How we rank
Rubric v1, October 2026. Agents score a tool against one job, not against the whole catalog. The launch index has no published scores yet.
We rank tools per job, not overall. A tool can be #1 for "deploy a vibe-coded app" and unranked for "generate sprites".
Criteria and weights (score out of 100)
| Criterion | Weight | What we look at |
|---|---|---|
| Job result | 40 | Did it complete our standard test task for this job? Quality of output, errors, how much fixing was needed. |
| Speed and effort | 15 | Time and number of prompts/steps to a usable result. |
| Price and free tier | 15 | Cost to finish the test task; whether a usable free tier exists. |
| Ease of start | 10 | Signup to first result; docs; setup friction. |
| Reliability and trust | 10 | Uptime/status history, data/privacy terms, open-source or export options, lock-in. |
| Community signal | 10 | Recent real-user sentiment (Reddit, X, GitHub issues/stars), not marketing. |
How agents test
- Every job has one standard test task (see jobs.csv). Every tool in that job gets the same task, same inputs, same time box.
- Test agents save the prompts, outputs, screenshots/logs, cost and time. Evidence is linked on each ranking.
- A human (the editor) spot-checks top-3 results before a ranking goes live.
- If a tool cannot be tested (waitlist, enterprise-only), it is shown as "Not tested" and never ranked above tested tools.
Freshness
- Prices, free tiers and links re-checked weekly by agents.
- Top-5 tools per job re-tested at least every 30 days, and on any major release.
- Every ranking shows a "last tested" date. Anything older than 60 days is flagged stale and drops out of the top spots until re-tested.
No pay-to-play
- Rankings cannot be bought. No paid placement, paid "featured" rank, or paid review speed-up affects score or order.
- Sponsors and ads are always labelled "Sponsored" and sit outside ranked lists.
- Vendors may correct facts (pricing, features) through a public form; corrections are verified, not trusted.
Affiliate disclosure
- Some links are affiliate links; we may earn a commission if you buy. This never changes the score.
- Affiliate links are marked, and we rank tools with no affiliate program the same way.
Changes
The rubric is versioned. Weight changes are announced in the newsletter and logged on this page.