Queue

VC1 · Build a landing page from a prompt · Not testedVC2 · Build a full-stack app from a prompt · 1 scoredVC3 · Deploy and host a vibe-coded app · Not testedVC4 · Add auth and a database to an app · 1 scoredVC5 · Find security holes in a vibe-coded repo · 1 scoredVC6 · Turn a design or screenshot into UI code · 1 scoredCA1 · Inline autocomplete and chat in the IDE · 3 scoredCA2 · Multi-file refactor with an agent · 3 scoredCA3 · Turn a GitHub issue into a PR autonomously · 2 scoredCA4 · AI code review on pull requests · 1 scoredCA5 · Generate unit and end-to-end tests · 4 scoredCA6 · Run a coding model locally or cheaply · 5 scoredMCP1 · Connect an agent to GitHub · 1 scoredMCP2 · Let an agent control a browser · 4 scoredMCP3 · Let an agent query a database · 1 scoredMCP4 · Give an agent web search and live docs · 1 scoredMCP5 · Build or host your own MCP server · 4 scoredMCP6 · Find and install MCP servers · 2 scoredGD1 · Generate 2D sprites and pixel art · 1 scoredGD2 · Generate 3D models for a game · 1 scoredGD3 · Make sound effects and music · 1 scoredGD4 · Voice and dialogue for NPCs · Not testedGD5 · Prototype a playable game from a prompt · Not testedGD6 · Animate characters with AI (mocap/rigging) · Not testedVC1 · Build a landing page from a prompt · Not testedVC2 · Build a full-stack app from a prompt · 1 scoredVC3 · Deploy and host a vibe-coded app · Not testedVC4 · Add auth and a database to an app · 1 scoredVC5 · Find security holes in a vibe-coded repo · 1 scoredVC6 · Turn a design or screenshot into UI code · 1 scoredCA1 · Inline autocomplete and chat in the IDE · 3 scoredCA2 · Multi-file refactor with an agent · 3 scoredCA3 · Turn a GitHub issue into a PR autonomously · 2 scoredCA4 · AI code review on pull requests · 1 scoredCA5 · Generate unit and end-to-end tests · 4 scoredCA6 · Run a coding model locally or cheaply · 5 scoredMCP1 · Connect an agent to GitHub · 1 scoredMCP2 · Let an agent control a browser · 4 scoredMCP3 · Let an agent query a database · 1 scoredMCP4 · Give an agent web search and live docs · 1 scoredMCP5 · Build or host your own MCP server · 4 scoredMCP6 · Find and install MCP servers · 2 scoredGD1 · Generate 2D sprites and pixel art · 1 scoredGD2 · Generate 3D models for a game · 1 scoredGD3 · Make sound effects and music · 1 scoredGD4 · Voice and dialogue for NPCs · Not testedGD5 · Prototype a playable game from a prompt · Not testedGD6 · Animate characters with AI (mocap/rigging) · Not tested

CA6 · measured scores only

Aider vs Qwen Code

For Run a coding model locally or cheaply.

Aider scored 78/100 and Qwen Code scored 40/100. The gap is 38 points on the rubric. Nothing else is inferred.

Last tested: 16 Oct 2026

Aider

78 / 100

Rank 2 on this job · tested 15 Oct 2026

Qwen Code

40 / 100

Rank 4 on this job · tested 16 Oct 2026

Rubric breakdown

Job result (40)32 Aider4 Qwen Code
Speed and effort (15)12 Aider3 Qwen Code
Price and free tier (15)15 Aider15 Qwen Code
Ease of start (10)5 Aider6 Qwen Code
Reliability and trust (10)8 Aider5 Qwen Code
Community signal (10)6 Aider7 Qwen Code

Evidence · Aider

Aider 0.86.2 with local ollama/qwen2.5-coder:3b. Two prompts, 49s then 82s. Final files passed tsc and the same checks. Evidence: evidence/2026-10-16/ca6/aider/.

Evidence · Qwen Code

Qwen Code 0.25.2 with --auth-type openai and Ollama's /v1 endpoint. A 60s ping returned tool-call text. A 20s task returned grep_search JSON and did not write the files. A 120s run with --max-tool-calls 0 was cancelled with an empty stdout. Evidence: evidence/2026-10-16/batch3/qwen-code/.

Back to Run a coding model locally or cheaply · How we rank