What is Devtool Arena, and why we built it
Devtool Arena is a public leaderboard, run by Lightsage, that scores how well coding agents use 150+ real developer tools. Twelve coding agents, including Claude Code, Codex and Cursor, get the same task for every tool in a category and a live API key. We score whether the integration works end to end, and what it cost.
For how a run works and how it's scored, see the methodology.
Why it matters which agent touches your API
A developer who asks Claude Code to add search to an app never reads your quickstart. The agent reads the docs, picks the SDK, gets a key and writes the code. If one of those steps breaks, the developer sees a failed task and rarely finds out why.
How Devtool Arena differs from model leaderboards and coding benchmarks
Devtool Arena holds the agent fixed and varies the tool. Most public leaderboards do the opposite and compare models.
| Term | What it measures | Where it happens | Who it's for |
|---|---|---|---|
| Devtool Arena | Whether a coding agent completes a real task with a specific developer tool | Sandboxed runs with live API keys, one board per agent | Devtool companies, and developers choosing tools |
| LMArena | Which model's answers people prefer | Blind head-to-head votes | Model builders and model buyers |
| SWE-bench | Whether a model can resolve real GitHub issues | Python repositories with test suites | Model and agent builders |
| Agent Landscape | How often coding agents recommend a tool | Questions developers ask agents | Devtool marketers |
How we keep the scores fair
A score is only useful if every tool in a category faced the same test. These are the rules every batch follows.
- Same test. Every tool in a category gets the same task, the same agent and model, and the same grader.
- No paying, no early look. No company can pay for a placement or see its results before they're published.
Why we built Devtool Arena
Before Devtool Arena, our team spent two years on one question: how fast someone new to an API can get a working integration. That meant docs tooling first, then working sample apps, then an MCP server for searching docs. By 2026 the newcomer was often a coding agent, and nobody published how those agents did with real products. Vendors guessed, and developers found out the hard way.
We made the results public and free because a benchmark only one company can see doesn't change the docs, SDKs and signup flows that need fixing. A public board gives every vendor the same evidence: the trace, the failed call, the missing file. They can fix something and check the next batch.
FAQ
Is Devtool Arena free? Yes. The boards and run reports are public and free to read, and there's no fee to be listed.
How often is Devtool Arena updated? Devtool Arena runs in batches. The first full batch ran in May 2026 and the second in October 2026, and new agents and tools are added between batches.
Can a company pay to rank higher on Devtool Arena? No. Every tool in a category gets the same task, agent and grader, and no company can pay for a placement.