LightsageClickTap any row to explore the full eval.
Get your API graded →
| Rank | Tool | Score | Grade | Cost | Calls | Errors | Time | Report |
|---|---|---|---|---|---|---|---|---|
🥇 | Deepgram | 95 | A | $0.03 | 5 | 0 | 38s | |
🥇 | OpenRouter | 95 | A | $0.02 | 4 | 0 | 50s | |
🥉 | Fireworks AI | 93 | A | $0.04 | 6 | 0 | 59s | |
🥉 | Prefect | 93 | A | $0.04 | 7 | 0 | 54s | |
🥉 | ElevenLabs | 93 | A | $0.03 | 6 | 0 | 53s | |
6, Tied with 4 others | Mesh API | 91 | A | $0.02 | 5 | 0 | 44s | |
6, Tied with 4 others | Rime | 91 | A | $0.01 | 4 | 0 | 42s | |
6, Tied with 4 others | Vonage | 91 | A | $0.03 | 5 | 0 | 53s | |
6, Tied with 4 others | OpenAI Whisper | 91 | A | $0.03 | 4 | 0 | 55s | |
6, Tied with 4 others | Neo4j | 91 | A | $0.02 | 5 | 0 | 45s | |
11 | Reducto | 89 | B | $0.05 | 6 | 1 | 56s | |
12 | Jina AI | 88 | B | $0.03 | 4 | 0 | 1m 21s | |
13, Tied with 3 others | Postman | 87 | B | $0.09 | 10 | 0 | 1m 23s | |
13, Tied with 3 others | Clerk | 87 | B | $0.07 | 7 | 0 | 1m 9s | |
13, Tied with 3 others | Tavily | 87 | B | $0.06 | 8 | 0 | 1m 14s | |
13, Tied with 3 others | Daytona | 87 | B | $0.07 | 8 | 0 | 1m 1s | |
17, Tied with 2 others | OpenCageData | 86 | B | $0.09 | 7 | 0 | 57s | |
17, Tied with 2 others | Chroma | 86 | B | $0.04 | 6 | 1 | 1m 20s | |
17, Tied with 2 others | Merge Gateway | 86 | B | $0.04 | 10 | 1 | 1m 8s | |
20, Tied with 3 others | Speechmatics | 84 | B | $0.04 | 6 | 0 | 54s | |
20, Tied with 3 others | Firecrawl | 84 | B | $0.07 | 9 | 1 | 1m 18s | |
20, Tied with 3 others | AssemblyAI | 84 | B | $0.05 | 9 | 1 | 1m 20s | |
20, Tied with 3 others | Vercel | 84 | B | $0.08 | 10 | 0 | 4m 3s | |
24, Tied with 3 others | Meetstream | 83 | B | $0.07 | 9 | 0 | 1m 2s | |
24, Tied with 3 others | You.com | 83 | B | $0.09 | 14 | 0 | 1m 54s | |
24, Tied with 3 others | Netlify | 83 | B | $0.05 | 8 | 0 | 6m 11s | |
24, Tied with 3 others | Browserbase | 83 | B | $0.08 | 15 | 0 | 1m 38s | |
28, Tied with 1 other | Groq | 82 | B | $0.04 | 5 | 0 | 40s | |
28, Tied with 1 other | Stability AI | 82 | B | $0.08 | 15 | 1 | 1m 26s | |
30, Tied with 5 others | LiveKit | 81 | B | $0.08 | 11 | 0 | 1m 4s | |
30, Tied with 5 others | Scrapfly | 81 | B | $0.04 | 7 | 0 | 1m 3s | |
30, Tied with 5 others | Cartesia | 81 | B | $0.06 | 10 | 0 | 1m 54s | |
30, Tied with 5 others | Nimble | 81 | B | $0.07 | 10 | 0 | 2m 17s | |
30, Tied with 5 others | Speechify | 81 | B | $0.07 | 11 | 0 | 1m 27s | |
30, Tied with 5 others | Steel | 81 | B | $0.15 | 19 | 0 | 1m 49s | |
36, Tied with 2 others | Qdrant | 80 | B | $0.14 | 11 | 1 | 1m 29s | |
36, Tied with 2 others | Meeting BaaS | 80 | B | $0.05 | 10 | 0 | 54s | |
36, Tied with 2 others | fastCRW | 80 | B | $0.13 | 15 | 2 | 1m 18s | |
39, Tied with 1 other | TinyFish | 79 | B | $0.07 | 11 | 0 | 1m 43s | |
39, Tied with 1 other | Dropbox | 79 | B | $0.14 | 13 | 0 | 1m 25s | |
41, Tied with 4 others | Tavus | 78 | B | $0.09 | 12 | 1 | 5m 10s | |
41, Tied with 4 others | Airwallex | 78 | B | $0.14 | 15 | 0 | 2m 10s | |
41, Tied with 4 others | Exa | 78 | B | $0.12 | 9 | 1 | 1m 8s | |
41, Tied with 4 others | Modal | 78 | B | $0.05 | 7 | 2 | 1m 2s | |
41, Tied with 4 others | Naive | 78 | B | $0.11 | 14 | 0 | 2m 10s | |
46, Tied with 4 others | WorkOS | 77 | B | $0.16 | 16 | 6 | 2m 15s | |
46, Tied with 4 others | Merge | 77 | B | $0.33 | 36 | 0 | 3m 40s | |
46, Tied with 4 others | Bolna | 77 | B | $0.11 | 10 | 0 | 1m 17s | |
46, Tied with 4 others | E2B | 77 | B | $0.12 | 17 | 1 | 1m 42s | |
46, Tied with 4 others | LlamaParse | 77 | B | $0.16 | 20 | 3 | 1m 58s | |
51, Tied with 5 others | Agentmail | 76 | B | $0.18 | 23 | 6 | 3m 51s | |
51, Tied with 5 others | Descope | 76 | B | $0.25 | 21 | 1 | 1m 42s | |
51, Tied with 5 others | Twilio | 76 | B | $0.15 | 19 | 8 | 3m 23s | |
51, Tied with 5 others | TABStack | 76 | B | $0.16 | 13 | 0 | 4m 19s | |
51, Tied with 5 others | Box | 76 | B | $0.25 | 25 | 1 | 1m 54s | |
51, Tied with 5 others | AIsa | 76 | B | $0.09 | 14 | 2 | 1m 31s | |
57, Tied with 6 others | Rev AI | 75 | B | $0.04 | 6 | 0 | 2m 12s | |
57, Tied with 6 others | Stripe | 75 | B | $0.20 | 27 | 4 | 1m 46s | |
57, Tied with 6 others | Plivo | 75 | B | $0.08 | 12 | 0 | 1m 44s | |
57, Tied with 6 others | Railway | 75 | B | $0.22 | 25 | 2 | 2m 39s | |
57, Tied with 6 others | Cognee | 75 | B | $1.59 | 67 | 0 | 14m 8s | |
57, Tied with 6 others | Tenki | 75 | B | $0.06 | 7 | 1 | 1m 27s | |
57, Tied with 6 others | Anchor Browser | 75 | B | $0.08 | 15 | 2 | 3m 16s | |
64, Tied with 12 others | VideoDB | 74 | C | $0.15 | 27 | 5 | 13m 10s | |
64, Tied with 12 others | Shopware | 74 | C | $0.26 | 42 | 7 | 4m 39s | |
64, Tied with 12 others | Zilliz Cloud | 74 | C | $0.17 | 15 | 2 | 1m 54s | |
64, Tied with 12 others | Render | 74 | C | $0.25 | 23 | 2 | 23m 39s | |
64, Tied with 12 others | Resend | 74 | C | $0.33 | 40 | 6 | 2m 30s | |
64, Tied with 12 others | Pinecone | 74 | C | $0.24 | 29 | 7 | 3m 42s | |
64, Tied with 12 others | PayPal | 74 | C | $0.19 | 31 | 0 | 3m 26s | |
64, Tied with 12 others | Superserve | 74 | C | $0.20 | 24 | 2 | 3m 14s | |
64, Tied with 12 others | Extend.ai | 74 | C | $0.10 | 11 | 1 | 1m 45s | |
64, Tied with 12 others | Mux | 74 | C | $0.22 | 31 | 2 | 3m 27s | |
64, Tied with 12 others | Zep | 74 | C | $0.24 | 22 | 1 | 5m 16s | |
64, Tied with 12 others | Mem0 | 74 | C | $0.23 | 29 | 4 | 3m 48s | |
64, Tied with 12 others | Oodle | 74 | C | $0.20 | 31 | 2 | 4m 20s | |
77, Tied with 5 others | Zernio | 73 | C | $0.16 | 19 | 2 | 1m 52s | |
77, Tied with 5 others | Paddle | 73 | C | $0.47 | 49 | 7 | 3m 18s | |
77, Tied with 5 others | Diffbot | 73 | C | $0.08 | 11 | 2 | 1m 25s | |
77, Tied with 5 others | HitPay | 73 | C | $0.10 | 12 | 0 | 1m 46s | |
77, Tied with 5 others | Blaxel | 73 | C | $0.12 | 14 | 2 | 2m 51s | |
77, Tied with 5 others | Twelve Labs | 73 | C | $0.12 | 25 | 3 | 2m 22s | |
83, Tied with 9 others | Pipecat | 72 | C | $0.56 | 66 | 10 | 8m 6s | |
83, Tied with 9 others | Weaviate | 72 | C | $0.49 | 36 | 1 | 5m 32s | |
83, Tied with 9 others | Nebius AI Cloud | 72 | C | $0.68 | 36 | 0 | 3m 58s | |
83, Tied with 9 others | Scalekit | 72 | C | $1.18 | 64 | 10 | 6m 3s | |
83, Tied with 9 others | Restate | 72 | C | $0.42 | 29 | 1 | 5m 32s | |
83, Tied with 9 others | Vapi | 72 | C | $0.52 | 43 | 4 | 8m 26s | |
83, Tied with 9 others | Browser Use | 72 | C | $0.39 | 33 | 1 | 7m 0s | |
83, Tied with 9 others | Cloudflare | 72 | C | $1.07 | 81 | 4 | 7m 53s | |
83, Tied with 9 others | Camunda | 72 | C | $0.66 | 43 | 7 | 5m 53s | |
83, Tied with 9 others | Google DeepMind | 72 | C | $0.19 | 20 | 6 | 2m 46s | |
93, Tied with 4 others | LanceDB | 71 | C | $0.13 | 12 | 1 | 1m 21s | |
93, Tied with 4 others | Stytch | 71 | C | $0.08 | 15 | 0 | 1m 31s | |
93, Tied with 4 others | Nylas | 71 | C | $0.19 | 29 | 6 | 3m 8s | |
93, Tied with 4 others | Coinbase Payments | 71 | C | $0.21 | 19 | 2 | 3m 52s | |
93, Tied with 4 others | Memgraph | 71 | C | $0.11 | 14 | 3 | 1m 11s | |
98, Tied with 3 others | Mollie | 70 | C | $0.16 | 11 | 1 | 2m 26s | |
98, Tied with 3 others | Agora | 70 | C | $0.34 | 30 | 3 | 3m 0s | |
98, Tied with 3 others | Letta | 70 | C | $0.41 | 25 | 2 | 3m 4s | |
98, Tied with 3 others | Hookdeck | 70 | C | $0.25 | 28 | 1 | 2m 46s | |
102, Tied with 2 others | Auth0 | 69 | C | $0.38 | 38 | 5 | 4m 37s | |
102, Tied with 2 others | Convex | 69 | C | $0.40 | 30 | 2 | 3m 30s | |
102, Tied with 2 others | CopilotKit | 69 | C | $0.42 | 33 | 5 | 4m 21s | |
105, Tied with 2 others | Docusign | 68 | C | $0.15 | 23 | 3 | 1m 52s | |
105, Tied with 2 others | Recall.ai | 68 | C | $0.13 | 9 | 0 | 1m 26s | |
105, Tied with 2 others | Zoom AI Services | 68 | C | $0.63 | 36 | 2 | 4m 33s | |
108, Tied with 2 others | Sprites | 67 | C | $0.54 | 47 | 7 | 5m 28s | |
108, Tied with 2 others | FalkorDB | 67 | C | $0.93 | 78 | 10 | 8m 53s | |
108, Tied with 2 others | TigerGraph | 67 | C | $0.91 | 73 | 14 | 15m 56s | |
111, Tied with 1 other | Prelude | 66 | C | $0.08 | 10 | 1 | 1m 16s | |
111, Tied with 1 other | Surge | 66 | C | $0.19 | 30 | 8 | 4m 29s | |
113, Tied with 1 other | DFNS | 64 | C | $0.37 | 32 | 0 | 2m 45s | |
113, Tied with 1 other | BetterStack | 64 | C | $0.34 | 26 | 0 | 5m 13s | |
115, Tied with 1 other | BoldSign | 63 | C | $0.62 | 47 | 10 | 8m 34s | |
115, Tied with 1 other | Freestyle | 63 | C | $0.90 | 64 | 9 | 7m 49s | |
117, Tied with 1 other | LemonSqueezy | 60 | C | $0.15 | 17 | 0 | 2m 2s | |
117, Tied with 1 other | Cadence | 60 | C | $0.60 | 31 | 2 | 2m 56s | |
119 | Square | 59 | C | $1.19 | 77 | 9 | 5m 44s | |
120 | Sensible | 54 | D | $0.78 | 61 | 5 | 15m 21s | |
121 | Pioneer AI | 53 | D | $0.24 | 24 | 6 | 2m 41s | |
122 | Netflix Conductor | 47 | D | $0.60 | 63 | 10 | 7m 49s | |
123 | Apideck | 45 | D | $0.30 | 31 | 7 | 4m 13s | |
124 | BlindPay | 43 | D | $0.62 | 38 | 2 | 7m 40s | |
| — | Cloudflare Workers Task needs a paid plan (Cloudflare Sandbox SDK requires Workers Paid) | — | — | — | — | — | — | |
| — | Fireblocks Phone number needed to sign up (developer sandbox form) | — | — | — | — | — | — | |
| — | Telnyx Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Ash Hosted Ash Cloud gives no working runtime on a free account (0 credits; credits from $10) | — | — | — | — | — | — | |
| — | Sonilo API key requires a paid plan | — | — | — | — | — | — | |
| — | Unstructured Unable to obtain API key | — | — | — | — | — | — | |
| — | Datadog Unable to obtain API key | — | — | — | — | — | — | |
| — | Dust API key requires a paid plan | — | — | — | — | — | — | |
| — | ElevenLabs Music Music API requires a paid plan | — | — | — | — | — | — | |
| — | Kosuke Unable to obtain API key | — | — | — | — | — | — | |
| — | Azion CLI build and deploy outlast the harness's no-output timeout | — | — | — | — | — | — | |
| — | Nebius Token Factory Credit card needed to sign up | — | — | — | — | — | — | |
| — | Circle Faucet API requires a Mainnet account with billing details | — | — | — | — | — | — | |
| — | Temporal Credit card required to start the free trial | — | — | — | — | — | — | |
| — | Razorpay Phone number or PAN needed before *****-mode keys | — | — | — | — | — | — | |
| — | DigitalOcean Credit card needed to sign up | — | — | — | — | — | — | |
| — | TogetherAI API key requires a paid plan | — | — | — | — | — | — | |
| — | HeyGen API key requires a paid plan | — | — | — | — | — | — | |
| — | Strapi Hosted Strapi Cloud requires a paid plan (no free plan or trial) | — | — | — | — | — | — | |
| — | Upsun Access requires vendor approval | — | — | — | — | — | — | |
| — | Perplexity API key requires a paid plan | — | — | — | — | — | — | |
| — | Chargebee Credit card or phone needed to sign up (site API answers 403 api_disabled 'site is not enabled'; new account needs phone verification) | — | — | — | — | — | — | |
| — | CocoIndex Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | Gumloop Credit card needed to sign up | — | — | — | — | — | — | |
| — | Relevance AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Replicate API key requires a paid plan | — | — | — | — | — | — | |
| — | Bargo Access requires vendor approval | — | — | — | — | — | — | |
| — | Massive Proxy access requires KYC verification | — | — | — | — | — | — | |
| — | Bandwidth Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | DeepInfra API key requires a paid plan | — | — | — | — | — | — | |
| — | Daily Task forbids raw REST, and Daily's Python SDK has no room or token methods | — | — | — | — | — | — | |
| — | Sinch Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | SambaNova API key requires a paid plan | — | — | — | — | — | — | |
| — | Adyen Phone number needed to sign up for a ***** account | — | — | — | — | — | — | |
| — | PandaDoc API trial used up; new accounts are suspended at signup | — | — | — | — | — | — | |
| — | Resemble AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Kite No API key offered (passkey Passport and CLI only) | — | — | — | — | — | — | |
| — | Ravion Task needs a connected AWS account (card and phone required); only aws-main in AWAITING_CONNECTION, pipeline fails Ravion:AwsAccount:NOT_CONNECTED | — | — | — | — | — | — | |
| — | Mubert API key requires a paid plan | — | — | — | — | — | — | |
| — | Visa Payment-link API requires Visa approval | — | — | — | — | — | — | |
| — | Cerebras Credit card required to activate API access | — | — | — | — | — | — | |
| — |  AWS Step Functions Free sandbox account expired; a new one can be requested once a week | — | — | — | — | — | — | |
| — | Infobip Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Akka Task needs a Python SDK; Akka's SDK is Java only | — | — | — | — | — | — | |
| — | Macroscope Credit card needed to sign up | — | — | — | — | — | — | |
| — | Entrig Task needs a paid plan (Entrig sends SMS only on its $100/month plan) | — | — | — | — | — | — | |
| — | Allo Task needs a Python SDK and outbound calls, which Allo doesn't offer | — | — | — | — | — | — | |
| — | GovSpend Access requires vendor approval | — | — | — | — | — | — | |
| — | MeetGeek Task needs a paid plan (bot join requires a paid subscription) | — | — | — | — | — | — | |
| — | Brave Search Credit card needed to sign up | — | — | — | — | — | — | |
| — | 100ms Credit card required to access API keys | — | — | — | — | — | — | |
| — | Roster No API key configured for this company | — | — | — | — | — | — | |
| — | Pursuit API requires a paid subscription arranged through a sales meeting | — | — | — | — | — | — | |
| — | Google Lyria API key requires a paid plan | — | — | — | — | — | — | |
| — | Apollo Required API access is excluded from free and trial plans | — | — | — | — | — | — | |
| — | Loudly API key requires a paid plan | — | — | — | — | — | — | |
| — | NationGraph Unable to obtain API key | — | — | — | — | — | — | |
| — | Unclone Access requires vendor approval | — | — | — | — | — | — | |
| — | Triple-A Sandbox access requires a reviewed business inquiry | — | — | — | — | — | — | |
| — | LMNT No longer in service | — | — | — | — | — | — | |
| — |  CueMeet Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | RampRadar API keys are not self-serve (request through the Contact page) | — | — | — | — | — | — |
Scores calculated from evaluations by AI coding agents across multiple benchmarks.
Frequently asked questions
Join the mailing list
Get updates on leaderboard changes, new benchmark releases, and product announcements.