LightsageClickTap any row to explore the full eval.
Get your API graded →
| Rank | Tool | Score | Grade | Cost | Calls | Errors | Time | Report |
|---|---|---|---|---|---|---|---|---|
🥇 | OpenAI Whisper | 95 | A | $0.0020 | 5 | 0 | 47s | |
🥈 | Prefect | 93 | A | $0.0043 | 8 | 0 | 54s | |
🥈 | fastCRW | 93 | A | $0.0033 | 10 | 0 | 34s | |
🥈 | Daytona | 93 | A | $0.0031 | 6 | 0 | 51s | |
5, Tied with 2 others | Descope | 90 | A | $0.0056 | 19 | 0 | 44s | |
5, Tied with 2 others | Dropbox | 90 | A | $0.0053 | 10 | 0 | 1m 9s | |
5, Tied with 2 others | E2B | 90 | A | $0.0032 | 8 | 0 | 1m 2s | |
8, Tied with 5 others | Agentmail | 89 | B | $0.0034 | 9 | 1 | 45s | |
8, Tied with 5 others | Stripe | 89 | B | $0.0044 | 10 | 1 | 52s | |
8, Tied with 5 others | Qdrant | 89 | B | $0.0044 | 8 | 1 | 55s | |
8, Tied with 5 others | Fireworks AI | 89 | B | $0.0043 | 10 | 2 | 46s | |
8, Tied with 5 others | AssemblyAI | 89 | B | $0.0024 | 6 | 1 | 50s | |
8, Tied with 5 others | OpenRouter | 89 | B | $0.0028 | 8 | 1 | 59s | |
14, Tied with 7 others | Mesh API | 88 | B | $0.0035 | 6 | 0 | 58s | |
14, Tied with 7 others | OpenCageData | 88 | B | $0.0029 | 7 | 0 | 38s | |
14, Tied with 7 others | CockroachDB | 88 | B | $0.0039 | 8 | 0 | 50s | |
14, Tied with 7 others | Reducto | 88 | B | $0.0044 | 11 | 0 | 1m 24s | |
14, Tied with 7 others | FalkorDB | 88 | B | $0.0038 | 7 | 0 | 57s | |
14, Tied with 7 others | Orkes | 88 | B | $0.0026 | 6 | 0 | 48s | |
14, Tied with 7 others | Vercel | 88 | B | $0.0062 | 6 | 0 | 1m 40s | |
14, Tied with 7 others | Neo4j | 88 | B | $0.0035 | 7 | 0 | 45s | |
22 | Clerk | 87 | B | $0.0058 | 14 | 1 | 51s | |
23, Tied with 6 others | Deepgram | 86 | B | $0.0060 | 10 | 1 | 1m 18s | |
23, Tied with 6 others | Weaviate | 86 | B | $0.0056 | 10 | 1 | 1m 10s | |
23, Tied with 6 others | You.com | 86 | B | $0.0136 | 18 | 0 | 1m 38s | |
23, Tied with 6 others | Tavily | 86 | B | $0.0150 | 17 | 0 | 1m 40s | |
23, Tied with 6 others | Exa | 86 | B | $0.0083 | 10 | 0 | 1m 9s | |
23, Tied with 6 others | Coinbase Payments | 86 | B | $0.0058 | 10 | 0 | 1m 7s | |
23, Tied with 6 others | Stability AI | 86 | B | $0.0047 | 10 | 1 | 1m 6s | |
30, Tied with 3 others | Bolna | 85 | B | $0.0043 | 7 | 1 | 57s | |
30, Tied with 3 others | Rime | 85 | B | $0.0021 | 7 | 1 | 43s | |
30, Tied with 3 others | Vonage | 85 | B | $0.0040 | 10 | 1 | 56s | |
30, Tied with 3 others | Browser Use | 85 | B | $0.0136 | 25 | 0 | 1m 34s | |
34, Tied with 7 others | VideoDB | 84 | B | $0.0082 | 16 | 0 | 4m 30s | |
34, Tied with 7 others | Shopware | 84 | B | $0.0353 | 33 | 0 | 2m 19s | |
34, Tied with 7 others | Tavus | 84 | B | $0.0078 | 18 | 0 | 3m 57s | |
34, Tied with 7 others | Meetstream | 84 | B | $0.0091 | 10 | 0 | 2m 24s | |
34, Tied with 7 others | Superserve | 84 | B | $0.0098 | 13 | 1 | 1m 11s | |
34, Tied with 7 others | Modal | 84 | B | $0.0046 | 10 | 0 | 50s | |
34, Tied with 7 others | ElevenLabs | 84 | B | $0.0037 | 11 | 1 | 1m 26s | |
34, Tied with 7 others | Steel | 84 | B | $0.0079 | 15 | 1 | 1m 25s | |
42, Tied with 10 others | Zernio | 83 | B | $0.0074 | 11 | 0 | 1m 8s | |
42, Tied with 10 others | Render | 83 | B | $0.0125 | 26 | 0 | 2m 33s | |
42, Tied with 10 others | Airwallex | 83 | B | $0.0073 | 14 | 0 | 1m 5s | |
42, Tied with 10 others | Apify | 83 | B | $0.0125 | 22 | 3 | 1m 29s | |
42, Tied with 10 others | Jina AI | 83 | B | $0.0355 | 32 | 0 | 4m 33s | |
42, Tied with 10 others | Nylas | 83 | B | $0.0123 | 18 | 0 | 1m 15s | |
42, Tied with 10 others | Box | 83 | B | $0.0115 | 18 | 3 | 1m 3s | |
42, Tied with 10 others | LlamaParse | 83 | B | $0.0093 | 16 | 1 | 1m 7s | |
42, Tied with 10 others | Speechify | 83 | B | $0.0059 | 15 | 3 | 52s | |
42, Tied with 10 others | Zoom AI Services | 83 | B | $0.0075 | 13 | 0 | 1m 3s | |
42, Tied with 10 others | Oodle | 83 | B | $0.0457 | 48 | 0 | 4m 14s | |
53, Tied with 2 others | WorkOS | 82 | B | $0.0067 | 20 | 1 | 1m 35s | |
53, Tied with 2 others | Resend | 82 | B | $0.0206 | 29 | 1 | 1m 29s | |
53, Tied with 2 others | Memgraph | 82 | B | $0.0039 | 9 | 1 | 1m 0s | |
56, Tied with 9 others | Pipecat | 81 | B | $0.0208 | 24 | 2 | 2m 25s | |
56, Tied with 9 others | Firecrawl | 81 | B | $0.0331 | 32 | 1 | 2m 26s | |
56, Tied with 9 others | Netflix Conductor | 81 | B | $0.0196 | 30 | 4 | 2m 10s | |
56, Tied with 9 others | Paddle | 81 | B | $0.0213 | 27 | 1 | 2m 26s | |
56, Tied with 9 others | Scalekit | 81 | B | $0.0346 | 40 | 1 | 2m 12s | |
56, Tied with 9 others | Pinecone | 81 | B | $0.0105 | 21 | 3 | 1m 30s | |
56, Tied with 9 others | Nimble | 81 | B | $0.0078 | 11 | 0 | 2m 34s | |
56, Tied with 9 others | Netlify | 81 | B | $0.0122 | 24 | 2 | 2m 8s | |
56, Tied with 9 others | Mux | 81 | B | $0.0187 | 28 | 2 | 1m 53s | |
56, Tied with 9 others | Merge Gateway | 81 | B | $0.0188 | 31 | 1 | 2m 11s | |
66, Tied with 8 others | Speechmatics | 80 | B | $0.0032 | 9 | 1 | 52s | |
66, Tied with 8 others | PayPal | 80 | B | $0.0265 | 28 | 0 | 2m 27s | |
66, Tied with 8 others | Railway | 80 | B | $0.0310 | 36 | 2 | 3m 26s | |
66, Tied with 8 others | Convex | 80 | B | $0.0168 | 35 | 0 | 1m 30s | |
66, Tied with 8 others | Restate | 80 | B | $0.0388 | 49 | 7 | 2m 33s | |
66, Tied with 8 others | Chroma | 80 | B | $0.0173 | 16 | 2 | 6m 49s | |
66, Tied with 8 others | Naive | 80 | B | $0.0074 | 14 | 1 | 1m 11s | |
66, Tied with 8 others | Zep | 80 | B | $0.0226 | 25 | 1 | 3m 39s | |
66, Tied with 8 others | AIsa | 80 | B | $0.0064 | 13 | 1 | 1m 17s | |
75, Tied with 7 others | Auth0 | 79 | B | $0.0072 | 20 | 3 | 1m 6s | |
75, Tied with 7 others | Twilio | 79 | B | $0.0335 | 35 | 6 | 2m 53s | |
75, Tied with 7 others | Blaxel | 79 | B | $0.0080 | 17 | 2 | 1m 8s | |
75, Tied with 7 others | Cognee | 79 | B | $0.0419 | 39 | 4 | 5m 4s | |
75, Tied with 7 others | Cloudflare | 79 | B | $0.0315 | 44 | 3 | 7m 51s | |
75, Tied with 7 others | Anchor Browser | 79 | B | $0.0073 | 15 | 1 | 1m 49s | |
75, Tied with 7 others | Camunda | 79 | B | $0.0444 | 52 | 6 | 6m 40s | |
75, Tied with 7 others | Google DeepMind | 79 | B | $0.0044 | 10 | 1 | 4m 6s | |
83, Tied with 7 others | Vectoree | 78 | B | $0.0081 | 11 | 0 | 56s | |
83, Tied with 7 others | Diagrid | 78 | B | $0.0137 | 29 | 3 | 1m 28s | |
83, Tied with 7 others | Zilliz Cloud | 78 | B | $0.0095 | 15 | 1 | 4m 37s | |
83, Tied with 7 others | Scrapfly | 78 | B | $0.0037 | 11 | 2 | 55s | |
83, Tied with 7 others | Diffbot | 78 | B | $0.0031 | 11 | 2 | 52s | |
83, Tied with 7 others | Sprites | 78 | B | $0.0120 | 27 | 2 | 1m 27s | |
83, Tied with 7 others | Extend.ai | 78 | B | $0.0085 | 18 | 1 | 1m 43s | |
83, Tied with 7 others | Tenki | 78 | B | $0.0053 | 13 | 1 | 48s | |
91, Tied with 2 others | LanceDB | 77 | B | $0.0052 | 13 | 0 | 3m 25s | |
91, Tied with 2 others | HitPay | 77 | B | $0.0159 | 17 | 0 | 1m 39s | |
91, Tied with 2 others | Twelve Labs | 77 | B | $0.0117 | 24 | 1 | 1m 44s | |
94, Tied with 9 others | Mollie | 76 | B | $0.0166 | 26 | 7 | 2m 6s | |
94, Tied with 9 others | Cartesia | 76 | B | $0.0139 | 23 | 1 | 2m 32s | |
94, Tied with 9 others | Nebius AI Cloud | 76 | B | $0.0132 | 26 | 1 | 1m 49s | |
94, Tied with 9 others | BlindPay | 76 | B | $0.0226 | 40 | 1 | 2m 11s | |
94, Tied with 9 others | Merge | 76 | B | $0.0603 | 55 | 2 | 4m 51s | |
94, Tied with 9 others | DFNS | 76 | B | $0.0078 | 22 | 0 | 52s | |
94, Tied with 9 others | Vapi | 76 | B | $0.0543 | 51 | 3 | 15m 15s | |
94, Tied with 9 others | Letta | 76 | B | $0.0296 | 21 | 1 | 2m 40s | |
94, Tied with 9 others | CopilotKit | 76 | B | $0.0147 | 26 | 2 | 1m 35s | |
94, Tied with 9 others | Hookdeck | 76 | B | $0.0493 | 59 | 1 | 2m 19s | |
104, Tied with 3 others | Prelude | 75 | B | $0.0031 | 8 | 0 | 42s | |
104, Tied with 3 others | TinyFish | 75 | B | $0.0601 | 37 | 0 | 6m 46s | |
104, Tied with 3 others | TABStack | 75 | B | $0.0262 | 29 | 1 | 3m 30s | |
104, Tied with 3 others | Recall.ai | 75 | B | $0.0039 | 10 | 0 | 45s | |
108, Tied with 1 other | LiveKit | 74 | C | $0.0479 | 44 | 1 | 8m 52s | |
108, Tied with 1 other | TigerGraph | 74 | C | $0.0395 | 45 | 1 | 7m 3s | |
110, Tied with 3 others | Rev AI | 73 | C | $0.0046 | 13 | 0 | 2m 26s | |
110, Tied with 3 others | Groq | 73 | C | $0.0084 | 8 | 2 | 1m 25s | |
110, Tied with 3 others | Square | 73 | C | $0.0189 | 30 | 5 | 1m 24s | |
110, Tied with 3 others | Surge | 73 | C | $0.0228 | 24 | 3 | 2m 14s | |
114, Tied with 3 others | BoldSign | 72 | C | $0.0111 | 26 | 1 | 1m 40s | |
114, Tied with 3 others | Agora | 72 | C | $0.0508 | 52 | 2 | 2m 48s | |
114, Tied with 3 others | Plivo | 72 | C | $0.0199 | 32 | 3 | 2m 8s | |
114, Tied with 3 others | Docusign | 72 | C | $0.0272 | 33 | 6 | 2m 23s | |
118, Tied with 2 others | Meeting BaaS | 71 | C | $0.0095 | 18 | 1 | 1m 3s | |
118, Tied with 2 others | Workato | 71 | C | $0.0808 | 59 | 0 | 6m 47s | |
118, Tied with 2 others | Stytch | 71 | C | $0.0196 | 33 | 0 | 1m 53s | |
121, Tied with 1 other | Freestyle | 70 | C | $0.0481 | 53 | 4 | 5m 46s | |
121, Tied with 1 other | BetterStack | 70 | C | $0.0442 | 58 | 0 | 3m 26s | |
123 | Cadence | 68 | C | $0.0287 | 41 | 2 | 2m 24s | |
124 | LemonSqueezy | 66 | C | $0.0112 | 15 | 0 | 1m 11s | |
125, Tied with 1 other | Postman | 62 | C | $0.0085 | 13 | 0 | 2m 1s | |
125, Tied with 1 other | Sensible | 62 | C | $0.0440 | 42 | 3 | 3m 19s | |
127 | Mem0 | 60 | C | $0.0084 | 12 | 1 | 1m 26s | |
128 | Apideck | 51 | D | $0.0375 | 40 | 5 | 4m 43s | |
129 | WSO2 Asgardeo | 43 | D | $0.0156 | 31 | 0 | 1m 35s | |
130 | Celigo | 40 | D | $0.1201 | 63 | 1 | 20m 19s | |
| — | Cloudflare Workers Task needs a paid plan (Cloudflare Sandbox SDK requires Workers Paid) | — | — | — | — | — | — | |
| — | Fireblocks Phone number needed to sign up (developer sandbox form) | — | — | — | — | — | — | |
| — | Telnyx Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Ash Hosted Ash Cloud gives no working runtime on a free account (0 credits; credits from $10) | — | — | — | — | — | — | |
| — | Sonilo API key requires a paid plan | — | — | — | — | — | — | |
| — | Unstructured Unable to obtain API key | — | — | — | — | — | — | |
| — | Datadog Unable to obtain API key | — | — | — | — | — | — | |
| — | Dust API key requires a paid plan | — | — | — | — | — | — | |
| — | ElevenLabs Music Music API requires a paid plan | — | — | — | — | — | — | |
| — | Kosuke Unable to obtain API key | — | — | — | — | — | — | |
| — | Azion CLI build and deploy outlast the harness's no-output timeout | — | — | — | — | — | — | |
| — | Nebius Token Factory Credit card needed to sign up | — | — | — | — | — | — | |
| — | Circle Faucet API requires a Mainnet account with billing details | — | — | — | — | — | — | |
| — | Temporal Credit card required to start the free trial | — | — | — | — | — | — | |
| — | Razorpay Phone number or PAN needed before *****-mode keys | — | — | — | — | — | — | |
| — | DigitalOcean Credit card needed to sign up | — | — | — | — | — | — | |
| — | TogetherAI API key requires a paid plan | — | — | — | — | — | — | |
| — | HeyGen API key requires a paid plan | — | — | — | — | — | — | |
| — | Strapi Hosted Strapi Cloud requires a paid plan (no free plan or trial) | — | — | — | — | — | — | |
| — | Upsun Access requires vendor approval | — | — | — | — | — | — | |
| — | Perplexity API key requires a paid plan | — | — | — | — | — | — | |
| — | Chargebee Credit card or phone needed to sign up (site API answers 403 api_disabled 'site is not enabled'; new account needs phone verification) | — | — | — | — | — | — | |
| — | CocoIndex Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | Gumloop Credit card needed to sign up | — | — | — | — | — | — | |
| — | Relevance AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Replicate API key requires a paid plan | — | — | — | — | — | — | |
| — | Bargo Access requires vendor approval | — | — | — | — | — | — | |
| — | Massive Proxy access requires KYC verification | — | — | — | — | — | — | |
| — | Bandwidth Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | DeepInfra API key requires a paid plan | — | — | — | — | — | — | |
| — | Daily Task forbids raw REST, and Daily's Python SDK has no room or token methods | — | — | — | — | — | — | |
| — | Sinch Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | SambaNova API key requires a paid plan | — | — | — | — | — | — | |
| — | Adyen Phone number needed to sign up for a ***** account | — | — | — | — | — | — | |
| — | PandaDoc API trial used up; new accounts are suspended at signup | — | — | — | — | — | — | |
| — | Resemble AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Kite No API key offered (passkey Passport and CLI only) | — | — | — | — | — | — | |
| — | Ravion Task needs a connected AWS account (card and phone required); only aws-main in AWAITING_CONNECTION, pipeline fails Ravion:AwsAccount:NOT_CONNECTED | — | — | — | — | — | — | |
| — | Mubert API key requires a paid plan | — | — | — | — | — | — | |
| — | Visa Payment-link API requires Visa approval | — | — | — | — | — | — | |
| — | Cerebras Credit card required to activate API access | — | — | — | — | — | — | |
| — |  AWS Step Functions Free sandbox account expired; a new one can be requested once a week | — | — | — | — | — | — | |
| — | Infobip Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Akka Task needs a Python SDK; Akka's SDK is Java only | — | — | — | — | — | — | |
| — | Macroscope Credit card needed to sign up | — | — | — | — | — | — | |
| — | Entrig Task needs a paid plan (Entrig sends SMS only on its $100/month plan) | — | — | — | — | — | — | |
| — | Allo Task needs a Python SDK and outbound calls, which Allo doesn't offer | — | — | — | — | — | — | |
| — | GovSpend Access requires vendor approval | — | — | — | — | — | — | |
| — | MeetGeek Task needs a paid plan (bot join requires a paid subscription) | — | — | — | — | — | — | |
| — | Brave Search Credit card needed to sign up | — | — | — | — | — | — | |
| — | 100ms Credit card required to access API keys | — | — | — | — | — | — | |
| — | Pioneer AI Credits required to create an API key (paid) | — | — | — | — | — | — | |
| — | Roster No API key configured for this company | — | — | — | — | — | — | |
| — | Pursuit API requires a paid subscription arranged through a sales meeting | — | — | — | — | — | — | |
| — | Google Lyria API key requires a paid plan | — | — | — | — | — | — | |
| — | Apollo Required API access is excluded from free and trial plans | — | — | — | — | — | — | |
| — | Loudly API key requires a paid plan | — | — | — | — | — | — | |
| — | NationGraph Unable to obtain API key | — | — | — | — | — | — | |
| — | Unclone Access requires vendor approval | — | — | — | — | — | — | |
| — | Triple-A Sandbox access requires a reviewed business inquiry | — | — | — | — | — | — | |
| — | LMNT No longer in service | — | — | — | — | — | — | |
| — |  CueMeet Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | RampRadar API keys are not self-serve (request through the Contact page) | — | — | — | — | — | — |
Scores calculated from evaluations by AI coding agents across multiple benchmarks.
Frequently asked questions
Join the mailing list
Get updates on leaderboard changes, new benchmark releases, and product announcements.