LightsageClickTap any row to explore the full eval.
Get your API graded →
| Rank | Tool | Score | Grade | Cost | Calls | Errors | Time | Report |
|---|---|---|---|---|---|---|---|---|
🥇 | Jina AI | 95 | A | $0.01 | 5 | 0 | 45s | |
🥈 | Deepgram | 93 | A | $0.00 | 7 | 0 | 39s | |
🥈 | WorkOS | 93 | A | $0.00 | 8 | 0 | 41s | |
🥈 | Stripe | 93 | A | $0.00 | 6 | 1 | 29s | |
🥈 | Weaviate | 93 | A | $0.01 | 8 | 0 | 43s | |
🥈 | Netflix Conductor | 93 | A | $0.01 | 9 | 0 | 42s | |
🥈 | AssemblyAI | 93 | A | $0.00 | 7 | 0 | 34s | |
🥈 | Prefect | 93 | A | $0.00 | 6 | 0 | 38s | |
🥈 | OpenRouter | 93 | A | $0.00 | 7 | 1 | 27s | |
🥈 | OpenAI Whisper | 93 | A | $0.00 | 6 | 0 | 46s | |
🥈 | Reducto | 93 | A | $0.01 | 7 | 0 | 42s | |
🥈 | ElevenLabs | 93 | A | $0.00 | 6 | 0 | 39s | |
13, Tied with 5 others | Postman | 90 | A | $0.01 | 13 | 0 | 56s | |
13, Tied with 5 others | Chroma | 90 | A | $0.00 | 11 | 0 | 58s | |
13, Tied with 5 others | E2B | 90 | A | $0.01 | 11 | 0 | 47s | |
13, Tied with 5 others | Box | 90 | A | $0.01 | 13 | 0 | 39s | |
13, Tied with 5 others | Stability AI | 90 | A | $0.01 | 13 | 0 | 45s | |
13, Tied with 5 others | Steel | 90 | A | $0.01 | 13 | 0 | 57s | |
19, Tied with 5 others | Qdrant | 89 | B | $0.01 | 8 | 2 | 45s | |
19, Tied with 5 others | Fireworks AI | 89 | B | $0.01 | 7 | 1 | 30s | |
19, Tied with 5 others | Tavily | 89 | B | $0.01 | 10 | 1 | 44s | |
19, Tied with 5 others | Superserve | 89 | B | $0.00 | 10 | 1 | 38s | |
19, Tied with 5 others | Vercel | 89 | B | $0.01 | 9 | 1 | 45s | |
19, Tied with 5 others | Daytona | 89 | B | $0.01 | 10 | 1 | 59s | |
25, Tied with 8 others | Mollie | 88 | B | $0.00 | 9 | 0 | 47s | |
25, Tied with 8 others | Airwallex | 88 | B | $0.01 | 7 | 0 | 31s | |
25, Tied with 8 others | Cartesia | 88 | B | $0.00 | 8 | 0 | 38s | |
25, Tied with 8 others | Bolna | 88 | B | $0.01 | 9 | 0 | 43s | |
25, Tied with 8 others | Rime | 88 | B | $0.00 | 7 | 0 | 30s | |
25, Tied with 8 others | OpenCageData | 88 | B | $0.01 | 8 | 0 | 34s | |
25, Tied with 8 others | CockroachDB | 88 | B | $0.01 | 8 | 0 | 47s | |
25, Tied with 8 others | Mux | 88 | B | $0.01 | 14 | 0 | 1m 2s | |
25, Tied with 8 others | Zoom AI Services | 88 | B | $0.01 | 7 | 0 | 45s | |
34, Tied with 3 others | Pipecat | 87 | B | $0.02 | 16 | 0 | 1m 5s | |
34, Tied with 3 others | Descope | 87 | B | $0.01 | 12 | 1 | 41s | |
34, Tied with 3 others | Dropbox | 87 | B | $0.02 | 13 | 1 | 41s | |
34, Tied with 3 others | fastCRW | 87 | B | $0.01 | 15 | 1 | 48s | |
38, Tied with 6 others | Render | 86 | B | $0.01 | 11 | 0 | 2m 22s | |
38, Tied with 6 others | Tavus | 86 | B | $0.01 | 14 | 0 | 3m 22s | |
38, Tied with 6 others | Paddle | 86 | B | $0.04 | 30 | 0 | 1m 19s | |
38, Tied with 6 others | Resend | 86 | B | $0.01 | 24 | 0 | 1m 20s | |
38, Tied with 6 others | Merge | 86 | B | $0.01 | 26 | 0 | 1m 19s | |
38, Tied with 6 others | You.com | 86 | B | $0.01 | 10 | 1 | 1m 0s | |
38, Tied with 6 others | Speechify | 86 | B | $0.01 | 12 | 0 | 40s | |
45, Tied with 5 others | Mesh API | 85 | B | $0.00 | 10 | 1 | 49s | |
45, Tied with 5 others | Exa | 85 | B | $0.00 | 8 | 1 | 44s | |
45, Tied with 5 others | Nylas | 85 | B | $0.02 | 21 | 0 | 52s | |
45, Tied with 5 others | Blaxel | 85 | B | $0.01 | 10 | 1 | 49s | |
45, Tied with 5 others | LlamaParse | 85 | B | $0.01 | 13 | 2 | 1m 13s | |
45, Tied with 5 others | Orkes | 85 | B | $0.01 | 10 | 1 | 37s | |
51, Tied with 4 others | LanceDB | 84 | B | $0.01 | 8 | 0 | 54s | |
51, Tied with 4 others | Speechmatics | 84 | B | $0.00 | 7 | 0 | 46s | |
51, Tied with 4 others | Scrapfly | 84 | B | $0.00 | 6 | 0 | 35s | |
51, Tied with 4 others | Groq | 84 | B | $0.00 | 6 | 0 | 23s | |
51, Tied with 4 others | Mem0 | 84 | B | $0.01 | 14 | 1 | 1m 28s | |
56, Tied with 12 others | Agentmail | 83 | B | $0.01 | 12 | 1 | 1m 46s | |
56, Tied with 12 others | TinyFish | 83 | B | $0.01 | 12 | 0 | 1m 9s | |
56, Tied with 12 others | Meetstream | 83 | B | $0.02 | 12 | 0 | 1m 14s | |
56, Tied with 12 others | Pinecone | 83 | B | $0.00 | 15 | 1 | 1m 31s | |
56, Tied with 12 others | Twilio | 83 | B | $0.01 | 16 | 6 | 1m 14s | |
56, Tied with 12 others | Convex | 83 | B | $0.01 | 17 | 0 | 1m 19s | |
56, Tied with 12 others | Nimble | 83 | B | $0.01 | 10 | 0 | 2m 36s | |
56, Tied with 12 others | Coinbase Payments | 83 | B | $0.01 | 15 | 0 | 1m 6s | |
56, Tied with 12 others | Naive | 83 | B | $0.01 | 13 | 3 | 52s | |
56, Tied with 12 others | Memgraph | 83 | B | $0.01 | 11 | 1 | 53s | |
56, Tied with 12 others | Merge Gateway | 83 | B | $0.02 | 17 | 1 | 1m 18s | |
56, Tied with 12 others | Zep | 83 | B | $0.04 | 26 | 0 | 4m 10s | |
56, Tied with 12 others | AIsa | 83 | B | $0.01 | 12 | 1 | 47s | |
69, Tied with 6 others | Zilliz Cloud | 82 | B | $0.01 | 9 | 2 | 1m 22s | |
69, Tied with 6 others | Clerk | 82 | B | $0.01 | 8 | 1 | 39s | |
69, Tied with 6 others | Apify | 82 | B | $0.02 | 16 | 2 | 1m 45s | |
69, Tied with 6 others | Scalekit | 82 | B | $0.03 | 34 | 1 | 1m 26s | |
69, Tied with 6 others | Docusign | 82 | B | $0.01 | 12 | 0 | 52s | |
69, Tied with 6 others | Vonage | 82 | B | $0.01 | 9 | 1 | 1m 29s | |
69, Tied with 6 others | Neo4j | 82 | B | $0.00 | 10 | 1 | 1m 1s | |
76, Tied with 2 others | Shopware | 81 | B | $0.02 | 28 | 1 | 2m 1s | |
76, Tied with 2 others | Browser Use | 81 | B | $0.02 | 36 | 4 | 2m 6s | |
76, Tied with 2 others | Camunda | 81 | B | $0.04 | 32 | 4 | 2m 6s | |
79, Tied with 7 others | Vectoree | 80 | B | $0.00 | 10 | 0 | 42s | |
79, Tied with 7 others | LiveKit | 80 | B | $0.01 | 14 | 1 | 1m 5s | |
79, Tied with 7 others | Auth0 | 80 | B | $0.01 | 14 | 1 | 1m 1s | |
79, Tied with 7 others | Meeting BaaS | 80 | B | $0.01 | 8 | 0 | 32s | |
79, Tied with 7 others | Diffbot | 80 | B | $0.00 | 9 | 1 | 38s | |
79, Tied with 7 others | Modal | 80 | B | $0.01 | 6 | 1 | 41s | |
79, Tied with 7 others | Tenki | 80 | B | $0.01 | 8 | 1 | 47s | |
79, Tied with 7 others | Cloudflare | 80 | B | $0.06 | 47 | 0 | 3m 7s | |
87, Tied with 8 others | VideoDB | 79 | B | $0.03 | 48 | 2 | 7m 27s | |
87, Tied with 8 others | Nebius AI Cloud | 79 | B | $0.03 | 24 | 1 | 1m 6s | |
87, Tied with 8 others | BlindPay | 79 | B | $0.02 | 23 | 1 | 1m 19s | |
87, Tied with 8 others | PayPal | 79 | B | $0.01 | 16 | 1 | 1m 7s | |
87, Tied with 8 others | Restate | 79 | B | $0.03 | 44 | 5 | 7m 31s | |
87, Tied with 8 others | Vapi | 79 | B | $0.05 | 54 | 1 | 6m 46s | |
87, Tied with 8 others | Netlify | 79 | B | $0.03 | 38 | 1 | 3m 11s | |
87, Tied with 8 others | FalkorDB | 79 | B | $0.02 | 17 | 3 | 1m 29s | |
87, Tied with 8 others | CopilotKit | 79 | B | $0.01 | 19 | 2 | 1m 8s | |
96, Tied with 5 others | Zernio | 78 | B | $0.01 | 26 | 1 | 1m 15s | |
96, Tied with 5 others | TABStack | 78 | B | $0.03 | 19 | 1 | 2m 19s | |
96, Tied with 5 others | Apideck | 78 | B | $0.01 | 19 | 2 | 2m 27s | |
96, Tied with 5 others | Sprites | 78 | B | $0.02 | 29 | 4 | 1m 14s | |
96, Tied with 5 others | Extend.ai | 78 | B | $0.01 | 14 | 1 | 3m 30s | |
96, Tied with 5 others | Twelve Labs | 78 | B | $0.01 | 16 | 1 | 1m 30s | |
102, Tied with 1 other | Rev AI | 77 | B | $0.00 | 5 | 0 | 3m 53s | |
102, Tied with 1 other | Anchor Browser | 77 | B | $0.01 | 21 | 2 | 2m 28s | |
104, Tied with 5 others | Diagrid | 76 | B | $0.01 | 27 | 2 | 1m 41s | |
104, Tied with 5 others | Firecrawl | 76 | B | $0.06 | 55 | 5 | 4m 8s | |
104, Tied with 5 others | Square | 76 | B | $0.02 | 23 | 0 | 54s | |
104, Tied with 5 others | Cognee | 76 | B | $0.05 | 55 | 2 | 10m 30s | |
104, Tied with 5 others | Oodle | 76 | B | $0.06 | 48 | 3 | 17m 8s | |
104, Tied with 5 others | Google DeepMind | 76 | B | $0.01 | 14 | 1 | 5m 6s | |
110, Tied with 4 others | BoldSign | 75 | B | $0.01 | 18 | 2 | 1m 9s | |
110, Tied with 4 others | Prelude | 75 | B | $0.01 | 9 | 0 | 33s | |
110, Tied with 4 others | Agora | 75 | B | $0.02 | 36 | 1 | 3m 19s | |
110, Tied with 4 others | BetterStack | 75 | B | $0.01 | 13 | 0 | 1m 0s | |
110, Tied with 4 others | Letta | 75 | B | $0.04 | 34 | 2 | 3m 52s | |
115, Tied with 1 other | Stytch | 74 | C | $0.01 | 17 | 0 | 1m 15s | |
115, Tied with 1 other | TigerGraph | 74 | C | $0.05 | 57 | 2 | 8m 0s | |
117, Tied with 1 other | Recall.ai | 73 | C | $0.01 | 13 | 0 | 50s | |
117, Tied with 1 other | HitPay | 73 | C | $0.01 | 15 | 1 | 3m 55s | |
119, Tied with 2 others | Plivo | 72 | C | $0.02 | 28 | 1 | 2m 8s | |
119, Tied with 2 others | Surge | 72 | C | $0.03 | 28 | 1 | 2m 0s | |
119, Tied with 2 others | Hookdeck | 72 | C | $0.05 | 37 | 5 | 2m 51s | |
122 | Cadence | 71 | C | $0.02 | 31 | 0 | 1m 33s | |
123 | Celigo | 70 | C | $0.04 | 44 | 1 | 5m 13s | |
124, Tied with 1 other | LemonSqueezy | 69 | C | $0.01 | 13 | 0 | 59s | |
124, Tied with 1 other | DFNS | 69 | C | $0.02 | 27 | 2 | 1m 4s | |
126 | Workato | 60 | C | $0.22 | 98 | 1 | 12m 42s | |
127 | Sensible | 56 | C | $0.18 | 73 | 1 | 20m 6s | |
128 | WSO2 Asgardeo | 36 | D | $0.03 | 27 | 1 | 1m 49s | |
| — | Cloudflare Workers Task needs a paid plan (Cloudflare Sandbox SDK requires Workers Paid) | — | — | — | — | — | — | |
| — | Fireblocks Phone number needed to sign up (developer sandbox form) | — | — | — | — | — | — | |
| — | Telnyx Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Ash Hosted Ash Cloud gives no working runtime on a free account (0 credits; credits from $10) | — | — | — | — | — | — | |
| — | Sonilo API key requires a paid plan | — | — | — | — | — | — | |
| — | Unstructured Unable to obtain API key | — | — | — | — | — | — | |
| — | Datadog Unable to obtain API key | — | — | — | — | — | — | |
| — | Dust API key requires a paid plan | — | — | — | — | — | — | |
| — | ElevenLabs Music Music API requires a paid plan | — | — | — | — | — | — | |
| — | Kosuke Unable to obtain API key | — | — | — | — | — | — | |
| — | Azion CLI build and deploy outlast the harness's no-output timeout | — | — | — | — | — | — | |
| — | Nebius Token Factory Credit card needed to sign up | — | — | — | — | — | — | |
| — | Circle Faucet API requires a Mainnet account with billing details | — | — | — | — | — | — | |
| — | Temporal Credit card required to start the free trial | — | — | — | — | — | — | |
| — | Razorpay Phone number or PAN needed before *****-mode keys | — | — | — | — | — | — | |
| — | DigitalOcean Credit card needed to sign up | — | — | — | — | — | — | |
| — | TogetherAI API key requires a paid plan | — | — | — | — | — | — | |
| — | HeyGen API key requires a paid plan | — | — | — | — | — | — | |
| — | Strapi Hosted Strapi Cloud requires a paid plan (no free plan or trial) | — | — | — | — | — | — | |
| — | Upsun Access requires vendor approval | — | — | — | — | — | — | |
| — | Perplexity API key requires a paid plan | — | — | — | — | — | — | |
| — | Chargebee Credit card or phone needed to sign up (site API answers 403 api_disabled 'site is not enabled'; new account needs phone verification) | — | — | — | — | — | — | |
| — | CocoIndex Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | Gumloop Credit card needed to sign up | — | — | — | — | — | — | |
| — | Relevance AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Replicate API key requires a paid plan | — | — | — | — | — | — | |
| — | Bargo Access requires vendor approval | — | — | — | — | — | — | |
| — | Massive Proxy access requires KYC verification | — | — | — | — | — | — | |
| — | Bandwidth Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Freestyle Credit card needed to sign up | — | — | — | — | — | — | |
| — | DeepInfra API key requires a paid plan | — | — | — | — | — | — | |
| — | Daily Task forbids raw REST, and Daily's Python SDK has no room or token methods | — | — | — | — | — | — | |
| — | Sinch Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | SambaNova API key requires a paid plan | — | — | — | — | — | — | |
| — | Adyen Phone number needed to sign up for a ***** account | — | — | — | — | — | — | |
| — | PandaDoc API trial used up; new accounts are suspended at signup | — | — | — | — | — | — | |
| — | Resemble AI API key requires a paid plan | — | — | — | — | — | — | |
| — | Kite No API key offered (passkey Passport and CLI only) | — | — | — | — | — | — | |
| — | Ravion Task needs a connected AWS account (card and phone required); only aws-main in AWAITING_CONNECTION, pipeline fails Ravion:AwsAccount:NOT_CONNECTED | — | — | — | — | — | — | |
| — | Mubert API key requires a paid plan | — | — | — | — | — | — | |
| — | Visa Payment-link API requires Visa approval | — | — | — | — | — | — | |
| — | Cerebras Credit card required to activate API access | — | — | — | — | — | — | |
| — |  AWS Step Functions Free sandbox account expired; a new one can be requested once a week | — | — | — | — | — | — | |
| — | Infobip Phone number needed (SMS verification) | — | — | — | — | — | — | |
| — | Akka Task needs a Python SDK; Akka's SDK is Java only | — | — | — | — | — | — | |
| — | Macroscope Credit card needed to sign up | — | — | — | — | — | — | |
| — | Entrig Task needs a paid plan (Entrig sends SMS only on its $100/month plan) | — | — | — | — | — | — | |
| — | Allo Task needs a Python SDK and outbound calls, which Allo doesn't offer | — | — | — | — | — | — | |
| — | GovSpend Access requires vendor approval | — | — | — | — | — | — | |
| — | MeetGeek Task needs a paid plan (bot join requires a paid subscription) | — | — | — | — | — | — | |
| — | Brave Search Credit card needed to sign up | — | — | — | — | — | — | |
| — | 100ms Credit card required to access API keys | — | — | — | — | — | — | |
| — | Pioneer AI Credits required to create an API key (paid) | — | — | — | — | — | — | |
| — | Roster No API key configured for this company | — | — | — | — | — | — | |
| — | Pursuit API requires a paid subscription arranged through a sales meeting | — | — | — | — | — | — | |
| — | Google Lyria API key requires a paid plan | — | — | — | — | — | — | |
| — | Apollo Required API access is excluded from free and trial plans | — | — | — | — | — | — | |
| — | Loudly API key requires a paid plan | — | — | — | — | — | — | |
| — | NationGraph Unable to obtain API key | — | — | — | — | — | — | |
| — | Unclone Access requires vendor approval | — | — | — | — | — | — | |
| — | Triple-A Sandbox access requires a reviewed business inquiry | — | — | — | — | — | — | |
| — | LMNT No longer in service | — | — | — | — | — | — | |
| — |  CueMeet Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | |
| — | RampRadar API keys are not self-serve (request through the Contact page) | — | — | — | — | — | — |
Scores calculated from evaluations by AI coding agents across multiple benchmarks.
Frequently asked questions
Join the mailing list
Get updates on leaderboard changes, new benchmark releases, and product announcements.