Back to leaderboard
Compare companies
Side-by-side MCP results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.
Model
Claude Code
CodexChoose 2-4 companies
Selected
Showing the first two companies on mobile. Use a wider screen to compare 3 companies.
Selected
Mobile compare is limited to two companies. Use a wider screen to compare all 3.
fastCRW
Webscraping
Qdrant
Vector Databases
Summary
Compare status
Comparable
Comparable
Overall score
79
93
BestGrade
B
A
Category
Webscraping
Vector Databases
Overall rank
#15
#1
BestCategory rank
#1
Best#1
BestFull eval
MCP Task Scoring
Task score
89%
96%
BestTask
Passed+30The agent successfully completed all required tasks: (1) discovered the available MCP tool (crw_scrape), (2) scraped htt...
BestPassed+30The agent successfully completed all required tasks using only MCP tools. It stored 3 FAQ-style entries about [REDACTED:...
BestUsed MCP Correctly
Passed+20Agent made 1 MCP tool calls
BestPassed+20Agent made 4 MCP tool calls
BestPartial Credit
Passed+10Partial credit: 100%
BestPassed+10Partial credit: 100%
BestZero Errors
Passed+10No errors
BestPassed+10No errors
BestEfficiency
Passed+102 tool calls (<=5, excellent)
BestPartial+66 tool calls (<=10, great)
Time
Partial+3.5134s (2-3min, good)
Passed+1054s (<60s, excellent)
BestCost
Partial+6$0.11 ($0.10-0.20, great)
Passed+10$0.09 (<$0.10, excellent)
BestRuntime (actual)
2m 13s
54s
BestCost (actual)
$0.1112
$0.0890
BestTool calls
2
Best6
Errors
0
Best0
BestTokens used
256,873
171,578
BestMCP Discovery
Discovery score
55%
89%
BestOfficial MCP
Passed5/5
BestPassed5/5
BestAuth Method
Passed15/15env var
BestPassed15/15env var
BestTool Count
Passed10/101 tools, lean
BestPassed10/102 tools, lean
BestTool Description Length
Passed10/106 tokens, lean
BestPassed10/10226 tokens, lean
Bestllms.txt
Passed10/10
BestPassed10/10
BestOutput Schema
Failed0/5
BestFailed0/5
BestTool Stability
Passed5/5
BestPassed5/5
BestSecurity Posture
—
Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are clearly typed, though the permiss...
Tool API Mapping
—
Partial14/15Tools map cleanly 1:1 to individual Qdrant operations (search and insert); each tool does one thing and an agent can com...
Description Clarity
—
Partial11/15Descriptions clearly explain the purpose of each tool (search vs. store memories) and when to use them, though they lack...
Eval Details
Integration task
# Task
Using the fastCRW MCP, scrape web content:
1. Discover available MCP tools and their capabilities
2. Scrape the page at "https://news.ycombinator.com"
3. Extract the top 5...
# Task
Using the Qdrant MCP, perform semantic memory operations:
1. Use the MCP tools relevant to storing and finding memories. You do not need to enumerate the full tool catalog...
Generated files
1
1
Trace tool calls
2
6
Completed
9/27/2026
10/14/2026
Stderr
None captured
None captured