Compare companies

Side-by-side MCP results for companies evaluated by Claude Code. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

Qdrant

Vector Databases

Exa

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Vector Databases

Overall rank

Best

#17

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

92%

Best

79%

Task

Passed+30The agent successfully completed all required steps using only MCP tools. It stored 3 FAQ-style entries about [REDACTED:...

Best

Passed+30The agent successfully completed all required steps of the research workflow using only Exa MCP tools. Both searches wer...

Best

Used MCP Correctly

Passed+20Agent made 4 MCP tool calls

Best

Passed+20Agent made 3 MCP tool calls

Best

Partial Credit

Passed+10Partial credit: 100%

Best

Passed+10Partial credit: 100%

Best

Zero Errors

Passed+10No errors

Best

Passed+10No errors

Best

Efficiency

Partial+66 tool calls (<=10, great)

Best

Partial+3.512 tool calls (<=15, good)

Time

Partial+677s (1-2min, great)

Best

Partial+2.0226s (3-5min, slow)

Cost

Passed+10$0.08 (<$0.10, excellent)

Best

Partial+3.5$0.29 ($0.20-0.40, good)

Runtime (actual)

1m 17s

Best

3m 45s

Cost (actual)

$0.0786

Best

$0.2917

Tool calls

Best

Errors

Best

Tokens used

116,404

Best

484,121

MCP Discovery

Discovery score

88%

91%

Best

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Passed15/15env var

Best

Tool Count

Passed10/102 tools, lean

Best

Passed10/103 tools, lean

Best

Tool Description Length

Passed10/10226 tokens, lean

Best

Passed10/101,156 tokens, lean

Best

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

Partial9/10No API keys, tokens, or credentials are visible in descriptions or schemas; however, the metadata parameter accepts arbi...

Best

Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are well-typed and constrained (e.g.,...

Best

Tool API Mapping

Partial14/15Tools map cleanly 1:1 to individual vector database operations (search and upsert/insert), with minimal workflow wrappin...

Best

Partial14/15Tools map cleanly 1:1 to individual API operations (web_search, web_fetch, agent_run) with minimal workflow wrapping; ag...

Best

Description Clarity

Partial10/15Descriptions explain the basic purpose of each tool (find vs. store memories), but lack detail about search behavior, re...

Partial13/15Descriptions are comprehensive and clearly explain when to use each tool, expected inputs/outputs, and important distinc...

Best

Eval Details

Integration task

# Task Using the Qdrant MCP, perform semantic memory operations: 1. Use the MCP tools relevant to storing and finding memories. You do not need to enumerate the full tool catalog...

# Task Using the Exa MCP, run a small research workflow: 1. Discover available MCP tools and their capabilities 2. Run TWO related searches to build a picture of a topic: a. A...

Generated files

Trace tool calls

Completed

10/4/2026

9/27/2026

Stderr

None captured

Metric	Qdrant Vector Databases	Exa Search	ElevenLabs Voice TTS
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	90 Best	82	89
Grade	A	B	B
Category	Vector Databases	Search	Voice TTS
Overall rank	#2 Best	#17	#3
Category rank	#1 Best	#4	#1 Best
Full eval	Open report	Open report	Open report
MCP Task Scoring
Task score	92%	79%	93% Best
Task	Passed+30The agent successfully completed all required steps using only MCP tools. It stored 3 FAQ-style entries about [REDACTED:... Best	Passed+30The agent successfully completed all required steps of the research workflow using only Exa MCP tools. Both searches wer... Best	Passed+30The agent successfully completed all task requirements: (1) discovered the text_to_speech MCP tool via ToolSearch, (2) c... Best
Used MCP Correctly	Passed+20Agent made 4 MCP tool calls Best	Passed+20Agent made 3 MCP tool calls Best	Passed+20Agent made 1 MCP tool calls Best
Partial Credit	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Passed+10No errors Best	Passed+10No errors Best	Passed+10No errors Best
Efficiency	Partial+66 tool calls (<=10, great)	Partial+3.512 tool calls (<=15, good)	Passed+105 tool calls (<=5, excellent) Best
Time	Partial+677s (1-2min, great) Best	Partial+2.0226s (3-5min, slow)	Partial+3.5167s (2-3min, good)
Cost	Passed+10$0.08 (<$0.10, excellent) Best	Partial+3.5$0.29 ($0.20-0.40, good)	Passed+10$0.07 (<$0.10, excellent) Best
Runtime (actual)	1m 17s Best	3m 45s	2m 46s
Cost (actual)	$0.0786	$0.2917	$0.0698 Best
Tool calls	6	12	5 Best
Errors	0 Best	0 Best	0 Best
Tokens used	116,404 Best	484,121	139,341
MCP Discovery
Discovery score	88%	91% Best	80%
Official MCP	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Passed15/15env var Best	Passed15/15env var Best
Tool Count	Passed10/102 tools, lean Best	Passed10/103 tools, lean Best	Partial7/1027 tools, moderate
Tool Description Length	Passed10/10226 tokens, lean Best	Passed10/101,156 tokens, lean Best	Partial4/107,840 tokens, heavy
llms.txt	Passed10/10 Best	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Security Posture	Partial9/10No API keys, tokens, or credentials are visible in descriptions or schemas; however, the metadata parameter accepts arbi... Best	Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are well-typed and constrained (e.g.,... Best	Partial9/10No API keys or credentials exposed in descriptions or schemas; parameters are properly typed; however, cost warnings are... Best
Tool API Mapping	Partial14/15Tools map cleanly 1:1 to individual vector database operations (search and upsert/insert), with minimal workflow wrappin... Best	Partial14/15Tools map cleanly 1:1 to individual API operations (web_search, web_fetch, agent_run) with minimal workflow wrapping; ag... Best	Partial13/15Tools predominantly map 1:1 to API endpoints (e.g., `create_agent`, `get_voice`, `list_conversations`), with minimal wor...
Description Clarity	Partial10/15Descriptions explain the basic purpose of each tool (find vs. store memories), but lack detail about search behavior, re...	Partial13/15Descriptions are comprehensive and clearly explain when to use each tool, expected inputs/outputs, and important distinc... Best	Partial12/15Most tools have clear, detailed descriptions explaining purpose and parameters, but some lack context on when to use the...
Eval Details
Integration task	# Task Using the Qdrant MCP, perform semantic memory operations: 1. Use the MCP tools relevant to storing and finding memories. You do not need to enumerate the full tool catalog...	# Task Using the Exa MCP, run a small research workflow: 1. Discover available MCP tools and their capabilities 2. Run TWO related searches to build a picture of a topic: a. A...	# Task Using the ElevenLabs MCP, convert text to speech. 1. Discover the MCP tools relevant to speech synthesis 2. Convert this text to speech: "Your order has been shipped an...
Generated files	1	2	1
Trace tool calls	6	12	5
Completed	10/4/2026	9/27/2026	9/27/2026
Stderr	None captured	None captured	None captured