Compare companies

Side-by-side MCP results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

Qdrant

Vector Databases

Exa

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Vector Databases

Overall rank

Best

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

96%

Best

92%

Task

Passed+30The agent successfully completed all required tasks using only MCP tools. It stored 3 FAQ-style entries about [REDACTED:...

Best

Passed+30The agent successfully completed all required steps: (1) discovered available MCP tools (web_search_exa and web_fetch_ex...

Best

Used MCP Correctly

Passed+20Agent made 4 MCP tool calls

Best

Passed+20Agent made 3 MCP tool calls

Best

Partial Credit

Passed+10Partial credit: 100%

Best

Passed+10Partial credit: 100%

Best

Zero Errors

Passed+10No errors

Best

Passed+10No errors

Best

Efficiency

Partial+66 tool calls (<=10, great)

Passed+105 tool calls (<=5, excellent)

Best

Time

Passed+1054s (<60s, excellent)

Best

Partial+6119s (1-2min, great)

Cost

Passed+10$0.09 (<$0.10, excellent)

Best

Partial+6$0.16 ($0.10-0.20, great)

Runtime (actual)

54s

Best

1m 58s

Cost (actual)

$0.0890

Best

$0.1613

Tool calls

Best

Errors

Best

Tokens used

171,578

Best

456,095

MCP Discovery

Discovery score

89%

91%

Best

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Passed15/15env var

Best

Tool Count

Passed10/102 tools, lean

Best

Passed10/103 tools, lean

Best

Tool Description Length

Passed10/10226 tokens, lean

Best

Passed10/101,023 tokens, lean

Best

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are clearly typed, though the permiss...

Best

Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are well-typed and safe; minor consid...

Best

Tool API Mapping

Partial14/15Tools map cleanly 1:1 to individual Qdrant operations (search and insert); each tool does one thing and an agent can com...

Best

Partial13/15Tools map cleanly to individual API operations (search, fetch, agent run) with minimal workflow wrapping; agent_run invo...

Description Clarity

Partial11/15Descriptions clearly explain the purpose of each tool (search vs. store memories) and when to use them, though they lack...

Partial14/15Descriptions are comprehensive and clearly explain tool purpose, when to use each tool, expected outputs, and practical...

Best

Eval Details

Integration task

# Task Using the Qdrant MCP, perform semantic memory operations: 1. Use the MCP tools relevant to storing and finding memories. You do not need to enumerate the full tool catalog...

# Task Using the Exa MCP, run a small research workflow: 1. Discover available MCP tools and their capabilities 2. Run TWO related searches to build a picture of a topic: a. A...

Generated files

Trace tool calls

Completed

10/14/2026

9/27/2026

Stderr

None captured

Metric	Qdrant Vector Databases	Exa Search	ElevenLabs Voice TTS
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	93 Best	91	89
Grade	A	A	B
Category	Vector Databases	Search	Voice TTS
Overall rank	#1 Best	#2	#3
Category rank	#1 Best	#1 Best	#1 Best
Full eval	Open report	Open report	Open report
MCP Task Scoring
Task score	96% Best	92%	93%
Task	Passed+30The agent successfully completed all required tasks using only MCP tools. It stored 3 FAQ-style entries about [REDACTED:... Best	Passed+30The agent successfully completed all required steps: (1) discovered available MCP tools (web_search_exa and web_fetch_ex... Best	Passed+30The agent successfully completed all task requirements: (1) Discovered the text_to_speech MCP tool, (2) Converted the sp... Best
Used MCP Correctly	Passed+20Agent made 4 MCP tool calls Best	Passed+20Agent made 3 MCP tool calls Best	Passed+20Agent made 1 MCP tool calls Best
Partial Credit	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Passed+10No errors Best	Passed+10No errors Best	Passed+10No errors Best
Efficiency	Partial+66 tool calls (<=10, great)	Passed+105 tool calls (<=5, excellent) Best	Passed+104 tool calls (<=5, excellent) Best
Time	Passed+1054s (<60s, excellent) Best	Partial+6119s (1-2min, great)	Partial+3.5138s (2-3min, good)
Cost	Passed+10$0.09 (<$0.10, excellent) Best	Partial+6$0.16 ($0.10-0.20, great)	Passed+10$0.06 (<$0.10, excellent) Best
Runtime (actual)	54s Best	1m 58s	2m 18s
Cost (actual)	$0.0890	$0.1613	$0.0625 Best
Tool calls	6	5	4 Best
Errors	0 Best	0 Best	0 Best
Tokens used	171,578 Best	456,095	181,168
MCP Discovery
Discovery score	89%	91% Best	80%
Official MCP	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Passed15/15env var Best	Passed15/15env var Best
Tool Count	Passed10/102 tools, lean Best	Passed10/103 tools, lean Best	Partial7/1027 tools, moderate
Tool Description Length	Passed10/10226 tokens, lean Best	Passed10/101,023 tokens, lean Best	Partial4/107,050 tokens, heavy
llms.txt	Passed10/10 Best	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Security Posture	Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are clearly typed, though the permiss... Best	Partial9/10No API keys, tokens, or credentials visible in descriptions or schemas; parameters are well-typed and safe; minor consid... Best	Partial9/10No API keys or credentials visible in descriptions or schemas; cost warnings are appropriately placed; minor concern is... Best
Tool API Mapping	Partial14/15Tools map cleanly 1:1 to individual Qdrant operations (search and insert); each tool does one thing and an agent can com... Best	Partial13/15Tools map cleanly to individual API operations (search, fetch, agent run) with minimal workflow wrapping; agent_run invo...	Partial13/155% - simulate_conversation and compose_music are the only tools that wrap multi-step workflows; the rest are direct API...
Description Clarity	Partial11/15Descriptions clearly explain the purpose of each tool (search vs. store memories) and when to use them, though they lack...	Partial14/15Descriptions are comprehensive and clearly explain tool purpose, when to use each tool, expected outputs, and practical... Best	Partial12/15Most tools have clear descriptions explaining purpose and parameters, but some lack detail on edge cases, error handling...
Eval Details
Integration task	# Task Using the Qdrant MCP, perform semantic memory operations: 1. Use the MCP tools relevant to storing and finding memories. You do not need to enumerate the full tool catalog...	# Task Using the Exa MCP, run a small research workflow: 1. Discover available MCP tools and their capabilities 2. Run TWO related searches to build a picture of a topic: a. A...	# Task Using the ElevenLabs MCP, convert text to speech. 1. Discover the MCP tools relevant to speech synthesis 2. Convert this text to speech: "Your order has been shipped an...
Generated files	1	1	1
Trace tool calls	6	5	4
Completed	10/14/2026	9/27/2026	10/14/2026
Stderr	None captured	None captured	None captured