Compare companies

Side-by-side MCP results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

Resend

LiveKit

Voice Infra

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice Infra

Overall rank

#15

Best

#34

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

84%

Best

49%

Task

Passed+30The agent successfully completed all required tasks using only MCP tools. It discovered MCP capabilities, listed availab...

Best

Failed+0The agent correctly used MCP tools (docs_search and get_pages) to discover what tools were available, which is step 1 of...

Used MCP Correctly

Passed+20Agent made 5 MCP tool calls

Best

Passed+20Agent made 2 MCP tool calls

Best

Partial Credit

Passed+10Partial credit: 100%

Best

Partial+1.5Partial credit: 15%

Zero Errors

Passed+10No errors

Best

Passed+10No errors

Best

Efficiency

Partial+67 tool calls (<=10, great)

Best

Partial+66 tool calls (<=10, great)

Best

Time

Partial+2.0215s (3-5min, slow)

Best

Partial+2.0187s (3-5min, slow)

Best

Cost

Partial+6$0.11 ($0.10-0.20, great)

Passed+10$0.09 (<$0.10, excellent)

Best

Runtime (actual)

3m 34s

3m 6s

Best

Cost (actual)

$0.1147

$0.0920

Best

Tool calls

Best

Errors

Best

Tokens used

372,658

213,444

Best

MCP Discovery

Discovery score

70%

Best

70%

Best

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Failed0/15none

Tool Count

Failed0/10106 tools, bloated

Passed10/109 tools, lean

Best

Tool Description Length

Failed0/1029,709 tokens, bloated

Partial4/106,352 tokens, heavy

Best

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce...

Best

Partial9/10No API keys or tokens visible in descriptions or schemas; parameters are clearly typed and non-destructive; minor concer...

Best

Tool API Mapping

Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...

Partial14/15Tools map cleanly 1:1 to individual operations (search docs, search code, get changelog, fetch pages, get pricing, submi...

Best

Description Clarity

Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi...

Best

Partial13/15Tool descriptions are comprehensive and explicit about purpose, with excellent guidance on when/how to use each tool (e....

Eval Details

Integration task

# Task Using the Resend MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3. S...

# Task Using the LiveKit MCP, manage real-time voice infrastructure. 1. Discover the MCP tools relevant to rooms, sessions, or calls 2. List existing rooms or sessions 3. Create...

Generated files

Trace tool calls

Completed

10/14/2026

9/27/2026

Stderr

None captured

Metric	Resend Email	LiveKit Voice Infra	Agentmail Email
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	79	55	80 Best
Grade	B	C	B
Category	Email	Voice Infra	Email
Overall rank	#15	#34	#13 Best
Category rank	#2	#1 Best	#1 Best
Full eval	Open report	Open report	Open report
MCP Task Scoring
Task score	84% Best	49%	83%
Task	Passed+30The agent successfully completed all required tasks using only MCP tools. It discovered MCP capabilities, listed availab... Best	Failed+0The agent correctly used MCP tools (docs_search and get_pages) to discover what tools were available, which is step 1 of...	Passed+30The agent successfully completed all required tasks using only MCP tools. It: (1) discovered and listed available MCP to... Best
Used MCP Correctly	Passed+20Agent made 5 MCP tool calls Best	Passed+20Agent made 2 MCP tool calls Best	Passed+20Agent made 4 MCP tool calls Best
Partial Credit	Passed+10Partial credit: 100% Best	Partial+1.5Partial credit: 15%	Passed+10Partial credit: 100% Best
Zero Errors	Passed+10No errors Best	Passed+10No errors Best	Passed+10No errors Best
Efficiency	Partial+67 tool calls (<=10, great) Best	Partial+66 tool calls (<=10, great) Best	Partial+66 tool calls (<=10, great) Best
Time	Partial+2.0215s (3-5min, slow) Best	Partial+2.0187s (3-5min, slow) Best	Partial+1.0301s (5-10min, very slow)
Cost	Partial+6$0.11 ($0.10-0.20, great)	Passed+10$0.09 (<$0.10, excellent) Best	Partial+6$0.12 ($0.10-0.20, great)
Runtime (actual)	3m 34s	3m 6s Best	5m 0s
Cost (actual)	$0.1147	$0.0920 Best	$0.1215
Tool calls	7	6 Best	6 Best
Errors	0 Best	0 Best	0 Best
Tokens used	372,658	213,444 Best	305,245
MCP Discovery
Discovery score	70%	70%	75% Best
Official MCP	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Failed0/15none	Passed15/15env var Best
Tool Count	Failed0/10106 tools, bloated	Passed10/109 tools, lean Best	Partial3/1041 tools, heavy
Tool Description Length	Failed0/1029,709 tokens, bloated	Partial4/106,352 tokens, heavy Best	Failed0/1013,234 tokens, bloated
llms.txt	Passed10/10 Best	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Security Posture	Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce... Best	Partial9/10No API keys or tokens visible in descriptions or schemas; parameters are clearly typed and non-destructive; minor concer... Best	Partial9/10No API keys or tokens visible in descriptions or schemas; destructive operations (delete_inbox, delete_thread, delete_dr... Best
Tool API Mapping	Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...	Partial14/15Tools map cleanly 1:1 to individual operations (search docs, search code, get changelog, fetch pages, get pricing, submi... Best	Partial14/15Tools map cleanly 1:1 to API operations with clear, granular operations (list, search, get, create, update, delete patte... Best
Description Clarity	Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi... Best	Partial13/15Tool descriptions are comprehensive and explicit about purpose, with excellent guidance on when/how to use each tool (e....	Partial14/15Descriptions are comprehensive and specific about what each tool does, with clear guidance on inputs, outputs, and usage... Best
Eval Details
Integration task	# Task Using the Resend MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3. S...	# Task Using the LiveKit MCP, manage real-time voice infrastructure. 1. Discover the MCP tools relevant to rooms, sessions, or calls 2. List existing rooms or sessions 3. Create...	# Task Using the Agentmail MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3...
Generated files	1	1	1
Trace tool calls	7	6	6
Completed	10/14/2026	9/27/2026	10/14/2026
Stderr	None captured	None captured	None captured