Compare companies

Side-by-side MCP results for companies evaluated by Claude Code. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

Resend

LiveKit

Voice Infra

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice Infra

Overall rank

#19

Best

#37

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

87%

Best

43%

Task

Passed+30The agent successfully completed all five required steps using only Resend MCP tools. Step 1: Tool discovery via ToolSea...

Best

Failed+0The agent correctly identified and used MCP tools (get_docs_overview, docs_search, get_pages) to discover that the avail...

Used MCP Correctly

Passed+20Agent made 4 MCP tool calls

Best

Passed+20Agent made 3 MCP tool calls

Best

Partial Credit

Passed+10Partial credit: 100%

Best

Partial+2.5Partial credit: 25%

Zero Errors

Passed+10No errors

Best

Passed+10No errors

Best

Efficiency

Partial+66 tool calls (<=10, great)

Best

Partial+69 tool calls (<=10, great)

Best

Time

Partial+1.0440s (5-10min, very slow)

Best

Partial+1.0307s (5-10min, very slow)

Best

Cost

Passed+10$0.09 (<$0.10, excellent)

Best

Partial+3.5$0.22 ($0.20-0.40, good)

Runtime (actual)

7m 19s

5m 7s

Best

Cost (actual)

$0.0893

Best

$0.2234

Tool calls

Best

Errors

Best

Tokens used

159,715

Best

368,381

MCP Discovery

Discovery score

70%

71%

Best

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Failed0/15none

Tool Count

Failed0/10106 tools, bloated

Passed10/109 tools, lean

Best

Tool Description Length

Failed0/1029,709 tokens, bloated

Partial4/105,300 tokens, heavy

Best

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce...

Best

Partial9/10No API keys or credentials visible in descriptions or schemas; parameters are well-typed; minor issue is that context pa...

Best

Tool API Mapping

Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...

Partial14/15Tools map cleanly 1:1 to individual API operations (search code, search docs, get changelog, etc.); each tool performs o...

Best

Description Clarity

Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi...

Best

Partial14/15Tool descriptions are comprehensive and clearly explain purpose, usage patterns, and parameters with specific examples;...

Best

Eval Details

Integration task

# Task Using the Resend MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3. S...

# Task Using the LiveKit MCP, manage real-time voice infrastructure. 1. Discover the MCP tools relevant to rooms, sessions, or calls 2. List existing rooms or sessions 3. Create...

Generated files

Trace tool calls

Completed

9/27/2026

Stderr

None captured

Metric	Resend Email	LiveKit Voice Infra	Agentmail Email
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	81	51	84 Best
Grade	B	D	B
Category	Email	Voice Infra	Email
Overall rank	#19	#37	#12 Best
Category rank	#2	#1 Best	#1 Best
Full eval	Open report	Open report	Open report
MCP Task Scoring
Task score	87%	43%	88% Best
Task	Passed+30The agent successfully completed all five required steps using only Resend MCP tools. Step 1: Tool discovery via ToolSea... Best	Failed+0The agent correctly identified and used MCP tools (get_docs_overview, docs_search, get_pages) to discover that the avail...	Passed+30The agent successfully completed all required tasks using only Agentmail MCP tools. It discovered available tools, liste... Best
Used MCP Correctly	Passed+20Agent made 4 MCP tool calls Best	Passed+20Agent made 3 MCP tool calls Best	Passed+20Agent made 5 MCP tool calls Best
Partial Credit	Passed+10Partial credit: 100% Best	Partial+2.5Partial credit: 25%	Passed+10Partial credit: 100% Best
Zero Errors	Passed+10No errors Best	Passed+10No errors Best	Passed+10No errors Best
Efficiency	Partial+66 tool calls (<=10, great) Best	Partial+69 tool calls (<=10, great) Best	Partial+67 tool calls (<=10, great) Best
Time	Partial+1.0440s (5-10min, very slow)	Partial+1.0307s (5-10min, very slow)	Partial+2.0280s (3-5min, slow) Best
Cost	Passed+10$0.09 (<$0.10, excellent) Best	Partial+3.5$0.22 ($0.20-0.40, good)	Passed+10$0.10 (<$0.10, excellent) Best
Runtime (actual)	7m 19s	5m 7s	4m 39s Best
Cost (actual)	$0.0893 Best	$0.2234	$0.0965
Tool calls	6 Best	9	7
Errors	0 Best	0 Best	0 Best
Tokens used	159,715 Best	368,381	180,827
MCP Discovery
Discovery score	70%	71%	75% Best
Official MCP	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Failed0/15none	Passed15/15env var Best
Tool Count	Failed0/10106 tools, bloated	Passed10/109 tools, lean Best	Partial3/1041 tools, heavy
Tool Description Length	Failed0/1029,709 tokens, bloated	Partial4/105,300 tokens, heavy Best	Failed0/1013,234 tokens, bloated
llms.txt	Passed10/10 Best	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Security Posture	Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce... Best	Partial9/10No API keys or credentials visible in descriptions or schemas; parameters are well-typed; minor issue is that context pa... Best	Partial9/10No API keys or tokens are exposed in tool descriptions or schemas; parameters are clearly typed and validated; however,... Best
Tool API Mapping	Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...	Partial14/15Tools map cleanly 1:1 to individual API operations (search code, search docs, get changelog, etc.); each tool performs o... Best	Partial14/15Tools map cleanly 1:1 to individual API operations (create_inbox, get_message, list_threads, etc.), with rare exceptions... Best
Description Clarity	Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi... Best	Partial14/15Tool descriptions are comprehensive and clearly explain purpose, usage patterns, and parameters with specific examples;... Best	Partial14/15Descriptions are comprehensive and explain tool purpose, parameters, expected outcomes, and edge cases clearly; agents c... Best
Eval Details
Integration task	# Task Using the Resend MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3. S...	# Task Using the LiveKit MCP, manage real-time voice infrastructure. 1. Discover the MCP tools relevant to rooms, sessions, or calls 2. List existing rooms or sessions 3. Create...	# Task Using the Agentmail MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3...
Generated files	1	1	1
Trace tool calls	6	9	7
Completed	9/27/2026	9/27/2026	10/14/2026
Stderr	None captured	None captured	None captured