Compare companies

Side-by-side MCP results for companies evaluated by Claude Code. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

Deepgram

Voice STT

Agentmail

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice STT

Overall rank

#51

#12

Best

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

88%

Best

Task

Failed+0The agent attempted to use MCP tools but failed to complete the core task. Only 1 MCP tool call was made (ListAgents), w...

Passed+30The agent successfully completed all required tasks using only Agentmail MCP tools. It discovered available tools, liste...

Best

Used MCP Correctly

Failed+0MCP usage issues: Only 1 MCP tool call made (ListAgents), which is unrelated to transcription, No transcription MCP tool...

Passed+20Agent made 5 MCP tool calls

Best

Partial Credit

Partial+1.5Partial credit: 15%

Passed+10Partial credit: 100%

Best

Zero Errors

Failed+0Failed tool calls: 13

Passed+10No errors

Best

Efficiency

Failed+026 tool calls (>25, inefficient)

Partial+67 tool calls (<=10, great)

Best

Time

Partial+1.0414s (5-10min, very slow)

Partial+2.0280s (3-5min, slow)

Best

Cost

Partial+2.0$0.45 ($0.40-0.70, expensive)

Passed+10$0.10 (<$0.10, excellent)

Best

Runtime (actual)

6m 53s

4m 39s

Best

Cost (actual)

$0.4500

$0.0965

Best

Tool calls

Best

Errors

Best

Tokens used

925,119

180,827

Best

MCP Discovery

Discovery score

92%

Best

75%

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Passed15/15env var

Best

Tool Count

Passed10/101 tools, lean

Best

Partial3/1041 tools, heavy

Tool Description Length

Passed10/10247 tokens, lean

Best

Failed0/1013,234 tokens, bloated

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

Partial9/10No API keys, tokens, or credentials are exposed in the tool definition or schema. The query parameter is a string type w...

Best

Partial9/10No API keys or tokens are exposed in tool descriptions or schemas; parameters are clearly typed and validated; however,...

Best

Tool API Mapping

Partial14/15This tool maps 1:1 to a single semantic search API operation with a minimal, focused input schema (one query parameter)....

Best

Partial14/15Tools map cleanly 1:1 to individual API operations (create_inbox, get_message, list_threads, etc.), with rare exceptions...

Best

Description Clarity

Partial14/15The description clearly explains the tool's purpose (semantic retrieval over Deepgram knowledge sources), what it return...

Best

Partial14/15Descriptions are comprehensive and explain tool purpose, parameters, expected outcomes, and edge cases clearly; agents c...

Best

Eval Details

Integration task

# Task Using the Deepgram MCP, transcribe audio. 1. Discover the MCP tools relevant to transcription 2. Transcribe this sample audio file: https://storage.googleapis.com/aai-w...

# Task Using the Agentmail MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3...

Generated files

Trace tool calls

Completed

9/27/2026

10/14/2026

Stderr

None captured

Metric	Deepgram Voice STT	Agentmail Email	Resend Email
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	30	84 Best	81
Grade	D	B	B
Category	Voice STT	Email	Email
Overall rank	#51	#12 Best	#19
Category rank	#1 Best	#1 Best	#2
Full eval	Open report	Open report	Open report
MCP Task Scoring
Task score	4%	88% Best	87%
Task	Failed+0The agent attempted to use MCP tools but failed to complete the core task. Only 1 MCP tool call was made (ListAgents), w...	Passed+30The agent successfully completed all required tasks using only Agentmail MCP tools. It discovered available tools, liste... Best	Passed+30The agent successfully completed all five required steps using only Resend MCP tools. Step 1: Tool discovery via ToolSea... Best
Used MCP Correctly	Failed+0MCP usage issues: Only 1 MCP tool call made (ListAgents), which is unrelated to transcription, No transcription MCP tool...	Passed+20Agent made 5 MCP tool calls Best	Passed+20Agent made 4 MCP tool calls Best
Partial Credit	Partial+1.5Partial credit: 15%	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Failed+0Failed tool calls: 13	Passed+10No errors Best	Passed+10No errors Best
Efficiency	Failed+026 tool calls (>25, inefficient)	Partial+67 tool calls (<=10, great) Best	Partial+66 tool calls (<=10, great) Best
Time	Partial+1.0414s (5-10min, very slow)	Partial+2.0280s (3-5min, slow) Best	Partial+1.0440s (5-10min, very slow)
Cost	Partial+2.0$0.45 ($0.40-0.70, expensive)	Passed+10$0.10 (<$0.10, excellent) Best	Passed+10$0.09 (<$0.10, excellent) Best
Runtime (actual)	6m 53s	4m 39s Best	7m 19s
Cost (actual)	$0.4500	$0.0965	$0.0893 Best
Tool calls	26	7	6 Best
Errors	13	0 Best	0 Best
Tokens used	925,119	180,827	159,715 Best
MCP Discovery
Discovery score	92% Best	75%	70%
Official MCP	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Passed15/15env var Best	Passed15/15env var Best
Tool Count	Passed10/101 tools, lean Best	Partial3/1041 tools, heavy	Failed0/10106 tools, bloated
Tool Description Length	Passed10/10247 tokens, lean Best	Failed0/1013,234 tokens, bloated	Failed0/1029,709 tokens, bloated
llms.txt	Passed10/10 Best	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Security Posture	Partial9/10No API keys, tokens, or credentials are exposed in the tool definition or schema. The query parameter is a string type w... Best	Partial9/10No API keys or tokens are exposed in tool descriptions or schemas; parameters are clearly typed and validated; however,... Best	Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce... Best
Tool API Mapping	Partial14/15This tool maps 1:1 to a single semantic search API operation with a minimal, focused input schema (one query parameter).... Best	Partial14/15Tools map cleanly 1:1 to individual API operations (create_inbox, get_message, list_threads, etc.), with rare exceptions... Best	Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...
Description Clarity	Partial14/15The description clearly explains the tool's purpose (semantic retrieval over Deepgram knowledge sources), what it return... Best	Partial14/15Descriptions are comprehensive and explain tool purpose, parameters, expected outcomes, and edge cases clearly; agents c... Best	Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi... Best
Eval Details
Integration task	# Task Using the Deepgram MCP, transcribe audio. 1. Discover the MCP tools relevant to transcription 2. Transcribe this sample audio file: https://storage.googleapis.com/aai-w...	# Task Using the Agentmail MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3...	# Task Using the Resend MCP, perform email operations: 1. Discover available MCP tools and their capabilities 2. Create a new email inbox (or list available inboxes/domains) 3. S...
Generated files	1	1	1
Trace tool calls	26	7	6
Completed	9/27/2026	10/14/2026	9/27/2026
Stderr	None captured	None captured	None captured