Compare companies

Side-by-side MCP results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Selected

Company 1

Company 2

Deepgram

Voice STT

Prefect

Durable Workflow

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice STT

Durable Workflow

Overall rank

#47

#37

Best

Category rank

Best

Full eval

Open report

MCP Task Scoring

Task score

31%

Best

31%

Best

Task

Failed+0The agent correctly attempted to use MCP tools as required (mcp__deepgram__list_mcp_resources, mcp__codex__list_mcp_reso...

Best

Failed+0The agent correctly used MCP tools for discovery and inspection (steps 1-2), properly identified that the Prefect MCP se...

Best

Used MCP Correctly

Passed+20Agent made 5 MCP tool calls

Best

Passed+20Agent made 10 MCP tool calls

Best

Partial Credit

Failed+0Partial credit: 0%

Partial+5Partial credit: 50%

Best

Zero Errors

Failed+0Tool outputs with errors: 5

Best

Failed+0Tool outputs with errors: 7

Best

Efficiency

Partial+3.515 tool calls (<=15, good)

Best

Partial+1.022 tool calls (<=25, below average)

Time

Partial+2.0242s (3-5min, slow)

Best

Partial+2.0189s (3-5min, slow)

Best

Cost

Partial+6$0.12 ($0.10-0.20, great)

Best

Partial+3.5$0.22 ($0.20-0.40, good)

Runtime (actual)

4m 1s

3m 8s

Best

Cost (actual)

$0.1215

Best

$0.2242

Tool calls

Best

Errors

Best

Tokens used

567,330

Best

835,661

MCP Discovery

Discovery score

55%

89%

Best

Official MCP

Passed5/5

Best

Passed5/5

Best

Auth Method

Passed15/15env var

Best

Passed15/15env var

Best

Tool Count

Passed10/103 tools, lean

Best

Passed10/1015 tools, lean

Best

Tool Description Length

Passed10/1029 tokens, lean

Best

Partial7/103,307 tokens, moderate

llms.txt

Passed10/10

Best

Passed10/10

Best

Output Schema

Failed0/5

Best

Failed0/5

Best

Tool Stability

Passed5/5

Best

Passed5/5

Best

Security Posture

—

Partial9/10No API keys or credentials visible in tool definitions; all tools are read-only inspection operations with no destructiv...

Tool API Mapping

—

Partial14/15Tools map cleanly 1:1 to individual API read operations (get_flows, get_deployments, get_task_runs, etc.) with consisten...

Description Clarity

—

Partial14/15Descriptions are comprehensive and explicit about tool purpose, parameters, filter operators, and usage examples; only m...

Eval Details

Integration task

# Task Using the Deepgram MCP, transcribe audio. 1. Discover the MCP tools relevant to transcription 2. Transcribe this sample audio file: https://storage.googleapis.com/aai-w...

# Task Using the Prefect MCP, interact with the workflow engine: 1. Discover available MCP tools and their capabilities 2. List existing workflow executions (or deployments/flows...

Generated files

Trace tool calls

Completed

9/27/2026

10/4/2026

Stderr

None captured

Metric	Deepgram Voice STT	Prefect Durable Workflow
Summary
Compare status	Comparable	Comparable
Overall score	38	48 Best
Grade	D	D
Category	Voice STT	Durable Workflow
Overall rank	#47	#37 Best
Category rank	#1 Best	#1 Best
Full eval	Open report	Open report
MCP Task Scoring
Task score	31% Best	31% Best
Task	Failed+0The agent correctly attempted to use MCP tools as required (mcp__deepgram__list_mcp_resources, mcp__codex__list_mcp_reso... Best	Failed+0The agent correctly used MCP tools for discovery and inspection (steps 1-2), properly identified that the Prefect MCP se... Best
Used MCP Correctly	Passed+20Agent made 5 MCP tool calls Best	Passed+20Agent made 10 MCP tool calls Best
Partial Credit	Failed+0Partial credit: 0%	Partial+5Partial credit: 50% Best
Zero Errors	Failed+0Tool outputs with errors: 5 Best	Failed+0Tool outputs with errors: 7 Best
Efficiency	Partial+3.515 tool calls (<=15, good) Best	Partial+1.022 tool calls (<=25, below average)
Time	Partial+2.0242s (3-5min, slow) Best	Partial+2.0189s (3-5min, slow) Best
Cost	Partial+6$0.12 ($0.10-0.20, great) Best	Partial+3.5$0.22 ($0.20-0.40, good)
Runtime (actual)	4m 1s	3m 8s Best
Cost (actual)	$0.1215 Best	$0.2242
Tool calls	15 Best	22
Errors	5 Best	7
Tokens used	567,330 Best	835,661
MCP Discovery
Discovery score	55%	89% Best
Official MCP	Passed5/5 Best	Passed5/5 Best
Auth Method	Passed15/15env var Best	Passed15/15env var Best
Tool Count	Passed10/103 tools, lean Best	Passed10/1015 tools, lean Best
Tool Description Length	Passed10/1029 tokens, lean Best	Partial7/103,307 tokens, moderate
llms.txt	Passed10/10 Best	Passed10/10 Best
Output Schema	Failed0/5 Best	Failed0/5 Best
Tool Stability	Passed5/5 Best	Passed5/5 Best
Security Posture	—	Partial9/10No API keys or credentials visible in tool definitions; all tools are read-only inspection operations with no destructiv...
Tool API Mapping	—	Partial14/15Tools map cleanly 1:1 to individual API read operations (get_flows, get_deployments, get_task_runs, etc.) with consisten...
Description Clarity	—	Partial14/15Descriptions are comprehensive and explicit about tool purpose, parameters, filter operators, and usage examples; only m...
Eval Details
Integration task	# Task Using the Deepgram MCP, transcribe audio. 1. Discover the MCP tools relevant to transcription 2. Transcribe this sample audio file: https://storage.googleapis.com/aai-w...	# Task Using the Prefect MCP, interact with the workflow engine: 1. Discover available MCP tools and their capabilities 2. List existing workflow executions (or deployments/flows...
Generated files	1	1
Trace tool calls	15	22
Completed	9/27/2026	10/4/2026
Stderr	None captured	None captured