Compare companies

Side-by-side CLI results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Selected

Company 1

Company 2

Deepgram

Voice STT

Vercel

Sandboxes

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice STT

Sandboxes

Overall rank

Best

#43

Category rank

Best

Full eval

Open report

CLI Task Scoring

Eval score

85%

Best

21%

Task Completed

Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the...

Best

Failed+0The agent successfully installed the vercel CLI, authenticated with `vercel whoami`, created the project directory struc...

Used CLI Correctly

Passed+20Agent used CLI commands correctly

Best

Failed+0CLI usage issues: Used curl instead of vercel CLI to call the function endpoint (command [9]), Did not successfully exec...

Partial Credit

Passed+10Partial credit: 100%

Best

Partial+6Partial credit: 60%

Zero Errors

Passed+10No errors

Best

Failed+0Failed tool calls: 1

Efficiency

Partial+3.511 tool calls (<=15, good)

Best

Partial+3.511 tool calls (<=15, good)

Best

Time

Partial+6103s (1-2min, great)

Best

Partial+685s (1-2min, great)

Best

Cost

Partial+6$0.11 ($0.10-0.20, great)

Best

Partial+6$0.12 ($0.10-0.20, great)

Best

Runtime (actual)

1m 42s

1m 25s

Best

Cost (actual)

$0.1075

Best

$0.1189

Tool calls

Best

Errors

Best

Tokens used

374,221

Best

406,332

CLI Discovery

Discovery score

95%

Best

78%

CLI exists

Passed5/5dg

Best

Passed5/5vercel

Best

Install docs

Passed5/5pip

Best

Passed5/5npm

Best

Auth method

Passed25/25env var

Best

Passed25/25env var

Best

Non-interactive

Passed25/25auto detect

Best

Partial18/25flag

JSON output

Passed15/15full

Best

Failed0/15none

llms.txt

Failed0/5

Passed5/5

Best

Agent skill

Passed20/20

Best

Passed20/20

Best

Eval Details

Integration task

# Task Using the dg CLI (Deepgram), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...

# Task Using the vercel CLI (Vercel), create and run a serverless function locally: 1. Check auth status: `vercel whoami` 2. Create a new project directory /home/daytona/app/verc...

Generated files

Trace tool calls

Completed

9/27/2026

Stderr

None captured

Metric	Deepgram Voice STT	Vercel Sandboxes
Summary
Compare status	Comparable	Comparable
Overall score	88 Best	38
Grade	B	D
Category	Voice STT	Sandboxes
Overall rank	#4 Best	#43
Category rank	#1 Best	#6
Full eval	Open report	Open report
CLI Task Scoring
Eval score	85% Best	21%
Task Completed	Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the... Best	Failed+0The agent successfully installed the vercel CLI, authenticated with `vercel whoami`, created the project directory struc...
Used CLI Correctly	Passed+20Agent used CLI commands correctly Best	Failed+0CLI usage issues: Used curl instead of vercel CLI to call the function endpoint (command [9]), Did not successfully exec...
Partial Credit	Passed+10Partial credit: 100% Best	Partial+6Partial credit: 60%
Zero Errors	Passed+10No errors Best	Failed+0Failed tool calls: 1
Efficiency	Partial+3.511 tool calls (<=15, good) Best	Partial+3.511 tool calls (<=15, good) Best
Time	Partial+6103s (1-2min, great) Best	Partial+685s (1-2min, great) Best
Cost	Partial+6$0.11 ($0.10-0.20, great) Best	Partial+6$0.12 ($0.10-0.20, great) Best
Runtime (actual)	1m 42s	1m 25s Best
Cost (actual)	$0.1075 Best	$0.1189
Tool calls	11 Best	11 Best
Errors	0 Best	1
Tokens used	374,221 Best	406,332
CLI Discovery
Discovery score	95% Best	78%
CLI exists	Passed5/5dg Best	Passed5/5vercel Best
Install docs	Passed5/5pip Best	Passed5/5npm Best
Auth method	Passed25/25env var Best	Passed25/25env var Best
Non-interactive	Passed25/25auto detect Best	Partial18/25flag
JSON output	Passed15/15full Best	Failed0/15none
llms.txt	Failed0/5	Passed5/5 Best
Agent skill	Passed20/20 Best	Passed20/20 Best
Eval Details
Integration task	# Task Using the dg CLI (Deepgram), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...	# Task Using the vercel CLI (Vercel), create and run a serverless function locally: 1. Check auth status: `vercel whoami` 2. Create a new project directory /home/daytona/app/verc...
Generated files	1	2
Trace tool calls	11	11
Completed	9/27/2026	9/27/2026
Stderr	None captured	None captured