Compare companies

Side-by-side CLI results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Selected

Company 1

Company 2

Deepgram

Voice STT

AssemblyAI

Voice STT

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Voice STT

Overall rank

Best

#15

Category rank

Best

Full eval

Open report

CLI Task Scoring

Eval score

85%

Best

79%

Task Completed

Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the...

Best

Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the...

Best

Used CLI Correctly

Passed+20Agent used CLI commands correctly

Best

Passed+20Agent used CLI commands correctly

Best

Partial Credit

Passed+10Partial credit: 100%

Best

Passed+10Partial credit: 100%

Best

Zero Errors

Passed+10No errors

Best

Failed+0Failed tool calls: 2

Efficiency

Partial+3.511 tool calls (<=15, good)

Best

Partial+3.512 tool calls (<=15, good)

Best

Time

Partial+6103s (1-2min, great)

Best

Partial+6105s (1-2min, great)

Best

Cost

Partial+6$0.11 ($0.10-0.20, great)

Passed+10$0.10 (<$0.10, excellent)

Best

Runtime (actual)

1m 42s

Best

1m 44s

Cost (actual)

$0.1075

$0.0972

Best

Tool calls

Best

Errors

Best

Tokens used

374,221

Best

381,257

CLI Discovery

Discovery score

95%

Best

70%

CLI exists

Passed5/5dg

Best

Passed5/5assemblyai

Best

Install docs

Passed5/5pip

Best

Passed5/5curl script

Best

Auth method

Passed25/25env var

Best

Passed25/25api key cmd

Best

Non-interactive

Passed25/25auto detect

Best

Failed0/25none

JSON output

Passed15/15full

Best

Passed15/15full

Best

llms.txt

Failed0/5

Best

Failed0/5

Best

Agent skill

Passed20/20

Best

Passed20/20

Best

Eval Details

Integration task

# Task Using the dg CLI (Deepgram), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...

# Task Using the assemblyai CLI (AssemblyAI), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sport...

Generated files

Trace tool calls

Completed

9/27/2026

Stderr

None captured

Metric	Deepgram Voice STT	AssemblyAI Voice STT
Summary
Compare status	Comparable	Comparable
Overall score	88 Best	76
Grade	B	B
Category	Voice STT	Voice STT
Overall rank	#4 Best	#15
Category rank	#1 Best	#2
Full eval	Open report	Open report
CLI Task Scoring
Eval score	85% Best	79%
Task Completed	Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the... Best	Passed+30The agent successfully completed all required steps: (1) downloaded the sample audio file using curl, (2) installed the... Best
Used CLI Correctly	Passed+20Agent used CLI commands correctly Best	Passed+20Agent used CLI commands correctly Best
Partial Credit	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Passed+10No errors Best	Failed+0Failed tool calls: 2
Efficiency	Partial+3.511 tool calls (<=15, good) Best	Partial+3.512 tool calls (<=15, good) Best
Time	Partial+6103s (1-2min, great) Best	Partial+6105s (1-2min, great) Best
Cost	Partial+6$0.11 ($0.10-0.20, great)	Passed+10$0.10 (<$0.10, excellent) Best
Runtime (actual)	1m 42s Best	1m 44s
Cost (actual)	$0.1075	$0.0972 Best
Tool calls	11 Best	12
Errors	0 Best	2
Tokens used	374,221 Best	381,257
CLI Discovery
Discovery score	95% Best	70%
CLI exists	Passed5/5dg Best	Passed5/5assemblyai Best
Install docs	Passed5/5pip Best	Passed5/5curl script Best
Auth method	Passed25/25env var Best	Passed25/25api key cmd Best
Non-interactive	Passed25/25auto detect Best	Failed0/25none
JSON output	Passed15/15full Best	Passed15/15full Best
llms.txt	Failed0/5 Best	Failed0/5 Best
Agent skill	Passed20/20 Best	Passed20/20 Best
Eval Details
Integration task	# Task Using the dg CLI (Deepgram), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...	# Task Using the assemblyai CLI (AssemblyAI), transcribe a sample audio file: 1. Download this sample audio file: https://storage.googleapis.com/aai-web-samples/5_common_sport...
Generated files	1	1
Trace tool calls	11	12
Completed	9/27/2026	9/27/2026
Stderr	None captured	None captured