Compare companies

Side-by-side CLI results for companies evaluated by Claude Code. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

OpenRouter

Inference

Daytona

Sandboxes

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Inference

Sandboxes

Overall rank

#50

#21

Best

Category rank

Best

Full eval

Open report

CLI Task Scoring

Eval score

12%

75%

Best

Task Completed

Failed+0The agent successfully extracted structured invoice data with all required fields (invoice_number, vendor, date, line_it...

Passed+30The agent successfully completed all four required steps: (1) Listed existing sandboxes using `daytona list --format jso...

Best

Used CLI Correctly

Failed+0CLI usage issues: Used @mrgoonie/openrouter-cli (third-party package) instead of the official @openrouter/cli for the in...

Passed+20Agent used CLI commands correctly

Best

Partial Credit

Partial+6Partial credit: 60%

Passed+10Partial credit: 100%

Best

Zero Errors

Failed+0Failed tool calls: 1

Best

Failed+0Failed tool calls: 1

Best

Efficiency

Partial+1.023 tool calls (<=25, below average)

Partial+3.512 tool calls (<=15, good)

Best

Time

Partial+2.0184s (3-5min, slow)

Partial+697s (1-2min, great)

Best

Cost

Partial+3.5$0.33 ($0.20-0.40, good)

Partial+6$0.15 ($0.10-0.20, great)

Best

Runtime (actual)

3m 4s

1m 36s

Best

Cost (actual)

$0.3338

$0.1458

Best

Tool calls

Best

Errors

Best

Tokens used

707,116

341,807

Best

CLI Discovery

Discovery score

60%

70%

Best

CLI exists

Passed5/5openrouter

Best

Passed5/5daytona

Best

Install docs

Passed5/5npm

Best

Passed5/5binary download

Best

Auth method

Passed25/25env var

Best

Passed25/25env var

Best

Non-interactive

Failed0/25none

Best

Failed0/25none

Best

JSON output

Failed0/15none

Partial10/15partial

Best

llms.txt

Passed5/5

Best

Passed5/5

Best

Agent skill

Passed20/20

Best

Passed20/20

Best

Eval Details

Integration task

# Task Using the openrouter CLI (OpenRouter), extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Ser...

# Task Using the daytona CLI (Daytona), manage cloud sandboxes: 1. List all existing sandboxes (or environments/instances) 2. Create a new sandbox environment using the CLI 3. Li...

Generated files

Trace tool calls

Completed

9/27/2026

Stderr

None captured

Metric	OpenRouter Inference	Daytona Sandboxes	Stripe Payment
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	26	73	80 Best
Grade	D	C	B
Category	Inference	Sandboxes	Payment
Overall rank	#50	#21	#9 Best
Category rank	#2	#5	#1 Best
Full eval	Open report	Open report	Open report
CLI Task Scoring
Eval score	12%	75% Best	75% Best
Task Completed	Failed+0The agent successfully extracted structured invoice data with all required fields (invoice_number, vendor, date, line_it...	Passed+30The agent successfully completed all four required steps: (1) Listed existing sandboxes using `daytona list --format jso... Best	Passed+30The agent successfully completed all required steps using the official Stripe CLI. Step [4] installed the stripe CLI via... Best
Used CLI Correctly	Failed+0CLI usage issues: Used @mrgoonie/openrouter-cli (third-party package) instead of the official @openrouter/cli for the in...	Passed+20Agent used CLI commands correctly Best	Passed+20Agent used CLI commands correctly Best
Partial Credit	Partial+6Partial credit: 60%	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Failed+0Failed tool calls: 1 Best	Failed+0Failed tool calls: 1 Best	Failed+0Failed tool calls: 1 Best
Efficiency	Partial+1.023 tool calls (<=25, below average)	Partial+3.512 tool calls (<=15, good) Best	Partial+3.511 tool calls (<=15, good) Best
Time	Partial+2.0184s (3-5min, slow)	Partial+697s (1-2min, great) Best	Partial+683s (1-2min, great) Best
Cost	Partial+3.5$0.33 ($0.20-0.40, good)	Partial+6$0.15 ($0.10-0.20, great) Best	Partial+6$0.17 ($0.10-0.20, great) Best
Runtime (actual)	3m 4s	1m 36s	1m 23s Best
Cost (actual)	$0.3338	$0.1458 Best	$0.1656
Tool calls	23	12	11 Best
Errors	1 Best	1 Best	1 Best
Tokens used	707,116	341,807	293,058 Best
CLI Discovery
Discovery score	60%	70%	93% Best
CLI exists	Passed5/5openrouter Best	Passed5/5daytona Best	Passed5/5stripe Best
Install docs	Passed5/5npm Best	Passed5/5binary download Best	Passed5/5binary download Best
Auth method	Passed25/25env var Best	Passed25/25env var Best	Passed25/25env var Best
Non-interactive	Failed0/25none	Failed0/25none	Partial18/25flag Best
JSON output	Failed0/15none	Partial10/15partial	Passed15/15default json Best
llms.txt	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Agent skill	Passed20/20 Best	Passed20/20 Best	Passed20/20 Best
Eval Details
Integration task	# Task Using the openrouter CLI (OpenRouter), extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Ser...	# Task Using the daytona CLI (Daytona), manage cloud sandboxes: 1. List all existing sandboxes (or environments/instances) 2. Create a new sandbox environment using the CLI 3. Li...	# Task Using the stripe CLI (Stripe), create payment resources: 1. Create a product called "Pro Plan" using the CLI 2. Create a price of $29/month (recurring) for that product 3....
Generated files	3	1	1
Trace tool calls	23	12	11
Completed	9/27/2026	9/27/2026	9/27/2026
Stderr	None captured	None captured	None captured