Compare companies

Side-by-side CLI results for companies evaluated by Codex. Compare score, rank, discoverability, cost, errors, runtime, and the underlying eval artifacts.

Model

Claude Code Codex

Codex

Tooling

API CLI MCP

Choose 2-4 companies

Selected

Company 1

Company 2

Showing the first two companies on mobile. Use a wider screen to compare 3 companies.

Selected

Company 1

Company 2

Company 3

Mobile compare is limited to two companies. Use a wider screen to compare all 3.

OpenRouter

Inference

Daytona

Sandboxes

Summary

Compare status

Comparable

Overall score

Best

Grade

Category

Inference

Sandboxes

Overall rank

#49

#21

Best

Category rank

Best

Full eval

Open report

CLI Task Scoring

Eval score

17%

74%

Best

Task Completed

Failed+0The agent successfully extracted structured invoice data and produced valid JSON output with all required fields (invoic...

Passed+30The agent successfully completed all required steps using the official daytona CLI. Step 1: Listed 6 existing sandboxes...

Best

Used CLI Correctly

Failed+0CLI usage issues: Used @mrgoonie/openrouter-cli instead of the official @openrouter/cli from the task specification, The...

Passed+20Agent used CLI commands correctly

Best

Partial Credit

Partial+6Partial credit: 60%

Passed+10Partial credit: 100%

Best

Zero Errors

Failed+0Failed tool calls: 2

Best

Failed+0Failed tool calls: 3

Best

Efficiency

Partial+2.019 tool calls (<=20, acceptable)

Best

Partial+2.017 tool calls (<=20, acceptable)

Best

Time

Partial+3.5147s (2-3min, good)

Partial+696s (1-2min, great)

Best

Cost

Partial+6$0.19 ($0.10-0.20, great)

Best

Partial+6$0.11 ($0.10-0.20, great)

Best

Runtime (actual)

2m 26s

1m 36s

Best

Cost (actual)

$0.1949

$0.1129

Best

Tool calls

Best

Errors

Best

Tokens used

817,040

442,158

Best

CLI Discovery

Discovery score

60%

70%

Best

CLI exists

Passed5/5openrouter

Best

Passed5/5daytona

Best

Install docs

Passed5/5npm

Best

Passed5/5binary download

Best

Auth method

Passed25/25env var

Best

Passed25/25env var

Best

Non-interactive

Failed0/25none

Best

Failed0/25none

Best

JSON output

Failed0/15none

Partial10/15partial

Best

llms.txt

Passed5/5

Best

Passed5/5

Best

Agent skill

Passed20/20

Best

Passed20/20

Best

Eval Details

Integration task

# Task Using the openrouter CLI (OpenRouter), extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Ser...

# Task Using the daytona CLI (Daytona), manage cloud sandboxes: 1. List all existing sandboxes (or environments/instances) 2. Create a new sandbox environment using the CLI 3. Li...

Generated files

Trace tool calls

Completed

9/27/2026

Stderr

None captured

Metric	OpenRouter Inference	Daytona Sandboxes	Stripe Payment
Summary
Compare status	Comparable	Comparable	Comparable
Overall score	30	72	79 Best
Grade	D	C	B
Category	Inference	Sandboxes	Payment
Overall rank	#49	#21	#10 Best
Category rank	#2	#4	#1 Best
Full eval	Open report	Open report	Open report
CLI Task Scoring
Eval score	17%	74% Best	74% Best
Task Completed	Failed+0The agent successfully extracted structured invoice data and produced valid JSON output with all required fields (invoic...	Passed+30The agent successfully completed all required steps using the official daytona CLI. Step 1: Listed 6 existing sandboxes... Best	Passed+30The agent successfully completed all required tasks using the official Stripe CLI. The trace shows: (1) installation of... Best
Used CLI Correctly	Failed+0CLI usage issues: Used @mrgoonie/openrouter-cli instead of the official @openrouter/cli from the task specification, The...	Passed+20Agent used CLI commands correctly Best	Passed+20Agent used CLI commands correctly Best
Partial Credit	Partial+6Partial credit: 60%	Passed+10Partial credit: 100% Best	Passed+10Partial credit: 100% Best
Zero Errors	Failed+0Failed tool calls: 2 Best	Failed+0Failed tool calls: 3 Best	Failed+0Failed tool calls: 2 Best
Efficiency	Partial+2.019 tool calls (<=20, acceptable) Best	Partial+2.017 tool calls (<=20, acceptable) Best	Partial+2.017 tool calls (<=20, acceptable) Best
Time	Partial+3.5147s (2-3min, good)	Partial+696s (1-2min, great) Best	Partial+6106s (1-2min, great) Best
Cost	Partial+6$0.19 ($0.10-0.20, great) Best	Partial+6$0.11 ($0.10-0.20, great) Best	Partial+6$0.11 ($0.10-0.20, great) Best
Runtime (actual)	2m 26s	1m 36s Best	1m 46s
Cost (actual)	$0.1949	$0.1129	$0.1091 Best
Tool calls	19	17 Best	17 Best
Errors	2 Best	3	2 Best
Tokens used	817,040	442,158	427,110 Best
CLI Discovery
Discovery score	60%	70%	93% Best
CLI exists	Passed5/5openrouter Best	Passed5/5daytona Best	Passed5/5stripe Best
Install docs	Passed5/5npm Best	Passed5/5binary download Best	Passed5/5binary download Best
Auth method	Passed25/25env var Best	Passed25/25env var Best	Passed25/25env var Best
Non-interactive	Failed0/25none	Failed0/25none	Partial18/25flag Best
JSON output	Failed0/15none	Partial10/15partial	Passed15/15default json Best
llms.txt	Passed5/5 Best	Passed5/5 Best	Passed5/5 Best
Agent skill	Passed20/20 Best	Passed20/20 Best	Passed20/20 Best
Eval Details
Integration task	# Task Using the openrouter CLI (OpenRouter), extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Ser...	# Task Using the daytona CLI (Daytona), manage cloud sandboxes: 1. List all existing sandboxes (or environments/instances) 2. Create a new sandbox environment using the CLI 3. Li...	# Task Using the stripe CLI (Stripe), create payment resources: 1. Create a product called "Pro Plan" using the CLI 2. Create a price of $29/month (recurring) for that product 3....
Generated files	1	1	1
Trace tool calls	19	17	17
Completed	9/27/2026	9/27/2026	9/27/2026
Stderr	None captured	None captured	None captured