Back to leaderboard

Reducto vs LlamaParse

Same Document Parsing task · View

#2 in Document Parsing

64CBest on 3 of 19

#3 in Document Parsing

63CBest on 1 of 19

CLI task scoring

Eval score
75%
73%
Task Completed
Passed+30The agent successfully completed all task requirements: (1) installed the official reducto CLI via `pip install reducto-...
Passed+30The agent successfully completed all required steps: (1) installed the official llama-parse CLI via pip, (2) downloaded...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Failed+0Failed tool calls: 1
Failed+0Failed tool calls: 1
Efficiency
Partial+3.514 tool calls (<=15, good)
Partial+3.514 tool calls (<=15, good)
Time
Partial+697s (1-2min, great)
Partial+3.5134s (2-3min, good)
Cost
Partial+6$0.15 ($0.10-0.20, great)
Partial+6$0.18 ($0.10-0.20, great)
Runtime (actual)Lower is better
1m 37s
2m 14s
Cost (actual)Lower is better
$0.1521
$0.1770
Tool callsLower is better
14
14
ErrorsLower is better
1
1
Tokens usedLower is better
385,339
380,664

CLI discovery

Discovery score
40%
40%
CLI exists
Passed5/5reducto
Passed5/5llama-parse
Install docs
Passed5/5pip
Passed5/5pip
Auth method
Passed25/25env var
Passed25/25env var
Non-interactive
Failed0/25none
Failed0/25none
JSON output
Failed0/15none
Failed0/15none
llms.txt
Passed5/5
Passed5/5
Agent skill
Failed0/20
Failed0/20

Eval details

Integration task
# Task Using the reducto CLI (Reducto), parse a document: 1. Download this sample PDF: https://www.irs.gov/pub/irs-pdf/fw9.pdf 2. Use the reducto CLI to parse the PDF into struct...
# Task Using the llama-parse CLI (LlamaParse), parse a document: 1. Download this sample PDF: https://www.irs.gov/pub/irs-pdf/fw9.pdf 2. Use the llama-parse CLI to parse the PDF...
Generated files
1
1
Trace tool calls
14
14
Completed
9/27/2026
9/27/2026
Stderr
None captured
None captured