Back to leaderboard

Reducto vs LlamaParse

Same Document Parsing task · View

#4 in Document Parsing

73CBest on 4 of 22

#1 in Document Parsing

81BBest on 7 of 22

MCP task scoring

Task score
66%
81%
Task
Passed+30The agent successfully completed all required task steps using only MCP tools for document parsing operations. (1) Disco...
Passed+30The agent successfully completed all required steps using only MCP tools for document parsing operations. The agent disc...
Used MCP Correctly
Passed+20Agent made 9 MCP tool calls
Passed+20Agent made 2 MCP tool calls
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Failed+0Failed tool calls: 6
Passed+10No errors
Efficiency
Partial+2.016 tool calls (<=20, acceptable)
Partial+66 tool calls (<=10, great)
Time
Partial+1.0398s (5-10min, very slow)
Partial+2.0194s (180s–300s, slow)
Cost
Partial+3.5$0.33 ($0.20-0.40, good)
Partial+3.5$0.34 ($0.20–$0.40, good)
Runtime (actual)Lower is better
6m 37s
3m 14s
Cost (actual)Lower is better
$0.3293
$0.3423
Tool callsLower is better
16
6
ErrorsLower is better
6
0
Tokens usedLower is better
696,440
263,983

MCP discovery

Discovery score
89%
82%
Official MCP
Passed5/5
Passed5/5
Auth Method
Passed15/15env var
Passed15/15env var
Tool Count
Passed10/109 tools, lean
Partial7/1027 tools, moderate
Tool Description Length
Partial7/102,290 tokens, moderate
Partial4/107,941 tokens, heavy
llms.txt
Passed10/10
Passed10/10
Output Schema
Failed0/5
Failed0/5
Tool Stability
Passed5/5
Passed5/5
Security Posture
Partial9/10No API keys, tokens, or internal URLs exposed in descriptions or schemas; all destructive operations (edit_document) acc...
Partial9/10No API keys or credentials visible in descriptions or schemas; parameters are clearly typed and validated; minor concern...
Tool API Mapping
Partial14/15Tools map cleanly 1:1 to individual API operations (parse, extract, classify, split, edit, upload, job retrieval) with m...
Partial13/15~5-10% workflow wrapping; most tools are granular API operations - Tools map cleanly to individual API operations with c...
Description Clarity
Partial14/15Descriptions are detailed and action-oriented, clearly explaining when to use each tool, expected inputs, output types,...
Partial14/15Descriptions are comprehensive and explain purpose, parameters, constraints, and edge cases well; nearly all tools inclu...

Eval details

Integration task
# Task Using the Reducto MCP, parse a document: 1. Discover available MCP tools for document parsing 2. Download this sample PDF: https://www.irs.gov/pub/irs-pdf/fw9.pdf 3. Use t...
# Task Using the LlamaParse MCP, parse a document: 1. Discover available MCP tools for document parsing 2. Download this sample PDF: https://www.irs.gov/pub/irs-pdf/fw9.pdf 3. Us...
Generated files
2
2
Trace tool calls
16
6
Completed
9/27/2026
11/11/2026
Stderr
None captured
None captured