Reducto GitHub Copilot API Evaluation

Reducto

reducto.aiDocument Parsing

GitHub Copilot

87/ 100

Score Breakdown87%

Lost points on 3 checks

cost6/10
time9/15
efficiency2/10

Eval Score (70% weight)82%

cost$0.13 ($0.10–$0.20, great)+6

time81s (60s–120s, great)+9

efficiency19 tool calls (≤20, acceptable)+2

found docsTouched docs at docs.reducto.ai: yes (9 matching tool calls)+10

zero errorsNo errors in tool outputs+5

syntax validCode compiles/parses correctly+5

created filesGenerated files: 1 (need ≥1)+10

full executionMethod: full_execution, Exit: 0+35

Checklist Score (30% weight)100%

Context7

Link+14

llms.txt

Link+14

MCP Server

Link+14

Typed SDK

Link+14

OpenAPI

Link+14

Agent Skills

Link+14

CLI

Link+14

Checklist determined by AI. If something looks wrong, .

Final Score

A: >=90%B: 75-89%C: 55-74%D: <55%

(82 × 0.7) + (100 × 0.3)= 87%

Replay

Using https://docs.reducto.ai/quickstart, complete the following task:

Using Reducto, build me a script that parses this PDF and extracts structured data: https://www.irs.gov/pub/irs-pdf/fw9.pdf

Print the result as JSON with fields: pages (total count), chunks (array of text blocks), tables (array if any found)

parse_w9.py

Unlock analytics

See Reducto's score trend, layer rankings, and competitive position.

Run completed Oct 26, 2026, 6:39 PM