Extend.ai GitHub Copilot API Evaluation

Extend.ai

extend.aiDocument Parsing

GitHub Copilot

79/ 100

Score Breakdown79%

Lost points on 4 checks

cost6/10
time5/15
efficiency1/10
Context70/14

Eval Score (70% weight)77%

cost$0.15 ($0.10–$0.20, great)+6

time125s (120s–180s, good)+5

efficiency23 tool calls (≤25, below average)+1

found docsTouched docs at docs.extend.ai: yes (13 matching tool calls)+10

zero errorsNo errors in tool outputs+5

syntax validCode compiles/parses correctly+5

created filesGenerated files: 1 (need ≥1)+10

full executionMethod: full_execution, Exit: 0+35

Checklist Score (30% weight)86%

Context7

llms.txt

Link+14

MCP Server

Link+14

Typed SDK

Link+14

OpenAPI

Link+14

Agent Skills

Link+14

CLI

Link+14

Checklist determined by AI. If something looks wrong, .

Final Score

A: >=90%B: 75-89%C: 55-74%D: <55%

(77 × 0.7) + (86 × 0.3)= 79%

Replay

Using https://docs.extend.ai/, complete the following task:

Using Extend.ai, build me a script that parses this PDF and extracts structured data: https://www.irs.gov/pub/irs-pdf/fw9.pdf

Print the result as JSON with fields: pages (total count), chunks (array of text blocks), tables (array if any found)

parse_w9.py

Unlock analytics

See Extend.ai's score trend, layer rankings, and competitive position.

Run completed Oct 26, 2026, 6:24 PM