Microsoft MarkItDown
85/ 100
BScore Breakdown85%
Lost points on 4 checks
- Time6 pts
- Cost6 pts
- Auth Method0 pts
- llms.txt0 pts
Task Score (70% weight)92%
TaskThe agent successfully completed all required steps: (1) discovered the available MCP tool (convert_to_markdown), (2) used the MCP tool to parse the PDF from the provided HTTPS URL without downloading separately, (3) extracted structured markdown/text output from the PDF (38,061 characters across 6 pages), and (4) printed and saved the parsed content showing form fields, sections, instructions, and extracted text. The agent used ONLY the MCP tool for document parsing operations and did not use curl, SDK calls, or raw API calls for the actual parsing. The pip install in step [1] was setup/configuration work, not a workaround. The agent correctly adapted when the PDF's tabular content was flattened to text rather than preserved as markdown tables, which is a limitation of the tool output, not an agent error.+30
Used MCP CorrectlyAgent made 1 MCP tool calls+20
Partial CreditPartial credit: 100%+10
Zero ErrorsNo errors+10
Efficiency3 tool calls (<=5, excellent)+10
Time90s (60s–120s, great)+6
Cost$0.16 ($0.10–$0.20, great)+6
Discovery Score (30% weight)70%
Official MCP+5
Auth Method+0
Tool Count+10
Tool Description Length+10
llms.txt+0
Output Schema+5
Tool Stability+5
Security PostureNo API keys, tokens, or credentials are exposed in the tool definition; the URI parameter is typed as a string with no overly permissive behavior, though the tool could potentially be abused to access sensitive local files via file: URIs depending on runtime restrictions.+9
Tool API MappingThe tool maps 1:1 to a single conversion operation with a clean input (URI) and output (markdown result), allowing agents to compose it freely with other tools without hidden multi-step workflows.+14
Description ClarityThe description clearly explains what the tool does (converts resources to markdown) and specifies supported URI schemes (http, https, file, data), though it lacks detail about expected output format, error handling, or use cases.+12
Discovery score comes from MCP metadata, live tools/list inspection, and judging of tool quality. If something looks wrong, .
Final Score
A: >=90%B: 75-89%C: 55-74%D: <55%
(92 × 0.7) + (70 × 0.3)= 85%
Replay
Using the Microsoft MarkItDown MCP, parse a document:
1. Discover available MCP tools for document parsing
2. Download this sample PDF: https://www.irs.gov/pub/irs-pdf/fw9.pdf
3. Use the MCP tools to parse the PDF into structured output (markdown, JSON, or text)
4. Print the parsed output showing extracted text content, tables, or structure
Use ONLY the MCP tools for document parsing operations. Do NOT use SDK or raw API calls.
You MUST use the Microsoft MarkItDown MCP server to accomplish this task.
Use the MCP tools for the actual product operations. Do NOT use curl, fetch, raw HTTP requests, CLI commands, or SDK calls as substitutes.
Use the MCP tools' help/descriptions to understand what each tool does.
If you need additional context, check the documentation: https://github.com/microsoft/markitdown/tree/main/packages/markitdown-mcp
## Setup
Do NOT assume the Microsoft MarkItDown MCP server is already installed or already running in this sandbox.
Review the installation and authentication details below before starting the task.
### Installation
Transport: `stdio`
Codex is configured to launch the `microsoft-markitdown` MCP server over stdio using: `/tmp/dta-markitdown-venv/bin/python -m markitdown_mcp`
Stored MCP package hint: `markitdown-mcp==0.0.1a8`
Run this setup command in the sandbox before the agent launches the server: `python3 -m venv /tmp/dta-markitdown-venv && /tmp/dta-markitdown-venv/bin/python -m pip install markitdown-mcp==0.0.1a8`
Bootstrap notes: Official Microsoft MCP verified from https://github.com/microsoft/markitdown/tree/main/packages/markitdown-mcp 2026-11-11. Published PyPI version0.0.1a8 pinned. MIT licensed free local parsing, no account/API key; only remote sandbox public fixture is accessible. MARKITDOWN_ENABLE_PLUGINS=false disables optional plugins and is not a credential. Harness setup correction2026-11-11: use explicit isolated venv interpreter for identical resolution under daytona setup and root Codex. Unqualified python3 differed across users causing initialize failure. No server code/tool change. Sandbox appdirectory is root-owned; create isolated venv under /tmp so setup daytona can write and agentroot can execute.
### Authentication
No authentication step is required for this MCP server.
## Summary
When you are done, write a JSON summary file to results.json with this structure:
{
"operations": [
{"step": 1, "description": "What you did", "mcp_tool": "tool_name", "success": true/false, "output_summary": "brief result"},
...
],
"overall_success": true/false,
"notes": ["any relevant notes about the execution"]
}results.json
Unlock analytics
See Microsoft MarkItDown's score trend, layer rankings, and competitive position.
Run completed Nov 11, 2026, 9:04 AM