Back to leaderboard

Scalekit

Same Auth task · View

#3 in Auth

35D

MCP task scoring

Task score
35%
Task
Failed+0The agent attempted to use MCP tools (ToolSearch) but encountered a fundamental blocker: the Scalekit MCP server require...
Used MCP Correctly
Failed+0MCP usage issues: No MCP tool calls were actually made (0 mcp__* calls vs. requirement to use Scalekit MCP tools), Users...
Partial Credit
Partial+1.5Partial credit: 15%
Zero Errors
Passed+10No errors
Efficiency
Passed+103 tool calls (<=5, excellent)
Time
Partial+3.5133s (2-3min, good)
Cost
Passed+10$0.08 (<$0.10, excellent)
Runtime (actual)Lower is better
2m 12s
Cost (actual)Lower is better
$0.0786
Tool callsLower is better
3
ErrorsLower is better
0
Tokens usedLower is better
99,971

MCP discovery

Discovery score
38%
Official MCP
Passed5/5
Auth Method
Partial8/15oauth rs dcr authcode
Tool Count
Failed0/100 tools, bloated
Tool Description Length
Passed10/100 tokens, lean
llms.txt
Passed10/10
Output Schema
Failed0/5
Tool Stability
Passed5/5
Security Posture
—
Tool API Mapping
—
Description Clarity
—

Eval details

Integration task
# Task Using the Scalekit MCP, perform user and organization management tasks: 1. Discover available MCP tools and understand what each one does 2. List all existing users (or se...
Generated files
1
Trace tool calls
3
Completed
9/27/2026
Stderr
None captured
Full eval