Scalekit vs Clerk
Both ran the same Auth task · Same Auth task · View the taskView the task
CompanyClick a name to swap it
#3 in Auth · #54 overall
35DBest on 8 of 22
#2 in Auth · #19 overall
81BBest on 5 of 22
Compare up to 4. Next in Auth:
MCP task scoring
Task score
35%
81%
Task
Failed+0The agent attempted to use MCP tools (ToolSearch) but encountered a fundamental blocker: the Scalekit MCP server require...
Passed+30The agent successfully completed the core task requirements using only MCP tools (4 MCP calls, no curl/SDK fallbacks). I...
Used MCP Correctly
Failed+0MCP usage issues: No MCP tool calls were actually made (0 mcp__* calls vs. requirement to use Scalekit MCP tools), Users...
Passed+20Agent made 4 MCP tool calls
Partial Credit
Partial+1.5Partial credit: 15%
Partial+9.5Partial credit: 95%
Zero Errors
Passed+10No errors
Passed+10No errors
Efficiency
Passed+103 tool calls (<=5, excellent)
Partial+3.511 tool calls (<=15, good)
Time
Partial+3.5133s (2-3min, good)
Partial+2.0202s (3-5min, slow)
Cost
Passed+10$0.08 (<$0.10, excellent)
Partial+6$0.16 ($0.10-0.20, great)
Runtime (actual)Lower is better
2m 12s
3m 22s
Cost (actual)Lower is better
$0.0786
$0.1577
Tool callsLower is better
3
11
ErrorsLower is better
0
0
Tokens usedLower is better
99,971
174,942
MCP discovery
Discovery score
38%
83%
Official MCP
Passed5/5
Passed5/5
Auth Method
Partial8/15oauth rs dcr authcode
Passed15/15env var
Tool Count
Failed0/100 tools, bloated
Passed10/1014 tools, lean
Tool Description Length
Passed10/100 tokens, lean
Partial4/105,871 tokens, heavy
llms.txt
Passed10/10
Passed10/10
Output Schema
Failed0/5
Failed0/5
Tool Stability
Passed5/5
Passed5/5
Security Posture
—
Partial8/10Tool definitions correctly hide API keys behind named schema properties (bearerAuth) rather than embedding them in descr...
Tool API Mapping
—
Partial14/15Tools map cleanly 1:1 to individual API operations (GetUser, CreateUser, DeleteUser, etc.); each tool performs a single,...
Description Clarity
—
Partial12/15Most tools have clear, purpose-driven descriptions explaining what they do; however, some lack detail about constraints...
Eval details
Integration task
# Task
Using the Scalekit MCP, perform user and organization management tasks:
1. Discover available MCP tools and understand what each one does
2. List all existing users (or se...
# Task
Using the Clerk MCP, perform user and organization management tasks:
1. Discover available MCP tools and understand what each one does
2. List all existing users (or searc...
Generated files
1
1
Trace tool calls
3
11
Completed
9/27/2026
9/27/2026
Stderr
None captured
None captured
#3 in Auth
35DBest on 8 of 22
#2 in Auth
81BBest on 5 of 22
MCP task scoring
Task score
35%
81%
Task
Failed+0The agent attempted to use MCP tools (ToolSearch) but encountered a fundamental blocker: the Scalekit MCP server require...
Passed+30The agent successfully completed the core task requirements using only MCP tools (4 MCP calls, no curl/SDK fallbacks). I...
Used MCP Correctly
Failed+0MCP usage issues: No MCP tool calls were actually made (0 mcp__* calls vs. requirement to use Scalekit MCP tools), Users...
Passed+20Agent made 4 MCP tool calls
Partial Credit
Partial+1.5Partial credit: 15%
Partial+9.5Partial credit: 95%
Zero Errors
Passed+10No errors
Passed+10No errors
Efficiency
Passed+103 tool calls (<=5, excellent)
Partial+3.511 tool calls (<=15, good)
Time
Partial+3.5133s (2-3min, good)
Partial+2.0202s (3-5min, slow)
Cost
Passed+10$0.08 (<$0.10, excellent)
Partial+6$0.16 ($0.10-0.20, great)
Runtime (actual)Lower is better
2m 12s
3m 22s
Cost (actual)Lower is better
$0.0786
$0.1577
Tool callsLower is better
3
11
ErrorsLower is better
0
0
Tokens usedLower is better
99,971
174,942
MCP discovery
Discovery score
38%
83%
Official MCP
Passed5/5
Passed5/5
Auth Method
Partial8/15oauth rs dcr authcode
Passed15/15env var
Tool Count
Failed0/100 tools, bloated
Passed10/1014 tools, lean
Tool Description Length
Passed10/100 tokens, lean
Partial4/105,871 tokens, heavy
llms.txt
Passed10/10
Passed10/10
Output Schema
Failed0/5
Failed0/5
Tool Stability
Passed5/5
Passed5/5
Security Posture
—
Partial8/10Tool definitions correctly hide API keys behind named schema properties (bearerAuth) rather than embedding them in descr...
Tool API Mapping
—
Partial14/15Tools map cleanly 1:1 to individual API operations (GetUser, CreateUser, DeleteUser, etc.); each tool performs a single,...
Description Clarity
—
Partial12/15Most tools have clear, purpose-driven descriptions explaining what they do; however, some lack detail about constraints...
Eval details
Integration task
# Task
Using the Scalekit MCP, perform user and organization management tasks:
1. Discover available MCP tools and understand what each one does
2. List all existing users (or se...
# Task
Using the Clerk MCP, perform user and organization management tasks:
1. Discover available MCP tools and understand what each one does
2. List all existing users (or searc...
Generated files
1
1
Trace tool calls
3
11
Completed
9/27/2026
9/27/2026
Stderr
None captured
None captured
Full eval