Scalekit
All 1 ran the same Auth task · Same Auth task · View the taskView the task
CompanyClick a name to swap it
#4 in Auth · #46 overall
32D
Compare up to 4. Next in Auth:
CLI task scoring
Eval score
32%
Task Completed
Failed+0The agent successfully installed and ran the official @scalekit-sdk/dryrun CLI tool with correct credentials, and verifi...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Partial+6.5Partial credit: 65%
Zero Errors
Failed+0Failed tool calls: 1
Efficiency
Partial+1.025 tool calls (<=25, below average)
Time
Partial+1.0328s (5-10min, very slow)
Cost
Partial+3.5$0.38 ($0.20-0.40, good)
Runtime (actual)Lower is better
5m 27s
Cost (actual)Lower is better
$0.3789
Tool callsLower is better
25
ErrorsLower is better
1
Tokens usedLower is better
825,747
CLI discovery
Discovery score
33%
CLI exists
Passed5/5npx @scalekit-sdk/dryrun
Install docs
Passed5/5npx
Auth method
Partial3/25oauth
Non-interactive
Failed0/25none
JSON output
Failed0/15none
llms.txt
Failed0/5
Agent skill
Passed20/20
Eval details
Integration task
# Task
Using the @scalekit-sdk/dryrun CLI (Scalekit), ***** an SSO/OIDC authentication flow:
1. Run `npx @scalekit-sdk/dryrun` with the Scalekit environment credentials (SCALEKIT...
Generated files
1
Trace tool calls
25
Completed
9/27/2026
Stderr
None captured
Full eval
#4 in Auth
32D
CLI task scoring
Eval score
32%
Task Completed
Failed+0The agent successfully installed and ran the official @scalekit-sdk/dryrun CLI tool with correct credentials, and verifi...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Partial+6.5Partial credit: 65%
Zero Errors
Failed+0Failed tool calls: 1
Efficiency
Partial+1.025 tool calls (<=25, below average)
Time
Partial+1.0328s (5-10min, very slow)
Cost
Partial+3.5$0.38 ($0.20-0.40, good)
Runtime (actual)Lower is better
5m 27s
Cost (actual)Lower is better
$0.3789
Tool callsLower is better
25
ErrorsLower is better
1
Tokens usedLower is better
825,747
CLI discovery
Discovery score
33%
CLI exists
Passed5/5npx @scalekit-sdk/dryrun
Install docs
Passed5/5npx
Auth method
Partial3/25oauth
Non-interactive
Failed0/25none
JSON output
Failed0/15none
llms.txt
Failed0/5
Agent skill
Passed20/20
Eval details
Integration task
# Task
Using the @scalekit-sdk/dryrun CLI (Scalekit), ***** an SSO/OIDC authentication flow:
1. Run `npx @scalekit-sdk/dryrun` with the Scalekit environment credentials (SCALEKIT...
Generated files
1
Trace tool calls
25
Completed
9/27/2026
Stderr
None captured
Full eval