Datadog
All 1 ran the same Observability task · Same Observability task · View the taskView the task
CompanyClick a name to swap it
#1 in Observability · #33 overall
63C
Compare up to 4.
CLI task scoring
Eval score
68%
Task Completed
Passed+30The agent successfully completed all three required tasks using the official pup CLI. (1) The metric 'test.leaderboard.c...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Zero Errors
Failed+0Failed tool calls: 3
Efficiency
Partial+1.021 tool calls (<=25, below average)
Time
Partial+3.5156s (2-3min, good)
Cost
Partial+3.5$0.25 ($0.20-0.40, good)
Runtime (actual)Lower is better
2m 35s
Cost (actual)Lower is better
$0.2530
Tool callsLower is better
21
ErrorsLower is better
3
Tokens usedLower is better
693,865
CLI discovery
Discovery score
53%
CLI exists
Passed5/5pup
Install docs
Passed5/5brew
Auth method
Partial3/25oauth
Non-interactive
Failed0/25none
JSON output
Passed15/15full
llms.txt
Passed5/5
Agent skill
Passed20/20
Eval details
Integration task
# Task
Using the pup CLI (Datadog), perform observability operations:
1. Submit a custom metric called "*****.leaderboard.cli_score" with value 42 and tags "env:*****,leaderboard...
Generated files
2
Trace tool calls
21
Completed
9/27/2026
Stderr
None captured
Full eval
#1 in Observability
63C
CLI task scoring
Eval score
68%
Task Completed
Passed+30The agent successfully completed all three required tasks using the official pup CLI. (1) The metric 'test.leaderboard.c...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Zero Errors
Failed+0Failed tool calls: 3
Efficiency
Partial+1.021 tool calls (<=25, below average)
Time
Partial+3.5156s (2-3min, good)
Cost
Partial+3.5$0.25 ($0.20-0.40, good)
Runtime (actual)Lower is better
2m 35s
Cost (actual)Lower is better
$0.2530
Tool callsLower is better
21
ErrorsLower is better
3
Tokens usedLower is better
693,865
CLI discovery
Discovery score
53%
CLI exists
Passed5/5pup
Install docs
Passed5/5brew
Auth method
Partial3/25oauth
Non-interactive
Failed0/25none
JSON output
Passed15/15full
llms.txt
Passed5/5
Agent skill
Passed20/20
Eval details
Integration task
# Task
Using the pup CLI (Datadog), perform observability operations:
1. Submit a custom metric called "*****.leaderboard.cli_score" with value 42 and tags "env:*****,leaderboard...
Generated files
2
Trace tool calls
21
Completed
9/27/2026
Stderr
None captured
Full eval