Back to leaderboard

Datadog

Same Observability task · View

#1 in Observability

63C

CLI task scoring

Eval score
68%
Task Completed
Passed+30The agent successfully completed all three required tasks using the official pup CLI. (1) The metric 'test.leaderboard.c...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Zero Errors
Failed+0Failed tool calls: 3
Efficiency
Partial+1.021 tool calls (<=25, below average)
Time
Partial+3.5156s (2-3min, good)
Cost
Partial+3.5$0.25 ($0.20-0.40, good)
Runtime (actual)Lower is better
2m 35s
Cost (actual)Lower is better
$0.2530
Tool callsLower is better
21
ErrorsLower is better
3
Tokens usedLower is better
693,865

CLI discovery

Discovery score
53%
CLI exists
Passed5/5pup
Install docs
Passed5/5brew
Auth method
Partial3/25oauth
Non-interactive
Failed0/25none
JSON output
Passed15/15full
llms.txt
Passed5/5
Agent skill
Passed20/20

Eval details

Integration task
# Task Using the pup CLI (Datadog), perform observability operations: 1. Submit a custom metric called "*****.leaderboard.cli_score" with value 42 and tags "env:*****,leaderboard...
Generated files
2
Trace tool calls
21
Completed
9/27/2026
Stderr
None captured
Full eval