Back to leaderboard

Twilio

Same Voice Telephony task · View

#2 in Voice Telephony

31D

CLI task scoring

Eval score
29%
Task Completed
Failed+0The agent correctly installed and used the official Twilio CLI (twilio-cli via npm, which provides the `twilio` binary)....
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Partial+3Partial credit: 30%
Zero Errors
Failed+0Failed tool calls: 14
Efficiency
Partial+1.025 tool calls (<=25, below average)
Time
Partial+2.0281s (3-5min, slow)
Cost
Partial+3.5$0.37 ($0.20-0.40, good)
Runtime (actual)Lower is better
4m 40s
Cost (actual)Lower is better
$0.3655
Tool callsLower is better
25
ErrorsLower is better
14
Tokens usedLower is better
811,111

CLI discovery

Discovery score
35%
CLI exists
Passed5/5twilio
Install docs
Passed5/5homebrew
Auth method
Passed25/25env var
Non-interactive
Failed0/25none
JSON output
Failed0/15none
llms.txt
Failed0/5
Agent skill
Failed0/20

Eval details

Integration task
# Task Using the twilio CLI (Twilio), perform a telephony workflow. If the CLI supports messaging, send a ***** SMS using sandbox or ***** credentials and print the response. If...
Generated files
1
Trace tool calls
25
Completed
9/27/2026
Stderr
None captured
Full eval