Back to leaderboard

Agentmail

Same Email task · View

#1 in Email

92A

CLI task scoring

Eval score
89%
Task Completed
Passed+30The agent successfully completed all required tasks using the agentmail CLI correctly. Step 1: Listed inboxes (empty) th...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Zero Errors
Passed+10No errors
Efficiency
Partial+3.512 tool calls (<=15, good)
Time
Passed+1060s (<60s, excellent)
Cost
Partial+6$0.13 ($0.10-0.20, great)
Runtime (actual)Lower is better
59s
Cost (actual)Lower is better
$0.1333
Tool callsLower is better
12
ErrorsLower is better
0
Tokens usedLower is better
340,255

CLI discovery

Discovery score
100%
CLI exists
Passed5/5agentmail
Install docs
Passed5/5npm
Auth method
Passed25/25env var
Non-interactive
Passed25/25auto detect
JSON output
Passed15/15full
llms.txt
Passed5/5
Agent skill
Passed20/20

Eval details

Integration task
# Task Using the agentmail CLI (Agentmail), perform email operations: 1. Create a new email inbox (or list available inboxes) 2. Send an email from an inbox to "support@lightsage...
Generated files
1
Trace tool calls
12
Completed
10/15/2026
Stderr
None captured
Full eval