Back to leaderboard

Groq

Same Inference task · View

#1 in Inference

38D

CLI task scoring

Eval score
31%
Task Completed
Failed+0The agent correctly installed and used the official `groq` CLI (groq-code-cli via npm), and attempted to invoke it for i...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Partial+1.5Partial credit: 15%
Zero Errors
Passed+10No errors
Efficiency
Failed+060 tool calls (>25, inefficient)
Time
Failed+0823s (>=10min, too slow)
Cost
Failed+0$1.07 (>=$1.00, too expensive)
Runtime (actual)Lower is better
13m 42s
Cost (actual)Lower is better
$1.07
Tool callsLower is better
60
ErrorsLower is better
0
Tokens usedLower is better
3,010,847

CLI discovery

Discovery score
55%
CLI exists
Passed5/5groq
Install docs
Passed5/5npm
Auth method
Passed25/25env var
Non-interactive
Failed0/25none
JSON output
Failed0/15none
llms.txt
Failed0/5
Agent skill
Passed20/20

Eval details

Integration task
# Task Using the groq CLI (Groq), extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Services: $500....
Generated files
4
Trace tool calls
60
Completed
10/13/2026
Stderr
None captured
Full eval