Back to leaderboard

Groq

Same Inference task · View

#2 in Inference

43D

MCP task scoring

Task score
31%
Task
Failed+0The agent correctly identified and attempted to use the Groq MCP tools (ask_with_code_execution and ask_with_realtime_in...
Used MCP Correctly
Passed+20Agent made 3 MCP tool calls
Partial Credit
Partial+1.0Partial credit: 10%
Zero Errors
Failed+0Failed tool calls: 3
Efficiency
Partial+3.512 tool calls (<=15, good)
Time
Partial+3.5163s (2-3min, good)
Cost
Partial+3.5$0.26 ($0.20-0.40, good)
Runtime (actual)Lower is better
2m 43s
Cost (actual)Lower is better
$0.2605
Tool callsLower is better
12
ErrorsLower is better
3
Tokens usedLower is better
359,911

MCP discovery

Discovery score
73%
Official MCP
Passed5/5
Auth Method
Passed15/15env var
Tool Count
Passed10/102 tools, lean
Tool Description Length
Passed10/10717 tokens, lean
llms.txt
Failed0/10
Output Schema
Failed0/5
Tool Stability
Passed5/5
Security Posture
Partial9/10No API keys, tokens, or internal URLs are exposed in descriptions or schemas; parameters are strictly typed with enums a...
Tool API Mapping
Partial7/15Both tools wrap complex workflows by bundling model selection, search filtering, and response formatting into single ope...
Description Clarity
Partial12/15Descriptions clearly explain the purpose of each tool and when to use them (real-time information vs. code execution), w...

Eval details

Integration task
# Task Using the Groq MCP, extract structured data from this invoice text: --- INVOICE #INV-2024-0042 From: Acme Corp Date: 2024-03-15 Items: - Consulting Services: $500.00 - Tr...
Generated files
1
Trace tool calls
12
Completed
10/14/2026
Stderr
None captured
Full eval