Back to leaderboard

Meeting BaaS

Same Meeting Bot task · View

#2 in Meeting Bot

38D

MCP task scoring

Task score
23%
Task
Failed+0The agent correctly identified and fixed a critical MCP server startup failure (ts-node/Node 25 incompatibility) and suc...
Used MCP Correctly
Passed+20Agent made 1 MCP tool calls
Partial Credit
Partial+3.5Partial credit: 35%
Zero Errors
Failed+0Failed tool calls: 7
Efficiency
Failed+061 tool calls (>25, inefficient)
Time
Failed+01043s (>=10min, too slow)
Cost
Failed+0$1.41 (>=$1.00, too expensive)
Runtime (actual)Lower is better
17m 23s
Cost (actual)Lower is better
$1.41
Tool callsLower is better
61
ErrorsLower is better
7
Tokens usedLower is better
3,613,169

MCP discovery

Discovery score
72%
Official MCP
Passed5/5
Auth Method
Passed15/15env var
Tool Count
Partial7/1029 tools, moderate
Tool Description Length
Partial4/105,899 tokens, heavy
llms.txt
Passed10/10
Output Schema
Failed0/5
Tool Stability
Passed5/5
Security Posture
Partial4/10Critical security issues: API keys and OAuth credentials (clientSecret, refreshToken) are accepted as tool parameters an...
Tool API Mapping
Partial10/15Tools generally map 1:1 to API endpoints, but several tools bundle multiple concerns: `joinMeeting` combines meeting par...
Description Clarity
Partial12/15Most tools have clear, descriptive explanations of purpose and parameters with good use of enums and format specificatio...

Eval details

Integration task
# Task Using the Meeting BaaS MCP, manage meeting recordings: 1. Discover available MCP tools and their capabilities 2. List existing meeting recordings or bots 3. Retrieve detai...
Generated files
4
Trace tool calls
61
Completed
9/27/2026
Stderr
None captured
Full eval