Resend vs Agentmail
Both ran the same Email task · Same Email task · View the taskView the task
CompanyClick a name to swap it
#2 in Email · #19 overall
81BBest on 3 of 22
#1 in Email · #13 overall
84BBest on 4 of 22
Compare up to 4. Next in Email:
MCP task scoring
Task score
87%
88%
Task
Passed+30The agent successfully completed all five required steps using only Resend MCP tools. Step 1: Tool discovery via ToolSea...
Passed+30The agent successfully completed all required tasks using only Agentmail MCP tools. It discovered available tools, liste...
Used MCP Correctly
Passed+20Agent made 4 MCP tool calls
Passed+20Agent made 5 MCP tool calls
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Passed+10No errors
Passed+10No errors
Efficiency
Partial+66 tool calls (<=10, great)
Partial+67 tool calls (<=10, great)
Time
Partial+1.0440s (5-10min, very slow)
Partial+2.0280s (3-5min, slow)
Cost
Passed+10$0.09 (<$0.10, excellent)
Passed+10$0.10 (<$0.10, excellent)
Runtime (actual)Lower is better
7m 19s
4m 39s
Cost (actual)Lower is better
$0.0893
$0.0965
Tool callsLower is better
6
7
ErrorsLower is better
0
0
Tokens usedLower is better
159,715
180,827
MCP discovery
Discovery score
70%
75%
Official MCP
Passed5/5
Passed5/5
Auth Method
Passed15/15env var
Passed15/15env var
Tool Count
Failed0/10106 tools, bloated
Partial3/1041 tools, heavy
Tool Description Length
Failed0/1029,709 tokens, bloated
Failed0/1013,234 tokens, bloated
llms.txt
Passed10/10
Passed10/10
Output Schema
Failed0/5
Failed0/5
Tool Stability
Passed5/5
Passed5/5
Security Posture
Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce...
Partial9/10No API keys or tokens are exposed in tool descriptions or schemas; parameters are clearly typed and validated; however,...
Tool API Mapping
Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...
Partial14/15Tools map cleanly 1:1 to individual API operations (create_inbox, get_message, list_threads, etc.), with rare exceptions...
Description Clarity
Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi...
Partial14/15Descriptions are comprehensive and explain tool purpose, parameters, expected outcomes, and edge cases clearly; agents c...
Eval details
Integration task
# Task
Using the Resend MCP, perform email operations:
1. Discover available MCP tools and their capabilities
2. Create a new email inbox (or list available inboxes/domains)
3. S...
# Task
Using the Agentmail MCP, perform email operations:
1. Discover available MCP tools and their capabilities
2. Create a new email inbox (or list available inboxes/domains)
3...
Generated files
1
1
Trace tool calls
6
7
Completed
9/27/2026
10/14/2026
Stderr
None captured
None captured
#2 in Email
81BBest on 3 of 22
#1 in Email
84BBest on 4 of 22
MCP task scoring
Task score
87%
88%
Task
Passed+30The agent successfully completed all five required steps using only Resend MCP tools. Step 1: Tool discovery via ToolSea...
Passed+30The agent successfully completed all required tasks using only Agentmail MCP tools. It discovered available tools, liste...
Used MCP Correctly
Passed+20Agent made 4 MCP tool calls
Passed+20Agent made 5 MCP tool calls
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Passed+10No errors
Passed+10No errors
Efficiency
Partial+66 tool calls (<=10, great)
Partial+67 tool calls (<=10, great)
Time
Partial+1.0440s (5-10min, very slow)
Partial+2.0280s (3-5min, slow)
Cost
Passed+10$0.09 (<$0.10, excellent)
Passed+10$0.10 (<$0.10, excellent)
Runtime (actual)Lower is better
7m 19s
4m 39s
Cost (actual)Lower is better
$0.0893
$0.0965
Tool callsLower is better
6
7
ErrorsLower is better
0
0
Tokens usedLower is better
159,715
180,827
MCP discovery
Discovery score
70%
75%
Official MCP
Passed5/5
Passed5/5
Auth Method
Passed15/15env var
Passed15/15env var
Tool Count
Failed0/10106 tools, bloated
Partial3/1041 tools, heavy
Tool Description Length
Failed0/1029,709 tokens, bloated
Failed0/1013,234 tokens, bloated
llms.txt
Passed10/10
Passed10/10
Output Schema
Failed0/5
Failed0/5
Tool Stability
Passed5/5
Passed5/5
Security Posture
Partial9/10No API keys or tokens visible in tool descriptions or schemas; parameters are clearly typed with validation; minor conce...
Partial9/10No API keys or tokens are exposed in tool descriptions or schemas; parameters are clearly typed and validated; however,...
Tool API Mapping
Partial12/15Most tools map 1:1 to individual API operations (e.g., create_contact, get_broadcast, add_suppression), but create-autom...
Partial14/15Tools map cleanly 1:1 to individual API operations (create_inbox, get_message, list_threads, etc.), with rare exceptions...
Description Clarity
Partial14/15Tool descriptions are comprehensive and clearly explain purpose, when to use, expected inputs/outputs, and workflows; mi...
Partial14/15Descriptions are comprehensive and explain tool purpose, parameters, expected outcomes, and edge cases clearly; agents c...
Eval details
Integration task
# Task
Using the Resend MCP, perform email operations:
1. Discover available MCP tools and their capabilities
2. Create a new email inbox (or list available inboxes/domains)
3. S...
# Task
Using the Agentmail MCP, perform email operations:
1. Discover available MCP tools and their capabilities
2. Create a new email inbox (or list available inboxes/domains)
3...
Generated files
1
1
Trace tool calls
6
7
Completed
9/27/2026
10/14/2026
Stderr
None captured
None captured
Full eval