Nylas vs Agentmail
Both ran the same Email task · Same Email task · View the taskView the task
CompanyClick a name to swap it
#4 in Email · #103 overall
69CBest on 0 of 14
#1 in Email · #7 overall
86BBest on 6 of 14
Compare up to 4. Next in Email:
Agent performance
Eval score
62%
87%
Task success
Passed
Passed
Cost per runLower is better
$0.7979
$0.1296
Tool callsLower is better
34
6
ErrorsLower is better
9
1
RuntimeLower is better
6m 50s
43s
Tokens usedLower is better
1,087,733
163,781
Discoverability
Discovery score
86%
86%
Context7
Not found
Not found
Eval details
Integration task
# Task
Using https://developer.nylas.com/docs/v3/email/, complete the following task:
Using Nylas, build me a script that:
1. Create a new email inbox
2. Send an email from that...
# Task
Using https://docs.agentmail.to, complete the following task:
Using Agentmail, build me a script that:
1. Create a new email inbox
2. Send an email from that inbox to "su...
Generated files
1
1
Trace tool calls
34
6
Completed
9/30/2026
10/15/2026
Stderr
None captured
None captured
#4 in Email
69CBest on 0 of 14
#1 in Email
86BBest on 6 of 14
Agent performance
Eval score
62%
87%
Task success
Passed
Passed
Cost per runLower is better
$0.7979
$0.1296
Tool callsLower is better
34
6
ErrorsLower is better
9
1
RuntimeLower is better
6m 50s
43s
Tokens usedLower is better
1,087,733
163,781
Discoverability
Discovery score
86%
86%
Context7
Not found
Not found
Eval details
Integration task
# Task
Using https://developer.nylas.com/docs/v3/email/, complete the following task:
Using Nylas, build me a script that:
1. Create a new email inbox
2. Send an email from that...
# Task
Using https://docs.agentmail.to, complete the following task:
Using Agentmail, build me a script that:
1. Create a new email inbox
2. Send an email from that inbox to "su...
Generated files
1
1
Trace tool calls
34
6
Completed
9/30/2026
10/15/2026
Stderr
None captured
None captured
Full eval