Deepgram vs AssemblyAI
Both ran the same Voice STT task · Same Voice STT task · View the taskView the task
CompanyClick a name to swap it
#1 in Voice STT · tied #1 overall
92ABest on 7 of 19
#2 in Voice STT · #17 overall
75BBest on 1 of 19
Compare up to 4.
CLI task scoring
Eval score
92%
78%
Task Completed
Passed+30The agent successfully completed all task requirements: (1) installed the official `dg` CLI via `pip install deepctl`, (...
Passed+30The agent successfully completed all required steps: (1) installed the official AssemblyAI CLI from the correct source,...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Passed+10No errors
Failed+0Failed tool calls: 2
Efficiency
Partial+67 tool calls (<=10, great)
Partial+68 tool calls (<=10, great)
Time
Partial+6109s (1-2min, great)
Partial+687s (1-2min, great)
Cost
Passed+10$0.10 (<$0.10, excellent)
Partial+6$0.13 ($0.10-0.20, great)
Runtime (actual)Lower is better
1m 49s
1m 27s
Cost (actual)Lower is better
$0.0959
$0.1278
Tool callsLower is better
7
8
ErrorsLower is better
0
2
Tokens usedLower is better
181,457
212,720
CLI discovery
Discovery score
95%
70%
CLI exists
Passed5/5dg
Passed5/5assemblyai
Install docs
Passed5/5pip
Passed5/5curl script
Auth method
Passed25/25env var
Passed25/25api key cmd
Non-interactive
Passed25/25auto detect
Failed0/25none
JSON output
Passed15/15full
Passed15/15full
llms.txt
Failed0/5
Failed0/5
Agent skill
Passed20/20
Passed20/20
Eval details
Integration task
# Task
Using the dg CLI (Deepgram), transcribe a sample audio file:
1. Download this sample audio file:
https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...
# Task
Using the assemblyai CLI (AssemblyAI), transcribe a sample audio file:
1. Download this sample audio file:
https://storage.googleapis.com/aai-web-samples/5_common_sport...
Generated files
1
1
Trace tool calls
7
8
Completed
9/27/2026
9/27/2026
Stderr
None captured
None captured
#1 in Voice STT
92ABest on 7 of 19
#2 in Voice STT
75BBest on 1 of 19
CLI task scoring
Eval score
92%
78%
Task Completed
Passed+30The agent successfully completed all task requirements: (1) installed the official `dg` CLI via `pip install deepctl`, (...
Passed+30The agent successfully completed all required steps: (1) installed the official AssemblyAI CLI from the correct source,...
Used CLI Correctly
Passed+20Agent used CLI commands correctly
Passed+20Agent used CLI commands correctly
Partial Credit
Passed+10Partial credit: 100%
Passed+10Partial credit: 100%
Zero Errors
Passed+10No errors
Failed+0Failed tool calls: 2
Efficiency
Partial+67 tool calls (<=10, great)
Partial+68 tool calls (<=10, great)
Time
Partial+6109s (1-2min, great)
Partial+687s (1-2min, great)
Cost
Passed+10$0.10 (<$0.10, excellent)
Partial+6$0.13 ($0.10-0.20, great)
Runtime (actual)Lower is better
1m 49s
1m 27s
Cost (actual)Lower is better
$0.0959
$0.1278
Tool callsLower is better
7
8
ErrorsLower is better
0
2
Tokens usedLower is better
181,457
212,720
CLI discovery
Discovery score
95%
70%
CLI exists
Passed5/5dg
Passed5/5assemblyai
Install docs
Passed5/5pip
Passed5/5curl script
Auth method
Passed25/25env var
Passed25/25api key cmd
Non-interactive
Passed25/25auto detect
Failed0/25none
JSON output
Passed15/15full
Passed15/15full
llms.txt
Failed0/5
Failed0/5
Agent skill
Passed20/20
Passed20/20
Eval details
Integration task
# Task
Using the dg CLI (Deepgram), transcribe a sample audio file:
1. Download this sample audio file:
https://storage.googleapis.com/aai-web-samples/5_common_sports_injuries...
# Task
Using the assemblyai CLI (AssemblyAI), transcribe a sample audio file:
1. Download this sample audio file:
https://storage.googleapis.com/aai-web-samples/5_common_sport...
Generated files
1
1
Trace tool calls
7
8
Completed
9/27/2026
9/27/2026
Stderr
None captured
None captured
Full eval