# CopilotKit Codex API Evaluation

**Category:** Agent UI
**Score:** 68 (C)

This is the agent-readable Devtool Arena evaluation page for CopilotKit on Codex API.

## Summary

| Metric | Value |
|--------|-------|
| Status | completed |
| Overall score | 68 |
| Grade | C |
| Eval score | 67 |
| Discovery score | 71 |
| Cost | $0.42 |
| Runtime | 2m 25s |
| Tool calls | 31 |
| Errors | 8 |
| Tokens used | 1963737 |
| Success | Yes |

## Agent-Readiness Checklist

| Signal | Evidence |
|--------|----------|
| Context7 | https://context7.com/copilotkit/copilotkit |
| llms.txt | https://docs.copilotkit.ai/llms.txt |
| MCP server | Not found |
| Typed SDK | https://github.com/CopilotKit/CopilotKit/tree/main/packages/runtime |
| OpenAPI | Not found |
| Agent skills | https://github.com/CopilotKit/CopilotKit/blob/main/CLAUDE.md |
| CLI | https://github.com/CopilotKit/CopilotKit |

## Evaluation Prompt

````text

# Task
Using https://docs.copilotkit.ai, complete the following task:

Using CopilotKit, build me a script that stands up an agent backend exposing two tools to a frontend, then exercises them over CopilotKit's own protocol:

1. Define TWO related tools/actions that a support assistant would call:
   a. `get_ticket_status(ticket_id)` — returns a dict with ticket_id, status, and priority. For ticket "T-1042" return status "open" and priority "high".
   b. `escalate_ticket(ticket_id, reason)` — returns a dict with ticket_id, escalated (boolean), and reason.
2. Serve both tools through CopilotKit's Python backend integration on localhost, and start the server in the background from the same script (a thread or subprocess is fine). Wait until it is accepting requests.
3. Query CopilotKit's own discovery/info endpoint to list the tools the backend advertises, and collect their names from that response (do NOT hardcode the list you defined in step 1).
4. Invoke the two tools through CopilotKit's protocol, where the SECOND call depends on the first: call `get_ticket_status` for "T-1042", then take the ticket_id from that response and pass it to `escalate_ticket` with reason "high priority ticket left open".
5. Shut the server down and print the output as JSON with these keys:
   - "discovered_tools": the sorted list of tool names read from the discovery endpoint in step 3
   - "ticket": the result payload returned by the `get_ticket_status` call
   - "escalation": the result payload returned by the `escalate_ticket` call

The two tool invocations in step 4 MUST go through CopilotKit's serving layer over HTTP (the endpoint CopilotKit mounts), not by calling your Python handler functions directly in-process. You MUST use the official CopilotKit Python SDK to define and serve the tools — do not hand-roll the protocol. 

## Execution
After creating the script, run it to verify it works:
```bash
cd /home/daytona/app && python <your_script>.py
```

The script should print output to stdout. If there are errors, debug and fix them until it runs successfully.

````

## Grader Results

| Check | Passed | Weight | Score | Details |
|-------|--------|--------|-------|---------|
| Cost | No | 0 | 0 | $0.42 ($0.40–0.70, expensive) |
| Time | Yes | 0 | 0 | 145s (2–3min, good) |
| Efficiency | No | 0 | 0 | 31 tool calls (>25, inefficient) |
| Found Docs | Yes | 0 | — | Touched docs at docs.copilotkit.ai: yes (12 matching tool calls) |
| Zero Errors | No | 0 | — | Tool outputs with errors/tracebacks: 8 |
| Syntax Valid | Yes | 0 | — | Code compiles/parses correctly |
| Created Files | Yes | 0 | — | Generated files: 2 (need ≥1) |
| Full Execution | Yes | 0 | — | Method: full_execution, Exit: 0 |

## Run Artifacts

| Artifact | Value |
|----------|-------|
| Conversation turns | 37 |
| Tool call traces | 31 |
| Generated files | 2 |
| Exit code | 0 |
| Completed at | 2026-09-26T17:44:13.748998+00:00 |

### Generated Files

- /home/daytona/app/requirements.txt
- /home/daytona/app/support_tools.py

## Related Pages

- [Codex API leaderboard](/leaderboard/codex/api)
- [Compare Codex API companies](/leaderboard/codex/api/compare)
- [Agent Landscape](/leaderboard/discoverability)

Canonical URL: https://devtoolarena.com/codex/api/copilotkit