# Statsig Codex MCP Evaluation

**Category:** Feature Flags
**Score:** 79 (B)

This is the agent-readable Devtool Arena evaluation page for Statsig on Codex MCP.

## Summary

| Metric | Value |
|--------|-------|
| Status | completed |
| Overall score | 79 |
| Grade | B |
| Eval score | 84 |
| Discovery score | 68 |
| Cost | $0.13 |
| Runtime | 3m 19s |
| Tool calls | 8 |
| Errors | 0 |
| Tokens used | 345722 |
| Success | Yes |

## Agent-Readiness Checklist

| Signal | Evidence |
|--------|----------|
| Context7 | Not found |
| llms.txt | Not found |
| MCP server | Not found |
| Typed SDK | Not found |
| OpenAPI | Not found |
| Agent skills | Not found |
| CLI | Not found |

## Evaluation Prompt

````text

# Task
Using the Statsig MCP, create and target a feature flag:

1. Show the active project, account, or environment context if the MCP tools expose it
2. List existing feature flags (feature gates)
3. Create a feature flag named "sapient_leaderboard_test", or locate it if it already exists
4. Add or update a targeting rule so users with plan="pro" (or user ID "leaderboard-test-user") receive the enabled value. If the tools support environments, scope the rule to a development or test environment. If the tools only support a rollout percentage, set that instead
5. Fetch the flag again and confirm the rule or rollout was saved
6. If the MCP tools can evaluate a flag for a user, evaluate it for user ID "leaderboard-test-user" with plan="pro" and region="us"
7. Print the result as JSON with fields: flag_key, flag_id, created (true if new, false if it already existed), targeting_rule, environment, evaluation (null if the tools cannot evaluate)

Use ONLY the MCP tools for all operations. Do not change or delete any other flag.

You MUST use the Statsig MCP server to accomplish this task.
Use the MCP tools for the actual product operations. Do NOT use curl, fetch, raw HTTP requests, CLI commands, or SDK calls as substitutes.

Use the MCP tools' help/descriptions to understand what each tool does.
If you need additional context, check the documentation: https://docs.statsig.com/integrations/mcp/claude-code

## Setup
Do NOT assume the Statsig MCP server is already installed or already running in this sandbox.
Review the installation and authentication details below before starting the task.

### Installation
Transport: `stdio`
Codex is configured to launch the `statsig` MCP server over stdio using: `npx -y mcp-remote@0.15.1 https://api.statsig.com/v3/mcp --header 'statsig-api-key:${STATSIG_CONSOLE_API_KEY}'`
No separate setup command is required before startup unless the docs indicate one.
Bootstrap notes: Added 2026-10-20 for Statsig MCP v3 (statsig.com/blog/introducing-statsig-mcp-v3, launched 2026-10-07). Config from docs.statsig.com/integrations/mcp/claude-code: remote https://api.statsig.com/v3/mcp with a Console API key in the statsig-api-key header (the API-key alternative to OAuth). Bridged over stdio with mcp-remote 0.15.1 so both harnesses send the header non-interactively.

### Authentication
The following environment variables are already set in this environment: `STATSIG_CONSOLE_API_KEY`
The MCP server should read them automatically when it starts.

## Execution
1. The harness has already written the launch configuration for `statsig`. Start with the task directly; do not rewrite agent settings, `.mcp-runtime.json`, or build a separate MCP client unless the connected server still fails after a normal retry.
2. You do not need to enumerate every MCP tool up front; inspect only the relevant tools you need. Agent-facing MCP tool names may be namespaced like `mcp__statsig__<tool_name>`.
3. Complete the task described above using MCP tools for the actual product operations.
4. If the server fails to start or authenticate, debug the documented setup and retry. Do not fall back to raw HTTP, SDK, CLI, or manual JSON-RPC/MCP calls as a substitute for exposed MCP tools.

Work in the /home/daytona/app directory.
If you use Bash during setup or debugging, limit it to installation/configuration work and then return to MCP tools for the task itself.
If there are errors, debug and fix them until the task runs successfully.

## Summary
When you are done, write a JSON summary file to /home/daytona/app/results.json with this structure:
{
  "operations": [
    {"step": 1, "description": "What you did", "mcp_tool": "tool_name", "success": true/false, "output_summary": "brief result"},
    ...
  ],
  "overall_success": true/false,
  "notes": ["any relevant notes about the execution"]
}

````

## Grader Results

| Check | Passed | Weight | Score | Details |
|-------|--------|--------|-------|---------|
| Task | Yes | — | — | — |
| Llms Txt | Yes | — | 10 | — |
| Tool Count | Yes | — | 7 | — |
| Auth Method | Yes | — | 15 | — |
| Official Mcp | Yes | — | 5 | — |
| Output Schema | No | — | 0 | — |
| Error Messages | — | — | 0 | api_destructive: Poor error message (generic or stack trace); fetch: Poor error message (generic or stack trace) |
| Tool Stability | Yes | — | 5 | — |
| Security Posture | — | — | 9 | — |
| Tool Api Mapping | — | — | 6 | — |
| Description Clarity | — | — | 11 | — |
| Tool Description Length | No | — | 0 | — |

## Run Artifacts

| Artifact | Value |
|----------|-------|
| Conversation turns | 12 |
| Tool call traces | 8 |
| Generated files | 1 |
| Exit code | — |
| Completed at | 2026-10-20T18:33:49.901542+00:00 |

### Generated Files

- /home/daytona/app/results.json

## Related Pages

- [Codex MCP leaderboard](/codex/mcp)
- [Compare Codex MCP companies](/codex/mcp/compare)
- [Agent Landscape](/discoverability)

Canonical URL: https://devtoolarena.com/codex/mcp/statsig