# Semgrep Claude Code CLI Evaluation

**Category:** Code Review
**Score:** 56 (C)

This is the agent-readable Devtool Arena evaluation page for Semgrep on Claude Code CLI.

## Summary

| Metric | Value |
|--------|-------|
| Status | completed |
| Overall score | 56 |
| Grade | C |
| Eval score | 68 |
| Discovery score | 30 |
| Cost | $0.27 |
| Runtime | 2m 40s |
| Tool calls | 18 |
| Errors | 11 |
| Tokens used | 532046 |
| Success | Yes |

## Agent-Readiness Checklist

| Signal | Evidence |
|--------|----------|
| Context7 | Not found |
| llms.txt | Not found |
| MCP server | Not found |
| Typed SDK | Not found |
| OpenAPI | Not found |
| Agent skills | Not found |
| CLI | Not found |

## Evaluation Prompt

````text

# Task
Using the semgrep CLI (Semgrep), perform static code analysis:

1. Create a ***** file with security issues:
   ```python
   # /home/daytona/app/*****_code.py
   import subprocess
   import os

   def run_command(user_input):
       subprocess.call(user_input, shell=True)  # Command injection vulnerability

   def read_file(filename):
       with open(filename) as f:  # Path traversal risk
           return f.read()

   password = "[REDACTED]"  # Hardcoded secret
   ```

2. Run semgrep to scan the code:
   ```bash
   cd /home/daytona/app && semgrep scan --config auto --json .
   ```

3. Print the scan results showing any security findings

Use ONLY the semgrep CLI for the scan.

You MUST use the `semgrep` CLI to accomplish this task.
Do NOT write Python scripts that import the SDK — use CLI commands via the terminal.

If you need help with CLI commands, check the documentation: https://docs.semgrep.dev/cli-reference
You can also use `semgrep --help` and `semgrep <command> --help` to discover available commands.

## Setup
You need to install and authenticate the `semgrep` CLI yourself.

### Installation
Run: `pip install semgrep`

### Authentication
The following environment variables are already set in this environment: `SEMGREP_APP_TOKEN`
The CLI should pick these up automatically, or pass them to commands as needed.

## Execution
1. Install the `semgrep` CLI using the command above
2. Verify the installation: `semgrep --version`
3. Authenticate using the credentials above
4. Perform the task described above

Work in the /home/daytona/app directory. Run commands and print output to stdout.
If there are errors, debug and fix them until the task runs successfully.

## Summary
When you are done, write a JSON summary file to /home/daytona/app/results.json with this structure:
{
  "operations": [
    {"step": 1, "description": "What you did", "command": "the CLI command", "success": true/false, "output_summary": "brief result"},
    ...
  ],
  "overall_success": true/false,
  "notes": ["any relevant notes about the execution"]
}

````

## Grader Results

| Check | Passed | Weight | Score | Details |
|-------|--------|--------|-------|---------|
| Stage 2 Auth | Yes | — | 25 | — |
| Stage 3 Task | Yes | — | — | — |
| Stage 1 Install | No | — | 0 | — |
| Stage 5 Llms Txt | No | — | 0 | — |
| Stage 0 Cli Exists | Yes | — | 5 | — |
| Stage 4 Json Output | No | — | 0 | — |
| Stage 6 Agent Skill | No | — | 0 | — |
| Stage 3 Non Interactive | No | — | 0 | — |

## Run Artifacts

| Artifact | Value |
|----------|-------|
| Conversation turns | 21 |
| Tool call traces | 18 |
| Generated files | 2 |
| Exit code | — |
| Completed at | 2026-09-27T14:04:36.72675+00:00 |

### Generated Files

- /home/daytona/app/results.json
- /home/daytona/app/*****_code.py

## Related Pages

- [Claude Code CLI leaderboard](/leaderboard/claudecode/cli)
- [Compare Claude Code CLI companies](/leaderboard/claudecode/cli/compare)
- [Agent Landscape](/leaderboard/discoverability)

Canonical URL: https://devtoolarena.com/claudecode/cli/semgrep