# LanceDB Claude Code MCP Evaluation

**Category:** Vector Databases
**Score:** 9 (D)

This is the agent-readable Devtool Arena evaluation page for LanceDB on Claude Code MCP.

## Summary

| Metric | Value |
|--------|-------|
| Status | completed |
| Overall score | 9 |
| Grade | D |
| Eval score | 0 |
| Discovery score | 30 |
| Cost | $0.32 |
| Runtime | 4m 7s |
| Tool calls | 15 |
| Errors | 3 |
| Tokens used | 400244 |
| Success | No |

## Agent-Readiness Checklist

| Signal | Evidence |
|--------|----------|
| Context7 | Not found |
| llms.txt | Not found |
| MCP server | Not found |
| Typed SDK | Not found |
| OpenAPI | Not found |
| Agent skills | Not found |
| CLI | Not found |

## Evaluation Prompt

````text

# Task
Using the LanceDB MCP, perform vector database operations:

1. Discover available MCP tools and their capabilities
2. Create a new collection/index called "faq-*****"
3. List collections to verify it was created
4. If the MCP tools support data operations, insert a ***** document and query the collection
5. Print all output clearly labeled for each step

Use ONLY the MCP tools for all operations.

You MUST use the LanceDB MCP server to accomplish this task.
Use the MCP tools for the actual product operations. Do NOT use curl, fetch, raw HTTP requests, CLI commands, or SDK calls as substitutes.

Use the MCP tools' help/descriptions to understand what each tool does.
If you need additional context, check the documentation: https://github.com/lancedb/lancedb-mcp-server

## Setup
Do NOT assume the LanceDB MCP server is already installed or already running in this sandbox.
Review the installation and authentication details below before starting the task.

### Installation
Transport: `stdio`
Claude Code is configured to launch the `lancedb` MCP server over stdio using: `uv --directory /home/daytona/lancedb-mcp-server run lancedb_mcp.py`
Stored MCP package hint: `lancedb-mcp-server`
Run this setup command in the sandbox before the agent launches the server: `bash -c '***** -d /home/daytona/lancedb-mcp-server || git clone https://github.com/lancedb/lancedb-mcp-server.git /home/daytona/lancedb-mcp-server'`
Run this setup command in the sandbox before the agent launches the server: `cd /home/daytona/lancedb-mcp-server && uv sync`
Bootstrap notes: Overriding stored hints: the vendor repo (github.com/lancedb/lancedb-mcp-server, confirmed owned by the 'lancedb' GitHub org) is a Python/uv project, NOT an npm package. The npm package 'lancedb-mcp-server' is an unrelated third-party package by a different maintainer whose own README describes a different Rust/cargo tool -- do not use `npx -y lancedb-mcp-server`. Read lancedb_mcp.py source directly: it uses FastMCP('lancedb') with mcp.run() (stdio transport, no port/URL), matching the vendor README's Claude config sample (command+args, no url). LANCEDB_API_KEY does not exist anywhere in the vendor source -- that stored hint is incorrect. Confirmed auth_method is 'none'.

### Authentication
No authentication step is required for this MCP server.

## Execution
1. The harness has already written the launch configuration for `lancedb`. Start with the task directly; do not rewrite agent settings, `.mcp-runtime.json`, or build a separate MCP client unless the connected server still fails after a normal retry.
2. You do not need to enumerate every MCP tool up front; inspect only the relevant tools you need. Agent-facing MCP tool names may be namespaced like `mcp__lancedb__<tool_name>`.
3. Complete the task described above using MCP tools for the actual product operations.
4. If the server fails to start or authenticate, debug the documented setup and retry. Do not fall back to raw HTTP, SDK, CLI, or manual JSON-RPC/MCP calls as a substitute for exposed MCP tools.

Work in the /home/daytona/app directory.
If you use Bash during setup or debugging, limit it to installation/configuration work and then return to MCP tools for the task itself.
If there are errors, debug and fix them until the task runs successfully.

## Summary
When you are done, write a JSON summary file to /home/daytona/app/results.json with this structure:
{
  "operations": [
    {"step": 1, "description": "What you did", "mcp_tool": "tool_name", "success": true/false, "output_summary": "brief result"},
    ...
  ],
  "overall_success": true/false,
  "notes": ["any relevant notes about the execution"]
}

````

## Grader Results

| Check | Passed | Weight | Score | Details |
|-------|--------|--------|-------|---------|
| Task | No | — | — | — |
| Llms Txt | Yes | — | 10 | — |
| Tool Count | No | — | 0 | — |
| Auth Method | No | — | 0 | — |
| Official Mcp | Yes | — | 5 | — |
| Output Schema | No | — | 0 | — |
| Tool Stability | Yes | — | 5 | — |
| Tool Description Length | Yes | — | 10 | — |

## Run Artifacts

| Artifact | Value |
|----------|-------|
| Conversation turns | 16 |
| Tool call traces | 15 |
| Generated files | 0 |
| Exit code | — |
| Completed at | 2026-09-27T15:06:37.459189+00:00 |

## Related Pages

- [Claude Code MCP leaderboard](/leaderboard/claudecode/mcp)
- [Compare Claude Code MCP companies](/leaderboard/claudecode/mcp/compare)
- [Agent Landscape](/leaderboard/discoverability)

Canonical URL: https://devtoolarena.com/claudecode/mcp/lancedb