mcp-scorecard

Three steps

1. Point it at a server mcp-scorecard scan ./server.py A Python file or a directory. Layer-only variants (footprint / scoping / security / name) also exist.
2. The scorecard comes back Overall grade: D (ORANGE) · tools=4 Overall grade + per-layer band, per-tool token counts, and one findings list per layer.
3. You trim --json / --format sarif Machine-readable output for CI. Exit codes: 0=GREEN/YELLOW, 1=ORANGE, 2=RED.

Example: a real scan

real run, 2026-07-13: mcp-scorecard scan on ~/repos/domain-pre-flight, condensed from the actual CLI output

What the scorecard weighs

Six axes are measured on the declared surface of the MCP server — no code is executed, only the tools/list output and the docstrings behind it:

• Initial token load sum of description + schema + name across all tools, tiktoken cl100k
• Per-tool bloat per-tool cost breakdown; descriptions >150 tokens flagged
• Vague verbs run / handle / process / manage / execute in descriptions caught with an action hint
• When-to-use trigger each tool checked for an explicit trigger phrase in its description
• Secret patterns AWS / GitHub / OpenAI / Anthropic / Slack / PEM regex sweep on descriptions
• Name namespace case collision, Levenshtein brand similarity (~50 brands + 23 known MCPs bundled), separator variants

Runtime security (prompt injection over live traffic, MCP-Scan wrap) and SARIF are on the roadmap for v0.2; registry-source targets (`github:` / `npm:`) for v0.4. The v0.1 scorecard is honest about what it inspects: the declared surface, not the running server.

Who it’s for

• MCP authors about to publish and wanting a token / naming / secret sweep before it ships
• Agent integrators evaluating a third-party MCP before adding it to a Claude Code / Cursor config
• Security reviewers wanting a fast structural pass before deeper runtime testing (MCP-Scan, Inspector)
• Book『MCP実践セキュリティ』readers wanting the checks from the book runnable against their own servers

When to reach for it

• Before publishing a new MCP to PyPI or npm: a single scan surfaces bloated descriptions and unspecified vague verbs
• Before adding a third-party MCP to your agent config: a fast structural read on cost, secrets and naming hygiene
• In CI: --json or --format sarif for a machine-readable gate, with ORANGE/RED as non-zero exit codes
• While writing『MCP実践セキュリティ』-style reviews: the same rules the book proposes, applied consistently

Install

pip install mcp-scorecard              # CLI + library
pip install "mcp-scorecard[mcp]"       # + MCP server (stdio)
mcp-scorecard scan ./server.py

The CLI works standalone. Add the optional `[mcp]` extra to expose `mcp-scorecard-mcp` as an MCP server so an LLM inside Claude Code / Cursor / Windsurf can audit other MCPs from chat.

What it looks at

• Layer A — Passive Footprint: tiktoken-counted description, schema and name per tool; initial_token_load summed; descriptions >150 tokens flagged as bloated
• Layer B — Use-Case Scoping: vague verb scan (run, handle, process, manage, execute), when-to-use trigger check, tool-pair overlap detection, naming style consistency (snake / camel / flat)
• Layer C — Security own rules: regex sweep for AWS / GitHub / OpenAI / Anthropic / Slack keys and PEM blocks in descriptions, tool shadowing check against ls / cat / rm / curl-family names, injection markers in docstrings. MCP-Scan wrap is planned for v0.2
• Layer D — Name Safety: case collision detection, Levenshtein similarity against a bundled set of ~50 brands and 23 known MCP names, separator variant sweep (mcp_scorecard vs mcp-scorecard), namespace hygiene
• 5 MCP tools via mcp-scorecard-mcp: preflight_scan (all four layers), preflight_footprint, preflight_scoping, preflight_security, preflight_name_check. All accept a path or a manifest JSON, so TypeScript / Node MCPs are supported through their manifest

Security patterns caught (samples)

Layer C runs regex sweeps on tool descriptions and structural checks on tool names. A few of the patterns it catches, with the fix pointed at:

source: src/mcp_preflight/rules/security.py · rules/footprint.py (patterns baked into v0.1)

Rewrites the scorecard actually rewards

Two before/after examples of the kind of change the scorecard grades higher:

Usage

# full scan
mcp-scorecard scan ./server.py

# CI-friendly JSON
mcp-scorecard scan ./server.py --json

# register as an MCP server in Claude Code
claude mcp add mcp-scorecard -- mcp-scorecard-mcp

Grading, not blocking

The scorecard is a pre-flight, not a firewall. It grades what it can inspect statically and points at what to fix:

• Each layer returns its own band (GREEN / YELLOW / ORANGE / RED); the overall grade is the worst of the four, so a single ORANGE layer drops the whole card
• Findings are typed with severity ("warn" / "error"), so CI can gate on severity via exit code (0=GREEN/YELLOW, 1=ORANGE, 2=RED)
• v0.1 inspects the declared surface only; runtime prompt injection over live MCP traffic is the job of the planned v0.2 MCP-Scan wrap

The exit-code contract matches SARIF conventions so `mcp-scorecard scan --format sarif` can drop into a CI security gate without extra glue.

Self-check: running the tool against its own MCP server returns overall grade D (ORANGE) — 557 tokens across 5 tools, all layers except scoping green. The scoping findings hit vague verbs (handle / process / manage / execute) because the docstrings explicitly mention those verbs as examples of what to avoid. That is a false positive in context, and it is labeled as such — but the same yardstick is applied to itself, unmodified.

Why this tool exists

The MCP ecosystem grew faster than the review rules for it. A single verbose server can silently burn thousands of tokens per turn just by being registered, because every description and inputSchema is sent to the LLM on every turn. Existing tools cover the runtime side well — MCP-Scan for injection and shadowing at call time, MCP Inspector for protocol compliance — but the LLM-facing surface (how expensive the server is to keep listed, how well-scoped its tools are, whether its names collide with common commands) had no dedicated scorecard. `mcp-scorecard` fills that gap, and does so as a companion to the checks proposed in the book『MCP実践セキュリティ』.

Alongside MCP-Scan and MCP Inspector

Three tools, three different questions. The honest difference is what each one actually inspects:

A dash means the tool does not cover that axis, not that it is worse. MCP-Scan and MCP Inspector are the reference points for runtime security and protocol debugging respectively; mcp-scorecard sits ahead of them, on the LLM-facing quality of the surface itself. Details on MCP-Scan and MCP Inspector reflect the state of their public docs as of 2026-07; axes not covered there are left as "—" rather than guessed.

FAQ

About the author

Built by Ken Imoto: 300+ technical articles across Zenn, Qiita, Dev.to and this site, 40+ books in 4 languages, 400K+ pageviews on Zenn and Qiita, 4 research papers on Zenodo, and creator of the LLMO Framework. The scorecard rules are the same ones baked into the forthcoming book『MCP実践セキュリティ』(Impress NextPublishing).

More about Ken →

Related

Pairs with domain-pre-flight, the pre-flight CLI + MCP for choosing a domain: same pre-flight naming line, same CLI + MCP dual shape, and the sample scan output on this page is a real run against that repo.

domain-pre-flight →