Pre-flight your MCP: four layers to grade a server before you publish it

The MCP ecosystem grew faster than the review rules for it. In the last few weeks I published three MCP servers of my own (domain-pre-flight, rag-db-advisor, opencut-mcp) and each time I found myself running the same mental checklist before shipping: how many tokens am I burning per turn just by being registered, are my tool descriptions telling the LLM enough to pick the right one, am I leaking anything into a description string, and does the tool name look like something else.

I moved that checklist into a tool and put it on PyPI. mcp-scorecard runs four pre-flight checks against an MCP server’s declared surface and returns a scorecard graded A–F with per-tool findings. The CLI is one command; the same four layers ship as five MCP tools, so an LLM inside Claude Code can audit another MCP without leaving the session.

pip install mcp-scorecard
mcp-scorecard scan ./your-server.py

Deliverables: kenimo49/mcp-scorecard v0.1.1 on PyPI (MIT). The full product page with the demo GIF and the compare table against MCP-Scan and MCP Inspector is at /products/mcp-scorecard/.

This post is the reasoning behind each of the four layers, why they exist in that order, and what the tool caught the first time I scanned my own MCPs. If you already run MCP-Scan or MCP Inspector, the last section explains where this one sits alongside them.

Why pre-flight for MCP at all

The default assumption when reviewing an MCP server is that the interesting risks live at call time: prompt injection in a tool response, credentials leaking through a shell exec, tool shadowing that steers the model to the wrong function. Those are real, and MCP-Scan covers them at runtime. What that framing misses is the surface the LLM sees before any tool is called.

Every tools/list entry is sent to the model on every turn — because the model needs the list to decide which tool to call. That is the passive cost of registering a server. And every description field in that list is what the model reads to choose. That is the passive quality of the server. Both are set at author time and neither depends on runtime traffic. Both are also invisible to a runtime scanner.

A pre-flight covers exactly that layer: what does the LLM see about your MCP, before any request goes out. The four layers below are one attempt at a compact answer.

Layer A — Passive Footprint

A single verbose MCP server can silently burn 5,000+ tokens per turn without anyone calling a tool. The tokens come from three places: the description string for each tool, the JSON Schema for each tool’s input, and the tool name itself. All three get concatenated into tools/list and shipped to the model every turn.

Layer A counts these with tiktoken on the cl100k_base encoding (the OpenAI GPT-4-family tokenizer, close enough to Claude’s to be a reasonable proxy). The output is a per-tool breakdown plus a global initial_token_load:

                per-tool footprint (top 10 by total)
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ Tool                   ┃ Desc tok ┃ Schema tok ┃ Name tok ┃ Total ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ check_domain           │      182 │         88 │        2 │   272 │
│ list_typo_permutations │       79 │         54 │        5 │   138 │
│ check_trademark        │       83 │         41 │        4 │   128 │
│ check_handles          │       76 │         40 │        2 │   118 │
└────────────────────────┴──────────┴────────────┴──────────┴───────┘
findings
  · 1 tool(s) have description > 150 tokens: check_domain

That is domain-pre-flight scanned by mcp-scorecard. Total initial_token_load is 656 tokens for four tools, well inside the GREEN band. But one tool (check_domain) is flagged: its description is 182 tokens, over the 150-token bloat threshold. When five servers like that are registered simultaneously, the passive drain adds up in a way that stays invisible until an “unused MCP” review.

The thresholds are calibrated but not sacred. v0.1 uses:

Metric	GREEN	YELLOW	ORANGE	RED
initial_token_load	≤ 1500	≤ 4000	≤ 8000	> 8000
Description tokens per tool	≤ 150	—	—	flagged as bloat if over
Tool count	≤ 15	—	—	≥ 30 raises tool_count warning

The RED threshold on initial_token_load is roughly where a single server starts eating into the context budget of long conversations. The 150-token bloat threshold on per-tool descriptions is where descriptions start reading like documentation instead of one-line usage.

The obvious way to fix a bloat finding is to trim the description to a single-purpose sentence and move examples out of the tool surface. Layer B has more to say about what “single-purpose sentence” should contain.

Layer B — Use-Case Scoping

Passive footprint answers how much the LLM sees. Layer B answers how well-targeted what it sees is. The failure mode is: the model has three tools with plausible descriptions and can’t tell which one to reach for, so it picks by heuristic and gets it wrong. Fixing that is not a security check; it is a UX check on the LLM as the user of your MCP.

Four rules run in v0.1:

Vague verbs. Descriptions that start with run, handle, process, manage, execute, do, perform, work with, deal with, or the generic helper / utility are flagged with an action hint. The flag on run on my own check_domain is a good example: the description said “run pre-flight checks on a domain”, which reads fine as documentation but leaves the LLM with no signal about what is being run. Rewriting to “check availability, run WHOIS, resolve DNS, and score TLD risk for a domain candidate” gives the same information plus enough disambiguation for the model to pick this tool over a sibling.

When-to-use trigger. Each tool’s description is checked for an explicit trigger phrase (use this, use when, call when, useful for, and the Japanese equivalents 使う, 使用, 呼び出). Three of my four domain-pre-flight tools do not have one, which is what the ORANGE band in the scan above is complaining about. The trigger is not a magic keyword; it is a marker for the model saying “here is the situation where I am the right tool”. A missing trigger tends to correlate with tools that get called from context inference rather than explicit reasoning, which is not a stable pattern.

Overlap detection. Each pair of tool descriptions is scored on shared stem-token overlap. If two tools use most of the same content words in their descriptions, they are probably telling the LLM roughly the same story and it will pick one at random. v0.1 flags overlap pairs above a threshold but does not fail on them; the fix is usually to rewrite one of the two descriptions to name the axis that separates them.

Naming style consistency. The bundled check classifies each tool name as snake_case, camelCase, kebab-case, or flat, and flags a server whose tools mix styles. Mixed styles do not cost tokens directly, but they cost model attention: switching between naming conventions in the same list makes the model spend a small budget on remembering which style a given tool used, and that budget could have gone into the reasoning.

Layer B is the layer where the tool grades its own MCP server harshly. Running mcp-scorecard scan against mcp-scorecard’s own MCP server returns overall grade D (ORANGE) because five scoping findings fire on the docstrings of the preflight_* tools. Four of those findings hit vague verbs (handle, process, manage, execute) that appear in the docstrings as examples of what the scoping check catches. That is a false positive in context, and it is flagged as such in the caveat on the product LP — but the same yardstick is applied to the tool itself, unmodified. The alternative would have been to whitelist my own docstrings, and that would have been the kind of trick that makes a rule set useless.

Layer C — Security own rules

Layer C is the layer that most obviously overlaps with existing tools, so it is also the layer with the tightest scope. v0.1 runs three families of own rules against the declared surface only; the runtime coverage that MCP-Scan does well is deliberately left for a v0.2 wrap.

Prompt injection markers in descriptions. A tool description is text the LLM reads on every turn, so any imperative language in it steers the model whether the author meant to or not. The rule catches known injection patterns (ignore previous instructions, disregard, system: prefix, closing </system> tags, you must, and the equivalent Japanese phrasing). This one comes straight out of the book『MCP実践セキュリティ』(Impress NextPublishing) whose review checklist informs several rules in this layer.

Tool shadowing. A tool named ls that lists remote objects will get called instead of the shell ls any time the model reads intent-imprecise instructions like “list the files here”. The rule checks tool names against a small set of common shell / filesystem / process names (ls, cat, rm, cp, curl, sudo, exec, eval, shell, execute) and flags any collision. The fix is namespace hygiene: rename to list_objects or s3_ls and the ambiguity goes away.

Hardcoded secrets in descriptions. Regex sweep for AWS access keys (AKIA...), GitHub PATs (ghp_...), OpenAI keys (sk-...), Anthropic keys (sk-ant-...), Google API keys (AIza...), Slack tokens (xox...), PEM private-key blocks (-----BEGIN PRIVATE KEY-----), and hardcoded Bearer patterns. The failure mode is not “the credential got committed to the repo” — that is gitleaks / trufflehog territory and deliberately not duplicated. The failure mode is “a description string sent to every LLM turn contains a credential”, which is a different and worse leak because it exfiltrates the credential through the model’s context on every request.

What the layer does not do in v0.1: full-source secret scanning, runtime injection over live traffic, or protocol-level validation. All three are covered better by existing tools; the v0.2 roadmap is a wrap around MCP-Scan for the runtime side, not a rewrite of it.

Layer D — Name Safety

Layer D covers the naming decisions the author already made and asks whether they are safe to publish. Four rules:

Case collision. The bundled dictionary of ~50 brand names and 23 known MCP names is normalized to lowercase; the candidate name is checked against it after the same normalization. GitHub-mcp collides with github-mcp under this rule, and the finding is “case collision under PEP 503 style normalization; will be indistinguishable from github-mcp once packaged”.

Brand Levenshtein similarity. Candidate names within Levenshtein distance 2 of a known brand (for brands ≥ 4 characters) trigger the typosquat warning. This is the same rule that runs in domain-pre-flight for domain typosquat detection, scaled down to package-name character counts. The 23 known-MCP list is small on purpose; the fix for a hit is not to make the list bigger but to pick a name that does not need to be adjacent to something else.

Separator variants. mcp_scorecard vs mcp-scorecard vs mcpscorecard all normalize to the same package name under PyPI’s PEP 503 rules, so shipping one variant while another already exists on PyPI is a namespace conflict. This layer catches it before publish; I ran into the same class of issue with mcp-scorecard itself, where the original name mcp-preflight was already occupied and mcp-pre-flight was rejected by PyPI as too similar. Layer D would have flagged the second name as “too close to an existing MCP” before I hit the PyPI validator.

Namespace hygiene. A regex on the candidate name checks for lowercase kebab-case with the optional @vendor/tool scoping. Names with mixed case or unusual characters are flagged as “prefer kebab-case or @vendor/tool scoping”. This one is cosmetic; the finding is a warn, not an error. It exists because the LLM’s tool-picking heuristics do work better on consistently formatted names, which loops back to Layer B.

The 23 known-MCP list is bundled as static data. It ships with obvious brands (github, google, anthropic, openai, aws, cloudflare, stripe, hubspot) plus a smaller set of common MCP names (mcp-scan, mcp-inspector, mcp-validator, and this tool’s own name). Additions are a plain PR against src/mcp_preflight/data/known_brands.py.

What it does not cover

The four layers above are the LLM-facing quality of an MCP server: what the model sees, how expensive that is, whether it can pick the right tool, whether the names collide with things it already knows. What they do not cover is the runtime security surface that MCP-Scan handles well: prompt injection at call time in tool outputs, credential handling during exec, tool-shadowing at request time. That is a separate scanner reading a live server; v0.1 of mcp-scorecard is an AST + manifest read, and it never executes the target. The compare table on the product LP has the full three-way position against MCP-Scan and MCP Inspector, with the axes each tool does and does not cover marked honestly rather than filled in.

A CI-friendly gate is on the roadmap: --format sarif output and non-zero exit codes for ORANGE / RED already work today, and the JSON schema is stable enough for local use. What is not stable in v0.1 is the specific band thresholds; they will move as I run the tool against more real MCPs and find where the current numbers are too permissive or too strict. The alpha label on v0.1 exists exactly for that reason.

Install

pip install mcp-scorecard              # CLI + library
pip install "mcp-scorecard[mcp]"       # + MCP server (stdio)

mcp-scorecard scan ./your-server.py    # full four-layer scan
mcp-scorecard footprint ./server.py    # Layer A only
mcp-scorecard scoping ./server.py      # Layer B only
mcp-scorecard security ./server.py     # Layer C only
mcp-scorecard name my-new-mcp          # Layer D only, on a candidate name

TypeScript / Node MCPs are supported through a manifest JSON: pass the tools/list output (or a saved copy) as JSON to --target, and Layers A / B / C / D all run against it. The AST path is Python-specific; the layers themselves are not.

To use it as an MCP server so Claude Code / Cursor / Windsurf can audit another MCP from chat:

{
  "mcpServers": {
    "mcp-scorecard": {
      "command": "mcp-scorecard-mcp"
    }
  }
}

Then ask the model to score an MCP by path; five tools (preflight_scan, preflight_footprint, preflight_scoping, preflight_security, preflight_name_check) route to the same layers as the CLI.

Related

mcp-scorecard is the LLM-facing quality companion to the runtime security and protocol tools already in the ecosystem, and the pre-flight companion to domain-pre-flight on the CLI + MCP naming line. It is also the tool half of a companion pair with the forthcoming book『MCP実践セキュリティ』(Impress NextPublishing): the book teaches the checks, the tool runs them.

Feedback on the rules and thresholds is exactly what a v0.1 alpha needs. Issues welcome on kenimo49/mcp-scorecard.