MCP Security · Defense-Coverage Benchmark
A vendor-neutral benchmark. Each tool is driven through its real code against 22 MCP attack vectors, crosswalked to the NIST AI RMF and the OWASP LLM & Agentic Top 10. Results are measured, with matched benign controls — not self-reported.
| Tool | Class | Weighted coverage | ● enforce | ◐ detect | none | False pos. |
|---|---|---|---|---|---|---|
| mcp-bastion | runtime-proxy | 34% (7.5/22) | 2 | 11 | 9 | 0 / 31 |
| mcp-firewall | runtime-proxy | 14% (3.0/22) | 3 | 0 | 19 | 0 / 31 |
| pipelock | egress-firewall | 11% (2.5/22) | 1 | 3 | 18 | 0 / 31 |
| null-baseline | none | 0% (0.0/22) | 0 | 0 | 22 | 0 / 31 |
Weighting: enforce = 1.0, detect = 0.5, none = 0, over 22 vectors. A vector counts only if the tool flags the attack and stays clean on the matched benign control (else it's a false positive). Zero false positives across all tools.
| Vector | Layer | mcp-bastion | mcp-firewall | pipelock | null-baseline |
|---|---|---|---|---|---|
| Tool poisoning | tool | ◐ | |||
| Tool name collision / shadowing | client | ◐ | |||
| Rug pull / dynamic capability mutation | tool | ● | |||
| Out-of-scope parameter injection | tool | ◐ | |||
| Prompt injection via tool results | tool | ◐ | ◐ | ||
| Indirect / retrieval injection | tool | ◐ | ● | ◐ | |
| Cross-tool data exfiltration / confused deputy | client | ◐ | ● | ● | |
| Tool-transfer / cross-server chaining | host-orchestration | ||||
| False-error escalation | tool | ||||
| Package / name squatting in registry | registry-supply-chain | ||||
| Supply-chain poisoning (unverified provenance) | registry-supply-chain | ||||
| Configuration drift | server | ||||
| Sandbox escape | server | ||||
| Schema / validation bypass | server | ◐ | |||
| Man-in-the-middle (transport) | transport | ◐ | |||
| DNS rebinding (local servers) | transport | ● | |||
| Server impersonation / identity spoofing | registry-supply-chain | ||||
| Excessive permission / privilege escalation | host-orchestration | ◐ | |||
| Credential / token theft via passthrough | host-orchestration | ◐ | ● | ||
| Consent fatigue / over-broad grants | client | ||||
| Command injection in tool execution | server | ||||
| System-prompt / context leakage via tools | client | ◐ | ◐ |
Defense-in-depth: the tools cover different layers; 13 of 22 vectors are covered by at least one tool and 9 by none (registry/supply-chain, OS-isolation, and semantic vectors that lie outside a runtime proxy's reach). No single proxy is sufficient.
| Attack | Encoding | mcp-bastion | mcp-firewall | pipelock | null-baseline |
|---|---|---|---|---|---|
| response-injection | homoglyph substitution | ◐ | |||
| response-injection | base64-wrapped payload | ◐ | |||
| tool-poisoning | zero-width / bidi-control characters | ◐ |
Robustness (evasion fixtures detected): mcp-bastion 1/3 · mcp-firewall 0/3 · pipelock 2/3 · null-baseline 0/3. The tools are robust to different obfuscations — no single tool survives all of them.
Every attack vector is crosswalked to the frameworks security and compliance teams govern by. NIST AI RMF is a U.S. federal framework (NIST, Dept. of Commerce); OWASP is an international open standard; STRIDE is the classic threat-modeling taxonomy.
| Vector | Layer | STRIDE | NIST AI RMF | OWASP LLM 2025 | OWASP Agentic 2026 |
|---|---|---|---|---|---|
| Tool poisoning | tool | Tampering/ElevationOfPrivilege | MAP, MEASURE, MANAGE | LLM01, LLM06 | ASI01, ASI02 |
| Tool name collision / shadowing | client | Spoofing/Tampering | MAP, MANAGE | LLM01, LLM03 | ASI01, ASI04 |
| Rug pull / dynamic capability mutation | tool | Tampering | MEASURE, MANAGE | LLM03, LLM06 | ASI04 |
| Out-of-scope parameter injection | tool | Tampering/ElevationOfPrivilege | MEASURE, MANAGE | LLM05, LLM06 | ASI02 |
| Prompt injection via tool results | tool | Tampering | MEASURE, MANAGE | LLM01, LLM05 | ASI01 |
| Indirect / retrieval injection | tool | Tampering | MEASURE, MANAGE | LLM01 | ASI01 |
| Cross-tool data exfiltration / confused deputy | client | InformationDisclosure/ElevationOfPrivilege | MAP, MEASURE, MANAGE | LLM02, LLM06 | ASI02, ASI03 |
| Tool-transfer / cross-server chaining | host-orchestration | ElevationOfPrivilege | MAP, MANAGE | LLM06 | ASI02, ASI03 |
| False-error escalation | tool | ElevationOfPrivilege/DenialOfService | MEASURE, MANAGE | LLM01, LLM06 | ASI01, ASI02 |
| Package / name squatting in registry | registry-supply-chain | Spoofing | GOVERN, MAP | LLM03 | ASI04 |
| Supply-chain poisoning (unverified provenance) | registry-supply-chain | Tampering/Spoofing | GOVERN, MAP, MANAGE | LLM03, LLM04 | ASI04 |
| Configuration drift | server | Tampering | GOVERN, MEASURE, MANAGE | LLM03 | ASI04 |
| Sandbox escape | server | ElevationOfPrivilege | MANAGE | LLM06 | ASI02, ASI03 |
| Schema / validation bypass | server | Tampering/ElevationOfPrivilege | MEASURE, MANAGE | LLM05 | ASI02 |
| Man-in-the-middle (transport) | transport | Tampering/InformationDisclosure/Spoofing | MANAGE | LLM02 | ASI03 |
| DNS rebinding (local servers) | transport | Spoofing/ElevationOfPrivilege | MANAGE | LLM06 | ASI03 |
| Server impersonation / identity spoofing | registry-supply-chain | Spoofing | GOVERN, MAP, MANAGE | LLM03 | ASI03, ASI04 |
| Excessive permission / privilege escalation | host-orchestration | ElevationOfPrivilege | GOVERN, MAP, MANAGE | LLM06 | ASI03 |
| Credential / token theft via passthrough | host-orchestration | InformationDisclosure/ElevationOfPrivilege | GOVERN, MANAGE | LLM02, LLM06 | ASI03 |
| Consent fatigue / over-broad grants | client | ElevationOfPrivilege/Repudiation | GOVERN, MANAGE | LLM06 | ASI03 |
| Command injection in tool execution | server | ElevationOfPrivilege/Tampering | MEASURE, MANAGE | LLM05 | ASI02 |
| System-prompt / context leakage via tools | client | InformationDisclosure | MEASURE, MANAGE | LLM07, LLM02 | ASI01 |
NIST AI RMF: GOVERN · MAP · MEASURE · MANAGE. OWASP LLM: LLM01 Prompt Injection · LLM02 Sensitive Info Disclosure · LLM03 Supply Chain · LLM04 Data/Model Poisoning · LLM05 Improper Output Handling · LLM06 Excessive Agency · LLM07 System Prompt Leakage · LLM08 Vector/Embedding. OWASP Agentic: ASI01 Agent Goal Hijack · ASI02 Tool Misuse · ASI03 Identity & Privilege Abuse · ASI04 Agentic Supply Chain.
Frameworks:
NIST AI RMF 1.0 (U.S. federal) ·
OWASP Top 10 for LLM Apps (2025) ·
OWASP Top 10 for Agentic Apps (2026) ·
STRIDE.
MCP security research:
SoK 2512.08290 ·
Formal framework 2604.05969 ·
STRIDE/DREAD 2603.22489 ·
MCP-DPT 2604.07551 ·
MSB 2510.15994 ·
MCPTox 2508.14925 ·
ETDI 2506.01333 ·
Trustworthy MCP Registry.
MCP protocol:
Specification ·
Registry.
Each adapter drives the tool's actual detection code or CLI — never a mock. A defender is scored only on what it inspects at runtime (a proxy that ignores tool results scores none there, even if its rules would match the text in isolation). Absolute totals are corpus-dependent; the per-vector matrix is the more reliable read. Scored tools run locally and deterministically; cloud-model tools are observed, not scored. Full methodology, prior-art positioning, and the whitepaper are in the repository.