MCP Security · Defense-Coverage Benchmark

How much of the MCP attack surface does each security proxy actually cover?

A vendor-neutral benchmark. Each tool is driven through its real code against 24 MCP attack vectors, crosswalked to the NIST AI RMF and the OWASP LLM & Agentic Top 10. Results are measured, with matched benign controls — not self-reported.

Coverage of the full attack surface

Weighting: enforce = 1.0, detect = 0.5, none = 0, over 24 vectors. CorpusRobustCoverage (headline) is the mean across all fixtures of a vector, including evasion variants; Capability is best-case (detects ≥ 1 fixture). Where the two diverge, the tool is brittle to evasion. A vector counts only if the tool flags the attack and stays clean on the matched benign control (else it's a false positive); a false positive on any benign variant scores that vector 0. Zero false positives across all tools.

Scope of this number: CorpusRobustCoverage is a descriptive coverage measure over a fixed, non-exhaustive corpus — not a procurement or substitutability ranking. A higher score does not mean a tool better closes your deployment's gap, and tools at different architectural layers (e.g. a runtime proxy vs. an egress firewall) are not substitutable. Read the per-vector matrix and the defense-in-depth view for decisions. Scoping clarified and the original metric fix credited to M. Brighindi; a decision-indexed RequiredGapClosure metric is planned — see CHANGELOG-scoring.

Per-vector coverage matrix

Defense-in-depth: the tools cover different layers; 19 of 24 vectors are covered by at least one tool and 5 by none (registry/supply-chain, OS-isolation, and semantic vectors that lie outside a runtime proxy's reach). No single proxy is sufficient.

Evasion robustness

Robustness (evasion fixtures detected): mcp-bastion 3/3 · mcp-firewall 0/3 · pipelock 2/3 · null-baseline 0/3. The tools are robust to different obfuscations — no single tool survives all of them.

Framework mapping

Every attack vector is crosswalked to the frameworks security and compliance teams govern by. NIST AI RMF is a U.S. federal framework (NIST, Dept. of Commerce); OWASP is an international open standard; STRIDE is the classic threat-modeling taxonomy.

NIST AI RMF: GOVERN · MAP · MEASURE · MANAGE. OWASP LLM: LLM01 Prompt Injection · LLM02 Sensitive Info Disclosure · LLM03 Supply Chain · LLM04 Data/Model Poisoning · LLM05 Improper Output Handling · LLM06 Excessive Agency · LLM07 System Prompt Leakage · LLM08 Vector/Embedding. OWASP Agentic: ASI01 Agent Goal Hijack · ASI02 Tool Misuse · ASI03 Identity & Privilege Abuse · ASI04 Agentic Supply Chain. NSA guidance (May 2026): AUTH Authentication & Access Control · LOG Monitoring & Logging · SEG Data Classification & Segmentation · RT Runtime Controls.

References

Frameworks: NIST AI RMF 1.0 (U.S. federal) · OWASP Top 10 for LLM Apps (2025) · OWASP Top 10 for Agentic Apps (2026) · STRIDE · NSA MCP Security CSI (U.S. federal — NSA AI Security Center, May 2026).
MCP security research: SoK 2512.08290 · Formal framework 2604.05969 · STRIDE/DREAD 2603.22489 · MCP-DPT 2604.07551 · MSB 2510.15994 · MCPTox 2508.14925 · ETDI 2506.01333 · Trustworthy MCP Registry · WebMCP/MSTI 2606.06387 · ShareLock 2606.27027.
MCP protocol: Specification · Registry.

Method & honesty

Each adapter drives the tool's actual detection code or CLI — never a mock. A defender is scored only on what it inspects at runtime (a proxy that ignores tool results scores none there, even if its rules would match the text in isolation). Absolute totals are corpus-dependent; the per-vector matrix is the more reliable read. Scored tools run locally and deterministically; cloud-model tools are observed, not scored. Full methodology, prior-art positioning, and the whitepaper are in the repository.