MCP server security has a measurement problem. On September 3, 2026 we ran NVIDIA's open source SkillSpector scanner over every claimed and verified server on MCPVault, 58 repositories in total. The scanner's own verdict: 42 of them CRITICAL, 47 scoring 51 or higher on its 0 to 100 risk scale. Microsoft's Playwright MCP, 36,000 stars, scored a perfect 100, "do not install".
That number is wrong. After reading every finding we stopped trying to grade servers at all. A browser automation server has to run commands. An API wrapper has to call its API. A tool that configures your editor has to write into your editor's config folder. None of that is a defect, so the new weekly security profile on every claimed listing does not say it is. It states what the code can reach, with a file and line behind each item, and leaves the decision to you.
Why a skill scanner misreads a server repository
SkillSpector was built for agent skills: a folder with a SKILL.md and a few scripts that an agent reads and runs. It scans 71 patterns across 17 categories, including prompt injection, data exfiltration, credential access, supply chain, tool poisoning, taint tracking and YARA malware signatures. In that setting the rules are sharp. A hidden HTML comment in a SKILL.md is a real attack, because the agent reads the file.
An MCP server repository is a different object. It has a README with generated sections wrapped in HTML comments, a CI pipeline that writes an .npmrc, a .gitignore listing .env, tests that spin up privileged containers, and code that calls the API it wraps. Every one of those trips a rule that makes sense for a skill.
Here is what the raw scan reported for Playwright MCP, and where each finding lived:
| Rule | Count | Where | What it actually was |
|---|---|---|---|
| Hidden instructions (P2) | 21 | README.md | HTML comments around generated docs sections |
| MCP rug pull (RP1) | 15 | README, CI, Dockerfile | Unpinned npx in install instructions |
| Credential access (PE3) | 5 | CI yaml, .gitignore, tests | The strings .npmrc and .env |
| Known vulnerable dependency (SC4) | 5 | package-lock.json | Real: fast-uri, hono, body-parser, qs |
| Agent config access (AS1) | 1 | README.md | Docs telling users to edit ~/.codex/config.toml |
Five findings out of 52 were real, all of them outdated dependencies. The scanner adds points per rule with diminishing returns, so a dozen distinct medium rules saturate at 100 no matter what the code does.
From findings to a profile
The weekly scan keeps every SkillSpector finding and turns it into a profile in three steps, documented in full at /docs/security-scan.
- Findings are classified by the file they sit in. Shipped code and agent-facing instruction files (AGENTS.md, CLAUDE.md, SKILL.md, the .claude, .cursor and .codex folders) count. README, other docs, configuration, CI and test files are recorded as notes and left out. Across the 58 repositories that split was 2,150 findings in code against 2,242 notes, so half of what the scanner flagged never runs when the server does.
- Each remaining finding maps to a capability, a plain statement of what the code can do: talks to external services, reads environment variables in bulk, runs shell commands, reads or writes other agents' configuration, accesses credential files, and so on. Dependencies with known advisories are listed separately, straight from OSV.dev.
- The profile opens with an exposure level, Minimal, Limited, Broad or Extensive, derived only from which capabilities are present. Limited means the server runs commands or loads code but does not reach secrets or other agents. Broad means it touches one or two sensitive areas such as the environment, credential files or another agent's config. Extensive means three or more.
Playwright MCP under this reading: exposure Limited, runs unpinned packages at runtime, five dependencies with advisories listed by CVE.
What the 58 servers reach
Snapshot date September 3, 2026, 58 claimed and verified repositories, static scan, nothing executed.
| Exposure | Servers | Share | |---|---|---| | Minimal | 12 | 21% | | Limited | 13 | 22% | | Broad | 27 | 47% | | Extensive | 6 | 10% |
Nine servers had no detected capability at all: single-purpose API wrappers whose only outbound traffic is the API they wrap, and small tools that stay inside their own process.
What the code most often reaches. Talking to external services was the most common capability, 39 servers, with hosts named in 43 profiles. Running unpinned packages at runtime came next with 35 servers, mostly npx -y in launch code, which is the ecosystem norm. Running shell commands appeared in 24 servers. Reading or writing other agents' configuration folders, ~/.claude, ~/.codex, ~/.gemini and the like, appeared in 19, mostly installers and hubs that register themselves with every client on the machine. Sending conversation content outward appeared in 12, accessing credential files in 10, and reading environment variables in bulk in 9. Elevated privileges, sudo or the Docker socket, appeared in 3.
Where the data goes. The hosts line collects the external services found in code. The most common were api.github.com (21 servers), api.openai.com (18), api.anthropic.com (6) and api.groq.com (5). Nothing about a host is a finding, but it is the first thing to check before handing a server a credential.
Vulnerable dependencies were the most common fixable item. Nine of the 58 servers shipped lockfiles with known-vulnerable versions: 63 dependency entries covering 58 distinct advisories. The most frequent packages were fast-uri (8 servers), qs (8), brace-expansion (6), hono (4), ip-address (4) and minimatch (4), all transitive Node dependencies with published CVEs and patched releases available. Two servers carried 13 entries each. Every one of these is cleared by a lockfile refresh, and two of the nine had otherwise empty profiles.
What put servers at Extensive. The six servers at the top of the scale share a shape: agent frameworks, orchestration CLIs and desktop apps rather than single-purpose wrappers. They read the environment in bulk, forward conversation content, write into other agents' configuration and spawn processes, because that is what an agent manager does. The profile lists each with file and line, and a user who wants an agent manager will recognise the feature set.
What is on the listing now
Every claimed and verified server on MCPVault now carries a security profile. It opens with the exposure level and one sentence explaining it, then lists what was detected, how many places each capability was seen and the external hosts, followed by a line naming the core capabilities that were not detected, the dependencies with advisories by package, version and identifier, and the count of notes left out. The scan runs weekly and picks up the latest commit each time.
Owners get the full picture in their dashboard: each capability with the file, line and matched snippet, every finding with a remediation, and every note with the reason it was excluded. The fastest way to change a profile is to refresh a lockfile; the next run reflects it.
A typical profile for a small API wrapper reads like this on the listing:
Security profile
Exposure: Minimal
Stays within its own process: at most external requests and local configuration.
Detected
Talks to external services 1 place
api.github.com
Not detected: reads environment variables in bulk, sends conversation
content outward, runs shell commands, reads or writes other agents'
configuration, accesses credential files, uses elevated privileges,
installs itself to run later, matches a known malicious pattern.
Dependencies with known advisories
qs 6.15.2 Low, CVE-2026-82417, CVE-2026-82562
fast-uri 3.1.4 High, CVE-2026-75931, CVE-2026-18446, +3
Verification proves the server starts and answers a real MCP handshake, inside a kernel sandbox. Quality grades summarise maintenance and adoption. The profile answers the remaining question: what does the code reach for while it runs.
How to read a profile before installing
Start with the detected list and ask whether each item fits what the server is for. A filesystem server that runs shell commands is doing its job. A weather server that reads environment variables in bulk deserves a look at the file where that happens, which the owner dashboard shows and the public card names. Then read the hosts line, because that is where your data can go. Then check the dependency list; an old lockfile is the most common and most fixable item. The not-detected line is the other half: a concrete statement you can hold the code to, not proof of safety. Keep the client-side tool approval prompts on either way.
Frequently asked questions
Does the security profile run the server?
No. The scan downloads a snapshot of the repository, removes tests, CI and documentation folders, and reads the remaining files. Dependency versions are checked against OSV.dev. Nothing is executed. Running the server is what verification does, inside a Landlock and seccomp sandbox on MCPVault's verifier host.
Why is there no security score?
Because a score is a verdict, and a static scanner cannot deliver one for an MCP server. SkillSpector's own score put Playwright MCP at 100, and any weighting we tried still punished servers for doing their job. The profile states what the code reaches and the exposure level says how much, without saying whether that is acceptable. That depends on what you wanted the server for, and you know that better than a scanner does.
My server shows Broad or Extensive exposure. Is that bad?
It means the code reaches sensitive areas: the environment, credential files, other agents' configuration, or it sends conversation content outward. For an agent manager or an installer, that is the feature. For a server that should not need it, it is worth a look at the file and line in your dashboard. A sentence in your README explaining why the server needs what it reaches helps users more than anything the scanner can say.
Can I get my unclaimed server profiled?
Profiles are built for claimed and verified servers only, because the result is public and a maintainer should be the first to see it. Claiming is free, takes a GitHub sign-in and about a minute, and the listing joins the next weekly run.
MCPVault lists 17,900+ MCP servers with quality grades, live verification and now a weekly security profile on every claimed listing. Browse the vault to read profiles, or claim your server to see yours first.