@kryptosai/mcp-observatory 0.28.0 → 1.28.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/COMMERCIAL.md +26 -5
- package/README.md +65 -12
- package/dist/src/action-receipt.d.ts +20 -0
- package/dist/src/action-receipt.js +85 -0
- package/dist/src/action-receipt.js.map +1 -0
- package/dist/src/audit.d.ts +64 -0
- package/dist/src/audit.js +359 -0
- package/dist/src/audit.js.map +1 -0
- package/dist/src/checks/attack-sim.d.ts +2 -0
- package/dist/src/checks/attack-sim.js +12 -0
- package/dist/src/checks/attack-sim.js.map +1 -1
- package/dist/src/cli.js +8 -1
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/attack-sim.js +54 -2
- package/dist/src/commands/attack-sim.js.map +1 -1
- package/dist/src/commands/audit.d.ts +2 -0
- package/dist/src/commands/audit.js +132 -0
- package/dist/src/commands/audit.js.map +1 -0
- package/dist/src/commands/init-ci.d.ts +2 -0
- package/dist/src/commands/init-ci.js +71 -11
- package/dist/src/commands/init-ci.js.map +1 -1
- package/dist/src/commands/legacy.js +2 -0
- package/dist/src/commands/legacy.js.map +1 -1
- package/dist/src/commands/receipt.d.ts +2 -0
- package/dist/src/commands/receipt.js +113 -0
- package/dist/src/commands/receipt.js.map +1 -0
- package/dist/src/commands/scan.js +23 -5
- package/dist/src/commands/scan.js.map +1 -1
- package/dist/src/commands/score.js +63 -0
- package/dist/src/commands/score.js.map +1 -1
- package/dist/src/commands/setup-ci-conversion.d.ts +2 -1
- package/dist/src/commands/setup-ci-conversion.js +22 -8
- package/dist/src/commands/setup-ci-conversion.js.map +1 -1
- package/dist/src/commands/test.js +18 -1
- package/dist/src/commands/test.js.map +1 -1
- package/dist/src/findings.d.ts +1 -0
- package/dist/src/findings.js +1 -0
- package/dist/src/findings.js.map +1 -1
- package/dist/src/index.d.ts +2 -0
- package/dist/src/index.js +2 -0
- package/dist/src/index.js.map +1 -1
- package/dist/src/receipt.d.ts +100 -0
- package/dist/src/receipt.js +375 -0
- package/dist/src/receipt.js.map +1 -0
- package/dist/src/reporters/attack-sim.js +6 -1
- package/dist/src/reporters/attack-sim.js.map +1 -1
- package/dist/src/reporters/sarif.js +11 -1
- package/dist/src/reporters/sarif.js.map +1 -1
- package/dist/src/security-profiles.d.ts +13 -0
- package/dist/src/security-profiles.js +59 -0
- package/dist/src/security-profiles.js.map +1 -0
- package/dist/src/telemetry.d.ts +7 -0
- package/dist/src/telemetry.js.map +1 -1
- package/docs/10x-agentic-growth-sprint.md +47 -0
- package/docs/agent-ecosystem-promotion-plan.md +1 -1
- package/docs/agentic-contributor-outreach.md +52 -0
- package/docs/code-scanning-demo.md +2 -2
- package/docs/commercial-boundary.md +83 -0
- package/docs/contributor-proof-cards/README.md +1 -1
- package/docs/examples/nsa-mcp-audit-report.md +372 -0
- package/docs/examples/nsa-mcp-results.sarif +930 -0
- package/docs/examples/nsa-mcp-score.json +13 -0
- package/docs/feishu-lark-mcp.md +1 -1
- package/docs/github-code-scanning-for-mcp.md +1 -1
- package/docs/government-enterprise-pilot.md +93 -0
- package/docs/mcp-receipts.md +109 -0
- package/docs/mcp-server-safety-index.md +26 -25
- package/docs/nsa-mcp-audit-ci.md +88 -0
- package/docs/procurement-one-pager.md +74 -0
- package/docs/proof.md +2 -2
- package/docs/public-guidance-crosswalk.md +48 -0
- package/docs/safety-index/artifacts/antv-chart-server.json +41 -23
- package/docs/safety-index/artifacts/antv-chart-server.md +23 -10
- package/docs/safety-index/artifacts/browsermcp-server.json +38 -20
- package/docs/safety-index/artifacts/browsermcp-server.md +23 -10
- package/docs/safety-index/artifacts/context7-server.json +85 -28
- package/docs/safety-index/artifacts/context7-server.md +35 -22
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.json +460 -36
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.md +31 -18
- package/docs/safety-index/artifacts/everything-server.json +38 -20
- package/docs/safety-index/artifacts/everything-server.md +25 -12
- package/docs/safety-index/artifacts/executeautomation-playwright-server.json +155 -20
- package/docs/safety-index/artifacts/executeautomation-playwright-server.md +26 -13
- package/docs/safety-index/artifacts/filesystem-server.json +116 -22
- package/docs/safety-index/artifacts/filesystem-server.md +27 -14
- package/docs/safety-index/artifacts/kubernetes-server.json +252 -27
- package/docs/safety-index/artifacts/kubernetes-server.md +44 -18
- package/docs/safety-index/artifacts/memory-server.json +39 -21
- package/docs/safety-index/artifacts/memory-server.md +24 -11
- package/docs/safety-index/artifacts/opentofu-server.json +38 -20
- package/docs/safety-index/artifacts/opentofu-server.md +24 -11
- package/docs/safety-index/artifacts/playwright-mcp-server.json +140 -20
- package/docs/safety-index/artifacts/playwright-mcp-server.md +26 -13
- package/docs/safety-index/artifacts/promptopia-server.json +41 -23
- package/docs/safety-index/artifacts/promptopia-server.md +26 -13
- package/docs/safety-index/artifacts/puppeteer-server.json +77 -34
- package/docs/safety-index/artifacts/puppeteer-server.md +45 -32
- package/docs/safety-index/artifacts/ref-tools-server.json +38 -20
- package/docs/safety-index/artifacts/ref-tools-server.md +22 -9
- package/docs/safety-index/artifacts/sequential-thinking-server.json +39 -21
- package/docs/safety-index/artifacts/sequential-thinking-server.md +23 -10
- package/docs/security-due-diligence.md +87 -0
- package/docs/setup-ci-doctor.md +11 -1
- package/docs/tool-call-receipts.md +1 -2
- package/examples/insecure-mcp-server/mcp-observatory.target.json +13 -0
- package/examples/insecure-mcp-server/server.mjs +62 -0
- package/package.json +12 -1
- package/scripts/postinstall.mjs +22 -7
package/docs/feishu-lark-mcp.md
CHANGED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Government And Enterprise MCP Readiness Pilot
|
|
2
|
+
|
|
3
|
+
MCP Observatory is a security release gate for MCP servers before deployment into sensitive, regulated, or mission-critical agentic AI environments.
|
|
4
|
+
|
|
5
|
+
## Problem Statement
|
|
6
|
+
|
|
7
|
+
MCP servers are a new security boundary. They expose tools, schemas, prompts, resources, filesystem access, browser automation, command execution, cloud APIs, and internal data paths directly to AI agents. Traditional dependency review catches package risk; it usually does not answer whether an agent-facing tool surface is safe to deploy.
|
|
8
|
+
|
|
9
|
+
Security teams need reproducible evidence before MCP servers become production dependencies.
|
|
10
|
+
|
|
11
|
+
## What The Audit Produces
|
|
12
|
+
|
|
13
|
+
An MCP Observatory pilot produces:
|
|
14
|
+
|
|
15
|
+
- profile-mapped audit reports using `--profile nsa-mcp`
|
|
16
|
+
- normalized findings with severity, confidence, evidence, recommendations, fingerprints, and control mappings
|
|
17
|
+
- SARIF output for GitHub Code Scanning
|
|
18
|
+
- trust status output: `enterprise_ready`, `scanned`, `needs_review`, `high_risk`, or `critical_risk`
|
|
19
|
+
- CI workflow examples for pull requests, scheduled checks, and security review artifacts
|
|
20
|
+
- remediation tracking across repeated scans
|
|
21
|
+
|
|
22
|
+
The `nsa-mcp` profile is not an official certification or government authorization. It operationalizes public security guidance into practical automated MCP checks for sensitive environments.
|
|
23
|
+
|
|
24
|
+
For buyer review, pair this pilot brief with the [public guidance crosswalk](./public-guidance-crosswalk.md), [procurement one-pager](./procurement-one-pager.md), and [security due diligence packet](./security-due-diligence.md).
|
|
25
|
+
|
|
26
|
+
## 30 Day Pilot
|
|
27
|
+
|
|
28
|
+
A practical 30 day pilot measures whether MCP Observatory can become the release gate for MCP server adoption.
|
|
29
|
+
|
|
30
|
+
Week 1:
|
|
31
|
+
- inventory MCP servers and target startup commands
|
|
32
|
+
- run baseline audits
|
|
33
|
+
- upload SARIF to GitHub Code Scanning for selected repos
|
|
34
|
+
|
|
35
|
+
Week 2:
|
|
36
|
+
- triage critical and high findings with security reviewers
|
|
37
|
+
- tune target configs and suppressions only where risk is accepted
|
|
38
|
+
- enable pull request audit checks for pilot repos
|
|
39
|
+
|
|
40
|
+
Week 3:
|
|
41
|
+
- remediate top findings
|
|
42
|
+
- rerun audits and compare trust status movement
|
|
43
|
+
- add scheduled weekly checks
|
|
44
|
+
|
|
45
|
+
Week 4:
|
|
46
|
+
- produce pilot summary with findings, false-positive feedback, remediation rate, and CI adoption status
|
|
47
|
+
- decide whether to expand to more repos or private fleet reporting
|
|
48
|
+
|
|
49
|
+
## Success Metrics
|
|
50
|
+
|
|
51
|
+
- number of MCP servers scanned
|
|
52
|
+
- critical and high findings discovered
|
|
53
|
+
- mean time to remediation
|
|
54
|
+
- percentage of findings fixed
|
|
55
|
+
- number of repos with CI gate enabled
|
|
56
|
+
- number of SARIF findings uploaded to GitHub Code Scanning
|
|
57
|
+
- false positive rate from security reviewer feedback
|
|
58
|
+
|
|
59
|
+
## Required Environment
|
|
60
|
+
|
|
61
|
+
The pilot needs:
|
|
62
|
+
|
|
63
|
+
- Node.js 20+
|
|
64
|
+
- reproducible MCP server startup commands or target JSON configs
|
|
65
|
+
- GitHub repository access for CI and SARIF upload when Code Scanning is in scope
|
|
66
|
+
- harmless fixtures for any tool invocation checks
|
|
67
|
+
- security reviewer feedback for false-positive classification
|
|
68
|
+
|
|
69
|
+
## Data Boundary
|
|
70
|
+
|
|
71
|
+
Local audit runs do not require hosted services.
|
|
72
|
+
|
|
73
|
+
By default, MCP Observatory should not collect secrets, environment dumps, file contents, command outputs, or private telemetry payloads. Audit artifacts should redact secret values and include only the evidence needed for security review.
|
|
74
|
+
|
|
75
|
+
If a hosted pilot is used, customer data boundaries should be documented before upload. The default local workflow can keep audit reports, SARIF, and score JSON inside the customer repository or CI artifact store.
|
|
76
|
+
|
|
77
|
+
## Workflow Fit
|
|
78
|
+
|
|
79
|
+
MCP Observatory fits existing security review surfaces:
|
|
80
|
+
|
|
81
|
+
- GitHub Actions runs the audit on pull requests and schedules
|
|
82
|
+
- SARIF uploads normalized findings to GitHub Code Scanning
|
|
83
|
+
- Markdown reports are published as CI artifacts for reviewer signoff
|
|
84
|
+
- score JSON can drive badges, dashboards, or release policy
|
|
85
|
+
- repeated audits measure remediation and drift
|
|
86
|
+
|
|
87
|
+
Start with:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md
|
|
91
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
92
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
93
|
+
```
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# MCP Receipts
|
|
2
|
+
|
|
3
|
+
A scan finds facts.
|
|
4
|
+
|
|
5
|
+
A report explains facts.
|
|
6
|
+
|
|
7
|
+
A receipt makes facts portable, reproducible, citable, and actionable.
|
|
8
|
+
|
|
9
|
+
MCP Observatory receipts are the canonical public trust artifact for MCP servers. A receipt records what was checked, when it was checked, which MCP Observatory version and profile were used, what evidence supports the verdict, what action a downstream system should take, and how another maintainer, agent, CI gate, registry, or buyer can rerun the same evaluation.
|
|
10
|
+
|
|
11
|
+
## What A Receipt Contains
|
|
12
|
+
|
|
13
|
+
Each receipt has eight stable sections:
|
|
14
|
+
|
|
15
|
+
- `subject`: server identity, startup command, package/repo/source, version, resolved identity, URL/path, commit, and package manager when available
|
|
16
|
+
- `run_context`: timestamp, MCP Observatory version, profile, safe-mode statement, environment class, invoked command, config file, OS, and safe working-directory context
|
|
17
|
+
- `evidence`: report paths and SHA-256 hashes, plus schema, tool surface, attack-simulation, and baseline/drift summaries
|
|
18
|
+
- `verdict`: score, trust status, receipt state, receipt action, and reason
|
|
19
|
+
- `findings`: the top findings only, with severity, control area, evidence summary, recommended fix, CI blocking status, and fingerprint
|
|
20
|
+
- `reproduction`: exact rerun command, CI command, SARIF upload hint, and expected artifacts
|
|
21
|
+
- `maintainer_cta`: claim/update receipt, add CI, fix findings, request rerun, and provide safe startup mode
|
|
22
|
+
- `buyer_cta`: request private fleet receipt pack, attack simulation evidence pack, CI/SARIF rollout, or government/enterprise pilot
|
|
23
|
+
|
|
24
|
+
The full audit report remains the place for every finding. The receipt is the compact trust record people and agents can cite.
|
|
25
|
+
|
|
26
|
+
## Verdict And Action Mapping
|
|
27
|
+
|
|
28
|
+
Receipts include both a human-readable state and an action receipt.
|
|
29
|
+
|
|
30
|
+
States:
|
|
31
|
+
|
|
32
|
+
- `ready_for_ci`
|
|
33
|
+
- `needs_review`
|
|
34
|
+
- `blocked`
|
|
35
|
+
- `could_not_evaluate`
|
|
36
|
+
|
|
37
|
+
Actions:
|
|
38
|
+
|
|
39
|
+
- `allow`
|
|
40
|
+
- `gate`
|
|
41
|
+
- `rerun`
|
|
42
|
+
- `quarantine`
|
|
43
|
+
- `escalate`
|
|
44
|
+
|
|
45
|
+
Mapping:
|
|
46
|
+
|
|
47
|
+
| Condition | State | Action |
|
|
48
|
+
|---|---|---|
|
|
49
|
+
| Critical findings | `blocked` | `escalate` |
|
|
50
|
+
| High findings | `blocked` | `gate` |
|
|
51
|
+
| Medium findings | `needs_review` | `gate` |
|
|
52
|
+
| Low/info findings only | `ready_for_ci` | `allow` |
|
|
53
|
+
| Evaluation error | `could_not_evaluate` | `rerun` |
|
|
54
|
+
|
|
55
|
+
Trust statuses remain compatible with audit output:
|
|
56
|
+
|
|
57
|
+
- `enterprise_ready`
|
|
58
|
+
- `scanned`
|
|
59
|
+
- `needs_review`
|
|
60
|
+
- `high_risk`
|
|
61
|
+
- `critical_risk`
|
|
62
|
+
|
|
63
|
+
## Emit A Receipt From Audit
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format json --output report.json --receipt receipt.json
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Markdown receipt:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output report.md --receipt receipt.md
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Standalone Receipt Command
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
mcp-observatory receipt npx -y my-mcp-server --profile nsa-mcp --format json --output receipt.json
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Public Safety Index receipt:
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
mcp-observatory receipt npx -y my-mcp-server --profile nsa-mcp --environment-class public_safety_index --format markdown --output receipt.md
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Supported `environment_class` values:
|
|
88
|
+
|
|
89
|
+
- `local`
|
|
90
|
+
- `ci`
|
|
91
|
+
- `public_safety_index`
|
|
92
|
+
- `private_fleet`
|
|
93
|
+
|
|
94
|
+
## Safe-Mode Guarantee
|
|
95
|
+
|
|
96
|
+
Receipts inherit the same safe-mode posture as the audit and attack-simulation flow. MCP Observatory inspects metadata, schemas, startup behavior, and inert attack-readiness evidence. It does not execute destructive payloads, exfiltrate secrets, or contact attacker-controlled callbacks.
|
|
97
|
+
|
|
98
|
+
Receipts also avoid leaking secrets or sensitive local paths. Working directories under the current user home directory are omitted, and private telemetry is not included.
|
|
99
|
+
|
|
100
|
+
## Why Receipts Matter
|
|
101
|
+
|
|
102
|
+
Receipts are meant to become the object that directories, agents, CI gates, maintainers, and buyers can pass around:
|
|
103
|
+
|
|
104
|
+
- a directory can link to the current receipt for a public MCP server
|
|
105
|
+
- a maintainer can claim the receipt and add CI
|
|
106
|
+
- an agent can decide whether to `allow`, `gate`, `rerun`, `quarantine`, or `escalate`
|
|
107
|
+
- a buyer can ask for a private fleet receipt pack before approving internal MCP dependencies
|
|
108
|
+
|
|
109
|
+
The public receipt proves the method. The private receipt pack turns it into a buyer-ready decision record.
|
|
@@ -1,41 +1,41 @@
|
|
|
1
|
-
# MCP Server Safety Index
|
|
1
|
+
# MCP Server Safety Index v1
|
|
2
2
|
|
|
3
3
|
The MCP Server Safety Index is an evidence standard for MCP readiness. It is not a leaderboard and does not rank maintainers.
|
|
4
4
|
|
|
5
|
-
Each row links to a reproducible command, a JSON run artifact, and a Markdown report generated by MCP Observatory.
|
|
5
|
+
Each row links to a reproducible command, an action receipt, a JSON run artifact, and a Markdown report generated by MCP Observatory. The goal is to show which failure classes matter before teams let agents depend on MCP servers.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
The safe attack simulation evidence checks metadata, schemas, and controlled canary surfaces only. It does not execute destructive payloads, contact external callback URLs, or claim exploitability from a static finding alone.
|
|
8
8
|
|
|
9
|
-
For the rules behind this page, see the [Safety Methodology](./methodology.md). For a smaller server-to-evidence map, see the [MCP Receipt Graph](./receipt-graph.md).
|
|
9
|
+
For the rules behind this page, see the [Safety Methodology](./methodology.md) and the [MCP Attack Simulator](./mcp-attack-simulator.md). For a smaller server-to-evidence map, see the [MCP Receipt Graph](./receipt-graph.md).
|
|
10
10
|
|
|
11
11
|
## Snapshot
|
|
12
12
|
|
|
13
13
|
- Evaluated servers: 15
|
|
14
|
-
- Ready for CI:
|
|
15
|
-
- Needs review before production:
|
|
16
|
-
- Unsafe default posture:
|
|
14
|
+
- Ready for CI: 7
|
|
15
|
+
- Needs review before production: 4
|
|
16
|
+
- Unsafe default posture: 4
|
|
17
17
|
- Not reproducible: 0
|
|
18
|
-
- Latest run: 2026-07-
|
|
18
|
+
- Latest run: 2026-07-06T19:49:58.878Z
|
|
19
19
|
|
|
20
20
|
## Evaluations
|
|
21
21
|
|
|
22
|
-
| # | Server | Category | Verdict | Failure Class | Reproduce | Evidence | Notes |
|
|
23
|
-
| ---: | --- | --- | --- | --- | --- | --- | --- |
|
|
24
|
-
| 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
|
|
25
|
-
| 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
|
|
26
|
-
| 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
|
|
27
|
-
| 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
|
|
28
|
-
| 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **
|
|
29
|
-
| 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | Prompt/resource contract | `npx -y promptopia-mcp` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
|
|
30
|
-
| 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | Prompt/tool inventory | `npx -y ref-tools-mcp` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
|
|
31
|
-
| 8 | [Git MCP Server](https://www.npmjs.com/package/@cyanheads/git-mcp-server) | Developer Tools | **Needs review before production** | Git tool boundary | `npx -y @cyanheads/git-mcp-server` | [JSON](./safety-index/artifacts/cyanheads-git-mcp-server.json) / [report](./safety-index/artifacts/cyanheads-git-mcp-server.md) | Zero-config public package. Keep tool invocation read-only unless a harmless fixture repository is added. |
|
|
32
|
-
| 9 | [OpenTofu MCP server](https://github.com/opentofu/opentofu-mcp-server) | Infrastructure | **
|
|
33
|
-
| 10 | [Puppeteer MCP server](https://www.npmjs.com/package/puppeteer-mcp-server) | Browser Automation | **
|
|
34
|
-
| 11 | [BrowserMCP](https://github.com/BrowserMCP/mcp) | Browser Automation | **Ready for CI** | Browser-control boundary | `npx -y @browsermcp/mcp` | [JSON](./safety-index/artifacts/browsermcp-server.json) / [report](./safety-index/artifacts/browsermcp-server.md) | Zero-config public package. [public proof](https://github.com/BrowserMCP/mcp/pull/189) |
|
|
35
|
-
| 12 | [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) | Browser Automation | **Unsafe default posture** | Browser/code execution boundary | `npx -y @playwright/mcp` | [JSON](./safety-index/artifacts/playwright-mcp-server.json) / [report](./safety-index/artifacts/playwright-mcp-server.md) | Zero-config public package; security findings represent policy-review prompts, not a vulnerability claim. |
|
|
36
|
-
| 13 | [AntV chart MCP server](https://github.com/antvis/mcp-server-chart) | Visualization | **Ready for CI** | Artifact-producing tools | `npx -y @antv/mcp-server-chart` | [JSON](./safety-index/artifacts/antv-chart-server.json) / [report](./safety-index/artifacts/antv-chart-server.md) | Zero-config public package. [public proof](https://github.com/antvis/mcp-server-chart/pull/312) |
|
|
37
|
-
| 14 | [ExecuteAutomation Playwright MCP](https://github.com/executeautomation/mcp-playwright) | Browser Automation | **Unsafe default posture** | Startup/listing reproducibility | `npx -y @executeautomation/playwright-mcp-server` | [JSON](./safety-index/artifacts/executeautomation-playwright-server.json) / [report](./safety-index/artifacts/executeautomation-playwright-server.md) | Evaluated as a public package; current result should be treated as a maintainer conversation starter. [public proof](https://github.com/executeautomation/mcp-playwright/pull/225) |
|
|
38
|
-
| 15 | [Flux159 Kubernetes MCP server](https://github.com/Flux159/mcp-server-kubernetes) | Infrastructure / Kubernetes | **
|
|
22
|
+
| # | Server | Category | Verdict | Action Receipt | Failure Class | Reproduce | CI Gate | Evidence | Notes |
|
|
23
|
+
| ---: | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
|
24
|
+
| 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | `allow` | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-everything" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
|
|
25
|
+
| 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | `allow` | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-sequential-thinking" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
|
|
26
|
+
| 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | `allow` | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-memory" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
|
|
27
|
+
| 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | `gate` | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
|
|
28
|
+
| 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **Needs review before production** | `gate` | Prompt-injection-sensitive retrieval | `npx -y @upstash/context7-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @upstash/context7-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/context7-server.json) / [report](./safety-index/artifacts/context7-server.md) | Zero-config public package. |
|
|
29
|
+
| 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | `allow` | Prompt/resource contract | `npx -y promptopia-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y promptopia-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
|
|
30
|
+
| 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | `allow` | Prompt/tool inventory | `npx -y ref-tools-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y ref-tools-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
|
|
31
|
+
| 8 | [Git MCP Server](https://www.npmjs.com/package/@cyanheads/git-mcp-server) | Developer Tools | **Needs review before production** | `gate` | Git tool boundary | `npx -y @cyanheads/git-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @cyanheads/git-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/cyanheads-git-mcp-server.json) / [report](./safety-index/artifacts/cyanheads-git-mcp-server.md) | Zero-config public package. Keep tool invocation read-only unless a harmless fixture repository is added. |
|
|
32
|
+
| 9 | [OpenTofu MCP server](https://github.com/opentofu/opentofu-mcp-server) | Infrastructure | **Needs review before production** | `gate` | Infrastructure tool surface | `npx -y @opentofu/opentofu-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @opentofu/opentofu-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/opentofu-server.json) / [report](./safety-index/artifacts/opentofu-server.md) | Zero-config public package. |
|
|
33
|
+
| 10 | [Puppeteer MCP server](https://www.npmjs.com/package/puppeteer-mcp-server) | Browser Automation | **Unsafe default posture** | `quarantine` | Browser/code execution boundary | `npx -y puppeteer-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y puppeteer-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/puppeteer-server.json) / [report](./safety-index/artifacts/puppeteer-server.md) | Intentional browser evaluation is suppressed so remaining findings stay readable. |
|
|
34
|
+
| 11 | [BrowserMCP](https://github.com/BrowserMCP/mcp) | Browser Automation | **Ready for CI** | `allow` | Browser-control boundary | `npx -y @browsermcp/mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @browsermcp/mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/browsermcp-server.json) / [report](./safety-index/artifacts/browsermcp-server.md) | Zero-config public package. [public proof](https://github.com/BrowserMCP/mcp/pull/189) |
|
|
35
|
+
| 12 | [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) | Browser Automation | **Unsafe default posture** | `quarantine` | Browser/code execution boundary | `npx -y @playwright/mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @playwright/mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/playwright-mcp-server.json) / [report](./safety-index/artifacts/playwright-mcp-server.md) | Zero-config public package; security findings represent policy-review prompts, not a vulnerability claim. |
|
|
36
|
+
| 13 | [AntV chart MCP server](https://github.com/antvis/mcp-server-chart) | Visualization | **Ready for CI** | `allow` | Artifact-producing tools | `npx -y @antv/mcp-server-chart` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @antv/mcp-server-chart" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/antv-chart-server.json) / [report](./safety-index/artifacts/antv-chart-server.md) | Zero-config public package. [public proof](https://github.com/antvis/mcp-server-chart/pull/312) |
|
|
37
|
+
| 14 | [ExecuteAutomation Playwright MCP](https://github.com/executeautomation/mcp-playwright) | Browser Automation | **Unsafe default posture** | `quarantine` | Startup/listing reproducibility | `npx -y @executeautomation/playwright-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @executeautomation/playwright-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/executeautomation-playwright-server.json) / [report](./safety-index/artifacts/executeautomation-playwright-server.md) | Evaluated as a public package; current result should be treated as a maintainer conversation starter. [public proof](https://github.com/executeautomation/mcp-playwright/pull/225) |
|
|
38
|
+
| 15 | [Flux159 Kubernetes MCP server](https://github.com/Flux159/mcp-server-kubernetes) | Infrastructure / Kubernetes | **Unsafe default posture** | `quarantine` | Command and cluster mutation boundary | `npx -y mcp-server-kubernetes` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y mcp-server-kubernetes" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/kubernetes-server.json) / [report](./safety-index/artifacts/kubernetes-server.md) | Zero-config public package. Evaluated without cluster credentials; findings are metadata and schema boundary prompts, not exploit claims. |
|
|
39
39
|
|
|
40
40
|
## Patterns Observed
|
|
41
41
|
|
|
@@ -57,6 +57,7 @@ For the rules behind this page, see the [Safety Methodology](./methodology.md).
|
|
|
57
57
|
## Publication Rules
|
|
58
58
|
|
|
59
59
|
- Use only public repositories, public package commands, public PRs, and generated sanitized artifacts.
|
|
60
|
+
- Include the action receipt and CI gate command so maintainers can rerun the evidence.
|
|
60
61
|
- Treat findings as reproducible evidence, not public shaming.
|
|
61
62
|
- Prefer “needs review” language unless there is clear artifact-backed proof of a dangerous default.
|
|
62
63
|
- Keep raw telemetry, emails, hostnames, private URLs, tokens, and customer claims out of public materials.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# NSA-MCP Audit CI Guide
|
|
2
|
+
|
|
3
|
+
Use this workflow when a repository needs MCP Observatory as a security release gate for an MCP server.
|
|
4
|
+
|
|
5
|
+
The `nsa-mcp` profile is not an official certification. It maps MCP Observatory findings to practical control areas for sensitive environments.
|
|
6
|
+
|
|
7
|
+
## Pull Request Audit
|
|
8
|
+
|
|
9
|
+
```yaml
|
|
10
|
+
name: MCP Observatory Audit
|
|
11
|
+
|
|
12
|
+
on:
|
|
13
|
+
pull_request:
|
|
14
|
+
push:
|
|
15
|
+
branches: [main]
|
|
16
|
+
schedule:
|
|
17
|
+
- cron: "0 9 * * 1"
|
|
18
|
+
|
|
19
|
+
permissions:
|
|
20
|
+
contents: read
|
|
21
|
+
security-events: write
|
|
22
|
+
|
|
23
|
+
jobs:
|
|
24
|
+
audit:
|
|
25
|
+
runs-on: ubuntu-latest
|
|
26
|
+
steps:
|
|
27
|
+
- uses: actions/checkout@v6
|
|
28
|
+
- uses: actions/setup-node@v5
|
|
29
|
+
with:
|
|
30
|
+
node-version: 24
|
|
31
|
+
- run: npm ci
|
|
32
|
+
- run: npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-high
|
|
33
|
+
- run: npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
34
|
+
- uses: github/codeql-action/upload-sarif@v4
|
|
35
|
+
if: always()
|
|
36
|
+
with:
|
|
37
|
+
sarif_file: mcp-audit.sarif
|
|
38
|
+
- uses: actions/upload-artifact@v4
|
|
39
|
+
if: always()
|
|
40
|
+
with:
|
|
41
|
+
name: mcp-observatory-audit
|
|
42
|
+
path: |
|
|
43
|
+
mcp-audit.md
|
|
44
|
+
mcp-audit.sarif
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Failing Policy
|
|
48
|
+
|
|
49
|
+
Use one of these gates:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-critical
|
|
53
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-high
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`--fail-on-critical` is appropriate while onboarding noisy repos. `--fail-on-high` is appropriate once maintainers have accepted the release gate.
|
|
57
|
+
|
|
58
|
+
## Badge Ready Score
|
|
59
|
+
|
|
60
|
+
The score command returns compact JSON for badge generators or dashboards:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Example:
|
|
67
|
+
|
|
68
|
+
```json
|
|
69
|
+
{
|
|
70
|
+
"target_id": "my-mcp-server",
|
|
71
|
+
"profile": "nsa-mcp",
|
|
72
|
+
"score": 87,
|
|
73
|
+
"status": "needs_review",
|
|
74
|
+
"finding_count": 2
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Local Fixture Demo
|
|
79
|
+
|
|
80
|
+
This repository includes an intentionally insecure inert fixture:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
npx tsx src/cli.ts audit examples/insecure-mcp-server --profile nsa-mcp --format markdown --output /tmp/nsa-mcp-audit-report.md
|
|
84
|
+
npx tsx src/cli.ts audit examples/insecure-mcp-server --profile nsa-mcp --format sarif --output /tmp/nsa-mcp-results.sarif
|
|
85
|
+
npx tsx src/cli.ts score examples/insecure-mcp-server --profile nsa-mcp --format json
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Generated examples are checked in under [`docs/examples`](./examples/).
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# MCP Observatory Procurement One-Pager
|
|
2
|
+
|
|
3
|
+
## Positioning
|
|
4
|
+
|
|
5
|
+
MCP Observatory is an open source security release gate for MCP servers before deployment into sensitive, regulated, or mission-critical agentic AI environments.
|
|
6
|
+
|
|
7
|
+
It helps platform and security teams answer a concrete procurement question:
|
|
8
|
+
|
|
9
|
+
> Can this MCP server be safely introduced into an agent workflow, and what evidence supports that decision?
|
|
10
|
+
|
|
11
|
+
## What It Does
|
|
12
|
+
|
|
13
|
+
- runs MCP server startup, conformance, schema, security, and safe attack-readiness checks
|
|
14
|
+
- maps findings to the `nsa-mcp` public-guidance profile
|
|
15
|
+
- produces Markdown audit reports for reviewers
|
|
16
|
+
- emits SARIF for GitHub Code Scanning
|
|
17
|
+
- emits compact trust score JSON for CI gates, badges, or dashboards
|
|
18
|
+
- supports recurring GitHub Actions checks and remediation tracking
|
|
19
|
+
|
|
20
|
+
## Buyer Value
|
|
21
|
+
|
|
22
|
+
| Buyer concern | MCP Observatory answer |
|
|
23
|
+
|---|---|
|
|
24
|
+
| Tool boundary risk | Detects shell, filesystem, browser, network, secrets, and destructive tool surfaces. |
|
|
25
|
+
| Agent manipulation risk | Flags hidden instructions and unsafe autonomous behavior in tool metadata. |
|
|
26
|
+
| Security review workflow | Produces SARIF, Markdown, JSON, and CI artifacts. |
|
|
27
|
+
| Procurement evidence | Provides a public-guidance crosswalk and repeatable audit commands. |
|
|
28
|
+
| Adoption risk | Runs locally with no hosted dependency required. |
|
|
29
|
+
| Remediation tracking | Supports repeatable audits, CI gates, and future baseline diffing. |
|
|
30
|
+
|
|
31
|
+
## Pilot Package
|
|
32
|
+
|
|
33
|
+
A 30 day pilot should include:
|
|
34
|
+
|
|
35
|
+
- 5 to 25 MCP servers scanned
|
|
36
|
+
- baseline `nsa-mcp` audit reports
|
|
37
|
+
- SARIF uploaded to GitHub Code Scanning for selected repos
|
|
38
|
+
- CI gate enabled on at least 3 repos
|
|
39
|
+
- weekly scheduled audit runs
|
|
40
|
+
- reviewer false-positive feedback
|
|
41
|
+
- final remediation and adoption summary
|
|
42
|
+
|
|
43
|
+
## Example Commands
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md
|
|
47
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
48
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Security And Data Boundary
|
|
52
|
+
|
|
53
|
+
Local audits do not require sending artifacts to a hosted service. Reports can stay inside the customer repository, CI logs, artifact store, or security tooling.
|
|
54
|
+
|
|
55
|
+
MCP Observatory should not collect secrets, environment dumps, file contents, or raw command outputs for audit reporting. Secret-like values are reported by name or pattern with values redacted.
|
|
56
|
+
|
|
57
|
+
## Procurement Notes
|
|
58
|
+
|
|
59
|
+
- License: MIT
|
|
60
|
+
- Runtime: Node.js 20+
|
|
61
|
+
- Integration points: CLI, GitHub Actions, SARIF, Markdown, JSON
|
|
62
|
+
- Hosted service: optional, not required for local audit evidence
|
|
63
|
+
- Certification claim: none
|
|
64
|
+
- Government endorsement claim: none
|
|
65
|
+
- Best fit: AI platform, AppSec, DevSecOps, AI assurance, software supply chain, and agent runtime security teams
|
|
66
|
+
|
|
67
|
+
## Useful Links
|
|
68
|
+
|
|
69
|
+
- [Government and enterprise pilot brief](./government-enterprise-pilot.md)
|
|
70
|
+
- [Public guidance crosswalk](./public-guidance-crosswalk.md)
|
|
71
|
+
- [NSA-MCP audit CI guide](./nsa-mcp-audit-ci.md)
|
|
72
|
+
- [Example audit report](./examples/nsa-mcp-audit-report.md)
|
|
73
|
+
- [Example SARIF](./examples/nsa-mcp-results.sarif)
|
|
74
|
+
- [Example score JSON](./examples/nsa-mcp-score.json)
|
package/docs/proof.md
CHANGED
|
@@ -5,8 +5,8 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
|
|
|
5
5
|
## Current Public Surface
|
|
6
6
|
|
|
7
7
|
- npm package: `@kryptosai/mcp-observatory`
|
|
8
|
-
- GitHub Action: `KryptosAI/mcp-observatory/action@
|
|
9
|
-
- Latest release: `
|
|
8
|
+
- GitHub Action: `KryptosAI/mcp-observatory/action@v1.28.0`
|
|
9
|
+
- Latest release: `v1.28.0`
|
|
10
10
|
- CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, setup-ci, init-ci, ci-report, enterprise-report, score, badge, cloud
|
|
11
11
|
- Test suite: 348 passing tests across 45 test files as of July 1, 2026
|
|
12
12
|
- GitHub traffic snapshot: 745 clones and 232 unique cloners in the visible June 2026 traffic window
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Public Guidance Crosswalk For MCP Security Audits
|
|
2
|
+
|
|
3
|
+
MCP Observatory maps MCP server evidence to public security guidance so security teams can review agent tool dependencies with familiar language.
|
|
4
|
+
|
|
5
|
+
This crosswalk is not a certification, endorsement, authorization to operate, or claim of compliance with any government program. It is a practical mapping from MCP Observatory checks to public guidance themes.
|
|
6
|
+
|
|
7
|
+
## Sources
|
|
8
|
+
|
|
9
|
+
Primary public sources used for this crosswalk:
|
|
10
|
+
|
|
11
|
+
- NSA, **Model Context Protocol (MCP): Security Design Considerations for Organizations Adopting MCP** (May 2026)
|
|
12
|
+
- NSA and partners, **Deploying AI Systems Securely: Best Practices for Deploying Secure and Resilient AI Systems** (April 2024)
|
|
13
|
+
- NSA, NCSC-UK, CISA, and partners, **Guidelines for Secure AI System Development** (November 2023)
|
|
14
|
+
- CISA and partners, **Careful Adoption of Agentic AI Services** (May 2026)
|
|
15
|
+
- NIST, **AI Risk Management Framework 1.0** and **Generative AI Profile NIST AI 600-1**
|
|
16
|
+
- OMB, **M-25-21 Accelerating Federal Use of AI through Innovation, Governance, and Public Trust**
|
|
17
|
+
- OMB, **M-25-22 Driving Efficient Acquisition of Artificial Intelligence in Government**
|
|
18
|
+
|
|
19
|
+
## Crosswalk
|
|
20
|
+
|
|
21
|
+
| MCP Observatory control | What Observatory checks | Public guidance theme | Audit evidence |
|
|
22
|
+
|---|---|---|---|
|
|
23
|
+
| `trust_boundaries` | Whether a server exposes risky tools, prompt/resource surfaces, or agent-steering metadata without clear boundaries. | Agentic AI systems need explicit governance, accountability, and threat-model-aware deployment. | Tool inventory, attack simulation findings, profile trust status. |
|
|
24
|
+
| `tool_permissions` | Command, filesystem, browser, network, database, secrets, and broad destructive tool surfaces. | MCP servers create new execution and permission boundaries that should be constrained and reviewed. | Security findings, permission-boundary findings, schema evidence. |
|
|
25
|
+
| `tool_description_integrity` | Hidden instructions, unsafe autonomous behavior, exfiltration language, and tool poisoning patterns in metadata. | AI system components should be secure by design and resistant to manipulation across the lifecycle. | Attack simulator findings with evidence excerpts. |
|
|
26
|
+
| `authentication` | Missing authentication on HTTP targets and target-level auth configuration gaps. | AI services and integrations should enforce identity and access controls appropriate to deployment context. | Target config evidence, normalized finding `mcp-observatory/security/no-auth-http`. |
|
|
27
|
+
| `secrets_exposure` | Secret-like environment variables and credential-like tool output. | Sensitive data used by AI systems needs protection throughout collection, processing, storage, and operation. | Redacted env-var evidence, canary/credential findings, SARIF results. |
|
|
28
|
+
| `schema_validation` | Weak, missing, broad, or permissive schemas and contract drift. | AI system inputs and interfaces should be validated, monitored, and constrained. | Schema quality findings, lock/diff evidence, contract-drift findings. |
|
|
29
|
+
| `input_validation` | Arbitrary command/code/file/path parameters and unconstrained additional properties. | Secure development guidance emphasizes input validation and least privilege. | Security findings and SARIF rule metadata. |
|
|
30
|
+
| `auditability` | Whether audit logging, structured event output, and review artifacts are present. | Government and enterprise adoption needs traceability, monitoring, and evidence for risk management. | Markdown report, JSON report, SARIF, CI artifacts, score JSON. |
|
|
31
|
+
| `runtime_safety` | Startup reliability, conformance, safe tool invocation behavior, and high-risk runtime boundaries. | Secure deployment guidance emphasizes resilient operation, monitoring, incident response, and recovery. | Run artifacts, check statuses, fatal errors, performance evidence. |
|
|
32
|
+
| `supply_chain` | Added destructive tools, broadened schemas, removed required fields, and future dependency indicators. | AI systems inherit software supply-chain risk and need continuous review of updates. | Drift findings, lock files, repeatable CI gates. |
|
|
33
|
+
|
|
34
|
+
## Why This Matters For MCP
|
|
35
|
+
|
|
36
|
+
MCP servers are not just libraries. They are agent-facing tool endpoints. A server can expose command execution, filesystem access, browser automation, database operations, cloud APIs, or internal business workflows through tool descriptions and schemas. That makes MCP server review a practical release-gate problem.
|
|
37
|
+
|
|
38
|
+
MCP Observatory turns that review into:
|
|
39
|
+
|
|
40
|
+
- one command a maintainer can run locally
|
|
41
|
+
- normalized findings a security reviewer can triage
|
|
42
|
+
- SARIF results GitHub Code Scanning can display
|
|
43
|
+
- trust status JSON a CI gate or badge generator can consume
|
|
44
|
+
- repeatable evidence for remediation tracking
|
|
45
|
+
|
|
46
|
+
## Review Boundary
|
|
47
|
+
|
|
48
|
+
MCP Observatory does not prove semantic safety and does not replace human authorization decisions. A passing audit means the tested target produced reproducible evidence for the mapped checks under the tested command and profile.
|