@kryptosai/mcp-observatory 0.27.0 → 1.28.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/COMMERCIAL.md +28 -2
- package/CONTRIBUTORS.md +39 -0
- package/README.md +114 -22
- package/dist/src/action-receipt.d.ts +20 -0
- package/dist/src/action-receipt.js +85 -0
- package/dist/src/action-receipt.js.map +1 -0
- package/dist/src/adapters/http.js +1 -5
- package/dist/src/adapters/http.js.map +1 -1
- package/dist/src/audit.d.ts +64 -0
- package/dist/src/audit.js +359 -0
- package/dist/src/audit.js.map +1 -0
- package/dist/src/cassette.js +3 -2
- package/dist/src/cassette.js.map +1 -1
- package/dist/src/checks/attack-sim.d.ts +20 -0
- package/dist/src/checks/attack-sim.js +316 -0
- package/dist/src/checks/attack-sim.js.map +1 -0
- package/dist/src/ci-issue.js +4 -3
- package/dist/src/ci-issue.js.map +1 -1
- package/dist/src/cli.js +25 -4
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/attack-sim.d.ts +2 -0
- package/dist/src/commands/attack-sim.js +220 -0
- package/dist/src/commands/attack-sim.js.map +1 -0
- package/dist/src/commands/audit.d.ts +2 -0
- package/dist/src/commands/audit.js +132 -0
- package/dist/src/commands/audit.js.map +1 -0
- package/dist/src/commands/init-ci.d.ts +3 -0
- package/dist/src/commands/init-ci.js +78 -13
- package/dist/src/commands/init-ci.js.map +1 -1
- package/dist/src/commands/legacy.js +8 -1
- package/dist/src/commands/legacy.js.map +1 -1
- package/dist/src/commands/receipt.d.ts +2 -0
- package/dist/src/commands/receipt.js +113 -0
- package/dist/src/commands/receipt.js.map +1 -0
- package/dist/src/commands/scan.js +30 -6
- package/dist/src/commands/scan.js.map +1 -1
- package/dist/src/commands/score.js +63 -0
- package/dist/src/commands/score.js.map +1 -1
- package/dist/src/commands/serve.js +9 -1
- package/dist/src/commands/serve.js.map +1 -1
- package/dist/src/commands/setup-ci-conversion.d.ts +3 -1
- package/dist/src/commands/setup-ci-conversion.js +23 -8
- package/dist/src/commands/setup-ci-conversion.js.map +1 -1
- package/dist/src/commands/telemetry.js +20 -14
- package/dist/src/commands/telemetry.js.map +1 -1
- package/dist/src/commands/test.js +31 -2
- package/dist/src/commands/test.js.map +1 -1
- package/dist/src/findings.d.ts +1 -0
- package/dist/src/findings.js +42 -1
- package/dist/src/findings.js.map +1 -1
- package/dist/src/history.js +3 -2
- package/dist/src/history.js.map +1 -1
- package/dist/src/index.d.ts +4 -0
- package/dist/src/index.js +4 -0
- package/dist/src/index.js.map +1 -1
- package/dist/src/integrations/smithery.js +5 -1
- package/dist/src/integrations/smithery.js.map +1 -1
- package/dist/src/receipt.d.ts +100 -0
- package/dist/src/receipt.js +375 -0
- package/dist/src/receipt.js.map +1 -0
- package/dist/src/reporters/attack-sim.d.ts +2 -0
- package/dist/src/reporters/attack-sim.js +89 -0
- package/dist/src/reporters/attack-sim.js.map +1 -0
- package/dist/src/reporters/pr-comment.js +20 -6
- package/dist/src/reporters/pr-comment.js.map +1 -1
- package/dist/src/reporters/sarif.js +11 -1
- package/dist/src/reporters/sarif.js.map +1 -1
- package/dist/src/runner.d.ts +3 -0
- package/dist/src/runner.js +5 -0
- package/dist/src/runner.js.map +1 -1
- package/dist/src/score.js +1 -1
- package/dist/src/score.js.map +1 -1
- package/dist/src/security-profiles.d.ts +13 -0
- package/dist/src/security-profiles.js +59 -0
- package/dist/src/security-profiles.js.map +1 -0
- package/dist/src/storage.js +3 -2
- package/dist/src/storage.js.map +1 -1
- package/dist/src/telemetry.d.ts +10 -0
- package/dist/src/telemetry.js +17 -1
- package/dist/src/telemetry.js.map +1 -1
- package/dist/src/types.d.ts +1 -1
- package/dist/src/utils/files.d.ts +1 -0
- package/dist/src/utils/files.js +21 -0
- package/dist/src/utils/files.js.map +1 -0
- package/dist/src/utils/ids.js +18 -5
- package/dist/src/utils/ids.js.map +1 -1
- package/dist/src/utils/url.d.ts +2 -0
- package/dist/src/utils/url.js +15 -0
- package/dist/src/utils/url.js.map +1 -0
- package/dist/src/validate.js +1 -1
- package/dist/src/validate.js.map +1 -1
- package/docs/10x-agentic-growth-sprint.md +47 -0
- package/docs/agent-ecosystem-promotion-plan.md +244 -0
- package/docs/agent-tasks.md +95 -0
- package/docs/agentic-contributor-outreach.md +52 -0
- package/docs/automatic-ci-integration.md +33 -0
- package/docs/campaign-attribution.md +58 -0
- package/docs/code-scanning-demo.md +57 -0
- package/docs/commercial-boundary.md +83 -0
- package/docs/contributor-proof-cards/README.md +33 -0
- package/docs/contributor-recognition.md +92 -0
- package/docs/examples/nsa-mcp-audit-report.md +372 -0
- package/docs/examples/nsa-mcp-results.sarif +930 -0
- package/docs/examples/nsa-mcp-score.json +13 -0
- package/docs/feishu-lark-mcp.md +1 -1
- package/docs/github-code-scanning-for-mcp.md +1 -1
- package/docs/government-enterprise-pilot.md +93 -0
- package/docs/launch.md +64 -0
- package/docs/mcp-attack-simulator.md +57 -0
- package/docs/mcp-receipts.md +109 -0
- package/docs/mcp-server-safety-index.md +29 -22
- package/docs/nsa-mcp-audit-ci.md +88 -0
- package/docs/procurement-one-pager.md +74 -0
- package/docs/proof.md +5 -2
- package/docs/public-guidance-crosswalk.md +48 -0
- package/docs/receipt-graph.md +54 -0
- package/docs/safety-index/artifacts/antv-chart-server.json +40 -22
- package/docs/safety-index/artifacts/antv-chart-server.md +22 -9
- package/docs/safety-index/artifacts/browsermcp-server.json +36 -18
- package/docs/safety-index/artifacts/browsermcp-server.md +22 -9
- package/docs/safety-index/artifacts/context7-server.json +83 -26
- package/docs/safety-index/artifacts/context7-server.md +33 -20
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.json +2438 -0
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.md +176 -0
- package/docs/safety-index/artifacts/everything-server.json +150 -57
- package/docs/safety-index/artifacts/everything-server.md +36 -23
- package/docs/safety-index/artifacts/executeautomation-playwright-server.json +162 -19
- package/docs/safety-index/artifacts/executeautomation-playwright-server.md +25 -12
- package/docs/safety-index/artifacts/filesystem-server.json +221 -19
- package/docs/safety-index/artifacts/filesystem-server.md +24 -11
- package/docs/safety-index/artifacts/kubernetes-server.attack.json +1559 -0
- package/docs/safety-index/artifacts/kubernetes-server.attack.md +43 -0
- package/docs/safety-index/artifacts/kubernetes-server.attack.sarif +2857 -0
- package/docs/safety-index/artifacts/kubernetes-server.json +1573 -0
- package/docs/safety-index/artifacts/kubernetes-server.md +176 -0
- package/docs/safety-index/artifacts/memory-server.json +93 -32
- package/docs/safety-index/artifacts/memory-server.md +45 -25
- package/docs/safety-index/artifacts/opentofu-server.json +44 -18
- package/docs/safety-index/artifacts/opentofu-server.md +23 -10
- package/docs/safety-index/artifacts/playwright-mcp-server.json +177 -21
- package/docs/safety-index/artifacts/playwright-mcp-server.md +27 -14
- package/docs/safety-index/artifacts/promptopia-server.json +45 -19
- package/docs/safety-index/artifacts/promptopia-server.md +21 -8
- package/docs/safety-index/artifacts/puppeteer-server.json +77 -22
- package/docs/safety-index/artifacts/puppeteer-server.md +28 -15
- package/docs/safety-index/artifacts/ref-tools-server.json +36 -18
- package/docs/safety-index/artifacts/ref-tools-server.md +21 -8
- package/docs/safety-index/artifacts/sequential-thinking-server.json +38 -21
- package/docs/safety-index/artifacts/sequential-thinking-server.md +22 -9
- package/docs/safety-index/targets.json +90 -14
- package/docs/security-due-diligence.md +87 -0
- package/docs/setup-ci-doctor.md +13 -1
- package/docs/target-gallery.md +49 -0
- package/docs/target-registry.md +5 -1
- package/docs/tool-call-receipts.md +115 -0
- package/examples/insecure-mcp-server/mcp-observatory.target.json +13 -0
- package/examples/insecure-mcp-server/server.mjs +62 -0
- package/package.json +31 -2
- package/schemas/run-artifact.schema.json +1 -0
- package/scripts/postinstall.mjs +138 -0
package/docs/feishu-lark-mcp.md
CHANGED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Government And Enterprise MCP Readiness Pilot
|
|
2
|
+
|
|
3
|
+
MCP Observatory is a security release gate for MCP servers before deployment into sensitive, regulated, or mission-critical agentic AI environments.
|
|
4
|
+
|
|
5
|
+
## Problem Statement
|
|
6
|
+
|
|
7
|
+
MCP servers are a new security boundary. They expose tools, schemas, prompts, resources, filesystem access, browser automation, command execution, cloud APIs, and internal data paths directly to AI agents. Traditional dependency review catches package risk; it usually does not answer whether an agent-facing tool surface is safe to deploy.
|
|
8
|
+
|
|
9
|
+
Security teams need reproducible evidence before MCP servers become production dependencies.
|
|
10
|
+
|
|
11
|
+
## What The Audit Produces
|
|
12
|
+
|
|
13
|
+
An MCP Observatory pilot produces:
|
|
14
|
+
|
|
15
|
+
- profile-mapped audit reports using `--profile nsa-mcp`
|
|
16
|
+
- normalized findings with severity, confidence, evidence, recommendations, fingerprints, and control mappings
|
|
17
|
+
- SARIF output for GitHub Code Scanning
|
|
18
|
+
- trust status output: `enterprise_ready`, `scanned`, `needs_review`, `high_risk`, or `critical_risk`
|
|
19
|
+
- CI workflow examples for pull requests, scheduled checks, and security review artifacts
|
|
20
|
+
- remediation tracking across repeated scans
|
|
21
|
+
|
|
22
|
+
The `nsa-mcp` profile is not an official certification or government authorization. It operationalizes public security guidance into practical automated MCP checks for sensitive environments.
|
|
23
|
+
|
|
24
|
+
For buyer review, pair this pilot brief with the [public guidance crosswalk](./public-guidance-crosswalk.md), [procurement one-pager](./procurement-one-pager.md), and [security due diligence packet](./security-due-diligence.md).
|
|
25
|
+
|
|
26
|
+
## 30 Day Pilot
|
|
27
|
+
|
|
28
|
+
A practical 30 day pilot measures whether MCP Observatory can become the release gate for MCP server adoption.
|
|
29
|
+
|
|
30
|
+
Week 1:
|
|
31
|
+
- inventory MCP servers and target startup commands
|
|
32
|
+
- run baseline audits
|
|
33
|
+
- upload SARIF to GitHub Code Scanning for selected repos
|
|
34
|
+
|
|
35
|
+
Week 2:
|
|
36
|
+
- triage critical and high findings with security reviewers
|
|
37
|
+
- tune target configs and suppressions only where risk is accepted
|
|
38
|
+
- enable pull request audit checks for pilot repos
|
|
39
|
+
|
|
40
|
+
Week 3:
|
|
41
|
+
- remediate top findings
|
|
42
|
+
- rerun audits and compare trust status movement
|
|
43
|
+
- add scheduled weekly checks
|
|
44
|
+
|
|
45
|
+
Week 4:
|
|
46
|
+
- produce pilot summary with findings, false-positive feedback, remediation rate, and CI adoption status
|
|
47
|
+
- decide whether to expand to more repos or private fleet reporting
|
|
48
|
+
|
|
49
|
+
## Success Metrics
|
|
50
|
+
|
|
51
|
+
- number of MCP servers scanned
|
|
52
|
+
- critical and high findings discovered
|
|
53
|
+
- mean time to remediation
|
|
54
|
+
- percentage of findings fixed
|
|
55
|
+
- number of repos with CI gate enabled
|
|
56
|
+
- number of SARIF findings uploaded to GitHub Code Scanning
|
|
57
|
+
- false positive rate from security reviewer feedback
|
|
58
|
+
|
|
59
|
+
## Required Environment
|
|
60
|
+
|
|
61
|
+
The pilot needs:
|
|
62
|
+
|
|
63
|
+
- Node.js 20+
|
|
64
|
+
- reproducible MCP server startup commands or target JSON configs
|
|
65
|
+
- GitHub repository access for CI and SARIF upload when Code Scanning is in scope
|
|
66
|
+
- harmless fixtures for any tool invocation checks
|
|
67
|
+
- security reviewer feedback for false-positive classification
|
|
68
|
+
|
|
69
|
+
## Data Boundary
|
|
70
|
+
|
|
71
|
+
Local audit runs do not require hosted services.
|
|
72
|
+
|
|
73
|
+
By default, MCP Observatory should not collect secrets, environment dumps, file contents, command outputs, or private telemetry payloads. Audit artifacts should redact secret values and include only the evidence needed for security review.
|
|
74
|
+
|
|
75
|
+
If a hosted pilot is used, customer data boundaries should be documented before upload. The default local workflow can keep audit reports, SARIF, and score JSON inside the customer repository or CI artifact store.
|
|
76
|
+
|
|
77
|
+
## Workflow Fit
|
|
78
|
+
|
|
79
|
+
MCP Observatory fits existing security review surfaces:
|
|
80
|
+
|
|
81
|
+
- GitHub Actions runs the audit on pull requests and schedules
|
|
82
|
+
- SARIF uploads normalized findings to GitHub Code Scanning
|
|
83
|
+
- Markdown reports are published as CI artifacts for reviewer signoff
|
|
84
|
+
- score JSON can drive badges, dashboards, or release policy
|
|
85
|
+
- repeated audits measure remediation and drift
|
|
86
|
+
|
|
87
|
+
Start with:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md
|
|
91
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
92
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
93
|
+
```
|
package/docs/launch.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# MCP Observatory Launch Page
|
|
2
|
+
|
|
3
|
+
```
|
|
4
|
+
███╗ ███╗ ██████╗██████╗
|
|
5
|
+
████╗ ████║██╔════╝██╔══██╗
|
|
6
|
+
██╔████╔██║██║ ██████╔╝
|
|
7
|
+
██║╚██╔╝██║██║ ██╔═══╝
|
|
8
|
+
██║ ╚═╝ ██║╚██████╗██║
|
|
9
|
+
╚═╝ ╚═╝ ╚═════╝╚═╝
|
|
10
|
+
O B S E R V A T O R Y
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
MCP Observatory is the GitHub-native CI and security gate for MCP servers before agents depend on them.
|
|
14
|
+
|
|
15
|
+
It turns a local MCP check into evidence maintainers already understand: GitHub Actions, SARIF, Code Scanning findings, score badges, reproducible run artifacts, and maintainer-ready PR copy.
|
|
16
|
+
|
|
17
|
+
## The Fast Path
|
|
18
|
+
|
|
19
|
+
1. Test one MCP server locally.
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npx @kryptosai/mcp-observatory test npx -y my-mcp-server --campaign launch-proof
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
2. Convert the passing check into CI.
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif --campaign launch-proof
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
3. Verify the adoption kit.
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
npx @kryptosai/mcp-observatory setup-ci --doctor
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Why Maintainers Use It
|
|
38
|
+
|
|
39
|
+
- Catch MCP startup, tool listing, prompt, resource, schema, and drift failures before agents rely on a server.
|
|
40
|
+
- Convert a one-off local check into a repeatable GitHub Action without hand-writing workflow YAML.
|
|
41
|
+
- Emit normalized SARIF so MCP findings can appear in GitHub Code Scanning.
|
|
42
|
+
- Share reproducible JSON and Markdown evidence without exposing private telemetry.
|
|
43
|
+
- Let coding agents run Observatory as an MCP server and inspect other MCP tools autonomously.
|
|
44
|
+
|
|
45
|
+
## Public Proof
|
|
46
|
+
|
|
47
|
+
- [GitHub Code Scanning for MCP servers](./github-code-scanning-for-mcp.md)
|
|
48
|
+
- [Code Scanning demo](./code-scanning-demo.md)
|
|
49
|
+
- [MCP Server Safety Index](./mcp-server-safety-index.md)
|
|
50
|
+
- [Target gallery](./target-gallery.md)
|
|
51
|
+
- [Public proof log](./proof.md)
|
|
52
|
+
- [Agent task pack](./agent-tasks.md)
|
|
53
|
+
|
|
54
|
+
## For Teams Running MCP In Production
|
|
55
|
+
|
|
56
|
+
If MCP servers are becoming production dependencies, use the same evidence model privately:
|
|
57
|
+
|
|
58
|
+
- CI rollout for selected MCP servers
|
|
59
|
+
- SARIF and GitHub Code Scanning setup
|
|
60
|
+
- private readiness review
|
|
61
|
+
- prioritized fixes and owner-ready next steps
|
|
62
|
+
- Safety Index-style report for internal agent dependencies
|
|
63
|
+
|
|
64
|
+
See the [MCP Readiness Review](./paid-pilot-offer.md). The default package starts at `$2,500`.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# MCP Attack Simulator
|
|
2
|
+
|
|
3
|
+
MCP Attack Simulator runs safe, inert simulation checks over MCP server metadata, schemas, captured evidence, and optional baselines. It produces reproducible Safety Index evidence without executing destructive payloads.
|
|
4
|
+
|
|
5
|
+
## Quick Start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npx @kryptosai/mcp-observatory attack-sim --target ./target.json --output attack-report.md --json attack-artifact.json
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Or run directly against a server command:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
npx @kryptosai/mcp-observatory attack-sim npx -y my-mcp-server
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
For GitHub Code Scanning:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npx @kryptosai/mcp-observatory attack-sim \
|
|
21
|
+
--target ./target.json \
|
|
22
|
+
--json attack-artifact.json \
|
|
23
|
+
--sarif attack-results.sarif \
|
|
24
|
+
--fail-on-high
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## What It Tests
|
|
28
|
+
|
|
29
|
+
- **Tool poisoning:** suspicious hidden or behavioral instructions in tool, prompt, resource, and schema metadata.
|
|
30
|
+
- **Exfiltration canaries:** fake Observatory canaries and credential-like values in captured evidence.
|
|
31
|
+
- **Permission boundaries:** broad path, command, network, or payload parameters combined with destructive or non-read-only behavior.
|
|
32
|
+
- **Contract drift:** when `--baseline <run-artifact>` is supplied, new destructive tools, broadened schemas, and removed required fields.
|
|
33
|
+
|
|
34
|
+
## Safe Simulation Only
|
|
35
|
+
|
|
36
|
+
The simulator does not run exploit payloads, contact attacker infrastructure, write or delete files, or exfiltrate data. It uses metadata, schema, drift, and already-captured response evidence.
|
|
37
|
+
|
|
38
|
+
If safe tool invocation is needed, run normal Observatory checks with existing deep invocation behavior and review the captured artifact. The simulator inspects captured evidence; it does not invent new dangerous calls.
|
|
39
|
+
|
|
40
|
+
## Outputs
|
|
41
|
+
|
|
42
|
+
- JSON: a normal MCP Observatory run artifact with an `attack-sim` check.
|
|
43
|
+
- Markdown: an attack simulation report with verdict, findings, reproduction command, and maintainer fixes.
|
|
44
|
+
- SARIF: Code Scanning findings for high and medium risk results.
|
|
45
|
+
|
|
46
|
+
## Baseline Drift
|
|
47
|
+
|
|
48
|
+
Use a prior run artifact as the baseline:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
npx @kryptosai/mcp-observatory attack-sim \
|
|
52
|
+
--target ./target.json \
|
|
53
|
+
--baseline .mcp-observatory/runs/previous.json \
|
|
54
|
+
--output attack-report.md
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
This flags agent-contract changes that can turn a previously acceptable MCP dependency into a review item.
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# MCP Receipts
|
|
2
|
+
|
|
3
|
+
A scan finds facts.
|
|
4
|
+
|
|
5
|
+
A report explains facts.
|
|
6
|
+
|
|
7
|
+
A receipt makes facts portable, reproducible, citable, and actionable.
|
|
8
|
+
|
|
9
|
+
MCP Observatory receipts are the canonical public trust artifact for MCP servers. A receipt records what was checked, when it was checked, which MCP Observatory version and profile were used, what evidence supports the verdict, what action a downstream system should take, and how another maintainer, agent, CI gate, registry, or buyer can rerun the same evaluation.
|
|
10
|
+
|
|
11
|
+
## What A Receipt Contains
|
|
12
|
+
|
|
13
|
+
Each receipt has eight stable sections:
|
|
14
|
+
|
|
15
|
+
- `subject`: server identity, startup command, package/repo/source, version, resolved identity, URL/path, commit, and package manager when available
|
|
16
|
+
- `run_context`: timestamp, MCP Observatory version, profile, safe-mode statement, environment class, invoked command, config file, OS, and safe working-directory context
|
|
17
|
+
- `evidence`: report paths and SHA-256 hashes, plus schema, tool surface, attack-simulation, and baseline/drift summaries
|
|
18
|
+
- `verdict`: score, trust status, receipt state, receipt action, and reason
|
|
19
|
+
- `findings`: the top findings only, with severity, control area, evidence summary, recommended fix, CI blocking status, and fingerprint
|
|
20
|
+
- `reproduction`: exact rerun command, CI command, SARIF upload hint, and expected artifacts
|
|
21
|
+
- `maintainer_cta`: claim/update receipt, add CI, fix findings, request rerun, and provide safe startup mode
|
|
22
|
+
- `buyer_cta`: request private fleet receipt pack, attack simulation evidence pack, CI/SARIF rollout, or government/enterprise pilot
|
|
23
|
+
|
|
24
|
+
The full audit report remains the place for every finding. The receipt is the compact trust record people and agents can cite.
|
|
25
|
+
|
|
26
|
+
## Verdict And Action Mapping
|
|
27
|
+
|
|
28
|
+
Receipts include both a human-readable state and an action receipt.
|
|
29
|
+
|
|
30
|
+
States:
|
|
31
|
+
|
|
32
|
+
- `ready_for_ci`
|
|
33
|
+
- `needs_review`
|
|
34
|
+
- `blocked`
|
|
35
|
+
- `could_not_evaluate`
|
|
36
|
+
|
|
37
|
+
Actions:
|
|
38
|
+
|
|
39
|
+
- `allow`
|
|
40
|
+
- `gate`
|
|
41
|
+
- `rerun`
|
|
42
|
+
- `quarantine`
|
|
43
|
+
- `escalate`
|
|
44
|
+
|
|
45
|
+
Mapping:
|
|
46
|
+
|
|
47
|
+
| Condition | State | Action |
|
|
48
|
+
|---|---|---|
|
|
49
|
+
| Critical findings | `blocked` | `escalate` |
|
|
50
|
+
| High findings | `blocked` | `gate` |
|
|
51
|
+
| Medium findings | `needs_review` | `gate` |
|
|
52
|
+
| Low/info findings only | `ready_for_ci` | `allow` |
|
|
53
|
+
| Evaluation error | `could_not_evaluate` | `rerun` |
|
|
54
|
+
|
|
55
|
+
Trust statuses remain compatible with audit output:
|
|
56
|
+
|
|
57
|
+
- `enterprise_ready`
|
|
58
|
+
- `scanned`
|
|
59
|
+
- `needs_review`
|
|
60
|
+
- `high_risk`
|
|
61
|
+
- `critical_risk`
|
|
62
|
+
|
|
63
|
+
## Emit A Receipt From Audit
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format json --output report.json --receipt receipt.json
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Markdown receipt:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output report.md --receipt receipt.md
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Standalone Receipt Command
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
mcp-observatory receipt npx -y my-mcp-server --profile nsa-mcp --format json --output receipt.json
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Public Safety Index receipt:
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
mcp-observatory receipt npx -y my-mcp-server --profile nsa-mcp --environment-class public_safety_index --format markdown --output receipt.md
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Supported `environment_class` values:
|
|
88
|
+
|
|
89
|
+
- `local`
|
|
90
|
+
- `ci`
|
|
91
|
+
- `public_safety_index`
|
|
92
|
+
- `private_fleet`
|
|
93
|
+
|
|
94
|
+
## Safe-Mode Guarantee
|
|
95
|
+
|
|
96
|
+
Receipts inherit the same safe-mode posture as the audit and attack-simulation flow. MCP Observatory inspects metadata, schemas, startup behavior, and inert attack-readiness evidence. It does not execute destructive payloads, exfiltrate secrets, or contact attacker-controlled callbacks.
|
|
97
|
+
|
|
98
|
+
Receipts also avoid leaking secrets or sensitive local paths. Working directories under the current user home directory are omitted, and private telemetry is not included.
|
|
99
|
+
|
|
100
|
+
## Why Receipts Matter
|
|
101
|
+
|
|
102
|
+
Receipts are meant to become the object that directories, agents, CI gates, maintainers, and buyers can pass around:
|
|
103
|
+
|
|
104
|
+
- a directory can link to the current receipt for a public MCP server
|
|
105
|
+
- a maintainer can claim the receipt and add CI
|
|
106
|
+
- an agent can decide whether to `allow`, `gate`, `rerun`, `quarantine`, or `escalate`
|
|
107
|
+
- a buyer can ask for a private fleet receipt pack before approving internal MCP dependencies
|
|
108
|
+
|
|
109
|
+
The public receipt proves the method. The private receipt pack turns it into a buyer-ready decision record.
|
|
@@ -2,36 +2,40 @@
|
|
|
2
2
|
|
|
3
3
|
The MCP Server Safety Index is an evidence standard for MCP readiness. It is not a leaderboard and does not rank maintainers.
|
|
4
4
|
|
|
5
|
-
Each row links to a reproducible command, a JSON run artifact, and a Markdown report generated by MCP Observatory. The goal is to show which failure classes matter before teams let agents depend on MCP servers.
|
|
5
|
+
Each row links to a reproducible command, an action receipt, a JSON run artifact, and a Markdown report generated by MCP Observatory. The goal is to show which failure classes matter before teams let agents depend on MCP servers.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
The safe attack simulation evidence checks metadata, schemas, and controlled canary surfaces only. It does not execute destructive payloads, contact external callback URLs, or claim exploitability from a static finding alone.
|
|
8
|
+
|
|
9
|
+
For the rules behind this page, see the [Safety Methodology](./methodology.md) and the [MCP Attack Simulator](./mcp-attack-simulator.md). For a smaller server-to-evidence map, see the [MCP Receipt Graph](./receipt-graph.md).
|
|
8
10
|
|
|
9
11
|
## Snapshot
|
|
10
12
|
|
|
11
|
-
- Evaluated servers:
|
|
12
|
-
- Ready for CI:
|
|
13
|
-
- Needs review before production:
|
|
14
|
-
- Unsafe default posture:
|
|
13
|
+
- Evaluated servers: 15
|
|
14
|
+
- Ready for CI: 7
|
|
15
|
+
- Needs review before production: 4
|
|
16
|
+
- Unsafe default posture: 4
|
|
15
17
|
- Not reproducible: 0
|
|
16
|
-
- Latest run: 2026-
|
|
18
|
+
- Latest run: 2026-07-06T19:49:58.878Z
|
|
17
19
|
|
|
18
20
|
## Evaluations
|
|
19
21
|
|
|
20
|
-
| # | Server | Category | Verdict | Failure Class | Reproduce | Evidence | Notes |
|
|
21
|
-
| ---: | --- | --- | --- | --- | --- | --- | --- |
|
|
22
|
-
| 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
|
|
23
|
-
| 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
|
|
24
|
-
| 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
|
|
25
|
-
| 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
|
|
26
|
-
| 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **
|
|
27
|
-
| 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | Prompt/resource contract | `npx -y promptopia-mcp` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
|
|
28
|
-
| 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | Prompt/tool inventory | `npx -y ref-tools-mcp` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
|
|
29
|
-
| 8 | [
|
|
30
|
-
| 9 | [
|
|
31
|
-
| 10 | [
|
|
32
|
-
| 11 | [
|
|
33
|
-
| 12 | [
|
|
34
|
-
| 13 | [
|
|
22
|
+
| # | Server | Category | Verdict | Action Receipt | Failure Class | Reproduce | CI Gate | Evidence | Notes |
|
|
23
|
+
| ---: | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
|
24
|
+
| 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | `allow` | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-everything" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
|
|
25
|
+
| 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | `allow` | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-sequential-thinking" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
|
|
26
|
+
| 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | `allow` | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-memory" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
|
|
27
|
+
| 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | `gate` | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
|
|
28
|
+
| 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **Needs review before production** | `gate` | Prompt-injection-sensitive retrieval | `npx -y @upstash/context7-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @upstash/context7-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/context7-server.json) / [report](./safety-index/artifacts/context7-server.md) | Zero-config public package. |
|
|
29
|
+
| 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | `allow` | Prompt/resource contract | `npx -y promptopia-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y promptopia-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
|
|
30
|
+
| 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | `allow` | Prompt/tool inventory | `npx -y ref-tools-mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y ref-tools-mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
|
|
31
|
+
| 8 | [Git MCP Server](https://www.npmjs.com/package/@cyanheads/git-mcp-server) | Developer Tools | **Needs review before production** | `gate` | Git tool boundary | `npx -y @cyanheads/git-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @cyanheads/git-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/cyanheads-git-mcp-server.json) / [report](./safety-index/artifacts/cyanheads-git-mcp-server.md) | Zero-config public package. Keep tool invocation read-only unless a harmless fixture repository is added. |
|
|
32
|
+
| 9 | [OpenTofu MCP server](https://github.com/opentofu/opentofu-mcp-server) | Infrastructure | **Needs review before production** | `gate` | Infrastructure tool surface | `npx -y @opentofu/opentofu-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @opentofu/opentofu-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/opentofu-server.json) / [report](./safety-index/artifacts/opentofu-server.md) | Zero-config public package. |
|
|
33
|
+
| 10 | [Puppeteer MCP server](https://www.npmjs.com/package/puppeteer-mcp-server) | Browser Automation | **Unsafe default posture** | `quarantine` | Browser/code execution boundary | `npx -y puppeteer-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y puppeteer-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/puppeteer-server.json) / [report](./safety-index/artifacts/puppeteer-server.md) | Intentional browser evaluation is suppressed so remaining findings stay readable. |
|
|
34
|
+
| 11 | [BrowserMCP](https://github.com/BrowserMCP/mcp) | Browser Automation | **Ready for CI** | `allow` | Browser-control boundary | `npx -y @browsermcp/mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @browsermcp/mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/browsermcp-server.json) / [report](./safety-index/artifacts/browsermcp-server.md) | Zero-config public package. [public proof](https://github.com/BrowserMCP/mcp/pull/189) |
|
|
35
|
+
| 12 | [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) | Browser Automation | **Unsafe default posture** | `quarantine` | Browser/code execution boundary | `npx -y @playwright/mcp` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @playwright/mcp" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/playwright-mcp-server.json) / [report](./safety-index/artifacts/playwright-mcp-server.md) | Zero-config public package; security findings represent policy-review prompts, not a vulnerability claim. |
|
|
36
|
+
| 13 | [AntV chart MCP server](https://github.com/antvis/mcp-server-chart) | Visualization | **Ready for CI** | `allow` | Artifact-producing tools | `npx -y @antv/mcp-server-chart` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @antv/mcp-server-chart" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/antv-chart-server.json) / [report](./safety-index/artifacts/antv-chart-server.md) | Zero-config public package. [public proof](https://github.com/antvis/mcp-server-chart/pull/312) |
|
|
37
|
+
| 14 | [ExecuteAutomation Playwright MCP](https://github.com/executeautomation/mcp-playwright) | Browser Automation | **Unsafe default posture** | `quarantine` | Startup/listing reproducibility | `npx -y @executeautomation/playwright-mcp-server` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @executeautomation/playwright-mcp-server" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/executeautomation-playwright-server.json) / [report](./safety-index/artifacts/executeautomation-playwright-server.md) | Evaluated as a public package; current result should be treated as a maintainer conversation starter. [public proof](https://github.com/executeautomation/mcp-playwright/pull/225) |
|
|
38
|
+
| 15 | [Flux159 Kubernetes MCP server](https://github.com/Flux159/mcp-server-kubernetes) | Infrastructure / Kubernetes | **Unsafe default posture** | `quarantine` | Command and cluster mutation boundary | `npx -y mcp-server-kubernetes` | `npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y mcp-server-kubernetes" --sarif --schedule weekly` | [JSON](./safety-index/artifacts/kubernetes-server.json) / [report](./safety-index/artifacts/kubernetes-server.md) | Zero-config public package. Evaluated without cluster credentials; findings are metadata and schema boundary prompts, not exploit claims. |
|
|
35
39
|
|
|
36
40
|
## Patterns Observed
|
|
37
41
|
|
|
@@ -39,6 +43,8 @@ For the rules behind this page, see the [Safety Methodology](./methodology.md).
|
|
|
39
43
|
- Artifact-producing tools: 1 server(s)
|
|
40
44
|
- Broad protocol surface: 1 server(s)
|
|
41
45
|
- Browser-control boundary: 1 server(s)
|
|
46
|
+
- Command and cluster mutation boundary: 1 server(s)
|
|
47
|
+
- Git tool boundary: 1 server(s)
|
|
42
48
|
- Infrastructure tool surface: 1 server(s)
|
|
43
49
|
- Persistent state tools: 1 server(s)
|
|
44
50
|
- Prompt-injection-sensitive retrieval: 1 server(s)
|
|
@@ -51,6 +57,7 @@ For the rules behind this page, see the [Safety Methodology](./methodology.md).
|
|
|
51
57
|
## Publication Rules
|
|
52
58
|
|
|
53
59
|
- Use only public repositories, public package commands, public PRs, and generated sanitized artifacts.
|
|
60
|
+
- Include the action receipt and CI gate command so maintainers can rerun the evidence.
|
|
54
61
|
- Treat findings as reproducible evidence, not public shaming.
|
|
55
62
|
- Prefer “needs review” language unless there is clear artifact-backed proof of a dangerous default.
|
|
56
63
|
- Keep raw telemetry, emails, hostnames, private URLs, tokens, and customer claims out of public materials.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# NSA-MCP Audit CI Guide
|
|
2
|
+
|
|
3
|
+
Use this workflow when a repository needs MCP Observatory as a security release gate for an MCP server.
|
|
4
|
+
|
|
5
|
+
The `nsa-mcp` profile is not an official certification. It maps MCP Observatory findings to practical control areas for sensitive environments.
|
|
6
|
+
|
|
7
|
+
## Pull Request Audit
|
|
8
|
+
|
|
9
|
+
```yaml
|
|
10
|
+
name: MCP Observatory Audit
|
|
11
|
+
|
|
12
|
+
on:
|
|
13
|
+
pull_request:
|
|
14
|
+
push:
|
|
15
|
+
branches: [main]
|
|
16
|
+
schedule:
|
|
17
|
+
- cron: "0 9 * * 1"
|
|
18
|
+
|
|
19
|
+
permissions:
|
|
20
|
+
contents: read
|
|
21
|
+
security-events: write
|
|
22
|
+
|
|
23
|
+
jobs:
|
|
24
|
+
audit:
|
|
25
|
+
runs-on: ubuntu-latest
|
|
26
|
+
steps:
|
|
27
|
+
- uses: actions/checkout@v6
|
|
28
|
+
- uses: actions/setup-node@v5
|
|
29
|
+
with:
|
|
30
|
+
node-version: 24
|
|
31
|
+
- run: npm ci
|
|
32
|
+
- run: npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-high
|
|
33
|
+
- run: npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
34
|
+
- uses: github/codeql-action/upload-sarif@v4
|
|
35
|
+
if: always()
|
|
36
|
+
with:
|
|
37
|
+
sarif_file: mcp-audit.sarif
|
|
38
|
+
- uses: actions/upload-artifact@v4
|
|
39
|
+
if: always()
|
|
40
|
+
with:
|
|
41
|
+
name: mcp-observatory-audit
|
|
42
|
+
path: |
|
|
43
|
+
mcp-audit.md
|
|
44
|
+
mcp-audit.sarif
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Failing Policy
|
|
48
|
+
|
|
49
|
+
Use one of these gates:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-critical
|
|
53
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md --fail-on-high
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`--fail-on-critical` is appropriate while onboarding noisy repos. `--fail-on-high` is appropriate once maintainers have accepted the release gate.
|
|
57
|
+
|
|
58
|
+
## Badge Ready Score
|
|
59
|
+
|
|
60
|
+
The score command returns compact JSON for badge generators or dashboards:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Example:
|
|
67
|
+
|
|
68
|
+
```json
|
|
69
|
+
{
|
|
70
|
+
"target_id": "my-mcp-server",
|
|
71
|
+
"profile": "nsa-mcp",
|
|
72
|
+
"score": 87,
|
|
73
|
+
"status": "needs_review",
|
|
74
|
+
"finding_count": 2
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Local Fixture Demo
|
|
79
|
+
|
|
80
|
+
This repository includes an intentionally insecure inert fixture:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
npx tsx src/cli.ts audit examples/insecure-mcp-server --profile nsa-mcp --format markdown --output /tmp/nsa-mcp-audit-report.md
|
|
84
|
+
npx tsx src/cli.ts audit examples/insecure-mcp-server --profile nsa-mcp --format sarif --output /tmp/nsa-mcp-results.sarif
|
|
85
|
+
npx tsx src/cli.ts score examples/insecure-mcp-server --profile nsa-mcp --format json
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Generated examples are checked in under [`docs/examples`](./examples/).
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# MCP Observatory Procurement One-Pager
|
|
2
|
+
|
|
3
|
+
## Positioning
|
|
4
|
+
|
|
5
|
+
MCP Observatory is an open source security release gate for MCP servers before deployment into sensitive, regulated, or mission-critical agentic AI environments.
|
|
6
|
+
|
|
7
|
+
It helps platform and security teams answer a concrete procurement question:
|
|
8
|
+
|
|
9
|
+
> Can this MCP server be safely introduced into an agent workflow, and what evidence supports that decision?
|
|
10
|
+
|
|
11
|
+
## What It Does
|
|
12
|
+
|
|
13
|
+
- runs MCP server startup, conformance, schema, security, and safe attack-readiness checks
|
|
14
|
+
- maps findings to the `nsa-mcp` public-guidance profile
|
|
15
|
+
- produces Markdown audit reports for reviewers
|
|
16
|
+
- emits SARIF for GitHub Code Scanning
|
|
17
|
+
- emits compact trust score JSON for CI gates, badges, or dashboards
|
|
18
|
+
- supports recurring GitHub Actions checks and remediation tracking
|
|
19
|
+
|
|
20
|
+
## Buyer Value
|
|
21
|
+
|
|
22
|
+
| Buyer concern | MCP Observatory answer |
|
|
23
|
+
|---|---|
|
|
24
|
+
| Tool boundary risk | Detects shell, filesystem, browser, network, secrets, and destructive tool surfaces. |
|
|
25
|
+
| Agent manipulation risk | Flags hidden instructions and unsafe autonomous behavior in tool metadata. |
|
|
26
|
+
| Security review workflow | Produces SARIF, Markdown, JSON, and CI artifacts. |
|
|
27
|
+
| Procurement evidence | Provides a public-guidance crosswalk and repeatable audit commands. |
|
|
28
|
+
| Adoption risk | Runs locally with no hosted dependency required. |
|
|
29
|
+
| Remediation tracking | Supports repeatable audits, CI gates, and future baseline diffing. |
|
|
30
|
+
|
|
31
|
+
## Pilot Package
|
|
32
|
+
|
|
33
|
+
A 30 day pilot should include:
|
|
34
|
+
|
|
35
|
+
- 5 to 25 MCP servers scanned
|
|
36
|
+
- baseline `nsa-mcp` audit reports
|
|
37
|
+
- SARIF uploaded to GitHub Code Scanning for selected repos
|
|
38
|
+
- CI gate enabled on at least 3 repos
|
|
39
|
+
- weekly scheduled audit runs
|
|
40
|
+
- reviewer false-positive feedback
|
|
41
|
+
- final remediation and adoption summary
|
|
42
|
+
|
|
43
|
+
## Example Commands
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md
|
|
47
|
+
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output mcp-audit.sarif
|
|
48
|
+
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Security And Data Boundary
|
|
52
|
+
|
|
53
|
+
Local audits do not require sending artifacts to a hosted service. Reports can stay inside the customer repository, CI logs, artifact store, or security tooling.
|
|
54
|
+
|
|
55
|
+
MCP Observatory should not collect secrets, environment dumps, file contents, or raw command outputs for audit reporting. Secret-like values are reported by name or pattern with values redacted.
|
|
56
|
+
|
|
57
|
+
## Procurement Notes
|
|
58
|
+
|
|
59
|
+
- License: MIT
|
|
60
|
+
- Runtime: Node.js 20+
|
|
61
|
+
- Integration points: CLI, GitHub Actions, SARIF, Markdown, JSON
|
|
62
|
+
- Hosted service: optional, not required for local audit evidence
|
|
63
|
+
- Certification claim: none
|
|
64
|
+
- Government endorsement claim: none
|
|
65
|
+
- Best fit: AI platform, AppSec, DevSecOps, AI assurance, software supply chain, and agent runtime security teams
|
|
66
|
+
|
|
67
|
+
## Useful Links
|
|
68
|
+
|
|
69
|
+
- [Government and enterprise pilot brief](./government-enterprise-pilot.md)
|
|
70
|
+
- [Public guidance crosswalk](./public-guidance-crosswalk.md)
|
|
71
|
+
- [NSA-MCP audit CI guide](./nsa-mcp-audit-ci.md)
|
|
72
|
+
- [Example audit report](./examples/nsa-mcp-audit-report.md)
|
|
73
|
+
- [Example SARIF](./examples/nsa-mcp-results.sarif)
|
|
74
|
+
- [Example score JSON](./examples/nsa-mcp-score.json)
|
package/docs/proof.md
CHANGED
|
@@ -5,8 +5,8 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
|
|
|
5
5
|
## Current Public Surface
|
|
6
6
|
|
|
7
7
|
- npm package: `@kryptosai/mcp-observatory`
|
|
8
|
-
- GitHub Action: `KryptosAI/mcp-observatory/action@
|
|
9
|
-
- Latest release: `
|
|
8
|
+
- GitHub Action: `KryptosAI/mcp-observatory/action@v1.28.0`
|
|
9
|
+
- Latest release: `v1.28.0`
|
|
10
10
|
- CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, setup-ci, init-ci, ci-report, enterprise-report, score, badge, cloud
|
|
11
11
|
- Test suite: 348 passing tests across 45 test files as of July 1, 2026
|
|
12
12
|
- GitHub traffic snapshot: 745 clones and 232 unique cloners in the visible June 2026 traffic window
|
|
@@ -76,6 +76,9 @@ Open and accepted third-party integrations are tracked here:
|
|
|
76
76
|
|
|
77
77
|
GitHub code search shows public references outside the main repo. These are discovery/listing signals, not customer claims:
|
|
78
78
|
|
|
79
|
+
- PulseMCP has a maintainer-recognized `io.github.KryptosAI/mcp-observatory` listing with `server.json` visibility.
|
|
80
|
+
- Smithery, Glama, LobeHub, Yarn/Vibehackers, and other MCP catalogs surface MCP Observatory install or scorecard pages.
|
|
81
|
+
- MseeP has an unclaimed listing for MCP Observatory. Treat it as a directory opportunity until ownership, scan evidence, and badge wording are reviewed.
|
|
79
82
|
- `punkpeye/awesome-mcp-devtools` lists MCP Observatory in an MCP developer-tools index.
|
|
80
83
|
- `linny006/mcp-servers-live` mirrors a public MCP Observatory listing page.
|
|
81
84
|
- `gabrielmoreira/awesome-ai-rabbit-holes` catalogs the GitHub project.
|