@kryptosai/mcp-observatory 0.21.0 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/COMMERCIAL.md +5 -3
  2. package/PRIVACY.md +5 -2
  3. package/README.md +27 -13
  4. package/dist/src/cli.js +1 -1
  5. package/dist/src/cli.js.map +1 -1
  6. package/dist/src/commands/init-ci.d.ts +16 -2
  7. package/dist/src/commands/init-ci.js +139 -2
  8. package/dist/src/commands/init-ci.js.map +1 -1
  9. package/dist/src/commercial.js +2 -2
  10. package/dist/src/commercial.js.map +1 -1
  11. package/dist/src/reporters/common.d.ts +16 -0
  12. package/dist/src/reporters/common.js +76 -0
  13. package/dist/src/reporters/common.js.map +1 -1
  14. package/dist/src/reporters/html.js +20 -0
  15. package/dist/src/reporters/html.js.map +1 -1
  16. package/dist/src/reporters/markdown.js +14 -2
  17. package/dist/src/reporters/markdown.js.map +1 -1
  18. package/dist/src/reporters/pr-comment.js +18 -1
  19. package/dist/src/reporters/pr-comment.js.map +1 -1
  20. package/dist/src/reporters/terminal.js +9 -1
  21. package/dist/src/reporters/terminal.js.map +1 -1
  22. package/dist/src/score.js +1 -1
  23. package/dist/src/score.js.map +1 -1
  24. package/dist/src/validate.js +58 -3
  25. package/dist/src/validate.js.map +1 -1
  26. package/docs/certification-campaign-template.md +42 -28
  27. package/docs/certification-distribution.md +21 -1
  28. package/docs/compatibility.md +2 -2
  29. package/docs/directory-listing-copy.md +13 -6
  30. package/docs/distribution-launch.md +5 -5
  31. package/docs/enterprise-outreach-playbook.md +2 -2
  32. package/docs/mcp-lock-files.md +63 -0
  33. package/docs/mcp-safety-report-latest.md +12 -8
  34. package/docs/mcp-security-field-guide.md +97 -0
  35. package/docs/mcp-server-safety-index.md +85 -0
  36. package/docs/paid-pilot-offer.md +58 -0
  37. package/docs/project-case-study.md +73 -43
  38. package/docs/proof.md +26 -9
  39. package/docs/public-post-drafts.md +86 -0
  40. package/docs/publish-readiness.md +13 -3
  41. package/docs/reference-evaluations.md +134 -0
  42. package/package.json +9 -6
@@ -0,0 +1,85 @@
1
+ # MCP Server Safety Index
2
+
3
+ The MCP Server Safety Index is a public, reproducible way to show how MCP servers behave under compatibility, schema quality, drift, and security checks.
4
+
5
+ The goal is constructive proof, not callouts. Each entry shows what should be tested, how to reproduce it, what risk class matters, and what a maintainer can do next.
6
+
7
+ ## Index v0
8
+
9
+ | # | Server | Category | Reproducible Command | What To Check | Risk Class | Status |
10
+ | ---: | --- | --- | --- | --- | --- | --- |
11
+ | 1 | [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) sequential thinking | Reference | `npx -y @modelcontextprotocol/server-sequential-thinking@latest` | Startup, tools/list, schema quality, security-lite | Reference compatibility | PR open: [#4392](https://github.com/modelcontextprotocol/servers/pull/4392) |
12
+ | 2 | [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) filesystem | Filesystem | `npx -y @modelcontextprotocol/server-filesystem .` | Startup in harmless temp dir, path tools, schema quality | Filesystem boundary | Researched |
13
+ | 3 | [`upstash/context7`](https://github.com/upstash/context7) | Documentation/search | `npx -y @upstash/context7-mcp@latest` | Startup, retrieval tools, schemas, prompt-injection-sensitive text flow | Untrusted content retrieval | PR open: [#2800](https://github.com/upstash/context7/pull/2800) |
14
+ | 4 | [`executeautomation/mcp-playwright`](https://github.com/executeautomation/mcp-playwright) | Browser automation | `npx -y @executeautomation/playwright-mcp-server@latest` | Browser tools, schema quality, intentional code-eval suppressions | Browser/code execution | PR open: [#225](https://github.com/executeautomation/mcp-playwright/pull/225) |
15
+ | 5 | [`microsoft/playwright-mcp`](https://github.com/microsoft/playwright-mcp) | Browser automation | `npx -y @playwright/mcp@latest` | Browser tools, skip-invoke policy, schema quality, suppressions | Browser/code execution | PR open: [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) |
16
+ | 6 | [`kazuph/mcp-taskmanager`](https://github.com/kazuph/mcp-taskmanager) | Developer tools | `npx -y @kazuph/mcp-taskmanager@latest` | Task tools, schema quality, mutation clarity | Project/task mutation | PR open: [#11](https://github.com/kazuph/mcp-taskmanager/pull/11) |
17
+ | 7 | [`cyanheads/filesystem-mcp-server`](https://github.com/cyanheads/filesystem-mcp-server) | Filesystem | `node dist/index.js` | Capability declarations, resources/list, sandboxed filesystem target | Filesystem boundary | PR open: [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) |
18
+ | 8 | [`browserbase/mcp-server-browserbase`](https://github.com/browserbase/mcp-server-browserbase) | Browser automation | `npx -y @browserbasehq/mcp-server-browserbase` | Auth-free startup, browser tools, network/browser boundaries | Hosted browser control | Researched; likely needs API key |
19
+ | 9 | [`redis/mcp-redis`](https://github.com/redis/mcp-redis) | Database | `uvx mcp-redis` | Startup without live database, command surface, destructive operations | Data mutation | Researched; may need service |
20
+ | 10 | [`mongodb-js/mongodb-mcp-server`](https://github.com/mongodb-js/mongodb-mcp-server) | Database | `npx -y mongodb-mcp-server` | Connection handling, read/write tools, auth posture | Data mutation/auth | Researched; likely needs connection string |
21
+ | 11 | [`supabase-community/supabase-mcp`](https://github.com/supabase-community/supabase-mcp) | Database/SaaS | `npx -y supabase-mcp` | Startup, token handling, project mutation tools | Cloud data access | Researched; likely needs token |
22
+ | 12 | [`cloudflare/mcp-server-cloudflare`](https://github.com/cloudflare/mcp-server-cloudflare) | Cloud | `npx -y @cloudflare/mcp-server-cloudflare` | Auth posture, deploy/config tools, schema clarity | Cloud control plane | Researched; likely needs auth |
23
+ | 13 | [`stripe/agent-toolkit`](https://github.com/stripe/agent-toolkit) | Payments | `npx -y @stripe/agent-toolkit` | MCP mode, payment/customer mutation tools, auth posture | Payments/destructive action | Researched; likely needs API key |
24
+ | 14 | [`github/github-mcp-server`](https://github.com/github/github-mcp-server) | Developer tools | `docker run ghcr.io/github/github-mcp-server` | Auth handling, repo mutation tools, schema clarity | Source-code control | Researched; likely needs token |
25
+ | 15 | [`jetbrains/mcpProxy`](https://github.com/JetBrains/mcpProxy) | IDE/developer tools | `npx -y @jetbrains/mcp-proxy` | IDE dependency, startup behavior, tool surface | Local IDE control | Researched; may need IDE process |
26
+ | 16 | [`BrowserMCP/mcp`](https://github.com/BrowserMCP/mcp) | Browser automation | `npx -y @browsermcp/mcp` | Browser tools, schema quality, browser-control boundary | Browser control | PR open: [#189](https://github.com/BrowserMCP/mcp/pull/189) |
27
+ | 17 | [`UI5/mcp-server`](https://github.com/UI5/mcp-server) | Developer tools | `npx -y @ui5/mcp-server` | UI5 tooling commands, schema quality, drift risk | App development tooling | PR open: [#348](https://github.com/UI5/mcp-server/pull/348) |
28
+ | 18 | [`antvis/mcp-server-chart`](https://github.com/antvis/mcp-server-chart) | Visualization/data | `npx -y @antv/mcp-server-chart` | Chart generation tools, schema quality, artifact-producing tools | Generated artifacts | PR open: [#312](https://github.com/antvis/mcp-server-chart/pull/312) |
29
+ | 19 | [`makenotion/notion-mcp-server`](https://github.com/makenotion/notion-mcp-server) | SaaS/API | `npx -y @notionhq/notion-mcp-server` | Auth handling, read/write tool separation, schema quality | Workspace data access | PR open: [#324](https://github.com/makenotion/notion-mcp-server/pull/324) |
30
+ | 20 | [`sentry/sentry-mcp`](https://github.com/getsentry/sentry-mcp) | Developer SaaS | `npx -y @sentry/mcp-server` | Auth handling, issue/project tools, schema quality | Production incident data | Researched; likely needs token |
31
+
32
+ ## Evaluation Command
33
+
34
+ For simple npm-backed servers:
35
+
36
+ ```bash
37
+ npx @kryptosai/mcp-observatory test --security npx -y <server-package>
38
+ ```
39
+
40
+ For safer campaign PRs:
41
+
42
+ ```bash
43
+ npx @kryptosai/mcp-observatory init-ci --all --command "npx -y <server-package>"
44
+ ```
45
+
46
+ For production-style review:
47
+
48
+ ```bash
49
+ npx @kryptosai/mcp-observatory lock
50
+ npx @kryptosai/mcp-observatory lock verify
51
+ ```
52
+
53
+ ## What Each Column Means
54
+
55
+ - What To Check: the minimum compatibility/security surface a maintainer or platform team should inspect.
56
+ - Risk Class: the operational reason the server matters before agents depend on it.
57
+ - Status: public proof such as PR open, PR accepted, badge added, researched, or needs maintainer review.
58
+
59
+ ## Publication Rules
60
+
61
+ - Use only public repositories, public package commands, public PRs, or sample artifacts.
62
+ - Include a reproduction command for every row.
63
+ - Link to the maintainer PR or public artifact when available.
64
+ - Phrase findings constructively: “needs review” rather than “unsafe” unless there is clear public proof.
65
+ - Keep customer/domain telemetry internal unless the customer gives permission or there is independent public evidence.
66
+
67
+ ## Five Patterns To Publish From v0
68
+
69
+ 1. Browser automation MCP servers need explicit policy around code execution, screenshots, navigation, and mutation.
70
+ 2. Filesystem MCP servers need harmless CI sandboxes and clear read/write boundaries.
71
+ 3. SaaS and cloud MCP servers often cannot be meaningfully checked without token-safe target configs.
72
+ 4. Database MCP servers need read/write classification and connection-string hygiene before CI rollout.
73
+ 5. Lock files turn MCP surface drift into a reviewable PR event instead of an invisible agent dependency change.
74
+
75
+ ## Next Wave Criteria
76
+
77
+ Prioritize 20-50 servers that have:
78
+
79
+ - active maintenance in the last 90 days
80
+ - visible stars, downloads, or directory listings
81
+ - simple `npx`, `uvx`, or Docker startup commands
82
+ - enterprise-relevant categories such as browser automation, filesystem, documentation/search, databases, cloud, productivity, and developer tools
83
+ - no existing MCP compatibility/security CI
84
+
85
+ One accepted PR in a respected repo is worth more than a large list of shallow checks.
@@ -0,0 +1,58 @@
1
+ # Paid Pilot Offer
2
+
3
+ ## Private MCP Readiness Review
4
+
5
+ Offer:
6
+
7
+ > Private MCP readiness review + CI rollout + drift/security report.
8
+
9
+ This is a manual pilot, not a self-serve SaaS promise.
10
+
11
+ ## Who It Is For
12
+
13
+ - teams running MCP servers in production or pre-production
14
+ - security/platform teams reviewing agent tool dependencies
15
+ - companies with private MCP repos
16
+ - teams that need proof before agents depend on internal tools
17
+
18
+ ## What The Pilot Includes
19
+
20
+ - review of the customer’s MCP config, repo, or startup commands
21
+ - MCP Observatory CI rollout for selected servers
22
+ - private readiness report covering startup, capabilities, schema quality, security findings, and drift risk
23
+ - MCP lock-file setup for contract drift review
24
+ - prioritized remediation notes
25
+ - optional certification language for servers that pass agreed checks
26
+
27
+ ## Starting Prices
28
+
29
+ - Business Pilot: starts at `$999/month`
30
+ - Enterprise Pilot: starts at `$3k/month`
31
+ - Strategic Accounts: custom, `$250k+/year`
32
+
33
+ Do not route major platforms, AI labs, or large enterprises to Team/Business pricing. Use a production/security pilot conversation and ask for the owner or procurement path.
34
+
35
+ ## Simple Outreach Copy
36
+
37
+ Subject: Private MCP readiness review
38
+
39
+ Hi,
40
+
41
+ I build MCP Observatory, the CI and security gate for MCP servers before agents depend on them.
42
+
43
+ I am opening a small number of private MCP readiness pilots for teams running MCP in production or pre-production. The pilot includes CI rollout, schema/security review, drift checks, and a private readiness report for your MCP servers.
44
+
45
+ If MCP is becoming part of your agent infrastructure, I can help you answer:
46
+
47
+ - which servers are safe enough for agents to depend on?
48
+ - which tool surfaces changed recently?
49
+ - where are the schema/security risks?
50
+ - what should block a PR before production?
51
+
52
+ Would it be useful to compare notes this week?
53
+
54
+ William
55
+
56
+ ## Delivery Shape
57
+
58
+ Start with static reports and CI setup. Do not build a dashboard until paid pilot feedback proves exactly what buyers need.
@@ -4,9 +4,17 @@
4
4
 
5
5
  MCP Observatory is CI/security infrastructure for production MCP servers.
6
6
 
7
- ## Problem
7
+ ## Project Narrative
8
8
 
9
- MCP servers are becoming dependencies for AI agents. Teams need to know whether those servers still start, expose usable tools, keep schemas stable, avoid obvious security footguns, and behave consistently as agents depend on them.
9
+ MCP Observatory identifies an emerging risk in AI agent infrastructure and turns it into a practical OSS control: CI checks, security reports, drift detection, telemetry intelligence, and certification workflows for production MCP servers.
10
+
11
+ The project is strongest as a signal because it connects product intuition with implementation depth. It starts from a real infrastructure shift, builds a working developer tool around that shift, instruments usage, and creates a credible path from open source adoption to production security workflows.
12
+
13
+ ## Problem Discovery
14
+
15
+ MCP servers are becoming dependencies for AI agents. They expose tools, prompts, resources, and data access that agents can call directly. When those servers drift, fail to start, expose broad capabilities, or return ambiguous schemas, the failure can propagate into agent workflows.
16
+
17
+ The control gap is simple: teams need a way to test MCP servers before agents depend on them. They also need artifacts that maintainers, platform engineers, and security reviewers can understand.
10
18
 
11
19
  ## Product
12
20
 
@@ -23,46 +31,57 @@ MCP Observatory provides:
23
31
  - static enterprise reports
24
32
  - telemetry intelligence for product and account-level learning
25
33
 
26
- ## Architecture
34
+ ## System Design
27
35
 
28
- The project is a TypeScript/Node CLI with modular command handlers, MCP adapters, check runners, reporters, artifact schemas, and a GitHub Action wrapper. A Cloudflare Worker handles hosted artifact upload pilots, and a separate telemetry Worker stores private aggregate usage events in D1.
36
+ The project is a TypeScript/Node CLI with modular command handlers, MCP adapters, check runners, reporters, artifact schemas, and a GitHub Action wrapper.
29
37
 
30
- ## Technical Proof
38
+ The system supports local-process and HTTP MCP targets, stores run artifacts, compares runs for regressions, generates reports for humans and CI systems, and can run as an MCP server itself. A Cloudflare Worker handles hosted artifact upload pilots. A separate telemetry Worker stores private aggregate usage events in D1 for product and account intelligence.
31
39
 
32
- As of June 19, 2026:
40
+ ## Security Model
33
41
 
34
- - 10k+ source lines in `src`
35
- - 40 test files
36
- - 321 passing tests
37
- - npm package published
38
- - GitHub Action available
39
- - MCP server mode available
40
- - telemetry export and company intelligence tooling available
42
+ MCP Observatory treats MCP servers as agent-facing infrastructure. The goal is not to claim formal semantic safety. The goal is to make compatibility, drift, and obvious security risk visible before deployment.
41
43
 
42
- ## Traction Snapshot
44
+ Current controls include:
43
45
 
44
- Safe public and aggregate signals:
46
+ - lightweight security checks for risky schema patterns
47
+ - schema quality analysis for agent usability
48
+ - SARIF output for security review workflows
49
+ - support for security suppressions when broad tools are intentional
50
+ - private-network rejection for hosted scans
51
+ - privacy disclosure and telemetry opt-out controls
52
+ - sanitized public reporting policy
45
53
 
46
- - 10,278 telemetry events
47
- - 7,211 telemetry sessions
48
- - 5,368 external sessions after separating internal activity
49
- - 582 GitHub clones and 175 unique cloners in the visible June 2026 traffic window
50
- - 104 npm downloads during June 11-17, 2026
54
+ For deeper context, see the [MCP Server Security Field Guide](./mcp-security-field-guide.md).
51
55
 
52
- These are early signals. Public social proof is still limited and should be improved through the certification campaign.
56
+ ## Telemetry Intelligence
53
57
 
54
- ## Security And Privacy Posture
58
+ Telemetry is used privately to understand product usage and identify account-level signals without publishing raw personal data.
55
59
 
56
- The project includes:
60
+ As of the latest local export on June 20, 2026:
57
61
 
58
- - telemetry opt-out controls
59
- - privacy disclosure
60
- - security policy
61
- - token-based hosted artifact upload
62
- - private-network rejection for hosted scans
63
- - sanitized public reporting policy
62
+ - 10,918 telemetry events
63
+ - 7,380 total sessions
64
+ - 5,379 external sessions after separating internal activity
65
+ - 2,446 external CI sessions
66
+ - 138 attributed company/org sessions
67
+ - 11 attributed company/org candidates
64
68
 
65
- Public claims should use aggregate metrics and accepted public integrations, not raw telemetry.
69
+ Public claims use aggregate or sanitized data only. Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
70
+
71
+ ## Distribution Strategy
72
+
73
+ The distribution wedge is useful CI for other MCP repositories. The certification campaign opens small, helpful PRs that add MCP compatibility/security checks and leave maintainers with a public trust signal.
74
+
75
+ Current public distribution proof includes:
76
+
77
+ - latest release: `v0.23.0`
78
+ - npm package: `@kryptosai/mcp-observatory`
79
+ - GitHub Action: `KryptosAI/mcp-observatory/action@main`
80
+ - visible GitHub traffic window: 721 clones and 221 unique cloners
81
+ - official MCP reference PR open and green: [`modelcontextprotocol/servers#4392`](https://github.com/modelcontextprotocol/servers/pull/4392)
82
+ - open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, and other MCP projects
83
+
84
+ See [reference evaluations](./reference-evaluations.md) and [public proof](./proof.md).
66
85
 
67
86
  ## Commercial Path
68
87
 
@@ -83,24 +102,35 @@ Current pilot anchors:
83
102
  - Enterprise: starts at `$3k/month`
84
103
  - Strategic: `$250k+/year`
85
104
 
86
- ## Job-Opportunity Value
105
+ ## Professional Signal
87
106
 
88
- This project demonstrates ability across:
107
+ MCP Observatory demonstrates applied work across:
89
108
 
90
- - AI infrastructure
109
+ - AI agent infrastructure
91
110
  - developer tooling
92
- - security tooling
93
- - MCP ecosystem work
111
+ - secure tool invocation
112
+ - software supply chain thinking
94
113
  - CI/CD integrations
95
114
  - telemetry and product analytics
96
- - commercialization and enterprise packaging
115
+ - open source distribution
116
+ - enterprise packaging
117
+
118
+ It is designed to be evaluated through public work: code, docs, CI integrations, reference evaluations, proof surfaces, and real maintainer PRs.
119
+
120
+ ## Future Roadmap
121
+
122
+ Near-term milestones:
97
123
 
98
- It is strongest as a portfolio asset when paired with public proof: accepted external PRs, badges in other repos, directory listings, and a short demo.
124
+ 1. Convert certification PRs into accepted public integrations.
125
+ 2. Publish recurring MCP safety reports.
126
+ 3. Add stronger policy/provenance language for production MCP adoption.
127
+ 4. Improve hosted artifact upload into a simple pilot workflow.
128
+ 5. Convert serious production users into paid pilots.
99
129
 
100
- ## Next Milestones
130
+ Longer-term opportunities:
101
131
 
102
- 1. Publish latest package with `init-ci`.
103
- 2. Open first certification PR wave.
104
- 3. Capture accepted PRs as public proof.
105
- 4. Publish recurring MCP safety reports.
106
- 5. Convert serious users into paid pilots.
132
+ - policy controls for agent tool use
133
+ - provenance for MCP packages and configurations
134
+ - schema locks and controlled drift review
135
+ - runtime monitoring for production agent tool calls
136
+ - fleet inventory across teams, repositories, and hosts
package/docs/proof.md CHANGED
@@ -6,22 +6,30 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
6
6
 
7
7
  - npm package: `@kryptosai/mcp-observatory`
8
8
  - GitHub Action: `KryptosAI/mcp-observatory/action@main`
9
+ - Latest release: `v0.23.0`
9
10
  - CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, init-ci, ci-report, enterprise-report, score, badge, cloud
10
- - Test suite: 321 passing tests across 40 test files as of June 19, 2026
11
- - GitHub traffic snapshot: 582 clones and 175 unique cloners in the visible June 2026 traffic window
11
+ - Test suite: 334 passing tests across 43 test files as of June 20, 2026
12
+ - GitHub traffic snapshot: 721 clones and 221 unique cloners in the visible June 2026 traffic window
12
13
  - npm downloads snapshot: 104 downloads for June 11-17, 2026
14
+ - Security guide: [MCP Server Security Field Guide](./mcp-security-field-guide.md)
15
+ - Safety index: [MCP Server Safety Index](./mcp-server-safety-index.md)
16
+ - Public examples: [Reference Evaluations](./reference-evaluations.md)
17
+ - Lock-file CI primitive: [MCP Lock Files](./mcp-lock-files.md)
18
+ - Public post drafts: [Launch Post Drafts](./public-post-drafts.md)
19
+ - Pilot offer: [Private MCP Readiness Review](./paid-pilot-offer.md)
13
20
 
14
21
  ## Safe Aggregate Telemetry Snapshot
15
22
 
16
23
  Internal telemetry is used for product analytics and account-level outreach. Public reporting uses only aggregate or sanitized data.
17
24
 
18
- As of the latest local export on June 19, 2026:
25
+ As of the latest local export on June 20, 2026:
19
26
 
20
- - 10,278 telemetry events
21
- - 7,211 unique sessions
22
- - 5,368 external sessions after separating internal/personal activity
23
- - 2,434 external CI sessions
24
- - 128 attributed company/org sessions
27
+ - 10,918 telemetry events
28
+ - 7,380 total sessions
29
+ - 5,379 external sessions after separating internal/personal activity
30
+ - 2,446 external CI sessions
31
+ - 138 attributed company/org sessions
32
+ - 11 attributed company/org candidates
25
33
  - top external commands: `serve`, `run`, `diff`, `test`, `scan`, `history`
26
34
 
27
35
  Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
@@ -50,7 +58,16 @@ Accepted third-party integrations will be tracked here:
50
58
 
51
59
  | Repo | PR | Check Added | Badge Added | Status |
52
60
  | --- | --- | --- | --- | --- |
53
- | _pending_ | | | | |
61
+ | `modelcontextprotocol/servers` | [#4392](https://github.com/modelcontextprotocol/servers/pull/4392) | Yes | No | Open, mergeable, MCP Observatory check passing |
62
+ | `microsoft/playwright-mcp` | [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) | Yes | No | Open |
63
+ | `upstash/context7` | [#2800](https://github.com/upstash/context7/pull/2800) | Yes | No | Open |
64
+ | `executeautomation/mcp-playwright` | [#225](https://github.com/executeautomation/mcp-playwright/pull/225) | Yes | No | Open |
65
+ | `kazuph/mcp-taskmanager` | [#11](https://github.com/kazuph/mcp-taskmanager/pull/11) | Yes | No | Open |
66
+ | `cyanheads/filesystem-mcp-server` | [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) | Yes | No | Open |
67
+ | `antvis/mcp-server-chart` | [#312](https://github.com/antvis/mcp-server-chart/pull/312) | Yes | No | Open |
68
+ | `BrowserMCP/mcp` | [#189](https://github.com/BrowserMCP/mcp/pull/189) | Yes | No | Open |
69
+ | `UI5/mcp-server` | [#348](https://github.com/UI5/mcp-server/pull/348) | Yes | No | Open |
70
+ | `makenotion/notion-mcp-server` | [#324](https://github.com/makenotion/notion-mcp-server/pull/324) | Yes | No | Open |
54
71
 
55
72
  ## Commercial Proof
56
73
 
@@ -0,0 +1,86 @@
1
+ # Public Post Drafts
2
+
3
+ Use these as launch posts, GitHub Discussion posts, LinkedIn posts, or short blog drafts. The framing is about MCP safety patterns, not “look at my tool.”
4
+
5
+ ## 1. I Tested 20 MCP Servers. The Pattern Was Not “Bad Servers”; It Was Missing Gates.
6
+
7
+ MCP servers are becoming production dependencies for agents, but many of them still ship without the kind of CI gate we expect from normal software dependencies.
8
+
9
+ The main pattern I saw while building the first MCP Server Safety Index was simple: the risky part is rarely that a server exists. The risky part is that agents may depend on a tool surface nobody is testing for startup reliability, schema quality, security posture, or drift.
10
+
11
+ The checks that matter most:
12
+
13
+ - does the server start cleanly in CI?
14
+ - do tools, prompts, and resources respond as advertised?
15
+ - are tool schemas precise enough for agents to call safely?
16
+ - did a release add, remove, or broaden a tool?
17
+ - are destructive tools clearly identifiable?
18
+
19
+ My takeaway: MCP needs a package-lock moment. Commit the agent-facing contract, then make drift visible before agents depend on it.
20
+
21
+ ## 2. Browser MCP Servers Need A Different Security Bar
22
+
23
+ Browser automation MCP servers are powerful because agents can navigate pages, click, type, inspect state, and sometimes execute scripts.
24
+
25
+ That is exactly why they need explicit CI and security gates.
26
+
27
+ For browser MCP servers, a useful review should separate:
28
+
29
+ - harmless inventory checks
30
+ - state-mutating browser actions
31
+ - code execution or page-evaluation tools
32
+ - network/navigation controls
33
+ - tool schemas that are too broad for safe agent planning
34
+
35
+ The goal is not to block browser MCP. The goal is to make the trust boundary visible before an agent gets a browser with hands.
36
+
37
+ ## 3. Filesystem MCP Servers Should Always Test In A Sandbox
38
+
39
+ Filesystem MCP servers are one of the clearest examples of why MCP CI needs context.
40
+
41
+ A server can be useful and still dangerous if the test command points at the wrong directory, if read/write boundaries are unclear, or if a tool schema makes broad path access look harmless.
42
+
43
+ The minimum safety pattern:
44
+
45
+ - run CI against a temporary harmless directory
46
+ - verify tools/resources respond as advertised
47
+ - flag broad filesystem access
48
+ - document which operations are read-only vs write-capable
49
+ - treat changes to path schemas as contract drift
50
+
51
+ Agents need tools. They do not need accidental access to everything.
52
+
53
+ ## 4. Token-Backed SaaS MCP Servers Need Issue-First Certification
54
+
55
+ Many SaaS, cloud, payments, database, and developer-platform MCP servers cannot be safely checked with a drive-by PR because meaningful startup requires tokens or live services.
56
+
57
+ For those repos, the right move is usually not a workflow PR first. It is an issue or maintainer question:
58
+
59
+ “What is the safest CI startup command for this server?”
60
+
61
+ Once maintainers provide a token-safe target config, the useful checks are:
62
+
63
+ - does startup fail cleanly without credentials?
64
+ - are auth requirements documented?
65
+ - are destructive tools obvious?
66
+ - are schemas narrow enough for agent use?
67
+ - can the repo publish a safe compatibility/security badge?
68
+
69
+ Security adoption works better when it starts by respecting maintainer context.
70
+
71
+ ## 5. MCP Drift Is An AI Supply Chain Problem
72
+
73
+ When a package dependency changes, teams have lock files, diffs, review, and release notes.
74
+
75
+ When an MCP server changes its tool surface, an agent dependency changed too.
76
+
77
+ That means tool additions, tool removals, schema broadening, new write actions, and prompt/resource changes should be visible in pull requests.
78
+
79
+ The useful primitive is an MCP lock file:
80
+
81
+ ```bash
82
+ npx @kryptosai/mcp-observatory lock
83
+ npx @kryptosai/mcp-observatory lock verify
84
+ ```
85
+
86
+ The point is not bureaucracy. It is to make the agent-facing contract reviewable before production workflows quietly depend on something new.
@@ -22,16 +22,25 @@ Confirm:
22
22
  - HTTP target examples use env references instead of inline tokens.
23
23
  - Security findings appear in artifact evidence as structured `findings`.
24
24
  - Hosted upload is available through `mcp-observatory cloud upload <artifact>` when `MCP_OBSERVATORY_CLOUD_TOKEN` is set.
25
+ - Hosted HTTP scans require `Authorization: Bearer <HOSTED_SCAN_TOKEN>` and are treated as an authenticated pilot surface.
26
+
27
+ Known audit note:
28
+
29
+ - `npm audit` may report `undici <=6.26.0` through the `npm@11.17.0` package bundled under `@semantic-release/npm`. As of June 20, 2026, `npm audit fix` cannot update this bundled copy and `npm@11.17.0` is the current published npm package. The remaining vulnerable `undici` copy is release tooling only and is not part of MCP Observatory runtime dependencies or the packed npm artifact. Recheck after npm publishes a newer package.
30
+
31
+ Known audit note:
32
+
33
+ - `npm audit` may report `undici <=6.26.0` through the `npm@11.17.0` package bundled under `@semantic-release/npm`. `npm audit fix` updates the fixable `@actions/http-client` path, but the remaining `undici` copy is bundled inside npm release tooling and is not part of MCP Observatory runtime dependencies or the packed npm artifact. Recheck after npm publishes a newer package.
25
34
 
26
35
  ## Public Distribution
27
36
 
28
37
  - Merge the health/commercialization PR.
29
38
  - Update the GitHub repo homepage to the README or commercial page.
30
39
  - Publish npm only after the release gate is green.
31
- - Refresh MCP directory listings with: “MCP Observatory helps teams test, secure, and monitor MCP servers before agents depend on them.”
32
- - Include “free for local OSS use; paid for hosted reporting, private repo CI, security reports, production monitoring, certification, support, and fleet visibility.”
40
+ - Refresh MCP directory listings with: “MCP Observatory is the CI and security gate for MCP servers before agents depend on them.”
41
+ - Include “free for local OSS use; paid for hosted reporting, private repo CI, recurring security reports, certification, support, and fleet visibility.”
33
42
  - Link production users to `COMMERCIAL.md` and `william@banksey.com`.
34
- - Submit or refresh listings on Glama, PulseMCP, Smithery, and relevant awesome-MCP lists with the tags: security, developer tools, CI/CD, testing, observability, schema drift.
43
+ - Submit or refresh listings on Glama, PulseMCP, Smithery, and relevant awesome-MCP lists with the tags: security, developer tools, CI/CD, testing, MCP security, schema drift.
35
44
  - Use the certification distribution loop to open helpful PRs against popular MCP server repos and convert accepted PRs into proof points.
36
45
  - Link public proof, the safety report, and directory listing copy from launch/outreach materials.
37
46
 
@@ -63,6 +72,7 @@ Worker:
63
72
 
64
73
  - `POST /api/v1/artifacts` stores a run artifact behind bearer-token auth.
65
74
  - `GET /api/v1/artifacts/:org` returns the org artifact index behind the same auth.
75
+ - `POST /api/v1/scan` requires `Authorization: Bearer <HOSTED_SCAN_TOKEN>`.
66
76
  - Hosted scans reject localhost/private-network targets; use local CLI for internal MCP servers.
67
77
 
68
78
  ## What Not To Do Yet
@@ -0,0 +1,134 @@
1
+ # MCP Observatory Reference Evaluations
2
+
3
+ Reference evaluations show how MCP Observatory applies to common MCP server categories. These are public, safe examples intended to help maintainers and security reviewers understand what the tool checks and what kind of risk each category can expose.
4
+
5
+ The examples below are not customer claims. They are public evaluation targets, public pull requests, or category examples that can be reproduced with the CLI.
6
+
7
+ ## Official MCP Reference Servers
8
+
9
+ Representative repo: [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers)
10
+
11
+ Public proof:
12
+
13
+ - PR: [`modelcontextprotocol/servers#4392`](https://github.com/modelcontextprotocol/servers/pull/4392)
14
+ - Status: open, mergeable, with a passing MCP Observatory check as of June 19, 2026
15
+
16
+ What this represents:
17
+
18
+ - reference MCP implementations
19
+ - simple tools that should behave predictably in CI
20
+ - a good baseline for model context protocol testing
21
+
22
+ What Observatory checks:
23
+
24
+ - server startup in GitHub Actions
25
+ - tools list/respond correctly
26
+ - schema quality and security scan output
27
+ - report generation for maintainers
28
+
29
+ Adoption command:
30
+
31
+ ```bash
32
+ npx @kryptosai/mcp-observatory init-ci --all --command "npx -y @modelcontextprotocol/server-sequential-thinking"
33
+ ```
34
+
35
+ ## Browser Automation MCP Servers
36
+
37
+ Representative public examples:
38
+
39
+ - [`microsoft/playwright-mcp`](https://github.com/microsoft/playwright-mcp)
40
+ - [`executeautomation/mcp-playwright`](https://github.com/executeautomation/mcp-playwright)
41
+
42
+ Public proof:
43
+
44
+ - PR: [`microsoft/playwright-mcp#1657`](https://github.com/microsoft/playwright-mcp/pull/1657)
45
+ - PR: [`executeautomation/mcp-playwright#225`](https://github.com/executeautomation/mcp-playwright/pull/225)
46
+
47
+ What this represents:
48
+
49
+ - high-capability browser tools
50
+ - agent access to pages, scripts, navigation, screenshots, and user-like actions
51
+ - a category where secure tool invocation and explicit trust boundaries matter
52
+
53
+ What Observatory checks:
54
+
55
+ - tool inventory
56
+ - schema quality
57
+ - risky browser/code-execution surfaces
58
+ - intentional suppressions for known acceptable findings
59
+ - whether deep invocation should be skipped for tools that can mutate browser state
60
+
61
+ Adoption command:
62
+
63
+ ```bash
64
+ npx @kryptosai/mcp-observatory test --security npx -y @playwright/mcp
65
+ ```
66
+
67
+ ## Filesystem MCP Servers
68
+
69
+ Representative public category: filesystem-backed MCP servers.
70
+
71
+ Public proof:
72
+
73
+ - PR: [`cyanheads/filesystem-mcp-server#19`](https://github.com/cyanheads/filesystem-mcp-server/pull/19)
74
+
75
+ What this represents:
76
+
77
+ - local file access exposed to agents
78
+ - read/write boundaries that should be explicit
79
+ - capability declarations that need to match observed MCP behavior
80
+
81
+ What Observatory checks:
82
+
83
+ - tools/resources capability consistency
84
+ - broad filesystem access findings
85
+ - schema quality for path-oriented tools
86
+ - safe sandbox target configuration for CI
87
+
88
+ Adoption command:
89
+
90
+ ```bash
91
+ npx @kryptosai/mcp-observatory test --security npx -y filesystem-mcp-server .
92
+ ```
93
+
94
+ Use a harmless temporary directory for CI checks when evaluating filesystem servers.
95
+
96
+ ## Documentation And Search MCP Servers
97
+
98
+ Representative public example: [`upstash/context7`](https://github.com/upstash/context7)
99
+
100
+ Public proof:
101
+
102
+ - PR: [`upstash/context7#2800`](https://github.com/upstash/context7/pull/2800)
103
+
104
+ What this represents:
105
+
106
+ - documentation retrieval and search tools
107
+ - untrusted or fast-changing text entering an agent context
108
+ - a category where prompt-injection-aware review matters
109
+
110
+ What Observatory checks:
111
+
112
+ - tool inventory
113
+ - schema quality
114
+ - startup reliability
115
+ - security findings around broad retrieval or response behavior
116
+ - report artifacts that maintainers can review in pull requests
117
+
118
+ Adoption command:
119
+
120
+ ```bash
121
+ npx @kryptosai/mcp-observatory init-ci --all --command "npx -y @upstash/context7-mcp"
122
+ ```
123
+
124
+ ## How To Read These Evaluations
125
+
126
+ Passing an Observatory check means the server passed the configured compatibility and security checks for that run. It does not mean the server is universally safe for every environment.
127
+
128
+ Use the results as an engineering control:
129
+
130
+ - add CI for repeatability
131
+ - compare artifacts between releases
132
+ - review security findings and suppressions
133
+ - document accepted risk for broad tools
134
+ - escalate production/private usage to hosted reporting, certification, or fleet visibility when the server becomes operationally important