@kryptosai/mcp-observatory 0.23.0 → 0.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/COMMERCIAL.md +14 -0
  2. package/README.md +21 -12
  3. package/dist/src/cli.js +2 -2
  4. package/dist/src/cli.js.map +1 -1
  5. package/dist/src/commands/init-ci.d.ts +3 -0
  6. package/dist/src/commands/init-ci.js +42 -22
  7. package/dist/src/commands/init-ci.js.map +1 -1
  8. package/dist/src/commands/test.js +20 -0
  9. package/dist/src/commands/test.js.map +1 -1
  10. package/dist/src/commercial.js +7 -0
  11. package/dist/src/commercial.js.map +1 -1
  12. package/dist/src/reporters/common.js +2 -2
  13. package/dist/src/reporters/common.js.map +1 -1
  14. package/dist/src/reporters/pr-comment.js +6 -2
  15. package/dist/src/reporters/pr-comment.js.map +1 -1
  16. package/docs/certification-campaign-template.md +4 -4
  17. package/docs/certification-distribution.md +1 -1
  18. package/docs/directory-listing-copy.md +1 -1
  19. package/docs/enterprise-outreach-playbook.md +10 -10
  20. package/docs/mcp-lock-files.md +1 -1
  21. package/docs/mcp-safety-report-latest.md +14 -9
  22. package/docs/mcp-security-field-guide.md +1 -1
  23. package/docs/mcp-server-safety-index.md +56 -80
  24. package/docs/methodology.md +90 -0
  25. package/docs/metrics-dashboard.md +108 -0
  26. package/docs/paid-pilot-offer.md +21 -5
  27. package/docs/project-case-study.md +12 -8
  28. package/docs/proof.md +29 -16
  29. package/docs/public-post-drafts.md +18 -6
  30. package/docs/publish-readiness.md +1 -5
  31. package/docs/reference-evaluations.md +3 -3
  32. package/docs/safety-index/artifacts/antv-chart-server.json +2765 -0
  33. package/docs/safety-index/artifacts/antv-chart-server.md +156 -0
  34. package/docs/safety-index/artifacts/browsermcp-server.json +416 -0
  35. package/docs/safety-index/artifacts/browsermcp-server.md +163 -0
  36. package/docs/safety-index/artifacts/context7-server.json +286 -0
  37. package/docs/safety-index/artifacts/context7-server.md +163 -0
  38. package/docs/safety-index/artifacts/everything-server.json +482 -0
  39. package/docs/safety-index/artifacts/everything-server.md +163 -0
  40. package/docs/safety-index/artifacts/executeautomation-playwright-server.json +955 -0
  41. package/docs/safety-index/artifacts/executeautomation-playwright-server.md +163 -0
  42. package/docs/safety-index/artifacts/filesystem-server.json +583 -0
  43. package/docs/safety-index/artifacts/filesystem-server.md +156 -0
  44. package/docs/safety-index/artifacts/memory-server.json +469 -0
  45. package/docs/safety-index/artifacts/memory-server.md +156 -0
  46. package/docs/safety-index/artifacts/opentofu-server.json +387 -0
  47. package/docs/safety-index/artifacts/opentofu-server.md +163 -0
  48. package/docs/safety-index/artifacts/playwright-mcp-server.json +919 -0
  49. package/docs/safety-index/artifacts/playwright-mcp-server.md +156 -0
  50. package/docs/safety-index/artifacts/promptopia-server.json +442 -0
  51. package/docs/safety-index/artifacts/promptopia-server.md +156 -0
  52. package/docs/safety-index/artifacts/puppeteer-server.json +377 -0
  53. package/docs/safety-index/artifacts/puppeteer-server.md +163 -0
  54. package/docs/safety-index/artifacts/ref-tools-server.json +262 -0
  55. package/docs/safety-index/artifacts/ref-tools-server.md +156 -0
  56. package/docs/safety-index/artifacts/sequential-thinking-server.json +286 -0
  57. package/docs/safety-index/artifacts/sequential-thinking-server.md +156 -0
  58. package/docs/safety-index/maintainer-note-template.md +25 -0
  59. package/docs/safety-index/targets.json +192 -0
  60. package/package.json +16 -9
@@ -42,7 +42,7 @@ Skip:
42
42
  | 5 | `browserbase/mcp-server-browserbase` | `npx -y @browserbasehq/mcp-server-browserbase` | Browser Automation | Hosted browser MCP category | verify auth-free startup | May require API key; issue-only if startup requires credentials | researched | | |
43
43
  | 6 | `smithery-ai/server-sequential-thinking` | `npx -y @smithery-ai/server-sequential-thinking` | Developer Tools | MCP directory ecosystem | verify package/repo naming | Good low-risk simple server if public package starts cleanly | researched | | |
44
44
  | 7 | `kazuph/mcp-taskmanager` | `npx -y @kazuph/mcp-taskmanager@latest` | Developer Tools | 200+ stars / task/project MCP category | Local validation passed: 10 tools | Scoped package name corrected from tracker | pr-opened | https://github.com/kazuph/mcp-taskmanager/pull/11 | Waiting for maintainer review |
45
- | 8 | `cyanheads/filesystem-mcp-server` | `node dist/index.js` | Filesystem | Filesystem MCP category | Local validation passed after fix: 10 tools | Fixed real conformance bug: advertised resources without `resources/list`; workflow uses temp sandbox | pr-opened | https://github.com/cyanheads/filesystem-mcp-server/pull/19 | Waiting for maintainer review |
45
+ | 8 | `cyanheads/filesystem-mcp-server` | `node dist/index.js` | Filesystem | Filesystem MCP category | Local validation passed after fix: 10 tools | Fixed real conformance bug: advertised resources without `resources/list`; workflow uses temp sandbox | closed-unmerged | https://github.com/cyanheads/filesystem-mcp-server/pull/19 | Closed by maintainer without merge |
46
46
  | 9 | `redis/mcp-redis` | `uvx mcp-redis` | Database | Enterprise database category | verify auth-free startup | Database target may require service; issue-only if credentials needed | researched | | |
47
47
  | 10 | `mongodb-js/mongodb-mcp-server` | `npx -y mongodb-mcp-server` | Database | Enterprise database category | verify auth-free startup | Likely needs connection string; issue-only first | researched | | |
48
48
  | 11 | `supabase-community/supabase-mcp` | `npx -y supabase-mcp` | Database | Enterprise/SaaS category | verify current package | Likely requires token; issue-only first | researched | | |
@@ -54,7 +54,7 @@ Skip:
54
54
  | 17 | `BrowserMCP/mcp` | `npx -y @browsermcp/mcp` | Browser Automation | 6k+ stars / browser-control MCP category | Local validation passed: 12 tools | Browser automation trust boundary; workflow is inventory/security only | pr-opened | https://github.com/BrowserMCP/mcp/pull/189 | Waiting for maintainer review |
55
55
  | 18 | `UI5/mcp-server` | `npx -y @ui5/mcp-server` | Developer Tools | SAP/UI5 ecosystem MCP package | Local validation passed: 10 tools | Developer tooling surface; no credentials required for inventory | pr-opened | https://github.com/UI5/mcp-server/pull/348 | Waiting for maintainer review |
56
56
  | 19 | `apify/actors-mcp-server` | `npx -y @apify/actors-mcp-server` | SaaS/API | Automation platform category | verify auth-free startup | Likely requires token; issue-only first | researched | | |
57
- | 20 | `makenotion/notion-mcp-server` | `npx -y @notionhq/notion-mcp-server` | SaaS/API | Major SaaS category | Local validation passed: 24 tools | Workspace-data MCP; PR is compatibility/schema/security inventory only | pr-opened | https://github.com/makenotion/notion-mcp-server/pull/324 | Waiting for maintainer review |
57
+ | 20 | `makenotion/notion-mcp-server` | `npx -y @notionhq/notion-mcp-server` | SaaS/API | Major SaaS category | Local validation passed: 24 tools | Workspace-data MCP; PR is compatibility/schema/security inventory only | pr-opened | https://github.com/makenotion/notion-mcp-server/pull/324 | Waiting for maintainer review; external Semgrep check failing |
58
58
  | 21 | `linear/linear-mcp` | `npx -y @linear/mcp-server` | SaaS/API | Developer SaaS category | verify package | Likely requires token; issue-only first | researched | | |
59
59
  | 22 | `sentry/sentry-mcp` | `npx -y @sentry/mcp-server` | Observability | Developer SaaS category | verify package | Likely requires token; issue-only first | researched | | |
60
60
  | 23 | `elastic/mcp-server-elasticsearch` | `npx -y @elastic/mcp-server-elasticsearch` | Search | Enterprise search category | verify package | Likely requires service; issue-only first | researched | | |
@@ -100,7 +100,7 @@ Statuses:
100
100
  ## PR Checklist
101
101
 
102
102
  - Generate the local kit first:
103
- `npx @kryptosai/mcp-observatory init-ci --all --command "<safe startup command>"`
103
+ `npx @kryptosai/mcp-observatory setup-ci --all --command "<safe startup command>"`
104
104
  - Add `.github/workflows/mcp-observatory.yml`
105
105
  - Add `mcp-observatory.target.json` when the startup command needs args, cwd, or env placeholders
106
106
  - Use `deep: true` and `security: true`
@@ -151,7 +151,7 @@ The goal is to give users a visible compatibility/security signal and catch sche
151
151
 
152
152
  ## Generated PR Body Printer
153
153
 
154
- After running `init-ci --all`, print the generated maintainer copy with:
154
+ After running `setup-ci --all`, print the generated maintainer copy with:
155
155
 
156
156
  ```bash
157
157
  npm run certification:pr-body -- docs/mcp-observatory-pr-body.md
@@ -35,7 +35,7 @@ npx @kryptosai/mcp-observatory badge npx -y <server-package> --output docs/mcp-h
35
35
  Fast path:
36
36
 
37
37
  ```bash
38
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y <server-package>"
38
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y <server-package>"
39
39
  ```
40
40
 
41
41
  That creates:
@@ -25,7 +25,7 @@ For security and platform teams, see the MCP Server Security Field Guide and MCP
25
25
  Add MCP CI in one command:
26
26
 
27
27
  ```bash
28
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y my-mcp-server"
28
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
29
29
  ```
30
30
 
31
31
  ## Tags
@@ -3,36 +3,36 @@
3
3
  Use this playbook after running:
4
4
 
5
5
  ```bash
6
- npm run telemetry:intelligence -- --input telemetry-exports/events-flat-full.json --out-dir reports
6
+ npm run telemetry:intelligence -- --input <private-telemetry-export.jsonl> --out-dir reports
7
7
  ```
8
8
 
9
9
  Start from `reports/telemetry-usage-summary.html` to confirm external usage before reading account rankings. Do not treat first-party CI, release workflows, or internal/personal sessions as market traction.
10
10
 
11
- Raw telemetry is allowed for internal account intelligence and may include git email, git remote URL, hostname, target command or URL, CI metadata, target IDs, and command outcomes. Do not include raw personal emails, hostnames, private URLs, target commands, tokens, or private telemetry exports in public issues, posts, docs, or customer-facing outreach. Use account domains, GitHub orgs, and aggregate telemetry evidence.
11
+ Raw telemetry is allowed for internal account intelligence only. Do not include personal emails, hostnames, private URLs, target commands, tokens, proprietary schemas, customer names, or private telemetry exports in public issues, posts, docs, or customer-facing outreach without explicit permission. Keep account-specific rankings in ignored `reports/` outputs or private notes.
12
12
 
13
13
  ## Priority Accounts
14
14
 
15
15
  | Priority | Account | Evidence | Motion |
16
16
  | ---: | --- | --- | --- |
17
- | 1 | `thinkingdata.cn` | High-confidence external usage, repeated sessions, private-network target signal, Feishu/Lark MCP targets | Enterprise pilot |
18
- | 2 | `kimquy.capital` | Recent light usage across multiple sessions | Business pilot / design partner |
19
- | 3 | `paperstreetdata.com` | Small usage cluster | Business pilot / testimonial ask |
20
- | 4 | `cyberneticsplus.com` | Single company signal | Team pilot / feedback ask |
17
+ | 1 | Private account A | Repeated external sessions, CI usage, private-target signal, or security workflow evidence | Enterprise pilot |
18
+ | 2 | Private account B | Recent usage across multiple sessions or repos | Business pilot / design partner |
19
+ | 3 | Private account C | Small usage cluster with clear owner signal | Business pilot / testimonial ask |
20
+ | 4 | Private account D | Single company or team signal | Team pilot / feedback ask |
21
21
  | 5 | GitHub org/user signals | CI or repo-based usage | Team pilot unless company identity is confirmed |
22
22
 
23
- ## ThinkingData First Email
23
+ ## First Email Template
24
24
 
25
- Subject: MCP security reporting for Feishu/Lark production workflows
25
+ Subject: MCP security reporting for production agent workflows
26
26
 
27
27
  Hi,
28
28
 
29
29
  I build MCP Observatory, an open source tool for testing, securing, and monitoring MCP servers before agents depend on them.
30
30
 
31
- We are seeing serious production-style usage patterns around Feishu/Lark MCP workflows and internal HTTP MCP targets. I am opening a small number of enterprise pilots for teams that want hosted MCP security reports, private-repo CI history, and fleet visibility across agent environments.
31
+ I am opening a small number of enterprise pilots for teams that want hosted MCP security reports, private-repo CI history, and fleet visibility across agent environments.
32
32
 
33
33
  If your team is running MCP servers in production, I can prepare a short evidence-based report and a pilot proposal focused on:
34
34
 
35
- - Feishu/Lark MCP compatibility
35
+ - MCP compatibility
36
36
  - private HTTP MCP health checks
37
37
  - security findings and schema drift
38
38
  - CI history and controlled drift review
@@ -21,7 +21,7 @@ npx @kryptosai/mcp-observatory lock verify
21
21
  Add CI:
22
22
 
23
23
  ```bash
24
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y my-mcp-server"
24
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
25
25
  ```
26
26
 
27
27
  ## Why It Matters
@@ -24,13 +24,18 @@ Safe aggregate telemetry from the latest local export:
24
24
 
25
25
  | Metric | Value |
26
26
  | --- | ---: |
27
- | Total telemetry events | 10,918 |
28
- | Total sessions | 7,380 |
29
- | External sessions | 5,379 |
27
+ | Total telemetry events | 11,481 |
28
+ | Total sessions | 7,571 |
29
+ | External sessions | 5,389 |
30
30
  | External CI sessions | 2,446 |
31
- | Attributed company/org sessions | 138 |
32
- | GitHub clones in visible traffic window | 721 |
33
- | Unique cloners in visible traffic window | 221 |
31
+ | Attributed company/org sessions | 148 |
32
+ | Attributed company/org candidates | 12 |
33
+ | Latest external activity | June 21, 2026 |
34
+ | npm downloads snapshot | 511 downloads, June 11-20, 2026 |
35
+ | GitHub clones in visible traffic window | 745 |
36
+ | Unique cloners in visible traffic window | 232 |
37
+ | GitHub page views in visible traffic window | 12 |
38
+ | Unique GitHub viewers in visible traffic window | 9 |
34
39
 
35
40
  Top external commands:
36
41
 
@@ -58,7 +63,7 @@ From public sample artifacts and Observatory check categories, the most importan
58
63
  ## How Maintainers Add The Check
59
64
 
60
65
  ```bash
61
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y my-mcp-server"
66
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
62
67
  ```
63
68
 
64
69
  That creates a GitHub Action and a README badge snippet. The action can comment on PRs and fail when MCP compatibility or security checks regress.
@@ -74,7 +79,7 @@ Production teams can use MCP Observatory for:
74
79
  - support and rollout review
75
80
  - fleet visibility across teams and repos
76
81
 
77
- Contact `william@banksey.com` for pilots.
82
+ See [Commercial Pilots](../COMMERCIAL.md) for production/private MCP usage.
78
83
 
79
84
  ## Launch Post
80
85
 
@@ -83,7 +88,7 @@ MCP servers are becoming production dependencies.
83
88
  I built MCP Observatory so MCP maintainers can add CI, security checks, schema drift detection, PR reports, and trust badges in one command:
84
89
 
85
90
  ```bash
86
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y my-mcp-server"
91
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
87
92
  ```
88
93
 
89
94
  Free for local OSS use. Paid pilots are available for hosted reporting, private repo CI, certification, support, and fleet visibility.
@@ -60,7 +60,7 @@ A useful MCP server CI gate should answer a few operational questions:
60
60
  MCP Observatory is designed to make that a one-command adoption path:
61
61
 
62
62
  ```bash
63
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y my-mcp-server"
63
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
64
64
  ```
65
65
 
66
66
  For a direct check:
@@ -1,85 +1,61 @@
1
- # MCP Server Safety Index
2
-
3
- The MCP Server Safety Index is a public, reproducible way to show how MCP servers behave under compatibility, schema quality, drift, and security checks.
4
-
5
- The goal is constructive proof, not callouts. Each entry shows what should be tested, how to reproduce it, what risk class matters, and what a maintainer can do next.
6
-
7
- ## Index v0
8
-
9
- | # | Server | Category | Reproducible Command | What To Check | Risk Class | Status |
10
- | ---: | --- | --- | --- | --- | --- | --- |
11
- | 1 | [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) sequential thinking | Reference | `npx -y @modelcontextprotocol/server-sequential-thinking@latest` | Startup, tools/list, schema quality, security-lite | Reference compatibility | PR open: [#4392](https://github.com/modelcontextprotocol/servers/pull/4392) |
12
- | 2 | [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) filesystem | Filesystem | `npx -y @modelcontextprotocol/server-filesystem .` | Startup in harmless temp dir, path tools, schema quality | Filesystem boundary | Researched |
13
- | 3 | [`upstash/context7`](https://github.com/upstash/context7) | Documentation/search | `npx -y @upstash/context7-mcp@latest` | Startup, retrieval tools, schemas, prompt-injection-sensitive text flow | Untrusted content retrieval | PR open: [#2800](https://github.com/upstash/context7/pull/2800) |
14
- | 4 | [`executeautomation/mcp-playwright`](https://github.com/executeautomation/mcp-playwright) | Browser automation | `npx -y @executeautomation/playwright-mcp-server@latest` | Browser tools, schema quality, intentional code-eval suppressions | Browser/code execution | PR open: [#225](https://github.com/executeautomation/mcp-playwright/pull/225) |
15
- | 5 | [`microsoft/playwright-mcp`](https://github.com/microsoft/playwright-mcp) | Browser automation | `npx -y @playwright/mcp@latest` | Browser tools, skip-invoke policy, schema quality, suppressions | Browser/code execution | PR open: [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) |
16
- | 6 | [`kazuph/mcp-taskmanager`](https://github.com/kazuph/mcp-taskmanager) | Developer tools | `npx -y @kazuph/mcp-taskmanager@latest` | Task tools, schema quality, mutation clarity | Project/task mutation | PR open: [#11](https://github.com/kazuph/mcp-taskmanager/pull/11) |
17
- | 7 | [`cyanheads/filesystem-mcp-server`](https://github.com/cyanheads/filesystem-mcp-server) | Filesystem | `node dist/index.js` | Capability declarations, resources/list, sandboxed filesystem target | Filesystem boundary | PR open: [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) |
18
- | 8 | [`browserbase/mcp-server-browserbase`](https://github.com/browserbase/mcp-server-browserbase) | Browser automation | `npx -y @browserbasehq/mcp-server-browserbase` | Auth-free startup, browser tools, network/browser boundaries | Hosted browser control | Researched; likely needs API key |
19
- | 9 | [`redis/mcp-redis`](https://github.com/redis/mcp-redis) | Database | `uvx mcp-redis` | Startup without live database, command surface, destructive operations | Data mutation | Researched; may need service |
20
- | 10 | [`mongodb-js/mongodb-mcp-server`](https://github.com/mongodb-js/mongodb-mcp-server) | Database | `npx -y mongodb-mcp-server` | Connection handling, read/write tools, auth posture | Data mutation/auth | Researched; likely needs connection string |
21
- | 11 | [`supabase-community/supabase-mcp`](https://github.com/supabase-community/supabase-mcp) | Database/SaaS | `npx -y supabase-mcp` | Startup, token handling, project mutation tools | Cloud data access | Researched; likely needs token |
22
- | 12 | [`cloudflare/mcp-server-cloudflare`](https://github.com/cloudflare/mcp-server-cloudflare) | Cloud | `npx -y @cloudflare/mcp-server-cloudflare` | Auth posture, deploy/config tools, schema clarity | Cloud control plane | Researched; likely needs auth |
23
- | 13 | [`stripe/agent-toolkit`](https://github.com/stripe/agent-toolkit) | Payments | `npx -y @stripe/agent-toolkit` | MCP mode, payment/customer mutation tools, auth posture | Payments/destructive action | Researched; likely needs API key |
24
- | 14 | [`github/github-mcp-server`](https://github.com/github/github-mcp-server) | Developer tools | `docker run ghcr.io/github/github-mcp-server` | Auth handling, repo mutation tools, schema clarity | Source-code control | Researched; likely needs token |
25
- | 15 | [`jetbrains/mcpProxy`](https://github.com/JetBrains/mcpProxy) | IDE/developer tools | `npx -y @jetbrains/mcp-proxy` | IDE dependency, startup behavior, tool surface | Local IDE control | Researched; may need IDE process |
26
- | 16 | [`BrowserMCP/mcp`](https://github.com/BrowserMCP/mcp) | Browser automation | `npx -y @browsermcp/mcp` | Browser tools, schema quality, browser-control boundary | Browser control | PR open: [#189](https://github.com/BrowserMCP/mcp/pull/189) |
27
- | 17 | [`UI5/mcp-server`](https://github.com/UI5/mcp-server) | Developer tools | `npx -y @ui5/mcp-server` | UI5 tooling commands, schema quality, drift risk | App development tooling | PR open: [#348](https://github.com/UI5/mcp-server/pull/348) |
28
- | 18 | [`antvis/mcp-server-chart`](https://github.com/antvis/mcp-server-chart) | Visualization/data | `npx -y @antv/mcp-server-chart` | Chart generation tools, schema quality, artifact-producing tools | Generated artifacts | PR open: [#312](https://github.com/antvis/mcp-server-chart/pull/312) |
29
- | 19 | [`makenotion/notion-mcp-server`](https://github.com/makenotion/notion-mcp-server) | SaaS/API | `npx -y @notionhq/notion-mcp-server` | Auth handling, read/write tool separation, schema quality | Workspace data access | PR open: [#324](https://github.com/makenotion/notion-mcp-server/pull/324) |
30
- | 20 | [`sentry/sentry-mcp`](https://github.com/getsentry/sentry-mcp) | Developer SaaS | `npx -y @sentry/mcp-server` | Auth handling, issue/project tools, schema quality | Production incident data | Researched; likely needs token |
31
-
32
- ## Evaluation Command
33
-
34
- For simple npm-backed servers:
35
-
36
- ```bash
37
- npx @kryptosai/mcp-observatory test --security npx -y <server-package>
38
- ```
39
-
40
- For safer campaign PRs:
41
-
42
- ```bash
43
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y <server-package>"
44
- ```
45
-
46
- For production-style review:
47
-
48
- ```bash
49
- npx @kryptosai/mcp-observatory lock
50
- npx @kryptosai/mcp-observatory lock verify
51
- ```
52
-
53
- ## What Each Column Means
54
-
55
- - What To Check: the minimum compatibility/security surface a maintainer or platform team should inspect.
56
- - Risk Class: the operational reason the server matters before agents depend on it.
57
- - Status: public proof such as PR open, PR accepted, badge added, researched, or needs maintainer review.
1
+ # MCP Server Safety Index v1
2
+
3
+ The MCP Server Safety Index is an evidence standard for MCP readiness. It is not a leaderboard and does not rank maintainers.
4
+
5
+ Each row links to a reproducible command, a JSON run artifact, and a Markdown report generated by MCP Observatory. The goal is to show which failure classes matter before teams let agents depend on MCP servers.
6
+
7
+ For the rules behind this page, see the [Safety Methodology](./methodology.md).
8
+
9
+ ## Snapshot
10
+
11
+ - Evaluated servers: 13
12
+ - Ready for CI: 10
13
+ - Needs review before production: 1
14
+ - Unsafe default posture: 2
15
+ - Not reproducible: 0
16
+ - Latest run: 2026-06-24T02:07:44.894Z
17
+
18
+ ## Evaluations
19
+
20
+ | # | Server | Category | Verdict | Failure Class | Reproduce | Evidence | Notes |
21
+ | ---: | --- | --- | --- | --- | --- | --- | --- |
22
+ | 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
23
+ | 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
24
+ | 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
25
+ | 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
26
+ | 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **Ready for CI** | Prompt-injection-sensitive retrieval | `npx -y @upstash/context7-mcp` | [JSON](./safety-index/artifacts/context7-server.json) / [report](./safety-index/artifacts/context7-server.md) | Zero-config public package. |
27
+ | 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | Prompt/resource contract | `npx -y promptopia-mcp` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
28
+ | 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | Prompt/tool inventory | `npx -y ref-tools-mcp` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
29
+ | 8 | [OpenTofu MCP server](https://github.com/opentofu/opentofu-mcp-server) | Infrastructure | **Ready for CI** | Infrastructure tool surface | `npx -y @opentofu/opentofu-mcp-server` | [JSON](./safety-index/artifacts/opentofu-server.json) / [report](./safety-index/artifacts/opentofu-server.md) | Zero-config public package. |
30
+ | 9 | [Puppeteer MCP server](https://www.npmjs.com/package/puppeteer-mcp-server) | Browser Automation | **Ready for CI** | Browser/code execution boundary | `npx -y puppeteer-mcp-server` | [JSON](./safety-index/artifacts/puppeteer-server.json) / [report](./safety-index/artifacts/puppeteer-server.md) | Intentional browser evaluation is suppressed so remaining findings stay readable. |
31
+ | 10 | [BrowserMCP](https://github.com/BrowserMCP/mcp) | Browser Automation | **Ready for CI** | Browser-control boundary | `npx -y @browsermcp/mcp` | [JSON](./safety-index/artifacts/browsermcp-server.json) / [report](./safety-index/artifacts/browsermcp-server.md) | Zero-config public package. [public proof](https://github.com/BrowserMCP/mcp/pull/189) |
32
+ | 11 | [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) | Browser Automation | **Unsafe default posture** | Browser/code execution boundary | `npx -y @playwright/mcp` | [JSON](./safety-index/artifacts/playwright-mcp-server.json) / [report](./safety-index/artifacts/playwright-mcp-server.md) | Zero-config public package; security findings represent policy-review prompts, not a vulnerability claim. |
33
+ | 12 | [AntV chart MCP server](https://github.com/antvis/mcp-server-chart) | Visualization | **Ready for CI** | Artifact-producing tools | `npx -y @antv/mcp-server-chart` | [JSON](./safety-index/artifacts/antv-chart-server.json) / [report](./safety-index/artifacts/antv-chart-server.md) | Zero-config public package. [public proof](https://github.com/antvis/mcp-server-chart/pull/312) |
34
+ | 13 | [ExecuteAutomation Playwright MCP](https://github.com/executeautomation/mcp-playwright) | Browser Automation | **Unsafe default posture** | Startup/listing reproducibility | `npx -y @executeautomation/playwright-mcp-server` | [JSON](./safety-index/artifacts/executeautomation-playwright-server.json) / [report](./safety-index/artifacts/executeautomation-playwright-server.md) | Evaluated as a public package; current result should be treated as a maintainer conversation starter. [public proof](https://github.com/executeautomation/mcp-playwright/pull/225) |
35
+
36
+ ## Patterns Observed
37
+
38
+ - Browser/code execution boundary: 2 server(s)
39
+ - Artifact-producing tools: 1 server(s)
40
+ - Broad protocol surface: 1 server(s)
41
+ - Browser-control boundary: 1 server(s)
42
+ - Infrastructure tool surface: 1 server(s)
43
+ - Persistent state tools: 1 server(s)
44
+ - Prompt-injection-sensitive retrieval: 1 server(s)
45
+ - Prompt/resource contract: 1 server(s)
46
+ - Prompt/tool inventory: 1 server(s)
47
+ - Sandboxed filesystem access: 1 server(s)
48
+ - Startup/listing reproducibility: 1 server(s)
49
+ - Tool schema clarity: 1 server(s)
58
50
 
59
51
  ## Publication Rules
60
52
 
61
- - Use only public repositories, public package commands, public PRs, or sample artifacts.
62
- - Include a reproduction command for every row.
63
- - Link to the maintainer PR or public artifact when available.
64
- - Phrase findings constructively: “needs review” rather than “unsafe” unless there is clear public proof.
65
- - Keep customer/domain telemetry internal unless the customer gives permission or there is independent public evidence.
66
-
67
- ## Five Patterns To Publish From v0
68
-
69
- 1. Browser automation MCP servers need explicit policy around code execution, screenshots, navigation, and mutation.
70
- 2. Filesystem MCP servers need harmless CI sandboxes and clear read/write boundaries.
71
- 3. SaaS and cloud MCP servers often cannot be meaningfully checked without token-safe target configs.
72
- 4. Database MCP servers need read/write classification and connection-string hygiene before CI rollout.
73
- 5. Lock files turn MCP surface drift into a reviewable PR event instead of an invisible agent dependency change.
74
-
75
- ## Next Wave Criteria
76
-
77
- Prioritize 20-50 servers that have:
53
+ - Use only public repositories, public package commands, public PRs, and generated sanitized artifacts.
54
+ - Treat findings as reproducible evidence, not public shaming.
55
+ - Prefer “needs review” language unless there is clear artifact-backed proof of a dangerous default.
56
+ - Keep raw telemetry, emails, hostnames, private URLs, tokens, and customer claims out of public materials.
57
+ - Send maintainers the report first; open CI PRs only when the report is useful and the target can run safely.
78
58
 
79
- - active maintenance in the last 90 days
80
- - visible stars, downloads, or directory listings
81
- - simple `npx`, `uvx`, or Docker startup commands
82
- - enterprise-relevant categories such as browser automation, filesystem, documentation/search, databases, cloud, productivity, and developer tools
83
- - no existing MCP compatibility/security CI
59
+ ## Next Step
84
60
 
85
- One accepted PR in a respected repo is worth more than a large list of shallow checks.
61
+ Use this index to start maintainer conversations and private readiness reviews. The buyer-facing offer is a private MCP readiness review with CI rollout, drift/security reporting, and safe-for-agent-dependency verdicts.
@@ -0,0 +1,90 @@
1
+ # MCP Observatory Safety Methodology
2
+
3
+ MCP Observatory treats MCP servers as agent-facing dependencies. The Safety Index is designed to answer one practical question:
4
+
5
+ > Is this server ready for agents and teams to depend on, and what evidence supports that answer?
6
+
7
+ The index is not a leaderboard. It is a reproducible evidence standard for maintainers, security teams, platform teams, and buyers evaluating MCP servers.
8
+
9
+ ## What Gets Tested
10
+
11
+ Each public evaluation runs MCP Observatory against a public repository or package command. A useful entry includes:
12
+
13
+ - server name and public source
14
+ - exact command and arguments
15
+ - run date
16
+ - MCP Observatory version
17
+ - JSON run artifact
18
+ - Markdown report
19
+ - verdict
20
+ - failure class
21
+ - reproduction notes
22
+
23
+ The default public check verifies startup, tools, prompts, resources, schema quality, and lightweight security findings. Some entries also include deeper security checks when the target can be evaluated without private credentials.
24
+
25
+ ## Verdicts
26
+
27
+ - **Ready for CI:** the server starts, lists expected MCP surfaces, and has no high- or medium-severity security finding in the generated artifact.
28
+ - **Needs review before production:** the server is reproducible but has findings or partial results a maintainer/security reviewer should inspect before production use.
29
+ - **Not reproducible:** the server cannot complete a basic startup or listing check from the documented public command.
30
+ - **Unsafe default posture:** the artifact contains high-severity security findings that deserve explicit policy review before agent dependency.
31
+ - **Could not evaluate:** the public command cannot be evaluated without credentials, private infrastructure, or maintainer-provided safe configuration.
32
+
33
+ These verdicts are intentionally operational. They are not formal vulnerability claims.
34
+
35
+ ## Scoring Inputs
36
+
37
+ MCP Observatory uses the same run artifact model across CLI, CI, reports, and the Safety Index. The health score considers:
38
+
39
+ - protocol compliance
40
+ - schema quality
41
+ - security and security-lite checks
42
+ - reliability/startup behavior
43
+ - performance where latency data is available
44
+
45
+ The Safety Index does not rank by score. Scores are supporting evidence; failure classes are the story.
46
+
47
+ ## Failure Classes
48
+
49
+ Common MCP readiness patterns include:
50
+
51
+ - startup/listing reproducibility
52
+ - browser/code execution boundary
53
+ - filesystem boundary
54
+ - prompt-injection-sensitive retrieval
55
+ - persistent state tools
56
+ - infrastructure or cloud control surfaces
57
+ - artifact-producing tools
58
+ - schema clarity and drift
59
+ - token-safe configuration
60
+
61
+ The first public index emphasizes these classes so maintainers can improve concrete surfaces rather than argue about a single trust score.
62
+
63
+ ## Reproducibility Rules
64
+
65
+ An index row should be included only when it can be reproduced from public information:
66
+
67
+ - public repo, package, or container reference
68
+ - no private telemetry
69
+ - no private customer evidence
70
+ - no raw emails, hostnames, private URLs, tokens, or response bodies
71
+ - a generated JSON artifact and Markdown report
72
+ - clear notes when credentials or maintainer context are required
73
+
74
+ If the safe public command is not known, the right next step is a maintainer note, not a drive-by CI PR.
75
+
76
+ ## Maintainer Posture
77
+
78
+ The index is constructive by default:
79
+
80
+ - send the report first
81
+ - describe the failure class, not the maintainer
82
+ - offer a CI PR only if the target can run safely and the maintainer wants it
83
+ - prefer read-only workflows and pinned action refs for third-party repos
84
+ - use issue-first outreach for token-backed SaaS, cloud, payments, database, and browser-control servers
85
+
86
+ ## Limitations
87
+
88
+ MCP Observatory cannot prove semantic safety. A passing result does not mean a server is safe for every workflow. It means the server produced reproducible evidence for compatibility, schema quality, and common security footguns under the tested command.
89
+
90
+ Production teams should pair these checks with their own threat model, policy, credential boundaries, sandboxing, approvals, and runtime monitoring.
@@ -0,0 +1,108 @@
1
+ # Local Metrics Dashboard
2
+
3
+ MCP Observatory includes a private laptop dashboard for collecting telemetry, GitHub activity, and npm download data going forward.
4
+
5
+ The dashboard is intentionally local. It stores raw telemetry in a SQLite database on your laptop, then renders a sanitized static HTML view that is safe to screenshot or skim without exposing raw emails, hostnames, private URLs, tokens, or private command bodies.
6
+
7
+ The layout follows an App Store Connect-style breakdown:
8
+
9
+ - Overview
10
+ - Strategy
11
+ - Acquisition
12
+ - Downloads
13
+ - Usage
14
+ - Reliability
15
+
16
+ The Strategy section turns raw telemetry into operator decisions: command funnel, version adoption, account/domain drilldown, CI/local source mix, and release-spike context.
17
+
18
+ Daily rows are shown newest to oldest so the most recent project activity is always at the top.
19
+
20
+ ## Refresh
21
+
22
+ ```bash
23
+ npm run metrics:refresh
24
+ npm run metrics:serve
25
+ ```
26
+
27
+ Outputs are written to:
28
+
29
+ - `.mcp-observatory-metrics/observatory.sqlite`
30
+ - `.mcp-observatory-metrics/dashboard/index.html`
31
+ - `.mcp-observatory-metrics/dashboard/latest.json`
32
+ - `.mcp-observatory-metrics/logs/`
33
+
34
+ The local metrics directory is ignored by git and should not be published.
35
+
36
+ ## Data Sources
37
+
38
+ The collector stores each source independently. If one source fails, the dashboard still builds from the last good SQLite data and records the failure in the Reliability section.
39
+
40
+ | Source | Data |
41
+ | --- | --- |
42
+ | Telemetry | Existing Cloudflare D1 export flow via `scripts/export-telemetry-d1.ts` |
43
+ | GitHub | clones, views, referrers, popular paths, repo snapshot, latest release, open issue/PR counts, workflow runs |
44
+ | npm | public daily downloads for `@kryptosai/mcp-observatory` |
45
+
46
+ GitHub traffic APIs have a limited visible window, so the collector stores snapshots locally going forward. npm daily buckets can lag; the dashboard labels npm data as complete public days rather than assuming current-day zero.
47
+
48
+ ## Credentials
49
+
50
+ GitHub collection uses either:
51
+
52
+ ```bash
53
+ gh auth login
54
+ ```
55
+
56
+ or:
57
+
58
+ ```bash
59
+ export GH_TOKEN=...
60
+ ```
61
+
62
+ Telemetry collection uses the same Wrangler configuration as the existing telemetry export script. If the config is not auto-discovered, set:
63
+
64
+ ```bash
65
+ export MCP_OBSERVATORY_TELEMETRY_WRANGLER_CONFIG=/path/to/wrangler.toml
66
+ export MCP_OBSERVATORY_TELEMETRY_D1_DATABASE=mcp-observatory-telemetry
67
+ ```
68
+
69
+ No npm token is required for public download counts.
70
+
71
+ ## Commands
72
+
73
+ ```bash
74
+ npm run metrics:collect # collect telemetry, GitHub, and npm into SQLite
75
+ npm run metrics:build # render HTML from the existing SQLite database
76
+ npm run metrics:refresh # collect + build
77
+ npm run metrics:open # open the static read-only dashboard
78
+ npm run metrics:serve # open the local dashboard with an Update Data button
79
+ ```
80
+
81
+ The **Update Data** button is available in `metrics:serve` mode. A static `file://` dashboard cannot run local commands from the browser, so `metrics:open` shows the same data but disables the button.
82
+
83
+ For tests or offline recovery, you can seed telemetry from an existing export:
84
+
85
+ ```bash
86
+ npm run metrics:refresh -- --telemetry-input telemetry-exports/events-flat-full.json
87
+ ```
88
+
89
+ ## Optional Hourly Refresh
90
+
91
+ Generate a small local refresh wrapper:
92
+
93
+ ```bash
94
+ npx tsx scripts/metrics-dashboard.ts scheduler
95
+ ```
96
+
97
+ The generated script lives under `.mcp-observatory-metrics/` and can be called by `launchd` or cron. It uses the same refresh command and writes logs to `.mcp-observatory-metrics/logs/refresh.log`.
98
+
99
+ ## Privacy
100
+
101
+ Raw telemetry remains available in the local SQLite database for account intelligence and product analytics. The dashboard view uses sanitized aggregates by default:
102
+
103
+ - company domains, not raw emails
104
+ - aggregate source and command counts, not private command bodies
105
+ - GitHub and npm public metrics
106
+ - collection errors and freshness status
107
+
108
+ Do not commit `.mcp-observatory-metrics/`, telemetry exports, private reports, or screenshots that reveal raw data.
@@ -17,11 +17,13 @@ This is a manual pilot, not a self-serve SaaS promise.
17
17
 
18
18
  ## What The Pilot Includes
19
19
 
20
- - review of the customer’s MCP config, repo, or startup commands
21
- - MCP Observatory CI rollout for selected servers
20
+ - MCP server inventory across selected repos, configs, or agent environments
21
+ - reproducible test artifacts for each reviewed server
22
22
  - private readiness report covering startup, capabilities, schema quality, security findings, and drift risk
23
- - MCP lock-file setup for contract drift review
24
- - prioritized remediation notes
23
+ - schema/tool drift baseline using MCP lock files
24
+ - MCP Observatory CI rollout plan for selected servers
25
+ - executive summary with “safe for agent dependency” verdicts
26
+ - prioritized remediation notes and owner-ready next steps
25
27
  - optional certification language for servers that pass agreed checks
26
28
 
27
29
  ## Starting Prices
@@ -55,4 +57,18 @@ William
55
57
 
56
58
  ## Delivery Shape
57
59
 
58
- Start with static reports and CI setup. Do not build a dashboard until paid pilot feedback proves exactly what buyers need.
60
+ Start with static reports and CI setup. The first deliverable should look like an internal security/readiness packet, not a SaaS login.
61
+
62
+ ## Evidence Standard
63
+
64
+ The public [Safety Methodology](./methodology.md) and [MCP Server Safety Index](./mcp-server-safety-index.md) are the template for private work:
65
+
66
+ - command/config used
67
+ - date and tool version
68
+ - JSON artifact
69
+ - Markdown or HTML report
70
+ - failure class
71
+ - verdict
72
+ - reproduction notes
73
+
74
+ Private pilots can include customer-specific details, but public/customer-facing summaries should use sanitized evidence unless the customer approves otherwise.
@@ -57,14 +57,15 @@ For deeper context, see the [MCP Server Security Field Guide](./mcp-security-fie
57
57
 
58
58
  Telemetry is used privately to understand product usage and identify account-level signals without publishing raw personal data.
59
59
 
60
- As of the latest local export on June 20, 2026:
60
+ As of the latest local export on June 21, 2026:
61
61
 
62
- - 10,918 telemetry events
63
- - 7,380 total sessions
64
- - 5,379 external sessions after separating internal activity
62
+ - 11,481 telemetry events
63
+ - 7,571 total sessions
64
+ - 5,389 external sessions after separating internal activity
65
65
  - 2,446 external CI sessions
66
- - 138 attributed company/org sessions
67
- - 11 attributed company/org candidates
66
+ - 148 attributed company/org sessions
67
+ - 12 attributed company/org candidates
68
+ - latest external activity: June 21, 2026
68
69
 
69
70
  Public claims use aggregate or sanitized data only. Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
70
71
 
@@ -77,9 +78,12 @@ Current public distribution proof includes:
77
78
  - latest release: `v0.23.0`
78
79
  - npm package: `@kryptosai/mcp-observatory`
79
80
  - GitHub Action: `KryptosAI/mcp-observatory/action@main`
80
- - visible GitHub traffic window: 721 clones and 221 unique cloners
81
+ - npm downloads snapshot: 511 downloads for June 11-20, 2026
82
+ - visible GitHub traffic window: 745 clones and 232 unique cloners
83
+ - visible GitHub page-view window: 12 views and 9 unique visitors
84
+ - public code-search references in MCP indexes, listing mirrors, and external experiment repos
81
85
  - official MCP reference PR open and green: [`modelcontextprotocol/servers#4392`](https://github.com/modelcontextprotocol/servers/pull/4392)
82
- - open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, and other MCP projects
86
+ - open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, AntV, BrowserMCP, UI5, Notion, and other MCP projects
83
87
 
84
88
  See [reference evaluations](./reference-evaluations.md) and [public proof](./proof.md).
85
89