@kryptosai/mcp-observatory 0.23.0 → 0.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/COMMERCIAL.md +14 -0
  2. package/README.md +21 -12
  3. package/dist/src/cli.js +2 -2
  4. package/dist/src/cli.js.map +1 -1
  5. package/dist/src/commands/init-ci.d.ts +3 -0
  6. package/dist/src/commands/init-ci.js +42 -22
  7. package/dist/src/commands/init-ci.js.map +1 -1
  8. package/dist/src/commands/test.js +20 -0
  9. package/dist/src/commands/test.js.map +1 -1
  10. package/dist/src/commercial.js +7 -0
  11. package/dist/src/commercial.js.map +1 -1
  12. package/dist/src/reporters/common.js +2 -2
  13. package/dist/src/reporters/common.js.map +1 -1
  14. package/dist/src/reporters/pr-comment.js +6 -2
  15. package/dist/src/reporters/pr-comment.js.map +1 -1
  16. package/docs/certification-campaign-template.md +4 -4
  17. package/docs/certification-distribution.md +1 -1
  18. package/docs/directory-listing-copy.md +1 -1
  19. package/docs/enterprise-outreach-playbook.md +10 -10
  20. package/docs/mcp-lock-files.md +1 -1
  21. package/docs/mcp-safety-report-latest.md +14 -9
  22. package/docs/mcp-security-field-guide.md +1 -1
  23. package/docs/mcp-server-safety-index.md +56 -80
  24. package/docs/methodology.md +90 -0
  25. package/docs/metrics-dashboard.md +108 -0
  26. package/docs/paid-pilot-offer.md +21 -5
  27. package/docs/project-case-study.md +12 -8
  28. package/docs/proof.md +29 -16
  29. package/docs/public-post-drafts.md +18 -6
  30. package/docs/publish-readiness.md +1 -5
  31. package/docs/reference-evaluations.md +3 -3
  32. package/docs/safety-index/artifacts/antv-chart-server.json +2765 -0
  33. package/docs/safety-index/artifacts/antv-chart-server.md +156 -0
  34. package/docs/safety-index/artifacts/browsermcp-server.json +416 -0
  35. package/docs/safety-index/artifacts/browsermcp-server.md +163 -0
  36. package/docs/safety-index/artifacts/context7-server.json +286 -0
  37. package/docs/safety-index/artifacts/context7-server.md +163 -0
  38. package/docs/safety-index/artifacts/everything-server.json +482 -0
  39. package/docs/safety-index/artifacts/everything-server.md +163 -0
  40. package/docs/safety-index/artifacts/executeautomation-playwright-server.json +955 -0
  41. package/docs/safety-index/artifacts/executeautomation-playwright-server.md +163 -0
  42. package/docs/safety-index/artifacts/filesystem-server.json +583 -0
  43. package/docs/safety-index/artifacts/filesystem-server.md +156 -0
  44. package/docs/safety-index/artifacts/memory-server.json +469 -0
  45. package/docs/safety-index/artifacts/memory-server.md +156 -0
  46. package/docs/safety-index/artifacts/opentofu-server.json +387 -0
  47. package/docs/safety-index/artifacts/opentofu-server.md +163 -0
  48. package/docs/safety-index/artifacts/playwright-mcp-server.json +919 -0
  49. package/docs/safety-index/artifacts/playwright-mcp-server.md +156 -0
  50. package/docs/safety-index/artifacts/promptopia-server.json +442 -0
  51. package/docs/safety-index/artifacts/promptopia-server.md +156 -0
  52. package/docs/safety-index/artifacts/puppeteer-server.json +377 -0
  53. package/docs/safety-index/artifacts/puppeteer-server.md +163 -0
  54. package/docs/safety-index/artifacts/ref-tools-server.json +262 -0
  55. package/docs/safety-index/artifacts/ref-tools-server.md +156 -0
  56. package/docs/safety-index/artifacts/sequential-thinking-server.json +286 -0
  57. package/docs/safety-index/artifacts/sequential-thinking-server.md +156 -0
  58. package/docs/safety-index/maintainer-note-template.md +25 -0
  59. package/docs/safety-index/targets.json +192 -0
  60. package/package.json +16 -9
package/docs/proof.md CHANGED
@@ -5,13 +5,15 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
5
5
  ## Current Public Surface
6
6
 
7
7
  - npm package: `@kryptosai/mcp-observatory`
8
- - GitHub Action: `KryptosAI/mcp-observatory/action@main`
8
+ - GitHub Action: `KryptosAI/mcp-observatory/action@v0.24.0`
9
9
  - Latest release: `v0.23.0`
10
- - CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, init-ci, ci-report, enterprise-report, score, badge, cloud
10
+ - CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, setup-ci, init-ci, ci-report, enterprise-report, score, badge, cloud
11
11
  - Test suite: 334 passing tests across 43 test files as of June 20, 2026
12
- - GitHub traffic snapshot: 721 clones and 221 unique cloners in the visible June 2026 traffic window
13
- - npm downloads snapshot: 104 downloads for June 11-17, 2026
12
+ - GitHub traffic snapshot: 745 clones and 232 unique cloners in the visible June 2026 traffic window
13
+ - GitHub page views snapshot: 12 views and 9 unique visitors in the visible June 2026 traffic window
14
+ - npm downloads snapshot: 511 downloads for June 11-20, 2026
14
15
  - Security guide: [MCP Server Security Field Guide](./mcp-security-field-guide.md)
16
+ - Safety methodology: [MCP Observatory Safety Methodology](./methodology.md)
15
17
  - Safety index: [MCP Server Safety Index](./mcp-server-safety-index.md)
16
18
  - Public examples: [Reference Evaluations](./reference-evaluations.md)
17
19
  - Lock-file CI primitive: [MCP Lock Files](./mcp-lock-files.md)
@@ -22,14 +24,15 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
22
24
 
23
25
  Internal telemetry is used for product analytics and account-level outreach. Public reporting uses only aggregate or sanitized data.
24
26
 
25
- As of the latest local export on June 20, 2026:
27
+ As of the latest local export on June 21, 2026:
26
28
 
27
- - 10,918 telemetry events
28
- - 7,380 total sessions
29
- - 5,379 external sessions after separating internal/personal activity
29
+ - 11,481 telemetry events
30
+ - 7,571 total sessions
31
+ - 5,389 external sessions after separating internal/personal activity
30
32
  - 2,446 external CI sessions
31
- - 138 attributed company/org sessions
32
- - 11 attributed company/org candidates
33
+ - 148 attributed company/org sessions
34
+ - 12 attributed company/org candidates
35
+ - latest external activity: June 21, 2026
33
36
  - top external commands: `serve`, `run`, `diff`, `test`, `scan`, `history`
34
37
 
35
38
  Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
@@ -54,20 +57,30 @@ MCP Observatory can:
54
57
 
55
58
  The certification campaign is designed to create public proof through accepted maintainer PRs.
56
59
 
57
- Accepted third-party integrations will be tracked here:
60
+ Open and accepted third-party integrations are tracked here:
58
61
 
59
62
  | Repo | PR | Check Added | Badge Added | Status |
60
63
  | --- | --- | --- | --- | --- |
61
64
  | `modelcontextprotocol/servers` | [#4392](https://github.com/modelcontextprotocol/servers/pull/4392) | Yes | No | Open, mergeable, MCP Observatory check passing |
62
- | `microsoft/playwright-mcp` | [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) | Yes | No | Open |
63
- | `upstash/context7` | [#2800](https://github.com/upstash/context7/pull/2800) | Yes | No | Open |
65
+ | `microsoft/playwright-mcp` | [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) | Yes | No | Closed, unmerged |
66
+ | `upstash/context7` | [#2800](https://github.com/upstash/context7/pull/2800) | Yes | No | Closed, maintainer declined third-party CI |
64
67
  | `executeautomation/mcp-playwright` | [#225](https://github.com/executeautomation/mcp-playwright/pull/225) | Yes | No | Open |
65
68
  | `kazuph/mcp-taskmanager` | [#11](https://github.com/kazuph/mcp-taskmanager/pull/11) | Yes | No | Open |
66
- | `cyanheads/filesystem-mcp-server` | [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) | Yes | No | Open |
69
+ | `cyanheads/filesystem-mcp-server` | [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) | Yes | No | Closed, unmerged |
67
70
  | `antvis/mcp-server-chart` | [#312](https://github.com/antvis/mcp-server-chart/pull/312) | Yes | No | Open |
68
71
  | `BrowserMCP/mcp` | [#189](https://github.com/BrowserMCP/mcp/pull/189) | Yes | No | Open |
69
- | `UI5/mcp-server` | [#348](https://github.com/UI5/mcp-server/pull/348) | Yes | No | Open |
70
- | `makenotion/notion-mcp-server` | [#324](https://github.com/makenotion/notion-mcp-server/pull/324) | Yes | No | Open |
72
+ | `UI5/mcp-server` | [#348](https://github.com/UI5/mcp-server/pull/348) | Yes | No | Closed, maintainer declined third-party CI |
73
+ | `makenotion/notion-mcp-server` | [#324](https://github.com/makenotion/notion-mcp-server/pull/324) | Yes | No | Closed after policy-style CI failure |
74
+
75
+ ## Public Discovery Snapshot
76
+
77
+ GitHub code search shows public references outside the main repo. These are discovery/listing signals, not customer claims:
78
+
79
+ - `punkpeye/awesome-mcp-devtools` lists MCP Observatory in an MCP developer-tools index.
80
+ - `linny006/mcp-servers-live` mirrors a public MCP Observatory listing page.
81
+ - `gabrielmoreira/awesome-ai-rabbit-holes` catalogs the GitHub project.
82
+ - `fmfg03/supermcp` includes an `apps/mcp-observatory` package path.
83
+ - `vellankikoti/mcp-observatory`, `LuKrlier/mcp-observatory`, and `shigeki7777/sasame-mcp-observatory` appear as separate public repos referencing or experimenting with the Observatory name/code surface.
71
84
 
72
85
  ## Commercial Proof
73
86
 
@@ -2,12 +2,22 @@
2
2
 
3
3
  Use these as launch posts, GitHub Discussion posts, LinkedIn posts, or short blog drafts. The framing is about MCP safety patterns, not “look at my tool.”
4
4
 
5
- ## 1. I Tested 20 MCP Servers. The Pattern Was Not “Bad Servers”; It Was Missing Gates.
5
+ ## Flagship Post: I Tested Popular MCP Servers. The Failure Pattern Was Not What I Expected.
6
6
 
7
7
  MCP servers are becoming production dependencies for agents, but many of them still ship without the kind of CI gate we expect from normal software dependencies.
8
8
 
9
9
  The main pattern I saw while building the first MCP Server Safety Index was simple: the risky part is rarely that a server exists. The risky part is that agents may depend on a tool surface nobody is testing for startup reliability, schema quality, security posture, or drift.
10
10
 
11
+ The industry does not need another vibes-based directory. It needs reproducible readiness evidence:
12
+
13
+ - exact command/config
14
+ - date and package version where available
15
+ - JSON artifact
16
+ - Markdown report
17
+ - verdict
18
+ - failure class
19
+ - reproduction notes
20
+
11
21
  The checks that matter most:
12
22
 
13
23
  - does the server start cleanly in CI?
@@ -18,7 +28,9 @@ The checks that matter most:
18
28
 
19
29
  My takeaway: MCP needs a package-lock moment. Commit the agent-facing contract, then make drift visible before agents depend on it.
20
30
 
21
- ## 2. Browser MCP Servers Need A Different Security Bar
31
+ I am publishing the Safety Methodology and the first MCP Server Safety Index as a small evidence standard, not a leaderboard. If your team is putting MCP into private or production agent workflows, I am doing a small number of private MCP readiness reviews: inventory, CI rollout, schema/tool drift baseline, security findings, and safe-for-agent-dependency verdicts.
32
+
33
+ ## Supporting Angle: Browser MCP Servers Need A Different Security Bar
22
34
 
23
35
  Browser automation MCP servers are powerful because agents can navigate pages, click, type, inspect state, and sometimes execute scripts.
24
36
 
@@ -32,9 +44,9 @@ For browser MCP servers, a useful review should separate:
32
44
  - network/navigation controls
33
45
  - tool schemas that are too broad for safe agent planning
34
46
 
35
- The goal is not to block browser MCP. The goal is to make the trust boundary visible before an agent gets a browser with hands.
47
+ The goal is not to block browser MCP. The goal is to make the trust boundary visible before an agent gets browser-control powers.
36
48
 
37
- ## 3. Filesystem MCP Servers Should Always Test In A Sandbox
49
+ ## Supporting Angle: Filesystem MCP Servers Should Always Test In A Sandbox
38
50
 
39
51
  Filesystem MCP servers are one of the clearest examples of why MCP CI needs context.
40
52
 
@@ -50,7 +62,7 @@ The minimum safety pattern:
50
62
 
51
63
  Agents need tools. They do not need accidental access to everything.
52
64
 
53
- ## 4. Token-Backed SaaS MCP Servers Need Issue-First Certification
65
+ ## Supporting Angle: Token-Backed SaaS MCP Servers Need Issue-First Certification
54
66
 
55
67
  Many SaaS, cloud, payments, database, and developer-platform MCP servers cannot be safely checked with a drive-by PR because meaningful startup requires tokens or live services.
56
68
 
@@ -68,7 +80,7 @@ Once maintainers provide a token-safe target config, the useful checks are:
68
80
 
69
81
  Security adoption works better when it starts by respecting maintainer context.
70
82
 
71
- ## 5. MCP Drift Is An AI Supply Chain Problem
83
+ ## Supporting Angle: MCP Drift Is An AI Supply Chain Problem
72
84
 
73
85
  When a package dependency changes, teams have lock files, diffs, review, and release notes.
74
86
 
@@ -26,11 +26,7 @@ Confirm:
26
26
 
27
27
  Known audit note:
28
28
 
29
- - `npm audit` may report `undici <=6.26.0` through the `npm@11.17.0` package bundled under `@semantic-release/npm`. As of June 20, 2026, `npm audit fix` cannot update this bundled copy and `npm@11.17.0` is the current published npm package. The remaining vulnerable `undici` copy is release tooling only and is not part of MCP Observatory runtime dependencies or the packed npm artifact. Recheck after npm publishes a newer package.
30
-
31
- Known audit note:
32
-
33
- - `npm audit` may report `undici <=6.26.0` through the `npm@11.17.0` package bundled under `@semantic-release/npm`. `npm audit fix` updates the fixable `@actions/http-client` path, but the remaining `undici` copy is bundled inside npm release tooling and is not part of MCP Observatory runtime dependencies or the packed npm artifact. Recheck after npm publishes a newer package.
29
+ - Release automation runs `semantic-release` ephemerally in GitHub Actions instead of installing it into the repository lockfile. This keeps release-only bundled dependencies out of the default-branch audit surface and out of the packed CLI artifact.
34
30
 
35
31
  ## Public Distribution
36
32
 
@@ -29,7 +29,7 @@ What Observatory checks:
29
29
  Adoption command:
30
30
 
31
31
  ```bash
32
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y @modelcontextprotocol/server-sequential-thinking"
32
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @modelcontextprotocol/server-sequential-thinking"
33
33
  ```
34
34
 
35
35
  ## Browser Automation MCP Servers
@@ -70,7 +70,7 @@ Representative public category: filesystem-backed MCP servers.
70
70
 
71
71
  Public proof:
72
72
 
73
- - PR: [`cyanheads/filesystem-mcp-server#19`](https://github.com/cyanheads/filesystem-mcp-server/pull/19)
73
+ - PR: [`cyanheads/filesystem-mcp-server#19`](https://github.com/cyanheads/filesystem-mcp-server/pull/19), closed unmerged by maintainer
74
74
 
75
75
  What this represents:
76
76
 
@@ -118,7 +118,7 @@ What Observatory checks:
118
118
  Adoption command:
119
119
 
120
120
  ```bash
121
- npx @kryptosai/mcp-observatory init-ci --all --command "npx -y @upstash/context7-mcp"
121
+ npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y @upstash/context7-mcp"
122
122
  ```
123
123
 
124
124
  ## How To Read These Evaluations