@kryptosai/mcp-observatory 0.23.0 → 0.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/COMMERCIAL.md +14 -0
- package/README.md +21 -12
- package/dist/src/cli.js +2 -2
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/init-ci.d.ts +3 -0
- package/dist/src/commands/init-ci.js +42 -22
- package/dist/src/commands/init-ci.js.map +1 -1
- package/dist/src/commands/test.js +20 -0
- package/dist/src/commands/test.js.map +1 -1
- package/dist/src/commercial.js +7 -0
- package/dist/src/commercial.js.map +1 -1
- package/dist/src/reporters/common.js +2 -2
- package/dist/src/reporters/common.js.map +1 -1
- package/dist/src/reporters/pr-comment.js +6 -2
- package/dist/src/reporters/pr-comment.js.map +1 -1
- package/docs/certification-campaign-template.md +4 -4
- package/docs/certification-distribution.md +1 -1
- package/docs/directory-listing-copy.md +1 -1
- package/docs/enterprise-outreach-playbook.md +10 -10
- package/docs/mcp-lock-files.md +1 -1
- package/docs/mcp-safety-report-latest.md +14 -9
- package/docs/mcp-security-field-guide.md +1 -1
- package/docs/mcp-server-safety-index.md +56 -80
- package/docs/methodology.md +90 -0
- package/docs/metrics-dashboard.md +108 -0
- package/docs/paid-pilot-offer.md +21 -5
- package/docs/project-case-study.md +12 -8
- package/docs/proof.md +29 -16
- package/docs/public-post-drafts.md +18 -6
- package/docs/publish-readiness.md +1 -5
- package/docs/reference-evaluations.md +3 -3
- package/docs/safety-index/artifacts/antv-chart-server.json +2765 -0
- package/docs/safety-index/artifacts/antv-chart-server.md +156 -0
- package/docs/safety-index/artifacts/browsermcp-server.json +416 -0
- package/docs/safety-index/artifacts/browsermcp-server.md +163 -0
- package/docs/safety-index/artifacts/context7-server.json +286 -0
- package/docs/safety-index/artifacts/context7-server.md +163 -0
- package/docs/safety-index/artifacts/everything-server.json +482 -0
- package/docs/safety-index/artifacts/everything-server.md +163 -0
- package/docs/safety-index/artifacts/executeautomation-playwright-server.json +955 -0
- package/docs/safety-index/artifacts/executeautomation-playwright-server.md +163 -0
- package/docs/safety-index/artifacts/filesystem-server.json +583 -0
- package/docs/safety-index/artifacts/filesystem-server.md +156 -0
- package/docs/safety-index/artifacts/memory-server.json +469 -0
- package/docs/safety-index/artifacts/memory-server.md +156 -0
- package/docs/safety-index/artifacts/opentofu-server.json +387 -0
- package/docs/safety-index/artifacts/opentofu-server.md +163 -0
- package/docs/safety-index/artifacts/playwright-mcp-server.json +919 -0
- package/docs/safety-index/artifacts/playwright-mcp-server.md +156 -0
- package/docs/safety-index/artifacts/promptopia-server.json +442 -0
- package/docs/safety-index/artifacts/promptopia-server.md +156 -0
- package/docs/safety-index/artifacts/puppeteer-server.json +377 -0
- package/docs/safety-index/artifacts/puppeteer-server.md +163 -0
- package/docs/safety-index/artifacts/ref-tools-server.json +262 -0
- package/docs/safety-index/artifacts/ref-tools-server.md +156 -0
- package/docs/safety-index/artifacts/sequential-thinking-server.json +286 -0
- package/docs/safety-index/artifacts/sequential-thinking-server.md +156 -0
- package/docs/safety-index/maintainer-note-template.md +25 -0
- package/docs/safety-index/targets.json +192 -0
- package/package.json +16 -9
|
@@ -42,7 +42,7 @@ Skip:
|
|
|
42
42
|
| 5 | `browserbase/mcp-server-browserbase` | `npx -y @browserbasehq/mcp-server-browserbase` | Browser Automation | Hosted browser MCP category | verify auth-free startup | May require API key; issue-only if startup requires credentials | researched | | |
|
|
43
43
|
| 6 | `smithery-ai/server-sequential-thinking` | `npx -y @smithery-ai/server-sequential-thinking` | Developer Tools | MCP directory ecosystem | verify package/repo naming | Good low-risk simple server if public package starts cleanly | researched | | |
|
|
44
44
|
| 7 | `kazuph/mcp-taskmanager` | `npx -y @kazuph/mcp-taskmanager@latest` | Developer Tools | 200+ stars / task/project MCP category | Local validation passed: 10 tools | Scoped package name corrected from tracker | pr-opened | https://github.com/kazuph/mcp-taskmanager/pull/11 | Waiting for maintainer review |
|
|
45
|
-
| 8 | `cyanheads/filesystem-mcp-server` | `node dist/index.js` | Filesystem | Filesystem MCP category | Local validation passed after fix: 10 tools | Fixed real conformance bug: advertised resources without `resources/list`; workflow uses temp sandbox |
|
|
45
|
+
| 8 | `cyanheads/filesystem-mcp-server` | `node dist/index.js` | Filesystem | Filesystem MCP category | Local validation passed after fix: 10 tools | Fixed real conformance bug: advertised resources without `resources/list`; workflow uses temp sandbox | closed-unmerged | https://github.com/cyanheads/filesystem-mcp-server/pull/19 | Closed by maintainer without merge |
|
|
46
46
|
| 9 | `redis/mcp-redis` | `uvx mcp-redis` | Database | Enterprise database category | verify auth-free startup | Database target may require service; issue-only if credentials needed | researched | | |
|
|
47
47
|
| 10 | `mongodb-js/mongodb-mcp-server` | `npx -y mongodb-mcp-server` | Database | Enterprise database category | verify auth-free startup | Likely needs connection string; issue-only first | researched | | |
|
|
48
48
|
| 11 | `supabase-community/supabase-mcp` | `npx -y supabase-mcp` | Database | Enterprise/SaaS category | verify current package | Likely requires token; issue-only first | researched | | |
|
|
@@ -54,7 +54,7 @@ Skip:
|
|
|
54
54
|
| 17 | `BrowserMCP/mcp` | `npx -y @browsermcp/mcp` | Browser Automation | 6k+ stars / browser-control MCP category | Local validation passed: 12 tools | Browser automation trust boundary; workflow is inventory/security only | pr-opened | https://github.com/BrowserMCP/mcp/pull/189 | Waiting for maintainer review |
|
|
55
55
|
| 18 | `UI5/mcp-server` | `npx -y @ui5/mcp-server` | Developer Tools | SAP/UI5 ecosystem MCP package | Local validation passed: 10 tools | Developer tooling surface; no credentials required for inventory | pr-opened | https://github.com/UI5/mcp-server/pull/348 | Waiting for maintainer review |
|
|
56
56
|
| 19 | `apify/actors-mcp-server` | `npx -y @apify/actors-mcp-server` | SaaS/API | Automation platform category | verify auth-free startup | Likely requires token; issue-only first | researched | | |
|
|
57
|
-
| 20 | `makenotion/notion-mcp-server` | `npx -y @notionhq/notion-mcp-server` | SaaS/API | Major SaaS category | Local validation passed: 24 tools | Workspace-data MCP; PR is compatibility/schema/security inventory only | pr-opened | https://github.com/makenotion/notion-mcp-server/pull/324 | Waiting for maintainer review |
|
|
57
|
+
| 20 | `makenotion/notion-mcp-server` | `npx -y @notionhq/notion-mcp-server` | SaaS/API | Major SaaS category | Local validation passed: 24 tools | Workspace-data MCP; PR is compatibility/schema/security inventory only | pr-opened | https://github.com/makenotion/notion-mcp-server/pull/324 | Waiting for maintainer review; external Semgrep check failing |
|
|
58
58
|
| 21 | `linear/linear-mcp` | `npx -y @linear/mcp-server` | SaaS/API | Developer SaaS category | verify package | Likely requires token; issue-only first | researched | | |
|
|
59
59
|
| 22 | `sentry/sentry-mcp` | `npx -y @sentry/mcp-server` | Observability | Developer SaaS category | verify package | Likely requires token; issue-only first | researched | | |
|
|
60
60
|
| 23 | `elastic/mcp-server-elasticsearch` | `npx -y @elastic/mcp-server-elasticsearch` | Search | Enterprise search category | verify package | Likely requires service; issue-only first | researched | | |
|
|
@@ -100,7 +100,7 @@ Statuses:
|
|
|
100
100
|
## PR Checklist
|
|
101
101
|
|
|
102
102
|
- Generate the local kit first:
|
|
103
|
-
`npx @kryptosai/mcp-observatory
|
|
103
|
+
`npx @kryptosai/mcp-observatory setup-ci --all --command "<safe startup command>"`
|
|
104
104
|
- Add `.github/workflows/mcp-observatory.yml`
|
|
105
105
|
- Add `mcp-observatory.target.json` when the startup command needs args, cwd, or env placeholders
|
|
106
106
|
- Use `deep: true` and `security: true`
|
|
@@ -151,7 +151,7 @@ The goal is to give users a visible compatibility/security signal and catch sche
|
|
|
151
151
|
|
|
152
152
|
## Generated PR Body Printer
|
|
153
153
|
|
|
154
|
-
After running `
|
|
154
|
+
After running `setup-ci --all`, print the generated maintainer copy with:
|
|
155
155
|
|
|
156
156
|
```bash
|
|
157
157
|
npm run certification:pr-body -- docs/mcp-observatory-pr-body.md
|
|
@@ -35,7 +35,7 @@ npx @kryptosai/mcp-observatory badge npx -y <server-package> --output docs/mcp-h
|
|
|
35
35
|
Fast path:
|
|
36
36
|
|
|
37
37
|
```bash
|
|
38
|
-
npx @kryptosai/mcp-observatory
|
|
38
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y <server-package>"
|
|
39
39
|
```
|
|
40
40
|
|
|
41
41
|
That creates:
|
|
@@ -25,7 +25,7 @@ For security and platform teams, see the MCP Server Security Field Guide and MCP
|
|
|
25
25
|
Add MCP CI in one command:
|
|
26
26
|
|
|
27
27
|
```bash
|
|
28
|
-
npx @kryptosai/mcp-observatory
|
|
28
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
## Tags
|
|
@@ -3,36 +3,36 @@
|
|
|
3
3
|
Use this playbook after running:
|
|
4
4
|
|
|
5
5
|
```bash
|
|
6
|
-
npm run telemetry:intelligence -- --input telemetry-
|
|
6
|
+
npm run telemetry:intelligence -- --input <private-telemetry-export.jsonl> --out-dir reports
|
|
7
7
|
```
|
|
8
8
|
|
|
9
9
|
Start from `reports/telemetry-usage-summary.html` to confirm external usage before reading account rankings. Do not treat first-party CI, release workflows, or internal/personal sessions as market traction.
|
|
10
10
|
|
|
11
|
-
Raw telemetry is allowed for internal account intelligence
|
|
11
|
+
Raw telemetry is allowed for internal account intelligence only. Do not include personal emails, hostnames, private URLs, target commands, tokens, proprietary schemas, customer names, or private telemetry exports in public issues, posts, docs, or customer-facing outreach without explicit permission. Keep account-specific rankings in ignored `reports/` outputs or private notes.
|
|
12
12
|
|
|
13
13
|
## Priority Accounts
|
|
14
14
|
|
|
15
15
|
| Priority | Account | Evidence | Motion |
|
|
16
16
|
| ---: | --- | --- | --- |
|
|
17
|
-
| 1 |
|
|
18
|
-
| 2 |
|
|
19
|
-
| 3 |
|
|
20
|
-
| 4 |
|
|
17
|
+
| 1 | Private account A | Repeated external sessions, CI usage, private-target signal, or security workflow evidence | Enterprise pilot |
|
|
18
|
+
| 2 | Private account B | Recent usage across multiple sessions or repos | Business pilot / design partner |
|
|
19
|
+
| 3 | Private account C | Small usage cluster with clear owner signal | Business pilot / testimonial ask |
|
|
20
|
+
| 4 | Private account D | Single company or team signal | Team pilot / feedback ask |
|
|
21
21
|
| 5 | GitHub org/user signals | CI or repo-based usage | Team pilot unless company identity is confirmed |
|
|
22
22
|
|
|
23
|
-
##
|
|
23
|
+
## First Email Template
|
|
24
24
|
|
|
25
|
-
Subject: MCP security reporting for
|
|
25
|
+
Subject: MCP security reporting for production agent workflows
|
|
26
26
|
|
|
27
27
|
Hi,
|
|
28
28
|
|
|
29
29
|
I build MCP Observatory, an open source tool for testing, securing, and monitoring MCP servers before agents depend on them.
|
|
30
30
|
|
|
31
|
-
|
|
31
|
+
I am opening a small number of enterprise pilots for teams that want hosted MCP security reports, private-repo CI history, and fleet visibility across agent environments.
|
|
32
32
|
|
|
33
33
|
If your team is running MCP servers in production, I can prepare a short evidence-based report and a pilot proposal focused on:
|
|
34
34
|
|
|
35
|
-
-
|
|
35
|
+
- MCP compatibility
|
|
36
36
|
- private HTTP MCP health checks
|
|
37
37
|
- security findings and schema drift
|
|
38
38
|
- CI history and controlled drift review
|
package/docs/mcp-lock-files.md
CHANGED
|
@@ -21,7 +21,7 @@ npx @kryptosai/mcp-observatory lock verify
|
|
|
21
21
|
Add CI:
|
|
22
22
|
|
|
23
23
|
```bash
|
|
24
|
-
npx @kryptosai/mcp-observatory
|
|
24
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
|
|
25
25
|
```
|
|
26
26
|
|
|
27
27
|
## Why It Matters
|
|
@@ -24,13 +24,18 @@ Safe aggregate telemetry from the latest local export:
|
|
|
24
24
|
|
|
25
25
|
| Metric | Value |
|
|
26
26
|
| --- | ---: |
|
|
27
|
-
| Total telemetry events |
|
|
28
|
-
| Total sessions | 7,
|
|
29
|
-
| External sessions | 5,
|
|
27
|
+
| Total telemetry events | 11,481 |
|
|
28
|
+
| Total sessions | 7,571 |
|
|
29
|
+
| External sessions | 5,389 |
|
|
30
30
|
| External CI sessions | 2,446 |
|
|
31
|
-
| Attributed company/org sessions |
|
|
32
|
-
|
|
|
33
|
-
|
|
|
31
|
+
| Attributed company/org sessions | 148 |
|
|
32
|
+
| Attributed company/org candidates | 12 |
|
|
33
|
+
| Latest external activity | June 21, 2026 |
|
|
34
|
+
| npm downloads snapshot | 511 downloads, June 11-20, 2026 |
|
|
35
|
+
| GitHub clones in visible traffic window | 745 |
|
|
36
|
+
| Unique cloners in visible traffic window | 232 |
|
|
37
|
+
| GitHub page views in visible traffic window | 12 |
|
|
38
|
+
| Unique GitHub viewers in visible traffic window | 9 |
|
|
34
39
|
|
|
35
40
|
Top external commands:
|
|
36
41
|
|
|
@@ -58,7 +63,7 @@ From public sample artifacts and Observatory check categories, the most importan
|
|
|
58
63
|
## How Maintainers Add The Check
|
|
59
64
|
|
|
60
65
|
```bash
|
|
61
|
-
npx @kryptosai/mcp-observatory
|
|
66
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
|
|
62
67
|
```
|
|
63
68
|
|
|
64
69
|
That creates a GitHub Action and a README badge snippet. The action can comment on PRs and fail when MCP compatibility or security checks regress.
|
|
@@ -74,7 +79,7 @@ Production teams can use MCP Observatory for:
|
|
|
74
79
|
- support and rollout review
|
|
75
80
|
- fleet visibility across teams and repos
|
|
76
81
|
|
|
77
|
-
|
|
82
|
+
See [Commercial Pilots](../COMMERCIAL.md) for production/private MCP usage.
|
|
78
83
|
|
|
79
84
|
## Launch Post
|
|
80
85
|
|
|
@@ -83,7 +88,7 @@ MCP servers are becoming production dependencies.
|
|
|
83
88
|
I built MCP Observatory so MCP maintainers can add CI, security checks, schema drift detection, PR reports, and trust badges in one command:
|
|
84
89
|
|
|
85
90
|
```bash
|
|
86
|
-
npx @kryptosai/mcp-observatory
|
|
91
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
|
|
87
92
|
```
|
|
88
93
|
|
|
89
94
|
Free for local OSS use. Paid pilots are available for hosted reporting, private repo CI, certification, support, and fleet visibility.
|
|
@@ -60,7 +60,7 @@ A useful MCP server CI gate should answer a few operational questions:
|
|
|
60
60
|
MCP Observatory is designed to make that a one-command adoption path:
|
|
61
61
|
|
|
62
62
|
```bash
|
|
63
|
-
npx @kryptosai/mcp-observatory
|
|
63
|
+
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server"
|
|
64
64
|
```
|
|
65
65
|
|
|
66
66
|
For a direct check:
|
|
@@ -1,85 +1,61 @@
|
|
|
1
|
-
# MCP Server Safety Index
|
|
2
|
-
|
|
3
|
-
The MCP Server Safety Index is
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
|
21
|
-
|
|
|
22
|
-
|
|
|
23
|
-
|
|
|
24
|
-
|
|
|
25
|
-
|
|
|
26
|
-
|
|
|
27
|
-
|
|
|
28
|
-
|
|
|
29
|
-
|
|
|
30
|
-
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
npx @kryptosai/mcp-observatory lock verify
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
## What Each Column Means
|
|
54
|
-
|
|
55
|
-
- What To Check: the minimum compatibility/security surface a maintainer or platform team should inspect.
|
|
56
|
-
- Risk Class: the operational reason the server matters before agents depend on it.
|
|
57
|
-
- Status: public proof such as PR open, PR accepted, badge added, researched, or needs maintainer review.
|
|
1
|
+
# MCP Server Safety Index v1
|
|
2
|
+
|
|
3
|
+
The MCP Server Safety Index is an evidence standard for MCP readiness. It is not a leaderboard and does not rank maintainers.
|
|
4
|
+
|
|
5
|
+
Each row links to a reproducible command, a JSON run artifact, and a Markdown report generated by MCP Observatory. The goal is to show which failure classes matter before teams let agents depend on MCP servers.
|
|
6
|
+
|
|
7
|
+
For the rules behind this page, see the [Safety Methodology](./methodology.md).
|
|
8
|
+
|
|
9
|
+
## Snapshot
|
|
10
|
+
|
|
11
|
+
- Evaluated servers: 13
|
|
12
|
+
- Ready for CI: 10
|
|
13
|
+
- Needs review before production: 1
|
|
14
|
+
- Unsafe default posture: 2
|
|
15
|
+
- Not reproducible: 0
|
|
16
|
+
- Latest run: 2026-06-24T02:07:44.894Z
|
|
17
|
+
|
|
18
|
+
## Evaluations
|
|
19
|
+
|
|
20
|
+
| # | Server | Category | Verdict | Failure Class | Reproduce | Evidence | Notes |
|
|
21
|
+
| ---: | --- | --- | --- | --- | --- | --- | --- |
|
|
22
|
+
| 1 | [Official everything server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Broad protocol surface | `npx -y @modelcontextprotocol/server-everything` | [JSON](./safety-index/artifacts/everything-server.json) / [report](./safety-index/artifacts/everything-server.md) | Zero-config official package; useful as a broad protocol baseline. [public proof](https://github.com/modelcontextprotocol/servers/pull/4392) |
|
|
23
|
+
| 2 | [Official sequential thinking server](https://github.com/modelcontextprotocol/servers) | Reference | **Ready for CI** | Tool schema clarity | `npx -y @modelcontextprotocol/server-sequential-thinking` | [JSON](./safety-index/artifacts/sequential-thinking-server.json) / [report](./safety-index/artifacts/sequential-thinking-server.md) | Zero-config official package. |
|
|
24
|
+
| 3 | [Official memory server](https://github.com/modelcontextprotocol/servers) | Reference / Memory | **Ready for CI** | Persistent state tools | `npx -y @modelcontextprotocol/server-memory` | [JSON](./safety-index/artifacts/memory-server.json) / [report](./safety-index/artifacts/memory-server.md) | Zero-config official package. |
|
|
25
|
+
| 4 | [Official filesystem server](https://github.com/modelcontextprotocol/servers) | Filesystem | **Needs review before production** | Sandboxed filesystem access | `npx -y @modelcontextprotocol/server-filesystem examples/filesystem-fixture` | [JSON](./safety-index/artifacts/filesystem-server.json) / [report](./safety-index/artifacts/filesystem-server.md) | Runs against the checked-in harmless fixture directory. |
|
|
26
|
+
| 5 | [Context7](https://github.com/upstash/context7) | Documentation / Search | **Ready for CI** | Prompt-injection-sensitive retrieval | `npx -y @upstash/context7-mcp` | [JSON](./safety-index/artifacts/context7-server.json) / [report](./safety-index/artifacts/context7-server.md) | Zero-config public package. |
|
|
27
|
+
| 6 | [Promptopia](https://www.npmjs.com/package/promptopia-mcp) | Prompts | **Ready for CI** | Prompt/resource contract | `npx -y promptopia-mcp` | [JSON](./safety-index/artifacts/promptopia-server.json) / [report](./safety-index/artifacts/promptopia-server.md) | Uses the checked-in prompt fixture through package defaults. |
|
|
28
|
+
| 7 | [Ref tools](https://www.npmjs.com/package/ref-tools-mcp) | Developer Tools | **Ready for CI** | Prompt/tool inventory | `npx -y ref-tools-mcp` | [JSON](./safety-index/artifacts/ref-tools-server.json) / [report](./safety-index/artifacts/ref-tools-server.md) | Zero-config public package. |
|
|
29
|
+
| 8 | [OpenTofu MCP server](https://github.com/opentofu/opentofu-mcp-server) | Infrastructure | **Ready for CI** | Infrastructure tool surface | `npx -y @opentofu/opentofu-mcp-server` | [JSON](./safety-index/artifacts/opentofu-server.json) / [report](./safety-index/artifacts/opentofu-server.md) | Zero-config public package. |
|
|
30
|
+
| 9 | [Puppeteer MCP server](https://www.npmjs.com/package/puppeteer-mcp-server) | Browser Automation | **Ready for CI** | Browser/code execution boundary | `npx -y puppeteer-mcp-server` | [JSON](./safety-index/artifacts/puppeteer-server.json) / [report](./safety-index/artifacts/puppeteer-server.md) | Intentional browser evaluation is suppressed so remaining findings stay readable. |
|
|
31
|
+
| 10 | [BrowserMCP](https://github.com/BrowserMCP/mcp) | Browser Automation | **Ready for CI** | Browser-control boundary | `npx -y @browsermcp/mcp` | [JSON](./safety-index/artifacts/browsermcp-server.json) / [report](./safety-index/artifacts/browsermcp-server.md) | Zero-config public package. [public proof](https://github.com/BrowserMCP/mcp/pull/189) |
|
|
32
|
+
| 11 | [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) | Browser Automation | **Unsafe default posture** | Browser/code execution boundary | `npx -y @playwright/mcp` | [JSON](./safety-index/artifacts/playwright-mcp-server.json) / [report](./safety-index/artifacts/playwright-mcp-server.md) | Zero-config public package; security findings represent policy-review prompts, not a vulnerability claim. |
|
|
33
|
+
| 12 | [AntV chart MCP server](https://github.com/antvis/mcp-server-chart) | Visualization | **Ready for CI** | Artifact-producing tools | `npx -y @antv/mcp-server-chart` | [JSON](./safety-index/artifacts/antv-chart-server.json) / [report](./safety-index/artifacts/antv-chart-server.md) | Zero-config public package. [public proof](https://github.com/antvis/mcp-server-chart/pull/312) |
|
|
34
|
+
| 13 | [ExecuteAutomation Playwright MCP](https://github.com/executeautomation/mcp-playwright) | Browser Automation | **Unsafe default posture** | Startup/listing reproducibility | `npx -y @executeautomation/playwright-mcp-server` | [JSON](./safety-index/artifacts/executeautomation-playwright-server.json) / [report](./safety-index/artifacts/executeautomation-playwright-server.md) | Evaluated as a public package; current result should be treated as a maintainer conversation starter. [public proof](https://github.com/executeautomation/mcp-playwright/pull/225) |
|
|
35
|
+
|
|
36
|
+
## Patterns Observed
|
|
37
|
+
|
|
38
|
+
- Browser/code execution boundary: 2 server(s)
|
|
39
|
+
- Artifact-producing tools: 1 server(s)
|
|
40
|
+
- Broad protocol surface: 1 server(s)
|
|
41
|
+
- Browser-control boundary: 1 server(s)
|
|
42
|
+
- Infrastructure tool surface: 1 server(s)
|
|
43
|
+
- Persistent state tools: 1 server(s)
|
|
44
|
+
- Prompt-injection-sensitive retrieval: 1 server(s)
|
|
45
|
+
- Prompt/resource contract: 1 server(s)
|
|
46
|
+
- Prompt/tool inventory: 1 server(s)
|
|
47
|
+
- Sandboxed filesystem access: 1 server(s)
|
|
48
|
+
- Startup/listing reproducibility: 1 server(s)
|
|
49
|
+
- Tool schema clarity: 1 server(s)
|
|
58
50
|
|
|
59
51
|
## Publication Rules
|
|
60
52
|
|
|
61
|
-
- Use only public repositories, public package commands, public PRs,
|
|
62
|
-
-
|
|
63
|
-
-
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
|
|
67
|
-
## Five Patterns To Publish From v0
|
|
68
|
-
|
|
69
|
-
1. Browser automation MCP servers need explicit policy around code execution, screenshots, navigation, and mutation.
|
|
70
|
-
2. Filesystem MCP servers need harmless CI sandboxes and clear read/write boundaries.
|
|
71
|
-
3. SaaS and cloud MCP servers often cannot be meaningfully checked without token-safe target configs.
|
|
72
|
-
4. Database MCP servers need read/write classification and connection-string hygiene before CI rollout.
|
|
73
|
-
5. Lock files turn MCP surface drift into a reviewable PR event instead of an invisible agent dependency change.
|
|
74
|
-
|
|
75
|
-
## Next Wave Criteria
|
|
76
|
-
|
|
77
|
-
Prioritize 20-50 servers that have:
|
|
53
|
+
- Use only public repositories, public package commands, public PRs, and generated sanitized artifacts.
|
|
54
|
+
- Treat findings as reproducible evidence, not public shaming.
|
|
55
|
+
- Prefer “needs review” language unless there is clear artifact-backed proof of a dangerous default.
|
|
56
|
+
- Keep raw telemetry, emails, hostnames, private URLs, tokens, and customer claims out of public materials.
|
|
57
|
+
- Send maintainers the report first; open CI PRs only when the report is useful and the target can run safely.
|
|
78
58
|
|
|
79
|
-
|
|
80
|
-
- visible stars, downloads, or directory listings
|
|
81
|
-
- simple `npx`, `uvx`, or Docker startup commands
|
|
82
|
-
- enterprise-relevant categories such as browser automation, filesystem, documentation/search, databases, cloud, productivity, and developer tools
|
|
83
|
-
- no existing MCP compatibility/security CI
|
|
59
|
+
## Next Step
|
|
84
60
|
|
|
85
|
-
|
|
61
|
+
Use this index to start maintainer conversations and private readiness reviews. The buyer-facing offer is a private MCP readiness review with CI rollout, drift/security reporting, and safe-for-agent-dependency verdicts.
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# MCP Observatory Safety Methodology
|
|
2
|
+
|
|
3
|
+
MCP Observatory treats MCP servers as agent-facing dependencies. The Safety Index is designed to answer one practical question:
|
|
4
|
+
|
|
5
|
+
> Is this server ready for agents and teams to depend on, and what evidence supports that answer?
|
|
6
|
+
|
|
7
|
+
The index is not a leaderboard. It is a reproducible evidence standard for maintainers, security teams, platform teams, and buyers evaluating MCP servers.
|
|
8
|
+
|
|
9
|
+
## What Gets Tested
|
|
10
|
+
|
|
11
|
+
Each public evaluation runs MCP Observatory against a public repository or package command. A useful entry includes:
|
|
12
|
+
|
|
13
|
+
- server name and public source
|
|
14
|
+
- exact command and arguments
|
|
15
|
+
- run date
|
|
16
|
+
- MCP Observatory version
|
|
17
|
+
- JSON run artifact
|
|
18
|
+
- Markdown report
|
|
19
|
+
- verdict
|
|
20
|
+
- failure class
|
|
21
|
+
- reproduction notes
|
|
22
|
+
|
|
23
|
+
The default public check verifies startup, tools, prompts, resources, schema quality, and lightweight security findings. Some entries also include deeper security checks when the target can be evaluated without private credentials.
|
|
24
|
+
|
|
25
|
+
## Verdicts
|
|
26
|
+
|
|
27
|
+
- **Ready for CI:** the server starts, lists expected MCP surfaces, and has no high- or medium-severity security finding in the generated artifact.
|
|
28
|
+
- **Needs review before production:** the server is reproducible but has findings or partial results a maintainer/security reviewer should inspect before production use.
|
|
29
|
+
- **Not reproducible:** the server cannot complete a basic startup or listing check from the documented public command.
|
|
30
|
+
- **Unsafe default posture:** the artifact contains high-severity security findings that deserve explicit policy review before agent dependency.
|
|
31
|
+
- **Could not evaluate:** the public command cannot be evaluated without credentials, private infrastructure, or maintainer-provided safe configuration.
|
|
32
|
+
|
|
33
|
+
These verdicts are intentionally operational. They are not formal vulnerability claims.
|
|
34
|
+
|
|
35
|
+
## Scoring Inputs
|
|
36
|
+
|
|
37
|
+
MCP Observatory uses the same run artifact model across CLI, CI, reports, and the Safety Index. The health score considers:
|
|
38
|
+
|
|
39
|
+
- protocol compliance
|
|
40
|
+
- schema quality
|
|
41
|
+
- security and security-lite checks
|
|
42
|
+
- reliability/startup behavior
|
|
43
|
+
- performance where latency data is available
|
|
44
|
+
|
|
45
|
+
The Safety Index does not rank by score. Scores are supporting evidence; failure classes are the story.
|
|
46
|
+
|
|
47
|
+
## Failure Classes
|
|
48
|
+
|
|
49
|
+
Common MCP readiness patterns include:
|
|
50
|
+
|
|
51
|
+
- startup/listing reproducibility
|
|
52
|
+
- browser/code execution boundary
|
|
53
|
+
- filesystem boundary
|
|
54
|
+
- prompt-injection-sensitive retrieval
|
|
55
|
+
- persistent state tools
|
|
56
|
+
- infrastructure or cloud control surfaces
|
|
57
|
+
- artifact-producing tools
|
|
58
|
+
- schema clarity and drift
|
|
59
|
+
- token-safe configuration
|
|
60
|
+
|
|
61
|
+
The first public index emphasizes these classes so maintainers can improve concrete surfaces rather than argue about a single trust score.
|
|
62
|
+
|
|
63
|
+
## Reproducibility Rules
|
|
64
|
+
|
|
65
|
+
An index row should be included only when it can be reproduced from public information:
|
|
66
|
+
|
|
67
|
+
- public repo, package, or container reference
|
|
68
|
+
- no private telemetry
|
|
69
|
+
- no private customer evidence
|
|
70
|
+
- no raw emails, hostnames, private URLs, tokens, or response bodies
|
|
71
|
+
- a generated JSON artifact and Markdown report
|
|
72
|
+
- clear notes when credentials or maintainer context are required
|
|
73
|
+
|
|
74
|
+
If the safe public command is not known, the right next step is a maintainer note, not a drive-by CI PR.
|
|
75
|
+
|
|
76
|
+
## Maintainer Posture
|
|
77
|
+
|
|
78
|
+
The index is constructive by default:
|
|
79
|
+
|
|
80
|
+
- send the report first
|
|
81
|
+
- describe the failure class, not the maintainer
|
|
82
|
+
- offer a CI PR only if the target can run safely and the maintainer wants it
|
|
83
|
+
- prefer read-only workflows and pinned action refs for third-party repos
|
|
84
|
+
- use issue-first outreach for token-backed SaaS, cloud, payments, database, and browser-control servers
|
|
85
|
+
|
|
86
|
+
## Limitations
|
|
87
|
+
|
|
88
|
+
MCP Observatory cannot prove semantic safety. A passing result does not mean a server is safe for every workflow. It means the server produced reproducible evidence for compatibility, schema quality, and common security footguns under the tested command.
|
|
89
|
+
|
|
90
|
+
Production teams should pair these checks with their own threat model, policy, credential boundaries, sandboxing, approvals, and runtime monitoring.
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Local Metrics Dashboard
|
|
2
|
+
|
|
3
|
+
MCP Observatory includes a private laptop dashboard for collecting telemetry, GitHub activity, and npm download data going forward.
|
|
4
|
+
|
|
5
|
+
The dashboard is intentionally local. It stores raw telemetry in a SQLite database on your laptop, then renders a sanitized static HTML view that is safe to screenshot or skim without exposing raw emails, hostnames, private URLs, tokens, or private command bodies.
|
|
6
|
+
|
|
7
|
+
The layout follows an App Store Connect-style breakdown:
|
|
8
|
+
|
|
9
|
+
- Overview
|
|
10
|
+
- Strategy
|
|
11
|
+
- Acquisition
|
|
12
|
+
- Downloads
|
|
13
|
+
- Usage
|
|
14
|
+
- Reliability
|
|
15
|
+
|
|
16
|
+
The Strategy section turns raw telemetry into operator decisions: command funnel, version adoption, account/domain drilldown, CI/local source mix, and release-spike context.
|
|
17
|
+
|
|
18
|
+
Daily rows are shown newest to oldest so the most recent project activity is always at the top.
|
|
19
|
+
|
|
20
|
+
## Refresh
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npm run metrics:refresh
|
|
24
|
+
npm run metrics:serve
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Outputs are written to:
|
|
28
|
+
|
|
29
|
+
- `.mcp-observatory-metrics/observatory.sqlite`
|
|
30
|
+
- `.mcp-observatory-metrics/dashboard/index.html`
|
|
31
|
+
- `.mcp-observatory-metrics/dashboard/latest.json`
|
|
32
|
+
- `.mcp-observatory-metrics/logs/`
|
|
33
|
+
|
|
34
|
+
The local metrics directory is ignored by git and should not be published.
|
|
35
|
+
|
|
36
|
+
## Data Sources
|
|
37
|
+
|
|
38
|
+
The collector stores each source independently. If one source fails, the dashboard still builds from the last good SQLite data and records the failure in the Reliability section.
|
|
39
|
+
|
|
40
|
+
| Source | Data |
|
|
41
|
+
| --- | --- |
|
|
42
|
+
| Telemetry | Existing Cloudflare D1 export flow via `scripts/export-telemetry-d1.ts` |
|
|
43
|
+
| GitHub | clones, views, referrers, popular paths, repo snapshot, latest release, open issue/PR counts, workflow runs |
|
|
44
|
+
| npm | public daily downloads for `@kryptosai/mcp-observatory` |
|
|
45
|
+
|
|
46
|
+
GitHub traffic APIs have a limited visible window, so the collector stores snapshots locally going forward. npm daily buckets can lag; the dashboard labels npm data as complete public days rather than assuming current-day zero.
|
|
47
|
+
|
|
48
|
+
## Credentials
|
|
49
|
+
|
|
50
|
+
GitHub collection uses either:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
gh auth login
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
or:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
export GH_TOKEN=...
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Telemetry collection uses the same Wrangler configuration as the existing telemetry export script. If the config is not auto-discovered, set:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
export MCP_OBSERVATORY_TELEMETRY_WRANGLER_CONFIG=/path/to/wrangler.toml
|
|
66
|
+
export MCP_OBSERVATORY_TELEMETRY_D1_DATABASE=mcp-observatory-telemetry
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
No npm token is required for public download counts.
|
|
70
|
+
|
|
71
|
+
## Commands
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
npm run metrics:collect # collect telemetry, GitHub, and npm into SQLite
|
|
75
|
+
npm run metrics:build # render HTML from the existing SQLite database
|
|
76
|
+
npm run metrics:refresh # collect + build
|
|
77
|
+
npm run metrics:open # open the static read-only dashboard
|
|
78
|
+
npm run metrics:serve # open the local dashboard with an Update Data button
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
The **Update Data** button is available in `metrics:serve` mode. A static `file://` dashboard cannot run local commands from the browser, so `metrics:open` shows the same data but disables the button.
|
|
82
|
+
|
|
83
|
+
For tests or offline recovery, you can seed telemetry from an existing export:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
npm run metrics:refresh -- --telemetry-input telemetry-exports/events-flat-full.json
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Optional Hourly Refresh
|
|
90
|
+
|
|
91
|
+
Generate a small local refresh wrapper:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
npx tsx scripts/metrics-dashboard.ts scheduler
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The generated script lives under `.mcp-observatory-metrics/` and can be called by `launchd` or cron. It uses the same refresh command and writes logs to `.mcp-observatory-metrics/logs/refresh.log`.
|
|
98
|
+
|
|
99
|
+
## Privacy
|
|
100
|
+
|
|
101
|
+
Raw telemetry remains available in the local SQLite database for account intelligence and product analytics. The dashboard view uses sanitized aggregates by default:
|
|
102
|
+
|
|
103
|
+
- company domains, not raw emails
|
|
104
|
+
- aggregate source and command counts, not private command bodies
|
|
105
|
+
- GitHub and npm public metrics
|
|
106
|
+
- collection errors and freshness status
|
|
107
|
+
|
|
108
|
+
Do not commit `.mcp-observatory-metrics/`, telemetry exports, private reports, or screenshots that reveal raw data.
|
package/docs/paid-pilot-offer.md
CHANGED
|
@@ -17,11 +17,13 @@ This is a manual pilot, not a self-serve SaaS promise.
|
|
|
17
17
|
|
|
18
18
|
## What The Pilot Includes
|
|
19
19
|
|
|
20
|
-
-
|
|
21
|
-
-
|
|
20
|
+
- MCP server inventory across selected repos, configs, or agent environments
|
|
21
|
+
- reproducible test artifacts for each reviewed server
|
|
22
22
|
- private readiness report covering startup, capabilities, schema quality, security findings, and drift risk
|
|
23
|
-
-
|
|
24
|
-
-
|
|
23
|
+
- schema/tool drift baseline using MCP lock files
|
|
24
|
+
- MCP Observatory CI rollout plan for selected servers
|
|
25
|
+
- executive summary with “safe for agent dependency” verdicts
|
|
26
|
+
- prioritized remediation notes and owner-ready next steps
|
|
25
27
|
- optional certification language for servers that pass agreed checks
|
|
26
28
|
|
|
27
29
|
## Starting Prices
|
|
@@ -55,4 +57,18 @@ William
|
|
|
55
57
|
|
|
56
58
|
## Delivery Shape
|
|
57
59
|
|
|
58
|
-
Start with static reports and CI setup.
|
|
60
|
+
Start with static reports and CI setup. The first deliverable should look like an internal security/readiness packet, not a SaaS login.
|
|
61
|
+
|
|
62
|
+
## Evidence Standard
|
|
63
|
+
|
|
64
|
+
The public [Safety Methodology](./methodology.md) and [MCP Server Safety Index](./mcp-server-safety-index.md) are the template for private work:
|
|
65
|
+
|
|
66
|
+
- command/config used
|
|
67
|
+
- date and tool version
|
|
68
|
+
- JSON artifact
|
|
69
|
+
- Markdown or HTML report
|
|
70
|
+
- failure class
|
|
71
|
+
- verdict
|
|
72
|
+
- reproduction notes
|
|
73
|
+
|
|
74
|
+
Private pilots can include customer-specific details, but public/customer-facing summaries should use sanitized evidence unless the customer approves otherwise.
|
|
@@ -57,14 +57,15 @@ For deeper context, see the [MCP Server Security Field Guide](./mcp-security-fie
|
|
|
57
57
|
|
|
58
58
|
Telemetry is used privately to understand product usage and identify account-level signals without publishing raw personal data.
|
|
59
59
|
|
|
60
|
-
As of the latest local export on June
|
|
60
|
+
As of the latest local export on June 21, 2026:
|
|
61
61
|
|
|
62
|
-
-
|
|
63
|
-
- 7,
|
|
64
|
-
- 5,
|
|
62
|
+
- 11,481 telemetry events
|
|
63
|
+
- 7,571 total sessions
|
|
64
|
+
- 5,389 external sessions after separating internal activity
|
|
65
65
|
- 2,446 external CI sessions
|
|
66
|
-
-
|
|
67
|
-
-
|
|
66
|
+
- 148 attributed company/org sessions
|
|
67
|
+
- 12 attributed company/org candidates
|
|
68
|
+
- latest external activity: June 21, 2026
|
|
68
69
|
|
|
69
70
|
Public claims use aggregate or sanitized data only. Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
|
|
70
71
|
|
|
@@ -77,9 +78,12 @@ Current public distribution proof includes:
|
|
|
77
78
|
- latest release: `v0.23.0`
|
|
78
79
|
- npm package: `@kryptosai/mcp-observatory`
|
|
79
80
|
- GitHub Action: `KryptosAI/mcp-observatory/action@main`
|
|
80
|
-
-
|
|
81
|
+
- npm downloads snapshot: 511 downloads for June 11-20, 2026
|
|
82
|
+
- visible GitHub traffic window: 745 clones and 232 unique cloners
|
|
83
|
+
- visible GitHub page-view window: 12 views and 9 unique visitors
|
|
84
|
+
- public code-search references in MCP indexes, listing mirrors, and external experiment repos
|
|
81
85
|
- official MCP reference PR open and green: [`modelcontextprotocol/servers#4392`](https://github.com/modelcontextprotocol/servers/pull/4392)
|
|
82
|
-
- open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, and other MCP projects
|
|
86
|
+
- open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, AntV, BrowserMCP, UI5, Notion, and other MCP projects
|
|
83
87
|
|
|
84
88
|
See [reference evaluations](./reference-evaluations.md) and [public proof](./proof.md).
|
|
85
89
|
|