@kryptosai/mcp-observatory 0.22.0 → 0.24.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/COMMERCIAL.md +5 -3
- package/PRIVACY.md +5 -2
- package/README.md +28 -13
- package/dist/src/cli.js +1 -1
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/init-ci.d.ts +3 -0
- package/dist/src/commands/init-ci.js +26 -9
- package/dist/src/commands/init-ci.js.map +1 -1
- package/dist/src/commercial.js +2 -2
- package/dist/src/commercial.js.map +1 -1
- package/dist/src/reporters/pr-comment.js +6 -2
- package/dist/src/reporters/pr-comment.js.map +1 -1
- package/dist/src/score.js +1 -1
- package/dist/src/score.js.map +1 -1
- package/dist/src/validate.js +58 -3
- package/dist/src/validate.js.map +1 -1
- package/docs/certification-campaign-template.md +10 -10
- package/docs/certification-distribution.md +16 -0
- package/docs/directory-listing-copy.md +12 -5
- package/docs/distribution-launch.md +5 -5
- package/docs/enterprise-outreach-playbook.md +2 -2
- package/docs/mcp-lock-files.md +63 -0
- package/docs/mcp-safety-report-latest.md +16 -7
- package/docs/mcp-security-field-guide.md +97 -0
- package/docs/mcp-server-safety-index.md +61 -0
- package/docs/methodology.md +90 -0
- package/docs/metrics-dashboard.md +105 -0
- package/docs/paid-pilot-offer.md +74 -0
- package/docs/project-case-study.md +77 -43
- package/docs/proof.md +42 -12
- package/docs/public-post-drafts.md +98 -0
- package/docs/publish-readiness.md +6 -4
- package/docs/reference-evaluations.md +134 -0
- package/docs/safety-index/artifacts/antv-chart-server.json +2765 -0
- package/docs/safety-index/artifacts/antv-chart-server.md +156 -0
- package/docs/safety-index/artifacts/browsermcp-server.json +416 -0
- package/docs/safety-index/artifacts/browsermcp-server.md +163 -0
- package/docs/safety-index/artifacts/context7-server.json +286 -0
- package/docs/safety-index/artifacts/context7-server.md +163 -0
- package/docs/safety-index/artifacts/everything-server.json +482 -0
- package/docs/safety-index/artifacts/everything-server.md +163 -0
- package/docs/safety-index/artifacts/executeautomation-playwright-server.json +955 -0
- package/docs/safety-index/artifacts/executeautomation-playwright-server.md +163 -0
- package/docs/safety-index/artifacts/filesystem-server.json +583 -0
- package/docs/safety-index/artifacts/filesystem-server.md +156 -0
- package/docs/safety-index/artifacts/memory-server.json +469 -0
- package/docs/safety-index/artifacts/memory-server.md +156 -0
- package/docs/safety-index/artifacts/opentofu-server.json +387 -0
- package/docs/safety-index/artifacts/opentofu-server.md +163 -0
- package/docs/safety-index/artifacts/playwright-mcp-server.json +919 -0
- package/docs/safety-index/artifacts/playwright-mcp-server.md +156 -0
- package/docs/safety-index/artifacts/promptopia-server.json +442 -0
- package/docs/safety-index/artifacts/promptopia-server.md +156 -0
- package/docs/safety-index/artifacts/puppeteer-server.json +377 -0
- package/docs/safety-index/artifacts/puppeteer-server.md +163 -0
- package/docs/safety-index/artifacts/ref-tools-server.json +262 -0
- package/docs/safety-index/artifacts/ref-tools-server.md +156 -0
- package/docs/safety-index/artifacts/sequential-thinking-server.json +286 -0
- package/docs/safety-index/artifacts/sequential-thinking-server.md +156 -0
- package/docs/safety-index/maintainer-note-template.md +25 -0
- package/docs/safety-index/targets.json +192 -0
- package/package.json +17 -13
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Local Metrics Dashboard
|
|
2
|
+
|
|
3
|
+
MCP Observatory includes a private laptop dashboard for collecting telemetry, GitHub activity, and npm download data going forward.
|
|
4
|
+
|
|
5
|
+
The dashboard is intentionally local. It stores raw telemetry in a SQLite database on your laptop, then renders a sanitized static HTML view that is safe to screenshot or skim without exposing raw emails, hostnames, private URLs, tokens, or private command bodies.
|
|
6
|
+
|
|
7
|
+
The layout follows an App Store Connect-style breakdown:
|
|
8
|
+
|
|
9
|
+
- Overview
|
|
10
|
+
- Acquisition
|
|
11
|
+
- Downloads
|
|
12
|
+
- Usage
|
|
13
|
+
- Reliability
|
|
14
|
+
|
|
15
|
+
Daily rows are shown newest to oldest so the most recent project activity is always at the top.
|
|
16
|
+
|
|
17
|
+
## Refresh
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npm run metrics:refresh
|
|
21
|
+
npm run metrics:serve
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Outputs are written to:
|
|
25
|
+
|
|
26
|
+
- `.mcp-observatory-metrics/observatory.sqlite`
|
|
27
|
+
- `.mcp-observatory-metrics/dashboard/index.html`
|
|
28
|
+
- `.mcp-observatory-metrics/dashboard/latest.json`
|
|
29
|
+
- `.mcp-observatory-metrics/logs/`
|
|
30
|
+
|
|
31
|
+
The local metrics directory is ignored by git and should not be published.
|
|
32
|
+
|
|
33
|
+
## Data Sources
|
|
34
|
+
|
|
35
|
+
The collector stores each source independently. If one source fails, the dashboard still builds from the last good SQLite data and records the failure in the Reliability section.
|
|
36
|
+
|
|
37
|
+
| Source | Data |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| Telemetry | Existing Cloudflare D1 export flow via `scripts/export-telemetry-d1.ts` |
|
|
40
|
+
| GitHub | clones, views, referrers, popular paths, repo snapshot, latest release, open issue/PR counts, workflow runs |
|
|
41
|
+
| npm | public daily downloads for `@kryptosai/mcp-observatory` |
|
|
42
|
+
|
|
43
|
+
GitHub traffic APIs have a limited visible window, so the collector stores snapshots locally going forward. npm daily buckets can lag; the dashboard labels npm data as complete public days rather than assuming current-day zero.
|
|
44
|
+
|
|
45
|
+
## Credentials
|
|
46
|
+
|
|
47
|
+
GitHub collection uses either:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
gh auth login
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
or:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
export GH_TOKEN=...
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Telemetry collection uses the same Wrangler configuration as the existing telemetry export script. If the config is not auto-discovered, set:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
export MCP_OBSERVATORY_TELEMETRY_WRANGLER_CONFIG=/path/to/wrangler.toml
|
|
63
|
+
export MCP_OBSERVATORY_TELEMETRY_D1_DATABASE=mcp-observatory-telemetry
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
No npm token is required for public download counts.
|
|
67
|
+
|
|
68
|
+
## Commands
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
npm run metrics:collect # collect telemetry, GitHub, and npm into SQLite
|
|
72
|
+
npm run metrics:build # render HTML from the existing SQLite database
|
|
73
|
+
npm run metrics:refresh # collect + build
|
|
74
|
+
npm run metrics:open # open the static read-only dashboard
|
|
75
|
+
npm run metrics:serve # open the local dashboard with an Update Data button
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The **Update Data** button is available in `metrics:serve` mode. A static `file://` dashboard cannot run local commands from the browser, so `metrics:open` shows the same data but disables the button.
|
|
79
|
+
|
|
80
|
+
For tests or offline recovery, you can seed telemetry from an existing export:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
npm run metrics:refresh -- --telemetry-input telemetry-exports/events-flat-full.json
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Optional Hourly Refresh
|
|
87
|
+
|
|
88
|
+
Generate a small local refresh wrapper:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
npx tsx scripts/metrics-dashboard.ts scheduler
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
The generated script lives under `.mcp-observatory-metrics/` and can be called by `launchd` or cron. It uses the same refresh command and writes logs to `.mcp-observatory-metrics/logs/refresh.log`.
|
|
95
|
+
|
|
96
|
+
## Privacy
|
|
97
|
+
|
|
98
|
+
Raw telemetry remains available in the local SQLite database for account intelligence and product analytics. The dashboard view uses sanitized aggregates by default:
|
|
99
|
+
|
|
100
|
+
- company domains, not raw emails
|
|
101
|
+
- aggregate source and command counts, not private command bodies
|
|
102
|
+
- GitHub and npm public metrics
|
|
103
|
+
- collection errors and freshness status
|
|
104
|
+
|
|
105
|
+
Do not commit `.mcp-observatory-metrics/`, telemetry exports, private reports, or screenshots that reveal raw data.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Paid Pilot Offer
|
|
2
|
+
|
|
3
|
+
## Private MCP Readiness Review
|
|
4
|
+
|
|
5
|
+
Offer:
|
|
6
|
+
|
|
7
|
+
> Private MCP readiness review + CI rollout + drift/security report.
|
|
8
|
+
|
|
9
|
+
This is a manual pilot, not a self-serve SaaS promise.
|
|
10
|
+
|
|
11
|
+
## Who It Is For
|
|
12
|
+
|
|
13
|
+
- teams running MCP servers in production or pre-production
|
|
14
|
+
- security/platform teams reviewing agent tool dependencies
|
|
15
|
+
- companies with private MCP repos
|
|
16
|
+
- teams that need proof before agents depend on internal tools
|
|
17
|
+
|
|
18
|
+
## What The Pilot Includes
|
|
19
|
+
|
|
20
|
+
- MCP server inventory across selected repos, configs, or agent environments
|
|
21
|
+
- reproducible test artifacts for each reviewed server
|
|
22
|
+
- private readiness report covering startup, capabilities, schema quality, security findings, and drift risk
|
|
23
|
+
- schema/tool drift baseline using MCP lock files
|
|
24
|
+
- MCP Observatory CI rollout plan for selected servers
|
|
25
|
+
- executive summary with “safe for agent dependency” verdicts
|
|
26
|
+
- prioritized remediation notes and owner-ready next steps
|
|
27
|
+
- optional certification language for servers that pass agreed checks
|
|
28
|
+
|
|
29
|
+
## Starting Prices
|
|
30
|
+
|
|
31
|
+
- Business Pilot: starts at `$999/month`
|
|
32
|
+
- Enterprise Pilot: starts at `$3k/month`
|
|
33
|
+
- Strategic Accounts: custom, `$250k+/year`
|
|
34
|
+
|
|
35
|
+
Do not route major platforms, AI labs, or large enterprises to Team/Business pricing. Use a production/security pilot conversation and ask for the owner or procurement path.
|
|
36
|
+
|
|
37
|
+
## Simple Outreach Copy
|
|
38
|
+
|
|
39
|
+
Subject: Private MCP readiness review
|
|
40
|
+
|
|
41
|
+
Hi,
|
|
42
|
+
|
|
43
|
+
I build MCP Observatory, the CI and security gate for MCP servers before agents depend on them.
|
|
44
|
+
|
|
45
|
+
I am opening a small number of private MCP readiness pilots for teams running MCP in production or pre-production. The pilot includes CI rollout, schema/security review, drift checks, and a private readiness report for your MCP servers.
|
|
46
|
+
|
|
47
|
+
If MCP is becoming part of your agent infrastructure, I can help you answer:
|
|
48
|
+
|
|
49
|
+
- which servers are safe enough for agents to depend on?
|
|
50
|
+
- which tool surfaces changed recently?
|
|
51
|
+
- where are the schema/security risks?
|
|
52
|
+
- what should block a PR before production?
|
|
53
|
+
|
|
54
|
+
Would it be useful to compare notes this week?
|
|
55
|
+
|
|
56
|
+
William
|
|
57
|
+
|
|
58
|
+
## Delivery Shape
|
|
59
|
+
|
|
60
|
+
Start with static reports and CI setup. The first deliverable should look like an internal security/readiness packet, not a SaaS login.
|
|
61
|
+
|
|
62
|
+
## Evidence Standard
|
|
63
|
+
|
|
64
|
+
The public [Safety Methodology](./methodology.md) and [MCP Server Safety Index](./mcp-server-safety-index.md) are the template for private work:
|
|
65
|
+
|
|
66
|
+
- command/config used
|
|
67
|
+
- date and tool version
|
|
68
|
+
- JSON artifact
|
|
69
|
+
- Markdown or HTML report
|
|
70
|
+
- failure class
|
|
71
|
+
- verdict
|
|
72
|
+
- reproduction notes
|
|
73
|
+
|
|
74
|
+
Private pilots can include customer-specific details, but public/customer-facing summaries should use sanitized evidence unless the customer approves otherwise.
|
|
@@ -4,9 +4,17 @@
|
|
|
4
4
|
|
|
5
5
|
MCP Observatory is CI/security infrastructure for production MCP servers.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Project Narrative
|
|
8
8
|
|
|
9
|
-
MCP
|
|
9
|
+
MCP Observatory identifies an emerging risk in AI agent infrastructure and turns it into a practical OSS control: CI checks, security reports, drift detection, telemetry intelligence, and certification workflows for production MCP servers.
|
|
10
|
+
|
|
11
|
+
The project is strongest as a signal because it connects product intuition with implementation depth. It starts from a real infrastructure shift, builds a working developer tool around that shift, instruments usage, and creates a credible path from open source adoption to production security workflows.
|
|
12
|
+
|
|
13
|
+
## Problem Discovery
|
|
14
|
+
|
|
15
|
+
MCP servers are becoming dependencies for AI agents. They expose tools, prompts, resources, and data access that agents can call directly. When those servers drift, fail to start, expose broad capabilities, or return ambiguous schemas, the failure can propagate into agent workflows.
|
|
16
|
+
|
|
17
|
+
The control gap is simple: teams need a way to test MCP servers before agents depend on them. They also need artifacts that maintainers, platform engineers, and security reviewers can understand.
|
|
10
18
|
|
|
11
19
|
## Product
|
|
12
20
|
|
|
@@ -23,46 +31,61 @@ MCP Observatory provides:
|
|
|
23
31
|
- static enterprise reports
|
|
24
32
|
- telemetry intelligence for product and account-level learning
|
|
25
33
|
|
|
26
|
-
##
|
|
34
|
+
## System Design
|
|
27
35
|
|
|
28
|
-
The project is a TypeScript/Node CLI with modular command handlers, MCP adapters, check runners, reporters, artifact schemas, and a GitHub Action wrapper.
|
|
36
|
+
The project is a TypeScript/Node CLI with modular command handlers, MCP adapters, check runners, reporters, artifact schemas, and a GitHub Action wrapper.
|
|
29
37
|
|
|
30
|
-
|
|
38
|
+
The system supports local-process and HTTP MCP targets, stores run artifacts, compares runs for regressions, generates reports for humans and CI systems, and can run as an MCP server itself. A Cloudflare Worker handles hosted artifact upload pilots. A separate telemetry Worker stores private aggregate usage events in D1 for product and account intelligence.
|
|
31
39
|
|
|
32
|
-
|
|
40
|
+
## Security Model
|
|
33
41
|
|
|
34
|
-
-
|
|
35
|
-
- 40 test files
|
|
36
|
-
- 321 passing tests
|
|
37
|
-
- npm package published
|
|
38
|
-
- GitHub Action available
|
|
39
|
-
- MCP server mode available
|
|
40
|
-
- telemetry export and company intelligence tooling available
|
|
42
|
+
MCP Observatory treats MCP servers as agent-facing infrastructure. The goal is not to claim formal semantic safety. The goal is to make compatibility, drift, and obvious security risk visible before deployment.
|
|
41
43
|
|
|
42
|
-
|
|
44
|
+
Current controls include:
|
|
43
45
|
|
|
44
|
-
|
|
46
|
+
- lightweight security checks for risky schema patterns
|
|
47
|
+
- schema quality analysis for agent usability
|
|
48
|
+
- SARIF output for security review workflows
|
|
49
|
+
- support for security suppressions when broad tools are intentional
|
|
50
|
+
- private-network rejection for hosted scans
|
|
51
|
+
- privacy disclosure and telemetry opt-out controls
|
|
52
|
+
- sanitized public reporting policy
|
|
45
53
|
|
|
46
|
-
|
|
47
|
-
- 7,211 telemetry sessions
|
|
48
|
-
- 5,368 external sessions after separating internal activity
|
|
49
|
-
- 582 GitHub clones and 175 unique cloners in the visible June 2026 traffic window
|
|
50
|
-
- 104 npm downloads during June 11-17, 2026
|
|
54
|
+
For deeper context, see the [MCP Server Security Field Guide](./mcp-security-field-guide.md).
|
|
51
55
|
|
|
52
|
-
|
|
56
|
+
## Telemetry Intelligence
|
|
53
57
|
|
|
54
|
-
|
|
58
|
+
Telemetry is used privately to understand product usage and identify account-level signals without publishing raw personal data.
|
|
55
59
|
|
|
56
|
-
|
|
60
|
+
As of the latest local export on June 21, 2026:
|
|
57
61
|
|
|
58
|
-
- telemetry
|
|
59
|
-
-
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
63
|
-
-
|
|
62
|
+
- 11,481 telemetry events
|
|
63
|
+
- 7,571 total sessions
|
|
64
|
+
- 5,389 external sessions after separating internal activity
|
|
65
|
+
- 2,446 external CI sessions
|
|
66
|
+
- 148 attributed company/org sessions
|
|
67
|
+
- 12 attributed company/org candidates
|
|
68
|
+
- latest external activity: June 21, 2026
|
|
64
69
|
|
|
65
|
-
Public claims
|
|
70
|
+
Public claims use aggregate or sanitized data only. Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
|
|
71
|
+
|
|
72
|
+
## Distribution Strategy
|
|
73
|
+
|
|
74
|
+
The distribution wedge is useful CI for other MCP repositories. The certification campaign opens small, helpful PRs that add MCP compatibility/security checks and leave maintainers with a public trust signal.
|
|
75
|
+
|
|
76
|
+
Current public distribution proof includes:
|
|
77
|
+
|
|
78
|
+
- latest release: `v0.23.0`
|
|
79
|
+
- npm package: `@kryptosai/mcp-observatory`
|
|
80
|
+
- GitHub Action: `KryptosAI/mcp-observatory/action@main`
|
|
81
|
+
- npm downloads snapshot: 511 downloads for June 11-20, 2026
|
|
82
|
+
- visible GitHub traffic window: 745 clones and 232 unique cloners
|
|
83
|
+
- visible GitHub page-view window: 12 views and 9 unique visitors
|
|
84
|
+
- public code-search references in MCP indexes, listing mirrors, and external experiment repos
|
|
85
|
+
- official MCP reference PR open and green: [`modelcontextprotocol/servers#4392`](https://github.com/modelcontextprotocol/servers/pull/4392)
|
|
86
|
+
- open certification PRs for Microsoft Playwright MCP, Upstash Context7, ExecuteAutomation Playwright MCP, AntV, BrowserMCP, UI5, Notion, and other MCP projects
|
|
87
|
+
|
|
88
|
+
See [reference evaluations](./reference-evaluations.md) and [public proof](./proof.md).
|
|
66
89
|
|
|
67
90
|
## Commercial Path
|
|
68
91
|
|
|
@@ -83,24 +106,35 @@ Current pilot anchors:
|
|
|
83
106
|
- Enterprise: starts at `$3k/month`
|
|
84
107
|
- Strategic: `$250k+/year`
|
|
85
108
|
|
|
86
|
-
##
|
|
109
|
+
## Professional Signal
|
|
87
110
|
|
|
88
|
-
|
|
111
|
+
MCP Observatory demonstrates applied work across:
|
|
89
112
|
|
|
90
|
-
- AI infrastructure
|
|
113
|
+
- AI agent infrastructure
|
|
91
114
|
- developer tooling
|
|
92
|
-
-
|
|
93
|
-
-
|
|
115
|
+
- secure tool invocation
|
|
116
|
+
- software supply chain thinking
|
|
94
117
|
- CI/CD integrations
|
|
95
118
|
- telemetry and product analytics
|
|
96
|
-
-
|
|
119
|
+
- open source distribution
|
|
120
|
+
- enterprise packaging
|
|
121
|
+
|
|
122
|
+
It is designed to be evaluated through public work: code, docs, CI integrations, reference evaluations, proof surfaces, and real maintainer PRs.
|
|
123
|
+
|
|
124
|
+
## Future Roadmap
|
|
125
|
+
|
|
126
|
+
Near-term milestones:
|
|
97
127
|
|
|
98
|
-
|
|
128
|
+
1. Convert certification PRs into accepted public integrations.
|
|
129
|
+
2. Publish recurring MCP safety reports.
|
|
130
|
+
3. Add stronger policy/provenance language for production MCP adoption.
|
|
131
|
+
4. Improve hosted artifact upload into a simple pilot workflow.
|
|
132
|
+
5. Convert serious production users into paid pilots.
|
|
99
133
|
|
|
100
|
-
|
|
134
|
+
Longer-term opportunities:
|
|
101
135
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
136
|
+
- policy controls for agent tool use
|
|
137
|
+
- provenance for MCP packages and configurations
|
|
138
|
+
- schema locks and controlled drift review
|
|
139
|
+
- runtime monitoring for production agent tool calls
|
|
140
|
+
- fleet inventory across teams, repositories, and hosts
|
package/docs/proof.md
CHANGED
|
@@ -5,23 +5,34 @@ MCP Observatory is early, but it is already a working MCP testing/security stack
|
|
|
5
5
|
## Current Public Surface
|
|
6
6
|
|
|
7
7
|
- npm package: `@kryptosai/mcp-observatory`
|
|
8
|
-
- GitHub Action: `KryptosAI/mcp-observatory/action@
|
|
8
|
+
- GitHub Action: `KryptosAI/mcp-observatory/action@v0.24.0`
|
|
9
|
+
- Latest release: `v0.23.0`
|
|
9
10
|
- CLI command count: scan, test, record, replay, verify, diff, watch, suggest, serve, lock, history, init-ci, ci-report, enterprise-report, score, badge, cloud
|
|
10
|
-
- Test suite:
|
|
11
|
-
- GitHub traffic snapshot:
|
|
12
|
-
-
|
|
11
|
+
- Test suite: 334 passing tests across 43 test files as of June 20, 2026
|
|
12
|
+
- GitHub traffic snapshot: 745 clones and 232 unique cloners in the visible June 2026 traffic window
|
|
13
|
+
- GitHub page views snapshot: 12 views and 9 unique visitors in the visible June 2026 traffic window
|
|
14
|
+
- npm downloads snapshot: 511 downloads for June 11-20, 2026
|
|
15
|
+
- Security guide: [MCP Server Security Field Guide](./mcp-security-field-guide.md)
|
|
16
|
+
- Safety methodology: [MCP Observatory Safety Methodology](./methodology.md)
|
|
17
|
+
- Safety index: [MCP Server Safety Index](./mcp-server-safety-index.md)
|
|
18
|
+
- Public examples: [Reference Evaluations](./reference-evaluations.md)
|
|
19
|
+
- Lock-file CI primitive: [MCP Lock Files](./mcp-lock-files.md)
|
|
20
|
+
- Public post drafts: [Launch Post Drafts](./public-post-drafts.md)
|
|
21
|
+
- Pilot offer: [Private MCP Readiness Review](./paid-pilot-offer.md)
|
|
13
22
|
|
|
14
23
|
## Safe Aggregate Telemetry Snapshot
|
|
15
24
|
|
|
16
25
|
Internal telemetry is used for product analytics and account-level outreach. Public reporting uses only aggregate or sanitized data.
|
|
17
26
|
|
|
18
|
-
As of the latest local export on June
|
|
27
|
+
As of the latest local export on June 21, 2026:
|
|
19
28
|
|
|
20
|
-
-
|
|
21
|
-
- 7,
|
|
22
|
-
- 5,
|
|
23
|
-
- 2,
|
|
24
|
-
-
|
|
29
|
+
- 11,481 telemetry events
|
|
30
|
+
- 7,571 total sessions
|
|
31
|
+
- 5,389 external sessions after separating internal/personal activity
|
|
32
|
+
- 2,446 external CI sessions
|
|
33
|
+
- 148 attributed company/org sessions
|
|
34
|
+
- 12 attributed company/org candidates
|
|
35
|
+
- latest external activity: June 21, 2026
|
|
25
36
|
- top external commands: `serve`, `run`, `diff`, `test`, `scan`, `history`
|
|
26
37
|
|
|
27
38
|
Raw emails, hostnames, private URLs, tokens, and response bodies are not published.
|
|
@@ -46,11 +57,30 @@ MCP Observatory can:
|
|
|
46
57
|
|
|
47
58
|
The certification campaign is designed to create public proof through accepted maintainer PRs.
|
|
48
59
|
|
|
49
|
-
|
|
60
|
+
Open and accepted third-party integrations are tracked here:
|
|
50
61
|
|
|
51
62
|
| Repo | PR | Check Added | Badge Added | Status |
|
|
52
63
|
| --- | --- | --- | --- | --- |
|
|
53
|
-
|
|
|
64
|
+
| `modelcontextprotocol/servers` | [#4392](https://github.com/modelcontextprotocol/servers/pull/4392) | Yes | No | Open, mergeable, MCP Observatory check passing |
|
|
65
|
+
| `microsoft/playwright-mcp` | [#1657](https://github.com/microsoft/playwright-mcp/pull/1657) | Yes | No | Closed, unmerged |
|
|
66
|
+
| `upstash/context7` | [#2800](https://github.com/upstash/context7/pull/2800) | Yes | No | Closed, maintainer declined third-party CI |
|
|
67
|
+
| `executeautomation/mcp-playwright` | [#225](https://github.com/executeautomation/mcp-playwright/pull/225) | Yes | No | Open |
|
|
68
|
+
| `kazuph/mcp-taskmanager` | [#11](https://github.com/kazuph/mcp-taskmanager/pull/11) | Yes | No | Open |
|
|
69
|
+
| `cyanheads/filesystem-mcp-server` | [#19](https://github.com/cyanheads/filesystem-mcp-server/pull/19) | Yes | No | Closed, unmerged |
|
|
70
|
+
| `antvis/mcp-server-chart` | [#312](https://github.com/antvis/mcp-server-chart/pull/312) | Yes | No | Open |
|
|
71
|
+
| `BrowserMCP/mcp` | [#189](https://github.com/BrowserMCP/mcp/pull/189) | Yes | No | Open |
|
|
72
|
+
| `UI5/mcp-server` | [#348](https://github.com/UI5/mcp-server/pull/348) | Yes | No | Closed, maintainer declined third-party CI |
|
|
73
|
+
| `makenotion/notion-mcp-server` | [#324](https://github.com/makenotion/notion-mcp-server/pull/324) | Yes | No | Closed after policy-style CI failure |
|
|
74
|
+
|
|
75
|
+
## Public Discovery Snapshot
|
|
76
|
+
|
|
77
|
+
GitHub code search shows public references outside the main repo. These are discovery/listing signals, not customer claims:
|
|
78
|
+
|
|
79
|
+
- `punkpeye/awesome-mcp-devtools` lists MCP Observatory in an MCP developer-tools index.
|
|
80
|
+
- `linny006/mcp-servers-live` mirrors a public MCP Observatory listing page.
|
|
81
|
+
- `gabrielmoreira/awesome-ai-rabbit-holes` catalogs the GitHub project.
|
|
82
|
+
- `fmfg03/supermcp` includes an `apps/mcp-observatory` package path.
|
|
83
|
+
- `vellankikoti/mcp-observatory`, `LuKrlier/mcp-observatory`, and `shigeki7777/sasame-mcp-observatory` appear as separate public repos referencing or experimenting with the Observatory name/code surface.
|
|
54
84
|
|
|
55
85
|
## Commercial Proof
|
|
56
86
|
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Public Post Drafts
|
|
2
|
+
|
|
3
|
+
Use these as launch posts, GitHub Discussion posts, LinkedIn posts, or short blog drafts. The framing is about MCP safety patterns, not “look at my tool.”
|
|
4
|
+
|
|
5
|
+
## Flagship Post: I Tested Popular MCP Servers. The Failure Pattern Was Not What I Expected.
|
|
6
|
+
|
|
7
|
+
MCP servers are becoming production dependencies for agents, but many of them still ship without the kind of CI gate we expect from normal software dependencies.
|
|
8
|
+
|
|
9
|
+
The main pattern I saw while building the first MCP Server Safety Index was simple: the risky part is rarely that a server exists. The risky part is that agents may depend on a tool surface nobody is testing for startup reliability, schema quality, security posture, or drift.
|
|
10
|
+
|
|
11
|
+
The industry does not need another vibes-based directory. It needs reproducible readiness evidence:
|
|
12
|
+
|
|
13
|
+
- exact command/config
|
|
14
|
+
- date and package version where available
|
|
15
|
+
- JSON artifact
|
|
16
|
+
- Markdown report
|
|
17
|
+
- verdict
|
|
18
|
+
- failure class
|
|
19
|
+
- reproduction notes
|
|
20
|
+
|
|
21
|
+
The checks that matter most:
|
|
22
|
+
|
|
23
|
+
- does the server start cleanly in CI?
|
|
24
|
+
- do tools, prompts, and resources respond as advertised?
|
|
25
|
+
- are tool schemas precise enough for agents to call safely?
|
|
26
|
+
- did a release add, remove, or broaden a tool?
|
|
27
|
+
- are destructive tools clearly identifiable?
|
|
28
|
+
|
|
29
|
+
My takeaway: MCP needs a package-lock moment. Commit the agent-facing contract, then make drift visible before agents depend on it.
|
|
30
|
+
|
|
31
|
+
I am publishing the Safety Methodology and the first MCP Server Safety Index as a small evidence standard, not a leaderboard. If your team is putting MCP into private or production agent workflows, I am doing a small number of private MCP readiness reviews: inventory, CI rollout, schema/tool drift baseline, security findings, and safe-for-agent-dependency verdicts.
|
|
32
|
+
|
|
33
|
+
## Supporting Angle: Browser MCP Servers Need A Different Security Bar
|
|
34
|
+
|
|
35
|
+
Browser automation MCP servers are powerful because agents can navigate pages, click, type, inspect state, and sometimes execute scripts.
|
|
36
|
+
|
|
37
|
+
That is exactly why they need explicit CI and security gates.
|
|
38
|
+
|
|
39
|
+
For browser MCP servers, a useful review should separate:
|
|
40
|
+
|
|
41
|
+
- harmless inventory checks
|
|
42
|
+
- state-mutating browser actions
|
|
43
|
+
- code execution or page-evaluation tools
|
|
44
|
+
- network/navigation controls
|
|
45
|
+
- tool schemas that are too broad for safe agent planning
|
|
46
|
+
|
|
47
|
+
The goal is not to block browser MCP. The goal is to make the trust boundary visible before an agent gets browser-control powers.
|
|
48
|
+
|
|
49
|
+
## Supporting Angle: Filesystem MCP Servers Should Always Test In A Sandbox
|
|
50
|
+
|
|
51
|
+
Filesystem MCP servers are one of the clearest examples of why MCP CI needs context.
|
|
52
|
+
|
|
53
|
+
A server can be useful and still dangerous if the test command points at the wrong directory, if read/write boundaries are unclear, or if a tool schema makes broad path access look harmless.
|
|
54
|
+
|
|
55
|
+
The minimum safety pattern:
|
|
56
|
+
|
|
57
|
+
- run CI against a temporary harmless directory
|
|
58
|
+
- verify tools/resources respond as advertised
|
|
59
|
+
- flag broad filesystem access
|
|
60
|
+
- document which operations are read-only vs write-capable
|
|
61
|
+
- treat changes to path schemas as contract drift
|
|
62
|
+
|
|
63
|
+
Agents need tools. They do not need accidental access to everything.
|
|
64
|
+
|
|
65
|
+
## Supporting Angle: Token-Backed SaaS MCP Servers Need Issue-First Certification
|
|
66
|
+
|
|
67
|
+
Many SaaS, cloud, payments, database, and developer-platform MCP servers cannot be safely checked with a drive-by PR because meaningful startup requires tokens or live services.
|
|
68
|
+
|
|
69
|
+
For those repos, the right move is usually not a workflow PR first. It is an issue or maintainer question:
|
|
70
|
+
|
|
71
|
+
“What is the safest CI startup command for this server?”
|
|
72
|
+
|
|
73
|
+
Once maintainers provide a token-safe target config, the useful checks are:
|
|
74
|
+
|
|
75
|
+
- does startup fail cleanly without credentials?
|
|
76
|
+
- are auth requirements documented?
|
|
77
|
+
- are destructive tools obvious?
|
|
78
|
+
- are schemas narrow enough for agent use?
|
|
79
|
+
- can the repo publish a safe compatibility/security badge?
|
|
80
|
+
|
|
81
|
+
Security adoption works better when it starts by respecting maintainer context.
|
|
82
|
+
|
|
83
|
+
## Supporting Angle: MCP Drift Is An AI Supply Chain Problem
|
|
84
|
+
|
|
85
|
+
When a package dependency changes, teams have lock files, diffs, review, and release notes.
|
|
86
|
+
|
|
87
|
+
When an MCP server changes its tool surface, an agent dependency changed too.
|
|
88
|
+
|
|
89
|
+
That means tool additions, tool removals, schema broadening, new write actions, and prompt/resource changes should be visible in pull requests.
|
|
90
|
+
|
|
91
|
+
The useful primitive is an MCP lock file:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
npx @kryptosai/mcp-observatory lock
|
|
95
|
+
npx @kryptosai/mcp-observatory lock verify
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The point is not bureaucracy. It is to make the agent-facing contract reviewable before production workflows quietly depend on something new.
|
|
@@ -22,20 +22,21 @@ Confirm:
|
|
|
22
22
|
- HTTP target examples use env references instead of inline tokens.
|
|
23
23
|
- Security findings appear in artifact evidence as structured `findings`.
|
|
24
24
|
- Hosted upload is available through `mcp-observatory cloud upload <artifact>` when `MCP_OBSERVATORY_CLOUD_TOKEN` is set.
|
|
25
|
+
- Hosted HTTP scans require `Authorization: Bearer <HOSTED_SCAN_TOKEN>` and are treated as an authenticated pilot surface.
|
|
25
26
|
|
|
26
27
|
Known audit note:
|
|
27
28
|
|
|
28
|
-
-
|
|
29
|
+
- Release automation runs `semantic-release` ephemerally in GitHub Actions instead of installing it into the repository lockfile. This keeps release-only bundled dependencies out of the default-branch audit surface and out of the packed CLI artifact.
|
|
29
30
|
|
|
30
31
|
## Public Distribution
|
|
31
32
|
|
|
32
33
|
- Merge the health/commercialization PR.
|
|
33
34
|
- Update the GitHub repo homepage to the README or commercial page.
|
|
34
35
|
- Publish npm only after the release gate is green.
|
|
35
|
-
- Refresh MCP directory listings with: “MCP Observatory
|
|
36
|
-
- Include “free for local OSS use; paid for hosted reporting, private repo CI, security reports,
|
|
36
|
+
- Refresh MCP directory listings with: “MCP Observatory is the CI and security gate for MCP servers before agents depend on them.”
|
|
37
|
+
- Include “free for local OSS use; paid for hosted reporting, private repo CI, recurring security reports, certification, support, and fleet visibility.”
|
|
37
38
|
- Link production users to `COMMERCIAL.md` and `william@banksey.com`.
|
|
38
|
-
- Submit or refresh listings on Glama, PulseMCP, Smithery, and relevant awesome-MCP lists with the tags: security, developer tools, CI/CD, testing,
|
|
39
|
+
- Submit or refresh listings on Glama, PulseMCP, Smithery, and relevant awesome-MCP lists with the tags: security, developer tools, CI/CD, testing, MCP security, schema drift.
|
|
39
40
|
- Use the certification distribution loop to open helpful PRs against popular MCP server repos and convert accepted PRs into proof points.
|
|
40
41
|
- Link public proof, the safety report, and directory listing copy from launch/outreach materials.
|
|
41
42
|
|
|
@@ -67,6 +68,7 @@ Worker:
|
|
|
67
68
|
|
|
68
69
|
- `POST /api/v1/artifacts` stores a run artifact behind bearer-token auth.
|
|
69
70
|
- `GET /api/v1/artifacts/:org` returns the org artifact index behind the same auth.
|
|
71
|
+
- `POST /api/v1/scan` requires `Authorization: Bearer <HOSTED_SCAN_TOKEN>`.
|
|
70
72
|
- Hosted scans reject localhost/private-network targets; use local CLI for internal MCP servers.
|
|
71
73
|
|
|
72
74
|
## What Not To Do Yet
|