@kryptosai/mcp-observatory 0.27.0 → 0.28.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/COMMERCIAL.md +6 -1
- package/CONTRIBUTORS.md +39 -0
- package/README.md +57 -18
- package/dist/src/adapters/http.js +1 -5
- package/dist/src/adapters/http.js.map +1 -1
- package/dist/src/cassette.js +3 -2
- package/dist/src/cassette.js.map +1 -1
- package/dist/src/checks/attack-sim.d.ts +18 -0
- package/dist/src/checks/attack-sim.js +304 -0
- package/dist/src/checks/attack-sim.js.map +1 -0
- package/dist/src/ci-issue.js +4 -3
- package/dist/src/ci-issue.js.map +1 -1
- package/dist/src/cli.js +17 -3
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/attack-sim.d.ts +2 -0
- package/dist/src/commands/attack-sim.js +168 -0
- package/dist/src/commands/attack-sim.js.map +1 -0
- package/dist/src/commands/init-ci.d.ts +1 -0
- package/dist/src/commands/init-ci.js +8 -3
- package/dist/src/commands/init-ci.js.map +1 -1
- package/dist/src/commands/legacy.js +6 -1
- package/dist/src/commands/legacy.js.map +1 -1
- package/dist/src/commands/scan.js +7 -1
- package/dist/src/commands/scan.js.map +1 -1
- package/dist/src/commands/serve.js +9 -1
- package/dist/src/commands/serve.js.map +1 -1
- package/dist/src/commands/setup-ci-conversion.d.ts +1 -0
- package/dist/src/commands/setup-ci-conversion.js +1 -0
- package/dist/src/commands/setup-ci-conversion.js.map +1 -1
- package/dist/src/commands/telemetry.js +20 -14
- package/dist/src/commands/telemetry.js.map +1 -1
- package/dist/src/commands/test.js +13 -1
- package/dist/src/commands/test.js.map +1 -1
- package/dist/src/findings.js +41 -1
- package/dist/src/findings.js.map +1 -1
- package/dist/src/history.js +3 -2
- package/dist/src/history.js.map +1 -1
- package/dist/src/index.d.ts +2 -0
- package/dist/src/index.js +2 -0
- package/dist/src/index.js.map +1 -1
- package/dist/src/integrations/smithery.js +5 -1
- package/dist/src/integrations/smithery.js.map +1 -1
- package/dist/src/reporters/attack-sim.d.ts +2 -0
- package/dist/src/reporters/attack-sim.js +84 -0
- package/dist/src/reporters/attack-sim.js.map +1 -0
- package/dist/src/reporters/pr-comment.js +20 -6
- package/dist/src/reporters/pr-comment.js.map +1 -1
- package/dist/src/runner.d.ts +3 -0
- package/dist/src/runner.js +5 -0
- package/dist/src/runner.js.map +1 -1
- package/dist/src/score.js +1 -1
- package/dist/src/score.js.map +1 -1
- package/dist/src/storage.js +3 -2
- package/dist/src/storage.js.map +1 -1
- package/dist/src/telemetry.d.ts +3 -0
- package/dist/src/telemetry.js +17 -1
- package/dist/src/telemetry.js.map +1 -1
- package/dist/src/types.d.ts +1 -1
- package/dist/src/utils/files.d.ts +1 -0
- package/dist/src/utils/files.js +21 -0
- package/dist/src/utils/files.js.map +1 -0
- package/dist/src/utils/ids.js +18 -5
- package/dist/src/utils/ids.js.map +1 -1
- package/dist/src/utils/url.d.ts +2 -0
- package/dist/src/utils/url.js +15 -0
- package/dist/src/utils/url.js.map +1 -0
- package/dist/src/validate.js +1 -1
- package/dist/src/validate.js.map +1 -1
- package/docs/agent-ecosystem-promotion-plan.md +244 -0
- package/docs/agent-tasks.md +95 -0
- package/docs/automatic-ci-integration.md +33 -0
- package/docs/campaign-attribution.md +58 -0
- package/docs/code-scanning-demo.md +57 -0
- package/docs/contributor-proof-cards/README.md +33 -0
- package/docs/contributor-recognition.md +92 -0
- package/docs/feishu-lark-mcp.md +1 -1
- package/docs/github-code-scanning-for-mcp.md +1 -1
- package/docs/launch.md +64 -0
- package/docs/mcp-attack-simulator.md +57 -0
- package/docs/mcp-server-safety-index.md +18 -12
- package/docs/proof.md +5 -2
- package/docs/receipt-graph.md +54 -0
- package/docs/safety-index/artifacts/antv-chart-server.json +21 -21
- package/docs/safety-index/artifacts/antv-chart-server.md +10 -10
- package/docs/safety-index/artifacts/browsermcp-server.json +17 -17
- package/docs/safety-index/artifacts/browsermcp-server.md +10 -10
- package/docs/safety-index/artifacts/context7-server.json +17 -17
- package/docs/safety-index/artifacts/context7-server.md +10 -10
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.json +2014 -0
- package/docs/safety-index/artifacts/cyanheads-git-mcp-server.md +163 -0
- package/docs/safety-index/artifacts/everything-server.json +131 -56
- package/docs/safety-index/artifacts/everything-server.md +24 -24
- package/docs/safety-index/artifacts/executeautomation-playwright-server.json +26 -18
- package/docs/safety-index/artifacts/executeautomation-playwright-server.md +10 -10
- package/docs/safety-index/artifacts/filesystem-server.json +128 -20
- package/docs/safety-index/artifacts/filesystem-server.md +10 -10
- package/docs/safety-index/artifacts/kubernetes-server.attack.json +1559 -0
- package/docs/safety-index/artifacts/kubernetes-server.attack.md +43 -0
- package/docs/safety-index/artifacts/kubernetes-server.attack.sarif +2857 -0
- package/docs/safety-index/artifacts/kubernetes-server.json +1348 -0
- package/docs/safety-index/artifacts/kubernetes-server.md +150 -0
- package/docs/safety-index/artifacts/memory-server.json +75 -32
- package/docs/safety-index/artifacts/memory-server.md +35 -28
- package/docs/safety-index/artifacts/opentofu-server.json +25 -17
- package/docs/safety-index/artifacts/opentofu-server.md +11 -11
- package/docs/safety-index/artifacts/playwright-mcp-server.json +56 -20
- package/docs/safety-index/artifacts/playwright-mcp-server.md +12 -12
- package/docs/safety-index/artifacts/promptopia-server.json +28 -20
- package/docs/safety-index/artifacts/promptopia-server.md +11 -11
- package/docs/safety-index/artifacts/puppeteer-server.json +42 -30
- package/docs/safety-index/artifacts/puppeteer-server.md +28 -28
- package/docs/safety-index/artifacts/ref-tools-server.json +17 -17
- package/docs/safety-index/artifacts/ref-tools-server.md +9 -9
- package/docs/safety-index/artifacts/sequential-thinking-server.json +20 -21
- package/docs/safety-index/artifacts/sequential-thinking-server.md +10 -10
- package/docs/safety-index/targets.json +90 -14
- package/docs/setup-ci-doctor.md +2 -0
- package/docs/target-gallery.md +49 -0
- package/docs/target-registry.md +5 -1
- package/docs/tool-call-receipts.md +116 -0
- package/package.json +20 -2
- package/schemas/run-artifact.schema.json +1 -0
- package/scripts/postinstall.mjs +123 -0
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# MCP Observatory Run Report
|
|
2
|
+
|
|
3
|
+
Generated at 2026-07-06T05:32:58.918Z
|
|
4
|
+
|
|
5
|
+
## Target and Environment Metadata
|
|
6
|
+
|
|
7
|
+
- Target: `mcp-server-kubernetes`
|
|
8
|
+
- Adapter: `local-process`
|
|
9
|
+
- Command: `npx -y mcp-server-kubernetes`
|
|
10
|
+
- Server: `kubernetes 3.9.2`
|
|
11
|
+
- Platform: `darwin 25.5.0`
|
|
12
|
+
- Node: `v22.22.1`
|
|
13
|
+
|
|
14
|
+
## Executive Summary
|
|
15
|
+
|
|
16
|
+
**Health Score: 72/100 (C)**
|
|
17
|
+
|
|
18
|
+
| Dimension | Score | Weight |
|
|
19
|
+
| --- | --- | --- |
|
|
20
|
+
| Protocol Compliance | 100/100 | 30% |
|
|
21
|
+
| Schema Quality | 60/100 | 20% |
|
|
22
|
+
| Security | 0/100 | 20% |
|
|
23
|
+
| Reliability | 100/100 | 20% |
|
|
24
|
+
| Performance | 100/100 | 10% |
|
|
25
|
+
|
|
26
|
+
| Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
|
|
27
|
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
28
|
+
| fail | 6 | 4 | 1 | 1 | 0 | 0 | 0 |
|
|
29
|
+
|
|
30
|
+
## At a Glance
|
|
31
|
+
|
|
32
|
+
- Safety verdict: **Blocked** — One or more checks can break agent dependence and should be fixed before production use.
|
|
33
|
+
- Top risks: schema-quality: Found 7 quality finding(s) across 29 item(s): 0 warnings, 7 info.; security-lite: Found 8 security finding(s): 3 high, 3 medium, 2 low.
|
|
34
|
+
- Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
|
|
35
|
+
- Failing checks: security-lite
|
|
36
|
+
- Partial or flaky checks: schema-quality
|
|
37
|
+
- Skipped checks: none
|
|
38
|
+
- Unsupported checks: none
|
|
39
|
+
- Suggested next step: Start with the failing checks: security-lite.
|
|
40
|
+
- CI next step: `Add CI: npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y <server-package>"`
|
|
41
|
+
|
|
42
|
+
## Regressions and Recoveries
|
|
43
|
+
|
|
44
|
+
_Use the `diff` command against another run artifact to classify regressions and recoveries over time._
|
|
45
|
+
|
|
46
|
+
## Full Capability Status Table
|
|
47
|
+
|
|
48
|
+
| Focus | Check | Status | Duration (ms) | Message |
|
|
49
|
+
| --- | --- | --- | --- | --- |
|
|
50
|
+
| healthy | conformance | pass | 7.21 | All 7 conformance checks passed. |
|
|
51
|
+
| healthy | prompts | pass | 0.62 | Advertised capability responded with the minimal expected shape (1 item). |
|
|
52
|
+
| healthy | resources | pass | 1.03 | Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported. |
|
|
53
|
+
| healthy | tools | pass | 2.26 | Advertised capability responded with the minimal expected shape (23 items). |
|
|
54
|
+
| review | schema-quality | partial | 0.90 | Found 7 quality finding(s) across 29 item(s): 0 warnings, 7 info. |
|
|
55
|
+
| act now | security-lite | fail | 0.60 | Found 8 security finding(s): 3 high, 3 medium, 2 low. |
|
|
56
|
+
|
|
57
|
+
## Evidence Snippets
|
|
58
|
+
|
|
59
|
+
### conformance — pass
|
|
60
|
+
|
|
61
|
+
Summary: All 7 conformance checks passed.
|
|
62
|
+
|
|
63
|
+
- Endpoint: `conformance/check`
|
|
64
|
+
- Advertised: `true`
|
|
65
|
+
- Responded: `true`
|
|
66
|
+
- Minimal shape present: `true`
|
|
67
|
+
- Item count: `7`
|
|
68
|
+
- Identifiers: none
|
|
69
|
+
- Diagnostics: [pass] capabilities-present: Server returned capabilities object., [pass] server-info: Server provided initialization info., [pass] tools-capability-match: tools/list returned 23 tool(s). (+4 more)
|
|
70
|
+
|
|
71
|
+
### prompts — pass
|
|
72
|
+
|
|
73
|
+
Summary: Advertised capability responded with the minimal expected shape (1 item).
|
|
74
|
+
|
|
75
|
+
- Endpoint: `prompts/list`
|
|
76
|
+
- Advertised: `true`
|
|
77
|
+
- Responded: `true`
|
|
78
|
+
- Minimal shape present: `true`
|
|
79
|
+
- Item count: `1`
|
|
80
|
+
- Identifiers: k8s-diagnose
|
|
81
|
+
- Diagnostics: Starting Kubernetes MCP server v3.9.2, handling commands..., Telemetry: Disabled, (node:54973) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (+1 more)
|
|
82
|
+
|
|
83
|
+
### resources — pass
|
|
84
|
+
|
|
85
|
+
Summary: Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported.
|
|
86
|
+
|
|
87
|
+
- Endpoint: `resources/list`
|
|
88
|
+
- Advertised: `true`
|
|
89
|
+
- Responded: `true`
|
|
90
|
+
- Minimal shape present: `true`
|
|
91
|
+
- Item count: `5`
|
|
92
|
+
- Identifiers: k8s://default/pods, k8s://default/deployments, k8s://default/services, k8s://namespaces, k8s://nodes
|
|
93
|
+
- Diagnostics: Starting Kubernetes MCP server v3.9.2, handling commands..., Telemetry: Disabled, (node:54973) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (+1 more)
|
|
94
|
+
- Endpoint: `resources/templates/list`
|
|
95
|
+
- Advertised: `true`
|
|
96
|
+
- Responded: `false`
|
|
97
|
+
- Minimal shape present: `false`
|
|
98
|
+
- Item count: `0`
|
|
99
|
+
- Identifiers: none
|
|
100
|
+
- Diagnostics: MCP error -32601: Method not found, Starting Kubernetes MCP server v3.9.2, handling commands..., Telemetry: Disabled (+2 more)
|
|
101
|
+
|
|
102
|
+
### tools — pass
|
|
103
|
+
|
|
104
|
+
Summary: Advertised capability responded with the minimal expected shape (23 items).
|
|
105
|
+
|
|
106
|
+
- Endpoint: `tools/list`
|
|
107
|
+
- Advertised: `true`
|
|
108
|
+
- Responded: `true`
|
|
109
|
+
- Minimal shape present: `true`
|
|
110
|
+
- Item count: `23`
|
|
111
|
+
- Identifiers: cleanup, kubectl_get, kubectl_describe, kubectl_apply, kubectl_delete (+18 more)
|
|
112
|
+
- Diagnostics: Starting Kubernetes MCP server v3.9.2, handling commands..., Telemetry: Disabled, (node:54973) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (+1 more)
|
|
113
|
+
|
|
114
|
+
### schema-quality — partial
|
|
115
|
+
|
|
116
|
+
Summary: Found 7 quality finding(s) across 29 item(s): 0 warnings, 7 info.
|
|
117
|
+
|
|
118
|
+
- Endpoint: `schema-quality/scan`
|
|
119
|
+
- Advertised: `true`
|
|
120
|
+
- Responded: `true`
|
|
121
|
+
- Minimal shape present: `true`
|
|
122
|
+
- Item count: `7`
|
|
123
|
+
- Identifiers: port_forward, stop_port_forward, list_api_resources
|
|
124
|
+
- Diagnostics: [info] tool "port_forward": Property 'resourceType' missing description, [info] tool "port_forward": Property 'resourceName' missing description, [info] tool "port_forward": Property 'localPort' missing description (+4 more)
|
|
125
|
+
|
|
126
|
+
### security-lite — fail
|
|
127
|
+
|
|
128
|
+
Summary: Found 8 security finding(s): 3 high, 3 medium, 2 low.
|
|
129
|
+
|
|
130
|
+
- Endpoint: `security/scan-lite`
|
|
131
|
+
- Advertised: `true`
|
|
132
|
+
- Responded: `true`
|
|
133
|
+
- Minimal shape present: `true`
|
|
134
|
+
- Item count: `8`
|
|
135
|
+
- Identifiers: cleanup, kubectl_apply, kubectl_delete, kubectl_create, kubectl_reconnect (+2 more)
|
|
136
|
+
- Diagnostics: [low] Tool "cleanup" has an empty schema but is marked as destructive., [medium] Tool "kubectl_apply" accepts filesystem paths and has destructive capabilities., [medium] Tool "kubectl_delete" accepts filesystem paths and has destructive capabilities. (+5 more)
|
|
137
|
+
|
|
138
|
+
## Reproduction Commands
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
npm run cli -- run --target <path-to-target-config.json>
|
|
142
|
+
npm run cli -- report --run <path-to-run-artifact.json> --format markdown
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Artifact Provenance
|
|
146
|
+
|
|
147
|
+
- Artifact type: `run`
|
|
148
|
+
- Schema version: `1.0.0`
|
|
149
|
+
- Run ID: `run_2026-07-06T053258917Z_2e4f693a`
|
|
150
|
+
- Gate: `fail`
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
"artifactType": "run",
|
|
3
3
|
"schemaVersion": "1.0.0",
|
|
4
4
|
"gate": "pass",
|
|
5
|
-
"runId": "run_2026-
|
|
6
|
-
"createdAt": "2026-
|
|
7
|
-
"toolVersion": "0.
|
|
5
|
+
"runId": "run_2026-07-06T014652851Z_87fbcb76",
|
|
6
|
+
"createdAt": "2026-07-06T01:46:52.851Z",
|
|
7
|
+
"toolVersion": "0.26.1",
|
|
8
8
|
"target": {
|
|
9
9
|
"targetId": "memory-server",
|
|
10
10
|
"adapter": "local-process",
|
|
@@ -25,15 +25,15 @@
|
|
|
25
25
|
"serverVersion": "0.6.3"
|
|
26
26
|
},
|
|
27
27
|
"environment": {
|
|
28
|
-
"platform": "darwin
|
|
29
|
-
"nodeVersion": "
|
|
28
|
+
"platform": "darwin 24.0.0",
|
|
29
|
+
"nodeVersion": "v25.8.1"
|
|
30
30
|
},
|
|
31
31
|
"summary": {
|
|
32
32
|
"total": 7,
|
|
33
|
-
"pass":
|
|
33
|
+
"pass": 5,
|
|
34
34
|
"fail": 0,
|
|
35
35
|
"partial": 1,
|
|
36
|
-
"unsupported":
|
|
36
|
+
"unsupported": 1,
|
|
37
37
|
"flaky": 0,
|
|
38
38
|
"skipped": 0,
|
|
39
39
|
"gate": "pass"
|
|
@@ -43,7 +43,7 @@
|
|
|
43
43
|
"id": "tools",
|
|
44
44
|
"capability": "tools",
|
|
45
45
|
"status": "pass",
|
|
46
|
-
"durationMs":
|
|
46
|
+
"durationMs": 8.82391699999971,
|
|
47
47
|
"message": "Advertised capability responded with the minimal expected shape (9 items).",
|
|
48
48
|
"evidence": [
|
|
49
49
|
{
|
|
@@ -295,7 +295,7 @@
|
|
|
295
295
|
"id": "prompts",
|
|
296
296
|
"capability": "prompts",
|
|
297
297
|
"status": "unsupported",
|
|
298
|
-
"durationMs": 0.
|
|
298
|
+
"durationMs": 0.0030410000003939786,
|
|
299
299
|
"message": "Prompts are not advertised by the target.",
|
|
300
300
|
"evidence": [
|
|
301
301
|
{
|
|
@@ -310,16 +310,33 @@
|
|
|
310
310
|
{
|
|
311
311
|
"id": "resources",
|
|
312
312
|
"capability": "resources",
|
|
313
|
-
"status": "
|
|
314
|
-
"durationMs": 0.
|
|
315
|
-
"message": "
|
|
313
|
+
"status": "pass",
|
|
314
|
+
"durationMs": 0.8521670000000086,
|
|
315
|
+
"message": "Advertised capability responded with the minimal expected shape (1 items).",
|
|
316
316
|
"evidence": [
|
|
317
317
|
{
|
|
318
|
-
"endpoint": "resources/list
|
|
319
|
-
"advertised":
|
|
320
|
-
"responded":
|
|
321
|
-
"minimalShapePresent":
|
|
322
|
-
"
|
|
318
|
+
"endpoint": "resources/list",
|
|
319
|
+
"advertised": true,
|
|
320
|
+
"responded": true,
|
|
321
|
+
"minimalShapePresent": true,
|
|
322
|
+
"itemCount": 1,
|
|
323
|
+
"identifiers": [
|
|
324
|
+
"memory://knowledge-graph"
|
|
325
|
+
],
|
|
326
|
+
"diagnostics": [
|
|
327
|
+
"Knowledge Graph MCP Server running on stdio"
|
|
328
|
+
]
|
|
329
|
+
},
|
|
330
|
+
{
|
|
331
|
+
"endpoint": "resources/templates/list",
|
|
332
|
+
"advertised": true,
|
|
333
|
+
"responded": true,
|
|
334
|
+
"minimalShapePresent": true,
|
|
335
|
+
"itemCount": 0,
|
|
336
|
+
"identifiers": [],
|
|
337
|
+
"diagnostics": [
|
|
338
|
+
"Knowledge Graph MCP Server running on stdio"
|
|
339
|
+
]
|
|
323
340
|
}
|
|
324
341
|
]
|
|
325
342
|
},
|
|
@@ -327,7 +344,7 @@
|
|
|
327
344
|
"id": "security-lite",
|
|
328
345
|
"capability": "security-lite",
|
|
329
346
|
"status": "pass",
|
|
330
|
-
"durationMs": 0.
|
|
347
|
+
"durationMs": 0.03016700000034689,
|
|
331
348
|
"message": "No security issues detected (lightweight scan).",
|
|
332
349
|
"evidence": [
|
|
333
350
|
{
|
|
@@ -343,7 +360,7 @@
|
|
|
343
360
|
"id": "conformance",
|
|
344
361
|
"capability": "conformance",
|
|
345
362
|
"status": "pass",
|
|
346
|
-
"durationMs":
|
|
363
|
+
"durationMs": 10.965208000000075,
|
|
347
364
|
"message": "All 7 conformance checks passed.",
|
|
348
365
|
"evidence": [
|
|
349
366
|
{
|
|
@@ -358,7 +375,7 @@
|
|
|
358
375
|
"[pass] server-info: Server provided initialization info.",
|
|
359
376
|
"[pass] tools-capability-match: tools/list returned 9 tool(s).",
|
|
360
377
|
"[pass] prompts-capability-match: Prompts not advertised — endpoint check skipped.",
|
|
361
|
-
"[pass] resources-capability-match:
|
|
378
|
+
"[pass] resources-capability-match: resources/list returned 1 resource(s).",
|
|
362
379
|
"[pass] tool-response-content: Tool \"read_graph\" response has valid content array.",
|
|
363
380
|
"[pass] error-handling: Server returned proper error code -32601 for unknown method."
|
|
364
381
|
]
|
|
@@ -369,8 +386,8 @@
|
|
|
369
386
|
"id": "schema-quality",
|
|
370
387
|
"capability": "schema-quality",
|
|
371
388
|
"status": "partial",
|
|
372
|
-
"durationMs": 4.
|
|
373
|
-
"message": "Found 4 quality finding(s) across
|
|
389
|
+
"durationMs": 4.120625000000018,
|
|
390
|
+
"message": "Found 4 quality finding(s) across 10 item(s): 0 warnings, 4 info.",
|
|
374
391
|
"evidence": [
|
|
375
392
|
{
|
|
376
393
|
"endpoint": "schema-quality/scan",
|
|
@@ -389,6 +406,32 @@
|
|
|
389
406
|
"[info] tool \"create_relations\": Property 'relations' missing description",
|
|
390
407
|
"[info] tool \"add_observations\": Property 'observations' missing description",
|
|
391
408
|
"[info] tool \"delete_observations\": Property 'deletions' missing description"
|
|
409
|
+
],
|
|
410
|
+
"findings": [
|
|
411
|
+
{
|
|
412
|
+
"itemType": "tool",
|
|
413
|
+
"itemName": "create_entities",
|
|
414
|
+
"issue": "Property 'entities' missing description",
|
|
415
|
+
"severity": "info"
|
|
416
|
+
},
|
|
417
|
+
{
|
|
418
|
+
"itemType": "tool",
|
|
419
|
+
"itemName": "create_relations",
|
|
420
|
+
"issue": "Property 'relations' missing description",
|
|
421
|
+
"severity": "info"
|
|
422
|
+
},
|
|
423
|
+
{
|
|
424
|
+
"itemType": "tool",
|
|
425
|
+
"itemName": "add_observations",
|
|
426
|
+
"issue": "Property 'observations' missing description",
|
|
427
|
+
"severity": "info"
|
|
428
|
+
},
|
|
429
|
+
{
|
|
430
|
+
"itemType": "tool",
|
|
431
|
+
"itemName": "delete_observations",
|
|
432
|
+
"issue": "Property 'deletions' missing description",
|
|
433
|
+
"severity": "info"
|
|
434
|
+
}
|
|
392
435
|
]
|
|
393
436
|
}
|
|
394
437
|
]
|
|
@@ -397,7 +440,7 @@
|
|
|
397
440
|
"id": "security",
|
|
398
441
|
"capability": "security",
|
|
399
442
|
"status": "pass",
|
|
400
|
-
"durationMs":
|
|
443
|
+
"durationMs": 3.843583999999737,
|
|
401
444
|
"message": "No security issues detected.",
|
|
402
445
|
"evidence": [
|
|
403
446
|
{
|
|
@@ -411,7 +454,7 @@
|
|
|
411
454
|
}
|
|
412
455
|
],
|
|
413
456
|
"healthScore": {
|
|
414
|
-
"overall":
|
|
457
|
+
"overall": 89,
|
|
415
458
|
"grade": "B",
|
|
416
459
|
"dimensions": [
|
|
417
460
|
{
|
|
@@ -442,11 +485,11 @@
|
|
|
442
485
|
{
|
|
443
486
|
"name": "Reliability",
|
|
444
487
|
"weight": 0.2,
|
|
445
|
-
"score":
|
|
488
|
+
"score": 83,
|
|
446
489
|
"details": [
|
|
447
490
|
"tools: pass (100/100)",
|
|
448
491
|
"prompts: unsupported (50/100)",
|
|
449
|
-
"resources:
|
|
492
|
+
"resources: pass (100/100)"
|
|
450
493
|
]
|
|
451
494
|
},
|
|
452
495
|
{
|
|
@@ -454,16 +497,16 @@
|
|
|
454
497
|
"weight": 0.1,
|
|
455
498
|
"score": 100,
|
|
456
499
|
"details": [
|
|
457
|
-
"Connect:
|
|
458
|
-
"p95 latency:
|
|
500
|
+
"Connect: 704ms",
|
|
501
|
+
"p95 latency: 9ms (3 operations)"
|
|
459
502
|
]
|
|
460
503
|
}
|
|
461
504
|
]
|
|
462
505
|
},
|
|
463
506
|
"performanceMetrics": {
|
|
464
|
-
"connectMs":
|
|
465
|
-
"toolsListMs":
|
|
466
|
-
"promptsListMs": 0.
|
|
467
|
-
"resourcesListMs": 0.
|
|
507
|
+
"connectMs": 704.0154159999997,
|
|
508
|
+
"toolsListMs": 8.82391699999971,
|
|
509
|
+
"promptsListMs": 0.0030410000003939786,
|
|
510
|
+
"resourcesListMs": 0.8521670000000086
|
|
468
511
|
}
|
|
469
512
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# MCP Observatory Run Report
|
|
2
2
|
|
|
3
|
-
Generated at 2026-
|
|
3
|
+
Generated at 2026-07-06T01:46:52.851Z
|
|
4
4
|
|
|
5
5
|
## Target and Environment Metadata
|
|
6
6
|
|
|
@@ -8,34 +8,34 @@ Generated at 2026-06-24T02:07:23.256Z
|
|
|
8
8
|
- Adapter: `local-process`
|
|
9
9
|
- Command: `npx -y @modelcontextprotocol/server-memory`
|
|
10
10
|
- Server: `memory-server 0.6.3`
|
|
11
|
-
- Platform: `darwin
|
|
12
|
-
- Node: `
|
|
11
|
+
- Platform: `darwin 24.0.0`
|
|
12
|
+
- Node: `v25.8.1`
|
|
13
13
|
|
|
14
14
|
## Executive Summary
|
|
15
15
|
|
|
16
|
-
**Health Score:
|
|
16
|
+
**Health Score: 89/100 (B)**
|
|
17
17
|
|
|
18
18
|
| Dimension | Score | Weight |
|
|
19
19
|
| --- | --- | --- |
|
|
20
20
|
| Protocol Compliance | 100/100 | 30% |
|
|
21
21
|
| Schema Quality | 60/100 | 20% |
|
|
22
22
|
| Security | 100/100 | 20% |
|
|
23
|
-
| Reliability |
|
|
23
|
+
| Reliability | 83/100 | 20% |
|
|
24
24
|
| Performance | 100/100 | 10% |
|
|
25
25
|
|
|
26
26
|
| Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
|
|
27
27
|
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
28
|
-
| pass | 7 |
|
|
28
|
+
| pass | 7 | 5 | 0 | 1 | 1 | 0 | 0 |
|
|
29
29
|
|
|
30
30
|
## At a Glance
|
|
31
31
|
|
|
32
32
|
- Safety verdict: **Needs review** — The server is usable, but caveated checks should be reviewed before agents depend on it.
|
|
33
|
-
- Top risks: schema-quality: Found 4 quality finding(s) across
|
|
33
|
+
- Top risks: schema-quality: Found 4 quality finding(s) across 10 item(s): 0 warnings, 4 info.
|
|
34
34
|
- Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
|
|
35
35
|
- Failing checks: none
|
|
36
36
|
- Partial or flaky checks: schema-quality
|
|
37
37
|
- Skipped checks: none
|
|
38
|
-
- Unsupported checks: prompts
|
|
38
|
+
- Unsupported checks: prompts
|
|
39
39
|
- Suggested next step: Review the caveated checks next: schema-quality.
|
|
40
40
|
- CI next step: `Add CI: npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y <server-package>"`
|
|
41
41
|
|
|
@@ -47,13 +47,13 @@ _Use the `diff` command against another run artifact to classify regressions and
|
|
|
47
47
|
|
|
48
48
|
| Focus | Check | Status | Duration (ms) | Message |
|
|
49
49
|
| --- | --- | --- | --- | --- |
|
|
50
|
-
| healthy | conformance | pass |
|
|
51
|
-
| healthy |
|
|
52
|
-
| healthy | security
|
|
53
|
-
| healthy |
|
|
54
|
-
|
|
|
50
|
+
| healthy | conformance | pass | 10.97 | All 7 conformance checks passed. |
|
|
51
|
+
| healthy | resources | pass | 0.85 | Advertised capability responded with the minimal expected shape (1 items). |
|
|
52
|
+
| healthy | security | pass | 3.84 | No security issues detected. |
|
|
53
|
+
| healthy | security-lite | pass | 0.03 | No security issues detected (lightweight scan). |
|
|
54
|
+
| healthy | tools | pass | 8.82 | Advertised capability responded with the minimal expected shape (9 items). |
|
|
55
|
+
| review | schema-quality | partial | 4.12 | Found 4 quality finding(s) across 10 item(s): 0 warnings, 4 info. |
|
|
55
56
|
| confirm intent | prompts | unsupported | 0.00 | Prompts are not advertised by the target. |
|
|
56
|
-
| confirm intent | resources | unsupported | 0.00 | Resources are not advertised by the target. |
|
|
57
57
|
|
|
58
58
|
## Evidence Snippets
|
|
59
59
|
|
|
@@ -69,6 +69,25 @@ Summary: All 7 conformance checks passed.
|
|
|
69
69
|
- Identifiers: none
|
|
70
70
|
- Diagnostics: [pass] capabilities-present: Server returned capabilities object., [pass] server-info: Server provided initialization info., [pass] tools-capability-match: tools/list returned 9 tool(s). (+4 more)
|
|
71
71
|
|
|
72
|
+
### resources — pass
|
|
73
|
+
|
|
74
|
+
Summary: Advertised capability responded with the minimal expected shape (1 items).
|
|
75
|
+
|
|
76
|
+
- Endpoint: `resources/list`
|
|
77
|
+
- Advertised: `true`
|
|
78
|
+
- Responded: `true`
|
|
79
|
+
- Minimal shape present: `true`
|
|
80
|
+
- Item count: `1`
|
|
81
|
+
- Identifiers: memory://knowledge-graph
|
|
82
|
+
- Diagnostics: Knowledge Graph MCP Server running on stdio
|
|
83
|
+
- Endpoint: `resources/templates/list`
|
|
84
|
+
- Advertised: `true`
|
|
85
|
+
- Responded: `true`
|
|
86
|
+
- Minimal shape present: `true`
|
|
87
|
+
- Item count: `0`
|
|
88
|
+
- Identifiers: none
|
|
89
|
+
- Diagnostics: Knowledge Graph MCP Server running on stdio
|
|
90
|
+
|
|
72
91
|
### security — pass
|
|
73
92
|
|
|
74
93
|
Summary: No security issues detected.
|
|
@@ -107,7 +126,7 @@ Summary: Advertised capability responded with the minimal expected shape (9 item
|
|
|
107
126
|
|
|
108
127
|
### schema-quality — partial
|
|
109
128
|
|
|
110
|
-
Summary: Found 4 quality finding(s) across
|
|
129
|
+
Summary: Found 4 quality finding(s) across 10 item(s): 0 warnings, 4 info.
|
|
111
130
|
|
|
112
131
|
- Endpoint: `schema-quality/scan`
|
|
113
132
|
- Advertised: `true`
|
|
@@ -129,18 +148,6 @@ Summary: Prompts are not advertised by the target.
|
|
|
129
148
|
- Identifiers: none
|
|
130
149
|
- Diagnostics: none
|
|
131
150
|
|
|
132
|
-
### resources — unsupported
|
|
133
|
-
|
|
134
|
-
Summary: Resources are not advertised by the target.
|
|
135
|
-
|
|
136
|
-
- Endpoint: `resources/list | resources/templates/list`
|
|
137
|
-
- Advertised: `false`
|
|
138
|
-
- Responded: `false`
|
|
139
|
-
- Minimal shape present: `false`
|
|
140
|
-
- Item count: `0`
|
|
141
|
-
- Identifiers: none
|
|
142
|
-
- Diagnostics: none
|
|
143
|
-
|
|
144
151
|
## Reproduction Commands
|
|
145
152
|
|
|
146
153
|
```bash
|
|
@@ -152,5 +159,5 @@ npm run cli -- report --run <path-to-run-artifact.json> --format markdown
|
|
|
152
159
|
|
|
153
160
|
- Artifact type: `run`
|
|
154
161
|
- Schema version: `1.0.0`
|
|
155
|
-
- Run ID: `run_2026-
|
|
162
|
+
- Run ID: `run_2026-07-06T014652851Z_87fbcb76`
|
|
156
163
|
- Gate: `pass`
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
"artifactType": "run",
|
|
3
3
|
"schemaVersion": "1.0.0",
|
|
4
4
|
"gate": "pass",
|
|
5
|
-
"runId": "run_2026-
|
|
6
|
-
"createdAt": "2026-
|
|
7
|
-
"toolVersion": "0.
|
|
5
|
+
"runId": "run_2026-07-06T014700071Z_aaaf7fe5",
|
|
6
|
+
"createdAt": "2026-07-06T01:47:00.071Z",
|
|
7
|
+
"toolVersion": "0.26.1",
|
|
8
8
|
"target": {
|
|
9
9
|
"targetId": "opentofu-server",
|
|
10
10
|
"adapter": "local-process",
|
|
@@ -25,8 +25,8 @@
|
|
|
25
25
|
"serverVersion": "0.1.0"
|
|
26
26
|
},
|
|
27
27
|
"environment": {
|
|
28
|
-
"platform": "darwin
|
|
29
|
-
"nodeVersion": "
|
|
28
|
+
"platform": "darwin 24.0.0",
|
|
29
|
+
"nodeVersion": "v25.8.1"
|
|
30
30
|
},
|
|
31
31
|
"summary": {
|
|
32
32
|
"total": 7,
|
|
@@ -43,7 +43,7 @@
|
|
|
43
43
|
"id": "tools",
|
|
44
44
|
"capability": "tools",
|
|
45
45
|
"status": "pass",
|
|
46
|
-
"durationMs": 2.
|
|
46
|
+
"durationMs": 2.2472500000003492,
|
|
47
47
|
"message": "Advertised capability responded with the minimal expected shape (5 items).",
|
|
48
48
|
"evidence": [
|
|
49
49
|
{
|
|
@@ -206,7 +206,7 @@
|
|
|
206
206
|
"id": "prompts",
|
|
207
207
|
"capability": "prompts",
|
|
208
208
|
"status": "unsupported",
|
|
209
|
-
"durationMs": 0.
|
|
209
|
+
"durationMs": 0.006666999999652035,
|
|
210
210
|
"message": "Prompts are not advertised by the target.",
|
|
211
211
|
"evidence": [
|
|
212
212
|
{
|
|
@@ -222,7 +222,7 @@
|
|
|
222
222
|
"id": "resources",
|
|
223
223
|
"capability": "resources",
|
|
224
224
|
"status": "pass",
|
|
225
|
-
"durationMs":
|
|
225
|
+
"durationMs": 0.5788329999995767,
|
|
226
226
|
"message": "Advertised capability responded with the minimal expected shape (1 items).",
|
|
227
227
|
"evidence": [
|
|
228
228
|
{
|
|
@@ -251,7 +251,7 @@
|
|
|
251
251
|
"id": "security-lite",
|
|
252
252
|
"capability": "security-lite",
|
|
253
253
|
"status": "pass",
|
|
254
|
-
"durationMs": 0.
|
|
254
|
+
"durationMs": 0.032666999999491964,
|
|
255
255
|
"message": "No security issues detected (lightweight scan).",
|
|
256
256
|
"evidence": [
|
|
257
257
|
{
|
|
@@ -267,7 +267,7 @@
|
|
|
267
267
|
"id": "conformance",
|
|
268
268
|
"capability": "conformance",
|
|
269
269
|
"status": "pass",
|
|
270
|
-
"durationMs":
|
|
270
|
+
"durationMs": 0.7163330000003043,
|
|
271
271
|
"message": "All 7 conformance checks passed.",
|
|
272
272
|
"evidence": [
|
|
273
273
|
{
|
|
@@ -293,7 +293,7 @@
|
|
|
293
293
|
"id": "schema-quality",
|
|
294
294
|
"capability": "schema-quality",
|
|
295
295
|
"status": "partial",
|
|
296
|
-
"durationMs": 0.
|
|
296
|
+
"durationMs": 0.5057919999999285,
|
|
297
297
|
"message": "Found 1 quality finding(s) across 6 item(s): 1 warnings, 0 info.",
|
|
298
298
|
"evidence": [
|
|
299
299
|
{
|
|
@@ -307,6 +307,14 @@
|
|
|
307
307
|
],
|
|
308
308
|
"diagnostics": [
|
|
309
309
|
"[warning] resource \"opentofu-registry-info\": Missing description"
|
|
310
|
+
],
|
|
311
|
+
"findings": [
|
|
312
|
+
{
|
|
313
|
+
"itemType": "resource",
|
|
314
|
+
"itemName": "opentofu-registry-info",
|
|
315
|
+
"issue": "Missing description",
|
|
316
|
+
"severity": "warning"
|
|
317
|
+
}
|
|
310
318
|
]
|
|
311
319
|
}
|
|
312
320
|
]
|
|
@@ -315,7 +323,7 @@
|
|
|
315
323
|
"id": "security",
|
|
316
324
|
"capability": "security",
|
|
317
325
|
"status": "pass",
|
|
318
|
-
"durationMs": 0.
|
|
326
|
+
"durationMs": 0.2513330000001588,
|
|
319
327
|
"message": "No security issues detected.",
|
|
320
328
|
"evidence": [
|
|
321
329
|
{
|
|
@@ -372,16 +380,16 @@
|
|
|
372
380
|
"weight": 0.1,
|
|
373
381
|
"score": 100,
|
|
374
382
|
"details": [
|
|
375
|
-
"Connect:
|
|
383
|
+
"Connect: 1133ms",
|
|
376
384
|
"p95 latency: 2ms (3 operations)"
|
|
377
385
|
]
|
|
378
386
|
}
|
|
379
387
|
]
|
|
380
388
|
},
|
|
381
389
|
"performanceMetrics": {
|
|
382
|
-
"connectMs":
|
|
383
|
-
"toolsListMs": 2.
|
|
384
|
-
"promptsListMs": 0.
|
|
385
|
-
"resourcesListMs":
|
|
390
|
+
"connectMs": 1133.1395840000005,
|
|
391
|
+
"toolsListMs": 2.2472500000003492,
|
|
392
|
+
"promptsListMs": 0.006666999999652035,
|
|
393
|
+
"resourcesListMs": 0.5788329999995767
|
|
386
394
|
}
|
|
387
395
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# MCP Observatory Run Report
|
|
2
2
|
|
|
3
|
-
Generated at 2026-
|
|
3
|
+
Generated at 2026-07-06T01:47:00.071Z
|
|
4
4
|
|
|
5
5
|
## Target and Environment Metadata
|
|
6
6
|
|
|
@@ -8,8 +8,8 @@ Generated at 2026-06-24T02:07:29.328Z
|
|
|
8
8
|
- Adapter: `local-process`
|
|
9
9
|
- Command: `npx -y @opentofu/opentofu-mcp-server`
|
|
10
10
|
- Server: `opentofu 0.1.0`
|
|
11
|
-
- Platform: `darwin
|
|
12
|
-
- Node: `
|
|
11
|
+
- Platform: `darwin 24.0.0`
|
|
12
|
+
- Node: `v25.8.1`
|
|
13
13
|
|
|
14
14
|
## Executive Summary
|
|
15
15
|
|
|
@@ -47,13 +47,13 @@ _Use the `diff` command against another run artifact to classify regressions and
|
|
|
47
47
|
|
|
48
48
|
| Focus | Check | Status | Duration (ms) | Message |
|
|
49
49
|
| --- | --- | --- | --- | --- |
|
|
50
|
-
| healthy | conformance | pass |
|
|
51
|
-
| healthy | resources | pass |
|
|
52
|
-
| healthy | security | pass | 0.
|
|
53
|
-
| healthy | security-lite | pass | 0.
|
|
54
|
-
| healthy | tools | pass | 2.
|
|
55
|
-
| review | schema-quality | partial | 0.
|
|
56
|
-
| confirm intent | prompts | unsupported | 0.
|
|
50
|
+
| healthy | conformance | pass | 0.72 | All 7 conformance checks passed. |
|
|
51
|
+
| healthy | resources | pass | 0.58 | Advertised capability responded with the minimal expected shape (1 items). |
|
|
52
|
+
| healthy | security | pass | 0.25 | No security issues detected. |
|
|
53
|
+
| healthy | security-lite | pass | 0.03 | No security issues detected (lightweight scan). |
|
|
54
|
+
| healthy | tools | pass | 2.25 | Advertised capability responded with the minimal expected shape (5 items). |
|
|
55
|
+
| review | schema-quality | partial | 0.51 | Found 1 quality finding(s) across 6 item(s): 1 warnings, 0 info. |
|
|
56
|
+
| confirm intent | prompts | unsupported | 0.01 | Prompts are not advertised by the target. |
|
|
57
57
|
|
|
58
58
|
## Evidence Snippets
|
|
59
59
|
|
|
@@ -159,5 +159,5 @@ npm run cli -- report --run <path-to-run-artifact.json> --format markdown
|
|
|
159
159
|
|
|
160
160
|
- Artifact type: `run`
|
|
161
161
|
- Schema version: `1.0.0`
|
|
162
|
-
- Run ID: `run_2026-
|
|
162
|
+
- Run ID: `run_2026-07-06T014700071Z_aaaf7fe5`
|
|
163
163
|
- Gate: `pass`
|