@kryptosai/mcp-observatory 0.23.0 → 0.24.0

package/docs/safety-index/artifacts/promptopia-server.md

@@ -0,0 +1,156 @@
+# MCP Observatory Run Report
+
+Generated at 2026-06-24T02:07:25.699Z
+
+## Target and Environment Metadata
+
+- Target: `promptopia-server`
+- Adapter: `local-process`
+- Command: `npx -y promptopia-mcp`
+- Server: `promptopia-mcp 1.1.0`
+- Platform: `darwin 25.5.0`
+- Node: `v22.22.1`
+
+## Executive Summary
+
+**Health Score: 89/100 (B)**
+
+| Dimension | Score | Weight |
+| --- | --- | --- |
+| Protocol Compliance | 100/100 | 30% |
+| Schema Quality | 60/100 | 20% |
+| Security | 100/100 | 20% |
+| Reliability | 83/100 | 20% |
+| Performance | 100/100 | 10% |
+
+| Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| pass | 7 | 5 | 0 | 1 | 1 | 0 | 0 |
+
+## At a Glance
+
+- Safety verdict: **Needs review** — The server is usable, but caveated checks should be reviewed before agents depend on it.
+- Top risks: schema-quality: Found 1 quality finding(s) across 8 item(s): 0 warnings, 1 info.
+- Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
+- Failing checks: none
+- Partial or flaky checks: schema-quality
+- Skipped checks: none
+- Unsupported checks: resources
+- Suggested next step: Review the caveated checks next: schema-quality.
+- CI next step: `Add CI: npx @kryptosai/mcp-observatory init-ci --all --command "npx -y <server-package>"`
+
+## Regressions and Recoveries
+
+_Use the `diff` command against another run artifact to classify regressions and recoveries over time._
+
+## Full Capability Status Table
+
+| Focus | Check | Status | Duration (ms) | Message |
+| --- | --- | --- | --- | --- |
+| healthy | conformance | pass | 1.66 | All 7 conformance checks passed. |
+| healthy | prompts | pass | 1.13 | Advertised capability responded with the minimal expected shape (1 item). |
+| healthy | security | pass | 0.19 | No security issues detected. |
+| healthy | security-lite | pass | 0.03 | No security issues detected (lightweight scan). |
+| healthy | tools | pass | 1.03 | Advertised capability responded with the minimal expected shape (7 items). |
+| review | schema-quality | partial | 0.66 | Found 1 quality finding(s) across 8 item(s): 0 warnings, 1 info. |
+| confirm intent | resources | unsupported | 0.00 | Resources are not advertised by the target. |
+
+## Evidence Snippets
+
+### conformance — pass
+
+Summary: All 7 conformance checks passed.
+
+- Endpoint: `conformance/check`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `7`
+  - Identifiers: none
+  - Diagnostics: [pass] capabilities-present: Server returned capabilities object., [pass] server-info: Server provided initialization info., [pass] tools-capability-match: tools/list returned 7 tool(s). (+4 more)
+
+### prompts — pass
+
+Summary: Advertised capability responded with the minimal expected shape (1 item).
+
+- Endpoint: `prompts/list`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `1`
+  - Identifiers: demo_welcome
+  - Diagnostics: Watching for changes in prompts directory: /Users/williamweishuhn/Documents/New project/mcp-observatory/examples/promptopia-prompts, promptopia-mcp MCP server running (v1.1.0)
+
+### security — pass
+
+Summary: No security issues detected.
+
+- Endpoint: `security/scan`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+### security-lite — pass
+
+Summary: No security issues detected (lightweight scan).
+
+- Endpoint: `security/scan-lite`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+### tools — pass
+
+Summary: Advertised capability responded with the minimal expected shape (7 items).
+
+- Endpoint: `tools/list`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `7`
+  - Identifiers: add_prompt, update_prompt, get_prompt, list_prompts, delete_prompt (+2 more)
+  - Diagnostics: Watching for changes in prompts directory: /Users/williamweishuhn/Documents/New project/mcp-observatory/examples/promptopia-prompts, promptopia-mcp MCP server running (v1.1.0)
+
+### schema-quality — partial
+
+Summary: Found 1 quality finding(s) across 8 item(s): 0 warnings, 1 info.
+
+- Endpoint: `schema-quality/scan`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `1`
+  - Identifiers: demo_welcome
+  - Diagnostics: [info] prompt "demo_welcome": Argument 'name' missing description
+
+### resources — unsupported
+
+Summary: Resources are not advertised by the target.
+
+- Endpoint: `resources/list | resources/templates/list`
+  - Advertised: `false`
+  - Responded: `false`
+  - Minimal shape present: `false`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+## Reproduction Commands
+
+```bash
+npm run cli -- run --target <path-to-target-config.json>
+npm run cli -- report --run <path-to-run-artifact.json> --format markdown
+```
+
+## Artifact Provenance
+
+- Artifact type: `run`
+- Schema version: `1.0.0`
+- Run ID: `run_2026-06-24T020725699Z_7cde9d35`
+- Gate: `pass`

package/docs/safety-index/artifacts/puppeteer-server.json

@@ -0,0 +1,377 @@
+{
+  "artifactType": "run",
+  "schemaVersion": "1.0.0",
+  "gate": "pass",
+  "runId": "run_2026-06-24T020730285Z_85ad0698",
+  "createdAt": "2026-06-24T02:07:30.285Z",
+  "toolVersion": "0.24.0",
+  "target": {
+    "targetId": "puppeteer-server",
+    "adapter": "local-process",
+    "command": "npx",
+    "args": [
+      "-y",
+      "puppeteer-mcp-server"
+    ],
+    "cwd": ".",
+    "metadata": {
+      "package": "puppeteer-mcp-server",
+      "purpose": "mcp-safety-index",
+      "riskClass": "Browser control",
+      "failureClass": "Browser/code execution boundary",
+      "whyItMatters": "Browser automation gives agents navigation, page state, and sometimes code-evaluation powers."
+    },
+    "serverName": "example-servers/puppeteer",
+    "serverVersion": "0.1.0"
+  },
+  "environment": {
+    "platform": "darwin 25.5.0",
+    "nodeVersion": "v22.22.1"
+  },
+  "summary": {
+    "total": 7,
+    "pass": 4,
+    "fail": 0,
+    "partial": 2,
+    "unsupported": 1,
+    "flaky": 0,
+    "skipped": 0,
+    "gate": "pass"
+  },
+  "checks": [
+    {
+      "id": "tools",
+      "capability": "tools",
+      "status": "pass",
+      "durationMs": 0.41391699999985576,
+      "message": "Advertised capability responded with the minimal expected shape (8 items).",
+      "evidence": [
+        {
+          "endpoint": "tools/list",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 8,
+          "identifiers": [
+            "puppeteer_connect_active_tab",
+            "puppeteer_navigate",
+            "puppeteer_screenshot",
+            "puppeteer_click",
+            "puppeteer_fill",
+            "puppeteer_select",
+            "puppeteer_hover",
+            "puppeteer_evaluate"
+          ],
+          "diagnostics": [],
+          "schemas": {
+            "puppeteer_connect_active_tab": {
+              "type": "object",
+              "properties": {
+                "targetUrl": {
+                  "type": "string",
+                  "description": "Optional URL of the target tab to connect to. If not provided, connects to the first available tab."
+                },
+                "debugPort": {
+                  "type": "number",
+                  "description": "Optional Chrome debugging port (default: 9222)",
+                  "default": 9222
+                }
+              },
+              "required": []
+            },
+            "puppeteer_navigate": {
+              "type": "object",
+              "properties": {
+                "url": {
+                  "type": "string"
+                }
+              },
+              "required": [
+                "url"
+              ]
+            },
+            "puppeteer_screenshot": {
+              "type": "object",
+              "properties": {
+                "name": {
+                  "type": "string",
+                  "description": "Name for the screenshot"
+                },
+                "selector": {
+                  "type": "string",
+                  "description": "CSS selector for element to screenshot"
+                },
+                "width": {
+                  "type": "number",
+                  "description": "Width in pixels (default: 800)"
+                },
+                "height": {
+                  "type": "number",
+                  "description": "Height in pixels (default: 600)"
+                }
+              },
+              "required": [
+                "name"
+              ]
+            },
+            "puppeteer_click": {
+              "type": "object",
+              "properties": {
+                "selector": {
+                  "type": "string",
+                  "description": "CSS selector for element to click"
+                }
+              },
+              "required": [
+                "selector"
+              ]
+            },
+            "puppeteer_fill": {
+              "type": "object",
+              "properties": {
+                "selector": {
+                  "type": "string",
+                  "description": "CSS selector for input field"
+                },
+                "value": {
+                  "type": "string",
+                  "description": "Value to fill"
+                }
+              },
+              "required": [
+                "selector",
+                "value"
+              ]
+            },
+            "puppeteer_select": {
+              "type": "object",
+              "properties": {
+                "selector": {
+                  "type": "string",
+                  "description": "CSS selector for element to select"
+                },
+                "value": {
+                  "type": "string",
+                  "description": "Value to select"
+                }
+              },
+              "required": [
+                "selector",
+                "value"
+              ]
+            },
+            "puppeteer_hover": {
+              "type": "object",
+              "properties": {
+                "selector": {
+                  "type": "string",
+                  "description": "CSS selector for element to hover"
+                }
+              },
+              "required": [
+                "selector"
+              ]
+            },
+            "puppeteer_evaluate": {
+              "type": "object",
+              "properties": {
+                "script": {
+                  "type": "string",
+                  "description": "JavaScript code to execute"
+                }
+              },
+              "required": [
+                "script"
+              ]
+            }
+          }
+        }
+      ]
+    },
+    {
+      "id": "prompts",
+      "capability": "prompts",
+      "status": "unsupported",
+      "durationMs": 0.0010000000002037268,
+      "message": "Prompts are not advertised by the target.",
+      "evidence": [
+        {
+          "endpoint": "prompts/list",
+          "advertised": false,
+          "responded": false,
+          "minimalShapePresent": false,
+          "diagnostics": []
+        }
+      ]
+    },
+    {
+      "id": "resources",
+      "capability": "resources",
+      "status": "pass",
+      "durationMs": 0.864040999998906,
+      "message": "Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported.",
+      "evidence": [
+        {
+          "endpoint": "resources/list",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 1,
+          "identifiers": [
+            "console://logs"
+          ],
+          "diagnostics": []
+        },
+        {
+          "endpoint": "resources/templates/list",
+          "advertised": true,
+          "responded": false,
+          "minimalShapePresent": false,
+          "diagnostics": [
+            "MCP error -32601: Method not found"
+          ]
+        }
+      ]
+    },
+    {
+      "id": "security-lite",
+      "capability": "security-lite",
+      "status": "pass",
+      "durationMs": 0.041500000001178705,
+      "message": "No security issues detected (lightweight scan).",
+      "evidence": [
+        {
+          "endpoint": "security/scan-lite",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 0
+        }
+      ]
+    },
+    {
+      "id": "conformance",
+      "capability": "conformance",
+      "status": "partial",
+      "durationMs": 2.417708999999377,
+      "message": "6/7 conformance checks passed, 1 failed.",
+      "evidence": [
+        {
+          "endpoint": "conformance/check",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": false,
+          "itemCount": 7,
+          "identifiers": [
+            "tool-response-content"
+          ],
+          "diagnostics": [
+            "[pass] capabilities-present: Server returned capabilities object.",
+            "[pass] server-info: Server provided initialization info.",
+            "[pass] tools-capability-match: tools/list returned 8 tool(s).",
+            "[pass] prompts-capability-match: Prompts not advertised — endpoint check skipped.",
+            "[pass] resources-capability-match: resources/list returned 1 resource(s).",
+            "[FAIL] tool-response-content: Tool response content check failed: MCP error -32603: Could not find Chrome (ver. 131.0.6778.204). This can occur if either\n 1. you did not perform an installation before running the script (e.g. `npx puppeteer browsers install chrome`) or\n 2. your cache path is incorrectly configured (which is: /Users/williamweishuhn/.cache/puppeteer).\nFor (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.",
+            "[pass] error-handling: Server returned proper error code -32601 for unknown method."
+          ]
+        }
+      ]
+    },
+    {
+      "id": "schema-quality",
+      "capability": "schema-quality",
+      "status": "partial",
+      "durationMs": 0.33158299999922747,
+      "message": "Found 2 quality finding(s) across 9 item(s): 1 warnings, 1 info.",
+      "evidence": [
+        {
+          "endpoint": "schema-quality/scan",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 2,
+          "identifiers": [
+            "puppeteer_navigate",
+            "Browser console logs"
+          ],
+          "diagnostics": [
+            "[info] tool \"puppeteer_navigate\": Property 'url' missing description",
+            "[warning] resource \"Browser console logs\": Missing description"
+          ]
+        }
+      ]
+    },
+    {
+      "id": "security",
+      "capability": "security",
+      "status": "pass",
+      "durationMs": 0.18262499999946158,
+      "message": "No security issues detected.",
+      "evidence": [
+        {
+          "endpoint": "security/scan",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 0
+        }
+      ]
+    }
+  ],
+  "healthScore": {
+    "overall": 77,
+    "grade": "C",
+    "dimensions": [
+      {
+        "name": "Protocol Compliance",
+        "weight": 0.3,
+        "score": 60,
+        "details": [
+          "conformance: partial (60/100)"
+        ]
+      },
+      {
+        "name": "Schema Quality",
+        "weight": 0.2,
+        "score": 60,
+        "details": [
+          "schema-quality: partial (60/100)"
+        ]
+      },
+      {
+        "name": "Security",
+        "weight": 0.2,
+        "score": 100,
+        "details": [
+          "security-lite: pass (100/100)",
+          "security: pass (100/100)"
+        ]
+      },
+      {
+        "name": "Reliability",
+        "weight": 0.2,
+        "score": 83,
+        "details": [
+          "tools: pass (100/100)",
+          "prompts: unsupported (50/100)",
+          "resources: pass (100/100)"
+        ]
+      },
+      {
+        "name": "Performance",
+        "weight": 0.1,
+        "score": 100,
+        "details": [
+          "Connect: 812ms",
+          "p95 latency: 1ms (3 operations)"
+        ]
+      }
+    ]
+  },
+  "performanceMetrics": {
+    "connectMs": 812.1966250000005,
+    "toolsListMs": 0.41391699999985576,
+    "promptsListMs": 0.0010000000002037268,
+    "resourcesListMs": 0.864040999998906
+  }
+}

package/docs/safety-index/artifacts/puppeteer-server.md

@@ -0,0 +1,163 @@
+# MCP Observatory Run Report
+
+Generated at 2026-06-24T02:07:30.285Z
+
+## Target and Environment Metadata
+
+- Target: `puppeteer-server`
+- Adapter: `local-process`
+- Command: `npx -y puppeteer-mcp-server`
+- Server: `example-servers/puppeteer 0.1.0`
+- Platform: `darwin 25.5.0`
+- Node: `v22.22.1`
+
+## Executive Summary
+
+**Health Score: 77/100 (C)**
+
+| Dimension | Score | Weight |
+| --- | --- | --- |
+| Protocol Compliance | 60/100 | 30% |
+| Schema Quality | 60/100 | 20% |
+| Security | 100/100 | 20% |
+| Reliability | 83/100 | 20% |
+| Performance | 100/100 | 10% |
+
+| Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| pass | 7 | 4 | 0 | 2 | 1 | 0 | 0 |
+
+## At a Glance
+
+- Safety verdict: **Needs review** — The server is usable, but caveated checks should be reviewed before agents depend on it.
+- Top risks: conformance: 6/7 conformance checks passed, 1 failed.; schema-quality: Found 2 quality finding(s) across 9 item(s): 1 warnings, 1 info.
+- Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
+- Failing checks: none
+- Partial or flaky checks: conformance, schema-quality
+- Skipped checks: none
+- Unsupported checks: prompts
+- Suggested next step: Review the caveated checks next: conformance, schema-quality.
+- CI next step: `Add CI: npx @kryptosai/mcp-observatory init-ci --all --command "npx -y <server-package>"`
+
+## Regressions and Recoveries
+
+_Use the `diff` command against another run artifact to classify regressions and recoveries over time._
+
+## Full Capability Status Table
+
+| Focus | Check | Status | Duration (ms) | Message |
+| --- | --- | --- | --- | --- |
+| healthy | resources | pass | 0.86 | Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported. |
+| healthy | security | pass | 0.18 | No security issues detected. |
+| healthy | security-lite | pass | 0.04 | No security issues detected (lightweight scan). |
+| healthy | tools | pass | 0.41 | Advertised capability responded with the minimal expected shape (8 items). |
+| review | conformance | partial | 2.42 | 6/7 conformance checks passed, 1 failed. |
+| review | schema-quality | partial | 0.33 | Found 2 quality finding(s) across 9 item(s): 1 warnings, 1 info. |
+| confirm intent | prompts | unsupported | 0.00 | Prompts are not advertised by the target. |
+
+## Evidence Snippets
+
+### resources — pass
+
+Summary: Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported.
+
+- Endpoint: `resources/list`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `1`
+  - Identifiers: console://logs
+  - Diagnostics: none
+- Endpoint: `resources/templates/list`
+  - Advertised: `true`
+  - Responded: `false`
+  - Minimal shape present: `false`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: MCP error -32601: Method not found
+
+### security — pass
+
+Summary: No security issues detected.
+
+- Endpoint: `security/scan`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+### security-lite — pass
+
+Summary: No security issues detected (lightweight scan).
+
+- Endpoint: `security/scan-lite`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+### tools — pass
+
+Summary: Advertised capability responded with the minimal expected shape (8 items).
+
+- Endpoint: `tools/list`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `8`
+  - Identifiers: puppeteer_connect_active_tab, puppeteer_navigate, puppeteer_screenshot, puppeteer_click, puppeteer_fill (+3 more)
+  - Diagnostics: none
+
+### conformance — partial
+
+Summary: 6/7 conformance checks passed, 1 failed.
+
+- Endpoint: `conformance/check`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `false`
+  - Item count: `7`
+  - Identifiers: tool-response-content
+  - Diagnostics: [pass] capabilities-present: Server returned capabilities object., [pass] server-info: Server provided initialization info., [pass] tools-capability-match: tools/list returned 8 tool(s). (+4 more)
+
+### schema-quality — partial
+
+Summary: Found 2 quality finding(s) across 9 item(s): 1 warnings, 1 info.
+
+- Endpoint: `schema-quality/scan`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `2`
+  - Identifiers: puppeteer_navigate, Browser console logs
+  - Diagnostics: [info] tool "puppeteer_navigate": Property 'url' missing description, [warning] resource "Browser console logs": Missing description
+
+### prompts — unsupported
+
+Summary: Prompts are not advertised by the target.
+
+- Endpoint: `prompts/list`
+  - Advertised: `false`
+  - Responded: `false`
+  - Minimal shape present: `false`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
+## Reproduction Commands
+
+```bash
+npm run cli -- run --target <path-to-target-config.json>
+npm run cli -- report --run <path-to-run-artifact.json> --format markdown
+```
+
+## Artifact Provenance
+
+- Artifact type: `run`
+- Schema version: `1.0.0`
+- Run ID: `run_2026-06-24T020730285Z_85ad0698`
+- Gate: `pass`