@kryptosai/mcp-observatory 0.27.0 → 1.28.1

package/docs/safety-index/artifacts/everything-server.md

@@ -1,6 +1,6 @@
 # MCP Observatory Run Report
 
-Generated at 2026-06-24T02:07:21.282Z
+Generated at 2026-07-06T19:49:44.860Z
 
 ## Target and Environment Metadata
 
@@ -25,12 +25,12 @@ Generated at 2026-06-24T02:07:21.282Z
 
 | Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| pass | 7 | 6 | 0 | 1 | 0 | 0 | 0 |
+| pass | 8 | 7 | 0 | 1 | 0 | 0 | 0 |
 
 ## At a Glance
 
 - Safety verdict: **Needs review** — The server is usable, but caveated checks should be reviewed before agents depend on it.
-- Top risks: schema-quality: Found 5 quality finding(s) across 24 item(s): 0 warnings, 5 info.
+- Top risks: schema-quality: Found 6 quality finding(s) across 24 item(s): 0 warnings, 6 info.
 - Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
 - Failing checks: none
 - Partial or flaky checks: schema-quality
@@ -47,16 +47,29 @@ _Use the `diff` command against another run artifact to classify regressions and
 
 | Focus | Check | Status | Duration (ms) | Message |
 | --- | --- | --- | --- | --- |
-| healthy | conformance | pass | 3.85 | All 7 conformance checks passed. |
-| healthy | prompts | pass | 0.74 | Advertised capability responded with the minimal expected shape (4 items). |
-| healthy | resources | pass | 1.01 | Advertised capability responded with the minimal expected shape (9 items). |
-| healthy | security | pass | 0.88 | No security issues detected. |
-| healthy | security-lite | pass | 0.48 | No security issues detected (lightweight scan). |
-| healthy | tools | pass | 7.35 | Advertised capability responded with the minimal expected shape (13 items). |
-| review | schema-quality | partial | 1.89 | Found 5 quality finding(s) across 24 item(s): 0 warnings, 5 info. |
+| healthy | attack-sim | pass | 2.65 | Safe attack simulation found no high-risk MCP attack-readiness findings. |
+| healthy | conformance | pass | 3.46 | All 7 conformance checks passed. |
+| healthy | prompts | pass | 0.66 | Advertised capability responded with the minimal expected shape (4 items). |
+| healthy | resources | pass | 1.08 | Advertised capability responded with the minimal expected shape (9 items). |
+| healthy | security | pass | 0.90 | Found 2 security finding(s): 0 high, 0 medium, 2 low. |
+| healthy | security-lite | pass | 0.50 | Found 2 security finding(s): 0 high, 0 medium, 2 low. |
+| healthy | tools | pass | 6.71 | Advertised capability responded with the minimal expected shape (13 items). |
+| review | schema-quality | partial | 1.34 | Found 6 quality finding(s) across 24 item(s): 0 warnings, 6 info. |
 
 ## Evidence Snippets
 
+### attack-sim — pass
+
+Summary: Safe attack simulation found no high-risk MCP attack-readiness findings.
+
+- Endpoint: `attack-sim/safe`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `0`
+  - Identifiers: none
+  - Diagnostics: none
+
 ### conformance — pass
 
 Summary: All 7 conformance checks passed.
@@ -102,27 +115,27 @@ Summary: Advertised capability responded with the minimal expected shape (9 item
 
 ### security — pass
 
-Summary: No security issues detected.
+Summary: Found 2 security finding(s): 0 high, 0 medium, 2 low.
 
 - Endpoint: `security/scan`
   - Advertised: `true`
   - Responded: `true`
   - Minimal shape present: `true`
-  - Item count: `0`
-  - Identifiers: none
-  - Diagnostics: none
+  - Item count: `2`
+  - Identifiers: toggle-simulated-logging, toggle-subscriber-updates
+  - Diagnostics: [low] Tool "toggle-simulated-logging" has an empty schema but is marked as destructive., [low] Tool "toggle-subscriber-updates" has an empty schema but is marked as destructive.
 
 ### security-lite — pass
 
-Summary: No security issues detected (lightweight scan).
+Summary: Found 2 security finding(s): 0 high, 0 medium, 2 low.
 
 - Endpoint: `security/scan-lite`
   - Advertised: `true`
   - Responded: `true`
   - Minimal shape present: `true`
-  - Item count: `0`
-  - Identifiers: none
-  - Diagnostics: none
+  - Item count: `2`
+  - Identifiers: toggle-simulated-logging, toggle-subscriber-updates
+  - Diagnostics: [low] Tool "toggle-simulated-logging" has an empty schema but is marked as destructive., [low] Tool "toggle-subscriber-updates" has an empty schema but is marked as destructive.
 
 ### tools — pass
 
@@ -138,15 +151,15 @@ Summary: Advertised capability responded with the minimal expected shape (13 ite
 
 ### schema-quality — partial
 
-Summary: Found 5 quality finding(s) across 24 item(s): 0 warnings, 5 info.
+Summary: Found 6 quality finding(s) across 24 item(s): 0 warnings, 6 info.
 
 - Endpoint: `schema-quality/scan`
   - Advertised: `true`
   - Responded: `true`
   - Minimal shape present: `true`
-  - Item count: `5`
-  - Identifiers: get-resource-links, get-resource-reference, gzip-file-as-resource, trigger-long-running-operation
-  - Diagnostics: [info] tool "get-resource-links": Has properties but no 'required' array declared, [info] tool "get-resource-reference": Has properties but no 'required' array declared, [info] tool "get-resource-reference": Property 'resourceType' missing description (+2 more)
+  - Item count: `6`
+  - Identifiers: get-resource-links, get-resource-reference, gzip-file-as-resource, trigger-long-running-operation, args-prompt
+  - Diagnostics: [info] tool "get-resource-links": Has properties but no 'required' array declared, [info] tool "get-resource-reference": Has properties but no 'required' array declared, [info] tool "get-resource-reference": Property 'resourceType' missing description (+3 more)
 
 ## Reproduction Commands
 
@@ -159,5 +172,5 @@ npm run cli -- report --run <path-to-run-artifact.json> --format markdown
 
 - Artifact type: `run`
 - Schema version: `1.0.0`
-- Run ID: `run_2026-06-24T020721281Z_c337e720`
+- Run ID: `run_2026-07-06T194944859Z_dd0288c3`
 - Gate: `pass`

package/docs/safety-index/artifacts/executeautomation-playwright-server.json

@@ -2,9 +2,9 @@
   "artifactType": "run",
   "schemaVersion": "1.0.0",
   "gate": "fail",
-  "runId": "run_2026-06-24T020744894Z_ce4e4a75",
-  "createdAt": "2026-06-24T02:07:44.894Z",
-  "toolVersion": "0.24.0",
+  "runId": "run_2026-07-06T194957955Z_4eaf8a90",
+  "createdAt": "2026-07-06T19:49:57.956Z",
+  "toolVersion": "0.28.0",
   "target": {
     "targetId": "executeautomation-playwright-server",
     "adapter": "local-process",
@@ -29,9 +29,9 @@
     "nodeVersion": "v22.22.1"
   },
   "summary": {
-    "total": 7,
+    "total": 8,
     "pass": 3,
-    "fail": 2,
+    "fail": 3,
     "partial": 1,
     "unsupported": 1,
     "flaky": 0,
@@ -43,7 +43,7 @@
       "id": "tools",
       "capability": "tools",
       "status": "pass",
-      "durationMs": 3.1655419999988226,
+      "durationMs": 1.1761660000011034,
       "message": "Advertised capability responded with the minimal expected shape (33 items).",
       "evidence": [
         {
@@ -746,7 +746,7 @@
       "id": "prompts",
       "capability": "prompts",
       "status": "unsupported",
-      "durationMs": 0.002541000001656357,
+      "durationMs": 0.0007080000013957033,
       "message": "Prompts are not advertised by the target.",
       "evidence": [
         {
@@ -762,7 +762,7 @@
       "id": "resources",
       "capability": "resources",
       "status": "pass",
-      "durationMs": 1.6509159999986878,
+      "durationMs": 0.6060830000005808,
       "message": "Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported.",
       "evidence": [
         {
@@ -791,7 +791,7 @@
       "id": "security-lite",
       "capability": "security-lite",
       "status": "fail",
-      "durationMs": 0.48133299999972223,
+      "durationMs": 0.05783300000075542,
       "message": "Found 1 security finding(s): 1 high, 0 medium, 0 low.",
       "evidence": [
         {
@@ -821,7 +821,7 @@
       "id": "conformance",
       "capability": "conformance",
       "status": "pass",
-      "durationMs": 7.248041000002559,
+      "durationMs": 2.294624999998632,
       "message": "All 7 conformance checks passed.",
       "evidence": [
         {
@@ -847,7 +847,7 @@
       "id": "schema-quality",
       "capability": "schema-quality",
       "status": "partial",
-      "durationMs": 1.8633340000014869,
+      "durationMs": 0.6172920000008162,
       "message": "Found 1 quality finding(s) across 34 item(s): 1 warnings, 0 info.",
       "evidence": [
         {
@@ -861,6 +861,14 @@
           ],
           "diagnostics": [
             "[warning] resource \"Browser console logs\": Missing description"
+          ],
+          "findings": [
+            {
+              "itemType": "resource",
+              "itemName": "Browser console logs",
+              "issue": "Missing description",
+              "severity": "warning"
+            }
           ]
         }
       ]
@@ -869,7 +877,7 @@
       "id": "security",
       "capability": "security",
       "status": "fail",
-      "durationMs": 0.8668330000000424,
+      "durationMs": 0.2935410000009142,
       "message": "Found 1 security finding(s): 1 high, 0 medium, 0 low.",
       "evidence": [
         {
@@ -894,6 +902,140 @@
           ]
         }
       ]
+    },
+    {
+      "id": "attack-sim",
+      "capability": "attack-sim",
+      "status": "fail",
+      "durationMs": 1.4405839999999444,
+      "message": "Safe attack simulation found 6 finding(s): 1 high, 5 medium, 0 low.",
+      "evidence": [
+        {
+          "endpoint": "attack-sim/safe",
+          "advertised": true,
+          "responded": true,
+          "minimalShapePresent": true,
+          "itemCount": 6,
+          "identifiers": [
+            "playwright_upload_file",
+            "playwright_evaluate",
+            "playwright_post",
+            "playwright_put",
+            "playwright_patch",
+            "playwright_delete"
+          ],
+          "diagnostics": [
+            "[medium] Tool \"playwright_upload_file\" combines broad parameters (filePath) with destructive or non-read-only behavior.",
+            "[high] Tool \"playwright_evaluate\" combines broad parameters (script) with destructive or non-read-only behavior.",
+            "[medium] Tool \"playwright_post\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+            "[medium] Tool \"playwright_put\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+            "[medium] Tool \"playwright_patch\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+            "[medium] Tool \"playwright_delete\" combines broad parameters (url, headers) with destructive or non-read-only behavior."
+          ],
+          "findings": [
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "medium",
+              "itemType": "tool",
+              "itemName": "playwright_upload_file",
+              "message": "Tool \"playwright_upload_file\" combines broad parameters (filePath) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "filePath"
+                ],
+                "description": "Upload a file to an input[type='file'] element on the page"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "gate"
+            },
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "high",
+              "itemType": "tool",
+              "itemName": "playwright_evaluate",
+              "message": "Tool \"playwright_evaluate\" combines broad parameters (script) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "script"
+                ],
+                "description": "Execute JavaScript in the browser console"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "quarantine"
+            },
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "medium",
+              "itemType": "tool",
+              "itemName": "playwright_post",
+              "message": "Tool \"playwright_post\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "url",
+                  "headers"
+                ],
+                "description": "Perform an HTTP POST request"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "gate"
+            },
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "medium",
+              "itemType": "tool",
+              "itemName": "playwright_put",
+              "message": "Tool \"playwright_put\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "url",
+                  "headers"
+                ],
+                "description": "Perform an HTTP PUT request"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "gate"
+            },
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "medium",
+              "itemType": "tool",
+              "itemName": "playwright_patch",
+              "message": "Tool \"playwright_patch\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "url",
+                  "headers"
+                ],
+                "description": "Perform an HTTP PATCH request"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "gate"
+            },
+            {
+              "ruleId": "attack-sim/permission-boundary/broad-destructive-tool",
+              "attackClass": "permission-boundary",
+              "severity": "medium",
+              "itemType": "tool",
+              "itemName": "playwright_delete",
+              "message": "Tool \"playwright_delete\" combines broad parameters (url, headers) with destructive or non-read-only behavior.",
+              "evidence": {
+                "broadParams": [
+                  "url",
+                  "headers"
+                ],
+                "description": "Perform an HTTP DELETE request"
+              },
+              "recommendation": "Constrain this tool with typed inputs, allowlists, explicit read-only/destructive annotations, and a harmless CI fixture.",
+              "recommendedAction": "gate"
+            }
+          ]
+        }
+      ]
     }
   ],
   "healthScore": {
@@ -922,7 +1064,8 @@
         "score": 0,
         "details": [
           "security-lite: fail (0/100)",
-          "security: fail (0/100)"
+          "security: fail (0/100)",
+          "attack-sim: fail (0/100)"
         ]
       },
       {
@@ -940,16 +1083,16 @@
         "weight": 0.1,
         "score": 100,
         "details": [
-          "Connect: 2061ms",
-          "p95 latency: 3ms (3 operations)"
+          "Connect: 908ms",
+          "p95 latency: 1ms (3 operations)"
         ]
       }
     ]
   },
   "performanceMetrics": {
-    "connectMs": 2060.7668749999975,
-    "toolsListMs": 3.1655419999988226,
-    "promptsListMs": 0.002541000001656357,
-    "resourcesListMs": 1.6509159999986878
+    "connectMs": 907.6346670000003,
+    "toolsListMs": 1.1761660000011034,
+    "promptsListMs": 0.0007080000013957033,
+    "resourcesListMs": 0.6060830000005808
   }
 }

package/docs/safety-index/artifacts/executeautomation-playwright-server.md

@@ -1,6 +1,6 @@
 # MCP Observatory Run Report
 
-Generated at 2026-06-24T02:07:44.894Z
+Generated at 2026-07-06T19:49:57.956Z
 
 ## Target and Environment Metadata
 
@@ -25,18 +25,18 @@ Generated at 2026-06-24T02:07:44.894Z
 
 | Gate | Total | Pass | Fail | Partial | Unsupported | Flaky | Skipped |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| fail | 7 | 3 | 2 | 1 | 1 | 0 | 0 |
+| fail | 8 | 3 | 3 | 1 | 1 | 0 | 0 |
 
 ## At a Glance
 
 - Safety verdict: **Blocked** — One or more checks can break agent dependence and should be fixed before production use.
-- Top risks: schema-quality: Found 1 quality finding(s) across 34 item(s): 1 warnings, 0 info.; security: Found 1 security finding(s): 1 high, 0 medium, 0 low.; security-lite: Found 1 security finding(s): 1 high, 0 medium, 0 low.
+- Top risks: schema-quality: Found 1 quality finding(s) across 34 item(s): 1 warnings, 0 info.; attack-sim: Safe attack simulation found 6 finding(s): 1 high, 5 medium, 0 low.; security: Found 1 security finding(s): 1 high, 0 medium, 0 low.
 - Regression/schema drift: Run `mcp-observatory diff <previous-run.json> <current-run.json>` to classify regressions and schema drift.
-- Failing checks: security-lite, security
+- Failing checks: security-lite, security, attack-sim
 - Partial or flaky checks: schema-quality
 - Skipped checks: none
 - Unsupported checks: prompts
-- Suggested next step: Start with the failing checks: security-lite, security.
+- Suggested next step: Start with the failing checks: security-lite, security, attack-sim.
 - CI next step: `Add CI: npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y <server-package>"`
 
 ## Regressions and Recoveries
@@ -47,13 +47,14 @@ _Use the `diff` command against another run artifact to classify regressions and
 
 | Focus | Check | Status | Duration (ms) | Message |
 | --- | --- | --- | --- | --- |
-| healthy | conformance | pass | 7.25 | All 7 conformance checks passed. |
-| healthy | resources | pass | 1.65 | Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported. |
-| healthy | tools | pass | 3.17 | Advertised capability responded with the minimal expected shape (33 items). |
-| review | schema-quality | partial | 1.86 | Found 1 quality finding(s) across 34 item(s): 1 warnings, 0 info. |
+| healthy | conformance | pass | 2.29 | All 7 conformance checks passed. |
+| healthy | resources | pass | 0.61 | Advertised capability responded with the minimal expected shape, but one optional resource endpoint appears unsupported. |
+| healthy | tools | pass | 1.18 | Advertised capability responded with the minimal expected shape (33 items). |
+| review | schema-quality | partial | 0.62 | Found 1 quality finding(s) across 34 item(s): 1 warnings, 0 info. |
 | confirm intent | prompts | unsupported | 0.00 | Prompts are not advertised by the target. |
-| act now | security | fail | 0.87 | Found 1 security finding(s): 1 high, 0 medium, 0 low. |
-| act now | security-lite | fail | 0.48 | Found 1 security finding(s): 1 high, 0 medium, 0 low. |
+| act now | attack-sim | fail | 1.44 | Safe attack simulation found 6 finding(s): 1 high, 5 medium, 0 low. |
+| act now | security | fail | 0.29 | Found 1 security finding(s): 1 high, 0 medium, 0 low. |
+| act now | security-lite | fail | 0.06 | Found 1 security finding(s): 1 high, 0 medium, 0 low. |
 
 ## Evidence Snippets
 
@@ -124,6 +125,18 @@ Summary: Prompts are not advertised by the target.
   - Identifiers: none
   - Diagnostics: none
 
+### attack-sim — fail
+
+Summary: Safe attack simulation found 6 finding(s): 1 high, 5 medium, 0 low.
+
+- Endpoint: `attack-sim/safe`
+  - Advertised: `true`
+  - Responded: `true`
+  - Minimal shape present: `true`
+  - Item count: `6`
+  - Identifiers: playwright_upload_file, playwright_evaluate, playwright_post, playwright_put, playwright_patch (+1 more)
+  - Diagnostics: [medium] Tool "playwright_upload_file" combines broad parameters (filePath) with destructive or non-read-only behavior., [high] Tool "playwright_evaluate" combines broad parameters (script) with destructive or non-read-only behavior., [medium] Tool "playwright_post" combines broad parameters (url, headers) with destructive or non-read-only behavior. (+3 more)
+
 ### security — fail
 
 Summary: Found 1 security finding(s): 1 high, 0 medium, 0 low.
@@ -159,5 +172,5 @@ npm run cli -- report --run <path-to-run-artifact.json> --format markdown
 
 - Artifact type: `run`
 - Schema version: `1.0.0`
-- Run ID: `run_2026-06-24T020744894Z_ce4e4a75`
+- Run ID: `run_2026-07-06T194957955Z_4eaf8a90`
 - Gate: `fail`