mcprigor 1.4.0 → 1.5.0

package/docs/PUBLISHING.md

@@ -0,0 +1,53 @@
+# Shareable hosted reports
+
+> Available since `1.5.0`.
+
+Turn a test run into a URL anyone can open — no repository access, no CI login:
+
+```bash
+export NETLIFY_AUTH_TOKEN=...   # personal or CI token
+mcprigor publish tests/catalog.mcpr --site your-netlify-site
+```
+
+`publish` runs the suite, builds the readable HTML report **with the clickable request/response session timeline**, deploys it to your Netlify site with the dependency-free digest API, waits until the deploy is live, and prints the shareable URL:
+
+```
+Published report: https://68b1c2--your-site.netlify.app
+```
+
+Each publish is a normal Netlify deploy of your own site, so access control, custom domains, deploy previews, and retention follow your existing hosting configuration. Unchanged files are skipped automatically via content digests.
+
+## Options
+
+```bash
+mcprigor publish suite.mcpr --site SITE [--include-json] [--test NAME] [--env qa]
+mcprigor publish suite.mcpr --out reports/latest
+```
+
+- `--site` — Netlify site ID or name. The token comes only from `NETLIFY_AUTH_TOKEN` (or `MCPRIGOR_PUBLISH_TOKEN`); tokens are never accepted as command-line flags.
+- `--out DIR` — write the same bundle to a local directory instead of (or in addition to) hosting it. Serve it from any static host: S3, GitHub Pages, nginx, an artifact store.
+- `--include-json` — also publish `result.json` for dashboards and programmatic consumers.
+- `--test`, `--env`, `--command`, `--url` — the same run-selection options as `mcprigor test`.
+
+The exit code still reflects the run (`0` passed, `1` failed), so `publish` can replace `test` in a pipeline step that both gates and shares.
+
+## Publishing from the QA workspace UI
+
+The visual workspace (`mcprigor workspace`) exposes the same features without the command line:
+
+- **HTML report** — every finished test run has an `HTML report` button that opens the full report, including the clickable session timeline, in a new tab.
+- **Publish** — start the workspace with hosting configured and a `Publish` button appears next to the export buttons:
+
+  ```bash
+  export MCPRIGOR_PUBLISH_SITE=your-netlify-site
+  export NETLIFY_AUTH_TOKEN=...
+  mcprigor workspace
+  ```
+
+  Clicking it deploys the selected run's report and opens the shareable URL; a `View published ↗` link stays on the run for re-opening or copying. Without both variables the button is hidden and the API answers with a clear configuration message — the token itself never reaches the browser.
+
+## Security
+
+- The report is produced by the same pipeline as `--html`: secrets and configured redaction patterns are removed **before** the report exists.
+- Publishing makes the report as public as the target site. Use a password-protected or team-restricted Netlify site for internal results.
+- The hosting token is read from the environment at the last moment and is never echoed, logged, or stored.

package/docs/QA-WORKSPACE.md

@@ -30,7 +30,7 @@ An empty folder shows a three-step welcome screen. Choose **＋ New test file**
 2. Edit the plain-language scenario. The editor provides syntax highlighting, line numbers, and autocomplete: top-level declarations at the start of a line, actions and assertions when indented, and comparison phrases after `Expect "field"`. Accept with `Tab` or `Enter`; force the list open with `Ctrl+Space`.
 3. Choose **Validate** (`Ctrl/⌘+S` saves, `Ctrl/⌘+Enter` runs). A wording problem highlights the offending line and moves the cursor to it.
 4. Choose **▶ Run tests** or **Parity**.
-5. Review the results panel: per-file pass/fail with durations; select a file for its full report.
+5. Review the results panel: per-file pass/fail with durations; select a file for its full report. Drag either panel divider to resize the file list, editor, and results areas; widths persist across reloads, arrow keys resize a focused divider, and double-click resets the layout.
 
 The editor marks unsaved changes; running or validating saves them first. If the file changed elsewhere after you opened it, the workspace refuses to overwrite it and asks you to reload.
 
@@ -52,6 +52,10 @@ Test runs are recorded in `.mcprigor/workspace-history.jsonl` (most recent 2000
 
 One search box filters all three tabs. It matches suite names, test names, and error text, and highlights matches, so you can answer questions like "when did `delivered` start failing?" without leaving the browser.
 
+Completed runs expose **PDF**, **CSV**, and **JUnit XML** downloads. The Trends tab exports a rich trends PDF, aggregate CSV, or raw history CSV. PDF reports include summary cards, pass-rate visuals, per-test detail, failures, durations, and evidence identifiers.
+
+An **HTML report** button opens the full report — including the clickable request/response session timeline — in a new tab. When the workspace is started with `MCPRIGOR_PUBLISH_SITE` and `NETLIFY_AUTH_TOKEN` set, a **Publish** button also appears: one click hosts the report at a shareable static URL and keeps a `View published ↗` link on the run. See [Shareable hosted reports](PUBLISHING.md).
+
 ## What is available
 
 - Creating, renaming, and editing `.mcpr` suites (YAML and JSON suites are listed and editable too)

package/docs/README.md

@@ -18,6 +18,12 @@ Choose the path that matches your role.
 - [State and dependencies](STATE-AND-DEPENDENCIES.md) — share outputs across tests and runs.
 - [Data engineering](DATA-ENGINEERING.md) — types, filters, joins, samples, and caches.
 - [Transport parity](TRANSPORT-PARITY.md) — compare stdio and Streamable HTTP.
+- [Performance governance](PERFORMANCE-GOVERNANCE.md) — call limits, percentile budgets, and regression gates.
+- [Multi-server compositions](MULTI-SERVER-COMPOSITIONS.md) — route tests across a fleet and gate combined drift.
+- [Coverage](COVERAGE.md) — find untested MCP surfaces and input-schema branches.
+- [Scheduled monitoring](MONITORING.md) — continuously test production HTTP endpoints and notify webhooks.
+- [GitHub Action](GITHUB-ACTION.md) — test, drift, flaky warnings, and rich pull-request comments.
+- [Shareable hosted reports](PUBLISHING.md) — `mcprigor publish` turns a run into a static report URL.
 
 ## Contracts and evidence
 
@@ -30,6 +36,7 @@ Choose the path that matches your role.
 
 - [Extension SDK](EXTENSION-SDK.md)
 - [Stable error model](ERROR-MODEL.md)
+- [Deterministic security audit](SECURITY-AUDIT.md) — built-in probe pack, severity gate, and rich reports.
 - [Security and retention](SECURITY-AND-RETENTION.md)
 - [Compatibility policy](COMPATIBILITY.md)
 

package/docs/SECURITY-AUDIT.md

@@ -0,0 +1,77 @@
+# Deterministic security audit
+
+> Available since `1.5.0`.
+
+`mcprigor audit` runs a fixed security and permissions probe pack against an MCP server and produces an auditable score. No AI chooses payloads or interprets outcomes.
+
+## Run the safe default pack
+
+Use a suite's configured target:
+
+```bash
+mcprigor audit tests/server.mcpr
+```
+
+Or supply a target directly:
+
+```bash
+mcprigor audit --url https://qa.example.com/mcp
+mcprigor audit --command "node dist/server.js"
+```
+
+The non-destructive default probes:
+
+| Probe | What MCP Rigor verifies |
+|---|---|
+| Malformed parameters | Invalid `tools/call` parameters are rejected |
+| Tool-name spoofing | An unknown administrative-looking tool cannot be invoked |
+| Oversized payload | A deterministic 1 MiB payload is rejected |
+| Path traversal | Resource reads reject Unix, Windows, and relative traversal URIs |
+| Tool inventory | Advertised tools are listed for opt-in injection/canary testing |
+
+## Opt into reviewed tool calls
+
+MCP Rigor never guesses that a tool is safe. Tool execution is disabled unless you allow an exact name:
+
+```bash
+mcprigor audit tests/server.mcpr \
+  --allow-tool search \
+  --allow-tool summarize
+```
+
+Only allow read-only, non-destructive tools in a disposable test environment. An allowed tool receives a fixed prompt-injection payload containing a deterministic canary. MCP Rigor checks whether the response follows or reflects the payload and whether it exposes the canary. Report evidence replaces the canary with `[CANARY REDACTED]`.
+
+## CI severity gate
+
+```bash
+mcprigor audit tests/server.mcpr --fail-on high
+```
+
+Values are `critical`, `high`, `medium`, `low`, and `none`; the default is `high`. The command exits nonzero when a failed finding meets or exceeds the selected severity.
+
+Scoring starts at 100:
+
+- critical: −35
+- high: −20
+- medium: −10
+- low: −4
+- skipped: no deduction
+
+Skipped probes remain visible with the exact `--allow-tool` needed. They are not represented as passes.
+
+## Reports
+
+```bash
+mcprigor audit tests/server.mcpr \
+  --pdf reports/security.pdf \
+  --json reports/security.json \
+  --csv reports/security.csv
+```
+
+The rich PDF includes score, grade, severity cards, per-finding status, explanations, and bounded evidence. Markdown output is available with `--markdown`.
+
+## Interpreting results
+
+The audit is deterministic black-box evidence, not a certification or a replacement for threat modeling. A passing probe proves the observed server rejected that exact payload under that exact configuration. Review skipped probes, environment permissions, downstream model behavior, and business authorization separately.
+
+Run intrusive tool probes only in an isolated environment. Do not expose production mutation tools through `--allow-tool`.

package/editors/mcpr-language.json

@@ -12,11 +12,11 @@
       "name": "keyword.control.version.mcpr"
     },
     {
-      "match": "^\\s*(Suite|Server|Server options|MCP URL|Compare target|Target options for|Default timeout|Redact|Snapshots|Ignore snapshot paths|Client behavior|Import flows from|Flow|Inputs|Outputs|Test|Scenario|Id|Depends on|Require|Require protocol|Variables|Skip|Setup|Steps|Cleanup|Data source):?",
+      "match": "^\\s*(Suite|Server|Named server|Server options|Server options for|MCP URL|Compare target|Target options for|Default timeout|Budget|Redact|Snapshots|Ignore snapshot paths|Client behavior|Import flows from|Flow|Inputs|Outputs|Test|Scenario|Id|Depends on|Require|Require protocol|Variables|Skip|Setup|Steps|Cleanup|Data source):?",
       "name": "keyword.control.mcpr"
     },
     {
-      "match": "^\\s*(Call tool|When the server asks for input|When the server requests sampling|Read resource|Get prompt|Send|Expect|Save|Export|Set|Use flow|For each row|Wait up to|Wait for notification|Subscribe to resource|Unsubscribe from resource|Set log level|List all|Get task|List tasks|Cancel task|From CSV|From JSON|From YAML|From Excel|From REST|Column|Derive|Keep rows where|Sample|Cache this source)\\b",
+      "match": "^\\s*(Call tool|On server|When the server asks for input|When the server requests sampling|Read resource|Get prompt|Send|Expect|Save|Export|Set|Use flow|For each row|Wait up to|Wait for notification|Subscribe to resource|Unsubscribe from resource|Set log level|List all|Get task|List tasks|Cancel task|From CSV|From JSON|From YAML|From Excel|From REST|Column|Derive|Keep rows where|Sample|Cache this source)\\b",
       "name": "keyword.other.mcpr"
     },
     {

package/editors/vscode/package.json

@@ -2,7 +2,7 @@
   "name": "mcprigor",
   "displayName": "MCP Rigor",
   "description": "Syntax highlighting and inline validation for MCP Rigor (.mcpr) natural-language MCP test suites.",
-  "version": "1.3.0",
+  "version": "1.4.0",
   "publisher": "fusionone",
   "license": "Apache-2.0",
   "repository": { "type": "git", "url": "https://github.com/FusionOnePlatform/mcprigor" },

package/editors/vscode/syntaxes/mcpr.tmLanguage.json

@@ -12,11 +12,11 @@
       "name": "keyword.control.version.mcpr"
     },
     {
-      "match": "^\\s*(Suite|Server|Server options|MCP URL|Compare target|Target options for|Default timeout|Redact|Snapshots|Ignore snapshot paths|Client behavior|Import flows from|Flow|Inputs|Outputs|Test|Scenario|Id|Depends on|Require|Require protocol|Variables|Skip|Setup|Steps|Cleanup|Data source):?",
+      "match": "^\\s*(Suite|Server|Named server|Server options|Server options for|MCP URL|Compare target|Target options for|Default timeout|Budget|Redact|Snapshots|Ignore snapshot paths|Client behavior|Import flows from|Flow|Inputs|Outputs|Test|Scenario|Id|Depends on|Require|Require protocol|Variables|Skip|Setup|Steps|Cleanup|Data source):?",
       "name": "keyword.control.mcpr"
     },
     {
-      "match": "^\\s*(Call tool|When the server asks for input|When the server requests sampling|Read resource|Get prompt|Send|Expect|Save|Export|Set|Use flow|For each row|Wait up to|Wait for notification|Subscribe to resource|Unsubscribe from resource|Set log level|List all|Get task|List tasks|Cancel task|From CSV|From JSON|From YAML|From Excel|From REST|Column|Derive|Keep rows where|Sample|Cache this source)\\b",
+      "match": "^\\s*(Call tool|On server|When the server asks for input|When the server requests sampling|Read resource|Get prompt|Send|Expect|Save|Export|Set|Use flow|For each row|Wait up to|Wait for notification|Subscribe to resource|Unsubscribe from resource|Set log level|List all|Get task|List tasks|Cancel task|From CSV|From JSON|From YAML|From Excel|From REST|Column|Derive|Keep rows where|Sample|Cache this source)\\b",
       "name": "keyword.other.mcpr"
     },
     {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcprigor",
-  "version": "1.4.0",
+  "version": "1.5.0",
   "description": "Plain-language MCP testing with isolated extensions and a local QA workspace",
   "type": "module",
   "bin": {
@@ -47,7 +47,6 @@
   "dependencies": {
     "@modelcontextprotocol/sdk": "1.27.1",
     "ajv": "^8.18.0",
-    "mcprigor": "^1.3.1",
     "yaml": "^2.8.2"
   },
   "overrides": {

package/workspace-assets/app.js

@@ -1,4 +1,4 @@
-const state = { csrf: '', path: '', etag: '', dirty: false, files: [], selected: new Set(), run: null, history: [], tab: 'run', runSel: -1 };
+const state = { csrf: '', path: '', etag: '', dirty: false, files: [], selected: new Set(), run: null, history: [], tab: 'run', runSel: -1, canPublish: false, publishing: false };
 const $ = id => document.getElementById(id);
 
 /* ---------- API ---------- */
@@ -14,6 +14,7 @@ async function api(path, options = {}) {
 async function start() {
   const boot = await api('/api/v1/bootstrap');
   state.csrf = boot.csrf;
+  state.canPublish = (boot.capabilities || []).includes('publish');
   $('version').textContent = `v${boot.version}`;
   $('workspace').textContent = `Folder: ${boot.root}`;
   $('connection').classList.add('ok');
@@ -613,16 +614,46 @@ start().catch(show);
 function updateExportBar() {
   const run = state.run;
   const bar = $('export-run');
-  const index = state.runSel || 0;
+  const index = state.runSel >= 0 ? state.runSel : 0;
   const item = run && run.items && run.items[index];
-  bar.hidden = !(run && run.mode === 'test' && item && item.status !== 'running' && item.tests);
+  const ready = run && run.mode === 'test' && item && item.status !== 'running' && item.tests;
+  bar.hidden = !ready;
+  const publish = $('publish-run');
+  const link = $('published-link');
+  publish.hidden = !(ready && state.canPublish);
+  publish.disabled = state.publishing;
+  publish.textContent = state.publishing ? 'Publishing…' : (item && item.publishedUrl ? 'Publish again' : 'Publish');
+  link.hidden = !(ready && item.publishedUrl);
+  if (item && item.publishedUrl) link.href = item.publishedUrl;
 }
 function exportRun(format) {
   const run = state.run;
   if (!run) return;
-  const index = state.runSel || 0;
+  const index = state.runSel >= 0 ? state.runSel : 0;
   window.open(`/api/v1/export/run?id=${encodeURIComponent(run.id)}&item=${index}&format=${format}`, '_blank');
 }
+async function publishRun() {
+  const run = state.run;
+  if (!run || state.publishing) return;
+  const index = state.runSel >= 0 ? state.runSel : 0;
+  state.publishing = true; updateExportBar();
+  try {
+    const value = await api('/api/v1/publish', { method: 'POST', body: JSON.stringify({ id: run.id, item: index }) });
+    const item = run.items[index];
+    if (item) item.publishedUrl = value.url;
+    window.open(value.url, '_blank', 'noopener');
+  } catch (error) {
+    toast(error.message || 'Publish failed', true);
+  } finally {
+    state.publishing = false; updateExportBar();
+  }
+}
+$('export-run-html').onclick = () => {
+  const run = state.run;
+  if (!run) return;
+  window.open(`/api/v1/report?id=${encodeURIComponent(run.id)}&item=${state.runSel >= 0 ? state.runSel : 0}`, '_blank');
+};
+$('publish-run').onclick = publishRun;
 $('export-run-pdf').onclick = () => exportRun('pdf');
 $('export-run-csv').onclick = () => exportRun('csv');
 $('export-run-junit').onclick = () => exportRun('junit');

package/workspace-assets/index.html

@@ -95,6 +95,9 @@
         <button id="export-run-pdf" class="ghost small" type="button">PDF</button>
         <button id="export-run-csv" class="ghost small" type="button">CSV</button>
         <button id="export-run-junit" class="ghost small" type="button">JUnit XML</button>
+        <button id="export-run-html" class="ghost small" type="button">HTML report</button>
+        <button id="publish-run" class="ghost small" type="button" hidden>Publish</button>
+        <a id="published-link" class="published-link" target="_blank" rel="noopener" hidden>View published ↗</a>
       </div>
       <div id="run-list"></div>
       <pre id="output" class="output"><span class="muted">Nothing has run yet.

package/workspace-assets/style.css

@@ -232,3 +232,5 @@ nav .suite-row button .rn:hover{opacity:1}
 .export-bar{display:flex;align-items:center;gap:6px;padding:8px 14px;border-bottom:1px solid var(--line-soft)}
 .export-label{font-size:11px;color:var(--text-dim);margin-right:2px}
 .export-bar .small{font-size:11px;padding:3px 10px}
+.published-link{font-size:11px;margin-left:4px;color:var(--accent, #2563eb);text-decoration:none}
+.published-link:hover{text-decoration:underline}