mcprigor 1.0.0-rc.1

package/docs/MCP-NATIVE.md

@@ -0,0 +1,111 @@
+# MCP-Native Behavior Testing
+
+MCP Rigor 0.10 extends deterministic request/response testing to asynchronous and client-side MCP behavior.
+
+## Notifications
+
+```text
+Wait for notification "notifications/resources/updated" within 5 seconds
+Expect "params.uri" equals "fixture://status"
+```
+
+Supported official notification handlers include:
+
+- progress
+- resource updated and resource list changed
+- tool and prompt list changed
+- logging messages
+- task status
+
+Events receive deterministic session-local sequence numbers and are available to the evidence recorder.
+
+## Resource subscriptions
+
+```text
+Subscribe to resource "fixture://status"
+Call tool "change_status"
+Wait for notification "notifications/resources/updated" within 5 seconds
+
+Cleanup:
+Unsubscribe from resource "fixture://status"
+```
+
+The server must declare resource subscription capability.
+
+## Progress and cancellation
+
+```text
+Call tool "import_catalog" with progress with:
+  file: "catalog.csv"
+
+Expect "progress" has 2 items
+```
+
+Cancellation can be configured in YAML/JSON native steps with `cancelAfterMs`. Cancellation uses `AbortSignal`, distinguishing it from request timeout.
+
+## Pagination
+
+```text
+List all tools
+Expect "items" has 20 items
+```
+
+Equivalent statements exist for resources, prompts, and resource templates. MCP Rigor follows `nextCursor` until absent and fails on repeated cursors.
+
+## Logging
+
+```text
+Set log level to "debug"
+Wait for notification "notifications/message" within 5 seconds
+```
+
+## Roots, sampling, and elicitation
+
+YAML/JSON suites can configure deterministic client behavior:
+
+```yaml
+client:
+  roots:
+    - uri: file:///workspace
+      name: Workspace
+  sampling:
+    model: fixture-model
+    text: deterministic response
+  elicitation:
+    action: accept
+    content:
+      approved: true
+```
+
+The SDK client advertises roots, sampling, and elicitation capabilities and installs handlers using official schemas. No real model is called and no human UI is invoked during deterministic runs.
+
+## Experimental tasks
+
+Native requests can stream task-aware tool calls through the SDK experimental task API. Generic task operations are also available:
+
+- `tasks/get`
+- `tasks/list`
+- `tasks/cancel`
+
+Task support remains explicitly experimental and SDK-version-sensitive. MCP Rigor records intermediate task events but does not claim to validate persistence or recovery across separate server processes.
+
+## Boundary of coverage
+
+Fully testable at the SDK boundary:
+
+- notification registration and delivery
+- progress callbacks
+- AbortSignal cancellation
+- subscriptions
+- logging level and messages
+- roots requests
+- deterministic sampling responses
+- form elicitation responses
+- cursor pagination
+
+Not claimed:
+
+- actual LLM response quality
+- human elicitation UI behavior
+- task durability across infrastructure failures
+- raw transport framing or reconnect behavior

package/docs/PLAIN-LANGUAGE-COOKBOOK.md

@@ -0,0 +1,219 @@
+# Plain-language cookbook
+
+Copy a pattern, replace the names and values, then run `mcprigor check FILE`.
+
+## Basic file
+
+```text
+MCP Test 1
+Suite: "Customer service"
+Server: node dist/server.js
+
+Test: "The server responds"
+  Send "ping"
+  Expect it succeeds
+```
+
+## Call a tool
+
+```text
+Test: "An active customer can be found"
+  Call tool "find_customer" with:
+    customerId: "C-100"
+
+  Expect "structuredContent.customerId" equals "C-100"
+  Expect "structuredContent.status" equals "active"
+```
+
+## Read a resource
+
+```text
+Test: "The catalog is ready"
+  Read resource "catalog://status"
+  Expect "contents[0].text" contains "ready"
+```
+
+## Get a prompt
+
+```text
+Test: "A review prompt can be created"
+  Get prompt "review_release" with:
+    release: "1.2.0"
+
+  Expect "messages" has 1 item
+```
+
+## Check result types and patterns
+
+```text
+Expect "structuredContent.total" is a number
+Expect "structuredContent.customerId" matches "^C-[0-9]+$"
+Expect "structuredContent.status" does not equal "deleted"
+Expect "structuredContent.items" exists
+Expect "structuredContent.items" has 3 items
+```
+
+## Expect an MCP error
+
+```text
+Test: "A missing customer returns an error"
+  Call tool "find_customer" with:
+    customerId: "DOES-NOT-EXIST"
+
+  Expect an error
+  Expect error code -32602
+  Expect error message matches "not found"
+```
+
+## Save and reuse a value
+
+```text
+Test: "Create and retrieve a customer"
+  Call tool "create_customer" with:
+    email: "qa@example.com"
+
+  Save "structuredContent.customerId" as "customerId"
+
+  Call tool "find_customer" with:
+    customerId: "${customerId}"
+
+  Expect "structuredContent.customerId" equals "${customerId}"
+```
+
+## Always clean up
+
+```text
+Test: "A temporary customer can be used"
+  Setup:
+  Call tool "create_customer" with:
+    email: "temporary@example.com"
+  Save "structuredContent.customerId" as "customerId"
+
+  Steps:
+  Call tool "find_customer" with:
+    customerId: "${customerId}"
+  Expect it succeeds
+
+  Cleanup:
+  Call tool "delete_customer" with:
+    customerId: "${customerId}"
+```
+
+Cleanup runs even if an earlier test action fails. Make cleanup actions safe to repeat.
+
+## Reuse a flow
+
+```text
+Flow: "Verify addition"
+  Inputs: a, b, expected
+
+  Call tool "add" with:
+    a: "${a}"
+    b: "${b}"
+
+  Expect "structuredContent.sum" equals "${expected}"
+
+Test: "Common addition cases"
+  Use flow "Verify addition" with:
+    a: 20
+    b: 22
+    expected: 42
+```
+
+## Run a table of examples
+
+```text
+Test: "Calculator examples"
+  For each row:
+    | caseId | a  | b  | expected |
+    | zero   | 0  | 0  | 0        |
+    | basic  | 2  | 3  | 5        |
+    | larger | 20 | 22 | 42       |
+
+  Call tool "add" with:
+    a: "${row.a}"
+    b: "${row.b}"
+
+  Expect "structuredContent.sum" equals "${row.expected}"
+```
+
+## Use CSV data
+
+```text
+Data source: "customers"
+  From CSV "data/customers.csv"
+  Column "customerId" is string required
+  Column "active" is boolean
+  Keep rows where "active" equals true
+  Sample 25 rows with seed 2025
+
+Test: "Active customers are available"
+  For each row from "customers"
+  Call tool "find_customer" with:
+    customerId: "${row.customerId}"
+  Expect it succeeds
+```
+
+## Share an output with another test
+
+```text
+Test: "Create a customer"
+  Id: create-customer
+  Call tool "create_customer" with:
+    email: "qa@example.com"
+  Export "structuredContent.customerId" as "customerId"
+
+Test: "Retrieve the created customer"
+  Id: retrieve-customer
+  Depends on: create-customer
+  Call tool "find_customer" with:
+    customerId: "${deps.create-customer.customerId}"
+  Expect it succeeds
+```
+
+## Compare two transports
+
+```text
+Compare target "Local": node dist/server.js
+Compare target "QA": https://qa.example.com/mcp
+
+Test: "Search behaves the same"
+  Call tool "search" with:
+    query: "red shoes"
+  Expect "structuredContent.total" equals 2
+```
+
+Run with `mcprigor parity FILE`.
+
+## Match a snapshot
+
+```text
+Expect "structuredContent" matches snapshot "customer" ignoring "$.generatedAt"
+```
+
+Create or approve the snapshot explicitly:
+
+```bash
+mcprigor test customer.mcpr --snapshot customer.snap.json --update-snapshots
+```
+
+## Notifications and progress
+
+```text
+Subscribe to resource "catalog://updates"
+Wait for notification "notifications/resources/updated" within 5 seconds
+Unsubscribe from resource "catalog://updates"
+```
+
+```text
+Call tool "import_catalog" with progress and cancel after 500 ms with:
+  file: "catalog.csv"
+```
+
+## Validate before running
+
+```bash
+mcprigor check tests/acceptance.mcpr
+```
+
+For every supported statement and rule, see the [language reference](LANGUAGE-SPEC.md).

package/docs/QA-GUIDE.md

@@ -0,0 +1,128 @@
+# QA guide
+
+Use this page as a short everyday checklist. For examples, open the [plain-language cookbook](PLAIN-LANGUAGE-COOKBOOK.md).
+
+## Everyday workflow
+
+```bash
+mcprigor check tests/acceptance.mcpr
+mcprigor test tests/acceptance.mcpr
+mcprigor test tests/acceptance.mcpr --html report.html
+```
+
+Or use the browser:
+
+```bash
+mcprigor workspace .
+```
+
+## Test structure
+
+```text
+MCP Test 1
+Suite: "Customer service"
+Server: node dist/server.js
+
+Test: "An active customer can be found"
+  Call tool "find_customer" with:
+    customerId: "C-100"
+
+  Expect "structuredContent.status" equals "active"
+```
+
+A file needs:
+
+1. A suite name
+2. A `Server` command or `MCP URL`
+3. One or more named tests
+4. At least one action in each test
+
+## Actions
+
+```text
+Call tool "name" with:
+  input: value
+
+Read resource "scheme://resource"
+
+Get prompt "name" with:
+  argument: value
+
+Send "ping"
+```
+
+## Expectations
+
+```text
+Expect it succeeds
+Expect an error
+Expect "field" equals value
+Expect "field" does not equal value
+Expect "field" contains value
+Expect "field" exists
+Expect "items" has 3 items
+Expect "field" is a string
+Expect "field" matches "pattern"
+```
+
+Arrays use indexes such as `items[0].name`.
+
+## Variables
+
+```text
+Save "structuredContent.id" as "createdId"
+```
+
+Use it later:
+
+```text
+Call tool "get_item" with:
+  id: "${createdId}"
+```
+
+## Setup and cleanup
+
+```text
+Setup:
+Call tool "create_fixture"
+
+Steps:
+Call tool "verify_fixture"
+
+Cleanup:
+Call tool "delete_fixture"
+```
+
+Cleanup is attempted even after a failed step. Keep it safe to repeat.
+
+## Data tables
+
+```text
+For each row:
+  | caseId | input | expected |
+  | first  | 2     | 4        |
+  | second | 3     | 6        |
+```
+
+Use values as `${row.input}` and `${row.expected}`.
+
+## Good QA practices
+
+- Assert stable business fields, not entire responses.
+- Use `exists` for generated IDs.
+- Avoid exact timestamp and token assertions.
+- Give tests behavior-focused names.
+- Keep each test independently repeatable.
+- Add cleanup for created data.
+- Validate with `check` before running.
+- Review snapshots and contract updates in pull requests.
+- Keep credentials in environment variables.
+
+## Where to go next
+
+- [Getting started](GETTING-STARTED.md)
+- [Plain-language cookbook](PLAIN-LANGUAGE-COOKBOOK.md)
+- [Guided authoring](GUIDED-AUTHORING.md)
+- [Data and reusable flows](DATA-AND-REUSE.md)
+- [Troubleshooting](TROUBLESHOOTING.md)
+- [Complete language reference](LANGUAGE-SPEC.md)

package/docs/QA-WORKSPACE.md

@@ -0,0 +1,78 @@
+# QA workspace
+
+Use MCP Rigor in a browser to edit, validate, and run saved tests.
+
+## Start it
+
+From your test directory:
+
+```bash
+mcprigor workspace .
+```
+
+Choose a port if needed:
+
+```bash
+mcprigor workspace . --port 4173
+```
+
+Open the printed local URL, for example `http://127.0.0.1:4173`.
+
+## Daily workflow
+
+1. Select a `.mcpr` suite in the left panel.
+2. Edit the plain-language scenario.
+3. Choose **Validate**.
+4. Fix any diagnostic shown below the editor.
+5. Choose **Run tests** or **Run parity**.
+6. Review the result panel.
+7. Save the file.
+
+The editor marks unsaved changes. If the file changed elsewhere after you opened it, the workspace refuses to overwrite it and asks you to reload.
+
+## What is available
+
+- Saved `.mcpr`, YAML, and JSON suites
+- Plain-language editing
+- Validation without server execution
+- Test execution
+- Transport parity execution
+- Terminal-style results
+- Local evidence indexing
+
+## Current candidate limits
+
+The browser currently focuses on the core edit/validate/run/parity loop. Use the CLI for:
+
+- guided test generation;
+- contract discovery and drift updates;
+- detailed evidence comparison;
+- snapshot acceptance;
+- replay;
+- cancellation and streaming run progress.
+
+See the [CLI reference](CLI-REFERENCE.md).
+
+## Security
+
+The workspace:
+
+- listens only on the local machine;
+- requires same-origin, CSRF-protected changes;
+- reads targets from saved suites;
+- does not accept arbitrary commands from browser requests;
+- restricts file paths and types;
+- uses atomic saves and content fingerprints;
+- limits files and requests to 1 MiB.
+
+Do not expose the workspace through a public proxy. Stop it with `Ctrl+C` when finished.
+
+## Problems
+
+If a suite does not appear, confirm that:
+
+- it is under the selected workspace directory;
+- its extension is `.mcpr`, `.yaml`, `.yml`, or `.json`;
+- it is not inside `node_modules`, `.git`, or `dist`.
+
+For server and test failures, see [troubleshooting](TROUBLESHOOTING.md).

package/docs/README.md

@@ -0,0 +1,55 @@
+# MCP Rigor documentation
+
+Choose the path that matches your role.
+
+## QA authors
+
+1. [Getting started](GETTING-STARTED.md) — install, connect, and run your first test.
+2. [Plain-language cookbook](PLAIN-LANGUAGE-COOKBOOK.md) — copy-ready scenarios and assertions.
+3. [QA workspace](QA-WORKSPACE.md) — edit and run tests in a browser.
+4. [Data and reusable flows](DATA-AND-REUSE.md) — tables, files, functions, and shared flows.
+5. [Troubleshooting](TROUBLESHOOTING.md) — understand errors and fix common failures.
+
+## Test and platform engineers
+
+- [Engineer setup](ENGINEER-SETUP.md) — targets, credentials, project layout, and CI.
+- [CLI reference](CLI-REFERENCE.md) — commands, options, outputs, and exit codes.
+- [Language reference](LANGUAGE-SPEC.md) — complete deterministic `.mcpr` syntax.
+- [File extension and migration](FILE-EXTENSION.md) — why MCP Rigor uses `.mcpr`.
+- [State and dependencies](STATE-AND-DEPENDENCIES.md) — share outputs across tests and runs.
+- [Data engineering](DATA-ENGINEERING.md) — types, filters, joins, samples, and caches.
+- [Transport parity](TRANSPORT-PARITY.md) — compare stdio and Streamable HTTP.
+
+## Contracts and evidence
+
+- [Contract drift](CONTRACT-DRIFT.md)
+- [Protocol evidence](EVIDENCE.md)
+- [Snapshots and replay](SNAPSHOTS-AND-REPLAY.md)
+- [MCP-native behavior](MCP-NATIVE.md)
+
+## Extensions and operations
+
+- [Extension SDK](EXTENSION-SDK.md)
+- [Stable error model](ERROR-MODEL.md)
+- [Security and retention](SECURITY-AND-RETENTION.md)
+- [Compatibility policy](COMPATIBILITY.md)
+
+## Project background
+
+These documents explain product design and research rather than everyday usage:
+
+- [Specification](SPECIFICATION.md)
+- [Competitive landscape](LANDSCAPE.md)
+
+## Fastest adoption path
+
+```text
+install → create .mcpr test → check → test → add CI → enable evidence
+```
+
+```bash
+npm install --save-dev mcprigor
+npx mcprigor init tests/acceptance.mcpr
+npx mcprigor check tests/acceptance.mcpr
+npx mcprigor test tests/acceptance.mcpr --html report.html
+```

package/docs/SECURITY-AND-RETENTION.md

@@ -0,0 +1,37 @@
+# Security and evidence retention
+
+> Production required: review this page before enabling remote data, custom extensions, or persistent CI evidence.
+
+## Implemented controls
+
+- URI userinfo and sensitive query/fragment fields are redacted.
+- Known secrets are redacted in raw, percent-encoded, double-encoded, base64, and base64url forms.
+- Terminal reports remove ANSI/CSI/OSC/DCS/APC/PM and unsafe C0/C1 controls.
+- Remote data requires explicit opt-in, credential-free HTTP(S), public DNS/IP destinations, at most three manually checked redirects, ten-second deadlines, and 10 MiB bodies.
+- Remote redirects are revalidated and private/link-local/loopback destinations are rejected.
+- YAML data uses the core schema, duplicate-key detection, alias limits, depth/node limits, and unsafe-key rejection.
+- CSV input caps fields at 1 MiB, rows at 1,000 columns, and total rows at one million; source files remain capped at 10 MiB.
+- XLSX input is capped at 25 MiB compressed and signature-checked before ExcelJS parsing.
+- Extension paths can be restricted with exact `extensions.allowlist` entries.
+- Server stderr is capped per chunk and per session.
+
+## Residual risks
+
+DNS validation followed by the platform `fetch` still has a rebinding window because the current implementation does not yet pin the resolved address through a custom dispatcher. ExcelJS preflight does not fully inspect ZIP central-directory expansion ratios. Use trusted QA data endpoints and files until strict broker/container backends are available.
+
+Worker threads are not a hard sandbox for hostile plugins.
+
+## Retention recommendation
+
+By default, do not retain raw authorization headers, environment variables, URI queries, downloaded data files, or full sensitive MCP payloads.
+
+Suggested policy:
+
+- run metadata and sanitized reports: 7–30 days;
+- contract baselines: for the supported release lifetime;
+- normalized traces: 7–14 days unless needed for audit;
+- raw trace payloads: opt-in, restricted, and 24–72 hours;
+- malicious parser samples: encrypted quarantine, audited access, and 24–72 hours;
+- debug logging: time-bounded with automatic expiry.
+
+Deletion procedures should cover CI artifacts, object storage, developer workstations, backups, caches, and third-party telemetry. Incident holds need an owner, reason, scope, and expiry.

package/docs/SNAPSHOTS-AND-REPLAY.md

@@ -0,0 +1,85 @@
+# Semantic Snapshots, Diffs, and Replay
+
+## Snapshots
+
+Add a plain-language snapshot assertion to the latest action:
+
+```text
+Call tool "get_customer" with:
+  id: "123"
+
+Expect "structuredContent" matches snapshot "customer"
+```
+
+Create or explicitly update snapshots:
+
+```bash
+mcprigor test customer.mcpr --snapshot customer.snap.json --update-snapshots
+```
+
+Verify without modifying the file:
+
+```bash
+mcprigor test customer.mcpr --snapshot customer.snap.json
+```
+
+Missing or changed snapshots fail unless `--update-snapshots` is present. Snapshot names are namespaced by test ID.
+
+YAML/JSON suites may ignore volatile paths:
+
+```yaml
+snapshots:
+  file: customer.snap.json
+  ignore:
+    - $.generatedAt
+```
+
+Or on one assertion:
+
+```yaml
+snapshot:
+  name: customer
+  ignore: [$.id, $.createdAt]
+```
+
+Ignored array positions are removed from the owned snapshot copy; application fields are never ignored automatically.
+
+## Semantic diffs
+
+Snapshot failures report path-level additions, removals, and replacements:
+
+```text
+- $.status: "created"
++ $.status: "pending"
++ $.metadata.region: "us-east"
+```
+
+Compare two JSON artifacts directly:
+
+```bash
+mcprigor snapshot-diff expected.json actual.json
+```
+
+Objects are compared by sorted keys and arrays by index. Equality remains type-sensitive.
+
+## Replay
+
+Replay request events from an MCP Rigor trace against a trusted target configuration:
+
+```bash
+mcprigor replay .mcprigor/run-1/trace.normalized.jsonl --target server.mcpr
+```
+
+Replay always creates a fresh MCP connection and initialization handshake. It never executes commands or URLs from the trace.
+
+Safe read/list methods are allowed by default. Tool calls are denied unless each exact tool is approved:
+
+```bash
+mcprigor replay trace.normalized.jsonl \
+  --target server.mcpr \
+  --allow-tool get_customer
+```
+
+Requests execute sequentially. Responses are compared semantically with their recorded correlated response. Changed responses return a path-level diff and exit status `1`.
+
+Replay is an API-boundary semantic check, not a simulation of original timing, concurrency, transport bytes, notifications, or external system state.