ptywright 0.3.0 → 0.5.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ptywright",
-  "version": "0.3.0",
+  "version": "0.5.0",
   "description": "Terminal/TUI automation driver over PTY + xterm, exposed as MCP tools",
   "keywords": [
     "agent",
@@ -37,6 +37,7 @@
   "exports": {
     ".": "./dist/cli.mjs",
     "./agent": "./dist/agent.mjs",
+    "./config": "./dist/config.mjs",
     "./mcp": "./dist/mcp.mjs",
     "./pty-cassette": "./dist/pty-cassette.mjs",
     "./session": "./dist/session.mjs",
@@ -62,6 +63,7 @@
     "prepublishOnly": "bun run build"
   },
   "dependencies": {
+    "@aitty/snapshot": "^0.5.1",
     "@modelcontextprotocol/sdk": "^1.25.2",
     "@xterm/headless": "^6.0.0",
     "asciinema-player": "3.9.0",

package/skills/ptywright-testing/SKILL.md

@@ -1,122 +1,156 @@
 ---
 name: ptywright-testing
-description: Terminal/TUI automation and regression testing using ptywright (PTY + xterm) via CLI or MCP tools. Use when you need to (1) drive a CLI/TUI app (send keys/mouse, wait, snapshot), (2) run scripted regressions (run/run-all) and review the HTML report (index.html + run.summary.json), or (3) record an interactive MCP-driven session into a replayable script with golden checkpoints.
+description: Build, run, record, replay, debug, and maintain deterministic terminal, TUI, PTY cassette, and browser-terminal agent regression tests with ptywright. Use when an agent needs to drive CLI/TUI apps, create ptywright scripts, configure ptywright.config.*, record or replay PTY output, solidify browser terminal agent flows into non-AI snapshot tests, inspect generated artifacts, or diagnose ptywright CI failures.
 ---
 
 # Ptywright Testing
 
-Use ptywright to run deterministic CLI/TUI regression tests with readable "terminal screenshots" and a Playwright-like HTML report.
+Use ptywright when the task involves terminal or browser-terminal behavior that should be repeatable without manual inspection. Prefer stable text, DOM, and terminal snapshots over screenshots unless the user explicitly needs visual media.
 
-## Installation & Usage
+## First Decision
 
-```bash
-# 推荐：bunx 一次性运行
-bunx ptywright@latest <command>
+Choose one workflow before editing:
+
+- **Browser terminal agent regression**: Use when a web app renders a terminal and exposes `[data-terminal-root]`, or when testing integrations such as Codex/Claude/Droid wrappers. Read `references/agent-regression.md`.
+- **Raw PTY recording and replay**: Use when the user wants to capture terminal bytes from `node-pty`, Bun Terminal, `bun-pty`, or an arbitrary command, then replay them into another renderer. Read `references/raw-pty-cassettes.md`.
+- **Scripted TUI tests**: Use when testing a CLI/TUI directly through ptywright scripts, golden snapshots, and HTML reports. Read `references/script-runner.md`.
+- **MCP interactive driving or recording**: Use when an agent should interact through ptywright MCP tools or record an MCP-driven session into a script. Read `references/mcp-tools.md`.
+- **CI/debugging/artifact triage**: Use when a ptywright run failed, snapshots mismatch, a manifest is stale, or reusable commands need to be executed. Read `references/ci-and-debugging.md`.
+
+If more than one workflow applies, start with the highest-level workflow that preserves determinism. For example, for an evolving browser terminal renderer, record a raw PTY cassette first, then create a browser agent regression that replays the cassette into the renderer.
 
-# 或全局安装
-bun add -g ptywright
-ptywright <command>
+## Installation And Entry Points
 
-# 本仓库内开发
+Prefer the local project command when working inside a ptywright checkout:
+
+```bash
 bun run bin/ptywright <command>
 ```
 
-## Choose the interface
+Prefer published package commands in downstream projects:
 
-- **MCP tools**: best for agent-driven interactive flows (`launch_session`, `wait_for_text`, snapshots, recording).
-- **CLI**: best for local deterministic regressions and reviewing HTML reports.
+```bash
+bunx ptywright@latest <command>
+# or
+npx ptywright@latest <command>
+```
 
-## Start the MCP server
+Common commands:
 
 ```bash
-# stdio (default)
-bunx ptywright@latest mcp
-
-# 精简 tools（降低上下文压力）
-bunx ptywright@latest mcp --caps core
+ptywright mcp
+ptywright mcp --caps core
+ptywright run <file.json|file.ts>
+ptywright run-all --dir scripts
+ptywright agent run <flow.json> --update-snapshots
+ptywright agent check
+ptywright pty record --out tests/cassettes/session.pty.json -- <command> [args...]
+```
 
-# HTTP 模式
-bunx ptywright@latest mcp-http --port 3000
+## Project Config
+
+Use `ptywright.config.ts` for project defaults, not as a second test DSL. The flow file remains the test case.
+
+```ts
+import { defineConfig } from "ptywright/config";
+
+export default defineConfig({
+  agent: {
+    artifactsRoot: ".tmp/agent",
+    cassetteDir: "tests/agent-cassettes",
+    snapshotDir: "tests/agent-snapshots",
+    defaults: {
+      headless: true,
+      timeoutMs: 45_000,
+      screenshot: false,
+      viewports: [{ name: "desktop", width: 1280, height: 820 }],
+      mask: [{ regex: "session_[a-z0-9]+", replacement: "<session>" }],
+    },
+  },
+});
 ```
 
-Capabilities (`--caps`): `all|core|debug|script|recording` (comma separated)
+Priority rule: explicit CLI args override flow fields, and flow fields override config defaults. Config-relative paths resolve from the config file directory.
+
+## Core Invariants
 
-## Configure MCP Client
+- Keep tests deterministic: fixed terminal size, explicit waits, stable snapshots, masks for random text.
+- Prefer structured APIs and generated reusable commands over shell string reconstruction.
+- Treat `--update-snapshots` as the only intentional baseline update path.
+- Use generated manifests and summaries as durable reproduction bundles.
+- Do not hand-edit cassette, run-record, summary, or manifest command metadata unless a test explicitly asks for malformed fixture data.
+- Avoid app-specific assumptions. ptywright should integrate with any renderer through commands, URLs, DOM roots, and cassette data.
 
-**Claude Desktop / Cursor** (`~/.config/claude/claude_desktop_config.json`):
+## Minimal Examples
+
+Browser agent flow:
 
 ```json
 {
-  "mcpServers": {
-    "ptywright": {
-      "command": "bunx",
-      "args": ["ptywright@latest", "mcp"]
-    }
-  }
+  "name": "browser_terminal_smoke",
+  "launch": {
+    "mode": "command",
+    "agentFlavor": "generic",
+    "command": "node",
+    "args": ["scripts/start-browser-terminal.js", "--print-url"],
+    "waitForUrlMs": 15000
+  },
+  "steps": [
+    { "type": "waitForStableDom" },
+    { "type": "snapshot", "name": "ready", "targets": ["terminal", "dom"] }
+  ]
 }
 ```
 
-## Run scripts (deterministic regression)
-
-### Run the whole suite (preferred)
+Raw PTY cassette:
 
 ```bash
-bunx ptywright@latest run-all --dir scripts
+ptywright pty record --out tests/cassettes/codex.pty.json -- codex --yolo
+ptywright pty replay tests/cassettes/codex.pty.json --speed 0
+ptywright pty validate tests/cassettes/codex.pty.json
 ```
 
-Output to focus on:
-- `reportPath` (open in a browser)
-- `summaryPath` (`run.summary.json` for agents/CI)
-
-MCP equivalent:
-- `run_all_scripts` (defaults: `dir="scripts"`, suite report in `.tmp/run-all/`)
-- Keep MCP output small: `run_all_scripts(includeEntries="failures", maxEntries=20)`
+Script runner:
 
-### Run one script
-
-```bash
-bunx ptywright@latest run <file.json|file.ts> [--artifacts-dir <dir>]
+```json
+{
+  "name": "tui_smoke",
+  "command": ["bun", "tests/fixtures/tui_demo.ts"],
+  "cols": 80,
+  "rows": 24,
+  "steps": [
+    { "type": "waitForText", "text": "Ready" },
+    { "type": "snapshot", "kind": "text", "saveAs": "ready" }
+  ]
+}
 ```
 
-MCP: `run_script(scriptPath=...)`
-
-## Debug a failure
-
-Script runner artifacts to check (paths are returned by CLI/MCP):
-
-- `*.report.html` (timeline + snapshots)
-- `*.cast` (full playback)
-- `failure.last.view.txt` / `failure.last.txt` (last screen)
-- `failure.error.txt` (stack trace)
+## Verification Commands
 
-Tip: for flaky waits, prefer `scope="buffer"` when the content may have scrolled into scrollback.
-
-## Record an interactive flow (MCP)
-
-1) `start_script_recording(name=...)`
-2) Drive the app with normal tools:
-   - `launch_session` → `send_text` / `press_key` / `wait_for_text` / `snapshot_*`
-3) Add golden checkpoints: `mark(label=...)`
-4) Export: `stop_script_recording(recordingId=..., writeFiles=true)`
-
-## All-tools smoke (recommended)
-
-To verify ptywright MCP tool coverage without relying on external apps/network, run:
+Use the narrowest useful verification first, then broaden when editing shared behavior:
 
 ```bash
-bun test tests/mcp_all_tools_smoke.test.ts
+bun run format:check
+bun run lint
+bun test tests/agent_config.test.ts
+bun test tests/agent_rerun.test.ts
+bun run build
+bun run check
 ```
 
-This exercises `core + debug + script + recording` tools end-to-end.
+For downstream projects:
 
-## Determinism tips
-
-- Fix terminal size (`cols/rows`) and `TERM` (`xterm-256color`) in `launch_session`.
-- Use `wait_for_stable_screen` before assertions/snapshots to reduce flake.
-- Use `mask` to redact timestamps, random IDs, spinners, etc.
-- For live LLM apps: assert on stable markers/state transitions, not exact prose.
+```bash
+ptywright agent validate <artifact-or-dir>
+ptywright agent inspect <artifact-or-dir>
+ptywright agent commands <artifact-or-dir> --json
+ptywright agent exec <artifact-or-dir> --command rerun
+```
 
-## Environment knobs
+## Resource Map
 
-- `TUI_TEST_PTY_BACKEND=auto|bun-terminal|bun-pty`
-  - default `auto`: macOS/Linux prefers `bun-terminal`, Windows uses `bun-pty`
+- `references/agent-regression.md`: Browser terminal agent flows, cassettes, snapshots, promote/check/rerun, and renderer integration.
+- `references/raw-pty-cassettes.md`: Raw PTY cassette recording, replay, wrapper integration, and renderer handoff.
+- `references/script-runner.md`: JSON/TS script runner, MCP script recording, goldens, masks, and reports.
+- `references/mcp-tools.md`: MCP setup and tool selection.
+- `references/ci-and-debugging.md`: Failure triage, manifests, reusable commands, snapshot updates, and CI gates.

package/skills/ptywright-testing/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Ptywright Testing"
+  short_description: "Terminal and browser-agent regression testing"
+  default_prompt: "Use $ptywright-testing to create or debug deterministic terminal, PTY cassette, or browser-agent regression tests."

package/skills/ptywright-testing/references/agent-regression.md

@@ -0,0 +1,132 @@
+# Browser Agent Regression
+
+Use this workflow when ptywright drives a browser-hosted terminal renderer. The renderer must expose a terminal root as `[data-terminal-root]`.
+
+## Contract
+
+`launch.mode=command` is the preferred integration:
+
+- `command` and `args` start a wrapper or app process.
+- The process prints a browser URL to stdout or stderr.
+- ptywright opens that URL with Playwright.
+- The page renders the terminal under `[data-terminal-root]`.
+- Steps drive browser input and compare terminal/DOM snapshots.
+
+Use `launch.mode=url` only when the page is already running.
+
+## Flow Lifecycle
+
+1. Create a flow JSON or TS file.
+2. Run live once and write baselines:
+
+   ```bash
+   ptywright agent run tests/agents/name.flow.json --update-snapshots
+   ```
+
+3. Compare later without updating:
+
+   ```bash
+   ptywright agent run tests/agents/name.flow.json
+   ```
+
+4. Replay a run record or cassette without the live agent:
+
+   ```bash
+   ptywright agent replay .tmp/agent/name/name.agent-run.json
+   ptywright agent replay .tmp/agent/name/name.cassette.json
+   ```
+
+5. Promote a good live run into committed non-AI regression:
+
+   ```bash
+   ptywright agent promote .tmp/agent/name/name.cassette.json --update-snapshots
+   ```
+
+6. Run the committed suite:
+
+   ```bash
+   ptywright agent check
+   ptywright agent replay-all tests/agent-cassettes --update-snapshots
+   ```
+
+## Recommended Flow Shape
+
+```json
+{
+  "name": "agent_renderer_smoke",
+  "launch": {
+    "mode": "command",
+    "agentFlavor": "codex",
+    "command": "node",
+    "args": [
+      "tests/harness/browser-terminal.js",
+      "--",
+      "codex",
+      "--yolo",
+      "--print-url"
+    ],
+    "waitForUrlMs": 20000,
+    "urlRegex": "(https?://\\S+)"
+  },
+  "defaults": {
+    "timeoutMs": 45000,
+    "screenshot": false,
+    "mask": [{ "regex": "req_[a-zA-Z0-9]+", "replacement": "<request-id>" }]
+  },
+  "viewports": [{ "name": "desktop", "width": 1280, "height": 820 }],
+  "steps": [
+    { "type": "waitForStableDom", "quietMs": 600 },
+    { "type": "snapshot", "name": "launch", "targets": ["terminal", "dom"] }
+  ]
+}
+```
+
+Keep the flow generic. ptywright should not import app internals. The downstream app should provide a command or test harness that prints a browser URL and can consume replay data if needed.
+
+## Recording Browser Interactions
+
+Use `agent record` when manually exploring a browser-terminal flow:
+
+```bash
+ptywright agent record tests/agents/base.flow.json \
+  --out tests/agents/recorded.flow.json \
+  --duration-ms 60000 \
+  --headed
+```
+
+End recording by waiting for `duration-ms` to elapse or by stopping the process. The output is a normal flow JSON containing keyboard/click steps plus a final checkpoint.
+
+## Non-AI Regression Strategy
+
+For evolving agent UIs:
+
+1. Capture or create a stable PTY or browser-agent cassette.
+2. Replay that cassette into the renderer.
+3. Snapshot terminal text and DOM.
+4. Commit cassette and snapshots.
+5. Use `agent check` in CI.
+
+This lets renderer changes be verified without asking the live AI to reproduce the same answer.
+
+## Artifact Meanings
+
+- `.agent-run.json`: Per-run record with `commands.replay.argv` and `commands.updateSnapshots.argv`.
+- `.cassette.json`: Normalized flow spec plus captured terminal/DOM frames and hashes.
+- `agent-replay.summary.json`: Replay-all suite summary.
+- `agent-check.summary.json`: Committed cassette check summary.
+- `agent-promote.summary.json`: Promote operation summary.
+- `ptywright-agent.manifest.json`: Hash-indexed portable artifact bundle.
+- `index.html`: Human-readable report with snapshots and reusable commands.
+
+## Common Commands
+
+```bash
+ptywright agent inspect .tmp/agent-check
+ptywright agent validate .tmp/agent-check
+ptywright agent commands .tmp/agent-check --json
+ptywright agent exec .tmp/agent-check --command rerun
+ptywright agent exec .tmp/agent-check --command updateSnapshots
+ptywright agent rerun .tmp/agent-check/agent-check.summary.json
+```
+
+Prefer `agent exec` when an artifact already contains a reusable command. It avoids shell parsing and relocates copied manifest bundles safely.

package/skills/ptywright-testing/references/ci-and-debugging.md

@@ -0,0 +1,95 @@
+# CI And Debugging
+
+Use this guide when a ptywright command fails, CI times out, snapshots mismatch, or generated artifact commands need to be reused.
+
+## First Triage
+
+1. Read the failing command and exact artifact paths from the log.
+2. Open the HTML report if available.
+3. Inspect the generated summary JSON.
+4. Run validation on the artifact or directory.
+5. Use generated commands instead of reconstructing shell strings manually.
+
+Commands:
+
+```bash
+ptywright agent inspect <artifact-or-dir>
+ptywright agent validate <artifact-or-dir>
+ptywright agent commands <artifact-or-dir> --json
+ptywright agent commands <artifact-or-dir> --command rerun
+ptywright agent exec <artifact-or-dir> --command rerun
+ptywright agent exec <artifact-or-dir> --command updateSnapshots
+```
+
+## Snapshot Mismatches
+
+Default replay/check mode compares snapshots. Only update baselines intentionally:
+
+```bash
+ptywright agent replay-all tests/agent-cassettes --update-snapshots
+ptywright agent exec <artifact-or-dir> --command updateSnapshots
+```
+
+For script runner:
+
+```bash
+ptywright run-all --dir scripts --update-goldens
+ptywright script exec <summary-or-dir> --command updateGoldens
+```
+
+Always inspect diffs before committing updated baselines.
+
+## Portable Bundles
+
+Agent run/check/promote/replay-all outputs include `ptywright-agent.manifest.json`. A manifest bundle can be copied and still supports:
+
+```bash
+ptywright agent inspect <copied-dir>
+ptywright agent commands <copied-dir> --json
+ptywright agent exec <copied-dir> --command rerun
+ptywright agent validate <copied-dir>
+```
+
+If a directory has artifacts but no top-level manifest, use `agent validate <dir>` for recursive validation. `agent commands` and `agent exec` expect a manifest-backed command bundle for directory arguments.
+
+## Common Failure Causes
+
+- Missing `[data-terminal-root]` in browser terminal pages.
+- Flow waits on unstable AI prose instead of stable markers.
+- Snapshot baseline was not updated after an intentional UI change.
+- Random text was not masked.
+- Relative cassette or snapshot paths were moved without a manifest bundle.
+- Stored command metadata in summaries was hand-edited and no longer matches schema expectations.
+- CI is too slow for tests that run multiple full browser replays in one case.
+
+## Timeout Reduction
+
+When a test times out:
+
+- Avoid running setup and rerun paths that both do full browser replay in the same test.
+- Use summary fixtures to test command metadata or override behavior.
+- Keep one full end-to-end test per workflow and make surrounding tests narrower.
+- Use committed deterministic cassettes instead of live agents.
+- Keep test timeouts realistic but do not hide structural slowness by only increasing timeouts.
+
+## Repository Gates
+
+For ptywright itself:
+
+```bash
+bun run format:check
+bun run lint
+bun test tests/agent_rerun.test.ts
+bun test tests/agent_promote.test.ts tests/agent_commands.test.ts
+bun run build
+bun run check
+```
+
+For downstream projects:
+
+```bash
+ptywright agent check
+ptywright agent validate .tmp/agent-check
+```
+
+Use the narrowest failing test while iterating, then broaden before finalizing shared behavior.

package/skills/ptywright-testing/references/mcp-tools.md

@@ -0,0 +1,91 @@
+# MCP Tools
+
+Use MCP when an agent should interact with a live terminal session, inspect terminal state, or record an exploratory flow into a script.
+
+## Start Server
+
+```bash
+ptywright mcp
+ptywright mcp --caps core
+ptywright mcp --caps core,script,recording
+ptywright mcp-http --port 3000
+```
+
+Capabilities:
+
+- `core`: Launch sessions, send input, wait, snapshot.
+- `debug`: Extra inspection and traces.
+- `script`: Run script files and suites.
+- `recording`: Record MCP tool calls into scripts.
+- `all`: Everything.
+
+Use smaller capability sets to reduce agent context pressure.
+
+## Client Config
+
+Example for clients that use a JSON MCP server config:
+
+```json
+{
+  "mcpServers": {
+    "ptywright": {
+      "command": "bunx",
+      "args": ["ptywright@latest", "mcp", "--caps", "core,script,recording"]
+    }
+  }
+}
+```
+
+Inside this repository, use:
+
+```json
+{
+  "mcpServers": {
+    "ptywright": {
+      "command": "bun",
+      "args": ["run", "src/cli.ts", "mcp"]
+    }
+  }
+}
+```
+
+## Tool Selection
+
+Typical interactive sequence:
+
+1. `launch_session` with fixed `cols`, `rows`, and `env.TERM`.
+2. `wait_for_text` for stable startup markers.
+3. `send_text`, `press_key`, or mouse tools.
+4. `wait_for_stable_screen` before snapshots.
+5. `snapshot_text`, `snapshot_view`, or `snapshot_grid`.
+6. `close_session` when done.
+
+Prefer semantic terminal snapshots over screenshots. Use screenshots only if the task explicitly needs visual proof.
+
+## Recording
+
+Use recording when an exploratory interaction should become a repeatable test:
+
+```text
+start_script_recording
+launch_session
+send_text / press_key / wait_for_text / snapshot_text
+mark
+stop_script_recording(writeFiles=true)
+```
+
+After export, run the generated script from the CLI to ensure it is deterministic:
+
+```bash
+ptywright run <exported-script.json>
+ptywright run <exported-script.json> --update-goldens
+```
+
+## Context Control
+
+When using MCP from an LLM agent:
+
+- Avoid returning huge terminal text unless needed.
+- Prefer `includeText=false` or failure-only entries for suite tools when available.
+- Use report and summary paths for detailed inspection.
+- Use masks early if non-deterministic output appears.

package/skills/ptywright-testing/references/raw-pty-cassettes.md

@@ -0,0 +1,82 @@
+# Raw PTY Cassettes
+
+Use raw PTY cassettes when the goal is to capture terminal output once and replay it later without relaunching the original CLI, AI agent, or TUI.
+
+## CLI Recording
+
+```bash
+ptywright pty record --out tests/cassettes/session.pty.json -- <command> [args...]
+ptywright pty validate tests/cassettes/session.pty.json
+ptywright pty inspect tests/cassettes/session.pty.json
+ptywright pty replay tests/cassettes/session.pty.json --speed 0
+```
+
+Examples:
+
+```bash
+ptywright pty record --out tests/cassettes/codex-yolo.pty.json -- codex --yolo
+ptywright pty record --out tests/cassettes/browser-terminal-codex.pty.json -- \
+  node tests/harness/browser-terminal.js -- codex --yolo
+```
+
+Use `--cols`, `--rows`, `--term`, and `--backend` to stabilize output:
+
+```bash
+ptywright pty record \
+  --out tests/cassettes/session.pty.json \
+  --cols 120 \
+  --rows 32 \
+  --term xterm-256color \
+  --backend auto \
+  -- <command>
+```
+
+## Programmatic Integration
+
+Use `ptywright/pty-cassette` in projects that already control a PTY-like object.
+
+```ts
+import { wrapPtyLike } from "ptywright/pty-cassette";
+
+const recorder = wrapPtyLike(ptyProcess, {
+  path: "tests/cassettes/session.pty.json",
+  command: ["codex", "--yolo"],
+  cols: 120,
+  rows: 32,
+  term: "xterm-256color",
+});
+
+// Use recorder.process like the original ptyProcess.
+// Close/finalize according to the package API.
+```
+
+Prefer wrapper integration when a downstream project wants to keep using native `node-pty`, Bun Terminal, or `bun-pty` while still producing ptywright-compatible data.
+
+## Renderer Handoff Pattern
+
+For browser terminal renderers:
+
+1. Record raw PTY output as `*.pty.json`.
+2. Add a small local harness in the renderer project that loads this cassette and renders it into the browser terminal.
+3. Print the browser URL from that harness.
+4. Use a ptywright agent flow to open the URL and snapshot `[data-terminal-root]`.
+
+This separates byte-level reproduction from renderer-level DOM regression.
+
+## Updating Scenarios Without Duplicating Huge Sessions
+
+Avoid repeatedly recording long sessions just to test one rendering edge.
+
+Recommended patterns:
+
+- Keep small, named cassettes for specific UI states: `code-block.pty.json`, `spinner.pty.json`, `long-line.pty.json`.
+- Prefer fixture commands that emit deterministic terminal sequences for a targeted state.
+- Trim at the source by recording a shorter command or a purpose-built harness.
+- Use masks to normalize timestamps, ids, spinner ticks, and model names.
+- Store cassettes under `tests/cassettes/` and keep renderer snapshots under `tests/agent-snapshots/`.
+
+If an existing long cassette is useful but contains irrelevant frames, create a derived fixture in the app's harness rather than hand-editing hashes unless the project has a supported cassette transform.
+
+## When To Use Browser Agent Cassettes Instead
+
+Use browser agent cassettes when you need DOM snapshots, viewport coverage, or Playwright interactions. Use raw PTY cassettes when you only need terminal bytes and want broad compatibility with any PTY provider.