explorbot 0.2.2 → 0.2.4

package/docs/contributing/ai-integration-tests.md

@@ -0,0 +1,57 @@
+# AI Agent Integration Tests
+
+We test AI agents with [`@copilotkit/aimock`](https://github.com/CopilotKit/aimock), an HTTP mock server that speaks LLM provider protocols. Unlike in-process model stubs, it runs the real `Provider` class and lets you inspect the HTTP requests through its Journal.
+
+Reference implementation: `tests/integration/planner.test.ts`.
+
+## Principles
+
+### What we mock
+
+- **The AI provider** — via the aimock HTTP server. Point the Vercel AI SDK at `mock.url/v1` with `createOpenAI({ compatibility: 'compatible' })` and `openai.chat('model-name')`. The `compatibility: 'compatible'` option keeps the mock on the Chat Completions API; without it, the SDK defaults to the Responses API, which aimock does not fully implement.
+- **Explorer, StateManager, Researcher, ExperienceTracker** — duck-typed mocks with only the methods the agent under test calls. Each agent runs in isolation; downstream agents (such as Researcher when testing Planner) return canned output.
+- **`withPage`** — resolves with a fake page object for agents that validate locators, since no browser runs in these tests.
+
+### What we don't mock
+
+- The real `Provider` class, so tests exercise serialization, retry, and telemetry paths.
+- Parsers, result objects, and markdown processing — the real pipeline runs.
+- `ActionResult.fromState()`, which works without the filesystem when `state.html` is inline.
+
+### What we test
+
+- **Output correctness**: the plan or research markdown the agent returns matches the canned AI response.
+- **Prompt construction** (via Journal): `mock.getLastRequest()` returns the messages sent to the AI, so you assert on what the agent prompted, not just what it returned. This is the main reason to use aimock.
+- **Control flow**: cache hits, dedup, retries, error paths, and style or feature injection. Inspect the request count and prompt content.
+
+### Fixture types
+
+- Text response (for `chat()` / `invokeConversation()`): `mock.on({}, { content: 'text' })`
+- Structured output (for `generateObject()`): `mock.on({}, { content: JSON.stringify(obj) })` — the SDK parses it back via `response_format: json_schema`
+- Sequential responses: `mock.on({ sequenceIndex: 0 }, ...)`, `sequenceIndex: 1`, ...
+- Errors: `mock.on({}, { error: {...}, status: 500 })`
+- See aimock docs for matching by user message, tool name, regex, or predicate.
+
+### Test data
+
+- Canned UI maps live in `test-data/ui-maps/`.
+- Use fictional applications (Task Tracker, and so on). Never use real product data or user names.
+- Two formats exist, depending on the agent under test:
+  - **Planner-input** (`task-board.md`): Element and Type columns only — what the Planner sees after its own table-column filtering.
+  - **Researcher-output** (`task-board-research.md`): full Element, Type, ARIA, CSS, and Coordinates columns — the raw Researcher AI output format.
+
+### Module-level caches
+
+Many agents hold module-level state: plan registry, session dedup, style cache, research cache. Each module that caches across calls must export a `clearXxx()` function for use in `beforeEach`. See the existing exports:
+
+- `src/ai/planner/subpages.ts` — `clearPlanRegistry()`
+- `src/ai/planner/session-dedup.ts` — `clearSessionDedup()`
+- `src/ai/planner/styles.ts` — `clearStyleCache()`
+- `src/ai/researcher/cache.ts` — `clearResearchCache()`
+
+## Running
+
+```bash
+bun test tests/integration/planner.test.ts
+bun test tests/integration/
+```

package/docs/contributing/contributing.md

@@ -0,0 +1,90 @@
+# Contributing to Explorbot
+
+## Project Philosophy
+
+Explorbot is general-purpose. It works with any web application, with no site-specific code. Keep this in mind when you contribute:
+
+- Solutions must work across different websites.
+- Don't hardcode locators or site-specific selectors.
+- Prefer universal patterns: ARIA, semantic HTML, common UI conventions.
+- A fix that only helps one site is probably the wrong approach.
+
+## Before You Start
+
+Discuss first, code second. Open an issue to propose your idea before you send a PR. This saves wasted effort and aligns on the approach.
+
+If Explorbot struggles on your site, try the built-in extension points first:
+
+- Knowledge files teach Explorbot about your pages (credentials, wait conditions, hints).
+- Rules add agent-specific guidance for navigation, research, and planning.
+- Hooks run custom logic before and after actions.
+- Configuration adjusts models, timeouts, and browser settings.
+
+See the [knowledge](../workflow/knowledge.md), [hooks](../web-testing/hooks.md), and [configuration](../reference/configuration.md) docs.
+
+## Pull Requests
+
+We accept PRs that help a variety of users. Here is what gets merged.
+
+### Tiny PRs get merged first
+
+Small, focused changes are reviewed and merged quickly. Examples:
+
+- Fix a typo in a prompt.
+- Add a missing ARIA selector pattern.
+- Improve an error message.
+- Fix a bug with a clear reproduction.
+
+### PRs that change agent logic require manual testing
+
+Changes to agents (tester, navigator, researcher, pilot, and others) affect how Explorbot interacts with every website. These PRs must be:
+
+- Polished — clean code, no leftover debug artifacts.
+- Tested by a person on real websites.
+- Described clearly: what changed and why.
+
+### PRs that change prompts require execution traces
+
+If you change internal AI prompts or rules, you must provide a [Langfuse](https://langfuse.com) execution trace that shows the change improved behavior. "It should work better" is not evidence. Show the before and after.
+
+### We accept agentic PRs
+
+AI-generated contributions are welcome if they follow the rules above: small, focused, tested, with clear explanations.
+
+### What NOT to send
+
+- PRs that reformat code differently. We use [Biome](https://biomejs.dev/) for formatting. Run `bun run format` before submitting.
+- Large refactoring PRs without prior discussion.
+- Site-specific fixes that only help one website.
+- Changes that break existing tests.
+
+## Development Setup
+
+```bash
+bun install
+bunx playwright install
+```
+
+## Code Style
+
+- Biome for formatting and linting. Run `bun run format` after every change.
+- No comments unless explicitly needed.
+- Early returns instead of nested if/else.
+- KISS/YAGNI — make the smallest change possible.
+- No code duplication. Check if it already exists.
+- Use dedent for formatting prompts.
+- No ternary operators.
+
+See `CLAUDE.md` for the full guidelines.
+
+## Testing
+
+Run `bun run format`, `bun run lint`, `bun run test:unit`, and `bun run test:node` before you submit a PR — all checks must pass. See the [testing guide](testing.md) for details, [AI integration tests](ai-integration-tests.md) for mocked-LLM agent tests, and [regression tests](regression-tests.md) for the real-AI end-to-end harness.
+
+## Using Claude Code
+
+This project includes Claude Code skills to help with development. See the `.claude/skills/` directory: changelog, demo-video, explorbot-debug, explorbot-fix-session, image-processing, prompt-audit, xpath-testing.
+
+## Questions?
+
+Open an issue on GitHub for questions or discussion.

package/docs/contributing/demo-videos.md

@@ -0,0 +1,36 @@
+# Demo Videos
+
+Explorbot sessions can be turned into demo videos for social media and presentations. The generator composites the browser screencast of a test with a simulated terminal that replays the session's real log lines at their original pace, over an abstract background with window chrome and shadows. Everything shown is real recorded data: no more than 1.25x speedup, successful runs only.
+
+Implementation lives in `.claude/skills/demo-video/` (`demo-video.ts` exports `analyzeDemoCandidates` and `createDemoVideo`; the Bunoshfile wraps them as commands).
+
+## Prerequisites
+
+- `vhs` and `ttyd` — render the terminal simulation (`go install github.com/charmbracelet/vhs@latest`)
+- `ffmpeg` and ImageMagick
+- IBM Plex Mono font (optional — VHS falls back to its default mono font)
+- A session recorded with screencasts: enable `ai.agents.historian.screencast` in `explorbot.config.js` so `output/screencasts/*.webm` exist alongside `output/explorbot.log`
+
+## Usage
+
+```bash
+bunx bunosh demo:analyze output/explorbot.log --screencasts output/screencasts
+bunx bunosh demo:video --size landscape --app-title "My App"
+bunx bunosh demo:video "upload a file" --size vertical --terminal-theme light
+```
+
+`demo:analyze` lists candidate segments ranked by how well they will read on video. `demo:video` renders the best one, or the test whose scenario name matches the first argument. Key options: `--duration` (target seconds, default 30), `--size` (`landscape`, `square`, `vertical`, or `WxH`), `--app-title` (browser window title, defaults to the tested app's host), `--terminal-theme` (`dark` or `light`), `--bg-image` (`auto` fetches a random abstract photo from Unsplash, `gradient` generates one offline, or pass a file/URL). Run `bunx bunosh demo:video --help` for the full list.
+
+The renderer verifies its output and writes three check frames (`*-frame-first/mid/last.png`) next to the video — review them before publishing.
+
+## How segments are picked
+
+Only successful tests with an existing screencast qualify. Within a test, the scorer prefers windows that:
+
+- are dense in visible actions (click, fill, type, navigate) rather than thinking or verification steps
+- have no dead air — inter-step gaps stay under 10 seconds
+- show forward progress — windows dominated by repeated retry steps are rejected
+- were executed live — batch-logged step bursts cannot sync with the video and are rejected
+- do not end on a failure note or mid-navigation on a blank page
+
+The log's `Saved screencast:` line joins a test to its `.webm`; video time is calibrated from that timestamp minus the video duration. If a screencast file was overwritten by a run outside the log, it is skipped.

package/docs/contributing/npm-package.md

@@ -0,0 +1,138 @@
+# Building and Publishing the npm Package
+
+Explorbot develops on Bun but ships to npm as a Node.js-compatible package. This page covers how the build works and how to publish it.
+
+## Prerequisites
+
+- Bun (for development and running the build)
+- Node.js >= 24 (for verifying the build output)
+- npm account with publish access to `explorbot` package
+
+## How the Build Works
+
+The source is TypeScript with `.ts` imports (enabled by `allowImportingTsExtensions` in `tsconfig.json`). Bun runs these natively, but Node.js needs compiled `.js` files.
+
+The build runs the TypeScript compiler (`tsc`) with a dedicated `tsconfig.build.json`:
+
+1. **TypeScript compilation** - Compiles `src/`, `bin/`, and `boat/` to `dist/`, preserving the directory structure.
+2. **Import rewriting** - `rewriteRelativeImportExtensions` rewrites `.ts` imports to `.js` in the output (a TypeScript 5.7+ feature).
+3. **Type declarations** - `scripts/build-types.ts` emits `.d.ts` files for the library API (see [Type Declarations](#type-declarations)).
+4. **Asset copying** - Copies `rules/` and `assets/sample-files/` into `dist/` so runtime path resolution works.
+5. **Shebang replacement** - Replaces `#!/usr/bin/env bun` with `#!/usr/bin/env node` in every CLI entry point declared in `bin`: `dist/bin/explorbot-cli.js` and `dist/boat/prima/bin/prima-cli.js`.
+
+### Build Configuration
+
+**`tsconfig.build.json`** extends the base `tsconfig.json` with:
+
+| Option | Value | Purpose |
+|--------|-------|---------|
+| `noEmit` | `false` | Enable output (base config has `true`) |
+| `outDir` | `dist` | Compilation output directory |
+| `rewriteRelativeImportExtensions` | `true` | Rewrite `.ts` → `.js` in imports |
+| `declaration` | `false` | The JS build emits no `.d.ts`; declarations are built separately (see below) |
+| `sourceMap` | `false` | No source maps in the published package |
+| `skipLibCheck` | `true` | Skip type checking of dependencies |
+
+The build skips type checking (`--noCheck` flag) because Bun is more permissive than `tsc` strict mode. Bun enforces type safety during development.
+
+### Package Structure
+
+After the build, the npm package contains:
+
+```
+dist/
+├── bin/explorbot-cli.js    # CLI entry point (#!/usr/bin/env node)
+├── src/                    # Compiled application code (.js) + type declarations (.d.ts)
+│   ├── index.js            # Library entry point
+│   └── index.d.ts          # Library type declarations
+├── boat/                   # Compiled API tester module
+├── rules/                  # Agent rule files (markdown)
+└── assets/sample-files/    # Sample files for testing
+```
+
+### Type Declarations
+
+`declaration: true` doesn't work directly on this codebase: the Researcher agent is built from generic mixin factories (`WithDeepAnalysis(Base)` etc.) that return anonymous classes with `private` members, which TypeScript can't serialize into a `.d.ts` (`TS4094`). Rather than refactor those hot-path agents, `scripts/build-types.ts` generates declarations from a transformed copy of the source:
+
+1. Copies `src/` into a temporary tree, rewriting every `private`/`protected` class modifier to `public` at its exact AST position (members are preserved; only the visibility keyword changes, which removes `TS4094`).
+2. Runs `tsc --emitDeclarationOnly` over the copy into `dist/src/`.
+3. Rewrites `.ts`/`.tsx` module specifiers to `.js` in the emitted `.d.ts` so they resolve for consumers.
+4. Deletes the temporary tree.
+
+The transform touches only the intermediate copy — the shipped `.js` keeps its real `private`/`protected` visibility. The `.d.ts` types are exact (unions, option shapes, and return types are all preserved), so Node.js/TypeScript consumers get full type-checking. Bun consumers resolve the TypeScript source directly via the `bun` export condition.
+
+Key `package.json` fields:
+
+```json
+{
+  "bin": {
+    "explorbot": "./dist/bin/explorbot-cli.js",
+    "prima": "./dist/boat/prima/bin/prima-cli.js"
+  },
+  "main": "dist/src/index.js",
+  "types": "dist/src/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/src/index.d.ts",
+      "bun": "./src/index.ts",
+      "import": "./dist/src/index.js"
+    }
+  },
+  "files": [
+    "dist/",
+    "src/**/*.ts",
+    "src/**/*.tsx",
+    "bin/**/*.ts",
+    "boat/api-tester/src/**/*.ts",
+    "boat/doc-collector/src/**/*.ts",
+    "boat/doc-collector/bin/**/*.ts",
+    "boat/doc-collector/package.json",
+    "boat/prima/src/**/*.ts",
+    "boat/prima/bin/**/*.ts",
+    "boat/prima/package.json",
+    "rules/",
+    "assets/sample-files/"
+  ],
+  "engines": { "node": ">=24.0.0" }
+}
+```
+
+The package ships two commands: `explorbot`, and `prima` for the [prima boat](../reference/commands.md#prima-boat), so `npx -p explorbot prima <command>` works without a separate install.
+
+Explorbot is both a CLI (`bin`) and a library (`exports`). The `.` entry point is `src/index.ts`, a side-effect-free barrel that re-exports the public API (`ExplorBot`, `Plan`, `Test`, and their types). The `exports` conditions are ordered so each consumer gets the right entry: `types` (the emitted `.d.ts`) for type-checking, `bun` (the TypeScript source) under Bun, and `import` (the compiled JS) under Node.js. This is why the source `src/**` files ship alongside `dist/`.
+
+## Building Locally
+
+```bash
+# Build the npm package
+bun run build:npm
+
+# Verify the CLIs work on Node.js
+node dist/bin/explorbot-cli.js --help
+node dist/boat/prima/bin/prima-cli.js --help
+
+# Check what would be published
+npm pack --dry-run
+```
+
+## Publishing
+
+GitHub Actions publishes automatically (see below), but you can also publish manually:
+
+```bash
+# Bump version
+npm version patch  # or minor, major
+
+# Build and publish (prepublishOnly runs build:npm automatically)
+npm publish
+```
+
+## Known Limitations
+
+- **Type declarations are transform-generated** - Declarations come from a transformed copy of the source (see [Type Declarations](#type-declarations)), not from `tsc --declaration` directly, because the mixin-based agents can't emit declarations as written. The published `.d.ts` types are exact; the workaround only concerns how they're produced.
+
+## CI/CD
+
+The `test.yml` workflow verifies the npm build on every push. On Node.js 24 it runs `bun run build:npm`, then the Node smoke tests: `node --test tests/node/*.mjs`. The `publish.yml` workflow additionally checks `node dist/bin/explorbot-cli.js --help` before publishing.
+
+The `publish.yml` workflow publishes to npm when you push a version tag (`v*` or a bare `1.2.3`-style tag). It overwrites the package version from the tag; tags containing `beta`, `alpha`, `pre`, or `rc` publish to the `beta` dist-tag instead of `latest`.

package/docs/contributing/observability.md

@@ -0,0 +1,227 @@
+# Observability & Debugging
+
+Explorbot integrates with [Langfuse](https://langfuse.com) for tracing. Use it to see what happened during a session: what data each agent received, which tools it called, and how it decided.
+
+![Langfuse Trace View](../assets/langfuse-trace.png)
+
+## Why Observability?
+
+Without traces, you only see the final test result and basic logs. When Explorbot runs on its own, you need to see:
+
+- What prompts went to the AI.
+- What tools were called, and with what parameters.
+- Token usage and cost per session.
+- How long each operation took.
+- Errors and retries.
+
+Use this data to:
+
+- Debug failed tests. See what the AI saw and decided.
+- Create Knowledge fixes. Find the context that was missing.
+- Tune prompts and agent performance.
+- Understand why a test passed or failed.
+- Export sessions for the `/explorbot-debug` skill.
+
+## Setting Up Langfuse
+
+### 1. Create a Langfuse Account
+
+Sign up at [langfuse.com](https://langfuse.com) (free tier available) or self-host.
+
+### 2. Get Your API Keys
+
+From your Langfuse project settings, copy:
+- **Public Key**
+- **Secret Key**
+
+### 3. Configure Explorbot
+
+Add credentials to your `.env` file:
+
+```bash
+LANGFUSE_PUBLIC_KEY=pk-lf-xxxxxxxx
+LANGFUSE_SECRET_KEY=sk-lf-xxxxxxxx
+```
+
+Or configure in `explorbot.config.js`:
+
+```javascript
+export default {
+  ai: {
+    model: groq('gpt-oss-20b'),
+    langfuse: {
+      enabled: true,
+      publicKey: process.env.LANGFUSE_PUBLIC_KEY,
+      secretKey: process.env.LANGFUSE_SECRET_KEY,
+      baseUrl: 'https://cloud.langfuse.com', // or your self-hosted URL
+    },
+  },
+};
+```
+
+### 4. Run Explorbot
+
+Once configured, Explorbot traces every AI call. No code changes needed.
+
+## What Gets Traced
+
+Explorbot uses the [Vercel AI SDK integration](https://langfuse.com/docs/integrations/vercel-ai-sdk) with Langfuse. Each session captures:
+
+| Trace | Description |
+|-------|-------------|
+| `test: <scenario>` | Full test execution cycle |
+| `researcher: <url>` | Page analysis by Researcher agent |
+| `planner: <url>` | Test scenario generation |
+| `driller: <url>` | Component drilling |
+| `ai.generateText` | Text generation calls |
+| `ai.generateObject` | Structured output calls |
+| `codeceptjs.step` | Individual browser actions |
+
+Navigator has no span of its own — its AI calls appear as `ai.*` spans under the parent trace.
+
+## Analyzing Sessions
+
+### In Langfuse Dashboard
+
+1. Open your Langfuse project
+2. Find the session by timestamp or name
+3. Click to see the full trace tree
+4. Inspect individual spans for:
+   - Input prompts
+   - Output responses
+   - Token counts
+   - Duration
+   - Errors
+
+## Debugging with Claude Code
+
+Explorbot includes a Claude Code skill that analyzes failed sessions.
+
+### Using the Debug Skill
+
+Find the failed `test: <scenario>` trace in Langfuse and copy its trace ID. Then, in Claude Code, run:
+
+```
+/explorbot-debug
+```
+
+Give it the trace ID — the skill fetches the trace with all its observations via `bun .claude/skills/explorbot-debug/langfuse-export.ts <trace-id>`. The trace holds the full context: prompts, tool calls, page states, and AI decisions. Without a trace ID, the skill analyzes `output/explorbot.log` instead.
+
+### What the Skill Analyzes
+
+The skill looks for three failure patterns:
+
+| Pattern | Symptoms | Solution |
+|---------|----------|----------|
+| **Missing Context** | Wrong element clicked, didn't understand UI | Add Knowledge file with disambiguation rules |
+| **Wrong Prompts** | Incorrect assumptions, wrong flow | Add Knowledge with business context |
+| **Wrong Tool Choice** | Used click when form needed, typing issues | Add Knowledge with CodeceptJS code examples |
+
+### How It Helps
+
+1. Extracts key data from the trace with jq:
+   - Failed tool calls
+   - URLs visited
+   - Prompts sent to the AI
+
+2. Identifies the root cause of failures.
+
+3. Suggests Knowledge files to fix the issue:
+   ```markdown
+   ---
+   url: /admin/users/*
+   ---
+
+   ## User Table
+   Each row has same buttons. Use container:
+   I.click('Delete', '[data-user-id="123"]')
+   ```
+
+4. Can try interactions with browser tools, if available, and record working CodeceptJS code.
+
+### Example Workflow
+
+```bash
+# 1. Test fails
+./bin/explorbot-cli.ts explore /admin/users
+
+# 2. Open Langfuse, find the failed "test: ..." trace, copy its trace ID
+
+# 3. In Claude Code:
+/explorbot-debug
+# Provide the trace ID
+
+# 4. Skill analyzes and suggests Knowledge fix
+# 5. Create knowledge file
+./bin/explorbot-cli.ts learn "/admin/users/*" "Use container context for table actions"
+
+# 6. Re-run test
+```
+
+## Debugging Tips
+
+### Enable Verbose Logging
+
+```bash
+./bin/explorbot-cli.ts explore /admin/users --verbose
+```
+
+Or set the environment variable:
+
+```bash
+DEBUG=explorbot:* ./bin/explorbot-cli.ts explore /admin/users
+```
+
+This shows detailed logs:
+
+- Prompts sent to the AI
+- Tool calls and results
+- State transitions
+
+### Specific Debug Namespaces
+
+```bash
+# AI provider calls only
+DEBUG=explorbot:provider ./bin/explorbot-cli.ts explore /admin/users
+
+# Navigator agent only
+DEBUG=explorbot:navigator ./bin/explorbot-cli.ts explore /admin/users
+
+# Multiple namespaces
+DEBUG=explorbot:tester,explorbot:navigator ./bin/explorbot-cli.ts explore /admin/users
+```
+
+### Available Namespaces
+
+| Namespace | What it shows |
+|-----------|---------------|
+| `explorbot:provider` | AI API calls, responses |
+| `explorbot:provider:out` | Outgoing prompts |
+| `explorbot:provider:in` | Incoming responses |
+| `explorbot:navigator` | Navigation decisions |
+| `explorbot:researcher` | Page analysis |
+| `explorbot:planner` | Test scenario generation |
+| `explorbot:tester` | Test execution |
+| `explorbot:historian` | Experience saving |
+| `explorbot:quartermaster` | A11y analysis |
+
+## Cost Tracking
+
+Langfuse tracks token usage per call. Use it to:
+
+- Monitor cost across sessions
+- Compare model efficiency
+- Find expensive operations
+- Tune prompts to reduce tokens
+
+## Self-Hosting Langfuse
+
+For privacy or compliance, you can self-host Langfuse. Langfuse v3 requires docker compose with Postgres, ClickHouse, and Redis — follow the [self-hosting docs](https://langfuse.com/self-hosting).
+
+Then set `baseUrl` in your config:
+
+```javascript
+langfuse: {
+  baseUrl: 'http://localhost:3000',
+}
+```

package/docs/contributing/regression-tests.md

@@ -0,0 +1,103 @@
+# Self-Regression Tests
+
+The regression harness runs Explorbot **end-to-end with real AI models** against a controlled local fixture app, then asserts on the artifacts Explorbot produces (research, plans, test results). Unlike the integration tests in `tests/integration/` — which mock the LLM — these runs use a live provider, so they are nondeterministic and gated behind owner approval in CI.
+
+The harness lives in `tests/regression/` and is driven by `regression:*` commands in the repo `Bunoshfile.js`.
+
+## Scenarios
+
+### Scenario A — fresh explore (`regression:basic`)
+
+Runs `explorbot explore /issues --headless` against the Trackly fixture and asserts:
+
+- **Login evidence** — a plan targets a post-login route (`/issues`, `/settings`) and at least one research file describes a post-login page (proves the agent applied the seeded credentials and got past the gate).
+- **Research** — at least one research file with a heading, a UI-map table, and enough domain keywords.
+- **Scenarios identified** — the plan holds at least `MIN_PLANNED_TESTS` tests covering at least `MIN_FEATURE_GROUPS` feature areas.
+- **Tests passed** — the `Results: N passed, N failed` stdout line shows no failures and at least `MIN_PASSED` passing tests, cross-checked against the reporter markdown.
+
+Credentials are supplied through a seeded knowledge file (`tests/regression/seeds/knowledge/login.md`) matched to `/login`, so this scenario also exercises knowledge rules.
+
+### Scenario B — experience reuse (`regression:experience`)
+
+Proves Explorbot reuses prior context to pass a test it cannot pass cold. The Trackly Archive vault (`/vault`) is unlocked only by an access code that exists nowhere in the DOM — only in the server and in the seed files.
+
+- **Control run** (always, once) — runs the seed plan with **empty** knowledge and experience dirs. It must NOT pass. A fully-passing control means the vault gate is broken and fails the harness.
+- **Seeded run** (retried) — runs the same plan with the seed knowledge and experience dirs. It must pass.
+
+The plan (`tests/regression/seeds/vault-plan.md`) describes *what* to do but never contains the code. The seeded context supplies it two ways: `seeds/knowledge/vault.md` carries the access **code** (data), and `seeds/experience/vault.md` carries the interaction **recipe** (fill field, click Unlock). Both are prior seeded context; the control has neither and fails.
+
+> The code is carried in a knowledge file, not only experience, on purpose. Knowledge is Explorbot's prominent, deterministic channel for page-specific data — the same mechanism scenario A uses for login credentials. Empirically, cheap models followed an experience-only code recipe only occasionally, so an experience-only gate was a coin-flip; knowledge makes the gate reliable while the experience file still exercises recipe reuse.
+
+## The fixture app — "Trackly"
+
+A self-contained issue tracker served in-process by `tests/regression/fixture/server.ts` (`Bun.serve`, ephemeral port, fresh in-memory store per attempt). No external network, no database.
+
+| Route | Auth | Contents |
+|-------|------|----------|
+| `/login` | no | credentials form (always native HTML) |
+| `/issues` | yes | issue list, search, status filter, label menu (start page) |
+| `/issues/new` | yes | create form: title, description, priority, labels + assignees multiselects |
+| `/issues/:id` | yes | detail, change-status menu, comment form, delete modal |
+| `/settings` | yes | Profile/Preferences tabs + an activity iframe |
+| `/vault` | no | scenario B access-code gate |
+| `/api/*` | cookie | REST API over the same store, `+ /api/openapi.json` |
+
+The REST API mirrors the store and is served for a future API-testing scenario; the current scenarios do not exercise it.
+
+### Component variants (ARIA on/off)
+
+Every widget (button, text field, select, multiselect, dropdown menu, modal, tabs) renders in one of three variants, selectable per run:
+
+- `native` — semantic HTML (`<button>`, `<select multiple>`, `<dialog>`, `<details>`).
+- `aria` — custom `<div>` widgets with correct roles (`role="combobox"`, `aria-modal`, `role="menu"`, …).
+- `plain` — bare `<div>` widgets with no roles or labels (the hostile case).
+- `random` — a seeded RNG picks a variant per widget; the same `--seed` reproduces identical markup.
+
+Pass `--variant` and `--seed` to any scenario. The login page is always native so variant runs don't fail at the door.
+
+## Running locally
+
+```bash
+# No-AI sanity check: fixture routes, auth, API, vault gate, seed parsing, variant rendering
+bunx bunosh regression:smoke
+
+# Serve the fixture for manual inspection (append ?variant=plain&seed=7 to any page)
+bunx bunosh regression:serve
+
+# Real-AI runs (need a provider key)
+export OPENROUTER_API_KEY=sk-...
+bunx bunosh regression:experience --retries 0   # cheapest AI path
+bunx bunosh regression:basic --retries 0
+bunx bunosh regression:all                       # both scenarios, default retries
+```
+
+Each attempt runs in an isolated throwaway directory under `tests/regression/.runs/` (gitignored). Inspect `tests/regression/.runs/<scenario>-<variant>-a<n>/output/` for the research, plans, and reporter markdown a run produced, and `tests/regression/.runs/report.md` for the summary.
+
+### Variant matrix
+
+`regression:variants` runs a scenario across variants and reports the outcome per variant **without gating** — a `plain`-variant failure is data about model robustness, not a regression. Only a crashed or timed-out run fails it.
+
+```bash
+bunx bunosh regression:variants --scenario basic --variants native,aria,plain
+```
+
+## CI and the approval gate
+
+`.github/workflows/regression.yml` runs on `pull_request` and `workflow_dispatch`. The job uses the `regression` GitHub environment, which holds `OPENROUTER_API_KEY` behind a **required reviewer**. Every run — including fork PRs — pauses at the environment gate before the secret is exposed. The job runs the PR's own code, so the reviewer must read the PR diff (workflow, Bunoshfile, fixture, lib) before approving; that review is the security boundary. The workflow never uses `pull_request_target`.
+
+The report is posted as a sticky PR comment (on `pull_request`) or a `Regression Reports` GitHub Discussion (on `workflow_dispatch`). Fork PRs get a read-only token, so comment posting may fail; the report is also written to the job step summary and uploaded as the `regression-runs` artifact.
+
+The report also embeds the **Session Analysis** from the fresh-explore run — Explorbot's Analyst agent writes a prose summary of what works, defects, UX issues, and execution issues, which the harness reads from `output/reports/<label>.md` and appends to the comment. The Analyst is enabled in the config template (`ai.agents.analyst.enabled: true`); it adds one AI call per basic run.
+
+## Tuning
+
+- **Thresholds** (`MIN_PLANNED_TESTS`, `MIN_PASSED`, `MIN_KEYWORD_HITS`, `MIN_FEATURE_GROUPS`) live at the top of `tests/regression/lib/assertions.ts`.
+- **Models** live in `tests/regression/fixture/explorbot.config.js`. If cheap models are too flaky, promote a stronger model to `ai.model` or lower `MIN_PASSED`.
+- **Run size** — adjust `--max-tests` in `runBasicScenario` and the per-CLI `timeout` values in `Bunoshfile.js`.
+
+## Owner prerequisites (one-time)
+
+1. Settings → Environments → create `regression`; add the repo owner as a **Required reviewer**; deployment branch policy "No restriction".
+2. Add the environment secret `OPENROUTER_API_KEY` (a dedicated key with a low spend limit).
+3. Create a Discussions category named exactly `Regression Reports`.
+4. Ensure the OpenRouter account has credit for the models in the config template.