@asrunjs/asrun 0.2.0 → 0.5.3

package/README.md

@@ -1,410 +1,221 @@
-# AsRun
-
-Automated tests against production and staging for multiple profiles. Humans write test cases in plain language; an AI
-agent turns them into Playwright specs; the specs run locally, once per profile, and produce videos, screenshots and
-traces for review.
-
-Profiles and environments: see `profiles.yaml`. A profile is one environment of the system under test (a URL, an
-account label, tags and an environment name), and each one becomes one Playwright project.
-
-## Profiles and environments
-Every entry in `profiles.yaml` is one environment: `id`, `name`, `env`, `baseURL`, optional `account` and `tags`, and
-`email`. One profile = one Playwright project named after its id (for example `example-staging`).
-- **There is no default environment for a run.** `pnpm asrun` and every AI command need `--env <env>`, which selects
-  the profiles whose `env` matches (the Run tests page needs one radio button picked). `pnpm signin <profile-id>` and
-  `pnpm explore <profile-id>` need no `--env`: the profile already names its environment.
-- **Separate logins.** Sessions are saved per profile in `.auth/<profile-id>.json`; log in with
-  `pnpm signin <profile-id>` (a human, including 2FA).
-- **Separate baselines and reports.** Screenshot baselines are stored per project (`...-example-staging-darwin.png`).
-  A run's folder, report title and summary carry the environment (`reports/2026-10-28T07-30-22-staging`); the web UI
-  shows PRODUCTION in red and STAGING in blue everywhere, and prints `ENV: PRODUCTION` / `ENV: STAGING` at the start of a CLI run.
-- **Tags.** `tags: [us, core]` groups profiles; `pnpm asrun --env staging --tag us` runs only the matching ones.
-- **Email.** `email: true` says a real mailbox is connected and tests may send email. It defaults to `false`, is set to
-  `false` for new profiles, and is rejected for `production`.
-- **Expected values are not here.** They live in each case document (`## Values`); `profiles.yaml` holds connection
-  settings only, so adding a profile never touches a case and two profiles can never fight over one case's values.
-
-## Installing asrun in a project
-
-`asrun` is a normal devDependency: it brings the Playwright fixtures and config, the case/spec tooling, the agent
-prompts and the web UI. A project commits only its own test content.
-
-```bash
-pnpm dlx @asrunjs/asrun init            # scaffolding only: writes .env, configs, AGENTS.md, commands, the build-script approval
-pnpm add -D @asrunjs/asrun              # or: pnpm add -D github:wizcas/asrun#v0.1.0
-pnpm exec asrun init --update  # picks up the installed version
-pnpm go                        # install + Chromium + doctor + web UI
-```
-
-Running `init` before `add` matters on pnpm 12: it writes the `allowBuilds` entry that pnpm needs before it will run
-the install scripts of `esbuild`/`ffmpeg-static` (without it, `pnpm add` exits with `ERR_PNPM_IGNORED_BUILDS`).
-`pnpm dlx @asrunjs/asrun init --install` does both in one step (the spec is optional:
-`--install github:wizcas/asrun#v0.1.0`).
-
-`pnpm asrun init` is safe to run again: it never overwrites a file you own.
-
-```bash
-pnpm asrun init --update   # refreshes only the generated files (playwright.config.ts, seed.spec.ts, commands)
-pnpm asrun doctor          # checks profiles, the tsconfig alias, generated files, the installed version, skill overrides
-```
-
-The project commits `profiles.yaml`, `cases/`, `tests/` (with their `*-snapshots/`), `AGENTS.md`, `.claude/`,
-`.mcp.json`, `opencode.jsonc` and the generated files. It never commits `.auth/`, `reports/`, `.tmp/`, `.explore/`,
-`.env` or `node_modules/`. See "Layout" below.
-
-**Which workspace a command runs against** comes from `.env` (`ASRUN_ROOT`, relative to that file; gitignored and
-written by `init`), then from the nearest folder with `profiles.yaml`/`.asrun.json`. A real environment variable wins
-over `.env`, so a CI job can point a checkout somewhere else without editing files.
-
-Update:
-
-```bash
-pnpm add -D @asrunjs/asrun@latest     # or: pnpm add -D github:wizcas/asrun#v0.2.0
-pnpm exec asrun init --update
-pnpm asrun doctor
-```
-
-There is no folder to move: the framework lives in `node_modules/@asrunjs/asrun`, which the package manager owns. To change
-the framework itself, `pnpm patch asrun` (a committed patch) or point the dependency at your fork's tag.
-
-`templates/` in the package holds what `init` writes. `AGENTS.md` and the skills stay in the package as the single
-source: a project narrows a rule in its own `AGENTS.md` (by naming its id, `F1`..`F10`; `[non-negotiable]` rules cannot
-be narrowed) and overrides a skill with `.claude/skills/<skill>.md`, which `doctor` lists.
-
-## Quick start in a project
-
-```bash
-node framework/scripts/go.mjs   # the very first time (no dependencies yet)
-pnpm go                         # every time after that
-```
-That is all. It checks Node (22+), lists the framework in `pnpm-workspace.yaml`, installs the dependencies and the
-browser (`pnpm install` + Chromium), writes `.env` and the workspace files, runs `doctor`, starts the web UI and opens
-http://127.0.0.1:4173. Run it again any day: the install step then takes seconds, and if the UI is already running it
-just opens it. The **?** icon in the top bar of the UI opens the step-by-step "How to start" page (install, log in,
-make cases, explore, generate specs, run, reports). `pnpm go --no-open` does not open the browser.
-
-The tab title shows where you are (`AsRun | Cases | documents/landing-page`, `AsRun | Job Log`). When it is your
-turn (a login to finish, a step that an attended exploration hands to you in the browser window, or answers that an
-AI run needs), the title gets a count in front (`(1) AsRun | ...`), the job shows a "Your turn" badge, and the bell
-icon in the top bar, once switched on, shows a browser notification while the UI tab is in the background. Clicking the
-notification opens that job.
-
-You need Node 22+ and pnpm. The AI steps (make cases, explore, generate) also need Claude Code or OpenCode; `pnpm go`
-warns if neither is found, everything else works without them.
-`ffmpeg` is also needed: it joins the videos of one test (a new browser tab has its own recording) into a single video
-for the report. `pnpm go` tries to install it (Homebrew on macOS, winget on Windows; on Linux run
-`sudo apt install ffmpeg`). If it is missing, the optional `ffmpeg-static` package is used instead; if neither works,
-the report shows the videos separately.
-Then check `profiles.yaml`: the profile ids, names, environments and addresses.
-
-## Every day
-
-| I want to... | Command |
-|---|---|
-| Check the workspace (profiles, tsconfig alias, generated files, skill overrides) | `pnpm asrun doctor` |
-| Create or refresh the generated workspace files | `pnpm asrun init`, `pnpm asrun init --update` |
-| See which profiles are logged in | `pnpm status` (all environments) or `pnpm status --env staging` |
-| Log in a profile (human, with 2FA) | `pnpm signin <profile-id>` |
-| Run everything in one environment | `pnpm asrun --env staging` |
-| Run one profile | `pnpm asrun --env production --profile example-production` |
-| Run all profiles except some (e.g. not logged in) | `pnpm asrun --env production --skip-profile example-production,example-staging` |
-| Run only the profiles that carry a tag | `pnpm asrun --env staging --tag us` |
-| Run only the profiles that are logged in | `pnpm asrun --env production --logged-in` |
-| Read what the last report says about a case | `pnpm heal:evidence --case <case-id> --env staging` |
-| Check the specs for guessed targets, sleeps and weak download checks | `pnpm spec:lint [case-id]` |
-| Render a PDF to images and print its text (to look at it) | `pnpm pdf:render <file.pdf> --out .explore/targets/<case-id>/<name> [--pages all] [--corner top-left]` |
-| List screenshot baselines next to what each must show | `pnpm baselines:review [case-id] [--new]` |
-| Overwrite the baselines of some cases | `pnpm asrun --env staging --grep <case-id> --update-baselines <case-id>` |
-| Run one case | `pnpm asrun --env production --grep hub-landing` |
-| Run cases of one suite | `pnpm asrun --env production --suite smoke` (several: `--suite smoke,regression`; `--priority` is the old name) |
-| Leave out the cases that write data | `pnpm asrun --env production --read-only` |
-| List test data a case could not remove | `pnpm cleanup --env <env>` (`--forget` after deleting it by hand) |
-| Watch a run step by step | `pnpm asrun --env staging --ui` |
-| Show the current test step as text on the videos | `pnpm asrun --env production --step-text` (also `=1\|true\|yes\|on`; off by default) |
-| Open the last report (videos, screenshots, traces) | `pnpm report` |
-| Look around a page as a profile (files you download in that window are saved to `~/Downloads`; the path is printed in the terminal / job log) | `pnpm explore <profile-id> /some/path/` |
-
-### Local web UI
-```bash
-pnpm go          # install + build + start + open the browser (use this the first time)
-pnpm ui          # builds the web app and serves http://127.0.0.1:4173
-pnpm ui:dev      # development: Vite on :5173 with hot reload, API on :4173
-```
-One page for the daily work (it only listens on `127.0.0.1`):
-
-| Page | What it does |
-|---|---|
-| Profiles | Login state of every profile, one row each (same check as `pnpm status`). "Log in" starts `pnpm signin <profile-id>`; log in in the browser window that opens, then click "I've logged in, save the session". "Visit" (shown for logged-in profiles) opens a logged-in browser window at `/app/home` (`pnpm explore <profile-id> /app/home`). While that window is open the button turns into a red "Close browser", which kills the browser process. Files downloaded in that window are saved to `~/Downloads` (an existing file is never overwritten, a ` (1)` is added) and the saved path is printed in the job log. "Add a new profile" has a form (environment, URL, account label, tags) that appends an entry to `profiles.yaml`, plus a guide for what to do next (log in, update the cases, a copyable AI prompt). |
-| Cases | List of `cases/**/*.md`, grouped by folder (collapsible), source editor with a rendered preview (both highlight what needs a human: `TODO` in any spelling such as `todo` / `TO-DO` / `to-do` (red for `🚫 TODO [blocking]`, gold otherwise), `🔍 to be confirmed by QA` / `👁️ not observed` / `TBC` / `TBD` in blue, and your `✍️ Answer:` lines in green; the emoji in front is part of the highlighted marker; hover a highlighted word for a short tooltip on what it means and how to resolve it, and click the small X after it to delete the marker, or the whole question block for a `TODO`, in one step, in the editor and in the preview (Undo brings it back; the file is only saved when you save). An `Answer:` gets its X once you typed something, and it clears what you typed), the editor and the preview scroll together, the preview block that holds the caret is shaded, and a click in the preview puts the caret on that line of the editor; `Ctrl/Cmd+S` to save, "New case" has an optional folder field and runs `pnpm case:new`. The **Spec** view is editable too (`Ctrl/Cmd+S` saves; the save is refused when the file changed on disk meanwhile): **Open in editor** opens the spec in the default application of your system for `.spec.ts` files, and **Copy path** copies its absolute path, to paste into a terminal editor such as nvim. Click the file name (or the pencil icon next to it) to rename the case or move it to another folder: the spec and the baseline images move with it and the spec is rewritten to match. **Delete** (type the id to confirm, or click the feather icon to fill it in) removes the case, its spec and its baseline images; both dialogs list exactly what they touch first, and are refused while an AI or test run is in progress. The editor toolbar has a command picker (explore / generate / heal, explore by default), an **Environment** picker (required for every command, no default), an **Attended** box (ticked by default, only for explore), a **Run AI** button (for **Heal spec** it opens a dialog first, see below) and icon buttons to copy the shell command and open the AI guide, for the one-shot AI command of the open case. The Claude / OpenCode switch (Claude by default, remembered in the browser) is in the top bar and applies to every AI command. **From memo** opens a text box for raw test notes: **Make cases** saves them to `.tmp/memos/` and runs `/make-case` one-shot (no questions: defaults for gaps, a `TODO` block for unknown values, no page exploration, existing cases are never overwritten); **Copy /make-case** copies the memo for the full interactive dialogue. |
-| Run tests | Form for `pnpm asrun`: environment (required, no default), profiles, suite, read-only, case, step text. The **Cases to run** list shows what the filters select, with an **Update baselines** box per case (`--update-baselines`, unticked by default; asks before it overwrites and shows the new images next to their target cards when the run ends). Asks for confirmation when write cases would run in production. |
-| Reports | One entry per run, each with a PRODUCTION / STAGING badge (runs from before environments existed are shown as PRODUCTION). **Results** lists every case x profile with the failure reason on one line (failed first); selecting one shows a red "Why it failed" block (failed step + error message) above the screenshot, video and a folded "Technical details" (call log, stack trace). **Full Playwright report** embeds the standard HTML report (videos and traces work). The latest run opens first. Runs made before summaries existed only have the full report. |
-| Help (the **?** icon in the top bar) | "How to start": seven short steps with screenshots, from `pnpm go` to the reports. The screenshots are in `ui/web/public/help/` (retake them when a page changes a lot). |
-| AI guide | Copyable `/make-case`, `/explore-case`, `/gen-spec`, `/heal-spec` commands for Claude Code and OpenCode. |
-
-The "Jobs" bar at the bottom (opening it shrinks the page above instead of covering it) shows every command started from the UI with live stdout/stderr and its exit
-code; a job can be cancelled there. Only these fixed actions can run (`signin <profile>`, `visit <profile>` and `run` with
-checked options) the three AI commands for an existing case and `/make-case` for a memo (`claude -p` / `opencode run`); there is no free shell. Set `ASRUN_UI_PORT` to change the port.
-
-### Checking what a test actually did
-Every run records a video and a trace per test. A new browser tab gets its own recording; the **Reports** page joins
-them in order into one video per test (`reports/<run>/videos/`, needs ffmpeg, see Quick start). The full Playwright report
-still lists the recordings separately. Add `--step-text`
-to `pnpm asrun` to overlay the current step's title on the videos; without it the videos are plain.
-In `pnpm report`, open the test: the steps are named after the steps in the case document. The trace shows each action with
-the page before and after it, plus every network request. `pnpm asrun` also prints the file paths of the
-videos and traces when it finishes; open a trace directly with `pnpm exec playwright show-trace <trace.zip>`.
-Recordings contain real production pages, so do not share them.
-
-### Logging in
-`pnpm signin <profile-id>` opens a normal browser. Log in yourself, including 2FA, wait for the logged-in
-page, then press Enter in the terminal. The session is saved to `.auth/<profile-id>.json` and verified.
-Nothing about 2FA is automated.
-
-Login itself is not tested. Before every run, `pnpm asrun` checks the session of each profile of the chosen
-environment. If a profile has no saved session, or the session expired (the check page redirects to `/login`), the
-tests of that profile fail right away (no retries) and the run prints:
-```
-Not logged in (tests for these profiles will FAIL):
-  example-production           expired  -> pnpm signin example-production
-Or skip them for this run:  pnpm asrun --env production --skip-profile example-production
-```
-Either log in (`pnpm signin <profile-id>`) or leave the profile out with `--skip-profile`. Profiles that are logged
-in are tested as usual. If a session expires in the middle of a run, the failed test is marked
-"session expired" in the output.
-
-Sessions are as sensitive as passwords: `.auth/` (one file per profile) is git-ignored, never share or commit it.
-
-## From a case document to a test (AI workflow)
-
-0. **(Optional) start from raw notes.** If you only have a memo or a list of things to check, run
-   `/make-case path/to/notes.md` (or paste the text after the command). The agent normalizes the text into English steps,
-   marks steps that write data (`[writes]`) or send email (`[email]`), keeps your original lines in a `## Source` table,
-   lists the gaps with a suggested default for each, and only then writes `cases/<id>.md`. It never writes specs and
-   never invents expected values (unknown ones become a default sentence with a `TODO` block under it). It never opens a
-   browser: what the pages really show is found in step 2. `pnpm case:check` validates the front matter of every case.
-1. **Write the case.** `pnpm case:new header-logo` creates `cases/header-logo.md` from a template; `pnpm case:new documents/header-logo` puts it in the
-   group folder `cases/documents/` (created on demand, one level only). The id of a case is its path without `.md`, its spec is
-   `tests/documents/header-logo.spec.ts`, and its expected values are in its own `## Values` block.
-   Fill in steps, what is expected for every profile, and the exact values the spec asserts under Values, one entry per
-   profile id (`default` applies to all). Plain language is fine for the checks. Write it in English; if it is in
-   another language, `/gen-spec` translates the case file to English first (and tells you).
-2. **(Optional) explore the case.** In OpenCode or Claude Code run `/explore-case header-logo production` (the case id is
-   required; the environment is optional: `production` or `staging`). This is the only step where a person and the
-   agent work together. The agent opens the real pages of the profiles that are logged in and only **reads** them: it
-   never creates, edits, publishes, shares or deletes anything, in staging or in production. It fixes wording in the
-   case file, writes what the pages show into it (marked `observed, 🔍 to be confirmed by QA`), and turns what it cannot
-   know into `🚫 TODO [blocking]` / `❓ TODO [non-blocking]` questions (every marker starts with its emoji). Every question is a block
-   under a sentence that is the agent's suggestion, written so that it can stand in the case as it is:
-   ```
-   - Clicking "Edit Draft" opens the document editor, in the same tab or in a new tab.
-     - ❓ TODO [non-blocking]: Same tab or new tab? (👁️ not observed)
-       - ✍️ Answer:
-   ```
-   It edits only the case file. **To accept the suggestion, delete the whole block** (the `TODO` line and the
-   `Answer:` line). **To give your own answer, write it after `Answer:`** (you may delete the rest of the block).
-   Then run the command again. It ends with `RESULT: ready` or `RESULT: needs-input`. In the web UI a run
-   that ends with `needs-input` shows an amber "Needs your input" badge and an answer box under its output: send your
-   answers (or "Use the suggested defaults", which accepts every suggestion) and the same agent conversation continues with the same permissions, folds
-   them into the case file and ends with a new `RESULT:`. A run can be continued once per turn, and only while the UI
-   server has not been restarted; after a restart, answer in the case file and run the command again.
-
-   **Attended exploration (`--attended`, or the "Attended" box on the Cases page, ticked by default).** Use it for a case that creates or
-   edits data, or needs a choice (an existing document, items). Stay at the computer: the browser window opens and the
-   agent waits while you do those steps. The window shows a small card in the bottom-left corner that says what to do (drag it by its title bar or minimize it if it is in the way), with **Done** (hand
-   control back) and **Cannot do this**; the agent also continues by itself when the finish condition it named shows up
-   on the page (for example a "Saved" message), and the card turns green when control is back. It waits up to 10
-   minutes per step. Delete the `[asrun]` items you created before you press Done on the last step. What you did is
-   saved in the case under `## Inputs` as a **general recipe**: positions instead of names ("select items #1 to #3"),
-   `[asrun]` names for created data, control labels but no selectors. Without `--attended` the agent never waits:
-   a step that needs a person becomes a `TODO [blocking]`.
-3. **Log in the profiles the case needs** (`pnpm status`, `pnpm signin ...`). Skip for `login: none` cases.
-4. **Generate.** In OpenCode or Claude Code run `/gen-spec header-logo staging` (the environment is required; there is
-   no default). The agent replays the case on the real pages with your saved sessions (it never writes data), writes `tests/header-logo.spec.ts`, fills the
-   case's `## Values` for the profiles it explored and checks the files (`pnpm typecheck`, `pnpm case:check`). It does not run the
-   spec and does not create screenshot baselines: running specs is a separate step (6). It does not stop to ask: what the
-   page answers is written into the case as observed (QA confirms it), and what is still unknown stays in the case as a
-   `TODO`. It ends with `RESULT: ready` or `RESULT: needs-input`. A case with a `[writes]` or `[input]` step that has no
-   recipe under `## Inputs` is not generated: it stops with `needs-input` and asks you to run step 2 with `--attended`.
-   If the spec already exists, the command leaves it alone and says so. Add `--force` (`/gen-spec header-logo staging
-   --force`) to regenerate it from the case and overwrite it (the case's `## Values` and baseline images are kept; the
-   report lists the ones that may be out of date). In the web UI, **Run** on a case that has a spec
-   asks first: "Overwrite the spec?". The server refuses to start `gen-spec` on an existing spec without that choice.
-5. **Review what the AI produced.** Check: the spec reads like your case, and the `## Values` block of the case has the
-   right values for each profile (you normally don't edit these by hand, but you can).
-6. **Run it:** `pnpm asrun --env <env> --grep <case-id>`. If the spec has visual checks, add `--update-snapshots`
-   the first time, then open the new images in `tests/<case>.spec.ts-snapshots/` and confirm each is the correct logo
-   for that profile. Baselines are the source of truth for pixel checks. Then run all profiles and `pnpm report`.
-
-When the product changes and a test fails, run `/heal-spec <case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`
-(or **Heal spec** in the Cases toolbar, or **Heal this failure** on a failed result on the Reports page). The agent
-decides whether the test needs updating or whether it found a real problem, and never edits an expectation just to
-make a failure disappear. How it works:
-- The environment is always required. Only the profiles that are logged in there are run (`pnpm asrun --logged-in`
-  leaves the others out instead of failing them).
-- The web UI first opens a dialog: the profiles to heal (every logged-in profile ticked) and an optional text box for what
-  you want healed or any evidence and clues. The text is stored in `.tmp/hints/` (git-ignored) and passed as `--hints`.
-  Clues guide the diagnosis; they never lower the bar for a product finding.
-- The agent reads the existing report first (`pnpm heal:evidence --case <id> --env <env>`: status, failed step, error,
-  call log and screenshots per profile, and whether the report is fresh or stale) and runs the case again only when
-  there is no report, it is stale, it does not explain the failure, or a clue has to be checked. After it edits the
-  spec it runs it once to confirm.
-
-### Targets, readiness and downloads (why generated specs can be trusted)
-Three rules keep an AI-written spec from passing or failing for the wrong reason. They hold in exploring, generating and
-healing (`AGENTS.md`, `<framework>/.claude/skills/exploring-cases/references/assertion-targets.md` and `readiness.md`).
-- **Targets.** A check on a logo, image, icon, file, PDF, screenshot or repeated text points to a card under `## Targets`
-  in the case (`[T1]`): what it is (and what it is not), where it is, how it is identified (never `.first()` or a bare
-  `img`), when it is ready, and a highlighted reference screenshot under `.explore/targets/`. When the agent is in doubt
-  it does not pick: it writes a `🚫 TODO [blocking]` that asks for the exact image, text, place or UI, with the
-  candidates listed. Answer it by editing the case (or, in an attended run, by outlining the target in the browser).
-- **Readiness.** After an action that loads something, the key requests must finish and the content must be visible before
-  anything is read. Specs use `actAndWait` (`@asrunjs/asrun/ready`); "missing" and "empty" checks need a positive ready signal.
-  Healing treats a "missing / empty / timeout" failure as a readiness problem first.
-- **Downloads.** `verifyDownload` (`@asrunjs/asrun/downloads`) checks that the download starts, finishes, the file exists and the
-  file is valid for its kind; any failure or timeout fails the whole step with a message that names the stage.
-- **PDFs are read like a person reads them.** `expectPdf` (`@asrunjs/asrun/pdf`) renders the pages to images (a logo region must not
-  be blank and must match the profile's baseline, 2% tolerance) and extracts the text (for example, no other profile's name).
-  Every page is checked unless the case limits the pages in strong words ("only", "exactly", "must"); "should",
-  "always", "usually" or no wording mean every page.
-  `pnpm pdf:render <file.pdf> --out .explore/targets/<case-id>/<name> --corner top-left` lets you (or the exploring agent)
-  look at the rendered pages.
-- **Checks you can run yourself:** `pnpm spec:lint` (fails on screenshots without a target, `.first()` on images, sleeps,
-  `networkidle`, raw download events, a `%PDF` check on its own), `pnpm baselines:review` (opens nothing; lists every
-  baseline with the card that says what it must show, so you can look at each new image), and `pnpm case:check`
-  (fails on an incomplete card, warns on an image check with no card).
-
-### Updating baselines on purpose
-If a profile really changed its logo, or a baseline is wrong: fix the spec first (the check must look at the element
-its target card names; a wrong locator just makes a new wrong image), then
-`pnpm asrun --env <env> --profile <id> --grep <case> --update-baselines <case-id>[,<case-id>]`, then look at the new
-image before keeping it (`pnpm baselines:review <case-id>`).
-- `--update-baselines` names the cases whose images are overwritten (`--update-snapshots=all`). They must be among the
-  cases the filters run. Playwright has one global switch, so they run as a second pass with their own report
-  (`<run-id>-baselines`); every other case is compared with its baselines as usual.
-- Without `--update-baselines`, `--update-snapshots` still works as before, but it rewrites every image of the run.
-- In the web UI, every case in **Cases to run** has an **Update baselines** box (nothing is ticked by default). A
-  confirm dialog follows, and when the run ends the job output shows each rewritten image next to the target card that
-  says what it must show.
-- Not part of exploring, generating or healing: an agent never updates a baseline.
-
-### Suites
-Each case lists its suites in the front matter: `suites: [smoke, regression]` (any lowercase words). A case runs in
-every suite it lists. The generated spec tags itself `@<suite>` for each one, which:
-- shows as badges next to the test in the terminal and in `pnpm report`, and can be typed into the report's
-  search box (`@smoke`) to filter there;
-- drives `pnpm asrun --suite <name>`, which can be combined with `--profile` and `--grep`. A case that is in two
-  of the chosen suites still runs once.
-
-An unknown suite is rejected with the list of suites the cases use. The old `priority: <level>` is read as
-`suites: [<level>]`.
-
-### Other front matter
-- `writes: [staging]` or `writes: [staging, production]` (omit it for a read-only case): the environments in which the
-  case may create or change data of the test accounts (and cleans it up). It then needs
-  `writes-note: <what it creates or changes>`. In an environment that is not listed, the case is **skipped with a
-  notice**, not failed. The old `writes: test-data` is refused with this hint.
-- `envs: [production, staging]` (optional): the case runs only in these environments; in others it is skipped. Every
-  environment in `writes` must also be in `envs`.
-
-Cases that write data get a red **WRITES** badge in the web UI and the report, and an `@writes` tag.
-
-## What a case may and may not do
-Tests run against production and staging. By default a case is read-only: look, read, compare. No creating,
-editing, deleting, submitting or accepting anything.
-
-A case may write only in the environments listed in its `writes` front matter (elsewhere it is skipped with a notice),
-and only data of the test account (named with the prefix `[asrun]`), which it must clean up.
-**AI agents never write data while they explore or generate**, in either environment: a person does those steps, in
-the browser, in an attended exploration (step 2 above).
-- **Staging:** no confirmation needed.
-- **Production (and any environment that is not staging)**, for a case that lists it under `writes`: a person confirms every run. `pnpm asrun --env production` lists
-  the write cases and asks you to type `production`. The web UI shows a confirm dialog. Unattended runs use
-  `--allow-writes=production`; AI agents never do. `--read-only` leaves the write cases out. The permission is also
-  enforced inside the test itself, so running `playwright test` directly cannot bypass it.
-- **Cleanup:** data is named `[asrun] ...`, registered with the `records` fixture and removed after the test, also
-  after a failure. Whatever could not be removed is shown in red in the report and listed by
-  `pnpm cleanup --env <env>`; delete it by hand, then `pnpm cleanup --env <env> --forget`.
-- **No retries, one at a time:** write cases are never retried, and run one at a time per profile.
-- **Email:** never sent from production. In other environments only if the profile has `email: true` in `profiles.yaml`.
-
-## Author
-Created by Zoro Chen.
-
-## Layout
-
-**In a project that uses the framework** (everything here is committed except the ignored folders):
-
-```
-profiles.yaml      one entry per environment: URL, account label, tags, environment (no expected values)
-cases/             human-written cases (+ _template.md); optional group folders: cases/<group>/<name>.md
-tests/             generated specs (same folders as cases/), and *-snapshots/ baseline images per profile
-tsconfig.json      plain compiler options: specs import the framework by package name (`@asrunjs/asrun/fixtures`)
-playwright.config.ts   re-exports node_modules/@asrunjs/asrun/dist/playwright.config.js
-AGENTS.md          the project rules: they point at node_modules/@asrunjs/asrun/AGENTS.md and narrow it (CLAUDE.md imports AGENTS.md)
-.claude/commands/  /make-case, /explore-case, /gen-spec, /heal-spec: thin shells that load the framework skill
-.claude/skills/    optional project overrides of a skill: <skill>.md
-.mcp.json, opencode.jsonc   the playwright-test MCP server, for Claude Code and OpenCode
-.env               ASRUN_ROOT (which workspace the commands use) and local settings (git-ignored)
-node_modules/@asrunjs/asrun/   the framework (installed dependency; never edited)
-.auth/             saved logins, one per profile (git-ignored)
-reports/           one folder per run; reports/latest points to the newest (git-ignored)
-```
-
-**In the framework repository** (no workspace content at the root: no `profiles.yaml`, `cases/`, `tests/` or reports):
-
-```
-lib/               fixtures, session guard, image assertions, case values (imported as asrun/<module>)
-scripts/           the pnpm commands above, plus init (workspace files) and doctor (checks)
-templates/         what `pnpm asrun init` writes into a project
-ui/                local web UI: server/ (Hono API, job runner), web/ (React + Vite), shared/ (types)
-docs/              static docs; theme-guide.html = the Inkstone theme (designed and created by Zoro Chen): colors, buttons, icons of the web UI (open in a browser, or via the "Inkstone theme" link in the UI's top bar)
-examples/demo/     a local workspace for developing the framework (only README and .gitignore are committed)
-.claude/           skills + slash commands for the AI workflow (shared by OpenCode and Claude Code)
-.opencode/         symlink: commands -> ../.claude/commands (OpenCode only reads commands here)
-AGENTS.md          the framework baseline rules (OpenCode); CLAUDE.md imports it for Claude Code
-opencode.jsonc     MCP config for OpenCode; .mcp.json is the same server for Claude Code
-```
-
-## Developing the framework
-
-`pnpm go`, `pnpm asrun`, `pnpm ui` work here too: without a workspace they use `examples/demo`. `init` writes the
-framework's `.env` for that (`ASRUN_ROOT=examples/demo`), so the commands need no environment variable:
-
-```bash
-pnpm asrun init --root examples/demo   # creates the demo workspace and the framework-root .env
-pnpm asrun --env staging --read-only
-```
-
-The demo is a workspace of its own as well:
-
-```bash
-cd examples/demo && ASRUN_ROOT=. pnpm asrun doctor
-```
-
-`ASRUN_ROOT` still wins over `.env` (a real environment variable always does), so one checkout can be pointed at
-another project for a single command:
-
-```bash
-ASRUN_ROOT=../my-real-project pnpm asrun --env staging --read-only
-```
-
-Two paths keep the two halves apart: `ROOT` (`lib/paths.ts`) is the workspace, `FRAMEWORK_ROOT` is this folder. The
-package is compiled to `dist/` (`pnpm build`) and the web UI to `ui/dist` (`pnpm ui`); both are shipped by `prepack`.
-Nothing writes into `FRAMEWORK_ROOT` at run time. After changing `lib/`, run `pnpm build` before exercising the demo's
-specs (they import the package by name, which resolves to `dist/`). After changing `templates/`, bump `version` in
-`package.json`; `pnpm asrun doctor` in a project then shows that it is behind.
-
-## How profiles become tests
-Every entry in `profiles.yaml` becomes a Playwright project (name = profile id) with its own `baseURL` and saved login.
-A spec is written once and runs once per profile; the `profile` fixture exposes that profile's `id`, `env`, `baseURL`,
-`account`, `tags` and `email`. The expected values come from the case document, through
-`requireValues(caseId, profile)` (its `## Values` block, `default` merged with the profile's own entry). Adding a
-profile = add a YAML entry (by hand or with the form on the Profiles page) + `pnpm signin <id>`, then add the values
-for it to the cases (see the guide on the Profiles page).
-
-## Troubleshooting
-- `Profile "x" is not logged in`: `pnpm signin <profile-id>`, or run with `--skip-profile <profile-id>`.
-- Screenshot mismatch: open the report; the diff view shows expected / actual / diff. If the change is
-  intended, update the baseline as above.
-- Report empty or missing: run `pnpm asrun --env <env>` first.
-- `Choose the environment to test`: there is no default; add `--env production` or `--env staging`.
+# AsRun
+
+End-to-end tests for web apps with more than one face. People write test cases in plain language; an AI agent turns
+them into Playwright specs; every spec runs once per profile — production and staging, account A and account B — and
+leaves a video, screenshots and a trace to review.
+
+AsRun is a devDependency, not a service. Your profiles, cases, specs and baselines are files in your repository.
+
+## What it does
+
+- **Plain-language cases.** `cases/<id>.md` is the source of truth: numbered steps, expected behaviour, the exact
+  values, and the visual targets a check may look at. A person can review it without reading code.
+- **One spec, every profile.** Each entry in `profiles.yaml` becomes a Playwright project with its own base URL,
+  saved login and screenshot baselines. Two accounts of one site give two independent runs.
+- **An AI workflow, not a prompt dump.** `/make-case` turns raw notes into cases, `/explore-case` checks them against
+  the real pages, `/gen-spec` writes the spec, `/heal-spec` repairs it from the last report. Each command ends with
+  `RESULT: ready` or `RESULT: needs-input`, so it works unattended.
+- **Evidence for every run.** One video per test (browser tabs joined), a trace with every action and request,
+  screenshots on failure, and a report that puts the failure first.
+- **Safe by default.** A case is read-only unless it lists environments under `writes`; writing anywhere except
+  staging needs a person's confirmation on every run; email never leaves production; logins are done by a person, 2FA
+  included.
+- **A local web UI.** Profiles and logins, a case editor with a form panel, AI runs with live logs, and reports. All
+  of it on `127.0.0.1`.
+- **Fits a terminal and a pipeline.** Filter by environment, profile, tag, suite or case; one report folder per run;
+  a failing test fails the command.
+- **No hidden state.** The framework lives in `node_modules/@asrunjs/asrun` and is never edited; a project commits
+  only its own test content.
+
+## Requirements
+
+- **Node.js 22+** and **pnpm**.
+- **ffmpeg** to join the recordings of one test into a video. `pnpm go` installs it with Homebrew or winget; on Linux
+  run `sudo apt install ffmpeg`. When it is missing, the bundled `ffmpeg-static` is used; if neither works, the report
+  lists the videos separately.
+- **Claude Code** or **OpenCode** for the AI steps. Everything else works without them.
+
+## Install
+
+```bash
+pnpm dlx @asrunjs/asrun init --install
+pnpm go
+```
+
+The first command adds AsRun and creates the workspace files (`profiles.yaml`, `cases/`, `tests/`, `AGENTS.md`, the
+commands for both AI tools, `.env`, the configs); run it in the project folder, which may be empty. The second installs
+the dependencies and Chromium, runs the checks, starts the web UI at http://127.0.0.1:4173 and opens it. After the
+first time, `pnpm go` is enough.
+
+To update:
+
+```bash
+pnpm add -D @asrunjs/asrun@latest
+pnpm asrun init --update
+pnpm asrun doctor
+```
+
+`init --update` refreshes only the generated files; it never overwrites `AGENTS.md`, `CLAUDE.md` or your skill
+overrides. `doctor` checks profiles, generated files, the installed version and the project rules.
+
+<details>
+<summary>Manual install or a pinned version</summary>
+
+```bash
+pnpm add -D @asrunjs/asrun        # or @asrunjs/asrun@<version>
+pnpm asrun init
+pnpm go
+```
+
+On pnpm 12, run `init` before adding the dependency: it writes the `allowBuilds` entry that `esbuild` and
+`ffmpeg-static` need, or `pnpm add` stops with `ERR_PNPM_IGNORED_BUILDS`. Details and flags:
+[install.md](./docs/install.md).
+
+</details>
+
+## Your first test
+
+1. **Point at your site.** Edit `profiles.yaml`: `id`, `name`, `env` (`staging` or `production`), `baseURL`, optional
+   `account` and `tags`. Expected values never go here; they live in the case.
+2. **Log in once.** `pnpm signin example-staging` opens a browser. Log in yourself, then press **Done, save the
+   session** on the card in the corner. The session is saved per profile in `.auth/`.
+3. **Write a case.** `pnpm case:new header-logo` creates `cases/header-logo.md` from the template; or start from raw
+   notes with `/make-case notes.md`. Fill in the steps, what is expected and the exact values.
+4. **Check it against the real pages.** `/explore-case header-logo staging` opens the pages (with your saved session),
+   writes what they show into the case and turns what it cannot know into questions. Answer them in the case file, or
+   in the UI's **Answer** sheet, and run it again.
+5. **Generate, run, read.** `/gen-spec header-logo staging` writes `tests/header-logo.spec.ts` and fills the case's
+   `## Values`. Then:
+
+   ```bash
+   pnpm asrun --env staging --grep header-logo
+   pnpm report
+   ```
+
+   The first run of a spec with visual checks writes its baselines: add `--update-snapshots` once, then look at every
+   new image before keeping it.
+
+In the web UI you never type a command: the **?** icon has the same steps with screenshots, and a case toolbar runs
+each AI step with **Run AI**. The full procedure is in [ai-workflow.md](./docs/ai-workflow.md).
+
+## Customize the AI for your project
+
+The commands are thin shells; the procedure is a skill in the framework package
+(`.claude/skills/{making-cases,exploring-cases,generating-specs}/`). A project refines a skill with one Markdown file
+named after the skill — not after the command:
+
+```markdown
+<!-- .claude/skills/generating-specs.md -->
+- Step titles in `test.step` keep the case's wording, including the number.
+- A helper used by more than one spec goes in `tests/helpers/<name>.ts`.
+- The staging account has no published documents: assert the empty state there instead.
+```
+
+It is read after the framework skill and before your project rules. Write only what differs.
+
+Rules combine in this order, and a later layer wins where they conflict:
+
+1. `F1`–`F6` of the framework baseline (safety, in every project). They cannot be changed.
+2. The framework baseline, `node_modules/@asrunjs/asrun/AGENTS.md`.
+3. The framework skill.
+4. Your override, `.claude/skills/<skill>.md`.
+5. Your own `AGENTS.md` §2: a table of rule ids (`F7`–`F10`) with `narrows` or `replaces`, for example the artifact
+   language.
+
+`pnpm asrun doctor` validates the table and lists the overrides it sees. An update never rewrites your rules:
+`init --force` refuses `AGENTS.md`, `CLAUDE.md` and `.claude/skills/**` without `--reset-rules`. Never edit the
+framework package itself: an update replaces it, and only the override files are yours. The AI tool stays configurable
+too: `.mcp.json` (Claude Code), `opencode.jsonc` (OpenCode and its model), `.claude/settings.json` (sandbox
+exceptions), or the model picker in the UI's top bar.
+
+Details: [ai-workflow.md](./docs/ai-workflow.md#customizing-the-prompts).
+
+## Every day (cookbook)
+
+| I want to... | Command |
+|---|---|
+| Check the workspace | `pnpm asrun doctor` |
+| See which profiles are logged in | `pnpm status [--env <env>]` |
+| Log in a profile (a person, with 2FA) | `pnpm signin <profile-id>` |
+| Run everything in one environment | `pnpm asrun --env staging` |
+| Run one profile | `pnpm asrun --env production --profile example-production` |
+| Leave out profiles (for example the ones not logged in) | `pnpm asrun --env production --skip-profile example-production` |
+| Run only the profiles with a tag | `pnpm asrun --env staging --tag us` |
+| Run only the profiles that are logged in | `pnpm asrun --env production --logged-in` |
+| Run one case | `pnpm asrun --env production --grep header-logo` |
+| Run a suite | `pnpm asrun --env production --suite smoke` |
+| Leave out the cases that write data | `pnpm asrun --env production --read-only` |
+| Look around a page as a profile | `pnpm explore example-staging /app/home` |
+| Open the last report | `pnpm report` |
+| Read what the last report says about a case | `pnpm heal:evidence --case <case-id> --env staging` |
+| Check the specs | `pnpm spec:lint [case-id]` |
+| List the baselines next to what they must show | `pnpm baselines:review [case-id] [--new]` |
+| Update the baselines of some cases on purpose | `pnpm asrun --env staging --grep <case-id> --update-baselines <case-id>` |
+| List test data that could not be removed | `pnpm cleanup --env staging` |
+| Watch a run step by step | `pnpm asrun --env staging --ui` |
+| Put the current step on the videos | `pnpm asrun --env production --step-text` |
+
+The web UI: `pnpm go` (install, build, start, open), `pnpm ui` (build and serve), `pnpm ui:dev` (development with hot
+reload). Each page covers one job: **Profiles** (login state, sign in, visit, add/duplicate/delete), **Cases** (list,
+editor, form panel, spec view, AI runs, notes-to-cases), **Run tests** (filters, update baselines, production
+confirmation), **Reports** (failure first, then video, screenshots, trace, full Playwright report), **Jobs** (live log
+of every command), **Help** and **AI guide**. See [web-ui.md](./docs/web-ui.md).
+
+## Safety
+
+Tests run against real sites, so the defaults are strict:
+
+- **Read-only unless the case says otherwise.** A case may create, edit or delete data only in the environments it
+  lists under `writes`, and only data of the test accounts, named `[asrun] ...`, which it must clean up. Elsewhere the
+  case is skipped with a notice, not failed.
+- **A person confirms non-staging writes on every run** by typing the environment name, or in a dialog in the UI.
+  `--read-only` leaves those cases out; `--allow-writes=<env>` is for unattended runs, and AI agents never use it.
+- **No email from production.** Elsewhere only when the profile has `email: true`.
+- **AI agents never write while exploring or generating.** A step that writes or needs a choice is done by a person in
+  the browser, in an attended exploration. Logins are always a person's job, 2FA included.
+- **A failing baseline is a person's decision.** An agent never updates a baseline to make a difference disappear.
+
+The rules that enforce this are `F1`–`F10` in [AGENTS.md](./AGENTS.md); a project can tighten or replace the last
+four, never the first six.
+
+## Documentation
+
+| I want to... | Read |
+|---|---|
+| Install, update, requirements, workspace location | [docs/install.md](./docs/install.md) |
+| `profiles.yaml`, environments, logins, sessions | [docs/profiles.md](./docs/profiles.md) |
+| Write or review a case | [docs/cases.md](./docs/cases.md) |
+| The four AI commands in depth | [docs/ai-workflow.md](./docs/ai-workflow.md) |
+| Every command and flag, reports, baselines | [docs/commands.md](./docs/commands.md) |
+| The local web UI, page by page | [docs/web-ui.md](./docs/web-ui.md) |
+| An error message | [docs/troubleshooting.md](./docs/troubleshooting.md) |
+| Change AsRun itself | [docs/agents/framework-dev.md](./docs/agents/framework-dev.md) |
+| The case document contract, for AI agents | [docs/agents/case-format.md](./docs/agents/case-format.md) |
+
+## Maintainers
+
+This section is for the people who maintain AsRun; the repository is private, so the instructions below assume
+access to it. It is developed against `examples/demo`, a committed workspace whose site is a local Docker stack
+(Conduit), so nothing runs against a public site.
+
+```bash
+pnpm install
+pnpm asrun init --root examples/demo
+pnpm build                         # the demo's specs import the package, which resolves to dist/
+pnpm demo:up                       # Docker Desktop must run
+pnpm demo:seed
+pnpm signin conduit-a              # password in examples/demo/README.md
+pnpm asrun --env staging --read-only
+pnpm ui
+```
+
+`pnpm demo:reset` returns the database to the state the cases expect (logins must be redone). Before a pull request:
+`pnpm typecheck`, `pnpm build`, `pnpm ui:build`, and the checks above on the demo. A change to `templates/` or the
+shipped docs bumps `version` in `package.json`.
+
+Framework layout, the two agent CLIs, the UI theme and copy rules, and where documentation belongs:
+[docs/agents/framework-dev.md](./docs/agents/framework-dev.md).
+
+## License
+
+MIT. Created by Zoro Chen.