@asrunjs/asrun 0.5.2 → 0.5.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -4,26 +4,50 @@ Read `AGENTS.md` first: it is the product baseline, and this file only adds how
4
4
  is relevant to a project that uses asrun as a dependency.
5
5
 
6
6
  ## Framework layout
7
+
7
8
  `<framework>` for a project is `node_modules/@asrunjs/asrun`. In this repository it is the root folder:
8
- - `lib/` fixtures (`profile`, session guard) and assertion helpers; specs import them as
9
+
10
+ - `lib/` fixtures (`profile`, session guard, write gate) and helpers; specs import them as
9
11
  `@asrunjs/asrun/<module>` (for example `@asrunjs/asrun/fixtures`, `@asrunjs/asrun/values`)
10
- - `scripts/` user-facing commands, see README.md
12
+ - `scripts/` the user-facing commands; every command and flag is documented in `docs/commands.md`
11
13
  - `templates/` the files `pnpm asrun init` writes into a workspace
12
14
  - `ui/` local web UI (`pnpm ui`). It may only run the fixed actions in `ui/server/actions.ts`
13
15
  and must never serve anything under `.auth/`.
14
- - `docs/theme-guide.html` the Inkstone theme guide of the web UI: color tokens, buttons, badges, icons, surfaces
16
+ - `docs/` user documentation (`install.md`, `profiles.md`, `cases.md`, `ai-workflow.md`,
17
+ `commands.md`, `web-ui.md`, `troubleshooting.md`) and `docs/theme-guide.html`
18
+ - `docs/agents/` documents for AI agents: this file, and `case-format.md` (the case document contract)
15
19
  - `examples/demo/` a workspace for developing the framework itself against a local Docker stack (Conduit);
16
20
  it is committed (`.auth/` and `reports/` never are), see its README
21
+ - `.claude/` skills + slash commands for Claude Code; the skills are shared with OpenCode
22
+ - `.opencode/commands/` the same commands for OpenCode (it does not read `.claude/commands`)
23
+ - `AGENTS.md` the product baseline rules (OpenCode reads it); `CLAUDE.md` imports it for Claude Code
24
+ - `README.md` the user-facing overview: what it does, install, first test, cookbook, links to `docs/`
25
+
26
+ `package.json` `files` ships `README.md`, `AGENTS.md`, `CLAUDE.md`, `docs/`, `templates/`, `.claude/`, `bin/`,
27
+ `dist/` and `ui/dist/`, so a project can read the same documentation from
28
+ `node_modules/@asrunjs/asrun/docs/`.
17
29
 
18
30
  ## Working on the framework itself
31
+
19
32
  - The framework repository holds no workspace content at the root: no `profiles.yaml`, `cases/`, `tests/` or
20
33
  reports there. The one workspace is `examples/demo`, which is committed because it runs its own local site; its
21
- `.auth/`, `reports/` and other run output are not. Develop against it: `pnpm demo:up` and `pnpm demo:seed` start
22
- and fill the local stack, `pnpm asrun init --root examples/demo` (it also writes the framework's `.env` with
23
- `ASRUN_ROOT=examples/demo`), then `pnpm asrun --env staging --read-only`.
34
+ `.auth/`, `reports/` and other run output are not.
35
+ - Develop against the demo:
36
+
37
+ ```bash
38
+ pnpm demo:up # build and start the local stack
39
+ pnpm demo:seed # accounts, articles, follows, favorites
40
+ pnpm asrun init --root examples/demo # also writes the framework's .env (ASRUN_ROOT=examples/demo)
41
+ pnpm build # the demo's specs import the package, which resolves to dist/
42
+ pnpm asrun --env staging --read-only
43
+ ```
44
+
45
+ `pnpm demo:reset` deletes the database and seeds it again; logins must be redone afterwards. Demo accounts are in
46
+ `examples/demo/README.md`.
24
47
  - Everything the framework reads for a project goes through `ROOT` in `lib/paths.ts` (the workspace root); its own
25
48
  assets (templates, docs, skills, its `node_modules`) go through `FRAMEWORK_ROOT`. Never write into
26
49
  `FRAMEWORK_ROOT` at run time.
50
+ - After changing `lib/`, run `pnpm build` before exercising the demo's specs.
27
51
  - `templates/` is the source of the files a workspace gets. When a template changes, bump `version` in
28
52
  `package.json` so `pnpm asrun doctor` in a project shows that it is behind.
29
53
  - `init` owns two lists: `GENERATED` (rewritten by `--update`; the commands and the Playwright files) and `ONCE`
@@ -33,7 +57,51 @@ is relevant to a project that uses asrun as a dependency.
33
57
  - A project's rules never flow back here. The framework reads a workspace only through `ROOT` at run time; nothing
34
58
  under `templates/` or `.claude/skills/` is generated from a project.
35
59
 
60
+ ### Installing this repository as a Git dependency (private)
61
+
62
+ The npm package is the public channel for users. This repository is private, so a Git dependency works only on a
63
+ machine that can clone it:
64
+
65
+ ```bash
66
+ pnpm add -D github:wizcas/asrun#<tag>
67
+ ```
68
+
69
+ pnpm 12 resolves a GitHub spec through the canonical HTTPS URL, not SSH. That machine therefore needs HTTPS
70
+ credentials (a credential helper or a token), or a rewrite to SSH:
71
+
72
+ ```bash
73
+ git config --global url."git@github.com:".insteadOf https://github.com/
74
+ ```
75
+
76
+ On CI, give the runner a `GITHUB_TOKEN`, for example:
77
+
78
+ ```bash
79
+ git config --global \
80
+ url."https://x-access-token:${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"
81
+ ```
82
+
83
+ or a deploy key with the SSH rewrite above. A Git install also runs the package's `prepare` (it builds `dist/` and
84
+ `ui/dist/`), so it is slower than the registry: use it only to pin a commit, a branch or a tag that is not published
85
+ yet. For a one-off change, `pnpm patch @asrunjs/asrun` and commit the patch instead.
86
+
87
+ ## Documentation
88
+
89
+ The split is intentional; keep it:
90
+
91
+ - `README.md` — the user's first page: what AsRun does, requirements, install, the first test, customizing the
92
+ prompts, the every-day cookbook, safety in short, a documentation index, maintainer notes. It should stay readable
93
+ in one sitting; a section that grows past a screen belongs in `docs/`.
94
+ - `docs/*.md` — the user reference. `docs/commands.md` is the only place that lists every command and flag.
95
+ - `docs/agents/*.md` — for AI agents. `case-format.md` is the contract of the case document (fixed tokens, values,
96
+ targets); `AGENTS.md` references it, and the template `AGENTS.md` does too.
97
+
98
+ When behavior changes, update the matching document in the same change. A command or flag goes into
99
+ `docs/commands.md` and, if it is a daily one, into the README cookbook. A change to the case format goes into
100
+ `docs/agents/case-format.md` and `docs/cases.md` together. Keep the relative links working and the fixed tokens
101
+ spelled exactly as the parser reads them.
102
+
36
103
  ## The commands and the two agent CLIs
104
+
37
105
  - Claude Code reads `.claude/commands/*` and `.claude/skills/<name>/SKILL.md`.
38
106
  - OpenCode reads `.opencode/commands/*`, `.claude/skills/<name>/SKILL.md`, `.opencode/skills/<name>/SKILL.md` and
39
107
  `.agents/skills/<name>/SKILL.md`. It does **not** read `.claude/commands`, and it does **not** read a flat
@@ -46,8 +114,10 @@ is relevant to a project that uses asrun as a dependency.
46
114
  plain file read. That is why the skill itself must read the project override (`.claude/skills/<skill>.md`).
47
115
 
48
116
  ## Web UI (`ui/web`)
117
+
49
118
  Before adding or changing any UI element, read `docs/theme-guide.html` (open the file; it is static HTML) and follow it.
50
119
  Its rules are binding, in particular:
120
+
51
121
  - Icons: Feather via the `react-feather` package only (`import {ChevronDown} from 'react-feather'`). Never draw, paste
52
122
  or hand-write SVGs, and never add another icon library. Size 14 in buttons, 16-20 elsewhere; decorative icons get
53
123
  `aria-hidden="true"`, icon-only buttons get an `aria-label`. Do not use Unicode symbols (`▲ ▼ ◀ ▶ ⤢`) as icons.
@@ -59,6 +129,9 @@ Its rules are binding, in particular:
59
129
  `ui/web/src/components/search-select.tsx`: an input with a filtered list below it, which a native `<select>` cannot
60
130
  do. It keeps `Select`'s height, border and focus ring, so the two look like one family; do not build a third variant.
61
131
  - If a change adds or alters a token, update `ui/web/src/index.css` and `docs/theme-guide.html` together.
132
+ - The Help page ("How to start") shows screenshots from `ui/web/public/help/`: retake one when the page it shows
133
+ changes.
134
+
62
135
  Before reaching for a new dependency or writing a helper component, grep `package.json` and `ui/web/src` for what the
63
136
  project already uses.
64
137
 
@@ -77,3 +150,18 @@ Every lead, label and tooltip is written for the person using the UI, never for
77
150
  `CopyBlock`, or a folded `<details>` block titled "For the command line".
78
151
  - **Say what the user sees.** "The run appears in the Jobs bar", not "the command runs in a one-shot session"; "the
79
152
  browser shows a small card", not "the agent pauses on a `writes` step".
153
+
154
+ ## Before you open a pull request
155
+
156
+ ```bash
157
+ pnpm typecheck
158
+ pnpm build # dist/ must match lib/ and scripts/
159
+ pnpm case:check # examples/demo cases
160
+ pnpm spec:lint # examples/demo specs
161
+ pnpm ui:build # ui/dist/ must match ui/web and ui/server
162
+ ```
163
+
164
+ - `pnpm build` and `pnpm ui:build` also run from `prepare` and `prepack`, so a published package is complete.
165
+ - Bump `version` when `templates/` or the shipped docs change; `pnpm asrun doctor` in a project compares it.
166
+ - Keep the demo's committed expectations (`profiles.yaml`, `cases/`, `tests/`, baselines) in step with the demo site.
167
+ Baselines are a person's decision: never let an agent regenerate them.
@@ -0,0 +1,235 @@
1
+ # The AI workflow
2
+
3
+ Four commands carry a test case from rough notes to a Playwright spec, and repair it when the product changes. They
4
+ work the same in Claude Code and OpenCode: each command is a thin shell that loads a skill from the framework package,
5
+ so the procedure lives in one place and a project can refine it (see
6
+ [Customizing the prompts](#customizing-the-prompts)).
7
+
8
+ | Command | What it does |
9
+ |---|---|
10
+ | `/make-case <notes-or-path>` | Turn raw notes into case documents. Optional step. |
11
+ | `/explore-case <case-id> <env> [--attended]` | Open the real pages and write what they show into the case. |
12
+ | `/gen-spec <case-id> <env> [--force]` | Write `tests/<case-id>.spec.ts` and fill the case's `## Values`. |
13
+ | `/heal-spec <case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]` | Diagnose a failing spec from its last report and fix it. |
14
+
15
+ The environment is **required** by every command except `/make-case`, which does not open pages. There is no default.
16
+
17
+ Every command ends with `RESULT: ready` or `RESULT: needs-input`. `needs-input` means somebody has to answer a
18
+ question in the case file (or, in the web UI, in the job's **Answer** sheet). A one-shot run cannot ask in the chat:
19
+ it writes its questions into the case and stops.
20
+
21
+ In the web UI you never type a command: choose the step in a case toolbar and press **Run AI**, or paste notes on
22
+ **Cases > From memo**. The UI runs the selected AI tool and streams the output into the Jobs bar.
23
+
24
+ ## 0. Optional: start from raw notes
25
+
26
+ ```
27
+ /make-case path/to/notes.md
28
+ ```
29
+
30
+ or paste the text after the command. The agent normalizes the notes into steps in the project's artifact language
31
+ (English by default), marks the steps that write data (`[writes]`) or send email (`[email]`), keeps the original lines
32
+ in a `## Source` table, and lists the gaps with a suggested default for each. It never invents expected values
33
+ (unknown ones become a sentence with a `TODO` block under it) and never opens a browser. What the pages really show
34
+ is found in step 2.
35
+
36
+ It writes `cases/<id>.md` only, and never overwrites an existing case. `pnpm case:check` validates the result.
37
+
38
+ ## 1. Write or open the case
39
+
40
+ `pnpm case:new header-logo` creates `cases/header-logo.md` from the template, or use a case you already have. The
41
+ format is in [cases.md](./cases.md). Write the steps, the expectations and — when you know them — the values.
42
+
43
+ ## 2. Explore the case (only where a person and the agent work together)
44
+
45
+ ```
46
+ /explore-case header-logo staging
47
+ ```
48
+
49
+ The agent opens the real pages of the profiles that are logged in and only **reads** them. It never creates, edits,
50
+ publishes, shares or deletes anything, in staging or in production. It:
51
+
52
+ - fixes wording in the case file and writes what the pages show into it, marked `observed, 🔍 to be confirmed by QA`;
53
+ - turns what it cannot know into `🚫 TODO [blocking]` / `❓ TODO [non-blocking]` questions, each under a sentence that
54
+ is its suggestion and can stand in the case as it is, with a last line `- ✍️ Answer:`;
55
+ - lists the candidates instead of guessing a target (`## Targets`);
56
+ - edits the case file only (plus reference screenshots under `.explore/targets/`).
57
+
58
+ **To accept the suggestion, delete the whole block.** **To answer, write after `Answer:`.** Then run the command
59
+ again. In the web UI, the run's **What to do next** bar offers **Answer**: the question list, an answer box,
60
+ **Use the suggested defaults** and **Generate spec**. A continued job keeps the same conversation and permissions, and
61
+ can be continued once per turn, only while the UI server has not restarted.
62
+
63
+ ### Attended exploration (`--attended`)
64
+
65
+ Use it for a case that creates or edits data, or needs a choice (an existing document, items). Stay at the computer:
66
+ the browser window opens and the agent waits while you do those steps.
67
+
68
+ - The window shows a small draggable card in the bottom-left corner with what to do, **Done** (hand control back) and
69
+ **Cannot do this**. The agent also continues by itself when the finish condition it named shows up (for example a
70
+ "Saved" message); the card turns green when control is back. It waits up to 10 minutes per step.
71
+ - What you did is recorded in the case under the step as **general recipe lines**
72
+ (`- Recipe (phase): ...`): positions instead of names of changeable data, `[asrun]` names for created data, control
73
+ labels but no selectors.
74
+ - Delete the `[asrun]` items you created before you press Done on the last step.
75
+ - Without `--attended` the agent never waits: a step that needs a person becomes a `TODO [blocking]`.
76
+
77
+ In the web UI the **Attended** box is ticked by default for Explore.
78
+
79
+ ## 3. Log in the profiles the case needs
80
+
81
+ ```bash
82
+ pnpm status
83
+ pnpm signin example-staging
84
+ ```
85
+
86
+ Skip this for a `login: none` case. Sessions are per profile; see [profiles.md](./profiles.md).
87
+
88
+ ## 4. Generate the spec
89
+
90
+ ```
91
+ /gen-spec header-logo staging
92
+ ```
93
+
94
+ The agent replays the case on the real pages with your saved sessions (it never writes data) and writes
95
+ `tests/header-logo.spec.ts`. It fills the case's `## Values` for the profiles it explored and checks the files
96
+ (`pnpm typecheck`, `pnpm case:check`; `pnpm spec:lint <case-id>` must have no failure).
97
+
98
+ It does **not** run the spec and does **not** create screenshot baselines: running is a separate step (6).
99
+
100
+ - A case must be in the project's artifact language. If it is in another language, `/gen-spec` translates the case
101
+ file first and tells you.
102
+ - A `[writes]` or `[input]` step with no `- Recipe:` line under it is not generated: the command stops with
103
+ `needs-input` and asks you to run step 2 with `--attended`.
104
+ - An existing spec is left alone; `--force` regenerates it from the case. The case's `## Values` and the baseline
105
+ images are kept, and the report lists the baselines that may be out of date. In the web UI, **Run** on a case that
106
+ has a spec asks "Overwrite the spec?" first.
107
+
108
+ ## 5. Review what the AI produced
109
+
110
+ Check that the spec reads like the case, and that `## Values` has the right values for each profile. The spec imports
111
+ `test`/`expect` from `@asrunjs/asrun/fixtures`, never from `@playwright/test`, and every step is a `test.step` named
112
+ after the case's step. You normally do not edit either file by hand, but you can.
113
+
114
+ ## 6. Run it and read the evidence
115
+
116
+ ```bash
117
+ pnpm asrun --env staging --grep header-logo
118
+ pnpm report
119
+ ```
120
+
121
+ The first run of a spec with visual checks writes its baselines. Run it once with `--update-snapshots`, then open
122
+ every new image under `tests/<case>.spec.ts-snapshots/` and confirm it shows the target card's element for that
123
+ profile. Baselines are the source of truth for pixel checks.
124
+
125
+ Then run all profiles and read the report: the video of each test, screenshots on failure, and the trace with the page
126
+ and the network requests. See [commands.md](./commands.md) for filters and baselines.
127
+
128
+ ## Healing a failing spec
129
+
130
+ ```
131
+ /heal-spec header-logo staging
132
+ ```
133
+
134
+ or **Heal spec** in the case toolbar, or **Heal this failure** on a failed result of the Reports page.
135
+
136
+ The agent decides whether the test needs updating or whether it found a real product problem. It **never** edits an
137
+ expectation just to make a failure disappear.
138
+
139
+ - The environment is required. Only the profiles that are logged in there run (`--logged-in` leaves the others out
140
+ instead of failing them).
141
+ - The web UI first opens a dialog: which profiles to heal (every logged-in profile ticked) and an optional text box
142
+ for what you want healed, with evidence or clues. The text is stored in `.tmp/hints/` and passed as `--hints`.
143
+ Clues guide the diagnosis; they never lower the bar for a product finding.
144
+ - It reads the existing report first:
145
+
146
+ ```bash
147
+ pnpm heal:evidence --case header-logo --env staging
148
+ ```
149
+
150
+ It prints status, failed step, error, call log and screenshots per profile, and whether the report is fresh or
151
+ stale. The case is run again only when there is no report, the report is stale, it does not explain the failure, or
152
+ a clue has to be checked. After editing the spec, it runs the case once to confirm.
153
+
154
+ ## Why a generated spec can be trusted
155
+
156
+ Three binding rules hold in exploring, generating and healing. They live in the framework's `AGENTS.md` and in
157
+ `.claude/skills/exploring-cases/references/assertion-targets.md` and `readiness.md`.
158
+
159
+ - **Targets.** A check on a logo, image, icon, file, PDF, screenshot or repeated text points to a `[T1]` card under
160
+ `## Targets`: what it is (and what it is not), where it is, how it is identified (never `.first()` or a bare `img`),
161
+ when it is ready, and a highlighted reference screenshot under `.explore/targets/`. When in doubt the agent does not
162
+ pick: it writes a `🚫 TODO [blocking]` that names the exact image, text, place or UI, with the candidates listed.
163
+ - **Readiness.** After an action that loads something, the key requests must finish and the content must be visible
164
+ before anything is read. Specs use `actAndWait` (`@asrunjs/asrun/ready`); "missing" and "empty" checks need a
165
+ positive ready signal first. Healing treats a "missing / empty / timeout" failure as a readiness problem first.
166
+ - **Downloads.** `verifyDownload` (`@asrunjs/asrun/downloads`) checks that the download starts, finishes, the file
167
+ exists and the file is valid for its kind. Any failure or timeout fails the whole step with a message that names the
168
+ stage.
169
+ - **PDFs are read like a person reads them.** `expectPdf` (`@asrunjs/asrun/pdf`) renders the pages to images (a logo
170
+ region must not be blank and must match the profile's baseline, 2% tolerance) and extracts the text. Every page is
171
+ checked unless the case limits the pages in strong words ("only", "exactly", "must"); "should", "always", "usually"
172
+ or no wording mean every page.
173
+
174
+ Checks you can run yourself:
175
+
176
+ | Command | What it checks |
177
+ |---|---|
178
+ | `pnpm spec:lint [case-id]` | fails on screenshots without a target, `.first()` on images, sleeps, `networkidle`, raw download events, a `%PDF` check on its own |
179
+ | `pnpm baselines:review [case-id] [--new]` | lists every baseline with the card that says what it must show, so you can look at each new image |
180
+ | `pnpm case:check [case-id]` | fails on an incomplete card or bad values; warns on an image check with no card |
181
+ | `pnpm pdf:render <file.pdf> --out <dir> [--pages all] [--corner top-left]` | renders the pages to images and prints the text, to look at a PDF by hand |
182
+
183
+ ## Customizing the prompts
184
+
185
+ The commands are shells; the procedure is a skill in the framework package:
186
+
187
+ - commands: `.claude/commands/*.md` (Claude Code) and `.opencode/commands/*.md` (OpenCode);
188
+ - skills: `<framework>/.claude/skills/{making-cases,exploring-cases,generating-specs}/`.
189
+
190
+ A project refines a skill with a **flat override file**, named after the skill (not the command):
191
+
192
+ ```
193
+ .claude/skills/generating-specs.md
194
+ .claude/skills/exploring-cases.md
195
+ .claude/skills/making-cases.md
196
+ ```
197
+
198
+ The command and the skill read it; it is not discovered as a standalone skill. Rules combine in this order, and a
199
+ later one wins where they conflict:
200
+
201
+ 1. `F1`–`F6` of the framework baseline: safety, in every project. Nothing can change them.
202
+ 2. The framework baseline, `<framework>/AGENTS.md`.
203
+ 3. The framework skill.
204
+ 4. The project override `.claude/skills/<skill>.md`.
205
+ 5. The project's own `AGENTS.md` §2 (Project rules).
206
+
207
+ An override is plain Markdown; write only what differs:
208
+
209
+ ```markdown
210
+ # generating-specs in this project
211
+
212
+ - Step titles in `test.step` keep the case's wording, including the number.
213
+ - A helper used by more than one spec goes in `tests/helpers/<name>.ts`.
214
+ - The staging account has no published documents: assert the empty state there instead.
215
+ ```
216
+
217
+ The project's `AGENTS.md` §2 is the other place: a table of rule ids (`F7`–`F10`; `F1`–`F6` are fixed) with `narrows`
218
+ (stricter) or `replaces` (a different default). For example, a project that writes cases in Simplified Chinese
219
+ `replaces` `F10`. `pnpm asrun doctor` fails on an unknown id, a missing `Effect`, or a non-negotiable rule, and lists
220
+ the overrides it sees.
221
+
222
+ An update never overwrites `AGENTS.md`, `CLAUDE.md` or `.claude/skills/**`: `pnpm asrun init --force` refuses them
223
+ without `--reset-rules`. Never edit the framework package itself.
224
+
225
+ Project-level AI tool configuration:
226
+
227
+ | File | What it sets |
228
+ |---|---|
229
+ | `.mcp.json` | the `playwright-test` MCP server for Claude Code |
230
+ | `opencode.jsonc` | the same MCP server for OpenCode, plus an optional `"model"` for one-shot runs |
231
+ | `.claude/settings.json` | the sandbox exceptions for `pnpm asrun`, `pnpm status`, `pnpm heal:evidence` |
232
+
233
+ In the web UI, the top bar chooses the AI tool and the model for every AI run, and **Thinking** passes `--thinking`
234
+ to OpenCode. The choice is remembered per CLI in the browser. Without a model, a one-shot OpenCode run uses its
235
+ built-in free model, which has a daily usage limit.
package/docs/cases.md ADDED
@@ -0,0 +1,189 @@
1
+ # Writing case documents
2
+
3
+ A case is a Markdown file that a person writes and reviews. It is the source of truth for the test: steps,
4
+ expectations, the exact values and the visual targets. The spec is generated from it and never replaces it.
5
+
6
+ ## Where cases live
7
+
8
+ - `cases/<name>.md`, or one group folder: `cases/<group>/<name>.md`.
9
+ - The **id** is the path below `cases/` without `.md`: `header-logo` or `documents/landing-page`. Names are lowercase
10
+ words with single dashes.
11
+ - The spec of a case lives at the same path below `tests/` (`tests/documents/landing-page.spec.ts`), with its
12
+ `*-snapshots/` folder next to it.
13
+ - Files starting with `_` are ignored (`cases/_template.md` is the template).
14
+
15
+ ```bash
16
+ pnpm case:new header-logo # creates cases/header-logo.md from the template
17
+ pnpm case:new documents/header-logo # also creates the group folder
18
+ pnpm case:check # validates every case (or: pnpm case:check <id>)
19
+ ```
20
+
21
+ ## The document
22
+
23
+ ````markdown
24
+ ---
25
+ id: header-logo
26
+ title: The header shows the product logo
27
+ login: required
28
+ suites: [smoke, regression]
29
+ ---
30
+
31
+ ## Goal
32
+ Why this case exists, in one or two sentences.
33
+
34
+ ## Steps
35
+ 1. Open `/`.
36
+ - Ready when: the header has rendered its menu.
37
+ 2. The header shows the product logo. [T1]
38
+
39
+ ## Expected
40
+ - The logo is the product logo for this profile, not a customer or partner logo.
41
+
42
+ ## Values
43
+ ```yaml
44
+ default:
45
+ logoAlt: Acme
46
+ example-staging:
47
+ logoAlt: Acme Staging
48
+ ```
49
+
50
+ ## Visual checks
51
+ - The header logo matches its baseline. [T1]
52
+
53
+ ## Targets
54
+ ### T1 Header logo
55
+ - What: the product logo, not the customer logo in the banner.
56
+ - Where: top-left of the header.
57
+ - Identified by: role `img`, name "Acme".
58
+ - Not: the customer logo in the banner and the favicon.
59
+ - Ready when: the image has a non-zero natural width.
60
+ - Reference example-staging: .explore/targets/header-logo/T1-example-staging.png
61
+
62
+ ## Notes
63
+ Anything a reviewer should know.
64
+ ````
65
+
66
+ The headings are fixed English tokens (`Goal`, `Steps`, `Expected`, `Values`, `Visual checks`, `Targets`, `Notes`,
67
+ and `Source` when the case was made from raw notes). A project language never translates them.
68
+
69
+ ## Front matter
70
+
71
+ | Key | Meaning |
72
+ |---|---|
73
+ | `id` | The case id (the path without `.md`). |
74
+ | `title` | One-line summary, shown in the editor and the spec. |
75
+ | `login` | `required` (the default) or `none` for a public page that needs no saved session. A `login: none` spec sets `test.use({mustLogin: false})`. |
76
+ | `suites` | Required. One or more lowercase words: `suites: [smoke, regression]`. A case runs in every suite it lists. |
77
+ | `writes` | The environments in which the case may create or change data, for example `writes: [staging]`. Omit it for a read-only case. |
78
+ | `writes-note` | Required with `writes`: what the case creates or changes, and that it is removed again. |
79
+ | `envs` | Optional: run the case only in these environments; elsewhere it is skipped. Must include every environment in `writes`. |
80
+
81
+ `priority: <level>` is the old name of `suites`; it is read as `suites: [<level>]`. Do not use both.
82
+
83
+ ## Steps
84
+
85
+ Numbered lines. Keep them short and in plain language: the spec uses each step's text verbatim in `test.step`, so the
86
+ report and the trace read like the case.
87
+
88
+ - **`- Ready when:`** under a step says what positive signal shows that what the step loads is really there (the list
89
+ has its first row, the editor is visible). The generated spec waits for it before it reads anything. It is not an
90
+ assertion; what must be true goes under `## Expected`.
91
+ - **Step tags**: `[writes]` for a step that creates or changes data, `[input]` for a step that needs a person's
92
+ choice (an existing document, items), `[email]` for a step that sends email. They are written at the end of the
93
+ step line.
94
+ - **`- Recipe (phase):`** under a tagged step records what a person did in the browser during an attended
95
+ exploration: positions instead of names of changeable data (`select items #1 to #3`), `[asrun]` names for created
96
+ data, control labels but no selectors. Several lines per step are normal. A tagged step with no recipe is not
97
+ generated: `/gen-spec` stops with `needs-input`.
98
+ - **Open questions** go in a `🚫 TODO [blocking]` or `❓ TODO [non-blocking]` block under the sentence they are about,
99
+ with the agent's suggestion written above and a last line `- ✍️ Answer:`. Accepting a question means deleting the
100
+ whole block; answering means writing after `Answer:`. The marker `🔍 to be confirmed by QA` / `👁️ not observed`
101
+ marks something seen but not settled.
102
+
103
+ ## Expected
104
+
105
+ Bullets: behaviour that must hold for every profile. Profile differences belong in `## Values`, not in prose.
106
+
107
+ ## Values
108
+
109
+ The exact literals the spec asserts, in one fenced `yaml` block: names, labels, options, test data names.
110
+
111
+ ```yaml
112
+ default:
113
+ createButton: Create New
114
+ emptyList: No documents yet
115
+ example-staging:
116
+ createButton: Create a document
117
+ ```
118
+
119
+ - `default` applies to every profile; an entry named after a profile id is **deep-merged over it**. An empty cell in
120
+ the UI removes the key from that profile.
121
+ - Only profile ids from `profiles.yaml` (or `default`) may appear at the top level. An unknown entry is an error.
122
+ - The case document is the only place where expected values live. `profiles.yaml` holds connection settings only.
123
+ - A profile with no entry (and no `default`) fails with "Missing values"; an empty value or one that starts with `TODO`
124
+ fails with "Incomplete values". Both messages say what to fill in.
125
+ - Specs read them with `requireValues('<id>', profile)` and never parse the YAML themselves.
126
+
127
+ ## Visual checks and targets
128
+
129
+ A visual check names the element that must match a screenshot baseline. A check on a logo, image, icon, file, PDF,
130
+ screenshot or repeated text refers to a card under `## Targets` as `[T1]`, and the spec carries `// target: T1` on the
131
+ line before the check.
132
+
133
+ Each card has these fields:
134
+
135
+ - `What`: the thing itself, and what it is **not** when a look-alike exists.
136
+ - `Where`: the region of the page, dialog, tab or file page.
137
+ - `Identified by`: role and name, test id, alt text or title; never a position.
138
+ - `Not`: the look-alikes that must not be matched.
139
+ - `Ready when`: what shows the content has loaded.
140
+ - `Reference <profile-id>:` a screenshot under `.explore/targets/<case-id>/`, or `none: <reason>`.
141
+
142
+ Do not guess a target: when it is unclear which element is meant, the exploring agent writes a `🚫 TODO [blocking]`
143
+ that names the candidates, and a person settles it. The binding rules are in
144
+ `../.claude/skills/exploring-cases/references/assertion-targets.md` of the framework package.
145
+
146
+ ## Data writing
147
+
148
+ By default a case is read-only: look, read, compare. No creating, editing, deleting, submitting or accepting
149
+ anything.
150
+
151
+ A case may write only in the environments listed under `writes`, and only data of the test account:
152
+
153
+ - everything it creates is named `[asrun] ...` (the `records` fixture refuses other names);
154
+ - created data is registered with `records.track` and removed after the test, also after a failure;
155
+ - the case's own last step is normally the deletion; `track` is the safety net;
156
+ - staging needs no confirmation. In any other environment a person confirms every run (typing the environment name,
157
+ or the dialog in the web UI). `--read-only` leaves the write cases out, and `--allow-writes=<env>` skips the prompt
158
+ for unattended runs (AI agents never use it);
159
+ - write cases are never retried and run one at a time per profile;
160
+ - a case that is not allowed to write in the current environment is **skipped with a notice**, not failed;
161
+ - whatever could not be removed is listed in red in the report and by `pnpm cleanup --env <env>`; delete it by hand,
162
+ then `--forget`.
163
+
164
+ Email is never sent in production, and elsewhere only when the profile has `email: true`. A step that sends email is
165
+ wrapped in the spec with `emailStepAllowed(profile, '<step title>')`, which records the skip in the report.
166
+
167
+ Never write a case for the login page or the login flow: login is a person's job (`pnpm signin`), not a test.
168
+
169
+ ## What `pnpm case:check` validates
170
+
171
+ It reads every case (or the ids you pass) and prints one line per case. It **fails** on:
172
+
173
+ - invalid front matter (missing `suites`, a bad `writes`/`envs` combination, `writes` without `writes-note`);
174
+ - a target card without its required fields (`What`, `Where`, `Identified by`), or a reference image outside
175
+ `.explore/targets/<case-id>/`;
176
+ - a `[T<n>]` reference with no card;
177
+ - an invalid `## Values` block (bad YAML, an unknown profile, a non-map entry).
178
+
179
+ It **warns** (usable, but something to settle) on: a reference image missing on this machine (`.explore/` is
180
+ git-ignored), a card without a reference line, a check on an image or file with no target card, no `## Values`, a
181
+ `[input]`/`[writes]` step with no `- Recipe:` line, the old `## Inputs` / `## Data` sections, and question blocks
182
+ that cannot be answered in place.
183
+
184
+ ## Rename or move a case
185
+
186
+ Use the Cases page (click the file name or the pencil icon). The spec and the baseline images move with it and the
187
+ spec is rewritten to match. By hand, move the three things together (`cases/...md`, `tests/...spec.ts`, its
188
+ `*-snapshots/`), and keep the spec's `// case:` line and its imports unchanged: specs import the framework by package
189
+ name, so their folder depth does not change the imports.