@asrunjs/asrun 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +11 -6
- package/.claude/commands/gen-spec.md +10 -6
- package/.claude/commands/heal-spec.md +13 -9
- package/.claude/commands/make-case.md +1 -1
- package/.claude/skills/exploring-cases/SKILL.md +84 -63
- package/.claude/skills/exploring-cases/references/assertion-targets.md +8 -4
- package/.claude/skills/exploring-cases/references/case-rules.md +110 -0
- package/.claude/skills/exploring-cases/references/handoff-card.js +73 -11
- package/.claude/skills/exploring-cases/references/notes.md +87 -0
- package/.claude/skills/exploring-cases/references/open-questions.md +58 -6
- package/.claude/skills/exploring-cases/references/readiness.md +7 -5
- package/.claude/skills/exploring-cases/references/tools.md +151 -0
- package/.claude/skills/exploring-cases/references/user-handoff.md +47 -18
- package/.claude/skills/generating-specs/SKILL.md +99 -54
- package/.claude/skills/generating-specs/references/conventions.md +4 -0
- package/.claude/skills/making-cases/SKILL.md +14 -7
- package/.claude/skills/making-cases/references/gaps.md +7 -4
- package/.claude/skills/making-cases/references/normalizing.md +1 -1
- package/AGENTS.md +61 -20
- package/README.md +10 -8
- package/bin/asrun.mjs +2 -0
- package/dist/lib/case-doc.d.ts +41 -4
- package/dist/lib/case-doc.js +109 -17
- package/dist/lib/explore-notes.d.ts +74 -0
- package/dist/lib/explore-notes.js +242 -0
- package/dist/lib/paths.d.ts +2 -0
- package/dist/lib/paths.js +2 -0
- package/dist/scripts/check-cases.js +89 -1
- package/dist/scripts/cli.js +9 -2
- package/dist/scripts/doctor.js +50 -3
- package/dist/scripts/explore-notes.d.ts +1 -0
- package/dist/scripts/explore-notes.js +131 -0
- package/dist/scripts/markers-confirm.d.ts +1 -0
- package/dist/scripts/markers-confirm.js +139 -0
- package/dist/ui/server/actions.js +50 -16
- package/dist/ui/server/app.js +7 -2
- package/dist/ui/server/job-store.d.ts +60 -0
- package/dist/ui/server/job-store.js +198 -0
- package/dist/ui/server/jobs.d.ts +46 -4
- package/dist/ui/server/jobs.js +361 -59
- package/dist/ui/server/process-info.d.ts +17 -0
- package/dist/ui/server/process-info.js +126 -0
- package/dist/ui/server/result-state.d.ts +10 -0
- package/dist/ui/server/result-state.js +106 -0
- package/dist/ui/shared/ai-commands.d.ts +4 -1
- package/dist/ui/shared/ai-commands.js +14 -4
- package/dist/ui/shared/types.d.ts +17 -2
- package/docs/agents/case-format.md +40 -5
- package/docs/agents/framework-dev.md +4 -0
- package/docs/ai-workflow.md +56 -16
- package/docs/cases.md +32 -8
- package/docs/commands.md +17 -1
- package/docs/theme-guide.html +56 -0
- package/docs/troubleshooting.md +40 -0
- package/docs/web-ui.md +59 -15
- package/package.json +3 -1
- package/templates/.claude/commands/explore-case.md +11 -6
- package/templates/.claude/commands/gen-spec.md +10 -5
- package/templates/.claude/commands/heal-spec.md +11 -7
- package/templates/.claude/commands/make-case.md +1 -1
- package/templates/AGENTS.md +9 -6
- package/templates/cases/_template.md +7 -0
- package/templates/package.json +2 -0
- package/ui/dist/assets/{dist-WvvCZeeu.js → dist--BXJInTq.js} +1 -1
- package/ui/dist/assets/{dist-B7HeglDd.js → dist-BMmNFIUH.js} +1 -1
- package/ui/dist/assets/{dist-DevASegc.js → dist-BXNaYHeh.js} +1 -1
- package/ui/dist/assets/{dist-Bb36rNUw.js → dist-ByLbEOlF.js} +1 -1
- package/ui/dist/assets/{dist-Dqc_tfzC.js → dist-C9ra6_BD.js} +1 -1
- package/ui/dist/assets/{dist-DfBpjgZT.js → dist-CK8mow8u.js} +1 -1
- package/ui/dist/assets/{dist-G9KNeh6u.js → dist-CQS0So-i.js} +1 -1
- package/ui/dist/assets/{dist-GvRUDL-2.js → dist-CTrwRc3Q.js} +1 -1
- package/ui/dist/assets/{dist-B9_GXR2G.js → dist-DTYj2vUD.js} +1 -1
- package/ui/dist/assets/{dist-Dq4Vnw5N.js → dist-mV0_nQZe.js} +1 -1
- package/ui/dist/assets/index-BzMqkmQ9.js +348 -0
- package/ui/dist/assets/index-DcJR-8Mt.css +2 -0
- package/ui/dist/index.html +2 -2
- package/ui/dist/assets/index-BJ5D2mFK.js +0 -346
- package/ui/dist/assets/index-BXY-eKBn.css +0 -2
|
@@ -8,8 +8,11 @@ description: Turns a human-written test case (cases/<id>.md, the id may include
|
|
|
8
8
|
Read `references/conventions.md` first. It defines the spec shape, fixtures, assertion layers and
|
|
9
9
|
the read-only rules. Read the workspace `AGENTS.md` for the hard rules and the project rules. Read `../exploring-cases/references/open-questions.md`: it
|
|
10
10
|
defines how open questions are written into the case file, resolved and reported. Also read
|
|
11
|
-
`../exploring-cases/references/
|
|
12
|
-
`../exploring-cases/references/
|
|
11
|
+
`../exploring-cases/references/case-rules.md` (how a case's `## Rules` are read, applied and maintained),
|
|
12
|
+
`../exploring-cases/references/assertion-targets.md` (the exact thing each check looks at),
|
|
13
|
+
`../exploring-cases/references/readiness.md` (wait for the content, verify downloads in five parts),
|
|
14
|
+
`../exploring-cases/references/notes.md` (the local exploration note this run reads first) and
|
|
15
|
+
`../exploring-cases/references/tools.md` (the browser tools this MCP can run). All six are binding here.
|
|
13
16
|
|
|
14
17
|
This skill never stops to ask the user in the chat, because it may run without anybody present (a one-shot
|
|
15
18
|
`claude -p` / `opencode run`). This holds whatever mode or agent the run is in: plan mode changes what you may write,
|
|
@@ -36,15 +39,18 @@ and the spec, and runs only the profiles that are logged in, and only when the r
|
|
|
36
39
|
## Before you start
|
|
37
40
|
Read the workspace `AGENTS.md`: its §2 holds the project rules, and it points at the framework baseline. If
|
|
38
41
|
`.claude/skills/generating-specs.md` exists in the workspace, read it and apply it after this skill. Then apply
|
|
39
|
-
`AGENTS.md` §2, which wins over both (the baseline, "Precedence"). F1–F6 always win. This is not a stop
|
|
40
|
-
continue with the procedure below.
|
|
42
|
+
`AGENTS.md` §2, which wins over both (the baseline, "Precedence"). F1–F6 and R2 always win. This is not a stop
|
|
43
|
+
condition: continue with the procedure below.
|
|
41
44
|
|
|
42
45
|
## Inputs
|
|
43
46
|
- The case id and the environment to explore in (`staging` or `production`). There is no default. If the
|
|
44
47
|
environment is missing, stop before exploring and report `RESULT: needs-input` ("pass the environment").
|
|
45
48
|
- Optionally `--force`: regenerate the spec from scratch and overwrite the existing one. See "An existing spec".
|
|
49
|
+
- Optionally `--hints <file>` (or free text after the arguments): what the person knows, or a change they want. Read it
|
|
50
|
+
before validating the case. It guides this run; after the work, a reusable instruction is written into `## Rules`
|
|
51
|
+
by the gate in `../exploring-cases/references/case-rules.md`, and a one-off clue stays in the report.
|
|
46
52
|
- No profile selection: the profile scope is the case's (see above). A case id and an environment are all this command
|
|
47
|
-
takes besides `--force`.
|
|
53
|
+
takes besides `--force` and `--hints`.
|
|
48
54
|
|
|
49
55
|
A case id is its path below `cases/` without `.md`: `hub-landing`, or `documents/landing-page` when the case is in a
|
|
50
56
|
folder (one level only). The spec is at the same path below `tests/` (`tests/documents/landing-page.spec.ts`); both import
|
|
@@ -80,14 +86,21 @@ Decide this first, before exploring anything. The spec path is `tests/<id>.spec.
|
|
|
80
86
|
2. **Validate the case.** Collect everything that is ambiguous, every check without an expected value and every
|
|
81
87
|
missing per-profile value, including the `TODO` items already in the case. Resolve them in the order of
|
|
82
88
|
`open-questions.md`; do not stop here:
|
|
89
|
+
- **Apply the rule pass (`R1`–`R4`).** Read `## Rules` before the steps. A rule that states an expectation, a
|
|
90
|
+
literal, a target or a real step belongs in that section: move it there (or ask). A rule that conflicts with the
|
|
91
|
+
case or the page is a `[blocking]` question, never an override. A rule that would write, send email or do a
|
|
92
|
+
person's step is never followed. Details: `case-rules.md`.
|
|
83
93
|
- **Steps that write or need a choice (`[input]`, `[writes]`) need a recipe.** Each must have at least one
|
|
84
94
|
`- Recipe (phase): ...` line under it in the case (written by `/explore-case`). An old case may still hold the
|
|
85
95
|
same recipe in `## Inputs`: use that line until it is moved under the step. If one has none, the case was
|
|
86
96
|
never explored with a person: stop before opening a browser, do not write the spec, and report `RESULT:
|
|
87
97
|
needs-input` with exactly this: "step <n> has no recorded input; run `/explore-case <id> <env> --attended`
|
|
88
98
|
first". Nothing else is written.
|
|
89
|
-
- Facts the page can answer and that the case does not hold yet are read in step 4 and written into the case
|
|
90
|
-
`(
|
|
99
|
+
- Facts the page can answer and that the case does not hold yet are read in step 4 and written into the case. Mark
|
|
100
|
+
them by the gate in `open-questions.md` ("Which sentences carry a marker"): a locator, a flow or a request is
|
|
101
|
+
`(seen on <profile>)`; a literal the spec asserts gets
|
|
102
|
+
`(observed on <profile>, 🔍 to be confirmed: <what a person must decide>)`, and that is the only thing a person
|
|
103
|
+
has to settle. A spec is generated from observed values.
|
|
91
104
|
- Safe defaults are applied: the suggestion is the sentence above the TODO block, and the block stays in the case
|
|
92
105
|
as `❓ TODO [non-blocking]` with an empty `✍️ Answer:`. A sentence with no TODO block under it is confirmed text.
|
|
93
106
|
- A non-empty `Answer:` is the user's answer: it wins over the sentence above it. Fold it into the case.
|
|
@@ -105,41 +118,47 @@ Decide this first, before exploring anything. The spec path is `tests/<id>.spec.
|
|
|
105
118
|
never for what the check asserts about it: that goes to `test.fixme` with the reason, or `needs-input`.
|
|
106
119
|
- Do not invent expected values. If the case's `## Values` and the spec disagree, the case wins: fix the spec and
|
|
107
120
|
mention the change in the report.
|
|
108
|
-
3. **Check sessions** if `login: required`:
|
|
109
|
-
|
|
110
|
-
information for is never dropped because it is not logged in: write its part from
|
|
111
|
-
`not verified on the page` in the report.
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
121
|
+
3. **Check sessions** if `login: required`: follow `../exploring-cases/references/tools.md`, "Starting a session". A
|
|
122
|
+
profile that needs login is named with `pnpm signin <profile-id>`, and the run continues with the profiles that are
|
|
123
|
+
ready. A profile the case holds information for is never dropped because it is not logged in: write its part from
|
|
124
|
+
the case and mark it `not verified on the page` in the report.
|
|
125
|
+
4. **Replay the case on the real page** with the `playwright-test` MCP, following
|
|
126
|
+
`../exploring-cases/references/tools.md` ("Starting a session", "Short calls") and `readiness.md` ("Exploring"). If
|
|
127
|
+
its tools are not registered in this session at all (no `browser_*` / `generator_*` tool) or it reports that it
|
|
128
|
+
failed to start, do not fall back to ad-hoc Playwright scripts: stop, tell the user which MCP server is missing and
|
|
129
|
+
to run `pnpm asrun doctor`, write no spec, and end with `RESULT: needs-input`.
|
|
130
|
+
**Read each profile's exploration note first** (`../exploring-cases/references/notes.md`): run
|
|
131
|
+
`pnpm asrun explore:notes <case-id> --json`. A fresh note (its `caseSha256` matches the case) supplies the locator,
|
|
132
|
+
the request pattern and the target's match count for a step; confirm each on the page instead of rediscovering it,
|
|
133
|
+
and skip what it does not hold. A missing or stale note means the plain replay below. After confirming an entry of
|
|
134
|
+
a fresh note, you may write the verified locator or request back into that file (`updatedAt` gets the time); never
|
|
135
|
+
add a step or a target, and never touch a stale note. Say in the report what came from a note and what was
|
|
136
|
+
refreshed. Perform the steps in the environment you were given (never ask for it; there is no default).
|
|
137
|
+
Do the read-only steps as written, and follow the `## Rules` that say how to prepare, wait or which control to
|
|
138
|
+
avoid. **Never perform a step that writes or needs a choice**: the agent does not write
|
|
139
|
+
data, in staging or in production, and it does not hand off to a person here or wait for one. A rule never
|
|
140
|
+
authorizes one of those steps. Write those steps
|
|
123
141
|
of the spec from the `- Recipe:` lines under the step (positions, `[asrun]` names, control labels and roles that
|
|
124
142
|
`/explore-case` recorded). Pages that only exist after such a step cannot be seen from here: write their locators
|
|
125
|
-
from
|
|
126
|
-
leave the first run of the spec (or `/heal-spec`) to confirm
|
|
127
|
-
per domain (example.com and exampleshop.com differ in markup). A profile the case holds
|
|
128
|
-
not logged in is skipped and named in the report; its part is written from the case and
|
|
129
|
-
the assertion table. Use this exploration to answer the open points of step 2, and read each
|
|
130
|
-
the case's `## Values`: open that profile's page only when a value or a behavior still has to be
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
143
|
+
from a fresh note first, then from the recipe, and mark them in the report as "observed by explore on <profile>" or
|
|
144
|
+
"not verified on the page"; leave the first run of the spec (or `/heal-spec`) to confirm the latter. Do this for at
|
|
145
|
+
least one logged-in profile per domain (example.com and exampleshop.com differ in markup). A profile the case holds
|
|
146
|
+
information for that is not logged in is skipped and named in the report; its part is written from the case and
|
|
147
|
+
marked `not verified` in the assertion table. Use this exploration to answer the open points of step 2, and read each
|
|
148
|
+
profile's value from the case's `## Values`: open that profile's page only when a value or a behavior still has to be
|
|
149
|
+
resolved for it.
|
|
150
|
+
**Ready before you read, here too**: follow `readiness.md`. The request URL patterns for `actAndWait` come from a
|
|
151
|
+
fresh note or from `browser_network_requests` during this replay, never from a guess; the `Ready when` signal comes
|
|
152
|
+
from the case.
|
|
153
|
+
**Check every target on the page:** follow `../exploring-cases/references/assertion-targets.md`, "What the spec does
|
|
154
|
+
with a card (generating)". Confirm that the card's `Identified by` matches exactly one element and that this element
|
|
155
|
+
is what `What` says (outline it with `locator.highlight()` and look at the screenshot, see `tools.md`). A locator that
|
|
138
156
|
matches several elements or the wrong one is fixed or becomes a question, never `.first()`.
|
|
139
157
|
Prefer `getByTestId` (the project uses `data-tn`), then role/label locators. No CSS chains or XPath.
|
|
140
|
-
Update the case file with what you learned: fold observations into the right section marked
|
|
141
|
-
|
|
142
|
-
and narrow the ones that remain. Keep the shape of
|
|
158
|
+
Update the case file with what you learned: fold observations into the right section, marked `(seen on <profile>)`
|
|
159
|
+
or, for a literal the spec asserts, `(observed on <profile>, 🔍 to be confirmed: <what to decide>)`; remove the
|
|
160
|
+
TODO blocks they answer (rewrite the sentence above the block), and narrow the ones that remain. Keep the shape of
|
|
161
|
+
`open-questions.md` for every block, and never put a marker inside one. Edit the case
|
|
143
162
|
with targeted edits; never touch its `## Source` table.
|
|
144
163
|
5. **Complete the case's `## Values`**: for every profile the case holds information for, add the entry with the
|
|
145
164
|
literals the spec asserts (the same keys as `default`, only what differs); put a literal that holds for every profile
|
|
@@ -154,12 +173,16 @@ Decide this first, before exploring anything. The spec path is `tests/<id>.spec.
|
|
|
154
173
|
(`@asrunjs/asrun/values`), which merges `default` with the running profile's entry.
|
|
155
174
|
Every action that loads something goes through `actAndWait` (`@asrunjs/asrun/ready`) with the key requests and the ready
|
|
156
175
|
signal; every download goes through `verifyDownload` (`@asrunjs/asrun/downloads`); every check on a target has
|
|
157
|
-
`// target: T<n>` on the line before it; every absence check has a `// ready: <signal>` before it
|
|
176
|
+
`// target: T<n>` on the line before it; every absence check has a `// ready: <signal>` before it; code that exists
|
|
177
|
+
only because of a `## Rules` bullet carries `// rule: <short quote>` before it; an assertion whose literal comes
|
|
178
|
+
from a `🔍 to be confirmed` line carries `// unconfirmed: <that case sentence>` before it, so
|
|
179
|
+
`pnpm markers:confirm` can settle it from a passing run. Never
|
|
158
180
|
`waitForTimeout`, `networkidle` or `.first()` to pick an image. A confirmation or modal that may or may not appear is
|
|
159
181
|
never decided by one `isVisible()` look (its `timeout` option is ignored): use `clickOptionalUntilHidden`
|
|
160
182
|
(`@asrunjs/asrun/ready`) or a waiting `expect(...).toBeVisible({timeout})`; see `readiness.md`, "Optional confirmations".
|
|
161
183
|
7. **Check the files**: run `pnpm typecheck`, `pnpm spec:lint <id>` and `pnpm case:check` (or `node --import tsx <framework>/scripts/check-cases.ts` if
|
|
162
|
-
it fails with a sandbox error such as `listen EPERM`).
|
|
184
|
+
it fails with a sandbox error such as `listen EPERM`). When you wrote an exploration note back, also run
|
|
185
|
+
`pnpm asrun explore:notes <id> --check --profile <profile-id>` for each profile you refreshed. Fix every error, including every `FAIL` of the spec lint (a lint failure is never silenced: fix the spec or turn the check
|
|
163
186
|
into a question). Every lint `WARN` is handled too: fix it, or name it in the report with the reason it is acceptable.
|
|
164
187
|
Then do the **self-review**: for each assertion answer in writing "if the product were broken, would this fail?"
|
|
165
188
|
and "if the product is right, could this fail by timing?" (a check that cannot fail, or that races the page, is rewritten).
|
|
@@ -168,14 +191,19 @@ Decide this first, before exploring anything. The spec path is `tests/<id>.spec.
|
|
|
168
191
|
Do NOT run `pnpm asrun`, do not use
|
|
169
192
|
`--update-snapshots` and do not create baselines: the first run of the spec does that, and a person reviews them.
|
|
170
193
|
8. **Report** in the user's language: spec path, **an assertion table** (step, what is asserted, target card, ready signal, and
|
|
171
|
-
`observed on the page` or `not verified`; no row may be missing), profiles explored,
|
|
194
|
+
`observed on the page` or `not verified`; no row may be missing), profiles explored, which steps were written from a
|
|
195
|
+
fresh exploration note and which notes were refreshed (or that there were none), profiles skipped (and why),
|
|
172
196
|
the profiles of `profiles.yaml` the case holds no information for (they run on `default` alone), whether the case file was
|
|
173
|
-
translated, what was added to the case file as
|
|
174
|
-
|
|
175
|
-
the
|
|
197
|
+
translated, what was added to the case file as `(seen on <profile>)` and which lines still need a person
|
|
198
|
+
(`🔍 to be confirmed`, with the reason), checks generated from observations, visual checks that need a baseline on
|
|
199
|
+
the first run, and the remaining TODO blocks (`[blocking]` ones with
|
|
200
|
+
the sentence that deleting the block would confirm). Add the rule table of `case-rules.md`: every `## Rules`
|
|
201
|
+
bullet, where it was applied or why it was not, and which rules were added from `--hints`. State that the spec was
|
|
202
|
+
not run and give the command to run it:
|
|
176
203
|
`pnpm asrun --env <env> --profile <ids> --grep <id>` (with `--update-snapshots` once if the spec has visual
|
|
177
204
|
checks, then run `pnpm baselines:review <id> --new` and open every new image: it must show what its target card says). The last line is `RESULT: ready` (no `[blocking]` question left, spec
|
|
178
|
-
written and the checks above pass) or `RESULT: needs-input`, as defined in `open-questions.md
|
|
205
|
+
written and the checks above pass) or `RESULT: needs-input`, as defined in `open-questions.md`: plain text on the
|
|
206
|
+
last line, never inside a code fence or in emphasis.
|
|
179
207
|
|
|
180
208
|
## Cases that write data
|
|
181
209
|
Only when the case front matter lists environments under `writes` (for example `writes: [staging]`). Never add
|
|
@@ -197,7 +225,8 @@ reason to do nothing when the user asked for something.
|
|
|
197
225
|
Heal from evidence, not from a habit of rerunning. A report that already exists says what failed.
|
|
198
226
|
|
|
199
227
|
1. **Read the user's request first** (the `--hints` file, or the free text). It says what to repair or improve and may
|
|
200
|
-
hold evidence the report lacks.
|
|
228
|
+
hold evidence the report lacks. Read the case's `## Rules` before the steps and apply the rule pass (`R1`–`R4`,
|
|
229
|
+
`../exploring-cases/references/case-rules.md`). Decide which kind of work it is:
|
|
201
230
|
- a failure to repair (the last run failed, or the user names one);
|
|
202
231
|
- an improvement, even when the last run passed: a check that is too loose, a step, expectation, value or target
|
|
203
232
|
that no longer matches the page, wording a person cannot follow;
|
|
@@ -225,10 +254,15 @@ Heal from evidence, not from a habit of rerunning. A report that already exists
|
|
|
225
254
|
leaves out the profiles that are not logged in instead of failing them. A profile that is not logged in is never healed:
|
|
226
255
|
name it in the report with `pnpm signin <profile-id>`.
|
|
227
256
|
4. **Look at the pages when the change depends on them.** Reading a page is not a run and is always allowed, also for an
|
|
228
|
-
improvement request with no failure behind it. Use the `playwright-test` MCP exactly as `/gen-spec` step 4 describes
|
|
229
|
-
|
|
257
|
+
improvement request with no failure behind it. Use the `playwright-test` MCP exactly as `/gen-spec` step 4 describes,
|
|
258
|
+
following `../exploring-cases/references/tools.md` ("Starting a session", "Short calls"), `readiness.md` and the
|
|
259
|
+
exploration note (`../exploring-cases/references/notes.md`). Do the read-only
|
|
230
260
|
steps only: **never perform a step that writes or needs a choice**, in any environment. What you see goes into the
|
|
231
|
-
case
|
|
261
|
+
case by the gate in `open-questions.md` ("Which sentences carry a marker"): `(seen on <profile>)` for a locator, a
|
|
262
|
+
flow or a request, and `(observed on <profile>, 🔍 to be confirmed: <what a person must decide>)` only for a
|
|
263
|
+
literal the spec asserts, so the person settles that in the review gate below. If the note was stale, edit the case
|
|
264
|
+
as usual; if it was fresh, a verified locator may be written back
|
|
265
|
+
(`../exploring-cases/references/notes.md`).
|
|
232
266
|
5. **Audit the assertion before you blame anyone.** Read the `assertion-targets.md` and `readiness.md` sections on healing.
|
|
233
267
|
- Is the locator still what the target card says (`What`, `Identified by`)? Open the failure screenshot and the
|
|
234
268
|
baseline (`pnpm baselines:review <case-id>`) and look at each: a baseline that shows something other than the card
|
|
@@ -250,12 +284,22 @@ Heal from evidence, not from a habit of rerunning. A report that already exists
|
|
|
250
284
|
more than a sentence (a new or removed step, different expected values, another target), regenerate the spec from
|
|
251
285
|
the case as "## Procedure" steps 5–6 describes instead of patching it; otherwise edit it in place. Say in the
|
|
252
286
|
report which of the two you did.
|
|
287
|
+
- A reusable instruction that is method, not an expectation: append it to `## Rules` under the four-question gate
|
|
288
|
+
of `case-rules.md`, in general terms, after checking that it is not already there. An agent-inferred rule
|
|
289
|
+
carries a marker only when it changes what is asserted; a method rule is `(seen on <profile>)` and a rule the
|
|
290
|
+
person stated carries nothing. A one-off request (this failure, this
|
|
291
|
+
account, this run) stays in the report. Never put a write, an email, a person's step, an expectation or a literal
|
|
292
|
+
into `## Rules`.
|
|
253
293
|
7. **Stop at the review gate.** After editing the case, read it back. If **this run** left a `🚫 TODO [blocking]` or
|
|
254
|
-
`❓ TODO [non-blocking]` block, or a `🔍 to be confirmed
|
|
294
|
+
`❓ TODO [non-blocking]` block, or a `🔍 to be confirmed` or `👁️ not observed` marker, do not edit or
|
|
255
295
|
regenerate the spec any further: a case that still asks something is not a source for a spec. Report exactly what
|
|
256
296
|
to review, that answering or deleting those blocks comes first, and that the next command is
|
|
257
|
-
`/explore-case <case-id> <env>` (with `--attended` when a step writes data)
|
|
258
|
-
|
|
297
|
+
`/explore-case <case-id> <env>` (with `--attended` when a step writes data), or
|
|
298
|
+
`pnpm markers:confirm --case <case-id> --env <env>` when a passing run already checks those lines. End with
|
|
299
|
+
`RESULT: needs-input`. Blocks and markers that were already in the case before this run and that it did not touch do
|
|
300
|
+
not stop it. A plain `(seen on <profile>)` note never stops it. One exception:
|
|
301
|
+
a `🔍 to be confirmed` marker inside `## Rules` does not stop the spec work unless the rule covers a literal the
|
|
302
|
+
spec asserts (a `🚫`/`❓` TODO block under a rule still does).
|
|
259
303
|
8. **After editing the spec, run it once to confirm** (same command as in step 3, for the profiles you healed). That rerun
|
|
260
304
|
is the only one that is always allowed. Run `pnpm typecheck`, `pnpm spec:lint <case-id>` and `pnpm case:check`, and
|
|
261
305
|
fix every failure of the checks you touched. An improvement may make the run fail on purpose: a tightened check that
|
|
@@ -264,8 +308,9 @@ Heal from evidence, not from a habit of rerunning. A report that already exists
|
|
|
264
308
|
whether the case was run again and why (or why not), what the request asked for and what came of it, what was
|
|
265
309
|
changed in the case and in the spec (and whether the spec was regenerated), what is a product finding (with paths),
|
|
266
310
|
which assertions were wrong or raced the page, which baselines are wrong and were deleted, what the user's request
|
|
267
|
-
contributed,
|
|
268
|
-
|
|
311
|
+
contributed, the rule table of `case-rules.md` (every `## Rules` bullet, where it was applied or why it was not,
|
|
312
|
+
and which rules this run added or changed), which profiles were left out because they are not logged in, and the
|
|
313
|
+
result of the confirming run. The last line is `RESULT: ready` (nothing left for a person, the spec changed and its checks pass)
|
|
269
314
|
or `RESULT: needs-input` (the case waits for a review, as in step 7, or a person has to decide something). A run
|
|
270
315
|
that could not write files at all (plan mode, a read-only sandbox, a refused permission) is `needs-input` too:
|
|
271
316
|
never end with a plan and a question instead.
|
|
@@ -108,6 +108,10 @@ Binding rules are in `../exploring-cases/references/readiness.md` and `.../asser
|
|
|
108
108
|
- A check on a logo, image, file or screenshot has `// target: T<n>` on the line before it (a card in the case's
|
|
109
109
|
`## Targets`). The element is found by the card's `Identified by` (role and name, test id, alt/title), never `.first()` or
|
|
110
110
|
a bare `img`/`svg`. Semantic assertion first (name/alt/src, where it is), pixels second, on that element only.
|
|
111
|
+
- An assertion whose literal comes from a `🔍 to be confirmed` line in the case carries
|
|
112
|
+
`// unconfirmed: <that case sentence>` before it (one line, the sentence without its marker). It is what
|
|
113
|
+
`pnpm markers:confirm --case <id> --env <env>` reads after a passing run to settle the marker, so the line must be
|
|
114
|
+
the case's own wording, not a paraphrase.
|
|
111
115
|
- A check for absence ("no", "missing", "empty", "hidden") has `// ready: <signal>` in front of it and a positive wait
|
|
112
116
|
before it. The page must have loaded before "nothing" means anything.
|
|
113
117
|
- A download is `await verifyDownload(page, () => option.click(), {kind: 'pdf'})`. It checks, and fails the step on
|
|
@@ -10,14 +10,16 @@ Read the workspace `AGENTS.md` (hard rules and project rules) and `cases/_templa
|
|
|
10
10
|
- `references/writes-and-email.md`: how to classify steps that write data or send email, and the data rules.
|
|
11
11
|
- `references/gaps.md`: what to ask, and the default to suggest for each question.
|
|
12
12
|
- `references/splitting.md`: how many cases to make.
|
|
13
|
+
- `../exploring-cases/references/case-rules.md`: what belongs in `## Rules` (a method constraint), and what stays in
|
|
14
|
+
Steps, Expected, Values or Targets.
|
|
13
15
|
|
|
14
16
|
This skill writes `cases/*.md` only. It never writes specs, `profiles.yaml`, or anything under `tests/`.
|
|
15
17
|
|
|
16
18
|
## Before you start
|
|
17
19
|
Read the workspace `AGENTS.md`: its §2 holds the project rules, and it points at the framework baseline. If
|
|
18
20
|
`.claude/skills/making-cases.md` exists in the workspace, read it and apply it after this skill. Then apply `AGENTS.md`
|
|
19
|
-
§2, which wins over both (the baseline, "Precedence"). F1–F6 always win. This is not a stop condition:
|
|
20
|
-
the procedure below.
|
|
21
|
+
§2, which wins over both (the baseline, "Precedence"). F1–F6 and R2 always win. This is not a stop condition:
|
|
22
|
+
continue with the procedure below.
|
|
21
23
|
|
|
22
24
|
## Inputs
|
|
23
25
|
- Raw notes: pasted text or a file path. Any language, any quality.
|
|
@@ -34,10 +36,11 @@ the procedure below.
|
|
|
34
36
|
creates, edits, publishes, shares, deletes or sends is not read-only, even if the notes do not say so.
|
|
35
37
|
4. **Find the gaps** (`references/gaps.md`): missing expected results, unknown values a spec must assert, unclear starting
|
|
36
38
|
state, steps that need data that may not exist, ordering that is not linear. Never invent an expected value.
|
|
37
|
-
Unknown values are written as the suggested default sentence with a `❓ TODO` block under it (see `gaps.md`)
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
`
|
|
39
|
+
Unknown values are written as the suggested default sentence with a `❓ TODO` block under it (see `gaps.md`). A fact
|
|
40
|
+
from the notes that nobody questions is a plain note `(seen on <profile>)`; only a literal the spec will assert and
|
|
41
|
+
that would pass while wrong gets `(observed on <profile>, 🔍 to be confirmed: <what to decide>)`. A marker never
|
|
42
|
+
goes inside a TODO block. A literal nobody knows yet goes into the case's `## Values` as `TODO` (the spec then fails
|
|
43
|
+
with a clear message until it is filled), never into `profiles.yaml`.
|
|
41
44
|
5. **Ask before writing.** Show the user a short plan and the open questions in one message: the proposed split
|
|
42
45
|
into cases, the classification (which steps write, which send email), and the gaps with your suggested default
|
|
43
46
|
for each. Wait for answers. Do not write case files before that.
|
|
@@ -48,10 +51,14 @@ the procedure below.
|
|
|
48
51
|
ask in step 5. The group does not repeat in the name: `documents/landing-page`, not `documents/documents-landing-page`.
|
|
49
52
|
- Front matter: `id`, `title`, `login`, `suites`, `writes`, `writes-note` (required when `writes` is not `none`),
|
|
50
53
|
`envs` when a case must not run everywhere. When unsure: `suites: [regression]`, no `writes` (read-only), or `writes: [staging]` when a step writes.
|
|
51
|
-
- Fill the sections that apply (Goal, Steps, Expected, Values, Visual checks, Source, Notes).
|
|
54
|
+
- Fill the sections that apply (Goal, Steps, Expected, Rules, Values, Visual checks, Source, Notes).
|
|
52
55
|
Remove template placeholders you do not use. Use `## Values` only for the literals a spec asserts; `default`
|
|
53
56
|
applies to every profile, and a profile id entry holds only what differs. Do not add `- Recipe:` lines: a person
|
|
54
57
|
records them during `/explore-case --attended`.
|
|
58
|
+
- Put a note in `## Rules` only when it states a **method** constraint (dismiss a banner before step 1, never use
|
|
59
|
+
an entry that needs 2FA). A constraint that is part of the tested flow is a step; one that creates, changes or
|
|
60
|
+
deletes data is a tagged step with a later recipe. Never invent a rule, and never put an expectation or a literal
|
|
61
|
+
there: those go to `## Expected` or `## Values`. The note that became a rule is listed in its `## Source` row.
|
|
55
62
|
- Case ids are lowercase kebab-case and name the thing, not the ticket: `landing-page`.
|
|
56
63
|
- Existing case files: do not overwrite. If a new case overlaps one (for example another case about the same page), say so,
|
|
57
64
|
propose "keep", "merge" or "replace", and wait for the user.
|
|
@@ -17,7 +17,7 @@ single lines. Never ask a question whose answer is already in the notes, `AGENTS
|
|
|
17
17
|
| Environments | Must this case run in production, or only staging? | All environments; production writes need the usual confirmation |
|
|
18
18
|
| Suites | Smoke, regression, or both? | A read-only landing check: `[smoke, regression]`; a flow that writes: `[regression]` |
|
|
19
19
|
| Profiles | Do all profiles see the same UI? | Yes; the exact literals that differ go in `## Values`, never in the steps |
|
|
20
|
-
| Unclear words | What does "collaborator button" / "..." button open? | Describe
|
|
20
|
+
| Unclear words | What does "collaborator button" / "..." button open? | Describe the safest reading and hang a TODO under it; a detail nobody questions is a plain note `(seen on <profile>)` |
|
|
21
21
|
| Unused lines | A statement that fits no case | Ask where it belongs; never drop it silently |
|
|
22
22
|
|
|
23
23
|
## Marking unknowns in a case
|
|
@@ -26,11 +26,14 @@ single lines. Never ask a question whose answer is already in the notes, `AGENTS
|
|
|
26
26
|
|
|
27
27
|
```
|
|
28
28
|
- The preview header shows the product logo, visible and not empty.
|
|
29
|
-
- ❓ TODO [non-blocking]: Which logo must it be? (
|
|
29
|
+
- ❓ TODO [non-blocking]: Which logo must it be? (the notes say only "the logo")
|
|
30
30
|
- ✍️ Answer:
|
|
31
31
|
```
|
|
32
32
|
|
|
33
33
|
The user accepts the default by deleting the block, or types an answer. Never write a bare `TODO: ...` line, a
|
|
34
|
-
question as the only text, or `Suggested:`.
|
|
35
|
-
- Seen on a page during exploration but not stated in the notes: `- <observation> (
|
|
34
|
+
question as the only text, or `Suggested:`. Never put a marker word inside the block: one claim carries one signal.
|
|
35
|
+
- Seen on a page during exploration but not stated in the notes: `- <observation> (seen on <profile>)`. A literal the
|
|
36
|
+
spec will assert is different: `(observed on <profile>, 🔍 to be confirmed: <what a person must decide>)`, and only
|
|
37
|
+
when a wrong reading would still pass. The gate is "Which sentences carry a marker" in
|
|
38
|
+
`exploring-cases/references/open-questions.md`.
|
|
36
39
|
- Never turn an observation into a hard requirement on your own.
|
|
@@ -34,6 +34,6 @@ Original notes, verbatim. Do not edit this table; it only exists so a reviewer c
|
|
|
34
34
|
```
|
|
35
35
|
|
|
36
36
|
- The original text is copied character by character, in a code span, in its original language.
|
|
37
|
-
- "Became" points at step numbers, or says "Expected", "Visual checks", "Not used: <reason>".
|
|
37
|
+
- "Became" points at step numbers, or says "Expected", "Rules", "Visual checks", "Not used: <reason>".
|
|
38
38
|
- A statement used by two cases appears in both tables.
|
|
39
39
|
- A statement that is not used anywhere is a gap: ask about it, do not silently drop it.
|
package/AGENTS.md
CHANGED
|
@@ -11,7 +11,7 @@ project that uses asrun, the framework files are in `node_modules/@asrunjs/asrun
|
|
|
11
11
|
## Precedence
|
|
12
12
|
|
|
13
13
|
Rules are applied in this order; a later layer wins where they conflict.
|
|
14
|
-
1. **F1–F6: safety.** Nothing overrides them, in any project. They are enforced in code as well
|
|
14
|
+
1. **F1–F6 and R2: safety.** Nothing overrides them, in any project. They are enforced in code as well
|
|
15
15
|
(`lib/write-gate.ts`, `lib/session.ts`, the run filters), so a rule file cannot weaken them.
|
|
16
16
|
2. **This file.** The generic facts and defaults.
|
|
17
17
|
3. **The framework skills** (`<framework>/.claude/skills/**`).
|
|
@@ -20,8 +20,11 @@ Rules are applied in this order; a later layer wins where they conflict.
|
|
|
20
20
|
5. **The project's `AGENTS.md` §2 (Project rules).** It states the project's intent and wins over layers 2–4.
|
|
21
21
|
|
|
22
22
|
A project writes §2 only for what differs from the defaults here, names the rule id, and says whether it `narrows` or
|
|
23
|
-
`replaces` the rule. `replaces` is refused for F1–F6; `pnpm asrun doctor` checks the table. A rule file never
|
|
24
|
-
the structural tokens under "Fixed tokens".
|
|
23
|
+
`replaces` the rule. `replaces` is refused for F1–F6 and R2; `pnpm asrun doctor` checks the table. A rule file never
|
|
24
|
+
renames the structural tokens under "Fixed tokens".
|
|
25
|
+
|
|
26
|
+
Inside a case, `## Rules` is a case-local layer below this file and the project's §2: it says how to run that case,
|
|
27
|
+
never what is true (`R1`), and a rule that conflicts with the rest of the case is a question, not an override.
|
|
25
28
|
|
|
26
29
|
## Hard rules
|
|
27
30
|
- **F1 — Read-only by default.** [non-negotiable] Tests never create, edit, delete, submit forms that change data,
|
|
@@ -62,8 +65,12 @@ These apply to every agent that drives a browser to explore pages for cases or s
|
|
|
62
65
|
Everything that writes is the person's (see F3).
|
|
63
66
|
- **B2 — Waiting for a person.** Only in an attended run. Announce the task in the browser window (a small draggable
|
|
64
67
|
card with a Done button and click-to-copy for texts to type), name the finish condition, wait, and tell the person
|
|
65
|
-
when control is back.
|
|
66
|
-
|
|
68
|
+
when control is back. The card starts and stops the recording itself and returns `recorded`; never call
|
|
69
|
+
`browser_start_recording` or `browser_stop_recording`: the `playwright-test` MCP lists many browser tools it cannot
|
|
70
|
+
run (`devtools`, `storage`, `network`, `pdf`, `vision`, and the `skillOnly` ones such as `browser_network_clear`),
|
|
71
|
+
and the substitutes for each are in `<framework>/.claude/skills/exploring-cases/references/tools.md`. Record what
|
|
72
|
+
they did as a general recipe under the step (`- Recipe (phase): ...`): positions instead of names of data that can
|
|
73
|
+
change.
|
|
67
74
|
- **B3 — Closing an email composer.** When closing an email composer makes a confirm dialog appear (for example
|
|
68
75
|
"Discard draft?"), choose "Discard" and continue. Never choose "Save" or "Send"; closing the composer must not keep
|
|
69
76
|
or send anything. The dialog can show up late: in a spec, clicking "Discard" and checking that the composer is
|
|
@@ -104,27 +111,60 @@ Details: `<framework>/.claude/skills/exploring-cases/references/assertion-target
|
|
|
104
111
|
- **A6 — Observe first, delete last.** Everything that needs the data (preview, share menus, composers, downloads) is
|
|
105
112
|
observed before the person deletes it.
|
|
106
113
|
|
|
114
|
+
## Case rules (`R`, binding for exploring, generating, healing and making cases)
|
|
115
|
+
A case document may hold a `## Rules` section: case-local instructions a person writes for the agents. They say **how**
|
|
116
|
+
to run that case, never **what is true**. Details: `<framework>/.claude/skills/exploring-cases/references/case-rules.md`.
|
|
117
|
+
- **R1 — Rules describe the method, not the outcome.** `## Steps`, `## Expected`, `## Values` and `## Targets` stay the
|
|
118
|
+
only place for behaviour, expectations and literals. A rule that contradicts them is a `🚫 TODO [blocking]` question,
|
|
119
|
+
never a silent override. A sentence that says something must be true belongs in `## Expected` (with a target card when
|
|
120
|
+
it looks at an image, file, PDF, screenshot or repeated text).
|
|
121
|
+
- **R2 — Rules never authorize what F1–F6 forbid.** [non-negotiable] A rule cannot allow creating, changing or
|
|
122
|
+
deleting data, sending email, accepting terms, or doing a person's step. Those stay tagged steps (`[writes]`,
|
|
123
|
+
`[email]`, `[input]`) with their `- Recipe:` lines and the front-matter `writes`/`writes-note` the case needs. The
|
|
124
|
+
agent never performs them while exploring or generating, whatever a rule says.
|
|
125
|
+
- **R3 — Every AI step reads every rule and accounts for it.** `/explore-case`, `/gen-spec` and `/heal-spec` (and
|
|
126
|
+
`/make-case` when it edits an existing case) read the whole `## Rules` section before they start, apply it, and list
|
|
127
|
+
each rule in the final report with where it was applied or why it was not. Exploration re-checks each rule against
|
|
128
|
+
the page: a rule that no longer holds gets a TODO with evidence, never silent deletion or rewriting. The agent may fix
|
|
129
|
+
a typo and may add a question under a rule; it never changes what a rule says.
|
|
130
|
+
- **R4 — Only reusable method is written back.** When a person supplies instructions before or during a run (`--hints`,
|
|
131
|
+
free text, an answer in the case), the agent appends to `## Rules` only what a future run of the same case needs,
|
|
132
|
+
stated in general terms (no account, date, run id or one-off failure), and only when it is method rather than an
|
|
133
|
+
expectation. A rule the agent inferred from a page carries a marker only when the rule changes what is asserted
|
|
134
|
+
(`(observed on <profile>, 🔍 to be confirmed: <what a person must decide>)`); a method rule whose wrong version would
|
|
135
|
+
fail the next run is a plain note `(seen on <profile>)`, and a rule the person stated carries nothing. Run-specific
|
|
136
|
+
information stays in the report. A marker inside `## Rules` does not stop spec generation the way an open question
|
|
137
|
+
about an assertion does.
|
|
138
|
+
|
|
107
139
|
## Fixed tokens
|
|
108
140
|
`lib/case-doc.ts`, the scripts and the web panel parse these literals. A project language never translates them, and
|
|
109
141
|
a project rule never renames them.
|
|
110
|
-
- Section headings: `## Goal`, `## Steps`, `## Expected`, `## Values`, `## Visual checks`, `## Targets`,
|
|
111
|
-
`## Source` (only when the case was made from notes).
|
|
142
|
+
- Section headings: `## Goal`, `## Rules`, `## Steps`, `## Expected`, `## Values`, `## Visual checks`, `## Targets`,
|
|
143
|
+
`## Notes`, `## Source` (only when the case was made from notes).
|
|
112
144
|
- A step's recipe lines: `- Recipe (phase): ...` under the step it belongs to.
|
|
113
145
|
- `### T<n>` target cards and the `[T<n>]` references to them.
|
|
114
146
|
- Question blocks: `🚫 TODO [blocking]`, `❓ TODO [non-blocking]`, `✍️ Answer:` (with its emoji), and the markers
|
|
115
|
-
`🔍 to be confirmed
|
|
147
|
+
`🔍 to be confirmed` and `👁️ not observed`. A marker may carry its reason and a screenshot in the same
|
|
148
|
+
parentheses: `(observed on <profile>, 🔍 to be confirmed: <what a person must decide>, evidence: <path>)`.
|
|
149
|
+
`(seen on <profile>)` is not a token: it is an ordinary note.
|
|
116
150
|
- Step tags `[input]`, `[writes]`, `[email]`, and the last line `RESULT: ready` / `RESULT: needs-input`.
|
|
151
|
+
The RESULT line stands on its own: nothing in front of it on that line (no label, in any language), nothing after
|
|
152
|
+
it but closing punctuation, and plain text: never inside a code fence or in emphasis.
|
|
117
153
|
- Front-matter keys: `id`, `title`, `login`, `suites`, `writes`, `writes-note`, `envs`.
|
|
118
154
|
|
|
119
155
|
## Where the details are
|
|
120
156
|
This file holds the rules. The exact shape those rules apply to is documented once, and not repeated here:
|
|
121
|
-
- `docs/agents/case-format.md`: the case document contract (front matter, steps, `## Values`, target cards,
|
|
122
|
-
spec's consumption of them, the write gate). Read it before editing a case document or a generated spec.
|
|
157
|
+
- `docs/agents/case-format.md`: the case document contract (front matter, steps, `## Rules`, `## Values`, target cards,
|
|
158
|
+
the spec's consumption of them, the write gate). Read it before editing a case document or a generated spec.
|
|
159
|
+
- `.claude/skills/exploring-cases/references/case-rules.md`: how a case's `## Rules` are read, applied and maintained.
|
|
160
|
+
- `.claude/skills/exploring-cases/references/tools.md`: which browser tools the `playwright-test` MCP can run, the
|
|
161
|
+
substitute for each one it only lists, and the arguments every call has to carry. Read it before the first browser
|
|
162
|
+
call of a run.
|
|
123
163
|
- `docs/commands.md`: every command and flag.
|
|
124
164
|
- `docs/cases.md`: the same case format for the people who write cases by hand.
|
|
125
165
|
- `docs/agents/framework-dev.md`: the framework's own layout and how to change it.
|
|
126
166
|
The project's own `AGENTS.md` §2 (project rules) and §3 (project facts) still win over the documentation where they
|
|
127
|
-
differ, and `F1`–`F6` win over everything.
|
|
167
|
+
differ, and `F1`–`F6` and `R2` win over everything.
|
|
128
168
|
|
|
129
169
|
## Layout
|
|
130
170
|
|
|
@@ -144,15 +184,16 @@ Three steps, each learns one thing: the intent, the facts and inputs, the implem
|
|
|
144
184
|
1. `making-cases` (or `/make-case <notes>`) turns raw notes into case documents. It writes `cases/*.md` only and
|
|
145
185
|
never opens a browser. Check them with `pnpm case:check`.
|
|
146
186
|
2. `exploring-cases` (or `/explore-case <case-id> <env> [--attended] [--profiles a,b]`) explores the real pages for one
|
|
147
|
-
case. It edits that case file
|
|
148
|
-
of the steps that write or need a choice
|
|
149
|
-
can be explored in
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
187
|
+
case. It edits that case file: wording, observations, TODO blocks, and, with `--attended`, the `- Recipe:` lines
|
|
188
|
+
of the steps that write or need a choice, and it writes a local note under `.explore/notes/` with the locators and
|
|
189
|
+
data-call paths it saw. `--profiles` narrows the run to those profiles; runs add up, so the case can be explored in
|
|
190
|
+
passes.
|
|
191
|
+
3. `generating-specs` (or `/gen-spec <case-id> <env>`) turns the case into a spec. It replays the case, its
|
|
192
|
+
step recipes and the exploration note, and `## Values`, writes the spec (values stay in the case document), but never
|
|
193
|
+
runs the tests, never writes data and never creates baselines. There is no profile selection here: it covers every
|
|
194
|
+
profile the case names, which is why a case explored in passes generates one spec for all of them. A tagged step with
|
|
195
|
+
no `- Recipe:` line (an old `## Inputs` line counts until it is moved) stops with `needs-input`: run
|
|
196
|
+
`/explore-case` first. Running specs is a separate step (`pnpm asrun`, or the Run page).
|
|
156
197
|
Workspace maintenance is separate from the AI steps: `pnpm asrun upgrade` installs the newest framework, refreshes the
|
|
157
198
|
generated files and applies the release's workspace migrations (it checks the target release for breaking changes
|
|
158
199
|
before installing), `pnpm asrun init [--update] [--force]` writes or refreshes the workspace files, `pnpm asrun
|
package/README.md
CHANGED
|
@@ -99,11 +99,12 @@ exempts `@asrunjs/asrun` from pnpm's 24-hour release age, so a fresh release ins
|
|
|
99
99
|
3. **Write a case.** `pnpm case:new header-logo` creates `cases/header-logo.md` from the template; or start from raw
|
|
100
100
|
notes with `/make-case notes.md`. Fill in the steps, what is expected and the exact values.
|
|
101
101
|
4. **Check it against the real pages.** `/explore-case header-logo staging` opens the pages (with your saved session),
|
|
102
|
-
writes what they show into the case and turns what it cannot know into questions.
|
|
103
|
-
|
|
104
|
-
with the others adds theirs, and the case
|
|
105
|
-
|
|
106
|
-
|
|
102
|
+
writes what they show into the case and turns what it cannot know into questions. It also saves a local note of the
|
|
103
|
+
locators and data calls it saw. Answer the questions in the case file, or in the UI's **Answer** sheet, and run it
|
|
104
|
+
again. With many profiles, `--profiles a,b` opens only those; a later run with the others adds theirs, and the case
|
|
105
|
+
keeps both.
|
|
106
|
+
5. **Generate, run, read.** `/gen-spec header-logo staging` reads those notes, writes `tests/header-logo.spec.ts` and
|
|
107
|
+
fills the case's `## Values`. Then:
|
|
107
108
|
|
|
108
109
|
```bash
|
|
109
110
|
pnpm asrun --env staging --grep header-logo
|
|
@@ -133,12 +134,13 @@ It is read after the framework skill and before your project rules. Write only w
|
|
|
133
134
|
|
|
134
135
|
Rules combine in this order, and a later layer wins where they conflict:
|
|
135
136
|
|
|
136
|
-
1. `F1`–`F6` of the framework baseline (safety, in every project). They cannot be changed.
|
|
137
|
+
1. `F1`–`F6` and `R2` of the framework baseline (safety, in every project). They cannot be changed.
|
|
137
138
|
2. The framework baseline, `node_modules/@asrunjs/asrun/AGENTS.md`.
|
|
138
139
|
3. The framework skill.
|
|
139
140
|
4. Your override, `.claude/skills/<skill>.md`.
|
|
140
|
-
5. Your own `AGENTS.md` §2: a table of rule ids (`F7`–`F10`) with `narrows` or `replaces`, for example
|
|
141
|
-
language.
|
|
141
|
+
5. Your own `AGENTS.md` §2: a table of rule ids (`F7`–`F10`, `R1`, `R3`, `R4`) with `narrows` or `replaces`, for example
|
|
142
|
+
the artifact language.
|
|
143
|
+
6. A case's `## Rules`: how to run that one case, never what is true.
|
|
142
144
|
|
|
143
145
|
`pnpm asrun doctor` validates the table and lists the overrides it sees. An update never rewrites your rules:
|
|
144
146
|
`init --force` refuses `AGENTS.md`, `CLAUDE.md` and `.claude/skills/**` without `--reset-rules`. Never edit the
|
package/bin/asrun.mjs
CHANGED
|
@@ -23,9 +23,11 @@ const SCRIPTS = {
|
|
|
23
23
|
status: 'status.js',
|
|
24
24
|
report: 'report.js',
|
|
25
25
|
explore: 'explore.js',
|
|
26
|
+
'explore:notes': 'explore-notes.js',
|
|
26
27
|
cleanup: 'cleanup.js',
|
|
27
28
|
'case:new': 'new-case.js',
|
|
28
29
|
'case:check': 'check-cases.js',
|
|
30
|
+
'markers:confirm': 'markers-confirm.js',
|
|
29
31
|
'spec:lint': 'lint-specs.js',
|
|
30
32
|
'baselines:review': 'review-baselines.js',
|
|
31
33
|
'pdf:render': 'render-pdf.js',
|