@asrunjs/asrun 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +11 -6
- package/.claude/commands/gen-spec.md +10 -6
- package/.claude/commands/heal-spec.md +13 -9
- package/.claude/commands/make-case.md +1 -1
- package/.claude/skills/exploring-cases/SKILL.md +84 -63
- package/.claude/skills/exploring-cases/references/assertion-targets.md +8 -4
- package/.claude/skills/exploring-cases/references/case-rules.md +110 -0
- package/.claude/skills/exploring-cases/references/handoff-card.js +73 -11
- package/.claude/skills/exploring-cases/references/notes.md +87 -0
- package/.claude/skills/exploring-cases/references/open-questions.md +58 -6
- package/.claude/skills/exploring-cases/references/readiness.md +7 -5
- package/.claude/skills/exploring-cases/references/tools.md +151 -0
- package/.claude/skills/exploring-cases/references/user-handoff.md +47 -18
- package/.claude/skills/generating-specs/SKILL.md +99 -54
- package/.claude/skills/generating-specs/references/conventions.md +4 -0
- package/.claude/skills/making-cases/SKILL.md +14 -7
- package/.claude/skills/making-cases/references/gaps.md +7 -4
- package/.claude/skills/making-cases/references/normalizing.md +1 -1
- package/AGENTS.md +61 -20
- package/README.md +10 -8
- package/bin/asrun.mjs +2 -0
- package/dist/lib/case-doc.d.ts +41 -4
- package/dist/lib/case-doc.js +109 -17
- package/dist/lib/explore-notes.d.ts +74 -0
- package/dist/lib/explore-notes.js +242 -0
- package/dist/lib/paths.d.ts +2 -0
- package/dist/lib/paths.js +2 -0
- package/dist/scripts/check-cases.js +89 -1
- package/dist/scripts/cli.js +9 -2
- package/dist/scripts/doctor.js +50 -3
- package/dist/scripts/explore-notes.d.ts +1 -0
- package/dist/scripts/explore-notes.js +131 -0
- package/dist/scripts/markers-confirm.d.ts +1 -0
- package/dist/scripts/markers-confirm.js +139 -0
- package/dist/ui/server/actions.js +50 -16
- package/dist/ui/server/app.js +7 -2
- package/dist/ui/server/job-store.d.ts +60 -0
- package/dist/ui/server/job-store.js +198 -0
- package/dist/ui/server/jobs.d.ts +46 -4
- package/dist/ui/server/jobs.js +361 -59
- package/dist/ui/server/process-info.d.ts +17 -0
- package/dist/ui/server/process-info.js +126 -0
- package/dist/ui/server/result-state.d.ts +10 -0
- package/dist/ui/server/result-state.js +106 -0
- package/dist/ui/shared/ai-commands.d.ts +4 -1
- package/dist/ui/shared/ai-commands.js +14 -4
- package/dist/ui/shared/types.d.ts +17 -2
- package/docs/agents/case-format.md +40 -5
- package/docs/agents/framework-dev.md +4 -0
- package/docs/ai-workflow.md +56 -16
- package/docs/cases.md +32 -8
- package/docs/commands.md +17 -1
- package/docs/theme-guide.html +56 -0
- package/docs/troubleshooting.md +40 -0
- package/docs/web-ui.md +59 -15
- package/package.json +3 -1
- package/templates/.claude/commands/explore-case.md +11 -6
- package/templates/.claude/commands/gen-spec.md +10 -5
- package/templates/.claude/commands/heal-spec.md +11 -7
- package/templates/.claude/commands/make-case.md +1 -1
- package/templates/AGENTS.md +9 -6
- package/templates/cases/_template.md +7 -0
- package/templates/package.json +2 -0
- package/ui/dist/assets/{dist-WvvCZeeu.js → dist--BXJInTq.js} +1 -1
- package/ui/dist/assets/{dist-B7HeglDd.js → dist-BMmNFIUH.js} +1 -1
- package/ui/dist/assets/{dist-DevASegc.js → dist-BXNaYHeh.js} +1 -1
- package/ui/dist/assets/{dist-Bb36rNUw.js → dist-ByLbEOlF.js} +1 -1
- package/ui/dist/assets/{dist-Dqc_tfzC.js → dist-C9ra6_BD.js} +1 -1
- package/ui/dist/assets/{dist-DfBpjgZT.js → dist-CK8mow8u.js} +1 -1
- package/ui/dist/assets/{dist-G9KNeh6u.js → dist-CQS0So-i.js} +1 -1
- package/ui/dist/assets/{dist-GvRUDL-2.js → dist-CTrwRc3Q.js} +1 -1
- package/ui/dist/assets/{dist-B9_GXR2G.js → dist-DTYj2vUD.js} +1 -1
- package/ui/dist/assets/{dist-Dq4Vnw5N.js → dist-mV0_nQZe.js} +1 -1
- package/ui/dist/assets/index-BzMqkmQ9.js +348 -0
- package/ui/dist/assets/index-DcJR-8Mt.css +2 -0
- package/ui/dist/index.html +2 -2
- package/ui/dist/assets/index-BJ5D2mFK.js +0 -346
- package/ui/dist/assets/index-BXY-eKBn.css +0 -2
package/docs/cases.md
CHANGED
|
@@ -31,6 +31,9 @@ suites: [smoke, regression]
|
|
|
31
31
|
## Goal
|
|
32
32
|
Why this case exists, in one or two sentences.
|
|
33
33
|
|
|
34
|
+
## Rules
|
|
35
|
+
- Before step 1: dismiss the cookie banner; it covers the header.
|
|
36
|
+
|
|
34
37
|
## Steps
|
|
35
38
|
1. Open `/`.
|
|
36
39
|
- Ready when: the header has rendered its menu.
|
|
@@ -63,8 +66,8 @@ example-staging:
|
|
|
63
66
|
Anything a reviewer should know.
|
|
64
67
|
````
|
|
65
68
|
|
|
66
|
-
The headings are fixed English tokens (`Goal`, `Steps`, `Expected`, `Values`, `Visual checks`, `Targets`,
|
|
67
|
-
and `Source` when the case was made from raw notes). A project language never translates them.
|
|
69
|
+
The headings are fixed English tokens (`Goal`, `Rules`, `Steps`, `Expected`, `Values`, `Visual checks`, `Targets`,
|
|
70
|
+
`Notes`, and `Source` when the case was made from raw notes). A project language never translates them.
|
|
68
71
|
|
|
69
72
|
## Front matter
|
|
70
73
|
|
|
@@ -80,6 +83,21 @@ and `Source` when the case was made from raw notes). A project language never tr
|
|
|
80
83
|
|
|
81
84
|
`priority: <level>` is the old name of `suites`; it is read as `suites: [<level>]`. Do not use both.
|
|
82
85
|
|
|
86
|
+
## Rules
|
|
87
|
+
|
|
88
|
+
Optional, case-local instructions for the AI agents that explore, generate and heal this case: **how** to run it, never
|
|
89
|
+
**what it proves**. One bullet per rule. A bullet may start with `Step 3:`, `Before step 3:` or `After step 3:` to tie
|
|
90
|
+
it to that step; `pnpm case:check` fails when the step does not exist. A rule without a prefix applies to the whole case.
|
|
91
|
+
|
|
92
|
+
- Use it for a page quirk, a mandatory read-only preparation (dismiss a cookie banner), a control to avoid, or a
|
|
93
|
+
waiting/attention hint.
|
|
94
|
+
- Do not put an expectation, a literal or a target here: those stay in `## Expected`, `## Values` and `## Targets`.
|
|
95
|
+
- A rule never allows creating, changing or deleting data, sending email or doing a person's step. Those stay tagged
|
|
96
|
+
steps with a recipe.
|
|
97
|
+
- A rule that conflicts with the rest of the case is a blocking question, not an override. `/explore-case` re-checks
|
|
98
|
+
each rule against the page and never deletes one silently.
|
|
99
|
+
- The framework baseline's `R1`–`R4` define exactly what a rule may say.
|
|
100
|
+
|
|
83
101
|
## Steps
|
|
84
102
|
|
|
85
103
|
Numbered lines. Keep them short and in plain language: the spec uses each step's text verbatim in `test.step`, so the
|
|
@@ -97,8 +115,13 @@ report and the trace read like the case.
|
|
|
97
115
|
generated: `/gen-spec` stops with `needs-input`.
|
|
98
116
|
- **Open questions** go in a `🚫 TODO [blocking]` or `❓ TODO [non-blocking]` block under the sentence they are about,
|
|
99
117
|
with the agent's suggestion written above and a last line `- ✍️ Answer:`. Accepting a question means deleting the
|
|
100
|
-
whole block; answering means writing after `Answer:`.
|
|
101
|
-
|
|
118
|
+
whole block; answering means writing after `Answer:`. Hang the block under the sentence that makes the claim, not
|
|
119
|
+
under a broader one above it, and one claim gets one block.
|
|
120
|
+
- **What the AI saw** is marked by what happens if it is wrong: `(seen on <profile>)` is a plain note the run checks by
|
|
121
|
+
itself, and `(observed on <profile>, 🔍 to be confirmed: <what a person must decide>)` is a line that would pass even
|
|
122
|
+
if it were wrong, so a person settles it. A marker may name a screenshot (`evidence: .explore/.../x.png`) so nobody
|
|
123
|
+
has to open the page again. Never put a marker inside a TODO block, on the sentence a block hangs under, or inside
|
|
124
|
+
the `## Values` yaml: one claim carries one signal. `pnpm case:check` fails on the first two.
|
|
102
125
|
|
|
103
126
|
## Expected
|
|
104
127
|
|
|
@@ -174,6 +197,7 @@ It reads every case (or the ids you pass) and prints one line per case. It **fai
|
|
|
174
197
|
- a target card without its required fields (`What`, `Where`, `Identified by`), or a reference image outside
|
|
175
198
|
`.explore/targets/<case-id>/`;
|
|
176
199
|
- a `[T<n>]` reference with no card;
|
|
200
|
+
- a `## Rules` bullet that names `Step <n>`, `Before step <n>` or `After step <n>` when the case has no such step;
|
|
177
201
|
- an invalid `## Values` block (bad YAML, an unknown profile, a non-map entry).
|
|
178
202
|
|
|
179
203
|
It **warns** (usable, but something to settle) on: a reference image missing on this machine (`.explore/` is
|
|
@@ -183,7 +207,7 @@ that cannot be answered in place.
|
|
|
183
207
|
|
|
184
208
|
## Rename or move a case
|
|
185
209
|
|
|
186
|
-
Use the Cases page (
|
|
187
|
-
spec is rewritten to match. By hand, move the three things together
|
|
188
|
-
`*-snapshots/`), and keep the spec's `// case:` line and its imports
|
|
189
|
-
name, so their folder depth does not change the imports.
|
|
210
|
+
Use the Cases page (open the case, then **Rename or move** in the **⋯** menu at the end of the tab strip). The spec
|
|
211
|
+
and the baseline images move with it and the spec is rewritten to match. By hand, move the three things together
|
|
212
|
+
(`cases/...md`, `tests/...spec.ts`, its `*-snapshots/`), and keep the spec's `// case:` line and its imports
|
|
213
|
+
unchanged: specs import the framework by package name, so their folder depth does not change the imports.
|
package/docs/commands.md
CHANGED
|
@@ -18,9 +18,11 @@ thin wrappers, so `pnpm asrun`, `pnpm signin` and the rest work as below. The sa
|
|
|
18
18
|
| Run tests | `pnpm asrun --env <env> [filters]` |
|
|
19
19
|
| Open the last report | `pnpm report` |
|
|
20
20
|
| Look around a page as a profile | `pnpm explore <profile-id> [path]` |
|
|
21
|
+
| See or validate what an exploration recorded for a case | `pnpm explore:notes <case-id> [--profile <id>] [--json] [--check]` |
|
|
21
22
|
| Read what the last report says about a case | `pnpm heal:evidence --case <case-id> --env <env>` |
|
|
22
23
|
| List or forget test data that could not be removed | `pnpm cleanup --env <env> [--forget]` |
|
|
23
24
|
| Create or validate a case | `pnpm case:new <id>`, `pnpm case:check [case-id]` |
|
|
25
|
+
| Settle the "to be confirmed" lines of a case from a passing run | `pnpm markers:confirm --case <case-id> --env <env> [--dry-run]` |
|
|
24
26
|
| Check a spec | `pnpm spec:lint [case-id]`, `pnpm baselines:review [case-id] [--new]` |
|
|
25
27
|
| Look at a PDF | `pnpm pdf:render <file.pdf> --out <dir> [--pages all] [--corner top-left]` |
|
|
26
28
|
| Type-check the workspace | `pnpm typecheck` |
|
|
@@ -125,7 +127,11 @@ pnpm pdf:render report.pdf --out .explore/targets/header-logo --corner top-left
|
|
|
125
127
|
```
|
|
126
128
|
|
|
127
129
|
- `case:new` creates the case from the template. One optional group folder, created on demand.
|
|
128
|
-
- `case:check` validates front matter, values and target cards;
|
|
130
|
+
- `case:check` validates front matter, values and target cards; it fails on a marker inside a question block, a marker
|
|
131
|
+
on the line a question hangs under, and a marker inside the `## Values` yaml. See [cases.md](./cases.md).
|
|
132
|
+
- `markers:confirm` reads the newest run report (or `--report <run-id|file>`) and removes the `🔍 to be confirmed`
|
|
133
|
+
markers whose line the spec carries as `// unconfirmed: ...`, but only when every test of the case passed. It answers
|
|
134
|
+
"does the page still say this?", never "is this the right expectation".
|
|
129
135
|
- `spec:lint` is a static check of the specs: it fails on sleeps, `networkidle`, image checks without a target,
|
|
130
136
|
`.first()` on images, raw download events, a magic-number-only PDF check. `pnpm spec:lint <case-id>` limits it.
|
|
131
137
|
- `baselines:review` opens nothing; it lists every baseline next to the target card that says what it must show.
|
|
@@ -136,6 +142,9 @@ pnpm pdf:render report.pdf --out .explore/targets/header-logo --corner top-left
|
|
|
136
142
|
|
|
137
143
|
```bash
|
|
138
144
|
pnpm explore example-staging /app/home
|
|
145
|
+
pnpm explore:notes conduit/header-navigation
|
|
146
|
+
pnpm explore:notes conduit/header-navigation --profile example-staging --json
|
|
147
|
+
pnpm explore:notes conduit/header-navigation --check
|
|
139
148
|
pnpm heal:evidence --case header-logo --env staging
|
|
140
149
|
```
|
|
141
150
|
|
|
@@ -144,6 +153,13 @@ are saved to `~/Downloads` and never overwrite an existing file. `heal:evidence`
|
|
|
144
153
|
step, error, call log and screenshots for a case, and whether the report is stale; it is what `/heal-spec` reads
|
|
145
154
|
first.
|
|
146
155
|
|
|
156
|
+
`explore:notes` reads the **exploration note** that `/explore-case` writes for a case under
|
|
157
|
+
`.explore/notes/<case-id>/<profile-id>.json`. It prints one line per profile (fresh or stale, the steps and targets it
|
|
158
|
+
covers) and `--json` prints the parsed notes for a run that reads them. A note is local, git-ignored and optional:
|
|
159
|
+
`/gen-spec` uses a fresh one to verify the locators and request patterns instead of rediscovering them, and replays the
|
|
160
|
+
case when it is missing or stale. `--check` fails on a malformed note and warns about a stale one (it prints the current
|
|
161
|
+
`caseSha256`); `--hash` prints that hash. Deleting `.explore/notes/` is always safe.
|
|
162
|
+
|
|
147
163
|
## Cleanup
|
|
148
164
|
|
|
149
165
|
```bash
|
package/docs/theme-guide.html
CHANGED
|
@@ -330,6 +330,7 @@
|
|
|
330
330
|
<a href="#icons">Icons</a>
|
|
331
331
|
<a href="#tooltips">Tooltips</a>
|
|
332
332
|
<a href="#editor">Markdown editor</a>
|
|
333
|
+
<a href="#document-tabs">Document tabs</a>
|
|
333
334
|
<a href="#surfaces">Surfaces</a>
|
|
334
335
|
<a href="#sections">Sidebar sections</a>
|
|
335
336
|
<a href="#checklist">Checklist</a>
|
|
@@ -959,6 +960,61 @@ import {FilterChips} from '../components/filter-chips';
|
|
|
959
960
|
<li><b>Every expand/collapse animates.</b> Use <code>Collapse</code> from <code>components/collapse.tsx</code> (height and fade, 200ms, <code>ease-out</code>) for content that appears and disappears, and rotate the chevron with <code>transition-transform duration-200</code> instead of swapping two icons. Always add <code>motion-reduce:transition-none</code>. Turn transitions off while the user drags a resize handle.</li>
|
|
960
961
|
</ul>
|
|
961
962
|
|
|
963
|
+
<h2 id="document-tabs">Document tabs</h2>
|
|
964
|
+
<p class="muted">
|
|
965
|
+
A page that holds more than one file of the same thing puts them in a tab strip: <code>DocumentTabs</code> in
|
|
966
|
+
<code>pages/cases.tsx</code>, where <b>Case</b> is the source a person writes and <b>Spec</b> what a run
|
|
967
|
+
generated from it. The strip is the only place the file names appear. The bar under it carries the tools of the
|
|
968
|
+
file on screen (<code>WorkbenchControls</code>, the layout and pane switches), never a second copy of the name,
|
|
969
|
+
and <b>Save</b> / <b>Discard</b> always sit at the same end of the strip and act on the file on screen.
|
|
970
|
+
</p>
|
|
971
|
+
<div class="demo" style="align-items:flex-end;gap:16px;padding:12px 16px 0;background:var(--warm);border-bottom:1px solid var(--line-mid)">
|
|
972
|
+
<span style="display:flex;align-items:center;gap:8px;margin-bottom:-1px;padding:6px 12px;border-bottom:2px solid var(--theme-btn);border-radius:6px 6px 0 0;background:var(--card);font-size:14px">
|
|
973
|
+
<span style="font-size:12px;font-weight:500;letter-spacing:.05em;color:var(--theme-ink)">Case</span>
|
|
974
|
+
<span class="mono">favorite-article.md</span>
|
|
975
|
+
<span title="Unsaved changes" style="width:6px;height:6px;border-radius:9999px;background:var(--gold-deep)"></span>
|
|
976
|
+
</span>
|
|
977
|
+
<span style="display:flex;align-items:center;gap:8px;padding:6px 12px;border-bottom:2px solid transparent;border-radius:6px 6px 0 0;font-size:14px;color:var(--ink-2)">
|
|
978
|
+
<span style="font-size:12px;font-weight:500;letter-spacing:.05em;color:var(--ink-3)">Spec</span>
|
|
979
|
+
<span class="mono">favorite-article.spec.ts</span>
|
|
980
|
+
</span>
|
|
981
|
+
<span style="display:flex;align-items:center;gap:8px;margin-left:auto;padding-bottom:8px">
|
|
982
|
+
<span class="btn btn-secondary">Discard</span>
|
|
983
|
+
<span class="btn btn-primary">Save</span>
|
|
984
|
+
<span class="btn btn-secondary">...</span>
|
|
985
|
+
</span>
|
|
986
|
+
</div>
|
|
987
|
+
<div class="demo" style="gap:16px;background:var(--card);border-bottom:1px solid var(--line-mid);padding:8px 16px">
|
|
988
|
+
<span style="display:flex;align-items:center;gap:6px">
|
|
989
|
+
<span style="font-size:12px;color:var(--ink-3)">Layout</span>
|
|
990
|
+
<span style="display:flex;overflow:hidden;border-radius:4px;box-shadow:0 0 0 1px var(--line-mid)">
|
|
991
|
+
<span style="padding:4px 10px;font-size:14px;background:var(--card);color:var(--ink-2)">Split</span>
|
|
992
|
+
<span style="padding:4px 10px;font-size:14px;background:var(--theme-btn);color:#fff">Single</span>
|
|
993
|
+
</span>
|
|
994
|
+
</span>
|
|
995
|
+
<span style="display:flex;align-items:center;gap:6px">
|
|
996
|
+
<span style="font-size:12px;color:var(--ink-3)">Pane</span>
|
|
997
|
+
<span style="display:flex;overflow:hidden;border-radius:4px;box-shadow:0 0 0 1px var(--line-mid)">
|
|
998
|
+
<span style="padding:4px 10px;font-size:14px;background:var(--theme-btn);color:#fff">Panel</span>
|
|
999
|
+
<span style="padding:4px 10px;font-size:14px;background:var(--card);color:var(--ink-2)">Editor</span>
|
|
1000
|
+
<span style="padding:4px 10px;font-size:14px;background:var(--card);color:var(--ink-2)">Doc</span>
|
|
1001
|
+
</span>
|
|
1002
|
+
</span>
|
|
1003
|
+
</div>
|
|
1004
|
+
<ul class="tight">
|
|
1005
|
+
<li>The active tab is <code>bg-card</code> with a <code>border-b-2 border-theme-btn</code> underline; the others are
|
|
1006
|
+
<code>text-ink-2</code> on the warm strip. The name is <code>font-mono</code>; the kind in front of it is
|
|
1007
|
+
<code>text-xs tracking-wider</code>, <code>text-theme-ink</code> when active and <code>text-ink-3</code> otherwise.</li>
|
|
1008
|
+
<li>Unsaved changes are a 6px <code>bg-gold-deep</code> dot on the tab, never a badge in the page header. A file
|
|
1009
|
+
that does not exist yet keeps its tab, disabled at 45% opacity, with a tooltip that says how to create it.</li>
|
|
1010
|
+
<li>The switches of the bar under the tabs are a segmented control, not buttons: active <code>bg-theme-btn</code>
|
|
1011
|
+
with white text, the others <code>bg-card</code>, the whole group ringed with <code>ring-1 ring-line-mid</code>, and
|
|
1012
|
+
a <code>text-xs text-ink-3</code> label in front (<code>Layout</code>, <code>Pane</code>). A switch group without a
|
|
1013
|
+
label is not discoverable; never ship one.</li>
|
|
1014
|
+
<li>Switching tabs keeps the other tab's draft, so <b>Save</b> and <b>Discard</b> take the active file; the tab
|
|
1015
|
+
dot is what says another file still has unsaved changes.</li>
|
|
1016
|
+
</ul>
|
|
1017
|
+
|
|
962
1018
|
<h2 id="sections">Sidebar sections and selection</h2>
|
|
963
1019
|
<div class="row" style="align-items:flex-start">
|
|
964
1020
|
<div class="side">
|
package/docs/troubleshooting.md
CHANGED
|
@@ -57,6 +57,11 @@ and regenerate the spec.
|
|
|
57
57
|
`Identified by`), refers to a reference image outside `.explore/targets/<case-id>/`, or a check refers to a `[T<n>]`
|
|
58
58
|
that has no card. A blocking question (`🚫 TODO [blocking]`) is the normal way to wait for a person's answer.
|
|
59
59
|
|
|
60
|
+
**`pnpm case:check` fails on a marker** — the same claim carries two signals: a marker word inside a question block, a
|
|
61
|
+
marker on the line a question hangs under, or a marker inside the `## Values` yaml (where it would become part of the
|
|
62
|
+
expected value). Keep one of the two, and move a `## Values` marker to the prose line above the block. A marker without
|
|
63
|
+
a reason is only a warning: write `🔍 to be confirmed: <what a person must decide>`.
|
|
64
|
+
|
|
60
65
|
**`pnpm spec:lint` fails** — read the rule name in the message: `no-sleep` (`waitForTimeout`), `no-networkidle`,
|
|
61
66
|
download handling, a screenshot or image check without `// target: T<n>`, a PDF check on a magic number alone. The
|
|
62
67
|
fixes are in the framework's `generating-specs` conventions; do not silence the rule.
|
|
@@ -117,6 +122,41 @@ cases, specs, `profiles.yaml` and `package.json` stay. Then run `pnpm dlx @asrun
|
|
|
117
122
|
`claude mcp list`; `failed: ... path specified` means the `playwright` path in that file does not resolve (repair it
|
|
118
123
|
with `pnpm asrun init --force <file>`).
|
|
119
124
|
|
|
125
|
+
**A `browser_*` tool says `Tool "<name>" not found`** — expected for part of them: the `playwright-test` MCP
|
|
126
|
+
(`run-test-mcp-server`) lists every browser tool Playwright has, but a call runs in a paused test whose backend installs
|
|
127
|
+
only the `core` and `testing` capabilities, plus it never installs the `skillOnly` tools. The server is up and the other
|
|
128
|
+
tools work: use the substitute in `.claude/skills/exploring-cases/references/tools.md` and do not report a missing MCP
|
|
129
|
+
server. `browser_highlight` / `browser_hide_highlight` become `locator.highlight()` / `page.hideHighlight()` through
|
|
130
|
+
`browser_run_code_unsafe`, `browser_network_clear` becomes a marker on the request list,
|
|
131
|
+
`browser_start_recording` / `browser_stop_recording` are never called (the handoff card,
|
|
132
|
+
`.claude/skills/exploring-cases/references/handoff-card.js`, starts and stops the recording itself and returns
|
|
133
|
+
`recorded`). `browser_annotate` cannot run either: the person names the element in the handoff card. `pnpm asrun doctor`
|
|
134
|
+
prints how many tools the installed Playwright runs and warns when `tools.md` names one it cannot.
|
|
135
|
+
|
|
136
|
+
**An AI run stops answering and never finishes** — the log keeps repeating `... still waiting for <tool call> (running
|
|
137
|
+
5m 30s)` or `... no output from the agent for 5m`, and nothing else arrives. A workspace has exactly one
|
|
138
|
+
`playwright-test` MCP server (the `run-test-mcp-server` child process OpenCode starts from `opencode.jsonc`), shared by
|
|
139
|
+
every session in that workspace, and Playwright runs one paused test in it at a time. One interrupted browser call —
|
|
140
|
+
closing the browser window by hand, cancelling a run while a call is in flight, or a second AI run using the same
|
|
141
|
+
server — leaves that queue wedged: the call never returns and every later run hangs on its first
|
|
142
|
+
`generator_setup_page` until the process is replaced. `pnpm asrun doctor` cannot see it, because it starts a fresh
|
|
143
|
+
server of its own. Cancel the stuck run, quit and restart OpenCode (that ends its MCP processes), then start the run
|
|
144
|
+
again. From the UI this is bounded: only one AI run at a time runs per workspace (the `ai` job lock, whatever case it is
|
|
145
|
+
on), and a run that says nothing for 10 minutes — 15 while a handoff card waits for a person — is stopped with this
|
|
146
|
+
instruction in its log.
|
|
147
|
+
|
|
148
|
+
**The UI server restarted while an AI run was going** — the job is still in the Jobs drawer, marked **Interrupted**,
|
|
149
|
+
and its log stops where the server died. If the agent (and the `playwright-test` MCP server below it) is still running,
|
|
150
|
+
the job offers **Stop the leftover process**; stop it before starting another AI run, or both agents share one MCP
|
|
151
|
+
server and wedge it. A job that had already written `RESULT: needs-input` is recovered as **Succeeded**, so its answers
|
|
152
|
+
can still be sent. Every finished job has **Run again**, which reuses the inputs it was started with.
|
|
153
|
+
|
|
154
|
+
**A call fails with `Missing key`** — the published schema requires two things that are easy to leave out: `intent` on
|
|
155
|
+
every action, input and assertion tool (`browser_click`, `browser_type`, `browser_evaluate`, `browser_run_code_unsafe`,
|
|
156
|
+
`browser_wait_for`, ...), and the fields that have a default (`browser_take_screenshot` needs `scale: "css"`,
|
|
157
|
+
`browser_network_requests` needs `static: false`, `generator_setup_page` needs `plan`). The full list is in
|
|
158
|
+
`.claude/skills/exploring-cases/references/tools.md`.
|
|
159
|
+
|
|
120
160
|
**An AI job runs but uses a slow or limited model** — the top bar's **Model** picker sets the model for UI runs;
|
|
121
161
|
without one, a one-shot OpenCode run uses its built-in free model, which has a daily usage limit. Set `"model"` in
|
|
122
162
|
`opencode.jsonc` to stop depending on the browser's choice. For Claude Code, the picker lists what its settings,
|
package/docs/web-ui.md
CHANGED
|
@@ -55,21 +55,39 @@ One row per profile, with the same login check as `pnpm status`.
|
|
|
55
55
|
|
|
56
56
|
## Cases
|
|
57
57
|
|
|
58
|
-
The case list is grouped by folder and collapses.
|
|
59
|
-
|
|
60
|
-
|
|
58
|
+
The case list is grouped by folder and collapses. A case opens on the **Case** tab, with the **Spec** tab beside it:
|
|
59
|
+
|
|
60
|
+
- **Case** (`favorite-article.md`) is the source a person writes: the steps, values, rules and questions.
|
|
61
|
+
- **Spec** (`favorite-article.spec.ts`) is what a generate or heal run wrote from it. It stays visible but disabled
|
|
62
|
+
until the case has a spec, and its tooltip says how to make one.
|
|
63
|
+
|
|
64
|
+
The tab is the page's state: it lives in the URL (`?view=spec`), so a reload, a shared link and the Back button land on
|
|
65
|
+
the same file, and every other link opens the case. Each tab keeps its own unsaved draft while you switch between them;
|
|
66
|
+
an amber dot on a tab means that file has unsaved changes, and **Save** (`Ctrl/Cmd+S`) and **Discard** always sit at the
|
|
67
|
+
right of the tab strip and act on the file on screen.
|
|
68
|
+
|
|
69
|
+
Under the tabs, the first row is the AI toolbar (command, environment, profiles, attended, **Run AI**, copy the shell
|
|
70
|
+
command, AI guide). It is the same on both tabs, because every AI command acts on the case. The row below holds the
|
|
71
|
+
tools of the file on screen: on **Case** the **Layout** (Split / Single) and **Pane** (Panel / Editor / Doc) switches
|
|
72
|
+
and the previous / next of an outstanding mark; on **Spec** the line that says the file is generated from the case
|
|
73
|
+
(with a link back to it), **Open in editor**, **Copy path** and copy the spec. **Rename or move** and **Delete** live
|
|
74
|
+
in the **⋯** menu at the end of the tab strip.
|
|
75
|
+
|
|
76
|
+
The editor opens in **Single** with the **Panel** full width; **Split** puts the source on the left and one pane on the
|
|
77
|
+
right (drag the divider to resize, double-click to reset).
|
|
61
78
|
|
|
62
79
|
- **Source and preview** scroll together, the preview block at the caret is shaded, and a click in the preview puts
|
|
63
80
|
the caret on that line. `Ctrl/Cmd+S` saves.
|
|
64
81
|
- **What needs a person is highlighted** in both: `TODO` in any spelling (red for `🚫 TODO [blocking]`, gold
|
|
65
|
-
otherwise), `🔍 to be confirmed
|
|
82
|
+
otherwise), `🔍 to be confirmed` / `👁️ not observed` / `TBC` / `TBD` in blue, and your `✍️ Answer:` lines in
|
|
66
83
|
green. Hover for a tooltip; the small X after a marker removes it (or the whole question block for a `TODO`) in one
|
|
67
84
|
step, with Undo; the file is saved when you save it.
|
|
68
85
|
- **New case** takes an optional folder and runs `pnpm case:new`.
|
|
69
86
|
- **The Panel** shows the case as a form, with the open questions first (accept one to delete the question, or write an
|
|
70
87
|
answer and the AI folds it in). Then the front matter (title, login, suites, writes and its note, environments), the
|
|
71
|
-
steps (sentence, `[writes]` / `[input]` / `[email]` tags, `- Ready when:` lines,
|
|
72
|
-
`## Expected` and `## Visual checks` bullets (drag to reorder), the `## Values` table (one
|
|
88
|
+
`## Rules` bullets, the steps (sentence, `[writes]` / `[input]` / `[email]` tags, `- Ready when:` lines,
|
|
89
|
+
`- Recipe:` lines), the `## Expected` and `## Visual checks` bullets (drag to reorder), the `## Values` table (one
|
|
90
|
+
column per profile; an
|
|
73
91
|
empty cell inherits `default`), and the `## Targets` cards read-only, with what each is missing. With two or more
|
|
74
92
|
profiles the Values section offers a **Rows: Keys / Profiles** switch, and from four profiles on it also adds a
|
|
75
93
|
search and environment chips. With more profiles than keys it starts as one row per profile, so every name and
|
|
@@ -78,15 +96,24 @@ Single, between **Panel**, **Editor** and **Doc**.
|
|
|
78
96
|
the card: what it is, where, how it is identified, what it must not be, and its reference screenshots. Every section
|
|
79
97
|
collapses and has a **?** with a short help card. Panel edits land in the same draft as the source editor, so one
|
|
80
98
|
save covers both, and **Discard** throws them away together.
|
|
81
|
-
- **
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
- **
|
|
99
|
+
- **The To confirm section** lists every line the AI marked: the sentence, where it is (`## Values · line 42`), **why it
|
|
100
|
+
needs you** (the reason the AI wrote after the marker's colon), **what it feeds** (an assertion or only a note) and,
|
|
101
|
+
when it saved one, the **screenshot** from the exploration, so you look at the image instead of opening the page
|
|
102
|
+
again. Rows that change an assertion are highlighted; **Confirm** settles one line and **Confirm all** settles the
|
|
103
|
+
list in one undo step. None of this blocks generating a spec; healing stops until the case is settled.
|
|
104
|
+
- **Spec view** is editable too (`Ctrl/Cmd+S`; refused when the file changed on disk meanwhile), and its tab links back
|
|
105
|
+
to the case it was generated from. **Open in editor** opens it in the default application for `.spec.ts`, **Copy
|
|
106
|
+
path** copies its absolute path.
|
|
107
|
+
- **Rename or move** (in the **⋯** menu) keeps the spec and the baselines with the case and rewrites the spec;
|
|
108
|
+
**Delete** (type the id, or fill it in with the feather icon) removes the case, its spec and its baselines. Both
|
|
109
|
+
dialogs list exactly what they touch, and are refused while an AI or test run is in progress or while a draft is
|
|
110
|
+
unsaved.
|
|
111
|
+
- **AI toolbar:** command (explore / generate / heal, explore by default), **Environment** (required, no default),
|
|
87
112
|
**Profiles** (explore only: every profile that is logged in is ticked, the choice is remembered per environment, and
|
|
88
113
|
only the ticked ones are opened), **Attended** (only for explore, ticked by default), **Run AI**, copy the shell
|
|
89
114
|
command, open the AI guide.
|
|
115
|
+
- **Writes in …** is one line under the toolbar, in red: the environments whose data the case writes, then its note,
|
|
116
|
+
with the full note in its tooltip. A case that only runs in some environments shows that line instead.
|
|
90
117
|
- **From memo** opens a Markdown editor with a preview for raw notes. **Make cases** saves them to `.tmp/memos/` and
|
|
91
118
|
runs `/make-case` one-shot (defaults for gaps, `TODO` for unknown values, no page exploration, existing cases never
|
|
92
119
|
overwritten); **Copy /make-case** copies the memo for the interactive dialogue. When the run ends, **What to do
|
|
@@ -121,6 +148,21 @@ One entry per run, with a PRODUCTION or STAGING badge (runs from before environm
|
|
|
121
148
|
The bottom bar shows every command started from the UI with live stdout/stderr and the exit code; a job can be
|
|
122
149
|
cancelled there. Opening it shrinks the page instead of covering it.
|
|
123
150
|
|
|
151
|
+
- Jobs are recorded under `.tmp/ui-jobs/<id>/` (`record.json`, `chunks.jsonl`, `output.log`), kept for 7 days and the
|
|
152
|
+
newest 50. A UI server restart does not forget them. A job that was running is shown as **Interrupted**: its output
|
|
153
|
+
stops at the last line the server received (the UI replays the last 4 MB of structured output; `output.log` keeps the
|
|
154
|
+
whole text). If its process is still alive, the page offers **Stop the leftover process** (the agent, its browser and
|
|
155
|
+
its MCP server), and a new AI run or test run is refused until it is stopped: one agent per workspace, as above. A
|
|
156
|
+
job that had already written `RESULT: needs-input` is recovered as **Succeeded**, so its answers can still be sent.
|
|
157
|
+
- **Run again** starts a finished job with the inputs it was started with. A heal keeps its hints, model, profiles and
|
|
158
|
+
report, so nothing has to be typed again. It is a new job with a new conversation and report; write cases, baseline
|
|
159
|
+
overwrites and an existing spec are confirmed again.
|
|
160
|
+
|
|
161
|
+
- Only one AI job runs at a time per workspace: `/explore-case`, `/gen-spec` and `/heal-spec` all drive the workspace's
|
|
162
|
+
single `playwright-test` MCP server, and two at once wedge it. Starting a second one is refused with the name of the
|
|
163
|
+
job it clashes with. A run that says nothing for 10 minutes — 15 while a handoff card waits for a person — is stopped
|
|
164
|
+
and its log says why; see [troubleshooting](./troubleshooting.md#ai-steps).
|
|
165
|
+
|
|
124
166
|
- An AI job's log is structured: a timestamp, one line per tool call (name, what it touched, how long it took), the
|
|
125
167
|
first lines of what it returned with the colors the command printed, the agent's thinking dimmed while **Thinking**
|
|
126
168
|
is ticked, and its messages rendered as Markdown. Long output is a preview with a count. The same text is kept in
|
|
@@ -128,8 +170,10 @@ cancelled there. Opening it shrinks the page instead of covering it.
|
|
|
128
170
|
- When an AI job ends, a thin **What to do next** bar appears under the log (or **Details** opens it in a sheet):
|
|
129
171
|
open questions with an answer box and **Use the suggested defaults**, the confirmations the exploration marked, and
|
|
130
172
|
the next action (**Generate spec**, **Open Run tests**, **View spec**, **Re-explore**). A run that ends with
|
|
131
|
-
`needs-input` gets an amber *Needs your input* badge; sending answers continues the same conversation, once per turn
|
|
132
|
-
|
|
173
|
+
`needs-input` gets an amber *Needs your input* badge; sending answers continues the same conversation, once per turn.
|
|
174
|
+
The record survives a UI server restart, so a run that had already written its questions can still be answered and a
|
|
175
|
+
finished job can be started again with **Run again**; after a restart, a job that did not get that far is answered in
|
|
176
|
+
the case file and run again.
|
|
133
177
|
- A run that ends without its `RESULT:` line gets an amber *No result* badge and a banner under the command instead of
|
|
134
178
|
a "what to do next" bar: nothing is known about what it did, so read the log before trusting the case or the spec.
|
|
135
179
|
A run that is not allowed to write files (plan mode, a read-only sandbox) ends that way; the framework starts
|
|
@@ -154,7 +198,7 @@ cancelled there. Opening it shrinks the page instead of covering it.
|
|
|
154
198
|
|
|
155
199
|
| Path | What |
|
|
156
200
|
|---|---|
|
|
157
|
-
| `.tmp/ui-jobs/` | the log of every job started from the UI (git-ignored) |
|
|
201
|
+
| `.tmp/ui-jobs/` | the record, chunks and log of every job started from the UI (git-ignored); 7 days, newest 50 |
|
|
158
202
|
| `.tmp/hints/` | the text you type in the Heal dialog, passed to the agent as `--hints` |
|
|
159
203
|
| `.explore/` | reference screenshots of the target cards (git-ignored) |
|
|
160
204
|
| `reports/` | one folder per run, `reports/latest` points at the newest |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@asrunjs/asrun",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Multi-profile end-to-end test suite: plain-language cases, AI-written Playwright specs, evidence for every run.",
|
|
5
5
|
"license": "UNLICENSED",
|
|
6
6
|
"author": "Zoro Chen",
|
|
@@ -52,9 +52,11 @@
|
|
|
52
52
|
"report": "tsx scripts/report.ts",
|
|
53
53
|
"heal:evidence": "node scripts/heal-evidence.mjs",
|
|
54
54
|
"explore": "tsx scripts/explore.ts",
|
|
55
|
+
"explore:notes": "tsx scripts/explore-notes.ts",
|
|
55
56
|
"cleanup": "tsx scripts/cleanup.ts",
|
|
56
57
|
"case:new": "tsx scripts/new-case.ts",
|
|
57
58
|
"case:check": "tsx scripts/check-cases.ts",
|
|
59
|
+
"markers:confirm": "tsx scripts/markers-confirm.ts",
|
|
58
60
|
"spec:lint": "tsx scripts/lint-specs.ts",
|
|
59
61
|
"baselines:review": "tsx scripts/review-baselines.ts",
|
|
60
62
|
"pdf:render": "tsx scripts/render-pdf.ts",
|
|
@@ -6,9 +6,9 @@ Load the framework skill `{{framework}}/.claude/skills/exploring-cases/SKILL.md`
|
|
|
6
6
|
|
|
7
7
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline
|
|
8
8
|
`{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/exploring-cases.md` if it exists, then the workspace
|
|
9
|
-
`AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
|
|
9
|
+
`AGENTS.md` §2 (Project rules). F1–F6 and R2 always win. Project facts are in `AGENTS.md` §3.
|
|
10
10
|
|
|
11
|
-
Arguments: `<case-id> <env> [--attended] [--profiles a,b]`. The case id is the path below `cases/` without `.md` (it may include a
|
|
11
|
+
Arguments: `<case-id> <env> [--attended] [--profiles a,b] [--hints <file>]`. The case id is the path below `cases/` without `.md` (it may include a
|
|
12
12
|
group folder, e.g. `documents/landing-page`) and is required. `<env>` is `staging` or `production`; it is required and there
|
|
13
13
|
is no default (without one, stop and say so). With `--attended` I am at the machine and the browser window is
|
|
14
14
|
visible: when a step writes data or needs a choice, hand it to me as the skill describes, wait for me, and record
|
|
@@ -16,12 +16,17 @@ what I did as a general recipe under the step it belongs to (`- Recipe:` lines).
|
|
|
16
16
|
`--profiles a,b` explores only those profiles (comma-separated ids from `profiles.yaml`); without it, at least one
|
|
17
17
|
logged-in profile per domain. A profile the list leaves out is not opened and nothing is written about it: runs add up,
|
|
18
18
|
so a later run covers the rest.
|
|
19
|
+
`--hints <file>` (or free text after the arguments) is what I know or want checked; read it first. After the work, a
|
|
20
|
+
reusable instruction goes into `## Rules` by the gate in
|
|
21
|
+
`{{framework}}/.claude/skills/exploring-cases/references/case-rules.md`; a one-off clue stays in the report.
|
|
19
22
|
|
|
20
|
-
|
|
23
|
+
Read `## Rules` before the steps and apply it. Follow
|
|
24
|
+
`{{framework}}/.claude/skills/exploring-cases/references/case-rules.md` (a rule says how to run the case, never what it
|
|
25
|
+
proves; never delete or rewrite one), `assertion-targets.md` (list the candidates, never
|
|
21
26
|
pick when in doubt, ask for the exact image/text/place/UI) and `readiness.md` (wait for the key requests and the
|
|
22
27
|
content before you read anything) of the skill.
|
|
23
28
|
|
|
24
29
|
This command edits `cases/<case-id>.md` only (plus screenshots under `.explore/targets/`). Do not ask me questions in
|
|
25
|
-
the chat: write open questions into the case file as the skill describes, and end with the `RESULT:` line
|
|
26
|
-
not allowed to write files at all (a plan-mode run,
|
|
27
|
-
`RESULT: needs-input`.
|
|
30
|
+
the chat: write open questions into the case file as the skill describes, and end with the `RESULT:` line as plain text
|
|
31
|
+
on the last line, never in a code fence or in emphasis. If you are not allowed to write files at all (a plan-mode run,
|
|
32
|
+
a read-only sandbox), say what blocked you and end with `RESULT: needs-input`.
|
|
@@ -7,18 +7,23 @@ $ARGUMENTS
|
|
|
7
7
|
|
|
8
8
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline
|
|
9
9
|
`{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace
|
|
10
|
-
`AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
|
|
10
|
+
`AGENTS.md` §2 (Project rules). F1–F6 and R2 always win. Project facts are in `AGENTS.md` §3.
|
|
11
11
|
|
|
12
|
-
Arguments: `<case-id> [<env>] [--force]`. The case file is `cases/<case-id>.md` (the id may include one group folder,
|
|
12
|
+
Arguments: `<case-id> [<env>] [--force] [--hints <file>]`. The case file is `cases/<case-id>.md` (the id may include one group folder,
|
|
13
13
|
e.g. `documents/landing-page`). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If
|
|
14
14
|
the spec already exists, it is left alone unless `--force` is given; with `--force` it is regenerated and overwritten.
|
|
15
|
+
`--hints <file>` (or free text after the arguments) is what I know or want changed; read it first. A reusable
|
|
16
|
+
instruction goes into `## Rules` by the gate in
|
|
17
|
+
`{{framework}}/.claude/skills/exploring-cases/references/case-rules.md`; a one-off clue stays in the report.
|
|
15
18
|
There is no profile option: the spec is generated for every profile the case holds information for (its `## Values`
|
|
16
19
|
keys, its target cards and its observations), not for a chosen subset and not only for the profiles that are logged in.
|
|
17
20
|
|
|
18
|
-
Follow the skill's procedure exactly, including
|
|
21
|
+
Follow the skill's procedure exactly, including `case-rules.md` (read `## Rules` before the steps; a rule says how to
|
|
22
|
+
run the case, never what it proves), its `assertion-targets.md` and `readiness.md`; run
|
|
19
23
|
`pnpm spec:lint <case-id>`. Only generate the spec: do not run `pnpm asrun` (running specs is a separate
|
|
20
24
|
process), and do not create baselines.
|
|
21
25
|
|
|
22
26
|
Do not ask me questions: resolve ambiguity as the skill describes (explore the page, use observed values marked for
|
|
23
|
-
QA, write the remaining questions into the case file), and end with the `RESULT:` line
|
|
24
|
-
files at all (a plan-mode run, a read-only
|
|
27
|
+
QA, write the remaining questions into the case file), and end with the `RESULT:` line as plain text on the last line,
|
|
28
|
+
never in a code fence or in emphasis. If you are not allowed to write files at all (a plan-mode run, a read-only
|
|
29
|
+
sandbox), say what blocked you and end with `RESULT: needs-input`.
|
|
@@ -7,7 +7,7 @@ spec", and follow it exactly.
|
|
|
7
7
|
|
|
8
8
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline
|
|
9
9
|
`{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace
|
|
10
|
-
`AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
|
|
10
|
+
`AGENTS.md` §2 (Project rules). F1–F6 and R2 always win. Project facts are in `AGENTS.md` §3.
|
|
11
11
|
|
|
12
12
|
Arguments: $ARGUMENTS
|
|
13
13
|
Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
|
|
@@ -16,23 +16,27 @@ Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`,
|
|
|
16
16
|
- `--profiles`: the profiles to run when evidence is needed. Without it, every profile that is logged in in `<env>`.
|
|
17
17
|
- `--report`: the report (folder name under `reports/`) to read first. Without it, the newest report of this case.
|
|
18
18
|
- `--hints <file>`: a file with what the user wants repaired or improved, with any evidence. Read it first. Any free
|
|
19
|
-
text after the arguments is the same kind of request. Both are optional.
|
|
19
|
+
text after the arguments is the same kind of request. Both are optional. Read `## Rules` before the steps and apply
|
|
20
|
+
it; after the work, a reusable method instruction goes into `## Rules` by the gate in
|
|
21
|
+
`{{framework}}/.claude/skills/exploring-cases/references/case-rules.md`, while a one-off clue stays in the report.
|
|
20
22
|
|
|
21
23
|
A passing run is not a reason to refuse a request: healing also refines a case and its spec. Never lower the bar for a
|
|
22
24
|
product finding to make a test pass.
|
|
23
25
|
|
|
24
26
|
Do not ask me questions in the chat: this may run unattended. Resolve what the report, the case and the page can
|
|
25
|
-
resolve, and write what still needs a person into the case as a `[blocking]` question. End with the `RESULT:` line
|
|
26
|
-
|
|
27
|
-
write files at all (a plan-mode run, a read-only
|
|
28
|
-
and end with `RESULT: needs-input`: never answer
|
|
27
|
+
resolve, and write what still needs a person into the case as a `[blocking]` question. End with the `RESULT:` line as
|
|
28
|
+
plain text on the last line, never in a code fence or in emphasis: `ready` when the healing or the refinement is
|
|
29
|
+
finished, `needs-input` when something is still open. If you could not write files at all (a plan-mode run, a read-only
|
|
30
|
+
sandbox, a refused permission), say in one sentence what blocked you and end with `RESULT: needs-input`: never answer
|
|
31
|
+
with a plan and a question instead.
|
|
29
32
|
|
|
30
33
|
Before you decide "product or test", audit the assertion itself (target card, baseline content, readiness): see step 5
|
|
31
34
|
of the skill's healing section, `assertion-targets.md` and `readiness.md`. Run `pnpm spec:lint <case-id>`.
|
|
32
35
|
Read the existing report first (`pnpm heal:evidence --case <case-id> --env <env>`), and run the case again only when
|
|
33
36
|
the skill says it is needed. Always run with `--logged-in`. If the change leaves an open question or an unconfirmed
|
|
34
37
|
marker in the case, stop there, tell me to review the case and run `/explore-case`, and end with
|
|
35
|
-
`RESULT: needs-input`.
|
|
38
|
+
`RESULT: needs-input`. A marker inside `## Rules` does not stop the spec unless the rule covers a literal the spec
|
|
39
|
+
asserts. Otherwise tell me whether it is a test problem or a product finding.
|
|
36
40
|
|
|
37
41
|
Run `pnpm asrun ...`, `pnpm heal:evidence ...`, `pnpm spec:lint ...`, `pnpm baselines:review ...` and
|
|
38
42
|
`pnpm status ...` as plain commands, optionally piped to `tail` or `head`, and do not chain other commands with them
|
|
@@ -7,7 +7,7 @@ documents: $ARGUMENTS
|
|
|
7
7
|
|
|
8
8
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline
|
|
9
9
|
`{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/making-cases.md` if it exists, then the workspace
|
|
10
|
-
`AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
|
|
10
|
+
`AGENTS.md` §2 (Project rules). F1–F6 and R2 always win. Project facts are in `AGENTS.md` §3.
|
|
11
11
|
|
|
12
12
|
The input is either text or a path to a file with the notes. Follow the skill's procedure exactly. Ask me the open
|
|
13
13
|
questions before writing any case file. Do not generate specs; that is `/gen-spec`.
|
package/templates/AGENTS.md
CHANGED
|
@@ -9,11 +9,13 @@ Read this file and the framework baseline it points at, before any AI step.
|
|
|
9
9
|
|
|
10
10
|
Rules are applied in this order, and a later layer wins where they conflict (the framework baseline, "Precedence"):
|
|
11
11
|
|
|
12
|
-
1. `F1`–`F6` of the framework baseline: safety, in every project. Nothing here can change them.
|
|
12
|
+
1. `F1`–`F6` and `R2` of the framework baseline: safety, in every project. Nothing here can change them.
|
|
13
13
|
2. The framework baseline, `{{framework}}/AGENTS.md`.
|
|
14
14
|
3. The framework skill (`{{framework}}/.claude/skills/<skill>/SKILL.md`).
|
|
15
15
|
4. A project skill override, `.claude/skills/<skill>.md` (optional; the command and the skill read it).
|
|
16
16
|
5. This file, §2 below: what this project does differently.
|
|
17
|
+
6. A case's `## Rules`: how to run that one case, never what is true. It is below §2 and cannot weaken `F1`–`F6` or
|
|
18
|
+
`R2`.
|
|
17
19
|
|
|
18
20
|
The framework skills are loaded by the commands in `.claude/commands/` (Claude Code) and `.opencode/commands/`
|
|
19
21
|
(OpenCode). The skills live in the framework package; §3 holds this project's facts.
|
|
@@ -21,8 +23,8 @@ The framework skills are loaded by the commands in `.claude/commands/` (Claude C
|
|
|
21
23
|
## 2. Project rules
|
|
22
24
|
|
|
23
25
|
Only what differs from the baseline: name the rule id, and whether it `narrows` (stricter) or `replaces` (a different
|
|
24
|
-
default, for example the artifact language) the baseline rule. `replaces` is refused for `F1`–`F6`. Rows
|
|
25
|
-
cell is not a rule id are ignored, so the placeholder below can stay until you need it.
|
|
26
|
+
default, for example the artifact language) the baseline rule. `replaces` is refused for `F1`–`F6` and `R2`. Rows
|
|
27
|
+
whose first cell is not a rule id are ignored, so the placeholder below can stay until you need it.
|
|
26
28
|
|
|
27
29
|
| Rule | Project rule | Effect |
|
|
28
30
|
|---|---|---|
|
|
@@ -39,9 +41,10 @@ Examples (delete what you do not use; the fenced block is an example, not a rule
|
|
|
39
41
|
`pnpm asrun doctor` fails when this table names a `[non-negotiable]` rule, an unknown rule id, or an `Effect` that is
|
|
40
42
|
not `narrows` or `replaces`.
|
|
41
43
|
|
|
42
|
-
The structural tokens are fixed English whatever the project language: the section headings (`## Values`, `##
|
|
43
|
-
...), a step's `- Recipe (phase): ...` lines, `### T<n>`, the `TODO` blocks with their `Answer:` line,
|
|
44
|
-
`[input]`/`[writes]`/`[email]`, `RESULT
|
|
44
|
+
The structural tokens are fixed English whatever the project language: the section headings (`## Values`, `## Rules`,
|
|
45
|
+
`## Targets`, ...), a step's `- Recipe (phase): ...` lines, `### T<n>`, the `TODO` blocks with their `Answer:` line,
|
|
46
|
+
`[input]`/`[writes]`/`[email]`, `RESULT:` (plain text on the last line, never inside a code fence or in emphasis),
|
|
47
|
+
and the front-matter keys. See "Fixed tokens" in the framework baseline.
|
|
45
48
|
|
|
46
49
|
## 3. Project facts
|
|
47
50
|
|
|
@@ -11,6 +11,13 @@ suites: [smoke, regression] # a case may be in several suites; filter with `pn
|
|
|
11
11
|
## Goal
|
|
12
12
|
Why this case exists, in one or two sentences.
|
|
13
13
|
|
|
14
|
+
## Rules
|
|
15
|
+
Optional instructions for the AI on how to run this case (a page quirk, a mandatory read-only preparation, a control
|
|
16
|
+
to avoid). They never state what the case expects and never allow writing data or acting for a person. One bullet per
|
|
17
|
+
rule; a bullet may start with `Step 3:`, `Before step 3:` or `After step 3:`. Delete this paragraph when the case needs
|
|
18
|
+
none.
|
|
19
|
+
- <e.g. Before step 1: dismiss the cookie banner; it covers the header.>
|
|
20
|
+
|
|
14
21
|
## Steps
|
|
15
22
|
1. Open <page or path, e.g. /account/>.
|
|
16
23
|
- Ready when: <the one positive signal that what this step loads is really there, e.g. the list shows its first
|
package/templates/package.json
CHANGED
|
@@ -14,9 +14,11 @@
|
|
|
14
14
|
"report": "asrun report",
|
|
15
15
|
"heal:evidence": "asrun heal:evidence",
|
|
16
16
|
"explore": "asrun explore",
|
|
17
|
+
"explore:notes": "asrun explore:notes",
|
|
17
18
|
"cleanup": "asrun cleanup",
|
|
18
19
|
"case:new": "asrun case:new",
|
|
19
20
|
"case:check": "asrun case:check",
|
|
21
|
+
"markers:confirm": "asrun markers:confirm",
|
|
20
22
|
"spec:lint": "asrun spec:lint",
|
|
21
23
|
"baselines:review": "asrun baselines:review",
|
|
22
24
|
"pdf:render": "asrun pdf:render",
|