@asrunjs/asrun 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +2 -1
- package/.claude/commands/gen-spec.md +2 -1
- package/.claude/commands/heal-spec.md +2 -1
- package/.claude/commands/make-case.md +2 -1
- package/.claude/skills/exploring-cases/SKILL.md +59 -64
- package/.claude/skills/exploring-cases/references/case-rules.md +5 -0
- package/.claude/skills/exploring-cases/references/notes.md +87 -0
- package/.claude/skills/exploring-cases/references/open-questions.md +8 -0
- package/.claude/skills/exploring-cases/references/readiness.md +3 -2
- package/.claude/skills/exploring-cases/references/tools.md +34 -0
- package/.claude/skills/exploring-cases/references/user-handoff.md +13 -9
- package/.claude/skills/generating-specs/SKILL.md +61 -47
- package/.claude/skills/generating-specs/references/conventions.md +6 -2
- package/.claude/skills/making-cases/SKILL.md +12 -5
- package/.claude/skills/making-cases/references/writes-and-email.md +5 -2
- package/AGENTS.md +30 -14
- package/README.md +16 -11
- package/bin/asrun.mjs +1 -0
- package/dist/lib/case-doc.d.ts +1 -1
- package/dist/lib/config.d.ts +26 -0
- package/dist/lib/config.js +51 -0
- package/dist/lib/explore-notes.d.ts +74 -0
- package/dist/lib/explore-notes.js +242 -0
- package/dist/lib/fixtures.d.ts +1 -1
- package/dist/lib/fixtures.js +7 -4
- package/dist/lib/paths.d.ts +2 -0
- package/dist/lib/paths.js +2 -0
- package/dist/lib/records.d.ts +17 -2
- package/dist/lib/records.js +32 -2
- package/dist/lib/rules.d.ts +57 -0
- package/dist/lib/rules.js +140 -0
- package/dist/scripts/cleanup.js +10 -1
- package/dist/scripts/cli.js +17 -2
- package/dist/scripts/doctor.js +27 -48
- package/dist/scripts/explore-notes.d.ts +1 -0
- package/dist/scripts/explore-notes.js +131 -0
- package/dist/scripts/lint-specs.js +23 -1
- package/dist/scripts/rules.d.ts +1 -0
- package/dist/scripts/rules.js +148 -0
- package/docs/agents/case-format.md +9 -5
- package/docs/agents/framework-dev.md +4 -0
- package/docs/ai-workflow.md +30 -13
- package/docs/cases.md +14 -10
- package/docs/commands.md +33 -4
- package/docs/install.md +2 -1
- package/docs/theme-guide.html +79 -2
- package/docs/troubleshooting.md +11 -2
- package/docs/web-ui.md +40 -12
- package/package.json +3 -1
- package/templates/.claude/commands/explore-case.md +3 -1
- package/templates/.claude/commands/gen-spec.md +3 -1
- package/templates/.claude/commands/heal-spec.md +3 -1
- package/templates/.claude/commands/make-case.md +3 -1
- package/templates/AGENTS.md +17 -8
- package/templates/cases/_template.md +3 -3
- package/templates/package.json +5 -0
- package/ui/dist/assets/{dist-BmWaNjqO.js → dist-BX88bMYN.js} +1 -1
- package/ui/dist/assets/{dist-2msWI-5M.js → dist-Bfv586Or.js} +1 -1
- package/ui/dist/assets/{dist-e-KhTWHJ.js → dist-BjlbPf3t.js} +1 -1
- package/ui/dist/assets/{dist-BVvHVJ3N.js → dist-CJnqQMvX.js} +1 -1
- package/ui/dist/assets/{dist-BMfJatHo.js → dist-CaAcCNRS.js} +1 -1
- package/ui/dist/assets/{dist-sfZKBSMm.js → dist-Clj0wGFy.js} +1 -1
- package/ui/dist/assets/{dist-J36xayOv.js → dist-D8TUm3Ai.js} +1 -1
- package/ui/dist/assets/{dist-4U_gPHXf.js → dist-DUDqh_3e.js} +1 -1
- package/ui/dist/assets/{dist-BwNz_xW2.js → dist-DkAx5a-Y.js} +1 -1
- package/ui/dist/assets/{dist-wjB7g6OJ.js → dist-cuboTVh-.js} +1 -1
- package/ui/dist/assets/index-DaUQORCy.js +353 -0
- package/ui/dist/assets/index-j3W4BBCj.css +2 -0
- package/ui/dist/index.html +2 -2
- package/ui/dist/assets/index-CGbRJT3b.js +0 -348
- package/ui/dist/assets/index-CI0i0LWc.css +0 -2
|
@@ -6,7 +6,8 @@ Load the `exploring-cases` skill and explore: $ARGUMENTS
|
|
|
6
6
|
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
7
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
8
|
framework skill, `.claude/skills/exploring-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
-
(Project rules). F1–F6 and R2 always win.
|
|
9
|
+
(Project rules). F1–F6 and R2 always win. The effective set is what `pnpm asrun rules` prints; §2 is applied as
|
|
10
|
+
written and is never turned into a question (a difference from a framework default is not a conflict).
|
|
10
11
|
Arguments: `<case-id> <env> [--attended] [--profiles a,b] [--hints <file>]`. The case id is the path below cases/ without `.md` (it may include a group
|
|
11
12
|
folder, e.g. documents/landing-page) and is required. `<env>` is `staging` or `production`; it is required and there is no
|
|
12
13
|
default (without one, stop and say so). With `--attended` I am at the machine and the browser window is visible: when a step
|
|
@@ -6,7 +6,8 @@ Load the `generating-specs` skill and generate the spec for: $ARGUMENTS
|
|
|
6
6
|
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
7
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
8
|
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
-
(Project rules). F1–F6 and R2 always win.
|
|
9
|
+
(Project rules). F1–F6 and R2 always win. The effective set is what `pnpm asrun rules` prints; §2 is applied as
|
|
10
|
+
written and is never turned into a question (a difference from a framework default is not a conflict).
|
|
10
11
|
Arguments: `<case-id> [<env>] [--force] [--hints <file>]`. The case file is cases/<case-id>.md (the id may include one group folder, e.g.
|
|
11
12
|
documents/landing-page). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If the spec already exists, it is
|
|
12
13
|
left alone unless `--force` is given; with `--force` it is regenerated and overwritten. `--hints <file>` (or free text
|
|
@@ -6,7 +6,8 @@ Load the `generating-specs` skill, section "Healing and refining a spec", and fo
|
|
|
6
6
|
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
7
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
8
|
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
-
(Project rules). F1–F6 and R2 always win.
|
|
9
|
+
(Project rules). F1–F6 and R2 always win. The effective set is what `pnpm asrun rules` prints; §2 is applied as
|
|
10
|
+
written and is never turned into a question (a difference from a framework default is not a conflict).
|
|
10
11
|
Arguments: $ARGUMENTS
|
|
11
12
|
Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
|
|
12
13
|
- `<case-id>` and `<env>` (`staging` or `production`) are required. There is no default environment. Without one, stop
|
|
@@ -6,6 +6,7 @@ Load the `making-cases` skill and turn this input into case documents: $ARGUMENT
|
|
|
6
6
|
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
7
|
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
8
|
framework skill, `.claude/skills/making-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
-
(Project rules). F1–F6 and R2 always win.
|
|
9
|
+
(Project rules). F1–F6 and R2 always win. The effective set is what `pnpm asrun rules` prints; §2 is applied as
|
|
10
|
+
written and is never turned into a question (a difference from a framework default is not a conflict).
|
|
10
11
|
The input is either text or a path to a file with the notes. Follow the skill's procedure exactly.
|
|
11
12
|
Ask me the open questions before writing any case file. Do not generate specs; that is `/gen-spec`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: exploring-cases
|
|
3
|
-
description: Explores the real pages for one case document (cases/<id>.md) before its spec is generated, for every logged-in profile or only the ones named with --profiles. Checks the case against profiles.yaml and the pages, fixes wording mistakes, records what the pages show, turns unknowns into answerable TODO blocks, and, in an attended run, lets the person do the steps that write data or need a choice while it waits and records a general recipe under that step's `- Recipe:` lines. Ends with RESULT ready or needs-input. Use when asked to explore, review, check, clean up or narrow down a case, or for /explore-case. Its open-question rules are shared with generating-specs.
|
|
3
|
+
description: Explores the real pages for one case document (cases/<id>.md) before its spec is generated, for every logged-in profile or only the ones named with --profiles. Checks the case against profiles.yaml and the pages, fixes wording mistakes, records what the pages show, turns unknowns into answerable TODO blocks, and, in an attended run, lets the person do the steps that write data or need a choice while it waits and records a general recipe under that step's `- Recipe:` lines. Also writes a local exploration note (locators and data-call paths) under `.explore/` for `/gen-spec` to verify instead of rediscovering. Ends with RESULT ready or needs-input. Use when asked to explore, review, check, clean up or narrow down a case, or for /explore-case. Its open-question rules are shared with generating-specs.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Exploring a case document
|
|
@@ -9,28 +9,35 @@ Read the workspace `AGENTS.md` (hard rules and project rules), `cases/_template.
|
|
|
9
9
|
(how a case's `## Rules` are read, applied and maintained), `references/open-questions.md` (how
|
|
10
10
|
questions are written, resolved and reported), `references/assertion-targets.md` (which exact image, text, file or
|
|
11
11
|
place a check looks at, and when to ask instead of picking), `references/readiness.md` (wait for the content before
|
|
12
|
-
you read it)
|
|
13
|
-
|
|
12
|
+
you read it), `references/tools.md` (the browser tools this MCP can run, and the substitutes for the ones it cannot),
|
|
13
|
+
`references/notes.md` (the local exploration note `/gen-spec` reads) and `references/user-handoff.md` (how a person works
|
|
14
|
+
in the browser while the agent waits, and how it is recorded). All but the last are binding for `/gen-spec` and healing
|
|
15
|
+
too.
|
|
14
16
|
|
|
15
17
|
## Before you start
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
`AGENTS.md`
|
|
19
|
-
|
|
18
|
+
Run `pnpm asrun rules --json` (or `pnpm asrun rules` for people): it prints the framework baseline, the project's
|
|
19
|
+
`AGENTS.md` §2 rules and the project settings (the test-data prefix). Those are the effective rules; if the command is
|
|
20
|
+
unavailable, read the workspace `AGENTS.md` directly. Also read `AGENTS.md` §2, and `.claude/skills/exploring-cases.md`
|
|
21
|
+
in the workspace, applying the override after this skill and §2 after both (the baseline, "Precedence"). F1–F6 and R2
|
|
22
|
+
always win. **Never turn a project rule into a question:** a difference between §2 and a framework default is not a
|
|
23
|
+
conflict; apply §2 and, if a row's id does not fit its text, apply the text and report the mismatch. This is not a stop
|
|
24
|
+
condition: continue with the procedure below. If `pnpm asrun rules` exits 1 (an invalid §2 row or a prefix that the
|
|
25
|
+
document and `package.json` disagree on), do not guess and do not turn it into a case question: report its errors and
|
|
26
|
+
end with `RESULT: needs-input`; a person fixes `AGENTS.md` or `package.json` first.
|
|
20
27
|
|
|
21
28
|
## Where this skill sits
|
|
22
29
|
Three steps, each learns one thing:
|
|
23
30
|
1. `/make-case` (making-cases): the **intent**. Turns raw notes into case documents. No browser.
|
|
24
31
|
2. `/explore-case` (this skill): the **facts and the inputs**. What the pages really show, and what a person has to
|
|
25
32
|
do for the steps that write or need a choice. The only step where a person and the agent work together.
|
|
26
|
-
3. `/gen-spec` (generating-specs): the **implementation**. Replays the case
|
|
27
|
-
writes the spec. It does not judge the case again.
|
|
33
|
+
3. `/gen-spec` (generating-specs): the **implementation**. Replays the case, its step recipes and the exploration note to
|
|
34
|
+
find locators and writes the spec. It does not judge the case again.
|
|
28
35
|
|
|
29
36
|
The purpose of this skill is to leave a case that can be replayed without anybody: no ambiguity, every fact marked
|
|
30
37
|
as observed, every input step described as a general recipe.
|
|
31
38
|
|
|
32
|
-
This skill edits `cases/<id>.md`
|
|
33
|
-
under `.auth/`.
|
|
39
|
+
This skill edits `cases/<id>.md` and writes its local exploration note under `.explore/notes/`. It never edits
|
|
40
|
+
`profiles.yaml`, `tests/`, the framework folder or anything under `.auth/`.
|
|
34
41
|
**The agent never writes data**, in staging or in production: a person does every step that writes or needs a
|
|
35
42
|
choice (see `references/user-handoff.md`).
|
|
36
43
|
|
|
@@ -68,7 +75,8 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
68
75
|
- ambiguous steps and checks without an expected result;
|
|
69
76
|
- steps that write data or need a choice: mark them `[writes]` / `[input]`. The front matter must list the
|
|
70
77
|
environments the finished spec may write in (`writes: [staging]`, or `[staging, production]`) with a
|
|
71
|
-
`writes-note`, the data must be named
|
|
78
|
+
`writes-note`, the data must be named with the project's test-data prefix (default `[asrun]`, printed by
|
|
79
|
+
`pnpm asrun rules`) followed by a run stamp, and cleaned up; a step that sends email is never
|
|
72
80
|
allowed in production and only in other environments when the profile has `email: true`;
|
|
73
81
|
- missing or doubtful `suites`/`envs`, a `writes` environment that `envs` excludes, and visual checks that are
|
|
74
82
|
too broad (screenshot a small element);
|
|
@@ -77,22 +85,14 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
77
85
|
3. **Repair the wording** (edit the file). Fix typos and grammar, make steps one action or one check each, put
|
|
78
86
|
assumptions in `## Notes` (not in `## Expected`), remove duplicates and template placeholders, make paths relative
|
|
79
87
|
to the app. Do not add checks the case did not ask for and do not drop any. Never edit the `## Source` table.
|
|
80
|
-
4. **Explore**. The exploration runs through the `playwright-test` MCP only
|
|
81
|
-
first browser call
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
Run `pnpm status --env <env>`. If it cannot run (a sandbox error such as
|
|
89
|
-
`listen EPERM` or a Chromium start failure), do not give up: open the profile's page through the MCP, and a page
|
|
90
|
-
that lands on the login page means the profile is not logged in. Explore each profile `--profiles` names that is
|
|
91
|
-
logged in, and, without `--profiles`, at least one logged-in profile per domain, with the `playwright-test` MCP
|
|
92
|
-
(`generator_setup_page` with `project=<profile-id>`, `seedFile=tests/seed.spec.ts` and `plan=""`). Never explore a
|
|
93
|
-
profile the selection leaves out, and never add a profile to fill
|
|
94
|
-
a gap. Profiles that are selected but not logged in are skipped and named in the report; a human runs
|
|
95
|
-
`pnpm signin <profile-id>`.
|
|
88
|
+
4. **Explore**. The exploration runs through the `playwright-test` MCP only: read `references/tools.md` before the
|
|
89
|
+
first browser call and follow its "Starting a session" and "Short calls" sections. If the tools are not registered
|
|
90
|
+
in this session at all (no `browser_*` / `generator_*` tool) or the server reports that it failed to start, do not
|
|
91
|
+
fall back to ad-hoc Playwright scripts: stop, tell the user which MCP server is missing and to run `pnpm asrun
|
|
92
|
+
doctor`, make no observations, and end with `RESULT: needs-input`.
|
|
93
|
+
Explore each profile `--profiles` names that is logged in, and, without `--profiles`, at least one logged-in profile
|
|
94
|
+
per domain. Never explore a profile the selection leaves out, and never add a profile to fill a gap. Profiles that
|
|
95
|
+
are selected but not logged in are skipped and named in the report; a human runs `pnpm signin <profile-id>`.
|
|
96
96
|
- Do the read-only steps yourself: open, look, read, hover, switch tabs, open and close dialogs and menus.
|
|
97
97
|
- **A step that writes or needs a choice is done by the person**, per `references/user-handoff.md`: announce it,
|
|
98
98
|
name the finish condition, run the wait loop (the card records what the person does), then write down what
|
|
@@ -109,38 +109,23 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
109
109
|
avoid is followed on every profile. Re-check each rule against the page: one that no longer holds gets a TODO
|
|
110
110
|
with evidence under it, never silent deletion or rewriting. A rule never lets you write data or do a person's
|
|
111
111
|
step.
|
|
112
|
-
- **Ready before you read
|
|
113
|
-
`
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
it
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
place or UI wanted and lists the candidates. In an attended run let the person name the element in the handoff
|
|
126
|
-
card, then outline what they named (`browser_annotate` is not runnable here).
|
|
127
|
-
- **Downloads and email composers are opened for real** (the composer is closed with "Discard"). A download is
|
|
128
|
-
verified in five parts: starts, finishes, file exists, file valid, and a failure or timeout fails the check
|
|
129
|
-
(see `references/readiness.md`). Record the file name, size and kind as observed. A downloaded PDF is rendered
|
|
130
|
-
(`pnpm pdf:render`) and its page images are opened and looked at: record which pages show the target, where, and
|
|
131
|
-
the text. Which pages the check covers follows the case: every page unless the case says ONLY (strong wording).
|
|
112
|
+
- **Ready before you read**: follow `references/readiness.md`, "Exploring". Record the signal it proves as a plain
|
|
113
|
+
note, `(seen on <profile>, after <signal>)`: the spec waits for it, so no person has to confirm it. When the first
|
|
114
|
+
read is empty or missing something, wait and read again before you record or ask anything.
|
|
115
|
+
- **Targets, not guesses**: follow `references/assertion-targets.md`, "How a card is made (exploring)". List ALL
|
|
116
|
+
candidates, outline the one that is meant, take the screenshot to `.explore/targets/<case-id>/`, **open it and look
|
|
117
|
+
at it**, clear the highlight and write the card under `## Targets`. When candidates cannot be told apart, or only a
|
|
118
|
+
position, a generic locator or dynamic data would decide it, write a `[blocking]` TODO that names the exact image,
|
|
119
|
+
text, place or UI wanted and lists the candidates. In an attended run let the person name the element in the
|
|
120
|
+
handoff card, then outline what they named (`browser_annotate` is not runnable here).
|
|
121
|
+
- **Downloads and email composers are opened for real** (the composer is closed with "Discard"). Verify a download
|
|
122
|
+
in the five parts of `references/readiness.md`, "Downloads", and record the file name, size and kind. Render a
|
|
123
|
+
downloaded PDF (`pnpm pdf:render`), open its page images and record which pages show the target, where, and the
|
|
124
|
+
text; the pages follow the case (every page unless the case says ONLY).
|
|
132
125
|
- Compare every step with what is really there.
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
click, wait for a dialog and read it in one call: split it, and read the page (`browser_snapshot`) between the
|
|
137
|
-
steps. Only the handoff wait loop may take minutes.
|
|
138
|
-
- The browser times out an action or a lookup after 15 s (the config sets it). Pass a shorter `{timeout: 5000}` when
|
|
139
|
-
you wait for something that may not appear, and use `.catch()` or `isVisible()` for a one-time look while exploring only.
|
|
140
|
-
`isVisible()` answers for this instant and ignores any `timeout` passed to it, so after an action take a snapshot or
|
|
141
|
-
wait for a signal first. Never copy this pattern into a spec (see `references/readiness.md`, "Optional confirmations").
|
|
142
|
-
- After a click that should open something, take a snapshot and see what opened before you wait for it by role. A
|
|
143
|
-
dialog that did not appear is a finding (what did open instead?), not something to wait for or retry.
|
|
126
|
+
- **Record the exploration note** for this profile as you go, in the shape of `references/notes.md`: the locator of
|
|
127
|
+
each step's main control, the data-call paths of each loading action, and each target it confirmed. It is local
|
|
128
|
+
cache under `.explore/`, never case content, and `/gen-spec` reads it; step 8 writes the file.
|
|
144
129
|
- A step that fails or times out twice is not tried a third time: write what you saw, add a `TODO [blocking]` under
|
|
145
130
|
the step and go on with the next one.
|
|
146
131
|
5. **Write the `## Targets` section and the ready signals.** One card per target, in the format of
|
|
@@ -149,7 +134,9 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
149
134
|
`[blocking]` TODO, never a guess. Check the screenshots you saved: each must show what its card says.
|
|
150
135
|
6. **Write the recipe lines** for every `[input]` or `[writes]` step that was handed off, from the card's `recorded`
|
|
151
136
|
result, as a general recipe under that step: one `- Recipe (phase): ...` line per phase. Format and rules are in
|
|
152
|
-
`references/user-handoff.md`.
|
|
137
|
+
`references/user-handoff.md`. What the case creates is named with the project's prefix and the run stamp
|
|
138
|
+
(`testDataName`); the recipe records how to build it, never the stamp itself. A step that was not handed off gets
|
|
139
|
+
no recipe line and a `[blocking]` TODO. A case
|
|
153
140
|
that still has the old `## Inputs` section: move the lines of the steps you touch under those steps as
|
|
154
141
|
`- Recipe:`; `pnpm case:check` names the rest.
|
|
155
142
|
7. **Resolve every open point** in the order of `references/open-questions.md`: answer it from the page, else apply
|
|
@@ -162,12 +149,18 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
162
149
|
the rules: apply what holds, move what belongs in another section, and leave a question under a rule that conflicts
|
|
163
150
|
or is obsolete. Then apply the `--hints` gate of `references/case-rules.md`: a reusable, general instruction goes
|
|
164
151
|
into `## Rules` (with a marker only when it changes what is asserted); a one-off clue stays in the report.
|
|
165
|
-
8. **
|
|
166
|
-
|
|
152
|
+
8. **Write the exploration note, then check.** After the case edits are done (so `caseSha256` is the case's final hash;
|
|
153
|
+
`pnpm asrun explore:notes <case-id> --hash`), write `.explore/notes/<case-id>/<profile-id>.json` for every profile
|
|
154
|
+
explored in this run, in the shape of `references/notes.md` and with only what was seen. Then run `pnpm case:check`
|
|
155
|
+
(it fails on an incomplete target card, a rule that names a missing step, and warns on a check that has none) and
|
|
156
|
+
`pnpm asrun explore:notes <case-id> --check` (it fails on a malformed note and warns on a stale one; the summary
|
|
157
|
+
line of every profile you explored must say `fresh`), and fix every FAIL. If
|
|
158
|
+
`case:check` fails with a sandbox error (`listen EPERM`), run
|
|
167
159
|
`node --import tsx <framework>/scripts/check-cases.ts` instead; it checks the same thing.
|
|
168
160
|
9. **Report** in the user's language: what was changed (wording fixes, observations added, TODOs closed or narrowed),
|
|
169
161
|
what was explored (profile ids), which profiles `--profiles` left out (and that a run with them covers the rest),
|
|
170
|
-
which steps the person did and what was recorded for them, what
|
|
162
|
+
which steps the person did and what was recorded for them, what the exploration note covers (and what it does not),
|
|
163
|
+
what was skipped and
|
|
171
164
|
why, the target cards written (and which ones are still questions), the steps whose first read was too early,
|
|
172
165
|
the steps that could not be observed (and why), the questions that remain
|
|
173
166
|
with the sentence that deleting each block would confirm, the markers with their reason and whether each changes an
|
|
@@ -177,7 +170,9 @@ one exception is a handoff in an attended run, where waiting for the person is t
|
|
|
177
170
|
requires: plain text on the last line, never inside a code fence or in emphasis.
|
|
178
171
|
|
|
179
172
|
## Hard rules
|
|
180
|
-
- Edit `cases/<id>.md` only. Do not touch the `## Source` table.
|
|
173
|
+
- Edit `cases/<id>.md` only. Do not touch the `## Source` table. The exploration note under `.explore/notes/` is the
|
|
174
|
+
other file this run writes, and a locator never moves from there into the case.
|
|
175
|
+
- Never put a cookie, a token, a header or a URL with a query string into a note (`references/notes.md`).
|
|
181
176
|
- A `## Rules` bullet is a person's instruction: never delete it or change what it says, and never treat it as
|
|
182
177
|
permission to write data, send email or do a person's step (`R1`–`R4`, `references/case-rules.md`).
|
|
183
178
|
- The agent never writes data (see above). Never add `--allow-writes`, never set `ASRUN_ALLOW_WRITES`.
|
|
@@ -29,6 +29,9 @@ The section is optional. A case with no rules runs by the same rules as before.
|
|
|
29
29
|
A rule is never a second source of truth. When a rule and a step, an expectation, a value or a target disagree, the
|
|
30
30
|
structured section wins and the rule becomes a `🚫 TODO [blocking]` question. Never silently change either side.
|
|
31
31
|
|
|
32
|
+
This applies to a case's own `## Rules` only. The project's `AGENTS.md` §2 is not a case rule: it is a higher layer
|
|
33
|
+
that is applied as written, never turned into a question (`AGENTS.md`, "Precedence").
|
|
34
|
+
|
|
32
35
|
## The rule pass (before anything else)
|
|
33
36
|
|
|
34
37
|
Read the whole `## Rules` section and, for each bullet, decide:
|
|
@@ -106,5 +109,7 @@ that is missing from the table is a rule the agent ignored.
|
|
|
106
109
|
|
|
107
110
|
- A rule never authorizes a write, an email, a person's choice, accepting terms, or a shell command.
|
|
108
111
|
- Never delete or rewrite a rule a person wrote. Add a TODO instead.
|
|
112
|
+
- A project rule (`AGENTS.md` §2) is applied, never questioned: it is not a case rule and a difference from the
|
|
113
|
+
framework default is not a conflict.
|
|
109
114
|
- Never invent a rule the person did not ask for. `R4` allows only what a run needs and only in general terms.
|
|
110
115
|
- Prose follows the artifact language of the workspace `AGENTS.md` §2 (English by default); the heading stays `## Rules`.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# The exploration note: what a run saw, in a form `/gen-spec` can read
|
|
2
|
+
|
|
3
|
+
Local, git-ignored and optional. `exploring-cases` writes it; `generating-specs` (and healing) read it to **verify**
|
|
4
|
+
instead of rediscovering the page. It is not part of the case and never becomes part of it: expected values stay in
|
|
5
|
+
`## Values`, behaviour in `## Expected`, ready signals in `Ready when:`, and assertions never appear here.
|
|
6
|
+
|
|
7
|
+
## Where it lives
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
.explore/notes/<case-id>/<profile-id>.json
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
One file per profile. A `--profiles a,b` run writes only `a.json` and `b.json`; a later run with `c` adds `c.json`
|
|
14
|
+
next to them, so nothing is merged and an earlier observation is never overwritten. A fresh checkout has no notes, and
|
|
15
|
+
that is a normal state: generation falls back to replaying the case.
|
|
16
|
+
|
|
17
|
+
## The shape
|
|
18
|
+
|
|
19
|
+
```json
|
|
20
|
+
{
|
|
21
|
+
"version": 1,
|
|
22
|
+
"caseId": "conduit/header-navigation",
|
|
23
|
+
"profile": "example-staging",
|
|
24
|
+
"env": "staging",
|
|
25
|
+
"caseSha256": "<pnpm asrun explore:notes <case-id> --hash>",
|
|
26
|
+
"capturedAt": "2026-11-09T10:20:31Z",
|
|
27
|
+
"steps": {
|
|
28
|
+
"2": {
|
|
29
|
+
"url": "/",
|
|
30
|
+
"locators": [{"purpose": "the header navigation link", "locator": "page.getByRole('link', {name: 'Home'})"}],
|
|
31
|
+
"requests": ["/api/articles"]
|
|
32
|
+
}
|
|
33
|
+
},
|
|
34
|
+
"targets": {"T1": {"locator": "page.getByRole('img', {name: 'Example'})", "matches": 1}}
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
- Keys of `steps` are the case's step numbers, keys of `targets` are its card ids (`T1`).
|
|
39
|
+
- `steps` may be `{}` and `targets` may be omitted; every other key is required (`env` and `updatedAt` are optional).
|
|
40
|
+
No other field is accepted.
|
|
41
|
+
- `caseSha256` is the 64-character hash `pnpm asrun explore:notes <case-id> --hash` prints (not the placeholder above).
|
|
42
|
+
- `pnpm asrun explore:notes <case-id> --check` validates the files: it fails on a malformed note and warns on a note
|
|
43
|
+
whose `caseSha256` no longer matches the case (the summary already prints it as `stale`; re-run the exploration, or
|
|
44
|
+
delete the note and let the case be replayed). A file that fails is ignored by `/gen-spec`.
|
|
45
|
+
|
|
46
|
+
## What goes in
|
|
47
|
+
|
|
48
|
+
`steps.<n>`
|
|
49
|
+
- `url`: the path the step runs on, relative to `baseURL` (`/documents/`), when it is not the current page.
|
|
50
|
+
- `locators`: the element the step acts on, not every element on the page. `purpose` says what it is in words; `locator`
|
|
51
|
+
is exactly what `browser_generate_locator` returned. Prefer the stable controls (`getByRole`, `getByTestId`,
|
|
52
|
+
`getByLabel`) and one entry per action.
|
|
53
|
+
- `requests`: the data-call paths the step's action started, taken from `browser_network_requests` above the marker
|
|
54
|
+
(`tools.md`, "Network requests without a clear"). Path only: `/api/articles`, never a host, a query string or a token.
|
|
55
|
+
Images, fonts and analytics are not data calls.
|
|
56
|
+
|
|
57
|
+
`targets.T1`
|
|
58
|
+
- `locator`: the card's `Identified by`, confirmed on the page.
|
|
59
|
+
- `matches`: how many elements it matched — `1` for a completed card. A card with open doubts gets no entry.
|
|
60
|
+
|
|
61
|
+
## What never goes in
|
|
62
|
+
|
|
63
|
+
- Expected values, counts, texts a check asserts: they belong in the case's `## Values` and `## Expected`.
|
|
64
|
+
- Ready signals: they belong in `Ready when:` (the note holds the request paths, not the promise).
|
|
65
|
+
- Assertions, test steps or a copy of the spec.
|
|
66
|
+
- Cookies, headers, storage state, tokens or a URL with a query string. `.explore/` is local, but a note is still not a
|
|
67
|
+
place for a credential (`F5`).
|
|
68
|
+
- A selector or an id that a future run would copy into `cases/*.md`. The case stays implementation-free; the note is
|
|
69
|
+
the only place a locator lives.
|
|
70
|
+
|
|
71
|
+
## Lists and changing data
|
|
72
|
+
|
|
73
|
+
A list, a row or a search result is recorded by its container and position (`page.getByRole('row').nth(0)`), never by
|
|
74
|
+
the name of the item, because the data changes between runs. The same rule as a recipe (`user-handoff.md`).
|
|
75
|
+
|
|
76
|
+
## Freshness and the write-back
|
|
77
|
+
|
|
78
|
+
- `caseSha256` is the sha256 of `cases/<case-id>.md` at the moment the note was written. If the case changes after that,
|
|
79
|
+
the note is **stale**: `/gen-spec` replays the case and does not use it.
|
|
80
|
+
- `/gen-spec` confirms an entry of a fresh note on the page and may write the verified locator or request back
|
|
81
|
+
(`updatedAt` gets the time). It never adds a step or a target, and it never touches a stale note.
|
|
82
|
+
- Deleting `.explore/notes/` is always safe: generation then re-derives everything from the page.
|
|
83
|
+
|
|
84
|
+
## Who reads it
|
|
85
|
+
|
|
86
|
+
Both skills read this file and `tools.md`, `readiness.md` and `assertion-targets.md`; the details of a browser call are
|
|
87
|
+
in `tools.md`, never repeated here.
|
|
@@ -110,6 +110,12 @@ answer, adds `(page shows <x> on <profile-id>)` and reports it as a finding. It
|
|
|
110
110
|
`[blocking]` question that now has an observation or a safe default is downgraded or closed in the file, and the
|
|
111
111
|
edit is part of the report. Do not carry old tags over.
|
|
112
112
|
|
|
113
|
+
## What is never a question
|
|
114
|
+
A rule in `AGENTS.md` §2 (a naming prefix, a language, a wait) is applied as written. A difference from the framework
|
|
115
|
+
default is not a conflict and is never a TODO, a marker or a question; a row whose id does not fit its text is applied
|
|
116
|
+
by its text and reported as a mismatch. A project-wide choice that the same answer would settle for every case belongs
|
|
117
|
+
in §2, not in one case's question.
|
|
118
|
+
|
|
113
119
|
## Resolution order (no question goes to the user while a better step exists)
|
|
114
120
|
1. **The page answers it.** Facts such as exact labels, which controls exist, where the logo is, what a menu holds
|
|
115
121
|
are answered by exploration (`/explore-case`; needs an environment and a logged-in profile; the agent only reads, a person does the steps that write). Rewrite the parent sentence
|
|
@@ -143,6 +149,8 @@ answer, adds `(page shows <x> on <profile-id>)` and reports it as a finding. It
|
|
|
143
149
|
names the exact image, text, place or UI wanted and lists the candidates. Never a safe default, because a wrong
|
|
144
150
|
guess passes silently.
|
|
145
151
|
- An intent question the page cannot answer ("should this also work for agents?").
|
|
152
|
+
- A literal the runner enforces and nobody settled in the case or in §2 (a name prefix, a value the code validates):
|
|
153
|
+
a wrong pick fails the run, so it is `[blocking]`, not `[non-blocking]`.
|
|
146
154
|
- No environment was given, or no profile is logged in, so a fact that needs the page cannot be observed.
|
|
147
155
|
|
|
148
156
|
## Final report
|
|
@@ -63,8 +63,9 @@ await expectImageLoaded(logo);
|
|
|
63
63
|
|
|
64
64
|
- Register the response wait **before** the action (`actAndWait` does).
|
|
65
65
|
- Every action that opens a dialog, a tab, a menu with data, a list or a preview waits for its key requests and its content
|
|
66
|
-
signal. The request URL patterns come from `
|
|
67
|
-
seen (the action was not run), say
|
|
66
|
+
signal. The request URL patterns come from the exploration note (`notes.md`) or, when it holds none, from
|
|
67
|
+
`browser_network_requests` during this replay; never from guesses. If none was seen (the action was not run), say
|
|
68
|
+
"not verified on the page" in the report.
|
|
68
69
|
- Never `waitForTimeout`, never `networkidle`. They wait for time or for silence, not for the content. `pnpm spec:lint` fails both.
|
|
69
70
|
- Never `if (await x.isVisible()) ...` to decide something that may not have loaded yet. Wait for a ready signal, then branch.
|
|
70
71
|
Where an element is genuinely optional, say what proves the page finished (`// ready: ...`) before looking for it.
|
|
@@ -23,6 +23,37 @@ substitute below instead, and say in the report which substitute you used.
|
|
|
23
23
|
`pnpm asrun doctor` prints how many tools the installed Playwright actually runs and warns when this file names one
|
|
24
24
|
it cannot (it happens when Playwright changes the filter; the file then needs the same edit).
|
|
25
25
|
|
|
26
|
+
## Starting a session
|
|
27
|
+
|
|
28
|
+
Both `exploring-cases` and `generating-specs` (and healing) drive the browser through this MCP, one profile at a time.
|
|
29
|
+
|
|
30
|
+
1. Check the session with `pnpm status --env <env>` (or `pnpm status` when the environment is the profile's). It can fail
|
|
31
|
+
with a sandbox error (`listen EPERM`, a Chromium start failure): then open the profile's page and judge by the login
|
|
32
|
+
page instead.
|
|
33
|
+
2. Start the session with `generator_setup_page` (`project=<profile-id>`, `seedFile=tests/seed.spec.ts`, `plan=""`).
|
|
34
|
+
Every later call belongs to that profile.
|
|
35
|
+
3. A profile that is not logged in is skipped and named in the report; a human runs `pnpm signin <profile-id>`. Never
|
|
36
|
+
explore a profile a selection leaves out (`--profiles` in exploration), and never add one to fill a gap.
|
|
37
|
+
|
|
38
|
+
If no `browser_*` / `generator_*` tool is registered at all, or the server reports that it failed to start, stop: tell
|
|
39
|
+
the user which MCP server is missing and to run `pnpm asrun doctor`, write nothing, and end with `RESULT: needs-input`.
|
|
40
|
+
|
|
41
|
+
## Short calls
|
|
42
|
+
|
|
43
|
+
A call that hangs is the normal failure mode of this MCP, so keep every call small.
|
|
44
|
+
|
|
45
|
+
- One `browser_run_code_unsafe` call does one thing and finishes within about 60 seconds. Do not close a tab, click,
|
|
46
|
+
wait for a dialog and read it in one call: split it and take a `browser_snapshot` between the steps. Only the handoff
|
|
47
|
+
wait loop may take minutes.
|
|
48
|
+
- The browser times out an action or a lookup after 15 s (the config sets it). Pass `{timeout: 5000}` when you wait for
|
|
49
|
+
something that may not appear, and use `.catch()` or `isVisible()` for a one-time look **while exploring only**.
|
|
50
|
+
`isVisible()` answers for this instant and ignores any `timeout` passed to it, so take a snapshot or wait for a signal
|
|
51
|
+
after an action first. Never copy this pattern into a spec (`readiness.md`, "Optional confirmations").
|
|
52
|
+
- After a click that should open something, take a snapshot and see what opened before you wait for it by role. A dialog
|
|
53
|
+
that did not appear is a finding (what did open instead?), not something to wait for or retry.
|
|
54
|
+
- A step that fails or times out twice is not tried a third time: record what you saw and go on (a `TODO [blocking]`
|
|
55
|
+
under the step in a case, a `needs-input` report when writing a spec).
|
|
56
|
+
|
|
26
57
|
## Runnable
|
|
27
58
|
|
|
28
59
|
Navigation and reading:
|
|
@@ -115,3 +146,6 @@ the card what the screenshot shows.
|
|
|
115
146
|
were started by the action. `browser_network_request({index})` gives one request's detail and status.
|
|
116
147
|
|
|
117
148
|
Record the marker as a plain note in the report (`after 12 new requests; the data call was 7`), never in the case.
|
|
149
|
+
|
|
150
|
+
4. While exploring, write the data-call paths of the action into the profile's exploration note (`notes.md`): path only,
|
|
151
|
+
no host, no query string. `/gen-spec` turns them into the `responses` of `actAndWait`.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# User handoff: a person works in the browser, the agent waits
|
|
2
2
|
|
|
3
3
|
Used by `exploring-cases` (and by `making-cases` never: it does not open a browser). `generating-specs` never hands
|
|
4
|
-
off; it replays the step recipes that an exploration recorded.
|
|
4
|
+
off; it replays the step recipes and the exploration note (`notes.md`) that an exploration recorded.
|
|
5
5
|
|
|
6
6
|
## The rule
|
|
7
7
|
**An agent never writes data.** Not in staging, not in production. Every step that creates, edits, publishes, shares,
|
|
@@ -25,10 +25,12 @@ that nobody watches.
|
|
|
25
25
|
## Before the handoff
|
|
26
26
|
1. Say what the user has to do and when it is finished, in one or two sentences ("Fill in the form, pick two
|
|
27
27
|
items, then press Done"). Name the finish condition first (below).
|
|
28
|
-
2. Put every text the user has to type into `copy` (the
|
|
29
|
-
copied with one click.
|
|
28
|
+
2. Put every text the user has to type into `copy` (the name of a document, a title), so it can be
|
|
29
|
+
copied with one click. Build a created name with the project's test-data prefix (default `[asrun]`, printed by
|
|
30
|
+
`pnpm asrun rules`) and a run stamp: `<prefix> <base> (<YYYY-MM-DD HH-mm-ss>, UTC)`, the same shape
|
|
31
|
+
`testDataName()` builds. The stamp is chosen now, not recorded in the recipe.
|
|
30
32
|
3. Never ask the user to send email (production: never; other environments: only when `email` is `true` for the
|
|
31
|
-
profile). Never ask for login or 2FA. Name everything the user creates
|
|
33
|
+
profile). Never ask for login or 2FA. Name everything the user creates `<prefix> <something> (<run stamp>)`.
|
|
32
34
|
4. Do not start a recording yourself: the card starts and stops it (see "Recording"). Never call
|
|
33
35
|
`browser_start_recording` or `browser_stop_recording`: the `playwright-test` MCP lists them but cannot run them
|
|
34
36
|
(`devtools` tools; see `tools.md`).
|
|
@@ -40,7 +42,7 @@ timeout), so no polling from outside is needed. The loop:
|
|
|
40
42
|
- **Shows a small floating card, not a bar.** It sits in the bottom-left corner and only the card takes clicks, so
|
|
41
43
|
nothing of the page is covered or blocked. The user can **drag** it by its title bar and **minimize** it to a small
|
|
42
44
|
pill that still has a Done button. Position and minimized state survive a page reload (kept in `sessionStorage`).
|
|
43
|
-
- **Click to copy.** Every text in `copy` (for example the
|
|
45
|
+
- **Click to copy.** Every text in `copy` (for example the `<prefix> <name> (<run stamp>)` the user has to type) is shown on its
|
|
44
46
|
own line with a copy icon; one click puts it on the clipboard and the icon shows "Copied". Pass only text that the
|
|
45
47
|
case says to type; leave `copy` empty when there is none.
|
|
46
48
|
- **Resets the flag first**: `window.__handoff = null` on every page it paints. A flag left over from an earlier
|
|
@@ -71,7 +73,7 @@ badge, no count in the tab title and no notification (a HINT in the job log says
|
|
|
71
73
|
async (page) => {
|
|
72
74
|
page.context().__handoffCfg = {
|
|
73
75
|
message: '<what the user does, one sentence>',
|
|
74
|
-
copy: [], // texts to type, e.g. ['[
|
|
76
|
+
copy: [], // texts to type, e.g. ['[autotest] published document (2026-11-09 12-34-56)']
|
|
75
77
|
doneText: null, // text that appears when finished, or null
|
|
76
78
|
maxMs: 600000,
|
|
77
79
|
};
|
|
@@ -114,8 +116,9 @@ The Done button is always there (also on the minimized card) as the fallback, an
|
|
|
114
116
|
- `outcome: 'timeout'` or `'skip'`: do not guess. Write a `TODO [blocking]` under the step and go on with what does
|
|
115
117
|
not depend on it.
|
|
116
118
|
- Take a snapshot, write down what the user did from the card's `recorded` (see "Recording"), then continue read-only.
|
|
117
|
-
- At the end of the exploration hand off once more when the user created data: "Delete the
|
|
118
|
-
created
|
|
119
|
+
- At the end of the exploration hand off once more when the user created data: "Delete the test items you
|
|
120
|
+
created (they start with <the project's prefix> and carry this run's timestamp), then press Done". The case must
|
|
121
|
+
clean up after itself, and so must the exploration.
|
|
119
122
|
|
|
120
123
|
## Recording
|
|
121
124
|
The card records what the person does: it starts the recording when it shows and stops it on every way out (Done,
|
|
@@ -134,7 +137,8 @@ Make the recipe as general as possible, because lists and data change:
|
|
|
134
137
|
name or id. "Select items #1 to #3 from the list."
|
|
135
138
|
- **Random is allowed only when the user asks for it** ("pick any 3"). The default is the position, because it can be
|
|
136
139
|
repeated. A random pick must be written into the test's annotations when the spec runs.
|
|
137
|
-
- **Text the case creates**:
|
|
140
|
+
- **Text the case creates**: `<prefix> <something> (<run stamp>)` (prefix from `package.json`
|
|
141
|
+
`"asrun.testDataPrefix"`, default `[asrun]`). The recipe says how to build it; it never records the stamp itself.
|
|
138
142
|
- **A value that has to exist** (an existing name, a search term): the recipe says what it must be like ("a name
|
|
139
143
|
that returns a result") and notes the observed value as plain text `(seen on <profile>: "<value>")`. Do not put a
|
|
140
144
|
brittle literal in as the only truth. One claim carries one signal: when a `[non-blocking]` TODO under the recipe
|