@asrunjs/asrun 0.2.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +18 -15
- package/.claude/commands/gen-spec.md +16 -13
- package/.claude/commands/heal-spec.md +26 -23
- package/.claude/commands/make-case.md +11 -8
- package/.claude/skills/exploring-cases/SKILL.md +146 -132
- package/.claude/skills/exploring-cases/references/open-questions.md +116 -112
- package/.claude/skills/exploring-cases/references/readiness.md +146 -137
- package/.claude/skills/exploring-cases/references/user-handoff.md +157 -147
- package/.claude/skills/generating-specs/SKILL.md +219 -207
- package/.claude/skills/generating-specs/references/conventions.md +148 -148
- package/.claude/skills/making-cases/SKILL.md +72 -64
- package/.claude/skills/making-cases/references/normalizing.md +39 -38
- package/.claude/skills/making-cases/references/writes-and-email.md +37 -36
- package/AGENTS.md +159 -154
- package/CLAUDE.md +2 -1
- package/README.md +512 -410
- package/dist/lib/build-artifacts.d.ts +8 -0
- package/dist/lib/build-artifacts.js +10 -0
- package/dist/lib/case-doc.d.ts +311 -0
- package/dist/lib/case-doc.js +1117 -0
- package/dist/lib/cases.d.ts +2 -15
- package/dist/lib/cases.js +5 -78
- package/dist/lib/command.d.ts +24 -0
- package/dist/lib/command.js +90 -0
- package/dist/lib/fixtures.js +2 -2
- package/dist/lib/login-card.d.ts +22 -0
- package/dist/lib/login-card.js +183 -0
- package/dist/lib/node-cli.d.ts +6 -2
- package/dist/lib/node-cli.js +16 -6
- package/dist/lib/profiles.d.ts +25 -1
- package/dist/lib/profiles.js +54 -1
- package/dist/lib/run-filters.js +10 -2
- package/dist/lib/session.d.ts +13 -1
- package/dist/lib/session.js +73 -7
- package/dist/lib/targets.d.ts +5 -13
- package/dist/lib/targets.js +4 -29
- package/dist/lib/values.js +2 -2
- package/dist/scripts/build.js +2 -1
- package/dist/scripts/check-cases.js +74 -6
- package/dist/scripts/doctor.js +115 -4
- package/dist/scripts/explore.js +22 -1
- package/dist/scripts/init.d.ts +7 -1
- package/dist/scripts/init.js +61 -13
- package/dist/scripts/lint-specs.js +4 -1
- package/dist/scripts/login.js +30 -2
- package/dist/ui/server/actions.d.ts +4 -0
- package/dist/ui/server/actions.js +25 -12
- package/dist/ui/server/ai-models.d.ts +3 -0
- package/dist/ui/server/ai-models.js +347 -0
- package/dist/ui/server/ai-stream.d.ts +34 -9
- package/dist/ui/server/ai-stream.js +283 -77
- package/dist/ui/server/app.js +74 -12
- package/dist/ui/server/jobs.d.ts +7 -1
- package/dist/ui/server/jobs.js +69 -15
- package/dist/ui/server/profiles.d.ts +20 -1
- package/dist/ui/server/profiles.js +214 -38
- package/dist/ui/shared/ai-commands.d.ts +16 -6
- package/dist/ui/shared/ai-commands.js +35 -12
- package/dist/ui/shared/types.d.ts +114 -0
- package/docs/agents/framework-dev.md +79 -0
- package/docs/theme-guide.html +922 -692
- package/package.json +12 -1
- package/templates/.claude/commands/explore-case.md +22 -21
- package/templates/.claude/commands/gen-spec.md +21 -20
- package/templates/.claude/commands/heal-spec.md +31 -30
- package/templates/.claude/commands/make-case.md +13 -12
- package/templates/.mcp.json +8 -8
- package/templates/AGENTS.md +64 -45
- package/templates/cases/_template.md +57 -63
- package/templates/opencode.jsonc +16 -11
- package/templates/profiles.yaml +42 -29
- package/ui/dist/assets/{dist-BHaIOZdW.js → dist-BGbe0-Aa.js} +1 -1
- package/ui/dist/assets/{dist-CBV7_vDN.js → dist-Ba-HC662.js} +1 -1
- package/ui/dist/assets/{dist-BiE4suo9.js → dist-BtMLMthJ.js} +1 -1
- package/ui/dist/assets/{dist-gF2bFn-5.js → dist-C-Oyo1ZT.js} +1 -1
- package/ui/dist/assets/{dist-DX8knLVr.js → dist-C8MdhqgG.js} +1 -1
- package/ui/dist/assets/{dist-DHCY9KpV.js → dist-CG1pHn5m.js} +1 -1
- package/ui/dist/assets/{dist-BwYRbe5s.js → dist-CmpteMYY.js} +1 -1
- package/ui/dist/assets/{dist-D_I45xgJ.js → dist-DSy3dVf4.js} +1 -1
- package/ui/dist/assets/{dist-BqAROxzI.js → dist-Dq6sW5Ur.js} +1 -1
- package/ui/dist/assets/{dist-BSJMM_jp.js → dist-jjxQutEm.js} +1 -1
- package/ui/dist/assets/index-CTmnF0Wp.css +2 -0
- package/ui/dist/assets/index-P_Yo0ZZn.js +338 -0
- package/ui/dist/help/ai-tools.png +0 -0
- package/ui/dist/help/case-explore.png +0 -0
- package/ui/dist/help/case-generate.png +0 -0
- package/ui/dist/help/cases-memo.png +0 -0
- package/ui/dist/help/profile-form.png +0 -0
- package/ui/dist/help/profiles-login.png +0 -0
- package/ui/dist/help/reports.png +0 -0
- package/ui/dist/help/run-tests.png +0 -0
- package/ui/dist/index.html +22 -16
- package/ui/dist/assets/index-DmasQ6Rq.css +0 -2
- package/ui/dist/assets/index-I7uU93kk.js +0 -294
|
@@ -1,36 +1,37 @@
|
|
|
1
|
-
# Steps that write data or send email
|
|
2
|
-
|
|
3
|
-
## Classify every step
|
|
4
|
-
| Class | Looks like | Marker |
|
|
5
|
-
|---|---|---|
|
|
6
|
-
| read-only | open, look, read, hover, switch tab or filter, open a preview, close a modal | none |
|
|
7
|
-
| writes | create, save, edit, publish, share, delete, upload, accept, assign a label, change a setting | `[writes]` |
|
|
8
|
-
| input | a step that needs a person's choice or typed value (an existing document, items, a contact name), writing or not | `[input]` |
|
|
9
|
-
| email | any step that makes the product send an email or notification (share by email, send to a recipient, invite a collaborator) | `[email]` |
|
|
10
|
-
|
|
11
|
-
A step can be several of these (a step that picks items and saves is `[input] [writes]`). A step is not read-only because the notes did not say it changes something: "Share" opens a
|
|
12
|
-
dialog (read-only) but sending the share is `[writes]` and probably `[email]`. When in doubt, mark it and ask.
|
|
13
|
-
|
|
14
|
-
## What a case with writes must look like
|
|
15
|
-
- Front matter: `writes: [<env>, ...]`, the environments the finished spec may write in (`[staging]`; add `production`
|
|
16
|
-
only when the notes or the user say so), and a `writes-note` that says what is created, changed and removed, in one
|
|
17
|
-
sentence ("Creates one draft document named '[asrun] ...', publishes it, deletes it at the end").
|
|
18
|
-
|
|
19
|
-
- Everything created is named `[asrun] <something>` (the spec enforces this).
|
|
20
|
-
- The case ends by deleting what it created. If the notes do not say how to delete it, that is a gap: ask, and
|
|
21
|
-
never guess a delete path. Until it is known, write the likeliest
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
- The case needs only data of the test account of the profile. It never touches data that existed before.
|
|
25
|
-
- Do not set `envs` just because a case writes. A case that does not list an environment under `writes` is skipped
|
|
26
|
-
there with a notice (not failed), and a human confirms a run in any environment except staging. When unsure, write
|
|
27
|
-
`writes: [staging]`.
|
|
28
|
-
- The agent never performs a write step while making, exploring or generating: a person does it in
|
|
29
|
-
`/explore-case --attended`, which records a general recipe under
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
1
|
+
# Steps that write data or send email
|
|
2
|
+
|
|
3
|
+
## Classify every step
|
|
4
|
+
| Class | Looks like | Marker |
|
|
5
|
+
|---|---|---|
|
|
6
|
+
| read-only | open, look, read, hover, switch tab or filter, open a preview, close a modal | none |
|
|
7
|
+
| writes | create, save, edit, publish, share, delete, upload, accept, assign a label, change a setting | `[writes]` |
|
|
8
|
+
| input | a step that needs a person's choice or typed value (an existing document, items, a contact name), writing or not | `[input]` |
|
|
9
|
+
| email | any step that makes the product send an email or notification (share by email, send to a recipient, invite a collaborator) | `[email]` |
|
|
10
|
+
|
|
11
|
+
A step can be several of these (a step that picks items and saves is `[input] [writes]`). A step is not read-only because the notes did not say it changes something: "Share" opens a
|
|
12
|
+
dialog (read-only) but sending the share is `[writes]` and probably `[email]`. When in doubt, mark it and ask.
|
|
13
|
+
|
|
14
|
+
## What a case with writes must look like
|
|
15
|
+
- Front matter: `writes: [<env>, ...]`, the environments the finished spec may write in (`[staging]`; add `production`
|
|
16
|
+
only when the notes or the user say so), and a `writes-note` that says what is created, changed and removed, in one
|
|
17
|
+
sentence ("Creates one draft document named '[asrun] ...', publishes it, deletes it at the end"). The `writes-note`
|
|
18
|
+
is the only summary; there is no separate Data section, and the removal itself is a step in `## Steps`.
|
|
19
|
+
- Everything created is named `[asrun] <something>` (the spec enforces this).
|
|
20
|
+
- The case ends by deleting what it created. If the notes do not say how to delete it, that is a gap: ask, and
|
|
21
|
+
never guess a delete path. Until it is known, write the likeliest delete step ("Select the 'Delete' option of the
|
|
22
|
+
item's menu on the list page.") and put a `TODO [blocking]: How is <thing> deleted?` block under that step (see
|
|
23
|
+
`gaps.md`).
|
|
24
|
+
- The case needs only data of the test account of the profile. It never touches data that existed before.
|
|
25
|
+
- Do not set `envs` just because a case writes. A case that does not list an environment under `writes` is skipped
|
|
26
|
+
there with a notice (not failed), and a human confirms a run in any environment except staging. When unsure, write
|
|
27
|
+
`writes: [staging]`.
|
|
28
|
+
- The agent never performs a write step while making, exploring or generating: a person does it in
|
|
29
|
+
`/explore-case --attended`, which records a general recipe under that step as `- Recipe (phase): ...`. Here, only
|
|
30
|
+
mark the step.
|
|
31
|
+
- Email: steps marked `[email]` are skipped by the runner in production and for profiles whose `email` is false.
|
|
32
|
+
Write the case so that skipping them does not break the following steps, or say in Notes that it does.
|
|
33
|
+
|
|
34
|
+
## Non-linear flows
|
|
35
|
+
Real flows often leave and come back (create a draft, stop before Publish, return to the list, check the draft,
|
|
36
|
+
edit it again, publish). Write them as one case with the steps in the real order, including the "return to the
|
|
37
|
+
list" steps. Do not split a flow only because it is long.
|
package/AGENTS.md
CHANGED
|
@@ -1,154 +1,159 @@
|
|
|
1
|
-
# AsRun — framework baseline
|
|
2
|
-
|
|
3
|
-
This file is the binding baseline for every project that uses the AsRun framework
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
Paths are relative to the **workspace root**: the project that owns `profiles.yaml`, `cases/` and `tests/`. In a
|
|
8
|
-
project that uses asrun, the framework files are in `node_modules/@asrunjs/asrun` (an installed dependency: never edit it).
|
|
9
|
-
`<framework>` below means that folder (`node_modules/@asrunjs/asrun` in a project, `.` in the framework repository).
|
|
10
|
-
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
- **
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
- **
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
`
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
- **
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
- `
|
|
103
|
-
|
|
104
|
-
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
-
|
|
117
|
-
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
project
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
case
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
1
|
+
# AsRun — framework baseline
|
|
2
|
+
|
|
3
|
+
This file is the binding baseline for every project that uses the AsRun framework. It describes no particular
|
|
4
|
+
product: a project adds its own rules in its own `AGENTS.md` §2, and they win where they differ (see "Precedence").
|
|
5
|
+
Its hard rules carry ids (`F1`..`F10`, `B1`..`B4`, `A1`..`A6`).
|
|
6
|
+
|
|
7
|
+
Paths are relative to the **workspace root**: the project that owns `profiles.yaml`, `cases/` and `tests/`. In a
|
|
8
|
+
project that uses asrun, the framework files are in `node_modules/@asrunjs/asrun` (an installed dependency: never edit it).
|
|
9
|
+
`<framework>` below means that folder (`node_modules/@asrunjs/asrun` in a project, `.` in the framework repository).
|
|
10
|
+
|
|
11
|
+
## Precedence
|
|
12
|
+
|
|
13
|
+
Rules are applied in this order; a later layer wins where they conflict.
|
|
14
|
+
1. **F1–F6: safety.** Nothing overrides them, in any project. They are enforced in code as well
|
|
15
|
+
(`lib/write-gate.ts`, `lib/session.ts`, the run filters), so a rule file cannot weaken them.
|
|
16
|
+
2. **This file.** The generic facts and defaults.
|
|
17
|
+
3. **The framework skills** (`<framework>/.claude/skills/**`).
|
|
18
|
+
4. **A project's skill override**: `.claude/skills/<skill>.md` in the workspace, if it exists. It refines that skill's
|
|
19
|
+
own procedure.
|
|
20
|
+
5. **The project's `AGENTS.md` §2 (Project rules).** It states the project's intent and wins over layers 2–4.
|
|
21
|
+
|
|
22
|
+
A project writes §2 only for what differs from the defaults here, names the rule id, and says whether it `narrows` or
|
|
23
|
+
`replaces` the rule. `replaces` is refused for F1–F6; `pnpm asrun doctor` checks the table. A rule file never renames
|
|
24
|
+
the structural tokens under "Fixed tokens".
|
|
25
|
+
|
|
26
|
+
## Hard rules
|
|
27
|
+
- **F1 — Read-only by default.** [non-negotiable] Tests never create, edit, delete, submit forms that change data,
|
|
28
|
+
send messages, or accept terms. Only a case that lists environments under `writes` in its front matter
|
|
29
|
+
(`writes: [staging]`, `writes: [staging, production]`; with a `writes-note`) may write, only in those environments,
|
|
30
|
+
and only data of the test accounts, which it must clean up. In any other environment the case is skipped with a
|
|
31
|
+
notice, not failed. The `writes` environments must be ones that `envs` lets the case run in. The permission is
|
|
32
|
+
checked at run time.
|
|
33
|
+
- **F2 — No bypassing the write confirmation.** [non-negotiable] A case that writes in any environment except staging
|
|
34
|
+
needs a human's confirmation on every run (typing the environment name in the CLI, or the confirm dialog in the web
|
|
35
|
+
UI). AI agents must NEVER add `--allow-writes`, set `ASRUN_ALLOW_WRITES`, or bypass the confirmation. Use
|
|
36
|
+
staging, or `--read-only`, instead.
|
|
37
|
+
- **F3 — Agents never write while exploring.** [non-negotiable] A step that creates, edits, publishes, shares or
|
|
38
|
+
deletes, and a step that needs a person's choice, is done by the person in the browser window
|
|
39
|
+
(`/explore-case --attended`, see `<framework>/.claude/skills/exploring-cases/references/user-handoff.md`). The
|
|
40
|
+
agent only reads. Without `--attended` an agent never waits for a person.
|
|
41
|
+
- **F4 — No email in production.** [non-negotiable] Never send email from a test in production. In other
|
|
42
|
+
environments only if the profile has `email: true`; new and AI-generated profile entries always get `email: false`.
|
|
43
|
+
- **F5 — Saved logins are passwords.** [non-negotiable] Never read, print, copy or commit anything under `.auth/`.
|
|
44
|
+
- **F6 — No 2FA automation.** [non-negotiable] Login is always done by a human via `pnpm signin <profile-id>`.
|
|
45
|
+
- **F7 — No default environment.** `--env <env>` is required by `pnpm asrun` and by every AI command.
|
|
46
|
+
`pnpm signin <profile-id>` and `pnpm explore <profile-id>` need no `--env`: the profile already names its
|
|
47
|
+
environment. Sessions are saved per profile in `.auth/<profile-id>.json`.
|
|
48
|
+
- **F8 — Login is not a test case.** `pnpm asrun` checks the session of every profile first; a profile that is not
|
|
49
|
+
logged in (missing or expired token) fails right away with a hint to run `pnpm signin <profile-id>` or to skip it
|
|
50
|
+
with `--skip-profile <profile-id>`. Never write specs for the login page or login flow.
|
|
51
|
+
- **F9 — Expected values live in the case document.** They go in its `## Values` block, never in `profiles.yaml`. A
|
|
52
|
+
profile holds connection settings only; adding or changing a profile must not touch any case value.
|
|
53
|
+
- **F10 — Artifact language.** Human-readable artifacts (case prose, report summaries, assertion messages) are
|
|
54
|
+
written in English by default. A project replaces this in §2 of its own `AGENTS.md` (for example Simplified Chinese
|
|
55
|
+
for case prose and reports). Code, comments, identifiers, commit messages and the structural tokens under
|
|
56
|
+
"Fixed tokens" always stay in English.
|
|
57
|
+
|
|
58
|
+
## Browser exploration rules (`B`)
|
|
59
|
+
These apply to every agent that drives a browser to explore pages for cases or specs (`making-cases`,
|
|
60
|
+
`exploring-cases`, `generating-specs`, the `playwright-test` MCP).
|
|
61
|
+
- **B1 — Read-only for the agent.** Open, look, read, hover, switch tabs, open and close dialogs and menus.
|
|
62
|
+
Everything that writes is the person's (see F3).
|
|
63
|
+
- **B2 — Waiting for a person.** Only in an attended run. Announce the task in the browser window (a small draggable
|
|
64
|
+
card with a Done button and click-to-copy for texts to type), name the finish condition, wait, and tell the person
|
|
65
|
+
when control is back. Record what they did as a general recipe under the step (`- Recipe (phase): ...`): positions
|
|
66
|
+
instead of names of data that can change.
|
|
67
|
+
- **B3 — Closing an email composer.** When closing an email composer makes a confirm dialog appear (for example
|
|
68
|
+
"Discard draft?"), choose "Discard" and continue. Never choose "Save" or "Send"; closing the composer must not keep
|
|
69
|
+
or send anything. The dialog can show up late: in a spec, clicking "Discard" and checking that the composer is
|
|
70
|
+
closed happen in one retry loop (`clickOptionalUntilHidden` in `@asrunjs/asrun/ready`), never after a single
|
|
71
|
+
`isVisible()` look.
|
|
72
|
+
- **B4 — Record it in the case.** If the case file does not already contain this step, do not edit the flow silently.
|
|
73
|
+
Add it to the case as a step with a `❓ TODO [non-blocking]` block for the user's confirmation, in the format of
|
|
74
|
+
`<framework>/.claude/skills/exploring-cases/references/open-questions.md`, for example:
|
|
75
|
+
```
|
|
76
|
+
4. Close the email composer; if a confirm dialog appears, choose "Discard".
|
|
77
|
+
- ❓ TODO [non-blocking]: Is choosing "Discard" in the close-composer confirm dialog correct for this case? (seen while exploring <profile-id>)
|
|
78
|
+
- ✍️ Answer:
|
|
79
|
+
```
|
|
80
|
+
Put the step right after the step that opens the composer (or after the last step that uses it). Mention it in the
|
|
81
|
+
final report so the user can confirm it. If the case already has the step, just follow it and add nothing.
|
|
82
|
+
|
|
83
|
+
## Assertions and readiness (`A`, binding for exploring, generating and healing)
|
|
84
|
+
Details: `<framework>/.claude/skills/exploring-cases/references/assertion-targets.md` and `readiness.md`.
|
|
85
|
+
- **A1 — Never guess what a check looks at.** A check on a logo, image, icon, file, PDF, screenshot or repeated text
|
|
86
|
+
names its exact target in a card in the case (`## Targets`: What, Where, Identified by, Not, Ready when, a
|
|
87
|
+
highlighted reference screenshot, referenced as `[T<n>]`). When in doubt (several candidates, only a position or a
|
|
88
|
+
generic locator, dynamic data in the area, no stable identity, the page differs from the case, it could not be
|
|
89
|
+
seen), write a `🚫 TODO [blocking]` that names the exact image, text, place or UI wanted and lists the candidates.
|
|
90
|
+
Do not pick. Look at the screenshot you saved: it must show what the card says.
|
|
91
|
+
- **A2 — Wait for the content before you read it.** After every action that loads something, wait for its key
|
|
92
|
+
requests to finish and for a positive content signal, then read. The same in exploration
|
|
93
|
+
(`browser_network_requests`, `browser_wait_for`), in specs (`actAndWait` in `@asrunjs/asrun/ready`) and in healing. A
|
|
94
|
+
"missing", "empty" or "hidden" result is only valid after a positive ready signal. Never `waitForTimeout`, never
|
|
95
|
+
`networkidle`.
|
|
96
|
+
- **A3 — Verify every download in five parts** with `verifyDownload` (`@asrunjs/asrun/downloads`): starts, finishes, file
|
|
97
|
+
exists, file is valid, and any failure or timeout fails the whole step.
|
|
98
|
+
- **A4 — A PDF is read like a person reads it.** Its pages are rendered to images (logo regions must not be blank and
|
|
99
|
+
match the profile's baseline, 2%) and its text is extracted (`expectPdf` in `@asrunjs/asrun/pdf`; `pnpm pdf:render`
|
|
100
|
+
while exploring). Every page is checked unless the case limits the pages in strong words ("only", "exactly",
|
|
101
|
+
"must"); "should", "always", "usually" and no wording at all mean every page.
|
|
102
|
+
- **A5 — No hard assertion on what nobody saw on a page.** Specs are checked with `pnpm spec:lint`; baselines with
|
|
103
|
+
`pnpm baselines:review` (open every new image and compare it with its card).
|
|
104
|
+
- **A6 — Observe first, delete last.** Everything that needs the data (preview, share menus, composers, downloads) is
|
|
105
|
+
observed before the person deletes it.
|
|
106
|
+
|
|
107
|
+
## Fixed tokens
|
|
108
|
+
`lib/case-doc.ts`, the scripts and the web panel parse these literals. A project language never translates them, and
|
|
109
|
+
a project rule never renames them.
|
|
110
|
+
- Section headings: `## Goal`, `## Steps`, `## Expected`, `## Values`, `## Visual checks`, `## Targets`, `## Notes`,
|
|
111
|
+
`## Source` (only when the case was made from notes).
|
|
112
|
+
- A step's recipe lines: `- Recipe (phase): ...` under the step it belongs to.
|
|
113
|
+
- `### T<n>` target cards and the `[T<n>]` references to them.
|
|
114
|
+
- Question blocks: `🚫 TODO [blocking]`, `❓ TODO [non-blocking]`, `✍️ Answer:` (with its emoji), and the markers
|
|
115
|
+
`🔍 to be confirmed by QA` / `👁️ not observed`.
|
|
116
|
+
- Step tags `[input]`, `[writes]`, `[email]`, and the last line `RESULT: ready` / `RESULT: needs-input`.
|
|
117
|
+
- Front-matter keys: `id`, `title`, `login`, `suites`, `writes`, `writes-note`, `envs`.
|
|
118
|
+
|
|
119
|
+
## Layout
|
|
120
|
+
|
|
121
|
+
**The workspace** (committed by the project; the framework never holds these):
|
|
122
|
+
- `cases/**/*.md` human-written test cases (input of the AI workflow); `cases/<group>/<name>.md` groups an area
|
|
123
|
+
- `tests/**/*.spec.ts` generated Playwright specs (output of the AI workflow), one per case, same folder as the case,
|
|
124
|
+
with their `*-snapshots/` baselines next to them
|
|
125
|
+
- `profiles.yaml` one entry per environment: URL, account label, tags, environment (connection settings only)
|
|
126
|
+
- `AGENTS.md` the project's own rules: they point at this baseline, and §2 holds what differs
|
|
127
|
+
- `.auth/` `reports/` `.tmp/` `.explore/` local state, never committed
|
|
128
|
+
|
|
129
|
+
The framework folder is read-only in a project: never edit it. Its own layout and how to change it are in
|
|
130
|
+
`docs/agents/framework-dev.md`.
|
|
131
|
+
|
|
132
|
+
## Workflow
|
|
133
|
+
Three steps, each learns one thing: the intent, the facts and inputs, the implementation.
|
|
134
|
+
1. `making-cases` (or `/make-case <notes>`) turns raw notes into case documents. It writes `cases/*.md` only and
|
|
135
|
+
never opens a browser. Check them with `pnpm case:check`.
|
|
136
|
+
2. `exploring-cases` (or `/explore-case <case-id> <env> [--attended]`) explores the real pages for one case. It
|
|
137
|
+
edits that case file only: wording, observations, TODO blocks, and, with `--attended`, the `- Recipe:` lines of
|
|
138
|
+
the steps that write or need a choice.
|
|
139
|
+
3. `generating-specs` (or `/gen-spec <case-id> <env>`) turns the case into a spec. It replays the case and its
|
|
140
|
+
step recipes and `## Values`, writes the spec (values stay in the case document), but never runs the tests, never
|
|
141
|
+
writes data and never creates baselines. A tagged step with no `- Recipe:` line (an old `## Inputs` line counts
|
|
142
|
+
until it is moved) stops with `needs-input`: run `/explore-case` first. Running specs is a separate step
|
|
143
|
+
(`pnpm asrun`, or the Run page).
|
|
144
|
+
Do not hand-write specs unless asked. Do not edit `lib/` to make a single spec pass.
|
|
145
|
+
Every case step is wrapped in `test.step('<n>. <case step text>', ...)` so reports and traces show the
|
|
146
|
+
case's own steps (see the skill's `references/conventions.md`).
|
|
147
|
+
Every AI command needs an environment (`--env`/`<env>`); there is no default and no "no environment" mode.
|
|
148
|
+
`/heal-spec <case-id> <env>` reads the existing report first (`pnpm heal:evidence --case <id> --env <env>`), reads the
|
|
149
|
+
user's optional clues (`--hints <file>`), runs only the profiles that are logged in (`--logged-in`) and runs the case
|
|
150
|
+
again only when the report is missing, stale or not enough.
|
|
151
|
+
Run `pnpm typecheck`, `pnpm spec:lint <case-id>` and `pnpm case:check` before reporting that a spec was generated. Run
|
|
152
|
+
`pnpm asrun --env <env> --profile <id> --grep <case-id>` only when healing a spec or when asked to.
|
|
153
|
+
Pick `--env staging` while developing. A case that does not list `production` under `writes` is skipped in
|
|
154
|
+
production with a notice; add `--read-only` to leave all write cases out.
|
|
155
|
+
|
|
156
|
+
## Working on the framework itself
|
|
157
|
+
This file is the product baseline. How to change asrun (repository layout, templates and versioning, the web UI
|
|
158
|
+
theme, developing against `examples/demo`) is in `docs/agents/framework-dev.md`; read it only when working in the
|
|
159
|
+
framework repository.
|
package/CLAUDE.md
CHANGED
|
@@ -1 +1,2 @@
|
|
|
1
|
-
@AGENTS.md
|
|
1
|
+
@AGENTS.md
|
|
2
|
+
@docs/agents/framework-dev.md
|