@asrunjs/asrun 0.2.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +18 -15
- package/.claude/commands/gen-spec.md +16 -13
- package/.claude/commands/heal-spec.md +26 -23
- package/.claude/commands/make-case.md +11 -8
- package/.claude/skills/exploring-cases/SKILL.md +146 -132
- package/.claude/skills/exploring-cases/references/open-questions.md +116 -112
- package/.claude/skills/exploring-cases/references/readiness.md +146 -137
- package/.claude/skills/exploring-cases/references/user-handoff.md +157 -147
- package/.claude/skills/generating-specs/SKILL.md +219 -207
- package/.claude/skills/generating-specs/references/conventions.md +148 -148
- package/.claude/skills/making-cases/SKILL.md +72 -64
- package/.claude/skills/making-cases/references/normalizing.md +39 -38
- package/.claude/skills/making-cases/references/writes-and-email.md +37 -36
- package/AGENTS.md +159 -154
- package/CLAUDE.md +2 -1
- package/README.md +512 -410
- package/dist/lib/build-artifacts.d.ts +8 -0
- package/dist/lib/build-artifacts.js +10 -0
- package/dist/lib/case-doc.d.ts +311 -0
- package/dist/lib/case-doc.js +1117 -0
- package/dist/lib/cases.d.ts +2 -15
- package/dist/lib/cases.js +5 -78
- package/dist/lib/command.d.ts +24 -0
- package/dist/lib/command.js +90 -0
- package/dist/lib/fixtures.js +2 -2
- package/dist/lib/login-card.d.ts +22 -0
- package/dist/lib/login-card.js +183 -0
- package/dist/lib/node-cli.d.ts +6 -2
- package/dist/lib/node-cli.js +16 -6
- package/dist/lib/profiles.d.ts +25 -1
- package/dist/lib/profiles.js +54 -1
- package/dist/lib/run-filters.js +10 -2
- package/dist/lib/session.d.ts +13 -1
- package/dist/lib/session.js +73 -7
- package/dist/lib/targets.d.ts +5 -13
- package/dist/lib/targets.js +4 -29
- package/dist/lib/values.js +2 -2
- package/dist/scripts/build.js +2 -1
- package/dist/scripts/check-cases.js +74 -6
- package/dist/scripts/doctor.js +115 -4
- package/dist/scripts/explore.js +22 -1
- package/dist/scripts/init.d.ts +7 -1
- package/dist/scripts/init.js +61 -13
- package/dist/scripts/lint-specs.js +4 -1
- package/dist/scripts/login.js +30 -2
- package/dist/ui/server/actions.d.ts +4 -0
- package/dist/ui/server/actions.js +25 -12
- package/dist/ui/server/ai-models.d.ts +3 -0
- package/dist/ui/server/ai-models.js +347 -0
- package/dist/ui/server/ai-stream.d.ts +34 -9
- package/dist/ui/server/ai-stream.js +283 -77
- package/dist/ui/server/app.js +74 -12
- package/dist/ui/server/jobs.d.ts +7 -1
- package/dist/ui/server/jobs.js +69 -15
- package/dist/ui/server/profiles.d.ts +20 -1
- package/dist/ui/server/profiles.js +214 -38
- package/dist/ui/shared/ai-commands.d.ts +16 -6
- package/dist/ui/shared/ai-commands.js +35 -12
- package/dist/ui/shared/types.d.ts +114 -0
- package/docs/agents/framework-dev.md +79 -0
- package/docs/theme-guide.html +922 -692
- package/package.json +12 -1
- package/templates/.claude/commands/explore-case.md +22 -21
- package/templates/.claude/commands/gen-spec.md +21 -20
- package/templates/.claude/commands/heal-spec.md +31 -30
- package/templates/.claude/commands/make-case.md +13 -12
- package/templates/.mcp.json +8 -8
- package/templates/AGENTS.md +64 -45
- package/templates/cases/_template.md +57 -63
- package/templates/opencode.jsonc +16 -11
- package/templates/profiles.yaml +42 -29
- package/ui/dist/assets/{dist-BHaIOZdW.js → dist-BGbe0-Aa.js} +1 -1
- package/ui/dist/assets/{dist-CBV7_vDN.js → dist-Ba-HC662.js} +1 -1
- package/ui/dist/assets/{dist-BiE4suo9.js → dist-BtMLMthJ.js} +1 -1
- package/ui/dist/assets/{dist-gF2bFn-5.js → dist-C-Oyo1ZT.js} +1 -1
- package/ui/dist/assets/{dist-DX8knLVr.js → dist-C8MdhqgG.js} +1 -1
- package/ui/dist/assets/{dist-DHCY9KpV.js → dist-CG1pHn5m.js} +1 -1
- package/ui/dist/assets/{dist-BwYRbe5s.js → dist-CmpteMYY.js} +1 -1
- package/ui/dist/assets/{dist-D_I45xgJ.js → dist-DSy3dVf4.js} +1 -1
- package/ui/dist/assets/{dist-BqAROxzI.js → dist-Dq6sW5Ur.js} +1 -1
- package/ui/dist/assets/{dist-BSJMM_jp.js → dist-jjxQutEm.js} +1 -1
- package/ui/dist/assets/index-CTmnF0Wp.css +2 -0
- package/ui/dist/assets/index-P_Yo0ZZn.js +338 -0
- package/ui/dist/help/ai-tools.png +0 -0
- package/ui/dist/help/case-explore.png +0 -0
- package/ui/dist/help/case-generate.png +0 -0
- package/ui/dist/help/cases-memo.png +0 -0
- package/ui/dist/help/profile-form.png +0 -0
- package/ui/dist/help/profiles-login.png +0 -0
- package/ui/dist/help/reports.png +0 -0
- package/ui/dist/help/run-tests.png +0 -0
- package/ui/dist/index.html +22 -16
- package/ui/dist/assets/index-DmasQ6Rq.css +0 -2
- package/ui/dist/assets/index-I7uU93kk.js +0 -294
|
@@ -1,15 +1,18 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Explore the real pages for one case (optionally attended: you do the steps that write data), before generating the spec
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
Load the `exploring-cases` skill and explore: $ARGUMENTS
|
|
6
|
-
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
1
|
+
---
|
|
2
|
+
description: Explore the real pages for one case (optionally attended: you do the steps that write data), before generating the spec
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `exploring-cases` skill and explore: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/exploring-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: `<case-id> <env> [--attended]`. The case id is the path below cases/ without `.md` (it may include a group
|
|
11
|
+
folder, e.g. documents/landing-page) and is required. `<env>` is `staging` or `production`; it is required and there is no
|
|
12
|
+
default (without one, stop and say so). With `--attended` I am at the machine and the browser window is visible: when a step
|
|
13
|
+
writes data or needs a choice, hand it to me as the skill describes, wait for me, and record what I did as a general
|
|
14
|
+
recipe under the step it belongs to (`- Recipe:` lines). Without `--attended`, never wait for me.
|
|
15
|
+
Follow `assertion-targets.md` (list the candidates, never pick when in doubt, ask for the exact image/text/place/UI)
|
|
16
|
+
and `readiness.md` (wait for the key requests and the content before you read anything) of the skill.
|
|
17
|
+
This command edits cases/<case-id>.md only (plus screenshots under `.explore/targets/`). Do not ask me questions in the chat: write open questions into the case
|
|
18
|
+
file as the skill describes, and end with the `RESULT:` line.
|
|
@@ -1,13 +1,16 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Generate or update tests/<id>.spec.ts from cases/<id>.md (the id may be <group>/<name>)
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
Load the `generating-specs` skill and generate the spec for: $ARGUMENTS
|
|
6
|
-
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
1
|
+
---
|
|
2
|
+
description: Generate or update tests/<id>.spec.ts from cases/<id>.md (the id may be <group>/<name>)
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `generating-specs` skill and generate the spec for: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: `<case-id> [<env>] [--force]`. The case file is cases/<case-id>.md (the id may include one group folder, e.g.
|
|
11
|
+
documents/landing-page). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If the spec already exists, it is
|
|
12
|
+
left alone unless `--force` is given; with `--force` it is regenerated and overwritten.
|
|
13
|
+
Follow the skill's procedure exactly, including `assertion-targets.md` and `readiness.md`; run `pnpm spec:lint <case-id>`. Only generate the spec: do not run `pnpm asrun` (running specs is a separate
|
|
14
|
+
process), and do not create baselines.
|
|
15
|
+
Do not ask me questions: resolve ambiguity as the skill describes (explore the page, use observed values marked for QA,
|
|
16
|
+
write the remaining questions into the case file), and end with the `RESULT:` line.
|
|
@@ -1,23 +1,26 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Diagnose and fix a failing spec from its last report (without hiding real product regressions)
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
Load the `generating-specs` skill, section "Healing a failing spec", and follow it exactly.
|
|
6
|
-
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
-
|
|
12
|
-
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
1
|
+
---
|
|
2
|
+
description: Diagnose and fix a failing spec from its last report (without hiding real product regressions)
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `generating-specs` skill, section "Healing a failing spec", and follow it exactly.
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: $ARGUMENTS
|
|
11
|
+
Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
|
|
12
|
+
- `<case-id>` and `<env>` (`staging` or `production`) are required. There is no default environment. Without one, stop
|
|
13
|
+
and say so.
|
|
14
|
+
- `--profiles`: the profiles to heal. Without it, every profile that is logged in in `<env>`.
|
|
15
|
+
- `--report`: the report (folder name under `reports/`) to read first. Without it, the newest report of this case.
|
|
16
|
+
- `--hints <file>`: a file with what the user knows (what to heal, evidence, clues). Read it first. Any free text
|
|
17
|
+
after the arguments is the same kind of clue. Both are optional.
|
|
18
|
+
Before you decide "product or test", audit the assertion itself (target card, baseline content, readiness): see steps 4
|
|
19
|
+
of the skill's healing section, `assertion-targets.md` and `readiness.md`. Run `pnpm spec:lint <case-id>`.
|
|
20
|
+
Read the existing report first (`pnpm heal:evidence --case <case-id> --env <env>`), and run the case again only when the
|
|
21
|
+
skill says it is needed. Always run with `--logged-in`. Tell me whether it is a test problem or a product finding.
|
|
22
|
+
|
|
23
|
+
Run `pnpm asrun ...`, `pnpm heal:evidence ...`, `pnpm spec:lint ...`, `pnpm baselines:review ...` and `pnpm status ...` as plain commands, optionally piped to `tail` or
|
|
24
|
+
`head`, and do not chain other commands with them (`;` or `&&`). Only these exact forms run outside Claude Code's
|
|
25
|
+
sandbox, which tsx and the browser need. If a command still fails with `listen EPERM` or a sandbox error, stop and
|
|
26
|
+
report it. Do not ask to disable the sandbox, and never add `--allow-writes` or set `ASRUN_ALLOW_WRITES`.
|
|
@@ -1,8 +1,11 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Turn raw test notes (pasted text or a file) into case documents in cases/
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
Load the `making-cases` skill and turn this input into case documents: $ARGUMENTS
|
|
6
|
-
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
-
|
|
8
|
-
|
|
1
|
+
---
|
|
2
|
+
description: Turn raw test notes (pasted text or a file) into case documents in cases/
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `making-cases` skill and turn this input into case documents: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/making-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
The input is either text or a path to a file with the notes. Follow the skill's procedure exactly.
|
|
11
|
+
Ask me the open questions before writing any case file. Do not generate specs; that is `/gen-spec`.
|
|
@@ -1,132 +1,146 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: exploring-cases
|
|
3
|
-
description: Explores the real pages for one case document (cases/<id>.md) before its spec is generated. Checks the case against profiles.yaml and the pages, fixes wording mistakes, records what the pages show, turns unknowns into answerable TODO blocks, and, in an attended run, lets the person do the steps that write data or need a choice while it waits and records a general recipe under
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Exploring a case document
|
|
7
|
-
|
|
8
|
-
Read `AGENTS.md` (hard rules), `cases/_template.md` (the target format), `references/open-questions.md` (how
|
|
9
|
-
questions are written, resolved and reported), `references/assertion-targets.md` (which exact image, text, file or
|
|
10
|
-
place a check looks at, and when to ask instead of picking), `references/readiness.md` (wait for the content before
|
|
11
|
-
you read it) and `references/user-handoff.md` (how a person works in the browser while the agent waits, and how it is
|
|
12
|
-
recorded). All but the last are binding for `/gen-spec` and healing too.
|
|
13
|
-
|
|
14
|
-
##
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
`
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
1
|
+
---
|
|
2
|
+
name: exploring-cases
|
|
3
|
+
description: Explores the real pages for one case document (cases/<id>.md) before its spec is generated. Checks the case against profiles.yaml and the pages, fixes wording mistakes, records what the pages show, turns unknowns into answerable TODO blocks, and, in an attended run, lets the person do the steps that write data or need a choice while it waits and records a general recipe under that step's `- Recipe:` lines. Ends with RESULT ready or needs-input. Use when asked to explore, review, check, clean up or narrow down a case, or for /explore-case. Its open-question rules are shared with generating-specs.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploring a case document
|
|
7
|
+
|
|
8
|
+
Read the workspace `AGENTS.md` (hard rules and project rules), `cases/_template.md` (the target format), `references/open-questions.md` (how
|
|
9
|
+
questions are written, resolved and reported), `references/assertion-targets.md` (which exact image, text, file or
|
|
10
|
+
place a check looks at, and when to ask instead of picking), `references/readiness.md` (wait for the content before
|
|
11
|
+
you read it) and `references/user-handoff.md` (how a person works in the browser while the agent waits, and how it is
|
|
12
|
+
recorded). All but the last are binding for `/gen-spec` and healing too.
|
|
13
|
+
|
|
14
|
+
## Before you start
|
|
15
|
+
Read the workspace `AGENTS.md`: its §2 holds the project rules, and it points at the framework baseline. If
|
|
16
|
+
`.claude/skills/exploring-cases.md` exists in the workspace, read it and apply it after this skill. Then apply
|
|
17
|
+
`AGENTS.md` §2, which wins over both (the baseline, "Precedence"). F1–F6 always win. This is not a stop condition:
|
|
18
|
+
continue with the procedure below.
|
|
19
|
+
|
|
20
|
+
## Where this skill sits
|
|
21
|
+
Three steps, each learns one thing:
|
|
22
|
+
1. `/make-case` (making-cases): the **intent**. Turns raw notes into case documents. No browser.
|
|
23
|
+
2. `/explore-case` (this skill): the **facts and the inputs**. What the pages really show, and what a person has to
|
|
24
|
+
do for the steps that write or need a choice. The only step where a person and the agent work together.
|
|
25
|
+
3. `/gen-spec` (generating-specs): the **implementation**. Replays the case and its step recipes to find locators and
|
|
26
|
+
writes the spec. It does not judge the case again.
|
|
27
|
+
|
|
28
|
+
The purpose of this skill is to leave a case that can be replayed without anybody: no ambiguity, every fact marked
|
|
29
|
+
as observed, every input step described as a general recipe.
|
|
30
|
+
|
|
31
|
+
This skill edits `cases/<id>.md` only. It never edits `profiles.yaml`, `tests/`, the framework folder or anything
|
|
32
|
+
under `.auth/`.
|
|
33
|
+
**The agent never writes data**, in staging or in production: a person does every step that writes or needs a
|
|
34
|
+
choice (see `references/user-handoff.md`).
|
|
35
|
+
|
|
36
|
+
It never stops to ask in the chat. A question that cannot be answered from the page goes into the case file, where
|
|
37
|
+
the user answers it by editing the file, and the report says `RESULT: needs-input`. The one exception is a handoff in
|
|
38
|
+
an attended run, where waiting for the person is the point.
|
|
39
|
+
|
|
40
|
+
## Inputs
|
|
41
|
+
- A case id (path below `cases/` without `.md`, e.g. `documents/landing-page`). It is required: an attended exploration of
|
|
42
|
+
every case at once is not a thing. Without an id, stop and say so.
|
|
43
|
+
- An environment to explore (`staging` or `production`). It is required and there is no default. Without one, stop
|
|
44
|
+
before exploring and report `RESULT: needs-input` ("pass the environment").
|
|
45
|
+
- Optionally `--attended`: a person is at the machine and the browser window is visible, so handoffs may happen.
|
|
46
|
+
Without it nothing waits, and a step that needs a person becomes a `[blocking]` TODO.
|
|
47
|
+
|
|
48
|
+
## Procedure
|
|
49
|
+
1. **Read** the case, `profiles.yaml` and `cases/_template.md`. Apply uncommitted user edits as they are: they are the
|
|
50
|
+
newest wording. Use targeted edits, never rewrite a whole file.
|
|
51
|
+
2. **Static review.** Find:
|
|
52
|
+
- missing values for a profile: compare the entries of `## Values` (keyed by `default` and profile ids) with the
|
|
53
|
+
ids in `profiles.yaml`. A missing entry is a gap, not something to invent: leave it to the observations and
|
|
54
|
+
`/gen-spec`. The case's `## Values` is the only place for expected values; never copy them into `profiles.yaml`;
|
|
55
|
+
- ambiguous steps and checks without an expected result;
|
|
56
|
+
- steps that write data or need a choice: mark them `[writes]` / `[input]`. The front matter must list the
|
|
57
|
+
environments the finished spec may write in (`writes: [staging]`, or `[staging, production]`) with a
|
|
58
|
+
`writes-note`, the data must be named `[asrun] ...` and cleaned up, and a step that sends email is never
|
|
59
|
+
allowed in production and only in other environments when the profile has `email: true`;
|
|
60
|
+
- missing or doubtful `suites`/`envs`, a `writes` environment that `envs` excludes, and visual checks that are
|
|
61
|
+
too broad (screenshot a small element).
|
|
62
|
+
3. **Repair the wording** (edit the file). Fix typos and grammar, make steps one action or one check each, put
|
|
63
|
+
assumptions in `## Notes` (not in `## Expected`), remove duplicates and template placeholders, make paths relative
|
|
64
|
+
to the app. Do not add checks the case did not ask for and do not drop any. Never edit the `## Source` table.
|
|
65
|
+
4. **Explore**. The exploration runs through the `playwright-test` MCP only. If its tools are not registered in this
|
|
66
|
+
session (no `browser_*` / `generator_*` tool) or it reports that it failed to start, do not fall back to ad-hoc
|
|
67
|
+
Playwright scripts: they see a different browser and their network view does not count as
|
|
68
|
+
`browser_network_requests` (A2). Stop, tell the user which MCP server is missing and to run `pnpm asrun doctor`
|
|
69
|
+
(it names the broken `.mcp.json` / `opencode.jsonc` and the fix), make no observations, and end with
|
|
70
|
+
`RESULT: needs-input`.
|
|
71
|
+
Run `pnpm status --env <env>`. If it cannot run (a sandbox error such as
|
|
72
|
+
`listen EPERM` or a Chromium start failure), do not give up: open the profile's page through the MCP, and a page
|
|
73
|
+
that lands on the login page means the profile is not logged in. Explore at least one logged-in profile per
|
|
74
|
+
domain with the `playwright-test` MCP (`generator_setup_page` with `project=<profile-id>` and
|
|
75
|
+
`seedFile=tests/seed.spec.ts`). Profiles that are not logged in are skipped and named in the report; a human runs
|
|
76
|
+
`pnpm signin <profile-id>`.
|
|
77
|
+
- Do the read-only steps yourself: open, look, read, hover, switch tabs, open and close dialogs and menus.
|
|
78
|
+
- **A step that writes or needs a choice is done by the person**, per `references/user-handoff.md`: announce it,
|
|
79
|
+
name the finish condition, start recording, run the wait loop, then write down what happened. In a run that
|
|
80
|
+
is not attended, write a `TODO [blocking]` under the step instead.
|
|
81
|
+
- Close any modal that opens by itself without confirming it (for example the follow-up modal that opens after
|
|
82
|
+
publishing: close it and confirm nothing).
|
|
83
|
+
- When a confirm dialog appears while closing an email composer, choose "Discard" (see `AGENTS.md`).
|
|
84
|
+
- **Plan the order before you start: observe, then let the person delete.** Every read-only observation that
|
|
85
|
+
needs the data (preview, share menu, email composers, downloads, menus, the dialogs) is done while the data
|
|
86
|
+
exists, and the handoff that deletes it comes last. If the data is gone before a step was observed, that step
|
|
87
|
+
is not observed: say so, do not fill it in from memory.
|
|
88
|
+
- **Ready before you read** (`references/readiness.md`): `browser_network_clear` before an action that loads
|
|
89
|
+
something, then the action, then wait for its key requests (`browser_network_requests`) and for the content
|
|
90
|
+
signal, and only then take the snapshot. When the first read is empty or missing something, wait and read
|
|
91
|
+
again before you record or ask anything. Record the signal as `(observed after <signal>, ...)`.
|
|
92
|
+
- **Targets, not guesses** (`references/assertion-targets.md`): for every check on a logo, image, icon, file, PDF,
|
|
93
|
+
screenshot or repeated text, list ALL candidates, outline the one that is meant, take the highlighted screenshot to
|
|
94
|
+
`.explore/targets/<case-id>/`, **open that screenshot and look at it**, and write the card under `## Targets`.
|
|
95
|
+
If there are several candidates, only a position to tell them apart, a generic locator, dynamic data in the area,
|
|
96
|
+
or the page differs from the case: write a `[blocking]` TODO that names the exact image, text, place or UI wanted
|
|
97
|
+
and lists the candidates. In an attended run let the person outline the target with `browser_annotate`.
|
|
98
|
+
- **Downloads and email composers are opened for real** (the composer is closed with "Discard"). A download is
|
|
99
|
+
verified in five parts: starts, finishes, file exists, file valid, and a failure or timeout fails the check
|
|
100
|
+
(see `references/readiness.md`). Record the file name, size and kind as observed. A downloaded PDF is rendered
|
|
101
|
+
(`pnpm pdf:render`) and its page images are opened and looked at: record which pages show the target, where, and
|
|
102
|
+
the text. Which pages the check covers follows the case: every page unless the case says ONLY (strong wording).
|
|
103
|
+
- Compare every step with what is really there.
|
|
104
|
+
|
|
105
|
+
**Keep every browser call short, so nothing hangs** (`generating-specs` follows the same rules):
|
|
106
|
+
- One `browser_run_code_unsafe` call does one thing and should finish within about 60 seconds. Do not close a tab,
|
|
107
|
+
click, wait for a dialog and read it in one call: split it, and read the page (`browser_snapshot`) between the
|
|
108
|
+
steps. Only the handoff wait loop may take minutes.
|
|
109
|
+
- The browser times out an action or a lookup after 15 s (the config sets it). Pass a shorter `{timeout: 5000}` when
|
|
110
|
+
you wait for something that may not appear, and use `.catch()` or `isVisible()` for a one-time look while exploring only.
|
|
111
|
+
`isVisible()` answers for this instant and ignores any `timeout` passed to it, so after an action take a snapshot or
|
|
112
|
+
wait for a signal first. Never copy this pattern into a spec (see `references/readiness.md`, "Optional confirmations").
|
|
113
|
+
- After a click that should open something, take a snapshot and see what opened before you wait for it by role. A
|
|
114
|
+
dialog that did not appear is a finding (what did open instead?), not something to wait for or retry.
|
|
115
|
+
- A step that fails or times out twice is not tried a third time: write what you saw, add a `TODO [blocking]` under
|
|
116
|
+
the step and go on with the next one.
|
|
117
|
+
5. **Write the `## Targets` section and the ready signals.** One card per target, in the format of
|
|
118
|
+
`references/assertion-targets.md`, and a `[T<n>]` reference in each check that uses it. Add `Ready when:` to every
|
|
119
|
+
step or target whose content loads after an action. A check with no card or a card with open doubts is a
|
|
120
|
+
`[blocking]` TODO, never a guess. Check the screenshots you saved: each must show what its card says.
|
|
121
|
+
6. **Write the recipe lines** for every `[input]` or `[writes]` step that was handed off, from the recording, as a
|
|
122
|
+
general recipe under that step: one `- Recipe (phase): ...` line per phase. Format and rules are in
|
|
123
|
+
`references/user-handoff.md`. A step that was not handed off gets no recipe line and a `[blocking]` TODO. A case
|
|
124
|
+
that still has the old `## Inputs` section: move the lines of the steps you touch under those steps as
|
|
125
|
+
`- Recipe:`; `pnpm case:check` names the rest.
|
|
126
|
+
7. **Resolve every open point** in the order of `references/open-questions.md`: answer it from the page, else apply
|
|
127
|
+
the safe default as the sentence above a `[non-blocking]` TODO block, else keep a `[blocking]` block. Write the
|
|
128
|
+
outcome into the case, every block with an empty `Answer:` line.
|
|
129
|
+
8. **Check**: run `pnpm case:check` (it fails on an incomplete target card and warns on a check that has none) and fix every FAIL. If it fails with a sandbox error (`listen EPERM`), run
|
|
130
|
+
`node --import tsx <framework>/scripts/check-cases.ts` instead; it checks the same thing.
|
|
131
|
+
9. **Report** in the user's language: what was changed (wording fixes, observations added, TODOs closed or narrowed),
|
|
132
|
+
what was explored (profile ids), which steps the person did and what was recorded for them, what was skipped and
|
|
133
|
+
why, the target cards written (and which ones are still questions), the steps whose first read was too early,
|
|
134
|
+
the steps that could not be observed (and why), the questions that remain with the sentence that deleting each block would confirm, and findings that look
|
|
135
|
+
like product bugs. End with the `RESULT:` line.
|
|
136
|
+
|
|
137
|
+
## Hard rules
|
|
138
|
+
- Edit `cases/<id>.md` only. Do not touch the `## Source` table.
|
|
139
|
+
- The agent never writes data (see above). Never add `--allow-writes`, never set `ASRUN_ALLOW_WRITES`.
|
|
140
|
+
- Never wait for a person in a run that is not attended.
|
|
141
|
+
- Never invent expected values, profile names, counts or UI texts. Observations are marked as such. Values that a
|
|
142
|
+
check asserts are written into the case's `## Values` under the profile id (or `default` when they hold for every
|
|
143
|
+
profile), and never into `profiles.yaml`.
|
|
144
|
+
- Never pick a target when in doubt, and never record a fact that was read before the content finished loading.
|
|
145
|
+
- Never read or print anything under `.auth/`. Never automate login or 2FA.
|
|
146
|
+
- Prose follows the artifact language of the workspace `AGENTS.md` §2 (English by default); structural tokens stay English.
|