@asrunjs/asrun 0.2.0 → 0.5.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (103) hide show
  1. package/.claude/commands/explore-case.md +18 -15
  2. package/.claude/commands/gen-spec.md +16 -13
  3. package/.claude/commands/heal-spec.md +26 -23
  4. package/.claude/commands/make-case.md +11 -8
  5. package/.claude/skills/exploring-cases/SKILL.md +146 -132
  6. package/.claude/skills/exploring-cases/references/open-questions.md +116 -112
  7. package/.claude/skills/exploring-cases/references/readiness.md +146 -137
  8. package/.claude/skills/exploring-cases/references/user-handoff.md +157 -147
  9. package/.claude/skills/generating-specs/SKILL.md +219 -207
  10. package/.claude/skills/generating-specs/references/conventions.md +148 -148
  11. package/.claude/skills/making-cases/SKILL.md +72 -64
  12. package/.claude/skills/making-cases/references/normalizing.md +39 -38
  13. package/.claude/skills/making-cases/references/writes-and-email.md +37 -36
  14. package/AGENTS.md +169 -154
  15. package/CLAUDE.md +2 -1
  16. package/README.md +221 -410
  17. package/dist/lib/build-artifacts.d.ts +8 -0
  18. package/dist/lib/build-artifacts.js +10 -0
  19. package/dist/lib/case-doc.d.ts +311 -0
  20. package/dist/lib/case-doc.js +1117 -0
  21. package/dist/lib/cases.d.ts +2 -15
  22. package/dist/lib/cases.js +5 -78
  23. package/dist/lib/command.d.ts +24 -0
  24. package/dist/lib/command.js +90 -0
  25. package/dist/lib/fixtures.js +2 -2
  26. package/dist/lib/login-card.d.ts +22 -0
  27. package/dist/lib/login-card.js +183 -0
  28. package/dist/lib/node-cli.d.ts +6 -2
  29. package/dist/lib/node-cli.js +16 -6
  30. package/dist/lib/profiles.d.ts +25 -1
  31. package/dist/lib/profiles.js +54 -1
  32. package/dist/lib/run-filters.js +10 -2
  33. package/dist/lib/session.d.ts +13 -1
  34. package/dist/lib/session.js +73 -7
  35. package/dist/lib/targets.d.ts +5 -13
  36. package/dist/lib/targets.js +4 -29
  37. package/dist/lib/values.js +2 -2
  38. package/dist/scripts/build.js +2 -1
  39. package/dist/scripts/check-cases.js +74 -6
  40. package/dist/scripts/doctor.js +115 -4
  41. package/dist/scripts/explore.js +22 -1
  42. package/dist/scripts/go.mjs +188 -188
  43. package/dist/scripts/init.d.ts +7 -1
  44. package/dist/scripts/init.js +62 -14
  45. package/dist/scripts/lint-specs.js +4 -1
  46. package/dist/scripts/login.js +30 -2
  47. package/dist/ui/server/actions.d.ts +4 -0
  48. package/dist/ui/server/actions.js +25 -12
  49. package/dist/ui/server/ai-models.d.ts +3 -0
  50. package/dist/ui/server/ai-models.js +347 -0
  51. package/dist/ui/server/ai-stream.d.ts +34 -9
  52. package/dist/ui/server/ai-stream.js +283 -77
  53. package/dist/ui/server/app.js +74 -12
  54. package/dist/ui/server/jobs.d.ts +7 -1
  55. package/dist/ui/server/jobs.js +69 -15
  56. package/dist/ui/server/profiles.d.ts +20 -1
  57. package/dist/ui/server/profiles.js +214 -38
  58. package/dist/ui/shared/ai-commands.d.ts +16 -6
  59. package/dist/ui/shared/ai-commands.js +35 -12
  60. package/dist/ui/shared/types.d.ts +114 -0
  61. package/docs/agents/case-format.md +193 -0
  62. package/docs/agents/framework-dev.md +167 -0
  63. package/docs/ai-workflow.md +235 -0
  64. package/docs/cases.md +189 -0
  65. package/docs/commands.md +175 -0
  66. package/docs/install.md +174 -0
  67. package/docs/profiles.md +139 -0
  68. package/docs/theme-guide.html +922 -692
  69. package/docs/troubleshooting.md +101 -0
  70. package/docs/web-ui.md +139 -0
  71. package/package.json +20 -1
  72. package/templates/.claude/commands/explore-case.md +22 -21
  73. package/templates/.claude/commands/gen-spec.md +21 -20
  74. package/templates/.claude/commands/heal-spec.md +31 -30
  75. package/templates/.claude/commands/make-case.md +13 -12
  76. package/templates/.mcp.json +8 -8
  77. package/templates/AGENTS.md +66 -45
  78. package/templates/cases/_template.md +57 -63
  79. package/templates/opencode.jsonc +16 -11
  80. package/templates/profiles.yaml +42 -29
  81. package/ui/dist/assets/{dist-D_I45xgJ.js → dist-BudNAjOv.js} +1 -1
  82. package/ui/dist/assets/{dist-CBV7_vDN.js → dist-C-eCSFja.js} +1 -1
  83. package/ui/dist/assets/{dist-gF2bFn-5.js → dist-CO4heSzh.js} +1 -1
  84. package/ui/dist/assets/{dist-DX8knLVr.js → dist-CPLhRzom.js} +1 -1
  85. package/ui/dist/assets/{dist-BqAROxzI.js → dist-DEPJZ3Me.js} +1 -1
  86. package/ui/dist/assets/{dist-BwYRbe5s.js → dist-DZ7RiqmA.js} +1 -1
  87. package/ui/dist/assets/{dist-BHaIOZdW.js → dist-LixzGniF.js} +1 -1
  88. package/ui/dist/assets/{dist-DHCY9KpV.js → dist-O_RVi4tP.js} +1 -1
  89. package/ui/dist/assets/{dist-BiE4suo9.js → dist-jPS6HCJ5.js} +1 -1
  90. package/ui/dist/assets/{dist-BSJMM_jp.js → dist-vmf3fnkM.js} +1 -1
  91. package/ui/dist/assets/index-BIGt8Hfz.js +338 -0
  92. package/ui/dist/assets/index-CTmnF0Wp.css +2 -0
  93. package/ui/dist/help/ai-tools.png +0 -0
  94. package/ui/dist/help/case-explore.png +0 -0
  95. package/ui/dist/help/case-generate.png +0 -0
  96. package/ui/dist/help/cases-memo.png +0 -0
  97. package/ui/dist/help/profile-form.png +0 -0
  98. package/ui/dist/help/profiles-login.png +0 -0
  99. package/ui/dist/help/reports.png +0 -0
  100. package/ui/dist/help/run-tests.png +0 -0
  101. package/ui/dist/index.html +22 -16
  102. package/ui/dist/assets/index-DmasQ6Rq.css +0 -2
  103. package/ui/dist/assets/index-I7uU93kk.js +0 -294
@@ -0,0 +1,101 @@
1
+ # Troubleshooting
2
+
3
+ ## Login
4
+
5
+ **`Profile "x" is not logged in`** — run `pnpm signin <profile-id>` and log in. To leave that profile out instead:
6
+ `pnpm asrun --env <env> --skip-profile <profile-id>`, or `--logged-in` to keep only the working ones. Login itself is
7
+ never a test; a missing or expired session fails right away, without retries.
8
+
9
+ **A saved session expires in the middle of a run** — the failed test is marked "session expired". Log in again and
10
+ rerun.
11
+
12
+ **`pnpm signin` opened a browser but nothing was saved** — the session is saved when you press **Done, save the
13
+ session** on the card in the bottom-left corner of that window (or press Enter in the terminal). Nothing about 2FA is
14
+ automated.
15
+
16
+ ## Running
17
+
18
+ **`Choose the environment to test`** — there is no default. Add `--env production` or `--env staging`; the Run tests
19
+ page needs one radio button.
20
+
21
+ **A case is skipped, not failed** — a case that writes data runs only in the environments under `writes`; a case with
22
+ `envs` runs only there. The run prints "skipped" with the reason.
23
+
24
+ **`This case writes data and has no permission to run in PRODUCTION`** — a person confirms production writes on every
25
+ run (type the environment name, or use the dialog in the web UI). `--read-only` leaves the write cases out. Unattended
26
+ runs use `--allow-writes=<env>`; AI agents must not.
27
+
28
+ **`--update-baselines` was refused** — the named cases must be among the cases the filters run, and cannot be combined
29
+ with Playwright's own `--update-snapshots`.
30
+
31
+ **The report is empty or missing** — run `pnpm asrun --env <env>` first. `pnpm report` opens `reports/latest`.
32
+
33
+ **Screenshot mismatch** — open the report: the diff view shows expected, actual and diff. If the change is intended,
34
+ update the baseline on purpose ([commands.md](./commands.md#baselines)); if not, fix the spec or the product. Never let
35
+ an agent overwrite a baseline to hide a difference.
36
+
37
+ **Videos are separate per tab, or there is no merged video** — ffmpeg is missing. `pnpm go` tries to install it
38
+ (Homebrew, winget); on Linux run `sudo apt install ffmpeg`. Without it, the optional `ffmpeg-static` package is used;
39
+ if neither works, the report lists the recordings separately.
40
+
41
+ **Downloads did not arrive** — files downloaded in a **Visit** window (the UI) or with `pnpm explore` are saved to
42
+ `~/Downloads` and never overwrite an existing file; the saved path is printed in the job log or terminal.
43
+
44
+ **The UI port is busy** — set `ASRUN_UI_PORT` (and `ASRUN_UI_WEB_PORT` for `pnpm ui:dev`).
45
+
46
+ ## Cases and specs
47
+
48
+ **`Missing values: cases/<id>.md has no "## Values" for profile ...`** — add the profile's values (or a `default`
49
+ entry) to the case's `## Values` block. See [cases.md](./cases.md#values).
50
+
51
+ **`Incomplete values: ... still has empty or TODO values`** — fill in the named keys, or answer the `TODO` in the case
52
+ and regenerate the spec.
53
+
54
+ **`pnpm case:check` fails on a target card** — the card is missing a required field (`What`, `Where`,
55
+ `Identified by`), refers to a reference image outside `.explore/targets/<case-id>/`, or a check refers to a `[T<n>]`
56
+ that has no card. A blocking question (`🚫 TODO [blocking]`) is the normal way to wait for a person's answer.
57
+
58
+ **`pnpm spec:lint` fails** — read the rule name in the message: `no-sleep` (`waitForTimeout`), `no-networkidle`,
59
+ download handling, a screenshot or image check without `// target: T<n>`, a PDF check on a magic number alone. The
60
+ fixes are in the framework's `generating-specs` conventions; do not silence the rule.
61
+
62
+ **A case step marked `[input]` or `[writes]` stops `/gen-spec` with `needs-input`** — that step has no
63
+ `- Recipe:` line. Run `/explore-case <case-id> <env> --attended` and do the step in the browser once, so the recipe
64
+ is recorded.
65
+
66
+ **`pnpm case:check` warns that a reference image is not on this machine** — `.explore/` is git-ignored: a fresh
67
+ checkout has the card but not the screenshot. Run the exploration again on this machine (or look at the card and the
68
+ report's baseline).
69
+
70
+ ## AI steps
71
+
72
+ **An AI command says the `playwright-test` MCP server is missing** — run `pnpm asrun doctor`. OpenCode reads only
73
+ `opencode.jsonc`; Claude Code reads only `.mcp.json`. `pnpm asrun init` writes both. Check with `opencode mcp list` /
74
+ `claude mcp list`; `failed: ... path specified` means the `playwright` path in that file does not resolve (repair it
75
+ with `pnpm asrun init --force <file>`).
76
+
77
+ **An AI job runs but uses a slow or limited model** — the top bar's **Model** picker sets the model for UI runs;
78
+ without one, a one-shot OpenCode run uses its built-in free model, which has a daily usage limit. Set `"model"` in
79
+ `opencode.jsonc` to stop depending on the browser's choice. For Claude Code, the picker lists what its settings,
80
+ environment, aliases or gateway expose.
81
+
82
+ **Thinking does nothing** — it is an OpenCode flag. Claude Code has no such flag; its model and settings decide.
83
+
84
+ **A project rule does nothing** — `AGENTS.md` §2 needs the rule id and an `Effect` of `narrows` or `replaces`.
85
+ `F1`..`F6` cannot be changed. `pnpm asrun doctor` names a wrong id, a missing `Effect` or a non-negotiable rule.
86
+
87
+ **A skill override does nothing** — it must be `.claude/skills/<skill>.md` (the skill name, for example
88
+ `generating-specs`, not the command name), and it is not discovered as a standalone skill: the command and the skill
89
+ read it. `doctor` lists the overrides it sees.
90
+
91
+ **`pnpm asrun init --force AGENTS.md` is refused** — `AGENTS.md`, `CLAUDE.md` and `.claude/skills/**` belong to the
92
+ project. Add `--reset-rules` to replace them with the template, and expect to lose the project's rules.
93
+
94
+ **`ERR_PNPM_IGNORED_BUILDS` while installing** — pnpm refused the build scripts of `esbuild` / `ffmpeg-static`. Run
95
+ `pnpm dlx @asrunjs/asrun init` first (it writes the `allowBuilds` entry), or add it to `pnpm-workspace.yaml` by hand;
96
+ see [install.md](./install.md#manual-install).
97
+
98
+ **The AI did something you did not expect** — exploring and generating only read pages and edit the case file; healing
99
+ edits the spec and runs it under the same write gate as any run (staging writes are allowed, any other environment
100
+ needs a person's confirmation). None of them writes product data on its own, and none updates a baseline. If a step
101
+ needs a person to write, run it with `--attended`; that is where a person acts, in the browser.
package/docs/web-ui.md ADDED
@@ -0,0 +1,139 @@
1
+ # The web UI
2
+
3
+ The UI is a local page for the daily work: profiles and logins, case editing, AI runs with live logs, and reports.
4
+ It runs on your machine and only listens on `127.0.0.1`.
5
+
6
+ ```bash
7
+ pnpm go # install + build + start + open the browser (the usual command)
8
+ pnpm ui # build the web app and serve http://127.0.0.1:4173
9
+ pnpm ui:dev # development: Vite on :5173 with hot reload, API in watch mode
10
+ ```
11
+
12
+ Set `ASRUN_UI_PORT` to change the port of the served app (`ASRUN_UI_WEB_PORT` for the Vite dev server).
13
+
14
+ Everything the UI does is a fixed action mapped to a command of this repository; there is no free shell. An AI job
15
+ runs the AI tool you selected (Claude Code or OpenCode) with the same commands as the terminal.
16
+
17
+ ## The top bar
18
+
19
+ - **Sections:** Profiles, Cases, Run tests, Reports, AI guide, Help, and the Jobs bar at the bottom.
20
+ - **AI tool switch:** Claude Code (the default) or OpenCode. The choice is remembered in this browser and applies to
21
+ every AI command.
22
+ - **Model picker:** the models that the selected CLI knows on this machine. OpenCode lists its providers
23
+ (`opencode models`); Claude Code lists what its own settings, environment, aliases and gateway expose, and does not
24
+ query cloud providers. The choice is remembered per CLI and shown in the command line of every AI run. On a
25
+ project that sets no model, a one-shot OpenCode run uses OpenCode's built-in free model, which has a daily limit.
26
+ - **Thinking:** passes `--thinking` to OpenCode. Off by default (it costs tokens and slows a run). Claude Code has no
27
+ such flag, so the icon is disabled there.
28
+ - **Bell:** browser notifications for "your turn" (a login to finish, an attended step waiting, answers an AI run
29
+ needs) and for a run that ends while the tab is in the background. It stays until you click or close it; clicking
30
+ opens that job. The tab title shows the section and, in front, how many things wait for you: `(1) AsRun | Cases`.
31
+
32
+ The UI's look is the **Inkstone** theme; the palette, buttons, badges and icons are documented in
33
+ `theme-guide.html` in the framework package (also linked from the top bar).
34
+
35
+ ## Profiles
36
+
37
+ One row per profile, with the same login check as `pnpm status`.
38
+
39
+ - **Log in** starts `pnpm signin <profile-id>`: log in in the browser window that opens, then press **Done, save the
40
+ session** on the card in that window (the job's "I've logged in, save the session" button does the same). A
41
+ logged-in profile shows `Login YES`.
42
+ - The **...** menu: **Verify** (the same read-only session check) and **Re-login** for a logged-in profile, **Logout**
43
+ for a saved session (deletes the session file, after a confirmation), **Duplicate** (a second entry with the same
44
+ settings under a new id; the saved login is not copied) and **Delete** (removes the entry and its saved login, after
45
+ a confirmation; baselines and old reports keep the id in their names and stay on disk).
46
+ - **Visit** opens a logged-in browser window at the profile's `startPath` (or the site root). While it is open the
47
+ button turns into **Close browser**. Downloads in that window are saved to `~/Downloads` without overwriting an
48
+ existing file, and the saved path is printed in the job log.
49
+ - **Add a new profile** has a form (environment, URL, account label, tags, start path) that appends an entry to
50
+ `profiles.yaml`, plus the next steps: log in, update the cases, a copyable AI prompt.
51
+
52
+ ## Cases
53
+
54
+ The case list is grouped by folder and collapses. The editor opens in **Single** with the **Panel** full width; the
55
+ toolbar switches to **Split** (source left, one pane right; drag the divider to resize, double-click to reset) or, in
56
+ Single, to **Editor** or **Doc**.
57
+
58
+ - **Source and preview** scroll together, the preview block at the caret is shaded, and a click in the preview puts
59
+ the caret on that line. `Ctrl/Cmd+S` saves.
60
+ - **What needs a person is highlighted** in both: `TODO` in any spelling (red for `🚫 TODO [blocking]`, gold
61
+ otherwise), `🔍 to be confirmed by QA` / `👁️ not observed` / `TBC` / `TBD` in blue, and your `✍️ Answer:` lines in
62
+ green. Hover for a tooltip; the small X after a marker removes it (or the whole question block for a `TODO`) in one
63
+ step, with Undo; the file is saved when you save it.
64
+ - **New case** takes an optional folder and runs `pnpm case:new`.
65
+ - **The Panel** shows the case as a form, with the open questions first (accept one to delete the question, or write an
66
+ answer and the AI folds it in). Then the front matter (title, login, suites, writes and its note, environments), the
67
+ steps (sentence, `[writes]` / `[input]` / `[email]` tags, `- Ready when:` lines, `- Recipe:` lines), the
68
+ `## Expected` and `## Visual checks` bullets (drag to reorder), the `## Values` table (one column per profile; an
69
+ empty cell inherits `default`), and the `## Targets` cards read-only, with what each is missing. `[T1]` chips open
70
+ the card: what it is, where, how it is identified, what it must not be, and its reference screenshots. Every section
71
+ collapses and has a **?** with a short help card. Panel edits land in the same draft as the source editor, so one
72
+ save covers both, and **Discard** throws them away together.
73
+ - **Spec view** is editable too (`Ctrl/Cmd+S`; refused when the file changed on disk meanwhile). **Open in editor**
74
+ opens it in the default application for `.spec.ts`, **Copy path** copies its absolute path.
75
+ - Click the file name (or the pencil icon) to **rename** or move the case: the spec and the baselines move with it and
76
+ the spec is rewritten. **Delete** (type the id, or fill it in with the feather icon) removes the case, its spec and
77
+ its baselines; both dialogs list exactly what they touch, and are refused while an AI or test run is in progress.
78
+ - **Toolbar:** command (explore / generate / heal, explore by default), **Environment** (required, no default),
79
+ **Attended** (only for explore, ticked by default), **Run AI**, copy the shell command, open the AI guide.
80
+ - **From memo** opens a Markdown editor with a preview for raw notes. **Make cases** saves them to `.tmp/memos/` and
81
+ runs `/make-case` one-shot (defaults for gaps, `TODO` for unknown values, no page exploration, existing cases never
82
+ overwritten); **Copy /make-case** copies the memo for the interactive dialogue. When the run ends, **What to do
83
+ next** shows the new cases and **Review** lists them, each with **Explore**: a case written from a memo has not been
84
+ checked against the real pages yet.
85
+
86
+ ## Run tests
87
+
88
+ The form builds `pnpm asrun`: environment (required; no run starts without one), profiles, suite, read-only, case,
89
+ step text.
90
+
91
+ - **Case** is a searchable picker of the cases that have a spec.
92
+ - **Cases to run** shows what the filters select, with an **Update baselines** box per case (`--update-baselines`,
93
+ unticked by default; it asks before overwriting and shows the new images next to their target cards when the run
94
+ ends).
95
+ - It asks for confirmation when write cases would run in production.
96
+
97
+ Running is a job: its output streams into the Jobs bar.
98
+
99
+ ## Reports
100
+
101
+ One entry per run, with a PRODUCTION or STAGING badge (runs from before environments existed show PRODUCTION).
102
+
103
+ - **Results** lists every case × profile with the failure reason on one line, failed first.
104
+ - Selecting one shows a red **Why it failed** block (failed step and error) above the screenshot and the video, with a
105
+ folded **Technical details** (call log, stack trace).
106
+ - **Full Playwright report** embeds the standard HTML report, where the videos and traces work.
107
+ - The latest run opens first.
108
+
109
+ ## Jobs
110
+
111
+ The bottom bar shows every command started from the UI with live stdout/stderr and the exit code; a job can be
112
+ cancelled there. Opening it shrinks the page instead of covering it.
113
+
114
+ - An AI job's log is structured: a timestamp, one line per tool call (name, what it touched, how long it took), the
115
+ first lines of what it returned with the colors the command printed, the agent's thinking dimmed while **Thinking**
116
+ is ticked, and its messages rendered as Markdown. Long output is a preview with a count. The same text is kept in
117
+ `.tmp/ui-jobs/`.
118
+ - When an AI job ends, a thin **What to do next** bar appears under the log (or **Details** opens it in a sheet):
119
+ open questions with an answer box and **Use the suggested defaults**, the confirmations the exploration marked, and
120
+ the next action (**Generate spec**, **Open Run tests**). A run that ends with `needs-input` gets an amber *Needs
121
+ your input* badge; sending answers continues the same conversation, once per turn and only while the UI server has
122
+ not restarted. After a restart, answer in the case file and run the command again.
123
+
124
+ ## Help and AI guide
125
+
126
+ - **Help** (the **?** icon) is "How to start": nine short steps with screenshots, from installing AsRun to reading a
127
+ report. The screenshots are in `ui/web/public/help/` in the framework package; retake them when a page changes.
128
+ - **AI guide** explains each AI step in the UI and has the same `/make-case`, `/explore-case`, `/gen-spec`,
129
+ `/heal-spec` commands for the terminal, ready to copy.
130
+
131
+ ## Where the UI keeps its state
132
+
133
+ | Path | What |
134
+ |---|---|
135
+ | `.tmp/ui-jobs/` | the log of every job started from the UI (git-ignored) |
136
+ | `.tmp/hints/` | the text you type in the Heal dialog, passed to the agent as `--hints` |
137
+ | `.explore/` | reference screenshots of the target cards (git-ignored) |
138
+ | `reports/` | one folder per run, `reports/latest` points at the newest |
139
+ | `.auth/` | the saved logins (git-ignored, never shared) |
package/package.json CHANGED
@@ -1,9 +1,17 @@
1
1
  {
2
2
  "name": "@asrunjs/asrun",
3
- "version": "0.2.0",
3
+ "version": "0.5.3",
4
4
  "description": "Multi-profile end-to-end test suite: plain-language cases, AI-written Playwright specs, evidence for every run.",
5
5
  "license": "MIT",
6
6
  "author": "Zoro Chen",
7
+ "repository": {
8
+ "type": "git",
9
+ "url": "git+https://github.com/wizcas/asrun.git"
10
+ },
11
+ "homepage": "https://github.com/wizcas/asrun#readme",
12
+ "bugs": {
13
+ "url": "https://github.com/wizcas/asrun/issues"
14
+ },
7
15
  "type": "module",
8
16
  "engines": {
9
17
  "node": ">=22"
@@ -55,7 +63,11 @@
55
63
  "@codemirror/lang-yaml": "^6.1.3",
56
64
  "@codemirror/language": "^6.13.1",
57
65
  "@codemirror/language-data": "^6.5.2",
66
+ "@codemirror/state": "^6.7.6",
58
67
  "@codemirror/view": "^6.43.15",
68
+ "@dnd-kit/core": "^6.3.1",
69
+ "@dnd-kit/sortable": "^10.0.0",
70
+ "@dnd-kit/utilities": "^3.2.2",
59
71
  "@hono/node-server": "^2.1.5",
60
72
  "@lezer/highlight": "^1.2.5",
61
73
  "@napi-rs/canvas": "^1.0.13",
@@ -92,6 +104,13 @@
92
104
  "spec:lint": "tsx scripts/lint-specs.ts",
93
105
  "baselines:review": "tsx scripts/review-baselines.ts",
94
106
  "pdf:render": "tsx scripts/render-pdf.ts",
107
+ "demo:up": "node scripts/demo.mjs up",
108
+ "demo:build": "node scripts/demo.mjs build",
109
+ "demo:seed": "node scripts/demo.mjs seed",
110
+ "demo:reset": "node scripts/demo.mjs reset",
111
+ "demo:down": "node scripts/demo.mjs down",
112
+ "demo:logs": "node scripts/demo.mjs logs",
113
+ "demo:status": "node scripts/demo.mjs status",
95
114
  "typecheck": "tsc --noEmit && tsc --noEmit -p ui/tsconfig.json",
96
115
  "ui": "tsx scripts/ui.ts",
97
116
  "ui:build": "tsx scripts/ui-build.ts",
@@ -1,21 +1,22 @@
1
- ---
2
- description: Explore the real pages for one case (optionally attended: you do the steps that write data), before generating the spec
3
- ---
4
-
5
- Load the framework skill `{{framework}}/.claude/skills/exploring-cases/SKILL.md` and explore: $ARGUMENTS
6
-
7
- If `.claude/skills/exploring-cases.md` exists, apply it after the framework skill; on conflict it wins, except rules
8
- tagged `[non-negotiable]`. Project facts are in `AGENTS.md` §3.
9
-
10
- Arguments: `<case-id> <env> [--attended]`. The case id is the path below `cases/` without `.md` (it may include a
11
- group folder, e.g. `documents/landing-page`) and is required. `<env>` is `staging` or `production`; it is required and there
12
- is no default (without one, stop and say so). With `--attended` I am at the machine and the browser window is
13
- visible: when a step writes data or needs a choice, hand it to me as the skill describes, wait for me, and record
14
- what I did as a general recipe under "## Inputs". Without `--attended`, never wait for me.
15
-
16
- Follow `{{framework}}/.claude/skills/exploring-cases/references/assertion-targets.md` (list the candidates, never
17
- pick when in doubt, ask for the exact image/text/place/UI) and `readiness.md` (wait for the key requests and the
18
- content before you read anything) of the skill.
19
-
20
- This command edits `cases/<case-id>.md` only (plus screenshots under `.explore/targets/`). Do not ask me questions in
21
- the chat: write open questions into the case file as the skill describes, and end with the `RESULT:` line.
1
+ ---
2
+ description: Explore the real pages for one case (optionally attended: you do the steps that write data), before generating the spec
3
+ ---
4
+
5
+ Load the framework skill `{{framework}}/.claude/skills/exploring-cases/SKILL.md` and explore: $ARGUMENTS
6
+
7
+ Rules combine in this order, and a later one wins where they conflict: the framework baseline
8
+ `{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/exploring-cases.md` if it exists, then the workspace
9
+ `AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
10
+
11
+ Arguments: `<case-id> <env> [--attended]`. The case id is the path below `cases/` without `.md` (it may include a
12
+ group folder, e.g. `documents/landing-page`) and is required. `<env>` is `staging` or `production`; it is required and there
13
+ is no default (without one, stop and say so). With `--attended` I am at the machine and the browser window is
14
+ visible: when a step writes data or needs a choice, hand it to me as the skill describes, wait for me, and record
15
+ what I did as a general recipe under the step it belongs to (`- Recipe:` lines). Without `--attended`, never wait for me.
16
+
17
+ Follow `{{framework}}/.claude/skills/exploring-cases/references/assertion-targets.md` (list the candidates, never
18
+ pick when in doubt, ask for the exact image/text/place/UI) and `readiness.md` (wait for the key requests and the
19
+ content before you read anything) of the skill.
20
+
21
+ This command edits `cases/<case-id>.md` only (plus screenshots under `.explore/targets/`). Do not ask me questions in
22
+ the chat: write open questions into the case file as the skill describes, and end with the `RESULT:` line.
@@ -1,20 +1,21 @@
1
- ---
2
- description: Generate or update tests/<id>.spec.ts from cases/<id>.md (the id may be <group>/<name>)
3
- ---
4
-
5
- Load the framework skill `{{framework}}/.claude/skills/generating-specs/SKILL.md` and generate the spec for:
6
- $ARGUMENTS
7
-
8
- If `.claude/skills/generating-specs.md` exists, apply it after the framework skill; on conflict it wins, except rules
9
- tagged `[non-negotiable]`. Project facts are in `AGENTS.md` §3.
10
-
11
- Arguments: `<case-id> [<env>] [--force]`. The case file is `cases/<case-id>.md` (the id may include one group folder,
12
- e.g. `documents/landing-page`). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If
13
- the spec already exists, it is left alone unless `--force` is given; with `--force` it is regenerated and overwritten.
14
-
15
- Follow the skill's procedure exactly, including its `assertion-targets.md` and `readiness.md`; run
16
- `pnpm spec:lint <case-id>`. Only generate the spec: do not run `pnpm asrun` (running specs is a separate
17
- process), and do not create baselines.
18
-
19
- Do not ask me questions: resolve ambiguity as the skill describes (explore the page, use observed values marked for
20
- QA, write the remaining questions into the case file), and end with the `RESULT:` line.
1
+ ---
2
+ description: Generate or update tests/<id>.spec.ts from cases/<id>.md (the id may be <group>/<name>)
3
+ ---
4
+
5
+ Load the framework skill `{{framework}}/.claude/skills/generating-specs/SKILL.md` and generate the spec for:
6
+ $ARGUMENTS
7
+
8
+ Rules combine in this order, and a later one wins where they conflict: the framework baseline
9
+ `{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace
10
+ `AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
11
+
12
+ Arguments: `<case-id> [<env>] [--force]`. The case file is `cases/<case-id>.md` (the id may include one group folder,
13
+ e.g. `documents/landing-page`). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If
14
+ the spec already exists, it is left alone unless `--force` is given; with `--force` it is regenerated and overwritten.
15
+
16
+ Follow the skill's procedure exactly, including its `assertion-targets.md` and `readiness.md`; run
17
+ `pnpm spec:lint <case-id>`. Only generate the spec: do not run `pnpm asrun` (running specs is a separate
18
+ process), and do not create baselines.
19
+
20
+ Do not ask me questions: resolve ambiguity as the skill describes (explore the page, use observed values marked for
21
+ QA, write the remaining questions into the case file), and end with the `RESULT:` line.
@@ -1,30 +1,31 @@
1
- ---
2
- description: Diagnose and fix a failing spec from its last report (without hiding real product regressions)
3
- ---
4
-
5
- Load the framework skill `{{framework}}/.claude/skills/generating-specs/SKILL.md`, section "Healing a failing
6
- spec", and follow it exactly.
7
-
8
- If `.claude/skills/generating-specs.md` exists, apply it after the framework skill; on conflict it wins, except rules
9
- tagged `[non-negotiable]`. Project facts are in `AGENTS.md` §3.
10
-
11
- Arguments: $ARGUMENTS
12
- Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
13
- - `<case-id>` and `<env>` (`staging` or `production`) are required. There is no default environment. Without one,
14
- stop and say so.
15
- - `--profiles`: the profiles to heal. Without it, every profile that is logged in in `<env>`.
16
- - `--report`: the report (folder name under `reports/`) to read first. Without it, the newest report of this case.
17
- - `--hints <file>`: a file with what the user knows (what to heal, evidence, clues). Read it first. Any free text
18
- after the arguments is the same kind of clue. Both are optional.
19
-
20
- Before you decide "product or test", audit the assertion itself (target card, baseline content, readiness): see
21
- step 4 of the skill's healing section, `assertion-targets.md` and `readiness.md`. Run `pnpm spec:lint <case-id>`.
22
- Read the existing report first (`pnpm heal:evidence --case <case-id> --env <env>`), and run the case again only when
23
- the skill says it is needed. Always run with `--logged-in`. Tell me whether it is a test problem or a product
24
- finding.
25
-
26
- Run `pnpm asrun ...`, `pnpm heal:evidence ...`, `pnpm spec:lint ...`, `pnpm baselines:review ...` and
27
- `pnpm status ...` as plain commands, optionally piped to `tail` or `head`, and do not chain other commands with them
28
- (`;` or `&&`). Only these exact forms run outside Claude Code's sandbox, which tsx and the browser need. If a
29
- command still fails with `listen EPERM` or a sandbox error, stop and report it. Do not ask to disable the sandbox,
30
- and never add `--allow-writes` or set `ASRUN_ALLOW_WRITES`.
1
+ ---
2
+ description: Diagnose and fix a failing spec from its last report (without hiding real product regressions)
3
+ ---
4
+
5
+ Load the framework skill `{{framework}}/.claude/skills/generating-specs/SKILL.md`, section "Healing a failing
6
+ spec", and follow it exactly.
7
+
8
+ Rules combine in this order, and a later one wins where they conflict: the framework baseline
9
+ `{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace
10
+ `AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
11
+
12
+ Arguments: $ARGUMENTS
13
+ Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
14
+ - `<case-id>` and `<env>` (`staging` or `production`) are required. There is no default environment. Without one,
15
+ stop and say so.
16
+ - `--profiles`: the profiles to heal. Without it, every profile that is logged in in `<env>`.
17
+ - `--report`: the report (folder name under `reports/`) to read first. Without it, the newest report of this case.
18
+ - `--hints <file>`: a file with what the user knows (what to heal, evidence, clues). Read it first. Any free text
19
+ after the arguments is the same kind of clue. Both are optional.
20
+
21
+ Before you decide "product or test", audit the assertion itself (target card, baseline content, readiness): see
22
+ step 4 of the skill's healing section, `assertion-targets.md` and `readiness.md`. Run `pnpm spec:lint <case-id>`.
23
+ Read the existing report first (`pnpm heal:evidence --case <case-id> --env <env>`), and run the case again only when
24
+ the skill says it is needed. Always run with `--logged-in`. Tell me whether it is a test problem or a product
25
+ finding.
26
+
27
+ Run `pnpm asrun ...`, `pnpm heal:evidence ...`, `pnpm spec:lint ...`, `pnpm baselines:review ...` and
28
+ `pnpm status ...` as plain commands, optionally piped to `tail` or `head`, and do not chain other commands with them
29
+ (`;` or `&&`). Only these exact forms run outside Claude Code's sandbox, which tsx and the browser need. If a
30
+ command still fails with `listen EPERM` or a sandbox error, stop and report it. Do not ask to disable the sandbox,
31
+ and never add `--allow-writes` or set `ASRUN_ALLOW_WRITES`.
@@ -1,12 +1,13 @@
1
- ---
2
- description: Turn raw test notes (pasted text or a file) into case documents in cases/
3
- ---
4
-
5
- Load the framework skill `{{framework}}/.claude/skills/making-cases/SKILL.md` and turn this input into case
6
- documents: $ARGUMENTS
7
-
8
- If `.claude/skills/making-cases.md` exists, apply it after the framework skill; on conflict it wins, except rules
9
- tagged `[non-negotiable]`. Project facts are in `AGENTS.md` §3.
10
-
11
- The input is either text or a path to a file with the notes. Follow the skill's procedure exactly. Ask me the open
12
- questions before writing any case file. Do not generate specs; that is `/gen-spec`.
1
+ ---
2
+ description: Turn raw test notes (pasted text or a file) into case documents in cases/
3
+ ---
4
+
5
+ Load the framework skill `{{framework}}/.claude/skills/making-cases/SKILL.md` and turn this input into case
6
+ documents: $ARGUMENTS
7
+
8
+ Rules combine in this order, and a later one wins where they conflict: the framework baseline
9
+ `{{framework}}/AGENTS.md`, the framework skill, `.claude/skills/making-cases.md` if it exists, then the workspace
10
+ `AGENTS.md` §2 (Project rules). F1–F6 always win. Project facts are in `AGENTS.md` §3.
11
+
12
+ The input is either text or a path to a file with the notes. Follow the skill's procedure exactly. Ask me the open
13
+ questions before writing any case file. Do not generate specs; that is `/gen-spec`.
@@ -1,8 +1,8 @@
1
- {
2
- "mcpServers": {
3
- "playwright-test": {
4
- "command": "./node_modules/.bin/playwright",
5
- "args": ["run-test-mcp-server"]
6
- }
7
- }
8
- }
1
+ {
2
+ "mcpServers": {
3
+ "playwright-test": {
4
+ "command": "{{playwright}}",
5
+ "args": ["run-test-mcp-server"]
6
+ }
7
+ }
8
+ }
@@ -1,45 +1,66 @@
1
- # <project name> — asrun workspace
2
-
3
- This project uses the asrun framework, which lives in `{{framework}}` as an installed dependency. The framework is
4
- read-only here: never edit files inside it, and never commit test data into it.
5
-
6
- Read this file, then the framework baseline, before any AI step.
7
-
8
- ## 1. Framework baseline (binding)
9
-
10
- Read `{{framework}}/AGENTS.md` in full. It is part of this project's rules. Its hard rules carry ids `F1`..`F10`,
11
- its browser rules `B1`..`B4`, its assertion rules `A1`..`A6`. Rules tagged `[non-negotiable]` cannot be narrowed.
12
-
13
- The framework skills are in `{{framework}}/.claude/skills/`. The commands in `.claude/commands/` load them. A
14
- project that needs different wording puts an override at `.claude/skills/<skill>.md` (for example
15
- `.claude/skills/exploring-cases.md`); the command applies the framework skill first and the override afterwards, and
16
- on conflict the override wins except for `[non-negotiable]` rules. `pnpm asrun doctor` lists every override.
17
-
18
- ## 2. Project rules (only stricter)
19
-
20
- Add rules that narrow the baseline for this project, and name the baseline rule each one narrows. The rules `F1`..`F6`
21
- cannot be narrowed, so they never appear here. Example:
22
-
23
- | Rule | Project rule |
24
- |---|---|
25
- | F9 | Staging writes are limited to accounts named `qa-*` and names starting with `[asrun]`. |
26
- | F7 | This project runs production profiles only from a scheduled job; people use `--env staging`. |
27
-
28
- `pnpm asrun doctor` fails when this table names a `[non-negotiable]` rule.
29
-
30
- ## 3. Project facts
31
-
32
- - Profiles and environments: `profiles.yaml`.
33
- - Areas: one folder per area below `cases/` (for example `cases/<area>/<name>.md`).
34
- - Expected values: they live in each case's `## Values` block, never here and never in `profiles.yaml`.
35
- - Who owns a finding, and where findings are reported: <fill in>.
36
-
37
- ## Layout of this workspace
38
-
39
- - `cases/**/*.md` human-written test cases (committed)
40
- - `tests/**/*.spec.ts` generated Playwright specs and their `*-snapshots/` baselines (committed)
41
- - `profiles.yaml` one entry per environment, connection settings only (committed)
42
- - `.auth/` saved logins: never read, never commit
43
- - `reports/` run reports: never commit
44
- - `{{framework}}/` the framework (installed package): never edit
45
- - `.env` local settings (`ASRUN_ROOT`): never commit
1
+ # <project name> — asrun workspace
2
+
3
+ This project uses the asrun framework, which lives in `{{framework}}` as an installed dependency. The framework is
4
+ read-only here: never edit files inside it, and never commit test data into it.
5
+
6
+ Read this file and the framework baseline it points at, before any AI step.
7
+
8
+ ## 1. How rules combine
9
+
10
+ Rules are applied in this order, and a later layer wins where they conflict (the framework baseline, "Precedence"):
11
+
12
+ 1. `F1`–`F6` of the framework baseline: safety, in every project. Nothing here can change them.
13
+ 2. The framework baseline, `{{framework}}/AGENTS.md`.
14
+ 3. The framework skill (`{{framework}}/.claude/skills/<skill>/SKILL.md`).
15
+ 4. A project skill override, `.claude/skills/<skill>.md` (optional; the command and the skill read it).
16
+ 5. This file, §2 below: what this project does differently.
17
+
18
+ The framework skills are loaded by the commands in `.claude/commands/` (Claude Code) and `.opencode/commands/`
19
+ (OpenCode). The skills live in the framework package; §3 holds this project's facts.
20
+
21
+ ## 2. Project rules
22
+
23
+ Only what differs from the baseline: name the rule id, and whether it `narrows` (stricter) or `replaces` (a different
24
+ default, for example the artifact language) the baseline rule. `replaces` is refused for `F1`–`F6`. Rows whose first
25
+ cell is not a rule id are ignored, so the placeholder below can stay until you need it.
26
+
27
+ | Rule | Project rule | Effect |
28
+ |---|---|---|
29
+ | <rule id, e.g. F10> | <what this project does differently from the baseline> | narrows |
30
+
31
+ Examples (delete what you do not use; the fenced block is an example, not a rule):
32
+
33
+ ```markdown
34
+ | F10 | Case prose, report summaries and assertion messages in Simplified Chinese; code and commit messages stay English. | replaces |
35
+ | F9 | Staging writes are limited to accounts named `qa-*` and names starting with `[asrun]`. | narrows |
36
+ | F7 | This project runs production profiles only from a scheduled job; people use `--env staging`. | narrows |
37
+ ```
38
+
39
+ `pnpm asrun doctor` fails when this table names a `[non-negotiable]` rule, an unknown rule id, or an `Effect` that is
40
+ not `narrows` or `replaces`.
41
+
42
+ The structural tokens are fixed English whatever the project language: the section headings (`## Values`, `## Targets`,
43
+ ...), a step's `- Recipe (phase): ...` lines, `### T<n>`, the `TODO` blocks with their `Answer:` line,
44
+ `[input]`/`[writes]`/`[email]`, `RESULT:`, and the front-matter keys. See "Fixed tokens" in the framework baseline.
45
+
46
+ ## 3. Project facts
47
+
48
+ - Profiles and environments: `profiles.yaml`.
49
+ - Areas: one folder per area below `cases/` (for example `cases/<area>/<name>.md`).
50
+ - Expected values: they live in each case's `## Values` block, never here and never in `profiles.yaml`.
51
+ - Case document contract (front matter, steps, values, target cards, how a spec consumes them):
52
+ `{{framework}}/docs/agents/case-format.md`.
53
+ - Who owns a finding, and where findings are reported: <fill in>.
54
+
55
+ ## Layout of this workspace
56
+
57
+ - `cases/**/*.md` human-written test cases (committed)
58
+ - `tests/**/*.spec.ts` generated Playwright specs and their `*-snapshots/` baselines (committed)
59
+ - `profiles.yaml` one entry per environment, connection settings only (committed)
60
+ - `.claude/commands/` `/make-case`, `/explore-case`, `/gen-spec`, `/heal-spec` for Claude Code
61
+ - `.opencode/commands/` the same commands for OpenCode (it does not read `.claude/commands`)
62
+ - `.claude/skills/` optional overrides of a framework skill: `<skill>.md`
63
+ - `.auth/` saved logins: never read, never commit
64
+ - `reports/` run reports: never commit
65
+ - `{{framework}}/` the framework (installed package): never edit
66
+ - `.env` local settings (`ASRUN_ROOT`): never commit