@asrunjs/asrun 0.0.0-stage → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/explore-case.md +18 -0
- package/.claude/commands/gen-spec.md +16 -0
- package/.claude/commands/heal-spec.md +26 -0
- package/.claude/commands/make-case.md +11 -0
- package/.claude/settings.json +5 -0
- package/.claude/skills/exploring-cases/SKILL.md +146 -0
- package/.claude/skills/exploring-cases/references/assertion-targets.md +103 -0
- package/.claude/skills/exploring-cases/references/handoff-card.js +128 -0
- package/.claude/skills/exploring-cases/references/open-questions.md +116 -0
- package/.claude/skills/exploring-cases/references/readiness.md +146 -0
- package/.claude/skills/exploring-cases/references/user-handoff.md +157 -0
- package/.claude/skills/generating-specs/SKILL.md +219 -0
- package/.claude/skills/generating-specs/references/conventions.md +148 -0
- package/.claude/skills/making-cases/SKILL.md +72 -0
- package/.claude/skills/making-cases/references/gaps.md +36 -0
- package/.claude/skills/making-cases/references/normalizing.md +39 -0
- package/.claude/skills/making-cases/references/splitting.md +28 -0
- package/.claude/skills/making-cases/references/writes-and-email.md +37 -0
- package/AGENTS.md +159 -0
- package/CLAUDE.md +2 -0
- package/LICENSE +21 -0
- package/README.md +511 -2
- package/bin/asrun.mjs +49 -0
- package/dist/lib/account-lock.d.ts +6 -0
- package/dist/lib/account-lock.js +52 -0
- package/dist/lib/assertions.d.ts +14 -0
- package/dist/lib/assertions.js +39 -0
- package/dist/lib/baselines.d.ts +43 -0
- package/dist/lib/baselines.js +108 -0
- package/dist/lib/build-artifacts.d.ts +8 -0
- package/dist/lib/build-artifacts.js +10 -0
- package/dist/lib/case-doc.d.ts +311 -0
- package/dist/lib/case-doc.js +1117 -0
- package/dist/lib/case-id.d.ts +11 -0
- package/dist/lib/case-id.js +17 -0
- package/dist/lib/case-ops.d.ts +26 -0
- package/dist/lib/case-ops.js +110 -0
- package/dist/lib/cases.d.ts +16 -0
- package/dist/lib/cases.js +51 -0
- package/dist/lib/cli.d.ts +10 -0
- package/dist/lib/cli.js +18 -0
- package/dist/lib/command.d.ts +24 -0
- package/dist/lib/command.js +90 -0
- package/dist/lib/downloads.d.ts +42 -0
- package/dist/lib/downloads.js +84 -0
- package/dist/lib/email.d.ts +10 -0
- package/dist/lib/email.js +19 -0
- package/dist/lib/ffmpeg.d.ts +8 -0
- package/dist/lib/ffmpeg.js +50 -0
- package/dist/lib/fixtures.d.ts +40 -0
- package/dist/lib/fixtures.js +149 -0
- package/dist/lib/login-card.d.ts +22 -0
- package/dist/lib/login-card.js +183 -0
- package/dist/lib/node-cli.d.ts +15 -0
- package/dist/lib/node-cli.js +36 -0
- package/dist/lib/paths.d.ts +25 -0
- package/dist/lib/paths.js +88 -0
- package/dist/lib/pdf.d.ts +71 -0
- package/dist/lib/pdf.js +140 -0
- package/dist/lib/profiles.d.ts +62 -0
- package/dist/lib/profiles.js +174 -0
- package/dist/lib/ready.d.ts +36 -0
- package/dist/lib/ready.js +53 -0
- package/dist/lib/records.d.ts +33 -0
- package/dist/lib/records.js +61 -0
- package/dist/lib/run-filters.d.ts +19 -0
- package/dist/lib/run-filters.js +38 -0
- package/dist/lib/run-summary.d.ts +90 -0
- package/dist/lib/run-summary.js +35 -0
- package/dist/lib/session.d.ts +30 -0
- package/dist/lib/session.js +111 -0
- package/dist/lib/summary-reporter.d.ts +16 -0
- package/dist/lib/summary-reporter.js +146 -0
- package/dist/lib/targets.d.ts +19 -0
- package/dist/lib/targets.js +26 -0
- package/dist/lib/ui-build.d.ts +10 -0
- package/dist/lib/ui-build.js +30 -0
- package/dist/lib/values.d.ts +29 -0
- package/dist/lib/values.js +85 -0
- package/dist/lib/write-gate.d.ts +41 -0
- package/dist/lib/write-gate.js +129 -0
- package/dist/playwright.config.d.ts +2 -0
- package/dist/playwright.config.js +92 -0
- package/dist/scripts/build.d.ts +1 -0
- package/dist/scripts/build.js +21 -0
- package/dist/scripts/check-cases.d.ts +1 -0
- package/dist/scripts/check-cases.js +149 -0
- package/dist/scripts/cleanup.d.ts +1 -0
- package/dist/scripts/cleanup.js +25 -0
- package/dist/scripts/cli.d.ts +1 -0
- package/dist/scripts/cli.js +28 -0
- package/dist/scripts/doctor.d.ts +1 -0
- package/dist/scripts/doctor.js +296 -0
- package/dist/scripts/explore.d.ts +1 -0
- package/dist/scripts/explore.js +75 -0
- package/dist/scripts/go.mjs +188 -0
- package/dist/scripts/heal-evidence.mjs +143 -0
- package/dist/scripts/init.d.ts +22 -0
- package/dist/scripts/init.js +332 -0
- package/dist/scripts/lint-specs.d.ts +1 -0
- package/dist/scripts/lint-specs.js +141 -0
- package/dist/scripts/login.d.ts +1 -0
- package/dist/scripts/login.js +75 -0
- package/dist/scripts/new-case.d.ts +1 -0
- package/dist/scripts/new-case.js +19 -0
- package/dist/scripts/render-pdf.d.ts +1 -0
- package/dist/scripts/render-pdf.js +44 -0
- package/dist/scripts/report.d.ts +1 -0
- package/dist/scripts/report.js +14 -0
- package/dist/scripts/review-baselines.d.ts +1 -0
- package/dist/scripts/review-baselines.js +30 -0
- package/dist/scripts/run.d.ts +1 -0
- package/dist/scripts/run.js +423 -0
- package/dist/scripts/status.d.ts +1 -0
- package/dist/scripts/status.js +31 -0
- package/dist/scripts/ui-build.d.ts +1 -0
- package/dist/scripts/ui-build.js +7 -0
- package/dist/scripts/ui-dev.d.ts +1 -0
- package/dist/scripts/ui-dev.js +3 -0
- package/dist/scripts/ui.d.ts +1 -0
- package/dist/scripts/ui.js +13 -0
- package/dist/ui/dev.d.ts +1 -0
- package/dist/ui/dev.js +36 -0
- package/dist/ui/server/actions.d.ts +95 -0
- package/dist/ui/server/actions.js +365 -0
- package/dist/ui/server/ai-models.d.ts +3 -0
- package/dist/ui/server/ai-models.js +347 -0
- package/dist/ui/server/ai-stream.d.ts +66 -0
- package/dist/ui/server/ai-stream.js +408 -0
- package/dist/ui/server/app.d.ts +4 -0
- package/dist/ui/server/app.js +395 -0
- package/dist/ui/server/cases.d.ts +23 -0
- package/dist/ui/server/cases.js +170 -0
- package/dist/ui/server/files.d.ts +5 -0
- package/dist/ui/server/files.js +67 -0
- package/dist/ui/server/index.d.ts +1 -0
- package/dist/ui/server/index.js +33 -0
- package/dist/ui/server/jobs.d.ts +80 -0
- package/dist/ui/server/jobs.js +317 -0
- package/dist/ui/server/profiles.d.ts +35 -0
- package/dist/ui/server/profiles.js +371 -0
- package/dist/ui/server/watch-files.d.ts +4 -0
- package/dist/ui/server/watch-files.js +36 -0
- package/dist/ui/shared/ai-commands.d.ts +88 -0
- package/dist/ui/shared/ai-commands.js +177 -0
- package/dist/ui/shared/case-id.d.ts +1 -0
- package/dist/ui/shared/case-id.js +1 -0
- package/dist/ui/shared/jobs.d.ts +6 -0
- package/dist/ui/shared/jobs.js +7 -0
- package/dist/ui/shared/product.d.ts +3 -0
- package/dist/ui/shared/product.js +3 -0
- package/dist/ui/shared/run-filters.d.ts +1 -0
- package/dist/ui/shared/run-filters.js +1 -0
- package/dist/ui/shared/site-address.d.ts +6 -0
- package/dist/ui/shared/site-address.js +11 -0
- package/dist/ui/shared/types.d.ts +371 -0
- package/dist/ui/shared/types.js +1 -0
- package/docs/agents/framework-dev.md +79 -0
- package/docs/theme-guide.html +922 -0
- package/package.json +109 -3
- package/templates/.claude/commands/explore-case.md +22 -0
- package/templates/.claude/commands/gen-spec.md +21 -0
- package/templates/.claude/commands/heal-spec.md +31 -0
- package/templates/.claude/commands/make-case.md +13 -0
- package/templates/.claude/settings.json +5 -0
- package/templates/.mcp.json +8 -0
- package/templates/AGENTS.md +64 -0
- package/templates/CLAUDE.md +1 -0
- package/templates/cases/_template.md +57 -0
- package/templates/gitignore +13 -0
- package/templates/opencode.jsonc +16 -0
- package/templates/package.json +31 -0
- package/templates/playwright.config.ts +3 -0
- package/templates/pnpm-workspace.yaml +5 -0
- package/templates/profiles.yaml +42 -0
- package/templates/seed.spec.ts +9 -0
- package/templates/tsconfig.json +14 -0
- package/ui/dist/apple-touch-icon.png +0 -0
- package/ui/dist/assets/apl-gfMMiTNf.js +1 -0
- package/ui/dist/assets/asciiarmor-C_eisQCp.js +1 -0
- package/ui/dist/assets/asn1-DCl_8MCW.js +1 -0
- package/ui/dist/assets/asterisk-BEn0Cwpx.js +1 -0
- package/ui/dist/assets/brainfuck-D5EjA2JK.js +1 -0
- package/ui/dist/assets/clike-xvONpD0Z.js +1 -0
- package/ui/dist/assets/clojure-BflJGmDX.js +1 -0
- package/ui/dist/assets/cmake-84nbDH0I.js +1 -0
- package/ui/dist/assets/cobol-DHLQUbN7.js +1 -0
- package/ui/dist/assets/coffeescript-De_zI4J0.js +1 -0
- package/ui/dist/assets/commonlisp-DgAKBe0r.js +1 -0
- package/ui/dist/assets/crystal-Bl6YKCG_.js +1 -0
- package/ui/dist/assets/css-CnaYoQib.js +1 -0
- package/ui/dist/assets/cypher-p9eYesGn.js +1 -0
- package/ui/dist/assets/d-DWkQyNnU.js +1 -0
- package/ui/dist/assets/diff-ChtP43wD.js +1 -0
- package/ui/dist/assets/dist-BGbe0-Aa.js +1 -0
- package/ui/dist/assets/dist-Ba-HC662.js +2 -0
- package/ui/dist/assets/dist-BtMLMthJ.js +1 -0
- package/ui/dist/assets/dist-Bw5-uxs9.js +1 -0
- package/ui/dist/assets/dist-BzulKuec.js +1 -0
- package/ui/dist/assets/dist-C-Oyo1ZT.js +1 -0
- package/ui/dist/assets/dist-C8MdhqgG.js +1 -0
- package/ui/dist/assets/dist-CG1pHn5m.js +1 -0
- package/ui/dist/assets/dist-CgIQYY1o.js +1 -0
- package/ui/dist/assets/dist-CmpteMYY.js +9 -0
- package/ui/dist/assets/dist-DDJNvivM.js +1 -0
- package/ui/dist/assets/dist-DF3F_I8N.js +1 -0
- package/ui/dist/assets/dist-DSy3dVf4.js +11 -0
- package/ui/dist/assets/dist-Dlorc3Sx.js +1 -0
- package/ui/dist/assets/dist-Dq6sW5Ur.js +1 -0
- package/ui/dist/assets/dist-Dr9crbBz.js +13 -0
- package/ui/dist/assets/dist-jjxQutEm.js +1 -0
- package/ui/dist/assets/dockerfile-DJkHQgkl.js +1 -0
- package/ui/dist/assets/dtd-DTF72YNO.js +1 -0
- package/ui/dist/assets/dylan-S7ucR4pR.js +1 -0
- package/ui/dist/assets/ebnf-D4c0_ac3.js +1 -0
- package/ui/dist/assets/ecl-BVIeWmwY.js +1 -0
- package/ui/dist/assets/eiffel-BEjRio4Q.js +1 -0
- package/ui/dist/assets/elm-Cshzl8qu.js +1 -0
- package/ui/dist/assets/erlang-BV0c4Hdn.js +1 -0
- package/ui/dist/assets/factor-LKH5gN8L.js +1 -0
- package/ui/dist/assets/fcl-ClOhbFR7.js +1 -0
- package/ui/dist/assets/forth-D6RzAl64.js +1 -0
- package/ui/dist/assets/fortran-Bt6PBEDR.js +1 -0
- package/ui/dist/assets/gas-BHEdbvp9.js +1 -0
- package/ui/dist/assets/gherkin-oBAE_ms0.js +1 -0
- package/ui/dist/assets/groovy-CUrWKPia.js +1 -0
- package/ui/dist/assets/haskell-BRsoo5mP.js +1 -0
- package/ui/dist/assets/haxe-DjVOVvNP.js +1 -0
- package/ui/dist/assets/http-CLVgA2GD.js +1 -0
- package/ui/dist/assets/idl-BUZw3wgd.js +1 -0
- package/ui/dist/assets/index-CTmnF0Wp.css +2 -0
- package/ui/dist/assets/index-P_Yo0ZZn.js +338 -0
- package/ui/dist/assets/javascript-lS2r2ifg.js +1 -0
- package/ui/dist/assets/julia-CQfgDRfP.js +1 -0
- package/ui/dist/assets/livescript-BXxG0Yva.js +1 -0
- package/ui/dist/assets/lua-DPCEkfD_.js +1 -0
- package/ui/dist/assets/mathematica-u8YMmbU0.js +1 -0
- package/ui/dist/assets/mbox-BEMVcqjs.js +1 -0
- package/ui/dist/assets/mirc-C9zpzAZU.js +1 -0
- package/ui/dist/assets/mllike-B45xgp2S.js +1 -0
- package/ui/dist/assets/modelica-CqHI_q-D.js +1 -0
- package/ui/dist/assets/mscgen-qgSLujhx.js +1 -0
- package/ui/dist/assets/mumps-BSzj6Xyz.js +1 -0
- package/ui/dist/assets/nginx-C-ORUuei.js +1 -0
- package/ui/dist/assets/nsis-viPvXFiq.js +1 -0
- package/ui/dist/assets/ntriples-CzMBulkp.js +1 -0
- package/ui/dist/assets/octave-D2Q8cUp1.js +1 -0
- package/ui/dist/assets/oz-BwTct_5T.js +1 -0
- package/ui/dist/assets/pascal-Dp_KdFgX.js +1 -0
- package/ui/dist/assets/perl-BkSkiI9m.js +1 -0
- package/ui/dist/assets/pig-AF4Hwhju.js +1 -0
- package/ui/dist/assets/powershell-D8WpQPaX.js +1 -0
- package/ui/dist/assets/properties-C357ku5U.js +1 -0
- package/ui/dist/assets/protobuf-D1ij_kNL.js +1 -0
- package/ui/dist/assets/pug-BqCHZ5xT.js +1 -0
- package/ui/dist/assets/puppet-BZyAS7I4.js +1 -0
- package/ui/dist/assets/python-B_1Jd8HM.js +1 -0
- package/ui/dist/assets/q-B9NhS7L_.js +1 -0
- package/ui/dist/assets/r-CxDmU1I4.js +1 -0
- package/ui/dist/assets/rpm-gpDK5sbt.js +1 -0
- package/ui/dist/assets/ruby-ChfNwlCY.js +1 -0
- package/ui/dist/assets/sas-C2OyD2Y6.js +1 -0
- package/ui/dist/assets/scheme-D9qYN0V5.js +1 -0
- package/ui/dist/assets/shell-DwuoZtxw.js +1 -0
- package/ui/dist/assets/sieve-tLWDfEkZ.js +1 -0
- package/ui/dist/assets/simple-mode-DXiLM1-G.js +1 -0
- package/ui/dist/assets/smalltalk-BLp5-jwC.js +1 -0
- package/ui/dist/assets/solr-B78wSQJQ.js +1 -0
- package/ui/dist/assets/sparql-Dz-mlCAC.js +1 -0
- package/ui/dist/assets/spreadsheet-CTIncYxj.js +1 -0
- package/ui/dist/assets/sql-DuQMFDvy.js +1 -0
- package/ui/dist/assets/stex-DNzd62Q_.js +1 -0
- package/ui/dist/assets/stylus-BOVgKnng.js +1 -0
- package/ui/dist/assets/swift-DKB6_1j6.js +1 -0
- package/ui/dist/assets/tcl-BGfruO1u.js +1 -0
- package/ui/dist/assets/textile-C4jVoHG1.js +1 -0
- package/ui/dist/assets/tiddlywiki-VlUiK86J.js +1 -0
- package/ui/dist/assets/tiki-CvUEFusT.js +1 -0
- package/ui/dist/assets/toml-BPTmHmyx.js +1 -0
- package/ui/dist/assets/troff-C2dAgh7P.js +1 -0
- package/ui/dist/assets/ttcn-PPSTk1ow.js +1 -0
- package/ui/dist/assets/ttcn-cfg-2b4Gb5sE.js +1 -0
- package/ui/dist/assets/turtle-Cx3rYX2g.js +1 -0
- package/ui/dist/assets/vb-BthdM_4N.js +1 -0
- package/ui/dist/assets/vbscript-D9f6rSsU.js +1 -0
- package/ui/dist/assets/velocity-CqftRsjg.js +1 -0
- package/ui/dist/assets/verilog-CjI9Y6RI.js +1 -0
- package/ui/dist/assets/vhdl-kS471soi.js +1 -0
- package/ui/dist/assets/webidl-D5jiJvOU.js +1 -0
- package/ui/dist/assets/xquery-BKJx4mf9.js +1 -0
- package/ui/dist/assets/yacas-CXXgtw0p.js +1 -0
- package/ui/dist/assets/z80-Co3PbHdE.js +1 -0
- package/ui/dist/favicon.ico +0 -0
- package/ui/dist/favicon.svg +11 -0
- package/ui/dist/help/ai-tools.png +0 -0
- package/ui/dist/help/case-explore.png +0 -0
- package/ui/dist/help/case-generate.png +0 -0
- package/ui/dist/help/cases-memo.png +0 -0
- package/ui/dist/help/profile-form.png +0 -0
- package/ui/dist/help/profiles-login.png +0 -0
- package/ui/dist/help/reports.png +0 -0
- package/ui/dist/help/run-tests.png +0 -0
- package/ui/dist/index.html +23 -0
- package/ui/dist/logos/claude.svg +1 -0
- package/ui/dist/logos/opencode.svg +1 -0
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Explore the real pages for one case (optionally attended: you do the steps that write data), before generating the spec
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `exploring-cases` skill and explore: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/exploring-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: `<case-id> <env> [--attended]`. The case id is the path below cases/ without `.md` (it may include a group
|
|
11
|
+
folder, e.g. documents/landing-page) and is required. `<env>` is `staging` or `production`; it is required and there is no
|
|
12
|
+
default (without one, stop and say so). With `--attended` I am at the machine and the browser window is visible: when a step
|
|
13
|
+
writes data or needs a choice, hand it to me as the skill describes, wait for me, and record what I did as a general
|
|
14
|
+
recipe under the step it belongs to (`- Recipe:` lines). Without `--attended`, never wait for me.
|
|
15
|
+
Follow `assertion-targets.md` (list the candidates, never pick when in doubt, ask for the exact image/text/place/UI)
|
|
16
|
+
and `readiness.md` (wait for the key requests and the content before you read anything) of the skill.
|
|
17
|
+
This command edits cases/<case-id>.md only (plus screenshots under `.explore/targets/`). Do not ask me questions in the chat: write open questions into the case
|
|
18
|
+
file as the skill describes, and end with the `RESULT:` line.
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Generate or update tests/<id>.spec.ts from cases/<id>.md (the id may be <group>/<name>)
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `generating-specs` skill and generate the spec for: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: `<case-id> [<env>] [--force]`. The case file is cases/<case-id>.md (the id may include one group folder, e.g.
|
|
11
|
+
documents/landing-page). `<env>` is the environment to explore in (`staging` or `production`); there is no default. If the spec already exists, it is
|
|
12
|
+
left alone unless `--force` is given; with `--force` it is regenerated and overwritten.
|
|
13
|
+
Follow the skill's procedure exactly, including `assertion-targets.md` and `readiness.md`; run `pnpm spec:lint <case-id>`. Only generate the spec: do not run `pnpm asrun` (running specs is a separate
|
|
14
|
+
process), and do not create baselines.
|
|
15
|
+
Do not ask me questions: resolve ambiguity as the skill describes (explore the page, use observed values marked for QA,
|
|
16
|
+
write the remaining questions into the case file), and end with the `RESULT:` line.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Diagnose and fix a failing spec from its last report (without hiding real product regressions)
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `generating-specs` skill, section "Healing a failing spec", and follow it exactly.
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/generating-specs.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
Arguments: $ARGUMENTS
|
|
11
|
+
Format: `<case-id> <env> [--profiles a,b] [--report <run-id>] [--hints <file>]`, with optional free text after them.
|
|
12
|
+
- `<case-id>` and `<env>` (`staging` or `production`) are required. There is no default environment. Without one, stop
|
|
13
|
+
and say so.
|
|
14
|
+
- `--profiles`: the profiles to heal. Without it, every profile that is logged in in `<env>`.
|
|
15
|
+
- `--report`: the report (folder name under `reports/`) to read first. Without it, the newest report of this case.
|
|
16
|
+
- `--hints <file>`: a file with what the user knows (what to heal, evidence, clues). Read it first. Any free text
|
|
17
|
+
after the arguments is the same kind of clue. Both are optional.
|
|
18
|
+
Before you decide "product or test", audit the assertion itself (target card, baseline content, readiness): see steps 4
|
|
19
|
+
of the skill's healing section, `assertion-targets.md` and `readiness.md`. Run `pnpm spec:lint <case-id>`.
|
|
20
|
+
Read the existing report first (`pnpm heal:evidence --case <case-id> --env <env>`), and run the case again only when the
|
|
21
|
+
skill says it is needed. Always run with `--logged-in`. Tell me whether it is a test problem or a product finding.
|
|
22
|
+
|
|
23
|
+
Run `pnpm asrun ...`, `pnpm heal:evidence ...`, `pnpm spec:lint ...`, `pnpm baselines:review ...` and `pnpm status ...` as plain commands, optionally piped to `tail` or
|
|
24
|
+
`head`, and do not chain other commands with them (`;` or `&&`). Only these exact forms run outside Claude Code's
|
|
25
|
+
sandbox, which tsx and the browser need. If a command still fails with `listen EPERM` or a sandbox error, stop and
|
|
26
|
+
report it. Do not ask to disable the sandbox, and never add `--allow-writes` or set `ASRUN_ALLOW_WRITES`.
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Turn raw test notes (pasted text or a file) into case documents in cases/
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
Load the `making-cases` skill and turn this input into case documents: $ARGUMENTS
|
|
6
|
+
In this repository the workspace is `examples/demo` (set `ASRUN_ROOT=examples/demo`); the framework is `.`.
|
|
7
|
+
Rules combine in this order, and a later one wins where they conflict: the framework baseline `AGENTS.md`, the
|
|
8
|
+
framework skill, `.claude/skills/making-cases.md` if it exists, then the workspace `AGENTS.md` §2 in `examples/demo`
|
|
9
|
+
(Project rules). F1–F6 always win.
|
|
10
|
+
The input is either text or a path to a file with the notes. Follow the skill's procedure exactly.
|
|
11
|
+
Ask me the open questions before writing any case file. Do not generate specs; that is `/gen-spec`.
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: exploring-cases
|
|
3
|
+
description: Explores the real pages for one case document (cases/<id>.md) before its spec is generated. Checks the case against profiles.yaml and the pages, fixes wording mistakes, records what the pages show, turns unknowns into answerable TODO blocks, and, in an attended run, lets the person do the steps that write data or need a choice while it waits and records a general recipe under that step's `- Recipe:` lines. Ends with RESULT ready or needs-input. Use when asked to explore, review, check, clean up or narrow down a case, or for /explore-case. Its open-question rules are shared with generating-specs.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploring a case document
|
|
7
|
+
|
|
8
|
+
Read the workspace `AGENTS.md` (hard rules and project rules), `cases/_template.md` (the target format), `references/open-questions.md` (how
|
|
9
|
+
questions are written, resolved and reported), `references/assertion-targets.md` (which exact image, text, file or
|
|
10
|
+
place a check looks at, and when to ask instead of picking), `references/readiness.md` (wait for the content before
|
|
11
|
+
you read it) and `references/user-handoff.md` (how a person works in the browser while the agent waits, and how it is
|
|
12
|
+
recorded). All but the last are binding for `/gen-spec` and healing too.
|
|
13
|
+
|
|
14
|
+
## Before you start
|
|
15
|
+
Read the workspace `AGENTS.md`: its §2 holds the project rules, and it points at the framework baseline. If
|
|
16
|
+
`.claude/skills/exploring-cases.md` exists in the workspace, read it and apply it after this skill. Then apply
|
|
17
|
+
`AGENTS.md` §2, which wins over both (the baseline, "Precedence"). F1–F6 always win. This is not a stop condition:
|
|
18
|
+
continue with the procedure below.
|
|
19
|
+
|
|
20
|
+
## Where this skill sits
|
|
21
|
+
Three steps, each learns one thing:
|
|
22
|
+
1. `/make-case` (making-cases): the **intent**. Turns raw notes into case documents. No browser.
|
|
23
|
+
2. `/explore-case` (this skill): the **facts and the inputs**. What the pages really show, and what a person has to
|
|
24
|
+
do for the steps that write or need a choice. The only step where a person and the agent work together.
|
|
25
|
+
3. `/gen-spec` (generating-specs): the **implementation**. Replays the case and its step recipes to find locators and
|
|
26
|
+
writes the spec. It does not judge the case again.
|
|
27
|
+
|
|
28
|
+
The purpose of this skill is to leave a case that can be replayed without anybody: no ambiguity, every fact marked
|
|
29
|
+
as observed, every input step described as a general recipe.
|
|
30
|
+
|
|
31
|
+
This skill edits `cases/<id>.md` only. It never edits `profiles.yaml`, `tests/`, the framework folder or anything
|
|
32
|
+
under `.auth/`.
|
|
33
|
+
**The agent never writes data**, in staging or in production: a person does every step that writes or needs a
|
|
34
|
+
choice (see `references/user-handoff.md`).
|
|
35
|
+
|
|
36
|
+
It never stops to ask in the chat. A question that cannot be answered from the page goes into the case file, where
|
|
37
|
+
the user answers it by editing the file, and the report says `RESULT: needs-input`. The one exception is a handoff in
|
|
38
|
+
an attended run, where waiting for the person is the point.
|
|
39
|
+
|
|
40
|
+
## Inputs
|
|
41
|
+
- A case id (path below `cases/` without `.md`, e.g. `documents/landing-page`). It is required: an attended exploration of
|
|
42
|
+
every case at once is not a thing. Without an id, stop and say so.
|
|
43
|
+
- An environment to explore (`staging` or `production`). It is required and there is no default. Without one, stop
|
|
44
|
+
before exploring and report `RESULT: needs-input` ("pass the environment").
|
|
45
|
+
- Optionally `--attended`: a person is at the machine and the browser window is visible, so handoffs may happen.
|
|
46
|
+
Without it nothing waits, and a step that needs a person becomes a `[blocking]` TODO.
|
|
47
|
+
|
|
48
|
+
## Procedure
|
|
49
|
+
1. **Read** the case, `profiles.yaml` and `cases/_template.md`. Apply uncommitted user edits as they are: they are the
|
|
50
|
+
newest wording. Use targeted edits, never rewrite a whole file.
|
|
51
|
+
2. **Static review.** Find:
|
|
52
|
+
- missing values for a profile: compare the entries of `## Values` (keyed by `default` and profile ids) with the
|
|
53
|
+
ids in `profiles.yaml`. A missing entry is a gap, not something to invent: leave it to the observations and
|
|
54
|
+
`/gen-spec`. The case's `## Values` is the only place for expected values; never copy them into `profiles.yaml`;
|
|
55
|
+
- ambiguous steps and checks without an expected result;
|
|
56
|
+
- steps that write data or need a choice: mark them `[writes]` / `[input]`. The front matter must list the
|
|
57
|
+
environments the finished spec may write in (`writes: [staging]`, or `[staging, production]`) with a
|
|
58
|
+
`writes-note`, the data must be named `[asrun] ...` and cleaned up, and a step that sends email is never
|
|
59
|
+
allowed in production and only in other environments when the profile has `email: true`;
|
|
60
|
+
- missing or doubtful `suites`/`envs`, a `writes` environment that `envs` excludes, and visual checks that are
|
|
61
|
+
too broad (screenshot a small element).
|
|
62
|
+
3. **Repair the wording** (edit the file). Fix typos and grammar, make steps one action or one check each, put
|
|
63
|
+
assumptions in `## Notes` (not in `## Expected`), remove duplicates and template placeholders, make paths relative
|
|
64
|
+
to the app. Do not add checks the case did not ask for and do not drop any. Never edit the `## Source` table.
|
|
65
|
+
4. **Explore**. The exploration runs through the `playwright-test` MCP only. If its tools are not registered in this
|
|
66
|
+
session (no `browser_*` / `generator_*` tool) or it reports that it failed to start, do not fall back to ad-hoc
|
|
67
|
+
Playwright scripts: they see a different browser and their network view does not count as
|
|
68
|
+
`browser_network_requests` (A2). Stop, tell the user which MCP server is missing and to run `pnpm asrun doctor`
|
|
69
|
+
(it names the broken `.mcp.json` / `opencode.jsonc` and the fix), make no observations, and end with
|
|
70
|
+
`RESULT: needs-input`.
|
|
71
|
+
Run `pnpm status --env <env>`. If it cannot run (a sandbox error such as
|
|
72
|
+
`listen EPERM` or a Chromium start failure), do not give up: open the profile's page through the MCP, and a page
|
|
73
|
+
that lands on the login page means the profile is not logged in. Explore at least one logged-in profile per
|
|
74
|
+
domain with the `playwright-test` MCP (`generator_setup_page` with `project=<profile-id>` and
|
|
75
|
+
`seedFile=tests/seed.spec.ts`). Profiles that are not logged in are skipped and named in the report; a human runs
|
|
76
|
+
`pnpm signin <profile-id>`.
|
|
77
|
+
- Do the read-only steps yourself: open, look, read, hover, switch tabs, open and close dialogs and menus.
|
|
78
|
+
- **A step that writes or needs a choice is done by the person**, per `references/user-handoff.md`: announce it,
|
|
79
|
+
name the finish condition, start recording, run the wait loop, then write down what happened. In a run that
|
|
80
|
+
is not attended, write a `TODO [blocking]` under the step instead.
|
|
81
|
+
- Close any modal that opens by itself without confirming it (for example the follow-up modal that opens after
|
|
82
|
+
publishing: close it and confirm nothing).
|
|
83
|
+
- When a confirm dialog appears while closing an email composer, choose "Discard" (see `AGENTS.md`).
|
|
84
|
+
- **Plan the order before you start: observe, then let the person delete.** Every read-only observation that
|
|
85
|
+
needs the data (preview, share menu, email composers, downloads, menus, the dialogs) is done while the data
|
|
86
|
+
exists, and the handoff that deletes it comes last. If the data is gone before a step was observed, that step
|
|
87
|
+
is not observed: say so, do not fill it in from memory.
|
|
88
|
+
- **Ready before you read** (`references/readiness.md`): `browser_network_clear` before an action that loads
|
|
89
|
+
something, then the action, then wait for its key requests (`browser_network_requests`) and for the content
|
|
90
|
+
signal, and only then take the snapshot. When the first read is empty or missing something, wait and read
|
|
91
|
+
again before you record or ask anything. Record the signal as `(observed after <signal>, ...)`.
|
|
92
|
+
- **Targets, not guesses** (`references/assertion-targets.md`): for every check on a logo, image, icon, file, PDF,
|
|
93
|
+
screenshot or repeated text, list ALL candidates, outline the one that is meant, take the highlighted screenshot to
|
|
94
|
+
`.explore/targets/<case-id>/`, **open that screenshot and look at it**, and write the card under `## Targets`.
|
|
95
|
+
If there are several candidates, only a position to tell them apart, a generic locator, dynamic data in the area,
|
|
96
|
+
or the page differs from the case: write a `[blocking]` TODO that names the exact image, text, place or UI wanted
|
|
97
|
+
and lists the candidates. In an attended run let the person outline the target with `browser_annotate`.
|
|
98
|
+
- **Downloads and email composers are opened for real** (the composer is closed with "Discard"). A download is
|
|
99
|
+
verified in five parts: starts, finishes, file exists, file valid, and a failure or timeout fails the check
|
|
100
|
+
(see `references/readiness.md`). Record the file name, size and kind as observed. A downloaded PDF is rendered
|
|
101
|
+
(`pnpm pdf:render`) and its page images are opened and looked at: record which pages show the target, where, and
|
|
102
|
+
the text. Which pages the check covers follows the case: every page unless the case says ONLY (strong wording).
|
|
103
|
+
- Compare every step with what is really there.
|
|
104
|
+
|
|
105
|
+
**Keep every browser call short, so nothing hangs** (`generating-specs` follows the same rules):
|
|
106
|
+
- One `browser_run_code_unsafe` call does one thing and should finish within about 60 seconds. Do not close a tab,
|
|
107
|
+
click, wait for a dialog and read it in one call: split it, and read the page (`browser_snapshot`) between the
|
|
108
|
+
steps. Only the handoff wait loop may take minutes.
|
|
109
|
+
- The browser times out an action or a lookup after 15 s (the config sets it). Pass a shorter `{timeout: 5000}` when
|
|
110
|
+
you wait for something that may not appear, and use `.catch()` or `isVisible()` for a one-time look while exploring only.
|
|
111
|
+
`isVisible()` answers for this instant and ignores any `timeout` passed to it, so after an action take a snapshot or
|
|
112
|
+
wait for a signal first. Never copy this pattern into a spec (see `references/readiness.md`, "Optional confirmations").
|
|
113
|
+
- After a click that should open something, take a snapshot and see what opened before you wait for it by role. A
|
|
114
|
+
dialog that did not appear is a finding (what did open instead?), not something to wait for or retry.
|
|
115
|
+
- A step that fails or times out twice is not tried a third time: write what you saw, add a `TODO [blocking]` under
|
|
116
|
+
the step and go on with the next one.
|
|
117
|
+
5. **Write the `## Targets` section and the ready signals.** One card per target, in the format of
|
|
118
|
+
`references/assertion-targets.md`, and a `[T<n>]` reference in each check that uses it. Add `Ready when:` to every
|
|
119
|
+
step or target whose content loads after an action. A check with no card or a card with open doubts is a
|
|
120
|
+
`[blocking]` TODO, never a guess. Check the screenshots you saved: each must show what its card says.
|
|
121
|
+
6. **Write the recipe lines** for every `[input]` or `[writes]` step that was handed off, from the recording, as a
|
|
122
|
+
general recipe under that step: one `- Recipe (phase): ...` line per phase. Format and rules are in
|
|
123
|
+
`references/user-handoff.md`. A step that was not handed off gets no recipe line and a `[blocking]` TODO. A case
|
|
124
|
+
that still has the old `## Inputs` section: move the lines of the steps you touch under those steps as
|
|
125
|
+
`- Recipe:`; `pnpm case:check` names the rest.
|
|
126
|
+
7. **Resolve every open point** in the order of `references/open-questions.md`: answer it from the page, else apply
|
|
127
|
+
the safe default as the sentence above a `[non-blocking]` TODO block, else keep a `[blocking]` block. Write the
|
|
128
|
+
outcome into the case, every block with an empty `Answer:` line.
|
|
129
|
+
8. **Check**: run `pnpm case:check` (it fails on an incomplete target card and warns on a check that has none) and fix every FAIL. If it fails with a sandbox error (`listen EPERM`), run
|
|
130
|
+
`node --import tsx <framework>/scripts/check-cases.ts` instead; it checks the same thing.
|
|
131
|
+
9. **Report** in the user's language: what was changed (wording fixes, observations added, TODOs closed or narrowed),
|
|
132
|
+
what was explored (profile ids), which steps the person did and what was recorded for them, what was skipped and
|
|
133
|
+
why, the target cards written (and which ones are still questions), the steps whose first read was too early,
|
|
134
|
+
the steps that could not be observed (and why), the questions that remain with the sentence that deleting each block would confirm, and findings that look
|
|
135
|
+
like product bugs. End with the `RESULT:` line.
|
|
136
|
+
|
|
137
|
+
## Hard rules
|
|
138
|
+
- Edit `cases/<id>.md` only. Do not touch the `## Source` table.
|
|
139
|
+
- The agent never writes data (see above). Never add `--allow-writes`, never set `ASRUN_ALLOW_WRITES`.
|
|
140
|
+
- Never wait for a person in a run that is not attended.
|
|
141
|
+
- Never invent expected values, profile names, counts or UI texts. Observations are marked as such. Values that a
|
|
142
|
+
check asserts are written into the case's `## Values` under the profile id (or `default` when they hold for every
|
|
143
|
+
profile), and never into `profiles.yaml`.
|
|
144
|
+
- Never pick a target when in doubt, and never record a fact that was read before the content finished loading.
|
|
145
|
+
- Never read or print anything under `.auth/`. Never automate login or 2FA.
|
|
146
|
+
- Prose follows the artifact language of the workspace `AGENTS.md` §2 (English by default); structural tokens stay English.
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# Assertion targets: never guess what a check looks at
|
|
2
|
+
|
|
3
|
+
Shared by `exploring-cases`, `generating-specs` and healing. It is binding in all three.
|
|
4
|
+
|
|
5
|
+
A check that looks at an image, a logo, a file, a piece of text or a place on the page must say **exactly which one**.
|
|
6
|
+
"The logo", "the header", "the first image" are not targets. A spec built on a guessed target passes or fails for
|
|
7
|
+
the wrong reason: a baseline once saved a content image instead of a logo, and a composer check once looked at a
|
|
8
|
+
close icon instead of the logo.
|
|
9
|
+
|
|
10
|
+
**When in doubt, ask. Never pick.** A wrong guess that passes is worse than a question.
|
|
11
|
+
|
|
12
|
+
## The target card
|
|
13
|
+
Every such check gets a card in the case, under `## Targets` (after `## Visual checks`). The checks refer to it as
|
|
14
|
+
`[T1]`, `[T2]`, ... (in "Expected", "Visual checks" and in steps).
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
## Targets
|
|
18
|
+
### T1 Preview logo
|
|
19
|
+
- What: the product logo (the ACME wordmark), not the avatar or a content image.
|
|
20
|
+
- Where: top-right corner of the preview page (`/preview/view/documents/...`).
|
|
21
|
+
- Identified by: the image with alt text "<observed alt>" inside the page header region (role and name, or test id; never `.first()`).
|
|
22
|
+
- Not: any element that shows item data (images, amounts, "Content #1").
|
|
23
|
+
- Ready when: the image is visible and its natural width is above 0.
|
|
24
|
+
- Reference example-staging: .explore/targets/documents/published-flow/T1-example-staging.png
|
|
25
|
+
- Reference example-production: .explore/targets/documents/published-flow/T1-example-production.png
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Fields:
|
|
29
|
+
- `What` (required): the thing itself, in words a QA person agrees with. Say what it is and, when a look-alike
|
|
30
|
+
exists on the page, what it is not.
|
|
31
|
+
- `Where` (required): the region of the page, dialog, tab or file page.
|
|
32
|
+
- `Identified by` (required): how it is told apart from every look-alike: role and accessible name, test id (`data-tn`),
|
|
33
|
+
alt text, `title`, or an anchor element and a relation to it. Never a position (`.first()`, `.nth()`, "the first
|
|
34
|
+
image") unless the case's author wrote that exact rule (then quote them).
|
|
35
|
+
- `Not` (recommended): the look-alikes that must not be matched. It is what `/gen-spec` and a reviewer check a baseline against.
|
|
36
|
+
- `Ready when` (required when the content loads after an action): see `readiness.md`.
|
|
37
|
+
- For a PDF or another multi-page file, also `Pages` (`all`, `first`, `last`, `1, 3`, plus the case's own words when it limits
|
|
38
|
+
the pages: see "Which pages are checked" in `readiness.md`; the default is every page) and `Region` (a corner or a box,
|
|
39
|
+
as fractions of the page: `top-left 30% x 12%`). Text that must or must not appear goes under `What` / `Not`.
|
|
40
|
+
- `Reference <profile-id>` (one per profile explored): a screenshot of the page with the target outlined
|
|
41
|
+
(`browser_highlight`, then `browser_take_screenshot`), saved under `.explore/targets/<case-id>/T<n>-<profile-id>.png`.
|
|
42
|
+
`.explore/` is git-ignored, because the pages are logged in. Write `none: <reason>` when it could not be taken.
|
|
43
|
+
A reference is evidence for the reader; it is never a test baseline.
|
|
44
|
+
|
|
45
|
+
`pnpm case:check` fails on a card without What/Where/Identified by and warns on a check about an image, logo, file,
|
|
46
|
+
screenshot or PDF that has no `[T<n>]`.
|
|
47
|
+
|
|
48
|
+
## Which checks need a card
|
|
49
|
+
Any check whose words include logo, image, icon, photo, screenshot, PDF, file, download, color, or "shows <profile>", and
|
|
50
|
+
every check that gets a screenshot baseline or an image/hash comparison. Text checks need a card when the text can
|
|
51
|
+
occur more than once on the page (a product name in a header and in a footer).
|
|
52
|
+
|
|
53
|
+
## When to ask (a `🚫 TODO [blocking]` block, not a guess)
|
|
54
|
+
Ask instead of choosing when ANY of these is true:
|
|
55
|
+
1. **More than one candidate** matches the description (two logos, a logo and a watermark, several images with alt text).
|
|
56
|
+
2. **The only way to find it is by position** (`.first()`, `.nth()`, "the image at the top").
|
|
57
|
+
3. **The only locator is generic**: `img`, `svg`, `img, svg`, `[alt]`, a CSS class chain.
|
|
58
|
+
4. **The area contains dynamic data**: rows, prices, dates, avatars, counts. A screenshot of it is never stable.
|
|
59
|
+
5. **It has no stable identity**: no alt, title, name or test id, so a rename or a reorder silently changes what is checked.
|
|
60
|
+
6. **The case says something different from the page** (the case says "top-right corner", the page has the logo top-left).
|
|
61
|
+
7. **It could not be opened or seen** in this run (a dialog, a file, a tab that needs data that is gone).
|
|
62
|
+
8. **A value is unknown** for a profile (no entry in the case's `## Values`), or the profiles differ where the case
|
|
63
|
+
says "same".
|
|
64
|
+
|
|
65
|
+
The question names the exact thing wanted: image, text, place or UI element. Write it like this, with the candidates
|
|
66
|
+
as evidence, so the person answers by pointing:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
- The preview shows the product logo in its top-right corner [T1].
|
|
70
|
+
- 🚫 TODO [blocking]: Which element is the logo? Candidates on example-staging: (a) the image with alt "Example" in the page header, (b) the first content image "Content #1", (c) the avatar. (candidates highlighted in .explore/targets/documents/published-flow/T1-candidates-example-staging.png)
|
|
71
|
+
- ✍️ Answer:
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
The parent sentence is the guess, so a `[blocking]` card is never turned into a check. `/gen-spec` writes
|
|
75
|
+
`test.fixme(true, '<the question>')` for it.
|
|
76
|
+
|
|
77
|
+
In an **attended** run, do not write the question first: ask the person to outline the target in the browser with
|
|
78
|
+
`browser_annotate`, then write the card from the annotation (the annotated screenshot is the Reference).
|
|
79
|
+
|
|
80
|
+
## How a card is made (exploring)
|
|
81
|
+
1. Find every candidate: `browser_find` / `browser_snapshot`, and count them. Do not stop at the first match.
|
|
82
|
+
2. Decide with the rules above: exactly one candidate with a stable identity and no dynamic data means a card;
|
|
83
|
+
anything else means a question.
|
|
84
|
+
3. Outline the target with `browser_highlight`, take the screenshot to `.explore/targets/...`, then `browser_hide_highlight`.
|
|
85
|
+
4. **Look at the screenshot** (read the image file) and say in the card what it shows. If it shows something other than
|
|
86
|
+
the card's `What` (a content image instead of a logo), the card is wrong: fix it before going on. This is the step that
|
|
87
|
+
would have caught the wrong baseline.
|
|
88
|
+
5. Do this for every profile that has an entry in the case's `## Values`. A target that differs between profiles is not
|
|
89
|
+
settled: write a `[blocking]` question instead of encoding one profile's locator as if it held for all.
|
|
90
|
+
|
|
91
|
+
## What the spec does with a card (generating)
|
|
92
|
+
- The check is written against `Identified by`, and the line before it is `// target: T<n>`. `pnpm spec:lint` fails a
|
|
93
|
+
screenshot, a logo/image locator or a position pick that has no target comment.
|
|
94
|
+
- A semantic assertion always comes before a pixel one: the element's alt/title/src or role matches, and it is where
|
|
95
|
+
`Where` says, then `toHaveScreenshot` on that element only.
|
|
96
|
+
- A card with `Not:` gets a check that the matched element is not one of those (for example it shows no item text).
|
|
97
|
+
- Screenshot only elements without dynamic data. If the target card cannot give such an element, it is a question.
|
|
98
|
+
|
|
99
|
+
## What healing does with a card
|
|
100
|
+
Before deciding "product changed" or "test is wrong", check **the assertion itself**: does the locator still point to
|
|
101
|
+
what the card's `What` says? Open the baseline and the failure screenshot and compare each with the card. A baseline
|
|
102
|
+
that shows something else is a test bug: delete it and report it (a person creates the new one). A card that no longer
|
|
103
|
+
matches the page is a question for the person, not something to rewrite quietly.
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
async (page) => {
|
|
2
|
+
// Floating "YOUR TURN" card for a handoff to a person. Do not copy, shorten or rewrite this file: load it with
|
|
3
|
+
// browser_run_code_unsafe({filename}) after setting the options (see user-handoff.md, "The wait loop").
|
|
4
|
+
const ctx = page.context();
|
|
5
|
+
const cfg = ctx.__handoffCfg || {};
|
|
6
|
+
const message = cfg.message || 'Do the step, then press Done';
|
|
7
|
+
const copy = Array.isArray(cfg.copy) ? cfg.copy : [];
|
|
8
|
+
const doneText = cfg.doneText || null;
|
|
9
|
+
const maxMs = cfg.maxMs || 600000;
|
|
10
|
+
await ctx.grantPermissions(['clipboard-read', 'clipboard-write']).catch(() => {});
|
|
11
|
+
|
|
12
|
+
const paint = (p) => p.evaluate(({message, copy}) => {
|
|
13
|
+
if (document.getElementById('__ASRUN_panel')) return;
|
|
14
|
+
const KEY = '__ASRUN_panel_state';
|
|
15
|
+
let state = {minimized: false, left: null, top: null};
|
|
16
|
+
try { state = {...state, ...JSON.parse(sessionStorage.getItem(KEY) || '{}')}; } catch (e) {}
|
|
17
|
+
const save = () => { try { sessionStorage.setItem(KEY, JSON.stringify(state)); } catch (e) {} };
|
|
18
|
+
const icon = (body) => '<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">' + body + '</svg>';
|
|
19
|
+
const ICON = {
|
|
20
|
+
copy: icon('<rect x="9" y="9" width="13" height="13" rx="2" ry="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/>'),
|
|
21
|
+
check: icon('<polyline points="20 6 9 17 4 12"/>'),
|
|
22
|
+
up: icon('<polyline points="18 15 12 9 6 15"/>'),
|
|
23
|
+
down: icon('<polyline points="6 9 12 15 18 9"/>'),
|
|
24
|
+
move: icon('<polyline points="5 9 2 12 5 15"/><polyline points="9 5 12 2 15 5"/><polyline points="15 19 12 22 9 19"/><polyline points="19 9 22 12 19 15"/><line x1="2" y1="12" x2="22" y2="12"/><line x1="12" y1="2" x2="12" y2="22"/>'),
|
|
25
|
+
};
|
|
26
|
+
const host = document.createElement('div');
|
|
27
|
+
host.id = '__ASRUN_panel';
|
|
28
|
+
host.style.cssText = 'all:initial;position:fixed;z-index:2147483647;left:16px;bottom:16px';
|
|
29
|
+
const root = host.attachShadow({mode: 'open'});
|
|
30
|
+
root.innerHTML = '<style>' +
|
|
31
|
+
'.card{width:300px;max-width:calc(100vw - 32px);background:#1e3340;color:#fff;font:13px/1.4 system-ui,sans-serif;border-radius:8px;box-shadow:0 6px 20px rgba(0,0,0,.35);overflow:hidden}' +
|
|
32
|
+
'.card.min{width:auto}.card.ok{background:#15803d}' +
|
|
33
|
+
'.bar{display:flex;align-items:center;gap:8px;padding:7px 8px 7px 12px;background:#b45309;cursor:grab;user-select:none;touch-action:none}' +
|
|
34
|
+
'.ok .bar{background:#166534;align-items:flex-start;padding-top:9px}.bar b{flex:1;min-width:0;font-size:12px;letter-spacing:.04em;white-space:nowrap}.ok .bar b{white-space:normal;line-height:1.35}.grip{display:inline-flex;opacity:.8}' +
|
|
35
|
+
'button{all:unset;box-sizing:border-box;cursor:pointer;display:inline-flex;align-items:center;justify-content:center;gap:4px;padding:3px 8px;border-radius:4px;background:rgba(255,255,255,.18);color:#fff;font:600 12px system-ui,sans-serif}' +
|
|
36
|
+
'button:hover{background:rgba(255,255,255,.3)}' +
|
|
37
|
+
'.body{padding:10px 12px 12px;display:flex;flex-direction:column;gap:8px}.min .body,.ok .body{display:none}' +
|
|
38
|
+
'.mini-done{display:none}.min .mini-done{display:inline-flex}' +
|
|
39
|
+
'.chip{display:flex;align-items:center;gap:6px;background:rgba(0,0,0,.28);padding:4px 4px 4px 8px;border-radius:4px}' +
|
|
40
|
+
'.chip span{flex:1;font:12px ui-monospace,Menlo,monospace;word-break:break-all;user-select:all}' +
|
|
41
|
+
'.row{display:flex;gap:8px}.row .primary{background:#15803d}.row .primary:hover{background:#16a34a}' +
|
|
42
|
+
'</style>' +
|
|
43
|
+
'<div class="card" id="card"><div class="bar" id="bar" title="Drag to move"><span class="grip">' + ICON.move + '</span><b id="title">YOUR TURN</b>' +
|
|
44
|
+
'<button class="mini-done" data-a="done">Done</button><button id="toggle" title="Minimize or expand" aria-label="Minimize or expand"></button></div>' +
|
|
45
|
+
'<div class="body"><div id="msg"></div><div id="chips"></div><div class="row">' +
|
|
46
|
+
'<button class="primary" data-a="done">Done</button><button data-a="skip">Cannot do this</button></div></div></div>';
|
|
47
|
+
const card = root.getElementById('card'), bar = root.getElementById('bar'), toggle = root.getElementById('toggle');
|
|
48
|
+
root.getElementById('msg').textContent = message;
|
|
49
|
+
const chips = root.getElementById('chips');
|
|
50
|
+
chips.style.cssText = 'display:' + (copy.length ? 'flex' : 'none') + ';flex-direction:column;gap:6px';
|
|
51
|
+
for (const text of copy) {
|
|
52
|
+
const chip = document.createElement('div');
|
|
53
|
+
chip.className = 'chip';
|
|
54
|
+
chip.innerHTML = '<span></span><button aria-label="Copy to clipboard" title="Copy">' + ICON.copy + '</button>';
|
|
55
|
+
chip.firstChild.textContent = text;
|
|
56
|
+
const btn = chip.lastChild;
|
|
57
|
+
btn.addEventListener('click', async () => {
|
|
58
|
+
try { await navigator.clipboard.writeText(text); } catch (e) {
|
|
59
|
+
const area = document.createElement('textarea');
|
|
60
|
+
area.value = text; area.style.cssText = 'position:fixed;opacity:0';
|
|
61
|
+
document.body.appendChild(area); area.select(); document.execCommand('copy'); area.remove();
|
|
62
|
+
}
|
|
63
|
+
btn.innerHTML = ICON.check + 'Copied';
|
|
64
|
+
setTimeout(() => { btn.innerHTML = ICON.copy; }, 1200);
|
|
65
|
+
});
|
|
66
|
+
chips.appendChild(chip);
|
|
67
|
+
}
|
|
68
|
+
const apply = () => {
|
|
69
|
+
card.classList.toggle('min', state.minimized);
|
|
70
|
+
toggle.innerHTML = state.minimized ? ICON.up : ICON.down;
|
|
71
|
+
root.getElementById('title').textContent = state.minimized ? 'YOUR TURN' : 'YOUR TURN: control is yours';
|
|
72
|
+
};
|
|
73
|
+
const place = () => {
|
|
74
|
+
if (state.left === null) return;
|
|
75
|
+
const r = host.getBoundingClientRect();
|
|
76
|
+
const left = Math.max(0, Math.min(innerWidth - Math.min(r.width, 80), state.left));
|
|
77
|
+
const top = Math.max(0, Math.min(innerHeight - 36, state.top));
|
|
78
|
+
host.style.left = left + 'px'; host.style.top = top + 'px'; host.style.bottom = 'auto';
|
|
79
|
+
};
|
|
80
|
+
root.addEventListener('mousedown', (e) => { if (e.target.closest('button')) e.preventDefault(); });
|
|
81
|
+
root.addEventListener('click', (e) => { const a = e.target.closest('[data-a]'); if (a) window.__handoff = a.dataset.a; });
|
|
82
|
+
toggle.addEventListener('click', () => { state.minimized = !state.minimized; save(); apply(); place(); });
|
|
83
|
+
let drag = null;
|
|
84
|
+
bar.addEventListener('pointerdown', (e) => {
|
|
85
|
+
if (e.target.closest('button')) return;
|
|
86
|
+
const r = host.getBoundingClientRect();
|
|
87
|
+
drag = {dx: e.clientX - r.left, dy: e.clientY - r.top};
|
|
88
|
+
bar.setPointerCapture(e.pointerId); bar.style.cursor = 'grabbing';
|
|
89
|
+
});
|
|
90
|
+
bar.addEventListener('pointermove', (e) => {
|
|
91
|
+
if (!drag) return;
|
|
92
|
+
state.left = e.clientX - drag.dx; state.top = e.clientY - drag.dy; place();
|
|
93
|
+
});
|
|
94
|
+
bar.addEventListener('pointerup', () => { drag = null; bar.style.cursor = 'grab'; save(); });
|
|
95
|
+
host.__finish = () => {
|
|
96
|
+
card.classList.remove('min'); card.classList.add('ok');
|
|
97
|
+
toggle.style.display = 'none';
|
|
98
|
+
root.getElementById('title').textContent = 'Done: control is back with Playwright. Do not touch the page.';
|
|
99
|
+
setTimeout(() => host.remove(), 2000);
|
|
100
|
+
};
|
|
101
|
+
apply();
|
|
102
|
+
document.documentElement.appendChild(host);
|
|
103
|
+
place();
|
|
104
|
+
if (!document.title.startsWith('YOUR TURN | ')) document.title = 'YOUR TURN | ' + document.title;
|
|
105
|
+
}, {message, copy}).catch(() => {});
|
|
106
|
+
|
|
107
|
+
for (const p of ctx.pages()) await p.evaluate(() => { window.__handoff = null; }).catch(() => {});
|
|
108
|
+
const started = Date.now();
|
|
109
|
+
let outcome = null;
|
|
110
|
+
while (!outcome && Date.now() - started < maxMs) {
|
|
111
|
+
for (const p of ctx.pages()) {
|
|
112
|
+
await paint(p);
|
|
113
|
+
const flag = await p.evaluate(() => window.__handoff || null).catch(() => null);
|
|
114
|
+
if (flag) outcome = flag;
|
|
115
|
+
else if (doneText && await p.getByText(doneText).first().isVisible().catch(() => false)) outcome = 'condition';
|
|
116
|
+
}
|
|
117
|
+
if (!outcome) await page.waitForTimeout(500);
|
|
118
|
+
}
|
|
119
|
+
for (const p of ctx.pages()) {
|
|
120
|
+
await p.evaluate(() => {
|
|
121
|
+
document.getElementById('__ASRUN_panel')?.__finish();
|
|
122
|
+
document.title = document.title.replace(/^YOUR TURN \| /, '');
|
|
123
|
+
}).catch(() => {});
|
|
124
|
+
}
|
|
125
|
+
await page.waitForTimeout(2300);
|
|
126
|
+
for (const p of ctx.pages()) await p.evaluate(() => sessionStorage.removeItem('__ASRUN_panel_state')).catch(() => {});
|
|
127
|
+
return {outcome: outcome || 'timeout', waitedMs: Date.now() - started};
|
|
128
|
+
}
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Open questions: write, resolve, report
|
|
2
|
+
|
|
3
|
+
Shared by `exploring-cases` and `generating-specs`. The agent never waits for an answer in the chat: it may run
|
|
4
|
+
without anybody present (a one-shot `claude -p` / `opencode run`). The case file is where the agent and the user
|
|
5
|
+
talk to each other, so nothing is lost between runs, tools or sessions.
|
|
6
|
+
|
|
7
|
+
## How a question is written in the case file
|
|
8
|
+
Every marker starts with its emoji, which is part of the marker: `🚫 TODO [blocking]`, `❓ TODO [non-blocking]`, `✍️ Answer:`, `🔍 to be confirmed by QA` and `👁️ not observed`. The web UI colors the emoji with the marker and offers an X button that deletes it.
|
|
9
|
+
Every question is a small block that hangs under a normal sentence. The sentence is the agent's suggestion, written
|
|
10
|
+
exactly as it should read in the final case. The block holds the question and an empty answer slot:
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
- <the suggested statement, a complete sentence that can stand in the case as it is>
|
|
14
|
+
- ❓ TODO [non-blocking]: <the question, one sentence>? (<evidence, e.g. seen on example-staging>)
|
|
15
|
+
- ✍️ Answer:
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
The same shape is used under a step, under a bullet of "Expected" and under the yaml block of "Values":
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
11. Delete the document with the "Delete" option of its "..." menu on the list page. [writes]
|
|
22
|
+
- 🚫 TODO [blocking]: How is the document deleted? (the same path was observed for drafts on example-staging)
|
|
23
|
+
- ✍️ Answer:
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Rules for writing it:
|
|
27
|
+
- **The suggestion is a statement, never a question or a half sentence.** "The logo is visible and not empty." is
|
|
28
|
+
right. "Suggested: visible." and "what is the logo?" are wrong. When no value is known, state the safest reading
|
|
29
|
+
(visible and not empty, the spec accepts both, no screenshot baseline is taken, ...). Never invent a concrete
|
|
30
|
+
value such as a text, a count or a color.
|
|
31
|
+
- **The suggestion is always the parent line**, so deleting the TODO block leaves a finished case. Do not put the
|
|
32
|
+
suggestion inside the TODO line, and do not write `Suggested:`.
|
|
33
|
+
- **Always leave the empty `Answer:` line** as the last line of the block, one level deeper than the `TODO` line.
|
|
34
|
+
Nothing goes after it, and there is exactly one per block.
|
|
35
|
+
- **Both the suggestion and the question may wrap.** A wrapped line is plain text (no bullet): the block ends at the
|
|
36
|
+
first blank line or at a line that is not indented deeper than the `TODO` line. Indent the block by 4 spaces under a
|
|
37
|
+
numbered step (2 under a `-` bullet) so Markdown renders it inside that step; `pnpm case:check` warns otherwise.
|
|
38
|
+
- The TODO line has the question and, when the page showed something, the evidence in parentheses. Nothing else.
|
|
39
|
+
- `[blocking]`: the spec cannot be written correctly without the answer. `[non-blocking]`: the suggestion is safe;
|
|
40
|
+
QA should confirm it later but nothing waits for it.
|
|
41
|
+
- One question per block; narrow broad questions down instead of copying them ("which profile elements?" becomes
|
|
42
|
+
"is the logo the only per-profile element in the left side bar?").
|
|
43
|
+
- A value read from a page is not a question: write it as a statement marked `(observed, 🔍 to be confirmed by QA)`.
|
|
44
|
+
It needs no TODO block; the reviewer confirms it by deleting the marker.
|
|
45
|
+
|
|
46
|
+
## How the user answers
|
|
47
|
+
Two ways, and the agent must understand both on the next run:
|
|
48
|
+
- **The suggestion is fine:** delete the whole TODO block (the `TODO` line and the `Answer:` line). The parent
|
|
49
|
+
sentence is now confirmed text.
|
|
50
|
+
- **Own answer:** write it after `Answer:`. The answer replaces the parent sentence where they disagree. The user
|
|
51
|
+
may delete the rest of the block, or leave it.
|
|
52
|
+
- An `Answer:` with no text is not an answer. A block that was left untouched stays open.
|
|
53
|
+
- An old `TODO: ...` line without a tag or without a parent suggestion (written before this format) counts as
|
|
54
|
+
`[blocking]` until it is rewritten into the shape above.
|
|
55
|
+
|
|
56
|
+
The agent folds an answer into plain text in the right place (the parent sentence, an expected bullet, an entry of
|
|
57
|
+
`## Values`, a step) and removes the `TODO` line and the `Answer:` line. If an answer contradicts the page, it keeps the
|
|
58
|
+
answer, adds `(page shows <x> on <profile-id>)` and reports it as a finding. It never overrides the user.
|
|
59
|
+
|
|
60
|
+
## What an open block means for the agent
|
|
61
|
+
- No TODO block under a sentence: the sentence is confirmed. Use it.
|
|
62
|
+
- `[non-blocking]` block: use the parent sentence as the expectation; keep the block for QA.
|
|
63
|
+
- `[blocking]` block: the parent sentence is only a guess. Do not generate a check from it: write
|
|
64
|
+
`test.fixme(true, '<the question>')` inside the step (or skip that profile).
|
|
65
|
+
- Every run re-evaluates the blocks that are already in the file against the rules below, including their tag: a
|
|
66
|
+
`[blocking]` question that now has an observation or a safe default is downgraded or closed in the file, and the
|
|
67
|
+
edit is part of the report. Do not carry old tags over.
|
|
68
|
+
|
|
69
|
+
## Resolution order (no question goes to the user while a better step exists)
|
|
70
|
+
1. **The page answers it.** Facts such as exact labels, which controls exist, where the logo is, what a menu holds
|
|
71
|
+
are answered by exploration (`/explore-case`; needs an environment and a logged-in profile; the agent only reads, a person does the steps that write). Rewrite the parent sentence
|
|
72
|
+
with the fact, mark it `(observed, 🔍 to be confirmed by QA)` and remove the block.
|
|
73
|
+
- A check with no expected value ("check X", "can see Y") becomes: the observed behaviour, marked as observed.
|
|
74
|
+
A spec may be generated from it.
|
|
75
|
+
- Values that a check asserts (logo name, label, option list) are observed per profile and written into the case's
|
|
76
|
+
`## Values`, under the profile id (or `default` when they hold for every profile).
|
|
77
|
+
2. **A default is safe.** Use the default of `gaps.md` in the `making-cases` skill (visible and not empty, all
|
|
78
|
+
environments, `suites: [regression]`, ...) as the parent sentence and keep the block as `[non-blocking]`.
|
|
79
|
+
Implementation questions are in this group: whether "visible" is enough, how long to wait, which of two equally
|
|
80
|
+
good locators for the SAME element to use. Pick the most robust option the page offers, state it, and do not
|
|
81
|
+
block on it. **Not in this group: which element a check looks at.** That is a target (`assertion-targets.md`):
|
|
82
|
+
one clear candidate becomes a card, anything else blocks.
|
|
83
|
+
3. **Otherwise it blocks.** Keep the block as `[blocking]`, narrowed as far as the page allows, with the best guess
|
|
84
|
+
as the parent sentence.
|
|
85
|
+
|
|
86
|
+
## What is always blocking (never guess, never generate around it)
|
|
87
|
+
- A step that would write data or send email while the front matter has no `writes` list (or no `writes-note`), or for an environment that `writes` does not list.
|
|
88
|
+
- A step that writes data or needs a choice, in a run that is not attended: no person can do it, so there is nothing to
|
|
89
|
+
record as a `- Recipe:` line under it.
|
|
90
|
+
- A step that cannot be performed after exploring: the path 404s, the control does not exist for any profile, or it
|
|
91
|
+
needs data that does not exist and the case does not create it.
|
|
92
|
+
- Observed behaviour that contradicts what the case states, or a profile that deviates from the others where the case
|
|
93
|
+
says "same for every profile". This may be a product bug: record it (`(observed: <what>, on <profile-id>)`),
|
|
94
|
+
do not encode the deviating value as the expectation, and say it in the report.
|
|
95
|
+
- **An assertion target that is in doubt** (`assertion-targets.md`): several candidates, only a position or a generic
|
|
96
|
+
locator to find it, dynamic data in its area, no stable identity, or the page differs from the case. The question
|
|
97
|
+
names the exact image, text, place or UI wanted and lists the candidates. Never a safe default, because a wrong
|
|
98
|
+
guess passes silently.
|
|
99
|
+
- An intent question the page cannot answer ("should this also work for agents?").
|
|
100
|
+
- No environment was given, or no profile is logged in, so a fact that needs the page cannot be observed.
|
|
101
|
+
|
|
102
|
+
## Final report
|
|
103
|
+
Report in the user's language. List every open block with its question and the parent sentence it would confirm, so
|
|
104
|
+
the user can see what deleting it accepts. The last line of the final message is exactly one of:
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
RESULT: ready
|
|
108
|
+
RESULT: needs-input
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- `ready`: no `[blocking]` question remains. Non-blocking ones may remain; list them so QA can confirm them.
|
|
112
|
+
- `needs-input`: at least one `[blocking]` question remains in the case file (list each with its parent sentence),
|
|
113
|
+
or the work could not be done (no environment, nobody logged in). Say what the user should edit in the case file,
|
|
114
|
+
then re-run.
|
|
115
|
+
|
|
116
|
+
Do not write anything after the `RESULT:` line.
|