mandrel 2.7.0 → 2.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (95) hide show
  1. package/.agents/README.md +104 -330
  2. package/.agents/agents/auditor.md +135 -0
  3. package/.agents/agents/plan-critic.md +80 -0
  4. package/.agents/audit-checklists/dependencies.md +7 -0
  5. package/.agents/audit-checklists/documentation.md +1 -0
  6. package/.agents/docs/SDLC.md +69 -162
  7. package/.agents/docs/configuration.md +159 -499
  8. package/.agents/docs/quality-gates.md +59 -180
  9. package/.agents/instructions.md +170 -295
  10. package/.agents/rules/changelog-style.md +8 -66
  11. package/.agents/rules/ci-remediation.md +65 -124
  12. package/.agents/rules/gherkin-standards.md +10 -31
  13. package/.agents/rules/git-conventions-reference.md +28 -61
  14. package/.agents/rules/git-conventions.md +1 -1
  15. package/.agents/rules/orchestration-error-handling.md +5 -15
  16. package/.agents/rules/security-baseline.md +7 -13
  17. package/.agents/rules/shell-conventions.md +4 -13
  18. package/.agents/rules/test-seams.md +2 -2
  19. package/.agents/rules/testing-standards.md +7 -17
  20. package/.agents/schemas/agentrc.schema.json +21 -0
  21. package/.agents/scripts/audit-to-stories.js +51 -0
  22. package/.agents/scripts/lib/audit-to-stories/dedupe-against-github.js +120 -55
  23. package/.agents/scripts/lib/config-settings-schema.js +32 -0
  24. package/.agents/scripts/lib/findings/semantic-issue-search.js +43 -5
  25. package/.agents/scripts/lib/observability/terse-result.js +114 -0
  26. package/.agents/scripts/lib/orchestration/complexity-gate.js +207 -0
  27. package/.agents/scripts/lib/orchestration/plan-context.js +3 -0
  28. package/.agents/scripts/lib/orchestration/single-story-close/phases/auto-merge.js +221 -8
  29. package/.agents/scripts/lib/orchestration/single-story-close/runner.js +55 -14
  30. package/.agents/scripts/lib/orchestration/story-close/emit-blocked.js +9 -3
  31. package/.agents/scripts/lib/orchestration/story-deliver-terminal.js +4 -1
  32. package/.agents/scripts/lib/orchestration/task-body-validator.js +13 -40
  33. package/.agents/scripts/lib/story-body/body-format-lints.js +215 -0
  34. package/.agents/scripts/lib/story-body/story-body.js +18 -2
  35. package/.agents/scripts/lib/templates/decomposer-prompts.js +16 -0
  36. package/.agents/scripts/providers/github/issues.js +54 -7
  37. package/.agents/scripts/providers/github/search-budget.js +124 -0
  38. package/.agents/scripts/providers/github/search-query.js +71 -0
  39. package/.agents/scripts/single-story-confirm-merge.js +14 -5
  40. package/.agents/scripts/single-story-init.js +19 -3
  41. package/.agents/scripts/sync-branch-from-base.js +9 -3
  42. package/.agents/skills/core/api-and-interface-design/SKILL.md +23 -297
  43. package/.agents/skills/core/api-and-interface-design/reference.md +76 -0
  44. package/.agents/skills/core/browser-testing-with-devtools/SKILL.md +20 -327
  45. package/.agents/skills/core/browser-testing-with-devtools/reference.md +74 -0
  46. package/.agents/skills/core/code-review-and-quality/reference.md +16 -398
  47. package/.agents/skills/core/debugging-and-error-recovery/reference.md +14 -281
  48. package/.agents/skills/core/documentation-and-adrs/reference.md +42 -338
  49. package/.agents/skills/core/git-workflow-and-versioning/SKILL.md +28 -326
  50. package/.agents/skills/core/idea-refinement/SKILL.md +67 -193
  51. package/.agents/skills/core/security-and-hardening/SKILL.md +15 -31
  52. package/.agents/skills/core/security-and-hardening/reference.md +15 -273
  53. package/.agents/skills/skills.index.json +5 -5
  54. package/.agents/skills/stack/qa/gherkin-authoring/SKILL.md +16 -222
  55. package/.agents/skills/stack/qa/gherkin-authoring/reference.md +157 -0
  56. package/.agents/skills/stack/qa/playwright/SKILL.md +0 -29
  57. package/.agents/skills/stack/qa/qa-explore-driving/SKILL.md +19 -23
  58. package/.agents/skills/stack/qa/qa-harness/SKILL.md +35 -53
  59. package/.agents/skills/stack/qa/vitest/SKILL.md +0 -29
  60. package/.agents/workflows/audit-accessibility.md +34 -108
  61. package/.agents/workflows/audit-architecture.md +35 -123
  62. package/.agents/workflows/audit-clean-code.md +19 -78
  63. package/.agents/workflows/audit-data-model.md +32 -100
  64. package/.agents/workflows/audit-dependencies.md +47 -111
  65. package/.agents/workflows/audit-devops.md +16 -83
  66. package/.agents/workflows/audit-documentation.md +46 -93
  67. package/.agents/workflows/audit-navigability.md +26 -80
  68. package/.agents/workflows/audit-performance.md +40 -106
  69. package/.agents/workflows/audit-privacy.md +17 -80
  70. package/.agents/workflows/audit-quality.md +35 -99
  71. package/.agents/workflows/audit-security.md +20 -78
  72. package/.agents/workflows/audit-seo.md +20 -98
  73. package/.agents/workflows/audit-sre.md +20 -88
  74. package/.agents/workflows/audit-to-stories.md +1 -8
  75. package/.agents/workflows/audit-ux-ui.md +17 -80
  76. package/.agents/workflows/deliver.md +54 -9
  77. package/.agents/workflows/git-cleanup.md +50 -275
  78. package/.agents/workflows/helpers/audit-lens-core.md +230 -0
  79. package/.agents/workflows/helpers/code-review.md +11 -23
  80. package/.agents/workflows/helpers/deliver-story-reference.md +114 -17
  81. package/.agents/workflows/helpers/deliver-story.md +36 -186
  82. package/.agents/workflows/helpers/qa-core.md +174 -0
  83. package/.agents/workflows/helpers/qa-run-scenario-reference.md +35 -0
  84. package/.agents/workflows/helpers/qa-run-scenario.md +11 -25
  85. package/.agents/workflows/helpers/worktree-lifecycle.md +6 -67
  86. package/.agents/workflows/mandrel-update.md +7 -13
  87. package/.agents/workflows/plan.md +44 -17
  88. package/.agents/workflows/qa-assist.md +140 -269
  89. package/.agents/workflows/qa-explore.md +125 -316
  90. package/.agents/workflows/qa-run.md +180 -380
  91. package/docs/CHANGELOG.md +30 -0
  92. package/package.json +1 -1
  93. package/.agents/workflows/helpers/audit-dual-path.md +0 -59
  94. package/.agents/workflows/helpers/audit-self-check.md +0 -70
  95. package/.agents/workflows/helpers/audit-severity-scale.md +0 -19
@@ -6,35 +6,31 @@ description: Drive Gherkin scenarios through a real browser as an agent-driven Q
6
6
 
7
7
  Execute a consumer's Gherkin `.feature` scenarios through a **real browser**
8
8
  (the chrome-devtools MCP surface), with the agent acting as the step executor
9
- and a human observing. The harness resolves the consumer's `qa` contract,
10
- **resolves a target environment**, selects a concrete scenario set, signs in
11
- via the environment's configured seam, then delegates each scenario to
12
- [`helpers/qa-run-scenario.md`](helpers/qa-run-scenario.md) which navigates
13
- **from a root** to drive each `Given/When/Then` and asserts `Then` outcomes
14
- **semantically** against the accessibility snapshot. Per-surface console and
15
- network are instrumented into structured findings; those findings are recorded
16
- as `QaLedgerItem`s on the shared session ledger under `temp/qa/` and routed —
17
- after the operator sign-off gate through the same
18
- classify/route/dedup/promote core `/qa-explore` and `/qa-assist` use, so re-run
19
- sweeps dedup previously-filed findings instead of re-drafting them. The harness
20
- never files tickets autonomously.
21
-
22
- This workflow is the agent-driven successor to the framework's earlier
23
- headless BDD runner. It is a **prose workflow**, not a Node orchestrator: the host LLM
24
- executes the procedure; deterministic Node helpers under
25
- `.agents/scripts/lib/qa/` do contract resolution, environment resolution,
26
- scenario selection, console filtering, evidence redaction, and session/ledger
27
- resolution, and the shared findings core under `.agents/scripts/lib/findings/`
28
- owns classification, dedup/route, and cluster/size/promote. The agent never
29
- invents those decisions in prose.
30
-
31
- > **When to run**: During sprint testing to exercise a targeted slice of the
9
+ and a human observing. The sweep resolves a **run envelope** — a target
10
+ environment, a concrete scenario set, and an authenticated persona session —
11
+ then delegates each scenario to
12
+ [`helpers/qa-run-scenario.md`](helpers/qa-run-scenario.md), which drives it
13
+ navigation-first and asserts `Then` outcomes semantically against the
14
+ accessibility snapshot. Per-surface console and network are captured as
15
+ structured findings, recorded as `QaLedgerItem`s on the shared session ledger,
16
+ and triaged after operator sign-off through the shared classify/route/
17
+ dedup/promote core. The harness never files tickets autonomously.
18
+
19
+ The shared machinery contract resolution + loud failure, the session & ledger
20
+ contract, redact-first, the `QaLedgerItem` shape, the triage procedure, and the
21
+ HITL write gate — lives once in [`helpers/qa-core.md`](helpers/qa-core.md); this
22
+ workflow states only the `/qa-run`-specific phases (env scope → sign-in →
23
+ drive) plus a Constraints delta. Deterministic Node helpers under
24
+ `.agents/scripts/lib/qa/` own contract, environment, and scenario resolution;
25
+ the agent never invents those decisions in prose.
26
+
27
+ > **When to run**: during sprint testing to exercise a targeted slice of the
32
28
  > acceptance suite (a feature, a tag expression, or a domain), for regression
33
29
  > passes before `/deliver`, or on demand while debugging a Story's
34
30
  > user-visible behavior in a live browser.
35
31
  >
36
- > **Persona**: `qa-engineer` · **Skills**: `stack/qa/gherkin-authoring`,
37
- > `stack/qa/playwright-bdd` (authoring reference; this harness owns execution)
32
+ > **Skills**: `stack/qa/gherkin-authoring`, `stack/qa/playwright-bdd`
33
+ > (authoring reference; this harness owns execution)
38
34
 
39
35
  ## Slash Command
40
36
 
@@ -45,33 +41,26 @@ invents those decisions in prose.
45
41
  ### Arguments
46
42
 
47
43
  Both arguments are **optional**. A bare `/qa-run` runs the interactive
48
- env-then-scope flow (Step 0.5); supplying the arguments skips the corresponding
49
- prompt.
44
+ env-then-scope flow; supplying an argument skips the corresponding prompt.
50
45
 
51
46
  | Name | Required | Shape / Example | Notes |
52
47
  | ---------- | -------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------- |
53
- | `env` | no | `local`, `staging`, `https://staging.app.example` | Selects one of the contract's `environments` (Step 0.5a). Omit to be **prompted** for the environment. A raw URL resolves by origin match. |
54
- | `selector` | no | `feature:login`, `tag:@smoke and not @wip`, `domain:billing` | Scopes the sweep to a concrete scenario set. Omit to be **prompted** for scope (Step 0.5b). One of three kinds — see below. |
55
-
56
- - **Bare `/qa-run`** → prompt for the environment (Step 0.5a), then prompt for
57
- scope (Step 0.5b).
58
- - **`/qa-run <env> <selector>`** → **skip both prompts**; resolve `<env>`
59
- directly and resolve `<selector>` directly.
48
+ | `env` | no | `local`, `staging`, `https://staging.app.example` | Selects one of the contract's `environments`. Omit to be **prompted**. A raw URL resolves by origin match. |
49
+ | `selector` | no | `feature:login`, `tag:@smoke and not @wip`, `domain:billing` | Scopes the sweep to a concrete scenario set. Omit to be **prompted** for scope. One of three kinds — see below. |
60
50
 
61
51
  The selector is resolved by
62
52
  [`resolve-selection.js`](../scripts/lib/qa/resolve-selection.js) into a
63
- deterministic, `(file, line)`-sorted scenario set under the contract's
64
- `featureRoot`. The three kinds map to that resolver's selector shapes:
65
-
66
- - **`feature:<id>`** → `{ kind: 'feature', id }` — the single `.feature` file
67
- whose `featureRoot`-relative path stem (or basename) equals the id
68
- (case-insensitive). Ambiguous ids throw; qualify with a relative path.
69
- - **`tag:<expression>`** `{ kind: 'tag', expression }` the scenario set
70
- whose tags satisfy the cucumber boolean expression (`@tag` atoms with
71
- `and` / `or` / `not` and parentheses). Quote expressions that contain
72
- spaces.
73
- - **`domain:<name>`** → `{ kind: 'domain', name }` — every scenario under the
74
- `featureRoot`-relative subdirectory `name`.
53
+ deterministic, `(file, line)`-sorted scenario set under `featureRoot`. Its
54
+ three kinds:
55
+
56
+ - **`feature:<id>`** — the single `.feature` file whose `featureRoot`-relative
57
+ path stem (or basename) equals the id (case-insensitive; ambiguous ids throw
58
+ qualify with a relative path).
59
+ - **`tag:<expression>`** the scenario set whose tags satisfy the cucumber
60
+ boolean expression (`@tag` atoms with `and` / `or` / `not` and parentheses;
61
+ quote expressions containing spaces).
62
+ - **`domain:<name>`** — every scenario under the `featureRoot`-relative
63
+ subdirectory `name`.
75
64
 
76
65
  ### Examples
77
66
 
@@ -83,365 +72,176 @@ deterministic, `(file, line)`-sorted scenario set under the contract's
83
72
  /qa-run https://staging.app.example domain:billing
84
73
  ```
85
74
 
86
- The canonical tag taxonomy `@smoke`, `@risk-high`, `@platform-web`,
87
- `@platform-mobile`, `@domain-*`, and the allowed extension syntax is defined
88
- in `.agents/rules/gherkin-standards.md`. Do not invent tags inside a feature
89
- file; add new tags to the rule first.
75
+ The canonical tag taxonomy (`@smoke`, `@risk-high`, `@platform-*`,
76
+ `@domain-*`, and the allowed extension syntax) is defined in
77
+ [`.agents/rules/gherkin-standards.md`](../rules/gherkin-standards.md). Do not
78
+ invent tags inside a feature file; add new tags to the rule first.
90
79
 
91
- ## Step 0 — Resolve the `qa` contract (fail loudly when absent)
80
+ ## Step 0 — Resolve the run envelope
92
81
 
93
- The harness is meaningless without the consumer's `qa` contract block in
94
- `.agentrc.json`. Resolve it through the single seam
95
- [`resolve-qa-contract.js`](../scripts/lib/qa/resolve-qa-contract.js) **before
96
- any browser work**:
82
+ The outcome of Steps 0–2 is a single resolved envelope **`{ environment,
83
+ scenario set, authenticated persona session }`** — that the per-scenario driver
84
+ (Step 3) consumes. The deterministic resolvers own every decision; the agent
85
+ narrates none of their internal return shapes. Every resolver **throws loudly**
86
+ on bad input, and the terminal behavior is the same in every case: **relay the
87
+ resolver's verbatim message and STOP** — never guess an environment, a
88
+ `featureRoot`, or a sign-in seam.
97
89
 
98
- ```bash
99
- node -e "import('./.agents/scripts/lib/qa/resolve-qa-contract.js').then(async (m) => { const { resolveConfig } = await import('./.agents/scripts/config-resolver.js'); const cfg = await resolveConfig(); console.log(JSON.stringify(m.resolveQaContract(cfg), null, 2)); })"
100
- ```
90
+ First resolve the `qa` contract and the session per
91
+ [`helpers/qa-core.md`](helpers/qa-core.md) (contract resolution + loud failure;
92
+ session & ledger under `temp/qa/`). Then build the rest of the envelope:
101
93
 
102
- (Use whatever config-resolution entry point the host exposes; the contract
103
- seam is `resolveQaContract(config)`.) The resolver returns the normalized
104
- contract:
105
-
106
- | Field | Use |
107
- | -------------------- | ------------------------------------------------------------------------- |
108
- | `featureRoot` | Root passed to `resolve-selection.js` for scenario discovery. |
109
- | `fixturesManifest` | Persona → seed binding loaded before sign-in. |
110
- | `environments` | Environment-keyed map (`{ baseUrl, signInSeam, allowWrites? }` per name). Resolved to a single target in Step 0.5a. |
111
- | `defaultEnvironment` | Environment name used when no `<env>` argument is supplied and the operator accepts the default. |
112
- | `personas` | Canonical object map keyed by persona name (`personaNames` lists the names). Authored as a plain name array under a `urlTemplate` seam, or as a per-persona credential/skill map under a `skill` (or credential) seam — see Step 2. |
113
- | `consoleAllowlist` | Inline benign-console patterns (default `[]`) — see Step 4. |
114
- | `designTokens` | Pointer to the token/style source for visual inspection (default `null`). |
115
-
116
- ### Loud-failure path (no `qa` block)
117
-
118
- `resolveQaContract` **throws** — there is no silent fallback to
119
- auto-detection — in three cases:
120
-
121
- - **Block absent** (no `qa` key, or an empty `qa: {}` with no harness-required
122
- fields): the error reads
123
- _"qa: this project has not bound the QA harness — add a `qa` block to
124
- .agentrc.json (featureRoot, fixturesManifest, environments, personas) before
125
- invoking the QA harness."_
126
- - **Malformed shape** (wrong-typed field, unknown field): the error names the
127
- offending field, e.g. `qa.featureRoot must be a string`.
128
- - **Missing required field**: the error names the first missing field.
129
-
130
- When you hit any of these, **STOP immediately**. Relay the resolver's
131
- verbatim message to the operator as the harness's terminal output and do not
132
- proceed to browser execution. Do not invent a `featureRoot`, do not guess a
133
- sign-in seam, and do not fall back to any retired headless BDD runner. The
134
- loud failure is the contract: a consumer that has not bound the harness has
135
- not opted into it.
136
-
137
- ### MCP availability check
94
+ ### The chrome-devtools MCP surface must be available
138
95
 
139
96
  The chrome-devtools MCP surface (`navigate_page`, `take_snapshot`, `click`,
140
97
  `fill_form`, `evaluate_script`, `wait_for`, `list_console_messages`,
141
- `list_network_requests`) is **host-provided** — it is an external runtime
142
- dependency, not in-repo code. If the host does not expose it, degrade with a
143
- clear error ("the chrome-devtools MCP server is unavailable; the QA harness
144
- requires a live browser surface") and stop. Do not attempt a headless
145
- fallback.
146
-
147
- ### Session & ledger (temp/qa/)
148
-
149
- Resolve the session and its ledger path **once**, up front, via
150
- [`qa-session.js`](../scripts/lib/qa/qa-session.js) — the same seam
151
- `/qa-explore` and `/qa-assist` use:
152
-
153
- ```js
154
- import { resolveQaSession } from '../scripts/lib/qa/qa-session.js';
155
- const { sessionId, ledgerPath, reused, untriaged } = resolveQaSession({ config });
156
- ```
157
-
158
- - The ledger is always written under **`temp/qa/<sessionId>.ndjson`**
159
- (`<tempRoot>/qa/`, resolved from `project.paths.tempRoot`). It is one
160
- `QaLedgerItem` per line (ndjson) validated against
161
- [`qa-ledger.schema.json`](../schemas/qa-ledger.schema.json). **Never** write
162
- the ledger anywhere else, and never commit it — `temp/` is gitignored per
163
- [`.agents/instructions.md` § 6](../instructions.md).
164
- - When `reused` is `true`, a prior session of the same id exists: **append**,
165
- never overwrite, and carry the `untriaged` items forward as the rolling
166
- backlog. Pass `--session-id <id>` (or `QA_SESSION_ID`) to resume a named
167
- session.
168
-
169
- The sweep's `F#` findings (Step 4) are recorded as `QaLedgerItem`s on this
170
- ledger, and Step 5 routes the ledger through the shared
171
- classify/route/dedup/promote core there is no separate `/qa-run` finding
172
- schema or draft-bundle path.
173
-
174
- ## Step 0.5 Resolve the environment, then the scope (interactive when unargued)
175
-
176
- ### Step 0.5a — Resolve the target environment
177
-
178
- Resolve which of the contract's `environments` this sweep runs against through
179
- [`resolveQaEnvironment`](../scripts/lib/qa/resolve-qa-contract.js). It returns
180
- the resolved target `{ name, baseUrl, signInSeam, allowWrites }` (with
181
- `allowWrites` defaulted to an explicit boolean `true` only for the
182
- conventional `local` environment, `false` for every other target unless the
183
- consumer opts in).
184
-
185
- - **`<env>` argument supplied** pass it straight to `resolveQaEnvironment`.
186
- An exact environment name wins; a raw URL resolves by **origin match**
187
- against each environment's `baseUrl`. An unknown name or an unmatched URL
188
- **throws** relay the resolver's message (it names the known environments)
189
- and stop; do **not** guess an environment.
190
- - **No `<env>` argument (bare `/qa-run`)** → **prompt the operator**. Enumerate
191
- every environment as `name → baseUrl` (marking `defaultEnvironment`), and ask
192
- which to run against. The operator may answer with an environment **name** or
193
- paste a **raw URL** — resolve either through `resolveQaEnvironment` (a raw
194
- URL matches by origin). If the answer resolves to nothing, **fail loudly**
195
- with the resolver's known-environments message; do not silently fall back to
196
- the default.
197
-
198
- Carry the resolved `{ name, baseUrl, allowWrites }` forward — `baseUrl` is the
199
- navigation root, and `allowWrites` drives the write guard (Step 1) and is part
200
- of the per-scenario helper's input contract (Step 3).
201
-
202
- ### Step 0.5b — Select the scope
203
-
204
- - **`<selector>` argument supplied** → parse it into the resolver's selector
205
- shape and proceed to Step 1.
206
- - **No `<selector>` argument (bare `/qa-run`)** → **prompt the operator** for
207
- scope. Enumerate the selectable scope under `featureRoot` via
208
- [`resolve-selection.js`](../scripts/lib/qa/resolve-selection.js): the
209
- available **domains** (first-level `featureRoot` subdirectories), **feature
210
- stems** (`.feature` path stems), and **canonical tags** (the tag atoms the
211
- scanner collected across the tree, per the `.agents/rules/gherkin-standards.md`
212
- taxonomy). Present them as a **multi-select** — the operator may pick any
213
- combination of domains, feature stems, and tags, which compose into the
214
- selector set. Always include an explicit final option:
215
-
216
- > **No coverage here → hand off to `/qa-explore`.** When the surface the
217
- > operator wants to test has no authored `.feature` coverage, choosing this
218
- > option ends the sweep and hands off to [`/qa-explore`](qa-explore.md) for
219
- > agent-led exploratory QA, rather than running an empty selection.
220
-
221
- If the operator picks the hand-off option, stop the sweep and route to
222
- `/qa-explore`. Otherwise resolve the multi-select into the concrete selector
223
- set and proceed to Step 1.
224
-
225
- ## Step 1 — Select the scenario set (and apply the write guard)
226
-
227
- Pass the resolved selector(s) and the contract's `featureRoot` to
228
- [`resolveSelection`](../scripts/lib/qa/resolve-selection.js). It returns
229
- `{ kind, featureRoot, files, scenarios }` where `scenarios` is the
230
- `(file, line)`-sorted set the sweep will execute. Determinism is load-bearing:
231
- re-running the same selector across sweeps scopes the identical set, so the
232
- evidence stays diffable.
233
-
234
- Load the `fixturesManifest` to resolve each persona's seed data before
235
- sign-in. If the selection is empty, report "no scenarios matched
236
- `<selector>`" and stop — an empty selection is operator error (a typo'd
237
- feature id or domain), not a passing sweep.
238
-
239
- ### `allowWrites` guardrail (non-local safety)
240
-
241
- When the resolved environment has **`allowWrites: false`**, exclude any
242
- scenario judged **mutating** from the selection before driving. Judge a
243
- scenario mutating from its **`When` steps** — a `When` that creates, updates,
244
- or deletes persisted state (submits a form that writes, deletes a record,
245
- changes a setting) is mutating; a read-only navigation/inspection `When` is
246
- not. Report the **exclusion count** ("N mutating scenarios excluded on
247
- read-only `<env>`") alongside the resolved scenario count so the operator sees
248
- what was skipped and why. The exclusion is overridable **only** by an explicit
249
- in-session operator confirmation (the operator affirms, in this session, that
250
- writes to `<env>` are acceptable) — never silently include mutating scenarios
251
- on a read-only target, and never widen `allowWrites` by editing the contract
252
- mid-sweep.
253
-
254
- ## Step 2 — Sign in via the environment's `signInSeam`
98
+ `list_network_requests`) is **host-provided** — an external runtime dependency,
99
+ not in-repo code. If the host does not expose it, degrade with a clear error
100
+ ("the chrome-devtools MCP server is unavailable; the QA harness requires a live
101
+ browser surface") and stop. Never attempt a headless fallback.
102
+
103
+ ### Step 1 — Resolve the environment, then the scope
104
+
105
+ **Environment** — resolve which of the contract's `environments` this sweep
106
+ runs against via
107
+ [`resolveQaEnvironment`](../scripts/lib/qa/resolve-qa-contract.js), yielding
108
+ `{ name, baseUrl, signInSeam, allowWrites }` (`allowWrites` defaults to an
109
+ explicit boolean — `true` only for the conventional `local` environment). When
110
+ `<env>` is supplied, pass it straight through (an exact name wins; a raw URL
111
+ matches by origin). When it is omitted (bare `/qa-run`), **prompt** the
112
+ operator, enumerating every environment as `name baseUrl` (marking
113
+ `defaultEnvironment`); they answer with a name or a raw URL. On an unknown name
114
+ or unmatched URL the resolver throws (naming the known environments) — relay it
115
+ and stop; never silently fall back to the default.
116
+
117
+ **Scope** when `<selector>` is supplied, parse it into the resolver's
118
+ selector shape. When omitted, **prompt** the operator: enumerate the selectable
119
+ scope under `featureRoot` via `resolve-selection.js` (domains, feature stems,
120
+ canonical tags) as a **multi-select** the operator composes into a selector
121
+ set. Always include a final **"No coverage here hand off to `/qa-explore`"**
122
+ option: when the surface has no authored `.feature` coverage, choosing it ends
123
+ the sweep and routes to [`/qa-explore`](qa-explore.md) rather than running an
124
+ empty selection.
125
+
126
+ **Scenario set + write guard** pass the resolved selector(s) and `featureRoot`
127
+ to [`resolveSelection`](../scripts/lib/qa/resolve-selection.js), which returns
128
+ the `(file, line)`-sorted scenario set (determinism is load-bearing: the same
129
+ selector scopes the identical set across sweeps). Load `fixturesManifest` to
130
+ resolve each persona's seed before sign-in. An empty selection is operator
131
+ error (a typo'd id or domain), not a passing sweep report "no scenarios
132
+ matched `<selector>`" and stop.
133
+
134
+ > **`allowWrites` guardrail (non-local safety).** When the resolved environment
135
+ > has `allowWrites: false`, **exclude any mutating scenario** from the selection
136
+ > before driving — judge a scenario mutating from its `When` steps (a `When`
137
+ > that creates, updates, or deletes persisted state is mutating; a read-only
138
+ > navigation/inspection `When` is not). Report the **exclusion count** ("N
139
+ > mutating scenarios excluded on read-only `<env>`") alongside the resolved
140
+ > count. The exclusion is overridable **only** by an explicit in-session
141
+ > operator confirmation — never silently include mutating scenarios on a
142
+ > read-only target, and never widen `allowWrites` by editing the contract
143
+ > mid-sweep.
144
+
145
+ ### Step 2 Sign in via the environment's `signInSeam`
255
146
 
256
147
  Sign in **once per persona** before driving that persona's scenarios, using the
257
- resolved environment's discriminated-union seam
258
- (`environment.signInSeam`, anchored on `environment.baseUrl`):
259
-
260
- - **`kind: 'url'`** — substitute `{persona}` into `template` (e.g.
261
- `/dev/sign-in-as/{persona}` `/dev/sign-in-as/admin`) and `navigate_page`
262
- to the resulting dev seam URL. The persona **name** (a `personaNames` entry)
263
- is the **sole input** the seam consumes — per-persona auth material is
264
- neither needed nor read here, so under a `urlTemplate` seam the contract is
265
- authored as a plain name array (`personas: ["athlete", "coach"]`).
266
- - **`kind: 'skill'`** invoke the named consumer skill for procedural
267
- (multi-step or non-URL) sign-in. Read the skill's `SKILL.md` and follow it.
268
-
269
- ### Credentials under a skill seam (bounded rule)
270
-
271
- Under a `skill` seam, real sign-in is permitted but **bounded**:
272
-
273
- - Real auth uses **only `credentialRef`-indirected material** — the persona's
274
- `credentialRef` names a stored credential the skill dereferences; raw
275
- passwords, tokens, or API keys are never inlined into the contract, the
276
- workflow, or chat.
277
- - **Secrets are never echoed** into chat, findings, or the ledger — do not
278
- print a credential, a session token, or a cookie value at any point.
279
- - **Captured evidence passes `redact-evidence.js`** (`redactEvidence`) before
280
- persistence, so any secret that leaks into console/network capture is
281
- scrubbed before it reaches a finding (see Step 4 and the per-scenario
282
- helper).
283
-
284
- Per-persona auth material (`credentialRef` / `signInSkill`, authored via the
285
- object-map `personas` shape) is consulted **only** under a `skill` or
286
- credential seam. Under a `urlTemplate` dev-impersonation seam the persona name
287
- is the only input, so the material is never read — author name-only personas
288
- there rather than fabricating `credentialRef`/`signInSkill` values the harness
289
- ignores. The resolver normalizes both authored shapes to one canonical object
290
- map keyed by persona name; a name-only persona resolves to an empty record.
291
-
292
- After sign-in, confirm the authenticated state with a `take_snapshot`
293
- (e.g. the user menu or persona badge is present) before driving any scenario.
294
- This confirmed authenticated session is the precondition the per-scenario
295
- helper's input contract requires (Step 3).
148
+ resolved environment's discriminated-union seam (anchored on `baseUrl`):
149
+
150
+ - **`kind: 'url'`** — substitute the persona **name** into `template` (e.g.
151
+ `/dev/sign-in-as/{persona}`) and `navigate_page` there. The name is the sole
152
+ input; under a `urlTemplate` seam the contract is authored as a plain name
153
+ array and no per-persona auth material is read.
154
+ - **`kind: 'skill'`** invoke the named consumer sign-in skill (procedural /
155
+ non-URL sign-in). Real auth uses **only `credentialRef`-indirected material**
156
+ the skill dereferences; raw passwords, tokens, or API keys are never inlined
157
+ into the contract, the workflow, or chat, and captured evidence is redacted
158
+ per [`helpers/qa-core.md`](helpers/qa-core.md) before persistence.
159
+
160
+ **Verification (the envelope's proof).** After sign-in, confirm the
161
+ authenticated state with a `take_snapshot` showing the persona badge (the user
162
+ menu / persona badge is present) before driving any scenario. This confirmed
163
+ session is the precondition the per-scenario helper re-verifies on entry.
296
164
 
297
165
  ## Step 3 — Drive each scenario via the per-scenario helper
298
166
 
299
167
  For each scenario in selection order, delegate driving, analysis, and reporting
300
- to [`helpers/qa-run-scenario.md`](helpers/qa-run-scenario.md). Pass its input
301
- contract:
168
+ to [`helpers/qa-run-scenario.md`](helpers/qa-run-scenario.md) the **one prose
169
+ home** for the driving rules (navigation-first / never URL-jump, semantic
170
+ `Then` assertion, the per-`When` write guard, mandatory evidence redaction, and
171
+ the sequential-only browser rule). Do not restate those rules here. Pass the
172
+ helper its input contract:
302
173
 
303
- - **`environment`** — the resolved `{ name, baseUrl, allowWrites }` from
304
- Step 0.5a.
174
+ - **`environment`** — the resolved `{ name, baseUrl, allowWrites }` (Step 1).
305
175
  - **`persona`** — the persona name **plus** the confirmed authenticated-session
306
- precondition established in Step 2 (the helper re-verifies it on entry).
307
- - **`scenario`** — the scenario ref (`.feature` file path and `(file, line)`
308
- locator).
176
+ precondition from Step 2 (the helper re-verifies it on entry).
177
+ - **`scenario`** — the scenario ref (`.feature` file path and `(file, line)`).
309
178
  - **`consoleAllowlist`** and **`designTokens`** — from the resolved contract.
310
179
 
311
- The helper owns the navigation-first / never-URL-jump rule, the semantic-`Then`
312
- assertion against the accessibility snapshot, the per-`When` write guard under
313
- `allowWrites: false`, and mandatory evidence redaction. It returns **one
314
- structured per-scenario result** — `{ scenario, intent, verdict (pass | fail |
315
- blocked), surface, findings[] }`. Collect one result per scenario; the sweep
316
- report shape (per-scenario `intent + verdict` lines plus totals, Step 6) is
317
- unchanged from the inlined procedure.
180
+ The helper returns **one structured per-scenario result**
181
+ `{ scenario, intent, verdict (pass | fail | blocked), surface, findings[] }`.
182
+ Collect one result per scenario for the sweep report (Step 5).
318
183
 
319
- ## Step 4 — Instrument & record findings onto the ledger
184
+ ## Step 4 — Record findings onto the ledger
320
185
 
321
186
  The per-scenario helper captures console and network per surface and turns
322
- genuine problems into structured findings, applying the contract's
187
+ genuine problems into structured `F#` findings, applying the contract's
323
188
  `consoleAllowlist` via
324
189
  [`filterConsoleMessages`](../scripts/lib/qa/console-allowlist.js) (each
325
- non-allowlisted console **error** becomes one `F#` finding; allowlisted
326
- patterns and non-error levels are suppressed) and, when `designTokens` is set,
327
- spot-checking the surface against the token source. The allowlist is a **noise
328
- filter, not a security control** — never expand it to silence a genuine error
329
- signal.
330
-
331
- Each returned `F#` finding the console/network-derived shape
332
- `{ id, classification, surface, symptom, likelyRootCause, disposition,
333
- acceptance, evidence: { console[], network[] } }`, with evidence already
334
- scrubbed of tokens, session cookies, and PII via
335
- [`redact-evidence.js`](../scripts/lib/qa/redact-evidence.js) per
336
- `.agents/rules/security-baseline.md` — is **recorded as a `QaLedgerItem`** on
337
- the session ledger resolved in Step 0 (`temp/qa/<sessionId>.ndjson`,
338
- [`qa-ledger.schema.json`](../schemas/qa-ledger.schema.json)). Map the finding
339
- onto the ledger shape: a stable `id` (`L1`, `L2`, in append order), the
340
- finding's `symptom` as the scrubbed `evidence`, the finding's `surface` as
341
- `coverage`, a `class` (map the finding's classification onto the
342
- ledger `class` enum a product defect is `product-bug`, a tooling gap is
343
- `tooling-dx`, etc.) and `severity`, `missingTest` (`null` when no test gap
344
- applies), and `disposition` left untriaged. **Append** to the ledger, never
345
- overwrite; a re-run appends to the same session. This is the single findings
346
- channel there is no separate `/qa-run` finding schema or draft bundle.
347
-
348
- ## Step 5 Triage the ledger (operator sign-off required)
349
-
350
- After the sweep completes, route the ledger through the same
351
- classify/route/dedup/promote core `/qa-explore` Triage and `/qa-assist` use.
352
- The **operator sign-off gate is preserved**: the harness MUST NOT create
353
- tickets autonomously present the routed dispositions and confirm each
354
- `file` / `defer` / `dismiss` with the operator before any write.
355
-
356
- For each untriaged ledger item:
357
-
358
- 1. **Classify** it via
359
- [`classify-finding.js`](../scripts/lib/findings/classify-finding.js). The
360
- item's `class` resolves to the focus/meta label set Triage applies when
361
- promoting it. The helper **throws** on an absent/unknown class — fix the
362
- ledger item's class rather than defaulting.
363
- 2. **Dedup / route** it against existing GitHub Issues via
364
- [`route-finding.js`](../scripts/lib/findings/route-finding.js):
365
-
366
- ```js
367
- import { routeFinding, fingerprintFooter } from '../scripts/lib/findings/route-finding.js';
368
- const { decision, matchedIssue, fingerprint } =
369
- await routeFinding(finding, { searchIssues });
370
- ```
371
-
372
- `decision` is one of `new` / `update-existing` / `duplicate` /
373
- `regression-of-closed`. Wire the `searchIssues` port to the GitHub provider,
374
- querying **both open and closed** Issues, and stamp the
375
- `fingerprintFooter(sha)` marker into any Issue body so future sweeps dedup
376
- against it. This is the **single** dedup implementation shared with
377
- `/qa-explore`, `/qa-assist`, and `audit-to-stories`.
378
- 3. **Decide the disposition** with the operator (`file` / `defer` / `dismiss`)
379
- and record it back onto the ledger item.
380
- 4. **Promote the `file`-dispositioned findings through `/plan`** via
381
- [`promote-finding.js`](../scripts/lib/findings/promote-finding.js) — the
382
- same cluster/size/route/file path `/qa-explore` and `/audit-to-stories`
383
- consume (`clusterLedgerItems` + `targetForCluster`: a cluster spanning ≤2
384
- coverage surfaces routes to `createStory` via `/plan --seed-file`, >2 to
385
- `createPlanSeed` via `/plan --seed`, with the cluster's `fingerprintFooter(sha)`
386
- stamped verbatim into the seed). A `file` disposition never opens a raw
387
- GitHub Issue; only `defer` and `dismiss` skip the `/plan` handoff.
388
- 5. **Gate:** any ticket-filing, seed write, `/plan` invocation, or label
389
- mutation is a write — confirm each with the operator before it happens.
390
-
391
- ## Step 6 — Report
392
-
393
- Summarize the sweep in chat with:
394
-
395
- - The resolved **environment** (`name → baseUrl`, and whether it is read-only)
396
- and the selector applied.
397
- - The resolved scenario count, plus the **`allowWrites` exclusion count** when
398
- mutating scenarios were skipped on a read-only environment.
399
- - Scenario totals: passed / failed / blocked.
400
- - Findings totals by classification, and the ledger routes
401
- (`new` / `update-existing` / `duplicate` / `regression-of-closed`).
402
- - A per-scenario line pairing each scenario's plain-English intent with its
403
- verdict (pass / fail / blocked), grouped by feature file or domain — so the
404
- digest reads as "what was checked → what happened", not a tag list.
405
- - For each failure, the scenario name, file path, the surface it ended on, and
406
- a one-line user-visible symptom.
407
- - The ledger path (under `temp/qa/`) and a pointer to the routed dispositions
408
- awaiting operator sign-off (if any).
190
+ non-allowlisted console **error** becomes one finding; allowlisted patterns and
191
+ non-error levels are suppressed) and spot-checking against `designTokens` when
192
+ set. The allowlist is a **noise filter, not a security control** — never expand
193
+ it to silence a genuine error signal.
194
+
195
+ Record each returned `F#` finding as a `QaLedgerItem` on the session ledger
196
+ (shape and append-never-overwrite rule per [`helpers/qa-core.md`](helpers/qa-core.md)):
197
+ map the finding's `symptom` to the redacted `evidence`, its `surface` to
198
+ `coverage`, its `classification` onto the ledger `class` enum (a product defect
199
+ is `product-bug`, a tooling gap is `tooling-dx`, …) plus a `severity`,
200
+ `missingTest` (`null` when no gap applies), and `disposition` left untriaged.
201
+
202
+ ## Step 5 Triage the ledger, then report
203
+
204
+ Route the ledger through the shared classify route disposition promote
205
+ procedure in [`helpers/qa-core.md`](helpers/qa-core.md), under its HITL write
206
+ gate: the harness MUST NOT create tickets autonomously — present the routed
207
+ dispositions and confirm each `file` / `defer` / `dismiss` before any write.
208
+
209
+ Then summarize the sweep in chat with:
210
+
211
+ - the resolved **environment** (`name baseUrl`, and whether it is read-only)
212
+ and the selector applied;
213
+ - the resolved scenario count, plus the **`allowWrites` exclusion count** when
214
+ mutating scenarios were skipped on a read-only environment;
215
+ - scenario totals (passed / failed / blocked);
216
+ - findings totals by classification and the ledger routes
217
+ (`new` / `update-existing` / `duplicate` / `regression-of-closed`);
218
+ - a per-scenario line pairing each scenario's plain-English intent with its
219
+ verdict, grouped by feature file or domain "what was checked → what
220
+ happened", not a tag list;
221
+ - for each failure: the scenario name, file path, the surface it ended on, and
222
+ a one-line user-visible symptom;
223
+ - the ledger path (under `temp/qa/`) and a pointer to any routed dispositions
224
+ awaiting operator sign-off.
409
225
 
410
226
  ## Constraints
411
227
 
412
- - **Always** resolve the `qa` contract first and **fail loudly** when it is
413
- absent or malformed. There is no auto-detection fallback.
414
- - **Always** resolve a target environment (Step 0.5a) — prompt for it when no
415
- `<env>` argument is supplied and **fail loudly** on an unknown name or an
416
- unmatched URL. Never silently fall back to the default on a bad answer.
228
+ Beyond the shared core ([`helpers/qa-core.md`](helpers/qa-core.md): contract +
229
+ loud failure, session/ledger, redact-first, QaLedgerItem, triage, HITL gate)
230
+ and the driving rules ([`helpers/qa-run-scenario.md`](helpers/qa-run-scenario.md):
231
+ navigation-first, semantic `Then`, redaction, sequential-only), the
232
+ `/qa-run`-specific deltas are:
233
+
234
+ - **Always** resolve a target environment (Step 1) — prompt when no `<env>` is
235
+ supplied — and **fail loudly** on an unknown name or unmatched URL; never
236
+ silently fall back to the default.
417
237
  - **Always** apply the `allowWrites` guardrail: on a read-only environment,
418
238
  exclude mutating scenarios and report the exclusion count; include them only
419
239
  on explicit in-session operator confirmation.
420
- - **Always** navigate from a root via UI affordances. **Never** URL-jump to a
421
- deep link to set up a scenario.
422
- - **Always** assert `Then` outcomes semantically against the accessibility
423
- snapshot. **Never** assert via DOM/CSS/XPath selectors, HTTP status codes,
424
- response bodies, or DB rows inside a scenario — push those to the contract
425
- tier per `.agents/rules/testing-standards.md`.
426
- - **Under a skill seam**, real sign-in uses only `credentialRef`-indirected
427
- material; secrets are never echoed into chat, findings, or the ledger; and
428
- captured evidence passes `redact-evidence.js` before persistence.
429
- - **Always** record findings as `QaLedgerItem`s on the shared session ledger
430
- under `temp/qa/` ([`qa-ledger.schema.json`](../schemas/qa-ledger.schema.json))
431
- and route them through the shared classify/route/dedup/promote core — there
432
- is no separate `/qa-run` finding schema or draft-bundle path. Never commit
433
- the ledger.
434
- - **Delegate Triage decisions to the helpers.** Classification
435
- ([`classify-finding.js`](../scripts/lib/findings/classify-finding.js)),
436
- dedup/route ([`route-finding.js`](../scripts/lib/findings/route-finding.js),
437
- fingerprint-footer against open + closed issues), and cluster/size/promote
438
- ([`promote-finding.js`](../scripts/lib/findings/promote-finding.js)) are
439
- deterministic — never re-derive them in prose.
440
- - **Never** file follow-up tickets autonomously; promote `file` findings
441
- through `/plan` only after operator sign-off, and never open a raw GitHub
442
- Issue for a `file` finding.
443
- - **Never** expand `consoleAllowlist` to suppress genuine error signal — it is
444
- a benign-noise filter, not a security control.
445
- - **Always** scrub captured evidence of secrets and PII before rendering a
446
- finding.
447
- - **Never** fall back to a retired headless BDD-runner workflow.
240
+ - **Always** sign in per persona through the environment's `signInSeam` and
241
+ confirm the authenticated state with a post-sign-in `take_snapshot` before
242
+ driving.
243
+ - An **empty selection is operator error**, not a passing sweep — report it and
244
+ stop.
245
+ - **Never** expand `consoleAllowlist` to suppress genuine error signal.
246
+ - **Never** file follow-up tickets autonomously, and **never** fall back to a
247
+ retired headless BDD runner.