mandrel 2.7.0 → 2.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/README.md +104 -330
- package/.agents/agents/auditor.md +135 -0
- package/.agents/agents/plan-critic.md +80 -0
- package/.agents/audit-checklists/dependencies.md +7 -0
- package/.agents/audit-checklists/documentation.md +1 -0
- package/.agents/docs/SDLC.md +69 -162
- package/.agents/docs/configuration.md +133 -499
- package/.agents/docs/quality-gates.md +59 -180
- package/.agents/instructions.md +170 -295
- package/.agents/rules/changelog-style.md +8 -66
- package/.agents/rules/ci-remediation.md +65 -124
- package/.agents/rules/gherkin-standards.md +10 -31
- package/.agents/rules/git-conventions-reference.md +28 -61
- package/.agents/rules/git-conventions.md +1 -1
- package/.agents/rules/orchestration-error-handling.md +5 -15
- package/.agents/rules/security-baseline.md +7 -13
- package/.agents/rules/shell-conventions.md +4 -13
- package/.agents/rules/test-seams.md +2 -2
- package/.agents/rules/testing-standards.md +7 -17
- package/.agents/skills/core/api-and-interface-design/SKILL.md +23 -297
- package/.agents/skills/core/api-and-interface-design/reference.md +76 -0
- package/.agents/skills/core/browser-testing-with-devtools/SKILL.md +20 -327
- package/.agents/skills/core/browser-testing-with-devtools/reference.md +74 -0
- package/.agents/skills/core/code-review-and-quality/reference.md +16 -398
- package/.agents/skills/core/debugging-and-error-recovery/reference.md +14 -281
- package/.agents/skills/core/documentation-and-adrs/reference.md +42 -338
- package/.agents/skills/core/git-workflow-and-versioning/SKILL.md +28 -326
- package/.agents/skills/core/idea-refinement/SKILL.md +67 -193
- package/.agents/skills/core/security-and-hardening/SKILL.md +15 -31
- package/.agents/skills/core/security-and-hardening/reference.md +15 -273
- package/.agents/skills/skills.index.json +5 -5
- package/.agents/skills/stack/qa/gherkin-authoring/SKILL.md +16 -222
- package/.agents/skills/stack/qa/gherkin-authoring/reference.md +157 -0
- package/.agents/skills/stack/qa/playwright/SKILL.md +0 -29
- package/.agents/skills/stack/qa/qa-explore-driving/SKILL.md +19 -23
- package/.agents/skills/stack/qa/qa-harness/SKILL.md +35 -53
- package/.agents/skills/stack/qa/vitest/SKILL.md +0 -29
- package/.agents/workflows/audit-accessibility.md +34 -108
- package/.agents/workflows/audit-architecture.md +35 -123
- package/.agents/workflows/audit-clean-code.md +19 -78
- package/.agents/workflows/audit-data-model.md +32 -100
- package/.agents/workflows/audit-dependencies.md +47 -111
- package/.agents/workflows/audit-devops.md +16 -83
- package/.agents/workflows/audit-documentation.md +46 -93
- package/.agents/workflows/audit-navigability.md +26 -80
- package/.agents/workflows/audit-performance.md +40 -106
- package/.agents/workflows/audit-privacy.md +17 -80
- package/.agents/workflows/audit-quality.md +35 -99
- package/.agents/workflows/audit-security.md +20 -78
- package/.agents/workflows/audit-seo.md +20 -98
- package/.agents/workflows/audit-sre.md +20 -88
- package/.agents/workflows/audit-to-stories.md +1 -8
- package/.agents/workflows/audit-ux-ui.md +17 -80
- package/.agents/workflows/deliver.md +54 -9
- package/.agents/workflows/git-cleanup.md +50 -275
- package/.agents/workflows/helpers/audit-lens-core.md +230 -0
- package/.agents/workflows/helpers/code-review.md +11 -23
- package/.agents/workflows/helpers/deliver-story-reference.md +114 -17
- package/.agents/workflows/helpers/deliver-story.md +26 -186
- package/.agents/workflows/helpers/qa-core.md +174 -0
- package/.agents/workflows/helpers/qa-run-scenario-reference.md +35 -0
- package/.agents/workflows/helpers/qa-run-scenario.md +11 -25
- package/.agents/workflows/helpers/worktree-lifecycle.md +6 -67
- package/.agents/workflows/mandrel-update.md +7 -13
- package/.agents/workflows/plan.md +17 -15
- package/.agents/workflows/qa-assist.md +140 -269
- package/.agents/workflows/qa-explore.md +125 -316
- package/.agents/workflows/qa-run.md +180 -380
- package/docs/CHANGELOG.md +10 -0
- package/package.json +1 -1
- package/.agents/workflows/helpers/audit-dual-path.md +0 -59
- package/.agents/workflows/helpers/audit-self-check.md +0 -70
- package/.agents/workflows/helpers/audit-severity-scale.md +0 -19
|
@@ -16,64 +16,44 @@ follow-up dispositions.
|
|
|
16
16
|
This is the **agent-led** front-end of exploratory QA: **the agent drives, the
|
|
17
17
|
operator watches and gates.** Its human-led sibling is
|
|
18
18
|
[`/qa-assist`](qa-assist.md) — there the *human* drives a single observation and
|
|
19
|
-
the agent scribes/enriches. No human-driven flow lives in `/qa-explore
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
resolution, redaction, coverage verdict, missing-test proposal, classification,
|
|
33
|
-
and dedup/route decisions. The agent never invents those decisions in prose.
|
|
19
|
+
the agent scribes/enriches. No human-driven flow lives in `/qa-explore`.
|
|
20
|
+
|
|
21
|
+
Unlike [`/qa-run`](qa-run.md) (which steps a known set of Gherkin `.feature`
|
|
22
|
+
scenarios through a browser), `/qa-explore` is **open-ended exploration**: the
|
|
23
|
+
agent probes the surface for product bugs, environment-setup friction,
|
|
24
|
+
tooling/DX gaps, missing tests, and enhancement ideas — each captured as a
|
|
25
|
+
`QaLedgerItem`.
|
|
26
|
+
|
|
27
|
+
The shared machinery — contract resolution + loud failure, the session & ledger
|
|
28
|
+
contract, redact-first, the `QaLedgerItem` shape, the triage procedure, and the
|
|
29
|
+
HITL write gate — lives once in [`helpers/qa-core.md`](helpers/qa-core.md); this
|
|
30
|
+
workflow states only the `/qa-explore`-specific phases (Plan / Capture) plus a
|
|
31
|
+
Constraints delta.
|
|
34
32
|
|
|
35
33
|
> **When to run**: ad-hoc agent-driven exploration of a freshly delivered Story
|
|
36
|
-
> or Feature, a regression sweep over a risky surface before `/deliver`, or
|
|
37
|
-
>
|
|
38
|
-
> triageable ledger.
|
|
34
|
+
> or Feature, a regression sweep over a risky surface before `/deliver`, or a
|
|
35
|
+
> structured agent-driven bug-hunt captured into a triageable ledger.
|
|
39
36
|
>
|
|
40
37
|
> **Skills**: `core/qa-coverage-mapping`, `stack/qa/qa-explore-driving`
|
|
41
38
|
|
|
42
39
|
## Role framing
|
|
43
40
|
|
|
44
41
|
You are the quality gatekeeper for this run: value coverage, hermetic
|
|
45
|
-
environments, and deterministic results. Do **not** invent signal — capture
|
|
46
|
-
|
|
47
|
-
persona pack.
|
|
42
|
+
environments, and deterministic results. Do **not** invent signal — capture what
|
|
43
|
+
the surface shows. Apply the QA skills below; there is no separate persona pack.
|
|
48
44
|
|
|
49
|
-
## Driving conventions
|
|
45
|
+
## Driving conventions
|
|
50
46
|
|
|
51
47
|
Before you drive a surface, read the
|
|
52
48
|
[`stack/qa/qa-explore-driving`](../skills/stack/qa/qa-explore-driving/SKILL.md)
|
|
53
|
-
skill
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
target environment, walk the surface from source, route definitions, and
|
|
62
|
-
rendered markup — chosen explicitly at Plan time, never as a silent fallback.
|
|
63
|
-
- **Authenticated driving follows the per-environment seam.** When the resolved
|
|
64
|
-
target environment carries a `signInSeam` — a dev `url` seam or a `skill` seam
|
|
65
|
-
with `credentialRef`-indirected sign-in — drive the authenticated surface via
|
|
66
|
-
that seam, including authenticated deployed hosts. Real credentials are never
|
|
67
|
-
typed inline: the seam consumes a persona **name** (`url`) or a stored
|
|
68
|
-
`credentialRef` (`skill`), and every evidence string passes through mandatory
|
|
69
|
-
redaction. Static is the interim **only** where the target environment
|
|
70
|
-
resolves no seam.
|
|
71
|
-
- **Broken navigation is a finding, not a workaround.** A missing affordance, a
|
|
72
|
-
nav 404, or a guard redirect loop is recorded and you move on — you do not
|
|
73
|
-
route around it with a direct URL.
|
|
74
|
-
|
|
75
|
-
The driving method (drive vs. static) is a **Plan-phase decision recorded in the
|
|
76
|
-
ledger**; do not switch methods mid-surface without a new Plan note.
|
|
49
|
+
skill — the **one** conventions reference for the *how* of agent-driven
|
|
50
|
+
exploration (navigation-first driving as the default; static driving as the
|
|
51
|
+
documented interim chosen at Plan time only where no seam resolves;
|
|
52
|
+
authenticated driving through the resolved environment's `signInSeam`; broken
|
|
53
|
+
navigation is a finding, not a workaround). The driving method (drive vs.
|
|
54
|
+
static) is a **Plan-phase decision recorded in the ledger**; do not switch
|
|
55
|
+
methods mid-surface without a new Plan note. Do not restate these conventions
|
|
56
|
+
inline — the skill owns them.
|
|
77
57
|
|
|
78
58
|
## Slash Command
|
|
79
59
|
|
|
@@ -87,45 +67,17 @@ ledger**; do not switch methods mid-surface without a new Plan note.
|
|
|
87
67
|
| --------- | -------- | ---------------------------------- | -------------------------------------------------------------------------------------- |
|
|
88
68
|
| `surface` | yes | `feature:login`, `area:onboarding` | A human label for the single surface to explore. Recorded as each ledger item's `coverage`. |
|
|
89
69
|
|
|
90
|
-
If no `surface` is supplied, **stop and ask** the operator to name one — do
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
## Project contract
|
|
96
|
-
|
|
97
|
-
Resolve the consumer's `qa` contract before exploring, via
|
|
98
|
-
[`resolve-qa-contract.js`](../scripts/lib/qa/resolve-qa-contract.js):
|
|
99
|
-
|
|
100
|
-
```js
|
|
101
|
-
import { resolveQaContract } from '../scripts/lib/qa/resolve-qa-contract.js';
|
|
102
|
-
const contract = resolveQaContract(config); // throws loudly if unbound
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
The resolver fails **loudly** when the project has not bound the QA harness
|
|
106
|
-
(no `qa` block in `.agentrc.json`) — there is no silent fallback. If it throws
|
|
107
|
-
the "this project has not bound the QA harness" message, surface that verbatim
|
|
108
|
-
to the operator and stop; do not pretend a contract exists.
|
|
109
|
-
|
|
110
|
-
## Session & ledger (temp/qa/)
|
|
111
|
-
|
|
112
|
-
Resolve the session and its ledger path **once**, up front, via
|
|
113
|
-
[`qa-session.js`](../scripts/lib/qa/qa-session.js):
|
|
70
|
+
If no `surface` is supplied, **stop and ask** the operator to name one — do not
|
|
71
|
+
invent scope. `/qa-explore` is **bounded to one surface per session**: explore
|
|
72
|
+
exactly the named surface, do not wander into adjacent surfaces, and start a
|
|
73
|
+
fresh session for a different surface.
|
|
114
74
|
|
|
115
|
-
|
|
116
|
-
import { resolveQaSession } from '../scripts/lib/qa/qa-session.js';
|
|
117
|
-
const { sessionId, ledgerPath, reused, untriaged } = resolveQaSession({ config });
|
|
118
|
-
```
|
|
75
|
+
## Contract & session
|
|
119
76
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
[`.agents/instructions.md` § 6](../instructions.md).
|
|
125
|
-
- When `reused` is `true`, a prior session of the same id exists: **append**,
|
|
126
|
-
never overwrite, and carry the `untriaged` items forward as the rolling
|
|
127
|
-
backlog (resume safety). Pass `--session-id <id>` (or `QA_SESSION_ID`) to
|
|
128
|
-
resume a named session.
|
|
77
|
+
Resolve the `qa` contract and the session (under `temp/qa/`) per
|
|
78
|
+
[`helpers/qa-core.md`](helpers/qa-core.md) before exploring — the resolver fails
|
|
79
|
+
**loudly** when the project has not bound the harness; surface that verbatim and
|
|
80
|
+
stop. A reused session appends and carries its `untriaged` backlog forward.
|
|
129
81
|
|
|
130
82
|
## Bounded per-surface session
|
|
131
83
|
|
|
@@ -133,9 +85,9 @@ A `/qa-explore` run is a **single bounded session over one named surface**, not
|
|
|
133
85
|
an open-ended sweep:
|
|
134
86
|
|
|
135
87
|
- **One surface.** The session explores exactly the `surface` argument. Driving
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
88
|
+
it may legitimately touch sub-surfaces reachable navigation-first, but the
|
|
89
|
+
session does not pivot to a different top-level surface — that is a new
|
|
90
|
+
session.
|
|
139
91
|
- **Bounded by the operator's gate.** Capture continues until the operator says
|
|
140
92
|
exploration is complete (the Capture → Triage gate), not until the agent has
|
|
141
93
|
exhausted the app. The agent proposes when it believes the surface is
|
|
@@ -145,12 +97,11 @@ an open-ended sweep:
|
|
|
145
97
|
|
|
146
98
|
## Phase gates (HITL)
|
|
147
99
|
|
|
148
|
-
Every phase transition is gated on **explicit operator confirmation
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
advances phases autonomously. If the operator does not confirm, stop and hold.
|
|
100
|
+
Every phase transition is gated on **explicit operator confirmation** (the HITL
|
|
101
|
+
write gate in [`helpers/qa-core.md`](helpers/qa-core.md)). Do not advance
|
|
102
|
+
Plan → Capture, or Capture → Triage, until the operator says so. State each gate
|
|
103
|
+
as a question, present the artifact (the plan with its chosen driving method,
|
|
104
|
+
then the captured ledger), and wait. If the operator does not confirm, hold.
|
|
154
105
|
|
|
155
106
|
---
|
|
156
107
|
|
|
@@ -159,57 +110,36 @@ advances phases autonomously. If the operator does not confirm, stop and hold.
|
|
|
159
110
|
Goal: agree on **what** will be explored and **how the agent will drive it**
|
|
160
111
|
before touching the surface.
|
|
161
112
|
|
|
162
|
-
1. Re-read the
|
|
163
|
-
|
|
164
|
-
skill, and resolve the `qa` contract and session (above).
|
|
113
|
+
1. Re-read the `stack/qa/qa-explore-driving` skill and resolve the contract and
|
|
114
|
+
session (above).
|
|
165
115
|
2. **Resolve the target environment** via
|
|
166
|
-
[`resolveQaEnvironment`](../scripts/lib/qa/resolve-qa-contract.js)
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
// `target` is an environment name or a raw URL; omit for the default.
|
|
174
|
-
const environment = resolveQaEnvironment(contract, target);
|
|
175
|
-
// → { name, baseUrl, signInSeam, allowWrites }
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
When the operator's `surface` does not pin an unambiguous target and the
|
|
179
|
-
contract declares more than one environment, **prompt** the operator to name
|
|
180
|
-
the environment (or accept the `defaultEnvironment`) — never silently pick
|
|
181
|
-
one. The resolver throws loudly (naming the known environments) on an unknown
|
|
182
|
-
name or an unmatched URL; surface that verbatim and stop. Record the resolved
|
|
183
|
-
**environment name** on the ledger alongside the driving method.
|
|
116
|
+
[`resolveQaEnvironment`](../scripts/lib/qa/resolve-qa-contract.js) — it keys
|
|
117
|
+
each deployment target to `{ name, baseUrl, signInSeam, allowWrites }`. When
|
|
118
|
+
the operator's `surface` does not pin an unambiguous target and the contract
|
|
119
|
+
declares more than one environment, **prompt** the operator (or accept
|
|
120
|
+
`defaultEnvironment`) — never silently pick one. The resolver throws loudly
|
|
121
|
+
(naming the known environments) on an unknown name or unmatched URL; surface
|
|
122
|
+
that verbatim and stop.
|
|
184
123
|
3. **Choose the driving method explicitly** for the named `surface` on the
|
|
185
124
|
resolved environment:
|
|
186
|
-
- **Drive (default):** a seam resolves
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
the
|
|
198
|
-
`environment: staging, method: drive, seam: skill` or
|
|
125
|
+
- **Drive (default):** a seam resolves, so drive through the browser MCP
|
|
126
|
+
navigation-first — including authenticated deployed hosts reached via a
|
|
127
|
+
`skill` seam (a stored `credentialRef` read by the named sign-in skill,
|
|
128
|
+
never a hand-typed secret) or a dev `url` seam (persona name substituted).
|
|
129
|
+
- **Static (documented interim):** **no seam resolves**, so walk the surface
|
|
130
|
+
from source, routes, and rendered markup — a deliberate Plan-time decision
|
|
131
|
+
with a recorded reason, never a silent fallback.
|
|
132
|
+
4. Draft an **exploration plan**: the resolved target environment (name +
|
|
133
|
+
`baseUrl`), the sub-surfaces / flows / states to drive, the classes of signal
|
|
134
|
+
being hunted (the [ledger `class` enum](../schemas/qa-ledger.schema.json)),
|
|
135
|
+
the chosen method and its rationale, and any rolling backlog carried forward.
|
|
136
|
+
Record the resolved **environment name**, the chosen method, and the reason on
|
|
137
|
+
the ledger (e.g. `environment: staging, method: drive, seam: skill` or
|
|
199
138
|
`environment: preview, method: static, reason: no seam resolves`).
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
tooling-dx, test-gap, enhancement — the
|
|
205
|
-
[ledger `class` enum](../schemas/qa-ledger.schema.json)),
|
|
206
|
-
- the chosen driving method and its rationale,
|
|
207
|
-
- any rolling backlog (`untriaged`) carried forward from a resumed session.
|
|
208
|
-
5. Present the plan, the resolved target environment, the chosen driving
|
|
209
|
-
method, and the resolved `ledgerPath` (under `temp/qa/`) to the operator.
|
|
210
|
-
6. **Gate:** ask the operator to confirm the plan, the target environment, and
|
|
211
|
-
the driving method (or amend the surface/scope/environment/method). Do
|
|
212
|
-
**not** proceed to Capture until they confirm.
|
|
139
|
+
5. Present the plan, the resolved environment, the chosen method, and the
|
|
140
|
+
resolved `ledgerPath` to the operator.
|
|
141
|
+
6. **Gate:** ask the operator to confirm the plan, the environment, and the
|
|
142
|
+
method (or amend). Do **not** proceed to Capture until they confirm.
|
|
213
143
|
|
|
214
144
|
---
|
|
215
145
|
|
|
@@ -220,73 +150,42 @@ observations. **This phase is strictly read-only.**
|
|
|
220
150
|
|
|
221
151
|
> **Read-only invariant.** The agent observes; it never mutates. Per
|
|
222
152
|
> [`stack/qa/qa-explore-driving`](../skills/stack/qa/qa-explore-driving/SKILL.md)
|
|
223
|
-
> § 3 (inviolable per
|
|
224
|
-
>
|
|
225
|
-
>
|
|
226
|
-
>
|
|
227
|
-
>
|
|
228
|
-
> `temp/qa/<sessionId>.ndjson`** — session scratch, not a repository or product
|
|
229
|
-
> mutation. Any action that would change state belongs in Triage (and only
|
|
230
|
-
> after the operator confirms). When a surface's only path forward is a
|
|
153
|
+
> § 3 (inviolable per [`security-baseline.md`](../rules/security-baseline.md)),
|
|
154
|
+
> do **not** edit source, run write commands, file or label GitHub issues,
|
|
155
|
+
> change tickets, submit destructive forms, or alter the product under test. The
|
|
156
|
+
> only write Capture performs is **appending ledger lines to
|
|
157
|
+
> `temp/qa/<sessionId>.ndjson`**. When a surface's only path forward is a
|
|
231
158
|
> mutating action, record the boundary as the finding and stop — do not cross
|
|
232
159
|
> it.
|
|
233
160
|
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
sign in through the resolved environment's `signInSeam` — a dev `url` seam
|
|
242
|
-
(substitute the persona name into the template) or a `skill` seam (invoke the
|
|
243
|
-
named sign-in skill, which reads a stored `credentialRef`). Never type real
|
|
244
|
-
credentials inline and never fabricate a session; the seam is the only path
|
|
245
|
-
to a logged-in surface, and all captured evidence is redacted (§ 1 below).
|
|
246
|
-
- **Static (documented interim):** walk the surface from source, route
|
|
247
|
-
definitions, and rendered markup. Treat its coverage as partial and say so in
|
|
248
|
-
the ledger — a static pass does not close the same coverage a driven pass
|
|
249
|
-
would.
|
|
161
|
+
Drive the surface using the method chosen in Plan (per the driving-conventions
|
|
162
|
+
skill): navigation-first through the browser MCP for **drive**, signing in
|
|
163
|
+
through the resolved environment's `signInSeam`; or walking source, routes, and
|
|
164
|
+
rendered markup for **static** (treat its coverage as partial and say so in the
|
|
165
|
+
ledger). Never URL-jump to establish a starting state, and reach an
|
|
166
|
+
authenticated surface only through the resolved environment's `signInSeam`:
|
|
167
|
+
**Never type real credentials inline** and never fabricate a session.
|
|
250
168
|
|
|
251
169
|
For each observation the agent makes while driving:
|
|
252
170
|
|
|
253
|
-
1. **Redact first
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
```js
|
|
257
|
-
import { redactEvidence } from '../scripts/lib/qa/redact-evidence.js';
|
|
258
|
-
const evidence = redactEvidence(rawObservation);
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
This is mandatory per [`security-baseline.md`](../rules/security-baseline.md)
|
|
262
|
-
(§ Data Leakage & Logging, § Secrets Management) — bearer tokens, session
|
|
263
|
-
cookies, Authorization headers, and emails are masked. The pass is
|
|
264
|
-
idempotent, so redact eagerly; captured console and network evidence is
|
|
265
|
-
untrusted until scrubbed.
|
|
266
|
-
|
|
171
|
+
1. **Redact first** (per [`helpers/qa-core.md`](helpers/qa-core.md)) — scrub the
|
|
172
|
+
evidence string through `redactEvidence` before it touches disk.
|
|
267
173
|
2. **Compute the coverage verdict** for the surface the observation points at,
|
|
268
174
|
via [`coverage-verdict.js`](../scripts/lib/qa/coverage-verdict.js) — the
|
|
269
175
|
deterministic seam behind the
|
|
270
176
|
[`core/qa-coverage-mapping`](../skills/core/qa-coverage-mapping/SKILL.md)
|
|
271
|
-
skill. Read that skill for how to assemble the `surface` input
|
|
272
|
-
|
|
273
|
-
`{present|absent}` verdict.
|
|
274
|
-
|
|
177
|
+
skill. Read that skill for how to assemble the `surface` input and read the
|
|
178
|
+
per-tier `{present|absent}` verdict.
|
|
275
179
|
3. **Propose the missing test** (if any) from that verdict, via
|
|
276
|
-
[`propose-missing-test.js`](../scripts/lib/qa/propose-missing-test.js)
|
|
277
|
-
names the lowest absent tier
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
`id` (`L1`, `L2`, … in capture order), the redacted `evidence`, the
|
|
284
|
-
`coverage` label (the `surface`, or `unknown`), a tentative `class` and
|
|
285
|
-
`severity`, the `missingTest`, and `disposition` left untriaged for now.
|
|
286
|
-
|
|
180
|
+
[`propose-missing-test.js`](../scripts/lib/qa/propose-missing-test.js) — it
|
|
181
|
+
names the lowest absent tier, or returns `null` when every tier is covered.
|
|
182
|
+
Record its `description` as the ledger item's `missingTest` (or `null`).
|
|
183
|
+
4. **Append a `QaLedgerItem`** to the ledger (shape per
|
|
184
|
+
[`helpers/qa-core.md`](helpers/qa-core.md)): a stable `id`, the redacted
|
|
185
|
+
`evidence`, the `coverage` label (the `surface`, or `unknown`), a tentative
|
|
186
|
+
`class` and `severity`, the `missingTest`, and `disposition` left untriaged.
|
|
287
187
|
5. Continue driving until the agent believes the surface is covered, then
|
|
288
188
|
propose that exploration is complete.
|
|
289
|
-
|
|
290
189
|
6. **Gate:** present the captured ledger (item count, classes, the driving
|
|
291
190
|
method used, the rolling backlog) and ask the operator to confirm moving to
|
|
292
191
|
Triage. Do **not** triage until they confirm.
|
|
@@ -295,144 +194,54 @@ For each observation the agent makes while driving:
|
|
|
295
194
|
|
|
296
195
|
## Phase 3 — Triage
|
|
297
196
|
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
[`classify-finding.js`](../scripts/lib/findings/classify-finding.js). The
|
|
305
|
-
item's `class` resolves to the focus/meta label set Triage applies when
|
|
306
|
-
promoting it (`tooling-dx` carries `meta::framework-gap`; `enhancement`
|
|
307
|
-
carries `meta::consumer-improvement`). The helper **throws** on an
|
|
308
|
-
absent/unknown class — fix the ledger item's class rather than defaulting.
|
|
309
|
-
|
|
310
|
-
2. **Dedup / route** it against existing GitHub Issues via
|
|
311
|
-
[`route-finding.js`](../scripts/lib/findings/route-finding.js):
|
|
312
|
-
|
|
313
|
-
```js
|
|
314
|
-
import { routeFinding, fingerprintFooter } from '../scripts/lib/findings/route-finding.js';
|
|
315
|
-
const { decision, matchedIssue, fingerprint } =
|
|
316
|
-
await routeFinding(finding, { searchIssues });
|
|
317
|
-
```
|
|
318
|
-
|
|
319
|
-
`decision` is one of `new` / `update-existing` / `duplicate` /
|
|
320
|
-
`regression-of-closed`. This is the **single** dedup implementation shared
|
|
321
|
-
with `/qa-assist` and `audit-to-stories`; stamp the `fingerprintFooter(sha)`
|
|
322
|
-
marker into any Issue body so future runs dedup against it. Wire the
|
|
323
|
-
`searchIssues` port to the GitHub provider (querying both open and closed
|
|
324
|
-
Issues).
|
|
325
|
-
|
|
326
|
-
3. **Decide the disposition** with the operator: `file` (promote through
|
|
327
|
-
`/plan` — never a raw GitHub Issue), `defer` (carry forward to a later
|
|
328
|
-
session as backlog), or `dismiss` (non-actionable). Record the chosen
|
|
329
|
-
`disposition` back onto the ledger item.
|
|
330
|
-
|
|
331
|
-
4. **Promote the `file`-dispositioned findings through `/plan`** via
|
|
332
|
-
[`promote-finding.js`](../scripts/lib/findings/promote-finding.js) — the
|
|
333
|
-
same cluster/size/route/file path `/qa-assist` and `/audit-to-stories`
|
|
334
|
-
consume. Never hand-roll the clustering, sizing, or promotion in prose:
|
|
335
|
-
|
|
336
|
-
```js
|
|
337
|
-
import { promoteFindings } from '../scripts/lib/findings/promote-finding.js';
|
|
338
|
-
const { promotions } = await promoteFindings(ledgerItems, {
|
|
339
|
-
searchIssues, // GitHub provider, open + closed
|
|
340
|
-
createStory, // tight cluster (≤2 surfaces): seed → /plan --seed-file
|
|
341
|
-
createPlanSeed, // broad cluster (>2 surfaces): same /plan --seed-file path (may N>1)
|
|
342
|
-
});
|
|
343
|
-
```
|
|
344
|
-
|
|
345
|
-
- **Sizing is delegated, not decided in prose.** `promoteFindings` runs
|
|
346
|
-
`clusterLedgerItems` + `targetForCluster`: a cluster spanning **≤2**
|
|
347
|
-
distinct coverage surfaces routes to `createStory`; **>2** routes to
|
|
348
|
-
`createPlanSeed`. Neither port opens an Epic ticket — both chain
|
|
349
|
-
`/plan --seed-file`. The workflow introduces no new sizing, clustering,
|
|
350
|
-
or dedup logic — `route-finding.js` / `promote-finding.js` remain the
|
|
351
|
-
single implementation.
|
|
352
|
-
- **`createStory` / `createPlanSeed` (`/plan --seed-file`)** — render a
|
|
353
|
-
**redacted** plan seed from the cluster (reuse the `/audit-to-stories`
|
|
354
|
-
Phase 5a seed shape; redaction already ran in Capture), **stamp the
|
|
355
|
-
cluster's `fingerprintFooter(sha)` verbatim into the seed body**, then
|
|
356
|
-
chain `/plan --seed-file <seed>`. Prefer one Story; split only under
|
|
357
|
-
the default-single policy. The footer must survive into the issue body
|
|
358
|
-
so a later `routeFinding` dedups the same finding instead of re-filing
|
|
359
|
-
it.
|
|
360
|
-
- **A `file` disposition never opens a raw GitHub Issue.** Every `file`
|
|
361
|
-
finding flows through `promoteFindings` → `/plan`; only `defer` and
|
|
362
|
-
`dismiss` skip the `/plan` handoff.
|
|
363
|
-
|
|
364
|
-
5. **Gate:** any ticket-filing, seed write, `/plan` invocation, or label
|
|
365
|
-
mutation is a write — confirm each one with the operator before it happens.
|
|
366
|
-
Capture stayed read-only precisely so that every state change lands here,
|
|
367
|
-
deliberately and confirmed. The plan→deliver hard stop is preserved: each
|
|
368
|
-
`/plan` chain pauses at its own HITL gates and never auto-delivers.
|
|
197
|
+
Route the captured ledger through the shared classify → route → disposition →
|
|
198
|
+
promote procedure in [`helpers/qa-core.md`](helpers/qa-core.md), with the
|
|
199
|
+
operator deciding each `file` / `defer` / `dismiss` and every write
|
|
200
|
+
operator-gated. `file` findings are promoted through `/plan` (never a raw
|
|
201
|
+
Issue); `defer` carries an item forward as backlog; `dismiss` marks it
|
|
202
|
+
non-actionable.
|
|
369
203
|
|
|
370
204
|
After triage, write the updated dispositions back to the ledger (still under
|
|
371
205
|
`temp/qa/`), and summarize: items captured, the driving method used, classes,
|
|
372
206
|
routes (`new`/`update-existing`/`duplicate`/`regression-of-closed`), the
|
|
373
|
-
Stories (`/plan --seed-file`) promoted, and the deferred rolling backlog
|
|
374
|
-
|
|
207
|
+
Stories (`/plan --seed-file`) promoted, and the deferred rolling backlog a
|
|
208
|
+
resumed session will pick up.
|
|
375
209
|
|
|
376
210
|
---
|
|
377
211
|
|
|
378
212
|
## Constraints
|
|
379
213
|
|
|
214
|
+
Beyond the shared core ([`helpers/qa-core.md`](helpers/qa-core.md): contract +
|
|
215
|
+
loud failure, session/ledger, redact-first, QaLedgerItem, triage, HITL gate)
|
|
216
|
+
and the driving conventions
|
|
217
|
+
([`stack/qa/qa-explore-driving`](../skills/stack/qa/qa-explore-driving/SKILL.md)),
|
|
218
|
+
the `/qa-explore`-specific deltas are:
|
|
219
|
+
|
|
380
220
|
- **Agent-led, bounded per surface.** The agent drives one named surface per
|
|
381
|
-
session and proposes when it is covered; the operator gates the boundary.
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
explicitly with a recorded reason, never a silent fallback. Do not switch
|
|
387
|
-
methods mid-surface without a new Plan note. See
|
|
388
|
-
[`stack/qa/qa-explore-driving`](../skills/stack/qa/qa-explore-driving/SKILL.md).
|
|
221
|
+
session and proposes when it is covered; the operator gates the boundary. No
|
|
222
|
+
human-driven flow lives here — that is [`/qa-assist`](qa-assist.md).
|
|
223
|
+
- **Pick the driving method at Plan time** (drive default; static the documented
|
|
224
|
+
interim, chosen with a recorded reason, never a silent fallback); do not
|
|
225
|
+
switch mid-surface without a new Plan note.
|
|
389
226
|
- **Capture is read-only.** The only Capture write is appending ledger lines
|
|
390
|
-
under `temp/qa/`. No source edits,
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
-
|
|
399
|
-
|
|
400
|
-
ticket, or mutate a label autonomously.
|
|
401
|
-
- **The ledger lives under `temp/qa/` only**, one `QaLedgerItem` per ndjson
|
|
402
|
-
line, conforming to [`qa-ledger.schema.json`](../schemas/qa-ledger.schema.json).
|
|
403
|
-
Never commit it.
|
|
404
|
-
- **Redact before persist.** Every evidence string passes through
|
|
405
|
-
[`redact-evidence.js`](../scripts/lib/qa/redact-evidence.js) before it
|
|
406
|
-
reaches disk or GitHub, per [`security-baseline.md`](../rules/security-baseline.md).
|
|
407
|
-
- **Delegate decisions to the helpers.** Coverage verdict
|
|
408
|
-
([`coverage-verdict.js`](../scripts/lib/qa/coverage-verdict.js)),
|
|
409
|
-
missing-test ([`propose-missing-test.js`](../scripts/lib/qa/propose-missing-test.js)),
|
|
410
|
-
classification ([`classify-finding.js`](../scripts/lib/findings/classify-finding.js)),
|
|
411
|
-
dedup/route ([`route-finding.js`](../scripts/lib/findings/route-finding.js)),
|
|
412
|
-
and cluster/size/promote
|
|
413
|
-
([`promote-finding.js`](../scripts/lib/findings/promote-finding.js)) are
|
|
414
|
-
deterministic — never re-derive them in prose.
|
|
415
|
-
- **Promote through `/plan`, never a raw Issue.** A `file`-dispositioned
|
|
416
|
-
finding is promoted via `promoteFindings`, which chains into
|
|
417
|
-
[`/plan`](plan.md) (`--seed-file` for a tight cluster, `--seed` for a broad
|
|
418
|
-
one) — mirroring [`/audit-to-stories`](audit-to-stories.md). `/qa-explore`
|
|
419
|
-
never opens a bare GitHub Issue for a `file` finding. The cluster's
|
|
420
|
-
`fingerprintFooter(sha)` is stamped verbatim into the seed so a future
|
|
421
|
-
`routeFinding` dedups it.
|
|
422
|
-
- **Resume safely.** A reused session appends and carries the un-triaged
|
|
423
|
-
backlog forward via [`qa-session.js`](../scripts/lib/qa/qa-session.js); it
|
|
424
|
-
never overwrites a prior ledger.
|
|
227
|
+
under `temp/qa/`. No source edits, ticket mutations, product writes, or
|
|
228
|
+
destructive form submissions. Reach an authenticated surface only through the
|
|
229
|
+
resolved environment's `signInSeam`; where no seam resolves, record the gap
|
|
230
|
+
and fall back to static.
|
|
231
|
+
- **Broken navigation is a finding, not a workaround** — never URL-jump around a
|
|
232
|
+
missing affordance, a nav 404, or a guard redirect loop.
|
|
233
|
+
- **Delegate coverage decisions to the helpers.** Coverage verdict
|
|
234
|
+
([`coverage-verdict.js`](../scripts/lib/qa/coverage-verdict.js)) and
|
|
235
|
+
missing-test ([`propose-missing-test.js`](../scripts/lib/qa/propose-missing-test.js))
|
|
236
|
+
are deterministic — never re-derive them in prose.
|
|
425
237
|
|
|
426
238
|
## See also
|
|
427
239
|
|
|
428
|
-
- [`/plan`](plan.md) — the planning pipeline
|
|
429
|
-
|
|
430
|
-
plan→deliver hard stop is preserved across the handoff.
|
|
240
|
+
- [`/plan`](plan.md) — the planning pipeline Triage chains into for a
|
|
241
|
+
`file`-dispositioned finding. The plan→deliver hard stop is preserved.
|
|
431
242
|
- [`/qa-assist`](qa-assist.md) — the human-led sibling that enriches a single
|
|
432
243
|
operator observation and triages through the same `/plan` handoff.
|
|
433
244
|
- [`/audit-to-stories`](audit-to-stories.md) — the precedent for the
|
|
434
245
|
findings → `/plan` handoff and the shared fingerprint-footer dedup contract.
|
|
435
|
-
- [`
|
|
436
|
-
|
|
437
|
-
cluster/size/promote and dedup/route/fingerprint-footer helpers. There is no
|
|
438
|
-
second clustering, sizing, or dedup implementation.
|
|
246
|
+
- [`helpers/qa-core.md`](helpers/qa-core.md) — the shared contract/session/
|
|
247
|
+
redaction/QaLedgerItem/triage/HITL core.
|