mandrel 2.7.0 → 2.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/README.md +104 -330
- package/.agents/agents/auditor.md +135 -0
- package/.agents/agents/plan-critic.md +80 -0
- package/.agents/audit-checklists/dependencies.md +7 -0
- package/.agents/audit-checklists/documentation.md +1 -0
- package/.agents/docs/SDLC.md +69 -162
- package/.agents/docs/configuration.md +133 -499
- package/.agents/docs/quality-gates.md +59 -180
- package/.agents/instructions.md +170 -295
- package/.agents/rules/changelog-style.md +8 -66
- package/.agents/rules/ci-remediation.md +65 -124
- package/.agents/rules/gherkin-standards.md +10 -31
- package/.agents/rules/git-conventions-reference.md +28 -61
- package/.agents/rules/git-conventions.md +1 -1
- package/.agents/rules/orchestration-error-handling.md +5 -15
- package/.agents/rules/security-baseline.md +7 -13
- package/.agents/rules/shell-conventions.md +4 -13
- package/.agents/rules/test-seams.md +2 -2
- package/.agents/rules/testing-standards.md +7 -17
- package/.agents/skills/core/api-and-interface-design/SKILL.md +23 -297
- package/.agents/skills/core/api-and-interface-design/reference.md +76 -0
- package/.agents/skills/core/browser-testing-with-devtools/SKILL.md +20 -327
- package/.agents/skills/core/browser-testing-with-devtools/reference.md +74 -0
- package/.agents/skills/core/code-review-and-quality/reference.md +16 -398
- package/.agents/skills/core/debugging-and-error-recovery/reference.md +14 -281
- package/.agents/skills/core/documentation-and-adrs/reference.md +42 -338
- package/.agents/skills/core/git-workflow-and-versioning/SKILL.md +28 -326
- package/.agents/skills/core/idea-refinement/SKILL.md +67 -193
- package/.agents/skills/core/security-and-hardening/SKILL.md +15 -31
- package/.agents/skills/core/security-and-hardening/reference.md +15 -273
- package/.agents/skills/skills.index.json +5 -5
- package/.agents/skills/stack/qa/gherkin-authoring/SKILL.md +16 -222
- package/.agents/skills/stack/qa/gherkin-authoring/reference.md +157 -0
- package/.agents/skills/stack/qa/playwright/SKILL.md +0 -29
- package/.agents/skills/stack/qa/qa-explore-driving/SKILL.md +19 -23
- package/.agents/skills/stack/qa/qa-harness/SKILL.md +35 -53
- package/.agents/skills/stack/qa/vitest/SKILL.md +0 -29
- package/.agents/workflows/audit-accessibility.md +34 -108
- package/.agents/workflows/audit-architecture.md +35 -123
- package/.agents/workflows/audit-clean-code.md +19 -78
- package/.agents/workflows/audit-data-model.md +32 -100
- package/.agents/workflows/audit-dependencies.md +47 -111
- package/.agents/workflows/audit-devops.md +16 -83
- package/.agents/workflows/audit-documentation.md +46 -93
- package/.agents/workflows/audit-navigability.md +26 -80
- package/.agents/workflows/audit-performance.md +40 -106
- package/.agents/workflows/audit-privacy.md +17 -80
- package/.agents/workflows/audit-quality.md +35 -99
- package/.agents/workflows/audit-security.md +20 -78
- package/.agents/workflows/audit-seo.md +20 -98
- package/.agents/workflows/audit-sre.md +20 -88
- package/.agents/workflows/audit-to-stories.md +1 -8
- package/.agents/workflows/audit-ux-ui.md +17 -80
- package/.agents/workflows/deliver.md +54 -9
- package/.agents/workflows/git-cleanup.md +50 -275
- package/.agents/workflows/helpers/audit-lens-core.md +230 -0
- package/.agents/workflows/helpers/code-review.md +11 -23
- package/.agents/workflows/helpers/deliver-story-reference.md +114 -17
- package/.agents/workflows/helpers/deliver-story.md +26 -186
- package/.agents/workflows/helpers/qa-core.md +174 -0
- package/.agents/workflows/helpers/qa-run-scenario-reference.md +35 -0
- package/.agents/workflows/helpers/qa-run-scenario.md +11 -25
- package/.agents/workflows/helpers/worktree-lifecycle.md +6 -67
- package/.agents/workflows/mandrel-update.md +7 -13
- package/.agents/workflows/plan.md +17 -15
- package/.agents/workflows/qa-assist.md +140 -269
- package/.agents/workflows/qa-explore.md +125 -316
- package/.agents/workflows/qa-run.md +180 -380
- package/docs/CHANGELOG.md +10 -0
- package/package.json +1 -1
- package/.agents/workflows/helpers/audit-dual-path.md +0 -59
- package/.agents/workflows/helpers/audit-self-check.md +0 -70
- package/.agents/workflows/helpers/audit-severity-scale.md +0 -19
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
# Gherkin Authoring — Reference (on-demand)
|
|
2
|
+
|
|
3
|
+
**Read this when** a task engages one of the sections below and the Policy
|
|
4
|
+
Capsule in [`SKILL.md`](SKILL.md) does not settle it on its own. The capsule is
|
|
5
|
+
the contract; this file is the reference material behind it. The enforcement
|
|
6
|
+
rules (tag taxonomy, forbidden patterns, Outline conventions) are owned by the
|
|
7
|
+
SSOT rule, [`gherkin-standards.md`](../../../../rules/gherkin-standards.md).
|
|
8
|
+
|
|
9
|
+
## Canonical Given / When / Then Phrasing
|
|
10
|
+
|
|
11
|
+
`.feature` prose is written for a product-minded reader, not a test harness.
|
|
12
|
+
Each clause has exactly one job:
|
|
13
|
+
|
|
14
|
+
- **Given** — a precondition that is *already true* when the scenario starts.
|
|
15
|
+
Stative, past or present tense. No actions.
|
|
16
|
+
- Good: `Given a signed-in account-owner with an unpaid invoice`.
|
|
17
|
+
- Bad: `Given the user clicks the login button` (that's a When).
|
|
18
|
+
- **When** — the single business action under test. One verb, one actor, one
|
|
19
|
+
event. Splitting a When into multiple clauses is almost always wrong — move
|
|
20
|
+
the extra clauses into Given.
|
|
21
|
+
- Good: `When they issue the invoice`.
|
|
22
|
+
- Bad: `When they log in and issue the invoice and download the PDF`.
|
|
23
|
+
- **Then** — an observable, user-visible outcome. Assert what the *user* sees or
|
|
24
|
+
what the *business* records, not what the system emits internally.
|
|
25
|
+
- Good: `Then the invoice appears in the issued invoices list`.
|
|
26
|
+
- Bad: `Then a 201 is returned` (forbidden; see
|
|
27
|
+
[gherkin-standards § Forbidden Patterns](../../../../rules/gherkin-standards.md#forbidden-patterns)).
|
|
28
|
+
- **And / But** — continuation clauses. They inherit the mood of the most recent
|
|
29
|
+
Given/When/Then. Never start a scenario with And/But.
|
|
30
|
+
|
|
31
|
+
Voice and tense: third-person, present tense, role-qualified noun actors
|
|
32
|
+
(`the user`, `a billing-admin`, not "I" or "we"). One clause, one fact —
|
|
33
|
+
comma-chained facts hide compound assertions. Numbers and identifiers go in
|
|
34
|
+
`Examples` tables or fixtures, not inline prose.
|
|
35
|
+
|
|
36
|
+
## Translating PRD Acceptance Criteria to Scenarios
|
|
37
|
+
|
|
38
|
+
PRD ACs are the raw material. The authoring move is **one AC → one scenario**
|
|
39
|
+
unless the AC encodes a matrix (then it becomes a Scenario Outline — see below).
|
|
40
|
+
|
|
41
|
+
Walkthrough — a PRD AC from a billing feature:
|
|
42
|
+
|
|
43
|
+
> **AC-3:** When a billing-admin issues an invoice for a customer with a
|
|
44
|
+
> negative balance, the system rejects the issue and shows an error naming the
|
|
45
|
+
> outstanding amount.
|
|
46
|
+
|
|
47
|
+
Translation steps:
|
|
48
|
+
|
|
49
|
+
1. **Identify the domain.** Billing → tag the scenario `@domain-billing`.
|
|
50
|
+
2. **Identify the actor and precondition.** "billing-admin", "customer with a
|
|
51
|
+
negative balance" → two Givens.
|
|
52
|
+
3. **Identify the single action.** "issues an invoice" → one When.
|
|
53
|
+
4. **Identify the observable outcome.** "rejects the issue and shows an error
|
|
54
|
+
naming the outstanding amount" → two Thens (business outcome, then
|
|
55
|
+
user-visible detail).
|
|
56
|
+
5. **Pick the risk tag.** If the originating ticket was `risk::high`, add
|
|
57
|
+
`@risk-high`.
|
|
58
|
+
|
|
59
|
+
Resulting scenario:
|
|
60
|
+
|
|
61
|
+
```gherkin
|
|
62
|
+
@domain-billing @risk-high
|
|
63
|
+
Scenario: Issuing an invoice is rejected when the customer has a negative balance
|
|
64
|
+
Given a signed-in billing-admin
|
|
65
|
+
And a customer with a negative account balance
|
|
66
|
+
When the billing-admin issues an invoice for that customer
|
|
67
|
+
Then the issue is rejected
|
|
68
|
+
And the rejection message names the outstanding amount
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Heuristics:
|
|
72
|
+
|
|
73
|
+
- If a single AC needs more than **one When**, it is really two ACs. Split
|
|
74
|
+
before you write.
|
|
75
|
+
- If a Then starts with "and the database has…" or "and the API returned…", you
|
|
76
|
+
have crossed into step-definition or contract-test territory. Rewrite it as
|
|
77
|
+
user-visible language or move the assertion to a contract test.
|
|
78
|
+
- If an AC says "the system logs X", that's an engineering non-functional
|
|
79
|
+
requirement — cover it with a focused unit or integration test, not a
|
|
80
|
+
`.feature` file.
|
|
81
|
+
|
|
82
|
+
## Background vs. Given, Outline vs. Multi-Scenario
|
|
83
|
+
|
|
84
|
+
Two authoring decisions authors routinely get wrong.
|
|
85
|
+
|
|
86
|
+
**Background vs. Given.** `Background` runs before **every** scenario in the
|
|
87
|
+
file. Use it only when every scenario genuinely shares the precondition **and**
|
|
88
|
+
it has no per-scenario variation (no `<placeholders>`, no per-scenario data).
|
|
89
|
+
Prefer a per-scenario Given when only some scenarios need the precondition, when
|
|
90
|
+
it varies by role/plan/fixture, or when a reader cannot understand the scenario
|
|
91
|
+
without scrolling up. Rule of thumb: if you find yourself writing *"except in
|
|
92
|
+
the admin scenario, where the Background step is actually…"*, delete the
|
|
93
|
+
Background.
|
|
94
|
+
|
|
95
|
+
**Scenario Outline vs. multiple Scenarios.** Use `Scenario Outline` when the
|
|
96
|
+
**same behavior** is exercised across a **bounded matrix** (roles, plan tiers,
|
|
97
|
+
locales) — the Givens/When/Thens are identical and only the data varies. Use
|
|
98
|
+
multiple `Scenario` blocks when the Givens differ structurally, the When verb
|
|
99
|
+
differs, or the Then outcomes differ in kind, not just value. Keep `Examples`
|
|
100
|
+
tables under ~12 rows, and split them by tag when rows need different tags
|
|
101
|
+
(e.g. `@risk-high` on admin rows only).
|
|
102
|
+
|
|
103
|
+
## Step-Definition Library Structure
|
|
104
|
+
|
|
105
|
+
Authoring scenarios and maintaining steps are the same job split across two
|
|
106
|
+
files. Keep that coupling visible:
|
|
107
|
+
|
|
108
|
+
```text
|
|
109
|
+
tests/
|
|
110
|
+
steps/
|
|
111
|
+
_common/ # actor, auth, navigation — reused across every domain
|
|
112
|
+
auth.steps.ts
|
|
113
|
+
navigation.steps.ts
|
|
114
|
+
billing/ # one directory per @domain-* tag
|
|
115
|
+
invoices.steps.ts
|
|
116
|
+
_deprecated/ # steps pending migration
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
- One directory per `@domain-*` tag; `_common/` holds cross-cutting steps.
|
|
120
|
+
Files are named by the noun the steps act on (`invoices.steps.ts`). Two levels
|
|
121
|
+
(`steps/<domain>/<noun>.steps.ts`) is the nesting ceiling.
|
|
122
|
+
- Step text follows the scenario text verbatim — divergence between scenario
|
|
123
|
+
prose and step text is a bug. Parameterize only over values that actually
|
|
124
|
+
vary; keep implementations ≤20 lines (a longer one signals a missing helper).
|
|
125
|
+
|
|
126
|
+
**Reuse before authoring** — the non-negotiable workflow is in
|
|
127
|
+
[gherkin-standards § Step Reuse](../../../../rules/gherkin-standards.md#step-reuse--grep-before-you-write):
|
|
128
|
+
|
|
129
|
+
1. Search for the verb: `rg -n "issues? an invoice" tests/steps`.
|
|
130
|
+
2. If the phrase exists, **change your scenario** to use it — do not fork a
|
|
131
|
+
near-duplicate step.
|
|
132
|
+
3. If a near-match exists, extend the existing step and update every call site
|
|
133
|
+
in the same PR.
|
|
134
|
+
4. Only when no reasonable match exists, add a new step in the correct domain
|
|
135
|
+
directory.
|
|
136
|
+
|
|
137
|
+
**Deprecation.** When a step is superseded, move the old definition into
|
|
138
|
+
`steps/_deprecated/` with a comment naming the replacement, migrate every call
|
|
139
|
+
site in the same PR, and delete the `_deprecated/` entry when the migration
|
|
140
|
+
lands. Never leave two live step definitions that mean the same thing.
|
|
141
|
+
|
|
142
|
+
## Authoring Checklist
|
|
143
|
+
|
|
144
|
+
Before opening a PR that adds or edits a `.feature` file:
|
|
145
|
+
|
|
146
|
+
- [ ] Every Scenario/Outline carries exactly one `@domain-*` tag.
|
|
147
|
+
- [ ] No forbidden patterns (SQL, status codes, selectors, URLs, payloads,
|
|
148
|
+
framework names, explicit waits) appear in prose.
|
|
149
|
+
- [ ] Each scenario has exactly one `When`.
|
|
150
|
+
- [ ] `Then` clauses assert user-visible outcomes, not implementation.
|
|
151
|
+
- [ ] `Background` is justified (applies to every scenario in the file).
|
|
152
|
+
- [ ] If a Scenario Outline is used, the matrix is bounded (≤12 rows) and the
|
|
153
|
+
shape is truly identical across rows.
|
|
154
|
+
- [ ] Every step phrase grep-matches an existing step definition **or** is
|
|
155
|
+
accompanied by a new step definition in the right domain directory.
|
|
156
|
+
- [ ] Reads standalone — a product reader who has never seen the codebase can
|
|
157
|
+
understand the intent without opening a step file.
|
|
@@ -19,32 +19,3 @@ vendor: playwright
|
|
|
19
19
|
- Write tests independent of one another so they run in parallel; clean up shared state in fixtures, not afterwards.
|
|
20
20
|
- Enable `trace: 'on-first-retry'` (or `'retain-on-failure'`) so CI failures are debuggable in the Trace Viewer.
|
|
21
21
|
- Use a unique data set per test run, or tear down state explicitly, to prevent cross-test contamination.
|
|
22
|
-
|
|
23
|
-
Standard operating procedures for robust, end-to-end (E2E) browser testing.
|
|
24
|
-
|
|
25
|
-
## 1. Core Principles
|
|
26
|
-
|
|
27
|
-
- **End-to-End focus:** Test the application as a user would, through the
|
|
28
|
-
browser.
|
|
29
|
-
- **Auto-waiting:** Leverage Playwright's built-in auto-waiting instead of
|
|
30
|
-
hardcoded `waitForTimeout` calls.
|
|
31
|
-
- **Resilience:** Write tests that survive minor UI changes (e.g., color tweaks)
|
|
32
|
-
by using robust selectors.
|
|
33
|
-
|
|
34
|
-
## 2. Technical Standards
|
|
35
|
-
|
|
36
|
-
- **Locators:** Use user-visible locators (e.g., `getByRole`, `getByText`,
|
|
37
|
-
`getByLabel`) over brittle CSS selectors or XPath.
|
|
38
|
-
- **State Management:** Use `storageState` to reuse authentication between
|
|
39
|
-
tests, avoiding repetitive login flows.
|
|
40
|
-
- **Visual Testing:** Use `toHaveScreenshot()` for critical UI layouts to detect
|
|
41
|
-
visual regressions.
|
|
42
|
-
|
|
43
|
-
## 3. Best Practices
|
|
44
|
-
|
|
45
|
-
- **Parallelism:** Ensure tests are independent so they can run concurrently to
|
|
46
|
-
reduce CI time.
|
|
47
|
-
- **Tracing:** Enable trace recording on failure to quickly debug CI issues with
|
|
48
|
-
the Playwright Trace Viewer.
|
|
49
|
-
- **Test Data:** Use a unique data set per test run or clean up state to prevent
|
|
50
|
-
cross-contamination.
|
|
@@ -41,29 +41,24 @@ the read-only and no-PII boundaries are inviolable per
|
|
|
41
41
|
[`security-baseline.md`](../../../../rules/security-baseline.md). Read this
|
|
42
42
|
skill before driving a live surface; read the workflow for the phase order.
|
|
43
43
|
|
|
44
|
-
## 1. Navigation-First Driving (the default)
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
the
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
missing, a nav link 404s, or a guard redirect loops, that is the finding. Do
|
|
63
|
-
not route around it with a direct URL — record it and move on.
|
|
64
|
-
- **Observe, do not fabricate.** Use app-provided UI affordances to reach and
|
|
65
|
-
observe a surface. Never script the runtime to manufacture an outcome the
|
|
66
|
-
exploration is meant to discover.
|
|
44
|
+
## 1. Navigation-First Driving (the default) — one prose home
|
|
45
|
+
|
|
46
|
+
Navigation-first driving is the default and load-bearing convention: reach every
|
|
47
|
+
surface the way a real user would so findings reflect a user-reachable state,
|
|
48
|
+
not an artifact of a deep link. Its full prose statement lives once in
|
|
49
|
+
[`qa-run-scenario.md`](../../../../workflows/helpers/qa-run-scenario.md) (shared
|
|
50
|
+
with the known-scenario sweep) — apply it from there rather than restating it.
|
|
51
|
+
In capsule:
|
|
52
|
+
|
|
53
|
+
- **Drive the running app by default** through the browser MCP; static (§ 2) is
|
|
54
|
+
the interim alternative, not the norm.
|
|
55
|
+
- **Start at a root and never URL-jump** — reach the surface under test by
|
|
56
|
+
clicking the affordances a real user has, never by deep-linking a starting
|
|
57
|
+
state.
|
|
58
|
+
- **Broken navigation is a finding, not a workaround** — a missing affordance, a
|
|
59
|
+
nav 404, or a guard redirect loop is recorded; do not route around it.
|
|
60
|
+
- **Observe, do not fabricate** — never script the runtime to manufacture an
|
|
61
|
+
outcome the exploration is meant to discover.
|
|
67
62
|
|
|
68
63
|
## 2. Static Driving — the Documented Interim
|
|
69
64
|
|
|
@@ -150,6 +145,7 @@ deployed surfaces are **driven**, not statically deferred.
|
|
|
150
145
|
## 5. Cross-References
|
|
151
146
|
|
|
152
147
|
- Run procedure (SSOT): [`qa-explore.md`](../../../../workflows/qa-explore.md).
|
|
148
|
+
- Driving rules (one prose home): [`qa-run-scenario.md`](../../../../workflows/helpers/qa-run-scenario.md).
|
|
153
149
|
- Known-scenario sibling sweep: [`qa-harness`](../qa-harness/SKILL.md).
|
|
154
150
|
- Browser instrumentation: [`browser-testing-with-devtools`](../../../core/browser-testing-with-devtools/SKILL.md).
|
|
155
151
|
- Read-only / no-PII boundary: [`security-baseline.md`](../../../../rules/security-baseline.md).
|
|
@@ -14,13 +14,12 @@ description:
|
|
|
14
14
|
|
|
15
15
|
## Policy Capsule
|
|
16
16
|
|
|
17
|
-
-
|
|
18
|
-
- Assert `Then` outcomes semantically against the accessibility snapshot (roles, accessible names, visible text); never via DOM/CSS/XPath selectors, HTTP status codes, response bodies, or DB rows.
|
|
17
|
+
- Driving rules (navigation-first, semantic `Then`, mandatory redaction, sequential-only) live in one prose home — [`qa-run-scenario.md`](../../../../workflows/helpers/qa-run-scenario.md); apply them, do not restate them.
|
|
19
18
|
- Capture console and network per surface; turn each non-allowlisted console error and each failed/error-status request into one structured `F#` finding.
|
|
20
19
|
- Filter console through `qa.consoleAllowlist` via `filterConsoleMessages`; treat the allowlist as a benign-noise filter, never as a security control to silence genuine errors.
|
|
21
20
|
- Spot-check surfaces against `qa.designTokens` when set; flag gross token violations (off-palette colors, off-scale spacing/typography) as findings.
|
|
22
21
|
- Scrub captured console and network of tokens, session cookies, and PII before rendering any finding — findings are posted to GitHub at approval time.
|
|
23
|
-
-
|
|
22
|
+
- Record findings as `QaLedgerItem`s and route them through the shared classify/route/promote core ([`qa-core.md`](../../../../workflows/helpers/qa-core.md)); the harness never files tickets autonomously.
|
|
24
23
|
- Resolve the `qa` contract first and fail loudly when it is absent or malformed; there is no auto-detection fallback and no headless degrade.
|
|
25
24
|
|
|
26
25
|
Guidance for executing the agent-driven QA harness through a real browser (the
|
|
@@ -35,45 +34,21 @@ live in [`testing-standards.md`](../../../../rules/testing-standards.md)
|
|
|
35
34
|
discipline is shared with [`playwright`](../playwright/SKILL.md). Read this
|
|
36
35
|
skill before instrumenting a live surface; read the workflow for the run order.
|
|
37
36
|
|
|
38
|
-
## 1.
|
|
39
|
-
|
|
40
|
-
The
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
Do not route around it with a direct URL — record it and move on.
|
|
54
|
-
- **Map Gherkin to browser actions.** `Given` establishes state by navigating
|
|
55
|
-
from the root (sign in as the persona, walk to the starting surface, seed via
|
|
56
|
-
UI where the manifest does not pre-seed). `When` performs the single user
|
|
57
|
-
action (`click`, `fill_form`, then `wait_for` the transition). Use
|
|
58
|
-
`evaluate_script` only for app-provided hooks — never to fabricate the
|
|
59
|
-
outcome a `Then` is meant to observe.
|
|
60
|
-
|
|
61
|
-
### Semantic `Then` assertion
|
|
62
|
-
|
|
63
|
-
Assert every `Then` against the accessibility snapshot from `take_snapshot`,
|
|
64
|
-
matching on **roles, accessible names, labels, and visible text** that express
|
|
65
|
-
the user-visible outcome — "a banner with text _Invoice sent_ is visible", "a
|
|
66
|
-
row for _ACME Corp_ appears in the invoices table".
|
|
67
|
-
|
|
68
|
-
- **Never** assert against brittle DOM/CSS/XPath selectors.
|
|
69
|
-
- **Never** assert on HTTP status codes, response bodies, or DB rows inside a
|
|
70
|
-
scenario — those are contract-tier concerns (see
|
|
71
|
-
[`testing-standards.md` § Assertion Placement](../../../../rules/testing-standards.md#assertion-placement)).
|
|
72
|
-
- A `Then` that can only be expressed as a wire-shape or DB check is a signal
|
|
73
|
-
the scenario is **mis-tiered**, not a license to break the semantic rule.
|
|
74
|
-
|
|
75
|
-
Record each scenario's result (pass / fail / blocked), the surface it ended on,
|
|
76
|
-
and a one-line user-visible symptom for any failure.
|
|
37
|
+
## 1. Driving Rules Live in One Prose Home
|
|
38
|
+
|
|
39
|
+
The driving rules the harness depends on — **navigation-first / never URL-jump**,
|
|
40
|
+
**semantic `Then` assertion** against the accessibility snapshot, the Gherkin →
|
|
41
|
+
browser-action mapping, the per-`When` write guard, mandatory evidence
|
|
42
|
+
redaction, and the **sequential-only** browser rule — are stated once in
|
|
43
|
+
[`qa-run-scenario.md`](../../../../workflows/helpers/qa-run-scenario.md), the
|
|
44
|
+
single-scenario driver `/qa-run` delegates to. Apply them from there; this skill
|
|
45
|
+
does not restate them. In short: reach every surface the way a real user would
|
|
46
|
+
(start at a root, click affordances, never deep-link a `Given`), assert `Then`
|
|
47
|
+
semantically (roles, accessible names, visible text — never DOM/CSS/XPath
|
|
48
|
+
selectors, HTTP status, response bodies, or DB rows), and record each scenario's
|
|
49
|
+
result (pass / fail / blocked), the surface it ended on, and a one-line symptom
|
|
50
|
+
for any failure. Assertion-tier rules are in
|
|
51
|
+
[`testing-standards.md` § Assertion Placement](../../../../rules/testing-standards.md#assertion-placement).
|
|
77
52
|
|
|
78
53
|
## 2. Per-Surface Console & Network Capture
|
|
79
54
|
|
|
@@ -126,7 +101,9 @@ skip this check entirely — do not invent a token source.
|
|
|
126
101
|
## 3. Findings — the `F#` Shape
|
|
127
102
|
|
|
128
103
|
Every captured problem is normalized into the structured `F#` finding shape so
|
|
129
|
-
the
|
|
104
|
+
the sweep can record it onto the shared ledger (each `F#` finding becomes one
|
|
105
|
+
`QaLedgerItem` — see [`qa-core.md`](../../../../workflows/helpers/qa-core.md))
|
|
106
|
+
and the schema validates:
|
|
130
107
|
|
|
131
108
|
```jsonc
|
|
132
109
|
{
|
|
@@ -137,7 +114,6 @@ the draft bundle stays diffable and the schema validates:
|
|
|
137
114
|
"likelyRootCause": null, // heuristic card output (§4); null until enriched
|
|
138
115
|
"disposition": "follow-up", // blocker | follow-up
|
|
139
116
|
"acceptance": null, // AC this folds into, when known
|
|
140
|
-
"foldsInto": "F2", // optional: another finding this is a duplicate facet of
|
|
141
117
|
"evidence": {
|
|
142
118
|
"console": [{ "level": "error", "text": "..." }],
|
|
143
119
|
"network": []
|
|
@@ -172,12 +148,14 @@ and let the operator triage from the symptom.
|
|
|
172
148
|
| `Failed to fetch` / `NetworkError` / CORS-rejected request | Misconfigured CORS allowlist, wrong origin, or a downed dependency | follow-up |
|
|
173
149
|
| Hydration / mismatch warning escalated to error | Server/client render divergence | follow-up |
|
|
174
150
|
| Off-palette color, off-scale spacing/typography | Design-token drift — hard-coded value bypassing the token | follow-up |
|
|
175
|
-
| Repeated identical console error across many surfaces | A shared component or global bootstrap fault |
|
|
151
|
+
| Repeated identical console error across many surfaces | A shared component or global bootstrap fault | record once; the shared route/dedup core collapses duplicates at triage |
|
|
176
152
|
|
|
177
153
|
Heuristics for working the cards:
|
|
178
154
|
|
|
179
|
-
- **
|
|
180
|
-
|
|
155
|
+
- **Record once, let dedup collapse.** When the same error fires on many
|
|
156
|
+
surfaces, record it once rather than filing N copies; the shared
|
|
157
|
+
classify/route/dedup core ([`qa-core.md`](../../../../workflows/helpers/qa-core.md))
|
|
158
|
+
collapses duplicates at triage against the fingerprint footer.
|
|
181
159
|
- **Blocker vs. follow-up.** A finding is a **blocker** when it breaks the
|
|
182
160
|
scenario's user-visible outcome or exposes an authorization gap. Everything
|
|
183
161
|
else (noise that does not break the journey, cosmetic token drift) is a
|
|
@@ -186,13 +164,15 @@ Heuristics for working the cards:
|
|
|
186
164
|
symptom and leave `likelyRootCause: null`. A wrong guess is worse than an
|
|
187
165
|
honest "unknown" the operator can triage.
|
|
188
166
|
|
|
189
|
-
## 5.
|
|
167
|
+
## 5. Record onto the Ledger & Triage (Never File Autonomously)
|
|
190
168
|
|
|
191
|
-
|
|
192
|
-
`
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
169
|
+
Record each `F#` finding as a `QaLedgerItem` on the shared session ledger under
|
|
170
|
+
`temp/qa/`, then route the ledger through the shared classify → route →
|
|
171
|
+
disposition → promote core — both stated once in
|
|
172
|
+
[`qa-core.md`](../../../../workflows/helpers/qa-core.md). The harness **MUST
|
|
173
|
+
NOT** create tickets autonomously: findings are promoted through `/plan` only
|
|
174
|
+
after the operator confirms each disposition at the HITL write gate. That gate
|
|
175
|
+
is the safety boundary against spurious filing.
|
|
196
176
|
|
|
197
177
|
## 6. Sign-In & Contract Discipline
|
|
198
178
|
|
|
@@ -211,6 +191,8 @@ filing.
|
|
|
211
191
|
## 7. Cross-References
|
|
212
192
|
|
|
213
193
|
- Run procedure (SSOT): [`qa-run.md`](../../../../workflows/qa-run.md).
|
|
194
|
+
- Driving rules (one prose home): [`qa-run-scenario.md`](../../../../workflows/helpers/qa-run-scenario.md).
|
|
195
|
+
- Shared QA core (contract/session/redaction/ledger/triage/HITL): [`qa-core.md`](../../../../workflows/helpers/qa-core.md).
|
|
214
196
|
- Console filter module: [`console-allowlist.js`](../../../../scripts/lib/qa/console-allowlist.js).
|
|
215
197
|
- Assertion-tier rules: [`testing-standards.md`](../../../../rules/testing-standards.md).
|
|
216
198
|
- Scenario prose: [`gherkin-authoring`](../gherkin-authoring/SKILL.md).
|
|
@@ -20,32 +20,3 @@ vendor: vitest
|
|
|
20
20
|
- Use snapshots only for large, stable data structures; avoid them for frequently changing UI to prevent snapshot fatigue.
|
|
21
21
|
- Aim for 80%+ coverage on business logic and edge cases; audit with `vitest --coverage`.
|
|
22
22
|
- Test observable behavior, not internal implementation details; refactors should not require rewriting passing tests.
|
|
23
|
-
|
|
24
|
-
Guidelines for writing fast, reliable unit and integration tests.
|
|
25
|
-
|
|
26
|
-
## 1. Core Principles
|
|
27
|
-
|
|
28
|
-
- **Speed:** Tests should be fast enough to run on every file save.
|
|
29
|
-
- **Isolation:** Each test must be independent. Avoid shared state between
|
|
30
|
-
tests.
|
|
31
|
-
- **Confidence:** Tests should verify behavior, not implementation details.
|
|
32
|
-
|
|
33
|
-
## 2. Technical Standards
|
|
34
|
-
|
|
35
|
-
- **Mocking:** Use `vi.mock()` for external dependencies (APIs, network calls)
|
|
36
|
-
and `vi.spyOn()` for monitoring function calls.
|
|
37
|
-
- **Snapshot Testing:** Use snapshots for large, stable data structures, but
|
|
38
|
-
avoid them for frequently changing UI components to prevent "snapshot
|
|
39
|
-
fatigue."
|
|
40
|
-
- **Coverage:** Aim for 80%+ coverage on business logic and edge cases. Use
|
|
41
|
-
`vitest --coverage` for auditing.
|
|
42
|
-
|
|
43
|
-
## 3. Best Practices
|
|
44
|
-
|
|
45
|
-
- **Descriptive Titles:** Use the
|
|
46
|
-
`describe('Component/Utility', () => { it('should [action] when [condition]') })`
|
|
47
|
-
pattern.
|
|
48
|
-
- **Arrange-Act-Assert (AAA):** Structure tests clearly into setup (Arrange),
|
|
49
|
-
execution (Act), and verification (Assert) phases.
|
|
50
|
-
- **Edge Cases:** Always include tests for error states, null/undefined inputs,
|
|
51
|
-
and boundary conditions.
|
|
@@ -4,6 +4,25 @@ description: Audit WCAG accessibility conformance (static-first) with an optiona
|
|
|
4
4
|
|
|
5
5
|
# Accessibility (WCAG) Audit
|
|
6
6
|
|
|
7
|
+
You are a Senior Accessibility Engineer holding the frontend to **WCAG 2.x
|
|
8
|
+
conformance** — semantic structure, ARIA correctness, keyboard operability, form
|
|
9
|
+
labelling, media alternatives, and contrast — grounding every finding in a
|
|
10
|
+
concrete element and the success criterion it violates. Default to **static**
|
|
11
|
+
detection over source; escalate to a **runtime** pass only when a live target is
|
|
12
|
+
configured. The shared lens machinery — read-only constraint, scope
|
|
13
|
+
interpretation, report envelope + finding-block skeleton, severity scale,
|
|
14
|
+
self-cross-check, and execution strategy — lives in
|
|
15
|
+
[`helpers/audit-lens-core.md`](helpers/audit-lens-core.md). Write the report to
|
|
16
|
+
`{{auditOutputDir}}/audit-accessibility-results.md`. Extra finding fields:
|
|
17
|
+
**WCAG:** (the violated success criterion, e.g. `1.1.1 Non-text Content (A)`)
|
|
18
|
+
and **Evidence:** (`measured | static` + the observable; single-run runtime
|
|
19
|
+
numbers are tagged `provisional`). The report adds a **Runtime Verification**
|
|
20
|
+
section.
|
|
21
|
+
|
|
22
|
+
> **No conformance certification.** The lens reports findings against WCAG
|
|
23
|
+
> success criteria; it does **not** assert a conformance level (A / AA / AAA)
|
|
24
|
+
> for the product. "No findings in scope" is not "certified conformant".
|
|
25
|
+
|
|
7
26
|
## Applicability
|
|
8
27
|
|
|
9
28
|
**Web targets only.** Registered with `target: "web"` in
|
|
@@ -12,28 +31,6 @@ on a project with no rendered frontend, since there are no components, templates
|
|
|
12
31
|
or routes to hold to WCAG. See the `target` key's schema description for how
|
|
13
32
|
applicability is probed from the consumer's checkout.
|
|
14
33
|
|
|
15
|
-
## Role
|
|
16
|
-
|
|
17
|
-
Senior Accessibility Engineer. You hold the frontend to **WCAG 2.x
|
|
18
|
-
conformance** — semantic structure, ARIA correctness, keyboard operability,
|
|
19
|
-
form labelling, media alternatives, and contrast — grounding every finding in a
|
|
20
|
-
concrete element and the success criterion it violates. You default to
|
|
21
|
-
**static** detection over the source, and escalate to a **runtime** pass only
|
|
22
|
-
when a live target is configured.
|
|
23
|
-
|
|
24
|
-
## Context & Objective
|
|
25
|
-
|
|
26
|
-
This is a **read-only** audit. Detect WCAG violations statically from the
|
|
27
|
-
component/template/route source, optionally corroborate them against a running
|
|
28
|
-
build, and emit a structured Markdown report at
|
|
29
|
-
`{{auditOutputDir}}/audit-accessibility-results.md`. Do not modify application
|
|
30
|
-
code — surfacing the violations (each keyed to a WCAG success criterion) is the
|
|
31
|
-
deliverable; fixing them is a separate pass.
|
|
32
|
-
|
|
33
|
-
> **No conformance certification.** The lens reports findings against WCAG
|
|
34
|
-
> success criteria; it does **not** assert a conformance level (A / AA / AAA)
|
|
35
|
-
> for the product. "No findings in scope" is not "certified conformant".
|
|
36
|
-
|
|
37
34
|
## Boundary with `audit-ux-ui`
|
|
38
35
|
|
|
39
36
|
These two web lenses share a border and must not double-report:
|
|
@@ -43,35 +40,26 @@ These two web lenses share a border and must not double-report:
|
|
|
43
40
|
understand the surface? Semantic HTML, ARIA, keyboard/focus, labelled
|
|
44
41
|
controls, text alternatives, and contrast against the WCAG ratio thresholds.
|
|
45
42
|
- **`audit-ux-ui`** owns **design-system adherence** — the consistency
|
|
46
|
-
question: do components and tokens match the project's own design system
|
|
47
|
-
(hardcoded values that bypass a token, raw elements that should defer to a
|
|
48
|
-
design-system component, interaction/loading/error states, premium feel)?
|
|
43
|
+
question: do components and tokens match the project's own design system?
|
|
49
44
|
|
|
50
45
|
Contrast is the one axis both can touch: **accessibility owns the WCAG ratio
|
|
51
46
|
verdict** (4.5:1 body / 3:1 large text / 3:1 non-text), while ux-ui owns whether
|
|
52
47
|
the colour came from a sanctioned token. When a contrast defect is in scope for
|
|
53
48
|
both, report the WCAG failure here and leave the token-adherence note to ux-ui.
|
|
54
49
|
|
|
55
|
-
## Scope
|
|
50
|
+
## Scope
|
|
56
51
|
|
|
57
|
-
|
|
58
|
-
following block is populated with the Story (or plan-run) change-set file list.
|
|
59
|
-
Otherwise — for any manual `/audit-<dimension>` invocation — the block
|
|
60
|
-
renders the literal substitution token and you MUST treat it as **no
|
|
61
|
-
scope filter — run the lens codebase-wide** exactly as you would have
|
|
62
|
-
before this section existed.
|
|
52
|
+
Interpret this lens's change-set fence per the core's Scope interpretation:
|
|
63
53
|
|
|
64
54
|
```text
|
|
65
55
|
{{changedFiles}}
|
|
66
56
|
```
|
|
67
57
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
proceed with the full codebase-wide scan defined in the remaining
|
|
74
|
-
steps.
|
|
58
|
+
## Execution strategy
|
|
59
|
+
|
|
60
|
+
Run this lens as a single `subagent_type: auditor` dispatch returning the report
|
|
61
|
+
path + Executive Summary; sequential inline execution is the fallback (see the
|
|
62
|
+
core's Execution strategy).
|
|
75
63
|
|
|
76
64
|
## Step 0: Discover the frontend surface and config (run first)
|
|
77
65
|
|
|
@@ -98,8 +86,6 @@ exists in scope, say so and emit an empty report rather than inventing findings.
|
|
|
98
86
|
|
|
99
87
|
## Step 1: Static WCAG detection, then triage
|
|
100
88
|
|
|
101
|
-
> Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
|
|
102
|
-
|
|
103
89
|
Run the **mechanical detectors first** (cheap, deterministic greps and the
|
|
104
90
|
static a11y linters discovered in Step 0), then apply **LLM triage** to each
|
|
105
91
|
candidate — a mechanical hit is a _candidate_, not automatically a finding.
|
|
@@ -173,71 +159,11 @@ Corroborate static findings against the runtime results (a statically-flagged
|
|
|
173
159
|
contrast defect confirmed by the engine graduates from provisional to
|
|
174
160
|
confirmed), and surface runtime-only violations the static pass could not see.
|
|
175
161
|
|
|
176
|
-
##
|
|
177
|
-
|
|
178
|
-
Generate and save a structured Markdown report to
|
|
179
|
-
`{{auditOutputDir}}/audit-accessibility-results.md`, using the exact template
|
|
180
|
-
below. The report MUST include all sections, even if empty (write
|
|
181
|
-
"_No findings._" rather than omitting a section).
|
|
182
|
-
|
|
183
|
-
> Grade every finding's severity on the shared
|
|
184
|
-
> [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md), and
|
|
185
|
-
> key every finding to the WCAG success criterion it violates.
|
|
186
|
-
|
|
187
|
-
```markdown
|
|
188
|
-
# Accessibility (WCAG) Audit report
|
|
189
|
-
|
|
190
|
-
## Executive Summary
|
|
191
|
-
|
|
192
|
-
[Overview of WCAG conformance health across the scope, the runtime mode's
|
|
193
|
-
status (ran against `<env>` / skipped — no target configured), and the
|
|
194
|
-
self-cross-check line.]
|
|
195
|
-
|
|
196
|
-
## Detailed Findings
|
|
197
|
-
|
|
198
|
-
[For every WCAG violation identified, use the following strict structure. Lead
|
|
199
|
-
each title with the primary file the finding lives in:]
|
|
200
|
-
|
|
201
|
-
### `path/to/primary-file.ext` — [Short title of the issue]
|
|
202
|
-
|
|
203
|
-
- **Dimension:** [e.g., Semantic Structure | ARIA | Keyboard & Focus | Forms & Labels | Media Alternatives | Contrast]
|
|
204
|
-
- **Severity:** [Critical | High | Medium | Low]
|
|
205
|
-
- **WCAG:** [success criterion — e.g. `1.1.1 Non-text Content (A)`]
|
|
206
|
-
- **Location:** `path/to/primary-file.ext:line`
|
|
207
|
-
- **Evidence:** [measured | static] [the observable — a quoted element, the
|
|
208
|
-
computed contrast ratio, the failing axe/Lighthouse audit id + median score.
|
|
209
|
-
Runtime numbers from a single run are tagged `provisional`.]
|
|
210
|
-
- **Current State:** [what is implemented and why it fails the criterion]
|
|
211
|
-
- **Recommendation & Rationale:** [the specific change — attribute to add,
|
|
212
|
-
element to swap, token to adjust — and the assistive-technology behaviour it
|
|
213
|
-
restores]
|
|
214
|
-
- **Acceptance signal:** [the command or observable that proves this finding is
|
|
215
|
-
remediated — e.g. the axe rule now passing on the route, or a re-run of this
|
|
216
|
-
lens]
|
|
217
|
-
- **Agent Prompt:**
|
|
218
|
-
`[A copy-pasteable, highly specific prompt to execute this a11y fix independently]`
|
|
219
|
-
|
|
220
|
-
## Runtime Verification
|
|
221
|
-
|
|
222
|
-
[Per-route median-of-3 accessibility scores when the runtime mode ran, or
|
|
223
|
-
"_Runtime corroboration unavailable — no `qa.environments` target configured._"]
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
## Constraint
|
|
227
|
-
|
|
228
|
-
This is a **read-only** audit. Provide the critique and implementation
|
|
229
|
-
suggestions, but do not modify components, styles, or configuration. The
|
|
230
|
-
runtime mode runs **non-mutating** measurements only and starts no arbitrary
|
|
231
|
-
dev server.
|
|
232
|
-
|
|
233
|
-
## Self-cross-check (mandatory — filter false positives before you finalize)
|
|
162
|
+
## Report additions
|
|
234
163
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
the orchestrated path's adversarial reviewer does. Drop every claimed
|
|
242
|
-
violation that names no concrete element and no specific WCAG success
|
|
243
|
-
criterion.
|
|
164
|
+
Beyond the shared skeleton, the Executive Summary states the runtime mode's
|
|
165
|
+
status (ran against `<env>` / skipped — no target configured), and the report
|
|
166
|
+
ends with a **Runtime Verification** section: per-route median-of-3
|
|
167
|
+
accessibility scores when the runtime mode ran, or "_Runtime corroboration
|
|
168
|
+
unavailable — no `qa.environments` target configured._" Drop every claimed
|
|
169
|
+
violation that names no concrete element and no specific WCAG success criterion.
|