@brunosps00/dev-workflow 2.2.0 → 2.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +46 -0
- package/README.md +12 -4
- package/lib/constants.js +24 -24
- package/lib/init.js +8 -0
- package/lib/instruction-health.js +38 -0
- package/lib/post-update.js +2 -1
- package/lib/validate.js +3 -1
- package/package.json +1 -1
- package/scaffold/config/routing.json +61 -23
- package/scaffold/en/agent-instructions.md +31 -137
- package/scaffold/en/commands/dw-analyze-project.md +2 -0
- package/scaffold/en/commands/dw-autopilot.md +22 -245
- package/scaffold/en/commands/dw-bugfix.md +9 -35
- package/scaffold/en/commands/dw-claude-run.md +29 -45
- package/scaffold/en/commands/dw-codex-run.md +30 -45
- package/scaffold/en/commands/dw-context-budget.md +6 -14
- package/scaffold/en/commands/dw-copilot-run.md +29 -41
- package/scaffold/en/commands/dw-generate-pr.md +2 -2
- package/scaffold/en/commands/dw-goal.md +11 -5
- package/scaffold/en/commands/dw-help.md +2 -2
- package/scaffold/en/commands/dw-plan.md +14 -7
- package/scaffold/en/commands/dw-qa.md +2 -0
- package/scaffold/en/commands/dw-resume.md +5 -11
- package/scaffold/en/commands/dw-review.md +4 -3
- package/scaffold/en/commands/dw-run.md +19 -166
- package/scaffold/en/commands/dw-skill-health.md +7 -0
- package/scaffold/en/commands/dw-update.md +18 -4
- package/scaffold/en/references/command-routing.md +53 -0
- package/scaffold/en/references/execution-contract.md +55 -0
- package/scaffold/en/templates/frontend-quality-template.md +43 -0
- package/scaffold/en/templates/task-template.md +20 -40
- package/scaffold/en/templates/tasks-template.md +9 -6
- package/scaffold/en/templates/techspec-template.md +4 -0
- package/scaffold/pt-br/agent-instructions.md +31 -137
- package/scaffold/pt-br/commands/dw-analyze-project.md +2 -0
- package/scaffold/pt-br/commands/dw-autopilot.md +22 -245
- package/scaffold/pt-br/commands/dw-bugfix.md +9 -35
- package/scaffold/pt-br/commands/dw-claude-run.md +29 -46
- package/scaffold/pt-br/commands/dw-codex-run.md +30 -45
- package/scaffold/pt-br/commands/dw-context-budget.md +6 -14
- package/scaffold/pt-br/commands/dw-copilot-run.md +29 -43
- package/scaffold/pt-br/commands/dw-generate-pr.md +2 -2
- package/scaffold/pt-br/commands/dw-goal.md +11 -5
- package/scaffold/pt-br/commands/dw-help.md +2 -2
- package/scaffold/pt-br/commands/dw-plan.md +14 -7
- package/scaffold/pt-br/commands/dw-qa.md +2 -0
- package/scaffold/pt-br/commands/dw-resume.md +7 -11
- package/scaffold/pt-br/commands/dw-review.md +4 -3
- package/scaffold/pt-br/commands/dw-run.md +19 -166
- package/scaffold/pt-br/commands/dw-skill-health.md +7 -0
- package/scaffold/pt-br/commands/dw-update.md +18 -4
- package/scaffold/pt-br/references/command-routing.md +53 -0
- package/scaffold/pt-br/references/execution-contract.md +55 -0
- package/scaffold/pt-br/templates/frontend-quality-template.md +43 -0
- package/scaffold/pt-br/templates/task-template.md +20 -40
- package/scaffold/pt-br/templates/tasks-template.md +9 -6
- package/scaffold/pt-br/templates/techspec-template.md +4 -0
- package/scaffold/scripts/lib/workflow-contract.mjs +101 -0
- package/scaffold/skill-registry.json +23 -23
- package/scaffold/skills/dw-cli-run/SKILL.md +33 -217
- package/scaffold/skills/dw-cli-run/references/dispatch-tuning.md +14 -72
- package/scaffold/skills/dw-codebase-intel/SKILL.md +5 -39
- package/scaffold/skills/dw-codebase-intel/agents/intel-updater.md +1 -1
- package/scaffold/skills/dw-codebase-intel/references/agents-detail.md +7 -0
- package/scaffold/skills/dw-codebase-intel/references/files-in-dw-intel-detail.md +16 -0
- package/scaffold/skills/dw-codebase-intel/references/how-dw-intel-reads-this-detail.md +12 -0
- package/scaffold/skills/dw-codebase-intel/references/how-to-compose-the-typical-flow-detail.md +11 -0
- package/scaffold/skills/dw-council/SKILL.md +2 -17
- package/scaffold/skills/dw-council/references/integration-with-other-dw-commands-detail.md +7 -0
- package/scaffold/skills/dw-council/references/phase-2-tensions-and-rebuttals-detail.md +15 -0
- package/scaffold/skills/dw-domain-modeling/SKILL.md +1 -1
- package/scaffold/skills/dw-execute-phase/SKILL.md +12 -32
- package/scaffold/skills/dw-execute-phase/agents/executor.md +15 -252
- package/scaffold/skills/dw-execute-phase/agents/plan-checker.md +4 -207
- package/scaffold/skills/dw-execute-phase/references/atomic-commits.md +8 -18
- package/scaffold/skills/dw-execute-phase/references/de-sloppify-pass-optional-cleanup-task-detail.md +12 -0
- package/scaffold/skills/dw-execute-phase/references/deviation-rules-during-execution-detail.md +8 -0
- package/scaffold/skills/dw-execute-phase/references/how-the-two-agents-compose-detail.md +7 -0
- package/scaffold/skills/dw-execute-phase/references/plan-verification.md +9 -155
- package/scaffold/skills/dw-execute-phase/references/wave-coordination.md +6 -97
- package/scaffold/skills/dw-git-discipline/SKILL.md +2 -2
- package/scaffold/skills/dw-grilling/SKILL.md +1 -1
- package/scaffold/skills/dw-incident-response/SKILL.md +1 -64
- package/scaffold/skills/dw-incident-response/references/the-five-phases-detail.md +66 -0
- package/scaffold/skills/dw-llm-eval/SKILL.md +5 -53
- package/scaffold/skills/dw-llm-eval/references/agent-tool-use-evaluation-detail.md +23 -0
- package/scaffold/skills/dw-llm-eval/references/anti-patterns-will-block-in-dw-review-code-only-detail.md +9 -0
- package/scaffold/skills/dw-llm-eval/references/consistency-run-to-run-stability-detail.md +10 -0
- package/scaffold/skills/dw-llm-eval/references/integration-with-dev-workflow-commands-detail.md +6 -0
- package/scaffold/skills/dw-llm-eval/references/the-oracle-ladder-detail.md +15 -0
- package/scaffold/skills/dw-memory/SKILL.md +4 -50
- package/scaffold/skills/dw-memory/references/context-budget.md +8 -56
- package/scaffold/skills/dw-memory/references/promotion-decision-test-detail.md +30 -0
- package/scaffold/skills/dw-minimalism/SKILL.md +4 -5
- package/scaffold/skills/dw-review-rigor/SKILL.md +1 -54
- package/scaffold/skills/dw-review-rigor/references/the-five-rules-detail.md +56 -0
- package/scaffold/skills/dw-simplification/SKILL.md +2 -55
- package/scaffold/skills/dw-simplification/references/pattern-recognition-detail.md +11 -0
- package/scaffold/skills/dw-simplification/references/the-five-rules-detail.md +48 -0
- package/scaffold/skills/dw-source-grounding/SKILL.md +1 -56
- package/scaffold/skills/dw-source-grounding/references/the-protocol-detect-fetch-implement-cite-detail.md +58 -0
- package/scaffold/skills/dw-testing-discipline/SKILL.md +25 -177
- package/scaffold/skills/dw-testing-discipline/references/agent-guardrails.md +9 -167
- package/scaffold/skills/dw-testing-discipline/references/core-rules.md +3 -3
- package/scaffold/skills/dw-testing-discipline/references/mutation-testing.md +19 -0
- package/scaffold/skills/dw-testing-discipline/references/patterns.md +3 -12
- package/scaffold/skills/dw-ui-discipline/SKILL.md +2 -59
- package/scaffold/skills/dw-ui-discipline/references/frontend-engineering.md +46 -0
- package/scaffold/skills/dw-ui-discipline/references/the-four-grounding-questions-detail.md +61 -0
- package/scaffold/skills/dw-verify/SKILL.md +26 -148
- package/scaffold/skills/dw-verify/references/verification-discovery-and-attribution.md +2 -2
- package/scaffold/skills/humanizer/SKILL.md +2 -1
- package/scaffold/skills/security-review/SKILL.md +3 -183
- package/scaffold/skills/security-review/references/do-not-flag-detail.md +49 -0
- package/scaffold/skills/security-review/references/quick-patterns-reference-detail.md +56 -0
- package/scaffold/skills/security-review/references/review-process-detail.md +84 -0
- package/scaffold/skills/vercel-react-best-practices/SKILL.md +1 -90
- package/scaffold/skills/vercel-react-best-practices/references/quick-reference-detail.md +92 -0
|
@@ -5,191 +5,39 @@ allowed-tools:
|
|
|
5
5
|
- Read
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
# Testing
|
|
8
|
+
# Testing discipline
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
Use for test design, test review or diagnosing failures/flakiness. Skip cosmetic test edits and reproductions that do not need a testing workflow. Tests expose defects in required behavior; a passing mock or coverage number does not establish correctness.
|
|
11
11
|
|
|
12
|
-
|
|
13
|
-
> A test that fails has done its job.
|
|
14
|
-
> A test that passes for the wrong reason is worse than no test.
|
|
12
|
+
## Core decisions
|
|
15
13
|
|
|
16
|
-
|
|
14
|
+
- Identify the behavior and lowest layer that can detect its regression. Extend an existing suite when it fits; do not create test-only production seams.
|
|
15
|
+
- Read the project's runner/configuration before choosing commands. Package manager and runner are different: `bun test` and `bun run test` need not run the same suite.
|
|
16
|
+
- Run actual production logic. Mock external boundaries only when justified, and cover integration boundaries with the appropriate real/contract test. Do not require a live external service for every unit change.
|
|
17
|
+
- On failures, inspect production behavior and expected contract before changing assertions. Explain changed expectations when requirements changed; do not weaken tests to obtain green output.
|
|
18
|
+
- Test relevant invalid/error cases, not a mandatory negative companion for every positive assertion. Avoid fixed sleeps and accidental clock/random/environment dependencies.
|
|
19
|
+
- Use project-required thresholds; no universal coverage target, assertion-count cap or mock-size gate. Assess whether the test can catch the intended defect.
|
|
17
20
|
|
|
18
|
-
##
|
|
21
|
+
## Read only the relevant reference
|
|
19
22
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
23
|
+
| Task | Reference |
|
|
24
|
+
|---|---|
|
|
25
|
+
| Choose layer or boundary | `references/core-rules.md` |
|
|
26
|
+
| Design tests | `references/patterns.md` |
|
|
27
|
+
| Review a concrete test smell | `references/anti-patterns.md` |
|
|
28
|
+
| Agent needs test-authoring checks | `references/agent-guardrails.md` |
|
|
29
|
+
| Explicit TDD / test first / red-green-refactor | `references/tdd-loop.md` |
|
|
30
|
+
| Diagnose flaky tests | `references/flaky-discipline.md` |
|
|
31
|
+
| Investigate weak tests or plan scoped mutation analysis | `references/mutation-testing.md` |
|
|
32
|
+
| Playwright browser tests | `references/playwright-recipes.md` |
|
|
33
|
+
| Browser auth/CSRF/header boundary | `references/security-boundary.md` |
|
|
34
|
+
| Choose UI/network/performance workflow | `references/three-workflow-patterns.md` |
|
|
28
35
|
|
|
29
|
-
|
|
36
|
+
The task/PR may record behavior, owning layer and suite in ordinary prose. Do not demand magic preamble strings or a new approval merely to name an already-defined invariant. Ambiguous product behavior needs clarification; discoverable runner/suite choices do not.
|
|
30
37
|
|
|
31
|
-
##
|
|
38
|
+
## Verification
|
|
32
39
|
|
|
33
|
-
-
|
|
34
|
-
- Reviewing a PR diff under test paths.
|
|
35
|
-
- Debugging a flaky test (or considering retry-as-fix — read `references/flaky-discipline.md` first).
|
|
36
|
-
- Generating tests via an AI agent → invokes `references/agent-guardrails.md` automatically.
|
|
37
|
-
- Explicit test-first requests ("TDD", "test first", "red-green-refactor") → use `references/tdd-loop.md` as an on-demand operating mode, not the default workflow.
|
|
38
|
-
- Browser-based E2E with Playwright → recipes in `references/playwright-recipes.md`.
|
|
39
|
-
- Verifying browser-side trust boundaries (auth, CSRF, headers) → `references/security-boundary.md`.
|
|
40
|
-
- Picking which test workflow applies (UI / network / perf) → `references/three-workflow-patterns.md`.
|
|
41
|
-
|
|
42
|
-
## Step 0 — detect the runner before writing any test
|
|
43
|
-
|
|
44
|
-
Do not assume `npm test`. The package manager and the test runner are different things — confirm both:
|
|
45
|
-
|
|
46
|
-
1. Read `package.json` `scripts.test` (plus `test:watch` / `test:coverage`), or the language equivalent (`pyproject.toml`/`tox.ini`, `Cargo.toml`, `*.csproj`, `go.mod`).
|
|
47
|
-
2. Identify the actual runner: jest / vitest / mocha / `node:test` / `bun:test` / pytest / `go test` / `cargo test` / `dotnet test`.
|
|
48
|
-
3. Invoke it through the project's package manager, not a hardcoded `npm`:
|
|
49
|
-
|
|
50
|
-
| Manager | test | watch | coverage |
|
|
51
|
-
|---|---|---|---|
|
|
52
|
-
| npm | `npm test` | `npm test -- --watch` | `npm run test:coverage` |
|
|
53
|
-
| pnpm | `pnpm test` | `pnpm test --watch` | `pnpm test:coverage` |
|
|
54
|
-
| yarn | `yarn test` | `yarn test --watch` | `yarn coverage` |
|
|
55
|
-
| bun (native) | `bun test` | `bun test --watch` | `bun test --coverage` |
|
|
56
|
-
|
|
57
|
-
**Bun gotcha:** `bun test` runs Bun's native runner (`bun:test`); `bun run test` runs whatever `scripts.test` is (often jest/vitest). They are not interchangeable — use the one the project actually configures. When unsure, confirm with `<runner> --help`.
|
|
58
|
-
|
|
59
|
-
## Reference router
|
|
60
|
-
|
|
61
|
-
| Doing what | Read |
|
|
62
|
-
|------------|------|
|
|
63
|
-
| Placing a new test (which layer?) | `references/core-rules.md` (Rule 2 deep-dive) |
|
|
64
|
-
| Writing new tests | `references/patterns.md` |
|
|
65
|
-
| Explicit test-first / TDD loop | `references/tdd-loop.md` |
|
|
66
|
-
| Reviewing tests / spotting smells | `references/anti-patterns.md` |
|
|
67
|
-
| Agent-generated tests | `references/agent-guardrails.md` + `references/anti-patterns.md` |
|
|
68
|
-
| Flaky tests | `references/flaky-discipline.md` |
|
|
69
|
-
| Playwright E2E | `references/playwright-recipes.md` |
|
|
70
|
-
| Browser trust boundary | `references/security-boundary.md` |
|
|
71
|
-
| Picking the right workflow | `references/three-workflow-patterns.md` |
|
|
72
|
-
|
|
73
|
-
## Patterns that produce reliable tests (one-liners; full in `references/patterns.md`)
|
|
74
|
-
|
|
75
|
-
1. Query by behavior and accessible role; never CSS selectors or DOM indices.
|
|
76
|
-
2. Selector ladder: role → label → text → test-id → structural. Stop at the highest rung that disambiguates.
|
|
77
|
-
3. Wait on observable conditions; never wall-clock sleeps.
|
|
78
|
-
4. Each test independent and order-free; lean on `beforeEach`, not `beforeAll`.
|
|
79
|
-
5. One behavior per test; as many assertions as that behavior requires.
|
|
80
|
-
6. Names read as specifications: `should <outcome> when <condition> given <state>`.
|
|
81
|
-
7. Table-driven / parameterized when inputs vary.
|
|
82
|
-
8. Build test data via factories; literal blobs only for fields under test.
|
|
83
|
-
9. Mock at boundaries you don't control; real wiring for the systems you own.
|
|
84
|
-
10. Real systems gate the final merge; contract tests bridge unit and E2E.
|
|
85
|
-
11. Mutation score, not coverage percentage, measures suite strength.
|
|
86
|
-
12. Page Object Model is a tool; collapse it for small suites where it adds noise.
|
|
87
|
-
|
|
88
|
-
## Anti-pattern catalog (four families, full in `references/anti-patterns.md`)
|
|
89
|
-
|
|
90
|
-
The four kinds of smell that produce most test debt:
|
|
91
|
-
|
|
92
|
-
**A. Fragile to refactor** — tests bound to internals, not behavior:
|
|
93
|
-
- Implementation-detail selectors.
|
|
94
|
-
- Asserting internal structure instead of observable outcome.
|
|
95
|
-
- Testing private methods directly.
|
|
96
|
-
- Snapshots replacing real assertions.
|
|
97
|
-
- Vague existence assertions (`toBeTruthy`, `should('exist')`).
|
|
98
|
-
- Actions with no assertion ("clicking save works").
|
|
99
|
-
|
|
100
|
-
**B. Non-deterministic outcomes** — tests that flip verdict on the same code:
|
|
101
|
-
- Static sleeps / fixed-timeout waits.
|
|
102
|
-
- Test order dependency / hidden shared state.
|
|
103
|
-
- Non-deterministic inputs (clock, RNG, locale).
|
|
104
|
-
|
|
105
|
-
**C. Mock-driven false confidence** — tests testing the test setup:
|
|
106
|
-
- Asserting the mock exists.
|
|
107
|
-
- Mock drift (mocked response no longer matches real API).
|
|
108
|
-
- Over-mocking child components.
|
|
109
|
-
- Incomplete mocks (missing fields the code reads).
|
|
110
|
-
- Mocking the wrong level (mocking methods of the SUT itself).
|
|
111
|
-
- Asserting on a value the test body fed into a mock.
|
|
112
|
-
|
|
113
|
-
**D. Suite hygiene problems** — team and suite-level pathologies:
|
|
114
|
-
- Coverage as vanity metric.
|
|
115
|
-
- Happy-path-only coverage.
|
|
116
|
-
- Eternal `beforeAll` hiding dependencies.
|
|
117
|
-
- Cleanup in `afterEach` (move to `beforeEach`).
|
|
118
|
-
- Magic strings and logic in tests.
|
|
119
|
-
- Testing against third-party sites.
|
|
120
|
-
- Quarantine-as-cemetery (skip without owner or deadline).
|
|
121
|
-
- Retry-as-fix (auto-retry hiding real bugs).
|
|
122
|
-
- Duplicate tests across pyramid layers.
|
|
123
|
-
- Weakening assertions to make tests pass.
|
|
124
|
-
|
|
125
|
-
Total: 25 specific patterns across the four families.
|
|
126
|
-
|
|
127
|
-
## Agent-authoring guardrails (mandatory when an LLM writes tests)
|
|
128
|
-
|
|
129
|
-
Six guardrails block the most common failure modes when an LLM produces test code. Each is a pre-condition before the diff goes to review. Full prompts and verification in `references/agent-guardrails.md`:
|
|
130
|
-
|
|
131
|
-
1. **State the invariant first** — agent prints `INVARIANT`, `OWNING_LAYER`, `EXISTING_SUITE` before writing code.
|
|
132
|
-
2. **Extend, don't sprawl** — agent extends an existing suite; new files require a named invariant.
|
|
133
|
-
3. **Real execution somewhere** — at least one test path runs against real systems before merge.
|
|
134
|
-
4. **Red? Read production** — on failure, the agent reads production code first and prints `ANALYSIS:` before changing tests.
|
|
135
|
-
5. **Classify before snapshot** — snapshots only with explicit `PRODUCT_CONTRACT` classification; `IMPLEMENTATION_DETAIL` forbids them.
|
|
136
|
-
6. **Negative companion** — every positive assertion ships with a negative test for invalid input or failure mode.
|
|
137
|
-
|
|
138
|
-
## Placement doctrine (tripwires)
|
|
139
|
-
|
|
140
|
-
Before writing test code:
|
|
141
|
-
|
|
142
|
-
- Name the invariant in **one sentence**. Fuzzy language signals unclear requirements — stop and clarify.
|
|
143
|
-
- Place the test at the **lowest layer** capable of detecting the defect when the invariant breaks.
|
|
144
|
-
- Reject tests where (`likelihood-of-bug` × `blast-radius`) falls below a ten-minute-maintenance threshold (the test is more expensive to maintain than the bug would be to fix).
|
|
145
|
-
|
|
146
|
-
## Flaky discipline (tripwires)
|
|
147
|
-
|
|
148
|
-
- Quarantine flaky tests within ONE HOUR of detection. Assign a named owner within 24 hours with a fix-by date.
|
|
149
|
-
- Track `flaky_rate` as a first-class metric: SLO under 1–2%; alert at >5%.
|
|
150
|
-
- Real systems at the final gate: mock at unit; contract-test boundaries; real DB/queue/route at integration; near-zero mocks at E2E.
|
|
151
|
-
|
|
152
|
-
Full taxonomy in `references/flaky-discipline.md`.
|
|
153
|
-
|
|
154
|
-
## Cross-cutting red flags
|
|
155
|
-
|
|
156
|
-
Any of these in a PR is enough to REJECT a verdict:
|
|
157
|
-
|
|
158
|
-
- Mock setup larger than the test logic.
|
|
159
|
-
- Test breaks when an internal method is renamed (not the public contract).
|
|
160
|
-
- Removing the assertion body leaves the test green.
|
|
161
|
-
- Test fails when run with `.only` in isolation.
|
|
162
|
-
- `sleep`, `Thread.sleep`, or `cy.wait(<number>)` appears.
|
|
163
|
-
- Selector contains CSS class, index, or `xpath`.
|
|
164
|
-
- Test asserts a third-party site is reachable.
|
|
165
|
-
- Snapshot diffs accepted without reading.
|
|
166
|
-
- Coverage percentage is the only metric quoted.
|
|
167
|
-
- Failing tests auto-retried until green; no investigation.
|
|
168
|
-
- Skipped/quarantined tests without named owner and fix-by date.
|
|
169
|
-
- Test depends on `new Date()`, `Math.random()`, or system locale.
|
|
170
|
-
- `afterEach` resets database state.
|
|
171
|
-
- Agent-written test has 6+ assertions and zero edge cases.
|
|
172
|
-
- The diff contains the phrase "I'll mock this to be safe."
|
|
173
|
-
|
|
174
|
-
## When NOT to use this skill
|
|
175
|
-
|
|
176
|
-
- General code review unrelated to tests.
|
|
177
|
-
- Library-specific debugging where the test is just a reproduction.
|
|
178
|
-
- Non-testing CI pipeline design (deploys, artifacts, secrets).
|
|
179
|
-
- Production observability and alerting.
|
|
180
|
-
- Single-line typo fixes in existing tests.
|
|
181
|
-
|
|
182
|
-
## Integration with dev-workflow commands
|
|
183
|
-
|
|
184
|
-
- `/dw-plan tasks` applies the placement doctrine — each test-adding task names the invariant.
|
|
185
|
-
- `/dw-run` runs the 6 agent guardrails when generating tests during implementation. When the task or user explicitly requests test-first work, `/dw-run` switches to `references/tdd-loop.md` for that task only.
|
|
186
|
-
- `/dw-review --code-only` runs the anti-pattern checks on diff hunks under test paths.
|
|
187
|
-
- `/dw-qa --fix` applies the flaky-discipline taxonomy in retest cycles.
|
|
188
|
-
- `/dw-qa` (UI mode) references `playwright-recipes.md` for concrete recipes.
|
|
189
|
-
|
|
190
|
-
## Bottom line
|
|
191
|
-
|
|
192
|
-
> A test that cannot fail is decorative. A test that fails for the wrong reason is misleading. Build tests that fail for exactly one reason — the reason the invariant was violated — and trust them when they do. Mocks isolate. Real systems validate. Coverage shines a light. Mutation score grades the suite. Agents will reach for the mock and the snapshot; the guardrails make them put both down. Tests reveal bugs, not just pass.
|
|
40
|
+
Use `dw-verify` for evidence validity and required delivery gates. Run affected tests while fixing, then complete invalidated required checks. Reuse equivalent passing evidence. Keep scenario-specific test discipline separate from generic CI/production policies.
|
|
193
41
|
|
|
194
42
|
## Structured Return
|
|
195
43
|
|
|
@@ -1,170 +1,12 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Test-authoring checks
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Read when designing or reviewing agent-authored tests. Apply checks to actual behavior rather than mandatory output tokens.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
1. Identify the behavior, owning layer and existing suite. Infer these from approved requirements and repository evidence; ask only if expected product behavior remains ambiguous.
|
|
6
|
+
2. Execute production logic. Use a real DB/integration fixture when the changed contract depends on it, a contract test for external boundaries, and browser tests for meaningful interactive risk. Pure logic can be verified in unit tests.
|
|
7
|
+
3. Investigate failures before changing tests. Distinguish a code defect, outdated expectation, environment issue or unrelated failure, and explain material expectation changes.
|
|
8
|
+
4. Reject self-confirming mock assertions. Snapshots require a meaningful stable contract and inspected diffs; prefer direct assertions when they express behavior more clearly.
|
|
9
|
+
5. Include relevant error/invalid-input cases where they can detect a real defect. A fixed negative-test quota or assertion count does not measure coverage quality.
|
|
10
|
+
6. Extend existing suites when suitable and avoid production branches solely for tests. Keep new files when a separate behavior boundary justifies them.
|
|
6
11
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
**Failure mode it blocks:** agent writes 200 lines of test code without articulating what the test is supposed to prove or where it belongs.
|
|
10
|
-
|
|
11
|
-
**What the agent must do:**
|
|
12
|
-
|
|
13
|
-
Before any test code, print:
|
|
14
|
-
|
|
15
|
-
```
|
|
16
|
-
INVARIANT: <one sentence — what behavior the test verifies>
|
|
17
|
-
OWNING_LAYER: <unit | integration | contract | e2e>
|
|
18
|
-
EXISTING_SUITE: <path to the existing test file the new test joins, or "NEW: <reason>">
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
If the agent can't fill any line, it stops and asks the user — it does NOT invent an invariant.
|
|
22
|
-
|
|
23
|
-
**Why it works:**
|
|
24
|
-
- "Invariant" forces specific behavior naming.
|
|
25
|
-
- "Owning layer" forces Rule 2 (lowest detectable layer).
|
|
26
|
-
- "Existing suite" forces extending coverage rather than spawning orphan files.
|
|
27
|
-
|
|
28
|
-
**Verification:** `/dw-review --code-only` looks for this 3-line preamble in the PR description or commit body. Missing = REJECTED.
|
|
29
|
-
|
|
30
|
-
## Guardrail 2 — Real execution somewhere
|
|
31
|
-
|
|
32
|
-
**Failure mode it blocks:** agent writes tests that mock everything. They pass green forever and validate nothing.
|
|
33
|
-
|
|
34
|
-
**What the agent must do:**
|
|
35
|
-
|
|
36
|
-
At SOME layer, the test path must run against real systems before merge:
|
|
37
|
-
|
|
38
|
-
- Pure logic: unit only is sufficient.
|
|
39
|
-
- Code touching DB: at least one integration test with real DB (testcontainers, ephemeral container, dedicated test DB).
|
|
40
|
-
- Code calling external services: a contract test OR a sandbox-account smoke test.
|
|
41
|
-
- UI interactions: at least one E2E run on a real preview environment.
|
|
42
|
-
|
|
43
|
-
**Verification:** PR description lists the real-system runs covering the touched code. If no real-system path covers the change → REJECTED.
|
|
44
|
-
|
|
45
|
-
## Guardrail 3 — On red, read production first
|
|
46
|
-
|
|
47
|
-
**Failure mode it blocks:** agent sees a test go red and modifies the test until green. Bug ships.
|
|
48
|
-
|
|
49
|
-
**What the agent must do:**
|
|
50
|
-
|
|
51
|
-
When a test fails (its own or pre-existing):
|
|
52
|
-
|
|
53
|
-
1. Print: `INVESTIGATING FAILURE: <test name>`.
|
|
54
|
-
2. Read production code in the path that produced the observed value.
|
|
55
|
-
3. Print: `ANALYSIS: <2-3 sentences — is production wrong, the test wrong, or has the invariant changed?>`.
|
|
56
|
-
4. Decide:
|
|
57
|
-
- Production wrong → fix production.
|
|
58
|
-
- Test wrong → fix the test AND document the change in the commit body.
|
|
59
|
-
- Invariant changed → update the test AND open an ADR if it's a public-contract change.
|
|
60
|
-
|
|
61
|
-
**Verification:** every commit changing a previously-green test must have an `ANALYSIS:` line. Missing = REJECTED.
|
|
62
|
-
|
|
63
|
-
## Guardrail 4 — No self-confirming assertions
|
|
64
|
-
|
|
65
|
-
**Failure mode it blocks:** two related shortcuts —
|
|
66
|
-
- Agent writes `mockFn.mockReturnValue('X')` then asserts `expect(mockFn()).toBe('X')`. Proves nothing — the test asserts what the test set up.
|
|
67
|
-
- Agent reaches for `toMatchSnapshot()` whenever unsure what to assert. The snapshot becomes the assertion; drift goes unnoticed.
|
|
68
|
-
|
|
69
|
-
**What the agent must do:**
|
|
70
|
-
|
|
71
|
-
**For mocks:** never assert on a value the test body fed into a mock. Assert on:
|
|
72
|
-
- The OUTPUT of production code that consumed the mock.
|
|
73
|
-
- The SIDE EFFECTS (DB state, network calls, event emissions) caused by production code.
|
|
74
|
-
- The VISIBLE behavior (UI change, log line, response) the user/caller observes.
|
|
75
|
-
|
|
76
|
-
**For snapshots:** before adding `toMatchSnapshot()`, classify the artifact:
|
|
77
|
-
- `PRODUCT_CONTRACT` — a stable contract worth pinning (serialized API output, stored-record schema). Snapshot OK; document the classification in a comment.
|
|
78
|
-
- `IMPLEMENTATION_DETAIL` — HTML structure, internal representation, component tree shape. Snapshot is FORBIDDEN; write specific assertions instead.
|
|
79
|
-
|
|
80
|
-
**Verification:**
|
|
81
|
-
- Mock value flowing directly from setup to assertion without passing through production code → REJECTED.
|
|
82
|
-
- Snapshot in diff without classification comment → REJECTED.
|
|
83
|
-
- Snapshot classified `IMPLEMENTATION_DETAIL` → REJECTED.
|
|
84
|
-
|
|
85
|
-
## Guardrail 5 — Negative companion
|
|
86
|
-
|
|
87
|
-
**Failure mode it blocks:** agent writes happy-path-only tests. Edge cases, error paths, boundary inputs uncovered.
|
|
88
|
-
|
|
89
|
-
**What the agent must do:**
|
|
90
|
-
|
|
91
|
-
Every positive assertion ships WITH at least one negative companion:
|
|
92
|
-
|
|
93
|
-
- Asserting `createUser(validInput)` succeeds → also assert `createUser(invalidInput)` fails with a specific error.
|
|
94
|
-
- Asserting `parseDate(validString)` returns a Date → also assert `parseDate(invalidString)` throws or returns null.
|
|
95
|
-
- Asserting `transferFunds(...)` succeeds with sufficient balance → also assert it fails with insufficient balance.
|
|
96
|
-
|
|
97
|
-
**Verification:** a PR adding N positive assertions to a public path must add ≥1 negative assertion. Imbalance >3:1 (positive:negative) on a public path → REJECTED.
|
|
98
|
-
|
|
99
|
-
## Guardrail 6 — Don't expand the surface to test it
|
|
100
|
-
|
|
101
|
-
**Failure mode it blocks:** agent exports internals, adds `*ForTesting` methods, or introduces `process.env.NODE_ENV === 'test'` branches in production code to make the test possible.
|
|
102
|
-
|
|
103
|
-
**What the agent must do:**
|
|
104
|
-
|
|
105
|
-
If the test needs access the production API doesn't grant:
|
|
106
|
-
- **Refactor production for testability** via dependency injection or interface seams.
|
|
107
|
-
- **Emit an observable side effect** the test can verify (event, log line, metric) that production also benefits from.
|
|
108
|
-
- **Use a dedicated test environment** with test credentials, not a backdoor flag.
|
|
109
|
-
|
|
110
|
-
The agent does NOT:
|
|
111
|
-
- Export `_internal` symbols just for tests.
|
|
112
|
-
- Add `// for testing only` methods on classes.
|
|
113
|
-
- Wrap production logic in `if (process.env.NODE_ENV !== 'test')` branches.
|
|
114
|
-
|
|
115
|
-
**Verification:** diff includes new production exports, env checks, or `*ForTesting`-style symbols → REJECTED. Refactor the surface or change the test approach.
|
|
116
|
-
|
|
117
|
-
## How the six guardrails compose
|
|
118
|
-
|
|
119
|
-
A test that passes all six:
|
|
120
|
-
1. States the invariant, layer, and host suite up front (Guardrail 1).
|
|
121
|
-
2. Exercises real systems somewhere in the pipeline (Guardrail 2).
|
|
122
|
-
3. When red, reads production first and documents the analysis (Guardrail 3).
|
|
123
|
-
4. Asserts on production's observable output, not its own setup (Guardrail 4).
|
|
124
|
-
5. Covers failures alongside successes (Guardrail 5).
|
|
125
|
-
6. Lives inside the production API's existing surface (Guardrail 6).
|
|
126
|
-
|
|
127
|
-
Tests passing all six are worth running. Tests missing any one are more likely to mislead than to help.
|
|
128
|
-
|
|
129
|
-
## Override procedure
|
|
130
|
-
|
|
131
|
-
To skip a guardrail explicitly:
|
|
132
|
-
1. State which guardrail is skipped.
|
|
133
|
-
2. State why in one sentence.
|
|
134
|
-
3. Add a `// SKIP-GUARDRAIL-N: <reason>` comment in the test.
|
|
135
|
-
4. Open a follow-up issue tracking the gap.
|
|
136
|
-
|
|
137
|
-
Without all four, the guardrail is enforced.
|
|
138
|
-
|
|
139
|
-
## Prompt block injected when an agent writes tests
|
|
140
|
-
|
|
141
|
-
```
|
|
142
|
-
You are about to write tests. Before producing test code, complete the
|
|
143
|
-
6-guardrail preamble:
|
|
144
|
-
|
|
145
|
-
INVARIANT: ___
|
|
146
|
-
OWNING_LAYER: ___
|
|
147
|
-
EXISTING_SUITE: ___
|
|
148
|
-
|
|
149
|
-
If you cannot complete these three lines, STOP and ask the user for
|
|
150
|
-
the requirement. Do not invent an invariant.
|
|
151
|
-
|
|
152
|
-
Then, while writing tests:
|
|
153
|
-
- Real execution: name the real-system path covering this code.
|
|
154
|
-
- On red: read production first; print ANALYSIS: before changing the test.
|
|
155
|
-
- Mocks: never assert on values fed into a mock.
|
|
156
|
-
- Snapshots: classify PRODUCT_CONTRACT or IMPLEMENTATION_DETAIL — the latter is forbidden.
|
|
157
|
-
- Coverage: every positive assertion needs a negative companion.
|
|
158
|
-
- Production surface: don't export internals or add test-only branches.
|
|
159
|
-
|
|
160
|
-
Tests violating guardrails without explicit SKIP-GUARDRAIL-N comments
|
|
161
|
-
will be REJECTED at review.
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
`/dw-run` and `/dw-bugfix` inject this prompt block before generating test code.
|
|
165
|
-
|
|
166
|
-
## Why six and not more
|
|
167
|
-
|
|
168
|
-
These are the highest-frequency LLM failure modes observed across multiple projects. Other tendencies exist but are either covered by the positive patterns (e.g., wall-clock waits) or are lower-frequency than these six.
|
|
169
|
-
|
|
170
|
-
If a new failure mode appears that none of the six catches, add a guardrail AND document the failure that motivated it. Don't add guardrails speculatively.
|
|
12
|
+
A review finding needs evidence that a test can pass for the wrong reason, fail nondeterministically, miss a relevant defect, or constrain implementation unnecessarily. Missing ceremonial preamble text is not a defect.
|
|
@@ -88,10 +88,10 @@ The discipline: no merge without a green E2E (or equivalent real-system check) f
|
|
|
88
88
|
|
|
89
89
|
**What it means:**
|
|
90
90
|
- **Coverage** tells you what lines executed. Useful as a NEGATIVE signal — 30% coverage means lots of dark code. Useless as a positive signal — 95% coverage with weak assertions is decorative.
|
|
91
|
-
- **Mutation
|
|
91
|
+
- **Mutation analysis** introduces small code changes and observes whether tests detect them. Inspect survivors and selected scope: a high score alone does not establish meaningful behavior coverage, and equivalent mutants may survive correct tests. Use `mutation-testing.md` for a scoped investigation.
|
|
92
92
|
- Neither should be a number you optimize for. They're diagnostics.
|
|
93
93
|
|
|
94
|
-
**Anti-pattern:**
|
|
94
|
+
**Anti-pattern:** inventing a universal coverage or mutation threshold and treating it as proof of correctness. Follow existing project-required gates; propose policy changes explicitly when warranted.
|
|
95
95
|
|
|
96
96
|
**Healthier framing:** "What lines in the touched diff are NOT covered? Why?" Sometimes the answer is "we don't care, it's logging." Sometimes it's "actually that's a critical branch — add a test."
|
|
97
97
|
|
|
@@ -122,7 +122,7 @@ A healthy test:
|
|
|
122
122
|
2. Sits at the lowest layer that can prove that behavior (Rule 2).
|
|
123
123
|
3. When red, sends you to read production code (Rule 3).
|
|
124
124
|
4. Has a sibling exercising real systems somewhere in the pipeline (Rule 4).
|
|
125
|
-
5.
|
|
125
|
+
5. Detects meaningful defects in the behavior it claims to cover; mutation analysis can probe this when justified (Rule 5).
|
|
126
126
|
6. Has zero footprint in production code (Rule 6).
|
|
127
127
|
|
|
128
128
|
Any test failing ≥2 of these is technical debt accumulating. `/dw-review --code-only` flags them.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Investigating weak tests with mutations
|
|
2
|
+
|
|
3
|
+
Use when an important behavior has weak evidence, when auditing tests, or when the approved testing strategy includes mutation analysis. Do not install a mutation framework for a copy edit or make every test change run a repository-wide campaign.
|
|
4
|
+
|
|
5
|
+
1. Name the behavior at risk and inspect its production path and tests. Select a bounded module and the existing runner. Confirm a clean baseline test run so unrelated failures are not misreported as detected mutants.
|
|
6
|
+
2. Run the installed mutation tool with an explicit source scope and inspect its execution status. If unavailable, a reversible deliberate defect in an isolated copy can probe one invariant; report it as a targeted experiment, not a complete mutation score. Never leave a deliberate defect in the delivery tree.
|
|
7
|
+
3. For a survivor, examine the changed expression, reachable inputs and expected observable result. Distinguish a missing assertion from equivalent behavior, unreachable code, exclusion, timeout or tooling failure. Report unassessed cases instead of calling them safe.
|
|
8
|
+
4. Add or strengthen a test only for a meaningful missing behavior. Derive expected values from requirements, fixtures or an independent oracle; do not repeat the algorithm under test. Ensure mocks match real module exports and API contracts. Assertions on shape, invocation or snapshots are useful only when that is the actual contract being protected.
|
|
9
|
+
5. Demonstrate that the selected defect is detected and the unmodified implementation passes. Record source scope, cases investigated, remaining limitations, command, tool version and report path. Do not weaken assertions, alter production semantics or exclude survivors just to improve a number.
|
|
10
|
+
|
|
11
|
+
Do not introduce a default mutation-score gate. Preserve existing project-required policy and raise a proposal if it needs revision. A run can succeed while reporting surviving mutants; a tool crash or zero selected files does not establish successful behavioral validation.
|
|
12
|
+
|
|
13
|
+
## Incremental execution
|
|
14
|
+
|
|
15
|
+
[StrykerJS incremental mode](https://stryker-mutator.io/docs/stryker-js/incremental/) reuses a prior report; it is not automatically selection of the current PR diff. Determine the PR base and intended source set separately. Include test changes and affected production scope. An empty selection must be explicitly justified.
|
|
16
|
+
|
|
17
|
+
Stryker's cache does not detect every dependency, configuration, snapshot or environment change. Invalidate reuse when those inputs affect results; use the installed version's supported force/full-scope mechanism. Preserve the unmutated dry run and check runner support for test-change detection. Reuse under `dw-verify` also requires valid input and environment evidence.
|
|
18
|
+
|
|
19
|
+
In CI, publish inspectable reports and distinguish infrastructure failures from investigated survivors. Select schedule and scope according to risk and measured cost. Broad campaigns belong in an agreed audit or scheduled job, not an implicit requirement on unrelated tasks.
|
|
@@ -192,20 +192,11 @@ No merge to main without a real-system path going green. Mocks are speed, real i
|
|
|
192
192
|
|
|
193
193
|
**Anti-pattern:** 100% mocked test suite. "It all passes locally" → first user request fails because the mock didn't match the real API shape.
|
|
194
194
|
|
|
195
|
-
## 11. Mutation
|
|
195
|
+
## 11. Mutation analysis investigates specific behavioral gaps
|
|
196
196
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
Set up mutation testing (Stryker for JS/TS, mutmut for Python, etc.) ONCE per project. Run weekly on critical modules.
|
|
200
|
-
|
|
201
|
-
```bash
|
|
202
|
-
npx stryker run
|
|
203
|
-
# Output: 87 mutants, 78 killed, 9 survived → mutation score 89.6%
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
A surviving mutant means: this code path runs in tests, but the tests don't actually assert anything that breaks when the code changes. Investigate each.
|
|
197
|
+
Use `mutation-testing.md` when critical behavior or weak assertions justify the cost. Inspect survivors individually: they may expose missing tests, equivalent behavior or analysis limits. Strengthen assertions against the intended contract and verify both the deliberate defect and unmodified behavior. Choose scope and cadence from project risk; do not install a tool in every project or introduce a score target.
|
|
207
198
|
|
|
208
|
-
**Anti-pattern:**
|
|
199
|
+
**Anti-pattern:** increasing coverage or killing mutants while assertions still repeat the implementation or only confirm mock setup.
|
|
209
200
|
|
|
210
201
|
## 12. Page Object Model is a tool, not a religion
|
|
211
202
|
|
|
@@ -23,70 +23,13 @@ If you're tempted to skip this "because it's just a small tweak" — that's the
|
|
|
23
23
|
|
|
24
24
|
## The four grounding questions
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
### 1. Where do design decisions come from?
|
|
29
|
-
|
|
30
|
-
Find the project's design source-of-truth in this order:
|
|
31
|
-
1. `.dw/rules/<frontend-module>.md` design system section.
|
|
32
|
-
2. `DESIGN.md`, `BRAND.md`, `STYLE_GUIDE.md` at project root.
|
|
33
|
-
3. Design token config — Tailwind theme, CSS variables in `theme.css`/`globals.css`, MUI/Chakra theme.
|
|
34
|
-
4. Component library config — `components.json` for shadcn, theme exports.
|
|
35
|
-
5. Storybook stories (implicit canonical components).
|
|
36
|
-
|
|
37
|
-
**Stop at the first match** in this list — that source is the authority; defer to it and do not override it with a lower rung or a training-data default. If a needed token is missing (e.g., a danger-secondary color), propose adding it to the authority FIRST, not inline.
|
|
38
|
-
|
|
39
|
-
If **none exists**: read `references/curated-defaults.md` and pick one of the 10 neutral palettes + one of the 10 font pairings shipped there. Mark the choice in the techspec/PR description: "Design source: no project authority found; using curated default `<name>`; recommend establishing `DESIGN.md`."
|
|
40
|
-
|
|
41
|
-
**Anti-patterns at this question:**
|
|
42
|
-
- Inventing color hex values inline (`bg-[#FF6B35]`).
|
|
43
|
-
- "I'll use Tailwind defaults" — that's training-data defaults, not project authority.
|
|
44
|
-
- Copying values from "a site I like" without understanding what it solved.
|
|
45
|
-
|
|
46
|
-
### 2. What does this surface help the user do?
|
|
47
|
-
|
|
48
|
-
Write one sentence: **"This surface helps the user `<verb-phrase>` so that `<outcome>`."**
|
|
49
|
-
|
|
50
|
-
Good examples:
|
|
51
|
-
- "...helps the user filter overdue invoices so they can chase late payers in under 30 seconds."
|
|
52
|
-
- "...helps the on-call engineer diagnose which deploy caused the spike so they can roll back without paging the team."
|
|
53
|
-
- "...helps the manager approve or reject expense reports without leaving Slack."
|
|
54
|
-
|
|
55
|
-
Bad examples:
|
|
56
|
-
- "This surface displays invoice data." (no user, no outcome)
|
|
57
|
-
- "Settings page for managing the account." (vague, no specificity)
|
|
58
|
-
- "Dashboard." (one word)
|
|
59
|
-
|
|
60
|
-
If you can't write the sentence, the requirements are unclear. Stop and clarify before proceeding.
|
|
61
|
-
|
|
62
|
-
### 3. What states does the surface have?
|
|
63
|
-
|
|
64
|
-
Enumerate all states before designing the happy path. Minimum nine, plus domain-specific ones — see `references/state-matrix.md`:
|
|
65
|
-
|
|
66
|
-
`default`, `hover`, `active`, `focus-visible`, `disabled`, `loading`, `empty`, `error`, `success` plus any domain states (read/unread, online/offline, stale/fresh, pending/approved/rejected, draft/saved/dirty, partial/complete, etc.).
|
|
67
|
-
|
|
68
|
-
Missing a state at design time = production bug later. The "we'll add empty state later" trap is real.
|
|
69
|
-
|
|
70
|
-
### 4. Who uses this surface, where, and in what mood?
|
|
71
|
-
|
|
72
|
-
One sentence: **"`<Who>` uses this on `<where>` in `<what light>` while `<what mood>`."**
|
|
73
|
-
|
|
74
|
-
Good examples:
|
|
75
|
-
- "An on-call engineer uses this on a dark-room laptop at 3am while troubleshooting a fire."
|
|
76
|
-
- "A field nurse uses this on a phone in bright outdoor light while juggling clipboards."
|
|
77
|
-
- "A receptionist uses this on a 24″ monitor at a busy front desk while talking to a visitor."
|
|
78
|
-
|
|
79
|
-
Decisions cascade from the answer:
|
|
80
|
-
- 3am dark room → dark mode, high contrast, no flashing animations.
|
|
81
|
-
- Bright outdoor → minimum 7:1 contrast, larger touch targets, no thin fonts.
|
|
82
|
-
- Busy front desk → glanceable info, no nested menus, large numbers.
|
|
83
|
-
|
|
84
|
-
Without this sentence, defaults take over: light mode, default contrasts, animations, regular touch targets. Production users hate it; you can't articulate why.
|
|
26
|
+
For the four grounding questions, read `references/the-four-grounding-questions-detail.md`. Load only when this part of the task applies.
|
|
85
27
|
|
|
86
28
|
## Required reading by context
|
|
87
29
|
|
|
88
30
|
| Doing what | Read |
|
|
89
31
|
|------------|------|
|
|
32
|
+
| Planning/auditing frontend data flow, dependencies or quality tooling | `references/frontend-engineering.md` (may be read directly without visual grounding for nonvisual work) |
|
|
90
33
|
| Any UI work (the full version of the grounding) | `references/hard-gate.md` |
|
|
91
34
|
| Reviewing or proposing a design | `references/visual-slop.md` (14 patterns + specific anti-default values) |
|
|
92
35
|
| Designing state coverage | `references/state-matrix.md` |
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Frontend engineering controls
|
|
2
|
+
|
|
3
|
+
Read when planning frontend architecture, auditing a frontend module, or reviewing changes to its data flow, dependencies or quality tooling. A visual/copy-only edit needs only the relevant UI checks. This reference can be used directly without running visual grounding for an API client or CI configuration change.
|
|
4
|
+
|
|
5
|
+
## Discover before changing
|
|
6
|
+
|
|
7
|
+
Inspect the target workspace's scripts, lockfile, framework, compiler, API source, lint configuration, entry points and CI. Reuse working controls. Record observed configuration separately from proposed policy; an absent tool is not a failed check. Installation of dev-workflow does not authorize replacing consumer tooling.
|
|
8
|
+
|
|
9
|
+
For a project audit, use the installed `frontend-quality-template.md` in `.dw/templates/` to document the baseline in the module's existing rules file. For a feature, include only affected decisions in its TechSpec. Read those decisions during implementation and review. A request to analyze produces findings and proposals; an authorized implementation may add compatible scripts/configuration and validate them.
|
|
10
|
+
|
|
11
|
+
When using this skill standalone without the scaffold template, record the same compact control fields from the adoption section below in the project's existing document; no template installation is required.
|
|
12
|
+
|
|
13
|
+
## Contract and type boundaries
|
|
14
|
+
|
|
15
|
+
- Identify the authoritative API schema and its owner/revision before introducing response fields. If generation is used, record the generator version, command and output paths. Review contract changes, regenerate reproducibly and inspect drift; correct the source or generator rather than editing generated output.
|
|
16
|
+
- When there is no maintained schema, derive a narrow boundary from verified backend code or documented responses and test it; label unresolved assumptions. Do not invent an OpenAPI specification from frontend guesses.
|
|
17
|
+
- Runtime data still needs validation at relevant trust boundaries. Generating a validator is insufficient unless the actual response path calls it and handles failure. [Hey API's Zod integration](https://heyapi.dev/docs/openapi/typescript/plugins/zod) is one option when compatible with the existing client.
|
|
18
|
+
- For TypeScript, inspect effective inherited configuration. Consider [`strict`](https://www.typescriptlang.org/tsconfig/strict.html), [`noUncheckedIndexedAccess`](https://www.typescriptlang.org/tsconfig/noUncheckedIndexedAccess.html) and [`exactOptionalPropertyTypes`](https://www.typescriptlang.org/tsconfig/exactOptionalPropertyTypes.html). Migrate by package or a coherent project boundary; passing filenames to a compiler invocation must not accidentally bypass project settings. Do not hide migration errors behind casts, `any` or non-null assertions.
|
|
19
|
+
|
|
20
|
+
## Architecture and lint feedback
|
|
21
|
+
|
|
22
|
+
Derive allowed imports from the project's architecture, including framework-specific server/client boundaries. Reuse an existing graph checker; [eslint-plugin-boundaries](https://github.com/javierbrea/eslint-plugin-boundaries) is an option for JS/TS. Verify aliases, workspace packages, type imports and legitimate framework entry points. A rejected dependency should identify its source, forbidden destination and the intended placement. Do not impose a universal folder layout.
|
|
23
|
+
|
|
24
|
+
Choose lint rules against concrete failure modes and the installed runner: missing assertion awaits, accidentally focused tests, unsafe type escapes and inaccessible JSX. For React state changes, check whether an Effect actually synchronizes an external system; derived values and event responses often belong in rendering or handlers. Preserve legitimate synchronization. See [React's Effect guidance](https://react.dev/learn/you-might-not-need-an-effect).
|
|
25
|
+
|
|
26
|
+
For a recurring repository-specific defect, prefer an existing rule. A custom detector needs a documented failure mode, an actionable diagnostic and valid/invalid fixtures. Exercise aliases, exclusions and unsupported syntax; an unresolved import is an analysis limitation, not proof of safety. Use an AST or the framework's parser when syntax matters. [ESLint's RuleTester workflow](https://eslint.org/docs/latest/extend/custom-rule-tutorial) supports rule validation. Avoid adding a regex-based import graph when the existing tool already models it.
|
|
27
|
+
|
|
28
|
+
## Code that should disappear
|
|
29
|
+
|
|
30
|
+
Evaluate dead-code findings against runtime reachability and public exports before deletion. Configure framework/plugin entry points, generated sources and workspace aliases. Tests alone can keep otherwise unused production code appearing live: [Knip production mode](https://knip.dev/features/production-mode) provides a separate view of shipped code and does not replace its normal analysis. Validate a newly configured detector with an intentionally unused fixture and a legitimately reachable fixture in an isolated workspace.
|
|
31
|
+
|
|
32
|
+
Use textual duplication reports such as [jscpd](https://github.com/kucherenko/jscpd) as candidates for inspection. Apply different policies to repeated markup/fixtures and business calculations. Search for semantic duplicates too. Consolidate only when copies represent the same rule and should evolve together; a shared abstraction must preserve callers' behavior. Record exclusions with a reason and review trigger, instead of growing blanket ignores to make a report green.
|
|
33
|
+
|
|
34
|
+
## Tests and adoption
|
|
35
|
+
|
|
36
|
+
When critical logic changes or existing tests appear weak, use the testing skill's `references/mutation-testing.md` if available. Mutation testing complements contract and user-flow tests; neither coverage nor a diagnostic score establishes acceptance.
|
|
37
|
+
|
|
38
|
+
For each adopted control record: risk, configuration, exact workspace command, scope, baseline, enforcement status, evidence and owner/review trigger. Use `required`, `advisory`, `deferred` or `not_applicable`; record actual execution separately (`passed`, `failed`, `not_run`). Start noisy new controls as advisory, triage findings, then promote demonstrated checks. Existing required gates remain required. Typechecking, builds and dependency graphs often need the whole affected project even when lint or mutation analysis can target a diff.
|
|
39
|
+
|
|
40
|
+
Local hooks may shorten feedback, but CI must execute required checks independently of editor or agent hooks. Keep failures identifiable; verify that deployment depends on successful required jobs and that repository rules actually require the intended status checks. Do not claim a workflow file alone establishes branch protection. Advisory mutation runs may publish findings without a score threshold; report execution failures explicitly. Never hide a required check behind `continue-on-error`, an empty selection or a silently skipped job.
|
|
41
|
+
|
|
42
|
+
Record command, inputs, environment and analysis scope through `dw-verify` when available. Treat new ignores, disabled rules, changed generation/configuration and lost CI coverage as reviewable changes. Delivery still requires checking product behavior, architecture and accessibility; automated findings reduce review work without replacing that judgment.
|
|
43
|
+
|
|
44
|
+
## Attribution
|
|
45
|
+
|
|
46
|
+
Workflow adaptation prompted by Yuri Mikhin's [Evil Martians article](https://evilmartians.com/chronicles/ten-anti-ai-slop-moves-for-frontend-projects-going-faster-than-humans-can-review), September 1, 2026. Procedures are independently written for dev-workflow and supported by the primary documentation linked above (checked September 9, 2026). No article text, source code or third-party tool is bundled here.
|