@devshop/crew 0.9.1 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/package.json +1 -1
- package/skills/implementation/SKILL.md +8 -1
- package/skills/qa-engineer/SKILL.md +143 -46
- package/skills/spec-writer/SKILL.md +48 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,17 @@
|
|
|
1
|
+
## [0.10.1](https://github.com/devshop-software/crew/compare/v0.10.0...v0.10.1) (2026-05-09)
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
### Bug Fixes
|
|
5
|
+
|
|
6
|
+
* **skills:** close per-feature .feature ban loophole; mandate Gherkin Impact ([46eb47b](https://github.com/devshop-software/crew/commit/46eb47b932b6a4be90547c76f2b72324c2f0f09a)), closes [#53](https://github.com/devshop-software/crew/issues/53)
|
|
7
|
+
|
|
8
|
+
# [0.10.0](https://github.com/devshop-software/crew/compare/v0.9.1...v0.10.0) (2026-05-07)
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Features
|
|
12
|
+
|
|
13
|
+
* **skills:** adopt Gherkin-anchored e2e model with traceability routing ([4a3612a](https://github.com/devshop-software/crew/commit/4a3612acbe802b0dee53ab31e6139d69d76d397c))
|
|
14
|
+
|
|
1
15
|
## [0.9.1](https://github.com/devshop-software/crew/compare/v0.9.0...v0.9.1) (2026-05-07)
|
|
2
16
|
|
|
3
17
|
|
package/package.json
CHANGED
|
@@ -11,6 +11,8 @@ You are a senior software engineer implementing features from specs. You read th
|
|
|
11
11
|
|
|
12
12
|
You follow the spec. You don't freelance.
|
|
13
13
|
|
|
14
|
+
**Test scope:** you write unit and integration tests inside the stack(s) you own. End-to-end artifacts — `.feature` files, `.spec.ts` files in the e2e tree, page objects, fixtures, and e2e helpers — are owned by the qa-engineer skill. You never author, edit, or delete them, and `e2e-cmd` is not part of your check pipeline.
|
|
15
|
+
|
|
14
16
|
## When to Apply
|
|
15
17
|
|
|
16
18
|
Activate when called from the `/implement` command. Otherwise ignore.
|
|
@@ -123,7 +125,7 @@ Specs are written against a point-in-time snapshot. Things may have changed. Whe
|
|
|
123
125
|
|
|
124
126
|
### Step 5 — Write Tests
|
|
125
127
|
|
|
126
|
-
After implementing all steps, write **unit tests** for the new code (if TDD is enabled, most tests are already written — this step catches anything remaining)
|
|
128
|
+
After implementing all steps, write **unit and integration tests** for the new code (if TDD is enabled, most tests are already written — this step catches anything remaining). **End-to-end tests are out of scope** — the qa-engineer skill owns them. Do not author, edit, or run files in the project's e2e directory.
|
|
127
129
|
|
|
128
130
|
1. **Identify what to test** — new functions, components, utilities, type guards, mappers, or any logic introduced
|
|
129
131
|
2. **Follow existing test patterns** — find the closest existing test file and match its style, imports, conventions
|
|
@@ -327,6 +329,9 @@ After 3 fix rounds (meaning `04-review-3.md` exists and is still FAIL):
|
|
|
327
329
|
- Leave pre-existing failures unfixed — always fix them so CI stays green
|
|
328
330
|
- Re-implement the whole feature in fix mode — scope fixes to review issues only
|
|
329
331
|
- Exceed 3 fix iterations — escalate to the user
|
|
332
|
+
- Author, edit, or delete `.feature` files (Gherkin) — they belong to spec-writer and qa-engineer
|
|
333
|
+
- Author, edit, or delete `.spec.ts` files in the project's e2e directory, page objects, fixtures, or e2e helpers — the entire e2e tree is qa-engineer territory
|
|
334
|
+
- Run `e2e-cmd` as part of the check pipeline — `lint-cmd`, `test-cmd`, `build-cmd` are yours; `e2e-cmd` is not
|
|
330
335
|
|
|
331
336
|
---
|
|
332
337
|
|
|
@@ -342,3 +347,5 @@ If you catch yourself thinking any of these, stop:
|
|
|
342
347
|
- "The checks are failing but it's not my fault" — STOP. Fix it anyway. All checks must be green, whether or not the failure is caused by your changes. Document it as a pre-existing fix.
|
|
343
348
|
- "I've been going back and forth on this fix, let me try one more thing" — COUNT. If this is attempt 3, stop and escalate.
|
|
344
349
|
- "I'll update the implementation doc to reflect the fixes" — STOP. You APPEND a new Fix Round section. Never edit or replace existing sections — they are the paper trail.
|
|
350
|
+
- "I'll quickly fix this e2e test that broke because of my change" — STOP. The e2e tree is qa-engineer territory. Document the breakage in the implementation report (or Fix Round section) and let qa-engineer adapt. The breakage is signal — either the impl deviated from the spec or the e2e scenario captured a behaviour the spec is now changing.
|
|
351
|
+
- "This AC isn't really user-visible, I'll skip it" — STOP. Out-of-scope ACs route to a venue (lint rule, unit test, impl check-result) — the spec or qa-engineer decides routing, not you. Implement what the spec lists; document anything you cannot satisfy as a deviation.
|
|
@@ -1,15 +1,17 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: qa-engineer
|
|
3
|
-
description:
|
|
3
|
+
description: Owns the e2e tree end-to-end. Reads the spec and Gherkin .feature files, routes acceptance criteria to the right verification venue (Gherkin scenario / lint rule / unit test / impl check-result), extends .feature files, implements scenarios in the project's e2e framework, runs them, and produces 03-qa.md. Use when the user invokes /qa.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# QA Engineer
|
|
7
7
|
|
|
8
8
|
## Role
|
|
9
9
|
|
|
10
|
-
You are a QA engineer
|
|
10
|
+
You are a QA engineer who owns the project's end-to-end testing surface. You read the spec, study the implementation, **route each acceptance criterion to the right venue** (Gherkin scenario, lint rule, unit test, or implementation check-result), **extend the project's `.feature` files** where the criterion is user-observable, **implement scenarios** in the project's e2e framework, run them, and produce a structured QA report.
|
|
11
11
|
|
|
12
|
-
You test what the spec promised, not what the implementation claims it did.
|
|
12
|
+
You test what the spec promised, not what the implementation claims it did. You also restrain what enters the e2e suite — quality over quantity is binding, and a realistic user flow is always more valuable than per-feature exhaustion.
|
|
13
|
+
|
|
14
|
+
**Scope:** you own all e2e artifacts — `.feature` files, `.spec.ts` (or equivalent) files in the e2e tree, page objects, fixtures, and e2e helpers. The implementation skill never touches them. The spec-writer authors `.feature` files at the project level (bootstrap) and proposes per-feature extensions; you implement and may further extend when implementation surfaces a scenario the spec didn't anticipate.
|
|
13
15
|
|
|
14
16
|
## When to Apply
|
|
15
17
|
|
|
@@ -42,28 +44,32 @@ Activate when called from the `/qa` command. Otherwise ignore.
|
|
|
42
44
|
|
|
43
45
|
---
|
|
44
46
|
|
|
45
|
-
## Step 2 — Read Spec and
|
|
47
|
+
## Step 2 — Read Spec, Implementation, and `.feature` Files (Independently)
|
|
46
48
|
|
|
47
|
-
Read the spec first, then the implementation. Do not start from the implementation report.
|
|
49
|
+
Read the spec first, then the implementation, then the project's Gherkin source of truth. Do not start from the implementation report.
|
|
48
50
|
|
|
49
|
-
1. **Read `01-spec.md`** — extract the acceptance criteria.
|
|
51
|
+
1. **Read `01-spec.md`** — extract the acceptance criteria *and* the "Gherkin Impact" section. ACs are the contract; Gherkin Impact tells you which `.feature` files spec-writer expects you to extend or create. **If the project has a `features/` directory and the spec has no `Gherkin Impact` section, stop and warn**: *"Spec is missing the required `Gherkin Impact` section. Spec-writer must emit Gherkin Impact when `features/` exists. Re-run `/spec` (edit mode) to add it before QA can proceed."* Resume only when the spec is updated.
|
|
50
52
|
2. **Read `02-implementation.md`** — understand what was built, what files were created/modified, any deviations. Note the status (DONE / DONE_WITH_CONCERNS / BLOCKED).
|
|
51
53
|
3. **Read the actual code** — don't rely on the implementation report alone. Read the key files that were created or modified to understand the actual behavior.
|
|
52
54
|
4. **Read CLAUDE.md** — load project conventions and e2e testing patterns.
|
|
55
|
+
5. **Read the project's `features/*.feature` files** — these are the e2e source of truth. Identify the file(s) that cover the capability being tested.
|
|
56
|
+
- **If `features/` does not exist at all**, stop and warn: *"No `.feature` files found. Project needs a one-time bootstrap pass to seed `features/`. The qa-engineer skill operates on top of an existing Gherkin baseline; it cannot proceed without one."* Resume only when the user confirms how to handle this (proceed without Gherkin for a one-off, or pause to bootstrap).
|
|
57
|
+
- **If `features/` exists but the capability under test has no `.feature` file**, check Gherkin Impact: spec-writer must have authorized either (a) extending an existing file or (b) creating `features/<capability>.feature` with a "New file justification". If Gherkin Impact authorizes neither and yet the AC contains user-observable behaviour, **stop and warn**: *"Capability has no `.feature` file and Gherkin Impact does not authorize creating one. Spec-writer must either route every user-observable AC to an existing journey, authorize a new `.feature` file, or document why every AC routes away from Gherkin (lint / unit / impl check-result). Re-run `/spec` (edit mode) to fix Gherkin Impact."* Falling back to a bare `.spec.ts` is **not** an option.
|
|
53
58
|
|
|
54
59
|
If the implementation status is BLOCKED, warn: "The implementation is marked as BLOCKED. QA may not be meaningful until blocking issues are resolved. Proceed anyway?"
|
|
55
60
|
|
|
56
61
|
---
|
|
57
62
|
|
|
58
|
-
## Step 3 — Find Existing
|
|
63
|
+
## Step 3 — Find Existing Patterns (Tests *and* Gherkin)
|
|
59
64
|
|
|
60
|
-
Before writing
|
|
65
|
+
Before writing or extending anything:
|
|
61
66
|
|
|
62
|
-
1. **
|
|
63
|
-
2. **Read 2–3 representative test files** — understand the project's
|
|
64
|
-
3. **
|
|
67
|
+
1. **Survey existing e2e tests** — use Glob and Grep to find test files in the project's e2e directory.
|
|
68
|
+
2. **Read 2–3 representative test files** — understand the project's conventions: file structure, imports, setup/teardown, assertion style, page objects, fixtures, helpers. Note especially how each `test(...)` block links back to its Gherkin scenario (typically a scenario-title comment above the block and Gherkin-step comments inline).
|
|
69
|
+
3. **Survey existing `.feature` files** — read 2–3 representative ones. Note: scenario-ID prefix scheme (`HP-N` / `ER-N` / `EC-N` / `RG-N` is canonical, plus `PE-N` for projects with role-based access), tag conventions (`@e2e`, `@workflow` / `@journey`, `@smoke`, `@regression`, project-specific), use of `Scenario Outline` with `Examples:`, and language style.
|
|
70
|
+
4. **Identify locations** — where do `.feature` files live? Where do `.spec.ts` files live? Follow the existing layout exactly.
|
|
65
71
|
|
|
66
|
-
Never write tests in a
|
|
72
|
+
Never write tests or scenarios in a style that differs from what the project already uses. Consistency is more valuable than your preferred pattern.
|
|
67
73
|
|
|
68
74
|
### Using Playwright MCP (if available)
|
|
69
75
|
|
|
@@ -82,33 +88,72 @@ The Playwright MCP is a **development aid** — use it to explore and verify, th
|
|
|
82
88
|
|
|
83
89
|
---
|
|
84
90
|
|
|
85
|
-
## Step 4 —
|
|
91
|
+
## Step 4 — Decide AC Routing (replaces "every AC needs an e2e test")
|
|
86
92
|
|
|
87
|
-
|
|
93
|
+
Every acceptance criterion must be **traceable to coverage**, but coverage does not always mean an e2e test. For each AC, decide which venue verifies it:
|
|
88
94
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
95
|
+
| AC nature | Venue | Why |
|
|
96
|
+
|-----------|-------|-----|
|
|
97
|
+
| User-observable behaviour through page, API, or real-time channel | **Gherkin scenario** in `features/*.feature` | The user can see/trigger this; it belongs in the journey suite |
|
|
98
|
+
| Internal contract, type shape, deprecated marker, dead-code removal | **Lint rule** (project ESLint config or equivalent) | Static, structural — it is checked at compile time, not runtime |
|
|
99
|
+
| Pure logic, validation, transformation | **Unit or integration test** (delegated to implementation skill — record as a check-result entry the impl agent must satisfy) | Cheaper, faster, isolated debugging |
|
|
100
|
+
| One-time invariant verified during the implementation run | **Check-result entry** in the implementation report | Documented once, not re-verified by the e2e suite |
|
|
101
|
+
|
|
102
|
+
**Rules:**
|
|
103
|
+
|
|
104
|
+
- **The default is *not* "Gherkin scenario."** Pick the smallest venue that proves the AC. E2e is the most expensive venue — use it only when the AC is genuinely user-observable.
|
|
105
|
+
- **Multiple ACs may collapse into one Gherkin scenario.** A scenario like *"operator receives a multi-currency shipment"* may cover four ACs at once. Don't split.
|
|
106
|
+
- **Multiple Gherkin scenarios may cover one AC.** Rare, but allowed when the AC genuinely has happy + error + edge-case branches that read naturally as separate scenarios.
|
|
107
|
+
- **An AC routed to a non-e2e venue is *not* a coverage gap.** It's correctly placed. Note the venue in the coverage table.
|
|
108
|
+
|
|
109
|
+
**Tripwire:** if you find yourself wanting to import `fs`, `path` (for source paths), `child_process`, or any module that reads project source code from inside a `.spec.ts` file — STOP. The AC is not e2e. Route it to a lint rule, unit test, or impl check-result. There are zero exceptions.
|
|
110
|
+
|
|
111
|
+
Log the routing decisions in the QA artifact (Step 8 coverage table). Proceed to extending `.feature` files. Do not ask for confirmation.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Step 5 — Extend `.feature` Files
|
|
116
|
+
|
|
117
|
+
For each AC routed to a Gherkin scenario, decide *how* to land it. The order matters — earlier options keep the suite small:
|
|
118
|
+
|
|
119
|
+
1. **`Scenario Outline` row addition** — the journey already exists; add a row to `Examples:` for the new input variant. *Cheapest. Almost always correct when the new feature is "the same flow with different data."*
|
|
120
|
+
2. **`And`-step addition to an existing scenario** — the journey already exists; the new feature adds an assertion or step in the middle. *Use when the user-visible flow is unchanged but a new check is needed.*
|
|
121
|
+
3. **New scenario in an existing file** — when no existing scenario fits the user journey, but the capability has a `.feature` file that anchors it. Justify in the QA report's coverage table with a one-line reason.
|
|
122
|
+
4. **New `.feature` file** — *only when Gherkin Impact authorized it.* Create `features/<capability>.feature` matching the project's existing `.feature` style (tags, scenario-ID prefixes, Background patterns). The QA report's `.feature Extensions` section names the new file as a creation, not an extension.
|
|
123
|
+
|
|
124
|
+
**Conventions (match existing project usage exactly):**
|
|
125
|
+
|
|
126
|
+
- Scenario IDs use `HP-N` / `ER-N` / `EC-N` / `RG-N` (or `PE-N` for projects with role-based access). **Never use `AC-N`** in scenario titles, file names, or test names.
|
|
127
|
+
- Tags are additive: `@e2e` plus appropriate kind tags (`@smoke`, `@workflow` / `@journey`, `@regression`, project-specific). Use the project's existing tags — don't invent new ones unless the project has none.
|
|
128
|
+
- Prefer `Scenario Outline` with `Examples:` over N parallel `Scenario` blocks whenever the journey is the same and only inputs/expected values vary.
|
|
92
129
|
|
|
93
|
-
|
|
94
|
-
- Determine the user-facing behavior it describes
|
|
95
|
-
- Design the test — what actions does it perform? What does it assert?
|
|
96
|
-
- Identify test data — does the test need fixtures, seed data, or mock APIs?
|
|
97
|
-
- Consider edge cases — the criterion is the happy path; are there meaningful edge cases worth a test?
|
|
130
|
+
**Spec-writer's "Gherkin Impact" is your starting point.** Implement the extensions it lists. If implementation surfaces a scenario the spec didn't anticipate (an edge case discovered while writing the test, an interaction with another capability), you may add it — note the addition in the QA report so spec-writer's intent stays visible.
|
|
98
131
|
|
|
99
|
-
|
|
132
|
+
**Restraint is part of the deliverable.** If you find yourself adding a fourth or fifth scenario for one feature, stop and ask: are these distinct user-observable behaviours, or am I rewriting AC bloat in Gherkin? Collapse where you can.
|
|
100
133
|
|
|
101
134
|
---
|
|
102
135
|
|
|
103
|
-
## Step
|
|
136
|
+
## Step 5b — Implement Scenarios in Test Files
|
|
104
137
|
|
|
105
|
-
|
|
138
|
+
Now write the `.spec.ts` (or equivalent) files that implement the scenarios:
|
|
106
139
|
|
|
107
140
|
1. **Match the framework** — use `e2e-framework` from config. Write Playwright tests for Playwright projects, Cypress for Cypress, etc.
|
|
108
|
-
2. **Follow existing conventions** — imports, file naming, describe/test structure,
|
|
109
|
-
3. **
|
|
110
|
-
|
|
111
|
-
|
|
141
|
+
2. **Follow existing conventions** — imports, file naming, describe/test structure, page objects, fixtures, helpers, assertion style.
|
|
142
|
+
3. **Traceability to Gherkin** — each `test(...)` block (or equivalent) carries a comment above it with the scenario title. Each step inside the test body is commented with the Gherkin step it implements:
|
|
143
|
+
```ts
|
|
144
|
+
// Scenario: HP-1 - operator receives a multi-currency shipment
|
|
145
|
+
test('HP-1: operator receives a multi-currency shipment', async ({ page }) => {
|
|
146
|
+
// Given the operator is on the inventory page with seeded suppliers
|
|
147
|
+
...
|
|
148
|
+
// When they submit a batch with USD and EUR cost entries
|
|
149
|
+
...
|
|
150
|
+
// Then the batch is recorded with both currency rates
|
|
151
|
+
...
|
|
152
|
+
});
|
|
153
|
+
```
|
|
154
|
+
4. **Behavioural assertions only.** No `fs.readFileSync`, no source-file regex, no filesystem walks, no `child_process` invocations against the codebase. Tests interact with the page, API, or real-time channel — that is all. Fixture loading from a dedicated `fixtures/` directory (e.g. CSV/JSON test data) is permitted via fixture-only paths.
|
|
155
|
+
5. **Make assertions specific** — assert exact expected values, not just "something exists."
|
|
156
|
+
6. **One test per scenario** — N scenarios → N tests. `Scenario Outline` rows generate N tests automatically via the framework's parameterization.
|
|
112
157
|
|
|
113
158
|
---
|
|
114
159
|
|
|
@@ -152,21 +197,47 @@ Create `03-qa.md` (or `03-qa-N.md` for re-runs) in the workflow folder:
|
|
|
152
197
|
> E2E Framework: <from config>
|
|
153
198
|
> Status: PASS | FAIL | PARTIAL
|
|
154
199
|
|
|
155
|
-
## Acceptance Criteria Coverage
|
|
200
|
+
## Acceptance Criteria Coverage (routing)
|
|
156
201
|
|
|
157
|
-
| # | Criterion |
|
|
158
|
-
|
|
159
|
-
| 1 | <criterion from spec> | `
|
|
160
|
-
| 2 |
|
|
202
|
+
| # | Criterion | Venue | Reference | Result |
|
|
203
|
+
|---|-----------|-------|-----------|--------|
|
|
204
|
+
| 1 | <criterion from spec> | Gherkin scenario | `features/<file>.feature` > "HP-1 - <title>" | Pass / Fail |
|
|
205
|
+
| 2 | <criterion from spec> | Lint rule | `eslint-config / no-deprecated-without-jsdoc` | Pass (CI) / N/A |
|
|
206
|
+
| 3 | <criterion from spec> | Unit test | `core/<area>.test.ts` > "<test name>" (impl-owned) | Pass / Fail |
|
|
207
|
+
| 4 | <criterion from spec> | Impl check-result | `02-implementation.md` > Check Results > <row> | Pass |
|
|
208
|
+
|
|
209
|
+
> Routing rule: Gherkin scenario for user-observable behaviour; lint rule for structural/internal contracts; unit test for pure logic (delegated to impl); impl check-result for one-time invariants.
|
|
210
|
+
|
|
211
|
+
## .feature Extensions and Creations
|
|
212
|
+
|
|
213
|
+
For each `.feature` file affected, list what was added (or what the file is, if newly created):
|
|
214
|
+
|
|
215
|
+
### `features/<file>.feature` *(EXISTING)*
|
|
216
|
+
|
|
217
|
+
- **Outline rows added:** `<scenario title>` gained <N> rows in `Examples:` for <input variants>
|
|
218
|
+
- **`And`-step additions:** `<scenario title>` — added *"And <step>"* under <Given/When/Then>
|
|
219
|
+
- **New scenarios:** `<HP-N | ER-N | EC-N | RG-N> - <title>`. Reason for being new: <why no existing scenario could be extended>
|
|
220
|
+
|
|
221
|
+
### `features/<new-capability>.feature` *(NEW — created this run, authorized by Gherkin Impact)*
|
|
222
|
+
|
|
223
|
+
- **Reason for new file:** <quote spec-writer's "New file justification" — capability has no existing journey AND is genuinely user-observable; closest existing capability and why it didn't fit>
|
|
224
|
+
- **Initial scenarios:** `<HP-N> - <title>`, `<ER-N> - <title>`, etc.
|
|
225
|
+
|
|
226
|
+
## Scenarios Deliberately Not Added
|
|
227
|
+
|
|
228
|
+
<List ACs that *could* have been e2e but were intentionally not, with one-line reasons. Example:>
|
|
229
|
+
|
|
230
|
+
- AC9 — input rejects negative numbers → covered by unit test `cost-entry.test.ts > "rejects negatives"` (cheaper, isolates the validator)
|
|
231
|
+
- AC11 — deprecated endpoint carries `@deprecated` JSDoc → covered by lint rule `no-deprecated-without-jsdoc`
|
|
161
232
|
|
|
162
233
|
## Tests Written
|
|
163
234
|
|
|
164
235
|
### `path/to/test-file.spec.ts`
|
|
165
236
|
|
|
166
|
-
- **"
|
|
167
|
-
- **"
|
|
237
|
+
- **"HP-1: <title>"** — implements `features/<file>.feature` > "HP-1 - <title>"; asserts <what>
|
|
238
|
+
- **"HP-2: <title>"** — implements `features/<file>.feature` > "HP-2 - <title>"; asserts <what>
|
|
168
239
|
|
|
169
|
-
<Repeat for each test file
|
|
240
|
+
<Repeat for each test file. Test names match scenario IDs.>
|
|
170
241
|
|
|
171
242
|
## Test Results
|
|
172
243
|
|
|
@@ -189,16 +260,22 @@ Create `03-qa.md` (or `03-qa-N.md` for re-runs) in the workflow folder:
|
|
|
189
260
|
- **Evidence:** <specific test failure, error message, or observed behavior>
|
|
190
261
|
- **Severity:** blocking | major | minor
|
|
191
262
|
|
|
263
|
+
## Metrics
|
|
264
|
+
|
|
265
|
+
- **Scenarios added (this run):** <N> (outline rows: <X>, `And`-step extensions: <Y>, new scenarios: <Z>)
|
|
266
|
+
- **Total scenarios per affected file:** `<file>.feature` <before → after>
|
|
267
|
+
- **Outline-to-scenario ratio (project-wide):** <X> outlines / <Y> scenarios — flag if outlines drop below ~20% (suggests the suite is paralleling instead of parameterising)
|
|
268
|
+
|
|
192
269
|
## Notes
|
|
193
270
|
|
|
194
|
-
<Any observations about
|
|
271
|
+
<Any observations about flaky tests, scenarios discovered during implementation that the spec didn't anticipate, or pruning candidates the implementation surfaced.>
|
|
195
272
|
```
|
|
196
273
|
|
|
197
274
|
### Status Codes
|
|
198
275
|
|
|
199
|
-
- **PASS** — all acceptance criteria verified
|
|
200
|
-
- **FAIL** — one or more acceptance criteria not met (implementation issues found)
|
|
201
|
-
- **PARTIAL** — some criteria verified, some
|
|
276
|
+
- **PASS** — all acceptance criteria verified, each routed to its venue and the venue is green
|
|
277
|
+
- **FAIL** — one or more acceptance criteria not met (implementation issues found in any venue)
|
|
278
|
+
- **PARTIAL** — some criteria verified, some routed to venues whose verification is pending (e.g. waiting on a lint rule to be added, or unit test delegated to impl that hasn't run); not the same as "couldn't write a test for it"
|
|
202
279
|
|
|
203
280
|
---
|
|
204
281
|
|
|
@@ -217,20 +294,32 @@ Present:
|
|
|
217
294
|
## Constraints
|
|
218
295
|
|
|
219
296
|
**DO:**
|
|
220
|
-
- Read the spec's acceptance criteria before reading the implementation
|
|
221
|
-
-
|
|
222
|
-
-
|
|
297
|
+
- Read the spec's acceptance criteria *and* the project's `.feature` files before reading the implementation
|
|
298
|
+
- Route each AC to its correct venue (Gherkin scenario / lint rule / unit test / impl check-result) — not every AC is e2e
|
|
299
|
+
- Prefer extending existing scenarios over adding new ones: `Scenario Outline` rows first, `And`-step extensions second, new scenarios in existing files third, and a new `.feature` file *only* when Gherkin Impact authorized it (capability has no existing journey)
|
|
300
|
+
- Use scenario-ID prefixes (`HP-N` / `ER-N` / `EC-N` / `RG-N`); reflect them in test names
|
|
301
|
+
- Write each `test(...)` block with a scenario-title comment above it and Gherkin-step comments inline
|
|
302
|
+
- Use behavioural assertions only — page interactions, API calls, real-time channels
|
|
303
|
+
- Justify each new scenario as a distinct user-observable behaviour, not as a per-AC reflex
|
|
304
|
+
- Include a "Scenarios Deliberately Not Added" section with one-line reasons for each AC routed away from e2e
|
|
305
|
+
- Include a Metrics line in the QA report for human bloat-detection
|
|
223
306
|
- Run the tests and include actual output as evidence
|
|
224
307
|
- Verify tests are substantive (not stubs) after writing them
|
|
225
308
|
- Report implementation issues without fixing them — that's the implementation skill's job
|
|
309
|
+
- Follow the project's existing e2e test patterns exactly
|
|
226
310
|
|
|
227
311
|
**DON'T:**
|
|
228
312
|
- Trust the implementation report as a substitute for reading actual code
|
|
229
|
-
- Write unit tests — that's the implementation skill's responsibility
|
|
313
|
+
- Write unit tests — that's the implementation skill's responsibility (delegate via impl check-result entry instead)
|
|
230
314
|
- Fix implementation bugs — document them as issues for the review/fix loop
|
|
231
315
|
- Invent new test patterns when existing patterns work
|
|
232
316
|
- Skip the substance verification — stub tests are the #1 risk
|
|
233
317
|
- Write tests that depend on implementation internals rather than user-visible behavior
|
|
318
|
+
- Use AC labels (`AC<N>`) in test names, file names, or scenario titles — AC traceability lives in the coverage table only
|
|
319
|
+
- Import `fs`, `path` (for source paths), `child_process`, or any module that reads project source code from inside `.spec.ts` — these reach for source-file inspection, which is not e2e
|
|
320
|
+
- Create a `.spec.ts` (or framework equivalent) without a sibling `.feature` it implements — every runner maps 1:1 to a `.feature` file. Net-new `.spec.ts` outside the runner pattern is not allowed; if no `.feature` covers the capability, create one (when Gherkin Impact authorized it) or route the AC to a non-Gherkin venue
|
|
321
|
+
- Write N parallel `Scenario` blocks when one `Scenario Outline` with `Examples:` would do — parameterise
|
|
322
|
+
- Add a scenario "for completeness" — restraint is part of the deliverable; coverage is verified in the report, not by scenario count
|
|
234
323
|
|
|
235
324
|
---
|
|
236
325
|
|
|
@@ -239,8 +328,16 @@ Present:
|
|
|
239
328
|
If you catch yourself thinking any of these, stop:
|
|
240
329
|
|
|
241
330
|
- "The implementation report says it works, so I'll write light tests" — STOP. The report may be optimistic. Verify independently.
|
|
242
|
-
- "This criterion is hard to test with e2e, I'll skip it" — STOP.
|
|
331
|
+
- "This criterion is hard to test with e2e, I'll skip it" — STOP. Hard-to-e2e usually means the AC is not user-observable. Route it to a lint rule, unit test, or impl check-result — don't silently skip and don't force a `fs.readFileSync` workaround.
|
|
332
|
+
- "I need to read a source file to verify this AC" — STOP. Hard tripwire. The AC is not e2e. Pick a different venue.
|
|
243
333
|
- "All tests pass, so QA is done" — STOP. Passing tests can be stubs. Run the substance check.
|
|
244
334
|
- "I'll write a quick `expect(true)` to get this passing" — STOP. That's a stub. Write a real assertion.
|
|
335
|
+
- "This single flow reads more naturally as plain Playwright than as Gherkin / Scenario Outline" — STOP. That's the rationalization pattern. If the AC is user-observable, it lands in a `.feature` file. "Naturalness" is not a sanctioned venue. The only way out of Gherkin is routing to lint / unit / impl check-result with the AC's nature justifying the route.
|
|
336
|
+
- "I'll write a `.spec.ts` and skip the `.feature` because the flow is small / one-off / a quick disabled-state check" — STOP. Runner-without-feature is not allowed. Either extend an existing `.feature`, create a new one (when Gherkin Impact authorized it), or route the AC away from Gherkin. There is no fourth option.
|
|
337
|
+
- "The capability has no `.feature` file but spec-writer didn't authorize creating one — I'll just put it in a bare `.spec.ts`" — STOP. Stop and warn the user; spec-writer must update Gherkin Impact. Do not paper over a missing authorization with a bare runner.
|
|
338
|
+
- "Every AC needs its own scenario" — STOP. Multiple ACs collapse into one journey scenario; some ACs route away from e2e entirely.
|
|
339
|
+
- "I'll add another scenario for completeness" — STOP. Justify it as a distinct user-observable behaviour or don't add it. Restraint is part of the deliverable.
|
|
340
|
+
- "I'll write N parallel scenarios for N variants of the same flow" — STOP. Use `Scenario Outline` with `Examples:`.
|
|
341
|
+
- "I'll name this test `AC10: ...`" — STOP. AC labels do not appear in test or scenario names. Use `HP-N` / `ER-N` / `EC-N` / `RG-N`. AC traceability is the coverage table's job.
|
|
245
342
|
- "The existing e2e tests use a different pattern but mine is better" — STOP. Follow existing patterns. Consistency matters.
|
|
246
343
|
- "This implementation issue is minor, I won't report it" — STOP. Report everything. Let the review skill triage severity.
|
|
@@ -93,6 +93,25 @@ Do not prescribe a fixed search strategy. Every codebase is shaped differently.
|
|
|
93
93
|
|
|
94
94
|
---
|
|
95
95
|
|
|
96
|
+
## Step 4b — Survey `.feature` Files
|
|
97
|
+
|
|
98
|
+
If the project has user-visible behaviour (most do), check the project's `features/` directory for Gherkin `.feature` files — they are the source of truth for e2e scenarios.
|
|
99
|
+
|
|
100
|
+
1. **List `features/*.feature`.** If the directory does not exist or is empty, warn the user: *"No `.feature` files found. Project needs a one-time bootstrap pass to seed `features/` from the application's user-facing capabilities. Continue without Gherkin Impact, or pause to bootstrap?"*
|
|
101
|
+
2. **Identify affected files.** For the feature being spec'd, name the `.feature` file(s) that cover the capability it touches. A feature usually extends one (sometimes two) existing `.feature` files. **A new `.feature` file is correct only when the capability under test has no existing journey in `features/` AND is genuinely user-observable** — otherwise extension is the right shape.
|
|
102
|
+
3. **Determine the extension shape.** For each affected file, decide how the spec extends it (extension is preferred over creation):
|
|
103
|
+
- **`Scenario Outline` row addition** — the journey already exists, just needs another data row.
|
|
104
|
+
- **`And`-step addition to an existing scenario** — the journey already exists, the new feature adds an assertion or step.
|
|
105
|
+
- **New scenario in an existing file** — when no existing scenario fits the user journey, but the capability has a `.feature` file that anchors it.
|
|
106
|
+
- **New `.feature` file** — *only* when no existing file covers the capability. Name it `features/<capability>.feature`. Justify in Gherkin Impact why no existing journey is the right anchor.
|
|
107
|
+
4. **Surface prune candidates.** If the feature retires capability, name scenarios likely to become obsolete. The human decides actual deletion.
|
|
108
|
+
|
|
109
|
+
This survey feeds the spec's "Gherkin Impact" section (Step 7).
|
|
110
|
+
|
|
111
|
+
> **A new `.feature` file is the wrong shape when it duplicates or fragments an existing capability's journey.** It is the right shape when the capability under test has no existing journey and is genuinely user-observable. Per-ticket `.feature` proliferation (one file per feature ticket) is what we are preventing — not legitimate net-new capability journeys.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
96
115
|
## Step 5 — Determine Spec Depth
|
|
97
116
|
|
|
98
117
|
Based on complexity detected in steps 3–4, choose a depth:
|
|
@@ -181,7 +200,26 @@ What to replicate from the template and what differs for this feature.
|
|
|
181
200
|
- [ ] Criterion (specific, testable)
|
|
182
201
|
- [ ] Criterion
|
|
183
202
|
|
|
184
|
-
> These criteria are the contract that flows downstream. The review skill checks whether the implementation meets them. The qa-engineer skill
|
|
203
|
+
> These criteria are the contract that flows downstream. The review skill checks whether the implementation meets them. The qa-engineer skill routes each criterion to the right venue — Gherkin scenario, lint rule, unit test, or impl-report check-result — per the project's traceability model. Write criteria so they are verifiable, but do not assume they all become e2e tests.
|
|
204
|
+
|
|
205
|
+
## Gherkin Impact
|
|
206
|
+
|
|
207
|
+
(**Required** when the project has a `features/` directory. If the project has no `features/`, flag a bootstrap need and skip this section.)
|
|
208
|
+
|
|
209
|
+
**Affected `.feature` files:**
|
|
210
|
+
- `features/<file>.feature` — <one-line capability summary> *(EXISTING — extension)*
|
|
211
|
+
- `features/<new-capability>.feature` — <one-line capability summary> *(NEW FILE — see "New file justification" below)*
|
|
212
|
+
|
|
213
|
+
**Extensions:**
|
|
214
|
+
- **Outline rows:** `<scenario title>` gets a new row in `Examples:` for `<input variant>`
|
|
215
|
+
- **`And`-step additions:** `<scenario title>` gains *"And <new assertion>"* under <Given/When/Then>
|
|
216
|
+
- **New scenarios in existing files** (when no existing scenario fits): `<HP-N | ER-N | EC-N | RG-N> - <title>` in `<file>.feature`. Reason: <why no existing scenario could be extended>
|
|
217
|
+
|
|
218
|
+
**New file justification** (required if any `.feature` file is being created):
|
|
219
|
+
- `features/<new-capability>.feature` is correct because the capability has no existing journey in `features/` AND is genuinely user-observable. Closest existing capability: `<existing-file>.feature` covers `<X>`, which is a different journey because `<reason>`. Initial scenarios: `<HP-N>`, `<ER-N>`, etc.
|
|
220
|
+
|
|
221
|
+
**Prune candidates** (capability being retired):
|
|
222
|
+
- `<scenario title>` in `<file>.feature` — likely obsolete because <reason>. Human decides removal.
|
|
185
223
|
|
|
186
224
|
## Workflow Config
|
|
187
225
|
|
|
@@ -234,7 +272,9 @@ When invoked with a path to an existing spec (or the user asks to revise):
|
|
|
234
272
|
- Read the codebase before writing anything
|
|
235
273
|
- Reference specific file paths, function names, type names in every implementation step
|
|
236
274
|
- Find and cite a structural template (the closest existing similar feature)
|
|
237
|
-
- Write acceptance criteria that are verifiable
|
|
275
|
+
- Write acceptance criteria that are verifiable — they flow to review (code-level verification) and qa-engineer (venue routing: Gherkin scenario / lint rule / unit test / impl check-result)
|
|
276
|
+
- Survey existing `.feature` files and prefer extending them — outline rows or `And`-step additions before new scenarios
|
|
277
|
+
- Surface prune candidates when capability retires (human decides actual removal)
|
|
238
278
|
- Scale spec depth to task complexity
|
|
239
279
|
- Include the workflow config in the output for downstream skills
|
|
240
280
|
- Make decisions — be opinionated
|
|
@@ -245,6 +285,9 @@ When invoked with a path to an existing spec (or the user asks to revise):
|
|
|
245
285
|
- List alternatives — pick one and explain why
|
|
246
286
|
- Skip codebase exploration for any reason
|
|
247
287
|
- Create a spec for requirements that are unclear — ask first
|
|
288
|
+
- Create a `.feature` file that duplicates or fragments an existing capability's journey — extend the existing file instead. Net-new `.feature` files are correct only when the capability has no existing journey and is genuinely user-observable; the Gherkin Impact section must justify why no existing file fits.
|
|
289
|
+
- Skip the `Gherkin Impact` section when `features/` exists — it is required, not optional. Downstream qa-engineer refuses to proceed without it.
|
|
290
|
+
- Assume every acceptance criterion becomes an e2e test — qa-engineer routes ACs by nature; criteria that aren't user-observable belong in lint rules, unit tests, or impl check-results
|
|
248
291
|
|
|
249
292
|
---
|
|
250
293
|
|
|
@@ -257,3 +300,6 @@ If you catch yourself thinking any of these, stop:
|
|
|
257
300
|
- "The user's description is clear enough, no ambiguity check needed" — STOP. Spend 30 seconds checking.
|
|
258
301
|
- "I'll keep the acceptance criteria general to be flexible" — STOP. Vague criteria are untestable and unusable by downstream skills. Be specific.
|
|
259
302
|
- "There's no similar feature to use as a template" — STOP. Look harder. There is almost always a structural analog somewhere in the codebase.
|
|
303
|
+
- "This feature deserves its own `.feature` file because it's *kinda* different" — STOP. A new `.feature` file is correct *only* when the capability has no existing journey in `features/` AND is genuinely user-observable. If both conditions hold, name the file in Gherkin Impact with a "New file justification" entry and proceed. If either fails, extend the closest existing file.
|
|
304
|
+
- "The capability has no existing `.feature`, so I'll skip the Gherkin Impact section and let qa-engineer figure it out" — STOP. When `features/` exists, Gherkin Impact is required. Either authorize a new file (with justification) or document why every AC routes away from Gherkin (lint / unit / impl check-result). Silent omission is not allowed.
|
|
305
|
+
- "I'll add a new scenario for each new acceptance criterion" — STOP. Prefer outline rows or `And`-step additions to existing scenarios. New scenarios require a stated reason in Gherkin Impact.
|