criteria-review 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +299 -0
  3. package/agents/README.md +123 -0
  4. package/agents/skills/claude-code/criteria-lookup/SKILL.md +236 -0
  5. package/bin/criteria-review.js +870 -0
  6. package/docs/decisions.md +310 -0
  7. package/docs/help/01-getting-started.md +170 -0
  8. package/docs/standard/00-roles-and-ownership.md +353 -0
  9. package/docs/standard/01-qa-approach.md +273 -0
  10. package/docs/standard/02-writing-acceptance-criteria.md +187 -0
  11. package/docs/standard/03-example-acceptance-document.md +244 -0
  12. package/docs/standard/04-playwright-journeys.md +329 -0
  13. package/docs/standard/05-example-journey.spec.ts +130 -0
  14. package/docs/standard/06-engineering-obligations.md +170 -0
  15. package/docs/standard/07-manual-qa-and-defects.md +138 -0
  16. package/docs/standard/08-definition-of-done.md +128 -0
  17. package/docs/standard/09-recording-journeys.md +644 -0
  18. package/docs/standard/README.md +70 -0
  19. package/docs/standard/adopting-the-standard.md +104 -0
  20. package/docs/standard/emit-contract.md +175 -0
  21. package/package.json +41 -0
  22. package/public/app.js +791 -0
  23. package/public/doc-name.js +35 -0
  24. package/public/index.html +100 -0
  25. package/public/markdown.js +0 -0
  26. package/public/queue-order.js +78 -0
  27. package/public/styles.css +414 -0
  28. package/src/config.js +104 -0
  29. package/src/emit.js +183 -0
  30. package/src/media.js +126 -0
  31. package/src/parse.js +298 -0
  32. package/src/scan.js +158 -0
  33. package/src/server.js +433 -0
  34. package/src/standard.js +174 -0
  35. package/src/version.js +19 -0
  36. package/src/videos.js +185 -0
  37. package/src/write.js +307 -0
@@ -0,0 +1,353 @@
1
+ # Roles and ownership
2
+
3
+ Who stands where. The rest of the pack describes the machinery; this file describes the people
4
+ the machinery is arranged around, what each of them owns, and where the boundary between
5
+ deciding and building falls.
6
+
7
+ Two roles carry the work, and this document names them as the tool does:
8
+
9
+ - **The architect** decides what the software *should* do. Organisations title this differently:
10
+ solutions architect, technical lead, product engineer. The title is not the point; the remit
11
+ is.
12
+ - **The implementers** decide how it is realised in code, and prove their own seams.
13
+
14
+ One person can hold both roles on a small project, and the boundary still matters, because it is
15
+ what makes the criteria worth writing rather than a formality performed after the fact.
16
+
17
+ ---
18
+
19
+ ## The one idea underneath the role
20
+
21
+ > Solutioning is finished when an implementer can ask *how do I build this cleanly* instead of
22
+ > *is this even the right thing*. Everything before that line is the architect's work.
23
+ > Everything after it is the team's.
24
+
25
+ The acceptance criteria are what draw that line, and this is what connects the pack to the whole
26
+ team: **a tier 1 document agreed before implementation is not a QA artefact that happens early.
27
+ It is the specification.** Written and confirmed up front, it removes the one question a skilled
28
+ implementer cannot answer for themselves, which is what the business actually wanted, and leaves
29
+ them the questions they are best placed to answer: how to build it, how to structure it, how to
30
+ make it fast and maintainable.
31
+
32
+ So the QA roadmap is not downstream of design. It is the guide to it. The obligation to
33
+ enumerate both tiers before anyone writes code ([`08-definition-of-done.md`](08-definition-of-done.md),
34
+ stage 1) is the same obligation as designing the feature. There is no separate design step that
35
+ the criteria then check. The criteria, agreed, are the design.
36
+
37
+ ---
38
+
39
+ ## 1. The role has two faces, and they are different jobs
40
+
41
+ Facing whoever wanted the software, which may be a customer, product, support or pre-sales, the
42
+ job is to **turn a want into an agreed intent**. This is where the "so that" is nailed down
43
+ ([`02-writing-acceptance-criteria.md`](02-writing-acceptance-criteria.md)), where a request stops
44
+ being a solution somebody guessed at and becomes the outcome it was really about. The output is
45
+ a tier 1 acceptance document, in the requester's language, that they have confirmed.
46
+
47
+ Facing the implementers, the job is to **turn agreed intent into a de-risked technical brief**.
48
+ This is where tier 2 obligations are named from the catalogue
49
+ ([`06-engineering-obligations.md`](06-engineering-obligations.md)), where the edge cases no user
50
+ would ever describe are worked through, and where hard unknowns are spiked before they reach a
51
+ sprint.
52
+
53
+ These are sequenced, not parallel, and the order is load bearing:
54
+
55
+ > Confirm the intent with whoever owns it **before** the technical edge-casing, not after.
56
+
57
+ The requester confirms behaviour in their own language, and the tier 2 work is invisible to them
58
+ ([`01-qa-approach.md`](01-qa-approach.md), section 2). Doing the technical edge-casing first
59
+ buries the intent they are being asked to approve under machinery they cannot read, and the
60
+ confirmation stops being a confirmation. Intent first, in their words. Obligations second, in
61
+ ours.
62
+
63
+ ---
64
+
65
+ ## 2. What the architect owns, and what the team owns
66
+
67
+ The single decision that determines whether the model scales or collapses back onto one person.
68
+ Draw the line in the wrong place and the role becomes a bottleneck with a better title.
69
+
70
+ The test is the one the standard uses to decide where a sentence lives: **does the decision stay
71
+ the same if you swapped the implementers for a different, equally skilled team?** If yes, the
72
+ architect owns it. If no, if it depends on how *this* person writes *this* code, it belongs to
73
+ the implementer.
74
+
75
+ | The architect owns | The team owns |
76
+ |---|---|
77
+ | The intent, and its confirmation with whoever owns it | How the intent is realised in code |
78
+ | The tier 1 scenarios and their wording | The internal structure of the implementation |
79
+ | The tier 2 obligations shortlist for the feature | Which patterns, types and abstractions sit behind the contract |
80
+ | Contracts across a boundary: API shape, event and message contracts, the data model | The implementation behind the contract |
81
+ | Cross-cutting choices: identity, authorisation surface, tenancy, where state lives | Local, cheap-to-reverse decisions inside a service |
82
+ | The hard-to-reverse and the high-risk | The many cheap-to-reverse decisions |
83
+ | Acceptance of the returned work against the brief | Getting the suite green and the evidence produced |
84
+
85
+ Both failure modes are worth naming, because each is invisible from inside it.
86
+
87
+ **Drawn too low:** if vetting reaches down to variable names, method shape, or which construct
88
+ you would have used, the architect has recreated themselves as the team's editor. Vetting asks
89
+ *does this honour the intent and the contract*, never *would I have written it this way*.
90
+
91
+ **Drawn too high:** handing the team a one-line want and calling it delegation. Intent left to be
92
+ inferred will be inferred fluently and wrongly, which is the failure
93
+ [`01-qa-approach.md`](01-qa-approach.md) section 6 exists to prevent. The brief is what stops
94
+ that, and its whole purpose is to make the intent unnecessary to guess.
95
+
96
+ ---
97
+
98
+ ## 3. Intake: vetting a request before it is real work
99
+
100
+ A request arriving from outside the team is a want, not a specification, and admitting it to the
101
+ backlog as though it were one is how unsourced intent enters the system. Intake is a gate, and it
102
+ sits deliberately at the front, before anyone has an interest in the list being short
103
+ ([`01-qa-approach.md`](01-qa-approach.md), section 3).
104
+
105
+ A request is not ready to be worked until it carries:
106
+
107
+ - **A named actor and a real "so that".** Not "add a filter", but who is blocked without it and
108
+ what they cannot currently do. A request whose "so that" is circular is a heading, not a story
109
+ ([`02-writing-acceptance-criteria.md`](02-writing-acceptance-criteria.md)).
110
+ - **Sourced intent, or an explicit question in its place.** Every claim about what was wanted
111
+ cites where it came from: a dated conversation with a named owner, a support ticket, a
112
+ commitment recorded somewhere. Where no source exists, the request carries the question rather
113
+ than a confident guess. Intent is never reverse engineered from what a competitor does or from
114
+ what seems reasonable.
115
+ - **The scope line.** One sentence naming what is in and, more usefully, what is deliberately
116
+ out. Most disputes at delivery are about the out.
117
+
118
+ The output of intake is not "approved". It is one of three: **admitted** with its intent sourced,
119
+ **returned** to the requester with the specific missing piece named, or **held** as a question
120
+ against the person who owns the answer. The three are visible states rather than a private
121
+ judgement, for the same reason the status ladder is visible: the unsettled surface has to stay on
122
+ screen ([`01-qa-approach.md`](01-qa-approach.md), section 5).
123
+
124
+ ---
125
+
126
+ ## 4. The two products
127
+
128
+ Per feature, the architect produces exactly two things, and the team should be able to start from
129
+ them alone.
130
+
131
+ ### 4a. The acceptance spec, which is the roadmap
132
+
133
+ The tier 1 document and the tier 2 shortlist, produced as
134
+ [`08-definition-of-done.md`](08-definition-of-done.md) stage 1 requires. What matters here is
135
+ what it *is* to the team: every tier 1 scenario is behaviour the implementer must deliver and
136
+ will be measured against, and every tier 2 obligation is an edge they are on notice to handle.
137
+ Enumerated up front, it is the map of the whole surface, drawn before the terrain tempts anyone
138
+ to shorten it.
139
+
140
+ Shared with the requester minus the tier 2 machinery, and confirmed, it becomes the thing that
141
+ carries force in a disagreement: *agreed*, not *assumed*.
142
+
143
+ ### 4b. The technical brief, which is the guide
144
+
145
+ What the acceptance spec does not carry: the design decisions the architect has taken so the team
146
+ does not have to, and the risks retired so the team does not hit them mid-sprint. It is short on
147
+ purpose. If it is long, the architect is doing the team's job.
148
+
149
+ | Section | What it is |
150
+ |---|---|
151
+ | Intent, in one paragraph | The agreed "so that", so the brief reads without the acceptance document open beside it |
152
+ | The obligations shortlist | The tier 2 ids this feature instantiates, each a concrete list of tests rather than a feeling |
153
+ | Contracts | API shape, event and message contracts, and the data model, stated as the boundary the team builds behind |
154
+ | Decisions taken | Cross-cutting calls already made, each with a one-line why, so they are not silently relitigated |
155
+ | Decisions left open | The local calls the team owns, named as theirs, so delegation is explicit rather than a gap |
156
+ | Risks retired | The spikes done and what they proved; the unknowns that would otherwise surface mid-sprint |
157
+ | Out of scope | What this deliberately does not do, so a returned implementation is not marked short for not doing it |
158
+
159
+ The "decisions left open" row is where the economics live. **Specify the outcome completely,
160
+ because a gap will not be correctly filled by someone who was not in the conversation. Do not
161
+ specify the mechanism**, because the moment the brief pre-writes the code, designing costs what
162
+ building costs and the reason for separating them is gone.
163
+
164
+ The line to hold: the *what* and the *must-be-true* are pinned to the point of no ambiguity; the
165
+ *how* is left as genuine mechanism the implementers own. That division is not a courtesy. It is
166
+ what keeps designing cheaper than building, and it is the hardest judgement in the role. Specify
167
+ too little and you get wrong software; too much and you have simply built it yourself, slowly.
168
+
169
+ ---
170
+
171
+ ## 5. The three levels: who builds which tests
172
+
173
+ The architect's output is not tests. It is the guidance tests are built from. Below it sit two
174
+ layers of proof, and the acceptance criteria are what let all three work independently without
175
+ drifting apart.
176
+
177
+ | Level | Builds | Against |
178
+ |---|---|---|
179
+ | Architect | The QA guidance: tier 1 scenarios and the tier 2 obligations shortlist | The agreed intent |
180
+ | Developers | The code, and the tests at their level: unit and integration | The scenarios and obligations handed down |
181
+ | End-to-end authors | The journeys | The tier 1 scenarios, taken by generation, not rewritten |
182
+
183
+ Read top to bottom, that is the flow of a feature. Developers prove their own seams close to the
184
+ code, and the tier 2 obligations are largely theirs, because partial writes, idempotency,
185
+ concurrency and boundaries live at the seams they build. The end-to-end layer proves the tier 1
186
+ scenarios through the interface a user actually walks.
187
+
188
+ The three levels are joined by one thing, the pack's shared primitive: **scenario ids**
189
+ ([`01-qa-approach.md`](01-qa-approach.md), section 4). They are written once. An integration test
190
+ cites the obligation id it instantiates; a journey cites the scenario id it proves. Nobody
191
+ restates the criteria, so coverage is computable across all three rather than counted separately
192
+ per team.
193
+
194
+ Two properties make this safe rather than merely tidy:
195
+
196
+ - **The end-to-end layer cannot drift from the guidance.** Journeys take their step text from the
197
+ acceptance documents by generation, not by copy
198
+ ([`04-playwright-journeys.md`](04-playwright-journeys.md), section 1), so rewording a scenario
199
+ breaks the citing journey until someone catches it up. Intent changes in one place and the
200
+ machinery forces the rest to follow, rather than the architect policing it by hand.
201
+ - **Automation follows the acceptance layer, never the reverse.** A journey with no scenario
202
+ behind it is a recording, not a proof. Neither the end-to-end authors nor the developers invent
203
+ criteria; both build against what was authored first.
204
+
205
+ What this means for the architect's own hands: they write criteria, not journeys, and not the
206
+ code. Authoring a scenario and automating it are different jobs, and conflating them pulls the
207
+ role back into building. The proof that the lower levels did their jobs is not that the architect
208
+ watched them work; it is the evidence bundle the journeys produce and the obligation citations
209
+ the integration tests carry, both read against the guidance.
210
+
211
+ ---
212
+
213
+ ## 6. The handoff
214
+
215
+ What the implementer receives is the acceptance spec (their roadmap), the technical brief (their
216
+ guide), and the decisions that are theirs to make (their room). What they should never receive is
217
+ the intent as a guess, an obligation list left to their diligence, or a contract they have to
218
+ invent by reading a mock-up.
219
+
220
+ **The handoff succeeds on the completeness of the brief, not on the implementer catching what it
221
+ missed.** Anything the brief leaves ambiguous may not be queried; it may simply be guessed, built
222
+ and reported done, and this becomes more likely the further the implementer sits from the
223
+ original conversation. So the bar is blunt: a faithful, literal reading of the acceptance spec
224
+ and the brief must produce the right software, with no discrimination required on the far side.
225
+
226
+ Pushback is welcome when it comes and occasionally catches something real, but it is not the
227
+ safety net. The safety net is the acceptance harness: generated journeys that cannot silently
228
+ skip a clause, and an evidence bundle that shows what was actually delivered rather than what was
229
+ claimed. That harness is what makes it safe to hand work to people who will build exactly what
230
+ was written, including its mistakes.
231
+
232
+ ---
233
+
234
+ ## 7. Vetting the returned work
235
+
236
+ The architect is the technical gate for work they did not write, and that gate has to be
237
+ affordable or it becomes a rubber stamp ([`01-qa-approach.md`](01-qa-approach.md), section 8). It
238
+ borrows the same principle as the rest of the pack: **check output, not input, and ask a specific
239
+ question derived from the change, never a general one.**
240
+
241
+ Vetting is not a line-by-line re-read. It is three questions, each answerable from an artefact
242
+ rather than from trust:
243
+
244
+ 1. **Does it honour the intent?** The tier 1 scenario's own words against what the journey
245
+ asserts ([`01-qa-approach.md`](01-qa-approach.md), section 4), watchable as a recording.
246
+ Seconds, not an afternoon.
247
+ 2. **Does it hold the contract?** The API shape, the event contract and the data model as handed
248
+ over, or with the divergence surfaced and agreed rather than quietly shipped.
249
+ 3. **Are the obligations discharged?** Each tier 2 id in the brief cited by a test that was
250
+ observed failing ([`06-engineering-obligations.md`](06-engineering-obligations.md)). A named id
251
+ with no citation is a gap, and it is a finite, checkable gap rather than a judgement call.
252
+
253
+ What is explicitly **not** the architect's to reject: how the implementer structured, named or
254
+ factored it, where the intent is honoured and the contract held. That is the line from section 2,
255
+ enforced at the moment it is most tempting to cross.
256
+
257
+ ---
258
+
259
+ ## 8. When the design is wrong: re-solutioning
260
+
261
+ A brief is a hypothesis, not a decree. It is the best available reading before the work exists,
262
+ and the work will teach things the reading could not. The pack already concedes this in the
263
+ status ladder: a scenario written at planning is `@status:proposed`, a proposal and not a
264
+ description ([`01-qa-approach.md`](01-qa-approach.md), section 5). Treating the first brief as
265
+ final is the waterfall error wearing a sprint's clothes.
266
+
267
+ Discovery mid-build is expected, and the response is **re-solutioning**, not silent absorption:
268
+
269
+ - **The loop runs through the architect, not around it.** When the build reveals the brief got
270
+ something wrong, or hit an edge nobody foresaw, the implementer surfaces it rather than
271
+ resolving it at the desk. The anomaly comes back up, it is re-solutioned, and an amended
272
+ criterion or brief flows back down.
273
+ - **The change is made visible by the machinery, not by memory.** Re-solutioning rewords a
274
+ clause, and rewording a clause breaks the journey that cited the old one
275
+ ([`04-playwright-journeys.md`](04-playwright-journeys.md), section 1). That is the feature, not
276
+ a nuisance: an amended requirement cannot quietly leave a test still proving the version it
277
+ replaced.
278
+ - **The loop is budgeted, not blamed.** A re-solutioned scenario is the model working. The real
279
+ failure is the same discovery being made three times by three implementers in parallel and
280
+ resolved three different ways, because it never came back up. Parallelism is what makes an
281
+ unsurfaced surprise expensive, which is why surfacing is the one obligation a team cannot
282
+ execute literally past.
283
+
284
+ And the point underneath it:
285
+
286
+ > Focus is the deliverable. A person carrying the cognitive load of building cannot also hold the
287
+ > whole edge-case and risk surface in view. A person whose only job is to design can.
288
+
289
+ Freed from implementing, attention goes where it is worth most: the edges, the failure modes, the
290
+ obligations, the risks. That is not a happy accident of the division of labour, it is a reason
291
+ for it. The model does not only change what delivery costs, it makes the design better, because
292
+ the designer is not also the builder.
293
+
294
+ ---
295
+
296
+ ## 9. Three horizons at once
297
+
298
+ The role does not run one sprint at a time. At any moment its work is spread across three
299
+ horizons, and holding all three is the shape of the job:
300
+
301
+ - **Ahead.** Designing the features of future cycles, so a de-risked brief is ready to pull
302
+ before the developers need it. This has to stay in front of delivery or the team runs dry.
303
+ - **Now.** Cycling with the developers on the work in flight: answering the open questions the
304
+ brief deliberately left them, and re-solutioning what the build turns up (section 8). Servicing
305
+ delivery without reaching into the how.
306
+ - **Behind.** Vetting and accepting the work completing from the last cycle, reading the evidence
307
+ against the guidance that was written for it (section 7).
308
+
309
+ That stagger is what makes the model work: the brief the developers pull today was designed while
310
+ they were building the last one. A role that only designed the current cycle would be a
311
+ bottleneck the team waited on. A role that designs ahead is a buffer the team draws from.
312
+
313
+ The honest hazard, worth managing deliberately: **reactive work crowds out proactive work.**
314
+ Servicing the current cycle is loud and immediate; designing ahead is quiet and can always be
315
+ done tomorrow. Left unmanaged, the current cycle consumes the day, forward design slips, and a
316
+ few cycles later the team pulls from an empty backlog and stalls. Design-ahead time is not the
317
+ slack in the role, it is the load-bearing part.
318
+
319
+ ---
320
+
321
+ ## 10. Where the roles stand in the lifecycle
322
+
323
+ The stages in [`08-definition-of-done.md`](08-definition-of-done.md) sequence the work. This
324
+ table names who stands at each gate, so the lifecycle and the roles are one description rather
325
+ than two.
326
+
327
+ | Stage | The architect | The team |
328
+ |---|---|---|
329
+ | Intake | Sources the intent; admits, returns or holds the request | |
330
+ | Plan | Writes tier 1, names the tier 2 shortlist, produces the brief | Challenges the brief and the contract |
331
+ | Confirm | Shares tier 1 with the intent owner, promotes off `derived` | |
332
+ | Build | Available for the open questions; does not reach into the how | Builds behind the contract, owns the local decisions |
333
+ | Gate | Runs the completeness sweep with the team; feeds gaps back into planning | Produces tests and evidence |
334
+ | Sign-off | Accepts against intent and contract; promotes status | Delivers the evidence bundle by one command |
335
+
336
+ The completeness sweep is worth calling out as a shared act rather than a checkpoint the
337
+ architect owns alone. Planning cannot enumerate what only emerges while building, and the person
338
+ who built it knows where the surfaces are. The sweep is where that knowledge feeds back into how
339
+ the next feature is planned, which is what turns the role from a bottleneck into a multiplier.
340
+
341
+ ---
342
+
343
+ ## What this file does not do
344
+
345
+ It does not restate the standard. The format, the tag vocabulary, the status ladder, the
346
+ lifecycle and the emit contract live where they live, and this file points at them rather than
347
+ copying them, for the same reason a consumer project holds pointers rather than restating the
348
+ pack ([`README.md`](README.md)). If a rule here appears to contradict one of those files, those
349
+ files are the authority and this one is the defect.
350
+
351
+ It also carries no organisation's instance data: no team's staffing, no business case for
352
+ resourcing the role, and no characterisation of a particular set of implementers. Those fail the
353
+ swap test in section 2 and belong with the project that has them, not here.
@@ -0,0 +1,273 @@
1
+ # The QA approach
2
+
3
+ The standard the rest of this pack implements. It is about connection, evidence and
4
+ enforcement rather than about adding rules, because reminders do not work: the rules that
5
+ failed me were already written down, already in front of me, and were read and then not
6
+ applied.
7
+
8
+ ---
9
+
10
+ ## 1. The constraint that shapes it
11
+
12
+ On a small team, and especially where one person designs and another executes, there is often
13
+ no peer who can challenge a claim. That makes **every claimed verification unfalsifiable
14
+ unless it produces an artefact somebody can inspect without redoing the work.**
15
+
16
+ That single sentence is why this standard is about evidence and traceability, and why the
17
+ cheapest artefact in the system is a two minute video rather than a paragraph asserting that
18
+ something was checked.
19
+
20
+ It has bitten me concretely. A feature branch once shipped documented, implemented, and with
21
+ every one of its nine manual test cases green, while the capability it described was
22
+ unreachable in the running product. The finding came from reading the server by hand. The QA
23
+ layer as it stood could not have produced it, because every test agreed with the software and
24
+ nothing in the process asked whether the software agreed with the intent.
25
+
26
+ ---
27
+
28
+ ## 2. Two tiers, split by readership
29
+
30
+ The split is by **who reads it**, and that is what keeps it honest.
31
+
32
+ **Tier 1, business acceptance.** Per actor, in the language of the person who wanted the
33
+ software. Ring up a sale, lock the till, hand over a shift, read the end of day report. No
34
+ concurrency, no retry semantics, no HTTP status codes. This is the tier a customer or product
35
+ owner reads and confirms.
36
+
37
+ **Tier 2, engineering obligations.** Partial writes, idempotency under retry and message
38
+ redelivery, optimistic concurrency races, audit ordering, malformed input, boundaries. Real,
39
+ mandatory, and invisible to the business. See [`06-engineering-obligations.md`](06-engineering-obligations.md).
40
+
41
+ The mapping between them is **asymmetric**, and this is the part that must not be softened:
42
+
43
+ ```
44
+ Tier 1 scenario -> 1..N UI journeys (near 1:1, user visible)
45
+ -> 1..N API tests (happy path / permission / invalid state)
46
+
47
+ Tier 2 obligation -> N API tests (NO tier 1 parent, and never will have one)
48
+ ```
49
+
50
+ > No operator will ever write "given the queue redelivers the drain message after the row was
51
+ > already written".
52
+
53
+ Deriving tier 2 from tier 1 loses it entirely. Every gap that motivated this standard was tier
54
+ 2. Putting tier 2 in front of the business is the opposite error: it buries the intent they
55
+ are being asked to approve.
56
+
57
+ **Edge cases a user could describe are not tier 2.** A wrong PIN, a lockout countdown, an
58
+ empty till, a plan tier that hides a button: all of those are ordinary use cases, each gets
59
+ its own scenario id, and each gets automated. An edge case only leaves tier 1 when no user
60
+ would ever phrase it.
61
+
62
+ ---
63
+
64
+ ## 3. Both tiers are enumerated at planning time
65
+
66
+ Before implementation, and before anyone has an interest in the list being short.
67
+
68
+ That placement is the whole point. Enumerating obligations at test writing time means the
69
+ person listing them is the person about to declare the work done, at the moment they most want
70
+ the list to be short.
71
+
72
+ Tier 2 additionally gets a **completeness sweep at the gate**, because planning cannot
73
+ enumerate surfaces that only emerge while building. Anything the sweep finds that planning
74
+ missed is fed back into how planning is done, so the next item plans better. Neither is
75
+ sufficient alone: planning alone misses what emerged, and the sweep alone is the fox counting
76
+ the hens.
77
+
78
+ ---
79
+
80
+ ## 4. The shared primitive: scenario ids
81
+
82
+ One vocabulary, cited by every stream, so coverage is computable across a feature rather than
83
+ per test suite.
84
+
85
+ - Scenarios live in `acceptance/<area>/<area>-acceptance.md` and carry stable ids as Gherkin
86
+ tags, for example `@LOCK-OPEN-001`.
87
+ - A UI journey **cites the id it proves**, and takes its step names from the scenario's own
88
+ words.
89
+ - An API test cites the scenario id for tier 1 behaviour, or the obligation it instantiates
90
+ for tier 2.
91
+
92
+ **Gherkin is the format** because it is the only candidate that is genuinely readable as prose
93
+ and structurally parseable at the same time: actor, precondition, action and outcome are
94
+ carried by the form itself, and a `Then` clause is an assertion in prose that maps close to
95
+ 1:1 onto a test assertion.
96
+
97
+ That mapping is what makes review affordable. The reviewable artefact is a side by side of the
98
+ scenario's own words against what the test actually asserts, so a mismatch is visible in
99
+ seconds without opening the test file:
100
+
101
+ ```
102
+ @TILL-PAY-004 A sale cannot be completed for less than the amount due
103
+ Then the sale is refused
104
+ and the drawer does not open
105
+ and the cart is left untouched
106
+ ------------------------------------------------------------
107
+ Test rejects_short_payment_and_leaves_cart_intact
108
+ Asserts result.success === false
109
+ drawerOpened === false
110
+ cart.lines.length === 2
111
+ ```
112
+
113
+ **The scenario's words reach the test by generation, never by retyping.** The clauses are
114
+ parsed out of the acceptance documents into a typed module the journeys reference, so a
115
+ scenario and the journey citing it cannot hold two different versions of the same sentence,
116
+ and a journey cannot silently prove three of five clauses. See section 1 of
117
+ [`04-playwright-journeys.md`](04-playwright-journeys.md). A convention would not survive this,
118
+ because keeping two copies of a sentence aligned by care is exactly the discipline that
119
+ already failed elsewhere.
120
+
121
+ **Known limit, and it must be stated wherever this is described.** That side by side catches
122
+ "the test does not do what it says". It does not catch "the scenario describes a situation
123
+ that cannot occur", and it does not catch a clause whose body asserts nothing. Only
124
+ demonstration against running software closes the first gap, which is why video is part of the
125
+ evidence and not decoration.
126
+
127
+ ### Traceability runs both ways
128
+
129
+ Or it decays into an instruction to be thorough, and gets skipped when the change is small and
130
+ the day is short:
131
+
132
+ - change a scenario, and you get the list of journeys and API tests citing it
133
+ - change an API contract, and you get the scenarios that depend on it
134
+ - delete or rename a scenario, and you get the citations that are now orphaned
135
+
136
+ The output is a finite list to work through, not a judgement call.
137
+
138
+ ---
139
+
140
+ ## 5. Confirmation status: derived, verified, accepted
141
+
142
+ Every scenario carries one. `derived` is the honest default for anything written up from
143
+ software that already exists.
144
+
145
+ | Status | Meaning |
146
+ |---|---|
147
+ | `proposed` | Written at planning, before the software exists. A proposal, not a description. |
148
+ | `derived` | Written up from the delivered software. Describes what it appears to do. |
149
+ | `verified` | A human watched the software do it, recorded with date and commit. |
150
+ | `accepted` | Confirmed as what the software **should** do. |
151
+
152
+ A test citing a `derived` scenario proves the test runs, not that the requirement is right.
153
+
154
+ **The gap between `derived` and `accepted` is where defects hide, and making it visible is
155
+ most of the value.** A single number ("94% pass") flattens that distinction into nothing. The
156
+ status ladder keeps the unconfirmed surface on screen and hands the backfill a concrete
157
+ worklist.
158
+
159
+ It is also the anti flood mechanism for review: `accepted` items drop out of the review queue
160
+ entirely, so what stays visible is precisely the unsettled surface. That costs nothing extra,
161
+ because the statuses are already tracked.
162
+
163
+ ---
164
+
165
+ ## 6. Intent cannot be reverse engineered
166
+
167
+ The most important rule in the pack, and the one most easily skipped.
168
+
169
+ Code contains what a system **does**. It never contains why anyone wanted it. Anyone, human or
170
+ agent, asked to document intent while reading only the implementation will produce intent
171
+ fluently, plausibly, and fabricated.
172
+
173
+ So every scenario written up from existing software carries a provenance line naming where its
174
+ intent came from:
175
+
176
+ ```
177
+ <!-- intent: design-notes/till-lock.md, section 4 -->
178
+ <!-- intent: stated by the product owner, 2026-03-11, recorded in decisions.md D-018 -->
179
+ <!-- intent: INFERRED from implementation - needs confirmation -->
180
+ ```
181
+
182
+ That is not bookkeeping. It is what makes review affordable, because attention goes to the
183
+ `INFERRED` ones instead of spreading evenly across everything. Where no source exists, intent
184
+ is a **question**, not a statement.
185
+
186
+ This is what decides whether writing criteria against existing software is worth doing at all:
187
+
188
+ > Use cases built from observed behaviour yield tests asserting that the software does what
189
+ > the software does. They cannot fail. Use cases built from sourced intent yield tests that
190
+ > can disagree with the delivered software, and a disagreement is a defect found.
191
+
192
+ Two corollaries:
193
+
194
+ - **Demonstration is of the running software, not of the code.** Citing a file and line proves
195
+ a line was read. Running the flow proves the system does it.
196
+ - **When comparing findings against intent, order matters.** Observed behaviour first, then
197
+ the intent stated by whoever owns it, stated **before** they see any inferred version, then
198
+ the written up intent. The other order confirms a guess instead of testing it.
199
+
200
+ **The highest value output of this exercise is the anomaly list, not the document.** States
201
+ nothing can write, guards no caller reaches, buttons with no behaviour behind them, comments
202
+ contradicting code. Derived blindly, a status with four guards and no writer reads as "the
203
+ system supports this". Derived carefully, it is a defect report.
204
+
205
+ ---
206
+
207
+ ## 7. Evidence, not ticks
208
+
209
+ **No question is answered "yes".** A checklist has already failed here: asked "did you cover
210
+ the edge cases?", I would have answered yes in good faith and been wrong.
211
+
212
+ Questions are of three kinds, and each is answered differently:
213
+
214
+ - **Enumerable** questions are answered with a **list**. "Which actors can reach this?" is
215
+ answered by naming every one, **including those that must be refused**. Answering from
216
+ assumption rather than enumeration is how two tests once shipped asserting a permission
217
+ boundary that did not exist.
218
+ - **Provable** questions are answered with a **test name**, and that test must have been
219
+ observed failing. "Is this safe under concurrency" is not a judgement call.
220
+ - **Judgement** questions are answered with a **specific sentence**. "Is this efficient" is
221
+ settled by naming the N+1 that was checked and is not there, or the lock scope and why it is
222
+ that wide. A bare yes launders an unexamined assumption into a recorded answer.
223
+
224
+ **Falsifiability is recorded, not implied.** A test never observed red is not evidence. Where
225
+ a test was proved able to fail, the file records the specific mutation and the result:
226
+
227
+ ```ts
228
+ // Proved red by removing the same-cashier guard in the unlock handler; this test
229
+ // failed on exactly the "shift is unchanged" assertion, not on setup.
230
+ ```
231
+
232
+ Inventing a plausible mutation is a much higher bar to bluff than a tick, and it is the manual
233
+ stand in for mutation testing over the changed files.
234
+
235
+ ---
236
+
237
+ ## 8. Design rule: optimise verification cost, not authoring cost
238
+
239
+ **Every artefact must be checkable in less time than it took to produce.**
240
+
241
+ A record needing twenty minutes of scrutiny will not be scrutinised. The reviewer becomes the
242
+ bottleneck and waves it through, which is precisely how bad work passes. A video takes two
243
+ minutes. A citation check is instant, because it is mechanical. Batches stay small and ordered
244
+ riskiest first, because forty items presented at once get rubber stamped.
245
+
246
+ This is also the argument for narrating a recorded run with the scenario's own wording: what a
247
+ viewer is told and what the journey proves are then the same sentence, and no second script
248
+ exists that could drift from the criteria.
249
+
250
+ ---
251
+
252
+ ## 9. What mechanical checking can and cannot do
253
+
254
+ Generating the clauses into the journeys (section 1 of
255
+ [`04-playwright-journeys.md`](04-playwright-journeys.md)) moves several of these from "a check
256
+ that has to be written and run" to "the build does not compile": a cited id that does not
257
+ exist, an orphaned citation after a rename, a missing clause, and a step the document does not
258
+ contain.
259
+
260
+ What is left for a citation check, all mechanically decidable:
261
+
262
+ 1. every scenario id is cited by at least one journey, or carries a recorded reason it cannot
263
+ be
264
+ 2. every cited test exists in the sources
265
+ 3. every cited test appears in the change under review, so citing pre existing unrelated tests
266
+ does not count
267
+ 4. every scenario carries a status and, where the status is `verified`, a date and commit
268
+ 5. no clause body is empty of assertions
269
+
270
+ **It never verifies that the answers are true.** Green means "well formed, and the citations
271
+ resolve to real tests in this change". It never means "the coverage is adequate". That
272
+ distinction has to be printed wherever the check is documented, or the check becomes exactly
273
+ the false assurance it was built to prevent.