criteria-review 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +299 -0
- package/agents/README.md +123 -0
- package/agents/skills/claude-code/criteria-lookup/SKILL.md +236 -0
- package/bin/criteria-review.js +870 -0
- package/docs/decisions.md +310 -0
- package/docs/help/01-getting-started.md +170 -0
- package/docs/standard/00-roles-and-ownership.md +353 -0
- package/docs/standard/01-qa-approach.md +273 -0
- package/docs/standard/02-writing-acceptance-criteria.md +187 -0
- package/docs/standard/03-example-acceptance-document.md +244 -0
- package/docs/standard/04-playwright-journeys.md +329 -0
- package/docs/standard/05-example-journey.spec.ts +130 -0
- package/docs/standard/06-engineering-obligations.md +170 -0
- package/docs/standard/07-manual-qa-and-defects.md +138 -0
- package/docs/standard/08-definition-of-done.md +128 -0
- package/docs/standard/09-recording-journeys.md +644 -0
- package/docs/standard/README.md +70 -0
- package/docs/standard/adopting-the-standard.md +104 -0
- package/docs/standard/emit-contract.md +175 -0
- package/package.json +41 -0
- package/public/app.js +791 -0
- package/public/doc-name.js +35 -0
- package/public/index.html +100 -0
- package/public/markdown.js +0 -0
- package/public/queue-order.js +78 -0
- package/public/styles.css +414 -0
- package/src/config.js +104 -0
- package/src/emit.js +183 -0
- package/src/media.js +126 -0
- package/src/parse.js +298 -0
- package/src/scan.js +158 -0
- package/src/server.js +433 -0
- package/src/standard.js +174 -0
- package/src/version.js +19 -0
- package/src/videos.js +185 -0
- package/src/write.js +307 -0
|
@@ -0,0 +1,353 @@
|
|
|
1
|
+
# Roles and ownership
|
|
2
|
+
|
|
3
|
+
Who stands where. The rest of the pack describes the machinery; this file describes the people
|
|
4
|
+
the machinery is arranged around, what each of them owns, and where the boundary between
|
|
5
|
+
deciding and building falls.
|
|
6
|
+
|
|
7
|
+
Two roles carry the work, and this document names them as the tool does:
|
|
8
|
+
|
|
9
|
+
- **The architect** decides what the software *should* do. Organisations title this differently:
|
|
10
|
+
solutions architect, technical lead, product engineer. The title is not the point; the remit
|
|
11
|
+
is.
|
|
12
|
+
- **The implementers** decide how it is realised in code, and prove their own seams.
|
|
13
|
+
|
|
14
|
+
One person can hold both roles on a small project, and the boundary still matters, because it is
|
|
15
|
+
what makes the criteria worth writing rather than a formality performed after the fact.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## The one idea underneath the role
|
|
20
|
+
|
|
21
|
+
> Solutioning is finished when an implementer can ask *how do I build this cleanly* instead of
|
|
22
|
+
> *is this even the right thing*. Everything before that line is the architect's work.
|
|
23
|
+
> Everything after it is the team's.
|
|
24
|
+
|
|
25
|
+
The acceptance criteria are what draw that line, and this is what connects the pack to the whole
|
|
26
|
+
team: **a tier 1 document agreed before implementation is not a QA artefact that happens early.
|
|
27
|
+
It is the specification.** Written and confirmed up front, it removes the one question a skilled
|
|
28
|
+
implementer cannot answer for themselves, which is what the business actually wanted, and leaves
|
|
29
|
+
them the questions they are best placed to answer: how to build it, how to structure it, how to
|
|
30
|
+
make it fast and maintainable.
|
|
31
|
+
|
|
32
|
+
So the QA roadmap is not downstream of design. It is the guide to it. The obligation to
|
|
33
|
+
enumerate both tiers before anyone writes code ([`08-definition-of-done.md`](08-definition-of-done.md),
|
|
34
|
+
stage 1) is the same obligation as designing the feature. There is no separate design step that
|
|
35
|
+
the criteria then check. The criteria, agreed, are the design.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## 1. The role has two faces, and they are different jobs
|
|
40
|
+
|
|
41
|
+
Facing whoever wanted the software, which may be a customer, product, support or pre-sales, the
|
|
42
|
+
job is to **turn a want into an agreed intent**. This is where the "so that" is nailed down
|
|
43
|
+
([`02-writing-acceptance-criteria.md`](02-writing-acceptance-criteria.md)), where a request stops
|
|
44
|
+
being a solution somebody guessed at and becomes the outcome it was really about. The output is
|
|
45
|
+
a tier 1 acceptance document, in the requester's language, that they have confirmed.
|
|
46
|
+
|
|
47
|
+
Facing the implementers, the job is to **turn agreed intent into a de-risked technical brief**.
|
|
48
|
+
This is where tier 2 obligations are named from the catalogue
|
|
49
|
+
([`06-engineering-obligations.md`](06-engineering-obligations.md)), where the edge cases no user
|
|
50
|
+
would ever describe are worked through, and where hard unknowns are spiked before they reach a
|
|
51
|
+
sprint.
|
|
52
|
+
|
|
53
|
+
These are sequenced, not parallel, and the order is load bearing:
|
|
54
|
+
|
|
55
|
+
> Confirm the intent with whoever owns it **before** the technical edge-casing, not after.
|
|
56
|
+
|
|
57
|
+
The requester confirms behaviour in their own language, and the tier 2 work is invisible to them
|
|
58
|
+
([`01-qa-approach.md`](01-qa-approach.md), section 2). Doing the technical edge-casing first
|
|
59
|
+
buries the intent they are being asked to approve under machinery they cannot read, and the
|
|
60
|
+
confirmation stops being a confirmation. Intent first, in their words. Obligations second, in
|
|
61
|
+
ours.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## 2. What the architect owns, and what the team owns
|
|
66
|
+
|
|
67
|
+
The single decision that determines whether the model scales or collapses back onto one person.
|
|
68
|
+
Draw the line in the wrong place and the role becomes a bottleneck with a better title.
|
|
69
|
+
|
|
70
|
+
The test is the one the standard uses to decide where a sentence lives: **does the decision stay
|
|
71
|
+
the same if you swapped the implementers for a different, equally skilled team?** If yes, the
|
|
72
|
+
architect owns it. If no, if it depends on how *this* person writes *this* code, it belongs to
|
|
73
|
+
the implementer.
|
|
74
|
+
|
|
75
|
+
| The architect owns | The team owns |
|
|
76
|
+
|---|---|
|
|
77
|
+
| The intent, and its confirmation with whoever owns it | How the intent is realised in code |
|
|
78
|
+
| The tier 1 scenarios and their wording | The internal structure of the implementation |
|
|
79
|
+
| The tier 2 obligations shortlist for the feature | Which patterns, types and abstractions sit behind the contract |
|
|
80
|
+
| Contracts across a boundary: API shape, event and message contracts, the data model | The implementation behind the contract |
|
|
81
|
+
| Cross-cutting choices: identity, authorisation surface, tenancy, where state lives | Local, cheap-to-reverse decisions inside a service |
|
|
82
|
+
| The hard-to-reverse and the high-risk | The many cheap-to-reverse decisions |
|
|
83
|
+
| Acceptance of the returned work against the brief | Getting the suite green and the evidence produced |
|
|
84
|
+
|
|
85
|
+
Both failure modes are worth naming, because each is invisible from inside it.
|
|
86
|
+
|
|
87
|
+
**Drawn too low:** if vetting reaches down to variable names, method shape, or which construct
|
|
88
|
+
you would have used, the architect has recreated themselves as the team's editor. Vetting asks
|
|
89
|
+
*does this honour the intent and the contract*, never *would I have written it this way*.
|
|
90
|
+
|
|
91
|
+
**Drawn too high:** handing the team a one-line want and calling it delegation. Intent left to be
|
|
92
|
+
inferred will be inferred fluently and wrongly, which is the failure
|
|
93
|
+
[`01-qa-approach.md`](01-qa-approach.md) section 6 exists to prevent. The brief is what stops
|
|
94
|
+
that, and its whole purpose is to make the intent unnecessary to guess.
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## 3. Intake: vetting a request before it is real work
|
|
99
|
+
|
|
100
|
+
A request arriving from outside the team is a want, not a specification, and admitting it to the
|
|
101
|
+
backlog as though it were one is how unsourced intent enters the system. Intake is a gate, and it
|
|
102
|
+
sits deliberately at the front, before anyone has an interest in the list being short
|
|
103
|
+
([`01-qa-approach.md`](01-qa-approach.md), section 3).
|
|
104
|
+
|
|
105
|
+
A request is not ready to be worked until it carries:
|
|
106
|
+
|
|
107
|
+
- **A named actor and a real "so that".** Not "add a filter", but who is blocked without it and
|
|
108
|
+
what they cannot currently do. A request whose "so that" is circular is a heading, not a story
|
|
109
|
+
([`02-writing-acceptance-criteria.md`](02-writing-acceptance-criteria.md)).
|
|
110
|
+
- **Sourced intent, or an explicit question in its place.** Every claim about what was wanted
|
|
111
|
+
cites where it came from: a dated conversation with a named owner, a support ticket, a
|
|
112
|
+
commitment recorded somewhere. Where no source exists, the request carries the question rather
|
|
113
|
+
than a confident guess. Intent is never reverse engineered from what a competitor does or from
|
|
114
|
+
what seems reasonable.
|
|
115
|
+
- **The scope line.** One sentence naming what is in and, more usefully, what is deliberately
|
|
116
|
+
out. Most disputes at delivery are about the out.
|
|
117
|
+
|
|
118
|
+
The output of intake is not "approved". It is one of three: **admitted** with its intent sourced,
|
|
119
|
+
**returned** to the requester with the specific missing piece named, or **held** as a question
|
|
120
|
+
against the person who owns the answer. The three are visible states rather than a private
|
|
121
|
+
judgement, for the same reason the status ladder is visible: the unsettled surface has to stay on
|
|
122
|
+
screen ([`01-qa-approach.md`](01-qa-approach.md), section 5).
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
## 4. The two products
|
|
127
|
+
|
|
128
|
+
Per feature, the architect produces exactly two things, and the team should be able to start from
|
|
129
|
+
them alone.
|
|
130
|
+
|
|
131
|
+
### 4a. The acceptance spec, which is the roadmap
|
|
132
|
+
|
|
133
|
+
The tier 1 document and the tier 2 shortlist, produced as
|
|
134
|
+
[`08-definition-of-done.md`](08-definition-of-done.md) stage 1 requires. What matters here is
|
|
135
|
+
what it *is* to the team: every tier 1 scenario is behaviour the implementer must deliver and
|
|
136
|
+
will be measured against, and every tier 2 obligation is an edge they are on notice to handle.
|
|
137
|
+
Enumerated up front, it is the map of the whole surface, drawn before the terrain tempts anyone
|
|
138
|
+
to shorten it.
|
|
139
|
+
|
|
140
|
+
Shared with the requester minus the tier 2 machinery, and confirmed, it becomes the thing that
|
|
141
|
+
carries force in a disagreement: *agreed*, not *assumed*.
|
|
142
|
+
|
|
143
|
+
### 4b. The technical brief, which is the guide
|
|
144
|
+
|
|
145
|
+
What the acceptance spec does not carry: the design decisions the architect has taken so the team
|
|
146
|
+
does not have to, and the risks retired so the team does not hit them mid-sprint. It is short on
|
|
147
|
+
purpose. If it is long, the architect is doing the team's job.
|
|
148
|
+
|
|
149
|
+
| Section | What it is |
|
|
150
|
+
|---|---|
|
|
151
|
+
| Intent, in one paragraph | The agreed "so that", so the brief reads without the acceptance document open beside it |
|
|
152
|
+
| The obligations shortlist | The tier 2 ids this feature instantiates, each a concrete list of tests rather than a feeling |
|
|
153
|
+
| Contracts | API shape, event and message contracts, and the data model, stated as the boundary the team builds behind |
|
|
154
|
+
| Decisions taken | Cross-cutting calls already made, each with a one-line why, so they are not silently relitigated |
|
|
155
|
+
| Decisions left open | The local calls the team owns, named as theirs, so delegation is explicit rather than a gap |
|
|
156
|
+
| Risks retired | The spikes done and what they proved; the unknowns that would otherwise surface mid-sprint |
|
|
157
|
+
| Out of scope | What this deliberately does not do, so a returned implementation is not marked short for not doing it |
|
|
158
|
+
|
|
159
|
+
The "decisions left open" row is where the economics live. **Specify the outcome completely,
|
|
160
|
+
because a gap will not be correctly filled by someone who was not in the conversation. Do not
|
|
161
|
+
specify the mechanism**, because the moment the brief pre-writes the code, designing costs what
|
|
162
|
+
building costs and the reason for separating them is gone.
|
|
163
|
+
|
|
164
|
+
The line to hold: the *what* and the *must-be-true* are pinned to the point of no ambiguity; the
|
|
165
|
+
*how* is left as genuine mechanism the implementers own. That division is not a courtesy. It is
|
|
166
|
+
what keeps designing cheaper than building, and it is the hardest judgement in the role. Specify
|
|
167
|
+
too little and you get wrong software; too much and you have simply built it yourself, slowly.
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## 5. The three levels: who builds which tests
|
|
172
|
+
|
|
173
|
+
The architect's output is not tests. It is the guidance tests are built from. Below it sit two
|
|
174
|
+
layers of proof, and the acceptance criteria are what let all three work independently without
|
|
175
|
+
drifting apart.
|
|
176
|
+
|
|
177
|
+
| Level | Builds | Against |
|
|
178
|
+
|---|---|---|
|
|
179
|
+
| Architect | The QA guidance: tier 1 scenarios and the tier 2 obligations shortlist | The agreed intent |
|
|
180
|
+
| Developers | The code, and the tests at their level: unit and integration | The scenarios and obligations handed down |
|
|
181
|
+
| End-to-end authors | The journeys | The tier 1 scenarios, taken by generation, not rewritten |
|
|
182
|
+
|
|
183
|
+
Read top to bottom, that is the flow of a feature. Developers prove their own seams close to the
|
|
184
|
+
code, and the tier 2 obligations are largely theirs, because partial writes, idempotency,
|
|
185
|
+
concurrency and boundaries live at the seams they build. The end-to-end layer proves the tier 1
|
|
186
|
+
scenarios through the interface a user actually walks.
|
|
187
|
+
|
|
188
|
+
The three levels are joined by one thing, the pack's shared primitive: **scenario ids**
|
|
189
|
+
([`01-qa-approach.md`](01-qa-approach.md), section 4). They are written once. An integration test
|
|
190
|
+
cites the obligation id it instantiates; a journey cites the scenario id it proves. Nobody
|
|
191
|
+
restates the criteria, so coverage is computable across all three rather than counted separately
|
|
192
|
+
per team.
|
|
193
|
+
|
|
194
|
+
Two properties make this safe rather than merely tidy:
|
|
195
|
+
|
|
196
|
+
- **The end-to-end layer cannot drift from the guidance.** Journeys take their step text from the
|
|
197
|
+
acceptance documents by generation, not by copy
|
|
198
|
+
([`04-playwright-journeys.md`](04-playwright-journeys.md), section 1), so rewording a scenario
|
|
199
|
+
breaks the citing journey until someone catches it up. Intent changes in one place and the
|
|
200
|
+
machinery forces the rest to follow, rather than the architect policing it by hand.
|
|
201
|
+
- **Automation follows the acceptance layer, never the reverse.** A journey with no scenario
|
|
202
|
+
behind it is a recording, not a proof. Neither the end-to-end authors nor the developers invent
|
|
203
|
+
criteria; both build against what was authored first.
|
|
204
|
+
|
|
205
|
+
What this means for the architect's own hands: they write criteria, not journeys, and not the
|
|
206
|
+
code. Authoring a scenario and automating it are different jobs, and conflating them pulls the
|
|
207
|
+
role back into building. The proof that the lower levels did their jobs is not that the architect
|
|
208
|
+
watched them work; it is the evidence bundle the journeys produce and the obligation citations
|
|
209
|
+
the integration tests carry, both read against the guidance.
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## 6. The handoff
|
|
214
|
+
|
|
215
|
+
What the implementer receives is the acceptance spec (their roadmap), the technical brief (their
|
|
216
|
+
guide), and the decisions that are theirs to make (their room). What they should never receive is
|
|
217
|
+
the intent as a guess, an obligation list left to their diligence, or a contract they have to
|
|
218
|
+
invent by reading a mock-up.
|
|
219
|
+
|
|
220
|
+
**The handoff succeeds on the completeness of the brief, not on the implementer catching what it
|
|
221
|
+
missed.** Anything the brief leaves ambiguous may not be queried; it may simply be guessed, built
|
|
222
|
+
and reported done, and this becomes more likely the further the implementer sits from the
|
|
223
|
+
original conversation. So the bar is blunt: a faithful, literal reading of the acceptance spec
|
|
224
|
+
and the brief must produce the right software, with no discrimination required on the far side.
|
|
225
|
+
|
|
226
|
+
Pushback is welcome when it comes and occasionally catches something real, but it is not the
|
|
227
|
+
safety net. The safety net is the acceptance harness: generated journeys that cannot silently
|
|
228
|
+
skip a clause, and an evidence bundle that shows what was actually delivered rather than what was
|
|
229
|
+
claimed. That harness is what makes it safe to hand work to people who will build exactly what
|
|
230
|
+
was written, including its mistakes.
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
## 7. Vetting the returned work
|
|
235
|
+
|
|
236
|
+
The architect is the technical gate for work they did not write, and that gate has to be
|
|
237
|
+
affordable or it becomes a rubber stamp ([`01-qa-approach.md`](01-qa-approach.md), section 8). It
|
|
238
|
+
borrows the same principle as the rest of the pack: **check output, not input, and ask a specific
|
|
239
|
+
question derived from the change, never a general one.**
|
|
240
|
+
|
|
241
|
+
Vetting is not a line-by-line re-read. It is three questions, each answerable from an artefact
|
|
242
|
+
rather than from trust:
|
|
243
|
+
|
|
244
|
+
1. **Does it honour the intent?** The tier 1 scenario's own words against what the journey
|
|
245
|
+
asserts ([`01-qa-approach.md`](01-qa-approach.md), section 4), watchable as a recording.
|
|
246
|
+
Seconds, not an afternoon.
|
|
247
|
+
2. **Does it hold the contract?** The API shape, the event contract and the data model as handed
|
|
248
|
+
over, or with the divergence surfaced and agreed rather than quietly shipped.
|
|
249
|
+
3. **Are the obligations discharged?** Each tier 2 id in the brief cited by a test that was
|
|
250
|
+
observed failing ([`06-engineering-obligations.md`](06-engineering-obligations.md)). A named id
|
|
251
|
+
with no citation is a gap, and it is a finite, checkable gap rather than a judgement call.
|
|
252
|
+
|
|
253
|
+
What is explicitly **not** the architect's to reject: how the implementer structured, named or
|
|
254
|
+
factored it, where the intent is honoured and the contract held. That is the line from section 2,
|
|
255
|
+
enforced at the moment it is most tempting to cross.
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## 8. When the design is wrong: re-solutioning
|
|
260
|
+
|
|
261
|
+
A brief is a hypothesis, not a decree. It is the best available reading before the work exists,
|
|
262
|
+
and the work will teach things the reading could not. The pack already concedes this in the
|
|
263
|
+
status ladder: a scenario written at planning is `@status:proposed`, a proposal and not a
|
|
264
|
+
description ([`01-qa-approach.md`](01-qa-approach.md), section 5). Treating the first brief as
|
|
265
|
+
final is the waterfall error wearing a sprint's clothes.
|
|
266
|
+
|
|
267
|
+
Discovery mid-build is expected, and the response is **re-solutioning**, not silent absorption:
|
|
268
|
+
|
|
269
|
+
- **The loop runs through the architect, not around it.** When the build reveals the brief got
|
|
270
|
+
something wrong, or hit an edge nobody foresaw, the implementer surfaces it rather than
|
|
271
|
+
resolving it at the desk. The anomaly comes back up, it is re-solutioned, and an amended
|
|
272
|
+
criterion or brief flows back down.
|
|
273
|
+
- **The change is made visible by the machinery, not by memory.** Re-solutioning rewords a
|
|
274
|
+
clause, and rewording a clause breaks the journey that cited the old one
|
|
275
|
+
([`04-playwright-journeys.md`](04-playwright-journeys.md), section 1). That is the feature, not
|
|
276
|
+
a nuisance: an amended requirement cannot quietly leave a test still proving the version it
|
|
277
|
+
replaced.
|
|
278
|
+
- **The loop is budgeted, not blamed.** A re-solutioned scenario is the model working. The real
|
|
279
|
+
failure is the same discovery being made three times by three implementers in parallel and
|
|
280
|
+
resolved three different ways, because it never came back up. Parallelism is what makes an
|
|
281
|
+
unsurfaced surprise expensive, which is why surfacing is the one obligation a team cannot
|
|
282
|
+
execute literally past.
|
|
283
|
+
|
|
284
|
+
And the point underneath it:
|
|
285
|
+
|
|
286
|
+
> Focus is the deliverable. A person carrying the cognitive load of building cannot also hold the
|
|
287
|
+
> whole edge-case and risk surface in view. A person whose only job is to design can.
|
|
288
|
+
|
|
289
|
+
Freed from implementing, attention goes where it is worth most: the edges, the failure modes, the
|
|
290
|
+
obligations, the risks. That is not a happy accident of the division of labour, it is a reason
|
|
291
|
+
for it. The model does not only change what delivery costs, it makes the design better, because
|
|
292
|
+
the designer is not also the builder.
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
## 9. Three horizons at once
|
|
297
|
+
|
|
298
|
+
The role does not run one sprint at a time. At any moment its work is spread across three
|
|
299
|
+
horizons, and holding all three is the shape of the job:
|
|
300
|
+
|
|
301
|
+
- **Ahead.** Designing the features of future cycles, so a de-risked brief is ready to pull
|
|
302
|
+
before the developers need it. This has to stay in front of delivery or the team runs dry.
|
|
303
|
+
- **Now.** Cycling with the developers on the work in flight: answering the open questions the
|
|
304
|
+
brief deliberately left them, and re-solutioning what the build turns up (section 8). Servicing
|
|
305
|
+
delivery without reaching into the how.
|
|
306
|
+
- **Behind.** Vetting and accepting the work completing from the last cycle, reading the evidence
|
|
307
|
+
against the guidance that was written for it (section 7).
|
|
308
|
+
|
|
309
|
+
That stagger is what makes the model work: the brief the developers pull today was designed while
|
|
310
|
+
they were building the last one. A role that only designed the current cycle would be a
|
|
311
|
+
bottleneck the team waited on. A role that designs ahead is a buffer the team draws from.
|
|
312
|
+
|
|
313
|
+
The honest hazard, worth managing deliberately: **reactive work crowds out proactive work.**
|
|
314
|
+
Servicing the current cycle is loud and immediate; designing ahead is quiet and can always be
|
|
315
|
+
done tomorrow. Left unmanaged, the current cycle consumes the day, forward design slips, and a
|
|
316
|
+
few cycles later the team pulls from an empty backlog and stalls. Design-ahead time is not the
|
|
317
|
+
slack in the role, it is the load-bearing part.
|
|
318
|
+
|
|
319
|
+
---
|
|
320
|
+
|
|
321
|
+
## 10. Where the roles stand in the lifecycle
|
|
322
|
+
|
|
323
|
+
The stages in [`08-definition-of-done.md`](08-definition-of-done.md) sequence the work. This
|
|
324
|
+
table names who stands at each gate, so the lifecycle and the roles are one description rather
|
|
325
|
+
than two.
|
|
326
|
+
|
|
327
|
+
| Stage | The architect | The team |
|
|
328
|
+
|---|---|---|
|
|
329
|
+
| Intake | Sources the intent; admits, returns or holds the request | |
|
|
330
|
+
| Plan | Writes tier 1, names the tier 2 shortlist, produces the brief | Challenges the brief and the contract |
|
|
331
|
+
| Confirm | Shares tier 1 with the intent owner, promotes off `derived` | |
|
|
332
|
+
| Build | Available for the open questions; does not reach into the how | Builds behind the contract, owns the local decisions |
|
|
333
|
+
| Gate | Runs the completeness sweep with the team; feeds gaps back into planning | Produces tests and evidence |
|
|
334
|
+
| Sign-off | Accepts against intent and contract; promotes status | Delivers the evidence bundle by one command |
|
|
335
|
+
|
|
336
|
+
The completeness sweep is worth calling out as a shared act rather than a checkpoint the
|
|
337
|
+
architect owns alone. Planning cannot enumerate what only emerges while building, and the person
|
|
338
|
+
who built it knows where the surfaces are. The sweep is where that knowledge feeds back into how
|
|
339
|
+
the next feature is planned, which is what turns the role from a bottleneck into a multiplier.
|
|
340
|
+
|
|
341
|
+
---
|
|
342
|
+
|
|
343
|
+
## What this file does not do
|
|
344
|
+
|
|
345
|
+
It does not restate the standard. The format, the tag vocabulary, the status ladder, the
|
|
346
|
+
lifecycle and the emit contract live where they live, and this file points at them rather than
|
|
347
|
+
copying them, for the same reason a consumer project holds pointers rather than restating the
|
|
348
|
+
pack ([`README.md`](README.md)). If a rule here appears to contradict one of those files, those
|
|
349
|
+
files are the authority and this one is the defect.
|
|
350
|
+
|
|
351
|
+
It also carries no organisation's instance data: no team's staffing, no business case for
|
|
352
|
+
resourcing the role, and no characterisation of a particular set of implementers. Those fail the
|
|
353
|
+
swap test in section 2 and belong with the project that has them, not here.
|
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
# The QA approach
|
|
2
|
+
|
|
3
|
+
The standard the rest of this pack implements. It is about connection, evidence and
|
|
4
|
+
enforcement rather than about adding rules, because reminders do not work: the rules that
|
|
5
|
+
failed me were already written down, already in front of me, and were read and then not
|
|
6
|
+
applied.
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## 1. The constraint that shapes it
|
|
11
|
+
|
|
12
|
+
On a small team, and especially where one person designs and another executes, there is often
|
|
13
|
+
no peer who can challenge a claim. That makes **every claimed verification unfalsifiable
|
|
14
|
+
unless it produces an artefact somebody can inspect without redoing the work.**
|
|
15
|
+
|
|
16
|
+
That single sentence is why this standard is about evidence and traceability, and why the
|
|
17
|
+
cheapest artefact in the system is a two minute video rather than a paragraph asserting that
|
|
18
|
+
something was checked.
|
|
19
|
+
|
|
20
|
+
It has bitten me concretely. A feature branch once shipped documented, implemented, and with
|
|
21
|
+
every one of its nine manual test cases green, while the capability it described was
|
|
22
|
+
unreachable in the running product. The finding came from reading the server by hand. The QA
|
|
23
|
+
layer as it stood could not have produced it, because every test agreed with the software and
|
|
24
|
+
nothing in the process asked whether the software agreed with the intent.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 2. Two tiers, split by readership
|
|
29
|
+
|
|
30
|
+
The split is by **who reads it**, and that is what keeps it honest.
|
|
31
|
+
|
|
32
|
+
**Tier 1, business acceptance.** Per actor, in the language of the person who wanted the
|
|
33
|
+
software. Ring up a sale, lock the till, hand over a shift, read the end of day report. No
|
|
34
|
+
concurrency, no retry semantics, no HTTP status codes. This is the tier a customer or product
|
|
35
|
+
owner reads and confirms.
|
|
36
|
+
|
|
37
|
+
**Tier 2, engineering obligations.** Partial writes, idempotency under retry and message
|
|
38
|
+
redelivery, optimistic concurrency races, audit ordering, malformed input, boundaries. Real,
|
|
39
|
+
mandatory, and invisible to the business. See [`06-engineering-obligations.md`](06-engineering-obligations.md).
|
|
40
|
+
|
|
41
|
+
The mapping between them is **asymmetric**, and this is the part that must not be softened:
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
Tier 1 scenario -> 1..N UI journeys (near 1:1, user visible)
|
|
45
|
+
-> 1..N API tests (happy path / permission / invalid state)
|
|
46
|
+
|
|
47
|
+
Tier 2 obligation -> N API tests (NO tier 1 parent, and never will have one)
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
> No operator will ever write "given the queue redelivers the drain message after the row was
|
|
51
|
+
> already written".
|
|
52
|
+
|
|
53
|
+
Deriving tier 2 from tier 1 loses it entirely. Every gap that motivated this standard was tier
|
|
54
|
+
2. Putting tier 2 in front of the business is the opposite error: it buries the intent they
|
|
55
|
+
are being asked to approve.
|
|
56
|
+
|
|
57
|
+
**Edge cases a user could describe are not tier 2.** A wrong PIN, a lockout countdown, an
|
|
58
|
+
empty till, a plan tier that hides a button: all of those are ordinary use cases, each gets
|
|
59
|
+
its own scenario id, and each gets automated. An edge case only leaves tier 1 when no user
|
|
60
|
+
would ever phrase it.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 3. Both tiers are enumerated at planning time
|
|
65
|
+
|
|
66
|
+
Before implementation, and before anyone has an interest in the list being short.
|
|
67
|
+
|
|
68
|
+
That placement is the whole point. Enumerating obligations at test writing time means the
|
|
69
|
+
person listing them is the person about to declare the work done, at the moment they most want
|
|
70
|
+
the list to be short.
|
|
71
|
+
|
|
72
|
+
Tier 2 additionally gets a **completeness sweep at the gate**, because planning cannot
|
|
73
|
+
enumerate surfaces that only emerge while building. Anything the sweep finds that planning
|
|
74
|
+
missed is fed back into how planning is done, so the next item plans better. Neither is
|
|
75
|
+
sufficient alone: planning alone misses what emerged, and the sweep alone is the fox counting
|
|
76
|
+
the hens.
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## 4. The shared primitive: scenario ids
|
|
81
|
+
|
|
82
|
+
One vocabulary, cited by every stream, so coverage is computable across a feature rather than
|
|
83
|
+
per test suite.
|
|
84
|
+
|
|
85
|
+
- Scenarios live in `acceptance/<area>/<area>-acceptance.md` and carry stable ids as Gherkin
|
|
86
|
+
tags, for example `@LOCK-OPEN-001`.
|
|
87
|
+
- A UI journey **cites the id it proves**, and takes its step names from the scenario's own
|
|
88
|
+
words.
|
|
89
|
+
- An API test cites the scenario id for tier 1 behaviour, or the obligation it instantiates
|
|
90
|
+
for tier 2.
|
|
91
|
+
|
|
92
|
+
**Gherkin is the format** because it is the only candidate that is genuinely readable as prose
|
|
93
|
+
and structurally parseable at the same time: actor, precondition, action and outcome are
|
|
94
|
+
carried by the form itself, and a `Then` clause is an assertion in prose that maps close to
|
|
95
|
+
1:1 onto a test assertion.
|
|
96
|
+
|
|
97
|
+
That mapping is what makes review affordable. The reviewable artefact is a side by side of the
|
|
98
|
+
scenario's own words against what the test actually asserts, so a mismatch is visible in
|
|
99
|
+
seconds without opening the test file:
|
|
100
|
+
|
|
101
|
+
```
|
|
102
|
+
@TILL-PAY-004 A sale cannot be completed for less than the amount due
|
|
103
|
+
Then the sale is refused
|
|
104
|
+
and the drawer does not open
|
|
105
|
+
and the cart is left untouched
|
|
106
|
+
------------------------------------------------------------
|
|
107
|
+
Test rejects_short_payment_and_leaves_cart_intact
|
|
108
|
+
Asserts result.success === false
|
|
109
|
+
drawerOpened === false
|
|
110
|
+
cart.lines.length === 2
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
**The scenario's words reach the test by generation, never by retyping.** The clauses are
|
|
114
|
+
parsed out of the acceptance documents into a typed module the journeys reference, so a
|
|
115
|
+
scenario and the journey citing it cannot hold two different versions of the same sentence,
|
|
116
|
+
and a journey cannot silently prove three of five clauses. See section 1 of
|
|
117
|
+
[`04-playwright-journeys.md`](04-playwright-journeys.md). A convention would not survive this,
|
|
118
|
+
because keeping two copies of a sentence aligned by care is exactly the discipline that
|
|
119
|
+
already failed elsewhere.
|
|
120
|
+
|
|
121
|
+
**Known limit, and it must be stated wherever this is described.** That side by side catches
|
|
122
|
+
"the test does not do what it says". It does not catch "the scenario describes a situation
|
|
123
|
+
that cannot occur", and it does not catch a clause whose body asserts nothing. Only
|
|
124
|
+
demonstration against running software closes the first gap, which is why video is part of the
|
|
125
|
+
evidence and not decoration.
|
|
126
|
+
|
|
127
|
+
### Traceability runs both ways
|
|
128
|
+
|
|
129
|
+
Or it decays into an instruction to be thorough, and gets skipped when the change is small and
|
|
130
|
+
the day is short:
|
|
131
|
+
|
|
132
|
+
- change a scenario, and you get the list of journeys and API tests citing it
|
|
133
|
+
- change an API contract, and you get the scenarios that depend on it
|
|
134
|
+
- delete or rename a scenario, and you get the citations that are now orphaned
|
|
135
|
+
|
|
136
|
+
The output is a finite list to work through, not a judgement call.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## 5. Confirmation status: derived, verified, accepted
|
|
141
|
+
|
|
142
|
+
Every scenario carries one. `derived` is the honest default for anything written up from
|
|
143
|
+
software that already exists.
|
|
144
|
+
|
|
145
|
+
| Status | Meaning |
|
|
146
|
+
|---|---|
|
|
147
|
+
| `proposed` | Written at planning, before the software exists. A proposal, not a description. |
|
|
148
|
+
| `derived` | Written up from the delivered software. Describes what it appears to do. |
|
|
149
|
+
| `verified` | A human watched the software do it, recorded with date and commit. |
|
|
150
|
+
| `accepted` | Confirmed as what the software **should** do. |
|
|
151
|
+
|
|
152
|
+
A test citing a `derived` scenario proves the test runs, not that the requirement is right.
|
|
153
|
+
|
|
154
|
+
**The gap between `derived` and `accepted` is where defects hide, and making it visible is
|
|
155
|
+
most of the value.** A single number ("94% pass") flattens that distinction into nothing. The
|
|
156
|
+
status ladder keeps the unconfirmed surface on screen and hands the backfill a concrete
|
|
157
|
+
worklist.
|
|
158
|
+
|
|
159
|
+
It is also the anti flood mechanism for review: `accepted` items drop out of the review queue
|
|
160
|
+
entirely, so what stays visible is precisely the unsettled surface. That costs nothing extra,
|
|
161
|
+
because the statuses are already tracked.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## 6. Intent cannot be reverse engineered
|
|
166
|
+
|
|
167
|
+
The most important rule in the pack, and the one most easily skipped.
|
|
168
|
+
|
|
169
|
+
Code contains what a system **does**. It never contains why anyone wanted it. Anyone, human or
|
|
170
|
+
agent, asked to document intent while reading only the implementation will produce intent
|
|
171
|
+
fluently, plausibly, and fabricated.
|
|
172
|
+
|
|
173
|
+
So every scenario written up from existing software carries a provenance line naming where its
|
|
174
|
+
intent came from:
|
|
175
|
+
|
|
176
|
+
```
|
|
177
|
+
<!-- intent: design-notes/till-lock.md, section 4 -->
|
|
178
|
+
<!-- intent: stated by the product owner, 2026-03-11, recorded in decisions.md D-018 -->
|
|
179
|
+
<!-- intent: INFERRED from implementation - needs confirmation -->
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
That is not bookkeeping. It is what makes review affordable, because attention goes to the
|
|
183
|
+
`INFERRED` ones instead of spreading evenly across everything. Where no source exists, intent
|
|
184
|
+
is a **question**, not a statement.
|
|
185
|
+
|
|
186
|
+
This is what decides whether writing criteria against existing software is worth doing at all:
|
|
187
|
+
|
|
188
|
+
> Use cases built from observed behaviour yield tests asserting that the software does what
|
|
189
|
+
> the software does. They cannot fail. Use cases built from sourced intent yield tests that
|
|
190
|
+
> can disagree with the delivered software, and a disagreement is a defect found.
|
|
191
|
+
|
|
192
|
+
Two corollaries:
|
|
193
|
+
|
|
194
|
+
- **Demonstration is of the running software, not of the code.** Citing a file and line proves
|
|
195
|
+
a line was read. Running the flow proves the system does it.
|
|
196
|
+
- **When comparing findings against intent, order matters.** Observed behaviour first, then
|
|
197
|
+
the intent stated by whoever owns it, stated **before** they see any inferred version, then
|
|
198
|
+
the written up intent. The other order confirms a guess instead of testing it.
|
|
199
|
+
|
|
200
|
+
**The highest value output of this exercise is the anomaly list, not the document.** States
|
|
201
|
+
nothing can write, guards no caller reaches, buttons with no behaviour behind them, comments
|
|
202
|
+
contradicting code. Derived blindly, a status with four guards and no writer reads as "the
|
|
203
|
+
system supports this". Derived carefully, it is a defect report.
|
|
204
|
+
|
|
205
|
+
---
|
|
206
|
+
|
|
207
|
+
## 7. Evidence, not ticks
|
|
208
|
+
|
|
209
|
+
**No question is answered "yes".** A checklist has already failed here: asked "did you cover
|
|
210
|
+
the edge cases?", I would have answered yes in good faith and been wrong.
|
|
211
|
+
|
|
212
|
+
Questions are of three kinds, and each is answered differently:
|
|
213
|
+
|
|
214
|
+
- **Enumerable** questions are answered with a **list**. "Which actors can reach this?" is
|
|
215
|
+
answered by naming every one, **including those that must be refused**. Answering from
|
|
216
|
+
assumption rather than enumeration is how two tests once shipped asserting a permission
|
|
217
|
+
boundary that did not exist.
|
|
218
|
+
- **Provable** questions are answered with a **test name**, and that test must have been
|
|
219
|
+
observed failing. "Is this safe under concurrency" is not a judgement call.
|
|
220
|
+
- **Judgement** questions are answered with a **specific sentence**. "Is this efficient" is
|
|
221
|
+
settled by naming the N+1 that was checked and is not there, or the lock scope and why it is
|
|
222
|
+
that wide. A bare yes launders an unexamined assumption into a recorded answer.
|
|
223
|
+
|
|
224
|
+
**Falsifiability is recorded, not implied.** A test never observed red is not evidence. Where
|
|
225
|
+
a test was proved able to fail, the file records the specific mutation and the result:
|
|
226
|
+
|
|
227
|
+
```ts
|
|
228
|
+
// Proved red by removing the same-cashier guard in the unlock handler; this test
|
|
229
|
+
// failed on exactly the "shift is unchanged" assertion, not on setup.
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
Inventing a plausible mutation is a much higher bar to bluff than a tick, and it is the manual
|
|
233
|
+
stand in for mutation testing over the changed files.
|
|
234
|
+
|
|
235
|
+
---
|
|
236
|
+
|
|
237
|
+
## 8. Design rule: optimise verification cost, not authoring cost
|
|
238
|
+
|
|
239
|
+
**Every artefact must be checkable in less time than it took to produce.**
|
|
240
|
+
|
|
241
|
+
A record needing twenty minutes of scrutiny will not be scrutinised. The reviewer becomes the
|
|
242
|
+
bottleneck and waves it through, which is precisely how bad work passes. A video takes two
|
|
243
|
+
minutes. A citation check is instant, because it is mechanical. Batches stay small and ordered
|
|
244
|
+
riskiest first, because forty items presented at once get rubber stamped.
|
|
245
|
+
|
|
246
|
+
This is also the argument for narrating a recorded run with the scenario's own wording: what a
|
|
247
|
+
viewer is told and what the journey proves are then the same sentence, and no second script
|
|
248
|
+
exists that could drift from the criteria.
|
|
249
|
+
|
|
250
|
+
---
|
|
251
|
+
|
|
252
|
+
## 9. What mechanical checking can and cannot do
|
|
253
|
+
|
|
254
|
+
Generating the clauses into the journeys (section 1 of
|
|
255
|
+
[`04-playwright-journeys.md`](04-playwright-journeys.md)) moves several of these from "a check
|
|
256
|
+
that has to be written and run" to "the build does not compile": a cited id that does not
|
|
257
|
+
exist, an orphaned citation after a rename, a missing clause, and a step the document does not
|
|
258
|
+
contain.
|
|
259
|
+
|
|
260
|
+
What is left for a citation check, all mechanically decidable:
|
|
261
|
+
|
|
262
|
+
1. every scenario id is cited by at least one journey, or carries a recorded reason it cannot
|
|
263
|
+
be
|
|
264
|
+
2. every cited test exists in the sources
|
|
265
|
+
3. every cited test appears in the change under review, so citing pre existing unrelated tests
|
|
266
|
+
does not count
|
|
267
|
+
4. every scenario carries a status and, where the status is `verified`, a date and commit
|
|
268
|
+
5. no clause body is empty of assertions
|
|
269
|
+
|
|
270
|
+
**It never verifies that the answers are true.** Green means "well formed, and the citations
|
|
271
|
+
resolve to real tests in this change". It never means "the coverage is adequate". That
|
|
272
|
+
distinction has to be printed wherever the check is documented, or the check becomes exactly
|
|
273
|
+
the false assurance it was built to prevent.
|