criteria-review 0.3.3 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/README.md +73 -1
- package/agents/skills/claude-code/criteria-architect/SKILL.md +149 -0
- package/agents/skills/claude-code/criteria-backfill/SKILL.md +184 -0
- package/agents/skills/claude-code/criteria-bundle/SKILL.md +137 -0
- package/agents/skills/claude-code/criteria-develop/SKILL.md +129 -0
- package/agents/skills/claude-code/criteria-glossary/SKILL.md +147 -0
- package/agents/skills/claude-code/criteria-help/SKILL.md +79 -0
- package/agents/skills/claude-code/criteria-plan/SKILL.md +118 -0
- package/agents/skills/claude-code/criteria-test/SKILL.md +124 -0
- package/bin/criteria-review.js +297 -3
- package/docs/decisions.md +69 -0
- package/docs/standard/10-glossary.md +134 -0
- package/docs/standard/README.md +2 -1
- package/package.json +1 -1
- package/public/app.js +14 -4
- package/public/index.html +1 -0
- package/src/config.js +5 -1
- package/src/emit.js +24 -0
- package/src/plan.js +142 -0
- package/src/server.js +39 -0
- package/src/terms.js +141 -0
- package/src/version.js +1 -1
package/agents/README.md
CHANGED
|
@@ -95,6 +95,19 @@ a network download inside the check and lets a missing dependency read as a pass
|
|
|
95
95
|
`node_modules/.bin/criteria-review`, falls back to `PATH`, and fails when neither exists. See
|
|
96
96
|
`docs/standard/emit-contract.md`.
|
|
97
97
|
|
|
98
|
+
### 5. A plan holds ids; the documents hold everything else
|
|
99
|
+
|
|
100
|
+
`criteria-review plan` records which scenarios a task covers and nothing more - no titles, no
|
|
101
|
+
statuses, no todo list of its own. Every other fact is read live, so nothing here can go stale
|
|
102
|
+
and disagree with a document.
|
|
103
|
+
|
|
104
|
+
There is therefore **no "mark it done"**. A scenario is done when its status moved and a
|
|
105
|
+
journey cites it, which is a fact rather than a tick. `plan check` answers from the documents
|
|
106
|
+
and exits non-zero while anything declared is still outstanding, which makes it usable as a
|
|
107
|
+
delivery gate.
|
|
108
|
+
|
|
109
|
+
`--plan` is a scope beside `--since`: what I chose, versus what the branch touched.
|
|
110
|
+
|
|
98
111
|
### And one standing rule
|
|
99
112
|
|
|
100
113
|
**A session that edits an acceptance document regenerates before running tests.** Otherwise the
|
|
@@ -121,9 +134,68 @@ not.
|
|
|
121
134
|
|
|
122
135
|
## What is here
|
|
123
136
|
|
|
137
|
+
**Work runs in two directions, and the skills split on that line.** Confusing them is the
|
|
138
|
+
mistake worth avoiding: one starts from what somebody wanted, the other from what the software
|
|
139
|
+
already does.
|
|
140
|
+
|
|
141
|
+
```
|
|
142
|
+
FORWARD - there is intent, the software does not exist yet
|
|
143
|
+
|
|
144
|
+
architect -> intake, tier 1 spec + tier 2 shortlist, brief, declaration
|
|
145
|
+
|
|
|
146
|
+
plan -> the scenarios a task covers
|
|
147
|
+
|
|
|
148
|
+
+-------------+-------------+
|
|
149
|
+
| |
|
|
150
|
+
test develop
|
|
151
|
+
"write the test" "write the code"
|
|
152
|
+
| |
|
|
153
|
+
+-------------+-------------+
|
|
154
|
+
|
|
|
155
|
+
lookup -> a person confirms, corrects, rejects
|
|
156
|
+
|
|
|
157
|
+
bundle -> the evidence
|
|
158
|
+
|
|
159
|
+
|
|
160
|
+
REVERSE - the software exists, nothing was written down
|
|
161
|
+
|
|
162
|
+
backfill -> criteria + characterisation journeys, from OBSERVATION
|
|
163
|
+
intent inferred, nothing confirmed
|
|
164
|
+
|
|
|
165
|
+
lookup -> a person verifies it, and the disagreements are the findings
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
In the forward direction a test that disagrees with its criterion is a defect in the software.
|
|
169
|
+
In the reverse direction the criterion came from the behaviour, so a disagreement means it was
|
|
170
|
+
observed wrongly. Same artefacts, opposite meaning.
|
|
171
|
+
|
|
124
172
|
| Skill | For |
|
|
125
173
|
|---|---|
|
|
126
|
-
| [`
|
|
174
|
+
| [`criteria-help`](skills/claude-code/criteria-help/SKILL.md) | **Start here.** Asks what you are doing and routes to the right skill, by situation rather than by name |
|
|
175
|
+
| [`criteria-architect`](skills/claude-code/criteria-architect/SKILL.md) | **Forward.** Vet the request, design the tier 1 spec and tier 2 shortlist, write the brief, save the declaration where the team takes work from |
|
|
176
|
+
| [`criteria-backfill`](skills/claude-code/criteria-backfill/SKILL.md) | **Reverse.** Reverse engineer a legacy area into criteria and characterisation journeys from observation, intent inferred and unconfirmed |
|
|
177
|
+
| [`criteria-plan`](skills/claude-code/criteria-plan/SKILL.md) | Turning a task into a working list of scenario ids: ask, verify, prepare, then act |
|
|
178
|
+
| [`criteria-lookup`](skills/claude-code/criteria-lookup/SKILL.md) | Looking a scenario up and answering it in conversation, and walking the queue one at a time |
|
|
179
|
+
| [`criteria-test`](skills/claude-code/criteria-test/SKILL.md) | Building the journey for a criterion, learning the behaviour by driving the running app |
|
|
180
|
+
| [`criteria-develop`](skills/claude-code/criteria-develop/SKILL.md) | Implementing code for criteria the build does not satisfy, red to green, without editing the criterion |
|
|
181
|
+
| [`criteria-glossary`](skills/claude-code/criteria-glossary/SKILL.md) | Finding where the vocabulary really lives and writing the script that emits the terms manifest, so a rename never stales a document |
|
|
182
|
+
| [`criteria-bundle`](skills/claude-code/criteria-bundle/SKILL.md) | Recording narrated videos for a plan or a branch, and packaging them with their documents |
|
|
183
|
+
|
|
184
|
+
### What every one of them refuses
|
|
185
|
+
|
|
186
|
+
Worth reading even if you port only one, because these are the rules that make the rest worth
|
|
187
|
+
having:
|
|
188
|
+
|
|
189
|
+
- **Never edit a criterion to make a test pass.** Easy, invisible, and the failure the whole
|
|
190
|
+
system exists to prevent. When the criterion is wrong, raise it and stop.
|
|
191
|
+
- **Never promote a status.** `verified` claims a person watched it; `accepted` is a claim about
|
|
192
|
+
intent. Neither is an agent's to make, and least of all about its own work.
|
|
193
|
+
- **Never invent intent.** Cite a source or write `INFERRED`. Code contains what a system does,
|
|
194
|
+
never why anyone wanted it.
|
|
195
|
+
- **Know which direction you are in.** A characterisation journey from `backfill` pins what the
|
|
196
|
+
software does and proves nothing about whether it is right; a journey from `criteria-test`
|
|
197
|
+
proves a criterion somebody chose. Reporting the first as if it were the second is how a green
|
|
198
|
+
suite comes to defend behaviour nobody ever wanted.
|
|
127
199
|
|
|
128
200
|
## What deliberately is not here
|
|
129
201
|
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: criteria-architect
|
|
3
|
+
description: Drive the forward direction: take a source of intent and design the acceptance spec and the technical brief a team can build from. Vets the request at intake, designs tier 1 scenarios and the tier 2 obligation shortlist, writes the brief, then asks where the declaration should be saved - a DevOps task, a GitHub issue, a document - so the scenario ids it names match what is committed. Use when the architect says "design this", "solution this", "new feature", "here is the intent", "prepare this task", or hands over a request that is not yet real work.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Design a feature from intent
|
|
7
|
+
|
|
8
|
+
The **forward direction**: intent, then criteria, then tests, then code. (The reverse - a legacy
|
|
9
|
+
application with nothing written down - is `criteria-backfill`, which works from observation and
|
|
10
|
+
infers intent. Do not mix them: one starts from what somebody wanted, the other from what the
|
|
11
|
+
software does.)
|
|
12
|
+
|
|
13
|
+
This skill drives the process the standard already defines. Read
|
|
14
|
+
`npx criteria-review guide` and the roles document before working; what follows is how to run
|
|
15
|
+
it, not a restatement of it.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 1. Intake: is this real work yet?
|
|
20
|
+
|
|
21
|
+
A request arriving from outside is **a want, not a specification**, and admitting it as though it
|
|
22
|
+
were one is how unsourced intent enters the system.
|
|
23
|
+
|
|
24
|
+
Take the source they give you - a conversation, a ticket, a document, a paragraph - and check it
|
|
25
|
+
carries three things:
|
|
26
|
+
|
|
27
|
+
- **A named actor and a real "so that".** Not "add a filter": who is blocked without it, and
|
|
28
|
+
what can they not currently do. A circular "so that" is a heading, not a story.
|
|
29
|
+
- **Sourced intent, or an explicit question in its place.** Every claim about what was wanted
|
|
30
|
+
cites where it came from - a dated conversation with a named owner, a ticket, a recorded
|
|
31
|
+
commitment. Where nothing exists, carry **the question**, not a confident guess. Never reverse
|
|
32
|
+
engineer intent from what a competitor does or from what seems reasonable.
|
|
33
|
+
- **The scope line.** One sentence naming what is in and, more usefully, what is deliberately
|
|
34
|
+
out. Most disputes at delivery are about the out.
|
|
35
|
+
|
|
36
|
+
The output is not "approved". It is one of three, and say which:
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
ADMITTED intent sourced, actor named, scope line written
|
|
40
|
+
RETURNED to the requester - naming the specific missing piece
|
|
41
|
+
HELD as a question against <who owns the answer>
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Do not design a returned or held request.** Designing around a missing intent is how a guess
|
|
45
|
+
becomes a requirement with an architect's authority on it.
|
|
46
|
+
|
|
47
|
+
## 2. Design the acceptance spec
|
|
48
|
+
|
|
49
|
+
The tier 1 document plus the tier 2 shortlist, enumerated **before** anyone has an interest in
|
|
50
|
+
the list being short.
|
|
51
|
+
|
|
52
|
+
**Tier 1** - what the requester would recognise, in their language, one actor per scenario. Work
|
|
53
|
+
outward from the happy path: the refusals, the permission boundaries, the states where the
|
|
54
|
+
action makes no sense, the limits. Each is an ordinary scenario with its own id, not a bullet at
|
|
55
|
+
the bottom.
|
|
56
|
+
|
|
57
|
+
Status is **`proposed`**: written at planning, before the software exists. That is exactly what
|
|
58
|
+
that status means, and it distinguishes these from anything derived off an implementation.
|
|
59
|
+
|
|
60
|
+
**Tier 2** - the obligations this feature instantiates: partial writes, idempotency under retry,
|
|
61
|
+
races, audit ordering, malformed input, boundaries, and authorisation enumerated including who
|
|
62
|
+
must be refused. Taken from the catalogue rather than from imagination, because a catalogue can
|
|
63
|
+
be checked against and diligence cannot.
|
|
64
|
+
|
|
65
|
+
They attach to **the unit of work, never to a scenario or a journey.** An obligation is by
|
|
66
|
+
definition something no user would describe, so it has no tier 1 parent and never will.
|
|
67
|
+
|
|
68
|
+
Write them to a sibling file in the area folder, `acceptance/<area>/<area>-obligations.md`, kept
|
|
69
|
+
separate from the acceptance document for one practical reason: tier 1 is shared with the
|
|
70
|
+
requester and tier 2 never is, so "send them that file" is a safer operation than "send that file
|
|
71
|
+
with the second half removed". Putting tier 2 in front of the requester buries the business
|
|
72
|
+
intent they are being asked to approve.
|
|
73
|
+
|
|
74
|
+
Present both for correction before writing anything. For tier 1, present the ids you propose and
|
|
75
|
+
let them be renamed - ids are stable forever and never reused, so a bad prefix is expensive
|
|
76
|
+
later.
|
|
77
|
+
|
|
78
|
+
## 3. Write the technical brief
|
|
79
|
+
|
|
80
|
+
Short on purpose. **If it is long, the architect is doing the team's job.**
|
|
81
|
+
|
|
82
|
+
| Section | What goes in it |
|
|
83
|
+
|---|---|
|
|
84
|
+
| Intent, one paragraph | The agreed "so that", so the brief reads without the acceptance document open |
|
|
85
|
+
| Obligations shortlist | The tier 2 ids, each a concrete list of tests rather than a feeling |
|
|
86
|
+
| Contracts | API shape, events and messages, the data model - the boundary the team builds behind |
|
|
87
|
+
| Decisions taken | Cross-cutting calls already made, each with a one-line why, so they are not relitigated |
|
|
88
|
+
| Decisions left open | The local calls the team owns, named as theirs, so delegation is explicit |
|
|
89
|
+
| Risks retired | Spikes done and what they proved; unknowns that would otherwise surface mid-sprint |
|
|
90
|
+
| Out of scope | What this deliberately does not do, so a returned implementation is not marked short |
|
|
91
|
+
|
|
92
|
+
**The hardest judgement in the role, and the one to get right here: specify the outcome
|
|
93
|
+
completely, and do not specify the mechanism.** A gap in the outcome will not be correctly
|
|
94
|
+
filled by someone who was not in the conversation. But the moment the brief pre-writes the code,
|
|
95
|
+
designing costs what building costs and the reason for separating them is gone.
|
|
96
|
+
|
|
97
|
+
The bar for the whole handoff is blunt: **a faithful, literal reading of the spec and the brief
|
|
98
|
+
must produce the right software**, with no discrimination required on the far side. Anything
|
|
99
|
+
ambiguous may not be queried - it may simply be guessed, built, and reported done.
|
|
100
|
+
|
|
101
|
+
## 4. Ask where the declaration is saved
|
|
102
|
+
|
|
103
|
+
Two destinations, and they are different things:
|
|
104
|
+
|
|
105
|
+
- **The criteria go in the repository**, committed, where the review tool reads them:
|
|
106
|
+
`acceptance/<area>/<area>-acceptance.md`.
|
|
107
|
+
- **The declaration goes in the work item** - the DevOps task, the GitHub issue, wherever the
|
|
108
|
+
team takes work from. It names the brief and **the scenario ids**, and those ids must match
|
|
109
|
+
what was committed exactly, because that is what `criteria-plan` reads back when someone picks
|
|
110
|
+
the task up.
|
|
111
|
+
|
|
112
|
+
So ask:
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
Where should the task declaration go?
|
|
116
|
+
1 a DevOps work item - id, or create a new one
|
|
117
|
+
2 a GitHub issue - number, or create a new one
|
|
118
|
+
3 a document in the repo
|
|
119
|
+
4 print it - I will paste it myself
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Then write it, and **verify the ids round-trip**: the ids in the work item must resolve against
|
|
123
|
+
the committed documents.
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
npx criteria-review plan set <the ids> --task "<name>" --source "<the work item>"
|
|
127
|
+
npx criteria-review plan # every id resolves, nothing unresolved
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
If any id does not resolve, the declaration and the repository have already drifted, on day one.
|
|
131
|
+
Fix it now rather than letting a developer find it.
|
|
132
|
+
|
|
133
|
+
## 5. Hand off, and say what is owed
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
ADMITTED. 11 tier 1 scenarios (proposed), 5 tier 2 obligations.
|
|
137
|
+
criteria -> acceptance/till-lock/till-lock-acceptance.md committed
|
|
138
|
+
brief -> issue #412
|
|
139
|
+
ids verified against the committed documents
|
|
140
|
+
Owed before build: the requester confirms the tier 1 spec (minus tier 2).
|
|
141
|
+
Next: `criteria-test` for the journeys, `criteria-develop` for the code.
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
**Share tier 1 with the requester and get it confirmed** before implementation. A document
|
|
145
|
+
carries force in a disagreement only if it was agreed - that is what turns "what the developer
|
|
146
|
+
assumed" into "what we agreed". Tier 2 stays inside the team.
|
|
147
|
+
|
|
148
|
+
**Never promote a status.** These are `proposed` until a person moves them, and the person is
|
|
149
|
+
the one who owns the intent, not the one who wrote it down.
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: criteria-backfill
|
|
3
|
+
description: Reverse engineer a legacy application into specs and tests that QA and the solution architect can verify. Asks which area to investigate, learns the behaviour by reading and running it, then recommends a filename, ids, roles, user stories and a numbered scenario list for correction before anything is written. Builds criteria and characterisation journeys from OBSERVATION; intent is inferred and marked as such until a human confirms it. Use when the architect says "backfill", "reverse engineer X", "we have no scenarios for this area", "document what this does", or a plan turns up work with no criteria behind it.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Reverse engineer an existing application
|
|
7
|
+
|
|
8
|
+
This is the **reverse direction** of the two the process has. Forward is intent to criteria to
|
|
9
|
+
tests to code, and the solution architect drives it. Reverse is this: a legacy application
|
|
10
|
+
exists, nobody wrote down what it promises, and the job is to produce specs and tests from
|
|
11
|
+
**observation** to the point where QA and the architect can verify them.
|
|
12
|
+
|
|
13
|
+
The output is a document of scenarios, and journeys that pin the behaviour, which a person can
|
|
14
|
+
confirm, correct or reject.
|
|
15
|
+
|
|
16
|
+
Four steps. **Nothing is written until step 3 has been corrected.**
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## 0. What is being produced, and what it is worth
|
|
21
|
+
|
|
22
|
+
> Code contains what a system **does**. It never contains why anyone wanted it.
|
|
23
|
+
|
|
24
|
+
So the behaviour here is observed and the **intent is inferred**, every time, and it is marked
|
|
25
|
+
`INFERRED` rather than asserted. An agent asked to document intent while reading only the
|
|
26
|
+
implementation will produce it fluently, plausibly and fabricated; the marking is what stops
|
|
27
|
+
that from becoming a requirement nobody chose.
|
|
28
|
+
|
|
29
|
+
**Yes, this produces tests as well as criteria** - characterisation journeys that pin what the
|
|
30
|
+
software does today. Be clear about what they are worth, because it is not the same as a forward
|
|
31
|
+
test:
|
|
32
|
+
|
|
33
|
+
- They **cannot disagree with the software.** They were written from it. A green one proves
|
|
34
|
+
nothing about whether the behaviour is right.
|
|
35
|
+
- What they buy is that the behaviour is now **pinned**: a change to it becomes visible instead
|
|
36
|
+
of silent, on a codebase where nothing was watching.
|
|
37
|
+
- And they make the criteria **verifiable**: QA and the architect get something concrete to
|
|
38
|
+
watch running, rather than a paragraph to agree with in the abstract.
|
|
39
|
+
|
|
40
|
+
The disagreement comes later, and from a person - reading a derived criterion and saying *that is
|
|
41
|
+
not what it should do*. That is the whole point of the reverse direction, and it is why the one
|
|
42
|
+
rule that must not bend is the next one.
|
|
43
|
+
|
|
44
|
+
**Nothing produced here is confirmed.** Every scenario is `derived`, every intent is `INFERRED`
|
|
45
|
+
unless genuinely cited, and nothing is ever `accepted` or `verified`. A characterisation test
|
|
46
|
+
whose criterion has been mistaken for a requirement is worse than no test, because it now defends
|
|
47
|
+
the current behaviour against anyone trying to change it.
|
|
48
|
+
|
|
49
|
+
## 1. Ask what to investigate
|
|
50
|
+
|
|
51
|
+
Never pick the area yourself. Ask, and offer help choosing:
|
|
52
|
+
|
|
53
|
+
```
|
|
54
|
+
Which area should I work through?
|
|
55
|
+
1 name it - "the till", "shift hand-over", "refunds"
|
|
56
|
+
2 show me candidates - I will list areas with code and no criteria
|
|
57
|
+
3 from a plan - items that resolved to nothing when the task was planned
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
An **area is a thing a user would name.** "Locking the till" is an area. "SessionController" is
|
|
61
|
+
not. If the answer is a module rather than a behaviour, say so and ask again.
|
|
62
|
+
|
|
63
|
+
Then ask what may be read as intent: a design note, a decision log, an issue, a specification,
|
|
64
|
+
a conversation they remember. **Ask explicitly, because this is the only chance to get real
|
|
65
|
+
intent**, and once the document is written the inferred version starts looking like a fact.
|
|
66
|
+
|
|
67
|
+
## 2. Investigate, in this order
|
|
68
|
+
|
|
69
|
+
**Documentation first, code second.** Reversing it means the code frames what you then go
|
|
70
|
+
looking for, and every document you read afterwards is read as confirmation.
|
|
71
|
+
|
|
72
|
+
1. **Read the sources they named**, and any decision record or phase document covering the
|
|
73
|
+
area. Note exact citations - `file.md:214`, an issue number, a dated decision. A citation is
|
|
74
|
+
a path, a line, an id or a URL. Prose that merely discusses intent is **not** a citation.
|
|
75
|
+
2. **Read the implementation** to learn what the software does: the actor's entry point, the
|
|
76
|
+
guards, the states, what is refused and what is silently allowed.
|
|
77
|
+
3. **Run it if you can.** Citing `file:line` proves a line was read; running the flow proves the
|
|
78
|
+
system does it. Anything you only read is a weaker claim, and the difference belongs in the
|
|
79
|
+
provenance line.
|
|
80
|
+
4. **Collect the anomalies as you go.** States nothing can write, guards no caller reaches,
|
|
81
|
+
buttons with no behaviour behind them, comments contradicting the code. **This is the highest
|
|
82
|
+
value output of the whole exercise**, higher than the document - derived blindly, a status
|
|
83
|
+
with four guards and no writer reads as "the system supports this"; derived carefully it is
|
|
84
|
+
a defect report.
|
|
85
|
+
|
|
86
|
+
## 3. Recommend, and wait for corrections
|
|
87
|
+
|
|
88
|
+
Present all of it before writing anything:
|
|
89
|
+
|
|
90
|
+
**Where it goes**
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
acceptance/till-lock/till-lock-acceptance.md new
|
|
94
|
+
acceptance/shift-lifecycle/shift-lifecycle-acceptance.md exists - 6 scenarios would be added
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
One folder per area, named for the area rather than the module. Say plainly when an area
|
|
98
|
+
already has a document, because adding to one is different from starting one.
|
|
99
|
+
|
|
100
|
+
**The id scheme**, checked against every registered project for collisions:
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
LOCK-SET-001..004 locking
|
|
104
|
+
LOCK-OPEN-001..004 reopening
|
|
105
|
+
LOCK-FAIL-001..003 lockout
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Ids are stable forever and never reused, so a bad prefix is expensive later. Offer the grouping
|
|
109
|
+
and let them rename it.
|
|
110
|
+
|
|
111
|
+
**The roles**, including any that must be refused.
|
|
112
|
+
|
|
113
|
+
**The user stories**, in prose, with the "so that" doing real work - "as a user I want a lock
|
|
114
|
+
button so that I can lock the screen" is circular and says nothing a test could disagree with.
|
|
115
|
+
|
|
116
|
+
**The scenarios**, numbered, each one line, each marked with where its intent came from:
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
1 LOCK-SET-001 sourced design-notes/till-lock.md:44 Locking de-authorises the terminal
|
|
120
|
+
2 LOCK-SET-002 sourced design-notes/till-lock.md:51 The shift and drawer stay open
|
|
121
|
+
3 LOCK-OPEN-002 INFERRED - A different cashier is recorded
|
|
122
|
+
...
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Then ask for corrections, and ask specifically about the `INFERRED` ones: **each is a question,
|
|
126
|
+
not a statement.** "I could not find why this behaves this way - is it deliberate, and is this
|
|
127
|
+
what it should do?" One sentence from them turns an unfalsifiable criterion into one that can
|
|
128
|
+
catch a defect.
|
|
129
|
+
|
|
130
|
+
Also present the anomalies, separately and plainly. They are not scenarios and should not be
|
|
131
|
+
smuggled in as if they were.
|
|
132
|
+
|
|
133
|
+
## 4. Write the criteria, then the journeys that pin them
|
|
134
|
+
|
|
135
|
+
Only after correction. Follow `criteria-review guide` and the standard's authoring rules; the
|
|
136
|
+
ones most often broken here:
|
|
137
|
+
|
|
138
|
+
- **One actor, in their language.** No route paths, no component names, no HTTP status codes.
|
|
139
|
+
- **`Then` clauses are assertions**, close to 1:1 with what a test would assert.
|
|
140
|
+
- **Negative outcomes are stated**, not implied. "And the drawer does not open" is a clause.
|
|
141
|
+
- **Edge cases get their own scenario and their own id.** A wrong PIN, a lockout countdown, an
|
|
142
|
+
expired voucher - each is an ordinary use case. A bullet list at the bottom headed "edge
|
|
143
|
+
cases" is a list of things nobody automated.
|
|
144
|
+
- **Tier 1 only.** Partial writes, retries, races, malformed input and boundaries are real and
|
|
145
|
+
belong to the engineering obligations, not here. No operator will ever describe them, and
|
|
146
|
+
putting them here buries the business intent this layer exists to make reviewable.
|
|
147
|
+
- **Every scenario is `derived`**, or `proposed` if the software does not exist yet. Never
|
|
148
|
+
`verified`, never `accepted` - those are claims only a person can make.
|
|
149
|
+
- **Every block carries a provenance comment**, a citation or `INFERRED`.
|
|
150
|
+
|
|
151
|
+
Where you believe the behaviour is wrong, **write the scenario as the wanted behaviour and say
|
|
152
|
+
so in the comment**, and do not write a journey pinning the behaviour you think is a defect.
|
|
153
|
+
|
|
154
|
+
Then the journeys, following `criteria-test`'s rules for binding, selectors and seeding - with
|
|
155
|
+
one difference that matters. There, a journey that disagrees with its criterion is a finding to
|
|
156
|
+
raise. Here the criterion came from the behaviour, so a disagreement means you observed it
|
|
157
|
+
wrongly: go back and look again rather than adjusting either artefact to match the other.
|
|
158
|
+
|
|
159
|
+
**A scenario you cannot exercise is a finding, not a gap.** A path no caller can reach, a state
|
|
160
|
+
nothing writes, an affordance with nothing behind it - record it in the anomaly list and leave
|
|
161
|
+
the scenario without a journey. Writing a journey that fakes its way into an unreachable state
|
|
162
|
+
would produce a green test for something that cannot happen.
|
|
163
|
+
|
|
164
|
+
## 5. Hand it on
|
|
165
|
+
|
|
166
|
+
Say what was produced and what is owed:
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
14 scenarios written to acceptance/till-lock/till-lock-acceptance.md
|
|
170
|
+
2 sourced, 12 INFERRED - the 12 are questions, not statements
|
|
171
|
+
11 characterisation journeys written and green - they PIN today's behaviour and
|
|
172
|
+
prove nothing about whether it is right
|
|
173
|
+
3 scenarios could not be exercised (listed) - and that is a finding, not a gap
|
|
174
|
+
4 anomalies found (listed above), none of them scenarios
|
|
175
|
+
Next: `criteria-lookup` to walk the inferred ones with the recordings beside them
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Say plainly that the green journeys are not evidence of correctness. A reader who takes "11
|
|
179
|
+
green" as reassurance has drawn the opposite conclusion from the truth: they show the software
|
|
180
|
+
does what it does, on a codebase where nobody had written down what it should do.
|
|
181
|
+
|
|
182
|
+
Never promote a status, never mark anything accepted, and do not fix the anomalies here. This
|
|
183
|
+
skill produces a document, a set of pinning tests and a list of questions. Answering them is the
|
|
184
|
+
architect's, and acting on them is another skill's.
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: criteria-bundle
|
|
3
|
+
description: Record narrated videos for the scenarios in the current plan or added on this branch, and assemble them with their acceptance documents into a delivery package. Runs the journeys at human-viewable pacing at 1080p, names each clip for the scenario it proves, and reports every scenario that has no recording rather than shipping a package that looks complete. Use when the architect says "record these", "bundle the videos", "make the handoff package", or a task is finished and needs its evidence.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Record and bundle the evidence
|
|
7
|
+
|
|
8
|
+
A test count is a claim about the software. A video is the software. This produces the second
|
|
9
|
+
kind, for the scenarios a task actually covered.
|
|
10
|
+
|
|
11
|
+
The full procedure, its failure modes and the reference plumbing are in the standard's
|
|
12
|
+
**Recording a journey** (`npx criteria-review guide` lists it, and the reading tab renders it).
|
|
13
|
+
This skill is how to run it and what to refuse.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## The rule that governs the whole thing
|
|
18
|
+
|
|
19
|
+
**Tests run at full speed. Videos published for people run at a speed a person can follow.**
|
|
20
|
+
|
|
21
|
+
Same journeys, same assertions, same code path, run twice, with an environment seam nobody sets
|
|
22
|
+
by default as the only difference. Two ways to get it wrong, and they are opposite:
|
|
23
|
+
|
|
24
|
+
- Pacing the gating suite for watchability gets the affordance deleted by whoever is waiting on
|
|
25
|
+
the build, and the demonstration goes with it.
|
|
26
|
+
- Publishing a clip at test speed produces footage of a cursor flickering around an unfamiliar
|
|
27
|
+
interface: cheaper to make and worth nothing, because the viewer cannot tell what happened.
|
|
28
|
+
|
|
29
|
+
If a set of clips runs in under a minute, it was filmed at test speed. If the everyday suite got
|
|
30
|
+
slower this week, something demonstration-only became unconditional.
|
|
31
|
+
|
|
32
|
+
## 1. Decide the scope, and say what it is
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
npx criteria-review queue --plan # what this task covers
|
|
36
|
+
npx criteria-review queue --since main # what this branch added
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Ask which, rather than assuming. A package labelled as a task's evidence that quietly contains
|
|
40
|
+
the whole suite is misleading in the direction that matters.
|
|
41
|
+
|
|
42
|
+
Then name the ones with no journey. **They cannot be recorded, and that is the finding** - a
|
|
43
|
+
scenario with no journey is not evidence missing from the package, it is coverage missing from
|
|
44
|
+
the work. Report it before recording rather than after.
|
|
45
|
+
|
|
46
|
+
## 2. Record
|
|
47
|
+
|
|
48
|
+
From a clean database, single worker, only the journeys in scope:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
E2E_VIDEO=1 E2E_SLOWMO=900 E2E_CARD_MS=4500 E2E_STEP_PAUSE=2000 \
|
|
52
|
+
npx playwright test --workers=1 <the journeys in scope>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Environment variable names are the project's own - read its watch-mode module rather than
|
|
56
|
+
assuming these. What must be true, whatever they are called: **1080p at a 1:1 capture size**,
|
|
57
|
+
per-test timeout raised for the recording run only, and every demonstration affordance inert
|
|
58
|
+
when the seam is unset.
|
|
59
|
+
|
|
60
|
+
**The recording run is a real pass, not a filming session.** Same assertions. If it goes red,
|
|
61
|
+
you have found something, and filming stops until it is understood - a green suite and a red
|
|
62
|
+
recording of the same journeys means the wider viewport broke something.
|
|
63
|
+
|
|
64
|
+
## 3. Copy the files out immediately
|
|
65
|
+
|
|
66
|
+
The runner wipes its output directory at the start of every run, so the next headless run
|
|
67
|
+
destroys them. This is the single most common way a recording session is lost, and it is
|
|
68
|
+
unrecoverable except by recording again.
|
|
69
|
+
|
|
70
|
+
Then rename each to lead with its scenario id, which is what maps a clip to a criterion.
|
|
71
|
+
|
|
72
|
+
## 4. Check each clip before it goes anywhere
|
|
73
|
+
|
|
74
|
+
Open them. A recording session that produced files is not a recording session that produced
|
|
75
|
+
evidence, and every one of these failures leaves the run green:
|
|
76
|
+
|
|
77
|
+
- **Blank, and unusually small** - the journey opened its own browser context, which inherited
|
|
78
|
+
no recording options. A thirty-second clip of about 55KB is this.
|
|
79
|
+
- **Soft, roughly 800x450** - the runner scaled the viewport into its default 800x800 box.
|
|
80
|
+
- **Two indistinguishable files from one test** - two contexts, both named by internal id.
|
|
81
|
+
- **No title card**, or captions that do not match the document's wording.
|
|
82
|
+
|
|
83
|
+
## 5. Assemble
|
|
84
|
+
|
|
85
|
+
Documents first, clips in the order the acceptance document introduces their scenarios:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
<target>/<area>-<recording date>/
|
|
89
|
+
00-<Area>-Acceptance.pdf
|
|
90
|
+
01-<SCENARIO-ID>-<slug>.webm
|
|
91
|
+
...
|
|
92
|
+
README.md
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
- **The numbering comes from the document**, not from a list you keep, so adding a scenario and
|
|
96
|
+
re-running is enough.
|
|
97
|
+
- **A gap in the numbers is meaningful** - a scenario written down with no recording. The README
|
|
98
|
+
names those, and names any clip that matched no scenario rather than dropping it.
|
|
99
|
+
- **Replace the target folder, do not merge.** A stale clip from a retired scenario is worse
|
|
100
|
+
than no folder, because nothing about it says it is out of date.
|
|
101
|
+
- **Label it with the date of the recording, not today's.** The package is evidence from a
|
|
102
|
+
moment.
|
|
103
|
+
|
|
104
|
+
**Write the manifest**, which is what ties the package to the work rather than leaving it a
|
|
105
|
+
folder of clips:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
npx criteria-review manifest --out <target>/manifest.json
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
It records the task and its source - the issue, the work item, the URL the plan was built from -
|
|
112
|
+
alongside every scenario, its status, and the clip that proves it. **Scenarios with no recording
|
|
113
|
+
are listed rather than omitted**, so a package cannot read as complete when it is not, and a
|
|
114
|
+
fuzzy-matched clip is marked as matched rather than named, because that is a guess that this
|
|
115
|
+
video shows this scenario.
|
|
116
|
+
|
|
117
|
+
A video not tied to a named scenario is only a recording. A package not tied to a work item has
|
|
118
|
+
the same problem one level up: six months later nobody can say which delivery it belonged to.
|
|
119
|
+
|
|
120
|
+
**Known gap, and say it rather than hiding it:** the tool writes the manifest but does not
|
|
121
|
+
assemble the folder - the copying, renaming and document conversion is still a project script or
|
|
122
|
+
hand work. The standard's own rule is that a package assembled by hand gets skipped exactly when
|
|
123
|
+
the delivery is under pressure. If you are doing it by hand, say so in the report and recommend
|
|
124
|
+
scripting it.
|
|
125
|
+
|
|
126
|
+
## 6. Report
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
6 scenarios in scope for "Till hardening" (issue #412)
|
|
130
|
+
5 recorded, checked, packaged to <path>
|
|
131
|
+
1 has no journey - LOCK-OPEN-002 - not recorded, and not covered
|
|
132
|
+
manifest.json written: 6 scenarios, 1 with no recording
|
|
133
|
+
folder assembled by hand (no package command); recommend scripting it
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Never promote a status because a recording exists. A video is what lets a person promote one
|
|
137
|
+
themselves in `criteria-lookup`, watching it - which is what `verified` has always meant.
|