qfai 1.10.0 → 1.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +19 -4
  2. package/assets/init/.qfai/assistant/README.md +27 -0
  3. package/assets/init/.qfai/assistant/agents/completion-reviewer.md +16 -0
  4. package/assets/init/.qfai/assistant/agents/delivery-planner.md +8 -0
  5. package/assets/init/.qfai/assistant/agents/implementation-reviewer.md +11 -0
  6. package/assets/init/.qfai/assistant/agents/qa-gatekeeper.md +99 -14
  7. package/assets/init/.qfai/assistant/agents/test-design-analyst.md +3 -1
  8. package/assets/init/.qfai/assistant/catalog/test-layers-ci-lanes.md +60 -0
  9. package/assets/init/.qfai/assistant/catalog/test-layers.md +125 -49
  10. package/assets/init/.qfai/assistant/constitution/constitution.md +1 -1
  11. package/assets/init/.qfai/assistant/constitution/drift-protocol.md +60 -5
  12. package/assets/init/.qfai/assistant/constitution/shared-skill-delegation-baseline.md +177 -2
  13. package/assets/init/.qfai/assistant/manifest/agent-catalog.yml +137 -15
  14. package/assets/init/.qfai/assistant/manifest/agent-routing.yml +69 -3
  15. package/assets/init/.qfai/assistant/skills/qfai-atdd/SKILL.md +126 -69
  16. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/credential-reuse.md +146 -0
  17. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/red-provenance.md +456 -0
  18. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/review-fix-rounds.md +128 -0
  19. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/scaffolding.md +29 -0
  20. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/shared-test-artifacts.md +96 -0
  21. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/stale-manifest.md +34 -0
  22. package/assets/init/.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md +29 -1
  23. package/assets/init/.qfai/assistant/skills/qfai-configure/SKILL.md +1 -1
  24. package/assets/init/.qfai/assistant/skills/qfai-discussion/references/review-cycle-playbook.md +4 -0
  25. package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/summary.json +2 -0
  26. package/assets/init/.qfai/assistant/skills/qfai-implement/SKILL.md +50 -50
  27. package/assets/init/.qfai/assistant/skills/qfai-implement/references/checkpoint-verification.md +136 -32
  28. package/assets/init/.qfai/assistant/skills/qfai-implement/references/cross-spec-ownership.md +1 -1
  29. package/assets/init/.qfai/assistant/skills/qfai-implement/references/evidence-revision.md +222 -8
  30. package/assets/init/.qfai/assistant/skills/qfai-implement/references/execution-ledger.md +110 -24
  31. package/assets/init/.qfai/assistant/skills/qfai-implement/references/final-checklist.md +12 -3
  32. package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-not-observable.md +48 -6
  33. package/assets/init/.qfai/assistant/skills/qfai-implement/references/review-artifact-layout.md +15 -2
  34. package/assets/init/.qfai/assistant/skills/qfai-implement/references/round-evidence.md +19 -3
  35. package/assets/init/.qfai/assistant/skills/qfai-implement/references/upstream-artifact-ordering.md +33 -0
  36. package/assets/init/.qfai/assistant/skills/qfai-sdd/SKILL.md +7 -2
  37. package/assets/init/.qfai/assistant/skills/qfai-sdd/references/review-cycle-playbook.md +4 -0
  38. package/assets/init/.qfai/assistant/skills/qfai-sdd/references/spec-traceability-rules.md +21 -8
  39. package/assets/init/.qfai/assistant/skills/qfai-verify/references/articles.md +1 -1
  40. package/assets/init/root/.github/workflows/qfai-tests.yml +318 -0
  41. package/assets/init/root/.github/workflows/qfai-validate.yml +327 -24
  42. package/dist/cli/index.cjs +12558 -9416
  43. package/dist/cli/index.cjs.map +1 -1
  44. package/dist/cli/index.mjs +8070 -4913
  45. package/dist/cli/index.mjs.map +1 -1
  46. package/dist/index.cjs +3175 -1575
  47. package/dist/index.cjs.map +1 -1
  48. package/dist/index.d.cts +146 -9
  49. package/dist/index.d.ts +146 -9
  50. package/dist/index.mjs +3096 -1502
  51. package/dist/index.mjs.map +1 -1
  52. package/package.json +5 -1
package/README.md CHANGED
@@ -121,13 +121,28 @@ npx qfai report
121
121
 
122
122
  ## ATDD annotation hard gate
123
123
 
124
- `qfai validate` enforces spec-to-test traceability with directory-based rules.
124
+ `qfai validate` enforces spec-to-test traceability. `US` and `CON-API` obligations are routed by ID type;
125
+ a `TC` obligation is routed by the `Level` its spec declares for it.
125
126
 
126
127
  - `tests/e2e/**`: annotate all covered user stories with concrete IDs such as `QFAI:SPEC-0001:US-0001`.
127
- - `tests/integration/**`: annotate all covered test cases with concrete IDs such as `QFAI:SPEC-0001:TC-0001`.
128
128
  - `tests/api/**`: annotate all covered API contracts with concrete IDs such as `QFAI:CON-API-0001`.
129
- - `tests/api/**` and `tests/e2e/**` must not use `TC` annotations.
129
+ - Annotate a covered test case with a concrete ID such as `QFAI:SPEC-0001:TC-0001`, in the directory its declared `Level` names:
130
+
131
+ | `Level` | Annotated in |
132
+ | ----------------------------- | ---------------------- |
133
+ | `L1`/`Unit`, `L2`/`Component` | no ATDD annotation |
134
+ | `L3`/`Integration` | `tests/integration/**` |
135
+ | `L4`/`API` | `tests/api/**` |
136
+ | `L5`/`E2E` | `tests/e2e/**` |
137
+ | none declared, or unreadable | `tests/integration/**` |
138
+
139
+ - Unit and Component test cases carry **no** ATDD annotation obligation. They are gated by the
140
+ `tdd/test-list.md` ledger instead, so do not copy them into `tests/integration/**` to satisfy this gate.
141
+ - A `TC` annotation outside the directory its declared `Level` names is rejected. The rule is `Level`-relative,
142
+ not a blanket ban: a `TC` in `tests/api/**` is accepted only for a test case that declares `L4`/`API`, and in
143
+ `tests/e2e/**` only for `L5`/`E2E`.
130
144
  - `AC` annotations are not required in code; AC coverage is treated as indirect through full `TC` coverage.
145
+ - These directories follow `paths.testsDir` from `qfai.config.yaml`; `tests/` above is the default.
131
146
 
132
147
  ## Operating model (skills-driven workflow)
133
148
 
@@ -145,7 +160,7 @@ The agent reads QFAI assets under `.qfai/assistant/` and produces or updates SDD
145
160
  QFAI includes a small set of custom skills (stored under `.qfai/assistant/skills/`) designed to keep the workflow opinionated and repeatable.
146
161
 
147
162
  - **qfai-configure**: Analyze the repository (language, frameworks, test layout, directory structure)
148
- and tailor `qfai.config.yaml` accordingly (especially `testFileGlobs`).
163
+ and adjust `qfai.config.yaml` accordingly (especially `testFileGlobs`).
149
164
  Run this once right after `npx qfai init`, and re-run it when the repository structure changes.
150
165
  - **qfai-discussion**: Run a unified structured discussion that produces and maintains the latest discussion pack
151
166
  as 15 required markdown files under `.qfai/discussion/discussion-<ts>/`.
@@ -0,0 +1,27 @@
1
+ # QFAI assistant tree
2
+
3
+ This directory is the canonical source for QFAI's skills, agents, constitution,
4
+ manifest and catalog. `npx qfai init` writes it once and never removes it, and the
5
+ tool-specific integration directories are built from it.
6
+
7
+ ## Canonical entrypoint
8
+
9
+ Every tool integration resolves through symlinks that point back here:
10
+
11
+ - .qfai/assistant/skills/ — skill documents
12
+ - .qfai/assistant/agents/ — agent definitions
13
+
14
+ These documents are the SSOT. Edit them here, not through the symlinks under
15
+ `.claude/`, `.agents/`, `.codex/` or `.github/`.
16
+
17
+ ## Integration surface
18
+
19
+ `npx qfai init` creates the wrappers under those four directories. They are
20
+ generated, so re-running `npx qfai init` restores any that a checkout flattened or
21
+ a cleanup removed; nothing there needs to be edited by hand.
22
+
23
+ `npx qfai validate` checks that the wrappers still resolve to the documents above.
24
+ It reads this file to tell "init has run here and the surface was deleted" from
25
+ "init has never run here" — the two look identical from the integration
26
+ directories alone once every wrapper is gone, and only one of them is a
27
+ problem. Leave it in place.
@@ -24,6 +24,22 @@ tools: [Read, Glob, Grep, Bash]
24
24
  - .qfai/assistant/catalog/test-layers.md
25
25
  - .qfai/specs/spec-\*/09_delta.md
26
26
  - Validation evidence and gate results
27
+ - `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
28
+ - The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
29
+ or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
30
+
31
+ **Validate evidence is a completion-gate input, not an item-cycle one.** When
32
+ this role is routed inside an item cycle — `/qfai-atdd` stage gate P1c hands a
33
+ single row to `/qfai-implement` and that run's reviewers gate its checkpoint —
34
+ `.qfai/report/validate.log`, the coverage reports and runtime evidence are P5/P6
35
+ artifacts of the calling stage and do not exist yet. Requiring them there
36
+ stopped the first branch-1 row at `refactor`, which Phase Red does not
37
+ re-select, so the calling stage never reached P2. Judge the row's own
38
+ phase-authored evidence; the completion gate is where the rest is owed. **The
39
+ two inputs above are what makes that possible** — without the ledger and the
40
+ evidence home its `Layer` selects, this role cannot identify the artifact it is
41
+ being asked to judge, and falls into its own Stop condition ("Required evidence
42
+ ... missing") on a correct branch-1 or branch-2 row.
27
43
 
28
44
  ## Deliverables
29
45
 
@@ -33,6 +33,14 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
33
33
  - .qfai/specs/spec-\*/01_Spec.md
34
34
  - `.qfai/specs/spec-*/tdd/test-list.md` — the execution ledger this role selects
35
35
  the next item from and whose Red-Green-Refactor ordering it enforces
36
+ - **The document the row's obligation column points at.** Item scope is "is this
37
+ selector a sufficient slice of the obligation", and the obligation is not
38
+ always a `TC-*`: an `E2E` row owes `US-Refs` and an `API` row owes
39
+ `CON-API-Refs`. Without these the role has nothing to compare such a row
40
+ against and can only guess a PASS or stall the gate.
41
+ - `.qfai/specs/spec-*/06_Test-Cases.md` for a `TC-Refs` row
42
+ - `.qfai/specs/spec-*/02_User-stories.md` for a `US-Refs` row
43
+ - `.qfai/contracts/api/**` for a `CON-API-Refs` row
36
44
  - .qfai/discussion/discussion-\*/04_Sources.md
37
45
  - .qfai/discussion/discussion-\*/06_REQ.md
38
46
  - .qfai/discussion/discussion-\*/11_OQ-Register.md
@@ -31,6 +31,17 @@ tools: [Read, Glob, Grep, Bash]
31
31
  - .github/instructions/principles.instructions.md
32
32
  - Diff of changed files
33
33
  - `.qfai/contracts/api/**` and `.qfai/contracts/db/**`
34
+ - `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
35
+ - The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
36
+ or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
37
+
38
+ **The last two are what the `Audited evidence hash` is computed over.** This
39
+ role records that hash itself, over the row's phase-authored fields — and those
40
+ live in an evidence file that is normally ignored, so the diff of changed files
41
+ does not contain them. Without the ledger and the evidence home the row's
42
+ `Layer` selects, this role cannot identify its own audit subject: the hash goes
43
+ missing and gate items 10-11 stop, or the orchestrator computes it instead,
44
+ which is the one thing the contract says must not happen.
34
45
 
35
46
  ## Deliverables
36
47
 
@@ -21,7 +21,9 @@ tools: [Read, Glob, Grep, Bash]
21
21
 
22
22
  ## Ownership boundaries
23
23
 
24
- - `delivery-planner` owns **item selection and item scope** — whether a ledger row's selector is a sufficient slice of its `TC-*` obligation. Do not adjudicate item scope here; a PASS on observation
24
+ - `delivery-planner` owns **item selection and item scope** — whether a ledger
25
+ row's selector is a sufficient slice of the obligation its `Layer` names
26
+ (`TC-Refs`, `US-Refs` or `CON-API-Refs`). Do not adjudicate item scope here; a PASS on observation
25
27
  evidence is explicitly scoped to that observation and never widens or ratifies item scope. See `.qfai/assistant/skills/qfai-implement/SKILL.md#precedence-between-delivery-planner-and-qa-gatekeeper`.
26
28
  - Refuse to evaluate RED/GREEN evidence while an unresolved `delivery-planner` scope REVISE is open on the same item.
27
29
 
@@ -32,7 +34,8 @@ the item owns, and nothing downstream re-asks: coverage is annotation presence
32
34
  and the Depth Matrix counts case categories. A test that cannot fail otherwise
33
35
  clears every gate.
34
36
 
35
- Require an `Oracle proof` on each item and **reject** it when:
37
+ Require an `Oracle proof` on each item **at a GREEN or completion gate**, and
38
+ **reject** it when:
36
39
 
37
40
  - the mutation is outside the code the item owns — breaking a shared helper
38
41
  proves the helper is used, not that this test discriminates;
@@ -41,6 +44,14 @@ Require an `Oracle proof` on each item and **reject** it when:
41
44
  - the failing output names a selector other than the row's;
42
45
  - the recorded command differs from the `GREEN command`.
43
46
 
47
+ **At a RED observation the proof is a plan, and a plan is enough.** Branch 1's
48
+ RED is taken before any production behaviour exists, so there is nothing to
49
+ mutate: the item names the predicate it will break and the command it will run.
50
+ Requiring a demonstrated mutation there made a correct observed RED unable to
51
+ pass P1b and so unable to reach Phase Green — the phase that builds the code the
52
+ mutation needs. Judge the plan for whether it names this row's predicate and
53
+ selector; judge the demonstration once the behaviour exists.
54
+
44
55
  `equivalent-mutant` is acceptable **only** when the named contract clause is
45
56
  genuinely weaker than the obligation. It is an upstream gap: route it as an
46
57
  advisory / Change Request, do not send the implementer to strengthen an
@@ -78,12 +89,46 @@ test:
78
89
  - "the suite is green" in place of the row's own GREEN.
79
90
 
80
91
  The one legitimate absence is the _RED not observable_ path: the obligation is
81
- already satisfied by a sibling row, so the correct test passes first run. Then
82
- require `Satisfied-by`, `Falsifiability command` and `Falsifiability result`
83
- instead — never both forms, never neither. See
92
+ already satisfied by something already in the tree, so the correct test passes
93
+ first run. Then require `Satisfied-by`, `Falsifiability command` and
94
+ `Falsifiability result` instead — never both forms, never neither.
95
+
96
+ **On an `E2E` / `API` / `Integration` row, `Satisfied-by` need not be a sibling `TDD-NNNN`.** A
97
+ production **path and symbol** is equally valid there and is the normal answer
98
+ for a row whose surface no ledger row owns; rejecting it sends every such row to
99
+ `exception`, the terminal state the path exists to avoid. Judge it on whether it
100
+ answers "what would I mutate to falsify this row".
101
+
102
+ **A commit id alone does not answer it — REVISE.** A commit that touched
103
+ several routes and a helper names no single predicate, so the ownership check
104
+ below has no boundary to apply and would accept a mutation anywhere inside it.
105
+ The producer contract requires the symbol for this reason
106
+ (`../skills/qfai-atdd/references/red-provenance.md#the-three-branches-must`); a
107
+ commit recorded **alongside** the path and symbol is provenance and is fine.
108
+
109
+ **And the mutation may touch it.** The Oracle Strength Check rejects a mutation
110
+ outside the code the item owns, which on an `E2E` / `API` / `Integration` row is every
111
+ production predicate there is — the same sentence above says no ledger row owns
112
+ that surface. Applied literally, no branch-2 row could ever produce
113
+ falsifiability evidence that passes. On a handed-over row, **the predicate
114
+ `Satisfied-by` names is the owned code** for this check; anything else is still
115
+ out of bounds.
116
+
117
+ **On any other row the sibling row is still required** — production code
118
+ no ledger row owns is the anomaly case there, not a substitute. See
84
119
  `.qfai/assistant/skills/qfai-implement/references/red-not-observable.md` and
85
120
  `.qfai/assistant/skills/qfai-implement/references/red-admissibility.md`.
86
121
 
122
+ **A `Layer = E2E` / `Layer = API` row from `/qfai-atdd` is judged the same
123
+ way.** Its journey is often written after the surface the same cycle built, so
124
+ the falsifiability form is the expected evidence rather than a concession —
125
+ accept it, with the mutated predicate being one the journey actually asserts
126
+ on. What is **not** acceptable is the third outcome appearing by default: a row
127
+ routed to `exception` whose `DR-*` says only that the surface came first has
128
+ not shown that either branch was unavailable, and that is a REVISE. See
129
+ `.qfai/assistant/skills/qfai-atdd/SKILL.md#red-provenance-for-an-atdd-owned-row-must`
130
+ and `.qfai/assistant/skills/qfai-implement/references/execution-ledger.md#atdd-owned-rows`.
131
+
87
132
  Verdict scope: a PASS covers the observation for that round and nothing else. It
88
133
  does not ratify item scope and does not clear the completion gate.
89
134
 
@@ -91,7 +136,11 @@ does not ratify item scope and does not clear the completion gate.
91
136
 
92
137
  In addition to traceability-based coverage (US/TC/CON-API existence), verify the **depth** of test cases:
93
138
 
94
- - Confirm a Coverage Depth Matrix exists (produced by `test-design-analyst`). Missing matrix: REVISE from the ATDD review cycle onward; on an SDD review cycle record it as a finding. See the scope note.
139
+ - Confirm a Coverage Depth Matrix exists at `.qfai/evidence/coverage-depth-<spec-id>.md` (produced by `test-design-analyst`).
140
+ Missing matrix: REVISE from the ATDD review cycle onward; on an SDD review cycle record it as a finding. See the scope note.
141
+ A matrix that exists only inside `.qfai/evidence/atdd-<spec-id>.md` is a **missing** matrix: that file is ignored by the
142
+ managed `.gitignore` block, so neither it nor the justification for any `❌` reaches a commit, and the "unjustified"
143
+ judgement cannot be re-made by anyone reading the repository.
95
144
  - Check that each US/TC has test cases for at minimum: normal path AND error/failure path.
96
145
  - Flag any US/TC that has only normal-path test cases as a coverage gap.
97
146
  - Reference: `.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`
@@ -105,8 +154,8 @@ In addition to traceability-based coverage (US/TC/CON-API existence), verify the
105
154
  The Coverage Depth Matrix is an **ATDD-stage artifact**: it is defined in
106
155
  `.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`, listed as an ATDD
107
156
  Mandatory Output,
108
- and written into `.qfai/evidence/atdd-<spec-id>.md`. `qfai-sdd` neither defines its layout nor
109
- ships a section for it, so:
157
+ and written to `.qfai/evidence/coverage-depth-<spec-id>.md` a committed path, unlike the rest of
158
+ `.qfai/evidence/**`. `qfai-sdd` neither defines its layout nor ships a section for it, so:
110
159
 
111
160
  - Apply this check from the **ATDD review cycle onward**, where
112
161
  `.qfai/assistant/skills/qfai-atdd/SKILL.md` lists
@@ -124,14 +173,47 @@ ships a section for it, so:
124
173
  - .qfai/assistant/catalog/test-layers.md
125
174
  - .qfai/specs/spec-\*/09_delta.md
126
175
  - `.qfai/specs/spec-*/tdd/test-list.md` — the ledger row under review
127
- - `.qfai/evidence/implement-<spec-id>.md` — the per-item RED/GREEN evidence this
128
- role adjudicates. Both are listed because the Stop condition below ("target
129
- artifacts are missing") is not checkable against an artifact this role was
130
- never told to open.
131
- - `.qfai/report/validate.log`
176
+ - **The per-item RED/GREEN evidence for the row under review — in the file its
177
+ `Layer` owns, and only that one.** `.qfai/evidence/atdd-<spec-id>.md`, under
178
+ `## Ledger rows advanced`, for a `Layer = E2E` / `Layer = API` / `Layer = Integration` row;
179
+ `.qfai/evidence/implement-<spec-id>.md` for every other row. Both are listed
180
+ because the Stop condition below ("target artifacts are missing") is not
181
+ checkable against an artifact this role was never told to open — but
182
+ requiring **both** makes that condition fire on a spec that legitimately has
183
+ one: a Unit-only spec never ran `/qfai-atdd`, and a spec whose rows are all
184
+ `E2E` / `API` / `Integration` has no implement file. Either way the gate would stop before
185
+ reading the evidence that does exist.
186
+ **The three below are required at a completion gate, not at a RED/GREEN
187
+ observation.** `/qfai-atdd` routes this role as blocking at stage gate P1b, and
188
+ validate output, coverage reports and runtime evidence are first produced at its
189
+ P5 and P6 — so requiring them there stopped a fresh run that had a perfectly
190
+ good RED pair, on artifacts its own ordering says cannot exist yet. At an
191
+ observation gate the row's own evidence above is the whole input.
192
+
193
+ - The scoped validate JSON for the spec under review — `validate.spec-<id>.json`
194
+ **beside the configured `output.validateJsonPath`**, not under a fixed
195
+ `.qfai/report/` — or the `run-*/` directory of the run that produced it, under
196
+ the configured `paths.outDir`. Read both from `qfai.config.yaml` the way the
197
+ SDD and discussion contracts do. A project that moved either output writes its
198
+ evidence where it said to, and looking for it at the default path reported a
199
+ missing artifact and stopped completion on a validate run that had succeeded
200
+ and left everything it owed. **Not `validate.log`**: it and the run-log pointer are shared by every
201
+ run, scoped or not, and nothing serializes them, so a sibling stage validating
202
+ at the same time overwrites what this one wrote — a failing run followed by a
203
+ sibling's success reads as this spec's PASS
132
204
  - `.qfai/report/specs-coverage/spec-*.md`
133
205
  - Runtime evidence and prototyping evidence artifacts
134
206
 
207
+ **Branch 3 gets its own verdict.** The observation gate admits an observed RED
208
+ or a falsifiability trio and calls anything else "never neither" — but a genuine
209
+ branch-3 row _has_ neither, by the finding that put it there. Judged by the two
210
+ forms it can only be REVISE, and skipping the gate leaves the stage's completion
211
+ condition unmet, so the row could not close either way. Judge these on their own
212
+ terms: a `DR-*` that records **what could not be observed and why each branch was
213
+ unavailable**, PASS or REVISE on that. A missing `DR-*`, or one that names no
214
+ unavailability, is still REVISE — this is a third form of evidence, not an
215
+ exemption from having any.
216
+
135
217
  ## Deliverables
136
218
 
137
219
  - Gate decision (PASS / REVISE) with rationale
@@ -140,7 +222,10 @@ ships a section for it, so:
140
222
 
141
223
  ## Stop conditions
142
224
 
143
- - Required evidence, governing specs, or target artifacts are missing.
225
+ - Required evidence, governing specs, or target artifacts are missing — judged
226
+ against what the invoking phase requires, per the note above the last three
227
+ inputs. At a RED/GREEN observation that is the row's own evidence; at a
228
+ completion gate it is all of them.
144
229
  - The request requires implementation or file editing instead of independent review.
145
230
  - The issue falls outside this review domain and must be rerouted to another specialist first.
146
231
 
@@ -58,7 +58,9 @@ At both stages: when business rules (BR-\*) exist, verify each BR has at least o
58
58
  - Coverage plan and layer ownership
59
59
  - Test-case quality and traceability findings
60
60
  - **Coverage Depth Matrix** (per spec, using the template in the depth checklist reference).
61
- Destination: `.qfai/evidence/atdd-<spec-id>.md` from the ATDD stage onward. During SDD there is
61
+ Destination: `.qfai/evidence/coverage-depth-<spec-id>.md` from the ATDD stage onward its own
62
+ file, because that path is committed while the rest of `.qfai/evidence/**` is ignored, and the
63
+ justification behind each `❌` is the input `qa-gatekeeper` reads. During SDD there is
62
64
  no evidence artifact that holds it, so report depth gaps as findings instead of producing the
63
65
  matrix format.
64
66
  - Volume estimate and risk notes
@@ -0,0 +1,60 @@
1
+ # Test layers to CI lanes
2
+
3
+ A crosswalk between the test layers in [`test-layers.md`](./test-layers.md) and the CI lanes
4
+ the shipped workflows run. It exists so that "which job runs my L3 tests" has one answer a
5
+ reader can find, instead of being rediscovered from a workflow file each time.
6
+
7
+ **The layer policy loader does not read this file.** (Written without a hyphen on purpose: the
8
+ loader extracts anything matching `layer-<word>` from the file it parses, and while it does not
9
+ parse this one, a catalog-directory file carrying such a token is a hazard waiting for the day
10
+ someone widens the loader to scan the directory.) It resolves `catalog/test-layers.md` by
11
+ exact path and reads nothing else in this directory, so nothing written here can widen or
12
+ narrow the layer vocabulary. That is deliberate: this document is a map, and a map that could
13
+ change the territory would be a second source of truth for something that already has one. If
14
+ you need to change what layers exist, change `test-layers.md` — this file follows.
15
+
16
+ ## The mapping
17
+
18
+ | Layer | What it exercises | Shipped CI lane |
19
+ | -------------- | --------------------------------------------- | --------------- |
20
+ | L1 Unit | one module, no I/O | `unit` |
21
+ | L2 Component | one component and its immediate collaborators | `component` |
22
+ | L3 Integration | several modules across a real boundary | `integration` |
23
+ | L4 API | a running interface, contract-first | `api` |
24
+ | L5 E2E | the product as a user reaches it | `e2e` |
25
+
26
+ Two lanes in the shipped workflow are not layers and have no row above:
27
+
28
+ - **`detection`** decides which of the lanes above need to run for a given change, and
29
+ publishes that decision as job outputs. It is infrastructure, not a test level.
30
+ - **`verdict`** aggregates the lanes into the single status a branch-protection rule can name.
31
+ It runs unconditionally so that a run where every lane was skipped is still distinguishable
32
+ from a run where nothing was verified.
33
+
34
+ ## Lane names are a project's choice
35
+
36
+ The names in the third column are the ones the shipped workflow uses. They are not a contract:
37
+ a project that calls its integration lane `service-tests` is not violating anything, and this
38
+ table is not a rename list. What matters is that each layer a project uses has some lane that
39
+ runs it, and that the lane's name stays put once branch protection refers to it — a check name
40
+ is a repository setting, and renaming one silently makes a required check unsatisfiable.
41
+
42
+ ## Per-level routing is not enforced
43
+
44
+ `test-layers.md` marks per-level annotation routing as a target state and says plainly: not
45
+ enforced, do not follow yet. This document does not change that and does not activate it. The
46
+ live traceability gate reads one directory for annotations, and it is the gate — not this
47
+ table — that decides where an annotation counts.
48
+
49
+ So: read this file to find out which lane runs a layer. Do not read it as instructions about
50
+ where anything belongs in the tree. That question is settled by the gate's own scope, and by
51
+ `test-layers.md` for the vocabulary.
52
+
53
+ ## Keeping this file honest
54
+
55
+ - Every layer code here also appears in `test-layers.md`'s crosswalk. If you add a layer there
56
+ and not here, this table is incomplete; if you add one here and not there, you have invented
57
+ a layer nothing enforces.
58
+ - The lane column describes the shipped workflow. If the shipped workflow's job set changes,
59
+ this table is stale, and stale is worse than absent for a document whose only job is to save
60
+ someone a lookup.
@@ -2,6 +2,11 @@
2
2
 
3
3
  This document is the SSOT for ATDD test-layer semantics and completion gates.
4
4
 
5
+ For which CI lane runs which layer, see the sibling map
6
+ [`test-layers-ci-lanes.md`](./test-layers-ci-lanes.md). That file is a crosswalk only — the
7
+ policy loader reads this file and not that one, so nothing there can change the
8
+ vocabulary declared below.
9
+
5
10
  ## Layer vocabulary crosswalk (normative)
6
11
 
7
12
  qfai spells the same layer four ways across shipped artifacts. This table is
@@ -31,17 +36,21 @@ Rules:
31
36
  - L1 and L2 have no mandated directory: unit and component tests live wherever
32
37
  the project's own convention puts them. Only L3-L5 are directory-pinned, and
33
38
  only those directories are scanned by the ATDD traceability rules.
34
- - **A `TC-*` row's `Level` is L1-L3.** The ATDD annotation hard gate routes an
35
- obligation by its ID, not by its `Level`: `US-*` is answered from
36
- `<testsDir>/e2e/**` (`QFAI-ATDD-111`), `TC-*` from
37
- `<testsDir>/integration/**` (`QFAI-ATDD-112`) and `CON-API-*` from
38
- `<testsDir>/api/**` (`QFAI-ATDD-113`) and `CON-DB-*` from
39
- `<testsDir>/integration/**` (`QFAI-ATDD-115`), while a `TC-*` reference inside
40
- `<testsDir>/api/**` or `<testsDir>/e2e/**` is rejected outright
41
- (`QFAI-ATDD-121` / `QFAI-ATDD-122`). L4's goal is `CON-API-*` and L5's is
42
- `US-*` (see the layer definitions below), so an oracle that lands at L4 or L5
43
- means the obligation is misfiled: record it as `CON-API-*` or `US-*` rather
44
- than as a `TC-*` row no test directory can carry.
39
+ - **A `TC-*` row's `Level` is L1-L3.** Of those, only L3 owes an ATDD
40
+ annotation: L1 and L2 have no mandated directory, so `QFAI-ATDD-112` does not
41
+ apply to them and `QFAI-ATDD-117` (`info`) names them instead. Their gate is
42
+ `tdd/test-list.md` / `TDDLIST_TC_NOT_COVERED`, under `/qfai-implement`.
43
+ `US-*` is answered from `<testsDir>/e2e/**` (`QFAI-ATDD-111`), `CON-API-*`
44
+ from `<testsDir>/api/**` (`QFAI-ATDD-113`) and `CON-DB-*` from
45
+ `<testsDir>/integration/**` (`QFAI-ATDD-115`) those three are fixed by the
46
+ ID type. A `TC-*` is answered from the directory **its own declared `Level`**
47
+ names, which for a correctly filed row is `<testsDir>/integration/**`; see
48
+ [Annotation routing](#annotation-routing) for the full table and the
49
+ misplacement rules.
50
+ L4's goal is `CON-API-*` and L5's is `US-*` (see the layer definitions
51
+ below), so an oracle that lands at L4 or L5 means the obligation is misfiled:
52
+ record it as `CON-API-*` or `US-*` rather than leaving a `TC-*` row at a
53
+ layer whose goal is another ID type.
45
54
  - The two code-side word lists (`tddHelpers.ts#UNIT_COMPONENT_LAYERS` /
46
55
  `#NON_COVERAGE_LAYERS`) accept both the code and the word form for the same
47
56
  layer; they MUST stay in step with this table.
@@ -73,7 +82,7 @@ writer targets a declared layer instead.
73
82
  - Scope: pure decision logic — a single module's inputs and return values, with
74
83
  no port collaboration and no real infrastructure.
75
84
  - Goal: verify `TC-*` obligations whose oracle observes inputs and outputs only.
76
- - Location rule: `tests/unit/**`.
85
+ - Convention: `tests/unit/**`. L1 has no mandated directory and owes no ATDD annotation — see the crosswalk and "Unit and Component owe no ATDD annotation".
77
86
 
78
87
  ### L2 Component
79
88
 
@@ -81,7 +90,7 @@ writer targets a declared layer instead.
81
90
  with no real infrastructure.
82
91
  - Goal: verify `TC-*` obligations whose oracle observes the interaction with a
83
92
  port rather than infrastructure state.
84
- - Location rule: `tests/component/**`.
93
+ - Convention: `tests/component/**`. L2 has no mandated directory and owes no ATDD annotation — see the crosswalk and "Unit and Component owe no ATDD annotation".
85
94
 
86
95
  ### L3 Integration
87
96
 
@@ -137,8 +146,23 @@ falsifying-oracle rule:
137
146
  observable at two layers, it is two obligations: write one TC per oracle and
138
147
  give each its own `Level`.
139
148
 
140
- - A multi-valued `Level` cell (`L3/L5`) is **illegal**. Nothing consumes it and
141
- no validator can route it.
149
+ - A multi-valued `Level` cell (`L3/L5`, `L1/L2`, `L1, L3`) is **illegal**. It
150
+ matches no entry in the crosswalk, so no rule can read a layer out of it.
151
+ - **What a reader does with one: split the row.** One TC per oracle, each with
152
+ its own single `Level`. Nothing else is a fix — in particular, do not record
153
+ a normalization that "drops one half": no tool performs one, and a note
154
+ saying an `L3/L5` row "normalizes to `L3`" is a claim about a value that only
155
+ `06_Test-Cases.md` can make.
156
+ - **What the validators do with one, until it is split.** They neither guess
157
+ nor let it through:
158
+ - `QFAI-ATDD-112` routes the TC to the same place a TC with no declared
159
+ `Level` goes — `<testsDir>/integration/**` — and keeps the obligation.
160
+ Unreadable is deliberately not "excused": if a cell qfai cannot read
161
+ discharged the obligation, `L1/L2` would be a one-keystroke way to delete
162
+ any TC from the gate. That default is where the obligation is _reported_,
163
+ not where the obligation _belongs_.
164
+ - `TDDLIST_UNKNOWN_LEVEL` (`warning`) names the cell, and the TC stays a
165
+ coverage target, so `tdd/test-list.md` still owes it a row.
142
166
  - If splitting is genuinely impossible, escalate through the Drift Protocol
143
167
  rather than inventing a combined value.
144
168
 
@@ -158,15 +182,43 @@ L4" applies only when the service-boundary values are themselves what the
158
182
  parent BR owns. Inverting this is what collapses a designed pyramid into an
159
183
  all-integration suite.
160
184
 
161
- ### Annotation routing is by ID type, not by `Level`
162
-
163
- The derived `Level` records which oracle owns the obligation. It does **not**
164
- move the traceability annotation. The [ATDD annotation hard gate](#atdd-annotation-hard-gate)
165
- routes by obligation ID: `US-*` is answered from `tests/e2e/**`
166
- (`QFAI-ATDD-111`), `TC-*` from `tests/integration/**` (`QFAI-ATDD-112`), and
167
- `CON-API-*` from `tests/api/**` (`QFAI-ATDD-113`); a `TC-*` reference inside
168
- `tests/api/**` or `tests/e2e/**` is rejected outright (`QFAI-ATDD-121` /
169
- `QFAI-ATDD-122`). Two consequences bind every `TC-*` row:
185
+ ### Annotation routing
186
+
187
+ The derived `Level` records which oracle owns the obligation, and the
188
+ [ATDD annotation hard gate](#atdd-annotation-hard-gate) routes each obligation
189
+ ID to exactly one directory. `US-*` is answered from `tests/e2e/**`
190
+ (`QFAI-ATDD-111`) and `CON-API-*` from `tests/api/**` (`QFAI-ATDD-113`); those
191
+ two are fixed by the ID type. A `TC-*` is answered from the directory **its own
192
+ declared `Level`** names (`QFAI-ATDD-112`):
193
+
194
+ | `Level` | Answered from |
195
+ | ----------------------------- | ------------------------------ |
196
+ | `L1`/`Unit` | no ATDD obligation |
197
+ | `L2`/`Component` | no ATDD obligation |
198
+ | `L3`/`Integration` | `tests/integration/**` |
199
+ | `L4`/`API` | `tests/api/**` (note) |
200
+ | `L5`/`E2E` | `tests/e2e/**` (note) |
201
+ | none declared | `tests/integration/**` |
202
+ | anything else — typo, `L3/L5` | `tests/integration/**` (note2) |
203
+
204
+ **(note)** A `TC-*` **should not be** at L4 or L5 — the first bullet below says
205
+ why and what to do instead. The gate routes it there rather than rejecting it so
206
+ a misfiled row is reported once, by the rule that names the real cause, instead
207
+ of twice as "uncovered in integration" and "forbidden in api".
208
+
209
+ **(note2)** A `Level` the crosswalk does not list — a typo, a project's own
210
+ word, or the illegal multi-valued cell — falls to the same default as an
211
+ undeclared one, and keeps its obligation. The default is the conservative
212
+ answer to a cell qfai cannot read, never a supported spelling: fix the cell
213
+ (see [Obligation spanning more than one layer](#obligation-spanning-more-than-one-layer)).
214
+ `TDDLIST_UNKNOWN_LEVEL` (`warning`) names such a cell on the ledger side.
215
+
216
+ Exactly one directory, never two: an annotation outside the one its `Level`
217
+ names is both uncovered and rejected (`QFAI-ATDD-121` / `QFAI-ATDD-122` /
218
+ `QFAI-ATDD-123`), and
219
+ the rejection is symmetric — an annotation left in `tests/integration/**` after
220
+ its TC moved to `L4`/`L5` is rejected the same way an early one in
221
+ `tests/api/**` is. Two consequences bind every `TC-*` row:
170
222
 
171
223
  - **A `TC-*` row's `Level` stays within L1–L3.** L4's goal is `CON-API-*` and
172
224
  L5's goal is `US-*` (see the layer definitions above), so an oracle that
@@ -185,14 +237,15 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
185
237
  parent BR keeps a spec-side obligation, split the row instead: keep a `TC-*`
186
238
  for the part the BR owns — by step 2 that part derives to L1–L3 — and re-file
187
239
  only the boundary assertion as `CON-API-*` / `US-*`.
188
- - **An L1/L2 `Level` does not relax `QFAI-ATDD-112`, and the gate does not
189
- change the `Level`.** The two are independent: the `Level` records what the
190
- oracle observes and is derived before any test exists, while the
191
- `QFAI:SPEC-XXXX:TC-YYYY` annotation is still owed to `tests/integration/**`.
192
- Never rewrite a derived L1/L2 to L3 because no integration trace exists yet
193
- that would make the recorded oracle depend on implementation order and hide
194
- the unit/component work `/qfai-implement` selects. The missing annotation is
195
- an open ATDD obligation to satisfy, not evidence that the `Level` was wrong.
240
+ - **An L1/L2 `Level` carries no `QFAI-ATDD-112` obligation, and the gate does
241
+ not change the `Level`.** The two are independent: the `Level` records what
242
+ the oracle observes and is derived before any test exists, while
243
+ `QFAI-ATDD-112` asks only about the layers ATDD owns. Never rewrite a derived
244
+ L1/L2 to L3 to make a gate quieter that would make the recorded oracle
245
+ depend on implementation order and hide the unit/component work
246
+ `/qfai-implement` selects. An L1/L2 row's obligation is discharged through
247
+ `tdd/test-list.md` and `TDDLIST_TC_NOT_COVERED`, not through an annotation in
248
+ a directory ATDD scans.
196
249
 
197
250
  ## TestKind resolution (single source)
198
251
 
@@ -241,11 +294,21 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
241
294
  - Every `TC-*` in specs must be referenced at least once from the directory
242
295
  its declared `Level` routes to: L3/Integration -> `tests/integration/**`,
243
296
  L4/API -> `tests/api/**`, L5/E2E -> `tests/e2e/**`. A TC with no declared
244
- `Level` defaults to `tests/integration/**`. This is what `QFAI-ATDD-112`
245
- checks. - Use `QFAI:SPEC-XXXX:TC-YYYY` annotations.
246
- - `tests/api/**` and `tests/e2e/**` must not carry `TC-*` annotations
247
- (`QFAI-ATDD-121` / `QFAI-ATDD-122`), so an L4 obligation is discharged as a
248
- `CON-API-*` reference, never as a `TC-*` one.
297
+ `Level` defaults to `tests/integration/**`. **L1/Unit and L2/Component owe
298
+ no reference at all** — see "Unit and Component owe no ATDD annotation"
299
+ below. This is what `QFAI-ATDD-112` checks.
300
+ - Use `QFAI:SPEC-XXXX:TC-YYYY` annotations.
301
+ - A `TC-*` annotation outside the directory its declared `Level` names is
302
+ rejected (`QFAI-ATDD-121` / `QFAI-ATDD-122` / `QFAI-ATDD-123`). The rule is
303
+ `Level`-relative,
304
+ not a blanket ban: a `TC-*` in `tests/api/**` is rejected **unless** that TC
305
+ declares `L4`/`API`, and in `tests/e2e/**` unless it declares `L5`/`E2E` —
306
+ an annotation matching its own `Level` is what discharges the obligation
307
+ there. A `TC-*` should not be at L4 or L5 in the first place (see
308
+ "Annotation routing"): re-file that obligation as `CON-API-*` or `US-*`.
309
+ But while the row exists at that `Level`, its annotation belongs in the one
310
+ directory the `Level` names, and putting it anywhere else leaves the TC
311
+ uncovered as well as forbidden.
249
312
  - Every declared `CON-DB-*` must be referenced at least once from
250
313
  `tests/integration/**` (`QFAI-ATDD-115`). Use `QFAI:CON-DB-XXXX`
251
314
  annotations. L3 owns this because a DB contract is only exercised against
@@ -255,16 +318,27 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
255
318
  current slice defers with a `-- x-qfai-status: planned` comment line,
256
319
  reported at `info` by `QFAI-ATDD-116` so the deferral stays visible.
257
320
 
258
- - Per-level routing (target state **not enforced, do not follow yet**):
259
- - The intended end state is one required location per declared `Level`:
260
- L1 -> `tests/unit/**`, L2 -> `tests/component/**`,
261
- L3 -> `tests/integration/**`. L4 stays `CON-API-*` in `tests/api/**` and
262
- L5 stays `US-*` in `tests/e2e/**`.
263
- - **This is not live.** `buildAtddTestGlobs` scans only
264
- `tests/{e2e,api,integration}`, so an annotation placed in `tests/unit/**`
265
- or `tests/component/**` is invisible to the scanner and `QFAI-ATDD-112`
266
- still reports the TC as uncovered. Until the scanner and `QFAI-ATDD-112`
267
- resolve per-TC, keep discharging every `TC-*` in `tests/integration/**`.
321
+ - **Unit and Component owe no ATDD annotation.** A `TC-*` whose declared
322
+ `Level` is L1 or L2 is outside `QFAI-ATDD-112` entirely: it is not required
323
+ in any directory, and an annotation for it inside a scanned directory is not
324
+ a violation either. `QFAI-ATDD-117` (`info`) names the excluded TCs on every
325
+ run so the exclusion is visible rather than silent.
326
+ - This is the only reading consistent with the rest of the package.
327
+ `qfai-atdd/SKILL.md` puts Unit and Component out of its scope, and the
328
+ crosswalk above gives L1/L2 no mandated directory only L3-L5 are
329
+ directory-pinned and only those three roots are scanned.
330
+ - Previously L1/L2 fell through to `tests/integration/**` — the fallback for
331
+ a spec with no `Level` column at all — so every declared Unit and Component
332
+ TC was an `error` demanding an annotation in a directory this file says is
333
+ not its home. `QFAI-WAIVER-002` refuses waivers on `error` rules, so a
334
+ project that filed unit tests where L1's own entry says to had no exit, and
335
+ the only validator-clean path was duplicating every annotation into
336
+ `tests/integration/**` — the all-integration collapse named under
337
+ Anti-patterns below.
338
+ - **They are still gated, by the other stage.** Every coverage-target `TC-*`
339
+ owes a `tdd/test-list.md` row, and `TDDLIST_TC_NOT_COVERED` (`error`)
340
+ reports a missing one. L1/L2 belong to `/qfai-implement`, which is the
341
+ stage that writes unit and component tests.
268
342
  - API obligations:
269
343
  - Every declared `CON-API-*` must be referenced at least once from `tests/api/**`.
270
344
  - Use `QFAI:CON-API-XXXX` annotations.
@@ -300,8 +374,10 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
300
374
  filed early into `tests/api/**`.
301
375
  - Unknown references (`US/TC/CON-API` not declared) are errors.
302
376
  - A `TC-*` annotation outside the directory its `Level` routes to is a
303
- misplacement, whichever directory it lands in including the two new
304
- locations `tests/unit/**` and `tests/component/**`.
377
+ misplacement, whichever directory it lands in. **This applies to L3-L5 only.**
378
+ L1 and L2 route nowhere — they carry no ATDD annotation obligation at all — so
379
+ an annotation for one is neither required nor misplaced, in
380
+ `tests/unit/**`, `tests/component/**` or anywhere else.
305
381
  - AC annotations are not required in code; AC coverage is treated as indirect through TC coverage.
306
382
  - `QFAI:CON-API-*` in `tests/e2e/**` is not forbidden, but contract guarantee belongs to API tests.
307
383