qfai 1.10.0 → 1.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -4
- package/assets/init/.qfai/assistant/README.md +27 -0
- package/assets/init/.qfai/assistant/agents/completion-reviewer.md +16 -0
- package/assets/init/.qfai/assistant/agents/delivery-planner.md +8 -0
- package/assets/init/.qfai/assistant/agents/implementation-reviewer.md +11 -0
- package/assets/init/.qfai/assistant/agents/qa-gatekeeper.md +99 -14
- package/assets/init/.qfai/assistant/agents/test-design-analyst.md +3 -1
- package/assets/init/.qfai/assistant/catalog/test-layers-ci-lanes.md +60 -0
- package/assets/init/.qfai/assistant/catalog/test-layers.md +125 -49
- package/assets/init/.qfai/assistant/constitution/constitution.md +1 -1
- package/assets/init/.qfai/assistant/constitution/drift-protocol.md +60 -5
- package/assets/init/.qfai/assistant/constitution/shared-skill-delegation-baseline.md +177 -2
- package/assets/init/.qfai/assistant/manifest/agent-catalog.yml +137 -15
- package/assets/init/.qfai/assistant/manifest/agent-routing.yml +69 -3
- package/assets/init/.qfai/assistant/skills/qfai-atdd/SKILL.md +126 -69
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/credential-reuse.md +146 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/red-provenance.md +456 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/review-fix-rounds.md +128 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/scaffolding.md +29 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/shared-test-artifacts.md +96 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/stale-manifest.md +34 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md +29 -1
- package/assets/init/.qfai/assistant/skills/qfai-configure/SKILL.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/review-cycle-playbook.md +4 -0
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/summary.json +2 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/SKILL.md +50 -50
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/checkpoint-verification.md +136 -32
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/cross-spec-ownership.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/evidence-revision.md +222 -8
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/execution-ledger.md +110 -24
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/final-checklist.md +12 -3
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-not-observable.md +48 -6
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/review-artifact-layout.md +15 -2
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/round-evidence.md +19 -3
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/upstream-artifact-ordering.md +33 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/SKILL.md +7 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/review-cycle-playbook.md +4 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/spec-traceability-rules.md +21 -8
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/articles.md +1 -1
- package/assets/init/root/.github/workflows/qfai-tests.yml +318 -0
- package/assets/init/root/.github/workflows/qfai-validate.yml +327 -24
- package/dist/cli/index.cjs +12558 -9416
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.mjs +8070 -4913
- package/dist/cli/index.mjs.map +1 -1
- package/dist/index.cjs +3175 -1575
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +146 -9
- package/dist/index.d.ts +146 -9
- package/dist/index.mjs +3096 -1502
- package/dist/index.mjs.map +1 -1
- package/package.json +5 -1
package/README.md
CHANGED
|
@@ -121,13 +121,28 @@ npx qfai report
|
|
|
121
121
|
|
|
122
122
|
## ATDD annotation hard gate
|
|
123
123
|
|
|
124
|
-
`qfai validate` enforces spec-to-test traceability
|
|
124
|
+
`qfai validate` enforces spec-to-test traceability. `US` and `CON-API` obligations are routed by ID type;
|
|
125
|
+
a `TC` obligation is routed by the `Level` its spec declares for it.
|
|
125
126
|
|
|
126
127
|
- `tests/e2e/**`: annotate all covered user stories with concrete IDs such as `QFAI:SPEC-0001:US-0001`.
|
|
127
|
-
- `tests/integration/**`: annotate all covered test cases with concrete IDs such as `QFAI:SPEC-0001:TC-0001`.
|
|
128
128
|
- `tests/api/**`: annotate all covered API contracts with concrete IDs such as `QFAI:CON-API-0001`.
|
|
129
|
-
-
|
|
129
|
+
- Annotate a covered test case with a concrete ID such as `QFAI:SPEC-0001:TC-0001`, in the directory its declared `Level` names:
|
|
130
|
+
|
|
131
|
+
| `Level` | Annotated in |
|
|
132
|
+
| ----------------------------- | ---------------------- |
|
|
133
|
+
| `L1`/`Unit`, `L2`/`Component` | no ATDD annotation |
|
|
134
|
+
| `L3`/`Integration` | `tests/integration/**` |
|
|
135
|
+
| `L4`/`API` | `tests/api/**` |
|
|
136
|
+
| `L5`/`E2E` | `tests/e2e/**` |
|
|
137
|
+
| none declared, or unreadable | `tests/integration/**` |
|
|
138
|
+
|
|
139
|
+
- Unit and Component test cases carry **no** ATDD annotation obligation. They are gated by the
|
|
140
|
+
`tdd/test-list.md` ledger instead, so do not copy them into `tests/integration/**` to satisfy this gate.
|
|
141
|
+
- A `TC` annotation outside the directory its declared `Level` names is rejected. The rule is `Level`-relative,
|
|
142
|
+
not a blanket ban: a `TC` in `tests/api/**` is accepted only for a test case that declares `L4`/`API`, and in
|
|
143
|
+
`tests/e2e/**` only for `L5`/`E2E`.
|
|
130
144
|
- `AC` annotations are not required in code; AC coverage is treated as indirect through full `TC` coverage.
|
|
145
|
+
- These directories follow `paths.testsDir` from `qfai.config.yaml`; `tests/` above is the default.
|
|
131
146
|
|
|
132
147
|
## Operating model (skills-driven workflow)
|
|
133
148
|
|
|
@@ -145,7 +160,7 @@ The agent reads QFAI assets under `.qfai/assistant/` and produces or updates SDD
|
|
|
145
160
|
QFAI includes a small set of custom skills (stored under `.qfai/assistant/skills/`) designed to keep the workflow opinionated and repeatable.
|
|
146
161
|
|
|
147
162
|
- **qfai-configure**: Analyze the repository (language, frameworks, test layout, directory structure)
|
|
148
|
-
and
|
|
163
|
+
and adjust `qfai.config.yaml` accordingly (especially `testFileGlobs`).
|
|
149
164
|
Run this once right after `npx qfai init`, and re-run it when the repository structure changes.
|
|
150
165
|
- **qfai-discussion**: Run a unified structured discussion that produces and maintains the latest discussion pack
|
|
151
166
|
as 15 required markdown files under `.qfai/discussion/discussion-<ts>/`.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# QFAI assistant tree
|
|
2
|
+
|
|
3
|
+
This directory is the canonical source for QFAI's skills, agents, constitution,
|
|
4
|
+
manifest and catalog. `npx qfai init` writes it once and never removes it, and the
|
|
5
|
+
tool-specific integration directories are built from it.
|
|
6
|
+
|
|
7
|
+
## Canonical entrypoint
|
|
8
|
+
|
|
9
|
+
Every tool integration resolves through symlinks that point back here:
|
|
10
|
+
|
|
11
|
+
- .qfai/assistant/skills/ — skill documents
|
|
12
|
+
- .qfai/assistant/agents/ — agent definitions
|
|
13
|
+
|
|
14
|
+
These documents are the SSOT. Edit them here, not through the symlinks under
|
|
15
|
+
`.claude/`, `.agents/`, `.codex/` or `.github/`.
|
|
16
|
+
|
|
17
|
+
## Integration surface
|
|
18
|
+
|
|
19
|
+
`npx qfai init` creates the wrappers under those four directories. They are
|
|
20
|
+
generated, so re-running `npx qfai init` restores any that a checkout flattened or
|
|
21
|
+
a cleanup removed; nothing there needs to be edited by hand.
|
|
22
|
+
|
|
23
|
+
`npx qfai validate` checks that the wrappers still resolve to the documents above.
|
|
24
|
+
It reads this file to tell "init has run here and the surface was deleted" from
|
|
25
|
+
"init has never run here" — the two look identical from the integration
|
|
26
|
+
directories alone once every wrapper is gone, and only one of them is a
|
|
27
|
+
problem. Leave it in place.
|
|
@@ -24,6 +24,22 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
24
24
|
- .qfai/assistant/catalog/test-layers.md
|
|
25
25
|
- .qfai/specs/spec-\*/09_delta.md
|
|
26
26
|
- Validation evidence and gate results
|
|
27
|
+
- `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
|
|
28
|
+
- The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
|
|
29
|
+
or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
|
|
30
|
+
|
|
31
|
+
**Validate evidence is a completion-gate input, not an item-cycle one.** When
|
|
32
|
+
this role is routed inside an item cycle — `/qfai-atdd` stage gate P1c hands a
|
|
33
|
+
single row to `/qfai-implement` and that run's reviewers gate its checkpoint —
|
|
34
|
+
`.qfai/report/validate.log`, the coverage reports and runtime evidence are P5/P6
|
|
35
|
+
artifacts of the calling stage and do not exist yet. Requiring them there
|
|
36
|
+
stopped the first branch-1 row at `refactor`, which Phase Red does not
|
|
37
|
+
re-select, so the calling stage never reached P2. Judge the row's own
|
|
38
|
+
phase-authored evidence; the completion gate is where the rest is owed. **The
|
|
39
|
+
two inputs above are what makes that possible** — without the ledger and the
|
|
40
|
+
evidence home its `Layer` selects, this role cannot identify the artifact it is
|
|
41
|
+
being asked to judge, and falls into its own Stop condition ("Required evidence
|
|
42
|
+
... missing") on a correct branch-1 or branch-2 row.
|
|
27
43
|
|
|
28
44
|
## Deliverables
|
|
29
45
|
|
|
@@ -33,6 +33,14 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
33
33
|
- .qfai/specs/spec-\*/01_Spec.md
|
|
34
34
|
- `.qfai/specs/spec-*/tdd/test-list.md` — the execution ledger this role selects
|
|
35
35
|
the next item from and whose Red-Green-Refactor ordering it enforces
|
|
36
|
+
- **The document the row's obligation column points at.** Item scope is "is this
|
|
37
|
+
selector a sufficient slice of the obligation", and the obligation is not
|
|
38
|
+
always a `TC-*`: an `E2E` row owes `US-Refs` and an `API` row owes
|
|
39
|
+
`CON-API-Refs`. Without these the role has nothing to compare such a row
|
|
40
|
+
against and can only guess a PASS or stall the gate.
|
|
41
|
+
- `.qfai/specs/spec-*/06_Test-Cases.md` for a `TC-Refs` row
|
|
42
|
+
- `.qfai/specs/spec-*/02_User-stories.md` for a `US-Refs` row
|
|
43
|
+
- `.qfai/contracts/api/**` for a `CON-API-Refs` row
|
|
36
44
|
- .qfai/discussion/discussion-\*/04_Sources.md
|
|
37
45
|
- .qfai/discussion/discussion-\*/06_REQ.md
|
|
38
46
|
- .qfai/discussion/discussion-\*/11_OQ-Register.md
|
|
@@ -31,6 +31,17 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
31
31
|
- .github/instructions/principles.instructions.md
|
|
32
32
|
- Diff of changed files
|
|
33
33
|
- `.qfai/contracts/api/**` and `.qfai/contracts/db/**`
|
|
34
|
+
- `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
|
|
35
|
+
- The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
|
|
36
|
+
or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
|
|
37
|
+
|
|
38
|
+
**The last two are what the `Audited evidence hash` is computed over.** This
|
|
39
|
+
role records that hash itself, over the row's phase-authored fields — and those
|
|
40
|
+
live in an evidence file that is normally ignored, so the diff of changed files
|
|
41
|
+
does not contain them. Without the ledger and the evidence home the row's
|
|
42
|
+
`Layer` selects, this role cannot identify its own audit subject: the hash goes
|
|
43
|
+
missing and gate items 10-11 stop, or the orchestrator computes it instead,
|
|
44
|
+
which is the one thing the contract says must not happen.
|
|
34
45
|
|
|
35
46
|
## Deliverables
|
|
36
47
|
|
|
@@ -21,7 +21,9 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
21
21
|
|
|
22
22
|
## Ownership boundaries
|
|
23
23
|
|
|
24
|
-
- `delivery-planner` owns **item selection and item scope** — whether a ledger
|
|
24
|
+
- `delivery-planner` owns **item selection and item scope** — whether a ledger
|
|
25
|
+
row's selector is a sufficient slice of the obligation its `Layer` names
|
|
26
|
+
(`TC-Refs`, `US-Refs` or `CON-API-Refs`). Do not adjudicate item scope here; a PASS on observation
|
|
25
27
|
evidence is explicitly scoped to that observation and never widens or ratifies item scope. See `.qfai/assistant/skills/qfai-implement/SKILL.md#precedence-between-delivery-planner-and-qa-gatekeeper`.
|
|
26
28
|
- Refuse to evaluate RED/GREEN evidence while an unresolved `delivery-planner` scope REVISE is open on the same item.
|
|
27
29
|
|
|
@@ -32,7 +34,8 @@ the item owns, and nothing downstream re-asks: coverage is annotation presence
|
|
|
32
34
|
and the Depth Matrix counts case categories. A test that cannot fail otherwise
|
|
33
35
|
clears every gate.
|
|
34
36
|
|
|
35
|
-
Require an `Oracle proof` on each item
|
|
37
|
+
Require an `Oracle proof` on each item **at a GREEN or completion gate**, and
|
|
38
|
+
**reject** it when:
|
|
36
39
|
|
|
37
40
|
- the mutation is outside the code the item owns — breaking a shared helper
|
|
38
41
|
proves the helper is used, not that this test discriminates;
|
|
@@ -41,6 +44,14 @@ Require an `Oracle proof` on each item and **reject** it when:
|
|
|
41
44
|
- the failing output names a selector other than the row's;
|
|
42
45
|
- the recorded command differs from the `GREEN command`.
|
|
43
46
|
|
|
47
|
+
**At a RED observation the proof is a plan, and a plan is enough.** Branch 1's
|
|
48
|
+
RED is taken before any production behaviour exists, so there is nothing to
|
|
49
|
+
mutate: the item names the predicate it will break and the command it will run.
|
|
50
|
+
Requiring a demonstrated mutation there made a correct observed RED unable to
|
|
51
|
+
pass P1b and so unable to reach Phase Green — the phase that builds the code the
|
|
52
|
+
mutation needs. Judge the plan for whether it names this row's predicate and
|
|
53
|
+
selector; judge the demonstration once the behaviour exists.
|
|
54
|
+
|
|
44
55
|
`equivalent-mutant` is acceptable **only** when the named contract clause is
|
|
45
56
|
genuinely weaker than the obligation. It is an upstream gap: route it as an
|
|
46
57
|
advisory / Change Request, do not send the implementer to strengthen an
|
|
@@ -78,12 +89,46 @@ test:
|
|
|
78
89
|
- "the suite is green" in place of the row's own GREEN.
|
|
79
90
|
|
|
80
91
|
The one legitimate absence is the _RED not observable_ path: the obligation is
|
|
81
|
-
already satisfied by
|
|
82
|
-
require `Satisfied-by`, `Falsifiability command` and
|
|
83
|
-
instead — never both forms, never neither.
|
|
92
|
+
already satisfied by something already in the tree, so the correct test passes
|
|
93
|
+
first run. Then require `Satisfied-by`, `Falsifiability command` and
|
|
94
|
+
`Falsifiability result` instead — never both forms, never neither.
|
|
95
|
+
|
|
96
|
+
**On an `E2E` / `API` / `Integration` row, `Satisfied-by` need not be a sibling `TDD-NNNN`.** A
|
|
97
|
+
production **path and symbol** is equally valid there and is the normal answer
|
|
98
|
+
for a row whose surface no ledger row owns; rejecting it sends every such row to
|
|
99
|
+
`exception`, the terminal state the path exists to avoid. Judge it on whether it
|
|
100
|
+
answers "what would I mutate to falsify this row".
|
|
101
|
+
|
|
102
|
+
**A commit id alone does not answer it — REVISE.** A commit that touched
|
|
103
|
+
several routes and a helper names no single predicate, so the ownership check
|
|
104
|
+
below has no boundary to apply and would accept a mutation anywhere inside it.
|
|
105
|
+
The producer contract requires the symbol for this reason
|
|
106
|
+
(`../skills/qfai-atdd/references/red-provenance.md#the-three-branches-must`); a
|
|
107
|
+
commit recorded **alongside** the path and symbol is provenance and is fine.
|
|
108
|
+
|
|
109
|
+
**And the mutation may touch it.** The Oracle Strength Check rejects a mutation
|
|
110
|
+
outside the code the item owns, which on an `E2E` / `API` / `Integration` row is every
|
|
111
|
+
production predicate there is — the same sentence above says no ledger row owns
|
|
112
|
+
that surface. Applied literally, no branch-2 row could ever produce
|
|
113
|
+
falsifiability evidence that passes. On a handed-over row, **the predicate
|
|
114
|
+
`Satisfied-by` names is the owned code** for this check; anything else is still
|
|
115
|
+
out of bounds.
|
|
116
|
+
|
|
117
|
+
**On any other row the sibling row is still required** — production code
|
|
118
|
+
no ledger row owns is the anomaly case there, not a substitute. See
|
|
84
119
|
`.qfai/assistant/skills/qfai-implement/references/red-not-observable.md` and
|
|
85
120
|
`.qfai/assistant/skills/qfai-implement/references/red-admissibility.md`.
|
|
86
121
|
|
|
122
|
+
**A `Layer = E2E` / `Layer = API` row from `/qfai-atdd` is judged the same
|
|
123
|
+
way.** Its journey is often written after the surface the same cycle built, so
|
|
124
|
+
the falsifiability form is the expected evidence rather than a concession —
|
|
125
|
+
accept it, with the mutated predicate being one the journey actually asserts
|
|
126
|
+
on. What is **not** acceptable is the third outcome appearing by default: a row
|
|
127
|
+
routed to `exception` whose `DR-*` says only that the surface came first has
|
|
128
|
+
not shown that either branch was unavailable, and that is a REVISE. See
|
|
129
|
+
`.qfai/assistant/skills/qfai-atdd/SKILL.md#red-provenance-for-an-atdd-owned-row-must`
|
|
130
|
+
and `.qfai/assistant/skills/qfai-implement/references/execution-ledger.md#atdd-owned-rows`.
|
|
131
|
+
|
|
87
132
|
Verdict scope: a PASS covers the observation for that round and nothing else. It
|
|
88
133
|
does not ratify item scope and does not clear the completion gate.
|
|
89
134
|
|
|
@@ -91,7 +136,11 @@ does not ratify item scope and does not clear the completion gate.
|
|
|
91
136
|
|
|
92
137
|
In addition to traceability-based coverage (US/TC/CON-API existence), verify the **depth** of test cases:
|
|
93
138
|
|
|
94
|
-
- Confirm a Coverage Depth Matrix exists (produced by `test-design-analyst`).
|
|
139
|
+
- Confirm a Coverage Depth Matrix exists at `.qfai/evidence/coverage-depth-<spec-id>.md` (produced by `test-design-analyst`).
|
|
140
|
+
Missing matrix: REVISE from the ATDD review cycle onward; on an SDD review cycle record it as a finding. See the scope note.
|
|
141
|
+
A matrix that exists only inside `.qfai/evidence/atdd-<spec-id>.md` is a **missing** matrix: that file is ignored by the
|
|
142
|
+
managed `.gitignore` block, so neither it nor the justification for any `❌` reaches a commit, and the "unjustified"
|
|
143
|
+
judgement cannot be re-made by anyone reading the repository.
|
|
95
144
|
- Check that each US/TC has test cases for at minimum: normal path AND error/failure path.
|
|
96
145
|
- Flag any US/TC that has only normal-path test cases as a coverage gap.
|
|
97
146
|
- Reference: `.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`
|
|
@@ -105,8 +154,8 @@ In addition to traceability-based coverage (US/TC/CON-API existence), verify the
|
|
|
105
154
|
The Coverage Depth Matrix is an **ATDD-stage artifact**: it is defined in
|
|
106
155
|
`.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`, listed as an ATDD
|
|
107
156
|
Mandatory Output,
|
|
108
|
-
and written
|
|
109
|
-
ships a section for it, so:
|
|
157
|
+
and written to `.qfai/evidence/coverage-depth-<spec-id>.md` — a committed path, unlike the rest of
|
|
158
|
+
`.qfai/evidence/**`. `qfai-sdd` neither defines its layout nor ships a section for it, so:
|
|
110
159
|
|
|
111
160
|
- Apply this check from the **ATDD review cycle onward**, where
|
|
112
161
|
`.qfai/assistant/skills/qfai-atdd/SKILL.md` lists
|
|
@@ -124,14 +173,47 @@ ships a section for it, so:
|
|
|
124
173
|
- .qfai/assistant/catalog/test-layers.md
|
|
125
174
|
- .qfai/specs/spec-\*/09_delta.md
|
|
126
175
|
- `.qfai/specs/spec-*/tdd/test-list.md` — the ledger row under review
|
|
127
|
-
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
176
|
+
- **The per-item RED/GREEN evidence for the row under review — in the file its
|
|
177
|
+
`Layer` owns, and only that one.** `.qfai/evidence/atdd-<spec-id>.md`, under
|
|
178
|
+
`## Ledger rows advanced`, for a `Layer = E2E` / `Layer = API` / `Layer = Integration` row;
|
|
179
|
+
`.qfai/evidence/implement-<spec-id>.md` for every other row. Both are listed
|
|
180
|
+
because the Stop condition below ("target artifacts are missing") is not
|
|
181
|
+
checkable against an artifact this role was never told to open — but
|
|
182
|
+
requiring **both** makes that condition fire on a spec that legitimately has
|
|
183
|
+
one: a Unit-only spec never ran `/qfai-atdd`, and a spec whose rows are all
|
|
184
|
+
`E2E` / `API` / `Integration` has no implement file. Either way the gate would stop before
|
|
185
|
+
reading the evidence that does exist.
|
|
186
|
+
**The three below are required at a completion gate, not at a RED/GREEN
|
|
187
|
+
observation.** `/qfai-atdd` routes this role as blocking at stage gate P1b, and
|
|
188
|
+
validate output, coverage reports and runtime evidence are first produced at its
|
|
189
|
+
P5 and P6 — so requiring them there stopped a fresh run that had a perfectly
|
|
190
|
+
good RED pair, on artifacts its own ordering says cannot exist yet. At an
|
|
191
|
+
observation gate the row's own evidence above is the whole input.
|
|
192
|
+
|
|
193
|
+
- The scoped validate JSON for the spec under review — `validate.spec-<id>.json`
|
|
194
|
+
**beside the configured `output.validateJsonPath`**, not under a fixed
|
|
195
|
+
`.qfai/report/` — or the `run-*/` directory of the run that produced it, under
|
|
196
|
+
the configured `paths.outDir`. Read both from `qfai.config.yaml` the way the
|
|
197
|
+
SDD and discussion contracts do. A project that moved either output writes its
|
|
198
|
+
evidence where it said to, and looking for it at the default path reported a
|
|
199
|
+
missing artifact and stopped completion on a validate run that had succeeded
|
|
200
|
+
and left everything it owed. **Not `validate.log`**: it and the run-log pointer are shared by every
|
|
201
|
+
run, scoped or not, and nothing serializes them, so a sibling stage validating
|
|
202
|
+
at the same time overwrites what this one wrote — a failing run followed by a
|
|
203
|
+
sibling's success reads as this spec's PASS
|
|
132
204
|
- `.qfai/report/specs-coverage/spec-*.md`
|
|
133
205
|
- Runtime evidence and prototyping evidence artifacts
|
|
134
206
|
|
|
207
|
+
**Branch 3 gets its own verdict.** The observation gate admits an observed RED
|
|
208
|
+
or a falsifiability trio and calls anything else "never neither" — but a genuine
|
|
209
|
+
branch-3 row _has_ neither, by the finding that put it there. Judged by the two
|
|
210
|
+
forms it can only be REVISE, and skipping the gate leaves the stage's completion
|
|
211
|
+
condition unmet, so the row could not close either way. Judge these on their own
|
|
212
|
+
terms: a `DR-*` that records **what could not be observed and why each branch was
|
|
213
|
+
unavailable**, PASS or REVISE on that. A missing `DR-*`, or one that names no
|
|
214
|
+
unavailability, is still REVISE — this is a third form of evidence, not an
|
|
215
|
+
exemption from having any.
|
|
216
|
+
|
|
135
217
|
## Deliverables
|
|
136
218
|
|
|
137
219
|
- Gate decision (PASS / REVISE) with rationale
|
|
@@ -140,7 +222,10 @@ ships a section for it, so:
|
|
|
140
222
|
|
|
141
223
|
## Stop conditions
|
|
142
224
|
|
|
143
|
-
- Required evidence, governing specs, or target artifacts are missing
|
|
225
|
+
- Required evidence, governing specs, or target artifacts are missing — judged
|
|
226
|
+
against what the invoking phase requires, per the note above the last three
|
|
227
|
+
inputs. At a RED/GREEN observation that is the row's own evidence; at a
|
|
228
|
+
completion gate it is all of them.
|
|
144
229
|
- The request requires implementation or file editing instead of independent review.
|
|
145
230
|
- The issue falls outside this review domain and must be rerouted to another specialist first.
|
|
146
231
|
|
|
@@ -58,7 +58,9 @@ At both stages: when business rules (BR-\*) exist, verify each BR has at least o
|
|
|
58
58
|
- Coverage plan and layer ownership
|
|
59
59
|
- Test-case quality and traceability findings
|
|
60
60
|
- **Coverage Depth Matrix** (per spec, using the template in the depth checklist reference).
|
|
61
|
-
Destination: `.qfai/evidence/
|
|
61
|
+
Destination: `.qfai/evidence/coverage-depth-<spec-id>.md` from the ATDD stage onward — its own
|
|
62
|
+
file, because that path is committed while the rest of `.qfai/evidence/**` is ignored, and the
|
|
63
|
+
justification behind each `❌` is the input `qa-gatekeeper` reads. During SDD there is
|
|
62
64
|
no evidence artifact that holds it, so report depth gaps as findings instead of producing the
|
|
63
65
|
matrix format.
|
|
64
66
|
- Volume estimate and risk notes
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Test layers to CI lanes
|
|
2
|
+
|
|
3
|
+
A crosswalk between the test layers in [`test-layers.md`](./test-layers.md) and the CI lanes
|
|
4
|
+
the shipped workflows run. It exists so that "which job runs my L3 tests" has one answer a
|
|
5
|
+
reader can find, instead of being rediscovered from a workflow file each time.
|
|
6
|
+
|
|
7
|
+
**The layer policy loader does not read this file.** (Written without a hyphen on purpose: the
|
|
8
|
+
loader extracts anything matching `layer-<word>` from the file it parses, and while it does not
|
|
9
|
+
parse this one, a catalog-directory file carrying such a token is a hazard waiting for the day
|
|
10
|
+
someone widens the loader to scan the directory.) It resolves `catalog/test-layers.md` by
|
|
11
|
+
exact path and reads nothing else in this directory, so nothing written here can widen or
|
|
12
|
+
narrow the layer vocabulary. That is deliberate: this document is a map, and a map that could
|
|
13
|
+
change the territory would be a second source of truth for something that already has one. If
|
|
14
|
+
you need to change what layers exist, change `test-layers.md` — this file follows.
|
|
15
|
+
|
|
16
|
+
## The mapping
|
|
17
|
+
|
|
18
|
+
| Layer | What it exercises | Shipped CI lane |
|
|
19
|
+
| -------------- | --------------------------------------------- | --------------- |
|
|
20
|
+
| L1 Unit | one module, no I/O | `unit` |
|
|
21
|
+
| L2 Component | one component and its immediate collaborators | `component` |
|
|
22
|
+
| L3 Integration | several modules across a real boundary | `integration` |
|
|
23
|
+
| L4 API | a running interface, contract-first | `api` |
|
|
24
|
+
| L5 E2E | the product as a user reaches it | `e2e` |
|
|
25
|
+
|
|
26
|
+
Two lanes in the shipped workflow are not layers and have no row above:
|
|
27
|
+
|
|
28
|
+
- **`detection`** decides which of the lanes above need to run for a given change, and
|
|
29
|
+
publishes that decision as job outputs. It is infrastructure, not a test level.
|
|
30
|
+
- **`verdict`** aggregates the lanes into the single status a branch-protection rule can name.
|
|
31
|
+
It runs unconditionally so that a run where every lane was skipped is still distinguishable
|
|
32
|
+
from a run where nothing was verified.
|
|
33
|
+
|
|
34
|
+
## Lane names are a project's choice
|
|
35
|
+
|
|
36
|
+
The names in the third column are the ones the shipped workflow uses. They are not a contract:
|
|
37
|
+
a project that calls its integration lane `service-tests` is not violating anything, and this
|
|
38
|
+
table is not a rename list. What matters is that each layer a project uses has some lane that
|
|
39
|
+
runs it, and that the lane's name stays put once branch protection refers to it — a check name
|
|
40
|
+
is a repository setting, and renaming one silently makes a required check unsatisfiable.
|
|
41
|
+
|
|
42
|
+
## Per-level routing is not enforced
|
|
43
|
+
|
|
44
|
+
`test-layers.md` marks per-level annotation routing as a target state and says plainly: not
|
|
45
|
+
enforced, do not follow yet. This document does not change that and does not activate it. The
|
|
46
|
+
live traceability gate reads one directory for annotations, and it is the gate — not this
|
|
47
|
+
table — that decides where an annotation counts.
|
|
48
|
+
|
|
49
|
+
So: read this file to find out which lane runs a layer. Do not read it as instructions about
|
|
50
|
+
where anything belongs in the tree. That question is settled by the gate's own scope, and by
|
|
51
|
+
`test-layers.md` for the vocabulary.
|
|
52
|
+
|
|
53
|
+
## Keeping this file honest
|
|
54
|
+
|
|
55
|
+
- Every layer code here also appears in `test-layers.md`'s crosswalk. If you add a layer there
|
|
56
|
+
and not here, this table is incomplete; if you add one here and not there, you have invented
|
|
57
|
+
a layer nothing enforces.
|
|
58
|
+
- The lane column describes the shipped workflow. If the shipped workflow's job set changes,
|
|
59
|
+
this table is stale, and stale is worse than absent for a document whose only job is to save
|
|
60
|
+
someone a lookup.
|
|
@@ -2,6 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
This document is the SSOT for ATDD test-layer semantics and completion gates.
|
|
4
4
|
|
|
5
|
+
For which CI lane runs which layer, see the sibling map
|
|
6
|
+
[`test-layers-ci-lanes.md`](./test-layers-ci-lanes.md). That file is a crosswalk only — the
|
|
7
|
+
policy loader reads this file and not that one, so nothing there can change the
|
|
8
|
+
vocabulary declared below.
|
|
9
|
+
|
|
5
10
|
## Layer vocabulary crosswalk (normative)
|
|
6
11
|
|
|
7
12
|
qfai spells the same layer four ways across shipped artifacts. This table is
|
|
@@ -31,17 +36,21 @@ Rules:
|
|
|
31
36
|
- L1 and L2 have no mandated directory: unit and component tests live wherever
|
|
32
37
|
the project's own convention puts them. Only L3-L5 are directory-pinned, and
|
|
33
38
|
only those directories are scanned by the ATDD traceability rules.
|
|
34
|
-
- **A `TC-*` row's `Level` is L1-L3.**
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
`<testsDir>/
|
|
39
|
-
`<testsDir>/
|
|
40
|
-
`<testsDir>/
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
39
|
+
- **A `TC-*` row's `Level` is L1-L3.** Of those, only L3 owes an ATDD
|
|
40
|
+
annotation: L1 and L2 have no mandated directory, so `QFAI-ATDD-112` does not
|
|
41
|
+
apply to them and `QFAI-ATDD-117` (`info`) names them instead. Their gate is
|
|
42
|
+
`tdd/test-list.md` / `TDDLIST_TC_NOT_COVERED`, under `/qfai-implement`.
|
|
43
|
+
`US-*` is answered from `<testsDir>/e2e/**` (`QFAI-ATDD-111`), `CON-API-*`
|
|
44
|
+
from `<testsDir>/api/**` (`QFAI-ATDD-113`) and `CON-DB-*` from
|
|
45
|
+
`<testsDir>/integration/**` (`QFAI-ATDD-115`) — those three are fixed by the
|
|
46
|
+
ID type. A `TC-*` is answered from the directory **its own declared `Level`**
|
|
47
|
+
names, which for a correctly filed row is `<testsDir>/integration/**`; see
|
|
48
|
+
[Annotation routing](#annotation-routing) for the full table and the
|
|
49
|
+
misplacement rules.
|
|
50
|
+
L4's goal is `CON-API-*` and L5's is `US-*` (see the layer definitions
|
|
51
|
+
below), so an oracle that lands at L4 or L5 means the obligation is misfiled:
|
|
52
|
+
record it as `CON-API-*` or `US-*` rather than leaving a `TC-*` row at a
|
|
53
|
+
layer whose goal is another ID type.
|
|
45
54
|
- The two code-side word lists (`tddHelpers.ts#UNIT_COMPONENT_LAYERS` /
|
|
46
55
|
`#NON_COVERAGE_LAYERS`) accept both the code and the word form for the same
|
|
47
56
|
layer; they MUST stay in step with this table.
|
|
@@ -73,7 +82,7 @@ writer targets a declared layer instead.
|
|
|
73
82
|
- Scope: pure decision logic — a single module's inputs and return values, with
|
|
74
83
|
no port collaboration and no real infrastructure.
|
|
75
84
|
- Goal: verify `TC-*` obligations whose oracle observes inputs and outputs only.
|
|
76
|
-
-
|
|
85
|
+
- Convention: `tests/unit/**`. L1 has no mandated directory and owes no ATDD annotation — see the crosswalk and "Unit and Component owe no ATDD annotation".
|
|
77
86
|
|
|
78
87
|
### L2 Component
|
|
79
88
|
|
|
@@ -81,7 +90,7 @@ writer targets a declared layer instead.
|
|
|
81
90
|
with no real infrastructure.
|
|
82
91
|
- Goal: verify `TC-*` obligations whose oracle observes the interaction with a
|
|
83
92
|
port rather than infrastructure state.
|
|
84
|
-
-
|
|
93
|
+
- Convention: `tests/component/**`. L2 has no mandated directory and owes no ATDD annotation — see the crosswalk and "Unit and Component owe no ATDD annotation".
|
|
85
94
|
|
|
86
95
|
### L3 Integration
|
|
87
96
|
|
|
@@ -137,8 +146,23 @@ falsifying-oracle rule:
|
|
|
137
146
|
observable at two layers, it is two obligations: write one TC per oracle and
|
|
138
147
|
give each its own `Level`.
|
|
139
148
|
|
|
140
|
-
- A multi-valued `Level` cell (`L3/L5`) is **illegal**.
|
|
141
|
-
no
|
|
149
|
+
- A multi-valued `Level` cell (`L3/L5`, `L1/L2`, `L1, L3`) is **illegal**. It
|
|
150
|
+
matches no entry in the crosswalk, so no rule can read a layer out of it.
|
|
151
|
+
- **What a reader does with one: split the row.** One TC per oracle, each with
|
|
152
|
+
its own single `Level`. Nothing else is a fix — in particular, do not record
|
|
153
|
+
a normalization that "drops one half": no tool performs one, and a note
|
|
154
|
+
saying an `L3/L5` row "normalizes to `L3`" is a claim about a value that only
|
|
155
|
+
`06_Test-Cases.md` can make.
|
|
156
|
+
- **What the validators do with one, until it is split.** They neither guess
|
|
157
|
+
nor let it through:
|
|
158
|
+
- `QFAI-ATDD-112` routes the TC to the same place a TC with no declared
|
|
159
|
+
`Level` goes — `<testsDir>/integration/**` — and keeps the obligation.
|
|
160
|
+
Unreadable is deliberately not "excused": if a cell qfai cannot read
|
|
161
|
+
discharged the obligation, `L1/L2` would be a one-keystroke way to delete
|
|
162
|
+
any TC from the gate. That default is where the obligation is _reported_,
|
|
163
|
+
not where the obligation _belongs_.
|
|
164
|
+
- `TDDLIST_UNKNOWN_LEVEL` (`warning`) names the cell, and the TC stays a
|
|
165
|
+
coverage target, so `tdd/test-list.md` still owes it a row.
|
|
142
166
|
- If splitting is genuinely impossible, escalate through the Drift Protocol
|
|
143
167
|
rather than inventing a combined value.
|
|
144
168
|
|
|
@@ -158,15 +182,43 @@ L4" applies only when the service-boundary values are themselves what the
|
|
|
158
182
|
parent BR owns. Inverting this is what collapses a designed pyramid into an
|
|
159
183
|
all-integration suite.
|
|
160
184
|
|
|
161
|
-
### Annotation routing
|
|
162
|
-
|
|
163
|
-
The derived `Level` records which oracle owns the obligation
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
(`QFAI-ATDD-111`)
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
185
|
+
### Annotation routing
|
|
186
|
+
|
|
187
|
+
The derived `Level` records which oracle owns the obligation, and the
|
|
188
|
+
[ATDD annotation hard gate](#atdd-annotation-hard-gate) routes each obligation
|
|
189
|
+
ID to exactly one directory. `US-*` is answered from `tests/e2e/**`
|
|
190
|
+
(`QFAI-ATDD-111`) and `CON-API-*` from `tests/api/**` (`QFAI-ATDD-113`); those
|
|
191
|
+
two are fixed by the ID type. A `TC-*` is answered from the directory **its own
|
|
192
|
+
declared `Level`** names (`QFAI-ATDD-112`):
|
|
193
|
+
|
|
194
|
+
| `Level` | Answered from |
|
|
195
|
+
| ----------------------------- | ------------------------------ |
|
|
196
|
+
| `L1`/`Unit` | no ATDD obligation |
|
|
197
|
+
| `L2`/`Component` | no ATDD obligation |
|
|
198
|
+
| `L3`/`Integration` | `tests/integration/**` |
|
|
199
|
+
| `L4`/`API` | `tests/api/**` (note) |
|
|
200
|
+
| `L5`/`E2E` | `tests/e2e/**` (note) |
|
|
201
|
+
| none declared | `tests/integration/**` |
|
|
202
|
+
| anything else — typo, `L3/L5` | `tests/integration/**` (note2) |
|
|
203
|
+
|
|
204
|
+
**(note)** A `TC-*` **should not be** at L4 or L5 — the first bullet below says
|
|
205
|
+
why and what to do instead. The gate routes it there rather than rejecting it so
|
|
206
|
+
a misfiled row is reported once, by the rule that names the real cause, instead
|
|
207
|
+
of twice as "uncovered in integration" and "forbidden in api".
|
|
208
|
+
|
|
209
|
+
**(note2)** A `Level` the crosswalk does not list — a typo, a project's own
|
|
210
|
+
word, or the illegal multi-valued cell — falls to the same default as an
|
|
211
|
+
undeclared one, and keeps its obligation. The default is the conservative
|
|
212
|
+
answer to a cell qfai cannot read, never a supported spelling: fix the cell
|
|
213
|
+
(see [Obligation spanning more than one layer](#obligation-spanning-more-than-one-layer)).
|
|
214
|
+
`TDDLIST_UNKNOWN_LEVEL` (`warning`) names such a cell on the ledger side.
|
|
215
|
+
|
|
216
|
+
Exactly one directory, never two: an annotation outside the one its `Level`
|
|
217
|
+
names is both uncovered and rejected (`QFAI-ATDD-121` / `QFAI-ATDD-122` /
|
|
218
|
+
`QFAI-ATDD-123`), and
|
|
219
|
+
the rejection is symmetric — an annotation left in `tests/integration/**` after
|
|
220
|
+
its TC moved to `L4`/`L5` is rejected the same way an early one in
|
|
221
|
+
`tests/api/**` is. Two consequences bind every `TC-*` row:
|
|
170
222
|
|
|
171
223
|
- **A `TC-*` row's `Level` stays within L1–L3.** L4's goal is `CON-API-*` and
|
|
172
224
|
L5's goal is `US-*` (see the layer definitions above), so an oracle that
|
|
@@ -185,14 +237,15 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
|
|
|
185
237
|
parent BR keeps a spec-side obligation, split the row instead: keep a `TC-*`
|
|
186
238
|
for the part the BR owns — by step 2 that part derives to L1–L3 — and re-file
|
|
187
239
|
only the boundary assertion as `CON-API-*` / `US-*`.
|
|
188
|
-
- **An L1/L2 `Level`
|
|
189
|
-
change the `Level`.** The two are independent: the `Level` records what
|
|
190
|
-
oracle observes and is derived before any test exists, while
|
|
191
|
-
`QFAI
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
240
|
+
- **An L1/L2 `Level` carries no `QFAI-ATDD-112` obligation, and the gate does
|
|
241
|
+
not change the `Level`.** The two are independent: the `Level` records what
|
|
242
|
+
the oracle observes and is derived before any test exists, while
|
|
243
|
+
`QFAI-ATDD-112` asks only about the layers ATDD owns. Never rewrite a derived
|
|
244
|
+
L1/L2 to L3 to make a gate quieter — that would make the recorded oracle
|
|
245
|
+
depend on implementation order and hide the unit/component work
|
|
246
|
+
`/qfai-implement` selects. An L1/L2 row's obligation is discharged through
|
|
247
|
+
`tdd/test-list.md` and `TDDLIST_TC_NOT_COVERED`, not through an annotation in
|
|
248
|
+
a directory ATDD scans.
|
|
196
249
|
|
|
197
250
|
## TestKind resolution (single source)
|
|
198
251
|
|
|
@@ -241,11 +294,21 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
|
|
|
241
294
|
- Every `TC-*` in specs must be referenced at least once from the directory
|
|
242
295
|
its declared `Level` routes to: L3/Integration -> `tests/integration/**`,
|
|
243
296
|
L4/API -> `tests/api/**`, L5/E2E -> `tests/e2e/**`. A TC with no declared
|
|
244
|
-
`Level` defaults to `tests/integration/**`.
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
297
|
+
`Level` defaults to `tests/integration/**`. **L1/Unit and L2/Component owe
|
|
298
|
+
no reference at all** — see "Unit and Component owe no ATDD annotation"
|
|
299
|
+
below. This is what `QFAI-ATDD-112` checks.
|
|
300
|
+
- Use `QFAI:SPEC-XXXX:TC-YYYY` annotations.
|
|
301
|
+
- A `TC-*` annotation outside the directory its declared `Level` names is
|
|
302
|
+
rejected (`QFAI-ATDD-121` / `QFAI-ATDD-122` / `QFAI-ATDD-123`). The rule is
|
|
303
|
+
`Level`-relative,
|
|
304
|
+
not a blanket ban: a `TC-*` in `tests/api/**` is rejected **unless** that TC
|
|
305
|
+
declares `L4`/`API`, and in `tests/e2e/**` unless it declares `L5`/`E2E` —
|
|
306
|
+
an annotation matching its own `Level` is what discharges the obligation
|
|
307
|
+
there. A `TC-*` should not be at L4 or L5 in the first place (see
|
|
308
|
+
"Annotation routing"): re-file that obligation as `CON-API-*` or `US-*`.
|
|
309
|
+
But while the row exists at that `Level`, its annotation belongs in the one
|
|
310
|
+
directory the `Level` names, and putting it anywhere else leaves the TC
|
|
311
|
+
uncovered as well as forbidden.
|
|
249
312
|
- Every declared `CON-DB-*` must be referenced at least once from
|
|
250
313
|
`tests/integration/**` (`QFAI-ATDD-115`). Use `QFAI:CON-DB-XXXX`
|
|
251
314
|
annotations. L3 owns this because a DB contract is only exercised against
|
|
@@ -255,16 +318,27 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
|
|
|
255
318
|
current slice defers with a `-- x-qfai-status: planned` comment line,
|
|
256
319
|
reported at `info` by `QFAI-ATDD-116` so the deferral stays visible.
|
|
257
320
|
|
|
258
|
-
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
-
|
|
264
|
-
`
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
321
|
+
- **Unit and Component owe no ATDD annotation.** A `TC-*` whose declared
|
|
322
|
+
`Level` is L1 or L2 is outside `QFAI-ATDD-112` entirely: it is not required
|
|
323
|
+
in any directory, and an annotation for it inside a scanned directory is not
|
|
324
|
+
a violation either. `QFAI-ATDD-117` (`info`) names the excluded TCs on every
|
|
325
|
+
run so the exclusion is visible rather than silent.
|
|
326
|
+
- This is the only reading consistent with the rest of the package.
|
|
327
|
+
`qfai-atdd/SKILL.md` puts Unit and Component out of its scope, and the
|
|
328
|
+
crosswalk above gives L1/L2 no mandated directory — only L3-L5 are
|
|
329
|
+
directory-pinned and only those three roots are scanned.
|
|
330
|
+
- Previously L1/L2 fell through to `tests/integration/**` — the fallback for
|
|
331
|
+
a spec with no `Level` column at all — so every declared Unit and Component
|
|
332
|
+
TC was an `error` demanding an annotation in a directory this file says is
|
|
333
|
+
not its home. `QFAI-WAIVER-002` refuses waivers on `error` rules, so a
|
|
334
|
+
project that filed unit tests where L1's own entry says to had no exit, and
|
|
335
|
+
the only validator-clean path was duplicating every annotation into
|
|
336
|
+
`tests/integration/**` — the all-integration collapse named under
|
|
337
|
+
Anti-patterns below.
|
|
338
|
+
- **They are still gated, by the other stage.** Every coverage-target `TC-*`
|
|
339
|
+
owes a `tdd/test-list.md` row, and `TDDLIST_TC_NOT_COVERED` (`error`)
|
|
340
|
+
reports a missing one. L1/L2 belong to `/qfai-implement`, which is the
|
|
341
|
+
stage that writes unit and component tests.
|
|
268
342
|
- API obligations:
|
|
269
343
|
- Every declared `CON-API-*` must be referenced at least once from `tests/api/**`.
|
|
270
344
|
- Use `QFAI:CON-API-XXXX` annotations.
|
|
@@ -300,8 +374,10 @@ routes by obligation ID: `US-*` is answered from `tests/e2e/**`
|
|
|
300
374
|
filed early into `tests/api/**`.
|
|
301
375
|
- Unknown references (`US/TC/CON-API` not declared) are errors.
|
|
302
376
|
- A `TC-*` annotation outside the directory its `Level` routes to is a
|
|
303
|
-
misplacement, whichever directory it lands in
|
|
304
|
-
|
|
377
|
+
misplacement, whichever directory it lands in. **This applies to L3-L5 only.**
|
|
378
|
+
L1 and L2 route nowhere — they carry no ATDD annotation obligation at all — so
|
|
379
|
+
an annotation for one is neither required nor misplaced, in
|
|
380
|
+
`tests/unit/**`, `tests/component/**` or anywhere else.
|
|
305
381
|
- AC annotations are not required in code; AC coverage is treated as indirect through TC coverage.
|
|
306
382
|
- `QFAI:CON-API-*` in `tests/e2e/**` is not forbidden, but contract guarantee belongs to API tests.
|
|
307
383
|
|