qfai 1.9.2 → 1.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +67 -6
- package/assets/init/.qfai/assistant/README.md +27 -0
- package/assets/init/.qfai/assistant/agents/acceptance-test-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/backend-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/completion-reviewer.md +29 -2
- package/assets/init/.qfai/assistant/agents/delivery-planner.md +17 -0
- package/assets/init/.qfai/assistant/agents/frontend-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/implementation-reviewer.md +17 -0
- package/assets/init/.qfai/assistant/agents/orchestrator.md +2 -2
- package/assets/init/.qfai/assistant/agents/qa-gatekeeper.md +183 -5
- package/assets/init/.qfai/assistant/agents/test-design-analyst.md +22 -3
- package/assets/init/.qfai/assistant/catalog/cli-ux-guidelines.md +2 -2
- package/assets/init/.qfai/assistant/catalog/spec_required_files.json +2 -1
- package/assets/init/.qfai/assistant/catalog/test-layers-ci-lanes.md +60 -0
- package/assets/init/.qfai/assistant/catalog/test-layers.md +430 -13
- package/assets/init/.qfai/assistant/catalog/worklog-entry.schema.md +165 -0
- package/assets/init/.qfai/assistant/constitution/communication.md +1 -1
- package/assets/init/.qfai/assistant/constitution/constitution.md +1 -1
- package/assets/init/.qfai/assistant/constitution/drift-protocol.md +359 -10
- package/assets/init/.qfai/assistant/constitution/quality.md +35 -5
- package/assets/init/.qfai/assistant/constitution/requirements-decomposition.md +37 -0
- package/assets/init/.qfai/assistant/constitution/shared-skill-delegation-baseline.md +419 -8
- package/assets/init/.qfai/assistant/constitution/shared-skill-operating-baseline.md +122 -5
- package/assets/init/.qfai/assistant/constitution/workflow.md +53 -7
- package/assets/init/.qfai/assistant/manifest/agent-catalog.yml +439 -946
- package/assets/init/.qfai/assistant/manifest/agent-routing.yml +116 -4
- package/assets/init/.qfai/assistant/manifest/review-profiles.yml +9 -0
- package/assets/init/.qfai/assistant/process/migrations/v1.4.27-atdd-alignment.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-atdd/SKILL.md +150 -53
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/credential-reuse.md +146 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/red-provenance.md +456 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/review-fix-rounds.md +128 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/scaffolding.md +29 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/shared-test-artifacts.md +96 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/stale-manifest.md +34 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md +52 -5
- package/assets/init/.qfai/assistant/skills/qfai-configure/SKILL.md +16 -8
- package/assets/init/.qfai/assistant/skills/qfai-discussion/SKILL.md +8 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/design-md-brand-catalog.md +2 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/discussion-completion-matrix.md +17 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/rcp_footer.md +10 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/review-cycle-playbook.md +5 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui-bearing-playbook.md +4 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux/review_audit_playbook.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux/trend_scan_playbook.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux_best_practices.md +17 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/01_Context.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/02_Inception-Deck.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/03_Story-Workshop.md +11 -3
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/05_Scope.md +5 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/07_NFR.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/09_Constraints.md +7 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/10_Policy.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/11_OQ-Register.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/12_OQ-Resolution-Log.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/14_Review-Request.md +14 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/99_delta.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/Rxx_reviewer.md +16 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/review_request.md +9 -6
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/summary.json +2 -0
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/uiux/40_screen_contracts.md +3 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/uiux/50_review_input_bundle.md +4 -2
- package/assets/init/.qfai/assistant/skills/qfai-implement/SKILL.md +252 -130
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/change-request-reset.md +93 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/checkpoint-verification.md +210 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/cross-spec-ownership.md +73 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/evidence-revision.md +291 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/execution-ledger.md +389 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/final-checklist.md +28 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/finding-classification.md +49 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/ledger-preconditions.md +55 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/oracle-strength.md +81 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/parallelization-policy.md +235 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-admissibility.md +91 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-not-observable.md +112 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/relevant-test-suite.md +87 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/review-artifact-layout.md +44 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/round-evidence.md +118 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/selector-granularity.md +24 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/upstream-artifact-ordering.md +33 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/volume-policy.md +149 -0
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/SKILL.md +35 -18
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/evidence-requirements.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/generator-prompt.md +106 -7
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/handoff.md +40 -11
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/iteration-loop.md +48 -1
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/templates/DESIGN.md.sample +6 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/SKILL.md +87 -22
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/contract-artifact-rules.md +148 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/rcp_footer.md +10 -4
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/review-cycle-playbook.md +5 -1
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-execution-playbook.md +4 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-phase-checklists.md +18 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-quality-gate.md +44 -3
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-triage.md +60 -7
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/spec-traceability-rules.md +171 -6
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/change-request.md +125 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/contracts/db-contract.sample.sql +6 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/evidence/sdd-spec.md +92 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/01_Objective.md +27 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/02_Initiative.md +30 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/05_Contracts.md +16 -6
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/06_Glossary.md +19 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/07_Constraints.md +25 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/08_Decisions.md +22 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/11_Slice-Policy.md +37 -10
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/02_User-stories.md +20 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/03_Acceptance-Criteria.md +19 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/04_Business-Rules.md +32 -3
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/06_Test-Cases.md +56 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/07_Decisions.md +29 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/10_Plan.md +41 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/16_Traceability-ledger.md +58 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/tdd/test-list.md +56 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/SKILL.md +57 -102
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/articles.md +24 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/context-load.md +22 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/verify-output-contract.md +48 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/templates/verify-evidence.md +48 -0
- package/assets/init/.qfai/assistant/skills/web-research/SKILL.md +25 -7
- package/assets/init/.qfai/waivers.yml +11 -5
- package/assets/init/root/.github/workflows/qfai-tests.yml +318 -0
- package/assets/init/root/.github/workflows/qfai-validate.yml +327 -24
- package/assets/init/root/DESIGN.md +6 -0
- package/assets/init/root/qfai.config.yaml +15 -12
- package/dist/cli/index.cjs +20646 -13620
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.mjs +22101 -15061
- package/dist/cli/index.mjs.map +1 -1
- package/dist/index.cjs +13954 -8829
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +423 -14
- package/dist/index.d.ts +423 -14
- package/dist/index.mjs +9330 -4224
- package/dist/index.mjs.map +1 -1
- package/package.json +22 -19
package/README.md
CHANGED
|
@@ -33,6 +33,28 @@ The agent reads the repository, produces the required artifacts, and iterates un
|
|
|
33
33
|
`index.html`, or `interaction.json`.
|
|
34
34
|
- Calibration SSOT is the calibration pack referenced by `calibrationRef.packPath`.
|
|
35
35
|
|
|
36
|
+
## Installation
|
|
37
|
+
|
|
38
|
+
qfai is published on npm as **`qfai`**. Install it as a dev dependency:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
npm i -D qfai
|
|
42
|
+
# or: pnpm add -D qfai / yarn add -D qfai
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Let the package manager write the `devDependencies` entry. Do not hand-pin a version
|
|
46
|
+
here: `package.json#version` in the published package is the only version source, and a
|
|
47
|
+
number copied from prose goes stale on the next release.
|
|
48
|
+
|
|
49
|
+
> **Do not install from the GitHub repository.** A git specifier such as
|
|
50
|
+
> `"qfai": "github:aganesy/QFAI"` maps the dependency key `qfai` to the private monorepo
|
|
51
|
+
> root — the manifest `name` is irrelevant, so it lands in `node_modules/qfai` regardless.
|
|
52
|
+
> That root ships no `bin` and no built `dist`, so nothing would be runnable or
|
|
53
|
+
> importable. Under npm or yarn a `preinstall` guard refuses the install with an
|
|
54
|
+
> explanatory error rather than completing silently; under pnpm — or any package manager
|
|
55
|
+
> that reports no user agent — it is not caught, so the mistake is yours to avoid. Use the
|
|
56
|
+
> npm package, or run it without installing via `npx qfai@latest <command>`.
|
|
57
|
+
|
|
36
58
|
## Quick start
|
|
37
59
|
|
|
38
60
|
> **Windows users:** `qfai init` creates symlinks internally.
|
|
@@ -99,13 +121,28 @@ npx qfai report
|
|
|
99
121
|
|
|
100
122
|
## ATDD annotation hard gate
|
|
101
123
|
|
|
102
|
-
`qfai validate` enforces spec-to-test traceability
|
|
124
|
+
`qfai validate` enforces spec-to-test traceability. `US` and `CON-API` obligations are routed by ID type;
|
|
125
|
+
a `TC` obligation is routed by the `Level` its spec declares for it.
|
|
103
126
|
|
|
104
127
|
- `tests/e2e/**`: annotate all covered user stories with concrete IDs such as `QFAI:SPEC-0001:US-0001`.
|
|
105
|
-
- `tests/integration/**`: annotate all covered test cases with concrete IDs such as `QFAI:SPEC-0001:TC-0001`.
|
|
106
128
|
- `tests/api/**`: annotate all covered API contracts with concrete IDs such as `QFAI:CON-API-0001`.
|
|
107
|
-
-
|
|
129
|
+
- Annotate a covered test case with a concrete ID such as `QFAI:SPEC-0001:TC-0001`, in the directory its declared `Level` names:
|
|
130
|
+
|
|
131
|
+
| `Level` | Annotated in |
|
|
132
|
+
| ----------------------------- | ---------------------- |
|
|
133
|
+
| `L1`/`Unit`, `L2`/`Component` | no ATDD annotation |
|
|
134
|
+
| `L3`/`Integration` | `tests/integration/**` |
|
|
135
|
+
| `L4`/`API` | `tests/api/**` |
|
|
136
|
+
| `L5`/`E2E` | `tests/e2e/**` |
|
|
137
|
+
| none declared, or unreadable | `tests/integration/**` |
|
|
138
|
+
|
|
139
|
+
- Unit and Component test cases carry **no** ATDD annotation obligation. They are gated by the
|
|
140
|
+
`tdd/test-list.md` ledger instead, so do not copy them into `tests/integration/**` to satisfy this gate.
|
|
141
|
+
- A `TC` annotation outside the directory its declared `Level` names is rejected. The rule is `Level`-relative,
|
|
142
|
+
not a blanket ban: a `TC` in `tests/api/**` is accepted only for a test case that declares `L4`/`API`, and in
|
|
143
|
+
`tests/e2e/**` only for `L5`/`E2E`.
|
|
108
144
|
- `AC` annotations are not required in code; AC coverage is treated as indirect through full `TC` coverage.
|
|
145
|
+
- These directories follow `paths.testsDir` from `qfai.config.yaml`; `tests/` above is the default.
|
|
109
146
|
|
|
110
147
|
## Operating model (skills-driven workflow)
|
|
111
148
|
|
|
@@ -123,7 +160,7 @@ The agent reads QFAI assets under `.qfai/assistant/` and produces or updates SDD
|
|
|
123
160
|
QFAI includes a small set of custom skills (stored under `.qfai/assistant/skills/`) designed to keep the workflow opinionated and repeatable.
|
|
124
161
|
|
|
125
162
|
- **qfai-configure**: Analyze the repository (language, frameworks, test layout, directory structure)
|
|
126
|
-
and
|
|
163
|
+
and adjust `qfai.config.yaml` accordingly (especially `testFileGlobs`).
|
|
127
164
|
Run this once right after `npx qfai init`, and re-run it when the repository structure changes.
|
|
128
165
|
- **qfai-discussion**: Run a unified structured discussion that produces and maintains the latest discussion pack
|
|
129
166
|
as 15 required markdown files under `.qfai/discussion/discussion-<ts>/`.
|
|
@@ -220,7 +257,10 @@ Operational notes.
|
|
|
220
257
|
- Skills should delegate work to multiple role-based sub-agents (Planner, Architect, Contract Designer, QA, Code Reviewer, etc.) to emulate a real delivery flow.
|
|
221
258
|
- Change classification (Primary/Tags) is required in `09_delta.md` and recommended in PRs. See `.qfai/assistant/constitution/change-classification.md`.
|
|
222
259
|
- Verification planning is recorded in `09_delta.md` (`Verification -> Plan`) and validated in CI (`VFY-*` rules).
|
|
223
|
-
- Review gate policies (required/optional layers
|
|
260
|
+
- Review gate policies (required/optional layers, default reviewers, optional
|
|
261
|
+
review modes) are documented in `.qfai/assistant/catalog/review-gate.rules.yml`.
|
|
262
|
+
This catalog is reference material for agents; it is not machine-enforced.
|
|
263
|
+
- Review pack structure — `.qfai/review/review-<YYYYMMDDhhmmssSSS>/{review_request.md,R01_*.md,summary.json}` — is the one layout enforced by validation (`QFAI-REVIEW-*`).
|
|
224
264
|
- Agent taxonomy and invocation SSOT are defined in `.qfai/assistant/manifest/agent-catalog.yml`, `.qfai/assistant/manifest/agent-routing.yml`, and `.qfai/assistant/manifest/review-profiles.yml`.
|
|
225
265
|
|
|
226
266
|
## Configuration
|
|
@@ -343,6 +383,11 @@ Recommended baseline.
|
|
|
343
383
|
|
|
344
384
|
Waiver policy.
|
|
345
385
|
|
|
386
|
+
- A waiver's `rule:` is the finding's `code`, copied verbatim from
|
|
387
|
+
`.qfai/report/validate.json` — `QFAI-ATDD-112`, `TDDLIST_UNKNOWN_LEVEL`,
|
|
388
|
+
`E_TC_ORPHAN`. Do not strip the `QFAI-` prefix; the stripped form
|
|
389
|
+
(`ATDD-112`) is kept working only for waiver files written against older
|
|
390
|
+
releases.
|
|
346
391
|
- Use waivers only for `warning` / `info` findings (false positives).
|
|
347
392
|
- Waivers that target `error` findings are invalid and fail validation (`QFAI-WAIVER-002`).
|
|
348
393
|
- Expired waivers are reported as warnings (`QFAI-WAIVER-003`) and must be renewed or removed with evidence.
|
|
@@ -434,7 +479,8 @@ Typical customizations.
|
|
|
434
479
|
│ │ ├── structure.md
|
|
435
480
|
│ │ ├── tech.md
|
|
436
481
|
│ │ ├── test-layers.md
|
|
437
|
-
│ │
|
|
482
|
+
│ │ ├── ui-definition-protocol.md
|
|
483
|
+
│ │ └── worklog-entry.schema.md
|
|
438
484
|
│ └── waivers.yml
|
|
439
485
|
└── qfai.config.yaml
|
|
440
486
|
```
|
|
@@ -443,6 +489,21 @@ Typical customizations.
|
|
|
443
489
|
contracts, evidence, reports, reviews, placeholder spec directories, or artifact
|
|
444
490
|
README files. Those files are created later by QFAI skills when real work exists.
|
|
445
491
|
|
|
492
|
+
### AI work-log surface (`.qfai/steering/`)
|
|
493
|
+
|
|
494
|
+
`qfai init` also creates `.qfai/steering/`, the per-project work-log surface for
|
|
495
|
+
AI coding agents, with a `README.md` and `_templates/entry.md`. Each entry is a
|
|
496
|
+
markdown file with YAML frontmatter, and `npx qfai validate` polices the surface in
|
|
497
|
+
the `sdd` and full profiles via `W-WORKLOG-SCHEMA`, `W-WORKLOG-BROKEN-LINK`,
|
|
498
|
+
`W-WORKLOG-STALE`, `W-PENDING-PROMOTION` and `R-HANDOFF-INCOMPLETE`.
|
|
499
|
+
|
|
500
|
+
The frontmatter contract and the **per-kind write trigger** — which `kind` an
|
|
501
|
+
agent writes when — are in the seeded
|
|
502
|
+
`.qfai/assistant/catalog/worklog-entry.schema.md`.
|
|
503
|
+
|
|
504
|
+
Note that `.qfai/steering/` (the work-log surface) is a different directory from
|
|
505
|
+
the legacy `.qfai/assistant/steering/` (the pre-recut assistant path).
|
|
506
|
+
|
|
446
507
|
Integration wrappers are also generated for immediate use:
|
|
447
508
|
|
|
448
509
|
- Agents/Codex VS Code: `.agents/skills/**`
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# QFAI assistant tree
|
|
2
|
+
|
|
3
|
+
This directory is the canonical source for QFAI's skills, agents, constitution,
|
|
4
|
+
manifest and catalog. `npx qfai init` writes it once and never removes it, and the
|
|
5
|
+
tool-specific integration directories are built from it.
|
|
6
|
+
|
|
7
|
+
## Canonical entrypoint
|
|
8
|
+
|
|
9
|
+
Every tool integration resolves through symlinks that point back here:
|
|
10
|
+
|
|
11
|
+
- .qfai/assistant/skills/ — skill documents
|
|
12
|
+
- .qfai/assistant/agents/ — agent definitions
|
|
13
|
+
|
|
14
|
+
These documents are the SSOT. Edit them here, not through the symlinks under
|
|
15
|
+
`.claude/`, `.agents/`, `.codex/` or `.github/`.
|
|
16
|
+
|
|
17
|
+
## Integration surface
|
|
18
|
+
|
|
19
|
+
`npx qfai init` creates the wrappers under those four directories. They are
|
|
20
|
+
generated, so re-running `npx qfai init` restores any that a checkout flattened or
|
|
21
|
+
a cleanup removed; nothing there needs to be edited by hand.
|
|
22
|
+
|
|
23
|
+
`npx qfai validate` checks that the wrappers still resolve to the documents above.
|
|
24
|
+
It reads this file to tell "init has run here and the surface was deleted" from
|
|
25
|
+
"init has never run here" — the two look identical from the integration
|
|
26
|
+
directories alone once every wrapper is gone, and only one of them is a
|
|
27
|
+
problem. Leave it in place.
|
|
@@ -32,6 +32,7 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
32
32
|
- Acceptance test plan and implemented coverage
|
|
33
33
|
- Mapping from US / TC / CON-API to test assets
|
|
34
34
|
- Execution proof and evidence summary
|
|
35
|
+
- TDD ledger Status + Evidence entry for each item processed, returned to the orchestrator (which owns the `test-list.md` write; do not edit that file directly)
|
|
35
36
|
- Gaps and follow-up actions
|
|
36
37
|
|
|
37
38
|
## Stop conditions
|
|
@@ -35,6 +35,7 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
35
35
|
- Backend implementation summary
|
|
36
36
|
- Changed files and affected contracts
|
|
37
37
|
- Local verification notes
|
|
38
|
+
- TDD ledger Status + Evidence entry for each item processed, returned to the orchestrator (which owns the `test-list.md` write; do not edit that file directly)
|
|
38
39
|
- Evidence summary for `.qfai/evidence/`
|
|
39
40
|
- Notes on invariants, validation, and why added abstractions/configuration are necessary now
|
|
40
41
|
|
|
@@ -12,7 +12,7 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
12
12
|
|
|
13
13
|
## Domain Responsibilities
|
|
14
14
|
|
|
15
|
-
- Return
|
|
15
|
+
- Return only PASS or REVISE, with actionable rework instructions on REVISE.
|
|
16
16
|
- Enforce validate evidence, required coverage obligations, and no self-approval.
|
|
17
17
|
- Verify rejected options are not reintroduced without RE-OPEN.
|
|
18
18
|
|
|
@@ -24,23 +24,50 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
24
24
|
- .qfai/assistant/catalog/test-layers.md
|
|
25
25
|
- .qfai/specs/spec-\*/09_delta.md
|
|
26
26
|
- Validation evidence and gate results
|
|
27
|
+
- `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
|
|
28
|
+
- The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
|
|
29
|
+
or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
|
|
30
|
+
|
|
31
|
+
**Validate evidence is a completion-gate input, not an item-cycle one.** When
|
|
32
|
+
this role is routed inside an item cycle — `/qfai-atdd` stage gate P1c hands a
|
|
33
|
+
single row to `/qfai-implement` and that run's reviewers gate its checkpoint —
|
|
34
|
+
`.qfai/report/validate.log`, the coverage reports and runtime evidence are P5/P6
|
|
35
|
+
artifacts of the calling stage and do not exist yet. Requiring them there
|
|
36
|
+
stopped the first branch-1 row at `refactor`, which Phase Red does not
|
|
37
|
+
re-select, so the calling stage never reached P2. Judge the row's own
|
|
38
|
+
phase-authored evidence; the completion gate is where the rest is owed. **The
|
|
39
|
+
two inputs above are what makes that possible** — without the ledger and the
|
|
40
|
+
evidence home its `Layer` selects, this role cannot identify the artifact it is
|
|
41
|
+
being asked to judge, and falls into its own Stop condition ("Required evidence
|
|
42
|
+
... missing") on a correct branch-1 or branch-2 row.
|
|
27
43
|
|
|
28
44
|
## Deliverables
|
|
29
45
|
|
|
30
|
-
- PASS or
|
|
46
|
+
- PASS or REVISE with concrete rework list
|
|
31
47
|
- Evidence summary and gaps
|
|
32
48
|
- Open risks or blocking assumptions
|
|
33
49
|
|
|
34
50
|
## Stop conditions
|
|
35
51
|
|
|
36
52
|
- Required evidence, governing specs, or target artifacts are missing.
|
|
53
|
+
- "Required evidence" means the evidence this review audits — the phase-authored record produced
|
|
54
|
+
before the review was requested. A completion record that this review's own verdict will be
|
|
55
|
+
written into is not yet expected to be finalized; its missing verdict fields are not a stop
|
|
56
|
+
condition and not a blocking finding. Report an unfinalized completion record as a sequencing
|
|
57
|
+
note, not as a gap.
|
|
37
58
|
- The request requires implementation or file editing instead of independent review.
|
|
38
59
|
- The issue falls outside this review domain and must be rerouted to another specialist first.
|
|
60
|
+
- The finding would add a product obligation upstream never asked for. Do not raise it as blocking;
|
|
61
|
+
raise it as an advisory finding plus a Change Request proposal per
|
|
62
|
+
`.qfai/assistant/constitution/drift-protocol.md#reviewer-originated-obligations`. A defect you can
|
|
63
|
+
demonstrate from the changed artifacts (correctness, security / data integrity, or a repository
|
|
64
|
+
quality gate) is NOT in this category: it stays blocking and traces to its `defect:*` class.
|
|
39
65
|
|
|
40
66
|
## Sign-off
|
|
41
67
|
|
|
42
68
|
- [ ] Review verdict is explicit
|
|
43
69
|
- [ ] Findings cite concrete artifacts or evidence
|
|
70
|
+
- [ ] Every finding declares `Severity:` and `Traces to:`; no blocking finding has `Traces to: none`
|
|
44
71
|
- [ ] Required gates and residual risks are recorded
|
|
45
72
|
|
|
46
73
|
## When to use
|
|
@@ -17,6 +17,13 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
17
17
|
- Define parallel-safe slices and post-merge verification conditions.
|
|
18
18
|
- Own schedule, priority, and risk trade-offs across roles.
|
|
19
19
|
- Produce actionable work orders for workers and reviewers.
|
|
20
|
+
- Own **item selection and item scope** in a TDD micro-cycle, and Red-Green-Refactor phase ordering. Raise a scope REVISE **before** RED/GREEN evidence is submitted to `qa-gatekeeper`; do not
|
|
21
|
+
re-litigate item scope after `qa-gatekeeper` has passed the observation for that round — open a new ledger row instead.
|
|
22
|
+
|
|
23
|
+
## Ownership boundaries
|
|
24
|
+
|
|
25
|
+
- `qa-gatekeeper` owns RED/GREEN **observation** evidence (did the test fail or pass for the expected reason). This role does not overrule that verdict, and that verdict does not widen item scope.
|
|
26
|
+
See `.qfai/assistant/skills/qfai-implement/SKILL.md#precedence-between-delivery-planner-and-qa-gatekeeper`.
|
|
20
27
|
|
|
21
28
|
## Inputs you must read
|
|
22
29
|
|
|
@@ -24,6 +31,16 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
24
31
|
- .qfai/assistant/{manifest,catalog}/\*\*
|
|
25
32
|
- .qfai/specs/spec-\*/09_delta.md
|
|
26
33
|
- .qfai/specs/spec-\*/01_Spec.md
|
|
34
|
+
- `.qfai/specs/spec-*/tdd/test-list.md` — the execution ledger this role selects
|
|
35
|
+
the next item from and whose Red-Green-Refactor ordering it enforces
|
|
36
|
+
- **The document the row's obligation column points at.** Item scope is "is this
|
|
37
|
+
selector a sufficient slice of the obligation", and the obligation is not
|
|
38
|
+
always a `TC-*`: an `E2E` row owes `US-Refs` and an `API` row owes
|
|
39
|
+
`CON-API-Refs`. Without these the role has nothing to compare such a row
|
|
40
|
+
against and can only guess a PASS or stall the gate.
|
|
41
|
+
- `.qfai/specs/spec-*/06_Test-Cases.md` for a `TC-Refs` row
|
|
42
|
+
- `.qfai/specs/spec-*/02_User-stories.md` for a `US-Refs` row
|
|
43
|
+
- `.qfai/contracts/api/**` for a `CON-API-Refs` row
|
|
27
44
|
- .qfai/discussion/discussion-\*/04_Sources.md
|
|
28
45
|
- .qfai/discussion/discussion-\*/06_REQ.md
|
|
29
46
|
- .qfai/discussion/discussion-\*/11_OQ-Register.md
|
|
@@ -34,6 +34,7 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
34
34
|
- Frontend implementation summary
|
|
35
35
|
- Changed files and affected flows
|
|
36
36
|
- Local verification notes
|
|
37
|
+
- TDD ledger Status + Evidence entry for each item processed, returned to the orchestrator (which owns the `test-list.md` write; do not edit that file directly)
|
|
37
38
|
- Evidence summary for `.qfai/evidence/`
|
|
38
39
|
- Notes on how simplicity, necessity, and reuse were preserved in the chosen implementation
|
|
39
40
|
|
|
@@ -31,6 +31,17 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
31
31
|
- .github/instructions/principles.instructions.md
|
|
32
32
|
- Diff of changed files
|
|
33
33
|
- `.qfai/contracts/api/**` and `.qfai/contracts/db/**`
|
|
34
|
+
- `.qfai/specs/<spec-id>/tdd/test-list.md` — the ledger, for the row under review
|
|
35
|
+
- The per-item evidence file that row's `Layer` owns: `.qfai/evidence/implement-<spec-id>.md`,
|
|
36
|
+
or `.qfai/evidence/atdd-<spec-id>.md` for an `E2E` / `API` / `Integration` row
|
|
37
|
+
|
|
38
|
+
**The last two are what the `Audited evidence hash` is computed over.** This
|
|
39
|
+
role records that hash itself, over the row's phase-authored fields — and those
|
|
40
|
+
live in an evidence file that is normally ignored, so the diff of changed files
|
|
41
|
+
does not contain them. Without the ledger and the evidence home the row's
|
|
42
|
+
`Layer` selects, this role cannot identify its own audit subject: the hash goes
|
|
43
|
+
missing and gate items 10-11 stop, or the orchestrator computes it instead,
|
|
44
|
+
which is the one thing the contract says must not happen.
|
|
34
45
|
|
|
35
46
|
## Deliverables
|
|
36
47
|
|
|
@@ -45,11 +56,17 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
45
56
|
- The request requires implementation or file editing instead of independent review.
|
|
46
57
|
- The issue falls outside this review domain and must be rerouted to another specialist first.
|
|
47
58
|
- The review would rely on speculative future requirements instead of current scope and evidence.
|
|
59
|
+
- The finding would add a product obligation upstream never asked for. Do not raise it as blocking;
|
|
60
|
+
raise it as an advisory finding plus a Change Request proposal per
|
|
61
|
+
`.qfai/assistant/constitution/drift-protocol.md#reviewer-originated-obligations`. A defect you can
|
|
62
|
+
demonstrate from the changed artifacts (correctness, security / data integrity, or a repository
|
|
63
|
+
quality gate) is NOT in this category: it stays blocking and traces to its `defect:*` class.
|
|
48
64
|
|
|
49
65
|
## Sign-off
|
|
50
66
|
|
|
51
67
|
- [ ] Review verdict is explicit
|
|
52
68
|
- [ ] Findings cite concrete artifacts or evidence
|
|
69
|
+
- [ ] Every finding declares `Severity:` and `Traces to:`; no blocking finding has `Traces to: none`
|
|
53
70
|
- [ ] Required gates and residual risks are recorded
|
|
54
71
|
|
|
55
72
|
## When to use
|
|
@@ -23,7 +23,7 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
23
23
|
- .qfai/assistant/constitution/\*\*
|
|
24
24
|
- .qfai/assistant/{manifest,catalog}/\*\*
|
|
25
25
|
- .qfai/assistant/catalog/test-layers.md (SSOT for hard coverage obligations)
|
|
26
|
-
- .qfai/specs/spec-\*/
|
|
26
|
+
- .qfai/specs/spec-\*/07_Decisions.md and .qfai/specs/\_policies/08_Decisions.md (Decision Records, DR-\*; check rejected)
|
|
27
27
|
- Prompt-specific artifacts (traceability, validation evidence, optional legacy scenario/ledger artifacts)
|
|
28
28
|
|
|
29
29
|
## Deliverables (MANDATORY)
|
|
@@ -37,7 +37,7 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
37
37
|
## Stop conditions
|
|
38
38
|
|
|
39
39
|
- Subagent delegation missing when required
|
|
40
|
-
- Validation gate evidence missing/failing (`qfai validate --fail-on error`)
|
|
40
|
+
- Validation gate evidence missing/failing (`npx qfai validate --fail-on error`)
|
|
41
41
|
- Required hard obligations in `test-layers.md` are unmet
|
|
42
42
|
- Reviewer sign-off missing
|
|
43
43
|
- Rejected option would be reintroduced without RE-OPEN DR
|
|
@@ -17,16 +17,153 @@ tools: [Read, Glob, Grep, Bash]
|
|
|
17
17
|
- Audit prototyping coverage evidence and unresolved spec coverage.
|
|
18
18
|
- Treat density or volume smells as review signals, not standalone hard gates.
|
|
19
19
|
- Verify test-case quality depth using the Coverage Depth Matrix (see below).
|
|
20
|
+
- Own RED/GREEN **observation** evidence in a TDD micro-cycle: did the test fail (or pass) for the expected reason.
|
|
21
|
+
|
|
22
|
+
## Ownership boundaries
|
|
23
|
+
|
|
24
|
+
- `delivery-planner` owns **item selection and item scope** — whether a ledger
|
|
25
|
+
row's selector is a sufficient slice of the obligation its `Layer` names
|
|
26
|
+
(`TC-Refs`, `US-Refs` or `CON-API-Refs`). Do not adjudicate item scope here; a PASS on observation
|
|
27
|
+
evidence is explicitly scoped to that observation and never widens or ratifies item scope. See `.qfai/assistant/skills/qfai-implement/SKILL.md#precedence-between-delivery-planner-and-qa-gatekeeper`.
|
|
28
|
+
- Refuse to evaluate RED/GREEN evidence while an unresolved `delivery-planner` scope REVISE is open on the same item.
|
|
29
|
+
|
|
30
|
+
## Oracle Strength Check (MUST)
|
|
31
|
+
|
|
32
|
+
GREEN is `exit code == 0`. That does not say the pass depends on the behaviour
|
|
33
|
+
the item owns, and nothing downstream re-asks: coverage is annotation presence
|
|
34
|
+
and the Depth Matrix counts case categories. A test that cannot fail otherwise
|
|
35
|
+
clears every gate.
|
|
36
|
+
|
|
37
|
+
Require an `Oracle proof` on each item **at a GREEN or completion gate**, and
|
|
38
|
+
**reject** it when:
|
|
39
|
+
|
|
40
|
+
- the mutation is outside the code the item owns — breaking a shared helper
|
|
41
|
+
proves the helper is used, not that this test discriminates;
|
|
42
|
+
- the mutation is a syntax error, a thrown "not implemented", or a deleted
|
|
43
|
+
export — that is a load failure, not a discriminating failure;
|
|
44
|
+
- the failing output names a selector other than the row's;
|
|
45
|
+
- the recorded command differs from the `GREEN command`.
|
|
46
|
+
|
|
47
|
+
**At a RED observation the proof is a plan, and a plan is enough.** Branch 1's
|
|
48
|
+
RED is taken before any production behaviour exists, so there is nothing to
|
|
49
|
+
mutate: the item names the predicate it will break and the command it will run.
|
|
50
|
+
Requiring a demonstrated mutation there made a correct observed RED unable to
|
|
51
|
+
pass P1b and so unable to reach Phase Green — the phase that builds the code the
|
|
52
|
+
mutation needs. Judge the plan for whether it names this row's predicate and
|
|
53
|
+
selector; judge the demonstration once the behaviour exists.
|
|
54
|
+
|
|
55
|
+
`equivalent-mutant` is acceptable **only** when the named contract clause is
|
|
56
|
+
genuinely weaker than the obligation. It is an upstream gap: route it as an
|
|
57
|
+
advisory / Change Request, do not send the implementer to strengthen an
|
|
58
|
+
assertion past the contract — that is reviewer-originated scope, which
|
|
59
|
+
`drift-protocol.md` forbids. Full criteria and the weak-oracle shapes:
|
|
60
|
+
`.qfai/assistant/skills/qfai-implement/references/oracle-strength.md`.
|
|
61
|
+
|
|
62
|
+
## RED/GREEN Observation Gate (MUST)
|
|
63
|
+
|
|
64
|
+
This is the gate `qfai-implement` routes here as blocking, per ledger row. Judge
|
|
65
|
+
the row's own evidence; nothing in the calling work order substitutes for it.
|
|
66
|
+
|
|
67
|
+
**Accept a RED** only when all hold:
|
|
68
|
+
|
|
69
|
+
- the test module loaded — the failure is not a collection, import, syntax,
|
|
70
|
+
missing-symbol or fixture error;
|
|
71
|
+
- an assertion (or expected-exception check) inside the row's own `Selector`
|
|
72
|
+
raised it, and the message names the predicate the row owns;
|
|
73
|
+
- the recorded output retains that assertion message and its location;
|
|
74
|
+
- when the `Selector` holds several entries, each entry's failure was observed
|
|
75
|
+
separately. One aggregate run is not a RED for several entries.
|
|
76
|
+
|
|
77
|
+
**Accept a GREEN** only when the same command shape ran after the production
|
|
78
|
+
change and the recorded output shows the row's own selector passing. A full-suite
|
|
79
|
+
pass that does not name the row's selector is not a GREEN for that row.
|
|
80
|
+
|
|
81
|
+
**Never accept as a substitute** for a captured failing run of the item's own
|
|
82
|
+
test:
|
|
83
|
+
|
|
84
|
+
- a narrative claim that the test failed, in any artifact, including a commit
|
|
85
|
+
message written by the implementing agent — that is self-attestation, which is
|
|
86
|
+
what this gate exists to prevent;
|
|
87
|
+
- a load error standing in for an assertion failure;
|
|
88
|
+
- evidence copied from a previous round or a sibling row;
|
|
89
|
+
- "the suite is green" in place of the row's own GREEN.
|
|
90
|
+
|
|
91
|
+
The one legitimate absence is the _RED not observable_ path: the obligation is
|
|
92
|
+
already satisfied by something already in the tree, so the correct test passes
|
|
93
|
+
first run. Then require `Satisfied-by`, `Falsifiability command` and
|
|
94
|
+
`Falsifiability result` instead — never both forms, never neither.
|
|
95
|
+
|
|
96
|
+
**On an `E2E` / `API` / `Integration` row, `Satisfied-by` need not be a sibling `TDD-NNNN`.** A
|
|
97
|
+
production **path and symbol** is equally valid there and is the normal answer
|
|
98
|
+
for a row whose surface no ledger row owns; rejecting it sends every such row to
|
|
99
|
+
`exception`, the terminal state the path exists to avoid. Judge it on whether it
|
|
100
|
+
answers "what would I mutate to falsify this row".
|
|
101
|
+
|
|
102
|
+
**A commit id alone does not answer it — REVISE.** A commit that touched
|
|
103
|
+
several routes and a helper names no single predicate, so the ownership check
|
|
104
|
+
below has no boundary to apply and would accept a mutation anywhere inside it.
|
|
105
|
+
The producer contract requires the symbol for this reason
|
|
106
|
+
(`../skills/qfai-atdd/references/red-provenance.md#the-three-branches-must`); a
|
|
107
|
+
commit recorded **alongside** the path and symbol is provenance and is fine.
|
|
108
|
+
|
|
109
|
+
**And the mutation may touch it.** The Oracle Strength Check rejects a mutation
|
|
110
|
+
outside the code the item owns, which on an `E2E` / `API` / `Integration` row is every
|
|
111
|
+
production predicate there is — the same sentence above says no ledger row owns
|
|
112
|
+
that surface. Applied literally, no branch-2 row could ever produce
|
|
113
|
+
falsifiability evidence that passes. On a handed-over row, **the predicate
|
|
114
|
+
`Satisfied-by` names is the owned code** for this check; anything else is still
|
|
115
|
+
out of bounds.
|
|
116
|
+
|
|
117
|
+
**On any other row the sibling row is still required** — production code
|
|
118
|
+
no ledger row owns is the anomaly case there, not a substitute. See
|
|
119
|
+
`.qfai/assistant/skills/qfai-implement/references/red-not-observable.md` and
|
|
120
|
+
`.qfai/assistant/skills/qfai-implement/references/red-admissibility.md`.
|
|
121
|
+
|
|
122
|
+
**A `Layer = E2E` / `Layer = API` row from `/qfai-atdd` is judged the same
|
|
123
|
+
way.** Its journey is often written after the surface the same cycle built, so
|
|
124
|
+
the falsifiability form is the expected evidence rather than a concession —
|
|
125
|
+
accept it, with the mutated predicate being one the journey actually asserts
|
|
126
|
+
on. What is **not** acceptable is the third outcome appearing by default: a row
|
|
127
|
+
routed to `exception` whose `DR-*` says only that the surface came first has
|
|
128
|
+
not shown that either branch was unavailable, and that is a REVISE. See
|
|
129
|
+
`.qfai/assistant/skills/qfai-atdd/SKILL.md#red-provenance-for-an-atdd-owned-row-must`
|
|
130
|
+
and `.qfai/assistant/skills/qfai-implement/references/execution-ledger.md#atdd-owned-rows`.
|
|
131
|
+
|
|
132
|
+
Verdict scope: a PASS covers the observation for that round and nothing else. It
|
|
133
|
+
does not ratify item scope and does not clear the completion gate.
|
|
20
134
|
|
|
21
135
|
## Test Case Quality Depth Check (MUST)
|
|
22
136
|
|
|
23
137
|
In addition to traceability-based coverage (US/TC/CON-API existence), verify the **depth** of test cases:
|
|
24
138
|
|
|
25
|
-
- Confirm a Coverage Depth Matrix exists (produced by `test-design-analyst`).
|
|
139
|
+
- Confirm a Coverage Depth Matrix exists at `.qfai/evidence/coverage-depth-<spec-id>.md` (produced by `test-design-analyst`).
|
|
140
|
+
Missing matrix: REVISE from the ATDD review cycle onward; on an SDD review cycle record it as a finding. See the scope note.
|
|
141
|
+
A matrix that exists only inside `.qfai/evidence/atdd-<spec-id>.md` is a **missing** matrix: that file is ignored by the
|
|
142
|
+
managed `.gitignore` block, so neither it nor the justification for any `❌` reaches a commit, and the "unjustified"
|
|
143
|
+
judgement cannot be re-made by anyone reading the repository.
|
|
26
144
|
- Check that each US/TC has test cases for at minimum: normal path AND error/failure path.
|
|
27
145
|
- Flag any US/TC that has only normal-path test cases as a coverage gap.
|
|
28
146
|
- Reference: `.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`
|
|
29
|
-
-
|
|
147
|
+
- Which verdict applies depends on the review cycle, per the scope note below.
|
|
148
|
+
On an **SDD** cycle this check is a review signal, not a hard gate that blocks validation.
|
|
149
|
+
From the **ATDD** cycle onward a missing matrix — or one whose ❌ cells are unjustified — is a REVISE.
|
|
150
|
+
Either way, unjustified gaps MUST be documented as findings.
|
|
151
|
+
|
|
152
|
+
### Scope of this check
|
|
153
|
+
|
|
154
|
+
The Coverage Depth Matrix is an **ATDD-stage artifact**: it is defined in
|
|
155
|
+
`.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`, listed as an ATDD
|
|
156
|
+
Mandatory Output,
|
|
157
|
+
and written to `.qfai/evidence/coverage-depth-<spec-id>.md` — a committed path, unlike the rest of
|
|
158
|
+
`.qfai/evidence/**`. `qfai-sdd` neither defines its layout nor ships a section for it, so:
|
|
159
|
+
|
|
160
|
+
- Apply this check from the **ATDD review cycle onward**, where
|
|
161
|
+
`.qfai/assistant/skills/qfai-atdd/SKILL.md` lists
|
|
162
|
+
the matrix under both Mandatory Outputs and Not-done criteria. A missing matrix is a REVISE there,
|
|
163
|
+
and so is one whose ❌ cells are unjustified.
|
|
164
|
+
- Do NOT evaluate it against an SDD spec pack that has no tests yet. On an SDD review cycle,
|
|
165
|
+
assess depth directly from `06_Test-Cases.md` (normal path plus error/boundary coverage per
|
|
166
|
+
AC) and record any gap as a finding, without requiring the matrix format.
|
|
30
167
|
|
|
31
168
|
## Inputs you must read
|
|
32
169
|
|
|
@@ -35,19 +172,60 @@ In addition to traceability-based coverage (US/TC/CON-API existence), verify the
|
|
|
35
172
|
- .qfai/assistant/{manifest,catalog}/\*\*
|
|
36
173
|
- .qfai/assistant/catalog/test-layers.md
|
|
37
174
|
- .qfai/specs/spec-\*/09_delta.md
|
|
38
|
-
- `.qfai/
|
|
175
|
+
- `.qfai/specs/spec-*/tdd/test-list.md` — the ledger row under review
|
|
176
|
+
- **The per-item RED/GREEN evidence for the row under review — in the file its
|
|
177
|
+
`Layer` owns, and only that one.** `.qfai/evidence/atdd-<spec-id>.md`, under
|
|
178
|
+
`## Ledger rows advanced`, for a `Layer = E2E` / `Layer = API` / `Layer = Integration` row;
|
|
179
|
+
`.qfai/evidence/implement-<spec-id>.md` for every other row. Both are listed
|
|
180
|
+
because the Stop condition below ("target artifacts are missing") is not
|
|
181
|
+
checkable against an artifact this role was never told to open — but
|
|
182
|
+
requiring **both** makes that condition fire on a spec that legitimately has
|
|
183
|
+
one: a Unit-only spec never ran `/qfai-atdd`, and a spec whose rows are all
|
|
184
|
+
`E2E` / `API` / `Integration` has no implement file. Either way the gate would stop before
|
|
185
|
+
reading the evidence that does exist.
|
|
186
|
+
**The three below are required at a completion gate, not at a RED/GREEN
|
|
187
|
+
observation.** `/qfai-atdd` routes this role as blocking at stage gate P1b, and
|
|
188
|
+
validate output, coverage reports and runtime evidence are first produced at its
|
|
189
|
+
P5 and P6 — so requiring them there stopped a fresh run that had a perfectly
|
|
190
|
+
good RED pair, on artifacts its own ordering says cannot exist yet. At an
|
|
191
|
+
observation gate the row's own evidence above is the whole input.
|
|
192
|
+
|
|
193
|
+
- The scoped validate JSON for the spec under review — `validate.spec-<id>.json`
|
|
194
|
+
**beside the configured `output.validateJsonPath`**, not under a fixed
|
|
195
|
+
`.qfai/report/` — or the `run-*/` directory of the run that produced it, under
|
|
196
|
+
the configured `paths.outDir`. Read both from `qfai.config.yaml` the way the
|
|
197
|
+
SDD and discussion contracts do. A project that moved either output writes its
|
|
198
|
+
evidence where it said to, and looking for it at the default path reported a
|
|
199
|
+
missing artifact and stopped completion on a validate run that had succeeded
|
|
200
|
+
and left everything it owed. **Not `validate.log`**: it and the run-log pointer are shared by every
|
|
201
|
+
run, scoped or not, and nothing serializes them, so a sibling stage validating
|
|
202
|
+
at the same time overwrites what this one wrote — a failing run followed by a
|
|
203
|
+
sibling's success reads as this spec's PASS
|
|
39
204
|
- `.qfai/report/specs-coverage/spec-*.md`
|
|
40
205
|
- Runtime evidence and prototyping evidence artifacts
|
|
41
206
|
|
|
207
|
+
**Branch 3 gets its own verdict.** The observation gate admits an observed RED
|
|
208
|
+
or a falsifiability trio and calls anything else "never neither" — but a genuine
|
|
209
|
+
branch-3 row _has_ neither, by the finding that put it there. Judged by the two
|
|
210
|
+
forms it can only be REVISE, and skipping the gate leaves the stage's completion
|
|
211
|
+
condition unmet, so the row could not close either way. Judge these on their own
|
|
212
|
+
terms: a `DR-*` that records **what could not be observed and why each branch was
|
|
213
|
+
unavailable**, PASS or REVISE on that. A missing `DR-*`, or one that names no
|
|
214
|
+
unavailability, is still REVISE — this is a third form of evidence, not an
|
|
215
|
+
exemption from having any.
|
|
216
|
+
|
|
42
217
|
## Deliverables
|
|
43
218
|
|
|
44
|
-
- Gate decision (PASS /
|
|
219
|
+
- Gate decision (PASS / REVISE) with rationale
|
|
45
220
|
- Hard gate status and required fixes
|
|
46
221
|
- Evidence summary and unresolved quality gaps
|
|
47
222
|
|
|
48
223
|
## Stop conditions
|
|
49
224
|
|
|
50
|
-
- Required evidence, governing specs, or target artifacts are missing
|
|
225
|
+
- Required evidence, governing specs, or target artifacts are missing — judged
|
|
226
|
+
against what the invoking phase requires, per the note above the last three
|
|
227
|
+
inputs. At a RED/GREEN observation that is the row's own evidence; at a
|
|
228
|
+
completion gate it is all of them.
|
|
51
229
|
- The request requires implementation or file editing instead of independent review.
|
|
52
230
|
- The issue falls outside this review domain and must be rerouted to another specialist first.
|
|
53
231
|
|
|
@@ -17,16 +17,30 @@ tools: [Read, Write, Edit, Glob, Grep, Bash]
|
|
|
17
17
|
- Estimate test volume as a planning signal, not a hard gate.
|
|
18
18
|
- Prevent unit/component scope creep and ambiguous layer ownership.
|
|
19
19
|
- Evaluate test-case depth using the structured checklist (see reference below).
|
|
20
|
-
- Produce a Coverage Depth Matrix
|
|
20
|
+
- Produce a Coverage Depth Matrix per spec from the ATDD stage onward, exposing gaps in boundary values, error paths, edge cases,
|
|
21
|
+
and combinatorial scenarios. During SDD, report the same gaps as findings instead — see the stage split below.
|
|
21
22
|
|
|
22
23
|
## Test Case Quality Depth (MUST)
|
|
23
24
|
|
|
24
25
|
When reviewing or producing test cases, apply the checklist in `.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md`.
|
|
25
26
|
|
|
27
|
+
Read the stage split first: the checklist's full category set and the Coverage Depth Matrix are ATDD-stage obligations. `qfai-sdd` neither
|
|
28
|
+
defines the matrix layout nor ships an artifact that holds it, and the tests do not exist yet, so the SDD rules below replace them.
|
|
29
|
+
|
|
30
|
+
From the ATDD stage onward:
|
|
31
|
+
|
|
26
32
|
- For each US/TC, verify that test cases exist for: normal path, error path, boundary values, special values, state transitions, and combinatorial scenarios.
|
|
27
33
|
- Produce the Coverage Depth Matrix as a required deliverable. Flag any ❌ cells as gaps.
|
|
28
34
|
- Test cases covering only normal (happy) paths are INCOMPLETE. Return REVISE with specific missing scenarios.
|
|
29
|
-
|
|
35
|
+
|
|
36
|
+
During SDD:
|
|
37
|
+
|
|
38
|
+
- Require normal path plus error/boundary coverage per AC, read directly from `06_Test-Cases.md`.
|
|
39
|
+
- Record any further depth gap (special values, state transitions, combinatorial) as a finding.
|
|
40
|
+
- Do NOT produce the matrix, and do NOT return REVISE solely because the matrix is absent or because
|
|
41
|
+
special / state-transition / combinatorial cases are not yet enumerated.
|
|
42
|
+
|
|
43
|
+
At both stages: when business rules (BR-\*) exist, verify each BR has at least one positive and one negative test case.
|
|
30
44
|
|
|
31
45
|
## Inputs you must read
|
|
32
46
|
|
|
@@ -43,7 +57,12 @@ When reviewing or producing test cases, apply the checklist in `.qfai/assistant/
|
|
|
43
57
|
|
|
44
58
|
- Coverage plan and layer ownership
|
|
45
59
|
- Test-case quality and traceability findings
|
|
46
|
-
- **Coverage Depth Matrix** (per spec, using the template in the depth checklist reference)
|
|
60
|
+
- **Coverage Depth Matrix** (per spec, using the template in the depth checklist reference).
|
|
61
|
+
Destination: `.qfai/evidence/coverage-depth-<spec-id>.md` from the ATDD stage onward — its own
|
|
62
|
+
file, because that path is committed while the rest of `.qfai/evidence/**` is ignored, and the
|
|
63
|
+
justification behind each `❌` is the input `qa-gatekeeper` reads. During SDD there is
|
|
64
|
+
no evidence artifact that holds it, so report depth gaps as findings instead of producing the
|
|
65
|
+
matrix format.
|
|
47
66
|
- Volume estimate and risk notes
|
|
48
67
|
- Scope-boundary decisions for tests
|
|
49
68
|
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
# CLI UX Guidelines
|
|
2
2
|
|
|
3
|
-
QFAI が定義する、`qfai validate` の UI/UX 関連出力ガイドライン。
|
|
3
|
+
QFAI が定義する、`npx qfai validate` の UI/UX 関連出力ガイドライン。
|
|
4
4
|
|
|
5
5
|
## Output Format
|
|
6
6
|
|
|
7
7
|
- Issue メッセージにタイムスタンプやランダム値を含めない(冪等性確保)
|
|
8
|
-
- `## Auto Check Results` — `qfai validate` による自動チェック結果セクション
|
|
8
|
+
- `## Auto Check Results` — `npx qfai validate` による自動チェック結果セクション
|
|
9
9
|
- `## Manual Review Results` — product-surface-reviewer による手動レビュー結果セクション
|
|
10
10
|
|
|
11
11
|
## Error Message Format
|