qfai 1.9.2 → 1.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +67 -6
- package/assets/init/.qfai/assistant/README.md +27 -0
- package/assets/init/.qfai/assistant/agents/acceptance-test-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/backend-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/completion-reviewer.md +29 -2
- package/assets/init/.qfai/assistant/agents/delivery-planner.md +17 -0
- package/assets/init/.qfai/assistant/agents/frontend-engineer.md +1 -0
- package/assets/init/.qfai/assistant/agents/implementation-reviewer.md +17 -0
- package/assets/init/.qfai/assistant/agents/orchestrator.md +2 -2
- package/assets/init/.qfai/assistant/agents/qa-gatekeeper.md +183 -5
- package/assets/init/.qfai/assistant/agents/test-design-analyst.md +22 -3
- package/assets/init/.qfai/assistant/catalog/cli-ux-guidelines.md +2 -2
- package/assets/init/.qfai/assistant/catalog/spec_required_files.json +2 -1
- package/assets/init/.qfai/assistant/catalog/test-layers-ci-lanes.md +60 -0
- package/assets/init/.qfai/assistant/catalog/test-layers.md +430 -13
- package/assets/init/.qfai/assistant/catalog/worklog-entry.schema.md +165 -0
- package/assets/init/.qfai/assistant/constitution/communication.md +1 -1
- package/assets/init/.qfai/assistant/constitution/constitution.md +1 -1
- package/assets/init/.qfai/assistant/constitution/drift-protocol.md +359 -10
- package/assets/init/.qfai/assistant/constitution/quality.md +35 -5
- package/assets/init/.qfai/assistant/constitution/requirements-decomposition.md +37 -0
- package/assets/init/.qfai/assistant/constitution/shared-skill-delegation-baseline.md +419 -8
- package/assets/init/.qfai/assistant/constitution/shared-skill-operating-baseline.md +122 -5
- package/assets/init/.qfai/assistant/constitution/workflow.md +53 -7
- package/assets/init/.qfai/assistant/manifest/agent-catalog.yml +439 -946
- package/assets/init/.qfai/assistant/manifest/agent-routing.yml +116 -4
- package/assets/init/.qfai/assistant/manifest/review-profiles.yml +9 -0
- package/assets/init/.qfai/assistant/process/migrations/v1.4.27-atdd-alignment.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-atdd/SKILL.md +150 -53
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/credential-reuse.md +146 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/red-provenance.md +456 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/review-fix-rounds.md +128 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/scaffolding.md +29 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/shared-test-artifacts.md +96 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/stale-manifest.md +34 -0
- package/assets/init/.qfai/assistant/skills/qfai-atdd/references/test-case-depth-checklist.md +52 -5
- package/assets/init/.qfai/assistant/skills/qfai-configure/SKILL.md +16 -8
- package/assets/init/.qfai/assistant/skills/qfai-discussion/SKILL.md +8 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/design-md-brand-catalog.md +2 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/discussion-completion-matrix.md +17 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/rcp_footer.md +10 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/review-cycle-playbook.md +5 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui-bearing-playbook.md +4 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux/review_audit_playbook.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux/trend_scan_playbook.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/references/ui_ux_best_practices.md +17 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/01_Context.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/02_Inception-Deck.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/03_Story-Workshop.md +11 -3
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/05_Scope.md +5 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/07_NFR.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/09_Constraints.md +7 -4
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/10_Policy.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/11_OQ-Register.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/12_OQ-Resolution-Log.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/14_Review-Request.md +14 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/99_delta.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/Rxx_reviewer.md +16 -7
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/review_request.md +9 -6
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/review/summary.json +2 -0
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/uiux/40_screen_contracts.md +3 -2
- package/assets/init/.qfai/assistant/skills/qfai-discussion/templates/uiux/50_review_input_bundle.md +4 -2
- package/assets/init/.qfai/assistant/skills/qfai-implement/SKILL.md +252 -130
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/change-request-reset.md +93 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/checkpoint-verification.md +210 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/cross-spec-ownership.md +73 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/evidence-revision.md +291 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/execution-ledger.md +389 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/final-checklist.md +28 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/finding-classification.md +49 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/ledger-preconditions.md +55 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/oracle-strength.md +81 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/parallelization-policy.md +235 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-admissibility.md +91 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/red-not-observable.md +112 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/relevant-test-suite.md +87 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/review-artifact-layout.md +44 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/round-evidence.md +118 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/selector-granularity.md +24 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/upstream-artifact-ordering.md +33 -0
- package/assets/init/.qfai/assistant/skills/qfai-implement/references/volume-policy.md +149 -0
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/SKILL.md +35 -18
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/evidence-requirements.md +1 -1
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/generator-prompt.md +106 -7
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/handoff.md +40 -11
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/references/iteration-loop.md +48 -1
- package/assets/init/.qfai/assistant/skills/qfai-prototyping/templates/DESIGN.md.sample +6 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/SKILL.md +87 -22
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/contract-artifact-rules.md +148 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/rcp_footer.md +10 -4
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/review-cycle-playbook.md +5 -1
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-execution-playbook.md +4 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-phase-checklists.md +18 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-quality-gate.md +44 -3
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/sdd-triage.md +60 -7
- package/assets/init/.qfai/assistant/skills/qfai-sdd/references/spec-traceability-rules.md +171 -6
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/change-request.md +125 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/contracts/db-contract.sample.sql +6 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/evidence/sdd-spec.md +92 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/01_Objective.md +27 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/02_Initiative.md +30 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/05_Contracts.md +16 -6
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/06_Glossary.md +19 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/07_Constraints.md +25 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/08_Decisions.md +22 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/_policies/11_Slice-Policy.md +37 -10
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/02_User-stories.md +20 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/03_Acceptance-Criteria.md +19 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/04_Business-Rules.md +32 -3
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/06_Test-Cases.md +56 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/07_Decisions.md +29 -2
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/10_Plan.md +41 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/16_Traceability-ledger.md +58 -0
- package/assets/init/.qfai/assistant/skills/qfai-sdd/templates/specs/spec/tdd/test-list.md +56 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/SKILL.md +57 -102
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/articles.md +24 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/context-load.md +22 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/references/verify-output-contract.md +48 -0
- package/assets/init/.qfai/assistant/skills/qfai-verify/templates/verify-evidence.md +48 -0
- package/assets/init/.qfai/assistant/skills/web-research/SKILL.md +25 -7
- package/assets/init/.qfai/waivers.yml +11 -5
- package/assets/init/root/.github/workflows/qfai-tests.yml +318 -0
- package/assets/init/root/.github/workflows/qfai-validate.yml +327 -24
- package/assets/init/root/DESIGN.md +6 -0
- package/assets/init/root/qfai.config.yaml +15 -12
- package/dist/cli/index.cjs +20646 -13620
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.mjs +22101 -15061
- package/dist/cli/index.mjs.map +1 -1
- package/dist/index.cjs +13954 -8829
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +423 -14
- package/dist/index.d.ts +423 -14
- package/dist/index.mjs +9330 -4224
- package/dist/index.mjs.map +1 -1
- package/package.json +22 -19
|
@@ -15,37 +15,248 @@ Skill files should reference this baseline and only add role-, stage-, or gate-s
|
|
|
15
15
|
|
|
16
16
|
1. Attempt the first required delegation at stage start using the platform's native delegation mechanism.
|
|
17
17
|
2. Treat that first real delegation attempt as the capability check. Do not gate execution on preflight availability questions or synthetic probe-only checks.
|
|
18
|
-
3. If the delegation fails,
|
|
18
|
+
3. If the delegation fails, classify the failure first (see `Delegation Failure Taxonomy`), then apply the response for that class. Never simulate roles and never continue with self-execution, whatever the class.
|
|
19
|
+
|
|
20
|
+
### Delegation Failure Taxonomy (MUST)
|
|
21
|
+
|
|
22
|
+
Every delegation failure belongs to exactly one of two classes.
|
|
23
|
+
|
|
24
|
+
| Class | Meaning | Sanctioned response |
|
|
25
|
+
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
|
|
26
|
+
| `unavailable` | The host has no usable delegation mechanism, the role is unknown, or the failure is a configuration / tooling / quota gap only the user can close — including a limit that waiting cannot clear. | Hard stop. |
|
|
27
|
+
| `saturated` | The host can delegate but is momentarily out of budget — `agent thread limit reached`, concurrency cap, queue full, rate limit, busy pool. The identical call would succeed later with no change by anyone. | Bounded wait-and-retry on the same stage. |
|
|
28
|
+
|
|
29
|
+
- Classify from the raw failure reason, and classify `saturated` only when the reason states or
|
|
30
|
+
plainly implies that the identical call would succeed later **with no change by anyone**: a queue
|
|
31
|
+
or pool that is currently full, a rate limit with a retry window, a concurrency cap that is
|
|
32
|
+
momentarily reached, an explicit "try again later".
|
|
33
|
+
- A limit or quota that only a user can lift is `unavailable`, not `saturated` — a configured
|
|
34
|
+
concurrency cap of 0, a maximum delegation depth, an input-size limit, an exhausted account
|
|
35
|
+
quota or plan. Waiting cannot clear those, so the retry loop would burn 30/60/120 seconds and
|
|
36
|
+
then report "no user action needed" about a condition that needs exactly that.
|
|
37
|
+
- When retryability is not explicit, default to `unavailable`. The two classes are not
|
|
38
|
+
symmetric: mis-classifying as `unavailable` costs one unnecessary stop the user can act on,
|
|
39
|
+
while mis-classifying as `saturated` hides an actionable failure behind a pointless wait.
|
|
40
|
+
- `saturated` never authorises self-execution of a primary artifact or of a blocking review, and never authorises discarding stage progress.
|
|
41
|
+
- When the `saturated` retry budget is exhausted, fall through to the hard stop and report the class as `saturated (retry budget exhausted)`.
|
|
42
|
+
|
|
43
|
+
### Delegation Failure — `saturated` (Bounded Retry)
|
|
44
|
+
|
|
45
|
+
- Retry the identical delegation with backoff: 30s, then 60s, then 120s. Attempt cap: 3 retries per work order.
|
|
46
|
+
- Do not re-scope, re-plan, or re-route the work order between retries — same role, same task.
|
|
47
|
+
- The stage stays open and resumable across the wait; completed work orders keep their `PASS` status.
|
|
48
|
+
- Report on entering the retry loop and on its outcome:
|
|
49
|
+
- `Delegation deferred: <raw reason or concise summary>`
|
|
50
|
+
- `Failure class: saturated`
|
|
51
|
+
- `Attempted role: <role>`
|
|
52
|
+
- `Attempted task: <task title>`
|
|
53
|
+
- `Retry condition: retry after <N> seconds / when a delegation slot frees`
|
|
54
|
+
- `Attempts used: <n>/3`
|
|
55
|
+
- `Stage state: held open and resumable — no stage progress discarded`
|
|
19
56
|
|
|
20
57
|
### Delegation Failure (Hard Stop)
|
|
21
58
|
|
|
59
|
+
Applies to `unavailable`, and to `saturated` once the retry budget is exhausted.
|
|
60
|
+
|
|
22
61
|
- Report all of:
|
|
23
62
|
- `Delegation failure: <raw reason or concise summary>`
|
|
63
|
+
- `Failure class: unavailable | saturated (retry budget exhausted)`
|
|
24
64
|
- `Attempted role: <role>`
|
|
25
65
|
- `Attempted task: <task title>`
|
|
26
66
|
- `Why stopped: QFAI requires real sub-agent delegation in this environment.`
|
|
27
|
-
- `User action needed: <settings or tooling changes required>`
|
|
67
|
+
- `User action needed: <settings or tooling changes required — or "none; wait for a delegation slot to free" when the class is saturated>`
|
|
28
68
|
- `Retry condition: rerun after the required delegation succeeds`
|
|
29
69
|
|
|
70
|
+
### Commit Scoping (MUST)
|
|
71
|
+
|
|
72
|
+
- A delegated agent stages only the paths it declared as deliverables in its
|
|
73
|
+
work order: `git add <path> …`.
|
|
74
|
+
- `git add -A`, `git add .` and `git commit -a` are forbidden for delegated
|
|
75
|
+
agents, in both isolation modes. In degraded / shared-index mode the
|
|
76
|
+
concurrent agents share one index, so a sweeping stage command commits a
|
|
77
|
+
sibling agent's in-flight files and misattributes work in the audit trail.
|
|
78
|
+
Under worktree separation there is no shared index and no sibling file to
|
|
79
|
+
sweep, but the command still stages everything else loose in that agent's own
|
|
80
|
+
worktree, so the commit still stops matching its declared deliverables.
|
|
81
|
+
- When the agent's deliverable paths are not known up front, it hands back an
|
|
82
|
+
unstaged diff and the orchestrator commits — under the same rule. The
|
|
83
|
+
orchestrator commits one handed-back diff at a time, stages that agent's
|
|
84
|
+
declared paths only, and is equally forbidden from `git add -A` / `git add .`
|
|
85
|
+
/ `git commit -a` while a parallel stage is in flight. Being the committer
|
|
86
|
+
does not exempt it; in degraded mode it is the only committer, so a sweeping
|
|
87
|
+
stage there mixes every sibling's work into one commit.
|
|
88
|
+
- Isolation requirements for concurrent stages are defined once in
|
|
89
|
+
`constitution/workflow.md#concurrency-stage-independent-mandatory`.
|
|
90
|
+
|
|
30
91
|
## Work Orders Summary
|
|
31
92
|
|
|
32
93
|
Every major artifact in the stage should include this table schema:
|
|
33
94
|
|
|
34
|
-
| Step | Role (sub-agent) | Task title | Input (refs) | Output (refs) | Status (PASS/REVISE) |
|
|
35
|
-
| ---- | ---------------- | ---------- | ------------ | ------------- |
|
|
36
|
-
| 1 | <role> | <task> | <refs> | <refs> | PASS/REVISE |
|
|
95
|
+
| Step | Role (sub-agent) | Agent instance | Task title | Input (refs) | Output (refs) | Status (PASS/REVISE/PENDING) |
|
|
96
|
+
| ---- | ---------------- | -------------- | ---------- | ------------ | ------------- | ---------------------------- |
|
|
97
|
+
| 1 | <role> | <instance id> | <task> | <refs> | <refs> | PASS/REVISE/PENDING |
|
|
37
98
|
|
|
38
99
|
- `Output (refs)` should point to in-file anchors or relative evidence paths.
|
|
100
|
+
- `Agent instance` is a run-stable identifier for the sub-agent that actually performed the step
|
|
101
|
+
(platform-supplied id where available, otherwise `<role>#<n>` assigned in order of first use).
|
|
102
|
+
It exists so an author→reviewer collision is detectable after the fact from the evidence alone;
|
|
103
|
+
the same instance appearing in an authoring step and in a review step over the same artifact is
|
|
104
|
+
a reviewer-independence violation.
|
|
105
|
+
- `PENDING` records a gate that could not be run — the only honest status for the exhausted-budget
|
|
106
|
+
branch below, which mandates it. It is never a substitute for `PASS`: DONE stays blocked while
|
|
107
|
+
any row is `PENDING`, and the stage stays resumable. A skill that allows only `PASS`/`REVISE`
|
|
108
|
+
would force an agent on that path to either break the schema or mislabel an unrun gate.
|
|
39
109
|
|
|
40
110
|
## Reviewer Gate Baseline
|
|
41
111
|
|
|
42
112
|
- Final completion gate must be delegated to an independent reviewer.
|
|
113
|
+
|
|
114
|
+
### Definition: independent reviewer (NORMATIVE)
|
|
115
|
+
|
|
116
|
+
An **independent reviewer** is a sub-agent that did **not** author or edit any artifact under
|
|
117
|
+
review in this run.
|
|
118
|
+
|
|
119
|
+
- The protected invariant is independence from authorship, not reviewer instance identity.
|
|
120
|
+
An agent that produced or modified none of the artifacts under review is independent even if
|
|
121
|
+
it filled another role earlier in the run; an agent that drafted or edited one of them is not
|
|
122
|
+
independent, however it is routed.
|
|
123
|
+
- Independence is judged per review target, over the whole run — not per phase. Authoring in an
|
|
124
|
+
earlier phase disqualifies the agent from reviewing that artifact in a later one.
|
|
125
|
+
- Role name alone never establishes independence. Routing dispatches by role; independence is a
|
|
126
|
+
separate constraint the routed agent must satisfy and attest to.
|
|
127
|
+
- A reviewer that discovers it authored or edited a review target MUST stop, declare the
|
|
128
|
+
conflict, and hand the same evidence set to a non-participating reviewer. It MUST NOT return
|
|
129
|
+
`PASS` on an artifact it authored.
|
|
130
|
+
- This definition governs every skill. Skill-local wording (e.g. `qfai-configure`'s "a reviewer
|
|
131
|
+
who did not modify the config") is an instance of it, not a competing rule.
|
|
132
|
+
|
|
43
133
|
- Reviewers must verify Drift Protocol enforcement.
|
|
44
134
|
- Reviewers must verify test-layer policy enforcement when relevant.
|
|
45
135
|
- Do not treat test volume ratios or floors as hard gates unless the skill explicitly says so.
|
|
46
136
|
- Do not declare DONE until all routed blocking reviewers return `PASS`.
|
|
47
137
|
- Every reviewer returning `FAIL` or `REVISE` must include a concrete fix proposal.
|
|
48
138
|
|
|
139
|
+
### Round budget (MUST)
|
|
140
|
+
|
|
141
|
+
- **Two rounds per reviewer per artifact.** Round 1 is the initial review;
|
|
142
|
+
round 2 reviews the fixes. **The budget is spent the moment round 2 returns
|
|
143
|
+
`REVISE`**: the orchestrator MUST NOT start a third review, and MUST stop and
|
|
144
|
+
escalate to the user with the open findings, the fixes already applied, and a
|
|
145
|
+
recommendation. The decision point is round 2's verdict, never a prediction
|
|
146
|
+
about a review that must not run.
|
|
147
|
+
- Escalation is not failure. The artifact stays at its current status and the
|
|
148
|
+
user decides: accept with the finding recorded as an Open Question, apply a
|
|
149
|
+
named fix, or drop the item from scope.
|
|
150
|
+
- **Completion after escalation.** The user's decision is the exception to
|
|
151
|
+
"no DONE until all blocking reviewers `PASS`", so the escalation has an exit:
|
|
152
|
+
- _Accept as Open Question_ or _drop from scope_ — the artifact may reach
|
|
153
|
+
DONE with the finding recorded; the reviewer's outstanding `REVISE` is
|
|
154
|
+
superseded by the recorded user decision. Cite the decision where the
|
|
155
|
+
stage records decisions (`*_delta.md` / `07_Decisions.md` / a Change
|
|
156
|
+
Request).
|
|
157
|
+
- _Apply a named fix_ — one **verification review** of exactly that fix is
|
|
158
|
+
permitted and does not consume budget (it is round 2b, not round 3). Its
|
|
159
|
+
remit is the named fix only. It may not raise findings unrelated to that
|
|
160
|
+
fix, but a defect the fix **introduced or exposed** is in remit and MUST be
|
|
161
|
+
reported rather than passed over: verifying only the named lines and
|
|
162
|
+
returning `PASS` while a regression sits next to them is a false `PASS`.
|
|
163
|
+
Such a finding escalates immediately (see the severity floor below) and
|
|
164
|
+
still does not start a round 3. The review returns `PASS` or escalates
|
|
165
|
+
again.
|
|
166
|
+
- **One 2b per artifact, total.** The verification review is free of budget,
|
|
167
|
+
not unbounded: a second escalation on the same artifact MUST NOT be
|
|
168
|
+
answered with another _apply a named fix_ + 2b cycle. Without this cap the
|
|
169
|
+
two rules compose into a loop — 2b costs nothing, and escalating again is
|
|
170
|
+
always allowed — so the gate has no guaranteed end. At the second
|
|
171
|
+
escalation the user is offered only _accept as Open Question_ or _drop the
|
|
172
|
+
item from scope_ (subject to the severity floor below); if the floor
|
|
173
|
+
withholds both, the artifact does not reach DONE and the stage stops with
|
|
174
|
+
the finding recorded.
|
|
175
|
+
- **Severity floor on the exit.** _Accept as Open Question_ is NOT available
|
|
176
|
+
for a finding that names a concrete security defect, data loss or
|
|
177
|
+
corruption, or a correctness defect that would break a released contract.
|
|
178
|
+
Present the user only _apply a named fix_ or _drop the item from scope_ for
|
|
179
|
+
those, and say why the third option is withheld. Without this the general
|
|
180
|
+
exit is a route around "deferring such a finding to an Open Question so a
|
|
181
|
+
`PASS` can be returned is prohibited" — one that needs no lateness and no
|
|
182
|
+
reviewer consent, only a user click.
|
|
183
|
+
- The round number MUST be recorded on each reviewer response
|
|
184
|
+
(`Round:` in the shared response template).
|
|
185
|
+
|
|
186
|
+
### Convergence (MUST)
|
|
187
|
+
|
|
188
|
+
- A finding first raised in round N > 1 MUST state why it was not raisable in
|
|
189
|
+
round N-1 — the fix introduced it, or the fix exposed it. A finding that was
|
|
190
|
+
raisable in round 1 and was not raised is **out of budget**: record it as an
|
|
191
|
+
Open Question or a `*_delta.md` Decision Record for the owning stage, do not
|
|
192
|
+
block on it.
|
|
193
|
+
- A reviewer MUST NOT open a new blocking _class_ of finding after the artifact
|
|
194
|
+
under review has been declared stable. New classes go to the owning stage.
|
|
195
|
+
- **Severity overrides lateness.** The out-of-budget rule is about review
|
|
196
|
+
discipline, not about shipping known harm. A late finding that names a
|
|
197
|
+
concrete security defect, data loss or corruption, or a correctness defect
|
|
198
|
+
that would break a released contract is **not** deferrable: the orchestrator
|
|
199
|
+
stops and escalates to the user immediately, exactly as it does when the
|
|
200
|
+
round budget is spent. It is still not a third round — no further review is
|
|
201
|
+
started, the finding goes straight to the user with its evidence. Deferring
|
|
202
|
+
such a finding to an Open Question so a `PASS` can be returned is prohibited.
|
|
203
|
+
That prohibition does not depend on lateness or on who proposes the deferral:
|
|
204
|
+
the escalation exit in the round budget withholds _Accept as Open Question_
|
|
205
|
+
for this same class, so a user choice cannot supersede it either.
|
|
206
|
+
|
|
207
|
+
### Reviewer remit (in scope per stage)
|
|
208
|
+
|
|
209
|
+
A finding outside the reviewing stage's remit is recorded and deferred, never
|
|
210
|
+
blocking:
|
|
211
|
+
|
|
212
|
+
| Stage | In scope | Out of scope (record and defer) |
|
|
213
|
+
| ------------------ | ----------------------------------------------------------------- | ---------------------------------------------- |
|
|
214
|
+
| `/qfai-discussion` | Requirement clarity, scope boundary, decision traceability | Spec structure, runtime behavior |
|
|
215
|
+
| `/qfai-sdd` | Spec / contract consistency, testability, traceability edges | Runtime enforcement correctness, code quality |
|
|
216
|
+
| `/qfai-atdd` | Obligation coverage, layer placement, annotation validity | Implementation structure |
|
|
217
|
+
| `/qfai-implement` | Code quality, spec alignment of the item, RED/GREEN evidence | Upstream spec content, contract design |
|
|
218
|
+
| `/qfai-configure` | Config / manifest validity and the surfaces the run generated | Spec content, implementation structure |
|
|
219
|
+
| `/qfai-verify` | Gate execution, evidence completeness, report / artifact fidelity | Authoring quality of the artifacts it verifies |
|
|
220
|
+
| `/web-research` | Source authority and freshness, citation accuracy, claim support | Spec content, implementation structure |
|
|
221
|
+
|
|
222
|
+
**Fallback for any stage not listed.** A stage that references this baseline
|
|
223
|
+
without a row above has, as its remit, the artifacts that stage itself
|
|
224
|
+
produces; everything upstream of them is out of scope, recorded and deferred.
|
|
225
|
+
Add the row when a new stage starts routing blocking reviewers, so the
|
|
226
|
+
in/out split is not re-derived per run.
|
|
227
|
+
|
|
228
|
+
### Finding provenance (MUST)
|
|
229
|
+
|
|
230
|
+
- Every finding must declare a severity (`blocking` or `advisory`) and a `Traces to:` value.
|
|
231
|
+
- `Traces to:` names what the finding enforces. Legal values:
|
|
232
|
+
- an upstream obligation — an `AC-*`, `BR-*`, `TC-*`, `CON-*` ID, or a named
|
|
233
|
+
constitution/catalog rule;
|
|
234
|
+
- `defect:correctness`, `defect:security`, or `defect:code-quality` — a defect demonstrable from
|
|
235
|
+
the changed artifacts themselves, cited together with the evidence that demonstrates it. See
|
|
236
|
+
`drift-protocol.md#defect-or-new-scope-decide-this-first`. A reviewer who can show the
|
|
237
|
+
deliverable is wrong on its own terms does not need an `AC-*` to say so;
|
|
238
|
+
- `none` — reviewer-originated scope, i.e. a new product obligation upstream never asked for.
|
|
239
|
+
- A finding whose `Traces to:` is `none` MUST be recorded as `advisory`. It cannot be `blocking`,
|
|
240
|
+
and it cannot gate `DONE`.
|
|
241
|
+
- An advisory finding is routed to the Change Request / Open Question path defined in
|
|
242
|
+
`drift-protocol.md#reviewer-originated-obligations`, not to the implementer.
|
|
243
|
+
- Only `blocking` findings — those citing an upstream obligation or a defect class — force
|
|
244
|
+
`REVISE`.
|
|
245
|
+
|
|
246
|
+
### Reviewer budget exhausted
|
|
247
|
+
|
|
248
|
+
A blocking review that cannot be delegated because the agent budget is spent is a `saturated`
|
|
249
|
+
failure, not a licence to skip the gate or to self-review.
|
|
250
|
+
|
|
251
|
+
- First apply the `saturated` bounded retry. A freed slot is the preferred outcome.
|
|
252
|
+
- If retries are exhausted, a reviewer role MAY be reused sequentially with a cleared context,
|
|
253
|
+
provided the reviewer did not author or edit any artifact under review in this run. The
|
|
254
|
+
protected invariant is independence from authorship, not reviewer instance identity.
|
|
255
|
+
- Record the reuse in the Work Orders Summary (`Task title` prefixed `re-review (sequential reuse)`).
|
|
256
|
+
- If even sequential reuse is impossible, hard stop with the review gate recorded as `PENDING`
|
|
257
|
+
rather than `PASS`. `PENDING` is not `PASS`; DONE stays blocked and the stage stays resumable.
|
|
258
|
+
- Never record a waived or self-performed review as `PASS`.
|
|
259
|
+
|
|
49
260
|
## Work order template
|
|
50
261
|
|
|
51
262
|
```text
|
|
@@ -54,10 +265,12 @@ Role: <sub-agent role>
|
|
|
54
265
|
Goal: <what to decide/produce>
|
|
55
266
|
Inputs (refs):
|
|
56
267
|
- <file/section>
|
|
268
|
+
- constitution/drift-protocol.md#core-rule <!-- the protected set, in front of the agent -->
|
|
57
269
|
Constraints:
|
|
58
270
|
- must: enforce Drift Protocol
|
|
59
271
|
- must: follow applicable test-layer or validation policy
|
|
60
|
-
- must_not: patch upstream artifacts directly
|
|
272
|
+
- must_not: patch upstream artifacts directly; every upstream change requires
|
|
273
|
+
STOP + Change Request + owner rerun per constitution/drift-protocol.md
|
|
61
274
|
Output format:
|
|
62
275
|
- <headings / bullet schema>
|
|
63
276
|
Quality bar:
|
|
@@ -68,15 +281,213 @@ Quality bar:
|
|
|
68
281
|
## Reviewer response template
|
|
69
282
|
|
|
70
283
|
```text
|
|
284
|
+
Round: 1 | 2 | 2b
|
|
71
285
|
Result: PASS | REVISE
|
|
286
|
+
Reviewed revision: <git rev> | working-tree+<content hash>
|
|
287
|
+
Audited evidence hash: <content hash of the evidence read> # one line per TDD-ID on a T1 group
|
|
288
|
+
Authored/edited under review: none | <artifact refs this reviewer authored or edited in this run>
|
|
72
289
|
Findings:
|
|
73
|
-
- <issue>
|
|
290
|
+
- <issue> | Severity: blocking|advisory | Traces to: <AC-*/BR-*/TC-*/CON-*/rule-name|defect:correctness|defect:security|defect:code-quality|none>
|
|
74
291
|
Required fixes:
|
|
75
|
-
- <action>
|
|
292
|
+
- <action> # blocking findings only
|
|
293
|
+
Advisory / Change Request proposals:
|
|
294
|
+
- <proposal> # findings with `Traces to: none`; not a DONE gate
|
|
76
295
|
Evidence checked:
|
|
77
296
|
- <refs>
|
|
78
297
|
```
|
|
79
298
|
|
|
299
|
+
`Round` is required — the round budget above is counted from it. `2b` is the
|
|
300
|
+
post-escalation verification review of a user-named fix.
|
|
301
|
+
|
|
302
|
+
- `Audited evidence hash` is REQUIRED wherever the evidence tree is what the
|
|
303
|
+
verdict is about. `Reviewed revision` excludes `.qfai/evidence/**` — see below
|
|
304
|
+
for why it has to — so nothing else pins the RED/GREEN output this reviewer
|
|
305
|
+
read, or the coverage justifications it accepted: edited after a PASS they
|
|
306
|
+
leave the revision unchanged and the verdict reads as fresh. Hash the row's
|
|
307
|
+
**phase-authored** entry, meaning the entry with the reviewer-appended fields
|
|
308
|
+
removed — which is exactly what was read — plus
|
|
309
|
+
`.qfai/evidence/coverage-depth-<spec-id>.md` where the row has one, in the
|
|
310
|
+
same `path + NUL + blob hash` manifest form, sorted by path. **The reviewer
|
|
311
|
+
computes it**, on the evidence it read; an orchestrator filling it in on the
|
|
312
|
+
reviewer's behalf is recording something nobody audited.
|
|
313
|
+
- **How to compute it, exactly.** The subject is part of a file, so a
|
|
314
|
+
file-level manifest alone is ambiguous, and two readers hashing different
|
|
315
|
+
extents produce a verdict that is either always stale or never checked. One
|
|
316
|
+
procedure, in four steps:
|
|
317
|
+
1. **Extract — the fields this observation could read, named.** Not "the
|
|
318
|
+
section minus what is written later": the entry keeps growing after every
|
|
319
|
+
observation, so subtracting a list only moved the problem to the next
|
|
320
|
+
field added. The RED gatekeeper hashes an entry that has no GREEN yet, the
|
|
321
|
+
GREEN gatekeeper one that has no `Refactor verify`, and each was stale as
|
|
322
|
+
soon as the phase wrote on. **Three subjects, from the row's
|
|
323
|
+
`### <TDD-ID>` section** of the evidence file its `Layer` owns — the
|
|
324
|
+
heading line through the line before the next `###` heading that names a
|
|
325
|
+
`TDD-` id, or the next `##` / `#` heading, or end of file, **counting only
|
|
326
|
+
headings outside a fenced block** (` ``` ` / `~~~`, closed at that length or
|
|
327
|
+
longer; a body that would close its own fence gets a longer one). Every
|
|
328
|
+
recorded output is fenced for this reason: a test asserting on Markdown
|
|
329
|
+
prints `## ...` of its own, and a boundary that took it dropped the GREEN,
|
|
330
|
+
the `Oracle proof` and the round evidence out of the subject. Each takes
|
|
331
|
+
only its own fields, in the order the contract lists them:
|
|
332
|
+
- **Row identity, in all three**: `TDD-ID`, `Layer`, `Test file` and
|
|
333
|
+
`Selector` — copied from the ledger, which the revision excludes.
|
|
334
|
+
Without them, changing `Selector` after a PASS to another valid test in
|
|
335
|
+
the same file left every hash and revision unmoved, and a verdict that
|
|
336
|
+
only ran the old selector stood as evidence for the new one. Mutable
|
|
337
|
+
bookkeeping — `Status`, `Evidence` — stays out: it moves on its own.
|
|
338
|
+
|
|
339
|
+
**The copy is checked against the ledger, not trusted** — the four
|
|
340
|
+
identity fields **and the obligation reference**. Hashing a value the
|
|
341
|
+
entry already holds proves only that the entry has not changed: edit
|
|
342
|
+
`Selector` in `test-list.md` after the PASS and copy, hash and revision
|
|
343
|
+
are all unmoved. Gate item 10 reads the four fields from
|
|
344
|
+
`test-list.md` and requires them to equal the copy; a difference is the
|
|
345
|
+
row moving under its own evidence, and the verdict is not fresh. The
|
|
346
|
+
obligation is on that list for the same reason: change `TC-Refs` /
|
|
347
|
+
`US-Refs` / `CON-API-Refs` alone after the PASS and the entry still holds
|
|
348
|
+
the old copy, so a verdict about one requirement stands for another.
|
|
349
|
+
|
|
350
|
+
- **RED observation**: the obligation reference the row's
|
|
351
|
+
`Layer` selects (`TC-ref`, or `US-ref` on `E2E` and `CON-API-ref` on
|
|
352
|
+
`API` — an ATDD-owned row has no `TC-ref`, so naming only that one left
|
|
353
|
+
its obligation outside every hash), `RED test hash`, the row's own
|
|
354
|
+
transient revision (`RED revision` or `Falsifiability revision`), and the
|
|
355
|
+
RED pair or the falsifiability trio with `RED failure mode`. **Not
|
|
356
|
+
`Revision`**: it names the tree the GREEN landed at and does not exist
|
|
357
|
+
yet, so including it made every correct RED PASS stale at GREEN.
|
|
358
|
+
- **GREEN observation**: the RED subject plus `Revision`, the GREEN pair,
|
|
359
|
+
`Oracle proof` and, where the row has one, `Replacement proof revision` —
|
|
360
|
+
it addresses the tree a re-taken proof ran against, which the revision
|
|
361
|
+
does not reach, so a subject without it let that proof be attributed to a
|
|
362
|
+
tree it never ran on.
|
|
363
|
+
- **Stage review** (a `completion-reviewer` judging a stage rather than a
|
|
364
|
+
row — a spec with no ATDD-owned rows is the ordinary case, and
|
|
365
|
+
`qfai-atdd/SKILL.md` treats zero as a legitimate count): the stage
|
|
366
|
+
evidence file **whole**, under its repo-relative path, plus
|
|
367
|
+
`.qfai/evidence/coverage-depth-<spec-id>.md` whole — **minus the
|
|
368
|
+
`## Final status` section**, which the P8 reviewer's own answer fills in.
|
|
369
|
+
Whole-file included it, so writing PASS and the confirmer's name straight
|
|
370
|
+
after hashing made the verdict stale on being recorded. There is no
|
|
371
|
+
`### <TDD-ID>` section to extract and no per-row boundary to draw, so the
|
|
372
|
+
rest of the file is the subject; step 2 normalizes it, steps 3 and 4 are
|
|
373
|
+
unchanged.
|
|
374
|
+
Without this the final review of such a spec either omitted a required
|
|
375
|
+
field or PASSed with nothing pinning the evidence it read.
|
|
376
|
+
- **Branch 3** (`exception`): row identity, the obligation reference the
|
|
377
|
+
row's `Layer` selects, the `DR-ID`, and the `DR-*` artifact it names. The
|
|
378
|
+
obligation is what the DR says cannot be observed, so a subject without
|
|
379
|
+
it let the reference be pointed at a different requirement after the PASS
|
|
380
|
+
— the ledger is out of the revision, and item 10's identity check covers
|
|
381
|
+
four fields, so nothing moved. Item 10 checks this one against the ledger
|
|
382
|
+
as well. There is no RED and no GREEN on this branch — the
|
|
383
|
+
claim is that neither could be had — so the DR **is** the evidence, and
|
|
384
|
+
leaving it out of every subject let the pointer be swapped after the PASS
|
|
385
|
+
for another existing `DR-*`, one already waived perhaps, with the
|
|
386
|
+
revision and the hash both unmoved. Gate item 10 also requires the
|
|
387
|
+
verdict to name the `DR-ID` the row currently carries.
|
|
388
|
+
- **Completion review** (`completion-reviewer` / `implementation-reviewer`):
|
|
389
|
+
the GREEN subject plus `Refactor verify command` / `result`, the
|
|
390
|
+
`Shared-artifact re-verify` block when the row has one — it records the
|
|
391
|
+
earlier rows' re-runs and re-taken proofs, which these reviewers are the
|
|
392
|
+
ones who audit, so leaving it out let it be edited or deleted without
|
|
393
|
+
moving either hash — and, from every `### Round N` block the row
|
|
394
|
+
carries, that block's **phase-authored** fields only. `Round N: reviewer verdict` is written by these reviewers
|
|
395
|
+
after they have read the block, so taking the whole block put their own
|
|
396
|
+
line inside what they hashed.
|
|
397
|
+
|
|
398
|
+
A field absent at that point contributes nothing — it is not a placeholder
|
|
399
|
+
and not an error. Nothing written after an observation is in its subject,
|
|
400
|
+
which is what makes a verdict re-checkable at all: gate item 10 recomputes
|
|
401
|
+
each one against its own list. **When in doubt about a new field, ask which
|
|
402
|
+
observation could have read it**; that is the whole rule, and it is why the
|
|
403
|
+
subjects are named rather than derived by subtraction. **A field written
|
|
404
|
+
after every reviewer is in no subject at all** — the checkpoint pair, which
|
|
405
|
+
the revision and the pack seal both miss — so it carries a seal of its own,
|
|
406
|
+
taken as it is written and recomputed by the gate that reads it.
|
|
407
|
+
|
|
408
|
+
2. **Normalize.** LF line endings; strip trailing whitespace from every line;
|
|
409
|
+
drop leading and trailing blank lines; end with exactly one newline.
|
|
410
|
+
3. **Serialize.** One record per artifact — the repo-relative path, a NUL
|
|
411
|
+
byte, then the SHA-256 of that artifact's normalized bytes — sorted by
|
|
412
|
+
path, joined with newlines. **This is the audit hash, not the working-tree
|
|
413
|
+
revision**: that one has its own four steps, including the untracked
|
|
414
|
+
record's `kind` and `mode`, in
|
|
415
|
+
`../skills/qfai-implement/references/evidence-revision.md`, and restating
|
|
416
|
+
it here is how the two came to disagree. Two artifacts: the extracted
|
|
417
|
+
section, recorded under the evidence file's path, and
|
|
418
|
+
on a branch-3 row the `DR-*` artifact the row names, whole, under its
|
|
419
|
+
repo-relative path — the subject says the DR is that branch's evidence, and
|
|
420
|
+
a subject with no record for it is a hash that does not move when the DR
|
|
421
|
+
text changes; and
|
|
422
|
+
the part of `.qfai/evidence/coverage-depth-<spec-id>.md` that belongs to
|
|
423
|
+
this row's obligation — not the file whole, and matched **exactly**. A
|
|
424
|
+
row may legitimately carry several (`TC-Refs: TC-0001, TC-0002`), so split
|
|
425
|
+
the copied column on commas first and take each id in the order the column
|
|
426
|
+
lists them; comparing the whole column against a single-id matrix cell
|
|
427
|
+
matched nothing, and a row with two obligations had no matrix rows in its
|
|
428
|
+
subject at all. For each id: the table rows whose obligation cell equals it
|
|
429
|
+
(`TC-0001` does not match `TC-00011`), plus each justification paragraph
|
|
430
|
+
whose first line names it. A justification that names no obligation belongs to
|
|
431
|
+
none of them and is left out; "everything after the table" was the other
|
|
432
|
+
reading, and two readers taking one each computed different hashes from one
|
|
433
|
+
file. The matrix is one document
|
|
434
|
+
for the spec and a later `/qfai-atdd` run recomputes it, so hashing all of
|
|
435
|
+
it made every existing verdict stale when an unrelated obligation's cell
|
|
436
|
+
moved, and a `done` row has no re-review path to clear that. Take the table
|
|
437
|
+
rows whose obligation column matches, with any justification lines under
|
|
438
|
+
them, normalized by step 2 as well; a row whose obligation appears nowhere
|
|
439
|
+
in the matrix contributes nothing.
|
|
440
|
+
4. **Hash.** SHA-256 of that record list; record the hex digest.
|
|
441
|
+
|
|
442
|
+
**A T1 coherent group is one pass and several rows** (`volume-policy.md`).
|
|
443
|
+
One hash over a representative would leave the other members' evidence free to
|
|
444
|
+
change after the PASS, and a private concatenation of their sections has no
|
|
445
|
+
defined member order or record shape for gate item 10 to reproduce. Record
|
|
446
|
+
**one `Audited evidence hash` per `TDD-ID` in the group**, each by these four
|
|
447
|
+
steps over that row's own subject, listed in the verdict beside the id it
|
|
448
|
+
belongs to. Nothing about a group is special then; it is the single-row rule
|
|
449
|
+
applied as many times as the group has members.
|
|
450
|
+
|
|
451
|
+
Gate item 10 runs the same four steps. A row with no coverage-depth file, or
|
|
452
|
+
none whose matrix names its obligation, has one record rather than a
|
|
453
|
+
placeholder — an absent artifact contributes nothing, not a name with an empty
|
|
454
|
+
hash.
|
|
455
|
+
|
|
456
|
+
- `Reviewed revision` is REQUIRED. It names the state the verdict describes — a `git rev-parse HEAD`
|
|
457
|
+
value, or `working-tree+<content hash>` when the tree is uncommitted, computed by
|
|
458
|
+
the four-step procedure in `../skills/qfai-implement/references/evidence-revision.md`:
|
|
459
|
+
collect, exclude, serialize, hash. **Do not restate it**: it was restated here as
|
|
460
|
+
`path + NUL + hash`, the canonical since gained the untracked record's `kind`
|
|
461
|
+
and `mode`, and the two spellings gave producer and reviewer different
|
|
462
|
+
addresses for one tree on any run with an untracked file in it — which a new
|
|
463
|
+
acceptance test is.
|
|
464
|
+
**The ledger, the evidence tree and the review pack are excluded**, exactly
|
|
465
|
+
as that contract says — `.qfai/specs/*/tdd/test-list.md`, `.qfai/evidence/**`
|
|
466
|
+
and `.qfai/review/**`. The pack is on the list because a project may
|
|
467
|
+
legitimately track it, and then storing R01 moves the address R02 computes,
|
|
468
|
+
so items 7-8 could not PASS on one revision. The phases write `test-list.md`
|
|
469
|
+
and `.qfai/evidence/**` between the
|
|
470
|
+
GREEN and the reviews, so hashing all of `git diff HEAD` here produced a
|
|
471
|
+
`Reviewed revision` that could never equal the phase-authored `Revision` —
|
|
472
|
+
and gate item 10 wants them equal. `references/evidence-revision.md` is the
|
|
473
|
+
field's contract and this restates it; the two have to agree or the verdict
|
|
474
|
+
cannot be re-checked. **Not** a
|
|
475
|
+
`git status --porcelain` digest: that names the changed paths and their states, so it does not
|
|
476
|
+
move when only the content of an already-changed file does, and a stale PASS reads as fresh.
|
|
477
|
+
Without
|
|
478
|
+
it a verdict cannot be re-checked, cannot be invalidated by a later commit, and cannot be told
|
|
479
|
+
apart from a stale one, so "stale evidence MUST NOT be reused" has nothing to compare against.
|
|
480
|
+
Reviewers are dispatched against the integrated tree by design, so the tree is legitimately
|
|
481
|
+
allowed to move under them: an honest, independent verdict on a tree that no longer exists is the
|
|
482
|
+
normal failure this field addresses. If the tree changed mid-review, say so and name the revision
|
|
483
|
+
the ruling is pinned to.
|
|
484
|
+
- `Authored/edited under review` is REQUIRED. A response omitting it is not a valid review verdict.
|
|
485
|
+
- Anything other than `none` is a declared independence conflict: the verdict cannot be `PASS`,
|
|
486
|
+
and the review must be handed to a non-participating reviewer (see
|
|
487
|
+
`Definition: independent reviewer`).
|
|
488
|
+
- `Result: REVISE` is legal only when at least one finding is `Severity: blocking`. A response
|
|
489
|
+
whose findings are all advisory returns `Result: PASS` with the proposals attached.
|
|
490
|
+
|
|
80
491
|
### Verdict vocabulary
|
|
81
492
|
|
|
82
493
|
- Reviewer responses in-flight use `Result: PASS | REVISE` (this file).
|
|
@@ -3,6 +3,32 @@
|
|
|
3
3
|
Use this document to keep SKILL bodies compact.
|
|
4
4
|
Skill files should reference this baseline and only restate skill-specific additions or overrides.
|
|
5
5
|
|
|
6
|
+
## SKILL.md Authoring Shape (Mandatory)
|
|
7
|
+
|
|
8
|
+
A `SKILL.md` states the contract and points at the file that carries the detail.
|
|
9
|
+
It is not where the detail lives.
|
|
10
|
+
|
|
11
|
+
- **Keep in `SKILL.md`**: what the skill is for, its non-goals, its hard
|
|
12
|
+
constraints, the phase/step order, and the gate conditions. Enough for an
|
|
13
|
+
agent to know what it must do and when it is done.
|
|
14
|
+
- **Move out**: command sets, table schemas, field-by-field contracts, worked
|
|
15
|
+
procedures, checklists and rationale. These go under the skill's own
|
|
16
|
+
directory:
|
|
17
|
+
- `references/` — normative detail the skill body cites (`references/<topic>.md`)
|
|
18
|
+
- `templates/` — artifacts the skill produces, as fillable skeletons
|
|
19
|
+
- `examples/` — worked instances that illustrate, and bind, nothing
|
|
20
|
+
- **One topic per file.** Do not replace an oversized `SKILL.md` with an
|
|
21
|
+
oversized `references/everything.md`; that is the same problem one directory
|
|
22
|
+
down. Split by topic and keep each file readable on its own — a reader who
|
|
23
|
+
followed one pointer should not have to scan past three unrelated subjects to
|
|
24
|
+
reach the one they came for.
|
|
25
|
+
- **Every pointer resolves.** A `SKILL.md` line that moves detail out must name
|
|
26
|
+
the file (and anchor, when the file covers more than one topic) so the reader
|
|
27
|
+
is never left guessing where the rule went.
|
|
28
|
+
|
|
29
|
+
A hard line ceiling backs this up in the asset tests, but it is a backstop, not
|
|
30
|
+
the rule. A skill approaching it is a signal to move a section out.
|
|
31
|
+
|
|
6
32
|
## User Questions (AskUserQuestion Protocol)
|
|
7
33
|
|
|
8
34
|
- When a question to the user is needed, use AskUserQuestion if the tool is available.
|
|
@@ -11,6 +37,53 @@ Skill files should reference this baseline and only restate skill-specific addit
|
|
|
11
37
|
- Preserve structured choice semantics when falling back.
|
|
12
38
|
- State why AskUserQuestion was unavailable.
|
|
13
39
|
|
|
40
|
+
## Canonical qfai Launcher (Mandatory)
|
|
41
|
+
|
|
42
|
+
- **Launcher preflight — run once, before the first gate.** Confirm the project
|
|
43
|
+
resolves qfai from its own dependencies. Either proof is sufficient:
|
|
44
|
+
- **A local binary at `node_modules/.bin/qfai`.** The normal case for npm,
|
|
45
|
+
pnpm and Yarn configured with `nodeLinker: node-modules`.
|
|
46
|
+
- **A Plug'n'Play install.** Yarn Berry's default `nodeLinker: pnp` writes no
|
|
47
|
+
`node_modules/.bin`, so the file check alone would report a correctly
|
|
48
|
+
installed project as UNRUN forever. Accept it when the project has a
|
|
49
|
+
`.pnp.cjs` / `.pnp.loader.mjs` at its root and lists `qfai` in
|
|
50
|
+
`package.json` `dependencies` / `devDependencies`; `yarn exec qfai --help`
|
|
51
|
+
exiting 0 is the direct confirmation.
|
|
52
|
+
|
|
53
|
+
If neither proof holds, every gate below is UNRUN: report it as a blocker and
|
|
54
|
+
stop. The fix is to install the dependency (`npm i -D qfai`, or the pnpm /
|
|
55
|
+
yarn equivalent). `qfai` does not add itself to `package.json` on init, so a
|
|
56
|
+
project bootstrapped with `npx qfai init` alone has no local dependency yet.
|
|
57
|
+
|
|
58
|
+
- Once the preflight passes, invoke every gate through the launcher that proof
|
|
59
|
+
established:
|
|
60
|
+
- local binary -> `npx qfai …`, which resolves to it.
|
|
61
|
+
`node_modules/.bin/qfai …` is the same thing spelled out; prefer it when
|
|
62
|
+
PATH reachability is uncertain.
|
|
63
|
+
- Plug'n'Play -> `yarn exec qfai …` (equivalently `yarn qfai …`), which sets
|
|
64
|
+
up the PnP environment for the child process. Do **not** fall back to
|
|
65
|
+
`npx qfai` there: outside the PnP runtime it cannot see the workspace
|
|
66
|
+
dependency and would fetch a remote copy instead.
|
|
67
|
+
|
|
68
|
+
Read every `npx qfai …` example in the shipped docs as "the launcher the
|
|
69
|
+
preflight established", not as a literal command for a PnP project.
|
|
70
|
+
|
|
71
|
+
- Never launch a gate as a bare `qfai` command: qfai is a project dependency,
|
|
72
|
+
not a global one, so that is `command not found` on a normal local install —
|
|
73
|
+
and a gate that cannot run is a gate that silently passes.
|
|
74
|
+
- The preflight is the guard, not a flag. `npx` runs "a command from a local
|
|
75
|
+
**or remote** npm package": with nothing resolvable locally it downloads and
|
|
76
|
+
runs one (non-interactive shells do this without prompting), and
|
|
77
|
+
`npx --no-install` still executes a copy already sitting in the npx cache.
|
|
78
|
+
Neither spelling can tell you the qfai that ran was this project's; only the
|
|
79
|
+
preflight can.
|
|
80
|
+
- If the launcher cannot be resolved at any point, the gate is UNRUN, not PASS.
|
|
81
|
+
Report it as a blocker instead of completing the stage.
|
|
82
|
+
- The same launcher is documented in `.qfai/assistant/catalog/tech.md` and
|
|
83
|
+
`.qfai/assistant/catalog/structure.md`. The CI workflow generated by
|
|
84
|
+
`npx qfai init` is the one deliberate exception: it runs before any project
|
|
85
|
+
install can be assumed and must still be able to bootstrap.
|
|
86
|
+
|
|
14
87
|
## FORMAT SSOT (Mandatory)
|
|
15
88
|
|
|
16
89
|
- Before writing or editing `.qfai/**`, read the relevant README/template/sample for the target artifact.
|
|
@@ -41,16 +114,60 @@ Rules:
|
|
|
41
114
|
|
|
42
115
|
## Gate Failure Autorepair Protocol
|
|
43
116
|
|
|
44
|
-
When validate, doctor, test, lint, typecheck, build, capture, or report gates fail:
|
|
117
|
+
When validate, doctor, test, lint, typecheck, build, capture, or report gates fail — **or when a blocking reviewer returns `REVISE`** (the in-flight verdict; `status: "FAIL"` is only what a review pack's `summary.json` serializes — see `shared-skill-delegation-baseline.md#verdict-vocabulary`):
|
|
45
118
|
|
|
46
119
|
- inspect exit code, logs, `validate.json`, and cited files before reporting;
|
|
47
120
|
- classify each finding as skill-owned artifact, upstream spec/contract, code/test defect, environment/tooling, or user decision;
|
|
48
121
|
- fix skill-owned artifacts and code/test defects autonomously when the fix is local and non-destructive;
|
|
49
|
-
- rerun the
|
|
122
|
+
- **upstream spec/contract findings: never repair.** STOP and follow `.qfai/assistant/constitution/drift-protocol.md` (Change Request + owner-skill rerun) — **even when the fix looks local and non-destructive, and even when it is one token and obviously correct**. "Local and non-destructive" is a permission for the two classes above it; it is not a test that upstream artifacts can pass. Ownership, not size, decides;
|
|
123
|
+
- environment/tooling findings: repair the environment when it is yours to repair (install a missing dev dependency, regenerate a lockfile, create a scratch directory). Stop for anything needing credentials, network access you do not have, or a change to CI configuration or the host machine;
|
|
124
|
+
- user decision findings: never decide by default. Record the question, state the option you would take and why, and stop — a decision taken silently to keep a gate green is the same failure as repairing upstream, one layer up;
|
|
125
|
+
- rerun the same failing gate after each fix batch, **and once with no intervening change** when the failure looks nondeterministic — see `#nondeterministic-gates` below. The confirmation rerun is bounded at one: after it the finding is classified, not re-rolled;
|
|
50
126
|
- do not weaken profiles, lower `--fail-on`, waive errors, invent evidence, or skip required reviewers;
|
|
51
|
-
- stop
|
|
52
|
-
|
|
53
|
-
|
|
127
|
+
- stop for destructive changes, **any upstream spec/contract finding**, ambiguous product/spec decisions, missing permissions/tools, or repeated no-progress failures — the stop list is closed over the classification above, so every class the agent is told to use has a defined next action;
|
|
128
|
+
- stop on **round count** as well as on lack of progress: a reviewer gate that would enter its third round escalates to the user, even when every round has made progress. See `shared-skill-delegation-baseline.md#round-budget-must`.
|
|
129
|
+
|
|
130
|
+
When stopping, report: cause, attempted fixes, remaining blocker, user action, retry gate, and **the work counts — how many items are complete, how many are blocked, and by which finding**.
|
|
131
|
+
|
|
132
|
+
The counts are not decoration. Restating the ownership rule does not change the incentive that breaks it: an agent facing "repair five upstream defects or report most of the batch as blocked" reaches for the repair because the alternative reads as failure. `26 items: 21 complete, 5 blocked on CON-DB-0007` is a report of work done, and it is what makes STOP a credible answer rather than a surrender. Blocked is a status, not a verdict on the run.
|
|
133
|
+
|
|
134
|
+
### Nondeterministic gates
|
|
135
|
+
|
|
136
|
+
A gate whose answer varies on identical inputs is a finding in its own right,
|
|
137
|
+
not a run to discard. The protocol classifies it as `environment/tooling`; what
|
|
138
|
+
follows is what that class obliges.
|
|
139
|
+
|
|
140
|
+
**When a gate fails and a rerun with no intervening change passes**, all of the
|
|
141
|
+
following are REQUIRED. A clean rerun on its own is not evidence for that gate.
|
|
142
|
+
|
|
143
|
+
- **Record it as an `environment/tooling` finding.** Not as a pass, and not as
|
|
144
|
+
a code/test defect — nothing was fixed between the two runs.
|
|
145
|
+
- **Disclose every run.** Report the results of all runs of that gate, in order,
|
|
146
|
+
with their commands. Reporting only the run that passed is
|
|
147
|
+
[selective reporting](#selective-reporting-is-invented-evidence) and is
|
|
148
|
+
forbidden: the evidence rules are satisfied by a clean run's command and
|
|
149
|
+
output, so nothing else stops it.
|
|
150
|
+
- **Re-run the failing selectors in isolation** and report that result too. A
|
|
151
|
+
selector that passes alone and fails in the suite is the signature of shared
|
|
152
|
+
state, not of a defect in that test.
|
|
153
|
+
- **Name the suspected cause**, concretely: a contended port, a shared database
|
|
154
|
+
or schema, an `os.tmpdir()` path, an un-namespaced cache or queue, ordering
|
|
155
|
+
between workers. "Flaky" is not a cause.
|
|
156
|
+
|
|
157
|
+
Do not fix the flake by rerunning until green, and do not fix it by weakening
|
|
158
|
+
the test. Either the shared resource is isolated per worker or the run is
|
|
159
|
+
serialised — both are real changes with a real cost, which is the point.
|
|
160
|
+
|
|
161
|
+
A gate reported this way has **not passed**. It is a blocker with a named cause,
|
|
162
|
+
and it goes in the stop report like any other.
|
|
163
|
+
|
|
164
|
+
#### Selective reporting is invented evidence
|
|
165
|
+
|
|
166
|
+
Reporting the clean run and omitting the red ones satisfies every existing
|
|
167
|
+
evidence rule — a real command, a real result, freshly obtained — and still
|
|
168
|
+
misrepresents what happened. Which of N runs is reported is itself part of the
|
|
169
|
+
evidence, so omitting runs of the same gate is on the same footing as inventing
|
|
170
|
+
one.
|
|
54
171
|
|
|
55
172
|
## Completion Contract (Shared)
|
|
56
173
|
|