qfai 2.1.0 → 2.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +327 -55
- package/assets/defaults/agent-routing/qfai-triage.yml +9 -0
- package/assets/defaults/workflows/change-tooling.yml +0 -4
- package/assets/defaults/workflows/run-owned-cleanup.yml +12 -0
- package/assets/init/.claude/settings.json +27 -9
- package/assets/init/.codex/hooks.json +20 -8
- package/assets/init/.qfai/assistant/agent/frontend-engineer.md +5 -0
- package/assets/init/.qfai/assistant/agent/implementation-reviewer.md +14 -1
- package/assets/init/.qfai/assistant/agent/product-experience-architect.md +4 -0
- package/assets/init/.qfai/assistant/agent/product-surface-reviewer.md +7 -0
- package/assets/init/.qfai/assistant/agent/qa-gatekeeper.md +29 -0
- package/assets/init/.qfai/assistant/agent/requirements-analyst.md +8 -0
- package/assets/init/.qfai/assistant/rule/communication.md +18 -8
- package/assets/init/.qfai/assistant/rule/constitution.md +5 -4
- package/assets/init/.qfai/assistant/rule/drift-protocol.md +72 -5
- package/assets/init/.qfai/assistant/rule/references/asset-authoring.md +72 -0
- package/assets/init/.qfai/assistant/rule/references/context-summary.md +30 -0
- package/assets/init/.qfai/assistant/rule/references/gate-failure-autorepair.md +42 -0
- package/assets/init/.qfai/assistant/rule/references/griller-recommendations.md +23 -0
- package/assets/init/.qfai/assistant/rule/references/host-backstops.md +60 -0
- package/assets/init/.qfai/assistant/rule/references/placeholder-scan.md +25 -0
- package/assets/init/.qfai/assistant/rule/references/reviewer-findings.md +41 -0
- package/assets/init/.qfai/assistant/rule/references/reviewer-remit.md +35 -0
- package/assets/init/.qfai/assistant/rule/references/worker-edit-boundary.md +48 -0
- package/assets/init/.qfai/assistant/rule/shared-skill-delegation-baseline.md +39 -162
- package/assets/init/.qfai/assistant/rule/shared-skill-operating-baseline.md +14 -145
- package/assets/init/.qfai/assistant/rule/stage-cost.md +1 -1
- package/assets/init/.qfai/assistant/rule/test-layers.md +42 -0
- package/assets/init/.qfai/assistant/rule/ui-definition-protocol.md +6 -1
- package/assets/init/.qfai/assistant/skill/qfai-configure/SKILL.md +11 -45
- package/assets/init/.qfai/assistant/skill/qfai-configure/references/principles.md +25 -0
- package/assets/init/.qfai/assistant/skill/qfai-discussion/SKILL.md +1 -1
- package/assets/init/.qfai/assistant/skill/qfai-discussion/references/discussion-completion-matrix.md +12 -9
- package/assets/init/.qfai/assistant/skill/qfai-discussion/references/ui-bearing-playbook.md +2 -4
- package/assets/init/.qfai/assistant/skill/qfai-discussion/references/ui_ux/platform_baselines.md +11 -0
- package/assets/init/.qfai/assistant/skill/qfai-discussion/references/ui_ux/review_audit_playbook.md +11 -0
- package/assets/init/.qfai/assistant/skill/qfai-discussion/references/ui_ux_best_practices.md +13 -42
- package/assets/init/.qfai/assistant/skill/qfai-discussion/templates/uiux/00_index.md +0 -16
- package/assets/init/.qfai/assistant/skill/qfai-grill/SKILL.md +1 -1
- package/assets/init/.qfai/assistant/skill/qfai-grilling/SKILL.md +16 -88
- package/assets/init/.qfai/assistant/skill/qfai-grilling/references/delegated-session.md +58 -0
- package/assets/init/.qfai/assistant/skill/qfai-grilling/references/session-record.md +42 -0
- package/assets/init/.qfai/assistant/skill/qfai-implement/SKILL.md +17 -4
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/credential-reuse.md +9 -0
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/cross-spec-ownership.md +41 -0
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/oracle-strength.md +6 -1
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/parallelization-policy.md +9 -1
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/red-admissibility.md +47 -1
- package/assets/init/.qfai/assistant/skill/qfai-implement/references/ui-affecting.md +4 -3
- package/assets/init/.qfai/assistant/skill/qfai-maintain/SKILL.md +4 -4
- package/assets/init/.qfai/assistant/skill/qfai-migration-v1-to-v2/SKILL.md +16 -10
- package/assets/init/.qfai/assistant/skill/qfai-migration-v1-to-v2/references/migration-guide.md +10 -260
- package/assets/init/.qfai/assistant/skill/qfai-migration-v1-to-v2/references/migration-placement.md +273 -0
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/SKILL.md +5 -4
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/design-md-spec.md +14 -5
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/generator-prompt.md +10 -0
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/handoff.md +1 -1
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/iteration-loop.md +3 -4
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/review-payload-schema.md +12 -5
- package/assets/init/.qfai/assistant/skill/qfai-prototyping/references/reviewer-prompt.md +33 -4
- package/assets/init/.qfai/assistant/skill/qfai-run/SKILL.md +34 -36
- package/assets/init/.qfai/assistant/skill/qfai-run/references/evidence-delivery.md +322 -0
- package/assets/init/.qfai/assistant/skill/qfai-run/references/extraction.md +67 -24
- package/assets/init/.qfai/assistant/skill/qfai-run/references/operator-screens.md +246 -5
- package/assets/init/.qfai/assistant/skill/qfai-run/references/plan.md +29 -5
- package/assets/init/.qfai/assistant/skill/qfai-run/references/stage-points.md +43 -0
- package/assets/init/.qfai/assistant/skill/qfai-sdd/SKILL.md +5 -4
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/concrete-abstract-cycle.md +10 -1
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/contract-artifact-rules.md +22 -9
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/design-anti-patterns.md +41 -6
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/design-md-authoring.md +11 -5
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/design-md-brand-catalog.md +17 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/requirements-decomposition.md +10 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/sdd-execution-playbook.md +34 -9
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/sdd-phase-checklists.md +4 -4
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/sdd-quality-gate.md +2 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/sdd-triage.md +177 -7
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/spec-traceability-rules.md +29 -5
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/ui-contract-guide.md +130 -1
- package/assets/init/.qfai/assistant/skill/qfai-sdd/references/ui-design-contract-normalization.md +4 -30
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/change-request.md +39 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/contracts/db-contract.sample.sql +2 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/contracts/ui-contract.sample.yaml +52 -5
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/spec/03_contract/cli/cli-NNNN-title.md +2 -2
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/spec/03_contract/tech.md +1 -1
- package/assets/init/.qfai/assistant/skill/qfai-sdd/templates/spec/decisions.md +7 -0
- package/assets/init/.qfai/assistant/skill/qfai-triage/SKILL.md +30 -20
- package/assets/init/.qfai/assistant/skill/qfai-verify/SKILL.md +6 -5
- package/assets/init/.qfai/assistant/skill/qfai-verify/references/context-load.md +9 -6
- package/assets/init/.qfai/assistant/skill/qfai-verify/references/verify-output-contract.md +92 -13
- package/assets/init/.qfai/assistant/skill/web-research/SKILL.md +37 -31
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/brave-search/config.toml +2 -4
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/brave-search/mcp-config.json +9 -5
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/firecrawl/.mcp.json +1 -2
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/firecrawl/config.toml +3 -3
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/firecrawl/mcp-config.json +11 -8
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/playwright/config.toml +1 -1
- package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/playwright/mcp-config.json +7 -3
- package/assets/init/.qfai/assistant/step/common-design-md/STEP.md +17 -7
- package/assets/init/.qfai/assistant/step/common-gate-run/STEP.md +126 -13
- package/assets/init/.qfai/assistant/step/common-review-cycle/STEP.md +3 -2
- package/assets/init/.qfai/assistant/step/discussion-pack/STEP.md +1 -0
- package/assets/init/.qfai/assistant/step/discussion-research/STEP.md +2 -0
- package/assets/init/.qfai/assistant/step/discussion-uiux/STEP.md +12 -4
- package/assets/init/.qfai/assistant/step/implement-backport/STEP.md +1 -1
- package/assets/init/.qfai/assistant/step/implement-benchmark/STEP.md +7 -4
- package/assets/init/.qfai/assistant/step/implement-dep-bump/STEP.md +11 -1
- package/assets/init/.qfai/assistant/step/implement-diagnose/STEP.md +2 -0
- package/assets/init/.qfai/assistant/step/implement-oracle-parity/STEP.md +2 -2
- package/assets/init/.qfai/assistant/step/implement-quarantine/STEP.md +3 -3
- package/assets/init/.qfai/assistant/step/implement-refactor/STEP.md +2 -2
- package/assets/init/.qfai/assistant/step/implement-regression-fix/STEP.md +4 -3
- package/assets/init/.qfai/assistant/step/implement-retire/STEP.md +6 -3
- package/assets/init/.qfai/assistant/step/implement-revert/STEP.md +4 -3
- package/assets/init/.qfai/assistant/step/implement-scaffold/STEP.md +3 -3
- package/assets/init/.qfai/assistant/step/implement-seam/STEP.md +9 -6
- package/assets/init/.qfai/assistant/step/implement-stress-harness/STEP.md +1 -1
- package/assets/init/.qfai/assistant/step/implement-sweep/STEP.md +4 -3
- package/assets/init/.qfai/assistant/step/implement-tdd/STEP.md +42 -6
- package/assets/init/.qfai/assistant/step/implement-test-fix/STEP.md +2 -0
- package/assets/init/.qfai/assistant/step/implement-tooling/STEP.md +22 -2
- package/assets/init/.qfai/assistant/step/maintain-edit/STEP.md +18 -14
- package/assets/init/.qfai/assistant/step/prototyping-grill/STEP.md +4 -3
- package/assets/init/.qfai/assistant/step/prototyping-loop/STEP.md +4 -0
- package/assets/init/.qfai/assistant/step/prototyping-preflight/STEP.md +1 -3
- package/assets/init/.qfai/assistant/step/sdd-contract/STEP.md +11 -0
- package/assets/init/.qfai/assistant/step/sdd-flow/STEP.md +4 -0
- package/assets/init/.qfai/assistant/step/sdd-gate/STEP.md +4 -0
- package/assets/init/.qfai/assistant/step/sdd-story/STEP.md +10 -0
- package/assets/init/.qfai/assistant/step/sdd-triage/STEP.md +37 -11
- package/assets/init/.qfai/assistant/step/triage-answer/STEP.md +5 -4
- package/assets/init/.qfai/assistant/step/triage-close/STEP.md +33 -23
- package/assets/init/.qfai/assistant/step/triage-cluster/STEP.md +5 -3
- package/assets/init/.qfai/assistant/step/triage-decompose/STEP.md +2 -0
- package/assets/init/.qfai/assistant/step/triage-dedupe/STEP.md +4 -0
- package/assets/init/.qfai/assistant/step/triage-handoff/STEP.md +13 -4
- package/assets/init/.qfai/assistant/step/triage-investigate/STEP.md +8 -5
- package/assets/init/.qfai/assistant/step/triage-owned-cleanup/STEP.md +86 -0
- package/assets/init/.qfai/assistant/step/triage-request-info/STEP.md +2 -0
- package/assets/init/.qfai/assistant/step/triage-security-intake/STEP.md +6 -3
- package/assets/init/.qfai/assistant/step/verify-change-note/STEP.md +9 -5
- package/assets/init/.qfai/assistant/step/verify-commit/STEP.md +39 -6
- package/assets/init/.qfai/assistant/step/verify-external/STEP.md +5 -2
- package/assets/init/.qfai/assistant/step/verify-qfai-gate/STEP.md +10 -2
- package/assets/init/.qfai/assistant/step/verify-release-notes/STEP.md +1 -0
- package/assets/init/.qfai/assistant/step/verify-repeat-run/STEP.md +7 -5
- package/assets/init/.qfai/assistant/step/verify-repo-gate/STEP.md +136 -28
- package/assets/init/.qfai/waivers.yml +1 -1
- package/assets/init/root/.agents/rules/ai-readable-markdown.md +98 -0
- package/assets/init/root/.agents/rules/api-budget.md +191 -13
- package/assets/init/root/.agents/rules/documentation-clarity.md +192 -35
- package/assets/init/root/.agents/rules/grilling.md +4 -4
- package/assets/init/root/.agents/rules/interface-clarity.md +5 -5
- package/assets/init/root/.agents/rules/minimal-implementation.md +11 -3
- package/assets/init/root/.agents/rules/reminders.json +27 -14
- package/assets/init/root/.agents/rules/session-feedback.md +136 -0
- package/assets/init/root/.agents/rules/temporary-files.md +47 -5
- package/assets/init/root/.agents/rules/untrusted-content.md +3 -3
- package/assets/init/root/.agents/rules/user-questions.md +74 -36
- package/assets/init/root/.gitattributes +9 -0
- package/assets/init/root/.github/workflows/qfai-docs.yml +44 -12
- package/assets/init/root/.github/workflows/qfai-tests.yml +58 -15
- package/assets/init/root/.github/workflows/qfai-validate.yml +49 -42
- package/assets/init/root/AGENTS.md +23 -21
- package/assets/mdschema/story/03_contract/tech.mdschema.yml +7 -3
- package/assets/sandbox-templates/default-deny.yaml +9 -5
- package/assets/schemas/workflow/extraction.schema.json +85 -5
- package/assets/schemas/workflow/plan.schema.json +9 -2
- package/assets/scripts/check-mdschema.mjs +172 -5
- package/assets/uix-rev/comparison-review.md +1 -1
- package/assets/uix-rev/contracts-review.md +1 -2
- package/dist/cli/index.cjs +6499 -7688
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.mjs +6449 -7641
- package/dist/cli/index.mjs.map +1 -1
- package/dist/index.cjs +5548 -7823
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +40 -666
- package/dist/index.d.ts +40 -666
- package/dist/index.mjs +5397 -7648
- package/dist/index.mjs.map +1 -1
- package/docs/MIGRATION-2.0.0.md +1 -1
- package/package.json +5 -7
- package/assets/init/root/CLAUDE.md +0 -45
|
@@ -1,23 +1,94 @@
|
|
|
1
1
|
# Verify Output Contract — `.qfai/report/verify.json`
|
|
2
2
|
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
- Verdict file
|
|
6
|
+
- Per-gate results
|
|
7
|
+
- CI-only checks
|
|
8
|
+
- Scope
|
|
9
|
+
- Example and rules
|
|
10
|
+
|
|
11
|
+
## Verdict file
|
|
12
|
+
|
|
3
13
|
`/qfai-verify` MUST write `.qfai/report/verify.json` at the end of the run. This file is the machine-readable verdict; the stage report is the human-readable account. No `qfai` command reads it, so an absent `verify.json` does not fail `npx qfai validate`.
|
|
4
14
|
|
|
5
|
-
Canonical path: `.qfai/report/verify.json
|
|
15
|
+
Canonical path: `.qfai/report/verify.json`. Create the `.qfai/report/` directory if absent — it is the same directory `validate.json` is written to.
|
|
6
16
|
|
|
7
|
-
|
|
17
|
+
| Field | Type | Required | Meaning |
|
|
18
|
+
| ------------ | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
19
|
+
| `status` | string | yes | `"PASS"` when every gate in scope passed, or is delegated to a CI check that is not red; `"FAIL"` otherwise. Only `"PASS"` satisfies a downstream gate. |
|
|
20
|
+
| `scope` | string | yes | Which stage's gate set this run covers. See the enum below. |
|
|
21
|
+
| `flowId` | string | no | The business flow this run targeted, when scoped to one (e.g. `"BF-0001"`). |
|
|
22
|
+
| `recordedAt` | ISO-8601 string | no | When the run completed. |
|
|
23
|
+
| `summary` | string | no | One or two sentences an operator can read without opening the stage report. |
|
|
24
|
+
| `gates` | array of objects | no | Per-gate results: as set out below. Advisory; no reader gates on it today. |
|
|
8
25
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
| `summary` | string | no | One or two sentences an operator can read without opening the stage report. |
|
|
16
|
-
| `gates` | array of objects | no | Per-gate results: `{ name, status, command }`. Advisory; no reader gates on it today. |
|
|
26
|
+
When the scope includes acceptance scaffolds or unfinished BF/AC acceptance
|
|
27
|
+
bodies, an optional `summary` stays one or two short sentences naming remaining
|
|
28
|
+
work, its owner or UNASSIGNED, and the detailed human-report reference. The
|
|
29
|
+
five-field detail belongs in `verify-repo-gate`'s `## Report` section, not a
|
|
30
|
+
new JSON field or table inside `summary`. Empty-body exemptions, verdicts and
|
|
31
|
+
completion rules stay unchanged.
|
|
17
32
|
|
|
18
33
|
`status` is a closed two-value enum: `"PASS"` / `"FAIL"`. There is no `"WARN"` — a run with only `warning` / `info` findings is `"PASS"` (waivers apply to those severities only). Any `error` finding makes it `"FAIL"`.
|
|
19
34
|
|
|
20
|
-
|
|
35
|
+
## Per-gate results
|
|
36
|
+
|
|
37
|
+
Each entry of `gates` is `{ name, status, command }`, plus these optional fields:
|
|
38
|
+
|
|
39
|
+
| Field | Type | Meaning |
|
|
40
|
+
| ---------- | ------ | --------------------------------------------------------------------------------------------------------------------------------- |
|
|
41
|
+
| `status` | string | `"PASS"`, `"FAIL"`, `"UNRUN"` or `"DELEGATED"`. `"UNRUN"` is a gate with no command, launcher or environment; it is never a pass. |
|
|
42
|
+
| `baseline` | string | On a `"FAIL"`: `"same"` when the base commit fails identically, `"different"` when it does not, `"unrun"` when it cannot run. |
|
|
43
|
+
| `reason` | string | On an `"UNRUN"` gate, or a `baseline` of `"unrun"`: what is missing. |
|
|
44
|
+
| `check` | string | On a `"DELEGATED"` gate: the CI check that runs it. |
|
|
45
|
+
| `ci` | string | On a `"DELEGATED"` gate: `"green"`, `"red"` or `"pending"`, the state of that check when the file is written. |
|
|
46
|
+
|
|
47
|
+
## CI-only checks
|
|
48
|
+
|
|
49
|
+
A gate the project's Standard commands declare `CI only: <check name>` is
|
|
50
|
+
recorded `"DELEGATED"` with its `check` and `ci` for the candidate being evaluated.
|
|
51
|
+
`check` names the actual CI check and `ci` its observed state.
|
|
52
|
+
A `"red"` check makes the top-level `status` `"FAIL"` for that candidate.
|
|
53
|
+
A `"pending"` one does not, and `summary` names it. A runnable local gate is
|
|
54
|
+
never delegated to avoid running it. Retain an observed FAIL or
|
|
55
|
+
UNRUN; neither becomes PASS or DELEGATED because a verification commit is wanted.
|
|
56
|
+
|
|
57
|
+
A `"PASS"` verdict with pending delegated checks supports only the existing
|
|
58
|
+
verification-commit allowance under a user instruction or project policy
|
|
59
|
+
covering that commit and its paths. It does not establish formal completion or
|
|
60
|
+
permission to push, create a pull request or merge. Every nondelegated gate
|
|
61
|
+
must pass. A delegated red check stops transport of the same unchanged candidate.
|
|
62
|
+
Explicit CI-only delegation covering a transport commit may instead carry actual
|
|
63
|
+
changes addressing that failure through the existing planned fix into a new
|
|
64
|
+
candidate, while HEAD still names the failed commit. Preserve the prior head,
|
|
65
|
+
check, run and observed result as FAIL; never relabel that evidence pending.
|
|
66
|
+
Report the actual fix and, after committing, the new head and resume condition.
|
|
67
|
+
The new candidate remains unverified until its own CI results are observed.
|
|
68
|
+
|
|
69
|
+
The stage report records the target, current head, run ID or URL when available,
|
|
70
|
+
check, actual outcome and counts, blocker and resume condition. No run yet means
|
|
71
|
+
no result; invent neither. `ci: "green"` needs actual passing evidence for that
|
|
72
|
+
same target and head. Missing or mismatched evidence remains pending. Formal
|
|
73
|
+
completion or merge waits for every required check to pass on the current head.
|
|
74
|
+
Keep this evidence in the stage report and `summary`; add no verdict fields or states.
|
|
75
|
+
|
|
76
|
+
A gate that fails identically on the base, or did not run and is not delegated, still makes the top-level `status` `"FAIL"`. The fields let a reader tell it from a failure the run caused.
|
|
77
|
+
|
|
78
|
+
Keep the full CI receipt in the stage report, as
|
|
79
|
+
`.qfai/assistant/step/common-gate-run/STEP.md#ci-evidence` requires. A branch
|
|
80
|
+
overview or a confirmed closed cleanup SKIP does not replace actual verification
|
|
81
|
+
or turn it into a behavioral PASS. Preserve failures for the current head;
|
|
82
|
+
unknown identity or purpose remains unverified. The receipt adds no fields or
|
|
83
|
+
states to this contract.
|
|
84
|
+
|
|
85
|
+
## Scope
|
|
86
|
+
|
|
87
|
+
`scope` is a closed enum. Write the gate set this invocation covers through
|
|
88
|
+
execution or an explicitly declared matching CI-only check. Delegation never
|
|
89
|
+
claims that the profile already ran; its check must perform the required scan
|
|
90
|
+
and profile. A whole-repository scope still requires full validation. Write the
|
|
91
|
+
one that matches the stage you were invoked for:
|
|
21
92
|
|
|
22
93
|
| `scope` | Written by | validate profile |
|
|
23
94
|
| ------------- | ------------------------------------------------------------------- | ----------------------------------------------- |
|
|
@@ -28,6 +99,8 @@ Canonical path: `.qfai/report/verify.json` (NOT `.qfai/output/`). Create the `.q
|
|
|
28
99
|
There is no `implement` value: the enum is closed at these three, and a run after
|
|
29
100
|
`/qfai-implement` is recorded as `full`.
|
|
30
101
|
|
|
102
|
+
## Example and rules
|
|
103
|
+
|
|
31
104
|
Minimal conforming example for the prototyping gate:
|
|
32
105
|
|
|
33
106
|
```json
|
|
@@ -42,6 +115,12 @@ Minimal conforming example for the prototyping gate:
|
|
|
42
115
|
|
|
43
116
|
Rules:
|
|
44
117
|
|
|
45
|
-
-
|
|
46
|
-
-
|
|
118
|
+
- A PASS gate needs actual command output or matching CI evidence. The existing
|
|
119
|
+
top-level pending-DELEGATED allowance supports only the covered verification
|
|
120
|
+
commit described above. A `FAIL` verdict is a legitimate output; downstream
|
|
121
|
+
work stops as its gate requires.
|
|
122
|
+
- Never write a `scope` this invocation does not cover by execution or its
|
|
123
|
+
declared matching CI-only check. Writing `"prototyping"` for a `full` gate set
|
|
124
|
+
is a false verdict, not a workaround. A `--profile verify` gate is
|
|
125
|
+
`scope: "full"`, not `scope: "atdd"`; `atdd` covers the ATDD gate set only.
|
|
47
126
|
- Do not add a `version` / `schemaVersion` field. No reader validates one, and a second version series in a distributed artifact is exactly what the distributed-surface rule forbids: the npm package version is the only version this surface has.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: web-research
|
|
3
3
|
title: "Web Research Pipeline"
|
|
4
|
-
description: "8-stage web research pipeline with MCP integration, caching, and citation generation."
|
|
4
|
+
description: "8-stage web research pipeline with MCP integration, caching, and citation generation. Use when asked to research a topic on the web and report the findings with cited sources."
|
|
5
5
|
argument-hint: "[query] [--max-depth N] [--yolo]"
|
|
6
6
|
allowed-tools: [Read, Glob, Bash, Write, Task, Agent, WebSearch, WebFetch]
|
|
7
7
|
roles: [Researcher, Analyst, FactChecker]
|
|
@@ -16,7 +16,7 @@ QFAI Skill Body (SSOT)
|
|
|
16
16
|
|
|
17
17
|
## /web-research — Web Research Pipeline
|
|
18
18
|
|
|
19
|
-
[DRIFT-PROTOCOL:
|
|
19
|
+
[DRIFT-PROTOCOL:REQUIRED]
|
|
20
20
|
|
|
21
21
|
## User Questions (AskUserQuestion Protocol)
|
|
22
22
|
|
|
@@ -38,14 +38,6 @@ baseline overlap, the baseline governs.
|
|
|
38
38
|
- The orchestrator may write the research artifact itself, or give independent searches to sub-agents that run in parallel.
|
|
39
39
|
- It never approves its own work.
|
|
40
40
|
|
|
41
|
-
### Capability Probe
|
|
42
|
-
|
|
43
|
-
- No additional overrides.
|
|
44
|
-
|
|
45
|
-
### Delegation Failure (Hard Stop)
|
|
46
|
-
|
|
47
|
-
- No additional overrides.
|
|
48
|
-
|
|
49
41
|
## Work Orders Summary
|
|
50
42
|
|
|
51
43
|
Every major research artifact must include a `## Work Orders Summary` table.
|
|
@@ -127,8 +119,9 @@ Configuration templates: `.qfai/assistant/skill/web-research/mcp-templates/playw
|
|
|
127
119
|
|
|
128
120
|
### 2.4 MCP Failure Recovery
|
|
129
121
|
|
|
130
|
-
- Crash detection
|
|
122
|
+
- Crash detection: notice an MCP server crash or a dropped connection within **10 seconds**.
|
|
131
123
|
- On MCP server crash, fallback to built-in tools (WebSearch / WebFetch).
|
|
124
|
+
- Tell the user that the MCP server is unavailable and that the built-in tools are in use.
|
|
132
125
|
- Rate limit: detect HTTP 429 responses and honour `Retry-After` header with exponential backoff.
|
|
133
126
|
|
|
134
127
|
## 3. Security
|
|
@@ -150,10 +143,11 @@ The sanitizer is idempotent: applying it twice produces byte-identical output.
|
|
|
150
143
|
Default policy: **default-deny**.
|
|
151
144
|
|
|
152
145
|
- Only domains listed in the project allowlist may be fetched.
|
|
153
|
-
- The allowlist is
|
|
154
|
-
- Unknown domains are logged and skipped; the pipeline continues with allowed sources.
|
|
146
|
+
- The allowlist is a domain list in `qfai.config.yaml` under `webResearch.allowlist`, read by the agent; `qfai` does not parse it.
|
|
147
|
+
- Unknown domains are logged with the blocked domain and skipped; the pipeline continues with allowed sources.
|
|
155
148
|
- Redirect chains are followed only while all hops remain on allowlisted domains.
|
|
156
|
-
A redirect to a non-allowlisted domain is blocked
|
|
149
|
+
A redirect to a non-allowlisted domain is blocked at that target, the fetch is rejected,
|
|
150
|
+
and the redirect chain up to the blocked target is logged.
|
|
157
151
|
|
|
158
152
|
### 3.3 --yolo Flag and Security Gates
|
|
159
153
|
|
|
@@ -167,14 +161,14 @@ cannot be bypassed.
|
|
|
167
161
|
|
|
168
162
|
Every pipeline execution produces a session log with **6 mandatory fields**:
|
|
169
163
|
|
|
170
|
-
| Field | Description
|
|
171
|
-
| ------------ |
|
|
172
|
-
| `session_id` | Unique identifier for this research run
|
|
173
|
-
| `query` | The
|
|
174
|
-
| `timestamp` | ISO-8601 start time
|
|
175
|
-
| `stages` |
|
|
176
|
-
| `sources` | List of fetched URLs with status codes
|
|
177
|
-
| `citations` | Final citation entries
|
|
164
|
+
| Field | Description |
|
|
165
|
+
| ------------ | ----------------------------------------------------------------------- |
|
|
166
|
+
| `session_id` | Unique identifier for this research run |
|
|
167
|
+
| `query` | The request and every search query issued |
|
|
168
|
+
| `timestamp` | ISO-8601 start time |
|
|
169
|
+
| `stages` | Stage results with timing, sanitization events and verification results |
|
|
170
|
+
| `sources` | List of fetched URLs with status codes and content hashes |
|
|
171
|
+
| `citations` | Final citation entries |
|
|
178
172
|
|
|
179
173
|
The session log is part of the research report, not a file of its own.
|
|
180
174
|
|
|
@@ -194,6 +188,9 @@ Risk-based gating strategy:
|
|
|
194
188
|
- **Low-risk queries**: Auto-approve. No human gate required.
|
|
195
189
|
- **High-risk queries** (e.g., medical, legal, financial): Gate before cite stage.
|
|
196
190
|
Human must confirm source selection and extracted claims.
|
|
191
|
+
- **A high-risk conclusion** is not applied to code until a human has reviewed it.
|
|
192
|
+
The gate blocks and shows the diff the conclusion would produce together with its citations.
|
|
193
|
+
A low-risk conclusion is applied without blocking.
|
|
197
194
|
- `--yolo` flag is **ignored** for security gates (see Section 3.3).
|
|
198
195
|
|
|
199
196
|
Risk classification is determined by query topic analysis and domain sensitivity rules.
|
|
@@ -212,7 +209,7 @@ Default TTL: **24 hours** (24h).
|
|
|
212
209
|
|
|
213
210
|
- Entries older than TTL are marked stale and re-fetched on next access.
|
|
214
211
|
- Staleness is tracked per-entry; partial cache invalidation is supported.
|
|
215
|
-
- TTL is configurable via `qfai.config.yaml` under `webResearch.cache.ttl
|
|
212
|
+
- TTL is configurable via `qfai.config.yaml` under `webResearch.cache.ttl`, read by the agent; `qfai` does not parse it.
|
|
216
213
|
|
|
217
214
|
### 7.3 Storage
|
|
218
215
|
|
|
@@ -235,8 +232,9 @@ the session log as the coordination artifact.
|
|
|
235
232
|
|
|
236
233
|
When the search stage returns no results:
|
|
237
234
|
|
|
238
|
-
- Log "no sources found" to the session log.
|
|
239
|
-
-
|
|
235
|
+
- Log "no web sources found" to the session log.
|
|
236
|
+
- Report "no web sources found" to the user together with every search query issued, for user review.
|
|
237
|
+
- Generate no citations: the citation block stays empty, and nothing is cited from memory or from a source that was not fetched.
|
|
240
238
|
- Do not proceed to fetch/extract stages.
|
|
241
239
|
|
|
242
240
|
### 9.2 Fetch Failure Isolation
|
|
@@ -245,14 +243,19 @@ Each URL is fetched independently. A fetch failure for one URL does not
|
|
|
245
243
|
abort the pipeline. Failed URLs are logged and excluded; the remaining
|
|
246
244
|
successful fetches produce a partial result.
|
|
247
245
|
|
|
246
|
+
When every fetch fails, no partial result exists. Report that every fetch failed,
|
|
247
|
+
with the failure reason (status code or timeout) for each URL, and do not run the
|
|
248
|
+
extract stage.
|
|
249
|
+
|
|
248
250
|
This isolation ensures that transient network errors or single-domain
|
|
249
251
|
outages do not block the entire research pipeline.
|
|
250
252
|
|
|
251
253
|
### 9.3 Rate Limiting
|
|
252
254
|
|
|
253
255
|
- Detect HTTP 429 (Too Many Requests) responses.
|
|
254
|
-
- Read and honour the `Retry-After` header.
|
|
255
|
-
- Apply exponential backoff with jitter for retries.
|
|
256
|
+
- Read and honour the `Retry-After` header: retry only after the delay it gives.
|
|
257
|
+
- Apply exponential backoff with jitter for further retries.
|
|
258
|
+
- Log each rate-limit event to the session log with the 429 status, the delay and the retry number.
|
|
256
259
|
|
|
257
260
|
## 10. Conservative Defaults
|
|
258
261
|
|
|
@@ -278,7 +281,8 @@ SKILL.md files follow a **progressive disclosure** loading strategy:
|
|
|
278
281
|
### 11.1 Invalid SKILL.md Handling
|
|
279
282
|
|
|
280
283
|
If the YAML front-matter is **invalid** or produces a **parse error** (malformed
|
|
281
|
-
YAML), the loader reports the error
|
|
284
|
+
YAML), the loader reports the parse error with its details (the YAML error
|
|
285
|
+
message and its location) to the session log and activates
|
|
282
286
|
**default behavior** as a fallback. The skill is still listed in the roster
|
|
283
287
|
but operates with built-in defaults until the YAML is corrected.
|
|
284
288
|
|
|
@@ -296,15 +300,17 @@ rules apply:
|
|
|
296
300
|
|
|
297
301
|
## 13. Golden Task Evaluation
|
|
298
302
|
|
|
299
|
-
**Golden task** sets are curated query-answer pairs
|
|
300
|
-
evaluation. Each golden task is
|
|
303
|
+
**Golden task** sets are curated query-answer pairs, each with its expected
|
|
304
|
+
sources and citations, used for regression evaluation. Each golden task is
|
|
305
|
+
scored against 4 metrics:
|
|
301
306
|
|
|
302
307
|
- **Citation precision** — accuracy of generated citations.
|
|
303
308
|
- **Coverage** — completeness of query facet coverage.
|
|
304
309
|
- **Freshness** — recency of cited sources.
|
|
305
310
|
- **Security hygiene** — sanitization pass rate.
|
|
306
311
|
|
|
307
|
-
|
|
312
|
+
Each score is compared with its target in Section 5, and the scores and the
|
|
313
|
+
comparison are reported with the evaluation run.
|
|
308
314
|
|
|
309
315
|
## Completion Contract (Shared)
|
|
310
316
|
|
package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/brave-search/mcp-config.json
CHANGED
|
@@ -1,9 +1,13 @@
|
|
|
1
1
|
{
|
|
2
|
-
"
|
|
3
|
-
"
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
"
|
|
2
|
+
"mcpServers": {
|
|
3
|
+
"brave-search": {
|
|
4
|
+
"type": "local",
|
|
5
|
+
"command": "npx",
|
|
6
|
+
"args": ["-y", "@modelcontextprotocol/server-brave-search@0.6.2"],
|
|
7
|
+
"env": {
|
|
8
|
+
"BRAVE_API_KEY": "${BRAVE_API_KEY}"
|
|
9
|
+
},
|
|
10
|
+
"tools": ["*"]
|
|
7
11
|
}
|
|
8
12
|
}
|
|
9
13
|
}
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Firecrawl MCP - supports both hosted URL and local npx modes
|
|
2
2
|
# Local npx mode: recommended for sensitive environments
|
|
3
3
|
# Hosted mode: set FIRECRAWL_API_URL to https://api.firecrawl.dev
|
|
4
|
-
[
|
|
4
|
+
[mcp_servers.firecrawl]
|
|
5
5
|
command = "npx"
|
|
6
6
|
args = ["-y", "firecrawl-mcp@3.11.0"]
|
|
7
|
+
env_vars = ["FIRECRAWL_API_KEY"]
|
|
7
8
|
|
|
8
|
-
[
|
|
9
|
-
FIRECRAWL_API_KEY = "${FIRECRAWL_API_KEY}"
|
|
9
|
+
[mcp_servers.firecrawl.env]
|
|
10
10
|
FIRECRAWL_API_URL = "https://api.firecrawl.dev"
|
package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/firecrawl/mcp-config.json
CHANGED
|
@@ -1,11 +1,14 @@
|
|
|
1
1
|
{
|
|
2
|
-
"
|
|
3
|
-
"
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
"
|
|
7
|
-
"
|
|
8
|
-
|
|
9
|
-
|
|
2
|
+
"mcpServers": {
|
|
3
|
+
"firecrawl": {
|
|
4
|
+
"type": "local",
|
|
5
|
+
"command": "npx",
|
|
6
|
+
"args": ["-y", "firecrawl-mcp@3.11.0"],
|
|
7
|
+
"env": {
|
|
8
|
+
"FIRECRAWL_API_KEY": "${FIRECRAWL_API_KEY}",
|
|
9
|
+
"FIRECRAWL_API_URL": "https://api.firecrawl.dev"
|
|
10
|
+
},
|
|
11
|
+
"tools": ["*"]
|
|
12
|
+
}
|
|
10
13
|
}
|
|
11
14
|
}
|
package/assets/init/.qfai/assistant/skill/web-research/mcp-templates/playwright/mcp-config.json
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
{
|
|
2
|
-
"
|
|
3
|
-
"
|
|
4
|
-
|
|
2
|
+
"mcpServers": {
|
|
3
|
+
"playwright": {
|
|
4
|
+
"type": "local",
|
|
5
|
+
"command": "npx",
|
|
6
|
+
"args": ["-y", "@executeautomation/playwright-mcp-server@1.0.12"],
|
|
7
|
+
"tools": ["*"]
|
|
8
|
+
}
|
|
5
9
|
}
|
|
6
10
|
}
|
|
@@ -27,12 +27,17 @@ for it.
|
|
|
27
27
|
lists it: `01_Context.md#Design Direction`, the reference registries in
|
|
28
28
|
`04_Sources.md`, and the `uiux/` sidecars. For an imported tree, the surface
|
|
29
29
|
the import source names, or the user's answer.
|
|
30
|
+
- The questions behind those answers:
|
|
31
|
+
`.qfai/assistant/skill/qfai-discussion/references/design-dna-intake.md`.
|
|
30
32
|
- The schema and validation rules:
|
|
31
33
|
`.qfai/assistant/skill/qfai-prototyping/references/design-md-spec.md`.
|
|
32
34
|
- The archetype catalog:
|
|
33
35
|
`.qfai/assistant/skill/qfai-sdd/references/design-md-brand-catalog.md`.
|
|
34
36
|
- The starting sample:
|
|
35
37
|
`.qfai/assistant/skill/qfai-prototyping/templates/DESIGN.md.sample`.
|
|
38
|
+
- The patterns that mark a design as generated, and their displacement table:
|
|
39
|
+
`.qfai/assistant/skill/qfai-sdd/references/design-anti-patterns.md`. Read it
|
|
40
|
+
for § Author and validate step 5.
|
|
36
41
|
|
|
37
42
|
## Author and validate
|
|
38
43
|
|
|
@@ -51,10 +56,12 @@ for it.
|
|
|
51
56
|
4. **Parse and validate.** `parseDesignMd(text)` must succeed; pass its `data`
|
|
52
57
|
to `validateDesignMd` and halt on any issue. Both are exported from the
|
|
53
58
|
`qfai` package entry.
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
59
|
+
5. **Check the finished file against the pattern list.** Review root
|
|
60
|
+
`DESIGN.md` against every aspect of
|
|
61
|
+
`.qfai/assistant/skill/qfai-sdd/references/design-anti-patterns.md` and its
|
|
62
|
+
Displacement section. A token that prescribes a listed pattern fails unless
|
|
63
|
+
`01_Context.md#Design Direction` asks for it. Name the pattern in the
|
|
64
|
+
finding, replace the value, and check the file again.
|
|
58
65
|
|
|
59
66
|
## Check before building
|
|
60
67
|
|
|
@@ -77,7 +84,8 @@ it, edit it; the next iteration is built from it.
|
|
|
77
84
|
Read first: the flow's classified surface set and root `DESIGN.md`. The step
|
|
78
85
|
passes when the flow is not on a visual prototyping surface, or when root
|
|
79
86
|
`DESIGN.md` exists, is not the unreplaced sample, and parses and validates with
|
|
80
|
-
no issue
|
|
87
|
+
no issue and shows no listed pattern that the recorded direction does not ask
|
|
88
|
+
for. The pass names the surface set and the validation result.
|
|
81
89
|
|
|
82
90
|
## Writes
|
|
83
91
|
|
|
@@ -87,5 +95,7 @@ no issue. The pass names the surface set and the validation result.
|
|
|
87
95
|
|
|
88
96
|
## Gate
|
|
89
97
|
|
|
90
|
-
`DESIGN.md` parses and validates with no issue,
|
|
91
|
-
the recorded direction or the named theme does not supply
|
|
98
|
+
`DESIGN.md` parses and validates with no issue, no value was written that
|
|
99
|
+
the recorded direction or the named theme does not supply, and no token
|
|
100
|
+
prescribes a pattern from `design-anti-patterns.md` that the recorded direction
|
|
101
|
+
does not ask for. A reviewer fails the gate by naming the pattern.
|
|
@@ -26,35 +26,111 @@ Neither is restated here.
|
|
|
26
26
|
globs.
|
|
27
27
|
- The launcher the preflight established, for a qfai command
|
|
28
28
|
(`.qfai/assistant/rule/shared-skill-operating-baseline.md#canonical-qfai-launcher-mandatory`).
|
|
29
|
+
- `.agents/rules/temporary-files.md`, when this step writes captured output or a script to a
|
|
30
|
+
file.
|
|
31
|
+
- `.agents/rules/api-budget.md`, when this step reads CI state or the project's items through
|
|
32
|
+
the forge's API.
|
|
29
33
|
|
|
30
34
|
## Procedure
|
|
31
35
|
|
|
32
|
-
1. **
|
|
36
|
+
1. **Determine applicable gates before resolving commands.** Use the calling
|
|
37
|
+
stage's scope and the project's declared scope. Do not narrow the gate set
|
|
38
|
+
from the diff alone. Decide whether the conditional `Build` (if relevant)
|
|
39
|
+
and `Pack / distribution` (if distributed) gates apply from those scopes.
|
|
40
|
+
When a condition is false, omit its gate from `gates` and record the reason
|
|
41
|
+
in the stage report. A required gate with a missing command, launcher or
|
|
42
|
+
environment remains UNRUN.
|
|
43
|
+
2. **Resolve the launcher once per stage.** Before the first qfai gate, run the
|
|
33
44
|
launcher preflight. Where it fails, every qfai gate is UNRUN: record it as a
|
|
34
45
|
blocker and stop.
|
|
35
|
-
|
|
46
|
+
3. **Resolve the command.** A qfai gate is the validate profile and scope the
|
|
36
47
|
calling step names, for example
|
|
37
48
|
`npx qfai validate --profile sdd --fail-on error --flow BF-NNNN`. A project
|
|
38
49
|
gate is the matching entry of the Standard commands section, used as
|
|
39
50
|
written. An entry that is missing makes the gate UNRUN: record it and route
|
|
40
51
|
the gap as the rule says; never substitute a command from another stack.
|
|
41
|
-
|
|
52
|
+
An entry written `CI only: <check name>` instead of a command is not run
|
|
53
|
+
here: record the gate DELEGATED with that check name. Record its `ci` state
|
|
54
|
+
as `green`, `red` or `pending` from the pull request when one exists; with no
|
|
55
|
+
result yet, record `pending`.
|
|
56
|
+
4. **Run it** on the tree the stage will hand to its reviewer, in the
|
|
42
57
|
environment the `Runtime` and `Platform` rows declare.
|
|
43
|
-
|
|
58
|
+
5. **Report** in the stage report: the exact
|
|
44
59
|
command, the exit code, the counts or outcome, the revision the run read,
|
|
45
60
|
and for a validate run the `<paths.outDir>/validate.log` path and the
|
|
46
61
|
`run-*/` directory its `run_log:` line names. The CLI writes that log on
|
|
47
62
|
every run; shell redirection is not needed.
|
|
48
|
-
|
|
63
|
+
6. **Read a validate result from its JSON** where the step selects work from
|
|
49
64
|
it. The result is usable only when the file exists, its `profile` is the one
|
|
50
65
|
run, and its `generatedAt` is no earlier than this run's start. Read `counts`
|
|
51
66
|
for the verdict and `issues[].code` for each finding; the array is `issues`
|
|
52
67
|
(`.qfai/assistant/skill/qfai-verify/references/validate-json-schema.md`).
|
|
53
|
-
|
|
68
|
+
7. **On a failure**, classify each finding and repair only what this stage
|
|
54
69
|
owns, then rerun the same gate on the same scope. A gate that failed and then
|
|
55
70
|
passed with no change in between is a nondeterministic gate: record every
|
|
56
71
|
run, in order.
|
|
57
72
|
|
|
73
|
+
## CI evidence
|
|
74
|
+
|
|
75
|
+
A branch overview locates runs; it is not a formal verification receipt.
|
|
76
|
+
For each required verification, record the following in the stage report:
|
|
77
|
+
|
|
78
|
+
- Target, current full head SHA, verification and intended purpose.
|
|
79
|
+
- Workflow identity and name, event, run ID or URL containing the run ID, and run attempt.
|
|
80
|
+
- Job identity and name, check identity and name, status and conclusion.
|
|
81
|
+
- For test checks, actual test counts and the executed behavior they cover, with the job log or
|
|
82
|
+
artifact that proves them. Distinguish missing counts from zero executed tests.
|
|
83
|
+
- For lint, build and other non-test checks, evidence of actual execution and
|
|
84
|
+
results from their logs or artifacts. Do not invent test counts for these checks.
|
|
85
|
+
- Any closed-pull-request cleanup purpose, with the evidence that confirms the
|
|
86
|
+
pull request was closed and this run was selected for cleanup rather than verification.
|
|
87
|
+
- Remaining blocker and resume condition, or an explicit statement that neither
|
|
88
|
+
remains. Record unavailable identity or purpose as unknown and unverified.
|
|
89
|
+
|
|
90
|
+
Identify job IDs and check IDs separately where present; record their absence
|
|
91
|
+
where the platform provides neither. An ID labelled only as check/job is
|
|
92
|
+
ambiguous: record its identity as unverified until the job and check are distinguished.
|
|
93
|
+
|
|
94
|
+
Use head, workflow, event, run attempt, job, check and purpose together. The same
|
|
95
|
+
head SHA and check name do not make two runs the same verification. A
|
|
96
|
+
`pull_request` event or later timestamp alone does not prove closed cleanup.
|
|
97
|
+
A confirmed closed cleanup SKIP is neither a behavioral PASS nor an override
|
|
98
|
+
of verification. A matrix job skipped by the declared selection policy may be
|
|
99
|
+
legitimate, but its SKIP does not prove that its behavior ran.
|
|
100
|
+
Retain an identified successful verification for its target and head when a
|
|
101
|
+
later run is confirmed closed cleanup; the cleanup SKIP does not replace it.
|
|
102
|
+
|
|
103
|
+
A green result for an old SHA does not verify the current head. A later failure
|
|
104
|
+
for the same verification on the current head cannot be replaced by its older
|
|
105
|
+
success. Keep the failure visible until a matching verification actually passes.
|
|
106
|
+
Missing logs or unknown purpose leave execution unverified. For test checks,
|
|
107
|
+
missing counts or zero executed tests do not prove behavioral coverage.
|
|
108
|
+
These receipt details belong in the stage report; do not
|
|
109
|
+
add fields or states to `verify.json`.
|
|
110
|
+
|
|
111
|
+
### Synthetic worked example
|
|
112
|
+
|
|
113
|
+
The following receipts are synthetic, not evidence from an actual repository.
|
|
114
|
+
They concern test checks for the required unit suite.
|
|
115
|
+
All target branch `example/topic` at full head
|
|
116
|
+
`aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`, with required verification
|
|
117
|
+
`unit suite` and check name `tests`. Each run, attempt, job, check and log is
|
|
118
|
+
identified separately.
|
|
119
|
+
|
|
120
|
+
| Receipt | Workflow identity / name; event | Run ID / attempt | Job identity / name; check identity / name | Purpose and proof | Status / conclusion | Actual counts and behavior | Log |
|
|
121
|
+
| -------------------- | -------------------------------- | ---------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------ | -------------------------------------------------------------------------------------------- |
|
|
122
|
+
| Earlier verification | `71` / `CI`; `push` | `91001` / `1` | `81001` / `unit`; `71001` / `tests` | Run configuration selects the unit suite | `completed` / `success` | 29 passed, 0 failed, 0 skipped; 29 unit tests executed | `receipts/unit-success.log` contains the test summary |
|
|
123
|
+
| Later verification | `71` / `CI`; `workflow_dispatch` | `91002` / `2` | `81002` / `unit`; `71002` / `tests` | Run configuration selects the same unit suite | `completed` / `failure` | 28 passed, 1 failed, 0 skipped; 29 unit tests executed | `receipts/unit-failure.log` contains the failing test and summary |
|
|
124
|
+
| Closed cleanup | `72` / `Cleanup`; `pull_request` | `91003` / `3` | `81003` / `unit`; `71003` / `tests` | Payload action is `closed`, target state is closed, and a selection job log confirms cleanup only | `completed` / `skipped` | 0 passed, 0 failed, 0 executed; no unit test coverage | `receipts/cleanup-selection.log` records the selection; the skipped test job has no test log |
|
|
125
|
+
|
|
126
|
+
The required unit verification remains FAIL after the later verification. The
|
|
127
|
+
cleanup SKIP neither clears it nor proves the unit behavior passed. Resume with
|
|
128
|
+
the actual repair and a matching verification on the resulting current head.
|
|
129
|
+
|
|
130
|
+
For a test check, if an actual reported run has only a head and check name, its missing workflow,
|
|
131
|
+
event, attempt, job, purpose, counts and logs remain unknown. Record it as
|
|
132
|
+
unverified with those gaps as blockers; the synthetic rows do not fill them.
|
|
133
|
+
|
|
58
134
|
## Scope
|
|
59
135
|
|
|
60
136
|
- A flow-scoped gate covers that flow. A result for another flow or another
|
|
@@ -64,14 +140,51 @@ Neither is restated here.
|
|
|
64
140
|
is neither repaired here nor counted as this gate's pass.
|
|
65
141
|
- A repository-wide PASS is never claimed from one flow's result.
|
|
66
142
|
|
|
143
|
+
### Selected-case execution
|
|
144
|
+
|
|
145
|
+
When the caller selects a focused Test case, record its command arguments,
|
|
146
|
+
cwd, file and case selection, actual runtime version, reporter, revision and raw output.
|
|
147
|
+
Link the EX and source declaration to the full file/parent/case identity and
|
|
148
|
+
observed body execution and terminal result. Keep exit, counts, containers
|
|
149
|
+
and static scanner linkage separate; none proves that selected behavior ran.
|
|
150
|
+
|
|
151
|
+
Zero selected executions or unidentifiable, skipped, TODO, cancelled or
|
|
152
|
+
interrupted target results cannot close the focused phase. Retain the output,
|
|
153
|
+
name the missing evidence, correct selection or observation and repeat it.
|
|
154
|
+
Read
|
|
155
|
+
`.qfai/assistant/skill/qfai-implement/references/red-admissibility.md#selected-case-execution`
|
|
156
|
+
before judging a focused RED or ordinary success. A focused result establishes
|
|
157
|
+
only its selected case; the existing verify run owns full collection coverage.
|
|
158
|
+
This human receipt changes no gate status or JSON field and adds no runner.
|
|
159
|
+
|
|
67
160
|
## Outcomes
|
|
68
161
|
|
|
69
|
-
| Outcome
|
|
70
|
-
|
|
|
71
|
-
| `PASS`
|
|
72
|
-
| `FAIL`
|
|
73
|
-
| `UNRUN`
|
|
162
|
+
| Outcome | Means |
|
|
163
|
+
| ----------- | ------------------------------------------------------------------ |
|
|
164
|
+
| `PASS` | The command ran on the reviewed tree and exited 0 with zero errors |
|
|
165
|
+
| `FAIL` | The command ran and reported an error or a non-zero exit |
|
|
166
|
+
| `UNRUN` | No command, no launcher, or no environment to run it in |
|
|
167
|
+
| `DELEGATED` | The project runs the gate in CI only, by the check its entry names |
|
|
74
168
|
|
|
75
|
-
Only `PASS` satisfies a gate.
|
|
76
|
-
|
|
169
|
+
Only `PASS` satisfies a gate here. A `DELEGATED` gate is satisfied by its CI
|
|
170
|
+
check, which runs on the pull request: it is not a pass, and it is not a
|
|
171
|
+
failure until that check is red. `FAIL` and `UNRUN` go in the stop report with
|
|
172
|
+
the reason and the retry condition. Do not weaken a profile, lower `--fail-on`,
|
|
77
173
|
waive an error or invent a result to reach `PASS`.
|
|
174
|
+
|
|
175
|
+
## A failure that predates the change
|
|
176
|
+
|
|
177
|
+
Before a failing gate is attributed to the change, run it again on the base
|
|
178
|
+
commit: the revision the change branched from, in a clean checkout that holds
|
|
179
|
+
none of the change. Build it the same way, so a failure that needs a build is
|
|
180
|
+
not read from a stale one.
|
|
181
|
+
|
|
182
|
+
| Base result | Record |
|
|
183
|
+
| -------------------------------------- | -------------------------------------------- |
|
|
184
|
+
| The same failures | `FAIL`, baseline `same`: it predates the run |
|
|
185
|
+
| Fewer, other or no failures | `FAIL`, baseline `different`: the run's own |
|
|
186
|
+
| The gate cannot run on the base either | `FAIL`, baseline `unrun`, with the reason |
|
|
187
|
+
|
|
188
|
+
The comparison is between the failing tests or findings, not the exit code.
|
|
189
|
+
The result stays `FAIL`; a baseline of `same` explains it and never turns it
|
|
190
|
+
into a pass. The run did not cause it, so it is reported and not repaired.
|