@aperant/framework 0.11.0 → 0.12.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (149) hide show
  1. package/CHANGELOG.md +226 -29
  2. package/agents/apt-planner.md +15 -0
  3. package/agents/apt-verifier.md +15 -10
  4. package/bin/apt-proof-video.mjs +256 -76
  5. package/dist/cli/commands/apply-recipe.d.mts +6 -0
  6. package/dist/cli/commands/apply-recipe.d.mts.map +1 -0
  7. package/dist/cli/commands/apply-recipe.mjs +98 -0
  8. package/dist/cli/commands/apply-recipe.mjs.map +1 -0
  9. package/dist/cli/commands/mobile-prepare.d.mts +53 -0
  10. package/dist/cli/commands/mobile-prepare.d.mts.map +1 -0
  11. package/dist/cli/commands/mobile-prepare.mjs +139 -0
  12. package/dist/cli/commands/mobile-prepare.mjs.map +1 -0
  13. package/dist/cli/commands/recipe-diff.d.mts +6 -0
  14. package/dist/cli/commands/recipe-diff.d.mts.map +1 -0
  15. package/dist/cli/commands/recipe-diff.mjs +75 -0
  16. package/dist/cli/commands/recipe-diff.mjs.map +1 -0
  17. package/dist/cli/commands/roadmap.d.mts.map +1 -1
  18. package/dist/cli/commands/roadmap.mjs +53 -0
  19. package/dist/cli/commands/roadmap.mjs.map +1 -1
  20. package/dist/cli/commands/validate-evidence.d.mts +1 -1
  21. package/dist/cli/commands/verify-sandbox.d.mts +6 -0
  22. package/dist/cli/commands/verify-sandbox.d.mts.map +1 -0
  23. package/dist/cli/commands/verify-sandbox.mjs +200 -0
  24. package/dist/cli/commands/verify-sandbox.mjs.map +1 -0
  25. package/dist/cli/consistency/parse-qa.d.mts.map +1 -1
  26. package/dist/cli/consistency/parse-qa.mjs +18 -4
  27. package/dist/cli/consistency/parse-qa.mjs.map +1 -1
  28. package/dist/cli/consistency/rules/r5-verdict-consistency.d.mts.map +1 -1
  29. package/dist/cli/consistency/rules/r5-verdict-consistency.mjs +11 -10
  30. package/dist/cli/consistency/rules/r5-verdict-consistency.mjs.map +1 -1
  31. package/dist/cli/coverage-check/user-outcomes.d.mts +7 -2
  32. package/dist/cli/coverage-check/user-outcomes.d.mts.map +1 -1
  33. package/dist/cli/coverage-check/user-outcomes.mjs +50 -17
  34. package/dist/cli/coverage-check/user-outcomes.mjs.map +1 -1
  35. package/dist/cli/design/frontmatter-schema.d.mts +8 -8
  36. package/dist/cli/dispatch.d.mts.map +1 -1
  37. package/dist/cli/dispatch.mjs +28 -0
  38. package/dist/cli/dispatch.mjs.map +1 -1
  39. package/dist/cli/gate/gates/verify-approved.d.mts +0 -48
  40. package/dist/cli/gate/gates/verify-approved.d.mts.map +1 -1
  41. package/dist/cli/gate/gates/verify-approved.mjs +23 -80
  42. package/dist/cli/gate/gates/verify-approved.mjs.map +1 -1
  43. package/dist/cli/install/mcp-server-specs.d.mts +6 -5
  44. package/dist/cli/install/mcp-server-specs.d.mts.map +1 -1
  45. package/dist/cli/install/mcp-server-specs.mjs +22 -4
  46. package/dist/cli/install/mcp-server-specs.mjs.map +1 -1
  47. package/dist/cli/personas/sidecar.d.mts +1 -1
  48. package/dist/cli/roadmap/backend-adapter.d.mts +43 -0
  49. package/dist/cli/roadmap/backend-adapter.d.mts.map +1 -0
  50. package/dist/cli/roadmap/backend-adapter.mjs +222 -0
  51. package/dist/cli/roadmap/backend-adapter.mjs.map +1 -0
  52. package/dist/cli/roadmap/lifecycle.d.mts +1 -1
  53. package/dist/cli/roadmap/rollup.d.mts +2 -2
  54. package/dist/cli/util/runtime-capabilities.d.mts +7 -0
  55. package/dist/cli/util/runtime-capabilities.d.mts.map +1 -1
  56. package/dist/cli/util/runtime-capabilities.mjs +65 -0
  57. package/dist/cli/util/runtime-capabilities.mjs.map +1 -1
  58. package/dist/cli/verify-proof/manifest-validator.mjs +13 -0
  59. package/dist/cli/verify-proof/manifest-validator.mjs.map +1 -1
  60. package/dist/cli/verify-proof/resolver.d.mts +10 -7
  61. package/dist/cli/verify-proof/resolver.d.mts.map +1 -1
  62. package/dist/cli/verify-proof/resolver.mjs +28 -7
  63. package/dist/cli/verify-proof/resolver.mjs.map +1 -1
  64. package/dist/cli/verify-proof/sandbox/apply.d.mts +64 -0
  65. package/dist/cli/verify-proof/sandbox/apply.d.mts.map +1 -0
  66. package/dist/cli/verify-proof/sandbox/apply.mjs +379 -0
  67. package/dist/cli/verify-proof/sandbox/apply.mjs.map +1 -0
  68. package/dist/cli/verify-proof/sandbox/gate-predicate.d.mts +63 -0
  69. package/dist/cli/verify-proof/sandbox/gate-predicate.d.mts.map +1 -0
  70. package/dist/cli/verify-proof/sandbox/gate-predicate.mjs +115 -0
  71. package/dist/cli/verify-proof/sandbox/gate-predicate.mjs.map +1 -0
  72. package/dist/cli/verify-proof/sandbox/pattern-matcher.d.mts +57 -0
  73. package/dist/cli/verify-proof/sandbox/pattern-matcher.d.mts.map +1 -0
  74. package/dist/cli/verify-proof/sandbox/pattern-matcher.mjs +124 -0
  75. package/dist/cli/verify-proof/sandbox/pattern-matcher.mjs.map +1 -0
  76. package/dist/cli/verify-proof/sandbox/recipe-diff.d.mts +28 -0
  77. package/dist/cli/verify-proof/sandbox/recipe-diff.d.mts.map +1 -0
  78. package/dist/cli/verify-proof/sandbox/recipe-diff.mjs +190 -0
  79. package/dist/cli/verify-proof/sandbox/recipe-diff.mjs.map +1 -0
  80. package/dist/cli/verify-proof/sandbox/sandbox-config.d.mts +25 -0
  81. package/dist/cli/verify-proof/sandbox/sandbox-config.d.mts.map +1 -0
  82. package/dist/cli/verify-proof/sandbox/sandbox-config.mjs +85 -0
  83. package/dist/cli/verify-proof/sandbox/sandbox-config.mjs.map +1 -0
  84. package/dist/cli/verify-proof/sandbox/self-test.d.mts +39 -0
  85. package/dist/cli/verify-proof/sandbox/self-test.d.mts.map +1 -0
  86. package/dist/cli/verify-proof/sandbox/self-test.mjs +248 -0
  87. package/dist/cli/verify-proof/sandbox/self-test.mjs.map +1 -0
  88. package/dist/plugin/.claude-plugin/plugin.json +1 -1
  89. package/dist/plugin/agents/apt-planner.md +15 -0
  90. package/dist/plugin/agents/apt-verifier.md +15 -10
  91. package/dist/plugin/skills/apt/SKILL.md +2 -2
  92. package/dist/plugin/skills/apt-plan/SKILL.md +23 -0
  93. package/dist/plugin/skills/apt-run/SKILL.md +1 -1
  94. package/dist/plugin/skills/apt-setup/SKILL.md +56 -0
  95. package/dist/plugin/skills/apt-verify/SKILL.md +20 -15
  96. package/dist/plugin/skills/apt-verify-proof/SKILL.md +146 -11
  97. package/dist/proof-report.d.ts.map +1 -1
  98. package/dist/proof-report.js +4 -1
  99. package/dist/proof-report.js.map +1 -1
  100. package/dist/types/config.d.ts +42 -4
  101. package/dist/types/config.d.ts.map +1 -1
  102. package/dist/types/qa-scoring.d.ts +45 -9
  103. package/dist/types/qa-scoring.d.ts.map +1 -1
  104. package/dist/types/qa-scoring.js +34 -13
  105. package/dist/types/qa-scoring.js.map +1 -1
  106. package/dist/types/state.d.ts +7 -2
  107. package/dist/types/state.d.ts.map +1 -1
  108. package/dist/types/task-record.d.ts +10 -2
  109. package/dist/types/task-record.d.ts.map +1 -1
  110. package/drivers/mobile/README.md +40 -0
  111. package/drivers/mobile/driver.mjs +106 -0
  112. package/drivers/mobile/manifest.json +49 -0
  113. package/package.json +138 -138
  114. package/prompts/inbox-clarification.md +17 -0
  115. package/prompts/inbox-triage.md +21 -0
  116. package/prompts/qa_orchestrator_agentic.md +18 -17
  117. package/prompts/qa_reviewer.md +17 -15
  118. package/skills/apt/SKILL.md +2 -2
  119. package/skills/apt-plan/SKILL.md +23 -0
  120. package/skills/apt-run/SKILL.md +1 -1
  121. package/skills/apt-setup/SKILL.md +56 -0
  122. package/skills/apt-verifier.md +7 -4
  123. package/skills/apt-verify/SKILL.md +20 -15
  124. package/skills/apt-verify-proof/SKILL.md +146 -11
  125. package/src/cli/commands/apply-recipe.mjs +105 -0
  126. package/src/cli/commands/mobile-prepare.mjs +151 -0
  127. package/src/cli/commands/recipe-diff.mjs +81 -0
  128. package/src/cli/commands/roadmap.mjs +54 -0
  129. package/src/cli/commands/verify-sandbox.mjs +231 -0
  130. package/src/cli/consistency/parse-qa.mjs +20 -4
  131. package/src/cli/consistency/rules/r5-verdict-consistency.mjs +11 -13
  132. package/src/cli/coverage-check/user-outcomes.mjs +52 -17
  133. package/src/cli/dispatch.mjs +27 -0
  134. package/src/cli/gate/gates/verify-approved.mjs +22 -81
  135. package/src/cli/install/mcp-server-specs.mjs +24 -4
  136. package/src/cli/roadmap/backend-adapter.mjs +231 -0
  137. package/src/cli/util/runtime-capabilities.mjs +67 -0
  138. package/src/cli/verify-proof/manifest-validator.mjs +15 -0
  139. package/src/cli/verify-proof/resolver.mjs +27 -7
  140. package/src/cli/verify-proof/sandbox/apply.mjs +401 -0
  141. package/src/cli/verify-proof/sandbox/gate-predicate.mjs +126 -0
  142. package/src/cli/verify-proof/sandbox/pattern-matcher.mjs +127 -0
  143. package/src/cli/verify-proof/sandbox/recipe-diff.mjs +208 -0
  144. package/src/cli/verify-proof/sandbox/sandbox-config.mjs +82 -0
  145. package/src/cli/verify-proof/sandbox/sandbox-schema.json +33 -0
  146. package/src/cli/verify-proof/sandbox/self-test.mjs +265 -0
  147. package/templates/config.json +10 -1
  148. package/templates/proof-verification.md +27 -8
  149. package/workflows/verify-proof.md +376 -302
@@ -1,10 +1,14 @@
1
1
  <purpose>
2
2
  Visual proof-of-work verification with structured evidence capture. Creates verification.json
3
- that tracks every test with action → evidence → verdict. Produces HTML proof report.
3
+ that tracks every outcome with action → evidence → verdict. Produces HTML proof report.
4
4
 
5
5
  This workflow is adversarial — it assumes things are broken until proven working.
6
6
  Every claim requires evidence. Every screenshot follows an interaction.
7
7
 
8
+ v0.9.0 — **outcome-walker**: iterate `spec.md ## User Outcomes` parent bullets
9
+ one at a time, reload ONLY that outcome card into working memory, drive the
10
+ stated bullet AND each inferred child, fail-loop on red, advance on green.
11
+
8
12
  OUTPUT FORMAT: verification.json (NOT VERIFICATION.md). JSON is the source of truth.
9
13
  </purpose>
10
14
 
@@ -40,7 +44,6 @@ Read spec and plan:
40
44
  ```bash
41
45
  cat spec.md 2>/dev/null
42
46
  cat implementation_plan.json 2>/dev/null
43
- cat testable_features.json 2>/dev/null
44
47
  ```
45
48
 
46
49
  Generate run ID and create proof directory. v0.8.6 — when the spec
@@ -65,12 +68,41 @@ screenshot for that surface.
65
68
  </step>
66
69
 
67
70
  <step name="pre_check">
68
- ## 2. Pre-Check + Driver Selection
71
+ ## 2. Pre-Check + Driver Selection + Outcome Loader
69
72
 
70
73
  If `implementation_plan.json` exists, verify all subtasks are completed.
71
74
  If any are pending: report and exit.
72
75
 
73
- ### 2a. Resolve the verify-proof driver
76
+ ### 2a. Load `## User Outcomes` from spec.md (HARD REQUIREMENT — v0.9.0)
77
+
78
+ `## User Outcomes` is the verify-proof input contract. Parse the
79
+ section via `parseUserOutcomes`:
80
+
81
+ ```js
82
+ import { parseUserOutcomes } from '.aperant/deps/node_modules/@aperant/framework/src/cli/coverage-check/user-outcomes.mjs'
83
+ const parsed = parseUserOutcomes(readFileSync('spec.md', 'utf-8'))
84
+ ```
85
+
86
+ If `parsed.body_kind !== 'outcomes'` AND `parsed.body_kind !== 'empty-with-note'`,
87
+ **ABORT** with a clear error: `"spec.md missing ## User Outcomes — verify-proof
88
+ requires the outcome contract. Run /apt:plan to author it."`
89
+
90
+ **Do NOT fall back to feature-registry-derived tests.** The legacy
91
+ "discover features from the registry" path is removed (A4 escape hatch).
92
+ The feature registry is informational only — it does NOT seed test
93
+ generation. Every test entry must trace back to a parent or inferred
94
+ outcome from `## User Outcomes`.
95
+
96
+ For each parent outcome, `parsed.outcomes[i]` carries:
97
+
98
+ - `id` — positional ID (`O1`, `O2`, ...)
99
+ - `surface` — closed set `{electron, web, cli, api, electron+web}`
100
+ - `text` — outcome prose (the parent bullet)
101
+ - `inferred` — string[] of indented child bullets (the planning LLM's
102
+ derived sub-checks; empty `[]` when the author wrote none)
103
+ - `requires` — optional dependency on a prior outcome's ID
104
+
105
+ ### 2b. Resolve the verify-proof driver
74
106
 
75
107
  The workflow is **runtime-agnostic** — it refers to IDL verbs
76
108
  (`driver.click(target)`, `driver.screenshot(path)`), never to concrete
@@ -95,9 +127,8 @@ If no driver satisfies a REQUIRED capability, the resolver throws
95
127
  `unsatisfied_required_capability` — fail-closed, NOT a silent
96
128
  `blocked_by_platform` skip (per ID-02).
97
129
 
98
- **v0.8.6 — per-outcome dispatch.** When the spec carries
99
- `## User Outcomes`, the workflow runner picks a driver PER OUTCOME
100
- SURFACE via `resolveDriversForOutcomes()` instead of a single
130
+ **v0.8.6 — per-outcome dispatch.** The workflow runner picks a driver
131
+ PER OUTCOME SURFACE via `resolveDriversForOutcomes()` instead of a single
101
132
  per-run driver pick. Each outcome's `[surface]` tag maps to the
102
133
  runtime-capability flag a driver must carry; `[electron+web]`
103
134
  outcomes dispatch to TWO drivers (one per surface) sharing the same
@@ -107,7 +138,7 @@ still works for the per-run "what driver would be picked?" preview
107
138
  the workflow runner imports `resolveDriversForOutcomes` directly
108
139
  from the resolver module in Step 7c.
109
140
 
110
- ### 2b. Probe optional evidence-capture tools
141
+ ### 2c. Probe optional evidence-capture tools
111
142
 
112
143
  ```bash
113
144
  # CDP video capture (optional)
@@ -120,223 +151,185 @@ sandbox_profile}` and `meta.fallback_attempts: [{driver_id, reason}]`
120
151
  (see ST11's schema extension in `templates/proof-verification.md`).
121
152
  </step>
122
153
 
123
- <step name="automated_checks">
124
- ## 3. Run Automated Checks (with output capture)
154
+ <step name="load_deferred_tools">
155
+ ## 2.5. Load Deferred Driver Tools (Claude Code / OpenCode hosts)
125
156
 
126
- Run each check and capture the FULL terminal output for the proof report.
157
+ > **Precondition.** When `--parallel` is requested, the sandbox gate at apt-verify-proof/SKILL.md Step 2c runs BEFORE this step. An unverified sandbox exits with `{status:"error", code:"E_SANDBOX_UNVERIFIED", verdict:"needs_human", reason:"sandbox_unverified"}` without loading any MCP driver tools or entering the outcome-walker loop. See `docs/verify-proof-sandbox-patterns.md` for setup.
127
158
 
128
- **IMPORTANT:** Capture output, don't just check exit codes.
159
+ On Claude Code and OpenCode hosts, the MCP driver tools listed in
160
+ `drivers/<surface>/manifest.json :: transport.mcp_server_id` are
161
+ **deferred** — they exist in the host's MCP server registry but their
162
+ schemas are NOT loaded by default. Before driving ANY UI-surface outcome
163
+ you MUST load them via `ToolSearch(query='select:...')`.
129
164
 
130
- ```bash
131
- # Tests
132
- pnpm test 2>&1 | head -200
133
- ```
165
+ **Worked example.** For the bundled `browser` driver (whose manifest
166
+ field `transport.mcp_server_id = "puppeteer"` exposes
167
+ `mcp__puppeteer__*` tools):
134
168
 
135
- ```bash
136
- # Type check
137
- pnpm typecheck 2>&1 | head -100
169
+ ```
170
+ ToolSearch(query='select:mcp__puppeteer__browser_navigate,mcp__puppeteer__browser_click,mcp__puppeteer__browser_screenshot,mcp__puppeteer__browser_type')
138
171
  ```
139
172
 
140
- ```bash
141
- # Lint
142
- pnpm lint 2>&1 | head -100
173
+ For the bundled `electron` driver (whose manifest field
174
+ `transport.mcp_server_id = "electron"` exposes `mcp__electron__*` tools):
175
+
176
+ ```
177
+ ToolSearch(query='select:mcp__electron__app_launch,mcp__electron__app_click,mcp__electron__app_screenshot,mcp__electron__app_type')
143
178
  ```
144
179
 
145
- Record results in verification.json `automated_checks` object with full output.
180
+ After the `ToolSearch` call, list the now-loaded tool names back to
181
+ confirm they appear in your tool surface. If any required driver tool
182
+ fails to load, emit `result: "needs_human_with_transport"` on every
183
+ outcome targeting that surface and proceed — do NOT fall back to
184
+ file-evidence (the legacy "commit hash as proof" pattern is rejected
185
+ by the proof_gate).
186
+
187
+ The manifest field is `transport.mcp_server_id` (and
188
+ `transport.skill_id` where applicable). The names
189
+ `mcp_transport.tool_prefix` and `transport.tool_prefix` are NOT real
190
+ manifest fields — do not search for them.
191
+
192
+ This step is a no-op on host CLIs whose entire MCP surface is loaded
193
+ eagerly (e.g. some Codex builds). Run it unconditionally; if the
194
+ selected tools are already in the surface, `ToolSearch` is idempotent.
146
195
  </step>
147
196
 
148
- <step name="discover_features">
149
- ## 4. Deep Feature Discovery + Coverage Gate
197
+ <step name="automated_checks">
198
+ ## 3. Run Automated Checks (HARD STOP gate — no "enforced at commit" dodge)
150
199
 
151
- **This step determines verification quality. Do not rush it.**
200
+ Run each of `pnpm lint`, `pnpm test`, `pnpm typecheck` (or the
201
+ configured equivalents) IN THIS RUN and capture the full terminal output.
152
202
 
153
- ### 4a. Check the Feature Registry FIRST
203
+ **Closed set for status.** Each check's `status` MUST be one of
204
+ `{pass, fail}`. The legacy string `"enforced_at_commit"` is no longer
205
+ valid — if you find yourself about to write it, STOP. You must actually
206
+ execute the check in this run.
154
207
 
155
208
  ```bash
156
- ls .aperant/features/ 2>/dev/null
157
- cat .aperant/features/{area}.json 2>/dev/null
209
+ # Tests
210
+ pnpm test 2>&1 | tee /tmp/proof-tests.log
158
211
  ```
159
212
 
160
- If `.aperant/features/{area}.json` exists for the area under test:
161
- - **Use it as your primary source** — it lists every sub-feature, wiring status, and test hints
162
- - Still verify against current code (the registry may be slightly stale)
163
- - If you discover features NOT in the registry, add them and flag: "Unregistered feature found"
164
-
165
- If no feature registry exists:
166
- - Suggest: "No feature registry found. Consider running `/apt:scan {area}` first for exhaustive coverage."
167
- - Proceed with manual discovery below, but warn that coverage may be incomplete
168
-
169
- **v0.8.6 — read `## User Outcomes` from spec.md FIRST when present.**
170
- The PRD-shaped spec.md (STANDARD/DEEP tracks) ships a
171
- `## User Outcomes` section that is the verify-proof input contract:
172
- `**Epic:**` headline + N atomic outcomes tagged by `[surface]`. When
173
- present, each outcome maps 1:1 to a test entry — the workflow runner
174
- does NOT have to invent test units from the feature registry. Fall
175
- back to the feature-registry-based discovery below ONLY when the spec
176
- lacks `## User Outcomes` (the v0.8.x warn-not-block migration window
177
- per spec ID-06; the section becomes a hard block in v0.9.0).
178
-
179
- ### 4b. Read the source code
180
-
181
- You MUST read the actual source code for the features under test. Do not guess.
182
- For example, if testing a terminal feature:
183
- - Read the component files to find every prop, every button, every interaction
184
- - Read settings/config components to find every slider, dropdown, toggle
185
- - Read toolbar components to find every button and its handler
186
- - Read any test files for hints about expected behavior
213
+ ```bash
214
+ # Type check
215
+ pnpm typecheck 2>&1 | tee /tmp/proof-typecheck.log
216
+ ```
187
217
 
188
- ### 4c. Enumerate ALL sub-features
218
+ ```bash
219
+ # Lint
220
+ pnpm lint 2>&1 | tee /tmp/proof-lint.log
221
+ ```
189
222
 
190
- For each feature area, list every individual testable element:
191
- - Every button → what does it do when clicked?
192
- - Every input/slider → what range does it accept? What does it change?
193
- - Every dropdown → what are the options? What does each option do?
194
- - Every toggle → what does it enable/disable?
195
- - Every keyboard shortcut → what does it trigger?
223
+ Record results in `verification.json :: automated_checks` with full
224
+ output. Each entry shape:
196
225
 
197
- ### 4d. Compare against reference (if available)
226
+ ```json
227
+ {
228
+ "automated_checks": {
229
+ "tests": { "status": "pass | fail", "output": "...", "count": 42 },
230
+ "typecheck": { "status": "pass | fail", "output": "..." },
231
+ "lint": { "status": "pass | fail", "output": "..." }
232
+ }
233
+ }
234
+ ```
198
235
 
199
- If the feature was ported from another codebase, or there is a design reference:
200
- - What should it look like when working correctly?
201
- - What visual quality markers indicate production-readiness?
202
- - A clean shell prompt with path and git branch is NOT the same as a blank terminal with cursor artifacts
236
+ **Hard STOP gate.** If any of the three checks has a `status` outside
237
+ `{pass, fail}` (e.g. `enforced_at_commit`, `skipped`, `unknown`), the
238
+ run aborts immediately with `automation_gate: fail` and overall
239
+ verdict `rejected`. The renderer / mapper rejects any input
240
+ verification.json carrying a non-conforming status.
203
241
 
204
- ### 4e. Assess what CAN be tested via MCP
242
+ **Verdict roll-up.** `automation_gate = pass` iff ALL THREE of tests,
243
+ typecheck, lint are `pass`. Any single `fail` → `automation_gate: fail`
244
+ → overall verdict `rejected` regardless of `proof_gate`.
245
+ </step>
205
246
 
206
- For each sub-feature, decide:
207
- - **testable**: can interact via MCP click/type/navigate + verify via screenshot
208
- - **observable**: can see in screenshot but cannot interact (e.g., xterm.js internal rendering)
209
- - **blocked**: cannot test or observe via available tools
247
+ <step name="execute_outcomes">
248
+ ## 4. Execute Outcomes (outcome-walker loop)
210
249
 
211
- Mark blocked items honestly. Do NOT mark them as pass.
250
+ **This step replaces the legacy "Deep Feature Discovery" + "Generate
251
+ Tests" + "Execute Tests" sequence.** The workflow runner iterates the
252
+ parent outcomes from Step 2a (`parsed.outcomes`) ONE AT A TIME, in
253
+ document order. For each parent outcome, drive the stated bullet AND
254
+ each inferred child, capture surface-appropriate evidence per child,
255
+ fix-loop on red, advance on green.
212
256
 
213
- ### 4f. Coverage Gate (MANDATORY blocks test execution)
257
+ ### 4a. Reload ONLY the current outcome card
214
258
 
215
- After generating your test list, calculate coverage against the feature registry:
259
+ Before driving outcome `Oi`, reload ONLY that outcome's card into
260
+ working memory:
216
261
 
217
262
  ```
218
- registered_count = total sub-features in .aperant/features/{area}.json
219
- tested_count = number of tests written (each linking to a registry_id)
220
- coverage = tested_count / registered_count
263
+ const outcome = parsed.outcomes[i]
264
+ // outcome = { id, surface, text, inferred[], requires? }
221
265
  ```
222
266
 
223
- **Coverage rules:**
224
- - If `coverage < 50%`: **BLOCK** "Only {tested_count}/{registered_count} features have tests. Add more tests before proceeding."
225
- - Do NOT proceed to step 5. Go back and write more tests.
226
- - If `coverage < 80%`: **WARN** — list the top untested features by importance. Proceed but note the gap.
227
- - If `coverage >= 80%`: Proceed normally.
267
+ Do NOT keep the prior outcome's evidence, screenshots, or driver state
268
+ in active memorythis defends against context bloat on long runs
269
+ (30+ outcomes). Persist progress to `verification.json :: tests[]`
270
+ between outcomes; that file is the durable source of truth.
228
271
 
229
- **Every untested registered feature MUST go in the `skipped` array** with a reason.
230
- Valid skip reasons:
231
- - `time_constraint` — would take too long relative to value
232
- - `unsatisfied_optional_capability` — the selected driver lacks an OPTIONAL IDL verb needed by this test (e.g. `assert_network` not implemented). Loud, deliberate skip.
233
- - `depends_on_other` — requires a feature not yet implemented
234
- - `not_applicable` — feature is deprecated or not relevant to this verification
272
+ ### 4b. Drive the stated bullet (IDL-verb dispatch, MUST-invoke directives)
235
273
 
236
- > **Legacy alias.** `blocked_by_platform` (pre-driver-architecture) is read by the schema reader as an alias for `unsatisfied_optional_capability` and emits a deprecation warning. New writes MUST use the new reason.
274
+ Use the per-outcome driver from `resolveDriversForOutcomes()`:
237
275
 
238
- > **Fail-closed, not a skip:** if a REQUIRED IDL capability has no implementing driver in the resolver chain, the resolver throws `unsatisfied_required_capability` (per ID-02). This aborts the run — it does NOT silently land in the `skipped` array. Coverage loss is loud, deliberate, and audited.
276
+ ```js
277
+ import { resolveDriversForOutcomes } from '.aperant/deps/node_modules/@aperant/framework/src/cli/verify-proof/resolver.mjs'
239
278
 
240
- Silence about untested features is NOT acceptable. Every registered sub-feature must appear
241
- either as a test (with `registry_id`) or in the `skipped` array.
242
- </step>
279
+ const entries = resolveDriversForOutcomes({
280
+ drivers, runtime,
281
+ outcomes: parsed.outcomes,
282
+ })
283
+ // → [{outcome_id: "O1", surface: "electron", driver: {...}}, ...]
284
+ // For [electron+web] outcomes, two entries share outcome_id.
285
+ ```
243
286
 
244
- <step name="generate_tests">
245
- ## 5. Generate Test Cases
246
-
247
- For EVERY sub-feature discovered in step 4, generate an **action-based test**.
248
-
249
- Each test MUST have:
250
- - **id**: sequential test ID (e.g., "TEST-01")
251
- - **registry_id**: the sub-feature ID from `.aperant/features/{area}.json` (e.g., "terminals-grid-auto-tile")
252
- - **name**: descriptive test name
253
- - **precondition**: what must be true before testing (e.g., "app is running, terminal tab is active")
254
- - **action**: a specific thing to DO (e.g., "Type 'echo hello' and press Enter")
255
- - **expected**: a specific observable result (e.g., "Terminal displays 'hello' on the next line")
256
-
257
- **Rules for writing tests:**
258
- 1. Every test involves an interaction (type, click, submit, navigate) OR a visual quality check
259
- 2. "It renders" is NOT a valid test — "it renders X with properties Y when you do Z" IS
260
- 3. If a feature has a text input, you must type into it
261
- 4. If a feature has a button, you must click it
262
- 5. If a feature has a slider, you must change it and verify the change took effect
263
- 6. If a feature creates something, verify the creation AND its quality
264
- 7. **Visual quality tests are valid** — "terminal shows clean prompt with path and git branch" IS a valid test
265
- 8. **Artifact detection is valid** — "no rendering artifacts, stray characters, or broken formatting" IS a valid test
266
- 9. Test sub-features exhaustively — if settings has 5 sliders, write 5 tests, not 1
267
- 10. **Every test must reference a `registry_id`** from the feature registry. This links tests to sub-features and makes coverage tracking automatic.
268
-
269
- Write ALL tests into verification.json `tests` array before starting execution.
270
-
271
- **Gate:** Do not proceed to step 6 until ALL tests are written AND the coverage gate from step 4f passes.
272
- </step>
287
+ For each `entry` whose `outcome_id` matches `outcome.id`:
273
288
 
274
- <step name="start_video">
275
- ## 6. Start Video Capture
289
+ 1. Compose an action sequence from `outcome.text` (the parent prose
290
+ describes the user-facing capability — translate into IDL verbs):
291
+ - If it says "type X" → `driver.type({ target, text: "X" })`
292
+ - If it says "click X" → `driver.click({ target })`
293
+ - If it says "navigate to X" → `driver.navigate({ url: "X" })`
294
+ - If it says "press key X" → `driver.key({ key: "X" })`
295
+ 2. Invoke IDL verbs on `entry.driver`.
296
+ 3. Capture surface-appropriate evidence per Step 4d.
297
+ 4. Record the test entry in `verification.json :: tests[]` with
298
+ `outcome_id = outcome.id` and `surface = entry.surface`.
276
299
 
277
- If CDP capture is available, start recording BEFORE executing tests:
300
+ **Transport dispatch directive (MUST honor — TD-08).** When `driver.transport.skill_id` is set AND the host CLI exposes the Skill tool AND the outcome surface is `web`, the workflow runner MUST invoke `Skill({skill: driver.transport.skill_id, args: ...})` (passing the IDL verb args) and MUST record `transport: "skill"` on the resulting test entry in verification.json. When the host CLI does NOT expose the Skill tool, fall back to the MCP transport via the existing puppeteer MCP server and record `transport: "mcp"`. The driver's `chooseTransport({has_skill_tool})` helper computes the choice; the workflow runner is the site that knows `has_skill_tool`. This directive is LLM-honored (the v1 verify-proof runner is the host LLM following this workflow as prose); a text-level grep guard (TD-08) pins the `MUST` language so future edits cannot silently soften it to `SHOULD`.
278
301
 
279
- ```bash
280
- apt-proof-video capture --port 9222 --duration {estimated_total_seconds} --fps 2 --output ${PROOF_DIR}
281
- ```
302
+ ### 4c. Drive each inferred child
282
303
 
283
- Run this in the background. The video will capture your ACTUAL test session —
284
- the same interactions that produce screenshots. This ensures video and screenshots
285
- are from the SAME session.
304
+ For each `child` in `outcome.inferred`:
286
305
 
287
- **Important:** Estimate duration generously. Better to have extra video than to miss tests.
288
- - ~10 seconds per test is a reasonable estimate
289
- - Add 30 seconds for navigation overhead
290
- - If 20 tests: --duration 230
306
+ 1. Compose an action sequence from the child prose (these are the
307
+ sub-checks the planning LLM derived from the parent — e.g. "X-button
308
+ closes", "outside-click closes", "Esc closes + returns focus").
309
+ 2. Invoke IDL verbs.
310
+ 3. Capture evidence per Step 4d.
311
+ 4. Record a SEPARATE test entry in `verification.json :: tests[]`
312
+ carrying:
313
+ - `outcome_id` = `outcome.id` (same as parent)
314
+ - `inferred_index` = index of the child in `outcome.inferred[]`
315
+ - `surface` = `entry.surface`
316
+ - the standard `action / expected / actual / evidence / result` fields.
291
317
 
292
- **Note:** Captured frames are auto-stitched into an MP4 file by the tool. No manual assembly needed.
318
+ ### 4d. Capture surface-appropriate evidence (fail-closed)
293
319
 
294
- If CDP is NOT available, note it and proceed without video.
295
- </step>
320
+ Surface evidence requirements:
296
321
 
297
- <step name="execute_tests">
298
- ## 7. Execute Tests (one at a time)
299
-
300
- For each test, in order:
301
-
302
- ### 7a. Perform the Action
303
- Actually do what the test says — invoke the IDL verb on the selected driver:
304
- - If it says "type X" → `driver.type({ target, text: "X" })`
305
- - If it says "click X" → `driver.click({ target })`
306
- - If it says "navigate to X" → `driver.navigate({ url: "X" })`
307
- - If it says "press key X" → `driver.key({ key: "X" })`
308
- - If it says "run command X" → `driver.execute_js({ script: "X" })`
309
-
310
- The driver layer translates IDL verbs to its transport (an MCP server,
311
- a local binary, or a built-in HTTP request). Workflow prose names ONLY
312
- IDL verbs per ID-01 — concrete tool names live in
313
- `packages/framework/drivers/<driver>/driver.mjs`.
314
-
315
- **v0.8.6 — transport dispatch directive (MUST honor).** When
316
- `driver.transport.skill_id` is set AND the host CLI exposes the
317
- Skill tool AND the outcome surface is `web`, the workflow runner
318
- MUST invoke `Skill({skill: driver.transport.skill_id, args: ...})`
319
- (passing the IDL verb args) and MUST record `transport: "skill"` on
320
- the resulting test entry in verification.json. When the host CLI
321
- does NOT expose the Skill tool, fall back to the MCP transport via
322
- the existing puppeteer MCP server and record `transport: "mcp"`.
323
- The driver's `chooseTransport({has_skill_tool})` helper computes
324
- the choice; the workflow runner is the site that knows
325
- `has_skill_tool`. This directive is LLM-honored (the v1 verify-proof
326
- runner is the host LLM following this workflow as prose); a
327
- text-level grep guard (TD-08) pins the `MUST` language so future
328
- edits cannot silently soften it to `SHOULD`.
329
-
330
- ### 7b. Wait for Result
331
- Wait for the action to complete. If it involves loading, wait for the load.
332
- Add a short delay (1-2 seconds) so the CDP capture also captures this state.
333
-
334
- ### 7c. Capture Evidence
335
- Take a screenshot AFTER the action completes. v0.8.6 — screenshots
336
- land under PER-SURFACE subdirectories, with the per-outcome driver
337
- returned by `resolveDriversForOutcomes()`:
322
+ | Surface | Acceptable evidence |
323
+ |---|---|
324
+ | `electron`, `web`, `electron+web` | Screenshot from `driver.screenshot()` under `${PROOF_DIR}/${surface}/screenshots/...` |
325
+ | `cli` | Captured command output (stdout/stderr) under `${PROOF_DIR}/cli/output/...` |
326
+ | `api` | HTTP request/response capture under `${PROOF_DIR}/api/captures/...` |
338
327
 
339
- ```
328
+ For UI-surface outcomes, take a screenshot AFTER each action completes.
329
+ Screenshots land under PER-SURFACE subdirectories — the per-outcome
330
+ driver returned by `resolveDriversForOutcomes()` carries the surface:
331
+
332
+ ```js
340
333
  // Pick a driver per outcome surface (NOT per run). `entries[]` is
341
334
  // the resolver's per-(outcome, surface) dispatch — for an
342
335
  // [electron+web] outcome you get TWO entries sharing outcome_id
@@ -351,130 +344,217 @@ for (const entry of entries) {
351
344
 
352
345
  Single-surface outcomes (`[electron]`, `[web]`, `[cli]`, `[api]`)
353
346
  produce one screenshot under `${PROOF_DIR}/${surface}/screenshots/`;
354
- cross-surface (`[electron+web]`) produces two — one per surface tree
355
- with both test entries in `verification.json:tests[]` carrying the
356
- same `id` but distinct `surface` and `evidence` fields.
357
-
358
- ### 7d. Analyze Evidence
359
- **MANDATORY and CRITICAL.** Every screenshot must be analyzed in detail.
360
-
361
- You MUST describe what you see — not what you expect to see. Be adversarial:
362
- - Are there rendering artifacts? Stray characters? Broken formatting?
363
- - Does the UI look production-quality or does it look broken?
364
- - Compare against what a real user would expect to see
365
- - If testing a terminal: is the prompt clean? Is the path correct? Are there garbage characters?
366
- - If testing settings: did the value actually change? Is the control responsive?
347
+ cross-surface (`[electron+web]`) produces two — one per surface tree
348
+ with both test entries in `verification.json :: tests[]` carrying the
349
+ same `outcome_id` but distinct `surface` and `evidence` fields.
367
350
 
368
- Write the analysis into the test's `analysis` field. Be specific and critical.
369
- "The terminal shows a cursor" is NOT analysis.
370
- "The terminal shows a blinking cursor at position 0,0 with no shell prompt, path, or git branch — this indicates the PTY session is not connected" IS analysis.
371
-
372
- If Gemini API is available, also run:
373
- ```bash
374
- apt-proof-video analyze "${PROOF_DIR}/screenshots/{feature-id}-test-{n}.png" \
375
- --model default \
376
- --spec "{expected result from this test}"
377
- ```
378
-
379
- ### 7e. Record Result to verification.json
380
- After each test, update the test entry in verification.json `tests` array.
381
- v0.8.6 adds `surface` and `transport` fields per test entry — the workflow
382
- runner populates them from the per-outcome dispatch directive:
351
+ Example test entry shape (electron surface, MCP transport):
383
352
 
384
353
  ```json
385
354
  {
386
355
  "id": "TEST-01",
387
- "registry_id": "terminals-grid-auto-tile",
388
- "name": "Three Terminal Grid Auto-Tile",
389
356
  "outcome_id": "O3",
390
357
  "surface": "electron",
391
358
  "transport": "mcp",
392
- "precondition": "2 terminals active",
393
359
  "action": "Press Cmd+T",
394
360
  "expected": "Grid reflows to 2+1 layout",
395
- "actual": "Grid correctly tiled",
396
- "evidence": "electron/screenshots/03-grid.png",
397
- "analysis": "Detailed adversarial analysis of what is actually visible in the screenshot...",
398
- "result": "pass",
399
- "severity": null
361
+ "evidence": "electron/screenshots/O3.png",
362
+ "result": "pass"
400
363
  }
401
364
  ```
402
365
 
403
- Compare `expected` vs `actual`:
404
- - If they match `result: "pass"`
405
- - If they partially match but have quality issues → `result: "fail"`, `severity: "minor"` or `"cosmetic"`
406
- - If they don't match `result: "fail"`, set `severity` (`"blocker"`, `"major"`, `"minor"`, `"cosmetic"`), add to `gaps` array
407
- - If you can't test → `result: "blocked"`, explain why in `analysis`
366
+ **`file-evidence: <commit-hash>` is NEVER sufficient.** The proof_gate
367
+ rejects that pattern for ALL surfaces — UI surfaces require screenshots,
368
+ CLI requires actual command output, API requires actual HTTP traces.
369
+ A commit hash is not proof a feature works.
370
+
371
+ **If a UI-surface outcome cannot capture screenshot evidence because
372
+ the host lacks the MCP driver transport (deferred tools didn't load,
373
+ puppeteer/electron-mcp-server unavailable):** write
374
+ `result: "needs_human_with_transport"` on that test entry, include
375
+ a `to_unblock` field explaining what the human needs to install /
376
+ load, and proceed. Do NOT fall back to file-evidence.
377
+
378
+ ### 4e. Analyze evidence (adversarial)
379
+
380
+ For each evidence artifact, write the `analysis` field with what you
381
+ **see**, not what you **expect to see**. Be adversarial:
382
+
383
+ - UI: Are there rendering artifacts? Stray characters? Broken layout?
384
+ Does the actual screenshot match the outcome's stated expectation?
385
+ - CLI: Did the command actually emit the expected output, or just exit 0?
386
+ - API: Did the response payload match the expected shape, or just 200?
387
+
388
+ "The terminal shows a cursor" is NOT analysis. "The terminal shows a
389
+ blinking cursor at position 0,0 with no shell prompt — the PTY session
390
+ is not connected" IS analysis.
391
+
392
+ ### 4f. Fail-loop on red, advance on green
393
+
394
+ After processing the parent + all inferred children for outcome `Oi`:
395
+
396
+ - If ALL test entries for `Oi` have `result ∈ {pass}` → advance to `Oi+1`.
397
+ - If ANY test entry has `result: "fail"` → fix the code, RE-RUN ONLY
398
+ THE FAILING ENTRIES for `Oi`, capture fresh evidence, write back to
399
+ `verification.json`. Loop up to `verification.max_iterations` times
400
+ (default 3 from config). After the iteration cap, leave `Oi` with
401
+ `result: "fail"` on the unresolved entries and continue — overall
402
+ verdict will be `rejected`.
403
+ - If ANY test entry has `result: "needs_human_with_transport"` →
404
+ advance to `Oi+1` (the human will install the transport and re-run).
405
+ Overall verdict will be `needs_human`.
406
+
407
+ ### 4g. Write verification.json after each outcome
408
+
409
+ Persist `verification.json :: tests[]` to disk after EVERY outcome
410
+ completes (success, fail, or needs_human_with_transport). This ensures
411
+ progress survives session interruption — `parsed.outcomes[i].id`
412
+ becomes the resume point.
413
+
414
+ ### 4h. Skip-reason taxonomy (legacy compatibility)
415
+
416
+ When an outcome cannot be driven because the selected driver lacks an
417
+ **OPTIONAL** IDL verb (e.g. `assert_network` not implemented), record
418
+ the test entry with `result: "skip"` and `reason: "unsatisfied_optional_capability"`.
419
+ This is the deliberate, loud skip path — log it in `skipped[]` so
420
+ coverage loss is auditable.
421
+
422
+ > **Legacy alias.** `blocked_by_platform` (pre-driver-architecture) is
423
+ > read by the schema reader as an alias for `unsatisfied_optional_capability`
424
+ > and emits a deprecation warning. New writes MUST use the new reason.
425
+
426
+ > **Fail-closed, not a skip:** if a REQUIRED IDL capability has no
427
+ > implementing driver in the resolver chain, the resolver throws
428
+ > `unsatisfied_required_capability` (per ID-02). This aborts the run —
429
+ > it does NOT silently land in `skipped[]`.
430
+
431
+ **Gate:** Do not proceed to Step 5 until every parent outcome has been
432
+ processed (success, fail, or needs_human_with_transport) AND
433
+ `verification.json :: tests[]` carries entries for every parent +
434
+ inferred-child combination.
435
+ </step>
408
436
 
409
- **Do NOT give a pass to something that has visual artifacts, broken rendering, or
410
- looks different from the expected production quality.**
437
+ <step name="start_video">
438
+ ## 5. Start Video Capture (optional)
411
439
 
412
- ### 7f. Write verification.json after each test
413
- Write the updated verification.json to disk after each test completes.
414
- This ensures progress is saved even if the session is interrupted.
440
+ If CDP capture is available, start recording BEFORE executing outcomes:
415
441
 
416
- **Gate:** Complete ALL tests before proceeding to step 8.
442
+ ```bash
443
+ apt-proof-video capture --port 9222 --duration {estimated_total_seconds} --fps 2 --output ${PROOF_DIR}
444
+ ```
445
+
446
+ Run this in the background. The video will capture your ACTUAL outcome
447
+ walker session — the same interactions that produce screenshots. This
448
+ ensures video and screenshots are from the SAME session.
449
+
450
+ **Important:** Estimate duration generously. Better to have extra video
451
+ than to miss outcomes.
452
+ - ~15 seconds per outcome (parent + inferred children) is a reasonable estimate
453
+ - Add 30 seconds for navigation overhead
454
+
455
+ **Note:** Captured frames are auto-stitched into an MP4 file by the tool.
456
+
457
+ If CDP is NOT available, note it and proceed without video.
458
+
459
+ (In practice this step is interleaved with Step 4 — kick off the
460
+ capture before the first outcome and stop it after the last.)
417
461
  </step>
418
462
 
419
463
  <step name="score_dimensions">
420
- ## 8. Score QA Dimensions
464
+ ## 6. Score QA Dimensions (informational rationale)
421
465
 
422
466
  Based on the evidence collected (not assumed), score each dimension.
423
- Write scores to verification.json `dimensions` object:
467
+ Write scores to verification.json `dimensions` object.
468
+
469
+ **v0.9.0 — dimensions are INFORMATIONAL rationale only.** The verdict
470
+ no longer derives from per-dimension minimums; it derives from the
471
+ two-gate envelope (Step 7).
424
472
 
425
473
  ```json
426
474
  {
427
475
  "dimensions": {
428
- "completeness": {
429
- "score": 7,
430
- "rationale": "18/22 spec requirements have passing tests. 4 blocked by platform."
431
- },
432
- "correctness": {
433
- "score": 8,
434
- "rationale": "All tested features behave as expected. No blockers or majors."
435
- },
436
- "code_quality": {
437
- "score": 9,
438
- "rationale": "Zero lint errors, zero type errors."
439
- },
440
- "test_coverage": {
441
- "score": 6,
442
- "rationale": "Unit tests cover core logic but no integration tests for new grid layout."
443
- }
476
+ "completeness": { "score": 7, "rationale": "All 12 outcomes processed; 2 needed transport, 10 driven." },
477
+ "correctness": { "score": 8, "rationale": "All driven outcomes matched expected; no regressions." },
478
+ "code_quality": { "score": 9, "rationale": "Zero lint errors, zero type errors." },
479
+ "test_coverage":{ "score": 6, "rationale": "Unit tests cover core logic; integration tests for the new outcome walker live in the framework package." }
444
480
  }
445
481
  }
446
482
  ```
447
483
 
448
- **Scoring rules:**
484
+ **Scoring rules** (all 1-10):
449
485
 
450
- **Completeness (1-10):**
451
- - Count how many spec requirements have passing tests
452
- - A requirement without a test scores 0
453
- - A requirement with a failing test scores 0
454
- - Sub-features not tested (e.g., individual settings) lower this score
486
+ - **Completeness**: count outcomes processed vs. authored (`tests` per
487
+ parent + inferred / total expected).
488
+ - **Correctness**: per-outcome verdict roll-up any blocker = max 3,
489
+ any major = max 5.
490
+ - **Code Quality**: lint + typecheck results.
491
+ - **Test Coverage**: unit + integration tests landed for the feature.
455
492
 
456
- **Correctness (1-10):**
457
- - Based on automated check results AND manual test results
458
- - Any blocker = max score 3
459
- - Any major = max score 5
460
- - Visual artifacts or rendering issues count as failures
493
+ Dimensions surface in the HTML report as rationale chips alongside the
494
+ two gate badges. They do NOT change the verdict.
495
+ </step>
496
+
497
+ <step name="derive_verdict">
498
+ ## 7. Derive Two-Gate Verdict
499
+
500
+ The verdict is the closed set `{approved, rejected, needs_human}`.
501
+ **`approved_with_notes` is DELETED — do not emit it.**
502
+
503
+ ### 7a. Compute `automation_gate`
504
+
505
+ ```
506
+ automation_gate = (
507
+ automated_checks.tests.status === 'pass' AND
508
+ automated_checks.typecheck.status === 'pass' AND
509
+ automated_checks.lint.status === 'pass'
510
+ ) ? 'pass' : 'fail'
511
+ ```
512
+
513
+ ### 7b. Compute `proof_gate`
514
+
515
+ For each test entry in `verification.json :: tests[]`:
461
516
 
462
- **Code Quality (1-10):**
463
- - Based on lint results and code review (if done)
517
+ - UI-surface (`electron`, `web`, `electron+web`) with
518
+ `evidence` matching `^file-evidence:` that test's effective result
519
+ is `needs_human_with_transport` regardless of the written `result`.
520
+ - Any test whose effective result is `fail` → contributes to
521
+ `proof_gate: fail`.
522
+ - Any test whose effective result is `needs_human_with_transport` →
523
+ contributes to overall verdict `needs_human` (does NOT fail `proof_gate`
524
+ on its own — the gate "passes" with the caveat that a human needs to
525
+ install the transport).
464
526
 
465
- **Test Coverage (1-10):**
466
- - Based on test suite results and whether new code has tests
527
+ ```
528
+ proof_gate = (
529
+ no test has effective result 'fail' AND
530
+ every outcome has at least one evidence entry per surface
531
+ ) ? 'pass' : 'fail'
532
+ ```
467
533
 
468
- **Derive verdict:**
469
- - ALL dimensions >= 7: `"approved"`
470
- - ALL dimensions >= 5: `"approved_with_notes"`
471
- - ANY dimension < 5: `"rejected"`
534
+ ### 7c. Derive overall verdict
472
535
 
473
- Write verdict to verification.json `verdict` field.
536
+ ```
537
+ if (automation_gate === 'fail') return 'rejected'
538
+ if (any test has effective result 'needs_human_with_transport') return 'needs_human'
539
+ if (proof_gate === 'fail') return 'rejected'
540
+ return 'approved'
541
+ ```
542
+
543
+ Write `verification.json :: meta.verdict` and `verification.json :: gates`:
544
+
545
+ ```json
546
+ {
547
+ "meta": { "verdict": "approved | rejected | needs_human" },
548
+ "gates": {
549
+ "automation_gate": "pass | fail",
550
+ "proof_gate": "pass | fail"
551
+ }
552
+ }
553
+ ```
474
554
  </step>
475
555
 
476
556
  <step name="multi_model" optional="true">
477
- ## 9. Multi-Model Verification (if enabled in config)
557
+ ## 8. Multi-Model Verification (if enabled in config)
478
558
 
479
559
  Only run if `.aperant/config.json` has `multi_model.enabled: true`.
480
560
 
@@ -485,7 +565,7 @@ Record each model's verdict in verification.json `model_verdicts` array.
485
565
  </step>
486
566
 
487
567
  <step name="generate_report">
488
- ## 10. Generate HTML Proof Report
568
+ ## 9. Generate HTML Proof Report
489
569
 
490
570
  Use the `apt-proof-video report` command to generate the HTML report from verification.json:
491
571
 
@@ -496,10 +576,12 @@ apt-proof-video report --proof-dir "${PROOF_DIR}" --title "{descriptive title}"
496
576
  The report command:
497
577
  1. Reads verification.json as the primary data source
498
578
  2. Recursively scans for screenshots in subdirectories
499
- 3. Generates feature checklists with pass/fail badges per test
500
- 4. Embeds screenshots inline per test
579
+ 3. Generates one card per `outcome_id` with stated prose + inferred
580
+ children + per-child evidence
581
+ 4. Renders two gate badges (`automation_gate`, `proof_gate`) plus
582
+ the overall verdict badge in the header
501
583
  5. Shows automated checks with expandable output
502
- 6. Shows QA dimension progress bars
584
+ 6. Shows QA dimension rationale chips
503
585
  7. Shows gaps with severity badges
504
586
  8. Embeds video player (frames are auto-stitched to MP4)
505
587
 
@@ -510,7 +592,7 @@ ln -sf "${PROOF_DIR}/report.html" .aperant/proof-report.html
510
592
  </step>
511
593
 
512
594
  <step name="open_report">
513
- ## 11. Open Report
595
+ ## 10. Open Report
514
596
 
515
597
  Read `verification.open_report` from the MERGED config (per-device override lives in `.aperant/config.local.json` per AUDIT-001; do not read raw `.aperant/config.json`). If true (default):
516
598
  ```bash
@@ -519,7 +601,7 @@ open "${PROOF_DIR}/report.html"
519
601
  </step>
520
602
 
521
603
  <step name="write_artifacts">
522
- ## 12. Write Artifacts
604
+ ## 11. Write Artifacts
523
605
 
524
606
  Write the following files:
525
607
  - `${PROOF_DIR}/verification.json` — the structured verification record (source of truth)
@@ -536,40 +618,35 @@ apt-tools commit "qa: proof verification {verdict}" --files ${PROOF_DIR}/verific
536
618
  </step>
537
619
 
538
620
  <step name="report_output">
539
- ## 13. Final Output
621
+ ## 12. Final Output
540
622
 
541
623
  ```
542
624
  Proof Verification: {VERDICT}
543
625
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
544
626
  Outcome: {one-line summary}
545
627
 
628
+ Gates:
629
+ automation_gate: {pass/fail}
630
+ proof_gate: {pass/fail}
631
+
546
632
  Checks:
547
633
  Tests: {pass/fail} ({count} tests)
548
634
  Types: {pass/fail}
549
635
  Lint: {pass/fail}
550
636
 
551
- Scores:
552
- Completeness: {n}/10
553
- Correctness: {n}/10
554
- Code Quality: {n}/10
555
- Test Coverage: {n}/10
556
-
557
- Coverage:
558
- Registry: {tested}/{total} features ({percent}%)
559
- Skipped: {n} features (see skipped array)
560
-
561
- Evidence:
562
- Features: {n} tested ({passed} pass, {failed} fail, {blocked} blocked)
563
- Screenshots: {n} captured, {n} analyzed
564
- Video: {captured/not available}
565
- Gaps: {n} issues ({blockers} blockers, {majors} major)
637
+ Outcomes:
638
+ Total: {n} parent outcomes (+ {m} inferred children)
639
+ Approved: {n}
640
+ Rejected: {n}
641
+ Needs Human: {n} (transport unavailable)
566
642
 
567
643
  Proof: {PROOF_DIR}/
568
- verification.json (structured test record)
644
+ verification.json (structured outcome record)
569
645
  report.html (opened in browser)
570
646
  report.md (for PR embedding)
571
647
 
572
648
  {If rejected:} Fix gaps and re-run /apt:verify-proof
649
+ {If needs_human:} Install the missing driver transports and re-run
573
650
  {If approved:} Next: /apt:review (optional code review)
574
651
  ```
575
652
  </step>
@@ -581,17 +658,14 @@ Proof: {PROOF_DIR}/
581
658
 
582
659
  These gates MUST be respected — do not skip them:
583
660
 
584
- 1. **Pre-check gate** (step 2): All subtasks must be completed before verification starts
585
- 2. **Feature discovery gate** (step 4): Source code MUST be read before writing tests
586
- 3. **Coverage gate** (step 4f): Coverage < 50% blocks test execution. Every untested feature must be in the skipped array.
587
- 4. **Test generation gate** (step 5): ALL tests must be written before execution starts
588
- 5. **Registry link gate** (step 5): Every test must have a `registry_id` linking to the feature registry
589
- 6. **Video gate** (step 6): Video capture starts BEFORE test execution, not after
590
- 7. **Test execution gate** (step 7): ALL tests must be executed before scoring
591
- 8. **Evidence gate** (step 7c-7d): Every test must have evidence AND critical analysis
592
- 9. **Quality gate** (step 7d): Visual artifacts, broken rendering, and quality issues MUST be reported as failures
593
- 10. **Scoring gate** (step 8): Scores must be derived from evidence, not assumed
594
- 11. **Output format gate**: Write verification.json, NEVER VERIFICATION.md
661
+ 1. **Pre-check gate** (Step 2): All subtasks must be completed before verification starts.
662
+ 2. **User Outcomes gate** (Step 2a): spec.md MUST carry `## User Outcomes` with `body_kind ∈ {outcomes, empty-with-note}`. No fallback to feature-registry-derived tests.
663
+ 3. **Deferred-tool-load gate** (Step 2.5): On Claude Code / OpenCode hosts, MCP driver tools listed under `drivers/<surface>/manifest.json :: transport.mcp_server_id` MUST be loaded via `ToolSearch(query='select:...')` BEFORE driving any UI-surface outcome.
664
+ 4. **Automation gate** (Step 3): `automated_checks.{tests,typecheck,lint}.status {pass, fail}` (closed set). No `"enforced_at_commit"`. Any single `fail` → `automation_gate: fail`.
665
+ 5. **Outcome-walker gate** (Step 4): every parent outcome AND every inferred child has its own test entry in `verification.json :: tests[]` with evidence appropriate to its surface OR `result: "needs_human_with_transport"`. File-evidence (`"file-evidence: ..."`) is NEVER sufficient for any surface.
666
+ 6. **Proof gate** (Step 7b): every outcome has surface-appropriate evidence. UI outcomes carrying `evidence: "file-evidence: ..."` are rejected as `needs_human_with_transport`.
667
+ 7. **Verdict gate** (Step 7c): the verdict is exactly one of `{approved, rejected, needs_human}`. The legacy `approved_with_notes` middle band is DELETED — emitting it is a hard error.
668
+ 8. **Output format gate** (Step 11): Write verification.json, NEVER VERIFICATION.md.
595
669
  </gates>
596
670
  </process>
597
671
  </output>