@aperant/framework 0.8.1 → 0.8.6

package/workflows/verify-proof.md

@@ -43,27 +43,81 @@ cat implementation_plan.json 2>/dev/null
 cat testable_features.json 2>/dev/null
 ```
 
-Generate run ID and create proof directory:
+Generate run ID and create proof directory. v0.8.6 — when the spec
+carries `## User Outcomes` with cross-surface tags (e.g.
+`[electron+web]`), proof artifacts MUST land under PER-SURFACE
+subdirectories so reviewers can tell at a glance which surface failed.
+Set up the path template here:
+
 ```bash
 RUN_ID="$(date +%H%M%S)"
 APP_NAME=$(node -e "try{console.log(JSON.parse(require('fs').readFileSync('package.json','utf8')).name)}catch{console.log('app')}" 2>/dev/null)
 PROOF_DIR=".aperant/proof/${APP_NAME}/$(date +%Y%m%d)/${RUN_ID}"
-mkdir -p "${PROOF_DIR}/screenshots"
+# Per-surface screenshot path template — Step 7c populates ${surface}
+# from the per-outcome resolver dispatch:
+#   ${PROOF_DIR}/${surface}/screenshots/${outcome-id}.png
+mkdir -p "${PROOF_DIR}"
 ```
+
+Sub-directories under `${PROOF_DIR}/${surface}/screenshots/` are
+created lazily in Step 7c when the workflow runner takes the first
+screenshot for that surface.
 </step>
 
 <step name="pre_check">
-## 2. Pre-Check
+## 2. Pre-Check + Driver Selection
 
 If `implementation_plan.json` exists, verify all subtasks are completed.
 If any are pending: report and exit.
 
-Check which MCP tools are available:
-- Try `mcp__electron__get_electron_window_info` — if responds, Electron is available
-- Try `mcp__puppeteer__puppeteer_navigate` — if responds, browser is available
-- Check `apt-proof-video --version` — if available, CDP video capture is enabled
+### 2a. Resolve the verify-proof driver
+
+The workflow is **runtime-agnostic** — it refers to IDL verbs
+(`driver.click(target)`, `driver.screenshot(path)`), never to concrete
+MCP tool names. The driver layer (`packages/framework/drivers/*/`)
+maps verbs to transports. Auto-detect the runtime and select a driver:
+
+```bash
+# Detect the project's runtime capabilities + select a driver
+node .aperant/deps/node_modules/@aperant/framework/bin/apt-tools.mjs detect-runtime .
+# → {status:"ok", runtime:{is_electron, is_tauri, is_nextjs, ...}, selected_driver_id:"..."}
+```
+
+The resolver picks the highest-scoring driver per ID-02 (capability_coverage
+> compatibility > stability > priority > locality). Adopters override
+auto-detect via `.aperant/config.json:verification.runtimes[]`:
+
+```json
+{ "verification": { "runtimes": ["electron", "cli"] } }
+```
+
+If no driver satisfies a REQUIRED capability, the resolver throws
+`unsatisfied_required_capability` — fail-closed, NOT a silent
+`blocked_by_platform` skip (per ID-02).
 
-Record available tools in verification.json `meta.tools` array.
+**v0.8.6 — per-outcome dispatch.** When the spec carries
+`## User Outcomes`, the workflow runner picks a driver PER OUTCOME
+SURFACE via `resolveDriversForOutcomes()` instead of a single
+per-run driver pick. Each outcome's `[surface]` tag maps to the
+runtime-capability flag a driver must carry; `[electron+web]`
+outcomes dispatch to TWO drivers (one per surface) sharing the same
+`outcome_id`. The legacy `apt-tools detect-runtime .` call above
+still works for the per-run "what driver would be picked?" preview
+— it answers the OLD question. For the new per-outcome question,
+the workflow runner imports `resolveDriversForOutcomes` directly
+from the resolver module in Step 7c.
+
+### 2b. Probe optional evidence-capture tools
+
+```bash
+# CDP video capture (optional)
+apt-proof-video --version  # if available, CDP video is on
+```
+
+Record the selected driver + auxiliary tools in verification.json:
+`meta.selected_driver: {driverId, version, manifest_sha256, trust_tier,
+sandbox_profile}` and `meta.fallback_attempts: [{driver_id, reason}]`
+(see ST11's schema extension in `templates/proof-verification.md`).
 </step>
 
 <step name="automated_checks">
@@ -112,6 +166,16 @@ If no feature registry exists:
 - Suggest: "No feature registry found. Consider running `/apt:scan {area}` first for exhaustive coverage."
 - Proceed with manual discovery below, but warn that coverage may be incomplete
 
+**v0.8.6 — read `## User Outcomes` from spec.md FIRST when present.**
+The PRD-shaped spec.md (STANDARD/DEEP tracks) ships a
+`## User Outcomes` section that is the verify-proof input contract:
+`**Epic:**` headline + N atomic outcomes tagged by `[surface]`. When
+present, each outcome maps 1:1 to a test entry — the workflow runner
+does NOT have to invent test units from the feature registry. Fall
+back to the feature-registry-based discovery below ONLY when the spec
+lacks `## User Outcomes` (the v0.8.x warn-not-block migration window
+per spec ID-06; the section becomes a hard block in v0.9.0).
+
 ### 4b. Read the source code
 
 You MUST read the actual source code for the features under test. Do not guess.
@@ -165,10 +229,14 @@ coverage = tested_count / registered_count
 **Every untested registered feature MUST go in the `skipped` array** with a reason.
 Valid skip reasons:
 - `time_constraint` — would take too long relative to value
-- `blocked_by_platform` — MCP tools cannot interact with this element
+- `unsatisfied_optional_capability` — the selected driver lacks an OPTIONAL IDL verb needed by this test (e.g. `assert_network` not implemented). Loud, deliberate skip.
 - `depends_on_other` — requires a feature not yet implemented
 - `not_applicable` — feature is deprecated or not relevant to this verification
 
+> **Legacy alias.** `blocked_by_platform` (pre-driver-architecture) is read by the schema reader as an alias for `unsatisfied_optional_capability` and emits a deprecation warning. New writes MUST use the new reason.
+
+> **Fail-closed, not a skip:** if a REQUIRED IDL capability has no implementing driver in the resolver chain, the resolver throws `unsatisfied_required_capability` (per ID-02). This aborts the run — it does NOT silently land in the `skipped` array. Coverage loss is loud, deliberate, and audited.
+
 Silence about untested features is NOT acceptable. Every registered sub-feature must appear
 either as a test (with `registry_id`) or in the `skipped` array.
 </step>
@@ -232,23 +300,61 @@ If CDP is NOT available, note it and proceed without video.
 For each test, in order:
 
 ### 7a. Perform the Action
-Actually do what the test says:
-- If it says "type X" → use `mcp__electron__send_command_to_electron` with `fill_input` or `send_keyboard_shortcut`
-- If it says "click X" → use the appropriate click tool
-- If it says "navigate to X" → navigate there
-- If it says "run command X" → run it
+Actually do what the test says — invoke the IDL verb on the selected driver:
+- If it says "type X" → `driver.type({ target, text: "X" })`
+- If it says "click X" → `driver.click({ target })`
+- If it says "navigate to X" → `driver.navigate({ url: "X" })`
+- If it says "press key X" → `driver.key({ key: "X" })`
+- If it says "run command X" → `driver.execute_js({ script: "X" })`
+
+The driver layer translates IDL verbs to its transport (an MCP server,
+a local binary, or a built-in HTTP request). Workflow prose names ONLY
+IDL verbs per ID-01 — concrete tool names live in
+`packages/framework/drivers/<driver>/driver.mjs`.
+
+**v0.8.6 — transport dispatch directive (MUST honor).** When
+`driver.transport.skill_id` is set AND the host CLI exposes the
+Skill tool AND the outcome surface is `web`, the workflow runner
+MUST invoke `Skill({skill: driver.transport.skill_id, args: ...})`
+(passing the IDL verb args) and MUST record `transport: "skill"` on
+the resulting test entry in verification.json. When the host CLI
+does NOT expose the Skill tool, fall back to the MCP transport via
+the existing puppeteer MCP server and record `transport: "mcp"`.
+The driver's `chooseTransport({has_skill_tool})` helper computes
+the choice; the workflow runner is the site that knows
+`has_skill_tool`. This directive is LLM-honored (the v1 verify-proof
+runner is the host LLM following this workflow as prose); a
+text-level grep guard (TD-08) pins the `MUST` language so future
+edits cannot silently soften it to `SHOULD`.
 
 ### 7b. Wait for Result
 Wait for the action to complete. If it involves loading, wait for the load.
 Add a short delay (1-2 seconds) so the CDP capture also captures this state.
 
 ### 7c. Capture Evidence
-Take a screenshot AFTER the action completes:
+Take a screenshot AFTER the action completes. v0.8.6 — screenshots
+land under PER-SURFACE subdirectories, with the per-outcome driver
+returned by `resolveDriversForOutcomes()`:
+
 ```
-Tool: mcp__electron__take_screenshot
-Args: {"outputPath": "${PROOF_DIR}/screenshots/{feature-id}-test-{n}.png"}
+// Pick a driver per outcome surface (NOT per run). `entries[]` is
+// the resolver's per-(outcome, surface) dispatch — for an
+// [electron+web] outcome you get TWO entries sharing outcome_id
+// and you loop both, capturing one screenshot per surface.
+for (const entry of entries) {
+  await entry.driver.screenshot({
+    target: { kind: "css", value: "body" },
+    output_path: `${PROOF_DIR}/${entry.surface}/screenshots/${outcome.id}.png`
+  })
+}
 ```
 
+Single-surface outcomes (`[electron]`, `[web]`, `[cli]`, `[api]`)
+produce one screenshot under `${PROOF_DIR}/${surface}/screenshots/`;
+cross-surface (`[electron+web]`) produces two — one per surface tree
+— with both test entries in `verification.json:tests[]` carrying the
+same `id` but distinct `surface` and `evidence` fields.
+
 ### 7d. Analyze Evidence
 **MANDATORY and CRITICAL.** Every screenshot must be analyzed in detail.
 
@@ -271,18 +377,23 @@ apt-proof-video analyze "${PROOF_DIR}/screenshots/{feature-id}-test-{n}.png" \
 ```
 
 ### 7e. Record Result to verification.json
-After each test, update the test entry in verification.json `tests` array:
+After each test, update the test entry in verification.json `tests` array.
+v0.8.6 adds `surface` and `transport` fields per test entry — the workflow
+runner populates them from the per-outcome dispatch directive:
 
 ```json
 {
   "id": "TEST-01",
   "registry_id": "terminals-grid-auto-tile",
   "name": "Three Terminal Grid Auto-Tile",
+  "outcome_id": "O3",
+  "surface": "electron",
+  "transport": "mcp",
   "precondition": "2 terminals active",
   "action": "Press Cmd+T",
   "expected": "Grid reflows to 2+1 layout",
   "actual": "Grid correctly tiled",
-  "evidence": "screenshots/03-grid.png",
+  "evidence": "electron/screenshots/03-grid.png",
   "analysis": "Detailed adversarial analysis of what is actually visible in the screenshot...",
   "result": "pass",
   "severity": null