session-orchestrator 4.2.0 → 5.0.0

package/skills/wave-executor/references/wave-loop-review.md

@@ -119,6 +119,19 @@ Log every non-`pass` result as an event to `.orchestrator/metrics/events.jsonl`
 
    Cross-reference `.claude/rules/verification-before-completion.md` § VBC-004 Exception 2: a subagent's `STATUS: done` is a claim that needs its own verification — this step is that verification for the file-write side effect. `$WAVE_PREDISPATCH_HEAD` is the HEAD snapshot captured before this wave dispatched (same snapshot used by `### 3c. File-level grounding`). When `persistence: false` (no STATE.md), still perform the check and surface any violation in the wave progress update; only the deviation-write is skipped.
 
+3d-bis. **Scope-echo check (#1092)** (per agent, INFORMATIONAL — never blocks): the send-side `orchestrator.wave_dispatch.scope_checked` record says what the guard saw in the prompt the coordinator handed to the dispatch tool; nothing observes the receive side. This step collects the one signal that is available — the agent's own echo of the digest injected in `wave-loop-dispatch.md` § Pre-Dispatch: File-Scope Injection.
+
+   For each agent that reported, write its final report text to a file and run:
+
+   ```bash
+   node scripts/lib/scope-echo.mjs --scope-file "<state-dir>/filescopes/wave-<N>/<agent-id>.json" \
+     --report-file "<report.txt>" --wave <N> --agent-id "<agent-id>" --emit
+   ```
+
+   `checkScopeEcho` prints the verdict as JSON and `--emit` appends one `orchestrator.wave_dispatch.scope_echo_checked` record (payload: `docs/events-schema.md`; counts, digests and closed enums only — no path, no prompt text). `match: false` or `echoed: false` is logged in the wave progress update as `scope-echo: <agent> missing|mismatch` and changes NOTHING else — no re-dispatch, no deviation write, no gate.
+
+   **What it proves:** the agent's final report carried the digest the coordinator handed it, i.e. the injected line survived the round trip into the agent's context and back. **What it does not prove:** that the model read the `FILE-SCOPE` block, understood it, or obeyed it — the digest is in the prompt, so it can be copied without ever looking at the paths (BV-004 ceiling, `docs/scope-collision-guard.md` § 4.2). Obedience is still measured at write time (`enforce-scope.mjs`) and by the W5 verification pass.
+
 3e. **Collect Open Questions** (Close Handover-Alignment-Gate, PRD 2026-07-07): scan every completed agent's report from this wave for an optional `OPEN-QUESTIONS:` line (see the report-line convention in `wave-loop-dispatch.md` `#### Agent-Type Resolution` — an agent MAY emit `OPEN-QUESTIONS: <question> | context: <...> | candidates: <opt A / opt B>`; most agents emit none). For each such line found:
 
    - Parse the question text (portion before the first ` | `).
@@ -317,9 +330,9 @@ If the commit itself fails (e.g., nothing to commit, pre-commit hook rejects), d
    - **Discovery** and **Finalization** waves: skip session-reviewer dispatch — Discovery is read-only and Finalization is a final git status check only.
    - This is complementary to the incremental verification in step 4 — the session-reviewer provides deeper analysis (security, silent failures, test depth, type design) that automated checks do not cover.
 6. **Pencil design review** (after Impl-Core and Impl-Polish roles only, if `pencil` configured in Session Config):
-   a. Check Pencil editor state: `get_editor_state({ include_schema: false })`. If no editor active, open the configured `.pen` file via `open_document({ filePathOrTemplate: "<pencil-path>" })`. If that also fails → skip with note "Pencil review skipped — .pen file unavailable."
-   b. Get design structure: `batch_get({ filePath: "<pencil-path>", patterns: [{ type: "frame" }], readDepth: 2, searchDepth: 2 })` — find frames relevant to this wave's UI work.
-   c. Screenshot relevant frames: `get_screenshot({ filePath: "<pencil-path>", nodeId: "<frame-id>" })` for each frame matching the wave's UI tasks.
+   a. Check Pencil app state: `mcp__pencil__get_app_state`. If the configured `.pen` file is not open, open it through the `mcp__pencil__execute` surface (an Open/activate call on `filePath` — `.pen` files are encrypted, so never `Read`/`Grep` them; follow the tool's own input schema, which the Pencil MCP server instructions describe as "be aware of the .pen schema and the instructions for the execute tool"). If that also fails → skip with note "Pencil review skipped — .pen file unavailable."
+   b. Get design structure: `mcp__pencil__execute` with a **Get visitor** scoped to `filePath: "<pencil-path>"` at depth 1 (top-level frames) or depth 2 (frame children) — find frames relevant to this wave's UI work. Read the execute tool's input schema for the exact visitor shape before calling; do not guess it.
+   c. Render relevant frames: there is no direct screenshot tool on the current surface — use `mcp__pencil__browser` to view/capture each frame matching the wave's UI tasks (`filePath` plus the frame's node id).
    d. Read the actual UI files changed in this wave (from agent outputs).
    e. **Compare**: layout structure, component hierarchy, visual elements (headings, buttons, inputs, cards), responsive behavior.
    f. **Report** in wave progress:
@@ -334,7 +347,7 @@ If the commit itself fails (e.g., nothing to commit, pre-commit hook rejects), d
         3. If "Revise" → re-run session-plan for remaining waves only
         4. If "Abort" → mark remaining waves as DEFERRED, proceed to session-end
    
-   Always use the `filePath` parameter on Pencil MCP calls. Only review frames relevant to the current wave, not the entire file.
+   Always use the `filePath` parameter on Pencil MCP calls. Only review frames relevant to the current wave, not the entire file. The live tool names (`mcp__pencil__get_app_state`, `mcp__pencil__execute`, `mcp__pencil__browser`) have one definition in code — `PENCIL_TOOL_NAMES` in `scripts/lib/ux-grill/pencil-coverage.mjs`; the pre-2026 tool surface it replaced no longer exists (dead names enumerated in that module's JSDoc).
 
 7. **Capture wave metrics**: If `persistence` is enabled in Session Config, record for this wave after all agents complete and quality checks run. If `persistence` is `false`, skip metrics capture entirely — do not accumulate in-memory metrics. Record:
    - `wave_number`, `role`, `started_at` (when agents were dispatched), `completed_at` (when all finished)
@@ -462,7 +475,7 @@ After each wave completes and before the progress update, update `<state-dir>/ST
 
 **Gate:** `persistence: true` in Session Config. When `persistence: false`, skip every push below — there is no runtime side-channel to feed.
 
-The helper is `scripts/lib/agent-status.mjs`. Its exports (`setStatus`, `setProgress`, `readCurrentStatus`) are all no-throw and return `{ ok: true } | { ok: false, reason }`; the coordinator ignores the return value (best-effort). Push at **three anchors** in the wave loop:
+The helper is `scripts/lib/agent-status.mjs`. The writers (`setStatus`, `setProgress`) are no-throw and return `{ ok: true } | { ok: false, reason }`; the coordinator ignores the return value (best-effort). The reader `readCurrentStatus()` returns `{ entries, source: 'live-map'|'rebuilt-log'|'stale-cache', at, degraded? }` — the ledger `agent-status.jsonl` is the source of truth and the current-map is a rebuildable cache (#1342), so a consumer must SHOW `source`/`at`; `readCurrentStatusEntries()` is the bare-map accessor. Push at **three anchors** in the wave loop:
 
 1. **dispatch** — in `### 1. Dispatch Agents`, as each agent is dispatched, push its status. Use `setProgress` when the wave's per-agent ordinal is meaningful, else `setStatus`:
 

package/templates/_shared/ux-manifest.template.md

@@ -0,0 +1,149 @@
+<!-- source: session-orchestrator plugin (canonical: templates/_shared/ux-manifest.template.md) -->
+---
+# ux-grill manifest — copy to `.orchestrator/ux-manifest.md` in the target repo.
+#
+# Every value below is a PLACEHOLDER. Replace them; never paste a credential,
+# a token or a production host into this file — it is tracked by git.
+
+# REQUIRED. Must be loopback (127.0.0.1 / localhost / [::1]). A non-loopback
+# base-url aborts the mechanical run with "base-url must be loopback".
+base-url: http://127.0.0.1:3100
+
+# REQUIRED. `dev` or `prod`. A dev build is not a geometry measurement basis,
+# so target-size findings from it are marked `provisional: true`.
+build: dev
+
+# Filename (relative to the target repo root) of the GITIGNORED env file that
+# holds the VALUES for every env NAME named below. Omit it if no run needs
+# credentials or guarded endpoints.
+env-file: .env.e2e.local
+
+# Env NAMES whose values must point at loopback. Use this for every endpoint
+# the app talks to (API base, database URL, storage) so a stray production
+# value cannot be written to by a seed or a journey. The value is checked but
+# never printed.
+guarded-url-envs:
+  - APP_API_BASE_URL
+  - APP_DATABASE_URL
+
+# Optional command that seeds deterministic test data before the run.
+seed-command: npm run seed:e2e
+
+# Evaluation personas. Credentials are env NAMES only — the values live in
+# `env-file`. Use `personas: []` for an app that needs no login.
+personas:
+  - name: solo-operator
+    login-env-email: LOGIN_EMAIL_SOLO
+    login-env-password: LOGIN_PASSWORD_SOLO
+    goal: Get from the dashboard to a finished document without help.
+  - name: accountant
+    login-env-email: LOGIN_EMAIL_ACCOUNTANT
+    login-env-password: LOGIN_PASSWORD_ACCOUNTANT
+    goal: Review a client's numbers and export them.
+
+# Routes to measure. `title-pattern` is a regular expression matched against
+# the page title. `frame` is optional and only used by the Pencil coverage
+# step (`desktop` | `mobile` | `both` | `none`).
+routes:
+  - path: /dashboard
+    title-pattern: ^Dashboard
+    persona: solo-operator
+    frame: both
+  - path: /documents/new
+    title-pattern: ^New document
+    persona: solo-operator
+
+# Journeys are replayed verbatim: every entry of `steps` is one agent-browser
+# command line, so the mechanical stage can count them without judging them.
+#
+# TRUST: each step line is split into an argv array and handed to the
+# `agent-browser` binary (never to a shell) — but NOT verbatim: the first token
+# must be one of the ALLOWLISTED UI verbs below, anything else aborts the run
+# with `step-verb-not-allowed`.
+#
+#   back  check  click  dblclick  drag  fill  find  focus  forward  get
+#   hover  is  keyboard  open  press  reload  scroll  scrollintoview
+#   select  snapshot  type  uncheck  wait
+#
+# The denied half of the CLI is what the allowlist exists for: `upload` and
+# `cookies set --curl` read arbitrary HOST FILES into the page, `download` and
+# `pdf` write arbitrary host paths, `eval` can exfiltrate off-origin, `connect`
+# retargets a foreign browser, and `close --all` kills every other agent's
+# session on the machine. So the trust model is "allowlisted UI verbs", not
+# "anything the CLI accepts" — whoever can commit this file can drive the
+# BROWSER, not the host.
+#
+# An `open` step must resolve to the same origin as `base-url`, else
+# `step-open-off-origin`. A step may not carry `--session`: the run owns its
+# session, and a step that retargets it aborts with `step-session-override`.
+#
+# `start` must resolve to the SAME ORIGIN as `base-url` (a path is the normal
+# form). An off-origin `start` aborts with `journey-start-off-origin` — with a
+# persona attached, an absolute foreign URL would type real credentials into a
+# foreign page. The same rule holds for every `routes[].path`.
+journeys:
+  - name: create-first-document
+    persona: solo-operator
+    start: /dashboard
+    steps:
+      - click "New document"
+      - type "#title" "Placeholder title"
+      - click "Save"
+    success: /documents/
+    max-steps: 6
+
+# Viewports. Defaults to exactly these two when the key is absent.
+#
+# Every viewport is VERIFIED after it is applied: `window.innerWidth` must equal
+# the expectation, else the viewport is skipped as `device-mismatch` rather than
+# captured under a wrong label. The expectation is the `viewport:` width, the
+# built-in width of a known `device:` name (iPhone 15/16 393, iPhone 16 Pro 402,
+# iPhone 17 402, iPad 820, iPad Pro 1024, Pixel 9 412, Galaxy S25 360 — measured
+# against agent-browser 0.37.1), or an explicit `expected-width:`. A `device:`
+# outside that list NEEDS `expected-width:`, otherwise the viewport is skipped:
+# an unknown device name leaves the previous device in place, so an unverified
+# width is exactly how desktop captures end up labelled `mobile`.
+viewports:
+  - name: desktop
+    viewport: 1440x900
+  - name: mobile
+    device: iPhone 15
+
+# Optional design-coverage source.
+pencil:
+  file: design/app.pen
+---
+
+# UX Manifest — <REPO NAME>
+
+> Per-repo UX truth: what a real user walks through, and what the mechanical
+> stage is allowed to touch. Last verified: <YYYY-MM-DD> by <operator>
+
+## Notes
+
+Free-form notes below the frontmatter. They are preserved verbatim and are not
+parsed — use them for context the fields cannot carry.
+
+**Credentials never go in this file.** The frontmatter names env VARIABLES
+(`login-env-email`, `login-env-password`, `guarded-url-envs`); their values are
+read at run time from the gitignored file named by `env-file`. A value pasted
+here is a secret in a tracked file, and the run itself will never print one:
+guarded-env and persona errors name the variable, never its content. `env-file`
+must resolve INSIDE the repo (`env-file-outside-repo` otherwise) — it names a
+file of this repo, never a host-wide secrets file.
+
+**Run artefacts can carry a credential even though findings cannot.** Three
+classes: journey step screenshots (one is taken after EVERY step, including the
+one right after `fill #pw ${LOGIN_PASSWORD}`), the text of `errors --json`, and
+the `html` snippets inside axe JSON. They all live under
+`.orchestrator/metrics/ux-grill/`, which `/ux-grill` adds to this repo's
+`.gitignore` on bootstrap — keep that line, and never commit a run directory.
+
+## Known exceptions
+
+Findings that are known and deliberately accepted — with the reason and a
+revisit trigger, so they are not re-filed every run.
+
+| Finding | Why accepted | Revisit trigger |
+|---------|--------------|-----------------|
+| <…> | <…> | <…> |

package/templates/_shared/journey-manifest.md

@@ -1,114 +0,0 @@
-<!-- source: session-orchestrator plugin (canonical: templates/_shared/journey-manifest.md) -->
-<!--
-  Retired with /journey-audit in 4.0.0 — kept as a template for a per-repo
-  product-truth manifest; no command consumes it.
-
-  Copy to `.orchestrator/journey-manifest.md` in the target repo and fill in every
-  section. While `/journey-audit` existed, it refused to run without this file;
-  with the command removed, filling this in is optional documentation only.
-
-  The `## SAFETY` block described what gated R5 (the real end-to-end run against
-  production with a real account) when the skill still ran. Kept below for
-  reference in case a future consumer re-adopts the same manifest shape.
-
-  Credentials: env-var NAMES only, never values. A value pasted here is a secret
-  in a tracked file.
--->
-
-# Journey Manifest — <REPO NAME>
-
-> Per-repo product-truth manifest, not code structure.
-> Retired with `/journey-audit` in 4.0.0 — no command consumes this file today.
-> Last verified: <YYYY-MM-DD> by <operator>
-
-## Personas & Einstiegspunkte
-
-| Persona | Ziel | Einstiegspunkt (URL/Route) | Erfolg heißt |
-|---------|------|---------------------------|--------------|
-| <anonymer Besucher> | <was er will> | `<https://…/>` | <messbarer Endzustand> |
-| <zahlender Kunde> | | | |
-| <Owner/Admin> | | | |
-
-List every entry point a real user can reach — including the ones marketing links
-to but the app never surfaces. A feature with no entry point is exactly the defect
-class this audit exists to find.
-
-## Wahrheits-SSOTs
-
-Where the truth about the product lives in code. R3 compares every claim against
-THESE files, not against other prose.
-
-| Gegenstand | SSOT-Datei/Konstante |
-|-----------|----------------------|
-| Preise / Pläne / Limits | `<src/config/plans.ts>` |
-| Feature-Flags | `<…>` |
-| Währung / Steuer / Region | `<…>` |
-| Quota / Kontingente | `<…>` |
-| i18n-Namespaces (Marketing, FAQ, Mail) | `<…>` |
-| Chat-/Bot-Faktenquelle | `<…>` |
-
-## Outbound-Touchpoints (Input für R1/R2)
-
-- Template-Verzeichnis: `<…>`
-- Send-Pfad / Provider-Adapter: `<…>`
-- Cron-/Scheduler-Quellen: `<…>`
-- Render-Idiom + Beispiel-Props für R2: `<z. B. react-email, props aus …>`
-
-## Chat-Interview (Frage → Soll-Antwort)
-
-R4 asks these verbatim and scores the answer against the expected one. The expected
-answer must be derivable from a Wahrheits-SSOT above — otherwise it is a second
-claim, not a truth key.
-
-| # | Frage (wörtlich) | Soll-Antwort (Kern-Fakt) | SSOT |
-|---|------------------|--------------------------|------|
-| 1 | <"Was kostet …?"> | <"…"> | `<datei>` |
-| 2 | | | |
-| 3 | | | |
-
-## SAFETY
-
-**MANDATORY for R5. Without this block filled in, R5 is not dispatched.**
-
-- **Erlaubte Konten:** `<test-account@…>` — and no other. Never a real customer account.
-- **Erlaubte Events/Objekte:** `<z. B. nur Events mit Präfix AUDIT-…>`
-- **No-Go-Aktionen:** <Löschen fremder Daten · Mailversand an echte Empfänger · Plan-Downgrade · Refunds · Webhook-Replays gegen Prod>
-- **Checkout-Grenze:** `<max. EUR X, Testkarte …, Live-Zahlungen verboten>`
-- **Cleanup-Regel:** <was nach dem Durchstich wieder entfernt wird, von wem, woran man erkennt dass es weg ist>
-- **Abbruchbedingung:** <woran R5 sofort stoppt und meldet statt weiterzumachen>
-
-## Credential-Quellen (env-Namen, NIE Werte)
-
-| Zweck | Env-Variable | Bezugsquelle |
-|-------|--------------|--------------|
-| Prod-Login R5 | `<AUDIT_TEST_ACCOUNT_EMAIL>` | `<1Password-Item / .env.local>` |
-| DB read-only R6 | `<AUDIT_DB_READONLY_URL>` | `<…>` |
-| Mail-Provider-API R6 | `<…>` | `<…>` |
-| Plattform-CLI R7 | `<…>` | `<…>` |
-
-## Realdaten-Queries (R6, read-only)
-
-Only `SELECT`. A query that writes does not belong in this file.
-
-```sql
--- Funnel: Registrierung → Aktivierung → Zahlung
-SELECT …;
-```
-
-- Identitäts-Regeln: <wie ein Nutzer über Tabellen hinweg identifiziert wird>
-- Nicht messbar (bewusst benennen): <was die Daten NICHT hergeben>
-
-## Plattform-Erwartung (R7)
-
-- CLI-Logins: `<vercel · supabase · cloudflare · …>`
-- Erwarteter Plan/Tier je Dienst: `<…>`
-- Offene Perf-/Kosten-Issues: `<#…>`
-
-## Bekannte Ausnahmen
-
-Findings that are known and deliberately accepted — with the reason and a revisit
-trigger. R1–R7 report them as `known-exception`, never as new findings.
-
-| Befund | Warum akzeptiert | Revisit-Trigger |
-|--------|------------------|-----------------|
-| <…> | <…> | <…> |