yam-flow 2.5.0 → 2.6.0

package/CHANGELOG.md

@@ -2,6 +2,57 @@
 
 All notable changes to yam-flow are documented here.
 
+## v2.6.0 - 2026-08-07
+
+### Added
+
+- Added effective executable identity validation for global yam install and rollback, including npm global root, package/bin mapping, PATH target, canonical realpaths, and observed version.
+- Added bounded paired-sample promotion receipts with explicit thresholds, deterministic evidence digests, operator-asserted measurement truth, and strict keep/revert gates.
+- Added a capability matrix to `yam tools doctor --json` that separates maturity from observed runtime readiness.
+- Added digest-bound strict gate results at release, external-update, Mission-completion, and benchmark-promotion boundaries.
+- Added the opt-in Demand-Gated Design Production Phase: pre-recorded operator demand, local static Plan Review Canvas Lite, at most two chronologically finding-backed revisions, final gallery manifest, and digest-verifiable immutable phase receipt.
+
+### Improved
+
+- External update receipts preserve the deepest actionable Doctor or rollback failure as the primary blocker while keeping receipt persistence failures as secondary evidence.
+- Ueye design-production artifacts bind one canonical revision state to its closed Canvas, enforce Canvas-comment and archive chronology at record time, and revalidate source, exact anchors, protected-asset editability, final-round gallery linkage, upstream digests, nested receipt semantics, session state, and aggregate state without upgrading those checks into visual, license, or implementation proof.
+
+### Security
+
+- Same-version PATH shadows, malformed global package/bin identities, path/symlink escapes, protected-asset replacement attempts, and inconsistent gate contracts fail closed.
+- Canvas rendering is local static HTML with a no-script/no-network CSP and escaped reviewer content; it starts no server or background process.
+
+### Verification
+
+- Focused external-update identity, strict-gate/promotion/capability/Mission, and design-production smokes.
+- TypeScript typecheck, clean build, self-verification, packaged CLI, package-boundary/secret, and bounded Deep checks.
+
+## v2.5.1 - 2026-07-31
+
+### Added
+
+- Added bounded stdin readers with a 1 MiB hook limit and a 4 MiB general CLI limit. Malformed or oversized hook payloads now return a valid fail-open response without echoing or acting on rejected input.
+- Added a final read-only `yam doctor --json` verification to yam updates and automatic rollbacks. The updater accepts success only when the command exits cleanly and returns the expected healthy Doctor contract.
+- Added a release-time secret guard over the actual npm dry-run packlist. It scans regular UTF-8 text files for high-signal private-key and provider credential patterns while reporting only pattern, path, and line metadata.
+
+### Improved
+
+- Install and Ueye integrity manifests now use locale-independent ordinal ordering. Existing yam 2.5.0-and-earlier receipts retain compatibility with their stored legacy order while new digests are deterministic across locales.
+
+### Security
+
+- Rejected stdin payloads are byte-bounded, are not included in diagnostics, and cannot supply hook context or trigger workspace mutation.
+- `verify:self`, `prepack`, `prepublishOnly`, and `release:check` now inherit the npm packlist secret guard; the inner pack inspection disables lifecycle scripts to avoid recursive execution.
+- Updater Doctor output recorded in receipts remains redacted and bounded, and an unhealthy rollback is not reported as an automatic recovery.
+
+### Verification
+
+- TypeScript typecheck and build
+- Bounded hook latency/read-only smoke and packaged CLI smoke
+- Locale-independent install/Ueye manifest regression smokes
+- External update/rollback Doctor smoke
+- Npm packlist secret-pattern and redaction smoke
+
 ## v2.5.0 - 2026-07-30
 
 ### Added

package/COMMANDS.md

@@ -154,7 +154,33 @@ yam update apply --component yam --json
 yam update apply --all --json
 ```
 
-`yam update check` is read-only. Every `apply` needs explicit authorization, uses a concurrent-run lock, records a component receipt, and stops on failed/manual results. `--all` runs yam last. Insane Search is never remove-first and `.codex/plugins/cache` is never edited directly.
+`yam update check` is read-only. Every `apply` needs explicit authorization, uses a concurrent-run lock, records a component receipt, and stops on failed/manual results. `--all` runs yam last. A yam install verifies the effective executable identity before any yam-side install command and repeats that check during rollback. This identity contract currently supports POSIX symlink layouts; Windows npm `.cmd` shims fail closed as unverified. Insane Search is never remove-first and `.codex/plugins/cache` is never edited directly.
+
+## Demand-Gated Design Production
+
+Use only after repeated plan-review or large multi-asset coordination demand is real. A one-off review remains on the ordinary Ueye/Mission path.
+
+```bash
+yam ueye production canvas create --session-id campaign --title "Campaign review" --demand-kind multi_asset_production --demand-evidence "Five coordinated assets require one review package" --artifact-spec ./canvas-artifact.json --json
+yam ueye production canvas comment --session-path .yam/ueye/design-production/campaign/plan-review.json --render-path .yam/ueye/design-production/campaign/canvas.html --comment-id finding-1 --artifact-id campaign-plan --anchor-kind line --locator L12 --finding "CTA hierarchy is ambiguous" --requested-change "Make the primary CTA explicit" --json
+yam ueye production canvas close --session-path .yam/ueye/design-production/campaign/plan-review.json --render-path .yam/ueye/design-production/campaign/canvas.html --verdict request_changes --json
+yam ueye production revision init --session-id campaign --json
+yam ueye production revision record --state-path .yam/ueye/design-production/campaign/revision-state.json --finding-id round-1 --source-comment-id finding-1 --finding-summary "CTA hierarchy is ambiguous" --finding-evidence "Canvas line L12" --planned-change "Clarify the primary CTA" --outcome accepted --revision-ref ./ueye-revision-ref.json --json
+yam ueye production gallery write --manifest-path .yam/ueye/design-production/campaign/gallery.json --session-id campaign --completion-state packaged --artifact-spec ./gallery-artifact.json --json
+yam ueye production gallery verify --manifest-path .yam/ueye/design-production/campaign/gallery.json --json
+yam ueye production finalize --demand-kind multi_asset_production --demand-evidence "Five coordinated assets require one review package" --canvas-session-path .yam/ueye/design-production/campaign/plan-review.json --revision-state-path .yam/ueye/design-production/campaign/revision-state.json --gallery-manifest-path .yam/ueye/design-production/campaign/gallery.json --receipt-path .yam/ueye/design-production/campaign/phase-receipt.json --json
+yam ueye production phase verify --receipt-path .yam/ueye/design-production/campaign/phase-receipt.json --json
+```
+
+Canvas artifact specs use `{ "id", "role", "file_path" }`; gallery specs include `{ "id", "role", "provenance", "revision_ref", "dimensions", "sha256", "completion_state", "final_path" }`. A revision reference also carries the archive receipt's exact `archived_at`, which must match its Ueye manifest entry. All project paths are relative and confined; Canvas session/render paths and the single revision state are fixed under the session-specific `.yam/ueye/design-production/` directory. Revision init binds that state to the closed `request_changes` Canvas path and digest; record/show/finalize reject alternate state files, and record immediately checks the source Canvas comment plus the current archive-time boundary. Demand is operator-asserted but must be recorded before Canvas creation and match at finalize. Canvas source hashes and exact anchors are rechecked before comment/close/finalize. Every phase round names its source Canvas comment and supplies at least one newly archived Ueye revision created within that round's review window; gallery artifacts bind to the final round. Ueye asset/revision hashes are rechecked, a protected `do_not_replace` asset requires `intent: "preserve"`, and `edit_copy` requires `allowed_for_edit: true`. The phase receipt stores upstream and canonical receipt digests, and `phase verify` re-reads their semantics as well as detecting later byte changes. Overall phase truth remains `partial`, and gallery visual, license, and implementation claims always remain `not_verified`.
+
+## Bounded Promotion
+
+```bash
+yam benchmark report --label "scout strategy" --candidate-digest <sha256> --baseline-digest <sha256> --sample seed-1:0.82:0.71 --sample seed-2:0.79:0.70 --unit score --evidence-source "fixed local fixture" --target higher --min-samples 2 --min-mean-delta 0.05 --min-win-rate 0.5 --json
+```
+
+This command records paired arithmetic; it does not run the benchmark. A `keep` decision still has `measurement_truth: operator_asserted` and overall `truth_status: partial`.
 
 ## Tool Doctor / Proof / Safety
 

package/README.md

@@ -99,6 +99,8 @@ yam hook status --global
 
 The profile installs `UserPromptSubmit` and `Stop` handlers. At `Stop`, yam checks the latest assistant message with the same read-only Study Note Guard used by `yam study-note check`. If changed artifacts lack the required role, execution point, before/after, expected behavior, syntax/structure, verification, limits, or relevant architecture hygiene, Codex receives one correction prompt. A second failed check warns but does not loop forever. If Git change scope is unavailable, the guard reports partial truth and keeps a manual-inspection warning visible instead of claiming the project is clean. The hook never writes the Study Note or runs verification for the agent.
 
+Hook stdin is limited to 1 MiB. Malformed JSON or an oversized payload produces a valid fail-open response without echoing the rejected content, deriving hook context from it, or mutating the workspace. Other CLI commands that read piped text use a 4 MiB limit and fail closed when it is exceeded.
+
 `yam hook status` reports stale or missing command targets as `broken` and exits nonzero. Run `yam hook enable <profile> --global` again to create a timestamped backup, preserve unrelated hooks, and migrate that profile to the current installed path and event coverage. Restart Codex or open a new task after changing hooks.
 
 ## Install
@@ -120,7 +122,7 @@ yam install --replace-user-skill quick
 
 Repeat `--replace-user-skill <name>` for each reviewed conflict. Unproven legacy, retired, and Codex mirror directories are preserved rather than deleted by name.
 
-A successful install writes `~/.agents/skills/.yam-flow-install-receipt.json`. The receipt records the package version, source identity, install timestamp, destination, managed skill inventory, and per-file hashes. `yam status` independently recomputes source and installed hashes, so a stale package install, local file change, unexpected file, or modified receipt is reported as drift.
+A successful install writes `~/.agents/skills/.yam-flow-install-receipt.json`. The receipt records the package version, source identity, install timestamp, destination, managed skill inventory, and per-file hashes. Integrity paths and digests use locale-independent ordinal ordering; receipts written by yam 2.5.0 and earlier are also checked against their stored legacy order during upgrade. `yam status` independently recomputes source and installed hashes, so a stale package install, local file change, unexpected file, or modified receipt is reported as drift.
 
 The receipt is local integrity provenance, not a cryptographic signature or a security boundary against another process running as the same OS user. An actor that can rewrite both the receipt and every recorded file can forge local ownership evidence.
 
@@ -148,7 +150,11 @@ yam update apply --component yam
 yam update apply --all --json
 ```
 
-`--all` processes Scrapling, then Insane Search, then yam so the running updater replaces itself last. It stops at the first failed or manual-only component. Each attempt writes a component receipt under `~/.local/state/yam/update-receipts/` with the old/new version, source revision, checks, side effects, outcome, and rollback guidance. A lock prevents concurrent apply runs.
+`--all` processes Scrapling, then Insane Search, then yam so the running updater replaces itself last. It stops at the first failed or manual-only component. After the global npm install, yam resolves `npm root -g`, the installed package manifest and bin mapping, the first `yam` on `PATH`, canonical realpaths, and the observed version before any `yam install` mutation. A same-version PATH shadow, malformed package, or identity mismatch fails closed; legitimate multi-hop symlinks remain valid. Rollback repeats the same identity check before rollback-side yam commands.
+
+A yam update is successful only after the installed version and status checks plus a final read-only `yam doctor --json` return the expected healthy contract. Doctor failure triggers the existing rollback path, and rollback is considered automatic only when its own identity, version, status, and Doctor checks pass. The deepest actionable rollback or Doctor cause remains primary even if receipt persistence also fails. Each attempt writes a bounded, redacted component receipt under `~/.local/state/yam/update-receipts/` with the old/new version, source revision, executable identity, checks, side effects, outcome, and rollback guidance. The aggregate apply response emits and verifies the strict `yam.gate-result.v1`; an already-held lock also returns a blocked gate without mutating components. A lock prevents concurrent apply runs.
+
+Effective yam identity currently verifies POSIX-style executable/symlink layouts through `which` and canonical realpaths. Windows npm `.cmd` shim identity is intentionally unverified and fails closed until a dedicated shim parser and fixture exist.
 
 Scrapling updates install an exact stable PyPI version into a new versioned virtual environment. `pip check`, ordinary HTTP extraction, and headless browser extraction must pass before the executable symlink changes atomically; the previous environment is retained. An existing environment must have a matching `.yam-scrapling-install.json` ownership marker before yam will replace its link. Insane Search uses only the official `codex plugin` commands. yam never edits `.codex/plugins/cache`, never removes the plugin first, and returns `manual_plugin_update_required` when the installed Codex CLI cannot update it safely in place.
 
@@ -214,6 +220,13 @@ yam ueye asset add --manifest .yam/ueye/assets.json --id official-logo --file ./
 yam ueye asset verify --manifest .yam/ueye/assets.json --json
 yam ueye revision archive --file .yam/screens/home.png --round 1 --artifact-id home --root .yam/ueye/revisions --json
 yam ueye revision verify --manifest .yam/ueye/revisions/manifest.json --json
+yam ueye production canvas create --session-id campaign --title "Campaign plan review" --demand-kind multi_asset_production --demand-evidence "Five coordinated campaign assets require one bounded review package" --artifact-spec ./canvas-artifact.json --json
+yam ueye production canvas comment --session-path .yam/ueye/design-production/campaign/plan-review.json --render-path .yam/ueye/design-production/campaign/canvas.html --comment-id finding-1 --artifact-id campaign-plan --anchor-kind line --locator L12 --finding "CTA hierarchy is ambiguous" --requested-change "Make the primary CTA explicit" --json
+yam ueye production canvas close --session-path .yam/ueye/design-production/campaign/plan-review.json --render-path .yam/ueye/design-production/campaign/canvas.html --verdict request_changes --json
+yam ueye production revision init --session-id campaign --json
+yam ueye production gallery verify --manifest-path .yam/ueye/design-production/campaign/gallery.json --json
+yam ueye production finalize --demand-kind multi_asset_production --demand-evidence "Five coordinated campaign assets require one bounded review package" --canvas-session-path .yam/ueye/design-production/campaign/plan-review.json --revision-state-path .yam/ueye/design-production/campaign/revision-state.json --gallery-manifest-path .yam/ueye/design-production/campaign/gallery.json --receipt-path .yam/ueye/design-production/campaign/phase-receipt.json --json
+yam ueye production phase verify --receipt-path .yam/ueye/design-production/campaign/phase-receipt.json --json
 yam proof --route ueye --truth verified --visual "implementation screenshot evidence recorded" --design-completion '{"completion_claim":"done","has_implementation_screenshot":true,"design_quality":"pass","states_checked":true,"mobile_checked":true,"contrast_checked":true,"cta_checked":true,"direction_locked":true,"truth_status":"verified"}'
 yam media proof --requested --attempted --output ./generated.png --wait-loop --json
 yam proof --route mission --mission-envelope '{"agent_id":"implementer","assigned_scope":"target component","changed_files":["src/file.ts"],"verification_hint":"npm run typecheck","truth_status":"partial"}'
@@ -255,6 +268,8 @@ yam status
 
 After `yam status` reports all skills and the install receipt as `ok`, restart Codex so the refreshed skills reload.
 
+Before packaging or publishing, `verify:self` inspects the exact file list returned by `npm pack --dry-run --json --ignore-scripts`. The release gate scans regular UTF-8 text files up to 2 MiB for high-signal private-key, npm, GitHub, OpenAI, Anthropic, and AWS credential patterns. Findings expose only the pattern id, package-relative path, and line number; matched text is never printed. Binary, symlink, non-regular, unreadable, outside-root, and oversized entries are reported as skipped rather than silently represented as scanned.
+
 Publishing still requires confirming the final npm package name and account access.
 
 ## Trust Kernel
@@ -316,6 +331,10 @@ These shapes keep reports machine-readable without turning normal work into a he
 - Ueye design brief and anti-slop review: `$ueye` reports can record operator-provided brief dimensions, constraints, invented metrics, placeholder copy, generic visuals, and custom anti-slop blockers. P0 anti-slop findings block `done` claims.
 - Ueye deep visual review: `$ueye` can record acceptance criteria, touched/read/verified files, skipped checks, residual risks, stop condition, resume hint, design-system evidence, implementation evidence, and a state matrix so serious UI work can carry Deep-grade verification without leaving the Ueye route.
 - Ueye asset/revision integrity: `$ueye` can record license/provenance and protected/editable flags for local references, then preserve pre-edit artifacts in hash-verified numbered rounds.
+- Demand-gated design production: when repeated Plan Review or large multi-asset work creates real coordination demand, Ueye can add a local static Canvas-lite review, at most two finding-backed revision rounds, a final gallery manifest, and one phase receipt. One-off review and single-asset work stay on the ordinary Ueye/Mission path. Integrity evidence never becomes proof of visual quality, licensing, or implementation correctness.
+- Bounded promotion receipt: `yam benchmark report` can compare paired candidate/baseline measurements under explicit sample, mean-delta, and win-rate thresholds. Arithmetic is verified, while measurement provenance remains `operator_asserted` and overall truth stays `partial`.
+- Honest capability matrix: `yam tools doctor --json` reports feature maturity separately from observed runtime readiness; instruction-only or unprobed capabilities are not reported as executed.
+- Strict gate result: release, update, benchmark-promotion, and Mission completion boundaries emit a digest-bound `yam.gate-result.v1` and fail closed on missing or inconsistent required evidence.
 
 ## Ueye Capture And Compare
 
@@ -327,6 +346,7 @@ yam ueye compare --reference ./reference.png --actual .yam/screens/home.png --js
 yam ueye preflight . --json
 yam ueye asset verify --manifest .yam/ueye/assets.json --json
 yam ueye revision verify --manifest .yam/ueye/revisions/manifest.json --json
+yam ueye production --help
 yam ueye report --reference ./reference.png --actual .yam/screens/home.png --brief-dimension "primary CTA clarity" --constraint "mobile first" --anti-slop "placeholder copy remains" --acceptance-criterion "pricing card CTA remains visible on mobile" --design-system-evidence "uses existing button token" --implementation-evidence "browser screenshot reviewed" --state-check default:pass --state-check mobile:partial --skipped-check "hover state not checked on touch viewport" --residual-risk "tablet breakpoint still needs visual pass" --stop-condition "stop after primary states are checked and residual risk is recorded" --resume-hint "capture tablet screenshot next" --completion-claim done --design-quality pass --direction-locked --reference-read --states-checked --mobile-checked --contrast-checked --cta-checked --provider-context local --execution-surface in-app-browser --browser-surface in-app-browser --preserved-state --json
 ```
 
@@ -338,6 +358,8 @@ yam ueye report --reference ./reference.png --actual .yam/screens/home.png --bri
 
 `preflight` does not claim visual verification. It is a pre-work checklist for reference source, target states, mobile/responsive needs, CTA/contrast/accessibility risk, screenshot availability, and likely P0/P1 candidates.
 
+The design production phase is deliberately dormant by default. Use it only when repeated plan annotation or a large coordinated asset set makes prose/file-by-file review costly. The operator-asserted demand trigger is recorded before Canvas creation and must match at finalize. Canvas is static local HTML with no server; its closed `request_changes` session binds one canonical `revision-state.json`, so init/record/show/finalize cannot select an alternate state to reset the two-round budget. Each recorded round must link a current Canvas comment and a newly archived Ueye revision created within that review boundary; the final gallery verifies path/hash/dimension/final-round linkage only and must be created no earlier than Canvas close or the final revision record. `finalize` binds SHA-256 digests for Canvas, revision state, and gallery into an immutable receipt, while `phase verify` re-reads nested upstream semantics and detects later evidence changes. Overall truth remains `partial`, correctness claims remain false, and a two-round unresolved stop remains draft.
+
 Generated media can guide visual direction, but it cannot prove the implemented UI by itself:
 
 ```bash

package/ROADMAP.md

@@ -172,7 +172,7 @@ Tasks:
 
 - Add `.yam/memory/records/*.json` storage. Done.
 - Add `yam memory init/add/list/summary/resolve`. Done.
-- Add lightweight secret-pattern blocking. Done.
+- Add lightweight secret-pattern blocking, including an actual npm dry-run packlist release guard with redacted findings. Done.
 - Teach context reuse to prefer `summary.md`, not full record reads. Done.
 - Keep memory opt-in only. Done.
 
@@ -308,6 +308,7 @@ Tasks:
 - Keep `yam-lite` advisory-only through `UserPromptSubmit`. Done.
 - Add Study Note `UserPromptSubmit` reminder plus one-pass `Stop` completion gate. Done.
 - Detect broken hook targets and migrate stale paths without replacing unrelated hooks. Done.
+- Bound hook/general stdin and fail open safely for rejected hook payloads. Done.
 - Add `references/tool-trust-layer.md`. Done.
 - Add read-only `yam tools doctor`. Done.
 - Add `yam proof` summary command. Done.
@@ -318,6 +319,8 @@ Tasks:
 - Add shallow `.sql` destructive keyword scan. Done.
 - Add `yam proof write`. Done.
 - Prepare npm/npx package metadata and dry-run packaging. Done.
+- Make integrity-critical install and Ueye manifest ordering locale-independent. Done.
+- Require a final read-only Doctor contract for yam update and rollback success. Done.
 - Add small route-scoped trust kernel: truth caps, fake/real policy, runtime truth matrix, and completion proof object. Done.
 
 ### 16. Deferred Triggers
@@ -432,7 +435,7 @@ Keep out for now:
 
 #### Plan Review Canvas Lite
 
-Status: deferred until repeated real-use demand.
+Status: available only inside the opt-in Demand-Gated Design Production Phase.
 
 Trigger:
 
@@ -440,15 +443,47 @@ Trigger:
 - Terminal-only approval causes revision ambiguity or costly re-reading.
 - A local browser review surface would materially shorten an approved Mission or Ueye feedback loop.
 
-First shape:
+Implemented shape:
 
-- local-only artifact review.
-- anchored comments and approve/request-changes verdict.
-- explicit session end and bounded idle shutdown.
-- Host/Origin checks, path confinement, and sandboxed artifact rendering.
+- local-only, path-confined static HTML artifact review with no server or background process.
+- exact line/text anchors and bounded operator-asserted image regions.
+- source hash and exact-anchor revalidation before comments, close, and finalize.
+- anchored findings plus explicit approve/request-changes verdict and session end.
 
 Keep out for now:
 
 - default local server.
 - remote sharing or hosting.
 - automatic implementation start without an explicit approval verdict.
+
+#### Demand-Gated Design Production Phase
+
+Status: implemented as an opt-in phase; remain dormant until repeated plan review or large multi-asset design production creates real coordination cost.
+
+Trigger:
+
+- Plan review repeatedly needs exact-element annotations that prose feedback cannot express efficiently.
+- A Ueye task produces several related visual assets that must preserve cross-asset consistency and provenance.
+- Reviewers need bounded before/after comparison or a packaged overview instead of inspecting files one by one.
+
+Phases:
+
+1. Record the operator-asserted demand kind and concrete evidence before Canvas creation, then require an exact match at finalize.
+2. Run Plan Review Canvas Lite as a local-only, explicitly closed review session.
+3. Bind one canonical revision state to the closed Canvas path/digest, then run at most two Canvas-comment-linked revision rounds, requiring at least one new hash-verified Ueye archive in the current review boundary per round.
+4. Write and verify a final gallery manifest with artifact ids, roles, provenance, final-round revision references, dimensions, hashes, completion state, and final paths.
+5. Finalize one immutable phase receipt with Canvas/revision/gallery and canonical receipt digests only when the demand trigger, closed Canvas, archive chronology, terminal revision state when required, matching session ids, and gallery integrity agree; re-verify it to detect later evidence changes.
+
+Entry and stop rules:
+
+- Enter only after the trigger appears in real work; a one-off plan, screenshot review, or single-asset edit stays on the existing Ueye/Mission path.
+- Each round requires a concrete finding from the bound Canvas; alternate state paths fail closed, and the phase ends after acceptance, two rounds, or a blocked integrity gate, whichever comes first.
+- The gallery is an inspection and packaging artifact, not proof that visual quality, licensing, or implementation correctness passed.
+- A two-round-limit stop may finalize only a draft gallery; it cannot be upgraded into a packaged or complete result.
+
+Keep out for now:
+
+- automatic creation of a design runtime or typed JSONL message bus.
+- unbounded revision loops or autonomous approval.
+- remote gallery hosting, public sharing, or background review services.
+- replacing protected source assets or weakening existing provenance and `do_not_replace` checks.