instar 1.3.554 → 1.3.556

package/upgrades/1.3.556.md

@@ -0,0 +1,34 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: minor -->
+
+## What Changed
+
+Implements the receiving end of the feedback-factory migration's Option-B write seam (`docs/specs/feedback-factory-migration.md`, Amendment A1 / Q2b — Dawn-confirmed 2026-06-11): the operated instance can now durably ACCEPT live fleet feedback with **no operated machine in the intake critical path**.
+
+- **Durable cloud inbox at the front.** `feedback-front/` (the canonical Vercel receiver) gains a PERSISTENCE mode: when a Vercel Blob token is present in the function's env, the FULL ported intake pipeline runs (`handleFeedbackSubmit` — rate limit, agent fingerprint, honeypot, non-blocking HMAC, validation, dedup) and every accepted report is written as one JSON object to a Blob inbox (`inbox/<feedbackId>.json`, random-suffixed → unguessable URLs). A report is durable the instant it is accepted — a sleeping/restarting operated machine only delays processing, never loses a report. With no token the deployed front's behavior is byte-identical Phase-0 (verify-only).
+- **Zero-dependency Blob client.** `BlobInboxClient` is a hand-rolled REST client (request shape from `@vercel/blob` 0.27.3, declared as `x-api-version: 7`) shared by the front bundle and the server — the front keeps its no-install deploy contract, and both ends of the inbox speak one protocol with zero drift. (Live-verified against the production Vercel Blob API: the client's pathname-in-URL request shape is rejected with `400 "Invalid pathname"` once the declared api-version is 9+, so the header is pinned to `7`, the version whose wire contract matches the request this client builds — verified across PUT/LIST/DELETE.)
+- **InboxDrainer on the operated machine.** Polls the inbox and ingests into the canonical store with at-least-once + feedbackId-dedup idempotency, delete-only-after-durable-commit ordering, poison-object quarantine (preserved under `quarantine/`, never dropped), reentrancy guard, and consume-safe pagination. Read-only status at `GET /feedback-inbox/status`.
+- **Durable canonical store.** `JsonlFeedbackStore` — append-only JSONL with last-write-wins load, boot-time atomic compaction, and torn-line tolerance. Deliberately format-compatible with `PersistedShadowImportTarget`, so the cutover's proven AS-IS import artifact (1,412 clusters + 148,115 rows, zero integrity issues) seeds the canonical store with no translation step.
+- **Receiver handler async seam.** `handleFeedbackSubmit` is now async over a narrow `ReceiverStore` (hasFeedback/addFeedback, sync or async) — decision order, status codes, and messages are unchanged (existing unit tests re-asserted; the front's HMAC round-trip harness green against the rebuilt bundle).
+- **Ships DARK.** Everything is behind `feedbackFactory.receiverPersistence.enabled` (default off) + a Blob token env (`FEEDBACK_INBOX_BLOB_TOKEN`); the route 503s when dark. Nothing changes for any install until the operated instance's cutover deploy.
+
+## What to Tell Your User
+
+- "The feedback system I report bugs to is moving to new infrastructure. Nothing changes for you — this update only installs the (dormant) receiving machinery; the switch-over happens later, announced separately."
+- "If this install ever runs its own feedback factory: incoming reports are now made durable in the cloud the moment they're accepted, so a machine being asleep or restarting can never lose one."
+
+## Summary of New Capabilities
+
+| Capability | How to Use |
+|-----------|-----------|
+| Feedback-inbox drain status | `GET /feedback-inbox/status` → `{ running, drained, duplicates, quarantined, errors, lastDrainAt, ... }` (503 when dark) |
+| Durable receiving end (operated instances) | `feedbackFactory.receiverPersistence.enabled: true` + `FEEDBACK_INBOX_BLOB_TOKEN` env |
+| Front persistence mode | Provide a Blob token in the Vercel function env (connect a Blob store); no token = Phase-0 verify-only |
+
+## Evidence
+
+- All three test tiers, new: 4 Tier-1 unit files (JsonlFeedbackStore durability/adoption/compaction, BlobInboxClient wire protocol, InboxDrainer semantics incl. both sides of the quarantine boundary, BlobInboxStore + async handler path), Tier-2 integration (the full accept → durable inbox → drain → canonical store pipeline over a real local Blob-protocol HTTP server, incl. retransmit-after-drain dedup), Tier-3 e2e lifecycle (dark → 503; enabled on the production init path → 200 + WIRING INTEGRITY: a seeded inbox blob lands as a durable row in the real on-disk store at the production default path).
+- Unit tests caught two real bugs pre-commit (consume-while-paginating skip; compaction threshold off-by-half) — both fixed, suites green.
+- Independent second-pass review of the side-effects artifact (`upgrades/side-effects/feedback-receiver-persistence.md`) raised one concern (a dropped cosmetic `phase: 0` field falsifying a byte-equivalence claim) — fixed in favor of byte-identity, front rebundled, verify harness re-run green.

package/upgrades/side-effects/feedback-receiver-persistence.md

@@ -0,0 +1,81 @@
+# Side-Effects Review — Feedback-factory receiver persistence (Option-B receiving end)
+
+**Version / slug:** `feedback-receiver-persistence`
+**Date:** `2026-06-11`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `required (receiver/intake decision surface) — see appended response`
+
+## Summary of the change
+
+Implements the receiving end of the migration spec's Option-B write seam (docs/specs/feedback-factory-migration.md, Amendment A1 / Q2b, Dawn-confirmed 2026-06-11): the canonical front (feedback-front/ on Vercel) durably persists each ACCEPTED fleet report into a cloud Vercel-Blob inbox, and a new `InboxDrainer` on the operated machine ingests them into a new durable `JsonlFeedbackStore` (format-compatible with the proven AS-IS import artifacts). Files: `src/feedback-factory/inbox/BlobInboxClient.ts` (new — pinned Blob REST, zero deps), `src/feedback-factory/inbox/InboxDrainer.ts` (new), `src/feedback-factory/store/JsonlFeedbackStore.ts` (new), `src/feedback-factory/receiver/BlobInboxStore.ts` (new), `src/feedback-factory/receiver/handlers.ts` (handler made async over a narrow `ReceiverStore` seam — decision logic untouched), `feedback-front/src/feedback.ts` + rebundled `api/feedback.js` (persistence mode behind a deploy-time env token; byte-equivalent Phase-0 behavior without it), server wiring (`AgentServer.ts`, `routes.ts` GET /feedback-inbox/status, `CapabilityIndex.ts`), config type (`types.ts feedbackFactory.receiverPersistence`, dark default), Agent Awareness (`templates.ts`, `PostUpdateMigrator.migrateClaudeMd` + shadow marker). All three test tiers added.
+
+## Decision-point inventory
+
+- `handleFeedbackSubmit` intake defense chain (rate limit / fingerprint / honeypot / HMAC / validation / dedup) — **pass-through** — made async-capable; decision ORDER, status codes, and messages unchanged (existing unit tests re-asserted as async; the front's HMAC round-trip harness re-run green against the rebuilt bundle).
+- Front mode selection (`inboxToken()` present ⇒ persistence mode, absent ⇒ Phase-0 verify-only) — **add** — a deploy-time structural gate, not a per-request heuristic; no content-based decision.
+- `parseInboxItem` quarantine boundary in the drainer — **add** — classifies an inbox object as ingestable vs malformed. Malformed is PRESERVED (quarantine/), never dropped; no user-visible block.
+- `feedbackFactory.receiverPersistence.enabled` config gate — **add** — dark default; enabled-without-token degrades to dark with a loud boot log line.
+
+---
+
+## 1. Over-block
+
+The intake defense chain is untouched, so no new rejection of legitimate reports. The new quarantine boundary could over-classify: a future receiver writing a richer shape would still pass (extra fields flow through `[k: string]: unknown`); only a missing/empty `feedbackId`, `title`, or `description` quarantines — and the receiver structurally always writes those (it validated them two layers earlier). Residual: an operator hand-seeding the inbox with a malformed object gets a quarantined (preserved) object, which is the intended behavior. Precision note (second-pass finding): the validator only empty-checks `feedbackId`; an empty-STRING title/description passes ingest — the safe, less-quarantine direction, and the receiver validated both two layers earlier. No issue identified beyond that.
+
+## 2. Under-block
+
+- The per-IP rate limiter on Vercel is per-warm-instance, so a multi-instance burst admits more than 10/hr/IP (the reference's limiter has the same scope on its single deployment; fingerprint + honeypot + HMAC layers are instance-independent). Known accepted gap, flagged in the Phase-0 deploy runbook; durable backing is a deploy-time follow-up. <!-- tracked: topic-12476 -->
+- Front-side dedup (`hasFeedback` = inbox prefix list) cannot see already-DRAINED ids — a retransmit after drain re-enters the inbox. By design: the drainer's canonical-store dedup drops it at ingest (integration test pins this), so end-to-end idempotency holds.
+
+## 3. Level-of-abstraction fit
+
+The front is stateless HTTP at the edge (Vercel — correct per spec §2.2); durability is a storage primitive at the platform layer (Blob — correct: a stateless function cannot host a durable buffer, which is exactly why the "forward + buffer in the function" alternative was rejected); the drainer is lightweight always-on at the operated-machine layer (Mini — matches §2.2's own placement taxonomy); the canonical store is data at the disk layer, format-aligned with the import machinery it inherits from. The drainer FEEDS the existing FeedbackStore seam rather than re-implementing any receiver logic. No layer inversion identified.
+
+## 4. Signal vs authority compliance
+
+No new blocking authority anywhere. Every accept/reject decision stays in the already-shipped, already-reviewed ported defense chain; the new components are pure transport/persistence; the status route is read-only observability; the quarantine path preserves rather than discards. Compliant with docs/signal-vs-authority.md — this change adds zero brittle checks with authority.
+
+## 5. Interactions
+
+- `cutoverReadiness` and the import machinery are untouched; the JsonlFeedbackStore deliberately READS the same JSONL shape `PersistedShadowImportTarget` writes, so the cutover import artifact seeds the canonical store without a translation step (unit test pins alias-id adoption).
+- The handler's async change touches its one existing consumer (its unit test, updated). The front imported only defense fns before; it now also imports the handler — bundled at dev time, no runtime dep added.
+- No double-fire risk: the drainer is single-instance (one server), `drainOnce` is reentrancy-guarded (`ticking`), and delete-after-commit + dedup makes overlap with a retransmit benign.
+- Server boot: the init block is try/caught and null-on-failure (deny-safe 503), so it cannot cascade into adjacent component init.
+
+## 6. External surfaces
+
+- New Bearer-gated read-only route `GET /feedback-inbox/status` (+ CapabilityIndex + CLAUDE.md template/migration). No unauthenticated surface added.
+- The front's PERSISTENCE mode changes what a fleet sender sees ONLY in that accepted reports now return the reference receiver's exact response shapes (it runs the faithful handler) — and the mode is unreachable until a Blob token reaches the function's env. Note (second-pass finding): the fallback to the store-injected `BLOB_READ_WRITE_TOKEN` means CONNECTING a Blob store to the Vercel project is itself the enabling act — deliberate, dashboard-gated, on a dedicated project, but it is the precise activation boundary, not "only the cutover deploy". Until a token exists the deployed front's no-token behavior is byte-identical Phase-0 (the 405/503 pre-gate bodies were restored to the original shape after the second-pass caught a dropped cosmetic `phase: 0` field; verify script re-run green against the rebuilt bundle).
+- Blob objects are public-but-unguessable URLs (random suffix ON, pinned by unit test). Report bodies are fleet bug reports — same sensitivity class as today's POST-to-Portal path; pathnames are never predictable, and the drainer reads URLs from list results only. Token lives in env (front: Vercel env; operated machine: `FEEDBACK_INBOX_BLOB_TOKEN`), never in config.json or source.
+- Timing dependence: the drainer is poll-based; a down machine only delays ingestion — the durable inbox is the buffer. That asymmetry is the design's point.
+
+## 7. Rollback cost
+
+- Dark by default: fleet installs see only an inert 503 route; no behavior change anywhere until BOTH the config flag and the token exist.
+- Operated instance back-out: set `enabled: false` (or drop the token) and restart — the inbox simply accumulates durably until re-enabled; nothing is lost, nothing dangles.
+- Front back-out: remove the token env from the Vercel project → next invocation is Phase-0 verify-only again. No deploy needed beyond an env change.
+- Full revert of the PR is clean: no migrations of durable data, no schema, no external state created at merge time. The PostUpdateMigrator section-append is idempotent and harmless if orphaned (documentation text only).
+
+---
+
+## Second-pass review response (independent reviewer subagent, 2026-06-11)
+
+Concern raised: the artifact's "byte-equivalent Phase-0 behavior without the token" claim was overstated — two responses in the front's no-token path changed. `feedback-front/src/feedback.ts` (405 method-not-allowed) and (503 receiver-not-configured) both dropped the `phase: 0` field that origin/main includes, and these two checks run BEFORE the token gate, so the change was live in today's no-token deployment. Materiality is low: the fleet sender never hits either path (POST-only, secret configured), the verify harness asserts neither, and dropping `phase: 0` there is arguably correct since those responses are now mode-independent — but the artifact asserted byte-equivalence twice and it was checkably false.
+
+Everything else audited independently checked out:
+- **Intake chain identical**: the `handlers.ts` diff vs origin/main touches only the new `ReceiverStore` interface, the async signature, and two awaited store calls — decision order, status codes, and messages unchanged; the handlers.test.ts diff reduces to pure async-call-shape edits.
+- **Delete-after-commit + reentrancy real**: `drainOnce` guards on `ticking` (cleared in `finally`); `drainBlob` commits via a synchronous `appendFileSync` before `await client.del`; the quarantine path preserves-then-clears; pagination re-lists the first page with zero-progress and maxBatches guards; the integration test pins retransmit-after-drain dedup.
+- **Dark default holds at all three gates**: server requires `rp?.enabled === true` AND the token env (warns + stays null otherwise); the route 503s on null; the front requires `inboxToken()`. Nuance: the `BLOB_READ_WRITE_TOKEN` fallback means connecting a Blob store to the Vercel project auto-enables persistence mode — deliberate, dashboard-gated, acceptable, but it weakens "unreachable until the cutover deploy".
+- **Signal-vs-authority compliant**: no new blocking authority — `parseInboxItem` is a boundary structural validator whose failure path preserves rather than drops; all accept/reject judgment stays in the pre-existing ported defense chain. Minor imprecision (no action): §1's "empty title/description quarantines" — the code only empty-checks `feedbackId`; empty-string title/description pass (the safe direction).
+
+**Resolution (author, same session):** the concern was fixed in the design's favor of byte-identity — `phase: 0` restored on both pre-gate responses (the no-token deployment is now literally byte-identical to origin/main), the front rebundled, and the verify harness re-run green. The two nuances were folded into §1 and §6 above. Iterated and closed.
+
+---
+
+## Post-merge live-API verification (2026-06-13, activation build)
+
+Bringing the operated receiver live against the **real** Vercel Blob API surfaced one external-dependency defect not catchable by the stubbed unit tier (§6 follow-up):
+
+- **`BlobInboxClient` x-api-version 9 → 7.** The hand-rolled client declared `x-api-version: 9` (carried over from the @vercel/blob source it was pinned against). The production Blob API REJECTS this client's v7-era request shape (pathname-in-URL, simple PUT body) with `400 bad_request "Invalid pathname"` once the declared version is 9+; the newer protocol versions expect a different request format the hand-rolled client does not build. Verified live against PUT/LIST/DELETE: all succeed at `7`, all fail at `9`/`11`. Pinned to `7` (the version whose wire contract matches the request this client builds). One-line fix in the shared client → corrects BOTH the front bundle and the operated-machine drainer. Adopting the @vercel/blob 2.x (`BLOB_API_VERSION=12`) protocol is a larger rewrite, tracked as receiver hardening. **Side-effect class:** external-API contract only — no data-shape, auth, dedup-key, or rollback impact (inbox object layout + drain semantics unchanged). This is the wire-version a request declares, not what it stores.
+- **Activation evidence:** a signed HMAC POST to `https://feedback.dawn-tunnel.dev/api/feedback` returns `200 {received:true}` and the report lands durably in the Blob inbox as `inbox/<feedbackId>.json` with `verified:true, status:unprocessed` — the full §6 external surface exercised end-to-end against the live store.
+- **Operator-env note (§6):** the operated receiver's webhook secret must be set with no trailing bytes — a prod env value of `instar-rising-tide-v1` + a literal backslash-n (NOT a real newline) was found and corrected. `normalizeWebhookSecret`/`trim()` strips real whitespace but NOT a literal `\n`, so a corrupt env value silently fails the HMAC. Set via `printf` (no trailing newline).

package/upgrades/side-effects/mm-stores-devgate.md

@@ -0,0 +1,92 @@
+# Side-Effects — multiMachine.stateSync memory stores → dev-gated (topic 13481)
+
+**Change:** Re-gate the 7 `multiMachine.stateSync.*` cross-machine memory stores
+(preferences, relationships, learnings, knowledge, evolutionActions, userRegistry,
+topicOperator) from `DARK_GATE_EXCLUSIONS` (off for EVERYONE, including dev agents) to
+`DEV_GATED_FEATURES` (live-on-dev / dark-fleet, `dryRun:false`), mirroring the
+`subscriptionPool.credentialRepointing` precedent.
+
+**Driver:** Operator directive (Justin, topic 13481, 2026-06-13): "NOTHING should ship dark
+on development agents — every multi-machine feature must be live on dev agents so it
+actually gets tested, not rot." A feature shipped dark everywhere never gets exercised; the
+WS2 replicated-store family was in exactly that state.
+
+**Spec:** `docs/specs/multi-machine-replicated-store-foundation.md` (the converged + approved
+spec the 7 stores were built against). This is a follow-up gating fix to the merged WS2
+stores, not a new feature.
+
+## What was wrong
+
+The 7 stores shipped with a literal `enabled: false` in `ConfigDefaults.ts` and were
+classified in `DARK_GATE_EXCLUSIONS` ("deliberate-fleet-default" — off for everyone,
+including dev). That is the #1001 anti-pattern the §12.5 dark-gate lint forbids for a
+dev-gated block: a written `enabled: false` literal force-darks even a development agent,
+so the stores never ran ANYWHERE — they could never be dogfooded on Echo/the Mini and never
+graduated.
+
+## What changed
+
+- `src/core/devGatedFeatures.ts` — moved all 7 entries from `DARK_GATE_EXCLUSIONS` to
+  `DEV_GATED_FEATURES`, each keyed on `multiMachine.stateSync.<store>.enabled` (the wiring
+  tests now prove each resolves live-on-dev / dark-on-fleet).
+- `src/config/ConfigDefaults.ts` — OMIT `enabled` from all 7 store blocks so the
+  developmentAgent gate decides; set `dryRun: false` (these stores replicate between the
+  operator's OWN machines with no destructive write, so unlike credentialRepointing they
+  need no write-safety dry-run canary — a dry-run would defeat "actually gets tested").
+- `src/core/devAgentGate.ts` — NEW `resolveStateSyncStores(config)` helper: returns a new
+  stores map where each store's `enabled` is the gate-resolved boolean (raw `enabled` ??
+  `!!developmentAgent`), preserving every other per-store field (e.g. `dryRun`). Non-store
+  foundation knobs (numbers like `maxDriftMs`) pass through untouched.
+- `src/commands/server.ts` — resolve the gate ONCE at the construction boundary
+  (`_stateSyncStoresResolved = resolveStateSyncStores(config)`) and feed that resolved map
+  into `selfStateSyncReceive`, all 7 `new ReplicatedStoreReader({ stores })` instances, and
+  `checkPoolFlagCoherence`. The funnels keep their unchanged `enabled === true` semantics —
+  they now read an already-resolved boolean, so a dev agent sees a LIVE flag. This is why
+  `ReplicatedStoreReader.ts`'s raw `.enabled === true` read is correct and was NOT touched.
+- `src/server/routes.ts` — the `/preferences/session-context` route resolves the
+  `preferences` store's `enabled` via `resolveDevAgentGate(..., ctx.config)` so the route's
+  own gate and `ctx.preferencesUnionReader.isLive()` agree.
+- `src/core/PostUpdateMigrator.ts` — NEW `migrateConfigStateSyncStoresDevGate(config)`:
+  strips ONLY the exact old-default signature `{ enabled:false, dryRun:true }` (2 keys, that
+  exact shape) per store, so `applyDefaults` backfills the new `{ dryRun:false }` and the
+  gate resolves `enabled`. Any divergence (operator-set `enabled:true`, a different dryRun,
+  extra keys) is treated as operator-touched and left ENTIRELY alone. Idempotent. Wired into
+  the migrate path (`upgraded`/`skipped` reporting), mirroring the credentialRepointing strip.
+- `tests/unit/lint-dev-agent-dark-gate.test.ts` — the 7 stateSync `enabled:`-line entries
+  are GONE from the EXPECTED attribution map (the literals were removed from ConfigDefaults,
+  so they have no attributed enabled:false path — exactly like credentialRepointing); the
+  three cartographer entries below them shift up (1125→1100, 1170→1145, 1195→1170). Verified
+  via the attributor against the edited ConfigDefaults.
+
+## Behavioral impact
+
+- **Dev agents (`developmentAgent: true`):** all 7 stores now resolve LIVE and `dryRun:false`
+  — replication genuinely runs so the family is dogfooded. Replication is between the
+  operator's OWN machines only: NO external egress, NO third-party spend. The PII stores
+  (relationships, userRegistry, topicOperator) carry the same at-rest honesty already
+  documented — at-rest plaintext per machine, transit-encrypted; an inbound-principal
+  RESOLUTION and "who is my verified operator?" stay LOCAL-authoritative (a replicated record
+  is untrusted advisory data). Fully reversible: the foundation's rollback-unmerge atomically
+  drops a peer's contribution on disable.
+- **Fleet (`developmentAgent` unset/false):** UNCHANGED — all 7 stores resolve dark, a
+  strict no-op, exactly as before. A single-machine agent is a no-op regardless.
+- **Reversibility:** set an explicit `multiMachine.stateSync.<store>.enabled: false` in
+  config to force-dark even a dev agent; an explicit `true` is the documented fleet-flip.
+
+## Migration parity
+
+`applyDefaults` is add-missing-only deep-merge, so a new agent gets the omitted-`enabled` /
+`dryRun:false` shape via `init`. Existing agents that already received the old
+`{ enabled:false, dryRun:true }` per store would, under add-missing-only, keep that stale
+shape (explicit values are not overwritten) and stay dark even on a dev agent — so
+`migrateConfigStateSyncStoresDevGate` strips the exact old-default signature on update,
+letting the gate resolve and `applyDefaults` backfill `dryRun:false`. An operator's
+hand-edited block (any divergence from the exact signature) is never touched — reach is not
+authority. The migration is idempotent (a second run finds nothing default-shaped to strip).
+
+## Tests
+
+Unit: `state-sync-stores-dark-gate` (40), `PostUpdateMigrator-stateSyncStoresDevGate` (7),
+the 5 ws2x wiring tests (ws22/23/24/25/26), `lint-dev-agent-dark-gate` (line-map recomputed),
+`no-silent-fallbacks`, `feature-delivery-completeness` — all green locally. Typecheck clean
+(`tsc --noEmit` exit 0). The full unit suite is left to CI (it is the authority).