instar 1.3.43 → 1.3.45

package/upgrades/side-effects/cross-machine-seamlessness-noloss-recovery.md

@@ -0,0 +1,61 @@
+# Side-Effects Review — Cross-Machine Seamlessness: no-loss stuck-message recovery (D-noloss)
+
+**Spec:** docs/specs/CROSS-MACHINE-SEAMLESSNESS-SPEC.md §8 G3a ("Stuck-`processing` recovery")
+
+The complement to the inbound dedup gate. With dedup alone, a crash AFTER claiming an
+inbound event but BEFORE the reply commits would lose the reply (Telegram won't redeliver
+once polled). This adds the spec's explicit "re-run from the stored input by the current
+lease holder" mechanism: a `processing` entry stuck past `maxProcessingMs` is re-routed from
+its `inputSnapshot`, producing the lost reply. Together with the dedup gate this makes the
+flag's guarantee genuinely *exactly-once* (no-loss AND no-duplicate). Still flag-gated dark
+(`multiMachine.exactlyOnceIngress`, default false).
+
+## What changed
+- `src/messaging/stuckMessageRecovery.ts` — NEW. `recoverStuckMessages(deps)`: lease-gated
+  (a standby never injects); `reclaimStuck(maxProcessingMs)` → for each entry with a stored
+  input and `attempts < maxReplayAttempts` (default 3), re-claim under the current epoch and
+  call the injected `reinject(topic, dedupeKey, text)`. Pure orchestration over injected deps.
+- `src/commands/server.ts` — after `server.start()`, ONLY when the ledger is wired (flag on)
+  AND telegram is present: builds `reinject` (sets the per-topic current-inbound key, then
+  routes the stored text via the same `telegram.onTopicMessage` path a fresh forward uses),
+  runs recovery at boot, and on a cadence (`max(30s, maxProcessingMs)`), lease-gated.
+
+## Over-block / under-block
+- OVER-re-run (the risk = re-answering a message the agent legitimately left unanswered, or a
+  persistently-failing turn looping): bounded by (1) only entries stuck past `maxProcessingMs`
+  (a genuinely timed-out turn, not one in flight), (2) `attempts < maxReplayAttempts` cap —
+  after the cap the entry is left alone, never re-run again, (3) lease-gate (only the awake
+  machine), (4) flag-off default. Proven both sides in stuck-message-recovery.test.ts.
+- UNDER-recover: an entry with no `inputSnapshot` can't be replayed (nothing captured) → skipped
+  honestly rather than guessed. Inputs ARE captured by the dedup gate's `decideIngress(input:text)`.
+- The known single-duplicate residual (spec §3 / §8 G3a impossibility floor): a crash AFTER the
+  reply physically sent but BEFORE `reply_committed` persisted → re-run produces one duplicate.
+  This is the documented Two-Generals floor, bounded to one, not a defect.
+
+## Signal vs authority
+- recoverStuckMessages decides which entries to re-run; the re-injection authority is the
+  existing onTopicMessage routing. The module holds no Telegram/server coupling (injected deps).
+
+## Interactions
+- Re-injection sets `currentInboundByTopic[topic]` BEFORE routing so the eventual reply commits
+  the correct entry (same coupling the inbound gate uses) — closing the loop: re-run → reply →
+  commit → not re-run again.
+- Telegram-only for v1 (the `--no-telegram` sessionManager path is a tracked refinement; the
+  block is guarded on `telegram` present).
+- Cadenced re-scan is `unref()`'d (never holds the process open) and lease-gated (a standby's
+  timer no-ops).
+
+## Rollback cost
+- Low. Flag default-off ⇒ the whole block is skipped (messageLedger undefined). Revert = drop the
+  post-start block + the module. No persisted schema beyond the opt-in ledger SQLite.
+
+## Tests
+- `tests/unit/stuck-message-recovery.test.ts` (7): re-runs a stuck entry from stored input;
+  standby no-op; already-replied not re-run; in-flight (within window) not re-run; attempts-cap
+  gives up (no storm); no-input skipped; + boot wiring-integrity (server.ts calls it after
+  server.start(), gated on the ledger + coordinator lease).
+
+## NOT in this increment (tracked, topic-13481)
+- Behavioral boot-recovery e2e (needs a live Telegram adapter) — exercised by the live flag-flip
+  test-as-self, which is the gate before defaulting the flag on. Cross-machine marker (D-xmachine)
+  + CONTINUATION resume (D3) remain.

package/upgrades/side-effects/cross-machine-seamlessness-reply-marker.md

@@ -0,0 +1,65 @@
+# Side-Effects Review — Cross-Machine Seamlessness: reply-marker propagation (D-xmachine)
+
+**Spec:** docs/specs/CROSS-MACHINE-SEAMLESSNESS-SPEC.md §8 G3a ("dual-medium reply_committed marker")
+
+The cross-machine half of exactly-once. The same-machine dedup gate + no-loss recovery handle
+one machine; this closes the window where, AFTER a handoff/failover, a provider redelivery of a
+message the OLD holder already answered arrives at the NEW holder — whose ledger wouldn't know,
+so it would re-answer. The holder now propagates a `reply_committed` marker to standby peers on
+each commit; the standby applies it via `applyRemoteReplyMarker`, so the new holder's dedup gate
+drops the redelivery. Still flag-gated dark (`multiMachine.exactlyOnceIngress`, default false).
+
+## What changed
+- `src/core/ReplyMarkerTransport.ts` — NEW. `broadcast(marker)` signs + POSTs
+  {dedupeKey, platform, replyIdempotencyKey, epoch, topic} to each standby peer's
+  `/api/message-marker` over the authenticated machine channel. NO encryption (the marker
+  carries no conversation content — authentication only). No peers → reachable no-op. Never
+  throws (best-effort; the dedup gate + provider redelivery are the backstop).
+- `src/server/machineRoutes.ts` — `POST /api/message-marker` (machineAuth) + `onReplyMarker`
+  ctx callback. Validates the marker shape; 503 when no receiver (exactly-once off).
+- `src/server/AgentServer.ts` — `replyMarkerTransport?` + `onReplyMarker?` options → routeCtx /
+  machineRoutes.
+- `src/server/routes.ts` — RouteContext.replyMarkerTransport; the outbound commit at
+  `/telegram/reply/:topicId` reads the just-committed `replyIdempotencyKey` from the ledger and
+  fires `replyMarkerTransport.broadcast(...)` (fire-and-forget, after the local commit).
+- `src/commands/server.ts` — constructs `ReplyMarkerTransport` (peer resolver = registry
+  machines with a lastKnownUrl) ONLY when `exactlyOnceIngress` + coordinator.enabled; wires
+  `onReplyMarker` → `messageLedger.applyRemoteReplyMarker`.
+
+## Over-block / under-block
+- The marker NEVER over-marks a fresh event: `applyRemoteReplyMarker` only sets reply_committed
+  on an entry not already acted-on, and the dedupeKey is the same provider-stable id both
+  machines compute — so a marker for event X can only dedup event X. The deterministic
+  `replyIdempotencyKey` (hash(dedupeKey+index)) is identical on both machines.
+- UNDER-propagate (a lost marker): best-effort — if the marker doesn't reach the standby, the
+  provider's at-least-once redelivery + the standby's own ledger (once it catches up) still
+  dedup; the residual is the documented single-duplicate Two-Generals floor, not a new failure.
+- Signed channel: `/api/message-marker` requires machineAuth (unsigned → 401, proven in the e2e);
+  the marker carries no secrets so no encryption is needed.
+
+## Signal vs authority
+- The transport propagates; the receiving ledger's `applyRemoteReplyMarker` is the authority on
+  its own state (only promotes a not-yet-acted entry). The dedup gate remains the single drop authority.
+
+## Interactions
+- Fires AFTER the local `commitInboundReply` (the local guarantee never depends on propagation).
+- Skipped for proxy/system sends (same guard as the commit) — a proxy message isn't a user reply.
+- Peer resolver mirrors the live-tail/handoff resolvers (registry machines with lastKnownUrl,
+  excludes self/revoked). Solo agent → no peers → no-op.
+
+## Rollback cost
+- Low. Flag default-off ⇒ transport not constructed, route 503s, no propagation. Revert = drop
+  the transport + route + the broadcast call. No persisted schema.
+
+## Tests
+- `tests/unit/reply-marker-transport.test.ts` (3): no-peers no-op; signs + POSTs to each peer;
+  resolves false (never throws) when all peers reject.
+- `tests/e2e/reply-marker-cross-machine-e2e.test.ts` (2): a PEER-signed marker over real HTTP +
+  machine-auth applies to the receiver's ledger, then a redelivery of that message is DEDUPED
+  (already-replied) — the cross-machine guarantee end-to-end; + unsigned marker rejected (401).
+
+## Completion status
+- With D-dedup + D-noloss + D-xmachine, the exactly-once guarantee is COMPLETE in all dimensions
+  (no-dup same+cross machine, no-loss on crash), all DARK behind the flag. Remaining before the
+  flag flips: D3 (CONTINUATION resume verify) + the over-Telegram two-machine test-as-self (the
+  live acceptance bar, needs the operator's machines).

package/upgrades/side-effects/process-health-tab-eli16-pass.md

@@ -0,0 +1,49 @@
+# Side-Effects Review — Process Health tab ELI16 + structure pass
+
+**Version / slug:** `process-health-tab-eli16-pass`
+**Date:** 2026-05-27
+**Author:** echo
+**Spec:** `docs/specs/PROCESS-HEALTH-DASHBOARD-TAB-SPEC.md` (v4, the original feature spec; this PR is a UX-refinement follow-up that does NOT change the spec's behavioral surface)
+
+## Summary of the change
+
+Eight rounds of user-driven copy + UX iteration on the Process Health dashboard tab (originally shipped in v1.3.27 via PR #435). Pure dashboard-only refinement — no `src/` / `scripts/` / `.husky/` / `skills/` changes — so this PR is outside the instar-dev gate's in-scope set. The reference implementation now backs a durable agent memory (`feedback_dashboard_copy_eli16.md`) that codifies the **Dashboard Standard** for future dashboard features.
+
+**Files changed (UI):**
+- `dashboard/process-health.js` (renderers): rewritten `CATEGORY_WORDS`, `ATTRIBUTION_LABELS`, `STAGES`, `INSIGHT_STATUS_WORDS`; new `TYPE_WORDS`, `STATUS_WORDS`, `CAUSE_CONFIDENCE_WORDS`, `statusDotClass`, `statusDot`, `labeledRow`. Restructured `renderCaptured` and `renderPatterns` around the fixed label vocabulary + `<details>` expandability + status color dots. `renderMaturation` reshaped per-stage as expandable `<details>` with plain-English descriptions. Dropped the redundant per-card framing line in v7.
+- `dashboard/index.html`: added page-intro paragraph + per-section subtitles that distinguish each section and cross-reference adjacent ones; new CSS for `.ph-intro`, `.ph-h-sub`, `.ph-item`, `.ph-item-summary`, `.ph-item-text`, `.ph-item-body`, `.ph-item-row`, `.ph-label`, `.ph-value`, `.ph-status-dot` (+ 5 status-color variants), `.ph-stage-summary`, `.ph-stage-body`; chevron disclosure indicator on every `<details>` (opacity 0.8).
+
+**Files changed (tests):**
+- `tests/unit/process-health-render.test.ts`: assertions rewritten for new label vocabulary + new status copy + new framing absence + structural assertions on `<details>` + status-dot CSS classes (3 new structural tests added).
+- `tests/integration/process-health-tab.test.ts`: assertions updated for new copy; new framing-absence check.
+- `tests/e2e/process-health-tab-lifecycle.test.ts`: assertions updated for new copy.
+
+**Files changed (release):**
+- `upgrades/NEXT.md` (NEW for this PR, `bump: patch`).
+
+## Decision-point inventory
+
+- **Pure UX refinement vs. functional change?** Pure UX. No new endpoints, no new data fields, no new auth surfaces, no behavioral changes to the polling controller, no changes to the route layer, no changes to migration parity. Only the rendered text + DOM structure changes.
+- **Drop the recommendation field entirely vs. keep + relabel?** Dropped. Justin explicitly directed "avoid language that implies user needs to act unless ABSOLUTELY essential". The recommendation field is upstream prose that is action-implying by content; relabeling chrome around it doesn't change that. Actionable insight delivery is the deferred §10 surface anyway; v1 of the tab is the calm informational surface.
+- **Drop the per-card framing line vs. keep + rephrase?** Dropped in v7. Once "verify before acting" was rephrased to "Same kind of problem has come up more than once" in v5, it became the third echo of the section title + section subtitle. The no-action-authority signal that the framing was originally carrying is now structural (the rendered card has nothing actionable on it), not chrome.
+- **Add status color dots vs. status text only?** Dots. Lets the user scan the collapsed list and read state at a glance without expanding each item. Colors are calm (amber/blue/green/gray) — never alarm reds, per the existing visual rules.
+
+## Side-effects analysis
+
+- **Behavioral:** None. The polling controller, route layer, and migration paths are unchanged. Only the rendered DOM differs.
+- **Architectural:** None. Same file structure; the renderer module is the same self-contained ESM as before. No new build step, no new runtime dependency.
+- **Security:** The safety contract (`sanitizeForDisplay`, textContent-only DOM writes, element ban, safeUrl) is unchanged. New label spans (`.ph-label`, `.ph-value`) use textContent only. The XSS negative fixture continues to fire zero live elements + no canary. The `detail.full` redaction is unchanged.
+- **Migration parity:** Not applicable — no agent-installed-file changes (`templates.ts` / `PostUpdateMigrator` / shadow markers untouched).
+- **Performance:** Marginal increase in DOM nodes per item (each captured item now has 4–5 `<div>`s of body content instead of 0–4 flat children). Diff-aware rendering still works (signature mechanism is structure-agnostic). No new fetches.
+- **Reversibility:** Fully reversible — revert the commit and the previous render returns.
+
+## Standard captured (the durable artifact this PR exists to back)
+
+The 8-round iteration converged on a design that's now codified as the **Dashboard Standard** in `feedback_dashboard_copy_eli16.md` (agent memory): 7 rules (ELI16 copy + codename maps + grounding intro + self-contained section titles with subtitles + expandable items + fixed label vocabulary + no action-implying language), 8 default substitutions, 4 calm status-color tokens, and the show-don't-tell iteration pattern. The Process Health tab is now the reference implementation.
+
+## Evidence pointers
+
+- All 46 tests green: 30 unit + 12 integration + 4 e2e — includes structural assertions for the unified label order, status-dot CSS classes, and that the dropped framing line is genuinely absent.
+- Safety verified end-to-end: XSS negative fixture still fires zero live elements + no canary; `detail.full` redaction unchanged.
+- Typecheck: `tsc --noEmit` clean.
+- Live-rendered both views (collapsed + expanded) to Chrome headless across 8 iterations and sent screenshots to the user for thumbs-up before each commit. Justin approved v8 with "this is good enough, let's go with it; save this as the standard for future dashboard features".