instar 1.3.106 → 1.3.108

package/upgrades/side-effects/compaction-busy-session-defer.md

@@ -0,0 +1,94 @@
+# Side-Effects Review — Compaction recovery: defer re-inject while the session is actively working
+
+**Version / slug:** `compaction-busy-session-defer`
+**Date:** `2026-05-29`
+**Author:** `echo`
+**Spec:** `docs/specs/compaction-busy-session-defer.md` (+ `.eli16.md`)
+**Convergence:** `docs/specs/reports/compaction-busy-session-defer-convergence.md` (fast-tracked — urgent fleet bug; see report)
+**Second-pass reviewer:** `not required` (see §"Phase 5 trigger check")
+
+## Summary of the change
+
+Fixes the user-reported false "session is restarting" loop (Justin, topic 15160):
+a live, actively-working session gets its inbound message buried under stacked
+compaction-recovery re-injects, so the message never reaches the session and the
+user sees repeated "restarting / starting up" narration.
+
+Root cause (verified in running code + `logs/server.log` on Echo): `CompactionSentinel`
+re-injects a recovery prompt when the session's JSONL doesn't grow within the
+verify window — but a long extended-think on a large context (typical right after
+resuming a near-full transcript) writes nothing to the JSONL until the turn
+lands, so a healthy session reads as "stuck." Caught firing 3× in ~52s against a
+live session.
+
+The change teaches the recovery surfaces to recognize a mid-turn session (the
+canonical `esc to interrupt` / `tokens · esc` footer OR a live non-baseline child
+process) and DEFER instead of re-injecting, bounded by `maxWorkingDefers`.
+
+## Files touched
+
+- `src/core/claudeActivityIndicators.ts` (**new**) — `CLAUDE_WORKING_INDICATORS`
+  single source of truth + pure `paneShowsClaudeWorking`.
+- `src/core/StuckInputSentinel.ts` — its `ACTIVITY_INDICATORS` now imports the
+  shared const (no behavior change; removes drift risk).
+- `src/core/SessionManager.ts` — `paneShowsActiveWork` + `isSessionActivelyWorking`;
+  `verifyInjection` skips the recovery Enter while the pane is actively working.
+- `src/monitoring/CompactionSentinel.ts` — `isActivelyWorking?` dep,
+  `maxWorkingDefers` config, `workingDefers` state, `deferForActiveWork()` +
+  `scheduleVerify()`, `compaction:deferred` event, `deferring` status.
+- `src/commands/server.ts` — wires `isActivelyWorking` to
+  `sessionManager.isSessionActivelyWorking`.
+- Tests: `tests/unit/CompactionSentinel.test.ts` (+6 cases),
+  `tests/unit/claudeActivityIndicators.test.ts` (new),
+  `tests/unit/session-active-work.test.ts` (new),
+  `tests/integration/compaction-busy-defer-wiring.test.ts` (new),
+  `tests/e2e/compaction-busy-defer-lifecycle.test.ts` (new).
+
+## Decision-point inventory
+
+- **Inject vs defer (CompactionSentinel)** — *modify, additive*. Was "no jsonl
+  growth → re-inject/fail." Now: "no growth AND actively working AND defer budget
+  remains → wait without injecting; else unchanged." A presence check, not a
+  judgment. Both branches test-covered. With the dep absent or
+  `maxWorkingDefers: 0`, behavior is byte-for-byte the old path.
+- **Recovery-Enter vs skip (verifyInjection)** — *modify, additive*. Was "marker
+  at prompt → fire Enter." Now: "marker at prompt AND actively working → skip
+  this tick, keep polling." Skipping is strictly less action; the input is
+  correctly queued by Claude Code and submits when the turn ends.
+- **Working signal** — *read-only*. Reuses the existing footer indicators +
+  `hasActiveProcesses`. No new file format, no write, no new tmux command.
+  `isSessionActivelyWorking` is wrapped so a capture/ps failure resolves to
+  `false` (cannot throw into recovery).
+
+## Blast radius
+
+- **No new authority, no new gate, no new detector, no new API route, no
+  external surface.** A defer behind an existing, already-rate-guarded recovery
+  primitive.
+- **Hot paths unchanged.** `getTopicForSession` and the message-inject hot path
+  are untouched. `isSessionActivelyWorking` runs only inside the recovery
+  lifecycle (already off the hot path).
+- **Strictly less aggressive.** The change can only PREVENT a re-inject/Enter; it
+  never adds one. It cannot starve a real recovery: an idle or wedged session
+  shows no working tell → no defer. Bounded by `maxWorkingDefers`.
+- **No agent-installed files changed** — no `.claude/settings.json` hooks, no
+  `.instar/config.json` defaults, no CLAUDE.md template, no hook scripts, no
+  skills. So **no PostUpdateMigrator entry is required**: pure `src/` logic that
+  reaches every agent through the normal dist update.
+
+## Phase 5 trigger check (second-pass reviewer)
+
+Second pass **not required**: no new authority, no destructive operation
+introduced (the only behavioral deltas REMOVE actions — a re-inject and a
+recovery Enter — against working sessions), no external surface, no migration.
+Additive, read-only signal behind an existing rate-guarded recovery primitive,
+with full three-tier coverage.
+
+## Verification
+
+- Unit: `CompactionSentinel.test.ts`, `claudeActivityIndicators.test.ts`,
+  `session-active-work.test.ts` (+ existing `StuckInputSentinel.test.ts` still
+  green after the shared-const refactor) — 125 pass across the related set.
+- Integration: `compaction-busy-defer-wiring.test.ts` — 4 pass.
+- E2E: `compaction-busy-defer-lifecycle.test.ts` — 4 pass (incl. WIRED guard).
+- `tsc --noEmit` clean.

package/upgrades/side-effects/failure-learning-sources-wiring-and-migration-parity.md

@@ -0,0 +1,127 @@
+# Side-effects review — Failure-Learning sources wiring + migration-parity assertion
+
+## What changed
+
+Two structural backstops shipped together, both targeting recurring bug classes
+named in the 2026-05-29 pipeline post-mortem.
+
+1. **Failure-Learning Loop: unimplemented-source warning + wiring-integrity test.**
+   `monitoring.failureLearning.sources.regression` and
+   `monitoring.failureLearning.sources.degradation` are config flags shipped
+   without implementations: setting them on did absolutely nothing. The flag
+   created a false sense of coverage — exactly the "specced but not wired"
+   class that produced PR #530 (AttributionResolver Phase 2 specced, never
+   wired, every burn alarm read 100% pre-attribution). A new boot-time
+   `console.warn` names the unimplemented sources, says they are silent
+   no-ops, and tells the operator to set them back off until impl ships.
+   A new unit-tier test asserts the CiFailurePoller and RevertDetector are
+   constructed iff their flags are on, and that the unimplemented-source
+   warning fires on the right configurations and stays quiet on the right
+   ones — so a future regression in the gating logic cannot silently
+   disable the loop.
+
+2. **Migration parity: a static assertion that fresh-init hook installs match
+   auto-update hook installs.** Per the "Migration parity skip" class
+   (telegram-reply.sh 403, autonomous-stop-hook broken path, the
+   slack-channel-context.sh divergence we found mid-PR in #542). The test
+   reads `installHooks()` (src/commands/init.ts) and `migrateHooks()`
+   (src/core/PostUpdateMigrator.ts), extracts the set of hook files each
+   writes, and asserts neither side has a file the other doesn't —
+   except for a small documented allowlist of migrator-only deferred-install
+   files. The allowlist's soft cap (10) trips a regression alarm if the
+   gap widens.
+
+## Why
+
+The May 2026 fix-rate to main was ~19% (150 of 781 commits over 14 days). The
+Failure-Learning Loop had captured exactly **one** event total — a manual
+agent-diagnosed entry I posted on the morning of the post-mortem itself. The
+loop's substrate (CiFailurePoller, RevertDetector, FailureLedger,
+FailureAttributionEngine, /failures routes) was already shipped (slice 1 in
+late April, slice 1b around #484), but every ingestion source was switched OFF
+in `ConfigDefaults.ts`. So the meta-trace lever was unpulled.
+
+I cannot flip a config default to a fleet-wide ON in a single PR — per-agent
+opt-in is the model the spec sets. What I CAN ship is the missing structural
+backstop: unimplemented sources must surface their gap visibly, and the
+wiring-integrity test makes any future regression in the gating impossible to
+miss. Plus the migration-parity test catches THE specific class that produced
+the silent failure that initiated this whole conversation (a fresh-init agent
+missing a hook the migrator installs, including telegram-topic-context.sh —
+the exact file behind the 2026-05-29 incoherent-reply incident).
+
+Echo's local config is being flipped to `sources.ci: true, sources.revert: true`
+out-of-band as a dogfooding step; that's an agent-local change, not part of
+this PR.
+
+## Risk surface
+
+- **Boot-time `console.warn` for unimplemented sources** — only fires when the
+  agent has explicitly turned on `sources.regression: true` or
+  `sources.degradation: ["..."]`. Default-off agents see nothing. The warning
+  text is plain-English and tells the operator the explicit fix (set them
+  back off). No state mutation; no API surface change.
+- **Wiring-integrity test** — pure unit test, uses an in-process mock
+  `SessionManager`. Doesn't construct an HTTP server, doesn't touch the
+  network, doesn't read git. Risk: zero.
+- **Migration-parity test** — static-analysis-only test. Parses
+  `src/commands/init.ts` and `src/core/PostUpdateMigrator.ts` source files
+  with a small regex (`fs.writeFileSync(path.join(hooksDir|instarHooksDir,
+  '<filename>'), …)`) and compares the resulting sets. Doesn't run either
+  function. Risk: zero.
+- **Allowlist semantics** — the test treats the allowlist as deliberate
+  accepted technical debt; growing it requires explicit code change + a
+  documented rationale per entry. The soft cap (10) protects against silent
+  drift.
+
+## Bug surfaces eliminated
+
+- `sources.regression: true` previously silently no-op'd → now warns loudly.
+- `sources.degradation: ["x"]` previously silently no-op'd → now warns loudly.
+- A future PR that adds a `migrateHooks()` write without an `installHooks()`
+  counterpart → caught at commit time by the parity test.
+- A future PR that adds an `installHooks()` write without a `migrateHooks()`
+  counterpart → caught at commit time by the parity test (this is the more
+  dangerous direction — the migrator overwrites on every cycle; a fresh-init
+  file the migrator doesn't maintain drifts forever).
+- A future regression that disables the CiFailurePoller construction when
+  `sources.ci: true` → caught by wiring-integrity test.
+
+## Migration footprint
+
+- No fleet migration required. The boot-warning is purely an in-process side
+  effect at AgentServer construction time; existing agents pick it up on the
+  next process restart (which auto-update will trigger anyway).
+- Default config defaults are unchanged. Per-agent opt-in remains the
+  intended model for the sources.
+
+## Testing
+
+- Unit (wiring): `tests/unit/failure-learning-sources-wiring.test.ts` —
+  9 tests. Constructed iff flag set (both directions), enabled:false
+  inert across all source flags, unimplemented sources trigger one
+  consolidated warning, implemented-only configs stay quiet.
+- Unit (migration parity): `tests/unit/migration-parity-hooks.test.ts` —
+  5 tests. Non-empty both sides, no install-only, no migrator-only outside
+  allowlist, allowlist soft cap. Positive + destructive-negative verified
+  (fake install-only write fails; fake migrator-only-without-allowlist
+  fails).
+
+## Test count
+
+14 new tests across the unit tier. Existing related tests
+(`CiFailurePoller.test.ts: 12`, `RevertDetector.test.ts: 9`,
+`PostUpdateMigrator-time-injection.test.ts: 13`) remain green.
+
+## Follow-ups (NOT in this PR — surfaced explicitly so the allowlist doesn't
+hide them indefinitely):
+
+- The `INSTALL_VS_MIGRATE_KNOWN_GAPS` allowlist enumerates 6 currently-accepted
+  migrator-only hooks. Each entry is technical debt that should be closed by
+  moving the file into `installHooks()` (or removing it from the migrator if
+  it's no longer needed). Tracked in the allowlist comments so the soft cap
+  surfaces drift.
+- The `regression` and `degradation` source IMPLEMENTATIONS still need to
+  ship. Those are separate features (the post-mortem's lever B —
+  real-world-state fixture tests — closely related). This PR's warning
+  closes the misleading-flag gap until then.

package/upgrades/side-effects/supervisor-cpu-starvation-restart-guard.md

@@ -0,0 +1,83 @@
+# Side-Effects Review — Supervisor CPU-starvation restart guard
+
+**Version / slug:** `supervisor-cpu-starvation-restart-guard`
+**Date:** `2026-05-29`
+**Author:** `echo`
+**Spec:** `docs/specs/supervisor-cpu-starvation-restart-guard.md` (+ `.eli16.md`)
+**Convergence:** `docs/specs/reports/supervisor-cpu-starvation-restart-guard-convergence.md` (fast-tracked — urgent fleet bug; see report)
+**Second-pass reviewer:** `not required` (see §"Phase 5 trigger check")
+
+## Summary of the change
+
+Fixes the overload-driven server restart loop that drops user messages (Justin,
+topic 15160: "Session restarting, message never lands"). Under CPU starvation
+the live server can't answer `/health`, so `ServerSupervisor` restarted it every
+~60s — but a fresh server is starved too, so it just dropped the in-flight
+message and looped. Verified in `logs/server.log`: 6 server restarts in ~35min
+during a load spike (load ratio ~2× cores).
+
+The guard makes the supervisor load-aware: while the box is CPU-starved
+(`loadavg[0]/cpuCount > 1.5`, the same signal SleepWakeDetector uses), an
+alive-but-unresponsive server's restart is DEFERRED (up to a 5-minute hard cap)
+instead of bounced.
+
+## Files touched
+
+- `src/core/cpuStarvation.ts` (**new**) — shared `cpuLoadRatio` / `isCpuStarved`
+  + `DEFAULT_MAX_LOAD_RATIO`.
+- `src/lifeline/ServerSupervisor.ts` — extracted the two identical
+  health-failure branches into `evaluateUnhealthyServer()`; added
+  `deferRestartForCpuStarvation()`, `maxLoadRatio`, `starvationRestartThreshold`,
+  and an injectable `loadRatioProvider`.
+- `tests/unit/cpu-starvation.test.ts` (new), `tests/unit/supervisor-cpu-starvation-defer.test.ts` (new).
+
+## Decision-point inventory
+
+- **Restart vs defer (alive + unresponsive)** — *modify, additive*. Was: restart
+  at `processAliveThreshold` (6 failures). Now: if CPU-starved and below the hard
+  cap, defer; else unchanged. A pure presence check on system load, not a
+  judgment. All branches covered by tests driving the real method.
+- **Branch extraction** — *refactor, behavior-preserving*. The two duplicated
+  failure paths (unhealthy `/health`, thrown check) now call one
+  `evaluateUnhealthyServer()`. The non-starved/dead/below-threshold behavior is
+  byte-for-byte the same; verified by the existing `supervisor-health-check`
+  test still passing + new real-method tests.
+- **Load read** — *read-only*. `os.loadavg()` / `os.cpus().length` via the shared
+  helper; returns 0 on error (fails toward "not starved" → restart, the safe
+  legacy behavior). No new file, no write, no new external surface.
+
+## Blast radius
+
+- **No new authority, no new gate, no new API route, no external surface.** A
+  defer behind the existing restart decision.
+- **Strictly less aggressive.** Can only DEFER a restart it would otherwise do,
+  only while genuinely CPU-starved, only up to a 5-min hard cap. Dead process →
+  immediate restart unchanged; not-starved → restart at 6 unchanged. Cannot
+  strand a hung server (hard cap) and the defer self-clears on the next healthy
+  tick.
+- **No agent-installed files changed** — pure `src/` (lifeline + a helper). No
+  `PostUpdateMigrator` entry required; the lifeline picks it up on its next
+  restart via the existing version-skew/drift coordination.
+
+## Pre-existing-bug note
+
+The two health-failure branches were identical copy-paste — a latent drift risk
+(a fix to one could miss the other). Extracting them into one method removes that
+hazard as a side benefit.
+
+## Phase 5 trigger check (second-pass reviewer)
+
+Second pass **not required**: no new authority, no new destructive operation (the
+only behavioral delta REMOVES a restart while starved), no external surface, no
+migration. Additive, read-only load signal behind an existing restart primitive,
+with real-method unit coverage + a wiring guard.
+
+## Verification
+
+- `tests/unit/cpu-starvation.test.ts` — pure helper (ratio + threshold).
+- `tests/unit/supervisor-cpu-starvation-defer.test.ts` — REAL
+  `evaluateUnhealthyServer()`: defer-while-starved, restart-when-not,
+  force-restart-past-cap, dead→immediate, below-threshold→wait,
+  defer-not-permanent + wiring guard. 19 pass with the existing
+  `supervisor-health-check` suite.
+- `tsc --noEmit` clean.