instar 1.1.1 → 1.1.3

package/upgrades/1.1.2.md

@@ -0,0 +1,130 @@
+# Upgrade Guide — vNEXT
+
+<!-- bump: patch -->
+<!-- patch = bug fixes, refactors, test additions, doc updates -->
+
+## What Changed
+
+**fix(boot-wrapper): plist↔wrapper extension drift no longer takes agents dark.**
+
+Three intertwined boot-time changes that together close a "agent goes
+silently dark and won't come back" failure mode observed in the field
+(Echo, 2026-05-20):
+
+1. **`installBootWrapper()` always writes `instar-boot.cjs`** (not `.js`),
+   regardless of the project's `package.json` `"type"` field. `.cjs`
+   forces CommonJS semantics in Node, so the wrapper's `require()` calls
+   work in both type=module and type=commonjs projects. Previously the
+   function picked an extension based on `package.json` AND deleted the
+   alt extension — so if a project gained `"type": "module"` after the
+   plist was generated (e.g., via a package upgrade), the next call to
+   `installBootWrapper` deleted the `.js` file the plist still pointed
+   at. launchd then exec'd a nonexistent file on every restart and none
+   of the downstream self-heal (`ServerSupervisor.preflightSelfHeal`,
+   `NativeModuleHealer`, `INSTAR_SUPERVISED` detection) ever ran because
+   the boot wrapper itself never loaded.
+
+2. **`TelegramLifeline.selfHealPlist` adds a wrapper-existence check.**
+   The existing three checks all passed when the plist referenced
+   `instar-boot.js` but the file was deleted (the plist content still
+   matched the legacy-detection patterns). A fourth check now extracts
+   the wrapper path from `ProgramArguments` and verifies the file
+   exists on disk; if not, the plist is regenerated immediately.
+   Defense-in-depth — the always-.cjs change above eliminates the
+   creation of this state going forward, but this check rescues any
+   already-existing drift before the next launchd restart hits the
+   bad path.
+
+3. **`WakeSocketServer.start()` recovers from EADDRINUSE.** The
+   pre-existing unlink-before-bind pass handled the common
+   "stale socket file from unclean exit" case but silently swallowed
+   unlink errors and offered no fallback if `listen()` itself fired
+   EADDRINUSE. On EADDRINUSE the server now probes the socket: if a
+   live peer answers, surface the error normally (don't clobber a real
+   second instance); if nothing answers, force-unlink and retry listen
+   once. Bounded retry — single attempt, no churn.
+
+4. **`PostUpdateMigrator.migrateBootWrapperToCjs`** — closes the gap
+   for in-the-wild agents whose launchd plists were generated before
+   this change with `instar-boot.js` in `ProgramArguments`. On darwin,
+   if the plist exists and references `instar-boot.js`, the migration
+   regenerates via `installAutoStart` so the plist now points at
+   `instar-boot.cjs`. Idempotent — re-runs that find a `.cjs`-referencing
+   plist are silent no-ops. The migration does NOT delete the legacy
+   `.js` file; rollback safety wins over file cleanup.
+
+Files touched (excluding tests / docs / upgrade notes):
+- `src/commands/setup.ts` — `installBootWrapper`, `ensureBootWrapper`
+- `src/commands/server.ts` — auto-start plist sanity check accepts `.cjs`
+- `src/lifeline/TelegramLifeline.ts` — selfHealPlist 4th check
+- `src/threadline/WakeSocketServer.ts` — EADDRINUSE recovery
+- `src/core/PostUpdateMigrator.ts` — `migrateBootWrapperToCjs`
+
+## Evidence
+
+**Repro of the original failure** (Echo, 2026-05-20, 10:25 PT):
+
+1. Echo's `package.json` has `"type": "module"` (instar's own package
+   has had this since v1.0.0).
+2. Earlier `installBootWrapper` calls picked `.cjs`, deleted `.js`, and
+   the plist was generated at that point referencing `.cjs`.
+3. A subsequent code path (`installBootWrapper` running while the boot
+   was actively crashing for a different reason — Node major-version
+   ABI break, since rolled back) wrote `.cjs` AGAIN and deleted any
+   `.js` that had been re-created by an even-older code path.
+4. The plist on disk still referenced `.cjs`, but ALSO any plist that
+   had been generated when echo's `package.json` was missing the
+   `type: module` declaration referenced `.js` — and the `.js` file
+   was gone.
+5. launchd's next restart exec'd the nonexistent path → process never
+   started → all downstream self-heal (`NativeModuleHealer.healBetterSqlite3`,
+   `ServerSupervisor.preflightSelfHeal`, bind-failure escalation)
+   silently never ran.
+6. Operator intervention (Dawn, manual plist edit) was required to
+   restore the agent.
+
+**Observed before:** `git log` shows multiple in-the-wild reports of
+launchd plists pointing at a deleted `instar-boot.js` after a project
+gained `"type": "module"`. The selfHealPlist check at
+`src/lifeline/TelegramLifeline.ts:2076` reads `content.includes('instar-boot.js')`,
+which returns `true` even after the file is deleted — so the existing
+checks all pass and no regen fires.
+
+**Observed after:**
+- New test `tests/unit/boot-wrapper-plist-coherence.test.ts` exercises
+  `installBootWrapper` with package.json `type: module`, type: commonjs,
+  and no package.json — all three write `instar-boot.cjs`. A test
+  explicitly plants a pre-existing `instar-boot.js` and verifies it is
+  NOT deleted by the new code path (rollback safety).
+- New test `tests/unit/PostUpdateMigrator-bootWrapperCjs.test.ts`
+  exercises the four migration branches (non-darwin skip, no plist
+  skip, already-.cjs idempotent skip, and the `.js → .cjs` detection +
+  regeneration path).
+- New test `tests/unit/wake-socket-server-stale-recovery.test.ts`
+  exercises the stale-socket cleanup path and asserts that a live peer
+  is NOT clobbered (the recovery surfaces EADDRINUSE in that case).
+- Existing test suites unchanged; `installBootWrapper` callers in
+  `installMacOSLaunchAgent` and `TelegramLifeline.selfHealPlist`
+  receive the new `.cjs` path through the unchanged `wrappers.js`
+  return field so no callsite needs adjustment.
+
+## What to Tell Your User
+
+- **Your agent stays online through package upgrades**: if I had an
+  internal package configuration change, my launchd entry point used
+  to risk getting renamed out from under itself. That whole class of
+  failure is closed off now — the boot file has a stable name that
+  works regardless of internal toggles.
+- **Stale socket files don't lock me out anymore**: if I exited
+  uncleanly and a leftover socket file blocked the next start, I now
+  detect that and clean it up safely, only after first checking that
+  nothing else is actually using it.
+
+## Summary of New Capabilities
+
+| Capability | How to Use |
+|-----------|-----------|
+| Plist↔wrapper coherence guarantee | automatic (no action required) |
+| selfHealPlist verifies wrapper-path-on-disk | automatic (runs in lifeline boot) |
+| WakeSocketServer EADDRINUSE recovery | automatic (runs in listener startup) |
+| PostUpdateMigrator regenerates `.js`-plists | automatic on next instar update |

package/upgrades/1.1.3.md

@@ -0,0 +1,124 @@
+# Upgrade Guide — vNEXT
+
+<!-- bump: patch -->
+<!-- patch = bug fixes, refactors, test additions, doc updates -->
+
+## What Changed
+
+**fix(lifeline): close the silent-drop version-skew failure class.**
+
+Five interlocking fixes that together address the 2026-05-19 →
+2026-05-20 b2lead-insights field incident (21h silent Telegram ingress
+drop after a server auto-update from v1.0.13 → v1.1.0 left the
+lifeline on v1.0.13; server's `/internal/telegram-forward` rejected
+the lifeline with HTTP 426 and the lifeline's drop-after-3-failures
+replay policy silently lost user messages).
+
+1. **`rateLimitState.decide` — `versionSkew` bucket bypasses
+   cooldown.** A hard incompatibility cannot be cured by waiting; the
+   per-minute cooldown wedged every restart attempt. The daily cap
+   (3 per 24h) and storm detection remain as backstops.
+
+2. **Replay-loop drop policy is gated on a version-skew episode
+   flag.** When `forwardToServer` raises `ForwardVersionSkewError`,
+   the lifeline sets `versionSkewActive = true` and sends one
+   user-visible Telegram alert ("ingress paused: version skew
+   detected, your messages are not lost"). The replay loop checks
+   the flag BEFORE the `MAX_REPLAY_FAILURES` drop check and re-queues
+   without incrementing the failure counter. Cleared on the next
+   successful forward.
+
+3. **`instar lifeline restart` CLI uses the correct service label.**
+   Was `com.instar.<projectName>.lifeline` — different domain AND
+   different suffix from what `installMacOSLaunchAgent` writes
+   (`ai.instar.<projectName>`). `launchctl kickstart` always failed,
+   silently fell back to pkill.
+
+4. **pkill fallback escalates to SIGKILL; lock-acquire recognizes
+   wedged sleeping processes.** SIGTERM, 3s grace, SIGKILL by name
+   pattern in the CLI. In `acquireLockFile`, 'S' (sleeping) state
+   with lock-write age > 5 min is treated as a wedged lifeline:
+   SIGTERM → 3s poll → SIGKILL → take the lock.
+
+5. **`npm rebuild` for better-sqlite3 uses `--build-from-source` and
+   `--ignore-scripts`.** Two call sites: `ServerSupervisor`
+   preflight rebuild and `NativeModuleHealer.healBetterSqlite3Sync`.
+   Without `--build-from-source`, npm can install a cached prebuilt
+   with the same wrong ABI and exit 0 — producing the "rebuild
+   succeeded but module still fails to load" pattern in the b2lead
+   logs.
+
+Files touched (excluding tests / docs / upgrade notes):
+- `src/lifeline/rateLimitState.ts` — `decide()`
+- `src/lifeline/TelegramLifeline.ts` — instance state,
+  `handleVersionSkew`, `forwardToServer`, `replayQueue`,
+  `acquireLockFile`
+- `src/cli.ts` — `lifelineCmd.command('restart')`
+- `src/lifeline/ServerSupervisor.ts` — preflight rebuild argv
+- `src/memory/NativeModuleHealer.ts` — in-line rebuild argv
+
+## Evidence
+
+**Repro of the original failure** (b2lead-insights, 2026-05-19 evening
+→ 2026-05-20 ~17:50 UTC):
+
+1. Server auto-updated v1.0.13 → v1.1.0.
+2. Lifeline kept running v1.0.13.
+3. User Telegram messages → lifeline forwards to server → server
+   returns HTTP 426 with `{ok: false, upgradeRequired: true,
+   serverVersion: '1.1.0', action: 'restart', reason: 'major-minor-mismatch'}`.
+4. Lifeline catches `ForwardVersionSkewError`, calls
+   `handleVersionSkew` → `initiateRestart('versionSkew', ...)`.
+5. `RestartOrchestrator.maybeRestart` consults `rateLimitState.decide`.
+   `elapsed < WATCHDOG_COOLDOWN_MS` → returns `{allowed: false, reason:
+   'cooldown-active'}`. No restart.
+6. Lifeline returns false from `forwardToServer`. Replay loop
+   increments `replayFailures`. After 3 attempts → drop.
+7. 40+ "server-down" notifications + at least 2 explicitly-logged
+   dropped messages (`tg-6577`, `tg-6580`). No user-visible alert.
+
+**Observed before:** `rateLimitState.ts:112` had no special-case for
+`versionSkew` — the cooldown check fired for all buckets. The replay
+loop at `TelegramLifeline.ts:1394` checked `failures >=
+MAX_REPLAY_FAILURES` unconditionally. The CLI restart command at
+`cli.ts:1639` hardcoded `com.instar.${projectName}.lifeline`.
+`ServerSupervisor.ts:751` and `NativeModuleHealer.ts:397` both ran
+`npm rebuild better-sqlite3` without `--build-from-source`.
+
+**Observed after:**
+- 26 new + extended unit tests (8 new in
+  `tests/unit/lifeline/version-skew-recovery.test.ts`, 1 new in
+  `tests/unit/lifeline/rateLimitState.test.ts`) cover all five paths.
+- Source assertions verify the supervisor rebuild argv includes
+  `--build-from-source` AND `--ignore-scripts`, the healer in-line
+  rebuild includes both flags, the CLI restart command builds
+  `ai.instar.${...}` and never `com.instar.${...}.lifeline`, the
+  pkill fallback contains both `-TERM` and `-KILL`, the
+  `versionSkewActive` flag is set in `handleVersionSkew`, cleared in
+  `forwardToServer` success, checked in `replayQueue` BEFORE the
+  drop branch, and the lock-acquire path recognizes wedged-S.
+
+Spec: `docs/specs/lifeline-version-skew-recovery.md`
+ELI16: `docs/specs/lifeline-version-skew-recovery.eli16.md`
+Side-effects review: `upgrades/side-effects/lifeline-version-skew-recovery.md`
+
+## What to Tell Your User
+
+- **You will never silently lose a message during an auto-update
+  again**: if my server and my Telegram lifeline temporarily get out
+  of sync after an update, I'll send you one clear note that ingress
+  is paused, keep all your messages safely in the queue, and replay
+  them as soon as the lifeline catches up.
+- **Recovery is automatic**: the bug class that needed a manual
+  restart to clear is now self-healing. No operator intervention.
+
+## Summary of New Capabilities
+
+| Capability | How to Use |
+|-----------|-----------|
+| Version-skew cooldown bypass | automatic |
+| Version-skew user alert (one per episode) | automatic |
+| No-drop policy during version-skew episodes | automatic |
+| CLI lifeline restart resolves correct service | automatic via `instar lifeline restart` |
+| Wedged-lifeline lock recovery | automatic |
+| Force-from-source native rebuilds | automatic |

package/upgrades/side-effects/boot-wrapper-plist-coherence.md

@@ -0,0 +1,176 @@
+# Side-Effects Review — Boot-Wrapper / Plist Coherence
+
+**Version / slug:** `boot-wrapper-plist-coherence`
+**Date:** `2026-05-20`
+**Author:** Echo (instar developer agent)
+**Trigger:** Field failure 2026-05-20, ~10:25 PT: instar agent "echo"
+went dark, required operator intervention (Dawn — manual plist edit) to
+recover. Root cause: launchd plist's `ProgramArguments` referenced
+`instar-boot.js` but that file had been deleted by `installBootWrapper`
+because `package.json` had gained `"type": "module"`.
+
+## Summary of the change
+
+Four interlocking edits that together eliminate the plist↔wrapper
+extension-drift failure class:
+
+1. **`installBootWrapper()` always writes `.cjs`.** No more reading
+   `package.json "type"` and switching extensions. No more deleting the
+   alt extension.
+2. **`ensureBootWrapper()` looks for `.cjs`.** The presence check used
+   to look for the type-dependent extension; now it always looks for
+   `.cjs`.
+3. **`TelegramLifeline.selfHealPlist` adds Check 4:** verify the wrapper
+   path referenced by the plist's `ProgramArguments` exists on disk. If
+   not, regenerate via `installAutoStart`. Defense-in-depth for any
+   already-drifted state.
+4. **`PostUpdateMigrator.migrateBootWrapperToCjs`** detects existing
+   `.js`-referencing plists and regenerates them to point at `.cjs` via
+   `installAutoStart`. Idempotent.
+5. **`WakeSocketServer.start()`** uses a probe-before-clobber pattern.
+   Previously: pre-bind unconditional `unlink`. New: listen first; on
+   EADDRINUSE, probe-connect to determine if the socket is live or
+   stale; only unlink+retry if confirmed stale. Prevents silently
+   stealing a path from a live peer.
+
+## Decision matrix
+
+### Over-block
+
+| Scenario | Outcome |
+|----------|---------|
+| Project has package.json `"type": "module"` | Always-`.cjs` is correct (no behavioral change vs. pre-existing `.cjs`-write path) |
+| Project has no package.json | Always-`.cjs` works (Node treats `.cjs` as CommonJS regardless of parent context) |
+| Project has package.json `"type": "commonjs"` | Always-`.cjs` works (CommonJS is the explicit context) |
+| Existing `.js` wrapper on disk | NOT touched (rollback-safe — old plists referencing `.js` keep working until migrator runs) |
+| Existing `.cjs` wrapper on disk | Overwritten with current content (preserves existing behavior) |
+
+No over-block. The `.cjs` extension is strictly broader than the previous `.js`+`.cjs` split.
+
+### Under-block
+
+| Scenario | Outcome |
+|----------|---------|
+| Plist references `.js`, no `.js` file on disk, no `.cjs` file on disk | Migrator regenerates; selfHealPlist Check 4 catches | 
+| Plist references `.cjs`, `.cjs` file deleted | selfHealPlist Check 4 catches (regenerates) |
+| Plist references `.js`, `.js` file present, `.cjs` file present | Migrator regenerates plist to `.cjs` (idempotent on re-run) |
+| Plist references `.js`, `.js` file present, `.cjs` file missing | Migrator regenerates plist (writes `.cjs` + updates plist) |
+
+No under-block.
+
+### Level of abstraction fit
+
+- The fix lives at the WRAPPER GENERATION + WRAPPER VERIFICATION layer,
+  which is the same layer where the bug originated. Not pushed up into
+  the Remediator (out of scope for this PR — Remediator is still under
+  Tier 2 rollout per the v3 spec) and not pushed down into launchd (no
+  way to influence launchd's interpretation of the plist).
+- `installBootWrapper`'s contract is unchanged from a caller's
+  perspective: it still returns `{ sh, js }`. The `js` field name is now
+  a slight misnomer (the file is `.cjs`) but every existing caller uses
+  it as an opaque path string, so no callsite changes.
+
+### Signal-vs-authority compliance
+
+| Component | Signal or Authority | Reason |
+|-----------|--------------------|--------|
+| `installBootWrapper` always-`.cjs` | Authority (writes files) | Deterministic, content-based decision. No judgment call. |
+| `ensureBootWrapper` `.cjs` lookup | Signal (returns boolean) | Caller decides what to do with "wrappers missing". |
+| `selfHealPlist` Check 4 | Authority (writes files via installAutoStart) | Existence check is deterministic. The regeneration is a documented recovery primitive. |
+| `migrateBootWrapperToCjs` | Authority (writes files via installAutoStart) | Detection is structural (plist content matches regex). Regeneration is the same primitive other migrations use. |
+| `WakeSocketServer` probe-then-recover | Signal-and-Authority hybrid | Probe is a signal (connect-or-not); the unlink+retry is bounded recovery (single retry, hard timeout). |
+
+Compliant. The new authorities are bounded recovery primitives, not
+judgmental gates. The new signals are deterministic structural checks.
+
+### Interactions with adjacent systems
+
+| System | Interaction | Risk |
+|--------|-------------|------|
+| `NativeModuleHealer` | None direct. NativeModuleHealer fires after boot wrapper loads; this fix ensures the boot wrapper CAN load. | Strictly improves NativeModuleHealer's blast radius. |
+| `ServerSupervisor.preflightSelfHeal` | None direct. Preflight runs after boot wrapper spawns the lifeline. | Same — strictly improves preflight's coverage. |
+| `detectLaunchdSupervised` | None. Runs inside the lifeline; the fix is upstream. | None. |
+| `INSTAR_SUPERVISED` plist env | Compatible — `installMacOSLaunchAgent` regenerates the full plist including this env var. | None. |
+| `RestartOrchestrator` | Same as above — fix is upstream. | None. |
+| `DegradationReporter` | None direct. Migrator failures land in `result.errors[]` which feed the migration report, not Degradation. Acceptable: migrator runs in a known supervised window, not in production hot paths. | None. |
+| `WakeSocketServer` listener daemon usage | The probe-before-clobber change only fires on EADDRINUSE. Normal startup path (no stale file) is unchanged. | Tested with planted-stale-file + live-peer scenarios. |
+| Existing `.js`-wrapper-using agents | Continue to work — `.js` file is NOT deleted; old plists still load; migrator regenerates plist on next update. | Backward compatible by design. |
+| Tests that simulate `instar-boot.js` (e2e/launchd-*) | These tests generate their OWN `.js` wrappers and do NOT go through `installBootWrapper`. Unchanged. | None — verified by reading the test files. |
+
+### Rollback cost
+
+- Pure code change. Revert is one git revert per affected file.
+- No persistent state changes that block rollback. The migrator runs
+  forward-only; rolling back leaves agents on `.cjs` plists (which work
+  fine) — the old `.js` files are still around if needed.
+- `WakeSocketServer` change: revert restores pre-bind-unlink behavior.
+  The live-peer-clobber behavior re-emerges but is rare in practice
+  (single-tenant agents on a machine).
+- Migrator runs idempotently — rolling back and re-deploying produces
+  the same end state.
+
+## Acceptance criteria
+
+1. **Repro of the Echo failure mode.**
+   - Test: `tests/unit/boot-wrapper-plist-coherence.test.ts > installBootWrapper — always writes .cjs > does NOT delete a pre-existing instar-boot.js (rollback safety)`
+   - Plants a pre-existing `instar-boot.js`, runs `installBootWrapper` with `package.json type=module`. Asserts the `.js` file still exists. Pre-change behavior: file deleted. Post-change: file preserved.
+
+2. **selfHealPlist catches drift.**
+   - Implicit in `TelegramLifeline.ts` source: `Check 4` extracts the wrapper path from `ProgramArguments` and verifies `fs.existsSync`. Covered by `Check 4` logic.
+
+3. **Migrator handles existing `.js`-referencing plists.**
+   - Test: `tests/unit/PostUpdateMigrator-bootWrapperCjs.test.ts > detects a plist that references instar-boot.js and attempts regeneration`
+   - Plants a plist referencing `instar-boot.js`. Runs the migrator. Asserts the migrator either succeeds (`upgraded`) or surfaces an error (`errors`) — both prove the detection-and-attempt path ran.
+
+4. **WakeSocketServer does NOT clobber live peers.**
+   - Test: `tests/unit/wake-socket-server-stale-recovery.test.ts > refuses to clobber a live peer (surfaces EADDRINUSE)`
+   - Binds a real `net.Server` to the socket path, then attempts to start a second `WakeSocketServer`. Asserts the second emits EADDRINUSE without taking the path away from the live peer.
+
+5. **WakeSocketServer recovers from stale socket.**
+   - Test: `tests/unit/wake-socket-server-stale-recovery.test.ts > cleans up a stale socket file on start (pre-bind unlink path)` and `> recovers from EADDRINUSE when no live peer is listening`
+   - Plants an empty file at the socket path, starts the server, asserts the listen succeeds.
+
+## Notes for second-pass reviewer
+
+- **Why always `.cjs` and not "always write both `.js` and `.cjs`?":**
+  Considered. Always-both would allow rollback to point at `.js` if
+  needed. Rejected because: (1) `.js` in a `type=module` project is a
+  broken artifact (Node can't load it as CJS); (2) the migrator
+  regenerates plists to `.cjs` automatically, so rollback isn't a
+  user-visible scenario; (3) keeping an unused broken file invites
+  confusion. The current code leaves `.js` files alone but doesn't
+  create new ones.
+
+- **Why not run the migrator on every lifeline boot instead of relying
+  on `PostUpdateMigrator`?** `selfHealPlist` already does the
+  equivalent on every lifeline boot — it's Check 4 of that function.
+  The migrator entry catches the case where the agent installs the new
+  instar version but hasn't yet booted the lifeline (e.g., the
+  postupdate migration runs first).
+
+- **Why probe-then-unlink instead of just `listen()`-then-error-bubble
+  on EADDRINUSE?** Without the recovery path, a stale socket file from
+  an unclean exit blocks startup forever until the operator manually
+  removes it. The probe-then-unlink preserves correctness (don't
+  clobber live peers) while restoring the auto-recovery the original
+  pre-bind-unlink path provided.
+
+- **Migrator side-effect on darwin only:** Linux systemd unit files
+  reference the bash wrapper (`instar-boot.sh`) which is unaffected
+  by this change. Non-darwin migrator path is a no-op skip.
+
+## Rollback playbook
+
+If this PR causes regressions:
+
+1. `git revert <pr-merge-sha>` on the canonical instar repo's `main`.
+2. Tag a patch release (e.g., `v1.1.2`) with the revert.
+3. `npx instar` auto-update propagates the revert to all in-the-wild
+   agents on next supervision cycle.
+4. Agents whose plists were already migrated to `.cjs` continue to
+   work — `installBootWrapper` post-revert writes `.cjs` for projects
+   with `type=module`, which matches what the migrated plists reference.
+5. Agents whose plists were NOT yet migrated (still on `.js`)
+   continue on `.js` — unchanged from pre-PR state.
+
+No persistent state cleanup required.

package/upgrades/side-effects/lifeline-version-skew-recovery.md

@@ -0,0 +1,180 @@
+# Side-Effects Review — Lifeline version-skew recovery
+
+**Version / slug:** `lifeline-version-skew-recovery`
+**Date:** `2026-05-20`
+**Author:** Echo (instar developer agent)
+**Trigger:** 2026-05-19 → 2026-05-20 b2lead-insights field incident (21h silent
+Telegram ingress drop after server auto-update from v1.0.13 → v1.1.0;
+lifeline stayed on v1.0.13; HTTP 426 from `/internal/telegram-forward`
+went to 3-attempts-then-drop replay policy; restart attempts blocked by
+cooldown; manual operator recovery required).
+
+## Summary of the change
+
+Five interlocking fixes that together close the silent-drop failure
+class. All independently revertable.
+
+1. **Rate-limit cooldown bypass for `versionSkew` bucket.** A
+   no-op-against-mismatch wait is replaced with allow-but-rate-limit
+   (daily cap still applies). One-line condition change in
+   `decide()`.
+
+2. **Drop policy guards on `versionSkewActive` instance flag.** Three
+   new state operations:
+   - `handleVersionSkew` sets `versionSkewActive = true` and fires
+     one user-visible alert (dedupe via `versionSkewAlertSentAt`).
+   - `forwardToServer` success clears the flag + dedupe timestamp.
+   - `replayQueue` re-queues without incrementing failures while
+     the flag is set.
+
+3. **CLI service label fix.** `com.instar.<name>.lifeline` →
+   `ai.instar.<name>`. Matches the plist label written by
+   `installMacOSLaunchAgent`. Stops the always-fall-back-to-pkill path.
+
+4. **pkill fallback escalates + lock-acquire recognizes wedged-S.**
+   CLI: SIGTERM, 3s grace, SIGKILL by pattern. Lock-acquire: detects
+   sleeping ('S') state with lock-write age > 5 min as a wedged
+   lifeline, sends its own SIGTERM → 3s poll → SIGKILL sequence.
+
+5. **`--build-from-source` + `--ignore-scripts` on every native
+   rebuild.** Two call sites: ServerSupervisor preflight rebuild
+   (line 751) and NativeModuleHealer.healBetterSqlite3Sync
+   (line 397). The Remediator-orchestrated path already had these
+   flags (NativeModuleHealer.healBetterSqlite3FromRemediator).
+
+Files touched (excluding tests / docs):
+- `src/lifeline/rateLimitState.ts` — `decide()` cooldown bypass
+- `src/lifeline/TelegramLifeline.ts` — instance state, `handleVersionSkew`,
+  `forwardToServer`, `replayQueue`, `acquireLockFile`
+- `src/cli.ts` — label fix + pkill escalation
+- `src/lifeline/ServerSupervisor.ts` — rebuild flags
+- `src/memory/NativeModuleHealer.ts` — rebuild flags
+
+## Decision matrix
+
+### Over-block
+
+| Scenario | Outcome |
+|----------|---------|
+| Lifeline at SAME version as server (no skew) | No change — `versionSkewActive` stays false |
+| Lifeline at OLD version, server unchanged (no auto-update) | No change — no 426, no skew |
+| Genuine transient 5xx | Drop-after-3 still fires (versionSkewActive is false) |
+| `ForwardServerBootError` (server starting) | Drop-after-3 still fires — known limitation; this PR doesn't try to also branch on serverBoot. Out of scope. Justified: serverBoot is short-lived (seconds), version skew is steady-state until restart. |
+| Lifeline restart loop after fix | Daily cap (3-per-24h) plus storm detection bound the loop |
+
+No over-block. The new authority only fires when ALL of:
+1. The forward threw `ForwardVersionSkewError`,
+2. The server version in the body differs from the lifeline's,
+3. The body's `upgradeRequired === true`.
+
+That's the same gate that already triggers the restart request, so we
+haven't widened the surface — just made the consequences during the
+window correct.
+
+### Under-block
+
+| Scenario | Outcome |
+|----------|---------|
+| Lifeline version "skews" because of a malformed 426 body | Guarded by existing `body.upgradeRequired !== true` and `serverVersion === this.lifelineVersion` checks — they were already in `handleVersionSkew` |
+| Lifeline tries to forge a skew to bypass cooldown | The 426 must come from the SERVER (loopback addr only on a privileged port); the lifeline cannot self-trigger the body shape |
+| Two version-skew episodes in <24h (daily cap reached) | Falls through to existing `version-skew-daily-cap` denial — observability path remains intact |
+| Stuck-lock detection: live healthy lifeline misidentified as wedged | Requires lock-age > 5 min AND 'S' state — a healthy lifeline writes the lock, immediately starts polling Telegram (lots of 'R' time), and answers heartbeats. Misidentification window is narrow but possible during long Telegram long-poll waits. Acceptable: false positive triggers SIGTERM → 3s grace → SIGKILL of a healthy lifeline → launchd respawns it within seconds. Worst case: ~5s downtime + a fresh boot. Far better than the 21h+ silent drop the current code permits. |
+| Forced rebuild with `--build-from-source` fails because node-gyp toolchain absent | Already-existing failure path: spawnSync returns non-zero, healer logs error, falls back to existing escalation. No new silent failure. |
+
+### Level of abstraction fit
+
+- Rate-limit bypass — at the decision-point that already knows about
+  buckets. Right layer.
+- Drop policy — at the replay loop that already has the failure-counter
+  state. Right layer. Could have lived in `forwardToServer` instead,
+  but the queue is what owns "should this message be dropped" — the
+  forward path only owns "did this single attempt succeed".
+- CLI label — at the kickstart callsite. One source of truth.
+- pkill escalation — at the CLI fallback. The CLI is the place where
+  "user typed `instar lifeline restart`" gets translated to a system
+  action.
+- Lock-acquire — at the lock-acquire function. The detector and the
+  recovery primitive are both bounded mechanics in the same scope.
+- Rebuild flags — at the rebuild callsites. No new abstraction.
+
+No over-engineering. Each change is at the layer that already owns
+the concern.
+
+### Signal-vs-authority compliance
+
+- `rateLimitState.decide` — returns signal. No authority change.
+- `versionSkewActive` flag — pure state signal. Replay loop is the
+  authority that decides re-queue vs drop.
+- `handleVersionSkew` — already had authority over "request a restart".
+  Now also: sets the flag and sends an alert. Both are bounded
+  mechanics with clear preconditions, not judgmental gates.
+- CLI label — mechanical change.
+- pkill escalation — bounded recovery (fixed 3s grace, single SIGKILL).
+- Lock-acquire stuck-S — bounded recovery (fixed 5min threshold for
+  age, 3s SIGTERM grace, single SIGKILL).
+- Rebuild flags — mechanical change.
+
+Compliant. No new judgmental gates; new authorities are bounded
+recovery primitives.
+
+### Interactions with adjacent systems
+
+| System | Interaction | Risk |
+|--------|-------------|------|
+| `MessageQueue` | Replay loop re-queues messages during skew (same enqueue path used for transient failures) | None — queue persistence semantics unchanged |
+| `notifyMessageDropped` | NOT called during version-skew episodes (drop is suppressed) | Strictly improves — these messages will deliver |
+| `DegradationReporter` | Still fires for non-skew drops; version-skew episode itself emits a degradation via the existing path | None |
+| `RestartOrchestrator` | Receives a `versionSkew` request that's now more likely to be granted (cooldown bypass) | Storm detection + daily cap remain as backstops |
+| `ServerSupervisor.preflightSelfHeal` | Rebuild flag change is a strict improvement | None |
+| Remediator (Tier 1+) | Not yet shipped; in-line healer is the steady-state path. When Remediator ships, its rebuild already used `--build-from-source` — no conflict | None |
+| `NativeModuleHealer.invokeFromRemediator` | Unchanged (already had the flag) | None |
+| `LifelineHealthWatchdog` | Tracks `consecutiveForwardFailures`; this PR clears it on success (existing behavior). No change to the watchdog's trip conditions | None |
+| `WatchdogTriggers.versionSkew` | The flag-set + alert-send happens BEFORE `initiateRestart`. If the restart is rate-limited, the alert still fires — good (user knows ingress is paused even while the lifeline is being patient about restarts) | None |
+| `acquireLockFile` callers | The added SIGTERM-then-SIGKILL path runs in the boot-startup window. New lifelines that find a wedged old one will spend up to 3s waiting for graceful exit. Acceptable | None |
+| Existing tests | All 18 `rateLimitState.test.ts` + 14 `lifeline-shadow-install-self-heal.test.ts` + 9 `NativeModuleHealer.test.ts` tests pass unchanged | Verified |
+
+### Rollback cost
+
+- Each fix is independently revertable.
+- No persistent state schema changes; the new lifeline instance
+  fields (`versionSkewActive`, `versionSkewAlertSentAt`) are not
+  serialized to disk.
+- No migration entry; existing in-the-wild agents pick up the fix on
+  their next `npx instar` update like any other patch.
+- Reverting the rebuild flag returns to the pre-existing flaky
+  rebuild behavior — agents that had drifted to "rebuild succeeded
+  but still fails" on the old code path continue exactly as they
+  did.
+
+## Acceptance criteria (test-mapped)
+
+1. `tests/unit/lifeline/rateLimitState.test.ts > versionSkew bucket
+   BYPASSES the cooldown` — asserts the new bypass.
+2. `tests/unit/lifeline/version-skew-recovery.test.ts > CLI service
+   label` — asserts `ai.instar.${...}` is in source and
+   `com.instar.${...}.lifeline` is gone.
+3. `tests/unit/lifeline/version-skew-recovery.test.ts > pkill
+   fallback escalates to SIGKILL` — asserts both signal calls
+   present in the restart command source.
+4. `tests/unit/lifeline/version-skew-recovery.test.ts >
+   ServerSupervisor preflight rebuild passes --build-from-source` —
+   asserts the rebuild argv on the supervisor side.
+5. `tests/unit/lifeline/version-skew-recovery.test.ts >
+   NativeModuleHealer in-line rebuild passes --build-from-source` —
+   asserts EVERY rebuild call in the healer source has the flag.
+6. `tests/unit/lifeline/version-skew-recovery.test.ts > lifeline
+   source guards drop with versionSkewActive` — asserts the source
+   order (skew bypass before drop check).
+7. `tests/unit/lifeline/version-skew-recovery.test.ts >
+   handleVersionSkew sets the active flag + alert dedupe`.
+8. `tests/unit/lifeline/version-skew-recovery.test.ts >
+   forwardToServer success clears the version-skew episode flag`.
+9. `tests/unit/lifeline/version-skew-recovery.test.ts > lock-acquire
+   treats sleeping (S) state > 5 min as recoverable`.
+
+## Forward note
+
+If the auto-update flow eventually grows a "tell the lifeline to
+restart" step, the failure class this PR addresses becomes
+impossible-by-construction. Until then, this is the steady-state
+recovery for the window between server-update and lifeline-restart.