instar 1.3.1302 → 1.3.1304

package/upgrades/1.3.1304.md

@@ -0,0 +1,89 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+An investigation into six `claude -p` builders SIGKILLed on the Mac Studio
+(2026-09-29/30) did not identify their cause. It did find automatic kill
+paths that picked their target by name, keyword or bare pid. None of those
+proves the target is the agent's own. Each path now proves ownership first,
+and when it cannot, it does not signal:
+
+- New `src/core/processIdentity.ts`: a recorded pid counts as the same process
+  only while its start time (`ps -o lstart`) matches the recorded one.
+- `instar lifeline restart` fallback (used when `launchctl kickstart` fails):
+  `lifeline.lock` and the startup marker now record `procStart`. The fallback
+  signals the lock holder and the marker pid only when each is proven to be
+  that same process, and it proves this again before SIGKILL. The old
+  `pkill -f '<agent>.*lifeline'` and the round-1 "command contains lifeline"
+  check are both gone. Legacy records without a start time are never
+  signalled.
+- Lifeline startup takeover (`acquireLockFile`, now in
+  `src/lifeline/lifelineLock.ts`): a stuck holder is terminated only when
+  proven. A pid that started after the lock was written counts as reused, and
+  the lock is taken without signalling anyone. If ownership is unproven, the
+  lock is respected. That includes a start time `ps` cannot read: the holder
+  counts as dead only when the kernel confirms (ESRCH) nothing runs there.
+- `PipeSessionSpawner`: records tmux `#{session_id}` + pane pid at creation. It
+  kills only that exact session, by id, and never a same-name replacement or a
+  stale pid's group. It signals the pane's process group only while tmux
+  reports the pane live; a retained dead pane (`remain-on-exit`) gets only its
+  session removed. It refuses to spawn over a `pipe-<threadId>` session it
+  did not start.
+- `SessionManager.spawnTriageSession`: it replaces an existing session under
+  the triage name only when `ownsLiveTmuxSession()` proves that session is its
+  own. Proof means the session's `INSTAR_SESSION_ID` names this agent's record
+  for that name.
+- `OrphanProcessReaper`: auto-kill requires `ownsLiveTmuxSession()`. A process
+  in a session that only reuses a recorded name is reported as external. Only
+  live panes map a process to a session. The start time seen when ownership
+  was established must still match before SIGTERM (unreadable sends nothing)
+  and before the delayed SIGKILL. Ownership is re-checked before the tmux
+  session is removed.
+- Tests: `createTempProject().cleanup()` removes only sessions named after its
+  unique temp dir. The broad `cleanupTmuxSessions('akit-integ-'/'akit-sched-')`
+  calls are removed. `scheduler-basic` now waits for spawns that are still in
+  flight before cleanup. Its every-second cron could land a spawn after
+  cleanup and leak a dead `instar-test-*-job-fast-test-*` session, which was
+  seen twice during a full `npm test` run here.
+
+## What to Tell Your User
+
+Several of my automatic clean-up paths for stuck sessions and processes now
+check that something is mine before they stop it. Before, a few repair paths went by a name or a
+process number, and on a busy machine those can end up belonging to some
+other program. If I can't prove something is mine, I leave it running.
+
+## Summary of New Capabilities
+
+- None new. Lifeline restart and takeover, pipe-session and triage cleanup,
+  and orphan cleanup now check that a process or session is their own before
+  stopping it, in the cases listed under Evidence.
+
+## Evidence
+
+- `tests/integration/foreign-process-survives-test-instance.test.ts` (real
+  tmux, 8 cases, including a retained dead pane that must get no signal). The orphan pass reaps an owned orphaned session. A foreign
+  `claude`-shaped process survives, and so does a foreign process in a session
+  that reuses a name the instance recorded. Fixture cleanup removes the
+  instance's own sessions and nothing else. Triage replaces its own session and
+  refuses a foreign one. The pipe spawner kills its own session, leaves a
+  same-name replacement and a stale pid alone, and refuses to spawn over a
+  foreign session. We restored each old behavior in turn: the reaper's name
+  match, the round-1 spawner, no triage guard, and the old helper. Each one
+  makes the matching case fail.
+- `tests/unit/lifeline/lifeline-lock-identity.test.ts` (real child processes).
+  A stale lock whose pid now runs a foreign command mentioning "lifeline" is
+  neither a restart target nor signalled at startup. The own proven holder is
+  a target and is terminated when wedged. Making the identity check accept any
+  live pid fails 3 cases. A `ps` that always fails leaves a live holder's lock
+  in place, and a confirmed-dead holder's lock is still taken over.
+- `tests/unit/orphan-reaper-pane-identity.test.ts` (scripted `ps`/tmux, no
+  real signals): a pid reused from a dead pane is not reaped; a changed or
+  unreadable start time sends nothing; a live owned orphan is still reaped.
+- These tests cover the specific cases listed. They do not cover every kill
+  path in instar, and PID reuse is simulated rather than reproduced. The six
+  builder deaths remain unattributed; these paths are not ruled out as their
+  cause.

package/upgrades/side-effects/feedback-drain-identity.md

@@ -0,0 +1,78 @@
+# Side-Effects Review — feedback drain file identity drops st_dev
+
+**Version / slug:** `feedback-drain-identity`
+**Date:** `2026-09-30`
+**Author:** `echo`
+**Second-pass reviewer:** `not required (one pure comparison function; checksum + epoch checks untouched)`
+
+## Summary of the change
+
+`FeedbackDrainStore.sqliteFileIdentity()` recorded `dev:ino:birthtimeMs`. macOS APFS renumbers
+`st_dev` across reboots/remounts. Live on the Mac Studio the checkpoint held
+`16777231:454176:1787103639952.4048` while the untouched file now stats as dev `16777232`, same
+inode and birth time. `restorePending()` saw a different identity, the checksum and epoch matched,
+so it returned `true`: the drain refused every tick and checkpoint (274 reports waiting) and the
+only exit was the PIN-gated failover-finalize route — a false operator escalation.
+
+Change: identity is now `ino:birthtimeMs`. New exported pure helper `sameSqliteFileIdentity(recorded, current)`
+compares a legacy three-part identity on its `ino:birthtime` parts. Both comparison sites
+(`restorePending()` and `finalizeRestore()`) use it. Checksum and epoch checks are unchanged.
+
+## Decision-point inventory
+
+- Is the DB file the checkpointed one? — **modify** — compare on inode + birth time only.
+
+## 1. Over-block
+
+Less than before: a reboot or remount no longer makes the drain refuse work.
+
+## 2. Under-block
+
+A replacement file that keeps the same inode AND the same birth time would pass. A restore by copy
+or rename produces a new inode and a new birth time (tested). The previous form had the same
+exposure for same-device replacements; st_dev only added detection for a cross-volume move with a
+colliding inode and birth time, which is not credible. On filesystems without birth time
+(`birthtimeMs` 0) identity falls to the inode alone, the same as before within one device.
+
+## 3. Level-of-abstraction fit
+
+The fix lives in the one function that defines identity; no caller changes.
+
+## 4. Signal vs authority compliance
+
+Unchanged: restore detection still blocks via `DrainConflictError`/restorePending, with the same
+checksum and epoch authority.
+
+## 5. Interactions
+
+`finalizeRestore()` requires the identity to DIFFER from the checkpoint; it now uses the same
+comparison, so a same-file finalize is still refused and a genuine restore still finalizes
+(existing destructive-restore test passes).
+
+## 6. External surfaces
+
+None. `/feedback-factory/drain/status` output shape unchanged; the stuck drain reports
+`restorePending:false` after the update.
+
+## 7. Multi-machine posture
+
+Per-machine file identity; not replicated. No change.
+
+## 7b. Constitutional Rules touched
+
+Durable intake floor: restores the drain's intake without weakening restore detection.
+Occam (Rule 116): one comparison helper, no new state or options.
+
+## 8. Rollback cost
+
+Release revert. Checkpoints written in the two-part form read as "different" to the old code,
+which reproduces today's escalation, no data loss.
+
+## Conclusion
+
+Safe to ship. Tests: unit (st_dev change on the same file → not restored; replaced file → still
+restored, new and legacy formats), integration backup-restore and e2e drain lifecycle all green.
+
+## Evidence pointers
+
+- `tests/unit/feedback-drain-store.test.ts` — new identity tests; the st_dev tests fail on the old code.

package/upgrades/side-effects/test-instance-foreign-kills.md

@@ -0,0 +1,261 @@
+# Side-Effects Review — automatic kills signal only what the agent can prove it started
+
+**Version / slug:** `test-instance-foreign-kills`
+**Date:** `2026-09-30`
+**Author:** `Echo (builder)`
+**Second-pass reviewer:** `independent reviewer subagent (see below)`
+
+## Summary of the change
+
+Round 3, after Astra's round-2 review (four must-fixes: M1 unreadable identity
+read as a dead lifeline; M2 and M3 retained dead tmux panes; M4 overreaching
+claims). The kill paths the reviews named now check ownership through one
+small shared check before they signal, in the cases the tests below exercise.
+Where a check cannot prove ownership, the path logs or reports and does not
+signal.
+
+Round 3 changes:
+
+- `processIdentity.checkRecordedProcess`: an unreadable start time (ps denied,
+  timed out, unparseable) is `gone` only when `kill(pid, 0)` returns ESRCH;
+  otherwise `unproven`, so a live lifeline lock is kept (M1). In
+  `acquireLockFile`, if the holder's identity turns unreadable during the
+  SIGTERM grace, the new lifeline neither sends SIGKILL nor takes the lock
+  (found by the round-3 second-pass review).
+- `PipeSessionSpawner.killPipeSession`: reads `#{pane_dead}`. The process
+  group is signalled only for an owned pane tmux reports live; an owned dead
+  or unreadable pane gets only `kill-session -t <session_id>`. The fallback
+  SIGKILL of the bare pane pid (when the pgid read failed) is removed (M2).
+- `OrphanProcessReaper`: only live panes (`pane_dead=0`) map a pid to a
+  session. The start time read at classification is carried on the orphan and
+  must still match right before SIGTERM; null sends nothing and the tmux
+  session is left alone. Session ownership is re-checked before the terminal
+  `kill-session`. The operator-requested external kill is a separate mode of
+  `killProcess` (M3).
+
+- `src/core/processIdentity.ts` (new, ~60 lines): `processStartMs(pid)` reads
+  `ps -o lstart=`. `checkRecordedProcess({pid, procStart, startedAt})` returns
+  one of four verdicts: `same` (the start time matches), `gone`, `reused` (the
+  live process started after the record was written), or `unproven`.
+- `src/lifeline/lifelineLock.ts` (new; `acquireLockFile` moved here from
+  `TelegramLifeline.ts`): the lock records `procStart`. Takeover signals are
+  sent only on `same`, and identity is proven again before SIGKILL. On
+  `reused`, the new lifeline takes the lock without a signal. On `unproven`,
+  it respects the lock. `provenLifelineRecords()` selects the `lifeline
+  restart` targets.
+- `src/lifeline/startupMarker.ts`: the marker records `procStart`.
+- `src/cli.ts` (`lifeline restart` fallback): targets come from
+  `provenLifelineRecords(lock, marker)`, and SIGKILL goes only to targets
+  that are still `same` after the grace period. The round-1 keyword check is
+  removed.
+- `src/threadline/PipeSessionSpawner.ts`: `new-session -P -F '#{session_id}
+  #{pane_pid}'` records the incarnation. `killPipeSession` signals the group
+  and runs `kill-session -t <session_id>` only while the named session still
+  lists that exact id and pid; otherwise it signals nothing. `spawn()` replaces
+  a same-name session only when it is recorded in `activeSessions`, and
+  otherwise refuses.
+- `src/core/SessionManager.ts`: new `ownsLiveTmuxSession(name)`. It reads the
+  live session's `INSTAR_SESSION_ID` (every spawn path sets it) and checks
+  that this agent's state holds a record with that id for that tmux name.
+  `spawnTriageSession` refuses to kill an occupied name without that proof.
+- `src/monitoring/OrphanProcessReaper.ts`: a historical-name match becomes
+  `instar-orphan` (auto-kill eligible) only with `ownsLiveTmuxSession`.
+  Without it, the process is classed `external` and only reported.
+  `killProcess` records the start time before SIGTERM and sends SIGKILL only
+  if it is unchanged.
+- Tests: the broad `cleanupTmuxSessions('akit-integ-'/'akit-sched-')` calls are
+  removed. `scheduler-basic` waits for in-flight spawns before cleanup (a
+  late spawn leaked a dead-pane session). The regression test is rewritten with 7 real-tmux cases, both sides
+  of each decision, with teardown in `finally`. There is a new unit test with
+  real child processes for the lifeline identity.
+
+## Decision-point inventory
+
+- Lifeline restart target selection — modify — keyword → proven incarnation.
+- Lifeline lock takeover — modify — any live pid → proven incarnation; reused → silent takeover; unproven → respect.
+- Pipe kill / pipe spawn over an existing name — modify — name → recorded session id + pane pid; foreign name → refuse.
+- Triage spawn over an existing name — modify — name → `ownsLiveTmuxSession`; foreign → refuse (throw).
+- Orphan auto-kill eligibility — modify — historical name → name + live `INSTAR_SESSION_ID` match.
+- Delayed SIGKILL escalations (reaper, lifeline) — modify — `kill(pid,0)` → same start time.
+
+---
+
+## 1. Over-block
+
+- A legacy lifeline lock or marker, written before this change and so with no
+  `procStart`, is never signalled by `lifeline restart`. At startup, a legacy
+  lock whose live pid started before the lock was written is respected, not
+  taken over. The first restart after the update normally goes through
+  launchd's `kickstart -k`, which does not use these paths. Once a lifeline
+  from this version is running, its records carry `procStart`.
+- A process whose start time `ps` cannot read is not signalled. For the
+  lifeline lock it counts as gone only on a confirmed ESRCH; a live holder
+  whose start time cannot be read keeps its lock, so a new lifeline refuses to
+  start until `ps` works again.
+- Pipe spawner: a live owned pane whose pgid cannot be read gets no process
+  signal; `kill-session` still hangs up the pane.
+- Orphan reaper: an owned orphan whose start time cannot be read is skipped
+  (reported as "Skipped orphan"), not killed.
+- Triage: if a session record was purged while its tmux session lived on, the
+  triage name stays occupied and the spawn throws. `TriageOrchestrator`
+  already handles a failed spawn. That is better than killing an unproven
+  session.
+- Orphan reaper: an orphan whose record was purged, or whose tmux env lacks
+  `INSTAR_SESSION_ID` (a very old session), is only reported. The operator
+  can still use the explicit external-process API.
+- Pipe spawner: a pipe session from before a server restart, which is no
+  longer in `activeSessions`, blocks a new pipe spawn for that thread. The
+  caller then falls through to the normal A2A path, as it does for any
+  `spawned:false`.
+
+## 2. Under-block
+
+- Start-time resolution is 1 s. A pid reused within the same second as the
+  original start would pass. That is not a realistic collision.
+- Two SessionManagers sharing one agent state dir are treated as one agent
+  and may clean up each other's sessions. That is the same agent, which is
+  intended.
+- The final ownership check and the signal are separate calls, a few
+  milliseconds apart. The reaper's terminal `kill-session` goes by name after
+  an ownership re-check.
+- Only the paths named here were changed. Other kill paths (for example the
+  SessionWatchdog, and SessionManager's own passes) were not re-audited in
+  this change.
+- This change does not identify what killed the six builders. Those deaths
+  remain unattributed, and these paths are not excluded as a possible cause.
+
+## 3. Level-of-abstraction fit
+
+There is one shared primitive for pids (`processIdentity`) and one for
+sessions (`ownsLiveTmuxSession`, which uses the `INSTAR_SESSION_ID` every
+spawn already sets). The pipe spawner uses tmux's own unique session id. No
+new subsystem, store or loop is added.
+
+## 4. Signal vs authority compliance
+
+- [x] No — no block/allow surface on information flow. These checks narrow an
+  irreversible action (a signal) to proven targets. Each one removes reach and
+  none adds authority.
+
+## 4b. Judgment-point check
+
+The checks are deterministic safety floors on a SIGKILL. They are not
+heuristics at a competing-signals decision point.
+
+## 5. Interactions
+
+- The SessionWatchdog already checks the parent pid, command and incarnation
+  before its descendant signals, and this change leaves it untouched.
+  SessionManager's own passes iterate its own records.
+- The reaper's `trackedNow` path to `terminateSession` is unchanged.
+  Ownership proof only gates the non-tracked, historical-record path.
+- `TelegramLifeline.releaseLockFile` still compares pid only when removing its
+  own lock file (a file, not a signal). That is unchanged.
+- `acquireLockFile` behavior for a dead pid (take over) and a fresh proven
+  holder (respect) is unchanged.
+
+## 6. External surfaces
+
+- New optional JSON fields: `procStart` in `state/lifeline.lock` and
+  `state/lifeline-started-at.json`. Readers ignore unknown fields, and
+  `readStartupMarker` keeps its existing validation.
+- A new `spawned:false` reason from the pipe spawner, and a new triage spawn
+  error message.
+- Orphan reaper reports can now list a process as `external` with the reason
+  "ownership unproven".
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+Machine-local BY DESIGN. Pids, start times and tmux sessions are per-host, and
+every check is on the host that would send the signal.
+
+## 8. Rollback cost
+
+Revert and ship a patch release. The new JSON fields are additive, and older
+code ignores them. No migration is needed.
+
+---
+
+## Conclusion
+
+Clear to ship pending review. In the cases the tests exercise, the paths
+Astra named signal only a proven incarnation, and the positive cleanup cases
+still work. The limits in section 2 remain.
+
+---
+
+## Second-pass review (if required)
+
+**Round 3 reviewer:** independent reviewer subagent (general-purpose,
+read-only). **Verdict: Concur.** M1-M4 closed in the code read; no automatic
+path read signals an unproven process. Its minor points: (a) identity turning
+unreadable during the lifeline SIGTERM grace fell through to a takeover —
+fixed in this round, with a test; (b) the reaper reads `procStart` just after
+the ownership check, not from the original `ps` scan (documented limit);
+(c) the integration test's teardown killed detached children on an `alive`
+check only — it now also requires the recorded start time.
+
+**Round 2 reviewer:** independent reviewer subagent (general-purpose, read-only).
+**Verdict: Concur with the review** (round 2). Astra's round-2 review then
+found M1-M3 below, so this verdict did not hold; the round-3 fixes await
+Astra's next review.
+
+- Round-2 reviewer's claim (superseded): all six must-fixes are closed in the
+  touched code. No touched path lets a
+  name, keyword or bare pid alone authorize an automatic signal. The lifeline
+  restart and takeover signal only on `same`, and SIGKILL re-checks. Pipe
+  teardown needs an exact id+pid match and kills by id. Triage throws on a
+  session it does not own. The reaper's name-only matches are report-only,
+  and its SIGKILL re-checks the start time.
+- `ownsLiveTmuxSession` checks the id against a regex before
+  `StateManager.getSession`, so its `validateKey` cannot throw. All four spawn
+  paths set `INSTAR_SESSION_ID`.
+- The over-block and under-block lists are accurate.
+- Minor residuals, none blocking. After proving ownership, the reaper kills
+  the tmux session by name, which leaves a window of milliseconds. Its first
+  SIGTERM goes to a pid from the same scan. A wedged legacy lifeline is left to
+  launchd's `kickstart -k`, as section 1 says.
+
+---
+
+## Evidence pointers
+
+- `tests/integration/foreign-process-survives-test-instance.test.ts`: 7/7
+  pass. We restored each old behavior in turn: the reaper name match, the
+  round-1 `PipeSessionSpawner.ts`, no triage guard, and the pre-fix
+  `tests/helpers/setup.ts`. Each makes its matching case fail (1, 2, 1 and 1
+  failures).
+- `tests/unit/lifeline/lifeline-lock-identity.test.ts`: 7/7 pass. With
+  `checkRecordedProcess` changed to accept any live pid, 3 fail. The legacy
+  "respect" case does not discriminate on its own, because a fresh child cannot
+  carry a 5-minute-old lock. Its verdict is covered by the `checkRecordedProcess` case.
+- Round 3: `tests/unit/lifeline/lifeline-lock-identity.test.ts` adds a `ps`
+  that always fails on PATH: a live holder is `unproven` and keeps its lock; a
+  holder confirmed dead (ESRCH) is `gone` and the lock is taken over.
+- Round 3: a holder that ignores SIGTERM, with `ps` failing after the first
+  identity check: `acquireLockFile` returns false, sends no SIGKILL, and the
+  lock is unchanged. This case fails against the round-2 `lifelineLock.ts`.
+- Round 3: the integration file adds a real retained dead pane
+  (`remain-on-exit`, its pane process killed by exact pid): `process.kill` is
+  never called and the owned session is removed. With the round-2 spawner this
+  case fails. The live-pane positive case now also asserts the pane process
+  exits. 8/8 pass.
+- Round 3: `tests/unit/orphan-reaper-pane-identity.test.ts` (scripted ps/tmux,
+  stubbed `process.kill`): a foreign pid reusing a dead or indeterminate
+  pane's pid is not an orphan and gets no signal; the same pid in a live owned
+  pane is reaped (SIGTERM then SIGKILL, ownership re-checked before tmux
+  cleanup); a changed or unreadable start time sends nothing; the operator
+  kill still works. All six fail against the round-2 reaper. PID reuse itself
+  is simulated, not reproduced.
+- Round-1 live check: a foreign `claude -p … fix the lifeline` process survived
+  the `lifeline restart` fallback.
+
+---
+
+## Class-Closure Declaration (display-only mirror)
+
+- **`defectClass`** — `unbounded-self-action`
+- **`closure`** — `n/a` (negative declaration)
+- **`reason`** — No new trigger or loop. The change only narrows which
+  pid or session existing kills may signal, and adds ownership proof before
+  each one.