instar 1.3.233 → 1.3.235

package/upgrades/1.3.235.md

@@ -0,0 +1,110 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: minor -->
+
+## What Changed
+
+**Fixed three latent ESM `require()` bugs and re-armed the two standards-gates that
+should have caught them.** Instar ships as ESM (`"type":"module"`, tsconfig Node16),
+where a bare CommonJS `require(...)` is undefined and throws `ReferenceError` the
+moment that line runs. Three runtime files had one on a live path:
+`reflect.ts` (hit by `instar reflect run`), `SessionWatchdog.ts` (the stuck-session
+watchdog's child-detection), and `PostUpdateMigrator.ts` (Codex-path detection — it
+silently returned null instead of throwing).
+
+The fix uses the repo's existing `createRequire(import.meta.url)` idiom for the first
+two (same lazy load, now ESM-legal), and for `PostUpdateMigrator.ts` simply calls the
+`detectCodexPath` it was already statically importing (the lazy require was redundant).
+
+These bugs reached `main` because the `esm-compliance` and `no-silent-fallbacks`
+gates — whose entire job is catching this class of issue — had been parked in
+`vitest.push.config.ts`'s flaky-exclude list, so CI stopped running them. This change
+re-arms both (taught `esm-compliance` to recognize the one legitimate ESM use of
+`require` — `createRequire`, needed to re-load a native module after a rebuild) and
+corrects the `no-silent-fallbacks` baseline to the true count.
+
+The per-agent **ResourceLedger** gains **Phase B: continuous CPU + memory
+sampling**, closing the gap where Instar tracked token usage well (the
+`TokenLedger`) but tracked *zero* CPU and *zero* memory per agent.
+
+A new `ResourceSampler` (mirroring `TokenLedgerPoller`) runs on a cadence and
+samples:
+- the agent's **own server process** — CPU% from a `process.cpuUsage()` delta over
+  the tick interval, plus RSS and V8 `heapUsed` from `process.memoryUsage()`;
+- each **spawned session** — by its tmux pane PID, via a single batched
+  `ps -o pid=,%cpu=,rss= -p …` call (one child process per tick, dead-PID tolerant);
+- a computed **aggregate** (server + all sessions).
+
+Each reading is written to the existing `ResourceLedger` SQLite store (a new
+`resource_samples` table, bounded by a retention prune). Two read-only routes
+expose it — `GET /resources/summary` (current + windowed avg/peak CPU%/RSS per
+source) and `GET /resources/samples` (recent raw samples) — alongside a new
+dashboard **Resource Usage** tab that renders it in plain language.
+
+The sampler rides the `developmentAgent` dark-feature gate: it is **live on
+development agents and dark on the fleet** until promoted. Configuration lives
+under `monitoring.resourceLedger` (`sampleIntervalMs`, `idleSampleIntervalMs`,
+`retentionDays`). It is read-only observability — it never gates, throttles, or
+mutates anything, and it fails open (a sampling error never crashes the agent).
+It is also cheap by design: a free CPU-usage delta, one batched `ps` per tick,
+bounded retention, and an idle-cadence backoff when no sessions are running.
+
+## What to Tell Your User
+
+A few features that could have thrown a runtime error on a rarely-hit path — job
+reflection, the stuck-session watchdog, and Codex detection during updates — are now
+fixed. Two safety checks that had been accidentally switched off in CI are switched
+back on, so this class of bug can't quietly return. Nothing for you to configure.
+
+I can now show you how much CPU and memory I'm actually using — my own server
+process and each session I have running — sampled continuously. Just ask "how
+much CPU or memory am I using right now?" and I'll give you the current numbers
+plus the recent peaks, or open the new Resource Usage tab on the dashboard to see
+it laid out: a right-now headline, a breakdown by process, and a recent trend.
+
+This is purely a window into resource use — it watches and reports, it never
+changes how I work or slows anything down. On most agents it ships turned off for
+now while it proves itself; if you'd like it switched on, just say so.
+
+## Summary of New Capabilities
+
+- Three latent ESM require() ReferenceErrors repaired (reflect, SessionWatchdog,
+  PostUpdateMigrator) using the established createRequire idiom or an existing static import.
+- The esm-compliance and no-silent-fallbacks CI gates are re-armed (removed from
+  FLAKY_TESTS), restoring enforcement of two standards that had silently stopped gating.
+
+- See current and recent CPU and memory use per agent, broken down by the server
+  and each running session, plus a combined total.
+- New read-only endpoints: GET /resources/summary and GET /resources/samples.
+- New dashboard Resource Usage tab.
+- Tunable cadence and retention; ships live on development agents, dark on the fleet.
+
+## Evidence
+
+- `tsc --noEmit` clean across the three source edits.
+- Re-armed gates verified green under the REAL CI config:
+  `vitest run --config vitest.push.config.ts tests/unit/esm-compliance.test.ts tests/unit/no-silent-fallbacks.test.ts`
+  → 2 files / 7 tests passed.
+- Baseline correction is evidence-backed: the heuristic count at d0fe838 (the
+  `[skip ci]` release that set 186) was already 431; at HEAD it is 437; a set-diff of
+  the two match-lists shows the +6 are the same catch blocks at shifted line numbers,
+  not new silent fallbacks.
+
+- New unit tests (`tests/unit/ResourceLedger.test.ts` Phase B block,
+  `tests/unit/ResourceSampler.test.ts`): record/summary/recentSamples/prune/count,
+  CPU% from a `cpuUsage` delta, batched-PID sampling, dead-PID tolerance, aggregate,
+  idle backoff, and fail-open-never-throws — 22 passing.
+- Wiring-integrity (`tests/unit/resource-sampler-wiring.test.ts`): the sampler is
+  constructed behind the `developmentAgent` gate, stopped on shutdown, delegates to
+  the real ledger (not a no-op), and is read-only / off the hot path — 8 passing.
+- Integration (`tests/integration/resources-summary-routes.test.ts`):
+  `/resources/summary` + `/resources/samples` return 401 without a bearer token,
+  503 when disabled, and 200 with per-source data; no raw event bodies leak —
+  5 passing.
+- E2E (`tests/e2e/resources-summary-lifecycle.test.ts`): on the production
+  `AgentServer` boot path the route is alive (200 under `developmentAgent`) and
+  503-stubs when the ledger is disabled — 5 passing.
+- `npm run build`, `npm run lint`, `node scripts/docs-coverage.mjs --check`, and the
+  parity guards (feature-delivery-completeness, capabilities-discoverability,
+  builtin-manifest, ConfigDefaults) all green.

package/upgrades/commitment-bare-continuation-guard.eli16.md

@@ -0,0 +1,26 @@
+# CommitmentSentinel bare-continuation guard — plain English
+
+## The problem
+The agent has a watcher that tries to notice when it has made a promise it should
+follow through on ("I'll report back when X"). But it was too eager: it treated
+almost every message you sent — even a plain "yes" or "please proceed" — as if it
+created a promise to track. The result was a registry full of fake "commitments"
+(36 of them showed as broken), which is exactly why it got hard to tell what was
+actually open.
+
+## The fix
+Before the watcher even considers an exchange, a simple deterministic check drops
+the ones where your message was just an approval or a "keep going" — those ask for
+nothing durable, so they can't be a promise. Real requests are untouched: anything
+with an action word ("deploy", "set", "restart", "turn off") still goes through, so
+"go ahead and deploy the latest" is correctly kept.
+
+## How safe is it?
+Very. The watcher only *detects* promises — it never deletes or changes one. The
+worst case is missing a promise from a one-word message with no verb (almost never),
+which is the safe direction. Explicitly opening a commitment via the API still works
+exactly as before. 50 tests cover both sides (drops the bare ones, keeps the real
+requests).
+
+## What you need to decide
+Nothing — it just makes the commitment registry trustworthy again.

package/upgrades/side-effects/commitment-bare-continuation-guard.md

@@ -0,0 +1,42 @@
+# Side-Effects Review — CommitmentSentinel bare-continuation guard
+
+**Version / slug:** `commitment-bare-continuation-guard`
+**Date:** `2026-06-04`
+**Author:** `echo`
+**Second-pass reviewer:** `not required (Tier-1: signal-only, favors false-negatives)`
+
+## Summary of the change
+
+The CommitmentSentinel registered a false-positive "commitment" for nearly every
+user message — including bare approvals/continuations ("please proceed", "yes") —
+because its LLM detector ran on every user→agent exchange. A deterministic
+pre-filter (`isBareContinuation`) now drops bare-approval exchanges before the LLM
+sees them. Reinforced with one line in the detection prompt.
+
+## Decision-point inventory
+
+One pure decision: `isBareContinuation(text)` — drop the exchange (true) or keep it
+(false). Unit-tested on both sides (50 cases).
+
+## 1. Over-block (what legitimate commitments could it wrongly DROP?)
+
+A genuine durable request phrased as a bare phrase under 30 chars with no
+imperative verb. Mitigated by the verb guard: any message containing a durable
+verb (change/set/turn/deploy/restart/report/…) is KEPT, so "go ahead and deploy",
+"sure, set the model to opus", "ok now restart the gemini server" all pass through
+(tested). The residual risk is a one-word durable instruction with no verb, which
+is vanishingly rare and favors the SAFE direction (a missed detection, not a false
+commitment) — the infrastructure `POST /commitments` path still registers explicit
+commitments regardless.
+
+## 2. Under-block (what false positives does it still MISS?)
+
+Substantive-but-non-committal exchanges (a real question the agent answered) still
+reach the LLM; the prompt reinforcement is the second line of defense there. This
+PR targets the dominant, deterministic false-positive class (bare approvals).
+
+## 3. Reversibility / blast radius
+
+Signal-only. The sentinel only DETECTS unregistered commitments; dropping an
+exchange cannot delete or alter an existing commitment. No persistent state, no
+migration, no route change. Fully reversible by reverting the filter.

package/upgrades/side-effects/esm-require-fixes-gate-rearm.md

@@ -0,0 +1,110 @@
+# Side-Effects Review — ESM require() bug fixes + standards-gate re-arm
+
+**Version / slug:** `esm-require-fixes-gate-rearm`
+**Date:** `2026-06-03`
+**Author:** `Echo (instar-dev agent)`
+**Tier:** `1` (small / low-risk — bugfix + test/config; no new capability, no block/allow surface)
+**Second-pass reviewer:** `not required`
+
+## Summary of the change
+
+Fixes three genuine latent ESM bugs and re-arms the two standards-gates that should
+have caught them but were parked in the CI flaky-exclude list.
+
+The package is ESM (`"type":"module"`, tsconfig `Node16`), where a bare CommonJS
+`require(...)` is undefined and throws `ReferenceError` at runtime. Three runtime
+files used a bare `require()`:
+- `src/commands/reflect.ts:355` — lazy `require('../core/intelligenceProviderFactory.js')`
+  inside `resolveIntelligence()`; `instar reflect run` would throw when it hits this.
+- `src/monitoring/SessionWatchdog.ts:713` — lazy `require('./frameworkProcessSignals.js')`
+  during child-process detection; throws on the watchdog's polling path.
+- `src/core/PostUpdateMigrator.ts:1137` — lazy `require('./Config.js')` for
+  `detectCodexPath` inside a try/catch (so it silently returned null → codex never
+  detected).
+
+Fixes:
+- `reflect.ts`, `SessionWatchdog.ts` — bind a real `require` via
+  `createRequire(import.meta.url)` (the repo's existing idiom, see
+  `NativeModuleHealer.ts:44`). Preserves the deliberate lazy-load semantics exactly,
+  but now ESM-legal.
+- `PostUpdateMigrator.ts` — `detectCodexPath` is ALREADY statically imported at the
+  top of the module (line 56), so the lazy require was redundant; replaced with a
+  direct call to the static import. No shim needed.
+
+Gate re-arm + test corrections:
+- `tests/unit/esm-compliance.test.ts` — teach the guard to recognize the legitimate
+  `require = createRequire(import.meta.url)` ESM binding (otherwise the naive
+  `require(` regex false-positives on `node-abi-mismatch.ts`, which already uses the
+  pattern correctly to re-load a native module after an ABI rebuild — which static
+  `import` cannot express).
+- `tests/unit/no-silent-fallbacks.test.ts` — baseline corrected 186→437 with
+  evidence (the 186 was set by a `[skip ci]` release while the true count was already
+  431; the +6 to 437 is line-shift re-counts, zero new fallbacks).
+- `tests/unit/feature-delivery-completeness.test.ts` — track the previously-untracked
+  `Token-Burn Alerts` migrator section.
+- `src/data/builtin-manifest.json` — regenerated against current source.
+- `vitest.push.config.ts` — RE-ARM `esm-compliance` + `no-silent-fallbacks` (removed
+  from `FLAKY_TESTS`) so these standards-gates run in CI again. They were quarantined,
+  which is exactly how the three bare-require bugs reached main unnoticed.
+
+## Decision-point inventory
+- `reflect.ts / SessionWatchdog.ts` require binding — **modify (mechanics only)** —
+  same lazy load, now via a real ESM `require`. No behavior change beyond "no longer
+  throws."
+- `PostUpdateMigrator.ts` — **modify** — use the existing static import; drops a
+  silent-null path (codex detection now actually works).
+- Test guards + push config + manifest — **modify/add** — no runtime surface.
+
+## 1. Over-block
+No block/allow surface. The two re-armed gates are CI test gates, not runtime gates.
+They cannot block a running agent; they can only fail a PR's CI (their intended job).
+
+## 2. Under-block
+The esm-compliance guard previously UNDER-blocked (it was excluded, so it caught
+nothing — three real require() bugs slipped through). Re-arming restores the intended
+coverage. The createRequire recognition is precise (`require\s*=\s*createRequire\s*\(`)
+so it does not newly under-block: a file only earns the skip by establishing a real
+ESM `require` binding.
+
+## 3. Level-of-abstraction fit
+Correct. The runtime fixes operate at the module-loading layer (the bug's layer). The
+gate changes operate at the CI-config layer. Neither reaches into business logic.
+
+## 4. Signal vs authority compliance
+**Required reference:** docs/signal-vs-authority.md
+- [x] No block/allow runtime surface of its own. The re-armed tests are SIGNALS to CI
+  (a PR fails), not runtime authorities. No agent decision is gated by this change.
+
+## 5. Interactions
+- **Double-fire / shadowing:** none. The require fixes are byte-equivalent loads.
+- **PostUpdateMigrator:** the static `detectCodexPath` import already existed and is
+  used elsewhere; calling it directly cannot introduce a cycle that the static import
+  didn't already have.
+- **no-silent-fallbacks baseline:** exact-count ratchet on a hyperactive main may need
+  a future bump if a large mechanical sweep lands (as happened 3× before) — that is the
+  ratchet working, not a regression. Re-measured fresh before merge.
+- **esm-compliance:** a future dependency-introduced `require()` will now (correctly)
+  fail CI — that is the gate doing its job.
+
+## 6. External surfaces
+None. No new route, no new persistent state, no config default change. `vitest.push.config.ts`
+is build/test tooling, not shipped agent config.
+
+## 7. Rollback cost
+Trivial. Revert the commit: the require fixes go back to throwing (status quo ante),
+and the two gates return to FLAKY_TESTS. No state, no migration, no user-visible change.
+
+## Conclusion
+Low-risk Tier-1 bugfix. Three real latent ESM `require()` ReferenceErrors fixed using
+the repo's established `createRequire` idiom (or, for PostUpdateMigrator, the existing
+static import). The standards-gates that exist to catch this class of bug are re-armed
+and verified green under the real CI push config. Clear to ship.
+
+## Evidence pointers
+- `tsc --noEmit` clean on the three src edits.
+- Re-armed gates pass under the real CI config:
+  `npx vitest run --config vitest.push.config.ts tests/unit/esm-compliance.test.ts tests/unit/no-silent-fallbacks.test.ts`
+  → 2 files / 7 tests passed.
+- Root-cause evidence for the baseline correction: heuristic count at d0fe838
+  (the `[skip ci]` release that set 186) = 431; at HEAD = 437; set-diff shows the +6
+  are line-shifted re-counts of the same catch blocks, not new fallbacks.

package/upgrades/side-effects/resource-ledger.md

@@ -0,0 +1,110 @@
+# Side-Effects Review — ResourceLedger Phase B (CPU + memory sampling)
+
+**Version / slug:** `resource-ledger`
+**Date:** `2026-06-03`
+**Author:** `echo`
+**Second-pass reviewer:** `not required`
+
+## Summary of the change
+
+Add continuous CPU + memory sampling to the per-agent `ResourceLedger`. A new
+`ResourceSampler` (mirroring `TokenLedgerPoller`) samples the agent's own server
+process (CPU% via a `process.cpuUsage()` delta, RSS + heapUsed via
+`process.memoryUsage()`) and each spawned session by its pane PID (one batched `ps`
+call, dead-PID tolerant), plus an aggregate, into a new `resource_samples` table.
+Two read-only routes (`/resources/summary`, `/resources/samples`) and a dashboard
+"Resource Usage" tab expose it. Constructed at boot in `AgentServer` behind the
+`developmentAgent` dark-feature gate; bounded by a retention prune.
+
+## Decision-point inventory
+
+One gate: whether to construct + start the sampler. It resolves
+`monitoring.resourceLedger.enabled ?? !!config.developmentAgent` — explicit config
+wins; otherwise live on dev agents, dark on the fleet. No other decision logic; the
+sampler is pure observation (no kill/throttle/route-block verdicts).
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject?** Nothing is rejected. The sampler
+gates nothing — it only reads `ps`/`process.*` and writes the ledger. The routes
+return data or a `503` stub when the ledger is null; they never block a caller.
+
+## 2. Under-block
+
+**What does this still miss?** CPU% for sessions comes from the pane PID's `ps
+%cpu`, which on macOS/Linux is a lifetime-average for the process, not a precise
+instantaneous interval rate (the own-process reading IS a true interval delta). It
+samples the pane shell PID, not a full descendant-tree roll-up, so a session's
+total tree CPU is approximated by the pane process line `ps` returns. Rate-limit
+aggregation across the fleet and mitigation actions are explicitly later slices —
+this slice only measures.
+
+## 3. Level-of-abstraction fit
+
+**Right layer?** Yes. The sampler lives in `monitoring/` beside `TokenLedgerPoller`
+and writes the existing `ResourceLedger`; boot wiring lives in `AgentServer`
+alongside the Phase-A ledger construction; the routes are inline in `routes.ts` next
+to `/resources/rate-limits` and `/tokens/*`. The pane-PID resolver is a read-only
+method on `SessionManager` (the owner of session/tmux truth), reusing the same
+`tmux list-panes #{pane_pid}` pattern already there.
+
+## 4. Signal vs authority compliance
+
+**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
+
+No blocking authority added. The ledger and sampler are pure signal: every write
+swallows its own error, every OS call is guarded, and a sampling failure records
+nothing and retries next tick (`@silent-fallback-ok`). The sampler never reaches
+back into any runtime flow — it cannot affect the `LlmCircuitBreaker`, the reaper,
+sessions, or messaging.
+
+## 5. Interactions
+
+Reads `SessionManager.getRunningSessionPanePids()` (new, read-only) once per tick;
+no mutation of session state. Shares the `ResourceLedger` SQLite handle with the
+Phase-A rate-limit path (same DB, separate `resource_samples` table — no schema
+collision; the rate-limit methods are untouched). The `ps` call is a child process,
+isolated from the event loop. No interaction with the SessionReaper, sentinels,
+compaction recovery, or the multi-machine lease. Idempotent boot (additive table
+DDL), clean shutdown (sampler stopped before the ledger closes).
+
+## 6. External surfaces
+
+Two new HTTP routes (`GET /resources/summary`, `GET /resources/samples`,
+Bearer-authed, read-only), registered in `CapabilityIndex`. One new dashboard tab
+(static HTML/JS, XSS-safe via escaping, friendly "not turned on" state on 503). New
+config block `monitoring.resourceLedger.{sampleIntervalMs,idleSampleIntervalMs,
+retentionDays}` with `applyDefaults` backfill. CLAUDE.md template + idempotent
+`migrateClaudeMd` backfill so existing agents learn the capability. No new
+notifications, no Telegram, no external network calls. One new SQLite table inside
+the already-present `resource-ledger.db`.
+
+## 7. Performance (a resource tracker must itself be cheap)
+
+This is the dimension that matters most here — the tracker must not become a
+resource hog.
+
+- **Sampling cost per tick is bounded and small.** The own-process reading is a free
+  `process.cpuUsage()` + `process.memoryUsage()` pair (no syscall fork). All session
+  PIDs are measured in **one** batched `ps` child process per tick — never one fork
+  per PID — with a 5s timeout and a 1 MB output cap. Default cadence is 60s
+  (active), backing off to 5 minutes when no sessions are running, so the idle CPU
+  floor is barely moved.
+- **Write cost is bounded.** Each tick writes a handful of rows (server + N sessions
+  + 1 aggregate) in a single transaction. No JSONL scan, no file walk (unlike the
+  TokenLedger), so there is no large-history boot hazard.
+- **Storage is bounded.** The `resource_samples` table is pruned every tick to a
+  retention window (default 7 days). At a 60s cadence with a few sessions that is on
+  the order of tens of thousands of small rows — kilobytes-to-low-megabytes, indexed
+  on `(ts)` and `(source, ts)` for cheap windowed queries.
+- **Fail-open never amplifies cost.** A slow/failed `ps` is caught (no stacking — a
+  re-entrant tick is skipped), and the timer is `unref()`'d so it can't keep the
+  process alive. The feature ships dark on the fleet (developmentAgent gate), so the
+  fleet-wide cost is zero until each agent is deliberately promoted.
+
+## Rollback
+
+Set `monitoring.resourceLedger.enabled` to `false` (disables the whole ledger; the
+routes 503) — or, to keep Phase A but stop sampling, the sampler simply is not
+constructed when the `developmentAgent` gate resolves off. No data migration to
+reverse; the `resource_samples` table is additive and inert when unused.