instar 1.3.329 → 1.3.331

package/upgrades/1.3.331.md

@@ -0,0 +1,77 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+The provider-portability substrate (both Anthropic adapters + the cost-aware
+routing policy, on main since 2026-05-18) shipped dark: server boot installed
+the routing policy with a literal `readSdkCredit: () => null` stub and
+registered ZERO adapters, while every internal LLM call (sentinels, gates,
+extractors — measured ~1,000 real calls / ~27M input tokens per 24h on one
+agent) hardcoded `claude -p`, the path that bills the Agent SDK credit pot
+after 2026-06-15 and fails with no reroute when it drains. This PR wires the
+substrate into production: (1) `registerAnthropicAdapters()` at server boot —
+gated (codex-only agents register nothing), idempotent (incl. concurrent
+single-flight), lazy (zero spawns at boot), with a TTL-cached real credit
+reader replacing the null stub; (2) a new `intelligence.subscriptionPath.mode`
+config — `off` (default; byte-for-byte today's behavior, argv pinned by test)
+/ `auto` (drain the prepaid SDK pot while healthy, fall back to the
+subscription interactive pool when unknown/at-margin, one cross-path fallback
+with DegradationReporter) / `force` (interactive pool ONLY, zero `claude -p` —
+the soak + June-15 emergency lever); (3) `AnthropicSubscriptionRouter` +
+`InteractivePoolIntelligenceProvider` inside the existing breaker wrap, with
+the pure `decideSdkVsSubscription` decision extracted and shared so the two
+routing layers cannot drift; (4) `GET /providers/registry` introspection;
+(5) pool production-hardening — `model` knob (intelligence pool runs haiku),
+poolSize validation, idle retirement (`maxIdleMinutes` was dead config),
+on-demand growth, agent-scoped session prefix + orphan recovery at start().
+
+## What to Tell Your User
+
+After June 15, Anthropic changes how background AI calls are billed: the
+"headless" path I use for internal housekeeping (message screening, safety
+checks, summaries) starts drawing from a prepaid credit pot instead of the
+flat subscription. Before this change, when that pot ran dry all my background
+thinking would simply FAIL — silently. Now I have a second lane: I can run
+those same internal calls through a normal interactive Claude session (the
+same kind you chat with me in), which stays on the subscription. There's a
+switch with three positions — today's behavior (default), automatic (use the
+prepaid pot while it's healthy, switch lanes when it runs low), and
+subscription-only (the June-15 emergency lever). Nothing changes for you at
+this release — the switch ships in the OFF position — but the lane now exists,
+is tested, and can be flipped per-agent when the billing change lands.
+
+## Summary of New Capabilities
+
+- Boot registration of both Anthropic providers (`headless` + `interactive
+  pool`) — lazy, gated, idempotent; routing policy now reads REAL SDK credit
+  state (TTL-cached) instead of a hardcoded null.
+- `intelligence.subscriptionPath.mode: off | auto | force` — per-agent control
+  of which Anthropic lane internal LLM calls use; `off` is pinned
+  byte-for-byte to today's `claude -p` argv by test.
+- `GET /providers/registry` — what is ACTUALLY registered (adapter ids +
+  capability flags only), the June-15 readiness diagnostic.
+- Interactive-pool hardening: configurable model (haiku for internal calls),
+  poolSize validation, idle-session retirement, on-demand growth, agent-scoped
+  tmux prefix + orphan REPL recovery after crashes.
+
+## Evidence
+
+Gap measured live on echo: ~1,000 internal `claude -p` calls / ~27M input
+tokens per 24h, all unrouted (boot stub `readSdkCredit: () => null`, registry
+empty — confirmed via the new `/providers/registry` on a pre-fix boot in the
+e2e test). 44+ new tests across all three tiers: unit (router decision
+boundaries both sides, factory argv pin, bootRegistration gating/idempotency/
+laziness, pool model flag + lifecycle hardening), integration
+(`providers-registry-route.test.ts` full HTTP pipeline), e2e
+(`provider-substrate-live-wiring.test.ts` — production-mirroring boot:
+registry populated, route 200, default-off invariance, codex-only gate,
+no-spawn-at-boot). 5-agent adversarial review panel: correctness/wiring
+(BLOCK→fixed: registration TOCTOU single-flight), security/cost (SHIP;
+poolSize validation added), ops/scale (BLOCK→fixed: idle retirement, orphan
+recovery), standards/lessons (BLOCK→fixed: Agent Awareness template +
+migration parity), spec-vs-reality (SHIP — truths T1–T7 all VERIFIED).
+Spec: `docs/specs/provider-substrate-live-wiring.md` (+ `.eli16.md`);
+side-effects: `upgrades/side-effects/provider-substrate-live-wiring.md`.

package/upgrades/side-effects/mesh-pull-brakes.md

@@ -0,0 +1,63 @@
+# Side-Effects Review — Mesh Lease-Transport Brakes
+
+**Version / slug:** `mesh-pull-brakes`
+**Date:** `2026-06-05`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `independent adversarial reviewer subagent — CONCUR with one substantive finding (10s timeout default → false self-suspend risk), APPLIED before commit (30s default + leaseTtlMs-derived wiring)`
+
+## Summary of the change
+
+`HttpLeaseTransport` (the fenced-lease wire) gains the two P19 brakes its deliberate fixed-cadence callers were missing: per-peer state-change failure logging via the new pure `PeerFailureLogGate` (first / every-Nth / recovery; exact bound ⌈F/N⌉+1 lines per streak; default N=360 ≈ 30min at the 5s pull cadence) replacing one-line-per-attempt (≈17k/day per down peer), with previously-silent non-ok responses now gated-logged; and `AbortSignal.timeout` on both fetches (default 30s; `server.ts` derives `min(leaseTtlMs/2, 30_000)`) closing the hung-socket wedge behind the pull loop's `leasePulling` re-entrancy guard. Files: `PeerFailureLogGate.ts` (new), `HttpLeaseTransport.ts`, one wiring line in `server.ts`, two test files.
+
+## Decision-point inventory
+
+- `HttpLeaseTransport.pullPeer` / `broadcast` — **modify (bounded)** — same requests, same return semantics; failures now abort at the timeout instead of hanging forever, and logging is gated.
+- `PeerFailureLogGate` — **add** — a pure log shaper; no authority of any kind.
+- `server.ts` lease-transport construction — **modify (one line)** — passes the config-derived timeout.
+
+## 1. Over-block
+
+The one real over-block risk was found BY the reviewer and closed BEFORE commit: a 10s timeout default sat inside the fleet's documented 5–40s receiver-stall envelope; because `LeaseCoordinator.renew()` treats an unconfirmed broadcast past `leaseTtlMs` as "no medium" and self-suspends the holder, a slow-but-alive peer would have falsely demoted a healthy awake machine on its FIRST slow renewal (renewal cadence 120s > TTL 60s leaves no retry headroom). Applied fix: 30s default (above the stall envelope's bulk; a truly hung socket never returns, so the wedge bound is intact) + derivation from `leaseTtlMs` at the construction site so operator-widened TTLs keep proportion. Residual: a peer consistently slower than 30s per request still reads as unreachable — at that point it is functionally unreachable (the TTL itself is 60s).
+
+## 2. Under-block
+
+(a) Sibling transports (`HttpLiveTailTransport`, `ReplyMarkerTransport`) share the no-timeout + per-attempt-log pattern at lower blast radius (no lease authority; live-tail already has #867's per-topic backoff bounding attempt rate) — next audit targets <!-- tracked: CMT-1109 -->. (b) A peer removed from the registry mid-failure-streak leaves one stale `Map` entry in the gate (never recovers, never deleted) — bounded by historical peer×op count (reviewer: "not worth fixing, worth knowing"). (c) The gate bounds LOG volume, not attempt volume — attempt cadence is the deliberate anti-blinding design and is out of scope by intent.
+
+## 3. Level-of-abstraction fit
+
+Yes. The brakes live in the transport that generates the cost; the cadence stays owned by the caller (`MultiMachineCoordinator`), exactly as the file's contract states. `PeerFailureLogGate` is the established pure-suppressor shape (`AgeKillBackoff`, `SlowRetrySentinelEscalation`) — count-based here (no clock) because the bound should be exact per attempt, not wall-time-dependent. Keying broadcast/pull separately mirrors `isReachable()`'s bidirectional reachability model.
+
+## 4. Signal vs authority compliance
+
+**Required reference:** `docs/signal-vs-authority.md`
+
+- [x] No — the gate shapes LOG output only. The timeout changes failure TIMING (hang → bounded abort), not failure handling: both callers already treated failures as advisisory data. Lease acquire/renew/suspend semantics are byte-identical; the reviewer traced the renewal path end-to-end to confirm the only semantic risk (timeout sizing) and it was resolved.
+
+## 5. Interactions
+
+- **Renewal-requires-medium:** the critical interaction, analyzed in §1 (reviewer probe 1) — resolved by the 30s/config-derived sizing.
+- **Anti-blinding pull loop:** a timed-out pull loses one learning tick and retries in ~5s; never feeds self-suspend. The timeout RESTORES anti-blinding in the hung-socket case (previously wedged forever).
+- **Test clocks:** `AbortSignal.timeout` uses real timers; the injected `now()` drives only reachability windows — orthogonal, no interaction (reviewer probe 2; all suites green).
+- **Log-format consumers:** no test or consumer pinned the old per-attempt strings (reviewer probe 5, grepped src+tests).
+- **Node engines:** `AbortSignal.timeout` needs ≥17.3; package.json engines is ≥20.12 (reviewer probe 4).
+
+## 6. External surfaces
+
+- **Logs only** — fewer lines in the failure steady-state, new (bounded) visibility for rejecting peers and recoveries. No API, schema, message, or notification surface. No topic creation (Bounded Notification Surface untouched).
+- **Persistent state / config / migration:** none. The derivation reads the existing `leaseTtlMs`; `requestTimeoutMs`/`failureLogEveryN` are dep options with in-code defaults.
+
+## 7. Rollback cost
+
+Revert the commit. No state, no config, no schema. Rollback's only observable effect: the log flood and the hung-socket wedge return.
+
+## Conclusion
+
+The transport carrying the mesh's most consequential state (the lease) now satisfies P19: bounded per-attempt cost (log gate), a hard bound on a single attempt's duration (abort timeout), with the deliberately-unbounded cadence left intact and explicitly justified (anti-blinding). The review process did its job visibly: the reviewer's false-self-suspend trace changed the shipped default before commit — recorded in the spec so the sizing rationale survives.
+
+---
+
+## Phase 5 — Second-pass review (lease/coherence-critical → required)
+
+An independent adversarial reviewer ran five probes at line level: (1) false unreachability via the renewal path — FOUND the 10s-default risk (traced `tickLease` 120s cadence → `renew()` → `broadcast` → self-suspend at TTL 60s against the fleet's 5–40s stall envelope) and proposed the exact fix that was then applied (30s default + `min(leaseTtlMs/2, 30s)` wired at `server.ts`); (2) AbortSignal real-timers vs injected test clocks — orthogonal, clean; (3) gate key independence + memory bounds — intended and bounded (one stale-key note, accepted); (4) Node engines — satisfied; (5) old-format/silence pins — none in src or tests. Ran the six lease/transport suites + tsc — all green. **Verdict: CONCUR** (with the recommended change, which is now in the diff).
+
+**Post-CI note:** the original commit was made before this worktree had its husky shims installed (`pnpm install` not yet run → zero hooks → no decision-audit entry staged) — the decision-audit CI gate caught the bypass exactly as designed (#830). This follow-up commit runs through the full gate and carries the entry.

package/upgrades/side-effects/provider-substrate-live-wiring.md

@@ -0,0 +1,145 @@
+# Side-Effects Review — Provider-Substrate Live Wiring (June-15 readiness PR 1)
+
+**Version / slug:** `provider-substrate-live-wiring`
+**Date:** `2026-06-05`
+**Author:** `echo (Claude Opus)`
+**Second-pass reviewer:** `5-agent verification panel (build Phase 3, LARGE)`
+
+## Summary of the change
+
+Wires the already-built provider substrate into production: registers both
+Anthropic adapters with the providers registry at server boot (gated,
+idempotent, lazy), plumbs a real TTL-cached SDK-credit reader into the
+CostAwareRoutingPolicy (replacing the `() => null` stub), and adds an
+opt-in `intelligence.subscriptionPath` mode (`off`/`auto`/`force`) that
+routes the internal-intelligence funnel between `claude -p` (SDK-credit
+path) and the interactive REPL pool (subscription floor) per spec 04
+Rule 1. Files: `src/providers/bootRegistration.ts` (new),
+`src/core/InteractivePoolIntelligenceProvider.ts` (new),
+`src/core/AnthropicSubscriptionRouter.ts` (new),
+`src/providers/costAwareRouting.ts` (decision extracted to shared pure fn),
+`src/providers/adapters/anthropic-interactive-pool/{config,pool}.ts`
+(model knob), `src/core/intelligenceProviderFactory.ts` (option),
+`src/core/types.ts` (config type), `src/commands/server.ts` (boot+shutdown
+wiring), `src/server/routes.ts` (GET /providers/registry).
+
+## Decision-point inventory
+
+- `registerAnthropicAdapters` gates (claudeForbidden, enabledFrameworks) — add — refuse Claude adapters on codex-only agents
+- `decideSdkVsSubscription` (extracted) — modify (refactor, semantics identical, existing tests pass) — SDK-pot vs subscription threshold
+- `AnthropicSubscriptionRouter.evaluate` mode branch — add — off/auto/force routing of internal LLM calls
+- `buildIntelligenceProvider` claude-code case — modify — wraps with router ONLY when the new option is passed; otherwise byte-identical
+- shutdown pool dispose — add — kills pool tmux sessions at server stop
+- `GET /providers/registry` — add — read-only introspection
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject that it shouldn't?**
+
+In `force` mode, a pool outage (tmux missing, spawn failure) makes internal
+LLM calls fail even though `claude -p` would have worked — by design (force
+mode's contract is zero `claude -p` traffic), and loudly. Fleet default is
+`off`, so nobody is exposed without an explicit flip. The codex-only gate
+refuses registration on codex-only agents — correct, mirrors the existing
+ClaudeCliIntelligenceProvider guard. No other block/allow surface.
+
+## 2. Under-block
+
+**What should be rejected but passes?**
+
+`auto` mode falls back across paths on ANY primary error, including
+prompt-shaped errors (e.g. timeout from an oversized prompt) where the
+retry will likely fail again — one wasted call, bounded (exactly one
+fallback attempt, then a loud throw). The pool's `--dangerously-skip-
+permissions` spawn means a prompt-injected judgment call could in principle
+invoke tools inside the pool session; mitigated by the empty scratch
+workdir and unchanged from the adapter's prototyped+parity-tested shape;
+hardening is tracked in the spec (CMT-1105).
+
+## 3. Level-of-abstraction fit
+
+Registration lives in `src/providers/` (substrate layer), the two
+IntelligenceProvider implementations in `src/core/` next to their peers,
+and the mode wiring at the composition root (server.ts) — matching the
+existing layering (IntelligenceRouter precedent). The threshold logic was
+EXTRACTED to one shared pure function rather than duplicated across the
+two routing layers.
+
+## 4. Signal vs authority compliance
+
+The router treats config (`mode`) as authority and credit snapshots as
+signal: unknown signal degrades conservatively (subscription floor), never
+blocks the call path. onRoute/onDegrade are observability taps with no
+authority. No LLM judgment gates any decision here — all routing is
+deterministic from config + credit state.
+
+## 5. Interactions
+
+- **Circuit breaker**: router sits INSIDE the breaker wrap — a rate-limit
+  on the surviving path still trips the account-global breaker. Unchanged
+  for mode off.
+- **Per-component IntelligenceRouter**: claude-code builds inherit the same
+  subscriptionPath option, so codex-default agents with claude-routed
+  components stay consistent.
+- **/metrics/features**: pool-served calls still attribute calls+latency;
+  token columns read 0 (pool reports no per-call usage; onUsage is
+  deliberately not invoked with fake zeros).
+- **SessionReaper/tmux tooling**: pool sessions are named `instar-pool-*`;
+  they are adapter-managed (maxIdle 30m, maxMessages 50, dispose at
+  shutdown). Reaper does not manage them (they are not instar sessions in
+  the session store).
+- **QuotaTracker**: unaffected — file-based scheduler shedding stays as-is.
+
+## 6. External surfaces
+
+`GET /providers/registry` (Bearer-authed like its router peers) returns
+adapter ids + capability flag names + a policy-installed boolean. No prompt
+content, no credentials; integration test asserts no key-material shapes in
+the payload. The usage-meter call (`/api/oauth/usage`) is the EXISTING
+read-only observability exception under spec 04 Rule 2, now TTL-cached
+(60s) so routing volume cannot hammer it.
+
+## 7. Rollback cost
+
+Mode is config-gated default-off: rollback = remove the config key (or set
+`off`) + restart — no data, no state, no migration. Registration itself is
+inert without the mode (lazy pool, no spawns). Worst-case orphan: pool tmux
+sessions if the server dies UNgracefully mid-soak; bounded by poolSize
+(default 2) and visible via `tmux ls` (`instar-pool-*`).
+
+## Conclusion
+
+Ship. Default-off + pinned-argv invariance test means zero fleet behavior
+change at merge; the June-15 lever becomes a config flip backed by 44 new
+tests across all three tiers.
+
+## Second-pass review (if required)
+
+Build Phase 3 runs the 5-agent verification panel (LARGE build) before
+commit; findings folded back into code/spec before the convergence tag.
+
+## Evidence pointers
+
+- Unit: `tests/unit/providers/bootRegistration.test.ts`,
+  `tests/unit/anthropic-subscription-router.test.ts`,
+  `tests/unit/intelligence-provider-factory-subscription-path.test.ts`,
+  `tests/unit/providers/adapters/anthropic-interactive-pool/pool-model-flag.test.ts`
+- Integration: `tests/integration/providers-registry-route.test.ts`
+- E2E: `tests/e2e/provider-substrate-live-wiring.test.ts`
+- Spec: `docs/specs/provider-substrate-live-wiring.md` (+ `.eli16.md`)
+- Live exposure measurement driving the work: /metrics/features 24h on echo
+  (~1,014 real internal calls, ~26.7M tokens-in).
+
+## CI-green follow-up (same PR)
+
+- New side effect: a FAILED boot registration/policy install now emits a
+  DegradationReporter event (`serverBoot.anthropicProviderRegistration`) in
+  addition to the yellow boot log line — so a dark June-15 routing install is
+  visible in the degradation feed, not just scrollback. No behavior change on
+  the success path.
+- Silent-fallback ratchet lowered 459 → 458 (that catch is now reporter-wired);
+  two non-degradation catches carry in-brace `@silent-fallback-ok`
+  justifications (raced tmux kill-session; HTTP 500 surfaced to caller).
+- `/providers/registry` tracked in the feature-delivery-completeness guard
+  (read-surface class, like `/session/clock`; no framework shadow — the lever
+  only applies to claude-code internal traffic).