instar 1.3.640 → 1.3.641

package/upgrades/1.3.641.md

@@ -0,0 +1,51 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: minor -->
+
+## What Changed
+
+The last rung of the rate-limit principle: **wait for capacity before falling back.** When an internal
+background check gets rate-limited and has already tried slowing-down-and-retrying and switching
+frameworks, it can now WAIT in a small queue for capacity to free up — instead of immediately giving up
+and dropping to a basic rule. The wait is bounded and rate-aware (it rides the same circuit-breaker
+timing that protects the account), so a burst of queued checks can't re-trip the limit the moment it
+recovers. Only background, non-urgent checks queue; anything the agent is actively waiting on still
+fails fast. If the queue is full or over its daily budget, the check falls through to its basic rule
+exactly as before — never silently dropped.
+
+This is **off by default** (on for the dev agent first), reuses the existing wedge-safe queue, and
+changes nothing for any other queue user.
+
+## What to Tell Your User
+
+Nothing changes unless you turn it on. Once on, if a background check hits a rate limit, it will
+briefly wait its turn for capacity to come back before falling back to a simpler rule — so you lose the
+smart answer less often. Urgent checks you are waiting on still respond fast, and if there is no spare
+capacity the check just uses its basic rule like before.
+
+## Summary of New Capabilities
+
+- `IntelligenceRouter` DEFERRABLE queue rung (`tryDeferrableQueue`): a non-gating call that exhausts
+  backoff + framework-swap enqueues onto a dedicated `LlmQueue` (background lane) and WAITS for
+  capacity (the enqueued `provider.evaluate` honors the account-global breaker's `retryAfterMs`)
+  before dropping to the caller's heuristic. Inserted at both fall-through points.
+- A GATING call is NEVER queued (`deferrable = !gating && …`, code-enforced — D5). An enqueue
+  rejection (daily-cap / interactive-reserve) or queued-call failure falls through to the heuristic,
+  never dropped. Two new `onDegrade` reasons (`queued` / `queue-rejected`) for `/metrics/features` (D7).
+- `LlmQueue` opt-in `backgroundDispatchMinGapMs` (§3c herd guard): a jittered minimum gap between
+  background dispatches; interactive bypasses; default 0 = off (zero change for existing callers).
+- Config `intelligence.degradationLadder.queue` → `{ enabled?, attemptTimeoutMs?, drainMinGapMs? }`;
+  `DEV_GATED_FEATURES` entry `degradationLadderQueue` (live-on-dev / dark-on-fleet).
+
+## Evidence
+
+**Before:** a deferrable internal call that exhausted backoff + framework-swap dropped STRAIGHT to its
+heuristic — even when capacity would have freed up moments later. **After:** with the rung enabled, it
+WAITS for capacity in a bounded, rate-aware queue first, and only falls back if the queue rejects it or
+the queued call fails. Reproduced by `tests/unit/degradation-ladder.test.ts` (7 queue cases: success +
+auto-resolve; order backoff/swap → queue → success; GATING never queued; enqueue-rejection ⇒ heuristic
+fallthrough; non-deferrable never queued; no-llmQueue no-op; queueEnabled:false not enqueued) and
+`tests/unit/LlmQueue.test.ts` (3 pacing cases). The rung reuses the existing wedge-safe `LlmQueue` and
+never calls the DegradationReporter alert machinery, so the 2026-06-21 event-loop wedge cannot recur.
+Full `tsc` + dark-gate lint green; `no-silent-fallbacks` holds at baseline 476.

package/upgrades/side-effects/resilient-degradation-ladder-increment-3.md

@@ -0,0 +1,108 @@
+# Side-Effects Review — Resilient Degradation Ladder Increment 3 (queue rung)
+
+**Slug:** `resilient-degradation-ladder-increment-3` · **Tier:** 2 (spec-driven; the spec converged +
+operator-approved, rode Increments 1/2). **Spec:** `docs/specs/resilient-degradation-ladder.md` §3b.3,
+§3c, D1/D5/D7.
+
+## Summary of the change
+
+The third (and final) rung of the operator's degradation principle — "prefer slowing down over
+falling back". Dark/dev-gated:
+
+- `IntelligenceRouter` gains a DEFERRABLE **queue rung** (`tryDeferrableQueue`): a non-gating call
+  that has exhausted backoff + framework-swap now WAITS for capacity in a dedicated `LlmQueue`
+  (`background` lane) before dropping to the caller's heuristic. The enqueued `provider.evaluate`
+  honors the account-global breaker's `retryAfterMs` via `acquireOrWait` — that IS the §3b.3
+  rate-awareness. Inserted at BOTH fall-through points (no-swap-configured AND swap-exhausted) before
+  the existing `onHeuristicFallthrough` + throw.
+- A structural `DeferrableQueue` interface declared in `core` (the real `LlmQueue` satisfies it) so
+  `core` does NOT import `monitoring` (no layering cycle).
+- `LlmQueue` gains an opt-in `backgroundDispatchMinGapMs` (§3c herd guard): a jittered minimum gap
+  between BACKGROUND-lane dispatches; interactive bypasses. Default 0 = OFF (today's greedy drain).
+- Server wires a DEDICATED `LlmQueue` for the rung (built ONLY when the gate-resolved `queueEnabled`
+  is true), gate-resolves `queueEnabled`/`queueAttemptTimeoutMs` via `resolveDevAgentGate`, and the
+  ladder object is now present when ANY rung (backoff OR queue) is active.
+- `DEV_GATED_FEATURES` registration (`degradationLadderQueue`).
+- Config type extended: `degradationLadder.queue` → `{ enabled?, attemptTimeoutMs?, drainMinGapMs? }`.
+
+## Decision-point inventory (frozen)
+
+D1 (config + dark rollout), D5 (`gating:true` ⇒ queue-skipped, code-enforced), D7 (reason taxonomy
+`queued` / `queue-rejected`). Increment-3 bounds (sensible defaults, dark/reversible, fit D1's
+`{enabled, …bounds}` structure): `queueAttemptTimeoutMs` 60000, dedicated queue `maxConcurrent` 1 +
+`maxDailyCents` 25, `drainMinGapMs` 0 (off).
+
+**D9 (new, recorded here) — the §3c herd guard is opt-in, realized two ways:** (1) the PRIMARY
+rate-awareness is the provider-layer `acquireOrWait` — each enqueued `evaluate` waits for the
+account-global breaker window, so queued calls don't hammer a rate-limited account; (2) the dedicated
+queue's `maxConcurrent: 1` serializes deferrable retries (naturally herd-safe); (3) `drainMinGapMs`
+adds an OPTIONAL jittered inter-dispatch gap (off by default). Explicit pacing is opt-in so existing
+`LlmQueue` callers (PresenceProxy / PromiseBeacon) see ZERO behavior change. This faithfully delivers
+§3b.3/§3c without rebuilding shared infrastructure (round-1 "extend, don't rebuild").
+
+## 1. Over-block / false positive
+
+The rung only ADDS a wait-for-capacity step on a DEFERRABLE call that already exhausted swap; it never
+blocks an interactive/gating call. A queued call that fails or is rejected falls through to exactly
+today's heuristic — strictly no worse than before. No new block surface.
+
+## 2. Under-block
+
+A GATING call NEVER reaches the queue rung: `deferrable = !gating && options.deferrable === true`, so
+`if (deferrable) …tryDeferrableQueue` is structurally unreachable for a gate (D5). Unit-tested
+(`GATING is NEVER queued`). The gating fail-closed boundary is unchanged.
+
+## 4. Signal vs authority
+
+The queue rung takes no destructive action — it waits for capacity then returns a real answer, or
+falls through. The two `onDegrade` emissions (`queued` / `queue-rejected`) are observability signals
+(D7), never gates. Consistent with Signal-vs-Authority.
+
+## 5. Interactions — wedge-safety
+
+Reuses the EXISTING `LlmQueue` (the wedge-safe, bounded, daily-capped queue). The new drain-pacing is
+a bounded `setTimeout` coalesced to at most one pending timer (no stacking), `unref`'d (never holds the
+process open). No new recursion, no growing array, no `report()`/`reportEvent()` call from the rung —
+the 2026-06-21 DegradationReporter wedge class cannot recur here. The dedicated queue is built only
+when the rung is active, so the fleet allocates nothing.
+
+## 6. External surfaces
+
+No new route. No new egress — a queued call is the SAME provider.evaluate that would otherwise have
+run; it just waits for capacity. The new config (`intelligence.degradationLadder.queue.*`) is opt-in.
+
+## Framework generality
+
+Framework-agnostic — the rung keys on the resolved framework and enqueues whatever provider the
+component routes to.
+
+## 7. Multi-machine posture
+
+Machine-local: each machine's router + its dedicated queue are independent. No replicated state, no
+cross-machine contract (the dedicated queue is per-process, same posture as the existing per-sentinel
+queues).
+
+## 8. Rollback cost
+
+Trivial: dark on the fleet (`queueEnabled` resolves dark; no dedicated queue is built, the ladder's
+`queueEnabled` is false, and `tryDeferrableQueue` returns `{ ok: false }` immediately ⇒ exactly
+today's heuristic-on-exhaustion). Revert = remove an unused-on-fleet code path. `drainMinGapMs` defaults
+0 so existing `LlmQueue` callers are untouched.
+
+## Evidence pointers
+
+- `tests/unit/degradation-ladder.test.ts` (+7 queue cases): queue success + onResolved; order
+  backoff/swap → queue → success; GATING never queued (D5); enqueue REJECTION ⇒ heuristic fallthrough
+  + onHeuristicFallthrough; non-deferrable never queued; queueEnabled-but-no-llmQueue no-op;
+  queueEnabled:false not enqueued.
+- `tests/unit/LlmQueue.test.ts` (+3 pacing cases): paces a 2nd background dispatch when the gap is set;
+  no pacing when off (both start at once); interactive bypasses pacing.
+- `tests/unit/devGatedFeatures-wiring.test.ts` auto-covers `degradationLadderQueue` (live-on-dev /
+  dark-on-fleet). `tests/unit/no-silent-fallbacks.test.ts` stays at baseline 476. Full `tsc` +
+  `lint-dev-agent-dark-gate` green.
+
+## Conclusion
+
+Delivers the final rung of the operator's degradation principle — a deferrable call now WAITS for
+capacity (slow down) before ever dropping to a brittle heuristic, with the provider breaker supplying
+rate-awareness and an opt-in herd gap on top. Dark/dev-gated, no-op when off, wedge-safe. Ship.