instar 1.3.783 → 1.3.785

package/upgrades/1.3.784.md

@@ -0,0 +1,59 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+- **`SpendAlertDispatcher`** (`src/core/SpendAlertDispatcher.ts`): lane-scoped dedup (money-critical
+  cap-hit/holder-dead vs informational — S-F8) + informational coalescing into ONE digest per window
+  + edge latch on CONFIRMED delivery only + dryRun-first + the scrubbed
+  `logs/routing-spend-alerts.jsonl` audit (metadata only — S-F7).
+- **`TelegramSpendTopicChannel`** (`src/core/TelegramSpendTopicChannel.ts`): message-INTO the ONE
+  "💰 Routing & Spend Alerts" topic; money-critical kinds prefer the DURABLE relay
+  (`PendingRelayStore` → `DeliveryFailureSentinel`, retry-until-delivered); lifeline fallback on ANY
+  failure; a repoint of `alerts.telegramTopicId` is audible in BOTH topics (G5).
+- **`SpendAlertEmitters`** (`src/core/SpendAlertEmitters.ts`): cap-approach 50/80% on BOTH caps (G4),
+  cap-hit on gate refusal (A-Min13 wording), door-dark with P19 brakes (episode budget = chain
+  length, widening backoff, flapping escalation), fallback-spike only at the hourly ceiling crossing
+  (Near-Silent), holder-dead surviving-voice (A2-2, stable pool-wide key), recon-drift surface
+  (fed by the Layer-1c sweep, next PR).
+- **Resolver rung 2 pool half (FD-6):** the created topic id is published as a content-free
+  `routingSpendAlertTopicId` field on the replicated machine registry
+  (`MachineIdentityManager.updateRoutingSpendAlertTopic` / `readAnyRoutingSpendAlertTopic`) — a
+  future serving-lease holder INHERITS the id instead of re-creating; a degraded registry read
+  falls through the lease-fenced ladder (never a duplicate mint).
+- **Router fan-out (I-9):** `onNatureRoutePlan` now fans to the env-gated console breadcrumb + the
+  emitters with per-subscriber throw-swallow; served fallbacks are counted from the existing
+  `onDegrade` seam.
+- **Gate observer:** `MeteredSpendGate` gains an OPTIONAL signal-only `onGateEvent` (admit +
+  cap-exceeded refusal), throw-swallowed — the admit/refuse path is unchanged (unit-pinned).
+- Self-action convergence models for the three new notifiers; config keys
+  `routingSpend.alerts.enabled` / `alerts.dryRun`.
+
+## What to Tell Your User
+
+Nothing yet — the alert layer ships in dry-run soak: it decides and audits but delivers nothing.
+Once flipped live (after the soak), every routing/spend notice — cap warnings at 50/80%, cap hits,
+a routing chain going fully dark, fallback-rate spikes — will land in exactly ONE "💰 Routing &
+Spend Alerts" Telegram topic (never a new topic per event), with only genuinely money-critical
+items escalating immediately and everything else coalesced into a quiet digest.
+
+## Summary of New Capabilities
+
+- (⚗️ Experimental, dryRun soak) The one-topic spend/routing alert layer: lane-deduped, digest-
+  coalesced, durable-delivery for money-critical kinds, pool-durable topic identity. Layer 1c
+  provider reconciliation (which feeds the drift alert) and the operator's amortized-subscription
+  display land next in this train (tracked: CMT-1929).
+
+## Evidence
+
+- `tests/unit/spend-alert-dispatcher.test.ts` — lanes, dryRun default, coalescing, edge latch
+  (confirmed-only), digest-failure un-latch, channel isolation, durable-relay preference, lifeline
+  fallback, G5 repoint, rung-2 pool inherit/publish/degraded-read.
+- `tests/unit/spend-alert-emitters.test.ts` — the full trigger matrix + P19 brakes + observer isolation.
+- `tests/integration/spend-alert-pipeline.test.ts` — the assembled pipeline: dryRun soak, durable
+  relay into the resolver-created topic, digest delivery, lifeline degradation.
+- `tests/e2e/routing-spend-lifecycle.test.ts` — the alert layer constructs on the REAL AgentServer
+  boot path on a dev agent (dryRun-first pinned) and stays DARK on the fleet.
+- `tests/unit/self-action-convergence.test.ts` — 47/47 with the three new models.

package/upgrades/1.3.785.md

@@ -0,0 +1,56 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+- **`ProviderCostReportStore`** (`src/monitoring/ProviderCostReportStore.ts`): the immutable
+  append-only provider-report record set + reconciliation records — `meteredCallId` join key,
+  supersession by greatest `capturedAt` (a late OpenRouter `/generation` cost never double-counts),
+  receive clamps (invalid rows audit-preserved + aggregate-excluded), declared 400d retention with
+  the batched prune, registered in `state-coherence-registry.json` WITH retention at birth.
+- **`extractProviderReport`**: the per-door capture parser — OpenRouter `usage.cost` +
+  cached-token detail + generation id; Groq token counts; Gemini native (`candidatesTokenCount` +
+  `thoughtsTokenCount`, erring HIGH) and OpenAI-compat shapes. Absent/reshaped fields degrade to
+  token-only or nothing — a first-class NORMAL state.
+- **`ProviderReconciliationSweep`** (`src/monitoring/ProviderReconciliationSweep.ts`): the
+  cadenced (default 6h) REPORTING-side cross-check — signed `driftPct` per (keyRef, door) window,
+  committed-figure enrichment on the metered-lease holder (via the READ surface, never the money
+  lock), drift ≥ `reconciliation.driftAlertPct` (default 10%) feeding the already-live
+  `SpendAlertEmitters.onReconciliationDrift` (Increment-C informational lane, dispatcher-latched).
+- **`feature_metrics.callId`** — the per-call join column, full I-4 discipline (column + type +
+  writer + INSERT), stamped only by the future metered dispatch.
+- **`GET /routing-spend/reconciliation`** — the drift-record read route (same dev gate as the
+  view); the spend summary now PREFERS provider-reported cost per row where reports exist
+  (`costBasis: "provider-reported"`, `providerReportedUsd`, `providerDriftPct`).
+- **FD-21 structural exclusion as a TEST:** the money gate/ledger modules reference nothing from
+  the provider store — pinned by `tests/unit/provider-cost-report-store.test.ts`.
+- Self-action convergence model `spend-recon-sweep` (24h-latch eternal sentinel).
+
+## What to Tell Your User
+
+Nothing yet — this layer is bookkeeping that stays empty until paid routing actually dispatches
+calls (a separate, still-unbuilt piece). Once real paid calls flow, your Spend tab's dollar figures
+will prefer what the PROVIDER says it charged (where the provider reports it), and a scheduled
+cross-check will quietly flag any drift between the provider's numbers and ours — so a stale price
+gets noticed and fixed instead of silently mispricing your books.
+
+## Summary of New Capabilities
+
+- (⚗️ Experimental, dark, honestly-empty) Provider-grounded cost reporting: the immutable
+  provider-report store, the per-door capture parser, the reconciliation sweep + drift alerts, and
+  the provider-preferred spend basis. Remaining in this train (tracked: CMT-1929): PR 4 — the
+  amortized subscription display with visible derivation math + scheduled web-research price checks.
+
+## Evidence
+
+- `tests/unit/provider-cost-report-store.test.ts` — 18 cases: clamps, supersession/no-double-count,
+  per-door extraction (incl. the Gemini thinking-token trap), signed drift + threshold + Near-Silent
+  below it + zero-internal guard + never-throws, the FD-21 structural exclusion, and the callId
+  end-to-end column discipline.
+- `tests/integration/routing-spend-routes.test.ts` — the reconciliation route (dark-503/live-200)
+  and the provider-preferred summary basis over real stores.
+- `tests/e2e/routing-spend-lifecycle.test.ts` — the reconciliation surface constructs on the REAL
+  AgentServer boot path (200 + the DB file on disk, honest empty records).
+- `tests/unit/self-action-convergence.test.ts` — 50/50 with the new model.

package/upgrades/side-effects/routing-spend-increment-1c.md

@@ -0,0 +1,44 @@
+# Side-Effects Review — Routing Control Room Layer 1c (provider-reported cost + reconciliation)
+
+**Spec:** `docs/specs/routing-control-room-spend-alerts.md` (review-convergence r7, approved, parent-principle: Token-Audit Completeness) — §Layer 1c + FD-21.
+**Worktree:** `echo/money-increment-1c` off `JKHeadley/main` @ `7c2064163` (contains merged Increments B + C).
+**Scope of this PR (PR 3 of the train, tracked CMT-1929):** the provider-grounded REPORTING anchor — `ProviderCostReportStore` (immutable append-only provider reports + reconciliation records, receive-clamped, 400d declared retention, `meteredCallId` join key with capturedAt supersession), the `callId` column on `feature_metrics` (full I-4 discipline: column + type + writer + INSERT), the per-door provider-report field extraction (`extractProviderReport` — OpenRouter usage.cost mandatory-include posture; Groq/Gemini token counts incl. the thinking/cached details), the capture SEAM the future metered dispatch calls strictly post-settle, the cadenced `ProviderReconciliationSweep` (reporting-side, NEVER the money lock; signed driftPct feeding the ALREADY-LIVE `SpendAlertEmitters.onReconciliationDrift`), the `GET /routing-spend/reconciliation` read route, and the summary composer's provider-preferred `costBasis` enrichment. The operator additions (amortized subscription display + web-research price checks) are PR 4.
+
+## Phase 1 — Principle check (signal-vs-authority)
+
+**Does this change involve a decision point?** No blocking decision anywhere: everything here is REPORTING truth. The load-bearing invariant is the opposite direction — the spec's structural exclusion (FD-21, the FD-9/FD-12 twin): NO provider report, reconciliation record, or drift figure is ever read by the money gate or the reserve/settle path. This PR ships that as a STRUCTURAL TEST (the gate/ledger modules import nothing from the provider store) plus the one-way drift rule: provider-LOWER only changes the report; provider-HIGHER raises a drift signal feeding the human-reviewed PIN price-promotion path — the committed counter is never rewritten in either direction. Compliant: signals only.
+
+## Phase 2 — Plan
+
+- **Decision points touched:** none with authority. The sweep DECIDES only whether a drift figure crosses the alert threshold — feeding the Increment-C dispatcher (itself notification-only, dryRun-soaked).
+- **Existing detectors/authorities interacted with:** `FeatureMetricsLedger` (additive `callId` column, same ensureAddedColumns discipline as `door`); `RoutingPriceAuthority` (read-only as-of pricing for the internal-derived side of the comparison); `SpendAlertEmitters.onReconciliationDrift` (already live, already convergence-modeled); the caps read surface (committed totals for booked-vs-reported — read-only, never the per-key money mutex); `state-coherence-registry.json` (the new store registered WITH declared retention — the Bounded Accumulation ratchet is satisfied at birth, never grandfathered).
+- **Store shape decision (documented deviation):** the spec says "a small SQLite table beside the rollup"; this ships as a small SQLite DB FILE beside the rollup's DB (`server-data/provider-cost-reports.db`) rather than a table inside `feature-metrics.db` — one owner per file, no cross-class schema coupling, identical query surface. Volume is bounded by metered calls only.
+- **Rollout:** rides the EXISTING dev-gated `routingSpend` view flag (FD-21: "capture + display in Increment A" posture — the store/route are read-only observability like the summary); the sweep additionally requires the reconciliation config (defaults on where the view is on; inert until provider reports exist, which requires the out-of-scope metered dispatch). No paid door can route regardless (FD-11 unchanged).
+- **Rollback path:** additive; disabling `routingSpend.enabled` reverts byte-for-byte (store not constructed, route 503, sweep never ticks). The DB file is inert data at rest.
+
+## Phase 4 — Side-effects review
+
+1. **Over-block:** N/A — nothing blocks. Over-REPORTING risk: a malformed provider body could pollute the reports; closed by receive-clamps (finite ≥0 numerics or the row is stored as `invalid: 1` with nulls — preserved for audit, excluded from every aggregate), length/charset clamps on strings, and HTML-escape at any render.
+2. **Under-block / missed data:** capture is best-effort BY SPEC (a swallowed append never blocks the call and never shares a failure domain with the fail-closed settle — the seam contract); a lost report degrades that row to `internal-derived`, a first-class labeled basis, and the reconciliation surface is what makes systematic loss visible.
+3. **Level-of-abstraction fit:** store + sweep live in `src/monitoring/` beside FeatureMetricsLedger (reporting side); the gate/ledger in `src/core/` remain import-free of them (structural test). The route rides the existing `/routing-spend/*` dev-gated family.
+4. **Signal vs authority:** compliant (Phase 1) — with the exclusion invariant unit-pinned.
+5. **Interactions:** the reserve-expiry sweep (money-side, takes the per-key mutex) and this provider-reconciliation sweep (reporting-side, never the lock) are named distinctly everywhere per the spec's vocabulary; the recon-drift alert rides the Increment-C informational lane and its already-registered convergence model. Supersession by `(meteredCallId, greatest capturedAt)` means a late `/generation` cost can never double-count.
+6. **External surfaces:** one new Bearer read route (503 when dark). No egress — the sweep reads local stores only; the drift ALERT egresses through the C dispatcher (dryRun-soaked).
+7. **Multi-machine posture:** `proxied-on-read` like `feature_metrics` (each machine records its own calls' reports; pool merge is a tracked follow-up with the pool summary <!-- tracked: CMT-1929 -->); the booked-vs-reported comparison that involves the money ledger runs on the metered-lease holder by construction (it reads the local caps/ledger read surface, which only the holder has live values for — a non-holder's sweep sees $0 committed and records the comparison as provider-vs-internal only).
+8. **Rollback cost:** config revert; no migration; the DB file and its records are inert audit history.
+
+## Phase 5 — Second-pass review
+
+Not-required by the Phase-5 trigger list (no block/allow messaging decisions, no session lifecycle, no gate/sentinel authority — this is read-side reporting; the one gate-adjacent surface is the STRUCTURAL EXCLUSION test, which reduces gate blast radius). Tier-2 chain otherwise complete; the FD-21 invariant is enforced by test rather than reviewer eyes.
+
+## Self-action convergence (unbounded-self-action — closure: guard)
+
+One new self-triggered behavior: the reconciliation sweep's drift ALERT — already covered by the registered `spend-recon-drift` emit surface riding the Increment-C dispatcher latch (24h re-arm per (keyRef, door, driftBucket)) and the existing dispatcher convergence posture; the sweep itself is a fixed-cadence read-only pass (no feedback loop: its output never changes its input — prices and reports are external facts). Registered model: `spend-recon-sweep` pins the once-per-bucket emission under permanently-drifting pressure.
+
+## No-deferrals accounting
+
+PR 4 (amortized subscription display with visible derivation math + scheduled web-research price checks) and the pool-scope merge of the reconciliation view are the remaining tracked items of this train, tracked under CMT-1929 <!-- tracked: CMT-1929 --> and enumerated in `.instar/plans/money-increment-b-brief.md`; nothing in THIS PR's claimed scope is partial.
+
+## Build outcome (staged with the change)
+
+Delivered exactly as reviewed: 18 unit + 2 integration + 1 e2e new tests green; FD-21 exclusion pinned; retention lint green with the store registered (never grandfathered); self-action ratchet 50/50 with `spend-recon-sweep`; docs-coverage floors held (class 55 / route 55).

package/upgrades/side-effects/routing-spend-increment-c.md

@@ -0,0 +1,44 @@
+# Side-Effects Review — Routing Control Room Increment C (alerts)
+
+**Spec:** `docs/specs/routing-control-room-spend-alerts.md` (review-convergence r7, approved, parent-principle: Token-Audit Completeness)
+**Worktree:** `echo/money-increment-c` off `JKHeadley/main` @ `220ebeb0b` (v1.3.783 — contains the merged Increment B) — remotes + version verified per Phase 2.
+**Scope of this PR:** Increment C — the channel-abstracted alert layer: `SpendAlertDispatcher` (lane-scoped dedup + coalescing BEFORE any channel send, edge latch on CONFIRMED delivery, dryRun-first, scrubbed jsonl audit), `TelegramSpendTopicChannel` (message-INTO the ONE dedicated topic; durable-relay path for money-critical kinds; lifeline fallback on ANY failure; audible repoint), the emitter set (cap-approach 50/80% on BOTH caps, cap-hit, door-dark with P19 brakes, fallback-spike rate detection, recon-drift surface, holder-dead surviving-voice), the router-signal fan-out (I-9, observer isolation), the gate event hook, and the POOL-PUBLISHED rung-2 topic record on the machine registry (FD-6). Layer 1c reconciliation (which feeds recon-drift) and the operator additions are the NEXT PRs (tracked: CMT-1929).
+
+## Phase 1 — Principle check (signal-vs-authority)
+
+**Does this change involve a decision point?** The alert layer decides WHETHER TO NOTIFY, never whether to act: it blocks no call, gates no money, and constrains no agent behavior. The only authority-shaped logic is the dispatcher's dedup/coalescing (bounded-notification-surface protection) — deterministic counters and latches whose worst failure is a suppressed or extra NOTIFICATION, never a suppressed action. The gate hook is signal-only (throw-swallowed; the gate's admit/refuse path is unchanged — unit-pinned). Compliant: signals feeding a notification surface, no brittle logic holding blocking authority.
+
+## Phase 2 — Plan
+
+- **Decision points touched:** none with blocking authority. The dispatcher's lanes (money-critical vs informational) govern coalescing only — both lanes land in the SAME dedicated topic (Amendment 2).
+- **Existing detectors/authorities interacted with:** `SpendAlertResolver` (extended with the pool-published rung-2 read/publish — creation stays serving-lease-fenced); `MeteredSpendGate` (gains an OPTIONAL `onGateEvent` observer — throw-swallowed, never on the refusal path); `onNatureRoutePlan` (single callback → small fan-out preserving observer isolation, I-9); `PendingRelayStore` (money-critical alerts ride the existing durable relay via injected enqueue); `MachineIdentity` registry (a content-free numeric `routingSpendAlertTopicId` field riding the SAME authenticated registry-sync path as `lastKnownUrl`/`endpoints`).
+- **Rollout (FD-16):** Increment C ships **dryRun-first live-on-dev** — `routingSpend.alerts.enabled` rides `resolveDevAgentGate` (dark on the fleet), and `alerts.dryRun` defaults TRUE even on dev (would-send lines to the scrubbed jsonl, nothing delivered) until a deliberate flip.
+- **Rollback path:** additive + dev-gated; disabling `alerts.enabled` (or leaving dryRun) reverts to Increment-B behavior byte-for-byte (the B stale-price cadence keeps its direct resolver path when the dispatcher is absent). No state migration; the registry field is inert data.
+
+## Phase 4 — Side-effects review
+
+1. **Over-block:** N/A for actions. For notifications: the per-kind edge latch + coalescing can suppress a repeat alert while a condition persists — by design (Bounded Notification Surface / Near-Silent Notifications); money-critical kinds ride a DISTINCT dedupe lane so a flapping door's volume can never coalesce a cap-hit into a digest (S-F8).
+2. **Under-block (missed notifications):** a transient send failure does NOT latch (stays eligible); money-critical kinds additionally fall back to the lifeline on ANY failure and ride the durable relay when wired — the failure mode is a DELAYED alert, never a silent drop. dryRun is the deliberate exception: it delivers nothing and says so in the jsonl (the FD-16 soak posture).
+3. **Level-of-abstraction fit:** dispatcher/channel/emitters live beside the B modules in `src/core/`; the fan-out lives at the single existing `onNatureRoutePlan` wiring point; NO new routes (the read surface is the jsonl + existing /guards posture). The `/attention` queue is deliberately NOT used (Amendment 2 — topic-per-item is the flood the operator forbids).
+4. **Signal vs authority:** compliant (Phase 1) — everything here is a signal consumer/notifier.
+5. **Interactions:** the dispatcher subsumes B's resolver-direct emission for stale-price/observed-drift (single voice — no double-fire: the cadence now emits THROUGH the dispatcher when present); the fan-out preserves the env-gated console breadcrumb; the gate hook never fires on the ledger path (only post-verdict); holder-dead is the ONE named exception to holder-single-voice (A2-2) and is keyed `spend-holder-dead:<keyEpoch>` pool-wide so two survivors dedupe.
+6. **External surfaces:** Telegram messages into the ONE dedicated topic (or lifeline). The registry field is content-free (a topic id number). No other egress.
+7. **Multi-machine posture:** topic identity becomes genuinely pool-durable this increment (FD-6 rung 2): the auto-created id is persisted machine-locally AND published as a content-free field on the replicated machine registry, so a future serving-lease holder INHERITS the id instead of re-creating; creation stays serving-lease-holder-only + single-flight + fenced. holder-dead emission is explicitly a surviving-machine act (single-machine: strict no-op). Emitter state (latches, backoff, spike baselines) is machine-local BY DESIGN — each machine notifies for its own observations; the dedupe keys carry machineId where cross-machine duplication is possible (door-dark), and pool-wide keys where it is not acceptable (holder-dead).
+8. **Rollback cost:** config-flag revert; no data migration; the jsonl and registry field are inert at rest.
+
+## Phase 5 — Second-pass review
+
+Required (touches messaging decisions — the dispatcher decides notification delivery). Cross-model independent audit; verdict appended below.
+
+**Reviewer verdict:** `VERDICT: Concur with the review. The provided artifact and code demonstrate robust mechanisms against blocking actions, silent drops of money-critical alerts, and notification floods, while adhering to signal-vs-authority principles.` (gemini cross-model, 2026-07-08 — scope: this artifact + SpendAlertDispatcher + TelegramSpendTopicChannel in full; questions: blocking-vs-notify, silent-drop of money-critical, flood protection, signal-vs-authority)
+
+## Self-action convergence (unbounded-self-action — closure: guard)
+
+New self-triggered notifiers are registered as convergence models in `src/testing/selfActionRegistry.ts` and proven to settle by `tests/unit/self-action-convergence.test.ts` (enforcement: ratchet):
+- **door-dark episode brakes** (`spend-door-dark-brakes`): max-attempts = chain length per episode bucket, widening backoff, flapping breaker → converges to the breaker bound under a permanently-dark chain.
+- **fallback-spike digest** (`spend-fallback-spike`): steady-state fallback churn is jsonl-only; a digest line fires only on a rate-spike edge with a per-window latch → ≤1 digest per window under sustained churn (eternal-sentinel rate floor).
+- **cap-approach thresholds** (`spend-cap-approach`): edge-triggered per (capKind, threshold, window) — at most 4 emissions per key per window (2 kinds × 2 thresholds), re-armed only by the window rolling.
+
+## No-deferrals accounting
+
+Layer 1c reconciliation (feeds recon-drift), the amortized-subscription display, and the web-research price checks are the next PRs of this increment train, tracked under CMT-1929 <!-- tracked: CMT-1929 --> and enumerated in `.instar/plans/money-increment-b-brief.md`; nothing in THIS PR's claimed scope is partial.