instar 1.3.784 → 1.3.786

package/src/scaffold/templates/jobs/instar/routing-price-web-verify.md

@@ -0,0 +1,32 @@
+---
+name: Routing Price Web-Verify
+description: "The SCHEDULED web-research price check (operator directive 2026-07-07; docs/specs/routing-control-room-spend-alerts.md FD-8 web-verify): for the metered doors whose prices are published on OFFICIAL WEB PAGES only (Groq, Google), the deterministic prober fetches groq.com/pricing + ai.google.dev/pricing and extracts the tracked models' per-Mtok prices with CONSERVATIVE fail-closed parsers (fixture-realness-tested; a reshaped page refuses, never guesses) plus a plausibility clamp vs the reviewed canonical price (>10x off → refused). Extracted points are OBSERVATIONS written forward-only, UTC-day-aligned, into the MACHINE-LOCAL observed cache ONLY — structurally never the canonical manifest; an observed price never becomes official without the operator's PIN promotion. The fetch is FREE (no LLM, no metered key) — any future LLM-assisted extraction stays manual + budget-capped. Drift between observed and canonical surfaces via the Spend tab promote hint and the observed-drift alert. Ships OFF by default (enabled:false). perMachineIndependent — the observed cache is machine-local."
+schedule: "0 6 * * 2"
+priority: low
+expectedDurationMinutes: 5
+model: haiku
+supervision: tier1
+enabled: false
+perMachineIndependent: true
+tags:
+  - cat:maintenance
+  - role:worker
+  - exec:prompt
+gate: curl -sf http://localhost:${INSTAR_PORT:-4042}/health >/dev/null 2>&1
+toolAllowlist: ["Bash"]
+unrestrictedTools: false
+mcpAccess: none
+---
+Run one routing-price web-verify pass. This job only INVOKES the deterministic prober and sanity-checks that it produced a well-formed result — the prober owns 100% of the network I/O, the conservative page parsing, the plausibility clamp, the forward-only day-alignment, and the observed-cache write. You have Bash ONLY — no Edit/Write tool. Do NOT try to edit any source file, and NEVER touch `scripts/routing-prices.manifest.json` (the canonical manifest is human/PIN-reviewed only; the prober is structurally forbidden from writing it).
+
+1. **Prober-presence gate.** Confirm the prober ships in this tree; if absent, exit cleanly (nothing to do):
+   `test -f scripts/routing-price-refresh.mjs`
+
+2. **Run the deterministic prober at the web-verify scope (official pricing pages; zero metered spend — the fetch is free and no LLM is involved):**
+   `node scripts/routing-price-refresh.mjs --scope +web-verify --plausibility-baseline scripts/routing-prices.manifest.json`
+
+   (The `--plausibility-baseline` path is READ-ONLY input for the >10x sanity clamp — the prober's own source is structurally baseline-blind (S2-2) and can never write that file; you pass its location, the prober only compares against it.)
+
+3. **Tier-1 supervision (your job) — sanity-check the run, do NOT re-surface anything yourself.** Confirm the prober printed a well-formed JSON result (`added`, `totalObserved`, `notes`). A note saying a page was "not confidently parseable — refused, never guessed" or "REFUSED by the plausibility clamp" is a HEALTHY fail-closed outcome, not an error. The observed cache is REPORTING-ONLY; a real price drift surfaces in the Routing Spend dashboard tab's promote hint and the observed-drift alert, never as an edit here.
+
+4. Exit. The observed cache is machine-local and forward-only; the canonical manifest is never touched by this job.

package/upgrades/1.3.785.md

@@ -0,0 +1,56 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+- **`ProviderCostReportStore`** (`src/monitoring/ProviderCostReportStore.ts`): the immutable
+  append-only provider-report record set + reconciliation records — `meteredCallId` join key,
+  supersession by greatest `capturedAt` (a late OpenRouter `/generation` cost never double-counts),
+  receive clamps (invalid rows audit-preserved + aggregate-excluded), declared 400d retention with
+  the batched prune, registered in `state-coherence-registry.json` WITH retention at birth.
+- **`extractProviderReport`**: the per-door capture parser — OpenRouter `usage.cost` +
+  cached-token detail + generation id; Groq token counts; Gemini native (`candidatesTokenCount` +
+  `thoughtsTokenCount`, erring HIGH) and OpenAI-compat shapes. Absent/reshaped fields degrade to
+  token-only or nothing — a first-class NORMAL state.
+- **`ProviderReconciliationSweep`** (`src/monitoring/ProviderReconciliationSweep.ts`): the
+  cadenced (default 6h) REPORTING-side cross-check — signed `driftPct` per (keyRef, door) window,
+  committed-figure enrichment on the metered-lease holder (via the READ surface, never the money
+  lock), drift ≥ `reconciliation.driftAlertPct` (default 10%) feeding the already-live
+  `SpendAlertEmitters.onReconciliationDrift` (Increment-C informational lane, dispatcher-latched).
+- **`feature_metrics.callId`** — the per-call join column, full I-4 discipline (column + type +
+  writer + INSERT), stamped only by the future metered dispatch.
+- **`GET /routing-spend/reconciliation`** — the drift-record read route (same dev gate as the
+  view); the spend summary now PREFERS provider-reported cost per row where reports exist
+  (`costBasis: "provider-reported"`, `providerReportedUsd`, `providerDriftPct`).
+- **FD-21 structural exclusion as a TEST:** the money gate/ledger modules reference nothing from
+  the provider store — pinned by `tests/unit/provider-cost-report-store.test.ts`.
+- Self-action convergence model `spend-recon-sweep` (24h-latch eternal sentinel).
+
+## What to Tell Your User
+
+Nothing yet — this layer is bookkeeping that stays empty until paid routing actually dispatches
+calls (a separate, still-unbuilt piece). Once real paid calls flow, your Spend tab's dollar figures
+will prefer what the PROVIDER says it charged (where the provider reports it), and a scheduled
+cross-check will quietly flag any drift between the provider's numbers and ours — so a stale price
+gets noticed and fixed instead of silently mispricing your books.
+
+## Summary of New Capabilities
+
+- (⚗️ Experimental, dark, honestly-empty) Provider-grounded cost reporting: the immutable
+  provider-report store, the per-door capture parser, the reconciliation sweep + drift alerts, and
+  the provider-preferred spend basis. Remaining in this train (tracked: CMT-1929): PR 4 — the
+  amortized subscription display with visible derivation math + scheduled web-research price checks.
+
+## Evidence
+
+- `tests/unit/provider-cost-report-store.test.ts` — 18 cases: clamps, supersession/no-double-count,
+  per-door extraction (incl. the Gemini thinking-token trap), signed drift + threshold + Near-Silent
+  below it + zero-internal guard + never-throws, the FD-21 structural exclusion, and the callId
+  end-to-end column discipline.
+- `tests/integration/routing-spend-routes.test.ts` — the reconciliation route (dark-503/live-200)
+  and the provider-preferred summary basis over real stores.
+- `tests/e2e/routing-spend-lifecycle.test.ts` — the reconciliation surface constructs on the REAL
+  AgentServer boot path (200 + the DB file on disk, honest empty records).
+- `tests/unit/self-action-convergence.test.ts` — 50/50 with the new model.

package/upgrades/1.3.786.md

@@ -0,0 +1,42 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+- **Amortized subscription display (operator decision #3 — "amortize but show the math"):**
+  `routingSpend.subscriptions` (reporting-only) declares a monthly price per CLI door; the spend
+  view amortizes it by CALENDAR TIME over the door's active days and shows the FULL derivation
+  (`$200.00/mo ÷ 30.4375 avg days/mo = $6.5708/day × 2 active day(s) = $13.1417`) per door —
+  never cap-enforced, counted once per door in totals; the Spend tab renders the figure inline
+  with a math footnote. Undeclared doors keep the honest `$0 (subscription)` wording.
+- **Scheduled web-research price checks (operator directive):** `--scope +web-verify` in the
+  deterministic prober fetches the OFFICIAL Groq + Google pricing pages, extracts tracked-model
+  prices with conservative fail-closed parsers (registered in SCRAPE_PARSERS, fed the REAL
+  captured page bytes) + a >10x plausibility clamp vs the reviewed canonical price, and writes
+  forward-only OBSERVATIONS into the gate-ineligible observed cache. Zero spend (no LLM, no
+  metered key — the FD-8 budget refusal now exempts exactly this deterministic scope). New
+  OFF-by-default weekly `routing-price-web-verify` job (tier-1 supervised, Bash-only).
+
+## What to Tell Your User
+
+If you tell me what your AI subscriptions cost per month (e.g. "Claude Max is $200/mo"), the Spend
+tab stops showing a bare $0 for them — you'll see an amortized share for the reporting window with
+the exact arithmetic printed underneath, clearly labeled as a time-based estimate that never
+affects any spending cap. And once you enable the weekly price-check job, I'll quietly re-read the
+providers' official pricing pages and flag drift against the reviewed prices — you always keep the
+final say (PIN) on what price the books use.
+
+## Summary of New Capabilities
+
+- (⚗️ Experimental) Amortized subscription costs with visible derivation math + scheduled
+  web-research price observations. This completes the money-increment train (B → C → 1c → PR 4);
+  the pool-scope reconciliation merge stays tracked under CMT-1929.
+
+## Evidence
+
+`tests/unit/routing-price-web-verify.test.ts` — 7 cases: both parsers over the REAL captured page
+bytes (groq $0.15/$0.60; flash-lite $0.25 text-rate/$1.50 thinking-inclusive), reshaped-page
+refusal, plausibility-clamp refusal, and the amortization math (derivation string pinned verbatim,
+door-level totals, undeclared-door nulls). Realness lint green; docs-coverage floors held.

package/upgrades/side-effects/routing-spend-increment-1c.md

@@ -0,0 +1,44 @@
+# Side-Effects Review — Routing Control Room Layer 1c (provider-reported cost + reconciliation)
+
+**Spec:** `docs/specs/routing-control-room-spend-alerts.md` (review-convergence r7, approved, parent-principle: Token-Audit Completeness) — §Layer 1c + FD-21.
+**Worktree:** `echo/money-increment-1c` off `JKHeadley/main` @ `7c2064163` (contains merged Increments B + C).
+**Scope of this PR (PR 3 of the train, tracked CMT-1929):** the provider-grounded REPORTING anchor — `ProviderCostReportStore` (immutable append-only provider reports + reconciliation records, receive-clamped, 400d declared retention, `meteredCallId` join key with capturedAt supersession), the `callId` column on `feature_metrics` (full I-4 discipline: column + type + writer + INSERT), the per-door provider-report field extraction (`extractProviderReport` — OpenRouter usage.cost mandatory-include posture; Groq/Gemini token counts incl. the thinking/cached details), the capture SEAM the future metered dispatch calls strictly post-settle, the cadenced `ProviderReconciliationSweep` (reporting-side, NEVER the money lock; signed driftPct feeding the ALREADY-LIVE `SpendAlertEmitters.onReconciliationDrift`), the `GET /routing-spend/reconciliation` read route, and the summary composer's provider-preferred `costBasis` enrichment. The operator additions (amortized subscription display + web-research price checks) are PR 4.
+
+## Phase 1 — Principle check (signal-vs-authority)
+
+**Does this change involve a decision point?** No blocking decision anywhere: everything here is REPORTING truth. The load-bearing invariant is the opposite direction — the spec's structural exclusion (FD-21, the FD-9/FD-12 twin): NO provider report, reconciliation record, or drift figure is ever read by the money gate or the reserve/settle path. This PR ships that as a STRUCTURAL TEST (the gate/ledger modules import nothing from the provider store) plus the one-way drift rule: provider-LOWER only changes the report; provider-HIGHER raises a drift signal feeding the human-reviewed PIN price-promotion path — the committed counter is never rewritten in either direction. Compliant: signals only.
+
+## Phase 2 — Plan
+
+- **Decision points touched:** none with authority. The sweep DECIDES only whether a drift figure crosses the alert threshold — feeding the Increment-C dispatcher (itself notification-only, dryRun-soaked).
+- **Existing detectors/authorities interacted with:** `FeatureMetricsLedger` (additive `callId` column, same ensureAddedColumns discipline as `door`); `RoutingPriceAuthority` (read-only as-of pricing for the internal-derived side of the comparison); `SpendAlertEmitters.onReconciliationDrift` (already live, already convergence-modeled); the caps read surface (committed totals for booked-vs-reported — read-only, never the per-key money mutex); `state-coherence-registry.json` (the new store registered WITH declared retention — the Bounded Accumulation ratchet is satisfied at birth, never grandfathered).
+- **Store shape decision (documented deviation):** the spec says "a small SQLite table beside the rollup"; this ships as a small SQLite DB FILE beside the rollup's DB (`server-data/provider-cost-reports.db`) rather than a table inside `feature-metrics.db` — one owner per file, no cross-class schema coupling, identical query surface. Volume is bounded by metered calls only.
+- **Rollout:** rides the EXISTING dev-gated `routingSpend` view flag (FD-21: "capture + display in Increment A" posture — the store/route are read-only observability like the summary); the sweep additionally requires the reconciliation config (defaults on where the view is on; inert until provider reports exist, which requires the out-of-scope metered dispatch). No paid door can route regardless (FD-11 unchanged).
+- **Rollback path:** additive; disabling `routingSpend.enabled` reverts byte-for-byte (store not constructed, route 503, sweep never ticks). The DB file is inert data at rest.
+
+## Phase 4 — Side-effects review
+
+1. **Over-block:** N/A — nothing blocks. Over-REPORTING risk: a malformed provider body could pollute the reports; closed by receive-clamps (finite ≥0 numerics or the row is stored as `invalid: 1` with nulls — preserved for audit, excluded from every aggregate), length/charset clamps on strings, and HTML-escape at any render.
+2. **Under-block / missed data:** capture is best-effort BY SPEC (a swallowed append never blocks the call and never shares a failure domain with the fail-closed settle — the seam contract); a lost report degrades that row to `internal-derived`, a first-class labeled basis, and the reconciliation surface is what makes systematic loss visible.
+3. **Level-of-abstraction fit:** store + sweep live in `src/monitoring/` beside FeatureMetricsLedger (reporting side); the gate/ledger in `src/core/` remain import-free of them (structural test). The route rides the existing `/routing-spend/*` dev-gated family.
+4. **Signal vs authority:** compliant (Phase 1) — with the exclusion invariant unit-pinned.
+5. **Interactions:** the reserve-expiry sweep (money-side, takes the per-key mutex) and this provider-reconciliation sweep (reporting-side, never the lock) are named distinctly everywhere per the spec's vocabulary; the recon-drift alert rides the Increment-C informational lane and its already-registered convergence model. Supersession by `(meteredCallId, greatest capturedAt)` means a late `/generation` cost can never double-count.
+6. **External surfaces:** one new Bearer read route (503 when dark). No egress — the sweep reads local stores only; the drift ALERT egresses through the C dispatcher (dryRun-soaked).
+7. **Multi-machine posture:** `proxied-on-read` like `feature_metrics` (each machine records its own calls' reports; pool merge is a tracked follow-up with the pool summary <!-- tracked: CMT-1929 -->); the booked-vs-reported comparison that involves the money ledger runs on the metered-lease holder by construction (it reads the local caps/ledger read surface, which only the holder has live values for — a non-holder's sweep sees $0 committed and records the comparison as provider-vs-internal only).
+8. **Rollback cost:** config revert; no migration; the DB file and its records are inert audit history.
+
+## Phase 5 — Second-pass review
+
+Not-required by the Phase-5 trigger list (no block/allow messaging decisions, no session lifecycle, no gate/sentinel authority — this is read-side reporting; the one gate-adjacent surface is the STRUCTURAL EXCLUSION test, which reduces gate blast radius). Tier-2 chain otherwise complete; the FD-21 invariant is enforced by test rather than reviewer eyes.
+
+## Self-action convergence (unbounded-self-action — closure: guard)
+
+One new self-triggered behavior: the reconciliation sweep's drift ALERT — already covered by the registered `spend-recon-drift` emit surface riding the Increment-C dispatcher latch (24h re-arm per (keyRef, door, driftBucket)) and the existing dispatcher convergence posture; the sweep itself is a fixed-cadence read-only pass (no feedback loop: its output never changes its input — prices and reports are external facts). Registered model: `spend-recon-sweep` pins the once-per-bucket emission under permanently-drifting pressure.
+
+## No-deferrals accounting
+
+PR 4 (amortized subscription display with visible derivation math + scheduled web-research price checks) and the pool-scope merge of the reconciliation view are the remaining tracked items of this train, tracked under CMT-1929 <!-- tracked: CMT-1929 --> and enumerated in `.instar/plans/money-increment-b-brief.md`; nothing in THIS PR's claimed scope is partial.
+
+## Build outcome (staged with the change)
+
+Delivered exactly as reviewed: 18 unit + 2 integration + 1 e2e new tests green; FD-21 exclusion pinned; retention lint green with the store registered (never grandfathered); self-action ratchet 50/50 with `spend-recon-sweep`; docs-coverage floors held (class 55 / route 55).

package/upgrades/side-effects/routing-spend-pr4-operator-additions.md

@@ -0,0 +1,50 @@
+# Side-Effects Review — Routing Control Room PR 4 (operator additions)
+
+**Spec:** `docs/specs/routing-control-room-spend-alerts.md` (review-convergence r7, approved, parent-principle: Token-Audit Completeness) — FD-7 deferral resolved by the operator + FD-8 web-verify, both operator decisions 2026-07-07 (topic 29723).
+**Worktree:** `echo/money-increment-pr4` off `JKHeadley/main` @ `d70a546f8` (contains merged Increments B, C, and Layer 1c).
+**Scope (PR 4 of the train, tracked CMT-1929 — the FINAL increment):** the two operator additions.
+
+## Phase 1 — Principle check (signal-vs-authority)
+
+**Does this change involve a decision point?** No blocking decision anywhere. The amortized display is REPORTING-only arithmetic over an operator-declared config value; the web-verify parsers write OBSERVATIONS into the gate-ineligible observed cache, and their failure mode is REFUSAL (no data) — never a wrong price entering any gate path. Compliant.
+
+## Phase 2 — Plan
+
+- **Decision points touched:** none with authority. The web-verify parsers DECIDE only whether a page is confidently parseable (fail-closed both ways: unparseable → no point; implausible vs canonical → refused).
+- **Existing detectors/authorities interacted with:** `RoutingPriceAuthority` (unchanged — the observed cache remains gate-ineligible by construction); the FD-8 budget refusal (narrowed: the DETERMINISTIC `+web-verify` scope is exempt because it spends nothing — no LLM, no metered key; metered probes stay budget-fail-closed); `SCRAPE_PARSERS` (both parsers registered with REAL captured fixtures per the Fixture Realness standard); `installBuiltinJobs` (the new template reaches existing agents on update — Migration Parity by the existing non-destructive install path).
+- **Rollback path:** additive; remove the config key / leave the job disabled and behavior reverts byte-for-byte. The fixtures and job template are inert.
+
+## Phase 4 — Side-effects review (rider content, reviewed)
+
+The operator additions land here as the train's final increment (both operator decisions 2026-07-07, topic 29723):
+
+1. **Amortized subscription display ("amortize but show the math"):** `routingSpend.subscriptions` (reporting-only config — never a gate input) declares a monthly price per CLI door; the composer amortizes by CALENDAR TIME over the door's ACTIVE days in the window (30.4375 avg-month constant, NAMED in the output) and emits the full `amortizationDerivation` string per door; totals count each door once; the Spend tab renders the amortized figure with the math as a visible footnote block. Undeclared doors keep the honest `$0 (subscription — not per-token billed)`. Over-block: none (display only). Under-block: an operator-declared wrong price shows wrong REPORTING — labeled as door-level calendar allocation, never cap-enforced.
+2. **Scheduled web-research price checks:** `--scope +web-verify` in the deterministic prober — fetches the OFFICIAL Groq + Google pricing pages (the two doors without machine-readable price APIs), extracts tracked-model prices with CONSERVATIVE fail-closed parsers (registered in SCRAPE_PARSERS; fed the REAL captured page bytes per the Fixture Realness standard) plus a plausibility clamp (>10x off the reviewed canonical → refused). Zero spend (no LLM, no metered key — the FD-8 budget refusal now exempts exactly this deterministic scope; metered probes stay budget-fail-closed). Observations only; PIN promotion unchanged. New OFF-by-default job template `routing-price-web-verify` (weekly, tier-1 supervised, Bash-only).
+
+Signal-vs-authority: both riders are reporting/observation surfaces with zero blocking authority; the web-verify parsers' failure mode is REFUSAL (no data), never a wrong price entering the gate path (observed cache is structurally gate-ineligible).
+
+5. **Interactions:** the amortized figure never feeds netUsd/grossUsd/committed (a separate labeled column + totals line, counted once per door); the web-verify observations ride the EXISTING forward-only merge + promote-me drift surface — no new interaction with the money layer. 6. **External surfaces:** two outbound HTTPS fetches to official public pricing pages from an OFF-by-default weekly job; no data leaves the machine. 7. **Multi-machine:** the observed cache stays machine-local BY DESIGN (declared in the sibling job); the subscriptions config is per-machine reporting (the pool-merge label already names the adjustments source). 8. **Rollback cost:** config/job revert; no state migration.
+
+## 6b. Operator-surface quality
+
+The one operator surface touched is the Spend tab's subscription-cost display (a READ surface — no new controls, no forms). Answered in writing:
+1. **Leads with its primary action?** Yes — the tab's primary content is unchanged (the spend headline + rows); the amortized figure lands IN the existing Net column where the operator already looks, as `~$13.14 amortized ⓘ` — a value, not a control.
+2. **Zero raw internals as primary content?** Yes — the derivation math is a hover title + a compact footnote block BELOW the table, in plain sentences ("$200.00/mo ÷ 30.4375 avg days/mo = $6.5708/day × 2 active day(s) = ..."), never JSON or config keys; the config key itself appears nowhere on the surface.
+3. **Destructive actions de-emphasized?** N/A — the change adds no actions at all (display only); the existing tab's control asymmetry (freeze easy, arm PIN-gated) is untouched.
+4. **Plain language at phone width?** Yes — one short suffix per row plus a wrapping footnote paragraph; no new table columns (phone tables were the 2026-06-12 lesson), no horizontal growth; the ⓘ affordance signals the hover/footnote for touch users who cannot hover.
+
+## Phase 5 — Second-pass review
+
+Not-required by the Phase-5 trigger list (no block/allow decisions, no session lifecycle, no gate/sentinel authority — display arithmetic + fail-closed observation parsers). The parsers' realness posture is enforced by the SCRAPE_PARSERS lint rather than reviewer eyes.
+
+## Self-action convergence (unbounded-self-action — closure: n/a)
+
+No new self-triggered loop: the web-verify job is a fixed-cadence, OFF-by-default scheduled job (calendar-driven, not feedback-driven — its output never changes its input), and the display change has no emission at all. Declared n/a: one-shot/cadenced observation work, not a self-triggered controller.
+
+## No-deferrals accounting
+
+This is the train's final increment; the pool-scope reconciliation merge remains the one tracked follow-up under CMT-1929 <!-- tracked: CMT-1929 -->.
+
+## S2-2 hardening round (post-CI, staged with this fix)
+
+CI's string-level S2-2 ratchet (`the prober source never names routing-prices.manifest.json`) caught the plausibility clamp's `readManifest` naming the canonical manifest in CODE — a genuine violation of the prober's structural manifest-blindness, not just a comment. Fixed by inverting the dependency: the prober now takes a generic READ-ONLY `--plausibility-baseline <path>` argument (absent → the clamp passes, no baseline), and the CALLER (the web-verify job template) passes the manifest's location. The prober source contains zero references to the reviewed price file; write-incapability is preserved by construction; the clamp behavior is unchanged (verified: sane passes, >10x refused, no-baseline passes).