pi-smart-router 0.22.0 → 1.0.0

package/README.md

@@ -2,7 +2,7 @@
 
 **Auto-model router middleware for the [pi](https://pi.dev) coding agent.**
 
-> Current release: **v0.16.2** (mirrors `package.json`; SemVer `0.y.z`). The public API and routing behavior may change until `1.0.0`.
+> **v1.0 — SemVer stable (honest defaults).** From the `1.0.0` release on, the public API and documented operator surface follow [semantic versioning](https://semver.org): breaking changes land only in a new major version, each accompanied by a migration guide — start with [docs/migration-v1.md](docs/migration-v1.md). **1.0 means stable API + honest calibration floors + gates that catch Sept-class failures** — not a claim that cheap-tier routing is behaviorally proven. [#95](https://github.com/beettlle/pi-smart-router/issues/95) / [#110](https://github.com/beettlle/pi-smart-router/issues/110) remain open for verifier-graded recalibration. The published release always mirrors `package.json` ([npm](https://www.npmjs.com/package/pi-smart-router)).
 
 pi-smart-router intercepts every LLM inference request and dynamically routes it to the optimal execution engine — balancing cost, capability, latency, and time-to-first-token (TTFT) — without requiring you to manually pick a model for each turn.
 
@@ -52,7 +52,9 @@ See [docs/PRD.md](docs/PRD.md) for full architectural justification, [docs/deep-
 
 | Dependency | Required | Notes |
 |------------|----------|-------|
-| [Node.js](https://nodejs.org/) >= 22 | Yes | ES module package; matches CI and `package.json` engines |
+| [Node.js](https://nodejs.org/) >= 22.19.0 | Yes | ES module package; matches `package.json` `engines.node` and CI (workflows pin Node `22.19.0`) |
+
+> **Engine floor:** `package.json` declares `engines.node >= 22.19.0`. If you (or your environment) enable `engine-strict=true` in `.npmrc`, installs on Node < 22.19.0 fail with `EBADENGINE`; on a supported Node (>= 22.19.0), `npm ci` completes with no `EBADENGINE` warnings.
 | [pi](https://pi.dev) coding agent | Yes | Extension host |
 | macOS Apple Silicon | MVP | Primary supported platform |
 | Linux (x64/arm64) | Experimental | Probe logic supported; not validated on real hardware |
@@ -94,7 +96,7 @@ Then in pi:
 npm install pi-smart-router
 ```
 
-Use `createRouter()` / `createRouterFromFleet()` for programmatic integration without the pi extension. See [Optional: YAML fleet (library API)](#optional-yaml-fleet-library-api).
+Use `createRouter()` / `createRouterFromFleet()` for programmatic integration without the pi extension. That path is the **catalog / routing-core** composition root (not full extension wiring). See [Library vs extension](#library-vs-extension) and [Optional: YAML fleet (library API)](#optional-yaml-fleet-library-api).
 
 ### From source (contributors)
 
@@ -128,7 +130,7 @@ After auth or model list changes, restart pi or run `/reload`.
 
 ### Develop from clone
 
-Requires **Pi ≥ 0.80.8** (`pi.minPiVersion`). The extension loads TypeScript from this repo via pi’s loader — no `npm run build` needed for dogfooding.
+Requires **Pi ≥ 0.85.1** (`pi.minPiVersion`). The extension loads TypeScript from this repo via pi’s loader — no `npm run build` needed for dogfooding.
 
 **Recommended (works from any cwd):** install the clone as a **path package** and remove any published npm copy. npm and path packages have different identities; leaving both installed can double-load and you may keep running a stale npm tarball.
 
@@ -261,7 +263,7 @@ npm run verify:ci
 
 ## Concurrency contract
 
-`RouterPipeline.route()` calls on a single router instance are **single-flight**: concurrent calls are serialized internally (SP-230, [#141](https://github.com/beettlle/pi-smart-router/issues/141)). The pipeline keeps per-route transient state on instance fields while stages run, so overlapping executions are queued rather than interleaved — each queued call waits at most one routing latency. This applies to `createRouter()` / `createRouterFromFleet()` handles: a shared `router.dispatch` is safe to call concurrently, and serialization does not change routing policy outcomes. For parallel routing throughput, create separate router instances.
+`RouterPipeline.route()` calls on a single router instance are **single-flight**: concurrent calls are serialized internally (SP-230, [#141](https://github.com/beettlle/pi-smart-router/issues/141)). Each exclusive route owns one per-route `RoutingContext`; overlapping executions are queued rather than interleaved — each queued call waits at most one routing latency. This applies to `createRouter()` / `createRouterFromFleet()` handles: a shared `router.dispatch` is safe to call concurrently, and serialization does not change routing policy outcomes. For parallel routing throughput, create separate router instances.
 
 ## Library vs extension
 
@@ -269,8 +271,8 @@ pi-smart-router ships in two shapes, and they are **not** feature-identical ([#1
 
 | Path | What you get |
 |------|--------------|
-| **Pi extension** (`pi install npm:pi-smart-router`, or project-local `.pi/extensions/smart-router/` when developing from clone) | The full product — the routing pipeline **plus** the stream-level behaviors below |
-| **npm library** (`createRouter()` / `createRouterFromFleet()` / `GatewayDispatch`) | The routing core — the 12-stage pipeline, fleet mapping, telemetry, and gateway health/failover *selection*. Stream-level behaviors are stubbed or left to your embedder loop |
+| **Pi extension** (`pi install npm:pi-smart-router`, or project-local `.pi/extensions/smart-router/` when developing from clone) — **supported product composition root** | The full product — the routing pipeline **plus** the stream-level behaviors below, with hardware probe + store-backed telemetry wired via `createDispatchOptions()` in `.pi/extensions/smart-router/fleet-bootstrap.ts` |
+| **npm library** (`createRouter()` / `createRouterFromFleet()` / `GatewayDispatch`) — **catalog / embedder composition root** | The routing core — the 12-stage pipeline, fleet mapping, gateway health/failover *selection*, and constructor defaults for **`costEstimator` + `localRuntime` only**. Hardware probe stays disabled and telemetry is opt-in unless you pass those ports. Stream-level behaviors are stubbed or left to your embedder loop |
 
 ### Extension-only capabilities
 
@@ -289,8 +291,8 @@ These behaviors run in `.pi/extensions/smart-router/` and have **no equivalent i
 
 ### Recommended integration path
 
-- **pi users:** install the **extension** (`pi install npm:pi-smart-router`). It is the full product — everything in the table above works out of the box, including failover, delegate spawn, headroom escalation, and quota reaction.
-- **npm embedders:** you get the **routing core** (12-stage pipeline, fleet mapping, telemetry, gateway health tracking, failover *selection*). Plan to implement your own stream delegation, failover iteration, headroom checks, and planning-delegate spawn around the decisions the pipeline returns — or track [#149](https://github.com/beettlle/pi-smart-router/issues/149) (**extension public facade**), the migration plan for exposing the extension's stream/delegation surface as supported library API so this gap closes over time. Until #149 lands, the extension modules also import `src/**` internals directly, so deep imports into `src/` are not a stable API. See [docs/extension-package-boundary.md](docs/extension-package-boundary.md) for the facade vs internal-API boundary, the deep-import lint guard, and the extension coverage gate.
+- **pi users:** install the **extension** (`pi install npm:pi-smart-router`). It is the supported composition root — everything in the table above works out of the box, including failover, delegate spawn, headroom escalation, quota reaction, hardware probe, and store-backed routing telemetry.
+- **npm embedders:** you get the **routing core** (12-stage pipeline, fleet mapping, gateway health tracking, failover *selection*). Bare `createRouter()` does **not** wire hardware probe or telemetry by default — pass `GatewayDispatchOptions` on `createRouterFromFleet(fleet, options)` when you need those ports. Plan to implement your own stream delegation, failover iteration, headroom checks, and planning-delegate spawn around the decisions the pipeline returns — or track [#149](https://github.com/beettlle/pi-smart-router/issues/149) (**extension public facade**), the migration plan for exposing the extension's stream/delegation surface as supported library API so this gap closes over time. Until #149 lands, the extension modules also import `src/**` internals directly, so deep imports into `src/` are not a stable API. See [docs/extension-package-boundary.md](docs/extension-package-boundary.md) for the facade vs internal-API boundary, the deep-import lint guard, and the extension coverage gate. See also [Composition roots in the 1.0 migration guide](docs/migration-v1.md#composition-roots-library-createrouter-vs-pi-extension).
 
 ```text
 pi extension path (full product)        npm library path (routing core)
@@ -323,7 +325,9 @@ To refresh after auth or settings changes, restart pi or `/reload` extensions.
 
 ## Optional: YAML fleet (library API)
 
-For programmatic integration **without** the pi extension, load a static fleet catalog from YAML and route via `GatewayDispatch.dispatch()`:
+For programmatic integration **without** the pi extension, load a static fleet catalog from YAML and route via `GatewayDispatch.dispatch()`.
+
+This path is **catalog-oriented**: `createRouter()` loads `models.yaml` and constructs `GatewayDispatch` with library defaults (`costEstimator` + `localRuntime`). It does **not** call the extension's `createDispatchOptions()` — so hardware probe remains disabled and no store-backed `telemetryEmitter` is attached unless you pass those options yourself. Prefer the [pi extension](#library-vs-extension) when you want the full product composition root.
 
 ```bash
 cp config/models.yaml.example ./config/models.yaml
@@ -331,13 +335,16 @@ cp config/models.yaml.example ./config/models.yaml
 ```
 
 ```typescript
-import { createRouter } from 'pi-smart-router';
+import { createRouter, createRouterFromFleet } from 'pi-smart-router';
 
 const router = createRouter({ modelsPath: './config/models.yaml' });
 router.register(piExtensionHooks); // lifecycle only: compaction + model override
 
 const decision = await router.dispatch.dispatch(routingRequest);
 // Embedder forwards inference to decision.selected_model_id
+
+// Optional: wire probe/telemetry yourself (same options bag as GatewayDispatch)
+// createRouterFromFleet(fleet, { systemInfoProvider, hardwareConfig, telemetryEmitter, ... })
 ```
 
 ### Embedder integration paths
@@ -698,18 +705,24 @@ Manual `/smart-router feedback good|bad` is **optional**. Passive dogfood signal
 
 | Passive field / signal | Role |
 |------------------------|------|
-| `model_override` | Failure — operator overrode the routed model |
+| `model_override` | Failure — operator overrode the routed model (behavioral) |
 | `compaction_pin_break` | Neutral/positive context — pin broke at compaction (not a cheap-tier failure by itself) |
-| Loop-escalation proxies (`tool_failure_chain`, pin reason `loop_escalation`) | Failure proxies for stuck tool loops |
+| Loop-escalation proxies (`tool_failure_chain`, pin reason `loop_escalation`) | Failure proxies for stuck tool loops (verifier-grade) |
 | `stop_reason` / `stop_reason_invalid` / `stop_reason_length` | Execution outcome — invalid or truncated stops mark failure |
 
+The train/aggregate paths derive a label per row (`deriveSuccessLabelFromExportRow` in `src/domain/routing/p-success-classifier.ts`) from the joined `outcome_signals` — **no operator annotation required**:
+
+- **Failure** (`success: false`) if any derived failure signal is present: behavioral (`model_override`, `feedback_bad`), verifier proxies (`tool_failure_chain`, `stop_reason_invalid`, `reprompt_detected`, `high_edit_distance`), or execution (`provider_failover`, `stop_reason_length`, `infra_error`).
+- **Success** (`success: true`) on explicit `feedback_good` only.
+- **Unlabeled** (`success: null`, skipped by training) when no outcome signals exist, or when only neutral signals such as `compaction_pin_break` were recorded — never coerced to success.
+
 Optional `feedback_good` / `feedback_bad` only refine labels when the operator chooses to annotate; they are not required for a valid train path. **Do not invent labels** — incomplete exports skip or stay unlabeled rather than fabricating outcomes.
 
 **Sample floor:** collect at least **≥30** labeled **economical-tier** rows (`minimum_training_samples.p_success_weights` / `isotonic_calibrator` in [`config/routing-calibration.json.example`](config/routing-calibration.json.example)) before relying on non-neutral `P(success)` or isotonic. Below that floor the classifier returns neutral `P_success_cheap = 0.5`.
 
-**Provenance today vs behavioral adoption:** the checked-in `config/p-success-weights.json` remains **synthetic/fixture** (SP-175 — trained on `scripts/fixtures/p-success-synthetic-train.jsonl`, not community dogfood). Treat those weights as an interim dogfood enablement until real passive-signal floors are met and artifacts are retrained/shipped ([#110](https://github.com/beettlle/pi-smart-router/issues/110) train/ship slice — SP-206). Do not claim synthetic rows are behavioral.
+**Provenance today (honest defaults, v1.0.0):** checked-in `config/p-success-weights.json` and `config/routing-calibration.json` ship **neutralized** (`trained_sample_count: 0` for P(success), isotonic, and triage; HyDRA remains 0/≥100). Serve-time routing uses **structural tier hints** and `P_success_cheap = 0.5` until a verifier-graded retrain clears hard ECE gates (`holdout_ece_calibrated ≤ holdout_ece_raw`, absolute ECE ≤ 0.10, non-degenerate `y_knots`). Scripted `scripts/qa/dogfood-gather.sh` labels are **quarantined** (`scripted_intent`) and must not feed ship trains. Bootstrap routing centroids remain in the bundle. TwinRouterBench CI corpus soft-fail (`mean_over_routing_rate ≈ 0.87` vs 0.15) is a **harness adapter artifact** ([#112](https://github.com/beettlle/pi-smart-router/issues/112) / [`over-routing-analysis.md`](spine-tasks/_authoring/release-v0.11.0/over-routing-analysis.md)), not a live-pipeline measurement — absolute release gates stay fixture-backed; frugality stays. [#95](https://github.com/beettlle/pi-smart-router/issues/95) / [#110](https://github.com/beettlle/pi-smart-router/issues/110) stay open for human dogfood + verifier-graded recalibration.
 
-**SP-206 status (v0.12.0):** **deferred / Partial (B).** Operator had no #95 dogfood exports in this window (labeled economical-tier rows = **0**, floor ≥30). No behavioral `config/p-success-weights.json` or `config/routing-calibration.json` was shipped. See [`spine-tasks/_authoring/release-v0.12.0/behavioral-calibration-partial.md`](spine-tasks/_authoring/release-v0.12.0/behavioral-calibration-partial.md). Leave [#110](https://github.com/beettlle/pi-smart-router/issues/110) open until floors are met; never invent labels.
+**SP-206 / hybrid interim:** July dogfood and a Sept hybrid were explored during the 1.0 train; Sept isotonic collapsed (same-x PAV overwrite) and scripted gather labels failed #110 honesty. Code now pools same-x scores and skips null labels; shipped artifacts stay honest-untrained until post-1.0 verifier-graded trains land.
 
 **Zero-manual-label path (aggregate → train → verify):**
 
@@ -731,6 +744,8 @@ npm run routing:train-calibration -- --input path/to/aggregated.jsonl
 npm run routing:verify-calibration -- config/routing-calibration.json
 ```
 
+Command cross-links: contrib export + aggregation details in [Community telemetry contribution (calibration)](#community-telemetry-contribution-calibration); standalone weights vs full bundle flags in [Operator train / reload](#operator-train--reload-no-prompt-text); OATS centroid refinement inside `routing:train-calibration` in [OATS cluster centroid refinement](#oats-cluster-centroid-refinement-offline-calibration); ECE/holdout gating in [Privacy-safe label packs + calibration dry-run](#privacy-safe-label-packs--calibration-dry-run-sp-189sp-191--102); one-line script summaries in the [Scripts](#scripts) table. The dogfood-side capture checklist lives in [`docs/qa/shadow-dogfood-protocol.md`](docs/qa/shadow-dogfood-protocol.md); the full dogfood → export → aggregate → train loop is diagrammed in [docs/migration-v1.md](docs/migration-v1.md#shadow-dogfood--calibration-behavioral-path).
+
 #### Operator train / reload (no prompt text)
 
 ```bash
@@ -742,7 +757,8 @@ SMART_ROUTER_DATASET=1
 # Train standalone weights (≥30 labeled rows required)
 npm run routing:train-p-success -- --input path/to/export.jsonl --output config/p-success-weights.json
 
-# Or regenerate the checked-in dogfood weights from the synthetic fixture (interim only):
+# Retrain standalone weights from the synthetic fixture (fixture demo only — NOT behavioral;
+# the checked-in weights are real dogfood, see "Provenance today" above):
 npm run routing:train-p-success
 
 # Optional: merge isotonic into an existing calibration bundle (does not rewrite hydra/centroids)
@@ -755,7 +771,7 @@ npm run routing:train-calibration -- --input path/to/aggregated.jsonl
 
 Reload is file-based: replace `config/p-success-weights.json` (and optionally `config/routing-calibration.json` for isotonic) and restart the host agent — no prompt text is ever written into training artifacts.
 
-**Isotonic gap:** serve-time isotonic calibration loads from `config/routing-calibration.json` (`isotonic_calibrator`). The checked-in dogfood path ships trained **logistic** weights only; isotonic is produced when you pass `--calibration-output` or run `routing:train-calibration` with ≥30 labeled samples. Until that bundle exists, the pipeline uses raw logistic `P(success)` (identity / no-op calibrator) and still exposes `p_success_raw` vs `p_success_calibrated` / `p_success_cheap` on explain and telemetry.
+**Isotonic calibration (shipped since v1.0.0):** serve-time isotonic calibration loads from `config/routing-calibration.json` (`isotonic_calibrator`). The checked-in bundle ships an **honest-untrained** calibrator (`trained_sample_count: 0` — identity knots), so serve-time `p_success_calibrated` / `p_success_cheap` stay at neutral `0.5` with structural tier hints until a verifier-graded retrain clears hard ECE gates (calibrated ECE ≤ raw, absolute ECE ≤ 0.10, non-degenerate `y_knots`). Operators who retrain should pass `--calibration-output` or run `routing:train-calibration` with ≥30 labeled samples (not `dogfood-gather.sh` scripted_intent). Explain and telemetry still expose `p_success_raw` vs `p_success_calibrated` / `p_success_cheap`.
 
 Library helpers (see `src/domain/routing/p-success-classifier.ts`):
 
@@ -1202,6 +1218,7 @@ npm run routing:twinrouterbench:full-track
 | **Gate soft-feed** | `npm run routing:assert-release-gates:corpus-report` |
 | **Over-routing breakdown** | `npm run routing:analyze-overrouting` · [v0.11.0 analysis](spine-tasks/_authoring/release-v0.11.0/over-routing-analysis.md) (#112 / #95) |
 | **Human QA protocol** | [`docs/qa/shadow-dogfood-protocol.md`](docs/qa/shadow-dogfood-protocol.md) · `npm run qa:shadow-dogfood` |
+| **Dogfood export → gates soft-feed** | `npm run qa:dogfood-soft-feed -- --export <track-b-export.json>` (dry-run, always exit 0 on gate outcome; never invents labels — #111) |
 
 **Absolute release gates stay on default fixtures.** `npm run release:functional-smoke` continues to assert `tests/eval/fixtures` against `config/release-gates.json` — do not point it at the corpus without operator review. Today the corpus subset fails `mean_over_routing_rate_max` (≈0.85 vs absolute max 0.15); that gap is intentional soft signal for the [#95](https://github.com/beettlle/pi-smart-router/issues/95) public static-track acceptance criteria alongside live dogfood traces. Use `--fixtures tests/eval/corpus/twinrouterbench` (or the corpus-report script) for #95 public-track scoring; keep absolute threshold edits out of band until operators approve. For live shadow dogfood steps and sign-off, see the [shadow dogfood protocol](docs/qa/shadow-dogfood-protocol.md).
 
@@ -1219,7 +1236,7 @@ First-class **local / optional nightly** path for the pinned ~970-row bank. **Do
 
 PR corpus smoke remains the vendored ≤150 subset. Absolute `config/release-gates.json` thresholds and `release:functional-smoke` stay fixture-backed.
 
-**[#95 dual-gate protocol](https://github.com/beettlle/pi-smart-router/issues/95):** (1) live shadow dogfood (`docs/qa/shadow-dogfood-protocol.md` · `npm run qa:shadow-dogfood`) and (2) public static-track soft-feed (CI subset report, or full-track report above). Neither path edits absolute release thresholds.
+**[#95 dual-gate protocol](https://github.com/beettlle/pi-smart-router/issues/95):** (1) live shadow dogfood (`docs/qa/shadow-dogfood-protocol.md` · `npm run qa:shadow-dogfood`, plus `npm run qa:dogfood-soft-feed` to attach labeled dogfood exports to the gates as a dry-run) and (2) public static-track soft-feed (CI subset report, or full-track report above). Neither path edits absolute release thresholds.
 
 **Deferred:** RouterBench classic (outcome-matrix) smoke is out of scope for SP-188; prefer TwinRouterBench static track + dogfood for #95.
 
@@ -1434,6 +1451,7 @@ Confirm https://pi.dev/packages/pi-smart-router shows the new version (may lag n
 
 | Document | Purpose |
 |----------|---------|
+| [docs/migration-v1.md](docs/migration-v1.md) | Upgrading to 1.0: raised requirements (pi ≥ 0.85.1, Node ≥ 22.19.0), calibration artifacts, pipeline architecture notes, SemVer stability surface |
 | [docs/PRD.md](docs/PRD.md) | Product requirements, research lineage, pipeline specification |
 | [docs/constitution.md](docs/constitution.md) | Project principles and non-negotiable rules |
 | [specs/001-build-smart-router/spec.md](specs/001-build-smart-router/spec.md) | Detailed feature specification |

package/config/benchmark-profiles.json

@@ -7,8 +7,8 @@
       "livecodebench": "https://livecodebench.github.io/leaderboard.html",
       "bfcl": "https://gorilla.cs.berkeley.edu/leaderboard.html"
     },
-    "scrape_date": "2026-09-05",
-    "catalog_freeze_date": "2026-09-05"
+    "scrape_date": "2026-09-10",
+    "catalog_freeze_date": "2026-09-10"
   },
   "aliases": {
     "anthropic/claude-opus-4": "claude-opus-4-5",

package/config/p-success-weights.json

@@ -13,24 +13,18 @@
     "routing_latency_norm",
     "economical_tier"
   ],
-  "intercept": 4.101841550841884,
+  "intercept": 0,
   "coefficients": [
-    -3.9458158506645473,
-    -3.9458158506645473,
-    -4.6221357618419106,
-    -4.702014247565688,
-    -1.9442887494123497,
-    -2.945430358303044,
-    -2.8373347243581435,
-    0,
-    0.09441030879566878,
-    4.101841550841884
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0
   ],
-  "trained_sample_count": 40,
-  "provenance": {
-    "source": "synthetic_fixture",
-    "task": "SP-175",
-    "note": "Privacy-safe feature vectors + labels only; no prompt text.",
-    "trained_at": "2026-07-10"
-  }
+  "trained_sample_count": 0
 }