token-harness 0.1.17 → 0.1.19

package/README.md

@@ -33,14 +33,15 @@ The first screen is **Overview**. There is no separate Setup page to learn.
 
 1. Token Harness detects Claude Code and Codex.
 2. Each detected coding agent says either **Ready** or **Setup incomplete**.
-3. If setup is incomplete, choose **Finish setup** on that agent. Token Harness prepares the
-   recommended **RTK + HarnessTrim** baseline and shows the exact safe plan before anything changes.
-4. The **Optimizers** section keeps the recommended baseline separate from optional tools
-   (mcptoon, GitNexus and Headroom). When a tool needs an action, that action appears on the same
-   card as the status it refers to.
+3. If setup is incomplete, use the **Optimizer setup** matrix. It shows every optimizer against
+   every detected agent and lets you select one harness or both in the same review. The recommended
+   **RTK + HarnessTrim** baseline has its own action in that matrix and shows the exact safe plan
+   before anything changes.
+4. Optional optimizers (mcptoon, GitNexus and Headroom) use the same matrix and per-optimizer action;
+   there is no repeated setup button under each coding-agent card.
 5. **Health and updates** is maintenance, not another onboarding checklist. Normal setup performs its
-   own safety checks. Use **Re-check health** for troubleshooting and **Check for updates** when you
-   want to inspect provider versions. If a reviewed update is available, the same dialog offers
+   own safety checks. Use **Re-check health** for troubleshooting and **Check for updates** to inspect
+   Token Harness and optimizer versions. If an update is available, the same dialog offers
    **Install updates** after showing the versions.
 6. Keep using Claude Code or Codex normally. Open **Results** when you want detailed evidence.
 
@@ -160,7 +161,7 @@ algorithms into this repository.
 | [RTK](https://github.com/rtk-ai/rtk) | Shell/tool output reduction | Managed on reviewed combinations |
 | [HarnessTrim](https://github.com/giuliastro/HarnessTrim) | Deterministic output/context reduction | Managed first-party integration |
 | mcptoon | MCP discovery / compact manifest guidance | Optional managed integration on exact reviewed 0.7.10 rows; no savings assumed |
-| GitNexus | Repository graph / MCP context | Optional managed Claude integration for already-installed 1.6.12; license review required |
+| GitNexus | Repository graph / MCP context | Optional managed Claude/Codex integration for reviewed 1.6.12; license review required |
 | Headroom | Local MCP context compression/retrieval | Optional config-only managed Claude/Codex integration for already-installed 0.37.0; package prerequisite stays user-owned |
 | [cclimits](https://github.com/cruzanstx/cclimits) | Optional Claude allowance evidence | Read-only evidence; not an optimizer |
 | [ccusage](https://github.com/ccusage/ccusage) | Local usage history | Read-only evidence; never subscription quota |
@@ -172,11 +173,12 @@ be accepted without another hard-coded version bump when their executable versio
 machine-readable `capabilities` version and the semantic surface/write-set comparison reports no
 drift.
 
-Provider **package updates are separate from harness configuration writes**. `token-harness update`
-can replace a reviewed provider target without requiring an exact historical Claude/Codex fixture
-for that package version; exact compatibility rows still gate any later managed agent-config
-mutation. HarnessTrim updates use its reviewed npm channel and capture the previous global version
-for rollback. On native Windows RTK still prefers WinGet, but when that catalog is behind the
+Token Harness and provider **package updates are separate from harness configuration writes**.
+`token-harness update` can update the app when it is running from its verified global npm
+installation, and can replace a reviewed provider target without requiring an exact historical
+Claude/Codex fixture for that package version; exact compatibility rows still gate any later
+managed agent-config mutation. HarnessTrim updates use its reviewed npm channel and capture the
+previous global version for rollback. On native Windows RTK still prefers WinGet, but when that catalog is behind the
 reviewed 0.49.0 target Token Harness can fall back to the exact official GitHub Windows x64 release:
 it verifies GitHub's published SHA-256, replaces only the uniquely resolved `rtk.exe`, verifies the
 new version, and restores and re-verifies the previous bytes on failure. This package-only fallback
@@ -194,7 +196,8 @@ verification, managed lifecycle, compatibility/reversibility, project maturity a
 validation. Broader context owners also require an explicit admission decision.
 
 See [docs/optimizer-priorities.md](docs/optimizer-priorities.md) and
-[RFC 0027](docs/rfcs/0027-optimization-stack-manager.md).
+[RFC 0027](docs/rfcs/0027-optimization-stack-manager.md) and
+[RFC 0028](docs/rfcs/0028-smart-model-routing.md).
 
 ## Stable-stack operating model
 
@@ -245,6 +248,7 @@ browser controller itself.
 | `apply` | Apply a reviewed stored plan | Yes, only with `--yes` |
 | `verify` | Check the declared integration tier | No |
 | `metrics` | Report attributable reducer savings | No |
+| `routing` | Export/configure an owned CCR rule or inspect routing decisions | Yes, only after preview and `--yes` |
 | `status` | Report pipelines, drift and importer modes | No |
 | `update` | Check/update reviewed provider packages | Yes, only with `--yes` |
 | `rollback` | Restore the latest transaction snapshot | Yes, only with `--yes` |
@@ -260,6 +264,79 @@ The older automation contracts remain available. `ui --json` preserves its exist
 report; `ui --read-only` opens the legacy read-only UI; `ui --no-open` starts the guided app without
 launching a browser.
 
+### Smart Model Routing (advanced)
+
+The local TypeScript classifier needs no API key or local model. On Node.js 22+, Token Harness can
+install and start the reviewed CCR 3.1.1 CLI in its own protected state directory. Installation and
+routing configuration are separate preview/apply steps. Existing authenticated CCR services can be
+used as-is; Token Harness does not adopt or update an external/global CCR installation. For a useful
+shadow report, the selected CCR profile needs an existing provider/model; when exactly one matching
+Claude Code or Codex provider is already configured, Token Harness adds a profile scoped to CCR CLI
+launches if the provider exposes one unambiguous default model. If it exposes several models, set
+`TOKEN_HARNESS_ROUTING_PROFILE_MODEL` to the exact `Provider/model` for the profile. To enable a
+conservative candidate, set `TOKEN_HARNESS_ROUTING_SIMPLE_MODEL` to an exact configured
+`Provider/model`; Token Harness validates it against CCR's provider catalog. It never imports OAuth
+credentials or edits native harness endpoints. Select provider login/import explicitly in CCR when
+needed. See CCR's
+[Agent Profiles guide](https://github.com/musistudio/claude-code-router/blob/main/docs/src/content/docs/en/configuration/profiles.md).
+On a first install, preview and approve CCR install/start, then preview and approve routing setup:
+
+```sh
+# First preview and approve CCR install/start.
+token-harness routing --configure-ccr --harness codex
+token-harness routing --configure-ccr --harness codex --yes
+# Then preview and approve the routing rule and optional profile.
+token-harness routing --configure-ccr --harness codex
+token-harness routing --configure-ccr --harness codex --yes
+```
+
+To update the Token Harness-owned CCR CLI to the current reviewed version pin, preview and apply:
+
+```sh
+token-harness routing --update-ccr
+token-harness routing --update-ccr --yes
+```
+
+After setup, launch the scoped profile shown by Token Harness (for Codex, typically
+`ccr "Token Harness Codex"`) and confirm a real request appears in CCR logs. The saved profile or
+gateway status alone does not prove interception. In shadow mode the rule records the proposed tier
+and leaves the current model unchanged.
+
+For a paired routing experiment, capture the baseline while the rule is in shadow mode, then record
+the task's actual quality outcome. To test conservative routing, set
+`TOKEN_HARNESS_ROUTING_SIMPLE_MODEL` in the Token Harness environment to an exact model already
+configured in CCR, roll back the owned shadow rule, and preview/apply the conservative rule. Token
+Harness validates the alias and embeds it in the script. Run the same task as the optimized variant,
+record its quality, then compare the receipt paths printed by Token Harness:
+
+```sh
+token-harness benchmark-start --benchmark-id routing-codex-1 --variant baseline --task mechanical --harness codex
+# Run the task with CCR in shadow mode, then finish with the actual quality and attempt counts.
+token-harness benchmark-finish --benchmark-id routing-codex-1 --variant baseline --quality passed --attempts 1 --failed-attempts 0
+
+token-harness routing --rollback-ccr --harness codex
+token-harness routing --rollback-ccr --harness codex --yes
+token-harness routing --configure-ccr --harness codex --route-mode conservative
+token-harness routing --configure-ccr --harness codex --route-mode conservative --yes
+token-harness benchmark-start --benchmark-id routing-codex-1 --variant optimized --task mechanical --harness codex
+# Repeat the same task under comparable conditions, then finish with its real quality outcome.
+token-harness benchmark-finish --benchmark-id routing-codex-1 --variant optimized --quality passed --attempts 1 --failed-attempts 0
+
+token-harness benchmark --baseline /path/to/baseline.json --optimized /path/to/optimized.json
+```
+
+To inspect decisions and local CCR request usage outside the task comparison, use
+`token-harness routing --route-metrics` or add `--ccr-usage`.
+
+The benchmark reads CCR session counters only when the task produced local routing events with a
+recognized harness identity. Receipts retain model and token aggregates, not prompts or session
+IDs. The comparator shows CCR token and provider-cost-estimate deltas separately and only when both
+observations are complete and both quality gates pass. This is not evidence of saved Codex/Claude
+subscription quota; quota deltas remain separately attributable, and no live savings are claimed
+until real paired tasks have been measured. `token-harness benchmark-matrix` also aggregates CCR
+usage across complete quality-passed pairs and reports withheld pairs separately. Shadow mode
+remains the default, and switching an owned rule's mode requires rollback before reconfiguration.
+
 ### Evaluation evidence (advanced / maintainers)
 
 Evaluation campaigns are an advanced maintainer workflow; managed setup stays in the unified **Optimization Stack**. The app
@@ -361,17 +438,22 @@ Update Token Harness itself:
 npm install --global token-harness@latest
 ```
 
-Provider checks and reviewed updates are in **Overview -> Health and updates**. The browser first
-checks reviewed channels and, when an installable update exists, offers **Install updates** in the
-same dialog. From the advanced CLI, the preview prints the exact confirmation command:
+Token Harness and optimizer checks are in **Overview -> Health and updates**. The browser checks
+the app's npm installation and the reviewed optimizer channels. When an installable update exists,
+it offers **Install updates** in the same dialog. After updating Token Harness from the browser,
+restart the app to load the new version. From the advanced CLI, the preview prints the exact
+confirmation command:
 
 ```sh
 token-harness update
 token-harness update --yes
 ```
 
-`update` replaces only installed providers whose target is inside the reviewed provider-package
-policy. HarnessTrim uses npm and captures the previous global version for rollback. On native
+`update` installs Token Harness itself only when the running copy matches its global npm package;
+other install methods remain available through their original updater. Optimizer updates replace
+only installed providers whose target is inside the reviewed provider-package policy. The npm
+package inventory captures the previous Token Harness version for rollback. HarnessTrim uses npm
+and captures the previous global version for rollback. On native
 Windows RTK prefers WinGet; when WinGet cannot yet reach the reviewed target, Token Harness can use
 the verified official GitHub Windows x64 release fallback described above. That fallback verifies the
 published digest and post-update version and restores the previous executable on failure.
@@ -471,6 +553,7 @@ does not require that Corepack shim write.
 
 Before changing public behavior or architecture, read
 [RFC 0027](docs/rfcs/0027-optimization-stack-manager.md),
+[RFC 0028](docs/rfcs/0028-smart-model-routing.md),
 [docs/optimizer-priorities.md](docs/optimizer-priorities.md),
 [docs/release-readiness.md](docs/release-readiness.md), [PLAN.md](PLAN.md), and the accepted
 [RFCs](docs/rfcs).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "token-harness",
-  "version": "0.1.17",
+  "version": "0.1.19",
   "description": "Quota-aware efficiency layer for Claude Code and Codex subscription limits.",
   "license": "Apache-2.0",
   "type": "module",

package/sbom.json

@@ -1,14 +1,14 @@
 {
   "bomFormat": "CycloneDX",
   "specVersion": "1.5",
-  "serialNumber": "urn:uuid:acbec121-baa9-de33-8ab5-e5e88efbd50e",
+  "serialNumber": "urn:uuid:fd379d86-ac5b-23d6-ac6a-6b9654496726",
   "version": 1,
   "metadata": {
     "component": {
       "type": "application",
       "bom-ref": "token-harness",
       "name": "token-harness",
-      "version": "0.1.17",
+      "version": "0.1.19",
       "description": "Quota-aware efficiency layer for Claude Code and Codex subscription limits.",
       "licenses": [
         {
@@ -20,7 +20,7 @@
       "hashes": [
         {
           "alg": "SHA-256",
-          "content": "acbec121baa9de338ab5e5e88efbd50e5c4b3f0a1f2c6f7171f80dd05b12f312"
+          "content": "fd379d86ac5b23d6ac6a6b96544967266bf50ee71039397bb7d6a3f3c34373f7"
         }
       ]
     },