claude-token-saver 3.12.1 → 3.13.0

package/README.en.md

@@ -214,11 +214,15 @@ For environments the learner cannot reach, write the mapping yourself in `<userD
 ```jsonc
 {
   "modelAliases": {
-    "arn:aws:bedrock:*:*:application-inference-profile/<PROFILE_ID>": "claude-opus-5"
+    "arn:aws:bedrock:*:*:application-inference-profile/<PROFILE_ID>": "claude-opus-5",
+    "prod-large": "claude-opus-5",   // house aliases map the same way
+    "team-*": "claude-haiku-4-5"
   }
 }
 ```
 
+**Map house aliases that carry no family name** (`prod-large`, `team-fast`) here too. Shapes that keep the family name are recognized as-is — Bedrock (`anthropic.claude-opus-4-5-v1:0`), Vertex (`claude-opus-4-5@20251101`), and the 1M suffix (`claude-sonnet-4-5[1m]`) — but an alias without one cannot be priced. Rather than report a wrong figure, routing-savings **drops those runs from the aggregate** (both sides of the comparison must be recognizable); one line in the table above brings them back.
+
 That file holds internal identifiers in plain text — do not commit it. On a direct-API machine it is never created and behaviour is unchanged.
 
 ## Spike issue codes
@@ -301,6 +305,13 @@ Also update `statusLine.command` in `~/.claude/settings.json` to `claude-token-s
 
 ## Release notes
 
+### v3.13.0 (2026-08-22)
+- **Routing savings are now priced "model before the rule → model that ran it"** — previously the counterfactual was the session's priciest model, which booked a saving even when that model had never handled work of this shape. Each rule now remembers the **model that handled its category before promotion** (its baseline) and prices only against that. The baseline is sticky once set: as a rule takes effect fewer episodes stay on the expensive model, so a recomputed baseline would drift down and shrink the very saving the rule produces.
+- **Delegations no rule covers are excluded** — `Explore`, hand-written agents, and plugin subagents were all being counted, crediting this tool for routing it never did. Ledger events now carry `rule` / `from` / `to`, so every dollar is traceable to the rule and model change behind it.
+- **Ledger schema version 2** — v1 events used the old counterfactual and are discarded rather than migrated; the next `route-scan` refills whatever is genuinely attributable.
+- **House alias model ids no longer price as Sonnet by accident** — when a gateway reports an alias carrying no family name (`prod-large`), the pricing table's default made it Sonnet, fabricating or erasing savings. Both sides of a comparison must now be recognizable ids. Bedrock (`anthropic.claude-opus-4-5-v1:0`), Vertex (`claude-opus-4-5@20251101`), and the `[1m]` suffix are recognized as-is; house aliases come back into the aggregate with one line in `modelAliases` (wildcards supported).
+- `harness check` also reports CLAUDE.md size and whether a `.claudeignore` exists (advisory; the 🅷 score is unaffected).
+
 ### v3.12.1 (2026-08-22)
 - **Routing-savings headline reads more clearly** — the amounts are now in the savings green (the one unambiguously good number on the line), `wk`/`mo`/`all` are spelled out as `weekly`/`monthly`/`total`, and the period now leads the amount. Three bare amounts in a row read as a single number until the eye found the trailing marker.
 

package/README.md

@@ -194,11 +194,15 @@ v3.10.0부터는 프로파일 ID를 역할(main·opus·sonnet·haiku)로 되돌
 ```jsonc
 {
   "modelAliases": {
-    "arn:aws:bedrock:*:*:application-inference-profile/<PROFILE_ID>": "claude-opus-5"
+    "arn:aws:bedrock:*:*:application-inference-profile/<PROFILE_ID>": "claude-opus-5",
+    "prod-large": "claude-opus-5",   // 사내 별칭도 같은 방식으로 매핑됩니다
+    "team-*": "claude-haiku-4-5"
   }
 }
 ```
 
+**모델명에 `opus`·`sonnet`·`haiku`·`fable` 이 들어 있지 않은 사내 별칭**(`prod-large`, `team-fast` 등)도 이 표로 매핑하십시오. Bedrock(`anthropic.claude-opus-4-5-v1:0`)·Vertex(`claude-opus-4-5@20251101`)·1M 접미사(`claude-sonnet-4-5[1m]`) 같이 계열명이 남아 있는 형태는 그대로 인식되지만, 계열명이 사라진 별칭은 가격표가 알아볼 수 없습니다. 이 경우 라우팅 절감 계산은 **틀린 금액을 내놓는 대신 그 실행을 집계에서 제외**하며(비교 양쪽 모두 인식 가능한 이름이어야 합니다), 위 표에 한 줄 추가하면 다시 집계에 들어옵니다.
+
 이 파일에는 사내 식별자가 평문으로 남으므로 저장소에 커밋하지 마십시오. 게이트웨이를 쓰지 않는 환경에서는 파일이 아예 만들어지지 않고 기존 동작이 그대로 유지됩니다.
 
 ## 토큰 급증 원인 코드
@@ -257,6 +261,13 @@ npm uninstall -g claude-cache-monitor && npm i -g claude-token-saver
 
 ## 릴리스 노트
 
+### v3.13.0 (2026-08-22)
+- **라우팅 절감액의 기준을 '승격 전 모델 → 위임 모델' 차액으로 바꿨습니다** — 이전에는 세션의 최상위 모델을 반사실로 잡아, 그 모델이 해당 유형을 실제로 처리한 적이 없어도 차액을 절감으로 기록했습니다. 이제 각 룰이 **승격 전 그 유형을 직접 처리하던 모델**(baseline)을 기억하고, 그 기준 대비로만 계산합니다. baseline은 한 번 정해지면 고정됩니다 — 룰이 효력을 낼수록 직접 처리 사례가 줄어 기준이 흘러내리고, 그러면 룰이 만든 절감이 스스로 작아지기 때문입니다.
+- **룰이 커버하지 않는 위임은 집계에서 뺐습니다** — `Explore`, 직접 만든 에이전트, 플러그인 에이전트처럼 이 도구와 무관하게 돌던 서브에이전트 실행까지 절감으로 잡히고 있었습니다. 도구가 라우팅하지 않은 작업의 절감을 도구 성과로 표시하면 안 됩니다. 원장 이벤트에 `rule`/`from`/`to`를 남겨 어느 룰이 어떤 모델 차이를 만들었는지 추적할 수 있습니다.
+- **원장 스키마 version 2** — 반사실 기준이 달라진 만큼 v1 항목은 마이그레이션 없이 폐기합니다(두 의미를 한 합계에 섞을 수 없습니다). 다음 `route-scan`이 귀속 가능한 절감만 다시 채웁니다.
+- **사내 별칭 모델명이 조용히 Sonnet으로 계산되지 않습니다** — 게이트웨이가 계열명 없는 별칭(`prod-large` 등)을 모델명으로 기록하면 가격표 기본값이 걸려 Sonnet으로 계산됐고, 그 결과 없는 절감이 생기거나 있는 절감이 지워졌습니다. 이제 비교 양쪽 모두 계열명이 남아 있는 id일 때만 집계합니다. Bedrock(`anthropic.claude-opus-4-5-v1:0`)·Vertex(`claude-opus-4-5@20251101`)·1M 접미사(`claude-sonnet-4-5[1m]`)는 그대로 인식되고, 사내 별칭은 `profile-map.json`의 `modelAliases`에 한 줄 추가하면 집계에 복귀합니다(와일드카드 가능).
+- `harness check`가 CLAUDE.md 크기와 `.claudeignore` 유무를 함께 보고합니다(자문 정보, 🅷 점수에는 미반영).
+
 ### v3.12.1 (2026-08-22)
 - **라우팅 절감 헤드라인의 가독성 정리** — 금액을 절감 녹색으로 칠하고(그 줄에서 유일하게 명확한 호재입니다), `wk`·`mo`·`all` 축약을 `weekly`·`monthly`·`total`로 풀었으며, 금액이 앞서던 순서를 뒤집어 기간이 먼저 오게 했습니다. 금액 셋이 연달아 나오면 뒤따르는 기간 표시를 찾기 전까지 한 덩어리로 읽혔습니다.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-token-saver",
-  "version": "3.12.1",
+  "version": "3.13.0",
   "description": "Route the easy work your expensive Claude model keeps repeating down to haiku/sonnet — post-hoc session analysis, no realtime router, no extra LLM calls.",
   "type": "module",
   "bin": {

package/src/cost.js

@@ -128,6 +128,29 @@ export function isUnknownModel(model) {
   return !model || String(model).toLowerCase() === 'unknown';
 }
 
+/**
+ * True when the id actually names a Claude family this table can price, as
+ * opposed to falling through to the Sonnet default.
+ *
+ * `detectPricingTier` must keep defaulting — a plain cost estimate is better
+ * off guessing Sonnet than refusing to answer. But anything that compares two
+ * models must not: behind a company gateway an id can be a house alias
+ * (`prod-large`, `team-fast`) carrying no family name, and pricing that as
+ * Sonnet silently invents or erases a delegation saving. Callers that need a
+ * real comparison gate on this and skip when it is false.
+ *
+ * Covers the shapes gateways actually emit — Bedrock
+ * (`anthropic.claude-opus-4-5-v1:0`, `us.anthropic.…`), Vertex
+ * (`claude-opus-4-5@20251101`), and the `[1m]` context suffix — because all of
+ * them keep the family name in the string. House aliases that do not are
+ * exactly what this returns false for; map those in profile-map.json's
+ * `modelAliases`.
+ */
+export function isRecognizedModelId(model) {
+  if (isUnknownModel(model)) return false;
+  return /fable|mythos|opus|sonnet|haiku/i.test(String(model));
+}
+
 export function modelRank(model) {
   // -1 sits below every real tier, so worthDelegating() rejects it and
   // tierForRank() attributes no saving to it: the run leaves the aggregate

package/src/model-alias.js

@@ -134,8 +134,10 @@ function overrideAlias(model, map) {
   for (const [pattern, alias] of Object.entries(map.modelAliases || {})) {
     if (globMatch(pattern, model)) return alias;
     // Overrides are usually written without the LiteLLM `converse/` prefix.
-    const bare = model.slice(model.indexOf('arn:aws:bedrock:'));
-    if (globMatch(pattern, bare)) return alias;
+    // Only meaningful for ARNs — on a plain id indexOf returns -1 and the
+    // slice would hand the matcher a single trailing character.
+    const at = model.indexOf('arn:aws:bedrock:');
+    if (at > 0 && globMatch(pattern, model.slice(at))) return alias;
   }
   return null;
 }
@@ -150,13 +152,20 @@ function overrideAlias(model, map) {
 export function resolveModelAlias(rawModel, { env = process.env } = {}) {
   if (!rawModel) return UNKNOWN_MODEL;
   const model = String(rawModel);
-  if (!isGatewayModelId(model)) return model;
 
   const map = loadProfileMap();
 
+  // Overrides are consulted for EVERY id, not just ARNs. A gateway can be
+  // configured to report a house alias (`prod-large`, `team-fast`) that names
+  // no Claude family at all; those never reach the ARN branch below, and left
+  // alone they price as Sonnet by default. One `modelAliases` entry maps them
+  // back. Ids that already name a family are left untouched — an override
+  // pattern has to match before anything changes.
   const override = overrideAlias(model, map);
   if (override) return override;
 
+  if (!isGatewayModelId(model)) return model;
+
   const pid = profileIdFrom(model);
   if (!pid) return UNKNOWN_MODEL;
 

package/src/model-rules.js

@@ -340,6 +340,14 @@ export function refreshModelRules(episodeStats, delegatedStats = new Map(), { no
     if (s) {
       r.count = s.count;
       r.errRate = s.epCount > 0 ? s.errCount / s.epCount : 0;
+      // Baseline = the model that handled this category before the rule moved
+      // it. Sticky once set: as a rule takes effect, fewer episodes stay on the
+      // expensive model, so a recomputed baseline would drift downward and
+      // shrink the very savings the rule is producing.
+      if (!r.baselineModel && s.baselineModel) {
+        r.baselineModel = s.baselineModel;
+        changed = true;
+      }
     }
     // Window snapshot, not a running total: these describe the current scan
     // window so a rule that stopped firing decays to zero instead of coasting

package/src/route-scan.js

@@ -23,7 +23,7 @@ import { userDataDir } from './paths.js';
 import { discoverSessionFiles } from './parser.js';
 import { collectSessionRecords } from './session-records.js';
 import { collectSubagentRuns, indexRuns, runsForEpisode } from './subagent-records.js';
-import { estimateCost, modelRank, TIER_TARGET_RANK, tierForRank } from './cost.js';
+import { estimateCost, modelRank, isRecognizedModelId, TIER_TARGET_RANK, tierForRank } from './cost.js';
 import { learnProfileMapping, resetModelAliasCache } from './model-alias.js';
 import { agentPhrase, agentPhraseEn } from './agents.js';
 
@@ -379,10 +379,24 @@ export async function runRouteScan({ days = 14 } = {}) {
   const episodeStats = new Map(); // "category|project" (+ "category|*") → outcome stats
   let tieredEpisodes = 0;
   const bumpStats = (key, ep) => {
-    const s = episodeStats.get(key) || { count: 0, errCount: 0, epCount: 0 };
+    const s = episodeStats.get(key) || { count: 0, errCount: 0, epCount: 0, baselineModel: null, baselineRank: -1 };
     s.count += 1;
     s.epCount += 1;
     if (ep.errors > 0) s.errCount += 1;
+    // Baseline model: what actually handled this category BEFORE any rule sent
+    // it elsewhere. This is the only honest counterfactual for "routing saved
+    // money" — the session's priciest model is not, since it may never have
+    // touched work of this shape. Episodes counted here are by definition ones
+    // an expensive model handled directly, so the priciest model seen on them
+    // is the before-picture the rule replaced.
+    for (const m of ep.models) {
+      // Only ids the pricing table really recognizes may become a baseline —
+      // an unresolved gateway id or a house alias would be priced as Sonnet
+      // and quietly rewrite every saving computed against it.
+      if (!isRecognizedModelId(m)) continue;
+      const r = modelRank(m);
+      if (r > s.baselineRank) { s.baselineRank = r; s.baselineModel = m; }
+    }
     episodeStats.set(key, s);
   };
 
@@ -454,7 +468,33 @@ export async function runRouteScan({ days = 14 } = {}) {
   // Ledger events feed the statusline's weekly/monthly "Routing saved"
   // totals. Keyed by run transcript path so overlapping re-scans upsert the
   // same event instead of double-counting it.
+  //
+  // What counts as a routing saving is narrower than what counts for
+  // rule-health above. A saving is the price difference a REGISTERED RULE
+  // caused: the model that used to handle this category (the rule's baseline,
+  // learned from episodes an expensive model handled directly) versus the
+  // model the run actually used. Subagent runs that no rule covers —
+  // Explore, a hand-written agent, a plugin's own subagent — would have gone
+  // to the same cheap model with or without this tool, so attributing their
+  // savings here would credit the tool for work it did not route.
   const ledgerEvents = [];
+  let ledgerRules = [];
+  try {
+    const { loadModelRules } = await import('./model-rules.js');
+    ledgerRules = loadModelRules().rules.filter((r) => r.status !== 'off');
+  } catch { /* no registry → no attributable savings, which is the honest zero */ }
+  // A rule's baseline: what it stored at promotion, else what this scan still
+  // observes handling the category directly (a rule promoted before baselines
+  // existed backfills on the next refresh).
+  const baselineFor = (rule, tier, catId, projectDir) => {
+    if (rule.baselineModel) return rule.baselineModel;
+    const s = episodeStats.get(`${tier}|${catId}|${projectDir}`)
+      || (rule.scope === 'global' ? episodeStats.get(`${tier}|${catId}|*`) : null);
+    return s?.baselineModel || null;
+  };
+  const ruleForRun = (tier, catId, projectDir) => ledgerRules.find((r) =>
+    r.tier === tier && r.category === catId &&
+    (r.scope === 'global' || r.project === projectDir));
   for (const [sessionPath, index] of runIndexBySession) {
     const used = new Set();
     for (const { ep, projectDir, sessionPath: epSession } of all) {
@@ -478,11 +518,27 @@ export async function runRouteScan({ days = 14 } = {}) {
         const saved = runSaving(run, mainModel);
         bumpDelegated(`${runTier}|${cat.id}|${projectDir}`, run, saved);
         bumpDelegated(`${runTier}|${cat.id}|*`, run, saved);
-        if (saved > 0) {
+        // Routing saving: priced against the rule's baseline (before → after),
+        // not against whatever the session's priciest model happened to be.
+        const rule = ruleForRun(runTier, cat.id, projectDir);
+        if (!rule) continue; // no rule routed this run — not our saving to claim
+        const baseline = baselineFor(rule, runTier, cat.id, projectDir);
+        if (!baseline) continue; // baseline unknown → no honest counterfactual
+        // Both sides of the comparison must be ids the pricing table really
+        // recognizes. A house alias from a company gateway prices as Sonnet by
+        // default, which would fabricate a saving against a cheap baseline or
+        // erase a real one — worse than showing nothing. Map such ids in
+        // profile-map.json's `modelAliases` to bring these runs back in.
+        if (!isRecognizedModelId(baseline) || !isRecognizedModelId(run.model)) continue;
+        const routed = runSaving(run, baseline);
+        if (routed > 0) {
           ledgerEvents.push({
             key: run.path,
             ts: run.endedAt ?? run.startedAt ?? Date.now(),
-            usd: saved,
+            usd: routed,
+            rule: rule.signature,
+            from: baseline,
+            to: run.model,
           });
         }
       }

package/src/savings-ledger.js

@@ -8,7 +8,15 @@
  * upsert instead of double-counting.
  *
  * File: <userDataDir>/delegation-ledger.json
- *   { "events": { "<run path>": { "ts": <ms epoch>, "usd": <number> } } }
+ *   { "version": 2,
+ *     "events": { "<run path>": { "ts", "usd", "rule", "from", "to" } } }
+ *
+ * `from`/`to` are the models the routing decision moved between, and `usd` is
+ * the price difference between them for this run's tokens. Version 1 priced
+ * every downgraded subagent run against the session's priciest model, which
+ * credited the tool for runs no rule of its own had routed; those events are
+ * discarded rather than migrated, since the number they carry cannot be
+ * recomputed without a rescan (which route-scan does anyway).
  *
  * Best-effort like every other state file here: an unreadable ledger reads as
  * empty, and the statusline renders totals of 0 as "no chip".
@@ -25,32 +33,48 @@ export function ledgerPath() {
   return join(userDataDir(), 'delegation-ledger.json');
 }
 
+export const LEDGER_VERSION = 2;
+
 export function loadLedger() {
   try {
     const data = JSON.parse(readFileSync(ledgerPath(), 'utf8'));
-    return data && typeof data.events === 'object' && data.events !== null
-      ? data
-      : { events: {} };
+    if (!data || typeof data.events !== 'object' || data.events === null) {
+      return { version: LEDGER_VERSION, events: {} };
+    }
+    // Pre-v2 events were priced against a different counterfactual — drop them
+    // instead of mixing two meanings into one total. The next scan rebuilds
+    // whatever is still attributable.
+    if (data.version !== LEDGER_VERSION) return { version: LEDGER_VERSION, events: {} };
+    return data;
   } catch {
-    return { events: {} };
+    return { version: LEDGER_VERSION, events: {} };
   }
 }
 
 /**
- * Upsert saving events. `events` is an array of { key, ts, usd } where `key`
- * is the run's transcript path (unique per subagent run). Zero-saving runs
- * are skipped — they carry no information the totals care about.
+ * Upsert saving events. `events` is an array of
+ * { key, ts, usd, rule, from, to } where `key` is the run's transcript path
+ * (unique per subagent run) and `from`/`to` name the models the routing
+ * decision moved between. Zero-saving runs are skipped — they carry no
+ * information the totals care about.
  */
 export function recordDelegationEvents(events) {
   if (!Array.isArray(events) || events.length === 0) return;
   const data = loadLedger();
+  data.version = LEDGER_VERSION;
   let changed = false;
   for (const e of events) {
     if (!e || !e.key || !(Number(e.usd) > 0) || !Number.isFinite(e.ts)) continue;
     const prev = data.events[e.key];
     const usd = Math.round(Number(e.usd) * 10000) / 10000;
     if (prev && prev.ts === e.ts && prev.usd === usd) continue;
-    data.events[e.key] = { ts: e.ts, usd };
+    data.events[e.key] = {
+      ts: e.ts,
+      usd,
+      ...(e.rule ? { rule: e.rule } : {}),
+      ...(e.from ? { from: e.from } : {}),
+      ...(e.to ? { to: e.to } : {}),
+    };
     changed = true;
   }
   if (!changed) return;