claude-token-saver 3.3.1 → 3.4.1

package/README.en.md

@@ -143,28 +143,16 @@ An auto `.bak` is kept, but **the session context that earned the rule its place
 
 ## 🔀 route-scan — "this recurring task could run on a cheaper tier"
 
-Analyzes your session history at the **episode (user request) level**, classifies the work your expensive model (opus/fable) keeps doing into **tiers**, and proposes delegation rules. Fully local, zero token cost. Criteria design and evidence: [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md).
+Finds the easy work your expensive model (opus/fable) keeps redoing in your session logs and proposes **haiku/sonnet delegation rules**. Fully local, zero token cost.
 
-- **T2 → haiku**: few calls, tiny output, near-zero mutation, no errors (lookups, pasted-screen Q&A, simple runs)
-- **T1 → sonnet**: moderate output, few mutations, ≤1 error (build pipelines, status checks)
-- **T0 stays**: repeated errors, heavy mutation, big output, design/analysis keywords — the session model keeps it
+- **T2 → haiku**: lookups, pasted-screen Q&A, simple runs — zero errors, near-zero mutation
+- **T1 → sonnet**: build pipelines, status checks — few mutations, ≤1 error
+- **T0 stays**: repeated errors, heavy mutation, design/analysis — the session model keeps it
 
-Signals: call count, output tokens, **mutating tool calls (Edit/Write/Bash)**, **tool errors**, and request text. Output thresholds are auto-calibrated from your own 14-day distribution (fixed constants drift with workload).
-
-<details>
-<summary>Why this design — research evidence (deep-research, 21 cross-verified sources + 553 locally measured episodes)</summary>
-
-A survey of existing LLM-routing research and systems shaped each axis of the design:
-
-- **Conservative demotion — "only send down what's clearly easy" — is the unexplored direction.** In RouterArena (2025), every academic and commercial router fell far short of the oracle (90.9%; best entrant 66.9), and the failure mode was consistently **over-routing to expensive models**. Per the routing-collapse work, the oracle needs the top model for under 20% of queries — while 73% of the author's episodes were still on the top tier. The headroom is large.
-- **Difficulty is defined by outcome, not text guessing.** RouterArena labels difficulty as "how many of 42 models actually got it right". Our logs already contain outcomes: tool errors, mutating tool calls, episode length. Local measurement confirms it — error incidence splits the tiers sharply (T2 candidates 2% / T1 8% / T0 36%).
-- **Thresholds calibrate to the user's own distribution.** RouteLLM's stated limitation is that fixed thresholds drift as the query distribution shifts — so output-token thresholds are recomputed from your last 14 days' p25/p75 (clamped).
-- **An escalation path makes demotion mistakes cheap.** The FrugalGPT cascade lesson ("start cheap, escalate on failure"). In Claude Code this falls out naturally: when a subagent gets stuck, the main model takes over — and promoted rules spell out that guard.
-- **Subagent `model:` is Anthropic's official cost-control mechanism.** The docs recommend routing to cheaper models like Haiku for cost control — yet Claude Code itself does no automatic model routing; even trivial turns resend the full context on the session model. That gap is exactly what route-scan fills.
-- **Learned classifiers (BERT/embeddings) deliberately excluded.** Even the best is 24 points off the oracle, we have no preference-pair training data, and it would break the zero-dependency principle. Regex categories + outcome signals are the current position.
-
-Sources (RouteLLM, FrugalGPT, RouterArena, routing-survey arxiv links), the full local-measurement table, and the complete tier criteria: [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md).
-</details>
+Three design pillars:
+1. Difficulty is judged by **outcome, not text guessing** — tool errors, mutating tool calls, output tokens
+2. Thresholds **auto-calibrate to your own 14-day distribution** — fixed constants drift with workload
+3. Promoted rules live in a tool-owned file (`.claude/ratchet-model.md`) that **refreshes itself every scan**, and a `⚠ rule-health` flag fires when a delegated category's error rate climbs — rules report their own staleness
 
 ```bash
 claude-token-saver route-scan                    # scan (24h cache) + tiered candidates
@@ -173,21 +161,7 @@ claude-token-saver route-scan dismiss 1          # not interested — won't resu
 claude-token-saver route-scan rules              # list model-fitting rules (rm <N> to remove)
 ```
 
-### Model-fitting ratchet — a separate file, continuously refreshed
-
-Promoted delegation rules never mix with hand-written ratchet rules: they live in a **separate, fully tool-owned file** — `.claude/ratchet-model.md` per project, `~/.claude/ratchet-model.md` for global scope — regenerated wholesale on every scan, and they stay alive afterward:
-
-- **Auto-refresh**: every rescan recomputes recurrence counts and the category's error rate from fresh logs and rewrites the file. Your `ratchet.md` is never touched by stat churn, so repos that commit `.claude/` see no diff noise (`ratchet-model.md` is safe to gitignore — it's always regenerable from the registry).
-- **rule-health**: when the delegated category's error rate exceeds 20%, the rule gets a `⚠ rule-health` flag suggesting you narrow or remove it — the "define difficulty by outcome" principle applied to rule lifecycle.
-- Your rules are managed by `harness list/rm`; model-fitting rules by `route-scan rules [rm <N>]` — separate files, separate indexes.
-- The CLAUDE.md ratchet section planted by `harness init` references both files, so Claude applies them together (existing users: re-run `harness init` to refresh the block).
-
-How it works (session-boundary calibration, NOT a real-time router):
-1. `install` registers a SessionStart hook that injects the cached scan results as session context on startup and `/clear`. Rescans are **data-triggered, not time-triggered**: ~5MB of new transcripts since the last scan rescans immediately, a small trickle rescans daily, and no change means no rescan at all (an unchanged-input scan is deterministic). A 1-hour minimum-interval guard applies, and promoting a rule triggers one immediate refresh to establish its stat baseline.
-2. When a recurring (≥3×) pattern exists, the statusline shows a `🅷⚠ route? R1` chip and Claude asks you whether to register it, and at which scope (`--project`/`--global`).
-3. Promoted rules make **the main model delegate that work type to a haiku/sonnet subagent automatically from the next session on**.
-
-Recommended companion setup: create `model: haiku` subagents under `~/.claude/agents/` (e.g. haiku-explore / haiku-runner / haiku-translate) plus a `model: sonnet` general worker so the rules are immediately actionable.
+Dig deeper: **tier criteria & research evidence** → [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) (Korean) · **rule-file mechanics, scan triggers, subagent setup** → [docs/ROUTE_SCAN.md](./docs/ROUTE_SCAN.md) (Korean + English)
 
 ## Spike issue codes
 

package/README.md

@@ -124,28 +124,16 @@ ratchet의 가치는 **한 방향 누적**에 있습니다. 룰을 가볍게 지
 
 ## 🔀 route-scan — "이 반복 작업, 더 싼 티어로 내려도 됩니다"
 
-세션 기록을 **에피소드(사용자 요청) 단위**로 분석해 상위 모델(opus/fable)이 반복 처리해 온 작업을 **티어로 분류**하고, 위임 룰로 승격하도록 제안합니다. 전 과정 로컬, 토큰 비용 0. 기준 설계와 근거는 [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) 참고.
+세션 로그에서 상위 모델(opus/fable)이 반복 처리해 온 쉬운 작업을 찾아 **haiku/sonnet 위임 룰로 승격**을 제안합니다. 전 과정 로컬, 토큰 비용 0.
 
-- **T2 → haiku**: 적은 호출·작은 출력·변경 거의 없음·에러 0으로 끝난 요청 (탐색·조회, 붙여넣은 화면 질문, 단순 실행 등)
-- **T1 → sonnet**: 중간 출력·변경 소수·에러 ≤1 (빌드 파이프라인, 상태 점검 등)
-- **T0 유지**: 에러 반복, 대량 변경, 큰 출력, 설계·분석 키워드 — 세션 모델이 계속 담당
+- **T2 → haiku**: 탐색·조회·단순 실행 — 에러 0, 변경 거의 없음
+- **T1 → sonnet**: 빌드·상태 점검 — 변경 소수, 에러 ≤1
+- **T0 유지**: 에러 반복·대량 변경·설계/분석 — 세션 모델이 계속 담당
 
-판정 신호는 호출 수·출력 토큰·**변경성 도구 수(Edit/Write/Bash)**·**도구 에러 수**·요청 텍스트이고, 출력 임계값은 사용자 자신의 최근 14일 분포에서 자동 보정됩니다 (고정 상수는 워크로드가 바뀌면 어긋나기 때문).
-
-<details>
-<summary>왜 이 설계인가 — 리서치 근거 (deep-research 소스 21개 교차검증 + 로컬 실측 553 에피소드)</summary>
-
-기존 LLM 라우팅 연구·시스템을 조사한 결과가 설계의 각 축을 결정했습니다:
-
-- **"쉬운 것만 확실히 내리는" 보수적 강등이 미개척 지대** — RouterArena(2025) 평가에서 학술·상용 라우터 전부 오라클(90.9%)에 크게 못 미쳤고(1위 66.9), 실패 방향은 일관되게 **비싼 모델로의 과잉 라우팅**이었습니다. 오라클 기준 최상위 모델이 정말 필요한 쿼리는 20% 미만인데(routing collapse 연구), 저자 로그에선 73%가 최상위 티어에 남아 있었습니다 — 헤드룸이 큽니다.
-- **난이도는 텍스트 추정이 아니라 결과(outcome)로 정의** — RouterArena는 난이도를 "42개 모델 중 몇 개가 실제로 맞혔나"로 라벨링했습니다. 우리 로그에도 결과가 이미 있습니다: 도구 에러·변경성 도구 수·에피소드 길이. 로컬 실측에서도 에러 경험률이 티어를 강하게 갈랐습니다 (T2 후보 2% / T1 8% / T0 36%).
-- **임계값은 사용자 분포에서 보정** — RouteLLM의 공식 한계가 "고정 임계값은 쿼리 분포가 바뀌면 어긋난다"입니다. 그래서 출력 토큰 임계값을 최근 14일 p25/p75에서 재계산합니다(클램프 포함).
-- **실패 시 승급 경로가 강등 오판 비용을 낮춤** — FrugalGPT 캐스케이드("싼 모델부터, 안 되면 위로")의 교훈. Claude Code에선 서브에이전트가 막히면 메인 모델이 이어받는 형태로 자연 구현되고, 승격 룰 텍스트에도 이 가드가 명시됩니다.
-- **서브에이전트 `model:`은 Anthropic 공식 비용 통제 메커니즘** — 공식 문서가 "Haiku 등 싼 모델로 라우팅해 비용 통제"를 권장하지만, Claude Code 본체는 자동 모델 라우팅을 하지 않습니다. 사소한 턴도 세션 모델이 전체 컨텍스트를 재전송하며 처리하는 이 공백이 route-scan이 메우는 자리입니다.
-- **학습형 분류기(BERT/임베딩)는 의도적으로 배제** — 최고 성능도 오라클과 24점 차이인 데다 학습 데이터가 없고 zero-dep 원칙에 어긋납니다. 정규식 카테고리 + outcome 신호 조합이 현재 위치.
-
-출처(RouteLLM·FrugalGPT·RouterArena·라우팅 서베이 arxiv 링크)와 로컬 실측 상세 테이블, 티어 기준 전문은 [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) 참고.
-</details>
+핵심 설계는 세 가지입니다:
+1. 난이도는 텍스트 추측이 아니라 **결과(outcome)로 판정** — 도구 에러·변경성 도구 수·출력 토큰
+2. 임계값은 **내 최근 14일 로그 분포에서 자동 보정** — 고정 상수는 워크로드가 바뀌면 어긋나므로
+3. 승격된 룰은 도구 소유 별도 파일(`.claude/ratchet-model.md`)에서 **자동 갱신**되고, 위임 후 에러율이 높아지면 `⚠ rule-health`로 경고 — 룰이 스스로 낡음을 알림
 
 ```bash
 claude-token-saver route-scan                    # 스캔 (24h 캐시) + 티어별 후보 출력
@@ -154,21 +142,7 @@ claude-token-saver route-scan dismiss 1          # 관심 없으면 무시 (재
 claude-token-saver route-scan rules              # 등록된 모델 피팅 룰 목록 (rm <N>으로 제거)
 ```
 
-### 모델 피팅 랫쳇 — 사용자 룰과 파일부터 분리, 로그 기반 자동 갱신
-
-승격된 위임 룰은 손으로 쓴 랫쳇 룰과 섞이지 않도록 **별도 파일**에 저장됩니다 — 프로젝트는 `.claude/ratchet-model.md`, 글로벌은 `~/.claude/ratchet-model.md`. 이 파일은 전적으로 도구 소유라 매 스캔마다 통째로 재생성되며, 이후에도 살아 움직입니다:
-
-- **자동 갱신**: 매 스캔마다 반복 횟수·해당 유형의 에러율을 최신 로그로 다시 계산해 파일을 재작성합니다. 통계가 바뀌어도 사용자의 `ratchet.md`는 전혀 건드리지 않으므로, `.claude/`를 커밋하는 프로젝트에서도 diff 소음이 없습니다 (`ratchet-model.md`는 gitignore해도 무방 — 레지스트리에서 항상 재생성 가능).
-- **rule-health**: 위임 대상 유형의 에러율이 20%를 넘으면 룰에 `⚠ rule-health` 경고가 붙어 조건을 좁히거나 제거하라고 알립니다 — "결과(outcome)로 난이도를 정의"하는 원칙을 룰 수명 관리에 재적용한 것.
-- 사용자 룰은 `harness list/rm`, 모델 피팅 룰은 `route-scan rules [rm <N>]`로 각각 관리 — 서로의 파일도 인덱스도 침범하지 않습니다.
-- `harness init`이 심는 CLAUDE.md 랫쳇 섹션이 두 파일을 모두 참조하므로 Claude가 세션에서 함께 적용합니다 (기존 사용자는 `harness init` 재실행으로 블록 갱신).
-
-동작 구조 (실시간 라우팅이 아니라 **세션 경계 캘리브레이션**):
-1. `install` 시 SessionStart 훅이 등록되어, 새 세션 시작·`/clear` 때 캐시된 스캔 결과를 세션 컨텍스트로 주입합니다. 재스캔은 시간이 아니라 **데이터가 트리거**: 마지막 스캔 이후 새 transcript가 ~5MB 이상 쌓이면 즉시, 소량이면 하루 1회, 아무 변화가 없으면 아예 돌지 않습니다 (변화 없는 재스캔은 결과가 동일하므로). 최소 간격 1시간 가드 포함, 룰 등록(promote) 직후에는 통계 기준선 확보를 위해 즉시 1회.
-2. 반복(≥3회) 패턴이 있으면 statusline에 `🅷⚠ route? R1` 칩이 뜨고, Claude가 등록 여부와 scope(`--project`/`--global`)를 물어봅니다.
-3. 등록된 룰은 **다음 세션부터 메인 모델이 해당 유형을 haiku/sonnet 서브에이전트로 자동 위임**하게 합니다.
-
-권장 사전 준비: `~/.claude/agents/`에 `model: haiku` 서브에이전트(예: haiku-explore / haiku-runner / haiku-translate)와 `model: sonnet` 범용 서브에이전트를 만들어 두면 룰이 바로 실행 가능해집니다.
+더 알아보기: **티어 기준·리서치 근거** → [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) · **룰 파일 구조·스캔 트리거·서브에이전트 준비** → [docs/ROUTE_SCAN.md](./docs/ROUTE_SCAN.md)
 
 ## 토큰 급증 원인 코드
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-token-saver",
-  "version": "3.3.1",
+  "version": "3.4.1",
   "description": "Save tokens on Claude Code — spike diagnosis, 1M-context detection, TTL countdown, statusline. (formerly claude-cache-monitor)",
   "type": "module",
   "bin": {

package/src/model-rules.js

@@ -35,6 +35,10 @@ import { homedir } from 'node:os';
 // Post-promotion delegated-category error rate above this flags the rule
 // for review (rule-health). Calibrated against local T0 avg error incidence.
 export const HEALTH_ERR_RATE = 0.2;
+// Below this many shape-eligible episodes in the window the error rate is
+// noise (1 error in 4 episodes = 25% — instant flag), so the review flag is
+// withheld until the sample is large enough to mean something.
+export const HEALTH_MIN_SAMPLE = 10;
 
 function stateDir() {
   if (process.platform === 'win32') {
@@ -104,12 +108,37 @@ export function renderModelRatchet(rules) {
     '## Rules',
     '',
   ];
+  const healthOf = (r) => r.status === 'review'
+    ? ` ⚠ rule-health: 최근 위임 대상 에러율 ${Math.round((r.errRate || 0) * 100)}% — 조건을 좁히거나 제거 검토`
+    : '';
+  const statsOf = (r) => `×${r.count || 0}, err ${Math.round((r.errRate || 0) * 100)}%, seen ${r.lastSeen || r.promotedAt}`;
+
+  // A category can carry both a T2 (haiku) and a T1 (sonnet) rule. Tier is
+  // only known after an episode finishes, so two separate bullets give the
+  // reading LLM no way to pick one at request time — merge such pairs into a
+  // single conditional rule with explicit request-time criteria (default to
+  // the cheap agent, escalate on multi-step scope, hand back on judgement /
+  // irreversible actions).
+  const byCategory = new Map();
   for (const r of rules) {
-    const health = r.status === 'review'
-      ? ` ⚠ rule-health: 최근 위임 대상 에러율 ${Math.round((r.errRate || 0) * 100)}% — 조건을 좁히거나 제거 검토`
-      : '';
-    const stats = ` <!-- ×${r.count || 0}, err ${Math.round((r.errRate || 0) * 100)}%, seen ${r.lastSeen || r.promotedAt} -->`;
-    lines.push(`- ${r.rule}${health}${stats}`);
+    if (!byCategory.has(r.category)) byCategory.set(r.category, []);
+    byCategory.get(r.category).push(r);
+  }
+  for (const group of byCategory.values()) {
+    const t2 = group.find((r) => r.tier === 'T2');
+    const t1 = group.find((r) => r.tier === 'T1');
+    if (t2 && t1) {
+      const rule =
+        `"${t2.label}" 유형 요청은 기본적으로 ${t2.agent}(haiku) 서브에이전트로 위임한다(예: "${t2.example}"). ` +
+        `여러 단계·여러 파일 수정이 얽힌 중간 난도 요청(예: "${t1.example}")은 model: sonnet 서브에이전트로 위임한다. ` +
+        `설계 판단·배포·스토어 제출 같은 비가역 작업이 섞이거나 위임 중 에러가 반복되면 위임하지 말고 메인 모델이 직접 처리한다`;
+      lines.push(`- ${rule}${healthOf(t2)}${healthOf(t1)} <!-- T2 ${statsOf(t2)} / T1 ${statsOf(t1)} -->`);
+      for (const r of group) {
+        if (r !== t2 && r !== t1) lines.push(`- ${r.rule}${healthOf(r)} <!-- ${statsOf(r)} -->`);
+      }
+    } else {
+      for (const r of group) lines.push(`- ${r.rule}${healthOf(r)} <!-- ${statsOf(r)} -->`);
+    }
   }
   return lines.join('\n') + '\n';
 }
@@ -171,8 +200,10 @@ export function syncAllFiles({ previousPaths = [] } = {}) {
  * for each registered rule, recompute recurrence count and the error rate
  * of episodes in its (tier-eligible) category — the rule-health signal.
  *
- * `episodeStats`: Map "category|project" (plus a "category|*" wildcard key
- * that global-scope rules fall back to) → { count, errCount, epCount }.
+ * `episodeStats`: Map "tier|category|project" (plus a "tier|category|*"
+ * wildcard key that global-scope rules fall back to) → { count, errCount,
+ * epCount }. Tier is part of the key so a category carrying both a T2 and a
+ * T1 rule doesn't double-count every episode into both rules' stats.
  * errCount/epCount measure the scan window's shape-eligible episodes — ones
  * an expensive model handled directly that still look T1/T2 by shape (tier
  * judged with the error signal zeroed; see route-scan's rule-health pass).
@@ -181,13 +212,13 @@ export function refreshModelRules(episodeStats, { now } = {}) {
   const data = loadModelRules();
   let changed = false;
   for (const r of data.rules) {
-    const s = episodeStats.get(`${r.category}|${r.project}`)
-      || (r.scope === 'global' ? episodeStats.get(`${r.category}|*`) : null);
+    const s = episodeStats.get(`${r.tier}|${r.category}|${r.project}`)
+      || (r.scope === 'global' ? episodeStats.get(`${r.tier}|${r.category}|*`) : null);
     if (!s) continue;
     r.count = s.count;
     r.errRate = s.epCount > 0 ? s.errCount / s.epCount : 0;
     r.lastSeen = now || r.lastSeen;
-    r.status = r.errRate > HEALTH_ERR_RATE ? 'review' : 'active';
+    r.status = r.errRate > HEALTH_ERR_RATE && s.epCount >= HEALTH_MIN_SAMPLE ? 'review' : 'active';
     changed = true;
   }
   if (changed) {

package/src/route-scan.js

@@ -42,7 +42,10 @@ export const T0_MIN_MUTATING = 7;
 // acks, feedback) — skip entirely.
 export const MIN_DELEGABLE_OUT = 100;
 // Escalation keywords: design/analysis judgement stays on the top tier.
-export const ESCALATE_RE = /설계|아키텍처|리팩토링|원인 분석|개선할|검토해보|비교|왜 |analyze|compare|evaluate|architect|refactor/i;
+// Irreversible/external actions (store submission, deploy, release, merge)
+// are included — they may look like light "run" episodes in the logs, but
+// delegating them defeats the harness's default-safe-path rule.
+export const ESCALATE_RE = /설계|아키텍처|리팩토링|원인 분석|개선할|검토해보|비교|왜 |제출|배포|출시|analyze|compare|evaluate|architect|refactor|submit|deploy|release|publish|merge/i;
 // A pattern must recur this often before we nag about it.
 export const MIN_RECURRENCE = 3;
 
@@ -58,50 +61,85 @@ export const RESCAN_MIN_INTERVAL_MS = 60 * 60 * 1000;      // never more than ho
 export const RESCAN_BIG_DELTA_BYTES = 5 * 1024 * 1024;     // this much new data → rescan now
 export const RESCAN_MAX_AGE_MS = 24 * 60 * 60 * 1000;      // any new data + a day old → rescan
 
-// Category → recommended subagent. First match wins; order matters
-// (paste before everything: keywords inside pasted UI/log text would
-// otherwise mislabel the episode; translate before read: "번역해줘" also
-// matches the read keywords).
+// Categories → recommended subagent. Classification is behavior-first with
+// weighted keyword scoring as fallback — see categorize().
 const PASTE_MIN_LEN = 400;
 const CATEGORIES = [
   {
     id: 'paste',
     label: '붙여넣은 화면·로그 질문',
     agent: 'haiku-explore',
-    re: null, // matched by length, see categorize()
+    kw: null, // matched by length, see categorize()
   },
   {
     id: 'translate',
     label: '배치 번역·정형 텍스트 변환',
     agent: 'haiku-translate',
-    re: /번역|translate|변환해|표로 정리|포맷팅/i,
+    kw: [[/번역|translate/i, 2], [/변환해|표로 정리|포맷팅/i, 1]],
   },
   {
     id: 'explore',
     label: '탐색·조회 (파일/값 찾기)',
     agent: 'haiku-explore',
-    re: /grep|검색|찾아|search|find|어디|위치|목록|살펴/i,
+    kw: [[/grep|검색|search|find/i, 2], [/찾아|어디|위치|목록|살펴/i, 1]],
   },
   {
     id: 'read',
     label: '읽기·요약·설명',
     agent: 'haiku-explore',
-    re: /읽어|요약|설명|정리해|summar|explain|보여줘|알려줘|뭐야|what/i,
-  },
-  {
-    id: 'run',
-    label: '명령 실행 (빌드·테스트·git)',
-    agent: 'haiku-runner',
-    re: /git |commit|push|실행|돌려|run |build|빌드|테스트|npm |pip|설치/i,
+    kw: [[/요약|summar|explain/i, 2], [/읽어|설명|정리해|보여줘|알려줘|뭐야|what/i, 1]],
   },
   {
     id: 'check',
     label: '상태 확인·검증',
     agent: 'haiku-explore',
-    re: /확인|맞아\?|되나|됐나|괜찮|체크|check|verify|status|점검/i,
+    kw: [[/확인|검증|verify|점검/i, 2], [/맞아\?|되나|됐나|됐어|되는지|괜찮|체크|check|status/i, 1]],
+  },
+  {
+    id: 'run',
+    label: '명령 실행 (빌드·테스트·git)',
+    agent: 'haiku-runner',
+    kw: [[/git |commit|push|npm |pip|빌드해|빌드 돌/i, 2], [/실행|돌려|run |build|빌드|테스트|설치/i, 1]],
   },
 ];
 
+// ── Behavior signal (1st) — what the episode actually DID ────────────────
+// The tool-call histogram is ground truth the prompt's wording is not:
+// "테스트 통과했는지 확인해줘" that actually ran `npx playwright test` IS a
+// run episode regardless of phrasing. Keyword scores only pick within (or,
+// when behavior is inconclusive, across) the plausible pool.
+const RUN_TOOLS = new Set(['Bash']);
+const LOOKUP_TOOLS = new Set(['Read', 'Grep', 'Glob', 'LS', 'WebFetch', 'WebSearch']);
+const WRITE_TOOLS = new Set(['Write', 'Edit', 'NotebookEdit']);
+const MIN_BEHAVIOR_CALLS = 2; // fewer calls than this → too little behavior to trust
+
+/**
+ * Narrow the candidate categories from the episode's tool mix.
+ * Returns an array of category ids (ordered — first is the default when
+ * keywords stay silent), or null when behavior is inconclusive.
+ */
+export function behaviorPool(toolCounts) {
+  let run = 0, lookup = 0, write = 0, total = 0;
+  for (const [name, n] of Object.entries(toolCounts || {})) {
+    total += n;
+    if (RUN_TOOLS.has(name)) run += n;
+    else if (LOOKUP_TOOLS.has(name)) lookup += n;
+    else if (WRITE_TOOLS.has(name)) write += n;
+  }
+  if (total < MIN_BEHAVIOR_CALLS) return null;
+  if (run > lookup + write) return ['run'];
+  if (lookup > run + write) return ['explore', 'read', 'check', 'translate'];
+  return null; // mixed / write-heavy — no reliable verdict, keywords decide
+}
+
+/** Weighted keyword score for one category (0 when it has no kw table). */
+function keywordScore(cat, text) {
+  if (!cat.kw) return 0;
+  let score = 0;
+  for (const [re, w] of cat.kw) if (re.test(text)) score += w;
+  return score;
+}
+
 // Episodes that are not user-delegable requests: bare continuations, injected
 // notifications, image pastes. These are easy but there is nothing to route.
 const SKIP_RE = /^(계속|이어서|continue|다음|proceed|진행|응|네|넵|ok|okay|yes|ㄱ+|고고)\b/i;
@@ -127,10 +165,27 @@ export function mungeProjectPath(p) {
   return String(p).replace(/[^a-zA-Z0-9-]/g, '-');
 }
 
-function categorize(text) {
+/**
+ * Classify an episode. Gates (paste by length) run first, then the tool-mix
+ * behavior signal narrows the candidate pool, then weighted keyword scores
+ * pick within it (highest score wins; ties fall back to CATEGORIES order).
+ * A behavior verdict without any keyword hit still classifies (pool's first
+ * id); no behavior AND no keyword hit → null (nothing delegable to name).
+ */
+export function categorize(text, toolCounts) {
   if (text.length >= PASTE_MIN_LEN) return CATEGORIES.find((c) => c.id === 'paste');
-  for (const c of CATEGORIES) if (c.re && c.re.test(text)) return c;
-  return null;
+  const pool = behaviorPool(toolCounts);
+  const eligible = pool
+    ? pool.map((id) => CATEGORIES.find((c) => c.id === id))
+    : CATEGORIES.filter((c) => c.kw);
+  let best = null;
+  let bestScore = 0;
+  for (const c of eligible) {
+    const s = keywordScore(c, text);
+    if (s > bestScore) { best = c; bestScore = s; }
+  }
+  if (best) return best;
+  return pool ? eligible[0] : null;
 }
 
 function isSkippable(text) {
@@ -146,7 +201,7 @@ function toEpisodes(records) {
   for (const r of records) {
     const text = (r.userText || '').trim();
     if (!cur || cur.text !== text) {
-      cur = { text, calls: 0, out: 0, mutating: 0, errors: 0, delegated: 0, models: new Set(), cwd: '' };
+      cur = { text, calls: 0, out: 0, mutating: 0, errors: 0, delegated: 0, models: new Set(), cwd: '', tools: {} };
       episodes.push(cur);
     }
     cur.calls += 1;
@@ -154,6 +209,9 @@ function toEpisodes(records) {
     cur.mutating += r.mutatingToolCalls || 0;
     cur.errors += r.toolErrors || 0;
     cur.delegated += r.delegationCalls || 0;
+    for (const [name, n] of Object.entries(r.toolCounts || {})) {
+      cur.tools[name] = (cur.tools[name] || 0) + n;
+    }
     cur.models.add(r.model);
     if (!cur.cwd && r.cwd) cur.cwd = r.cwd;
   }
@@ -244,17 +302,20 @@ export async function runRouteScan({ days = 14 } = {}) {
   for (const { ep, projectDir } of all) {
     if (isSkippable(ep.text)) continue;
     if (![...ep.models].some(isExpensiveModel)) continue; // already cheap
-    const cat = categorize(ep.text);
+    const cat = categorize(ep.text, ep.tools);
     if (cat) {
       // rule-health denominator: episodes that LOOK delegable by shape
       // (tier judged with the error signal zeroed — using real errors here
       // would be circular, since T2 requires errors=0 by definition). The
       // numerator is those that still hit errors: exactly the "light-looking
       // work in this category keeps failing" risk a delegation rule cares about.
+      // Keyed by tier as well: a category can carry both a T2 and a T1 rule,
+      // and sharing one category-wide stat would double-count every episode
+      // into both rules (identical ×N / err% on unrelated tiers).
       const shapeTier = tierOf({ ...ep, errors: 0 }, cat, thresholds);
       if (shapeTier === 'T1' || shapeTier === 'T2') {
-        bumpStats(`${cat.id}|${projectDir}`, ep);
-        bumpStats(`${cat.id}|*`, ep);
+        bumpStats(`${shapeTier}|${cat.id}|${projectDir}`, ep);
+        bumpStats(`${shapeTier}|${cat.id}|*`, ep);
       }
     }
     const tier = tierOf(ep, cat, thresholds);
@@ -299,8 +360,8 @@ export async function runRouteScan({ days = 14 } = {}) {
     (r.scope === 'global' || r.project === g.project));
 
   const ruleText = (g) => g.tier === 'T2'
-    ? `"${g.label}" 유형의 단순 요청(예: "${g.example}")은 ${g.agent}(haiku) 서브에이전트로 위임한다`
-    : `"${g.label}" 유형의 중간 난도 요청(예: "${g.example}")은 model: sonnet 서브에이전트로 위임한다 (설계 판단·반복 에러 발생 시 메인 모델이 이어받음)`;
+    ? `"${g.label}" 유형의 단순 요청(예: "${g.example}")은 ${g.agent}(haiku) 서브에이전트로 위임한다 (설계 판단·배포·스토어 제출 같은 비가역 작업이 섞이면 위임하지 않음)`
+    : `"${g.label}" 유형의 중간 난도 요청(예: "${g.example}")은 model: sonnet 서브에이전트로 위임한다 (설계 판단·비가역 작업·반복 에러 발생 시 메인 모델이 이어받음)`;
 
   const candidates = [...groups.values()]
     .filter((g) => g.count >= MIN_RECURRENCE && !hasRule(g))

package/src/session-records.js

@@ -79,14 +79,18 @@ export async function collectSessionRecords(filePath, { includeContent = true }
 
     let mutatingToolCalls = 0;
     let delegationCalls = 0;
+    const prev = records.get(reqId);
+    // Per-tool call histogram — what the call actually DID. route-scan's
+    // categorizer trusts this over the prompt's wording (behavior-first).
+    const toolCounts = { ...(prev?.toolCounts || {}) };
     if (Array.isArray(msg.content)) {
       for (const b of msg.content) {
         if (!b || b.type !== 'tool_use') continue;
         if (MUTATING_TOOLS.has(b.name)) mutatingToolCalls += 1;
         if (DELEGATION_TOOLS.has(b.name)) delegationCalls += 1;
+        if (typeof b.name === 'string') toolCounts[b.name] = (toolCounts[b.name] || 0) + 1;
       }
     }
-    const prev = records.get(reqId);
 
     lastRecord = {
       model: normalizeModelId(msg.model),
@@ -104,6 +108,7 @@ export async function collectSessionRecords(filePath, { includeContent = true }
       mutatingToolCalls: (prev?.mutatingToolCalls || 0) + mutatingToolCalls,
       delegationCalls: (prev?.delegationCalls || 0) + delegationCalls,
       toolErrors: prev?.toolErrors || 0,
+      toolCounts,
     };
     records.set(reqId, lastRecord);
   }