claude-token-saver 3.3.0 โ 3.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +12 -35
- package/README.md +12 -35
- package/bin/cli.js +29 -14
- package/package.json +1 -1
- package/src/model-rules.js +36 -9
- package/src/route-scan.js +110 -25
- package/src/session-records.js +6 -1
package/README.en.md
CHANGED
|
@@ -143,28 +143,16 @@ An auto `.bak` is kept, but **the session context that earned the rule its place
|
|
|
143
143
|
|
|
144
144
|
## ๐ route-scan โ "this recurring task could run on a cheaper tier"
|
|
145
145
|
|
|
146
|
-
|
|
146
|
+
Finds the easy work your expensive model (opus/fable) keeps redoing in your session logs and proposes **haiku/sonnet delegation rules**. Fully local, zero token cost.
|
|
147
147
|
|
|
148
|
-
- **T2 โ haiku**:
|
|
149
|
-
- **T1 โ sonnet**:
|
|
150
|
-
- **T0 stays**: repeated errors, heavy mutation,
|
|
148
|
+
- **T2 โ haiku**: lookups, pasted-screen Q&A, simple runs โ zero errors, near-zero mutation
|
|
149
|
+
- **T1 โ sonnet**: build pipelines, status checks โ few mutations, โค1 error
|
|
150
|
+
- **T0 stays**: repeated errors, heavy mutation, design/analysis โ the session model keeps it
|
|
151
151
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
A survey of existing LLM-routing research and systems shaped each axis of the design:
|
|
158
|
-
|
|
159
|
-
- **Conservative demotion โ "only send down what's clearly easy" โ is the unexplored direction.** In RouterArena (2025), every academic and commercial router fell far short of the oracle (90.9%; best entrant 66.9), and the failure mode was consistently **over-routing to expensive models**. Per the routing-collapse work, the oracle needs the top model for under 20% of queries โ while 73% of the author's episodes were still on the top tier. The headroom is large.
|
|
160
|
-
- **Difficulty is defined by outcome, not text guessing.** RouterArena labels difficulty as "how many of 42 models actually got it right". Our logs already contain outcomes: tool errors, mutating tool calls, episode length. Local measurement confirms it โ error incidence splits the tiers sharply (T2 candidates 2% / T1 8% / T0 36%).
|
|
161
|
-
- **Thresholds calibrate to the user's own distribution.** RouteLLM's stated limitation is that fixed thresholds drift as the query distribution shifts โ so output-token thresholds are recomputed from your last 14 days' p25/p75 (clamped).
|
|
162
|
-
- **An escalation path makes demotion mistakes cheap.** The FrugalGPT cascade lesson ("start cheap, escalate on failure"). In Claude Code this falls out naturally: when a subagent gets stuck, the main model takes over โ and promoted rules spell out that guard.
|
|
163
|
-
- **Subagent `model:` is Anthropic's official cost-control mechanism.** The docs recommend routing to cheaper models like Haiku for cost control โ yet Claude Code itself does no automatic model routing; even trivial turns resend the full context on the session model. That gap is exactly what route-scan fills.
|
|
164
|
-
- **Learned classifiers (BERT/embeddings) deliberately excluded.** Even the best is 24 points off the oracle, we have no preference-pair training data, and it would break the zero-dependency principle. Regex categories + outcome signals are the current position.
|
|
165
|
-
|
|
166
|
-
Sources (RouteLLM, FrugalGPT, RouterArena, routing-survey arxiv links), the full local-measurement table, and the complete tier criteria: [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md).
|
|
167
|
-
</details>
|
|
152
|
+
Three design pillars:
|
|
153
|
+
1. Difficulty is judged by **outcome, not text guessing** โ tool errors, mutating tool calls, output tokens
|
|
154
|
+
2. Thresholds **auto-calibrate to your own 14-day distribution** โ fixed constants drift with workload
|
|
155
|
+
3. Promoted rules live in a tool-owned file (`.claude/ratchet-model.md`) that **refreshes itself every scan**, and a `โ rule-health` flag fires when a delegated category's error rate climbs โ rules report their own staleness
|
|
168
156
|
|
|
169
157
|
```bash
|
|
170
158
|
claude-token-saver route-scan # scan (24h cache) + tiered candidates
|
|
@@ -173,21 +161,7 @@ claude-token-saver route-scan dismiss 1 # not interested โ won't resu
|
|
|
173
161
|
claude-token-saver route-scan rules # list model-fitting rules (rm <N> to remove)
|
|
174
162
|
```
|
|
175
163
|
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
Promoted delegation rules never mix with hand-written ratchet rules: they live in a **separate, fully tool-owned file** โ `.claude/ratchet-model.md` per project, `~/.claude/ratchet-model.md` for global scope โ regenerated wholesale on every scan, and they stay alive afterward:
|
|
179
|
-
|
|
180
|
-
- **Auto-refresh**: every rescan recomputes recurrence counts and the category's error rate from fresh logs and rewrites the file. Your `ratchet.md` is never touched by stat churn, so repos that commit `.claude/` see no diff noise (`ratchet-model.md` is safe to gitignore โ it's always regenerable from the registry).
|
|
181
|
-
- **rule-health**: when the delegated category's error rate exceeds 20%, the rule gets a `โ rule-health` flag suggesting you narrow or remove it โ the "define difficulty by outcome" principle applied to rule lifecycle.
|
|
182
|
-
- Your rules are managed by `harness list/rm`; model-fitting rules by `route-scan rules [rm <N>]` โ separate files, separate indexes.
|
|
183
|
-
- The CLAUDE.md ratchet section planted by `harness init` references both files, so Claude applies them together (existing users: re-run `harness init` to refresh the block).
|
|
184
|
-
|
|
185
|
-
How it works (session-boundary calibration, NOT a real-time router):
|
|
186
|
-
1. `install` registers a SessionStart hook that injects the cached scan results as session context on startup and `/clear`. Rescans are **data-triggered, not time-triggered**: ~5MB of new transcripts since the last scan rescans immediately, a small trickle rescans daily, and no change means no rescan at all (an unchanged-input scan is deterministic). A 1-hour minimum-interval guard applies, and promoting a rule triggers one immediate refresh to establish its stat baseline.
|
|
187
|
-
2. When a recurring (โฅ3ร) pattern exists, the statusline shows a `๐
ทโ route? R1` chip and Claude asks you whether to register it, and at which scope (`--project`/`--global`).
|
|
188
|
-
3. Promoted rules make **the main model delegate that work type to a haiku/sonnet subagent automatically from the next session on**.
|
|
189
|
-
|
|
190
|
-
Recommended companion setup: create `model: haiku` subagents under `~/.claude/agents/` (e.g. haiku-explore / haiku-runner / haiku-translate) plus a `model: sonnet` general worker so the rules are immediately actionable.
|
|
164
|
+
Dig deeper: **tier criteria & research evidence** โ [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) (Korean) ยท **rule-file mechanics, scan triggers, subagent setup** โ [docs/ROUTE_SCAN.md](./docs/ROUTE_SCAN.md) (Korean + English)
|
|
191
165
|
|
|
192
166
|
## Spike issue codes
|
|
193
167
|
|
|
@@ -269,6 +243,9 @@ Also update `statusLine.command` in `~/.claude/settings.json` to `claude-token-s
|
|
|
269
243
|
|
|
270
244
|
## Release notes
|
|
271
245
|
|
|
246
|
+
### v3.3.1 (2026-07-13)
|
|
247
|
+
- **Plain language instead of codes** โ bare `R1`/`T2` codes were unreadable for first-time users. Every user-facing listing (SessionStart hook briefing, `route-scan` candidates, `route-scan rules`) now spells tiers out as `T2 (simple โ haiku-class is enough)` and scopes as `this project only` / `all projects (global)`. The hook briefing also instructs the model to relay the spelled-out explanation, not the codes. (Statusline chips keep the terse `route? R1` for width; the session briefing carries the meaning.)
|
|
248
|
+
|
|
272
249
|
### v3.3.0 (2026-07-13)
|
|
273
250
|
- **Delegation visibility & briefing** โ the whole model-fitting loop is now surfaced to the user:
|
|
274
251
|
- The SessionStart hook injects, per candidate, **the exact rule text that would be written to ratchet-model.md** โ you see precisely what you're approving.
|
package/README.md
CHANGED
|
@@ -124,28 +124,16 @@ ratchet์ ๊ฐ์น๋ **ํ ๋ฐฉํฅ ๋์ **์ ์์ต๋๋ค. ๋ฃฐ์ ๊ฐ๋ณ๊ฒ ์ง
|
|
|
124
124
|
|
|
125
125
|
## ๐ route-scan โ "์ด ๋ฐ๋ณต ์์
, ๋ ์ผ ํฐ์ด๋ก ๋ด๋ ค๋ ๋ฉ๋๋ค"
|
|
126
126
|
|
|
127
|
-
์ธ์
|
|
127
|
+
์ธ์
๋ก๊ทธ์์ ์์ ๋ชจ๋ธ(opus/fable)์ด ๋ฐ๋ณต ์ฒ๋ฆฌํด ์จ ์ฌ์ด ์์
์ ์ฐพ์ **haiku/sonnet ์์ ๋ฃฐ๋ก ์น๊ฒฉ**์ ์ ์ํฉ๋๋ค. ์ ๊ณผ์ ๋ก์ปฌ, ํ ํฐ ๋น์ฉ 0.
|
|
128
128
|
|
|
129
|
-
- **T2 โ haiku**:
|
|
130
|
-
- **T1 โ sonnet**:
|
|
131
|
-
- **T0 ์ ์ง**: ์๋ฌ
|
|
129
|
+
- **T2 โ haiku**: ํ์ยท์กฐํยท๋จ์ ์คํ โ ์๋ฌ 0, ๋ณ๊ฒฝ ๊ฑฐ์ ์์
|
|
130
|
+
- **T1 โ sonnet**: ๋น๋ยท์ํ ์ ๊ฒ โ ๋ณ๊ฒฝ ์์, ์๋ฌ โค1
|
|
131
|
+
- **T0 ์ ์ง**: ์๋ฌ ๋ฐ๋ณตยท๋๋ ๋ณ๊ฒฝยท์ค๊ณ/๋ถ์ โ ์ธ์
๋ชจ๋ธ์ด ๊ณ์ ๋ด๋น
|
|
132
132
|
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
๊ธฐ์กด LLM ๋ผ์ฐํ
์ฐ๊ตฌยท์์คํ
์ ์กฐ์ฌํ ๊ฒฐ๊ณผ๊ฐ ์ค๊ณ์ ๊ฐ ์ถ์ ๊ฒฐ์ ํ์ต๋๋ค:
|
|
139
|
-
|
|
140
|
-
- **"์ฌ์ด ๊ฒ๋ง ํ์คํ ๋ด๋ฆฌ๋" ๋ณด์์ ๊ฐ๋ฑ์ด ๋ฏธ๊ฐ์ฒ ์ง๋** โ RouterArena(2025) ํ๊ฐ์์ ํ์ ยท์์ฉ ๋ผ์ฐํฐ ์ ๋ถ ์ค๋ผํด(90.9%)์ ํฌ๊ฒ ๋ชป ๋ฏธ์ณค๊ณ (1์ 66.9), ์คํจ ๋ฐฉํฅ์ ์ผ๊ด๋๊ฒ **๋น์ผ ๋ชจ๋ธ๋ก์ ๊ณผ์ ๋ผ์ฐํ
**์ด์์ต๋๋ค. ์ค๋ผํด ๊ธฐ์ค ์ต์์ ๋ชจ๋ธ์ด ์ ๋ง ํ์ํ ์ฟผ๋ฆฌ๋ 20% ๋ฏธ๋ง์ธ๋ฐ(routing collapse ์ฐ๊ตฌ), ์ ์ ๋ก๊ทธ์์ 73%๊ฐ ์ต์์ ํฐ์ด์ ๋จ์ ์์์ต๋๋ค โ ํค๋๋ฃธ์ด ํฝ๋๋ค.
|
|
141
|
-
- **๋์ด๋๋ ํ
์คํธ ์ถ์ ์ด ์๋๋ผ ๊ฒฐ๊ณผ(outcome)๋ก ์ ์** โ RouterArena๋ ๋์ด๋๋ฅผ "42๊ฐ ๋ชจ๋ธ ์ค ๋ช ๊ฐ๊ฐ ์ค์ ๋ก ๋งํ๋"๋ก ๋ผ๋ฒจ๋งํ์ต๋๋ค. ์ฐ๋ฆฌ ๋ก๊ทธ์๋ ๊ฒฐ๊ณผ๊ฐ ์ด๋ฏธ ์์ต๋๋ค: ๋๊ตฌ ์๋ฌยท๋ณ๊ฒฝ์ฑ ๋๊ตฌ ์ยท์ํผ์๋ ๊ธธ์ด. ๋ก์ปฌ ์ค์ธก์์๋ ์๋ฌ ๊ฒฝํ๋ฅ ์ด ํฐ์ด๋ฅผ ๊ฐํ๊ฒ ๊ฐ๋์ต๋๋ค (T2 ํ๋ณด 2% / T1 8% / T0 36%).
|
|
142
|
-
- **์๊ณ๊ฐ์ ์ฌ์ฉ์ ๋ถํฌ์์ ๋ณด์ ** โ RouteLLM์ ๊ณต์ ํ๊ณ๊ฐ "๊ณ ์ ์๊ณ๊ฐ์ ์ฟผ๋ฆฌ ๋ถํฌ๊ฐ ๋ฐ๋๋ฉด ์ด๊ธ๋๋ค"์
๋๋ค. ๊ทธ๋์ ์ถ๋ ฅ ํ ํฐ ์๊ณ๊ฐ์ ์ต๊ทผ 14์ผ p25/p75์์ ์ฌ๊ณ์ฐํฉ๋๋ค(ํด๋จํ ํฌํจ).
|
|
143
|
-
- **์คํจ ์ ์น๊ธ ๊ฒฝ๋ก๊ฐ ๊ฐ๋ฑ ์คํ ๋น์ฉ์ ๋ฎ์ถค** โ FrugalGPT ์บ์ค์ผ์ด๋("์ผ ๋ชจ๋ธ๋ถํฐ, ์ ๋๋ฉด ์๋ก")์ ๊ตํ. Claude Code์์ ์๋ธ์์ด์ ํธ๊ฐ ๋งํ๋ฉด ๋ฉ์ธ ๋ชจ๋ธ์ด ์ด์ด๋ฐ๋ ํํ๋ก ์์ฐ ๊ตฌํ๋๊ณ , ์น๊ฒฉ ๋ฃฐ ํ
์คํธ์๋ ์ด ๊ฐ๋๊ฐ ๋ช
์๋ฉ๋๋ค.
|
|
144
|
-
- **์๋ธ์์ด์ ํธ `model:`์ Anthropic ๊ณต์ ๋น์ฉ ํต์ ๋ฉ์ปค๋์ฆ** โ ๊ณต์ ๋ฌธ์๊ฐ "Haiku ๋ฑ ์ผ ๋ชจ๋ธ๋ก ๋ผ์ฐํ
ํด ๋น์ฉ ํต์ "๋ฅผ ๊ถ์ฅํ์ง๋ง, Claude Code ๋ณธ์ฒด๋ ์๋ ๋ชจ๋ธ ๋ผ์ฐํ
์ ํ์ง ์์ต๋๋ค. ์ฌ์ํ ํด๋ ์ธ์
๋ชจ๋ธ์ด ์ ์ฒด ์ปจํ
์คํธ๋ฅผ ์ฌ์ ์กํ๋ฉฐ ์ฒ๋ฆฌํ๋ ์ด ๊ณต๋ฐฑ์ด route-scan์ด ๋ฉ์ฐ๋ ์๋ฆฌ์
๋๋ค.
|
|
145
|
-
- **ํ์ตํ ๋ถ๋ฅ๊ธฐ(BERT/์๋ฒ ๋ฉ)๋ ์๋์ ์ผ๋ก ๋ฐฐ์ ** โ ์ต๊ณ ์ฑ๋ฅ๋ ์ค๋ผํด๊ณผ 24์ ์ฐจ์ด์ธ ๋ฐ๋ค ํ์ต ๋ฐ์ดํฐ๊ฐ ์๊ณ zero-dep ์์น์ ์ด๊ธ๋ฉ๋๋ค. ์ ๊ท์ ์นดํ
๊ณ ๋ฆฌ + outcome ์ ํธ ์กฐํฉ์ด ํ์ฌ ์์น.
|
|
146
|
-
|
|
147
|
-
์ถ์ฒ(RouteLLMยทFrugalGPTยทRouterArenaยท๋ผ์ฐํ
์๋ฒ ์ด arxiv ๋งํฌ)์ ๋ก์ปฌ ์ค์ธก ์์ธ ํ
์ด๋ธ, ํฐ์ด ๊ธฐ์ค ์ ๋ฌธ์ [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) ์ฐธ๊ณ .
|
|
148
|
-
</details>
|
|
133
|
+
ํต์ฌ ์ค๊ณ๋ ์ธ ๊ฐ์ง์
๋๋ค:
|
|
134
|
+
1. ๋์ด๋๋ ํ
์คํธ ์ถ์ธก์ด ์๋๋ผ **๊ฒฐ๊ณผ(outcome)๋ก ํ์ ** โ ๋๊ตฌ ์๋ฌยท๋ณ๊ฒฝ์ฑ ๋๊ตฌ ์ยท์ถ๋ ฅ ํ ํฐ
|
|
135
|
+
2. ์๊ณ๊ฐ์ **๋ด ์ต๊ทผ 14์ผ ๋ก๊ทธ ๋ถํฌ์์ ์๋ ๋ณด์ ** โ ๊ณ ์ ์์๋ ์ํฌ๋ก๋๊ฐ ๋ฐ๋๋ฉด ์ด๊ธ๋๋ฏ๋ก
|
|
136
|
+
3. ์น๊ฒฉ๋ ๋ฃฐ์ ๋๊ตฌ ์์ ๋ณ๋ ํ์ผ(`.claude/ratchet-model.md`)์์ **์๋ ๊ฐฑ์ **๋๊ณ , ์์ ํ ์๋ฌ์จ์ด ๋์์ง๋ฉด `โ rule-health`๋ก ๊ฒฝ๊ณ โ ๋ฃฐ์ด ์ค์ค๋ก ๋ก์์ ์๋ฆผ
|
|
149
137
|
|
|
150
138
|
```bash
|
|
151
139
|
claude-token-saver route-scan # ์ค์บ (24h ์บ์) + ํฐ์ด๋ณ ํ๋ณด ์ถ๋ ฅ
|
|
@@ -154,21 +142,7 @@ claude-token-saver route-scan dismiss 1 # ๊ด์ฌ ์์ผ๋ฉด ๋ฌด์ (์ฌ
|
|
|
154
142
|
claude-token-saver route-scan rules # ๋ฑ๋ก๋ ๋ชจ๋ธ ํผํ
๋ฃฐ ๋ชฉ๋ก (rm <N>์ผ๋ก ์ ๊ฑฐ)
|
|
155
143
|
```
|
|
156
144
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
์น๊ฒฉ๋ ์์ ๋ฃฐ์ ์์ผ๋ก ์ด ๋ซ์ณ ๋ฃฐ๊ณผ ์์ด์ง ์๋๋ก **๋ณ๋ ํ์ผ**์ ์ ์ฅ๋ฉ๋๋ค โ ํ๋ก์ ํธ๋ `.claude/ratchet-model.md`, ๊ธ๋ก๋ฒ์ `~/.claude/ratchet-model.md`. ์ด ํ์ผ์ ์ ์ ์ผ๋ก ๋๊ตฌ ์์ ๋ผ ๋งค ์ค์บ๋ง๋ค ํต์งธ๋ก ์ฌ์์ฑ๋๋ฉฐ, ์ดํ์๋ ์ด์ ์์ง์
๋๋ค:
|
|
160
|
-
|
|
161
|
-
- **์๋ ๊ฐฑ์ **: ๋งค ์ค์บ๋ง๋ค ๋ฐ๋ณต ํ์ยทํด๋น ์ ํ์ ์๋ฌ์จ์ ์ต์ ๋ก๊ทธ๋ก ๋ค์ ๊ณ์ฐํด ํ์ผ์ ์ฌ์์ฑํฉ๋๋ค. ํต๊ณ๊ฐ ๋ฐ๋์ด๋ ์ฌ์ฉ์์ `ratchet.md`๋ ์ ํ ๊ฑด๋๋ฆฌ์ง ์์ผ๋ฏ๋ก, `.claude/`๋ฅผ ์ปค๋ฐํ๋ ํ๋ก์ ํธ์์๋ diff ์์์ด ์์ต๋๋ค (`ratchet-model.md`๋ gitignoreํด๋ ๋ฌด๋ฐฉ โ ๋ ์ง์คํธ๋ฆฌ์์ ํญ์ ์ฌ์์ฑ ๊ฐ๋ฅ).
|
|
162
|
-
- **rule-health**: ์์ ๋์ ์ ํ์ ์๋ฌ์จ์ด 20%๋ฅผ ๋์ผ๋ฉด ๋ฃฐ์ `โ rule-health` ๊ฒฝ๊ณ ๊ฐ ๋ถ์ด ์กฐ๊ฑด์ ์ขํ๊ฑฐ๋ ์ ๊ฑฐํ๋ผ๊ณ ์๋ฆฝ๋๋ค โ "๊ฒฐ๊ณผ(outcome)๋ก ๋์ด๋๋ฅผ ์ ์"ํ๋ ์์น์ ๋ฃฐ ์๋ช
๊ด๋ฆฌ์ ์ฌ์ ์ฉํ ๊ฒ.
|
|
163
|
-
- ์ฌ์ฉ์ ๋ฃฐ์ `harness list/rm`, ๋ชจ๋ธ ํผํ
๋ฃฐ์ `route-scan rules [rm <N>]`๋ก ๊ฐ๊ฐ ๊ด๋ฆฌ โ ์๋ก์ ํ์ผ๋ ์ธ๋ฑ์ค๋ ์นจ๋ฒํ์ง ์์ต๋๋ค.
|
|
164
|
-
- `harness init`์ด ์ฌ๋ CLAUDE.md ๋ซ์ณ ์น์
์ด ๋ ํ์ผ์ ๋ชจ๋ ์ฐธ์กฐํ๋ฏ๋ก Claude๊ฐ ์ธ์
์์ ํจ๊ป ์ ์ฉํฉ๋๋ค (๊ธฐ์กด ์ฌ์ฉ์๋ `harness init` ์ฌ์คํ์ผ๋ก ๋ธ๋ก ๊ฐฑ์ ).
|
|
165
|
-
|
|
166
|
-
๋์ ๊ตฌ์กฐ (์ค์๊ฐ ๋ผ์ฐํ
์ด ์๋๋ผ **์ธ์
๊ฒฝ๊ณ ์บ๋ฆฌ๋ธ๋ ์ด์
**):
|
|
167
|
-
1. `install` ์ SessionStart ํ
์ด ๋ฑ๋ก๋์ด, ์ ์ธ์
์์ยท`/clear` ๋ ์บ์๋ ์ค์บ ๊ฒฐ๊ณผ๋ฅผ ์ธ์
์ปจํ
์คํธ๋ก ์ฃผ์
ํฉ๋๋ค. ์ฌ์ค์บ์ ์๊ฐ์ด ์๋๋ผ **๋ฐ์ดํฐ๊ฐ ํธ๋ฆฌ๊ฑฐ**: ๋ง์ง๋ง ์ค์บ ์ดํ ์ transcript๊ฐ ~5MB ์ด์ ์์ด๋ฉด ์ฆ์, ์๋์ด๋ฉด ํ๋ฃจ 1ํ, ์๋ฌด ๋ณํ๊ฐ ์์ผ๋ฉด ์์ ๋์ง ์์ต๋๋ค (๋ณํ ์๋ ์ฌ์ค์บ์ ๊ฒฐ๊ณผ๊ฐ ๋์ผํ๋ฏ๋ก). ์ต์ ๊ฐ๊ฒฉ 1์๊ฐ ๊ฐ๋ ํฌํจ, ๋ฃฐ ๋ฑ๋ก(promote) ์งํ์๋ ํต๊ณ ๊ธฐ์ค์ ํ๋ณด๋ฅผ ์ํด ์ฆ์ 1ํ.
|
|
168
|
-
2. ๋ฐ๋ณต(โฅ3ํ) ํจํด์ด ์์ผ๋ฉด statusline์ `๐
ทโ route? R1` ์นฉ์ด ๋จ๊ณ , Claude๊ฐ ๋ฑ๋ก ์ฌ๋ถ์ scope(`--project`/`--global`)๋ฅผ ๋ฌผ์ด๋ด
๋๋ค.
|
|
169
|
-
3. ๋ฑ๋ก๋ ๋ฃฐ์ **๋ค์ ์ธ์
๋ถํฐ ๋ฉ์ธ ๋ชจ๋ธ์ด ํด๋น ์ ํ์ haiku/sonnet ์๋ธ์์ด์ ํธ๋ก ์๋ ์์**ํ๊ฒ ํฉ๋๋ค.
|
|
170
|
-
|
|
171
|
-
๊ถ์ฅ ์ฌ์ ์ค๋น: `~/.claude/agents/`์ `model: haiku` ์๋ธ์์ด์ ํธ(์: haiku-explore / haiku-runner / haiku-translate)์ `model: sonnet` ๋ฒ์ฉ ์๋ธ์์ด์ ํธ๋ฅผ ๋ง๋ค์ด ๋๋ฉด ๋ฃฐ์ด ๋ฐ๋ก ์คํ ๊ฐ๋ฅํด์ง๋๋ค.
|
|
145
|
+
๋ ์์๋ณด๊ธฐ: **ํฐ์ด ๊ธฐ์คยท๋ฆฌ์์น ๊ทผ๊ฑฐ** โ [docs/TIER_CRITERIA.md](./docs/TIER_CRITERIA.md) ยท **๋ฃฐ ํ์ผ ๊ตฌ์กฐยท์ค์บ ํธ๋ฆฌ๊ฑฐยท์๋ธ์์ด์ ํธ ์ค๋น** โ [docs/ROUTE_SCAN.md](./docs/ROUTE_SCAN.md)
|
|
172
146
|
|
|
173
147
|
## ํ ํฐ ๊ธ์ฆ ์์ธ ์ฝ๋
|
|
174
148
|
|
|
@@ -226,6 +200,9 @@ npm uninstall -g claude-cache-monitor && npm i -g claude-token-saver
|
|
|
226
200
|
|
|
227
201
|
## ๋ฆด๋ฆฌ์ค ๋
ธํธ
|
|
228
202
|
|
|
203
|
+
### v3.3.1 (2026-07-13)
|
|
204
|
+
- **์ฝ๋ ๋์ ํ์ด์ด ์ค๋ช
** โ `R1`, `T2` ๊ฐ์ ์ฝ๋๊ฐ ์ค๋ช
์์ด ๋
ธ์ถ๋ผ ์ฒ์ ์ฐ๋ ์ฌ๋์ด ์ ์ ์๋ ๋ฌธ์ ์์ . ๋ชจ๋ ์ฌ์ฉ์ ๋๋ฉด ์ถ๋ ฅ(SessionStart ํ
๋ธ๋ฆฌํ, `route-scan` ํ๋ณด ๋ชฉ๋ก, `route-scan rules` ๋ชฉ๋ก)์์ ํฐ์ด๋ฅผ `T2 (๋จ์ ์์
โ haiku๊ธ์ด๋ฉด ์ถฉ๋ถ)` ์์ผ๋ก ํ์ด์ฐ๊ณ , scope๋ `์ด ํ๋ก์ ํธ๋ง`/`๋ชจ๋ ํ๋ก์ ํธ(๊ธ๋ก๋ฒ)`๋ก ํ๊ธฐ. ํ
๋ธ๋ฆฌํ์๋ "์ฌ์ฉ์์๊ฒ ์ ๋ฌํ ๋ ์ฝ๋๊ฐ ์๋๋ผ ํ์ด์ด ์ค๋ช
์ผ๋ก ๋ธ๋ฆฌํํ๋ผ"๋ ์ง์ ํฌํจ. (statusline ์นฉ์ ํญ ์ ์ฝ์ `route? R1` ์ ์ง โ ์๋ฏธ๋ ์ธ์
๋ธ๋ฆฌํ์ด ์ค๋ช
)
|
|
205
|
+
|
|
229
206
|
### v3.3.0 (2026-07-13)
|
|
230
207
|
- **์์ ๊ฐ์ํยท๋ธ๋ฆฌํ ๊ฐํ** โ ๋ชจ๋ธ ํผํ
์ ์ ๊ณผ์ ์ด ์ฌ์ฉ์์๊ฒ ๋ณด์ด๋๋ก:
|
|
231
208
|
- SessionStart ํ
์ด ํ๋ณด๋ง๋ค **๋ฑ๋ก ์ ratchet-model.md์ ๊ธฐ๋ก๋ ๋ฃฐ ์๋ฌธ**์ ํจ๊ป ์ฃผ์
โ ๋ฌด์์ด ๋ฑ๋ก๋ ์ง ์ ํํ ๋ณด๊ณ ์น์ธ.
|
package/bin/cli.js
CHANGED
|
@@ -520,8 +520,13 @@ async function main() {
|
|
|
520
520
|
}
|
|
521
521
|
console.log(lang === 'ko' ? '๐ ๋ชจ๋ธ ํผํ
๋ฃฐ (๋ก๊ทธ ๊ธฐ๋ฐ ์๋ ๊ฐฑ์ ):' : '๐ Model-fitting rules (auto-refreshed from logs):');
|
|
522
522
|
rules.forEach((r, i) => {
|
|
523
|
-
const health = r.status === 'review'
|
|
524
|
-
|
|
523
|
+
const health = r.status === 'review'
|
|
524
|
+
? (lang === 'ko' ? ' โ ์๋ฌ์จ ์ด๊ณผ โ ์ฌ๊ฒํ ํ์' : ' โ error rate over threshold โ needs review')
|
|
525
|
+
: '';
|
|
526
|
+
const stat = lang === 'ko'
|
|
527
|
+
? `${r.tier} (${rs.tierLabel(r.tier)}) ยท ${rs.scopeLabel(r.scope)} ยท ๋ฐ๋ณต ${r.count || 0}ํ ยท ์๋ฌ์จ ${Math.round((r.errRate || 0) * 100)}%`
|
|
528
|
+
: `${r.tier} (${rs.tierLabel(r.tier, 'en')}) ยท ${rs.scopeLabel(r.scope, 'en')} ยท seen ร${r.count || 0} ยท err ${Math.round((r.errRate || 0) * 100)}%`;
|
|
529
|
+
console.log(` #${i + 1} ${stat}${health}`);
|
|
525
530
|
console.log(` ${r.rule}`);
|
|
526
531
|
});
|
|
527
532
|
console.log('\n์ ๊ฑฐ: claude-token-saver route-scan rules rm <N>');
|
|
@@ -571,21 +576,24 @@ async function main() {
|
|
|
571
576
|
if (open.length === 0 && reviewRules.length === 0) return; // silent โ nothing to inject
|
|
572
577
|
const lines = [];
|
|
573
578
|
if (open.length > 0) {
|
|
574
|
-
lines.push(`[claude-token-saver route-scan] ์ต๊ทผ ${cache.days}์ผ ์ธ์
์์
|
|
579
|
+
lines.push(`[claude-token-saver route-scan] ์ต๊ทผ ${cache.days}์ผ ์ธ์
์์ ๋น์ผ ๋ชจ๋ธ(opus/fable)์ด ๋ฐ๋ณต ์ฒ๋ฆฌํด ์จ, ๋ ์ผ ๋ชจ๋ธ๋ก ๋๊ฒจ๋ ๋๋ ์์
์ด ๊ฐ์ง๋์์ต๋๋ค.`);
|
|
580
|
+
lines.push('(R<N>์ ํ๋ณด ๋ฒํธ, T2/T1์ ๋์ด๋ ๋ฑ๊ธ์
๋๋ค โ ์ฌ์ฉ์์๊ฒ ์ ๋ฌํ ๋๋ ์ฝ๋๊ฐ ์๋๋ผ ์๋ ํ์ด์ด ์ค๋ช
์ผ๋ก ๋ธ๋ฆฌํํ์ธ์)');
|
|
575
581
|
for (const c of open) {
|
|
576
|
-
const
|
|
577
|
-
lines.push(` R${c.id}
|
|
578
|
-
lines.push(`
|
|
582
|
+
const tier = c.tier || 'T2';
|
|
583
|
+
lines.push(` ํ๋ณด R${c.id} โ "${c.label}" ์ ํ, ${c.count}ํ ๋ฐ๋ณต (ํ๋ก์ ํธ: ${c.project})`);
|
|
584
|
+
lines.push(` ํ์ : ${tier} (${rs.tierLabel(tier)}) โ ${c.agent} ์๋ธ์์ด์ ํธ ์์ ๊ถ์ฅ ยท ์ ์ฉ ๋ฒ์ ์ ์: ${rs.scopeLabel(c.suggestedScope)}`);
|
|
585
|
+
lines.push(` ์์ ์์ฒญ: "${c.example}"`);
|
|
579
586
|
lines.push(` ๋ฑ๋ก ์ ratchet-model.md์ ๊ธฐ๋ก๋ ๋ฃฐ: "${c.rule}"`);
|
|
580
587
|
}
|
|
581
|
-
lines.push('๋ฑ๋กํ๋ฉด ๋ค์ ์ธ์
๋ถํฐ ์๋ ์์๋ฉ๋๋ค. ์ฌ์ฉ์์๊ฒ ๋ฑ๋ก ์ฌ๋ถ๋ฅผ ๋ฌผ์ ๋ ์ ๋ฃฐ ์๋ฌธ์ ๊ทธ๋๋ก ๋ณด์ฌ์ฃผ๊ณ ,
|
|
582
|
-
lines.push(' claude-token-saver harness promote R<N> --project|--global #
|
|
588
|
+
lines.push('๋ฑ๋กํ๋ฉด ๋ค์ ์ธ์
๋ถํฐ ์๋ ์์๋ฉ๋๋ค. ์ฌ์ฉ์์๊ฒ ๋ฑ๋ก ์ฌ๋ถ๋ฅผ ๋ฌผ์ ๋ ์ ๋ฃฐ ์๋ฌธ์ ๊ทธ๋๋ก ๋ณด์ฌ์ฃผ๊ณ , ์ ์ฉ ๋ฒ์๊น์ง ํ์ธํ ๋ค ์คํํ์ธ์:');
|
|
589
|
+
lines.push(' claude-token-saver harness promote R<N> --project|--global # ์ ์ฉ ๋ฒ์๋ ๋ฐ๋์ ์ฌ์ฉ์์๊ฒ ํ์ธ');
|
|
583
590
|
lines.push(' claude-token-saver route-scan dismiss <N> # ์ฌ์ฉ์๊ฐ ์์น ์์ผ๋ฉด');
|
|
584
591
|
}
|
|
585
592
|
if (reviewRules.length > 0) {
|
|
586
|
-
lines.push(`[claude-token-saver rule-health]
|
|
593
|
+
lines.push(`[claude-token-saver rule-health] ์ฌ์ฉ์๊ฐ ์น์ธํ ์์ ๋ฃฐ ์ค, ์์ ๋์ ์ ํ์ ์ต๊ทผ ์๋ฌ์จ์ด ๊ธฐ์ค(20%)์ ๋์ด ์ฌ๊ฒํ ๊ฐ ํ์ํ ๋ฃฐ์ด ์์ต๋๋ค โ ์ฌ์ฉ์์๊ฒ ๋ธ๋ฆฌํํ๊ณ ์กฐ๊ฑด ์ขํ๊ธฐ/์ ๊ฑฐ๋ฅผ ์์ํ์ธ์:`);
|
|
587
594
|
for (const r of reviewRules) {
|
|
588
|
-
lines.push(` #${r.n}
|
|
595
|
+
lines.push(` ๋ฃฐ #${r.n} (${r.tier} ${rs.tierLabel(r.tier)} ยท ${rs.scopeLabel(r.scope)}) โ ์ต๊ทผ ์๋ฌ์จ ${Math.round((r.errRate || 0) * 100)}%`);
|
|
596
|
+
lines.push(` "${r.rule}"`);
|
|
589
597
|
}
|
|
590
598
|
lines.push(' ์ ๊ฑฐ: claude-token-saver route-scan rules rm <N>');
|
|
591
599
|
}
|
|
@@ -614,11 +622,18 @@ async function main() {
|
|
|
614
622
|
: 'No delegation candidates (below recurrence threshold or already resolved).');
|
|
615
623
|
return;
|
|
616
624
|
}
|
|
617
|
-
console.log(lang === 'ko' ? '\n์์
|
|
625
|
+
console.log(lang === 'ko' ? '\n์์ ํ๋ณด (R<N>=ํ๋ณด ๋ฒํธ, T2/T1=๋์ด๋ ๋ฑ๊ธ):' : '\nDelegation candidates (R<N> = candidate id, T2/T1 = difficulty tier):');
|
|
618
626
|
for (const c of open) {
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
627
|
+
const tier = c.tier || 'T2';
|
|
628
|
+
if (lang === 'ko') {
|
|
629
|
+
console.log(` R${c.id} "${c.label}" ร${c.count}ํ [${c.project}]`);
|
|
630
|
+
console.log(` ํ์ : ${tier} (${rs.tierLabel(tier)}) โ ${c.agent} ์์ ๊ถ์ฅ ยท ์ ์ฉ ๋ฒ์ ์ ์: ${rs.scopeLabel(c.suggestedScope)}`);
|
|
631
|
+
} else {
|
|
632
|
+
console.log(` R${c.id} "${c.label}" ร${c.count} [${c.project}]`);
|
|
633
|
+
console.log(` verdict: ${tier} (${rs.tierLabel(tier, 'en')}) โ delegate to ${c.agent} ยท suggested scope: ${rs.scopeLabel(c.suggestedScope, 'en')}`);
|
|
634
|
+
}
|
|
635
|
+
console.log(` ${lang === 'ko' ? '์์' : 'example'}: "${c.example}"`);
|
|
636
|
+
console.log(` ${lang === 'ko' ? '๋ฃฐ' : 'rule'}: ${c.rule}`);
|
|
622
637
|
}
|
|
623
638
|
console.log('');
|
|
624
639
|
console.log(lang === 'ko' ? '๋ฑ๋ก / ๋ฌด์:' : 'Promote / dismiss:');
|
package/package.json
CHANGED
package/src/model-rules.js
CHANGED
|
@@ -104,12 +104,37 @@ export function renderModelRatchet(rules) {
|
|
|
104
104
|
'## Rules',
|
|
105
105
|
'',
|
|
106
106
|
];
|
|
107
|
+
const healthOf = (r) => r.status === 'review'
|
|
108
|
+
? ` โ rule-health: ์ต๊ทผ ์์ ๋์ ์๋ฌ์จ ${Math.round((r.errRate || 0) * 100)}% โ ์กฐ๊ฑด์ ์ขํ๊ฑฐ๋ ์ ๊ฑฐ ๊ฒํ `
|
|
109
|
+
: '';
|
|
110
|
+
const statsOf = (r) => `ร${r.count || 0}, err ${Math.round((r.errRate || 0) * 100)}%, seen ${r.lastSeen || r.promotedAt}`;
|
|
111
|
+
|
|
112
|
+
// A category can carry both a T2 (haiku) and a T1 (sonnet) rule. Tier is
|
|
113
|
+
// only known after an episode finishes, so two separate bullets give the
|
|
114
|
+
// reading LLM no way to pick one at request time โ merge such pairs into a
|
|
115
|
+
// single conditional rule with explicit request-time criteria (default to
|
|
116
|
+
// the cheap agent, escalate on multi-step scope, hand back on judgement /
|
|
117
|
+
// irreversible actions).
|
|
118
|
+
const byCategory = new Map();
|
|
107
119
|
for (const r of rules) {
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
120
|
+
if (!byCategory.has(r.category)) byCategory.set(r.category, []);
|
|
121
|
+
byCategory.get(r.category).push(r);
|
|
122
|
+
}
|
|
123
|
+
for (const group of byCategory.values()) {
|
|
124
|
+
const t2 = group.find((r) => r.tier === 'T2');
|
|
125
|
+
const t1 = group.find((r) => r.tier === 'T1');
|
|
126
|
+
if (t2 && t1) {
|
|
127
|
+
const rule =
|
|
128
|
+
`"${t2.label}" ์ ํ ์์ฒญ์ ๊ธฐ๋ณธ์ ์ผ๋ก ${t2.agent}(haiku) ์๋ธ์์ด์ ํธ๋ก ์์ํ๋ค(์: "${t2.example}"). ` +
|
|
129
|
+
`์ฌ๋ฌ ๋จ๊ณยท์ฌ๋ฌ ํ์ผ ์์ ์ด ์ฝํ ์ค๊ฐ ๋๋ ์์ฒญ(์: "${t1.example}")์ model: sonnet ์๋ธ์์ด์ ํธ๋ก ์์ํ๋ค. ` +
|
|
130
|
+
`์ค๊ณ ํ๋จยท๋ฐฐํฌยท์คํ ์ด ์ ์ถ ๊ฐ์ ๋น๊ฐ์ญ ์์
์ด ์์ด๊ฑฐ๋ ์์ ์ค ์๋ฌ๊ฐ ๋ฐ๋ณต๋๋ฉด ์์ํ์ง ๋ง๊ณ ๋ฉ์ธ ๋ชจ๋ธ์ด ์ง์ ์ฒ๋ฆฌํ๋ค`;
|
|
131
|
+
lines.push(`- ${rule}${healthOf(t2)}${healthOf(t1)} <!-- T2 ${statsOf(t2)} / T1 ${statsOf(t1)} -->`);
|
|
132
|
+
for (const r of group) {
|
|
133
|
+
if (r !== t2 && r !== t1) lines.push(`- ${r.rule}${healthOf(r)} <!-- ${statsOf(r)} -->`);
|
|
134
|
+
}
|
|
135
|
+
} else {
|
|
136
|
+
for (const r of group) lines.push(`- ${r.rule}${healthOf(r)} <!-- ${statsOf(r)} -->`);
|
|
137
|
+
}
|
|
113
138
|
}
|
|
114
139
|
return lines.join('\n') + '\n';
|
|
115
140
|
}
|
|
@@ -171,8 +196,10 @@ export function syncAllFiles({ previousPaths = [] } = {}) {
|
|
|
171
196
|
* for each registered rule, recompute recurrence count and the error rate
|
|
172
197
|
* of episodes in its (tier-eligible) category โ the rule-health signal.
|
|
173
198
|
*
|
|
174
|
-
* `episodeStats`: Map "category|project" (plus a "category|*"
|
|
175
|
-
* that global-scope rules fall back to) โ { count, errCount,
|
|
199
|
+
* `episodeStats`: Map "tier|category|project" (plus a "tier|category|*"
|
|
200
|
+
* wildcard key that global-scope rules fall back to) โ { count, errCount,
|
|
201
|
+
* epCount }. Tier is part of the key so a category carrying both a T2 and a
|
|
202
|
+
* T1 rule doesn't double-count every episode into both rules' stats.
|
|
176
203
|
* errCount/epCount measure the scan window's shape-eligible episodes โ ones
|
|
177
204
|
* an expensive model handled directly that still look T1/T2 by shape (tier
|
|
178
205
|
* judged with the error signal zeroed; see route-scan's rule-health pass).
|
|
@@ -181,8 +208,8 @@ export function refreshModelRules(episodeStats, { now } = {}) {
|
|
|
181
208
|
const data = loadModelRules();
|
|
182
209
|
let changed = false;
|
|
183
210
|
for (const r of data.rules) {
|
|
184
|
-
const s = episodeStats.get(`${r.category}|${r.project}`)
|
|
185
|
-
|| (r.scope === 'global' ? episodeStats.get(`${r.category}|*`) : null);
|
|
211
|
+
const s = episodeStats.get(`${r.tier}|${r.category}|${r.project}`)
|
|
212
|
+
|| (r.scope === 'global' ? episodeStats.get(`${r.tier}|${r.category}|*`) : null);
|
|
186
213
|
if (!s) continue;
|
|
187
214
|
r.count = s.count;
|
|
188
215
|
r.errRate = s.epCount > 0 ? s.errCount / s.epCount : 0;
|
package/src/route-scan.js
CHANGED
|
@@ -42,7 +42,10 @@ export const T0_MIN_MUTATING = 7;
|
|
|
42
42
|
// acks, feedback) โ skip entirely.
|
|
43
43
|
export const MIN_DELEGABLE_OUT = 100;
|
|
44
44
|
// Escalation keywords: design/analysis judgement stays on the top tier.
|
|
45
|
-
|
|
45
|
+
// Irreversible/external actions (store submission, deploy, release, merge)
|
|
46
|
+
// are included โ they may look like light "run" episodes in the logs, but
|
|
47
|
+
// delegating them defeats the harness's default-safe-path rule.
|
|
48
|
+
export const ESCALATE_RE = /์ค๊ณ|์ํคํ
์ฒ|๋ฆฌํฉํ ๋ง|์์ธ ๋ถ์|๊ฐ์ ํ |๊ฒํ ํด๋ณด|๋น๊ต|์ |์ ์ถ|๋ฐฐํฌ|์ถ์|analyze|compare|evaluate|architect|refactor|submit|deploy|release|publish|merge/i;
|
|
46
49
|
// A pattern must recur this often before we nag about it.
|
|
47
50
|
export const MIN_RECURRENCE = 3;
|
|
48
51
|
|
|
@@ -58,50 +61,85 @@ export const RESCAN_MIN_INTERVAL_MS = 60 * 60 * 1000; // never more than ho
|
|
|
58
61
|
export const RESCAN_BIG_DELTA_BYTES = 5 * 1024 * 1024; // this much new data โ rescan now
|
|
59
62
|
export const RESCAN_MAX_AGE_MS = 24 * 60 * 60 * 1000; // any new data + a day old โ rescan
|
|
60
63
|
|
|
61
|
-
//
|
|
62
|
-
//
|
|
63
|
-
// otherwise mislabel the episode; translate before read: "๋ฒ์ญํด์ค" also
|
|
64
|
-
// matches the read keywords).
|
|
64
|
+
// Categories โ recommended subagent. Classification is behavior-first with
|
|
65
|
+
// weighted keyword scoring as fallback โ see categorize().
|
|
65
66
|
const PASTE_MIN_LEN = 400;
|
|
66
67
|
const CATEGORIES = [
|
|
67
68
|
{
|
|
68
69
|
id: 'paste',
|
|
69
70
|
label: '๋ถ์ฌ๋ฃ์ ํ๋ฉดยท๋ก๊ทธ ์ง๋ฌธ',
|
|
70
71
|
agent: 'haiku-explore',
|
|
71
|
-
|
|
72
|
+
kw: null, // matched by length, see categorize()
|
|
72
73
|
},
|
|
73
74
|
{
|
|
74
75
|
id: 'translate',
|
|
75
76
|
label: '๋ฐฐ์น ๋ฒ์ญยท์ ํ ํ
์คํธ ๋ณํ',
|
|
76
77
|
agent: 'haiku-translate',
|
|
77
|
-
|
|
78
|
+
kw: [[/๋ฒ์ญ|translate/i, 2], [/๋ณํํด|ํ๋ก ์ ๋ฆฌ|ํฌ๋งทํ
/i, 1]],
|
|
78
79
|
},
|
|
79
80
|
{
|
|
80
81
|
id: 'explore',
|
|
81
82
|
label: 'ํ์ยท์กฐํ (ํ์ผ/๊ฐ ์ฐพ๊ธฐ)',
|
|
82
83
|
agent: 'haiku-explore',
|
|
83
|
-
|
|
84
|
+
kw: [[/grep|๊ฒ์|search|find/i, 2], [/์ฐพ์|์ด๋|์์น|๋ชฉ๋ก|์ดํด/i, 1]],
|
|
84
85
|
},
|
|
85
86
|
{
|
|
86
87
|
id: 'read',
|
|
87
88
|
label: '์ฝ๊ธฐยท์์ฝยท์ค๋ช
',
|
|
88
89
|
agent: 'haiku-explore',
|
|
89
|
-
|
|
90
|
-
},
|
|
91
|
-
{
|
|
92
|
-
id: 'run',
|
|
93
|
-
label: '๋ช
๋ น ์คํ (๋น๋ยทํ
์คํธยทgit)',
|
|
94
|
-
agent: 'haiku-runner',
|
|
95
|
-
re: /git |commit|push|์คํ|๋๋ ค|run |build|๋น๋|ํ
์คํธ|npm |pip|์ค์น/i,
|
|
90
|
+
kw: [[/์์ฝ|summar|explain/i, 2], [/์ฝ์ด|์ค๋ช
|์ ๋ฆฌํด|๋ณด์ฌ์ค|์๋ ค์ค|๋ญ์ผ|what/i, 1]],
|
|
96
91
|
},
|
|
97
92
|
{
|
|
98
93
|
id: 'check',
|
|
99
94
|
label: '์ํ ํ์ธยท๊ฒ์ฆ',
|
|
100
95
|
agent: 'haiku-explore',
|
|
101
|
-
|
|
96
|
+
kw: [[/ํ์ธ|๊ฒ์ฆ|verify|์ ๊ฒ/i, 2], [/๋ง์\?|๋๋|๋๋|๋์ด|๋๋์ง|๊ด์ฐฎ|์ฒดํฌ|check|status/i, 1]],
|
|
97
|
+
},
|
|
98
|
+
{
|
|
99
|
+
id: 'run',
|
|
100
|
+
label: '๋ช
๋ น ์คํ (๋น๋ยทํ
์คํธยทgit)',
|
|
101
|
+
agent: 'haiku-runner',
|
|
102
|
+
kw: [[/git |commit|push|npm |pip|๋น๋ํด|๋น๋ ๋/i, 2], [/์คํ|๋๋ ค|run |build|๋น๋|ํ
์คํธ|์ค์น/i, 1]],
|
|
102
103
|
},
|
|
103
104
|
];
|
|
104
105
|
|
|
106
|
+
// โโ Behavior signal (1st) โ what the episode actually DID โโโโโโโโโโโโโโโโ
|
|
107
|
+
// The tool-call histogram is ground truth the prompt's wording is not:
|
|
108
|
+
// "ํ
์คํธ ํต๊ณผํ๋์ง ํ์ธํด์ค" that actually ran `npx playwright test` IS a
|
|
109
|
+
// run episode regardless of phrasing. Keyword scores only pick within (or,
|
|
110
|
+
// when behavior is inconclusive, across) the plausible pool.
|
|
111
|
+
const RUN_TOOLS = new Set(['Bash']);
|
|
112
|
+
const LOOKUP_TOOLS = new Set(['Read', 'Grep', 'Glob', 'LS', 'WebFetch', 'WebSearch']);
|
|
113
|
+
const WRITE_TOOLS = new Set(['Write', 'Edit', 'NotebookEdit']);
|
|
114
|
+
const MIN_BEHAVIOR_CALLS = 2; // fewer calls than this โ too little behavior to trust
|
|
115
|
+
|
|
116
|
+
/**
|
|
117
|
+
* Narrow the candidate categories from the episode's tool mix.
|
|
118
|
+
* Returns an array of category ids (ordered โ first is the default when
|
|
119
|
+
* keywords stay silent), or null when behavior is inconclusive.
|
|
120
|
+
*/
|
|
121
|
+
export function behaviorPool(toolCounts) {
|
|
122
|
+
let run = 0, lookup = 0, write = 0, total = 0;
|
|
123
|
+
for (const [name, n] of Object.entries(toolCounts || {})) {
|
|
124
|
+
total += n;
|
|
125
|
+
if (RUN_TOOLS.has(name)) run += n;
|
|
126
|
+
else if (LOOKUP_TOOLS.has(name)) lookup += n;
|
|
127
|
+
else if (WRITE_TOOLS.has(name)) write += n;
|
|
128
|
+
}
|
|
129
|
+
if (total < MIN_BEHAVIOR_CALLS) return null;
|
|
130
|
+
if (run > lookup + write) return ['run'];
|
|
131
|
+
if (lookup > run + write) return ['explore', 'read', 'check', 'translate'];
|
|
132
|
+
return null; // mixed / write-heavy โ no reliable verdict, keywords decide
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
/** Weighted keyword score for one category (0 when it has no kw table). */
|
|
136
|
+
function keywordScore(cat, text) {
|
|
137
|
+
if (!cat.kw) return 0;
|
|
138
|
+
let score = 0;
|
|
139
|
+
for (const [re, w] of cat.kw) if (re.test(text)) score += w;
|
|
140
|
+
return score;
|
|
141
|
+
}
|
|
142
|
+
|
|
105
143
|
// Episodes that are not user-delegable requests: bare continuations, injected
|
|
106
144
|
// notifications, image pastes. These are easy but there is nothing to route.
|
|
107
145
|
const SKIP_RE = /^(๊ณ์|์ด์ด์|continue|๋ค์|proceed|์งํ|์|๋ค|๋ต|ok|okay|yes|ใฑ+|๊ณ ๊ณ )\b/i;
|
|
@@ -127,10 +165,27 @@ export function mungeProjectPath(p) {
|
|
|
127
165
|
return String(p).replace(/[^a-zA-Z0-9-]/g, '-');
|
|
128
166
|
}
|
|
129
167
|
|
|
130
|
-
|
|
168
|
+
/**
|
|
169
|
+
* Classify an episode. Gates (paste by length) run first, then the tool-mix
|
|
170
|
+
* behavior signal narrows the candidate pool, then weighted keyword scores
|
|
171
|
+
* pick within it (highest score wins; ties fall back to CATEGORIES order).
|
|
172
|
+
* A behavior verdict without any keyword hit still classifies (pool's first
|
|
173
|
+
* id); no behavior AND no keyword hit โ null (nothing delegable to name).
|
|
174
|
+
*/
|
|
175
|
+
export function categorize(text, toolCounts) {
|
|
131
176
|
if (text.length >= PASTE_MIN_LEN) return CATEGORIES.find((c) => c.id === 'paste');
|
|
132
|
-
|
|
133
|
-
|
|
177
|
+
const pool = behaviorPool(toolCounts);
|
|
178
|
+
const eligible = pool
|
|
179
|
+
? pool.map((id) => CATEGORIES.find((c) => c.id === id))
|
|
180
|
+
: CATEGORIES.filter((c) => c.kw);
|
|
181
|
+
let best = null;
|
|
182
|
+
let bestScore = 0;
|
|
183
|
+
for (const c of eligible) {
|
|
184
|
+
const s = keywordScore(c, text);
|
|
185
|
+
if (s > bestScore) { best = c; bestScore = s; }
|
|
186
|
+
}
|
|
187
|
+
if (best) return best;
|
|
188
|
+
return pool ? eligible[0] : null;
|
|
134
189
|
}
|
|
135
190
|
|
|
136
191
|
function isSkippable(text) {
|
|
@@ -146,7 +201,7 @@ function toEpisodes(records) {
|
|
|
146
201
|
for (const r of records) {
|
|
147
202
|
const text = (r.userText || '').trim();
|
|
148
203
|
if (!cur || cur.text !== text) {
|
|
149
|
-
cur = { text, calls: 0, out: 0, mutating: 0, errors: 0, delegated: 0, models: new Set(), cwd: '' };
|
|
204
|
+
cur = { text, calls: 0, out: 0, mutating: 0, errors: 0, delegated: 0, models: new Set(), cwd: '', tools: {} };
|
|
150
205
|
episodes.push(cur);
|
|
151
206
|
}
|
|
152
207
|
cur.calls += 1;
|
|
@@ -154,6 +209,9 @@ function toEpisodes(records) {
|
|
|
154
209
|
cur.mutating += r.mutatingToolCalls || 0;
|
|
155
210
|
cur.errors += r.toolErrors || 0;
|
|
156
211
|
cur.delegated += r.delegationCalls || 0;
|
|
212
|
+
for (const [name, n] of Object.entries(r.toolCounts || {})) {
|
|
213
|
+
cur.tools[name] = (cur.tools[name] || 0) + n;
|
|
214
|
+
}
|
|
157
215
|
cur.models.add(r.model);
|
|
158
216
|
if (!cur.cwd && r.cwd) cur.cwd = r.cwd;
|
|
159
217
|
}
|
|
@@ -244,17 +302,20 @@ export async function runRouteScan({ days = 14 } = {}) {
|
|
|
244
302
|
for (const { ep, projectDir } of all) {
|
|
245
303
|
if (isSkippable(ep.text)) continue;
|
|
246
304
|
if (![...ep.models].some(isExpensiveModel)) continue; // already cheap
|
|
247
|
-
const cat = categorize(ep.text);
|
|
305
|
+
const cat = categorize(ep.text, ep.tools);
|
|
248
306
|
if (cat) {
|
|
249
307
|
// rule-health denominator: episodes that LOOK delegable by shape
|
|
250
308
|
// (tier judged with the error signal zeroed โ using real errors here
|
|
251
309
|
// would be circular, since T2 requires errors=0 by definition). The
|
|
252
310
|
// numerator is those that still hit errors: exactly the "light-looking
|
|
253
311
|
// work in this category keeps failing" risk a delegation rule cares about.
|
|
312
|
+
// Keyed by tier as well: a category can carry both a T2 and a T1 rule,
|
|
313
|
+
// and sharing one category-wide stat would double-count every episode
|
|
314
|
+
// into both rules (identical รN / err% on unrelated tiers).
|
|
254
315
|
const shapeTier = tierOf({ ...ep, errors: 0 }, cat, thresholds);
|
|
255
316
|
if (shapeTier === 'T1' || shapeTier === 'T2') {
|
|
256
|
-
bumpStats(`${cat.id}|${projectDir}`, ep);
|
|
257
|
-
bumpStats(`${cat.id}|*`, ep);
|
|
317
|
+
bumpStats(`${shapeTier}|${cat.id}|${projectDir}`, ep);
|
|
318
|
+
bumpStats(`${shapeTier}|${cat.id}|*`, ep);
|
|
258
319
|
}
|
|
259
320
|
}
|
|
260
321
|
const tier = tierOf(ep, cat, thresholds);
|
|
@@ -299,8 +360,8 @@ export async function runRouteScan({ days = 14 } = {}) {
|
|
|
299
360
|
(r.scope === 'global' || r.project === g.project));
|
|
300
361
|
|
|
301
362
|
const ruleText = (g) => g.tier === 'T2'
|
|
302
|
-
? `"${g.label}" ์ ํ์ ๋จ์ ์์ฒญ(์: "${g.example}")์ ${g.agent}(haiku) ์๋ธ์์ด์ ํธ๋ก
|
|
303
|
-
: `"${g.label}" ์ ํ์ ์ค๊ฐ ๋๋ ์์ฒญ(์: "${g.example}")์ model: sonnet ์๋ธ์์ด์ ํธ๋ก ์์ํ๋ค (์ค๊ณ
|
|
363
|
+
? `"${g.label}" ์ ํ์ ๋จ์ ์์ฒญ(์: "${g.example}")์ ${g.agent}(haiku) ์๋ธ์์ด์ ํธ๋ก ์์ํ๋ค (์ค๊ณ ํ๋จยท๋ฐฐํฌยท์คํ ์ด ์ ์ถ ๊ฐ์ ๋น๊ฐ์ญ ์์
์ด ์์ด๋ฉด ์์ํ์ง ์์)`
|
|
364
|
+
: `"${g.label}" ์ ํ์ ์ค๊ฐ ๋๋ ์์ฒญ(์: "${g.example}")์ model: sonnet ์๋ธ์์ด์ ํธ๋ก ์์ํ๋ค (์ค๊ณ ํ๋จยท๋น๊ฐ์ญ ์์
ยท๋ฐ๋ณต ์๋ฌ ๋ฐ์ ์ ๋ฉ์ธ ๋ชจ๋ธ์ด ์ด์ด๋ฐ์)`;
|
|
304
365
|
|
|
305
366
|
const candidates = [...groups.values()]
|
|
306
367
|
.filter((g) => g.count >= MIN_RECURRENCE && !hasRule(g))
|
|
@@ -406,6 +467,30 @@ export async function shouldRescan(cache, { days = 14 } = {}) {
|
|
|
406
467
|
return age >= RESCAN_MAX_AGE_MS;
|
|
407
468
|
}
|
|
408
469
|
|
|
470
|
+
/**
|
|
471
|
+
* Human-readable labels for the T0/T1/T2 codes and scope values โ used by
|
|
472
|
+
* every user-facing listing so a bare code never appears without its meaning
|
|
473
|
+
* (first-time users can't be expected to know the tier vocabulary).
|
|
474
|
+
*/
|
|
475
|
+
export function tierLabel(tier, lang = 'ko') {
|
|
476
|
+
const ko = {
|
|
477
|
+
T2: '๋จ์ ์์
โ haiku๊ธ์ด๋ฉด ์ถฉ๋ถ',
|
|
478
|
+
T1: '์ค๊ฐ ๋๋ โ sonnet๊ธ์ด๋ฉด ์ถฉ๋ถ',
|
|
479
|
+
T0: '๊ณ ๋๋ โ ์ง๊ธ ๋ชจ๋ธ ์ ์ง',
|
|
480
|
+
};
|
|
481
|
+
const en = {
|
|
482
|
+
T2: 'simple โ haiku-class is enough',
|
|
483
|
+
T1: 'moderate โ sonnet-class is enough',
|
|
484
|
+
T0: 'hard โ stays on the session model',
|
|
485
|
+
};
|
|
486
|
+
return (lang === 'ko' ? ko : en)[tier] || tier;
|
|
487
|
+
}
|
|
488
|
+
|
|
489
|
+
export function scopeLabel(scope, lang = 'ko') {
|
|
490
|
+
if (lang === 'ko') return scope === 'global' ? '๋ชจ๋ ํ๋ก์ ํธ(๊ธ๋ก๋ฒ)' : '์ด ํ๋ก์ ํธ๋ง';
|
|
491
|
+
return scope === 'global' ? 'all projects (global)' : 'this project only';
|
|
492
|
+
}
|
|
493
|
+
|
|
409
494
|
/** Candidates not yet promoted/dismissed. */
|
|
410
495
|
export function openCandidates(cache) {
|
|
411
496
|
if (!cache?.candidates) return [];
|
package/src/session-records.js
CHANGED
|
@@ -79,14 +79,18 @@ export async function collectSessionRecords(filePath, { includeContent = true }
|
|
|
79
79
|
|
|
80
80
|
let mutatingToolCalls = 0;
|
|
81
81
|
let delegationCalls = 0;
|
|
82
|
+
const prev = records.get(reqId);
|
|
83
|
+
// Per-tool call histogram โ what the call actually DID. route-scan's
|
|
84
|
+
// categorizer trusts this over the prompt's wording (behavior-first).
|
|
85
|
+
const toolCounts = { ...(prev?.toolCounts || {}) };
|
|
82
86
|
if (Array.isArray(msg.content)) {
|
|
83
87
|
for (const b of msg.content) {
|
|
84
88
|
if (!b || b.type !== 'tool_use') continue;
|
|
85
89
|
if (MUTATING_TOOLS.has(b.name)) mutatingToolCalls += 1;
|
|
86
90
|
if (DELEGATION_TOOLS.has(b.name)) delegationCalls += 1;
|
|
91
|
+
if (typeof b.name === 'string') toolCounts[b.name] = (toolCounts[b.name] || 0) + 1;
|
|
87
92
|
}
|
|
88
93
|
}
|
|
89
|
-
const prev = records.get(reqId);
|
|
90
94
|
|
|
91
95
|
lastRecord = {
|
|
92
96
|
model: normalizeModelId(msg.model),
|
|
@@ -104,6 +108,7 @@ export async function collectSessionRecords(filePath, { includeContent = true }
|
|
|
104
108
|
mutatingToolCalls: (prev?.mutatingToolCalls || 0) + mutatingToolCalls,
|
|
105
109
|
delegationCalls: (prev?.delegationCalls || 0) + delegationCalls,
|
|
106
110
|
toolErrors: prev?.toolErrors || 0,
|
|
111
|
+
toolCounts,
|
|
107
112
|
};
|
|
108
113
|
records.set(reqId, lastRecord);
|
|
109
114
|
}
|