claude-token-saver 3.44.0 → 3.50.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -21
- package/NOTICE +35 -0
- package/README.ko.md +99 -48
- package/README.md +154 -200
- package/bin/cli.js +67 -29
- package/examples/statusline-command.ps1 +11 -5
- package/examples/statusline-command.sh +9 -4
- package/package.json +8 -5
- package/presets/model-rules.json +50 -9
- package/src/changelog.js +70 -0
- package/src/cli-name.js +25 -0
- package/src/commands/brief.js +13 -1
- package/src/commands/install.js +1 -0
- package/src/commands/mode.js +22 -2
- package/src/commands/profile-map.js +164 -0
- package/src/commands/route-scan.js +70 -7
- package/src/commands/uninstall.js +5 -4
- package/src/config.js +22 -3
- package/src/effort-palette.js +145 -0
- package/src/formatters/statusline.js +349 -122
- package/src/formatters/table.js +5 -4
- package/src/gateway-auth.js +86 -0
- package/src/glyphs.js +103 -0
- package/src/hook-manager.js +3 -2
- package/src/installer.js +132 -16
- package/src/litellm-budget.js +38 -67
- package/src/litellm-models.js +264 -0
- package/src/model-alias.js +170 -16
- package/src/model-rules.js +53 -17
- package/src/route-inject.js +94 -0
- package/src/route-scan.js +24 -1
- package/src/seed-rules.js +29 -16
- package/src/statusline-mode.js +95 -0
- package/src/stdin-payload.js +40 -0
- package/src/ultracode.js +202 -0
- package/src/update-check.js +178 -1
- package/src/window-labels.js +17 -7
package/README.md
CHANGED
|
@@ -10,231 +10,170 @@
|
|
|
10
10
|
<img alt="Sprag" src="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/site/assets/logo/sprag-lockup-light.svg" width="220">
|
|
11
11
|
</picture>
|
|
12
12
|
|
|
13
|
-
**A quality ratchet for
|
|
13
|
+
**A quality ratchet for your coding agent. Forward motion passes, backspin locks.**
|
|
14
14
|
|
|
15
15
|
[](https://www.npmjs.com/package/sprag-cli)
|
|
16
|
-
[](https://www.npmjs.com/package/sprag-cli)
|
|
17
|
+
[](./LICENSE)
|
|
18
18
|
|
|
19
|
-
[
|
|
19
|
+
[English](https://github.com/rootstudioyaml/sprag/blob/main/README.md) · [한국어](https://github.com/rootstudioyaml/sprag/blob/main/README.ko.md)
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
<!-- ko:begin (generated from README.ko.md — edit that file, then run npm run sync:ko) -->
|
|
26
|
-
**🇰🇷 한국어 문서: [README.ko.md](https://github.com/rootstudioyaml/sprag/blob/main/README.ko.md)** (아래 토글로도 펼칠 수 있습니다.)
|
|
21
|
+
[sprag.io](https://sprag.io) · [Benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md)
|
|
27
22
|
|
|
28
|
-
|
|
29
|
-
<summary><strong>🇰🇷 한국어로 보기 (클릭하면 한국어 전문이 펼쳐집니다)</strong></summary>
|
|
23
|
+
</div>
|
|
30
24
|
|
|
31
25
|
---
|
|
32
26
|
|
|
33
|
-
|
|
27
|
+
Sprag is named after the sprag clutch: forward motion passes, backspin locks.
|
|
28
|
+
Your agent keeps working, it just stops regressing.
|
|
34
29
|
|
|
35
|
-
|
|
30
|
+
It reads the sessions you already ran, and what it finds there holds for every
|
|
31
|
+
session after. Repeated failures become rules loaded at session start. Work that
|
|
32
|
+
a cheaper tier has never got wrong goes to a sub-agent on that tier, and every
|
|
33
|
+
delegated run writes its own price difference to a ledger. Cache and rate-limit
|
|
34
|
+
trouble reaches your statusline while you can still act on it.
|
|
36
35
|
|
|
37
|
-
|
|
36
|
+
Nothing leaves your machine, because it reads only the logs Claude Code already
|
|
37
|
+
writes there. No API key, no extra model calls, no runtime dependencies.
|
|
38
38
|
|
|
39
39
|
```bash
|
|
40
|
-
npm i -g sprag-cli #
|
|
40
|
+
npm i -g sprag-cli # the old claude-token-saver package still gets the same releases
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-

|
|
46
44
|
|
|
47
|
-
|
|
48
|
-
|---|---|---|---|
|
|
49
|
-
| 공개 벤치마크 정확도 | **59.1%** · 단일 최고 모델 57.9% | `▰▰▰▰▰▰▰▰▰▰▰▰` | [벤치마크](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
50
|
-
| 같은 문항 총비용 | **$268** · gpt-5 $388 · gemini-2.5-pro $734 | `▰▰▰▰▰▱▱▱▱▱▱▱` | [벤치마크](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
51
|
-
| 문서 토큰 | **−95.8%** · 540,429 → 22,610 | `▰▱▱▱▱▱▱▱▱▱▱▱` | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
52
|
-
| 메시지당 비용 | **−18.6%** · $2.345 → $1.910 | `▰▰▰▰▰▰▰▰▰▰▱▱` | [실측 리포트](#실제-효과-도입-전후-리포트) |
|
|
53
|
-
|
|
54
|
-
```text
|
|
55
|
-
정확도 총비용 (11,696문항)
|
|
56
|
-
티어 기준 라우터 59.1% ← 최고 $268 ▰▰▰▰▰▱▱▱▱▱▱▱▱▱▱
|
|
57
|
-
gpt-5 57.8% $388 ▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱
|
|
58
|
-
gemini-2.5-pro 57.9% $734 ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰
|
|
59
|
-
```
|
|
45
|
+
## How it works
|
|
60
46
|
|
|
61
|
-
|
|
47
|
+
<picture>
|
|
48
|
+
<source srcset="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/docs/how-it-works.svg" type="image/svg+xml">
|
|
49
|
+
<img src="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/docs/how-it-works.png" alt="What goes in: your past sessions, the task you just asked for, a document you would attach, Korean prose being written, and the tokens this turn is spending. What sprag does with it: writes a ratchet rule loaded every session, converts the document, checks the Korean as you write, prints the cost in your prompt, and routes a matched task to a haiku or sonnet sub-agent whose output the main agent reviews.">
|
|
50
|
+
</picture>
|
|
62
51
|
|
|
63
|
-
|
|
52
|
+
Everything on the left is something your session already produces. Nothing is
|
|
53
|
+
sent anywhere to read it. The tier decision at the bottom is the whole routing
|
|
54
|
+
argument in three tests: where the answer lives, whether a mistake surfaces, and
|
|
55
|
+
what error rate the rule was measured at.
|
|
64
56
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
| 📊 **토큰 텔레메트리** | 캐시 적중률과 TTL, 컨텍스트 크기, 출력 급증, 두 가지 사용량 한도를 매 턴 표시합니다. | [statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) |
|
|
71
|
-
| 📄 **doc2md** | pptx와 xlsx, pdf, docx, fig를 필요할 때 Markdown 으로 바꿔 읽습니다. | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
72
|
-
| 🇰🇷 **문체 게이트** | 쓰는 시점에 hook으로 검사합니다. 이중 피동과 번역투, 문장 연결을 살핍니다. | [문체 지침](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) |
|
|
57
|
+
Sprag does not swap the model you asked for. It reads what you have already run,
|
|
58
|
+
finds the kinds of task that never went wrong on a cheaper tier, and spawns a
|
|
59
|
+
sub-agent for exactly those. The main agent reviews what comes back before
|
|
60
|
+
anything counts as done, and the work returns to it the moment a rule stops
|
|
61
|
+
holding.
|
|
73
62
|
|
|
74
|
-
|
|
63
|
+
## Measured results
|
|
75
64
|
|
|
76
|
-
|
|
65
|
+
| Metric | Result | | Evidence |
|
|
66
|
+
|---|---|---|---|
|
|
67
|
+
| Public benchmark accuracy | **59.1%** vs 57.9% best single model | `▰▰▰▰▰▰▰▰▰▰▰▰` | [benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
68
|
+
| Cost for the same queries | **$268** vs gpt-5 $388, gemini-2.5-pro $734 | `▰▰▰▰▰▱▱▱▱▱▱▱` | [benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
69
|
+
| Tokens for the same documents | **−95.9%**, 2,011,178 → 82,209 over 12 files | `▰▱▱▱▱▱▱▱▱▱▱▱` | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
70
|
+
| Delegation savings, from the ledger | **$44.67** over 69 delegated runs | `▰▰▰▰▰▰▰▰▰▰▰▱` | `sprag route-scan savings` |
|
|
71
|
+
| Cheap tier on the same prompts | **24% of the cost**, one answer worse out of eight | `▰▰▰▱▱▱▱▱▱▱▱▱` | [A/B run](#the-cheap-tier-on-the-same-prompts) |
|
|
72
|
+
|
|
73
|
+
The first two rows answer the objection that comes up first: that moving work to
|
|
74
|
+
a cheaper model has to cost you accuracy. It did not. The same criteria, wired up
|
|
75
|
+
as a router, scored 1.2 points above the best single model at a third of the
|
|
76
|
+
cost. Nothing is best at everything, which is why choosing per task beats
|
|
77
|
+
choosing once.
|
|
78
|
+
|
|
79
|
+
Measured offline on LLMRouterBench (Findings of ACL 2026): 11,696 instances, 13
|
|
80
|
+
models whose answers, scores and costs ship with the dataset. No model was
|
|
81
|
+
called and nothing was spent. The router chose; the numbers were looked up. Run
|
|
82
|
+
it again and you get the same figures. It validates the criteria, not sprag's own
|
|
83
|
+
routing: the dataset has no Claude models, so deepseek-v3, qwen3-235b and gpt-5
|
|
84
|
+
stood in for cheap, mid and flagship.
|
|
85
|
+
|
|
86
|
+
The last three rows are ledgers on this machine. They grow as it runs. These were
|
|
87
|
+
read on 2026-09-18.
|
|
88
|
+
|
|
89
|
+
### Current, on one machine, last 30 days
|
|
77
90
|
|
|
78
91
|
```
|
|
79
|
-
🔀
|
|
80
|
-
🚨 5H █████▓ 94% 🔄 12:36 · 🅷 5/5 · 🤖 Opus 5 · 🧠 Cache hit 98.8% · ⏳ Cache expires 59:46 · 📅 weekly ██▒░░░ 38% · 💵 Sep $42 · 📦 Ctx 47% of 1M
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
문제가 생기면 경고 칩이 줄 맨 앞으로 옵니다. `🚨 5H/7D NN%`, `⚠ Ctx 500k+`, `⚠ Cache miss`, `⚠ Input spike`, `⚠ Output heavy`, `⚠ Call surge`, `⚠ Rebuild churn`, `⚠ 5m TTL` 여덟 가지이며, 칩 문구를 그대로 말하면 Skill이 원인 코드와 조치 순서를 꺼내 줍니다. 세그먼트별 의미와 색 기준은 [statusline 문서](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md)에 정리했습니다.
|
|
84
|
-
|
|
85
|
-
## 실제 효과: 도입 전후 리포트
|
|
86
|
-
|
|
87
|
-

|
|
88
|
-
|
|
89
|
-
harness 5/5 + ratchet을 실제 적용한 전후 비교입니다 (저자 Claude Code 로그, **사용자 메시지 1건당** 정규화, 2026-05-02 기준, Opus 4.7 가격).
|
|
90
|
-
|
|
91
|
-
| 메트릭 | 도입 전 (7일/739msg) | 도입 후 (2일/157msg) | 변화 |
|
|
92
|
-
|---|---:|---:|---:|
|
|
93
|
-
| 메시지당 비용 | $2.345 | $1.910 | **−18.6%** |
|
|
94
|
-
| 메시지당 출력 토큰 | 7,391 | 6,052 | −18.1% |
|
|
95
|
-
| 메시지당 assistant 왕복 | 9.73 | 8.83 | −9.2% |
|
|
96
|
-
| 메시지당 도구 호출 | 5.72 | 5.25 | −8.2% |
|
|
97
|
-
|
|
98
|
-
같은 요청을 더 적은 왕복으로 끝내므로 첫 시도 적중률이 오릅니다. PEV와 Structured Task가 한 번에 가게 만든 효과로 보입니다. 표본이 2일치(157msg)라 통계적 의미는 약하고, 주별 작업 토픽 차이도 섞여 있습니다.
|
|
99
|
-
|
|
100
|
-
## 자주 쓰는 명령
|
|
101
|
-
|
|
102
|
-
| 명령 | 하는 일 |
|
|
103
|
-
|---|---|
|
|
104
|
-
| `sprag` | 최근 1일 진단 리포트 |
|
|
105
|
-
| `sprag handoff` | 한도에 막히기 전에 작업 상태를 백업 |
|
|
106
|
-
| `sprag route-scan` | 내 기록에서 위임 후보 찾기 (LLM 호출 0회) |
|
|
107
|
-
| `sprag route-scan savings` | 라우팅 절감 원장 조회 |
|
|
108
|
-
| `sprag harness check` | 🅷 하네스 점수 확인 |
|
|
109
|
-
| `sprag doc2md <파일>` | 문서 한 건을 Markdown 으로 변환 |
|
|
110
|
-
|
|
111
|
-
전체 명령과 옵션, 출력 언어 설정은 [명령 문서](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md)에 있습니다.
|
|
112
|
-
|
|
113
|
-
## 더 깊은 내용
|
|
114
|
-
|
|
115
|
-
| 문서 | 다루는 내용 |
|
|
116
|
-
|---|---|
|
|
117
|
-
| [설치와 기본 설정](https://github.com/rootstudioyaml/sprag/blob/main/docs/INSTALL.md) | 설치 직후 켜지는 기능, 직접 켜야 하는 기능, 끄는 방법 |
|
|
118
|
-
| [statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) | 세그먼트 의미, 경고 칩, 토큰 급증 원인 코드, 업데이트 안내 |
|
|
119
|
-
| [명령](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md) | 전체 서브커맨드와 CLI 옵션 |
|
|
120
|
-
| [Harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) | 5원칙, 래칫 승격, compact-window 고정, seed 프리셋 |
|
|
121
|
-
| [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md) | 티어 판정, 룰 파일 구조, 스캔 시점, 서브에이전트 설정 |
|
|
122
|
-
| [티어 판별 기준](https://github.com/rootstudioyaml/sprag/blob/main/docs/TIER_CRITERIA.md) | T0·T1·T2 를 가르는 근거와 연구 출처 |
|
|
123
|
-
| [벤치마크](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) | LLMRouterBench 11,696문항 재현 결과 |
|
|
124
|
-
| [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) | 변환 대상 포맷, 절감 근거, 문서 수정 절차, DRM 문서 |
|
|
125
|
-
| [한국어 문체 지침](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) | 주입 내용, 쓰기 시점 검사, 적용 전후 비교 |
|
|
126
|
-
| [라우터가 아닌 이유](https://github.com/rootstudioyaml/sprag/blob/main/docs/NOT-A-ROUTER.md) | 실시간 라우팅이 캐시를 깨서 비용을 키우는 구조 |
|
|
127
|
-
| [게이트웨이와 환경](https://github.com/rootstudioyaml/sprag/blob/main/docs/GATEWAYS.md) | Bedrock·Vertex·LiteLLM, 가격표, FAQ, 동작 원리 |
|
|
128
|
-
|
|
129
|
-
## 릴리스 노트
|
|
130
|
-
|
|
131
|
-
전체 내역은 [CHANGELOG.md](./CHANGELOG.md)로 옮겼습니다. 최근 변경은 다음과 같습니다.
|
|
132
|
-
|
|
133
|
-
- **v3.39.0**: `feedback` 서브커맨드가 터미널이나 Claude 세션에서 버그 제보·기능 제안을 바로 제출합니다. gh CLI가 있으면 이슈 직접 등록, 없으면 로그인 없는 익명 제출(릴레이가 이슈로 자동 등록), 오프라인이면 로컬 저장. `install`은 기존 statusline이 있으면 조용히 건너뛰지 않고 교체 여부를 묻습니다.
|
|
134
|
-
- **v3.38.0**: `cohesion on`이 응집성 다섯 원칙을 영어 블록으로 주입합니다. 옵트인이고, korean 지침이 켜져 있으면 중복 주입을 막습니다.
|
|
135
|
-
- **v3.37.0**: 한국어 지침에 보강 지침(supplement)이 붙습니다. 번역투·상투 패턴·응집성(문장 이어짐) 조항이며, 쓰기 시점 검사에도 번역투 5종이 추가됐습니다. 실파일 255개 실측에서 오탐 1건으로 검증했습니다.
|
|
136
|
-
- **v3.35.0**: 이번 달 1일 00시 이후 지출 추정치를 `💵 Sep $42` 세그먼트로 상시 표시합니다. LiteLLM 게이트웨이 사용자는 키의 max_budget/spend 를 `🔑 budget` 게이지로 봅니다 (5h/7d cap 이 없는 Bedrock·LiteLLM 환경 대응).
|
|
137
|
-
- **v3.34.0**: seed 프리셋 제안, 설치 시 출력 언어 선택, 컨텍스트 경고 500k 상향. 상세는 CHANGELOG 참고.
|
|
138
|
-
|
|
139
|
-
## 피드백
|
|
140
|
-
|
|
141
|
-
버그 제보와 기능 제안은 GitHub 이슈로 받습니다: https://github.com/rootstudioyaml/sprag/issues
|
|
92
|
+
🔀 Routing saved $44.67 total (last 7d $23.21 · last 30d $42.73)
|
|
142
93
|
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
```bash
|
|
146
|
-
sprag feedback "Bedrock에서 5m TTL 칩이 사라지지 않아요"
|
|
94
|
+
opus-5 → sonnet-5 59 runs $39.75 ▰▰▰▰▰▰▰▰▰▰▱▱
|
|
95
|
+
opus-5 → haiku-4.5 10 runs $4.92 ▰▱▱▱▱▱▱▱▱▱▱▱
|
|
147
96
|
```
|
|
148
97
|
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
MIT
|
|
156
|
-
|
|
157
|
-
---
|
|
98
|
+
| Also measured | Over 30 days | |
|
|
99
|
+
|---|---|---|
|
|
100
|
+
| Documents converted | 12 files, 2,011,178 → 82,209 tokens | `▰▱▱▱▱▱▱▱▱▱▱▱` |
|
|
101
|
+
| Cache hit rate | 92.2% | `▰▰▰▰▰▰▰▰▰▰▰▱` |
|
|
102
|
+
| Volume behind both | 175 sessions · 14,682 API calls · 3.81B input tokens | |
|
|
158
103
|
|
|
159
|
-
|
|
104
|
+
Real output, from `sprag route-scan savings`, `sprag --days 30`, and the doc2md
|
|
105
|
+
ledger. Nothing here is a projection.
|
|
160
106
|
|
|
161
|
-
[
|
|
162
|
-
[](https://www.youtube.com/@DeepPulseEN)
|
|
163
|
-
[](https://rootstudioyaml.github.io/)
|
|
107
|
+
Every delegated run records its own price difference, so `sprag route-scan savings` traces any amount back to the rule that caused it. What the ledger deliberately leaves out is written down in [the command reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md).
|
|
164
108
|
|
|
165
|
-
|
|
109
|
+
## Everything ships in one install, working from day one
|
|
166
110
|
|
|
167
|
-
|
|
168
|
-
|
|
111
|
+
| | What it does | More |
|
|
112
|
+
|---|---|---|
|
|
113
|
+
| ⚙️ **Ratchet rules** | Repeated failures become one-line rules loaded every session. Candidates are detected from your logs, and you choose project or global scope. | [harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) |
|
|
114
|
+
| 🔀 **Model fitting** | Log-driven delegation rules carrying the error rate they were measured at and the savings they reported, written to `ratchet-model.md`. | [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md) |
|
|
115
|
+
| 🅷 **Harness score** | Five operating principles checked live. Skip the verify step and `🅷 4/5` says so before you report done. | [harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) |
|
|
116
|
+
| 📊 **Token telemetry** | Cache hit rate, TTL, context size, output spikes and both rate-limit windows, in the prompt every turn. | [statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) |
|
|
117
|
+
| 📄 **doc2md** | pptx, xlsx, pdf, docx and fig converted on demand, so a document costs one read instead of your whole context. | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
118
|
+
| 🇰🇷 **Style gates** | Write-time prose lint enforced by hook. Shipping today for Korean technical writing: double passives, translationese, cohesion. | [style](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) |
|
|
169
119
|
|
|
170
|
-
|
|
120
|
+
The two savings figures are never added together. They measure different things:
|
|
121
|
+
the routing ledger records the price difference on delegated runs, and the
|
|
122
|
+
conversion ledger records tokens a document did not cost.
|
|
171
123
|
|
|
172
|
-
|
|
124
|
+
Neither one counts what the harness and the ratchet rules save. That saving
|
|
125
|
+
arrives as fewer round-trips rather than as cheaper ones.
|
|
173
126
|
|
|
174
|
-
|
|
127
|
+
## The statusline in one line
|
|
175
128
|
|
|
176
|
-
```bash
|
|
177
|
-
npm i -g sprag-cli # same package as the old claude-token-saver name
|
|
178
129
|
```
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
## Four numbers
|
|
183
|
-
|
|
184
|
-
| Metric | Result | | Evidence |
|
|
185
|
-
|---|---|---|---|
|
|
186
|
-
| Public benchmark accuracy | **59.1%** vs 57.9% best single model | `▰▰▰▰▰▰▰▰▰▰▰▰` | [benchmark](./docs/BENCHMARK.md) |
|
|
187
|
-
| Cost for the same queries | **$268** vs gpt-5 $388, gemini-2.5-pro $734 | `▰▰▰▰▰▱▱▱▱▱▱▱` | [benchmark](./docs/BENCHMARK.md) |
|
|
188
|
-
| Tokens per document | **−95.8%**, 540,429 → 22,610 | `▰▱▱▱▱▱▱▱▱▱▱▱` | [doc2md](./docs/DOC2MD.md) |
|
|
189
|
-
| Cost per user message | **−18.6%**, $2.345 → $1.910 | `▰▰▰▰▰▰▰▰▰▰▱▱` | [report](#real-world-impact--beforeafter-report) |
|
|
190
|
-
|
|
191
|
-
```text
|
|
192
|
-
accuracy total cost, 11,696 queries
|
|
193
|
-
tier-criteria router 59.1% ← best $268 ▰▰▰▰▰▱▱▱▱▱▱▱▱▱▱
|
|
194
|
-
gpt-5 57.8% $388 ▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱
|
|
195
|
-
gemini-2.5-pro 57.9% $734 ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰
|
|
130
|
+
🔀 Routing saved $44.67 | opus→sonnet 59× $39.75 · opus→haiku 10× $4.92
|
|
131
|
+
🚨 5H ▰▰▰▰▰▰▰▰▰▰▰▱ 94% 🔄 12:36 · 🅷 5/5 · 🤖 Opus 5 · 🧠 Cache hit 98.8% · ⏳ Cache expires 59:46 · 📅 weekly ▰▰▰▰▰▱▱▱▱▱▱▱ 38% · 💵 Sep $42 · 📦 Ctx 47% of 1M
|
|
196
132
|
```
|
|
197
133
|
|
|
198
|
-
|
|
134
|
+
When something is wrong the warning chip leads the line: `🚨 5H/7D NN%`, `⚠ Ctx 500k+`, `⚠ Cache miss`, `⚠ Input spike`, `⚠ Output heavy`, `⚠ Call surge`, `⚠ Rebuild churn`, `⚠ 5m TTL`. Paste the chip text into Claude and the Skill names the root-cause code and the fix. Every segment is explained in [the statusline reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md).
|
|
199
135
|
|
|
200
|
-
|
|
136
|
+
Inside a JetBrains IDE terminal the chips render as single-cell glyphs instead of emoji (`◉ Cache hit 98.8% · ◧ Ctx 47% of 1M`). The IDE's default font has no emoji glyphs, so emoji leave characters from the previous frame on screen, like `Cache expires 4:545`. [Label modes](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md#label-modes) covers the detail, plus `sprag mode narrow` and `mode icon-force`.
|
|
201
137
|
|
|
202
|
-
|
|
203
|
-
|---|---|---|
|
|
204
|
-
| ⚙️ **Ratchet rules** | Repeated failures become one-line rules loaded every session; candidates are detected from your logs. | [harness](./docs/HARNESS.md) |
|
|
205
|
-
| 🔀 **Model fitting** | Log-driven delegation rules with measured error rates and savings, written to `ratchet-model.md`. | [route-scan](./docs/ROUTE_SCAN.md) |
|
|
206
|
-
| 🅷 **Harness score** | Five operating principles checked live. Skip the verify step and `🅷 4/5` says so before you report done. | [harness](./docs/HARNESS.md) |
|
|
207
|
-
| 📊 **Token telemetry** | Cache hit rate, TTL, context size, output spikes and both rate-limit windows, every turn. | [statusline](./docs/STATUSLINE.md) |
|
|
208
|
-
| 📄 **doc2md** | pptx, xlsx, pdf, docx and fig converted on demand instead of pasted into context. | [doc2md](./docs/DOC2MD.md) |
|
|
209
|
-
| 🇰🇷 **Style gates** | Write-time prose lint enforced by hook: double passives, translationese, cohesion. | [style](./docs/KOREAN-STYLE.md) |
|
|
138
|
+
## The cheap tier on the same prompts
|
|
210
139
|
|
|
211
|
-
The
|
|
140
|
+
The benchmark uses public data. This asks the same question of my own work: eight
|
|
141
|
+
prompts from real sessions in this repository, sent to both tiers, graded against
|
|
142
|
+
an answer written before the run (2026-09-17, raw data in
|
|
143
|
+
`video/out/measure2.json`).
|
|
212
144
|
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
```
|
|
145
|
+
| | flagship (opus-5) | cheap tier (haiku-4.5) |
|
|
146
|
+
|---|---:|---:|
|
|
147
|
+
| Cost for all eight | $2.5652 | **$0.6132** (23.9%) |
|
|
148
|
+
| Graded correct | 7 / 8 | 6 / 8 |
|
|
149
|
+
| Gave the same answer | | 6 / 8 |
|
|
219
150
|
|
|
220
|
-
|
|
151
|
+
The prompt both tiers missed is the useful one. It asked which files reference
|
|
152
|
+
`process.env` without saying whether subdirectories counted, and the two read it
|
|
153
|
+
differently. Adding that one condition made their answers identical. The question
|
|
154
|
+
was ambiguous, not hard.
|
|
221
155
|
|
|
222
|
-
|
|
156
|
+
The remaining miss is the cheap tier leaving entries out of a file listing. A
|
|
157
|
+
wrong listing is one you can see is wrong, which is the second criterion doing
|
|
158
|
+
its job.
|
|
223
159
|
|
|
224
|
-
|
|
160
|
+
So the cheap tier is not free. On this sample one lookup in eight comes back
|
|
161
|
+
worse, for a quarter of the price. That is why rules only send work whose failure
|
|
162
|
+
surfaces, and never send design or diagnosis.
|
|
225
163
|
|
|
226
|
-
|
|
164
|
+
## What the method has produced here
|
|
227
165
|
|
|
228
|
-
|
|
|
229
|
-
|
|
230
|
-
|
|
|
231
|
-
|
|
|
232
|
-
|
|
|
233
|
-
| tool calls / message | 5.72 | 5.25 | −8.2% |
|
|
166
|
+
| Measured | Value | Sample | Window |
|
|
167
|
+
|---|---|---|---|
|
|
168
|
+
| Documents converted | 2,011,178 → 82,209 tokens | 12 files (xlsx, pptx, pdf) | to 2026-09-18 |
|
|
169
|
+
| Delegated runs | $44.67 saved, $0.65 per run | 69 runs | to 2026-09-18 |
|
|
170
|
+
| Ratchet rules accumulated | 37 (30 global, 7 per-project) | 4 repositories | 2026-05-08 to 09-16 |
|
|
234
171
|
|
|
235
|
-
The
|
|
172
|
+
The two counts grow differently. Ratchet rules come from mistakes a person made,
|
|
173
|
+
so they keep coming. Delegation rules stop: the criteria recognize a fixed set of
|
|
174
|
+
work types, and once each has a rule there is nothing left to promote.
|
|
236
175
|
|
|
237
|
-
##
|
|
176
|
+
## Day-to-day commands
|
|
238
177
|
|
|
239
178
|
| Command | What it does |
|
|
240
179
|
|---|---|
|
|
@@ -245,51 +184,66 @@ The same request resolves in fewer round-trips, so first-try success goes up: th
|
|
|
245
184
|
| `sprag harness check` | Current 🅷 harness score |
|
|
246
185
|
| `sprag doc2md <file>` | Convert one document to Markdown |
|
|
247
186
|
|
|
248
|
-
Every subcommand, flag and the output-language setting: [command reference](
|
|
187
|
+
Every subcommand, flag and the output-language setting: [command reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md).
|
|
249
188
|
|
|
250
|
-
##
|
|
189
|
+
## Documentation
|
|
251
190
|
|
|
252
191
|
| Doc | Covers |
|
|
253
192
|
|---|---|
|
|
254
|
-
| [Install](
|
|
255
|
-
| [Statusline](
|
|
256
|
-
| [Commands](
|
|
257
|
-
| [Harness](
|
|
258
|
-
| [route-scan](
|
|
259
|
-
| [Tier criteria](
|
|
260
|
-
| [Benchmark](
|
|
261
|
-
| [doc2md](
|
|
262
|
-
| [Korean style](
|
|
263
|
-
| [Not a router](
|
|
264
|
-
| [Gateways & environment](
|
|
193
|
+
| [Install](https://github.com/rootstudioyaml/sprag/blob/main/docs/INSTALL.md) | What the install turns on, what stays manual, how to turn each part off |
|
|
194
|
+
| [Statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) | Segments, warning chips, spike issue codes, update notifications |
|
|
195
|
+
| [Commands](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md) | Full CLI surface |
|
|
196
|
+
| [Harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) | The five principles, rule promotion, compact-window pinning, seed presets |
|
|
197
|
+
| [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md) | Tier verdicts, rule-file mechanics, scan triggers, subagent setup |
|
|
198
|
+
| [Tier criteria](https://github.com/rootstudioyaml/sprag/blob/main/docs/TIER_CRITERIA.md) | Why T0/T1/T2 split where they do, with sources (Korean) |
|
|
199
|
+
| [Benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) | The 11,696-instance LLMRouterBench run |
|
|
200
|
+
| [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) | Supported formats, savings evidence, editing documents, DRM cases |
|
|
201
|
+
| [Korean style](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) | What gets injected, the write-time check, before/after |
|
|
202
|
+
| [Not a router](https://github.com/rootstudioyaml/sprag/blob/main/docs/NOT-A-ROUTER.md) | Why realtime routing breaks the cache and costs more |
|
|
203
|
+
| [Gateways & environment](https://github.com/rootstudioyaml/sprag/blob/main/docs/GATEWAYS.md) | Bedrock, Vertex, LiteLLM budgets, pricing table, FAQ, how it works |
|
|
204
|
+
|
|
205
|
+
Behind a LiteLLM gateway, `sprag profile-map --refresh` now reads model aliases straight from the gateway's own `GET /model/info`, so route-scan no longer needs to wait on learned votes or a hand-edited `profile-map.json`. The refresh uses the same 5-minute check and 24-hour cache as the LiteLLM budget gauge. It never stores your auth token: each refresh reads it from `apiKeyHelper` at call time and lets it expire on the helper's own schedule. Details: [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md).
|
|
265
206
|
|
|
266
207
|
## Release notes
|
|
267
208
|
|
|
268
209
|
The full history moved to [CHANGELOG.md](./CHANGELOG.md) (Korean; version headings and command names are language-neutral). Recent changes:
|
|
269
210
|
|
|
270
|
-
- **v3.39.0**: `feedback` subcommand
|
|
271
|
-
- **v3.38.0**: `cohesion on`
|
|
211
|
+
- **v3.39.0**: `feedback` subcommand for filing bug reports and feature requests straight from the terminal or a Claude session, via the gh CLI, an anonymous no-login form (auto-filed as a GitHub issue by an Apps Script relay), or a local fallback. `install` now asks before replacing an existing statusline instead of silently skipping.
|
|
212
|
+
- **v3.38.0**: `cohesion on` turns the language-neutral cohesion rules from the Korean supplement into a standalone English injection (given-before-new, one referent per pronoun, subject consistency, bridging, merging choppy sentences). Opt-in, ~0.5k tokens per session, suppressed while `korean on` already carries them.
|
|
272
213
|
- **v3.37.0**: Korean guidance grows a conservative supplement (translationese, AI-writing tics, a research-backed cohesion section whose principles apply to English prose too) and the write-time lint gains 5 translationese patterns, validated at 1 false positive across 255 real files.
|
|
273
|
-
- **v3.35.0**: A `💵 Sep $42` segment now shows estimated spend since 00:00 on the 1st of the current month, always on
|
|
214
|
+
- **v3.35.0**: A `💵 Sep $42` segment now shows estimated spend since 00:00 on the 1st of the current month, always on, including gateway setups with no 5h/7d caps. LiteLLM gateway users get a `🔑 budget ▰▱ 34% $34/$100` gauge built from the key's budget (`GET /key/info` + `GET /user/info`, team-membership budget first, then key, then internal user, verified against a Dockerized LiteLLM).
|
|
274
215
|
- **v3.34.0**: seed presets offered one at a time, output-language choice at install, context warning raised to 500k.
|
|
275
216
|
|
|
276
217
|
## Feedback
|
|
277
218
|
|
|
278
|
-
|
|
219
|
+
Bugs and feature requests go in the issue tracker: https://github.com/rootstudioyaml/sprag/issues
|
|
279
220
|
|
|
280
|
-
|
|
221
|
+
With no browser or GitHub login at hand (corporate network, mid-session), submit straight from the terminal instead, or ask Claude to do it for you:
|
|
281
222
|
|
|
282
223
|
```bash
|
|
283
224
|
sprag feedback "the 5m TTL chip never clears on Bedrock"
|
|
284
225
|
```
|
|
285
226
|
|
|
286
|
-
It files a GitHub issue
|
|
227
|
+
It files a GitHub issue through the `gh` CLI when one is authenticated. Otherwise it submits anonymously, with no login, which works where github.com is blocked. Pass `--anonymous` to skip the `gh` path. Version and OS metadata are attached automatically.
|
|
287
228
|
|
|
288
|
-
|
|
229
|
+
For a bug, please include the tool version (`sprag --version`) and your OS. For a statusline or warning issue, add the statusline output or the `sprag last` result.
|
|
289
230
|
|
|
290
231
|
## License
|
|
291
232
|
|
|
292
|
-
|
|
233
|
+
Apache License 2.0. The full text is in [LICENSE](./LICENSE), and the attributions
|
|
234
|
+
it requires are in [NOTICE](./NOTICE).
|
|
235
|
+
|
|
236
|
+
Releases up to and including v3.47.0 went out under the MIT License. A copy
|
|
237
|
+
received under those terms stays under them; the new license applies to
|
|
238
|
+
releases after that.
|
|
239
|
+
|
|
240
|
+
**Running sprag inside a product or a paid service?** The license already permits
|
|
241
|
+
that, so this is not a request for permission. We would like to hear about it
|
|
242
|
+
anyway, because the parts that get awkward at scale are the parts we would rather
|
|
243
|
+
fix than have you work around: shared ratchet rules across a team, gateway setups
|
|
244
|
+
where the model id is an opaque profile, and per-seat budget reporting. [Open an
|
|
245
|
+
issue](https://github.com/rootstudioyaml/sprag/issues) or reach us through the
|
|
246
|
+
channels below.
|
|
293
247
|
|
|
294
248
|
---
|
|
295
249
|
|
|
@@ -299,4 +253,4 @@ MIT
|
|
|
299
253
|
[](https://www.youtube.com/@DeepPulseEN)
|
|
300
254
|
[](https://rootstudioyaml.github.io/)
|
|
301
255
|
|
|
302
|
-
Built
|
|
256
|
+
Built at **DeepPulse**, a channel about AI developer tooling, and used there every day. The [launch Short (60s)](https://www.youtube.com/shorts/RaD8qMsPTnA) covers where it came from and how it gets used.
|