sprag-cli 3.46.2 → 3.48.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -21
- package/NOTICE +35 -0
- package/README.ko.md +80 -23
- package/README.md +142 -39
- package/bin/cli.js +23 -3
- package/package.json +4 -3
- package/src/commands/profile-map.js +164 -0
- package/src/commands/route-scan.js +11 -7
- package/src/gateway-auth.js +86 -0
- package/src/litellm-budget.js +22 -65
- package/src/litellm-models.js +264 -0
- package/src/model-alias.js +170 -16
package/README.md
CHANGED
|
@@ -5,11 +5,11 @@
|
|
|
5
5
|
<img alt="Sprag" src="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/site/assets/logo/sprag-lockup-light.svg" width="220">
|
|
6
6
|
</picture>
|
|
7
7
|
|
|
8
|
-
**A quality ratchet for
|
|
8
|
+
**A quality ratchet for your coding agent. Forward motion passes, backspin locks.**
|
|
9
9
|
|
|
10
10
|
[](https://www.npmjs.com/package/sprag-cli)
|
|
11
11
|
[](https://www.npmjs.com/package/sprag-cli)
|
|
12
|
-
[](./LICENSE)
|
|
13
13
|
|
|
14
14
|
[English](https://github.com/rootstudioyaml/sprag/blob/main/README.md) · [한국어](https://github.com/rootstudioyaml/sprag/blob/main/README.ko.md)
|
|
15
15
|
|
|
@@ -19,15 +19,41 @@
|
|
|
19
19
|
|
|
20
20
|
---
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
Sprag is named after the sprag clutch: forward motion passes, backspin locks.
|
|
23
|
+
Your agent keeps working, it just stops regressing.
|
|
23
24
|
|
|
24
|
-
|
|
25
|
+
It reads the sessions you already ran, and what it finds there holds for every
|
|
26
|
+
session after. Repeated failures become rules loaded at session start. Work that
|
|
27
|
+
a cheaper tier has never got wrong goes to a sub-agent on that tier, and every
|
|
28
|
+
delegated run writes its own price difference to a ledger. Cache and rate-limit
|
|
29
|
+
trouble reaches your statusline while you can still act on it.
|
|
30
|
+
|
|
31
|
+
Nothing leaves your machine, because it reads only the logs Claude Code already
|
|
32
|
+
writes there. No API key, no extra model calls, no runtime dependencies.
|
|
25
33
|
|
|
26
34
|
```bash
|
|
27
|
-
npm i -g sprag-cli #
|
|
35
|
+
npm i -g sprag-cli # the old claude-token-saver package still gets the same releases
|
|
28
36
|
```
|
|
29
37
|
|
|
30
|
-
|
|
39
|
+
|
|
40
|
+
## How it works
|
|
41
|
+
|
|
42
|
+
<picture>
|
|
43
|
+
<source srcset="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/docs/how-it-works.svg" type="image/svg+xml">
|
|
44
|
+
<img src="https://raw.githubusercontent.com/rootstudioyaml/sprag/main/docs/how-it-works.png" alt="What goes in: your past sessions, the task you just asked for, a document you would attach, Korean prose being written, and the tokens this turn is spending. What sprag does with it: writes a ratchet rule loaded every session, converts the document, checks the Korean as you write, prints the cost in your prompt, and routes a matched task to a haiku or sonnet sub-agent whose output the main agent reviews.">
|
|
45
|
+
</picture>
|
|
46
|
+
|
|
47
|
+
Everything on the left is something your session already produces. Nothing is
|
|
48
|
+
sent anywhere to read it. The tier decision at the bottom is the whole routing
|
|
49
|
+
argument in three tests: where the answer lives, whether a mistake surfaces, and
|
|
50
|
+
what error rate the rule was measured at.
|
|
51
|
+
|
|
52
|
+
Sprag does not swap the model you asked for. It reads what you have already run,
|
|
53
|
+
finds the kinds of task that never went wrong on a cheaper tier, and spawns a
|
|
54
|
+
sub-agent for exactly those. The main agent reviews what comes back before
|
|
55
|
+
anything counts as done, and the work returns to it the moment a rule stops
|
|
56
|
+
holding.
|
|
31
57
|
|
|
32
58
|
## Measured results
|
|
33
59
|
|
|
@@ -35,52 +61,114 @@ npm i -g sprag-cli # same package as the old claude-token-saver name
|
|
|
35
61
|
|---|---|---|---|
|
|
36
62
|
| Public benchmark accuracy | **59.1%** vs 57.9% best single model | `▰▰▰▰▰▰▰▰▰▰▰▰` | [benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
37
63
|
| Cost for the same queries | **$268** vs gpt-5 $388, gemini-2.5-pro $734 | `▰▰▰▰▰▱▱▱▱▱▱▱` | [benchmark](https://github.com/rootstudioyaml/sprag/blob/main/docs/BENCHMARK.md) |
|
|
38
|
-
| Tokens
|
|
39
|
-
|
|
|
64
|
+
| Tokens for the same documents | **−95.9%**, 2,011,178 → 82,209 over 12 files | `▰▱▱▱▱▱▱▱▱▱▱▱` | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
65
|
+
| Delegation savings, from the ledger | **$44.67** over 69 delegated runs | `▰▰▰▰▰▰▰▰▰▰▰▱` | `sprag route-scan savings` |
|
|
66
|
+
| Cheap tier on the same prompts | **24% of the cost**, one answer worse out of eight | `▰▰▰▱▱▱▱▱▱▱▱▱` | [A/B run](#the-cheap-tier-on-the-same-prompts) |
|
|
67
|
+
|
|
68
|
+
The first two rows answer the objection that comes up first: that moving work to
|
|
69
|
+
a cheaper model has to cost you accuracy. It did not. The same criteria, wired up
|
|
70
|
+
as a router, scored 1.2 points above the best single model at a third of the
|
|
71
|
+
cost. Nothing is best at everything, which is why choosing per task beats
|
|
72
|
+
choosing once.
|
|
73
|
+
|
|
74
|
+
Measured offline on LLMRouterBench (Findings of ACL 2026): 11,696 instances, 13
|
|
75
|
+
models whose answers, scores and costs ship with the dataset. No model was
|
|
76
|
+
called and nothing was spent. The router chose; the numbers were looked up. Run
|
|
77
|
+
it again and you get the same figures. It validates the criteria, not sprag's own
|
|
78
|
+
routing: the dataset has no Claude models, so deepseek-v3, qwen3-235b and gpt-5
|
|
79
|
+
stood in for cheap, mid and flagship.
|
|
80
|
+
|
|
81
|
+
The last three rows are ledgers on this machine. They grow as it runs. These were
|
|
82
|
+
read on 2026-09-18.
|
|
83
|
+
|
|
84
|
+
### Current, on one machine, last 30 days
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
🔀 Routing saved $44.67 total (last 7d $23.21 · last 30d $42.73)
|
|
40
88
|
|
|
89
|
+
opus-5 → sonnet-5 59 runs $39.75 ▰▰▰▰▰▰▰▰▰▰▱▱
|
|
90
|
+
opus-5 → haiku-4.5 10 runs $4.92 ▰▱▱▱▱▱▱▱▱▱▱▱
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
| Also measured | Over 30 days | |
|
|
94
|
+
|---|---|---|
|
|
95
|
+
| Documents converted | 12 files, 2,011,178 → 82,209 tokens | `▰▱▱▱▱▱▱▱▱▱▱▱` |
|
|
96
|
+
| Cache hit rate | 92.2% | `▰▰▰▰▰▰▰▰▰▰▰▱` |
|
|
97
|
+
| Volume behind both | 175 sessions · 14,682 API calls · 3.81B input tokens | |
|
|
98
|
+
|
|
99
|
+
Real output, from `sprag route-scan savings`, `sprag --days 30`, and the doc2md
|
|
100
|
+
ledger. Nothing here is a projection.
|
|
41
101
|
|
|
42
|
-
|
|
102
|
+
Every delegated run records its own price difference, so `sprag route-scan savings` traces any amount back to the rule that caused it. What the ledger deliberately leaves out is written down in [the command reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md).
|
|
43
103
|
|
|
44
104
|
## Everything ships in one install, working from day one
|
|
45
105
|
|
|
46
106
|
| | What it does | More |
|
|
47
107
|
|---|---|---|
|
|
48
|
-
| ⚙️ **Ratchet rules** | Repeated failures become one-line rules loaded every session
|
|
49
|
-
| 🔀 **Model fitting** | Log-driven delegation rules
|
|
108
|
+
| ⚙️ **Ratchet rules** | Repeated failures become one-line rules loaded every session. Candidates are detected from your logs, and you choose project or global scope. | [harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) |
|
|
109
|
+
| 🔀 **Model fitting** | Log-driven delegation rules carrying the error rate they were measured at and the savings they reported, written to `ratchet-model.md`. | [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md) |
|
|
50
110
|
| 🅷 **Harness score** | Five operating principles checked live. Skip the verify step and `🅷 4/5` says so before you report done. | [harness](https://github.com/rootstudioyaml/sprag/blob/main/docs/HARNESS.md) |
|
|
51
|
-
| 📊 **Token telemetry** | Cache hit rate, TTL, context size, output spikes and both rate-limit windows, every turn. | [statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) |
|
|
52
|
-
| 📄 **doc2md** | pptx, xlsx, pdf, docx and fig converted on demand instead of
|
|
53
|
-
| 🇰🇷 **Style gates** | Write-time prose lint enforced by hook: double passives, translationese, cohesion. | [style](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) |
|
|
111
|
+
| 📊 **Token telemetry** | Cache hit rate, TTL, context size, output spikes and both rate-limit windows, in the prompt every turn. | [statusline](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md) |
|
|
112
|
+
| 📄 **doc2md** | pptx, xlsx, pdf, docx and fig converted on demand, so a document costs one read instead of your whole context. | [doc2md](https://github.com/rootstudioyaml/sprag/blob/main/docs/DOC2MD.md) |
|
|
113
|
+
| 🇰🇷 **Style gates** | Write-time prose lint enforced by hook. Shipping today for Korean technical writing: double passives, translationese, cohesion. | [style](https://github.com/rootstudioyaml/sprag/blob/main/docs/KOREAN-STYLE.md) |
|
|
54
114
|
|
|
55
|
-
The
|
|
115
|
+
The two savings figures are never added together. They measure different things:
|
|
116
|
+
the routing ledger records the price difference on delegated runs, and the
|
|
117
|
+
conversion ledger records tokens a document did not cost.
|
|
118
|
+
|
|
119
|
+
Neither one counts what the harness and the ratchet rules save. That saving
|
|
120
|
+
arrives as fewer round-trips rather than as cheaper ones.
|
|
56
121
|
|
|
57
122
|
## The statusline in one line
|
|
58
123
|
|
|
59
124
|
```
|
|
60
|
-
🔀 Routing saved $
|
|
125
|
+
🔀 Routing saved $44.67 | opus→sonnet 59× $39.75 · opus→haiku 10× $4.92
|
|
61
126
|
🚨 5H ▰▰▰▰▰▰▰▰▰▰▰▱ 94% 🔄 12:36 · 🅷 5/5 · 🤖 Opus 5 · 🧠 Cache hit 98.8% · ⏳ Cache expires 59:46 · 📅 weekly ▰▰▰▰▰▱▱▱▱▱▱▱ 38% · 💵 Sep $42 · 📦 Ctx 47% of 1M
|
|
62
127
|
```
|
|
63
128
|
|
|
64
|
-
When something is wrong the warning chip leads the line: `🚨 5H/7D NN%`, `⚠ Ctx 500k+`, `⚠ Cache miss`, `⚠ Input spike`, `⚠ Output heavy`, `⚠ Call surge`, `⚠ Rebuild churn`, `⚠ 5m TTL`.
|
|
129
|
+
When something is wrong the warning chip leads the line: `🚨 5H/7D NN%`, `⚠ Ctx 500k+`, `⚠ Cache miss`, `⚠ Input spike`, `⚠ Output heavy`, `⚠ Call surge`, `⚠ Rebuild churn`, `⚠ 5m TTL`. Paste the chip text into Claude and the Skill names the root-cause code and the fix. Every segment is explained in [the statusline reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md).
|
|
130
|
+
|
|
131
|
+
Inside a JetBrains IDE terminal the chips render as single-cell glyphs instead of emoji (`◉ Cache hit 98.8% · ◧ Ctx 47% of 1M`). The IDE's default font has no emoji glyphs, so emoji leave characters from the previous frame on screen, like `Cache expires 4:545`. [Label modes](https://github.com/rootstudioyaml/sprag/blob/main/docs/STATUSLINE.md#label-modes) covers the detail, plus `sprag mode narrow` and `mode icon-force`.
|
|
65
132
|
|
|
66
|
-
|
|
133
|
+
## The cheap tier on the same prompts
|
|
67
134
|
|
|
68
|
-
|
|
135
|
+
The benchmark uses public data. This asks the same question of my own work: eight
|
|
136
|
+
prompts from real sessions in this repository, sent to both tiers, graded against
|
|
137
|
+
an answer written before the run (2026-09-17, raw data in
|
|
138
|
+
`video/out/measure2.json`).
|
|
69
139
|
|
|
70
|
-
|
|
140
|
+
| | flagship (opus-5) | cheap tier (haiku-4.5) |
|
|
141
|
+
|---|---:|---:|
|
|
142
|
+
| Cost for all eight | $2.5652 | **$0.6132** (23.9%) |
|
|
143
|
+
| Graded correct | 7 / 8 | 6 / 8 |
|
|
144
|
+
| Gave the same answer | | 6 / 8 |
|
|
71
145
|
|
|
72
|
-
|
|
146
|
+
The prompt both tiers missed is the useful one. It asked which files reference
|
|
147
|
+
`process.env` without saying whether subdirectories counted, and the two read it
|
|
148
|
+
differently. Adding that one condition made their answers identical. The question
|
|
149
|
+
was ambiguous, not hard.
|
|
73
150
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
| output tokens / message | 7,391 | 6,052 | −18.1% |
|
|
78
|
-
| assistant turns / message | 9.73 | 8.83 | −9.2% |
|
|
79
|
-
| tool calls / message | 5.72 | 5.25 | −8.2% |
|
|
151
|
+
The remaining miss is the cheap tier leaving entries out of a file listing. A
|
|
152
|
+
wrong listing is one you can see is wrong, which is the second criterion doing
|
|
153
|
+
its job.
|
|
80
154
|
|
|
81
|
-
|
|
155
|
+
So the cheap tier is not free. On this sample one lookup in eight comes back
|
|
156
|
+
worse, for a quarter of the price. That is why rules only send work whose failure
|
|
157
|
+
surfaces, and never send design or diagnosis.
|
|
82
158
|
|
|
83
|
-
##
|
|
159
|
+
## What the method has produced here
|
|
160
|
+
|
|
161
|
+
| Measured | Value | Sample | Window |
|
|
162
|
+
|---|---|---|---|
|
|
163
|
+
| Documents converted | 2,011,178 → 82,209 tokens | 12 files (xlsx, pptx, pdf) | to 2026-09-18 |
|
|
164
|
+
| Delegated runs | $44.67 saved, $0.65 per run | 69 runs | to 2026-09-18 |
|
|
165
|
+
| Ratchet rules accumulated | 37 (30 global, 7 per-project) | 4 repositories | 2026-05-08 to 09-16 |
|
|
166
|
+
|
|
167
|
+
The two counts grow differently. Ratchet rules come from mistakes a person made,
|
|
168
|
+
so they keep coming. Delegation rules stop: the criteria recognize a fixed set of
|
|
169
|
+
work types, and once each has a rule there is nothing left to promote.
|
|
170
|
+
|
|
171
|
+
## Day-to-day commands
|
|
84
172
|
|
|
85
173
|
| Command | What it does |
|
|
86
174
|
|---|---|
|
|
@@ -93,7 +181,7 @@ The same request resolves in fewer round-trips, so first-try success goes up: th
|
|
|
93
181
|
|
|
94
182
|
Every subcommand, flag and the output-language setting: [command reference](https://github.com/rootstudioyaml/sprag/blob/main/docs/COMMANDS.md).
|
|
95
183
|
|
|
96
|
-
##
|
|
184
|
+
## Documentation
|
|
97
185
|
|
|
98
186
|
| Doc | Covers |
|
|
99
187
|
|---|---|
|
|
@@ -109,33 +197,48 @@ Every subcommand, flag and the output-language setting: [command reference](http
|
|
|
109
197
|
| [Not a router](https://github.com/rootstudioyaml/sprag/blob/main/docs/NOT-A-ROUTER.md) | Why realtime routing breaks the cache and costs more |
|
|
110
198
|
| [Gateways & environment](https://github.com/rootstudioyaml/sprag/blob/main/docs/GATEWAYS.md) | Bedrock, Vertex, LiteLLM budgets, pricing table, FAQ, how it works |
|
|
111
199
|
|
|
200
|
+
Behind a LiteLLM gateway, `sprag profile-map --refresh` now reads model aliases straight from the gateway's own `GET /model/info`, so route-scan no longer needs to wait on learned votes or a hand-edited `profile-map.json`. The refresh uses the same 5-minute check and 24-hour cache as the LiteLLM budget gauge. It never stores your auth token: each refresh reads it from `apiKeyHelper` at call time and lets it expire on the helper's own schedule. Details: [route-scan](https://github.com/rootstudioyaml/sprag/blob/main/docs/ROUTE_SCAN.md).
|
|
201
|
+
|
|
112
202
|
## Release notes
|
|
113
203
|
|
|
114
204
|
The full history moved to [CHANGELOG.md](./CHANGELOG.md) (Korean; version headings and command names are language-neutral). Recent changes:
|
|
115
205
|
|
|
116
|
-
- **v3.39.0**: `feedback` subcommand
|
|
117
|
-
- **v3.38.0**: `cohesion on`
|
|
206
|
+
- **v3.39.0**: `feedback` subcommand for filing bug reports and feature requests straight from the terminal or a Claude session, via the gh CLI, an anonymous no-login form (auto-filed as a GitHub issue by an Apps Script relay), or a local fallback. `install` now asks before replacing an existing statusline instead of silently skipping.
|
|
207
|
+
- **v3.38.0**: `cohesion on` turns the language-neutral cohesion rules from the Korean supplement into a standalone English injection (given-before-new, one referent per pronoun, subject consistency, bridging, merging choppy sentences). Opt-in, ~0.5k tokens per session, suppressed while `korean on` already carries them.
|
|
118
208
|
- **v3.37.0**: Korean guidance grows a conservative supplement (translationese, AI-writing tics, a research-backed cohesion section whose principles apply to English prose too) and the write-time lint gains 5 translationese patterns, validated at 1 false positive across 255 real files.
|
|
119
|
-
- **v3.35.0**: A `💵 Sep $42` segment now shows estimated spend since 00:00 on the 1st of the current month, always on
|
|
209
|
+
- **v3.35.0**: A `💵 Sep $42` segment now shows estimated spend since 00:00 on the 1st of the current month, always on, including gateway setups with no 5h/7d caps. LiteLLM gateway users get a `🔑 budget ▰▱ 34% $34/$100` gauge built from the key's budget (`GET /key/info` + `GET /user/info`, team-membership budget first, then key, then internal user, verified against a Dockerized LiteLLM).
|
|
120
210
|
- **v3.34.0**: seed presets offered one at a time, output-language choice at install, context warning raised to 500k.
|
|
121
211
|
|
|
122
212
|
## Feedback
|
|
123
213
|
|
|
124
|
-
|
|
214
|
+
Bugs and feature requests go in the issue tracker: https://github.com/rootstudioyaml/sprag/issues
|
|
125
215
|
|
|
126
|
-
|
|
216
|
+
With no browser or GitHub login at hand (corporate network, mid-session), submit straight from the terminal instead, or ask Claude to do it for you:
|
|
127
217
|
|
|
128
218
|
```bash
|
|
129
219
|
sprag feedback "the 5m TTL chip never clears on Bedrock"
|
|
130
220
|
```
|
|
131
221
|
|
|
132
|
-
It files a GitHub issue
|
|
222
|
+
It files a GitHub issue through the `gh` CLI when one is authenticated. Otherwise it submits anonymously, with no login, which works where github.com is blocked. Pass `--anonymous` to skip the `gh` path. Version and OS metadata are attached automatically.
|
|
133
223
|
|
|
134
|
-
|
|
224
|
+
For a bug, please include the tool version (`sprag --version`) and your OS. For a statusline or warning issue, add the statusline output or the `sprag last` result.
|
|
135
225
|
|
|
136
226
|
## License
|
|
137
227
|
|
|
138
|
-
|
|
228
|
+
Apache License 2.0. The full text is in [LICENSE](./LICENSE), and the attributions
|
|
229
|
+
it requires are in [NOTICE](./NOTICE).
|
|
230
|
+
|
|
231
|
+
Releases up to and including v3.47.0 went out under the MIT License. A copy
|
|
232
|
+
received under those terms stays under them; the new license applies to
|
|
233
|
+
releases after that.
|
|
234
|
+
|
|
235
|
+
**Running sprag inside a product or a paid service?** The license already permits
|
|
236
|
+
that, so this is not a request for permission. We would like to hear about it
|
|
237
|
+
anyway, because the parts that get awkward at scale are the parts we would rather
|
|
238
|
+
fix than have you work around: shared ratchet rules across a team, gateway setups
|
|
239
|
+
where the model id is an opaque profile, and per-seat budget reporting. [Open an
|
|
240
|
+
issue](https://github.com/rootstudioyaml/sprag/issues) or reach us through the
|
|
241
|
+
channels below.
|
|
139
242
|
|
|
140
243
|
---
|
|
141
244
|
|
|
@@ -145,4 +248,4 @@ MIT
|
|
|
145
248
|
[](https://www.youtube.com/@DeepPulseEN)
|
|
146
249
|
[](https://rootstudioyaml.github.io/)
|
|
147
250
|
|
|
148
|
-
Built
|
|
251
|
+
Built at **DeepPulse**, a channel about AI developer tooling, and used there every day. The [launch Short (60s)](https://www.youtube.com/shorts/RaD8qMsPTnA) covers where it came from and how it gets used.
|
package/bin/cli.js
CHANGED
|
@@ -87,7 +87,7 @@ function readUpdateChip() {
|
|
|
87
87
|
const KNOWN_SUBCOMMANDS = new Set([
|
|
88
88
|
'last', 'brief', 'history', 'handoff', 'install', 'uninstall', 'mode', 'korean', 'cohesion',
|
|
89
89
|
'doc2md', 'harness', 'route-scan', 'compact-window', 'update-check', 'upgrade',
|
|
90
|
-
'seed', 'litellm-budget', 'feedback',
|
|
90
|
+
'seed', 'litellm-budget', 'profile-map', 'feedback',
|
|
91
91
|
]);
|
|
92
92
|
|
|
93
93
|
const USAGE = `sprag — Claude Code token usage, cache health, and model routing
|
|
@@ -99,6 +99,7 @@ Usage:
|
|
|
99
99
|
sprag --format csv CSV output
|
|
100
100
|
sprag --project myproj filter by project
|
|
101
101
|
sprag route-scan detect recurring easy work → delegation candidates
|
|
102
|
+
sprag profile-map show/refresh the gateway model map (LiteLLM /model/info)
|
|
102
103
|
sprag install set up skill/hooks/statusline
|
|
103
104
|
sprag install --yes take the defaults without asking
|
|
104
105
|
sprag uninstall remove everything install added
|
|
@@ -296,12 +297,22 @@ async function main() {
|
|
|
296
297
|
// sprag litellm-budget --json # 캐시 원본 JSON 출력
|
|
297
298
|
// sprag litellm-budget --refresh # 지금 프록시에 물어봄 (detached 자식이 사용)
|
|
298
299
|
if (args[0] === 'litellm-budget') {
|
|
299
|
-
const { gatewayBase, readBudgetState, refreshBudgetState, formatBudgetReport } =
|
|
300
|
+
const { gatewayBase, readBudgetState, refreshBudgetState, formatBudgetReport, resolveKey } =
|
|
300
301
|
await import('../src/litellm-budget.js');
|
|
301
302
|
const quiet = hasFlag('--quiet');
|
|
302
303
|
if (hasFlag('--refresh')) {
|
|
303
304
|
try {
|
|
304
|
-
|
|
305
|
+
// apiKeyHelper 가 토큰 수명(TTL)을 소유합니다. 헬퍼를 한 번만 불러 그
|
|
306
|
+
// 키를 예산 갱신과 게이트웨이 모델맵 갱신 양쪽에 재사용합니다. 갱신
|
|
307
|
+
// 경로마다 resolveKey() 를 다시 부르면 사본을 만드는 셈이 되고, 헬퍼
|
|
308
|
+
// 호출 횟수도 그만큼 늘어납니다.
|
|
309
|
+
const key = resolveKey();
|
|
310
|
+
let next = null;
|
|
311
|
+
if (key) {
|
|
312
|
+
next = await refreshBudgetState(process.env, fetch, key);
|
|
313
|
+
const { maybeRefreshGatewayModelMap } = await import('../src/litellm-models.js');
|
|
314
|
+
await maybeRefreshGatewayModelMap({ base: gatewayBase(), key });
|
|
315
|
+
}
|
|
305
316
|
if (!quiet) console.log(JSON.stringify(next, null, 2));
|
|
306
317
|
} catch (e) {
|
|
307
318
|
debug('litellm-budget:refresh', e);
|
|
@@ -334,6 +345,15 @@ async function main() {
|
|
|
334
345
|
return;
|
|
335
346
|
}
|
|
336
347
|
|
|
348
|
+
// Subcommand: profile-map · LiteLLM 게이트웨이의 프로파일 ID → 모델 별칭
|
|
349
|
+
// 매핑을 보여 주거나(GET /model/info) 갱신합니다.
|
|
350
|
+
// sprag profile-map # 캐시된 게이트웨이 모델맵 출력
|
|
351
|
+
// sprag profile-map --json # 캐시 원본 JSON 출력
|
|
352
|
+
// sprag profile-map --refresh # 지금 프록시에 물어봄 (사람이 직접 실행)
|
|
353
|
+
if (args[0] === 'profile-map') {
|
|
354
|
+
return (await import('../src/commands/profile-map.js')).run({ args, hasFlag });
|
|
355
|
+
}
|
|
356
|
+
|
|
337
357
|
// Subcommand: upgrade — run the install command that matches how this copy
|
|
338
358
|
// got here, then confirm the new version.
|
|
339
359
|
// sprag upgrade # install the latest release
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sprag-cli",
|
|
3
|
-
"version": "3.
|
|
3
|
+
"version": "3.48.0",
|
|
4
4
|
"description": "Sprag - a quality ratchet harness for AI coding agents: model-fitting delegation, ratchet rules, doc2md, Korean style gate. Same tool as claude-token-saver, under its new name.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -22,7 +22,8 @@
|
|
|
22
22
|
"examples/",
|
|
23
23
|
"README.md",
|
|
24
24
|
"README.ko.md",
|
|
25
|
-
"LICENSE"
|
|
25
|
+
"LICENSE",
|
|
26
|
+
"NOTICE"
|
|
26
27
|
],
|
|
27
28
|
"engines": {
|
|
28
29
|
"node": ">=18"
|
|
@@ -52,7 +53,7 @@
|
|
|
52
53
|
"model-fitting",
|
|
53
54
|
"agent-harness"
|
|
54
55
|
],
|
|
55
|
-
"license": "
|
|
56
|
+
"license": "Apache-2.0",
|
|
56
57
|
"repository": {
|
|
57
58
|
"type": "git",
|
|
58
59
|
"url": "git+https://github.com/rootstudioyaml/sprag.git"
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Subcommand: profile-map. LiteLLM 게이트웨이의 프로파일 ID → 모델 계열 매핑을
|
|
3
|
+
* 보여 주거나 갱신합니다.
|
|
4
|
+
*
|
|
5
|
+
* sprag profile-map # 캐시된 매핑을 사람이 읽을 수 있게 출력
|
|
6
|
+
* sprag profile-map --json # profile-map.json 원본을 그대로 출력
|
|
7
|
+
* sprag profile-map --refresh # LiteLLM GET /model/info 를 지금 호출해 캐시를 갱신
|
|
8
|
+
* sprag profile-map --quiet # 위 어느 모드든 출력을 억제 (스크립트/자식 프로세스용)
|
|
9
|
+
*
|
|
10
|
+
* 왜 이 명령이 필요한가: model-alias.js 의 학습 경로(투표)는 서브에이전트
|
|
11
|
+
* 실행이 MIN_VOTES 건 이상 쌓여야 판정이 서므로, 새 프로파일 ID 나 사용량이
|
|
12
|
+
* 적은 티어는 오랫동안(또는 영원히) role=null 로 남는다. 반면 LiteLLM
|
|
13
|
+
* `/model/info` 는 게이트웨이가 이미 알고 있는 배포 목록을 즉시 돌려주므로,
|
|
14
|
+
* 이 명령으로 그 매핑을 캐시해 두면 학습을 기다릴 필요가 없다. 실제 판정은
|
|
15
|
+
* model-alias.js 의 resolveModelAlias() 가 하고, 이 명령은 그 판정에 쓰이는
|
|
16
|
+
* 세 출처(override/gateway/learned)를 사람이 확인할 수 있게 보여 줄 뿐이다.
|
|
17
|
+
*
|
|
18
|
+
* 왜 --refresh 에서만 resolveKey() 를 부르는가: apiKeyHelper 가 인증 토큰의
|
|
19
|
+
* 수명(TTL)을 소유한다. 이 명령의 읽기 경로(플래그 없음/--json)는 캐시 파일만
|
|
20
|
+
* 보므로 키가 전혀 필요 없고, --refresh 는 사람이 터미널에서 직접 실행하는
|
|
21
|
+
* 경로라서 헬퍼가 Okta 로그인 창을 띄우더라도 괜찮다(사용자 지시). 얻은 키는
|
|
22
|
+
* fetch 요청 헤더에만 쓰고, 화면에 출력하거나 파일에 쓰지 않는다.
|
|
23
|
+
*/
|
|
24
|
+
|
|
25
|
+
export async function run({ args, hasFlag }) {
|
|
26
|
+
const { userLanguage } = await import('../config.js');
|
|
27
|
+
const { loadProfileMap, aliasForRole, MIN_VOTES } = await import('../model-alias.js');
|
|
28
|
+
const { gatewayBase, resolveKey } = await import('../gateway-auth.js');
|
|
29
|
+
const { readGatewayModelMap, refreshGatewayModelMap } = await import('../litellm-models.js');
|
|
30
|
+
|
|
31
|
+
const lang = userLanguage();
|
|
32
|
+
const ko = lang === 'ko';
|
|
33
|
+
const quiet = hasFlag('--quiet');
|
|
34
|
+
const say = (line) => { if (!quiet) console.log(line); };
|
|
35
|
+
|
|
36
|
+
const base = gatewayBase();
|
|
37
|
+
if (!base) {
|
|
38
|
+
// 오류가 아니다: 공식 API 를 직접 쓰는 사용자에게는 이 명령이 원래 할 일이
|
|
39
|
+
// 없다. --refresh 와 --json 도 base 없이는 의미가 없으므로 여기서 함께
|
|
40
|
+
// 걸러낸다.
|
|
41
|
+
say(ko
|
|
42
|
+
? '게이트웨이가 감지되지 않았습니다 (ANTHROPIC_BASE_URL 필요). LiteLLM 등 게이트웨이 뒤에서 실행할 때만 쓰는 명령입니다.'
|
|
43
|
+
: 'No gateway detected (ANTHROPIC_BASE_URL required). This command only matters behind a gateway such as LiteLLM.');
|
|
44
|
+
return;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
if (hasFlag('--refresh')) {
|
|
48
|
+
const key = resolveKey();
|
|
49
|
+
if (!key) {
|
|
50
|
+
say(ko
|
|
51
|
+
? 'apiKeyHelper 가 토큰을 내주지 못했습니다. 인증을 마친 뒤 다시 실행하십시오.'
|
|
52
|
+
: 'apiKeyHelper did not return a token. Authenticate, then run this again.');
|
|
53
|
+
process.exitCode = 1;
|
|
54
|
+
return;
|
|
55
|
+
}
|
|
56
|
+
const result = await refreshGatewayModelMap({ base, key });
|
|
57
|
+
if (!result) {
|
|
58
|
+
say(ko
|
|
59
|
+
? `LiteLLM 에서 모델 정보를 가져오지 못했습니다 (GET ${base}/model/info). CTS_DEBUG=1 로 다시 실행하면 원인이 보입니다.`
|
|
60
|
+
: `Could not fetch model info from LiteLLM (GET ${base}/model/info). Re-run with CTS_DEBUG=1 to see why.`);
|
|
61
|
+
process.exitCode = 1;
|
|
62
|
+
return;
|
|
63
|
+
}
|
|
64
|
+
const resolved = Object.keys(result.aliases).length;
|
|
65
|
+
say(ko
|
|
66
|
+
? `배포 ${result.count}건 중 ${resolved}개 키를 해석했습니다${result.skipped.length ? ` (미해석 ${result.skipped.length}건)` : ''}.`
|
|
67
|
+
: `Resolved ${resolved} key(s) out of ${result.count} deployment(s)${result.skipped.length ? ` (${result.skipped.length} unresolved)` : ''}.`);
|
|
68
|
+
if (result.skipped.length) {
|
|
69
|
+
say(ko
|
|
70
|
+
? ` 미해석 model_name: ${result.skipped.join(', ')}`
|
|
71
|
+
: ` Unresolved model_name: ${result.skipped.join(', ')}`);
|
|
72
|
+
}
|
|
73
|
+
return;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
const map = loadProfileMap();
|
|
77
|
+
const gatewayMap = readGatewayModelMap();
|
|
78
|
+
|
|
79
|
+
if (hasFlag('--json')) {
|
|
80
|
+
say(JSON.stringify(map, null, 2));
|
|
81
|
+
return;
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
say(ko ? `게이트웨이: ${base}` : `Gateway: ${base}`);
|
|
85
|
+
|
|
86
|
+
if (gatewayMap) {
|
|
87
|
+
const fetchedMs = Date.parse(gatewayMap.fetchedAt);
|
|
88
|
+
const ageMin = Number.isFinite(fetchedMs) ? Math.max(0, Math.round((Date.now() - fetchedMs) / 60000)) : null;
|
|
89
|
+
const resolved = Object.keys(gatewayMap.aliases || {}).length;
|
|
90
|
+
say(ko
|
|
91
|
+
? `조회 ${ageMin === null ? '알 수 없음' : `${ageMin}분 전`} · 배포 ${gatewayMap.count}건 · 해석된 키 ${resolved}개`
|
|
92
|
+
: `checked ${ageMin === null ? 'unknown' : `${ageMin}m ago`} · ${gatewayMap.count} deployment(s) · ${resolved} key(s) resolved`);
|
|
93
|
+
} else {
|
|
94
|
+
say(ko
|
|
95
|
+
? '게이트웨이 모델맵이 아직 없습니다. `sprag profile-map --refresh` 로 지금 가져오십시오.'
|
|
96
|
+
: 'No gateway model map cached yet. Fetch it now with `sprag profile-map --refresh`.');
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
const overrideEntries = Object.entries(map.modelAliases || {});
|
|
100
|
+
if (overrideEntries.length) {
|
|
101
|
+
say(ko ? '\n사용자 매핑 (modelAliases):' : '\nUser overrides (modelAliases):');
|
|
102
|
+
for (const [pattern, alias] of overrideEntries) {
|
|
103
|
+
say(` ${pattern} → ${alias} (override)`);
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
const gwAliases = gatewayMap?.aliases || {};
|
|
108
|
+
const learned = map.learned || {};
|
|
109
|
+
// Set 으로 합치는 이유: 같은 프로파일 ID 가 두 출처 모두에 나타날 수 있고,
|
|
110
|
+
// 그때 서로 다른 답을 준다면 그 자체가 사용자에게 알려야 할 신호다(학습이
|
|
111
|
+
// 틀렸거나 게이트웨이 설정이 바뀌었다는 뜻).
|
|
112
|
+
const keys = [...new Set([...Object.keys(gwAliases), ...Object.keys(learned)])].sort();
|
|
113
|
+
|
|
114
|
+
if (keys.length) {
|
|
115
|
+
say(ko ? '\n게이트웨이 · 학습 매핑:' : '\nGateway & learned mappings:');
|
|
116
|
+
for (const key of keys) {
|
|
117
|
+
const gwAlias = gwAliases[key];
|
|
118
|
+
const entry = learned[key];
|
|
119
|
+
|
|
120
|
+
if (gwAlias) {
|
|
121
|
+
say(` ${key} → ${gwAlias} (gateway)`);
|
|
122
|
+
if (entry?.role) {
|
|
123
|
+
const learnedAlias = aliasForRole(entry.role);
|
|
124
|
+
if (learnedAlias && learnedAlias !== gwAlias) {
|
|
125
|
+
say(ko
|
|
126
|
+
? ` ⚠ 학습 결과는 ${learnedAlias} 였습니다. 학습이 틀렸거나 게이트웨이 설정이 바뀌었을 수 있습니다.`
|
|
127
|
+
: ` ⚠ the learned result was ${learnedAlias}. Either the learning was wrong, or the gateway config changed.`);
|
|
128
|
+
}
|
|
129
|
+
} else if (entry && !entry.role) {
|
|
130
|
+
say(ko
|
|
131
|
+
? ` 학습은 표 ${entry.total || 0}개로 판정 대기였지만, 게이트웨이가 이미 해석했으므로 더 이상 문제되지 않습니다.`
|
|
132
|
+
: ` Learning was still undecided at ${entry.total || 0} vote(s), but the gateway already resolves this, so it is no longer an issue.`);
|
|
133
|
+
}
|
|
134
|
+
continue;
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
if (entry?.role) {
|
|
138
|
+
const learnedAlias = aliasForRole(entry.role);
|
|
139
|
+
say(learnedAlias
|
|
140
|
+
? ` ${key} → ${learnedAlias} (learned · ${entry.role})`
|
|
141
|
+
: (ko ? ` ${key} → ${entry.role} 역할 · 별칭 미확정 (learned)` : ` ${key} → ${entry.role} role · alias undecided (learned)`));
|
|
142
|
+
} else if (entry) {
|
|
143
|
+
say(ko
|
|
144
|
+
? ` ${key} → 표 ${entry.total || 0}개 · 판정 대기 (기준 ${MIN_VOTES}개, learned)`
|
|
145
|
+
: ` ${key} → ${entry.total || 0} vote(s) · undecided (needs ${MIN_VOTES}, learned)`);
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
if (!overrideEntries.length && !keys.length) {
|
|
151
|
+
say(ko
|
|
152
|
+
? '\n등록된 매핑이 없습니다.'
|
|
153
|
+
: '\nNo mappings recorded yet.');
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
if (gatewayMap?.skipped?.length) {
|
|
157
|
+
say(ko
|
|
158
|
+
? `\n게이트웨이가 계열명을 알아내지 못한 배포: ${gatewayMap.skipped.join(', ')}.`
|
|
159
|
+
: `\nDeployments the gateway could not name a family for: ${gatewayMap.skipped.join(', ')}.`);
|
|
160
|
+
say(ko
|
|
161
|
+
? ' profile-map.json 의 modelAliases 에 해당 이름을 직접 매핑하십시오.'
|
|
162
|
+
: ' Map those names directly under modelAliases in profile-map.json.');
|
|
163
|
+
}
|
|
164
|
+
}
|
|
@@ -183,15 +183,19 @@ export async function run({ args, hasFlag, numArg }) {
|
|
|
183
183
|
const { MIN_VOTES } = await import('../model-alias.js');
|
|
184
184
|
const learning = scan.unresolvedRuns < MIN_VOTES;
|
|
185
185
|
const remedyKo = learning
|
|
186
|
-
? `\n 위임이 ${MIN_VOTES}건 이상 쌓이면 자동으로 해석될 수 있습니다. 그때까지
|
|
187
|
-
+ '\n
|
|
188
|
-
+ '\n
|
|
189
|
-
|
|
186
|
+
? `\n 위임이 ${MIN_VOTES}건 이상 쌓이면 자동으로 해석될 수 있습니다. 그때까지 기다리는 동안`
|
|
187
|
+
+ '\n sprag profile-map --refresh 를 먼저 실행해 보십시오. LiteLLM 게이트웨이라면 배포 목록에서'
|
|
188
|
+
+ '\n 매핑을 곧바로 가져옵니다. 그래도 남는 ID 는 profile-map.json 의 modelAliases 에 직접'
|
|
189
|
+
+ '\n 매핑한 뒤 route-scan --refresh 를 실행하십시오.'
|
|
190
|
+
: '\n sprag profile-map --refresh 를 먼저 실행해 보십시오. LiteLLM 게이트웨이라면 배포 목록에서 매핑을 곧바로 가져옵니다.'
|
|
191
|
+
+ '\n 그래도 남는 ID 는 profile-map.json 의 modelAliases 에 직접 매핑한 뒤 route-scan --refresh 를 실행하십시오.';
|
|
190
192
|
const remedyEn = learning
|
|
191
|
-
? `\n These may resolve on their own once ${MIN_VOTES} or more delegated runs accumulate
|
|
192
|
-
+ '\n
|
|
193
|
+
? `\n These may resolve on their own once ${MIN_VOTES} or more delegated runs accumulate; check again then.`
|
|
194
|
+
+ '\n Meanwhile, try sprag profile-map --refresh first: on a LiteLLM gateway it pulls the mapping'
|
|
195
|
+
+ '\n straight from the deployment list. Anything still left over, map it under modelAliases in'
|
|
193
196
|
+ '\n profile-map.json and run route-scan --refresh.'
|
|
194
|
-
: '\n
|
|
197
|
+
: '\n Try sprag profile-map --refresh first: on a LiteLLM gateway it pulls the mapping straight from the deployment list.'
|
|
198
|
+
+ '\n Anything still left over, map it under modelAliases in profile-map.json and run route-scan --refresh.';
|
|
195
199
|
console.log(lang === 'ko'
|
|
196
200
|
? `\n⚠ 해석되지 않은 모델 ID 때문에 위임 ${scan.unresolvedRuns}건이 집계에서 제외됐습니다${ids ? ` (${ids})` : ''}.` + remedyKo
|
|
197
201
|
: `\n⚠ ${scan.unresolvedRuns} delegated run(s) were excluded — unpriceable model id${ids ? ` (${ids})` : ''}.` + remedyEn);
|