@bushizhanghan/dsh-context-budget 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +98 -2
- package/cordis.patch.yml +4 -0
- package/lib/engine.js +590 -0
- package/lib/index.js +300 -0
- package/package.json +49 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 bushizhanghan
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,99 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-context-budget — 上下文预算审计(DSH 插件)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
> 每一轮对话,模型都被重新喂一遍同一堆东西:system prompt、AGENTS.md、60+ 个技能的目录、几十个工具的 schema、运行时快照。
|
|
4
|
+
> 它们逐字节相同,**模型不需要再知道一遍**。这个插件就是来量这件事的。
|
|
5
|
+
|
|
6
|
+
在真实机器的会话日志上实测:一个 34 轮的会话,注入上下文 **256k token,其中 37.4%(95.8k)是逐字节重复的内容**。
|
|
7
|
+
system prompt 被重发 6 遍(16.8k token 白付),skill catalog 被重发 3 遍(12.7k token 白付)。
|
|
8
|
+
这不是压缩能解决的问题——压缩省的是历史,这是**固定开销在复利**。
|
|
9
|
+
|
|
10
|
+
## 安装
|
|
11
|
+
|
|
12
|
+
```sh
|
|
13
|
+
dsh plugin --profile desktop add @bushizhanghan/dsh-context-budget
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
重启 DSH 后生效。**只读**:从不写你的会话日志,也不调 LLM——每个数字都能从日志里手算复核。
|
|
17
|
+
|
|
18
|
+
## Agent 工具
|
|
19
|
+
|
|
20
|
+
| 工具 | 作用 |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `ctx_budget` | 审计单个会话:每轮注入构成(system / AGENTS.md / skill catalog / tools / runtime / team / history),以及逐字节重复的纯浪费 |
|
|
23
|
+
| `ctx_budget_sweep` | 扫描所有会话,按浪费量排序,看哪段对话的固定开销最重 |
|
|
24
|
+
| `ctx_budget_skills` | 技能目录治理:每轮要为 60+ 个技能的描述付多少 token,哪些该懒加载 |
|
|
25
|
+
| `ctx_budget_tools` | 工具 schema 膨胀:哪个插件的工具定义每轮占用最多上下文 |
|
|
26
|
+
|
|
27
|
+
## 分类口径
|
|
28
|
+
|
|
29
|
+
| 类别 | 来源 |
|
|
30
|
+
|---|---|
|
|
31
|
+
| `system` | `system/message`:组装好的 system prompt,每轮重建 |
|
|
32
|
+
| `agents` | `user/message` + `source.kind = agent-instructions`:AGENTS.md 等工作区指令 |
|
|
33
|
+
| `skills` | `user/message` + `source.kind = skill-catalog`:`<system-reminder>` 里的技能目录 |
|
|
34
|
+
| `tools` | `request/header.data.header.tools`:实际发出的工具 schema 数组 |
|
|
35
|
+
| `runtime` | `runtime-context` / `time-context` / `model-selection` |
|
|
36
|
+
| `team` | `agent-teams` / `agent-message` / `subagent-settled` |
|
|
37
|
+
| `history` | `user` / `compact-checkpoint`:真实对话与压缩检查点 |
|
|
38
|
+
| `other` | 未识别的新注入面——**不静默丢弃**,这样未来 DSH 新增注入面会自己冒出来 |
|
|
39
|
+
|
|
40
|
+
## 浪费判定
|
|
41
|
+
|
|
42
|
+
一个块在第 N 轮和第 N+k 轮签名相同(长度 + 头 32 字符 + 尾 32 字符)即为重复。
|
|
43
|
+
**第一次出现不是浪费**(模型总得看一遍),之后每次都是。历史不计入浪费——它是合法累积的。
|
|
44
|
+
|
|
45
|
+
## 它在真实数据上发现了什么
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
context budget · session-88fbc2a8…: 34 turns, 68 request(s)
|
|
49
|
+
injected context total (est): 256079 tok
|
|
50
|
+
repeated unchanged content: 95792 tok (37.4% of injected)
|
|
51
|
+
|
|
52
|
+
composition (est tokens, summed over turns)
|
|
53
|
+
system 105034 (33 block(s))
|
|
54
|
+
skills 63633 (16 block(s))
|
|
55
|
+
history 36098 (170 block(s))
|
|
56
|
+
tools 19486 (1 block(s))
|
|
57
|
+
team 18320 (40 block(s))
|
|
58
|
+
runtime 10765 (135 block(s))
|
|
59
|
+
agents 2355 (3 block(s))
|
|
60
|
+
|
|
61
|
+
tool schema inflation (est tokens per turn)
|
|
62
|
+
plugin:agent-teams 5170 (18 tool(s))
|
|
63
|
+
tool:code_review 2325 (6 tool(s))
|
|
64
|
+
mcp:wincu 2284 (10 tool(s))
|
|
65
|
+
|
|
66
|
+
skill catalog: 59 skills, 5022 est tok per turn
|
|
67
|
+
academic-research-suite 133 tok used 0x
|
|
68
|
+
scientific-toolkit-skill 133 tok used 0x
|
|
69
|
+
wechat-miniprogram-design 133 tok used 0x
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
三条可执行的结论:
|
|
73
|
+
|
|
74
|
+
1. **AgentTeams 插件 18 个工具 = 5170 token/轮**,是最大的单一固定开销;其中多数在非团队会话里根本用不到。
|
|
75
|
+
2. **59 个技能目录 = 5022 token/轮**,而实测整台机器被调用过的技能只有 7 个。最贵的 10 个技能从未被调用,却每轮全价付费。
|
|
76
|
+
3. **system prompt 每轮 3370 token**,跨轮几乎不变——唯一真正在变的是 GUI URL 和工作目录两行。
|
|
77
|
+
|
|
78
|
+
## 设计取舍
|
|
79
|
+
|
|
80
|
+
- **先读日志再写规则。** 第一版假设会话日志是 UTF-8,在真实机器上 94 个会话全部解析失败——GUI 启动的会话是 **UTF-16LE 带 BOM**。现在按 BOM 嗅探,两种编码都见过。
|
|
81
|
+
- **目录层级是 `<workspace>/session-<uuid>/`**,根和会话目录之间隔着一层 workspace,平铺扫描根目录什么都找不到。向下走两层。
|
|
82
|
+
- **CJK 按 1 字符 = 1 token 算。** 这台机器上的会话以中文为主,按 4 字符 = 1 token 会把中文提示低估约 4 倍。
|
|
83
|
+
- **`estimateTokens` 是估计值,报告里一律标 `est`。** 这个插件要的是**比例**——同一除数在比值里抵消。需要精确数字时用 `request/context` 的 `contextWindow` 做分母。
|
|
84
|
+
- **懒加载规则故意保守**:只标记"本轮从未调用且描述 ≥ 60 token"的技能。用过一次的技能永不标记;没有使用信号的技能不猜测它已死(它可能在别的会话里被用)。
|
|
85
|
+
- **未识别的 `source.kind` 归入 `other` 而不是丢弃**,否则未来 DSH 新增注入面会让审计数字悄悄变小而没人发现。
|
|
86
|
+
|
|
87
|
+
## 测试
|
|
88
|
+
|
|
89
|
+
```sh
|
|
90
|
+
node --test test/
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
`test/engine.test.mjs` 一条规则一个测试(token 估计、CJK、分类映射、签名、重复判定、技能治理、工具分组、空会话不除零)。
|
|
94
|
+
`test/integration.test.mjs` 用假 `ctx = { tools: { register } }` 启动插件,驱动全部 4 个工具打真实会话日志,
|
|
95
|
+
并断言反向情况:不存在的会话/根目录必须返回 `ok: false`。
|
|
96
|
+
|
|
97
|
+
## License
|
|
98
|
+
|
|
99
|
+
MIT
|
package/cordis.patch.yml
ADDED
package/lib/engine.js
ADDED
|
@@ -0,0 +1,590 @@
|
|
|
1
|
+
// dsh-context-budget engine.
|
|
2
|
+
//
|
|
3
|
+
// The question this engine answers: of the context a model is handed on each
|
|
4
|
+
// turn, how much of it does the model not need to know?
|
|
5
|
+
//
|
|
6
|
+
// Established by reading real session logs on this machine, not by guessing:
|
|
7
|
+
//
|
|
8
|
+
// 1. A DSH session log (session.v4.jsonl.zstd) is UTF-16LE when the session
|
|
9
|
+
// was started by the GUI and UTF-8 otherwise. Both shapes exist in the
|
|
10
|
+
// same sessions tree, so the loader sniffs the BOM instead of assuming.
|
|
11
|
+
// 2. Per-turn injected context is spread over four event kinds:
|
|
12
|
+
// - system/message the assembled system prompt (identity,
|
|
13
|
+
// team protocol, tool-usage rules, runtime
|
|
14
|
+
// facts). Rebuilt every turn.
|
|
15
|
+
// - user/message injected instruction blocks, tagged by
|
|
16
|
+
// data.source.kind: agent-instructions
|
|
17
|
+
// (AGENTS.md), skill-catalog, runtime-context,
|
|
18
|
+
// time-context, agent-teams, model-selection, …
|
|
19
|
+
// - request/header the tool schema array actually sent, with a
|
|
20
|
+
// data.reason (initial / change / resume).
|
|
21
|
+
// - request/context provider/model/contextWindow, for the
|
|
22
|
+
// denominator of the budget.
|
|
23
|
+
// 3. History (assistant/tool output) grows per turn; everything above is
|
|
24
|
+
// either rebuilt or resent byte-identically. So the audit measures, per
|
|
25
|
+
// turn: composition (how the budget splits), repetition (the same bytes
|
|
26
|
+
// resent), and provenance (which surface owns each byte).
|
|
27
|
+
//
|
|
28
|
+
// Pure functions, no ctx, no I/O: every function here takes data it was handed.
|
|
29
|
+
// The session reader lives in engine.js too because it has no host
|
|
30
|
+
// dependencies beyond `zstd` on PATH, which is the same dependency the other
|
|
31
|
+
// audit plugins in this profile use.
|
|
32
|
+
import { execFileSync } from 'node:child_process'
|
|
33
|
+
import { existsSync } from 'node:fs'
|
|
34
|
+
|
|
35
|
+
// =====================================================================
|
|
36
|
+
// Session log reading
|
|
37
|
+
// =====================================================================
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* Load a session.v4.jsonl.zstd into ordered events.
|
|
41
|
+
*
|
|
42
|
+
* Encoding is sniffed, not assumed: GUI-started sessions are UTF-16LE with a
|
|
43
|
+
* BOM, CLI-started ones are plain UTF-8. Reading a UTF-16LE file as UTF-8
|
|
44
|
+
* yields a file where every JSON line fails to parse — which is exactly the
|
|
45
|
+
* bug this sniff prevents.
|
|
46
|
+
*/
|
|
47
|
+
export function loadSession(file) {
|
|
48
|
+
if (!file || !existsSync(file)) return { events: [], error: `not found: ${file}` }
|
|
49
|
+
let text
|
|
50
|
+
try {
|
|
51
|
+
const buf = execFileSync('zstd', ['-dc', file], { maxBuffer: 512 * 1024 * 1024 })
|
|
52
|
+
text = buf[0] === 0xff && buf[1] === 0xfe ? buf.toString('utf16le').slice(1) : buf.toString('utf8')
|
|
53
|
+
} catch (e) {
|
|
54
|
+
return { events: [], error: `cannot decompress (is zstd installed?): ${String(e.message).slice(0, 120)}` }
|
|
55
|
+
}
|
|
56
|
+
const events = []
|
|
57
|
+
for (const line of text.split('\n')) {
|
|
58
|
+
if (!line.trim()) continue
|
|
59
|
+
try {
|
|
60
|
+
events.push(JSON.parse(line))
|
|
61
|
+
} catch {}
|
|
62
|
+
}
|
|
63
|
+
events.sort((a, b) => (a.seq ?? 0) - (b.seq ?? 0))
|
|
64
|
+
return { events, error: null }
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
// =====================================================================
|
|
68
|
+
// Token estimation
|
|
69
|
+
// =====================================================================
|
|
70
|
+
|
|
71
|
+
/**
|
|
72
|
+
* Rough token count for a piece of text.
|
|
73
|
+
*
|
|
74
|
+
* Deliberately simple and monotone: 1 token ≈ 4 characters. It is an estimate,
|
|
75
|
+
* so every report that uses it says "est" — the point of this plugin is the
|
|
76
|
+
* *ratio* between surfaces and turns, and a constant divisor cancels out of a
|
|
77
|
+
* ratio. The one real-world correction is CJK, where a single character is a
|
|
78
|
+
* token on its own: counting CJK as 4 chars would understate Chinese-heavy
|
|
79
|
+
* prompts (this machine's sessions are mostly Chinese) by ~4x.
|
|
80
|
+
*/
|
|
81
|
+
export function estimateTokens(text) {
|
|
82
|
+
if (!text) return 0
|
|
83
|
+
const s = String(text)
|
|
84
|
+
let cjk = 0
|
|
85
|
+
for (const ch of s) {
|
|
86
|
+
const c = ch.codePointAt(0)
|
|
87
|
+
if ((c >= 0x2e80 && c <= 0x9fff) || (c >= 0xf900 && c <= 0xfaff) || (c >= 0xff00 && c <= 0xffef) || (c >= 0x3000 && c <= 0x303f)) cjk += 1
|
|
88
|
+
}
|
|
89
|
+
return Math.max(1, Math.ceil((s.length - cjk) / 4 + cjk))
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/** Token estimate for a measured character count of ASCII-ish text (JSON schemas, ASCII lines). */
|
|
93
|
+
export function tokensForChars(chars) {
|
|
94
|
+
const n = Number(chars)
|
|
95
|
+
if (!Number.isFinite(n) || n <= 0) return 0
|
|
96
|
+
return Math.ceil(n / 4)
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
// =====================================================================
|
|
100
|
+
// Classification: which surface owns a block of injected text
|
|
101
|
+
// =====================================================================
|
|
102
|
+
|
|
103
|
+
// Category constants. Order matters only for stable report output.
|
|
104
|
+
export const CATEGORIES = {
|
|
105
|
+
SYSTEM: 'system',
|
|
106
|
+
AGENTS: 'agents',
|
|
107
|
+
SKILLS: 'skills',
|
|
108
|
+
TOOLS: 'tools',
|
|
109
|
+
RUNTIME: 'runtime',
|
|
110
|
+
TEAM: 'team',
|
|
111
|
+
HISTORY: 'history',
|
|
112
|
+
OTHER: 'other',
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
/** user/message source.kind → category. Anything unlisted lands in OTHER. */
|
|
116
|
+
const USER_KIND_CATEGORY = {
|
|
117
|
+
'agent-instructions': CATEGORIES.AGENTS,
|
|
118
|
+
'skill-catalog': CATEGORIES.SKILLS,
|
|
119
|
+
'runtime-context': CATEGORIES.RUNTIME,
|
|
120
|
+
'time-context': CATEGORIES.RUNTIME,
|
|
121
|
+
'agent-teams': CATEGORIES.TEAM,
|
|
122
|
+
'agent-message': CATEGORIES.TEAM,
|
|
123
|
+
'subagent-settled': CATEGORIES.TEAM,
|
|
124
|
+
'model-selection': CATEGORIES.RUNTIME,
|
|
125
|
+
user: CATEGORIES.HISTORY,
|
|
126
|
+
goal: CATEGORIES.OTHER,
|
|
127
|
+
'tool-jobs': CATEGORIES.OTHER,
|
|
128
|
+
'dsh-rewind': CATEGORIES.OTHER,
|
|
129
|
+
'user-approval': CATEGORIES.OTHER,
|
|
130
|
+
'compact-checkpoint': CATEGORIES.HISTORY,
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
/** Which user/message kinds are re-injected standing instructions rather than turns of conversation. */
|
|
134
|
+
export function categoryOfUserKind(kind) {
|
|
135
|
+
return USER_KIND_CATEGORY[kind] ?? CATEGORIES.OTHER
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
/** Total character length of a message event's content. */
|
|
139
|
+
export function contentChars(content) {
|
|
140
|
+
if (!Array.isArray(content)) return typeof content === 'string' ? content.length : 0
|
|
141
|
+
let n = 0
|
|
142
|
+
for (const p of content) {
|
|
143
|
+
if (typeof p === 'string') n += p.length
|
|
144
|
+
else if (p && typeof p === 'object' && typeof p.text === 'string') n += p.text.length
|
|
145
|
+
}
|
|
146
|
+
return n
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
/**
|
|
150
|
+
* Joined text of a message event's content.
|
|
151
|
+
*
|
|
152
|
+
* Every part that carries `text` counts, including `reasoning` parts: they are
|
|
153
|
+
* sent to the model, so they spend budget whether or not they are instructions.
|
|
154
|
+
*/
|
|
155
|
+
export function contentText(content) {
|
|
156
|
+
if (!Array.isArray(content)) return typeof content === 'string' ? content : ''
|
|
157
|
+
const out = []
|
|
158
|
+
for (const p of content) {
|
|
159
|
+
if (typeof p === 'string') out.push(p)
|
|
160
|
+
else if (p && typeof p === 'object' && typeof p.text === 'string') out.push(p.text)
|
|
161
|
+
}
|
|
162
|
+
return out.filter((s) => s !== '').join('\n')
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
// =====================================================================
|
|
166
|
+
// Per-turn budget model
|
|
167
|
+
// =====================================================================
|
|
168
|
+
|
|
169
|
+
/**
|
|
170
|
+
* Build the per-turn budget model of a session.
|
|
171
|
+
*
|
|
172
|
+
* For each turn we record the bytes of every injected block, tagged with the
|
|
173
|
+
* category that owns it, plus a stable signature (first 64 chars + length) so
|
|
174
|
+
* two blocks can be compared for byte-identity without keeping the whole text
|
|
175
|
+
* in memory across turns.
|
|
176
|
+
*
|
|
177
|
+
* Returns { turns, categories, requests, toolSets } where `turns` is ordered by
|
|
178
|
+
* turn number.
|
|
179
|
+
*/
|
|
180
|
+
export function buildTurns(events) {
|
|
181
|
+
const turnOf = (ev) => (ev && ev.data && typeof ev.data.turn === 'number' ? ev.data.turn : null)
|
|
182
|
+
const byTurn = new Map()
|
|
183
|
+
const ensure = (n) => {
|
|
184
|
+
if (!byTurn.has(n)) byTurn.set(n, { turn: n, blocks: [], requests: 0, historyChars: 0, historyTokens: 0 })
|
|
185
|
+
return byTurn.get(n)
|
|
186
|
+
}
|
|
187
|
+
const push = (turn, block) => {
|
|
188
|
+
const t = ensure(turn)
|
|
189
|
+
t.blocks.push(block)
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
// system/message carries the assembled system prompt, but only inside its
|
|
193
|
+
// first turn is it new; the same text is resent every turn after. We record
|
|
194
|
+
// it per turn with a signature so the repetition analysis can prove it.
|
|
195
|
+
for (const ev of events) {
|
|
196
|
+
const t = String(ev.type || '')
|
|
197
|
+
const turn = turnOf(ev)
|
|
198
|
+
if (t === 'system/message') {
|
|
199
|
+
const text = contentText(ev.data?.message?.content)
|
|
200
|
+
push(turn ?? 0, {
|
|
201
|
+
category: CATEGORIES.SYSTEM,
|
|
202
|
+
chars: text.length,
|
|
203
|
+
tokens: estimateTokens(text),
|
|
204
|
+
signature: signature(text),
|
|
205
|
+
seq: ev.seq,
|
|
206
|
+
label: 'system prompt',
|
|
207
|
+
})
|
|
208
|
+
} else if (t === 'user/message') {
|
|
209
|
+
const d = ev.data || {}
|
|
210
|
+
const text = contentText(d.content)
|
|
211
|
+
const kind = d.source?.kind ?? 'user'
|
|
212
|
+
const cat = categoryOfUserKind(kind)
|
|
213
|
+
if (turn == null) {
|
|
214
|
+
// pre-turn injection (the first user message of a session)
|
|
215
|
+
push(0, block(kind, cat, text, ev.seq))
|
|
216
|
+
continue
|
|
217
|
+
}
|
|
218
|
+
if (cat === CATEGORIES.HISTORY) {
|
|
219
|
+
const t = ensure(turn)
|
|
220
|
+
t.historyChars = (t.historyChars || 0) + text.length
|
|
221
|
+
t.historyTokens = (t.historyTokens || 0) + estimateTokens(text)
|
|
222
|
+
continue
|
|
223
|
+
}
|
|
224
|
+
push(turn, block(kind, cat, text, ev.seq))
|
|
225
|
+
} else if (t === 'request/header') {
|
|
226
|
+
const tools = ev.data?.header?.tools
|
|
227
|
+
if (Array.isArray(tools)) {
|
|
228
|
+
const t = ensure(turn ?? 0)
|
|
229
|
+
t.requests += 1
|
|
230
|
+
// Keep the last (largest) schema set sent this turn: a request that
|
|
231
|
+
// changed its tools mid-turn is rare, and the newest set is the one
|
|
232
|
+
// the model was actually shown.
|
|
233
|
+
t.toolNames = tools.map((x) => x.name)
|
|
234
|
+
t.toolChars = JSON.stringify(tools).length
|
|
235
|
+
t.toolTokens = estimateTokens(JSON.stringify(tools))
|
|
236
|
+
t.toolSignature = signature(tools.map((x) => `${x.name}:${JSON.stringify(x.parameters ?? {})}`).join('|'))
|
|
237
|
+
t.reason = ev.data.reason ?? null
|
|
238
|
+
}
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
const turns = [...byTurn.values()].sort((a, b) => a.turn - b.turn)
|
|
243
|
+
return turns
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
function block(kind, category, text, seq) {
|
|
247
|
+
return {
|
|
248
|
+
kind,
|
|
249
|
+
category,
|
|
250
|
+
chars: text.length,
|
|
251
|
+
tokens: estimateTokens(text),
|
|
252
|
+
signature: signature(text),
|
|
253
|
+
seq,
|
|
254
|
+
label: labelFor(kind, text),
|
|
255
|
+
}
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
function labelFor(kind, text) {
|
|
259
|
+
if (kind === 'agent-instructions') {
|
|
260
|
+
const m = text.match(/Instructions from:\s*([^\n]+)/)
|
|
261
|
+
return m ? `instructions · ${m[1].trim()}` : 'instructions'
|
|
262
|
+
}
|
|
263
|
+
if (kind === 'skill-catalog') return 'skill catalog'
|
|
264
|
+
if (kind === 'runtime-context') return 'runtime context'
|
|
265
|
+
if (kind === 'time-context') return 'time context'
|
|
266
|
+
if (kind === 'model-selection') return 'model change notice'
|
|
267
|
+
if (kind === 'agent-teams' || kind === 'agent-message') return `team · ${kind}`
|
|
268
|
+
return kind
|
|
269
|
+
}
|
|
270
|
+
|
|
271
|
+
/** Stable short identity of a text: length + head + tail. Cheap and collision-safe enough for repetition detection. */
|
|
272
|
+
export function signature(text) {
|
|
273
|
+
const s = String(text ?? '')
|
|
274
|
+
if (s.length === 0) return 'empty'
|
|
275
|
+
return `${s.length}:${s.slice(0, 32)}|${s.slice(-32)}`
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
/**
|
|
279
|
+
* Aggregate a turn list into a composition table.
|
|
280
|
+
*
|
|
281
|
+
* Returns { categories: { <cat>: {tokens, blocks} }, totalTokens, perTurn }.
|
|
282
|
+
*/
|
|
283
|
+
export function composition(turns) {
|
|
284
|
+
const categories = {}
|
|
285
|
+
let total = 0
|
|
286
|
+
const perTurn = []
|
|
287
|
+
const bump = (cat, tokens) => {
|
|
288
|
+
categories[cat] = categories[cat] || { tokens: 0, blocks: 0 }
|
|
289
|
+
categories[cat].tokens += tokens
|
|
290
|
+
categories[cat].blocks += 1
|
|
291
|
+
}
|
|
292
|
+
for (const t of turns) {
|
|
293
|
+
const row = { turn: t.turn, categories: {}, total: 0 }
|
|
294
|
+
for (const b of t.blocks) {
|
|
295
|
+
row.categories[b.category] = (row.categories[b.category] || 0) + b.tokens
|
|
296
|
+
bump(b.category, b.tokens)
|
|
297
|
+
}
|
|
298
|
+
if (t.toolTokens) {
|
|
299
|
+
row.categories[CATEGORIES.TOOLS] = (row.categories[CATEGORIES.TOOLS] || 0) + t.toolTokens
|
|
300
|
+
bump(CATEGORIES.TOOLS, t.toolTokens)
|
|
301
|
+
}
|
|
302
|
+
if (t.historyTokens) {
|
|
303
|
+
row.categories[CATEGORIES.HISTORY] = (row.categories[CATEGORIES.HISTORY] || 0) + t.historyTokens
|
|
304
|
+
bump(CATEGORIES.HISTORY, t.historyTokens)
|
|
305
|
+
}
|
|
306
|
+
row.total = Object.values(row.categories).reduce((a, b) => a + b, 0)
|
|
307
|
+
total += row.total
|
|
308
|
+
perTurn.push(row)
|
|
309
|
+
}
|
|
310
|
+
return { categories, totalTokens: total, perTurn }
|
|
311
|
+
}
|
|
312
|
+
|
|
313
|
+
// =====================================================================
|
|
314
|
+
// Repetition: the pure-waste detector
|
|
315
|
+
// =====================================================================
|
|
316
|
+
|
|
317
|
+
/**
|
|
318
|
+
* Find injected content that is resent byte-identically.
|
|
319
|
+
*
|
|
320
|
+
* A block that appears in turn N and again in turn N+k with the same signature
|
|
321
|
+
* costs the same tokens again and teaches the model nothing new: that is the
|
|
322
|
+
* pure waste this plugin exists to quantify. The first occurrence is not waste
|
|
323
|
+
* (the model has to see it once); every later occurrence is.
|
|
324
|
+
*
|
|
325
|
+
* Returns { repeated: [{category,label,occurrences,tokensEach,wasteTokens,firstTurn,lastTurn}], wasteTokens, totalTokens }.
|
|
326
|
+
*/
|
|
327
|
+
export function detectRepetition(turns) {
|
|
328
|
+
const seen = new Map()
|
|
329
|
+
const order = []
|
|
330
|
+
for (const t of turns) {
|
|
331
|
+
for (const b of t.blocks) {
|
|
332
|
+
const key = `${b.category}\u0000${b.signature}`
|
|
333
|
+
if (!seen.has(key)) {
|
|
334
|
+
seen.set(key, { category: b.category, label: b.label, tokensEach: b.tokens, turns: [] })
|
|
335
|
+
order.push(key)
|
|
336
|
+
}
|
|
337
|
+
const rec = seen.get(key)
|
|
338
|
+
rec.turns.push(t.turn)
|
|
339
|
+
}
|
|
340
|
+
}
|
|
341
|
+
const repeated = []
|
|
342
|
+
let waste = 0
|
|
343
|
+
let total = 0
|
|
344
|
+
for (const key of order) {
|
|
345
|
+
const rec = seen.get(key)
|
|
346
|
+
total += rec.tokensEach * rec.turns.length
|
|
347
|
+
if (rec.turns.length > 1) {
|
|
348
|
+
const wasteTokens = rec.tokensEach * (rec.turns.length - 1)
|
|
349
|
+
waste += wasteTokens
|
|
350
|
+
repeated.push({
|
|
351
|
+
category: rec.category,
|
|
352
|
+
label: rec.label,
|
|
353
|
+
occurrences: rec.turns.length,
|
|
354
|
+
tokensEach: rec.tokensEach,
|
|
355
|
+
wasteTokens,
|
|
356
|
+
firstTurn: rec.turns[0],
|
|
357
|
+
lastTurn: rec.turns[rec.turns.length - 1],
|
|
358
|
+
})
|
|
359
|
+
}
|
|
360
|
+
}
|
|
361
|
+
repeated.sort((a, b) => b.wasteTokens - a.wasteTokens)
|
|
362
|
+
return { repeated, wasteTokens: waste, totalTokens: total }
|
|
363
|
+
}
|
|
364
|
+
|
|
365
|
+
/**
|
|
366
|
+
* Per-turn waste: how much of a turn's injected context is a repeat of
|
|
367
|
+
* something the model has already been sent.
|
|
368
|
+
*
|
|
369
|
+
* This is the number that matters for "why is this turn expensive": history is
|
|
370
|
+
* legitimately cumulative, but a system prompt, a skill catalog and a tool set
|
|
371
|
+
* resent unchanged are the same bytes paying rent again.
|
|
372
|
+
*/
|
|
373
|
+
export function perTurnWaste(turns) {
|
|
374
|
+
const seen = new Set()
|
|
375
|
+
const out = []
|
|
376
|
+
for (const t of turns) {
|
|
377
|
+
let repeatTokens = 0
|
|
378
|
+
let freshTokens = 0
|
|
379
|
+
for (const b of t.blocks) {
|
|
380
|
+
const key = `${b.category}\u0000${b.signature}`
|
|
381
|
+
if (seen.has(key)) repeatTokens += b.tokens
|
|
382
|
+
else {
|
|
383
|
+
freshTokens += b.tokens
|
|
384
|
+
seen.add(key)
|
|
385
|
+
}
|
|
386
|
+
}
|
|
387
|
+
out.push({ turn: t.turn, repeatTokens, freshTokens, requests: t.requests })
|
|
388
|
+
}
|
|
389
|
+
return out
|
|
390
|
+
}
|
|
391
|
+
|
|
392
|
+
// =====================================================================
|
|
393
|
+
// Skill catalog
|
|
394
|
+
// =====================================================================
|
|
395
|
+
|
|
396
|
+
/** One skill line in a skill-catalog block: `- \`name\`: description`. */
|
|
397
|
+
export function parseSkillCatalog(text) {
|
|
398
|
+
if (!text) return []
|
|
399
|
+
const out = []
|
|
400
|
+
for (const line of String(text).split('\n')) {
|
|
401
|
+
const m = line.match(/^- `([^`]+)`:\s*(.*)$/)
|
|
402
|
+
if (!m) continue
|
|
403
|
+
out.push({ name: m[1], description: m[2], chars: line.length })
|
|
404
|
+
}
|
|
405
|
+
return out
|
|
406
|
+
}
|
|
407
|
+
|
|
408
|
+
/**
|
|
409
|
+
* Skill-catalog governance.
|
|
410
|
+
*
|
|
411
|
+
* The catalog is injected as one block per turn, so every installed skill's
|
|
412
|
+
* description is paid for on every turn whether or not it is ever used. This
|
|
413
|
+
* measures that standing cost and ranks skills by description length, because
|
|
414
|
+
* length is the only signal available without an LLM: a 500-character
|
|
415
|
+
* description for a skill that is invoked once a month is a bad trade.
|
|
416
|
+
*
|
|
417
|
+
* `usage` maps skill name → number of `skill` tool calls observed in the
|
|
418
|
+
* session, so the report can separate "big and used" from "big and dead".
|
|
419
|
+
*/
|
|
420
|
+
/**
|
|
421
|
+
* Rank skills by the tokens their description costs on every turn.
|
|
422
|
+
*
|
|
423
|
+
* `skills` entries carry either `chars` (a measured line length, as produced by
|
|
424
|
+
* parseSkillCatalog) or just `description` (as supplied by a caller that has
|
|
425
|
+
* the catalog but not the line), so the cost is computed from whichever is
|
|
426
|
+
* present. `usage` maps skill name → number of `skill` tool calls observed, so
|
|
427
|
+
* the report can separate "big and used" from "big and dead".
|
|
428
|
+
*/
|
|
429
|
+
export function rankSkills(skills, usage = {}) {
|
|
430
|
+
return skills
|
|
431
|
+
.map((s) => {
|
|
432
|
+
const chars = typeof s.chars === 'number' ? s.chars : String(s.description ?? '').length
|
|
433
|
+
return {
|
|
434
|
+
name: s.name,
|
|
435
|
+
chars,
|
|
436
|
+
tokensPerTurn: tokensForChars(chars),
|
|
437
|
+
uses: Number(usage[s.name] || 0),
|
|
438
|
+
}
|
|
439
|
+
})
|
|
440
|
+
.sort((a, b) => b.tokensPerTurn - a.tokensPerTurn)
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
/**
|
|
444
|
+
* Which skills should be lazy-loaded.
|
|
445
|
+
*
|
|
446
|
+
* Rule: a skill whose description costs at least `minTokens` (default 60) on
|
|
447
|
+
* every turn and was never invoked in this session is a lazy-load candidate.
|
|
448
|
+
* That is deliberately conservative — it never flags a skill the session leaned
|
|
449
|
+
* on, and it ignores skills with no usage signal at all rather than guessing
|
|
450
|
+
* that an unused-looking skill is dead (it may be used in another session).
|
|
451
|
+
*/
|
|
452
|
+
export function lazyLoadCandidates(ranked, { minTokens = 60 } = {}) {
|
|
453
|
+
return ranked.filter((s) => s.uses === 0 && s.tokensPerTurn >= minTokens)
|
|
454
|
+
}
|
|
455
|
+
|
|
456
|
+
// =====================================================================
|
|
457
|
+
// Tool schema inflation
|
|
458
|
+
// =====================================================================
|
|
459
|
+
|
|
460
|
+
/**
|
|
461
|
+
* Attribute tool-schema bytes to a plugin group.
|
|
462
|
+
*
|
|
463
|
+
* Tool names carry their owner: `mcp__<server>__<tool>` is an MCP server,
|
|
464
|
+
* `agent_teams_*` / `team_task_*` are the AgentTeams plugin, `dsh_ppt_*` is the
|
|
465
|
+
* ppt plugin, and everything else is a built-in. The grouping is a prefix rule,
|
|
466
|
+
* not a registry lookup, so it works on any machine without a plugin list.
|
|
467
|
+
*/
|
|
468
|
+
export function toolGroup(name) {
|
|
469
|
+
const n = String(name || '')
|
|
470
|
+
if (n.startsWith('mcp__')) {
|
|
471
|
+
const parts = n.split('__')
|
|
472
|
+
return `mcp:${parts[1] || 'unknown'}`
|
|
473
|
+
}
|
|
474
|
+
if (n.startsWith('agent_teams_') || n.startsWith('team_task_')) return 'plugin:agent-teams'
|
|
475
|
+
if (n.startsWith('dsh_ppt_')) return 'plugin:ppt-fusion'
|
|
476
|
+
if (n.startsWith('ctx_budget')) return 'plugin:context-budget'
|
|
477
|
+
const twoWords = n.split('_').slice(0, 2).join('_')
|
|
478
|
+
if (twoWords && twoWords !== n) return `tool:${twoWords}`
|
|
479
|
+
return `tool:${n || 'unknown'}`
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
/** Rank tool groups by the tokens their schemas cost per turn. */
|
|
483
|
+
export function toolInflation(tools) {
|
|
484
|
+
const groups = new Map()
|
|
485
|
+
let total = 0
|
|
486
|
+
for (const t of tools || []) {
|
|
487
|
+
const chars = JSON.stringify(t).length
|
|
488
|
+
const g = toolGroup(t.name)
|
|
489
|
+
if (!groups.has(g)) groups.set(g, { group: g, tools: 0, chars: 0 })
|
|
490
|
+
const rec = groups.get(g)
|
|
491
|
+
rec.tools += 1
|
|
492
|
+
rec.chars += chars
|
|
493
|
+
total += chars
|
|
494
|
+
}
|
|
495
|
+
const rows = [...groups.values()].map((r) => ({ ...r, tokensPerTurn: tokensForChars(r.chars) }))
|
|
496
|
+
rows.sort((a, b) => b.tokensPerTurn - a.tokensPerTurn)
|
|
497
|
+
return { rows, totalChars: total, totalTokens: tokensForChars(total) }
|
|
498
|
+
}
|
|
499
|
+
|
|
500
|
+
// =====================================================================
|
|
501
|
+
// Whole-session audit
|
|
502
|
+
// =====================================================================
|
|
503
|
+
|
|
504
|
+
/**
|
|
505
|
+
* Audit one session's context budget.
|
|
506
|
+
*
|
|
507
|
+
* Returns { ok, session, turns, composition, repetition, perTurn, skills,
|
|
508
|
+
* tools, report }.
|
|
509
|
+
*/
|
|
510
|
+
export function auditSession(events, { usage = {}, skills = [], tools = [] } = {}) {
|
|
511
|
+
const turns = buildTurns(events)
|
|
512
|
+
const comp = composition(turns)
|
|
513
|
+
const rep = detectRepetition(turns)
|
|
514
|
+
const perTurn = perTurnWaste(turns)
|
|
515
|
+
const inflated = toolInflation(tools)
|
|
516
|
+
const ranked = rankSkills(skills, usage)
|
|
517
|
+
const lazy = lazyLoadCandidates(ranked)
|
|
518
|
+
|
|
519
|
+
const turnCount = turns.length
|
|
520
|
+
const requests = turns.reduce((a, t) => a + t.requests, 0)
|
|
521
|
+
const wasteRatio = comp.totalTokens > 0 ? rep.wasteTokens / comp.totalTokens : 0
|
|
522
|
+
// The model actually used comes from the request the host sent, which is the
|
|
523
|
+
// authoritative record; model/selection is only the last UI change.
|
|
524
|
+
const model =
|
|
525
|
+
events.find((e) => e.type === 'request/header')?.data?.header?.config?.model ??
|
|
526
|
+
events.find((e) => e.type === 'model/selection')?.data?.model ??
|
|
527
|
+
null
|
|
528
|
+
const contextWindow = events.find((e) => e.type === 'request/context')?.data?.contextWindow ?? null
|
|
529
|
+
|
|
530
|
+
return {
|
|
531
|
+
ok: true,
|
|
532
|
+
session: {
|
|
533
|
+
events: events.length,
|
|
534
|
+
turns: turnCount,
|
|
535
|
+
requests,
|
|
536
|
+
model,
|
|
537
|
+
contextWindow,
|
|
538
|
+
cwd: events.find((e) => e.type === 'session')?.data?.cwd ?? null,
|
|
539
|
+
},
|
|
540
|
+
turns,
|
|
541
|
+
composition: comp,
|
|
542
|
+
repetition: rep,
|
|
543
|
+
perTurn,
|
|
544
|
+
skills: { ranked, lazy, total: skills.length, tokensPerTurn: ranked.reduce((a, s) => a + s.tokensPerTurn, 0) },
|
|
545
|
+
tools: inflated,
|
|
546
|
+
wasteRatio,
|
|
547
|
+
}
|
|
548
|
+
}
|
|
549
|
+
|
|
550
|
+
/** Render the human-readable report used as the tool's text output. */
|
|
551
|
+
export function renderReport(result, { title }) {
|
|
552
|
+
const L = []
|
|
553
|
+
const head = title ? `${title}: ` : ''
|
|
554
|
+
L.push('')
|
|
555
|
+
L.push(`${head}${result.session.turns} turns, ${result.session.requests} request(s)`)
|
|
556
|
+
L.push(` injected context total (est): ${result.composition.totalTokens} tok`)
|
|
557
|
+
L.push(` repeated unchanged content: ${result.repetition.wasteTokens} tok (${(result.wasteRatio * 100).toFixed(1)}% of injected)`)
|
|
558
|
+
L.push('')
|
|
559
|
+
L.push('composition (est tokens, summed over turns)')
|
|
560
|
+
const cats = Object.entries(result.composition.categories).sort((a, b) => b[1].tokens - a[1].tokens)
|
|
561
|
+
for (const [cat, v] of cats) {
|
|
562
|
+
L.push(` ${cat.padEnd(10)} ${String(v.tokens).padStart(8)} (${v.blocks} block(s))`)
|
|
563
|
+
}
|
|
564
|
+
if (result.repetition.repeated.length) {
|
|
565
|
+
L.push('')
|
|
566
|
+
L.push('repeated unchanged content (waste)')
|
|
567
|
+
for (const r of result.repetition.repeated.slice(0, 8)) {
|
|
568
|
+
L.push(` ${r.label} ${r.occurrences}x ${r.tokensEach} tok each = ${r.wasteTokens} tok waste`)
|
|
569
|
+
}
|
|
570
|
+
}
|
|
571
|
+
if (result.tools.rows.length) {
|
|
572
|
+
L.push('')
|
|
573
|
+
L.push('tool schema inflation (est tokens per turn)')
|
|
574
|
+
for (const r of result.tools.rows.slice(0, 8)) {
|
|
575
|
+
L.push(` ${r.group.padEnd(24)} ${String(r.tokensPerTurn).padStart(7)} (${r.tools} tool(s))`)
|
|
576
|
+
}
|
|
577
|
+
}
|
|
578
|
+
if (result.skills.ranked.length) {
|
|
579
|
+
L.push('')
|
|
580
|
+
L.push(`skill catalog: ${result.skills.total} skills, ${result.skills.tokensPerTurn} est tok per turn`)
|
|
581
|
+
for (const s of result.skills.ranked.slice(0, 6)) {
|
|
582
|
+
L.push(` ${s.name.padEnd(30)} ${String(s.tokensPerTurn).padStart(5)} tok used ${s.uses}x`)
|
|
583
|
+
}
|
|
584
|
+
if (result.skills.lazy.length) {
|
|
585
|
+
L.push(` lazy-load candidates: ${result.skills.lazy.slice(0, 10).map((s) => s.name).join(', ')}`)
|
|
586
|
+
}
|
|
587
|
+
}
|
|
588
|
+
L.push('')
|
|
589
|
+
return L.join('\n')
|
|
590
|
+
}
|
package/lib/index.js
ADDED
|
@@ -0,0 +1,300 @@
|
|
|
1
|
+
// dsh-context-budget: agent tools over the per-turn context budget model.
|
|
2
|
+
//
|
|
3
|
+
// Four tools, one per question an operator actually asks:
|
|
4
|
+
//
|
|
5
|
+
// ctx_budget what is this turn's context made of, and how much of it
|
|
6
|
+
// is the same bytes as last turn?
|
|
7
|
+
// ctx_budget_sweep every session, ranked by waste.
|
|
8
|
+
// ctx_budget_skills the standing cost of the skill catalog, and which
|
|
9
|
+
// skills should be lazy-loaded.
|
|
10
|
+
// ctx_budget_tools which plugin's tool schemas cost the most per turn.
|
|
11
|
+
//
|
|
12
|
+
// Deterministic throughout: no LLM is called, so every number here can be
|
|
13
|
+
// recomputed by hand from the session log.
|
|
14
|
+
import { join } from 'node:path'
|
|
15
|
+
import { homedir } from 'node:os'
|
|
16
|
+
import { existsSync, readdirSync } from 'node:fs'
|
|
17
|
+
import {
|
|
18
|
+
auditSession,
|
|
19
|
+
buildTurns,
|
|
20
|
+
loadSession,
|
|
21
|
+
parseSkillCatalog,
|
|
22
|
+
renderReport,
|
|
23
|
+
toolInflation,
|
|
24
|
+
tokensForChars,
|
|
25
|
+
} from './engine.js'
|
|
26
|
+
|
|
27
|
+
export const name = 'context-budget'
|
|
28
|
+
export const inject = ['tools']
|
|
29
|
+
|
|
30
|
+
const Config = {
|
|
31
|
+
// Where DSH keeps session logs.
|
|
32
|
+
sessionsDir: '',
|
|
33
|
+
// Cap the number of sessions a sweep will read, so a large history cannot
|
|
34
|
+
// hang the tool. Override with a bigger number when you want the full sweep.
|
|
35
|
+
maxSessions: 30,
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/** Resolve the sessions dir: explicit arg, config, or the default per-profile layout. */
|
|
39
|
+
function resolveSessionsDir(dir) {
|
|
40
|
+
if (dir) return dir
|
|
41
|
+
if (Config.sessionsDir) return Config.sessionsDir
|
|
42
|
+
return join(homedir(), '.dsh', 'sessions')
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
/**
|
|
46
|
+
* Every session log under a sessions root.
|
|
47
|
+
*
|
|
48
|
+
* The layout is `<sessions>/<workspace-slug>/session-<uuid>/session.v4.jsonl.zstd`
|
|
49
|
+
* — a workspace level sits between the root and the session dirs, so a flat
|
|
50
|
+
* scan of the root finds nothing. Walk down to the session dirs and keep the
|
|
51
|
+
* workspace slug with each row.
|
|
52
|
+
*/
|
|
53
|
+
function listSessions(root, limit) {
|
|
54
|
+
if (!existsSync(root)) return []
|
|
55
|
+
const out = []
|
|
56
|
+
const walk = (dir, workspace, depth) => {
|
|
57
|
+
if (depth > 2 || out.length > (typeof limit === 'number' ? limit : 500)) return
|
|
58
|
+
let entries
|
|
59
|
+
try {
|
|
60
|
+
entries = readdirSync(dir, { withFileTypes: true })
|
|
61
|
+
} catch {
|
|
62
|
+
return
|
|
63
|
+
}
|
|
64
|
+
for (const e of entries) {
|
|
65
|
+
if (!e.isDirectory()) continue
|
|
66
|
+
const full = join(dir, e.name)
|
|
67
|
+
const log = join(full, 'session.v4.jsonl.zstd')
|
|
68
|
+
if (existsSync(log)) {
|
|
69
|
+
out.push({ name: e.name, workspace, file: log })
|
|
70
|
+
continue
|
|
71
|
+
}
|
|
72
|
+
walk(full, e.name, depth + 1)
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
walk(root, '(root)', 0)
|
|
76
|
+
return typeof limit === 'number' ? out.slice(0, limit) : out
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
/** Accept a session path, a session dir, or a bare session id. */
|
|
80
|
+
function resolveSessionFile(arg) {
|
|
81
|
+
if (!arg) return null
|
|
82
|
+
if (existsSync(arg) && arg.endsWith('.zstd')) return arg
|
|
83
|
+
const direct = join(arg, 'session.v4.jsonl.zstd')
|
|
84
|
+
if (existsSync(direct)) return direct
|
|
85
|
+
const byId = listSessions(resolveSessionsDir('')).find((s) => s.name === arg || s.name === `session-${arg}`)
|
|
86
|
+
return byId ? byId.file : null
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
/**
|
|
90
|
+
* The extra inputs the pure engine cannot see in the turn model: the raw skill
|
|
91
|
+
* catalog text and the tool schema array actually sent. Both come from the same
|
|
92
|
+
* events the engine already read, so no second pass over the file is needed.
|
|
93
|
+
*/
|
|
94
|
+
function extractInputs(events) {
|
|
95
|
+
let skills = []
|
|
96
|
+
let tools = []
|
|
97
|
+
let catalogSeen = null
|
|
98
|
+
for (const ev of events) {
|
|
99
|
+
if (ev.type === 'user/message' && ev.data?.source?.kind === 'skill-catalog' && catalogSeen === null) {
|
|
100
|
+
catalogSeen = ev.data.content.map((c) => c.text || '').join('\n')
|
|
101
|
+
skills = parseSkillCatalog(catalogSeen)
|
|
102
|
+
}
|
|
103
|
+
if (ev.type === 'request/header' && Array.isArray(ev.data?.header?.tools) && tools.length === 0) {
|
|
104
|
+
tools = ev.data.header.tools
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
return { skills, tools }
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/** How many times each skill was actually invoked, from `skill` tool calls. */
|
|
111
|
+
function extractSkillUsage(events) {
|
|
112
|
+
const usage = {}
|
|
113
|
+
for (const ev of events) {
|
|
114
|
+
if (ev.type !== 'tool/call' || ev.data?.name !== 'skill') continue
|
|
115
|
+
let args = {}
|
|
116
|
+
try {
|
|
117
|
+
args = JSON.parse(ev.data.arguments || '{}')
|
|
118
|
+
} catch {}
|
|
119
|
+
const n = args?.name
|
|
120
|
+
if (n) usage[n] = (usage[n] || 0) + 1
|
|
121
|
+
}
|
|
122
|
+
return usage
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
export function apply(ctx, config = {}) {
|
|
126
|
+
const cfg = { ...Config, ...config }
|
|
127
|
+
const OUTPUT_SCHEMA = { type: 'json' }
|
|
128
|
+
const outRender = (_args, value) => [{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value, null, 2) }]
|
|
129
|
+
|
|
130
|
+
const tools = [
|
|
131
|
+
{
|
|
132
|
+
name: 'ctx_budget',
|
|
133
|
+
description:
|
|
134
|
+
'Audit how one session spends its per-turn context budget: break down each turn into the surfaces that inject it (system prompt, AGENTS.md instructions, skill catalog, tool schemas, runtime context, team messages, history), then measure how much of it is the same content resent byte-identically — the part the model does not need to be told again. Also reports the model and context window for scale. Deterministic, no LLM. Returns {ok, session, composition, repetition, perTurn, skills, tools, wasteRatio, report}.',
|
|
135
|
+
parameters: {
|
|
136
|
+
type: 'object',
|
|
137
|
+
properties: {
|
|
138
|
+
session: { type: 'string', description: 'Path to a session.v4.jsonl.zstd file, a session directory, or a session id. Omit to audit the newest session.' },
|
|
139
|
+
dir: { type: 'string', description: 'Sessions root used to resolve a bare session id, and to pick the newest session when none is given. Defaults to ~/.dsh/sessions.' },
|
|
140
|
+
},
|
|
141
|
+
},
|
|
142
|
+
async execute(args) {
|
|
143
|
+
const root = resolveSessionsDir(args.dir)
|
|
144
|
+
let file = args.session ? resolveSessionFile(args.session) : null
|
|
145
|
+
if (args.session && !file) return { ok: false, error: `no session.v4.jsonl.zstd for ${args.session}` }
|
|
146
|
+
if (!file) {
|
|
147
|
+
const newest = listSessions(root, cfg.maxSessions).sort((a, b) => b.name.localeCompare(a.name))[0]
|
|
148
|
+
if (!newest) return { ok: false, error: `no sessions found under ${root}` }
|
|
149
|
+
file = newest.file
|
|
150
|
+
}
|
|
151
|
+
const { events, error } = loadSession(file)
|
|
152
|
+
if (error) return { ok: false, error }
|
|
153
|
+
if (events.length === 0) return { ok: false, error: 'session log is empty' }
|
|
154
|
+
const { skills, tools: toolSchemas } = extractInputs(events)
|
|
155
|
+
const result = auditSession(events, { skills, tools: toolSchemas, usage: extractSkillUsage(events) })
|
|
156
|
+
result.session.file = file
|
|
157
|
+
result.session.workspace = file.split(/[\\/]/).slice(-3, -2)[0] ?? null
|
|
158
|
+
result.report = renderReport(result, { title: `context budget · ${file}` })
|
|
159
|
+
return result
|
|
160
|
+
},
|
|
161
|
+
},
|
|
162
|
+
{
|
|
163
|
+
name: 'ctx_budget_sweep',
|
|
164
|
+
description:
|
|
165
|
+
'Sweep every session under a sessions root and rank them by how much of their injected context is repeated unchanged content, so the worst offenders are the first ones you look at. Read-only; a big history is capped by `limit` (default 30 newest). Returns {ok, sessions:[{name,workspace,turns,injectedTokens,wasteTokens,wasteRatio,topSurface}], totals}.',
|
|
166
|
+
parameters: {
|
|
167
|
+
type: 'object',
|
|
168
|
+
properties: {
|
|
169
|
+
dir: { type: 'string', description: 'Sessions root. Defaults to ~/.dsh/sessions.' },
|
|
170
|
+
limit: { type: 'integer', description: 'Max sessions to read (default 30).' },
|
|
171
|
+
},
|
|
172
|
+
},
|
|
173
|
+
async execute(args) {
|
|
174
|
+
const root = resolveSessionsDir(args.dir)
|
|
175
|
+
const sessions = listSessions(root, Math.max(1, Number(args.limit) || cfg.maxSessions))
|
|
176
|
+
if (sessions.length === 0) return { ok: false, error: `no sessions found under ${root}` }
|
|
177
|
+
const rows = []
|
|
178
|
+
for (const s of sessions) {
|
|
179
|
+
const { events, error } = loadSession(s.file)
|
|
180
|
+
if (error || events.length === 0) continue
|
|
181
|
+
const turns = buildTurns(events)
|
|
182
|
+
if (turns.length === 0) continue
|
|
183
|
+
const result = auditSession(events, extractInputs(events))
|
|
184
|
+
const top = Object.entries(result.composition.categories).sort((a, b) => b[1].tokens - a[1].tokens)[0]
|
|
185
|
+
rows.push({
|
|
186
|
+
name: s.name,
|
|
187
|
+
workspace: s.workspace,
|
|
188
|
+
turns: result.session.turns,
|
|
189
|
+
injectedTokens: result.composition.totalTokens,
|
|
190
|
+
wasteTokens: result.repetition.wasteTokens,
|
|
191
|
+
wasteRatio: result.wasteRatio,
|
|
192
|
+
topSurface: top ? `${top[0]} ${top[1].tokens}` : null,
|
|
193
|
+
})
|
|
194
|
+
}
|
|
195
|
+
rows.sort((a, b) => b.wasteTokens - a.wasteTokens)
|
|
196
|
+
const totals = rows.reduce(
|
|
197
|
+
(acc, r) => ({ injectedTokens: acc.injectedTokens + r.injectedTokens, wasteTokens: acc.wasteTokens + r.wasteTokens, turns: acc.turns + r.turns }),
|
|
198
|
+
{ injectedTokens: 0, wasteTokens: 0, turns: 0 },
|
|
199
|
+
)
|
|
200
|
+
return { ok: true, sessions: rows, totals }
|
|
201
|
+
},
|
|
202
|
+
},
|
|
203
|
+
{
|
|
204
|
+
name: 'ctx_budget_skills',
|
|
205
|
+
description:
|
|
206
|
+
'Skill-catalog governance: how many tokens of skill descriptions the model reads on every single turn, ranked by cost, and which skills should be lazy-loaded (expensive to describe, rarely invoked). With 60+ skills installed the catalog is one of the largest standing costs in the prompt, and it is paid whether or not a skill is ever used. Returns {ok, skills:[{name,tokensPerTurn,uses}], tokensPerTurn, lazyLoad, turns}.',
|
|
207
|
+
parameters: {
|
|
208
|
+
type: 'object',
|
|
209
|
+
properties: {
|
|
210
|
+
session: { type: 'string', description: 'Path to a session.v4.jsonl.zstd file, a session directory, or a session id. Omit to audit the newest session.' },
|
|
211
|
+
dir: { type: 'string', description: 'Sessions root used to resolve a bare session id and to pick the newest session. Defaults to ~/.dsh/sessions.' },
|
|
212
|
+
maxSkills: { type: 'integer', description: 'How many skills to list at most (default 40).' },
|
|
213
|
+
},
|
|
214
|
+
},
|
|
215
|
+
async execute(args) {
|
|
216
|
+
const root = resolveSessionsDir(args.dir)
|
|
217
|
+
let file = args.session ? resolveSessionFile(args.session) : null
|
|
218
|
+
if (args.session && !file) return { ok: false, error: `no session.v4.jsonl.zstd for ${args.session}` }
|
|
219
|
+
if (!file) {
|
|
220
|
+
const newest = listSessions(root, cfg.maxSessions).sort((a, b) => b.name.localeCompare(a.name))[0]
|
|
221
|
+
if (!newest) return { ok: false, error: `no sessions found under ${root}` }
|
|
222
|
+
file = newest.file
|
|
223
|
+
}
|
|
224
|
+
const { events, error } = loadSession(file)
|
|
225
|
+
if (error) return { ok: false, error }
|
|
226
|
+
const { skills } = extractInputs(events)
|
|
227
|
+
if (skills.length === 0) return { ok: false, error: 'no skill catalog found in this session' }
|
|
228
|
+
const usage = extractSkillUsage(events)
|
|
229
|
+
const ranked = skills
|
|
230
|
+
.map((s) => ({ name: s.name, chars: s.chars, tokensPerTurn: tokensForChars(s.chars), uses: usage[s.name] || 0 }))
|
|
231
|
+
.sort((a, b) => b.tokensPerTurn - a.tokensPerTurn)
|
|
232
|
+
const max = Math.max(1, Number(args.maxSkills) || 40)
|
|
233
|
+
const lazy = ranked.filter((s) => s.uses === 0 && s.tokensPerTurn >= 60)
|
|
234
|
+
return {
|
|
235
|
+
ok: true,
|
|
236
|
+
file,
|
|
237
|
+
skills: ranked.slice(0, max),
|
|
238
|
+
totalSkills: ranked.length,
|
|
239
|
+
tokensPerTurn: ranked.reduce((a, s) => a + s.tokensPerTurn, 0),
|
|
240
|
+
lazyLoad: lazy.slice(0, max).map((s) => ({ name: s.name, tokensPerTurn: s.tokensPerTurn, uses: 0 })),
|
|
241
|
+
turns: events.filter((e) => e.type === 'turn/start').length,
|
|
242
|
+
}
|
|
243
|
+
},
|
|
244
|
+
},
|
|
245
|
+
{
|
|
246
|
+
name: 'ctx_budget_tools',
|
|
247
|
+
description:
|
|
248
|
+
'Tool-schema inflation: which tool group occupies the most context on every turn. Tool definitions are sent in full on every request, so a chatty MCP server or a plugin with wide parameters is a standing tax the model pays per turn. Groups are derived from tool-name prefixes (mcp__server__, plugin families, built-ins). Returns {ok, groups:[{group,tools,chars,tokensPerTurn}], totalTokensPerTurn, tools}.',
|
|
249
|
+
parameters: {
|
|
250
|
+
type: 'object',
|
|
251
|
+
properties: {
|
|
252
|
+
session: { type: 'string', description: 'Path to a session.v4.jsonl.zstd file, a session directory, or a session id. Omit to audit the newest session.' },
|
|
253
|
+
dir: { type: 'string', description: 'Sessions root used to resolve a bare session id and to pick the newest session. Defaults to ~/.dsh/sessions.' },
|
|
254
|
+
maxGroups: { type: 'integer', description: 'How many groups to list at most (default 20).' },
|
|
255
|
+
},
|
|
256
|
+
},
|
|
257
|
+
async execute(args) {
|
|
258
|
+
const root = resolveSessionsDir(args.dir)
|
|
259
|
+
let file = args.session ? resolveSessionFile(args.session) : null
|
|
260
|
+
if (args.session && !file) return { ok: false, error: `no session.v4.jsonl.zstd for ${args.session}` }
|
|
261
|
+
if (!file) {
|
|
262
|
+
const newest = listSessions(root, cfg.maxSessions).sort((a, b) => b.name.localeCompare(a.name))[0]
|
|
263
|
+
if (!newest) return { ok: false, error: `no sessions found under ${root}` }
|
|
264
|
+
file = newest.file
|
|
265
|
+
}
|
|
266
|
+
const { events, error } = loadSession(file)
|
|
267
|
+
if (error) return { ok: false, error }
|
|
268
|
+
const { tools } = extractInputs(events)
|
|
269
|
+
if (tools.length === 0) return { ok: false, error: 'no tool schemas found in this session' }
|
|
270
|
+
const inflated = toolInflation(tools)
|
|
271
|
+
const max = Math.max(1, Number(args.maxGroups) || 20)
|
|
272
|
+
return {
|
|
273
|
+
ok: true,
|
|
274
|
+
file,
|
|
275
|
+
groups: inflated.rows.slice(0, max),
|
|
276
|
+
toolCount: tools.length,
|
|
277
|
+
totalChars: inflated.totalChars,
|
|
278
|
+
totalTokensPerTurn: inflated.totalTokens,
|
|
279
|
+
}
|
|
280
|
+
},
|
|
281
|
+
},
|
|
282
|
+
]
|
|
283
|
+
|
|
284
|
+
for (const t of tools) {
|
|
285
|
+
try {
|
|
286
|
+
ctx.tools.register({
|
|
287
|
+
name: t.name,
|
|
288
|
+
description: t.description,
|
|
289
|
+
parameters: t.parameters,
|
|
290
|
+
output: { schema: OUTPUT_SCHEMA, render: outRender },
|
|
291
|
+
timeoutMs: 300_000,
|
|
292
|
+
isConcurrencySafe: () => true,
|
|
293
|
+
presentCall: (args) => ({ card: 'generic', title: t.name, kind: 'read', rawInput: args }),
|
|
294
|
+
execute: t.execute,
|
|
295
|
+
})
|
|
296
|
+
} catch (error) {
|
|
297
|
+
console.error(`[context-budget] ${t.name} registration skipped: ${error}`)
|
|
298
|
+
}
|
|
299
|
+
}
|
|
300
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,51 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bushizhanghan/dsh-context-budget",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Audit where each turn's context budget goes: per-turn composition (AGENTS.md / skill catalog / tool schemas / runtime context / history), repeated injection of byte-identical content (pure waste), skill-catalog governance, and tool-schema inflation. Deterministic, no LLM. 上下文预算审计:每轮注入构成、重复注入浪费、技能目录治理、工具 schema 膨胀。",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "lib/index.js",
|
|
7
|
+
"exports": {
|
|
8
|
+
".": "./lib/index.js",
|
|
9
|
+
"./engine": "./lib/engine.js",
|
|
10
|
+
"./package.json": "./package.json"
|
|
11
|
+
},
|
|
12
|
+
"files": [
|
|
13
|
+
"lib",
|
|
14
|
+
"cordis.patch.yml",
|
|
15
|
+
"README.md",
|
|
16
|
+
"LICENSE"
|
|
17
|
+
],
|
|
18
|
+
"keywords": [
|
|
19
|
+
"dsh",
|
|
20
|
+
"dsh-plugin",
|
|
21
|
+
"deepseek-harness",
|
|
22
|
+
"cordis",
|
|
23
|
+
"cordis-plugin",
|
|
24
|
+
"context",
|
|
25
|
+
"tokens",
|
|
26
|
+
"budget",
|
|
27
|
+
"audit",
|
|
28
|
+
"agent-tools"
|
|
29
|
+
],
|
|
30
|
+
"author": "bushizhanghan",
|
|
31
|
+
"license": "MIT",
|
|
32
|
+
"repository": {
|
|
33
|
+
"type": "git",
|
|
34
|
+
"url": "git+https://github.com/bushizhanghan/dsh-context-budget.git"
|
|
35
|
+
},
|
|
36
|
+
"bugs": {
|
|
37
|
+
"url": "https://github.com/bushizhanghan/dsh-context-budget/issues"
|
|
38
|
+
},
|
|
39
|
+
"homepage": "https://github.com/bushizhanghan/dsh-context-budget#readme",
|
|
40
|
+
"engines": {
|
|
41
|
+
"node": ">=22"
|
|
42
|
+
},
|
|
43
|
+
"dsh": {
|
|
44
|
+
"bundle": {
|
|
45
|
+
"patch": "./cordis.patch.yml"
|
|
46
|
+
}
|
|
47
|
+
},
|
|
48
|
+
"scripts": {
|
|
49
|
+
"test": "node --test test/"
|
|
50
|
+
}
|
|
51
|
+
}
|