dsh-xray 0.9.1 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -0
- package/README.zh.md +4 -0
- package/lib/client.js +157 -1
- package/lib/collect/requests.js +160 -0
- package/lib/index.js +20 -2
- package/lib/model.js +36 -0
- package/lib/panel.js +30 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -13,6 +13,8 @@
|
|
|
13
13
|
|
|
14
14
|
<p align="center">
|
|
15
15
|
<strong>X-ray for your <a href="https://github.com/deepseek-ai/deepseek-harness">DeepSeek Harness</a></strong> — see what's actually loaded, why, and what it costs you.
|
|
16
|
+
<br>
|
|
17
|
+
<em>LLM context-cost observability: token attribution per plugin, prompt-section and tool-schema pricing, skill catalog tax, dependency cascades.</em>
|
|
16
18
|
</p>
|
|
17
19
|
|
|
18
20
|
<p align="center">
|
|
@@ -64,6 +66,7 @@ The cost view answers the question no other tool asks: **who put this in my cont
|
|
|
64
66
|
- **By-plugin rollup** — each plugin's per-request context tax: sections + schemas + tokens + share, ranked.
|
|
65
67
|
- **Entry inspection** — `/xray/api/entry` returns any entry's live text with a chars/tokens ruler. Computed per request, never persisted.
|
|
66
68
|
- **Skill cost** — a dedicated view prices every skill twice: its catalog line (resident on every request once any model-invocable skill exists) and its body (billed per load). Pricing only — toggling belongs to the ecosystem's skill managers.
|
|
69
|
+
- **Per-request ledger** — one classified bill per LLM call: system / tool schemas / history / tool results (aggregated per tool), a **Δprev** growth column, and prefix-stability markers (⚡ system+tools byte-identical to the previous request — KV-cache-friendly; ✂ prefix broken). Compaction and title calls are tagged apart from chat. Only counts, names, and hashes are kept — never message text.
|
|
67
70
|
- **Explained UI** — every view opens with a one-line "what am I looking at"; terms carry plain-language tooltips; the whole tab is localized (English / 中文) through the host locale service.
|
|
68
71
|
|
|
69
72
|
The same data flows through three surfaces: the **X-Ray tab** (native GUI), the standalone **`/xray` page** (works even when the client-module pipeline it diagnoses is broken), and the **CLI**.
|
|
@@ -208,6 +211,7 @@ Diagnostic imaging for a running composition — complementary to [dsh-doctor](h
|
|
|
208
211
|
| --- | --- |
|
|
209
212
|
| Context-tax attribution & entry inspection | 💰 Optimization |
|
|
210
213
|
| Skill cost (catalog line + body pricing) | 💰 Optimization |
|
|
214
|
+
| Per-request ledger (Δprev, prefix stability) | 💰 Optimization |
|
|
211
215
|
| Layer attribution | 🔍 Inspection |
|
|
212
216
|
| Declared vs. actual diff | 🔍 Inspection |
|
|
213
217
|
| Conflict detection | 🔍 Inspection |
|
package/README.zh.md
CHANGED
|
@@ -12,6 +12,8 @@
|
|
|
12
12
|
|
|
13
13
|
<p align="center">
|
|
14
14
|
<strong>给 <a href="https://github.com/deepseek-ai/deepseek-harness">DeepSeek Harness</a> 拍 X 光</strong>——看清到底加载了什么、为什么在那、以及它悄悄花掉了你什么。
|
|
15
|
+
<br>
|
|
16
|
+
<em>LLM 上下文成本可观测:按插件归因 token、prompt section 与工具 schema 计价、skill catalog 常驻税、依赖停用级联。</em>
|
|
15
17
|
</p>
|
|
16
18
|
|
|
17
19
|
<p align="center">
|
|
@@ -63,6 +65,7 @@ cost 视图回答一个别的工具都不问的问题:**这段上下文是谁放
|
|
|
63
65
|
- **按插件汇总** —— 每个插件的每请求上下文税:sections + schemas + tokens + 占比,排序呈现。
|
|
64
66
|
- **条目原文查看** —— `/xray/api/entry` 返回任意条目的实时文本,附字符/token 标尺。按请求现算,绝不落盘。
|
|
65
67
|
- **skill 成本** —— 专属视图为每个 skill 计两笔价:catalog 行(只要存在模型可调用的 skill 就随每次请求常驻)与正文(每次加载计费)。只计价——启停归生态里的 skill 管理器。
|
|
68
|
+
- **逐请求账单** —— 每次 LLM 调用一张分类账单:system / 工具 schema / 历史 / 工具结果(按工具聚合),**Δprev** 增量列,前缀稳定性标记(⚡ system+工具与上一请求逐字节一致——KV 缓存友好;✂ 前缀击穿)。compaction 与标题生成单独标注。只存计数、名字和 hash——绝不存消息正文。
|
|
66
69
|
- **界面自解释** —— 每个视图开头一句"你在看什么";术语带白话提示;整个标签页通过宿主 locale 服务双语呈现(English / 中文)。
|
|
67
70
|
|
|
68
71
|
同一份数据流经三个界面:**X 光标签页**(原生 GUI)、独立 **`/xray` 页面**(连它所诊断的 client-module 加载链路挂了都能用)、以及 **CLI**。
|
|
@@ -207,6 +210,7 @@ npx dsh-xray health # 读取运行时快照
|
|
|
207
210
|
| --- | --- |
|
|
208
211
|
| 上下文税归因 & 条目原文查看 | 💰 优化 |
|
|
209
212
|
| skill 成本(catalog 行 + 正文计价) | 💰 优化 |
|
|
213
|
+
| 逐请求账单(Δprev、前缀稳定性) | 💰 优化 |
|
|
210
214
|
| 层归因 | 🔍 检视 |
|
|
211
215
|
| 声明 vs 实际 diff | 🔍 检视 |
|
|
212
216
|
| 冲突检测 | 🔍 检视 |
|
package/lib/client.js
CHANGED
|
@@ -68,7 +68,7 @@ window.__ModuleLoader__.load({
|
|
|
68
68
|
//#endregion
|
|
69
69
|
|
|
70
70
|
//#region tiny view primitives
|
|
71
|
-
const VIEWS = ['summary', 'health', 'deps', 'cost', 'shadow', 'skills'];
|
|
71
|
+
const VIEWS = ['summary', 'health', 'deps', 'cost', 'shadow', 'skills', 'requests'];
|
|
72
72
|
const NS = 'xray';
|
|
73
73
|
|
|
74
74
|
/** English messages (also the key vocabulary; zh mirrors every key). */
|
|
@@ -169,6 +169,30 @@ window.__ModuleLoader__.load({
|
|
|
169
169
|
'no skill observation yet — the skills service is absent or discovery has not run',
|
|
170
170
|
'skills.yes': 'yes',
|
|
171
171
|
'skills.userOnly': 'user-only',
|
|
172
|
+
'intro.requests':
|
|
173
|
+
'One bill per LLM call, newest first: system + tool schemas + history + tool results. ⚡ = the system/tools prefix matched the previous request (cache-friendly); ✂ = the prefix changed. Δ is the total against the previous request.',
|
|
174
|
+
'col.seq': '#',
|
|
175
|
+
'col.when': 'when',
|
|
176
|
+
'col.purpose': 'purpose',
|
|
177
|
+
'col.total': 'total',
|
|
178
|
+
'col.system': 'system',
|
|
179
|
+
'col.toolSchemas2': 'tools',
|
|
180
|
+
'col.history': 'history',
|
|
181
|
+
'col.results': 'results',
|
|
182
|
+
'col.prefix': 'prefix',
|
|
183
|
+
'col.delta': 'Δ',
|
|
184
|
+
'tip.total': 'Estimated tokens of the whole request payload (~4 chars/token)',
|
|
185
|
+
'tip.history': 'Conversation messages (user + assistant), excluding tool results',
|
|
186
|
+
'tip.results': 'Tool-result messages, aggregated per tool (hover a cell for the top tools)',
|
|
187
|
+
'tip.prefix':
|
|
188
|
+
'Whether system prompt + tool schemas were byte-identical to the previous request in this session (⚡ cache-friendly, ✂ prefix broken)',
|
|
189
|
+
'requests.none': 'no requests observed yet — send one agent message first',
|
|
190
|
+
'requests.session': 'session {id} ({n} request(s))',
|
|
191
|
+
'requests.chat': 'chat',
|
|
192
|
+
'requests.copyId': 'copy call id',
|
|
193
|
+
'requests.copied': 'copied',
|
|
194
|
+
'tip.copyId':
|
|
195
|
+
"Opaque id of this tool's largest result in this request — find the call in the Trajectory tab",
|
|
172
196
|
'entry.loading': 'loading entry…',
|
|
173
197
|
'entry.stats': '{chars} chars · ~{tokens} tokens ({estimator})',
|
|
174
198
|
'entry.close': 'close',
|
|
@@ -264,6 +288,29 @@ window.__ModuleLoader__.load({
|
|
|
264
288
|
'skills.none': '尚无 skill 观测——skills 服务缺失或发现未运行',
|
|
265
289
|
'skills.yes': '是',
|
|
266
290
|
'skills.userOnly': '仅用户',
|
|
291
|
+
'intro.requests':
|
|
292
|
+
'每次 LLM 调用一张账单,最新在前:system + 工具 schema + 历史 + 工具结果。⚡ = system/工具前缀与上一请求逐字节一致(缓存友好);✂ = 前缀变了。Δ 为相对上一请求的总量增减。',
|
|
293
|
+
'col.seq': '#',
|
|
294
|
+
'col.when': '时间',
|
|
295
|
+
'col.purpose': '用途',
|
|
296
|
+
'col.total': '总量',
|
|
297
|
+
'col.system': 'system',
|
|
298
|
+
'col.toolSchemas2': '工具',
|
|
299
|
+
'col.history': '历史',
|
|
300
|
+
'col.results': '结果',
|
|
301
|
+
'col.prefix': '前缀',
|
|
302
|
+
'col.delta': 'Δ',
|
|
303
|
+
'tip.total': '整个请求负载的估算 tokens(约 4 字符/token)',
|
|
304
|
+
'tip.history': '对话消息(user + assistant),不含工具结果',
|
|
305
|
+
'tip.results': '工具结果消息,按工具聚合(悬停单元格看 top 工具)',
|
|
306
|
+
'tip.prefix':
|
|
307
|
+
'system prompt + 工具 schema 是否与本会话上一请求逐字节一致(⚡ 缓存友好,✂ 前缀击穿)',
|
|
308
|
+
'requests.none': '尚未观测到请求——先发送一条 agent 消息',
|
|
309
|
+
'requests.session': '会话 {id}({n} 个请求)',
|
|
310
|
+
'requests.chat': 'chat',
|
|
311
|
+
'requests.copyId': '复制 call id',
|
|
312
|
+
'requests.copied': '已复制',
|
|
313
|
+
'tip.copyId': '该工具在本请求中最大结果的不透明 id——在 Trajectory 标签页中定位那次调用',
|
|
267
314
|
'entry.loading': '正在加载条目…',
|
|
268
315
|
'entry.stats': '{chars} 字符 · 约 {tokens} tokens({estimator})',
|
|
269
316
|
'entry.close': '关闭',
|
|
@@ -393,6 +440,56 @@ window.__ModuleLoader__.load({
|
|
|
393
440
|
return h(Table, { t, headers: ['plugin', 'sections', 'tools', 'tokens', 'share', ''], rows });
|
|
394
441
|
}
|
|
395
442
|
|
|
443
|
+
/** Expandable tool-results cell: unfolds top tools with the opaque call
|
|
444
|
+
* id of each tool's largest result — the manual bridge to Trajectory
|
|
445
|
+
* until the host exposes a requestInspect write-side. */
|
|
446
|
+
function ResultsCell({ tokens, rows, t }) {
|
|
447
|
+
const [open, setOpen] = react.useState(false);
|
|
448
|
+
const [copied, setCopied] = react.useState(null);
|
|
449
|
+
if (!rows?.length) return h('span', { className: 'xray-num' }, String(tokens));
|
|
450
|
+
return h(
|
|
451
|
+
react.Fragment,
|
|
452
|
+
null,
|
|
453
|
+
h(
|
|
454
|
+
'button',
|
|
455
|
+
{ className: 'xray-entry-link xray-num', onClick: () => setOpen(!open) },
|
|
456
|
+
`${open ? '▾' : '▸'} ${tokens}`,
|
|
457
|
+
),
|
|
458
|
+
open
|
|
459
|
+
? h(
|
|
460
|
+
'div',
|
|
461
|
+
{ style: { marginTop: 4 } },
|
|
462
|
+
rows.map((row) =>
|
|
463
|
+
h(
|
|
464
|
+
'div',
|
|
465
|
+
{ key: row.name, style: { whiteSpace: 'nowrap' } },
|
|
466
|
+
h('span', { className: 'xray-muted' }, `${row.name} ~${row.tokens} `),
|
|
467
|
+
row.topCallId
|
|
468
|
+
? h(
|
|
469
|
+
'button',
|
|
470
|
+
{
|
|
471
|
+
className: 'xray-entry-link',
|
|
472
|
+
title: t('tip.copyId'),
|
|
473
|
+
onClick: () => {
|
|
474
|
+
try {
|
|
475
|
+
navigator.clipboard.writeText(String(row.topCallId));
|
|
476
|
+
setCopied(row.name);
|
|
477
|
+
setTimeout(() => setCopied(null), 1500);
|
|
478
|
+
} catch {
|
|
479
|
+
/* clipboard unavailable: the id is still visible */
|
|
480
|
+
}
|
|
481
|
+
},
|
|
482
|
+
},
|
|
483
|
+
copied === row.name ? t('requests.copied') : t('requests.copyId'),
|
|
484
|
+
)
|
|
485
|
+
: null,
|
|
486
|
+
),
|
|
487
|
+
),
|
|
488
|
+
)
|
|
489
|
+
: null,
|
|
490
|
+
);
|
|
491
|
+
}
|
|
492
|
+
|
|
396
493
|
/** Raw-text inspector: fetches one entry's live text on open. Fetched per
|
|
397
494
|
* view, never cached — the text IS the audit artifact. */
|
|
398
495
|
function EntryModal({ target, onClose, t }) {
|
|
@@ -701,6 +798,65 @@ window.__ModuleLoader__.load({
|
|
|
701
798
|
),
|
|
702
799
|
}),
|
|
703
800
|
),
|
|
801
|
+
|
|
802
|
+
requests: (d, t) =>
|
|
803
|
+
!d.available
|
|
804
|
+
? h('p', { className: 'xray-muted' }, t('requests.none'))
|
|
805
|
+
: h(
|
|
806
|
+
react.Fragment,
|
|
807
|
+
null,
|
|
808
|
+
d.sessions.map((session) =>
|
|
809
|
+
h(
|
|
810
|
+
react.Fragment,
|
|
811
|
+
{ key: session.sessionId },
|
|
812
|
+
h(
|
|
813
|
+
'div',
|
|
814
|
+
{ className: 'xray-h3' },
|
|
815
|
+
fill(t('requests.session'), {
|
|
816
|
+
id: String(session.sessionId).slice(0, 12),
|
|
817
|
+
n: session.requests.length,
|
|
818
|
+
}),
|
|
819
|
+
),
|
|
820
|
+
h(Table, {
|
|
821
|
+
t,
|
|
822
|
+
headers: [
|
|
823
|
+
'seq',
|
|
824
|
+
'when',
|
|
825
|
+
'purpose',
|
|
826
|
+
'total',
|
|
827
|
+
'system',
|
|
828
|
+
'toolSchemas2',
|
|
829
|
+
'history',
|
|
830
|
+
'results',
|
|
831
|
+
'prefix',
|
|
832
|
+
'delta',
|
|
833
|
+
],
|
|
834
|
+
rows: session.requests.map((r) => {
|
|
835
|
+
return Row(`${session.sessionId}/${r.seq}`, [
|
|
836
|
+
{ cls: 'xray-num', text: String(r.seq) },
|
|
837
|
+
{ cls: 'xray-muted', text: new Date(r.at).toTimeString().slice(0, 8) },
|
|
838
|
+
r.purpose
|
|
839
|
+
? h('span', { className: 'xray-muted' }, r.purpose)
|
|
840
|
+
: t('requests.chat'),
|
|
841
|
+
{ cls: 'xray-num', text: `~${r.total}` },
|
|
842
|
+
{ cls: 'xray-num', text: String(r.system) },
|
|
843
|
+
{ cls: 'xray-num', text: String(r.toolSchemas) },
|
|
844
|
+
{ cls: 'xray-num', text: String(r.history) },
|
|
845
|
+
h(ResultsCell, { tokens: r.toolResults, rows: r.toolResultRows, t }),
|
|
846
|
+
r.prefixStable === null ? '—' : r.prefixStable ? '⚡' : '✂',
|
|
847
|
+
{
|
|
848
|
+
cls: 'xray-num',
|
|
849
|
+
text:
|
|
850
|
+
r.deltaTotal === null
|
|
851
|
+
? '—'
|
|
852
|
+
: `${r.deltaTotal >= 0 ? '+' : ''}${r.deltaTotal}`,
|
|
853
|
+
},
|
|
854
|
+
]);
|
|
855
|
+
}),
|
|
856
|
+
}),
|
|
857
|
+
),
|
|
858
|
+
),
|
|
859
|
+
),
|
|
704
860
|
};
|
|
705
861
|
//#endregion
|
|
706
862
|
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
// Request ledger: observe the `llm/stream` waterfall and record ONE
|
|
2
|
+
// classified bill per LLM call — system prompt, tool schemas, conversation
|
|
3
|
+
// history, tool results — plus prefix stability. Purely observational: the
|
|
4
|
+
// options object passes through `next()` untouched, and nothing but counts,
|
|
5
|
+
// names, and hashes is retained (never message text).
|
|
6
|
+
//
|
|
7
|
+
// Every request in the process flows through here: main sessions, subagents,
|
|
8
|
+
// compaction and title calls (tagged by `purpose`). Entries bucket per
|
|
9
|
+
// sessionId in a bounded ring so one fan-out cannot evict another session's
|
|
10
|
+
// history.
|
|
11
|
+
|
|
12
|
+
const crypto = require('node:crypto');
|
|
13
|
+
|
|
14
|
+
/** Rough token estimate mirroring collect/runtime.js (~4 chars/token). */
|
|
15
|
+
function estimateTokens(text) {
|
|
16
|
+
return Math.ceil(text.length / 4);
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
/** Serialize one content block to its model-facing text size, recursively. */
|
|
20
|
+
function blockChars(block) {
|
|
21
|
+
if (!block || typeof block !== 'object') return 0;
|
|
22
|
+
switch (block.type) {
|
|
23
|
+
case 'text':
|
|
24
|
+
case 'reasoning':
|
|
25
|
+
return String(block.text ?? '').length;
|
|
26
|
+
case 'image':
|
|
27
|
+
// Images are billed by the provider in provider units; approximate by
|
|
28
|
+
// the transported payload size so the ledger at least ranks them.
|
|
29
|
+
return String(block.data ?? block.url ?? '').length;
|
|
30
|
+
case 'tool-call':
|
|
31
|
+
return String(block.name ?? '').length + String(block.arguments ?? '').length;
|
|
32
|
+
case 'tool-result':
|
|
33
|
+
return (block.content ?? []).reduce((sum, inner) => sum + blockChars(inner), 0);
|
|
34
|
+
default:
|
|
35
|
+
try {
|
|
36
|
+
return JSON.stringify(block).length;
|
|
37
|
+
} catch {
|
|
38
|
+
return 0;
|
|
39
|
+
}
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
/** Short content hash for prefix-stability comparison (never stored text). */
|
|
44
|
+
function digest(text) {
|
|
45
|
+
return crypto.createHash('sha256').update(text).digest('hex').slice(0, 12);
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* Classify one GenerateOptions into the ledger entry. Tool results resolve
|
|
50
|
+
* their tool name through the preceding assistant tool-call blocks.
|
|
51
|
+
*/
|
|
52
|
+
function classify(options) {
|
|
53
|
+
const system = String(options.system ?? '');
|
|
54
|
+
const toolsJson = JSON.stringify(options.tools ?? []);
|
|
55
|
+
const callNames = new Map(); // callId -> tool name
|
|
56
|
+
let historyChars = 0;
|
|
57
|
+
let historyMessages = 0;
|
|
58
|
+
let userMessages = 0;
|
|
59
|
+
let assistantMessages = 0;
|
|
60
|
+
const toolResults = new Map(); // tool name -> {count, chars, topCallId, topChars}
|
|
61
|
+
for (const message of options.messages ?? []) {
|
|
62
|
+
const content = message.content ?? [];
|
|
63
|
+
for (const block of content) {
|
|
64
|
+
if (block?.type === 'tool-call') callNames.set(block.id, block.name);
|
|
65
|
+
}
|
|
66
|
+
const isToolResult =
|
|
67
|
+
message.source?.kind === 'tool' || content.some((b) => b?.type === 'tool-result');
|
|
68
|
+
const chars = content.reduce((sum, block) => sum + blockChars(block), 0);
|
|
69
|
+
if (isToolResult) {
|
|
70
|
+
let name = 'unknown';
|
|
71
|
+
let callId = null;
|
|
72
|
+
for (const block of content) {
|
|
73
|
+
if (block?.type === 'tool-result') {
|
|
74
|
+
callId = block.toolCallId ?? null;
|
|
75
|
+
name = callNames.get(block.toolCallId) ?? 'unknown';
|
|
76
|
+
break;
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
const row = toolResults.get(name) ?? { count: 0, chars: 0, topCallId: null, topChars: -1 };
|
|
80
|
+
row.count += 1;
|
|
81
|
+
row.chars += chars;
|
|
82
|
+
if (chars > row.topChars) {
|
|
83
|
+
row.topChars = chars;
|
|
84
|
+
row.topCallId = callId;
|
|
85
|
+
}
|
|
86
|
+
toolResults.set(name, row);
|
|
87
|
+
} else {
|
|
88
|
+
historyChars += chars;
|
|
89
|
+
historyMessages += 1;
|
|
90
|
+
if (message.role === 'user') userMessages += 1;
|
|
91
|
+
else if (message.role === 'assistant') assistantMessages += 1;
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
const toolResultRows = [...toolResults.entries()]
|
|
95
|
+
.map(([name, row]) => ({
|
|
96
|
+
name,
|
|
97
|
+
count: row.count,
|
|
98
|
+
tokens: estimateTokens(' '.repeat(row.chars)),
|
|
99
|
+
// Opaque provider call id of this tool's LARGEST result — paste it
|
|
100
|
+
// into Trajectory to find the culprit call. No text is retained.
|
|
101
|
+
topCallId: row.topCallId,
|
|
102
|
+
}))
|
|
103
|
+
.sort((a, b) => b.tokens - a.tokens);
|
|
104
|
+
const toolResultTokens = toolResultRows.reduce((sum, r) => sum + r.tokens, 0);
|
|
105
|
+
const entry = {
|
|
106
|
+
at: Date.now(),
|
|
107
|
+
provider: options.provider ?? null,
|
|
108
|
+
model: options.model ?? null,
|
|
109
|
+
purpose: options.purpose ?? null,
|
|
110
|
+
system: estimateTokens(system),
|
|
111
|
+
toolSchemas: estimateTokens(toolsJson),
|
|
112
|
+
toolCount: (options.tools ?? []).length,
|
|
113
|
+
history: estimateTokens(' '.repeat(historyChars)),
|
|
114
|
+
historyMessages,
|
|
115
|
+
userMessages,
|
|
116
|
+
assistantMessages,
|
|
117
|
+
toolResults: toolResultTokens,
|
|
118
|
+
toolResultRows: toolResultRows.slice(0, 8),
|
|
119
|
+
prefixDigest: digest(system + '\u0000' + toolsJson),
|
|
120
|
+
};
|
|
121
|
+
entry.total = entry.system + entry.toolSchemas + entry.history + entry.toolResults;
|
|
122
|
+
return entry;
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
/**
|
|
126
|
+
* Install the ledger observer.
|
|
127
|
+
* @param ctx - the mounted plugin's context.
|
|
128
|
+
* @param options - { maxPerSession } ring bound (default 50).
|
|
129
|
+
* @returns { sessions, dispose } — sessions: Map<sessionId, entries[]>.
|
|
130
|
+
*/
|
|
131
|
+
function installRequestLedger(ctx, options = {}) {
|
|
132
|
+
const maxPerSession = options.maxPerSession ?? 50;
|
|
133
|
+
const sessions = new Map(); // sessionId -> [entry, ...] (oldest first)
|
|
134
|
+
const dispose = ctx.on('llm/stream', (generateOptions, next) => {
|
|
135
|
+
try {
|
|
136
|
+
const key = String(generateOptions.sessionId ?? 'unscoped');
|
|
137
|
+
const entry = classify(generateOptions);
|
|
138
|
+
const ring = sessions.get(key) ?? [];
|
|
139
|
+
const prev = ring[ring.length - 1];
|
|
140
|
+
entry.prefixStable = prev === undefined ? null : prev.prefixDigest === entry.prefixDigest;
|
|
141
|
+
entry.deltaTotal = prev === undefined ? null : entry.total - prev.total;
|
|
142
|
+
ring.push(entry);
|
|
143
|
+
if (ring.length > maxPerSession) ring.shift();
|
|
144
|
+
sessions.set(key, ring);
|
|
145
|
+
// Bound the session table itself: keep the most recently active 20.
|
|
146
|
+
if (sessions.size > 20) {
|
|
147
|
+
const oldest = [...sessions.entries()].sort(
|
|
148
|
+
(a, b) => (a[1][a[1].length - 1]?.at ?? 0) - (b[1][b[1].length - 1]?.at ?? 0),
|
|
149
|
+
)[0];
|
|
150
|
+
if (oldest) sessions.delete(oldest[0]);
|
|
151
|
+
}
|
|
152
|
+
} catch {
|
|
153
|
+
/* the ledger must never break a model call */
|
|
154
|
+
}
|
|
155
|
+
return next();
|
|
156
|
+
});
|
|
157
|
+
return { sessions, dispose };
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
module.exports = { installRequestLedger, classify, blockChars };
|
package/lib/index.js
CHANGED
|
@@ -9,7 +9,15 @@ const {
|
|
|
9
9
|
estimateTokens,
|
|
10
10
|
} = require('./collect/runtime.js');
|
|
11
11
|
const { installSectionAttribution } = require('./collect/attribution.js');
|
|
12
|
-
const {
|
|
12
|
+
const { installRequestLedger } = require('./collect/requests.js');
|
|
13
|
+
const {
|
|
14
|
+
serviceGraph,
|
|
15
|
+
health,
|
|
16
|
+
shadowing,
|
|
17
|
+
contextCost,
|
|
18
|
+
skillCost,
|
|
19
|
+
requestLedger,
|
|
20
|
+
} = require('./model.js');
|
|
13
21
|
|
|
14
22
|
const name = 'dsh-xray';
|
|
15
23
|
|
|
@@ -36,6 +44,7 @@ function apply(ctx) {
|
|
|
36
44
|
let lastAssembly = null; // latest system-prompt assembly observation
|
|
37
45
|
let attribution = null; // section name -> plugin name (live table)
|
|
38
46
|
let skillCatalog = null; // latest skill-cost observation (async, throttled)
|
|
47
|
+
let ledger = null; // per-session request ledger (llm/stream observer)
|
|
39
48
|
let skillRefreshing = false;
|
|
40
49
|
const refreshSkills = (lookup) => {
|
|
41
50
|
// Discovery hits the filesystem and is scope/cwd-sensitive: the
|
|
@@ -66,6 +75,7 @@ function apply(ctx) {
|
|
|
66
75
|
snap.sectionOwners = attribution ? Object.fromEntries(attribution.table) : {};
|
|
67
76
|
snap.toolOwners = attribution ? Object.fromEntries(attribution.toolTable) : {};
|
|
68
77
|
snap.skillCatalog = skillCatalog;
|
|
78
|
+
snap.requestLedger = ledger ? Object.fromEntries(ledger.sessions) : {};
|
|
69
79
|
fs.mkdirSync(dir, { recursive: true });
|
|
70
80
|
const tmp = `${file}.tmp`;
|
|
71
81
|
fs.writeFileSync(tmp, JSON.stringify(snap, null, 2));
|
|
@@ -125,12 +135,15 @@ function apply(ctx) {
|
|
|
125
135
|
// Section attribution: diff-based name->plugin table over
|
|
126
136
|
// system-prompt/change (see collect/attribution.js for the strategy).
|
|
127
137
|
attribution = installSectionAttribution(ctx);
|
|
138
|
+
ledger = installRequestLedger(ctx);
|
|
128
139
|
return [
|
|
129
140
|
disposeStatus,
|
|
130
141
|
disposeAssemble,
|
|
131
142
|
() => {
|
|
132
143
|
attribution.dispose();
|
|
133
144
|
attribution = null;
|
|
145
|
+
ledger.dispose();
|
|
146
|
+
ledger = null;
|
|
134
147
|
},
|
|
135
148
|
() => {
|
|
136
149
|
clearTimeout(timer);
|
|
@@ -153,6 +166,7 @@ function apply(ctx) {
|
|
|
153
166
|
snap.sectionOwners = attribution ? Object.fromEntries(attribution.table) : {};
|
|
154
167
|
snap.toolOwners = attribution ? Object.fromEntries(attribution.toolTable) : {};
|
|
155
168
|
snap.skillCatalog = skillCatalog;
|
|
169
|
+
snap.requestLedger = ledger ? Object.fromEntries(ledger.sessions) : {};
|
|
156
170
|
return snap;
|
|
157
171
|
};
|
|
158
172
|
wctx.effect(
|
|
@@ -175,6 +189,7 @@ function apply(ctx) {
|
|
|
175
189
|
cost: () => contextCost(freshSnap()),
|
|
176
190
|
shadow: () => shadowing(freshSnap()),
|
|
177
191
|
skills: () => skillCost(freshSnap()),
|
|
192
|
+
requests: () => requestLedger(freshSnap()),
|
|
178
193
|
},
|
|
179
194
|
// Entry text is computed per request from the live registries and
|
|
180
195
|
// never persisted — the audit answer to "what exactly is ~N tokens?".
|
|
@@ -219,7 +234,8 @@ function apply(ctx) {
|
|
|
219
234
|
parameters: {
|
|
220
235
|
view: {
|
|
221
236
|
type: 'string',
|
|
222
|
-
description:
|
|
237
|
+
description:
|
|
238
|
+
'summary | deps | health | cost | shadow | skills | requests (default summary)',
|
|
223
239
|
},
|
|
224
240
|
},
|
|
225
241
|
output: {
|
|
@@ -233,11 +249,13 @@ function apply(ctx) {
|
|
|
233
249
|
snap.sectionOwners = attribution ? Object.fromEntries(attribution.table) : {};
|
|
234
250
|
snap.toolOwners = attribution ? Object.fromEntries(attribution.toolTable) : {};
|
|
235
251
|
snap.skillCatalog = skillCatalog;
|
|
252
|
+
snap.requestLedger = ledger ? Object.fromEntries(ledger.sessions) : {};
|
|
236
253
|
if (args.view === 'deps') return serviceGraph(snap);
|
|
237
254
|
if (args.view === 'health') return health(snap);
|
|
238
255
|
if (args.view === 'cost') return contextCost(snap);
|
|
239
256
|
if (args.view === 'shadow') return shadowing(snap);
|
|
240
257
|
if (args.view === 'skills') return skillCost(snap);
|
|
258
|
+
if (args.view === 'requests') return requestLedger(snap);
|
|
241
259
|
return {
|
|
242
260
|
plugins: snap.plugins.length,
|
|
243
261
|
unhealthy: health(snap).unhealthy.length,
|
package/lib/model.js
CHANGED
|
@@ -377,6 +377,41 @@ function skillCost(snap) {
|
|
|
377
377
|
};
|
|
378
378
|
}
|
|
379
379
|
|
|
380
|
+
/** F11: per-request ledger — one classified bill per LLM call, newest
|
|
381
|
+
* first, with prefix-stability and delta markers. The time axis the static
|
|
382
|
+
* cost view lacks. */
|
|
383
|
+
function requestLedger(snap) {
|
|
384
|
+
const observed = snap.requestLedger;
|
|
385
|
+
if (!observed || Object.keys(observed).length === 0)
|
|
386
|
+
return { available: false, sessions: [], capturedAt: snap.capturedAt };
|
|
387
|
+
const sessions = Object.entries(observed)
|
|
388
|
+
.map(([sessionId, entries]) => ({
|
|
389
|
+
sessionId,
|
|
390
|
+
lastAt: entries[entries.length - 1]?.at ?? 0,
|
|
391
|
+
requests: entries
|
|
392
|
+
.slice()
|
|
393
|
+
.reverse()
|
|
394
|
+
.map((e, i, arr) => ({
|
|
395
|
+
seq: entries.length - i,
|
|
396
|
+
at: e.at,
|
|
397
|
+
purpose: e.purpose,
|
|
398
|
+
provider: e.provider,
|
|
399
|
+
model: e.model,
|
|
400
|
+
total: e.total,
|
|
401
|
+
system: e.system,
|
|
402
|
+
toolSchemas: e.toolSchemas,
|
|
403
|
+
history: e.history,
|
|
404
|
+
toolResults: e.toolResults,
|
|
405
|
+
toolResultRows: e.toolResultRows,
|
|
406
|
+
historyMessages: e.historyMessages,
|
|
407
|
+
prefixStable: e.prefixStable,
|
|
408
|
+
deltaTotal: e.deltaTotal,
|
|
409
|
+
})),
|
|
410
|
+
}))
|
|
411
|
+
.sort((a, b) => b.lastAt - a.lastAt);
|
|
412
|
+
return { available: true, sessions, capturedAt: snap.capturedAt };
|
|
413
|
+
}
|
|
414
|
+
|
|
380
415
|
module.exports = {
|
|
381
416
|
replayLayers,
|
|
382
417
|
attribute,
|
|
@@ -388,4 +423,5 @@ module.exports = {
|
|
|
388
423
|
shadowing,
|
|
389
424
|
contextCost,
|
|
390
425
|
skillCost,
|
|
426
|
+
requestLedger,
|
|
391
427
|
};
|
package/lib/panel.js
CHANGED
|
@@ -38,7 +38,7 @@ const PAGE = `<!doctype html>
|
|
|
38
38
|
<div id="status"></div>
|
|
39
39
|
<div id="content"></div>
|
|
40
40
|
<script>
|
|
41
|
-
const views = ['summary', 'health', 'deps', 'cost', 'shadow', 'skills'];
|
|
41
|
+
const views = ['summary', 'health', 'deps', 'cost', 'shadow', 'skills', 'requests'];
|
|
42
42
|
const esc = (s) => String(s ?? '').replace(/[&<>]/g, (c) => ({'&':'&','<':'<','>':'>'}[c]));
|
|
43
43
|
const escAttr = (s) => esc(s).replace(/"/g, '"');
|
|
44
44
|
|
|
@@ -50,6 +50,7 @@ const INTRO = {
|
|
|
50
50
|
cost: 'What every LLM request carries before your message: prompt sections + tool schemas, attributed to the plugin that registered each. "By plugin" is each plugin\\'s per-request context tax.',
|
|
51
51
|
shadow: 'Same-name registrations. A service provided by two plugins means one silently wins — usually intended (an override), occasionally a conflict.',
|
|
52
52
|
skills: 'What each skill costs: its catalog line rides every request once any model-invocable skill exists; its body is billed only when loaded. Pricing only — to enable/disable skills, use a skill manager.',
|
|
53
|
+
requests: 'One bill per LLM call, newest first: system + tool schemas + history + tool results. \u26a1 = the system/tools prefix matched the previous request (cache-friendly); \u2702 = the prefix changed. \u0394 is the total against the previous request.',
|
|
53
54
|
};
|
|
54
55
|
|
|
55
56
|
// Hover glossary: term -> plain-language meaning (native title tooltips).
|
|
@@ -69,6 +70,10 @@ const TIPS = {
|
|
|
69
70
|
catalog: 'Tokens of this skill's line in the durable session catalog, carried on every request',
|
|
70
71
|
body: 'Tokens of the full rendered skill body, billed per load and then resident in history',
|
|
71
72
|
invocable: 'Whether the model may load this skill itself; user-only skills stay out of the catalog',
|
|
73
|
+
total: 'Estimated tokens of the whole request payload (~4 chars/token)',
|
|
74
|
+
history: 'Conversation messages (user + assistant), excluding tool results',
|
|
75
|
+
results: 'Tool-result messages, aggregated per tool below each row',
|
|
76
|
+
prefix: 'Whether system prompt + tool schemas were byte-identical to the previous request in this session',
|
|
72
77
|
};
|
|
73
78
|
const th = (h) => '<th' + (TIPS[h] ? ' title="' + escAttr(TIPS[h]) + '"' : '') + '>' + esc(h) + '</th>';
|
|
74
79
|
const intro = (v) => '<p class="muted" style="margin:0 0 10px">' + INTRO[v] + '</p>';
|
|
@@ -181,6 +186,30 @@ const renderers = {
|
|
|
181
186
|
+ '</td><td>' + (s.modelInvocable ? '<span class="ok">yes</span>' : '<span class="muted">user-only</span>') + '</td></tr>'));
|
|
182
187
|
return html;
|
|
183
188
|
},
|
|
189
|
+
requests(d) {
|
|
190
|
+
if (!d.available) return '<p class="muted">no requests observed yet — send one agent message first</p>';
|
|
191
|
+
let html = '';
|
|
192
|
+
for (const session of d.sessions) {
|
|
193
|
+
html += '<h3 style="margin:12px 0 4px">session ' + esc(String(session.sessionId).slice(0, 12))
|
|
194
|
+
+ ' <span class="muted">(' + session.requests.length + ' request(s))</span></h3>';
|
|
195
|
+
html += table(['#', 'when', 'purpose', 'total', 'system', 'tools', 'history', 'results', 'prefix', 'Δ'],
|
|
196
|
+
session.requests.map((r) => {
|
|
197
|
+
const when = new Date(r.at).toTimeString().slice(0, 8);
|
|
198
|
+
const purpose = r.purpose ?? 'chat';
|
|
199
|
+
const prefix = r.prefixStable === null ? '—' : r.prefixStable ? '⚡' : '✂';
|
|
200
|
+
const delta = r.deltaTotal === null ? '—'
|
|
201
|
+
: (r.deltaTotal >= 0 ? '+' : '') + r.deltaTotal;
|
|
202
|
+
const top = (r.toolResultRows ?? []).slice(0, 3).map((t) => esc(t.name) + ' ~' + t.tokens).join(', ');
|
|
203
|
+
return '<tr><td class="num">' + r.seq + '</td><td class="muted">' + when
|
|
204
|
+
+ '</td><td>' + (purpose === 'chat' ? 'chat' : '<span class="muted">' + esc(purpose) + '</span>')
|
|
205
|
+
+ '</td><td class="num">~' + r.total + '</td><td class="num">' + r.system
|
|
206
|
+
+ '</td><td class="num">' + r.toolSchemas + '</td><td class="num">' + r.history
|
|
207
|
+
+ '</td><td class="num" title="' + escAttr(top) + '">' + r.toolResults
|
|
208
|
+
+ '</td><td>' + prefix + '</td><td class="num">' + delta + '</td></tr>';
|
|
209
|
+
}));
|
|
210
|
+
}
|
|
211
|
+
return html;
|
|
212
|
+
},
|
|
184
213
|
};
|
|
185
214
|
|
|
186
215
|
async function render() {
|