dsh-context-truth 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README-en.md +10 -7
- package/README.md +16 -7
- package/lib/index.js +90 -24
- package/package.json +1 -1
- package/test/index.test.js +132 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.3
|
|
4
|
+
|
|
5
|
+
Four changes, all from capturing a real compaction request in a live session (and from the host
|
|
6
|
+
log line that capture explained):
|
|
7
|
+
|
|
8
|
+
- **A request that merely quotes the directive is no longer mistaken for a compaction call.**
|
|
9
|
+
Detection trusted a substring match on the trailing message, so an ordinary turn whose last tool
|
|
10
|
+
result dumped the engine source or a session log was treated as the summarizer call — one real
|
|
11
|
+
tool result contained 10 marker hits. A caller-set `purpose` is now authoritative, and the
|
|
12
|
+
tagless fallback requires the trailing message to *begin* with the directive.
|
|
13
|
+
- **A look-alike can no longer spend the one-shot failure warning.** In the wild it did: the
|
|
14
|
+
warning fired for an ordinary turn, which would have left a later *real* failure completely
|
|
15
|
+
silent. Only a call tagged `purpose: 'compaction'` may consume the warning now.
|
|
16
|
+
- **A reading below one display step says so instead of rendering `0`.** After a compaction the
|
|
17
|
+
meter reads 30–50K against a 50K step, so the line printed `0 / 1.00M tokens used (0%)` while
|
|
18
|
+
the same line reported ~600K of headroom — an empty window and a nearly full one at once. It now
|
|
19
|
+
prints `under 50K / 1.00M tokens used (under 5%)` (`不足 50K`), and a headroom inside its own
|
|
20
|
+
step prints `Under 2K` rather than `less than 1K`.
|
|
21
|
+
- **Rule 1 forbids quoting, not just asserting.** It now bans recording such claims quoted,
|
|
22
|
+
paraphrased, or listed as an example, and tells the summarizer to drop the wording when an
|
|
23
|
+
earlier checkpoint already carries it. The rule embeds no quotable example phrases of its own.
|
|
24
|
+
- 28 unit tests (was 21), including the real look-alike request shape and the sub-step rendering.
|
|
25
|
+
|
|
3
26
|
## 0.1.2
|
|
4
27
|
|
|
5
28
|
- Maintenance release: identical code and docs to 0.1.0. Published directly because npm staged
|
package/README-en.md
CHANGED
|
@@ -35,21 +35,24 @@ you work. This host-measured line is the only authority on context pressure; nev
|
|
|
35
35
|
estimate it from the conversation's length.
|
|
36
36
|
```
|
|
37
37
|
|
|
38
|
-
Four bands (`ample` / `moderate` / `tight` / `critical`) keyed to the **auto-compaction point**, with 5% hysteresis. The `ample` band additionally declares that every earlier "context is nearly full" claim — including ones inside compacted checkpoints — is stale.
|
|
38
|
+
Four bands (`ample` / `moderate` / `tight` / `critical`) keyed to the **auto-compaction point**, with 5% hysteresis. The `ample` band additionally declares that every earlier "context is nearly full" claim — including ones inside compacted checkpoints — is stale. A reading that rounds below one display step renders as `under 50K / 1.00M tokens used (under 5%)` — never as a `0` that would contradict the headroom printed on the same line.
|
|
39
39
|
|
|
40
40
|
### 2. Compaction directive rules (on by default)
|
|
41
41
|
|
|
42
|
-
The last message of a compaction call
|
|
42
|
+
The last message of a compaction call is the summarizer directive. Through the **public `llm/stream` waterfall** the plugin appends two rules to its tail — tail-only, so the cached prefix stays byte-identical:
|
|
43
43
|
|
|
44
44
|
```
|
|
45
45
|
- Never record the assistant's own statements about its remaining context, token budget, or
|
|
46
|
-
context-window pressure
|
|
47
|
-
|
|
48
|
-
|
|
46
|
+
context-window pressure — not even quoted, paraphrased, or listed as an example, and not
|
|
47
|
+
when an earlier checkpoint already carries one: drop that wording instead of copying it
|
|
48
|
+
forward. Those readings are measured by the host and delivered separately; a model's guess
|
|
49
|
+
about them is not a durable fact.
|
|
49
50
|
- Never record a plan to stop, defer, or hand off work whose only justification was context
|
|
50
51
|
pressure. Record the actual task state instead.
|
|
51
52
|
```
|
|
52
53
|
|
|
54
|
+
The call is identified by the `purpose: 'compaction'` tag its caller sets. When a deployment sets no tag, the fallback requires the trailing message to **begin** with the directive — so a tool result that merely quotes the engine (a source dump, a log excerpt, a grep hit) is never patched by mistake, and can never spend the one-shot failure warning either.
|
|
55
|
+
|
|
53
56
|
## Data sources (all host-measured — nothing is estimated)
|
|
54
57
|
|
|
55
58
|
| Item | Source |
|
|
@@ -116,14 +119,14 @@ What you get for it: the model stops halting or handing off because it *thinks*
|
|
|
116
119
|
|
|
117
120
|
* Measured on **dsh 0.2.0-rc.2**; only public seams are used (`systemPrompt.variable/context`, the `llm/stream` waterfall, the `tokenMeter` / `sessionProjections` services) — no private fields.
|
|
118
121
|
* If an agent preset sets `includeRuntimeContext: false` (for example `liangshen`), the runtime suppresses **every** runtime-context contribution and capability 1 does not apply (capability 2 is unaffected).
|
|
119
|
-
* If a host froze the request envelope (this version does not), capability 2 logs one warning and skips — it never breaks a model call.
|
|
122
|
+
* If a host froze the request envelope (**this version does not** — verified by capturing a real compaction request), capability 2 logs one warning and skips — it never breaks a model call. That warning is reserved for the engine's own tagged call, so a look-alike request can never silence a later real failure.
|
|
120
123
|
* **Mounting twice in one scope does not fail**: bundle assembly plus a runtime injection are two assembly paths for the same plugin; the second instance detects the taken name, stays silent (one `info` line) and the first keeps serving — it never becomes a failed plugin row.
|
|
121
124
|
* The plugin registers no tools and does not touch the tool catalog, so it adds nothing to the first-turn prefill.
|
|
122
125
|
|
|
123
126
|
## Verify
|
|
124
127
|
|
|
125
128
|
```sh
|
|
126
|
-
npm test #
|
|
129
|
+
npm test # 28 unit tests: banding, hysteresis, quantization, sub-step rendering, degradation, directive patching, look-alike rejection, idempotence
|
|
127
130
|
```
|
|
128
131
|
|
|
129
132
|
After installing, open a new session and send one message: expanding that turn's runtime context block should show `Context occupancy (host-measured): …`, and the log carries `context-truth: none -> ample (used=… window=1000000 compactionAt=678464)`.
|
package/README.md
CHANGED
|
@@ -38,21 +38,28 @@ estimate it from the conversation's length.
|
|
|
38
38
|
|
|
39
39
|
四档语气(`ample` / `moderate` / `tight` / `critical`)按**自动压缩阈值**的比例分档,
|
|
40
40
|
带 5% 迟滞;`ample` 档额外声明"此前所有'上下文快满了'的说法(含压缩检查点里的)都已作废"。
|
|
41
|
+
读数不足一个显示台阶时渲染为 `under 50K / 1.00M tokens used (under 5%)`——不会渲染成 `0`,
|
|
42
|
+
那会和同一行里的剩余量自相矛盾。
|
|
41
43
|
|
|
42
44
|
### 2. 压缩指令补规则(默认开)
|
|
43
45
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
+
压缩摘要调用的最后一条消息就是摘要指令。本插件通过**公开的 `llm/stream` 瀑布**在它末尾追加
|
|
47
|
+
两条规则,只改尾巴、不动被缓存的前缀:
|
|
46
48
|
|
|
47
49
|
```
|
|
48
50
|
- Never record the assistant's own statements about its remaining context, token budget, or
|
|
49
|
-
context-window pressure
|
|
50
|
-
|
|
51
|
-
|
|
51
|
+
context-window pressure — not even quoted, paraphrased, or listed as an example, and not
|
|
52
|
+
when an earlier checkpoint already carries one: drop that wording instead of copying it
|
|
53
|
+
forward. Those readings are measured by the host and delivered separately; a model's guess
|
|
54
|
+
about them is not a durable fact.
|
|
52
55
|
- Never record a plan to stop, defer, or hand off work whose only justification was context
|
|
53
56
|
pressure. Record the actual task state instead.
|
|
54
57
|
```
|
|
55
58
|
|
|
59
|
+
识别依据是调用方打的 `purpose: 'compaction'` 标记;部署没有打标记时,兜底判据要求末尾那条
|
|
60
|
+
消息**以指令开头**——所以"只是引用了引擎"的工具结果(源码转储、日志片段、grep 命中)不会被
|
|
61
|
+
误改,也不会吃掉那条一次性失败告警。
|
|
62
|
+
|
|
56
63
|
## 数据来源(都是宿主的真实数字,没有估算)
|
|
57
64
|
|
|
58
65
|
| 项目 | 来源 |
|
|
@@ -120,7 +127,9 @@ runtime-context 快照**字节不变就不会重发**,所以读数按台阶量
|
|
|
120
127
|
`tokenMeter` / `sessionProjections` 服务),不读私有字段。
|
|
121
128
|
* 若某个 agent preset 设置了 `includeRuntimeContext: false`(例如 `liangshen`),运行时会抑制
|
|
122
129
|
**所有** runtime-context 贡献,此时第 1 个能力不生效(第 2 个不受影响)。
|
|
123
|
-
*
|
|
130
|
+
* 若宿主把请求信封冻结(**本版没有**——已通过抓取真实压缩请求验证),第 2 个能力会打一条
|
|
131
|
+
warn 并跳过,绝不阻断模型调用。该告警只为引擎自己打了标记的调用保留,误报的"形似请求"
|
|
132
|
+
不可能吃掉它、让后面真失败变成静默。
|
|
124
133
|
* **同一作用域挂载两次不会失败**:bundle 装配 + 运行时注入是同一插件的两条装配路径,
|
|
125
134
|
第二个实例检测到重名后保持静默(只记一条 info),由第一个实例继续服务——不会变成
|
|
126
135
|
一行 failed 的插件记录。
|
|
@@ -129,7 +138,7 @@ runtime-context 快照**字节不变就不会重发**,所以读数按台阶量
|
|
|
129
138
|
## 验证
|
|
130
139
|
|
|
131
140
|
```sh
|
|
132
|
-
npm test #
|
|
141
|
+
npm test # 28 项单测:分档/迟滞/量化/不足一档的渲染/降级/摘要改写/形似请求排除/幂等/重复挂载
|
|
133
142
|
```
|
|
134
143
|
|
|
135
144
|
装好后开一个新会话发一条消息,展开该轮的 runtime context 块应看到
|
package/lib/index.js
CHANGED
|
@@ -37,6 +37,13 @@
|
|
|
37
37
|
* the public `llm/stream` waterfall. Only the tail changes: the replayed
|
|
38
38
|
* conversation prefix — the part the provider caches — stays byte-identical.
|
|
39
39
|
*
|
|
40
|
+
* The call is identified by that `purpose` tag whenever the caller sets one; the
|
|
41
|
+
* directive-text fallback demands that the trailing message *begin* with the
|
|
42
|
+
* directive, so a tool result that merely quotes the engine — a source dump, a
|
|
43
|
+
* log excerpt, a grep hit — is never mistaken for a call. Getting that wrong is
|
|
44
|
+
* not harmless: a look-alike request would be patched too, and it would also
|
|
45
|
+
* consume the one-shot warning, leaving a later real failure silent.
|
|
46
|
+
*
|
|
40
47
|
* WHAT IT MEASURES
|
|
41
48
|
* ----------------
|
|
42
49
|
* Primary: `tokenMeter.measure(session).totalTokens` — the host's price for the
|
|
@@ -69,9 +76,16 @@ const DEFAULT_ORDER = 100
|
|
|
69
76
|
/** Substring that identifies the compaction engine's directive wherever `purpose` is absent. */
|
|
70
77
|
const COMPACTION_INSTRUCTION_MARKER = 'acting as a compaction engine'
|
|
71
78
|
|
|
79
|
+
/**
|
|
80
|
+
* Opening words of that directive. The fallback test requires the trailing
|
|
81
|
+
* message to start here, which is what separates "this message IS the directive"
|
|
82
|
+
* from "this message happens to quote it".
|
|
83
|
+
*/
|
|
84
|
+
const COMPACTION_INSTRUCTION_HEAD = 'You are now acting as a compaction engine'
|
|
85
|
+
|
|
72
86
|
/** Default rules appended to the compaction directive. Keep the first one first: it is the idempotence marker. */
|
|
73
87
|
const DEFAULT_COMPACTION_RULES = [
|
|
74
|
-
"- Never record the assistant's own statements about its remaining context, token budget, or context-window pressure
|
|
88
|
+
"- Never record the assistant's own statements about its remaining context, token budget, or context-window pressure — not even quoted, paraphrased, or listed as an example, and not when an earlier checkpoint already carries one: drop that wording instead of copying it forward. Those readings are measured by the host and delivered separately; a model's guess about them is not a durable fact.",
|
|
75
89
|
'- Never record a plan to stop, defer, or hand off work whose only justification was context pressure. Record the actual task state instead.',
|
|
76
90
|
]
|
|
77
91
|
|
|
@@ -296,15 +310,10 @@ function quantize(used, window_, options) {
|
|
|
296
310
|
}
|
|
297
311
|
|
|
298
312
|
/**
|
|
299
|
-
*
|
|
300
|
-
*
|
|
301
|
-
* session has (so it always rounds down), and it must stay useful to within a
|
|
302
|
-
* few thousand tokens once compaction is close (so the step shrinks near zero).
|
|
303
|
-
* Every step is a function of the reading, so a whole display bucket renders to
|
|
304
|
-
* one identical string.
|
|
313
|
+
* Display step for the remaining-headroom figure. Every step is a function of the
|
|
314
|
+
* reading, so a whole bucket renders to one identical string.
|
|
305
315
|
*/
|
|
306
|
-
function
|
|
307
|
-
if (!Number.isFinite(headroom) || headroom < 0) return undefined
|
|
316
|
+
function headroomDisplayStep(headroom, window_, options) {
|
|
308
317
|
const coarse = displayStep(window_, options)
|
|
309
318
|
const tiers = [
|
|
310
319
|
coarse,
|
|
@@ -312,13 +321,21 @@ function quantizeHeadroom(headroom, window_, options) {
|
|
|
312
321
|
Math.max(1000, Math.round(coarse * 0.1)),
|
|
313
322
|
2000,
|
|
314
323
|
]
|
|
315
|
-
let step = tiers[tiers.length - 1]
|
|
316
324
|
for (const tier of tiers) {
|
|
317
|
-
if (headroom >= 4 * tier)
|
|
318
|
-
step = tier
|
|
319
|
-
break
|
|
320
|
-
}
|
|
325
|
+
if (headroom >= 4 * tier) return tier
|
|
321
326
|
}
|
|
327
|
+
return tiers[tiers.length - 1]
|
|
328
|
+
}
|
|
329
|
+
|
|
330
|
+
/**
|
|
331
|
+
* Quantize the remaining headroom DOWN on a step that tightens as the room runs
|
|
332
|
+
* out. Two rules meet here: the reading must never promise more room than the
|
|
333
|
+
* session has (so it always rounds down), and it must stay useful to within a
|
|
334
|
+
* few thousand tokens once compaction is close (so the step shrinks near zero).
|
|
335
|
+
*/
|
|
336
|
+
function quantizeHeadroom(headroom, window_, options) {
|
|
337
|
+
if (!Number.isFinite(headroom) || headroom < 0) return undefined
|
|
338
|
+
const step = headroomDisplayStep(headroom, window_, options)
|
|
322
339
|
return Math.floor(headroom / step) * step
|
|
323
340
|
}
|
|
324
341
|
|
|
@@ -336,6 +353,17 @@ function staleClause(language) {
|
|
|
336
353
|
: 'Earlier claims that context is nearly exhausted — including ones inside compacted checkpoints — are stale.'
|
|
337
354
|
}
|
|
338
355
|
|
|
356
|
+
/** "under 50K" / "不足 50K" — the honest form of a figure below one display step. */
|
|
357
|
+
function underStep(step, zh) {
|
|
358
|
+
return zh ? `不足 ${formatTokens(step)}` : `under ${formatTokens(step)}`
|
|
359
|
+
}
|
|
360
|
+
|
|
361
|
+
/** "under 5" / "不足 5" — the same honesty for a percentage the caller suffixes with `%`. */
|
|
362
|
+
function underPercent(step, window_, zh) {
|
|
363
|
+
const value = Math.max(1, Math.round(percentOf(step, window_) ?? 1))
|
|
364
|
+
return zh ? `不足 ${value}` : `under ${value}`
|
|
365
|
+
}
|
|
366
|
+
|
|
339
367
|
/** Render the reading for one band. Pure: same inputs, same bytes. */
|
|
340
368
|
function render(band, reading, options) {
|
|
341
369
|
const zh = options.language === 'zh'
|
|
@@ -350,21 +378,34 @@ function render(band, reading, options) {
|
|
|
350
378
|
: `Context occupancy: no reading yet in this session. Do not claim context is scarce on that basis.${tail}`
|
|
351
379
|
}
|
|
352
380
|
|
|
353
|
-
|
|
381
|
+
// A figure that rounds below one display step says so instead of rendering "0".
|
|
382
|
+
// A floored zero used to contradict the headroom on the same line — the reading
|
|
383
|
+
// was 30–50K against a 50K step — and read as an empty window.
|
|
384
|
+
const step = Number.isFinite(reading.step) && reading.step > 0 ? reading.step : undefined
|
|
385
|
+
const flooredUsed = reading.quantized === 0 && Number.isFinite(reading.used) && reading.used > 0
|
|
386
|
+
const used = flooredUsed && step !== undefined ? underStep(step, zh) : formatTokens(reading.quantized)
|
|
354
387
|
const window_ = formatTokens(reading.window)
|
|
355
|
-
const percent =
|
|
388
|
+
const percent =
|
|
389
|
+
flooredUsed && step !== undefined ? underPercent(step, reading.window, zh) : `${reading.percent}`
|
|
356
390
|
const head = reading.headroom
|
|
391
|
+
const headStep =
|
|
392
|
+
Number.isFinite(reading.headroomStep) && reading.headroomStep > 0 ? reading.headroomStep : undefined
|
|
393
|
+
const flooredHead = head === 0 && Number.isFinite(reading.headroomRaw) && reading.headroomRaw > 0
|
|
357
394
|
const point = Number.isFinite(reading.threshold)
|
|
358
395
|
? zh
|
|
359
396
|
? ` 自动压缩阈值 ${formatTokens(reading.threshold)}(窗口的 ${percentOf(reading.threshold, reading.window)}%)。`
|
|
360
397
|
: ` Automatic compaction fires at ${formatTokens(reading.threshold)} (${percentOf(reading.threshold, reading.window)}% of the window).`
|
|
361
398
|
: ''
|
|
362
399
|
const remaining =
|
|
363
|
-
head
|
|
364
|
-
?
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
400
|
+
head === undefined
|
|
401
|
+
? ''
|
|
402
|
+
: flooredHead && headStep !== undefined
|
|
403
|
+
? zh
|
|
404
|
+
? ` 距阈值不足 ${formatTokens(headStep)} tokens。`
|
|
405
|
+
: ` Under ${formatTokens(headStep)} tokens of headroom remain before it.`
|
|
406
|
+
: zh
|
|
407
|
+
? ` 距阈值还有约 ${formatTokens(head)} tokens。`
|
|
408
|
+
: ` About ${formatTokens(head)} tokens of headroom remain before it.`
|
|
368
409
|
const head0 = zh
|
|
369
410
|
? `上下文实况(宿主实测):已用 ${used} / ${window_}(${percent}%)。`
|
|
370
411
|
: `Context occupancy (host-measured): ${used} / ${window_} tokens used (${percent}%).`
|
|
@@ -422,6 +463,12 @@ function patchCompactionInstruction(text, rules) {
|
|
|
422
463
|
|
|
423
464
|
/**
|
|
424
465
|
* Test whether one LLM request is the compaction summarization call.
|
|
466
|
+
*
|
|
467
|
+
* A tag set by the caller is authoritative and is never second-guessed. Without
|
|
468
|
+
* one, the trailing message must *be* the directive: a message that merely
|
|
469
|
+
* contains the marker somewhere — a tool result dumping the engine source or a
|
|
470
|
+
* session log is the common case — is an ordinary request.
|
|
471
|
+
*
|
|
425
472
|
* @param options - request envelope observed at the `llm/stream` waterfall.
|
|
426
473
|
* @returns whether the request carries the compaction directive.
|
|
427
474
|
*/
|
|
@@ -429,9 +476,18 @@ function isCompactionRequest(options) {
|
|
|
429
476
|
if (options === null || typeof options !== 'object') return false
|
|
430
477
|
const messages = options.messages
|
|
431
478
|
if (!Array.isArray(messages) || messages.length === 0) return false
|
|
479
|
+
if (options.purpose !== undefined) return options.purpose === 'compaction'
|
|
432
480
|
const text = messageText(messages[messages.length - 1])
|
|
433
|
-
|
|
434
|
-
|
|
481
|
+
return typeof text === 'string' && text.trimStart().startsWith(COMPACTION_INSTRUCTION_HEAD)
|
|
482
|
+
}
|
|
483
|
+
|
|
484
|
+
/**
|
|
485
|
+
* Whether the request identifies itself as the engine's call. Used only to decide
|
|
486
|
+
* who may spend the one-shot warning: a heuristic match must never be able to
|
|
487
|
+
* silence a later, real failure.
|
|
488
|
+
*/
|
|
489
|
+
function isTaggedCompactionRequest(options) {
|
|
490
|
+
return options !== null && typeof options === 'object' && options.purpose === 'compaction'
|
|
435
491
|
}
|
|
436
492
|
|
|
437
493
|
/**
|
|
@@ -500,7 +556,12 @@ function installCompactionPatch(ctx, options) {
|
|
|
500
556
|
let warned = false
|
|
501
557
|
ctx.on('llm/stream', (request, next) => {
|
|
502
558
|
try {
|
|
503
|
-
|
|
559
|
+
const applied = patchCompactionRequest(request, options.compactionRules)
|
|
560
|
+
// Only a request its own caller tagged as compaction may spend the one-shot
|
|
561
|
+
// warning. Warning on a heuristic match let an ordinary turn — one whose
|
|
562
|
+
// trailing tool result quoted the directive — burn the warning, and a real
|
|
563
|
+
// failure after that would have been completely silent.
|
|
564
|
+
if (!applied && isTaggedCompactionRequest(request) && !warned) {
|
|
504
565
|
warned = true
|
|
505
566
|
const logger = ctx.logger
|
|
506
567
|
if (logger !== undefined && typeof logger.warn === 'function') {
|
|
@@ -572,7 +633,10 @@ function installContextLine(ctx, options) {
|
|
|
572
633
|
band,
|
|
573
634
|
{
|
|
574
635
|
quantized,
|
|
636
|
+
step: displayStep(window_, options),
|
|
575
637
|
headroom,
|
|
638
|
+
headroomStep:
|
|
639
|
+
headroomRaw === undefined ? undefined : headroomDisplayStep(headroomRaw, window_, options),
|
|
576
640
|
headroomRaw,
|
|
577
641
|
used: reading.used,
|
|
578
642
|
window: window_,
|
|
@@ -608,6 +672,7 @@ export const internals = {
|
|
|
608
672
|
settleBand,
|
|
609
673
|
quantize,
|
|
610
674
|
quantizeHeadroom,
|
|
675
|
+
headroomDisplayStep,
|
|
611
676
|
displayStep,
|
|
612
677
|
render,
|
|
613
678
|
renderUnknown,
|
|
@@ -615,4 +680,5 @@ export const internals = {
|
|
|
615
680
|
patchCompactionInstruction,
|
|
616
681
|
patchCompactionRequest,
|
|
617
682
|
isCompactionRequest,
|
|
683
|
+
isTaggedCompactionRequest,
|
|
618
684
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-context-truth",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "DeepSeek Harness plugin: tells the model its real, host-measured context occupancy every turn — and keeps compaction summaries from carrying false \"context is nearly exhausted\" claims.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"deepseek-harness",
|
package/test/index.test.js
CHANGED
|
@@ -3,7 +3,7 @@ import test from 'node:test'
|
|
|
3
3
|
|
|
4
4
|
import { apply, internals, name, inject, CONTEXT_NAME, VARIABLE_NAME } from '../lib/index.js'
|
|
5
5
|
|
|
6
|
-
const { resolveOptions, readOccupancy, settleBand, quantize, quantizeHeadroom, render, patchCompactionInstruction, patchCompactionRequest, isCompactionRequest, messageText, DEFAULT_OPTIONS } = internals
|
|
6
|
+
const { resolveOptions, readOccupancy, settleBand, quantize, quantizeHeadroom, headroomDisplayStep, render, patchCompactionInstruction, patchCompactionRequest, isCompactionRequest, isTaggedCompactionRequest, messageText, DEFAULT_OPTIONS, DEFAULT_COMPACTION_RULES } = internals
|
|
7
7
|
|
|
8
8
|
/** A 1M window reserving the DeepSeek adapter's default 256K completion tokens. */
|
|
9
9
|
const WINDOW = 1000000
|
|
@@ -329,3 +329,134 @@ test('the rules can be replaced or switched off', () => {
|
|
|
329
329
|
assert.equal(registered.listeners['llm/stream'], undefined)
|
|
330
330
|
assert.ok(registered.variable, 'the measured line stays installed')
|
|
331
331
|
})
|
|
332
|
+
|
|
333
|
+
test('a request that merely quotes the directive is not a compaction call', () => {
|
|
334
|
+
const rules = resolveOptions({}).compactionRules
|
|
335
|
+
// The shape that produced a false positive in the wild: an ordinary user turn
|
|
336
|
+
// whose trailing tool result dumped the engine's own directive text.
|
|
337
|
+
const lookAlike = {
|
|
338
|
+
messages: [
|
|
339
|
+
{ role: 'user', content: [{ type: 'text', text: 'grep the engine for the marker' }] },
|
|
340
|
+
{
|
|
341
|
+
role: 'user',
|
|
342
|
+
content: [
|
|
343
|
+
{
|
|
344
|
+
type: 'text',
|
|
345
|
+
text: `--- hit#1 type=tool/result at=1200/9000\n${INSTRUCTION}\n--- total hits: 10`,
|
|
346
|
+
},
|
|
347
|
+
],
|
|
348
|
+
},
|
|
349
|
+
],
|
|
350
|
+
}
|
|
351
|
+
assert.equal(isCompactionRequest(lookAlike), false)
|
|
352
|
+
assert.equal(patchCompactionRequest(lookAlike, rules), false)
|
|
353
|
+
assert.equal(messageText(lookAlike.messages.at(-1)).includes(rules[0]), false, 'nothing was appended')
|
|
354
|
+
|
|
355
|
+
const quoted = {
|
|
356
|
+
messages: [{ role: 'user', content: [{ type: 'text', text: `grep output: ${INSTRUCTION}` }] }],
|
|
357
|
+
}
|
|
358
|
+
assert.equal(isCompactionRequest(quoted), false)
|
|
359
|
+
assert.equal(isTaggedCompactionRequest(quoted), false)
|
|
360
|
+
})
|
|
361
|
+
|
|
362
|
+
test('a caller tag is authoritative in both directions', () => {
|
|
363
|
+
const rules = resolveOptions({}).compactionRules
|
|
364
|
+
// Tagged as something else: never the compaction call, even though the trailing
|
|
365
|
+
// message is the directive verbatim.
|
|
366
|
+
const other = { ...compactionRequest(), purpose: 'session-title' }
|
|
367
|
+
assert.equal(isCompactionRequest(other), false)
|
|
368
|
+
assert.equal(patchCompactionRequest(other, rules), false)
|
|
369
|
+
assert.equal(messageText(other.messages.at(-1)).includes(rules[0]), false)
|
|
370
|
+
|
|
371
|
+
// Untagged: a directive-shaped tail is still recognised, so an engine that sets
|
|
372
|
+
// no purpose at all keeps working.
|
|
373
|
+
const untagged = compactionRequest()
|
|
374
|
+
delete untagged.purpose
|
|
375
|
+
assert.equal(isCompactionRequest(untagged), true)
|
|
376
|
+
assert.equal(isTaggedCompactionRequest(untagged), false)
|
|
377
|
+
assert.equal(patchCompactionRequest(untagged, rules), true)
|
|
378
|
+
assert.ok(messageText(untagged.messages.at(-1)).includes(rules[0]))
|
|
379
|
+
|
|
380
|
+
// A tagged call with no messages has nothing to patch.
|
|
381
|
+
assert.equal(isCompactionRequest({ purpose: 'compaction', messages: [] }), false)
|
|
382
|
+
assert.equal(isCompactionRequest({ purpose: 'compaction', messages: 'not an array' }), false)
|
|
383
|
+
})
|
|
384
|
+
|
|
385
|
+
test('a look-alike can no longer spend the one-shot warning', () => {
|
|
386
|
+
const { ctx, registered } = stubContext({ used: 100000 })
|
|
387
|
+
const warnings = []
|
|
388
|
+
ctx.logger = { warn: (message) => warnings.push(message), info: () => {} }
|
|
389
|
+
apply(ctx, {})
|
|
390
|
+
const stream = registered.listeners['llm/stream']
|
|
391
|
+
const next = () => 'stream'
|
|
392
|
+
|
|
393
|
+
// An ordinary request with a frozen envelope that quotes the directive. This is
|
|
394
|
+
// exactly what burned the warning in the wild, and it must stay silent.
|
|
395
|
+
const lookAlike = Object.freeze({
|
|
396
|
+
messages: Object.freeze([
|
|
397
|
+
Object.freeze({ role: 'user', content: Object.freeze([{ type: 'text', text: `excerpt: ${INSTRUCTION}` }]) }),
|
|
398
|
+
]),
|
|
399
|
+
})
|
|
400
|
+
assert.equal(stream(lookAlike, next), 'stream')
|
|
401
|
+
assert.deepEqual(warnings, [], 'a look-alike must not warn')
|
|
402
|
+
|
|
403
|
+
// A genuine tagged call it cannot patch: warns, exactly once.
|
|
404
|
+
assert.equal(stream(Object.freeze(compactionRequest()), next), 'stream')
|
|
405
|
+
assert.equal(warnings.length, 1)
|
|
406
|
+
assert.match(warnings[0], /could not extend the compaction directive/)
|
|
407
|
+
stream(Object.freeze(compactionRequest()), next)
|
|
408
|
+
assert.equal(warnings.length, 1, 'the warning stays one-shot')
|
|
409
|
+
})
|
|
410
|
+
|
|
411
|
+
test('a reading below one display step says so instead of rendering zero', () => {
|
|
412
|
+
const { ctx, registered } = stubContext({ used: 35000 })
|
|
413
|
+
apply(ctx, {})
|
|
414
|
+
const line = lineFor(registered, stubSession())
|
|
415
|
+
assert.match(line, /Context occupancy \(host-measured\): under 50K \/ 1\.00M tokens used \(under 5%\)/)
|
|
416
|
+
assert.doesNotMatch(line, /: 0 \/ 1\.00M/, 'a sub-step reading must never render as an empty window')
|
|
417
|
+
// The line must not contradict itself: 643K of real headroom, floored to 600K.
|
|
418
|
+
assert.match(line, /About 600K tokens of headroom remain/)
|
|
419
|
+
})
|
|
420
|
+
|
|
421
|
+
test('an empty session still reads a true zero, and Chinese floors the same way', () => {
|
|
422
|
+
const empty = stubContext({ used: 0 })
|
|
423
|
+
apply(empty.ctx, {})
|
|
424
|
+
assert.match(lineFor(empty.registered, stubSession()), /: 0 \/ 1\.00M tokens used \(0%\)/)
|
|
425
|
+
|
|
426
|
+
const zh = stubContext({ used: 35000 })
|
|
427
|
+
apply(zh.ctx, { language: 'zh' })
|
|
428
|
+
assert.match(lineFor(zh.registered, stubSession()), /已用 不足 50K \/ 1\.00M(不足 5%)/)
|
|
429
|
+
})
|
|
430
|
+
|
|
431
|
+
test('a floored headroom names its own step instead of claiming less than 1K', () => {
|
|
432
|
+
const options = resolveOptions({})
|
|
433
|
+
// 2464 floors to one full step; only a reading inside the step floors to zero.
|
|
434
|
+
assert.equal(headroomDisplayStep(2464, WINDOW, options), 2000)
|
|
435
|
+
assert.equal(quantizeHeadroom(2464, WINDOW, options), 2000)
|
|
436
|
+
assert.equal(quantizeHeadroom(1500, WINDOW, options), 0)
|
|
437
|
+
const rendered = render(
|
|
438
|
+
'tight',
|
|
439
|
+
{
|
|
440
|
+
quantized: 650000,
|
|
441
|
+
step: 50000,
|
|
442
|
+
used: 676000,
|
|
443
|
+
window: WINDOW,
|
|
444
|
+
threshold: THRESHOLD,
|
|
445
|
+
percent: 65,
|
|
446
|
+
headroom: 0,
|
|
447
|
+
headroomRaw: 1500,
|
|
448
|
+
headroomStep: 2000,
|
|
449
|
+
},
|
|
450
|
+
options,
|
|
451
|
+
)
|
|
452
|
+
assert.match(rendered, /Under 2K tokens of headroom remain before it\./)
|
|
453
|
+
assert.doesNotMatch(rendered, /less than 1K/)
|
|
454
|
+
})
|
|
455
|
+
|
|
456
|
+
test('the first rule forbids quoting a budget claim and embeds none itself', () => {
|
|
457
|
+
const rule = DEFAULT_COMPACTION_RULES[0]
|
|
458
|
+
assert.match(rule, /not even quoted, paraphrased, or listed as an example/)
|
|
459
|
+
assert.match(rule, /earlier checkpoint already carries one/)
|
|
460
|
+
assert.equal(rule.includes('"'), false, 'the rule must not hand the model a quotable false claim')
|
|
461
|
+
assert.match(DEFAULT_COMPACTION_RULES[1], /whose only justification was context pressure/)
|
|
462
|
+
})
|