billion-context-dsh 0.2.8 → 0.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.en.md CHANGED
@@ -3,7 +3,7 @@
3
3
  [English](./README.en.md) | [中文](./README.md)
4
4
 
5
5
  > **⚠️ Beta notice — not for production use**
6
- > This project (**v0.2.8**) is a work-in-progress beta. The [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) itself is also in **public beta**. **Do not use either in engineering / production environments** — expect breaking changes and rough edges.
6
+ > This project (**v0.2.10**) is a work-in-progress beta. The [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) itself is also in **public beta**. **Do not use either in engineering / production environments** — expect breaking changes and rough edges.
7
7
 
8
8
  <p align="center">
9
9
  <strong>Built with gratitude on top of these projects</strong> — please give them a ⭐:
@@ -130,9 +130,10 @@ DSH derives every model request from its append-only session log (the *surface*)
130
130
  | nudge ("efficiency note — compress early and keep context lean") | injected at `agent/pre-step` by the kernel's pressure decision — efficiency note + context breakdown + compression rules, tone aligned with kernel/pi; never an order |
131
131
  | `decompress` | read-only recovery of shadowed originals from the log |
132
132
  | `search_context` | scores a unified doc set (block summaries + shadowed originals) rebuilt from the log via acp-kernel `searchBlocks` (hybrid: stemming + CJK bigrams + char n-gram fuzzy); hits link back to the owning block |
133
- | `acp_status` | CONTEXT BREAKDOWN (tool/text/summaries shares of the visible total) + compressed-block ledger + nudge decision line; no context-window rows; scope/view/tool/sort/limit drilldown supported |
133
+ | `acp_status` | CONTEXT BREAKDOWN (tool/text/summaries shares of the visible total) + compressed-block ledger + nudge decision line + a `Checkpoint seqs` row mapping each ACTIVE block's kernel ref (`bN`) to its checkpoint summary seq — compressing a checkpoint seq distills that block (issue #60); no context-window rows; scope/view/tool/sort/limit drilldown supported |
134
134
  | block state | in-memory kernel state + **log-rebuilt ledger** (no sidecar files) |
135
135
  | tiered distillation (T2/T3) | re-compressing a block's summary node distills that block (tier 2); distilling a tier-2 block yields tier 3. Tier + kernel block ids are persisted to the log, so kernel state rehydrates from the log after a restart and stays distillable |
136
+ | compression accounting (shadow price) | `shadowedTokenCount` (what the host occupancy display deducts) is priced in the **host token-meter's vocabulary** (`ctx.tokenMeter.measure` preferred; exact mirror in `src/host-tokens.ts` as fallback) — never the plugin's internal CJK-aware estimate (that is display currency; mixing it into the host ledger can drive `messageTokens` negative and brick a CJK-heavy session, issue #54) |
136
137
 
137
138
  The load-bearing compression guidance (tools, philosophy, summary rules, tier rules) is registered as a one-time system-prompt section; each nudge carries a condensed version (efficiency note + philosophy + context breakdown + HOW_TO_COMPRESS_RULES + range table + batch tip). There is deliberately **no automatic summarization**: automatic policy only nudges the model (`compactIfNeeded` returns null).
138
139
 
@@ -149,7 +150,7 @@ A walkthrough of the ACP philosophy this project inherits — how active context
149
150
  | `compress` | Replace a seq range with a dense summary you write (edges auto-balanced to tool-pair boundaries); re-compressing a block's summary node distills it (tier 2/3) |
150
151
  | `decompress` | Restore a previously compressed block's original content (read-only); accepts the `bN` ref shown by acp_status or a compaction id |
151
152
  | `search_context` | Search compressed block summaries and originals by keyword (acp-kernel hybrid retrieval: stemming + CJK bigrams + fuzzy); hits link back to the owning block |
152
- | `acp_status` | CONTEXT BREAKDOWN (tool/text/summaries shares of the visible total) + compressed-block ledger + nudge decision line; no context-window rows. Drilldown supported: `scope:"compressed"` per block, `scope:"uncompressed"` + `view:"messages"`/`"ranges"` per message/range, with `tool` filter, `sort` order and `limit` cap. Drilldown row refs are kernel ids (mN) — feed them straight to `compress` as `startSeq`/`endSeq` (auto-mapped to the live surface seq); `Surface:` seqs work too |
153
+ | `acp_status` | CONTEXT BREAKDOWN (tool/text/summaries shares of the visible total) + compressed-block ledger + nudge decision line + a `Checkpoint seqs` row mapping each ACTIVE block's kernel ref (`bN`) to its checkpoint summary seq (the distill entry point, issue #60); no context-window rows. Drilldown supported: `scope:"compressed"` per block, `scope:"uncompressed"` + `view:"messages"`/`"ranges"` per message/range, with `tool` filter, `sort` order and `limit` cap. Drilldown row refs are kernel ids (mN) — feed them straight to `compress` as `startSeq`/`endSeq` (auto-mapped to the live surface seq); `Surface:` seqs work too |
153
154
  | `/acp` | status / compress / decompress from the command bar; status also shows human-side window info (estimated context, window source, compressed-block ledger, and **nudge arbitration** — `nudge: idle/ACTIVE — reason` plus how many tokens remain until the next nudge, decided by the same kernel turn as the nudge path) |
154
155
 
155
156
  ## Upstream & credits
package/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  [中文](./README.md) | [English](./README.en.md)
4
4
 
5
5
  > **⚠️ 测试版声明——请勿用于生产环境**
6
- > 本项目(**v0.2.8**)仍处于开发中的测试版。[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 本身也处于**公开测试版**阶段。**请勿将两者用于工程化 / 生产环境**——预期会有破坏性变更与粗糙之处。
6
+ > 本项目(**v0.2.10**)仍处于开发中的测试版。[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 本身也处于**公开测试版**阶段。**请勿将两者用于工程化 / 生产环境**——预期会有破坏性变更与粗糙之处。
7
7
 
8
8
  <p align="center">
9
9
  <strong>衷心感谢以下项目——请给它们一个 ⭐:</strong>
@@ -131,9 +131,10 @@ DSH 的每个模型请求都派生自其 append-only 会话日志(*surface*)
131
131
  | nudge("效率提示——尽早压缩保持精简") | 由内核的压力决策在 `agent/pre-step` 注入——效率通知 + 上下文分解 + 压缩规则,语气对齐 kernel/pi;绝非命令 |
132
132
  | `decompress` | 从日志只读恢复被遮蔽的原文 |
133
133
  | `search_context` | 从日志重建块摘要 + 被遮蔽原文的统一文档集,交 acp-kernel `searchBlocks`(hybrid:词干化 + CJK bigram + 字符 n-gram 模糊)打分;命中回链所属块 |
134
- | `acp_status` | CONTEXT BREAKDOWN(tool/text/summaries 占可见总量)+ 压缩块账本 + nudge 决策行;不含上下文窗口;支持 scope/view/tool/sort/limit 钻取 |
134
+ | `acp_status` | CONTEXT BREAKDOWN(tool/text/summaries 占可见总量)+ 压缩块账本 + nudge 决策行 + `Checkpoint seqs` 行(active 块的 `bN → seq` 映射——压缩某个 checkpoint seq 即蒸馏该块,issue #60);不含上下文窗口;支持 scope/view/tool/sort/limit 钻取 |
135
135
  | 块状态 | 内存内核状态 + **日志重建账本**(无旁车文件) |
136
136
  | 分层蒸馏(T2/T3) | 再次压缩某块的摘要节点 = 蒸馏该块(tier 2),蒸馏 tier-2 块得 tier 3;tier 与内核块 id 持久化进日志,重启后内核状态从日志再水合、可继续蒸馏 |
137
+ | 压缩记账(影子价格) | `shadowedTokenCount`(宿主占用率据此扣减)**用宿主 token-meter 的词汇计价**(`ctx.tokenMeter.measure` 优先,`src/host-tokens.ts` 精确镜像兜底)——绝不混用插件内部的 CJK 感知估算(那是展示货币,混用会把宿主账本扣成负数、卡死中文会话,issue #54) |
137
138
 
138
139
  承载性的压缩指引(工具、哲学、摘要规则、tier 蒸馏/浓缩规则)注册为一次性系统提示段;每条 nudge 携带精简版(效率提示 + 哲学 + 上下文分解 + 压缩规则 + 范围表 + 批量提示)。刻意**不做自动摘要**:自动策略只 nudge 模型(`compactIfNeeded` 返回 null)。
139
140
 
@@ -150,7 +151,7 @@ DSH 的每个模型请求都派生自其 append-only 会话日志(*surface*)
150
151
  | `compress` | 用你书写的紧凑摘要替换 seq 范围(边界自动平衡到 tool-call/result 配对点);对某块的摘要节点再次压缩 = 分层蒸馏(tier 2/3) |
151
152
  | `decompress` | 恢复已压缩块的原始内容(只读);接受 acp_status 显示的 `bN` 或 compaction id |
152
153
  | `search_context` | 按关键词搜索压缩块摘要与原文(acp-kernel hybrid 检索:词干化 + CJK bigram + 模糊);命中回链所属块 |
153
- | `acp_status` | CONTEXT BREAKDOWN(tool/text/summaries 占可见总量)+ 压缩块账本 + nudge 决策行;不含上下文窗口。支持钻取:`scope:"compressed"` 逐块、`scope:"uncompressed"` + `view:"messages"`/`"ranges"` 逐消息/区间,`tool` 过滤、`sort` 排序、`limit` 截断。钻取行 ref 是内核 mN——可直接作为 `compress` 的 `startSeq`/`endSeq`(自动映射为 live surface seq);`Surface:` 的 seq 同样可用 |
154
+ | `acp_status` | CONTEXT BREAKDOWN(tool/text/summaries 占可见总量)+ 压缩块账本 + nudge 决策行 + `Checkpoint seqs` 行(active 块的 `bN → seq` 蒸馏入口,issue #60);不含上下文窗口。支持钻取:`scope:"compressed"` 逐块、`scope:"uncompressed"` + `view:"messages"`/`"ranges"` 逐消息/区间,`tool` 过滤、`sort` 排序、`limit` 截断。钻取行 ref 是内核 mN——可直接作为 `compress` 的 `startSeq`/`endSeq`(自动映射为 live surface seq);`Surface:` 的 seq 同样可用 |
154
155
  | `/acp` | 从命令栏执行 status / compress / decompress;status 额外展示 human-side 窗口信息(estimated context、context window 来源、压缩账本、**nudge 仲裁**——`nudge: idle/ACTIVE — reason` 及距下一次 nudge 还差多少 token,与 nudge 路径同一内核判定) |
155
156
 
156
157
  ## 上游项目与致谢
@@ -0,0 +1,72 @@
1
+ /**
2
+ * Host-vocabulary token pricing for the durable shadow-price protocol.
3
+ *
4
+ * The host token-meter prices every appended message with a fixed flat-4
5
+ * heuristic (`estimateContent` / `estimateMessage` in `dsh-token-meter`) and
6
+ * the producer contract requires every `compaction/summary`/`compaction/prune`
7
+ * `shadowedTokenCount` claim to be derived from the SAME estimator. Writing
8
+ * claims with the engine's CJK-aware `defaultCountTokens` overdraws the meter
9
+ * on CJK-heavy sessions and permanently bricks them (live session
10
+ * `session-3aa366c3`, issue #54; AGENTS.md rule 12 — `defaultCountTokens` is
11
+ * display currency, NEVER event currency).
12
+ *
13
+ * This module prices claims in the host's vocabulary: it prefers the live
14
+ * meter's own per-node prices (`ctx.tokenMeter.measure(session).nodes` —
15
+ * exact by construction, follows host estimator changes automatically, the
16
+ * same path the host's own `compaction-basic` uses) and falls back to an
17
+ * exact mirror of the host's estimator when the meter is unreachable.
18
+ */
19
+ import type { Session, SessionEvent } from '@deepseek-ai/dsh-session';
20
+ /** The host's model-visible content block union (structural, mirror-side only). */
21
+ export type HostBlock = {
22
+ type: 'text';
23
+ text: string;
24
+ } | {
25
+ type: 'reasoning';
26
+ text: string;
27
+ } | {
28
+ type: 'tool-call';
29
+ name: string;
30
+ arguments: string;
31
+ } | {
32
+ type: 'tool-result';
33
+ toolCallId: string;
34
+ content: HostContent;
35
+ } | ({
36
+ type?: string;
37
+ } & Record<string, unknown>);
38
+ /** A content block list, or a bare string (`tool-result` content may be either). */
39
+ export type HostContent = readonly HostBlock[] | string;
40
+ /**
41
+ * Exact mirror of the host's `estimateContent`
42
+ * (`@deepseek-ai/dsh-token-meter/lib/types/estimate.js`): text/reasoning
43
+ * `ceil(len/4)+4`, tool-call `ceil(name/4)+ceil(arguments/4)+4`, tool-result
44
+ * recursive over its content, unknown blocks `4+ceil(JSON.stringify/4)` over
45
+ * the ORIGINAL block object. A string content is iterated as an iterable, so
46
+ * every CHARACTER falls to the default branch (`4+ceil(JSON.stringify(char)/4)`
47
+ * — 5 tokens for any single unescaped character).
48
+ */
49
+ export declare function estimateHostContent(blocks: HostContent): number;
50
+ /** Exact mirror of the host's `estimateMessage` (content + role framing). */
51
+ export declare function estimateHostMessage(message: {
52
+ content: HostContent;
53
+ }): number;
54
+ /**
55
+ * Host price of ONE session event under the mirror: project it through the
56
+ * host's `deriveEventMessage` (null for non-surface events and empty-content
57
+ * assistant messages) and price the derived message; null derives to 0.
58
+ */
59
+ export declare function hostPriceEvent(event: SessionEvent): number;
60
+ /** Mirror price of a set of surface seqs (the fallback claim computation). */
61
+ export declare function shadowedHostTokens(session: Session, seqs: readonly number[]): number;
62
+ /**
63
+ * Claim price for `seqs` in the host's vocabulary. Prefers the live meter's
64
+ * own per-node prices when `ctx.tokenMeter` is reachable and covers every
65
+ * shadowed seq (exact by construction, follows host estimator changes); ANY
66
+ * failure — meter absent, `measure` throwing (e.g. a step-less log), or a seq
67
+ * missing from the measurement — falls back to the exact mirror. Never returns
68
+ * a `defaultCountTokens` price (rule 12).
69
+ */
70
+ export declare function shadowedTokensViaMeter(session: Session, seqs: readonly number[], ctx?: {
71
+ get?(name: string): unknown;
72
+ } | null): number;
package/dist/index.js CHANGED
@@ -2775,6 +2775,84 @@ function extractEventText(event) {
2775
2775
  }
2776
2776
  }
2777
2777
 
2778
+ // src/host-tokens.ts
2779
+ import { deriveEventMessage } from "@deepseek-ai/dsh-session";
2780
+ var CHARS_PER_TOKEN = 4;
2781
+ var BLOCK_OVERHEAD = 4;
2782
+ var ROLE_OVERHEAD = 4;
2783
+ function blockType(block) {
2784
+ if (typeof block !== "object" || block === null) return void 0;
2785
+ const type = block.type;
2786
+ return typeof type === "string" ? type : void 0;
2787
+ }
2788
+ function estimateHostContent(blocks) {
2789
+ if (typeof blocks === "string") {
2790
+ let tokens2 = 0;
2791
+ for (const char of blocks) {
2792
+ tokens2 += BLOCK_OVERHEAD + Math.ceil(JSON.stringify(char).length / CHARS_PER_TOKEN);
2793
+ }
2794
+ return tokens2;
2795
+ }
2796
+ let tokens = 0;
2797
+ for (const block of blocks) {
2798
+ switch (blockType(block)) {
2799
+ case "text":
2800
+ case "reasoning": {
2801
+ tokens += Math.ceil(block.text.length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD;
2802
+ break;
2803
+ }
2804
+ case "tool-call": {
2805
+ const call = block;
2806
+ tokens += Math.ceil(call.name.length / CHARS_PER_TOKEN) + Math.ceil(call.arguments.length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD;
2807
+ break;
2808
+ }
2809
+ case "tool-result": {
2810
+ tokens += estimateHostContent(block.content) + BLOCK_OVERHEAD;
2811
+ break;
2812
+ }
2813
+ default:
2814
+ tokens += BLOCK_OVERHEAD + Math.ceil(JSON.stringify(block).length / CHARS_PER_TOKEN);
2815
+ }
2816
+ }
2817
+ return tokens;
2818
+ }
2819
+ function estimateHostMessage(message) {
2820
+ return estimateHostContent(message.content) + ROLE_OVERHEAD;
2821
+ }
2822
+ function hostPriceEvent(event) {
2823
+ const message = deriveEventMessage(event);
2824
+ return message === null ? 0 : estimateHostMessage(message);
2825
+ }
2826
+ function shadowedHostTokens(session, seqs) {
2827
+ let total = 0;
2828
+ for (const seq of seqs) {
2829
+ const event = session.events[seq];
2830
+ if (event !== void 0) total += hostPriceEvent(event);
2831
+ }
2832
+ return total;
2833
+ }
2834
+ function shadowedTokensViaMeter(session, seqs, ctx) {
2835
+ try {
2836
+ const meter = ctx?.get?.("tokenMeter");
2837
+ if (meter?.measure !== void 0) {
2838
+ const bySeq = new Map(meter.measure(session).nodes.map((node) => [node.seq, node.tokens]));
2839
+ let total = 0;
2840
+ let missing = false;
2841
+ for (const seq of seqs) {
2842
+ const tokens = bySeq.get(seq);
2843
+ if (tokens === void 0) {
2844
+ missing = true;
2845
+ break;
2846
+ }
2847
+ total += tokens;
2848
+ }
2849
+ if (!missing) return total;
2850
+ }
2851
+ } catch {
2852
+ }
2853
+ return shadowedHostTokens(session, seqs);
2854
+ }
2855
+
2778
2856
  // src/region.ts
2779
2857
  function findOpenTurn(events) {
2780
2858
  let open = null;
@@ -3027,14 +3105,14 @@ function assistantProviderModel(event) {
3027
3105
  }
3028
3106
  return { provider: "billion-context-dsh", model: "surface-prune" };
3029
3107
  }
3030
- function hideSurfaceSeqs(session, seqs, provider, model, text) {
3108
+ function hideSurfaceSeqs(session, seqs, provider, model, text, priceEvent = hostPriceEvent) {
3031
3109
  if (seqs.length === 0) return;
3032
3110
  const start = seqs[0];
3033
3111
  const end = seqs[seqs.length - 1];
3034
3112
  let shadowedTokenCount = 0;
3035
3113
  for (const seq of seqs) {
3036
3114
  const event = session.events[seq];
3037
- if (event !== void 0) shadowedTokenCount += defaultCountTokens(extractEventText(event));
3115
+ if (event !== void 0) shadowedTokenCount += priceEvent(event);
3038
3116
  }
3039
3117
  session.append("compaction/prune", {
3040
3118
  shadowedRange: { start, end },
@@ -3447,7 +3525,7 @@ var NUDGE_ALLOWED = {
3447
3525
  normal: /* @__PURE__ */ new Set(["pct", "philosophy"]),
3448
3526
  emergency: /* @__PURE__ */ new Set(["pct", "philosophy"]),
3449
3527
  guidance: /* @__PURE__ */ new Set(),
3450
- tier: /* @__PURE__ */ new Set(["tier", "count", "prevTier", "tokens", "seqs"]),
3528
+ tier: /* @__PURE__ */ new Set(["tier", "count", "prevTier", "tokens", "seqs", "firstSeq", "lastSeq"]),
3451
3529
  breakdown: /* @__PURE__ */ new Set(["system", "tool", "summaries", "code", "text"]),
3452
3530
  growth: /* @__PURE__ */ new Set(["growth"]),
3453
3531
  tip: /* @__PURE__ */ new Set()
@@ -3522,7 +3600,7 @@ var DEFAULT_PROMPTS = {
3522
3600
  normal: "This is an efficiency nudge to compress early and keep context lean \u2014 not an overflow warning. A separate, stronger alert will appear if the context is actually full.\n\n{philosophy}",
3523
3601
  emergency: "\u26A0\uFE0F Context limit reached \u2014 compress now. Prioritize consumed tool outputs.\n\n{philosophy}",
3524
3602
  guidance: HOW_TO_COMPRESS_RULES,
3525
- tier: "Tier {tier}: {count} tier-{prevTier} block(s) distillable ({tokens} tokens) \u2014 compress their summary node(s) [seqs {seqs}] to reclaim the original messages.",
3603
+ tier: "Tier {tier}: {count} tier-{prevTier} block(s) distillable ({tokens} tokens) \u2014 distill them by compressing their checkpoint seq(s) [seqs {seqs}] as one range: compress({ content: [{ startSeq: {firstSeq}, endSeq: {lastSeq}, summary }] }).",
3526
3604
  breakdown: "Context breakdown: {system}K system | {tool}K tool | {summaries}K summaries | {code}K code | {text}K text",
3527
3605
  growth: "+{growth}K since last nudge",
3528
3606
  tip: "\u{1F4A1} Compress all ranges in one call (pass multiple content entries: `content: [{...}, {...}]`)."
@@ -3536,7 +3614,7 @@ var DEFAULT_PROMPTS = {
3536
3614
  tools: {
3537
3615
  compress: "Replace older conversation ranges with dense summaries you write. Each message seq is a surface reference. Single range: compress({ content: [{ startSeq, endSeq, summary }] }). Batch multiple unrelated ranges in one call (each content entry becomes its own block); keep ranges disjoint. Never compress content the current step is actively using. Compress boundaries are SURFACE SEQS (acp_status Surface: row, latest nudge table) \u2014 NOT the block refs (bN, e.g. b1) that acp_status COMPRESSED BLOCKS shows, which are for decompress only. Drilldown mN refs (e.g. m00306) are ALSO accepted as startSeq/endSeq \u2014 they are auto-mapped to the live surface seq; an unknown mN (never assigned on the current surface) fails with guidance. Seq refs must come from the CURRENT surface (acp_status or the latest nudge): a span whose edges were shadowed by an earlier compress is auto-remapped to its still-live content, a fully compressed span is reported as already compressed, and invented/other-session seqs fail with guidance. Good compression moments: stage or subtask completion, strategy switches, intermediate milestones, and wrapping up failed exploration \u2014 when the details are consumed and no longer critical for the task ahead. When you write a summary, turn dead-end exploration into a conclusion (what was tried, why it failed, the next step) \u2014 not a blow-by-blow; and keep the summary the ONLY record: self-contained, so a later reader (or you, after decompress) can continue without the original.",
3538
3616
  decompress: "Recover the original content of a compressed block by its blockId \u2014 the kernel block ref `bN` shown by acp_status (e.g. b1), or a compaction id from search_context (read-only; does not unshadow the range).",
3539
- searchContext: "Search inside compressed blocks (summaries and original content) for information the model no longer sees in context.",
3617
+ searchContext: "Search inside compressed blocks (summaries and original content) for information the model no longer sees in context. When a summary lacks a detail you need (exact values, error strings, decisions, verbatim code), SEARCH the compressed blocks FIRST \u2014 never guess or reconstruct from memory: search_context(query) locates the right block, then decompress only that block to recover the original.",
3540
3618
  acpStatus: 'Context status: overview of the current context \u2014 CONTEXT BREAKDOWN (tool/text/summaries token shares of the visible total), COMPRESSED BLOCKS ledger, and the nudge decision. No args = overview. Percentages are shares of the visible content, not the context window. Note: the block refs in COMPRESSED BLOCKS (bN, e.g. b1) are for decompress; compress uses the Surface: seq range, not bN. Drilldown: pass scope:"compressed" for a per-block list, or scope:"uncompressed" with view:"messages" (every visible message) / view:"ranges" (merged ranges); tool filters to one tool name, sort reorders (size/time/tool; age for compressed), limit caps rows (default 30). Drilldown row refs are kernel ids (mN) \u2014 feed them straight to compress as startSeq/endSeq (auto-mapped to the live surface seq); bN is for decompress, Surface: seqs also work in compress.'
3541
3619
  },
3542
3620
  systemPromptTemplate: `Active Context Pruning \u2014 model-driven context management
@@ -3564,7 +3642,7 @@ WHEN NOT TO COMPRESS:
3564
3642
  Compression tools (refs are SURFACE SEQS, not ids):
3565
3643
  - compress: replace one or more seq ranges, each with your own dense summary. Single range: compress({ content: [{ startSeq, endSeq, summary }] }). Batch multiple unrelated segments in one call (each entry becomes its own block): compress({ content: [{ startSeq: 1, endSeq: 5, summary: '...' }, { startSeq: 12, endSeq: 18, summary: '...' }] }). Keep ranges disjoint \u2014 overlapping entries in one batch are skipped. Edges are auto-balanced to tool-call/result boundaries; a trailing #callId fragment in a seq is ignored. Seq refs must be on the current surface: seqs from older nudges or earlier compresses go stale as the surface moves, so a stale span is auto-remapped to its still-live remainder (the result reports the adjusted span), a fully compressed span is reported as already compressed, and invented/other-session seqs fail with guidance. The block refs (bN, e.g. b1) in acp_status COMPRESSED BLOCKS are for decompress, NOT compress boundaries.
3566
3644
  - decompress: recover a compressed block's original content, read-only. decompress({ blockId }) \u2014 accept the bN ref shown by acp_status (e.g. b1) or a compaction id.
3567
- - search_context: find information inside compressed blocks BEFORE decompressing. search_context({ query }).
3645
+ - search_context: when a summary lacks the details you need (exact values, error strings, decisions, verbatim code), SEARCH the compressed blocks FIRST \u2014 never guess or reconstruct from memory; search_context(query) locates the right block, decompress only that block.
3568
3646
  - acp_status: current context usage and the live compressible-range list. Run it right before compressing \u2014 the only seqs that never go stale are the ones you just read. Drilldown (scope/view/tool/sort/limit) lists per-message or per-block sizes; drilldown rows are kernel ids (mN) \u2014 compress accepts them directly (auto-mapped to the live surface seq).
3569
3647
 
3570
3648
  Tiered compression: each compressed block appears on the surface as one summary node. Compressing that node again DISTILLS the block (tier 2): the parent summary folds into your new summary and the original messages are freed. Distilling a tier-2 block yields tier 3. Distill when a summary itself is consumed \u2014 decompress on the tier-2 block recovers the full originals.
@@ -3671,7 +3749,7 @@ function replaceTierTrigger(text, nudge, session, prompts) {
3671
3749
  const next = rest.match(/\n\nHOW TO COMPRESS/);
3672
3750
  const end = next !== null ? start + 2 + next.index : text.length;
3673
3751
  const targets = nudge.tierTargetBlocks;
3674
- const summarySeqs = targets.map((block) => summarySeqOfKernelBlock(session, block.blockId)).filter((seq) => seq !== null);
3752
+ const summarySeqs = targets.map((block) => summarySeqOfKernelBlock(session, block.blockId)).filter((seq) => seq !== null).sort((a, b) => a - b);
3675
3753
  const pending = nudge.tier === 2 ? nudge.breakdown?.pendingT2 : nudge.breakdown?.pendingT3;
3676
3754
  const tokens = typeof pending === "number" ? pending : 0;
3677
3755
  const tierValue = nudge.tier === null ? 2 : nudge.tier;
@@ -3680,7 +3758,9 @@ function replaceTierTrigger(text, nudge, session, prompts) {
3680
3758
  count: targets.length,
3681
3759
  prevTier: tierValue - 1,
3682
3760
  tokens,
3683
- seqs: summarySeqs.join(", ")
3761
+ seqs: summarySeqs.join(", "),
3762
+ firstSeq: summarySeqs[0] ?? "n/a",
3763
+ lastSeq: summarySeqs[summarySeqs.length - 1] ?? "n/a"
3684
3764
  });
3685
3765
  return text.slice(0, start) + "\n\n" + tierLine + text.slice(end);
3686
3766
  }
@@ -3717,7 +3797,7 @@ function renderNudgeFromTemplates(nudge, emergency, session, prompts) {
3717
3797
  if (prompts.nudge.guidance !== "") parts.push("", prompts.nudge.guidance);
3718
3798
  if ((nudge.tier === 2 || nudge.tier === 3) && (nudge.tierTargetBlocks?.length ?? 0) > 0) {
3719
3799
  const targets = nudge.tierTargetBlocks;
3720
- const summarySeqs = targets.map((block) => summarySeqOfKernelBlock(session, block.blockId)).filter((seq) => seq !== null);
3800
+ const summarySeqs = targets.map((block) => summarySeqOfKernelBlock(session, block.blockId)).filter((seq) => seq !== null).sort((a, b) => a - b);
3721
3801
  const pending = nudge.tier === 2 ? nudge.breakdown?.pendingT2 : nudge.breakdown?.pendingT3;
3722
3802
  const tokens = typeof pending === "number" ? pending : 0;
3723
3803
  const tierLine = renderTemplate(prompts.nudge.tier, {
@@ -3725,7 +3805,9 @@ function renderNudgeFromTemplates(nudge, emergency, session, prompts) {
3725
3805
  count: targets.length,
3726
3806
  prevTier: nudge.tier - 1,
3727
3807
  tokens,
3728
- seqs: summarySeqs.join(", ")
3808
+ seqs: summarySeqs.join(", "),
3809
+ firstSeq: summarySeqs[0] ?? "n/a",
3810
+ lastSeq: summarySeqs[summarySeqs.length - 1] ?? "n/a"
3729
3811
  });
3730
3812
  if (tierLine !== "") parts.push(tierLine);
3731
3813
  const tierRules = nudge.tier === 2 ? TIER2_DISTILL_RULES : TIER3_CONDENSE_RULES;
@@ -3968,11 +4050,7 @@ async function handleCompress(env, args, exec) {
3968
4050
  }
3969
4051
  const { start, end } = range;
3970
4052
  const shadowed = shadowedSeqsOf(session, start, end);
3971
- let shadowedTokens = 0;
3972
- for (const seq of shadowed) {
3973
- const event = session.events[seq];
3974
- if (event !== void 0) shadowedTokens += defaultCountTokens(extractEventText(event));
3975
- }
4053
+ const shadowedTokens = shadowedTokensViaMeter(session, shadowed, agent.ctx);
3976
4054
  const tier = block.tier === 2 || block.tier === 3 ? block.tier : 1;
3977
4055
  const parentBlockIds = compactionIdsOfKernelBlocks(session, block.directBlockIds);
3978
4056
  const { compactionId } = runCompactionTransaction(session, {
@@ -3994,7 +4072,7 @@ async function handleCompress(env, args, exec) {
3994
4072
  effectiveMessageIds: block.effectiveMessageIds
3995
4073
  });
3996
4074
  const adjusted = start !== range.startSeq || end !== range.endSeq;
3997
- const tierLabel2 = tier === 1 ? "" : `, tier ${tier}`;
4075
+ const tierLabel2 = `, tier ${tier}`;
3998
4076
  const note = range.recovered === true ? ` (seqs ${range.startSeq}..${range.endSeq} were already shadowed \u2014 compressed the live remainder ${start}..${end})` : adjusted ? ` (adjusted from ${range.startSeq}..${range.endSeq} to balanced edges)` : "";
3999
4077
  lines.push(
4000
4078
  ` block ${compactionId.slice(0, 8)}: seqs ${start}..${end}, ${shadowed.length} messages shadowed${tierLabel2}${note}`
@@ -4166,6 +4244,10 @@ async function handleStatus(env, rawArgs, exec) {
4166
4244
  if (nudge !== void 0) {
4167
4245
  lines.push("", `Nudge: ${nudge.shouldInject ? "ACTIVE" : "idle"} \u2014 ${nudge.reason}`);
4168
4246
  }
4247
+ const checkpointRows = blockRegistry(session).filter((entry) => entry.active && entry.summarySeq !== null).map((entry) => `${entry.kernelBlockId} \u2192 seq ${entry.summarySeq}`);
4248
+ if (checkpointRows.length > 0) {
4249
+ lines.push("", `Checkpoint seqs (active blocks \u2014 compress a checkpoint seq to distill it): ${checkpointRows.join(", ")}`);
4250
+ }
4169
4251
  }
4170
4252
  lines.push("", `Surface: ${surfaceSummary(session)}`);
4171
4253
  if (args.scope === "uncompressed") {
@@ -4288,12 +4370,8 @@ function compressText(env, agent, args) {
4288
4370
  if (blockRefForSummarySeq(session, start) !== null || blockRefForSummarySeq(session, end) !== null) {
4289
4371
  return "/acp compress: the range touches a compressed block summary node \u2014 distill it with the compress tool (seq-based batch), not /acp compress";
4290
4372
  }
4291
- const shadowed = shadowedSeqsOf(session, startSeq, endSeq);
4292
- let shadowedTokens = 0;
4293
- for (const seq of shadowed) {
4294
- const event = session.events[seq];
4295
- if (event !== void 0) shadowedTokens += defaultCountTokens(extractEventText(event));
4296
- }
4373
+ const shadowed = shadowedSeqsOf(session, start, end);
4374
+ const shadowedTokens = shadowedTokensViaMeter(session, shadowed, agent.ctx);
4297
4375
  const { compactionId } = runCompactionTransaction(session, {
4298
4376
  start,
4299
4377
  end,