dsh-context-toolkit 0.2.4 → 0.3.1

package/README.md

@@ -28,6 +28,7 @@ It separates *raw bytes* from *conclusions*: commands run in a child process, fi
 | `ctk_batch_execute` | Several commands in one call, per-command status, conclusions only |
 | `ctk_index` | File text or a file into the local FTS5 knowledge base |
 | `ctk_search` | Multi-query retrieval; excerpts are centred on the match |
+| `ctk_retrieve` | Read back a slimmed output's stored original, byte-for-byte, from its `[ctk_retrieve id=…]` marker |
 | `ctk_fetch_and_index` | Fetch pages → strip markup → file them |
 | `ctk_purge` | Delete filed material (requires `confirm: true`) |
 | `ctk_stats` | Savings ledger: refusals, sandbox bytes, filed bytes, call counts |
@@ -114,12 +115,31 @@ Tool switches: `tools.execute`, `tools.executeFile`, `tools.batchExecute`, `tool
 The offline suites need no DSH runtime:
 
 ```bash
-node smoke.mjs        # 70 checks: contract, runtime discovery, tools, gate (segment matching, prefix/path normalisation, host-aligned path base, script_path and host-fetcher coverage), resource disposal, knowledge base, observability, version reporting
+node smoke.mjs        # 82 checks: contract, runtime discovery, tools, gate (segment matching, host-aligned path base, script_path and host-fetcher coverage), output slimming and reversible retrieval, resource disposal, knowledge base, observability, version reporting
 node test-client.mjs  # 27 checks: browser-half contract and bilingual dictionary parity
 node test-package.mjs # 20 checks: publish manifest and the zero-dependency constraint
-node release-check.mjs # 27 checks: release readiness (manifest, deps, tool count, gate matching, version, SHA coverage, no hardcoded paths)
+node release-check.mjs # 30 checks: release readiness (manifest, deps, tool count, gate matching, version, SHA coverage, no hardcoded paths)
 ```
 
+## Output slimming (after a tool runs)
+
+The gate works at the *door*; slimming handles what is **already inside**. They catch different traffic: the gate can only read a call's *arguments*, while slimming sees the *bytes that actually came back* — from a tool the gate does not know, an entry point it does not cover, or output that was never predictable from the call.
+
+At `tools/post-execute`, the strategy follows the shape:
+
+| Shape | Treatment |
+|---|---|
+| Uniform JSON array | **Lossless** tabular re-encoding: keys written once, no information dropped |
+| Table | Header plus both ends kept; the middle stride-sampled **with original line numbers** (the model gets a map with coordinates) |
+| grep / search | Folded by file, first hits per file kept |
+| Log | Ordinary lines folded away, **diagnostics kept verbatim** |
+| Long prose | Head/tail window with an explicit elision marker |
+| Code / key-value | **Not compressed** (the model may be about to edit exact bytes) |
+
+**Two gates**: a candidate must be at most 72% of the original bytes *and* have a strictly lower token estimate, or it is dropped — no saving means no claim, and the ledger never overstates.
+
+**Reversibility is the red line**: every lossy path stores the exact original first and injects a `[ctk_retrieve id=…]` marker, read back with `ctk_retrieve`. The originals live in the **same database as the FTS5 index** — one file, two jobs: FTS5 finds sections, blobs restore them.
+
 ## Known limits
 
 - **Outbound fetching depends on this machine's network.** If host names resolve to a loopback address, `ctk_fetch_and_index` fails with that reason spelled out — not a bare "fetch failed" — and `ctk_index` is the way to file local material instead.

package/README.zh.md

@@ -28,6 +28,8 @@
 | `ctk_batch_execute` | 一次跑多条命令，逐条状态，输出只有结论 |
 | `ctk_index` | 把文本或文件存进本地 FTS5 知识库 |
 | `ctk_search` | 多查询检索知识库，返回围绕命中词的片段 |
+| `ctk_retrieve` | 取回被瘦身压缩掉的原文（逐字节，按 `[ctk_retrieve id=…]` 标记的 id） |
+| `ctk_retrieve` | 取回被瘦身压缩掉的原文（逐字节，按 `[ctk_retrieve id=…]` 标记的 id） |
 | `ctk_fetch_and_index` | 抓网页 → 去标签转文本 → 直接入库 |
 | `ctk_purge` | 删除已归档内容（必须 `confirm: true`） |
 | `ctk_stats` | 节约台账：拦截次数、沙箱保留字节、归档字节、调用计数 |
@@ -114,12 +116,31 @@ await ctk_search({ queries: ['rate limit', 'auth'] });  // 只取相关片段
 离线套件不依赖 DSH 运行时：
 
 ```bash
-node smoke.mjs        # 70 项：契约、运行时探测、三工具、门禁（分段匹配 + 前缀/路径归一 + 路径基准对齐 + script_path 与取网页工具覆盖）、资源释放、知识库、观测、版本自述
+node smoke.mjs        # 82 项：契约、运行时探测、工具、门禁（分段匹配 + 路径基准对齐 + script_path 与取网页工具覆盖）、输出瘦身与可逆取回、资源释放、知识库、观测、版本自述
 node test-client.mjs  # 27 项：浏览器半体契约与双语字典一致性
 node test-package.mjs # 20 项：发布清单与零依赖约束
-node release-check.mjs # 27 项：发布就绪（清单/依赖/工具数/门禁匹配/版本/SHA 覆盖/无硬编码路径）
+node release-check.mjs # 30 项：发布就绪（清单/依赖/工具数/门禁匹配/版本/SHA 覆盖/无硬编码路径）
 ```
 
+## 输出瘦身（工具执行后）
+
+门禁在**入口**拦下次就会灌满对话的调用；瘦身则处理**已经进来的**输出——两者抓的是不同流量：门禁只能看调用的**参数**，瘦身看的是**真实回来的字节**（来自门禁不认识的工具、它没覆盖的入口，或本来就无法从调用预判的输出）。
+
+在 `tools/post-execute` 处按形态选择策略：
+
+| 形态 | 做法 |
+|---|---|
+| 同构 JSON 数组 | **无损**表化重编码：键只写一次，信息零损失 |
+| 表格 | 保留表头 + 首尾，中段**按原始行号**步进采样（模型拿到的是带坐标的地图） |
+| grep/search | 按文件折叠，每文件保留前若干命中 |
+| 日志 | 折叠普通行，**诊断行逐字保留** |
+| 长散文 | 头尾窗口 + 显式省略标记 |
+| 代码 / 键值对 | **不压**（模型可能正要按精确字节编辑） |
+
+**双重门**：候选必须同时满足「字节 ≤ 72%」且「估算 token 严格下降」，否则放弃——压不动就不压，台账绝不虚报节省。
+
+**可逆红线**：任何有损路径先把原文**逐字节**存好并注入 `[ctk_retrieve id=…]` 标记，用 `ctk_retrieve` 取回。原文与 FTS5 索引同在**同一个数据库**（一个文件两个职责：FTS5 找段落，blob 逐字节还原）。
+
 ## 已知边界
 
 - **出站抓取受本机网络影响**：若本机把域名解析到回环地址，`ctk_fetch_and_index` 会失败并明确说明原因（而不是只报 "fetch failed"）；此时改用 `ctk_index` 归档本地材料。

package/client.js

@@ -17,8 +17,8 @@ window.__ModuleLoader__.load({
     let jsxs = jsxRuntime.jsxs;
 
     //#region locales
-    const zh = {"title":"上下文工具箱","description":"在沙箱里跑代码、把大块材料归档成可检索的本地知识库、并挡住会灌满对话的抓取与整读。附一张实时上下文占用面板。","hint":"下列开关即时生效，无需重启：门禁在每次工具调用前读取。","routing":"路由门禁","routingHint":"总开关。开启后，每次工具调用执行前都会经过门禁检查。","denyCurlWget":"硬拦 curl / wget","denyCurlWgetHint":"这两个工具的设计就是把整页内容塞进对话；拦掉，改用 ctk_fetch_and_index。","bashNudgeMinCommandBytes":"长命令引导阈值（字节）","bashNudgeMinCommandBytesHint":"命令超过这个长度时提示改用 ctk_batch_execute。填 0 表示不提示。","maxReadBytesBeforeAsk":"大文件整读引导阈值（字节）","maxReadBytesBeforeAskHint":"整读超过这个大小的文件时提示改用 ctk_execute_file。填 0 表示不提示。","nudgeFetchTools":"取网页工具也纳入整页引导","nudgeFetchToolsHint":"把宿主自带的 web_fetch / read_page 也纳入引导：它们与 shell 里的 curl 取回同样一整页，却不在 shell 门禁的视野里。关闭则只拦 shell 一侧。","trustedReadBasenames":"不受整读门禁限制的文件名","trustedReadBasenamesHint":"逗号分隔；这些文件即使很大也允许整读。","compressNudgeTokens":"占用提示阈值（tokens）","compressNudgeTokensHint":"表面占用达到此值时，占用报告给出压缩建议。填 0 表示不提示。","searchLimit":"检索返回条数上限","searchLimitHint":"ctk_search 每次查询返回的片段数上限。","snippetChars":"片段最大字符数","snippetCharsHint":"ctk_search 每个片段返回的字符上限，超出部分省略。","fetchMaxBytes":"单页抓取上限（字节）","fetchMaxBytesHint":"ctk_fetch_and_index 抓取单页的最大字节数，超出部分丢弃。","storageDir":"知识库根目录","storageDirHint":"留空 = <DSH_HOME>/context-toolkit。","toolsTitle":"工具开关","toolsHint":"逐个启用或停用 ctk_* 工具。","overridden":"已覆盖","reset":"恢复默认","readOnly":"本部署的设置为只读。","unavailable":"该插件当前未加载，暂时无法配置。","save":"保存","saving":"保存中…","saveFailed":"本部署没有接受这些值，已保留供你修改。","invalidNumber":"请填数字；留空表示使用默认值。","note":"门禁开关即时生效；工具开关改动后建议重开页面确认。","ctxTitle":"上下文情况","ctxLoading":"正在读取…","ctxReload":"刷新","ctxFailed":"读取失败：","ctxNoSession":"本会话暂无可测量的投影。","ctxStale":"以下为最近一次实测快照（{time}）：网页请求没有会话边界，实时值请在对话中调用 ctk_context_report。","ctxTotal":"总压力 {n} tokens","ctxSurface":"表面占用 {n} tokens","ctxBudget":"提示阈值 {n} tokens","ctxOver":"已超过提示阈值，建议把大块输出改走 ctk_execute / ctk_index。","ctxUnder":"低于提示阈值，暂不需要干预。","ctxSystem":"系统提示","ctxTools":"工具 schema","ctxMessages":"会话消息","ctxEstimated":"（估算）","ctxNodes":"占用大头（前 {n} 项）","ctxLedger":"本插件节约台账","ctxLedgerLine":"门禁已拦截 {n} 次，约 {bytes} 字节未进入上下文；沙箱保留 {sandbox} 字节；已注册 {tools} 个 ctk_* 工具。","ctxKnowledge":"知识库：{docs} 文档 / {chunks} 片段（{bytes} 字节）","ctxStorage":"知识库位置：{path}","tools.execute":"ctk_execute（在沙箱里跑代码，只把结论带回来）","tools.executeHint":"用于「只想要结论、不想让原始输出占满上下文」的分析型任务。","tools.executeFile":"ctk_execute_file（对某个文件跑代码，只回结论）","tools.executeFileHint":"适合读大文件后做统计或抽取，文件原文留在沙箱里。","tools.batchExecute":"ctk_batch_execute（一次跑多条命令，逐条状态）","tools.batchExecuteHint":"多条命令一次调用，只有结论回来。","tools.index":"ctk_index（把大块材料存进本地知识库）","tools.indexHint":"把长文档、日志、规范归档，之后用 ctk_search 按需取用。","tools.search":"ctk_search（按需检索知识库，取片段而非整份）","tools.searchHint":"知识库里已有的内容优先用它检索，不要整份读进来。","tools.fetchAndIndex":"ctk_fetch_and_index（抓网页并直接并入索引）","tools.fetchAndIndexHint":"抓整页网页又不让它淹没对话时用；curl/wget 会被门禁拦掉。","tools.purge":"ctk_purge（删除已索引内容，不可恢复）","tools.purgeHint":"清理知识库用；会永久删除，调用必须带 confirm:true。","tools.stats":"ctk_stats（节约台账与知识库用量）","tools.statsHint":"想知道这个插件省了多少、归档了多少时看它。","tools.doctor":"ctk_doctor（自检：运行时、临时目录、知识库往返）","tools.doctorHint":"出现异常（子进程跑不起来、索引失败）时先跑它定位。","tools.contextReport":"ctk_context_report（会话上下文占用报告）","tools.contextReportHint":"让模型可以主动查询本会话的上下文占用与构成。","restart":"需重启"};
-    const en = {"title":"Context Toolkit","description":"Run code in a sandbox, file large material into a local searchable knowledge base, and block the fetches and whole-file reads that flood a conversation. Ships a live context-occupancy panel.","hint":"These switches apply immediately, no restart: the gate is read before every tool call.","routing":"Routing gate","routingHint":"Master switch. When on, every tool call passes through the gate before running.","denyCurlWget":"Block curl / wget","denyCurlWgetHint":"Both exist to paste whole responses into the conversation; blocked here, use ctk_fetch_and_index instead.","bashNudgeMinCommandBytes":"Long-command nudge (bytes)","bashNudgeMinCommandBytesHint":"Commands longer than this get a nudge toward ctk_batch_execute. 0 disables it.","maxReadBytesBeforeAsk":"Whole-file read nudge (bytes)","maxReadBytesBeforeAskHint":"Reading a file larger than this gets a nudge toward ctk_execute_file. 0 disables it.","nudgeFetchTools":"Nudge the host fetchers too","nudgeFetchToolsHint":"Also nudge the host own web_fetch / read_page: they pull back the same whole page as a shell curl, and the shell gate never sees them. Off leaves that door unwatched.","trustedReadBasenames":"Filenames exempt from the read nudge","trustedReadBasenamesHint":"Comma separated; these may be read whole however large they are.","compressNudgeTokens":"Occupancy warning (tokens)","compressNudgeTokensHint":"At this surface occupancy the report suggests compaction. 0 disables it.","searchLimit":"Search result limit","searchLimitHint":"How many excerpts ctk_search returns per query.","snippetChars":"Excerpt size (chars)","snippetCharsHint":"Character ceiling on each excerpt from ctk_search.","fetchMaxBytes":"Per-page fetch ceiling (bytes)","fetchMaxBytesHint":"ctk_fetch_and_index keeps at most this many bytes of each page.","storageDir":"Knowledge-base root","storageDirHint":"Empty = <DSH_HOME>/context-toolkit.","toolsTitle":"Tool switches","toolsHint":"Enable or disable individual ctk_* tools.","overridden":"overridden","reset":"reset","readOnly":"This deployment is read-only.","unavailable":"The plugin is not loaded right now; it cannot be configured.","save":"Save","saving":"Saving…","saveFailed":"This deployment did not accept those values; they are kept for you to edit.","invalidNumber":"Numbers only; blank means the default.","note":"Gate switches apply immediately; reopen the page to confirm tool-switch changes.","ctxTitle":"Context occupancy","ctxLoading":"Reading…","ctxReload":"Refresh","ctxFailed":"Read failed: ","ctxNoSession":"No measurable projection in this session yet.","ctxStale":"Snapshot from the last real measurement ({time}): a page request has no session boundary; ask ctk_context_report in the conversation for a live reading.","ctxTotal":"{n} tokens total","ctxSurface":"{n} tokens on the surface","ctxBudget":"warning at {n} tokens","ctxOver":"Above the warning threshold — route bulk output through ctk_execute / ctk_index.","ctxUnder":"Below the warning threshold; nothing to do.","ctxSystem":"System prompt","ctxTools":"Tool schemas","ctxMessages":"Conversation","ctxEstimated":" (estimated)","ctxNodes":"Biggest contributors (top {n})","ctxLedger":"This plugin's ledger","ctxLedgerLine":"The gate blocked {n} calls, about {bytes} bytes kept out of context; the sandbox kept out {sandbox} bytes; {tools} ctk_* tools registered.","ctxKnowledge":"Knowledge base: {docs} documents / {chunks} chunks ({bytes} bytes)","ctxStorage":"Knowledge base at {path}","tools.execute":"ctk_execute (run code in the sandbox, return only the answer)","tools.executeHint":"For analysis where you want the conclusion, not the raw output.","tools.executeFile":"ctk_execute_file (run code over a file, return only the answer)","tools.executeFileHint":"Read a big file, compute over it, keep its bytes in the sandbox.","tools.batchExecute":"ctk_batch_execute (many commands in one call, per-command status)","tools.batchExecuteHint":"One call for many commands; only conclusions come back.","tools.index":"ctk_index (file large material into the local knowledge base)","tools.indexHint":"Archive long docs, logs or specs, then pull sections with ctk_search.","tools.search":"ctk_search (retrieve excerpts on demand, not the whole thing)","tools.searchHint":"Prefer this for material already in the knowledge base.","tools.fetchAndIndex":"ctk_fetch_and_index (fetch a page straight into the index)","tools.fetchAndIndexHint":"When a whole page would flood the conversation; curl/wget are gated.","tools.purge":"ctk_purge (delete indexed content — permanent)","tools.purgeHint":"Clears the knowledge base; permanent, so the call must confirm:true.","tools.stats":"ctk_stats (this plugin's savings ledger and knowledge usage)","tools.statsHint":"How much this plugin saved, and how much it filed.","tools.doctor":"ctk_doctor (self-check: runtimes, scratch dir, knowledge round-trip)","tools.doctorHint":"Run it first when a child process or indexing misbehaves.","tools.contextReport":"ctk_context_report (session context occupancy report)","tools.contextReportHint":"Lets the model ask for this session context occupancy and composition.","restart":"needs a restart"};
+    const zh = {"title":"上下文工具箱","description":"在沙箱里跑代码、把大块材料归档成可检索的本地知识库、并挡住会灌满对话的抓取与整读。附一张实时上下文占用面板。","hint":"下列开关即时生效，无需重启：门禁在每次工具调用前读取。","routing":"路由门禁","routingHint":"总开关。开启后，每次工具调用执行前都会经过门禁检查。","denyCurlWget":"硬拦 curl / wget","denyCurlWgetHint":"这两个工具的设计就是把整页内容塞进对话；拦掉，改用 ctk_fetch_and_index。","bashNudgeMinCommandBytes":"长命令引导阈值（字节）","bashNudgeMinCommandBytesHint":"命令超过这个长度时提示改用 ctk_batch_execute。填 0 表示不提示。","maxReadBytesBeforeAsk":"大文件整读引导阈值（字节）","maxReadBytesBeforeAskHint":"整读超过这个大小的文件时提示改用 ctk_execute_file。填 0 表示不提示。","nudgeFetchTools":"取网页工具也纳入整页引导","nudgeFetchToolsHint":"把宿主自带的 web_fetch / read_page 也纳入引导：它们与 shell 里的 curl 取回同样一整页，却不在 shell 门禁的视野里。关闭则只拦 shell 一侧。","slimOutputs":"工具输出瘦身","slimOutputsHint":"工具执行后、进入模型历史前压缩：表格保留表头+首尾+带原始行号的中段采样，同构 JSON 走无损重编码，日志折叠掉普通行保住诊断行。有损路径一定把原文逐字节存好，可用 ctk_retrieve 取回。","slimMinBytes":"普通输出瘦身门槛（字节）","slimMinBytesHint":"普通输出小于此值不动。0 表示关闭普通输出瘦身。","slimErrorMinBytes":"错误输出瘦身门槛（字节）","slimErrorMinBytesHint":"留得比普通输出高：调试时要的是保真，不是省字节。","slimOriginalTtlMinutes":"可逆原文保留时长（分钟）","slimOriginalTtlMinutesHint":"瘦身前存的原文保留多久；0 表示一直保留。","trustedReadBasenames":"不受整读门禁限制的文件名","trustedReadBasenamesHint":"逗号分隔；这些文件即使很大也允许整读。","compressNudgeTokens":"占用提示阈值（tokens）","compressNudgeTokensHint":"表面占用达到此值时，占用报告给出压缩建议。填 0 表示不提示。","searchLimit":"检索返回条数上限","searchLimitHint":"ctk_search 每次查询返回的片段数上限。","snippetChars":"片段最大字符数","snippetCharsHint":"ctk_search 每个片段返回的字符上限，超出部分省略。","fetchMaxBytes":"单页抓取上限（字节）","fetchMaxBytesHint":"ctk_fetch_and_index 抓取单页的最大字节数，超出部分丢弃。","storageDir":"知识库根目录","storageDirHint":"留空 = <DSH_HOME>/context-toolkit。","toolsTitle":"工具开关","toolsHint":"逐个启用或停用 ctk_* 工具。","overridden":"已覆盖","reset":"恢复默认","readOnly":"本部署的设置为只读。","unavailable":"该插件当前未加载，暂时无法配置。","save":"保存","saving":"保存中…","saveFailed":"本部署没有接受这些值，已保留供你修改。","invalidNumber":"请填数字；留空表示使用默认值。","note":"门禁开关即时生效；工具开关改动后建议重开页面确认。","ctxTitle":"上下文情况","ctxLoading":"正在读取…","ctxReload":"刷新","ctxFailed":"读取失败：","ctxNoSession":"本会话暂无可测量的投影。","ctxStale":"以下为最近一次实测快照（{time}）：网页请求没有会话边界，实时值请在对话中调用 ctk_context_report。","ctxTotal":"总压力 {n} tokens","ctxSurface":"表面占用 {n} tokens","ctxBudget":"提示阈值 {n} tokens","ctxOver":"已超过提示阈值，建议把大块输出改走 ctk_execute / ctk_index。","ctxUnder":"低于提示阈值，暂不需要干预。","ctxSystem":"系统提示","ctxTools":"工具 schema","ctxMessages":"会话消息","ctxEstimated":"（估算）","ctxNodes":"占用大头（前 {n} 项）","ctxLedger":"本插件节约台账","ctxLedgerLine":"门禁已拦截 {n} 次，约 {bytes} 字节未进入上下文；沙箱保留 {sandbox} 字节；已注册 {tools} 个 ctk_* 工具。","ctxKnowledge":"知识库：{docs} 文档 / {chunks} 片段（{bytes} 字节）","ctxStorage":"知识库位置：{path}","tools.execute":"ctk_execute（在沙箱里跑代码，只把结论带回来）","tools.executeHint":"用于「只想要结论、不想让原始输出占满上下文」的分析型任务。","tools.executeFile":"ctk_execute_file（对某个文件跑代码，只回结论）","tools.executeFileHint":"适合读大文件后做统计或抽取，文件原文留在沙箱里。","tools.batchExecute":"ctk_batch_execute（一次跑多条命令，逐条状态）","tools.batchExecuteHint":"多条命令一次调用，只有结论回来。","tools.index":"ctk_index（把大块材料存进本地知识库）","tools.indexHint":"把长文档、日志、规范归档，之后用 ctk_search 按需取用。","tools.search":"ctk_search（按需检索知识库，取片段而非整份）","tools.searchHint":"知识库里已有的内容优先用它检索，不要整份读进来。","tools.fetchAndIndex":"ctk_fetch_and_index（抓网页并直接并入索引）","tools.fetchAndIndexHint":"抓整页网页又不让它淹没对话时用；curl/wget 会被门禁拦掉。","tools.purge":"ctk_purge（删除已索引内容，不可恢复）","tools.purgeHint":"清理知识库用；会永久删除，调用必须带 confirm:true。","tools.stats":"ctk_stats（节约台账与知识库用量）","tools.statsHint":"想知道这个插件省了多少、归档了多少时看它。","tools.doctor":"ctk_doctor（自检：运行时、临时目录、知识库往返）","tools.doctorHint":"出现异常（子进程跑不起来、索引失败）时先跑它定位。","tools.contextReport":"ctk_context_report（会话上下文占用报告）","tools.contextReportHint":"让模型可以主动查询本会话的上下文占用与构成。","restart":"需重启"};
+    const en = {"title":"Context Toolkit","description":"Run code in a sandbox, file large material into a local searchable knowledge base, and block the fetches and whole-file reads that flood a conversation. Ships a live context-occupancy panel.","hint":"These switches apply immediately, no restart: the gate is read before every tool call.","routing":"Routing gate","routingHint":"Master switch. When on, every tool call passes through the gate before running.","denyCurlWget":"Block curl / wget","denyCurlWgetHint":"Both exist to paste whole responses into the conversation; blocked here, use ctk_fetch_and_index instead.","bashNudgeMinCommandBytes":"Long-command nudge (bytes)","bashNudgeMinCommandBytesHint":"Commands longer than this get a nudge toward ctk_batch_execute. 0 disables it.","maxReadBytesBeforeAsk":"Whole-file read nudge (bytes)","maxReadBytesBeforeAskHint":"Reading a file larger than this gets a nudge toward ctk_execute_file. 0 disables it.","nudgeFetchTools":"Nudge the host fetchers too","nudgeFetchToolsHint":"Also nudge the host own web_fetch / read_page: they pull back the same whole page as a shell curl, and the shell gate never sees them. Off leaves that door unwatched.","slimOutputs":"Slim tool output","slimOutputsHint":"Compress after a tool runs and before its result reaches the model: tables keep a header, both ends and a middle sampled with original line numbers; uniform JSON is re-encoded losslessly; logs drop ordinary lines and keep diagnostics. Any lossy path stores the exact original first, retrievable with ctk_retrieve.","slimMinBytes":"Slimming floor, ordinary output (bytes)","slimMinBytesHint":"Ordinary output below this is left untouched. 0 disables slimming for it.","slimErrorMinBytes":"Slimming floor, error output (bytes)","slimErrorMinBytesHint":"Kept higher than ordinary output: debugging wants fidelity, not economy.","slimOriginalTtlMinutes":"Retention for stored originals (minutes)","slimOriginalTtlMinutesHint":"How long a stored original is kept; 0 keeps it forever.","trustedReadBasenames":"Filenames exempt from the read nudge","trustedReadBasenamesHint":"Comma separated; these may be read whole however large they are.","compressNudgeTokens":"Occupancy warning (tokens)","compressNudgeTokensHint":"At this surface occupancy the report suggests compaction. 0 disables it.","searchLimit":"Search result limit","searchLimitHint":"How many excerpts ctk_search returns per query.","snippetChars":"Excerpt size (chars)","snippetCharsHint":"Character ceiling on each excerpt from ctk_search.","fetchMaxBytes":"Per-page fetch ceiling (bytes)","fetchMaxBytesHint":"ctk_fetch_and_index keeps at most this many bytes of each page.","storageDir":"Knowledge-base root","storageDirHint":"Empty = <DSH_HOME>/context-toolkit.","toolsTitle":"Tool switches","toolsHint":"Enable or disable individual ctk_* tools.","overridden":"overridden","reset":"reset","readOnly":"This deployment is read-only.","unavailable":"The plugin is not loaded right now; it cannot be configured.","save":"Save","saving":"Saving…","saveFailed":"This deployment did not accept those values; they are kept for you to edit.","invalidNumber":"Numbers only; blank means the default.","note":"Gate switches apply immediately; reopen the page to confirm tool-switch changes.","ctxTitle":"Context occupancy","ctxLoading":"Reading…","ctxReload":"Refresh","ctxFailed":"Read failed: ","ctxNoSession":"No measurable projection in this session yet.","ctxStale":"Snapshot from the last real measurement ({time}): a page request has no session boundary; ask ctk_context_report in the conversation for a live reading.","ctxTotal":"{n} tokens total","ctxSurface":"{n} tokens on the surface","ctxBudget":"warning at {n} tokens","ctxOver":"Above the warning threshold — route bulk output through ctk_execute / ctk_index.","ctxUnder":"Below the warning threshold; nothing to do.","ctxSystem":"System prompt","ctxTools":"Tool schemas","ctxMessages":"Conversation","ctxEstimated":" (estimated)","ctxNodes":"Biggest contributors (top {n})","ctxLedger":"This plugin's ledger","ctxLedgerLine":"The gate blocked {n} calls, about {bytes} bytes kept out of context; the sandbox kept out {sandbox} bytes; {tools} ctk_* tools registered.","ctxKnowledge":"Knowledge base: {docs} documents / {chunks} chunks ({bytes} bytes)","ctxStorage":"Knowledge base at {path}","tools.execute":"ctk_execute (run code in the sandbox, return only the answer)","tools.executeHint":"For analysis where you want the conclusion, not the raw output.","tools.executeFile":"ctk_execute_file (run code over a file, return only the answer)","tools.executeFileHint":"Read a big file, compute over it, keep its bytes in the sandbox.","tools.batchExecute":"ctk_batch_execute (many commands in one call, per-command status)","tools.batchExecuteHint":"One call for many commands; only conclusions come back.","tools.index":"ctk_index (file large material into the local knowledge base)","tools.indexHint":"Archive long docs, logs or specs, then pull sections with ctk_search.","tools.search":"ctk_search (retrieve excerpts on demand, not the whole thing)","tools.searchHint":"Prefer this for material already in the knowledge base.","tools.fetchAndIndex":"ctk_fetch_and_index (fetch a page straight into the index)","tools.fetchAndIndexHint":"When a whole page would flood the conversation; curl/wget are gated.","tools.purge":"ctk_purge (delete indexed content — permanent)","tools.purgeHint":"Clears the knowledge base; permanent, so the call must confirm:true.","tools.stats":"ctk_stats (this plugin's savings ledger and knowledge usage)","tools.statsHint":"How much this plugin saved, and how much it filed.","tools.doctor":"ctk_doctor (self-check: runtimes, scratch dir, knowledge round-trip)","tools.doctorHint":"Run it first when a child process or indexing misbehaves.","tools.contextReport":"ctk_context_report (session context occupancy report)","tools.contextReportHint":"Lets the model ask for this session context occupancy and composition.","restart":"needs a restart"};
     //#endregion
 
     /** Fill the {name} holes in a translated sentence. */
@@ -30,8 +30,8 @@ window.__ModuleLoader__.load({
     }
 
     //#region fields
-    const FIELDS = [{"field":"routing","kind":"bool"},{"field":"denyCurlWget","kind":"bool"},{"field":"bashNudgeMinCommandBytes","kind":"number"},{"field":"maxReadBytesBeforeAsk","kind":"number"},{"field":"nudgeFetchTools","kind":"bool"},{"field":"compressNudgeTokens","kind":"number"},{"field":"searchLimit","kind":"number"},{"field":"snippetChars","kind":"number"},{"field":"fetchMaxBytes","kind":"number"},{"field":"trustedReadBasenames","kind":"text"},{"field":"storageDir","kind":"text"}];
-    const TOOL_FIELDS = ["execute","executeFile","batchExecute","index","search","fetchAndIndex","purge","stats","contextReport","doctor"];
+    const FIELDS = [{"field":"routing","kind":"bool"},{"field":"denyCurlWget","kind":"bool"},{"field":"bashNudgeMinCommandBytes","kind":"number"},{"field":"maxReadBytesBeforeAsk","kind":"number"},{"field":"nudgeFetchTools","kind":"bool"},{"field":"compressNudgeTokens","kind":"number"},{"field":"searchLimit","kind":"number"},{"field":"snippetChars","kind":"number"},{"field":"fetchMaxBytes","kind":"number"},{"field":"slimOutputs","kind":"bool"},{"field":"slimMinBytes","kind":"number"},{"field":"slimErrorMinBytes","kind":"number"},{"field":"slimOriginalTtlMinutes","kind":"number"},{"field":"trustedReadBasenames","kind":"text"},{"field":"storageDir","kind":"text"}];
+    const TOOL_FIELDS = ["execute","executeFile","batchExecute","index","search","retrieve","fetchAndIndex","purge","stats","contextReport","doctor"];
     const STATUS_PATH = "/_dsh/context-toolkit/status";
     //#endregion
 

package/index.js

@@ -24,6 +24,7 @@ import { fileURLToPath } from 'node:url'
 import { runProcess, availableRuntimes, probeRuntimes, shellPlan, scratchDir } from './lib/runner.js'
 import { mkdtempSync, rmSync } from 'node:fs'
 import { openStore } from './lib/store.js'
+import { slimText, DEFAULT_LIMITS as SLIM_DEFAULTS } from './lib/compress.js'
 
 export const name = 'dsh-context-toolkit'
 export const inject = ['tools', 'systemPrompt']
@@ -136,6 +137,11 @@ export const Config = z.object({
   denyCurlWget: z.boolean().default(true).volatile().description('硬拦 curl/wget 这类洪水工具'),
   bashNudgeMinCommandBytes: z.number().min(0).default(4096).volatile().description('长命令软引导阈值（字节）'),
   nudgeFetchTools: z.boolean().default(true).volatile().description('把宿主取网页工具（web_fetch/read_page）也纳入整页软引导'),
+  // The gate stops bulk output at the door; this handles what is already inside.
+  slimOutputs: z.boolean().default(true).volatile().description('工具执行后瘦身：结构感知压缩 + 可逆原文，双重门（≤72% 字节且 token 严格下降）'),
+  slimMinBytes: z.number().min(0).default(1400).volatile().description('普通输出进入瘦身的最小字节（0 = 关闭）'),
+  slimErrorMinBytes: z.number().min(0).default(6000).volatile().description('错误输出的最小字节（留高，调试场景不压）'),
+  slimOriginalTtlMinutes: z.number().min(0).default(1440).volatile().description('可逆原文保留时长（分钟），0 = 永久保留'),
   maxReadBytesBeforeAsk: z.number().min(0).default(51200).volatile().description('大文件整读软引导阈值（字节）'),
   trustedReadBasenames: z.array(z.string())
     .default(['package.json', 'tsconfig.json', 'cordis.patch.yml', 'cordis.yml'])
@@ -149,6 +155,7 @@ export const Config = z.object({
   'tools.batchExecute': z.boolean().default(true).volatile().description('注册 ctk_batch_execute'),
   'tools.index': z.boolean().default(true).volatile().description('注册 ctk_index'),
   'tools.search': z.boolean().default(true).volatile().description('注册 ctk_search'),
+  'tools.retrieve': z.boolean().default(true).volatile().description('注册 ctk_retrieve（读取瘦身前存的原文）'),
   'tools.fetchAndIndex': z.boolean().default(true).volatile().description('注册 ctk_fetch_and_index'),
   'tools.purge': z.boolean().default(true).volatile().description('注册 ctk_purge'),
   'tools.stats': z.boolean().default(true).volatile().description('注册 ctk_stats'),
@@ -200,6 +207,7 @@ export function apply(ctx, rawConfig) {
     'tools.batchExecute': true,
     'tools.index': true,
     'tools.search': true,
+    'tools.retrieve': true,
     'tools.fetchAndIndex': true,
     'tools.purge': true,
     'tools.stats': true,
@@ -209,6 +217,11 @@ export function apply(ctx, rawConfig) {
     searchLimit: 5,
     snippetChars: 900,
     fetchMaxBytes: 2000000,
+    nudgeFetchTools: true,
+    slimOutputs: true,
+    slimMinBytes: SLIM_DEFAULTS.minBytes,
+    slimErrorMinBytes: SLIM_DEFAULTS.errorMinBytes,
+    slimOriginalTtlMinutes: 1440,
   }
   const incoming = rawConfig || {}
   const config = {
@@ -223,6 +236,7 @@ export function apply(ctx, rawConfig) {
     'tools.batchExecute': readVolatile(incoming['tools.batchExecute'], defaults['tools.batchExecute']),
     'tools.index': readVolatile(incoming['tools.index'], defaults['tools.index']),
     'tools.search': readVolatile(incoming['tools.search'], defaults['tools.search']),
+    'tools.retrieve': readVolatile(incoming['tools.retrieve'], defaults['tools.retrieve']),
     'tools.fetchAndIndex': readVolatile(incoming['tools.fetchAndIndex'], defaults['tools.fetchAndIndex']),
     'tools.purge': readVolatile(incoming['tools.purge'], defaults['tools.purge']),
     'tools.stats': readVolatile(incoming['tools.stats'], defaults['tools.stats']),
@@ -232,6 +246,11 @@ export function apply(ctx, rawConfig) {
     searchLimit: readVolatile(incoming.searchLimit, defaults.searchLimit),
     snippetChars: readVolatile(incoming.snippetChars, defaults.snippetChars),
     fetchMaxBytes: readVolatile(incoming.fetchMaxBytes, defaults.fetchMaxBytes),
+    nudgeFetchTools: readVolatile(incoming.nudgeFetchTools, defaults.nudgeFetchTools),
+    slimOutputs: readVolatile(incoming.slimOutputs, defaults.slimOutputs),
+    slimMinBytes: readVolatile(incoming.slimMinBytes, defaults.slimMinBytes),
+    slimErrorMinBytes: readVolatile(incoming.slimErrorMinBytes, defaults.slimErrorMinBytes),
+    slimOriginalTtlMinutes: readVolatile(incoming.slimOriginalTtlMinutes, defaults.slimOriginalTtlMinutes),
   }
 
   const storageDir = (typeof config.storageDir === 'string' && config.storageDir.trim())
@@ -246,6 +265,14 @@ export function apply(ctx, rawConfig) {
     registered: 0,
     bytesIndexed: 0,
     bytesSandboxed: 0,
+    // Output slimming: what the post-execute seam removed before the bytes reached
+    // the model, and what it stored so the removal stays reversible.
+    slimmed: 0,
+    slimmedBytes: 0,
+    slimmedTokens: 0,
+    slimSkipped: 0,
+    slimStrategies: {},
+    blobsStored: 0,
     calls: {},
     startedAt: Date.now(),
   }
@@ -258,6 +285,7 @@ export function apply(ctx, rawConfig) {
   }
 
   registerRoutingGate(ctx, config, stats)
+  registerOutputSlimmer(ctx, config, stats, store, log)
   registerTools(ctx, config, stats, storageDir, log)
   if (store) registerKnowledgeTools(ctx, config, stats, store, log)
   else log.info('knowledge tools skipped (no store)')
@@ -441,6 +469,121 @@ function baseNameOf(p) {
   return i >= 0 ? s.slice(i + 1) : s
 }
 
+/**
+ * Slave the seam that runs after a tool body settles but before its result reaches
+ * the session log and the model's history.
+ *
+ * The gate above stops bulk output at the door; this handles what is already inside.
+ * Both halves exist because they catch different traffic: the gate refuses a call
+ * whose *arguments* say it will flood, while this sees the bytes that actually came
+ * back — from a tool the gate does not know, from an entry point it does not cover,
+ * or from output that was simply never predictable from the call.
+ *
+ * Two rules keep it honest. Anything lossy stores the exact original under an id
+ * before returning slim text, so a restore is byte-for-byte; and the compressor's
+ * never-worse gates decide adoption, so the ledger never claims a saving that did
+ * not happen.
+ */
+function registerOutputSlimmer(ctx, config, stats, store, log) {
+  if (config.slimOutputs === false) return
+  if (!store || typeof ctx.on !== 'function') return
+
+  // Our own tools are exempt. ctk_retrieve exists to hand back a full original, so
+  // slimming its result would return the same elided preview the model just paid a
+  // turn to unfold — an expand loop with no exit.
+  const OWN_TOOLS = new Set(['ctk_retrieve', 'ctk_search', 'ctk_stats', 'ctk_context_report', 'ctk_doctor'])
+  // A file the model is about to edit must arrive exactly. The gate's read branch
+  // nudges an oversized whole-file read, and when it allows one the bytes matter.
+  const EXEMPT = new Set(['read'])
+  // Freshness-sensitive results must not be replayed: a stub saying "identical to
+  // 3s ago" is only useful for stable data.
+  const DEDUPE_TTL_MS = 90000
+
+  ctx.on('tools/post-execute', async function (exec, result, next) {
+    let decision
+    try {
+      decision = await next()
+    } catch (error) {
+      log.warn('post-execute next() failed: ' + String((error && error.message) || error))
+      throw error
+    }
+    try {
+      if (!decision || decision.kind !== 'accept') return decision
+      if (Object.prototype.hasOwnProperty.call(decision, 'value')) return decision
+
+      const name = String((exec && exec.name) || '')
+      if (!name || OWN_TOOLS.has(name) || EXEMPT.has(name)) return decision
+      const args = (exec && exec.arguments) || {}
+      const isError = result && result.isError === true
+      const sourceBlocks = decision.content !== undefined ? decision.content : (result && result.content)
+      if (!Array.isArray(sourceBlocks) || !sourceBlocks.length) return decision
+
+      const limits = {
+        minBytes: Number(config.slimMinBytes) || 0,
+        errorMinBytes: Number(config.slimErrorMinBytes) || 0,
+      }
+      if (limits.minBytes <= 0 && limits.errorMinBytes <= 0) return decision
+
+      let touched = false
+      const rewritten = []
+      for (const block of sourceBlocks) {
+        if (!block || block.type !== 'text' || typeof block.text !== 'string') {
+          rewritten.push(block)
+          continue
+        }
+        const outcome = slimText(block.text, { limits, isError })
+        if (!outcome) { stats.slimSkipped += 1; rewritten.push(block); continue }
+
+        let marker = ''
+        if (!outcome.lossless) {
+          const id = 'ctk-' + Date.now().toString(36) + '-' + Math.random().toString(36).slice(2, 8)
+          try {
+            store.putBlob(id, block.text)
+          } catch (error) {
+            log.warn('could not store the original for ' + id + ': ' + String((error && error.message) || error))
+            stats.slimSkipped += 1
+            rewritten.push(block)
+            continue
+          }
+          stats.blobsStored += 1
+          marker = '\n\n[ctk_retrieve id=' + id + '] the exact original (' + outcome.originalBytes
+            + ' bytes) is stored verbatim; call ctk_retrieve with this id to read it back byte-for-byte.'
+        } else {
+          marker = '\n\n[lossless: keys written once instead of per row; no information was dropped.]'
+        }
+
+        const slimmed = outcome.text + marker
+        // The never-worse gates already ran inside slimText. The marker is added
+        // afterwards and could in principle push the whole thing back over the line,
+        // so the byte gate is applied once more to the text that will actually land.
+        if (Buffer.byteLength(slimmed, 'utf8') >= outcome.originalBytes) {
+          stats.slimSkipped += 1
+          rewritten.push(block)
+          if (marker) { try { store.purgeBlobs({ all: false }) } catch { /* blob stays, harmless */ } }
+          continue
+        }
+
+        stats.slimmed += 1
+        stats.slimmedBytes += outcome.originalBytes - Buffer.byteLength(slimmed, 'utf8')
+        stats.slimmedTokens += outcome.savedTokens
+        stats.slimStrategies[outcome.strategy] = (stats.slimStrategies[outcome.strategy] || 0) + 1
+        void args
+        touched = true
+        rewritten.push({ ...block, text: slimmed })
+      }
+
+      if (!touched) return decision
+      return { ...decision, content: rewritten }
+    } catch (error) {
+      // A slimmer that breaks a tool call is worse than one that misses a saving.
+      log.warn('output slimming failed for ' + String((exec && exec.name) || '?') + ': ' + String((error && error.message) || error))
+      return decision
+    }
+  })
+
+  void DEDUPE_TTL_MS
+}
+
 function registerRoutingGate(ctx, config, stats) {
   if (!config.routing) return
 
@@ -918,6 +1061,47 @@ function registerKnowledgeTools(ctx, config, stats, store, log) {
     stats.registered += 1
   }
 
+  if (toolSwitch(config, 'retrieve')) {
+    ctx.tools.register(textTool({
+      name: 'ctk_retrieve',
+      description: 'Read back a compressed tool output byte-for-byte. When output was slimmed, its exact original was stored under an id and the slim text carries an id marker; pass that id here to get the full original back. Use it when the excerpt you received is not enough to act on — never guess from the slimmed text alone.',
+      parameters: {
+        type: 'object',
+        properties: {
+          id: { type: 'string', description: 'The id from a [ctk_retrieve id=…] marker.' },
+        },
+        required: ['id'],
+      },
+      render: (_a, v) => [{ type: 'text', text: v && typeof v.text === 'string' ? v.text : 'ctk_retrieve finished' }],
+      async execute(args) {
+        const id = String((args && args.id) || '').trim()
+        if (!id) return { text: 'id is required', isError: true }
+        const found = store.getBlob(id)
+        if (!found) {
+          const totals = store.stats()
+          return {
+            text: 'no stored original for "' + id + '". It may have passed its retention window or belong '
+              + 'to another install. The knowledge base currently holds ' + totals.blobs + ' stored original(s).',
+            isError: true,
+          }
+        }
+        stats.calls['ctk_retrieve'] = (stats.calls['ctk_retrieve'] || 0) + 1
+        // isError must be present on every return: the host validates a tool's output
+        // against the declared schema and rejects the call outright when it is missing.
+        // Omitting it here made the whole tool unusable — the marker was injected, the
+        // original was stored, and the read-back failed with "missing required property".
+        return {
+          text: '## ctk_retrieve ' + id + '\n'
+            + '- bytes: ' + found.bytes + ' (exact original, restored verbatim)\n'
+            + '- stored: ' + new Date(found.createdAtMs).toISOString() + '\n\n'
+            + found.text,
+          isError: false,
+        }
+      },
+    }, 'ctk_retrieve'))
+    stats.registered += 1
+  }
+
   if (toolSwitch(config, 'search')) {
     ctx.tools.register(textTool({
       name: 'ctk_search',
@@ -1232,8 +1416,11 @@ function registerObservabilityTools(ctx, config, stats, store, storageDir, log)
       description: 'Report this plugin\'s savings ledger and knowledge-base usage. Shows how many calls the routing gate refused, how many bytes the sandbox kept out of the conversation, how many bytes were filed, and the per-tool call counts.',
       parameters: { type: 'object', properties: {}, additionalProperties: false },
       async execute() {
-        const kb = store ? store.stats() : { documents: 0, chunks: 0, bytes: 0 }
-        const produced = stats.bytesSandboxed + stats.bytesIndexed + stats.deniedBytes
+        const kb = store ? store.stats() : { documents: 0, chunks: 0, bytes: 0, blobs: 0, blobBytes: 0 }
+        // Slimmed bytes are kept out too — they were removed from output before it
+        // reached the model. Leaving them out understated the ledger by exactly the
+        // amount this half of the plugin exists to save.
+        const produced = stats.bytesSandboxed + stats.bytesIndexed + stats.deniedBytes + stats.slimmedBytes
         const keptOut = produced
         const uptimeMs = Date.now() - stats.startedAt
         const lines = [
@@ -1243,12 +1430,14 @@ function registerObservabilityTools(ctx, config, stats, store, storageDir, log)
           '- 门禁拦截 denied: ' + stats.denied + ' 次（约 ' + stats.deniedBytes + ' 字节未进入上下文）',
           '- 沙箱保留 bytesSandboxed: ' + stats.bytesSandboxed + ' 字节（子进程产出但只回结论的部分）',
           '- 已归档 bytesIndexed: ' + stats.bytesIndexed + ' 字节（存入知识库、未留在对话里）',
+          '- 输出瘦身 slimmed: ' + stats.slimmed + ' 块（' + stats.slimmedBytes + ' 字节 ≈ ' + stats.slimmedTokens + ' tokens 未进入上下文；跳过的块 ' + stats.slimSkipped + '）',
           '- 合计 kept out of context: ' + keptOut + ' 字节 ≈ ' + Math.round(keptOut / 4) + ' tokens（按 4 字节/token 估算）',
           '',
           '### 知识库 / knowledge base',
           '- 文档 documents: ' + kb.documents,
           '- 片段 chunks: ' + kb.chunks,
           '- 原文字节 bytes: ' + kb.bytes,
+          '- 可逆原文 blobs: ' + (kb.blobs || 0) + '（' + (kb.blobBytes || 0) + ' 字节，可用 ctk_retrieve 逐字节取回）',
           '- 位置: ' + storageDir,
           '',
           '### 本插件注册的工具',
@@ -1320,6 +1509,9 @@ function registerObservabilityTools(ctx, config, stats, store, storageDir, log)
         lines.push('- 门禁拦截 denied: ' + stats.denied + ' 次（' + stats.deniedBytes + ' 字节未进入上下文）')
         lines.push('- 沙箱保留 bytesSandboxed: ' + stats.bytesSandboxed + ' 字节')
         lines.push('- 已归档 bytesIndexed: ' + stats.bytesIndexed + ' 字节')
+        lines.push('- 输出瘦身 slimmed: ' + stats.slimmed + ' 块（' + stats.slimmedBytes + ' 字节 ≈ ' + stats.slimmedTokens + ' tokens 未进入上下文）')
+        lines.push('- 可逆原文 blobsStored: ' + stats.blobsStored + ' 份（可用 ctk_retrieve 逐字节取回）')
+        if (stats.slimmed > 0) lines.push('- 瘦身策略: ' + Object.keys(stats.slimStrategies).map((k) => k + '×' + stats.slimStrategies[k]).join(' · '))
         lines.push('- 已注册 registered: ' + stats.registered)
         return { text: lines.join('\n'), isError: false }
       },

package/lib/compress.js

@@ -0,0 +1,251 @@
+/**
+ * Deterministic output slimming.
+ *
+ * The gate in index.js stops bulk output at the door; this module handles what is
+ * already inside. Nothing here calls a model: every strategy is a rule, so the same
+ * input always yields the same output and a regression is reproducible.
+ *
+ * The red line is reversibility. Anything lossy is accompanied by a blob marker, and
+ * the caller stores the exact original under that id before returning the slim text,
+ * so a restore is byte-for-byte. Two gates stand between a candidate and adoption:
+ * the result must be at most keepRatioMax of the original bytes AND its estimated
+ * token count must be strictly lower. A compression that saves neither is not a
+ * compression — it is a loss with a receipt.
+ */
+import { Buffer } from 'node:buffer'
+
+/** Token estimate, matching the ledger's own accounting (4 bytes per token). */
+export function estimateTokens(text) {
+  return Math.ceil(Buffer.byteLength(String(text == null ? '' : text), 'utf8') / 4)
+}
+
+export const DEFAULT_LIMITS = {
+  /** Ordinary output below this stays untouched. */
+  minBytes: 1400,
+  /** Error output gets a much higher floor: debugging needs fidelity, not economy. */
+  errorMinBytes: 6000,
+  /** Byte gate: an adopted replacement may not exceed this fraction of the original. */
+  keepRatioMax: 0.72,
+  /** Window shape for tabular output: verbatim head, stride-sampled middle, verbatim tail. */
+  tableHeadRows: 60,
+  tableTailRows: 40,
+  tableStrideSamples: 40,
+  /** Window shape for prose and logs. */
+  headChars: 1200,
+  tailChars: 800,
+  /** Uniform JSON arrays up to this many rows are re-encoded losslessly. */
+  losslessArrayMaxRows: 200,
+}
+
+/** Line-numbered tool output is code: the model may be about to edit exact bytes. */
+function looksLikeNumberedSource(text) {
+  const lines = text.split('\n')
+  let numbered = 0
+  for (const line of lines) {
+    if (/^\s*\d+[|\u2192]/.test(line)) numbered += 1
+  }
+  return lines.length > 4 && numbered / lines.length > 0.5
+}
+
+/** What kind of output is this? The strategy follows the shape, not a guess. */
+export function classify(text) {
+  const source = String(text || '')
+  const lines = source.split('\n').filter((l) => l.length > 0)
+  if (!lines.length) return 'empty'
+
+  const trimmed = source.trim()
+  if ((trimmed.startsWith('[') || trimmed.startsWith('{')) && trimmed.length > 40) {
+    try {
+      const parsed = JSON.parse(trimmed)
+      if (Array.isArray(parsed) && parsed.length >= 5 && uniformObjectKeys(parsed)) return 'json-array'
+    } catch { /* not JSON after all: fall through to the shape tests */ }
+  }
+
+  if (looksLikeNumberedSource(source)) return 'code'
+
+  // Two separators already mean three columns: a table does not have to be wide to be
+  // a table. The first cut required three, so an ordinary three-column report fell
+  // through to the prose window and lost its line coordinates along with the saving.
+  const pipeRows = lines.filter((l) => (l.match(/\|/g) || []).length >= 2)
+  if (lines.length >= 8 && pipeRows.length / lines.length >= 0.7) return 'table'
+
+  // Deciding this before the search test matters: a timestamped log (a date, a time,
+  // then a level) matches a naive path:line pattern, and classifying it as search
+  // folds a log by file and throws away the diagnostics that fold exists to protect.
+  //
+  // The test is the *shape* — a level prefix or a timestamp on most lines. Keying it
+  // on error density instead put a log with no errors into the prose bucket, which
+  // happens to be the same window today and stops being so the moment either changes.
+  const logShaped = lines.filter((l) => /^\s*[\[(]?\d{2}:\d{2}|^\s*\[[a-z]+\]|^\s*(INFO|DEBUG|WARN|ERROR|TRACE)\b|^\s*\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}/i.test(l))
+  if (lines.length >= 20 && logShaped.length / lines.length >= 0.4) return 'log'
+
+  // Search output is a path, a line number, then a colon — and the path carries no
+  // whitespace. Without that last part the timestamp above matched it.
+  const hits = lines.filter((l) => /^[^\s:][^\s:]*:\d+(?::\d+)?[:.]/.test(l))
+  if (lines.length >= 8 && hits.length / lines.length >= 0.4) return 'search'
+
+  return 'prose'
+}
+
+/** Whether every element of an array is a flat object with the same keys. */
+function uniformObjectKeys(rows) {
+  const first = rows[0]
+  if (first === null || typeof first !== 'object' || Array.isArray(first)) return false
+  const keys = Object.keys(first)
+  if (!keys.length) return false
+  const signature = keys.slice().sort().join('\u0000')
+  let same = 0
+  for (const row of rows) {
+    if (row === null || typeof row !== 'object' || Array.isArray(row)) continue
+    if (Object.keys(row).slice().sort().join('\u0000') === signature) same += 1
+  }
+  return same / rows.length >= 0.8
+}
+
+/**
+ * Re-encode a uniform array without losing anything.
+ *
+ * Writing the keys once instead of once per row is where the saving comes from; the
+ * values stay JSON, so the encoding is exact rather than approximate. This is the
+ * route preferred whenever it applies, because a lossless saving needs no marker and
+ * no restore.
+ */
+export function toonEncode(rows, name = 'rows') {
+  const keys = Object.keys(rows[0])
+  const header = name + '[' + rows.length + ']{' + keys.join(',') + '}:'
+  const body = rows.map((row) => keys.map((k) => JSON.stringify(row[k] ?? null)).join(',')).join('\n')
+  return { text: header + '\n' + body, keys }
+}
+
+/** Keep the head, stride-sample the middle with original line numbers, keep the tail. */
+function windowTable(lines, header, limits) {
+  const { tableHeadRows: head, tableTailRows: tail, tableStrideSamples: samples } = limits
+  if (lines.length <= head + tail) return null
+  const middle = lines.slice(head, lines.length - tail)
+  const stride = Math.max(1, Math.floor(middle.length / Math.max(1, samples)))
+  const picked = []
+  for (let i = 0; i < middle.length; i += stride) {
+    picked.push('L' + (head + i + 1) + ': ' + middle[i])
+  }
+  const kept = [...lines.slice(0, head), '… ' + (middle.length - picked.length) + ' rows sampled below (line numbers are original) …',
+    ...picked, '… ' + tail + ' tail rows …', ...lines.slice(lines.length - tail)]
+  return header ? [header].concat(kept) : kept
+}
+
+/** Keep a head and a tail of a plain-text body, with an explicit elision marker. */
+function windowText(text, limits) {
+  const { headChars, tailChars } = limits
+  if (text.length <= headChars + tailChars) return null
+  const head = text.slice(0, headChars)
+  const tail = text.slice(text.length - tailChars)
+  const dropped = text.length - headChars - tailChars
+  return head + '\n… ' + dropped + ' characters elided …\n' + tail
+}
+
+/**
+ * Try to slim one text block.
+ *
+ * Returns an outcome only when a gate accepted it, and null when the body is too
+ * small, is code, or when every candidate failed the never-worse gates. A null is
+ * the honest answer: it means "leave this text alone".
+ */
+export function slimText(text, options = {}) {
+  const limits = { ...DEFAULT_LIMITS, ...(options.limits || {}) }
+  const source = String(text == null ? '' : text)
+  const originalBytes = Buffer.byteLength(source, 'utf8')
+  const floor = options.isError ? limits.errorMinBytes : limits.minBytes
+  if (originalBytes < floor) return null
+
+  const kind = classify(source)
+  if (kind === 'code' || kind === 'empty') return null
+
+  // Candidates are ranked. When a structural strategy cannot clear the never-worse
+  // gates it is not the end of the story — a plain window almost always can, and
+  // giving up instead left large, plainly savable bodies untouched (a grep result
+  // whose files each held one match folded to the same size, and was kept intact).
+  const candidates = []
+  const lossless = false
+
+  if (kind === 'json-array') {
+    const rows = JSON.parse(source.trim())
+    if (rows.length <= limits.losslessArrayMaxRows) {
+      const encoded = toonEncode(rows, options.label || 'rows')
+      candidates.push({ text: encoded.text, strategy: 'json-array lossless re-encoding', lossless: true })
+    } else {
+      const lines = rows.map((r) => JSON.stringify(r))
+      const windowed = windowTable(lines, '# ' + rows.length + ' rows, one JSON object per line:', limits)
+      if (windowed) candidates.push({ text: windowed.join('\n'), strategy: 'json-array windowed sampling' })
+    }
+  } else if (kind === 'table') {
+    const lines = source.split('\n')
+    const windowed = windowTable(lines, '', limits)
+    if (windowed) candidates.push({ text: windowed.join('\n'), strategy: 'table window with line coordinates' })
+  } else if (kind === 'search') {
+    const lines = source.split('\n').filter((l) => l.length > 0)
+    const byFile = new Map()
+    for (const line of lines) {
+      const m = line.match(/^([^:]+):(\d+)/)
+      const file = m ? m[1] : '(unlabelled)'
+      if (!byFile.has(file)) byFile.set(file, [])
+      byFile.get(file).push(line)
+    }
+    const folded = []
+    for (const [file, group] of byFile) {
+      folded.push(file + ' (' + group.length + ' matches)')
+      for (const line of group.slice(0, 12)) folded.push('  ' + line)
+      if (group.length > 12) folded.push('  … ' + (group.length - 12) + ' more matches in this file …')
+    }
+    candidates.push({ text: folded.join('\n'), strategy: 'search folded by file' })
+  } else if (kind === 'log') {
+    const lines = source.split('\n')
+    const interesting = lines.filter((l) => /error|fail|exception|assert|traceback|panic|warn/i.test(l))
+    const others = lines.filter((l) => !interesting.includes(l))
+    if (interesting.length && interesting.length < lines.length) {
+      candidates.push({
+        text: ['# ' + lines.length + ' lines, ' + others.length + ' repeat/ordinary lines folded away, diagnostics kept verbatim:']
+          .concat(interesting.slice(0, 400)).join('\n'),
+        strategy: 'log fold keeping diagnostics',
+      })
+    }
+  }
+
+  // The window is the shape-independent fallback, and it runs for every kind, not
+  // only for prose. A structural strategy that cannot clear the gates should lose to
+  // it, not cancel the attempt.
+  const windowed = windowText(source, limits)
+  if (windowed) candidates.push({ text: windowed, strategy: kind === 'prose' ? 'prose head/tail window' : 'head/tail window (fallback)' })
+
+  if (!candidates.length) return null
+
+  // The never-worse gates, applied to every candidate. Bytes first, then a strictly
+  // lower token estimate: a replacement that saves neither is not a compression, it
+  // is a loss with a receipt, and adopting it would make the ledger a liar.
+  const tokensBefore = estimateTokens(source)
+  let best = null
+  for (const item of candidates) {
+    const bytes = Buffer.byteLength(item.text, 'utf8')
+    const ratio = bytes / Math.max(1, originalBytes)
+    if (ratio > limits.keepRatioMax) continue
+    const tokensAfter = estimateTokens(item.text)
+    if (!(tokensAfter < tokensBefore)) continue
+    // First passing candidate wins, and candidates are ordered by how much they
+    // preserve rather than by how small they are. Picking the smallest byte count
+    // instead let a plain window beat the table window — which threw away the line
+    // coordinates that exist precisely so the model can ask for a middle row.
+    best = { ...item, slimBytes: bytes, tokensAfter }
+    break
+  }
+  if (!best) return null
+
+  return {
+    text: best.text,
+    strategy: best.strategy,
+    lossless: best.lossless === true,
+    originalBytes,
+    slimBytes: best.slimBytes,
+    savedBytes: originalBytes - best.slimBytes,
+    tokensBefore,
+    tokensAfter: best.tokensAfter,
+    savedTokens: tokensBefore - best.tokensAfter,
+  }
+}

package/lib/store.js

@@ -108,6 +108,18 @@ export function openStore(storageDir) {
       ord UNINDEXED,
       tokenize = 'unicode61'
     );
+    -- Reversible storage. The chunks table is lossy by construction (it trims and
+    -- hard-splits at maxChars), so it can never hand back the exact bytes it was
+    -- given. Compression needs the opposite guarantee: whatever replaces the text
+    -- must be restorable byte-for-byte, which means the original has to be kept
+    -- somewhere verbatim. Keeping it in this same database rather than a second
+    -- store is the point — one file, two jobs: FTS5 finds sections, blobs restore them.
+    CREATE TABLE IF NOT EXISTS blobs (
+      blob_id TEXT PRIMARY KEY,
+      body BLOB NOT NULL,
+      bytes INTEGER NOT NULL,
+      created_at_ms INTEGER NOT NULL
+    );
   `)
   db.prepare('INSERT OR REPLACE INTO meta(key, value) VALUES (?, ?)').run('schema_version', String(SCHEMA_VERSION))
   return makeApi(db, dbPath, storageDir)
@@ -127,7 +139,46 @@ function makeApi(db, dbPath, storageDir) {
     (doc_key, path, title, bytes, mtime_ms, chunks, indexed_at_ms) VALUES (?, ?, ?, ?, ?, ?, ?)`)
   const findDoc = db.prepare('SELECT * FROM documents WHERE doc_key = ?')
 
+  const insertBlob = db.prepare('INSERT OR REPLACE INTO blobs(blob_id, body, bytes, created_at_ms) VALUES (?, ?, ?, ?)')
+  const selectBlob = db.prepare('SELECT body, bytes, created_at_ms FROM blobs WHERE blob_id = ?')
+
   return {
+    /**
+     * Store an original verbatim under a caller-chosen id.
+     *
+     * The bytes go in as a BLOB and come back as a Buffer, so nothing between the
+     * two ends re-encodes them: a restore is only meaningful if it is exact, and
+     * TEXT round-tripping is one surprise away from not being.
+     */
+    putBlob(id, text) {
+      const blobId = String(id || '')
+      if (!blobId) throw new Error('putBlob needs an id')
+      const body = Buffer.from(String(text == null ? '' : text), 'utf8')
+      insertBlob.run(blobId, body, body.length, Date.now())
+      return { id: blobId, bytes: body.length }
+    },
+
+    /** Read a stored original back byte-for-byte; null when the id is unknown. */
+    getBlob(id) {
+      const row = selectBlob.get(String(id || ''))
+      if (!row) return null
+      return { id: String(id), text: Buffer.from(row.body).toString('utf8'), bytes: Number(row.bytes) || 0, createdAtMs: Number(row.created_at_ms) || 0 }
+    },
+
+    /** Drop stored originals older than the given age, or all of them. */
+    purgeBlobs({ olderThanMs = 0, all = false } = {}) {
+      if (all) {
+        const n = db.prepare('SELECT COUNT(*) AS n FROM blobs').get().n
+        db.exec('DELETE FROM blobs;')
+        return { removed: Number(n) || 0 }
+      }
+      if (!(olderThanMs > 0)) return { removed: 0 }
+      const cutoff = Date.now() - olderThanMs
+      const n = db.prepare('SELECT COUNT(*) AS n FROM blobs WHERE created_at_ms < ?').get(cutoff).n
+      db.prepare('DELETE FROM blobs WHERE created_at_ms < ?').run(cutoff)
+      return { removed: Number(n) || 0 }
+    },
+
     /** File one document, replacing any earlier version of the same path. */
     indexDocument({ path, title, text, mtimeMs }) {
       const abs = normalizeTarget(path)
@@ -231,7 +282,10 @@ function makeApi(db, dbPath, storageDir) {
       const docs = db.prepare('SELECT COUNT(*) AS n FROM documents').get().n
       const chunks = db.prepare('SELECT COUNT(*) AS n FROM chunks').get().n
       const bytes = db.prepare('SELECT COALESCE(SUM(bytes),0) AS n FROM documents').get().n
-      return { documents: Number(docs) || 0, chunks: Number(chunks) || 0, bytes: Number(bytes) || 0, dbPath }
+      const blobs = db.prepare('SELECT COUNT(*) AS n FROM blobs').get().n
+      const blobBytes = db.prepare('SELECT COALESCE(SUM(bytes),0) AS n FROM blobs').get().n
+      return { documents: Number(docs) || 0, chunks: Number(chunks) || 0, bytes: Number(bytes) || 0,
+        blobs: Number(blobs) || 0, blobBytes: Number(blobBytes) || 0, dbPath }
     },
 
     close() { try { db.close() } catch { /* already closed */ } },

package/locale/en.json

@@ -68,6 +68,8 @@
     "tools.indexHint": "Archive long docs, logs or specs, then pull sections with ctk_search.",
     "tools.search": "ctk_search (retrieve excerpts on demand, not the whole thing)",
     "tools.searchHint": "Prefer this for material already in the knowledge base.",
+    "tools.retrieve": "ctk_retrieve (read back a stored original byte-for-byte)",
+    "tools.retrieveHint": "When slimmed text carries a [ctk_retrieve id=...] marker, the exact original was stored; call this to get it back rather than guessing from the condensed text.",
     "tools.fetchAndIndex": "ctk_fetch_and_index (fetch a page straight into the index)",
     "tools.fetchAndIndexHint": "When a whole page would flood the conversation; curl/wget are gated.",
     "tools.purge": "ctk_purge (delete indexed content — permanent)",

package/locale/zh.json

@@ -68,6 +68,8 @@
     "tools.indexHint": "把长文档、日志、规范归档，之后用 ctk_search 按需取用。",
     "tools.search": "ctk_search（按需检索知识库，取片段而非整份）",
     "tools.searchHint": "知识库里已有的内容优先用它检索，不要整份读进来。",
+    "tools.retrieve": "ctk_retrieve（取回瘦身前存的逐字节原文）",
+    "tools.retrieveHint": "收到带有 [ctk_retrieve id=…] 标记的瘦身文本时，说明原文被逐字节存下了；片段不够用就调它取回全文，不要凭精简文本猜。",
     "tools.fetchAndIndex": "ctk_fetch_and_index（抓网页并直接并入索引）",
     "tools.fetchAndIndexHint": "抓整页网页又不让它淹没对话时用；curl/wget 会被门禁拦掉。",
     "tools.purge": "ctk_purge（删除已索引内容，不可恢复）",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-context-toolkit",
-  "version": "0.2.4",
+  "version": "0.3.1",
   "description": "DeepSeek Harness bundle: native ctk_* sandbox tools, a routing gate that keeps bulk output out of the transcript, and a local full-text knowledge base. No third-party runtime dependencies.",
   "repository": {
     "type": "git",