dsh-context-toolkit 0.3.1 → 0.3.3

package/CHANGELOG.md

@@ -0,0 +1,45 @@
+# Changelog (dsh-context-toolkit)
+
+> [简体中文](./CHANGELOG.zh.md) | **English**
+
+User-visible changes, newest first. Commit hashes in brackets.
+
+## 0.3.3
+- **Docs**: adds this bilingual changelog, listed in the manifest and gated by release-check; the README and repository description were refreshed too. The npm package now carries the changelog with each release. [`01d7107`]
+
+## 0.3.2
+- **Fixed**: `slimOriginalTtlMinutes` was displayed in the settings page and never read by the code, so the expiry the panel promised never happened and stored originals accumulated without limit; the sweep now runs once on every load.
+- **Fixed**: the rollback call `purgeBlobs({ all: false })` passed no age and was therefore a no-op, leaving the original forever; a rollback now removes exactly the id it wrote and cannot take an unrelated original with it.
+- The ledger reports the effective retention, making a consumed setting visible. [`98f6a06`]
+
+## 0.3.1
+- **Fixed**: `ctk_retrieve` omitted `isError` on its success path; the host validates a tool's output against its schema and rejects the call when a required property is missing, so read-back was unusable — the marker was injected and the original stored, and it could not be read. [`75ea559`]
+- The ledger now renders slimming counts (blocks, bytes, tokens, skipped) and stored originals, and counts slimmed bytes as kept out of context.
+
+## 0.3.0
+- **New: output slimming at `tools/post-execute`** — the gate guards the door; this handles what is already inside: uniform JSON re-encoded losslessly, tables keeping header and both ends with the middle sampled by original line number, grep/search folded by file, logs folded with diagnostics kept verbatim, code and key-value left alone.
+- **Two never-worse gates**: at most 72% of the bytes and a strictly lower token estimate, or the candidate is dropped.
+- **Reversibility is the red line**: every lossy path stores the exact original first and injects a `[ctk_retrieve id=…]` marker, read back with the new `ctk_retrieve`. Originals live in the **same SQLite database as the FTS5 index** (a `blobs` table; BLOB access so nothing re-encodes the bytes). [`61f162d`]
+
+## 0.2.4
+- **Fixed**: the `read` branch repeated `script_path`'s path-base mistake — a relative path was resolved against the plugin process's cwd while the host opens it against the session working directory; measured on live code, the same 70 KB file was refused by absolute path and read whole by relative path. [`8607fb2`]
+- **New**: the host's own fetchers (`web_fetch`, `read_page`) are nudged too — previously curl in a shell was refused while the same page walked in through them.
+
+## 0.2.3
+- **Fixed**: a relative `script_path` was resolved against a different base than the host uses, so the gate read nothing and waved the call through; resolution now uses the session working directory. [`5820210`]
+
+## 0.2.2
+- **Fixed**: gate matching moved from a whole-line substring with delimiters to splitting at command separators and comparing each segment's command word — fixing both directions (misses on non-leading invocations and prefixed/pathed ones, false denials where the tool name is just an argument).
+- **Fixed**: `ops_bash` runs a file when handed `script_path`, which the gate never saw; the script is now read (first 256 KB).
+- **Fixed**: the suite leaked its own handles (the mock discarded the disposer, so the SQLite handle stayed open and Windows could not delete the temp directory). [`da1b0fd`]
+
+## 0.2.1
+- **Fixed**: `ctk_doctor` printed a hardcoded `0.1.0` while the bundle shipped 0.2.0; it now reads its own `package.json` at load.
+- **New**: `release-check.mjs`, the release-readiness gate. [`d98835a`]
+
+## 0.2.0
+- **Fixed**: `guard` returned `null` instead of `undefined` when allowing, and the host denies whenever the reason is not undefined — every tool call was denied (run_code failed with a bare `Error: null`).
+- **Fixed**: the gate recognised only the shell names it shipped with, so `curl` through `ops_bash` slipped past; every shell name is covered now.
+
+## 0.1.0
+- First release: `ctk_execute` / `ctk_execute_file` / `ctk_batch_execute` (run code in a child process, return only the conclusion), `ctk_index` / `ctk_search` (a local FTS5 knowledge base on `node:sqlite`), the routing gate, `ctk_stats` / `ctk_doctor` and a live occupancy panel. [`87b35fe`]

package/CHANGELOG.zh.md

@@ -0,0 +1,45 @@
+# 更新说明（dsh-context-toolkit）
+
+> **简体中文** | [English](./CHANGELOG.md)
+
+本文件按版本记录用户可感知的变化。方括号内为对应提交。
+
+## 0.3.3
+- **文档**：新增中英双语更新说明（本文件），并纳入发布清单与发布门；README 与仓库首页说明同步更新。npm 包开始随版本携带 CHANGELOG。[`01d7107`]
+
+## 0.3.2
+- **修复**：`slimOriginalTtlMinutes`（原文保留期）此前只在设置页展示、代码从未读取 ⇒ 面板承诺的过期从不发生，原文无限累积；现于**每次加载**执行一次过期清理。
+- **修复**：标记投递失败时的回滚调用 `purgeBlobs({ all: false })` 不传年龄 ⇒ **是空操作**，原文永久留存；现按**精确 id** 回滚，不会牵连其他原文。
+- 台账新增「原文保留期 retention」一行，让被消费的配置可见。[`98f6a06`]
+
+## 0.3.1
+- **修复**：`ctk_retrieve` 成功路径漏 `isError` ⇒ 宿主按 schema 校验工具输出、缺必需属性直接拒调用，**可逆取回完全不可用**（标记已注入、原文已入库，却取不回来）；现补上。[`75ea559`]
+- 台账渲染「输出瘦身 slimmed（块/字节/token/跳过数）」与「可逆原文 blobs」，并把瘦身字节计入 *kept out of context*。
+
+## 0.3.0
+- **新增：出口压缩（`tools/post-execute`）**——门禁守入口，这里处理**已经进来**的输出：同构 JSON **无损**表化重编码、表格保留表头+首尾并按**原始行号**采样中段、grep/search 按文件折叠、日志折叠普通行而**逐字保留诊断行**、代码与键值对**不压**。
+- **双重门**：字节 ≤ 72% 且估算 token 严格下降，压不动就不压。
+- **可逆红线**：有损路径先把原文**逐字节**存好并注入 `[ctk_retrieve id=…]` 标记；新增 `ctk_retrieve` 取回。原文与 FTS5 索引同在**同一个 SQLite 库**（`blobs` 表，BLOB 存取避免重编码）。[`61f162d`]
+
+## 0.2.4
+- **修复**：`read` 分支重犯了 `script_path` 的路径基准错——相对路径按插件进程 cwd 解析，而宿主按**会话工作目录**打开；实测同一份 70000 字节文件绝对路径被拦、相对路径整段进对话。[`8607fb2`]
+- **新增**：把宿主自带取网页工具（`web_fetch` / `read_page`）纳入整页软引导（此前 shell 里的 curl 被拦而同页可从它们走进来）。
+
+## 0.2.3
+- **修复**：相对 `script_path` 解析基准与宿主不一致 ⇒ 读不到文件即当作「无脚本」放行；现统一按会话 cwd 解析。[`5820210`]
+
+## 0.2.2
+- **修复**：门禁匹配从「整行子串 + 两侧分隔符」改为**按命令分隔符分段、只比较各段命令词**——修掉双向错误（漏检非首位调用与带前缀/路径的调用；误拦工具名只是参数的文本）。
+- **修复**：`ops_bash` 用 `script_path` 时载荷在文件里、门禁完全看不见 ⇒ 补读脚本前 256 KB。
+- **修复**：套件自身的句柄泄漏（mock 丢弃 disposer ⇒ SQLite 句柄常开、Windows 删不掉临时目录）。[`da1b0fd`]
+
+## 0.2.1
+- **修复**：`ctk_doctor` 版本号硬编码 `0.1.0` 而包已 0.2.0 ⇒ 改为启动时读自身 `package.json`。
+- **新增**：`release-check.mjs`（发布就绪门）。[`d98835a`]
+
+## 0.2.0
+- **修复**：`guard` 放行时返回 `null` 而非 `undefined` ⇒ 宿主按 `reason !== undefined` 判拒，**每个工具调用都被拒**（run_code 报裸 `Error: null`）。
+- **修复**：门禁只认识自带的 shell 名 ⇒ `ops_bash` 上同样的 curl 可绕过；现覆盖全部 shell 名。
+
+## 0.1.0
+- 首个版本：`ctk_execute` / `ctk_execute_file` / `ctk_batch_execute`（子进程跑码只回结论）、`ctk_index` / `ctk_search`（基于 `node:sqlite` 的本地 FTS5 知识库）、路由门禁、`ctk_stats` / `ctk_doctor` 与实时占用面板。[`87b35fe`]

package/README.md

@@ -1,6 +1,6 @@
 # dsh-context-toolkit
 
-> [简体中文](./README.zh.md) | **English**
+> [简体中文](./README.zh.md) | **English** | [Changelog](./CHANGELOG.md)
 
 A DeepSeek Harness bundle whose single job is **keeping bulk output out of the conversation**.
 
@@ -115,7 +115,7 @@ Tool switches: `tools.execute`, `tools.executeFile`, `tools.batchExecute`, `tool
 The offline suites need no DSH runtime:
 
 ```bash
-node smoke.mjs        # 82 checks: contract, runtime discovery, tools, gate (segment matching, host-aligned path base, script_path and host-fetcher coverage), output slimming and reversible retrieval, resource disposal, knowledge base, observability, version reporting
+node smoke.mjs        # 85 checks: contract, runtime discovery, tools, gate (segment matching, host-aligned path base, script_path and host-fetcher coverage), output slimming with reversible retrieval and retention, resource disposal, knowledge base, observability, version reporting
 node test-client.mjs  # 27 checks: browser-half contract and bilingual dictionary parity
 node test-package.mjs # 20 checks: publish manifest and the zero-dependency constraint
 node release-check.mjs # 30 checks: release readiness (manifest, deps, tool count, gate matching, version, SHA coverage, no hardcoded paths)
@@ -139,6 +139,7 @@ At `tools/post-execute`, the strategy follows the shape:
 **Two gates**: a candidate must be at most 72% of the original bytes *and* have a strictly lower token estimate, or it is dropped — no saving means no claim, and the ledger never overstates.
 
 **Reversibility is the red line**: every lossy path stores the exact original first and injects a `[ctk_retrieve id=…]` marker, read back with `ctk_retrieve`. The originals live in the **same database as the FTS5 index** — one file, two jobs: FTS5 finds sections, blobs restore them.
+- **Retention**: `slimOriginalTtlMinutes` (default 1440 = one day, 0 = forever) is swept once on every load; when a marker cannot be delivered the blob is rolled back by id, so an unrelated original is never taken with it.
 
 ## Known limits
 

package/README.zh.md

@@ -1,6 +1,6 @@
 # 上下文工具箱（dsh-context-toolkit）
 
-> **简体中文** | [English](./README.md)
+> **简体中文** | [English](./README.md) | [更新说明](./CHANGELOG.zh.md)
 
 一个给 DeepSeek Harness 用的插件：**让大块输出不进对话**。
 
@@ -116,7 +116,7 @@ await ctk_search({ queries: ['rate limit', 'auth'] });  // 只取相关片段
 离线套件不依赖 DSH 运行时：
 
 ```bash
-node smoke.mjs        # 82 项：契约、运行时探测、工具、门禁（分段匹配 + 路径基准对齐 + script_path 与取网页工具覆盖）、输出瘦身与可逆取回、资源释放、知识库、观测、版本自述
+node smoke.mjs        # 85 项：契约、运行时探测、工具、门禁（分段匹配 + 路径基准对齐 + script_path 与取网页工具覆盖）、输出瘦身与可逆取回及保留期、资源释放、知识库、观测、版本自述
 node test-client.mjs  # 27 项：浏览器半体契约与双语字典一致性
 node test-package.mjs # 20 项：发布清单与零依赖约束
 node release-check.mjs # 30 项：发布就绪（清单/依赖/工具数/门禁匹配/版本/SHA 覆盖/无硬编码路径）
@@ -140,6 +140,7 @@ node release-check.mjs # 30 项：发布就绪（清单/依赖/工具数/门禁
 **双重门**：候选必须同时满足「字节 ≤ 72%」且「估算 token 严格下降」，否则放弃——压不动就不压，台账绝不虚报节省。
 
 **可逆红线**：任何有损路径先把原文**逐字节**存好并注入 `[ctk_retrieve id=…]` 标记，用 `ctk_retrieve` 取回。原文与 FTS5 索引同在**同一个数据库**（一个文件两个职责：FTS5 找段落，blob 逐字节还原）。
+- **原文保留期**：`slimOriginalTtlMinutes`（默认 1440 = 一天，0 = 永久）由插件在**每次加载时**执行一次清理；标记投递失败时按 id 精确回滚，不会牵连其他原文。
 
 ## 已知边界
 

package/index.js

@@ -284,6 +284,19 @@ export function apply(ctx, rawConfig) {
     log.warn('knowledge base unavailable: ' + String((error && error.message) || error))
   }
 
+  // Retention is enforced here, once per load, because the alternative was a setting
+  // the user could see and the code never read: slimOriginalTtlMinutes was exposed in
+  // the settings page while nothing consumed it, so stored originals accumulated
+  // forever and the promise the panel made was not kept.
+  if (store) {
+    try {
+      const swept = store.purgeBlobsOlderThan(Number(config.slimOriginalTtlMinutes))
+      if (swept.removed > 0) log.info('retention sweep removed ' + swept.removed + ' stored original(s)')
+    } catch (error) {
+      log.warn('retention sweep failed: ' + String((error && error.message) || error))
+    }
+  }
+
   registerRoutingGate(ctx, config, stats)
   registerOutputSlimmer(ctx, config, stats, store, log)
   registerTools(ctx, config, stats, storageDir, log)
@@ -559,7 +572,10 @@ function registerOutputSlimmer(ctx, config, stats, store, log) {
         if (Buffer.byteLength(slimmed, 'utf8') >= outcome.originalBytes) {
           stats.slimSkipped += 1
           rewritten.push(block)
-          if (marker) { try { store.purgeBlobs({ all: false }) } catch { /* blob stays, harmless */ } }
+          // The blob was written but the marker could not be delivered, so nothing will
+          // ever reference it. Removing this specific id is the point: the previous call
+          // passed { all: false } with no age, which purgeBlobs treats as "nothing to do".
+          if (marker) { try { store.purgeBlobs({ ids: [id] }) } catch { /* blob stays, harmless */ } }
           continue
         }
 
@@ -1438,6 +1454,7 @@ function registerObservabilityTools(ctx, config, stats, store, storageDir, log)
           '- 片段 chunks: ' + kb.chunks,
           '- 原文字节 bytes: ' + kb.bytes,
           '- 可逆原文 blobs: ' + (kb.blobs || 0) + '（' + (kb.blobBytes || 0) + ' 字节，可用 ctk_retrieve 逐字节取回）',
+          '- 原文保留期 retention: ' + (Number(config.slimOriginalTtlMinutes) > 0 ? Number(config.slimOriginalTtlMinutes) + ' 分钟（每次加载时清理过期项）' : '永久保留'),
           '- 位置: ' + storageDir,
           '',
           '### 本插件注册的工具',

package/lib/store.js

@@ -165,13 +165,27 @@ function makeApi(db, dbPath, storageDir) {
       return { id: String(id), text: Buffer.from(row.body).toString('utf8'), bytes: Number(row.bytes) || 0, createdAtMs: Number(row.created_at_ms) || 0 }
     },
 
-    /** Drop stored originals older than the given age, or all of them. */
-    purgeBlobs({ olderThanMs = 0, all = false } = {}) {
+    /**
+     * Drop stored originals: specific ids, everything older than an age, or all.
+     *
+     * The three modes answer different questions. A caller whose marker never reached
+     * the model removes the one id it just wrote; a retention policy removes by age; a
+     * reset removes all. Removing by id first means a rollback cannot take an unrelated
+     * original down with it.
+     */
+    purgeBlobs({ ids = [], olderThanMs = 0, all = false } = {}) {
       if (all) {
         const n = db.prepare('SELECT COUNT(*) AS n FROM blobs').get().n
         db.exec('DELETE FROM blobs;')
         return { removed: Number(n) || 0 }
       }
+      const list = (Array.isArray(ids) ? ids : [ids]).map((v) => String(v || '')).filter(Boolean)
+      if (list.length) {
+        const del = db.prepare('DELETE FROM blobs WHERE blob_id = ?')
+        let removed = 0
+        for (const id of list) removed += Number(del.run(id).changes) || 0
+        return { removed }
+      }
       if (!(olderThanMs > 0)) return { removed: 0 }
       const cutoff = Date.now() - olderThanMs
       const n = db.prepare('SELECT COUNT(*) AS n FROM blobs WHERE created_at_ms < ?').get(cutoff).n
@@ -179,6 +193,15 @@ function makeApi(db, dbPath, storageDir) {
       return { removed: Number(n) || 0 }
     },
 
+    /** Drop originals older than a retention in minutes; 0 means keep forever. */
+    purgeBlobsOlderThan(minutes) {
+      const n = Number(minutes)
+      // An explicit 0 is a policy ("keep forever"), not an oversight, so it must not be
+      // read as "expire immediately".
+      if (!(n > 0)) return { removed: 0 }
+      return this.purgeBlobs({ olderThanMs: n * 60000 })
+    },
+
     /** File one document, replacing any earlier version of the same path. */
     indexDocument({ path, title, text, mtimeMs }) {
       const abs = normalizeTarget(path)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-context-toolkit",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "description": "DeepSeek Harness bundle: native ctk_* sandbox tools, a routing gate that keeps bulk output out of the transcript, and a local full-text knowledge base. No third-party runtime dependencies.",
   "repository": {
     "type": "git",
@@ -27,6 +27,8 @@
     "cordis.patch.yml",
     "README.md",
     "README.zh.md",
+    "CHANGELOG.md",
+    "CHANGELOG.zh.md",
     "LICENSE",
     "locale/*.json",
     "icon.svg"