dsh-agent-speedup 0.2.0 → 0.2.1

package/README.md

@@ -19,13 +19,14 @@ Within that 22%, three different problems exist, and each capability targets one
 
 | Capability | Problem it solves | Measured |
 |---|---|---|
-| **Cache** | Same read-only call repeated | wall **−13.5%** on the release-candidate run; **−46%** in the earlier calibration on the same task shape |
-| **Prefetch** | Files the model will ask for right after a `glob` | 12/12 prefetch stored, 11 subsequent hits, 28 s of execution avoided across 6 runs |
-| **Shell concurrency** | Multiple readonly shell commands serialized | 5 pwsh calls in 19.4 s → concurrent judgment in 8 ms |
+| **Cache** | Same read-only call repeated | wall **−13.5%** (T4) and **−10.5%** (T5), release build; an earlier calibration on the same task shape measured **−46%** |
+| **Prefetch** | Files the model will ask for right after a `glob` | 12/12 prefetch calls completed, results served from cache on the model's next reads (ledger: 14 hits, 28.1 s of instrumented execution avoided across 6 runs) |
+| **Shell concurrency** | Multiple readonly shell commands serialized | 4 pwsh calls: **7,103 ms serialized → 317 ms concurrent (22×)**, identical task and model |
 
 And two things it deliberately does not fix: on local millisecond-scale tools the plugin is
 invisible (tools were 0.1–9.4% of wall in our runs), and DeepSeek Harness already parallel-batches
-concurrency-safe tools — 6 reads completed within 21 ms in our baseline, with no plugin installed.
+concurrency-safe tools — in our baseline 7 tool calls (6 reads + 1 shell) were dispatched within
+19 ms with no plugin installed, and 5 adjacent tool results landed within 100 ms of each other.
 
 Full method, sample sizes, and the findings that argue against this plugin:
 [docs/BENCHMARK.md](docs/BENCHMARK.md).
@@ -70,8 +71,9 @@ handled identically.
 ### Shell concurrency
 
 dsh's `bash` and `pwsh` tools declare no `isConcurrencySafe`, so every shell call runs exclusively —
-even five readonly `Get-ChildItem` calls take turns (measured: 19.4 s for what should be one
-parallel window). This capability attaches a conservative classifier that marks a command
+four readonly `Get-*` calls that should share one parallel window instead took 7.1 s
+(measured: per-call gaps of 2.2–2.5 s). With this capability enabled, the same four calls completed
+in 317 ms. The capability attaches a conservative classifier that marks a command
 concurrency-safe only when every segment matches a readonly whitelist and no write pattern appears.
 
 **This is off by default.** A classifier mistake means two shell commands with side effects run at
@@ -108,14 +110,32 @@ All keys with zod defaults; a patch override replaces the whole object.
 | `shellTools` | `["bash","pwsh"]` | Shell tools to patch |
 | `debugLogFile` | unset | Write scheduling events here |
 
+## Safety
+
+Read-only tools only. `bash`, `pwsh`, `write` and `edit` are passed through untouched and are never
+cached or prefetched. Error results, including guard denials, are not remembered. Cache and
+in-flight prefetches are cleared when the plugin unloads (in-flight prefetch aborts included).
+
+Two known boundaries, found in independent review:
+
+- **Working-directory assumption.** Cache keys and stat checks resolve relative paths against the
+  process working directory, while dsh's `read` resolves against the agent's working directory. In
+  headless mode (one process per workspace) these coincide. If an agent `cd`s elsewhere, entries
+  whose file cannot be stat'd are simply not cached (fail-safe), but same-named files across
+  different agent working directories are not disambiguated.
+- **Cache hits short-circuit the waterfall.** A hit returns without calling `next()`, which also
+  skips `tools/execute` listeners registered by plugins loaded after this one. If you run other
+  execute-stage plugins, load them before this one.
+
 ## 中文
 
 给 DeepSeek Harness 用的工具执行效率套件，三个能力各自对应一类可实测的问题：
 
-- **缓存**：只读工具重复调用直接命中（重复读场景实测免掉 28 秒真实执行 / 6 轮）
-- **预取**：`glob` 之后预读文件，模型开口时结果已在（实测 12/12 预取全部完成并命中）
-- **只读 shell 并发**：dsh 原生会把 shell 命令串行（实测 5 条毫秒级命令跑了 19.4 秒），
-  本插件给只读命令打上并发标记，让它们进入 dsh 原生并行批（默认关闭，需显式开启）
+- **缓存**：只读工具重复调用直接命中（发布候选回归实测免掉 28 秒真实执行 / 6 轮）
+- **预取**：`glob` 之后预读文件，模型开口时结果已在（实测 12/12 预取完成、结果全部从缓存供给）
+- **只读 shell 并发**：dsh 原生会把 shell 命令串行（实测 4 条毫秒级命令串行 7.1 秒，
+  开启后 317 毫秒完成，22 倍），本插件给只读命令打上并发标记，
+  让它们进入 dsh 原生并行批（默认关闭，需显式开启）
 
 前提要清楚：模型延迟占关键路径的 87%，工具占 22%。本地毫秒级工具上这个插件无感；
 收益集中在"慢工具 + 重复调用"场景。数字、方法和反面结果在

package/docs/BENCHMARK.md

@@ -66,9 +66,14 @@ Mechanism-level confirmation, independent of wall time:
 - T5: prefetch launched 4 of 4 files and the model's subsequent four reads all hit
   (`speculative: true`), with zero prefetch failures.
 
-A later run of the same suite after the plugin was refactored for release reproduced the mechanism
-(per-call tool time −49.7% on T4, −79.4% on T5) but a smaller wall-time delta (−7.4% and −17.5%),
-which is consistent with model-side variance at this sample size.
+Two follow-up regression runs, with the plugin versions they belong to (kept separate because the
+numbers are not interchangeable):
+
+- **0.1.x plugin** (−49.7% per-call tool time on T4, −79.4% on T5; wall −7.4% and −17.5%): measured
+  before the suite rewrite. Wall deltas at this sample size were dominated by model-side variance.
+- **0.2.0 release candidate** (this package): T4 wall −14.7%, T5 wall −10.5%, ledger showing
+  14 cache hits and 28.1 s of instrumented execution avoided across 6 runs. Confirms the mechanism
+  survived the rewrite; the wall deltas differ from 0.1.x within model-side variance.
 
 ## Two results that argue against the plugin
 

package/lib/index.js

@@ -74,15 +74,21 @@ function normPath(value) {
   }
 }
 
-/** 稳定键：工具名 + 规范化参数（键序固定） */
+/** 稳定键：工具名 + 规范化参数（键序固定）。
+ *  只有明确的路径类参数才做路径归一——grep 的 pattern 是正则语义，
+ *  对它做 resolve/relative 会把 `a\db` 与 `a/db` 归一成同一键（假命中）。 */
 function keyOf(tool, args) {
   const normalized = {};
   for (const k of Object.keys(args ?? {}).sort()) {
-    normalized[k] = typeof args[k] === "string" ? normPath(args[k]) : args[k];
+    const v = args[k];
+    normalized[k] = typeof v === "string" && PATH_ARGS.has(k) ? normPath(v) : v;
   }
   return `${tool}|${JSON.stringify(normalized)}`;
 }
 
+/** 已知承载文件路径的参数名；白名单外的字符串一律原样入键 */
+const PATH_ARGS = new Set(["file_path", "path", "dir", "directory", "cwd", "file", "target"]);
+
 /** 文件 stat 指纹；失败返回 undefined（无法校验） */
 function statFingerprint(path) {
   try {
@@ -162,12 +168,21 @@ export function apply(ctx, config) {
     if (result?.isError) { stats.miss++; return result; }
     const filePath = exec.arguments?.file_path;
     const execMs = Date.now() - t0;
+    // 带路径的条目必须拿到 stat 指纹才可缓存：指纹取不到说明该路径不在本进程 cwd
+    // 视野内（agent 工作目录与进程 cwd 不一致），缓存它有跨目录内容污染风险。
+    // glob/grep 等无 file_path 的条目标记 volatile，由写意图与 shell 调用触发整体失效。
+    let fp = null;
+    if (typeof filePath === "string") {
+      fp = statFingerprint(normPath(filePath));
+      if (fp === undefined) { stats.miss++; return result; }
+    }
     cache.set(key, {
       result,
       spec: prefetching.has(key),
       execMs,
+      volatile: typeof filePath !== "string", // glob/grep 等无指纹条目：写意图/shell 调用后整体失效
       ...(typeof filePath === "string" ? { path: normPath(filePath) } : {}),
-      ...(typeof filePath === "string" ? { fingerprint: statFingerprint(normPath(filePath)) } : {}),
+      ...(fp !== null ? { fingerprint: fp } : {}),
     });
     stats.miss++;
     log({ kind: "cache-miss", tool: exec.name, key, speculative: prefetching.has(key), execMs });
@@ -175,6 +190,8 @@ export function apply(ctx, config) {
   });
 
   // ================= 能力 2：推测预取 =================
+  /** 在途预取的控制器：卸载时统一 abort，防止 teardown 后仍写缓存/日志 */
+  const inflightControllers = new Set();
   const prefetch = (paths, tool) => {
     const targets = [];
     for (const p of paths) {
@@ -182,34 +199,42 @@ export function apply(ctx, config) {
       const key = keyOf(tool, { file_path: p });
       if (cache.has(key) || prefetching.has(key)) continue;
       prefetching.add(key);
-      targets.push({ path: p, key });
+      targets.push({ path: p, key, startedAt: 0 });
     }
     for (const t of targets) {
       stats.speculativeLaunched++;
       log({ kind: "spec-start", tool, path: t.path });
-      const settle = (fn) => { prefetching.delete(t.key); fn(); };
+      const controller = new AbortController();
+      inflightControllers.add(controller);
+      t.startedAt = Date.now();
+      const settle = (fn) => { prefetching.delete(t.key); inflightControllers.delete(controller); fn(); };
       try {
         ctx.tools
           .execute({
             name: tool,
             arguments: { file_path: t.path },
             callId: `spec-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
-            signal: new AbortController().signal,
+            signal: controller.signal,
           })
           .then((result) => settle(() => {
+            if (controller.signal.aborted) return;
             if (result?.isError) { stats.speculativeFailed++; log({ kind: "spec-fail", path: t.path, reason: "error-result" }); return; }
             if (cache.has(t.key)) { log({ kind: "spec-done", path: t.path, stored: false, note: "real-call-won" }); return; }
-            cache.set(t.key, { result, spec: true, execMs: 0, path: t.path, fingerprint: statFingerprint(t.path) });
-            log({ kind: "spec-done", path: t.path, stored: true });
+            cache.set(t.key, { result, spec: true, execMs: Date.now() - t.startedAt, path: t.path, fingerprint: statFingerprint(t.path) });
+            log({ kind: "spec-done", path: t.path, stored: true, execMs: Date.now() - t.startedAt });
           }))
           .catch((e) => settle(() => { stats.speculativeFailed++; log({ kind: "spec-fail", path: t.path, error: String(e).slice(0, 160) }); }));
       } catch (e) {
         settle(() => { stats.speculativeFailed++; log({ kind: "spec-fail", path: t.path, error: String(e).slice(0, 160) }); });
       }
+      t.startedAt = Date.now();
     }
   };
 
   ctx.on("tools/post-execute", async (exec, result, next) => {
+    // shell 调用后清空 volatile 条目：bash/pwsh 可能写了文件且不发 fs 事件，
+    // glob/grep 的无指纹条目此时已不可信（R1a Issue 1 的 shell 侧兜底）
+    if (cfg.shellTools.includes(exec?.name)) dropVolatile("shell-call");
     if (cfg.speculate && cfg.speculateOn.has(exec?.name) && result != null) {
       const paths = pathsFromResult(result, cfg.speculateRootPrefix);
       if (paths.length) prefetch(paths, SPECULATE_READ_OF[exec.name] ?? "read");
@@ -218,21 +243,38 @@ export function apply(ctx, config) {
   });
 
   // ================= 能力 2.5：写意图即时失效 =================
-  // 只监听写意图。fs/observed 会在每次成功的 read 上触发，若在此失效会把刚写入的
-  // 缓存条目立刻删掉。bash/pwsh 的写入不发事件，由 stat 指纹校验兜底。
+  // ⚠ 契约：fs/write-intent 与 fs/edit-intent 是 waterfall（不是 emit），监听器签名
+  //   (target, exec, next)，返回值就是写入意图。必须 return next()，否则：
+  //   - 默认顺序下（fs-observation-policy 先注册且不调 next）本监听器永远不执行；
+  //   - 顺序翻转时返回 undefined 会禁用宿主 fs-observation-policy 的 CAS 写入保护。
+  // 两条失效规则：
+  //   a) 条目路径与写意图匹配 → 精确删除；
+  //   b) volatile 条目（glob/grep 等无 file_path 指纹的）→ 任何写意图都清空——
+  //      其正确性前提是"目录内容未变"，无法用单文件 stat 校验。
+  // bash/pwsh 的写入不发事件，由 stat 校验 + shell 调用后清空 volatile 兜底。
+  // （fs/observed 故意不监听：它与 stat 指纹完全冗余，且随每次读取触发。）
+  const dropVolatile = (reason) => {
+    let dropped = 0;
+    for (const [k, entry] of cache) {
+      if (entry.volatile) { cache.delete(k); dropped++; }
+    }
+    if (dropped) log({ kind: "cache-volatile-drop", reason, dropped });
+  };
   const invalidateByTarget = (target) => {
     const key = target?.targetKey ?? target?.displayPath;
-    if (!key) return;
-    const normalized = normPath(String(key));
-    for (const [k, entry] of cache) {
-      if (entry.path !== undefined && normPath(entry.path) === normalized) {
-        cache.delete(k);
-        log({ kind: "cache-invalidated", key: k });
+    if (key) {
+      const normalized = normPath(String(key));
+      for (const [k, entry] of cache) {
+        if (entry.path !== undefined && normPath(entry.path) === normalized) {
+          cache.delete(k);
+          log({ kind: "cache-invalidated", key: k });
+        }
       }
     }
+    dropVolatile("write-intent");
   };
-  ctx.on("fs/write-intent", (target) => invalidateByTarget(target));
-  ctx.on("fs/edit-intent", (target) => invalidateByTarget(target));
+  ctx.on("fs/write-intent", (target, exec, next) => { invalidateByTarget(target); return next(); });
+  ctx.on("fs/edit-intent", (target, exec, next) => { invalidateByTarget(target); return next(); });
 
   // ================= 能力 3：只读 shell 并发 =================
   const shellPatched = new Set();
@@ -272,6 +314,8 @@ export function apply(ctx, config) {
   const bootTimers = [100, 500, 1500].map((ms) => setTimeout(patchShellConcurrency, ms));
   ctx.effect(() => () => {
     for (const t of bootTimers) clearTimeout(t);
+    for (const c of inflightControllers) c.abort(new Error("plugin unloaded"));
+    inflightControllers.clear();
     cache.clear();
     prefetching.clear();
   }, "agent-speedup teardown");

package/lib/util/classify-command.js

@@ -25,6 +25,8 @@ const READONLY_PREFIXES = [
 const WRITE_PATTERNS = [
   /\s>\s*\S/,            // 输出重定向
   /\s>>\s*\S/,           // 追加重定向
+  /\s\d>\s*\S/,          // fd 重定向到文件（2>err.log）
+  /\bdelete\b|\b-delete\b|-exec\b|-execdir\b|\bfd\s+(-x|--exec)\b/i,  // find -delete/-exec、fd -x
   /\b(rm|rmdir|del|erase|mv|move|cp|copy|install|uninstall)\b/i,
   /\b(mkdir|md|touch|new-item|set-content|add-content|out-file)\b/i,
   /\b(sed\s+-i|tee|truncate|chmod|chown|chgrp)\b/i,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-agent-speedup",
-  "version": "0.2.0",
+  "version": "0.2.1",
   "description": "Tool-execution efficiency suite for DeepSeek Harness: read-only result cache, speculative prefetch, and concurrent readonly shell commands. Honest scope: helps when tools are slow and calls repeat; model latency dominates otherwise.",
   "type": "module",
   "main": "lib/index.js",
@@ -41,8 +41,12 @@
     "@deepseek-ai/dsh-tools": ">=0.2.0-rc.0"
   },
   "peerDependenciesMeta": {
-    "@deepseek-ai/cordis": { "optional": true },
-    "@deepseek-ai/dsh-tools": { "optional": true }
+    "@deepseek-ai/cordis": {
+      "optional": true
+    },
+    "@deepseek-ai/dsh-tools": {
+      "optional": true
+    }
   },
   "scripts": {
     "test": "node --test test/"