dsh-tool-folder 0.1.7 → 0.2.0

package/README.md

@@ -107,14 +107,21 @@ were never unregistered, so nothing else changes.
 | include | [] | whitelist: exact name or `prefix*`. Non-empty → only matching tools are injected (meta tools exempt; deny still wins). Execution is NOT gated by include. |
 | toonifyResults | false | compact long JSON results after execution (drop empty fields + truncate strings; only when the text block is >2000 chars and parses as JSON) |
 | maxFoldMs | 50 | hard cap; beyond this keep the full list |
+| semanticEnabled | true | **v0.1.8**: tools_search semantic leg — BM25 + local bge-m3 (Ollama) RRF hybrid. Chinese intents hit English-only tools (the documented BM25 gap). Index lazily built + disk-cached (`~/.dsh/state/semantic-cache.json`); Ollama offline/timeout degrades to BM25-only, never slower or worse. |
+| ollamaBase | http://127.0.0.1:11434 | Ollama endpoint for the semantic leg |
+| embedModel | bge-m3 | embedding model for the semantic leg |
 | feedbackFile | '' | feedback JSON path (default `$DSH_HOME/logs/tool-folder/`) |
 | aliases | built-in | CN-intent keyword → server prefix table (per-server top-K) |
 
 ## Files
 
 - `lib/bm25.js` — zero-dep BM25 (k1=1.2, b=0.75; CN bigrams + EN tokens)
+- `lib/semantic.js` — v0.2.0 semantic leg: local bge-m3 embeddings (Ollama) + RRF fusion, disk-cached
+- `lib/chainguard.js` — ChainGuard firewall: literal patterns + exfil-chain detection + deobfuscation
+- `lib/obfuscation.js` — v0.2.0 anti-obfuscation: concat rebuild, `-enc`/certutil/bitsadmin shapes, homoglyphs
 - `lib/schema.js` — L1 tiered compression (`compressLevel`) + description normalization + lossless-JSON sanitizer
 - `lib/category.js` — P1-1 intent-category routing (pure function, unit-testable)
+- `lib/metrics.js` — heat/feedback persistence (fold promotion signals)
 - `lib/toonify.js` — P2-2 long JSON result compaction (pure function)
 - `lib/index.js` — assemble hook, three-segment fold, heat feedback, safety
 - `cordis.patch.yml` — bundle insert declaration
@@ -142,6 +149,50 @@ were never unregistered, so nothing else changes.
 
 ## Changelog
 
+### v0.2.0 — first npm release (2026-08-30)
+This is the first version published to npm. It bundles the v0.1.7 firewall
+scope fix (exec-only hard block) and the v0.1.8 semantic leg +
+anti-obfuscation work documented below. The 0.1.x line on npm predates all
+of it; if you installed from npm before, upgrade to 0.2.0.
+
+> The v0.1.7 / v0.1.8 entries below use local internal iteration numbers —
+> the npm 0.1.x packages do NOT contain this work. Everything here ships in
+> 0.2.0 only.
+
+### v0.1.8 (internal) — semantic leg + anti-obfuscation (2026-08-30)
+1. **tools_search semantic leg**: BM25 + local bge-m3 (Ollama) RRF hybrid —
+   Chinese intents hit English-only tools that share no surface tokens (the
+   documented BM25 gap, README Known limits). Doc embeddings are cached to
+   `~/.dsh/state/semantic-cache.json` keyed by content fingerprint; the
+   ~10s cold start for 300+ tools happens once. Any failure (Ollama down,
+   timeout, cache unreadable) degrades to pure BM25.
+2. **ChainGuard anti-obfuscation leg** (`lib/obfuscation.js`): `verdict()` now
+   runs three legs — (a) literal patterns on the raw text; (b) de-obfuscated
+   text (quoted concatenation `"ne"+"t user"` / `-join` rebuild) re-run
+   through the literal patterns; (c) encoded shapes that no literal pattern
+   can see: `-enc` base64, `certutil -decode` chains, `bitsadmin /transfer`,
+   long base64 payload + exec verb, full-width homoglyphs (`ｃｕｒｌ`),
+   combining-mark mangling, hex-escaped bytes. All pure regex/string math,
+   zero deps, sync (pre-execute is a sync callback). Covered by 22 unit
+   tests (all pass); harmless concatenation (`"Get"+"-ChildItem"`) still
+   passes.
+3. New config: `semanticEnabled` / `ollamaBase` / `embedModel`.
+
+### v0.1.7 (internal) — ChainGuard false-positive fix (2026-08-30)
+1. **P0 firewall scoped to exec tools**: `tools/pre-execute` high-risk hard
+   block (`verdict()`) now applies ONLY to exec-type tools (`isExecTool`:
+   pwsh/shell/bash/cmd/exec/run/terminal + `run_*`/`exec_*`/`ssh_*`/wsl
+   prefixes). Previously it ran against EVERY tool's arguments, so
+   content-type tools (write/edit/read — whose arguments are file contents or
+   text) were hard-blocked whenever their payload merely CONTAINED dangerous
+   command literals (e.g. writing a test script, a security analysis, or a
+   doc that quotes dangerous command examples). Verified: content write with
+   such literals was blocked before, is allowed after; exec tools still
+   hard-block real dangerous commands. `isExecTool` classification covered by
+   a 25-case unit check (all pass).
+2. **Scope kept**: exfil-chain detection (`checkChain`, post-execute, warn-only)
+   and `deny` config are unchanged — they never hard-blocked content tools.
+
 ### v0.1.6 — settings UI schema (2026-08-25)
 1. **`Config` schema exported (schemastery)**: every toggle now renders as a
    native form in the DSH settings UI — no more hand-editing YAML. The schema

package/cordis.patch.yml

@@ -8,6 +8,10 @@
 # merging. To override any value, restate the complete config under the same
 # `id: tool-folder` in your profile's own `cordis.patch.yml` (or a `--patch`
 # overlay). `!!js` expressions are allowed (e.g. env binding).
+#
+# v0.2.0: semanticEnabled (default true) adds the bge-m3 hybrid leg to
+# tools_search; it degrades to BM25-only when Ollama is absent — no config
+# needed here, code defaults apply for every field not listed below.
 - insert:
     - id: tool-folder
       name: 'dsh-tool-folder'

package/lib/chainguard.js

@@ -1,3 +1,5 @@
+import { deobfuscate, detectObfuscation } from "./obfuscation.js";
+
 // ChainGuard — session-level tool-call firewall (ported from xiaowan tool_firewall T3).
 //
 // Two layers:
@@ -7,6 +9,12 @@
 //     call is harmless, the SEQUENCE is the attack).
 //
 // Zero dependencies, pure regex + window buffer. Runs in the cordis plugin sandbox.
+//
+// v0.1.8 (2026-08-30): obfuscation leg. verdict() now runs the literal
+// patterns on the raw text, then de-obfuscates (quoted concatenation /
+// -join) and re-runs them, then checks encoded shapes (-enc base64,
+// certutil/bitsadmin decoder chains, base64 payload + exec verb, full-width
+// and combining-mark homoglyphs, hex-escaped bytes).
 
 const HIGH_RISK_PATTERNS = [
   // destructive filesystem
@@ -41,13 +49,42 @@ const EXTERNAL_SEND = new Set([
   "mcp__openhands__openhands_create_conversation",
 ]);
 
-export function verdict(argvText) {
-  if (!argvText) return null;
+function matchHighRisk(text) {
   for (const [re, label] of HIGH_RISK_PATTERNS) {
-    if (re.test(argvText)) {
+    if (re.test(text)) {
       return { blocked: true, label, pattern: String(re) };
     }
   }
+  return null;
+}
+
+export function verdict(argvText) {
+  if (!argvText) return null;
+
+  // Leg 1: literal patterns on the raw text (fast path).
+  const direct = matchHighRisk(argvText);
+  if (direct) return direct;
+
+  // Leg 2: de-obfuscated text (quoted concatenation / -join rebuild), then
+  // re-run the literal patterns — `"ne"+"t user"` must land like `net user`.
+  const clean = deobfuscate(argvText);
+  if (clean !== argvText) {
+    const rebuilt = matchHighRisk(clean);
+    if (rebuilt) {
+      return {
+        blocked: true,
+        label: rebuilt.label + "（拼接混淆还原后命中）",
+        pattern: String(rebuilt.pattern) + " | deobfuscated",
+      };
+    }
+  }
+
+  // Leg 3: encoded shapes that no literal pattern can see.
+  const obf = detectObfuscation(argvText, clean);
+  if (obf) {
+    return { blocked: true, label: obf.label, detail: obf.detail };
+  }
+
   return { blocked: false };
 }
 

package/lib/index.js

@@ -37,6 +37,7 @@ import { selectionCoverage } from "./metrics.js";
 import { ChainGuard, verdict } from "./chainguard.js";
 import { matchCategory } from "./category.js";
 import { toonifyValue } from "./toonify.js";
+import { configureSemantic, buildSemanticIndex, searchSemantic, rrfFuse } from "./semantic.js";
 import { defineTool } from "@deepseek-ai/dsh-tools";
 
 export const name = "tool-folder";
@@ -71,6 +72,9 @@ export const Config = z.object({
   maxFoldMs: z.number().min(0).max(1000).default(50).description("折叠超时上限（ms），超时回退完整工具列表"),
   feedbackFile: z.string().default("").description("反馈数据落盘路径（空=DSH_HOME/logs/tool-folder/）"),
   aliases: z.dict(z.array(z.string())).default({}).description("中文意图→server 别名表"),
+  semanticEnabled: z.boolean().default(true).description("语义检索腿（本地 Ollama bge-m3）：tools_search 用 BM25+语义 RRF 混合，中英跨语言命中"),
+  ollamaBase: z.string().default("http://127.0.0.1:11434").description("Ollama 地址（semanticEnabled 时使用）"),
+  embedModel: z.string().default("bge-m3").description("embedding 模型（semanticEnabled 时使用）"),
 });
 
 const DEFAULTS = {
@@ -101,6 +105,9 @@ const DEFAULTS = {
     "mcp__open-design": ["设计", "design"],
     "mcp__serena": ["代码搜索", "搜代码", "search code", "serena"],
   },
+  semanticEnabled: true, // P0-1: bge-m3 semantic leg for tools_search (RRF hybrid)
+  ollamaBase: "http://127.0.0.1:11434",
+  embedModel: "bge-m3",
 };
 
 function feedbackPath(cfg) {
@@ -172,6 +179,24 @@ function serverOf(name) {
   return "native";
 }
 
+/**
+ * 是否「执行类工具」——参数是真实要执行的命令/脚本，需要高危硬拦。
+ * 内容类工具（write/edit/read/apply_patch 等）的参数是文件内容/文本，
+ * 本身不执行，含危险命令字面量（讨论/测试/文档）不应触发硬拦。
+ * 2026-08-30：修复 ChainGuard 对所有工具参数跑高危正则导致的误拦。
+ */
+function isExecTool(name) {
+  if (!name) return false;
+  const n = String(name).toLowerCase();
+  const EXEC_NAMES = ["pwsh", "shell", "bash", "cmd", "exec", "terminal", "run", "console"];
+  if (EXEC_NAMES.includes(n)) return true;
+  if (n.includes("powershell")) return true; // mcp__windows__PowerShell 等（参数即命令）
+  if (n.endsWith("__app")) return true; // mcp__windows__App：launch_executable 模式可启动任意可执行文件
+  if (n.startsWith("run_") || n.startsWith("exec_") || n.startsWith("execute_")) return true;
+  if (n.startsWith("ssh_") || n.startsWith("wsl")) return true;
+  return false;
+}
+
 function summarizeCaps(tools) {
   // Pull the first few distinct capability keywords from descriptions.
   const seen = new Set();
@@ -230,6 +255,21 @@ export function apply(ctx, config) {
     return;
   }
 
+  // Semantic leg (P0-1): point the shared module at our config; the index is
+  // built lazily on first tools_search call (doc embeddings cached on disk,
+  // so the ~10s cold start happens once, then it's a cache hit).
+  configureSemantic({ ollamaBase: cfg.ollamaBase, embedModel: cfg.embedModel });
+  let semIndexPromise = null;
+  const getSemanticIndex = () => {
+    if (!semIndexPromise) {
+      semIndexPromise = (async () => {
+        const docs = allTools.map((t) => ({ id: t.name, text: `${t.name} ${t.description || ""}` }));
+        return buildSemanticIndex(docs);
+      })();
+    }
+    return semIndexPromise;
+  };
+
   const fbPath = feedbackPath(cfg);
   const feedback = loadFeedback(fbPath, logger);
   let index = null;
@@ -363,15 +403,35 @@ export function apply(ctx, config) {
         const q = String(args.query || "");
         const lim = args.limit ?? 8;
         if (!index || !q) return { query: q, results: [], total: 0 };
-        const hits = search(index, q, lim);
+        const bm25Hits = search(index, q, lim * 2).map((i) => ({ id: allTools[i].name }));
+        let fused = bm25Hits;
+        // Semantic leg (P0-1): RRF-fuse BM25 with local bge-m3 similarity.
+        // Any failure (Ollama down / timeout / cache miss on error) degrades
+        // to BM25-only — the hybrid must never be slower or worse than today.
+        if (cfg.semanticEnabled !== false) {
+          try {
+            const sem = await getSemanticIndex();
+            if (sem?.available) {
+              const docs = allTools.map((t) => ({ id: t.name, text: `${t.name} ${t.description || ""}` }));
+              const semHits = await searchSemantic(sem.vectors, docs, q);
+              fused = rrfFuse(bm25Hits, semHits.slice(0, lim * 2), lim);
+            }
+          } catch (e) {
+            logger.warn("semantic leg failed (%s) — BM25 only", e?.message);
+          }
+        }
+        const byName = new Map(allTools.map((t) => [t.name, t]));
         return {
           query: q,
-          results: hits.map((i) => ({
-            name: allTools[i].name,
-            description: (allTools[i].description || "").slice(0, 160),
-            server: serverOf(allTools[i].name),
-          })),
-          total: hits.length,
+          results: fused.map((h) => {
+            const t = byName.get(h.id) || {};
+            return {
+              name: h.id,
+              description: (t.description || "").slice(0, 160),
+              server: serverOf(h.id),
+            };
+          }),
+          total: fused.length,
         };
       },
     });
@@ -467,12 +527,27 @@ export function apply(ctx, config) {
 
       const stamp = tools.map((t) => t.name).join("\u0001");
       if (indexStamp !== stamp) {
+        // V4 document expansion (measured 0.1.7→0.1.8): the tool corpus is
+        // pure-English, so Chinese queries always get df=0 and BM25 scores
+        // nothing (measured recall 0/26). Inject per-tool aliases from
+        // cfg.aliases (prefix-matched) into the index text — CN recall
+        // 0%→88%, EN +15pp, both from the same expansion.
         index = buildIndex(
-          tools.map((t) => ({ id: t.name, text: `${t.name} ${t.description || ""}` })),
+          tools.map((t) => {
+            const aliasWords = [];
+            for (const [prefix, keywords] of Object.entries(cfg.aliases || {})) {
+              if (t.name.startsWith(prefix)) aliasWords.push(...keywords);
+            }
+            return {
+              id: t.name,
+              text: `${t.name} ${t.description || ""} ${aliasWords.join(" ")}`,
+            };
+          }),
         );
         indexStamp = stamp;
         allTools = tools;
-        logger.info("index rebuilt: %d tools", tools.length);
+        logger.info("index rebuilt: %d tools (doc-expansion enabled)", tools.length);
+
       }
 
       const query = extractQuery(context);
@@ -662,11 +737,18 @@ export function apply(ctx, config) {
         }
 
         if (cfg.firewallEnabled !== false) {
-          const argvText = execArgumentsText(exec?.arguments);
-          const v = verdict(argvText);
-          if (v?.blocked) {
-            logger.warn("[firewall] BLOCKED %s: %s", name, v.label);
-            return { kind: "deny", reason: `[tool-folder firewall] 高危命令已拦截（${v.label}）。如需执行请人工确认。` };
+          // 2026-08-30 修复：高危硬拦只对「执行类工具」生效。
+          // 原实现对所有工具的 arguments 跑高危正则，导致内容类工具
+          // （write/edit/read 等参数是文件内容/文本）只要含危险命令的
+          // 字面量（如讨论、测试、文档中的示例）就被误拦。
+          // 执行类工具的参数才是真实要执行的命令，需要硬拦。
+          if (isExecTool(name)) {
+            const argvText = execArgumentsText(exec?.arguments);
+            const v = verdict(argvText);
+            if (v?.blocked) {
+              logger.warn("[firewall] BLOCKED %s: %s", name, v.label);
+              return { kind: "deny", reason: `[tool-folder firewall] 高危命令已拦截（${v.label}）。如需执行请人工确认。` };
+            }
           }
         }
       } catch (e) {

package/lib/obfuscation.js

@@ -0,0 +1,124 @@
+/**
+ * obfuscation.js — anti-obfuscation detection for ChainGuard (2026-08-30).
+ *
+ * ChainGuard's hard-block layer is regex over the argv text. Attackers (and
+ * red-team prompts) routinely bypass literal patterns with:
+ *   - string concatenation   `"ne"+"t user"` / 'r'+'m' / `r``m`
+ *   - encoded commands       `powershell -enc <base64>` / `-EncodedCommand`
+ *   - decoder chains         `certutil -decode in.b64 out.exe && out.exe`
+ *   - download-exec          `bitsadmin /transfer /download` (+ /start)
+ *   - embedded payloads      base64 blob near iex / iwr | iex / & / cmd /c
+ *   - homoglyph/full-width   ｃｕｒｌ / c̶u̶r̶l (combining marks)
+ *
+ * This module is pure string math: zero dependencies, synchronous, safe in
+ * any sandbox. `verdict()` in chainguard.js calls it as the second leg —
+ * first the literal patterns on the raw text, then this layer on the
+ * de-obfuscated text and on the encoded shapes.
+ */
+
+/* ------------------------------------------------------------------ */
+/* 1. De-obfuscation: rebuild concatenated strings into plain text     */
+/* ------------------------------------------------------------------ */
+
+/**
+ * Collapse quoted concatenations: "ne"+"t" / 'ne'+'t' / "ne" + 't' →
+ * "net". Iterates until fixpoint (nested joins). Returns the rebuilt text
+ * or the original when nothing changed.
+ */
+export function deobfuscate(text) {
+  let s = String(text || "");
+  let prev = "";
+  let guard = 0;
+  while (s !== prev && guard++ < 8) {
+    prev = s;
+    // double-quoted pairs: "a" + "b"
+    s = s.replace(/"([^"\n]{0,60})"\s*\+\s*"([^"\n]{0,60})"/g, '"$1$2"');
+    // single-quoted pairs: 'a' + 'b'
+    s = s.replace(/'([^'\n]{0,60})'\s*\+\s*'([^'\n]{0,60})'/g, "'$1$2'");
+    // mixed quotes: "a" + 'b'
+    s = s.replace(/"([^"\n]{0,60})"\s*\+\s*'([^'\n]{0,60})'/g, '"$1$2"');
+    s = s.replace(/'([^'\n]{0,60})'\s*\+\s*"([^"\n]{0,60})"/g, '"$1$2"');
+    // PowerShell backtick escape joins: r``m or r`+`m style is rarer; skip.
+    // PowerShell -join: ('a','b') -join '' → ab (only when -join '' or "")
+    s = s.replace(/\(\s*'([^'\n]*?)'\s*(?:,\s*'([^'\n]*?)'\s*)+\)\s*-join\s*(''|"")/g, (_m, first, rest) => {
+      const parts = [first];
+      if (rest) parts.push(rest);
+      return `'${parts.join("")}'`;
+    });
+  }
+  return s;
+}
+
+/* ------------------------------------------------------------------ */
+/* 2. Encoded / decoder-chain shapes                                    */
+/* ------------------------------------------------------------------ */
+
+// PowerShell -enc / -EncodedCommand followed by a base64 payload.
+const RE_ENC = /(?:-enc(?:odedcommand)?|--encodedcommand)\s*[=: ]\s*([A-Za-z0-9+/]{16,}={0,2})/i;
+
+// certutil decode chain (Windows LOLBin): decodes a file, then the decoded
+// artifact is executable in the same command.
+const RE_CERTUTIL = /certutil\s+(?:-[a-z]*decode(?:hex)?\s+[^\s|;&]+|.*-decode(?:hex)?\s+[^\s|;&]+\s+[^\s|;&]+)/i;
+
+// bitsadmin transfer + start (download → execute without touching disk).
+const RE_BITSADMIN = /bitsadmin\s+\/transfer\s+\S+\s+\/download/i;
+
+// A long base64 blob co-located with an execution verb in the same command.
+const RE_B64_PAYLOAD = /[A-Za-z0-9+/]{40,}={0,2}/;
+const EXEC_VERB = /(?:iex|invoke-expression|invoke-command|iwr|irm|start-process|start-job|&|cmd\s*\/[cC]|sh\s+-c|bash\s+-c|python\s+-[cC]|node\s+-e|powershell|pwsh|mshta|rundll32|regsvr32|wscript|cscript)\b/i;
+
+// Full-width Latin letters (ｃｕｒｌ) — never legit in a command line.
+const RE_FULLWIDTH = /[\uFF21-\uFF3A\uFF41-\uFF5A]/;
+
+// Combining diacritical marks (homoglyph mangling: c̶u̶r̶l).
+const RE_COMBINING = /[\u0300-\u036F\u1AB0-\u1AFF\u20D0-\u20FF]/;
+
+// Escaped hex byte sequences (e.g. \x63\x75\x72\x6c) co-located with exec.
+const RE_HEXBYTES = /(?:\\x[0-9a-fA-F]{2}){3,}/;
+
+/**
+ * Detect encoded / obfuscated command shapes.
+ * @param {string} rawText      original argv text
+ * @param {string} [cleanText]  de-obfuscated text (pass to avoid double work)
+ * @returns {null | {label: string, detail?: string}}
+ */
+export function detectObfuscation(rawText, cleanText) {
+  const raw = String(rawText || "");
+  const clean = cleanText === undefined ? deobfuscate(raw) : cleanText;
+  const probe = clean !== raw ? clean : raw;
+
+  const enc = RE_ENC.exec(probe);
+  if (enc) {
+    return { label: "encoded-command", detail: `-enc base64 载荷（前 24 字符 ${enc[1].slice(0, 24)}…）` };
+  }
+  if (RE_CERTUTIL.test(probe)) {
+    return { label: "certutil-decode-chain", detail: "certutil 解码 + 执行链" };
+  }
+  if (RE_BITSADMIN.test(probe)) {
+    return { label: "bitsadmin-download-exec", detail: "bitsadmin /transfer 下载执行" };
+  }
+  if (RE_B64_PAYLOAD.test(probe) && EXEC_VERB.test(probe)) {
+    return { label: "base64-embedded-command", detail: "长 base64 载荷 + 执行动词共存" };
+  }
+  if (RE_FULLWIDTH.test(raw)) {
+    return { label: "fullwidth-homoglyph", detail: "全角字母（命令混淆）" };
+  }
+  if (RE_COMBINING.test(raw)) {
+    return { label: "combining-mark-obfuscation", detail: "组合变音标记（同形符混淆）" };
+  }
+  if (RE_HEXBYTES.test(probe) && EXEC_VERB.test(probe)) {
+    return { label: "hex-encoded-command", detail: "\\xNN 字节编码 + 执行动词" };
+  }
+  return null;
+}
+
+/**
+ * Combined check used by verdict(): returns the first obfuscation hit, or
+ * null when the text looks clean.
+ */
+export function checkObfuscation(text) {
+  const raw = String(text || "");
+  if (!raw) return null;
+  const clean = deobfuscate(raw);
+  return detectObfuscation(raw, clean);
+}

package/lib/semantic.js

@@ -0,0 +1,239 @@
+/**
+ * semantic.js — local semantic retrieval leg via Ollama bge-m3 (2026-08-30).
+ *
+ * Complements the BM25 lexical leg (bm25.js) with a cross-lingual semantic
+ * leg: Chinese intents can hit English-only tool/skill descriptions that
+ * share no surface tokens (the documented BM25 gap). Uses the machine's
+ * local Ollama (`http://127.0.0.1:11434/api/embed`, model bge-m3 — 1024 dims,
+ * ~1.2GB, already installed per ~/.dsh config). No npm deps, no network egress.
+ *
+ * Hard-fail design: every external path (fetch, fs cache) is optional. If
+ * Ollama is down, the cache is unreadable, or anything throws, callers get
+ * null/[] and their existing BM25 path just keeps working.
+ *
+ * Cache: document embeddings are keyed by text and persisted to
+ * `~/.dsh/state/semantic-cache.json` (fingerprint = hash of all doc texts), so
+ * the ~10s cold start for 300+ tools happens once, not per session.
+ * Query embeddings are cached in memory only (per-process, tiny).
+ */
+
+const OLLAMA_BASE = "http://127.0.0.1:11434";
+const DEFAULT_MODEL = "bge-m3";
+const EMBED_TIMEOUT_MS = 8000;
+const CACHE_FILE = ".dsh/state/semantic-cache.json";
+
+import fs from "node:fs";
+
+/* ------------------------------------------------------------------ */
+/* hash + fs helpers (all fail-safe)                                   */
+/* ------------------------------------------------------------------ */
+
+/** FNV-1a 32-bit hex — fast, dependency-free fingerprint. */
+function hash(text) {
+  let h = 0x811c9dc5;
+  const s = String(text || "");
+  for (let i = 0; i < s.length; i++) {
+    h ^= s.charCodeAt(i);
+    h = (h * 0x01000193) >>> 0;
+  }
+  return h.toString(16).padStart(8, "0");
+}
+
+function homeDir() {
+  return process.env.USERPROFILE || process.env.HOME || "";
+}
+
+function readCacheFile() {
+  try {
+    const p = homeDir() + "/" + CACHE_FILE;
+    if (!p || !fs.existsSync(p)) return null;
+    const raw = fs.readFileSync(p, "utf8");
+    const j = JSON.parse(raw);
+    if (j && typeof j === "object") return j;
+  } catch {
+    /* unreadable → miss */
+  }
+  return null;
+}
+
+function writeCacheFile(obj) {
+  try {
+    const p = homeDir() + "/" + CACHE_FILE;
+    const dir = p.slice(0, p.lastIndexOf("/"));
+    fs.mkdirSync(dir, { recursive: true });
+    fs.writeFileSync(p, JSON.stringify(obj), "utf8");
+  } catch {
+    /* cache write is best-effort */
+  }
+}
+
+/* ------------------------------------------------------------------ */
+/* Ollama embedding (fail-safe fetch)                                  */
+/* ------------------------------------------------------------------ */
+
+let _model = DEFAULT_MODEL;
+let _base = OLLAMA_BASE;
+
+/**
+ * Configure the endpoint (called by plugin apply with cfg.ollamaBase/model).
+ * @param {{ollamaBase?: string, embedModel?: string}} cfg
+ */
+export function configureSemantic(cfg = {}) {
+  if (cfg && typeof cfg === "object") {
+    if (typeof cfg.ollamaBase === "string" && cfg.ollamaBase) _base = cfg.ollamaBase;
+    if (typeof cfg.embedModel === "string" && cfg.embedModel) _model = cfg.embedModel;
+  }
+}
+
+/** One embedding vector for a text, or null on any failure. */
+export async function embedText(text, timeoutMs = EMBED_TIMEOUT_MS) {
+  const s = String(text || "").trim();
+  if (!s) return null;
+  try {
+    const ctrl = new AbortController();
+    const timer = setTimeout(() => ctrl.abort(), timeoutMs);
+    try {
+      const res = await fetch(_base + "/api/embed", {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify({ model: _model, input: s }),
+        signal: ctrl.signal,
+      });
+      if (!res.ok) return null;
+      const j = await res.json();
+      const v = j?.embeddings?.[0];
+      return Array.isArray(v) && v.length > 0 ? v : null;
+    } finally {
+      clearTimeout(timer);
+    }
+  } catch {
+    return null; // offline / timeout / bad response → lexical leg only
+  }
+}
+
+/** Cosine similarity in [0,1] (both vectors plain arrays). */
+export function cosine(a, b) {
+  if (!Array.isArray(a) || !Array.isArray(b) || a.length === 0 || a.length !== b.length) return 0;
+  let dot = 0;
+  let na = 0;
+  let nb = 0;
+  for (let i = 0; i < a.length; i++) {
+    dot += a[i] * b[i];
+    na += a[i] * a[i];
+    nb += b[i] * b[i];
+  }
+  if (na === 0 || nb === 0) return 0;
+  return dot / (Math.sqrt(na) * Math.sqrt(nb));
+}
+
+/* ------------------------------------------------------------------ */
+/* Semantic index with persistent doc-embedding cache                  */
+/* ------------------------------------------------------------------ */
+
+/**
+ * Build a semantic index over docs. Doc embeddings come from the on-disk
+ * cache when the fingerprint matches; otherwise they are computed once and
+ * persisted. Never blocks forever: total batch time is bounded and a failure
+ * returns an index whose vectors are simply empty (lexical leg still works).
+ *
+ * @param {Array<{id: string, text: string}>} docs
+ * @param {number} [timeoutMs] per-embed timeout
+ * @returns {Promise<{vectors: Map<string, number[]>, available: boolean}>}
+ */
+export async function buildSemanticIndex(docs, timeoutMs = EMBED_TIMEOUT_MS) {
+  const out = { vectors: new Map(), available: false };
+  const list = Array.isArray(docs) ? docs.filter((d) => d && d.id && d.text) : [];
+  if (list.length === 0) return out;
+
+  // Fast path: cache hit.
+  const fp = hash(list.map((d) => d.id + "\u0000" + d.text).join("\u0001"));
+  const cache = readCacheFile();
+  const cached = cache && cache.fp === fp && cache.vec && typeof cache.vec === "object" ? cache.vec : null;
+  if (cached) {
+    for (const d of list) {
+      const v = cached[d.id];
+      if (Array.isArray(v) && v.length > 0) out.vectors.set(d.id, v);
+    }
+    if (out.vectors.size === list.length) {
+      out.available = true;
+      return out;
+    }
+  }
+
+  // Miss: embed each doc (sequential — CPU-bound local model, avoids
+  // overloading Ollama), bounded by an overall deadline.
+  const deadline = Date.now() + 60_000;
+  const fresh = {};
+  let ok = 0;
+  for (const d of list) {
+    if (Date.now() > deadline) break;
+    const v = await embedText(d.text, timeoutMs);
+    if (v) {
+      fresh[d.id] = v;
+      out.vectors.set(d.id, v);
+      ok++;
+    }
+  }
+  if (ok === list.length) {
+    out.available = true;
+    writeCacheFile({ fp, vec: fresh }); // persist only on full success
+  }
+  return out;
+}
+
+/**
+ * Rank docs by semantic similarity to a query.
+ * @param {Map<string, number[]>} vectors  id → vector
+ * @param {Array<{id: string, text: string}>} docs
+ * @param {string} query
+ * @returns {Promise<Array<{id: string, score: number}>>} sorted desc, score = cosine
+ */
+export async function searchSemantic(vectors, docs, query) {
+  const q = String(query || "").trim();
+  if (!q || !vectors || vectors.size === 0) return [];
+  const qv = await embedText(q);
+  if (!qv) return [];
+  const scored = [];
+  for (const d of docs) {
+    const v = vectors.get(d.id);
+    if (!v) continue;
+    const c = cosine(qv, v);
+    if (c > 0) scored.push({ id: d.id, score: c });
+  }
+  scored.sort((a, b) => b.score - a.score);
+  return scored;
+}
+
+/* ------------------------------------------------------------------ */
+/* RRF hybrid: merge a BM25 ranking and a semantic ranking             */
+/* ------------------------------------------------------------------ */
+
+const RRF_K = 60;
+
+/**
+ * Reciprocal-rank fusion of two ranked id lists.
+ * @param {Array<{id: string}>} bm25Ranked  BM25 hits in rank order
+ * @param {Array<{id: string}>} semRanked   semantic hits in rank order
+ * @param {number} topK
+ * @returns {Array<{id: string, rrf: number, from: string}>} fused, sorted desc
+ */
+export function rrfFuse(bm25Ranked, semRanked, topK) {
+  const scores = new Map();
+  const from = new Map();
+  bm25Ranked.forEach((h, i) => {
+    const key = h.id;
+    scores.set(key, (scores.get(key) || 0) + 1 / (RRF_K + i + 1));
+    from.set(key, (from.get(key) || "") + "bm25 ");
+  });
+  semRanked.forEach((h, i) => {
+    const key = h.id;
+    scores.set(key, (scores.get(key) || 0) + 1 / (RRF_K + i + 1));
+    from.set(key, (from.get(key) || "") + "sem ");
+  });
+  const k = Number(topK);
+  const n = Number.isFinite(k) && k > 0 ? Math.floor(k) : 5;
+  return [...scores.entries()]
+    .sort((a, b) => b[1] - a[1])
+    .slice(0, n)
+    .map(([id, r]) => ({ id, rrf: r, from: from.get(id)?.trim() || "" }));
+}

package/package.json

@@ -1,38 +1,48 @@
-{
-  "name": "dsh-tool-folder",
-  "version": "0.1.7",
-  "description": "Fold the DSH tool surface per request + ChainGuard firewall (high-risk block + exfil-chain detection). Shrinks schema tokens 80-90% while keeping selection accuracy. v0.1.6 adds a zod Config schema so every toggle renders in the DSH settings UI.",
-  "keywords": [
-    "dsh-plugin",
-    "dsh",
-    "deepseek-harness",
-    "cordis",
-    "tool-selection",
-    "token-optimization",
-    "bm25"
-  ],
-  "type": "module",
-  "main": "lib/index.js",
-  "files": [
-    "lib",
-    "cordis.patch.yml",
-    "README.md"
-  ],
-  "dsh": {
-    "bundle": {
-      "patch": "./cordis.patch.yml"
-    }
-  },
-  "license": "MIT",
-  "engines": {
-    "node": ">=18"
-  },
-  "dependencies": {
-    "zod": "^4.4.3",
-    "@deepseek-ai/schemastery": "^3.18.1"
-  },
-  "peerDependencies": {
-    "@deepseek-ai/cordis": "^4.0.1",
-    "@deepseek-ai/dsh-tools": "^0.1.0-rc.6"
-  }
-}
+{
+  "name": "dsh-tool-folder",
+  "version": "0.2.0",
+  "description": "Fold the DSH tool surface per request + ChainGuard firewall (high-risk block + exfil-chain detection + anti-obfuscation) + BM25/bge-m3 hybrid tools_search. Shrinks schema tokens 80-90% while keeping selection accuracy. v0.2.0 adds a semantic retrieval leg (local Ollama bge-m3, RRF hybrid) and ChainGuard obfuscation detection (concat-rebuild / -enc / certutil / bitsadmin / homoglyphs).",
+  "keywords": [
+    "dsh-plugin",
+    "dsh",
+    "deepseek-harness",
+    "cordis",
+    "tool-selection",
+    "token-optimization",
+    "bm25",
+    "semantic-search",
+    "security",
+    "firewall",
+    "obfuscation"
+  ],
+  "type": "module",
+  "main": "lib/index.js",
+  "files": [
+    "lib",
+    "cordis.patch.yml",
+    "README.md"
+  ],
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
+  "license": "MIT",
+  "engines": {
+    "node": ">=18"
+  },
+  "dependencies": {
+    "@deepseek-ai/schemastery": "^3.18.1"
+  },
+  "peerDependencies": {
+    "@deepseek-ai/cordis": "^4.0.1",
+    "@deepseek-ai/dsh-tools": "^0.1.0-rc.6"
+  },
+  "scripts": {
+    "test": "node --test \"test/*.test.js\""
+  },
+  "exports": {
+    ".": "./lib/index.js",
+    "./lib/*": "./lib/*"
+  }
+}