pi-supernova 0.0.15 → 0.2.0

package/CHANGELOG.md

@@ -2,6 +2,37 @@
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-09-04
+
+### Changed
+
+- `read`, `write`, `edit`, and `bash` are the main interface. Source questions and directory selection use `read`; no separate search-tool discovery is needed.
+- Internal adapters remain callable when the host hides their top-level tools. Explicit exclusions, delegated-tool permissions, and session guards still apply.
+- Source selection favors declarations over callers. It reports ambiguous, missing, and incomplete results instead of arbitrary file choices.
+- Bounded ripgrep searches replace whole-repository content loading for source questions. Large source files remain searchable; context and signatures have explicit limits.
+
+### Fixed
+
+- Directory reads include staged entries before commit. Explicit test-directory searches include test files without extra query words.
+
+## [0.1.0] - 2026-09-04
+
+### Fixed
+
+- Each program uses a separate worker and transaction. Late callbacks, idle worker errors, concurrent runs, startup cancellation, and startup deadlines no longer share state.
+- Command deadlines stop descendants. Signal exits report failure. Native commits recover partial replacements; writes reject missing content and edits reject overlapping matches.
+- Unified patches handle zero-context insertions, empty files, final-newline changes, and CRLF. Reads see external changes; evidence search includes staged files.
+- OMP tool calls use the current session’s enabled, permission-aware registry. Discovery supplies complete schemas. Batch options, failure envelopes, scheduling, and call identifiers remain consistent.
+- Output limits cover aggregate batch text, markers, and spill footers. Details remain valid JSON; logs report truncation. DataView ranges and own `__proto__` values survive serialization.
+
+### Changed
+
+- Acorn parses supported function and body forms. Return hints ignore comments and nested callbacks. Invalid collection arguments report errors.
+- The result ledger checks text equality, retains explicit-read pins across aliases, expires source records, and accepts a zero window.
+- Terminal width checks use Unicode graphemes. Expanded cards wrap all bounded result and log text without a preview-line limit.
+- Pi 0.85.0 native adapters respect active-tool changes. Pi has no cross-extension execution API; the README states this limit.
+- Live checks cover Pi 0.85.0 and OMP 18.1.10, including expanded terminal output.
+
 ## [0.0.15] - 2026-09-04
 
 ### Removed

package/README.md

@@ -5,14 +5,22 @@
 [![node](https://img.shields.io/node/v/pi-supernova.svg)](https://nodejs.org)
 [![pi-package](https://img.shields.io/badge/pi--package-extension-7aa2f7)](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/packages.md)
 
-**CodeMode for [Pi](https://pi.dev) and [OMP](https://omp.sh).** One tool, one guest program: progressive discovery, a hard result bottleneck, and Amdahl-aware parallel waves.
+**CodeMode for [Pi](https://pi.dev) and [OMP](https://omp.sh).** One program with `read`, `write`, `edit`, and `bash`. Search, source context, and result limits stay inside these commands.
 
 ```bash
 pi install npm:pi-supernova
 omp install npm:pi-supernova
 ```
 
-Load **before** other tool-owning packages so `registerTool` capture works. Restart the host after install.
+Restart the host after install. OMP uses the current session’s enabled tool registry, independent of extension load order.
+
+Pi 0.85.0 supplies tool schemas but has no cross-extension execution API. Supernova uses native adapters and executors registered through its API instance.
+
+An unrelated Pi extension’s tools are not callable through Supernova. OMP 18.1.10 supports those calls through its session registry.
+
+Both hosts use the same runtime, transactions, and result limits. Hiding a top-level native tool does not remove its internal Supernova adapter.
+
+Use `excludeTools` to disable an internal adapter. Delegated extension tools still require host permission and active status. Replaced or disposed sessions cannot call tools.
 
 ---
 
@@ -20,39 +28,63 @@ Load **before** other tool-owning packages so `registerTool` capture works. Rest
 
 | | Stock multi-tool prompt | **pi-supernova** |
 |--|-------------------------|------------------|
-| Schema tax | Every tool in context | Thin catalog + on-demand `describe` |
+| Schema tax | Every tool in context | Four known commands; optional host schemas on demand |
 | Multi-step | Model stitches turns | One in-process program |
 | Parallel reads | Ad hoc | `callMany` Auto / `parallel()` |
 | Hosts | Separate packages | Same tarball for Pi **and** OMP |
 
-Guest code runs as an `AsyncFunction` in a worker thread (same trust class as host `bash`), not an OS/VM sandbox. Tool calls go through `nova.call` RPC to the host thread; a hard timeout, abort, `process.exit`, or a memory blow-up terminates the worker without touching the harness.
+Guest code runs as an `AsyncFunction` in a new worker for each program. It has the same trust level as host `bash`.
+
+The host parses the bounded source with Acorn before execution. Tool calls use RPC. Deadlines and cancellation cover worker startup, tool discovery, and execution.
+
+A completed worker cannot supply callbacks or globals to another program. Concurrent programs have separate transactions, traces, cancellation signals, and call budgets.
 
 ---
 
 ## Quick example
 
 ```js
-async () => {
-  const hits = await nova.search("read file contents");
-  await nova.describe("read");
+const hit = JSON.parse(await read("src", {about: "validateJwtRefreshToken"}));
+if (hit.status !== "found") return hit;
+return await read(hit.path, {about: "expired tokens"});
+```
 
-  const a = await nova.call("read", { path: "src/index.ts" });
-  const wave = await nova.callMany([
-    { name: "read", args: { path: "a.ts" } },
-    { name: "read", args: { path: "b.ts" } },
-  ]);
+No catalog lookup or separate search command is needed.
 
-  return { preview: a.value.slice(0, 200), mode: wave.mode, n: wave.length };
-}
-```
+| Command | Result |
+|---------|--------|
+| `read(file, offset?, limit?)` | File text; line numbers start at one |
+| `read(files)` | An array of file texts |
+| `read(directory)` | Direct directory entries |
+| `read("symbol or question")` | JSON text with source status, path, line, signature, and context |
+| `read(path, {about: question})` | Relevant file bodies, or source selection inside a directory |
+| `write(path, text)` | Write a file |
+| `edit(path, oldText, newText)` | Post-edit lines, a structural check, and references |
+| `bash(command, options?)` | Bounded command output; failure throws |
+
+### Source selection
+
+`read` reports `found`, `ambiguous`, `not_found`, or `incomplete`.
+Only `found` selects a path. Uncertain results contain at most three candidates.
+
+Identifier declarations take priority over callers and filenames. Natural-language selection uses text matches, not semantic inference. Use a symbol or a narrower directory when needed.
+
+The confidence value is a ranking heuristic, not a measured probability. Duplicate declarations do not produce a confident winner.
+
+Search uses ripgrep without loading every file into the text cache. It can locate declarations beyond the 512 KiB whole-file cache limit.
 
-Globals: `nova`, `parallel`, `pipeline`, `console`, plus shorthand `read` (path or path array), `write`, `edit`, `patch`, `evidence`, `surface`, `snap`, `bash`, and `exec`.
+Each candidate contains up to seven source lines. Signatures and individual lines have a 240-character limit, with explicit truncation markers.
 
-Read discipline that keeps context small: `evidence(question)` across the repo, or `read(path, {about: question})` for one file (full structure, only relevant bodies expanded, ~65% fewer tokens than the file) → `read(path, offset, limit)` only for the lines you will edit.
+File-list and content-search output have separate 2,097,152-character budgets. Incomplete searches report their status instead of selecting a file.
 
-Results never repeat what the model already saw: a run of lines from an earlier result collapses to `⋯ 23 lines same as #12 · host-bridge.js:40–62 ⋯`; changed lines always show; `read(path, a, n)` shows a cited range again. `edit` returns the post-edit lines, a structural check, and who else references a changed declaration; a failing `bash` attaches the source behind the `path:line` it printed. A read→edit→verify loop costs ~15% of the naive token count.
+The measured 10,001-file fixture returned a 199-character result. This is about 50 tokens at four characters per token, not an exact token count.
 
-File search is an in-process port of [fff](https://github.com/dmtrKovalenko/fff): `glob("hostbrdge")` finds `host-bridge.js` (typo-tolerant, frecency- and git-status-ranked), `grep` is smart-case with declaration lines first and a fuzzy fallback. None of it spawns a process.
+In the 23-question source check, 17 questions selected the correct file and six returned candidates. No result selected a wrong file.
+All 15 identifier questions selected the correct file. Natural-language questions can still miss the intended file in their candidates.
+
+Search scans disk on each question. Warm searches can be slower than the previous full-content cache, but do not retain every source file.
+
+Already-seen result lines collapse to references. `read(path, firstLine, lineCount)` shows a range again. `edit` returns changed lines without a verification read.
 
 The returned value is rendered as a compact JS literal (unquoted keys, one item per line only when a container exceeds 120 columns) and capped at `maxReturnChars`. Strings are returned raw. This costs ~43% fewer tokens than pretty JSON. Return small shaped values, not raw file dumps.
 
@@ -69,27 +101,36 @@ Unified Pi/OMP card: one aligned row per call (status · tool · duration · tar
 ╰───────────────────────────────────────────────────────────────╯
 ```
 
-Multi-line commands show their first line plus a hidden-line count. Press Enter for a larger hunk budget, the full returned value, and logs.
+Multi-line commands show their first line and a hidden-line count. Expand the card to show the complete bounded return value and logs.
+
+Expanded text wraps at the terminal width without a preview-line limit. Operation lists and mutation diffs keep their separate 24-row budgets.
 
 ---
 
-## API
+## Optional APIs
 
 | API | Role |
 |-----|------|
 | `nova.search(query, limit?)` | Thin catalog hits |
-| `nova.describe(name)` | Parameter summary on demand |
+| `nova.describe(name)` | Complete JSON input schema on demand; explicit failure when no usable schema is available |
 | `nova.call(name, args)` | Host tool or native adapter |
 | `nova.callMany([{name,args}])` | Auto parallel wave; iterable array with `.mode` / `.results` |
 | `nova.evidence(query, {k?, path?, maxChars?})` | Top-K source spans (path, lines, verbatim text) that answer a question. Zero-token evidence selection after Zero-Mem; ~68% fewer tokens than reading the files |
-| `read(path, {about})` | Whole-file outline with only the relevant bodies expanded; folded bodies show `line … N lines` |
 | `nova.surface(path)` | Structural outline for a source file |
-| `nova.snap(query, searchRoot?)` | Defining file (workspace-relative), line, signature, confidence, and context for a concept; served from the in-process index in well under 1ms |
+| `nova.snap(query, searchRoot?)` | Structured source-selection result; the same engine as a source question passed to `read` |
 | `nova.has(name)` | Whether a catalog or native tool is callable (sync) |
-| `parallel(thunks)` / `pipeline(items, …stages)` | Raw `Promise.all` helpers |
+| `parallel(thunks)` / `pipeline(items, …stages)` | Array-based helpers; pipeline stages must be functions |
 | `nova.speculate(fn)` | Counterfactual branch (rollback / commit) |
 
-Root Snap searches ignore hidden files. Passing a hidden search root includes hidden files beneath that root; Git metadata is always excluded.
+`nova.call` returns an explicit `{ok, value}` envelope. Convenience helpers throw when a host tool reports failure.
+
+`callMany` runs known read-only tools concurrently. Unknown tools and mutating actions run in order, including LSP rename operations.
+
+`nova.describe` preserves required fields, unions, enums, nested objects, and numeric constraints. OMP ArkType and Zod schemas convert to JSON Schema.
+
+Source questions ignore hidden files by default. An explicit hidden directory includes its hidden files. Git metadata stays excluded.
+
+An explicit test directory includes test files without extra query words. Directory reads include staged entries before commit.
 
 ---
 
@@ -112,11 +153,17 @@ Optional `~/.pi/agent/supernova.json` or `~/.omp/agent/supernova.json`
 }
 ```
 
+`maxCallResultChars` bounds text per call. A batch shares one text budget across all items; it does not repeat a joined copy.
+
+Envelope fields and JSON encoding add overhead. Details have a separate 2,000-character JSON budget. Truncation flags cover returns, raw objects, and logs.
+
+An optional `spillDir` retains complete truncated text. Inline spill footers and truncation markers fit within the configured text limit.
+
 `seenWindow` is how many programs back the seen-ledger remembers (set 0 to disable collapsing). `maxHeapMb` caps the guest worker heap (V8 `resourceLimits` on Node) and arms a process-RSS watchdog that terminates a runaway program on both Node and Bun.
 
 Defaults also set `excludeTools` (includes `supernova` and DCE helpers). An empty `"excludeTools": []` **replaces** those defaults, so omit the key unless you mean that.
 
-Slash command `/supernova`: catalog size, captured executors, and session token stats.
+Slash command `/supernova`: callable catalog size, external tools, native adapters, and session token stats.
 
 ---
 
@@ -141,8 +188,8 @@ Pair with DCE last if you use it: `omp install npm:pi-deferred-context-engine`.
 
 | Symptom | Fix |
 |---------|-----|
-| `unknown tool "…"` | Follow the `Did you mean` hint, or `nova.search("")`; for host tools install supernova **first**; restart; `/supernova` |
-| `Rendered line exceeds terminal width` | ≥0.0.1 and restart so `render.js` reloads |
+| `unknown tool "…"` | Check `excludeTools` for native adapters. Enable delegated tools in the host. Restart after package changes. |
+| `Rendered line exceeds terminal width` | Install the current package and restart so the Unicode width code reloads. |
 | `callMany` / not iterable | ≥0.0.1; the return is an array with `.mode` / `.results` |
 | Extension missing on OMP | `omp install npm:pi-supernova` (needs `"omp".extensions`) |
 
@@ -152,9 +199,16 @@ Pair with DCE last if you use it: `omp install npm:pi-deferred-context-engine`.
 
 - Guest JS is **unsandboxed**. Adapter path jails are not a boundary against `import("node:fs")`. The worker only contains hangs, exits, and memory, not intent.
 - Guest error messages carry `(line:col)` on Node; Bun's engine does not expose guest-relative positions.
-- `bash` / mutating tools flush speculative writes (transaction barrier); error rollback cannot undo that.
-- The workspace index refreshes its file list every 10s or on any supernova mutation; a file created by an external process can take up to 10s to appear in `glob`/`snap` (`read` is never stale).
-- Pre-1.0 package: APIs and TUI may still evolve between minor releases.
+
+## Transactions and file freshness
+
+- `bash` and mutating host tools commit pending writes before execution. Later rollback cannot undo those changes or external effects.
+- Native commits stage replacements and backups before installation. A commit failure restores earlier replacements; a recovery failure reports retained backup paths.
+- A nested `nova.speculate` branch cannot call external mutators. Await each branch before returning.
+- Native reads use current disk content unless a staged write replaces it. Evidence search includes new staged files.
+- Workspace file-list updates use filesystem watchers. Without a working watcher, external new files can take 10 seconds to appear in indexed searches.
+
+This is a pre-1.0 package. APIs and the terminal display can change between minor releases.
 
 ## Research and prior art
 
@@ -163,7 +217,7 @@ Supernova's retrieval and result shaping implement published methods. Where a pa
 | Work | What we use it for | Where |
 |------|--------------------|-------|
 | **Zero-Mem: Zero-Token Memory Operations for LLM Agents**, Xiao, Zhu, Zhang, Chen, Hong, Zhuang, Zhang, Chen, Ouyang, Ren, Huang (arXiv:2607.29377) | `evidence(query)`: entity–context graph with co-occurrence weights (eq. 3–4), turn/window/episode hierarchy as line/span/file (eq. 5, 11), query profile and relational/local routing (eq. 6–7), lexical entity alignment and one propagation step (eq. 8–9), personalized PageRank over spans (eq. 10), per-view normalisation and ρ-weighted fusion (eq. 12–13), closure with bridges and neighbours (eq. 14), deterministic calibration (eq. 15). Top-K = 5 follows the paper's Top-5 ≈ Top-10 finding. | `evidence.js` |
-| **Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents**, Zhu, Wu (arXiv:2608.29606) | Every returned unit carries provenance (path, line range, verbatim text); the L0→L1→L2 read discipline (`surface` → `evidence`/`read(path, {about})` → `read(path, offset, limit)`); the citation-lock idea that a model should only cite what it actually opened. | `evidence.js`, `outline.js`, tool guidance |
+| **Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents**, Zhu, Wu (arXiv:2608.29606) | Every returned unit carries provenance (path, line range, verbatim text); the L0→L1→L2 read discipline (`read(query)` → `read(path, {about})` → `read(path, offset, limit)`); the citation-lock idea that a model should only cite what it actually opened. | `evidence.js`, `outline.js`, tool guidance |
 | **Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement**, Yan, Su, et al. (arXiv:2609.01481) | Progressive disclosure (index first, detail on demand) and carrying evidence forward instead of reconstructing it from code. | outline / result shaping |
 | **Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents**, Yang, Jin, Zhao, et al. (arXiv:2609.02042) | Framing: one supernova program is an action chunk (one model decision, many primitive actions, stop at the first failing one). | runtime design |
 | **fff**, Dmitriy Kovalenko, MIT, <https://github.com/dmtrKovalenko/fff> | File search. We reimplemented fff's ranking in plain JavaScript after reading its Rust sources (`crates/fff-core/src/score.rs`, `dbs/frecency.rs`, `path_utils.rs`); the formulas and constants are fff's, the code is ours, and nothing runs out of process. Ported: typo-tolerant fuzzy path matching with boundary/consecutive/case bonuses and smart-case; exact-filename +40% and filename +20% bonuses; frecency boost `base·f/100` with fff's AI-mode decay (3-day half-life, 7-day window) and modification-recency steps (30s/5m/15m/1h/4h); git-modified +15%; directory-distance penalty from the current file (−1 per hop, floor −20); definition-first result hinting; fuzzy fallback on zero literal matches; weak-match cutoff; watcher-driven index refresh. Not ported: fff's SIMD/frizbee matcher (ours is an fzf-style greedy match with backward tightening), LMDB persistence (frecency is per session), and the MCP/Neovim surfaces. | `fuzzy.js`, `repo-index.js`, `host-bridge.js` |

package/bottleneck.js

@@ -4,120 +4,121 @@ import { randomUUID } from "node:crypto";
 import { isString, isObject } from "./decode.js";
 import { truncateChars, formatValue } from "./format.js";
 
-function serializeBounded(value, maxChars, label = "value") {
-  let serialized;
-  try {
-    serialized = isString(value) ? value : JSON.stringify(value);
-  } catch {
-    serialized = String(value);
-  }
-  if (serialized === undefined) serialized = "null";
-  return truncateChars(serialized, maxChars, label);
+function json(value) {
+  try { return JSON.stringify(value) ?? "null"; } catch { return JSON.stringify(String(value)); }
 }
-
-function extractContentText(content) {
-  if (!Array.isArray(content)) return "";
-  return content
-    .filter((part) => part && isObject(part) && part.type === "text" && isString(part.text))
-    .map((part) => part.text)
-    .join("\n");
+function detailsOf(raw) {
+  const details = raw?.details;
+  if (!isString(details)) return details;
+  try { return JSON.parse(details); } catch { return details; }
 }
-
-function extractRawString(raw, maxChars) {
+export function hostResultFailed(raw) {
+  const details = detailsOf(raw);
+  return raw?.isError === true || details?.ok === false || (Number.isInteger(details?.exitCode) && details.exitCode !== 0);
+}
+function extractRawString(raw) {
   if (raw == null) return "";
   if (isString(raw)) return raw;
   if (!isObject(raw)) return String(raw);
-  if (Array.isArray(raw.content)) return extractContentText(raw.content);
+  if (Array.isArray(raw.content)) return raw.content.filter(part => part?.type === "text" && isString(part.text)).map(part => part.text).join("\n");
   if (isString(raw.text)) return raw.text;
-  return serializeBounded(raw, maxChars, "host-result").text;
-}
-
-function summarizeDetails(details, maxChars) {
-  return serializeBounded(details, maxChars, "details").text;
+  return json(raw);
 }
 
-function maybeSpill(cappedText, fullText, config) {
-  if (fullText.length <= (config.maxCallResultChars ?? 65536)) return null;
-  if (!isString(config.spillDir) || config.spillDir.length === 0) return null;
-  try {
-    const dir = config.spillDir;
-    fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
-    const filePath = path.join(dir, `${Date.now()}-${randomUUID().slice(0, 8)}.txt`);
-    fs.writeFileSync(filePath, fullText, { encoding: "utf8", mode: 0o600 });
-    return { pointer: filePath };
-  } catch {
-    return null;
-  }
-}
-
-function batchItems(details, maxChars) {
-  if (!isObject(details) || details.batch !== true) return undefined;
-  if (!Array.isArray(details.items)) return undefined;
-  return details.items.map((item) => truncateChars(item, maxChars, "host-result").text);
+/** Bound JSON before serialization; preserve small scalar fields such as exitCode. */
+function summarizeDetails(value, budget = 2000) {
+  const encoded = json(value);
+  if (encoded.length <= budget) return encoded;
+  const snapshot = JSON.parse(encoded);
+  const fit = (input, limit) => {
+    const serialized = json(input);
+    if (serialized.length <= limit) return input;
+    if (isString(input)) {
+      let low = 0;
+      let high = Math.min(input.length, limit);
+      while (low < high) {
+        const mid = Math.ceil((low + high) / 2);
+        if (json(truncateChars(input, mid).text).length <= limit) low = mid;
+        else high = mid - 1;
+      }
+      return truncateChars(input, low).text;
+    }
+    if (!isObject(input) && !Array.isArray(input)) return null;
+    const out = Array.isArray(input) ? [] : { truncated: true };
+    const entries = Object.entries(input);
+    if (!Array.isArray(input)) entries.sort((a, b) => json(a[1]).length - json(b[1]).length);
+    for (const [key, child] of entries) {
+      const used = json(out).length;
+      const overhead = Array.isArray(out) ? 1 : json(key).length + 2;
+      const available = limit - used - overhead;
+      if (available < 4) break;
+      const bounded = fit(child, available);
+      if (Array.isArray(out)) out.push(bounded);
+      else Object.defineProperty(out, key, { value: bounded, enumerable: true, configurable: true });
+      if (json(out).length > limit) {
+        if (Array.isArray(out)) out.pop();
+        else delete out[key];
+      }
+    }
+    return out;
+  };
+  return json(fit(snapshot, budget));
 }
 
-function spillSuffix(capped, text, config) {
-  const spill = maybeSpill(capped.text, text, config);
-  if (!spill?.pointer) return { spill, value: capped.text };
-  return { spill, value: `${capped.text}\n\n[full output spilled to ${spill.pointer}]` };
+function spill(fullText, config) {
+  if (!isString(config.spillDir) || !config.spillDir) return undefined;
+  fs.mkdirSync(config.spillDir, { recursive: true, mode: 0o700 });
+  const file = path.join(config.spillDir, Date.now() + "-" + randomUUID().slice(0, 8) + ".txt");
+  fs.writeFileSync(file, fullText, { encoding: "utf8", mode: 0o600, flag: "wx" });
+  return file;
 }
 
 export function packageHostResult(raw, config) {
   const maxChars = config.maxCallResultChars ?? 65536;
-  const isError = isObject(raw) && raw.isError === true;
-  const details = isObject(raw) ? raw.details : undefined;
-  const upstreamTruncated = isObject(details) && details.outputTruncated === true;
-  const text = extractRawString(raw, maxChars);
-  const items = batchItems(details, maxChars);
-  // Batch items travel as their own field; keep the details summary small and parseable.
-  const summarizedDetails =
-    details === undefined ? undefined : summarizeDetails(items ? { ...details, items: undefined } : details, 2000);
-
+  const details = detailsOf(raw);
+  const batch = details?.batch === true && Array.isArray(details.items) ? details.items : undefined;
+  const text = batch ? "" : extractRawString(raw);
   const capped = truncateChars(text, maxChars, "host-result");
-  if (!capped.truncated) {
-    return {
-      ok: !isError,
-      value: capped.text,
-      truncated: upstreamTruncated,
-      details: summarizedDetails,
-      items,
-    };
+  let truncated = capped.truncated || details?.outputTruncated === true;
+  const result = { ok: !hostResultFailed(raw), value: capped.text, truncated };
+  if (details !== undefined) result.details = summarizeDetails(batch ? { ...details, items: undefined } : details);
+  if (batch) {
+    let remaining = maxChars;
+    result.items = batch.map((item, index) => {
+      const bounded = truncateChars(item, Math.floor(remaining / (batch.length - index)), "host-result");
+      remaining -= bounded.text.length;
+      truncated ||= bounded.truncated;
+      return bounded.text;
+    });
   }
-
-  const { spill, value } = spillSuffix(capped, text, config);
-
-  return {
-    ok: !isError,
-    value,
-    truncated: true,
-    originalChars: capped.originalChars,
-    spill: spill?.pointer,
-    details: summarizedDetails,
-    items,
-  };
-}
-
-function clipLogs(logs, config) {
-  const logLines = Array.isArray(logs) ? logs : [];
-  const maxLogLines = config.maxLogLines ?? 100;
-  const maxLogLineChars = config.maxLogLineChars ?? 4096;
-  const clippedLogs = logLines.slice(0, maxLogLines).map((line) => {
-    const s = isString(line) ? line : String(line);
-    if (s.length <= maxLogLineChars) return s;
-    return s.slice(0, maxLogLineChars) + "…";
-  });
-  return { clippedLogs, logTruncated: logLines.length > maxLogLines };
+  result.truncated = truncated;
+  if (truncated) {
+    result.originalChars = batch ? batch.reduce((sum, item) => sum + String(item).length, 0) : text.length;
+    if (config.spillDir) {
+      const pointer = spill(batch ? batch.join("\n---\n") : text, config);
+      if (pointer) {
+        result.spill = pointer;
+        if (!batch) {
+          const footer = "\n[full output spilled to " + pointer + "]";
+          result.value = footer.length <= maxChars
+            ? truncateChars(text, maxChars - footer.length, "host-result").text + footer
+            : capped.text;
+        }
+      }
+    }
+  }
+  return result;
 }
 
 export function packageFinalReturn(value, logs, config) {
-  const maxReturn = config.maxReturnChars ?? 32000;
-  const serialized = truncateChars(isString(value) ? value.replace(/\n+$/, "") : formatValue(value), maxReturn, "return");
-  const { clippedLogs, logTruncated } = clipLogs(logs, config);
-  return {
-    returnValue: serialized.truncated ? serialized.text : value,
-    returnText: serialized.text,
-    returnTruncated: serialized.truncated,
-    logs: clippedLogs,
-    logTruncated,
-  };
+  const serialized = truncateChars(isString(value) ? value : formatValue(value), config.maxReturnChars ?? 32000, "return");
+  const maxLines = config.maxLogLines ?? 100;
+  let logTruncated = logs.length > maxLines;
+  const clipped = logs.slice(0, maxLines).map(line => {
+    const result = truncateChars(line, config.maxLogLineChars ?? 4096, "log");
+    logTruncated ||= result.truncated;
+    return result.text;
+  });
+  return { returnValue: serialized.truncated ? serialized.text : value, returnText: serialized.text,
+    returnTruncated: serialized.truncated, logs: clipped, logTruncated };
 }

package/catalog.js

@@ -4,13 +4,13 @@ import { isString, isObject } from "./decode.js";
 const NATIVE_TOOL_DEFINITIONS = [
   {
     name: "read",
-    description: "Read UTF-8 workspace files by path, or resolve a concept query to source. Supports path arrays, offset, and limit.",
+    description: "Read files, directories, or source questions. Directory questions use about. Source selection reports uncertainty instead of guessing.",
     parameters: { type: "object", properties: {
-      path: { type: "string", description: "Workspace-relative file path or concept query" },
+      path: { anyOf: [{ type: "string" }, { type: "array", items: { type: "string" } }], description: "Workspace-relative file or directory, source question, or array of paths" },
       target: { anyOf: [{ type: "string" }, { type: "array" }], description: "File path/query or array of paths" },
       offset: { type: "number", description: "One-based starting line" },
       limit: { type: "number", description: "Maximum lines to return" },
-      about: { type: "string", description: "Question or symbol: returns the whole file as an outline with only the relevant bodies expanded" },
+      about: { type: "string", description: "Question or symbol: expand relevant file bodies, or select source inside a directory" },
     } },
   },
   {
@@ -28,7 +28,7 @@ const NATIVE_TOOL_DEFINITIONS = [
     parameters: { type: "object", properties: { path: { type: "string" }, patch: { type: "string" } }, required: ["patch"] },
   },
   {
-    name: "snap", description: "Resolve a concept query to the most relevant workspace source location.",
+    name: "snap", description: "Select source with found, ambiguous, not_found, or incomplete status. The read command uses the same engine.",
     parameters: { type: "object", properties: {
       query: { type: "string", description: "Source concept to resolve" },
       path: { type: "string", description: "Optional workspace search root; explicitly targeting a hidden directory includes its hidden files, but Git metadata is always excluded" },
@@ -110,6 +110,7 @@ function normalizeTool(tool) {
     description,
     descLower: description.toLowerCase(),
     parameters: tool.parameters,
+    schemaError: tool.schemaError,
     sourcePath: sourcePathOf(tool),
   };
 }
@@ -158,33 +159,6 @@ export function searchCatalog(catalog, query, limit = 12) {
   return scored.slice(0, Math.max(1, limit)).map(({ score: _s, ...hit }) => hit);
 }
 
-function fieldSummary(key, schema, required) {
-  const s = schema && isObject(schema) ? schema : {};
-  // Only signal-bearing keys: `required:false` and empty descriptions cost tokens and say nothing.
-  return {
-    type: s.type || (Array.isArray(s.anyOf) ? "union" : "unknown"),
-    required: required.has(key) || undefined,
-    description: isString(s.description) ? s.description.slice(0, 120) : undefined,
-  };
-}
-
-function schemaSummary(parameters) {
-  if (!parameters || !isObject(parameters)) return { type: "unknown" };
-  const props = parameters.properties;
-  if (!props || !isObject(props)) {
-    return {
-      type: parameters.type || "object",
-      note: "schema present (no enumerable properties)",
-    };
-  }
-  const required = new Set(Array.isArray(parameters.required) ? parameters.required : []);
-  const fields = {};
-  for (const [key, schema] of Object.entries(props)) {
-    fields[key] = fieldSummary(key, schema, required);
-  }
-  return { type: "object", fields };
-}
-
 /** Optimal string alignment distance: insert/delete/substitute/adjacent-transpose cost 1. */
 function editDistance(a, b) {
   const rows = Array.from({ length: a.length + 1 }, (_, i) => [i, ...new Array(b.length).fill(0)]);
@@ -232,12 +206,13 @@ export function describeTool(catalog, name) {
   if (!row) {
     return { ok: false, error: unknownToolMessage(name, catalog.map((t) => t.name)) };
   }
+  if (!isObject(row.parameters)) return { ok: false, name, error: "tool schema unavailable: " + (row.schemaError ?? name) };
   if (!row._described) {
     row._described = {
       ok: true,
       name: row.name,
       description: row.description,
-      parameters: schemaSummary(row.parameters),
+      parameters: row.parameters,
       sourcePath: row.sourcePath,
     };
   }

package/config.js

@@ -8,7 +8,7 @@ const require = createRequire(import.meta.url);
 const DEFAULTS = require("./config.default.json");
 
 const KNOWN_KEYS = new Set(Object.keys(DEFAULTS));
-const NONNEGATIVE_INTEGER_KEYS = new Set(["maxLogLines"]);
+const NONNEGATIVE_INTEGER_KEYS = new Set(["maxLogLines", "seenWindow"]);
 const POSITIVE_INTEGER_KEYS = new Set([
   "timeoutMs",
   "maxCodeChars",
@@ -18,7 +18,6 @@ const POSITIVE_INTEGER_KEYS = new Set([
   "maxLogLineChars",
   "maxSearchResults",
   "maxHeapMb",
-  "seenWindow",
 ]);
 
 const VALIDATORS = {