token-goat 2.9.4 → 2.9.5

package/README.md

@@ -11,7 +11,7 @@ permalink: /
 
 ***Give the model what it needs, not everything you have.***
 
-**85%** smaller reads · **97.4%** image compression · **180+** filter & interception rules · **94–99%** skill overhead cut · compaction memory · **prompt injection** guard · **3.7 GB** never reached the model · **1.1 Gt** tokens saved
+**85%** smaller reads · **49%** of first-read bytes withheld · **97.4%** image compression · **180+** filter & interception rules · **94–99%** skill overhead cut · compaction memory · **prompt injection** guard · **3.7 GB** never reached the model · **1.1 Gt** tokens saved
 
 **Reduces AI token use/costs by 40–90%, and improves its focus. Fully automated, always online.**
 
@@ -70,7 +70,8 @@ The fastest way to reduce AI token costs is fixing these five, not writing short
 | 3.3 MB screenshot lands in model context | 84 KB compressed copy, 97.4% smaller |
 | Agent re-reads files from earlier in the session | "Already read this" reminder with narrow slice suggestion |
 | Read tool asks for lines the session was already given | Answered with a pointer at the copy already delivered instead of the file, when the text this read would return matches the text already served for it whole line for whole line. Proof rather than a read count, so it also covers a file inside the recent-read protection window; a changed file, a wider range, and `reread_deny = false` all pass through |
-| Same unchanged file read again through the shell (`cat`, `head`, `tail`, `sed -n '1,40p'`) | Byte-identical repeat replaced with a one-line pointer instead of the whole body — 4,031 bytes to 189 on this project. Only fires when the two runs match exactly, so a changed file is untouched; full text stays available via `token-goat bash-output <id>` |
+| Same unchanged file read again through the shell (`cat`, `head`, `tail`, `sed -n '1,40p'`) | Byte-identical repeat replaced with a one-line pointer instead of the whole body: 4,031 bytes to 189 on this project. A later read wholly inside an earlier one collapses the same way |
+| A shell read of a file overlaps an earlier one without sitting inside it (`sed -n '100,140p'` after `sed -n '120,160p'`) | Only the already-shown stretch is withheld, replaced in place by one line naming its line numbers and the command that returns it; every line not shown before arrives untouched. 56% of the second read on a real pair over this project's README. Line numbers come from the command's own ranges, so a `tail`, whose start the command does not state, is left alone. Full text stays available via `token-goat bash-output <id>` |
 | Part of a file re-read under a different command (`head -40 F` then `sed -n '1,30p' F`) | Repeat replaced with a pointer when every line coming back was already served for that same file this session. Matched on the text, on whole-line boundaries, so a read that adds any new lines is left whole; the record is dropped when the file is edited or the conversation is compacted. A file read with the Read tool counts as served too, limited to the lines that read actually handed over, and a read that came back truncated counts for nothing |
 | Read tool asks for a range that overlaps one already delivered | Only the overlapping stretch is withheld, replaced by one line naming the line numbers and the recall command; the new lines come through with their own numbers and spacing, byte for byte. A stretch is kept whenever dropping it would save less than the line replacing it costs, and a result carrying anything the redactor would strip, a truncated result, or a file changed on disk all pass through whole. Set `elide_served_lines = false` to disable |
 | Shell command run with colour on (`git`, `npm`, `npx`, `token-goat`) | Terminal escape sequences removed, leaving byte-for-byte the same output with colour off: 13.2% smaller on a real `git diff` here, and about a quarter of the bytes on the calls it fires for. Nothing is summarized or withheld, so there is no recall pointer; it runs on failed commands too, and only when the escapes are worth a rewrite |
@@ -114,7 +115,7 @@ The fastest way to reduce AI token costs is fixing these five, not writing short
 | Index-only files (lockfiles, source maps, bundles) read on every session | Pre-Read suppression for read-only files (package-lock.json, *.map, dist/), skipped unless explicitly edited |
 | Large markdown file read in full (README.md, CHANGELOG.md, CLAUDE.md ≥8 KB) | Heading tree intercepted instead — H1–H3 with `#2`/`#3` disambiguation; `token-goat section` shortcuts listed for well-known files; post-edit injects a re-read suggestion rather than the full file |
 | Subagent reads a ≥30 KB markdown file whole, on its first look at it | Off by default. Set `subagent_markdown_first_read_deny = true` under `[hints]` to block that read and answer with the heading tree instead. Only fires in a subagent, only on a first, un-ranged read of a `.md`/`.mdx`/`.markdown` file with at least three headings; a read that already asks for a line range, and every main-session read, pass through untouched |
-| A source file is read whole for the first time, with no earlier copy to compare it against | Off by default. Set `fold_code_bodies = true` under `[hints]` to keep each function's declaration and its first eight lines, replacing the rest with one line naming the symbol, the exact range removed, and the command that returns it. Comment blocks of twelve lines or more fold to their first two on the same terms. Imports, signatures, type declarations, classes and interfaces are never folded, and a file whose index no longer matches what is on disk keeps its bodies. Measured on this repository's own source: 229 of 256 files fold, and 43% of what a read of them delivers is removed. A read that already names an offset or a limit, and any file holding something that looks like a secret, pass through untouched |
+| A source file is read whole for the first time, with no earlier copy to compare it against | Keeps each function's declaration and its first eight lines, replacing the rest with one line naming the symbol, the exact range removed, and the command that returns it. Set `fold_code_bodies = false` under `[hints]` to turn it off. Comment blocks of twelve lines or more fold to their first two on the same terms. Imports, signatures, type declarations, classes and interfaces are never folded, and a file whose index no longer matches what is on disk keeps its bodies. Measured on this repository's own source: 229 of 256 files fold, and 43% of what a read of them delivers is removed. A read that already names an offset or a limit, and any file holding something that looks like a secret, pass through untouched |
 | PDF opened via Read | Full read denied; PDF shows page count and outline (`token-goat pdf-extract` pulls the actual text, optionally paged/sliced, when the outline isn't enough) |
 | Excel/PowerPoint/Word file (.xlsx/.pptx/.docx) opened via Read | Full read denied; redirects to the matching narrow-slice command family (`xlsx-sheets`/`xlsx-head`/`xlsx-range`/`xlsx-query`, `pptx-outline`/`pptx-slide`/`pptx-notes`/`pptx-text`, `docx-outline`/`docx-text`) instead of extracting the whole document as text |
 | Other Office binary (.odt, .ods, .ott, .odp) opened via Read | Full read denied; redirects to `pandoc` for text extraction (no dedicated reader for these formats yet) |
@@ -185,10 +186,12 @@ Returns relevance-ranked, distance-scored hits straight from the notes, the same
 
 ## Token savings, measured
 
-Numbers below come from synthetic-fixture benchmarks in the test suite. Each row points at the source file where the measurement is reproduced.
+Numbers below come from synthetic-fixture benchmarks in the test suite, except the first two rows, which are measured on a corpus of real sessions and say so. Each row points at the source file where the measurement is reproduced.
 
 | Source | Improvement | Measured impact | Where |
 |--------|-------------|-----------------|-------|
+| Large-document outline | A read naming no `offset` or `limit` on a markdown document over 8 KB with six or more headings returns the lead-in and heading tree, plus the `token-goat section` command that reads any section verbatim | Measured on 13,870 real reads across 5,104 sessions: withholds 40.4% of all bytes the Read tool delivers. A 104,890 B document returns 1,541 B with its opening paragraph intact. Sections sum to the document, so a reader who pulls every one back pays about 3% over reading it whole | `src/hooks_read.ts` (`foldMarkdownOutline`) |
+| Large-source skeleton | A read naming no `offset` or `limit` on a source file over 12 KB with eight or more declarations returns the preamble and one line per declaration, each withheld run naming the `token-goat read "file::Symbol"` or ranged `Read` that returns it | Measured on the same corpus: withholds up to 8.6% of all bytes the Read tool delivers, a ceiling rather than a booked total, since the shipping gates are narrower than the measurement's. The skeleton is 10.5% of a `.ts` file. Median symbol body is 4.1% of its file, so a reader breaks even at roughly 22 bodies pulled back out of an average 30.8 | `src/hooks_read.ts` (`foldSourceSkeleton`) |
 | Image shrink | Every still is encoded both ways and the smaller file wins, rather than one codec being assumed | 2560x1440 screenshot-shaped PNG: 2,782,963 -> 259,475 bytes (91% smaller) | `src/image_shrink.ts` (codec selection) |
 | Repomap output | `--compact` trims the top-symbols list to 10 (vs 30) and drops the recent-files section and per-symbol locations | Denser overview for the same byte budget | `src/baseline.ts` (`buildProjectMap`, `token-goat map --compact`) |
 | DB reindex | Batched single transaction + composite indexes on `(file_id, kind)` | 100 files / 10K rows: 84 s → 1 s (~80× faster) | `src/parser.ts`, `src/db.ts` (index migration) |

package/dist/{token-goat-chunk-GEXHIIKM.mjs → token-goat-chunk-2FVSKFZU.mjs}

@@ -7,11 +7,11 @@ import {
   expandGlobs,
   leftoverIntegrations,
   run
-} from "./token-goat-chunk-6MU6FHE3.mjs";
-import "./token-goat-chunk-US57I7GN.mjs";
-import "./token-goat-chunk-FSC44QOH.mjs";
-import "./token-goat-chunk-CBO45DYZ.mjs";
-import "./token-goat-chunk-3ZR4NLB3.mjs";
+} from "./token-goat-chunk-5T2K7DEE.mjs";
+import "./token-goat-chunk-4P6GTCMM.mjs";
+import "./token-goat-chunk-JO5JX72D.mjs";
+import "./token-goat-chunk-U4FTM2SB.mjs";
+import "./token-goat-chunk-ZOKNDG6V.mjs";
 import "./token-goat-chunk-EEIDFMEM.mjs";
 import "./token-goat-chunk-A37V4PBF.mjs";
 export {

package/dist/{token-goat-chunk-US57I7GN.mjs → token-goat-chunk-4P6GTCMM.mjs}

@@ -1,6 +1,7 @@
 import { createRequire as __cjsRequire } from 'node:module';
 const require = __cjsRequire(import.meta.url);
 import {
+  DEFAULT_QUERY_LIMIT,
   IMPORT_RE,
   ImageDecodeError,
   MAX_OVER_FETCH,
@@ -25,6 +26,7 @@ import {
   extractPdfText,
   fenceUntrusted,
   fenceUntrustedContent,
+  fenceUntrustedFileContent,
   formatCsvProfile,
   formatCsvTable,
   getEmbeddingCoverage,
@@ -68,7 +70,7 @@ import {
   walkProject,
   yamlLineClosesQuote,
   yamlOpenQuoteAfter
-} from "./token-goat-chunk-CBO45DYZ.mjs";
+} from "./token-goat-chunk-U4FTM2SB.mjs";
 import {
   Database,
   PER_FILE_COUNTERFACTUAL_CEILING,
@@ -122,7 +124,7 @@ import {
   unsupportedLanguageName,
   windowsCmdQuoteArg,
   withExtension
-} from "./token-goat-chunk-3ZR4NLB3.mjs";
+} from "./token-goat-chunk-ZOKNDG6V.mjs";
 import {
   registerReset
 } from "./token-goat-chunk-EEIDFMEM.mjs";
@@ -458,6 +460,69 @@ import * as path6 from "node:path";
 import { execFileSync, spawnSync as spawnSync2 } from "node:child_process";
 import { randomUUID } from "node:crypto";
 
+// src/ref_blindness.ts
+init_define_import_meta_env();
+var REF_INDEXED_LANGUAGES = /* @__PURE__ */ new Set([
+  "typescript",
+  "javascript",
+  "python",
+  "go",
+  "rust",
+  "java",
+  "c",
+  "cpp",
+  "ruby"
+]);
+function isRefIndexedLanguage(language) {
+  return REF_INDEXED_LANGUAGES.has(language);
+}
+function isRefIndexedFile(filePath) {
+  return isRefIndexedLanguage(detectLanguage(filePath));
+}
+var LANGUAGE_LABELS = /* @__PURE__ */ new Map([
+  ["csharp", "C#"],
+  ["cpp", "C++"],
+  ["c", "C"],
+  ["php", "PHP"],
+  ["powershell", "PowerShell"],
+  ["sql", "SQL"],
+  ["graphql", "GraphQL"],
+  ["proto", "Protocol Buffers"],
+  ["html", "HTML"],
+  ["css", "CSS"],
+  ["toml", "TOML"],
+  ["json", "JSON"],
+  ["yaml", "YAML"],
+  ["ini", "INI"],
+  ["env_file", "env file"],
+  ["ipynb", "Jupyter notebook"],
+  ["salesforce_metadata", "Salesforce metadata"],
+  ["salesforce_markup", "Salesforce markup"],
+  ["unknown", "this file type"]
+]);
+function languageLabel(language) {
+  return LANGUAGE_LABELS.get(language) ?? language;
+}
+function refBlindLanguageNotice(symbolName, language, displayPath) {
+  return `Cannot determine references for '${symbolName}': ${languageLabel(language)} call sites are not indexed (${displayPath}). This is a gap in token-goat's index, not evidence the symbol is unreferenced. Search the source directly instead, e.g. \`rg -n -w ${symbolName}\`.`;
+}
+var REF_BLIND_KIND_REASON = "the index records value-position references only (calls, `new`, macro invocations), never type annotations, so a type declaration has no ref rows however widely it is used";
+var REF_BLIND_DEF_PROBE_LIMIT = 50;
+function kindWithArticle(kind) {
+  return `${/^[aeiou]/.test(kind) ? "an" : "a"} ${kind}`;
+}
+function refBlindKindClause(symbolName, kinds) {
+  const first = kinds[0];
+  if (kinds.length === 1 && first !== void 0) return `'${symbolName}' is ${kindWithArticle(first)}`;
+  return `every definition of '${symbolName}' is a type declaration (${kinds.map((k) => `'${k}'`).join(", ")})`;
+}
+function refBlindKindNotice(symbolName, kinds) {
+  return `Cannot determine references for '${symbolName}': ${refBlindKindClause(symbolName, kinds)}, and ${REF_BLIND_KIND_REASON}. This is a gap in token-goat's index, not evidence the symbol is unreferenced. Search the source directly instead, e.g. \`rg -n -w ${symbolName}\`.`;
+}
+function refBlindKindPartialNote(symbolName, kinds, blindCount, totalCount) {
+  return `Note: ${blindCount} of ${totalCount} definitions of '${symbolName}' (${kinds.map((k) => `'${k}'`).join(", ")}) are not covered by this result -- ${REF_BLIND_KIND_REASON}. The empty result above speaks only for the other ${totalCount - blindCount}; for the rest, search the source directly, e.g. \`rg -n -w ${symbolName}\`.`;
+}
+
 // src/read_commands.ts
 init_define_import_meta_env();
 import * as fs4 from "node:fs";
@@ -5435,6 +5500,15 @@ function guardJsonRows(items) {
   if (!cfg.overflow_guard.enabled) return { items: [...items], truncated: false, totalCount: items.length };
   return capJsonRows(items, cfg.overflow_guard.max_tokens);
 }
+function changedDiffBaselineBytes(cwd, ref, files) {
+  if (files.length === 0) return 0;
+  try {
+    const result = runGit(["diff", ref, "--unified=0", "--", ...files], { cwd });
+    if (result.exitCode === 0) return Buffer.byteLength(result.stdout, "utf8");
+  } catch {
+  }
+  return 0;
+}
 function sumFileSizes(filePaths) {
   let total = 0;
   for (const fp of new Set(filePaths)) {
@@ -5822,9 +5896,9 @@ function truncationFooter(shown, limit, total, plural, flag) {
 
 token-goat: ${notice}`;
 }
-function refsTotal(clientFiltered, filteredTotal, shown, countExact, preScanCount) {
+function refsTotal(clientFiltered, filteredTotal, shown, countExact, preScanCount, scanLimit) {
   if (!clientFiltered) return { count: countExact(), exact: true };
-  return { count: filteredTotal ?? shown, exact: preScanCount < REFS_TOP_SCAN_LIMIT };
+  return { count: filteredTotal ?? shown, exact: preScanCount < scanLimit };
 }
 function truncationNotice(shown, limit, total, plural, flag) {
   if (shown < limit) return null;
@@ -6037,9 +6111,11 @@ function resolveSymbolSpec(spec, forceRefresh, projectRoot) {
     });
   }
   if (methodName !== void 0 && candidates.length > 1) {
+    const containerBaseName = file.slice(Math.max(file.lastIndexOf("/"), file.lastIndexOf("\\")) + 1);
     const containers = querySymbols({
       name: symBase,
       limit: 50,
+      ...containerBaseName !== "" ? { fileBaseName: containerBaseName } : {},
       ...projectRoot !== void 0 ? { rootDir: projectRoot } : {}
     });
     const symBaseLower = symBase.toLowerCase();
@@ -6355,7 +6431,9 @@ function renderRefsTargets(targets, opts, {
     if (rootDir !== void 0) queryOpts.rootDir = rootDir;
     const scanned = queryRefs(queryOpts);
     const preScanCount = scanned.length;
+    const scanLimit = queryOpts.limit ?? DEFAULT_QUERY_LIMIT;
     let results = applyTypedRefsTier(symbol, file, scanned);
+    const typedFilterDropped = results.length < scanned.length;
     let suppressed = 0;
     if (opts.excludeTests === true) {
       const f = applyExcludeTestsFilter(results);
@@ -6365,9 +6443,10 @@ function renderRefsTargets(targets, opts, {
     const preGrepCount = results.length;
     const matchesGrep = refGrepFilter(opts.grep);
     if (matchesGrep !== void 0) results = results.filter(matchesGrep);
+    const clientFiltered = opts.excludeTests === true || matchesGrep !== void 0 || typedFilterDropped;
     let filteredTotal;
-    if (opts.excludeTests === true || matchesGrep !== void 0) filteredTotal = results.length;
-    if ((opts.excludeTests === true || matchesGrep !== void 0) && opts.top === void 0) {
+    if (clientFiltered) filteredTotal = results.length;
+    if (clientFiltered && opts.top === void 0) {
       results = results.slice(0, opts.limit ?? 100);
     }
     if (results.length > 0) anyFound = true;
@@ -6379,7 +6458,7 @@ function renderRefsTargets(targets, opts, {
         jsonOut[key] = withHidden(topFilesJsonPayload(results, opts.top));
       } else {
         const capped = guardJsonRows(results);
-        const trueTotal = opts.excludeTests === true || matchesGrep !== void 0 ? filteredTotal ?? results.length : countRefs(queryOpts);
+        const trueTotal = clientFiltered ? filteredTotal ?? results.length : countRefs(queryOpts);
         jsonOut[key] = withHidden({ items: refsJsonItems(capped.items, opts.context ?? 0), truncated: capped.truncated || trueTotal > results.length, totalCount: trueTotal });
       }
       continue;
@@ -6406,7 +6485,7 @@ function renderRefsTargets(targets, opts, {
       const notice = truncationNotice(
         results.length,
         opts.limit ?? 100,
-        () => refsTotal(opts.excludeTests === true || matchesGrep !== void 0, filteredTotal, results.length, () => countRefs(queryOpts), preScanCount),
+        () => refsTotal(clientFiltered, filteredTotal, results.length, () => countRefs(queryOpts), preScanCount, scanLimit),
         "references",
         "--limit"
       );
@@ -6482,7 +6561,9 @@ function runRefsSingle(opts) {
   if (rootDir !== void 0) queryOpts.rootDir = rootDir;
   const scanned = queryRefs(queryOpts);
   const preScanCount = scanned.length;
+  const scanLimit = queryOpts.limit ?? DEFAULT_QUERY_LIMIT;
   let results = applyTypedRefsTier(symName, defFileHint, scanned);
+  const typedFilterDropped = results.length < scanned.length;
   let suppressed = 0;
   if (opts.excludeTests === true) {
     const f = applyExcludeTestsFilter(results);
@@ -6492,9 +6573,10 @@ function runRefsSingle(opts) {
   const preGrepCount = results.length;
   const matchesGrep = refGrepFilter(opts.grep);
   if (matchesGrep !== void 0) results = results.filter(matchesGrep);
+  const clientFiltered = opts.excludeTests === true || matchesGrep !== void 0 || typedFilterDropped;
   let filteredTotal;
-  if (opts.excludeTests === true || matchesGrep !== void 0) filteredTotal = results.length;
-  if ((opts.excludeTests === true || matchesGrep !== void 0) && opts.top === void 0) {
+  if (clientFiltered) filteredTotal = results.length;
+  if (clientFiltered && opts.top === void 0) {
     results = results.slice(0, opts.limit ?? 100);
   }
   if (results.length === 0) {
@@ -6511,12 +6593,26 @@ function runRefsSingle(opts) {
       return 1;
     }
     const rootDir2 = opts.projectRoot ?? resolveProjectRoot({ project: process.cwd() });
-    if (querySymbols({ name: symName, rootDir: rootDir2, limit: 1 }).length === 0) {
+    const defRows = querySymbols({ name: symName, rootDir: rootDir2, limit: REF_BLIND_DEF_PROBE_LIMIT });
+    if (defRows.length === 0) {
       emitErr(`Symbol not found: ${symName}${unknownSymbolSuggestion(symName, rootDir2)}`);
       if (opts.json !== true && isIndexEmptyForProject(globalDbPath(), rootDir2)) emitErr(emptyIndexMessage(rootDir2));
       return 1;
     }
+    const defPaths = defFileHint !== void 0 ? [defFileHint] : defRows.map((r) => r.filePath);
+    const firstDefPath = defPaths[0];
+    if (firstDefPath !== void 0 && defPaths.every((fp) => !isRefIndexedFile(fp))) {
+      emitErr(refBlindLanguageNotice(symName, detectLanguage(firstDefPath), refsDisplayPath(firstDefPath)));
+      return 1;
+    }
+    const kindRows = defFileHint !== void 0 ? querySymbols({ name: symName, filePath: defFileHint, rootDir: rootDir2, limit: REF_BLIND_DEF_PROBE_LIMIT }) : defRows;
+    const kindVerdict = refBlindKindVerdict(kindRows);
+    if (kindVerdict.allBlind) {
+      emitErr(refBlindKindNotice(symName, kindVerdict.blindKinds));
+      return 1;
+    }
     emitErr(`No references found for '${symName}'`);
+    if (kindVerdict.blindCount > 0) emitErr(refBlindKindPartialNote(symName, kindVerdict.blindKinds, kindVerdict.blindCount, kindRows.length));
     if (opts.json !== true) {
       if (isIndexEmptyForProject(globalDbPath(), rootDir2)) emitErr(emptyIndexMessage(rootDir2));
     }
@@ -6529,7 +6625,7 @@ function runRefsSingle(opts) {
       payload = topFilesJsonPayload(results, opts.top);
     } else {
       const capped = guardJsonRows(results);
-      const trueTotal = opts.excludeTests === true || matchesGrep !== void 0 ? filteredTotal ?? results.length : countRefs(queryOpts);
+      const trueTotal = clientFiltered ? filteredTotal ?? results.length : countRefs(queryOpts);
       payload = { items: refsJsonItems(capped.items, opts.context ?? 0), truncated: capped.truncated || trueTotal > results.length, totalCount: trueTotal };
     }
     const hiddenByGrep = matchesGrep !== void 0 ? preGrepCount - (filteredTotal ?? results.length) : 0;
@@ -6539,7 +6635,7 @@ function runRefsSingle(opts) {
     return 0;
   }
   const lines = opts.top !== void 0 ? renderTopFilesSummary(results, opts.top, suppressed) : opts.callers === true ? [...opts.excludeTests === true && suppressed > 0 ? [`${countNoun(results.length, "reference")} (${excludeTestsHiddenNote(suppressed)})`] : [], ...renderCallerGroups(results, opts.context ?? 0)] : [...opts.excludeTests === true && suppressed > 0 ? [`${countNoun(results.length, "reference")} (${excludeTestsHiddenNote(suppressed)})`] : [], ...results.flatMap((ref) => renderRefLines(ref, opts.context ?? 0, ""))];
-  const refsFooter = opts.top !== void 0 ? "" : truncationFooter(results.length, opts.limit ?? 100, () => refsTotal(opts.excludeTests === true || matchesGrep !== void 0, filteredTotal, results.length, () => countRefs(queryOpts), preScanCount), "references", "--limit");
+  const refsFooter = opts.top !== void 0 ? "" : truncationFooter(results.length, opts.limit ?? 100, () => refsTotal(clientFiltered, filteredTotal, results.length, () => countRefs(queryOpts), preScanCount, scanLimit), "references", "--limit");
   const text = lines.join("\n");
   emit(guardText(text, "symbol") + refsFooter);
   recordReadStat("symbol_read", fullSourceBytes, text + refsFooter, symName);
@@ -6726,7 +6822,23 @@ function runSkeleton(opts) {
   return { text, code: 0 };
 }
 var DOC_SUMMARY_MAX_CHARS = 140;
+function firstSentenceEnd(line) {
+  const MIN_SENTENCE_CHARS = 30;
+  const ABBREV = new RegExp("(?:\\b(?:e\\.g|i\\.e|vs|cf|etc|approx|al|Dr|Mr|Ms|St|Fig|No)\\.|\\b\\p{L}\\.)$", "u");
+  for (const m of line.matchAll(/[.!?](?=\s|$)/gu)) {
+    const end = m.index + 1;
+    if (end < MIN_SENTENCE_CHARS) continue;
+    const head = line.slice(0, end);
+    if (ABBREV.test(head)) continue;
+    return end;
+  }
+  return null;
+}
 function clipDocSummary(firstLine) {
+  const sentence = firstSentenceEnd(firstLine);
+  if (sentence !== null && sentence <= DOC_SUMMARY_MAX_CHARS) {
+    return sentence === firstLine.length ? firstLine : `${firstLine.slice(0, sentence)}\u2026`;
+  }
   if (firstLine.length <= DOC_SUMMARY_MAX_CHARS) return firstLine;
   const cut = firstLine.lastIndexOf(" ", DOC_SUMMARY_MAX_CHARS);
   return `${firstLine.slice(0, cut > 40 ? cut : DOC_SUMMARY_MAX_CHARS).trimEnd()}\u2026`;
@@ -7930,10 +8042,12 @@ function runChanged(opts = {}) {
   }
   if (opts.symbolMode === true) {
     let hunksByFile = /* @__PURE__ */ new Map();
+    let symbolDiffBaselineBytes = 0;
     try {
-      const diffResult = runGit(["diff", ref, "--unified=0"], { cwd });
+      const diffResult = runGit(["diff", ref, "--unified=0", "--", ...changedFiles], { cwd });
       if (diffResult.exitCode === 0) {
         hunksByFile = parseDiffHunks(diffResult.stdout);
+        symbolDiffBaselineBytes = Buffer.byteLength(diffResult.stdout, "utf8");
       }
     } catch {
     }
@@ -7949,7 +8063,7 @@ function runChanged(opts = {}) {
       emit("No symbols changed.");
       return 0;
     }
-    const symbolFullBytes = sumFileSizes(changedFiles.map((f) => resolveIndexPath(f, projectRoot)));
+    const symbolFullBytes = symbolDiffBaselineBytes;
     if (opts.json === true) {
       const capped = guardJsonRows(allSymbols);
       const text = JSON.stringify({ items: capped.items, truncated: capped.truncated, totalCount: capped.totalCount }, null, 2);
@@ -7964,7 +8078,7 @@ function runChanged(opts = {}) {
     recordReadStat("changed_lookup", symbolFullBytes, symbolText, ref);
     return 0;
   }
-  const fullBytes = sumFileSizes(changedFiles.map((f) => resolveIndexPath(f, projectRoot)));
+  const fullBytes = changedDiffBaselineBytes(cwd, ref, changedFiles);
   if (opts.json === true) {
     const capped = guardJsonRows(changedFiles);
     const text = JSON.stringify({ items: capped.items, truncated: capped.truncated, totalCount: capped.totalCount }, null, 2);
@@ -9701,12 +9815,26 @@ function runCallers(opts) {
       emitErr2(`No non-test references found for '${opts.symbol}' (${excludeTestsHiddenNote(suppressed)})`);
       return 1;
     }
-    if (querySymbols({ name, rootDir, limit: 1 }).length === 0) {
+    const defRows = querySymbols({ name, rootDir, limit: REF_BLIND_DEF_PROBE_LIMIT });
+    if (defRows.length === 0) {
       emitErr2(`Symbol not found: ${opts.symbol}${unknownSymbolSuggestion(name, rootDir)}`);
       if (opts.json !== true && isIndexEmptyForProject(globalDbPath(), rootDir)) emitErr2(emptyIndexMessage(rootDir));
       return 1;
     }
+    const defPaths = fileHint !== void 0 ? [fileHint] : defRows.map((r) => r.filePath);
+    const firstDefPath = defPaths[0];
+    if (firstDefPath !== void 0 && defPaths.every((fp) => !isRefIndexedFile(fp))) {
+      emitErr2(refBlindLanguageNotice(name, detectLanguage(firstDefPath), toDisplayPath(rootDir, firstDefPath)));
+      return 1;
+    }
+    const kindRows = fileHint !== void 0 ? querySymbols({ name, filePath: fileHint, limit: REF_BLIND_DEF_PROBE_LIMIT }) : defRows;
+    const kindVerdict = refBlindKindVerdict(kindRows);
+    if (kindVerdict.allBlind) {
+      emitErr2(refBlindKindNotice(name, kindVerdict.blindKinds));
+      return 1;
+    }
     emitErr2(`No references found for '${opts.symbol}'`);
+    if (kindVerdict.blindCount > 0) emitErr2(refBlindKindPartialNote(name, kindVerdict.blindKinds, kindVerdict.blindCount, kindRows.length));
     if (opts.json !== true && isIndexEmptyForProject(globalDbPath(), rootDir)) emitErr2(emptyIndexMessage(rootDir));
     return 1;
   }
@@ -9754,6 +9882,14 @@ function runCallChain(opts) {
     if (opts.json !== true && isIndexEmptyForProject(globalDbPath(), rootDir)) emitErr2(emptyIndexMessage(rootDir));
     return 1;
   }
+  const refBlindRootPath = () => {
+    const paths = fileHint !== void 0 ? [fileHint] : querySymbols({ name, rootDir, limit: REF_BLIND_DEF_PROBE_LIMIT }).map((r) => r.filePath);
+    return paths.length > 0 && paths.every((fp) => !isRefIndexedFile(fp)) ? paths[0] : void 0;
+  };
+  const refBlindRootKinds = () => {
+    const rows = fileHint !== void 0 ? querySymbols({ name, filePath: fileHint, limit: REF_BLIND_DEF_PROBE_LIMIT }) : querySymbols({ name, rootDir, limit: REF_BLIND_DEF_PROBE_LIMIT });
+    return { ...refBlindKindVerdict(rows), total: rows.length };
+  };
   const getSyms = buildFileSymCache();
   let suppressedCount = 0;
   const callersOf = (n) => {
@@ -9778,7 +9914,10 @@ function runCallChain(opts) {
   if (opts.json === true) {
     const hiddenByGrep = chains.length - filteredChains.length;
     const hiddenByExcludeTests = noCallers && opts.excludeTests === true ? suppressedCount : 0;
-    emit2(JSON.stringify({ chains: filteredChains, ...hiddenByGrep > 0 ? { hiddenByGrep } : {}, ...hiddenByExcludeTests > 0 ? { hiddenByExcludeTests } : {} }, null, 2));
+    const refBlindKinds = noCallers ? refBlindRootKinds().blindKinds : [];
+    const blindRootPath = noCallers ? refBlindRootPath() : void 0;
+    const refBlindLanguage = blindRootPath !== void 0 ? { language: detectLanguage(blindRootPath), definedIn: toDisplayPath(rootDir, blindRootPath) } : void 0;
+    emit2(JSON.stringify({ chains: filteredChains, ...hiddenByGrep > 0 ? { hiddenByGrep } : {}, ...hiddenByExcludeTests > 0 ? { hiddenByExcludeTests } : {}, ...refBlindKinds.length > 0 ? { refBlindKinds } : {}, ...refBlindLanguage !== void 0 ? { refBlindLanguage } : {} }, null, 2));
     return 0;
   }
   if (noCallers) {
@@ -9786,7 +9925,20 @@ function runCallChain(opts) {
       emit2(`${name}  (no non-test callers; ${excludeTestsHiddenNote(suppressedCount)})`);
       return 0;
     }
+    const blindRoot = refBlindRootPath();
+    if (blindRoot !== void 0) {
+      emitErr2(refBlindLanguageNotice(name, detectLanguage(blindRoot), toDisplayPath(rootDir, blindRoot)));
+      emit2(`${name}  (no callers recorded)`);
+      return 0;
+    }
+    const rootKinds = refBlindRootKinds();
+    if (rootKinds.allBlind) {
+      emitErr2(refBlindKindNotice(name, rootKinds.blindKinds));
+      emit2(`${name}  (no callers recorded)`);
+      return 0;
+    }
     emit2(`${name}  (no callers)`);
+    if (rootKinds.blindCount > 0) emitErr2(refBlindKindPartialNote(name, rootKinds.blindKinds, rootKinds.blindCount, rootKinds.total));
     return 0;
   }
   if (matchesGrep !== void 0 && filteredChains.length === 0) {
@@ -9881,11 +10033,25 @@ function runImpact(opts) {
       emitErr2(`No non-test impact found for '${opts.symbol}' (${excludeTestsHiddenNote(suppressedCount)})`);
       return 1;
     }
-    if (querySymbols({ name: rootName, rootDir, limit: 1 }).length === 0) {
+    const defRows = querySymbols({ name: rootName, rootDir, limit: REF_BLIND_DEF_PROBE_LIMIT });
+    if (defRows.length === 0) {
       emitErr2(`Symbol not found: ${opts.symbol}${unknownSymbolSuggestion(rootName, rootDir)}`);
       return 1;
     }
+    const defPaths = fileHint !== void 0 ? [fileHint] : defRows.map((r) => r.filePath);
+    const firstDefPath = defPaths[0];
+    if (firstDefPath !== void 0 && defPaths.every((fp) => !isRefIndexedFile(fp))) {
+      emitErr2(refBlindLanguageNotice(rootName, detectLanguage(firstDefPath), toDisplayPath(rootDir, firstDefPath)));
+      return 1;
+    }
+    const kindRows = fileHint !== void 0 ? querySymbols({ name: rootName, filePath: fileHint, limit: REF_BLIND_DEF_PROBE_LIMIT }) : defRows;
+    const kindVerdict = refBlindKindVerdict(kindRows);
+    if (kindVerdict.allBlind) {
+      emitErr2(refBlindKindNotice(rootName, kindVerdict.blindKinds));
+      return 1;
+    }
     emitErr2(`No callers found for '${opts.symbol}'`);
+    if (kindVerdict.blindCount > 0) emitErr2(refBlindKindPartialNote(rootName, kindVerdict.blindKinds, kindVerdict.blindCount, kindRows.length));
     return 1;
   }
   if (opts.json === true) {
@@ -9918,7 +10084,10 @@ function hasAncestorDispatchRef(methodName, scopeName, classFile, rootDir) {
     const baseSyms = querySymbols({ name: baseName, kind: "class", limit: 1, rootDir });
     const baseSym = baseSyms[0];
     if (baseSym === void 0) return false;
-    const baseRefs = queryRefs({ name: methodName, filePath: baseSym.filePath, limit: DEFAULT_REF_QUERY_LIMIT, rootDir });
+    let baseRefs = queryRefs({ name: methodName, filePath: baseSym.filePath, limit: DEFAULT_REF_QUERY_LIMIT, rootDir });
+    if (baseRefs.length >= DEFAULT_REF_QUERY_LIMIT && !baseRefs.some((ref) => ref.line >= baseSym.lineStart && ref.line <= baseSym.lineEnd)) {
+      baseRefs = queryRefs({ name: methodName, filePath: baseSym.filePath, limit: UNBOUNDED_REF_LIMIT, rootDir });
+    }
     if (baseRefs.some((ref) => ref.line >= baseSym.lineStart && ref.line <= baseSym.lineEnd)) return true;
     currentName = baseName;
     currentFile = baseSym.filePath;
@@ -9992,11 +10161,26 @@ function runDead(opts) {
     }
     return 1;
   }
-  const syms = kinds.flatMap((k) => querySymbols({ kind: k, limit: UNBOUNDED_REF_LIMIT, rootDir }));
+  const blindKinds = kinds.filter((k) => REF_BLIND_KINDS.includes(k));
+  const assessableKinds = kinds.filter((k) => !REF_BLIND_KINDS.includes(k));
+  if (blindKinds.length > 0 && assessableKinds.length === 0) {
+    const blindLabel = blindKinds.length === 1 ? "kind" : "kinds";
+    emitErr2(`Cannot assess deadness for ${blindLabel}: ${blindKinds.map((k) => `'${k}'`).join(", ")} -- ${REF_BLIND_KIND_REASON}.`);
+    emitErr2(`Every symbol of ${blindKinds.length === 1 ? "this kind" : "these kinds"} would be reported dead, so no result is emitted rather than a wrong one. To hunt unused type declarations, list them with 'token-goat types --json' and search the source for each name directly.`);
+    return 1;
+  }
+  const blindKindNote = blindKinds.length > 0 ? `Note: ${blindKinds.map((k) => `'${k}'`).join(", ")} excluded -- ${REF_BLIND_KIND_REASON}.` : void 0;
+  if (blindKindNote !== void 0 && opts.json !== true) emitErr2(blindKindNote);
+  const syms = assessableKinds.flatMap((k) => querySymbols({ kind: k, limit: UNBOUNDED_REF_LIMIT, rootDir }));
   const getSyms = buildFileSymCache();
   const results = [];
   let suppressed = 0;
+  let refBlindByLanguage = 0;
   for (const sym of syms) {
+    if (!isRefIndexedFile(sym.filePath)) {
+      refBlindByLanguage += 1;
+      continue;
+    }
     if (opts.includePrivate !== true && sym.name.startsWith("_")) continue;
     const refs = queryRefs({ name: sym.name, limit: DEFAULT_REF_QUERY_LIMIT, rootDir });
     let scoped = filterRefsForSymbol(refs, sym.name, sym.filePath, getSyms);
@@ -10014,6 +10198,9 @@ function runDead(opts) {
     }
     results.push({ name: sym.name, kind: sym.kind, file: sym.filePath, line: sym.lineStart });
   }
+  if (refBlindByLanguage > 0 && opts.json !== true) {
+    emitErr2(`Note: ${countNoun(refBlindByLanguage, "symbol")} skipped -- defined in a language whose call sites token-goat does not index, so deadness cannot be determined for them.`);
+  }
   const preGrepCount = results.length;
   const matchesGrep = opts.grep !== void 0 ? compileGrepMatcher(opts.grep) : void 0;
   const grepped = matchesGrep !== void 0 ? results.filter((r) => matchesGrep(r.name)) : results;
@@ -10025,7 +10212,7 @@ function runDead(opts) {
     }));
     const topTruncated = sliced.length < grepped.length;
     const hiddenByGrep = preGrepCount - grepped.length;
-    emit2(JSON.stringify({ items: capped.items, truncated: capped.truncated || topTruncated, totalCount: grepped.length, ...hiddenByGrep > 0 ? { hiddenByGrep } : {}, ...opts.excludeTests === true && suppressed > 0 ? { hiddenByExcludeTests: suppressed } : {} }, null, 2));
+    emit2(JSON.stringify({ items: capped.items, truncated: capped.truncated || topTruncated, totalCount: grepped.length, ...hiddenByGrep > 0 ? { hiddenByGrep } : {}, ...opts.excludeTests === true && suppressed > 0 ? { hiddenByExcludeTests: suppressed } : {}, ...blindKinds.length > 0 ? { excludedKinds: blindKinds, excludedKindsReason: REF_BLIND_KIND_REASON } : {}, ...refBlindByLanguage > 0 ? { unassessableByLanguage: refBlindByLanguage } : {} }, null, 2));
     return 0;
   }
   if (sliced.length === 0) {
@@ -10220,6 +10407,15 @@ var TYPE_KINDS = [
   // Kotlin/Scala object.
   "graphql_extend"
 ];
+var REF_BLIND_KINDS = [.../* @__PURE__ */ new Set([...TYPE_KINDS, "impl"])];
+function refBlindKindVerdict(rows) {
+  const blind = rows.filter((r) => REF_BLIND_KINDS.includes(r.kind));
+  return {
+    blindKinds: [...new Set(blind.map((r) => r.kind))],
+    blindCount: blind.length,
+    allBlind: rows.length > 0 && blind.length === rows.length
+  };
+}
 var CORE_SYMBOL_KINDS = [
   .../* @__PURE__ */ new Set([...GENERIC_SYMBOL_KINDS, ...TYPE_KINDS, ...OTHER_ADAPTER_SYMBOL_KINDS])
 ];
@@ -10728,8 +10924,9 @@ function runAsk(opts) {
   } catch {
   }
   if (!backendPath) return degrade(`${BACKEND_ENV}=${backendLabel} is set, but '${backendLabel}' was not found on PATH`);
-  const context = hits.map((h, i) => `[${i + 1}] ${h.filePath}
+  const rawContext = hits.map((h, i) => `[${i + 1}] ${h.filePath}
 ${h.body ?? ""}`).join("\n\n");
+  const context = fenceUntrustedFileContent(redactSecrets(rawContext).text);
   const prompt = `Answer the QUESTION using only the CODE SNIPPETS below.
 QUESTION: ${opts.question}