sweet-search 2.6.17 → 2.7.0

package/README.md

@@ -23,7 +23,7 @@ Every coding agent today reaches for grep + Read by reflex. *sweet-search* chall
 ## ✨ Highlights
 
 - **Hybrid retrieval** — one of the six tools uses BM25F lexical + dense semantic + structural graph signals, fused per query and reranked by late-interaction
-- **Agent-native by design** — token-budgeted output tiers, an optional MCP server (and default zero-overhead CLI), and a GEPA-evolved system prompt installed into Claude Code, Codex, Gemini CLI, and Cursor with one command
+- **Agent-native by design** — token-budgeted output tiers, an optional MCP server (and default zero-overhead CLI), and a GEPA-evolved system prompt — one `init` installs it into Claude Code by default (Codex, Gemini CLI, and Cursor via flags)
 - **Indexed grep, ~10× faster than ripgrep** — a sparse n-gram prefilter skips the files that provably can't match
 - **ColBERT-style reranking, locally** — per-token MaxSim late interaction on hand-written SIMD kernels
 - **GPU-accelerated indexing** — Apple Metal, CUDA, CoreML Neural Engine, or plain CPU via ORT; same engine, auto-selected
@@ -263,6 +263,7 @@ We're SOTA in June 2026 on 3/4 attempted benchmarks at HARDER settings (running
 - **Reproduction:** result artifacts live in [`eval/results/`](eval/results/); rerun via `eval/run_all.js`. The canonical full-pool loaders are in `eval/download_data.py`.
 - **Full corpus, not distractors.** Published baselines for GCSN- and CoSQA-style benchmarks typically rank the gold against 99 sampled distractors; every number here ranks against the benchmark's *full* corpus (6k–19k candidates) — strictly harder.
 - **Zero-shot + docstring-stripped.** We never fine-tune on these tasks. For docstring-derived benchmarks (AdvTest, M2CRB) we strip the docstring from the indexed code — otherwise the NL query matches itself verbatim (a no-strip AdvTest run scores a meaningless 0.98). This is the standard protocol; it is also why our AdvTest is lower than naïve setups that leave the docstring in.
+- **Dev/held-out split.** Our ranking work iterates against a fixed dev split of each benchmark (e.g. GCSN: 600 dev + 400 held-out per language, stratified, seed=42) and treats the remainder as held-out, inspected aggregate-only at milestones. The table figures are full-corpus runs — they include the dev portion we tuned against; a held-out-only breakdown ships with the next results refresh.
 - **What we deliberately don't claim yet.** CoIR (official metric NDCG@10 over per-subtask corpora up to ~1M docs), CoSQA+ (multi-positive, MAP-primary), and CLARC (per-group pools) use protocols and metrics our single-pool MRR@10 harness doesn't currently match. Rather than publish apples-to-oranges numbers, we omit them; faithful per-subtask CoIR (NDCG@10) runs are queued.
 - **M2CRB** — the paper's metric is *auMRRc* (area under the MRR-vs-pool-size curve; best published **52.7**, fine-tuned). Because that area averages over easier small pools, `auMRRc ≥ full-pool MRR` for any model — so our **54.0 full-pool MRR@10** (all 5,795 functions, zero-shot) clears their best on a strictly harder measure. No one publishes a plain full-corpus MRR@10 on M2CRB, so ours is the best available.
 - **AdvTest honesty note.** We could not reproduce the commonly-cited 59.5 for the bare CodeRankEmbed encoder on our corpus: the reference FP32 model scores 54.7 on our leak-free, docstring-stripped, full-19,210 setup, and our shipped INT8 build 51.4. We report our measured numbers and the reference check rather than the leaderboard figure.
@@ -399,7 +400,7 @@ flowchart TD
 | 🎯 **Intent Rerank** | • demote docs / tests / config when you want implementation<br/>• log-scaled call-site boosts surface the most-referenced function |
 | 🕸️ **Graph Expansion** | • typed-edge walks (`imports`/`extends`/`calls`/`uses`) · adaptive 2-hop on the AST graph · edges picked by intent<br/>• **PathRAG** flow pruning + degree normalization → hubs can't dominate |
 | 🧮 **Late interaction Rerank** | • Query embedded per-token by **LateOn-Code** (149M; a 17M **edge** variant auto-selected on low-RAM hosts)<br/>• **MaxSim** against the pre-indexed quantized token vectors<br/>• native Rust+Rayon MaxSim kernel ⚡ · WASM-SIMD fallback (1.26 s → 27 ms on a 231-candidate rerank) |
-| 📦 **Package** | • entity-aware expansion → whole functions (imports, docstrings, decorators)<br/>• same-file overlap demotion → diverse, non-overlapping spans<br/>• auto-selected **3k / 8k / 12k** token budget |
+| 📦 **Package** | • entity-aware expansion → whole functions (imports, docstrings, decorators)<br/>• same-file overlap demotion → diverse, non-overlapping spans<br/>• **symbol-family completion** (agent mode) — generated/width families surface as a compact indexed manifest instead of truncating silently, inside the same budget<br/>• auto-selected **3k / 8k / 12k** token budget |
 
 <details>
 <summary><b>🌶️ Extra spice — the bits that didn't fit the diagram</b></summary>
@@ -418,7 +419,7 @@ flowchart TD
 - **Quality priors:** every chunk carries a 0–1 prior from test proximity, git recency, symbol centrality (PageRank), comment density, and complexity — production code surfaces, stale fixtures sink.
 - **Community structure:** a canonical **Leiden** pass detects code communities on the entity graph at index time, feeding vocabulary prewarming and structural signals — it understands your modules, not just your directories.
 - **Multilingual:** 14 languages get full tree-sitter AST treatment; a 39-config registry covers 70+ extensions beyond that. Router features handle camelCase/snake_case, CJK density, and German compounds.
-- **Format-gated signals:** structure-aware boosts and demotions (symbol-exact, path-token, mega-entity) fire only in agent mode — they help agent-shaped queries and would hurt plain NL, so they stay gated by default.
+- **Format-gated signals:** structure-aware boosts and demotions (symbol-exact, path-token, anomalous-chunk, mega-entity) fire only in agent mode — they help agent-shaped queries and would hurt plain NL, so they stay gated by default.
 
 **🛟 Rescues & honest trade-offs.**
 - **Long-query rescue:** wordy NL queries that FTS5 would tokenize into an unsatisfiable `AND` fall back to multi-query BM25F + RRF — one query per content keyword, fused.
@@ -449,6 +450,7 @@ Every match comes back in stable `file:line` order — ripgrep-identical counts,
 
 - Full methodology, per-repo table, and the optimization log: [`docs/GREP_INDEXING_STRATEGY.md`](docs/GREP_INDEXING_STRATEGY.md).
 - Regexes with no extractable literals fall back to native grep over the indexed file set; fixed-string and glob queries use a ripgrep fallback.
+- **Dialect recovery** (agent mode): patterns written in GNU-grep BRE muscle memory (`foo\|bar`, `\(group\)`) are literals in Rust's regex dialect and used to silently match nothing — a zero-hit exact search now gets one gated auto-retry with the translated pattern instead of a false "no matches".
 
 </details>
 
@@ -553,6 +555,8 @@ without another search.
 <summary><b>More</b></summary>
 
 - The CLI/MCP form scales it up: `sweet-search read <file...>` (and the `read` MCP tool) batches **1–20 files in a single call**, each with the same symbol metadata — twenty files for the price of one tool invocation.
+- **Query-aware continuation** (agent mode): a range read that stops before EOF names the unread symbols below it — ranked by relevance to the session's recent queries, not just declaration order — plus the exact command to continue.
+- **Shown-span receipts** (agent mode, default-on): the daemon remembers what a session has already been shown; an exact re-read collapses to a one-line receipt instead of resending the bytes.
 
 </details>
 

package/core/infrastructure/code-graph-repository.js

@@ -13,6 +13,10 @@ import { existsSync, statSync } from 'fs';
 import { applyReadPragmas, prepareCached } from './db-utils.js';
 import { readAdjacentManifest, resolveManifestCodeGraphPath, sqlAliasPrefix } from './code-graph-visibility.js';
 
+function pathIsUnsafe(value) {
+  return value.startsWith('/') || value === '..' || value.startsWith('../') || value.includes('/../');
+}
+
 export class CodeGraphRepository {
   constructor(dbPath, options = {}) {
     this._baseDbPath = dbPath;
@@ -383,6 +387,60 @@ export class CodeGraphRepository {
     }
   }
 
+  /**
+   * Find exact indexed symbol-family candidates under a bounded directory.
+   * The search domain supplies an identifier stem discovered from indexed
+   * seeds; this adapter performs only parameterized persistence work.
+   *
+   * @param {string} stem - alphanumeric identifier fragment (3-64 chars)
+   * @param {{ filePrefix:string, types?:string[], limit?:number }} opts
+   * @returns {Array<{id,name,type,filePath,startLine,endLine,parentClass}>}
+   */
+  findFamilyCandidates(stem, opts = {}) {
+    const db = this._open();
+    if (!db || typeof stem !== 'string' || !/^[A-Za-z][A-Za-z0-9]{2,63}$/.test(stem)) return [];
+    const prefix = typeof opts.filePrefix === 'string'
+      ? opts.filePrefix.replace(/\\/g, '/').replace(/^\.\//, '').replace(/\/$/, '')
+      : '';
+    if (!prefix || prefix === '.' || pathIsUnsafe(prefix)) return [];
+    const types = Array.isArray(opts.types)
+      ? [...new Set(opts.types.filter((type) => typeof type === 'string' && /^[A-Za-z][\w-]{0,31}$/.test(type)))].slice(0, 16)
+      : [];
+    const limit = Math.max(1, Math.min(128, opts.limit ?? 64));
+    const escapeLike = (value) => value.replace(/[\\%_]/g, '\\$&');
+    try {
+      const typeSql = types.length > 0 ? `AND type IN (${types.map(() => '?').join(',')})` : '';
+      const sql = `
+        SELECT id, name, type, file_path, start_line, end_line, parent_class
+        FROM entities
+        WHERE lower(name) LIKE lower(?) ESCAPE '\\'
+          AND file_path LIKE ? ESCAPE '\\'
+          ${typeSql}
+          AND ${this._entityVisibilitySql(db)}
+        ORDER BY name ASC, file_path ASC, start_line ASC
+        LIMIT ?
+      `;
+      const args = [
+        `%${escapeLike(stem)}%`,
+        `${escapeLike(prefix)}/%`,
+        ...types,
+        ...this._entityVisibilityParams(db),
+        limit,
+      ];
+      return prepareCached(db, sql).all(...args).map((row) => ({
+        id: row.id,
+        name: row.name,
+        type: row.type,
+        filePath: row.file_path,
+        startLine: row.start_line,
+        endLine: row.end_line,
+        parentClass: row.parent_class || null,
+      }));
+    } catch {
+      return [];
+    }
+  }
+
   /**
    * Get a single entity by id, with file:line metadata.
    *

package/core/infrastructure/model-fetcher.js

@@ -292,6 +292,13 @@ export async function fetchModelFile(hfId, filePath, destDir, options = {}) {
       if (startByte > 0) {
         headers['Range'] = `bytes=${startByte}-`;
       }
+      // HF 403s anonymous downloads from datacenter IPs (CI runners), even on
+      // public files. A read token lifts that. Only sent to huggingface.co —
+      // never to a custom hfEndpoint, which may be an untrusted mirror.
+      const hfToken = process.env.HF_TOKEN || process.env.HUGGING_FACE_HUB_TOKEN;
+      if (hfToken && url.startsWith('https://huggingface.co/')) {
+        headers['Authorization'] = `Bearer ${hfToken}`;
+      }
 
       const resp = await fetch(url, { headers, signal });
 

package/core/search/agent-pack-completion.js

@@ -0,0 +1,493 @@
+/**
+ * Agent-only pack completion.
+ *
+ * This is a post-ranking presentation policy: it never changes retrieval
+ * scores or asks the model to search again. A compact continuation or indexed
+ * family manifest is paid for by removing a lower-ranked code body (or the
+ * older same-file hint), so the configured and realized pack ceilings stay
+ * unchanged.
+ */
+
+import path from 'node:path';
+import { readFileRange } from './search-pattern-chunks.js';
+import { containsToken, extractQueryEvidence, informativeSubtokens } from './query-sufficiency.js';
+
+const MAX_BOUNDARY_GAP_LINES = 12;
+const MAX_IMMEDIATE_GAP_LINES = 3;
+const MAX_REFERENCED_SIBLING_GAP_LINES = 160;
+const MAX_FAMILY_SEEDS = 24;
+const MAX_FAMILY_STEMS = 3;
+const MAX_FAMILY_CANDIDATES = 64;
+const BODY_REFERENCE_GENERIC_TOKENS = new Set(['get', 'set', 'has', 'can', 'could', 'should', 'needed', 'result', 'value', 'data', 'info']);
+const IDENTIFIER_RE = /\b[A-Za-z_$][A-Za-z0-9_$]{2,79}\b/g;
+const TRUNCATION_MARKER_RE = /^\s*\/\/ \.\.\. \(\d+ (?:more lines|lines elided)\)(?: \.\.\.)?\s*$/;
+
+function defaultEstimateTokens(text) {
+  return text ? Math.ceil(text.length / 3.5) : 0;
+}
+
+function normalizedLines(text) {
+  if (typeof text !== 'string' || !text) return [];
+  const lines = text.replace(/\r\n/g, '\n').split('\n');
+  if (lines.at(-1) === '') lines.pop();
+  return lines;
+}
+
+/** Return the last contiguous source coordinate actually rendered. */
+export function shownSourceEndLine(startLine, code, truncated = false) {
+  if (!Number.isInteger(startLine) || startLine < 1) return null;
+  const lines = normalizedLines(code);
+  if (lines.length === 0) return startLine - 1;
+  const marker = truncated ? lines.findIndex((line) => TRUNCATION_MARKER_RE.test(line)) : -1;
+  const sourceLineCount = marker >= 0 ? marker : lines.length;
+  return startLine + Math.max(0, sourceLineCount - 1);
+}
+
+function identifierParts(name) {
+  return String(name || '').match(/[A-Z]+(?=[A-Z][a-z]|\d|$)|[A-Z]?[a-z]+|[A-Z]+|\d+/g) || [];
+}
+
+function familyStem(name) {
+  const alpha = identifierParts(name).filter((part) => /^[A-Za-z]{3,}$/.test(part));
+  if (alpha.length === 0) return null;
+  alpha.sort((a, b) => b.length - a.length || a.localeCompare(b));
+  return alpha[0].toLowerCase();
+}
+
+function groupSortKey(group) {
+  const width = Number(group.prefix.match(/\d+/)?.[0] || 0);
+  const alpha = group.prefix.replace(/\d+/g, '').toLowerCase();
+  return { width, alpha, prefix: group.prefix.toLowerCase() };
+}
+
+function compareNumericLabels(a, b) {
+  const normalizedA = a.replace(/^0+(?=\d)/, '');
+  const normalizedB = b.replace(/^0+(?=\d)/, '');
+  return normalizedA.length - normalizedB.length
+    || normalizedA.localeCompare(normalizedB)
+    || a.length - b.length
+    || a.localeCompare(b);
+}
+
+/**
+ * Compact exact indexed names by their final numeric slot. No Cartesian
+ * products are generated: every rendered member came from `candidates`.
+ */
+export function buildIndexedFamilyManifest(candidates, { seedNames = [] } = {}) {
+  if (!Array.isArray(candidates) || candidates.length < 2) return null;
+  const seedStems = new Set(seedNames.map(familyStem).filter(Boolean));
+  const byName = new Map();
+  for (const candidate of candidates) {
+    const name = candidate?.name;
+    if (typeof name !== 'string' || !/[A-Za-z]/.test(name) || !/\d/.test(name)) continue;
+    const stem = familyStem(name);
+    if (!stem || (seedStems.size > 0 && !seedStems.has(stem))) continue;
+    if (!byName.has(name)) byName.set(name, candidate);
+  }
+
+  const groups = new Map();
+  for (const [name, candidate] of byName) {
+    const match = name.match(/^(.*?)(\d+)([^\d]*)$/);
+    if (!match) continue;
+    const [, prefix, numeric, suffix] = match;
+    if (!prefix || !/[A-Za-z]/.test(prefix)) continue;
+    const key = `${prefix}\u0000${suffix}`;
+    if (!groups.has(key)) groups.set(key, { prefix, suffix, values: new Map() });
+    groups.get(key).values.set(numeric, { candidate, numeric });
+  }
+
+  const compact = [...groups.values()]
+    .filter((group) => group.values.size >= 2)
+    .sort((a, b) => {
+      const ka = groupSortKey(a);
+      const kb = groupSortKey(b);
+      return ka.width - kb.width || ka.alpha.localeCompare(kb.alpha)
+        || ka.prefix.localeCompare(kb.prefix);
+    });
+  if (compact.length === 0) return null;
+
+  const labels = [];
+  const members = [];
+  for (const group of compact) {
+    const values = [...group.values.keys()].sort(compareNumericLabels);
+    labels.push(`${group.prefix}{${values.join(',')}}${group.suffix}`);
+    for (const value of values) members.push(group.values.get(value).candidate);
+  }
+  const rendered = `# indexed family: ${labels.join(' · ')}`;
+  return {
+    rendered,
+    tokens: defaultEstimateTokens(rendered),
+    groups: labels,
+    members,
+  };
+}
+
+function overlapsShown(candidate, results) {
+  return results.some((result) => {
+    if (!result?.code || result.file !== candidate.file) return false;
+    const start = result.shownStartLine ?? result.startLine;
+    const end = result.shownEndLine ?? result.endLine;
+    return Number.isFinite(start) && Number.isFinite(end)
+      && candidate.startLine <= end && candidate.endLine >= start;
+  });
+}
+
+function boundaryScore(entity, queryEvidence, parentClass, gap, allowImmediate = false) {
+  const name = String(entity?.name || '');
+  if (!name) return 0;
+  let exact = 0;
+  for (const anchor of queryEvidence.anchors) {
+    const caseSensitive = /[A-Z]/.test(anchor);
+    if (containsToken(name, anchor, { caseSensitive })) exact = Math.max(exact, 100);
+    else if (name.toLowerCase().includes(anchor.toLowerCase())) exact = Math.max(exact, 60);
+  }
+  const nameTokens = informativeSubtokens(name);
+  let matched = 0;
+  for (const token of queryEvidence.subtokens) if (nameTokens.has(token)) matched++;
+  if (exact === 0 && matched === 0) {
+    // The top-ranked result already establishes query relevance. Do not let a
+    // budget cutoff land immediately before its next complete sibling merely
+    // because the model phrased the behavior rather than the sibling's name.
+    return allowImmediate && gap <= MAX_IMMEDIATE_GAP_LINES
+      ? MAX_IMMEDIATE_GAP_LINES - gap + 1 : 0;
+  }
+  const sameParent = parentClass && entity.parentClass === parentClass ? 5 : 0;
+  return exact + matched * 10 + sameParent + (MAX_BOUNDARY_GAP_LINES - gap) / 100;
+}
+
+function bodySiblingScore(entity, code) {
+  const candidateTokens = [...informativeSubtokens(entity?.name)]
+    .filter((token) => !BODY_REFERENCE_GENERIC_TOKENS.has(token));
+  if (candidateTokens.length < 2 || !code) return 0;
+  let inspected = 0;
+  for (const [identifier] of String(code).matchAll(IDENTIFIER_RE)) {
+    if (++inspected > 256) break;
+    const referenceTokens = informativeSubtokens(identifier);
+    let overlap = 0;
+    for (const token of candidateTokens) if (referenceTokens.has(token)) overlap++;
+    if (overlap >= 2) return 40 + overlap * 10;
+  }
+  return 0;
+}
+
+function findBoundaryContinuation(results, query, regex, codeGraphRepo) {
+  if (!codeGraphRepo || typeof codeGraphRepo.findAdjacentEntities !== 'function') return null;
+  const evidence = extractQueryEvidence(query, regex);
+  if (evidence.anchors.length === 0 && evidence.subtokens.size === 0) return null;
+
+  for (const result of results) {
+    if (!result?.code || result.presentation !== 'full'
+        || !Number.isInteger(result.shownStartLine)
+        || !Number.isInteger(result.shownEndLine)) continue;
+    let adjacent;
+    try {
+      adjacent = codeGraphRepo.findAdjacentEntities(
+        result.file,
+        result.shownStartLine,
+        result.shownEndLine,
+        { perSide: 8 },
+      );
+    } catch {
+      continue;
+    }
+    let parentClass = result.parentClass || null;
+    if (!parentClass && result.entityId && typeof codeGraphRepo.getEntityById === 'function') {
+      try { parentClass = codeGraphRepo.getEntityById(result.entityId)?.parentClass || null; }
+      catch { parentClass = null; }
+    }
+    const candidates = (adjacent?.below || []).flatMap((entity) => {
+      const gap = entity.startLine - result.shownEndLine;
+      if (!Number.isInteger(entity.startLine) || !Number.isInteger(entity.endLine)
+          || gap < 1 || gap > (result.rank === 1
+            ? MAX_REFERENCED_SIBLING_GAP_LINES : MAX_BOUNDARY_GAP_LINES)) return [];
+      const candidate = { ...entity, file: result.file };
+      if (overlapsShown(candidate, results)) return [];
+      const namedScore = gap <= MAX_BOUNDARY_GAP_LINES
+        ? boundaryScore(entity, evidence, parentClass, gap, result.rank === 1) : 0;
+      const referencedScore = result.rank === 1 ? bodySiblingScore(entity, result.code) : 0;
+      const score = Math.max(namedScore, referencedScore);
+      return score > 0 ? [{ trigger: result, entity, score, gap }] : [];
+    }).sort((a, b) => b.score - a.score || a.gap - b.gap || a.entity.startLine - b.entity.startLine);
+    if (candidates.length > 0) return candidates[0];
+  }
+  return null;
+}
+
+function identifierMatchesQuery(name, evidence) {
+  for (const anchor of evidence.anchors) {
+    const caseSensitive = /[A-Z]/.test(anchor);
+    if (containsToken(name, anchor, { caseSensitive })
+        || name.toLowerCase().includes(anchor.toLowerCase())) return true;
+  }
+  const nameTokens = informativeSubtokens(name);
+  for (const token of evidence.subtokens) if (nameTokens.has(token)) return true;
+  return false;
+}
+
+function extractSeedNames(results, query, regex) {
+  const out = [];
+  const seen = new Set();
+  const evidence = extractQueryEvidence(query, regex);
+  const push = (name) => {
+    if (out.length >= MAX_FAMILY_SEEDS || typeof name !== 'string'
+        || !/[A-Za-z]/.test(name) || !/\d/.test(name) || seen.has(name)) return;
+    seen.add(name);
+    out.push(name);
+  };
+  for (const result of results) {
+    // A selected summary row is still indexed evidence. Generated-family
+    // searches often return the template/output map with code while concrete
+    // members (IVec2/UVec2, etc.) survive only as lower summary symbols.
+    push(result?.symbol);
+    if (result?.code) {
+      for (const name of result.code.match(IDENTIFIER_RE) || []) {
+        if (identifierMatchesQuery(name, evidence)) push(name);
+      }
+    }
+    if (out.length >= MAX_FAMILY_SEEDS) break;
+  }
+  return out;
+}
+
+function commonDirectory(filePaths) {
+  const dirs = filePaths
+    .filter((file) => typeof file === 'string' && file.length > 0)
+    .map((file) => path.posix.dirname(file.replace(/\\/g, '/')));
+  if (dirs.length === 0) return null;
+  const parts = dirs.map((dir) => dir.split('/').filter(Boolean));
+  const common = [];
+  for (let i = 0; i < Math.min(...parts.map((entry) => entry.length)); i++) {
+    if (!parts.every((entry) => entry[i] === parts[0][i])) break;
+    common.push(parts[0][i]);
+  }
+  let prefix = common.join('/');
+  if (!prefix || prefix === '.') return null;
+  if (new Set(dirs).size === 1 && /\d/.test(path.posix.basename(prefix))) {
+    const parent = path.posix.dirname(prefix);
+    if (parent && parent !== '.') prefix = parent;
+  }
+  return prefix;
+}
+
+function findIndexedFamily(indexedSeeds, codeGraphRepo) {
+  const stems = [...new Set(indexedSeeds.map((seed) => familyStem(seed.name)).filter(Boolean))]
+    .slice(0, MAX_FAMILY_STEMS);
+  let best = null;
+  for (const stem of stems) {
+    const seeds = indexedSeeds.filter((seed) => familyStem(seed.name) === stem);
+    const filePrefix = commonDirectory(seeds.map((seed) => seed.filePath));
+    if (!filePrefix) continue;
+    const types = [...new Set(seeds.map((seed) => seed.type).filter(Boolean))];
+    let candidates;
+    try {
+      candidates = codeGraphRepo.findFamilyCandidates(stem, {
+        filePrefix, types, limit: MAX_FAMILY_CANDIDATES,
+      });
+    } catch {
+      continue;
+    }
+    const manifest = buildIndexedFamilyManifest(candidates, {
+      seedNames: seeds.map((seed) => seed.name),
+    });
+    if (manifest && (!best || manifest.members.length > best.manifest.members.length)) {
+      best = { manifest, seeds };
+    }
+  }
+  return best;
+}
+
+/** Build grep family closure only from symbols indexed at exact match lines. */
+export function buildIndexedGrepFamilyManifest(results, codeGraphRepo) {
+  if (!Array.isArray(results) || results.length < 2
+      || typeof codeGraphRepo?.findEntitiesInRange !== 'function'
+      || typeof codeGraphRepo?.findFamilyCandidates !== 'function') return null;
+  const seeds = [];
+  const seen = new Set();
+  for (const result of results.slice(0, MAX_FAMILY_SEEDS)) {
+    let entities = [];
+    try { entities = codeGraphRepo.findEntitiesInRange(result.file, result.line, result.line) || []; }
+    catch { entities = []; }
+    if (entities.length === 0 && typeof codeGraphRepo.findEnclosingEntity === 'function') {
+      try { entities = [codeGraphRepo.findEnclosingEntity(result.file, result.line, result.line)].filter(Boolean); }
+      catch { entities = []; }
+    }
+    for (const entity of entities) {
+      if (typeof entity?.name !== 'string' || !/\d/.test(entity.name) || seen.has(entity.name)) continue;
+      seen.add(entity.name);
+      seeds.push({ ...entity, filePath: result.file });
+    }
+  }
+  if (seeds.length < 2) return null;
+  return findIndexedFamily(seeds, codeGraphRepo)?.manifest || null;
+}
+
+function discoverFamily(results, codeGraphRepo, query, regex) {
+  if (!codeGraphRepo
+      || typeof codeGraphRepo.findEntitiesByAnyName !== 'function'
+      || typeof codeGraphRepo.findFamilyCandidates !== 'function') return null;
+  const seedNames = extractSeedNames(results, query, regex);
+  if (seedNames.length === 0) return null;
+  let indexedSeeds;
+  try {
+    indexedSeeds = codeGraphRepo.findEntitiesByAnyName(seedNames, {
+      limit: MAX_FAMILY_SEEDS * 2,
+    });
+  } catch {
+    return null;
+  }
+  if (!Array.isArray(indexedSeeds) || indexedSeeds.length === 0) return null;
+  const family = findIndexedFamily(indexedSeeds, codeGraphRepo);
+  if (!family) return null;
+  const owner = results.find((result) => result?.code && (
+    family.manifest.members.some((member) => member.name === result.symbol)
+    || family.seeds.some((seed) => result.code.includes(seed.name))
+  )) || results.find((result) => result?.code);
+  return owner ? { manifest: family.manifest, owner } : null;
+}
+
+function summaryFor(result) {
+  const symbol = result.symbol || 'code block';
+  const type = result.symbolType ? ` (${result.symbolType})` : '';
+  return `${result.file}:${result.startLine} — ${symbol}${type}`;
+}
+
+function demoteDonor(result) {
+  result.summary ||= summaryFor(result);
+  result.presentation = 'summary';
+  result.code = null;
+  result.codeTokens = 0;
+  result.expanded = false;
+  delete result.shownStartLine;
+  delete result.shownEndLine;
+  delete result.boundaryTruncated;
+}
+
+function completeContinuation(boundary, fileCache, projectRoot, estimateTokens) {
+  if (!boundary) return null;
+  const { trigger, entity } = boundary;
+  const header = `# continues at ${trigger.file}:${entity.startLine} ${entity.name}`;
+  const code = readFileRange(fileCache, trigger.file, entity.startLine, entity.endLine, projectRoot);
+  const expectedLines = entity.endLine - entity.startLine + 1;
+  if (code && normalizedLines(code).length === expectedLines) {
+    return {
+      kind: 'symbol',
+      file: trigger.file,
+      startLine: entity.startLine,
+      endLine: entity.endLine,
+      symbol: entity.name,
+      symbolType: entity.type || null,
+      code,
+      rendered: header,
+      tokens: estimateTokens(`${header}\n${code}`),
+    };
+  }
+  return null;
+}
+
+function trailerContinuation(boundary, estimateTokens) {
+  if (!boundary) return null;
+  const { trigger, entity } = boundary;
+  const rendered = `# continues at ${trigger.file}:${entity.startLine} ${entity.name}`;
+  return {
+    kind: 'trailer',
+    file: trigger.file,
+    startLine: entity.startLine,
+    endLine: entity.endLine,
+    symbol: entity.name,
+    symbolType: entity.type || null,
+    code: null,
+    rendered,
+    tokens: estimateTokens(rendered),
+  };
+}
+
+/** Apply token-neutral completion in place and return updated accounting. */
+export function applyAgentPackCompletion({
+  results,
+  query,
+  regex,
+  codeGraphRepo,
+  fileCache,
+  projectRoot,
+  tokensUsed,
+  tokenBudget,
+  estimateTokens = defaultEstimateTokens,
+  isAgentFormat = false,
+}) {
+  if (isAgentFormat !== true || !Array.isArray(results) || results.length === 0) {
+    return { tokensUsed, changed: false };
+  }
+  const originalTokens = Math.max(0, Number(tokensUsed) || 0);
+  const boundary = findBoundaryContinuation(results, query, regex, codeGraphRepo);
+  const family = discoverFamily(results, codeGraphRepo, query, regex);
+  if (!boundary && !family) return { tokensUsed: originalTokens, changed: false };
+
+  // Reallocate only a lower-ranked tail. The top result and the rows that own
+  // the new evidence must remain code-bearing; otherwise a family-only pack
+  // can demote its sole useful hit and attach an invisible manifest to a
+  // summary row.
+  const donor = results.slice(1).reverse().find((result) => (
+    result !== boundary?.trigger
+    && result !== family?.owner
+    && result?.code
+    && Number(result.codeTokens) > 0
+  ));
+  const donorTokens = donor ? Number(donor.codeTokens) : 0;
+  const mapTokens = boundary?.trigger?.sameFile?.tokens || 0;
+  let available = donorTokens + mapTokens;
+  if (available <= 0) return { tokensUsed: originalTokens, changed: false };
+
+  let spent = 0;
+  let continuation = null;
+  if (boundary) {
+    const complete = completeContinuation(boundary, fileCache, projectRoot, estimateTokens);
+    const trailer = trailerContinuation(boundary, estimateTokens);
+    if (complete && complete.tokens <= available) continuation = complete;
+    else if (trailer.tokens <= available) continuation = trailer;
+    if (continuation) {
+      spent += continuation.tokens;
+      available -= continuation.tokens;
+    } else {
+      // The old same-file line cannot fund another feature unless it is
+      // actually replaced by a continuation.
+      available = donorTokens;
+    }
+  }
+
+  let familyManifest = null;
+  if (family) {
+    const familyTokens = estimateTokens(family.manifest.rendered);
+    if (familyTokens <= available) {
+      familyManifest = { ...family.manifest, tokens: familyTokens };
+      spent += familyTokens;
+      available -= familyTokens;
+    }
+  }
+  if (!continuation && !familyManifest) {
+    return { tokensUsed: originalTokens, changed: false };
+  }
+
+  const reclaimMap = !!continuation && mapTokens > 0;
+  const mapReclaimed = reclaimMap ? mapTokens : 0;
+  const needsDonor = spent > mapReclaimed;
+  if (needsDonor && !donor) return { tokensUsed: originalTokens, changed: false };
+
+  const reclaimed = mapReclaimed + (needsDonor ? donorTokens : 0);
+  const nextTokens = Math.max(0, originalTokens - reclaimed + spent);
+  const numericBudget = Number(tokenBudget);
+  const effectiveBudget = Number.isFinite(numericBudget) && numericBudget >= 0
+    ? numericBudget : originalTokens;
+  if (nextTokens > originalTokens || nextTokens > effectiveBudget) {
+    return { tokensUsed: originalTokens, changed: false };
+  }
+
+  if (reclaimMap) delete boundary.trigger.sameFile;
+  if (needsDonor) demoteDonor(donor);
+  if (continuation) boundary.trigger.continuation = continuation;
+  if (familyManifest) family.owner.familyManifest = familyManifest;
+
+  return {
+    tokensUsed: nextTokens,
+    changed: true,
+  };
+}