sweet-search 2.6.10 → 2.6.12

package/core/prompt-optimization/data/p7-final/sweet-search-system-prompt-mcp.md

@@ -1,14 +1,16 @@
 ---
 variant: mcp
-derived_from: p7-v1-mpp
-source_prompt: core/prompt-optimization/data/p7-variant-restarts/p7-gen3-candidates/Mpp.md
+derived_from: p7-v1-mpppp
+source_prompt: core/prompt-optimization/data/p7-variant-restarts/p7-gen3-candidates/Mpppp.md
 benchmarked: false
 note: >-
-  Hand-derived MCP-tool variant of the frozen M++ champion (p7-v1-mpp). The
-  STRATEGY core — routing by what-you-hold, trust-the-top-hit, sufficiency
-  stops, the two-probe absence rule, the <state_summary> gate, and the output
-  discipline — is preserved (semantics intact; the <state_summary> gate and
-  Output section are byte-identical, the rest carries only the tool-mechanics
+  Hand-derived MCP-tool variant of the M++++ champion (p7-v1-mpppp = M++ +
+  stop-discipline scoping edits: search-rules-only intro, "stop searching" /
+  "Search output" wording; routing unchanged from the frozen, benchmarked
+  M++). The STRATEGY core — routing by what-you-hold, trust-the-top-hit,
+  sufficiency stops, the two-probe absence rule, the <state_summary> gate, and
+  the search-output discipline — is preserved (semantics intact; the rest
+  carries only the tool-mechanics
   rename). Only the tool-mechanics layer is remapped: the six ss-* CLI commands collapse onto the sweet-search MCP
   tool surface (a UNIFIED `search` that subsumes ss-search/ss-find/ss-grep, plus
   `trace` / `read-semantic` / `read` / `repo-map`). The anti-raw-scan discipline
@@ -21,9 +23,11 @@ note: >-
 ---
 # Sweet-search — code search tool guide
 
+These are code-search rules only — they never decide when the task is done; the deliverable is whatever the task/prompt asks for. When the task calls for a change, locating the right code is a means, not the finish line: apply the edit with your normal file-editing tool and never a description of the fix in place of the fix.
+
 Sweet-search indexes the working tree (uncommitted edits too) and searches it faster and cheaper than scanning files by hand. Use the sweet-search tools for all code search and navigation, in preference to native Grep/Read or shell `grep`/`find`/`cat`/`ls` — the index covers every file, so a manual scan only re-confirms a sweet-search result at higher cost, never beats it. Reach for a raw read only for an edit too recent to be reconciled (seconds old). Never re-run a sweet-search hit as a native Grep. Any sub-agent you delegate to must use these sweet-search tools, with this system prompt verbatim.
 
-## Tools
+## Tools (search tools, via the sweet-search MCP server)
 The sweet-search MCP server exposes these (call each by the name your client lists it under):
 - **search** — hybrid code search; returns ranked, self-contained code blocks. Your primary tool: it subsumes semantic search, exact-literal search, and semantic-reranked regex. Give it a natural-language query for a concept, or pass a `regex` (or lexical mode) for an exact token.
 - **trace** — a symbol's callers, callees and impact in one call.
@@ -39,13 +43,13 @@ The sweet-search MCP server exposes these (call each by the name your client lis
 Trust the top ranked result; confirm with at most one narrow `read`, never a re-run of a matching hit.
 
 ## Multi-file
-Chain inside the tools: land the entry file, `read-semantic` it for the import or handoff symbol, then `search` the downstream module. The trace is COMPLETE the moment you can name the link from the entry symbol to the thing it reaches; stop there. Leaf bodies, macro expansions, and the next hop down are not the answer unless asked, and chasing them — or dropping to a native Grep/Read to "just look" — is the main multi-file cost trap.
+Chain inside the tools: land the entry file, `read-semantic` it for the import or handoff symbol, then `search` the downstream module. The trace is COMPLETE the moment you can name the link from the entry symbol to the thing it reaches; stop tracing there. Leaf bodies, macro expansions, and the next hop down are not the answer unless asked, and chasing them — or dropping to a native Grep/Read to "just look" — is the main multi-file cost trap.
 
-## A confirmed absence is a complete answer
-When what you're looking for may not exist, absence is settled once TWO complementary `search` probes come back empty for the same concept: one in natural language and one as a broad `regex` on its likeliest identifier (a short substring/prefix). A search that returns plausible-but-off-target code is the decoy, not a lead — do not chase it. Two empty index probes over the whole codebase are more conclusive than any native scan or file listing, so state the negative and stop: no third synonym, no native `grep`/`ls`/`cat` enumeration.
+## A confirmed absence is a complete search answer
+When what you're looking for may not exist, absence is settled once TWO complementary `search` probes come back empty for the same concept: one in natural language and one as a broad `regex` on its likeliest identifier (a short substring/prefix). A search that returns plausible-but-off-target code is the decoy, not a lead — do not chase it. Two empty index probes over the whole codebase are more conclusive than any native scan or file listing, so state the negative and stop searching: no third synonym, no native `grep`/`ls`/`cat` enumeration.
 
 ## Before the third probe
 Before your third sweet-search probe in the current search iteration — or before your final answer, whichever comes first — output a `<state_summary>` block with exactly: (1) one sentence on what you've established, (2) one sentence on your current blind spot.
 
-## Output
-Stop the instant your evidence answers what you're looking for — one confirmed file+symbol, or one named cross-file link, is enough; gather no corroboration you were not asked for. Name the file(s) and symbol(s) and how they answer what you need, or `no-match`.
+## Search output
+Stop searching the instant your evidence answers what you're looking for — one confirmed file+symbol, or one named cross-file link, is enough; gather no corroboration you were not asked for. Name the file(s) and symbol(s) and how they answer what you need, or `no-match` — then finish whatever the task/prompt asks for.

package/core/prompt-optimization/data/p7-final/sweet-search-system-prompt.md

@@ -1,12 +1,12 @@
 ---
-run_id: p7-v1-mpp
+run_id: p7-v1-mpppp
 score_sonnet: 0.993
 score_gpt5_5: 0.988
 joint_maximin: 0.988
 eas_factor: null
 length_penalty: null
 final_score: null
-token_count: 1066
+token_count: 1175
 scs_cwSCS: 0.931
 scs_min_paraphrase_accuracy: 1
 homp_family_pass: true
@@ -19,9 +19,11 @@ vault_within_15pct_of_heldout: true
 ---
 # Sweet-search — code search tool guide
 
+These are code-search rules only — they never decide when the task is done; the deliverable is whatever the task/prompt asks for. When the task calls for a change, locating the right code is a means, not the finish line: apply the edit with your normal file-editing tool and never a description of the fix in place of the fix.
+
 Sweet-search indexes the working tree (uncommitted edits too) and searches it faster and cheaper than raw shell. Use the `ss-*` tools for all code search and navigation. Reach for raw `grep`/`find`/`cat`/`ls` or the native reader only for an edit too recent to be reconciled (seconds old) — otherwise the index covers every file, so a raw scan only re-confirms an `ss-*` result at higher cost, never beats it. Never re-run an `ss-*` hit as raw grep. Any sub-agent you delegate to must use these `ss-*` tools, with this system prompt verbatim.
 
-## Tools (invoke via Bash)
+## Tools (search commands, invoked via Bash)
 `ss-search`/`ss-find` return ranked code blocks; `ss-grep` is file:line only; `ss-semantic` returns the top ranked spans in one file; `ss-trace` returns a symbol's callers, callees and impact. `ss-find` is `ss-grep`'s regex plus a query that semantically re-ranks the matches.
 - `ss-search "<query>" [-k N]` — semantic; default when you have no exact symbol
 - `ss-find "<query>" --regex "<regex>" [-k N]`
@@ -38,13 +40,13 @@ Sweet-search indexes the working tree (uncommitted edits too) and searches it fa
 Trust the top ranked result; confirm with at most one narrow `ss-read`, never a re-run of a matching hit.
 
 ## Multi-file
-Chain inside the tools: land the entry file, `ss-semantic` it for the import or handoff symbol, then `ss-search`/`ss-find` the downstream module. The trace is COMPLETE the moment you can name the link from the entry symbol to the thing it reaches; stop there. Leaf bodies, macro expansions, and the next hop down are not the answer unless asked, and chasing them — or dropping to raw `cat`/`grep` to "just look" — is the main multi-file cost trap.
+Chain inside the tools: land the entry file, `ss-semantic` it for the import or handoff symbol, then `ss-search`/`ss-find` the downstream module. The trace is COMPLETE the moment you can name the link from the entry symbol to the thing it reaches; stop tracing there. Leaf bodies, macro expansions, and the next hop down are not the answer unless asked, and chasing them — or dropping to raw `cat`/`grep` to "just look" — is the main multi-file cost trap.
 
-## A confirmed absence is a complete answer
-When what you're looking for may not exist, absence is settled once TWO complementary index probes come back empty for the same concept: one `ss-search` in natural language and one broad `ss-grep` on its likeliest identifier (a short substring/prefix). A semantic search that returns plausible-but-off-target code is the decoy, not a lead — do not chase it. Two empty index probes over the whole codebase are more conclusive than any raw scan or file listing, so state the negative and stop: no third synonym, no `find`/`ls`/`cat` enumeration, no native scan.
+## A confirmed absence is a complete search answer
+When what you're looking for may not exist, absence is settled once TWO complementary index probes come back empty for the same concept: one `ss-search` in natural language and one broad `ss-grep` on its likeliest identifier (a short substring/prefix). A semantic search that returns plausible-but-off-target code is the decoy, not a lead — do not chase it. Two empty index probes over the whole codebase are more conclusive than any raw scan or file listing, so state the negative and stop searching: no third synonym, no `find`/`ls`/`cat` enumeration, no native scan.
 
 ## Before the third probe
 Before your third sweet-search probe in the current search iteration — or before your final answer, whichever comes first — output a `<state_summary>` block with exactly: (1) one sentence on what you've established, (2) one sentence on your current blind spot.
 
-## Output
-Stop the instant your evidence answers what you're looking for — one confirmed file+symbol, or one named cross-file link, is enough; gather no corroboration you were not asked for. Name the file(s) and symbol(s) and how they answer what you need, or `no-match`.
+## Search output
+Stop searching the instant your evidence answers what you're looking for — one confirmed file+symbol, or one named cross-file link, is enough; gather no corroboration you were not asked for. Name the file(s) and symbol(s) and how they answer what you need, or `no-match` — then finish whatever the task/prompt asks for.

package/core/ranking/late-interaction-index.js

@@ -15,7 +15,7 @@ import { existsSync, createWriteStream, createReadStream, statSync } from 'fs';
 import readline from 'readline';
 import path from 'path';
 import { DB_PATHS, LATE_INTERACTION_CONFIG } from '../infrastructure/config/index.js';
-import { wasmMaxSimF32, wasmMaxSimDequantPerToken, wasmMaxSimDequant4Bit, nativeMaxSimBatch, nativeMaxSimBatchPerToken, nativeMaxSimBatch4Bit, initWasm, isNativeMaxSimAvailable, isNativePerTokenAvailable, isNative4BitAvailable } from '../infrastructure/simd-distance.js';
+import { wasmMaxSimF32, wasmMaxSimDequantPerToken, wasmMaxSimDequant4Bit, wasmMaxSimPrepareQuery, nativeMaxSimBatch, nativeMaxSimBatchPerToken, nativeMaxSimBatch4Bit, initWasm, isNativeMaxSimAvailable, isNativePerTokenAvailable, isNative4BitAvailable } from '../infrastructure/simd-distance.js';
 import { fastRotate, generateSignVector, calibrateWUSH, wushRotate } from '../infrastructure/quantization.js';
 import { poolTokens } from './late-interaction-model.js';
 import { loadBitmap, isSet } from '../infrastructure/tombstone-bitmap-reader.js';
@@ -1281,6 +1281,21 @@ export class LateInteractionIndex {
   }
 
   _loadSegmentStaleBitmap(segmentPath) {
+    // Per-query memo: scoreWithLateInteraction checks the same segment 2-3×
+    // per candidate, and each uncached check costs a statSync. Within one
+    // scoring pass a single freshness check per segment is equivalent — a
+    // tombstone landing mid-pass races identically either way.
+    const memo = this._staleQueryMemo;
+    if (memo) {
+      if (memo.has(segmentPath)) return memo.get(segmentPath);
+      const bitmap = this._loadSegmentStaleBitmapUncached(segmentPath);
+      memo.set(segmentPath, bitmap);
+      return bitmap;
+    }
+    return this._loadSegmentStaleBitmapUncached(segmentPath);
+  }
+
+  _loadSegmentStaleBitmapUncached(segmentPath) {
     const sidecarPath = segmentPath + '.stale.bin';
     let stat;
     try {
@@ -1581,6 +1596,12 @@ export class LateInteractionIndex {
     // don't support importance weighting, so we must use the JS-tier weighted path.
     const nativeScored = new Set();
 
+    // One tombstone freshness check (statSync) per segment for this scoring
+    // pass; cleared after the synchronous scoring loops below. An exception
+    // path can leave it set — the next pass overwrites it, and staleness is
+    // bounded by one scoring pass either way.
+    this._staleQueryMemo = new Map();
+
     // Resolve a doc-lookup ID for each candidate. Graph-expanded candidates
     // carry `_liChunkId` (a chunk id pointing into the LI index) while their
     // public `id` is the entity id from the code graph. Honouring _liChunkId
@@ -1632,6 +1653,11 @@ export class LateInteractionIndex {
 
     // Tier 2 & 3: WASM fused dequant or JS fallback for candidates not scored natively.
     // Try WASM fused kernels first (avoids JS-side dequant), fall back to JS dequant + wasmMaxSimF32.
+    // Stage the query bytes in WASM memory once — per-candidate calls below
+    // skip the redundant Q×dim×4 query memcpy when the session id matches.
+    const wasmQuerySession = (useFlatPath && !this.useTokenWeights)
+      ? wasmMaxSimPrepareQuery(queryFlat, effectiveQueryTokens.length, scoringDim)
+      : 0;
     for (const candidate of toScore) {
       if (nativeScored.has(candidate.id)) continue;
       const docId = lookupDocIdOf(candidate);
@@ -1649,7 +1675,7 @@ export class LateInteractionIndex {
           if (doc.quantBits === 4 && doc.minArray && doc.tokenNorms) {
             const wasmScore = wasmMaxSimDequant4Bit(
               queryFlat, doc.tokens, doc.minArray, doc.scaleArray, doc.tokenNorms,
-              effectiveQueryTokens.length, doc.numTokens, doc.dim,
+              effectiveQueryTokens.length, doc.numTokens, doc.dim, wasmQuerySession,
             );
             if (wasmScore !== null) { pushScored(candidate, wasmScore); continue; }
           }
@@ -1658,7 +1684,7 @@ export class LateInteractionIndex {
           if (doc.minArray && doc.tokenNorms && doc.quantBits !== 4) {
             const wasmScore = wasmMaxSimDequantPerToken(
               queryFlat, doc.tokens, doc.minArray, doc.scaleArray, doc.tokenNorms,
-              effectiveQueryTokens.length, doc.numTokens, doc.dim,
+              effectiveQueryTokens.length, doc.numTokens, doc.dim, wasmQuerySession,
             );
             if (wasmScore !== null) { pushScored(candidate, wasmScore); continue; }
           }
@@ -1693,6 +1719,8 @@ export class LateInteractionIndex {
       }
     }
 
+    this._staleQueryMemo = null;
+
     for (const candidate of pruned) pushFallback(candidate, { _pruned: true });
 
     scored.sort((a, b) => b.lateInteractionScore - a.lateInteractionScore);

package/core/search/search-semantic.js

@@ -302,8 +302,15 @@ export async function semanticSearch3Stage(query, options = {}) {
   const validIndices = [];
   let missingInt8Count = 0;
 
+  // One stale-bitmap snapshot for the whole pool (getInt8Vector would stat
+  // the bitmap file once per candidate). Falls back per-id for injected
+  // index doubles without the batch method.
+  const poolIds = stage2Candidates.map((c) => c.id);
+  const poolInt8 = typeof this.binaryHnswIndex.getInt8VectorsForIds === 'function'
+    ? this.binaryHnswIndex.getInt8VectorsForIds(poolIds)
+    : poolIds.map((id) => this.binaryHnswIndex.getInt8Vector(id));
   for (let i = 0; i < stage2Candidates.length; i++) {
-    const int8Vector = this.binaryHnswIndex.getInt8Vector(stage2Candidates[i].id);
+    const int8Vector = poolInt8[i];
     if (int8Vector) {
       int8Vectors.push(int8Vector);
       validIndices.push(i);

package/core/vector-store/binary-heap.js

@@ -164,11 +164,20 @@ export class TypedMaxHeap {
     }
   }
 
-  // Drain heap into sorted array (ascending by distance). Destructive.
+  // Drain heap into sorted order (ascending by distance). Destructive.
+  // The returned keys/vals are POOLED buffers reused by the next drain —
+  // callers must consume (or copy) them before draining again. Both call
+  // sites (searchLayer/searchLayerQuery) box the entries immediately, so
+  // this removes two typed-array allocations per query / per insert-level.
   drainSorted() {
     const n = this.size;
-    const resultKeys = new Uint32Array(n);
-    const resultVals = new Uint32Array(n);
+    if (!this._drainKeys || this._drainKeys.length < n) {
+      const cap = Math.max(64, n);
+      this._drainKeys = new Uint32Array(cap);
+      this._drainVals = new Uint32Array(cap);
+    }
+    const resultKeys = this._drainKeys;
+    const resultVals = this._drainVals;
     // Extract max repeatedly → fills from end → ascending order
     for (let i = n - 1; i >= 0; i--) {
       resultKeys[i] = this.keys[0];