@tobilu/qmd 1.0.7 → 1.1.2

package/CHANGELOG.md

@@ -2,6 +2,96 @@
 
 ## [Unreleased]
 
+## [1.1.2] - 2026-03-07
+
+13 community PRs merged. GPU initialization replaced with node-llama-cpp's
+built-in `autoAttempt` — deleting ~220 lines of manual fallback code and
+fixing GPU issues reported across 10+ PRs in one shot. Reranking is faster
+through chunk deduplication and a parallelism cap that prevents VRAM
+exhaustion.
+
+### Changes
+
+- **GPU init**: use node-llama-cpp's `build: "autoAttempt"` instead of manual
+  GPU backend detection. Automatically tries Metal/CUDA/Vulkan and falls back
+  gracefully. #310 (thanks @giladgd — the node-llama-cpp author)
+- **Query `--explain`**: `qmd query --explain` exposes retrieval score traces
+  — backend scores, per-list RRF contributions, top-rank bonus, reranker
+  score, and final blended score. Works in JSON and CLI output. #242
+  (thanks @vyalamar)
+- **Collection ignore patterns**: `ignore: ["Sessions/**", "*.tmp"]` in
+  collection config to exclude files from indexing. #304 (thanks @sebkouba)
+- **Multilingual embeddings**: `QMD_EMBED_MODEL` env var lets you swap in
+  models like Qwen3-Embedding for non-English collections. #273 (thanks
+  @daocoding)
+- **Configurable expansion context**: `QMD_EXPAND_CONTEXT_SIZE` env var
+  (default 2048) — previously used the model's full 40960-token window,
+  wasting VRAM. #313 (thanks @0xble)
+- **`candidateLimit` exposed**: `-C` / `--candidate-limit` flag and MCP
+  parameter to tune how many candidates reach the reranker. #255 (thanks
+  @pandysp)
+- **MCP multi-session**: HTTP transport now supports multiple concurrent
+  client sessions, each with its own server instance. #286 (thanks @joelev)
+
+### Fixes
+
+- **Reranking performance**: cap parallel rerank contexts at 4 to prevent
+  VRAM exhaustion on high-core machines. Deduplicate identical chunk texts
+  before reranking — same content from different files now shares a single
+  reranker call. Cache scores by content hash instead of file path.
+- Deactivate stale docs when all files are removed from a collection and
+  `qmd update` is run. #312 (thanks @0xble)
+- Handle emoji-only filenames (`🐘.md` → `1f418.md`) instead of crashing.
+  #308 (thanks @debugerman)
+- Skip unreadable files during indexing (e.g. iCloud-evicted files returning
+  EAGAIN) instead of crashing. #253 (thanks @jimmynail)
+- Suppress progress bar escape sequences when stderr is not a TTY. #230
+  (thanks @dgilperez)
+- Emit format-appropriate empty output (`[]` for JSON, CSV header for CSV,
+  etc.) instead of plain text "No results." #228 (thanks @amsminn)
+- Correct Windows sqlite-vec package name (`sqlite-vec-windows-x64`) and add
+  `sqlite-vec-linux-arm64`. #225 (thanks @ilepn)
+- Fix claude plugin setup CLI commands in README. #311 (thanks @gi11es)
+
+## [1.1.1] - 2026-03-06
+
+### Fixes
+
+- Reranker: truncate documents exceeding the 2048-token context window
+  instead of silently producing garbage scores. Long chunks (e.g. from
+  PDF ingestion) now get a fair ranking.
+- Nix: add python3 and cctools to build dependencies. #214 (thanks
+  @pcasaretto)
+
+## [1.1.0] - 2026-02-20
+
+QMD now speaks in **query documents** — structured multi-line queries where every line is typed (`lex:`, `vec:`, `hyde:`), combining keyword precision with semantic recall. A single plain query still works exactly as before (it's treated as an implicit `expand:` and auto-expanded by the LLM). Lex now supports quoted phrases and negation (`"C++ performance" -sports -athlete`), making intent-aware disambiguation practical. The formal query grammar is documented in `docs/SYNTAX.md`.
+
+The npm package now uses the standard `#!/usr/bin/env node` bin convention, replacing the custom bash wrapper. This fixes native module ABI mismatches when installed via bun and works on any platform with node >= 22 on PATH.
+
+### Changes
+
+- **Query document format**: multi-line queries with typed sub-queries (`lex:`, `vec:`, `hyde:`). Plain queries remain the default (`expand:` implicit, but not written inside the document). First sub-query gets 2× fusion weight — put your strongest signal first. Formal grammar in `docs/SYNTAX.md`.
+- **Lex syntax**: full BM25 operator support. `"exact phrase"` for verbatim matching; `-term` and `-"phrase"` for exclusions. Essential for disambiguation when a term is overloaded across domains (e.g. `performance -sports -athlete`).
+- **`expand:` shortcut**: send a single plain query (or start the document with `expand:` on its only line) to auto-expand via the local LLM. Query documents themselves are limited to `lex`, `vec`, and `hyde` lines.
+- **MCP `query` tool** (renamed from `structured_search`): rewrote the tool description to fully teach AI agents the query document format, lex syntax, and combination strategy. Includes worked examples with intent-aware lex.
+- **HTTP `/query` endpoint** (renamed from `/search`; `/search` kept as silent alias).
+- **`collections` array filter**: filter by multiple collections in a single query (`collections: ["notes", "brain"]`). Removed the single `collection` string param — array only.
+- **Collection `include`/`exclude`**: `includeByDefault: false` hides a collection from all queries unless explicitly named via `collections`. CLI: `qmd collection exclude <name>` / `qmd collection include <name>`.
+- **Collection `update-cmd`**: attach a shell command that runs before every `qmd update` (e.g. `git stash && git pull --rebase --ff-only && git stash pop`). CLI: `qmd collection update-cmd <name> '<cmd>'`.
+- **`qmd status` tips**: shows actionable tips when collections lack context descriptions or update commands.
+- **`qmd collection` subcommands**: `show`, `update-cmd`, `include`, `exclude`. Bare `qmd collection` now prints help.
+- **Packaging**: replaced custom bash wrapper with standard `#!/usr/bin/env node` shebang on `dist/qmd.js`. Fixes native module ABI mismatches when installed via bun, and works on any platform where node >= 22 is on PATH.
+- **Removed MCP tools** `search`, `vector_search`, `deep_search` — all superseded by `query`.
+- **Removed** `qmd context check` command.
+- **CLI timing**: each LLM step (expand, embed, rerank) prints elapsed time inline (`Expanding query... (4.2s)`).
+
+### Fixes
+
+- `qmd collection list` shows `[excluded]` tag for collections with `includeByDefault: false`.
+- Default searches now respect `includeByDefault` — excluded collections are skipped unless explicitly named.
+- Fix main module detection when installed globally via npm/bun (symlink resolution).
+
 ## [1.0.7] - 2026-02-18
 
 ### Changes
@@ -333,4 +423,3 @@ notes, journals, and meeting transcripts.
 [Unreleased]: https://github.com/tobi/qmd/compare/v1.0.0...HEAD
 [1.0.0]: https://github.com/tobi/qmd/releases/tag/v1.0.0
 [0.9.0]: https://github.com/tobi/qmd/compare/v0.8.0...v0.9.0
-

package/README.md

@@ -97,8 +97,8 @@ Although the tool works perfectly fine when you just tell your agent to use it o
 **Claude Code** — Install the plugin (recommended):
 
 ```bash
-claude marketplace add tobi/qmd
-claude plugin add qmd@qmd
+claude plugin marketplace add tobi/qmd
+claude plugin install qmd@qmd
 ```
 
 Or configure MCP manually in `~/.claude/settings.json`:
@@ -252,12 +252,34 @@ QMD uses three local GGUF models (auto-downloaded on first use):
 
 | Model | Purpose | Size |
 |-------|---------|------|
-| `embeddinggemma-300M-Q8_0` | Vector embeddings | ~300MB |
+| `embeddinggemma-300M-Q8_0` | Vector embeddings (default) | ~300MB |
 | `qwen3-reranker-0.6b-q8_0` | Re-ranking | ~640MB |
 | `qmd-query-expansion-1.7B-q4_k_m` | Query expansion (fine-tuned) | ~1.1GB |
 
 Models are downloaded from HuggingFace and cached in `~/.cache/qmd/models/`.
 
+### Custom Embedding Model
+
+Override the default embedding model via the `QMD_EMBED_MODEL` environment variable.
+This is useful for multilingual corpora (e.g. Chinese, Japanese, Korean) where
+`embeddinggemma-300M` has limited coverage.
+
+```sh
+# Use Qwen3-Embedding-0.6B for better multilingual (CJK) support
+export QMD_EMBED_MODEL="hf:Qwen/Qwen3-Embedding-0.6B-GGUF/qwen3-embedding-0.6b-q8_0.gguf"
+
+# After changing the model, re-embed all collections:
+qmd embed -f
+```
+
+Supported model families:
+- **embeddinggemma** (default) — English-optimized, small footprint
+- **Qwen3-Embedding** — Multilingual (119 languages including CJK), MTEB top-ranked
+
+> **Note:** When switching embedding models, you must re-index with `qmd embed -f`
+> since vectors are not cross-compatible between models. The prompt format is
+> automatically adjusted for each model family.
+
 ## Installation
 
 ```sh
@@ -366,6 +388,7 @@ qmd query "user authentication"
 --min-score <num>  # Minimum score threshold (default: 0)
 --full             # Show full document content
 --line-numbers     # Add line numbers to output
+--explain          # Include retrieval score traces (query, JSON/CLI output)
 --index <name>     # Use named index
 
 # Output formats (for search and multi-get)
@@ -428,6 +451,9 @@ qmd search --md --full "error handling"
 # JSON output for scripting
 qmd query --json "quarterly reports"
 
+# Inspect how each result was scored (RRF + rerank blend)
+qmd query --json --explain "quarterly reports"
+
 # Use separate index for different knowledge base
 qmd --index work search "quarterly reports"
 ```

package/dist/collections.d.ts

@@ -16,8 +16,10 @@ export type ContextMap = Record<string, string>;
 export interface Collection {
     path: string;
     pattern: string;
+    ignore?: string[];
     context?: ContextMap;
     update?: string;
+    includeByDefault?: boolean;
 }
 /**
  * The complete configuration file structure
@@ -55,6 +57,21 @@ export declare function getCollection(name: string): NamedCollection | null;
  * List all collections
  */
 export declare function listCollections(): NamedCollection[];
+/**
+ * Get collections that are included by default in queries
+ */
+export declare function getDefaultCollections(): NamedCollection[];
+/**
+ * Get collection names that are included by default
+ */
+export declare function getDefaultCollectionNames(): string[];
+/**
+ * Update a collection's settings
+ */
+export declare function updateCollectionSettings(name: string, settings: {
+    update?: string | null;
+    includeByDefault?: boolean;
+}): boolean;
 /**
  * Add or update a collection
  */

package/dist/collections.js

@@ -117,6 +117,46 @@ export function listCollections() {
         ...collection,
     }));
 }
+/**
+ * Get collections that are included by default in queries
+ */
+export function getDefaultCollections() {
+    return listCollections().filter(c => c.includeByDefault !== false);
+}
+/**
+ * Get collection names that are included by default
+ */
+export function getDefaultCollectionNames() {
+    return getDefaultCollections().map(c => c.name);
+}
+/**
+ * Update a collection's settings
+ */
+export function updateCollectionSettings(name, settings) {
+    const config = loadConfig();
+    const collection = config.collections[name];
+    if (!collection)
+        return false;
+    if (settings.update !== undefined) {
+        if (settings.update === null) {
+            delete collection.update;
+        }
+        else {
+            collection.update = settings.update;
+        }
+    }
+    if (settings.includeByDefault !== undefined) {
+        if (settings.includeByDefault === true) {
+            // true is default, remove the field
+            delete collection.includeByDefault;
+        }
+        else {
+            collection.includeByDefault = settings.includeByDefault;
+        }
+    }
+    saveConfig(config);
+    return true;
+}
 /**
  * Add or update a collection
  */

package/dist/llm.d.ts

@@ -4,16 +4,23 @@
  * Provides embeddings, text generation, and reranking using local GGUF models.
  */
 import { type Token as LlamaToken } from "node-llama-cpp";
+/**
+ * Detect if a model URI uses the Qwen3-Embedding format.
+ * Qwen3-Embedding uses a different prompting style than nomic/embeddinggemma.
+ */
+export declare function isQwen3EmbeddingModel(modelUri: string): boolean;
 /**
  * Format a query for embedding.
- * Uses nomic-style task prefix format for embeddinggemma.
+ * Uses nomic-style task prefix format for embeddinggemma (default).
+ * Uses Qwen3-Embedding instruct format when a Qwen embedding model is active.
  */
-export declare function formatQueryForEmbedding(query: string): string;
+export declare function formatQueryForEmbedding(query: string, modelUri?: string): string;
 /**
  * Format a document for embedding.
- * Uses nomic-style format with title and text fields.
+ * Uses nomic-style format with title and text fields (default).
+ * Qwen3-Embedding encodes documents as raw text without special prefixes.
  */
-export declare function formatDocForEmbedding(text: string, title?: string): string;
+export declare function formatDocForEmbedding(text: string, title?: string, modelUri?: string): string;
 /**
  * Token with log probability
  */
@@ -130,7 +137,7 @@ export type RerankDocument = {
 };
 export declare const LFM2_GENERATE_MODEL = "hf:LiquidAI/LFM2-1.2B-GGUF/LFM2-1.2B-Q4_K_M.gguf";
 export declare const LFM2_INSTRUCT_MODEL = "hf:LiquidAI/LFM2.5-1.2B-Instruct-GGUF/LFM2.5-1.2B-Instruct-Q4_K_M.gguf";
-export declare const DEFAULT_EMBED_MODEL_URI = "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf";
+export declare const DEFAULT_EMBED_MODEL_URI: string;
 export declare const DEFAULT_RERANK_MODEL_URI = "hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf";
 export declare const DEFAULT_GENERATE_MODEL_URI = "hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf";
 export declare const DEFAULT_MODEL_CACHE_DIR: string;
@@ -183,6 +190,11 @@ export type LlamaCppConfig = {
     generateModel?: string;
     rerankModel?: string;
     modelCacheDir?: string;
+    /**
+     * Context size used for query expansion generation contexts.
+     * Default: 2048. Can also be set via QMD_EXPAND_CONTEXT_SIZE.
+     */
+    expandContextSize?: number;
     /**
      * Inactivity timeout in ms before unloading contexts (default: 2 minutes, 0 to disable).
      *
@@ -210,6 +222,7 @@ export declare class LlamaCpp implements LLM {
     private generateModelUri;
     private rerankModelUri;
     private modelCacheDir;
+    private expandContextSize;
     private embedModelLoadPromise;
     private generateModelLoadPromise;
     private rerankModelLoadPromise;
@@ -318,6 +331,8 @@ export declare class LlamaCpp implements LLM {
         context?: string;
         includeLexical?: boolean;
     }): Promise<Queryable[]>;
+    private static readonly RERANK_TEMPLATE_OVERHEAD;
+    private static readonly RERANK_TARGET_DOCS_PER_CONTEXT;
     rerank(query: string, documents: RerankDocument[], options?: RerankOptions): Promise<RerankResult>;
     /**
      * Get device/GPU info for status display.

package/dist/llm.js

@@ -3,25 +3,43 @@
  *
  * Provides embeddings, text generation, and reranking using local GGUF models.
  */
-import { getLlama, getLlamaGpuTypes, resolveModelFile, LlamaChatSession, LlamaLogLevel, } from "node-llama-cpp";
+import { getLlama, resolveModelFile, LlamaChatSession, LlamaLogLevel, } from "node-llama-cpp";
 import { homedir } from "os";
 import { join } from "path";
 import { existsSync, mkdirSync, statSync, unlinkSync, readdirSync, readFileSync, writeFileSync } from "fs";
 // =============================================================================
 // Embedding Formatting Functions
 // =============================================================================
+/**
+ * Detect if a model URI uses the Qwen3-Embedding format.
+ * Qwen3-Embedding uses a different prompting style than nomic/embeddinggemma.
+ */
+export function isQwen3EmbeddingModel(modelUri) {
+    return /qwen.*embed/i.test(modelUri) || /embed.*qwen/i.test(modelUri);
+}
 /**
  * Format a query for embedding.
- * Uses nomic-style task prefix format for embeddinggemma.
+ * Uses nomic-style task prefix format for embeddinggemma (default).
+ * Uses Qwen3-Embedding instruct format when a Qwen embedding model is active.
  */
-export function formatQueryForEmbedding(query) {
+export function formatQueryForEmbedding(query, modelUri) {
+    const uri = modelUri ?? process.env.QMD_EMBED_MODEL ?? DEFAULT_EMBED_MODEL;
+    if (isQwen3EmbeddingModel(uri)) {
+        return `Instruct: Retrieve relevant documents for the given query\nQuery: ${query}`;
+    }
     return `task: search result | query: ${query}`;
 }
 /**
  * Format a document for embedding.
- * Uses nomic-style format with title and text fields.
+ * Uses nomic-style format with title and text fields (default).
+ * Qwen3-Embedding encodes documents as raw text without special prefixes.
  */
-export function formatDocForEmbedding(text, title) {
+export function formatDocForEmbedding(text, title, modelUri) {
+    const uri = modelUri ?? process.env.QMD_EMBED_MODEL ?? DEFAULT_EMBED_MODEL;
+    if (isQwen3EmbeddingModel(uri)) {
+        // Qwen3-Embedding: documents are raw text, no task prefix
+        return title ? `${title}\n${text}` : text;
+    }
     return `title: ${title || "none"} | text: ${text}`;
 }
 // =============================================================================
@@ -29,7 +47,8 @@ export function formatDocForEmbedding(text, title) {
 // =============================================================================
 // HuggingFace model URIs for node-llama-cpp
 // Format: hf:<user>/<repo>/<file>
-const DEFAULT_EMBED_MODEL = "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf";
+// Override via QMD_EMBED_MODEL env var (e.g. hf:Qwen/Qwen3-Embedding-0.6B-GGUF/qwen3-embedding-0.6b-q8_0.gguf)
+const DEFAULT_EMBED_MODEL = process.env.QMD_EMBED_MODEL ?? "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf";
 const DEFAULT_RERANK_MODEL = "hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf";
 // const DEFAULT_GENERATE_MODEL = "hf:ggml-org/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf";
 const DEFAULT_GENERATE_MODEL = "hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf";
@@ -126,6 +145,24 @@ export async function pullModels(models, options = {}) {
  */
 // Default inactivity timeout: 5 minutes (keep models warm during typical search sessions)
 const DEFAULT_INACTIVITY_TIMEOUT_MS = 5 * 60 * 1000;
+const DEFAULT_EXPAND_CONTEXT_SIZE = 2048;
+function resolveExpandContextSize(configValue) {
+    if (configValue !== undefined) {
+        if (!Number.isInteger(configValue) || configValue <= 0) {
+            throw new Error(`Invalid expandContextSize: ${configValue}. Must be a positive integer.`);
+        }
+        return configValue;
+    }
+    const envValue = process.env.QMD_EXPAND_CONTEXT_SIZE?.trim();
+    if (!envValue)
+        return DEFAULT_EXPAND_CONTEXT_SIZE;
+    const parsed = Number.parseInt(envValue, 10);
+    if (!Number.isInteger(parsed) || parsed <= 0) {
+        process.stderr.write(`QMD Warning: invalid QMD_EXPAND_CONTEXT_SIZE="${envValue}", using default ${DEFAULT_EXPAND_CONTEXT_SIZE}.\n`);
+        return DEFAULT_EXPAND_CONTEXT_SIZE;
+    }
+    return parsed;
+}
 export class LlamaCpp {
     llama = null;
     embedModel = null;
@@ -137,6 +174,7 @@ export class LlamaCpp {
     generateModelUri;
     rerankModelUri;
     modelCacheDir;
+    expandContextSize;
     // Ensure we don't load the same model/context concurrently (which can allocate duplicate VRAM).
     embedModelLoadPromise = null;
     generateModelLoadPromise = null;
@@ -152,6 +190,7 @@ export class LlamaCpp {
         this.generateModelUri = config.generateModel || DEFAULT_GENERATE_MODEL;
         this.rerankModelUri = config.rerankModel || DEFAULT_RERANK_MODEL;
         this.modelCacheDir = config.modelCacheDir || MODEL_CACHE_DIR;
+        this.expandContextSize = resolveExpandContextSize(config.expandContextSize);
         this.inactivityTimeoutMs = config.inactivityTimeoutMs ?? DEFAULT_INACTIVITY_TIMEOUT_MS;
         this.disposeModelsOnInactivity = config.disposeModelsOnInactivity ?? false;
     }
@@ -249,27 +288,12 @@ export class LlamaCpp {
      */
     async ensureLlama() {
         if (!this.llama) {
-            // Detect available GPU types and use the best one.
-            // We can't rely on gpu:"auto" — it returns false even when CUDA is available
-            // (likely a binary/build config issue in node-llama-cpp).
-            // @ts-expect-error node-llama-cpp API compat
-            const gpuTypes = await getLlamaGpuTypes();
-            // Prefer CUDA > Metal > Vulkan > CPU
-            const preferred = ["cuda", "metal", "vulkan"].find(g => gpuTypes.includes(g));
-            let llama;
-            if (preferred) {
-                try {
-                    llama = await getLlama({ gpu: preferred, logLevel: LlamaLogLevel.error });
-                }
-                catch {
-                    llama = await getLlama({ gpu: false, logLevel: LlamaLogLevel.error });
-                    process.stderr.write(`QMD Warning: ${preferred} reported available but failed to initialize. Falling back to CPU.\n`);
-                }
-            }
-            else {
-                llama = await getLlama({ gpu: false, logLevel: LlamaLogLevel.error });
-            }
-            if (!llama.gpu) {
+            const llama = await getLlama({
+                // attempt to build
+                build: "autoAttempt",
+                logLevel: LlamaLogLevel.error
+            });
+            if (llama.gpu === false) {
                 process.stderr.write("QMD Warning: no GPU acceleration, running on CPU (slow). Run 'qmd status' for details.\n");
             }
             this.llama = llama;
@@ -466,7 +490,7 @@ export class LlamaCpp {
         if (this.rerankContexts.length === 0) {
             const model = await this.ensureRerankModel();
             // ~960 MB per context with flash attention at contextSize 2048
-            const n = await this.computeParallelism(1000);
+            const n = Math.min(await this.computeParallelism(1000), 4);
             const threads = await this.threadsPerContext(n);
             for (let i = 0; i < n; i++) {
                 try {
@@ -668,8 +692,10 @@ export class LlamaCpp {
       `
         });
         const prompt = `/no_think Expand this search query: ${query}`;
-        // Create fresh context for each call
-        const genContext = await this.generateModel.createContext();
+        // Create a bounded context for expansion to prevent large default VRAM allocations.
+        const genContext = await this.generateModel.createContext({
+            contextSize: this.expandContextSize,
+        });
         const sequence = genContext.getSequence();
         const session = new LlamaChatSession({ contextSequence: sequence });
         try {
@@ -731,38 +757,73 @@ export class LlamaCpp {
             await genContext.dispose();
         }
     }
+    // Qwen3 reranker chat template overhead (system prompt, tags, separators)
+    static RERANK_TEMPLATE_OVERHEAD = 200;
+    static RERANK_TARGET_DOCS_PER_CONTEXT = 10;
     async rerank(query, documents, options = {}) {
         // Ping activity at start to keep models alive during this operation
         this.touchActivity();
         const contexts = await this.ensureRerankContexts();
-        // Build a map from document text to original indices (for lookup after sorting)
-        const textToDoc = new Map();
-        documents.forEach((doc, index) => {
-            textToDoc.set(doc.text, { file: doc.file, index });
+        const model = await this.ensureRerankModel();
+        // Truncate documents that would exceed the rerank context size.
+        // Budget = contextSize - template overhead - query tokens
+        const queryTokens = model.tokenize(query).length;
+        const maxDocTokens = LlamaCpp.RERANK_CONTEXT_SIZE - LlamaCpp.RERANK_TEMPLATE_OVERHEAD - queryTokens;
+        const truncationCache = new Map();
+        const truncatedDocs = documents.map((doc) => {
+            const cached = truncationCache.get(doc.text);
+            if (cached !== undefined) {
+                return cached === doc.text ? doc : { ...doc, text: cached };
+            }
+            const tokens = model.tokenize(doc.text);
+            const truncatedText = tokens.length <= maxDocTokens
+                ? doc.text
+                : model.detokenize(tokens.slice(0, maxDocTokens));
+            truncationCache.set(doc.text, truncatedText);
+            if (truncatedText === doc.text)
+                return doc;
+            return { ...doc, text: truncatedText };
+        });
+        // Deduplicate identical effective texts before scoring.
+        // This avoids redundant work for repeated chunks and fixes collisions where
+        // multiple docs map to the same chunk text.
+        const textToDocs = new Map();
+        truncatedDocs.forEach((doc, index) => {
+            const existing = textToDocs.get(doc.text);
+            if (existing) {
+                existing.push({ file: doc.file, index });
+            }
+            else {
+                textToDocs.set(doc.text, [{ file: doc.file, index }]);
+            }
         });
         // Extract just the text for ranking
-        const texts = documents.map((doc) => doc.text);
+        const texts = Array.from(textToDocs.keys());
         // Split documents across contexts for parallel evaluation.
         // Each context has its own sequence with a lock, so parallelism comes
         // from multiple contexts evaluating different chunks simultaneously.
-        const n = contexts.length;
-        const chunkSize = Math.ceil(texts.length / n);
-        const chunks = Array.from({ length: n }, (_, i) => texts.slice(i * chunkSize, (i + 1) * chunkSize)).filter(chunk => chunk.length > 0);
-        const allScores = await Promise.all(chunks.map((chunk, i) => contexts[i].rankAll(query, chunk)));
+        const activeContextCount = Math.max(1, Math.min(contexts.length, Math.ceil(texts.length / LlamaCpp.RERANK_TARGET_DOCS_PER_CONTEXT)));
+        const activeContexts = contexts.slice(0, activeContextCount);
+        const chunkSize = Math.ceil(texts.length / activeContexts.length);
+        const chunks = Array.from({ length: activeContexts.length }, (_, i) => texts.slice(i * chunkSize, (i + 1) * chunkSize)).filter(chunk => chunk.length > 0);
+        const allScores = await Promise.all(chunks.map((chunk, i) => activeContexts[i].rankAll(query, chunk)));
         // Reassemble scores in original order and sort
         const flatScores = allScores.flat();
         const ranked = texts
             .map((text, i) => ({ document: text, score: flatScores[i] }))
             .sort((a, b) => b.score - a.score);
-        // Map back to our result format using the text-to-doc map
-        const results = ranked.map((item) => {
-            const docInfo = textToDoc.get(item.document);
-            return {
-                file: docInfo.file,
-                score: item.score,
-                index: docInfo.index,
-            };
-        });
+        // Map back to our result format.
+        const results = [];
+        for (const item of ranked) {
+            const docInfos = textToDocs.get(item.document) ?? [];
+            for (const docInfo of docInfos) {
+                results.push({
+                    file: docInfo.file,
+                    score: item.score,
+                    index: docInfo.index,
+                });
+            }
+        }
         return {
             results,
             model: this.rerankModelUri,
@@ -1019,7 +1080,8 @@ let defaultLlamaCpp = null;
  */
 export function getDefaultLlamaCpp() {
     if (!defaultLlamaCpp) {
-        defaultLlamaCpp = new LlamaCpp();
+        const embedModel = process.env.QMD_EMBED_MODEL;
+        defaultLlamaCpp = new LlamaCpp(embedModel ? { embedModel } : {});
     }
     return defaultLlamaCpp;
 }