@tobilu/qmd 2.1.0 → 2.5.2

package/CHANGELOG.md

@@ -2,6 +2,94 @@
 
 ## [Unreleased]
 
+## [2.5.2] - 2026-05-22
+
+### Fixes
+
+- Launcher: Rewrite `bin/qmd` as a Node-based shebang polyglot to fix global npm installation execution failures on Windows (#668 / #452), while supporting seamless fallback to Bun in Node-less environments.
+
+
+## [2.5.1] - 2026-05-20
+
+### Changes
+
+- Release: publish from GitHub Actions via npm Trusted Publishing/OIDC instead of a long-lived `NPM_TOKEN` secret.
+
+## [2.5.0] - 2026-05-19
+
+### Changes
+
+- Dependencies: update core SQLite/config/chunking packages (`better-sqlite3`, `yaml`, `web-tree-sitter`, `tree-sitter-go`, and `tree-sitter-python`) while keeping incompatible `zod`, `tsx`, and `vitest` majors pinned.
+- Agent skills: add `qmd skills list|get|path` to serve version-matched runtime skill instructions from the installed CLI, and make `qmd skill install` write a stable discovery stub so installed agent skills do not go stale after QMD upgrades.
+- CLI: add `qmd doctor` for index/runtime diagnostics, including SQLite/sqlite-vec versions, embedding fingerprint freshness, mixed-fingerprint detection, safe legacy fingerprint adoption, and content-hash sampling.
+
+### Fixes
+
+- Launcher: prefer runnable TypeScript source in git checkouts even when ignored `dist/` artifacts exist, while packaged installs continue to run `dist/`.
+- GPU: keep node-llama-cpp's documented `gpu: "auto"` initialization as the primary path, then perform no-build packaged CUDA/Vulkan/Metal probes only if auto falls back to CPU.
+- CLI: move GPU/CPU runtime diagnostics out of `qmd status`; use `qmd doctor` for device probing and related environment guidance.
+- CLI: point unexpected command/setup failures toward `qmd doctor` so diagnostics are the default next step when QMD behaves incorrectly.
+- Doctor: explicitly warn when `content_vectors` contains multiple non-empty embedding fingerprint names, with the per-fingerprint document/chunk breakdown.
+- Embed: make the TTY progress line label byte-based input progress explicitly, show embedded chunks as a count, and shorten the displayed model name.
+- Embed: retain per-chunk failure details, retry failed chunks after later successful embeds and again when no other chunks remain, clear recovered errors, and cap retries to avoid endless loops.
+- Tests: expand the container smoke harness to cover npm-global, npx-style, and Bun-global install scenarios, always checking auto and `QMD_FORCE_CPU=1` doctor modes, with opt-in tiny `qmd embed` and GPU probe runs for supported container runtimes.
+- Embedding: fingerprint vector metadata using the active embedding model and formatting/chunking parameters so stale vectors are treated as pending after search semantics change. Legacy `content_vectors` columns are migrated lazily on first vector-health/write use to preserve fast QMD startup.
+
+- Skill: expand the packaged QMD skill with retrieval-first workflows, structured query examples, wiki/source collection guidance, and safe fallbacks when model-backed search is unavailable.
+- Tests: make `bun run test` execute the local unit suite under both Node/Vitest and Bun (`test:node` + `test:bun`) so runtime-specific regressions are caught before CI.
+- Model config: centralize embedding/rerank/generation model resolution so `qmd embed`, `status`, `query`, `vsearch`, `pull`, SDK vector search, and `bench` use the same active `.qmd/index.yaml` model hints and environment fallbacks.
+- GPU/status: `qmd status` now uses the same embedding model identity as `qmd embed` when computing pending embeddings, so URI-backed embeddings are not incorrectly reported as pending under the legacy `embeddinggemma` alias.
+- GPU status: `qmd status` now always shows GPU mode/configuration without unsafe native probing, and CPU-fallback warnings point to `QMD_STATUS_DEVICE_PROBE=1 qmd status` for an actual backend probe. The no-GPU warning is emitted once per process instead of once per LLM instance during benchmarks.
+- GPU: add `QMD_FORCE_CPU=1` / `--no-gpu` to bypass CUDA/Vulkan/Metal probing entirely, and route native llama.cpp stdout noise to stderr so JSON output stays parseable during search/query commands.
+- Snippet line numbers: `qmd_query` (MCP), HTTP `/query`, and `qmd query`
+  (CLI JSON output and snippet headers) now return absolute source-file
+  line numbers instead of chunk-local ones, so the `line` field can be
+  passed back to `qmd_get` as `fromLine` without a separate lookup.
+  Snippet selection remains scoped to the best matching chunk
+  (preserves #149).
+- CLI: `qmd query --full` now emits the full document body in all output
+  formats (json, csv, md, xml), restoring the documented behavior of the
+  flag. Previously it returned only the best matching chunk (~3.6KB max
+  per result). Output payload for `--full` queries is now proportional
+  to total document size.
+- macOS Metal: `qmd query --json` now flushes successful JSON output and uses a safe immediate-exit path on Darwin to avoid ggml Metal finalizer aborts; other commands still dispose LLM contexts/models before the llama runtime. #368
+- Embedding: require complete chunk coverage before treating a document as
+  embedded, remove partial vectors when chunk/session failures leave a
+  document incomplete, and keep `qmd status` pending counts honest after
+  interrupted long embed runs. #637 #378
+- Embedding: `qmd embed -c <collection>` now scopes pending-doc selection
+  to the requested collection instead of embedding global pending work.
+  Scoped `--force` clears only collection-owned vectors, preserves shared
+  hashes referenced by sibling collections, and drops `vectors_vec` only
+  when the scoped clear empties all vectors.
+- Hybrid search: weight RRF lists by query type so original FTS and original vector evidence get the intended 2x boost, instead of accidentally boosting the first lexical expansion. #591
+- MCP: seed llama.cpp/GGML quiet env vars before launching `qmd mcp` so native logs cannot pollute stdio JSON-RPC framing. #593
+- CLI: remove CommonJS `require()` calls from ESM index path normalization so `qmd --index <path>` no longer crashes with `ERR_AMBIGUOUS_MODULE_SYNTAX` on Node 22+. #634
+- Windows CUDA: serialize llama.cpp embedding/reranking contexts by default to avoid intermittent `ggml-cuda.cu:98` crashes in `qmd query`; set `QMD_EMBED_PARALLELISM` to opt back into parallel contexts if your driver is stable. #519
+- MCP: make `qmd mcp --index <name>` use the selected index for both foreground and daemon HTTP servers instead of falling back to the default store. #343
+- Embedding: respect `QMD_EMBED_MODEL` consistently for vector indexing and vector-backed search, with default-model fallback when unset.
+- Config: use one home-directory resolver for YAML config and the default SQLite cache path, avoiding Windows CLI/MCP split-brain when `HOME` is unset.
+- GPU: respect explicit `QMD_LLAMA_GPU=metal|vulkan|cuda` backend overrides instead of always using auto GPU selection. #529
+- Fix: preserve original filename case in `handelize()`. The previous
+  `.toLowerCase()` call made indexed paths unreachable on case-sensitive
+  filesystems (Linux). `qmd update` automatically migrates legacy
+  lowercase paths without re-embedding.
+- CLI: make `qmd status` skip native `node-llama-cpp` device probing by
+  default so status stays safe on machines with broken or unsupported GPU
+  drivers. Set `QMD_STATUS_DEVICE_PROBE=1` to opt in.
+- CLI: lazy-load `node-llama-cpp` so lightweight commands such as
+  `qmd status` do not import native ML dependencies or trigger llama.cpp
+  builds on ARM/no-GPU machines. #491
+- Store: keep content rows referenced by inactive documents during orphan
+  cleanup so `qmd update` preserves soft-deleted tombstones for removed
+  files. #585
+- Packaging: install AST grammar WASM packages as required dependencies so
+  Bun global installs include TypeScript/TSX/JavaScript grammars, and add a
+  `smoke:package-grammars` verification command. #595
+- Launcher: add wrapper smoke coverage for scoped package, npm/npx,
+  Homebrew/Linuxbrew, Bun global symlink layouts, and `$BUN_INSTALL`
+  false-positive runtime selection regressions. #351 #353 #354 #356 #358 #359
+
 ## [2.1.0] - 2026-04-05
 
 Code files now chunk at function and class boundaries via tree-sitter,

package/README.md

@@ -797,6 +797,9 @@ llm_cache       -- Cached LLM responses (query expansion, rerank scores)
 | Variable | Default | Description |
 |----------|---------|-------------|
 | `XDG_CACHE_HOME` | `~/.cache` | Cache directory location |
+| `QMD_LLAMA_GPU` | `auto` | Force llama.cpp GPU backend (`metal`, `vulkan`, `cuda`) or disable GPU with `false` |
+| `QMD_FORCE_CPU` | unset | Set to `1`/`true` to force CPU mode before any CUDA/Vulkan/Metal probing. Equivalent CLI flag: `--no-gpu`. |
+| `QMD_EMBED_PARALLELISM` | automatic | Override embedding/reranking context parallelism (1-8). Windows CUDA defaults to `1` because parallel CUDA contexts can crash with `ggml-cuda.cu:98`; use Vulkan or raise this only if your driver is stable. |
 
 ## How It Works
 

package/bin/qmd

@@ -1,32 +1,111 @@
-#!/bin/sh
-# Resolve symlinks so global installs (npm link / npm install -g) can find the
-# actual package directory instead of the global bin directory.
-SOURCE="$0"
-while [ -L "$SOURCE" ]; do
-  SOURCE_DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
-  TARGET="$(readlink "$SOURCE")"
-  case "$TARGET" in
-    /*) SOURCE="$TARGET" ;;
-    *) SOURCE="$SOURCE_DIR/$TARGET" ;;
-  esac
-done
-
-# Detect the runtime used to install this package and use the matching one
-# to avoid native module ABI mismatches (e.g., better-sqlite3 compiled for bun vs node)
-DIR="$(cd -P "$(dirname "$SOURCE")/.." && pwd)"
-
-# Detect the package manager that installed dependencies by checking lockfiles.
-# $BUN_INSTALL is intentionally NOT checked — it only indicates that bun exists
-# on the system, not that it was used to install this package (see #361).
-#
-# package-lock.json takes priority: if it exists, npm installed the native
-# modules for Node.  The repo ships bun.lock, so without this check, source
-# builds that use npm would be incorrectly routed to bun, causing ABI
-# mismatches with better-sqlite3 / sqlite-vec (see #381).
-if [ -f "$DIR/package-lock.json" ]; then
-  exec node "$DIR/dist/cli/qmd.js" "$@"
-elif [ -f "$DIR/bun.lock" ] || [ -f "$DIR/bun.lockb" ]; then
-  exec bun "$DIR/dist/cli/qmd.js" "$@"
-else
-  exec node "$DIR/dist/cli/qmd.js" "$@"
-fi
+#!/usr/bin/env node
+// 2>/dev/null; if command -v node >/dev/null 2>&1; then exec node "$0" "$@"; else exec bun "$0" "$@"; fi
+// Cross-platform launcher for qmd.
+//
+// Previously this was a POSIX shell script with `#!/bin/sh`, which meant npm
+// on Windows generated shims that tried to route through `/bin/sh` — a path
+// that doesn't exist on Windows, so `qmd` failed immediately after a global
+// install. Rewriting the launcher in Node.js lets npm generate native
+// cmd/ps1/sh shims that invoke `node` directly on every platform.
+
+import { spawn, spawnSync } from "node:child_process";
+import { existsSync, realpathSync } from "node:fs";
+import { dirname, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+
+// Resolve symlinks so global installs (npm link / npm install -g) can find
+// the actual package directory instead of the global bin directory.
+const self = realpathSync(fileURLToPath(import.meta.url));
+const pkgDir = resolve(dirname(self), "..");
+const jsEntry = resolve(pkgDir, "dist/cli/qmd.js");
+const tsEntry = resolve(pkgDir, "src/cli/qmd.ts");
+
+// MCP stdio reserves stdout exclusively for JSON-RPC frames. node-llama-cpp
+// / llama.cpp / ggml can write native logs directly to stdout before JS-level
+// log handlers are attached, so seed the native quiet env before Node/Bun imports
+// the CLI and its LLM modules. Preserve explicit user values when provided.
+if (process.argv[2] === "mcp") {
+  process.env.LLAMA_LOG_LEVEL = process.env.LLAMA_LOG_LEVEL || "error";
+  process.env.GGML_LOG_LEVEL = process.env.GGML_LOG_LEVEL || "error";
+  process.env.GGML_BACKEND_SILENT = process.env.GGML_BACKEND_SILENT || "1";
+}
+
+function hasBun() {
+  try {
+    const res = spawnSync("bun", ["--version"], { stdio: "ignore", shell: process.platform === "win32" });
+    return res.status === 0;
+  } catch {
+    return false;
+  }
+}
+
+// In published packages, bin/qmd must run dist/. In a git checkout, however,
+// dist/ is often ignored and can be stale after git reset or branch switches.
+// Prefer source mode only for checkouts so ./bin/qmd reflects the checked-out
+// source without changing packaged/runtime behavior.
+let useSourceMode = false;
+let sourceRunner = null;
+let sourceArgs = [];
+
+if (existsSync(resolve(pkgDir, ".git")) && existsSync(tsEntry)) {
+  if (existsSync(resolve(pkgDir, "bun.lock")) || existsSync(resolve(pkgDir, "bun.lockb"))) {
+    if (hasBun()) {
+      useSourceMode = true;
+      sourceRunner = "bun";
+      sourceArgs = [tsEntry, ...process.argv.slice(2)];
+    }
+  }
+  if (!useSourceMode && existsSync(resolve(pkgDir, "node_modules/tsx/dist/cli.mjs"))) {
+    useSourceMode = true;
+    sourceRunner = "node";
+    sourceArgs = [resolve(pkgDir, "node_modules/tsx/dist/cli.mjs"), tsEntry, ...process.argv.slice(2)];
+  }
+}
+
+if (!useSourceMode && !existsSync(jsEntry)) {
+  console.error(`qmd is not built: missing ${jsEntry}`);
+  console.error("Run: bun install && bun run build");
+  console.error("Or:  npm install && npm run build");
+  console.error("After building, run: qmd doctor");
+  process.exit(1);
+}
+
+// Detect the package manager that installed dependencies by checking lockfiles.
+// $BUN_INSTALL is intentionally NOT checked — it only indicates that bun exists
+// on the system, not that it was used to install this package (see #361).
+//
+// package-lock.json takes priority: if it exists, npm installed the native
+// modules for Node. The repo ships bun.lock, so without this check, source
+// builds that use npm would be incorrectly routed to bun, causing ABI
+// mismatches with better-sqlite3 / sqlite-vec (see #381).
+let runnerName = "node";
+if (existsSync(resolve(pkgDir, "package-lock.json"))) {
+  runnerName = "node";
+} else if (existsSync(resolve(pkgDir, "bun.lock")) || existsSync(resolve(pkgDir, "bun.lockb"))) {
+  runnerName = "bun";
+} else {
+  runnerName = "node";
+}
+
+const runner = useSourceMode ? sourceRunner : (runnerName === "node" ? "node" : "bun");
+const args = useSourceMode ? sourceArgs : [jsEntry, ...process.argv.slice(2)];
+const needsShell = (runner === "bun") && process.platform === "win32";
+
+const child = spawn(runner, args, {
+  stdio: "inherit",
+  shell: needsShell,
+});
+
+child.on("exit", (code, signal) => {
+  if (signal) {
+    process.kill(process.pid, signal);
+  } else {
+    process.exit(code ?? 0);
+  }
+});
+
+child.on("error", (err) => {
+  const name = useSourceMode ? sourceRunner : runnerName;
+  console.error(`qmd: failed to launch ${name}: ${err.message}`);
+  process.exit(1);
+});

package/dist/ast.d.ts

@@ -24,6 +24,7 @@ export type SupportedLanguage = "typescript" | "tsx" | "javascript" | "python" |
  * Returns null for unsupported or unknown extensions (including .md).
  */
 export declare function detectLanguage(filepath: string): SupportedLanguage | null;
+export declare function formatGrammarLoadError(language: SupportedLanguage, err: unknown): string;
 /**
  * Parse a source file and return break points at AST node boundaries.
  *

package/dist/ast.js

@@ -47,13 +47,19 @@ export function detectLanguage(filepath) {
  * Maps language to the npm package and wasm filename for the grammar.
  */
 const GRAMMAR_MAP = {
-    typescript: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-typescript.wasm" },
-    tsx: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-tsx.wasm" },
-    javascript: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-typescript.wasm" },
-    python: { pkg: "tree-sitter-python", wasm: "tree-sitter-python.wasm" },
-    go: { pkg: "tree-sitter-go", wasm: "tree-sitter-go.wasm" },
-    rust: { pkg: "tree-sitter-rust", wasm: "tree-sitter-rust.wasm" },
+    typescript: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-typescript.wasm", version: "0.23.2" },
+    tsx: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-tsx.wasm", version: "0.23.2" },
+    javascript: { pkg: "tree-sitter-typescript", wasm: "tree-sitter-typescript.wasm", version: "0.23.2" },
+    python: { pkg: "tree-sitter-python", wasm: "tree-sitter-python.wasm", version: "0.23.4" },
+    go: { pkg: "tree-sitter-go", wasm: "tree-sitter-go.wasm", version: "0.23.4" },
+    rust: { pkg: "tree-sitter-rust", wasm: "tree-sitter-rust.wasm", version: "0.24.0" },
 };
+export function formatGrammarLoadError(language, err) {
+    const grammar = GRAMMAR_MAP[language];
+    const detail = err instanceof Error ? err.message : String(err);
+    return `${grammar.pkg}/${grammar.wasm} failed to load (${detail}); falling back to regex chunking. ` +
+        `Repair a broken global install with: bun add ${grammar.pkg}@${grammar.version}`;
+}
 // =============================================================================
 // Per-Language Query Definitions
 // =============================================================================
@@ -152,6 +158,8 @@ let QueryClass = null;
 let initPromise = null;
 /** Languages that have already failed to load — warn only once per process. */
 const failedLanguages = new Set();
+/** Last grammar load error by language, for status output. */
+const grammarLoadErrors = new Map();
 /** Cached grammar load promises. */
 const grammarCache = new Map();
 /** Cached compiled queries per language. */
@@ -200,7 +208,9 @@ async function loadGrammar(language) {
     catch (err) {
         failedLanguages.add(language);
         grammarCache.delete(wasmKey);
-        console.warn(`[qmd] Failed to load tree-sitter grammar for ${language}: ${err}`);
+        const message = formatGrammarLoadError(language, err);
+        grammarLoadErrors.set(language, message);
+        console.warn(`[qmd] AST grammar unavailable for ${language}: ${message}`);
         return null;
     }
 }
@@ -299,7 +309,7 @@ export async function getASTStatus() {
                 languages.push({ language: lang, available: true });
             }
             else {
-                languages.push({ language: lang, available: false, error: "grammar failed to load" });
+                languages.push({ language: lang, available: false, error: grammarLoadErrors.get(lang) ?? "grammar failed to load" });
             }
         }
         catch (err) {

package/dist/bench/bench.d.ts

@@ -18,4 +18,6 @@ export declare function runBenchmark(fixturePath: string, options?: {
     json?: boolean;
     collection?: string;
     backends?: string[];
+    dbPath?: string;
+    configPath?: string;
 }): Promise<BenchmarkResult>;

package/dist/bench/bench.js

@@ -17,32 +17,113 @@ import { readFileSync } from "node:fs";
 import { resolve } from "node:path";
 import { createStore, getDefaultDbPath, } from "../index.js";
 import { scoreResults } from "./score.js";
+function parseStructuredQuery(query) {
+    const lines = query.split("\n").map((line, idx) => ({
+        trimmed: line.trim(),
+        number: idx + 1,
+    })).filter(line => line.trimmed.length > 0);
+    if (lines.length === 0)
+        return undefined;
+    const prefixRe = /^(lex|vec|hyde):\s*/i;
+    const intentRe = /^intent:\s*/i;
+    const searches = [];
+    let intent;
+    for (const line of lines) {
+        if (intentRe.test(line.trimmed)) {
+            if (intent !== undefined) {
+                throw new Error(`Line ${line.number}: only one intent: line is allowed per benchmark query.`);
+            }
+            intent = line.trimmed.replace(intentRe, "").trim();
+            if (!intent) {
+                throw new Error(`Line ${line.number}: intent: must include text.`);
+            }
+            continue;
+        }
+        const match = line.trimmed.match(prefixRe);
+        if (match) {
+            const type = match[1].toLowerCase();
+            const text = line.trimmed.slice(match[0].length).trim();
+            if (!text) {
+                throw new Error(`Line ${line.number} (${type}:) must include text.`);
+            }
+            searches.push({ type, query: text, line: line.number });
+            continue;
+        }
+        if (lines.length === 1) {
+            return undefined;
+        }
+        throw new Error(`Line ${line.number} is missing a lex:/vec:/hyde:/intent: prefix.`);
+    }
+    if (intent && searches.length === 0) {
+        throw new Error("intent: cannot appear alone. Add at least one lex:, vec:, or hyde: line.");
+    }
+    return searches.length > 0 ? { searches, intent } : undefined;
+}
+function uniqueFiles(files, limit) {
+    const seen = new Set();
+    const out = [];
+    for (const file of files) {
+        if (seen.has(file))
+            continue;
+        seen.add(file);
+        out.push(file);
+        if (out.length >= limit)
+            break;
+    }
+    return out;
+}
 const BACKENDS = [
     {
         name: "bm25",
         run: async (store, query, limit, collection) => {
-            const results = await store.searchLex(query, { limit, collection });
+            const structured = parseStructuredQuery(query.query);
+            const lexQueries = structured?.searches.filter(q => q.type === "lex");
+            if (structured) {
+                const files = [];
+                for (const lex of lexQueries ?? []) {
+                    const results = await store.searchLex(lex.query, { limit, collection });
+                    files.push(...results.map((r) => r.filepath));
+                }
+                return uniqueFiles(files, limit);
+            }
+            const results = await store.searchLex(query.query, { limit, collection });
             return results.map((r) => r.filepath);
         },
     },
     {
         name: "vector",
         run: async (store, query, limit, collection) => {
-            const results = await store.searchVector(query, { limit, collection });
+            const structured = parseStructuredQuery(query.query);
+            const vectorQueries = structured?.searches.filter(q => q.type === "vec" || q.type === "hyde");
+            if (structured) {
+                const files = [];
+                for (const vectorQuery of vectorQueries ?? []) {
+                    const results = await store.searchVector(vectorQuery.query, { limit, collection });
+                    files.push(...results.map((r) => r.filepath));
+                }
+                return uniqueFiles(files, limit);
+            }
+            const results = await store.searchVector(query.query, { limit, collection });
             return results.map((r) => r.filepath);
         },
     },
     {
         name: "hybrid",
         run: async (store, query, limit, collection) => {
-            const results = await store.search({ query, limit, collection, rerank: false });
+            const structured = parseStructuredQuery(query.query);
+            const results = structured
+                ? await store.search({ queries: structured.searches, intent: structured.intent, limit, collection, rerank: false })
+                : await store.search({ query: query.query, limit, collection, rerank: false });
             return results.map((r) => r.file);
         },
     },
     {
         name: "full",
         run: async (store, query, limit, collection) => {
-            const results = await store.search({ query, limit, collection, rerank: true });
+            const structured = parseStructuredQuery(query.query);
+            const results = structured
+                ? await store.search({ queries: structured.searches, intent: structured.intent, limit, collection, rerank: true })
+                : await store.search({ query: query.query, limit, collection, rerank: true });
             return results.map((r) => r.file);
         },
     },
@@ -52,19 +133,24 @@ async function runQuery(store, backend, query, collection) {
     const start = Date.now();
     let resultFiles;
     try {
-        resultFiles = await backend.run(store, query.query, limit, collection);
+        resultFiles = await backend.run(store, query, limit, collection);
     }
-    catch (err) {
+    catch {
         // Backend may not be available (e.g., no embeddings for vector search)
         return {
             precision_at_k: 0,
             recall: 0,
+            recall_at_1: 0,
+            recall_at_3: 0,
+            recall_at_5: 0,
             mrr: 0,
             f1: 0,
             hits_at_k: 0,
             total_expected: query.expected_files.length,
             latency_ms: Date.now() - start,
             top_files: [],
+            matched_files: [],
+            unmatched_expected_files: query.expected_files,
         };
     }
     const latency_ms = Date.now() - start;
@@ -80,11 +166,11 @@ function formatTable(results) {
     const lines = [];
     const pad = (s, n) => s.slice(0, n).padEnd(n);
     const num = (n) => n.toFixed(2).padStart(5);
-    lines.push(`${pad("Query", 25)} ${pad("Backend", 8)} ${pad("P@k", 6)} ${pad("Recall", 7)} ${pad("MRR", 6)} ${pad("F1", 6)} ${pad("ms", 8)}`);
-    lines.push("-".repeat(70));
+    lines.push(`${pad("Query", 25)} ${pad("Backend", 8)} ${pad("P@k", 6)} ${pad("R@1", 6)} ${pad("R@3", 6)} ${pad("R@5", 6)} ${pad("MRR", 6)} ${pad("F1", 6)} ${pad("ms", 8)}`);
+    lines.push("-".repeat(88));
     for (const r of results) {
         for (const [backend, br] of Object.entries(r.backends)) {
-            lines.push(`${pad(r.id, 25)} ${pad(backend, 8)} ${num(br.precision_at_k)} ${num(br.recall)}  ${num(br.mrr)} ${num(br.f1)} ${String(Math.round(br.latency_ms)).padStart(7)}ms`);
+            lines.push(`${pad(r.id, 25)} ${pad(backend, 8)} ${num(br.precision_at_k)} ${num(br.recall_at_1)} ${num(br.recall_at_3)} ${num(br.recall_at_5)} ${num(br.mrr)} ${num(br.f1)} ${String(Math.round(br.latency_ms)).padStart(7)}ms`);
         }
         lines.push("");
     }
@@ -99,14 +185,17 @@ function computeSummary(results) {
             backendNames.add(name);
         }
     }
-    for (const name of backendNames) {
-        let totalP = 0, totalR = 0, totalMrr = 0, totalF1 = 0, totalLat = 0, count = 0;
+    for (const name of Array.from(backendNames)) {
+        let totalP = 0, totalR = 0, totalR1 = 0, totalR3 = 0, totalR5 = 0, totalMrr = 0, totalF1 = 0, totalLat = 0, count = 0;
         for (const r of results) {
             const br = r.backends[name];
             if (!br)
                 continue;
             totalP += br.precision_at_k;
             totalR += br.recall;
+            totalR1 += br.recall_at_1;
+            totalR3 += br.recall_at_3;
+            totalR5 += br.recall_at_5;
             totalMrr += br.mrr;
             totalF1 += br.f1;
             totalLat += br.latency_ms;
@@ -116,6 +205,9 @@ function computeSummary(results) {
             summary[name] = {
                 avg_precision: totalP / count,
                 avg_recall: totalR / count,
+                avg_recall_at_1: totalR1 / count,
+                avg_recall_at_3: totalR3 / count,
+                avg_recall_at_5: totalR5 / count,
                 avg_mrr: totalMrr / count,
                 avg_f1: totalF1 / count,
                 avg_latency_ms: totalLat / count,
@@ -132,7 +224,10 @@ export async function runBenchmark(fixturePath, options = {}) {
         throw new Error("Invalid fixture: missing 'queries' array");
     }
     // Open store
-    const store = await createStore({ dbPath: getDefaultDbPath() });
+    const store = await createStore({
+        dbPath: options.dbPath ?? getDefaultDbPath(),
+        ...(options.configPath ? { configPath: options.configPath } : {}),
+    });
     // Filter backends if requested
     const activeBackends = options.backends
         ? BACKENDS.filter(b => options.backends.includes(b.name))
@@ -178,7 +273,7 @@ export async function runBenchmark(fixturePath, options = {}) {
         const pad = (s, n) => s.slice(0, n).padEnd(n);
         const num = (n) => n.toFixed(3).padStart(6);
         for (const [name, s] of Object.entries(summary)) {
-            console.log(`  ${pad(name, 8)} P@k=${num(s.avg_precision)} Recall=${num(s.avg_recall)} MRR=${num(s.avg_mrr)} F1=${num(s.avg_f1)} Avg=${Math.round(s.avg_latency_ms)}ms`);
+            console.log(`  ${pad(name, 8)} P@k=${num(s.avg_precision)} R@1=${num(s.avg_recall_at_1)} R@3=${num(s.avg_recall_at_3)} R@5=${num(s.avg_recall_at_5)} MRR=${num(s.avg_mrr)} F1=${num(s.avg_f1)} Avg=${Math.round(s.avg_latency_ms)}ms`);
         }
     }
     return benchResult;

package/dist/bench/score.d.ts

@@ -14,13 +14,20 @@ export declare function normalizePath(p: string): string;
  * Handles different path formats by comparing normalized suffixes.
  */
 export declare function pathsMatch(result: string, expected: string): boolean;
-/**
- * Score a set of search results against expected files.
- */
-export declare function scoreResults(resultFiles: string[], expectedFiles: string[], topK: number): {
+type ScoreMetrics = {
     precision_at_k: number;
     recall: number;
+    recall_at_1: number;
+    recall_at_3: number;
+    recall_at_5: number;
     mrr: number;
     f1: number;
     hits_at_k: number;
+    matched_files: string[];
+    unmatched_expected_files: string[];
 };
+/**
+ * Score a set of search results against expected files.
+ */
+export declare function scoreResults(resultFiles: string[], expectedFiles: string[], topK: number): ScoreMetrics;
+export {};

package/dist/bench/score.js

@@ -10,7 +10,7 @@
  */
 export function normalizePath(p) {
     if (p.startsWith("qmd://")) {
-        // qmd://collection/path/to/file → path/to/file
+        // qmd://collection/docs/readme.md → docs/readme.md
         const withoutScheme = p.slice("qmd://".length);
         const slashIdx = withoutScheme.indexOf("/");
         p = slashIdx >= 0 ? withoutScheme.slice(slashIdx + 1) : withoutScheme;
@@ -30,23 +30,30 @@ export function pathsMatch(result, expected) {
         return true;
     return false;
 }
+function hitsWithin(resultFiles, expectedFiles, k) {
+    const topKResults = resultFiles.slice(0, k);
+    let hits = 0;
+    for (const expected of expectedFiles) {
+        if (topKResults.some(r => pathsMatch(r, expected))) {
+            hits++;
+        }
+    }
+    return hits;
+}
 /**
  * Score a set of search results against expected files.
  */
 export function scoreResults(resultFiles, expectedFiles, topK) {
     // Count hits in top-k
-    const topKResults = resultFiles.slice(0, topK);
-    let hitsAtK = 0;
-    for (const expected of expectedFiles) {
-        if (topKResults.some(r => pathsMatch(r, expected))) {
-            hitsAtK++;
-        }
-    }
-    // Count total hits anywhere
-    let totalHits = 0;
+    const hitsAtK = hitsWithin(resultFiles, expectedFiles, topK);
+    const matchedFiles = [];
+    const unmatchedExpectedFiles = [];
     for (const expected of expectedFiles) {
         if (resultFiles.some(r => pathsMatch(r, expected))) {
-            totalHits++;
+            matchedFiles.push(expected);
+        }
+        else {
+            unmatchedExpectedFiles.push(expected);
         }
     }
     // MRR: reciprocal rank of first relevant result
@@ -59,9 +66,23 @@ export function scoreResults(resultFiles, expectedFiles, topK) {
     }
     const denominator = Math.min(topK, expectedFiles.length);
     const precision_at_k = denominator > 0 ? hitsAtK / denominator : 0;
-    const recall = expectedFiles.length > 0 ? totalHits / expectedFiles.length : 0;
+    const recall = expectedFiles.length > 0 ? matchedFiles.length / expectedFiles.length : 0;
+    const recall_at_1 = expectedFiles.length > 0 ? hitsWithin(resultFiles, expectedFiles, 1) / expectedFiles.length : 0;
+    const recall_at_3 = expectedFiles.length > 0 ? hitsWithin(resultFiles, expectedFiles, 3) / expectedFiles.length : 0;
+    const recall_at_5 = expectedFiles.length > 0 ? hitsWithin(resultFiles, expectedFiles, 5) / expectedFiles.length : 0;
     const f1 = precision_at_k + recall > 0
         ? 2 * (precision_at_k * recall) / (precision_at_k + recall)
         : 0;
-    return { precision_at_k, recall, mrr, f1, hits_at_k: hitsAtK };
+    return {
+        precision_at_k,
+        recall,
+        recall_at_1,
+        recall_at_3,
+        recall_at_5,
+        mrr,
+        f1,
+        hits_at_k: hitsAtK,
+        matched_files: matchedFiles,
+        unmatched_expected_files: unmatchedExpectedFiles,
+    };
 }