grepmax 0.26.70 → 0.26.72

package/.claude-plugin/marketplace.json

@@ -10,7 +10,7 @@
       "name": "grepmax",
       "source": "./plugins/grepmax",
       "description": "Semantic code search for Claude Code. Automatically indexes your project and provides intelligent search capabilities.",
-      "version": "0.26.70",
+      "version": "0.26.72",
       "author": {
         "name": "Robert Owens",
         "email": "robowens@me.com"

package/DOCUMENT-SEARCH.md

@@ -0,0 +1,134 @@
+# Existing-index document search (contract v1)
+
+`gmax mcp --existing-index-only` serves two read-only stdio MCP tools. The exact
+entry dispatch precedes the normal CLI. Server identity is
+`gmax-document-search`; the 2025-06-18 initialize exchange is supported by the
+current MCP SDK. Contract version is independent of package version.
+
+Every tool returns `structuredContent` with `contractVersion: 1` and capabilities
+`{existingIndexOnly: 1, queryLogging: false, watch: false, runtimeStartup: false}`.
+The text content contains only the result state. No source/snippet bodies are
+returned. Runlist has no gmax, model or SDK runtime dependency.
+
+- `document_search_status({paths})` returns `state: ready`, a required positive
+  safe-integer `generation`, `project: {root,store,lastIndexed}`,
+  actual metadata `covered` paths, `coverage: {requested,indexed,partial}`,
+  `embeddingReady`, `queryState` and index state. Readiness does not guarantee
+  worker availability when the subsequent query arrives. Coverage is not a
+  registry/store-wide chunk count and does not prove current source freshness.
+  Covered paths retain a validated `indexedMtimeMs` where available. A missing
+  sampled Markdown file reduces coverage instead of discarding other paths;
+  malformed or out-of-scope paths still refuse. Counts are recomputed, and
+  metadata uses a bounded field allowlist without free-form diagnostics.
+- `semantic_search({query,prefixes})` returns `state: ready`, canonical `root`,
+  selected `store`, required `generation`, `matches` and index state. Each pointer is
+  `{path,startLine,endLine,score,hash?,hashAlgorithm?}`. Ranges are one-based full
+  source ranges, converted from indexed zero-based lines. Score is
+  `1/(1+max(0,cosineDistance))`, not a relevance probability.
+- `hashAlgorithm: sha256-bytes` is declared only when the vector row digest agrees
+  with current version-1 metadata whose digest hashes the exact indexed bytes.
+  Legacy/mismatched provenance retains document-only pointers. Consumers must
+  reread and authorize current source, verify the digest and map body lines before
+  jumping to a section. This bridge never hashes new bytes to bless old lines.
+
+Limits: 500 query characters, 32 explicit prefixes, 2,000 coverage paths and a
+512 KiB serialized path budget, 50 vector candidates, 1 MiB transport frames and
+a 10-second bridge operation deadline. The most specific registered ancestor
+of the canonical checkout selects the primary store; registered spelling is
+preserved for daemon/row lookups. Coverage and prefixes must resolve inside the
+checkout and selected project. Secondary stores are unavailable. No candidate
+refill broadens scope. Symlink targets are rechecked before pointers are returned.
+
+Unavailable states: `unsupported_tool`, `unsupported_daemon`, `no_index`,
+`index_unavailable`, `daemon_unavailable`, `store_unavailable`,
+`embedding_unavailable`, `embedding_mismatch`, `host_pressure`, `busy`,
+`cancelled`, `timeout`, `no_coverage`, `search_unavailable`. Envelopes contain
+stable states, never raw query-bearing diagnostics. Schema/unknown-tool errors
+remain standard MCP errors.
+
+The supported bridge negotiates daemon `existingIndexOnlySearch: 1` and resource
+generation. Missing, null, non-integer and changed generations refuse; Runlist
+also fences status against subsequent retrieval. Private daemon commands are `documents.status` and `documents.search`;
+Runlist does not call the socket API. The daemon validates roots, stores, bounds
+and generation again under its shared-operation coordinator, including after
+encoding and native retrieval. Paused/contained resources refuse rather than
+return keyword results.
+
+Retrieval in this first slice is **dense-only** against an already-open LanceDB
+connection/table. It never invokes normal hybrid search, FTS creation, reranking,
+ColBERT inference, `ensureTable`, schema evolution, index mutation or local-store
+fallback. Query encoding uses a currently idle, already-warm compatible worker:
+CPU requires a loaded Granite tokenizer/session; GPU requires cached positive
+confirmation of the existing configured MLX model. Idle time alone does not revoke
+that identity: each bounded `/embed` response revalidates model and dimensions,
+and successful inference renews the normal health cache. Failure revokes restricted
+eligibility until external work confirms readiness. A worker rechecks before inference;
+no initialization, health polling, retry or backend fallback is allowed. Cold
+workers refuse immediately. When compatible warm workers are occupied, at most
+four document queries may wait FIFO for up to two seconds for a warm worker.
+They get a turn between indexing files; after four consecutive document-query
+assignments, waiting ordinary work gets a turn. A full queue or expired worker
+wait returns `busy`. `queryState: ready` describes admission availability, including
+this bounded wait, and does not guarantee immediate execution or query success.
+Waiting requests live outside the ordinary worker queues and cannot trigger
+worker startup, replacement or scaling. Existing ordinary work retains its own
+admitted lifecycle, including independent expansion/replacement.
+
+Admission and warm readiness are rechecked when a worker becomes available.
+Queued cancellation/deadlines withdraw the request; cancellation after dispatch
+keeps the shared worker assigned until its actual terminal response or exit.
+The ten-second request bound includes waiting rather than restarting at dispatch.
+Synchronous admission has its own probe bound and is checked against the absolute
+request deadline before any query is sent. Native retrieval still uses the
+remaining overall deadline. An in-flight indexing file is never interrupted;
+long files or backend outages can still exceed the worker wait and return `busy`.
+
+Read-only admission honors the configured host guard policy without creating locks,
+reservations, pruning records or changing containment markers. Under the default
+`strict` policy, existing Darwin inference requires fresh known-normal aggregate
+resources and reads the existing ledger; absent ledgers are treated as empty and
+corrupt or symlinked ledgers refuse. Explicit `critical-only` uses fresh OS/kernel
+pressure probes without aggregate sampling or ledger access: warning/unknown
+measurements remain diagnostic, while confirmed critical pressure refuses.
+Containment is checked before and after sampling in either policy. This read-only
+path never creates a safety latch; the daemon's safety machinery remains authoritative.
+Other platforms retain the existing no-footprint-probe policy; this is not a
+cross-platform native qualification claim. Markdown coverage
+and returned pointers require regular files, including after symlink resolution;
+a directory whose name ends in `.md` can be a prefix but never a source pointer.
+
+Cancellation closes only the request/owned bridge. It rejects the caller and
+discards late results; an already executing shared worker remains assigned until
+its terminal response or exit. Caller deadlines/heartbeats cannot trigger worker
+replacement or native session initialization. Idle RSS retirement and consecutive
+oversized completions remain enforced, including after restricted work, without
+filling the worker floor on that path. Further restricted calls then report
+unavailable; ordinary work may expand under ordinary admission. Busy inference is
+never interrupted by RSS retirement. The daemon's independent safety/shutdown
+machinery remains authoritative.
+
+Native retrieval receives the remaining overall deadline and holds its shared
+operation until the actual native promise settles. A JavaScript timer cannot
+release resources that native code still uses. Caller cancellation/bridge timeout
+remains bounded and late results are discarded. If the SDK fails to settle even
+after its native deadline, exclusive resource teardown stays blocked rather than
+closing underneath the read; no worker/store replacement is induced to hide the
+failure. Cancellation cannot interrupt an already running ONNX native call.
+Global logging settings are unchanged, and this path never logs query text.
+
+## Isolated qualification
+
+Use `node scripts/build-document-search-candidate.cjs /absolute/new/output` to
+compile/package without overwriting `dist`, installing, publishing, or invoking
+lifecycle scripts. The script copies the pinned native JS runtime with provenance.
+`node scripts/audit-document-search.cjs /absolute/extracted/package` exercises a
+synthetic daemon and checks packaged files/import boundaries. The normal consumer
+audit invokes that check against its freshly installed tarball too.
+
+The optional Runlist fixture is
+`RUNLIST_GMAX_QUALIFICATION_ENTRY=/absolute/extracted/package/dist/bin.js node --test desktop/test/semantic-gmax-contract.test.mjs`.
+Fixture homes/IPC are private temporary directories; no real daemon, index,
+watcher, model or grammar download is used. Reusing checkout dependencies for a
+local package probe must be reported separately from fresh dependency resolution.
+Installation, a shared-daemon restart, watcher recovery and live retrieval require
+separate explicit actions. Source/package fixture success proves none of those.

package/README.md

@@ -233,6 +233,33 @@ Architecture audits omit unresolved same-name definitions from rankings, cycles
 
 CLI `gmax skeleton --agent` and MCP `code_skeleton` text show at most 120 lines / 6,000 characters per file and report omitted lines and characters. CLI human output and JSON structure remain complete.
 
+## Existing-index document search
+
+`gmax mcp --existing-index-only` exposes a separate read-only stdio MCP contract
+for Markdown document coverage and semantic pointers. It uses an existing daemon,
+index and already-warm compatible query worker. It cannot start a daemon, watch a
+project, initialize a model, fall back to another backend or mutate the index.
+Cold, busy or unavailable resources return an explicit unavailable state.
+
+Document query admission follows the configured host guard policy. On macOS,
+default `strict` requires fresh aggregate resource admission; explicit
+`critical-only` checks OS/kernel critical pressure without an aggregate scan or
+ledger access. Warning or unavailable measurements stay diagnostic in that mode.
+Both policies preserve containment checks. Up to four document queries can wait
+FIFO for an already-warm compatible worker for up to two seconds, taking a turn
+between indexing files. After four consecutive document queries, waiting ordinary
+work gets a turn. Saturation or wait expiry returns `busy`; cold workers still
+refuse. This wait cannot start, replace or scale workers, and it never interrupts
+an in-flight file. Admission/readiness are checked again before dispatch, and the
+overall ten-second deadline includes waiting. Readiness means a request can be
+admitted, including this bounded wait. Long indexing files can still return `busy`.
+
+Results contain canonical paths and one-based source ranges, without cached source
+bodies. The first slice uses dense retrieval with explicit path scope and bounded
+deadlines. Consumers must reread and authorize current source before using a
+pointer. See [the document-search contract](DOCUMENT-SEARCH.md) for tool schemas,
+limits, hash provenance and isolated qualification.
+
 ## Background Daemon
 
 A single daemon watches your projects via native OS file events (FSEvents/inotify). Changes are detected in sub-second and incrementally reindexed. All writes to LanceDB are routed through the daemon via IPC, eliminating lock contention.
@@ -580,6 +607,75 @@ The candidate trace contains pointers and stage ranks for at most the first 200
 
 Each traced candidate also has an `outcome`: returned, stage-one/pooled cut, deduplicated, per-file limit or display limit. Display limit means the production loop filled its result window before examining that candidate; it does not assert that the candidate would pass the per-file cap.
 
+#### Existing-index document retrieval evaluation
+
+In a source checkout, [scripts/eval-documents.cjs](scripts/eval-documents.cjs) measures the released [document MCP contract](DOCUMENT-SEARCH.md) through an explicit installed entry point. It uses only Node built-ins and sequential `document_search_status` / `semantic_search` calls. It performs no model warmup, daemon startup, indexing, refusal retries or fallback to normal hybrid search.
+
+```bash
+node scripts/eval-documents.cjs \
+  --fixture docs/measurements/documents/fixture-v1.json \
+  --sha256 <reviewed-fixture-digest> \
+  --root /absolute/already-indexed/project \
+  --entry /absolute/installed/grepmax/dist/bin.js \
+  --output docs/measurements/documents/baseline-v1.json
+```
+
+Freeze the JSON fixture before retrieving results. It contains `schemaVersion: 1`, `purpose`, explicit project-relative `prefixes`, and 1–100 `cases`. Each case has a unique `id`, a `query` (maximum 500 characters), and one or more `expected` targets: `{file, startLine, endLine, sourceSha256}`. Files are contained relative Markdown paths; ranges are one-based and answer-bearing; hashes cover exact source bytes. Keep private fixtures and evidence in ignored `docs/measurements/`. Optional `--repeats 1-3` repeats the same frozen cases without retries or warmup.
+
+The report separates the rank of the expected document among distinct returned documents from the rank of an overlapping expected section among chunk pointers. Document Recall@10 credits the fraction of declared target documents retrieved; MRR@10 credits only ranks 1–10. Section overlap indicates source-context discovery, not complete answer coverage. Only the first 50 returned candidates are available; document deduplication cannot recover targets outside that window. Reviewed targets are not exhaustive judgments of all useful documents, so these metrics describe the frozen targets, not general recall.
+
+Source and indexed hashes must match the fixture before querying; generation and project/store identity must agree across coverage and retrieval. Returned source ranges require reread/hash verification; new artifacts retain both the indexed pointer digest and current-source digest so later reviews can check provenance. Refusals, changed targets and unverified expected pointers are excluded from relevance metrics and counted separately; `usableFraction` includes every planned sample. Background indexing and partial/degraded index context are recorded without excluding targets whose coverage and hashes remain current. No queries, source bodies or raw diagnostics are copied into result artifacts. Reads are bounded to 2 MiB per source; MCP frames to 1 MiB. The report and sibling `.samples.jsonl` checkpoint refuse overwrite and use owner-only permissions. Exit 0 means an evaluation with usable samples completed, not that a quality threshold passed; exit 2 means no usable samples; exit 1 means preflight or transport/contract failure. No consumer rollout is implied.
+
+To investigate scope competition, [scripts/eval-document-scopes.cjs](scripts/eval-document-scopes.cjs) compares the original fixture prefixes against declared narrower directories without changing queries or targets:
+
+```bash
+node scripts/eval-document-scopes.cjs \
+  --fixture docs/measurements/documents/fixture-v1.json --sha256 <digest> \
+  --root /absolute/already-indexed/project \
+  --entry /absolute/installed/grepmax/dist/bin.js \
+  --output docs/measurements/documents/paired-scope-v1.json \
+  --narrow-prefix docs/services --narrow-prefix docs/modules
+```
+
+Declare and freeze the scope hypothesis before querying. Narrower prefixes must stay within the original scope and retain every target. Arm order alternates per case, with a one-second gap between pairs and no retries. Reports show each arm's standalone observations and separately compare only pairs where both arms are usable under the same resource generation, project and store. The live competing corpus can still change; this is a diagnostic comparison rather than an isolated causal test. Scopes chosen from known target families introduce a useful prior, so gains do not establish unscoped or held-out retrieval quality. Owner-only checkpoints, overwrite refusal and source/transport bounds are shared with the base evaluator. Exit 2 means no comparable pairs.
+
+### Offline lexical candidate experiment
+
+[scripts/eval-document-lexical.cjs](scripts/eval-document-lexical.cjs) tests lexical candidates and fixed document reciprocal-rank fusion against a prior paired scope report. It uses Node built-ins and filesystem Markdown only; it never contacts a daemon, opens a native store, loads a model or changes production retrieval.
+
+Freeze an owner-only study plan before scoring, including the original fixture checksum, archived report checksum, canonical root, narrower relative prefixes, fixed algorithm parameters and resource limits. The selected scope must retain every target and match the archived narrow arm. Pass that plan's checksum explicitly:
+
+```bash
+NODE_OPTIONS=--max-old-space-size=384 node scripts/eval-document-lexical.cjs \
+  --fixture docs/measurements/documents/fixture-v1.json \
+  --plan docs/measurements/documents/lexical-plan.json --plan-sha256 <digest> \
+  --archive docs/measurements/documents/paired-scope-v1.json \
+  --snapshot docs/measurements/documents/new-private-snapshot \
+  --output docs/measurements/documents/new-lexical-report.json
+```
+
+The prototype uses fixed 48-line windows with eight-line overlap, BM25 `k1=1.2` / `b=0.75`, 50 positive-score lexical candidates and equal-weight document RRF with constant 60. Targets are used for grading only. Stable path/line ties, no per-case overrides and no zero-score padding keep the experiment reproducible. Bounds are 512 files, 16 MiB total source, 2 MiB per file, 10,000 windows, one million token occurrences and 20,000 directory entries. Missing, aliased, changed or oversized source fails snapshot creation rather than silently dropping competing documents. A private snapshot preserves exact source bytes and a digest manifest; the report stores pointers, digests, IDs and scores without questions or source bodies. Existing output/snapshot paths are refused.
+
+Lexical metrics cover cases whose frozen targets match the snapshot. Archive replay additionally requires original ready state/resource identity and every competitor pointer's indexed/current digest and range to match the snapshot; an invalid competitor excludes the entire case. Refused archive cases stay separate. Offline lexical availability does not exercise deployed busy/pressure admission. Archived dense candidates are not re-encoded over this snapshot, and narrower families chosen from known target locations introduce a prior. Report document gains/losses on the verified shared replay cohort; section overlap is diagnostic because window sizes and hybrid representatives differ from production chunks. A passing exploratory gate only supports proposing further matched qualification. No production integration or rollout follows automatically. Exit 0 means scoring completed with usable lexical cases, even when the gate fails; exit 2 means no usable lexical cases, and exit 1 means preflight or evaluation failure.
+
+### Protected document fusion qualification
+
+[scripts/eval-document-fusion.cjs](scripts/eval-document-fusion.cjs) qualifies a fixed candidate ordering in two modes: replay cached verified lexical/dense candidates, or collect one broader pass through the released existing-only MCP. It interleaves each arm's first three distinct documents (`lex1,dense1,lex2,dense2,lex3,dense3`), deduplicates while preserving the selected arm's pointer, then fills to 50 with the fixed RRF tail. This protects high-ranked lexical-only evidence; spending up to six slots can displace useful lower-ranked dense candidates. Expected targets never affect ranking.
+
+```bash
+NODE_OPTIONS=--max-old-space-size=384 node scripts/eval-document-fusion.cjs \
+  --mode live --design <frozen-design.json> --design-sha256 <digest> \
+  --fixture <frozen-fixture.json> --fixture-sha256 <digest> \
+  --entry /absolute/path/to/installed/grepmax/dist/bin.js --entry-sha256 <digest> \
+  --snapshot <new-private-snapshot-directory> --output <new-private-report.json>
+```
+
+For `--mode replay`, replace the entry arguments with `--input <prior-lexical-report.json> --input-sha256 <digest>` and point `--snapshot` at its immutable snapshot. Replay uses saved lexical candidates without rescoring queries. Live mode snapshots the complete predefined scope and builds the bounded filesystem index before MCP initialization; it permits one case-order pass with one-second gaps, without retries, warmup, runtime startup or native fallback. The existing-only entry must expose the expected restricted capabilities. Coverage, generation, project/store identity and every indexed/current competitor digest and range must verify. Refusals remain separate from relevance. A final full source-byte/file-set fence invalidates comparisons if the corpus changed; checkpoints are provisional and the final JSON is authoritative.
+
+The lexical CLI retains its one-million-token default. A frozen fusion design may explicitly allow 1,100,000 token occurrences; all other source/window limits and the 384 MiB Node heap setting remain unchanged. The heap setting is not an operating-system physical memory cap. Outputs and snapshots refuse overwrite and use private permissions; reports exclude question text and source bodies. The broader study requires at least 16 of 20 comparable cases, no dense top-ten target losses, and protected document recall at least as high as either arm on the same cohort. Other predefined positive cohort bounds are supported. Exit 0 means collection completed with some comparable cases, even if the gate fails; exit 2 means none, and exit 1 means interrupted/preflight failure.
+
+The October 8 experiment recovered both original residuals: Redis rank 1, People rank 3, 15/15 replay document hits with no dense top-ten target loss. Its broader 393-file, 20-case pass produced only two comparable cases (15 busy and three pressure refusals; one busy case also lacked indexed coverage), below the frozen 16-case minimum. Broader qualification is **insufficient**. Questions were source-curated after the rule freeze and held out from parameter selection, not independently adjudicated real-user questions. Representative/window differences prevent like-for-like section accuracy claims. This evaluation does not change production retrieval or authorize integration, release or rollout.
+
 ## Attribution
 
 grepmax is built upon the foundation of [mgrep](https://github.com/mixedbread-ai/mgrep) by MixedBread. See the [NOTICE](NOTICE) file for details.

package/dist/bin.js

@@ -152,6 +152,15 @@ function seedConfig(home) {
     }
 }
 if (require.main === module) {
-    selectHome();
-    require("./index");
+    if (process.argv.slice(2).join(" ") === "mcp --existing-index-only") {
+        require("./lib/mcp/document-search").runDocumentSearchBridge();
+    }
+    else if (process.argv.includes("--existing-index-only")) {
+        process.stderr.write("gmax: use exactly mcp --existing-index-only\n");
+        process.exitCode = 2;
+    }
+    else {
+        selectHome();
+        require("./index");
+    }
 }

package/dist/lib/daemon/daemon.js

@@ -84,6 +84,7 @@ const root_availability_1 = require("../utils/root-availability");
 const safety_latch_1 = require("../utils/safety-latch");
 const watcher_store_1 = require("../utils/watcher-store");
 const pool_1 = require("../workers/pool");
+const document_search_handler_1 = require("./document-search-handler");
 const ipc_handler_1 = require("./ipc-handler");
 const mlx_server_manager_1 = require("./mlx-server-manager");
 const paused_reads_1 = require("./paused-reads");
@@ -1021,6 +1022,55 @@ class Daemon {
             ? { queue: progress.queue }
             : {})), { failedFiles: Math.max((_d = progress === null || progress === void 0 ? void 0 : progress.failedFiles) !== null && _d !== void 0 ? _d : 0, (_e = health.failedFiles) !== null && _e !== void 0 ? _e : 0) });
     }
+    /** Separate shared-operation path: never paused keyword fallback or runtime setup. */
+    documentSearch(cmd, signal) {
+        return __awaiter(this, void 0, void 0, function* () {
+            const state = () => this.pausedReason !== null || (0, autostart_1.daemonStartDeniedReason)()
+                ? "host_pressure"
+                : this.shuttingDown || this.recycling
+                    ? "busy"
+                    : !this.vectorDb || !this.metaCache
+                        ? "store_unavailable"
+                        : this.activeConfigurationError()
+                            ? "embedding_mismatch"
+                            : "ready";
+            return this.operations.runShared("documents", signal, (operationSignal) => (0, document_search_handler_1.handleDocumentSearch)({
+                home: config_1.PATHS.globalRoot,
+                state,
+                generation: () => this.resourceGenerationId(),
+                embeddingState: (root) => {
+                    const project = (0, project_registry_1.getProject)(root);
+                    if ((project === null || project === void 0 ? void 0 : project.status) !== "indexed")
+                        return "no_index";
+                    if (!this.activeGeneration)
+                        return "embedding_unavailable";
+                    if (this.workerPool &&
+                        this.workerPool.generation.fingerprint !==
+                            this.activeGeneration.fingerprint)
+                        return "embedding_mismatch";
+                    try {
+                        return (0, embedding_generation_1.compareEmbeddingGeneration)(project, this.activeGeneration)
+                            .state === "stale"
+                            ? "embedding_mismatch"
+                            : "ready";
+                    }
+                    catch (_a) {
+                        return "embedding_mismatch";
+                    }
+                },
+                meta: (file) => { var _a; return (_a = this.metaCache) === null || _a === void 0 ? void 0 : _a.get(file); },
+                queryState: () => { var _a; var _b; return (_b = (_a = this.workerPool) === null || _a === void 0 ? void 0 : _a.existingQueryState()) !== null && _b !== void 0 ? _b : "embedding_unavailable"; },
+                encode: (query, requestSignal) => this.workerPool
+                    ? this.workerPool.encodeQueryExisting(query, requestSignal)
+                    : Promise.reject(new Error("embedding_unavailable")),
+                table: () => this.vectorDb
+                    ? this.vectorDb.existingTableForRead()
+                    : Promise.reject(new Error("store_unavailable")),
+                indexState: (root) => this.indexState(root),
+                lastIndexed: (root) => { var _a; return (_a = (0, project_registry_1.getProject)(root)) === null || _a === void 0 ? void 0 : _a.lastIndexed; },
+            }, cmd, operationSignal));
+        });
+    }
     search(payload, signal) {
         return __awaiter(this, void 0, void 0, function* () {
             // Search handling lives in search-handler.ts (Phase 12 split). The daemon

package/dist/lib/daemon/document-search-handler.js

@@ -0,0 +1,194 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
+    function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
+    return new (P || (P = Promise))(function (resolve, reject) {
+        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
+        function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
+        function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
+        step((generator = generator.apply(thisArg, _arguments || [])).next());
+    });
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.documentFilter = documentFilter;
+exports.handleDocumentSearch = handleDocumentSearch;
+const path = __importStar(require("node:path"));
+const document_contract_1 = require("../mcp/document-contract");
+function sqlString(input) {
+    return `'${input.replace(/'/g, "''")}'`;
+}
+function documentFilter(prefixes) {
+    return `(${prefixes
+        .map((prefix) => {
+        const dir = prefix.endsWith(path.sep) ? prefix : prefix + path.sep;
+        return `(path = ${sqlString(prefix)} OR (path >= ${sqlString(dir)} AND path < ${sqlString(`${dir}\uffff`)}))`;
+    })
+        .join(" OR ")}) AND lower(path) LIKE '%.md'`;
+}
+function handleDocumentSearch(deps, cmd, signal) {
+    return __awaiter(this, void 0, void 0, function* () {
+        var _a, _b;
+        var _c, _d;
+        try {
+            const deadline = performance.now() + document_contract_1.DOCUMENT_LIMITS.deadlineMs;
+            signal = AbortSignal.any([
+                signal,
+                AbortSignal.timeout(document_contract_1.DOCUMENT_LIMITS.deadlineMs),
+            ]);
+            signal.throwIfAborted();
+            if (cmd.contractVersion !== document_contract_1.DOCUMENT_CONTRACT_VERSION)
+                return (0, document_contract_1.documentFailure)("unsupported_tool");
+            if (deps.state() !== "ready")
+                return (0, document_contract_1.documentFailure)(deps.state());
+            if (typeof cmd.checkout !== "string")
+                return (0, document_contract_1.documentFailure)("no_index");
+            const context = (0, document_contract_1.selectDocumentContext)(deps.home, cmd.checkout);
+            if (cmd.projectRoot !== context.wireRoot || cmd.store !== context.store)
+                return (0, document_contract_1.documentFailure)("store_unavailable");
+            const embeddingState = deps.embeddingState(context.wireRoot);
+            if (embeddingState !== "ready")
+                return (0, document_contract_1.documentFailure)(embeddingState);
+            const generation = deps.generation();
+            if (!(0, document_contract_1.documentGeneration)(generation))
+                return (0, document_contract_1.documentFailure)("index_unavailable");
+            if (!(0, document_contract_1.documentGeneration)(cmd.generation) || cmd.generation !== generation)
+                return (0, document_contract_1.documentFailure)("embedding_mismatch");
+            const recheck = () => {
+                signal.throwIfAborted();
+                if (performance.now() >= deadline)
+                    throw new Error("timeout");
+                if (deps.state() !== "ready")
+                    throw new Error(deps.state());
+                if (deps.generation() !== generation ||
+                    deps.embeddingState(context.wireRoot) !== "ready")
+                    throw new Error("embedding_mismatch");
+            };
+            const base = {
+                ok: true,
+                root: context.root,
+                store: context.store,
+                generation,
+            };
+            if (cmd.cmd === "documents.status") {
+                const { paths, requested } = (0, document_contract_1.documentCoveragePaths)(cmd.paths, context);
+                const covered = paths.flatMap((file) => {
+                    const entry = deps.meta(file);
+                    return (entry === null || entry === void 0 ? void 0 : entry.hasVectors) === true
+                        ? [
+                            Object.assign(Object.assign({ path: file }, (entry.hashVersion === 1 && /^[a-f0-9]{64}$/.test(entry.hash)
+                                ? { hash: entry.hash, hashAlgorithm: "sha256-bytes" }
+                                : {})), ((0, document_contract_1.documentMtime)(entry.mtimeMs) !== null
+                                ? { indexedMtimeMs: entry.mtimeMs }
+                                : {})),
+                        ]
+                        : [];
+                });
+                recheck();
+                const state = (0, document_contract_1.documentQueryState)(deps.queryState());
+                return Object.assign(Object.assign({}, base), { embeddingReady: state === "ready", queryState: state, lastIndexed: (_c = (0, document_contract_1.documentIndexedAt)((_a = deps.lastIndexed) === null || _a === void 0 ? void 0 : _a.call(deps, context.wireRoot))) !== null && _c !== void 0 ? _c : context.lastIndexed, indexState: (0, document_contract_1.documentIndexState)(deps.indexState(context.wireRoot)), covered, coverage: {
+                        requested,
+                        indexed: covered.length,
+                        partial: covered.length !== requested,
+                    } });
+            }
+            if (cmd.cmd !== "documents.search")
+                return (0, document_contract_1.documentFailure)("unsupported_tool");
+            if (typeof cmd.query !== "string" ||
+                !cmd.query.trim() ||
+                cmd.query.length > document_contract_1.DOCUMENT_LIMITS.query)
+                return (0, document_contract_1.documentFailure)("search_unavailable");
+            const prefixes = (0, document_contract_1.documentPaths)(cmd.prefixes, context, "prefixes");
+            const state = deps.queryState();
+            if (state !== "ready")
+                return (0, document_contract_1.documentFailure)(state);
+            recheck();
+            const encoded = yield deps.encode(cmd.query, signal);
+            recheck();
+            if (!encoded.dense.length || !encoded.dense.every(Number.isFinite))
+                return (0, document_contract_1.documentFailure)("embedding_unavailable");
+            const table = yield deps.table();
+            recheck();
+            const remainingMs = Math.floor(deadline - performance.now());
+            if (remainingMs <= 0)
+                throw new Error("timeout");
+            // Await the actual native promise, not a JS timeout race. The caller's
+            // bridge/signal has its own deadline; shared ownership must remain until
+            // native completion, even if the SDK fails to honor its native timeout.
+            const rows = (yield table
+                .vectorSearch(encoded.dense)
+                .distanceType("cosine")
+                .where(documentFilter(prefixes))
+                .select(["path", "hash", "start_line", "end_line", "_distance"])
+                .limit(document_contract_1.DOCUMENT_LIMITS.candidates)
+                .toArray({ timeoutMs: remainingMs }));
+            recheck();
+            const matches = rows.slice(0, document_contract_1.DOCUMENT_LIMITS.candidates).flatMap((row) => {
+                if (typeof row.path !== "string" ||
+                    !prefixes.some((prefix) => row.path === prefix || row.path.startsWith(prefix + path.sep)))
+                    return [];
+                const file = (0, document_contract_1.canonicalPointer)(row.path, context);
+                if (!file ||
+                    !Number.isInteger(row.start_line) ||
+                    row.start_line < 0 ||
+                    !Number.isInteger(row.end_line) ||
+                    row.end_line < row.start_line ||
+                    !Number.isFinite(row._distance))
+                    return [];
+                const entry = deps.meta(row.path);
+                // Only a matching current metadata revision proves this row's digest is bytes.
+                const hash = (entry === null || entry === void 0 ? void 0 : entry.hashVersion) === 1 &&
+                    entry.hasVectors === true &&
+                    row.hash === entry.hash &&
+                    /^[a-f0-9]{64}$/.test(row.hash)
+                    ? { hash: row.hash, hashAlgorithm: "sha256-bytes" }
+                    : {};
+                return [
+                    Object.assign({ path: file, startLine: row.start_line + 1, endLine: row.end_line + 1, score: 1 / (1 + Math.max(0, row._distance)) }, hash),
+                ];
+            });
+            return Object.assign(Object.assign({}, base), { indexState: (0, document_contract_1.documentIndexState)(deps.indexState(context.wireRoot)), matches, retrieval: "dense", candidateLimit: document_contract_1.DOCUMENT_LIMITS.candidates });
+        }
+        catch (error) {
+            const e = error;
+            return (0, document_contract_1.documentFailure)(signal.aborted
+                ? ((_b = signal.reason) === null || _b === void 0 ? void 0 : _b.name) === "TimeoutError"
+                    ? "timeout"
+                    : "cancelled"
+                : e.name === "QueryTimeoutError"
+                    ? "timeout"
+                    : ((_d = e.code) !== null && _d !== void 0 ? _d : e.message));
+        }
+    });
+}

package/dist/lib/daemon/ipc-handler.js

@@ -133,13 +133,17 @@ function handleCommand(daemon, cmd, conn) {
                 cmd.cmd !== "shutdown" &&
                 (daemon.operationStatus() === "closing" ||
                     daemon.operationStatus() === "closed")) {
-                return errorResponse(new operation_coordinator_1.OperationClosedError());
+                return cmd.cmd === "documents.status" || cmd.cmd === "documents.search"
+                    ? { ok: false, state: "busy" }
+                    : errorResponse(new operation_coordinator_1.OperationClosedError());
             }
             // The socket listens before LanceDB/MetaCache are open so liveness probes
             // succeed during slow startup. Gate resource-dependent commands until those
             // stores exist; ping/shutdown must always work (probes + restart).
             if (cmd.cmd !== "ping" && cmd.cmd !== "shutdown" && !daemon.isReady()) {
-                return { ok: false, error: "daemon initializing" };
+                return cmd.cmd === "documents.status" || cmd.cmd === "documents.search"
+                    ? { ok: false, state: "index_unavailable" }
+                    : { ok: false, error: "daemon initializing" };
             }
             switch (cmd.cmd) {
                 case "ping":
@@ -157,6 +161,7 @@ function handleCommand(daemon, cmd, conn) {
                             exclusiveGenerationRebuild: exports.EXCLUSIVE_GENERATION_REBUILD_PROTOCOL,
                             readVerbs: read_verbs_1.READ_VERBS_PROTOCOL,
                             searchDiagnostics: 1,
+                            existingIndexOnlySearch: 1,
                             perFileSearch: 1,
                             // `watch`/`unwatch` accept holder/pid/ttlMs. A daemon without this
                             // treats `unwatch` as unwatch-for-everyone, so clients releasing a
@@ -225,6 +230,28 @@ function handleCommand(daemon, cmd, conn) {
                     setImmediate(() => daemon.shutdown());
                     return { ok: true };
                 }
+                case "documents.status":
+                case "documents.search": {
+                    const ac = new AbortController();
+                    const close = () => ac.abort();
+                    conn.once("close", close);
+                    try {
+                        return yield daemon.documentSearch(cmd, ac.signal);
+                    }
+                    catch (error) {
+                        return {
+                            ok: false,
+                            state: ac.signal.aborted
+                                ? "cancelled"
+                                : error instanceof operation_coordinator_1.OperationBusyError
+                                    ? "busy"
+                                    : "index_unavailable",
+                        };
+                    }
+                    finally {
+                        conn.off("close", close);
+                    }
+                }
                 case "search":
                 case "search-v2": {
                     const isV2 = cmd.cmd === "search-v2";