@intentic/iq-engine 1.360.0 → 1.361.0

package/README.md

@@ -34,6 +34,11 @@ flowchart LR
 - `weak` overrides both when no reranked passage reaches `WEAK_FLOOR` (0.05), a cross-encoder probability, and adds a hint to stop or rephrase once. Every other score in the pipeline is relative to the best hit, so something always leads; this is the one absolute test, calibrated on [iq-bench](../iq-bench)'s no-answer slice. Without a reranker no answer is ever called weak.
   - 2026-10-06: raised from 0.005. It catches 8 of 12 no-answer cases instead of 5 and calls 3 of 76 answerable ones weak instead of 2. Over 393 replayed transcript queries it fires on 9 instead of none, and none of those sessions went on to open the file iq answered with.
 - The embedder and reranker are ONNX models baked at image build time by `scripts/fetch-model.mjs`. Without them search degrades to BM25 and ripgrep; it never downloads at runtime.
+- The resident query worker starts on its first model request, and loads each model only when its stage is needed.
+  It releases its thread and model state after fifteen quiet minutes, or two under cgroup memory pressure (full
+  avg10 at least 5%). Requests in flight prevent release; the next request recreates the worker. The text/index host
+  stays available. (2026-10-26) Permanent model retention was rejected in favor of a later cold load after a long
+  quiet period; frequent searches stay warm.
 - Embeddings are bge-small-en-v1.5 (q8): an attention-masked mean over the token states, L2-normalised. Queries carry bge's retrieval instruction ("Represent this sentence for searching relevant passages: "); passages never do. `VECTOR_SPACE` names what a stored vector depends on (model, pooling, normalisation), and both the index and the vector-cache sidecar are keyed by it: changing it empties both stores once and re-embeds every chunk, so vectors from two spaces never mix. The mean config keeps the bare model id as its value, the key existing caches already carry.
   - 2026-09-29: kept mean pooling over bge's reference [CLS] pooling. On iq-bench's 58 natural-language cases (hono, click, intentic), nDCG@10 for mean vs CLS was 0.716 vs 0.699 in the full pipeline (CLS better on 6 cases, worse on 4, sign p 0.75) and 0.610 vs 0.600 for the vector leg alone (12 better, 14 worse). That is a tie, and switching would force every index through a full re-embed (about 80 minutes for this repository's 138k chunks) for no measured gain. Revisit if a larger bench shows a real difference. Dropping the query instruction lowered nDCG under both poolings (vector leg with mean pooling 0.610 to 0.582).
 - `verify(text)` is the grounding check behind `iq verify`: it pulls the file paths, `path:line` anchors and code names out of an answer and checks them against the sweep and the index. A missing file or a name defined and written nowhere is an issue with the closest real one suggested; an anchor past its file's end is one; and an anchor whose line cites names the file holds, none of them inside the cited range, its enclosing function, or 15 lines around it, has drifted. Names are matched case-exactly, and a line citing one file at several lines is not judged for drift, since which name belongs to which anchor is a guess there. A file the index has not caught up with is read from disk and its index rows ignored, since an agent verifies right after editing, ahead of the daemon's re-index.

package/dist/query/query-worker.js

@@ -10,12 +10,12 @@ if (port === null) {
 }
 const { indexDir, modelDir } = workerData;
 const db = openIndex(indexDir, "read");
-const embedderReady = loadEmbedder(modelDir);
-const rerankerReady = loadReranker(modelDir);
+let embedderReady;
+let rerankerReady;
 const answer = async (request) => {
     const { id } = request;
     if (request.type === "semantic") {
-        const embedder = await embedderReady;
+        const embedder = await (embedderReady ??= loadEmbedder(modelDir));
         if (embedder === undefined) {
             return { type: "absent", id };
         }
@@ -23,7 +23,7 @@ const answer = async (request) => {
         const hits = semanticSearch(db, await embedder.embedQuery(request.query), new Set(request.allowed));
         return { type: "semantic", id, hits, pending };
     }
-    const reranker = await rerankerReady;
+    const reranker = await (rerankerReady ??= loadReranker(modelDir));
     if (reranker === undefined) {
         return { type: "absent", id };
     }

package/dist/query/query-worker.js.map

@@ -1 +1 @@
-{"version":3,"file":"query-worker.js","sourceRoot":"","sources":["../../src/query/query-worker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AAC7D,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAyB3C,MAAM,IAAI,GAAG,UAAU,CAAC;AACxB,IAAI,IAAI,KAAK,IAAI,EAAE,CAAC;IAChB,MAAM,IAAI,KAAK,CAAC,iDAAiD,CAAC,CAAC;AACvE,CAAC;AAED,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,GAAG,UAA6B,CAAC;AAE7D,MAAM,EAAE,GAAG,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;AACvC,MAAM,aAAa,GAAG,YAAY,CAAC,QAAQ,CAAC,CAAC;AAC7C,MAAM,aAAa,GAAG,YAAY,CAAC,QAAQ,CAAC,CAAC;AAE7C,MAAM,MAAM,GAAG,KAAK,EAAE,OAA2B,EAAgC,EAAE;IAC/E,MAAM,EAAE,EAAE,EAAE,GAAG,OAAO,CAAC;IACvB,IAAI,OAAO,CAAC,IAAI,KAAK,UAAU,EAAE,CAAC;QAC9B,MAAM,QAAQ,GAAG,MAAM,aAAa,CAAC;QACrC,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YACzB,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC;QAClC,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,CAAC,EAAE,CAAC,GAAG,CAAC,qDAAqD,CAAC,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;QAClG,MAAM,IAAI,GAAG,cAAc,CAAC,EAAE,EAAE,MAAM,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,IAAI,GAAG,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC;QACpG,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;IACnD,CAAC;IACD,MAAM,QAAQ,GAAG,MAAM,aAAa,CAAC;IACrC,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QACzB,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC;IAClC,CAAC;IACD,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,MAAM,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,QAAQ,CAAC,EAAE,CAAC;AAClG,CAAC,CAAC;AAEF,IAAI,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,OAA2B,EAAE,EAAE;IAC/C,KAAK,MAAM,CAAC,OAAO,CAAC,CAAC,IAAI,CACrB,CAAC,QAAQ,EAAE,EAAE;QACT,IAAI,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC;IAC/B,CAAC,EACD,CAAC,KAAc,EAAE,EAAE;QAEf,IAAI,CAAC,WAAW,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,OAAO,CAAC,EAAE,EAAE,KAAK,EAAE,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IACrF,CAAC,CACJ,CAAC;AACN,CAAC,CAAC,CAAC"}
+{"version":3,"file":"query-worker.js","sourceRoot":"","sources":["../../src/query/query-worker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AAC7D,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAyB3C,MAAM,IAAI,GAAG,UAAU,CAAC;AACxB,IAAI,IAAI,KAAK,IAAI,EAAE,CAAC;IAChB,MAAM,IAAI,KAAK,CAAC,iDAAiD,CAAC,CAAC;AACvE,CAAC;AAED,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,GAAG,UAA6B,CAAC;AAE7D,MAAM,EAAE,GAAG,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;AACvC,IAAI,aAA0D,CAAC;AAC/D,IAAI,aAA0D,CAAC;AAE/D,MAAM,MAAM,GAAG,KAAK,EAAE,OAA2B,EAAgC,EAAE;IAC/E,MAAM,EAAE,EAAE,EAAE,GAAG,OAAO,CAAC;IACvB,IAAI,OAAO,CAAC,IAAI,KAAK,UAAU,EAAE,CAAC;QAC9B,MAAM,QAAQ,GAAG,MAAM,CAAC,aAAa,KAAK,YAAY,CAAC,QAAQ,CAAC,CAAC,CAAC;QAClE,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YACzB,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC;QAClC,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,CAAC,EAAE,CAAC,GAAG,CAAC,qDAAqD,CAAC,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;QAClG,MAAM,IAAI,GAAG,cAAc,CAAC,EAAE,EAAE,MAAM,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,IAAI,GAAG,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC;QACpG,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;IACnD,CAAC;IACD,MAAM,QAAQ,GAAG,MAAM,CAAC,aAAa,KAAK,YAAY,CAAC,QAAQ,CAAC,CAAC,CAAC;IAClE,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QACzB,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,CAAC;IAClC,CAAC;IACD,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,MAAM,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,QAAQ,CAAC,EAAE,CAAC;AAClG,CAAC,CAAC;AAEF,IAAI,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,OAA2B,EAAE,EAAE;IAC/C,KAAK,MAAM,CAAC,OAAO,CAAC,CAAC,IAAI,CACrB,CAAC,QAAQ,EAAE,EAAE;QACT,IAAI,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC;IAC/B,CAAC,EACD,CAAC,KAAc,EAAE,EAAE;QAEf,IAAI,CAAC,WAAW,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,OAAO,CAAC,EAAE,EAAE,KAAK,EAAE,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IACrF,CAAC,CACJ,CAAC;AACN,CAAC,CAAC,CAAC"}

package/dist/query/worker-idle.d.ts

@@ -0,0 +1,11 @@
+export interface WorkerIdleOptions {
+    readonly idleMs?: number;
+    readonly pressureIdleMs?: number;
+    readonly underPressure?: () => Promise<boolean>;
+    readonly onError?: (error: Error) => void;
+}
+export declare const workerIdle: (options: WorkerIdleOptions, release: () => Promise<void>) => {
+    busy: () => void;
+    quiet: () => void;
+};
+//# sourceMappingURL=worker-idle.d.ts.map

package/dist/query/worker-idle.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"worker-idle.d.ts","sourceRoot":"","sources":["../../src/query/worker-idle.ts"],"names":[],"mappings":"AAEA,MAAM,WAAW,iBAAiB;IAC9B,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;IAChD,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,CAAC;CAC7C;AAUD,eAAO,MAAM,UAAU,YAAa,iBAAiB,WAAW,MAAM,OAAO,CAAC,IAAI,CAAC;gBAM9D,IAAI;iBAwBN,IAAI;CAMtB,CAAC"}

package/dist/query/worker-idle.js

@@ -0,0 +1,48 @@
+import { readFile } from "node:fs/promises";
+const underPressure = async () => {
+    const text = await readFile("/sys/fs/cgroup/memory.pressure", "utf8").catch(() => "");
+    return Number(/^full avg10=([\d.]+)/mu.exec(text)?.[1] ?? 0) >= 5;
+};
+export const workerIdle = (options, release) => {
+    const idleMs = options.idleMs ?? 15 * 60_000;
+    const pressureIdleMs = Math.min(idleMs, options.pressureIdleMs ?? 2 * 60_000);
+    let timer;
+    let generation = 0;
+    let quietSince = 0;
+    const busy = () => {
+        generation += 1;
+        clearTimeout(timer);
+    };
+    const arm = () => {
+        const own = generation;
+        timer = setTimeout(() => {
+            void (async () => {
+                const pressured = await (options.underPressure ?? underPressure)();
+                if (own !== generation) {
+                    return;
+                }
+                if (Date.now() - quietSince >= (pressured ? pressureIdleMs : idleMs)) {
+                    await release();
+                }
+                else {
+                    arm();
+                }
+            })().catch((error) => {
+                options.onError?.(error);
+                if (own === generation) {
+                    arm();
+                }
+            });
+        }, Math.max(1, Math.min(30_000, pressureIdleMs)));
+        timer.unref();
+    };
+    return {
+        busy,
+        quiet: () => {
+            busy();
+            quietSince = Date.now();
+            arm();
+        },
+    };
+};
+//# sourceMappingURL=worker-idle.js.map

package/dist/query/worker-idle.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"worker-idle.js","sourceRoot":"","sources":["../../src/query/worker-idle.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAU5C,MAAM,aAAa,GAAG,KAAK,IAAsB,EAAE;IAE/C,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,gCAAgC,EAAE,MAAM,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;IACtF,OAAO,MAAM,CAAC,wBAAwB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC;AACtE,CAAC,CAAC;AAGF,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,OAA0B,EAAE,OAA4B,EAAE,EAAE;IACnF,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,IAAI,EAAE,GAAG,MAAM,CAAC;IAC7C,MAAM,cAAc,GAAG,IAAI,CAAC,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,cAAc,IAAI,CAAC,GAAG,MAAM,CAAC,CAAC;IAC9E,IAAI,KAAgD,CAAC;IACrD,IAAI,UAAU,GAAG,CAAC,CAAC;IACnB,IAAI,UAAU,GAAG,CAAC,CAAC;IACnB,MAAM,IAAI,GAAG,GAAS,EAAE;QACpB,UAAU,IAAI,CAAC,CAAC;QAChB,YAAY,CAAC,KAAK,CAAC,CAAC;IACxB,CAAC,CAAC;IACF,MAAM,GAAG,GAAG,GAAS,EAAE;QACnB,MAAM,GAAG,GAAG,UAAU,CAAC;QACvB,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE;YACpB,KAAK,CAAC,KAAK,IAAI,EAAE;gBACb,MAAM,SAAS,GAAG,MAAM,CAAC,OAAO,CAAC,aAAa,IAAI,aAAa,CAAC,EAAE,CAAC;gBACnE,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;oBAAA,OAAO;gBAAA,CAAC;gBACjC,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,UAAU,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC,CAAC,MAAM,CAAC,EAAE,CAAC;oBACnE,MAAM,OAAO,EAAE,CAAC;gBACpB,CAAC;qBAAM,CAAC;oBACJ,GAAG,EAAE,CAAC;gBACV,CAAC;YACL,CAAC,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC,KAAY,EAAE,EAAE;gBACxB,OAAO,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,CAAC;gBACzB,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;oBAAA,GAAG,EAAE,CAAC;gBAAA,CAAC;YACpC,CAAC,CAAC,CAAC;QACP,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC;QAClD,KAAK,CAAC,KAAK,EAAE,CAAC;IAClB,CAAC,CAAC;IACF,OAAO;QACH,IAAI;QACJ,KAAK,EAAE,GAAS,EAAE;YACd,IAAI,EAAE,CAAC;YACP,UAAU,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;YACxB,GAAG,EAAE,CAAC;QACV,CAAC;KACJ,CAAC;AACN,CAAC,CAAC"}

package/dist/query/worker-scorer.d.ts

@@ -1,8 +1,18 @@
+import { type WorkerIdleOptions } from "./worker-idle.js";
+import type { QueryWorkerRequest, QueryWorkerResponse } from "./query-worker.js";
 import type { QueryScorer } from "./scorer.js";
-export interface WorkerScorerOptions {
+export interface QueryThread {
+    postMessage(request: QueryWorkerRequest): void;
+    on(event: "message", listener: (response: QueryWorkerResponse) => void): void;
+    on(event: "error", listener: (error: Error) => void): void;
+    on(event: "exit", listener: () => void): void;
+    terminate(): Promise<number>;
+}
+export interface WorkerScorerOptions extends WorkerIdleOptions {
     readonly indexDir: string;
     readonly modelDir: string | undefined;
     readonly onError?: (error: Error) => void;
+    readonly spawn?: () => QueryThread;
 }
 export interface WorkerScorer extends QueryScorer {
     metrics(): {

package/dist/query/worker-scorer.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"worker-scorer.d.ts","sourceRoot":"","sources":["../../src/query/worker-scorer.ts"],"names":[],"mappings":"AAKA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAK/C,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,SAAS,CAAC;IAEtC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,CAAC;CAC7C;AAED,MAAM,WAAW,YAAa,SAAQ,WAAW;IAC7C,OAAO,IAAI;QAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;QAAC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;KAAE,CAAC;IACxE,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAID,eAAO,MAAM,YAAY,YAAa,mBAAmB,KAAG,YA2F3D,CAAC"}
+{"version":3,"file":"worker-scorer.d.ts","sourceRoot":"","sources":["../../src/query/worker-scorer.ts"],"names":[],"mappings":"AAGA,OAAO,EAAc,KAAK,iBAAiB,EAAE,MAAM,kBAAkB,CAAC;AAEtE,OAAO,KAAK,EAAmB,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,mBAAmB,CAAC;AAClG,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAK/C,MAAM,WAAW,WAAW;IACxB,WAAW,CAAC,OAAO,EAAE,kBAAkB,GAAG,IAAI,CAAC;IAC/C,EAAE,CAAC,KAAK,EAAE,SAAS,EAAE,QAAQ,EAAE,CAAC,QAAQ,EAAE,mBAAmB,KAAK,IAAI,GAAG,IAAI,CAAC;IAC9E,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,GAAG,IAAI,CAAC;IAC3D,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,IAAI,GAAG,IAAI,CAAC;IAC9C,SAAS,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC;CAChC;AAED,MAAM,WAAW,mBAAoB,SAAQ,iBAAiB;IAC1D,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,SAAS,CAAC;IAEtC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,CAAC;IAC1C,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,WAAW,CAAC;CACtC;AAED,MAAM,WAAW,YAAa,SAAQ,WAAW;IAC7C,OAAO,IAAI;QAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;QAAC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;KAAE,CAAC;IACxE,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAaD,eAAO,MAAM,YAAY,YAAa,mBAAmB,KAAG,YAiG3D,CAAC"}

package/dist/query/worker-scorer.js

@@ -1,38 +1,52 @@
 import { join } from "node:path";
 import { pathToFileURL } from "node:url";
 import { Worker } from "node:worker_threads";
+import { workerIdle } from "./worker-idle.js";
 import { packageRoot } from "@intentic/constants/node";
 const WORKER_URL = pathToFileURL(join(packageRoot(import.meta.url), "dist/query/query-worker.js"));
+const settleAll = (pending) => {
+    const waiting = [...pending.values()];
+    pending.clear();
+    for (const settle of waiting) {
+        settle(undefined);
+    }
+};
 export const workerScorer = (options) => {
     const pending = new Map();
     let worker;
     let nextId = 0;
     let closed = false;
-    const settleAll = () => {
-        const waiting = [...pending.values()];
-        pending.clear();
-        for (const settle of waiting) {
-            settle(undefined);
-        }
-    };
+    const idle = workerIdle(options, async () => {
+        const held = worker;
+        worker = undefined;
+        await held?.terminate();
+    });
     const abandon = (dead, error) => {
-        if (worker === dead) {
-            worker = undefined;
+        if (worker !== dead) {
+            return;
         }
-        settleAll();
+        worker = undefined;
+        idle.busy();
+        settleAll(pending);
         options.onError?.(error);
     };
     const live = () => {
         if (worker !== undefined) {
             return worker;
         }
-        const spawned = new Worker(WORKER_URL, {
+        const spawned = options.spawn?.() ?? new Worker(WORKER_URL, {
             workerData: { indexDir: options.indexDir, modelDir: options.modelDir },
         });
         spawned.on("message", (response) => {
+            if (worker !== spawned) {
+                return;
+            }
             const settle = pending.get(response.id);
             pending.delete(response.id);
             settle?.(response);
+            if (pending.size === 0) {
+                idle.quiet();
+            }
         });
         spawned.on("error", (error) => {
             abandon(spawned, error);
@@ -45,17 +59,27 @@ export const workerScorer = (options) => {
         worker = spawned;
         return spawned;
     };
-    live();
     const send = async (request) => {
         if (closed) {
             return undefined;
         }
         const id = nextId++;
+        idle.busy();
         const message = request(id);
         const answered = new Promise((resolve) => {
             pending.set(id, resolve);
         });
-        live().postMessage(message);
+        try {
+            live().postMessage(message);
+        }
+        catch (error) {
+            pending.delete(id);
+            options.onError?.(error instanceof Error ? error : new Error(String(error)));
+            if (worker !== undefined && pending.size === 0) {
+                idle.quiet();
+            }
+            return undefined;
+        }
         const response = await answered;
         if (response?.type === "failed") {
             options.onError?.(new Error(`iq query worker: ${message.type} failed: ${response.error}`));
@@ -75,9 +99,10 @@ export const workerScorer = (options) => {
         },
         async close() {
             closed = true;
+            idle.busy();
             const running = worker;
             worker = undefined;
-            settleAll();
+            settleAll(pending);
             await running?.terminate();
         },
     };

package/dist/query/worker-scorer.js.map

@@ -1 +1 @@
-{"version":3,"file":"worker-scorer.js","sourceRoot":"","sources":["../../src/query/worker-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAAE,MAAM,EAAE,MAAM,qBAAqB,CAAC;AAC7C,OAAO,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAKvD,MAAM,UAAU,GAAG,aAAa,CAAC,IAAI,CAAC,WAAW,CAAC,OAAO,IAAI,CAAC,GAAG,CAAC,EAAE,4BAA4B,CAAC,CAAC,CAAC;AAgBnG,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,OAA4B,EAAgB,EAAE;IACvE,MAAM,OAAO,GAAG,IAAI,GAAG,EAA+D,CAAC;IACvF,IAAI,MAA0B,CAAC;IAC/B,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,IAAI,MAAM,GAAG,KAAK,CAAC;IAGnB,MAAM,SAAS,GAAG,GAAS,EAAE;QACzB,MAAM,OAAO,GAAG,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;QACtC,OAAO,CAAC,KAAK,EAAE,CAAC;QAChB,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;YAC3B,MAAM,CAAC,SAAS,CAAC,CAAC;QACtB,CAAC;IACL,CAAC,CAAC;IAGF,MAAM,OAAO,GAAG,CAAC,IAAY,EAAE,KAAY,EAAQ,EAAE;QACjD,IAAI,MAAM,KAAK,IAAI,EAAE,CAAC;YAClB,MAAM,GAAG,SAAS,CAAC;QACvB,CAAC;QACD,SAAS,EAAE,CAAC;QACZ,OAAO,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,CAAC;IAC7B,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,GAAW,EAAE;QACtB,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;YACvB,OAAO,MAAM,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,MAAM,CAAC,UAAU,EAAE;YACnC,UAAU,EAAE,EAAE,QAAQ,EAAE,OAAO,CAAC,QAAQ,EAAE,QAAQ,EAAE,OAAO,CAAC,QAAQ,EAA4B;SACnG,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,QAA6B,EAAE,EAAE;YACpD,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;YACxC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;YAC5B,MAAM,EAAE,CAAC,QAAQ,CAAC,CAAC;QACvB,CAAC,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,KAAY,EAAE,EAAE;YACjC,OAAO,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;QAC5B,CAAC,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,MAAM,EAAE,GAAG,EAAE;YAEpB,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;gBACrB,OAAO,CAAC,OAAO,EAAE,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAC,CAAC;YAC1D,CAAC;QACL,CAAC,CAAC,CAAC;QACH,MAAM,GAAG,OAAO,CAAC;QACjB,OAAO,OAAO,CAAC;IACnB,CAAC,CAAC;IAGF,IAAI,EAAE,CAAC;IAEP,MAAM,IAAI,GAAG,KAAK,EAAE,OAA2C,EAA4C,EAAE;QACzG,IAAI,MAAM,EAAE,CAAC;YACT,OAAO,SAAS,CAAC;QACrB,CAAC;QACD,MAAM,EAAE,GAAG,MAAM,EAAE,CAAC;QACpB,MAAM,OAAO,GAAG,OAAO,CAAC,EAAE,CAAC,CAAC;QAC5B,MAAM,QAAQ,GAAG,IAAI,OAAO,CAAkC,CAAC,OAAO,EAAE,EAAE;YACtE,OAAO,CAAC,GAAG,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC;QAC7B,CAAC,CAAC,CAAC;QAEH,IAAI,EAAE,CAAC,WAAW,CAAC,OAAO,CAAC,CAAC;QAC5B,MAAM,QAAQ,GAAG,MAAM,QAAQ,CAAC;QAChC,IAAI,QAAQ,EAAE,IAAI,KAAK,QAAQ,EAAE,CAAC;YAC9B,OAAO,CAAC,OAAO,EAAE,CAAC,IAAI,KAAK,CAAC,oBAAoB,OAAO,CAAC,IAAI,YAAY,QAAQ,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YAC3F,OAAO,SAAS,CAAC;QACrB,CAAC;QACD,OAAO,QAAQ,CAAC;IACpB,CAAC,CAAC;IAEF,OAAO;QACH,OAAO,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,KAAK,SAAS,EAAE,eAAe,EAAE,OAAO,CAAC,IAAI,EAAE,CAAC;QAC9E,KAAK,CAAC,QAAQ,CAAC,KAAK,EAAE,OAAO;YACzB,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,KAAK,EAAE,OAAO,EAAE,CAAC,GAAG,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC;YAC9F,OAAO,QAAQ,EAAE,IAAI,KAAK,UAAU,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,EAAE,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;QAC1G,CAAC;QACD,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,QAAQ;YACxB,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,KAAK,EAAE,QAAQ,EAAE,CAAC,GAAG,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC;YAC9F,OAAO,QAAQ,EAAE,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC;QACrE,CAAC;QACD,KAAK,CAAC,KAAK;YACP,MAAM,GAAG,IAAI,CAAC;YACd,MAAM,OAAO,GAAG,MAAM,CAAC;YACvB,MAAM,GAAG,SAAS,CAAC;YAEnB,SAAS,EAAE,CAAC;YACZ,MAAM,OAAO,EAAE,SAAS,EAAE,CAAC;QAC/B,CAAC;KACJ,CAAC;AACN,CAAC,CAAC"}
+{"version":3,"file":"worker-scorer.js","sourceRoot":"","sources":["../../src/query/worker-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAAE,MAAM,EAAE,MAAM,qBAAqB,CAAC;AAC7C,OAAO,EAAE,UAAU,EAA0B,MAAM,kBAAkB,CAAC;AACtE,OAAO,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAC;AAKvD,MAAM,UAAU,GAAG,aAAa,CAAC,IAAI,CAAC,WAAW,CAAC,OAAO,IAAI,CAAC,GAAG,CAAC,EAAE,4BAA4B,CAAC,CAAC,CAAC;AA0BnG,MAAM,SAAS,GAAG,CAAC,OAAgB,EAAQ,EAAE;IACzC,MAAM,OAAO,GAAG,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IACtC,OAAO,CAAC,KAAK,EAAE,CAAC;IAChB,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAAA,MAAM,CAAC,SAAS,CAAC,CAAC;IAAA,CAAC;AACtD,CAAC,CAAC;AAIF,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,OAA4B,EAAgB,EAAE;IACvE,MAAM,OAAO,GAAY,IAAI,GAAG,EAAE,CAAC;IACnC,IAAI,MAA+B,CAAC;IACpC,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,MAAM,IAAI,GAAG,UAAU,CAAC,OAAO,EAAE,KAAK,IAAI,EAAE;QACxC,MAAM,IAAI,GAAG,MAAM,CAAC;QACpB,MAAM,GAAG,SAAS,CAAC;QACnB,MAAM,IAAI,EAAE,SAAS,EAAE,CAAC;IAC5B,CAAC,CAAC,CAAC;IAGH,MAAM,OAAO,GAAG,CAAC,IAAiB,EAAE,KAAY,EAAQ,EAAE;QACtD,IAAI,MAAM,KAAK,IAAI,EAAE,CAAC;YAAA,OAAO;QAAA,CAAC;QAC9B,MAAM,GAAG,SAAS,CAAC;QACnB,IAAI,CAAC,IAAI,EAAE,CAAC;QACZ,SAAS,CAAC,OAAO,CAAC,CAAC;QACnB,OAAO,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,CAAC;IAC7B,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,GAAgB,EAAE;QAC3B,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;YACvB,OAAO,MAAM,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,EAAE,EAAE,IAAI,IAAI,MAAM,CAAC,UAAU,EAAE;YACxD,UAAU,EAAE,EAAE,QAAQ,EAAE,OAAO,CAAC,QAAQ,EAAE,QAAQ,EAAE,OAAO,CAAC,QAAQ,EAA4B;SACnG,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,QAA6B,EAAE,EAAE;YACpD,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;gBAAA,OAAO;YAAA,CAAC;YACjC,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;YACxC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;YAC5B,MAAM,EAAE,CAAC,QAAQ,CAAC,CAAC;YACnB,IAAI,OAAO,CAAC,IAAI,KAAK,CAAC,EAAE,CAAC;gBACrB,IAAI,CAAC,KAAK,EAAE,CAAC;YACjB,CAAC;QACL,CAAC,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,KAAY,EAAE,EAAE;YACjC,OAAO,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;QAC5B,CAAC,CAAC,CAAC;QACH,OAAO,CAAC,EAAE,CAAC,MAAM,EAAE,GAAG,EAAE;YAEpB,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;gBACrB,OAAO,CAAC,OAAO,EAAE,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAC,CAAC;YAC1D,CAAC;QACL,CAAC,CAAC,CAAC;QACH,MAAM,GAAG,OAAO,CAAC;QACjB,OAAO,OAAO,CAAC;IACnB,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,KAAK,EAAE,OAA2C,EAA4C,EAAE;QACzG,IAAI,MAAM,EAAE,CAAC;YACT,OAAO,SAAS,CAAC;QACrB,CAAC;QACD,MAAM,EAAE,GAAG,MAAM,EAAE,CAAC;QACpB,IAAI,CAAC,IAAI,EAAE,CAAC;QACZ,MAAM,OAAO,GAAG,OAAO,CAAC,EAAE,CAAC,CAAC;QAC5B,MAAM,QAAQ,GAAG,IAAI,OAAO,CAAkC,CAAC,OAAO,EAAE,EAAE;YACtE,OAAO,CAAC,GAAG,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC;QAC7B,CAAC,CAAC,CAAC;QACH,IAAI,CAAC;YAED,IAAI,EAAE,CAAC,WAAW,CAAC,OAAO,CAAC,CAAC;QAChC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACb,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;YACnB,OAAO,CAAC,OAAO,EAAE,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YAC7E,IAAI,MAAM,KAAK,SAAS,IAAI,OAAO,CAAC,IAAI,KAAK,CAAC,EAAE,CAAC;gBAAA,IAAI,CAAC,KAAK,EAAE,CAAC;YAAA,CAAC;YAC/D,OAAO,SAAS,CAAC;QACrB,CAAC;QACD,MAAM,QAAQ,GAAG,MAAM,QAAQ,CAAC;QAChC,IAAI,QAAQ,EAAE,IAAI,KAAK,QAAQ,EAAE,CAAC;YAC9B,OAAO,CAAC,OAAO,EAAE,CAAC,IAAI,KAAK,CAAC,oBAAoB,OAAO,CAAC,IAAI,YAAY,QAAQ,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YAC3F,OAAO,SAAS,CAAC;QACrB,CAAC;QACD,OAAO,QAAQ,CAAC;IACpB,CAAC,CAAC;IAEF,OAAO;QACH,OAAO,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,KAAK,SAAS,EAAE,eAAe,EAAE,OAAO,CAAC,IAAI,EAAE,CAAC;QAC9E,KAAK,CAAC,QAAQ,CAAC,KAAK,EAAE,OAAO;YACzB,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAE,KAAK,EAAE,OAAO,EAAE,CAAC,GAAG,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC;YAC9F,OAAO,QAAQ,EAAE,IAAI,KAAK,UAAU,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,EAAE,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;QAC1G,CAAC;QACD,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,QAAQ;YACxB,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,EAAE,KAAK,EAAE,QAAQ,EAAE,CAAC,GAAG,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC;YAC9F,OAAO,QAAQ,EAAE,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC;QACrE,CAAC;QACD,KAAK,CAAC,KAAK;YACP,MAAM,GAAG,IAAI,CAAC;YACd,IAAI,CAAC,IAAI,EAAE,CAAC;YACZ,MAAM,OAAO,GAAG,MAAM,CAAC;YACvB,MAAM,GAAG,SAAS,CAAC;YAEnB,SAAS,CAAC,OAAO,CAAC,CAAC;YACnB,MAAM,OAAO,EAAE,SAAS,EAAE,CAAC;QAC/B,CAAC;KACJ,CAAC;AACN,CAAC,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@intentic/iq-engine",
-  "version": "1.360.0",
+  "version": "1.361.0",
   "description": "Search engine behind the iq CLI, disk index (node:sqlite), lexical/structural/semantic/git engines, rank fusion, budget renderer",
   "license": "MIT",
   "type": "module",
@@ -57,28 +57,27 @@
     "@ast-grep/lang-java": "0.0.7",
     "@ast-grep/lang-python": "0.0.6",
     "@ast-grep/lang-rust": "0.0.7",
-    "@ast-grep/napi": "0.45.3",
+    "@ast-grep/napi": "0.51.0",
     "@huggingface/hub": "2.17.5",
     "@huggingface/transformers": "4.3.0",
-    "@intentic/base": "1.360.0",
-    "@intentic/constants": "1.360.0",
-    "@intentic/sandbox-contract": "1.360.0",
-    "@intentic/workspace-ignore": "1.360.0",
-    "sqlite-vec": "0.1.9",
-    "tslib": "2.8.1"
+    "@intentic/base": "1.361.0",
+    "@intentic/constants": "1.361.0",
+    "@intentic/sandbox-contract": "1.361.0",
+    "@intentic/workspace-ignore": "1.361.0",
+    "sqlite-vec": "0.1.9"
   },
   "devDependencies": {
     "@intentic/testing": "0.0.0",
     "@intentic/tsconfig": "0.0.0",
-    "@types/bun": "1.4.2",
-    "@types/node": "24.19.1",
-    "@typescript/native-preview": "7.0.0-dev.20260707.2"
+    "@types/bun": "1.4.4",
+    "@types/node": "24.19.2",
+    "typescript": "7.0.2"
   },
   "scripts": {
-    "build": "tsgo",
+    "build": "tsc",
     "typecheck": "bun check -p tsconfig.test.json",
-    "watch": "tsgo --build --watch",
-    "test": "tsgo && suites",
+    "watch": "tsc --build --watch",
+    "test": "tsc && suites",
     "test:watch": "suites --watch"
   }
 }