clawmem 0.15.1 → 0.17.0

package/AGENTS.md

@@ -174,7 +174,7 @@ compositeScore = (0.50·searchScore + 0.25·recencyScore + 0.25·confidenceScore
 - **Vector search empty but BM25 works** → missing embeddings (the watcher indexes but does NOT embed). Run `clawmem embed`.
 - **`intent_search` weak for WHY/ENTITY** → sparse graph. Run `build_graphs`. Don't run it after every reindex (A-MEM links per-doc automatically).
 - **Rankings look RRF-flat / reranker suspect** → `clawmem rerank-health`. A mis-served reranker (e.g. a GGUF that drops the score head) returns HTTP 200 but inert scores, silently collapsing ranking to RRF.
-- **Intermittent `UserPromptSubmit hook timed out after 8s — output discarded`** → almost always the context-surfacing hook's **cold-start** (fresh Bun process + opening a large `index.sqlite` + cold OS page cache), NOT inference. Recurs on memory-constrained hosts (e.g. WSL with a low memory cap) as the cache is evicted between turns. **Durable fix: give the host enough RAM to keep the index cached**; raising the hook `timeout` in `~/.claude/settings.json` (8s default; no CLI knob) is a secondary margin — avoid 15s+ as a standing default. Detail: [docs/troubleshooting.md](docs/troubleshooting.md) → *Tuning the context-surfacing hook timeout*.
+- **Intermittent `UserPromptSubmit hook timed out after 8s — output discarded`** → **fixed in v0.16.0** (upgrade). Root cause was not inference or host RAM alone: the `context-surfacing` vector leg ran a *synchronous* `sqlite-vec` scan that the `Promise.race(vectorTimeout)` guard could not bound (a synchronous call blocks the event loop, so the timer never fires), and every writable hook open ran an unconditional backfill `UPDATE` that could wait out `busy_timeout` under writer contention. v0.16.0 bounds both vector legs with real deadlines + timer cleanup, read-guards the init backfill, caps the init `busy_timeout`, and adds a watcher-side vector prewarm. A cold OS page cache still adds latency to the first post-boot call, so RAM headroom helps the margin — but on a large vault the scan cost, not RAM, was the dominant trigger. Still seeing it after upgrading? Raise the hook `timeout` in `~/.claude/settings.json` (8s default; no CLI knob) as a secondary margin. Detail: [docs/troubleshooting.md](docs/troubleshooting.md).
 - **Anything setup-shaped** (download blocked, server unreachable, watcher memory bloat, indexer bugs) → [docs/troubleshooting.md](docs/troubleshooting.md).
 
 ---

package/docs/troubleshooting.md

@@ -136,7 +136,7 @@ Common issues when running ClawMem with hooks, MCP server, or OpenClaw plugin. O
 - If the error persists after v0.1.8: restart the watcher to clear accumulated state (`systemctl --user restart clawmem-watcher.service`). Check `systemctl --user status clawmem-watcher.service` for memory usage — healthy is under 100MB, bloated is 400MB+.
 - **v0.2.4 fix:** Hook's SQLite `busy_timeout` was 500ms — too tight. During A-MEM enrichment or heavy indexing, the watcher can hold write locks for 500ms+, causing the hook's DB open to fail with SQLITE_BUSY. Raised to 5000ms (matches MCP server). The hook's 8s outer timeout still leaves 3s for actual work after a 5s busy wait.
 - **v0.3.1 fix:** Shell `timeout` wrappers (e.g., `timeout 8 clawmem hook context-surfacing`) kill the process with exit 124 and no stderr — Claude Code reports "Failed with non-blocking status code: No stderr output". This affects all hook events (UserPromptSubmit, Stop, SessionStart, PreCompact), not just Stop hooks. Fix: Remove shell `timeout` from all hook commands and use Claude Code's native `timeout` property instead. Run `clawmem setup hooks` to reinstall with correct config (v0.3.1+), or manually update `~/.claude/settings.json` — see [setup-hooks](guides/setup-hooks.md).
-- **Large vault + memory-constrained host (intermittent cold-start, even with `llama-server`).** A correctly-configured `llama-server` (the ~200ms row under *Hooks slow or near timeout*) can still intermittently exceed the 8s budget on the **first/cold call of a session — or any later turn after the OS evicts the page cache**. The cost here is not inference and not the `node-llama-cpp` import; it is process + data cold-start: spawning a fresh Bun process, opening a large `index.sqlite` (this grows with vault size — a multi-hundred-MB to >1GB index is the trigger), and re-reading index pages dropped from the OS page cache. On a memory-constrained host this *recurs* — e.g. a WSL2 instance with a low `.wslconfig` `memory` cap (or any box under memory pressure) evicts the cached index + Bun modules between turns, so "cold start" is no longer once-per-session. Warm calls stay sub-second; only the cold ones blow the budget, which is why it presents as intermittent "timed out after 8s" on *certain* turns. **Durable fix: give the host enough RAM that the index + Bun modules stay cached** (e.g. raise the WSL2 `memory` cap) — this removes the recurrence at the source. A modest `timeout` bump (see the tradeoffs table under *Hooks slow or near timeout*) is a secondary margin for the genuine cold call, not a substitute for memory headroom. The `deep` profile additionally reranks (extra remote round-trips), widening the cold-call window; `balanced` (default) does not rerank.
+- **Large vault + intermittent hook timeout (`timed out after 8s`) — FIXED in v0.16.0.** Earlier this was diagnosed as pure cold-start (fresh Bun process, opening a large `index.sqlite`, re-reading evicted index pages) with "give the host more RAM" as the durable fix — but the dominant causes were two code-level defects: (1) the `context-surfacing` vector leg ran a *synchronous* `sqlite-vec` scan that the `Promise.race(vectorTimeout)` guard could not bound (a synchronous call blocks the event loop, so the timer never fires), and (2) every writable hook open ran an unconditional backfill `UPDATE` that could wait out `busy_timeout` under writer contention. **v0.16.0 fixes both:** `searchVec` takes a real wall-clock deadline and self-aborts before the blocking scan; both vector legs race the embed against the remaining budget and clear their timers; the init backfill is read-guarded and the init `busy_timeout` is capped to the caller's value; and the watcher prewarms the sqlite-vec payload into the page cache on startup (embed-independent, watcher-only). A cold page cache still adds latency to the genuine first post-boot call, so host RAM headroom + the prewarm help the margin — but on a large vault the scan cost, not RAM, was the trigger. A modest `timeout` bump (see the tradeoffs table under *Hooks slow or near timeout*) remains a secondary margin. The `deep` profile additionally reranks (extra remote round-trips), widening the cold-call window; `balanced` (default) does not rerank.
 
 **Watcher memory bloat (400MB+)**
 - The watcher accumulates memory when processing high-frequency file change events. The most common trigger was Claude Code session transcript `.jsonl` files changing on every keystroke during active conversations. Each event opened the database briefly, and over hours of active use, memory grew to 400-800MB.
@@ -222,7 +222,7 @@ Common issues when running ClawMem with hooks, MCP server, or OpenClaw plugin. O
 
   The timeout applies per invocation. A slow first prompt (cold start) doesn't mean subsequent prompts will be slow — Bun caches modules after the first load, and `node-llama-cpp` model files are cached on disk after the first download. Subsequent prompts in the same session are typically faster.
 
-  **Exception — memory-constrained hosts / very large vaults:** if the host is under memory pressure (e.g. a WSL2 instance with a low `.wslconfig` `memory` cap) or the vault's `index.sqlite` is large, the OS can evict the cached index + Bun modules *between* turns, so the cold-start cost recurs intermittently rather than once per session — the hook times out on *certain* turns, not just the first. There the durable fix is host memory headroom (so the working set stays cached), not a longer timeout. See *"UserPromptSubmit hook error" (intermittent)* above.
+  **Exception — large vaults (intermittent, pre-v0.16.0):** before v0.16.0 the hook could time out on *certain* turns (not just the first) because of an unbounded synchronous `sqlite-vec` scan plus an init-time write-lock wait — see *"UserPromptSubmit hook error" (intermittent)* above. **Upgrade to v0.16.0**, which bounds the scan and the init path and prewarms the cache. Host RAM headroom + the watcher prewarm still help the genuine cold-call margin, but they were not the root cause.
 
   **Stop hooks** (`decision-extractor`, `handoff-generator`, `feedback-loop`) default to 30s (v0.3.1+, was 10s prior) because they run LLM inference (observer model). These run at session end, so latency doesn't block the user.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.15.1",
+  "version": "0.17.0",
   "description": "On-device memory layer for AI agents. Claude Code, OpenClaw, and Hermes. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/clawmem.ts

@@ -8,6 +8,9 @@ import { existsSync, mkdirSync, readFileSync } from "fs";
 import { resolve as pathResolve, basename } from "path";
 import {
   createStore,
+  prewarmVectors,
+  startPeriodicPrewarm,
+  resolvePrewarmIntervalMs,
   enableProductionMode,
   getDefaultDbPath,
   canonicalDocId,
@@ -1070,6 +1073,15 @@ function printResults(results: Array<{ displayPath: string; title: string; compo
 // Hook dispatch
 // =============================================================================
 
+// B3: the context-surfacing UserPromptSubmit hook runs under a tight budget
+// (8s repo default). Its OWN writes — dedup UPSERT, context_usage, recall
+// events, co-activations — are all best-effort/fail-open, but under writer
+// contention each could otherwise wait up to the store default busy_timeout
+// (5000ms) and blow the budget. Cap this process's busy_timeout so a contended
+// write fails fast (SQLITE_BUSY → skipped by the fail-open guards) instead of
+// stalling. Reads are unaffected (WAL readers never wait on the write lock).
+const CONTEXT_SURFACING_WRITE_BUSY_TIMEOUT_MS = 1500;
+
 async function cmdHook(args: string[]) {
   const hookName = args[0];
   if (!hookName) die("Usage: clawmem hook <name>");
@@ -1081,6 +1093,11 @@ async function cmdHook(args: string[]) {
   try {
     switch (hookName) {
       case "context-surfacing":
+        // Scope the small busy_timeout to THIS process only. Each `clawmem
+        // hook` invocation runs exactly one hook, so the Stop hooks
+        // (decision-extractor / handoff-generator / feedback-loop, 30s budget)
+        // run in separate processes and keep the store default (5000ms).
+        try { s.db.exec(`PRAGMA busy_timeout = ${CONTEXT_SURFACING_WRITE_BUSY_TIMEOUT_MS}`); } catch { /* non-fatal */ }
         output = await contextSurfacing(s, input);
         break;
       case "session-bootstrap":
@@ -1830,6 +1847,7 @@ async function cmdWatch() {
   let stopHeavyLane: (() => Promise<void>) | null = null;
   let watcherHandle: { close: () => void } | null = null;
   let checkpointTimerHandle: Timer | null = null;
+  let prewarmTimerHandle: ReturnType<typeof setInterval> | null = null;
 
   // Graceful shutdown — stop workers, close watchers, then exit. SIGTERM
   // handling is critical for systemd `systemctl --user stop` to shut down
@@ -1838,6 +1856,13 @@ async function cmdWatch() {
   // its own withWorkerLease finally block before we close the store.
   const shutdown = async (signal: string) => {
     console.log(`\n${c.dim}[watch] Received ${signal}, shutting down...${c.reset}`);
+    // Clear the periodic prewarm FIRST — before the awaited worker drains below. The timer is
+    // unref'd but still fires while the loop is alive; a tick landing mid-drain would run the
+    // synchronous ~1.5 GB scan and delay shutdown. Clearing it here is the only guard against that.
+    if (prewarmTimerHandle) {
+      clearInterval(prewarmTimerHandle);
+      prewarmTimerHandle = null;
+    }
     if (stopHeavyLane) {
       await stopHeavyLane();
       stopHeavyLane = null;
@@ -1886,6 +1911,35 @@ async function cmdWatch() {
     stopHeavyLane = startHeavyMaintenanceWorker(s, llm, cfg);
   }
 
+  // Prewarm the sqlite-vec chunks into OS page cache ONCE, in the single long-lived watcher
+  // process only (never in the per-session MCP processes — N concurrent cold scans would be an
+  // I/O storm). The context-surfacing UserPromptSubmit hook runs a SYNCHRONOUS sqlite-vec MATCH
+  // that cannot be time-bounded in-thread (bun:sqlite exposes no interrupt); a cold ~1.5 GB scan
+  // can blow the hook's 8-15s budget. A single warm scan here keeps the hook-path scan sub-second.
+  // Deferred + best-effort so it never delays watcher startup and never throws.
+  setTimeout(() => {
+    try {
+      // prewarmVectors is embed-independent and returns true ONLY when a scan actually ran,
+      // so we never log a false-positive "prewarmed" (embed down at boot / no vectors yet).
+      if (prewarmVectors(s.db)) console.log(`${c.dim}[watch] vector cache prewarmed${c.reset}`);
+    } catch { /* best-effort: unexpected SQL error */ }
+  }, 0);
+
+  // B5 Option C: keep the vector payload warm against OS page-cache eviction BETWEEN hook calls.
+  // The one-shot prewarm above warms once; on a long-running host under memory pressure the kernel
+  // can evict the payload and let a cold synchronous MATCH creep back into the context-surfacing
+  // hook path. Re-touching the pages on an interval biases the kernel LRU toward keeping them
+  // resident (a PROBABILITY reduction, not a hard cap — the hard cap is the deferred BACKLOG
+  // Source 46 daemon). Cleared FIRST in shutdown(); the handle is unref'd so it never keeps the
+  // process alive by itself. resolvePrewarmIntervalMs enforces a strict parse + 60s floor so a
+  // stray tiny value (e.g. "1", or "1e3" which parseInt would read as 1) cannot schedule a
+  // near-continuous scan loop. Set CLAWMEM_PREWARM_INTERVAL_MS=0 to disable. Default 10 min.
+  const prewarmIntervalMs = resolvePrewarmIntervalMs(Bun.env.CLAWMEM_PREWARM_INTERVAL_MS);
+  prewarmTimerHandle = startPeriodicPrewarm(s.db, prewarmIntervalMs);
+  if (prewarmTimerHandle) {
+    console.log(`${c.dim}[watch] periodic vector prewarm every ${Math.round(prewarmIntervalMs / 1000)}s${c.reset}`);
+  }
+
   watcherHandle = startWatcher(dirs, {
     debounceMs: 2000,
     onChanged: async (fullPath, event) => {

package/src/hooks/context-surfacing.ts

@@ -149,6 +149,21 @@ export async function contextSurfacing(
   const tokenBudget = profile.tokenBudget;
   const startTime = Date.now();
 
+  // High-fix (B3): the hook's writes to the MAIN store are bounded by the
+  // busy_timeout cmdHook set for this process (1500ms for context-surfacing).
+  // But skill-vault stores are opened separately via resolveStore(), which
+  // would otherwise use the 5000ms operational default — so a contended
+  // skill-vault write (the recall mirror below) could still stall the hook up
+  // to 5s. Inherit the main store's current cap and pass it to EVERY
+  // skill-vault open so those opens/writes are bounded identically. (Reads are
+  // WAL-safe regardless; this primarily bounds the mirror write.)
+  let hookBusyTimeout = 5000;
+  try {
+    const bt = (store.db.prepare("PRAGMA busy_timeout").get() as { timeout?: number } | undefined)?.timeout;
+    if (typeof bt === "number" && bt > 0) hookBusyTimeout = bt;
+  } catch { /* keep default */ }
+  const skillStoreOpts = { busyTimeout: hookBusyTimeout };
+
   // §11.4: Resolve session-scoped focus topic. Primary signal is the
   // per-session focus file at ~/.cache/clawmem/sessions/<id>.focus
   // (file > env var precedence via resolveSessionTopic). Env var
@@ -183,14 +198,24 @@ export async function contextSurfacing(
   // When vector succeeds, also supplement with FTS for keyword-exact recall
   let results: SearchResult[] = [];
   if (profile.useVector) {
+    let vectorTimer: ReturnType<typeof setTimeout> | undefined;
     try {
-      const vectorPromise = store.searchVec(retrievalQuery, DEFAULT_EMBED_MODEL, maxResults);
-      const timeoutPromise = new Promise<SearchResult[]>((_, reject) =>
-        setTimeout(() => reject(new Error("vector timeout")), profile.vectorTimeout)
-      );
+      // Pass a wall-clock deadline into searchVec: the Promise.race below abandons the vector
+      // promise on timeout but cannot CANCEL it (and cannot interrupt its synchronous scan). The
+      // deadline makes searchVec self-abort before the blocking MATCH, so a slow embed cannot let
+      // an already-timed-out vector leg resume and re-block the hook after it fell back to FTS.
+      const vectorDeadline = Date.now() + profile.vectorTimeout;
+      const vectorPromise = store.searchVec(retrievalQuery, DEFAULT_EMBED_MODEL, maxResults, undefined, undefined, undefined, vectorDeadline);
+      const timeoutPromise = new Promise<SearchResult[]>((_, reject) => {
+        vectorTimer = setTimeout(() => reject(new Error("vector timeout")), profile.vectorTimeout);
+      });
       results = await Promise.race([vectorPromise, timeoutPromise]);
     } catch {
       // Vector search unavailable, timed out, or errored — fall back to BM25
+    } finally {
+      // Clear the timer when the vector promise won the race: a pending (ref'd) setTimeout keeps the
+      // Bun hook process alive for the full vectorTimeout after results are already in hand.
+      if (vectorTimer) clearTimeout(vectorTimer);
     }
   }
 
@@ -211,7 +236,7 @@ export async function contextSurfacing(
   // Dual-query: also search skill vault if configured (secondary source)
   if (getVaultPath("skill")) {
     try {
-      const skillStore = resolveStore("skill");
+      const skillStore = resolveStore("skill", skillStoreOpts);
       const skillResults = skillStore.searchFTS(retrievalQuery, 5);
       // Tag skill vault results for identification in output
       for (const r of skillResults) {
@@ -269,7 +294,21 @@ export async function contextSurfacing(
             if (eq.type === 'lex') {
               hits = store.searchFTS(eq.query, 5);
             } else if (profile.useVector) {
-              try { hits = await store.searchVec(eq.query, DEFAULT_EMBED_MODEL, 5); } catch { /* vector leg non-fatal */ }
+              // Bound BOTH the async embed wait (Promise.race on the remaining 6s budget) AND the
+              // late synchronous MATCH (the deadline arg makes the abandoned promise self-abort
+              // before the scan) — mirroring the balanced leg above. The loop guard only breaks
+              // BETWEEN iterations, so without the race a slow embed here can still blow the budget.
+              const remainingMs = startTime + 6000 - Date.now();
+              if (remainingMs <= 0) break;
+              let deepTimer: ReturnType<typeof setTimeout> | undefined;
+              try {
+                const deepVec = store.searchVec(eq.query, DEFAULT_EMBED_MODEL, 5, undefined, undefined, undefined, startTime + 6000);
+                const deepTimeout = new Promise<SearchResult[]>((_, reject) => {
+                  deepTimer = setTimeout(() => reject(new Error("vector timeout")), remainingMs);
+                });
+                hits = await Promise.race([deepVec, deepTimeout]);
+              } catch { /* vector leg non-fatal (timed out or errored) */ }
+              finally { if (deepTimer) clearTimeout(deepTimer); }  // don't let a pending timer keep the hook process alive
             }
             for (const r of hits) {
               if (!seen.has(r.filepath)) {
@@ -322,7 +361,7 @@ export async function contextSurfacing(
     // expects the collection-relative path, not the full virtual path
     const parsed = r.filepath.startsWith('clawmem://') ? r.filepath.replace(/^clawmem:\/\/[^/]+\/?/, '') : r.filepath;
     // Use the correct store for skill-vault results
-    const targetStore = (r as any)._fromVault === "skill" ? (() => { try { return resolveStore("skill"); } catch { return store; } })() : store;
+    const targetStore = (r as any)._fromVault === "skill" ? (() => { try { return resolveStore("skill", skillStoreOpts); } catch { return store; } })() : store;
     const doc = targetStore.findActiveDocument(r.collectionName, parsed);
     if (!doc) return true;
     if (doc.snoozed_until && new Date(doc.snoozed_until) > now) return false;
@@ -350,7 +389,7 @@ export async function contextSurfacing(
   let enriched = enrichResults(store, generalResults, prompt);
   if (skillResults.length > 0) {
     try {
-      const skillStore = resolveStore("skill");
+      const skillStore = resolveStore("skill", skillStoreOpts);
       enriched = [...enriched, ...enrichResults(skillStore, skillResults, prompt)];
     } catch {
       // Skill store unavailable — enrich with general store as fallback
@@ -476,7 +515,7 @@ export async function contextSurfacing(
           writeRecallEvents(store, input.sessionId, qHash, mappedDocs, validUsageId, turnIndex);
         } else {
           try {
-            const vaultStore = resolveStore(vault);
+            const vaultStore = resolveStore(vault, skillStoreOpts);
             // Mirror context_usage row into named vault for correct FK + attribution
             const vaultPaths = docs.map(r => r.displayPath);
             const vaultUsageId = vaultStore.insertUsage({

package/src/hooks.ts

@@ -213,13 +213,25 @@ export function wasPromptSeenRecently(store: Store, hookName: string, prompt: st
   }
 
   const preview = normalized.slice(0, 120);
-  store.db.prepare(`
-    INSERT INTO hook_dedupe (hook_name, prompt_hash, prompt_preview, last_seen_at)
-    VALUES (?, ?, ?, ?)
-    ON CONFLICT(hook_name, prompt_hash) DO UPDATE SET
-      prompt_preview = excluded.prompt_preview,
-      last_seen_at = excluded.last_seen_at
-  `).run(hookName, hash, preview, nowIso);
+  // Best-effort dedup bookkeeping. Under writer contention this UPSERT can hit
+  // SQLITE_BUSY — especially from the context-surfacing hook, which caps its
+  // busy_timeout low (B3) so its own writes cannot stall the tight
+  // UserPromptSubmit budget. A failed write only means the next identical
+  // prompt won't be suppressed; it is never a reason to throw and abort the
+  // hook. The `recent` verdict comes from the READ above (WAL-safe, does not
+  // wait on the write lock), so same-prompt dedup still works when the row
+  // already exists even if this refresh write is skipped.
+  try {
+    store.db.prepare(`
+      INSERT INTO hook_dedupe (hook_name, prompt_hash, prompt_preview, last_seen_at)
+      VALUES (?, ?, ?, ?)
+      ON CONFLICT(hook_name, prompt_hash) DO UPDATE SET
+        prompt_preview = excluded.prompt_preview,
+        last_seen_at = excluded.last_seen_at
+    `).run(hookName, hash, preview, nowIso);
+  } catch {
+    /* best-effort: contended/failed dedup write must never abort the hook */
+  }
 
   return recent;
 }

package/src/llm.ts

@@ -109,6 +109,14 @@ export type EmbedOptions = {
   model?: string;
   isQuery?: boolean;
   title?: string;
+  /**
+   * Abort signal for the remote embed fetch AND its 429-retry backoff (B4).
+   * The query path passes AbortSignal.timeout(<remaining budget>) so a slow or
+   * rate-limited embed cannot outlive the caller's deadline. Without it, the
+   * hook's Promise.race only ABANDONS the embed promise — the underlying fetch
+   * and retry sleeps keep running; the signal actually CANCELS them.
+   */
+  signal?: AbortSignal;
 };
 
 /**
@@ -697,7 +705,7 @@ export class LlamaCpp implements LLM {
     // Remote server or cloud API — preferred path
     if (this.remoteEmbedUrl && !this.isRemoteEmbedDown()) {
       const extraParams = this.getCloudEmbedParams(!!options.isQuery);
-      const result = await this.embedRemote(text, extraParams);
+      const result = await this.embedRemote(text, extraParams, undefined, options.signal);
       if (result) return result;
       // Cloud providers don't fall back — if API key is set, the user chose cloud
       if (this.isCloudEmbedding()) return null;
@@ -710,6 +718,13 @@ export class LlamaCpp implements LLM {
     // Remote is in cooldown or was never configured — try local fallback
     if (this.remoteEmbedUrl && this.isRemoteEmbedDown()) {
       if (process.env.CLAWMEM_NO_LOCAL_MODELS === "true") return null;
+      // Medium-fix (B4): a deadline-bounded caller (query path, signal set)
+      // cannot afford a local model load/download during a remote cooldown —
+      // embedLocal ignores the abort signal and can run for seconds/minutes.
+      // Skip the local fallback and let the caller degrade (searchVec → [] →
+      // FTS). Pure-local mode (no remoteEmbedUrl) never enters this branch, so
+      // local-only deployments still embed.
+      if (options.signal) return null;
       this.noteRemoteFallback(
         "embed",
         this.isLoopbackUrl(this.remoteEmbedUrl)
@@ -952,22 +967,47 @@ export class LlamaCpp implements LLM {
     return Math.floor(delayMs * (0.75 + Math.random() * 0.5));
   }
 
-  private async embedRemote(text: string, extraParams: Record<string, unknown> = {}, retries = 5): Promise<EmbeddingResult | null> {
+  /**
+   * Sleep for `ms`, resolving early if `signal` aborts. Returns true if the
+   * wait was cut short by an abort (caller should stop retrying), false if it
+   * slept the full duration. Without this, a 429 backoff (up to 30s) would run
+   * to completion even after the caller's deadline elapsed (B4).
+   */
+  private async abortableDelay(ms: number, signal?: AbortSignal): Promise<boolean> {
+    if (signal?.aborted) return true;
+    if (!signal) {
+      await new Promise(r => setTimeout(r, ms));
+      return false;
+    }
+    return await new Promise<boolean>((resolve) => {
+      const onAbort = () => { clearTimeout(timer); resolve(true); };
+      const timer = setTimeout(() => {
+        signal.removeEventListener("abort", onAbort);
+        resolve(false);
+      }, ms);
+      signal.addEventListener("abort", onAbort, { once: true });
+    });
+  }
+
+  private async embedRemote(text: string, extraParams: Record<string, unknown> = {}, retries = 5, signal?: AbortSignal): Promise<EmbeddingResult | null> {
     if (this.isRemoteEmbedDown()) return null;
     const input = this.truncateForEmbed(text);
     for (let attempt = 0; attempt < retries; attempt++) {
+      if (signal?.aborted) return null; // caller deadline already elapsed — do not start another attempt
       try {
         const body: Record<string, unknown> = { input, model: this.remoteEmbedModel, ...extraParams };
         const resp = await fetch(`${this.remoteEmbedUrl}/v1/embeddings`, {
           method: "POST",
           headers: this.getEmbedHeaders(),
           body: JSON.stringify(body),
+          signal,
         });
         if (resp.status === 429) {
           const retryAfter = this.parseRetryAfter(resp);
           const delay = retryAfter ?? Math.min(1000 * 2 ** attempt, 30000);
-          console.error(`Remote embed rate-limited, retry ${attempt + 1}/${retries} in ${this.jitter(delay)}ms`);
-          await new Promise(r => setTimeout(r, this.jitter(delay)));
+          const jittered = this.jitter(delay);
+          console.error(`Remote embed rate-limited, retry ${attempt + 1}/${retries} in ${jittered}ms`);
+          if (await this.abortableDelay(jittered, signal)) return null; // deadline elapsed during backoff
           continue;
         }
         if (!resp.ok) {
@@ -983,6 +1023,13 @@ export class LlamaCpp implements LLM {
           model: data.model || this.remoteEmbedUrl!,
         };
       } catch (error) {
+        // An abort/timeout is an intentional caller-driven cancellation (the
+        // query-path deadline), NOT a transport failure — do not trip the 60s
+        // remote-down cooldown, which would needlessly force local fallback.
+        const name = (error as { name?: string })?.name;
+        if (signal?.aborted || name === "AbortError" || name === "TimeoutError") {
+          return null;
+        }
         if (this.isTransportError(error)) {
           this.markRemoteEmbedDown();
         } else {

package/src/store.ts

@@ -365,7 +365,7 @@ function loadVecExtension(db: Database): void {
   }
 }
 
-function initializeDatabase(db: Database): void {
+function initializeDatabase(db: Database, busyTimeoutMs: number = 15000): void {
   // Set busy_timeout FIRST so subsequent PRAGMAs (journal_mode in particular,
   // which acquires a write lock when switching or initializing WAL state) wait
   // instead of returning SQLITE_BUSY when concurrent Stop hooks
@@ -373,10 +373,12 @@ function initializeDatabase(db: Database): void {
   // before_reset hook fan-out in src/openclaw/engine.ts — open the DB
   // simultaneously. busy_timeout is a connection-level setting that only
   // governs *subsequent* statements (default busy handler is NULL → SQLITE_BUSY
-  // returns immediately), so it must precede the contending PRAGMAs. 15s is
-  // well within the 30s Stop hook timeout. createStore() resets to operational
+  // returns immediately), so it must precede the contending PRAGMAs. The init
+  // busy_timeout defaults to 15s (well within the 30s Stop hook timeout) but is
+  // capped to the caller's opts.busyTimeout — hook opens pass 5000 so init cannot
+  // wait out the 8-15s UserPromptSubmit budget. createStore() resets to operational
   // value (5000ms or opts.busyTimeout) after DDL completes. Issue #13.
-  db.exec("PRAGMA busy_timeout = 15000");
+  db.exec(`PRAGMA busy_timeout = ${busyTimeoutMs}`);
   loadVecExtension(db);
   db.exec("PRAGMA journal_mode = WAL");
   db.exec("PRAGMA foreign_keys = ON");
@@ -450,9 +452,17 @@ function initializeDatabase(db: Database): void {
     }
   }
 
-  // Backfill last_accessed_at from modified_at for existing docs
+  // Backfill last_accessed_at from modified_at for existing docs.
+  // Guarded by a read first: on an already-backfilled DB the UPDATE is skipped, so a writable
+  // open takes NO write lock here and cannot wait on busy_timeout under concurrent writers.
+  // (The unconditional UPDATE previously ran on EVERY writable open — including the
+  // context-surfacing UserPromptSubmit hook — and could block up to busy_timeout when another
+  // process held the write lock, pushing the hook past its 8-15s deadline.)
   try {
-    db.exec(`UPDATE documents SET last_accessed_at = modified_at WHERE last_accessed_at IS NULL`);
+    const needsBackfill = db.prepare(`SELECT 1 FROM documents WHERE last_accessed_at IS NULL LIMIT 1`).get();
+    if (needsBackfill) {
+      db.exec(`UPDATE documents SET last_accessed_at = modified_at WHERE last_accessed_at IS NULL`);
+    }
   } catch { /* ignore if already backfilled */ }
 
   db.exec(`CREATE INDEX IF NOT EXISTS idx_documents_collection ON documents(collection, active)`);
@@ -1136,6 +1146,83 @@ export function getVecTableDim(db: Database): number | null {
   return readVecTableDim(db);
 }
 
+/**
+ * Prewarm the sqlite-vec payload into OS page cache with a single brute-force MATCH using a ZERO
+ * query vector. Decoupled from the embedding server on purpose — it works when the embed server is
+ * down at boot or CLAWMEM_NO_LOCAL_MODELS=true, unlike a searchVec()-based prewarm (which embeds
+ * first and would silently no-op). Returns true ONLY if a scan actually ran (a vector table with a
+ * known dimension exists), so callers never log a false-positive "warmed". The k-NN MATCH is
+ * brute-force, so it touches every vector chunk — exactly the payload we want cache-resident.
+ */
+export function prewarmVectors(db: Database): boolean {
+  const dim = getVecTableDim(db);
+  if (!dim || dim <= 0) return false;
+  const zero = new Float32Array(dim);
+  db.prepare(`SELECT hash_seq FROM vectors_vec WHERE embedding MATCH ? AND k = ?`).all(zero, 1);
+  return true;
+}
+
+/**
+ * Keep the sqlite-vec payload resident in the OS page cache by re-running the brute-force
+ * prewarm on an interval. The one-shot prewarm at watcher startup warms the cache ONCE; on a
+ * long-running host under memory pressure the kernel can evict the (potentially ~1.5 GB) vector
+ * payload between hook calls, and the next cold SYNCHRONOUS MATCH in the context-surfacing hook
+ * path can then blow the 8-15s hook budget (bun:sqlite exposes no interrupt, so an in-flight
+ * scan cannot be abandoned). Re-touching the pages biases the kernel LRU toward keeping them
+ * resident — a PROBABILITY reduction, NOT a hard cap. The hard cap (moving the blocking scan off
+ * the hook's event loop so its deadline can fire) is the deferred BACKLOG Source 46 daemon.
+ *
+ * Best-effort: never throws; a per-tick failure is swallowed so the timer keeps running. Returns
+ * the interval handle (the caller MUST clear it on shutdown) or null when disabled (intervalMs
+ * <= 0 or non-finite). The handle is unref'd so it never by itself keeps the process alive.
+ * `onPrewarm(ran)` is an optional observability hook fired after each attempt — `ran` is whether
+ * a scan actually executed (i.e. a dimensioned vector table exists).
+ */
+export function startPeriodicPrewarm(
+  db: Database,
+  intervalMs: number,
+  onPrewarm?: (ran: boolean) => void,
+): ReturnType<typeof setInterval> | null {
+  if (!Number.isFinite(intervalMs) || intervalMs <= 0) return null;
+  const timer = setInterval(() => {
+    let ran = false;
+    try { ran = prewarmVectors(db); } catch { /* best-effort: unexpected SQL error */ }
+    if (onPrewarm) { try { onPrewarm(ran); } catch { /* observer must never break the timer */ } }
+  }, intervalMs);
+  (timer as { unref?: () => void }).unref?.();
+  return timer;
+}
+
+/** Floor for the periodic re-prewarm interval. Below this, a large-vault (~1.5 GB) brute-force scan
+ *  runs near-continuously and can saturate the watcher event loop + I/O, so any smaller positive
+ *  request is clamped UP to this value. */
+export const PREWARM_MIN_INTERVAL_MS = 60_000;
+/** Default periodic re-prewarm interval when CLAWMEM_PREWARM_INTERVAL_MS is unset or unparseable. */
+export const PREWARM_DEFAULT_INTERVAL_MS = 600_000;
+
+/**
+ * Resolve the raw CLAWMEM_PREWARM_INTERVAL_MS env value into a SAFE interval for the watcher. This
+ * policy is kept OUT of the permissive `startPeriodicPrewarm` mechanism (so unit tests can still use
+ * tiny intervals) and applied only on the production env path.
+ *   - unset / empty / unparseable → default (600000). Garbage must NOT silently disable the
+ *     mitigation, nor be read as a tiny interval.
+ *   - exactly 0 → 0 (the documented off switch; `startPeriodicPrewarm` then returns null).
+ *   - negative → default (nonsensical; neither an intentional disable nor a fast loop).
+ *   - 0 < n < floor → clamped UP to the 60s floor. Prevents the near-continuous scan loop that e.g.
+ *     "1" or "1e3" would otherwise schedule. NOTE: `Number("1e3") === 1000` whereas
+ *     `parseInt("1e3", 10) === 1`, so `Number()` is used deliberately (parseInt silently truncates
+ *     at the "e").
+ *   - n >= floor → floored to an integer and used as-is.
+ */
+export function resolvePrewarmIntervalMs(raw: string | undefined): number {
+  if (raw === undefined || raw.trim() === "") return PREWARM_DEFAULT_INTERVAL_MS;
+  const n = Number(raw);
+  if (!Number.isFinite(n)) return PREWARM_DEFAULT_INTERVAL_MS;
+  if (n === 0) return 0;
+  if (n < 0) return PREWARM_DEFAULT_INTERVAL_MS;
+  return Math.max(PREWARM_MIN_INTERVAL_MS, Math.floor(n));
+}
+
 /**
  * The DISTINCT non-empty embedding models stored in the vault (empty array if no
  * embeddings exist). Used to detect model drift BETWEEN runs — a different model at
@@ -1204,7 +1291,7 @@ export type Store = {
 
   // Search
   searchFTS: (query: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }) => SearchResult[];
-  searchVec: (query: string, model: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }) => Promise<SearchResult[]>;
+  searchVec: (query: string, model: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }, deadlineMs?: number) => Promise<SearchResult[]>;
 
   // Query expansion & reranking
   expandQuery: (query: string, model?: string, intent?: string) => Promise<ExpandedQuery[]>;
@@ -1339,7 +1426,7 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
     ? new Database(resolvedPath, { readonly: true })
     : new Database(resolvedPath);
   if (!opts?.readonly) {
-    initializeDatabase(db);
+    initializeDatabase(db, opts?.busyTimeout ?? 15000);
   } else {
     // Readonly: set busy_timeout FIRST so the journal_mode PRAGMA below
     // doesn't race when concurrent processes open the DB. PRAGMA
@@ -1352,7 +1439,7 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
     db.exec("PRAGMA journal_mode = WAL");
     db.exec("PRAGMA query_only = ON");
   }
-  // For the writable branch: initializeDatabase() set 15000 during DDL —
+  // For the writable branch: initializeDatabase() set opts.busyTimeout (default 15000) during DDL —
   // reset to operational value here. For readonly: already set inside the
   // branch above; this assignment is a no-op rewrite to the same value.
   db.exec(`PRAGMA busy_timeout = ${opts?.busyTimeout ?? 5000}`);
@@ -1398,7 +1485,7 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
 
     // Search
     searchFTS: (query: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }) => searchFTS(db, query, limit, collectionId, collections, dateRange),
-    searchVec: (query: string, model: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }) => searchVec(db, query, model, limit, collectionId, collections, dateRange),
+    searchVec: (query: string, model: string, limit?: number, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }, deadlineMs?: number) => searchVec(db, query, model, limit, collectionId, collections, dateRange, deadlineMs),
 
     // Query expansion & reranking
     expandQuery: (query: string, model?: string, intent?: string) => expandQuery(query, model, db, intent),
@@ -3361,13 +3448,19 @@ export function searchFTS(db: Database, query: string, limit: number = 20, colle
 // Vector Search
 // =============================================================================
 
-export async function searchVec(db: Database, query: string, model: string, limit: number = 20, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }): Promise<SearchResult[]> {
+export async function searchVec(db: Database, query: string, model: string, limit: number = 20, collectionId?: number, collections?: string[], dateRange?: { start: string; end: string }, deadlineMs?: number): Promise<SearchResult[]> {
   const tableExists = db.prepare(`SELECT name FROM sqlite_master WHERE type='table' AND name='vectors_vec'`).get();
   if (!tableExists) return [];
 
-  const embedding = await getEmbedding(query, model, true);
+  const embedding = await getEmbedding(query, model, true, deadlineMs);
   if (!embedding) return [];
 
+  // Guard-defect fix: the caller's Promise.race(vectorTimeout) cannot interrupt the SYNCHRONOUS
+  // sqlite-vec MATCH below (bun:sqlite blocks the event loop) and does NOT cancel this promise.
+  // If the wall-clock budget already elapsed during the async embed above, bail here so a
+  // timed-out vector leg cannot resume and re-block the hook after it fell back to FTS.
+  if (deadlineMs !== undefined && Date.now() >= deadlineMs) return [];
+
   // IMPORTANT: We use a two-step query approach here because sqlite-vec virtual tables
   // hang indefinitely when combined with JOINs in the same query. Do NOT try to
   // "optimize" this by combining into a single query with JOINs - it will break.
@@ -3467,11 +3560,22 @@ export async function searchVec(db: Database, query: string, model: string, limi
 // Embeddings
 // =============================================================================
 
-async function getEmbedding(text: string, model: string, isQuery: boolean): Promise<number[] | null> {
+async function getEmbedding(text: string, model: string, isQuery: boolean, deadlineMs?: number): Promise<number[] | null> {
   const llm = getDefaultLlamaCpp();
   // Format text using the appropriate prompt template
   const formattedText = isQuery ? formatQueryForEmbedding(text) : formatDocForEmbedding(text);
-  const result = await llm.embed(formattedText, { model, isQuery });
+  // B4: bound the remote embed fetch + its 429 backoff to the caller's wall-clock
+  // deadline. Under the context-surfacing hook's Promise.race the abandoned embed
+  // promise otherwise keeps its fetch + retry sleeps running; AbortSignal.timeout
+  // actually cancels them, so a slow/rate-limited embed can no longer outlive the
+  // hook budget.
+  let signal: AbortSignal | undefined;
+  if (deadlineMs !== undefined) {
+    const remaining = deadlineMs - Date.now();
+    if (remaining <= 0) return null; // deadline already elapsed — skip the embed entirely
+    signal = AbortSignal.timeout(remaining);
+  }
+  const result = await llm.embed(formattedText, { model, isQuery, signal });
   return result?.embedding || null;
 }