wazap-mcp 0.18.2 → 0.18.4

package/README.md

@@ -8,7 +8,7 @@
 ```
 
 **WhatsApp for your AI agent.** An MCP server that puts your WhatsApp account —
-chats, messages, media, contacts, groups — behind 33 tools any MCP client can
+chats, messages, media, contacts, groups — behind 34 tools any MCP client can
 call. Pairing-code login, no browser, no phone-number reseller, ~20 MB of RAM.
 
 Built on [Baileys](https://github.com/WhiskeySockets/Baileys), which speaks the
@@ -284,6 +284,7 @@ them. `--dry-run` prints the plan and touches nothing.
 | `get_stories` | read | The stories (status updates) received in the last day, by author, with previews on request. They show nowhere else. |
 | `wait_for_messages` | read | Block up to 55 s until a message arrives, then return it with a cursor for the next call. `addressed_to_me` wakes only for direct messages, @-mentions and replies. |
 | `search_messages` | read | Text search across the locally held messages; `since`, `until` and `from` narrow it. |
+| `recall` | read | Semantic search over the whole indexed history: matches by meaning, so a paraphrase or another language still hits, and it finds messages too old for `search_messages`. Off until [turned on](#semantic-recall). |
 | `get_message` | read | One message in full, with its quoted message and reactions. |
 | `search_contacts` | read | Find contacts by name or number. |
 | `sync_contacts` | read | Fetch the phone's address book from WhatsApp again, when names are missing. |
@@ -423,6 +424,41 @@ transcribed once and not again after a restart. Audio *files* are left alone,
 since one can be an hour long; call `transcribe_audio(message_id)` for those.
 `WAZAP_TRANSCRIBE_AUTO=0` keeps the tool and stops the background work.
 
+## Semantic recall
+
+`search_messages` matches exact words; `recall` matches what was meant. A
+paraphrase or another language still hits, and it keeps finding messages too
+old for `search_messages` to see — those come back marked `index only`, which
+`get_message` and `download_media` cannot open. For an exact string — an id,
+a phone number, a URL — `search_messages` stays the right tool.
+
+Off by default, and fully local: a `llama-server` sidecar bound to loopback
+does the embedding, so nothing leaves the machine. It needs llama.cpp, the
+pinned model and persisted history (`WAZAP_PERSIST_HISTORY`, on by default):
+
+```bash
+brew install llama.cpp      # macOS; elsewhere build llama.cpp and put llama-server on PATH
+wazap embed download        # fetch the embedding model, ~318 MB sha256-verified
+wazap config recall local   # then restart the service
+```
+
+`wazap embed download` offers the `brew install` itself when `llama-server`
+is missing. `wazap status` runs the three checks — `recall`, `llama-server`,
+`embed model` — and `get_status` reports the index as `off`, `indexing`,
+`ready` or `degraded`.
+
+`chat_id`, `since`, `until` and `from` narrow a recall exactly like
+`search_messages`. Hits rank by similarity scaled by recency, and anything
+under the similarity floor is dropped rather than listed. The index lives at
+`accounts/<id>/recall/` (dir `0700`, files `0600`), holds only what persisted
+history already stores, and tombstones a message out when it is deleted or
+revoked.
+
+The knobs — `WAZAP_RECALL`, `WAZAP_EMBED_MODEL` (`embeddinggemma-300m` by
+default, `e5-base-multilingual` for an older llama.cpp), `WAZAP_EMBED_BIN`,
+`WAZAP_RECALL_MAX`, `WAZAP_RECALL_MIN_SIMILARITY` — are documented in
+`.env.example`.
+
 ## Skills
 
 wazap ships five [Agent Skills](https://agentskills.io) that teach an agent the workflows behind the tools, not just the tools:
@@ -512,11 +548,12 @@ accounts moves into `accounts/default/` the first time a wazap command runs.
     auth/           WhatsApp credentials — treat this like a password
     media/          downloads from download_media
     history/        per-chat message history, so a restart is not amnesia
+    recall/         the semantic index, when recall is on
     previews/       one small JPEG per photo or video already previewed
     notes.json      notes on contacts and "handled" marks; never sent anywhere
     store.json      chat-list snapshot
     qr.png          last QR, when login showed one
-  models/           whisper.cpp models, when transcription runs locally
+  models/           whisper.cpp and embedding models, when transcription or recall run locally
   server.lock       pid of the running server
   daemon.json       loopback endpoint a second wazap bridges to
   oauth.json        registered agents and hashed OAuth grants, when OAuth is on

package/dist/cli.js

@@ -4,14 +4,11 @@ import { mkdirSync } from "node:fs";
 import { createInterface } from "node:readline/promises";
 import { setTimeout as sleep } from "node:timers/promises";
 import { DisconnectReason } from "baileys";
-import qrcode from "qrcode";
-import qrcodeTerminal from "qrcode-terminal";
 import { accountRows, describeAccount, describeStatusAccount } from "./account-cli.js";
 import { AccountHub } from "./account-hub.js";
 import { AccountRegistry, resolveAccount } from "./accounts.js";
 import { clearSession, readLinkedAccount } from "./auth-state.js";
 import { banner } from "./banner.js";
-import { runBridge } from "./bridge.js";
 import { BAILEYS_VERSION, WAZAP_VERSION, paths } from "./config.js";
 import { connectNext, whereInstalled } from "./connect.js";
 import { decideRole, readDaemon, removeDaemon, writeDaemon } from "./daemon.js";
@@ -460,6 +457,9 @@ export async function runServe(config) {
             process.exit(2);
         }
         if (role.kind === "bridge") {
+            // Bridging onto a running daemon is the rare role; the MCP client stack
+            // it needs stays out of the common path until then.
+            const { runBridge } = await import("./bridge.js");
             await runBridge(role.daemon, p.daemonFile);
             return;
         }
@@ -717,6 +717,11 @@ async function linkByCode(authDir, phone, waiting, w) {
     return pairing.done;
 }
 async function linkByQr(p, waiting, w) {
+    // The QR drawers are only needed by this one login path, not by serve.
+    const [{ default: qrcode }, { default: qrcodeTerminal }] = await Promise.all([
+        import("qrcode"),
+        import("qrcode-terminal"),
+    ]);
     if (w) {
         await w.next("Scan this with WhatsApp");
         waiting.start("Waiting for a QR from WhatsApp…");

package/dist/doctor.js

@@ -313,8 +313,11 @@ async function checkRecall(config) {
     const spec = EMBED_MODELS[settings.model];
     const size = fileSize(embedModelPath(settings.modelsDir, spec));
     const readiness = await embedReady(settings, spec);
+    // An env-set floor is the user's own calibration; only the model's
+    // unmeasured default gets flagged.
+    const uncalibrated = !spec.floorCalibrated && process.env.WAZAP_RECALL_MIN_SIMILARITY === undefined ? ", uncalibrated floor" : "";
     return [
-        { name: "recall", state: "ok", detail: `local (${settings.model})` },
+        { name: "recall", state: "ok", detail: `local (${settings.model}${uncalibrated})` },
         readiness.ok
             ? { name: "llama-server", state: "ok", detail: settings.embedUrl ?? "found" }
             : { name: "llama-server", state: "fail", detail: readiness.detail, fix: readiness.fix },

package/dist/errors.js

@@ -44,6 +44,7 @@ export const ERROR_GUIDE = {
     TRANSCRIBE_FAILED: "The transcription provider ran and failed. Read the message; retry once at most.",
     RECALL_UNAVAILABLE: "Semantic recall is off, or llama.cpp or the embedding model is missing. Tell the user to run the command in the fix; do not retry.",
     RECALL_FAILED: "The embedding backend ran and failed. Read the message; retry once at most.",
+    RECALL_BAD_INPUT: "The embedding server refused the input itself. Do not retry it unchanged.",
     TIMEOUT: "WhatsApp did not answer in time. Retry once; if it fails again, call get_status.",
     SERVICE_ERROR: "wazap's own background service could not be managed. This is a machine problem, not a WhatsApp one: read the fix and tell the user.",
     DRAFT_NOT_FOUND: "That draft_id is unknown or was already sent. Call the send tool again to draft, show the new preview, then confirm_send.",

package/dist/recall/engine.js

@@ -1,7 +1,13 @@
 /**
  * The embedding backend: one `llama-server --embedding` child bound to
- * loopback, restarted when it dies, killed when wazap stops. The queue is the
- * only caller, so a slow restart stalls indexing — never ingestion.
+ * loopback, restarted when it dies, reaped when idle, killed when wazap
+ * stops. The queue is the only caller, so a slow restart stalls indexing —
+ * never ingestion.
+ *
+ * The /embedding API is stateless, so every account in the process shares one
+ * server: the registry below spawns on the first acquire and kills on the
+ * last release, keyed by the only things that make a sidecar distinct — the
+ * resolved binary and the model file.
  *
  * WAZAP_EMBED_URL points at an already-running compatible server instead of
  * spawning one; that seam exists for the test stub, not as a supported option.
@@ -204,6 +210,133 @@ class UrlBackend {
         return Promise.resolve();
     }
 }
+/** The seam the tests replace; production always spawns a real llama-server. */
+export const sidecarFactory = {
+    open: (bin, model, onLog) => new LlamaSidecar(bin, model, onLog),
+};
+const sharedSidecars = new Map();
+/**
+ * The claim side of the registry: an existing entry is joined, a missing one
+ * is spawned. A failed start frees the slot, so the next claim spawns fresh
+ * instead of joining a rejection; a successful one starts the idle clock.
+ * Consumers still holding refs on a failed or reaped entry release into a
+ * stopped sidecar — stop() on it is a safe no-op.
+ */
+function claimEntry(key, bin, model, onLog, idleMs) {
+    let entry = sharedSidecars.get(key);
+    if (entry === undefined) {
+        const sidecar = sidecarFactory.open(bin, model, onLog);
+        entry = { refs: 0, sidecar, started: sidecar.start(), idleMs, idleTimer: null, reaped: false };
+        sharedSidecars.set(key, entry);
+        const spawned = entry;
+        spawned.started.then(() => touchIdle(key, spawned), () => {
+            if (sharedSidecars.get(key) === spawned)
+                sharedSidecars.delete(key);
+        });
+    }
+    entry.refs++;
+    return entry;
+}
+/**
+ * The idle clock only runs while the server is up and unused: every embed
+ * through the entry re-arms it, and an entry the registry no longer holds —
+ * stopped or already reaped — is never armed.
+ */
+function touchIdle(key, entry) {
+    if (entry.idleMs <= 0 || entry.reaped)
+        return;
+    if (sharedSidecars.get(key) !== entry)
+        return;
+    if (entry.idleTimer !== null)
+        clearTimeout(entry.idleTimer);
+    entry.idleTimer = setTimeout(() => reapIdle(key, entry), entry.idleMs);
+    entry.idleTimer.unref();
+}
+/**
+ * Idle means nobody is calling embed — the claims themselves can stay. The
+ * entry is evicted and the child killed while consumers still hold it; their
+ * next embed finds the slot empty and claims a fresh spawn.
+ */
+function reapIdle(key, entry) {
+    entry.idleTimer = null;
+    entry.reaped = true;
+    if (sharedSidecars.get(key) === entry)
+        sharedSidecars.delete(key);
+    entry.sidecar.stop().catch(() => { });
+}
+/**
+ * One consumer's claim on a shared sidecar. Embedding calls reach the child
+ * through waitReady(); stop() only gives this claim back — the last one out
+ * is the one that kills the server. The child's restart lines keep arriving
+ * through the first consumer's onLog, which in production is the same `log`
+ * for everyone.
+ *
+ * An idle reap can evict the entry out from under a live claim; the next
+ * embed then re-claims the key — joining a respawn already under way or
+ * starting one — so a live engine always reaches a running server.
+ */
+class SharedSidecar {
+    key;
+    bin;
+    model;
+    onLog;
+    idleMs;
+    released = false;
+    entry;
+    constructor(key, bin, model, onLog, idleMs) {
+        this.key = key;
+        this.bin = bin;
+        this.model = model;
+        this.onLog = onLog;
+        this.idleMs = idleMs;
+        this.entry = claimEntry(key, bin, model, onLog, idleMs);
+    }
+    /** First claim on a key spawns; later ones join the start already under way. */
+    static acquire(bin, model, onLog, idleMs) {
+        return new SharedSidecar(`${bin}\n${model}`, bin, model, onLog, idleMs);
+    }
+    /** The shared start; the engine releases its claim when this rejects. */
+    started() {
+        return this.entry.started;
+    }
+    get base() {
+        return this.entry.sidecar.base;
+    }
+    /**
+     * The per-embed heartbeat on the registry: a reaped entry is swapped for a
+     * live claim and the idle clock restarts before the health gate runs.
+     */
+    async waitReady() {
+        const entry = this.liveEntry();
+        touchIdle(this.key, entry);
+        await entry.started;
+        await entry.sidecar.waitReady();
+    }
+    /** The claim's entry while the registry holds it; a fresh claim once it was evicted. */
+    liveEntry() {
+        if (!this.released && sharedSidecars.get(this.key) !== this.entry) {
+            this.entry = claimEntry(this.key, this.bin, this.model, this.onLog, this.idleMs);
+        }
+        return this.entry;
+    }
+    async stop() {
+        if (this.released)
+            return;
+        this.released = true;
+        const entry = this.entry;
+        entry.refs--;
+        // A reaped entry was already evicted and stopped; only the claim is left.
+        if (entry.refs > 0 || entry.reaped)
+            return;
+        if (entry.idleTimer !== null) {
+            clearTimeout(entry.idleTimer);
+            entry.idleTimer = null;
+        }
+        if (sharedSidecars.get(this.key) === entry)
+            sharedSidecars.delete(this.key);
+        await entry.sidecar.stop();
+    }
+}
 /**
  * The queue's handle on embeddings. `embed` waits out a restart for up to
  * START_TIMEOUT_MS, then fails the batch — the queue retries, so a dying
@@ -225,9 +358,17 @@ export class EmbedEngine {
         if (bin === null) {
             throw new WazapError("RECALL_UNAVAILABLE", "llama-server not found; semantic recall needs llama.cpp", llamaInstallFix());
         }
-        const sidecar = new LlamaSidecar(bin, embedModelPath(settings.modelsDir, spec), onLog);
-        await sidecar.start();
-        return new EmbedEngine(sidecar, spec);
+        // Accounts on the same binary and model share one server: acquire bumps
+        // the registry's refcount, this engine's stop() hands just this claim back.
+        const target = SharedSidecar.acquire(bin, embedModelPath(settings.modelsDir, spec), onLog, settings.embedIdleMs);
+        try {
+            await target.started();
+        }
+        catch (err) {
+            await target.stop();
+            throw err;
+        }
+        return new EmbedEngine(target, spec);
     }
     /**
      * `kind` picks the model's task prefix: the index holds "document" texts,
@@ -253,7 +394,11 @@ export class EmbedEngine {
         }
         if (!response.ok) {
             const body = (await response.text().catch(() => "")).slice(0, 300);
-            throw new WazapError("RECALL_FAILED", `embedding server answered ${response.status}: ${body}`);
+            // A 4xx means the input itself is unembeddable — over the model's
+            // context, malformed — and no retry will change that, so the queue
+            // treats it differently from a sick backend.
+            const code = response.status >= 400 && response.status < 500 ? "RECALL_BAD_INPUT" : "RECALL_FAILED";
+            throw new WazapError(code, `embedding server answered ${response.status}: ${body}`);
         }
         const reply = (await response.json());
         if (!Array.isArray(reply)) {

package/dist/recall/models.js

@@ -24,6 +24,11 @@ export const EMBED_MODELS = {
         url: "https://huggingface.co/ggml-org/embeddinggemma-300M-GGUF/resolve/main/embeddinggemma-300M-Q8_0.gguf",
         // EmbeddingGemma's own retrieval task, from its model card.
         prompts: { query: "task: search result | query: ", document: "title: none | text: " },
+        maxChars: 2048,
+        // Measured on a real index under those prompts: noise tops out ~0.31,
+        // real paraphrases start ~0.35.
+        defaultMinSimilarity: 0.35,
+        floorCalibrated: true,
     },
     "e5-base-multilingual": {
         alias: "e5-base-multilingual",
@@ -34,6 +39,17 @@ export const EMBED_MODELS = {
         url: "https://huggingface.co/dinab/multilingual-e5-base-Q8_0-GGUF/resolve/main/multilingual-e5-base-q8_0.gguf",
         // e5's documented asymmetric prefixes.
         prompts: { query: "query: ", document: "passage: " },
+        // e5's context window is 512 tokens; the cap leaves headroom for the
+        // document prefix and diacritic-heavy text.
+        maxChars: 800,
+        // Measured on a real 12k-message index: noise tops out ~0.84 while real
+        // paraphrases run 0.82-0.88 — the bands overlap, so no floor separates
+        // cleanly. 0.85 deliberately errs toward silence: a missed hit answers
+        // "nothing found", a false hit is a wrong memory an agent will repeat.
+        // Gemma stays the recommended model; e5 remains the fallback for a
+        // llama.cpp too old for gemma embeddings.
+        defaultMinSimilarity: 0.85,
+        floorCalibrated: true,
     },
 };
 export function embedModelSpec(name) {

package/dist/recall/queue.js

@@ -10,6 +10,7 @@
  * by the put it deletes; inside one batch the last op per sid wins.
  */
 import { setTimeout as sleep } from "node:timers/promises";
+import { WazapError } from "../errors.js";
 import { logError } from "../logger.js";
 /** Texts per embedding request; llama.cpp pools them in one pass. */
 const BATCH = 32;
@@ -167,10 +168,33 @@ export class RecallQueue {
         const puts = ops.flatMap((op) => (op.item === undefined ? [] : [op.item]));
         if (dels.length > 0)
             await this.store.remove(dels);
+        await this.commitPuts(puts);
+    }
+    /**
+     * A 4xx from the embed server names the input, not the backend — one text
+     * over the model's context window fails the whole request and would retry
+     * forever. The batch is bisected until only the bad text is left, and that
+     * single message is dropped: the index loses one unembeddable entry rather
+     * than dying behind a poison pill that every restart would re-feed.
+     */
+    async commitPuts(puts) {
         if (puts.length === 0)
             return;
-        const vectors = await this.embed(puts.map((item) => item.text));
-        await this.store.add(puts, vectors);
+        try {
+            const vectors = await this.embed(puts.map((item) => item.text));
+            await this.store.add(puts, vectors);
+        }
+        catch (err) {
+            if (!(err instanceof WazapError && err.code === "RECALL_BAD_INPUT"))
+                throw err;
+            if (puts.length === 1) {
+                logError(`recall index: dropping unembeddable message ${puts[0].sid}`, err);
+                return;
+            }
+            const mid = Math.ceil(puts.length / 2);
+            await this.commitPuts(puts.slice(0, mid));
+            await this.commitPuts(puts.slice(mid));
+        }
     }
     async applySeals() {
         for (const [file, seal] of this.seals) {

package/dist/recall/settings.js

@@ -6,18 +6,13 @@
 import { join } from "node:path";
 import { WazapError } from "../errors.js";
 import { stripPasted } from "../transcribe/index.js";
+import { EMBED_MODELS } from "./models.js";
 const OFF = new Set(["", "off", "0", "no", "none", "false"]);
 const ON = new Set(["local", "on", "1", "yes", "true"]);
 const MODEL_ALIASES = ["embeddinggemma-300m", "e5-base-multilingual"];
 const DEFAULT_MAX_ROWS = 50_000;
 const MIN_MAX_ROWS = 100;
-/**
- * Cosine floor for embeddinggemma-300m under its task prompts, measured on a
- * real index: noise tops out ~0.31, real paraphrases start ~0.35. The prompts
- * widened the noise/signal gap enough for the floor to mean something.
- * e5-base needs its own calibration.
- */
-const DEFAULT_MIN_SIMILARITY = 0.35;
+const DEFAULT_EMBED_IDLE_MINUTES = 30;
 function parseEnabled(raw) {
     const value = stripPasted(raw ?? "").toLowerCase();
     if (OFF.has(value))
@@ -43,15 +38,28 @@ function parseMaxRows(raw) {
         return n;
     throw new WazapError("INVALID_ID", `WAZAP_RECALL_MAX must be a number >= ${MIN_MAX_ROWS}, got "${value}".`, "Fix WAZAP_RECALL_MAX or remove it");
 }
-function parseMinSimilarity(raw) {
+/**
+ * The env wins over the model's own floor — a cosine that means "real match"
+ * is the model's to price, the override is the user's.
+ */
+function parseMinSimilarity(raw, fallback) {
     const value = stripPasted(raw ?? "");
     if (value === "")
-        return DEFAULT_MIN_SIMILARITY;
+        return fallback;
     const n = Number(value);
     if (Number.isFinite(n) && n >= 0 && n <= 1)
         return n;
     throw new WazapError("INVALID_ID", `WAZAP_RECALL_MIN_SIMILARITY must be a number between 0 and 1, got "${value}".`, "Fix WAZAP_RECALL_MIN_SIMILARITY or remove it");
 }
+function parseEmbedIdle(raw) {
+    const value = stripPasted(raw ?? "");
+    if (value === "")
+        return DEFAULT_EMBED_IDLE_MINUTES * 60_000;
+    const minutes = Number(value);
+    if (Number.isFinite(minutes) && minutes >= 0)
+        return Math.round(minutes * 60_000);
+    throw new WazapError("INVALID_ID", `WAZAP_EMBED_IDLE_MINUTES must be a number of minutes >= 0, got "${value}".`, "Fix WAZAP_EMBED_IDLE_MINUTES or remove it");
+}
 function parseUrl(raw) {
     const value = stripPasted(raw ?? "");
     if (value === "")
@@ -68,13 +76,15 @@ function parseUrl(raw) {
 }
 export function readRecallSettings(env, dataDir) {
     const embedBin = stripPasted(env.WAZAP_EMBED_BIN ?? "");
+    const model = parseModel(env.WAZAP_EMBED_MODEL);
     return {
         enabled: parseEnabled(env.WAZAP_RECALL),
-        model: parseModel(env.WAZAP_EMBED_MODEL),
+        model,
         embedBin: embedBin === "" ? null : embedBin,
         embedUrl: parseUrl(env.WAZAP_EMBED_URL),
         modelsDir: join(dataDir, "models"),
+        embedIdleMs: parseEmbedIdle(env.WAZAP_EMBED_IDLE_MINUTES),
         maxRows: parseMaxRows(env.WAZAP_RECALL_MAX),
-        minSimilarity: parseMinSimilarity(env.WAZAP_RECALL_MIN_SIMILARITY),
+        minSimilarity: parseMinSimilarity(env.WAZAP_RECALL_MIN_SIMILARITY, EMBED_MODELS[model].defaultMinSimilarity),
     };
 }

package/dist/recall/store.js

@@ -29,6 +29,18 @@ const DIR_MODE = 0o700;
 const FILE_MODE = 0o600;
 /** Owner call: fresh matches rank first. 0.5^(age/half-life) scales similarity. */
 const RECENCY_HALF_LIFE_MS = 30 * 24 * 60 * 60 * 1000;
+/** A literal query token shorter than this is too common to name anything. */
+const MIN_TOKEN_LEN = 4;
+/** What one matched rare token adds, before its document frequency scales it down. */
+const TOKEN_BOOST = 0.02;
+/** All token bonuses together may add at most this — a neighbour reorder, never a rescue. */
+const TOKEN_BOOST_MAX = 0.05;
+/** Reranking costs a text pass per candidate; only the top of the list gets it. */
+const RERANK_WINDOW = 100;
+/** One chat holds at most this many leading slots; further hits yield to other chats first. */
+const CHAT_SLOT_CAP = 3;
+/** Word overlap at or above this marks a candidate a near-duplicate of a picked hit. */
+const NEAR_DUP_JACCARD = 0.8;
 function normalize(vector) {
     let norm = 0;
     for (const x of vector)
@@ -52,6 +64,39 @@ function quantize(vector, dims) {
 function recencyDecay(ageMs) {
     return Math.pow(0.5, Math.max(0, ageMs) / RECENCY_HALF_LIFE_MS);
 }
+/** Case- and diacritic-insensitive fold, so "Cata" and "cată" are one token. */
+function fold(text) {
+    return text
+        .toLowerCase()
+        .normalize("NFD")
+        .replace(/[\u0300-\u036f]/g, "");
+}
+function textProfile(text) {
+    const folded = fold(text);
+    const words = new Set();
+    const literals = new Set();
+    for (const match of folded.matchAll(/[\p{L}\p{N}]+/gu)) {
+        words.add(match[0]);
+        if (match[0].length >= MIN_TOKEN_LEN)
+            literals.add(match[0]);
+    }
+    for (const chunk of folded.split(/\s+/)) {
+        const compact = chunk.replace(/[^\p{L}\p{N}]+/gu, "");
+        if (compact.length >= MIN_TOKEN_LEN)
+            literals.add(compact);
+    }
+    return { norm: folded.replace(/\s+/g, " ").trim(), words, literals };
+}
+/** Word Jaccard — two texts sharing most of their words are one answer. */
+function nearDuplicate(a, b) {
+    if (a.size === 0 || b.size === 0)
+        return false;
+    let shared = 0;
+    for (const token of a)
+        if (b.has(token))
+            shared++;
+    return shared / (a.size + b.size - shared) >= NEAR_DUP_JACCARD;
+}
 export class RecallStore {
     dir;
     spec;
@@ -331,7 +376,9 @@ export class RecallStore {
     /**
      * Brute-force cosine over live rows, filtered the way search_messages
      * filters, then ranked by similarity × recency decay. Exact, zero-dep and
-     * fast enough at living-memory sizes (50k × 768d int8).
+     * fast enough at living-memory sizes (50k × 768d int8). The floor applies
+     * to raw similarity — rerank() then reorders the survivors, never rescues
+     * what the floor dropped.
      */
     query(q, nowMs = Date.now()) {
         const unit = normalize(q.vector);
@@ -356,7 +403,74 @@ export class RecallStore {
             hits.push({ record, similarity, score: similarity * recencyDecay(nowMs - record.ts) });
         }
         hits.sort((a, b) => b.score - a.score);
-        return hits.slice(0, q.limit);
+        return this.rerank(hits, q);
+    }
+    /**
+     * The bounded reordering pass over the strongest candidates, run after the
+     * similarity floor has already decided what counts as an answer.
+     *
+     * First, literal tokens: embeddings are weak on names and identifiers, so a
+     * query token a hit carries verbatim earns a small bonus. A token's weight
+     * fades with its document frequency in the candidate set — a token in every
+     * candidate is common and adds nothing — and the total is capped, so the
+     * bonus reorders near-equal scores and can never lift a weak hit over a
+     * strong semantic one.
+     *
+     * Then diversity: a greedy walk picks in score order, but a near-duplicate
+     * of a picked hit, or a hit from a chat that already holds CHAT_SLOT_CAP
+     * slots, trails the list instead of filling it. Nothing is dropped and no
+     * score is invented — demoted hits keep their score and sit behind the
+     * picked ones, so a query scoped to a single chat comes back unchanged.
+     */
+    rerank(sorted, q) {
+        // A raw caller may omit limit; then the window itself is the cap.
+        const cap = Number.isFinite(q.limit) ? q.limit : sorted.length;
+        const candidates = sorted.slice(0, Math.max(cap, RERANK_WINDOW)).map((hit) => ({ hit, ...textProfile(hit.record.text) }));
+        if (candidates.length === 0)
+            return [];
+        const wanted = q.text === undefined ? new Set() : textProfile(q.text).literals;
+        if (wanted.size > 0) {
+            const df = new Map();
+            for (const token of wanted) {
+                let count = 0;
+                for (const c of candidates)
+                    if (c.literals.has(token))
+                        count++;
+                if (count > 0 && count < candidates.length)
+                    df.set(token, count);
+            }
+            if (df.size > 0) {
+                for (const c of candidates) {
+                    let bonus = 0;
+                    for (const [token, count] of df) {
+                        if (c.literals.has(token))
+                            bonus += TOKEN_BOOST * (1 - count / candidates.length);
+                    }
+                    c.hit = { ...c.hit, score: c.hit.score + Math.min(TOKEN_BOOST_MAX, bonus) };
+                }
+                candidates.sort((a, b) => b.hit.score - a.hit.score);
+            }
+        }
+        const picked = [];
+        const overflow = [];
+        const dups = [];
+        const perChat = new Map();
+        const chosen = [];
+        for (const c of candidates) {
+            if (chosen.some((p) => p.norm === c.norm || nearDuplicate(p.words, c.words))) {
+                dups.push(c.hit);
+                continue;
+            }
+            const held = perChat.get(c.hit.record.jid) ?? 0;
+            if (held >= CHAT_SLOT_CAP) {
+                overflow.push(c.hit);
+                continue;
+            }
+            perChat.set(c.hit.record.jid, held + 1);
+            picked.push(c.hit);
+            chosen.push(c);
+        }
+        return [...picked, ...overflow, ...dups].slice(0, cap);
     }
     async maybeCompact() {
         if (this.nextRow === 0 || this.deadRows <= this.nextRow * COMPACT_DEAD_RATIO)

package/dist/server.js

@@ -2,12 +2,8 @@ import { randomUUID, timingSafeEqual } from "node:crypto";
 import { createServer } from "node:http";
 import { createConnection } from "node:net";
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
-import { mcpAuthRouter, getOAuthProtectedResourceMetadataUrl } from "@modelcontextprotocol/sdk/server/auth/router.js";
 import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
-import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
 import { isInitializeRequest } from "@modelcontextprotocol/sdk/types.js";
-import express from "express";
-import { rateLimit } from "express-rate-limit";
 import { WAZAP_VERSION, paths, writesHints } from "./config.js";
 import { APPROVE_PATH, OAUTH_SCOPES, WazapOAuthProvider } from "./oauth.js";
 import { loadSkills, registerSkillPrompts, skillInstructions } from "./skills.js";
@@ -108,6 +104,9 @@ async function listenHttp(server, host, port) {
 }
 /** Serve /mcp and /healthz on one address. Resolves with the bound port, so port 0 works. */
 export async function startHttpEndpoint(hub, config, endpoint) {
+    // express is only needed once a listener is actually bound; a stdio server
+    // with sharing off and a bridge onto a running daemon never reach here.
+    const { default: express } = await import("express");
     const app = express();
     app.use(express.json());
     app.use((req, res, next) => {
@@ -131,7 +130,14 @@ export async function startHttpEndpoint(hub, config, endpoint) {
     const oauth = endpoint.oauth;
     // A server that advertises sign-in must not also answer strangers.
     const openRead = endpoint.openRead && !oauth;
+    let resourceMetadataUrl = null;
     if (oauth) {
+        // The OAuth stack (the SDK's auth router and its own limiter) is the one
+        // part of this endpoint only a public server pays for, so it loads here.
+        const [{ mcpAuthRouter, getOAuthProtectedResourceMetadataUrl }, { rateLimit }] = await Promise.all([
+            import("@modelcontextprotocol/sdk/server/auth/router.js"),
+            import("express-rate-limit"),
+        ]);
         // Reached through a TLS proxy: on this machine, or the Docker bridge when
         // the container binds 0.0.0.0. The proxy's idea of the caller is the one
         // the password lockout and the SDK's limiters should count.
@@ -149,8 +155,8 @@ export async function startHttpEndpoint(hub, config, endpoint) {
         }));
         app.post(APPROVE_PATH, rateLimit({ windowMs: 15 * 60 * 1000, limit: 30, standardHeaders: true, legacyHeaders: false }), express.urlencoded({ extended: false }), oauth.approve);
         log(`OAuth on: agents sign in at ${oauth.issuerUrl.href}`);
+        resourceMetadataUrl = getOAuthProtectedResourceMetadataUrl(oauth.resourceUrl);
     }
-    const resourceMetadataUrl = oauth ? getOAuthProtectedResourceMetadataUrl(oauth.resourceUrl) : null;
     // The first credential the bearer token matches decides the session's tools,
     // so a leaked read token can never message anyone. An OAuth token carries
     // the scope the person picked on the consent page.
@@ -196,6 +202,9 @@ export async function startHttpEndpoint(hub, config, endpoint) {
             const sessionId = req.headers["mcp-session-id"];
             let transport = typeof sessionId === "string" ? transports.get(sessionId) : undefined;
             if (!transport && req.method === "POST" && isInitializeRequest(req.body)) {
+                // Session state is only needed when a client actually posts initialize;
+                // loading it here keeps it off the bind path that daemon.json waits on.
+                const { StreamableHTTPServerTransport } = await import("@modelcontextprotocol/sdk/server/streamableHttp.js");
                 const newTransport = new StreamableHTTPServerTransport({
                     sessionIdGenerator: () => randomUUID(),
                     onsessioninitialized: (sid) => {

package/dist/tools.js

@@ -500,7 +500,9 @@ search_messages to see. For an exact string — an id, a phone number, a URL —
 search_messages is the better tool.
 
 Each result carries its date and a score: semantic similarity scaled by
-recency, so fresh matches rank first. chat_id, since, until and from narrow
+recency, plus a small bonus when the hit repeats a rare query token
+verbatim — a name, a number — so fresh and exact matches rank first.
+chat_id, since, until and from narrow
 the search exactly like search_messages. A hit marked "index only" lives in
 the index alone: quote it, but get_message and download_media cannot see it.
 Results under the similarity floor are dropped rather than listed; when only

package/dist/whatsapp.js

@@ -627,7 +627,16 @@ export class WhatsAppService {
             const [vector] = await this.recallEmbed([query], "query");
             const minSimilarity = this.recallEnv instanceof WazapError ? undefined : this.recallEnv.minSimilarity;
             const hits = store
-                .query({ vector: vector, chatId: scope, sinceMs: opts.sinceMs, untilMs: opts.untilMs, from, minSimilarity, limit })
+                .query({
+                vector: vector,
+                text: query,
+                chatId: scope,
+                sinceMs: opts.sinceMs,
+                untilMs: opts.untilMs,
+                from,
+                minSimilarity,
+                limit,
+            })
                 .map((hit) => {
                 const live = this.store.messages.has(hit.record.sid);
                 return {
@@ -1248,8 +1257,9 @@ export class WhatsAppService {
         }
     }
     /**
-     * The sidecar, started on the first embedding request. A failed start is not
-     * cached — the next queued batch tries again.
+     * The sidecar, started on the first embedding request and shared with every
+     * other account in the process on the same binary and model. A failed start
+     * is not cached — the next queued batch tries again.
      */
     recallEngine() {
         if (this.stopped)
@@ -1289,8 +1299,10 @@ export class WhatsAppService {
         if (text !== null) {
             // Capped here, not only in the store, so the feed diff compares the text
             // the index would actually keep — an over-cap message is not fresh work
-            // on every boot.
-            const capped = text.slice(0, RECALL_TEXT_CAP);
+            // on every boot. The cap is the model's: e5's 512-token window takes
+            // far less text than gemma's.
+            const maxChars = this.recallEnv instanceof WazapError ? RECALL_TEXT_CAP : EMBED_MODELS[this.recallEnv.model].maxChars;
+            const capped = text.slice(0, maxChars);
             ops.push({
                 sid,
                 item: { sid, jid, ts: messageTimestampMs(raw), sender: this.recallSender(raw, jid), type: messageType(raw), text: capped },
@@ -1390,9 +1402,10 @@ export class WhatsAppService {
         return { state: pending > 0 ? "indexing" : "ready", indexed, pending };
     }
     /**
-     * Queue first, then the sidecar — stopping it unblocks an embedding call in
-     * flight — then the store, whose own write queue drains before it closes.
-     * An engine still coming up is stopped whenever its start resolves.
+     * Queue first, then the engine — releasing its claim on the shared sidecar
+     * unblocks an embedding call in flight — then the store, whose own write
+     * queue drains before it closes. An engine still coming up is released
+     * whenever its start resolves.
      */
     async stopRecall() {
         const queueStop = this.recallQueue?.stop() ?? Promise.resolve();

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wazap-mcp",
-  "version": "0.18.2",
+  "version": "0.18.4",
   "mcpName": "io.github.razvangirgiz/wazap",
   "description": "WhatsApp for your AI agent. MCP server over Baileys: pairing-code login, several accounts, 33 tools, stdio or HTTP.",
   "license": "MIT",