clawmem 0.11.2 → 0.11.3

package/AGENTS.md

@@ -2,7 +2,7 @@
 
 ## Inference Services
 
-ClawMem uses three `llama-server` instances for neural inference. By default, the `bin/clawmem` wrapper points at `localhost:8088/8089/8090`.
+ClawMem uses three inference services (embedding, LLM, reranker). The **default** stack runs all three as `llama-server` instances; the **SOTA** stack swaps the reranker for a transformers sidecar (same `/v1/rerank` contract). The `bin/clawmem` wrapper points at `localhost:8088/8089/8090`.
 
 **Default (QMD native combo, any GPU or in-process):**
 
@@ -14,15 +14,15 @@ ClawMem uses three `llama-server` instances for neural inference. By default, th
 
 All three models auto-download via `node-llama-cpp` if no server is running (Metal on Apple Silicon, Vulkan where available, CPU as last resort). Fast with GPU acceleration (Metal/Vulkan); significantly slower on CPU-only.
 
-**SOTA upgrade (12GB+ GPU):** CC-BY-NC-4.0 — non-commercial only.
+**SOTA upgrade (16GB+ GPU):** CC-BY-NC-4.0 — non-commercial only.
 
 | Service | Port | Model | VRAM | Protocol |
 |---|---|---|---|---|
 | Embedding | 8088 | zembed-1-Q4_K_M | ~4.4GB | `/v1/embeddings` |
 | LLM | 8089 | qmd-query-expansion-1.7B-q4_k_m | ~2.2GB | `/v1/chat/completions` |
-| Reranker | 8090 | zerank-2-Q4_K_M | ~3.3GB | `/v1/rerank` |
+| Reranker | 8090 | zerank-2 seq-cls sidecar | ~9GB (bf16) | `/v1/rerank` |
 
-Total ~10GB VRAM. zembed-1 (2560d, 32K context, SOTA retrieval) distilled from zerank-2 via zELO. Optimal pairing.
+zembed-1 (2560d, 32K context, SOTA retrieval) distilled from zerank-2 via zELO — optimal pairing. The SOTA reranker is the **zerank-2 seq-cls sidecar** at `extras/rerankers/zerank-2-seq/` (transformers, bf16, ~9GB VRAM) — **not** a GGUF. The older `zerank-2-Q4_K_M` GGUF is **deprecated**: llama.cpp drops zerank's score head, so its scores are near-zero and uninformative (final ordering stays RRF-dominated). Full SOTA stack ≈ 16GB VRAM.
 
 **Remote option:** Set `CLAWMEM_EMBED_URL`, `CLAWMEM_LLM_URL`, `CLAWMEM_RERANK_URL` to the remote host. Set `CLAWMEM_NO_LOCAL_MODELS=true` to prevent surprise fallback downloads.
 
@@ -36,7 +36,7 @@ Total ~10GB VRAM. zembed-1 (2560d, 32K context, SOTA retrieval) distilled from z
 |---|---|---|---|
 | Embedding | [EmbeddingGemma-300M-Q8_0](https://huggingface.co/ggml-org/embeddinggemma-300M-GGUF) (314MB, 768d) | [zembed-1-Q4_K_M](https://huggingface.co/Abhiray/zembed-1-Q4_K_M-GGUF) (2.4GB, 2560d) | zembed-1: 32K context, SOTA retrieval. `-ub` must match `-b`. |
 | LLM | [qmd-query-expansion-1.7B-q4_k_m](https://huggingface.co/tobil/qmd-query-expansion-1.7B-gguf) (~1.1GB) | Same | QMD's Qwen3-1.7B finetune for query expansion. |
-| Reranker | [qwen3-reranker-0.6B-Q8_0](https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF) (~600MB) | [zerank-2-Q4_K_M](https://huggingface.co/keisuke-miyako/zerank-2-gguf-q4_k_m) (2.4GB) | zerank-2: outperforms Cohere rerank-3.5. `-ub` must match `-b`. |
+| Reranker | [qwen3-reranker-0.6B-Q8_0](https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF) (~600MB) | [zerank-2 seq-cls sidecar](extras/rerankers/zerank-2-seq/) (bf16, ~9GB) | zerank-2: NDCG@10 ahead of Cohere rerank-3.5. Served via transformers sidecar, **not** GGUF (the GGUF drops the score head → inert). CC-BY-NC. |
 
 ### Server Setup (all three use llama-server)
 
@@ -55,15 +55,15 @@ llama-server -m qmd-query-expansion-1.7B-q4_k_m.gguf \
 llama-server -m Qwen3-Reranker-0.6B-Q8_0.gguf \
   --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 --batch-size 512
 
-# === SOTA upgrade (12GB+ GPU) — -ub must match -b for non-causal attention ===
+# === SOTA upgrade (16GB+ GPU) ===
 
-# Embedding
+# Embedding (zembed-1) — -ub must match -b for non-causal attention
 llama-server -m zembed-1-Q4_K_M.gguf \
   --embeddings --port 8088 --host 0.0.0.0 -ngl 99 -c 8192 -b 2048 -ub 2048
 
-# Reranker
-llama-server -m zerank-2-Q4_K_M.gguf \
-  --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 -b 2048 -ub 2048
+# Reranker (zerank-2) — seq-cls SIDECAR (transformers, bf16), NOT a llama-server GGUF:
+#   cd extras/rerankers/zerank-2-seq && docker compose run --rm convert && docker compose up -d reranker
+# The zerank-2 GGUF is deprecated: llama.cpp drops its score head -> near-zero, uninformative scores.
 ```
 
 ### Verify Endpoints
@@ -734,7 +734,7 @@ clawmem focus clear --session-id abc123
 - **Entity resolution (v0.2.0+):** A-MEM enrichment extracts named entities via LLM, resolves to canonical forms using FTS5 + Levenshtein fuzzy matching with **type-agnostic compatibility buckets** (person, org, location stay separate; project/service/tool/concept merge freely as "tech" bucket). Quality filters reject title-as-entity, long names, template placeholders, and invalid locations. Entity edges use IDF-based specificity scoring (rare entities create edges; ubiquitous entities alone cannot). See `docs/internals/entity-resolution.md` for customization (extending type vocabulary and buckets).
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
-- Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.
+- Three inference services (embedding, LLM, reranker) on local or remote GPU — `llama-server` for all three in the default stack; the SOTA stack serves the reranker as a transformers sidecar. Wrapper defaults to `localhost:8088/8089/8090`.
 - `CLAWMEM_NO_LOCAL_MODELS=false` (default) allows in-process LLM/reranker fallback via `node-llama-cpp`. Set `true` for remote-only setups to fail fast on unreachable endpoints.
 - Consolidation worker (`CLAWMEM_ENABLE_CONSOLIDATION=true`) backfills unenriched docs with A-MEM notes + links. Only runs if the MCP process stays alive long enough to tick (every 5min).
 - Beads integration: `syncBeadsIssues()` queries `bd` CLI (Dolt backend, v0.58.0+) for live issue data, creates markdown docs in `beads` collection, maps all dependency edge types into `memory_relations`, and triggers A-MEM enrichment for new docs. Watcher auto-triggers on `.beads/` directory changes; `beads_sync` MCP tool for manual sync. Requires `bd` binary on PATH or at `~/go/bin/bd`.

package/CLAUDE.md

@@ -2,7 +2,7 @@
 
 ## Inference Services
 
-ClawMem uses three `llama-server` instances for neural inference. By default, the `bin/clawmem` wrapper points at `localhost:8088/8089/8090`.
+ClawMem uses three inference services (embedding, LLM, reranker). The **default** stack runs all three as `llama-server` instances; the **SOTA** stack swaps the reranker for a transformers sidecar (same `/v1/rerank` contract). The `bin/clawmem` wrapper points at `localhost:8088/8089/8090`.
 
 **Default (QMD native combo, any GPU or in-process):**
 
@@ -14,15 +14,15 @@ ClawMem uses three `llama-server` instances for neural inference. By default, th
 
 All three models auto-download via `node-llama-cpp` if no server is running (Metal on Apple Silicon, Vulkan where available, CPU as last resort). Fast with GPU acceleration (Metal/Vulkan); significantly slower on CPU-only.
 
-**SOTA upgrade (12GB+ GPU):** CC-BY-NC-4.0 — non-commercial only.
+**SOTA upgrade (16GB+ GPU):** CC-BY-NC-4.0 — non-commercial only.
 
 | Service | Port | Model | VRAM | Protocol |
 |---|---|---|---|---|
 | Embedding | 8088 | zembed-1-Q4_K_M | ~4.4GB | `/v1/embeddings` |
 | LLM | 8089 | qmd-query-expansion-1.7B-q4_k_m | ~2.2GB | `/v1/chat/completions` |
-| Reranker | 8090 | zerank-2-Q4_K_M | ~3.3GB | `/v1/rerank` |
+| Reranker | 8090 | zerank-2 seq-cls sidecar | ~9GB (bf16) | `/v1/rerank` |
 
-Total ~10GB VRAM. zembed-1 (2560d, 32K context, SOTA retrieval) distilled from zerank-2 via zELO. Optimal pairing.
+zembed-1 (2560d, 32K context, SOTA retrieval) distilled from zerank-2 via zELO — optimal pairing. The SOTA reranker is the **zerank-2 seq-cls sidecar** at `extras/rerankers/zerank-2-seq/` (transformers, bf16, ~9GB VRAM) — **not** a GGUF. The older `zerank-2-Q4_K_M` GGUF is **deprecated**: llama.cpp drops zerank's score head, so its scores are near-zero and uninformative (final ordering stays RRF-dominated). Full SOTA stack ≈ 16GB VRAM.
 
 **Remote option:** Set `CLAWMEM_EMBED_URL`, `CLAWMEM_LLM_URL`, `CLAWMEM_RERANK_URL` to the remote host. Set `CLAWMEM_NO_LOCAL_MODELS=true` to prevent surprise fallback downloads.
 
@@ -36,7 +36,7 @@ Total ~10GB VRAM. zembed-1 (2560d, 32K context, SOTA retrieval) distilled from z
 |---|---|---|---|
 | Embedding | [EmbeddingGemma-300M-Q8_0](https://huggingface.co/ggml-org/embeddinggemma-300M-GGUF) (314MB, 768d) | [zembed-1-Q4_K_M](https://huggingface.co/Abhiray/zembed-1-Q4_K_M-GGUF) (2.4GB, 2560d) | zembed-1: 32K context, SOTA retrieval. `-ub` must match `-b`. |
 | LLM | [qmd-query-expansion-1.7B-q4_k_m](https://huggingface.co/tobil/qmd-query-expansion-1.7B-gguf) (~1.1GB) | Same | QMD's Qwen3-1.7B finetune for query expansion. |
-| Reranker | [qwen3-reranker-0.6B-Q8_0](https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF) (~600MB) | [zerank-2-Q4_K_M](https://huggingface.co/keisuke-miyako/zerank-2-gguf-q4_k_m) (2.4GB) | zerank-2: outperforms Cohere rerank-3.5. `-ub` must match `-b`. |
+| Reranker | [qwen3-reranker-0.6B-Q8_0](https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF) (~600MB) | [zerank-2 seq-cls sidecar](extras/rerankers/zerank-2-seq/) (bf16, ~9GB) | zerank-2: NDCG@10 ahead of Cohere rerank-3.5. Served via transformers sidecar, **not** GGUF (the GGUF drops the score head → inert). CC-BY-NC. |
 
 ### Server Setup (all three use llama-server)
 
@@ -55,15 +55,15 @@ llama-server -m qmd-query-expansion-1.7B-q4_k_m.gguf \
 llama-server -m Qwen3-Reranker-0.6B-Q8_0.gguf \
   --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 --batch-size 512
 
-# === SOTA upgrade (12GB+ GPU) — -ub must match -b for non-causal attention ===
+# === SOTA upgrade (16GB+ GPU) ===
 
-# Embedding
+# Embedding (zembed-1) — -ub must match -b for non-causal attention
 llama-server -m zembed-1-Q4_K_M.gguf \
   --embeddings --port 8088 --host 0.0.0.0 -ngl 99 -c 8192 -b 2048 -ub 2048
 
-# Reranker
-llama-server -m zerank-2-Q4_K_M.gguf \
-  --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 -b 2048 -ub 2048
+# Reranker (zerank-2) — seq-cls SIDECAR (transformers, bf16), NOT a llama-server GGUF:
+#   cd extras/rerankers/zerank-2-seq && docker compose run --rm convert && docker compose up -d reranker
+# The zerank-2 GGUF is deprecated: llama.cpp drops its score head -> near-zero, uninformative scores.
 ```
 
 ### Verify Endpoints
@@ -734,7 +734,7 @@ clawmem focus clear --session-id abc123
 - **Entity resolution (v0.2.0+):** A-MEM enrichment extracts named entities via LLM, resolves to canonical forms using FTS5 + Levenshtein fuzzy matching with **type-agnostic compatibility buckets** (person, org, location stay separate; project/service/tool/concept merge freely as "tech" bucket). Quality filters reject title-as-entity, long names, template placeholders, and invalid locations. Entity edges use IDF-based specificity scoring (rare entities create edges; ubiquitous entities alone cannot). See `docs/internals/entity-resolution.md` for customization (extending type vocabulary and buckets).
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
-- Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.
+- Three inference services (embedding, LLM, reranker) on local or remote GPU — `llama-server` for all three in the default stack; the SOTA stack serves the reranker as a transformers sidecar. Wrapper defaults to `localhost:8088/8089/8090`.
 - `CLAWMEM_NO_LOCAL_MODELS=false` (default) allows in-process LLM/reranker fallback via `node-llama-cpp`. Set `true` for remote-only setups to fail fast on unreachable endpoints.
 - Consolidation worker (`CLAWMEM_ENABLE_CONSOLIDATION=true`) backfills unenriched docs with A-MEM notes + links. Only runs if the MCP process stays alive long enough to tick (every 5min).
 - Beads integration: `syncBeadsIssues()` queries `bd` CLI (Dolt backend, v0.58.0+) for live issue data, creates markdown docs in `beads` collection, maps all dependency edge types into `memory_relations`, and triggers A-MEM enrichment for new docs. Watcher auto-triggers on `.beads/` directory changes; `beads_sync` MCP tool for manual sync. Requires `bd` binary on PATH or at `~/go/bin/bd`.

package/README.md

@@ -114,8 +114,8 @@ After installing, here's the full journey from zero to working memory:
 | Step | What | How | Details |
 |------|------|-----|---------|
 | **1. Bootstrap** | Create a vault, index your first collection, embed, install hooks and MCP | `clawmem bootstrap ~/notes --name notes` | One command does it all. Or run each step manually (see below). |
-| **2. Choose models** | Pick embedding + reranker models based on your hardware | 12GB+ VRAM → SOTA stack (zembed-1 + zerank-2). Less → QMD native combo. No GPU → cloud embedding or CPU fallback. | [GPU Services](#gpu-services) |
-| **3. Download models** | Get the GGUF files for your chosen stack | `wget` from HuggingFace, or let `node-llama-cpp` auto-download the QMD native models on first use | [Embedding](#embedding), [LLM Server](#llm-server), [Reranker Server](#reranker-server) |
+| **2. Choose models** | Pick embedding + reranker models based on your hardware | 16GB+ VRAM → SOTA stack (zembed-1 + zerank-2 sidecar). Less → QMD native combo. No GPU → cloud embedding or CPU fallback. | [GPU Services](#gpu-services) |
+| **3. Download models** | Get the model files for your chosen stack (GGUFs for embedding/LLM/default-reranker; the zerank-2 SOTA reranker builds its own sidecar artifact) | `wget` from HuggingFace, let `node-llama-cpp` auto-download the QMD native models, or run the sidecar recipe | [Embedding](#embedding), [LLM Server](#llm-server), [Reranker Server](#reranker-server) |
 | **4. Start services** | Run GPU servers (if using dedicated GPU) and background services. Optionally enable the v0.8.2 background maintenance workers in the watcher unit so consolidation + deductive synthesis run automatically. | `llama-server` for each model. systemd units for watcher + embed timer. Drop-in for the watcher to enable workers + tune intervals + set the quiet window. | [systemd services](docs/guides/systemd-services.md), [background workers](docs/guides/systemd-services.md#background-maintenance-workers-v082) |
 | **5. Decide what to index** | Add collections for your projects, notes, research, and domain docs | `clawmem collection add ~/project --name project` | The more relevant markdown you index, the better retrieval works. See [building a rich context field](docs/introduction.md#building-a-rich-context-field). |
 | **6. Connect your agent** | Hook into Claude Code, OpenClaw, Hermes, or any MCP client | `clawmem setup hooks && clawmem setup mcp` for Claude Code. `clawmem setup openclaw` for OpenClaw. Copy `src/hermes/` to Hermes plugins for Hermes. | [Integration](#integration) |
@@ -305,17 +305,19 @@ vault_sync(vault="work", content_root="~/work/docs")
 
 ### GPU Services
 
-ClawMem uses three `llama-server` (llama.cpp) instances for neural inference. All three have in-process fallbacks via `node-llama-cpp` (auto-downloads on first use), so ClawMem works without a dedicated GPU. `node-llama-cpp` auto-detects the best available backend — Metal on Apple Silicon, Vulkan where available, CPU as last resort. With GPU acceleration (Metal/Vulkan), in-process inference is fast for these small models (0.3B–1.7B); on CPU-only systems it is significantly slower. For production use, run the servers via [systemd services](docs/guides/systemd-services.md) to prevent silent fallback.
+ClawMem uses three inference services — embedding, LLM, and reranker. In the **default** stack all three run as `llama-server` (llama.cpp) instances, each with an in-process `node-llama-cpp` fallback (auto-downloads on first use), so ClawMem works without a dedicated GPU. (The **SOTA** reranker is the exception — it is a transformers sidecar, see [Reranker Server](#reranker-server).) `node-llama-cpp` auto-detects the best available backend — Metal on Apple Silicon, Vulkan where available, CPU as last resort. With GPU acceleration (Metal/Vulkan), in-process inference is fast for these small models (0.3B–1.7B); on CPU-only systems it is significantly slower. For production use, run the servers via [systemd services](docs/guides/systemd-services.md) to prevent silent fallback.
 
-**GPU with VRAM to spare (12GB+, recommended):** ZeroEntropy's distillation-paired stack delivers best retrieval quality — total ~10GB VRAM.
+**GPU with VRAM to spare (16GB+, recommended):** ZeroEntropy's distillation-paired stack delivers best retrieval quality — total ~16GB VRAM.
 
 | Service | Port | Model | VRAM | Purpose |
 |---|---|---|---|---|
 | Embedding | 8088 | [zembed-1-Q4_K_M](https://huggingface.co/Abhiray/zembed-1-Q4_K_M-GGUF) | ~4.4GB | SOTA embedding (2560d, 32K context). Distilled from zerank-2 via zELO. |
 | LLM | 8089 | [qmd-query-expansion-1.7B-q4_k_m](https://huggingface.co/tobil/qmd-query-expansion-1.7B-gguf) | ~2.2GB | Intent classification, query expansion, A-MEM |
-| Reranker | 8090 | [zerank-2-Q4_K_M](https://huggingface.co/keisuke-miyako/zerank-2-gguf-q4_k_m) | ~3.3GB | SOTA reranker. Outperforms Cohere rerank-3.5. Optimal pairing with zembed-1. |
+| Reranker | 8090 | [zerank-2 seq-cls sidecar](extras/rerankers/zerank-2-seq/) | ~9GB (bf16) | SOTA reranker. NDCG@10 ahead of Cohere rerank-3.5. Optimal pairing with zembed-1. |
 
-**Important:** zembed-1 and zerank-2 use non-causal attention — `-ub` must equal `-b` on llama-server (e.g. `-b 2048 -ub 2048`). See [Reranker Server](#reranker-server) for details.
+**The SOTA reranker is a sidecar, not a GGUF.** It is served by the [zerank-2 seq-cls sidecar](extras/rerankers/zerank-2-seq/) (transformers, bf16). The previously-listed `zerank-2-Q4_K_M` GGUF is **deprecated** — llama.cpp's converter drops zerank's score head, so under `--reranking` it produces near-zero, uninformative scores (final ordering stays RRF-dominated). See the [sidecar README](extras/rerankers/zerank-2-seq/) and [Reranker Server](#reranker-server).
+
+**Important:** zembed-1 uses non-causal attention — `-ub` must equal `-b` on llama-server (e.g. `-b 2048 -ub 2048`). See [Embedding](#embedding) for details.
 
 **License:** zembed-1 and zerank-2 are released under **CC-BY-NC-4.0** — non-commercial only. The QMD native models below have no such restriction.
 
@@ -466,15 +468,15 @@ Cross-encoder reranking for `query` and `intent_search` pipelines on port 8090.
 
 Scores each candidate against the original query (cross-encoder architecture). `query` pipeline: 4000 char context per doc (deep reranking); `intent_search`: 200 char context per doc (fast reranking).
 
-**GPU with VRAM to spare (recommended):** [zerank-2-Q4_K_M](https://huggingface.co/keisuke-miyako/zerank-2-gguf-q4_k_m) (2.4GB, ~3.3GB VRAM). Outperforms Cohere rerank-3.5 and Gemini 2.5 Flash. Optimal pairing with zembed-1 (same distillation architecture via zELO). **CC-BY-NC-4.0** — non-commercial only.
+**GPU with VRAM to spare (recommended):** the **[zerank-2 seq-cls sidecar](extras/rerankers/zerank-2-seq/)** (bf16, ~9GB VRAM). zerank-2's NDCG@10 is ahead of Cohere rerank-3.5 and Gemini 2.5 Flash. Optimal pairing with zembed-1 (same distillation architecture via zELO). **CC-BY-NC-4.0** — non-commercial only.
 
-```bash
-wget https://huggingface.co/keisuke-miyako/zerank-2-gguf-q4_k_m/resolve/main/zerank-2-Q4_k_m.gguf
+zerank-2 is **not** servable as a llama.cpp GGUF — its CrossEncoder/LogitScore head is dropped by the GGUF converter (the model becomes a headless causal LM that produces near-zero, uninformative scores under `--reranking`, leaving final ordering RRF-dominated). The sidecar serves the real head via transformers, with a reproducible correctness gate:
 
-# -ub must match -b for non-causal attention
-llama-server -m zerank-2-Q4_K_M.gguf \
-  --reranking --port 8090 --host 0.0.0.0 \
-  -ngl 99 -c 2048 -b 2048 -ub 2048
+```bash
+cd extras/rerankers/zerank-2-seq
+docker compose build
+HF_TOKEN=hf_xxx docker compose run --rm convert   # download + convert + verify (all gates must pass)
+docker compose up -d reranker                      # serves /v1/rerank on :8090
 ```
 
 **CPU / GPU without VRAM to spare:** [qwen3-reranker-0.6B-Q8_0](https://huggingface.co/ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF) (~600MB, ~1.3GB VRAM). The QMD native reranker — auto-downloaded by `node-llama-cpp` if no server is running.
@@ -487,7 +489,7 @@ llama-server -m Qwen3-Reranker-0.6B-Q8_0.gguf \
   -ngl 99 -c 2048 --batch-size 512
 ```
 
-**Note:** zerank-2 and zembed-1 use non-causal attention — `-ub` (ubatch) must equal `-b` (batch). Omitting `-ub` or setting it lower causes assertion crashes. qwen3-reranker-0.6B does not have this requirement. See [llama.cpp#12836](https://github.com/ggml-org/llama.cpp/issues/12836).
+**Note:** zembed-1 (embedding) uses non-causal attention — `-ub` (ubatch) must equal `-b` (batch); omitting `-ub` or setting it lower causes assertion crashes. The qwen3-reranker-0.6B GGUF does not have this requirement, and the zerank-2 sidecar is served via transformers (no llama.cpp batch flags). See [llama.cpp#12836](https://github.com/ggml-org/llama.cpp/issues/12836).
 
 ### MCP Server
 
@@ -1151,7 +1153,7 @@ clawmem serve --port 7438 &
 
 ## Deployment
 
-Three-tier retrieval architecture: infrastructure (watcher + embed timer) → hooks (~90%) → agent MCP (~10%). Works out of the box without a dedicated GPU (all models auto-download via `node-llama-cpp`, uses Metal on Apple Silicon). For best performance, run three `llama-server` instances — see [GPU Services](#gpu-services) for model tiers (SOTA vs QMD native) and [Cloud Embedding](#option-c-cloud-embedding-api) for cloud embedding alternatives.
+Three-tier retrieval architecture: infrastructure (watcher + embed timer) → hooks (~90%) → agent MCP (~10%). Works out of the box without a dedicated GPU (all models auto-download via `node-llama-cpp`, uses Metal on Apple Silicon). For best performance, run the inference services on GPU (three `llama-server` instances in the default stack; the SOTA reranker is a transformers sidecar) — see [GPU Services](#gpu-services) for model tiers (SOTA vs QMD native) and [Cloud Embedding](#option-c-cloud-embedding-api) for cloud embedding alternatives.
 
 Key services: `clawmem-watcher` (auto-index on file change + beads sync), `clawmem-embed` timer (daily embedding sweep), 7 Claude Code hooks installed by default (context injection, curator nudge, compaction support, decision extraction, handoffs, feedback). Optional `clawmem-curator` agent for on-demand lifecycle triage, retrieval health checks, and maintenance (`clawmem setup curator`).
 

package/SKILL.md

@@ -18,7 +18,7 @@ Two tiers: **hooks** handle automatic context flow (surfacing, extraction, compa
 
 ## Inference Services
 
-Three `llama-server` instances for neural inference. The `bin/clawmem` wrapper defaults to `localhost:8088/8089/8090`.
+Three inference services (embedding, LLM, reranker). The **default** stack runs all three as `llama-server` instances; the **SOTA** stack serves the reranker as a transformers sidecar (same `/v1/rerank` contract). The `bin/clawmem` wrapper defaults to `localhost:8088/8089/8090`.
 
 **Default (QMD native combo, any GPU or in-process):**
 
@@ -30,7 +30,7 @@ Three `llama-server` instances for neural inference. The `bin/clawmem` wrapper d
 
 All three models auto-download via `node-llama-cpp` if no server is running (Metal on Apple Silicon, Vulkan where available, CPU as last resort). Fast with GPU acceleration (Metal/Vulkan); significantly slower on CPU-only.
 
-**SOTA upgrade (12GB+ GPU):** zembed-1-Q4_K_M (embedding, 2560d, ~4.4GB) + zerank-2-Q4_K_M (reranker, ~3.3GB). Total ~10GB with LLM. Distillation-paired via zELO. `-ub` must match `-b` for both. **CC-BY-NC-4.0** — non-commercial only.
+**SOTA upgrade (16GB+ GPU):** zembed-1-Q4_K_M (embedding, 2560d, ~4.4GB) + the **zerank-2 seq-cls sidecar** (reranker, bf16 ~9GB — see `extras/rerankers/zerank-2-seq/`). Total ~16GB with LLM. Distillation-paired via zELO. zembed-1's `-ub` must match `-b`. **CC-BY-NC-4.0** — non-commercial only. The older `zerank-2-Q4_K_M` GGUF reranker is **deprecated** (llama.cpp drops zerank's score head → inert reranking); use the sidecar.
 
 **Remote option:** Set `CLAWMEM_EMBED_URL`, `CLAWMEM_LLM_URL`, `CLAWMEM_RERANK_URL` to remote host. Set `CLAWMEM_NO_LOCAL_MODELS=true` to prevent fallback downloads.
 
@@ -53,15 +53,14 @@ llama-server -m qmd-query-expansion-1.7B-q4_k_m.gguf \
 llama-server -m Qwen3-Reranker-0.6B-Q8_0.gguf \
   --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 --batch-size 512
 
-# === SOTA upgrade (12GB+ GPU) — -ub must match -b ===
+# === SOTA upgrade (16GB+ GPU) ===
 
-# Embedding (zembed-1)
+# Embedding (zembed-1) — -ub must match -b
 llama-server -m zembed-1-Q4_K_M.gguf \
   --embeddings --port 8088 --host 0.0.0.0 -ngl 99 -c 8192 -b 2048 -ub 2048
 
-# Reranker (zerank-2)
-llama-server -m zerank-2-Q4_K_M.gguf \
-  --reranking --port 8090 --host 0.0.0.0 -ngl 99 -c 2048 -b 2048 -ub 2048
+# Reranker (zerank-2) — seq-cls SIDECAR (transformers, bf16), NOT GGUF:
+#   see extras/rerankers/zerank-2-seq/  (the zerank-2 GGUF is deprecated — drops the score head)
 ```
 
 ### Verify Endpoints
@@ -791,7 +790,7 @@ clawmem focus clear --session-id abc123
 
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
-- Three `llama-server` instances on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.
+- Three inference services on local or remote GPU — `llama-server` for all three in the default stack; the SOTA stack serves the reranker as a transformers sidecar. Wrapper defaults to `localhost:8088/8089/8090`.
 - `CLAWMEM_NO_LOCAL_MODELS=false` (default) allows in-process fallback. Set `true` for remote-only to fail fast.
 - Consolidation worker (`CLAWMEM_ENABLE_CONSOLIDATION=true`) backfills unenriched docs and runs Phase 2 merge / Phase 3 deductive synthesis. **v0.8.2:** hosted by either `clawmem watch` (long-lived, canonical) or `clawmem mcp` (per-session fallback); every tick acquires a `light-consolidation` `worker_leases` row before doing work, so dual-hosting against the same vault is safe.
 - Beads integration: `syncBeadsIssues()` queries `bd` CLI (Dolt backend, v0.58.0+), creates markdown docs, maps dependency edges into `memory_relations`. Watcher auto-triggers on `.beads/` changes; `beads_sync` MCP for manual sync.

package/extras/rerankers/zerank-2-seq/Dockerfile

@@ -0,0 +1,26 @@
+# ClawMem zerank-2 seq-cls reranker sidecar.
+# Serves zeroentropy/zerank-2-reranker as a faithful Qwen3ForSequenceClassification cross-encoder
+# (the llama.cpp GGUF route drops zerank's score head; this serves the real one via transformers).
+#
+# LICENSE: this image bundles only the MIT recipe code. zerank-2's WEIGHTS are CC-BY-NC-4.0
+# (non-commercial) and are downloaded at convert time for YOUR own use — never redistributed here.
+FROM pytorch/pytorch:2.6.0-cuda12.4-cudnn9-runtime
+
+RUN pip install --no-cache-dir \
+      "transformers>=4.51" \
+      "huggingface_hub>=0.24" \
+      "uvicorn[standard]" \
+      fastapi \
+      safetensors \
+      accelerate
+
+WORKDIR /app
+COPY zr_common.py build_and_verify.py server.py /app/
+
+ENV ZR_OUT=/models/zerank-2-seq \
+    ZR_PORT=8090 \
+    ZR_TEMP=5.0 \
+    ZR_MAXLEN=8192
+
+EXPOSE 8090
+CMD ["python3", "server.py"]

package/extras/rerankers/zerank-2-seq/README.md

@@ -0,0 +1,107 @@
+# zerank-2 seq-cls reranker sidecar (SOTA, non-commercial)
+
+A drop-in `/v1/rerank` sidecar that serves **ZeroEntropy [zerank-2](https://huggingface.co/zeroentropy/zerank-2-reranker)** — a state-of-the-art cross-encoder (NDCG@10 ≈ 0.671, ahead of Cohere rerank-3.5 and Gemini-2.5-Flash listwise) — **faithfully**, via transformers, with a reproducible correctness gate.
+
+Point ClawMem at it and nothing else changes:
+
+```bash
+export CLAWMEM_RERANK_URL=http://<this-host>:8090
+```
+
+ClawMem's `query` and `intent_search` pipelines call `/v1/rerank` on that URL exactly as before.
+
+---
+
+## Why this exists (and why the GGUF is deprecated)
+
+The previously-recommended `zerank-2-Q4_K_M.gguf` reranker is **broken** and is deprecated. zerank-2 is a `Qwen3ForCausalLM` that scores a `(query, document)` pair on the logit of a single relevance token (`"Yes"`, id `9454`) via a sentence-transformers `LogitScore` head. llama.cpp's `convert_hf_to_gguf.py` only synthesizes a rerank head when it finds the literal string `# Qwen3-Reranker` in the model card — zerank-2's card lacks it, so the previously-recommended GGUF (and anything built by the current/standard llama.cpp converter path) is a **headless causal LM**. Served under `--reranking` it produces near-zero, uninformative scores → reranking degrades to an inert RRF-dominated passthrough, silently.
+
+This recipe sidesteps GGUF entirely. Because zerank-2 uses **tied embeddings**, the relevance logit `hidden · embed_tokens.weight[9454]` is reproduced **exactly** by a standard `Qwen3ForSequenceClassification` whose `num_labels=1` score head is that one embedding row. We convert to that form, **prove** the conversion is bit-exact, and serve it with the model's real chat template and `sigmoid(logit/5)` calibration.
+
+> **License:** zerank-2 is **CC-BY-NC-4.0 (non-commercial)**. This recipe is MIT code; it **never bundles the weights** — `build_and_verify.py` downloads them for *your own* non-commercial use. ClawMem's default reranker stays the permissively-licensed `qwen3-reranker-0.6B`; this is an opt-in upgrade.
+
+---
+
+## Requirements
+
+- An NVIDIA GPU with **~9 GiB free VRAM** (bf16) for the reranker. A 12 GB card is comfortable for the sidecar **alone**; budget **16 GB+** for the full co-located SOTA stack (embedding + LLM + reranker).
+- **NVIDIA Container Toolkit** (for `--gpus`/`docker compose` GPU access).
+- A Hugging Face account that has **accepted the zerank-2 license**, and an `HF_TOKEN` for the one-off convert step.
+
+---
+
+## Quick start (Docker Compose)
+
+```bash
+cd extras/rerankers/zerank-2-seq
+docker compose build
+
+# One-off: download + convert + run ALL correctness gates into ./models/zerank-2-seq
+HF_TOKEN=hf_xxx docker compose run --rm convert
+
+# Serve on :8090 (boot-persistent)
+docker compose up -d reranker
+
+# Verify
+curl -s localhost:8090/health
+curl -s -X POST localhost:8090/v1/rerank -H 'Content-Type: application/json' \
+  -d '{"query":"capital of France","documents":["Paris is the capital of France.","Bananas grow in the tropics."]}'
+# -> relevant ~0.96, irrelevant ~0.08
+```
+
+Then set `CLAWMEM_RERANK_URL=http://<this-host>:8090` for ClawMem (see the main README's *GPU Services*).
+
+The convert step **refuses to finish unless every gate passes** (`RESULT PASS`, exit 0). If a gate fails it exits non-zero and does not leave a half-baked model serving.
+
+---
+
+## What the gate proves
+
+`build_and_verify.py` is the correctness proof — it converts, then asserts:
+
+| Gate | Asserts |
+|---|---|
+| **GATE 1** | the score-head row equals `lm_head[9454]` in fp32, saved as bf16 (no silent fp16 downcast) |
+| **GATE 1.5** | the served tokenizer == the source tokenizer (formatted strings + token ids + `encode_pair` through truncation); the transformers `fix_mistral_regex` warning is benign |
+| **GATE-AP** | the assistant-generation prefix survives even a near-`MAXLEN` input (truncating it would move the scored position) |
+| **GATE 2** | the seq-cls logit equals the causal model's token-9454 logit **bit-exactly** over the real served path, incl. batched right-padded pooling and empty/whitespace-doc edges |
+
+It runs against the source model, so it verifies **any** copy of the weights — see the two source paths below.
+
+---
+
+## Two ways to get the weights
+
+1. **Reproduce (trustless, default).** `build_and_verify.py` downloads `zeroentropy/zerank-2-reranker` and performs the conversion itself, then gates it. You trust only ZeroEntropy's official weights + this MIT code.
+2. **Pull a pre-converted upload, then verify it.** A community conversion exists at [`baseten-admin/zerank-2-reranker-seq`](https://huggingface.co/baseten-admin/zerank-2-reranker-seq). Download it into `./models/zerank-2-seq`, then run the gate in **verify-only** mode — it skips conversion but still downloads the official `ZR_SRC` to prove the seq-cls logits match: `ZR_VERIFY_ONLY=1 HF_TOKEN=hf_xxx docker compose run --rm convert`. The gate verifies whatever you point `ZR_OUT` at, and **fails a bad upload** (e.g. one converted on the wrong token).
+
+---
+
+## How it works
+
+- **Conversion** (`build_and_verify.py`): copy `embed_tokens.weight[9454]` into a `Linear(hidden, 1, bias=False)` score head of `Qwen3ForSequenceClassification(num_labels=1)`, in **bf16**. Tied embeddings make this logit identical to the native causal score by construction.
+- **Serving** (`server.py`): `batch=1` (deterministic, no padding drift), applies zerank's chat template (`query`→system, `document`→user, assistant prefix), returns `sigmoid(logit/5)` — the native calibration.
+- **Shared encoder** (`zr_common.py`): the gate and the server import the *same* `encode_pair`, so served scores == gated scores by construction, and the assistant prefix is never truncated.
+
+---
+
+## Configuration
+
+| Env | Default | Meaning |
+|---|---|---|
+| `ZR_SRC` | `zeroentropy/zerank-2-reranker` | source model to convert |
+| `ZR_OUT` | `/models/zerank-2-seq` | converted seq-cls model path (mounted volume) |
+| `ZR_PORT` | `8090` | sidecar port |
+| `ZR_TEMP` | `5.0` | calibration temperature → `sigmoid(logit/ZR_TEMP)` |
+| `ZR_MAXLEN` | `8192` | max tokens per pair (doc tail truncated to fit; prefix preserved) |
+| `HF_TOKEN` | — | required for the convert download (gated model) |
+
+---
+
+## Rollback
+
+This sidecar only occupies `:8090` and speaks the standard contract, so reverting is just pointing `CLAWMEM_RERANK_URL` back at your previous reranker (e.g. a `qwen3-reranker-0.6B` `llama-server`) and stopping the container:
+
+```bash
+docker compose down
+```

package/extras/rerankers/zerank-2-seq/build_and_verify.py

@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+"""Reproduce zerank-2 -> Qwen3ForSequenceClassification(num_labels=1), bf16, token 9454, then GATE.
+GATE1   : fp32 weight-equality (score row == lm_head[9454], bf16 preserved).
+GATE1.5 : served-tokenizer path identity  -> OUT tok == SRC tok: (a) formatted-string + ids,
+          (b) encode_pair(SRC)==encode_pair(OUT) INCLUDING the truncation path,
+          (c) fix_mistral_regex default-vs-flag token-id identity (persisted).
+GATE-AP : assistant-prefix preserved under near-MAXLEN via encode_pair.
+GATE2   : runtime FIDELITY (seq-cls logit == causal token-9454 logit) over the EXACT served path
+          (OUT tokenizer + encode_pair, batch=1) + a batched right-padded pooling check + edge cases.
+Exit 0 only if every gate passes. Runs in the provided Dockerfile image (torch + transformers + CUDA)."""
+import os, sys, gc, math
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import torch
+from transformers import AutoModelForCausalLM, AutoModelForSequenceClassification, AutoTokenizer
+from zr_common import format_pair, encode_pair, ASSISTANT_PREFIX
+
+SRC = os.environ.get("ZR_SRC", "zeroentropy/zerank-2-reranker")
+OUT = os.environ.get("ZR_OUT", "/models/zerank-2-seq")
+TOKEN_ID = 9454
+TEMP = 5.0
+MAXLEN = int(os.environ.get("ZR_MAXLEN", "8192"))
+DEV = "cuda" if torch.cuda.is_available() else "cpu"
+VERIFY_ONLY = os.environ.get("ZR_VERIFY_ONLY", "").lower() in ("1", "true", "yes")  # gate an existing OUT (e.g. a pulled upload) without re-converting
+def log(*a): print("[build]", *a, flush=True)
+
+# ---- Phase A: convert (CPU, bf16, explicit token 9454) — skipped under ZR_VERIFY_ONLY ----
+if VERIFY_ONLY:
+    if not os.path.isdir(OUT):
+        log(f"ZR_VERIFY_ONLY set but {OUT} does not exist — nothing to verify"); sys.exit(2)
+    log(f"ZR_VERIFY_ONLY: skipping conversion; gating the existing model at {OUT} against {SRC}")
+else:
+    log(f"loading causal {SRC} (bf16, cpu)")
+    causal = AutoModelForCausalLM.from_pretrained(SRC, torch_dtype=torch.bfloat16)
+    assert causal.lm_head.bias is None
+    assert getattr(causal.config, "tie_word_embeddings", False)
+    hidden = causal.lm_head.in_features
+    row = causal.lm_head.weight[TOKEN_ID].detach().clone()
+    log(f"extracted lm_head row {TOKEN_ID}: shape={tuple(row.shape)} dtype={row.dtype}")
+    del causal; gc.collect()
+
+    log("loading seq-cls backbone (bf16, cpu, num_labels=1)")
+    seq = AutoModelForSequenceClassification.from_pretrained(SRC, num_labels=1, torch_dtype=torch.bfloat16)
+    lin = torch.nn.Linear(hidden, 1, bias=False, dtype=torch.bfloat16)
+    with torch.no_grad():
+        lin.weight.copy_(row.unsqueeze(0))
+    seq.score = lin
+    seq.config.num_labels = 1
+    seq.config.id2label = {0: "relevant"}; seq.config.label2id = {"relevant": 0}
+    tok = AutoTokenizer.from_pretrained(SRC)
+    pad_id = tok.pad_token_id if tok.pad_token_id is not None else tok.convert_tokens_to_ids("<|endoftext|>")
+    seq.config.pad_token_id = pad_id
+    tok.padding_side = "right"
+    log(f"pad_token_id={pad_id} padding_side=right score.weight.dtype={seq.score.weight.dtype}")
+    os.makedirs(OUT, exist_ok=True)
+    seq.save_pretrained(OUT); tok.save_pretrained(OUT)
+    log(f"saved seq-cls -> {OUT}")
+    del seq; gc.collect()
+
+# ---- GATE 1: fp32 weight equality ----
+log("GATE1: reload + fp32 weight equality")
+c2 = AutoModelForCausalLM.from_pretrained(SRC, torch_dtype=torch.bfloat16)
+src_row = c2.lm_head.weight[TOKEN_ID].float().cpu(); del c2; gc.collect()
+s2 = AutoModelForSequenceClassification.from_pretrained(OUT, num_labels=1, torch_dtype=torch.bfloat16)
+saved_dtype = s2.score.weight.dtype
+saved_row = s2.score.weight[0].float().cpu(); del s2; gc.collect()
+maxd1 = (src_row - saved_row).abs().max().item()
+gate1 = maxd1 < 1e-7
+log(f"GATE1 saved_score_dtype={saved_dtype} weight_maxdiff(fp32)={maxd1:.3e} -> {'PASS' if gate1 else 'FAIL'}")
+if not gate1:
+    log("GATE1 FAIL — aborting."); sys.exit(2)
+
+PAIRS = [
+    ("What is the capital of France?", "The capital of France is Paris."),
+    ("What is the capital of France?", "Bananas are rich in potassium and grow in tropical climates."),
+    ("How do I configure TLS in nginx?", "In nginx set ssl_certificate and ssl_certificate_key in the server block on 443, then reload."),
+    ("python async database connection pooling", "asyncpg exposes create_pool; set min_size/max_size and acquire via 'async with pool.acquire()'."),
+    ("short q", "x"),
+    ("distributed consensus and leader election in raft", ("Raft elects a leader via randomized election timeouts. " * 12)[:400]),
+    ("empty doc edge", ""),
+    ("whitespace doc edge", "   "),
+]
+
+# ---- GATE 1.5: served-tokenizer path identity (OUT == SRC) ----
+src_tok = AutoTokenizer.from_pretrained(SRC); src_tok.padding_side = "right"
+out_tok = AutoTokenizer.from_pretrained(OUT); out_tok.padding_side = "right"
+pad_id = out_tok.pad_token_id if out_tok.pad_token_id is not None else out_tok.convert_tokens_to_ids("<|endoftext|>")  # set here so GATE2 has it under ZR_VERIFY_ONLY too
+# (a) formatted-string + token-id identity
+tokpath_ok = True
+for q, d in PAIRS:
+    s_str, o_str = format_pair(src_tok, q, d), format_pair(out_tok, q, d)
+    if (s_str != o_str) or (src_tok(s_str, add_special_tokens=False)["input_ids"] != out_tok(o_str, add_special_tokens=False)["input_ids"]):
+        tokpath_ok = False; log(f"  GATE1.5a MISMATCH q={q!r}")
+log(f"GATE1.5a served==source tokenizer (string+ids) -> {'PASS' if tokpath_ok else 'FAIL'}")
+# (b) encode_pair(SRC) == encode_pair(OUT) INCLUDING the truncation path (gate path == served path)
+ep_ok = True
+ep_pairs = PAIRS + [("very long query " + "stress " * 3000, "document body " + "lorem ipsum " * 6000)]
+for q, d in ep_pairs:
+    a = encode_pair(src_tok, q, d, max_total_tokens=MAXLEN)["input_ids"].tolist()
+    b = encode_pair(out_tok, q, d, max_total_tokens=MAXLEN)["input_ids"].tolist()
+    if a != b: ep_ok = False; log(f"  GATE1.5b encode_pair MISMATCH q[:30]={q[:30]!r} lens {len(a[0])} vs {len(b[0])}")
+log(f"GATE1.5b encode_pair(SRC)==encode_pair(OUT) incl truncation -> {'PASS' if ep_ok else 'FAIL'}")
+# (c) fix_mistral_regex default-vs-flag token-id identity (the transformers warning is benign if 0 diffs)
+try:
+    fix_tok = AutoTokenizer.from_pretrained(OUT, fix_mistral_regex=True); fix_flag = "supported"
+except TypeError as e:
+    fix_tok = None; fix_flag = f"UNSUPPORTED({type(e).__name__})"
+regex_diffs = 0
+for q, d in PAIRS:
+    base = out_tok(format_pair(out_tok, q, d), add_special_tokens=False)["input_ids"]
+    cmp = (fix_tok or out_tok)(format_pair((fix_tok or out_tok), q, d), add_special_tokens=False)["input_ids"]
+    if base != cmp: regex_diffs += 1
+mixed = "cafe naive 2026 x=1;y=2 https://a.b/c?q=1 path/to::thing emoji"
+mixed_diff = out_tok(mixed, add_special_tokens=False)["input_ids"] != (fix_tok or out_tok)(mixed, add_special_tokens=False)["input_ids"]
+regex_ok = (regex_diffs == 0) and (not mixed_diff)
+log(f"GATE1.5c fix_mistral_regex flag={fix_flag} pair_id_diffs={regex_diffs} mixed_diff={mixed_diff} -> {'BENIGN/PASS' if regex_ok else 'MATERIAL/FAIL'}")
+
+# ---- GATE-AP: assistant-prefix preserved under near-MAXLEN input ----
+ap_ids = out_tok(ASSISTANT_PREFIX, add_special_tokens=False)["input_ids"]
+long_doc = "lorem ipsum dolor sit amet consectetur " * 4000
+enc_long = encode_pair(out_tok, "very long reranking stress-test query " * 200, long_doc, max_total_tokens=MAXLEN, device="cpu")
+long_ids = enc_long["input_ids"][0].tolist()
+ap_ok = (len(long_ids) <= MAXLEN) and (long_ids[-len(ap_ids):] == ap_ids)
+log(f"GATE-AP near-MAXLEN: len={len(long_ids)}<= {MAXLEN} and tail==assistant_prefix -> {'PASS' if ap_ok else 'FAIL'}")
+
+# ---- GATE 2: runtime fidelity over the EXACT served path (OUT tokenizer + encode_pair) + batched pooling ----
+log("GATE2: loading causal on GPU")
+c3 = AutoModelForCausalLM.from_pretrained(SRC, torch_dtype=torch.bfloat16).to(DEV).eval()
+causal_served, causal_single, causal_batched = [], [], []
+with torch.no_grad():
+    for q, d in PAIRS:
+        enc = encode_pair(out_tok, q, d, max_total_tokens=MAXLEN, device=DEV)     # EXACT served path (OUT tok)
+        li = int(enc["attention_mask"][0].sum().item()) - 1
+        causal_served.append(c3(**enc).logits[0, li, TOKEN_ID].float().item())
+    texts = [format_pair(out_tok, q, d) for q, d in PAIRS]
+    for t in texts:
+        e = out_tok(t, return_tensors="pt", add_special_tokens=False).to(DEV)
+        causal_single.append(c3(**e).logits[0, -1, TOKEN_ID].float().item())
+    eb = out_tok(texts, return_tensors="pt", padding=True, add_special_tokens=False).to(DEV)
+    ob = c3(**eb).logits; last = eb["attention_mask"].sum(1) - 1
+    causal_batched = [ob[i, last[i], TOKEN_ID].float().item() for i in range(len(texts))]
+del c3; gc.collect(); torch.cuda.empty_cache()
+
+log("GATE2: loading seq-cls on GPU")
+s3 = AutoModelForSequenceClassification.from_pretrained(OUT, num_labels=1, torch_dtype=torch.bfloat16).to(DEV).eval()
+s3.config.pad_token_id = pad_id
+seq_served, seq_single, seq_batched = [], [], []
+with torch.no_grad():
+    for q, d in PAIRS:
+        enc = encode_pair(out_tok, q, d, max_total_tokens=MAXLEN, device=DEV)     # EXACT served path (OUT tok)
+        seq_served.append(s3(**enc).logits.reshape(-1)[0].float().item())
+    for t in texts:
+        e = out_tok(t, return_tensors="pt", add_special_tokens=False).to(DEV)
+        seq_single.append(s3(**e).logits.reshape(-1)[0].float().item())
+    eb = out_tok(texts, return_tensors="pt", padding=True, add_special_tokens=False).to(DEV)
+    seq_batched = s3(**eb).logits.reshape(-1).float().tolist()
+del s3; gc.collect(); torch.cuda.empty_cache()
+
+def sig(x): return 1.0 / (1.0 + math.exp(-x / TEMP))
+max_served = max_eqB = max_eqS = max_dS = max_dC = 0.0
+log("idx servd(prod) seqServd eqServed eqB|seqB-causalB| eqS|seq1-causal1| driftSeq driftCausal score")
+for i, (q, d) in enumerate(PAIRS):
+    eqSrv = abs(seq_served[i] - causal_served[i])      # PRODUCTION path fidelity (OUT tok + encode_pair, batch=1)
+    eqB = abs(seq_batched[i] - causal_batched[i]); eqS = abs(seq_single[i] - causal_single[i])
+    dS = abs(seq_batched[i] - seq_single[i]); dC = abs(causal_batched[i] - causal_single[i])
+    max_served = max(max_served, eqSrv); max_eqB = max(max_eqB, eqB); max_eqS = max(max_eqS, eqS)
+    max_dS = max(max_dS, dS); max_dC = max(max_dC, dC)
+    log(f"{i:>3} {causal_served[i]:+8.3f} {seq_served[i]:+8.3f}  {eqSrv:.1e}    {eqB:.1e}        {eqS:.1e}    {dS:.1e}  {dC:.1e}  {sig(seq_served[i]):.4f}")
+
+TOL = 1e-3
+gate2 = (max_served < TOL) and (max_eqB < TOL) and (max_eqS < TOL)
+log(f"GATE2 FIDELITY served={max_served:.2e} batched={max_eqB:.2e} single={max_eqS:.2e} TOL={TOL} -> {'PASS' if gate2 else 'FAIL'}")
+log(f"INFO backbone padding drift max|seqB-seq1|={max_dS:.2e} ~= max|causalB-causal1|={max_dC:.2e} (shared bf16 right-pad)")
+log(f"SANITY relevant>irrelevant: {sig(seq_served[0]):.4f} > {sig(seq_served[1]):.4f} -> {'OK' if seq_served[0] > seq_served[1] else 'WARN'}")
+allpass = gate1 and tokpath_ok and ep_ok and regex_ok and ap_ok and gate2
+log(f"GATES gate1={gate1} tokpath={tokpath_ok} encode_pair_id={ep_ok} fix_mistral_regex={regex_ok} assistant_prefix={ap_ok} gate2={gate2}")
+print("RESULT", "PASS" if allpass else "FAIL", flush=True)
+sys.exit(0 if allpass else 3)

package/extras/rerankers/zerank-2-seq/docker-compose.yml

@@ -0,0 +1,49 @@
+# ClawMem zerank-2 seq-cls reranker sidecar.
+#
+#   1. Build:              docker compose build
+#   2. Convert + verify:   HF_TOKEN=hf_xxx docker compose run --rm convert
+#   3. Serve on :8090:     docker compose up -d reranker
+#
+# Then point ClawMem at it:   export CLAWMEM_RERANK_URL=http://<this-host>:8090
+#
+# Requires the NVIDIA Container Toolkit. zerank-2 weights are CC-BY-NC-4.0 (non-commercial);
+# accept the model license on Hugging Face and pass HF_TOKEN for the one-off convert step.
+x-gpu: &gpu
+  deploy:
+    resources:
+      reservations:
+        devices:
+          - driver: nvidia
+            count: all
+            capabilities: [gpu]
+
+services:
+  reranker:                     # the live sidecar — serves the verified model on :8090
+    build: .
+    image: clawmem-zerank-2-seq
+    restart: unless-stopped
+    ports:
+      - "8090:8090"
+    volumes:
+      - ./models:/models
+    environment:
+      ZR_OUT: /models/zerank-2-seq
+      ZR_PORT: "8090"
+    <<: *gpu
+
+  convert:                      # one-off: download + convert + run all gates into ./models
+    build: .
+    image: clawmem-zerank-2-seq
+    profiles: ["tools"]
+    environment:
+      ZR_SRC: zeroentropy/zerank-2-reranker
+      ZR_OUT: /models/zerank-2-seq
+      HF_TOKEN: ${HF_TOKEN:-}
+    volumes:
+      - ./models:/models
+      - hf-cache:/root/.cache/huggingface
+    command: ["python3", "build_and_verify.py"]
+    <<: *gpu
+
+volumes:
+  hf-cache:

package/extras/rerankers/zerank-2-seq/server.py

@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+"""ClawMem zerank-2 seq-cls reranker sidecar. Drop-in for CLAWMEM_RERANK_URL.
+POST /v1/rerank {query, documents:[...]} -> {results:[{index, relevance_score}]}.
+Uses zr_common.encode_pair (the SAME path the build gate verified) -> served scores == gated scores by
+construction, and the assistant prefix is never truncated. batch=1 -> deterministic, drift-free scores."""
+import os, sys, math, torch
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from fastapi import FastAPI
+from pydantic import BaseModel
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+from zr_common import encode_pair
+
+MODEL = os.environ.get("ZR_OUT", "/models/zerank-2-seq")
+TEMP = float(os.environ.get("ZR_TEMP", "5.0"))
+MAXLEN = int(os.environ.get("ZR_MAXLEN", "8192"))
+DEV = "cuda" if torch.cuda.is_available() else "cpu"
+
+tok = AutoTokenizer.from_pretrained(MODEL); tok.padding_side = "right"
+model = AutoModelForSequenceClassification.from_pretrained(MODEL, num_labels=1, torch_dtype=torch.bfloat16).to(DEV).eval()
+if model.config.pad_token_id is None:
+    model.config.pad_token_id = tok.pad_token_id or tok.convert_tokens_to_ids("<|endoftext|>")
+
+app = FastAPI()
+
+class RerankReq(BaseModel):
+    query: str
+    documents: list[str]
+
+@app.post("/v1/rerank")
+@torch.no_grad()
+def rerank(req: RerankReq):
+    results = []
+    for i, d in enumerate(req.documents):                       # batch=1: deterministic, no padding drift
+        enc = encode_pair(tok, req.query, d, max_total_tokens=MAXLEN, device=DEV)
+        lg = model(**enc).logits.reshape(-1)[0].float().item()
+        results.append({"index": i, "relevance_score": 1.0 / (1.0 + math.exp(-lg / TEMP))})
+    return {"results": results}
+
+@app.get("/v1/models")
+def models():
+    return {"models": [{"id": "zerank-2-seq", "object": "model"}]}
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "model": MODEL, "device": DEV}
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("ZR_PORT", "8090")))

package/extras/rerankers/zerank-2-seq/zr_common.py

@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+"""Shared formatting + token-budgeting for the zerank-2 seq-cls reranker.
+Imported by BOTH build_and_verify.py (the gate) and server.py (serving) so the gated path and the
+served path are identical BY CONSTRUCTION.
+
+Guarantees the assistant-generation prefix is NEVER truncated: truncating it would move the scored/pooled
+position off the assistant prefix -> wrong score. We right-truncate the DOCUMENT
+tail (then the QUERY tail if needed) BEFORE templating, leaving room for the template scaffold + assistant
+prefix, with a final left-truncation belt so the prefix (always at the end) survives any re-encode overshoot."""
+
+ASSISTANT_PREFIX = "<|im_start|>assistant\n"
+
+
+def format_pair(tok, query, document):
+    return tok.apply_chat_template(
+        [{"role": "query", "content": query}, {"role": "document", "content": document}],
+        tokenize=False, add_generation_prompt=True)
+
+
+def _ids(tok, text):
+    return tok(text, add_special_tokens=False)["input_ids"]
+
+
+def encode_pair(tok, query, document, max_total_tokens=8192, device="cpu", safety=16):
+    """Return a tokenized (query, document) pair whose total length <= max_total_tokens and whose tail is
+    always the assistant prefix. Truncates document tail first, then query tail, then (belt) left-truncates."""
+    scaffold = len(_ids(tok, format_pair(tok, "", "")))
+    avail = max(2, max_total_tokens - scaffold - safety)
+    q_ids = _ids(tok, query)
+    if len(q_ids) > avail - 1:                      # cap query so a document always has room
+        q_ids = q_ids[:avail - 1]
+        query = tok.decode(q_ids)
+    d_budget = max(1, avail - len(q_ids))
+    d_ids = _ids(tok, document)
+    if len(d_ids) > d_budget:                        # truncate the document TAIL (least-important part)
+        d_ids = d_ids[:d_budget]
+        document = tok.decode(d_ids)
+    text = format_pair(tok, query, document)
+    enc = tok(text, return_tensors="pt", add_special_tokens=False)
+    if enc["input_ids"].shape[1] > max_total_tokens:  # belt: keep the LAST tokens -> assistant prefix survives
+        enc = {k: v[:, -max_total_tokens:] for k, v in enc.items()}
+        return {k: v.to(device) for k, v in enc.items()}
+    return enc.to(device)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.11.2",
+  "version": "0.11.3",
   "description": "On-device memory layer for AI agents. Claude Code, OpenClaw, and Hermes. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {
@@ -9,6 +9,7 @@
   "files": [
     "bin/",
     "src/",
+    "extras/",
     "README.md",
     "LICENSE",
     "CLAUDE.md",