threadkeeper 0.6.2__tar.gz → 0.8.0__tar.gz

{threadkeeper-0.6.2/threadkeeper.egg-info → threadkeeper-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: threadkeeper
-Version: 0.6.2
+Version: 0.8.0
 Summary: Multi-agent shared brain across Claude Code/Desktop, Codex, Gemini, Copilot, VS Code. Cross-session memory, self-improving skill loops, inter-agent signaling — one local MCP server.
 Author: thread-keeper contributors
 License: MIT
@@ -24,12 +24,17 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: mcp>=1.0.0
 Provides-Extra: semantic
-Requires-Dist: sentence-transformers>=2.2.0; extra == "semantic"
+Requires-Dist: fastembed>=0.3; extra == "semantic"
 Requires-Dist: numpy>=1.24.0; extra == "semantic"
 Requires-Dist: sqlite-vec>=0.1.9; extra == "semantic"
+Provides-Extra: semantic-st
+Requires-Dist: sentence-transformers>=2.2.0; extra == "semantic-st"
+Requires-Dist: numpy>=1.24.0; extra == "semantic-st"
+Requires-Dist: sqlite-vec>=0.1.9; extra == "semantic-st"
 Provides-Extra: dev
 Requires-Dist: pytest>=8.0; extra == "dev"
 Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: pytest-forked>=1.6; extra == "dev"
 Dynamic: license-file
 
 # thread-keeper
@@ -189,7 +194,7 @@ autonomous learning daemons cannot recursively start inside review forks.
 A daemon measures combined child RSS every 10 s; admission control
 refuses a new spawn that would exceed `THREADKEEPER_SPAWN_BUDGET_MB`
 (3 GB default). Slim children that need semantic search delegate to the
-parent via `search_via_parent` — no per-child copy of sentence-transformers.
+parent via `search_via_parent` — no per-child copy of the embedding model.
 
 ### Learning loops
 
@@ -435,7 +440,9 @@ The most-used env knobs (full list in `threadkeeper/config.py`):
 | `THREADKEEPER_MEMORY_GUARD_RETIRE_LIVE` | "" (off) | allow retiring parent-alive MCP servers; off protects live clients |
 | `THREADKEEPER_MEMORY_GUARD_NOTIFY` | "1" | send macOS desktop notification when possible |
 | `THREADKEEPER_INGEST_INTERVAL_S` | 3 | transcript ingest tick (s) |
-| `THREADKEEPER_NO_EMBEDDINGS` | "" | force-disable sentence-transformers |
+| `THREADKEEPER_NO_EMBEDDINGS` | "" | force-disable the embedding model (FTS5 + delegate only) |
+| `THREADKEEPER_EMBED_BACKEND` | `onnx` | embedding runtime: `onnx` (fastembed, no PyTorch) or `sentence-transformers` (legacy fallback) |
+| `THREADKEEPER_EMBED_MODEL` | `paraphrase-multilingual-MiniLM-L12-v2` | 384-dim cross-lingual embedding model |
 | `THREADKEEPER_SPAWNED_CHILD` | "" | spawn-internal marker; disables autonomous daemons in children |
 | `THREADKEEPER_SKILL_NUDGE_INTERVAL` | 10 | events between `skill_hint` nudges |
 
@@ -525,6 +532,34 @@ Hooks and small runtime artifacts: `~/.threadkeeper/hooks/`.
 
 ---
 
+## Embeddings
+
+Semantic search runs `paraphrase-multilingual-MiniLM-L12-v2` (384-dim,
+RU+EN+50 langs). The default backend is **fastembed / ONNX Runtime** — no
+PyTorch. A model-loaded process sits at ~700 MB physical footprint
+(~850 MB RSS), down from ~1.8 GB on the PyTorch backend.
+
+A **sentence-transformers** (PyTorch) backend is kept as an opt-in fallback.
+It is heavier (~1.8 GB RSS) and produces vectors that are *not numerically
+identical* to the ONNX backend's, so switching backends warrants a recompute:
+
+```bash
+# Install the fallback runtime and switch to it:
+pip install -e '.[semantic-st]'
+export THREADKEEPER_EMBED_BACKEND=sentence-transformers
+
+# After any backend switch, homogenize the stored corpus so queries and
+# stored vectors live in the same space:
+tk-migrate-embeddings --all          # or --notes-only / --dialog-only
+tk-migrate-embeddings --dry-run      # report stale counts only
+```
+
+The migration is batched, resumable, and idempotent (a second run finds
+nothing stale). Both backends emit 384-dim vectors, so the `vec0` schema is
+unchanged.
+
+---
+
 ## Verifying ingest across CLIs
 
 ```bash

{threadkeeper-0.6.2 → threadkeeper-0.8.0}/README.md

@@ -155,7 +155,7 @@ autonomous learning daemons cannot recursively start inside review forks.
 A daemon measures combined child RSS every 10 s; admission control
 refuses a new spawn that would exceed `THREADKEEPER_SPAWN_BUDGET_MB`
 (3 GB default). Slim children that need semantic search delegate to the
-parent via `search_via_parent` — no per-child copy of sentence-transformers.
+parent via `search_via_parent` — no per-child copy of the embedding model.
 
 ### Learning loops
 
@@ -401,7 +401,9 @@ The most-used env knobs (full list in `threadkeeper/config.py`):
 | `THREADKEEPER_MEMORY_GUARD_RETIRE_LIVE` | "" (off) | allow retiring parent-alive MCP servers; off protects live clients |
 | `THREADKEEPER_MEMORY_GUARD_NOTIFY` | "1" | send macOS desktop notification when possible |
 | `THREADKEEPER_INGEST_INTERVAL_S` | 3 | transcript ingest tick (s) |
-| `THREADKEEPER_NO_EMBEDDINGS` | "" | force-disable sentence-transformers |
+| `THREADKEEPER_NO_EMBEDDINGS` | "" | force-disable the embedding model (FTS5 + delegate only) |
+| `THREADKEEPER_EMBED_BACKEND` | `onnx` | embedding runtime: `onnx` (fastembed, no PyTorch) or `sentence-transformers` (legacy fallback) |
+| `THREADKEEPER_EMBED_MODEL` | `paraphrase-multilingual-MiniLM-L12-v2` | 384-dim cross-lingual embedding model |
 | `THREADKEEPER_SPAWNED_CHILD` | "" | spawn-internal marker; disables autonomous daemons in children |
 | `THREADKEEPER_SKILL_NUDGE_INTERVAL` | 10 | events between `skill_hint` nudges |
 
@@ -491,6 +493,34 @@ Hooks and small runtime artifacts: `~/.threadkeeper/hooks/`.
 
 ---
 
+## Embeddings
+
+Semantic search runs `paraphrase-multilingual-MiniLM-L12-v2` (384-dim,
+RU+EN+50 langs). The default backend is **fastembed / ONNX Runtime** — no
+PyTorch. A model-loaded process sits at ~700 MB physical footprint
+(~850 MB RSS), down from ~1.8 GB on the PyTorch backend.
+
+A **sentence-transformers** (PyTorch) backend is kept as an opt-in fallback.
+It is heavier (~1.8 GB RSS) and produces vectors that are *not numerically
+identical* to the ONNX backend's, so switching backends warrants a recompute:
+
+```bash
+# Install the fallback runtime and switch to it:
+pip install -e '.[semantic-st]'
+export THREADKEEPER_EMBED_BACKEND=sentence-transformers
+
+# After any backend switch, homogenize the stored corpus so queries and
+# stored vectors live in the same space:
+tk-migrate-embeddings --all          # or --notes-only / --dialog-only
+tk-migrate-embeddings --dry-run      # report stale counts only
+```
+
+The migration is batched, resumable, and idempotent (a second run finds
+nothing stale). Both backends emit 384-dim vectors, so the `vec0` schema is
+unchanged.
+
+---
+
 ## Verifying ingest across CLIs
 
 ```bash

{threadkeeper-0.6.2 → threadkeeper-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "threadkeeper"
-version = "0.6.2"
+version = "0.8.0"
 description = "Multi-agent shared brain across Claude Code/Desktop, Codex, Gemini, Copilot, VS Code. Cross-session memory, self-improving skill loops, inter-agent signaling — one local MCP server."
 requires-python = ">=3.11"
 authors = [{ name = "thread-keeper contributors" }]
@@ -32,15 +32,27 @@ dependencies = [
 [project.optional-dependencies]
 # Semantic cross-language search + sub-linear vector index. Recommended
 # for any real use — without it, dialog_search falls back to FTS5 only.
+# Default backend is fastembed/ONNX Runtime: no PyTorch, ~700MB footprint.
 semantic = [
+    "fastembed>=0.3",
+    "numpy>=1.24.0",
+    "sqlite-vec>=0.1.9",
+]
+# Legacy PyTorch backend, kept as an opt-in fallback. Install this AND set
+# THREADKEEPER_EMBED_BACKEND=sentence-transformers to use it. ~1.8GB RSS.
+semantic-st = [
     "sentence-transformers>=2.2.0",
     "numpy>=1.24.0",
     "sqlite-vec>=0.1.9",
 ]
-# Test runner + coverage.
+# Test runner + coverage. pytest-forked isolates each test in its own
+# process: the per-test package re-import (tests/conftest.py) accumulates
+# native ONNX/tokenizer thread pools that can deadlock sqlite finalize in a
+# single long-lived process, so CI runs `pytest --forked`.
 dev = [
     "pytest>=8.0",
     "pytest-cov>=5.0",
+    "pytest-forked>=1.6",
 ]
 
 [project.urls]
@@ -54,6 +66,9 @@ Changelog     = "https://github.com/po4erk91/thread-keeper/releases"
 # After `pip install threadkeeper`, the user gets `thread-keeper-setup`
 # directly on PATH. Equivalent to `python -m threadkeeper._setup`.
 thread-keeper-setup = "threadkeeper._setup:main"
+# Recompute stored embeddings with the active backend (e.g. after switching to
+# the ONNX default). Equivalent to `python -m threadkeeper.migrate_embeddings`.
+tk-migrate-embeddings = "threadkeeper.migrate_embeddings:main"
 
 [tool.setuptools.packages.find]
 include = ["threadkeeper*"]

{threadkeeper-0.6.2 → threadkeeper-0.8.0}/tests/test_curator.py

@@ -321,3 +321,85 @@ def test_advisory_mode_default_excludes_destructive_tools(
     assert "lesson_append" not in allowed
     assert "ADVISORY MODE" in kw["prompt"]
     assert "DESTRUCTIVE MODE ENABLED" not in kw["prompt"]
+
+
+# ──────────────────────────────────────────────────────────────────────
+# Concepts review (F1) — curator also audits the concepts store
+# ──────────────────────────────────────────────────────────────────────
+
+def _add_concept(conn, cid, desc, confidence="medium",
+                 registered_at=None, last_evidence_at=None):
+    now = int(time.time())
+    conn.execute(
+        "INSERT INTO concepts (id, description, confidence, registered_at, "
+        "last_evidence_at) VALUES (?,?,?,?,?)",
+        (cid, desc, confidence, registered_at or now, last_evidence_at),
+    )
+    conn.commit()
+
+
+def test_collect_concepts_empty(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    text, n = pkg["curator"]._collect_concepts(conn)
+    assert n == 0
+    assert text == ""
+
+
+def test_collect_concepts_lists_with_age(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    now = int(time.time())
+    _add_concept(conn, "Cfresh", "fresh high-conf idea",
+                 confidence="high", last_evidence_at=now - 86400)  # 1d
+    _add_concept(conn, "Cstale", "stale low-conf idea",
+                 confidence="low",
+                 registered_at=now - 40 * 86400,
+                 last_evidence_at=None)  # never corroborated, 40d old
+    text, n = pkg["curator"]._collect_concepts(conn)
+    assert n == 2
+    assert "Cfresh" in text and "Cstale" in text
+    assert "CONCEPTS (n=2)" in text
+    # stale concept (no last_evidence, registered 40d ago) shows ~40d age
+    assert "40d_ago" in text
+    # oldest-first ordering: stale concept appears before fresh one
+    assert text.index("Cstale") < text.index("Cfresh")
+
+
+def test_run_curator_pass_includes_concepts_in_inventory(
+    tmp_path, monkeypatch,
+):
+    pkg = _bootstrap(tmp_path, monkeypatch, min_lessons="2")
+    pkg["lessons"].append_lesson(title="a", body="b1", source="shadow")
+    pkg["lessons"].append_lesson(title="b", body="b2", source="shadow")
+    conn = pkg["db"].get_db()
+    _add_concept(conn, "Cabc", "asymmetric in-band reactivity",
+                 confidence="high")
+
+    import threadkeeper.tools.spawn as spawn_mod
+    captured: list[dict] = []
+    monkeypatch.setattr(
+        spawn_mod, "spawn",
+        lambda **kw: captured.append(kw) or "spawn task_id=fake pid=0",
+    )
+    pkg["curator"].run_curator_pass(force=True)
+    prompt = captured[0]["prompt"]
+    assert "CONCEPTS (n=1)" in prompt
+    assert "Cabc" in prompt
+    assert "asymmetric in-band reactivity" in prompt
+
+
+def test_concepts_alone_do_not_trigger_pass(tmp_path, monkeypatch):
+    """Concepts enrich the review but don't lower the lesson threshold —
+    a pass still requires CURATOR_MIN_LESSONS lessons."""
+    pkg = _bootstrap(tmp_path, monkeypatch, min_lessons="3")
+    conn = pkg["db"].get_db()
+    _add_concept(conn, "Conly", "a lone concept", confidence="high")
+
+    import threadkeeper.tools.spawn as spawn_mod
+    called = []
+    monkeypatch.setattr(spawn_mod, "spawn",
+                        lambda **kw: called.append(kw) or "x")
+    out = pkg["curator"].run_curator_pass(force=True)
+    assert out.startswith("below_threshold")
+    assert called == []

threadkeeper-0.8.0/tests/test_evolve_daemon.py

@@ -0,0 +1,187 @@
+"""Evolve reviewer daemon — autonomous triage of the format-evolution queue.
+
+The daemon never APPLIES a suggestion (that edits format/code). It spawns a
+child that calls evolve_decide(promote|dismiss) to keep the queue honest.
+Tests exercise the pure logic + dispatch with spawn monkeypatched; no real
+child is launched.
+"""
+from __future__ import annotations
+
+import sys
+import time
+from pathlib import Path
+
+
+_FAKE_CID = "dddd4444-5555-6666-7777-888899990000"
+
+
+def _bootstrap(tmp_path, monkeypatch, interval="0", review_min="2"):
+    env = {
+        "THREADKEEPER_DB": str(tmp_path / "db.sqlite"),
+        "CLAUDE_PROJECTS_DIR": str(tmp_path / "fake_claude_projects"),
+        "THREADKEEPER_INGEST_INTERVAL_S": "0",
+        "THREADKEEPER_INGEST_CAP": "0",
+        "THREADKEEPER_SKILL_WATCH_INTERVAL_S": "0",
+        "THREADKEEPER_SPAWN_BUDGET_POLL_S": "0",
+        "THREADKEEPER_SEARCH_PROXY_POLL_S": "0",
+        "THREADKEEPER_MEMORY_GUARD_POLL_S": "0",
+        "THREADKEEPER_SHADOW_REVIEW_INTERVAL_S": "0",
+        "THREADKEEPER_CURATOR_INTERVAL_S": "0",
+        "THREADKEEPER_EXTRACT_INTERVAL_S": "0",
+        "THREADKEEPER_CANDIDATE_REVIEW_INTERVAL_S": "0",
+        "THREADKEEPER_PROBE_INTERVAL_S": "0",
+        "THREADKEEPER_EVOLVE_REVIEW_INTERVAL_S": interval,
+        "THREADKEEPER_EVOLVE_REVIEW_MIN": review_min,
+        "THREADKEEPER_TASK_LOG_DIR": str(tmp_path / "tasks"),
+        "THREADKEEPER_CLIENT": "pytest",
+        "THREADKEEPER_FORCE_CID": _FAKE_CID,
+        "THREADKEEPER_NO_EMBEDDINGS": "1",
+    }
+    for k, v in env.items():
+        monkeypatch.setenv(k, v)
+    Path(env["CLAUDE_PROJECTS_DIR"]).mkdir(parents=True, exist_ok=True)
+    for name in [m for m in list(sys.modules) if m.startswith("threadkeeper")]:
+        del sys.modules[name]
+    import threadkeeper.server  # noqa: F401
+    from threadkeeper import _mcp, db, evolve_daemon, identity
+    return {"mcp": _mcp.mcp, "db": db, "ed": evolve_daemon, "identity": identity}
+
+
+def _tool(pkg, name):
+    return pkg["mcp"]._tool_manager._tools[name].fn
+
+
+def _add_evolve(conn, suggestion, rationale=None, applied=0, status="pending"):
+    conn.execute(
+        "INSERT INTO evolve (suggestion, rationale, applied, status, created_at) "
+        "VALUES (?,?,?,?,?)",
+        (suggestion, rationale, applied, status, int(time.time())),
+    )
+    conn.commit()
+
+
+# ── pending selection ──────────────────────────────────────────────────
+
+def test_pending_excludes_applied_and_decided(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "pending one")
+    _add_evolve(conn, "already applied", applied=1)
+    _add_evolve(conn, "already dismissed", status="dismissed")
+    _add_evolve(conn, "already promoted", status="promoted")
+    pend = pkg["ed"]._pending(conn)
+    sugg = [r["suggestion"] for r in pend]
+    assert sugg == ["pending one"]
+
+
+# ── evolve_decide tool ─────────────────────────────────────────────────
+
+def test_evolve_decide_promote(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "make briefs shorter")
+    eid = conn.execute("SELECT id FROM evolve").fetchone()["id"]
+    out = _tool(pkg, "evolve_decide")(evolve_id=eid, decision="promote",
+                                      reason="clear win")
+    assert "status=promoted" in out
+    row = conn.execute("SELECT status, review_reason, reviewed_at FROM evolve "
+                       "WHERE id=?", (eid,)).fetchone()
+    assert row["status"] == "promoted"
+    assert row["review_reason"] == "clear win"
+    assert row["reviewed_at"] is not None
+
+
+def test_evolve_decide_dismiss_and_bad_inputs(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "dup suggestion")
+    eid = conn.execute("SELECT id FROM evolve").fetchone()["id"]
+    assert "status=dismissed" in _tool(pkg, "evolve_decide")(
+        evolve_id=eid, decision="dismiss", reason="duplicate of #1")
+    assert _tool(pkg, "evolve_decide")(
+        evolve_id=eid, decision="banana").startswith("ERR bad_decision")
+    assert _tool(pkg, "evolve_decide")(
+        evolve_id=9999, decision="promote").startswith("ERR evolve_not_found")
+
+
+# ── run_evolve_pass dispatch ────────────────────────────────────────────
+
+def test_run_evolve_pass_disabled(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    assert pkg["ed"].run_evolve_pass() == "disabled"
+
+
+def test_run_evolve_pass_no_pending(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    assert pkg["ed"].run_evolve_pass(force=True) == "no_pending"
+
+
+def test_run_evolve_pass_below_min(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch, review_min="2")
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "only one")
+    assert pkg["ed"].run_evolve_pass(force=True) == "below_min n=1"
+
+
+def test_run_evolve_pass_spawns_reviewer(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch, review_min="2")
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "suggestion alpha", rationale="friction A")
+    _add_evolve(conn, "suggestion beta")
+    calls = {}
+    import threadkeeper.tools.spawn as spawn_mod
+    monkeypatch.setattr(spawn_mod, "spawn",
+                        lambda **kw: calls.update(kw) or "ok task=tk_ev pid=1")
+    out = pkg["ed"].run_evolve_pass(force=True)
+    assert out.startswith("spawned n=2")
+    # both suggestions reached the child prompt
+    assert "suggestion alpha" in calls["prompt"]
+    assert "suggestion beta" in calls["prompt"]
+    assert "friction A" in calls["prompt"]
+    assert calls["write_origin"] == "evolve"
+    assert calls["role"] == "evolve_reviewer"
+    # narrow tool surface: triage only, never applies
+    assert "evolve_decide" in calls["extra_allowed_tools"]
+    assert "skill_manage" not in calls["extra_allowed_tools"]
+    assert pkg["ed"]._last_evolve_ts(conn) > 0
+
+
+def test_run_evolve_pass_single_flight(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch, review_min="1")
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "s1")
+    import os
+    conn.execute(
+        "INSERT INTO tasks (id, pid, cwd, prompt, started_at) "
+        "VALUES (?,?,?,?,?)",
+        ("tk_evr", os.getpid(), "/tmp",
+         "You are an EVOLVE REVIEWER triaging the queue.", int(time.time())),
+    )
+    conn.commit()
+
+    def _boom(**kw):
+        raise AssertionError("must not spawn while a reviewer runs")
+    import threadkeeper.tools.spawn as spawn_mod
+    monkeypatch.setattr(spawn_mod, "spawn", _boom)
+    assert "reviewer_running" in pkg["ed"].run_evolve_pass(force=True)
+
+
+# ── brief surfaces promoted ★ first, drops dismissed ───────────────────
+
+def test_brief_evolve_promoted_marked_dismissed_hidden(tmp_path, monkeypatch):
+    pkg = _bootstrap(tmp_path, monkeypatch)
+    conn = pkg["db"].get_db()
+    _add_evolve(conn, "promoted one", status="promoted")
+    _add_evolve(conn, "pending one", status="pending")
+    _add_evolve(conn, "dismissed one", status="dismissed")
+    from threadkeeper.brief import render_brief
+    text = render_brief(conn)
+    # suggestion text is wrapped by q(); assert on the ★ marker + substring
+    assert "★" in text
+    assert "promoted one" in text
+    assert "pending one" in text
+    assert "dismissed one" not in text
+    # the ★ marker attaches to the promoted suggestion, not the pending one
+    assert text.index("★") < text.index("promoted one")
+    # promoted sorts before pending
+    assert text.index("promoted one") < text.index("pending one")

{threadkeeper-0.6.2 → threadkeeper-0.8.0}/tests/test_nudges.py

@@ -37,6 +37,19 @@ def _bootstrap_with_env(tmp_path, monkeypatch,
         "CLAUDE_PROJECTS_DIR": str(tmp_path / "fake_claude_projects"),
         "THREADKEEPER_INGEST_INTERVAL_S": "0",
         "THREADKEEPER_INGEST_CAP": "0",
+        # Zero every background-daemon interval so no daemon thread fires a
+        # pass mid-test and emits a counted `spawn` event that races the
+        # nudge-counter assertions. Inherited from the real shell env
+        # otherwise (a dev box with probe/evolve daemons enabled in
+        # settings.json leaks the interval into pytest).
+        "THREADKEEPER_PROBE_INTERVAL_S": "0",
+        "THREADKEEPER_EVOLVE_REVIEW_INTERVAL_S": "0",
+        "THREADKEEPER_SHADOW_REVIEW_INTERVAL_S": "0",
+        "THREADKEEPER_CURATOR_INTERVAL_S": "0",
+        "THREADKEEPER_EXTRACT_INTERVAL_S": "0",
+        "THREADKEEPER_CANDIDATE_REVIEW_INTERVAL_S": "0",
+        "THREADKEEPER_SPAWN_BUDGET_POLL_S": "0",
+        "THREADKEEPER_MEMORY_GUARD_POLL_S": "0",
         "THREADKEEPER_TASK_LOG_DIR": str(tmp_path / "tasks"),
         "THREADKEEPER_CLIENT": "pytest",
         "THREADKEEPER_MEMORY_NUDGE_INTERVAL": str(memory_interval),

threadkeeper-0.8.0/tests/test_onnx_embeddings.py

@@ -0,0 +1,133 @@
+"""ONNX embedding backend + tk-migrate-embeddings.
+
+Verifies that:
+- the active backend encodes to L2-normalized 384-dim float32 vectors
+- embed_tag stamps the active backend for a real blob, None otherwise
+- freshly inserted notes carry the embed_backend tag
+- the migration recomputes stale (NULL-tagged) rows, tags them, and is
+  idempotent + dry-run-safe
+
+Skips entirely when no embedding backend is installed.
+"""
+from __future__ import annotations
+
+import time
+
+import pytest
+
+pytestmark = pytest.mark.slow  # model warmup on first encode
+
+
+def _tool(pkg, name):
+    return pkg["mcp"]._tool_manager._tools[name].fn
+
+
+@pytest.fixture()
+def sem_pkg(fresh_mp):
+    """Fresh package against a clean tmp DB; skip if semantic search is off."""
+    if not fresh_mp["config"].SEMANTIC_AVAILABLE:
+        pytest.skip("no embedding backend installed in this environment")
+    return fresh_mp
+
+
+def _seed_legacy_notes(conn, n: int):
+    """Insert n notes with a real embedding blob but a NULL backend tag,
+    simulating rows written before the ONNX migration."""
+    from threadkeeper import embeddings as emb
+    for i in range(n):
+        blob = emb._embed(f"legacy seeded note {i} about webhooks and retries")
+        conn.execute(
+            "INSERT INTO notes (content, kind, created_at, embedding, embed_backend) "
+            "VALUES (?,?,?,?,NULL)",
+            (f"legacy seeded note {i}", "insight", int(time.time()), blob),
+        )
+    conn.commit()
+
+
+# ── encode primitives ────────────────────────────────────────────────
+
+def test_encode_is_normalized_384_float32(sem_pkg):
+    import numpy as np
+    from threadkeeper import embeddings as emb
+    arr = emb._encode(["привет мир", "hello world"])
+    assert arr is not None
+    assert arr.shape == (2, 384)
+    assert arr.dtype == np.dtype("float32")
+    assert np.allclose(np.linalg.norm(arr, axis=1), 1.0, atol=1e-3)
+
+
+def test_encode_is_cross_lingual(sem_pkg):
+    """A RU/EN translation pair must score higher than an unrelated phrase."""
+    from threadkeeper import embeddings as emb
+    v = emb._encode(["кошка", "cat", "quarterly financial report"])
+    assert float(v[0] @ v[1]) > float(v[0] @ v[2])
+
+
+def test_embed_tag(sem_pkg):
+    from threadkeeper import embeddings as emb
+    active = sem_pkg["config"].EMBED_BACKEND
+    assert emb.embed_tag(b"\x00\x01") == active
+    assert emb.embed_tag(None) is None
+
+
+# ── write-path tagging ───────────────────────────────────────────────
+
+def test_new_note_carries_backend_tag(sem_pkg):
+    tid = _tool(sem_pkg, "open_thread")(question="backend tag test")
+    _tool(sem_pkg, "note")(thread_id=tid,
+                           content="tagged note about idempotency keys",
+                           kind="insight")
+    conn = sem_pkg["db"].get_db()
+    active = sem_pkg["config"].EMBED_BACKEND
+    row = conn.execute(
+        "SELECT embedding, embed_backend FROM notes "
+        "WHERE thread_id=? ORDER BY id DESC LIMIT 1",
+        (tid,),
+    ).fetchone()
+    assert row["embedding"] is not None
+    assert row["embed_backend"] == active
+
+
+# ── migration ────────────────────────────────────────────────────────
+
+def test_migration_recomputes_tags_and_is_idempotent(sem_pkg):
+    from threadkeeper import migrate_embeddings as mig
+    active = sem_pkg["config"].EMBED_BACKEND
+    conn = sem_pkg["db"].get_db()
+    _seed_legacy_notes(conn, 3)
+
+    assert mig._count_stale(conn, "notes", active) == 3
+
+    rc = mig.run(do_notes=True, do_dialog=False, batch=2,
+                 dry_run=False, log=lambda _m: None)
+    assert rc == 0
+    assert mig._count_stale(conn, "notes", active) == 0
+    tagged = conn.execute(
+        "SELECT COUNT(*) FROM notes WHERE embed_backend=?", (active,)
+    ).fetchone()[0]
+    assert tagged >= 3
+
+    # idempotent: a second pass finds nothing stale and changes nothing.
+    rc2 = mig.run(do_notes=True, do_dialog=False, batch=2,
+                  dry_run=False, log=lambda _m: None)
+    assert rc2 == 0
+    assert mig._count_stale(conn, "notes", active) == 0
+
+
+def test_migration_dry_run_writes_nothing(sem_pkg):
+    from threadkeeper import migrate_embeddings as mig
+    active = sem_pkg["config"].EMBED_BACKEND
+    conn = sem_pkg["db"].get_db()
+    _seed_legacy_notes(conn, 2)
+
+    assert mig._count_stale(conn, "notes", active) == 2
+    mig.run(do_notes=True, do_dialog=False, batch=10,
+            dry_run=True, log=lambda _m: None)
+    # still stale — dry run must not touch the rows
+    assert mig._count_stale(conn, "notes", active) == 2
+
+
+def test_migration_requires_a_scope_flag(sem_pkg):
+    from threadkeeper import migrate_embeddings as mig
+    with pytest.raises(SystemExit):
+        mig.main([])  # argparse error → SystemExit(2)