synapt 0.6.2__tar.gz → 0.7.2__tar.gz

{synapt-0.6.2/src/synapt.egg-info → synapt-0.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: synapt
-Version: 0.6.2
+Version: 0.7.2
 Summary: Persistent conversational memory for AI coding assistants
 Author: Layne Penney
 License-Expression: MIT
@@ -58,9 +58,17 @@ Dynamic: license-file
   remembers what you worked on, decisions you made, and patterns you established.
 </p>
 
+<p align="center">
+  <a href="https://synapt.dev">Website</a> &middot;
+  <a href="https://synapt.dev/blog/">Blog</a> &middot;
+  <a href="https://x.com/synapt_dev">@synapt_dev</a>
+</p>
+
 ---
 
-Works as an [MCP server](https://modelcontextprotocol.io/) for Claude Code and other MCP-compatible tools.
+**#2 on LOCOMO** (76.04%, within 1.5pp of Engram) and **+14.51pp over Mem0** on CodeMemo (90.51% vs 76.0%). Local-first — runs on a laptop, no cloud dependency for memory.
+
+Works as an [MCP server](https://modelcontextprotocol.io/) for Claude Code, Codex CLI, OpenCode, and other MCP-compatible tools.
 
 ## Install
 
@@ -100,20 +108,57 @@ Add to your Claude Code config (`~/.claude/mcp.json`):
 }
 ```
 
-This gives your AI assistant 13 tools for searching past sessions, managing a journal, setting reminders, and building a durable knowledge base.
+This gives your AI assistant tools for searching past sessions, managing a journal, setting reminders, and building a durable knowledge base.
+
+### Codex CLI
+
+Add to `~/.codex/config.toml`:
+
+```toml
+[mcp_servers.synapt]
+command = "synapt"
+args = ["server"]
+```
+
+Synapt automatically discovers and indexes Codex transcripts from `~/.codex/sessions/`.
+
+### OpenCode
+
+Add to `~/.config/opencode/opencode.json`:
+
+```json
+{
+  "mcp": {
+    "synapt": {
+      "type": "local",
+      "command": ["synapt", "server"],
+      "enabled": true
+    }
+  }
+}
+```
 
 ## Features
 
-- **Hybrid search** — BM25 full-text search fused with semantic embeddings via [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf). Surfaces results that keyword search alone would miss.
+- **FTS5 + embedding hybrid search** — BM25 full-text search fused with semantic embeddings via [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf), plus cross-encoder reranking. Surfaces results that keyword search alone would miss.
+- **Sub-chunk splitting** — Splits transcripts at tool-use boundaries so each chunk captures a coherent action (code edit, test run, error trace) rather than arbitrary fixed-length windows.
+- **Cross-session link expansion** — When retrieving a chunk, automatically surfaces related chunks from other sessions, enabling multi-hop reasoning across your project history.
+- **Content-aware adaptive filtering** — Classifies conversations as code/personal/mixed and adjusts consolidation filters and retrieval parameters per content type.
 - **Query intent routing** — Classifies queries as factual, temporal, debug, decision, aggregation, exploratory, or procedural and adjusts search parameters (recency decay, knowledge boost, embedding weight) automatically.
+- **Enrichment + consolidation + knowledge graph** — Optional LLM-powered session summaries, durable knowledge extraction, and a knowledge graph that connects facts across sessions.
 - **Knowledge embeddings** — Durable knowledge nodes get 384-dim embeddings for semantic retrieval, built at index time.
 - **Topic clustering** — Jaccard token-overlap clustering groups related chunks across sessions.
 - **Session journal** — Rich entries with focus, decisions, done items, and next steps.
 - **Reminders** — Cross-session sticky reminders that surface at session start.
 - **Timeline** — Chronological work arcs showing project narrative.
 - **Working memory** — Frequency-boosted search results for active topics.
-- **LLM enrichment** — Optional LLM-powered summaries and cluster upgrades.
-- **Knowledge consolidation** — Extracts durable knowledge from session journals.
+- **Local-first** — Runs entirely on your laptop. Indexing, embedding, and retrieval are all local — no cloud dependency for memory.
+- **MCP server** — 16 tools for Claude Code integration: search, journal, channels, reminders, knowledge, and more.
+- **Agent channels** — Cross-session communication via append-only channels. Agents can post messages, send directives, and coordinate work across worktrees.
+- **Directive notifications** — Targeted directives are automatically surfaced in MCP tool responses. Broadcast directives (`to="*"`) reach all agents.
+- **Contradiction flagging** — Flag conflicting information from free text or search results. Auto-matches existing knowledge nodes via FTS, creates new nodes on resolution.
+- **Codex CLI support** — Indexes Codex transcripts from `~/.codex/sessions/` automatically. Cross-editor memory between Claude Code and Codex.
+- **Agent-aware consolidation** — Journal entries capture agent identity (griptree, agent_id). Consolidation detects concurrent sessions and annotates them for the LLM.
 - **Plugin system** — Extend with additional tools via entry-point discovery.
 
 ## MCP tools
@@ -132,7 +177,10 @@ This gives your AI assistant 13 tools for searching past sessions, managing a jo
 | `recall_remind` | Set cross-session reminders |
 | `recall_enrich` | LLM-powered chunk summarization |
 | `recall_consolidate` | Extract knowledge from journals |
-| `recall_contradict` | Flag contradictions in knowledge |
+| `recall_contradict` | Flag contradictions in knowledge (supports free-text claims) |
+| `recall_channel` | Cross-session agent communication (post, read, directives, who) |
+| `recall_quick` | Fast knowledge check (no transcript chunks) |
+| `recall_reload` | Restart MCP server to pick up code changes |
 
 ## CLI reference
 
@@ -148,23 +196,39 @@ synapt server                    # Start MCP server
 
 ## Benchmarks
 
-Evaluated on [LOCOMO](https://snap-research.github.io/locomo/) (Long Conversational Memory) — 10 conversations, 1540 QA pairs — following [Mem0's methodology](https://arxiv.org/abs/2504.19413) (J-score via LLM-as-Judge).
+### LOCOMO — Conversational Memory
+
+Evaluated on [LOCOMO](https://snap-research.github.io/locomo/) (Long Conversational Memory) — 10 conversations, 1540 QA pairs — following [Mem0's methodology](https://arxiv.org/abs/2504.19413) (J-score via LLM-as-Judge). Competitor data from the [Mem0 paper](https://arxiv.org/abs/2504.19413) and [Memobase benchmark](https://github.com/memodb-io/memobase/blob/main/docs/experiments/locomo-benchmark/README.md).
 
-| System | Multi-Hop | Temporal | Single-Hop | Open-Domain | **Overall** | Infra |
-|--------|-----------|----------|------------|-------------|-------------|-------|
-| **synapt** | **70.21** | **61.68** | **62.50** | **80.14** | **73.38** | **local 3B model** |
-| Full-Context | — | — | — | — | 72.90 | upper bound |
-| Mem0+Graph | 47.19 | 58.13 | 65.71 | 75.71 | 68.44 | cloud GPT-4 |
-| Mem0 | 51.15 | 55.51 | 67.13 | 72.93 | 66.88 | cloud GPT-4 |
-| Zep | 41.35 | 49.31 | 61.70 | 76.60 | 65.99 | cloud service |
-| LangMem | 47.92 | 23.43 | 62.23 | 71.12 | 58.10 | cloud |
-| OpenAI Memory | 42.92 | 21.71 | 63.79 | 62.29 | 52.90 | cloud |
+All systems use gpt-4o-mini as shared backbone (generation + judge) for fair comparison. Competitor data from the [Engram paper](https://arxiv.org/abs/2511.12960) (3 runs, stddev reported) and [Mem0 paper](https://arxiv.org/abs/2504.19413).
 
-Synapt scores **73.38% overall** — beating the Full-Context upper bound (72.90%), Mem0+Graph (68.44%), Mem0 (66.88%), Zep (65.99%), and all other tested systems — using Ministral 3B locally for enrichment. No cloud API calls.
+| System | **Overall** | Multi-Hop | Temporal | Infra |
+|--------|-------------|-----------|----------|-------|
+| Engram | 77.55 ± 0.13 | — | — | cloud (BM25+ColBERT+KG) |
+| **synapt v0.6.1 (8B)** | **76.04** | **70.92** | 66.36 | Ministral 8B cloud enrich |
+| Memobase | 75.78 | 46.88 | **85.05** | cloud |
+| **synapt v0.6.1 (3B)** | **73.38** | 70.21 | 61.68 | **local 3B on M2 Air** |
+| memOS | 72.99 ± 0.14 | — | — | cloud |
+| Full-Context | 72.60 ± 0.07 | — | — | upper bound |
+| Mem0 | 64.73 ± 0.17 | 51.15 | 55.51 | cloud GPT-4 |
+| Zep | 42.29 ± 0.18 | — | — | cloud |
 
-> **What is Full-Context?** The entire conversation history is passed directly to GPT-4 as context — no retrieval, no memory extraction. It represents the theoretical upper bound: the LLM has access to every fact. Synapt beats it because focused retrieval surfaces only what's relevant, reducing noise for the answer model.
+Synapt is **#2 on LOCOMO** at 76.04% — 1.51pp behind Engram (within their stddev of ±0.13) and ahead of Memobase (75.78%), the Full-Context upper bound (72.60%), and all other systems. The 3B local configuration (73.38%) beats the Full-Context upper bound using only a Ministral 3B model running on an M2 MacBook Air.
 
-**Best-in-class**: Multi-hop (70.21%) and open-domain (80.14%) — highest of any system tested, including those using GPT-4 for memory extraction.
+**Best-in-class multi-hop**: 70.92% — highest of any system tested, including those using GPT-4 for memory extraction. Engram is cloud-only; synapt runs entirely on a laptop.
+
+> **What is Full-Context?** The entire conversation history is passed directly to the LLM as context — no retrieval, no memory extraction. It represents the theoretical upper bound: the LLM has access to every fact. Synapt beats it because focused retrieval surfaces only what's relevant, reducing noise for the answer model.
+
+### CodeMemo — Coding Memory
+
+First benchmark specifically testing coding session memory — 158 questions across 3 projects, 6 categories. Same gpt-4o-mini judge and answer model for both systems.
+
+| System | Factual | Debug | Architecture | Temporal | Convention | Cross-Session | **Overall** |
+|--------|---------|-------|-------------|----------|------------|---------------|-------------|
+| **synapt v0.6.2** | **97.14** | **100.0** | 92.86 | **90.91** | **80.0** | **86.36** | **90.51** |
+| Mem0 (OSS) | 72.73 | 77.78 | **100.0** | 87.50 | 42.86 | 71.43 | 76.0 |
+
+Synapt leads by **+14.51pp overall**. The biggest gaps are in convention (+37pp), factual (+24pp), and debug (+22pp) — categories that depend on raw evidence preservation. Synapt runs entirely locally; Mem0 requires OpenAI API calls for memory extraction, embedding, and search.
 
 ## How search works
 
@@ -235,6 +299,13 @@ pip install -e ".[test]"
 pytest tests/ -v
 ```
 
+## Links
+
+- [synapt.dev](https://synapt.dev) — Website
+- [synapt.dev/blog](https://synapt.dev/blog/) — Blog
+- [@synapt_dev](https://x.com/synapt_dev) — X / Twitter
+- [PyPI](https://pypi.org/project/synapt/) — `pip install synapt`
+
 ## License
 
 MIT

{synapt-0.6.2 → synapt-0.7.2}/README.md

@@ -14,9 +14,17 @@
   remembers what you worked on, decisions you made, and patterns you established.
 </p>
 
+<p align="center">
+  <a href="https://synapt.dev">Website</a> &middot;
+  <a href="https://synapt.dev/blog/">Blog</a> &middot;
+  <a href="https://x.com/synapt_dev">@synapt_dev</a>
+</p>
+
 ---
 
-Works as an [MCP server](https://modelcontextprotocol.io/) for Claude Code and other MCP-compatible tools.
+**#2 on LOCOMO** (76.04%, within 1.5pp of Engram) and **+14.51pp over Mem0** on CodeMemo (90.51% vs 76.0%). Local-first — runs on a laptop, no cloud dependency for memory.
+
+Works as an [MCP server](https://modelcontextprotocol.io/) for Claude Code, Codex CLI, OpenCode, and other MCP-compatible tools.
 
 ## Install
 
@@ -56,20 +64,57 @@ Add to your Claude Code config (`~/.claude/mcp.json`):
 }
 ```
 
-This gives your AI assistant 13 tools for searching past sessions, managing a journal, setting reminders, and building a durable knowledge base.
+This gives your AI assistant tools for searching past sessions, managing a journal, setting reminders, and building a durable knowledge base.
+
+### Codex CLI
+
+Add to `~/.codex/config.toml`:
+
+```toml
+[mcp_servers.synapt]
+command = "synapt"
+args = ["server"]
+```
+
+Synapt automatically discovers and indexes Codex transcripts from `~/.codex/sessions/`.
+
+### OpenCode
+
+Add to `~/.config/opencode/opencode.json`:
+
+```json
+{
+  "mcp": {
+    "synapt": {
+      "type": "local",
+      "command": ["synapt", "server"],
+      "enabled": true
+    }
+  }
+}
+```
 
 ## Features
 
-- **Hybrid search** — BM25 full-text search fused with semantic embeddings via [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf). Surfaces results that keyword search alone would miss.
+- **FTS5 + embedding hybrid search** — BM25 full-text search fused with semantic embeddings via [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf), plus cross-encoder reranking. Surfaces results that keyword search alone would miss.
+- **Sub-chunk splitting** — Splits transcripts at tool-use boundaries so each chunk captures a coherent action (code edit, test run, error trace) rather than arbitrary fixed-length windows.
+- **Cross-session link expansion** — When retrieving a chunk, automatically surfaces related chunks from other sessions, enabling multi-hop reasoning across your project history.
+- **Content-aware adaptive filtering** — Classifies conversations as code/personal/mixed and adjusts consolidation filters and retrieval parameters per content type.
 - **Query intent routing** — Classifies queries as factual, temporal, debug, decision, aggregation, exploratory, or procedural and adjusts search parameters (recency decay, knowledge boost, embedding weight) automatically.
+- **Enrichment + consolidation + knowledge graph** — Optional LLM-powered session summaries, durable knowledge extraction, and a knowledge graph that connects facts across sessions.
 - **Knowledge embeddings** — Durable knowledge nodes get 384-dim embeddings for semantic retrieval, built at index time.
 - **Topic clustering** — Jaccard token-overlap clustering groups related chunks across sessions.
 - **Session journal** — Rich entries with focus, decisions, done items, and next steps.
 - **Reminders** — Cross-session sticky reminders that surface at session start.
 - **Timeline** — Chronological work arcs showing project narrative.
 - **Working memory** — Frequency-boosted search results for active topics.
-- **LLM enrichment** — Optional LLM-powered summaries and cluster upgrades.
-- **Knowledge consolidation** — Extracts durable knowledge from session journals.
+- **Local-first** — Runs entirely on your laptop. Indexing, embedding, and retrieval are all local — no cloud dependency for memory.
+- **MCP server** — 16 tools for Claude Code integration: search, journal, channels, reminders, knowledge, and more.
+- **Agent channels** — Cross-session communication via append-only channels. Agents can post messages, send directives, and coordinate work across worktrees.
+- **Directive notifications** — Targeted directives are automatically surfaced in MCP tool responses. Broadcast directives (`to="*"`) reach all agents.
+- **Contradiction flagging** — Flag conflicting information from free text or search results. Auto-matches existing knowledge nodes via FTS, creates new nodes on resolution.
+- **Codex CLI support** — Indexes Codex transcripts from `~/.codex/sessions/` automatically. Cross-editor memory between Claude Code and Codex.
+- **Agent-aware consolidation** — Journal entries capture agent identity (griptree, agent_id). Consolidation detects concurrent sessions and annotates them for the LLM.
 - **Plugin system** — Extend with additional tools via entry-point discovery.
 
 ## MCP tools
@@ -88,7 +133,10 @@ This gives your AI assistant 13 tools for searching past sessions, managing a jo
 | `recall_remind` | Set cross-session reminders |
 | `recall_enrich` | LLM-powered chunk summarization |
 | `recall_consolidate` | Extract knowledge from journals |
-| `recall_contradict` | Flag contradictions in knowledge |
+| `recall_contradict` | Flag contradictions in knowledge (supports free-text claims) |
+| `recall_channel` | Cross-session agent communication (post, read, directives, who) |
+| `recall_quick` | Fast knowledge check (no transcript chunks) |
+| `recall_reload` | Restart MCP server to pick up code changes |
 
 ## CLI reference
 
@@ -104,23 +152,39 @@ synapt server                    # Start MCP server
 
 ## Benchmarks
 
-Evaluated on [LOCOMO](https://snap-research.github.io/locomo/) (Long Conversational Memory) — 10 conversations, 1540 QA pairs — following [Mem0's methodology](https://arxiv.org/abs/2504.19413) (J-score via LLM-as-Judge).
+### LOCOMO — Conversational Memory
+
+Evaluated on [LOCOMO](https://snap-research.github.io/locomo/) (Long Conversational Memory) — 10 conversations, 1540 QA pairs — following [Mem0's methodology](https://arxiv.org/abs/2504.19413) (J-score via LLM-as-Judge). Competitor data from the [Mem0 paper](https://arxiv.org/abs/2504.19413) and [Memobase benchmark](https://github.com/memodb-io/memobase/blob/main/docs/experiments/locomo-benchmark/README.md).
 
-| System | Multi-Hop | Temporal | Single-Hop | Open-Domain | **Overall** | Infra |
-|--------|-----------|----------|------------|-------------|-------------|-------|
-| **synapt** | **70.21** | **61.68** | **62.50** | **80.14** | **73.38** | **local 3B model** |
-| Full-Context | — | — | — | — | 72.90 | upper bound |
-| Mem0+Graph | 47.19 | 58.13 | 65.71 | 75.71 | 68.44 | cloud GPT-4 |
-| Mem0 | 51.15 | 55.51 | 67.13 | 72.93 | 66.88 | cloud GPT-4 |
-| Zep | 41.35 | 49.31 | 61.70 | 76.60 | 65.99 | cloud service |
-| LangMem | 47.92 | 23.43 | 62.23 | 71.12 | 58.10 | cloud |
-| OpenAI Memory | 42.92 | 21.71 | 63.79 | 62.29 | 52.90 | cloud |
+All systems use gpt-4o-mini as shared backbone (generation + judge) for fair comparison. Competitor data from the [Engram paper](https://arxiv.org/abs/2511.12960) (3 runs, stddev reported) and [Mem0 paper](https://arxiv.org/abs/2504.19413).
 
-Synapt scores **73.38% overall** — beating the Full-Context upper bound (72.90%), Mem0+Graph (68.44%), Mem0 (66.88%), Zep (65.99%), and all other tested systems — using Ministral 3B locally for enrichment. No cloud API calls.
+| System | **Overall** | Multi-Hop | Temporal | Infra |
+|--------|-------------|-----------|----------|-------|
+| Engram | 77.55 ± 0.13 | — | — | cloud (BM25+ColBERT+KG) |
+| **synapt v0.6.1 (8B)** | **76.04** | **70.92** | 66.36 | Ministral 8B cloud enrich |
+| Memobase | 75.78 | 46.88 | **85.05** | cloud |
+| **synapt v0.6.1 (3B)** | **73.38** | 70.21 | 61.68 | **local 3B on M2 Air** |
+| memOS | 72.99 ± 0.14 | — | — | cloud |
+| Full-Context | 72.60 ± 0.07 | — | — | upper bound |
+| Mem0 | 64.73 ± 0.17 | 51.15 | 55.51 | cloud GPT-4 |
+| Zep | 42.29 ± 0.18 | — | — | cloud |
 
-> **What is Full-Context?** The entire conversation history is passed directly to GPT-4 as context — no retrieval, no memory extraction. It represents the theoretical upper bound: the LLM has access to every fact. Synapt beats it because focused retrieval surfaces only what's relevant, reducing noise for the answer model.
+Synapt is **#2 on LOCOMO** at 76.04% — 1.51pp behind Engram (within their stddev of ±0.13) and ahead of Memobase (75.78%), the Full-Context upper bound (72.60%), and all other systems. The 3B local configuration (73.38%) beats the Full-Context upper bound using only a Ministral 3B model running on an M2 MacBook Air.
 
-**Best-in-class**: Multi-hop (70.21%) and open-domain (80.14%) — highest of any system tested, including those using GPT-4 for memory extraction.
+**Best-in-class multi-hop**: 70.92% — highest of any system tested, including those using GPT-4 for memory extraction. Engram is cloud-only; synapt runs entirely on a laptop.
+
+> **What is Full-Context?** The entire conversation history is passed directly to the LLM as context — no retrieval, no memory extraction. It represents the theoretical upper bound: the LLM has access to every fact. Synapt beats it because focused retrieval surfaces only what's relevant, reducing noise for the answer model.
+
+### CodeMemo — Coding Memory
+
+First benchmark specifically testing coding session memory — 158 questions across 3 projects, 6 categories. Same gpt-4o-mini judge and answer model for both systems.
+
+| System | Factual | Debug | Architecture | Temporal | Convention | Cross-Session | **Overall** |
+|--------|---------|-------|-------------|----------|------------|---------------|-------------|
+| **synapt v0.6.2** | **97.14** | **100.0** | 92.86 | **90.91** | **80.0** | **86.36** | **90.51** |
+| Mem0 (OSS) | 72.73 | 77.78 | **100.0** | 87.50 | 42.86 | 71.43 | 76.0 |
+
+Synapt leads by **+14.51pp overall**. The biggest gaps are in convention (+37pp), factual (+24pp), and debug (+22pp) — categories that depend on raw evidence preservation. Synapt runs entirely locally; Mem0 requires OpenAI API calls for memory extraction, embedding, and search.
 
 ## How search works
 
@@ -191,6 +255,13 @@ pip install -e ".[test]"
 pytest tests/ -v
 ```
 
+## Links
+
+- [synapt.dev](https://synapt.dev) — Website
+- [synapt.dev/blog](https://synapt.dev/blog/) — Blog
+- [@synapt_dev](https://x.com/synapt_dev) — X / Twitter
+- [PyPI](https://pypi.org/project/synapt/) — `pip install synapt`
+
 ## License
 
 MIT

{synapt-0.6.2 → synapt-0.7.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "synapt"
-version = "0.6.2"
+version = "0.7.2"
 description = "Persistent conversational memory for AI coding assistants"
 readme = "README.md"
 license = "MIT"

{synapt-0.6.2 → synapt-0.7.2}/src/synapt/__init__.py

@@ -1,3 +1,3 @@
 """Synapt: persistent conversational memory for AI coding assistants."""
 
-__version__ = "0.6.0"
+__version__ = "0.7.2"

synapt-0.7.2/src/synapt/_models/vllm_client.py

@@ -0,0 +1,61 @@
+"""vLLM inference client for decoder-only models.
+
+Fast GPU inference for models like Ministral-8B using vLLM's
+offline LLM engine. No server process needed.
+
+Requires: pip install vllm
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import List
+
+from .base import Message, ModelClient
+
+logger = logging.getLogger(__name__)
+
+_ENGINE_CACHE: dict[str, object] = {}
+
+
+class VLLMClient(ModelClient):
+    """Decoder-only model client via vLLM offline inference."""
+
+    def __init__(self, max_tokens: int = 800, model_name: str | None = None):
+        self.max_tokens = max_tokens
+        self._model_name = model_name
+
+    def _get_engine(self, model: str):
+        if model in _ENGINE_CACHE:
+            return _ENGINE_CACHE[model]
+
+        from vllm import LLM
+
+        engine = LLM(model=model, max_model_len=4096, trust_remote_code=True)
+        _ENGINE_CACHE[model] = engine
+        return engine
+
+    def chat(
+        self,
+        model: str,
+        messages: List[Message],
+        temperature: float = 0.2,
+        **kwargs,
+    ) -> str:
+        from vllm import SamplingParams
+
+        max_tokens = kwargs.get("max_tokens") or self.max_tokens
+        engine = self._get_engine(self._model_name or model)
+
+        # Use vLLM's chat API with the tokenizer's chat template
+        chat_messages = [{"role": m.role, "content": m.content} for m in messages]
+
+        params = SamplingParams(
+            temperature=temperature,
+            max_tokens=max_tokens,
+        )
+
+        outputs = engine.chat(chat_messages, params)
+        if outputs and outputs[0].outputs:
+            return outputs[0].outputs[0].text.strip()
+        return ""

{synapt-0.6.2 → synapt-0.7.2}/src/synapt/recall/_model_router.py

@@ -144,6 +144,8 @@ def get_client(
         return _get_transformers_client(max_tokens)
     elif override == "ollama":
         return _get_ollama_client(max_tokens)
+    elif override == "vllm":
+        return _get_vllm_client(max_tokens)
     elif override in _extra_backends:
         key = (override, max_tokens)
         cached = _client_cache.get(key, _NOT_FOUND)
@@ -255,7 +257,8 @@ def _get_ollama_client(max_tokens: int) -> object | None:
 
     try:
         from synapt._models.ollama_client import OllamaClient
-        client = OllamaClient(max_tokens=max_tokens)
+        client = OllamaClient()
+        client.max_tokens = max_tokens
         _client_cache[key] = client
         logger.debug("OllamaClient available for decoder-only inference")
         return client
@@ -264,6 +267,23 @@ def _get_ollama_client(max_tokens: int) -> object | None:
         return None
 
 
+def _get_vllm_client(max_tokens: int) -> object | None:
+    """Try to create a VLLMClient. Returns None if unavailable."""
+    key = ("vllm", max_tokens)
+    if key in _client_cache:
+        return _client_cache[key]
+
+    try:
+        from synapt._models.vllm_client import VLLMClient
+        client = VLLMClient(max_tokens=max_tokens)
+        _client_cache[key] = client
+        logger.debug("VLLMClient available for decoder-only inference")
+        return client
+    except ImportError:
+        logger.debug("VLLMClient not available (vllm not installed)")
+        return None
+
+
 def _get_llamacpp_client(max_tokens: int) -> object | None:
     """Try to create a LlamaCppClient. Returns None if unavailable."""
     key = ("llamacpp", max_tokens)