undertow-llm 0.2.1__tar.gz

undertow_llm-0.2.1/PKG-INFO

@@ -0,0 +1,228 @@
+Metadata-Version: 2.4
+Name: undertow-llm
+Version: 0.2.1
+Summary: Provider-agnostic LLM observability & reliability SDK — semantic caching, retries, rate limiting, cost tracking, and a local dashboard via a single @track() decorator.
+Author: Amogh Arora
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/ShriAmogh/undertow-llm
+Project-URL: Repository, https://github.com/ShriAmogh/undertow-llm
+Keywords: llm,observability,caching,openai,gemini,anthropic,sdk,ai-infra
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: sentence-transformers>=2.7.0
+Requires-Dist: numpy>=1.26.0
+Requires-Dist: fastapi>=0.111.0
+Requires-Dist: uvicorn[standard]>=0.29.0
+Requires-Dist: jinja2>=3.1.4
+Requires-Dist: click>=8.1.7
+Requires-Dist: python-dotenv>=1.0.1
+Requires-Dist: httpx>=0.27.0
+Provides-Extra: gemini
+Requires-Dist: google-genai>=1.0.0; extra == "gemini"
+Provides-Extra: ollama
+Requires-Dist: ollama>=0.2.0; extra == "ollama"
+Provides-Extra: all-providers
+Requires-Dist: google-genai>=1.0.0; extra == "all-providers"
+Requires-Dist: ollama>=0.2.0; extra == "all-providers"
+Provides-Extra: postgres
+Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"
+Requires-Dist: pgvector>=0.2.0; extra == "postgres"
+Provides-Extra: redis
+Requires-Dist: redis>=5.0.0; extra == "redis"
+Provides-Extra: prod
+Requires-Dist: psycopg2-binary>=2.9.0; extra == "prod"
+Requires-Dist: pgvector>=0.2.0; extra == "prod"
+Requires-Dist: redis>=5.0.0; extra == "prod"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.2.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
+Requires-Dist: locust>=2.28.0; extra == "dev"
+
+# undertow_llm
+
+**Wrap any LLM call, get caching, retries, rate limiting, and a real-time dashboard — with zero code changes to your model.**
+
+[![PyPI version](https://img.shields.io/pypi/v/undertow_llm.svg)](https://pypi.org/project/undertow_llm/)
+[![Python versions](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue.svg)](https://pypi.org/project/undertow_llm/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Status](https://img.shields.io/badge/status-active-brightgreen.svg)](https://pypi.org/project/undertow_llm/)
+
+---
+
+## Problem Statement
+
+Without an observability and resilience layer, production LLM applications suffer from soaring API costs due to redundant prompt calls, unexpected provider outages with zero fallback protection, and complete lack of visibility into latency and errors. `undertow_llm` solves this by wrapping your existing Python LLM functions in a single decorator—providing semantic caching, automated retries, rate limiting, and a live dashboard without modifying your model logic.
+
+---
+
+## Install
+
+```bash
+pip install undertow-llm
+```
+
+Or install with provider & production backend extras:
+
+```bash
+pip install "undertow-llm[gemini]"    # Google Gemini support
+pip install "undertow-llm[ollama]"    # Ollama local model support
+pip install "undertow-llm[postgres]"  # PostgreSQL + pgvector backend
+pip install "undertow-llm[redis]"     # Redis rate-limiting backend
+pip install "undertow-llm[prod]"      # Production stack (Postgres + Redis)
+```
+
+---
+
+## Quickstart
+
+**Before** (bare LLM call — no caching, no fallback, no observability):
+```python
+def generate_response(prompt: str) -> str:
+    return client.models.generate_content("gemini-2.5-flash", prompt).text
+```
+
+**After** (wrapped with `@track()` — fully resilient & tracked):
+```python
+from undertow_llm import track
+
+@track(cache=True, retries=3, rate_limit_rate=2.0)
+def generate_response(prompt: str) -> str:
+    return client.models.generate_content("gemini-2.5-flash", prompt).text
+```
+
+Copy-paste into your application and run. Zero edits required except setting your provider API key.
+
+---
+
+## Real-Time Dashboard
+
+Start the live observability dashboard in one command:
+
+```bash
+undertow-llm serve
+```
+
+Open `http://localhost:8080` to inspect real-time metrics, cache hit ratios, latency charts, cost estimates, distributed traces, and request logs.
+
+---
+
+## How It Works
+
+1. The `@track()` decorator wraps your function, intercepting incoming prompts before execution.
+2. It performs a vector similarity search (using `SentenceTransformers`) to serve semantic cache hits instantly and applies token-bucket rate limits.
+3. Upon function completion, it records latency, token usage, estimated cost, and execution traces to storage.
+4. It is provider-agnostic because it wraps your Python function call directly and never touches your underlying model SDK.
+
+For a detailed architectural breakdown of the 8-stage execution pipeline and backend dispatcher, see [ARCHITECTURE.md](ARCHITECTURE.md).
+
+---
+
+## Configuration
+
+**Local dev needs zero config** — defaults out-of-the-box to local SQLite (`undertow_llm.db`).
+
+For production environments, configure via environment variables or `configure()`:
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `UNDERTOW_LLM_POSTGRES_URL` | `None` (SQLite) | PostgreSQL URL with `pgvector` for production vector storage & metrics |
+| `UNDERTOW_LLM_REDIS_URL` | `None` (Local) | Redis URL for distributed rate limiting & token buckets |
+| `UNDERTOW_LLM_DB_PATH` | `"undertow-llm.db"` | File path for local SQLite database fallback |
+| `UNDERTOW_LLM_DASHBOARD_PORT` | `8080` | HTTP port for `undertow-llm serve` dashboard |
+
+---
+
+## `@track()` Parameter Reference
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `cache` | `bool` | `True` | Enable semantic caching for responses |
+| `similarity_threshold` | `float` | `0.92` | Cosine similarity threshold for cache hits (0.0 to 1.0) |
+| `cache_ttl` | `int \| None` | `None` | Optional time-to-live in seconds for cached entries |
+| `retries` | `int` | `3` | Max retry attempts for transient LLM failures |
+| `base_delay` | `float` | `1.0` | Initial exponential backoff delay (seconds) |
+| `max_delay` | `float` | `60.0` | Cap on exponential backoff delay (seconds) |
+| `jitter` | `bool` | `True` | Add randomized jitter to retry delays to prevent thundering herds |
+| `retry_on` | `tuple` | `(Exception,)` | Exception types that trigger automatic retries |
+| `fallback` | `list` | `[]` | Ordered list of fallback functions to call if primary function fails |
+| `rate_limit_rate` | `float` | `2.0` | Token-bucket refill rate (tokens/second) |
+| `rate_limit_max_tokens` | `float` | `10.0` | Token-bucket maximum capacity (burst limit) |
+| `max_concurrency` | `int \| None` | `None` | Max concurrent executions allowed across processes |
+| `cost_per_call` | `float \| None` | `None` | Explicit cost override per call ($/call) |
+| `policy` | `callable \| None` | `None` | Custom safety hook returning `"allow"`, `"block"`, or `"flag"` |
+| `canary` | `dict \| None` | `None` | Canary routing config `{"fn": alternate_fn, "weight": 0.10}` |
+
+---
+
+## Supported Providers
+
+`undertow_llm` works with **any provider** — OpenAI, Anthropic, Google Gemini, Ollama, HuggingFace, or custom local models — since it wraps your existing Python function call rather than a specific provider SDK.
+
+---
+
+## Examples
+
+See [`demo/example_usage.py`](demo/example_usage.py) for complete runnable examples.
+
+### 1. Multi-Provider Fallback Chain
+```python
+from undertow_llm import track
+
+def fallback_anthropic(prompt: str) -> str:
+    return anthropic_client.messages.create(model="claude-3-5-sonnet", messages=[{"role": "user", "content": prompt}]).content[0].text
+
+@track(retries=2, fallback=[fallback_anthropic])
+def primary_openai(prompt: str) -> str:
+    return openai_client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": prompt}]).choices[0].message.content
+```
+
+### 2. Streaming LLM Response
+```python
+@track(cache=False)
+def stream_gemini(prompt: str):
+    response = gemini_client.models.generate_content_stream("gemini-2.5-flash", prompt)
+    for chunk in response:
+        yield chunk.text
+```
+
+### 3. Local Model (Ollama) with Custom Usage Extractor
+```python
+@track(
+    cost_per_call=0.0,  # Local model — zero API cost
+    usage_extractor=lambda res: {"prompt_tokens": len(res.get("prompt", "")), "completion_tokens": len(res.get("response", ""))}
+)
+def ask_ollama(prompt: str) -> dict:
+    return ollama.generate(model="llama3", prompt=prompt)
+```
+
+---
+
+## Known Limitations & Roadmap
+
+### Limitations
+- **SQLite Concurrency**: Local SQLite storage (`undertow_llm.db`) is zero-config and ideal for development and single-instance apps, but is not designed for multi-node production scale. For high concurrency, set `UNDERTOW_LLM_POSTGRES_URL` and `UNDERTOW_LLM_REDIS_URL`.
+
+### Near-Term Roadmap
+- [ ] OpenTelemetry trace exporter integration
+- [ ] Multi-tenant workspace tagging & dashboard authentication
+- [ ] Automated PII redaction and sensitive prompt masking filters
+
+---
+
+## Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for developer setup instructions.
+
+---
+
+## License
+
+MIT — see [LICENSE](LICENSE)

undertow_llm-0.2.1/README.md

@@ -0,0 +1,180 @@
+# undertow_llm
+
+**Wrap any LLM call, get caching, retries, rate limiting, and a real-time dashboard — with zero code changes to your model.**
+
+[![PyPI version](https://img.shields.io/pypi/v/undertow_llm.svg)](https://pypi.org/project/undertow_llm/)
+[![Python versions](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue.svg)](https://pypi.org/project/undertow_llm/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Status](https://img.shields.io/badge/status-active-brightgreen.svg)](https://pypi.org/project/undertow_llm/)
+
+---
+
+## Problem Statement
+
+Without an observability and resilience layer, production LLM applications suffer from soaring API costs due to redundant prompt calls, unexpected provider outages with zero fallback protection, and complete lack of visibility into latency and errors. `undertow_llm` solves this by wrapping your existing Python LLM functions in a single decorator—providing semantic caching, automated retries, rate limiting, and a live dashboard without modifying your model logic.
+
+---
+
+## Install
+
+```bash
+pip install undertow-llm
+```
+
+Or install with provider & production backend extras:
+
+```bash
+pip install "undertow-llm[gemini]"    # Google Gemini support
+pip install "undertow-llm[ollama]"    # Ollama local model support
+pip install "undertow-llm[postgres]"  # PostgreSQL + pgvector backend
+pip install "undertow-llm[redis]"     # Redis rate-limiting backend
+pip install "undertow-llm[prod]"      # Production stack (Postgres + Redis)
+```
+
+---
+
+## Quickstart
+
+**Before** (bare LLM call — no caching, no fallback, no observability):
+```python
+def generate_response(prompt: str) -> str:
+    return client.models.generate_content("gemini-2.5-flash", prompt).text
+```
+
+**After** (wrapped with `@track()` — fully resilient & tracked):
+```python
+from undertow_llm import track
+
+@track(cache=True, retries=3, rate_limit_rate=2.0)
+def generate_response(prompt: str) -> str:
+    return client.models.generate_content("gemini-2.5-flash", prompt).text
+```
+
+Copy-paste into your application and run. Zero edits required except setting your provider API key.
+
+---
+
+## Real-Time Dashboard
+
+Start the live observability dashboard in one command:
+
+```bash
+undertow-llm serve
+```
+
+Open `http://localhost:8080` to inspect real-time metrics, cache hit ratios, latency charts, cost estimates, distributed traces, and request logs.
+
+---
+
+## How It Works
+
+1. The `@track()` decorator wraps your function, intercepting incoming prompts before execution.
+2. It performs a vector similarity search (using `SentenceTransformers`) to serve semantic cache hits instantly and applies token-bucket rate limits.
+3. Upon function completion, it records latency, token usage, estimated cost, and execution traces to storage.
+4. It is provider-agnostic because it wraps your Python function call directly and never touches your underlying model SDK.
+
+For a detailed architectural breakdown of the 8-stage execution pipeline and backend dispatcher, see [ARCHITECTURE.md](ARCHITECTURE.md).
+
+---
+
+## Configuration
+
+**Local dev needs zero config** — defaults out-of-the-box to local SQLite (`undertow_llm.db`).
+
+For production environments, configure via environment variables or `configure()`:
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `UNDERTOW_LLM_POSTGRES_URL` | `None` (SQLite) | PostgreSQL URL with `pgvector` for production vector storage & metrics |
+| `UNDERTOW_LLM_REDIS_URL` | `None` (Local) | Redis URL for distributed rate limiting & token buckets |
+| `UNDERTOW_LLM_DB_PATH` | `"undertow-llm.db"` | File path for local SQLite database fallback |
+| `UNDERTOW_LLM_DASHBOARD_PORT` | `8080` | HTTP port for `undertow-llm serve` dashboard |
+
+---
+
+## `@track()` Parameter Reference
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `cache` | `bool` | `True` | Enable semantic caching for responses |
+| `similarity_threshold` | `float` | `0.92` | Cosine similarity threshold for cache hits (0.0 to 1.0) |
+| `cache_ttl` | `int \| None` | `None` | Optional time-to-live in seconds for cached entries |
+| `retries` | `int` | `3` | Max retry attempts for transient LLM failures |
+| `base_delay` | `float` | `1.0` | Initial exponential backoff delay (seconds) |
+| `max_delay` | `float` | `60.0` | Cap on exponential backoff delay (seconds) |
+| `jitter` | `bool` | `True` | Add randomized jitter to retry delays to prevent thundering herds |
+| `retry_on` | `tuple` | `(Exception,)` | Exception types that trigger automatic retries |
+| `fallback` | `list` | `[]` | Ordered list of fallback functions to call if primary function fails |
+| `rate_limit_rate` | `float` | `2.0` | Token-bucket refill rate (tokens/second) |
+| `rate_limit_max_tokens` | `float` | `10.0` | Token-bucket maximum capacity (burst limit) |
+| `max_concurrency` | `int \| None` | `None` | Max concurrent executions allowed across processes |
+| `cost_per_call` | `float \| None` | `None` | Explicit cost override per call ($/call) |
+| `policy` | `callable \| None` | `None` | Custom safety hook returning `"allow"`, `"block"`, or `"flag"` |
+| `canary` | `dict \| None` | `None` | Canary routing config `{"fn": alternate_fn, "weight": 0.10}` |
+
+---
+
+## Supported Providers
+
+`undertow_llm` works with **any provider** — OpenAI, Anthropic, Google Gemini, Ollama, HuggingFace, or custom local models — since it wraps your existing Python function call rather than a specific provider SDK.
+
+---
+
+## Examples
+
+See [`demo/example_usage.py`](demo/example_usage.py) for complete runnable examples.
+
+### 1. Multi-Provider Fallback Chain
+```python
+from undertow_llm import track
+
+def fallback_anthropic(prompt: str) -> str:
+    return anthropic_client.messages.create(model="claude-3-5-sonnet", messages=[{"role": "user", "content": prompt}]).content[0].text
+
+@track(retries=2, fallback=[fallback_anthropic])
+def primary_openai(prompt: str) -> str:
+    return openai_client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": prompt}]).choices[0].message.content
+```
+
+### 2. Streaming LLM Response
+```python
+@track(cache=False)
+def stream_gemini(prompt: str):
+    response = gemini_client.models.generate_content_stream("gemini-2.5-flash", prompt)
+    for chunk in response:
+        yield chunk.text
+```
+
+### 3. Local Model (Ollama) with Custom Usage Extractor
+```python
+@track(
+    cost_per_call=0.0,  # Local model — zero API cost
+    usage_extractor=lambda res: {"prompt_tokens": len(res.get("prompt", "")), "completion_tokens": len(res.get("response", ""))}
+)
+def ask_ollama(prompt: str) -> dict:
+    return ollama.generate(model="llama3", prompt=prompt)
+```
+
+---
+
+## Known Limitations & Roadmap
+
+### Limitations
+- **SQLite Concurrency**: Local SQLite storage (`undertow_llm.db`) is zero-config and ideal for development and single-instance apps, but is not designed for multi-node production scale. For high concurrency, set `UNDERTOW_LLM_POSTGRES_URL` and `UNDERTOW_LLM_REDIS_URL`.
+
+### Near-Term Roadmap
+- [ ] OpenTelemetry trace exporter integration
+- [ ] Multi-tenant workspace tagging & dashboard authentication
+- [ ] Automated PII redaction and sensitive prompt masking filters
+
+---
+
+## Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for developer setup instructions.
+
+---
+
+## License
+
+MIT — see [LICENSE](LICENSE)

undertow_llm-0.2.1/pyproject.toml

@@ -0,0 +1,81 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "undertow-llm"
+version = "0.2.1"
+description = "Provider-agnostic LLM observability & reliability SDK — semantic caching, retries, rate limiting, cost tracking, and a local dashboard via a single @track() decorator."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+keywords = ["llm", "observability", "caching", "openai", "gemini", "anthropic", "sdk", "ai-infra"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+authors = [{ name = "Amogh Arora" }]
+dependencies = [
+    # Semantic cache
+    "sentence-transformers>=2.7.0",
+    "numpy>=1.26.0",
+
+    # Dashboard backend
+    "fastapi>=0.111.0",
+    "uvicorn[standard]>=0.29.0",
+    "jinja2>=3.1.4",
+
+    # CLI
+    "click>=8.1.7",
+
+    # Utilities
+    "python-dotenv>=1.0.1",
+    "httpx>=0.27.0",       # async HTTP for alerting webhook
+]
+
+# LLM providers are optional — undertow_llm is provider-agnostic.
+# Install the provider you use, e.g.: pip install undertow-llm[gemini]
+[project.optional-dependencies]
+gemini = ["google-genai>=1.0.0"]
+ollama = ["ollama>=0.2.0"]
+all-providers = ["google-genai>=1.0.0", "ollama>=0.2.0"]
+postgres = ["psycopg2-binary>=2.9.0", "pgvector>=0.2.0"]
+redis = ["redis>=5.0.0"]
+prod = ["psycopg2-binary>=2.9.0", "pgvector>=0.2.0", "redis>=5.0.0"]
+dev = [
+    "pytest>=8.2.0",
+    "pytest-asyncio>=0.23.0",
+    "pytest-cov>=5.0.0",
+    "locust>=2.28.0",
+]
+
+[project.scripts]
+undertow-llm = "undertow_llm.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/ShriAmogh/undertow-llm"
+Repository = "https://github.com/ShriAmogh/undertow-llm"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["undertow_llm*"]
+
+# Bundle the dashboard HTML/CSS/JS — without this the dashboard silently breaks
+# when the package is installed via pip (non-Python files are ignored by default).
+[tool.setuptools.package-data]
+"undertow_llm" = [
+    "server/templates/*.html",
+    "server/static/*.css",
+    "server/static/*.js",
+    "server/static/*.png",
+    "server/static/*.svg",
+]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]

undertow_llm-0.2.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

undertow_llm-0.2.1/tests/test_cache.py

@@ -0,0 +1,156 @@
+"""
+tests/test_cache.py
+===================
+Phase 2 tests: semantic cache — embedding, cosine similarity, store.
+
+We mock the sentence-transformer model so tests run fast (no 80MB model
+download in CI), but keep the cosine similarity math unpatched so we
+validate the algorithm itself.
+"""
+
+from __future__ import annotations
+
+import time
+import numpy as np
+import pytest
+from unittest.mock import patch, MagicMock
+
+from undertow_llm.cache.semantic import cosine_similarity, embedding_to_bytes, bytes_to_embedding
+from undertow_llm.cache.store import CacheStore
+
+
+# ── Fixtures ──────────────────────────────────────────────────────────────────
+
+@pytest.fixture
+def cache_store(temp_db):
+    return CacheStore(db_path=temp_db)
+
+
+def _fake_embed(text: str) -> np.ndarray:
+    """
+    Deterministic fake embedding for tests.
+    Near-duplicates (same first 20 chars) get a nearly identical vector.
+    Dissimilar texts get a near-orthogonal vector.
+    """
+    np.random.seed(hash(text[:20]) % (2**31))
+    base = np.random.rand(384).astype(np.float32)
+    # Add a small perturbation for texts with same prefix (simulates near-duplicates)
+    if len(text) > 20:
+        np.random.seed(hash(text) % (2**31))
+        perturbation = np.random.rand(384).astype(np.float32) * 0.05
+        base = base + perturbation
+    # Normalise
+    return base / np.linalg.norm(base)
+
+
+# ── Cosine Similarity ─────────────────────────────────────────────────────────
+
+class TestCosineSimilarity:
+    def test_identical_vectors_score_one(self):
+        v = np.array([1.0, 0.0, 0.0], dtype=np.float32)
+        assert cosine_similarity(v, v) == pytest.approx(1.0, abs=1e-6)
+
+    def test_orthogonal_vectors_score_zero(self):
+        a = np.array([1.0, 0.0, 0.0], dtype=np.float32)
+        b = np.array([0.0, 1.0, 0.0], dtype=np.float32)
+        assert cosine_similarity(a, b) == pytest.approx(0.0, abs=1e-6)
+
+    def test_opposite_vectors_score_minus_one(self):
+        a = np.array([1.0, 0.0], dtype=np.float32)
+        b = np.array([-1.0, 0.0], dtype=np.float32)
+        assert cosine_similarity(a, b) == pytest.approx(-1.0, abs=1e-6)
+
+    def test_similar_vectors_score_high(self):
+        # Two nearly-identical unit vectors
+        a = np.array([0.6, 0.8, 0.0], dtype=np.float32)
+        b = np.array([0.601, 0.799, 0.01], dtype=np.float32)
+        b = b / np.linalg.norm(b)
+        score = cosine_similarity(a, b)
+        assert score > 0.99
+
+
+# ── Serialisation ─────────────────────────────────────────────────────────────
+
+class TestSerialization:
+    def test_roundtrip_preserves_vector(self):
+        original = np.random.rand(384).astype(np.float32)
+        blob = embedding_to_bytes(original)
+        recovered = bytes_to_embedding(blob)
+        np.testing.assert_array_almost_equal(original, recovered)
+
+    def test_blob_is_bytes(self):
+        v = np.ones(384, dtype=np.float32)
+        assert isinstance(embedding_to_bytes(v), bytes)
+
+
+# ── CacheStore ────────────────────────────────────────────────────────────────
+
+class TestCacheStore:
+    def test_miss_on_empty_store(self, cache_store):
+        emb = _fake_embed("What is Python?")
+        result = cache_store.lookup(emb, threshold=0.92)
+        assert result is None
+
+    def test_write_then_exact_hit(self, cache_store):
+        """Exact same prompt embedding should always hit."""
+        prompt = "What is Python?"
+        emb = _fake_embed(prompt)
+
+        cache_store.write(prompt, emb, "Python is a programming language.", ttl=None)
+        result = cache_store.lookup(emb, threshold=0.92)
+
+        assert result is not None
+        assert result.response == "Python is a programming language."
+        assert result.similarity == pytest.approx(1.0, abs=1e-4)
+
+    def test_dissimilar_prompt_misses(self, cache_store):
+        """A semantically unrelated prompt must not hit the cache."""
+        prompt_a = "What is Python?"
+        emb_a = _fake_embed(prompt_a)
+        cache_store.write(prompt_a, emb_a, "Python is a language.", ttl=None)
+
+        # Chocolate cake is completely unrelated → orthogonal embedding
+        emb_b = _fake_embed("What is the best chocolate cake recipe?") * -1  # force dissimilar
+        emb_b = emb_b / np.linalg.norm(emb_b)
+
+        result = cache_store.lookup(emb_b, threshold=0.92)
+        assert result is None
+
+    def test_ttl_expiry(self, cache_store):
+        """An entry with expired TTL must not be returned."""
+        prompt = "Test TTL"
+        emb = _fake_embed(prompt)
+        # Write with TTL of 1 second
+        cache_store.write(prompt, emb, "cached response", ttl=1)
+
+        # Should hit immediately
+        assert cache_store.lookup(emb, threshold=0.90) is not None
+
+        # Wait for expiry
+        time.sleep(1.1)
+        assert cache_store.lookup(emb, threshold=0.90) is None
+
+    def test_hit_count_increments(self, cache_store):
+        """hit_count must increment on each cache hit."""
+        emb = _fake_embed("Python question")
+        cache_store.write("Python question", emb, "answer", ttl=None)
+
+        result1 = cache_store.lookup(emb, threshold=0.90)
+        result2 = cache_store.lookup(emb, threshold=0.90)
+
+        assert result1.hit_count == 1
+        assert result2.hit_count == 2
+
+    def test_count_returns_valid_entries(self, cache_store):
+        """count() should return the number of non-expired entries."""
+        assert cache_store.count() == 0
+        cache_store.write("q1", _fake_embed("q1"), "a1", ttl=None)
+        cache_store.write("q2", _fake_embed("q2"), "a2", ttl=None)
+        assert cache_store.count() == 2
+
+    def test_expired_entries_excluded_from_count(self, cache_store):
+        """count() must not include expired entries."""
+        cache_store.write("q", _fake_embed("q"), "a", ttl=1)
+        assert cache_store.count() == 1
+        time.sleep(1.1)
+        assert cache_store.count() == 0