generic-ml-cache-daemon 0.13.0__tar.gz

generic_ml_cache_daemon-0.13.0/.gitignore

@@ -0,0 +1,18 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+build/
+*.egg-info/
+.coverage
+coverage.xml
+.pytest_cache/
+.ruff_cache/
+
+# editors / IDEs
+.idea/
+.vscode/
+
+# machine-local repo tooling (not project content)
+secret-audit.sh
+setup-repo.sh

generic_ml_cache_daemon-0.13.0/PKG-INFO

@@ -0,0 +1,36 @@
+Metadata-Version: 2.4
+Name: generic-ml-cache-daemon
+Version: 0.13.0
+Summary: Local HTTP daemon for generic-ml-cache: REST API, gateway proxy, and session transport. A thin inbound driver over generic-ml-cache-core.
+Project-URL: Homepage, https://github.com/danielslobozian/generic-ml-cache
+Project-URL: Repository, https://github.com/danielslobozian/generic-ml-cache
+Project-URL: Issues, https://github.com/danielslobozian/generic-ml-cache/issues
+Project-URL: Changelog, https://github.com/danielslobozian/generic-ml-cache/blob/main/CHANGELOG.md
+Author: Daniel Slobozian
+License-Expression: Apache-2.0
+Keywords: ai,cache,daemon,fastapi,gateway,http,llm,proxy
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Utilities
+Requires-Python: >=3.9
+Requires-Dist: fastapi>=0.115
+Requires-Dist: generic-ml-cache-core>=0.12.0
+Requires-Dist: sse-starlette>=2.0
+Requires-Dist: uvicorn>=0.30
+Provides-Extra: dev
+Requires-Dist: coverage>=7; extra == 'dev'
+Requires-Dist: httpx>=0.27; extra == 'dev'
+Requires-Dist: prometheus-client>=0.20; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff>=0.15; extra == 'dev'
+Provides-Extra: metrics
+Requires-Dist: prometheus-client>=0.20; extra == 'metrics'

generic_ml_cache_daemon-0.13.0/README.md

@@ -0,0 +1,165 @@
+# generic-ml-cache-daemon
+
+The HTTP daemon for [generic-ml-cache](https://github.com/danielslobozian/generic-ml-cache).
+Exposes the cache store and all session/execution functionality as a local REST API with
+server-sent event (SSE) streaming, plus a caching proxy gateway for the Anthropic Messages API.
+
+## Installation
+
+```bash
+pip install generic-ml-cache-daemon           # runtime only
+pip install "generic-ml-cache-daemon[metrics]" # + Prometheus /metrics endpoint
+```
+
+## Starting the daemon
+
+### Via the CLI (recommended)
+
+```bash
+gmlcache daemon start                 # foreground, default 127.0.0.1:8765
+gmlcache daemon start --port 9000     # custom port
+gmlcache daemon start --metrics       # enable /metrics endpoint
+gmlcache daemon start --session abc   # bind to session "abc"
+```
+
+Check status or stop:
+
+```bash
+gmlcache daemon status
+gmlcache daemon stop
+```
+
+### Direct launch
+
+```bash
+python -m generic_ml_cache_daemon             # uses defaults
+GMLCACHE_STORE=/path/to/store python -m generic_ml_cache_daemon
+GMLCACHE_SESSION=abc GMLCACHE_METRICS=1 python -m generic_ml_cache_daemon
+```
+
+Environment variables:
+
+| Variable | Default | Description |
+|---|---|---|
+| `GMLCACHE_STORE` | config store path | Path to the cache store directory |
+| `GMLCACHE_SESSION` | *(none)* | Bind all intercepted calls to a session |
+| `GMLCACHE_METRICS` | `0` | Set `1` to enable the Prometheus `/metrics` endpoint |
+
+## HTTP API
+
+The daemon listens on `http://127.0.0.1:8765` by default.
+Interactive API docs are available at `/docs` (Swagger UI) and `/redoc`.
+
+### Observability
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/health` | Liveness: `{"status":"ok"}` |
+| `GET` | `/ready` | Readiness: probes the store; 503 if inaccessible |
+| `GET` | `/info` | Version, store path, adapters, bound session |
+| `GET` | `/metrics` | Prometheus text (requires `[metrics]` extra + `--metrics`) |
+
+### Sessions
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/sessions` | List all session IDs |
+| `POST` | `/sessions` | Create a session (body: `{tags, spec}`) |
+| `GET` | `/sessions/{id}` | Get session tags and spec (404 if unknown) |
+| `GET` | `/sessions/{id}/stats` | Calls, hits, hit rate |
+| `PUT` | `/sessions/{id}/spec` | Set or replace execution spec |
+| `DELETE` | `/sessions/{id}/spec` | Remove execution spec |
+| `POST` | `/sessions/{id}/tags` | Add a tag |
+| `DELETE` | `/sessions/{id}/tags/{tag}` | Remove a tag |
+
+### Executions & Global Stats
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/executions` | List all current (servable) executions |
+| `GET` | `/executions/{key}` | Inspect by exact key or prefix (409 on ambiguous prefix) |
+| `GET` | `/stats` | Global execution count + event counts |
+| `POST` | `/purge` | Purge by scope: `all`, `key`, `tag`, `session`, `session_tag` |
+
+**Purge body examples:**
+
+```json
+{"by": "all"}
+{"by": "key", "target": "deadbeef"}
+{"by": "session", "target": "abc123"}
+```
+
+### Run (synchronous or SSE)
+
+```
+POST /run
+{
+  "client": "anthropic",
+  "model": "claude-opus-4-8",
+  "prompt": "Summarise the paper.",
+  "effort": "medium",
+  "session_id": "abc"
+}
+```
+
+- `Accept: application/json` (default) — blocks and returns `{execution_key, state, cache_hit, stdout, stderr}`
+- `Accept: text/event-stream` — SSE: `{"type":"accepted"}` immediately, then `{"type":"complete", ...}` on finish
+
+### Jobs (detached / async)
+
+| Method | Path | Description |
+|---|---|---|
+| `POST` | `/jobs` | Submit a background execution; returns `{job_id, state}` with 202 |
+| `GET` | `/jobs` | List all job IDs |
+| `GET` | `/jobs/{id}` | Poll state: `pending`, `running`, `done`, `error` |
+| `GET` | `/jobs/{id}/stream` | SSE: periodic `status` events, then `complete` or `error` |
+
+### Claude Gateway
+
+```
+POST /gateway/claude/v1/messages
+```
+
+A cache-transparent proxy for the Anthropic Messages API. Requests that hit the
+cache are returned without a network call to Anthropic. The response shape matches
+the Anthropic Messages API exactly, with one extra field: `x_cache_hit: bool`.
+
+**Limitations (0.13.0):** single-turn conversations only (one `role: user` message,
+no prior assistant turns). Multi-turn support is planned.
+
+**Example:**
+
+```bash
+curl http://127.0.0.1:8765/gateway/claude/v1/messages \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "claude-opus-4-8",
+    "messages": [{"role": "user", "content": "Hello, world!"}],
+    "max_tokens": 256
+  }'
+```
+
+Point any Anthropic SDK client at the gateway by overriding the base URL:
+
+```python
+import anthropic
+
+client = anthropic.Anthropic(
+    api_key="...",
+    base_url="http://127.0.0.1:8765/gateway/claude",
+)
+```
+
+## Architecture
+
+The daemon is a thin FastAPI layer over the `generic-ml-cache-core` hexagonal
+architecture. It does not own any state — all persistence goes through the
+existing `JournalMetrics` (SQLite registry) and `SqliteExecutionRepository`
+that the core library manages.
+
+Background jobs run in a `ThreadPoolExecutor` inside an in-process
+`JobRegistry`; job state is not persisted across daemon restarts.
+
+## License
+
+Apache-2.0

generic_ml_cache_daemon-0.13.0/pyproject.toml

@@ -0,0 +1,64 @@
+[build-system]
+requires = ["hatchling>=1.18"]
+build-backend = "hatchling.build"
+
+[project]
+name = "generic-ml-cache-daemon"
+version = "0.13.0"
+description = "Local HTTP daemon for generic-ml-cache: REST API, gateway proxy, and session transport. A thin inbound driver over generic-ml-cache-core."
+requires-python = ">=3.9"
+license = "Apache-2.0"
+license-files = ["LICENSE", "NOTICE"]
+authors = [{ name = "Daniel Slobozian" }]
+keywords = ["cache", "llm", "ai", "daemon", "http", "fastapi", "gateway", "proxy"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: Apache Software License",
+  "Operating System :: OS Independent",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Utilities",
+]
+dependencies = [
+  "generic-ml-cache-core>=0.12.0",
+  "fastapi>=0.115",
+  "uvicorn>=0.30",
+  "sse-starlette>=2.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/danielslobozian/generic-ml-cache"
+Repository = "https://github.com/danielslobozian/generic-ml-cache"
+Issues = "https://github.com/danielslobozian/generic-ml-cache/issues"
+Changelog = "https://github.com/danielslobozian/generic-ml-cache/blob/main/CHANGELOG.md"
+
+[project.optional-dependencies]
+# Optional Prometheus /metrics endpoint. Off by default.
+metrics = ["prometheus-client>=0.20"]
+dev = [
+  "pytest>=7",
+  "pytest-cov",
+  "coverage>=7",
+  "ruff>=0.15",
+  "httpx>=0.27",
+  "prometheus-client>=0.20",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/generic_ml_cache_daemon"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"
+
+[tool.coverage.run]
+omit = ["*/generic_ml_cache_daemon/__main__.py"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py39"

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/__init__.py

@@ -0,0 +1,3 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+__version__ = "0.13.0"

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/__main__.py

@@ -0,0 +1,27 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Entry point: run the daemon via ``python -m generic_ml_cache_daemon``."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+import uvicorn
+
+from generic_ml_cache_daemon.app import create_app
+
+_DEFAULT_HOST = "127.0.0.1"
+_DEFAULT_PORT = 8765
+
+
+def main() -> None:
+    store_root = Path(os.environ.get("GMLCACHE_STORE", str(Path.home() / ".gmlcache")))
+    session_id = os.environ.get("GMLCACHE_SESSION") or None
+    enable_metrics = os.environ.get("GMLCACHE_METRICS", "").lower() in ("1", "true", "yes")
+    application = create_app(store_root, session_id=session_id, enable_metrics=enable_metrics)
+    uvicorn.run(application, host=_DEFAULT_HOST, port=_DEFAULT_PORT)
+
+
+if __name__ == "__main__":
+    main()

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/app.py

@@ -0,0 +1,63 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""FastAPI application factory for the generic-ml-cache daemon."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Optional
+
+from fastapi import FastAPI
+
+from generic_ml_cache_core.adapter.inbound.composition import build_use_cases
+
+from generic_ml_cache_daemon import __version__
+
+
+def create_app(
+    store_root: Path,
+    *,
+    session_id: Optional[str] = None,
+    enable_metrics: bool = False,
+) -> FastAPI:
+    """Create and configure the daemon FastAPI application.
+
+    Args:
+        store_root: path to the gmlcache store directory (the injected data source).
+        session_id: optional session all intercepted calls are recorded under.
+        enable_metrics: expose the Prometheus /metrics endpoint.
+
+    Returns:
+        A fully wired FastAPI application. Routes are mounted by this function;
+        callers should not mount additional routes after construction.
+    """
+    application = FastAPI(
+        title="generic-ml-cache daemon",
+        version=__version__,
+        docs_url="/docs",
+        redoc_url="/redoc",
+    )
+
+    wired_use_cases = build_use_cases(store_root)
+    application.state.wired = wired_use_cases
+    application.state.store_root = store_root
+    application.state.session_id = session_id
+    application.state.enable_metrics = enable_metrics
+
+    from generic_ml_cache_daemon.jobs import JobRegistry
+    from generic_ml_cache_daemon.routes.executions import router as executions_router
+    from generic_ml_cache_daemon.routes.gateway import router as gateway_router
+    from generic_ml_cache_daemon.routes.health import router as health_router
+    from generic_ml_cache_daemon.routes.jobs import router as jobs_router
+    from generic_ml_cache_daemon.routes.run import router as run_router
+    from generic_ml_cache_daemon.routes.sessions import router as sessions_router
+
+    application.state.job_registry = JobRegistry()
+    application.include_router(health_router)
+    application.include_router(sessions_router)
+    application.include_router(executions_router)
+    application.include_router(run_router)
+    application.include_router(jobs_router)
+    application.include_router(gateway_router)
+
+    return application

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/jobs.py

@@ -0,0 +1,86 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""In-process job registry for detached background executions.
+
+Each POST /jobs submission gets a unique job_id. The execution runs in a
+background thread; callers poll GET /jobs/{id} or stream GET /jobs/{id}/stream.
+The registry is in-process memory only — jobs are not persisted across restarts.
+"""
+
+from __future__ import annotations
+
+import concurrent.futures
+import secrets
+import threading
+from enum import Enum
+from typing import Dict, Optional
+
+from generic_ml_cache_core.application.domain.model.execution.ml_execution import MlExecution
+
+
+class JobState(str, Enum):
+    PENDING = "pending"
+    RUNNING = "running"
+    DONE = "done"
+    ERROR = "error"
+
+
+class Job:
+    def __init__(self, job_id: str) -> None:
+        self.job_id = job_id
+        self.state = JobState.PENDING
+        self.execution: Optional[MlExecution] = None
+        self.error: Optional[str] = None
+        self._done_event = threading.Event()
+
+    def wait(self, timeout: Optional[float] = None) -> bool:
+        return self._done_event.wait(timeout=timeout)
+
+    def mark_running(self) -> None:
+        self.state = JobState.RUNNING
+
+    def mark_done(self, execution: MlExecution) -> None:
+        self.execution = execution
+        self.state = JobState.DONE
+        self._done_event.set()
+
+    def mark_error(self, error: str) -> None:
+        self.error = error
+        self.state = JobState.ERROR
+        self._done_event.set()
+
+
+class JobRegistry:
+    """Thread-safe in-memory registry of submitted jobs."""
+
+    def __init__(self) -> None:
+        self._jobs: Dict[str, Job] = {}
+        self._lock = threading.Lock()
+        self._executor = concurrent.futures.ThreadPoolExecutor(
+            max_workers=4, thread_name_prefix="gmlc-job"
+        )
+
+    def submit(self, fn, *args) -> Job:
+        job_id = secrets.token_hex(8)
+        job = Job(job_id)
+        with self._lock:
+            self._jobs[job_id] = job
+
+        def _run() -> None:
+            job.mark_running()
+            try:
+                execution = fn(*args)
+                job.mark_done(execution)
+            except Exception as exc:
+                job.mark_error(str(exc))
+
+        self._executor.submit(_run)
+        return job
+
+    def get(self, job_id: str) -> Optional[Job]:
+        with self._lock:
+            return self._jobs.get(job_id)
+
+    def list_ids(self) -> list:
+        with self._lock:
+            return list(self._jobs.keys())

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/metrics.py

@@ -0,0 +1,17 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Prometheus metrics setup for the daemon. Requires the optional [metrics] extra."""
+
+from __future__ import annotations
+
+try:
+    import prometheus_client  # type: ignore[import-untyped]  # noqa: F401
+
+    _AVAILABLE = True
+except ImportError:  # pragma: no cover
+    _AVAILABLE = False
+
+
+def is_prometheus_available() -> bool:
+    """Return True when the prometheus-client extra is installed."""
+    return _AVAILABLE

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/__init__.py

@@ -0,0 +1,2 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/execution.py

@@ -0,0 +1,59 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Pydantic models for the Executions HTTP API and global stats/purge."""
+
+from __future__ import annotations
+
+from typing import Dict, List, Literal, Union
+
+from pydantic import BaseModel
+
+
+class ExecutionSummaryResponse(BaseModel):
+    execution_key: str
+    kind: str
+    client: str
+    model: str
+
+
+class ExecutionListResponse(BaseModel):
+    executions: List[ExecutionSummaryResponse]
+    total: int
+
+
+class GlobalStatsResponse(BaseModel):
+    executions: int
+    event_counts: Dict[str, int]
+
+
+class PurgeByAll(BaseModel):
+    by: Literal["all"]
+
+
+class PurgeByKey(BaseModel):
+    by: Literal["key"]
+    target: str
+
+
+class PurgeByTag(BaseModel):
+    by: Literal["tag"]
+    target: str
+
+
+class PurgeBySession(BaseModel):
+    by: Literal["session"]
+    target: str
+
+
+class PurgeBySessionTag(BaseModel):
+    by: Literal["session_tag"]
+    target: str
+
+
+PurgeBody = Union[PurgeByAll, PurgeByKey, PurgeByTag, PurgeBySession, PurgeBySessionTag]
+
+
+class PurgeResponse(BaseModel):
+    executions_removed: int
+    bytes_freed: int
+    blobs_removed: int

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/gateway.py

@@ -0,0 +1,39 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Pydantic models for the Claude gateway (/gateway/claude/v1/messages)."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel
+
+
+class MessageParam(BaseModel):
+    role: str
+    content: str
+
+
+class MessagesRequest(BaseModel):
+    model: str
+    messages: List[MessageParam]
+    max_tokens: int = 8192
+    system: Optional[str] = None
+    session_id: Optional[str] = None
+
+
+class ContentBlock(BaseModel):
+    type: str = "text"
+    text: str
+
+
+class MessagesResponse(BaseModel):
+    id: str
+    type: str = "message"
+    role: str = "assistant"
+    content: List[ContentBlock]
+    model: str
+    stop_reason: str = "end_turn"
+    stop_sequence: Optional[str] = None
+    usage: Dict[str, Any]
+    x_cache_hit: bool = False

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/health.py

@@ -0,0 +1,25 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Pydantic response models for /health, /ready, and /info."""
+
+from __future__ import annotations
+
+from typing import List, Optional
+
+from pydantic import BaseModel
+
+
+class HealthResponse(BaseModel):
+    status: str
+
+
+class ReadyResponse(BaseModel):
+    status: str
+    detail: Optional[str] = None
+
+
+class InfoResponse(BaseModel):
+    version: str
+    store_root: str
+    session_id: Optional[str] = None
+    adapters: List[str]

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/job.py

@@ -0,0 +1,28 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Pydantic models for the Jobs HTTP API (detached background executions)."""
+
+from __future__ import annotations
+
+from typing import List, Optional
+
+from pydantic import BaseModel
+
+
+class JobSubmitBody(BaseModel):
+    client: str
+    model: str
+    effort: str = ""
+    prompt: str = ""
+    context: str = ""
+    tags: List[str] = []
+    session_id: Optional[str] = None
+
+
+class JobResponse(BaseModel):
+    job_id: str
+    state: str
+    execution_key: Optional[str] = None
+    stdout: Optional[str] = None
+    stderr: Optional[str] = None
+    error: Optional[str] = None

generic_ml_cache_daemon-0.13.0/src/generic_ml_cache_daemon/models/run.py

@@ -0,0 +1,27 @@
+# SPDX-FileCopyrightText: 2026 Daniel Slobozian
+# SPDX-License-Identifier: Apache-2.0
+"""Pydantic models for the /run endpoint."""
+
+from __future__ import annotations
+
+from typing import List, Optional
+
+from pydantic import BaseModel
+
+
+class RunBody(BaseModel):
+    client: str
+    model: str
+    effort: str = ""
+    prompt: str = ""
+    context: str = ""
+    tags: List[str] = []
+    session_id: Optional[str] = None
+
+
+class RunResponse(BaseModel):
+    execution_key: str
+    state: str
+    cache_hit: bool
+    stdout: Optional[str] = None
+    stderr: Optional[str] = None