lifeboatcache 0.1.0__tar.gz

lifeboatcache-0.1.0/.gitignore

@@ -0,0 +1,15 @@
+.venv/
+vendor/
+__pycache__/
+*.pyc
+*.so
+*.egg-info/
+build/
+dist/
+.pytest_cache/
+.coverage
+htmlcov/
+scratch_*.log
+*.tmp
+target/
+.lifeboat_kv/

lifeboatcache-0.1.0/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.5
+Name: lifeboatcache
+Version: 0.1.0
+Summary: Lifeboat Cache — tiered, compressed, persistent KV cache for vLLM, SGLang and Lifeboat, with a built-in dashboard
+Author-email: "Iterate.ai" <brian@iterate.ai>
+License: Proprietary
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.24
+Requires-Dist: pydantic<3,>=2.7
+Requires-Dist: pyyaml<7,>=6
+Provides-Extra: dev
+Requires-Dist: cython; extra == 'dev'
+Requires-Dist: maturin<2,>=1.7; extra == 'dev'
+Requires-Dist: moto<6,>=5; extra == 'dev'
+Requires-Dist: openai; extra == 'dev'
+Requires-Dist: pytest-cov>=5; extra == 'dev'
+Requires-Dist: pytest<9,>=8; extra == 'dev'
+Provides-Extra: engines
+Requires-Dist: lmcache<0.6,>=0.5.6; extra == 'engines'
+Requires-Dist: nixl<2,>=1.5.0; extra == 'engines'
+Requires-Dist: sglang<0.6,>=0.5.19; extra == 'engines'
+Requires-Dist: vllm<0.33,>=0.32.0; extra == 'engines'
+Provides-Extra: s3
+Requires-Dist: boto3>=1.28; extra == 's3'
+Provides-Extra: sglang
+Provides-Extra: vllm
+Requires-Dist: lmcache<0.6,>=0.5.6; extra == 'vllm'
+Description-Content-Type: text/markdown
+
+# Lifeboat Cache
+
+Tiered, compressed, persistent KV cache for **vLLM**, **SGLang**, and **Lifeboat** (which
+embeds SGLang), with a built-in dashboard to configure, monitor and observe it.
+
+```bash
+pip install "lifeboatcache[vllm]"   # into the environment that runs vLLM
+pip install lifeboatcache           # into the environment that runs SGLang / Lifeboat
+```
+
+Installing registers engine plugins; nothing else is needed to start. With no config file,
+Lifeboat writes KV to local disk under `~/.lifeboat_cache/kv`, compresses it with fp8, and
+serves the dashboard on <http://127.0.0.1:9400>.
+
+## Turn it on
+
+| Engine | How |
+|---|---|
+| vLLM | `LIFEBOAT_CACHE=1 vllm serve <model>` — or explicitly `--kv-transfer-config '{"kv_connector":"LifeboatConnector","kv_role":"kv_both"}'` |
+| SGLang | `LIFEBOAT_CACHE=1 python -m sglang.launch_server --model-path <model>` |
+| Lifeboat | set `LIFEBOAT_CACHE=1` in the Lifeboat container's environment, with `lifeboatcache` installed in its Python env |
+
+What happens underneath:
+
+- **vLLM** — `LifeboatConnector` is LMCache's multiprocess connector with Lifeboat defaults: it
+  auto-starts a local LMCache server (port 6555) whose L2 storage is the Lifeboat adapter.
+  GPU ↔ host movement is LMCache's; everything below host RAM is Lifeboat.
+- **SGLang / Lifeboat** — Lifeboat is a HiCache storage backend (`--hicache-storage-backend
+  dynamic`). With `LIFEBOAT_CACHE=1` the plugin enables HiCache and selects the backend unless
+  the launch already chose a storage backend. Explicit flags work too:
+
+  ```bash
+  python -m sglang.launch_server --model-path <model> --enable-hierarchical-cache \
+    --hicache-storage-backend dynamic \
+    --hicache-storage-backend-extra-config '{"backend_name":"lifeboat","module_path":"lifeboat_cache.integrations.sglang_backend","class_name":"LifeboatHiCacheStorage"}'
+  ```
+
+## Storage targets and the local fallback
+
+Targets are tried in order; the first usable one takes new writes. Each is write-probed at
+startup. One that can't be reached (ONTAP/NFS export not mounted, S3 without credentials) is
+skipped with the reason shown on the dashboard, and if nothing persistent is left, KV goes to
+`storage.fallback_local` on local disk.
+
+```yaml
+storage:
+  tiers:
+    - {name: ontap, type: ontap, enabled: true, mount: /mnt/kvcache, protocol: pnfs_rdma}
+    - {name: nfs,   type: nfs,   enabled: true, mount: /mnt/nfs, path: /mnt/nfs/lifeboat/kv}
+    - {name: s3,    type: s3,    enabled: true, bucket: kv-archive, region: us-east-1}
+  fallback_local: {enabled: true, path: ~/.lifeboat_cache/kv, capacity_gb: 50}
+```
+
+Persisted KV survives restarts: on startup the index is rebuilt from disk in the background.
+
+## Dashboard
+
+`http://127.0.0.1:9400` on the node (use `ssh -L 9400:localhost:9400 <node>` from your
+laptop). It shows the cache hit rate, KV served, bytes saved by compression and the codec's
+sampled reconstruction error, per-target status (circuit breaker, usage, queue), evictions,
+and startup recovery. The **Configure** tab edits storage targets, the local fallback, the
+compression codec per rung, the eviction policy and the write mode; changes are validated,
+applied live, and saved to the config file (startup-bound sections are flagged "restart
+required"). `/metrics` serves Prometheus text (also on port 9402).
+
+To expose it beyond localhost, set `dashboard.host: 0.0.0.0` **and** `dashboard.auth_token`;
+open the page once with `?token=<token>`.
+
+CLI: `lifeboat-cache status` prints the same numbers; `lifeboat-cache init` writes a starter
+config; `lifeboat-cache server` runs a standalone Lifeboat-backed LMCache server for several
+vLLM instances to share.
+
+## Configuration
+
+One YAML file, resolved as: `--config` / `$LIFEBOAT_CACHE_CONFIG` →
+`~/.lifeboat_cache/lifeboat.yaml` → built-in defaults (`$LIFEBOAT_CACHE_HOME` moves the home
+directory). `lifeboat-cache config show` prints the resolved config. Presets ship with the
+package: `extends: single_node_gpu`.
+
+## Compression — what the numbers say
+
+Measured on real Qwen3 KV (2,048-token prefix, see `docs/`):
+
+| codec | bits/value | Qwen3-4B perplexity (base 2.297) |
+|---|---|---|
+| fp8 (default) | 8.0 | 2.293 |
+| turboquant4, keep first/last 2 layers | 6.2 | 2.300 |
+| turboquant3 | 4.0 | 824 (breaks the model) |
+| oscar2 on 2,048-token chunks | 5.0 | 72 (breaks the model) |
+
+fp8 is the default for every stored rung. The dashboard shows the sampled reconstruction error
+of whatever codec is live.

lifeboatcache-0.1.0/README.md

@@ -0,0 +1,92 @@
+# Lifeboat Cache
+
+Tiered, compressed, persistent KV cache for **vLLM**, **SGLang**, and **Lifeboat** (which
+embeds SGLang), with a built-in dashboard to configure, monitor and observe it.
+
+```bash
+pip install "lifeboatcache[vllm]"   # into the environment that runs vLLM
+pip install lifeboatcache           # into the environment that runs SGLang / Lifeboat
+```
+
+Installing registers engine plugins; nothing else is needed to start. With no config file,
+Lifeboat writes KV to local disk under `~/.lifeboat_cache/kv`, compresses it with fp8, and
+serves the dashboard on <http://127.0.0.1:9400>.
+
+## Turn it on
+
+| Engine | How |
+|---|---|
+| vLLM | `LIFEBOAT_CACHE=1 vllm serve <model>` — or explicitly `--kv-transfer-config '{"kv_connector":"LifeboatConnector","kv_role":"kv_both"}'` |
+| SGLang | `LIFEBOAT_CACHE=1 python -m sglang.launch_server --model-path <model>` |
+| Lifeboat | set `LIFEBOAT_CACHE=1` in the Lifeboat container's environment, with `lifeboatcache` installed in its Python env |
+
+What happens underneath:
+
+- **vLLM** — `LifeboatConnector` is LMCache's multiprocess connector with Lifeboat defaults: it
+  auto-starts a local LMCache server (port 6555) whose L2 storage is the Lifeboat adapter.
+  GPU ↔ host movement is LMCache's; everything below host RAM is Lifeboat.
+- **SGLang / Lifeboat** — Lifeboat is a HiCache storage backend (`--hicache-storage-backend
+  dynamic`). With `LIFEBOAT_CACHE=1` the plugin enables HiCache and selects the backend unless
+  the launch already chose a storage backend. Explicit flags work too:
+
+  ```bash
+  python -m sglang.launch_server --model-path <model> --enable-hierarchical-cache \
+    --hicache-storage-backend dynamic \
+    --hicache-storage-backend-extra-config '{"backend_name":"lifeboat","module_path":"lifeboat_cache.integrations.sglang_backend","class_name":"LifeboatHiCacheStorage"}'
+  ```
+
+## Storage targets and the local fallback
+
+Targets are tried in order; the first usable one takes new writes. Each is write-probed at
+startup. One that can't be reached (ONTAP/NFS export not mounted, S3 without credentials) is
+skipped with the reason shown on the dashboard, and if nothing persistent is left, KV goes to
+`storage.fallback_local` on local disk.
+
+```yaml
+storage:
+  tiers:
+    - {name: ontap, type: ontap, enabled: true, mount: /mnt/kvcache, protocol: pnfs_rdma}
+    - {name: nfs,   type: nfs,   enabled: true, mount: /mnt/nfs, path: /mnt/nfs/lifeboat/kv}
+    - {name: s3,    type: s3,    enabled: true, bucket: kv-archive, region: us-east-1}
+  fallback_local: {enabled: true, path: ~/.lifeboat_cache/kv, capacity_gb: 50}
+```
+
+Persisted KV survives restarts: on startup the index is rebuilt from disk in the background.
+
+## Dashboard
+
+`http://127.0.0.1:9400` on the node (use `ssh -L 9400:localhost:9400 <node>` from your
+laptop). It shows the cache hit rate, KV served, bytes saved by compression and the codec's
+sampled reconstruction error, per-target status (circuit breaker, usage, queue), evictions,
+and startup recovery. The **Configure** tab edits storage targets, the local fallback, the
+compression codec per rung, the eviction policy and the write mode; changes are validated,
+applied live, and saved to the config file (startup-bound sections are flagged "restart
+required"). `/metrics` serves Prometheus text (also on port 9402).
+
+To expose it beyond localhost, set `dashboard.host: 0.0.0.0` **and** `dashboard.auth_token`;
+open the page once with `?token=<token>`.
+
+CLI: `lifeboat-cache status` prints the same numbers; `lifeboat-cache init` writes a starter
+config; `lifeboat-cache server` runs a standalone Lifeboat-backed LMCache server for several
+vLLM instances to share.
+
+## Configuration
+
+One YAML file, resolved as: `--config` / `$LIFEBOAT_CACHE_CONFIG` →
+`~/.lifeboat_cache/lifeboat.yaml` → built-in defaults (`$LIFEBOAT_CACHE_HOME` moves the home
+directory). `lifeboat-cache config show` prints the resolved config. Presets ship with the
+package: `extends: single_node_gpu`.
+
+## Compression — what the numbers say
+
+Measured on real Qwen3 KV (2,048-token prefix, see `docs/`):
+
+| codec | bits/value | Qwen3-4B perplexity (base 2.297) |
+|---|---|---|
+| fp8 (default) | 8.0 | 2.293 |
+| turboquant4, keep first/last 2 layers | 6.2 | 2.300 |
+| turboquant3 | 4.0 | 824 (breaks the model) |
+| oscar2 on 2,048-token chunks | 5.0 | 72 (breaks the model) |
+
+fp8 is the default for every stored rung. The dashboard shows the sampled reconstruction error
+of whatever codec is live.

lifeboatcache-0.1.0/presets/base.yaml

@@ -0,0 +1,16 @@
+# Base preset: codec definitions + fleet-neutral defaults. Other presets extend this.
+# (Defining codecs here means any preset keeping the default ladder validates.)
+compression:
+  recompress_on_demote: true
+  codecs:
+    turboquant4: {type: turboquant, bits: 4, keep_bf16_layers: [0, 1, -2, -1], group_size: 64}
+    turboquant3: {type: turboquant, bits: 3}
+    oscar2:      {type: oscar, bits: 2, rotation: calibrated, sink_tokens: 64, recent_tokens: 256}
+    kvtc:        {type: kvtc, pca_rank: auto, entropy_coder: ans}
+    fp8:         {type: fp8}
+  accuracy_guard:
+    enabled: true
+    sample_rate: 0.01
+    metric: logit_kl
+    threshold: 0.05
+    action: raise_precision

lifeboatcache-0.1.0/presets/dev_cpu.yaml

@@ -0,0 +1,22 @@
+# dev_cpu: single-process CPU dev on a Mac ("fleet of size one"). No control plane.
+extends: base.yaml
+device: cpu
+compression:
+  ladder: {gpu: turboquant3, cpu: turboquant3, local_nvme: kvtc, shared: kvtc, archive: kvtc}
+control_plane:
+  enabled: false
+  membership: static
+  metadata_store: memory
+  offline_mode: local_only
+eviction:
+  l1_policy: lru
+  episodic: {enabled: false}
+storage:
+  enabled: true
+  write_mode: low_latency
+  tiers:
+    - {name: local, type: filesystem, enabled: true, path: ./.lifeboat_kv, scope: node, capacity_gb: 2}
+tenancy:
+  enabled: false
+telemetry:
+  prometheus: {enabled: false}

lifeboatcache-0.1.0/presets/fleet_50k.yaml

@@ -0,0 +1,53 @@
+# fleet_50k: 50,000+ GPU fleet. Control plane on, gossip membership, etcd metadata,
+# heavy sharding + replication, topology-aware placement, full telemetry.
+extends: base.yaml
+device: auto
+topology:
+  discovery: k8s
+  numa_aware: true
+control_plane:
+  enabled: true
+  membership: gossip
+  metadata_store: etcd
+  index_shards: 4096
+  replication_factor: 3
+  heartbeat_ms: 1000
+  offline_mode: cached_placement
+  config_push: {enabled: true, canary_percent: 5, auto_rollback_on_regression: true}
+compression:
+  ladder: {gpu: turboquant4, cpu: turboquant3, local_nvme: kvtc, shared: kvtc, archive: kvtc}
+eviction:
+  l1_policy: episodic
+  episodic: {enabled: true, persist_evicted_episodes: true, prefetch_on_topic_return: true}
+reuse:
+  cross_node_p2p: {enabled: true, transport: nixl, max_hops: 1, prefer_nvlink_domain: true}
+storage:
+  enabled: true
+  write_mode: auto
+  placement:
+    write_local_first: true
+    replicate_to_shared_async: true
+    read_nearest: true
+    hot_object_replicas: 3
+    hotspot_threshold_rps: 500
+  tiers:
+    - {name: nvme,  type: nvme,  enabled: true,  path: /mnt/nvme/kv, io: io_uring,
+       direct_io: true, capacity_gb: 800, scope: node}
+    - {name: ontap, type: ontap, enabled: true, mount: /mnt/kvcache, protocol: pnfs_rdma,
+       prefer_nearest_flexcache: true, fabricpool_cold_after_h: 24, scope: cluster}
+    - {name: s3,    type: s3,    enabled: true, bucket: kv-archive, multipart_parallelism: 8,
+       byte_range_reads: true, scope: region}
+    - {name: rdma,  type: nixl,  enabled: true, scope: pod}
+  backpressure:
+    queue_depth_per_tier: 16384
+    shed_below_reuse_score: 0.1
+    circuit_breaker: {error_rate: 0.2, open_s: 30}
+tenancy:
+  enabled: true
+  default_qos: throughput
+  fair_share: true
+telemetry:
+  prometheus: {enabled: true, port: 9402, max_label_cardinality: 10000}
+  otel: {enabled: true, endpoint: "otel-collector:4317", trace_sample_rate: 0.001}
+  chargeback_per_tenant: true
+  export_locality_to_router: {enabled: true, target: dynamo}

lifeboatcache-0.1.0/presets/max_accuracy.yaml

@@ -0,0 +1,15 @@
+# max_accuracy: preserve quality. Higher-precision codecs, strict accuracy guard.
+extends: base.yaml
+device: auto
+compression:
+  ladder: {gpu: fp8, cpu: turboquant4, local_nvme: turboquant4, shared: turboquant4, archive: kvtc}
+  recompress_on_demote: true
+  accuracy_guard:
+    enabled: true
+    sample_rate: 0.05
+    metric: logit_kl
+    threshold: 0.01
+    action: raise_precision
+eviction:
+  l1_policy: episodic
+  episodic: {enabled: true, per_layer_budget: sensitivity}

lifeboatcache-0.1.0/presets/max_concurrency.yaml

@@ -0,0 +1,15 @@
+# max_concurrency: throughput-first. Aggressive coalescing, write-behind, low-latency codecs.
+extends: base.yaml
+device: auto
+compression:
+  ladder: {gpu: turboquant4, cpu: turboquant4, local_nvme: turboquant4, shared: kvtc, archive: kvtc}
+eviction:
+  l1_policy: arc
+  admission: {enabled: true, min_reuse_score: 0.1, write_behind: always}
+storage:
+  write_mode: low_latency
+  coalesce: {target_tokens: 4096, min_tokens: 512, max_tokens: 8192, adaptive: true, flush_interval_ms: 20}
+  backpressure: {queue_depth_per_tier: 16384, shed_below_reuse_score: 0.2}
+tenancy:
+  default_qos: throughput
+  fair_share: true

lifeboatcache-0.1.0/presets/single_node_gpu.yaml

@@ -0,0 +1,18 @@
+# single_node_gpu: one GPU node, local NVMe tier, no control plane.
+extends: base.yaml
+device: cuda
+control_plane:
+  enabled: false
+  membership: static
+  metadata_store: memory
+  offline_mode: local_only
+eviction:
+  l1_policy: episodic
+storage:
+  enabled: true
+  write_mode: auto
+  tiers:
+    - {name: nvme, type: nvme, enabled: true, path: /mnt/nvme/kv, io: io_uring,
+       direct_io: true, capacity_gb: 800, scope: node}
+telemetry:
+  prometheus: {enabled: true, port: 9402}

lifeboatcache-0.1.0/presets/storage_efficient.yaml

@@ -0,0 +1,13 @@
+# storage_efficient: minimize bytes. Aggressive low-bit codecs, recompress on demote, dedup.
+extends: base.yaml
+device: auto
+compression:
+  ladder: {gpu: turboquant4, cpu: turboquant3, local_nvme: oscar2, shared: kvtc, archive: kvtc}
+  recompress_on_demote: true
+eviction:
+  l1_policy: episodic
+  admission: {enabled: true, min_reuse_score: 0.35, write_behind: idle}
+storage:
+  write_mode: efficient
+  coalesce: {target_tokens: 8192, min_tokens: 1024, max_tokens: 8192, adaptive: true, flush_interval_ms: 100}
+  index: {content_addressed: true, dedup: true, dedup_scope: global, tenant_salted_hashing: true}

lifeboatcache-0.1.0/pyproject.toml

@@ -0,0 +1,78 @@
+[project]
+# Distribution name is `lifeboatcache` (pip install lifeboatcache); the import package is
+# `lifeboat_cache`.
+name = "lifeboatcache"
+version = "0.1.0"
+description = "Lifeboat Cache — tiered, compressed, persistent KV cache for vLLM, SGLang and Lifeboat, with a built-in dashboard"
+readme = "README.md"
+# 3.10+: SGLang (and the Lifeboat product that embeds it) supports 3.10.
+requires-python = ">=3.10"
+license = { text = "Proprietary" }
+authors = [{ name = "Iterate.ai", email = "brian@iterate.ai" }]
+
+# Supported upstream ranges. Vendored at: vllm v0.32.0, lmcache v0.5.6,
+# sglang v0.5.22, nixl v1.5.0 (see docs/VERSIONS.md). These are installed
+# editable from vendor/ during dev; ranges here gate the version-compat test.
+dependencies = [
+    "pydantic>=2.7,<3",
+    "pyyaml>=6,<7",
+    "numpy>=1.24",
+]
+
+[project.optional-dependencies]
+# vLLM path: Lifeboat sits under LMCache's MP server. Install into the env that runs vLLM.
+vllm = ["lmcache>=0.5.6,<0.6"]
+# SGLang / Lifeboat path: nothing extra beyond SGLang itself (already in that env).
+sglang = []
+s3 = ["boto3>=1.28"]
+# Upstream engines — pinned supported ranges (installed editable from vendor/ in dev).
+# vllm 0.32.0: builds on Apple clang (0.31.0 does NOT — OpenMP/structured-binding
+# error), but 0.32.0 has a stale CPU slot-mapping fallback. We repair it at runtime
+# via lifeboat_cache.integrations.vllm_cpu_patches. See upstream_patches/README.md §2.
+engines = [
+    "vllm>=0.32.0,<0.33",
+    "lmcache>=0.5.6,<0.6",
+    "sglang>=0.5.19,<0.6",
+    "nixl>=1.5.0,<2",
+]
+dev = [
+    "maturin>=1.7,<2",
+    "pytest>=8,<9",
+    "pytest-cov>=5",
+    "moto>=5,<6",
+    "Cython",
+    "openai",
+]
+
+[project.scripts]
+lifeboat-cache = "lifeboat_cache.config.cli:main"
+lifeboat-controld = "lifeboat_cache.control.controld:main"
+
+# Loaded automatically by the engines in every process once the package is installed.
+[project.entry-points."vllm.general_plugins"]
+lifeboat = "lifeboat_cache.integrations.vllm_plugin:register"
+
+[project.entry-points."sglang.srt.plugins"]
+lifeboat = "lifeboat_cache.integrations.sglang_plugin:register"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/lifeboat_cache"]
+
+[tool.hatch.build.targets.wheel.force-include]
+"presets" = "lifeboat_cache/presets"
+
+[tool.hatch.build.targets.sdist]
+include = ["src/lifeboat_cache", "presets", "README.md", "pyproject.toml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]
+markers = [
+    "requires_gpu: needs a CUDA/ROCm GPU (skipped on Mac)",
+    "requires_docker: needs docker/docker-compose",
+    "requires_cluster: needs a multi-node cluster",
+]

lifeboatcache-0.1.0/src/lifeboat_cache/__init__.py

@@ -0,0 +1,7 @@
+"""Lifeboat Cache — commercial LMCache plugin (L2 storage, serde, policies).
+
+See the project brief / docs/ for architecture. This package is import-safe on a
+CPU-only Mac (the "fleet of size one") and on GPU fleets alike.
+"""
+
+__version__ = "0.0.0"

lifeboatcache-0.1.0/src/lifeboat_cache/compress/__init__.py

@@ -0,0 +1,57 @@
+"""CPU reference compression layer for Lifeboat Cache.
+
+A set of KV-cache codecs behind one :class:`Codec` interface, driven by the
+``compression`` config section. Every codec is pure-numpy (torch optional) and produces a
+**self-describing blob** so ``decode`` needs no external arguments.
+
+Codecs (each approximates a published method; see the module docstrings for exactly what is
+reproduced vs. approximated):
+
+* ``turboquant`` — group-wise affine int3/int4 with float16-kept layers.
+* ``fp8``        — e4m3 / e5m2 emulation via an exact FP8 value grid.
+* ``oscar``      — 2-bit middle + full-precision sink/recent tokens + optional rotation.
+* ``kvtc``       — PCA + quantized coefficients/residual + zlib (ANS stand-in).
+
+Importing this package registers all four codec factories, so
+:func:`build_codecs` can instantiate any ladder from config.
+"""
+from __future__ import annotations
+
+from .accuracy import cosine, logit_kl, mse, tensor_relative_error
+from .base import (
+    Codec,
+    build_codecs,
+    create_codec,
+    pack_arrays,
+    pack_blob,
+    register_codec,
+    unpack_arrays,
+    unpack_blob,
+)
+from .calibration import CalibrationProfile
+from .demote import recompress
+from .fp8 import FP8Codec
+from .kvtc import KVTCCodec
+from .oscar2 import Oscar2Codec
+from .turboquant import TurboQuantCodec
+
+__all__ = [
+    "Codec",
+    "register_codec",
+    "create_codec",
+    "build_codecs",
+    "pack_blob",
+    "unpack_blob",
+    "pack_arrays",
+    "unpack_arrays",
+    "TurboQuantCodec",
+    "FP8Codec",
+    "Oscar2Codec",
+    "KVTCCodec",
+    "CalibrationProfile",
+    "recompress",
+    "logit_kl",
+    "cosine",
+    "mse",
+    "tensor_relative_error",
+]

lifeboatcache-0.1.0/src/lifeboat_cache/compress/accuracy.py

@@ -0,0 +1,53 @@
+"""Accuracy metrics for the compression accuracy-guard.
+
+The guard samples a fraction of writes, decodes them, and compares against the original to
+decide whether a codec is degrading quality (config ``compression.accuracy_guard.metric``).
+The headline metric is softmax KL on logits — the distributional distance that actually
+matters for generation quality — plus cheaper tensor-space helpers.
+"""
+from __future__ import annotations
+
+import numpy as np
+
+_EPS = 1e-12
+
+
+def _log_softmax(z: np.ndarray) -> np.ndarray:
+    z = z - z.max(axis=-1, keepdims=True)
+    return z - np.log(np.exp(z).sum(axis=-1, keepdims=True) + _EPS)
+
+
+def logit_kl(p_logits: np.ndarray, q_logits: np.ndarray) -> float:
+    """Mean KL(softmax(p) || softmax(q)) over the last axis.
+
+    Zero iff the two logit tensors induce the same distribution (identical logits, or logits
+    differing only by a per-row additive constant), strictly positive otherwise. Computed in
+    log-space for numerical stability.
+    """
+    lp = _log_softmax(np.asarray(p_logits, dtype=np.float64))
+    lq = _log_softmax(np.asarray(q_logits, dtype=np.float64))
+    p = np.exp(lp)
+    kl = (p * (lp - lq)).sum(axis=-1)
+    return float(np.maximum(kl, 0.0).mean())
+
+
+def cosine(a: np.ndarray, b: np.ndarray) -> float:
+    """Mean cosine similarity in [-1, 1] over flattened vectors (1.0 == identical)."""
+    a = np.asarray(a, dtype=np.float64).reshape(-1)
+    b = np.asarray(b, dtype=np.float64).reshape(-1)
+    denom = (np.linalg.norm(a) * np.linalg.norm(b)) + _EPS
+    return float(np.dot(a, b) / denom)
+
+
+def mse(a: np.ndarray, b: np.ndarray) -> float:
+    """Mean squared error between two arrays."""
+    a = np.asarray(a, dtype=np.float64)
+    b = np.asarray(b, dtype=np.float64)
+    return float(np.mean((a - b) ** 2))
+
+
+def tensor_relative_error(a: np.ndarray, b: np.ndarray) -> float:
+    """Relative reconstruction error ``||a - b|| / (||a|| + eps)`` (Frobenius norm)."""
+    a = np.asarray(a, dtype=np.float64)
+    b = np.asarray(b, dtype=np.float64)
+    return float(np.linalg.norm(a - b) / (np.linalg.norm(a) + _EPS))