vllm-kvblockd 0.1.0__tar.gz

vllm_kvblockd-0.1.0/.gitignore

@@ -0,0 +1,56 @@
+# ── PRIVATE RESEARCH VAULT — never commit ─────────────────────────
+# All competitive research, strategy, business plans, drafts, and course material.
+research/
+
+# OS noise
+.DS_Store
+
+# Build artifacts
+bin/
+dist/
+*.prof
+*.test
+testdata/corpus-large/
+
+# External build dependencies fetched at build time (never vendored):
+# ai-dynamo/nixl checkout for the adapters/nixl plugin (see its README).
+build-deps/
+adapters/nixl/build/
+
+# Local env
+.env
+
+# Soak ARTIFACTS (logs, JSONL, pprof snapshots) — never the driver source
+/soak
+/soak.jsonl
+bench/rigs/aws-soak/results/
+bench/rigs/aws-dramgate/allocs-*.pb.gz
+
+# Claude session rules — private, never push
+CLAUDE.md
+
+# Compiled command binaries accidentally built into the tree
+/xferspike
+/kvblockd
+/kvbctl
+cmd/xferspike/xferspike
+
+# Python
+__pycache__/
+*.pyc
+.pytest_cache/
+.ruff_cache/
+*.egg-info/
+python/**/dist/
+
+# mutation-testing artifacts
+report.json
+bench/rigs/*/.rig-state
+
+# locally built bench binaries
+/kvbench
+/crash
+/getbench
+internal/store/modeltest/testdata/rapid/
+bench/rigs/aws-s3soak/results/
+.claude/

vllm_kvblockd-0.1.0/DEFER.md

@@ -0,0 +1,31 @@
+# DEFERRED: GPU end-to-end for the SecondaryTierManager altitude
+
+`tier_manager.py` (KvblockdTierManager under vLLM's `OffloadingConnector` +
+`TieringOffloadingSpec`) is **code-complete and unit-tested** against the
+pinned v0.25.0 contract (UPSTREAM.lock): synthetic primary memoryview,
+hand-built `JobMetadata`, byte-exact round-trips, RETRY→HIT lookup
+transitions, failure jobs, and `drain_jobs` semantics — all green with a real
+kvblockd daemon on the other end of the wire.
+
+What is NOT yet proven: the `OffloadingConnector` path end-to-end on a real
+GPU. That path cannot run on the vLLM CPU backend (it is CUDA/ROCm/XPU only),
+so the GPU e2e is deferred rather than faked. There is no silently-skipped
+green: the CPU e2e gate is carried by `connector.py` alone
+(`.github/workflows/vllm-native-cpu.yml`).
+
+## Revisit trigger
+
+Run the GPU validation session before any release or announcement that claims
+the tier-manager (`OffloadingConnector`) path — the claim must not outrun the
+evidence.
+
+## Validation checklist for that session
+
+- vLLM v0.25.x + `vllm-kvblockd` installed; kvblockd daemon co-located.
+- `OffloadingConnector` + `TieringOffloadingSpec` with the `kvblockd`
+  secondary tier; `kv_offload_benchmark.py`-style workload: 10k unique
+  512-token requests, prefix caching off, Llama-3.1-8B (or a comparable
+  model that fits the available GPU).
+- PASS = non-zero secondary-tier hits, byte-correct outputs,
+  `drain_jobs` clean shutdown under load, TTFT before/after captured.
+- Result logged in `docs/INTEGRATIONS.md`.

vllm_kvblockd-0.1.0/PKG-INFO

@@ -0,0 +1,26 @@
+Metadata-Version: 2.4
+Name: vllm-kvblockd
+Version: 0.1.0
+Summary: vLLM native KV-transfer adapter backed by kvblockd (KVConnectorBase_V1 + SecondaryTierManager)
+Author-email: Rudray Mehra <rudraymehra@gmail.com>
+License-Expression: Apache-2.0
+Requires-Python: >=3.10
+Requires-Dist: blake3>=0.4
+Requires-Dist: kvblockd>=0.1.0
+Requires-Dist: numpy
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == 'test'
+Requires-Dist: torch; extra == 'test'
+Provides-Extra: vllm
+Requires-Dist: vllm<0.26,>=0.22; extra == 'vllm'
+Description-Content-Type: text/markdown
+
+# vllm-kvblockd
+
+Native vLLM KVConnector-v1 for [kvblockd](https://github.com/rudraymehra/KVStash) —
+the single-binary LLM KV-cache store (DRAM → NVMe → S3 over plain TCP).
+This connector produced every published TTFT number in the repo
+(receipts: `docs/BENCHMARKS.md`, `docs/CLAIMS.md`).
+
+Quick start: `docs/INTEGRATIONS.md` → "vLLM native connector".
+TP=1 only (refuses multi-GPU at boot); requires `PYTHONHASHSEED=0`.

vllm_kvblockd-0.1.0/README.md

@@ -0,0 +1,9 @@
+# vllm-kvblockd
+
+Native vLLM KVConnector-v1 for [kvblockd](https://github.com/rudraymehra/KVStash) —
+the single-binary LLM KV-cache store (DRAM → NVMe → S3 over plain TCP).
+This connector produced every published TTFT number in the repo
+(receipts: `docs/BENCHMARKS.md`, `docs/CLAIMS.md`).
+
+Quick start: `docs/INTEGRATIONS.md` → "vLLM native connector".
+TP=1 only (refuses multi-GPU at boot); requires `PYTHONHASHSEED=0`.

vllm_kvblockd-0.1.0/UPSTREAM.lock

@@ -0,0 +1,58 @@
+# UPSTREAM.lock — the exact vLLM sources this adapter was written against.
+#
+# Pinned: vllm-project/vllm tag v0.25.0
+#   commit 702f4814fe54fabff350d43cb753ae3e47c0c276  (tag object, fetched 2026-07-20)
+#
+# Files read at that tag (copies vendored under upstream_refs/, names
+# disambiguated because two of them are both called base.py):
+#
+#   vllm/distributed/kv_transfer/kv_connector/v1/base.py
+#       -> upstream_refs/kv_connector_v1_base.py        (KVConnectorBase_V1)
+#   vllm/distributed/kv_transfer/kv_connector/v1/example_connector.py
+#       -> upstream_refs/example_connector.py           (SharedStorageConnector was
+#          RENAMED to ExampleConnector before v0.25.0 — the model this
+#          connector's design follows)
+#   vllm/distributed/kv_transfer/kv_connector/factory.py
+#       -> upstream_refs/kv_connector_factory.py        (external load path +
+#          3-arg-constructor enforcement + HMA gate)
+#   vllm/v1/kv_offload/base.py
+#       -> upstream_refs/kv_offload_base.py             (OffloadKey, ReqContext,
+#          LookupResult, OffloadingSpec)
+#   vllm/v1/kv_offload/tiering/base.py
+#       -> upstream_refs/tiering_base.py                (SecondaryTierManager,
+#          JobMetadata, JobResult)
+#   vllm/v1/kv_offload/tiering/factory.py
+#       -> upstream_refs/tiering_factory.py             (SecondaryTierFactory)
+#   vllm/v1/kv_offload/tiering/async_lookup.py
+#       -> upstream_refs/tiering_async_lookup.py        (AsyncLookupManager — the
+#          model for our _AsyncExistsBatcher; we vendor the PATTERN, not the import)
+#   vllm/v1/kv_offload/tiering/fs/manager.py
+#       -> upstream_refs/tiering_fs_manager.py          (FileSystemTierManager — the
+#          model implementation: strides[0] block size, DualQueueThreadPool)
+#
+# Churn survey across the tested version window (verified against raw tags 2026-07-20):
+#   - KVConnectorBase_V1.__init__(vllm_config, role, kv_cache_config) — IDENTICAL
+#     3-arg signature at v0.22.1, v0.23.0, v0.24.0, v0.25.0. The connector factory
+#     at v0.25.0 additionally REFUSES external connectors without a
+#     kv_cache_config kwarg, so the 3-arg form is mandatory and safe window-wide.
+#   - vllm/v1/kv_offload/tiering/{base,factory}.py present with the same
+#     SecondaryTierManager.__init__(offloading_spec, primary_kv_view, tier_type)
+#     at v0.22.1 and v0.23.0 (and v0.25.0). No documented skip is currently
+#     needed in the CI version matrix; the import check still prints SKIP (exit 0) if a
+#     future/older release lacks the tiering module.
+#   - HMA gate (v0.25.0 factory): a connector that does not implement SupportsHMA
+#     cannot be created unless --disable-hybrid-kv-cache-manager is passed.
+#     KvblockdConnector does not implement SupportsHMA in v0.1 -> the CPU e2e
+#     serve line carries that flag (bench/e2e/vllm-native-cpu.sh).
+#   - RFC-vs-merged delta: the RFC-era tier config carried module_path/class_name
+#     INSIDE the tier dict. The MERGED v0.25.0 SecondaryTierFactory ignores
+#     those keys (registry-only resolution) and passes every non-"type" key to
+#     the tier constructor. Working out-of-tree loading goes through
+#     OffloadingSpecFactory: spec_name=KvblockdTieringSpec +
+#     spec_module_path=vllm_kvblockd.tier_manager (importing our module is what
+#     registers the "kvblockd" tier type). KvblockdTierManager still ACCEPTS
+#     and ignores module_path/class_name kwargs so RFC-shaped configs don't
+#     crash. The merged code is the contract, not the RFC.
+#
+# Re-verify this file's survey (and refresh upstream_refs/) before every vLLM
+# version bump — see the CHURN-WATCH header in src/vllm_kvblockd/connector.py.

vllm_kvblockd-0.1.0/pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "vllm-kvblockd"
+version = "0.1.0"
+description = "vLLM native KV-transfer adapter backed by kvblockd (KVConnectorBase_V1 + SecondaryTierManager)"
+requires-python = ">=3.10"
+readme = "README.md"
+license = "Apache-2.0"
+authors = [{ name = "Rudray Mehra", email = "rudraymehra@gmail.com" }]
+dependencies = [
+    "kvblockd>=0.1.0",
+    "numpy",  # JobMetadata.block_ids is an np.ndarray; also zero-copy tensor views
+    "blake3>=0.4",  # config.py imports it directly (key derivation core) — declare
+    # what we import, never lean on kvblockd's transitive pin (matching version)
+]
+
+[project.optional-dependencies]
+# vllm is NEVER a hard runtime dep: the plugin is loaded BY vllm
+# (kv_connector_module_path / spec_module_path), so a hard pin would make the
+# package uninstallable next to a vllm the resolver dislikes. The pin lives
+# here, as the tested window (the last 4 minor releases).
+vllm = ["vllm>=0.22,<0.26"]
+test = ["pytest>=8", "torch"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/vllm_kvblockd"]
+
+[tool.hatch.build.targets.sdist]
+# Vendored upstream reference copies are for contract review, not distribution.
+exclude = ["upstream_refs"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+# Vendored vLLM reference copies (UPSTREAM.lock) are upstream's style, not ours.
+extend-exclude = ["upstream_refs"]

vllm_kvblockd-0.1.0/src/vllm_kvblockd/__init__.py

@@ -0,0 +1,49 @@
+"""vllm-kvblockd — vLLM native adapter for the kvblockd KV-cache block store.
+
+Two integration altitudes, one package (see UPSTREAM.lock for the pinned
+vLLM contract):
+
+  CONNECTOR (CPU-capable, the e2e-gated one):
+    --kv-transfer-config '{"kv_connector": "KvblockdConnector",
+                           "kv_role": "kv_both",
+                           "kv_connector_module_path": "vllm_kvblockd.connector",
+                           "kv_connector_extra_config": {
+                             "kvblockd_endpoint": "kvblockd://127.0.0.1:9440",
+                             "kvblockd_namespace": "vllm",
+                             "kvblockd_token": "..."}}'
+    (v0.25 factory note: serve with --disable-hybrid-kv-cache-manager.)
+
+  SECONDARY TIER (GPU serving via OffloadingConnector; e2e deferred, DEFER.md):
+    --kv-transfer-config '{"kv_connector": "OffloadingConnector",
+                           "kv_role": "kv_both",
+                           "kv_connector_extra_config": {
+                             "cpu_bytes_to_use": 64000000000,
+                             "spec_name": "KvblockdTieringSpec",
+                             "spec_module_path": "vllm_kvblockd.tier_manager",
+                             "secondary_tiers": [{
+                               "type": "kvblockd",
+                               "endpoint": "kvblockd://127.0.0.1:9440",
+                               "namespace": "vllm",
+                               "token": "..."}]}}'
+    Importing vllm_kvblockd.tier_manager (which spec_module_path does) is
+    what registers the "kvblockd" tier type with SecondaryTierFactory.
+
+Both classes import and instantiate WITHOUT vllm installed (the CI
+import check relies on this); vllm is an optional extra, never a hard
+dependency — the plugin is loaded BY vllm.
+"""
+
+from vllm_kvblockd.config import AdapterConfig, block_chain_keys, chain_seed, fingerprint
+from vllm_kvblockd.connector import KvblockdConnector, KvblockdConnectorMetadata
+from vllm_kvblockd.tier_manager import KvblockdTierManager
+
+__all__ = [
+    "AdapterConfig",
+    "KvblockdConnector",
+    "KvblockdConnectorMetadata",
+    "KvblockdTierManager",
+    "block_chain_keys",
+    "chain_seed",
+    "fingerprint",
+]
+__version__ = "0.1.0"