gridbook 0.1.0__tar.gz

gridbook-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Robert Tand
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

gridbook-0.1.0/MANIFEST.in

@@ -0,0 +1,33 @@
+# Sdist contents beyond the package itself. Release artifacts are built from
+# THIS tree (the public repo), so the sdist gets the full public README, the
+# docs/ set and the roadmap; the in-tree plugin has none of those.
+#
+# The runtime CUDA sources need no rule here: they live inside the package
+# (gridbook/csrc) and are declared as [tool.setuptools.package-data] in
+# pyproject.toml, which setuptools also honours when building the sdist.
+#
+# LICENSE / NOTICE need no rule either: setuptools' default `license-files`
+# globs (LICEN[CS]E*, COPYING*, NOTICE*, AUTHORS*) put them in both the sdist
+# and the wheel's .dist-info/licenses/.
+include ROADMAP.md
+graft docs
+recursive-include tests *.py
+
+# ...but `graft docs` is indiscriminate, and two things under docs/ are internal
+# working material, not documentation for someone who ran `pip install gridbook`.
+# A published sdist is permanent (a PyPI file cannot be re-uploaded), so these
+# are excluded at the source rather than trimmed later:
+#
+#   docs/DISTRIBUTION.md  the distribution decision record -- internal machine
+#                         paths, internal task numbers, and a not-yet-executed
+#                         launch plan. Reaches the public via the GitHub repo,
+#                         which is a revisable surface; it has no business in an
+#                         immutable release artifact.
+#   docs/hf-cards/        staged HuggingFace card edits that have not been
+#                         applied yet, each carrying its own APPLY GATE. Shipping
+#                         un-applied drafts inside the wheel's own sdist invites
+#                         someone to read them as the current cards.
+exclude docs/DISTRIBUTION.md
+prune docs/hf-cards
+
+global-exclude *.py[cod]

gridbook-0.1.0/PKG-INFO

@@ -0,0 +1,353 @@
+Metadata-Version: 2.4
+Name: gridbook
+Version: 0.1.0
+Summary: vLLM plugin for the NVFP4-CB / FP8-CB product-codebook weight formats: 2-6 bit-per-weight LLM quantization served by dedicated CUDA decode and prefill kernels.
+Author-email: Robert Tand <robert.tand@icloud.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/RobTand/gridbook
+Project-URL: Repository, https://github.com/RobTand/gridbook
+Project-URL: Issues, https://github.com/RobTand/gridbook/issues
+Project-URL: Documentation, https://github.com/RobTand/gridbook/blob/master/docs/SPEC.md
+Keywords: vllm,quantization,llm,inference,cuda,nvfp4,fp8,codebook,vector-quantization,blackwell
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: GPU :: NVIDIA CUDA
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch
+Requires-Dist: triton
+Requires-Dist: safetensors
+Requires-Dist: huggingface-hub
+Provides-Extra: serve
+Requires-Dist: vllm; extra == "serve"
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
+
+# gridbook
+
+[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](docs/INSTALL.md)
+[![Status](https://img.shields.io/badge/status-beta-orange.svg)](ROADMAP.md)
+
+**Codebook-quantized LLM weights that serve at native tensor-core speed.**
+`gridbook` is a **format specification** plus an **out-of-tree vLLM plugin** that
+serves it: weights are stored as product-vector-quantization codebook indices
+whose codebook entries lie exactly on a hardware grid (NVFP4 / FP8), so a decoded
+tile *is* a native tile and feeds the same Blackwell tensor-core GEMM vLLM
+already runs. No forked runtime, no patched vLLM core, no custom serving stack.
+
+Ready-to-serve models are published on Hugging Face — [jump to the
+list](#published-artifacts).
+
+---
+
+## What you need
+
+| | Requirement |
+|---|---|
+| **GPU** | NVIDIA **Blackwell `sm_120` / `sm_121`** for the native-speed path (GB10 / DGX Spark is the reference; RTX 5090 is user-reported). Older NVIDIA cards partly work — read the [compatibility table](#compatibility) before assuming. |
+| **CUDA toolchain** | `nvcc` on `PATH` **in the process that serves**, matching your torch build. Kernels are **JIT-compiled on first model load** (~30 s once, then cached), *not* at `pip install` time. `nvcc` 13.0 is the tested toolchain. |
+| **PyTorch** | whatever build your vLLM uses (measured: `2.11.0+cu130`). |
+| **vLLM** | already installed — gridbook is a plugin, not a runtime. Measured against `0.23.1rc1.dev764+g54b16d8a9`; see [compatibility](#compatibility). |
+| **Parallelism** | **`tp=1` only.** The plugin has no tensor-parallel handling; multi-GPU sharding of CB weights is not implemented. |
+| **Python** | ≥ 3.10 (measured on 3.12). |
+
+Without `nvcc`, or on a non-Blackwell GPU, the plugin still loads and produces
+**correct** output through fallback kernels — but they are a correctness path,
+not a speed path. For dense decode the fallback is unchanged code and the gap is
+measured: the Triton decode-GEMM did **4.20 tok/s** on the 27B where the CUDA
+GEMV does **10.28**. For MoE the grouped CUDA GEMV is skipped entirely and the
+regression is larger but artifact-dependent — that is a *historical*
+before/after, not a fresh benchmark of today's fallback, and
+[`BENCHMARKS.md`](docs/BENCHMARKS.md#what-the-fallback-costs) says exactly what
+was and was not measured. See
+[TROUBLESHOOTING](docs/TROUBLESHOOTING.md#the-cuda-extension-did-not-load-triton-fallback)
+for how to tell which path you are on.
+
+---
+
+## Install
+
+Install **into the environment that already has vLLM and torch** — gridbook does
+not pin either.
+
+```bash
+pip install git+https://github.com/RobTand/gridbook
+```
+
+That is the only route that works today. `pip install gridbook` is **planned but
+not yet published** — the PyPI name is unclaimed, so that command fails right
+now. Track it on the [roadmap](ROADMAP.md#distribution).
+
+There is nothing else to configure: the plugin registers itself through vLLM's
+`vllm.general_plugins` entry point, and vLLM auto-detects a gridbook checkpoint
+from its `config.json`. No `--quantization` flag.
+
+Full matrix — toolchain versions, the three install routes, the first-run JIT
+build, how to persist the build cache, and a copy-pasteable install check:
+**[`docs/INSTALL.md`](docs/INSTALL.md)**. For the container route — a
+[`Dockerfile`](Dockerfile) that builds vLLM + gridbook + the kernels in one
+image — see **[`docs/CONTAINER.md`](docs/CONTAINER.md)**.
+
+---
+
+## Quickstart — serve a real model
+
+```bash
+pip install git+https://github.com/RobTand/gridbook
+
+vllm serve rdtand/Qwen3.6-27B-prismaquant-gridbook-5.5bit-vllm \
+  --host 0.0.0.0 --port 8000 \
+  --max-model-len 32768 \
+  --gpu-memory-utilization 0.85 \
+  --enforce-eager
+```
+
+```bash
+curl -s http://localhost:8000/v1/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"model": "rdtand/Qwen3.6-27B-prismaquant-gridbook-5.5bit-vllm",
+       "prompt": "The capital of France is", "max_tokens": 8, "temperature": 0}'
+```
+
+**How to tell it is actually working.** The failure is silent by design (the
+plugin fail-softs to Triton rather than refusing to serve), so check for the
+warning rather than for a crash:
+
+- **Bad** — a `[prismaquant-cb] WARNING:` line on stderr saying the CUDA
+  decode-GEMV extension could not be built means you are on the slow path.
+  (Grep your vLLM log for `[prismaquant-cb]`, not for `gridbook` — the runtime
+  log prefix still uses the project's older name.)
+- **Good** — no such line, and the first model load pauses ~30 s the first time
+  ever on that machine (the JIT build) and not on subsequent starts.
+- A definitive check that does not require serving anything is in
+  [`docs/INSTALL.md`](docs/INSTALL.md#verify-the-install).
+
+Notes:
+
+- **`--enforce-eager` is the tested configuration.** The decode dispatch branches
+  on the token count on the host, which is capture-hostile; naive graph capture
+  made decode *worse*, because vLLM pads captured decode batches past the
+  prefill threshold and every graphed step then took the prefill path. Details
+  and the exceptions: [`KERNELS.md`](docs/KERNELS.md#cuda-graph-safety-rules).
+- **`--gpu-memory-utilization 0.85` is a deliberately conservative start**, not a
+  tuned value. On a unified-memory box (GB10 / DGX Spark) that flag does not
+  carve up private VRAM — it claims from the pool the host also lives in, and
+  0.94/0.95 killed the whole machine under long-prefill spikes. Bring the server
+  up, check several GiB of `MemAvailable` remain, then raise it. On a discrete
+  GPU you can generally go higher. Details:
+  [`INSTALL.md`](docs/INSTALL.md#per-artifact-requirements) and
+  [`TROUBLESHOOTING.md`](docs/TROUBLESHOOTING.md#out-of-memory--box-hang-on-a-dgx-spark).
+- Set `--max-model-len` to fit your KV budget. Adding `--kv-cache-dtype fp8`
+  roughly halves KV bytes and is what the larger artifacts were measured with.
+- Serving by Hub id (as above) or from a local directory both work — the
+  codebook sidecar is fetched from the Hub when you pass a repo id.
+
+---
+
+## Published artifacts
+
+All are standard Hugging Face directories (`config.json` + safetensors + the
+`cb_codebooks.pqcb` codebook sidecar + tokenizer). Sizes are the sum of repo
+files, read from the HF API on 2026-07-28.
+
+| Model | Repo | Size on disk | bpp | Notes |
+|---|---|---|---|---|
+| Qwen3.6-27B (dense hybrid VL) | [`rdtand/Qwen3.6-27B-prismaquant-gridbook-5.5bit-vllm`](https://huggingface.co/rdtand/Qwen3.6-27B-prismaquant-gridbook-5.5bit-vllm) | 23.0 GB (21.4 GiB) | 5.5 | The measured quality result below. Vision tower is stock NVFP4 W4A16, so it needs a vLLM NVFP4 backend as well as gridbook. |
+| Laguna-S-2.1 (117B sparse MoE, coding) | [`rdtand/Laguna-S-2.1-prismaquant-gridbook-6bit-vllm`](https://huggingface.co/rdtand/Laguna-S-2.1-prismaquant-gridbook-6bit-vllm) | 89.4 GB (83.2 GiB) | 6.0 | 256k context on one 128 GB box. **No quality claims** (no servable BF16 teacher at this scale). |
+| Hy3-295B-A21B (295B MoE) | [`rdtand/Hy3-295B-A21B-prismaquant-gridbook-2.9bit-vllm`](https://huggingface.co/rdtand/Hy3-295B-A21B-prismaquant-gridbook-2.9bit-vllm) | 105.7 GB (98.5 GiB) | 2.9 | Serves on a **single** 128 GB box. **No quality-vs-teacher claim** at this scale. |
+
+Each artifact needs enough memory for its weights **plus** KV cache and
+activation headroom; see [`docs/INSTALL.md`](docs/INSTALL.md#per-artifact-requirements).
+
+---
+
+## Compatibility
+
+The native path is Blackwell-targeted, but the decode kernel is *not*
+arch-specific — this table separates what was **measured** from what is
+**inferred from the code and untested**. Nothing here is inferred silently.
+
+| GPU class | Decode (M ≤ 16) | Dense FP8-CB prefill | Mid-M fused prefill (16 < M ≤ 128) | Verdict |
+|---|---|---|---|---|
+| **GB10 / DGX Spark, `sm_121`** | CUDA GEMV | expand → stock W8A8 GEMM | CUTLASS `sm120` | **MEASURED.** Every published number comes from this box. |
+| **RTX 5090, `sm_120`** | same | same | same | **USER-REPORTED** working (vLLM 0.25.1, 27B artifact, [issue #1](https://github.com/RobTand/gridbook/issues/1)) after that issue's memory patches. No speed numbers published; the exact `nvcc` arch flag (`sm_120` vs the measured `sm_121`) is untested here. |
+| **H100 `sm_90`, RTX 4090 / L40S `sm_89`** | expected to work — the decode kernel contains no arch guards, no inline PTX and no `-arch` flag | expected to work (needs `sm_89+` fp8 support) | **fails, fail-softs** to the expand path with a warning (the fused kernel is `sm_120`-family only) | **INFERRED, UNTESTED.** Also gated by whatever the artifact's non-CB groups need (e.g. the 27B vision tower's NVFP4). |
+| **A100 `sm_80`** | expected to work | **expected to fail**: the dense FP8-CB prefill calls `cutlass_scaled_mm` on fp8 with no capability guard and no fallback | n/a | **NOT RECOMMENDED.** The declared capability floor is 8.0, but a prompt longer than 16 tokens is expected to error. INFERRED from code; no A100 was tested. |
+| **No `nvcc`, or non-NVIDIA** | Triton fallback | Triton fallback | n/a | **Correct, slow.** Use for numerics verification and CI, not for serving. |
+
+Tested software stack (verified in-container, 2026-07-28): vLLM
+`0.23.1rc1.dev764+g54b16d8a9`, torch `2.11.0+cu130`, triton `3.6.0`,
+transformers `5.13.0`, CUDA `13.0.88`, Python `3.12.3`, **arm64**. vLLM `0.25.1`
+and `0.23.1rc1.dev1060` are user-reported working on x86 + RTX 5090 (issue #1).
+Everything else is untested. See
+[`docs/INSTALL.md`](docs/INSTALL.md#tested-software-stack) for how versions can
+drift and what that looks like.
+
+---
+
+## Why this exists
+
+Below ~4 bits per weight, single-format scalar quantization loses quality fast,
+and the methods that recover it (learned codebooks / vector quantization) have
+historically been **slow to serve**: a VQ weight must be *decoded* before it can
+be multiplied, and that decode usually runs on general CUDA cores, not tensor
+cores.
+
+Concretely, in the `llama.cpp` / GGUF ecosystem — today's most common low-bit
+serving path:
+
+- **k-quant** formats have a good fused CUDA matmul path (MMQ), so they serve
+  reasonably fast, but their quality rungs stop around 3–4 bits.
+- **IQ** formats (the sub-3-bit "I-quant" rungs) reach much better quality, but
+  their decode falls to a **slower CUDA-core dequant path** — a *prefill tax* you
+  pay on every prompt token.
+
+`gridbook` targets **IQ-class quality at native-kernel speed**. The dtype in each
+format name is the **grid the codebook's values live on, not the storage width**:
+storage is a k-bit index per 8-weight vector, so `FP8_CB_K32` is a 4.0-bit/weight
+encoding whose reconstructed values are exact fp8 numbers, and `NVFP4_CB_K16` is
+a ~2.3-bit/weight encoding reconstructing exact NVFP4 codes. Because a decoded CB
+tile *is* an NVFP4/FP8 tile:
+
+1. **Decode (batch-1 / small M):** a bandwidth-bound fused GEMV streams the
+   packed indices, expands per group in registers, and multiplies — fewer bytes
+   per weight means *less* memory traffic than a wider format, so decode is at or
+   above native-format speed.
+2. **Prefill (large M):** the layer's weight tile is expanded **transiently** into
+   a scratch buffer (bounded — never a resident dense weight), then run through
+   the stock native GEMM.
+
+Formats mix per-Linear with plain NVFP4, FP8 and BF16 inside one standard
+`safetensors` checkpoint. See [`docs/MOTIVATION.md`](docs/MOTIVATION.md) for the
+full argument and [`docs/KERNELS.md`](docs/KERNELS.md) for the kernel design.
+
+---
+
+## Headline results
+
+Quality numbers are **at matched bits-per-weight** against a baseline quantized
+to stock per-Linear NVFP4/FP8 (served natively by vLLM's `compressed-tensors`
+path). Measured on a single **NVIDIA GB10 / DGX Spark** (Blackwell `sm_121`,
+128 GB unified memory, ~273 GB/s), single calibration seed. Read the caveats in
+[`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) — especially the ±17% measurement-
+arithmetic band and the single-seed noise band.
+
+| Model | bpp | Quality vs matched-bpp NVFP4/FP8 baseline | Serving speed |
+|---|---|---|---|
+| **27B dense hybrid** | 5.5 | **ALL-KL −58.3%**, confident-KL −52.9%, PPL gap to BF16 ~3× smaller | decode **at native parity** (10.3 vs 10.26 tok/s); dense prefill 1.44× the baseline's TTFT |
+| **35B MoE**, 256 experts (**not published**) | 4.75 | **confident-KL −53%**, ALL-KL −43%, top-1 agreement higher on both slices | decode 32.6–33.3 tok/s — **faster than BF16** (28.4) |
+| **295B MoE** (published) | 2.9 | *no quality-vs-teacher claim at this scale* | **serves on ONE DGX Spark**; prefill **~109 tok/s vs the matched-byte GGUF IQ build's 42 = ~2.6×**; base decode 14.6 tok/s (16.1 with the MTP draft) — below the GGUF build's ~18 |
+
+- **The 27B row is the A/B build, not the published file — and the published
+  file's own card reports different absolute numbers.** The −58.3% comes from
+  the 2026-07-18 head-to-head, whose CB arm used a 4-rung menu
+  (`FP8_CB_K36/K40/K44/K48`). The artifact actually on the Hub is a later,
+  finer 8-rung ladder (`K36`–`K47`), measured in a **different** evaluation
+  session, and its [model
+  card](https://huggingface.co/rdtand/Qwen3.6-27B-prismaquant-gridbook-5.5bit-vllm)
+  reports ALL-KL **0.0049** and **−77%** where this table's session reads
+  0.0134 and −58.3%. Both are real; **absolute KL is not comparable across
+  sessions** (the *same unchanged* NVFP4/FP8 baseline artifact reads 0.02407
+  confident-KL in one session and 0.01302 in the other), while the within-session
+  relative verdict — CB wins big at matched bytes — reproduces in both. The
+  arithmetic is worked through in
+  [`BENCHMARKS.md`](docs/BENCHMARKS.md#two-sessions-two-builds-why-the-model-card-says-77-and-this-page-says-583).
+  This page quotes the more conservative figure.
+- The 35B MoE result is an internal measurement; **there is no downloadable 35B
+  gridbook artifact**. It is reported because it is the evidence that the format
+  win reproduces on Mixture-of-Experts, not as something you can reproduce
+  without an encoder.
+- The 295B result carries **no quality-vs-teacher claim**: a 295B BF16 reference
+  cannot be run on a single box to compute KL against. What is validated is that
+  it loads, serves, and generates coherent, arithmetically-correct output on one
+  128 GB box, and its speed against a matched-byte GGUF build. Tool-use
+  (ToolEvalBench) is **parity within the plateau band**, not a quality win: the
+  shipped artifact scored 88/100 (130/148) on the ship config, against 87 for the
+  GGUF IQ build and 86 for GGUF k-quant, with a measured single-seed churn band of
+  ±2–3 points across serving configs.
+- **MoE prefill is solved and on by default.** Measured on Laguna-S-2.1 (117B
+  MoE): 293 → **1,821 tok/s** at 8k and 207 → **1,822 tok/s** at 63k after the
+  CUDA chunk-expander landed (commit `8829c16`); the current default is a
+  measured per-layer path selection. Docs that still describe MoE prefill as a
+  per-expert loop are stale — see [`ROADMAP.md`](ROADMAP.md) for what is actually
+  open.
+- **Large-M dense prefill is the honest remaining gap**: ~1.44× the native GEMM's
+  TTFT on the 27B.
+
+---
+
+## Documentation
+
+| Doc | What it covers |
+|---|---|
+| [`docs/INSTALL.md`](docs/INSTALL.md) | Hardware/toolchain matrix, the three install routes, first-run JIT build, cache persistence, install verification. |
+| [`docs/TROUBLESHOOTING.md`](docs/TROUBLESHOOTING.md) | Real failure modes keyed to the exact message you see, with cause and fix. |
+| [`docs/SPEC.md`](docs/SPEC.md) | **Normative format specification** — byte layout, product-VQ, v1/v2 scale coding, codebook sidecar, config vocabulary, extensibility contract. Implementation-independent, MUST/SHOULD language. |
+| [`docs/MOTIVATION.md`](docs/MOTIVATION.md) | Why the 2–4 bpp regime needs codebooks, why serving has been the blocker, and an honest comparison to GGUF k-quant/IQ. |
+| [`docs/KERNELS.md`](docs/KERNELS.md) | Serving kernel design: transient-expand prefill, fused act-QDQ decode GEMV, two-tier in-register scale compose, grouped MoE GEMV, Triton fallbacks, CUDA-graph rules. |
+| [`docs/PLUGIN.md`](docs/PLUGIN.md) | Operator reference for the plugin itself: dispatch, kernel set, environment switches. |
+| [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | The measured results with full hardware/protocol context and caveats. |
+| [`ROADMAP.md`](ROADMAP.md) | What is done, what is open, and what was measured and rejected. |
+| [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to run the tests, what a good bug report contains, and what is in scope. |
+
+---
+
+## Naming
+
+One project, several strings — this trips people up when grepping logs, so here
+is the whole list:
+
+| Where | String |
+|---|---|
+| Repo / PyPI package / Python import | `gridbook` |
+| vLLM quantization registry key | `"gridbook"` — what every published artifact carries in `config.json` |
+| Legacy registry key (still accepted) | `"prismaquant"` — artifacts exported before the rename |
+| Custom-op namespace | `prismaquant::` |
+| Environment variables | `PRISMAQUANT_*` |
+| Log/warning prefix | `[prismaquant-cb]` |
+
+The `prismaquant`-prefixed runtime names are kept because renaming them would
+break compatibility with already-published artifacts and with in-flight A/B
+tooling; a rename of the *non*-load-bearing ones (build-cache directory, log
+prefix) is a pre-1.0 item on the [roadmap](ROADMAP.md). **PrismaQuant** is the
+separate research pipeline that *produces* gridbook artifacts —
+[github.com/RobTand/prismaquant](https://github.com/RobTand/prismaquant).
+
+---
+
+## Scope and honesty notes
+
+- The plugin **does not patch vLLM core**. It does install a thin `load_weights`
+  wrapper on specific *model classes* (HunYuan-V3, Laguna, Qwen3.5-MoE and their
+  MTP drafters) whose loaders map MoE experts at the top level and would
+  otherwise not recognise stacked codebook expert tensors. That wrap is inert for
+  non-CB checkpoints. Nothing in `vllm/` is modified.
+- This repository **serves** the format; it does not yet **produce** artifacts.
+  A reference encoder is a roadmap item — until then, artifacts come from the
+  authors' pipeline, and the [spec](docs/SPEC.md) is published so an encoder can
+  be written independently.
+- Every number in this repo comes from a measurement on the hardware named beside
+  it. Where something is inferred rather than measured, it says so.
+
+## License
+
+Apache License 2.0 — see [`LICENSE`](LICENSE). The format specification in
+[`docs/SPEC.md`](docs/SPEC.md) is published under the same license: the format is
+free, open, and intended to be implemented by anyone, in any runtime, without
+permission.
+
+## Citation and attribution
+
+Robert Tand — <robert.tand@icloud.com>. Machine-readable metadata in
+[`CITATION.cff`](CITATION.cff).