unalloc 0.2.1__tar.gz → 0.2.3__tar.gz

{unalloc-0.2.1 → unalloc-0.2.3}/.gitignore

@@ -2,6 +2,7 @@ __pycache__/
 *.py[cod]
 .venv/
 venv/
+.venv-research/
 dist/
 build/
 *.egg-info/
@@ -16,3 +17,4 @@ build/
 .claude/settings.local.json
 .ipynb_checkpoints/
 .cache/
+.tmp/

unalloc-0.2.3/CHANGELOG.md

@@ -0,0 +1,63 @@
+# Changelog
+
+## 0.2.3 — 2026-09-21
+
+### Fixed
+- Figure 9's within-run spread is drawn as its own vertical bar rather than an error bar hung off the whole-run point. The old form had to clip whenever the whole-run share fell outside the window quartiles, which it does at three of the eight plotted points — the whole-run share is weighted by each window's traffic, so it need not lie inside the quartiles of its own windows.
+- §9 no longer presents the window statistics as a bootstrap confidence interval or as a bound on uncertainty. Consecutive windows of one run are not independent observations, so the paragraph now reports median, interquartile range and sign as descriptive statistics of that run, and the cross-reference matches what the figure actually draws. `bootstrap_ci` is renamed `resample_range`, and the metrics key `divergence_mean_ci95` becomes `divergence_mean_resample_range`.
+- arXiv kit refreshed against the built PDF: 14 pages, AI-use disclosure on page 13, and the concept DOI in the Comments field. The standalone abstract gained the synthetic-scenario qualification and the session-initial arrival-rate clarification that the paper's abstract already carried.
+
+### Added
+- `tests/test_paper_numbers.py` also checks the submission kit: the page, figure and table counts against the committed PDF, the concept DOI in both the kit and the paper, and the standalone abstract's length and scope wording.
+
+## 0.2.2 — 2026-09-21
+
+### Fixed
+- Paper Table 3: the time-share entries at the 8 and 16 requests/s loads printed the raw shares while the caption promised overhead redistributed; they are 4.7% and 5.3%. The 12–14 point headline result is unaffected — the figure and the divergence metric already redistributed.
+- Paper: the configured load is now stated as the arrival rate of session-initial requests, distinguished throughout from completed throughput (3.7–26.9 requests/s), which includes the follow-up turns of multi-turn sessions.
+- Paper: "none waited in the queue" is now scoped to what was measured — no queued request in any half-second telemetry sample.
+- Paper: the CPU-versus-GPU gap is reported as a difference between two experiments rather than an effect with a measured cause; the fixed-overhead explanation is labelled a hypothesis, since model, software, hardware and workload all differ.
+- Paper: the abstract marks the 66% and 61% distribution results as a constructed scenario built on synthetic allocations.
+- Paper and README: OpenCost's OpenAI plugin and inference accounting are acknowledged, and the novelty claim narrowed to the implementation and the reproducible characterization of the failure modes. FOCUS added as related work.
+- Paper: Shapley is described as a reference for a specified cost function rather than a ground truth for rental bills; the Pope, DistServe, ABACUS and Cost-Governed RAG comparisons are bounded to what those works support; CPU collective fractions are no longer called an upper bound for NVLink GPUs.
+- Bibliography: official proceedings links, retrieval dates for mutable documentation, Shapley page range, and the ABACUS preprint dated to its 22 December 2024 submission.
+
+### Added
+- `bench.py --repeats n`: independent runs per load level, each with a fresh seed, with the load levels interleaved so session drift does not land on one of them. `analyze.py` groups repeats by rate and reports the median and range across them.
+- Within-run stability: both meters recomputed in ten-second windows of each run, with quartiles and a bootstrap confidence interval for the mean, warm-up window excluded. The published single runs now carry error bars in the GPU figure and a confidence interval in §9; none of the intervals approaches zero.
+- `tests/test_paper_numbers.py`: the GPU results table, its prose ranges and its scope claims are checked against `metrics.json`, so a stale table fails the build. This is the test that would have caught the redistribution error above.
+- `paper/REFERENCE-AUDIT.md`: the 23-reference source audit behind the corrections.
+
+## 0.2.1 — 2026-09-14
+
+### Changed
+- Paper prepared for arXiv: abstract leads with the attribution failures at the seams between ledgers and the H100 result; metering results are framed as disagreement between rules rather than over-charging, since none is a ground truth; new Related Work section (JouleShare, LLMVisor, PrefixShield, Cost-Governed RAG and others); defect catalogue moved to Appendix A; use of AI tools disclosed.
+- `paper/build.py` fails on Typst that compiles but renders wrong (`~`, `;` after a citation, unescaped dollar amounts).
+
+### Added
+- `case_studies/gpu_validation`: benchmark client, fake server, remote setup script, analysis and evidence renderer for validating the metering results with real vLLM on a GPU, plus a DigitalOcean runbook. First run: H100, vLLM 0.29.0, Qwen2.5-7B-Instruct, 6,241 requests, 0 errors.
+- Blog addendum with the verified GPU specs, results and bring-up/teardown evidence.
+- Paper §9, "Validation on a datacenter GPU", with the H100 results table and figure; abstract, methodology table and limitations updated to match.
+
+## 0.2.0 — 2026-09-13
+
+Validated against five inference case studies; see `paper/unalloc-case-studies.pdf`.
+
+### Fixed
+- OpenAI and Anthropic adapters lost their default API URLs whenever the CLI passed an unset environment variable, so every live fetch failed.
+- The Anthropic Admin API is now called with `x-api-key` and `anthropic-version` instead of a Bearer token.
+- Provider cost APIs are paged with `has_more` / `next_page`; previously only the first page was read.
+- OpenCost component costs are summed as `Decimal` when `totalCost` is absent, never through `float`.
+- Stacked label prefixes are all stripped (`label_app_kubernetes_io_name` → `name`).
+- Canonical-key collisions resolve by a fixed precedence instead of arrival order (explicit key, alias, rewritten key, path-derived key).
+- OpenCost annotations no longer override labels with the same key.
+- `unalloc labels` honours `--fallback`, and exits 1 when no rows load instead of printing "Nothing unallocated".
+
+### Added
+- `report --budget PCT` exits 2 when the unallocated share exceeds `PCT`, for CI gates.
+- Reports show the spend attributed only through a fallback key (`fallback_usd` in `--json`).
+- Case studies (`case_studies/`), a Typst paper (`paper/`), a ledger explorer, a companion notebook and a dev container with Postgres.
+
+## 0.1.0
+
+Initial release: OpenCost, LiteLLM, OpenAI and Anthropic adapters; `report`, `labels` and `reconcile`.

{unalloc-0.2.1 → unalloc-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: unalloc
-Version: 0.2.1
+Version: 0.2.3
 Summary: Find the AI spend nobody owns. Joins OpenCost allocation data with LLM provider bills and reports what's unattributed.
 Project-URL: Homepage, https://github.com/timurista/unalloc
 Project-URL: Issues, https://github.com/timurista/unalloc/issues
@@ -30,6 +30,7 @@ Requires-Dist: ruff>=0.6; extra == 'dev'
 Provides-Extra: research
 Requires-Dist: ipykernel>=6.29; extra == 'research'
 Requires-Dist: jupyterlab>=4.2; extra == 'research'
+Requires-Dist: markdown>=3.6; extra == 'research'
 Requires-Dist: matplotlib>=3.8; extra == 'research'
 Requires-Dist: nbclient>=0.10; extra == 'research'
 Requires-Dist: nbformat>=5.10; extra == 'research'
@@ -47,9 +48,12 @@ Description-Content-Type: text/markdown
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/timurista/unalloc/blob/main/LICENSE)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://github.com/timurista/unalloc/blob/main/pyproject.toml)
 [![Paper](https://img.shields.io/badge/paper-PDF-1f5b45.svg)](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)
-[![Explorer](https://img.shields.io/badge/explorer-live-1f5b45.svg)](https://timurista.github.io/unalloc/)
+[![Explorer](https://img.shields.io/badge/explorer-live-1f5b45.svg)](https://timurista.github.io/unalloc/explorer.html)
+[![DOI](https://zenodo.org/badge/1368528624.svg)](https://doi.org/10.5281/zenodo.22761012)
 
-OpenCost tells you what your Kubernetes workloads cost. Your provider dashboard tells you what your OpenAI and Anthropic calls cost. Neither tells you what a single AI feature costs, because the two live in different systems with different keys. `unalloc` pulls both into one normalized ledger, joins them on a label dimension you choose, and reports the number your finance team keeps asking for: how much of this month's AI spend can't be attributed to any team.
+**Project site: [timurista.github.io/unalloc](https://timurista.github.io/unalloc/)**, with the findings, case studies, [paper (PDF)](https://timurista.github.io/unalloc/paper/unalloc-case-studies.pdf), [blog post](https://timurista.github.io/unalloc/blog/who-pays-for-the-kv-cache.html), [GPU addendum](https://timurista.github.io/unalloc/blog/gpu-validation.html) and [ledger explorer](https://timurista.github.io/unalloc/explorer.html).
+
+OpenCost tells you what your Kubernetes workloads cost. Your provider dashboard tells you what your OpenAI and Anthropic calls cost. Getting both into one view is possible today — OpenCost has an OpenAI plugin — but the answer still depends on a join nobody checks: the same dollar can arrive through both your gateway and your provider bill, and the labels that say who owns it are set per workload, not per pod template. `unalloc` pulls all four sources into one normalized ledger, joins them on a label dimension you choose, and reports the number your finance team keeps asking for: how much of this month's AI spend can't be attributed to any team — and how much of the rest was only rescued by a fallback key.
 
 ## The output
 
@@ -189,7 +193,7 @@ Two design choices worth calling out:
 
 ## Case studies and paper
 
-Five studies push real (or realistically simulated) inference workloads through unalloc's own adapters. They exposed nine defects in the tool along the way, all fixed with regression tests, and they're written up as a paper: **[paper/unalloc-case-studies.pdf](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)**.
+Six studies push real (or realistically simulated) inference workloads through unalloc's own adapters, including a run with vLLM on an NVIDIA H100. They exposed nine defects in the tool along the way, all fixed with regression tests, and they're written up as a paper: **[paper/unalloc-case-studies.pdf](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)**. For the readable version, see the [blog post](https://timurista.github.io/unalloc/blog/who-pays-for-the-kv-cache.html) and the [GPU addendum](https://timurista.github.io/unalloc/blog/gpu-validation.html) on the [project site](https://timurista.github.io/unalloc/).
 
 | Study | What runs | Headline |
 | --- | --- | --- |
@@ -201,7 +205,8 @@ Five studies push real (or realistically simulated) inference workloads through
 | [`gpu_validation`](https://github.com/timurista/unalloc/blob/main/case_studies/gpu_validation) | Real vLLM 0.29 serving Qwen2.5-7B on a DigitalOcean H100, driven with the simulator's tenants ([runbook](https://github.com/timurista/unalloc/blob/main/case_studies/gpu_validation/RUNBOOK.md), [evidence](https://github.com/timurista/unalloc/tree/main/case_studies/results/gpu_validation/evidence)) | Token metering over-charges search by 12–14 points vs time share; GPU "utilization" reads 97% at 2 req/s and 99% at 16 req/s |
 
 ```bash
-make research            # CPU torch, notebook tooling, typst
+make research            # build ./.venv-research: CPU torch, notebook tooling, typst
+                         # (the targets below find it on their own; or pass PY=...)
 make case-studies        # all five, full size (a few minutes)
 make paper               # figures + PDF
 make notebook            # execute case_studies/unalloc_case_studies.ipynb
@@ -210,7 +215,7 @@ make ui                  # ledger explorer on http://127.0.0.1:8765
 
 ![Share of a shared vLLM pod's bill under five metering rules](https://raw.githubusercontent.com/timurista/unalloc/main/paper/figures/kv_shares.png)
 
-The **[ledger explorer](https://timurista.github.io/unalloc/)** (or `python -m case_studies.ui` locally) shows every dataset row by row: pick the label that means "owner", add fallbacks, and watch each row resolve to a team, a fallback value, or the backlog. `python -m case_studies.ui --export explorer.html` writes a self-contained copy you can share.
+The **[ledger explorer](https://timurista.github.io/unalloc/explorer.html)** (or `python -m case_studies.ui` locally) shows every dataset row by row: pick the label that means "owner", add fallbacks, and watch each row resolve to a team, a fallback value, or the backlog. `python -m case_studies.ui --export explorer.html` writes a self-contained copy you can share.
 
 ## Development environment
 
@@ -225,7 +230,9 @@ make devcontainer-check    # lint, tests and every case study at smoke size, ins
 
 OpenCost 1.121.0 added inference cost tracking for vLLM and llm-d deployments: cost per million tokens, KV-cache-corrected pricing, shared infrastructure attribution. That is excellent and `unalloc` does not reimplement it. Those rows arrive through the same `/allocation` shape and land in this ledger tagged `unalloc_layer=inference`.
 
-What OpenCost structurally cannot do is see your OpenAI and Anthropic bills, because they aren't Kubernetes objects. What LiteLLM cannot do is see your GPU nodes. `unalloc` lives in exactly that seam and nowhere else. It is:
+OpenCost can also bring external provider spend alongside Kubernetes cost through its plugin interface, including an OpenAI plugin announced in November 2024. So "Kubernetes cost and provider cost in one place" is not a gap `unalloc` invented, and it is not the claim here.
+
+What `unalloc` does is narrower: read all four sources yourself — OpenCost, a LiteLLM gateway, and the OpenAI and Anthropic billing APIs directly — and answer one question about the join. Which dollars carry no owner, which were rescued only by a fallback key that may be pointing at a Helm chart rather than a team, and which arrived twice because the gateway and the provider both reported them. That is the seam it lives in, and nowhere else. It is:
 
 - **not** a dashboard — it prints a number and exits (the explorer is a case-study companion, not part of the CLI)
 - **not** an optimizer — it makes no recommendations about rightsizing or commitments
@@ -250,6 +257,24 @@ pip install -e ".[dev]"
 ruff check . && pytest
 ```
 
+## Citing
+
+If you use unalloc or its case studies, please cite the archived release. GitHub's "Cite this repository" button produces the same entry from [`CITATION.cff`](https://github.com/timurista/unalloc/blob/main/CITATION.cff).
+
+- **All versions:** [doi:10.5281/zenodo.22761012](https://doi.org/10.5281/zenodo.22761012), which always resolves to the latest release — currently 0.2.3, the code the paper describes
+- **Version 0.2.1:** [doi:10.5281/zenodo.22761013](https://doi.org/10.5281/zenodo.22761013)
+
+```bibtex
+@software{urista_unalloc_2026,
+  author  = {Urista, Timothy},
+  title   = {unalloc: find the AI spend nobody owns},
+  version = {0.2.3},
+  year    = {2026},
+  doi     = {10.5281/zenodo.22761012},
+  url     = {https://github.com/timurista/unalloc}
+}
+```
+
 ## License
 
 Apache 2.0, matching OpenCost.

{unalloc-0.2.1 → unalloc-0.2.3}/README.md

@@ -6,9 +6,12 @@
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/timurista/unalloc/blob/main/LICENSE)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://github.com/timurista/unalloc/blob/main/pyproject.toml)
 [![Paper](https://img.shields.io/badge/paper-PDF-1f5b45.svg)](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)
-[![Explorer](https://img.shields.io/badge/explorer-live-1f5b45.svg)](https://timurista.github.io/unalloc/)
+[![Explorer](https://img.shields.io/badge/explorer-live-1f5b45.svg)](https://timurista.github.io/unalloc/explorer.html)
+[![DOI](https://zenodo.org/badge/1368528624.svg)](https://doi.org/10.5281/zenodo.22761012)
 
-OpenCost tells you what your Kubernetes workloads cost. Your provider dashboard tells you what your OpenAI and Anthropic calls cost. Neither tells you what a single AI feature costs, because the two live in different systems with different keys. `unalloc` pulls both into one normalized ledger, joins them on a label dimension you choose, and reports the number your finance team keeps asking for: how much of this month's AI spend can't be attributed to any team.
+**Project site: [timurista.github.io/unalloc](https://timurista.github.io/unalloc/)**, with the findings, case studies, [paper (PDF)](https://timurista.github.io/unalloc/paper/unalloc-case-studies.pdf), [blog post](https://timurista.github.io/unalloc/blog/who-pays-for-the-kv-cache.html), [GPU addendum](https://timurista.github.io/unalloc/blog/gpu-validation.html) and [ledger explorer](https://timurista.github.io/unalloc/explorer.html).
+
+OpenCost tells you what your Kubernetes workloads cost. Your provider dashboard tells you what your OpenAI and Anthropic calls cost. Getting both into one view is possible today — OpenCost has an OpenAI plugin — but the answer still depends on a join nobody checks: the same dollar can arrive through both your gateway and your provider bill, and the labels that say who owns it are set per workload, not per pod template. `unalloc` pulls all four sources into one normalized ledger, joins them on a label dimension you choose, and reports the number your finance team keeps asking for: how much of this month's AI spend can't be attributed to any team — and how much of the rest was only rescued by a fallback key.
 
 ## The output
 
@@ -148,7 +151,7 @@ Two design choices worth calling out:
 
 ## Case studies and paper
 
-Five studies push real (or realistically simulated) inference workloads through unalloc's own adapters. They exposed nine defects in the tool along the way, all fixed with regression tests, and they're written up as a paper: **[paper/unalloc-case-studies.pdf](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)**.
+Six studies push real (or realistically simulated) inference workloads through unalloc's own adapters, including a run with vLLM on an NVIDIA H100. They exposed nine defects in the tool along the way, all fixed with regression tests, and they're written up as a paper: **[paper/unalloc-case-studies.pdf](https://github.com/timurista/unalloc/blob/main/paper/unalloc-case-studies.pdf)**. For the readable version, see the [blog post](https://timurista.github.io/unalloc/blog/who-pays-for-the-kv-cache.html) and the [GPU addendum](https://timurista.github.io/unalloc/blog/gpu-validation.html) on the [project site](https://timurista.github.io/unalloc/).
 
 | Study | What runs | Headline |
 | --- | --- | --- |
@@ -160,7 +163,8 @@ Five studies push real (or realistically simulated) inference workloads through
 | [`gpu_validation`](https://github.com/timurista/unalloc/blob/main/case_studies/gpu_validation) | Real vLLM 0.29 serving Qwen2.5-7B on a DigitalOcean H100, driven with the simulator's tenants ([runbook](https://github.com/timurista/unalloc/blob/main/case_studies/gpu_validation/RUNBOOK.md), [evidence](https://github.com/timurista/unalloc/tree/main/case_studies/results/gpu_validation/evidence)) | Token metering over-charges search by 12–14 points vs time share; GPU "utilization" reads 97% at 2 req/s and 99% at 16 req/s |
 
 ```bash
-make research            # CPU torch, notebook tooling, typst
+make research            # build ./.venv-research: CPU torch, notebook tooling, typst
+                         # (the targets below find it on their own; or pass PY=...)
 make case-studies        # all five, full size (a few minutes)
 make paper               # figures + PDF
 make notebook            # execute case_studies/unalloc_case_studies.ipynb
@@ -169,7 +173,7 @@ make ui                  # ledger explorer on http://127.0.0.1:8765
 
 ![Share of a shared vLLM pod's bill under five metering rules](https://raw.githubusercontent.com/timurista/unalloc/main/paper/figures/kv_shares.png)
 
-The **[ledger explorer](https://timurista.github.io/unalloc/)** (or `python -m case_studies.ui` locally) shows every dataset row by row: pick the label that means "owner", add fallbacks, and watch each row resolve to a team, a fallback value, or the backlog. `python -m case_studies.ui --export explorer.html` writes a self-contained copy you can share.
+The **[ledger explorer](https://timurista.github.io/unalloc/explorer.html)** (or `python -m case_studies.ui` locally) shows every dataset row by row: pick the label that means "owner", add fallbacks, and watch each row resolve to a team, a fallback value, or the backlog. `python -m case_studies.ui --export explorer.html` writes a self-contained copy you can share.
 
 ## Development environment
 
@@ -184,7 +188,9 @@ make devcontainer-check    # lint, tests and every case study at smoke size, ins
 
 OpenCost 1.121.0 added inference cost tracking for vLLM and llm-d deployments: cost per million tokens, KV-cache-corrected pricing, shared infrastructure attribution. That is excellent and `unalloc` does not reimplement it. Those rows arrive through the same `/allocation` shape and land in this ledger tagged `unalloc_layer=inference`.
 
-What OpenCost structurally cannot do is see your OpenAI and Anthropic bills, because they aren't Kubernetes objects. What LiteLLM cannot do is see your GPU nodes. `unalloc` lives in exactly that seam and nowhere else. It is:
+OpenCost can also bring external provider spend alongside Kubernetes cost through its plugin interface, including an OpenAI plugin announced in November 2024. So "Kubernetes cost and provider cost in one place" is not a gap `unalloc` invented, and it is not the claim here.
+
+What `unalloc` does is narrower: read all four sources yourself — OpenCost, a LiteLLM gateway, and the OpenAI and Anthropic billing APIs directly — and answer one question about the join. Which dollars carry no owner, which were rescued only by a fallback key that may be pointing at a Helm chart rather than a team, and which arrived twice because the gateway and the provider both reported them. That is the seam it lives in, and nowhere else. It is:
 
 - **not** a dashboard — it prints a number and exits (the explorer is a case-study companion, not part of the CLI)
 - **not** an optimizer — it makes no recommendations about rightsizing or commitments
@@ -209,6 +215,24 @@ pip install -e ".[dev]"
 ruff check . && pytest
 ```
 
+## Citing
+
+If you use unalloc or its case studies, please cite the archived release. GitHub's "Cite this repository" button produces the same entry from [`CITATION.cff`](https://github.com/timurista/unalloc/blob/main/CITATION.cff).
+
+- **All versions:** [doi:10.5281/zenodo.22761012](https://doi.org/10.5281/zenodo.22761012), which always resolves to the latest release — currently 0.2.3, the code the paper describes
+- **Version 0.2.1:** [doi:10.5281/zenodo.22761013](https://doi.org/10.5281/zenodo.22761013)
+
+```bibtex
+@software{urista_unalloc_2026,
+  author  = {Urista, Timothy},
+  title   = {unalloc: find the AI spend nobody owns},
+  version = {0.2.3},
+  year    = {2026},
+  doi     = {10.5281/zenodo.22761012},
+  url     = {https://github.com/timurista/unalloc}
+}
+```
+
 ## License
 
 Apache 2.0, matching OpenCost.

{unalloc-0.2.1 → unalloc-0.2.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "unalloc"
-version = "0.2.1"
+version = "0.2.3"
 description = "Find the AI spend nobody owns. Joins OpenCost allocation data with LLM provider bills and reports what's unattributed."
 readme = "README.md"
 requires-python = ">=3.11"
@@ -40,6 +40,7 @@ research = [
   "ipykernel>=6.29",
   "jupyterlab>=4.2",
   "psycopg[binary]>=3.2",
+  "markdown>=3.6",
 ]
 
 [project.urls]
@@ -69,8 +70,12 @@ extend-exclude = ["*.ipynb"]
 [tool.ruff.lint]
 select = ["E", "F", "I", "UP", "B", "SIM", "RUF"]
 ignore = ["B008"]
-# Figure labels use real typographic signs ("36×", "÷").
-allowed-confusables = ["×", "÷"]
+# Figure labels and page copy use real typographic signs ("36×", "÷", "12–14").
+allowed-confusables = ["×", "÷", "–"]
+
+[tool.ruff.lint.per-file-ignores]
+# HTML page templates; wrapping them for line length hurts more than it helps.
+"docs_build/__main__.py" = ["E501"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

{unalloc-0.2.1 → unalloc-0.2.3}/src/unalloc/__init__.py

@@ -2,5 +2,5 @@
 
 from __future__ import annotations
 
-__version__ = "0.2.1"
+__version__ = "0.2.3"
 __all__ = ["__version__"]

unalloc-0.2.3/tests/test_docs_build.py

@@ -0,0 +1,33 @@
+"""The Pages site builds from the repository with every local link resolving."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+pytest.importorskip("markdown")
+
+from docs_build.__main__ import _rewrite, build
+
+
+def test_site_builds_with_no_broken_links(tmp_path):
+    written = build(tmp_path / "docs")
+    names = {p.relative_to(tmp_path / "docs").as_posix() for p in written}
+    assert {"index.html", "explorer.html", "blog/who-pays-for-the-kv-cache.html",
+            "blog/gpu-validation.html"} <= names
+    assert (tmp_path / "docs" / "paper" / "unalloc-case-studies.pdf").stat().st_size > 100_000
+    landing = (tmp_path / "docs" / "index.html").read_text()
+    assert "paper/unalloc-case-studies.pdf" in landing
+    post = (tmp_path / "docs" / "blog" / "who-pays-for-the-kv-cache.html").read_text()
+    assert 'href="../explorer.html"' in post
+    assert ".md\"" not in post
+
+
+def test_unknown_blog_link_fails_the_build():
+    assert _rewrite("../paper/figures/kv_shares.png") == "../assets/figures/kv_shares.png"
+    with pytest.raises(SystemExit):
+        _rewrite("notes/draft.md")

unalloc-0.2.3/tests/test_paper_numbers.py

@@ -0,0 +1,162 @@
+"""Every number in the GPU results table must still match the measured run.
+
+The published table once printed two time-share entries without redistributing
+overhead, which no test could see: the figure code redistributes, the table was
+typed by hand. This reads the numbers back out of the Typst source and compares
+them against `metrics.json`, so a stale table fails the build rather than a
+reviewer's arithmetic.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+
+import pytest
+
+ROOT = Path(__file__).resolve().parents[1]
+SOURCE = ROOT / "paper" / "unalloc.typ"
+METRICS = ROOT / "case_studies" / "results" / "gpu_validation" / "metrics.json"
+
+pytestmark = pytest.mark.skipif(not METRICS.exists(), reason="GPU metrics not present")
+
+# [2], [446 (3.7)], [773], [67%], [28 / 44], [6.3], [97%], [469], [16.5%], [4.8%],
+ROW = re.compile(
+    r"^\s*\[(?P<load>\d+)\], \[(?P<reqs>[\d,]+) \((?P<done>[\d.]+)\)\], \[(?P<out>[\d,]+)\], "
+    r"\[(?P<cache>\d+)%\], \[(?P<ttft50>[\d.]+) / (?P<ttft95>[\d.]+)\], \[(?P<tpot>[\d.]+)\], "
+    r"\[(?P<util>\d+)%\], \[(?P<power>\d+)\], \[(?P<tok>[\d.]+)%\], \[(?P<time>[\d.]+)%\],\s*$",
+    re.MULTILINE,
+)
+
+
+def _num(text: str) -> float:
+    return float(text.replace(",", ""))
+
+
+def _close(printed: float, actual: float, places: int) -> bool:
+    """Printed to `places` decimals is honest if it is within half a unit there."""
+    return abs(printed - actual) <= 0.5 * 10**-places + 1e-9
+
+
+@pytest.fixture(scope="module")
+def rows():
+    runs = {r["rate_rps"]: r for r in json.loads(METRICS.read_text())["runs"]}
+    printed = [m.groupdict() for m in ROW.finditer(SOURCE.read_text())]
+    assert len(printed) == len(runs), f"table has {len(printed)} rows, data has {len(runs)}"
+    return [(p, runs[float(p["load"])]) for p in printed]
+
+
+def test_table_reports_completed_requests_and_throughput(rows):
+    for p, run in rows:
+        assert _num(p["reqs"]) == run["requests"]
+        assert _close(_num(p["done"]), run["requests"] / run["wall_s"], 1), p["load"]
+        assert _close(_num(p["out"]), run["output_tokens_per_s"], 0), p["load"]
+        assert _close(_num(p["cache"]), run["cache_hit_rate_usage"] * 100, 0), p["load"]
+
+
+def test_table_latency_and_device_columns(rows):
+    for p, run in rows:
+        lat = run["latency"]["all"]
+        assert _close(_num(p["ttft50"]), lat["ttft_s"]["p50"] * 1000, 0), p["load"]
+        assert _close(_num(p["ttft95"]), lat["ttft_s"]["p95"] * 1000, 0), p["load"]
+        assert _close(_num(p["tpot"]), lat["tpot_s"]["p50"] * 1000, 1), p["load"]
+        assert _close(_num(p["util"]), run["gpu"]["gpu_util_pct_mean"], 0), p["load"]
+        assert _close(_num(p["power"]), run["gpu"]["power_w_mean"], 0), p["load"]
+
+
+def test_meter_shares_are_printed_with_overhead_redistributed(rows):
+    """The caption promises redistribution; both meter columns must honour it."""
+    for p, run in rows:
+        for column, meter in (("tok", "tokens"), ("time", "time_share")):
+            split = run["meters"][meter]
+            redistributed = split["search"] / (1 - split["__overhead__"]) * 100
+            assert _close(_num(p[column]), redistributed, 1), (
+                f"load {p['load']} {meter}: paper prints {p[column]}%, "
+                f"redistributed is {redistributed:.3f}% "
+                f"(raw {split['search'] * 100:.3f}%)"
+            )
+
+
+def test_prose_ranges_match_the_table(rows):
+    text = " ".join(SOURCE.read_text().split())
+    tok = [_num(p["tok"]) for p, _ in rows]
+    share = [_num(p["time"]) for p, _ in rows]
+    assert f"search {min(tok):.1f}–{max(tok):.1f}% of the bill" in text
+    assert f"time-share meter assigns {min(share):.1f}–{max(share):.1f}%" in text
+
+    divergence = [run["divergence_search_tokens_vs_time_pts"] for _, run in rows]
+    lo, hi = min(divergence), max(divergence)
+    assert f"a disagreement of {lo:.1f}–{hi:.1f} points" in text
+
+
+def test_totals_and_error_free_claim(rows):
+    flowed = " ".join(SOURCE.read_text().split())
+    total = sum(run["requests"] for _, run in rows)
+    assert f"All {total:,} requests completed without error" in flowed
+    assert all(run["errors"] == 0 for _, run in rows)
+
+
+def test_queue_claim_is_scoped_to_the_sampling_interval(rows):
+    """max_waiting is a half-second Prometheus sample, not a continuous guarantee."""
+    text = " ".join(SOURCE.read_text().split())
+    assert all(run["max_waiting"] == 0 for _, run in rows)
+    assert "no queued request was observed in any half-second telemetry sample" in text
+    assert "none waited in the queue" not in text
+
+
+def test_configured_rate_is_distinguished_from_completed_traffic(rows):
+    text = " ".join(SOURCE.read_text().split())
+    assert "session-initial" in text
+    done = [run["requests"] / run["wall_s"] for _, run in rows]
+    assert f"{min(done):.1f} to {max(done):.1f} completed requests per second" in text
+
+
+# --- the arXiv submission kit must describe the PDF that is actually committed ---
+
+PDF = ROOT / "paper" / "unalloc-case-studies.pdf"
+KIT = ROOT / "paper" / "arxiv" / "SUBMISSION.md"
+ABSTRACT = ROOT / "paper" / "arxiv" / "abstract.txt"
+
+
+def _pdf_pages(path: Path) -> int:
+    """Page count from the PDF's page tree, without a PDF library."""
+    blob = path.read_bytes()
+    counts = re.findall(rb"/Type\s*/Pages[^>]*?/Count\s+(\d+)", blob)
+    assert counts, "no page tree found in the PDF"
+    return max(int(c) for c in counts)
+
+
+@pytest.mark.skipif(not PDF.exists() or not KIT.exists(), reason="paper artifacts not present")
+def test_submission_comments_match_the_built_pdf():
+    kit = KIT.read_text()
+    pages = _pdf_pages(PDF)
+    assert f"| Comments | {pages} pages," in kit, (
+        f"SUBMISSION.md Comments claims a different length; the PDF has {pages} pages"
+    )
+    source = SOURCE.read_text()
+    figures = source.count("#figure(image(")
+    tables = len(re.findall(r"^#figure\($", source, re.MULTILINE))
+    assert f"{pages} pages, {figures} figures, {tables} tables" in kit
+
+
+@pytest.mark.skipif(not KIT.exists(), reason="submission kit not present")
+def test_submission_kit_cites_the_concept_doi():
+    """A version DOI goes stale on the next release; the concept DOI does not."""
+    kit = KIT.read_text()
+    concept = "10.5281/zenodo.22761012"
+    assert f"Software: doi:{concept}" in kit
+    assert concept in SOURCE.read_text(), "the paper itself should cite the same DOI"
+
+
+@pytest.mark.skipif(not ABSTRACT.exists(), reason="abstract not present")
+def test_standalone_abstract_carries_the_same_qualifications_as_the_paper():
+    text = ABSTRACT.read_text().strip()
+    assert len(text) <= 1920, f"arXiv caps the abstract at 1920 characters; this is {len(text)}"
+    # The two scope corrections must survive in the copy that readers see first.
+    assert "constructed multi-pod deployment scenario" in text
+    assert "synthetic OpenCost allocations" in text
+    assert "session-initial" in text
+    assert "completed requests per second" in text
+    # The bare claim the review flagged must not come back.
+    assert "97-99% from 2 to 16 requests per second" not in text

unalloc-0.2.1/CHANGELOG.md

@@ -1,35 +0,0 @@
-# Changelog
-
-## 0.2.1 — 2026-09-14
-
-### Changed
-- Paper prepared for arXiv: abstract leads with the attribution failures at the seams between ledgers and the H100 result; metering results are framed as disagreement between rules rather than over-charging, since none is a ground truth; new Related Work section (JouleShare, LLMVisor, PrefixShield, Cost-Governed RAG and others); defect catalogue moved to Appendix A; use of AI tools disclosed.
-- `paper/build.py` fails on Typst that compiles but renders wrong (`~`, `;` after a citation, unescaped dollar amounts).
-
-### Added
-- `case_studies/gpu_validation`: benchmark client, fake server, remote setup script, analysis and evidence renderer for validating the metering results with real vLLM on a GPU, plus a DigitalOcean runbook. First run: H100, vLLM 0.29.0, Qwen2.5-7B-Instruct, 6,241 requests, 0 errors.
-- Blog addendum with the verified GPU specs, results and bring-up/teardown evidence.
-- Paper §9, "Validation on a datacenter GPU", with the H100 results table and figure; abstract, methodology table and limitations updated to match.
-
-## 0.2.0 — 2026-09-13
-
-Validated against five inference case studies; see `paper/unalloc-case-studies.pdf`.
-
-### Fixed
-- OpenAI and Anthropic adapters lost their default API URLs whenever the CLI passed an unset environment variable, so every live fetch failed.
-- The Anthropic Admin API is now called with `x-api-key` and `anthropic-version` instead of a Bearer token.
-- Provider cost APIs are paged with `has_more` / `next_page`; previously only the first page was read.
-- OpenCost component costs are summed as `Decimal` when `totalCost` is absent, never through `float`.
-- Stacked label prefixes are all stripped (`label_app_kubernetes_io_name` → `name`).
-- Canonical-key collisions resolve by a fixed precedence instead of arrival order (explicit key, alias, rewritten key, path-derived key).
-- OpenCost annotations no longer override labels with the same key.
-- `unalloc labels` honours `--fallback`, and exits 1 when no rows load instead of printing "Nothing unallocated".
-
-### Added
-- `report --budget PCT` exits 2 when the unallocated share exceeds `PCT`, for CI gates.
-- Reports show the spend attributed only through a fallback key (`fallback_usd` in `--json`).
-- Case studies (`case_studies/`), a Typst paper (`paper/`), a ledger explorer, a companion notebook and a dev container with Postgres.
-
-## 0.1.0
-
-Initial release: OpenCost, LiteLLM, OpenAI and Anthropic adapters; `report`, `labels` and `reconcile`.