quantprobe 1.3.0__tar.gz → 1.5.0__tar.gz

quantprobe-1.5.0/PKG-INFO

@@ -0,0 +1,145 @@
+Metadata-Version: 2.4
+Name: quantprobe
+Version: 1.5.0
+Summary: Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes.
+Author: Federico Sciuca
+License: MIT
+Project-URL: Homepage, https://github.com/FedericoTs/quantprobe
+Keywords: llm,quantization,llama.cpp,gguf,inference,moe
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: gguf>=0.9
+Requires-Dist: requests>=2.28
+Dynamic: license-file
+
+# quantprobe
+
+### Placement beats budget
+
+**Where your bits sit — which layers, which memory tier — matters more than how many you have. Four falsification-tested laws for running big LLMs on hardware you already own, every number measured on one 2016 desktop (GTX 1060 6 GB · 16 GB DDR4 · SATA SSD).**
+
+![smoke](https://github.com/FedericoTs/quantprobe/actions/workflows/smoke.yml/badge.svg) ![license](https://img.shields.io/badge/license-MIT-0f766e) ![hardware](https://img.shields.io/badge/measured_on-2016_GTX_1060_6GB-d73027) ![data--free](https://img.shields.io/badge/quantization-data--free-1d9e70) ![models](https://img.shields.io/badge/validated-7B_→_744B-378add) [![x](https://img.shields.io/badge/author-@federico__sciuca-14181f)](https://x.com/federico_sciuca)
+
+<p align="center"><img src="weights/data/hero_placement.png" width="880" alt="Placement across memory tiers: disk 2-bit cold experts, RAM 2-bit experts + probed fragile band at 4-bit, VRAM 4-bit attention — one law ties them"></p>
+
+> **▶ Try the interactive calculator: [Will it run — and how fast?](https://federicots.github.io/quantprobe/)**
+> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
+
+
+**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds.** Two tiers: `plan`/`target` and the web calculator need *nothing* installed; `quantize`/`probe`/`run`/`bench`/`dashboard` drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`). Preview any command without llama.cpp using `--dry`. 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
+
+
+---
+
+## What this is
+
+My 2016 desktop can't run frontier models the way a datacenter does — so instead of brute force, I asked *where* every bit and byte should go, and answered it by measurement. Months of experiments later, the result is four laws, a 30-minute probe tool, copy-paste llama.cpp recipes, and one equation that predicts decode speed from 7B to 744B — validated against my own pre-registered predictions and against [colibri](https://github.com/JustVugg/colibri)'s independently published 744B numbers.
+
+## Headline results
+
+| result | number |
+|---|---|
+| 16B MoE, 2-bit, **data-free**, resident on a 6 GB card | ppl 6.31 → 6.96 (**1.10×**) — beats calibrated SOTA's gap-ratio |
+| Same bytes, different layers (Gemma 4 12B, stock llama.cpp) | **byte-identical files, 2.25 ppl apart** (12.27 vs 10.02) |
+| Gemma 4 12B depth-aware 2-bit | 1.91× → **1.45×** quality cost, ~4.5 GB resident |
+| Qwen3-30B-A3B on the 2016 desktop | **19.3 tok/s** — hybrid placement, *predicted 19 before measuring* |
+| GLM-4.5-Air **110B** from a SATA drive, 16 GB RAM | 0.19 tok/s — inside the law's pre-registered 0.2–0.3 band |
+| RAM overclock (XMP, 2133→3000) | dense **+52%**, pre-registered ×1.41+ |
+
+## Why the evidence is unusually strong
+
+Most benchmark posts report what happened. I report what I **predicted before it happened** — I wrote the number down, then ran the hardware, and this is the strongest form of empirical evidence I know how to produce:
+
+| prediction (made first) | measured (after) |
+|---|---|
+| 110B streamed from SATA: **0.2–0.3 tok/s** | **0.19** |
+| RAM overclock scales in-RAM decode **×1.41+** | **×1.52** |
+| 30B hybrid placement: **~19 tok/s** | **19.30 ± 0.88** |
+| a day-old 118B (Laguna S 2.1) streamed from this SATA drive: **0.2–0.4** ([staked pre-download](preregistrations/2026-07-23-laguna-s-2.1-on-2016-desktop.md)) | **0.38 ± 0.17** |
+| colibri's own 128 GB / 25 GB tiers, from our η bands | land **inside** the bands |
+
+Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apart — only placement differs), a full **claim → script → log manifest** (every number reproducible in-tree), and a set of **documented dead ends** (dynamic top-k, semantic paging, self-speculation — all measured-dead, because a law you only confirm is a law you haven't tested).
+
+<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
+<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
+
+**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
+
+
+## The four placement laws
+
+Full statements, each with its establishing measurement and a falsifiable prediction, in **[LAWS.md](LAWS.md)**.
+
+1. **Rotation is rank-conditional.** Incoherence rotation (QuIP#/QTIP/QuaRot) helps full-rank tensors (+0.006 ppl) and destroys low-rank bottlenecks (+1623 ppl) — a ~270,000× swing on effective rank alone.
+2. **Trained networks are dense everywhere.** Experts sit *exactly* at the rate-distortion floor; routing is flat (even across domains — Jaccard 1.00 prose vs code); activations are diffuse. **2-bit is the floor.**
+3. **Fragility is measurable, not predictable.** Gemma late-fragile 4×, Mistral **early-fragile 25×** — architectural near-twins pointing opposite ways. Weight statistics mislead. **Only a 30-minute functional probe decides.**
+4. **The tiered decode law.** `tok/s = η(tier)·BW ÷ active-bytes`, η collapsing per tier across 7B→744B and both projects' hardware. **v1.1 adds the context term**: each generated token also re-reads the whole KV cache from *its* tier — `--ctx` prices it, `bench --depth` measures it (measured here: 20.02 → 16.12 tok/s at 16k depth).
+
+<p align="center"><img src="weights/data/x_chart_E_scalinglaw.png" width="700" alt="One scaling law, 7B to 744B, predicted vs measured, including colibri's published tiers"></p>
+
+<p align="center"><img src="weights/data/x_chart_D_placement.png" width="700" alt="Byte-identical GGUF files, 2.25 perplexity apart — placement is worth twice the byte budget"></p>
+<p align="center"><img src="weights/data/x_chart_I_kvdepth.png" width="700" alt="Decode speed versus context depth: measured 20.02 tok/s at depth zero falling to 16.12 at 16384, on the Law 4 v2 curve with eta_kv 0.70"></p>
+
+
+## Install — and the eleven commands
+
+```bash
+pip install quantprobe
+quantprobe auto qwen3-coder --tps 15 --run   # empty machine -> optimal quant chosen, fetched, chatting
+```
+
+**Zero-config on your own box**: `quantprobe plan --gguf model.gguf` auto-detects the machine and reads the model from the file. Presets/flags estimate any *other* machine. `hw`/`plan`/`target`/`optimize` need nothing else installed (`auto` needs network for the fetch); the rest drive stock [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point at it with `--llama-dir`/`QUANTPROBE_LLAMA_DIR`/`PATH`; preview any command with `--dry`).
+
+```bash
+quantprobe auto qwen3-30b --tps 15                       # ONE command: optimizer picks bits, closest quant fetched, run command printed
+quantprobe hw                                            # what the law sees on THIS machine (every value source-tagged)
+quantprobe plan     --gguf model.gguf                    # zero-config prediction: placement + tok/s + the launch command
+quantprobe optimize --tps 20                             # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
+quantprobe target   --tps 5 --machine gaming --ladder    # inverse: target -> smartest model + speed-intelligence ladder
+quantprobe fetch    qwen3-30b ./models                   # robust, resumable download
+quantprobe quantize --gguf f16.gguf --out 2bit.gguf      # COMPRESS: depth-aware ~2-bit GGUF (verified: loads + generates)
+quantprobe probe    --gguf f16.gguf --eval wiki.test.raw # measure YOUR model's fragile band (~30 min); --apply builds it
+quantprobe run      --gguf 2bit.gguf                     # plan the placement, then LAUNCH llama.cpp chat
+quantprobe bench    --gguf 2bit.gguf --contribute        # predicted vs measured on your box; opt-in datapoint
+quantprobe dashboard --gguf 2bit.gguf                    # the law LIVE: neuron galaxy + thinking toggle, every reply scored vs prediction
+```
+
+The loop is self-validating: `plan` predicted 17.5 for a file we then measured at **18.32 ± 0.17**; the config months of research converged to is what `optimize` picks blind. A measured example of what that's worth: the same model, mis-specified vs law-routed, is **3.38 vs 18.32 tok/s (×5.4)** — [worked examples](docs/EXAMPLES.md).
+
+## Help grow the law
+
+`quantprobe bench --contribute` prints exactly what would be shared (hardware label, model, predicted-vs-measured) plus a pre-filled issue link — **you review and submit; nothing is ever sent automatically**. Contributed points land on the law chart; the ones *outside* the bands are the most valuable. Open falsifiable predictions anyone can settle: [preregistrations/](preregistrations/).
+
+## Deep dives
+
+| | |
+|---|---|
+| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; Ollama interop; llama.cpp version notes |
+| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
+| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
+| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
+| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
+| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
+| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
+| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.4, every release |
+
+## Honest limitations
+
+- Perplexity on WikiText-2 is my primary metric; I haven't run task-level evals (MMLU/HellaSwag) yet.
+- My fragility atlas covers four model families — enough to *disprove* universality, not to chart every architecture.
+- 0.19 tok/s for a 110B is a **capacity demonstration, not usable inference** — the honest speed only arrives with faster storage.
+- Speed numbers are single-stream decode on one machine (±25% across environments); the tiered-decode η values are fitted, not derived.
+- No custom runtime: everything rides stock llama.cpp and streaming eval harnesses. The one CUDA kernel is verified in reference, not built.
+
+
+## Credits
+
+[colibri](https://github.com/JustVugg/colibri) (744B on 25 GB, pure C) inspired the tier-streaming exploration. The quantization stack builds on [llama.cpp](https://github.com/ggml-org/llama.cpp) and the QTIP/QuIP# incoherence codecs — whose central tool our first law bounds. Independent research by Federico Sciuca, AI-supported, on one desktop; every claim is measured, and every negative that redirected the work is documented. The Law 4 context term (v1.1) was prompted by **u/RogerAI--fyi** (Reddit), who correctly observed the original formulation omitted per-token KV reads — measured, confirmed, and shipped within a day.
+
+## License
+
+MIT — see [LICENSE](LICENSE). © 2026 Federico Sciuca.

quantprobe-1.5.0/README.md

@@ -0,0 +1,127 @@
+# quantprobe
+
+### Placement beats budget
+
+**Where your bits sit — which layers, which memory tier — matters more than how many you have. Four falsification-tested laws for running big LLMs on hardware you already own, every number measured on one 2016 desktop (GTX 1060 6 GB · 16 GB DDR4 · SATA SSD).**
+
+![smoke](https://github.com/FedericoTs/quantprobe/actions/workflows/smoke.yml/badge.svg) ![license](https://img.shields.io/badge/license-MIT-0f766e) ![hardware](https://img.shields.io/badge/measured_on-2016_GTX_1060_6GB-d73027) ![data--free](https://img.shields.io/badge/quantization-data--free-1d9e70) ![models](https://img.shields.io/badge/validated-7B_→_744B-378add) [![x](https://img.shields.io/badge/author-@federico__sciuca-14181f)](https://x.com/federico_sciuca)
+
+<p align="center"><img src="weights/data/hero_placement.png" width="880" alt="Placement across memory tiers: disk 2-bit cold experts, RAM 2-bit experts + probed fragile band at 4-bit, VRAM 4-bit attention — one law ties them"></p>
+
+> **▶ Try the interactive calculator: [Will it run — and how fast?](https://federicots.github.io/quantprobe/)**
+> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
+
+
+**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds.** Two tiers: `plan`/`target` and the web calculator need *nothing* installed; `quantize`/`probe`/`run`/`bench`/`dashboard` drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`). Preview any command without llama.cpp using `--dry`. 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
+
+
+---
+
+## What this is
+
+My 2016 desktop can't run frontier models the way a datacenter does — so instead of brute force, I asked *where* every bit and byte should go, and answered it by measurement. Months of experiments later, the result is four laws, a 30-minute probe tool, copy-paste llama.cpp recipes, and one equation that predicts decode speed from 7B to 744B — validated against my own pre-registered predictions and against [colibri](https://github.com/JustVugg/colibri)'s independently published 744B numbers.
+
+## Headline results
+
+| result | number |
+|---|---|
+| 16B MoE, 2-bit, **data-free**, resident on a 6 GB card | ppl 6.31 → 6.96 (**1.10×**) — beats calibrated SOTA's gap-ratio |
+| Same bytes, different layers (Gemma 4 12B, stock llama.cpp) | **byte-identical files, 2.25 ppl apart** (12.27 vs 10.02) |
+| Gemma 4 12B depth-aware 2-bit | 1.91× → **1.45×** quality cost, ~4.5 GB resident |
+| Qwen3-30B-A3B on the 2016 desktop | **19.3 tok/s** — hybrid placement, *predicted 19 before measuring* |
+| GLM-4.5-Air **110B** from a SATA drive, 16 GB RAM | 0.19 tok/s — inside the law's pre-registered 0.2–0.3 band |
+| RAM overclock (XMP, 2133→3000) | dense **+52%**, pre-registered ×1.41+ |
+
+## Why the evidence is unusually strong
+
+Most benchmark posts report what happened. I report what I **predicted before it happened** — I wrote the number down, then ran the hardware, and this is the strongest form of empirical evidence I know how to produce:
+
+| prediction (made first) | measured (after) |
+|---|---|
+| 110B streamed from SATA: **0.2–0.3 tok/s** | **0.19** |
+| RAM overclock scales in-RAM decode **×1.41+** | **×1.52** |
+| 30B hybrid placement: **~19 tok/s** | **19.30 ± 0.88** |
+| a day-old 118B (Laguna S 2.1) streamed from this SATA drive: **0.2–0.4** ([staked pre-download](preregistrations/2026-07-23-laguna-s-2.1-on-2016-desktop.md)) | **0.38 ± 0.17** |
+| colibri's own 128 GB / 25 GB tiers, from our η bands | land **inside** the bands |
+
+Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apart — only placement differs), a full **claim → script → log manifest** (every number reproducible in-tree), and a set of **documented dead ends** (dynamic top-k, semantic paging, self-speculation — all measured-dead, because a law you only confirm is a law you haven't tested).
+
+<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
+<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
+
+**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
+
+
+## The four placement laws
+
+Full statements, each with its establishing measurement and a falsifiable prediction, in **[LAWS.md](LAWS.md)**.
+
+1. **Rotation is rank-conditional.** Incoherence rotation (QuIP#/QTIP/QuaRot) helps full-rank tensors (+0.006 ppl) and destroys low-rank bottlenecks (+1623 ppl) — a ~270,000× swing on effective rank alone.
+2. **Trained networks are dense everywhere.** Experts sit *exactly* at the rate-distortion floor; routing is flat (even across domains — Jaccard 1.00 prose vs code); activations are diffuse. **2-bit is the floor.**
+3. **Fragility is measurable, not predictable.** Gemma late-fragile 4×, Mistral **early-fragile 25×** — architectural near-twins pointing opposite ways. Weight statistics mislead. **Only a 30-minute functional probe decides.**
+4. **The tiered decode law.** `tok/s = η(tier)·BW ÷ active-bytes`, η collapsing per tier across 7B→744B and both projects' hardware. **v1.1 adds the context term**: each generated token also re-reads the whole KV cache from *its* tier — `--ctx` prices it, `bench --depth` measures it (measured here: 20.02 → 16.12 tok/s at 16k depth).
+
+<p align="center"><img src="weights/data/x_chart_E_scalinglaw.png" width="700" alt="One scaling law, 7B to 744B, predicted vs measured, including colibri's published tiers"></p>
+
+<p align="center"><img src="weights/data/x_chart_D_placement.png" width="700" alt="Byte-identical GGUF files, 2.25 perplexity apart — placement is worth twice the byte budget"></p>
+<p align="center"><img src="weights/data/x_chart_I_kvdepth.png" width="700" alt="Decode speed versus context depth: measured 20.02 tok/s at depth zero falling to 16.12 at 16384, on the Law 4 v2 curve with eta_kv 0.70"></p>
+
+
+## Install — and the eleven commands
+
+```bash
+pip install quantprobe
+quantprobe auto qwen3-coder --tps 15 --run   # empty machine -> optimal quant chosen, fetched, chatting
+```
+
+**Zero-config on your own box**: `quantprobe plan --gguf model.gguf` auto-detects the machine and reads the model from the file. Presets/flags estimate any *other* machine. `hw`/`plan`/`target`/`optimize` need nothing else installed (`auto` needs network for the fetch); the rest drive stock [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point at it with `--llama-dir`/`QUANTPROBE_LLAMA_DIR`/`PATH`; preview any command with `--dry`).
+
+```bash
+quantprobe auto qwen3-30b --tps 15                       # ONE command: optimizer picks bits, closest quant fetched, run command printed
+quantprobe hw                                            # what the law sees on THIS machine (every value source-tagged)
+quantprobe plan     --gguf model.gguf                    # zero-config prediction: placement + tok/s + the launch command
+quantprobe optimize --tps 20                             # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
+quantprobe target   --tps 5 --machine gaming --ladder    # inverse: target -> smartest model + speed-intelligence ladder
+quantprobe fetch    qwen3-30b ./models                   # robust, resumable download
+quantprobe quantize --gguf f16.gguf --out 2bit.gguf      # COMPRESS: depth-aware ~2-bit GGUF (verified: loads + generates)
+quantprobe probe    --gguf f16.gguf --eval wiki.test.raw # measure YOUR model's fragile band (~30 min); --apply builds it
+quantprobe run      --gguf 2bit.gguf                     # plan the placement, then LAUNCH llama.cpp chat
+quantprobe bench    --gguf 2bit.gguf --contribute        # predicted vs measured on your box; opt-in datapoint
+quantprobe dashboard --gguf 2bit.gguf                    # the law LIVE: neuron galaxy + thinking toggle, every reply scored vs prediction
+```
+
+The loop is self-validating: `plan` predicted 17.5 for a file we then measured at **18.32 ± 0.17**; the config months of research converged to is what `optimize` picks blind. A measured example of what that's worth: the same model, mis-specified vs law-routed, is **3.38 vs 18.32 tok/s (×5.4)** — [worked examples](docs/EXAMPLES.md).
+
+## Help grow the law
+
+`quantprobe bench --contribute` prints exactly what would be shared (hardware label, model, predicted-vs-measured) plus a pre-filled issue link — **you review and submit; nothing is ever sent automatically**. Contributed points land on the law chart; the ones *outside* the bands are the most valuable. Open falsifiable predictions anyone can settle: [preregistrations/](preregistrations/).
+
+## Deep dives
+
+| | |
+|---|---|
+| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; Ollama interop; llama.cpp version notes |
+| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
+| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
+| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
+| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
+| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
+| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
+| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.4, every release |
+
+## Honest limitations
+
+- Perplexity on WikiText-2 is my primary metric; I haven't run task-level evals (MMLU/HellaSwag) yet.
+- My fragility atlas covers four model families — enough to *disprove* universality, not to chart every architecture.
+- 0.19 tok/s for a 110B is a **capacity demonstration, not usable inference** — the honest speed only arrives with faster storage.
+- Speed numbers are single-stream decode on one machine (±25% across environments); the tiered-decode η values are fitted, not derived.
+- No custom runtime: everything rides stock llama.cpp and streaming eval harnesses. The one CUDA kernel is verified in reference, not built.
+
+
+## Credits
+
+[colibri](https://github.com/JustVugg/colibri) (744B on 25 GB, pure C) inspired the tier-streaming exploration. The quantization stack builds on [llama.cpp](https://github.com/ggml-org/llama.cpp) and the QTIP/QuIP# incoherence codecs — whose central tool our first law bounds. Independent research by Federico Sciuca, AI-supported, on one desktop; every claim is measured, and every negative that redirected the work is documented. The Law 4 context term (v1.1) was prompted by **u/RogerAI--fyi** (Reddit), who correctly observed the original formulation omitted per-token KV reads — measured, confirmed, and shipped within a day.
+
+## License
+
+MIT — see [LICENSE](LICENSE). © 2026 Federico Sciuca.

{quantprobe-1.3.0 → quantprobe-1.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "quantprobe"
-version = "1.3.0"
+version = "1.5.0"
 description = "Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes."
 readme = "README.md"
 license = { text = "MIT" }

{quantprobe-1.3.0 → quantprobe-1.5.0}/quantprobe/__init__.py

@@ -8,4 +8,4 @@ Three commands, all grounded in measured laws (see LAWS.md in the repo):
 Every constant in `plan` is fitted from measurements published in the repo, including two
 pre-registered hardware predictions confirmed within 8%.
 """
-__version__ = "1.3.0"
+__version__ = "1.5.0"

quantprobe-1.5.0/quantprobe/auto.py

@@ -0,0 +1,106 @@
+"""quantprobe auto — one command: model in, running setup out.
+
+    quantprobe auto qwen3-30b --tps 15          # preset
+    quantprobe auto unsloth/Qwen3-30B-A3B-GGUF  # any HF GGUF repo (params read from filenames' size)
+
+The two-speed design:
+  FAST PATH (this command): detect the machine -> ask the optimizer for the best effective-bits ->
+  scan the HF repo's file list and pick the closest-matching quant BY SIZE (bits = size*8/params —
+  no fragile name parsing) -> fetch it -> print the prediction and the run command (--run launches).
+  CUSTOM PATH (the actual product, printed at the end): probe YOUR model's fragile band and build a
+  depth-aware GGUF at the same bits — better quality at the same bytes. `probe --apply` does it.
+"""
+from __future__ import annotations
+import json, urllib.request
+
+from . import plan as planmod
+from . import optimize as optmod
+
+# preset -> (repo, total params B, active B, always-active B, moe)
+MODEL_REPOS = {
+    "qwen3-30b":  ("unsloth/Qwen3-30B-A3B-GGUF", 30.5, 3.3, 1.2, True),
+    "qwen3-coder": ("unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF", 30.5, 3.3, 1.2, True),
+    "glm-air":    ("unsloth/GLM-4.5-Air-GGUF", 110, 12, 2.7, True),
+    "laguna-s":   ("unsloth/Laguna-S-2.1-GGUF", 117.6, 8, 2.5, True),
+    "gemma-12b":  ("unsloth/gemma-4-12b-it-GGUF", 11.9, 11.9, 11.9, False),
+    "mistral-7b": ("unsloth/Mistral-7B-Instruct-v0.3-GGUF", 7.2, 7.2, 7.2, False),
+}
+
+
+def list_ggufs(repo):
+    """[(path, size_bytes)] for a HF repo, via the public tree API."""
+    req = urllib.request.Request(f"https://huggingface.co/api/models/{repo}/tree/main",
+                                 headers={"User-Agent": "quantprobe-auto"})
+    with urllib.request.urlopen(req, timeout=30) as r:
+        return [(f["path"], f.get("size", 0)) for f in json.load(r)
+                if f["path"].endswith(".gguf") and f.get("size", 0) > 1e8
+                and "mmproj" not in f["path"].lower() and "draft" not in f["path"].lower()]
+
+
+def run(a):
+    target = a.target
+    if target in MODEL_REPOS:
+        repo, t, ac, ne, moe = MODEL_REPOS[target]
+    else:
+        repo = target
+        if not getattr(a, "total", None):
+            raise SystemExit(f"'{target}' is not a preset ({', '.join(MODEL_REPOS)}) - for a raw HF "
+                             "repo also pass --total (B params) and, for MoE, --active/--always-active")
+        t, ac = a.total, a.active or a.total
+        ne = a.always_active or (ac if ac >= t * 0.9 else ac * 0.35)
+        moe = ac < t * 0.9
+    a.total, a.active, a.always_active, a.model = t, ac, ne, None
+
+    # 1. what does the optimizer want on THIS machine?
+    a.gguf = None
+    ranked = optmod.run(a)
+    want_bits = ranked[0]["bits"]
+    _, _, _, _, _, (vc, vb, rc, rb, db, geta, gl), ctx, kvp = optmod.resolve(a)
+
+    # 2. pick the closest file BY SIZE (bits = size*8/params; honest, format-agnostic)
+    try:
+        files = list_ggufs(repo)
+    except Exception as e:
+        raise SystemExit(f"could not list {repo}: {e}")
+    if not files:
+        raise SystemExit(f"no GGUF files found in {repo}")
+    scored = []
+    for path, size in files:
+        bits = size * 8 / (t * 1e9)
+        if bits < 1.0 or bits > 9:
+            continue
+        _, _, cfgs = planmod.evaluate(t, ac, ne, moe, bits, vc, vb, rc, rb, db, geta, 1.0, gl,
+                                      ctx=ctx, kvp=kvp)
+        cfgs = [c for c in cfgs if "expert cache" not in c[0]]
+        if not cfgs:
+            continue
+        scored.append((abs(bits - want_bits), -cfgs[0][1], path, size, bits, cfgs[0]))
+    if not scored:
+        raise SystemExit("no usable quant in that repo for this machine")
+    scored.sort()
+    _, _, path, size, bits, best = scored[0]
+    print(f"\n[quantprobe auto] optimizer wants ~{want_bits:g}-bit; closest file in {repo}:")
+    print(f"  {path}  ({size/1e9:.1f} GB, {bits:.2f} effective bits)")
+    print(f"  predicted on this machine: {best[1]:.1f} tok/s  ({best[0]})")
+    if getattr(a, "dry", False):
+        print("  (--dry: nothing downloaded)")
+        return path
+    # 3. fetch it (resumable), then hand off
+    from . import fetch as fetchmod
+    dest = getattr(a, "dir", None) or "./models"
+    import os
+    os.makedirs(dest, exist_ok=True)
+    ok = fetchmod.fetch(repo, dest, path, fetchmod.token())
+    if not ok:
+        raise SystemExit("download failed (it resumes: re-run the same command)")
+    full = os.path.join(dest, os.path.basename(path))
+    print(f"\n[quantprobe auto] ready. Run it:")
+    print(f"  quantprobe run --gguf {full}")
+    print(f"\n  Want better quality at the SAME size? That's the custom path (the actual product):")
+    print(f"  fetch the f16, then:  quantprobe probe --gguf <f16>.gguf --eval wiki.test.raw --apply")
+    if getattr(a, "run", False):
+        from . import runtime
+        a.gguf = full
+        a.bits = None                              # let autospec read the real file
+        runtime.run(a)
+    return full

{quantprobe-1.3.0 → quantprobe-1.5.0}/quantprobe/cli.py

@@ -84,6 +84,24 @@ def main():
     z.add_argument("--llama-dir", default=None)
     z.add_argument("--dry", action="store_true")
 
+    o = sub.add_parser("optimize", help="cheapest path to a target speed: search bits x placement x KV x hardware over the law")
+    o.add_argument("--tps", type=float, default=None, help="target tok/s (omit for the speed frontier)")
+    o.add_argument("--max-quality", type=float, default=None, help="quality-cost ceiling (default 1.12)")
+    o.add_argument("--any-runtime", action="store_true", help="include placements needing expert-caching runtimes (ktransformers-class), not just stock llama.cpp")
+    o.add_argument("--allow-prune", action="store_true", help="include REAP-class pruned variants (domain-specialized: +39%% out-of-domain ppl measured)")
+    hwargs(o)
+
+    au = sub.add_parser("auto", help="ONE command: model in, running setup out - optimizer picks the bits, the closest quant is fetched, run command printed")
+    au.add_argument("target", help="model preset (qwen3-30b, qwen3-coder, glm-air, laguna-s, gemma-12b, mistral-7b) or a HF GGUF repo id")
+    au.add_argument("--dir", default="./models", help="download directory (default ./models)")
+    au.add_argument("--run", action="store_true", help="launch chat immediately after the download")
+    hwargs(au)
+    au.add_argument("--tps", type=float, default=None, help="target tok/s for the optimizer")
+    au.add_argument("--max-quality", type=float, default=None)
+    au.add_argument("--allow-prune", action="store_true"); au.add_argument("--any-runtime", action="store_true")
+    au.add_argument("--serve", action="store_true", help="with --run: llama-server instead of chat")
+    au.add_argument("--extra", default=None)
+
     hwp = sub.add_parser("hw", help="detect THIS machine's memory tiers (no flags needed); every value tagged with its source")
     hwp.add_argument("--measure", default=None, metavar="FILE", help="also MEASURE sequential disk read on a real file (e.g. a GGUF)")
 
@@ -109,6 +127,12 @@ def main():
     elif a.cmd == "target":
         from . import target
         target.run(a)
+    elif a.cmd == "auto":
+        from . import auto
+        auto.run(a)
+    elif a.cmd == "optimize":
+        from . import optimize
+        optimize.run(a)
     elif a.cmd == "hw":
         from . import detect
         detect.run(a)