quantprobe 1.5.0__tar.gz → 1.5.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {quantprobe-1.5.0/quantprobe.egg-info → quantprobe-1.5.2}/PKG-INFO +9 -5
- {quantprobe-1.5.0 → quantprobe-1.5.2}/README.md +8 -4
- {quantprobe-1.5.0 → quantprobe-1.5.2}/pyproject.toml +1 -1
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/__init__.py +1 -1
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/auto.py +67 -2
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/cli.py +1 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/optimize.py +1 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/plan.py +18 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/target.py +2 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2/quantprobe.egg-info}/PKG-INFO +9 -5
- {quantprobe-1.5.0 → quantprobe-1.5.2}/LICENSE +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/dashboard.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/detect.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/fetch.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/probe.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/runtime.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe/spec.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe.egg-info/SOURCES.txt +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe.egg-info/dependency_links.txt +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe.egg-info/entry_points.txt +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe.egg-info/requires.txt +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/quantprobe.egg-info/top_level.txt +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/setup.cfg +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/tests/test_engine.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/tests/test_evaluator.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/tests/test_pipeline.py +0 -0
- {quantprobe-1.5.0 → quantprobe-1.5.2}/tests/test_transforms.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: quantprobe
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.2
|
|
4
4
|
Summary: Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes.
|
|
5
5
|
Author: Federico Sciuca
|
|
6
6
|
License: MIT
|
|
@@ -30,7 +30,7 @@ Dynamic: license-file
|
|
|
30
30
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
31
31
|
|
|
32
32
|
|
|
33
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
33
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
34
34
|
|
|
35
35
|
|
|
36
36
|
---
|
|
@@ -67,7 +67,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
67
67
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
68
68
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
69
69
|
|
|
70
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
70
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
71
|
+
|
|
72
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
71
73
|
|
|
72
74
|
|
|
73
75
|
## The four placement laws
|
|
@@ -96,6 +98,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
96
98
|
|
|
97
99
|
```bash
|
|
98
100
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
101
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
99
102
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
100
103
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
101
104
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -118,14 +121,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
118
121
|
|
|
119
122
|
| | |
|
|
120
123
|
|---|---|
|
|
121
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
124
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
122
125
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
123
126
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
124
127
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
125
128
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
126
129
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
130
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
127
131
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
128
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
132
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
129
133
|
|
|
130
134
|
## Honest limitations
|
|
131
135
|
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
13
13
|
|
|
14
14
|
|
|
15
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
15
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
16
16
|
|
|
17
17
|
|
|
18
18
|
---
|
|
@@ -49,7 +49,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
49
49
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
50
50
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
51
51
|
|
|
52
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
52
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
53
|
+
|
|
54
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
53
55
|
|
|
54
56
|
|
|
55
57
|
## The four placement laws
|
|
@@ -78,6 +80,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
78
80
|
|
|
79
81
|
```bash
|
|
80
82
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
83
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
81
84
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
82
85
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
83
86
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -100,14 +103,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
100
103
|
|
|
101
104
|
| | |
|
|
102
105
|
|---|---|
|
|
103
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
106
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
104
107
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
105
108
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
106
109
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
107
110
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
108
111
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
112
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
109
113
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
110
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
114
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
111
115
|
|
|
112
116
|
## Honest limitations
|
|
113
117
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "quantprobe"
|
|
7
|
-
version = "1.5.
|
|
7
|
+
version = "1.5.2"
|
|
8
8
|
description = "Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = { text = "MIT" }
|
|
@@ -8,4 +8,4 @@ Three commands, all grounded in measured laws (see LAWS.md in the repo):
|
|
|
8
8
|
Every constant in `plan` is fitted from measurements published in the repo, including two
|
|
9
9
|
pre-registered hardware predictions confirmed within 8%.
|
|
10
10
|
"""
|
|
11
|
-
__version__ = "1.5.
|
|
11
|
+
__version__ = "1.5.2"
|
|
@@ -37,6 +37,37 @@ def list_ggufs(repo):
|
|
|
37
37
|
and "mmproj" not in f["path"].lower() and "draft" not in f["path"].lower()]
|
|
38
38
|
|
|
39
39
|
|
|
40
|
+
WIKI_URL = "https://huggingface.co/datasets/ggml-org/ci/resolve/main/wikitext-2-raw-v1.zip"
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def ensure_eval(dest_dir):
|
|
44
|
+
"""The held-out corpus for probing, fetched once (1.3 MB)."""
|
|
45
|
+
import os, zipfile, io
|
|
46
|
+
path = os.path.join(dest_dir, "wiki.test.raw")
|
|
47
|
+
if os.path.isfile(path):
|
|
48
|
+
return path
|
|
49
|
+
print("[quantprobe auto] fetching the WikiText-2 eval corpus (1.3 MB, one time)...")
|
|
50
|
+
with urllib.request.urlopen(WIKI_URL, timeout=60) as r:
|
|
51
|
+
z = zipfile.ZipFile(io.BytesIO(r.read()))
|
|
52
|
+
with z.open("wikitext-2-raw/wiki.test.raw") as f, open(path, "wb") as out:
|
|
53
|
+
out.write(f.read())
|
|
54
|
+
return path
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def pick_source(files, t):
|
|
58
|
+
"""High-precision source for the custom build: prefer Q8-class (smallest requantizable), else max bits."""
|
|
59
|
+
cands = []
|
|
60
|
+
for path, size in files:
|
|
61
|
+
if "-of-" in path.lower():
|
|
62
|
+
continue # split multi-part files: out of scope for v1
|
|
63
|
+
bits = size * 8 / (t * 1e9)
|
|
64
|
+
cands.append((bits, size, path))
|
|
65
|
+
if not cands:
|
|
66
|
+
return None
|
|
67
|
+
hi = sorted([c for c in cands if c[0] >= 7.5], key=lambda c: c[1])
|
|
68
|
+
return hi[0] if hi else max(cands)
|
|
69
|
+
|
|
70
|
+
|
|
40
71
|
def run(a):
|
|
41
72
|
target = a.target
|
|
42
73
|
if target in MODEL_REPOS:
|
|
@@ -79,6 +110,40 @@ def run(a):
|
|
|
79
110
|
raise SystemExit("no usable quant in that repo for this machine")
|
|
80
111
|
scored.sort()
|
|
81
112
|
_, _, path, size, bits, best = scored[0]
|
|
113
|
+
|
|
114
|
+
if getattr(a, "custom", False):
|
|
115
|
+
src = pick_source(files, t)
|
|
116
|
+
if not src:
|
|
117
|
+
raise SystemExit("no usable high-precision source in that repo (split files unsupported)")
|
|
118
|
+
sbits, ssize, spath = src
|
|
119
|
+
import os
|
|
120
|
+
dest = getattr(a, "dir", None) or "./models"
|
|
121
|
+
os.makedirs(dest, exist_ok=True)
|
|
122
|
+
print("\n[quantprobe auto --custom] source: " + spath + f" ({ssize/1e9:.1f} GB, {sbits:.1f}-bit)")
|
|
123
|
+
print(" pipeline: fetch source -> probe the fragile band (~30-60 min) -> build the")
|
|
124
|
+
print(" depth-aware GGUF. Interrupt anytime; the fetch resumes.")
|
|
125
|
+
if getattr(a, "dry", False):
|
|
126
|
+
print(" (--dry: nothing downloaded)")
|
|
127
|
+
return spath
|
|
128
|
+
from . import fetch as fetchmod
|
|
129
|
+
if not fetchmod.fetch(repo, dest, spath, fetchmod.token()):
|
|
130
|
+
raise SystemExit("source download failed (re-run: it resumes)")
|
|
131
|
+
srcfull = os.path.join(dest, os.path.basename(spath))
|
|
132
|
+
evalf = ensure_eval(dest)
|
|
133
|
+
from . import probe as probemod
|
|
134
|
+
import argparse
|
|
135
|
+
out = os.path.join(dest, os.path.basename(spath).rsplit(".gguf", 1)[0] + "-depthaware.gguf")
|
|
136
|
+
pa = argparse.Namespace(gguf=srcfull, bands=4, chunks=32, eval=evalf, ngl=99,
|
|
137
|
+
workdir=dest, llama_dir=getattr(a, "llama_dir", None),
|
|
138
|
+
apply=True, out=out, dry_run=False)
|
|
139
|
+
probemod.run(pa)
|
|
140
|
+
print("\n[quantprobe auto --custom] your personalized model:")
|
|
141
|
+
print(" quantprobe run --gguf " + out)
|
|
142
|
+
if getattr(a, "run", False):
|
|
143
|
+
from . import runtime
|
|
144
|
+
a.gguf = out; a.bits = None
|
|
145
|
+
runtime.run(a)
|
|
146
|
+
return out
|
|
82
147
|
print(f"\n[quantprobe auto] optimizer wants ~{want_bits:g}-bit; closest file in {repo}:")
|
|
83
148
|
print(f" {path} ({size/1e9:.1f} GB, {bits:.2f} effective bits)")
|
|
84
149
|
print(f" predicted on this machine: {best[1]:.1f} tok/s ({best[0]})")
|
|
@@ -96,8 +161,8 @@ def run(a):
|
|
|
96
161
|
full = os.path.join(dest, os.path.basename(path))
|
|
97
162
|
print(f"\n[quantprobe auto] ready. Run it:")
|
|
98
163
|
print(f" quantprobe run --gguf {full}")
|
|
99
|
-
print(
|
|
100
|
-
print(
|
|
164
|
+
print("\n Better quality at the SAME size: rerun with --custom - it probes YOUR model's")
|
|
165
|
+
print(" fragile band (~30-60 min) and builds a depth-aware GGUF personalized to it.")
|
|
101
166
|
if getattr(a, "run", False):
|
|
102
167
|
from . import runtime
|
|
103
168
|
a.gguf = full
|
|
@@ -99,6 +99,7 @@ def main():
|
|
|
99
99
|
au.add_argument("--tps", type=float, default=None, help="target tok/s for the optimizer")
|
|
100
100
|
au.add_argument("--max-quality", type=float, default=None)
|
|
101
101
|
au.add_argument("--allow-prune", action="store_true"); au.add_argument("--any-runtime", action="store_true")
|
|
102
|
+
au.add_argument("--custom", action="store_true", help="THE PRODUCT: fetch a high-precision source, probe YOUR model (~30-60 min), build a depth-aware GGUF personalized to it")
|
|
102
103
|
au.add_argument("--serve", action="store_true", help="with --run: llama-server instead of chat")
|
|
103
104
|
au.add_argument("--extra", default=None)
|
|
104
105
|
|
|
@@ -144,9 +144,27 @@ def qual_of(moe, bits):
|
|
|
144
144
|
return QUAL[moe][min(keys, key=lambda k: abs(k - bits))]
|
|
145
145
|
|
|
146
146
|
|
|
147
|
+
def check_presets(args):
|
|
148
|
+
"""Refuse unknown preset names LOUDLY instead of silently falling back to defaults."""
|
|
149
|
+
mdl = getattr(args, "model", None)
|
|
150
|
+
if mdl and mdl not in MODELS and not getattr(args, "total", None):
|
|
151
|
+
raise SystemExit(
|
|
152
|
+
"unknown --model '%s' (and no --total to describe it).\n"
|
|
153
|
+
" presets: %s\n"
|
|
154
|
+
" or describe it: --total <B> --active <B> [--always-active <B>]\n"
|
|
155
|
+
" or point at a file: --gguf model.gguf (exact spec read from the GGUF)" % (mdl, ", ".join(sorted(MODELS))))
|
|
156
|
+
mac = getattr(args, "machine", None)
|
|
157
|
+
if mac and mac not in MACHINES:
|
|
158
|
+
raise SystemExit(
|
|
159
|
+
"unknown --machine '%s'.\n"
|
|
160
|
+
" presets: %s\n"
|
|
161
|
+
" or pass flags: --vram/--vram-bw/--ram/--ram-bw/--disk-bw (no flags = auto-detect this box)" % (mac, ", ".join(sorted(MACHINES))))
|
|
162
|
+
|
|
163
|
+
|
|
147
164
|
def run(args):
|
|
148
165
|
from . import spec as specmod
|
|
149
166
|
specmod.apply(args)
|
|
167
|
+
check_presets(args)
|
|
150
168
|
if getattr(args, "bits", None) is None:
|
|
151
169
|
args.bits = 2.5
|
|
152
170
|
m = dict(MODELS[args.model]) if args.model in MODELS else {}
|
|
@@ -52,6 +52,8 @@ def feasible(a, tps_target):
|
|
|
52
52
|
|
|
53
53
|
|
|
54
54
|
def run(a):
|
|
55
|
+
from . import plan as planmod
|
|
56
|
+
planmod.check_presets(a)
|
|
55
57
|
hw, vc, vb, rc, rb, db, geta, gl = hw_of(a)
|
|
56
58
|
rows = feasible(a, a.tps)
|
|
57
59
|
print(f"\nquantprobe target - smartest model meeting >= {a.tps:g} tok/s "
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: quantprobe
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.2
|
|
4
4
|
Summary: Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes.
|
|
5
5
|
Author: Federico Sciuca
|
|
6
6
|
License: MIT
|
|
@@ -30,7 +30,7 @@ Dynamic: license-file
|
|
|
30
30
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
31
31
|
|
|
32
32
|
|
|
33
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
33
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
34
34
|
|
|
35
35
|
|
|
36
36
|
---
|
|
@@ -67,7 +67,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
67
67
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
68
68
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
69
69
|
|
|
70
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
70
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
71
|
+
|
|
72
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
71
73
|
|
|
72
74
|
|
|
73
75
|
## The four placement laws
|
|
@@ -96,6 +98,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
96
98
|
|
|
97
99
|
```bash
|
|
98
100
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
101
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
99
102
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
100
103
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
101
104
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -118,14 +121,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
118
121
|
|
|
119
122
|
| | |
|
|
120
123
|
|---|---|
|
|
121
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
124
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
122
125
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
123
126
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
124
127
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
125
128
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
126
129
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
130
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
127
131
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
128
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
132
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
129
133
|
|
|
130
134
|
## Honest limitations
|
|
131
135
|
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|