quantprobe 1.5.1__tar.gz → 1.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {quantprobe-1.5.1/quantprobe.egg-info → quantprobe-1.6.0}/PKG-INFO +9 -6
- {quantprobe-1.5.1 → quantprobe-1.6.0}/README.md +8 -5
- {quantprobe-1.5.1 → quantprobe-1.6.0}/pyproject.toml +1 -1
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/__init__.py +1 -1
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/auto.py +35 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/cli.py +4 -2
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/optimize.py +1 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/plan.py +18 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/target.py +2 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0/quantprobe.egg-info}/PKG-INFO +9 -6
- {quantprobe-1.5.1 → quantprobe-1.6.0}/LICENSE +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/dashboard.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/detect.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/fetch.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/probe.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/runtime.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe/spec.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe.egg-info/SOURCES.txt +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe.egg-info/dependency_links.txt +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe.egg-info/entry_points.txt +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe.egg-info/requires.txt +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/quantprobe.egg-info/top_level.txt +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/setup.cfg +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/tests/test_engine.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/tests/test_evaluator.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/tests/test_pipeline.py +0 -0
- {quantprobe-1.5.1 → quantprobe-1.6.0}/tests/test_transforms.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: quantprobe
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.6.0
|
|
4
4
|
Summary: Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes.
|
|
5
5
|
Author: Federico Sciuca
|
|
6
6
|
License: MIT
|
|
@@ -30,7 +30,7 @@ Dynamic: license-file
|
|
|
30
30
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
31
31
|
|
|
32
32
|
|
|
33
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
33
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
34
34
|
|
|
35
35
|
|
|
36
36
|
---
|
|
@@ -67,7 +67,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
67
67
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
68
68
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
69
69
|
|
|
70
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
70
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
71
|
+
|
|
72
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
71
73
|
|
|
72
74
|
|
|
73
75
|
## The four placement laws
|
|
@@ -96,7 +98,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
96
98
|
|
|
97
99
|
```bash
|
|
98
100
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
99
|
-
quantprobe auto qwen3-30b --custom
|
|
101
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
100
102
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
101
103
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
102
104
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -119,14 +121,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
119
121
|
|
|
120
122
|
| | |
|
|
121
123
|
|---|---|
|
|
122
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
124
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
123
125
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
124
126
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
125
127
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
126
128
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
127
129
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
130
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
128
131
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
129
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
132
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
130
133
|
|
|
131
134
|
## Honest limitations
|
|
132
135
|
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
13
13
|
|
|
14
14
|
|
|
15
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
15
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
16
16
|
|
|
17
17
|
|
|
18
18
|
---
|
|
@@ -49,7 +49,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
49
49
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
50
50
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
51
51
|
|
|
52
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
52
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
53
|
+
|
|
54
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
53
55
|
|
|
54
56
|
|
|
55
57
|
## The four placement laws
|
|
@@ -78,7 +80,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
78
80
|
|
|
79
81
|
```bash
|
|
80
82
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
81
|
-
quantprobe auto qwen3-30b --custom
|
|
83
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
82
84
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
83
85
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
84
86
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -101,14 +103,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
101
103
|
|
|
102
104
|
| | |
|
|
103
105
|
|---|---|
|
|
104
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
106
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
105
107
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
106
108
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
107
109
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
108
110
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
109
111
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
112
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
110
113
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
111
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
114
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
112
115
|
|
|
113
116
|
## Honest limitations
|
|
114
117
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "quantprobe"
|
|
7
|
-
version = "1.
|
|
7
|
+
version = "1.6.0"
|
|
8
8
|
description = "Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = { text = "MIT" }
|
|
@@ -8,4 +8,4 @@ Three commands, all grounded in measured laws (see LAWS.md in the repo):
|
|
|
8
8
|
Every constant in `plan` is fitted from measurements published in the repo, including two
|
|
9
9
|
pre-registered hardware predictions confirmed within 8%.
|
|
10
10
|
"""
|
|
11
|
-
__version__ = "1.
|
|
11
|
+
__version__ = "1.6.0"
|
|
@@ -68,7 +68,36 @@ def pick_source(files, t):
|
|
|
68
68
|
return hi[0] if hi else max(cands)
|
|
69
69
|
|
|
70
70
|
|
|
71
|
+
def _wizard(a):
|
|
72
|
+
"""No model given: detect, ask two questions, go. The full pipeline with zero flags."""
|
|
73
|
+
print("quantprobe auto - interactive (no flags needed; `quantprobe auto --help` for the full list)")
|
|
74
|
+
try:
|
|
75
|
+
from . import detect as detmod
|
|
76
|
+
d, _ = detmod.detect()
|
|
77
|
+
print(f"\n this machine [auto-detected]: {d['vram']:g} GB VRAM @ {d['vram_bw']:g} GB/s | "
|
|
78
|
+
f"{d['ram']:g} GB RAM @ {d['ram_bw']:g} GB/s | disk {d['disk_bw']:g} GB/s")
|
|
79
|
+
except Exception:
|
|
80
|
+
pass
|
|
81
|
+
try:
|
|
82
|
+
m = input(f"\n model - preset ({', '.join(MODEL_REPOS)}) or HF GGUF repo id [qwen3-30b]: ").strip() or "qwen3-30b"
|
|
83
|
+
print("\n [1] best standard quant for this machine, ready to run (skips quantization)")
|
|
84
|
+
print(" [2] full custom: probe YOUR model's fragile layers (~30-60 min), build its personalized GGUF")
|
|
85
|
+
print(" [3] hit a speed target (asks for tok/s)")
|
|
86
|
+
c = input(" choice [1]: ").strip() or "1"
|
|
87
|
+
if c == "2":
|
|
88
|
+
a.custom = True
|
|
89
|
+
elif c == "3":
|
|
90
|
+
a.tps = float(input(" target tok/s: ").strip())
|
|
91
|
+
r = input(" launch chat when ready? [Y/n]: ").strip().lower()
|
|
92
|
+
a.run = r != "n"
|
|
93
|
+
except EOFError:
|
|
94
|
+
raise SystemExit("no model given and no terminal to ask. Pass one: quantprobe auto qwen3-30b [--custom] [--run]")
|
|
95
|
+
a.target = m
|
|
96
|
+
|
|
97
|
+
|
|
71
98
|
def run(a):
|
|
99
|
+
if a.target is None:
|
|
100
|
+
_wizard(a)
|
|
72
101
|
target = a.target
|
|
73
102
|
if target in MODEL_REPOS:
|
|
74
103
|
repo, t, ac, ne, moe = MODEL_REPOS[target]
|
|
@@ -111,6 +140,12 @@ def run(a):
|
|
|
111
140
|
scored.sort()
|
|
112
141
|
_, _, path, size, bits, best = scored[0]
|
|
113
142
|
|
|
143
|
+
if getattr(a, "custom", False) and want_bits >= 3.5 and not getattr(a, "force_custom", False):
|
|
144
|
+
print(f"\n[quantprobe auto --custom] this machine doesn't need the surgery: the optimizer")
|
|
145
|
+
print(f" wants ~{want_bits:g}-bit here, and at >=3.5 bits standard community quants match the")
|
|
146
|
+
print(f" depth-aware recipe on quality - the fragile-band fix only pays below ~3 bits (Laws 1-2).")
|
|
147
|
+
print(f" Fetching the optimal standard quant instead. Build anyway: --force-custom.")
|
|
148
|
+
a.custom = False
|
|
114
149
|
if getattr(a, "custom", False):
|
|
115
150
|
src = pick_source(files, t)
|
|
116
151
|
if not src:
|
|
@@ -92,14 +92,16 @@ def main():
|
|
|
92
92
|
hwargs(o)
|
|
93
93
|
|
|
94
94
|
au = sub.add_parser("auto", help="ONE command: model in, running setup out - optimizer picks the bits, the closest quant is fetched, run command printed")
|
|
95
|
-
au.add_argument("target",
|
|
95
|
+
au.add_argument("target", nargs="?", default=None,
|
|
96
|
+
help="model preset (qwen3-30b, qwen3-coder, glm-air, laguna-s, gemma-12b, mistral-7b) or a HF GGUF repo id; omit it and quantprobe ASKS (interactive)")
|
|
96
97
|
au.add_argument("--dir", default="./models", help="download directory (default ./models)")
|
|
97
98
|
au.add_argument("--run", action="store_true", help="launch chat immediately after the download")
|
|
98
99
|
hwargs(au)
|
|
99
100
|
au.add_argument("--tps", type=float, default=None, help="target tok/s for the optimizer")
|
|
100
101
|
au.add_argument("--max-quality", type=float, default=None)
|
|
101
102
|
au.add_argument("--allow-prune", action="store_true"); au.add_argument("--any-runtime", action="store_true")
|
|
102
|
-
au.add_argument("--custom", action="store_true", help="THE PRODUCT: fetch a high-precision source, probe YOUR model (~30-60 min), build a depth-aware GGUF personalized to it")
|
|
103
|
+
au.add_argument("--custom", action="store_true", help="THE PRODUCT: fetch a high-precision source, probe YOUR model (~30-60 min), build a depth-aware GGUF personalized to it (machine-gated: skipped with an explanation when this box doesn't need sub-3-bit surgery)")
|
|
104
|
+
au.add_argument("--force-custom", action="store_true", help="build the depth-aware GGUF even when the optimizer says this machine doesn't need it")
|
|
103
105
|
au.add_argument("--serve", action="store_true", help="with --run: llama-server instead of chat")
|
|
104
106
|
au.add_argument("--extra", default=None)
|
|
105
107
|
|
|
@@ -144,9 +144,27 @@ def qual_of(moe, bits):
|
|
|
144
144
|
return QUAL[moe][min(keys, key=lambda k: abs(k - bits))]
|
|
145
145
|
|
|
146
146
|
|
|
147
|
+
def check_presets(args):
|
|
148
|
+
"""Refuse unknown preset names LOUDLY instead of silently falling back to defaults."""
|
|
149
|
+
mdl = getattr(args, "model", None)
|
|
150
|
+
if mdl and mdl not in MODELS and not getattr(args, "total", None):
|
|
151
|
+
raise SystemExit(
|
|
152
|
+
"unknown --model '%s' (and no --total to describe it).\n"
|
|
153
|
+
" presets: %s\n"
|
|
154
|
+
" or describe it: --total <B> --active <B> [--always-active <B>]\n"
|
|
155
|
+
" or point at a file: --gguf model.gguf (exact spec read from the GGUF)" % (mdl, ", ".join(sorted(MODELS))))
|
|
156
|
+
mac = getattr(args, "machine", None)
|
|
157
|
+
if mac and mac not in MACHINES:
|
|
158
|
+
raise SystemExit(
|
|
159
|
+
"unknown --machine '%s'.\n"
|
|
160
|
+
" presets: %s\n"
|
|
161
|
+
" or pass flags: --vram/--vram-bw/--ram/--ram-bw/--disk-bw (no flags = auto-detect this box)" % (mac, ", ".join(sorted(MACHINES))))
|
|
162
|
+
|
|
163
|
+
|
|
147
164
|
def run(args):
|
|
148
165
|
from . import spec as specmod
|
|
149
166
|
specmod.apply(args)
|
|
167
|
+
check_presets(args)
|
|
150
168
|
if getattr(args, "bits", None) is None:
|
|
151
169
|
args.bits = 2.5
|
|
152
170
|
m = dict(MODELS[args.model]) if args.model in MODELS else {}
|
|
@@ -52,6 +52,8 @@ def feasible(a, tps_target):
|
|
|
52
52
|
|
|
53
53
|
|
|
54
54
|
def run(a):
|
|
55
|
+
from . import plan as planmod
|
|
56
|
+
planmod.check_presets(a)
|
|
55
57
|
hw, vc, vb, rc, rb, db, geta, gl = hw_of(a)
|
|
56
58
|
rows = feasible(a, a.tps)
|
|
57
59
|
print(f"\nquantprobe target - smartest model meeting >= {a.tps:g} tok/s "
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: quantprobe
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.6.0
|
|
4
4
|
Summary: Probe-then-quantize for LLMs: measure a model's fragility curve, plan bit/tier placement by the tiered decode law, and emit ready-to-run llama.cpp recipes.
|
|
5
5
|
Author: Federico Sciuca
|
|
6
6
|
License: MIT
|
|
@@ -30,7 +30,7 @@ Dynamic: license-file
|
|
|
30
30
|
> Pick any model + your machine → predicted tok/s, memory fit, quality cost, and your cheapest next upgrade — from the law below, with your config plotted against every validated measurement.
|
|
31
31
|
|
|
32
32
|
|
|
33
|
-
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds
|
|
33
|
+
**New here? → [QUICKSTART.md](QUICKSTART.md) gets you running in 60 seconds — or just `pip install quantprobe && quantprobe auto qwen3-coder --run`.** The calculators (`hw`/`plan`/`target`/`optimize`) need nothing installed; the weight-touching commands drive [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) (point quantprobe at it with `--llama-dir`, `QUANTPROBE_LLAMA_DIR`, or `PATH`; preview anything with `--dry`). 16 machine presets ship in (`--machine`): GTX 1060 → RTX 5090, Apple `mac-m2/m3/m4-*`, DGX Spark, Epyc — or pass raw specs. Multi-GPU / RAID? Comma lists aggregate: `--vram 24,24 --disk-bw 14,14`. Big-VRAM + disk-streaming rigs get the three-tier expert-cache row (v1.3).
|
|
34
34
|
|
|
35
35
|
|
|
36
36
|
---
|
|
@@ -67,7 +67,9 @@ Add to that: a **byte-identical control** (two GGUFs the same size, 2.25 ppl apa
|
|
|
67
67
|
<p align="center"><img src="weights/data/validation_19tok/live_run_20tps.png" width="880" alt="One frame: Task Manager showing 16 GB DDR4-3000 and the GTX 1060 6GB beside llama.cpp chatting Qwen3-30B-A3B live at 20.4 tok/s generation"></p>
|
|
68
68
|
<p align="center"><em>One frame, no cuts: Task Manager (16 GB @ 3000 MT/s, GTX 1060 6 GB, RAM at 91% — the hybrid placement using the whole machine) beside llama.cpp chatting Qwen3-30B-A3B at <b>20.4 tok/s generation</b> — above the pre-registered 19. Raw logs, hardware attestation + GGUF SHA256: <a href="weights/data/validation_19tok/EVIDENCE.txt">EVIDENCE.txt</a>. Third bench run: 19.26 ± 0.45 (series 19.30 → 19.55 → 19.26).</em></p>
|
|
69
69
|
|
|
70
|
-
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism.
|
|
70
|
+
**Open pre-registrations** — predictions staked publicly *before* measurement: [colibri v1.1, five falsifiable predictions (2026-07-23)](preregistrations/2026-07-23-colibri-v1.1.md) — dual-SSD scaling, int3 speedup, lattice-vs-scalar, AVX-512 tier-scoping, MTP×MoE antagonism — plus open bands on the CPU-only context slope and expert-pruning decode ([all verdicts](preregistrations/)).
|
|
71
|
+
|
|
72
|
+
**Live research track — Law 5 (prefill):** protocol-first and scored same-day in [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) — 6 staked hits, 4 published misses, and one self-caught retraction corrected in public within the hour, including the measured recipe that cuts an agent's 8k-context turn cost from ~75 s to ~0.5 s. Next up, staked before launch: [Law 6 — speculation economics](preregistrations/2026-07-24-law6-speculation-economics.md), scoring in public the week of 2026-07-27.
|
|
71
73
|
|
|
72
74
|
|
|
73
75
|
## The four placement laws
|
|
@@ -96,7 +98,7 @@ quantprobe auto qwen3-coder --tps 15 --run # empty machine -> optimal quant ch
|
|
|
96
98
|
|
|
97
99
|
```bash
|
|
98
100
|
quantprobe auto qwen3-30b --tps 15 # ONE command: optimizer picks bits, closest quant fetched, run command printed
|
|
99
|
-
quantprobe auto qwen3-30b --custom
|
|
101
|
+
quantprobe auto qwen3-30b --custom --run # THE PRODUCT: probe YOUR model, build its personalized depth-aware GGUF, launch it
|
|
100
102
|
quantprobe hw # what the law sees on THIS machine (every value source-tagged)
|
|
101
103
|
quantprobe plan --gguf model.gguf # zero-config prediction: placement + tok/s + the launch command
|
|
102
104
|
quantprobe optimize --tps 20 # CHEAPEST PATH to a target: bits x placement x hardware, Pareto-ranked
|
|
@@ -119,14 +121,15 @@ The loop is self-validating: `plan` predicted 17.5 for a file we then measured a
|
|
|
119
121
|
|
|
120
122
|
| | |
|
|
121
123
|
|---|---|
|
|
122
|
-
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels;
|
|
124
|
+
| [QUICKSTART.md](QUICKSTART.md) | 60-second start, three levels; recipes (own fine-tune, coding agents, hardware buying); Ollama interop |
|
|
123
125
|
| [LAWS.md](LAWS.md) | the four laws — statements, measurements, falsifiable predictions, the general form |
|
|
124
126
|
| [docs/EXAMPLES.md](docs/EXAMPLES.md) | worked examples with real outputs: zero-config, the ×5.4 optimizer A/B, probe walkthrough, troubleshooting |
|
|
125
127
|
| [docs/HARDWARE.md](docs/HARDWARE.md) | the 2016 box: exact specs, measured bandwidths, what the next euro buys |
|
|
126
128
|
| [docs/DEEP-DIVE.md](docs/DEEP-DIVE.md) | what's new vs. what's built-on, parity tables, the 744B-at-home projection, repository map |
|
|
127
129
|
| [preregistrations/](preregistrations/) | every staked prediction with its verdict — hits, the near-miss, and the honest miss |
|
|
130
|
+
| [weights/LAW5_PROTOCOL.md](weights/LAW5_PROTOCOL.md) | the live Law-5 research ledger — conventions, stakes, scores, and the retraction |
|
|
128
131
|
| [papers/arxiv/](papers/arxiv/) | the paper (submission-ready LaTeX) |
|
|
129
|
-
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.
|
|
132
|
+
| [CHANGELOG.md](CHANGELOG.md) | v1.0 → v1.5, every release |
|
|
130
133
|
|
|
131
134
|
## Honest limitations
|
|
132
135
|
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|