effbelt 0.1.0__tar.gz

effbelt-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sardor Razikov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

effbelt-0.1.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: effbelt
+Version: 0.1.0
+Summary: LLM efficiency toolbelt: fit, tokenwaste, evalsanity, quantcheck, cheapest, modelcard — save money and catch silent bugs, on a laptop.
+Author: Sardor Razikov
+License: MIT
+Project-URL: Homepage, https://github.com/SRKRZ23/effbelt
+Project-URL: Source, https://github.com/SRKRZ23/effbelt
+Project-URL: Sponsor, https://github.com/sponsors/SRKRZ23
+Keywords: llm,gpu,memory,kv-cache,quantization,cost,efficiency,benchmark
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: torch
+Requires-Dist: torch>=2.1; extra == "torch"
+Provides-Extra: hf
+Requires-Dist: torch>=2.1; extra == "hf"
+Requires-Dist: transformers>=5; extra == "hf"
+Requires-Dist: kvquant-hf; extra == "hf"
+Dynamic: license-file
+
+# effbelt — the LLM efficiency toolbelt
+
+Six small, honest tools that **save money and catch silent bugs** — all runnable
+on a laptop with **no GPU and no API key**. Four of them need nothing but Python;
+two can optionally measure real numbers on a tiny local model.
+
+```bash
+pip install effbelt            # the four pure tools
+pip install "effbelt[hf]"      # + quantcheck tensors and live modelcard
+```
+
+| tool | one line | needs a model? |
+|------|----------|:---:|
+| `fit` | will this model fit on my card? (weights + KV cache + overhead) | no |
+| `tokenwaste` | where your LLM **API bill leaks** (duplicate prompts, retries, oversized model) | no |
+| `evalsanity` | **is your benchmark lying?** (contamination, degenerate data/preds) | no |
+| `quantcheck` | **did your quantization actually do anything?** (silent no-op guard) | tensors only |
+| `cheapest` | the cheapest config that still clears **your** quality bar | your eval |
+| `modelcard` | one-page quant tradeoff card (exact memory + *measured* ppl) | optional |
+
+Every number here was produced on an Apple-silicon MacBook, offline. Nothing is invented.
+
+---
+
+## Why trust the memory math? It matches a real cache to the byte.
+
+`fit`'s KV-cache formula is cross-checked in the test suite against the **real
+measured bytes** of a live Hugging Face cache (via the sister project
+[kvquant](https://github.com/SRKRZ23/kvquant)):
+
+```
+KV formula check: analytic 180.0KB vs measured 180.0KB at seq=10 (0.00% off)
+```
+
+## `fit` — stop guessing at OOM
+
+```
+$ effbelt fit llama-3.1-70b --gpu rtx-4090 --seq 8192
+llama-3.1-70b  @  seq=8192 batch=1  weights=fp16 kv=fp16
+  weights   131.50 GB
+  kv cache    2.50 GB   (8 kv-heads GQA)
+  overhead    1.00 GB
+  --------  135.00 GB total
+  DOES NOT FIT on a 24 GB card  (headroom -111.00 GB)
+```
+```
+$ effbelt fit llama-3.1-8b --gpu rtx-4090 --seq 32768 --kv int8
+  ... FITS on a 24 GB card  (headroom +6.04 GB)
+```
+Use `--hf <repo>` to pull authoritative dimensions straight from a model's config.
+
+## `tokenwaste` — audit an API request log (JSONL)
+
+```
+$ effbelt tokenwaste requests.jsonl
+22 requests, ~$0.0780 total (prices as of 2026-10)
+  recoverable up to ~$0.0780 (<= 100% of bill; levers below can overlap):
+  - oversized-model   ~$0.0718  large model, <=20 output tok -> try a small model
+  - over-long-context ~$0.0300  context over 8000 tok
+  - no-prefix-cache   ~$0.0183  22x identical system prompt (~290 tok)
+  - retries           ~$0.0013  requests marked retry=true
+```
+The levers overlap on the same request, so the headline is an honest **upper
+bound** capped at 100% of the bill — never a made-up "save 300%".
+
+## `evalsanity` — before you believe a leaderboard number
+
+```
+$ effbelt evalsanity eval.json
+41 test items: 2 flag(s)
+  [HIGH] 1/41 test items appear VERBATIM in train -> contamination
+  [MED ] labels 93% one class ('A') -> imbalanced
+  -> at least one HIGH flag: do NOT trust this number yet.
+```
+
+## `quantcheck` — the silent no-op killer
+
+The lesson behind kvquant, generalized to **any** tensor transform you think
+quantizes. A no-op (e.g. you mutated a non-persisting view) passes every smoke
+test and moves no metric — quantcheck proves it *bit*, estimates effective
+bits, and flags the two classic bugs:
+
+```python
+from effbelt import quantcheck
+r = quantcheck.check(my_quant_fn, some_tensor, claimed_bits=8)
+print(r.report())
+# quantcheck: SILENT NO-OP   <- if your "quantizer" changed nothing
+# quantcheck: SUSPICIOUS     <- not idempotent / coarser than claimed
+# quantcheck: OK
+```
+
+## `modelcard` — know the tradeoff before you quantize
+
+Exact memory arithmetic, and *measured* quality when you point it at a model:
+
+```
+$ effbelt modelcard gemma-3-270m --seq 8192 --live google/gemma-3-270m-it
+  kv dtype   kv cache  vs fp16   quality
+  fp16         0.14GB    1.00x   +0.00% ppl
+  int8         0.07GB    2.00x   -0.33% ppl     <- free: 2x smaller, lossless
+  int4         0.04GB    4.00x  +68.41% ppl     <- breaks on this small model
+  quality measured on google/gemma-3-270m-it (160 tok, MPS/CPU)
+```
+
+## `cheapest` — smallest model that still passes
+
+```python
+from effbelt import cheapest
+res = cheapest.sweep([
+    cheapest.Candidate("7b-int4",  cost=1, quality_fn=lambda: my_eval("7b-int4")),
+    cheapest.Candidate("7b-fp16",  cost=2, quality_fn=lambda: my_eval("7b-fp16")),
+    cheapest.Candidate("70b-int4", cost=8, quality_fn=lambda: my_eval("70b-int4")),
+], bar=0.80)
+print(res.report())   # -> use the cheapest one that clears 0.80
+```
+
+---
+
+## Honest scope
+- Prices and the small model table are **approximate and editable** (see
+  `effbelt/models.py`); for authoritative model dims use `fit --hf`.
+- `quantcheck`'s effective-bit estimate is advisory at small group sizes — the
+  *silent-no-op* and *idempotency* checks are the reliable ones.
+- `modelcard` quality numbers are measured only when you pass `--live`; otherwise
+  the quality column is left blank. We don't fill it with guesses.
+
+## Tests
+All validation runs on a laptop, offline:
+```bash
+python -m pytest tests/ -v      # 18 tests, incl. the real-cache cross-check
+```
+
+## Support
+Free and MIT. If it saved you a GPU or a bill, consider sponsoring:
+**https://github.com/sponsors/SRKRZ23**
+
+Crypto (receive-only):
+- BTC `bc1q9hm4upksmvkzhl8z3zvxgz94da358qfe3tred0`
+- ETH / Polygon / USDT-ERC20 `0xa9e3b63799f5E5cc6258042D45869a5a1d39F021`
+- USDT-TRC20 `TUyNkMXme1CYAc1wZ8SZ64X9upBc9nwTZ5`
+- SOL `j8zZCsWwM8WmUZwMSDHikNuz6E49Vn8nwo9FMQnHSoW`
+
+Sister project: [kvquant](https://github.com/SRKRZ23/kvquant) — the engine behind
+`quantcheck` and `modelcard`.

effbelt-0.1.0/README.md

@@ -0,0 +1,144 @@
+# effbelt — the LLM efficiency toolbelt
+
+Six small, honest tools that **save money and catch silent bugs** — all runnable
+on a laptop with **no GPU and no API key**. Four of them need nothing but Python;
+two can optionally measure real numbers on a tiny local model.
+
+```bash
+pip install effbelt            # the four pure tools
+pip install "effbelt[hf]"      # + quantcheck tensors and live modelcard
+```
+
+| tool | one line | needs a model? |
+|------|----------|:---:|
+| `fit` | will this model fit on my card? (weights + KV cache + overhead) | no |
+| `tokenwaste` | where your LLM **API bill leaks** (duplicate prompts, retries, oversized model) | no |
+| `evalsanity` | **is your benchmark lying?** (contamination, degenerate data/preds) | no |
+| `quantcheck` | **did your quantization actually do anything?** (silent no-op guard) | tensors only |
+| `cheapest` | the cheapest config that still clears **your** quality bar | your eval |
+| `modelcard` | one-page quant tradeoff card (exact memory + *measured* ppl) | optional |
+
+Every number here was produced on an Apple-silicon MacBook, offline. Nothing is invented.
+
+---
+
+## Why trust the memory math? It matches a real cache to the byte.
+
+`fit`'s KV-cache formula is cross-checked in the test suite against the **real
+measured bytes** of a live Hugging Face cache (via the sister project
+[kvquant](https://github.com/SRKRZ23/kvquant)):
+
+```
+KV formula check: analytic 180.0KB vs measured 180.0KB at seq=10 (0.00% off)
+```
+
+## `fit` — stop guessing at OOM
+
+```
+$ effbelt fit llama-3.1-70b --gpu rtx-4090 --seq 8192
+llama-3.1-70b  @  seq=8192 batch=1  weights=fp16 kv=fp16
+  weights   131.50 GB
+  kv cache    2.50 GB   (8 kv-heads GQA)
+  overhead    1.00 GB
+  --------  135.00 GB total
+  DOES NOT FIT on a 24 GB card  (headroom -111.00 GB)
+```
+```
+$ effbelt fit llama-3.1-8b --gpu rtx-4090 --seq 32768 --kv int8
+  ... FITS on a 24 GB card  (headroom +6.04 GB)
+```
+Use `--hf <repo>` to pull authoritative dimensions straight from a model's config.
+
+## `tokenwaste` — audit an API request log (JSONL)
+
+```
+$ effbelt tokenwaste requests.jsonl
+22 requests, ~$0.0780 total (prices as of 2026-10)
+  recoverable up to ~$0.0780 (<= 100% of bill; levers below can overlap):
+  - oversized-model   ~$0.0718  large model, <=20 output tok -> try a small model
+  - over-long-context ~$0.0300  context over 8000 tok
+  - no-prefix-cache   ~$0.0183  22x identical system prompt (~290 tok)
+  - retries           ~$0.0013  requests marked retry=true
+```
+The levers overlap on the same request, so the headline is an honest **upper
+bound** capped at 100% of the bill — never a made-up "save 300%".
+
+## `evalsanity` — before you believe a leaderboard number
+
+```
+$ effbelt evalsanity eval.json
+41 test items: 2 flag(s)
+  [HIGH] 1/41 test items appear VERBATIM in train -> contamination
+  [MED ] labels 93% one class ('A') -> imbalanced
+  -> at least one HIGH flag: do NOT trust this number yet.
+```
+
+## `quantcheck` — the silent no-op killer
+
+The lesson behind kvquant, generalized to **any** tensor transform you think
+quantizes. A no-op (e.g. you mutated a non-persisting view) passes every smoke
+test and moves no metric — quantcheck proves it *bit*, estimates effective
+bits, and flags the two classic bugs:
+
+```python
+from effbelt import quantcheck
+r = quantcheck.check(my_quant_fn, some_tensor, claimed_bits=8)
+print(r.report())
+# quantcheck: SILENT NO-OP   <- if your "quantizer" changed nothing
+# quantcheck: SUSPICIOUS     <- not idempotent / coarser than claimed
+# quantcheck: OK
+```
+
+## `modelcard` — know the tradeoff before you quantize
+
+Exact memory arithmetic, and *measured* quality when you point it at a model:
+
+```
+$ effbelt modelcard gemma-3-270m --seq 8192 --live google/gemma-3-270m-it
+  kv dtype   kv cache  vs fp16   quality
+  fp16         0.14GB    1.00x   +0.00% ppl
+  int8         0.07GB    2.00x   -0.33% ppl     <- free: 2x smaller, lossless
+  int4         0.04GB    4.00x  +68.41% ppl     <- breaks on this small model
+  quality measured on google/gemma-3-270m-it (160 tok, MPS/CPU)
+```
+
+## `cheapest` — smallest model that still passes
+
+```python
+from effbelt import cheapest
+res = cheapest.sweep([
+    cheapest.Candidate("7b-int4",  cost=1, quality_fn=lambda: my_eval("7b-int4")),
+    cheapest.Candidate("7b-fp16",  cost=2, quality_fn=lambda: my_eval("7b-fp16")),
+    cheapest.Candidate("70b-int4", cost=8, quality_fn=lambda: my_eval("70b-int4")),
+], bar=0.80)
+print(res.report())   # -> use the cheapest one that clears 0.80
+```
+
+---
+
+## Honest scope
+- Prices and the small model table are **approximate and editable** (see
+  `effbelt/models.py`); for authoritative model dims use `fit --hf`.
+- `quantcheck`'s effective-bit estimate is advisory at small group sizes — the
+  *silent-no-op* and *idempotency* checks are the reliable ones.
+- `modelcard` quality numbers are measured only when you pass `--live`; otherwise
+  the quality column is left blank. We don't fill it with guesses.
+
+## Tests
+All validation runs on a laptop, offline:
+```bash
+python -m pytest tests/ -v      # 18 tests, incl. the real-cache cross-check
+```
+
+## Support
+Free and MIT. If it saved you a GPU or a bill, consider sponsoring:
+**https://github.com/sponsors/SRKRZ23**
+
+Crypto (receive-only):
+- BTC `bc1q9hm4upksmvkzhl8z3zvxgz94da358qfe3tred0`
+- ETH / Polygon / USDT-ERC20 `0xa9e3b63799f5E5cc6258042D45869a5a1d39F021`
+- USDT-TRC20 `TUyNkMXme1CYAc1wZ8SZ64X9upBc9nwTZ5`
+- SOL `j8zZCsWwM8WmUZwMSDHikNuz6E49Vn8nwo9FMQnHSoW`
+
+Sister project: [kvquant](https://github.com/SRKRZ23/kvquant) — the engine behind
+`quantcheck` and `modelcard`.

effbelt-0.1.0/effbelt/__init__.py

@@ -0,0 +1,21 @@
+"""effbelt — the LLM efficiency toolbelt.
+
+Six small, honest tools that save money and catch silent bugs, all runnable
+on a laptop with no GPU and no API key:
+
+  fit         will this model fit on my card? (weights + KV cache + overhead)
+  tokenwaste  where your LLM API bill leaks (duplicate prompts, retries, ...)
+  evalsanity  is your benchmark lying? (contamination, degenerate data/preds)
+  quantcheck  did your quantization actually do anything? (silent no-op guard)
+  cheapest    the cheapest config that still clears your quality bar
+  modelcard   one-page quant tradeoff card (arithmetic memory + measured ppl)
+
+Sister project: kvquant (the engine behind quantcheck/modelcard).
+MIT. If it saved you a GPU, consider sponsoring.
+"""
+__version__ = "0.1.0"
+
+from . import memory, tokenwaste, evalsanity, quantcheck, cheapest, modelcard
+
+__all__ = ["memory", "tokenwaste", "evalsanity", "quantcheck", "cheapest",
+           "modelcard", "__version__"]

effbelt-0.1.0/effbelt/cheapest.py

@@ -0,0 +1,57 @@
+"""`cheapest` — the smallest/cheapest config that still passes YOUR bar.
+
+You give candidate configs (each with a relative cost and a way to score
+quality on your own task) and a quality bar. cheapest runs the sweep and
+returns the lowest-cost config that clears the bar. The quality function is
+yours (an eval on your data); the logic here is just the honest sweep +
+pick, so the recommendation is only as good as your bar and your eval.
+"""
+from __future__ import annotations
+from dataclasses import dataclass
+from typing import Callable
+
+
+@dataclass
+class Candidate:
+    name: str
+    cost: float                 # relative $ or GB or latency -- your unit
+    quality_fn: Callable[[], float]   # returns a score; higher is better
+
+
+@dataclass
+class SweepRow:
+    name: str
+    cost: float
+    quality: float
+    passes: bool
+
+
+@dataclass
+class CheapestResult:
+    bar: float
+    rows: list[SweepRow]
+    pick: SweepRow | None
+
+    def report(self) -> str:
+        lines = [f"quality bar = {self.bar:g}"]
+        for r in sorted(self.rows, key=lambda x: x.cost):
+            mark = "PASS" if r.passes else "fail"
+            star = "  <= cheapest passing" if self.pick and r.name == self.pick.name else ""
+            lines.append(f"  cost {r.cost:8.3f}  q={r.quality:.4f}  [{mark}]  "
+                         f"{r.name}{star}")
+        if self.pick:
+            lines.append(f"-> use {self.pick.name!r} (cost {self.pick.cost:g}, "
+                         f"q={self.pick.quality:.4f})")
+        else:
+            lines.append("-> NOTHING clears the bar; raise budget or lower the bar.")
+        return "\n".join(lines)
+
+
+def sweep(candidates: list[Candidate], bar: float) -> CheapestResult:
+    rows = []
+    for c in candidates:
+        q = float(c.quality_fn())
+        rows.append(SweepRow(c.name, c.cost, q, q >= bar))
+    passing = [r for r in rows if r.passes]
+    pick = min(passing, key=lambda r: r.cost) if passing else None
+    return CheapestResult(bar, rows, pick)

effbelt-0.1.0/effbelt/cli.py

@@ -0,0 +1,101 @@
+"""effbelt command-line interface: `effbelt <tool> ...`."""
+from __future__ import annotations
+import argparse
+import sys
+
+
+def _cmd_fit(a):
+    from .memory import fit, shape_from_hf
+    from .models import GPUS
+    gpu = a.gpu
+    if isinstance(gpu, str):
+        if gpu.lower() not in GPUS:
+            print(f"unknown gpu {gpu!r}; known: {sorted(GPUS)}", file=sys.stderr)
+            return 2
+        gpu = GPUS[gpu.lower()]
+    shape = shape_from_hf(a.hf) if a.hf else a.model
+    r = fit(shape, gpu_gb=float(gpu), seq_len=a.seq, batch=a.batch,
+            weight_dtype=a.wdtype, kv_dtype=a.kv, overhead_gb=a.overhead)
+    print(r.report())
+    return 0
+
+
+def _cmd_tokenwaste(a):
+    from .tokenwaste import analyze, load_jsonl
+    recs = load_jsonl(a.log)
+    print(analyze(recs, long_context_tokens=a.long_context).report())
+    return 0
+
+
+def _cmd_evalsanity(a):
+    from .evalsanity import check
+    import json
+    with open(a.data, encoding="utf-8") as f:
+        d = json.load(f)
+    rep = check(d["test"], d.get("train"), d.get("labels"), d.get("predictions"))
+    print(rep.report())
+    return 0 if rep.clean else 1
+
+
+def _cmd_modelcard(a):
+    from .modelcard import build, measure_kv_quality
+    measured, on = None, None
+    if a.live:
+        measured = measure_kv_quality(a.live, max_tokens=a.max_tokens)
+        on = f"{a.live} ({a.max_tokens} tok, MPS/CPU)"
+    print(build(a.model, seq_len=a.seq, batch=a.batch,
+                weight_dtype=a.wdtype, measured_ppl=measured,
+                measured_on=on).report())
+    return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(prog="effbelt",
+                                description="LLM efficiency toolbelt")
+    sub = p.add_subparsers(dest="cmd", required=True)
+
+    f = sub.add_parser("fit", help="will this model fit on my card?")
+    f.add_argument("model", help="known model name, or use --hf")
+    f.add_argument("--hf", help="authoritative dims from a HF repo id")
+    f.add_argument("--gpu", default="rtx-4090", help="gpu name or GB number")
+    f.add_argument("--seq", type=int, default=4096)
+    f.add_argument("--batch", type=int, default=1)
+    f.add_argument("--wdtype", default="fp16")
+    f.add_argument("--kv", default="fp16")
+    f.add_argument("--overhead", type=float, default=1.0)
+    f.set_defaults(func=_cmd_fit)
+
+    t = sub.add_parser("tokenwaste", help="audit an API request log (jsonl)")
+    t.add_argument("log")
+    t.add_argument("--long-context", type=int, default=8000)
+    t.set_defaults(func=_cmd_tokenwaste)
+
+    e = sub.add_parser("evalsanity", help="is your benchmark lying?")
+    e.add_argument("data", help="json with keys: test[, train, labels, predictions]")
+    e.set_defaults(func=_cmd_evalsanity)
+
+    m = sub.add_parser("modelcard", help="quant tradeoff card")
+    m.add_argument("model")
+    m.add_argument("--seq", type=int, default=4096)
+    m.add_argument("--batch", type=int, default=1)
+    m.add_argument("--wdtype", default="fp16")
+    m.add_argument("--live", help="HF repo id to MEASURE real ppl deltas")
+    m.add_argument("--max-tokens", type=int, default=200)
+    m.set_defaults(func=_cmd_modelcard)
+
+    return p
+
+
+def main(argv=None) -> int:
+    args = build_parser().parse_args(argv)
+    try:
+        gpu = getattr(args, "gpu", None)
+        if gpu is not None and gpu.replace(".", "", 1).isdigit():
+            args.gpu = float(gpu)
+    except Exception:
+        pass
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

effbelt-0.1.0/effbelt/evalsanity.py

@@ -0,0 +1,115 @@
+"""`evalsanity` — is your benchmark lying to you?
+
+Cheap, deterministic checks that catch the usual ways an eval number is
+inflated or meaningless BEFORE you trust it:
+  - train/test contamination (exact + near-dup via shingled Jaccard)
+  - degenerate data (duplicate rows, near-constant labels)
+  - degenerate predictions (model emits one answer, or copies the input)
+  - too-good-to-be-true accuracy on a tiny / imbalanced set
+No model required. Works on lists of strings.
+"""
+from __future__ import annotations
+from dataclasses import dataclass, field
+import re
+
+
+def _shingles(text: str, k: int = 5) -> set[str]:
+    toks = re.findall(r"\w+", text.lower())
+    if len(toks) < k:
+        return {" ".join(toks)} if toks else set()
+    return {" ".join(toks[i:i + k]) for i in range(len(toks) - k + 1)}
+
+
+def _jaccard(a: set[str], b: set[str]) -> float:
+    if not a or not b:
+        return 0.0
+    return len(a & b) / len(a | b)
+
+
+@dataclass
+class SanityReport:
+    n_test: int
+    flags: list[tuple[str, str]] = field(default_factory=list)  # (severity, msg)
+
+    @property
+    def clean(self) -> bool:
+        return not any(s == "HIGH" for s, _ in self.flags)
+
+    def report(self) -> str:
+        if not self.flags:
+            return f"{self.n_test} test items: no sanity problems found."
+        lines = [f"{self.n_test} test items: {len(self.flags)} flag(s)"]
+        for sev, msg in sorted(self.flags, key=lambda x: {"HIGH": 0, "MED": 1,
+                                                          "LOW": 2}[x[0]]):
+            lines.append(f"  [{sev:4s}] {msg}")
+        if not self.clean:
+            lines.append("  -> at least one HIGH flag: do NOT trust this number yet.")
+        return "\n".join(lines)
+
+
+def check(test_texts: list[str],
+          train_texts: list[str] | None = None,
+          labels: list | None = None,
+          predictions: list | None = None,
+          near_dup_threshold: float = 0.8) -> SanityReport:
+    flags: list[tuple[str, str]] = []
+    n = len(test_texts)
+
+    # ---- contamination ----
+    if train_texts:
+        train_set = set(t.strip() for t in train_texts)
+        exact = sum(1 for t in test_texts if t.strip() in train_set)
+        if exact:
+            flags.append(("HIGH", f"{exact}/{n} test items appear VERBATIM in "
+                                  f"train -> contamination"))
+        # near-dup (sampled to stay cheap)
+        tr_sh = [_shingles(t) for t in train_texts[:2000]]
+        near = 0
+        for t in test_texts[:2000]:
+            sh = _shingles(t)
+            if any(_jaccard(sh, trs) >= near_dup_threshold for trs in tr_sh):
+                near += 1
+        if near > exact:
+            flags.append(("HIGH", f"{near}/{min(n,2000)} test items are NEAR "
+                                  f"-duplicates of train (Jaccard>="
+                                  f"{near_dup_threshold})"))
+
+    # ---- degenerate test data ----
+    uniq = len(set(test_texts))
+    if n and uniq < n:
+        flags.append(("MED", f"{n-uniq}/{n} duplicate test items"))
+    if labels is not None:
+        from collections import Counter
+        c = Counter(labels)
+        top, topn = c.most_common(1)[0]
+        frac = topn / len(labels)
+        if frac >= 0.95:
+            flags.append(("HIGH", f"labels {frac*100:.0f}% one class ({top!r}) "
+                                  f"-> accuracy is meaningless; use balanced metric"))
+        elif frac >= 0.8:
+            flags.append(("MED", f"labels {frac*100:.0f}% one class ({top!r}) "
+                                 f"-> imbalanced"))
+
+    # ---- degenerate predictions ----
+    if predictions is not None:
+        from collections import Counter
+        pc = Counter(predictions)
+        ptop, ptopn = pc.most_common(1)[0]
+        if ptopn / len(predictions) >= 0.95 and len(pc) > 1:
+            flags.append(("HIGH", f"predictions {100*ptopn/len(predictions):.0f}% "
+                                  f"a single value ({ptop!r}) -> model collapsed"))
+        elif len(pc) == 1:
+            flags.append(("HIGH", f"predictions are ALL the same value ({ptop!r})"))
+        # copying the input?
+        if len(predictions) == len(test_texts):
+            copied = sum(1 for p, t in zip(predictions, test_texts)
+                         if isinstance(p, str) and p.strip() == t.strip())
+            if copied / len(predictions) >= 0.3:
+                flags.append(("MED", f"{copied}/{len(predictions)} predictions "
+                                     f"equal the input text -> possible copying"))
+
+    # ---- tiny set ----
+    if 0 < n < 30:
+        flags.append(("LOW", f"only {n} test items -> wide confidence interval; "
+                             f"a few flips swing the score"))
+    return SanityReport(n, flags)