chad-code 2.1.0__tar.gz → 2.2.0__tar.gz

{chad_code-2.1.0 → chad_code-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: chad-code
-Version: 2.1.0
+Version: 2.2.0
 Summary: Claude-Code-style coding agent that runs entirely on a 24 GB Apple Silicon Mac (MLX, Qwen3.8-27B)
 License-Expression: MIT
 Project-URL: Repository, https://github.com/nathansutton/chad
@@ -59,13 +59,13 @@ uvx chad-code          # runs anywhere; the command is still `chad`
 uvx chad-code prove    # offline smoke test: 4 tiny fix-it tasks, verified, timed 🗿
 ```
 
-The first run asks, then downloads the model once (~13 GB) into the shared Hugging
+The first run asks, then downloads the model once (~8 GB) into the shared Hugging
 Face cache. While it downloads, `cd` into a project and think of a scoped first ask:
 *"fix the failing test in `tests/test_x.py`"* lands, *"improve my codebase"* flails.
 
 chad targets 24 GB and nothing smaller. It runs below that and tells you it is doing so,
-but 13 GB of weights sit resident before a single token of context, so a 16 GB Mac gets a
-window too small to work in.
+but 8 GB of weights and drafter plus a ~4 GB prefill transient sit resident before a single
+token of context, so a 16 GB Mac gets a window too small to work in.
 
 The PyPI package is `chad-code`. Bare `chad` is an unrelated squatted package.
 
@@ -79,9 +79,9 @@ The PyPI package is `chad-code`. Bare `chad` is an unrelated squatted package.
 ## Same model, same Mac, stock engine
 
 What do you gain over pointing a generic local-model tool at the same weights?
-Qwen3.8-27B at the same `UD-Q3_K_XL` recipe (Unsloth's GGUF for
-llama.cpp, chad's MLX conversion of the same bit map), the same M4 Pro (24 GB), one engine
-resident at a time, each measured with its own benchmark on a 512-token prompt and a
+Qwen3.8-27B at the `UD-Q3_K_XL` recipe (Unsloth's GGUF for llama.cpp, chad's MLX
+conversion of the same bit map; chad runs it with `--model`), the same M4 Pro (24 GB), one
+engine resident at a time, each measured with its own benchmark on a 512-token prompt and a
 128-token generation.
 
 | Engine | Prefill (512-tok prompt) | Decode (128 tok) | Speculative decoding |
@@ -90,11 +90,17 @@ resident at a time, each measured with its own benchmark on a 512-token prompt a
 | llama.cpp `llama-server` (build 10917), serial | 97 tok/s | 11.3 tok/s | off |
 | llama.cpp `llama-server` (build 10917) | 95 tok/s | 11.1 tok/s² | DFlash2 drafter (Q4_K_M GGUF) |
 | **chad**, serial (`CHAD_NO_DFLASH=1`) | 100 tok/s | 17.9 tok/s | off |
-| **chad**, default | 101 tok/s | **62.9 tok/s**¹ | DFlash2 block drafter |
+| **chad** | 101 tok/s | **62.9 tok/s**¹ | DFlash2 block drafter |
 
 A 200-token function body takes roughly 18 seconds at 10.9 tok/s and 3 at 63. You wait for
 the first one and you talk to the second.
 
+The weights chad ships by default are the same model in Prism ML's **ternary** build
+(every projection Hadamard-rotated and stored at 2 bits): 7.2 GB resident instead of 12.3,
+the same 64 tok/s drafted and 21 serial on this Mac, and the 5 GB it gives back is
+context, roughly a **150k-token window** where the 3-bit landed near 56k. The like-for-like
+table above stays on the 3-bit recipe because that is the quant llama.cpp has a GGUF of.
+
 Ollama does not get its own row: it is llama.cpp underneath, measured without speculative
 decoding, and on the same GGUF (0.32.15, Modelfile `FROM` only) it measures 96 tok/s
 prefill and the same **10.9** decode.
@@ -202,20 +208,24 @@ chad ships exactly one, downloaded once into the shared Hugging Face cache
 
 | Model | Quant | Footprint |
 |---|---|---|
-| [Qwen3.8-27B `UD-Q3_K_XL-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX) | 3-bit group-64 body, 5-bit `lm_head`, bundled 4-bit DFlash2 drafter | ~13 GB resident, 262k native context |
+| [Qwen3.8-27B `Ternary-Bonsai-2-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX) (default) | Prism ML's ternary build: Hadamard-rotated, 2-bit group-128, levels {−s, 0, +s}; bundled 4-bit DFlash2 drafter | ~8 GB resident, 262k native context, ~150k usable on 24 GB |
+| [Qwen3.8-27B `UD-Q3_K_XL-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX) via `--model` | 3-bit group-64 body, 5-bit `lm_head`, the same bundled drafter | ~13 GB resident, ~56k usable on 24 GB |
 
 Qwen3.8-27B is **dense** (64 layers: 48 GatedDeltaNet + 16 full attention), so every
-parameter is on the critical path for every token and the quant is where decode speed comes
-from. The bits go where held-out perplexity says they pay: `lm_head` is a second full
-1.27B-param tensor and is held at 5-bit, while `embed_tokens` is a lookup table whose error
-never compounds through a matmul, so it is cheapest. The name follows
-[Unsloth's convention](https://docs.unsloth.ai/) (`UD-…`), though the quant is MLX group-64
-affine, not a llama.cpp k-quant. The drafter ships in the same repo, pre-quantized.
+parameter is on the critical path for every token and the quant is where both decode speed
+and context come from: the governor prices a gigabyte of weights at about 29k tokens of
+window. The ternary build is Prism ML's [Bonsai](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit)
+pack of this checkpoint, repacked text-only with the base tokenizer and the drafter
+bundled (Apache-2.0; created using Bonsai by Prism ML). Its projections are stored in a
+rotated basis that an ordinary MLX loader would silently get wrong, so chad carries its own
+loader for it and fits the decode fast-path, the verify kernel and the drafter to it. What it
+costs: on code, teacher-forced perplexity is 4.49 against the 3-bit's 3.99 (+12%), while
+the private eval tiers tie at 56/56. The 3-bit stays one flag away.
 
 `--model <repo or local dir>` runs different weights through the same engine and stays a
 first-class escape hatch. The drafter, the fused-attention coverage, the decode fastpath and
-the context governor are all fitted to the shipped checkpoint, so other weights run slower;
-they do not break.
+the context governor are all fitted to the two checkpoints above, so other weights run
+slower; they do not break.
 
 ## Installing & upgrading
 

{chad_code-2.1.0 → chad_code-2.2.0}/README.md

@@ -23,13 +23,13 @@ uvx chad-code          # runs anywhere; the command is still `chad`
 uvx chad-code prove    # offline smoke test: 4 tiny fix-it tasks, verified, timed 🗿
 ```
 
-The first run asks, then downloads the model once (~13 GB) into the shared Hugging
+The first run asks, then downloads the model once (~8 GB) into the shared Hugging
 Face cache. While it downloads, `cd` into a project and think of a scoped first ask:
 *"fix the failing test in `tests/test_x.py`"* lands, *"improve my codebase"* flails.
 
 chad targets 24 GB and nothing smaller. It runs below that and tells you it is doing so,
-but 13 GB of weights sit resident before a single token of context, so a 16 GB Mac gets a
-window too small to work in.
+but 8 GB of weights and drafter plus a ~4 GB prefill transient sit resident before a single
+token of context, so a 16 GB Mac gets a window too small to work in.
 
 The PyPI package is `chad-code`. Bare `chad` is an unrelated squatted package.
 
@@ -43,9 +43,9 @@ The PyPI package is `chad-code`. Bare `chad` is an unrelated squatted package.
 ## Same model, same Mac, stock engine
 
 What do you gain over pointing a generic local-model tool at the same weights?
-Qwen3.8-27B at the same `UD-Q3_K_XL` recipe (Unsloth's GGUF for
-llama.cpp, chad's MLX conversion of the same bit map), the same M4 Pro (24 GB), one engine
-resident at a time, each measured with its own benchmark on a 512-token prompt and a
+Qwen3.8-27B at the `UD-Q3_K_XL` recipe (Unsloth's GGUF for llama.cpp, chad's MLX
+conversion of the same bit map; chad runs it with `--model`), the same M4 Pro (24 GB), one
+engine resident at a time, each measured with its own benchmark on a 512-token prompt and a
 128-token generation.
 
 | Engine | Prefill (512-tok prompt) | Decode (128 tok) | Speculative decoding |
@@ -54,11 +54,17 @@ resident at a time, each measured with its own benchmark on a 512-token prompt a
 | llama.cpp `llama-server` (build 10917), serial | 97 tok/s | 11.3 tok/s | off |
 | llama.cpp `llama-server` (build 10917) | 95 tok/s | 11.1 tok/s² | DFlash2 drafter (Q4_K_M GGUF) |
 | **chad**, serial (`CHAD_NO_DFLASH=1`) | 100 tok/s | 17.9 tok/s | off |
-| **chad**, default | 101 tok/s | **62.9 tok/s**¹ | DFlash2 block drafter |
+| **chad** | 101 tok/s | **62.9 tok/s**¹ | DFlash2 block drafter |
 
 A 200-token function body takes roughly 18 seconds at 10.9 tok/s and 3 at 63. You wait for
 the first one and you talk to the second.
 
+The weights chad ships by default are the same model in Prism ML's **ternary** build
+(every projection Hadamard-rotated and stored at 2 bits): 7.2 GB resident instead of 12.3,
+the same 64 tok/s drafted and 21 serial on this Mac, and the 5 GB it gives back is
+context, roughly a **150k-token window** where the 3-bit landed near 56k. The like-for-like
+table above stays on the 3-bit recipe because that is the quant llama.cpp has a GGUF of.
+
 Ollama does not get its own row: it is llama.cpp underneath, measured without speculative
 decoding, and on the same GGUF (0.32.15, Modelfile `FROM` only) it measures 96 tok/s
 prefill and the same **10.9** decode.
@@ -166,20 +172,24 @@ chad ships exactly one, downloaded once into the shared Hugging Face cache
 
 | Model | Quant | Footprint |
 |---|---|---|
-| [Qwen3.8-27B `UD-Q3_K_XL-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX) | 3-bit group-64 body, 5-bit `lm_head`, bundled 4-bit DFlash2 drafter | ~13 GB resident, 262k native context |
+| [Qwen3.8-27B `Ternary-Bonsai-2-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX) (default) | Prism ML's ternary build: Hadamard-rotated, 2-bit group-128, levels {−s, 0, +s}; bundled 4-bit DFlash2 drafter | ~8 GB resident, 262k native context, ~150k usable on 24 GB |
+| [Qwen3.8-27B `UD-Q3_K_XL-DFlash2`](https://huggingface.co/nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX) via `--model` | 3-bit group-64 body, 5-bit `lm_head`, the same bundled drafter | ~13 GB resident, ~56k usable on 24 GB |
 
 Qwen3.8-27B is **dense** (64 layers: 48 GatedDeltaNet + 16 full attention), so every
-parameter is on the critical path for every token and the quant is where decode speed comes
-from. The bits go where held-out perplexity says they pay: `lm_head` is a second full
-1.27B-param tensor and is held at 5-bit, while `embed_tokens` is a lookup table whose error
-never compounds through a matmul, so it is cheapest. The name follows
-[Unsloth's convention](https://docs.unsloth.ai/) (`UD-…`), though the quant is MLX group-64
-affine, not a llama.cpp k-quant. The drafter ships in the same repo, pre-quantized.
+parameter is on the critical path for every token and the quant is where both decode speed
+and context come from: the governor prices a gigabyte of weights at about 29k tokens of
+window. The ternary build is Prism ML's [Bonsai](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit)
+pack of this checkpoint, repacked text-only with the base tokenizer and the drafter
+bundled (Apache-2.0; created using Bonsai by Prism ML). Its projections are stored in a
+rotated basis that an ordinary MLX loader would silently get wrong, so chad carries its own
+loader for it and fits the decode fast-path, the verify kernel and the drafter to it. What it
+costs: on code, teacher-forced perplexity is 4.49 against the 3-bit's 3.99 (+12%), while
+the private eval tiers tie at 56/56. The 3-bit stays one flag away.
 
 `--model <repo or local dir>` runs different weights through the same engine and stays a
 first-class escape hatch. The drafter, the fused-attention coverage, the decode fastpath and
-the context governor are all fitted to the shipped checkpoint, so other weights run slower;
-they do not break.
+the context governor are all fitted to the two checkpoints above, so other weights run
+slower; they do not break.
 
 ## Installing & upgrading
 

{chad_code-2.1.0 → chad_code-2.2.0}/pyproject.toml

@@ -4,7 +4,7 @@
 # import name, and command name are independent. `uvx chad-code` runs the alias
 # script added under [project.scripts].
 name = "chad-code"
-version = "2.1.0"
+version = "2.2.0"
 description = "Claude-Code-style coding agent that runs entirely on a 24 GB Apple Silicon Mac (MLX, Qwen3.8-27B)"
 readme = "README.md"
 license = "MIT"

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/__init__.py

@@ -4,7 +4,7 @@ A flat collection of cooperating modules behind one console script (``chad``):
 the inference engine, the tool layer, the agent loop, and the terminal UI.
 """
 
-__version__ = "2.1.0"
+__version__ = "2.2.0"
 
 # chad sets no MLX_* runtime vars. MLX_METAL_FAST_SYNCH, MLX_MAX_OPS_PER_BUFFER
 # and MLX_MAX_MB_PER_BUFFER were each measured end-to-end on the 35B and every

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/agent.py

@@ -753,6 +753,12 @@ class Agent:
         self._split_ok = ok
         return ok
 
+    def _template_extra(self) -> dict:
+        """The template kwargs beyond messages/tools/thinking: the reasoning effort,
+        when one is set (the user's, else the engine's loader-carried default)."""
+        effort = self.reasoning_effort or self.engine.reasoning_effort_default
+        return {"reasoning_effort": effort} if effort else {}
+
     def _render(self, thinking: bool = None):
         # `thinking` overrides self.thinking for THIS render only (no-think
         # escalation renders one step with <think> off, then restores). None => self.thinking.
@@ -772,13 +778,14 @@ class Agent:
         # malformed TypeScript it cannot repair. `low` keeps the reasoning block and
         # shrinks it. Passed only when set, and via **kwargs, so templates that do not
         # accept the argument (q3_s6's, which has no reasoning_effort at all) render
-        # byte-identically to before.
-        extra = {}
-        if self.reasoning_effort:
-            extra["reasoning_effort"] = self.reasoning_effort
+        # byte-identically to before. A loader may carry a default of its own
+        # (engine.reasoning_effort_default) for a template whose default it disagrees
+        # with; the user's setting still wins. `_prefix_ids` renders with the same
+        # kwargs: the effort lands in the system block, so a prefix priced without
+        # it would not be a prefix of the prompt.
         ids = self._template_ids(self.engine.tok.apply_chat_template(
             messages, tools=self._active_schemas(), add_generation_prompt=True,
-            enable_thinking=thinking, **extra,
+            enable_thinking=thinking, **self._template_extra(),
         ))
         # Debug hook (env-gated, off by default): dump the first decoded render so a
         # rendered-prompt difference across environments can be diffed. Best-effort.
@@ -808,7 +815,8 @@ class Agent:
         def render1(u):
             return self._template_ids(self.engine.tok.apply_chat_template(
                 [sysm, {"role": "user", "content": u}], tools=schemas,
-                add_generation_prompt=True, enable_thinking=self.thinking))
+                add_generation_prompt=True, enable_thinking=self.thinking,
+                **self._template_extra()))
         a, b = render1("a"), render1("the quick brown fox jumps")
         n = 0
         for x, y in zip(a, b):
@@ -835,7 +843,8 @@ class Agent:
         def render1(m):
             return self._template_ids(self.engine.tok.apply_chat_template(
                 [m, {"role": "user", "content": "a"}], tools=schemas,
-                add_generation_prompt=True, enable_thinking=self.thinking))
+                add_generation_prompt=True, enable_thinking=self.thinking,
+                **self._template_extra()))
         a, b = render1({"role": "system", "content": static}), render1(sysm)
         n = 0
         for x, y in zip(a, b):

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/base_engine.py

@@ -168,6 +168,9 @@ class BaseEngine(Protocol):
     effective_ctx: int           # usable context window (tokens)
     cache_dir: Optional[str]     # on-disk KV checkpoint dir; None disables warm-start
     _cached_ids: list            # tokens currently resident in the prefix cache
+    # A chat-template `reasoning_effort` the loaded weights want passed when the user
+    # set none (a loader that disagrees with its template's default); None = template's.
+    reasoning_effort_default: Optional[str]
 
     # --- generation ------------------------------------------------------
     def generate(

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/cli.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 """chad — a local, MLX-backed, Claude-Code-style coding agent.
 
-One model (Qwen3.8-27B, 3-bit, with its DFlash2 drafter), one entrypoint, run with uv:
+One model (Qwen3.8-27B, ternary, with its DFlash2 drafter), one entrypoint, run with uv:
 
     uv run chad                                # interactive full-screen TUI
     uv run chad "fix the bug in greet.py"      # one-shot, headless
@@ -38,31 +38,27 @@ if TYPE_CHECKING:
 _HERE = os.path.dirname(os.path.abspath(__file__))
 _PROJECT_ROOT = os.path.dirname(os.path.dirname(_HERE))
 
-# The shipped model, on Hugging Face. Naming follows Unsloth's dynamic-quant
-# convention so the quant scheme is recognizable (UD = Unsloth Dynamic; Q3_K_XL =
-# a 3-bit body with extra bits where they pay), plus an -MLX suffix for format
-# discoverability. The quant itself is MLX group-64 affine, not llama.cpp Q3_K
-# k-quants — the model card says so; the tag is for recognition, not bit-for-bit
-# equivalence. The repo also carries the DFlash2 block drafter, pre-quantized, in
-# `dflash/` (~1.1 GB): one download gets the model and its speculative decoder,
-# with nothing built on first run (mlx_dflash.py).
+# The shipped model, on Hugging Face. The repo carries the weights AND the DFlash2
+# block drafter, pre-quantized, in `dflash/` (~1.1 GB): one download gets the model
+# and its speculative decoder, with nothing built on first run (mlx_dflash.py).
 #
 # Qwen3.8-27B is `qwen3_5` — DENSE (64 layers: 48 GatedDeltaNet + 16 full attention),
 # so every parameter is on the critical path for every token and shrinking the model
-# is the only decode lever there is. The recipe (`q3_e3h5`) is 3-bit group-64
-# throughout, except lm_head at 5-bit: with vocab_size 248,320 and tie_word_embeddings
-# false, the head is a second full 1.27B-param tensor, and the calibrated GGUF builds
-# of this same checkpoint are unanimous that it is the tier worth protecting while
-# embed_tokens — a lookup table, whose per-row error never compounds through a matmul
-# — is the cheapest. ~12.1 GB resident.
-_HF_MODEL = "nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX"
+# is the only decode lever there is. The shipped weights are Prism ML's ternary build:
+# every projection Hadamard-rotated offline and stored as 2-bit affine group-128 whose
+# three levels are {-s, 0, +s}, ~7.2 GB resident against the 3-bit recipe's 12.1 — and
+# on a dense model those 5 GB are context, at the governor's measured 34,816 B/token.
+# The rotation has to be undone on the activations at runtime (prism_pack.py); the
+# repo is our repack of the pack with the base tokenizer and the drafter bundled.
+_HF_MODEL = "nathansutton/Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX"
 # A dev clone that already built the weights locally should use them rather than
 # re-download — prefer this dir when present.
-_LOCAL_MODEL = os.path.join(_PROJECT_ROOT, "models", "Qwen3.8-27B-q3_e3h5")
+_LOCAL_MODEL = os.path.join(_PROJECT_ROOT, "models", "Qwen3.8-27B-Ternary-Bonsai-2")
 # chad targets 24 GB Apple Silicon and nothing smaller. Below this the model still
-# loads, but the context governor has almost nothing left to spend after ~12.1 GB of
-# weights and the ~4.3 GB prefill transient, so the window collapses toward its floor.
-# We warn and proceed rather than refuse: the harness advises, the caller decides.
+# loads, but the context governor has little left to spend after ~7.2 GB of weights,
+# the ~1.1 GB drafter and the ~4.3 GB prefill transient, so the window shrinks toward
+# its floor. We warn and proceed rather than refuse: the harness advises, the caller
+# decides.
 _MIN_RAM_GB = 23.5
 
 
@@ -487,10 +483,10 @@ def _pick_model(spec=None, *, host: Host = HOST, local_model: str = _LOCAL_MODEL
 
 def _model_download_gb(model_id):
     """Approximate download size in GiB for the shipped model (for the disk preflight
-    and the confirm prompt — display honesty, not accounting): ~12.1 GB of weights
+    and the confirm prompt — display honesty, not accounting): ~7.2 GB of weights
     plus the ~1.1 GB bundled DFlash2 drafter. An arbitrary `--model` is unknowable
     ahead of the resolve, so it gets the same figure."""
-    return 13.2
+    return 8.3
 
 
 def _cached_weights_complete(

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/completion_engine.py

@@ -142,6 +142,7 @@ class CompletionEngine:
         self.timeout = timeout
         # 0 = auto: load() asks the server (/props n_ctx) so chad's window matches
         # the wall the server actually enforces; falls back to 32768 offline.
+        self.reasoning_effort_default = None
         self.effective_ctx = effective_ctx
         # tokenizer_id points at the HF repo whose tokenizer matches the served GGUF's
         # vocab (the served model_id is usually a GGUF repo with no tokenizer files).

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/engine.py

@@ -518,6 +518,9 @@ class Engine:
     model: Any = field(init=False, default=None)
     tok: Any = field(init=False, default=None)
     effective_ctx: int = field(init=False, default=32768)
+    # A chat-template default the loaded weights need overridden (the Prism pack's
+    # template says xhigh where the shipped model's says medium); None = the template's.
+    reasoning_effort_default: Optional[str] = field(init=False, default=None)
     _cache: Any = field(init=False, default=None)
     _cached_ids: list = field(init=False, default_factory=list)
     # Token ids generated in the CURRENT turn, for the presence penalty. Reset at
@@ -541,6 +544,8 @@ class Engine:
     # dense — the MoE arm is here for `--model`.
     _is_moe: bool = field(init=False, default=False)
     _dflash: Any = field(init=False, default=None)
+    fastpath: bool = field(init=False, default=False)   # mlx_fastpath installed
+    _dflash_ladder: Any = field(init=False, default=None)   # learned round-cost seed
     _n_attn_heads: int = field(init=False, default=16)
     _n_kv_heads: int = field(init=False, default=0)
     _head_dim: int = field(init=False, default=0)
@@ -624,7 +629,11 @@ class Engine:
         # dense qwen3_5 hybrid; silent no-op on any other model or on failure.
         from . import mlx_fastpath
         _log_mlx_provenance()
-        mlx_fastpath.install(self.model, model_path=path)
+        self.fastpath = mlx_fastpath.install(self.model, model_path=path)
+        if not self.fastpath and not config.flag("CHAD_NO_FASTPATH"):
+            # install() says why when it declines a Prism pack or fails outright;
+            # this is the line that makes a stock-graph run visible at all.
+            log.info("FASTPATH not installed: decoding on the stock op graph")
         # Fused quantized-KV decode attention: makes kv_bits=8 a speed win
         # instead of a loss. Patches mlx_lm's quantized SDPA branch
         # only; inert unless a QuantizedKVCache is actually in play.
@@ -662,9 +671,26 @@ class Engine:
         does, including the tokenizer's stop ids from the model config (which folds in
         generation_config.json; the override never touches them)."""
         model_path = _download(path)
-        eos = load_config(model_path).get("eos_token_id")
-        self.tok = load_tokenizer(model_path, eos_token_ids=eos)
+        cfg = load_config(model_path)
+        eos = cfg.get("eos_token_id")
+        # Prism's Hadamard-folded ternary packs carry weights in a rotated basis and
+        # declare their own model_type. mlx-lm's affine loader would find the right
+        # shapes, skip the activation transform and return garbage without erroring,
+        # so they route to chad's own loader on the declared type.
+        from . import prism_pack
+        prism = prism_pack.is_prism_pack(cfg)
+        with prism_pack.quiet_tokenizer_load() if prism else contextlib.nullcontext():
+            self.tok = load_tokenizer(model_path, eos_token_ids=eos)
         override, self.effective_ctx = self._ctx_override(path)
+        if prism:
+            if override:
+                cfg = {**cfg, "text_config": {**cfg["text_config"], **override}}
+            self.model, _ = prism_pack.load(str(model_path), cfg)
+            self.reasoning_effort_default = prism_pack.REASONING_EFFORT_DEFAULT
+            # The speculative schedule's round-cost seed is per weight width.
+            from . import mlx_dflash
+            self._dflash_ladder = list(mlx_dflash.round_costs(prism_pack.BITS))
+            return
         self.model, _ = load_model(model_path, model_config=override)
 
     def _read_model_shape(self, path: str) -> None:
@@ -2552,6 +2578,8 @@ class Engine:
             stats.generated_tokens = len(out_ids)
             stats.gen_ids = list(out_ids)
             self._cached_ids = fed_ids
+            if policy is not None:
+                self._dflash_ladder = policy.learned_seed()
             mx.clear_cache()
             return detok.text, stats
         except BaseException:
@@ -2597,7 +2625,9 @@ class _DFlashDrafter:
                               cfg.block_size - 1))
         # Fresh per-turn schedule state: acceptance statistics are a property
         # of the current prompt/content, not of the session.
-        self.policy = (mlx_dflash.block_policy(self.cap)
+        # The round-cost ladder is the exception: it belongs to the machine and the
+        # checkpoint, so each turn starts from what the last one measured.
+        self.policy = (mlx_dflash.block_policy(self.cap, eng._dflash_ladder)
                        if eng.dflash_adaptive else None)
         self._ids = list(cfg.target_layer_ids)
         self._mask = int(cfg.mask_token_id)

{chad_code-2.1.0 → chad_code-2.2.0}/src/chad/mlx_dflash.py

@@ -450,6 +450,24 @@ def build(config: DFlashConfig):
 # first extra row too cheaply and collapsed the schedule on medium acceptance).
 BLOCK_ROUND_COSTS = (1.0, 1.76, 1.93, 2.30, 2.18, 2.20, 2.19, 2.20)
 
+# The same ladder on the 2-bit ternary pack (benchmarks/spec_decode.py, a fixed-width
+# arm per depth, round wall over the serial step of the same prompt). It is NOT the
+# 3-bit ladder rescaled: the serial step is shorter (46-51 ms against 57), so the
+# drafter forward, the rollback and the host sync are a larger share of every round,
+# and rows 2..4 ride mlx's 2-bit matmul, which pays more per row than its 3-bit one.
+# Measured twice, six prompt x preset cells each agreeing to +-0.05: on a cool machine
+# (21.5 tok/s serial) 1.68 / 2.21 / 2.60 / 2.56 / 2.57 / 2.57 / 2.60, on a warm one
+# (19.7) 1.58 / 2.09 / 2.44 / 2.39 / 2.44 / 2.44 / 2.46 — the ratio moves with the
+# serial step, which is the dispatch-bound half. The seed is the midpoint; either way
+# the 3-bit seed priced every width past 1 some 9-18% too cheap here, and
+# WidthPolicy.learned_seed carries what a turn measures into the next.
+BLOCK_ROUND_COSTS_2BIT = (1.0, 1.63, 2.15, 2.52, 2.48, 2.50, 2.50, 2.53)
+
+
+def round_costs(weight_bits: Optional[int]) -> tuple:
+    """The measured seed ladder for a target quantized at `weight_bits`."""
+    return BLOCK_ROUND_COSTS_2BIT if weight_bits == 2 else BLOCK_ROUND_COSTS
+
 
 class WidthPolicy:
     """Cost-model width schedule: choose the verified width that maximizes
@@ -537,6 +555,29 @@ class WidthPolicy:
             return ob[0]
         return self._seed[depth] * self._unit()
 
+    LEARN_MIN = 4        # rounds at a depth before its wall may re-shape the seed
+
+    def learned_seed(self) -> list:
+        """The seed ladder for the NEXT turn: this one's, with every depth the turn
+        measured enough moved halfway to what it measured, in seed units (so the
+        result is scale-free like the seed itself — the next turn's context is a
+        different length and its walls a different scale).
+
+        The policy is rebuilt per turn because acceptance is a property of the
+        content, but the cost ladder is a property of the machine and the
+        checkpoint, and a turn that re-learns it from a wrong seed spends its first
+        rounds at the wrong width. Halfway, and only past LEARN_MIN rounds: a depth
+        priced out by one inflated sample (a kernel build, a rollback) is never
+        visited again to correct it."""
+        if not self._obs:
+            return list(self._seed)
+        unit = self._unit()
+        out = list(self._seed)
+        for d, (wall, n) in self._obs.items():
+            if n >= self.LEARN_MIN and unit > 0:
+                out[d] = 0.5 * out[d] + 0.5 * wall / unit
+        return out
+
     def _tail_p(self) -> float:
         """Acceptance estimate for positions >= TAIL_SPLIT. Real pooled
         observations once any wide round has run; before that, INFERENCE
@@ -806,6 +847,57 @@ def bundle_dir(model_dir: str) -> Optional[str]:
     return d if os.path.isfile(os.path.join(d, "config.json")) else None
 
 
+# Drafters transfer across quantizations of one base model: a drafter reads the
+# target's residual stream at its tapped layers, which a different weight quantization
+# only perturbs (measured on the Prism ternary pack: 93% acceptance with the 3-bit
+# model's sidecar). A checkpoint that bundles no drafter borrows the one a sibling
+# repo bundles, keyed on the (hidden, layers, vocab) shape the tap needs anyway.
+DONORS: dict = {(5120, 64, 248320): "nathansutton/Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX"}
+
+
+def _donor_file(repo_id: str, filename: str, cached: bool = True) -> str:
+    """The local path of one file of a hub repo: the cache when it holds it (and
+    `cached`), a download — into the repo's CURRENT snapshot — otherwise."""
+    from huggingface_hub import hf_hub_download, try_to_load_from_cache
+    hit = try_to_load_from_cache(repo_id, filename) if cached else None
+    return hit if isinstance(hit, str) else hf_hub_download(repo_id, filename)
+
+
+def donor_bundle_dir(key, repo_id: Optional[str], donors: Optional[dict] = None,
+                     fetch: Optional[Callable[[str, str], str]] = None) -> Optional[str]:
+    """The sidecar dir a `donors` (default DONORS) sibling bundles for a target of
+    shape `key`, or None (no donor for the shape, the model IS the donor, or the
+    fetch failed). `fetch(repo_id, filename)` resolves one file; None = the hub."""
+    donor = (DONORS if donors is None else donors).get(key)
+    if not donor or donor == repo_id:
+        return None
+    resolve = fetch or _donor_file
+    try:
+        cfg = resolve(donor, f"{_BUNDLE}/config.json")
+        wts = resolve(donor, f"{_BUNDLE}/model.safetensors")
+        d = os.path.dirname(wts)
+        if os.path.dirname(cfg) != d and not _complete(d):
+            # The two files resolve independently, and a cache hit on the config
+            # from an older revision's snapshot next to weights downloaded into the
+            # current one is two directories, neither a bundle. The weights' snapshot
+            # is the live one: fetch the config into it.
+            if fetch is not None:
+                fetch(donor, f"{_BUNDLE}/config.json")
+            else:
+                _donor_file(donor, f"{_BUNDLE}/config.json", cached=False)
+    except Exception as e:  # noqa: BLE001 — offline/gated: decode without the drafter
+        log.warning("DFlash drafter: donor bundle %s unavailable (%s); decoding "
+                    "without it", donor, e)
+        return None
+    if not _complete(d):
+        log.warning("DFlash drafter: donor bundle %s resolved to a config and weights "
+                    "in different snapshots (%s, %s); decoding without it", donor,
+                    os.path.dirname(cfg), d)
+        return None
+    log.info("DFlash drafter: no bundle with these weights; borrowing %s's", donor)
+    return d
+
+
 def _complete(d: str) -> bool:
     return (os.path.isfile(os.path.join(d, "config.json"))
             and os.path.isfile(os.path.join(d, "model.safetensors")))
@@ -849,18 +941,22 @@ def ensure_bundle(model_dir: str, repo_id: Optional[str],
 
 
 def load_drafter(model: "nn.Module", model_dir: str, repo_id: Optional[str] = None,
-                 bits: int = 4, gs: int = 64) -> Optional["nn.Module"]:
+                 bits: int = 4, gs: int = 64, donors: Optional[dict] = None,
+                 fetch: Optional[Callable[[str, str], str]] = None) -> Optional["nn.Module"]:
     """Load the DFlash drafter bundled with the target's weights (or the dir
     CHAD_DFLASH_PATH names), bound to the target's embedding/lm_head, with the
     target tap installed. None when no drafter ships for this model or on any
     failure — DFlash is a pure speed feature, never load-bearing.
 
     `repo_id` is the model's HF repo when it came from the hub, so a bundle whose
-    weights the base download filtered out can be completed (see ensure_bundle)."""
+    weights the base download filtered out can be completed (see ensure_bundle).
+    `donors`/`fetch` are donor_bundle_dir's (tests inject them)."""
     try:
         key = _target_key(model)
         ensure_bundle(model_dir, repo_id)
         sdir = bundle_dir(model_dir)
+        if sdir is None and key is not None:
+            sdir = donor_bundle_dir(key, repo_id, donors, fetch)
         if key is None or sdir is None:
             return None
         if not _is_sidecar(sdir):