chad-code 1.0.7__tar.gz → 1.0.9__tar.gz

{chad_code-1.0.7/src/chad_code.egg-info → chad_code-1.0.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: chad-code
-Version: 1.0.7
+Version: 1.0.9
 Summary: Local MLX-backed, Claude-Code-style coding agent (Apple Silicon, Ornith 35B/9B)
 License-Expression: MIT
 Project-URL: Repository, https://github.com/nathansutton/chad
@@ -139,6 +139,21 @@ uv run chad "add a --json flag to main.py and update the tests"   # one-shot, he
 uv run chad -c               # resume this directory's last conversation
 ```
 
+**Optional extras.** Everything core is in the base install; two features are opt-in
+because they pull deps not every install wants — `speech` (voice mode: a mic library,
+no torch) and `highlight` (syntax colour in diffs/previews). An extra rides on the
+**install spec**, not on a separate command, so how you add it depends on how you
+installed chad:
+
+```bash
+uv tool install --force 'chad-code[speech]'   # add to an existing `uv tool` install
+uvx --from 'chad-code[speech]' chad           # one-off run, nothing installed
+uv sync --extra speech                        # from a clone
+```
+
+`/speech` in the TUI prints whichever of those matches your install, so you never have
+to work it out from here.
+
 **The model.** chad picks one for you by RAM and downloads it once into the shared Hugging
 Face cache (`~/.cache/huggingface`, reused across every project). Override with
 `--model 9b` / `--model 35b`, or `--model <repo or local dir>` for anything else.
@@ -164,8 +179,8 @@ re-downloads the model.
 **Development.** `uv sync` once, then `uv run pytest -q` — the fast unit gate loads **no
 model weights**, runs in seconds, and is what CI runs. Throughput on your own machine:
 `uv run chad-bench` (see [Throughput & performance](docs/benchmarks.md)). LSP-precise
-find-references / rename need the `lsp` extra (`uv tool install 'chad-code[lsp]'`); without
-it chad uses the tree-sitter fallback automatically.
+find-references / rename need no extra — chad fetches pyright via `uvx` on first use and
+falls back to tree-sitter when a language server can't start.
 
 ## Interactive UX
 
@@ -191,8 +206,9 @@ it chad uses the tree-sitter fallback automatically.
   isn't clipped, and a personal word table (`~/.chad/speech_words.json`) teaches it your
   identifiers — `{"pie test": "pytest"}`. Dictation cost is linear in take length, so a
   long thought is fine; `/speech` off releases both the mic and the weights. Nothing
-  leaves the machine. Needs the `speech` extra (`uv sync --extra speech` — just a mic
-  library; no torch, no numba).
+  leaves the machine. Needs the `speech` extra — just a mic library; no torch, no numba
+  (see [Installing & upgrading](#installing--upgrading); on a `uv tool` install that's
+  `uv tool install --force 'chad-code[speech]'`).
 
 **Usage.** `uv run chad --help` is the source of truth:
 

{chad_code-1.0.7 → chad_code-1.0.9}/README.md

@@ -103,6 +103,21 @@ uv run chad "add a --json flag to main.py and update the tests"   # one-shot, he
 uv run chad -c               # resume this directory's last conversation
 ```
 
+**Optional extras.** Everything core is in the base install; two features are opt-in
+because they pull deps not every install wants — `speech` (voice mode: a mic library,
+no torch) and `highlight` (syntax colour in diffs/previews). An extra rides on the
+**install spec**, not on a separate command, so how you add it depends on how you
+installed chad:
+
+```bash
+uv tool install --force 'chad-code[speech]'   # add to an existing `uv tool` install
+uvx --from 'chad-code[speech]' chad           # one-off run, nothing installed
+uv sync --extra speech                        # from a clone
+```
+
+`/speech` in the TUI prints whichever of those matches your install, so you never have
+to work it out from here.
+
 **The model.** chad picks one for you by RAM and downloads it once into the shared Hugging
 Face cache (`~/.cache/huggingface`, reused across every project). Override with
 `--model 9b` / `--model 35b`, or `--model <repo or local dir>` for anything else.
@@ -128,8 +143,8 @@ re-downloads the model.
 **Development.** `uv sync` once, then `uv run pytest -q` — the fast unit gate loads **no
 model weights**, runs in seconds, and is what CI runs. Throughput on your own machine:
 `uv run chad-bench` (see [Throughput & performance](docs/benchmarks.md)). LSP-precise
-find-references / rename need the `lsp` extra (`uv tool install 'chad-code[lsp]'`); without
-it chad uses the tree-sitter fallback automatically.
+find-references / rename need no extra — chad fetches pyright via `uvx` on first use and
+falls back to tree-sitter when a language server can't start.
 
 ## Interactive UX
 
@@ -155,8 +170,9 @@ it chad uses the tree-sitter fallback automatically.
   isn't clipped, and a personal word table (`~/.chad/speech_words.json`) teaches it your
   identifiers — `{"pie test": "pytest"}`. Dictation cost is linear in take length, so a
   long thought is fine; `/speech` off releases both the mic and the weights. Nothing
-  leaves the machine. Needs the `speech` extra (`uv sync --extra speech` — just a mic
-  library; no torch, no numba).
+  leaves the machine. Needs the `speech` extra — just a mic library; no torch, no numba
+  (see [Installing & upgrading](#installing--upgrading); on a `uv tool` install that's
+  `uv tool install --force 'chad-code[speech]'`).
 
 **Usage.** `uv run chad --help` is the source of truth:
 

{chad_code-1.0.7 → chad_code-1.0.9}/pyproject.toml

@@ -4,7 +4,7 @@
 # import name, and command name are independent. `uvx chad-code` runs the alias
 # script added under [project.scripts].
 name = "chad-code"
-version = "1.0.7"
+version = "1.0.9"
 description = "Local MLX-backed, Claude-Code-style coding agent (Apple Silicon, Ornith 35B/9B)"
 readme = "README.md"
 license = "MIT"
@@ -72,11 +72,15 @@ dependencies = [
     "mcp>=2,<3",
 ]
 
+# Extras ride on the INSTALL SPEC, so the command depends on how chad was installed:
+# `uv tool install --force 'chad-code[<extra>]'` (or `uvx --from 'chad-code[<extra>]' chad`)
+# for a PyPI install, `uv sync --extra <extra>` only in a clone. speech.install_hint()
+# derives the right one at runtime — keep it in step with any extra added here.
+# NOTE: the PyPI distribution is `chad-code`; 'pip install chad' installs an unrelated
+# squatted package.
+#
 # Optional syntax highlighting in diffs / confirm previews. Pure-Python,
 # import-guarded in render.py: absent, output is byte-identical to the un-highlighted path.
-# Install with `uv sync --extra highlight`.
-# NOTE: no PyPI release exists; 'pip install chad' installs an unrelated squatted
-# package. Document uv/git installs only.
 [project.optional-dependencies]
 highlight = ["pygments>=2.17"]
 # Voice mode (/speech in the TUI): push-to-talk dictation via chad's vendored
@@ -85,7 +89,7 @@ highlight = ["pygments>=2.17"]
 # `say`. All on-device, nothing leaves the machine. mlx/numpy/huggingface-hub
 # are already core deps, so the extra adds only the mic (sounddevice/PortAudio).
 # Optional and import-guarded in speech.py: absent, /speech prints the install
-# hint and everything else is untouched. Install with `uv sync --extra speech`.
+# hint and everything else is untouched.
 speech = [
     "sounddevice>=0.5",
 ]
@@ -170,10 +174,10 @@ module = "chad.parakeet.*"
 ignore_errors = true
 
 [[tool.mypy.overrides]]
-# mlx_fastpath patches mlx_lm classes that only exist on macOS: with mlx installed
-# (dev Macs, the macos CI jobs) its `type: ignore[method-assign]` comments are load-
+# These modules patch mlx_lm classes that only exist on macOS: with mlx installed
+# (dev Macs, the macos CI jobs) their `type: ignore[method-assign]` comments are load-
 # bearing; on the Linux lint runner mlx_lm is absent, the patched classes type as Any,
 # and warn_unused_ignores flags those same comments — unfixable in the code for both
-# platforms at once, so the unused-ignore warning alone is relaxed for this module.
-module = "chad.mlx_fastpath"
+# platforms at once, so the unused-ignore warning alone is relaxed for them.
+module = ["chad.mlx_fastpath", "chad.mlx_moe_fused"]
 warn_unused_ignores = false

{chad_code-1.0.7 → chad_code-1.0.9}/src/chad/__init__.py

@@ -4,7 +4,7 @@ A flat collection of cooperating modules behind one console script (``chad``):
 the inference engine, the tool layer, the agent loop, and the terminal UI.
 """
 
-__version__ = "1.0.7"
+__version__ = "1.0.9"
 
 # chad sets no MLX_* runtime vars. MLX_METAL_FAST_SYNCH, MLX_MAX_OPS_PER_BUFFER
 # and MLX_MAX_MB_PER_BUFFER were each measured end-to-end on the 35B and every

{chad_code-1.0.7 → chad_code-1.0.9}/src/chad/mlx_fastpath.py

@@ -61,6 +61,10 @@ def install(model: Any) -> bool:
             return False
         _concat_expert_gate_up(model)
         _concat_gdn_in_projs(model)
+        # Fused MoE decode kernels + router|seg / shared gate|up concats;
+        # engages only on the exact 35B geometry, silent no-op elsewhere.
+        from . import mlx_moe_fused
+        mlx_moe_fused.install(model)
         _install_layer_fastpath(model)
         log.info("FASTPATH installed: fused expert/GDN projections + compiled "
                  "S=1 layer step")
@@ -129,6 +133,8 @@ def _patch_switch_glu() -> None:
     import mlx.core as mx
     from mlx_lm.models import switch_layers as sl
 
+    if getattr(sl.SwitchGLU.__call__, "_chad_fastpath", False):
+        return  # already patched; re-wrapping would stack guards
     stock_call = sl.SwitchGLU.__call__
 
     def fused_call(self, x, indices):
@@ -151,6 +157,7 @@ def _patch_switch_glu() -> None:
             y = sl._scatter_unsort(y, inv_order, indices.shape)
         return y.squeeze(-2)
 
+    fused_call._chad_fastpath = True  # type: ignore[attr-defined]
     sl.SwitchGLU.__call__ = fused_call  # type: ignore[method-assign]
 
 
@@ -187,6 +194,8 @@ def _patch_gdn_call() -> None:
     import mlx.nn as nn
     from mlx_lm.models import qwen3_5 as q35
 
+    if getattr(q35.GatedDeltaNet.__call__, "_chad_fastpath", False):
+        return
     stock_call = q35.GatedDeltaNet.__call__
 
     def call(self, inputs, mask=None, cache=None):
@@ -236,6 +245,7 @@ def _patch_gdn_call() -> None:
         out = self.norm(out, z)
         return self.out_proj(out.reshape(B, S, -1))
 
+    call._chad_fastpath = True  # type: ignore[attr-defined]
     q35.GatedDeltaNet.__call__ = call  # type: ignore[method-assign]
 
 
@@ -246,13 +256,32 @@ def _install_layer_fastpath(model) -> None:
     from mlx_lm.models import qwen3_5 as q35
 
     for layer in model.language_model.model.layers:
-        layer._moe_fast = _compile_moe_step(layer)
+        body = _compile_moe_step(layer)
+        fused = getattr(layer.mlp, "_fused_step", None)
+        if fused is not None:
+            # mlx_moe_fused kernels handle the single-token (1,1,2048) decode
+            # step; anything else (batched decode, foreign width) takes the
+            # compiled stock-graph body. Pinning the exact element count keeps
+            # a future geometry change from reaching kernels that hardcode it.
+            from .mlx_moe_fused import HID as moe_hid
+            layer._moe_body_fast = body
+            layer._moe_fast = (lambda h, f=fused, b=body, n=moe_hid:
+                               f(h) if h.size == n else b(h))
+        else:
+            layer._moe_fast = body
         if layer.is_linear and hasattr(layer.linear_attn, "_fused_w"):
             layer._gdn_fast = _compile_gdn_step(layer)
 
+    if getattr(q35.DecoderLayer.__call__, "_chad_fastpath", False):
+        return
     stock_layer_call = q35.DecoderLayer.__call__
 
     def layer_call(self, x, mask=None, cache=None):
+        # S==1 cannot distinguish decode from a stray 1-token prefill chunk
+        # (mask is None for both); such a chunk takes this path and pays one
+        # token's worth of the same rounding class decode itself injects into
+        # the cache every step — bounded, unlike S>1 compile-fusion drift,
+        # which is why prefill proper stays on the stock graph.
         if mask is None and x.shape[1] == 1 and cache is not None \
                 and getattr(self, "_moe_fast", None) is not None:
             if self.is_linear:
@@ -270,6 +299,7 @@ def _install_layer_fastpath(model) -> None:
                 return self._moe_fast(x + r)
         return stock_layer_call(self, x, mask=mask, cache=cache)
 
+    layer_call._chad_fastpath = True  # type: ignore[attr-defined]
     q35.DecoderLayer.__call__ = layer_call  # type: ignore[method-assign]
 
 
@@ -300,14 +330,22 @@ def _moe_body(layer):
     sw = mlp.switch_mlp
     ln_w = layer.post_attention_layernorm.weight
     ln_eps = layer.post_attention_layernorm.eps
-    gate = mlp.gate
     fw, fs, fb = sw._fused_w, sw._fused_s, sw._fused_b
     sgs, sbits = sw._fused_gs, sw._fused_bits
     dp = sw.down_proj
     se = mlp.shared_expert
-    seg = mlp.shared_expert_gate
     k = mlp.top_k
     norm_topk = mlp.norm_topk_prob
+    E = mlp.num_experts
+    # mlx_moe_fused may have concatenated router|seg and shared gate|up; this
+    # body must then read the fused copies (originals are placeholders). Both
+    # variants are the same math per row.
+    moek = hasattr(mlp, "_rt_w")
+    if moek:
+        rt = (mlp._rt_w, mlp._rt_s, mlp._rt_b)
+        shg = (mlp._sh_w, mlp._sh_s, mlp._sh_b)
+    else:
+        gate, seg = mlp.gate, mlp.shared_expert_gate
 
     def qmm(x, m):
         return mx.quantized_matmul(x, m.weight, scales=m.scales, biases=m.biases,
@@ -316,7 +354,13 @@ def _moe_body(layer):
 
     def fwd(h):
         x = mx.fast.rms_norm(h, ln_w, ln_eps)
-        gates = mx.softmax(qmm(x, gate), axis=-1, precise=True)
+        if moek:
+            rg = mx.quantized_matmul(x, rt[0], scales=rt[1], biases=rt[2],
+                                     transpose=True, group_size=mlp._rt_gs,
+                                     bits=mlp._rt_bits)
+            gates = mx.softmax(rg[..., :E], axis=-1, precise=True)
+        else:
+            gates = mx.softmax(qmm(x, gate), axis=-1, precise=True)
         inds = mx.argpartition(gates, kth=-k, axis=-1)[..., -k:]
         scores = mx.take_along_axis(gates, inds, axis=-1)
         if norm_topk:
@@ -329,6 +373,13 @@ def _moe_body(layer):
         y = mx.gather_qmm(hh, dp.weight, dp.scales, dp.biases, rhs_indices=inds,
                           transpose=True, group_size=dp.group_size, bits=dp.bits)
         y = (y.squeeze(-2) * scores[..., None]).sum(axis=-2)
+        if moek:
+            sgu = mx.quantized_matmul(x, shg[0], scales=shg[1], biases=shg[2],
+                                      transpose=True, group_size=mlp._sh_gs,
+                                      bits=mlp._sh_bits)
+            g, u = mx.split(sgu, 2, axis=-1)
+            sh = qmm(nn.silu(g) * u, se.down_proj)
+            return h + y + mx.sigmoid(rg[..., E:E + 1]) * sh
         sh = qmm(nn.silu(qmm(x, se.gate_proj)) * qmm(x, se.up_proj), se.down_proj)
         return h + y + mx.sigmoid(qmm(x, seg)) * sh