mtplx 0.1.0rc1__tar.gz

mtplx-0.1.0rc1/CITATION.cff

@@ -0,0 +1,18 @@
+cff-version: 1.2.0
+title: MTPLX
+message: "If MTPLX helps your project, product, benchmark, article, or research, please cite it using this metadata."
+type: software
+authors:
+  - family-names: Altoukhi
+    given-names: Youssof
+repository-code: "https://github.com/youssofal/mtplx"
+url: "https://github.com/youssofal/mtplx"
+license: Apache-2.0
+version: 0.1.0rc1
+abstract: "Native MTP speculative decoding for Qwen3-Next on Apple Silicon, using built-in MTP heads with math-correct rejection sampling and an OpenAI/Anthropic-compatible serving surface."
+keywords:
+  - speculative decoding
+  - MTP
+  - MLX
+  - Apple Silicon
+  - local AI

mtplx-0.1.0rc1/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

mtplx-0.1.0rc1/MANIFEST.in

@@ -0,0 +1 @@
+include CITATION.cff

mtplx-0.1.0rc1/NOTICE

@@ -0,0 +1,13 @@
+MTPLX
+Copyright 2026 Youssof Altoukhi
+
+MTPLX is a native MTP speculative decoding project for Apple Silicon.
+
+Preferred attribution for public projects, products, benchmarks, articles, and
+research that use or build on MTPLX:
+
+  Powered by MTPLX by Youssof Altoukhi
+  https://github.com/youssofal/mtplx
+
+If MTPLX informs academic or technical writing, please cite the repository using
+the included CITATION.cff metadata.

mtplx-0.1.0rc1/PKG-INFO

@@ -0,0 +1,413 @@
+Metadata-Version: 2.4
+Name: mtplx
+Version: 0.1.0rc1
+Summary: Native MTP speculative decoding for Qwen3-Next on Apple Silicon.
+Author-email: Youssof Altoukhi <business@youssofal.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/youssofal/mtplx
+Project-URL: Documentation, https://github.com/youssofal/mtplx/tree/main/docs
+Project-URL: Issues, https://github.com/youssofal/mtplx/issues
+Project-URL: Citation, https://github.com/youssofal/mtplx/blob/main/CITATION.cff
+Keywords: mlx,apple-silicon,speculative-decoding,qwen,mtp,local-ai
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: MacOS
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Requires-Dist: fastapi>=0.136
+Requires-Dist: huggingface-hub>=0.36
+Requires-Dist: mlx<0.32,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
+Requires-Dist: mlx-lm<0.32,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
+Requires-Dist: numpy>=2
+Requires-Dist: pydantic>=2
+Requires-Dist: rich>=14
+Requires-Dist: safetensors>=0.6
+Requires-Dist: uvicorn>=0.46
+Provides-Extra: competitors
+Requires-Dist: dflash-mlx==0.1.0; extra == "competitors"
+Provides-Extra: server
+Requires-Dist: fastapi>=0.136; extra == "server"
+Requires-Dist: uvicorn>=0.46; extra == "server"
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.8; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Dynamic: license-file
+
+<div align="center">
+
+```
+  ███╗   ███╗ ████████╗ ██████╗  ██╗      ██╗  ██╗
+  ████╗ ████║ ╚══██╔══╝ ██╔══██╗ ██║      ╚██╗██╔╝
+  ██╔████╔██║    ██║    ██████╔╝ ██║       ╚███╔╝
+  ██║╚██╔╝██║    ██║    ██╔═══╝  ██║       ██╔██╗
+  ██║ ╚═╝ ██║    ██║    ██║      ███████╗ ██╔╝ ██╗
+  ╚═╝     ╚═╝    ╚═╝    ╚═╝      ╚══════╝ ╚═╝  ╚═╝
+```
+
+# **Native MTP speculative decoding on Apple Silicon**
+
+**~2.24× over no-MTP AR at `temp=0.6`** on Qwen3.6-27B · math-correct rejection sampling · MLX-native · zero external drafter
+
+<sub>Multiplier is hardware-independent. Absolute tok/s scales with memory bandwidth — current public record on M5 Max: **63.056 / 62.886 tok/s** D3, [`Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`](https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed).</sub>
+
+[![CI](https://github.com/youssofal/mtplx/actions/workflows/ci.yml/badge.svg)](https://github.com/youssofal/mtplx/actions/workflows/ci.yml)
+[![Python](https://img.shields.io/badge/python-3.11%E2%80%933.13-blue)](https://www.python.org/)
+[![macOS Apple Silicon](https://img.shields.io/badge/macOS-Apple%20Silicon-black?logo=apple)](https://developer.apple.com/metal/)
+[![Status](https://img.shields.io/badge/status-v0.1.0--preview.1-orange)](CHANGELOG.md)
+[![License](https://img.shields.io/badge/license-Apache--2.0-green)](LICENSE)
+
+</div>
+
+---
+
+MTPLX runs **the model's own built-in MTP heads** as a speculative drafter, with **exact probability-ratio acceptance + residual correction** — not the greedy-argmax trick most fast-decode tools use at T>0. That means real coding settings (`temperature=0.6`, `top_p=0.95`, `top_k=20`) actually get the speculative speedup *and* keep the target model's distribution.
+
+This is **not** DFlash, DDTree, llama-spec, or an external-drafter system. It's a native-MTP runtime built around MLX, Apple Silicon, and a real OpenAI/Anthropic-compatible serving surface.
+
+```bash
+python3 -m pip install -U mtplx
+mtplx start            # interactive: pick model → mode → web/CLI, then chat
+```
+
+That's it. The wizard handles the default speed model (`Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`), runtime mode, and surface (browser chat at `127.0.0.1:8000/` or terminal chat) on first run. On every subsequent run it asks "same as last time?" so you're one keypress from chatting.
+
+---
+
+## What you get
+
+- **Native MTP speculative decoding.** Built-in MTP heads, no external drafter, no RAM hit for a second model.
+- **Math-correct sampling at T=0.6.** Probability-ratio acceptance with residual correction. Verified `max_diff = 0.0` against reference single-token AR on the verified Qwen3.6-27B path.
+- **~2.24× over no-MTP AR at `temp=0.6`.** The hardware-independent number, which the CLI reports as `mean_speedup_vs_ar`. Verified contract on the public default `Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`: `63.056 / 62.886 tok/s` MTP-D3 vs `28.156 tok/s` no-MTP AR, on Apple Silicon M5 Max with `--max` fans, target sampler `temp=0.6 top_p=0.95 top_k=20`, draft sampler `temp=0.70`. Absolute tok/s scales with memory bandwidth; the 2.24× multiplier doesn't.
+- **Real serving surface.** OpenAI-compatible `/v1/chat/completions` + `/v1/completions` + `/v1/models`, Anthropic-compatible `/v1/messages` (streaming SSE), `/health`, `/metrics`. Plug it into Open WebUI, Claude Code, Cline, Continue, or anything that speaks OpenAI.
+- **In-browser chat UI** with auto-detected model context (256k for Qwen3.6), live tokens-per-second, markdown rendering, code-block copy buttons, a stop button, and a settings sidebar that persists per-machine.
+- **Interactive start wizard.** Pick model, mode, and surface in three numbered prompts. Returning users get "same as last time?". No flag-soup required.
+- **Local-folder model picker.** Point the wizard at any parent directory — your `~/models/`, the LM Studio cache, the HuggingFace cache — and it walks the tree, classifies each model into the four-tier compatibility contract, and presents a numbered picker. Config-only classification, never mmaps a tensor file, so a single APFS-dataless or partial download in the tree can't crash the picker.
+- **One-line live download progress.** Single rich-rendered line with bar / percent / GB / speed / ETA, streamed at 8 fps. HuggingFace's tqdm bars are suppressed during the download so they don't fight the MTPLX UI for terminal real estate.
+- **Honest profile names that tell you what they do.**
+  - `Medium` — default native-MTP speed path (`performance-cold`), in the **~2.2× over no-MTP AR** lane, not sustained without fan control.
+  - `Max` — Medium + ThermalForge fans pinned at 100%, **~2.24× over no-MTP AR** in the recorded lane (`63.056 / 62.886 tok/s` MTP vs `28.156 tok/s` AR on M5 Max), loud by design.
+  - `Stable` — hidden compatibility flag (`--profile stable` / `--profile safe`) for the exact/staged long-reply path.
+- **Crash-safe fan control.** When Max is on, MTPLX spawns a detached watchdog that restores fans to auto if the parent dies for any reason — including `kill -9` and "I closed the terminal". Verified live on hardware.
+- **Idle-aware Max mode.** Server tracks request activity; after 15 minutes of no chat, fans drop to auto, then ramp back up on the next message.
+- **Four-tier model compatibility contract.** `mtplx inspect <model>` reports: verified / arch-compatible-unverified / incompatible-architecture / no-MTP. No silent garbage runs.
+- **Lazy imports.** `mtplx --help`, `doctor`, `inspect`, `init`, `setup` work on a fresh venv *without MLX installed*. Generation and serving pull in MLX only when needed.
+- **Preview status: 562-test suite green**, including end-to-end onboarding, local-folder picker, live download progress, fan-control crash safety, OpenAI server fake-state, lazy-import survival, exactness gates.
+
+> **Preview honesty.** The cold path is verified at the **~2.24× multiplier** above. *Sustained* no-fan long-context throughput is currently in a worse lane on Flappy 10k versus the v0.2 target — the v0.1 release ships with this gap explicit. Closing it is the v0.2 deliverable; see [Roadmap](#roadmap).
+
+---
+
+## Quick start (full)
+
+```bash
+# 1. Install from PyPI
+python3 -m pip install -U mtplx
+
+# 2. Verify the install
+mtplx help
+mtplx doctor --json
+
+# 3. Chat (the wizard does everything)
+mtplx start
+```
+
+Power-user shortcuts (any of these skip the wizard):
+
+```bash
+mtplx start --fresh                         # re-run the wizard from scratch
+mtplx start cli                             # terminal chat directly
+mtplx start --max                           # browser chat with fan boost
+mtplx start --model /path/to/model          # use a specific local or HF model
+mtplx pull Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed
+mtplx quickstart --port 8000                # API server only, no chat
+```
+
+OpenAI-compatible smoke test:
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"mtplx","messages":[{"role":"user","content":"hi"}],"stream":true}'
+```
+
+The GitHub release wheel remains available for reproducible Preview 1 installs, but PyPI is the primary public path. If your Python blocks global installs, create and activate a virtual environment first:
+
+```bash
+python3 -m venv .venv
+. .venv/bin/activate
+python -m pip install -U pip mtplx
+```
+
+---
+
+## How it actually works
+
+Most "fast decode on Apple Silicon" projects fall into one of three buckets:
+
+| Approach | What they do at T>0 | What MTPLX does |
+|---|---|---|
+| llama.cpp / mlx-lm AR | No speculation, target model only | Speculative with a built-in drafter |
+| DFlash, prefix-match speculation | Greedy-argmax equality (silently breaks at T>0) | Probability-ratio acceptance + residual correction |
+| External-drafter speculation | Loads a second model into RAM | Uses the target's own MTP heads — zero extra RAM |
+
+The math-correctness wedge is real. At `temperature=0.6`, the difference between "rejected because the draft argmax disagrees" and "rejected via the Leviathan/Chen rejection-sampling theorem" is the difference between a benchmark trick and a runtime your code editor can trust. MTPLX does the latter, including residual correction `(p − q)+` for the cases where the draft was rejected.
+
+**Verified evidence (current public default `Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`):**
+- **~2.24× over matched no-MTP AR at `temp=0.6`** on Apple Silicon M5 Max: `63.056 / 62.886 tok/s` MTP-D3 paired runs vs `28.156 tok/s` no-MTP AR, same machine, same target sampler (`temp=0.6 top_p=0.95 top_k=20`), draft sampler `temp=0.70 top_p=0.95 top_k=20`, performance-cold profile, fans pinned by `--max`, thinking mode off. Recorded in `mtplx_runtime.json` under the model.
+- The multiplier is what the CLI reports as `mean_speedup_vs_ar`. Absolute tok/s above is M5-Max-with-614-GB/s-bandwidth-specific; if your Mac is slower you keep the **2.24×** ratio, the absolute number drops with bandwidth.
+- Per-position acceptance on the recorded prompt: `[100%, 97.96%, 93.88%]` at D3 (corrections=3 over 49 verify calls).
+- Distribution exactness vs reference single-token AR: `max_diff = 0.0`. Greedy diagnostic on the same cleaned window: `60.108 tok/s`.
+
+```mermaid
+flowchart LR
+    A[Prompt] --> B[Target model<br/>Qwen3.6-27B]
+    B --> C[Built-in MTP heads<br/>draft K=4]
+    C --> D[Probability-ratio<br/>acceptance + residual correction]
+    D --> E[Verified tokens]
+    E -->|loop| B
+    F[OpenAI-compatible server<br/>Anthropic-compatible /v1/messages]
+    E --> F
+    G[Browser chat<br/>or terminal chat]
+    F --> G
+```
+
+No second model, no greedy hack, no external drafter, no silent distribution drift.
+
+---
+
+## Modes
+
+Picked by `mtplx start`, or set explicitly via `--profile`. Every mode preserves exactness; the difference is the runtime path and whether MTPLX touches your fans.
+
+| Mode | Profile | Mechanics | Speed lane | Best for |
+|---|---|---|---|---|
+| **Medium** | `performance-cold` | Native-MTP speed path, Apple fan curve | ~2.2× over no-MTP AR, not sustained without fans | Default first run, short replies, snappy chat |
+| **Max** | `performance-cold` + `--max` | Medium path plus ThermalForge pinned to 100% | **~2.24× over no-MTP AR** (recorded: 63.056/62.886 vs 28.156 tok/s on M5 Max) | Sustained workloads, you don't mind fans |
+| **Stable** | `stable` / `safe` | Exact/staged long-reply path, hidden from onboarding | Lower peak speed, steadier shape | Compatibility and conservative long replies |
+
+`Max` requires ThermalForge. `mtplx max --install` installs it from source into `~/.mtplx/bin/thermalforge`, sets up a passwordless sudoers rule scoped to that one binary, and verifies fans actually ramp before declaring success. One sudo prompt, end-to-end. Crash safety covers SIGINT, SIGTERM, SIGHUP, terminal close, and `kill -9` via a detached sidecar process.
+
+---
+
+## Compatibility
+
+```bash
+mtplx inspect <model-path-or-hf-repo> --json
+```
+
+| Tier | Means | Behavior |
+|---|---|---|
+| **Verified** | Has `mtplx_runtime.json` and passed MTPLX gates | Runs |
+| **Arch-compatible, unverified** | Qwen3-Next MTP markers detected, no runtime contract | Refuses unless `--unsafe-force-unverified` |
+| **Incompatible architecture** | MTP exists but not Qwen3-Next | Clear error, roadmap pointer |
+| **No MTP** | No MTP head detected | Clear error, no garbage runs |
+
+v0.1 ships verified Qwen3.6-27B via `Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed`, with public served model id `mtplx-qwen36-27b-optimized-speed`. The compatibility registry already detects DeepSeek V3 / V3.2, GLM-4 MoE / MoE-Lite, MiMo, and MiniMax M2 — unsupported runtime families stay behind explicit compatibility gates rather than silently running.
+
+### Support matrix
+
+| Area | Preview support |
+|---|---|
+| Mac | Apple Silicon only (`arm64`) |
+| macOS | 14.0+; Sequoia is supported |
+| Python | native arm64 Python 3.10+ |
+| MLX | `python3 -m pip install mlx` in the same native environment |
+| Memory | dynamic preflight; warns below 48 GiB, fails when the selected model/profile estimate exceeds 80% of unified memory |
+| Storage | first download requires `max(model_size * 2.5, model_size + 20 GiB)` free on the model-cache filesystem |
+| Docker/Open WebUI | Docker Desktop current plus previous two macOS major releases |
+
+Run `mtplx doctor --summary`, `mtplx doctor --deep --json`, or `mtplx doctor --bundle` before filing a bug. Bundles are redacted by default under `~/.mtplx/reports/`.
+
+---
+
+## CLI surface
+
+```bash
+mtplx start                 # interactive setup, then chat
+mtplx help                  # detailed help; `mtplx help <command>` for any
+mtplx doctor                # install + model + integration health
+mtplx inspect <model>       # four-tier compatibility report
+mtplx init                  # write ~/.mtplx/config.toml
+mtplx setup                 # download verified model, prepare cache
+mtplx pull                  # download the default HF model safely
+mtplx models                # cached models, validation, size, delete command
+mtplx run "..."             # one-shot ask
+mtplx chat                  # terminal chat
+mtplx start                 # OpenAI/Anthropic-compatible server
+mtplx connect openwebui     # paste settings for Open WebUI
+mtplx openwebui docker-command
+mtplx bench run --suite cold-long-code-192
+mtplx max --install         # install ThermalForge for Max mode
+mtplx max --status          # fan / thermal state
+```
+
+Every command has `--json` for machine-readable output and `--help` for context-specific docs.
+
+---
+
+## Architecture
+
+The architectural achievement is **a single-model native-MTP runtime that's mathematically exact at temperature**, with a real serving surface bolted on. There is no second drafter, no greedy hack, and no "drop in a fast-decode library" wrapper. Four layers, drawn the way they actually run.
+
+### 0. MLX runtime layer (the kernel stack we own)
+
+MTPLX is not a thin wrapper over stock MLX — the speed lane sits on top of an **MLX source fork** plus a small set of **custom Metal kernels** registered as primitives. Stock `mlx-lm` cannot reproduce the multiplier above; the runtime layer is what makes the speculative cycle in §2 tractable on Apple Silicon.
+
+What we changed at the MLX source level (fork: `mlx-mtplx-0.31.2-qmm`, commit `2377a99f` "Tune small-M qmv for MTPLX 60TPS path"):
+
+- **Small-M `qmv` retuning.** The verify forward is dominated by quantized-matrix-vector ops at `M ≈ 3..6` (one position per accepted draft). Stock MLX's `qmv_fast_impl` is tuned for large M and stalls dispatch at small M. Our fork: `BN16` group-size, **4-simdgroup** instead of 2-simdgroup, `unroll_count(4)` on the inner loop. Cuts the verify-MLP region by enough to be the difference between "MTP loses to AR" and "MTP at ~2.24×".
+- **Source-primitive registration.** Custom kernels (below) are registered through `mlx.core.fast.metal_kernel` and integrated into MLX's graph the same way stock primitives are, so `mx.compile` can fuse around them and `mx.eval` doesn't see them as opaque blocks.
+
+Custom Metal kernels we shipped on top of the fork:
+
+- **`linear-gdn-from-conv-tape`** — the GDN linear-attention path during verify. Records an *innovation tape* of `(token, gate, state-delta)` tuples during the draft phase, then **replays** them deterministically on rollback when a draft is rejected. Replaces stock MLX's `Conv1d` + recurrent-state restore with a single fused kernel that's bit-exact (`max_diff = 0.0` against batched-vs-sequential reference) and shape-stable.
+- **`verify_qmv` (small-M qmv kernel).** Direct successor of dflash-mlx's M=16 idea, retuned for MTPLX's M=3..6 verify shapes. Now subsumed by the MLX-source qmv tuning above for the verify hot path; remains as a standalone primitive for diagnostic regressions.
+- **GraphBank.** A cache of `mx.compile`-compiled verify graphs, keyed by `(suffix_length, depth, profile)`. Each verify shape gets one compiled graph reused across cycles — no per-cycle Python dispatch overhead. Capture-commit + GraphBank together hit `capture_commit_time_s ≈ 0.073 ms` per cycle (vs `verify_time_s ≈ 47 ms` per cycle), i.e. the commit step is three orders of magnitude smaller than the verify itself.
+- **Draft-only 4-bit / 3-bit LM head** built in memory by `scripts/probe_draft_lm_head_requant.py`. The target's `lm_head` stays at the model's actual precision (BF16 / INT4 affine); the drafter gets a separate, much smaller LM-head requantized for proposal use only. Cuts draft time by ~29% without touching target accuracy.
+
+Runtime knobs that ship on by default in `performance-cold`:
+
+- `MTPLX_LAZY_VERIFY_LOGITS=1` · `MTPLX_BATCH_TARGET_ARRAYS=1` · `MTPLX_LAZY_MTP_HISTORY_APPEND=1` · `MTPLX_DROP_EVENTS=1` · `MTPLX_SKIP_VERIFY_SNAPSHOT=1`.
+
+Numerical hygiene (these are correctness fixes, not speed):
+
+- **`fp32` `p/q` ratio** during probability-ratio acceptance. The Leviathan–Chen ratio underflows in BF16 at small `q`; fp32 is the only safe path.
+- **`mx.random.split` per draft position** so each acceptance roll uses an independent RNG key. Without this, depth>1 would silently correlate accept decisions.
+
+```mermaid
+flowchart TB
+    subgraph FORK["MLX source fork · mlx-mtplx-0.31.2-qmm"]
+        QMV["small-M qmv: BN16 · 4-simdgroup · unroll_count(4)<br/>tuned for M=3..6 verify shapes"]
+        REG["mx.fast.metal_kernel + source-primitive registration<br/>(mx.compile fuses across our kernels)"]
+    end
+    subgraph KERNS["Custom Metal kernels"]
+        TAPE["linear-gdn-from-conv-tape<br/>fused GDN verify + innovation-tape rollback"]
+        VQMV["verify_qmv · small-M qmv (diagnostic)"]
+        DLM["Draft-only 4/3-bit LM head<br/>built in memory, target lm_head untouched"]
+    end
+    subgraph GRAPH["Compiled graphs"]
+        BANK2["GraphBank · mx.compile per (suffix_len, depth, profile)<br/>capture_commit_time ≈ 0.073 ms / cycle"]
+    end
+    subgraph HYGIENE["Numerical hygiene"]
+        FP32["fp32 p/q ratio (BF16 underflow at small q)"]
+        RNG["mx.random.split per draft position"]
+    end
+    FORK --> KERNS
+    KERNS --> GRAPH
+    GRAPH --> HOT["used by speculative cycle in §2"]
+    HYGIENE --> HOT
+```
+
+### 1. Single-model runtime
+
+The target model and the drafter are the **same checkpoint**. Qwen3.6-27B ships native MTP heads; MTPLX uses them as the speculative drafter. Zero RAM cost for a second model, zero distillation, zero "we trained a drafter" handoff. The trunk's KV cache obeys a **committed-history contract** (verified against the vLLM CUDA reference at cosine > 0.9998 through D5) so recursive draft depth holds together — that's what lets D2/D3/D4 acceptance reach the 90s instead of collapsing.
+
+```mermaid
+flowchart LR
+    subgraph TGT["Target model · Qwen3.6-27B (single checkpoint)"]
+        TRUNK["Trunk · 64 layers (48 GDN + 16 full-attn)<br/>committed-history KV cache"]
+        HEAD["Built-in MTP heads · recursive depth K=3 default"]
+        TRUNK -.shares hidden states.-> HEAD
+    end
+```
+
+### 2. Speculative cycle (the hot loop)
+
+Per cycle: the MTP head drafts K tokens, the target verifies all K in parallel via one batched forward, **probability-ratio acceptance** (Leviathan–Chen) decides per-position, **residual correction `(p − q)+`** emits a clean replacement on rejection, and a **bonus token** falls out for free when all K accept. Verify cost is paid by `capture_commit` + the `linear-gdn-from-conv-tape` GDN kernel + a **GraphBank** of compiled verify shapes; the math is exact at any temperature.
+
+```mermaid
+flowchart LR
+    DRAFT["MTP head drafts q₁..q_K<br/>+ proposal probabilities"] --> VERIFY
+    VERIFY["Target batched verify forward<br/>capture-commit · linear-gdn-from-conv-tape · GraphBank"] --> ACCEPT
+    ACCEPT["Probability-ratio acceptance<br/>(Leviathan–Chen at any T)"] -->|all K accepted| BONUS
+    ACCEPT -->|rejected at i| CORR
+    BONUS["Bonus token at K+1 (free)"] --> COMMIT
+    CORR["Residual (p − q)+ token at i"] --> COMMIT
+    COMMIT["Committed-history KV writeback"] -->|next cycle| DRAFT
+```
+
+### 3. Serving stack
+
+The runtime is wrapped in a real serving surface so you can point Open WebUI / Claude Code / Cline / Continue / `curl` / `openai-python` / `anthropic-python` at it. **Engine sessions** keep per-chat state; the **Session Bank** preserves warm-prefix exact state across turns (verified `logits_max_abs_diff = 0.0` against fresh forwards) so multi-turn TTFT doesn't collapse the way a stateless shim would.
+
+```mermaid
+flowchart TB
+    subgraph CLIENTS["Clients"]
+        BR["Browser chat<br/>127.0.0.1:8000"]
+        OW["Open WebUI · Cline · Claude Code · Continue"]
+        CURL["curl · openai-python · anthropic-python"]
+        TERM["Terminal chat (mtplx start cli)"]
+    end
+    subgraph API["FastAPI server"]
+        OAI["/v1/chat/completions · /v1/completions · /v1/models"]
+        ANT["/v1/messages (Anthropic SSE translator)"]
+        OBS["/health · /metrics"]
+    end
+    subgraph ENG["Engine layer"]
+        SESS["Engine sessions (per-chat context + cache)"]
+        BANK["Session bank · warm-prefix exact-state reuse"]
+    end
+    BR --> OAI
+    OW --> OAI
+    OW --> ANT
+    CURL --> OAI
+    CURL --> ANT
+    TERM --> ENG
+    OAI --> SESS
+    ANT --> SESS
+    SESS --> BANK
+    BANK -->|drives| RUNTIME["Native-MTP runtime (cycle above)"]
+    OBS --- ENG
+```
+
+The CLI (`mtplx start` / `pull` / `doctor` / `inspect` / `max`) is the on-ramp to all of the above and not the architectural story — it lazy-imports MLX so `--help`, `doctor`, `inspect`, `init`, `setup` work on a fresh venv with no GPU/Apple-Silicon stack installed.
+
+---
+
+## Roadmap
+
+**v0.1.0-preview.1 (today).** Verified Qwen3-Next-MTP cold path, OpenAI/Anthropic-compatible serving, in-browser chat, interactive `mtplx start` wizard with local-folder model picker and one-line live download progress, four-tier compatibility, crash-safe Max mode, lazy-import CLI surface, 562-test suite green.
+
+**v0.2 — sustained throughput.** Diagnostic-gated kernel ladder targeting `last64/first64 ≥ 0.90` no-fan on 10k generations while preserving the **~2.24× multiplier** lane. Mechanism-driven: lazy-graph severance + output narrowing if graph history is the bottleneck; MLX-primitive-registered cache-update + `mx.compile` if dispatch tax dominates; an owned GDN+MLP verify-cycle kernel via `mx.fast.metal_kernel` only if the cheaper paths don't close the gap.
+
+**v0.3 — broader fleet.** DeepSeek V3 / V3.2 MTP backend (registered, runtime pending), GLM-4 MoE backend, MiMo backend, generic MTP backend behind `mtplx_runtime.json`. Optional Homebrew tap. Multi-session server concurrency.
+
+The kernel-ladder direction is grounded in a six-agent deep-research synthesis (Compass / GPT Pro / Gemini ×2 / Claude ×2 / final validation pass) plus a closed-branch failure ledger that's already 35+ entries deep. We don't ship benchmark theater.
+
+---
+
+## What MTPLX is *not*
+
+- It's not DFlash. DFlash uses greedy-argmax prefix matching and breaks the target distribution at T>0. MTPLX implements exact probability-ratio rejection sampling.
+- It's not an external-drafter system. There's no second model. The drafter is the target's own MTP heads.
+- It's not a generic "speculative decoding library". It's a runtime + serving stack with an explicit model-compatibility contract.
+- It's not a CUDA project. MTPLX is MLX-native and Apple-Silicon-first. Linux/CUDA is not on the roadmap; for that, use vLLM.
+- It's not finished. v0.1 is a preview. The **~2.24× multiplier** cold-lane target is met, the sustained-no-fan target is not, and the README says so.
+
+---
+
+## License, citation, and attribution
+
+MTPLX builds on [MLX](https://github.com/ml-explore/mlx) and the Qwen3-Next model family. The speculative-sampling math follows Leviathan & Chen 2023 ("Fast Inference from Transformers via Speculative Decoding") and the MTP heads ship with Qwen. Design and diagnostics are informed by vLLM speculative decoding, vLLM-Metal (issues #188 and #281), DFlash-MLX, DDTree-MLX, and DeepSeek V3.2's `mx.depends` precedent. Optional fan control via [ThermalForge](https://github.com/ProducerGuy/ThermalForge). Model weights and licenses remain governed by their upstream model cards.
+
+MTPLX is released under the [Apache License 2.0](LICENSE). If you redistribute MTPLX or derivative works, preserve the Apache license and the attribution notices from [NOTICE](NOTICE) as required by Apache-2.0.
+
+If MTPLX powers a public project, product, benchmark, article, or research result, please include clear credit in your README, docs, paper, or public writeup:
+
+> Powered by MTPLX by Youssof Altoukhi
+>
+> https://github.com/youssofal/mtplx
+
+For academic or technical writing, cite the repository using [CITATION.cff](CITATION.cff).
+
+— Built by [Youssof Altoukhi](https://github.com/youssofal). Contributions, bug reports, and benchmark replications welcome via [Issues](https://github.com/youssofal/mtplx/issues).