mtplx 2.4.0__tar.gz → 2.4.2__tar.gz

{mtplx-2.4.0 → mtplx-2.4.2}/CHANGELOG.md

@@ -4,6 +4,157 @@ All notable user-facing changes to MTPLX. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [2.4.2] - 2026-08-02
+
+The agentic-cache release: the session cache stops losing warm state
+mid-run, tool-turn commits stop being ghosts, every serve keeps a durable
+per-request trail by default, an experimental DeepSeek-V4-Flash backend
+lands, and the documentation now matches the code everywhere it was
+audited.
+
+### Added
+
+- DeepSeek-V4-Flash: experimental native AR backend
+  (`model_type: deepseek_v4`) — Hyper-Connections, compressed sparse
+  attention, hash-routed MoE, grouped output-LoRA — loading the
+  mlx-community checkpoints directly, with an optional single-block MTP
+  speculative lane when the checkpoint carries `mtp.0.*` weights
+  (spec == AR gated; K=1-3 measured up to 2.28x on the 2bit-DQ build).
+  MTP-declaring checkpoints that ship no draft weights degrade to AR
+  with a clear message instead of failing at bind. Thanks @davidtai
+  (#216).
+- Request log, default on: every serve writes numeric/hash per-request
+  telemetry to `~/.mtplx/logs/request-log-<port>.jsonl` (64 MB x 4
+  rotation; no prompt or completion content; disable with
+  `MTPLX_REQUEST_LOG_JSONL=off`). Pairs with 2.4.1's opt-in bit-exact
+  request capture to make agent-session incidents diagnosable after the
+  fact (#196/#197). New helpers: `scripts/gauntlet_scoreboard.py`
+  per-session summarizer, `scripts/oc_tap.py` recording proxy and
+  `scripts/oc_tap_diff.py` request-mutation analyzer for content-level
+  wire truth.
+- Session bank, active-session eviction protection: sessions that
+  touched the bank within `MTPLX_SESSION_BANK_ACTIVE_PIN_TTL_S`
+  (default 600 s) are eviction-last, so cross-session pressure evicts
+  idle victims instead of the session that is mid-run.
+- Session bank, newest-K per-session snapshot retention
+  (`MTPLX_SESSION_BANK_PER_SESSION_MAX_ENTRIES`, default 3): divergent
+  per-turn sibling snapshots no longer accumulate unreclaimed.
+  `/health` now reports active sessions, the pin TTL, and recent
+  evictions.
+- Postcommit foreground grace (`MTPLX_POSTCOMMIT_FOREGROUND_GRACE_S`,
+  default 2 s): a nearly-finished background cache commit lands instead
+  of being preempted by the next fast agent-loop request.
+- Session identity honors `x-session-affinity` / `x-session-id` request
+  headers (OpenCode sends these per request), ending cross-request
+  identity churn on that client.
+
+### Fixed
+
+- Tool-turn "ghost re-prefills": the tool-rewrite async commit rendered
+  a canonical history that matched neither the generation nor the next
+  prompt, burning full-history re-forwards (26.8 s observed) without
+  ever storing. It is disabled pending a byte-proven canonical render
+  (`MTPLX_IDLE_POSTCOMMIT_TOOL_REWRITE` re-enables);
+  store-on-prefill and block salvage cover the lane.
+- The bridge's convergence guard now states explicitly that editing and
+  verification tools remain allowed and that its restriction covers
+  only the current reply — a model read the old wording as a
+  session-wide tool ban and stalled an entire session.
+- `mtplx profile thermal`, `profile eval-attribution`,
+  `profile dispatch --trace`, and `thermal fanmax-run` invoked
+  research-workspace scripts that are not part of the distribution, and
+  `--dry-run` printed those phantom paths as runnable commands. They
+  now report availability honestly (exit 2, machine-readable
+  `available: false`) and run the real script when present.
+- `mtplx doctor`: Python floor corrected to 3.11 (matching
+  `requires-python`); remediation texts no longer tell end users to
+  edit source constants or to move a healthy server off its port;
+  `--port` is documented and, when passed explicitly, aims the server
+  connectivity checks.
+- Session-bank near-prefix restores on backends with bounded rollback
+  (DeepSeek-V4) pre-check `max_rollback` and fall back to a cold
+  prefill instead of raising (#216).
+- Help surfaces match their own parsers: the onboarding help no longer
+  promises a Turbo wizard choice that does not exist (Turbo
+  auto-selects for the quantized flagships), `--strict-cold` names the
+  enforced 59 tok/s gate, `--open-dashboard` opens alongside the chosen
+  client (as it always did), and the command reference teaches
+  `mtplx <command> --help`, which also works for multi-word commands.
+
+### Documentation
+
+- Full truth sweep: ~450 documentation claims reconciled against the
+  code across 27 files. Highlights: INSTALL.md no longer references an
+  MLX fork removed in 2.0.0; turbo-verify.md no longer calls the
+  shipped default "experimental, off by default" nor excludes the
+  6-bit lane that ships; the Anthropic base-URL instruction (docs and
+  the canonical example) no longer 404s; `/metrics` no longer claims a
+  Prometheus mode that never existed; the README modes table shows
+  Turbo as the default for the quantized 27B/9B flagships; the Laguna
+  memory requirement states the real ~85.3 GiB preflight gate;
+  version-era staleness ("v0.1", "preview", v0.3.x runbook pins) is
+  cleared; historical release notes gain bracketed corrections where
+  they documented commands that never worked. Thanks
+  @PhilipJohnBasile for #218 (removed the unsupported MTP-sidecar
+  graft guidance; seeded by #215).
+- Dependency-record correction: the transformers pin has been
+  `<5.14,!=5.13.0` since shortly after 2.0.0; the changelog never
+  recorded the relaxation from `<5.13`.
+
+### Dependencies
+
+- pypa/gh-action-pypi-publish 1.14.1 -> 1.14.2 (#217).
+
+## [2.4.1] - 2026-08-01
+
+The smooth-streaming release: the app's chat render path is overhauled
+(no more freeze-then-catch-up stutter, scroll bounce, or plain-text code
+blocks — real syntax coloring, live code cards, tables, and actual math
+notation), and the 2.4.0 short-turn regression is fixed.
+
+### Added
+
+- Live syntax coloring for code blocks (12 languages + generic) from a
+  freeze-time lexer that colors each line exactly once; streaming cost
+  is O(new text), never O(document).
+- Streaming code card: an open fence renders as a live card with
+  colored lines and flips once to its settled form at close.
+- Pipe tables render as real tables; math renders as real notation
+  (Unicode super/subscripts, stacked matrices and fractions, inline
+  conversion instead of dollar-sign leaks).
+- Typewriter pacing for streamed text with geometric catch-up and a
+  hard drain bound (`MTPLX_STREAM_TYPEWRITER=0` to disable), and a live
+  tok/s chip computed over a sliding ~5 s window.
+- Performance mode is a true kill switch: plain text only, through both
+  the streaming and settled render paths.
+- Opt-in per-request capture for bit-exact failure replay:
+  `MTPLX_REQUEST_CAPTURE_DIR=<dir>` persists each request's
+  reproduction envelope at dispatch time (#196/#197, third layer).
+- Opt-in frontend stream-performance probe (`MTPLX_UI_PERF=1`, HUD via
+  `MTPLX_UI_PERF_HUD=1`) with a per-turn JSONL trace joinable to engine
+  stats by request id.
+- Experimental: cost-model speculative-depth policy
+  (`--adaptive-policy cost`) and blocked-sequential GDN prefill
+  (`MTPLX_GDN_BLOCKED_PREFILL=1`). Defaults unchanged.
+
+### Fixed
+
+- 2.4.0 short-turn regression: the compiled-verify path could reserve
+  KV budget above the configured ceiling, taxing short requests with
+  setup work they never used; the reserve is now clamped.
+- Warming prefills yield to real traffic within one small chunk instead
+  of delaying a freshly arrived request.
+- Derivative model artifacts whose names extend a first-party model
+  name are served under their own id, not the flagship's — the health
+  payload, OpenAI `model` field, and app model chip now report the
+  artifact actually loaded.
+- Streaming render: line-segment coalescing keeps realized view count
+  bounded on long answers; the bottom-pin scroll correction runs in the
+  same display cycle as layout so the streaming bubble can no longer
+  visibly bounce; a per-display-cycle window-sizing walk that floored
+  every update at ~50 ms is removed (`MTPLX_APP_SIZING_TUNER=0`
+  restores it).
+
 ## [2.4.0] - 2026-07-31
 
 The 35B speed release: the 35B-A3B MoE gets a compiled decode stack and
@@ -676,4 +827,17 @@ working as one product. Full notes:
   completions, and Anthropic `stop_sequences`) and `/v1/completions`
   streams tokens as they are generated with real finish reasons.
 
-[1.0.0]: https://github.com/youssofal/mtplx/releases/tag/v1.0.0
+[2.4.2]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.2
+[2.4.1]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.1
+[2.4.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.0
+[2.3.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.3.0
+[2.2.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.2.0
+[2.1.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.1.0
+[2.0.2]: https://github.com/youssofal/MTPLX/releases/tag/v2.0.2
+[2.0.1]: https://github.com/youssofal/MTPLX/releases/tag/v2.0.1
+[2.0.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.0.0
+[1.0.4]: https://github.com/youssofal/MTPLX/releases/tag/v1.0.4
+[1.0.3]: https://github.com/youssofal/MTPLX/releases/tag/v1.0.3
+[1.0.2]: https://github.com/youssofal/MTPLX/releases/tag/v1.0.2
+[1.0.1]: https://github.com/youssofal/MTPLX/releases/tag/v1.0.1
+[1.0.0]: https://github.com/youssofal/MTPLX/releases/tag/v1.0.0

{mtplx-2.4.0 → mtplx-2.4.2}/CITATION.cff

@@ -8,7 +8,7 @@ authors:
 repository-code: "https://github.com/youssofal/mtplx"
 url: "https://github.com/youssofal/mtplx"
 license: Apache-2.0
-version: 0.1.0rc1
+version: 2.4.2
 abstract: "Native MTP speculative decoding for Qwen3-Next on Apple Silicon, using built-in MTP heads with math-correct rejection sampling and an OpenAI/Anthropic-compatible serving surface."
 keywords:
   - speculative decoding

{mtplx-2.4.0 → mtplx-2.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mtplx
-Version: 2.4.0
+Version: 2.4.2
 Summary: Native MTP speculative decoding for Qwen3-Next on Apple Silicon.
 Author-email: Youssof Altoukhi <business@youssofal.com>
 License-Expression: Apache-2.0
@@ -105,9 +105,11 @@ On a 16 GB M4 Mac mini, tuning the 9B model lands on depth 1: 14.4 tok/s baselin
 
 <img src="docs/assets/readme/app-forge.jpg" alt="Forge verifying a freshly built MTP model" width="100%" />
 
-Forge takes a Hugging Face repo and turns it into an MTPLX-ready MTP model: convert to MLX, train the MTP adapter, verify that the result is actually faster and still exact, and publish back to the Hub if you want to share it. The honest part matters: Forge measures before and after on your hardware and shows you the verdict ("Depth 1 is fastest: 227.1 to 296.1, 1.30x") rather than assuming the adapter helped. Available in the app and as `mtplx forge`.
+Forge takes a Hugging Face repo and turns it into an MTPLX-ready MTP model: convert to MLX, train the MTP adapter, verify that the result is actually faster and still exact, and publish back to the Hub if you want to share it. The honest part matters: Forge measures before and after on your hardware and shows you the verdict ("Depth 1 is fastest: 227.1 to 296.1, 1.30x") rather than assuming the adapter helped. Available in the app and as `mtplx forge` subcommands.
 
-The official catalog lives on Hugging Face under [Youssofal](https://huggingface.co/Youssofal): Qwen 3.5 (4B, 9B), Qwen 3.6 (27B, 35B MoE) in speed, balance, and quality builds, plus Gemma 4. The app recommends from these based on your hardware.
+MTPLX does not support attaching a separately supplied MTP sidecar to an arbitrary MLX trunk. Matching architecture fields, tensor shapes, or provenance labels cannot prove that the head was trained against those exact trunk weights. Use a complete model that already includes its matching MTP weights, or use Forge to build and verify an artifact from its original source checkpoint.
+
+The official catalog lives on Hugging Face under [Youssofal](https://huggingface.co/Youssofal): Qwen 3.5 (4B, 9B), Qwen 3.6 (27B, 35B MoE) in speed and quality builds (the 35B MoE adds a balance build), plus Gemma 4. The app recommends from these based on your hardware.
 
 ## The server
 
@@ -119,7 +121,7 @@ curl http://127.0.0.1:8000/v1/chat/completions \
   -d '{"model":"mtplx","messages":[{"role":"user","content":"hi"}],"stream":true}'
 ```
 
-Sessions survive: a warm-prefix session bank keeps multi-turn chats fast, and an optional SSD cache restores sessions near-instantly across restarts.
+Sessions survive: a warm-prefix session bank keeps multi-turn chats fast, and a default-on SSD session cache restores sessions near-instantly across restarts (disable with `--ssd-session-cache off`).
 
 Sampler controls cover `temperature`, `top_p`, `top_k`, and the OpenAI penalty pair `presence_penalty` / `frequency_penalty` — per request, as server defaults (`--default-presence-penalty` / `--default-frequency-penalty` on `start`/`serve`/`quickstart`), or live via `mtplx settings set` and the app's Presence Penalty dial. Penalties default to 0, which is an exact no-op that preserves MTP exactness. Qwen's guidance: leave them at 0 for coding and agent work; ~0.5–1.5 presence penalty helps creative writing or when a model loops on itself.
 
@@ -133,20 +135,21 @@ mtplx pull <hf-repo>       # download a model safely
 mtplx models               # what is cached, sizes, validation
 mtplx inspect <model>      # compatibility report before anything runs
 mtplx tune --retune        # measure AR vs D1/D2/D3 on your Mac
-mtplx forge                # build, verify, and publish MTP models
+mtplx forge --help         # build, verify, and publish MTP models (probe/build/publish/verify subcommands)
 mtplx bench aime --quick   # run the AIME benchmark from the terminal
 mtplx doctor               # install and integration health
 mtplx max --install        # fan control (one sudo prompt, crash-safe)
 mtplx settings get/set     # read or change live server settings
 ```
 
-Every command takes `--json` and `--help`. The CLI works without MLX installed for everything that does not need a model, so `doctor` and `inspect` run on any machine.
+Every command takes `--help`, and most inspection/diagnostic commands take `--json`. The CLI works without MLX installed for everything that does not need a model, so `doctor` and `inspect` run on any machine.
 
 ## Modes
 
 | Mode | What it does | When |
 |---|---|---|
-| **Sustained** | Default. Long-context MTP path with chunked prefill and request-sized KV | Everyday use, big files, 16K-200K prompts |
+| **Turbo** | NAX verify kernels + compiled verify; the default for the quantized 27B and 9B flagship models | Picked automatically for those models |
+| **Sustained** | Default for all other models. Long-context MTP path with chunked prefill and request-sized KV | Everyday use, big files, 16K-200K prompts |
 | **Sustained Max** | Sustained with fans pinned at 100% | Long work where you want maximum cooling |
 | **Burst** | Legacy short-context benchmark lane, loud | Short prompts and benchmarks only |
 
@@ -154,7 +157,7 @@ Fan-backed modes restore your fans to automatic if MTPLX dies for any reason, in
 
 ## Compatibility, honestly
 
-`mtplx inspect` classifies models before anything runs: verified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+`mtplx inspect` classifies models before anything runs: verified, family-compatible but unverified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models load with an explicit unverified label. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
 
 [Laguna-S-2.1 oQ4e](https://huggingface.co/mlx-community/Laguna-S-2.1-oQ4e) is supported through its exact MLX architecture in target-only AR mode:
 
@@ -170,8 +173,10 @@ MTPLX pins that model to revision
 tokenizer, generation config, special tokens map, and Poolside chat template
 before admitting it. The checkpoint has no native MTP head, so an MTP launch is
 rejected before weights load instead of falling back during execution. The
-weights occupy 59.72 GiB (64.13 GB); use a Mac with at least 96 GiB unified
-memory (128 GiB is recommended). MTPLX defaults Laguna to a 32,768-token context
+weights occupy 59.72 GiB, a 64.13 GB snapshot on disk. The launch preflight
+requires about 85 GiB of unified memory (weights, runtime headroom, and a
+16 GiB system reserve) — in practice a 96 GB Mac; 128 GB is
+comfortable. MTPLX defaults Laguna to a 32,768-token context
 and response cap, and checks larger explicit server contexts against the active
 Metal memory cap.
 

{mtplx-2.4.0 → mtplx-2.4.2}/README.md

@@ -55,9 +55,11 @@ On a 16 GB M4 Mac mini, tuning the 9B model lands on depth 1: 14.4 tok/s baselin
 
 <img src="docs/assets/readme/app-forge.jpg" alt="Forge verifying a freshly built MTP model" width="100%" />
 
-Forge takes a Hugging Face repo and turns it into an MTPLX-ready MTP model: convert to MLX, train the MTP adapter, verify that the result is actually faster and still exact, and publish back to the Hub if you want to share it. The honest part matters: Forge measures before and after on your hardware and shows you the verdict ("Depth 1 is fastest: 227.1 to 296.1, 1.30x") rather than assuming the adapter helped. Available in the app and as `mtplx forge`.
+Forge takes a Hugging Face repo and turns it into an MTPLX-ready MTP model: convert to MLX, train the MTP adapter, verify that the result is actually faster and still exact, and publish back to the Hub if you want to share it. The honest part matters: Forge measures before and after on your hardware and shows you the verdict ("Depth 1 is fastest: 227.1 to 296.1, 1.30x") rather than assuming the adapter helped. Available in the app and as `mtplx forge` subcommands.
 
-The official catalog lives on Hugging Face under [Youssofal](https://huggingface.co/Youssofal): Qwen 3.5 (4B, 9B), Qwen 3.6 (27B, 35B MoE) in speed, balance, and quality builds, plus Gemma 4. The app recommends from these based on your hardware.
+MTPLX does not support attaching a separately supplied MTP sidecar to an arbitrary MLX trunk. Matching architecture fields, tensor shapes, or provenance labels cannot prove that the head was trained against those exact trunk weights. Use a complete model that already includes its matching MTP weights, or use Forge to build and verify an artifact from its original source checkpoint.
+
+The official catalog lives on Hugging Face under [Youssofal](https://huggingface.co/Youssofal): Qwen 3.5 (4B, 9B), Qwen 3.6 (27B, 35B MoE) in speed and quality builds (the 35B MoE adds a balance build), plus Gemma 4. The app recommends from these based on your hardware.
 
 ## The server
 
@@ -69,7 +71,7 @@ curl http://127.0.0.1:8000/v1/chat/completions \
   -d '{"model":"mtplx","messages":[{"role":"user","content":"hi"}],"stream":true}'
 ```
 
-Sessions survive: a warm-prefix session bank keeps multi-turn chats fast, and an optional SSD cache restores sessions near-instantly across restarts.
+Sessions survive: a warm-prefix session bank keeps multi-turn chats fast, and a default-on SSD session cache restores sessions near-instantly across restarts (disable with `--ssd-session-cache off`).
 
 Sampler controls cover `temperature`, `top_p`, `top_k`, and the OpenAI penalty pair `presence_penalty` / `frequency_penalty` — per request, as server defaults (`--default-presence-penalty` / `--default-frequency-penalty` on `start`/`serve`/`quickstart`), or live via `mtplx settings set` and the app's Presence Penalty dial. Penalties default to 0, which is an exact no-op that preserves MTP exactness. Qwen's guidance: leave them at 0 for coding and agent work; ~0.5–1.5 presence penalty helps creative writing or when a model loops on itself.
 
@@ -83,20 +85,21 @@ mtplx pull <hf-repo>       # download a model safely
 mtplx models               # what is cached, sizes, validation
 mtplx inspect <model>      # compatibility report before anything runs
 mtplx tune --retune        # measure AR vs D1/D2/D3 on your Mac
-mtplx forge                # build, verify, and publish MTP models
+mtplx forge --help         # build, verify, and publish MTP models (probe/build/publish/verify subcommands)
 mtplx bench aime --quick   # run the AIME benchmark from the terminal
 mtplx doctor               # install and integration health
 mtplx max --install        # fan control (one sudo prompt, crash-safe)
 mtplx settings get/set     # read or change live server settings
 ```
 
-Every command takes `--json` and `--help`. The CLI works without MLX installed for everything that does not need a model, so `doctor` and `inspect` run on any machine.
+Every command takes `--help`, and most inspection/diagnostic commands take `--json`. The CLI works without MLX installed for everything that does not need a model, so `doctor` and `inspect` run on any machine.
 
 ## Modes
 
 | Mode | What it does | When |
 |---|---|---|
-| **Sustained** | Default. Long-context MTP path with chunked prefill and request-sized KV | Everyday use, big files, 16K-200K prompts |
+| **Turbo** | NAX verify kernels + compiled verify; the default for the quantized 27B and 9B flagship models | Picked automatically for those models |
+| **Sustained** | Default for all other models. Long-context MTP path with chunked prefill and request-sized KV | Everyday use, big files, 16K-200K prompts |
 | **Sustained Max** | Sustained with fans pinned at 100% | Long work where you want maximum cooling |
 | **Burst** | Legacy short-context benchmark lane, loud | Short prompts and benchmarks only |
 
@@ -104,7 +107,7 @@ Fan-backed modes restore your fans to automatic if MTPLX dies for any reason, in
 
 ## Compatibility, honestly
 
-`mtplx inspect` classifies models before anything runs: verified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+`mtplx inspect` classifies models before anything runs: verified, family-compatible but unverified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models load with an explicit unverified label. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
 
 [Laguna-S-2.1 oQ4e](https://huggingface.co/mlx-community/Laguna-S-2.1-oQ4e) is supported through its exact MLX architecture in target-only AR mode:
 
@@ -120,8 +123,10 @@ MTPLX pins that model to revision
 tokenizer, generation config, special tokens map, and Poolside chat template
 before admitting it. The checkpoint has no native MTP head, so an MTP launch is
 rejected before weights load instead of falling back during execution. The
-weights occupy 59.72 GiB (64.13 GB); use a Mac with at least 96 GiB unified
-memory (128 GiB is recommended). MTPLX defaults Laguna to a 32,768-token context
+weights occupy 59.72 GiB, a 64.13 GB snapshot on disk. The launch preflight
+requires about 85 GiB of unified memory (weights, runtime headroom, and a
+16 GiB system reserve) — in practice a 96 GB Mac; 128 GB is
+comfortable. MTPLX defaults Laguna to a 32,768-token context
 and response cap, and checks larger explicit server contexts against the active
 Metal memory cap.
 

{mtplx-2.4.0 → mtplx-2.4.2}/mtplx/adaptive.py

@@ -250,3 +250,254 @@ class ExpectedValueDepthPolicy:
             prob_term = 2.0 * _clamp(float(top1_prob), 0.0, 1.0) - 1.0
         raw = 1.0 + self.confidence_weight * (0.75 * margin_term + 0.25 * prob_term)
         return _clamp(raw, 0.25, 1.75)
+
+
+class CostModelDepthPolicy:
+    """Cost-model adaptive depth: maximize expected committed tokens per
+    wall-clock cycle, not acceptance streaks.
+
+    Ported from omlx's ``_DepthController`` (jundot/omlx v0.5.4rc1,
+    omlx/patches/mlx_lm_mtp/batch_generator.py) with the depth-0
+    park/exit machinery deliberately left out for now (our 27B lanes
+    always profit from speculation; the escape hatch matters for
+    head_dim-512/MoE models and can come later). Their measured design
+    decisions preserved verbatim:
+
+    - ``score(d) = (1 + p1 + p1 p2 + ...) / t_est(d)``.
+    - Acceptance is a token-domain EMA (a property of model/content);
+      cost is a wall-clock-horizon EMA (tracks context growth, thermal
+      state, and external GPU load at constant real-time responsiveness)
+      with a one-off-spike damp.
+    - The marginal cost of an extra verify row is the measured slope
+      between the cheapest and priciest measured depths, not a constant.
+    - Probes are bidirectional and staleness-directed, duty-bounded to
+      ~15% of cycles: re-measuring a SHALLOWER rival is what breaks the
+      depth-2 lock omlx measured (stale-high t[1] hides depth 1 forever).
+    - The cost EMA is per-cycle, NOT staleness-age-weighted: omlx
+      measured age-weighting worse (probe-burst noise injected straight
+      into the decision; prose re-over-drafted 1.6%).
+
+    Drop-in for the ``AdaptiveDepthPolicy`` interface: ``current_depth``
+    plus ``observe(attempted_depth=, accepted_depths=)``. Cycle cost is
+    self-timed as the wall interval between observe calls (one observe
+    per verify cycle), which deliberately includes the loop's host
+    bookkeeping — that tax is part of the real cost of running a cycle.
+    """
+
+    ALPHA = 0.08
+    TAU_MS = 400.0
+    PROBE_PERIOD_MS = 1000.0
+    PROBE_PERIOD_MAX_MS = 5000.0
+    PROBE_LEN = 4
+    PROBE_DUTY = 0.15
+    PROBE_MARGIN = 1.15
+    SPIKE_RATIO = 2.0
+    SPIKE_DAMP = 0.25
+    MARGINAL_MS = 7.0
+    HYSTERESIS = 1.03
+    # Ignore absurd inter-observe gaps (queue waits, tool round-trips in
+    # agent serving): a "cycle" above this is not a cycle measurement.
+    MAX_CYCLE_MS = 5000.0
+    # The generation loop passes its own measured cycle wall-time when it
+    # sees this flag; the self-timed inter-observe fallback stays for
+    # callers that do not.
+    accepts_cycle_ms = True
+
+    def __init__(
+        self,
+        max_depth: int,
+        min_depth: int = 1,
+        marginal_ms: float | None = None,
+    ) -> None:
+        if max_depth < 1:
+            raise ValueError("max_depth must be >= 1")
+        self.max_depth = int(max_depth)
+        self.min_depth = max(1, min(int(min_depth), self.max_depth))
+        if marginal_ms:
+            self.MARGINAL_MS = float(marginal_ms)
+        self.current_depth = self.max_depth
+        self.p = [0.6] * self.max_depth
+        self.t: dict[int, float] = {}
+        self.t_age: dict[int, float] = {}
+        self.cycles = 0
+        self.probe_left = 0
+        self._ms_probe = 0.0
+        self._ms_explore = 0.0
+        self._warmup = list(range(self.max_depth, self.min_depth - 1, -1))
+        self._last_observe_s: float | None = None
+
+    # -- cost bookkeeping --------------------------------------------------
+
+    def _time_alpha(self, cycle_ms: float) -> float:
+        return 1.0 - math.exp(-max(0.0, float(cycle_ms)) / self.TAU_MS)
+
+    def _update_time(self, used: int, cycle_ms: float) -> None:
+        prev = self.t.get(used)
+        if prev is None:
+            self.t[used] = cycle_ms
+            return
+        if self._warmup:
+            self.t[used] = min(prev, cycle_ms)
+            return
+        a = self._time_alpha(cycle_ms)
+        if cycle_ms > self.SPIKE_RATIO * prev:
+            a *= self.SPIKE_DAMP
+        self.t[used] = (1.0 - a) * prev + a * cycle_ms
+
+    def _marginal_est(self) -> float:
+        if len(self.t) >= 2:
+            depths = sorted(self.t)
+            lo, hi = depths[0], depths[-1]
+            if hi > lo:
+                slope = (self.t[hi] - self.t[lo]) / (hi - lo)
+                if slope > 0.0:
+                    return slope
+        return self.MARGINAL_MS
+
+    def _t_est(self, d: int) -> float:
+        if d in self.t:
+            return self.t[d]
+        if not self.t:
+            return 30.0 + self.MARGINAL_MS * d
+        ref = min(self.t, key=lambda x: abs(x - d))
+        return max(1e-3, self.t[ref] + self._marginal_est() * (d - ref))
+
+    def _score(self, d: int) -> float:
+        expected = 1.0
+        run = 1.0
+        for j in range(d):
+            run *= self.p[j]
+            expected += run
+        return expected / max(1e-6, self._t_est(d))
+
+    # -- selection ---------------------------------------------------------
+
+    def _depths(self) -> list[int]:
+        return list(range(self.min_depth, self.max_depth + 1))
+
+    def _best(self) -> int:
+        cur_score = self._score(self.current_depth)
+        best_d, best_score = self.current_depth, cur_score
+        for d in self._depths():
+            s = self._score(d)
+            if s > best_score:
+                best_d, best_score = d, s
+        if best_d != self.current_depth and best_score < cur_score * self.HYSTERESIS:
+            return self.current_depth
+        return best_d
+
+    def _best_rival(self) -> int | None:
+        best = self._score(self.current_depth)
+        if best <= 0.0:
+            return self._most_stale()
+        rival, rival_score = None, 0.0
+        for d in self._depths():
+            if d == self.current_depth:
+                continue
+            s = self._score(d)
+            if s > rival_score:
+                rival, rival_score = d, s
+        if rival is not None and rival_score * self.PROBE_MARGIN >= best:
+            return rival
+        return None
+
+    def _most_stale(self) -> int | None:
+        candidates = [d for d in self._depths() if d != self.current_depth]
+        if not candidates:
+            return None
+        never = [d for d in candidates if d not in self.t]
+        if never:
+            return never[0]
+        return max(candidates, key=lambda d: self.t_age.get(d, 0.0))
+
+    # -- the drop-in interface ---------------------------------------------
+
+    def observe(
+        self,
+        *,
+        attempted_depth: int,
+        accepted_depths: int,
+        cycle_ms: float | None = None,
+    ) -> dict:
+        import time as _time
+
+        now = _time.perf_counter()
+        if cycle_ms is None and self._last_observe_s is not None:
+            cycle_ms = (now - self._last_observe_s) * 1000.0
+        if cycle_ms is not None and (cycle_ms > self.MAX_CYCLE_MS or cycle_ms <= 0.0):
+            cycle_ms = None
+        self._last_observe_s = now
+
+        self.cycles += 1
+        used = max(1, min(int(attempted_depth), self.max_depth))
+        accepted = max(0, min(int(accepted_depths), used))
+        previous_depth = self.current_depth
+
+        a = self.ALPHA
+        for j in range(used):
+            hit = 1.0 if j < accepted else 0.0
+            self.p[j] = (1.0 - a) * self.p[j] + a * hit
+            if j >= accepted:
+                break
+
+        if cycle_ms is not None:
+            self._update_time(used, cycle_ms)
+            for d in list(self.t_age):
+                self.t_age[d] += cycle_ms
+            self.t_age[used] = 0.0
+            self._ms_probe += cycle_ms
+            self._ms_explore += cycle_ms
+
+        action = "hold"
+        if self._warmup:
+            # A warmup slot is consumed only once its depth has a real cost
+            # sample; the very first observe has no prior timestamp to diff
+            # against, so that cycle repeats its depth instead of advancing.
+            if cycle_ms is not None and used == self._warmup[0]:
+                self._warmup.pop(0)
+            if self._warmup:
+                self.current_depth = self._warmup[0]
+                action = "warmup"
+            else:
+                self.current_depth = self._best()
+                self._ms_probe = 0.0
+                action = "warmup_done"
+        elif self.probe_left > 0:
+            self.probe_left -= 1
+            if self.probe_left == 0:
+                self.current_depth = self._best()
+                self._ms_probe = 0.0
+                action = "probe_done"
+            else:
+                action = "probing"
+        else:
+            self.current_depth = self._best()
+            if self.current_depth != previous_depth:
+                action = (
+                    "increase" if self.current_depth > previous_depth else "decrease"
+                )
+            if self.max_depth > self.min_depth and cycle_ms is not None:
+                period = max(
+                    self.PROBE_PERIOD_MS,
+                    self.PROBE_LEN * cycle_ms / self.PROBE_DUTY,
+                )
+                if self._ms_probe >= period:
+                    explore_due = self._ms_explore >= max(
+                        self.PROBE_PERIOD_MAX_MS, 2.0 * period
+                    )
+                    target = self._most_stale() if explore_due else self._best_rival()
+                    if target is not None:
+                        self.current_depth = target
+                        self.probe_left = self.PROBE_LEN
+                        self._ms_probe = 0.0
+                        if explore_due:
+                            self._ms_explore = 0.0
+                        action = "probe"
+
+        return {
+            "previous_depth": previous_depth,
+            "attempted_depth": used,
+            "accepted_depths": accepted,
+            "next_depth": self.current_depth,
+            "action": action,
+        }

{mtplx-2.4.0 → mtplx-2.4.2}/mtplx/artifacts.py

@@ -313,6 +313,62 @@ def expected_mtp_file(model_dir: Path | str, config: dict[str, Any] | None = Non
     return model_path / "mtp.safetensors"
 
 
+def mtp_weights_present_on_disk(
+    model_dir: Path | str, config: dict[str, Any] | None = None
+) -> bool:
+    """Whether a model that declares MTP layers actually ships MTP weights.
+
+    A conversion can declare ``num_nextn_predict_layers`` in the config while
+    dropping the MTP weights themselves (e.g. the DeepSeek-V4-Flash 2bit-DQ
+    build). The runtime uses this probe to tell that benign case (config field
+    only -> degrade to autoregressive) apart from a genuine injection failure
+    (weights present but unusable -> raise).
+
+    Conservative by design: it only returns ``False`` when it can *positively*
+    confirm absence via a shard index that carries no MTP-shaped keys under any
+    known naming convention. A sidecar file, a missing/unreadable index, or any
+    ambiguity returns ``True`` so the existing injection + validation path runs
+    unchanged and a real detection bug on an MTP-bearing model still surfaces.
+    """
+    model_path = Path(model_dir)
+    config = config if config is not None else load_config(model_path)
+
+    # 1. Explicit MTP sidecar file (Qwen/GLM/hy3 external draft head).
+    if expected_mtp_file(model_path, config).exists():
+        return True
+
+    index_path = model_path / "model.safetensors.index.json"
+    if not index_path.exists():
+        # No index to inspect: cannot prove absence, preserve legacy behavior.
+        return True
+    try:
+        weight_map = json.loads(index_path.read_text(encoding="utf-8")).get(
+            "weight_map", {}
+        )
+    except Exception:
+        return True
+    keys = [str(k) for k in weight_map]
+
+    # 2. Namespaced embedded MTP weights ("mtp.*" / "language_model.mtp.*").
+    if any(is_mtp_key(k) for k in keys):
+        return True
+
+    # 3. DeepSeek-style trailing MTP decoder layer(s) appended after the trunk:
+    #    model.layers.{num_hidden_layers + i}.*
+    start = int(
+        text_config(config).get("num_hidden_layers")
+        or config.get("num_hidden_layers")
+        or 0
+    )
+    count = _num_mtp_layers(config)
+    if start and count:
+        wanted = tuple(f"model.layers.{start + i}." for i in range(count))
+        if any(k.startswith(wanted) for k in keys):
+            return True
+
+    return False
+
+
 @dataclass(frozen=True)
 class TensorInfo:
     key: str