mtplx 2.4.2__tar.gz → 2.5.0__tar.gz

{mtplx-2.4.2 → mtplx-2.5.0}/CHANGELOG.md

@@ -4,6 +4,66 @@ All notable user-facing changes to MTPLX. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [2.5.0] - 2026-08-03
+
+The next-model release: MTPLX can load new architecture aliases without a
+hard-coded model-type wait, multi-layer MTP drafts are supported for the
+upcoming Qwen generation, HY3 becomes a first-class serving target, and the
+coding-agent bridge is materially more reliable. Experimental DeepSeek V4,
+Laguna, and GDN speed lanes from David Tai land behind explicit opt-ins; the
+shipping defaults remain unchanged.
+
+### Added
+
+- Upcoming-Qwen architecture readiness: the native draft path now supports
+  `mtp_num_hidden_layers = N`, and unknown `model_type` values can resolve
+  through their declared architecture class. A synthetic `qwen3_8` alias
+  drill exercised load, generate, CLI, and app discovery before public weights
+  existed; this is compatibility preparation, not a claim that unreleased
+  weights were benchmarked.
+- HY3 first-class serving: a vendored MTP-capable model class, official
+  defaults, suffixed think-tag handling, model discovery, and native OpenCode
+  tool calls. AR-only exports fail safely to the AR path rather than touching
+  an uninitialized draft head.
+- Experimental DeepSeek-V4 shape-specialized verification lanes: prebound
+  output-LoRA routes, adaptive speculative width, exact M3 attention
+  projection, sinkhorn and attention-island kernels, and compiled
+  post-attention verifier islands. On the 128 GB M5 Max release machine, the
+  exact 2-bit DQ model plus official MTP shard measured about 31 AR tok/s and
+  36 MTP tok/s versus about 4/6 tok/s on the conservative path. These routes
+  remain opt-in while broader model-quality validation continues. Contributed
+  by David Tai (@davidtai, #223).
+- Experimental Laguna S-2.1 `mlx.fast` ports covering decode and prefill
+  kernels, including size-gated prefill MoE combine; fail-loud guards and
+  portable scratch space keep unsupported shapes on the safe route.
+  Contributed by David Tai (@davidtai, #222).
+- Experimental GDN headquarter execution layout for verify tape capture,
+  env-gated with bit-exact coverage and loud fallback. Contributed by David
+  Tai (@davidtai, #209).
+
+### Fixed
+
+- Coding-agent JSON tool calls can carry large write bodies without the hidden
+  tool guard aborting them, and common near-miss argument keys are repaired at
+  the protocol boundary. This was verified through real OpenCode CLI and
+  Desktop sessions, including a multi-file edit with all generated tests
+  passing.
+- App and CLI launches now share the same coding-agent engine environment, so
+  a workflow does not silently change behavior depending on which Start button
+  launched it.
+- `start` and `quickstart --dry-run` report the resolved profile instead of the
+  parser's placeholder default.
+- Smart-fan mode holds through the post-generation heat-soak window before
+  restoring automatic control, avoiding the early restore that could distort
+  back-to-back performance runs (#227).
+
+### Compatibility
+
+- `transformers` 5.14 is allowed after tokenizer and tool-template parity were
+  verified; the incompatible 5.13.0 release remains excluded (#175).
+- No speculative-depth, cache, sampler, or model-speed default changed in this
+  release. The opt-in model kernels fail closed to established implementations.
+
 ## [2.4.2] - 2026-08-02
 
 The agentic-cache release: the session cache stops losing warm state
@@ -827,6 +887,7 @@ working as one product. Full notes:
   completions, and Anthropic `stop_sequences`) and `/v1/completions`
   streams tokens as they are generated with real finish reasons.
 
+[2.5.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.5.0
 [2.4.2]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.2
 [2.4.1]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.1
 [2.4.0]: https://github.com/youssofal/MTPLX/releases/tag/v2.4.0

{mtplx-2.4.2 → mtplx-2.5.0}/CITATION.cff

@@ -8,7 +8,7 @@ authors:
 repository-code: "https://github.com/youssofal/mtplx"
 url: "https://github.com/youssofal/mtplx"
 license: Apache-2.0
-version: 2.4.2
+version: 2.5.0
 abstract: "Native MTP speculative decoding for Qwen3-Next on Apple Silicon, using built-in MTP heads with math-correct rejection sampling and an OpenAI/Anthropic-compatible serving surface."
 keywords:
   - speculative decoding

{mtplx-2.4.2 → mtplx-2.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mtplx
-Version: 2.4.2
+Version: 2.5.0
 Summary: Native MTP speculative decoding for Qwen3-Next on Apple Silicon.
 Author-email: Youssof Altoukhi <business@youssofal.com>
 License-Expression: Apache-2.0
@@ -28,7 +28,7 @@ Requires-Dist: huggingface-hub>=0.36
 Requires-Dist: pillow>=10
 Requires-Dist: mlx<0.33,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
 Requires-Dist: mlx-lm<0.32,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
-Requires-Dist: transformers!=5.13.0,<5.14; sys_platform == "darwin" and platform_machine == "arm64"
+Requires-Dist: transformers!=5.13.0,<5.15; sys_platform == "darwin" and platform_machine == "arm64"
 Requires-Dist: nanobind>=2; sys_platform == "darwin" and platform_machine == "arm64"
 Requires-Dist: numpy>=2
 Requires-Dist: pydantic>=2

{mtplx-2.4.2 → mtplx-2.5.0}/mtplx/artifacts.py

@@ -12,18 +12,14 @@ from typing import Any
 
 from .constants import (
     EXPECTED_ALL_PREQUANTIZED_MTP_KEYS,
-    EXPECTED_ALL_PREQUANTIZED_MTP_TENSOR_COUNT,
     EXPECTED_MTP_KEYS,
-    EXPECTED_PREQUANTIZED_MTP_KEYS,
-    EXPECTED_PREQUANTIZED_MTP_TENSOR_COUNT,
     EXPECTED_MTP_TENSOR_COUNT,
+    EXPECTED_PREQUANTIZED_MTP_KEYS,
     EXPECTED_QWEN_MOE_MTP_KEYS,
-    EXPECTED_QWEN_MOE_MTP_TENSOR_COUNT,
     EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS,
-    EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_TENSOR_COUNT,
     EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_KEYS,
-    EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_TENSOR_COUNT,
     MULTIMODAL_SIDECARS,
+    expand_mtp_layer_keys,
 )
 from .models.laguna_config import (
     LAGUNA_S_2_1_REPO_ID,
@@ -194,6 +190,13 @@ def _mtp_expected_key_set(
     prequantized = isinstance(mtp_quant, dict) and bool(mtp_quant.get("prequantized"))
     quant_policy = str(mtp_quant.get("policy") or "") if isinstance(mtp_quant, dict) else ""
     normalized = {normalize_mtp_key(key) for key in keys}
+    # Every named key set below is the canonical depth-1 template; checkpoints
+    # declaring mtp_num_hidden_layers > 1 replicate the layer keys per index.
+    n_layers = max(_num_mtp_layers(config), 1)
+
+    def _expanded(base: tuple[str, ...]) -> set[str]:
+        return expand_mtp_layer_keys(base, n_layers)
+
     if _is_qwen_moe_mtp_layout(config, normalized):
         if any(".mlp.switch_mlp." in key for key in normalized):
             has_prequantized_aux = any(
@@ -202,7 +205,7 @@ def _mtp_expected_key_set(
             )
             if prequantized or has_prequantized_aux:
                 expected = _expected_prequantized_keys_for_present_aux(
-                    set(EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_KEYS),
+                    _expanded(EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_KEYS),
                     normalized,
                 )
                 return (
@@ -210,9 +213,10 @@ def _mtp_expected_key_set(
                     len(expected),
                     "prequantized-mlx-affine-qwen-moe-switch-mlx",
                 )
+            expected = _expanded(EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_KEYS)
             return (
-                set(EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_KEYS),
-                EXPECTED_QWEN_MOE_SWITCH_MLP_MTP_TENSOR_COUNT,
+                expected,
+                len(expected),
                 "bf16-qwen-moe-switch-mlx",
             )
         if _has_numbered_moe_experts(normalized):
@@ -224,42 +228,47 @@ def _mtp_expected_key_set(
                 config,
                 prequantized=prequantized or has_prequantized_aux,
             )
-        if prequantized or normalized == set(EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS):
+        if prequantized or normalized == _expanded(EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS):
+            expected = _expanded(EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS)
             return (
-                set(EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS),
-                EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_TENSOR_COUNT,
+                expected,
+                len(expected),
                 "prequantized-mlx-affine-qwen-moe",
             )
+        expected = _expanded(EXPECTED_QWEN_MOE_MTP_KEYS)
         return (
-            set(EXPECTED_QWEN_MOE_MTP_KEYS),
-            EXPECTED_QWEN_MOE_MTP_TENSOR_COUNT,
+            expected,
+            len(expected),
             "bf16-qwen-moe",
         )
     if prequantized and quant_policy == "all":
+        expected = _expanded(EXPECTED_ALL_PREQUANTIZED_MTP_KEYS)
         return (
-            set(EXPECTED_ALL_PREQUANTIZED_MTP_KEYS),
-            EXPECTED_ALL_PREQUANTIZED_MTP_TENSOR_COUNT,
+            expected,
+            len(expected),
             "prequantized-mlx-affine",
         )
     if prequantized:
+        expected = _expanded(EXPECTED_PREQUANTIZED_MTP_KEYS)
         return (
-            set(EXPECTED_PREQUANTIZED_MTP_KEYS),
-            EXPECTED_PREQUANTIZED_MTP_TENSOR_COUNT,
+            expected,
+            len(expected),
             "prequantized-mlx-affine",
         )
-    if normalized == set(EXPECTED_ALL_PREQUANTIZED_MTP_KEYS):
+    if normalized == _expanded(EXPECTED_ALL_PREQUANTIZED_MTP_KEYS):
         return (
-            set(EXPECTED_ALL_PREQUANTIZED_MTP_KEYS),
-            EXPECTED_ALL_PREQUANTIZED_MTP_TENSOR_COUNT,
+            set(normalized),
+            len(normalized),
             "prequantized-mlx-affine",
         )
-    if normalized == set(EXPECTED_PREQUANTIZED_MTP_KEYS):
+    if normalized == _expanded(EXPECTED_PREQUANTIZED_MTP_KEYS):
         return (
-            set(EXPECTED_PREQUANTIZED_MTP_KEYS),
-            EXPECTED_PREQUANTIZED_MTP_TENSOR_COUNT,
+            set(normalized),
+            len(normalized),
             "prequantized-mlx-affine",
         )
-    return set(EXPECTED_MTP_KEYS), EXPECTED_MTP_TENSOR_COUNT, "bf16"
+    expected = _expanded(EXPECTED_MTP_KEYS)
+    return expected, len(expected), "bf16"
 
 
 def _observed_sidecar_format(sidecar_format: str, tensors: tuple[TensorInfo, ...]) -> str:

{mtplx-2.4.2 → mtplx-2.5.0}/mtplx/attention_context.py

@@ -13,11 +13,20 @@ VALID_ATTENTION_PHASES = {
     "postcommit",
     "unknown",
 }
+VALID_MODEL_FORWARD_KINDS = {
+    "target_verify",
+    "repair",
+    "other",
+}
 
 _ATTENTION_PHASE: ContextVar[str] = ContextVar(
     "mtplx_attention_phase",
     default="unknown",
 )
+_MODEL_FORWARD_KIND: ContextVar[str] = ContextVar(
+    "mtplx_model_forward_kind",
+    default="other",
+)
 
 
 def normalize_attention_phase(phase: str | None) -> str:
@@ -29,6 +38,15 @@ def current_attention_phase() -> str:
     return normalize_attention_phase(_ATTENTION_PHASE.get())
 
 
+def normalize_model_forward_kind(kind: str | None) -> str:
+    value = (kind or "other").strip().lower()
+    return value if value in VALID_MODEL_FORWARD_KINDS else "other"
+
+
+def current_model_forward_kind() -> str:
+    return normalize_model_forward_kind(_MODEL_FORWARD_KIND.get())
+
+
 @contextmanager
 def attention_phase(phase: str | None) -> Iterator[None]:
     token = _ATTENTION_PHASE.set(normalize_attention_phase(phase))
@@ -36,3 +54,14 @@ def attention_phase(phase: str | None) -> Iterator[None]:
         yield
     finally:
         _ATTENTION_PHASE.reset(token)
+
+
+@contextmanager
+def model_forward_kind(kind: str | None) -> Iterator[None]:
+    """Identify whether one decode-verify-phase target call verifies or repairs."""
+
+    token = _MODEL_FORWARD_KIND.set(normalize_model_forward_kind(kind))
+    try:
+        yield
+    finally:
+        _MODEL_FORWARD_KIND.reset(token)

{mtplx-2.4.2 → mtplx-2.5.0}/mtplx/backends/descriptors.py

@@ -570,6 +570,48 @@ GLM_MTP_DESCRIPTOR = BackendDescriptor(
 )
 
 
+HY_V3_MTP_DESCRIPTOR = BackendDescriptor(
+    backend_id="hy_v3_mtp",
+    architecture_id="hy-v3-mtp",
+    model_family="hy",
+    display_name="Hy3 native MTP",
+    artifact_layout="single_mlx_folder_native_mtp",
+    runtime_capabilities=NATIVE_CONTRACT_DESCRIPTOR.runtime_capabilities,
+    # Official Tencent inference settings (tencent/Hy3 generation_config.json):
+    # temperature 0.9, top_p 1.0, top_k disabled. Do NOT inherit the Qwen
+    # 0.6/0.95/20 coding sampler; Hy3's CoT measurably degrades at greedy/cold
+    # settings (community reports on the release thread).
+    sampler_defaults=SamplerDefaults(temperature=0.9, top_p=1.0, top_k=0),
+    # Hy3 emits <think:opensource>...</think:opensource> (suffixed single
+    # tokens). The qwen3-style splitter handles suffixed spellings.
+    reasoning_codec=ReasoningCodec(
+        parser="qwen3",
+        display_name="Hy3 think tags",
+        default_mode="auto",
+    ),
+    draft_semantics=NATIVE_CONTRACT_DESCRIPTOR.draft_semantics,
+    uses_external_assistant=False,
+    uses_draft_lm_head=True,
+    hidden_variant="pre_norm",
+    tune_policy=TunePolicy(
+        supported=False,
+        unsupported_reason="Tune is supported for Qwen 3.5, Qwen 3.6, and Gemma 4 MTPLX models only.",
+    ),
+    kv_quant_policy=KVQuantPolicy(
+        supported=False,
+        disabled_reason="KV quantization is not supported for Hy3.",
+    ),
+    validation_status="experimental_contract_gated",
+    status="experimental_contract_gated",
+    notes=(
+        "Single appended NextN MoE layer (depth 1); 192-expert sigmoid top-8 "
+        "routing with expert bias; draft input eh_proj(concat[enorm(embedding), "
+        "hnorm(pre-final-norm hidden)]).",
+        "Official sampler: temperature 0.9, top_p 1.0, top_k off.",
+    ),
+)
+
+
 GEMMA4_TARGET_DISTRIBUTION_POLICY = TargetDistributionPolicy(
     modes=("gemma4_target_prefix_exact",),
     default_mode="gemma4_target_prefix_exact",
@@ -695,6 +737,7 @@ DESCRIPTORS_BY_BACKEND_ID: dict[str, BackendDescriptor] = {
     STEP3P5_MTP_DESCRIPTOR.backend_id: STEP3P5_MTP_DESCRIPTOR,
     DEEPSEEK_MTP_DESCRIPTOR.backend_id: DEEPSEEK_MTP_DESCRIPTOR,
     GLM_MTP_DESCRIPTOR.backend_id: GLM_MTP_DESCRIPTOR,
+    HY_V3_MTP_DESCRIPTOR.backend_id: HY_V3_MTP_DESCRIPTOR,
     "mimo_mtp": NATIVE_CONTRACT_DESCRIPTOR,
     "nemotron_h_mtp": NATIVE_CONTRACT_DESCRIPTOR,
 }

{mtplx-2.4.2 → mtplx-2.5.0}/mtplx/commands/public.py

@@ -307,6 +307,14 @@ def _opencode_memory_env_defaults() -> dict[str, str]:
         else _OPENCODE_DEFAULT_MAX_ENTRIES
     )
     return {
+        # Long-context decode route (>=32k): same keys the app's
+        # codingAgentRuntimeEnvironment and the CLI hermes lane already set —
+        # the OpenCode CLI lane missing them was surface drift (2026-08-03
+        # parity audit), not intent.
+        "MTPLX_VLLM_METAL_PAGED_GQA_SDPA_ROUTE": "async_per_head",
+        "MTPLX_VLLM_METAL_PAGED_GQA_SDPA_MIN_CONTEXT": "32768",
+        "MTPLX_VLLM_METAL_PAGED_GQA_SDPA_MIN_Q": "3",
+        "MTPLX_VLLM_METAL_PAGED_GQA_SDPA_MAX_Q": "5",
         "MTPLX_SESSION_BLOCK_PREFIX_RESTORE": "1",
         "MTPLX_SESSION_BANK_MAX_ENTRIES": max_entries,
         # "auto" = the engine budgets half the RAM surplus left after the
@@ -997,7 +1005,13 @@ def _resolved_default_profile_name(args: Any, model: str | None = None) -> str:
 
 
 def _apply_qwen36_35b_optimized_speed_defaults(args: Any, model_id: str) -> None:
-    if model_id != QWEN36_35B_OPTIMIZED_SPEED_PUBLIC_MODEL_ID:
+    # The -FP16 sibling shares the byte-identical INT packs and the measured
+    # launch defaults; the app's substring detection already applied them to
+    # it while this exact-id gate skipped it (2026-08-03 parity audit).
+    if model_id not in {
+        QWEN36_35B_OPTIMIZED_SPEED_PUBLIC_MODEL_ID,
+        QWEN36_35B_OPTIMIZED_SPEED_FP16_PUBLIC_MODEL_ID,
+    }:
         return
     cli_flags = getattr(args, "_cli_flags", set()) or set()
     injected = set(getattr(args, "_injected_default_flags", set()) or set())
@@ -1359,12 +1373,19 @@ def _pi_preserve_thinking_policy(args: Any) -> str:
 
 
 def _apply_pi_history_budget_env_defaults(env: dict[str, str]) -> None:
+    """Pi lane = the shared coding-agent engine block + Pi history budgets.
+
+    Mirrors the app's composition exactly (codingAgentRuntimeEnvironment then
+    the Pi-specific overrides in MTPLXCommandBuilder.swift). Before the
+    2026-08-03 parity audit the CLI Pi lane carried only the history keys —
+    no session bank, SDPA route, postcommit wait, or frontier flags — and
+    three of its values (96/16/150) diverged from the app-lane numbers
+    (72/8/120) every app Pi user already runs; unified to the app values.
+    """
+    _apply_opencode_memory_env_defaults(env)
     env.setdefault("MTPLX_TOOL_RESULT_COMPACT_THRESHOLD_CHARS", "1200")
     env.setdefault("MTPLX_ACTIVE_READ_INSPECTION_COMPACT_MAX_LINES", "32")
     env.setdefault("MTPLX_ACTIVE_READ_INSPECTION_LINE_MAX_CHARS", "180")
-    env.setdefault("MTPLX_ACTIVE_READ_INSPECTION_TOTAL_MAX_LINES", "96")
-    env.setdefault("MTPLX_ACTIVE_READ_INSPECTION_MIN_LINES_PER_FILE", "16")
-    env.setdefault("MTPLX_ACTIVE_READ_INSPECTION_MULTI_FILE_LINE_MAX_CHARS", "150")
     env.setdefault("MTPLX_ACTIVE_TOOL_RESULT_COMPACT_MAX_LINES", "32")
     env.setdefault("MTPLX_ACTIVE_TOOL_RESULT_LINE_MAX_CHARS", "220")
 
@@ -12584,7 +12605,11 @@ def cmd_quickstart_public(args: Any) -> int:
             "target": target,
             "model": model,
             "cache_dir": cache_dir,
-            "profile": getattr(args, "profile", DEFAULT_PROFILE_NAME),
+            # Display the profile the launch will actually resolve (per-model
+            # turbo rewrite included) — the raw parser default here made the
+            # dry-run advertise "sustained" for the turbo-default flagships
+            # (the 2026-07-16 stale-display bug class, on one more surface).
+            "profile": _resolved_default_profile_name(args, model),
             "generation_mode": _generation_mode_from_args(args),
             "max": bool(getattr(args, "max", False)),
             "download_if_missing": download,

{mtplx-2.4.2 → mtplx-2.5.0}/mtplx/constants.py

@@ -184,6 +184,29 @@ EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_TENSOR_COUNT = len(
     EXPECTED_QWEN_MOE_PREQUANTIZED_MTP_KEYS
 )
 
+_MTP_LAYER_KEY_MARKER = "mtp.layers.0."
+
+
+def expand_mtp_layer_keys(keys: tuple[str, ...] | set[str], n_layers: int) -> set[str]:
+    """Expand a depth-1 MTP key template across ``n_layers`` draft layers.
+
+    Every expected-key set in this module describes the canonical
+    single-layer (``mtp.layers.0.*``) head. Upstream checkpoints may declare
+    ``mtp_num_hidden_layers > 1`` (the config key is N-generic in the vLLM
+    reference contract); their weight layout replicates the per-layer
+    template at each index. Identity for ``n_layers <= 1``.
+    """
+    n = max(int(n_layers), 1)
+    expanded: set[str] = set()
+    for key in keys:
+        if _MTP_LAYER_KEY_MARKER in key:
+            for index in range(n):
+                expanded.add(key.replace(_MTP_LAYER_KEY_MARKER, f"mtp.layers.{index}.", 1))
+        else:
+            expanded.add(key)
+    return expanded
+
+
 MULTIMODAL_SIDECARS = (
     "preprocessor_config.json",
     "processor_config.json",