mtplx 2.2.0__tar.gz → 2.4.0__tar.gz

{mtplx-2.2.0 → mtplx-2.4.0}/CHANGELOG.md

@@ -4,6 +4,195 @@ All notable user-facing changes to MTPLX. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [2.4.0] - 2026-07-31
+
+The 35B speed release: the 35B-A3B MoE gets a compiled decode stack and
+continuous batched serving, the 2.3.0 fan regression is root-caused and
+fixed, tool calling gets another round of contract hardening, and
+structured output can no longer be eaten by an unbounded reasoning
+prelude. Four community contributors landed code in this release.
+
+### Added
+
+- 35B-A3B compiled decode stack: target-prefix compiled route, whole-MoE
+  fusion, GDN post-conv fusion, and a row-owned router (David Tai, #174).
+- Continuous batched serving for the A3B lane: fixed-shape cohorts,
+  ragged KV, fold-in repair, and AR row-packing (David Tai, #200).
+- Laguna S-2.1 support (exact-pin oQ4e, AR-only) with an app catalog
+  entry, plus a Poolside `arg_key`/`arg_value` tool-call dialect parser
+  (David Tai, #195).
+- Hy3 295B full-residency lane and generic MTP draft-contract hardening
+  whose loud recurrent-cache failure also caught a real bug on the Qwen
+  lane (David Tai, #208).
+- `/health` now reports smart-fan restore state (`restore_verified`,
+  `restore_failures`, `stale_leases_reconciled`) so stuck-fan reports
+  are diagnosable from the field (#201).
+
+### Fixed
+
+- Fans no longer stay pinned at max after a request ends (#201). A
+  failed fan restore was logged once and then treated as restored, so
+  the hardware stayed ramped while the server believed it was clean;
+  restores now verify the fan rows are back on the Apple auto curve and
+  retry with backoff until they are. The ThermalForge daemon-socket
+  restore path no longer trusts the daemon's "ok" reply without
+  verifying, and falls back to the CLI in the same call. A stale-lease
+  watchdog drops any fan lease held while the engine has been
+  continuously idle (default 120s, `MTPLX_SMART_FAN_STALE_LEASE_S`).
+- A generation cut by `max_tokens` mid-tool-call now reports
+  `finish_reason: "length"` instead of `"tool_calls"`, so agent clients
+  continue the turn instead of executing a truncated call (#196, #197
+  layers one and two).
+- The think-splitter no longer leaks reasoning into visible content when
+  the text contains bare `function=` or `parameter=` strings (#196/#197
+  companion fix).
+- Streaming tool-call parsing handles bracket-style dialects with a
+  balanced string/escape-aware scanner, buffers incomplete calls instead
+  of double-delivering them, and passes through calls to undeclared
+  tools per the OpenAI contract instead of dropping them (David Tai,
+  #195).
+- Constrained generation bounds the `<think>` prelude at 4000 characters
+  (`MTPLX_THINK_PRELUDE_MAX_CHARS`, 0 restores unbounded), so an
+  unclosed think block can no longer consume the entire token budget and
+  return no document (Jozef Kristek, #213).
+- Forge model probes recover from slow Hugging Face config responses:
+  30s timeout, pinned-SHA retry, positive-MTP-only indexed acceptance,
+  and revision-string validation (Philip John Basile, #210).
+
+### Changed
+
+- Dependency bumps: pillow 12.3.0, actions/checkout 7.0.1,
+  actions/setup-python 7.0.0, pypa/gh-action-pypi-publish 1.14.1.
+
+## [2.3.0] - 2026-07-21
+
+The agent reliability release: the #170 tool-argument collapse is
+root-caused and fixed, structured output ships with full speculative
+speed (community-contributed), agent sessions keep their warm cache
+through client history rewrites, and eight findings from an independent
+source review of v2.2.0 are fixed, each with a regression test that
+fails on the pre-fix code.
+
+### Fixed
+
+- Agent tool calls with nested arguments no longer collapse to empty `{}`
+  arguments (#170). The streaming Qwen-XML tool parser silently discarded
+  function-body text that was not wrapped in `<parameter=>` blocks, so a
+  model that wrote its arguments as a JSON object inside the
+  `<function=...>` envelope — the common slip on nested `edits` arrays —
+  produced a schema-valid call with empty arguments on requiredless tools
+  and a silently vanished call otherwise. Both the streaming and the final
+  parser now accept a pure JSON-object function body as the arguments
+  payload, and every unwrapped, lead, or trailing text lane is a loud
+  protocol fallback with identical contracts in the two parsers.
+- The injected tool contract no longer shows degenerate `[]` examples for
+  array parameters: exemplars are populated from the item schema's own
+  keys (an `edits` array renders as
+  `[{"search": "ARGUMENT_VALUE", "replace": "ARGUMENT_VALUE"}]`), removing
+  an in-prompt template for empty-array emissions.
+- The final tool-call extraction lane (non-stream parsing and the stream
+  fallback) no longer fabricates `{}`-argument calls out of function bodies
+  it could not read — live-reproduced on v2.2.0 as a `grep` call arriving
+  with empty arguments. All extraction dialects now share the strict
+  parsers' contract: a pure JSON-object function body is the arguments
+  payload, an unreadable body stays visible content, blank-body no-argument
+  calls still parse, and partial arguments are delivered exactly as the
+  model wrote them (schema validation remains the client's job, per the
+  OpenAI protocol).
+- Context-copy speculative rounds now stop accepting at the first
+  accepted stop token, exactly like the MTP acceptance loop. Previously
+  the copy lane could accept an entire block past a stop, leaving the
+  target cache, logits/hidden selection, and MTP history advanced beyond
+  the emitted response while the final state was still marked safe to
+  commit — a session-cache poisoning risk on recurrent (GDN) models.
+- Context-copy is prompt-only again at the boundary: proposal blocks are
+  sliced from the prompt and capped at its edge instead of running into
+  the model's own generated output (the self-repetition case the feature
+  contract excludes). Boundary-less candidates are skipped inside the
+  n-gram index so the best *valid* match still fires.
+- Non-divisor packed MTP quantization widths (5-bit, 3-bit, 6-bit) now
+  infer the correct group size via total-bit arithmetic (#182; fix by
+  @Jonathangadeaharder in #183). A 5-bit group-64 head previously
+  inferred group 60, overwrote the artifact's correct declared contract,
+  and made the model unloadable.
+- Long-running daemons no longer accumulate unbounded telemetry: the
+  SessionBank eviction log is a bounded ring, the scheduler's
+  started-by-batch-key counter collapses per-session key suffixes to
+  stable classes, the dashboard per-session TPS map is LRU-bounded, and
+  the OpenCode title fast path trims request metrics like every other
+  lane (#145-adjacent slow-creep hygiene).
+- The desktop Hugging Face probe now recognizes rootless assistant-pair
+  bundles (`mtplx_pair.json` + `target/config.json`) the same way the
+  Python runtime does, so the official Gemma4 pair repos classify as
+  ready to download instead of unreadable (#107); Forge routes them to
+  install-instead-of-rebuild.
+- `mtplx doctor --json` stdout is now guaranteed machine-parseable: the
+  report is built with stdout routed to stderr, so third-party lazy
+  importers that print() their errors (huggingface_hub does, whenever its
+  HTTP dependencies are broken or split across install locations) can no
+  longer prefix the JSON document with prose and break `--json` consumers.
+- Agent tool-loop sessions no longer lose their warm cache when the
+  client rewrites history (transcript compaction, retroactive tool-result
+  digests): a common-prefix identity fallback (≥4096 shared tokens and
+  ≥25% of the prompt) keeps the session id instead of minting an
+  anonymous one, ending the cold full-context re-prefills those
+  rotations caused mid-session.
+- Fan restore goes through the ThermalForge daemon socket instead of the
+  app-killing CLI path, and `mtplx pull` detects interrupted downloads
+  as incomplete instead of treating them as ready (thanks @titan550,
+  #178/#179/#180/#181).
+
+### Changed
+
+- `parallel_tool_calls` is now honored: a request that declares it gets
+  the declared behavior in both directions (false → at most one tool
+  call per turn), with the previous client-profile heuristic kept only
+  as the fallback when the field is absent (thanks @PhilipJohnBasile,
+  #190 — Android Studio declares false today and was being ignored).
+- Agent tool contract v13: instructs whole-file reads (the old
+  "smallest read range" clause provoked storms of 1-to-5-line
+  micro-reads) and forbids echoing file contents into visible text
+  (double-emitted file bodies inflated agent turns by tens of
+  thousands of characters). Measured on live agent sessions: thinking
+  share of generated tokens 75% → 6-29%, double emissions eliminated.
+
+### Added
+
+- Structured output: `response_format` `json_object` and `json_schema`
+  are now enforced with llguidance token masks instead of silently
+  ignored — and the grammar composes with the MTP verify loop, so
+  constrained requests keep full speculative speed (measured: decode
+  parity within noise, under 5ms total mask cost per request). On
+  thinking templates a prelude grammar lets reasoning finish before the
+  document is forced. Opt-in strict tool-call grammars
+  (`MTPLX_TOOL_CALL_STRICT=1`) force every tool call to a declared tool
+  name with schema-valid arguments — a real OpenCode session built a
+  complete project through the strict lane with zero malformed calls.
+  Thanks @PhilipJohnBasile (#186, #187, #188). Requires the `[server]`
+  extra, which the desktop app installs by default; bare pip installs
+  get a clear 400 with an install hint.
+- Durable per-request telemetry: `--request-log-jsonl` (env
+  `MTPLX_REQUEST_LOG_JSONL`) appends every request record as one JSON
+  line, and `scripts/session_forensics.py` correlates that log with an
+  OpenCode database into a single timeline with detectors for re-prefill
+  rewinds, TTFT stalls, thinking marathons, double emissions,
+  session-identity rotations, and usage mismatches.
+- An **opt-in** agent-lane reasoning budget (`--agent-thinking-budget`,
+  env `MTPLX_THINKING_BUDGET`; OFF by default): at the budget the
+  reasoning segment is force-closed with a visible bridge so the turn
+  proceeds to its answer or tool call, and every engagement is surfaced
+  per-request in telemetry (`thinking_guard`). Below the budget decoding
+  is bit-exact; plain chat is never touched.
+- Stream stall watchdog (#86 containment): if a stream receives nothing
+  while the model owner's progress heartbeat is frozen for
+  `MTPLX_STREAM_STALL_DEADLINE_S` (default 300s, 0 disables), the
+  request fails with a structured, diagnosable error and releases its
+  slot instead of hanging forever. Healthy long prefills and model loads
+  tick the heartbeat and are never affected; the daemon is never killed.
+- `pip install mtplx` now includes Pillow, matching the advertised image
+  support in the app and server (#103); previously vision failed at
+  import unless the `[server]` extra was installed.
+
 ## [2.2.0] - 2026-07-19
 
 The copy-drafting and small-Mac release. Decoding: context-copy

{mtplx-2.2.0 → mtplx-2.4.0}/NOTICE

@@ -20,3 +20,7 @@ not include or depend on the vLLM serving stack.
 This product includes Metal kernel code adapted from dflash-mlx
 (https://github.com/bstnxbt/dflash-mlx), Copyright dflash-mlx contributors,
 licensed under the Apache License 2.0. See mtplx/nax_verify.py for details.
+
+This product includes the Apache-2.0 licensed MLX implementation for
+Laguna-S-2.1 from PipeNetwork, Copyright 2026 PipeNetwork, under
+mtplx/models/laguna.py.

{mtplx-2.2.0 → mtplx-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mtplx
-Version: 2.2.0
+Version: 2.4.0
 Summary: Native MTP speculative decoding for Qwen3-Next on Apple Silicon.
 Author-email: Youssof Altoukhi <business@youssofal.com>
 License-Expression: Apache-2.0
@@ -25,6 +25,7 @@ License-File: LICENSE
 License-File: NOTICE
 Requires-Dist: fastapi>=0.136
 Requires-Dist: huggingface-hub>=0.36
+Requires-Dist: pillow>=10
 Requires-Dist: mlx<0.33,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
 Requires-Dist: mlx-lm<0.32,>=0.31; sys_platform == "darwin" and platform_machine == "arm64"
 Requires-Dist: transformers!=5.13.0,<5.14; sys_platform == "darwin" and platform_machine == "arm64"
@@ -38,7 +39,7 @@ Provides-Extra: competitors
 Requires-Dist: dflash-mlx==0.1.0; extra == "competitors"
 Provides-Extra: server
 Requires-Dist: fastapi>=0.136; extra == "server"
-Requires-Dist: pillow>=10; extra == "server"
+Requires-Dist: llguidance>=1.7; extra == "server"
 Requires-Dist: uvicorn>=0.46; extra == "server"
 Provides-Extra: dev
 Requires-Dist: build>=1.2; extra == "dev"
@@ -153,7 +154,26 @@ Fan-backed modes restore your fans to automatic if MTPLX dies for any reason, in
 
 ## Compatibility, honestly
 
-`mtplx inspect` classifies any model into four tiers before anything runs: verified, architecture-compatible but unverified, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+`mtplx inspect` classifies models before anything runs: verified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+
+[Laguna-S-2.1 oQ4e](https://huggingface.co/mlx-community/Laguna-S-2.1-oQ4e) is supported through its exact MLX architecture in target-only AR mode:
+
+```bash
+mtplx start cli \
+  --model mlx-community/Laguna-S-2.1-oQ4e \
+  --download \
+  --no-mtp
+```
+
+MTPLX pins that model to revision
+`8e3f5cad513746264940c1c4195de48d7ea345a5` and verifies the 13-shard layout,
+tokenizer, generation config, special tokens map, and Poolside chat template
+before admitting it. The checkpoint has no native MTP head, so an MTP launch is
+rejected before weights load instead of falling back during execution. The
+weights occupy 59.72 GiB (64.13 GB); use a Mac with at least 96 GiB unified
+memory (128 GiB is recommended). MTPLX defaults Laguna to a 32,768-token context
+and response cap, and checks larger explicit server contexts against the active
+Metal memory cap.
 
 ## What MTPLX is not
 

{mtplx-2.2.0 → mtplx-2.4.0}/README.md

@@ -104,7 +104,26 @@ Fan-backed modes restore your fans to automatic if MTPLX dies for any reason, in
 
 ## Compatibility, honestly
 
-`mtplx inspect` classifies any model into four tiers before anything runs: verified, architecture-compatible but unverified, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+`mtplx inspect` classifies models before anything runs: verified, architecture-compatible but unverified, AR-only, incompatible architecture, or no MTP heads at all. Unverified models refuse to run unless you explicitly force them. There are no silent fallbacks: if MTPLX cannot run a model correctly, it tells you instead of running it badly.
+
+[Laguna-S-2.1 oQ4e](https://huggingface.co/mlx-community/Laguna-S-2.1-oQ4e) is supported through its exact MLX architecture in target-only AR mode:
+
+```bash
+mtplx start cli \
+  --model mlx-community/Laguna-S-2.1-oQ4e \
+  --download \
+  --no-mtp
+```
+
+MTPLX pins that model to revision
+`8e3f5cad513746264940c1c4195de48d7ea345a5` and verifies the 13-shard layout,
+tokenizer, generation config, special tokens map, and Poolside chat template
+before admitting it. The checkpoint has no native MTP head, so an MTP launch is
+rejected before weights load instead of falling back during execution. The
+weights occupy 59.72 GiB (64.13 GB); use a Mac with at least 96 GiB unified
+memory (128 GiB is recommended). MTPLX defaults Laguna to a 32,768-token context
+and response cap, and checks larger explicit server contexts against the active
+Metal memory cap.
 
 ## What MTPLX is not