pi-makora-provider 1.1.0 → 1.3.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Open-weight models through [Makora](https://inference.makora.com)**
 
-_DeepSeek V4, Kimi K2.6, GLM 5.1, Qwen 3.6 — with client-side tool call repair for [pi](https://github.com/earendil-works/pi-coding-agent)._
+_DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 — with client-side tool call repair and preserved-thinking flags for [pi](https://github.com/earendil-works/pi-coding-agent)._
 
 [![pi extension](https://img.shields.io/badge/pi-extension-blueviolet)](https://github.com/earendil-works/pi-coding-agent)
 [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
@@ -19,14 +19,14 @@ _DeepSeek V4, Kimi K2.6, GLM 5.1, Qwen 3.6 — with client-side tool call repair
 | Model | ID | Reasoning | Notes |
 |-------|----|-----------|-------|
 | DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes | `include_reasoning` + `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field |
-| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning_content` field |
-| GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | `enable_thinking` via `qwen-chat-template`; thinking levels copied from neuralwatt GLM 5.2; returns `reasoning_content` field |
-| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default; client-side tool call parsing (vLLM streaming parser bypass) |
+| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field |
+| GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field |
+| GLM 5.2 NVFP4 | `zai-org/GLM-5.2-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field |
+| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
 | Llama 3.3 70B FP8 | `amd/Llama-3.3-70B-Instruct-FP8-KV` | No |  |
 | Llama 3.3 70B Instruct | `meta-llama/Llama-3.3-70B-Instruct` | No |  |
-| MiniMax M3 MXFP8 | `MiniMaxAI/MiniMax-M3-MXFP8` | Yes | Reasoning via `chat_template_kwargs.enable_thinking`; returns `reasoning_content` field |
-| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; client-side tool call parsing (vLLM streaming parser bypass) |
-| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; client-side tool call parsing (vLLM streaming parser bypass) |
+| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
+| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
 <!-- MODELS_TABLE_END -->
 
 ## Installation
@@ -163,15 +163,17 @@ These issues are common to all vLLM-hosted providers and affect Makora models:
 
 - **GLM tool calls — follow-up `tool_calls` crash workaround**: the ZAI chat template calls `.items()` on any assistant message that carries a `tool_calls` field (on the `tool_calls` list or on its JSON-string `arguments` field), raising a leaked Python `AttributeError` that vLLM returns as HTTP 400 (`'list object' has no attribute 'items'` / `'str object' has no attribute 'items'`). The `before_provider_request` hook (`stripGlmToolCalls`) strips `tool_calls` from assistant messages and converts them to GLM-native `<tool_call>` XML text in `content` — which the model natively understands in conversation history — before the follow-up request is sent. The hook is scoped to Makora's own GLM models via an exact model-id match (`isMakoraGlmVllmModel`): `before_provider_request` is global in pi (it fires for every loaded provider's requests and the event carries no `provider` field), so gating by model id is what keeps it from touching GLM models served by other providers. If upstream fixes the `.items()` crash, this transform is a harmless no-op (the XML text is valid GLM input either way).
 
-- **Kimi K2.6 + Qwen 3.6 tool calling**: vLLM's server-side streaming tool-call handling is broken or missing for these models — Makora's vLLM is missing both `--enable-auto-tool-choice` and `--tool-call-parser` for them, so they emit tool calls as raw text tokens rather than structured `tool_call` deltas.
+- **Kimi K2.7 Code + Qwen 3.6 tool calling**: vLLM's server-side streaming tool-call handling is broken or missing for these models — Makora's vLLM is missing both `--enable-auto-tool-choice` and `--tool-call-parser` for them, so they emit tool calls as raw text tokens rather than structured `tool_call` deltas.
 
-    - **Kimi K2.6**: Uses `<|tool_call_begin|>...<|tool_call_end|>` tokens.
+    - **Kimi K2.7 Code**: Uses `<|tool_call_begin|>...<|tool_call_end|>` tokens.
     - **Qwen 3.6**: Uses hermes-style `<function=...>` XML, sometimes with `█` delimiters.
 
-- **GLM 5.1 CoT leak**: On some vLLM builds, disabling reasoning may still leak chain-of-thought into `content` terminated by a ``` marker. See [vllm-project/vllm#31319](https://github.com/vllm-project/vllm/issues/31319).
+- **GLM 5.2 CoT leak**: On some vLLM builds, disabling reasoning may still leak chain-of-thought into `content` terminated by a ``` marker. See [vllm-project/vllm#31319](https://github.com/vllm-project/vllm/issues/31319).
 
 - **DeepSeek V4 reasoning**: The official DeepSeek API uses `thinking: { type: "enabled" }` which Makora's vLLM silently ignores. The `before_provider_request` hook rewrites the payload to use vLLM-native params instead:
-  - **DS V4 Pro**: `chat_template_kwargs: { thinking: true }`. Returns `reasoning_content`.
+  - **DS V4 Pro**: `chat_template_kwargs: { thinking: true }`. Returns `reasoning` (vLLM reasoning parser; the official DeepSeek API returns `reasoning_content`, but Makora's vLLM build returns `reasoning`).
   - **DS V4 Flash**: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`. `include_reasoning` alone returns `reasoning: null` on this vLLM build — both params are required. Returns `reasoning`.
-- **GLM 5.1 reasoning**: Returns `reasoning_content` (not `reasoning`). pi's OpenAI completions handler checks `reasoning_content` first, so this is handled correctly.
-- **MiniMax M3 reasoning**: Uses `chat_template_kwargs.enable_thinking` to toggle thinking (not `chat_template_kwargs.thinking` like DeepSeek). The `before_provider_request` hook rewrites the DeepSeek API-style `thinking` param into vLLM-native `chat_template_kwargs: { enable_thinking: true }`. Returns `reasoning_content` field.
+- **Preserved thinking (multi-turn reasoning continuity)**: every reasoning model on Makora returns reasoning in the `reasoning` field (not `reasoning_content`). Pi sends prior assistant reasoning back on follow-up turns; Makora's vLLM chat templates accept it under either `reasoning` or `reasoning_content`. For models whose chat template defines an explicit preserved-thinking flag, the flag is sent via `compat.chatTemplateKwargs`:
+  - **Kimi K2.7 Code**: `preserve_thinking: true` (the model forces thinking-only mode with preserve_thinking on, per the vLLM recipe).
+  - **Qwen 3.6 27B / 35B**: `preserve_thinking: true` (Qwen 3.6 ships with preserve_thinking).
+  - **GLM 5.2**: no extra flag — the vLLM GLM-5.2 recipe uses only `enable_thinking` + `reasoning_effort`; preserved thinking works without a dedicated flag. Effort honors only `high` and `max` (lower pi levels resolve to the default `max`).

package/index.ts

@@ -17,7 +17,7 @@
  *     pi sends thinking: { type } via the "deepseek" thinkingFormat, but vLLM
  *     ignores that — the before_provider_request hook rewrites the payload to
  *     use chat_template_kwargs: { thinking: true } instead.
- *     Returns reasoning_content field.
+ *     Returns reasoning field (vLLM reasoning parser; not reasoning_content).
  *   - DeepSeek V4 Flash: reasoning via include_reasoning +
  *     chat_template_kwargs.thinking on vLLM.
  *     The before_provider_request hook rewrites the payload to replace
@@ -25,26 +25,18 @@
  *     chat_template_kwargs: { thinking: true }.
  *     include_reasoning alone returns reasoning: null on this vLLM build.
  *     Returns reasoning field.
- *   - GLM 5.1 FP8: reasoning via chat_template_kwargs.enable_thinking.
- *     NOTE: vLLM may leak chain-of-thought into content instead of the
- *     reasoning field on some builds. See
- *     https://github.com/vllm-project/vllm/issues/31319
- *     Also: vLLM's streaming parser omits delta.tool_calls when the model
- *     calls tools, finishing with finish_reason: "tool_calls" but an empty
- *     delta. Setting zaiToolStream: true sends tool_stream: true in the
- *     request, which forces vLLM to use the explicit tool streaming path
- *     that correctly emits tool call chunks.
- *   - GLM 5.2 FP8: reasoning via chat_template_kwargs.enable_thinking.
- *     Thinking levels (minimal/low/medium/high/max) are copied from the
- *     neuralwatt provider's GLM 5.2 configuration and mapped through pi's
- *     qwen-chat-template thinkingFormat.
- *   - GPT-OSS 120B: reasoning always on; returns `reasoning` field.
- *   - Kimi K2.6 NVFP4 / Kimi K2.7 Code: reasoning always on by default;
- *     returns `reasoning` field. Can be toggled via enable_thinking.
+ *   - GLM 5.2 FP8 / NVFP4: reasoning via chat_template_kwargs.enable_thinking.
+ *     Effort uses vLLM's reasoning_effort field (only `high` and `max` are
+ *     distinct levels per the vLLM GLM-5.2 recipe; lower pi levels resolve to
+ *     the default). Thinking levels aligned with the neuralwatt provider's
+ *     GLM 5.2 configuration and mapped through pi's qwen-chat-template
+ *     thinkingFormat. Returns `reasoning` field.
+ *   - Kimi K2.7 Code: reasoning always on (thinking-only model);
+ *     chatTemplateKwargs.preserve_thinking forces multi-turn reasoning
+ *     continuity. Returns `reasoning` field. Can be toggled via enable_thinking.
  *   - Qwen 3.6 models: reasoning via chat_template_kwargs.enable_thinking;
- *     returns `reasoning` field.
- *   - MiniMax M3 MXFP8: reasoning via chat_template_kwargs.enable_thinking;
- *     returns reasoning_content field.
+ *     chatTemplateKwargs.preserve_thinking for multi-turn continuity.
+ *     Returns `reasoning` field.
  *   - Llama 3.3 70B: not a reasoning model.
  *
  * Developer role is NOT supported by any of the chat templates on Makora's
@@ -109,6 +101,10 @@ interface JsonModel {
     requiresToolResultName?: boolean;
     requiresAssistantAfterToolResult?: boolean;
     cacheControlFormat?: "anthropic";
+    /** Extra keys merged into vLLM `chat_template_kwargs` on every request.
+     *  Used for preserved-thinking flags like `preserve_thinking` / `clear_thinking`
+     *  that some chat templates require for multi-turn reasoning continuity. */
+    chatTemplateKwargs?: Record<string, unknown>;
   };
 }
 
@@ -218,10 +214,8 @@ const BASE_URL = "https://inference.makora.com/v1";
 
 const DS_PRO_ID = "deepseek-ai/DeepSeek-V4-Pro";
 const DS_FLASH_ID = "deepseek-ai/DeepSeek-V4-Flash";
-const MINIMAX_M3_ID = "MiniMaxAI/MiniMax-M3-MXFP8";
 
 const DS_VLLM_MODELS = new Set([DS_PRO_ID, DS_FLASH_ID]);
-const ENABLE_THINKING_VLLM_MODELS = new Set([MINIMAX_M3_ID]);
 
 /**
  * Makora's GLM models, built from the same models list this extension
@@ -266,8 +260,6 @@ export function isMakoraGlmVllmModel(model: string): boolean {
  *   - DS V4 Flash: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`
  *     + `reasoning_effort`. `include_reasoning` alone returns `reasoning: null`
  *     on this vLLM build — both params are required.
- *   - MiniMax M3: `chat_template_kwargs: { enable_thinking: true }` +
- *     `reasoning_effort`. Returns `reasoning_content` field.
  *
  * This hook rewrites the payload accordingly.
  */
@@ -293,11 +285,6 @@ function rewriteVllmPayload(payload: Record<string, unknown>): Record<string, un
       const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
       p.chat_template_kwargs = { ...ctq, thinking: true };
     }
-  } else if (ENABLE_THINKING_VLLM_MODELS.has(model)) {
-    // Models using chat_template_kwargs.enable_thinking (e.g. MiniMax M3)
-    delete p.thinking;
-    const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
-    p.chat_template_kwargs = { ...ctq, enable_thinking: true };
   }
 
   return p;

package/models.json

@@ -63,8 +63,8 @@
     }
   },
   {
-    "id": "MiniMaxAI/MiniMax-M3-MXFP8",
-    "name": "MiniMax M3 MXFP8",
+    "id": "moonshotai/Kimi-K2.7-Code",
+    "name": "Kimi K2.7 Code",
     "reasoning": false,
     "input": [
       "text"
@@ -75,7 +75,7 @@
       "cacheRead": 0,
       "cacheWrite": 0
     },
-    "contextWindow": 1048576,
+    "contextWindow": 262144,
     "maxTokens": 0,
     "compat": {
       "supportsDeveloperRole": false,
@@ -84,8 +84,8 @@
     }
   },
   {
-    "id": "moonshotai/Kimi-K2.7-Code",
-    "name": "Kimi K2.7 Code",
+    "id": "unsloth/Qwen3.6-27B-NVFP4",
+    "name": "Qwen 3.6 27B NVFP4",
     "reasoning": false,
     "input": [
       "text"
@@ -105,8 +105,8 @@
     }
   },
   {
-    "id": "unsloth/Qwen3.6-27B-NVFP4",
-    "name": "Qwen 3.6 27B NVFP4",
+    "id": "unsloth/Qwen3.6-35B-A3B-NVFP4",
+    "name": "Qwen 3.6 35B A3B NVFP4",
     "reasoning": false,
     "input": [
       "text"
@@ -126,8 +126,8 @@
     }
   },
   {
-    "id": "unsloth/Qwen3.6-35B-A3B-NVFP4",
-    "name": "Qwen 3.6 35B A3B NVFP4",
+    "id": "zai-org/GLM-5.2-FP8",
+    "name": "GLM 5.2 FP8",
     "reasoning": false,
     "input": [
       "text"
@@ -138,7 +138,7 @@
       "cacheRead": 0,
       "cacheWrite": 0
     },
-    "contextWindow": 262144,
+    "contextWindow": 1048576,
     "maxTokens": 0,
     "compat": {
       "supportsDeveloperRole": false,
@@ -147,8 +147,8 @@
     }
   },
   {
-    "id": "zai-org/GLM-5.2-FP8",
-    "name": "GLM 5.2 FP8",
+    "id": "zai-org/GLM-5.2-NVFP4",
+    "name": "GLM 5.2 NVFP4",
     "reasoning": false,
     "input": [
       "text"
@@ -159,7 +159,7 @@
       "cacheRead": 0,
       "cacheWrite": 0
     },
-    "contextWindow": 980000,
+    "contextWindow": 1048576,
     "maxTokens": 0,
     "compat": {
       "supportsDeveloperRole": false,

package/package.json

@@ -1,17 +1,9 @@
 {
   "name": "pi-makora-provider",
-  "version": "1.1.0",
-  "description": "Makora provider extension for pi - Access DeepSeek V4, GLM 5.1, Kimi K2.6, Llama 3.3, Qwen 3.6, and more through the Makora inference API",
+  "version": "1.3.0",
+  "description": "Makora provider extension for pi - Access DeepSeek V4, GLM 5.2, Kimi K2.7 Code, Llama 3.3, Qwen 3.6, and more through the Makora inference API",
   "type": "module",
   "main": "index.ts",
-  "scripts": {
-    "clean": "echo 'nothing to clean'",
-    "build": "echo 'nothing to build'",
-    "check": "echo 'nothing to check'",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "update-models": "node scripts/update-models.js"
-  },
   "keywords": [
     "pi",
     "extension",
@@ -42,5 +34,13 @@
   },
   "devDependencies": {
     "vitest": "^4.1.9"
+  },
+  "scripts": {
+    "clean": "echo 'nothing to clean'",
+    "build": "echo 'nothing to build'",
+    "check": "echo 'nothing to check'",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "update-models": "node scripts/update-models.js"
   }
-}
+}

package/patch.json

@@ -16,7 +16,7 @@
   },
   "deepseek-ai/DeepSeek-V4-Pro": {
     "reasoning": true,
-    "notes": "`chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning_content` field",
+    "notes": "`chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": null,
       "low": null,
@@ -30,76 +30,34 @@
       "requiresReasoningContentOnAssistantMessages": true
     }
   },
-  "nvidia/Kimi-K2.6-NVFP4": {
-    "reasoning": true,
-    "input": [
-      "text",
-      "image"
-    ],
-    "notes": "Reasoning on by default; client-side tool call parsing (vLLM streaming parser bypass)",
-    "thinkingLevelMap": {
-      "minimal": "low",
-      "xhigh": "high"
-    },
-    "compat": {
-      "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
-    }
-  },
-  "openai/gpt-oss-120b": {
-    "reasoning": true,
-    "notes": "Reasoning always on",
-    "thinkingLevelMap": {
-      "minimal": "low",
-      "xhigh": "high"
-    },
-    "compat": {
-      "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
-    }
-  },
   "unsloth/Qwen3.6-27B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
     },
     "compat": {
       "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
+      "supportsReasoningEffort": true,
+      "chatTemplateKwargs": {
+        "preserve_thinking": true
+      }
     }
   },
   "unsloth/Qwen3.6-35B-A3B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
     },
     "compat": {
       "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
-    }
-  },
-  "MiniMaxAI/MiniMax-M3-MXFP8": {
-    "reasoning": true,
-    "input": [
-      "text",
-      "image"
-    ],
-    "notes": "Reasoning via `chat_template_kwargs.enable_thinking`; returns `reasoning_content` field",
-    "thinkingLevelMap": {
-      "minimal": null,
-      "low": null,
-      "medium": null,
-      "high": "high",
-      "xhigh": "max"
-    },
-    "compat": {
-      "thinkingFormat": "deepseek",
       "supportsReasoningEffort": true,
-      "requiresReasoningContentOnAssistantMessages": true
+      "chatTemplateKwargs": {
+        "preserve_thinking": true
+      }
     }
   },
   "moonshotai/Kimi-K2.7-Code": {
@@ -108,37 +66,41 @@
       "text",
       "image"
     ],
-    "notes": "Reasoning on by default; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
     },
     "compat": {
       "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
+      "supportsReasoningEffort": true,
+      "chatTemplateKwargs": {
+        "preserve_thinking": true
+      }
     }
   },
-  "zai-org/GLM-5.1-FP8": {
-    "contextWindow": 200000,
+  "zai-org/GLM-5.2-FP8": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; returns `reasoning_content` field; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "`enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field",
     "thinkingLevelMap": {
-      "minimal": "low",
-      "xhigh": "high"
+      "minimal": null,
+      "low": null,
+      "medium": null,
+      "high": "high",
+      "xhigh": "max"
     },
     "compat": {
       "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true,
-      "zaiToolStream": true
+      "supportsReasoningEffort": true
     }
   },
-  "zai-org/GLM-5.2-FP8": {
+  "zai-org/GLM-5.2-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; thinking levels copied from neuralwatt GLM 5.2; returns `reasoning_content` field",
+    "notes": "`enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field",
     "thinkingLevelMap": {
-      "minimal": "minimal",
-      "low": "low",
-      "medium": "medium",
+      "minimal": null,
+      "low": null,
+      "medium": null,
       "high": "high",
       "xhigh": "max"
     },