pi-lilac-provider 1.3.0 → 1.4.0

package/README.md

@@ -103,13 +103,33 @@ pi --provider lilac --model moonshotai/kimi-k2.6
 
 ### Thinking Mode
 
-All Lilac models support chain-of-thought reasoning via `chat_template_kwargs`. Pi uses the `qwen-chat-template` thinking format to send both `thinking` and `enable_thinking` keys, which works across all model families:
-
-- **Kimi K2.6**: Honors `thinking` key (Moonshot template)
-- **GLM 5.1**: Honors `enable_thinking` key (Z.ai template)
-- **Gemma 4**: Honors `enable_thinking` key (Google template)
-
-In pi, reasoning models automatically use the appropriate thinking format. Use Shift+Tab to control thinking level.
+All Lilac models toggle reasoning via `chat_template_kwargs`, but the key each
+model's chat template honors differs per family. The provider uses pi's
+`chat-template` thinkingFormat with per-model `chatTemplateKwargs` (configured in
+`patch.json`) so the right key reaches each template:
+
+| Model | Reasoning key | Default |
+|-------|---------------|---------|
+| Kimi K2.6 | `thinking` (bool) | on |
+| GLM 5.1 | `enable_thinking` (bool) | on |
+| GLM 5.2 | `enable_thinking` (bool) + `reasoning_effort` (`max`\|`high`) | on (`max`) |
+| Gemma 4 | `enable_thinking` (bool) | off |
+| MiniMax M2.7 | `thinking` + `enable_thinking` (bool) | on |
+| MiniMax M3 | `thinking_mode` (`disabled`\|`adaptive`\|`enabled`) | adaptive (server) |
+
+Kimi K2.6, GLM 5.1, Gemma 4, and MiniMax M2.7 use the forward-compatible form
+that sends **both** `thinking` and `enable_thinking`, so whichever key the
+template honors is set. GLM 5.2 additionally maps pi's thinking levels to
+`reasoning_effort` (`high` = lower-latency, `xhigh` = `max`). MiniMax M3 uses
+the `thinking_mode` enum, exposed as three pi thinking levels: `off` →
+`disabled` (never think), `minimal` → `adaptive` (the model decides), `high` →
+`enabled` (always think). Pi starts at `off` (`disabled`); cycle to `minimal`
+for M3's adaptive "model decides" mode. (The selector/footer show pi's level
+names — `minimal`/`high` — not the `thinking_mode` values; pi has no per-model
+level-relabel hook.)
+
+In pi, reasoning models automatically use the appropriate thinking format. Use
+Shift+Tab to control thinking level.
 
 ### Vision
 
@@ -153,7 +173,7 @@ Add to your pi configuration for automatic loading:
 
 Lilac's API is OpenAI-compatible with these specifics:
 
-- **`thinkingFormat: "qwen-chat-template"`** — All reasoning models. Lilac uses `chat_template_kwargs` (with `thinking` and `enable_thinking` keys) to toggle reasoning. Pi sends both keys for forward compatibility.
+- **`thinkingFormat: "chat-template"`** — All reasoning models. Lilac's vLLM backend toggles reasoning via `chat_template_kwargs`, but the honored key differs per model family. Per-model `chatTemplateKwargs` in `patch.json` send the right key(s): `thinking`+`enable_thinking` (bool) for Kimi K2.6, GLM 5.1, Gemma 4, and MiniMax M2.7; `enable_thinking` + `reasoning_effort` for GLM 5.2; `thinking_mode` (adaptive|enabled|disabled) for MiniMax M3.
 - **`maxTokensField: "max_completion_tokens"`** — All models. Lilac supports `max_completion_tokens` (preferred for reasoning models as it includes reasoning tokens).
 - **`supportsDeveloperRole: true`** — All models. Lilac's vLLM backend maps the developer role to system.
 - **`supportsStore: false`** — All models. Lilac doesn't support the `store` parameter.

package/index.ts

@@ -5,14 +5,28 @@
  * Base URL: https://api.getlilac.com/v1
  *
  * Lilac serves models via a customized fork of vLLM tuned for idle-GPU scheduling
- * and shared warm endpoints. All models use chat_template_kwargs to toggle reasoning:
+ * and shared warm endpoints. Reasoning is toggled via chat_template_kwargs, but
+ * the key each model's chat template honors differs per family, so per-model
+ * chatTemplateKwargs are configured in patch.json:
  *
- *   - Kimi K2.6: reasoning ON by default, honors `thinking` key
- *   - GLM 5.1:   reasoning ON by default, honors `enable_thinking` key
- *   - Gemma 4:   reasoning OFF by default, honors `enable_thinking` key
+ *   - Kimi K2.6:    honors `thinking` (bool); `enable_thinking` ignored. ON by default.
+ *   - GLM 5.1:      honors `enable_thinking` (bool). ON by default.
+ *   - GLM 5.2:      honors `enable_thinking` (bool) + `reasoning_effort` (max|high). ON by default.
+ *   - Gemma 4:      honors `enable_thinking` (bool). OFF by default.
+ *   - MiniMax M2.7: forward-compatible `thinking` + `enable_thinking` (bool).
+ *   - MiniMax M3:   honors `thinking_mode` (disabled|adaptive|enabled); bool keys ignored.
  *
- * The forward-compatible approach is to send both `thinking` and `enable_thinking`
- * in chat_template_kwargs — pi's `qwen-chat-template` thinkingFormat does this.
+ * We use pi's `chat-template` thinkingFormat (NOT `qwen-chat-template`, which
+ * sends only `enable_thinking` + `preserve_thinking` and is ignored by Kimi and
+ * MiniMax M3). The forward-compatible form sends BOTH `thinking` and
+ * `enable_thinking` so whichever key the template honors is set:
+ *   { chat_template_kwargs: { thinking: <bool>, enable_thinking: <bool> } }
+ * GLM 5.2 adds `reasoning_effort` (high = lower-latency, xhigh = max) via a
+ * thinkingLevelMap. MiniMax M3 maps to the `thinking_mode` enum as three pi
+ * thinking levels — off→disabled, minimal→adaptive (model decides), high→enabled
+ * — so adaptive is selectable via pi's Shift+Tab cycle (off→minimal→high). Pi
+ * shows the pi level names (minimal/high) in the selector/footer, not the
+ * thinking_mode values; there's no per-model level-relabel hook.
  *
  * Key API notes:
  *   - Uses `max_completion_tokens` (preferred for reasoning models)
@@ -99,6 +113,17 @@ type ThinkingLevelMap = {
   xhigh?: string | null;
 };
 
+// A chat_template_kwargs value, mirroring pi-ai's ChatTemplateKwargSchema. Scalar
+// values are passed through verbatim; { $var } values are resolved by pi-ai from
+// the turn's thinking state ("thinking.enabled" → bool, "thinking.effort" → the
+// mapped effort string). omitWhenOff drops the key entirely when thinking is off.
+type ChatTemplateKwargValue =
+  | string
+  | number
+  | boolean
+  | null
+  | { $var: "thinking.enabled" | "thinking.effort"; omitWhenOff?: boolean };
+
 interface JsonModel {
   id: string;
   name: string;
@@ -117,7 +142,19 @@ interface JsonModel {
     supportsDeveloperRole?: boolean;
     supportsStore?: boolean;
     maxTokensField?: "max_completion_tokens" | "max_tokens";
-    thinkingFormat?: "openai" | "zai" | "qwen" | "qwen-chat-template";
+    thinkingFormat?:
+      | "openai"
+      | "openrouter"
+      | "together"
+      | "deepseek"
+      | "zai"
+      | "qwen"
+      | "chat-template"
+      | "qwen-chat-template"
+      | "string-thinking"
+      | "ant-ling";
+    chatTemplateKwargs?: Record<string, ChatTemplateKwargValue>;
+    zaiToolStream?: boolean;
     supportsReasoningEffort?: boolean;
   };
   discount?: JsonDiscount;
@@ -252,14 +289,21 @@ function transformApiModel(apiModel: any): JsonModel | null {
     maxTokens: apiModel.top_provider?.max_completion_tokens || apiModel.context_length || 131072,
   };
 
-  // All Lilac models use chat_template_kwargs for reasoning toggle
+  // All Lilac models toggle reasoning via chat_template_kwargs, but the key each
+  // model's chat template honors differs per family. Default newly discovered
+  // models to the forward-compatible both-keys form (works across all current
+  // Lilac templates); per-model overrides in patch.json refine this — e.g. GLM
+  // 5.2 adds reasoning_effort, MiniMax M3 uses the thinking_mode enum.
   if (features.includes("reasoning")) {
     model.compat = {
       supportsDeveloperRole: true,
       supportsStore: false,
       maxTokensField: "max_completion_tokens",
-      thinkingFormat: "qwen-chat-template",
-      supportsReasoningEffort: true,
+      thinkingFormat: "chat-template",
+      chatTemplateKwargs: {
+        thinking: { $var: "thinking.enabled" },
+        enable_thinking: { $var: "thinking.enabled" },
+      },
     };
   } else {
     model.compat = {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-lilac-provider",
-  "version": "1.3.0",
+  "version": "1.4.0",
   "description": "Lilac provider extension for pi - Access Kimi K2.6, GLM 5.1, and Gemma 4 models through Lilac's OpenAI-compatible API on idle GPUs",
   "type": "module",
   "main": "index.ts",

package/patch.json

@@ -7,11 +7,14 @@
       "cacheRead": 0.20
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "thinking": { "$var": "thinking.enabled" },
+        "enable_thinking": { "$var": "thinking.enabled" }
+      },
       "maxTokensField": "max_completion_tokens",
       "supportsDeveloperRole": false,
-      "supportsStore": false,
-      "supportsReasoningEffort": true
+      "supportsStore": false
     }
   },
   "zai-org/glm-5.1": {
@@ -23,20 +26,29 @@
       "cacheRead": 0.27
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "thinking": { "$var": "thinking.enabled" },
+        "enable_thinking": { "$var": "thinking.enabled" }
+      },
       "maxTokensField": "max_completion_tokens",
       "supportsDeveloperRole": false,
       "supportsStore": false,
-      "zaiToolStream": true,
-      "supportsReasoningEffort": true
+      "zaiToolStream": true
     }
   },
   "zai-org/glm-5.2": {
+    "compat": {
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "enable_thinking": { "$var": "thinking.enabled" },
+        "reasoning_effort": { "$var": "thinking.effort", "omitWhenOff": true }
+      }
+    },
     "thinkingLevelMap": {
-      "off": "minimal",
       "minimal": null,
-      "low": null,
-      "medium": null,
+      "low": "high",
+      "medium": "high",
       "high": "high",
       "xhigh": "max"
     }
@@ -49,16 +61,40 @@
       "cacheRead": 0
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "thinking": { "$var": "thinking.enabled" },
+        "enable_thinking": { "$var": "thinking.enabled" }
+      },
       "maxTokensField": "max_completion_tokens",
       "supportsDeveloperRole": true,
-      "supportsStore": false,
-      "supportsReasoningEffort": true
+      "supportsStore": false
     }
   },
   "minimaxai/minimax-m2.7": {
     "compat": {
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "thinking": { "$var": "thinking.enabled" },
+        "enable_thinking": { "$var": "thinking.enabled" }
+      },
       "supportsDeveloperRole": false
     }
+  },
+  "minimaxai/minimax-m3": {
+    "compat": {
+      "thinkingFormat": "chat-template",
+      "chatTemplateKwargs": {
+        "thinking_mode": { "$var": "thinking.effort" }
+      }
+    },
+    "thinkingLevelMap": {
+      "off": "disabled",
+      "minimal": "adaptive",
+      "low": null,
+      "medium": null,
+      "high": "enabled",
+      "xhigh": null
+    }
   }
 }

package/scripts/update-models.js

@@ -147,7 +147,11 @@ function transformApiModel(apiModel, existingModelsMap) {
     maxTokens: apiModel.top_provider?.max_completion_tokens || apiModel.context_length || 131072,
   };
 
-  // Add compat — all Lilac models use chat_template_kwargs for reasoning toggle
+  // Add compat — all Lilac models toggle reasoning via chat_template_kwargs, but
+  // the honored key differs per model family. Default newly discovered models to
+  // the forward-compatible both-keys form (works across all current Lilac
+  // templates); refine per-model in patch.json (e.g. GLM 5.2 adds reasoning_effort,
+  // MiniMax M3 uses the thinking_mode enum).
   const compat = {
     supportsDeveloperRole: true,
     supportsStore: false,
@@ -155,7 +159,11 @@ function transformApiModel(apiModel, existingModelsMap) {
   };
 
   if (hasReasoning) {
-    compat.thinkingFormat = 'qwen-chat-template';
+    compat.thinkingFormat = 'chat-template';
+    compat.chatTemplateKwargs = {
+      thinking: { $var: 'thinking.enabled' },
+      enable_thinking: { $var: 'thinking.enabled' },
+    };
   }
 
   model.compat = compat;