pi-makora-provider 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -27
- package/index.ts +5 -208
- package/package.json +1 -1
- package/patch.json +5 -67
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
**Open-weight models through [Makora](https://inference.makora.com)**
|
|
6
6
|
|
|
7
|
-
_DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6
|
|
7
|
+
_DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 for [pi](https://github.com/earendil-works/pi-coding-agent)._
|
|
8
8
|
|
|
9
9
|
[](https://github.com/earendil-works/pi-coding-agent)
|
|
10
10
|
[](./LICENSE)
|
|
@@ -18,15 +18,15 @@ _DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 — with client-side tool call r
|
|
|
18
18
|
<!-- MODELS_TABLE_START -->
|
|
19
19
|
| Model | ID | Reasoning | Notes |
|
|
20
20
|
|-------|----|-----------|-------|
|
|
21
|
-
| DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes |
|
|
22
|
-
| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes |
|
|
21
|
+
| DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes | returns `reasoning` field |
|
|
22
|
+
| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | returns `reasoning` field |
|
|
23
23
|
| GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field |
|
|
24
24
|
| GLM 5.2 NVFP4 | `zai-org/GLM-5.2-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field |
|
|
25
|
-
| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
25
|
+
| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
|
|
26
26
|
| Llama 3.3 70B FP8 | `amd/Llama-3.3-70B-Instruct-FP8-KV` | No | |
|
|
27
27
|
| Llama 3.3 70B Instruct | `meta-llama/Llama-3.3-70B-Instruct` | No | |
|
|
28
|
-
| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
29
|
-
| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
28
|
+
| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
|
|
29
|
+
| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
|
|
30
30
|
<!-- MODELS_TABLE_END -->
|
|
31
31
|
|
|
32
32
|
## Installation
|
|
@@ -156,24 +156,3 @@ Do **not** edit `models.json` directly — it is auto-generated from the API. To
|
|
|
156
156
|
- The API is OpenAI-compatible (chat completions format)
|
|
157
157
|
- All models are hosted on vLLM
|
|
158
158
|
- The `developer` role is not supported (prompts are silently dropped); `supportsDeveloperRole` is set to `false` for all models
|
|
159
|
-
|
|
160
|
-
## vLLM Caveats
|
|
161
|
-
|
|
162
|
-
These issues are common to all vLLM-hosted providers and affect Makora models:
|
|
163
|
-
|
|
164
|
-
- **GLM tool calls — follow-up `tool_calls` crash workaround**: the ZAI chat template calls `.items()` on any assistant message that carries a `tool_calls` field (on the `tool_calls` list or on its JSON-string `arguments` field), raising a leaked Python `AttributeError` that vLLM returns as HTTP 400 (`'list object' has no attribute 'items'` / `'str object' has no attribute 'items'`). The `before_provider_request` hook (`stripGlmToolCalls`) strips `tool_calls` from assistant messages and converts them to GLM-native `<tool_call>` XML text in `content` — which the model natively understands in conversation history — before the follow-up request is sent. The hook is scoped to Makora's own GLM models via an exact model-id match (`isMakoraGlmVllmModel`): `before_provider_request` is global in pi (it fires for every loaded provider's requests and the event carries no `provider` field), so gating by model id is what keeps it from touching GLM models served by other providers. If upstream fixes the `.items()` crash, this transform is a harmless no-op (the XML text is valid GLM input either way).
|
|
165
|
-
|
|
166
|
-
- **Kimi K2.7 Code + Qwen 3.6 tool calling**: vLLM's server-side streaming tool-call handling is broken or missing for these models — Makora's vLLM is missing both `--enable-auto-tool-choice` and `--tool-call-parser` for them, so they emit tool calls as raw text tokens rather than structured `tool_call` deltas.
|
|
167
|
-
|
|
168
|
-
- **Kimi K2.7 Code**: Uses `<|tool_call_begin|>...<|tool_call_end|>` tokens.
|
|
169
|
-
- **Qwen 3.6**: Uses hermes-style `<function=...>` XML, sometimes with `█` delimiters.
|
|
170
|
-
|
|
171
|
-
- **GLM 5.2 CoT leak**: On some vLLM builds, disabling reasoning may still leak chain-of-thought into `content` terminated by a ``` marker. See [vllm-project/vllm#31319](https://github.com/vllm-project/vllm/issues/31319).
|
|
172
|
-
|
|
173
|
-
- **DeepSeek V4 reasoning**: The official DeepSeek API uses `thinking: { type: "enabled" }` which Makora's vLLM silently ignores. The `before_provider_request` hook rewrites the payload to use vLLM-native params instead:
|
|
174
|
-
- **DS V4 Pro**: `chat_template_kwargs: { thinking: true }`. Returns `reasoning` (vLLM reasoning parser; the official DeepSeek API returns `reasoning_content`, but Makora's vLLM build returns `reasoning`).
|
|
175
|
-
- **DS V4 Flash**: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`. `include_reasoning` alone returns `reasoning: null` on this vLLM build — both params are required. Returns `reasoning`.
|
|
176
|
-
- **Preserved thinking (multi-turn reasoning continuity)**: every reasoning model on Makora returns reasoning in the `reasoning` field (not `reasoning_content`). Pi sends prior assistant reasoning back on follow-up turns; Makora's vLLM chat templates accept it under either `reasoning` or `reasoning_content`. For models whose chat template defines an explicit preserved-thinking flag, the flag is sent via `compat.chatTemplateKwargs`:
|
|
177
|
-
- **Kimi K2.7 Code**: `preserve_thinking: true` (the model forces thinking-only mode with preserve_thinking on, per the vLLM recipe).
|
|
178
|
-
- **Qwen 3.6 27B / 35B**: `preserve_thinking: true` (Qwen 3.6 ships with preserve_thinking).
|
|
179
|
-
- **GLM 5.2**: no extra flag — the vLLM GLM-5.2 recipe uses only `enable_thinking` + `reasoning_effort`; preserved thinking works without a dedicated flag. Effort honors only `high` and `max` (lower pi levels resolve to the default `max`).
|
package/index.ts
CHANGED
|
@@ -13,42 +13,11 @@
|
|
|
13
13
|
* Model resolution strategy: static models.json merged with custom-models.json
|
|
14
14
|
*
|
|
15
15
|
* Reasoning notes:
|
|
16
|
-
* - DeepSeek V4 Pro: reasoning
|
|
17
|
-
*
|
|
18
|
-
*
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
* - DeepSeek V4 Flash: reasoning via include_reasoning +
|
|
22
|
-
* chat_template_kwargs.thinking on vLLM.
|
|
23
|
-
* The before_provider_request hook rewrites the payload to replace
|
|
24
|
-
* thinking: { type } with include_reasoning: true +
|
|
25
|
-
* chat_template_kwargs: { thinking: true }.
|
|
26
|
-
* include_reasoning alone returns reasoning: null on this vLLM build.
|
|
27
|
-
* Returns reasoning field.
|
|
28
|
-
* - GLM 5.1 FP8: reasoning via chat_template_kwargs.enable_thinking.
|
|
29
|
-
* NOTE: vLLM may leak chain-of-thought into content instead of the
|
|
30
|
-
* reasoning field on some builds. See
|
|
31
|
-
* https://github.com/vllm-project/vllm/issues/31319
|
|
32
|
-
* Also: vLLM's streaming parser omits delta.tool_calls when the model
|
|
33
|
-
* calls tools, finishing with finish_reason: "tool_calls" but an empty
|
|
34
|
-
* delta. Setting zaiToolStream: true sends tool_stream: true in the
|
|
35
|
-
* request, which forces vLLM to use the explicit tool streaming path
|
|
36
|
-
* that correctly emits tool call chunks.
|
|
37
|
-
* - GLM 5.2 FP8 / NVFP4: reasoning via chat_template_kwargs.enable_thinking.
|
|
38
|
-
* Effort uses vLLM's reasoning_effort field (only `high` and `max` are
|
|
39
|
-
* distinct levels per the vLLM GLM-5.2 recipe; lower pi levels resolve to
|
|
40
|
-
* the default). Thinking levels aligned with the neuralwatt provider's
|
|
41
|
-
* GLM 5.2 configuration and mapped through pi's qwen-chat-template
|
|
42
|
-
* thinkingFormat. Returns `reasoning` field.
|
|
43
|
-
* - GPT-OSS 120B: reasoning always on; returns `reasoning` field.
|
|
44
|
-
* - Kimi K2.7 Code: reasoning always on (thinking-only model);
|
|
45
|
-
* chatTemplateKwargs.preserve_thinking forces multi-turn reasoning
|
|
46
|
-
* continuity. Returns `reasoning` field. Can be toggled via enable_thinking.
|
|
47
|
-
* - Qwen 3.6 models: reasoning via chat_template_kwargs.enable_thinking;
|
|
48
|
-
* chatTemplateKwargs.preserve_thinking for multi-turn continuity.
|
|
49
|
-
* Returns `reasoning` field.
|
|
50
|
-
* - MiniMax M3 MXFP8: reasoning via chat_template_kwargs.enable_thinking;
|
|
51
|
-
* returns reasoning_content field.
|
|
16
|
+
* - DeepSeek V4 Pro: returns `reasoning` field.
|
|
17
|
+
* - DeepSeek V4 Flash: returns `reasoning` field.
|
|
18
|
+
* - GLM 5.2 FP8 / NVFP4: returns `reasoning` field.
|
|
19
|
+
* - Kimi K2.7 Code: returns `reasoning` field.
|
|
20
|
+
* - Qwen 3.6 models: returns `reasoning` field.
|
|
52
21
|
* - Llama 3.3 70B: not a reasoning model.
|
|
53
22
|
*
|
|
54
23
|
* Developer role is NOT supported by any of the chat templates on Makora's
|
|
@@ -141,11 +110,6 @@ interface PatchEntry {
|
|
|
141
110
|
|
|
142
111
|
type PatchMap = Record<string, PatchEntry>;
|
|
143
112
|
|
|
144
|
-
/** Type guard: non-null, non-array object. */
|
|
145
|
-
export function isObject(value: unknown): value is Record<string, unknown> {
|
|
146
|
-
return value !== null && typeof value === "object" && !Array.isArray(value);
|
|
147
|
-
}
|
|
148
|
-
|
|
149
113
|
// Patch Application
|
|
150
114
|
|
|
151
115
|
function applyPatch(model: JsonModel, patch: PatchEntry): JsonModel {
|
|
@@ -224,166 +188,11 @@ function buildModels(
|
|
|
224
188
|
const PROVIDER_ID = "makora";
|
|
225
189
|
const BASE_URL = "https://inference.makora.com/v1";
|
|
226
190
|
|
|
227
|
-
const DS_PRO_ID = "deepseek-ai/DeepSeek-V4-Pro";
|
|
228
|
-
const DS_FLASH_ID = "deepseek-ai/DeepSeek-V4-Flash";
|
|
229
|
-
const MINIMAX_M3_ID = "MiniMaxAI/MiniMax-M3-MXFP8";
|
|
230
|
-
|
|
231
|
-
const DS_VLLM_MODELS = new Set([DS_PRO_ID, DS_FLASH_ID]);
|
|
232
|
-
const ENABLE_THINKING_VLLM_MODELS = new Set([MINIMAX_M3_ID]);
|
|
233
|
-
|
|
234
|
-
/**
|
|
235
|
-
* Makora's GLM models, built from the same models list this extension
|
|
236
|
-
* registers. `before_provider_request` is GLOBAL in pi — it fires for every
|
|
237
|
-
* loaded provider's requests, not just makora's — and its event carries only
|
|
238
|
-
* `payload` (no `provider` field to gate on). So the GLM tool_calls strip MUST
|
|
239
|
-
* be scoped to makora's own GLM model ids. A loose `/glm/i` regex would also
|
|
240
|
-
* rewrite tool_calls for every sibling provider that serves any GLM model
|
|
241
|
-
* (baseten, io, lilac, neuralwatt, tensorix, fireworks, crofai, hypercharm,
|
|
242
|
-
* wafer, parasail, umans, opencode, ...), silently breaking their tool calling
|
|
243
|
-
* and clobbering the input their own before_provider_request handlers expect.
|
|
244
|
-
*
|
|
245
|
-
* Derived (not hardcoded) so it auto-tracks models.json refreshes — models.json
|
|
246
|
-
* is auto-generated by scripts/update-models.js. All makora models run on vLLM,
|
|
247
|
-
* so every GLM id here shares the ZAI chat-template `.items()` crash that
|
|
248
|
-
* stripGlmToolCalls works around.
|
|
249
|
-
*/
|
|
250
191
|
const allMakoraModels = buildModels(
|
|
251
192
|
modelsData as JsonModel[],
|
|
252
193
|
customModelsData as JsonModel[],
|
|
253
194
|
patchData as PatchMap,
|
|
254
195
|
);
|
|
255
|
-
const GLM_VLLM_MODELS = new Set(
|
|
256
|
-
allMakoraModels.filter((m) => /glm/i.test(m.id)).map((m) => m.id),
|
|
257
|
-
);
|
|
258
|
-
|
|
259
|
-
/** Whether `model` is one of makora's GLM models — the only ids the
|
|
260
|
-
* before_provider_request hook should strip tool_calls for. Exported so the
|
|
261
|
-
* scoping can be unit-tested in isolation. */
|
|
262
|
-
export function isMakoraGlmVllmModel(model: string): boolean {
|
|
263
|
-
return GLM_VLLM_MODELS.has(model);
|
|
264
|
-
}
|
|
265
|
-
|
|
266
|
-
/**
|
|
267
|
-
* Intercept the request payload for models that need vLLM-specific thinking
|
|
268
|
-
* param rewrites.
|
|
269
|
-
*
|
|
270
|
-
* pi's "deepseek" thinkingFormat sends `thinking: { type: "enabled" }` which
|
|
271
|
-
* is the official DeepSeek API format — but Makora's vLLM deployment ignores
|
|
272
|
-
* it. vLLM requires different params depending on the model:
|
|
273
|
-
* - DS V4 Pro: `chat_template_kwargs: { thinking: true }` + `reasoning_effort`
|
|
274
|
-
* - DS V4 Flash: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`
|
|
275
|
-
* + `reasoning_effort`. `include_reasoning` alone returns `reasoning: null`
|
|
276
|
-
* on this vLLM build — both params are required.
|
|
277
|
-
* - MiniMax M3: `chat_template_kwargs: { enable_thinking: true }` +
|
|
278
|
-
* `reasoning_effort`. Returns `reasoning_content` field.
|
|
279
|
-
*
|
|
280
|
-
* This hook rewrites the payload accordingly.
|
|
281
|
-
*/
|
|
282
|
-
function rewriteVllmPayload(payload: Record<string, unknown>): Record<string, unknown> {
|
|
283
|
-
const model = payload.model as string | undefined;
|
|
284
|
-
if (!model) return payload;
|
|
285
|
-
|
|
286
|
-
const p = { ...payload };
|
|
287
|
-
|
|
288
|
-
if (DS_VLLM_MODELS.has(model)) {
|
|
289
|
-
// Remove the DeepSeek API-style `thinking` param that vLLM ignores
|
|
290
|
-
delete p.thinking;
|
|
291
|
-
|
|
292
|
-
if (model === DS_PRO_ID) {
|
|
293
|
-
// DS Pro: chat_template_kwargs.thinking + reasoning_effort
|
|
294
|
-
const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
|
|
295
|
-
p.chat_template_kwargs = { ...ctq, thinking: true };
|
|
296
|
-
} else if (model === DS_FLASH_ID) {
|
|
297
|
-
// DS Flash: include_reasoning + chat_template_kwargs.thinking + reasoning_effort
|
|
298
|
-
// vLLM requires *both* include_reasoning and chat_template_kwargs.thinking:
|
|
299
|
-
// include_reasoning alone returns reasoning: null.
|
|
300
|
-
p.include_reasoning = true;
|
|
301
|
-
const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
|
|
302
|
-
p.chat_template_kwargs = { ...ctq, thinking: true };
|
|
303
|
-
}
|
|
304
|
-
} else if (ENABLE_THINKING_VLLM_MODELS.has(model)) {
|
|
305
|
-
// Models using chat_template_kwargs.enable_thinking (e.g. MiniMax M3)
|
|
306
|
-
delete p.thinking;
|
|
307
|
-
const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
|
|
308
|
-
p.chat_template_kwargs = { ...ctq, enable_thinking: true };
|
|
309
|
-
}
|
|
310
|
-
|
|
311
|
-
return p;
|
|
312
|
-
}
|
|
313
|
-
|
|
314
|
-
/**
|
|
315
|
-
* GLM models on Makora's vLLM crash with a leaked Python AttributeError
|
|
316
|
-
* (`'list object' has no attribute 'items'` or `'str object' has no attribute 'items'`)
|
|
317
|
-
* when any assistant message in the request contains a `tool_calls` field.
|
|
318
|
-
* The ZAI/vLLM chat template calls `.items()` on the tool_calls list (or on the
|
|
319
|
-
* JSON-string `arguments` field), which raises AttributeError and leaks into the
|
|
320
|
-
* HTTP 400 response body.
|
|
321
|
-
*
|
|
322
|
-
* Fix: for makora's GLM models only (see GLM_VLLM_MODELS / isMakoraGlmVllmModel),
|
|
323
|
-
* strip `tool_calls` from assistant messages in the before_provider_request hook
|
|
324
|
-
* and convert them back to GLM's native
|
|
325
|
-
* `<tool_call>` XML text in `content`. The model natively understands this
|
|
326
|
-
* format in conversation history, and the `role: "tool"` result messages that
|
|
327
|
-
* follow are rendered fine by the chat template's tool-observation branch.
|
|
328
|
-
*
|
|
329
|
-
* If upstream fixes both the streaming parser and the 500/400 crash, this
|
|
330
|
-
* transform becomes a harmless no-op (the XML text is still valid GLM input).
|
|
331
|
-
*/
|
|
332
|
-
|
|
333
|
-
export function toolCallToGlmXml(tc: Record<string, unknown>): string {
|
|
334
|
-
const fn = (isObject(tc.function) ? tc.function : {}) as Record<string, unknown>;
|
|
335
|
-
const name = typeof fn.name === "string" ? fn.name : "";
|
|
336
|
-
const argsStr = typeof fn.arguments === "string" ? fn.arguments : "{}";
|
|
337
|
-
let args: Record<string, unknown>;
|
|
338
|
-
try {
|
|
339
|
-
args = JSON.parse(argsStr) as Record<string, unknown>;
|
|
340
|
-
} catch {
|
|
341
|
-
args = {};
|
|
342
|
-
}
|
|
343
|
-
if (!isObject(args)) args = {};
|
|
344
|
-
const argLines = Object.entries(args).map(
|
|
345
|
-
([key, value]) =>
|
|
346
|
-
`<arg_key>${key}</arg_key>\n<arg_value>${
|
|
347
|
-
typeof value === "string" ? value : JSON.stringify(value)
|
|
348
|
-
}</arg_value>`,
|
|
349
|
-
);
|
|
350
|
-
return `<tool_call>${name}\n${argLines.join("\n")}\n</tool_call>`;
|
|
351
|
-
}
|
|
352
|
-
|
|
353
|
-
export function stripGlmToolCalls(payload: Record<string, unknown>): Record<string, unknown> {
|
|
354
|
-
const messages = payload.messages;
|
|
355
|
-
if (!Array.isArray(messages)) return payload;
|
|
356
|
-
|
|
357
|
-
let modified = false;
|
|
358
|
-
const newMessages = messages.map((msg) => {
|
|
359
|
-
if (!isObject(msg)) return msg;
|
|
360
|
-
if (msg.role !== "assistant") return msg;
|
|
361
|
-
|
|
362
|
-
const toolCalls = msg.tool_calls;
|
|
363
|
-
if (!Array.isArray(toolCalls) || toolCalls.length === 0) return msg;
|
|
364
|
-
|
|
365
|
-
const xmlBlocks = toolCalls
|
|
366
|
-
.filter((tc): tc is Record<string, unknown> => isObject(tc))
|
|
367
|
-
.map(toolCallToGlmXml);
|
|
368
|
-
if (xmlBlocks.length === 0) return msg;
|
|
369
|
-
|
|
370
|
-
const toolCallText = xmlBlocks.join("\n");
|
|
371
|
-
// pi's openai-completions provider always serializes assistant content as a
|
|
372
|
-
// plain string, but guard against null/array content for robustness.
|
|
373
|
-
const existingContent = typeof msg.content === "string" ? msg.content : "";
|
|
374
|
-
const newContent = existingContent ? `${existingContent}\n${toolCallText}` : toolCallText;
|
|
375
|
-
|
|
376
|
-
modified = true;
|
|
377
|
-
const rest: Record<string, unknown> = {};
|
|
378
|
-
for (const [k, v] of Object.entries(msg)) {
|
|
379
|
-
if (k !== "tool_calls") rest[k] = v;
|
|
380
|
-
}
|
|
381
|
-
return { ...rest, content: newContent };
|
|
382
|
-
});
|
|
383
|
-
|
|
384
|
-
if (!modified) return payload;
|
|
385
|
-
return { ...payload, messages: newMessages };
|
|
386
|
-
}
|
|
387
196
|
|
|
388
197
|
export default function (pi: ExtensionAPI) {
|
|
389
198
|
const models = allMakoraModels;
|
|
@@ -396,16 +205,4 @@ export default function (pi: ExtensionAPI) {
|
|
|
396
205
|
api: "openai-completions",
|
|
397
206
|
models,
|
|
398
207
|
});
|
|
399
|
-
|
|
400
|
-
pi.on("before_provider_request", (event) => {
|
|
401
|
-
const payload = event.payload as Record<string, unknown> | undefined;
|
|
402
|
-
if (!payload || typeof payload.model !== "string") return;
|
|
403
|
-
|
|
404
|
-
let result = rewriteVllmPayload(payload);
|
|
405
|
-
if (isMakoraGlmVllmModel(payload.model)) {
|
|
406
|
-
result = stripGlmToolCalls(result);
|
|
407
|
-
}
|
|
408
|
-
return result;
|
|
409
|
-
});
|
|
410
208
|
}
|
|
411
|
-
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-makora-provider",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.4.0",
|
|
4
4
|
"description": "Makora provider extension for pi - Access DeepSeek V4, GLM 5.2, Kimi K2.7 Code, Llama 3.3, Qwen 3.6, and more through the Makora inference API",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.ts",
|
package/patch.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"deepseek-ai/DeepSeek-V4-Flash": {
|
|
3
3
|
"reasoning": true,
|
|
4
|
-
"notes": "
|
|
4
|
+
"notes": "returns `reasoning` field",
|
|
5
5
|
"thinkingLevelMap": {
|
|
6
6
|
"minimal": null,
|
|
7
7
|
"low": null,
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
},
|
|
17
17
|
"deepseek-ai/DeepSeek-V4-Pro": {
|
|
18
18
|
"reasoning": true,
|
|
19
|
-
"notes": "
|
|
19
|
+
"notes": "returns `reasoning` field",
|
|
20
20
|
"thinkingLevelMap": {
|
|
21
21
|
"minimal": null,
|
|
22
22
|
"low": null,
|
|
@@ -30,37 +30,9 @@
|
|
|
30
30
|
"requiresReasoningContentOnAssistantMessages": true
|
|
31
31
|
}
|
|
32
32
|
},
|
|
33
|
-
"nvidia/Kimi-K2.6-NVFP4": {
|
|
34
|
-
"reasoning": true,
|
|
35
|
-
"input": [
|
|
36
|
-
"text",
|
|
37
|
-
"image"
|
|
38
|
-
],
|
|
39
|
-
"notes": "Reasoning on by default; client-side tool call parsing (vLLM streaming parser bypass)",
|
|
40
|
-
"thinkingLevelMap": {
|
|
41
|
-
"minimal": "low",
|
|
42
|
-
"xhigh": "high"
|
|
43
|
-
},
|
|
44
|
-
"compat": {
|
|
45
|
-
"thinkingFormat": "qwen-chat-template",
|
|
46
|
-
"supportsReasoningEffort": true
|
|
47
|
-
}
|
|
48
|
-
},
|
|
49
|
-
"openai/gpt-oss-120b": {
|
|
50
|
-
"reasoning": true,
|
|
51
|
-
"notes": "Reasoning always on",
|
|
52
|
-
"thinkingLevelMap": {
|
|
53
|
-
"minimal": "low",
|
|
54
|
-
"xhigh": "high"
|
|
55
|
-
},
|
|
56
|
-
"compat": {
|
|
57
|
-
"thinkingFormat": "qwen-chat-template",
|
|
58
|
-
"supportsReasoningEffort": true
|
|
59
|
-
}
|
|
60
|
-
},
|
|
61
33
|
"unsloth/Qwen3.6-27B-NVFP4": {
|
|
62
34
|
"reasoning": true,
|
|
63
|
-
"notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
35
|
+
"notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
|
|
64
36
|
"thinkingLevelMap": {
|
|
65
37
|
"minimal": "low",
|
|
66
38
|
"xhigh": "high"
|
|
@@ -75,7 +47,7 @@
|
|
|
75
47
|
},
|
|
76
48
|
"unsloth/Qwen3.6-35B-A3B-NVFP4": {
|
|
77
49
|
"reasoning": true,
|
|
78
|
-
"notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
50
|
+
"notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
|
|
79
51
|
"thinkingLevelMap": {
|
|
80
52
|
"minimal": "low",
|
|
81
53
|
"xhigh": "high"
|
|
@@ -88,33 +60,13 @@
|
|
|
88
60
|
}
|
|
89
61
|
}
|
|
90
62
|
},
|
|
91
|
-
"MiniMaxAI/MiniMax-M3-MXFP8": {
|
|
92
|
-
"reasoning": true,
|
|
93
|
-
"input": [
|
|
94
|
-
"text",
|
|
95
|
-
"image"
|
|
96
|
-
],
|
|
97
|
-
"notes": "Reasoning via `chat_template_kwargs.enable_thinking`; returns `reasoning_content` field",
|
|
98
|
-
"thinkingLevelMap": {
|
|
99
|
-
"minimal": null,
|
|
100
|
-
"low": null,
|
|
101
|
-
"medium": null,
|
|
102
|
-
"high": "high",
|
|
103
|
-
"xhigh": "max"
|
|
104
|
-
},
|
|
105
|
-
"compat": {
|
|
106
|
-
"thinkingFormat": "deepseek",
|
|
107
|
-
"supportsReasoningEffort": true,
|
|
108
|
-
"requiresReasoningContentOnAssistantMessages": true
|
|
109
|
-
}
|
|
110
|
-
},
|
|
111
63
|
"moonshotai/Kimi-K2.7-Code": {
|
|
112
64
|
"reasoning": true,
|
|
113
65
|
"input": [
|
|
114
66
|
"text",
|
|
115
67
|
"image"
|
|
116
68
|
],
|
|
117
|
-
"notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field
|
|
69
|
+
"notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
|
|
118
70
|
"thinkingLevelMap": {
|
|
119
71
|
"minimal": "low",
|
|
120
72
|
"xhigh": "high"
|
|
@@ -127,20 +79,6 @@
|
|
|
127
79
|
}
|
|
128
80
|
}
|
|
129
81
|
},
|
|
130
|
-
"zai-org/GLM-5.1-FP8": {
|
|
131
|
-
"contextWindow": 200000,
|
|
132
|
-
"reasoning": true,
|
|
133
|
-
"notes": "`enable_thinking` via `qwen-chat-template`; returns `reasoning_content` field; client-side tool call parsing (vLLM streaming parser bypass)",
|
|
134
|
-
"thinkingLevelMap": {
|
|
135
|
-
"minimal": "low",
|
|
136
|
-
"xhigh": "high"
|
|
137
|
-
},
|
|
138
|
-
"compat": {
|
|
139
|
-
"thinkingFormat": "qwen-chat-template",
|
|
140
|
-
"supportsReasoningEffort": true,
|
|
141
|
-
"zaiToolStream": true
|
|
142
|
-
}
|
|
143
|
-
},
|
|
144
82
|
"zai-org/GLM-5.2-FP8": {
|
|
145
83
|
"reasoning": true,
|
|
146
84
|
"notes": "`enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field",
|