pi-makora-provider 1.3.0 → 1.5.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Open-weight models through [Makora](https://inference.makora.com)**
 
-_DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 — with client-side tool call repair and preserved-thinking flags for [pi](https://github.com/earendil-works/pi-coding-agent)._
+_DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 for [pi](https://github.com/earendil-works/pi-coding-agent)._
 
 [![pi extension](https://img.shields.io/badge/pi-extension-blueviolet)](https://github.com/earendil-works/pi-coding-agent)
 [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
@@ -18,15 +18,16 @@ _DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 — with client-side tool call r
 <!-- MODELS_TABLE_START -->
 | Model | ID | Reasoning | Notes |
 |-------|----|-----------|-------|
-| DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes | `include_reasoning` + `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field |
-| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field |
+| DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes | returns `reasoning` field |
+| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | returns `reasoning` field |
+| Gemma 4 26B A4B | `google/gemma-4-26B-A4B` | No |  |
 | GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field |
 | GLM 5.2 NVFP4 | `zai-org/GLM-5.2-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field |
-| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
+| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
 | Llama 3.3 70B FP8 | `amd/Llama-3.3-70B-Instruct-FP8-KV` | No |  |
 | Llama 3.3 70B Instruct | `meta-llama/Llama-3.3-70B-Instruct` | No |  |
-| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
-| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass) |
+| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
+| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
 <!-- MODELS_TABLE_END -->
 
 ## Installation
@@ -157,23 +158,32 @@ Do **not** edit `models.json` directly — it is auto-generated from the API. To
 - All models are hosted on vLLM
 - The `developer` role is not supported (prompts are silently dropped); `supportsDeveloperRole` is set to `false` for all models
 
-## vLLM Caveats
-
-These issues are common to all vLLM-hosted providers and affect Makora models:
-
-- **GLM tool calls — follow-up `tool_calls` crash workaround**: the ZAI chat template calls `.items()` on any assistant message that carries a `tool_calls` field (on the `tool_calls` list or on its JSON-string `arguments` field), raising a leaked Python `AttributeError` that vLLM returns as HTTP 400 (`'list object' has no attribute 'items'` / `'str object' has no attribute 'items'`). The `before_provider_request` hook (`stripGlmToolCalls`) strips `tool_calls` from assistant messages and converts them to GLM-native `<tool_call>` XML text in `content` — which the model natively understands in conversation history — before the follow-up request is sent. The hook is scoped to Makora's own GLM models via an exact model-id match (`isMakoraGlmVllmModel`): `before_provider_request` is global in pi (it fires for every loaded provider's requests and the event carries no `provider` field), so gating by model id is what keeps it from touching GLM models served by other providers. If upstream fixes the `.items()` crash, this transform is a harmless no-op (the XML text is valid GLM input either way).
-
-- **Kimi K2.7 Code + Qwen 3.6 tool calling**: vLLM's server-side streaming tool-call handling is broken or missing for these models — Makora's vLLM is missing both `--enable-auto-tool-choice` and `--tool-call-parser` for them, so they emit tool calls as raw text tokens rather than structured `tool_call` deltas.
-
-    - **Kimi K2.7 Code**: Uses `<|tool_call_begin|>...<|tool_call_end|>` tokens.
-    - **Qwen 3.6**: Uses hermes-style `<function=...>` XML, sometimes with `█` delimiters.
-
-- **GLM 5.2 CoT leak**: On some vLLM builds, disabling reasoning may still leak chain-of-thought into `content` terminated by a ``` marker. See [vllm-project/vllm#31319](https://github.com/vllm-project/vllm/issues/31319).
-
-- **DeepSeek V4 reasoning**: The official DeepSeek API uses `thinking: { type: "enabled" }` which Makora's vLLM silently ignores. The `before_provider_request` hook rewrites the payload to use vLLM-native params instead:
-  - **DS V4 Pro**: `chat_template_kwargs: { thinking: true }`. Returns `reasoning` (vLLM reasoning parser; the official DeepSeek API returns `reasoning_content`, but Makora's vLLM build returns `reasoning`).
-  - **DS V4 Flash**: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`. `include_reasoning` alone returns `reasoning: null` on this vLLM build — both params are required. Returns `reasoning`.
-- **Preserved thinking (multi-turn reasoning continuity)**: every reasoning model on Makora returns reasoning in the `reasoning` field (not `reasoning_content`). Pi sends prior assistant reasoning back on follow-up turns; Makora's vLLM chat templates accept it under either `reasoning` or `reasoning_content`. For models whose chat template defines an explicit preserved-thinking flag, the flag is sent via `compat.chatTemplateKwargs`:
-  - **Kimi K2.7 Code**: `preserve_thinking: true` (the model forces thinking-only mode with preserve_thinking on, per the vLLM recipe).
-  - **Qwen 3.6 27B / 35B**: `preserve_thinking: true` (Qwen 3.6 ships with preserve_thinking).
-  - **GLM 5.2**: no extra flag — the vLLM GLM-5.2 recipe uses only `enable_thinking` + `reasoning_effort`; preserved thinking works without a dedicated flag. Effort honors only `high` and `max` (lower pi levels resolve to the default `max`).
+## Death-Loop Guard
+
+GLM 5.2 (NVFP4 / FP8) occasionally degenerates into an unbroken `!` repetition
+loop (`!!!!...`) that eats the whole response. This extension ships a guard
+that watches the streamed assistant output (both the visible answer and the
+reasoning trace) and, when it detects a long run of `!` characters, **aborts
+the runaway generation and resumes the agentic loop invisibly** — no new user
+message is injected, using the
+[pi-invisible-continue](https://github.com/monotykamary/pi-invisible-continue)
+pattern (`agent.prompt([])`).
+
+On abort, pi finalizes the in-flight assistant message **with its accumulated
+`!!!` content** into the transcript, so the guard drops that partial message
+before resuming — otherwise the model would just re-read its own `!!!` and loop
+again. The recovery is bounded per prompt (default 3 attempts) to avoid
+abort/continue thrash.
+
+The guard is scoped to the Makora GLM 5.2 family by default and is tunable via
+constants at the top of [`death-loop-guard.ts`](./death-loop-guard.ts):
+
+| Constant | Default | Meaning |
+|---|---|---|
+| `GUARDED_MODEL_IDS` | `zai-org/GLM-5.2-NVFP4`, `zai-org/GLM-5.2-FP8` | Which model IDs to guard. Add `'*'` to guard every Makora model. |
+| `BANG_THRESHOLD` | `40` | Consecutive `!` characters that trip the guard. 40 is far above anything normal prose/code produces. |
+| `MAX_RECOVERS_PER_RUN` | `3` | Max invisible recoveries per user prompt before the guard stops intervening. |
+
+It is only active for the `makora` provider, so it never interferes when you
+switch to another provider. If you also run `pi-invisible-continue`, the two
+coexist — both chain the `Agent.prototype.subscribe` patch.

package/death-loop-guard.ts

@@ -0,0 +1,238 @@
+/**
+ * Death-loop guard for Makora reasoning models.
+ *
+ * Some Makora models (notably GLM 5.2 NVFP4 / FP8) occasionally fall into a
+ * degenerate repetition loop, emitting an unbroken run of '!' characters
+ * (e.g. "!!!!...") that consumes the whole response. This guard watches the
+ * streamed assistant output (both the visible answer and the reasoning
+ * trace); when that run is detected it aborts the runaway generation, drops
+ * the partial (toxic) assistant message from the transcript, and resumes the
+ * agentic loop invisibly via agent.prompt([]) — the same pattern
+ * pi-invisible-continue uses, so no new user message pollutes the context.
+ *
+ * Why trim the aborted message: on abort, pi finalizes the in-flight
+ * assistant message WITH its accumulated '!!!' content and stopReason
+ * "aborted" into the transcript. Resuming from that context would re-feed
+ * the toxic text to the model and likely re-trigger the loop. Dropping the
+ * last (aborted) assistant message leaves the context ending at the prior
+ * user/toolResult message — a clean continuation point.
+ *
+ * Module resolution: @earendil-works/pi-agent-core is a devDependency only
+ * (types + test resolution). At runtime pi's extension loader aliases that
+ * specifier to its bundled copy, so the Agent class patched below is the
+ * SAME class AgentSession uses. A static import is required — jiti's alias
+ * applies to static imports (which it rewrites to its own resolver) but not
+ * to native dynamic import() calls.
+ */
+
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import { Agent } from "@earendil-works/pi-agent-core";
+
+const PROVIDER_ID = "makora";
+
+/** Makora model IDs to guard. Add ids to widen coverage, or include "*"
+ *  to guard every Makora model. Defaults to the GLM 5.2 family (the known
+ *  offender). Kept exported so tests and downstream forks can introspect. */
+export const GUARDED_MODEL_IDS = new Set<string>([
+  "zai-org/GLM-5.2-NVFP4",
+  "zai-org/GLM-5.2-FP8",
+]);
+
+/** Trip after this many consecutive '!' characters in the streamed answer.
+ *  40 is far above anything normal prose or code produces. */
+export const BANG_THRESHOLD = 40;
+
+/** Max invisible recoveries per user prompt, to bound abort/continue thrash. */
+export const MAX_RECOVERS_PER_RUN = 3;
+
+export interface GuardedMessage {
+  role: string;
+  stopReason?: string;
+  content?: unknown;
+}
+
+export interface GuardedAgent {
+  abort(): void;
+  waitForIdle(): Promise<void>;
+  prompt(input: unknown[] | string): Promise<void>;
+  state: { messages: GuardedMessage[] };
+}
+
+let _agent: GuardedAgent | null = null;
+
+/** Mutex held only across the abort+trim critical section — NOT across the
+ *  resumed prompt([]) run, so the resumed stream stays monitored. */
+let _recovering = false;
+
+/** Trailing '!' run length in the current text block. */
+let _trailingBangs = 0;
+
+/** Latch: already tripped for the current assistant message. */
+let _tripped = false;
+
+/** Recoveries performed in the current user-initiated agent run. */
+let _recoversThisRun = 0;
+
+export function isGuardedModel(
+  model: { provider?: string; id?: string } | undefined | null,
+): boolean {
+  if (!model) return false;
+  if (model.provider !== PROVIDER_ID) return false;
+  if (GUARDED_MODEL_IDS.has("*")) return true;
+  return model.id != null && GUARDED_MODEL_IDS.has(model.id);
+}
+
+/** Update the trailing-'!' run length given a new text delta. O(len(delta)).
+ *  Trailing run depends only on the delta's suffix: if the delta contains any
+ *  non-'!' char, the prior run is cut off at that char; if the delta is all
+ *  '!', it extends the prior run. */
+export function nextTrailingBangs(prev: number, delta: string): number {
+  const len = delta.length;
+  if (len === 0) return prev;
+  let i = len - 1;
+  while (i >= 0 && delta.charCodeAt(i) === 0x21) i--; // '!' === 0x21
+  const trailingInDelta = len - 1 - i;
+  return i >= 0 ? trailingInDelta : prev + len;
+}
+
+export function extractText(content: unknown): string {
+  if (typeof content === "string") return content;
+  if (Array.isArray(content)) {
+    let out = "";
+    for (const block of content) {
+      if (
+        block &&
+        typeof block === "object" &&
+        (block as { type?: string }).type === "text" &&
+        typeof (block as { text?: unknown }).text === "string"
+      ) {
+        out += (block as { text: string }).text;
+      }
+    }
+    return out;
+  }
+  return "";
+}
+
+/** Trailing '!' run length of a finalized message's text content. */
+export function messageTrailingBangs(msg: GuardedMessage): number {
+  const text = extractText(msg.content);
+  let i = text.length - 1;
+  while (i >= 0 && text.charCodeAt(i) === 0x21) i--;
+  return text.length - 1 - i;
+}
+
+async function recover(agent: GuardedAgent): Promise<void> {
+  _recovering = true;
+  try {
+    agent.abort();
+    await agent.waitForIdle();
+    const msgs = agent.state.messages;
+    const last = msgs[msgs.length - 1];
+    if (
+      last &&
+      last.role === "assistant" &&
+      (last.stopReason === "aborted" || messageTrailingBangs(last) >= BANG_THRESHOLD)
+    ) {
+      // Drop the toxic partial so it isn't re-fed to the model on resume.
+      // The state setter copies the array, leaving a clean transcript that
+      // ends at the prior user/toolResult message.
+      agent.state.messages = msgs.slice(0, -1);
+    }
+  } finally {
+    // Release before resuming so the resumed stream stays monitored.
+    _recovering = false;
+  }
+  try {
+    // Invisible continue: fresh agent loop, no new message injected.
+    await agent.prompt([]);
+  } catch {
+    // "Agent is already processing" or other transient error — best effort.
+  }
+}
+
+export function registerDeathLoopGuard(pi: ExtensionAPI): void {
+  // Capture the live Agent instance by chaining Agent.prototype.subscribe.
+  // subscribe() fires when AgentSession attaches — on every fresh session
+  // and every resume — so _agent always points at the active Agent. Chain
+  // any prior patch (e.g. pi-invisible-continue) so both extensions coexist.
+  const proto = Agent.prototype as unknown as {
+    subscribe: (this: GuardedAgent, ...args: unknown[]) => unknown;
+  };
+  const origSubscribe = proto.subscribe;
+  proto.subscribe = function (this: GuardedAgent, ...args: unknown[]) {
+    _agent = this;
+    return origSubscribe.apply(this, args);
+  };
+
+  pi.on("session_start", () => {
+    _recovering = false;
+    _trailingBangs = 0;
+    _tripped = false;
+    _recoversThisRun = 0;
+  });
+
+  // before_agent_start fires only for user prompts (the AgentSession path),
+  // not for the recovery's direct agent.prompt([]), so the recovery cap is
+  // bounded per user prompt instead of reset on every recovery continuation.
+  pi.on("before_agent_start", () => {
+    _recovering = false;
+    _trailingBangs = 0;
+    _tripped = false;
+    _recoversThisRun = 0;
+  });
+
+  pi.on("message_start", (event) => {
+    if (event.message?.role === "assistant") {
+      _trailingBangs = 0;
+      _tripped = false;
+    }
+  });
+
+  pi.on("message_update", (event, ctx) => {
+    if (_recovering || _tripped) return;
+    const ame = event.assistantMessageEvent;
+    if (ame.type === "text_start" || ame.type === "thinking_start") {
+      // New content block (answer or reasoning) — trailing run starts fresh.
+      _trailingBangs = 0;
+      return;
+    }
+    // Watch both the visible answer and the reasoning trace: GLM 5.2 is a
+    // reasoning model, and the loop can surface in either.
+    if (ame.type !== "text_delta" && ame.type !== "thinking_delta") return;
+    if (!isGuardedModel(ctx.model)) return;
+
+    _trailingBangs = nextTrailingBangs(_trailingBangs, ame.delta);
+    if (_trailingBangs < BANG_THRESHOLD) return;
+
+    if (_recoversThisRun >= MAX_RECOVERS_PER_RUN) {
+      _tripped = true;
+      ctx.ui.notify(
+        "Makora death-loop guard: runaway '!' output detected, but the " +
+          "recovery limit for this prompt was reached — stopping " +
+          "intervention. Try /continue or rephrase.",
+        "warning",
+      );
+      return;
+    }
+    _tripped = true;
+    _recoversThisRun++;
+    const agent = _agent;
+    if (!agent) {
+      ctx.ui.notify(
+        "Makora death-loop guard: runaway '!' output detected but the Agent " +
+          "instance was not captured; cannot recover automatically.",
+        "warning",
+      );
+      return;
+    }
+    ctx.ui.notify(
+      `Makora death-loop guard: aborting runaway '!' output and resuming ` +
+        `(${_recoversThisRun}/${MAX_RECOVERS_PER_RUN}).`,
+      "warning",
+    );
+    // Detach: the handler must return so the run can unwind to idle before
+    // recover() awaits waitForIdle() and calls prompt([]).
+    void recover(agent);
+  });
+}

package/index.ts

@@ -13,32 +13,19 @@
  * Model resolution strategy: static models.json merged with custom-models.json
  *
  * Reasoning notes:
- *   - DeepSeek V4 Pro: reasoning via chat_template_kwargs.thinking on vLLM.
- *     pi sends thinking: { type } via the "deepseek" thinkingFormat, but vLLM
- *     ignores that — the before_provider_request hook rewrites the payload to
- *     use chat_template_kwargs: { thinking: true } instead.
- *     Returns reasoning field (vLLM reasoning parser; not reasoning_content).
- *   - DeepSeek V4 Flash: reasoning via include_reasoning +
- *     chat_template_kwargs.thinking on vLLM.
- *     The before_provider_request hook rewrites the payload to replace
- *     thinking: { type } with include_reasoning: true +
- *     chat_template_kwargs: { thinking: true }.
- *     include_reasoning alone returns reasoning: null on this vLLM build.
- *     Returns reasoning field.
- *   - GLM 5.2 FP8 / NVFP4: reasoning via chat_template_kwargs.enable_thinking.
- *     Effort uses vLLM's reasoning_effort field (only `high` and `max` are
- *     distinct levels per the vLLM GLM-5.2 recipe; lower pi levels resolve to
- *     the default). Thinking levels aligned with the neuralwatt provider's
- *     GLM 5.2 configuration and mapped through pi's qwen-chat-template
- *     thinkingFormat. Returns `reasoning` field.
- *   - Kimi K2.7 Code: reasoning always on (thinking-only model);
- *     chatTemplateKwargs.preserve_thinking forces multi-turn reasoning
- *     continuity. Returns `reasoning` field. Can be toggled via enable_thinking.
- *   - Qwen 3.6 models: reasoning via chat_template_kwargs.enable_thinking;
- *     chatTemplateKwargs.preserve_thinking for multi-turn continuity.
- *     Returns `reasoning` field.
+ *   - DeepSeek V4 Pro: returns `reasoning` field.
+ *   - DeepSeek V4 Flash: returns `reasoning` field.
+ *   - GLM 5.2 FP8 / NVFP4: returns `reasoning` field.
+ *   - Kimi K2.7 Code: returns `reasoning` field.
+ *   - Qwen 3.6 models: returns `reasoning` field.
  *   - Llama 3.3 70B: not a reasoning model.
  *
+ * A death-loop guard (see ./death-loop-guard.ts) is registered alongside the
+ * provider. It watches the assistant text stream on the GLM 5.2 family and,
+ * if the model falls into an unbroken '!' repetition loop, aborts the runaway
+ * generation and resumes the agentic loop invisibly via agent.prompt([]) (the
+ * pi-invisible-continue pattern) — no new user message is injected.
+ *
  * Developer role is NOT supported by any of the chat templates on Makora's
  * vLLM deployment (prompts with role: "developer" are silently dropped).
  * supportsDeveloperRole is set to false for all models.
@@ -61,6 +48,7 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import modelsData from "./models.json" with { type: "json" };
 import customModelsData from "./custom-models.json" with { type: "json" };
 import patchData from "./patch.json" with { type: "json" };
+import { registerDeathLoopGuard } from "./death-loop-guard.js";
 
 // Types
 
@@ -129,11 +117,6 @@ interface PatchEntry {
 
 type PatchMap = Record<string, PatchEntry>;
 
-/** Type guard: non-null, non-array object. */
-export function isObject(value: unknown): value is Record<string, unknown> {
-  return value !== null && typeof value === "object" && !Array.isArray(value);
-}
-
 // Patch Application
 
 function applyPatch(model: JsonModel, patch: PatchEntry): JsonModel {
@@ -212,157 +195,11 @@ function buildModels(
 const PROVIDER_ID = "makora";
 const BASE_URL = "https://inference.makora.com/v1";
 
-const DS_PRO_ID = "deepseek-ai/DeepSeek-V4-Pro";
-const DS_FLASH_ID = "deepseek-ai/DeepSeek-V4-Flash";
-
-const DS_VLLM_MODELS = new Set([DS_PRO_ID, DS_FLASH_ID]);
-
-/**
- * Makora's GLM models, built from the same models list this extension
- * registers. `before_provider_request` is GLOBAL in pi — it fires for every
- * loaded provider's requests, not just makora's — and its event carries only
- * `payload` (no `provider` field to gate on). So the GLM tool_calls strip MUST
- * be scoped to makora's own GLM model ids. A loose `/glm/i` regex would also
- * rewrite tool_calls for every sibling provider that serves any GLM model
- * (baseten, io, lilac, neuralwatt, tensorix, fireworks, crofai, hypercharm,
- * wafer, parasail, umans, opencode, ...), silently breaking their tool calling
- * and clobbering the input their own before_provider_request handlers expect.
- *
- * Derived (not hardcoded) so it auto-tracks models.json refreshes — models.json
- * is auto-generated by scripts/update-models.js. All makora models run on vLLM,
- * so every GLM id here shares the ZAI chat-template `.items()` crash that
- * stripGlmToolCalls works around.
- */
 const allMakoraModels = buildModels(
   modelsData as JsonModel[],
   customModelsData as JsonModel[],
   patchData as PatchMap,
 );
-const GLM_VLLM_MODELS = new Set(
-  allMakoraModels.filter((m) => /glm/i.test(m.id)).map((m) => m.id),
-);
-
-/** Whether `model` is one of makora's GLM models — the only ids the
- *  before_provider_request hook should strip tool_calls for. Exported so the
- *  scoping can be unit-tested in isolation. */
-export function isMakoraGlmVllmModel(model: string): boolean {
-  return GLM_VLLM_MODELS.has(model);
-}
-
-/**
- * Intercept the request payload for models that need vLLM-specific thinking
- * param rewrites.
- *
- * pi's "deepseek" thinkingFormat sends `thinking: { type: "enabled" }` which
- * is the official DeepSeek API format — but Makora's vLLM deployment ignores
- * it. vLLM requires different params depending on the model:
- *   - DS V4 Pro:  `chat_template_kwargs: { thinking: true }` + `reasoning_effort`
- *   - DS V4 Flash: `include_reasoning: true` + `chat_template_kwargs: { thinking: true }`
- *     + `reasoning_effort`. `include_reasoning` alone returns `reasoning: null`
- *     on this vLLM build — both params are required.
- *
- * This hook rewrites the payload accordingly.
- */
-function rewriteVllmPayload(payload: Record<string, unknown>): Record<string, unknown> {
-  const model = payload.model as string | undefined;
-  if (!model) return payload;
-
-  const p = { ...payload };
-
-  if (DS_VLLM_MODELS.has(model)) {
-    // Remove the DeepSeek API-style `thinking` param that vLLM ignores
-    delete p.thinking;
-
-    if (model === DS_PRO_ID) {
-      // DS Pro: chat_template_kwargs.thinking + reasoning_effort
-      const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
-      p.chat_template_kwargs = { ...ctq, thinking: true };
-    } else if (model === DS_FLASH_ID) {
-      // DS Flash: include_reasoning + chat_template_kwargs.thinking + reasoning_effort
-      // vLLM requires *both* include_reasoning and chat_template_kwargs.thinking:
-      // include_reasoning alone returns reasoning: null.
-      p.include_reasoning = true;
-      const ctq = (p.chat_template_kwargs as Record<string, unknown>) ?? {};
-      p.chat_template_kwargs = { ...ctq, thinking: true };
-    }
-  }
-
-  return p;
-}
-
-/**
- * GLM models on Makora's vLLM crash with a leaked Python AttributeError
- * (`'list object' has no attribute 'items'` or `'str object' has no attribute 'items'`)
- * when any assistant message in the request contains a `tool_calls` field.
- * The ZAI/vLLM chat template calls `.items()` on the tool_calls list (or on the
- * JSON-string `arguments` field), which raises AttributeError and leaks into the
- * HTTP 400 response body.
- *
- * Fix: for makora's GLM models only (see GLM_VLLM_MODELS / isMakoraGlmVllmModel),
- * strip `tool_calls` from assistant messages in the before_provider_request hook
- * and convert them back to GLM's native
- * `<tool_call>` XML text in `content`. The model natively understands this
- * format in conversation history, and the `role: "tool"` result messages that
- * follow are rendered fine by the chat template's tool-observation branch.
- *
- * If upstream fixes both the streaming parser and the 500/400 crash, this
- * transform becomes a harmless no-op (the XML text is still valid GLM input).
- */
-
-export function toolCallToGlmXml(tc: Record<string, unknown>): string {
-  const fn = (isObject(tc.function) ? tc.function : {}) as Record<string, unknown>;
-  const name = typeof fn.name === "string" ? fn.name : "";
-  const argsStr = typeof fn.arguments === "string" ? fn.arguments : "{}";
-  let args: Record<string, unknown>;
-  try {
-    args = JSON.parse(argsStr) as Record<string, unknown>;
-  } catch {
-    args = {};
-  }
-  if (!isObject(args)) args = {};
-  const argLines = Object.entries(args).map(
-    ([key, value]) =>
-      `<arg_key>${key}</arg_key>\n<arg_value>${
-        typeof value === "string" ? value : JSON.stringify(value)
-      }</arg_value>`,
-  );
-  return `<tool_call>${name}\n${argLines.join("\n")}\n</tool_call>`;
-}
-
-export function stripGlmToolCalls(payload: Record<string, unknown>): Record<string, unknown> {
-  const messages = payload.messages;
-  if (!Array.isArray(messages)) return payload;
-
-  let modified = false;
-  const newMessages = messages.map((msg) => {
-    if (!isObject(msg)) return msg;
-    if (msg.role !== "assistant") return msg;
-
-    const toolCalls = msg.tool_calls;
-    if (!Array.isArray(toolCalls) || toolCalls.length === 0) return msg;
-
-    const xmlBlocks = toolCalls
-      .filter((tc): tc is Record<string, unknown> => isObject(tc))
-      .map(toolCallToGlmXml);
-    if (xmlBlocks.length === 0) return msg;
-
-    const toolCallText = xmlBlocks.join("\n");
-    // pi's openai-completions provider always serializes assistant content as a
-    // plain string, but guard against null/array content for robustness.
-    const existingContent = typeof msg.content === "string" ? msg.content : "";
-    const newContent = existingContent ? `${existingContent}\n${toolCallText}` : toolCallText;
-
-    modified = true;
-    const rest: Record<string, unknown> = {};
-    for (const [k, v] of Object.entries(msg)) {
-      if (k !== "tool_calls") rest[k] = v;
-    }
-    return { ...rest, content: newContent };
-  });
-
-  if (!modified) return payload;
-  return { ...payload, messages: newMessages };
-}
 
 export default function (pi: ExtensionAPI) {
   const models = allMakoraModels;
@@ -376,15 +213,7 @@ export default function (pi: ExtensionAPI) {
     models,
   });
 
-  pi.on("before_provider_request", (event) => {
-    const payload = event.payload as Record<string, unknown> | undefined;
-    if (!payload || typeof payload.model !== "string") return;
-
-    let result = rewriteVllmPayload(payload);
-    if (isMakoraGlmVllmModel(payload.model)) {
-      result = stripGlmToolCalls(result);
-    }
-    return result;
-  });
+  // Abort runaway '!' repetition loops on the GLM 5.2 family and resume the
+  // agentic loop invisibly (no new user message). See ./death-loop-guard.ts.
+  registerDeathLoopGuard(pi);
 }
-

package/models.json

@@ -41,6 +41,27 @@
       "maxTokensField": "max_completion_tokens"
     }
   },
+  {
+    "id": "google/gemma-4-26B-A4B",
+    "name": "Gemma 4 26B A4B",
+    "reasoning": false,
+    "input": [
+      "text"
+    ],
+    "cost": {
+      "input": 0,
+      "output": 0,
+      "cacheRead": 0,
+      "cacheWrite": 0
+    },
+    "contextWindow": 262144,
+    "maxTokens": 0,
+    "compat": {
+      "supportsDeveloperRole": false,
+      "supportsStore": false,
+      "maxTokensField": "max_completion_tokens"
+    }
+  },
   {
     "id": "meta-llama/Llama-3.3-70B-Instruct",
     "name": "Llama 3.3 70B Instruct",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-makora-provider",
-  "version": "1.3.0",
+  "version": "1.5.0",
   "description": "Makora provider extension for pi - Access DeepSeek V4, GLM 5.2, Kimi K2.7 Code, Llama 3.3, Qwen 3.6, and more through the Makora inference API",
   "type": "module",
   "main": "index.ts",
@@ -21,6 +21,7 @@
   "license": "MIT",
   "files": [
     "index.ts",
+    "death-loop-guard.ts",
     "models.json",
     "custom-models.json",
     "patch.json",
@@ -33,6 +34,8 @@
     ]
   },
   "devDependencies": {
+    "@earendil-works/pi-agent-core": "^0.80.2",
+    "@earendil-works/pi-coding-agent": "^0.80.2",
     "vitest": "^4.1.9"
   },
   "scripts": {

package/patch.json

@@ -1,7 +1,7 @@
 {
   "deepseek-ai/DeepSeek-V4-Flash": {
     "reasoning": true,
-    "notes": "`include_reasoning` + `chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field",
+    "notes": "returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": null,
       "low": null,
@@ -16,7 +16,7 @@
   },
   "deepseek-ai/DeepSeek-V4-Pro": {
     "reasoning": true,
-    "notes": "`chat_template_kwargs.thinking` via `before_provider_request` payload rewrite; returns `reasoning` field",
+    "notes": "returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": null,
       "low": null,
@@ -32,7 +32,7 @@
   },
   "unsloth/Qwen3.6-27B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
@@ -47,7 +47,7 @@
   },
   "unsloth/Qwen3.6-35B-A3B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
@@ -66,7 +66,7 @@
       "text",
       "image"
     ],
-    "notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field; client-side tool call parsing (vLLM streaming parser bypass)",
+    "notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"