pi-makora-provider 1.7.0 → 1.9.0

package/README.md

@@ -21,13 +21,13 @@ _DeepSeek V4, Kimi K2.7 Code, GLM 5.2, Qwen 3.6 for [pi](https://github.com/eare
 | DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` | Yes | returns `reasoning` field |
 | DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` | Yes | returns `reasoning` field |
 | Gemma 4 26B A4B | `google/gemma-4-26B-A4B` | No |  |
-| GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field |
-| GLM 5.2 NVFP4 | `zai-org/GLM-5.2-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field |
-| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
+| GLM 5.2 FP8 | `zai-org/GLM-5.2-FP8` | Yes | off via top-level `reasoning_effort: "none"` (only lever GLM responds to; `enable_thinking`/`thinking`/chat_template_kwargs toggles are ignored); effort `high`/`max` only (minimal/low/medium clamp to `high`); multi-turn continuity via `clear_thinking` (false when thinking on = preserve, true when off = clear; `preserve_thinking` is inert for GLM) injected via onPayload; returns `reasoning` field |
+| GLM 5.2 NVFP4 | `zai-org/GLM-5.2-NVFP4` | Yes | off via top-level `reasoning_effort: "none"` (only lever GLM responds to; `enable_thinking`/`thinking`/chat_template_kwargs toggles are ignored); effort `high`/`max` only (minimal/low/medium clamp to `high`); multi-turn continuity via `clear_thinking` (false when thinking on = preserve, true when off = clear; `preserve_thinking` is inert for GLM) injected via onPayload; returns `reasoning` field |
+| Kimi K2.7 Code | `moonshotai/Kimi-K2.7-Code` | Yes | on/off via namespaced `chat_template_kwargs.thinking` (toggled with thinking switch); `preserve_thinking` (toggled) for multi-turn continuity; onPayload aliases each assistant message's `reasoning` → `reasoning_content` (Kimi's template reads `reasoning_content` but pi-ai sends `reasoning`); returns `reasoning` field. `reasoning_effort` is not emitted (Kimi's template reads namespaced `thinking`, not `reasoning_effort`) |
 | Llama 3.3 70B FP8 | `amd/Llama-3.3-70B-Instruct-FP8-KV` | No |  |
 | Llama 3.3 70B Instruct | `meta-llama/Llama-3.3-70B-Instruct` | No |  |
-| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
-| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | `enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field |
+| Qwen 3.6 27B NVFP4 | `unsloth/Qwen3.6-27B-NVFP4` | Yes | off via top-level `reasoning_effort: "none"`; `preserve_thinking` (toggled with thinking switch) injected via onPayload for multi-turn continuity; returns `reasoning` field |
+| Qwen 3.6 35B A3B NVFP4 | `unsloth/Qwen3.6-35B-A3B-NVFP4` | Yes | off via top-level `reasoning_effort: "none"`; `preserve_thinking` (toggled with thinking switch) injected via onPayload for multi-turn continuity; returns `reasoning` field |
 <!-- MODELS_TABLE_END -->
 
 ## Installation
@@ -160,21 +160,31 @@ Do **not** edit `models.json` directly — it is auto-generated from the API. To
 
 ## Death-Loop Guard
 
-GLM 5.2 (NVFP4 / FP8) occasionally degenerates into a repetition loop — an
-unbroken run of a single character or token, spaced or not (observed: `!` →
-`!!!!...`, `0` → `0000...`, and `0` → `0 0 0 ...`) — that eats the whole
-response. This extension ships a guard that watches the streamed assistant
-output (both the visible answer and the reasoning trace) and, when it detects a
-long run of any one non-whitespace character **or token**, **aborts
-the runaway generation and resumes the agentic loop invisibly** — no new user
-message is injected, using the
-[pi-invisible-continue](https://github.com/monotykamary/pi-invisible-continue)
+GLM 5.2 (NVFP4 / FP8) occasionally degenerates into a repetition loop that
+eats the whole response. Several shapes are possible: an unbroken run of one
+character (`!` → `!!!!...`, `0` → `0000...`), a spaced token loop (`0` →
+`0 0 0 ...`), a short unit repeated under any delimiter (`{}` → `{},{},{},...`),
+and a line/template loop where every line differs at the token level (a log-line
+loop with incrementing timestamps and cycling names). This extension ships a
+guard that watches the streamed assistant output (both the visible answer and
+the reasoning trace) with four detectors — character run, token run,
+trailing-unit run, and normalized-line run — and on a trip it **aborts the
+runaway generation, removes the toxic message from the agent's transcript, and
+resumes the agentic loop invisibly** — no new user message is injected, using
+the [pi-invisible-continue](https://github.com/monotykamary/pi-invisible-continue)
 pattern (`agent.prompt([])`).
 
-On abort, pi finalizes the in-flight assistant message **with its accumulated
-repeated-character content** into the transcript, so the guard drops that
-partial message before resuming — otherwise the model would just re-read its
-own repeated text and loop again. Recovery retries **indefinitely with exponential backoff (2s→60s,
+Aborting alone isn't enough: the toxic text would stay in the agent's
+persisted transcript and bias later turns (the model re-rolls into ever more
+obscure loops), and any loop that completes without being aborted is committed
+outright. So the guard also **removes the toxic output from the agent itself**:
+a `message_end` handler replaces a degenerate (or aborted) assistant message
+with a clean stub before it is saved to the session file / shown in the TUI,
+and a `context` handler strips degenerate (and stub) assistant messages from
+the per-LLM-call message list so the model never re-sees them. The recovery
+trim also drops the last assistant message from `state.messages` before
+resuming, so the continued `agent.prompt([])` sends a clean context. Recovery
+retries **indefinitely with exponential backoff (2s→60s,
 2×)**, like [pi-retry](https://github.com/monotykamary/pi-retry) — long-horizon
 agent work can trip the loop many times in a session, so a retry cap would
 strand the agent mid-task. The loop exits only on a clean turn, a user abort
@@ -192,6 +202,12 @@ constants at the top of [`death-loop-guard.ts`](./death-loop-guard.ts):
 | `IGNORED_REPEAT_CHARS` | `space, tab, LF, CR, FF, VT` | Characters whose repetition never trips the guard. |
 | `TOKEN_REPEAT_THRESHOLD` | `40` | Consecutive identical whitespace-delimited tokens that trip the guard. Catches spaced loops like `0 0 0` that the character run can't see. |
 | `TOKEN_REPEAT_BUFFER_CHARS` | `1024` | Tail length kept for token-run detection. Must hold ≥ `TOKEN_REPEAT_THRESHOLD` copies of the repeated unit. |
+| `LINE_REPEAT_THRESHOLD` | `100` | Consecutive trailing lines sharing one normalized structure (digits→`#`, identifiers→`A`, whitespace collapsed) that trip the guard. Catches line/template loops (e.g. a log-line loop with incrementing timestamps / cycling names) invisible to the char and token runs. Set high to spare typical structured output (tables, CSVs); degenerate loops produce hundreds-to-thousands of lines. |
+| `LINE_REPEAT_BUFFER_CHARS` | `65536` | Tail length kept for line-run detection. Only the last `LINE_REPEAT_THRESHOLD + 8` complete lines are normalized per check. |
+| `UNIT_REPEAT_THRESHOLD` | `40` | Trailing repeats of a short unit (1–`UNIT_MAX_LENGTH` chars) under ANY delimiter (or none) that trip the guard. The delimiter-agnostic catch-all: catches `{},{},{}` (unit `{},`), `();();();` (unit `();`), and any separator the char/token runs can't see. |
+| `UNIT_MAX_LENGTH` | `16` | Maximum candidate unit length for trailing-unit detection. Repeating units are short; 16 covers `{},`, `();`, `->` with room to spare. |
+| `UNIT_REPEAT_BUFFER_CHARS` | `1024` | Tail length kept for trailing-unit detection. Must hold ≥ `UNIT_REPEAT_THRESHOLD` copies of the longest unit (40 × 16 = 640). |
+| `DEATH_LOOP_STUB_TEXT` | `[Makora death-loop guard: discarded a degenerate repetition loop.]` | Clean replacement swapped in at `message_end` so a death-loop message is removed from the saved transcript. A `context` handler also strips this stub from what the model sees. |
 | `BACKOFF_BASE_MS` | `2000` | Initial recovery backoff delay. |
 | `BACKOFF_MAX_MS` | `60000` | Maximum backoff delay (cap). |
 | `BACKOFF_MULTIPLIER` | `2` | Backoff growth factor per retry. |

package/death-loop-guard.ts

@@ -2,16 +2,29 @@
  * Death-loop guard for Makora reasoning models.
  *
  * Some Makora models (notably GLM 5.2 NVFP4 / FP8) occasionally fall into a
- * degenerate repetition loop, emitting an unbroken run of a single character
- * (observed: '!' -> "!!!!..." and '0' -> "0000...") or a repeated token
- * (observed: '0' -> "0 0 0 ...") that consumes the whole response. This guard
- * watches the
- * streamed assistant output (both the visible answer and the reasoning
- * trace); when that run is detected it aborts the runaway generation, drops
- * the partial (toxic) assistant message from the transcript, and resumes the
- * agentic loop invisibly via agent.prompt([]) — the same pattern
- * pi-invisible-continue / pi-retry uses, so no new user message pollutes the
- * context.
+ * degenerate repetition loop that consumes the whole response. Several shapes
+ * are possible: an unbroken run of one character ('!' -> "!!!!...", '0' ->
+ * "0000..."), a spaced token loop ('0' -> "0 0 0 ..."), a short unit repeated
+ * under any delimiter ('{}' -> "{},{},{},..."), and a line/template loop where
+ * every line differs at the token level (a log-line loop with incrementing
+ * timestamps and cycling names). This guard watches the streamed assistant
+ * output (both the visible answer and the reasoning trace) with four detectors
+ * — character run, token run, trailing-unit run, and normalized-line run —
+ * and on a trip it (1) aborts the runaway generation, (2) removes the toxic
+ * message from the agent itself, and (3) resumes the agentic loop invisibly
+ * via agent.prompt([]) (the pi-invisible-continue / pi-retry pattern, so no
+ * new user message pollutes the context).
+ *
+ * Removal, not just suppression: aborting alone leaves the toxic text in the
+ * agent's persisted transcript, and any pattern that completes without being
+ * aborted is committed outright — both bias later turns (the model re-rolls
+ * into ever more obscure loops). So a message_end handler replaces a
+ * degenerate/aborted assistant message with a clean stub BEFORE it is saved to
+ * the session file / shown in the TUI, and a context handler strips degenerate
+ * (and stub) assistant messages from the per-LLM-call message list so the
+ * model never re-sees them. The recovery trim also drops the last assistant
+ * message from state.messages so the resumed prompt([]) sends a clean
+ * continuation context.
  *
  * Infinite retries with exponential backoff. Long-horizon agent work can trip
  * the loop many times in one session; capping retries would strand the agent
@@ -21,12 +34,16 @@
  * window to intervene; interruptible sleep polls every 100ms so Esc and /new
  * take effect within 100ms instead of waiting out the full delay.
  *
- * Why trim the aborted message: on abort, pi finalizes the in-flight
- * assistant message WITH its accumulated '!!!' content and stopReason
- * "aborted" into the transcript. Resuming from that context would re-feed
- * the toxic text to the model and likely re-trigger the loop. Dropping the
- * last (aborted) assistant message leaves the context ending at the prior
- * user/toolResult message — a clean continuation point.
+ * Why trim AND scrub the aborted message: on abort, pi finalizes the
+ * in-flight assistant message WITH its accumulated toxic content and
+ * stopReason "aborted" into the transcript (the Agent pushes it to
+ * state.messages before event listeners run). Resuming from that context
+ * would re-feed the toxic text to the model and likely re-trigger the loop.
+ * The message_end handler swaps that content for a clean stub in the saved
+ * transcript; trimAbortedDeathLoop then drops the last assistant message from
+ * state.messages so the resumed prompt([]) sends a context ending at the prior
+ * user/toolResult — a clean continuation point. The context handler
+ * backstops any case that bypasses the recovery path.
  *
  * Distinguishing our abort from a user Esc: both call agent.abort(), so both
  * surface as stopReason "aborted" at turn_end. The handler sets _weAborted
@@ -97,6 +114,47 @@ export const TOKEN_REPEAT_THRESHOLD = 40;
  *  copies of tokens up to ~25 chars. */
 export const TOKEN_REPEAT_BUFFER_CHARS = 1024;
 
+/** Trip after this many consecutive trailing lines that share the same
+ *  normalized structure (digits -> '#', identifiers -> 'A', whitespace
+ *  collapsed). Catches template/line-level repetition where every line differs
+ *  at the token level (incrementing timestamps, cycling names) — e.g. a
+ *  log-line loop — which neither the character nor the token run can see. Set
+ *  high (100): structural repetition is the most false-positive-prone (markdown
+ *  tables, CSVs), while degenerate loops produce hundreds-to-thousands of
+ *  lines, so a high threshold catches them fast while sparing typical structured
+ *  output. Tunable. */
+export const LINE_REPEAT_THRESHOLD = 100;
+
+/** Bounded tail kept for line-run detection. Only the last
+ *  (LINE_REPEAT_THRESHOLD + 8) complete lines are normalized per check, so cost
+ *  stays bounded regardless of buffer size; the buffer just needs to hold that
+ *  many lines. 65536 fits ~100 lines up to ~600 chars. */
+export const LINE_REPEAT_BUFFER_CHARS = 65536;
+
+/** Trip after this many trailing repeats of a short unit (1..UNIT_MAX_LENGTH
+ *  chars) under any delimiter (or none). The delimiter-agnostic catch-all: it
+ *  catches `{},{},{}` (unit `{},`), `();();();` (unit `();`), and any future
+ *  separator the char/token runs can't see. 40 mirrors the other thresholds. */
+export const UNIT_REPEAT_THRESHOLD = 40;
+
+/** Maximum candidate unit length for trailing-unit detection. Repeating units
+ *  are short; 16 covers multi-char units like `{},` (3), `();` (3), `->` (2)
+ *  with room to spare. */
+export const UNIT_MAX_LENGTH = 16;
+
+/** Bounded tail kept for trailing-unit detection. Must hold >=
+ *  UNIT_REPEAT_THRESHOLD copies of the longest unit: 40 * 16 = 640, so 1024
+ *  gives comfortable margin. */
+export const UNIT_REPEAT_BUFFER_CHARS = 1024;
+
+/** Replacement content for a finalized assistant message that was a death
+ *  loop. The toxic text is removed from the agent's persisted transcript
+ *  (session file + TUI) by replacing it with this clean stub at message_end,
+ *  and a context handler strips even this stub from what the model sees, so
+ *  neither the toxic output nor the stub biases later turns. */
+export const DEATH_LOOP_STUB_TEXT =
+  "[Makora death-loop guard: discarded a degenerate repetition loop.]";
+
 /** Exponential backoff for recovery retries. Mirrors pi-retry's defaults:
  *  2s base, 60s cap, 2× multiplier. Tunable via these constants. */
 export const BACKOFF_BASE_MS = 2000;
@@ -119,6 +177,8 @@ export interface GuardedMessage {
   role: string;
   stopReason?: string;
   content?: unknown;
+  provider?: string;
+  model?: string;
 }
 
 export interface GuardedAgent {
@@ -145,11 +205,22 @@ let _trailingRunLen = 0;
  *  identical tokens. Capped to keep memory/cost bounded on long streams. */
 let _tokenRepeatBuffer = "";
 
+/** Bounded trailing buffer for line/template repetition detection (e.g. a
+ *  log-line loop where every line differs at the token level). Only complete
+ *  lines are inspected, and only the last few are normalized per check. */
+let _lineRepeatBuffer = "";
+
+/** Bounded trailing buffer for trailing-unit (delimiter-agnostic) repetition
+ *  detection (e.g. "{},{},{}"). */
+let _unitRepeatBuffer = "";
+
 /** Reset all per-turn/per-block detection state. */
 function resetDetection(): void {
   _trailingRunChar = "";
   _trailingRunLen = 0;
   _tokenRepeatBuffer = "";
+  _lineRepeatBuffer = "";
+  _unitRepeatBuffer = "";
 }
 
 /** Latch: already tripped for the current assistant message. */
@@ -322,6 +393,147 @@ export function isDegenerateTokenRun(
   return run.count >= threshold && run.token !== "";
 }
 
+/** A trailing run of consecutive lines sharing one normalized structure.
+ *  `template` is "" when `count` === 0. */
+export interface LineRun {
+  template: string;
+  count: number;
+}
+
+/** Normalize a line for structural repetition detection: identifiers -> "A",
+ *  digit runs -> "#", runs of spaces/tabs collapsed to one, trailing whitespace
+ *  trimmed. Lines that differ only in filler (timestamps, names, numbers, ids)
+ *  collapse to the same template, so a log-line loop like
+ *  "... User logged in: alice" / "... bob" / "... carol" maps to one template. */
+export function normalizeLine(line: string): string {
+  return line
+    .replace(/[A-Za-z_][A-Za-z0-9_]*/g, "A")
+    .replace(/[0-9]+/g, "#")
+    .replace(/[ \t]+/g, " ")
+    .replace(/[ \t]+$/, "");
+}
+
+/** Trailing run of consecutive lines with the same normalized structure in
+ *  `text`. Only COMPLETE lines (terminated by a newline) are counted — the
+ *  trailing partial line is ignored so a mid-stream truncation can't break or
+ *  inflate the run. Only the last `maxScan` complete lines are normalized, so
+ *  cost is bounded regardless of `text` length. Blank trailing lines are
+ *  skipped (legitimate blank lines must not trip the guard). */
+export function trailingLineRun(
+  text: string,
+  maxScan: number = LINE_REPEAT_THRESHOLD + 8,
+): LineRun {
+  if (!text.includes("\n")) return { template: "", count: 0 };
+  const parts = text.split("\n");
+  parts.pop(); // drop the partial/empty segment after the last newline
+  if (parts.length === 0) return { template: "", count: 0 };
+  const lines = parts.slice(-maxScan).map(normalizeLine);
+  let end = lines.length;
+  while (end > 0 && lines[end - 1] === "") end--;
+  if (end === 0) return { template: "", count: 0 };
+  const template = lines[end - 1];
+  let count = 1;
+  let i = end - 1;
+  while (i > 0 && lines[i - 1] === template) {
+    count++;
+    i--;
+  }
+  return { template, count };
+}
+
+/** True when a trailing line run is long enough to be a degenerate loop.
+ *  Blank-line runs never count (template is ""). Exported for unit testing. */
+export function isDegenerateLineRun(
+  run: LineRun,
+  threshold: number = LINE_REPEAT_THRESHOLD,
+): boolean {
+  return run.count >= threshold && run.template !== "";
+}
+
+/** A trailing run of one repeated unit (1..UNIT_MAX_LENGTH chars), under any
+ *  delimiter (or none). `unit` is "" when `count` === 0. */
+export interface UnitRun {
+  unit: string;
+  count: number;
+}
+
+/** True when every char of `s` is a whitespace char we ignore (mirrors
+ *  IGNORED_REPEAT_CHARS). Keeps the unit detector from tripping on
+ *  pure-whitespace units (indentation, blank lines). */
+function isAllWhitespace(s: string): boolean {
+  for (let i = 0; i < s.length; i++) {
+    if (!IGNORED_REPEAT_CHARS.has(s[i])) return false;
+  }
+  return s.length > 0;
+}
+
+/** Best trailing repetition of a short unit in `text`. For each candidate
+ *  period p (1..maxLength), takes the unit as the last p chars and counts how
+ *  many consecutive p-blocks ending at the tail equal it (a leading partial
+ *  block is ignored). Returns the run with the highest count. Delimiter-
+ *  agnostic, so it catches a short unit repeated under ANY separator (or none):
+ *  `!!!!` (unit `!`), `0 0 0` (unit `0 `), `{},{},{}` (unit `{},`), `();();();`
+ *  (unit `();`), etc.
+ *
+ *  Cost is paid only when repetition is present: on normal text every
+ *  candidate mismatches its first block, so the whole check is O(maxLength)
+ *  slices; a real repeat does O(text.length) work — exactly what we want to
+ *  detect. Operates on a bounded tail buffer. */
+export function trailingUnitRun(
+  text: string,
+  maxLength: number = UNIT_MAX_LENGTH,
+): UnitRun {
+  const n = text.length;
+  if (n < 2) return { unit: "", count: 0 };
+  let best: UnitRun = { unit: "", count: 0 };
+  const maxP = Math.min(maxLength, n);
+  for (let p = 1; p <= maxP; p++) {
+    const unit = text.slice(n - p);
+    let count = 1;
+    let start = n - p;
+    while (start - p >= 0 && text.slice(start - p, start) === unit) {
+      count++;
+      start -= p;
+    }
+    if (count > best.count) best = { unit, count };
+  }
+  return best;
+}
+
+/** True when a trailing unit run is long enough to be a degenerate loop.
+ *  Pure-whitespace units never count (e.g. a run of spaces), matching the
+ *  character detector's whitespace exclusion. Exported for unit testing. */
+export function isDegenerateUnitRun(
+  run: UnitRun,
+  threshold: number = UNIT_REPEAT_THRESHOLD,
+): boolean {
+  return run.count >= threshold && run.unit !== "" && !isAllWhitespace(run.unit);
+}
+
+/** True when a finalized assistant message is a death loop by ANY detector
+ *  (character run, token run, trailing-unit run, or line/template run). Used by
+ *  the message_end scrubber, the context filter, and the recovery trim to
+ *  decide what to remove from the agent. */
+export function isDeathLoopMessage(msg: GuardedMessage): boolean {
+  if (msg.role !== "assistant") return false;
+  const text = extractText(msg.content);
+  if (text.length === 0) return false;
+  return (
+    isDegenerateRun(messageTrailingRun(msg)) ||
+    isDegenerateTokenRun(trailingTokenRun(text)) ||
+    isDegenerateUnitRun(trailingUnitRun(text)) ||
+    isDegenerateLineRun(trailingLineRun(text))
+  );
+}
+
+/** True when `msg` was produced by a guarded Makora model (so we only scrub
+ *  our own models' output, never another provider's). */
+export function isGuardedMessage(msg: GuardedMessage): boolean {
+  if (msg.provider !== PROVIDER_ID) return false;
+  if (GUARDED_MODEL_IDS.has("*")) return true;
+  return msg.model != null && GUARDED_MODEL_IDS.has(msg.model);
+}
+
 /** Exponential backoff delay, capped at maxDelayMs. Mirrors pi-retry. */
 export function calculateDelay(
   attempt: number,
@@ -369,9 +581,7 @@ function trimAbortedDeathLoop(agent: GuardedAgent): void {
   if (
     last &&
     last.role === "assistant" &&
-    (last.stopReason === "aborted" ||
-      isDegenerateRun(messageTrailingRun(last)) ||
-      isDegenerateTokenRun(trailingTokenRun(extractText(last.content))))
+    (last.stopReason === "aborted" || isDeathLoopMessage(last))
   ) {
     agent.state.messages = msgs.slice(0, -1);
   }
@@ -404,8 +614,8 @@ async function triggerRecovery(): Promise<void> {
       const delay = calculateDelay(attempt);
       if (_notifyFn) {
         _notifyFn(
-          `Makora death-loop guard: resuming after runaway repeated-character ` +
-            `output (retry ${attempt}, backoff ${formatDuration(delay)})...`,
+          `Makora death-loop guard: resuming after a runaway repetition loop ` +
+            `(retry ${attempt}, backoff ${formatDuration(delay)})...`,
           "warning",
         );
       }
@@ -520,7 +730,39 @@ export function registerDeathLoopGuard(pi: ExtensionAPI): void {
     );
     const tokenRun = trailingTokenRun(_tokenRepeatBuffer);
 
-    if (!isDegenerateRun(run) && !isDegenerateTokenRun(tokenRun)) return;
+    // Line/template repetition: a structured loop (e.g. a log-line loop where
+    // every line differs at the token level — incrementing timestamps, cycling
+    // names) is invisible to both the char and token runs. Track the recent
+    // tail and, when a line completes (a newline arrives), count consecutive
+    // lines sharing one normalized structure. Recomputed only on newline
+    // deltas; the trailing partial line is ignored by trailingLineRun.
+    let lineRun: LineRun = { template: "", count: 0 };
+    if (ame.delta.includes("\n")) {
+      _lineRepeatBuffer = (_lineRepeatBuffer + ame.delta).slice(
+        -LINE_REPEAT_BUFFER_CHARS,
+      );
+      lineRun = trailingLineRun(_lineRepeatBuffer);
+    }
+
+    // Trailing-unit repetition: a short unit repeated under ANY delimiter (or
+    // none) — e.g. "{},{},{}" (unit "{},"), "();();();" (unit "();") — is
+    // invisible to the char run (the separator resets it) and the token run
+    // (whitespace-only splitter). This general detector catches it by finding
+    // any short period whose trailing blocks repeat. Runs every delta; cheap
+    // on normal text (each candidate mismatches its first block).
+    _unitRepeatBuffer = (_unitRepeatBuffer + ame.delta).slice(
+      -UNIT_REPEAT_BUFFER_CHARS,
+    );
+    const unitRun = trailingUnitRun(_unitRepeatBuffer);
+
+    if (
+      !isDegenerateRun(run) &&
+      !isDegenerateTokenRun(tokenRun) &&
+      !isDegenerateLineRun(lineRun) &&
+      !isDegenerateUnitRun(unitRun)
+    ) {
+      return;
+    }
 
     // Trip: abort the runaway stream and kick off the recovery loop (once,
     // mutex-gated). The loop driver handles re-detection on resumed turns.
@@ -529,7 +771,7 @@ export function registerDeathLoopGuard(pi: ExtensionAPI): void {
     const agent = _agent;
     if (!agent) {
       ctx.ui.notify(
-        "Makora death-loop guard: runaway repeated-character output detected " +
+        "Makora death-loop guard: runaway repetition loop detected " +
           "but the Agent instance was not captured; cannot recover automatically.",
         "warning",
       );
@@ -561,4 +803,53 @@ export function registerDeathLoopGuard(pi: ExtensionAPI): void {
       _notifyFn = (message, level) => ctx.ui.notify(message, level);
     }
   });
+
+  // Remove degenerate output from the agent's persisted transcript. When a
+  // finalized assistant message is a death loop (or one we aborted), replace
+  // its content with a clean stub BEFORE it is saved to the session file /
+  // shown in the TUI. Trimming state.messages only cleans the in-memory
+  // continuation context; this cleans the saved record too, so the toxic text
+  // can't bias a later /resume. (The Agent pushes the original message to
+  // state.messages before this event fires; trimAbortedDeathLoop drops it.)
+  pi.on("message_end", (event) => {
+    const msg = event.message as GuardedMessage;
+    if (msg.role !== "assistant") return;
+    if (!isGuardedMessage(msg)) return;
+    if (!isDeathLoopMessage(msg) && !_weAborted) return;
+    return {
+      message: {
+        ...event.message,
+        content: [{ type: "text", text: DEATH_LOOP_STUB_TEXT }],
+      },
+    };
+  });
+
+  // Defense-in-depth: before each LLM call, strip any degenerate (or stub)
+  // assistant message from the trailing messages so the model never re-sees
+  // toxic output — covering user-prompt turns and resumed sessions even when a
+  // loop completed without being aborted. The recovery's direct
+  // agent.prompt([]) bypasses this event (the state.messages trim covers that
+  // path). Only the last ~32 messages are scanned: degenerate output, if
+  // present, is recent, and stubs left in context are clean (non-biasing).
+  pi.on("context", (event) => {
+    const messages = event.messages;
+    if (!messages || messages.length === 0) return;
+    const scanFrom = Math.max(0, messages.length - 32);
+    let changed = false;
+    const filtered = messages.filter((m, i) => {
+      if (i < scanFrom || m.role !== "assistant") return true;
+      const text = extractText((m as GuardedMessage).content);
+      const isToxic =
+        text.length >= 1000 &&
+        isGuardedMessage(m as GuardedMessage) &&
+        isDeathLoopMessage(m as GuardedMessage);
+      if (isToxic || text === DEATH_LOOP_STUB_TEXT) {
+        changed = true;
+        return false;
+      }
+      return true;
+    });
+    if (!changed) return;
+    return { messages: filtered };
+  });
 }

package/index.ts

@@ -21,12 +21,14 @@
  *   - Llama 3.3 70B: not a reasoning model.
  *
  * A death-loop guard (see ./death-loop-guard.ts) is registered alongside the
- * provider. It watches the assistant text stream on the GLM 5.2 family and,
- * if the model falls into a degenerate repetition loop — an unbroken run of a
- * single character or token, spaced or not (observed: `!!!!`, `0000`, `0 0 0`)
- * — it aborts the runaway generation and resumes the agentic loop invisibly
- * via agent.prompt([]) (the pi-invisible-continue pattern); no new user
- * message is injected.
+ * provider. It watches the assistant text stream on the GLM 5.2 family with
+ * four detectors — character run, token run, trailing-unit run, and
+ * normalized-line run — and, if the model falls into a degenerate repetition
+ * loop (observed: `!!!!`, `0000`, `0 0 0`, `{},{},{}`, and a structured
+ * log-line loop), it aborts the runaway generation, removes the toxic message
+ * from the agent's transcript (so it can't bias later turns), and resumes the
+ * agentic loop invisibly via agent.prompt([]) (the pi-invisible-continue
+ * pattern); no new user message is injected.
  *
  * Developer role is NOT supported by any of the chat templates on Makora's
  * vLLM deployment (prompts with role: "developer" are silently dropped).
@@ -47,6 +49,9 @@
  */
 
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import type { SimpleStreamOptions, AssistantMessageEventStream } from "@earendil-works/pi-ai";
+import { clampThinkingLevel } from "@earendil-works/pi-ai";
+import { streamOpenAICompletions } from "@earendil-works/pi-ai/compat";
 import modelsData from "./models.json" with { type: "json" };
 import customModelsData from "./custom-models.json" with { type: "json" };
 import patchData from "./patch.json" with { type: "json" };
@@ -91,6 +96,13 @@ interface JsonModel {
     requiresToolResultName?: boolean;
     requiresAssistantAfterToolResult?: boolean;
     cacheControlFormat?: "anthropic";
+    /** When set, onPayload copies each assistant message's `reasoning` field into
+     *  this field name before sending. Needed when a model's chat template reads
+     *  prior reasoning from a different field than the one pi-ai sends it in.
+     *  e.g. Kimi K2.7's template reads `reasoning_content`, but pi-ai sends
+     *  `reasoning` (the field the model returns) — without the copy, preserve_thinking
+     *  can't render the prior trace. */
+    assistantReasoningField?: string;
     /** Extra keys merged into vLLM `chat_template_kwargs` on every request.
      *  Used for preserved-thinking flags like `preserve_thinking` / `clear_thinking`
      *  that some chat templates require for multi-turn reasoning continuity. */
@@ -192,6 +204,177 @@ function buildModels(
   return Array.from(modelMap.values());
 }
 
+// Thinking-off + preserved-thinking via streamSimple
+//
+// Makora's reasoning models need BOTH a thinking on/off switch AND multi-turn
+// reasoning continuity (`preserve_thinking`). pi-ai's built-in `thinkingFormat`
+// branches are mutually exclusive: the `chat-template` branch emits
+// `chat_template_kwargs` (so it can carry `preserve_thinking`) but never a
+// top-level `reasoning_effort`; the OpenAI fallback emits `reasoning_effort`
+// (the lever GLM 5.2 and Qwen 3.6 actually respond to for off) but never
+// `chat_template_kwargs`. No single format emits both.
+//
+// Behavioral E2E (see test-thinking-triggers.ts / test-preserve-deterministic.ts):
+//   GLM 5.2   — off ONLY via top-level `reasoning_effort: "none"`;
+//               `enable_thinking`/`thinking`/chat_template_kwargs toggles are ignored.
+//               Multi-turn continuity uses `clear_thinking: false` (NOT preserve_thinking
+//               — that flag is inert for GLM); toggled with the thinking switch
+//               (false when on = preserve, true when off = clear).
+//   Qwen 3.6  — off via top-level `reasoning_effort: "none"` (or `enable_thinking`).
+//               Multi-turn continuity uses `preserve_thinking: true`; reads the
+//               `reasoning` field pi-ai sends. Toggled with the thinking switch.
+//   Kimi K2.7 — off via namespaced `chat_template_kwargs.thinking: false`.
+//               Multi-turn continuity uses `preserve_thinking: true`, BUT the
+//               template reads `reasoning_content` while pi-ai sends `reasoning`
+//               (Makora doesn't gateway-alias the fields) — so onPayload also
+//               copies each assistant message's `reasoning` → `reasoning_content`.
+//   preserve_thinking IS functional (deterministic E2E: Qwen 3.6 27B recalls
+//     1.00 with it on, 0.00 off) — it gates whether the prior reasoning trace
+//     is rendered into the next turn's prompt.
+//
+// So this provider registers a `streamSimple` wrapper that delegates to pi-ai's
+// `streamOpenAICompletions` (keeping all its streaming/tool-calling/caching) and
+// uses pi-ai's `onPayload` hook — which runs AFTER `buildParams` — to inject the
+// `chat_template_kwargs` that the chosen `thinkingFormat` branch can't reach.
+// `buildParams` still owns `reasoning_effort` (on → mapped effort; off → "none");
+// `onPayload` owns `chat_template_kwargs` (preserve_thinking + Kimi's `thinking`).
+// Per-model config lives in `compat.chatTemplateKwargs` (patch.json), using the
+// same `{ "$var": "thinking.enabled" }` schema pi-ai's chat-template format uses.
+
+/** Resolve one `chatTemplateKwargs` value (scalar or `{ $var }`) against the
+ *  current thinking state. Mirrors pi-ai's `resolveChatTemplateKwargValue` so
+ *  onPayload-injected values match what the built-in chat-template format would
+ *  produce — but injected after buildParams so they coexist with reasoning_effort. */
+export function resolveChatTemplateKwarg(
+  value: unknown,
+  model: JsonModel,
+  thinkingOn: boolean,
+  reasoningEffort: string | undefined,
+): unknown {
+  // Scalars pass through unchanged (e.g. static `preserve_thinking: true`).
+  if (value === null || typeof value !== "object") {
+    return value;
+  }
+  const spec = value as { $var?: string; omitWhenOff?: boolean; invert?: boolean };
+  if (spec.omitWhenOff && !thinkingOn) {
+    return undefined;
+  }
+  if (spec.$var === "thinking.enabled") {
+    return spec.invert ? !thinkingOn : thinkingOn;
+  }
+  if (spec.$var === "thinking.effort") {
+    const mapped = thinkingOn
+      ? model.thinkingLevelMap?.[reasoningEffort as string]
+      : model.thinkingLevelMap?.off;
+    if (mapped === undefined) return reasoningEffort;
+    return typeof mapped === "string" ? mapped : undefined;
+  }
+  return undefined;
+}
+
+/** Custom streamSimple: delegate to pi-ai's OpenAI completions streamer and
+ *  inject per-model `chat_template_kwargs` (preserve_thinking / clear_thinking +
+ *  Kimi's thinking toggle) via the onPayload hook, and alias each assistant
+ *  message's reasoning field when a template reads a different field name.
+ *  Models with neither chatTemplateKwargs nor assistantReasoningField (DeepSeek,
+ *  Llama, Gemma) pass through unchanged — onPayload is not registered. */
+export function streamMakora(
+  model: any,
+  context: any,
+  options?: SimpleStreamOptions,
+): AssistantMessageEventStream {
+  const apiKey = options?.apiKey || "";
+  if (!apiKey) {
+    throw new Error(
+      `No API key for Makora. Add it to ~/.pi/agent/auth.json under "makora", ` +
+        `set the MAKORA_OPTIMIZE_TOKEN env var, or use --api-key.`,
+    );
+  }
+
+  // pi-ai's streamer reads `model.api` to pick the OpenAI completions client. Our
+  // provider registers under `api: "makora"` (so pi routes to this streamSimple);
+  // override to `openai-completions` here so streamOpenAICompletions uses the
+  // standard client. Per-model baseUrl overrides (per-slug endpoints) are kept.
+  const makoraModel = { ...model, api: "openai-completions", baseUrl: model.baseUrl || BASE_URL };
+
+  // pi hands streamSimple providers the raw thinking selection as
+  // `options.reasoning` (a ThinkingLevel). The raw streamOpenAICompletions only
+  // reads `options.reasoningEffort`, so replicate the clamp+convert pi-ai's own
+  // streamSimple wrapper does — otherwise reasoning_effort never reaches the
+  // body and thinking levels silently do nothing. "off" → undefined (off).
+  const clampedReasoning = options?.reasoning
+    ? clampThinkingLevel(makoraModel, options.reasoning)
+    : undefined;
+  const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning;
+  const thinkingOn = reasoningEffort !== undefined;
+  const { reasoning: _reasoning, ...streamOptions } = options ?? {};
+
+  // Inject chat_template_kwargs after buildParams via onPayload. Any caller-
+  // supplied onPayload is chained first so it can inspect/replace the payload;
+  // our injection then merges into whatever chat_template_kwargs already exist.
+  // We also register onPayload when the model needs its prior-reasoning field
+  // aliased (assistantReasoningField) — e.g. Kimi's template reads
+  // `reasoning_content` but pi-ai sends `reasoning`; the copy makes the
+  // preserved trace visible to the template so preserve_thinking can render it.
+  const userOnPayload = (streamOptions as any).onPayload;
+  const extraKwargs = (makoraModel as JsonModel).compat?.chatTemplateKwargs;
+  const hasExtraKwargs =
+    !!extraKwargs && typeof extraKwargs === "object" && Object.keys(extraKwargs).length > 0;
+  const assistantReasoningField = (makoraModel as JsonModel).compat?.assistantReasoningField;
+  const needsFieldCopy = !!assistantReasoningField;
+  const onPayload =
+    hasExtraKwargs || needsFieldCopy || userOnPayload
+      ? async (params: any, mdl: any) => {
+          let p = params;
+          if (userOnPayload) {
+            const next = await userOnPayload(p, mdl);
+            if (next !== undefined) p = next;
+          }
+          if (needsFieldCopy && thinkingOn) {
+            // Only alias the prior reasoning field when thinking is ON — when the
+            // user turns thinking off we must NOT carry prior reasoning forward
+            // (preserve_thinking is inert for Kimi's field rendering; the field's
+            // presence is what gates continuity). Kimi's template ignores the
+            // `reasoning` field, so skipping the copy leaves nothing to render.
+            const field = assistantReasoningField!;
+            const msgs = Array.isArray(p?.messages) ? p.messages : [];
+            p = {
+              ...p,
+              messages: msgs.map((m: any) =>
+                m && m.role === "assistant" &&
+                typeof m.reasoning === "string" && m.reasoning.length > 0 &&
+                m[field] === undefined
+                  ? { ...m, [field]: m.reasoning }
+                  : m
+              ),
+            };
+          }
+          if (hasExtraKwargs) {
+            const resolved: Record<string, unknown> = {};
+            for (const [k, v] of Object.entries(extraKwargs!)) {
+              const r = resolveChatTemplateKwarg(v, makoraModel, thinkingOn, reasoningEffort as string | undefined);
+              if (r !== undefined) resolved[k] = r;
+            }
+            p = {
+              ...p,
+              chat_template_kwargs: {
+                ...(p?.chat_template_kwargs ?? {}),
+                ...resolved,
+              },
+            };
+          }
+          return p;
+        }
+      : undefined;
+
+  return streamOpenAICompletions(makoraModel, context, {
+    ...streamOptions,
+    reasoningEffort,
+    apiKey,
+    ...(onPayload ? { onPayload } : {}),
+  });
+}
+
 // Extension Entry Point
 
 const PROVIDER_ID = "makora";
@@ -206,17 +389,23 @@ const allMakoraModels = buildModels(
 export default function (pi: ExtensionAPI) {
   const models = allMakoraModels;
 
-  // apiKey resolution order: auth.json ("makora" key) → MAKORA_OPTIMIZE_TOKEN env var
+  // apiKey resolution order: auth.json ("makora" key) → MAKORA_OPTIMIZE_TOKEN env var.
+  // `api: "makora"` + `streamSimple` routes every Makora model through streamMakora
+  // (above), which delegates to pi-ai's OpenAI completions streamer and injects
+  // per-model chat_template_kwargs via onPayload.
   pi.registerProvider(PROVIDER_ID, {
     name: "Makora",
     baseUrl: BASE_URL,
     apiKey: "$MAKORA_OPTIMIZE_TOKEN",
-    api: "openai-completions",
+    api: "makora",
+    streamSimple: streamMakora,
     models,
   });
 
-  // Abort runaway repetition loops — a single character or token, spaced or
-  // not (e.g. '!!!!', '0000', '0 0 0') — on the GLM 5.2 family and resume the
-  // agentic loop invisibly (no new user message). See ./death-loop-guard.ts.
+  // Abort runaway repetition loops — a single character, a spaced token, a
+  // delimiter-joined unit, or a structured line/template loop (e.g. '!!!!',
+  // '0000', '0 0 0', '{},{}', a log-line loop) — on the GLM 5.2 family; remove
+  // the toxic output from the agent's transcript and resume the agentic loop
+  // invisibly (no new user message). See ./death-loop-guard.ts.
   registerDeathLoopGuard(pi);
 }

package/models.json

@@ -159,7 +159,7 @@
       "cacheRead": 0,
       "cacheWrite": 0
     },
-    "contextWindow": 1048576,
+    "contextWindow": 980000,
     "maxTokens": 0,
     "compat": {
       "supportsDeveloperRole": false,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-makora-provider",
-  "version": "1.7.0",
+  "version": "1.9.0",
   "description": "Makora provider extension for pi - Access DeepSeek V4, GLM 5.2, Kimi K2.7 Code, Llama 3.3, Qwen 3.6, and more through the Makora inference API",
   "type": "module",
   "main": "index.ts",

package/patch.json

@@ -32,31 +32,31 @@
   },
   "unsloth/Qwen3.6-27B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
+    "notes": "off via top-level `reasoning_effort: \"none\"`; `preserve_thinking` (toggled with thinking switch) injected via onPayload for multi-turn continuity; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": "low",
-      "xhigh": "high"
+      "xhigh": "high",
+      "off": "none"
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
       "supportsReasoningEffort": true,
       "chatTemplateKwargs": {
-        "preserve_thinking": true
+        "preserve_thinking": { "$var": "thinking.enabled" }
       }
     }
   },
   "unsloth/Qwen3.6-35B-A3B-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
+    "notes": "off via top-level `reasoning_effort: \"none\"`; `preserve_thinking` (toggled with thinking switch) injected via onPayload for multi-turn continuity; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": "low",
-      "xhigh": "high"
+      "xhigh": "high",
+      "off": "none"
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
       "supportsReasoningEffort": true,
       "chatTemplateKwargs": {
-        "preserve_thinking": true
+        "preserve_thinking": { "$var": "thinking.enabled" }
       }
     }
   },
@@ -66,47 +66,54 @@
       "text",
       "image"
     ],
-    "notes": "Reasoning on by default (thinking-only model); `preserve_thinking` via `chatTemplateKwargs` for multi-turn continuity; returns `reasoning` field",
+    "notes": "on/off via namespaced `chat_template_kwargs.thinking` (toggled with thinking switch); `preserve_thinking` (toggled) for multi-turn continuity; onPayload aliases each assistant message's `reasoning` → `reasoning_content` (Kimi's template reads `reasoning_content` but pi-ai sends `reasoning`); returns `reasoning` field. `reasoning_effort` is not emitted (Kimi's template reads namespaced `thinking`, not `reasoning_effort`)",
     "thinkingLevelMap": {
       "minimal": "low",
       "xhigh": "high"
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true,
+      "supportsReasoningEffort": false,
+      "assistantReasoningField": "reasoning_content",
       "chatTemplateKwargs": {
-        "preserve_thinking": true
+        "thinking": { "$var": "thinking.enabled" },
+        "preserve_thinking": { "$var": "thinking.enabled" }
       }
     }
   },
   "zai-org/GLM-5.2-FP8": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); thinking levels aligned with neuralwatt GLM 5.2; returns `reasoning` field",
+    "notes": "off via top-level `reasoning_effort: \"none\"` (only lever GLM responds to; `enable_thinking`/`thinking`/chat_template_kwargs toggles are ignored); effort `high`/`max` only (minimal/low/medium clamp to `high`); multi-turn continuity via `clear_thinking` (false when thinking on = preserve, true when off = clear; `preserve_thinking` is inert for GLM) injected via onPayload; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": null,
       "low": null,
       "medium": null,
       "high": "high",
-      "xhigh": "max"
+      "xhigh": "max",
+      "off": "none"
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
+      "supportsReasoningEffort": true,
+      "chatTemplateKwargs": {
+        "clear_thinking": { "$var": "thinking.enabled", "invert": true }
+      }
     }
   },
   "zai-org/GLM-5.2-NVFP4": {
     "reasoning": true,
-    "notes": "`enable_thinking` via `qwen-chat-template`; effort via `reasoning_effort` (only `high`/`max` distinct, per vLLM GLM-5.2 recipe); returns `reasoning` field",
+    "notes": "off via top-level `reasoning_effort: \"none\"` (only lever GLM responds to; `enable_thinking`/`thinking`/chat_template_kwargs toggles are ignored); effort `high`/`max` only (minimal/low/medium clamp to `high`); multi-turn continuity via `clear_thinking` (false when thinking on = preserve, true when off = clear; `preserve_thinking` is inert for GLM) injected via onPayload; returns `reasoning` field",
     "thinkingLevelMap": {
       "minimal": null,
       "low": null,
       "medium": null,
       "high": "high",
-      "xhigh": "max"
+      "xhigh": "max",
+      "off": "none"
     },
     "compat": {
-      "thinkingFormat": "qwen-chat-template",
-      "supportsReasoningEffort": true
+      "supportsReasoningEffort": true,
+      "chatTemplateKwargs": {
+        "clear_thinking": { "$var": "thinking.enabled", "invert": true }
+      }
     }
   }
 }