@gtrabanco/pi-nan-provider 0.6.3 → 0.6.4

package/AGENTS.md

@@ -37,6 +37,27 @@ Regenerate the catalog after touching `scripts/generate-models.ts`:
 bun run generate-models
 ```
 
+## Live NaN API during diagnosis
+
+Diagnostic calls against the real gateway are allowed — they spend the
+maintainer's quota, so they are **permission-gated**:
+
+- **Ask the maintainer before running any live probe, with an approximate token
+  cost (input + output).** No silent probing. If the cost is not worth it, report
+  the behavior to NaN and let them reproduce it instead of debugging it here.
+- **Tests must never hit the network.** `bunfig.toml` preloads
+  `test/network-guard.ts`, which makes any un-injected `fetch` throw. Keep it:
+  inject `fetchImpl` / `options.fetch`, or use the local fixture. The permission
+  gate covers ad-hoc diagnosis only, never `bun test`.
+- **Default to `qwen3.6` — it is unlimited.**
+- **For massive/bulk probes prefer a model the maintainer uses less with a large
+  token budget, e.g. `mimo-v2.5`** (1M context).
+- **When the model under investigation is the point** (e.g. reproducing a
+  model-specific 400), use it, but minimize tokens: smallest viable prompt,
+  lowest `max_tokens`, stop at the first decisive response.
+- Repro commands that run a real `pi` session (`pi --fork ... -p ...`) use the
+  same key; keep them minimal and delete the forked session files afterwards.
+
 ## One shared implementation for all providers
 
 `nan` (and any future provider, e.g. `helmcode`) must stay behind the single shared

package/README.es.md

@@ -1,7 +1,7 @@
 # @gtrabanco/pi-nan-provider
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![Version](https://img.shields.io/badge/version-0.6.3-blue)](https://github.com/gtrabanco/pi-nan-provider/releases)
+[![Version](https://img.shields.io/badge/version-0.6.4-blue)](https://github.com/gtrabanco/pi-nan-provider/releases)
 
 [NaN Builders](https://nan.builders) model provider + MCP bridges para [pi](https://github.com/earendil-works/pi). 
 
@@ -47,7 +47,7 @@ El registro es síncrono a propósito: el catálogo de fallback está disponible
 
 Al cambiar de modelo, pi-ai reenvía el razonamiento del modelo anterior como texto plano de asistente — **sin límite de tamaño**. Un único razonamiento largo o degenerado puede desbordar la ventana de un modelo de 262K, y NaN responde con un `400 Invalid request. Check your request parameters.` genérico que parece un bug del proveedor (seguimiento upstream: [pi-nan-provider#3](https://github.com/gtrabanco/pi-nan-provider/issues/3); issue abierta upstream: [pi#6167](https://github.com/earendil-works/pi/issues/6167)).
 
-Este paquete limita cada bloque de razonamiento **cross-model** a 16.000 caracteres con un marcador visible de truncado. El razonamiento del mismo modelo no se toca nunca, y el guard solo actúa sobre peticiones dirigidas a los proveedores de este paquete. Pon `NAN_THINKING_GUARD=0` para desactivarlo.
+Este paquete **elimina todos los bloques de razonamiento cross-model reenviados**, de modo que cambiar de un modelo de 1M de contexto a uno de 262K (`qwen3.6`) ya no desborda la ventana. Las respuestas y los tool results de los modelos no se tocan — solo se quitan sus trazas internas de razonamiento, así que `qwen3.6` puede seguir respondiendo sobre lo que hizo otro modelo. El razonamiento del mismo modelo no se toca nunca, y el guard solo actúa sobre peticiones dirigidas a los proveedores de este paquete. Pon `NAN_THINKING_GUARD=0` para desactivarlo.
 
 ## 🔑 Autenticación
 

package/README.md

@@ -1,7 +1,7 @@
 # @gtrabanco/pi-nan-provider
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![Version](https://img.shields.io/badge/version-0.6.3-blue)](https://github.com/gtrabanco/pi-nan-provider/releases)
+[![Version](https://img.shields.io/badge/version-0.6.4-blue)](https://github.com/gtrabanco/pi-nan-provider/releases)
 
 [NaN Builders](https://nan.builders) model provider + MCP bridges for [pi](https://github.com/earendil-works/pi). 
 
@@ -47,7 +47,7 @@ The registration is synchronous on purpose: the generated fallback catalog is av
 
 When you switch models, pi-ai replays the previous model's reasoning as plain assistant text — with **no size bound**. A single long or degenerate reasoning trace can therefore overflow a 262K-context model's window, and NaN answers with a generic `400 Invalid request. Check your request parameters.` that looks like a provider bug (upstream tracking: [pi-nan-provider#3](https://github.com/gtrabanco/pi-nan-provider/issues/3); open upstream issue: [pi#6167](https://github.com/earendil-works/pi/issues/6167)).
 
-This package caps every replayed **cross-model** reasoning block at 16,000 chars with a visible truncation marker. Same-model reasoning is never altered, and the guard only acts on requests targeting this package's providers. Set `NAN_THINKING_GUARD=0` to disable it.
+This package **drops every replayed cross-model reasoning block**, so switching from a 1M-context model to a 262K one (`qwen3.6`) no longer overflows the window. The models' answers and tool results are untouched — only their internal reasoning traces are removed, so `qwen3.6` can still answer about what another model did. Same-model reasoning is never altered, and the guard only acts on requests targeting this package's providers. Set `NAN_THINKING_GUARD=0` to disable it.
 
 ## 🔑 Authentication
 

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@gtrabanco/pi-nan-provider",
-	"version": "0.6.3",
+	"version": "0.6.4",
 	"description": "NaN Builders (api.nan.builders) model provider for pi - OpenAI-compatible registration with a models.dev-generated fallback, tier-aware live catalog, and MCP bridges (official web search + optional community media server)",
 	"keywords": [
 		"pi",

package/scripts/models.generated.ts

@@ -1,7 +1,7 @@
 // This file is auto-generated by scripts/generate-models.ts
 // Do not edit manually — run `bun run generate-models` to update.
 //
-// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-10T11:57:39.263Z
+// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-11T11:47:14.840Z
 // Provenance: every contextWindow/maxTokens/input/cost value traces to
 // models.dev or to the per-entry notes below. Nothing is invented; entries
 // models.dev documents incompletely are omitted and flagged instead.
@@ -381,7 +381,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 export const GENERATED_CATALOG_META = {
 	source: "https://models.dev/api.json",
 	modelsDevProvider: "nan",
-	fetchedAt: "2026-09-10T11:57:39.263Z",
+	fetchedAt: "2026-09-11T11:47:14.840Z",
 	modelCount: 6,
 	models: ["deepseek-v4-flash","gemma4","glm5.3-flash","mimo-v2.5","qwen3.6","qwen3.8-flash"],
 	notes: [

package/src/cross-model-thinking-guard.ts

@@ -6,45 +6,36 @@
  * every non-redacted `thinking` block to a plain `text` block verbatim
  * (`packages/ai/src/api/transform-messages.ts`), and `openai-completions`
  * serializes that into the assistant `content` string. Nothing bounds
- * `block.thinking`, so a long/degenerate reasoning trace from the previous
- * model (seen in the wild: a 445,888-char / 131,072-token thinking block after
- * `glm5.3-flash` ended with `stopReason: "length"`) is replayed as a 445 KB
- * assistant text message. The request then exceeds the destination model's
- * context window and NaN's gateway answers with a generic
- * `400 Invalid request. Check your request parameters.` (verified live
- * 2026-09-10: the same payload returns 200 for deepseek-v4-flash /
- * glm5.3-flash and 200 for qwen3.6 once that message is removed).
+ * `block.thinking`, and nothing bounds the SUM across messages.
+ *
+ * Measured on real sessions: replayed reasoning is 30–60% of the whole context
+ * (e.g. 356,723 of 903,464 chars in one session; 349,882 of 749,525 in another).
+ * Switching from a 1M-context model (glm5.3-flash, deepseek-v4-flash) to a
+ * 262K-context one (`qwen3.6`) then overflows the window, and NaN's gateway
+ * answers a generic `400 Invalid request. Check your request parameters.`
+ *
+ * A per-block cap was tried first and is NOT enough: many medium blocks sum to
+ * hundreds of thousands of tokens. This guard therefore DROPS every replayed
+ * cross-model `thinking` block outright — it is the reasoning trace that
+ * pi-ai would have replayed as plain text, not the model's answers or tool
+ * results, so qwen can still answer questions about what glm/deepseek did.
+ * Same-model reasoning is never touched (signatures and continuity depend on
+ * it), and the guard only acts on requests targeting this package's providers.
  *
  * The extension runs in pi's `context` hook, which fires BEFORE pi-ai's
  * `transformMessages` (pi-agent-core `transformContext` → `convertToLlm` →
  * provider stream). It therefore sees the original `thinking` blocks and can
- * bound what they will become. Only messages whose (provider, api, model)
- * differ from the target are touched; same-model replay keeps its reasoning
- * byte-for-byte because signatures/continuity depend on it.
+ * remove them before they become text.
  *
- * This is a bounded mitigation, not a fix: it caps each replayed cross-model
- * reasoning block, which is sufficient for the observed single-degenerate-trace
- * failure. The real fix belongs upstream (see
- * https://github.com/earendil-works/pi/issues/9433).
+ * This is a bounded mitigation, not the fix: the unbounded conversion (and pi
+ * not re-checking the context size on a model switch) belongs upstream. See
+ * https://github.com/gtrabanco/pi-nan-provider/issues/3.
  */
 
 /** Env var that opts out of the guard (`0`, `false`, `no` or `off`). Default: enabled. */
 export const NAN_THINKING_GUARD_ENV = "NAN_THINKING_GUARD";
 
-/**
- * Maximum characters of a single cross-model reasoning block replayed as text.
- * 16,000 chars is ~4K tokens — far above any real reasoning trace, far below
- * the 445,888-char degenerate trace that caused the 400.
- */
-export const MAX_CROSS_MODEL_THINKING_CHARS = 16_000;
-
-/** Appended after the kept prefix so the substitution is visible, never silent. */
-export const CROSS_MODEL_THINKING_TRUNCATION_MARKER =
-	"\n\n[…previous-model reasoning truncated by pi-nan-provider to keep the request within NaN's context]";
-
 export interface CrossModelThinkingGuardOptions {
-	/** Max chars kept per replayed cross-model reasoning block. */
-	maxCharsPerBlock?: number;
 	/** Provider ids this guard applies to (the caller's registered providers). */
 	providerIds: ReadonlySet<string>;
 }
@@ -57,7 +48,6 @@ interface GuardTarget {
 
 interface ThinkingBlock {
 	type?: string;
-	thinking?: unknown;
 	[k: string]: unknown;
 }
 
@@ -84,19 +74,17 @@ function isThinkingBlock(value: unknown): value is ThinkingBlock {
 }
 
 /**
- * Bound every cross-model `thinking` block to `maxCharsPerBlock` characters.
- *
- * Returns the SAME array reference when nothing changed, so callers can skip
- * cloning the whole context on the common path. Never mutates the input.
+ * Remove every `thinking` block from assistant messages produced by a model
+ * other than the target. Returns the SAME array reference when nothing changed,
+ * so callers can skip cloning on the common path. Never mutates the input.
  */
-export function boundCrossModelThinking<T>(
+export function stripCrossModelThinking<T>(
 	messages: readonly T[],
 	target: GuardTarget | undefined,
 	options: CrossModelThinkingGuardOptions,
 ): readonly T[] {
 	if (!target?.provider || !options.providerIds.has(target.provider)) return messages;
 
-	const maxChars = options.maxCharsPerBlock ?? MAX_CROSS_MODEL_THINKING_CHARS;
 	let changed = false;
 
 	const next = messages.map((raw) => {
@@ -106,16 +94,8 @@ export function boundCrossModelThinking<T>(
 		const isSameModel = raw.provider === target.provider && raw.api === target.api && raw.model === target.id;
 		if (isSameModel) return raw;
 
-		let messageChanged = false;
-		const content = (raw.content as unknown[]).map((block) => {
-			if (!isThinkingBlock(block)) return block;
-			const text = block.thinking;
-			if (typeof text !== "string" || text.length <= maxChars) return block;
-			messageChanged = true;
-			return { ...block, thinking: text.slice(0, maxChars) + CROSS_MODEL_THINKING_TRUNCATION_MARKER };
-		});
-
-		if (!messageChanged) return raw;
+		const content = (raw.content as unknown[]).filter((block) => !isThinkingBlock(block));
+		if (content.length === (raw.content as unknown[]).length) return raw;
 		changed = true;
 		return { ...raw, content };
 	});

package/src/index.ts

@@ -29,9 +29,8 @@ import type { ContextEvent, ExtensionAPI, ProviderConfig } from "@earendil-works
 import type { Provider } from "@earendil-works/pi-ai";
 import { registerNanMcpCommand } from "./commands.ts";
 import {
-	boundCrossModelThinking,
 	crossModelThinkingGuardEnabled,
-	MAX_CROSS_MODEL_THINKING_CHARS,
+	stripCrossModelThinking,
 } from "./cross-model-thinking-guard.ts";
 import { baselineModels } from "./fetch-models.ts";
 import { createNanWebSearchTool, webSearchBridgeEnabled, NAN_API_KEY_ENV } from "./mcp/nan-search.ts";
@@ -114,29 +113,27 @@ function registerMcpToolsCompat(pi: ExtensionAPI): void {
 }
 
 /**
- * Bound the reasoning pi-ai replays across a model switch.
+ * Drop the reasoning pi-ai replays across a model switch.
  *
  * pi-ai's `transformMessages` downgrades a previous model's `thinking` blocks
- * to plain text with no size bound (still true on 0.85.1 / main), so a long or
- * degenerate reasoning trace is re-inlined into the assistant `content` and
- * can push the request past the destination model's context window. NaN's
- * gateway answers that with a generic `400 Invalid request. Check your request
- * parameters.`, which reads as a provider bug and not as an oversized prompt.
- * This hook runs before pi-ai converts the blocks, so capping them here keeps
- * the replayed context bounded. See src/cross-model-thinking-guard.ts.
+ * to plain text with no size bound (still true on 0.85.1 / main), and nothing
+ * bounds the sum across messages — measured at 30–60% of the whole context on
+ * real sessions. Switching from a 1M-context model to a 262K one (`qwen3.6`)
+ * then overflows the window, and NaN's gateway answers a generic
+ * `400 Invalid request. Check your request parameters.` This hook runs before
+ * pi-ai converts the blocks, so removing the cross-model reasoning here keeps
+ * the replayed context small. Answers and tool results are untouched. See
+ * src/cross-model-thinking-guard.ts.
  *
  * Scope: only requests targeting this package's providers are touched, and
- * only messages from a DIFFERENT model — same-model reasoning is never altered.
+ * only messages from a DIFFERENT model — same-model reasoning is never removed.
  */
 export function registerCrossModelThinkingGuard(pi: ExtensionAPI): void {
 	if (typeof pi.on !== "function") return; // old pi without the context hook
 	const providerIds = new Set(PROVIDERS.map((provider) => provider.id));
 	pi.on("context", (event, ctx) => {
 		if (!crossModelThinkingGuardEnabled()) return;
-		const guarded = boundCrossModelThinking(event.messages, ctx.model, {
-			maxCharsPerBlock: MAX_CROSS_MODEL_THINKING_CHARS,
-			providerIds,
-		});
+		const guarded = stripCrossModelThinking(event.messages, ctx.model, { providerIds });
 		if (guarded === event.messages) return;
 		return { messages: guarded as ContextEvent["messages"] };
 	});