@gtrabanco/pi-nan-provider 0.6.5 → 0.6.7

package/AGENTS.md

@@ -136,9 +136,25 @@ Every PR that changes code MUST bump `package.json` version in the same PR; CI p
   `supportsFinishReason: true` so pi-ai raises the retryable
   `Stream ended without finish_reason` (pi-ai's `RETRYABLE_PROVIDER_ERROR_PATTERN`
   matches `"ended without"`, so the turn is retried) instead of silently
-  synthesizing `stop`/`toolUse`. `supportsUsageInStreaming` is `false` because
-  `src/openai-compat-sanitizer.ts` strips `stream_options`. Regression tests:
-  `test/issue-2-truncated-stream.test.ts`; issue #2.
+  synthesizing `stop`/`toolUse`. `supportsUsageInStreaming` stays `false` by
+  default (NaN's published schema does not document `stream_options`), but the
+  sanitizer gates its `stream_options` removal on the model's effective
+  `compat.supportsUsageInStreaming`, so a confirmed per-model `models.json`
+  override now yields real usage instead of being silently undone
+  (`test/issue-4-token-usage.test.ts`; issues #2, #4). Regression tests:
+  `test/issue-2-truncated-stream.test.ts`, `test/issue-4-token-usage.test.ts`;
+  issues #2 and #4.
+- A NaN request that still exceeds the destination model's context window (the
+  cross-model thinking guard is disabled with `NAN_THINKING_GUARD=0`, the
+  inflation is not a `thinking` block, or the window is smaller) gets NaN's
+  generic 400 `Invalid request. Check your request parameters.`, which pi-ai's
+  `isContextOverflow()` does NOT match — so pi never compacts and the session
+  wedges (upstream `earendil-works/pi#9409`). `src/context-overflow-classifier.ts`
+  re-checks the request size at the provider boundary and rewrites that error
+  into a pi-recognizable overflow message (chars/3.47 estimate; conservative:
+  only when estimated over the window). Wired in `src/provider-factory.ts` after
+  the sanitizer. Regression tests: `test/issue-3-model-switch-overflow.test.ts`,
+  `test/context-overflow-classifier.test.ts`; issue #3.
 - Relative imports inside this package use `.ts` extensions (pi's official
   extension examples do the same; pi transpiles extension sources).
 - pi intentionally has NO built-in MCP client (docs/usage.md). MCP integration

package/README.es.md

@@ -49,6 +49,8 @@ Al cambiar de modelo, pi-ai reenvía el razonamiento del modelo anterior como te
 
 Este paquete **elimina todos los bloques de razonamiento cross-model reenviados**, de modo que cambiar de un modelo de 1M de contexto a uno de 262K (`qwen3.6`) ya no desborda la ventana. Las respuestas y los tool results de los modelos no se tocan — solo se quitan sus trazas internas de razonamiento, así que `qwen3.6` puede seguir respondiendo sobre lo que hizo otro modelo. El razonamiento del mismo modelo no se toca nunca, y el guard solo actúa sobre peticiones dirigidas a los proveedores de este paquete. Pon `NAN_THINKING_GUARD=0` para desactivarlo.
 
+Si una petición sigue desbordando — guard desactivado, inflado que no es un bloque de razonamiento (tool outputs grandes, imágenes) o una ventana de destino más pequeña — NaN responde con el mismo 400 genérico en lugar de nombrar el desbordamiento, y la auto-compactación de pi no lo reconoce, así que la sesión se queda atascada en el techo. Por eso el proveedor **vuelve a comprobar el tamaño de la petición a la salida**: cuando llega ese 400 genérico para una petición estimada por encima de la ventana del modelo, el error se reescribe como un mensaje de context overflow que pi reconoce, de modo que compacta y reintenta en vez de bloquearse. Un 400 genérico en una petición dentro de la ventana no se toca, así que nunca se etiquetan mal errores no relacionados.
+
 ### ⏱️ Streams truncados intermitentes (auto-retry, sin stall silencioso)
 
 El gateway LiteLLM de NaN cierra ocasionalmente un stream SSE **antes** de emitir el chunk final `finish_reason` (observado en `glm5.3-flash`; [issue #2](https://github.com/gtrabanco/pi-nan-provider/issues/2)). El catálogo declara `supportsFinishReason: true`, así que pi-ai lo convierte en el error `Stream ended without finish_reason` — que coincide con el patrón de errores reintentables de pi y se **reintenta automáticamente**, en vez de aceptar en silencio una respuesta a medias. Si una versión del gateway nunca manda `finish_reason`, el turno ahora falla de forma visible al agotar los reintentos.
@@ -69,7 +71,19 @@ Puedes sobrescribir el `compat` de cualquier modelo en `~/.pi/agent/models.json`
 
 > Poner `supportsFinishReason: false` restaura el antiguo stall silencioso — no recomendado.
 
-**Uso de tokens en streaming:** `supportsUsageInStreaming` es `false` porque el esquema estricto de NaN no documenta `stream_options` y el sanitizer de peticiones lo elimina. Por eso el usage aparece a cero; la flag ahora coincide con lo que realmente se envía.
+**Uso de tokens en streaming:** `supportsUsageInStreaming` es `false` por defecto porque el esquema publicado de NaN no documenta `stream_options`; sin él, el usage aparece a cero. Si has confirmado que tu modelo devuelve el chunk de usage en streaming, activalo por modelo — el sanitizer de peticiones entonces reenvía `stream_options: { "include_usage": true }` y pi muestra los tokens reales en lugar de ceros:
+
+```json
+{
+  "providers": {
+    "nan": {
+      "modelOverrides": {
+        "qwen3.6": { "compat": { "supportsUsageInStreaming": true } }
+      }
+    }
+  }
+}
+```
 
 ## 🔑 Autenticación
 

package/README.md

@@ -49,6 +49,8 @@ When you switch models, pi-ai replays the previous model's reasoning as plain as
 
 This package **drops every replayed cross-model reasoning block**, so switching from a 1M-context model to a 262K one (`qwen3.6`) no longer overflows the window. The models' answers and tool results are untouched — only their internal reasoning traces are removed, so `qwen3.6` can still answer about what another model did. Same-model reasoning is never altered, and the guard only acts on requests targeting this package's providers. Set `NAN_THINKING_GUARD=0` to disable it.
 
+If a request still overflows — the guard is disabled, the inflation is not a reasoning block (large tool outputs, images), or the destination window is simply smaller — NaN answers the same generic 400 instead of naming the overflow, and pi's auto-compaction does not recognize it, so the session wedges at the ceiling. The provider therefore **re-checks the request size on the way out**: when that generic 400 arrives for a request estimated over the model's window, the error is rewritten into a context-overflow message pi recognizes, so it compacts and retries instead of stalling. A generic 400 on a within-window request is left untouched, so unrelated errors are never mislabelled.
+
 ### ⏱️ Intermittent truncated streams (auto-retry, no silent stall)
 
 NaN's LiteLLM gateway occasionally closes an SSE stream **before** emitting the final `finish_reason` chunk (observed on `glm5.3-flash`; [issue #2](https://github.com/gtrabanco/pi-nan-provider/issues/2)). The catalog declares `supportsFinishReason: true`, so pi-ai turns that into the error `Stream ended without finish_reason` — which matches pi's retryable-provider pattern and is **retried automatically**, instead of silently accepting a half-finished answer. If a gateway version never sends `finish_reason`, the turn now fails visibly once the retry budget is exhausted.
@@ -69,7 +71,19 @@ You can override any model's `compat` per-model in `~/.pi/agent/models.json` (pi
 
 > Setting `supportsFinishReason: false` restores the old silent-stall behavior — not recommended.
 
-**Streaming token usage:** `supportsUsageInStreaming` is `false` because NaN's strict schema does not document `stream_options` and the request sanitizer removes it. Usage therefore reads as zero; the flag now matches what is actually sent.
+**Streaming token usage:** `supportsUsageInStreaming` is `false` by default because NaN's published schema does not document `stream_options`; without it usage reads as zero. If you have confirmed that your model returns a streaming usage chunk, opt in per model — the request sanitizer then forwards `stream_options: { "include_usage": true }` and pi reports real token counts instead of zeros:
+
+```json
+{
+  "providers": {
+    "nan": {
+      "modelOverrides": {
+        "qwen3.6": { "compat": { "supportsUsageInStreaming": true } }
+      }
+    }
+  }
+}
+```
 
 ## 🔑 Authentication
 

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@gtrabanco/pi-nan-provider",
-	"version": "0.6.5",
+	"version": "0.6.7",
 	"description": "NaN Builders (api.nan.builders) model provider for pi - OpenAI-compatible registration with a models.dev-generated fallback, tier-aware live catalog, and MCP bridges (official web search + optional community media server)",
 	"keywords": [
 		"pi",

package/scripts/generate-models.ts

@@ -84,9 +84,12 @@ const NAN_COMPAT = {
 	supportsDeveloperRole: false,
 	supportsReasoningEffort: true,
 	// pi-ai only sends `stream_options: { include_usage: true }` when this is
-	// true, but src/openai-compat-sanitizer.ts strips `stream_options` before
-	// sending (NaN's schema does not document it). Declaring true would be a
-	// lie the sanitizer immediately undoes; false matches what is sent.
+	// not false. NaN's published schema does not document `stream_options`, so
+	// the conservative default is false: the sanitizer strips it and usage
+	// stays zero. A user who confirms their gateway reports streaming usage
+	// can opt in per model with a models.json compat override
+	// (`supportsUsageInStreaming: true`); the sanitizer then forwards
+	// `stream_options` instead of deleting it (issue #4, 2026-09-13).
 	supportsUsageInStreaming: false,
 	// The NaN/LiteLLM gateway intermittently closes SSE streams before emitting
 	// `finish_reason`. With true, pi-ai raises "Stream ended without
@@ -99,7 +102,7 @@ const NAN_COMPAT = {
 };
 
 const NAN_COMPAT_NOTE =
-	"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.";
+	"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4).";
 
 interface ModelsDevModel {
 	id?: string;

package/scripts/models.generated.ts

@@ -1,7 +1,7 @@
 // This file is auto-generated by scripts/generate-models.ts
 // Do not edit manually — run `bun run generate-models` to update.
 //
-// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-13T16:55:47.183Z
+// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-13T18:08:44.200Z
 // Provenance: every contextWindow/maxTokens/input/cost value traces to
 // models.dev or to the per-entry notes below. Nothing is invented; entries
 // models.dev documents incompletely are omitted and flagged instead.
@@ -36,7 +36,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4).",
 			"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models). models.dev provider nan also lists text+image now (DeepSeek V4.1 Flash entry, checked 2026-09-13; its 2026-09-07 snapshot listed text only), so this override is kept as a pin for the vision capability rather than as a divergence."
 		],
 		"extras": {
@@ -97,7 +97,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4)."
 		],
 		"extras": {
 			"id": "gemma4",
@@ -160,7 +160,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4)."
 		],
 		"extras": {
 			"id": "glm5.3-flash",
@@ -219,7 +219,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4)."
 		],
 		"extras": {
 			"id": "mimo-v2.5",
@@ -279,7 +279,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4)."
 		],
 		"extras": {
 			"id": "qwen3.6",
@@ -342,7 +342,7 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 			"maxTokensField": "max_tokens"
 		},
 		"notes": [
-			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
+			"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4).",
 			"contextWindow 262,144: the earlier 1,000,000 override (maintainer-confirmed 2026-09-05) was withdrawn 2026-09-07 — the updated https://nan.builders/docs/models still states '262K token context, the model's native window' and models.dev agrees at 262,144; NaN docs are treated as the most reliable source (maintainer instruction, 2026-09-07)."
 		],
 		"extras": {
@@ -382,13 +382,13 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
 export const GENERATED_CATALOG_META = {
 	source: "https://models.dev/api.json",
 	modelsDevProvider: "nan",
-	fetchedAt: "2026-09-13T16:55:47.183Z",
+	fetchedAt: "2026-09-13T18:08:44.200Z",
 	modelCount: 6,
 	models: ["deepseek-v4-flash","gemma4","glm5.3-flash","mimo-v2.5","qwen3.6","qwen3.8-flash"],
 	notes: [
 	"live-only: \"glm5.3\" kept out of the static catalog (premium-tier model (models.dev now documents it with 1M context / 131,072 max output; NaN docs https://nan.builders/docs/models + https://nan.builders/openapi.json, checked 2026-09-13) kept live-only so a non-premium key never sees a model it cannot call when the live /models fetch is unavailable; premium keys still get it via the /models refresh with conservative placeholder limits)",
 	"provider-removed: \"glm5.2\" excluded from the catalog (removed by NaN (2026-09-05); absent from the official chat model list in https://nan.builders/openapi.json and https://nan.builders/docs/models (checked 2026-09-07) while models.dev provider nan still listed it — excluded so regeneration does not resurrect it)",
-	"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
+	"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13): pi-ai only sends stream_options when this is not false, and NaN's published schema does not document it, so the conservative default stays false (sanitizer removes stream_options, usage reads zero). A user who has confirmed that their model returns a streaming usage chunk can opt in per model with a models.json compat override (supportsUsageInStreaming: true); the sanitizer then forwards stream_options instead of deleting it (issue #4).",
 	"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models). models.dev provider nan also lists text+image now (DeepSeek V4.1 Flash entry, checked 2026-09-13; its 2026-09-07 snapshot listed text only), so this override is kept as a pin for the vision capability rather than as a divergence.",
 	"contextWindow 262,144: the earlier 1,000,000 override (maintainer-confirmed 2026-09-05) was withdrawn 2026-09-07 — the updated https://nan.builders/docs/models still states '262K token context, the model's native window' and models.dev agrees at 262,144; NaN docs are treated as the most reliable source (maintainer instruction, 2026-09-07)."
 ],

package/src/context-overflow-classifier.ts

@@ -0,0 +1,181 @@
+/**
+ * NaN-aware context-overflow classification.
+ *
+ * Root cause (pi-ai, still present on 0.85.1 / main): when history is replayed
+ * into a DIFFERENT model, `transformMessages` downgrades every non-redacted
+ * `thinking` block to plain `text` verbatim, with no size bound — and nothing
+ * bounds the SUM across messages. Switching from a 1M-context model
+ * (`glm5.3-flash`, `deepseek-v4-flash`) to a 262K one (`qwen3.6`) can therefore
+ * push the request past the destination window (see
+ * `src/cross-model-thinking-guard.ts` for the primary mitigation, which drops
+ * that replayed reasoning before pi-ai converts it).
+ *
+ * When the request still overflows — the guard is disabled
+ * (`NAN_THINKING_GUARD=0`), the trace is not a `thinking` block (large tool
+ * outputs, images), or the destination window is simply smaller — NaN's LiteLLM
+ * gateway answers the generic
+ *   HTTP 400 `Invalid request. Check your request parameters.`
+ * instead of naming the context overflow. pi's auto-compaction keys on pi-ai's
+ * `isContextOverflow()`, whose documented patterns do NOT match that text, so
+ * the session wedges permanently at the ceiling (the issue's upstream
+ * earendil-works/pi#9409, "Sessions wedge permanently at the context ceiling on
+ * reasoning models").
+ *
+ * This module makes the package **re-check the request size on the way out**
+ * (one of the two upstream fixes the tracking issue names). After the request is
+ * sent and the gateway answers, if
+ *   - the terminal assistant message is an error whose text carries NaN's
+ *     generic 400 marker, AND
+ *   - the request we sent was estimated to exceed the model's context window,
+ * then the error message is rewritten into a form that matches pi-ai's
+ * documented overflow patterns, so pi compacts and retries instead of wedging.
+ * The original provider text is preserved in the rewritten message. The
+ * reclassification is deliberately conservative: a generic 400 on a
+ * within-window request is left untouched, so unrelated 400s are never
+ * mislabelled as overflow.
+ *
+ * Estimation ratio: the same chars/3.47 the issue's offline session measurements
+ * use (`chars/3.47 + ~31K tools + ~12K system`). The system prompt and tool
+ * schemas are counted explicitly, so the constant is only applied to the
+ * message/character budget. This is a heuristic for deciding whether a generic
+ * 400 is plausibly an overflow — it never fabricates model metadata.
+ */
+
+import type {
+	AssistantMessage,
+	AssistantMessageEventStream,
+	Context,
+	Model,
+	ProviderStreams,
+} from "@earendil-works/pi-ai";
+
+/** The exact generic body NaN's gateway returns for an over-window request. */
+export const NAN_GENERIC_BAD_REQUEST = "Invalid request. Check your request parameters.";
+
+/**
+ * Characters per token used for the request-size estimate. Same ratio as the
+ * issue's offline measurements (chars/3.47); a conservative, provider-agnostic
+ * heuristic, never a capability claim.
+ */
+export const ESTIMATED_CHARS_PER_TOKEN = 3.47;
+
+/**
+ * Estimate the input tokens of an outgoing request from its character size.
+ * Returns 0 when the context cannot be serialized (nothing is reclassified).
+ */
+export function estimateRequestTokens(
+	context: Pick<Context, "systemPrompt" | "messages" | "tools"> | undefined,
+): number {
+	if (!context) return 0;
+	const systemChars = typeof context.systemPrompt === "string" ? context.systemPrompt.length : 0;
+	const toolsChars = safeJsonLength(context.tools);
+	const messagesChars = safeJsonLength(context.messages);
+	const total = systemChars + toolsChars + messagesChars;
+	if (!Number.isFinite(total) || total <= 0) return 0;
+	return Math.ceil(total / ESTIMATED_CHARS_PER_TOKEN);
+}
+
+function safeJsonLength(value: unknown): number {
+	if (value === undefined || value === null) return 0;
+	if (Array.isArray(value) && value.length === 0) return 0;
+	try {
+		return JSON.stringify(value)?.length ?? 0;
+	} catch {
+		return 0;
+	}
+}
+
+/**
+ * True when a terminal error message carries NaN's generic-400 marker (with or
+ * without the `400: {...}` wrapper pi-ai adds).
+ */
+export function isGenericNanBadRequest(message: Pick<AssistantMessage, "errorMessage"> | undefined): boolean {
+	return typeof message?.errorMessage === "string" && message.errorMessage.includes(NAN_GENERIC_BAD_REQUEST);
+}
+
+/**
+ * Reclassify a terminal assistant error as a context overflow when — and only
+ * when — NaN returned its generic 400 for a request estimated to exceed the
+ * model's context window. Mutates and returns the message (the same object
+ * flows through the error event and `result()`).
+ */
+export function classifyContextOverflowError<T extends AssistantMessage>(
+	message: T,
+	model: Pick<Model<"openai-completions">, "contextWindow"> | { contextWindow?: number } | undefined,
+	estimatedInputTokens: number,
+): T {
+	if (message.stopReason !== "error") return message;
+	if (!isGenericNanBadRequest(message)) return message;
+	const contextWindow = model?.contextWindow ?? 0;
+	if (!Number.isFinite(contextWindow) || contextWindow <= 0) return message;
+	if (!Number.isFinite(estimatedInputTokens) || estimatedInputTokens <= contextWindow) return message;
+
+	message.errorMessage =
+		`Requested token count exceeds the model's maximum context length of ${contextWindow} tokens ` +
+		`(estimated ${Math.ceil(estimatedInputTokens)} input tokens). ` +
+		`NaN's gateway answered HTTP 400 "${NAN_GENERIC_BAD_REQUEST}" instead of naming the overflow, ` +
+		`so the context must be compacted before retrying.`;
+	return message;
+}
+
+/**
+ * Wrap a provider stream so its terminal error is classified against the size
+ * of the request that produced it. Both the `error` event and `result()` yield
+ * the same (mutated) message, so iterating consumers and `result()`-only
+ * consumers agree.
+ */
+export function classifyStreamContextOverflow(
+	stream: AssistantMessageEventStream,
+	model: { contextWindow?: number } | undefined,
+	estimatedInputTokens: number,
+): AssistantMessageEventStream {
+	const rewrite = (message: AssistantMessage | undefined): AssistantMessage | undefined => {
+		if (message) classifyContextOverflowError(message, model, estimatedInputTokens);
+		return message;
+	};
+	const resultPromise = stream.result().then(rewrite);
+
+	return new Proxy(stream, {
+		get(target, property, receiver) {
+			if (property === "result") return () => resultPromise;
+			if (property === Symbol.asyncIterator) {
+				return () => {
+					const iterator = target[Symbol.asyncIterator]();
+					return {
+						async next() {
+							const step = await iterator.next();
+							if (!step.done && step.value?.type === "error") rewrite(step.value.error);
+							return step;
+						},
+						async return(value?: unknown) {
+							return iterator.return ? iterator.return(value) : { done: true, value };
+						},
+						async throw(error?: unknown) {
+							if (iterator.throw) return iterator.throw(error);
+							throw error;
+						},
+						[Symbol.asyncIterator]() {
+							return this;
+						},
+					};
+				};
+			}
+			const value = Reflect.get(target, property, receiver);
+			return typeof value === "function" ? value.bind(target) : value;
+		},
+	}) as AssistantMessageEventStream;
+}
+
+/**
+ * Wrap every `stream`/`streamSimple` call so a NaN over-window request whose
+ * gateway answer is the opaque generic 400 is surfaced as a context overflow.
+ */
+export function withContextOverflowClassification(api: ProviderStreams): ProviderStreams {
+	return {
+		...api,
+		stream: (model, context, options) =>
+			classifyStreamContextOverflow(api.stream(model, context, options), model, estimateRequestTokens(context)),
+		streamSimple: (model, context, options) =>
+			classifyStreamContextOverflow(api.streamSimple(model, context, options), model, estimateRequestTokens(context)),
+	};
+}

package/src/fetch-models.ts

@@ -206,8 +206,10 @@ export function mergeLiveWithGenerated(
 			// NAN_COMPAT: the NaN/LiteLLM gateway cuts SSE streams before
 			// finish_reason, so supportsFinishReason must stay true (pi-ai then
 			// raises the retryable "Stream ended without finish_reason" instead
-			// of silently stalling), and supportsUsageInStreaming must stay false
-			// because src/openai-compat-sanitizer.ts strips stream_options.
+			// of silently stalling), and supportsUsageInStreaming stays false
+			// because NaN's schema does not document `stream_options`: the
+			// sanitizer strips it unless the model's effective compat opts in
+			// (issue #4), so false matches the default wire payload.
 			models.push({
 				id,
 				name: id,

package/src/openai-compat-sanitizer.ts

@@ -34,9 +34,13 @@
  *  4. Top-level fields NaN's schema does not list: `store` and
  *     `stream_options`. These are opt-in/usage fields pi-ai sends by default
  *     for a "standard" provider; NaN does not document them, so they are
- *     removed. (Removing `stream_options` only costs live token-usage in the
- *     stream; NaN models are membership-quota based with zero per-token cost,
- *     so this is a safe trade.)
+ *     removed by default. `stream_options` is the one exception that can be
+ *     explicitly opted into: when the model's effective
+ *     `compat.supportsUsageInStreaming` is true (a catalog flag or a user
+ *     `models.json` override), the caller has confirmed the gateway reports
+ *     usage and `stream_options` is preserved — deleting it unconditionally
+ *     would silently zero out `message.usage` (issue #4). `store` is always
+ *     removed.
  *  5. An EMPTY `tools` array. Verified against the live gateway (2026-09-09):
  *     NaN rejects `tools: []` with the same 400, while `stream: true`, a
  *     `system` message, string content, and a `tool` role message are all
@@ -183,12 +187,23 @@ function sanitizeMessage(message: unknown): unknown {
 	return out;
 }
 
+export interface SanitizerOptions {
+	/**
+	 * Keep `stream_options` in the payload. Set only when the model's effective
+	 * `compat.supportsUsageInStreaming` is true: pi-ai then asks the gateway for
+	 * `stream_options: { include_usage: true }` and expects the terminal usage
+	 * chunk. When false/absent, `stream_options` is stripped from the strict
+	 * NaN payload as before.
+	 */
+	preserveStreamOptions?: boolean;
+}
+
 /**
  * Rewrite an OpenAI-compatible `/chat/completions` payload so every field
  * conforms to NaN's published schema. Returns the updated payload; if the
  * payload has no `messages` array it is returned unchanged.
  */
-export function sanitizeOpenAICompatPayload(payload: unknown): unknown {
+export function sanitizeOpenAICompatPayload(payload: unknown, options: SanitizerOptions = {}): unknown {
 	if (!isObject(payload) || !Array.isArray(payload.messages)) return payload;
 
 	const messages = payload.messages.map(sanitizeMessage);
@@ -221,7 +236,11 @@ export function sanitizeOpenAICompatPayload(payload: unknown): unknown {
 
 	// Top-level fields absent from NaN's schema.
 	delete out.store;
-	delete out.stream_options;
+	// `stream_options` is preserved only when the model declares that the
+	// gateway reports usage in the stream (issue #4). Otherwise it is stripped:
+	// pi-ai would not send it for such a model anyway, and a hand-built payload
+	// must stay schema-strict.
+	if (options.preserveStreamOptions !== true) delete out.stream_options;
 	// NaN documents `max_tokens`, not `max_completion_tokens`.
 	if ("max_completion_tokens" in out && !("max_tokens" in out)) {
 		out.max_tokens = out.max_completion_tokens;

package/src/provider-factory.ts

@@ -26,6 +26,7 @@ import type {
 	ProviderStreams,
 	RefreshModelsContext,
 } from "@earendil-works/pi-ai";
+import { withContextOverflowClassification } from "./context-overflow-classifier.ts";
 import {
 	baselineModels,
 	DEFAULT_MODELS_TIMEOUT_MS,
@@ -92,6 +93,12 @@ export async function resolveOpenAICompletionsApi(): Promise<OpenAICompletionsAp
  * which pi-ai version the runtime bundles, so the fix is not tied to a
  * specific upstream build.
  *
+ * `stream_options` is the one field whose removal is conditional: when the
+ * model's effective `compat.supportsUsageInStreaming` is true (catalog value
+ * or user `models.json` override), pi-ai requested usage and the gateway will
+ * return it — stripping the field would silently zero `message.usage`
+ * (issue #4). Every other model keeps the strict payload.
+ *
  * Any caller-supplied `onPayload` (e.g. pi's own debug/passthrough hook) is
  * preserved and chained AFTER sanitization, so the final payload is always
  * schema-valid.
@@ -100,13 +107,27 @@ function isObject(value: unknown): value is Record<string, unknown> {
 	return typeof value === "object" && value !== null;
 }
 
+/**
+ * Whether the model's effective compat asks pi-ai for streaming usage. pi-ai
+ * emits `stream_options: { include_usage: true }` when this is not false, and
+ * `models.json` overrides compose above the registered catalog, so this reads
+ * the user's confirmed value rather than the generated default.
+ */
+function modelSupportsUsageInStreaming(model: unknown): boolean {
+	if (!isObject(model)) return false;
+	const compat = model.compat;
+	return isObject(compat) && compat.supportsUsageInStreaming === true;
+}
+
 export function wrapApiForStrictSanitization(api: ProviderStreams): ProviderStreams {
 	const withSanitizer = <TOptions extends object | undefined>(options: TOptions): TOptions => {
 		const userOnPayload = isObject(options) ? (options.onPayload as unknown) : undefined;
 		return {
 			...((options ?? {}) as Record<string, unknown>),
 			onPayload: async (payload: unknown, model: unknown) => {
-				const sanitized = sanitizeOpenAICompatPayload(payload);
+				const sanitized = sanitizeOpenAICompatPayload(payload, {
+					preserveStreamOptions: modelSupportsUsageInStreaming(model),
+				});
 				if (typeof userOnPayload === "function") {
 					const userResult = await (userOnPayload as (p: unknown, m: unknown) => unknown)(sanitized, model);
 					return userResult ?? sanitized;
@@ -173,6 +194,12 @@ export async function createNanCompatibleProvider(
 			const current = liveIds;
 			return current ? models.filter((model) => current.has(model.id)) : models;
 		},
-		api: wrapApiForStrictSanitization(apiFactory()),
+		// Sanitize the payload for NaN's strict schema, then classify an opaque
+		// generic 400 as a context overflow when the request we just sent was over
+		// the model's window. The second layer keeps a replayed cross-model
+		// reasoning trace (or any other over-window request the context-hook guard
+		// cannot reach, e.g. NAN_THINKING_GUARD=0) recoverable instead of wedging
+		// the session — see src/context-overflow-classifier.ts.
+		api: withContextOverflowClassification(wrapApiForStrictSanitization(apiFactory())),
 	});
 }