@gtrabanco/pi-nan-provider 0.6.3 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +36 -1
- package/README.es.md +24 -2
- package/README.md +24 -2
- package/package.json +1 -1
- package/scripts/generate-models.ts +42 -14
- package/scripts/manual-overrides.ts +5 -1
- package/scripts/models.generated.ts +32 -31
- package/src/cross-model-thinking-guard.ts +25 -45
- package/src/fetch-models.ts +8 -6
- package/src/index.ts +12 -15
package/AGENTS.md
CHANGED
|
@@ -37,6 +37,27 @@ Regenerate the catalog after touching `scripts/generate-models.ts`:
|
|
|
37
37
|
bun run generate-models
|
|
38
38
|
```
|
|
39
39
|
|
|
40
|
+
## Live NaN API during diagnosis
|
|
41
|
+
|
|
42
|
+
Diagnostic calls against the real gateway are allowed — they spend the
|
|
43
|
+
maintainer's quota, so they are **permission-gated**:
|
|
44
|
+
|
|
45
|
+
- **Ask the maintainer before running any live probe, with an approximate token
|
|
46
|
+
cost (input + output).** No silent probing. If the cost is not worth it, report
|
|
47
|
+
the behavior to NaN and let them reproduce it instead of debugging it here.
|
|
48
|
+
- **Tests must never hit the network.** `bunfig.toml` preloads
|
|
49
|
+
`test/network-guard.ts`, which makes any un-injected `fetch` throw. Keep it:
|
|
50
|
+
inject `fetchImpl` / `options.fetch`, or use the local fixture. The permission
|
|
51
|
+
gate covers ad-hoc diagnosis only, never `bun test`.
|
|
52
|
+
- **Default to `qwen3.6` — it is unlimited.**
|
|
53
|
+
- **For massive/bulk probes prefer a model the maintainer uses less with a large
|
|
54
|
+
token budget, e.g. `mimo-v2.5`** (1M context).
|
|
55
|
+
- **When the model under investigation is the point** (e.g. reproducing a
|
|
56
|
+
model-specific 400), use it, but minimize tokens: smallest viable prompt,
|
|
57
|
+
lowest `max_tokens`, stop at the first decisive response.
|
|
58
|
+
- Repro commands that run a real `pi` session (`pi --fork ... -p ...`) use the
|
|
59
|
+
same key; keep them minimal and delete the forked session files afterwards.
|
|
60
|
+
|
|
40
61
|
## One shared implementation for all providers
|
|
41
62
|
|
|
42
63
|
`nan` (and any future provider, e.g. `helmcode`) must stay behind the single shared
|
|
@@ -90,7 +111,9 @@ Every PR that changes code MUST bump `package.json` version in the same PR; CI p
|
|
|
90
111
|
`MANUAL_OVERRIDES` (mandatory provenance note) in `scripts/manual-overrides.ts`,
|
|
91
112
|
applied by `scripts/generate-models.ts` — never hand-edited into
|
|
92
113
|
`scripts/models.generated.ts` and never invented. e.g. deepseek-v4-flash
|
|
93
|
-
image input (Vision-Exp variant; models.dev lists text
|
|
114
|
+
image input (Vision-Exp variant; models.dev now also lists text+image,
|
|
115
|
+
checked 2026-09-13, so the override is kept as a pin rather than a
|
|
116
|
+
divergence). Re-verify
|
|
94
117
|
overrides when the sources update: the qwen3.8-flash contextWindow 1,000,000
|
|
95
118
|
override (maintainer-confirmed 2026-09-05) was withdrawn 2026-09-07 — the
|
|
96
119
|
updated NaN docs still say 262K "the model's native window" and models.dev
|
|
@@ -100,10 +123,22 @@ Every PR that changes code MUST bump `package.json` version in the same PR; CI p
|
|
|
100
123
|
community `deepseek-v4-flash`, `mimo-v2.5`, `qwen3.8-flash`, `glm5.3-flash`,
|
|
101
124
|
`qwen3.6`, `gemma4` (all text+image vision) + premium-tier `glm5.3`
|
|
102
125
|
(~753B MoE, text-only input, 1M context, 400M tokens/rolling 4h window).
|
|
126
|
+
glm5.3 is now documented by models.dev too (1M context / 131,072 max output,
|
|
127
|
+
checked 2026-09-13) but stays out of the static catalog via
|
|
128
|
+
`LIVE_ONLY_MODEL_IDS` (premium tier) so a non-premium key never sees a
|
|
129
|
+
model it cannot call when the live `/models` fetch is unavailable; premium
|
|
130
|
+
keys still receive it live with conservative placeholder limits.
|
|
103
131
|
`glm5.2` was removed by the provider (2026-09-05); models.dev may still list
|
|
104
132
|
it — the generator excludes it via `PROVIDER_REMOVED_MODEL_IDS`. Non-chat
|
|
105
133
|
endpoints: qwen3-embedding, rerank, kokoro (TTS), whisper (STT),
|
|
106
134
|
flux-2-klein (images) — MCP-bridge territory, not chat catalog models.
|
|
135
|
+
- NaN can close an SSE stream **before** `finish_reason`. The catalog sets
|
|
136
|
+
`supportsFinishReason: true` so pi-ai raises the retryable
|
|
137
|
+
`Stream ended without finish_reason` (pi-ai's `RETRYABLE_PROVIDER_ERROR_PATTERN`
|
|
138
|
+
matches `"ended without"`, so the turn is retried) instead of silently
|
|
139
|
+
synthesizing `stop`/`toolUse`. `supportsUsageInStreaming` is `false` because
|
|
140
|
+
`src/openai-compat-sanitizer.ts` strips `stream_options`. Regression tests:
|
|
141
|
+
`test/issue-2-truncated-stream.test.ts`; issue #2.
|
|
107
142
|
- Relative imports inside this package use `.ts` extensions (pi's official
|
|
108
143
|
extension examples do the same; pi transpiles extension sources).
|
|
109
144
|
- pi intentionally has NO built-in MCP client (docs/usage.md). MCP integration
|
package/README.es.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# @gtrabanco/pi-nan-provider
|
|
2
2
|
|
|
3
3
|
[](https://opensource.org/licenses/MIT)
|
|
4
|
-
[](https://github.com/gtrabanco/pi-nan-provider/releases)
|
|
5
5
|
|
|
6
6
|
[NaN Builders](https://nan.builders) model provider + MCP bridges para [pi](https://github.com/earendil-works/pi).
|
|
7
7
|
|
|
@@ -47,7 +47,29 @@ El registro es síncrono a propósito: el catálogo de fallback está disponible
|
|
|
47
47
|
|
|
48
48
|
Al cambiar de modelo, pi-ai reenvía el razonamiento del modelo anterior como texto plano de asistente — **sin límite de tamaño**. Un único razonamiento largo o degenerado puede desbordar la ventana de un modelo de 262K, y NaN responde con un `400 Invalid request. Check your request parameters.` genérico que parece un bug del proveedor (seguimiento upstream: [pi-nan-provider#3](https://github.com/gtrabanco/pi-nan-provider/issues/3); issue abierta upstream: [pi#6167](https://github.com/earendil-works/pi/issues/6167)).
|
|
49
49
|
|
|
50
|
-
Este paquete
|
|
50
|
+
Este paquete **elimina todos los bloques de razonamiento cross-model reenviados**, de modo que cambiar de un modelo de 1M de contexto a uno de 262K (`qwen3.6`) ya no desborda la ventana. Las respuestas y los tool results de los modelos no se tocan — solo se quitan sus trazas internas de razonamiento, así que `qwen3.6` puede seguir respondiendo sobre lo que hizo otro modelo. El razonamiento del mismo modelo no se toca nunca, y el guard solo actúa sobre peticiones dirigidas a los proveedores de este paquete. Pon `NAN_THINKING_GUARD=0` para desactivarlo.
|
|
51
|
+
|
|
52
|
+
### ⏱️ Streams truncados intermitentes (auto-retry, sin stall silencioso)
|
|
53
|
+
|
|
54
|
+
El gateway LiteLLM de NaN cierra ocasionalmente un stream SSE **antes** de emitir el chunk final `finish_reason` (observado en `glm5.3-flash`; [issue #2](https://github.com/gtrabanco/pi-nan-provider/issues/2)). El catálogo declara `supportsFinishReason: true`, así que pi-ai lo convierte en el error `Stream ended without finish_reason` — que coincide con el patrón de errores reintentables de pi y se **reintenta automáticamente**, en vez de aceptar en silencio una respuesta a medias. Si una versión del gateway nunca manda `finish_reason`, el turno ahora falla de forma visible al agotar los reintentos.
|
|
55
|
+
|
|
56
|
+
Puedes sobrescribir el `compat` de cualquier modelo en `~/.pi/agent/models.json` (docs de pi → Per-model Overrides); los overrides se componen por encima del proveedor registrado. Ejemplo (forzando el comportamiento de retry explícitamente):
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{
|
|
60
|
+
"providers": {
|
|
61
|
+
"nan": {
|
|
62
|
+
"modelOverrides": {
|
|
63
|
+
"glm5.3-flash": { "compat": { "supportsFinishReason": true } }
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
> Poner `supportsFinishReason: false` restaura el antiguo stall silencioso — no recomendado.
|
|
71
|
+
|
|
72
|
+
**Uso de tokens en streaming:** `supportsUsageInStreaming` es `false` porque el esquema estricto de NaN no documenta `stream_options` y el sanitizer de peticiones lo elimina. Por eso el usage aparece a cero; la flag ahora coincide con lo que realmente se envía.
|
|
51
73
|
|
|
52
74
|
## 🔑 Autenticación
|
|
53
75
|
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# @gtrabanco/pi-nan-provider
|
|
2
2
|
|
|
3
3
|
[](https://opensource.org/licenses/MIT)
|
|
4
|
-
[](https://github.com/gtrabanco/pi-nan-provider/releases)
|
|
5
5
|
|
|
6
6
|
[NaN Builders](https://nan.builders) model provider + MCP bridges for [pi](https://github.com/earendil-works/pi).
|
|
7
7
|
|
|
@@ -47,7 +47,29 @@ The registration is synchronous on purpose: the generated fallback catalog is av
|
|
|
47
47
|
|
|
48
48
|
When you switch models, pi-ai replays the previous model's reasoning as plain assistant text — with **no size bound**. A single long or degenerate reasoning trace can therefore overflow a 262K-context model's window, and NaN answers with a generic `400 Invalid request. Check your request parameters.` that looks like a provider bug (upstream tracking: [pi-nan-provider#3](https://github.com/gtrabanco/pi-nan-provider/issues/3); open upstream issue: [pi#6167](https://github.com/earendil-works/pi/issues/6167)).
|
|
49
49
|
|
|
50
|
-
This package
|
|
50
|
+
This package **drops every replayed cross-model reasoning block**, so switching from a 1M-context model to a 262K one (`qwen3.6`) no longer overflows the window. The models' answers and tool results are untouched — only their internal reasoning traces are removed, so `qwen3.6` can still answer about what another model did. Same-model reasoning is never altered, and the guard only acts on requests targeting this package's providers. Set `NAN_THINKING_GUARD=0` to disable it.
|
|
51
|
+
|
|
52
|
+
### ⏱️ Intermittent truncated streams (auto-retry, no silent stall)
|
|
53
|
+
|
|
54
|
+
NaN's LiteLLM gateway occasionally closes an SSE stream **before** emitting the final `finish_reason` chunk (observed on `glm5.3-flash`; [issue #2](https://github.com/gtrabanco/pi-nan-provider/issues/2)). The catalog declares `supportsFinishReason: true`, so pi-ai turns that into the error `Stream ended without finish_reason` — which matches pi's retryable-provider pattern and is **retried automatically**, instead of silently accepting a half-finished answer. If a gateway version never sends `finish_reason`, the turn now fails visibly once the retry budget is exhausted.
|
|
55
|
+
|
|
56
|
+
You can override any model's `compat` per-model in `~/.pi/agent/models.json` (pi's `docs/models.md` → Per-model Overrides); overrides compose above the registered provider. Example (forcing the retry behavior explicitly):
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{
|
|
60
|
+
"providers": {
|
|
61
|
+
"nan": {
|
|
62
|
+
"modelOverrides": {
|
|
63
|
+
"glm5.3-flash": { "compat": { "supportsFinishReason": true } }
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
> Setting `supportsFinishReason: false` restores the old silent-stall behavior — not recommended.
|
|
71
|
+
|
|
72
|
+
**Streaming token usage:** `supportsUsageInStreaming` is `false` because NaN's strict schema does not document `stream_options` and the request sanitizer removes it. Usage therefore reads as zero; the flag now matches what is actually sent.
|
|
51
73
|
|
|
52
74
|
## 🔑 Authentication
|
|
53
75
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@gtrabanco/pi-nan-provider",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.5",
|
|
4
4
|
"description": "NaN Builders (api.nan.builders) model provider for pi - OpenAI-compatible registration with a models.dev-generated fallback, tier-aware live catalog, and MCP bridges (official web search + optional community media server)",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi",
|
|
@@ -57,15 +57,19 @@ const PROVIDER_REMOVED_MODEL_IDS: Record<string, string> = {
|
|
|
57
57
|
};
|
|
58
58
|
|
|
59
59
|
/**
|
|
60
|
-
*
|
|
61
|
-
*
|
|
62
|
-
*
|
|
63
|
-
*
|
|
64
|
-
*
|
|
60
|
+
* Premium/tier-gated models that models.dev documents but that are deliberately
|
|
61
|
+
* kept OUT of the static fallback catalog. The static baseline is what pi
|
|
62
|
+
* registers with zero network, and it must not advertise a premium model to a
|
|
63
|
+
* key that cannot call it: NaN's live `/models` response is the tier
|
|
64
|
+
* authority, so premium keys still receive these models through the live
|
|
65
|
+
* refresh with conservative placeholder limits (UNKNOWN_MODEL_LIMITS).
|
|
66
|
+
* Excluded at generation time with the recorded reason so a regeneration
|
|
67
|
+
* cannot resurrect them into the baseline.
|
|
65
68
|
*/
|
|
66
|
-
const
|
|
67
|
-
"glm5.3:
|
|
68
|
-
|
|
69
|
+
const LIVE_ONLY_MODEL_IDS: Record<string, string> = {
|
|
70
|
+
"glm5.3":
|
|
71
|
+
"premium-tier model (models.dev now documents it with 1M context / 131,072 max output; NaN docs https://nan.builders/docs/models + https://nan.builders/openapi.json, checked 2026-09-13) kept live-only so a non-premium key never sees a model it cannot call when the live /models fetch is unavailable; premium keys still get it via the /models refresh with conservative placeholder limits",
|
|
72
|
+
};
|
|
69
73
|
|
|
70
74
|
/**
|
|
71
75
|
* LiteLLM compat confirmed against the live api.nan.builders gateway by the
|
|
@@ -79,13 +83,23 @@ const KNOWN_UNEMITTABLE_MODEL_NOTES: readonly string[] = [
|
|
|
79
83
|
const NAN_COMPAT = {
|
|
80
84
|
supportsDeveloperRole: false,
|
|
81
85
|
supportsReasoningEffort: true,
|
|
82
|
-
|
|
83
|
-
|
|
86
|
+
// pi-ai only sends `stream_options: { include_usage: true }` when this is
|
|
87
|
+
// true, but src/openai-compat-sanitizer.ts strips `stream_options` before
|
|
88
|
+
// sending (NaN's schema does not document it). Declaring true would be a
|
|
89
|
+
// lie the sanitizer immediately undoes; false matches what is sent.
|
|
90
|
+
supportsUsageInStreaming: false,
|
|
91
|
+
// The NaN/LiteLLM gateway intermittently closes SSE streams before emitting
|
|
92
|
+
// `finish_reason`. With true, pi-ai raises "Stream ended without
|
|
93
|
+
// finish_reason", which its retryable-provider pattern ("ended without")
|
|
94
|
+
// matches, so the turn is retried automatically. With false, pi-ai
|
|
95
|
+
// silently synthesizes stop/toolUse and the turn stalls mid-answer
|
|
96
|
+
// (observed 2026-09-13 on glm5.3-flash; issue #2).
|
|
97
|
+
supportsFinishReason: true,
|
|
84
98
|
maxTokensField: "max_tokens" as const,
|
|
85
99
|
};
|
|
86
100
|
|
|
87
101
|
const NAN_COMPAT_NOTE =
|
|
88
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
102
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.";
|
|
89
103
|
|
|
90
104
|
interface ModelsDevModel {
|
|
91
105
|
id?: string;
|
|
@@ -144,6 +158,10 @@ function convertModel(modelId: string, m: ModelsDevModel): GeneratedModel | { sk
|
|
|
144
158
|
if (removedReason) {
|
|
145
159
|
return { skip: `provider-removed: "${modelId}" excluded from the catalog (${removedReason})` };
|
|
146
160
|
}
|
|
161
|
+
const liveOnlyReason = LIVE_ONLY_MODEL_IDS[modelId];
|
|
162
|
+
if (liveOnlyReason) {
|
|
163
|
+
return { skip: `live-only: "${modelId}" kept out of the static catalog (${liveOnlyReason})` };
|
|
164
|
+
}
|
|
147
165
|
const contextWindow = m.limit?.context;
|
|
148
166
|
const maxTokens = m.limit?.output;
|
|
149
167
|
if (typeof contextWindow !== "number" || contextWindow <= 0) {
|
|
@@ -225,9 +243,19 @@ async function main(): Promise<void> {
|
|
|
225
243
|
|
|
226
244
|
const fetchedAt = new Date().toISOString();
|
|
227
245
|
const allNotes = [
|
|
228
|
-
...
|
|
229
|
-
|
|
230
|
-
|
|
246
|
+
...new Set([
|
|
247
|
+
...skipped,
|
|
248
|
+
// Record every exclusion unconditionally (not only when models.dev
|
|
249
|
+
// still lists the id) so a regeneration can never drop the reason a
|
|
250
|
+
// model is absent from the catalog.
|
|
251
|
+
...Object.entries(PROVIDER_REMOVED_MODEL_IDS).map(
|
|
252
|
+
([id, reason]) => `provider-removed: "${id}" excluded from the catalog (${reason})`,
|
|
253
|
+
),
|
|
254
|
+
...Object.entries(LIVE_ONLY_MODEL_IDS).map(
|
|
255
|
+
([id, reason]) => `live-only: "${id}" kept out of the static catalog (${reason})`,
|
|
256
|
+
),
|
|
257
|
+
...entries.flatMap((entry) => entry.notes ?? []),
|
|
258
|
+
]),
|
|
231
259
|
];
|
|
232
260
|
|
|
233
261
|
const generated = `// This file is auto-generated by scripts/generate-models.ts
|
|
@@ -16,6 +16,10 @@
|
|
|
16
16
|
*
|
|
17
17
|
* Fields override the models.dev-derived entry one-for-one:
|
|
18
18
|
* name, reasoning, input, cost, contextWindow, maxTokens.
|
|
19
|
+
*
|
|
20
|
+
* An override may also act as a pin — a value models.dev already agrees with,
|
|
21
|
+
* kept so an upstream regression cannot silently drop a confirmed capability.
|
|
22
|
+
* A pin's note must say it is a pin; never present it as a divergence.
|
|
19
23
|
*/
|
|
20
24
|
|
|
21
25
|
export interface ManualModelOverride {
|
|
@@ -38,6 +42,6 @@ export interface ManualModelOverride {
|
|
|
38
42
|
export const MANUAL_OVERRIDES: Record<string, ManualModelOverride> = {
|
|
39
43
|
"deepseek-v4-flash": {
|
|
40
44
|
input: ["text", "image"],
|
|
41
|
-
note: "input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models
|
|
45
|
+
note: "input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models). models.dev provider nan also lists text+image now (DeepSeek V4.1 Flash entry, checked 2026-09-13; its 2026-09-07 snapshot listed text only), so this override is kept as a pin for the vision capability rather than as a divergence.",
|
|
42
46
|
},
|
|
43
47
|
};
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// This file is auto-generated by scripts/generate-models.ts
|
|
2
2
|
// Do not edit manually — run `bun run generate-models` to update.
|
|
3
3
|
//
|
|
4
|
-
// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-
|
|
4
|
+
// Source: https://models.dev/api.json (provider "nan"), fetched 2026-09-13T16:55:47.183Z
|
|
5
5
|
// Provenance: every contextWindow/maxTokens/input/cost value traces to
|
|
6
6
|
// models.dev or to the per-entry notes below. Nothing is invented; entries
|
|
7
7
|
// models.dev documents incompletely are omitted and flagged instead.
|
|
@@ -14,7 +14,7 @@ import type { GeneratedModelEntry } from "../src/fetch-models.ts";
|
|
|
14
14
|
export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
15
15
|
{
|
|
16
16
|
"id": "deepseek-v4-flash",
|
|
17
|
-
"name": "DeepSeek V4 Flash",
|
|
17
|
+
"name": "DeepSeek V4.1 Flash",
|
|
18
18
|
"reasoning": true,
|
|
19
19
|
"input": [
|
|
20
20
|
"text",
|
|
@@ -31,31 +31,32 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
31
31
|
"compat": {
|
|
32
32
|
"supportsDeveloperRole": false,
|
|
33
33
|
"supportsReasoningEffort": true,
|
|
34
|
-
"supportsUsageInStreaming":
|
|
35
|
-
"supportsFinishReason":
|
|
34
|
+
"supportsUsageInStreaming": false,
|
|
35
|
+
"supportsFinishReason": true,
|
|
36
36
|
"maxTokensField": "max_tokens"
|
|
37
37
|
},
|
|
38
38
|
"notes": [
|
|
39
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
40
|
-
"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models
|
|
39
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
|
|
40
|
+
"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models). models.dev provider nan also lists text+image now (DeepSeek V4.1 Flash entry, checked 2026-09-13; its 2026-09-07 snapshot listed text only), so this override is kept as a pin for the vision capability rather than as a divergence."
|
|
41
41
|
],
|
|
42
42
|
"extras": {
|
|
43
43
|
"id": "deepseek-v4-flash",
|
|
44
|
-
"name": "DeepSeek V4 Flash",
|
|
45
|
-
"description": "
|
|
44
|
+
"name": "DeepSeek V4.1 Flash",
|
|
45
|
+
"description": "DeepSeek V4.1 Flash model for reasoning and agentic coding",
|
|
46
46
|
"family": "deepseek-flash",
|
|
47
|
-
"attachment":
|
|
47
|
+
"attachment": true,
|
|
48
48
|
"reasoning": true,
|
|
49
49
|
"reasoning_options": [],
|
|
50
50
|
"tool_call": true,
|
|
51
51
|
"structured_output": true,
|
|
52
52
|
"temperature": true,
|
|
53
53
|
"knowledge": "2025-05",
|
|
54
|
-
"release_date": "2026-
|
|
55
|
-
"last_updated": "2026-
|
|
54
|
+
"release_date": "2026-09-10",
|
|
55
|
+
"last_updated": "2026-09-10",
|
|
56
56
|
"modalities": {
|
|
57
57
|
"input": [
|
|
58
|
-
"text"
|
|
58
|
+
"text",
|
|
59
|
+
"image"
|
|
59
60
|
],
|
|
60
61
|
"output": [
|
|
61
62
|
"text"
|
|
@@ -91,12 +92,12 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
91
92
|
"compat": {
|
|
92
93
|
"supportsDeveloperRole": false,
|
|
93
94
|
"supportsReasoningEffort": true,
|
|
94
|
-
"supportsUsageInStreaming":
|
|
95
|
-
"supportsFinishReason":
|
|
95
|
+
"supportsUsageInStreaming": false,
|
|
96
|
+
"supportsFinishReason": true,
|
|
96
97
|
"maxTokensField": "max_tokens"
|
|
97
98
|
},
|
|
98
99
|
"notes": [
|
|
99
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
100
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
|
|
100
101
|
],
|
|
101
102
|
"extras": {
|
|
102
103
|
"id": "gemma4",
|
|
@@ -154,12 +155,12 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
154
155
|
"compat": {
|
|
155
156
|
"supportsDeveloperRole": false,
|
|
156
157
|
"supportsReasoningEffort": true,
|
|
157
|
-
"supportsUsageInStreaming":
|
|
158
|
-
"supportsFinishReason":
|
|
158
|
+
"supportsUsageInStreaming": false,
|
|
159
|
+
"supportsFinishReason": true,
|
|
159
160
|
"maxTokensField": "max_tokens"
|
|
160
161
|
},
|
|
161
162
|
"notes": [
|
|
162
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
163
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
|
|
163
164
|
],
|
|
164
165
|
"extras": {
|
|
165
166
|
"id": "glm5.3-flash",
|
|
@@ -213,12 +214,12 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
213
214
|
"compat": {
|
|
214
215
|
"supportsDeveloperRole": false,
|
|
215
216
|
"supportsReasoningEffort": true,
|
|
216
|
-
"supportsUsageInStreaming":
|
|
217
|
-
"supportsFinishReason":
|
|
217
|
+
"supportsUsageInStreaming": false,
|
|
218
|
+
"supportsFinishReason": true,
|
|
218
219
|
"maxTokensField": "max_tokens"
|
|
219
220
|
},
|
|
220
221
|
"notes": [
|
|
221
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
222
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
|
|
222
223
|
],
|
|
223
224
|
"extras": {
|
|
224
225
|
"id": "mimo-v2.5",
|
|
@@ -273,12 +274,12 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
273
274
|
"compat": {
|
|
274
275
|
"supportsDeveloperRole": false,
|
|
275
276
|
"supportsReasoningEffort": true,
|
|
276
|
-
"supportsUsageInStreaming":
|
|
277
|
-
"supportsFinishReason":
|
|
277
|
+
"supportsUsageInStreaming": false,
|
|
278
|
+
"supportsFinishReason": true,
|
|
278
279
|
"maxTokensField": "max_tokens"
|
|
279
280
|
},
|
|
280
281
|
"notes": [
|
|
281
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
282
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect."
|
|
282
283
|
],
|
|
283
284
|
"extras": {
|
|
284
285
|
"id": "qwen3.6",
|
|
@@ -336,12 +337,12 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
336
337
|
"compat": {
|
|
337
338
|
"supportsDeveloperRole": false,
|
|
338
339
|
"supportsReasoningEffort": true,
|
|
339
|
-
"supportsUsageInStreaming":
|
|
340
|
-
"supportsFinishReason":
|
|
340
|
+
"supportsUsageInStreaming": false,
|
|
341
|
+
"supportsFinishReason": true,
|
|
341
342
|
"maxTokensField": "max_tokens"
|
|
342
343
|
},
|
|
343
344
|
"notes": [
|
|
344
|
-
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true,
|
|
345
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
|
|
345
346
|
"contextWindow 262,144: the earlier 1,000,000 override (maintainer-confirmed 2026-09-05) was withdrawn 2026-09-07 — the updated https://nan.builders/docs/models still states '262K token context, the model's native window' and models.dev agrees at 262,144; NaN docs are treated as the most reliable source (maintainer instruction, 2026-09-07)."
|
|
346
347
|
],
|
|
347
348
|
"extras": {
|
|
@@ -381,14 +382,14 @@ export const NAN_GENERATED_MODELS: readonly GeneratedModelEntry[] = [
|
|
|
381
382
|
export const GENERATED_CATALOG_META = {
|
|
382
383
|
source: "https://models.dev/api.json",
|
|
383
384
|
modelsDevProvider: "nan",
|
|
384
|
-
fetchedAt: "2026-09-
|
|
385
|
+
fetchedAt: "2026-09-13T16:55:47.183Z",
|
|
385
386
|
modelCount: 6,
|
|
386
387
|
models: ["deepseek-v4-flash","gemma4","glm5.3-flash","mimo-v2.5","qwen3.6","qwen3.8-flash"],
|
|
387
388
|
notes: [
|
|
389
|
+
"live-only: \"glm5.3\" kept out of the static catalog (premium-tier model (models.dev now documents it with 1M context / 131,072 max output; NaN docs https://nan.builders/docs/models + https://nan.builders/openapi.json, checked 2026-09-13) kept live-only so a non-premium key never sees a model it cannot call when the live /models fetch is unavailable; premium keys still get it via the /models refresh with conservative placeholder limits)",
|
|
388
390
|
"provider-removed: \"glm5.2\" excluded from the catalog (removed by NaN (2026-09-05); absent from the official chat model list in https://nan.builders/openapi.json and https://nan.builders/docs/models (checked 2026-09-07) while models.dev provider nan still listed it — excluded so regeneration does not resurrect it)",
|
|
389
|
-
"
|
|
390
|
-
"
|
|
391
|
-
"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models, checked 2026-09-07; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models); models.dev provider nan lists text only.",
|
|
391
|
+
"compat matches the maintainer's working ~/.pi/agent/models.json LiteLLM config for api.nan.builders (2026-09-04): supportsDeveloperRole false, supportsReasoningEffort true, maxTokensField max_tokens. NaN's docs example sets only supportsDeveloperRole: true and is not battle-tested. supportsFinishReason true (2026-09-13, issue #2): the LiteLLM gateway intermittently closes SSE streams before emitting finish_reason; with true pi-ai raises 'Stream ended without finish_reason', which matches pi-ai's retryable-provider pattern ('ended without') and is retried automatically, whereas false silently synthesized stop/toolUse and stalled the turn mid-answer. supportsUsageInStreaming false (2026-09-13, issue #2): pi-ai only sends stream_options when this is true, but src/openai-compat-sanitizer.ts strips stream_options before sending, so requesting it would contradict the sanitizer with no effect.",
|
|
392
|
+
"input includes image: NaN serves the Vision-Exp variant ('takes images as input', https://nan.builders/docs/models; the image_url content-parts in https://nan.builders/openapi.json list deepseek-v4-flash among the vision models). models.dev provider nan also lists text+image now (DeepSeek V4.1 Flash entry, checked 2026-09-13; its 2026-09-07 snapshot listed text only), so this override is kept as a pin for the vision capability rather than as a divergence.",
|
|
392
393
|
"contextWindow 262,144: the earlier 1,000,000 override (maintainer-confirmed 2026-09-05) was withdrawn 2026-09-07 — the updated https://nan.builders/docs/models still states '262K token context, the model's native window' and models.dev agrees at 262,144; NaN docs are treated as the most reliable source (maintainer instruction, 2026-09-07)."
|
|
393
394
|
],
|
|
394
395
|
} as const;
|
|
@@ -6,45 +6,36 @@
|
|
|
6
6
|
* every non-redacted `thinking` block to a plain `text` block verbatim
|
|
7
7
|
* (`packages/ai/src/api/transform-messages.ts`), and `openai-completions`
|
|
8
8
|
* serializes that into the assistant `content` string. Nothing bounds
|
|
9
|
-
* `block.thinking`,
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
9
|
+
* `block.thinking`, and nothing bounds the SUM across messages.
|
|
10
|
+
*
|
|
11
|
+
* Measured on real sessions: replayed reasoning is 30–60% of the whole context
|
|
12
|
+
* (e.g. 356,723 of 903,464 chars in one session; 349,882 of 749,525 in another).
|
|
13
|
+
* Switching from a 1M-context model (glm5.3-flash, deepseek-v4-flash) to a
|
|
14
|
+
* 262K-context one (`qwen3.6`) then overflows the window, and NaN's gateway
|
|
15
|
+
* answers a generic `400 Invalid request. Check your request parameters.`
|
|
16
|
+
*
|
|
17
|
+
* A per-block cap was tried first and is NOT enough: many medium blocks sum to
|
|
18
|
+
* hundreds of thousands of tokens. This guard therefore DROPS every replayed
|
|
19
|
+
* cross-model `thinking` block outright — it is the reasoning trace that
|
|
20
|
+
* pi-ai would have replayed as plain text, not the model's answers or tool
|
|
21
|
+
* results, so qwen can still answer questions about what glm/deepseek did.
|
|
22
|
+
* Same-model reasoning is never touched (signatures and continuity depend on
|
|
23
|
+
* it), and the guard only acts on requests targeting this package's providers.
|
|
17
24
|
*
|
|
18
25
|
* The extension runs in pi's `context` hook, which fires BEFORE pi-ai's
|
|
19
26
|
* `transformMessages` (pi-agent-core `transformContext` → `convertToLlm` →
|
|
20
27
|
* provider stream). It therefore sees the original `thinking` blocks and can
|
|
21
|
-
*
|
|
22
|
-
* differ from the target are touched; same-model replay keeps its reasoning
|
|
23
|
-
* byte-for-byte because signatures/continuity depend on it.
|
|
28
|
+
* remove them before they become text.
|
|
24
29
|
*
|
|
25
|
-
* This is a bounded mitigation, not
|
|
26
|
-
*
|
|
27
|
-
*
|
|
28
|
-
* https://github.com/earendil-works/pi/issues/9433).
|
|
30
|
+
* This is a bounded mitigation, not the fix: the unbounded conversion (and pi
|
|
31
|
+
* not re-checking the context size on a model switch) belongs upstream. See
|
|
32
|
+
* https://github.com/gtrabanco/pi-nan-provider/issues/3.
|
|
29
33
|
*/
|
|
30
34
|
|
|
31
35
|
/** Env var that opts out of the guard (`0`, `false`, `no` or `off`). Default: enabled. */
|
|
32
36
|
export const NAN_THINKING_GUARD_ENV = "NAN_THINKING_GUARD";
|
|
33
37
|
|
|
34
|
-
/**
|
|
35
|
-
* Maximum characters of a single cross-model reasoning block replayed as text.
|
|
36
|
-
* 16,000 chars is ~4K tokens — far above any real reasoning trace, far below
|
|
37
|
-
* the 445,888-char degenerate trace that caused the 400.
|
|
38
|
-
*/
|
|
39
|
-
export const MAX_CROSS_MODEL_THINKING_CHARS = 16_000;
|
|
40
|
-
|
|
41
|
-
/** Appended after the kept prefix so the substitution is visible, never silent. */
|
|
42
|
-
export const CROSS_MODEL_THINKING_TRUNCATION_MARKER =
|
|
43
|
-
"\n\n[…previous-model reasoning truncated by pi-nan-provider to keep the request within NaN's context]";
|
|
44
|
-
|
|
45
38
|
export interface CrossModelThinkingGuardOptions {
|
|
46
|
-
/** Max chars kept per replayed cross-model reasoning block. */
|
|
47
|
-
maxCharsPerBlock?: number;
|
|
48
39
|
/** Provider ids this guard applies to (the caller's registered providers). */
|
|
49
40
|
providerIds: ReadonlySet<string>;
|
|
50
41
|
}
|
|
@@ -57,7 +48,6 @@ interface GuardTarget {
|
|
|
57
48
|
|
|
58
49
|
interface ThinkingBlock {
|
|
59
50
|
type?: string;
|
|
60
|
-
thinking?: unknown;
|
|
61
51
|
[k: string]: unknown;
|
|
62
52
|
}
|
|
63
53
|
|
|
@@ -84,19 +74,17 @@ function isThinkingBlock(value: unknown): value is ThinkingBlock {
|
|
|
84
74
|
}
|
|
85
75
|
|
|
86
76
|
/**
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
*
|
|
90
|
-
* cloning the whole context on the common path. Never mutates the input.
|
|
77
|
+
* Remove every `thinking` block from assistant messages produced by a model
|
|
78
|
+
* other than the target. Returns the SAME array reference when nothing changed,
|
|
79
|
+
* so callers can skip cloning on the common path. Never mutates the input.
|
|
91
80
|
*/
|
|
92
|
-
export function
|
|
81
|
+
export function stripCrossModelThinking<T>(
|
|
93
82
|
messages: readonly T[],
|
|
94
83
|
target: GuardTarget | undefined,
|
|
95
84
|
options: CrossModelThinkingGuardOptions,
|
|
96
85
|
): readonly T[] {
|
|
97
86
|
if (!target?.provider || !options.providerIds.has(target.provider)) return messages;
|
|
98
87
|
|
|
99
|
-
const maxChars = options.maxCharsPerBlock ?? MAX_CROSS_MODEL_THINKING_CHARS;
|
|
100
88
|
let changed = false;
|
|
101
89
|
|
|
102
90
|
const next = messages.map((raw) => {
|
|
@@ -106,16 +94,8 @@ export function boundCrossModelThinking<T>(
|
|
|
106
94
|
const isSameModel = raw.provider === target.provider && raw.api === target.api && raw.model === target.id;
|
|
107
95
|
if (isSameModel) return raw;
|
|
108
96
|
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
if (!isThinkingBlock(block)) return block;
|
|
112
|
-
const text = block.thinking;
|
|
113
|
-
if (typeof text !== "string" || text.length <= maxChars) return block;
|
|
114
|
-
messageChanged = true;
|
|
115
|
-
return { ...block, thinking: text.slice(0, maxChars) + CROSS_MODEL_THINKING_TRUNCATION_MARKER };
|
|
116
|
-
});
|
|
117
|
-
|
|
118
|
-
if (!messageChanged) return raw;
|
|
97
|
+
const content = (raw.content as unknown[]).filter((block) => !isThinkingBlock(block));
|
|
98
|
+
if (content.length === (raw.content as unknown[]).length) return raw;
|
|
119
99
|
changed = true;
|
|
120
100
|
return { ...raw, content };
|
|
121
101
|
});
|
package/src/fetch-models.ts
CHANGED
|
@@ -201,11 +201,13 @@ export function mergeLiveWithGenerated(
|
|
|
201
201
|
} else {
|
|
202
202
|
// Conservative placeholder for allowlisted uncatalogued live ids
|
|
203
203
|
// (e.g. premium glm5.3): limits are the documented safe envelope and
|
|
204
|
-
// capabilities stay "unknown".
|
|
205
|
-
//
|
|
206
|
-
//
|
|
207
|
-
//
|
|
208
|
-
// ended without finish_reason"
|
|
204
|
+
// capabilities stay "unknown". The compat keys are the same
|
|
205
|
+
// gateway-behavior flags applied to every catalog model in
|
|
206
|
+
// NAN_COMPAT: the NaN/LiteLLM gateway cuts SSE streams before
|
|
207
|
+
// finish_reason, so supportsFinishReason must stay true (pi-ai then
|
|
208
|
+
// raises the retryable "Stream ended without finish_reason" instead
|
|
209
|
+
// of silently stalling), and supportsUsageInStreaming must stay false
|
|
210
|
+
// because src/openai-compat-sanitizer.ts strips stream_options.
|
|
209
211
|
models.push({
|
|
210
212
|
id,
|
|
211
213
|
name: id,
|
|
@@ -217,7 +219,7 @@ export function mergeLiveWithGenerated(
|
|
|
217
219
|
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
218
220
|
contextWindow: UNKNOWN_MODEL_LIMITS.contextWindow,
|
|
219
221
|
maxTokens: UNKNOWN_MODEL_LIMITS.maxTokens,
|
|
220
|
-
compat: { supportsFinishReason: false },
|
|
222
|
+
compat: { supportsFinishReason: true, supportsUsageInStreaming: false },
|
|
221
223
|
});
|
|
222
224
|
unknown.push(id);
|
|
223
225
|
}
|
package/src/index.ts
CHANGED
|
@@ -29,9 +29,8 @@ import type { ContextEvent, ExtensionAPI, ProviderConfig } from "@earendil-works
|
|
|
29
29
|
import type { Provider } from "@earendil-works/pi-ai";
|
|
30
30
|
import { registerNanMcpCommand } from "./commands.ts";
|
|
31
31
|
import {
|
|
32
|
-
boundCrossModelThinking,
|
|
33
32
|
crossModelThinkingGuardEnabled,
|
|
34
|
-
|
|
33
|
+
stripCrossModelThinking,
|
|
35
34
|
} from "./cross-model-thinking-guard.ts";
|
|
36
35
|
import { baselineModels } from "./fetch-models.ts";
|
|
37
36
|
import { createNanWebSearchTool, webSearchBridgeEnabled, NAN_API_KEY_ENV } from "./mcp/nan-search.ts";
|
|
@@ -114,29 +113,27 @@ function registerMcpToolsCompat(pi: ExtensionAPI): void {
|
|
|
114
113
|
}
|
|
115
114
|
|
|
116
115
|
/**
|
|
117
|
-
*
|
|
116
|
+
* Drop the reasoning pi-ai replays across a model switch.
|
|
118
117
|
*
|
|
119
118
|
* pi-ai's `transformMessages` downgrades a previous model's `thinking` blocks
|
|
120
|
-
* to plain text with no size bound (still true on 0.85.1 / main),
|
|
121
|
-
*
|
|
122
|
-
*
|
|
123
|
-
*
|
|
124
|
-
*
|
|
125
|
-
*
|
|
126
|
-
* the replayed context
|
|
119
|
+
* to plain text with no size bound (still true on 0.85.1 / main), and nothing
|
|
120
|
+
* bounds the sum across messages — measured at 30–60% of the whole context on
|
|
121
|
+
* real sessions. Switching from a 1M-context model to a 262K one (`qwen3.6`)
|
|
122
|
+
* then overflows the window, and NaN's gateway answers a generic
|
|
123
|
+
* `400 Invalid request. Check your request parameters.` This hook runs before
|
|
124
|
+
* pi-ai converts the blocks, so removing the cross-model reasoning here keeps
|
|
125
|
+
* the replayed context small. Answers and tool results are untouched. See
|
|
126
|
+
* src/cross-model-thinking-guard.ts.
|
|
127
127
|
*
|
|
128
128
|
* Scope: only requests targeting this package's providers are touched, and
|
|
129
|
-
* only messages from a DIFFERENT model — same-model reasoning is never
|
|
129
|
+
* only messages from a DIFFERENT model — same-model reasoning is never removed.
|
|
130
130
|
*/
|
|
131
131
|
export function registerCrossModelThinkingGuard(pi: ExtensionAPI): void {
|
|
132
132
|
if (typeof pi.on !== "function") return; // old pi without the context hook
|
|
133
133
|
const providerIds = new Set(PROVIDERS.map((provider) => provider.id));
|
|
134
134
|
pi.on("context", (event, ctx) => {
|
|
135
135
|
if (!crossModelThinkingGuardEnabled()) return;
|
|
136
|
-
const guarded =
|
|
137
|
-
maxCharsPerBlock: MAX_CROSS_MODEL_THINKING_CHARS,
|
|
138
|
-
providerIds,
|
|
139
|
-
});
|
|
136
|
+
const guarded = stripCrossModelThinking(event.messages, ctx.model, { providerIds });
|
|
140
137
|
if (guarded === event.messages) return;
|
|
141
138
|
return { messages: guarded as ContextEvent["messages"] };
|
|
142
139
|
});
|