@xingjisoft/neutrino 0.1.3 → 0.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -76,7 +76,7 @@ Type a message on the start screen and press `Enter` to begin a new chat.
76
76
  | Scroll wheel | Scroll the chat/log, or move through lists |
77
77
  | Right-click | Go back (like `Esc`) |
78
78
 
79
- The selection highlight is drawn over whatever is on screen, so it works uniformly on the chat log, the input box and every menu. Selected text is copied with trailing whitespace trimmed per line (shell-style), so it can be pasted straight back into a terminal.
79
+ The selection highlight is drawn over whatever is on screen, so it works uniformly on the chat log, the input box and every menu. Selected text is copied with trailing whitespace trimmed per line (shell-style), so it can be pasted straight back into a terminal. UI chrome (such as the scrollbar) is skipped when copying and highlighting, so a copy never picks up the bar or the padding in front of it.
80
80
 
81
81
  ## Configuration
82
82
 
@@ -121,17 +121,43 @@ node scripts/test-paste.mjs # verify pasted newlines are flattened to space
121
121
  node scripts/test-mouse.mjs # verify mouse escape-sequence parsing
122
122
  node scripts/test-cursor.mjs # verify click-to-position-cursor math
123
123
  node scripts/test-selection.mjs # verify selection highlighting & text extraction
124
+ node scripts/test-copy.mjs # verify a copy never includes the scrollbar or trailing padding
124
125
  node scripts/test-scrollbar.mjs # verify the conversation scrollbar size/position and drag maths
125
126
  node scripts/test-markdown.mjs # verify markdown/table rendering (incl. ragged rows)
126
127
  node scripts/test-wrap.mjs # verify indentation & spacing are preserved when wrapping
127
128
  node scripts/test-render.mjs # fuzz the diff renderer + render the chat screen,
128
129
  # asserting no stale/residual cells are left behind
130
+ node scripts/test-sanitize.mjs # verify ANSI/control codes never reach the screen
131
+ node scripts/test-drift.mjs # verify the input area self-heals after terminal drift
132
+ node scripts/test-fetch.mjs # verify streamed requests release aborts and retry drops
133
+ node scripts/test-compact.mjs # verify context-compression planning
134
+ node scripts/test-tools.mjs # verify tool-output folding
129
135
  ```
130
136
 
131
137
  ### Rendering model
132
138
 
133
139
  The screen is drawn into a `Buffer` of cells which is diffed against the previous frame, so only changed rows are emitted. Rows are erased to the end of the line before being repainted, and the buffer keeps double-width glyphs consistent (writing over one half of a wide character blanks the other half). Together these guarantee that a repainted row can never leave a stale cell behind.
134
140
 
141
+ Because only *changed* rows are emitted, a row whose contents happen to be identical from one frame to the next is never rewritten — so if the terminal shifts its own contents behind our back (ConPTY reflowing the buffer mid-resize, a size-race repaint, a stray scroll) that row would stay stale. Two defences cover this: the persistent chrome (input box, tabs, key hints) is repainted on every frame via `Screen.alwaysRepaintRows()`, and the app periodically re-emits the whole cached frame (`Renderer.repaint()`) as a safety net. Both are no-ops when nothing drifted.
142
+
143
+ ### Text hygiene
144
+
145
+ A cell's glyph is copied straight into the output stream, so a cell holding a *control* character becomes a command the terminal executes — an ESC sequence can move the cursor, clear the screen or leave the alternate buffer, and a TAB shifts the rest of the row. Tool output (ANSI colours from `ls --color`, `git`, `npm`, progress bars, …), file contents and pasted text all carry such bytes, so they are neutralised in three places:
146
+
147
+ 1. `src/sanitize.ts` — `sanitizeText()` removes whole escape sequences (not just the ESC byte, which would leave `[31m` behind as literal text), expands tabs so measured widths match what the terminal draws, and drops any remaining control code. It is applied where untrusted text enters the display (chat messages, tool results, pasted input).
148
+ 2. `Buffer` — writing a control character stores a space instead, preserving the column width the caller measured, so no cell can ever hold one.
149
+ 3. `Renderer` — a last-resort check when emitting a row, so the byte stream cannot carry a control code even if a cell were populated another way.
150
+
151
+ ### Network resilience
152
+
153
+ Model responses arrive over a streamed connection that can drop at any point. A single blip should not end the session, so three cases are handled:
154
+
155
+ - **The request never goes out** (`fetch` rejects) — retried a couple of times with a short backoff, bounded by a connect timeout.
156
+ - **The stream dies before the first byte** — the server or a proxy closed the connection mid-flight (`TypeError: terminated`, `other side closed`, …). Nothing has reached the user, so the request is simply re-issued.
157
+ - **The stream dies after output has started** — retrying would duplicate text, so the error is surfaced instead, with the real reason expanded out of the `error.cause` chain (`fetch failed` alone says nothing useful).
158
+
159
+ Any stream the consumer walks away from (pause, or a newer turn taking over) always has its body cancelled, so sockets are not leaked.
160
+
135
161
  ## License
136
162
 
137
163
  Apache-2.0. See [LICENSE](./LICENSE).
package/dist/app.js CHANGED
@@ -16,6 +16,10 @@ import { SettingsScreen } from "./screens/settings.js";
16
16
  import { StartScreen } from "./screens/start.js";
17
17
  const CURSOR_BLINK_MS = 530;
18
18
  const TOAST_MS = 1600;
19
+ // How often the whole frame is re-emitted as a safety net, so terminal-side
20
+ // drift (ConPTY reflow, stray scroll) can never leave any row stale for long.
21
+ // Re-emitting the cached frame is a no-op when nothing drifted.
22
+ const REPAINT_MS = 2000;
19
23
  export class App {
20
24
  state = defaultState();
21
25
  width;
@@ -28,6 +32,7 @@ export class App {
28
32
  blinkTimer = null;
29
33
  cursorTimer = null;
30
34
  toastTimer = null;
35
+ repaintTimer = null;
31
36
  toast = "";
32
37
  // Most recently rendered frame. Kept so a selection can be turned back into
33
38
  // text without having to re-run the screen's render.
@@ -81,6 +86,7 @@ export class App {
81
86
  if (this.syncSize())
82
87
  this.render();
83
88
  }, 200);
89
+ this.repaintTimer = setInterval(() => this.renderer.repaint(), REPAINT_MS);
84
90
  this.pushScreen(new StartScreen());
85
91
  }
86
92
  onResize() {
@@ -133,7 +139,7 @@ export class App {
133
139
  if (this.state.selection)
134
140
  applySelection(buf, this.state.selection);
135
141
  this.drawToast(buf);
136
- this.renderer.render(buf);
142
+ this.renderer.render(buf, this.top?.alwaysRepaintRows() ?? []);
137
143
  }
138
144
  drawToast(buf) {
139
145
  if (!this.toast)
@@ -240,6 +246,15 @@ export class App {
240
246
  return;
241
247
  }
242
248
  if (ev.type === "move") {
249
+ // A release can be lost (the pointer left the window, the terminal did
250
+ // not report button-up, …). Motion with no button held means the gesture
251
+ // is over, so drop any stale press/drag instead of letting a bare hover
252
+ // keep scrolling the log or grow a phantom selection.
253
+ if (!ev.drag) {
254
+ this.pressed = false;
255
+ this.pressStart = null;
256
+ this.barDrag = null;
257
+ }
243
258
  if (this.barDrag) {
244
259
  const bar = this.top?.scrollbar();
245
260
  if (bar) {
@@ -373,6 +388,10 @@ export class App {
373
388
  this.running = true;
374
389
  if (this.sizeTimer)
375
390
  clearInterval(this.sizeTimer);
391
+ if (this.repaintTimer) {
392
+ clearInterval(this.repaintTimer);
393
+ this.repaintTimer = null;
394
+ }
376
395
  if (this.blinkTimer) {
377
396
  clearInterval(this.blinkTimer);
378
397
  this.blinkTimer = null;
@@ -0,0 +1,68 @@
1
+ /**
2
+ * Helpers for compressing a long conversation so it fits back inside the
3
+ * model's context window.
4
+ *
5
+ * Everything here is pure: deciding *what* to summarise, and flattening that
6
+ * part into a plain transcript. The actual "ask the model to summarise" step
7
+ * lives in `LLMClient.summarize`, so this module stays trivially unit-testable.
8
+ */
9
+ /** How many trailing messages are always kept verbatim when compressing. */
10
+ export const COMPACT_KEEP = 4;
11
+ /** Per-message character budget fed to the summariser, to bound its own input. */
12
+ export const COMPACT_MAX_CHARS_PER_MESSAGE = 4000;
13
+ /**
14
+ * Split `messages` into an older prefix to summarise and a recent suffix to
15
+ * keep. The kept suffix never begins with a bare tool result — that would be an
16
+ * orphan without the assistant tool-call that produced it — so such messages
17
+ * are folded into the prefix instead.
18
+ *
19
+ * Returns null when there is nothing worth compressing (the conversation is
20
+ * already short enough).
21
+ */
22
+ export function planCompaction(messages, keep = COMPACT_KEEP) {
23
+ if (keep < 1)
24
+ keep = 1;
25
+ const head = [];
26
+ const tail = [];
27
+ for (let i = messages.length - 1; i >= 0; i--) {
28
+ if (tail.length < keep)
29
+ tail.unshift(messages[i]);
30
+ else
31
+ head.push(messages[i]);
32
+ }
33
+ head.reverse();
34
+ // Never open the kept suffix with an orphan tool result.
35
+ while (tail.length > 1 && tail[0].role === "tool") {
36
+ head.push(tail.shift());
37
+ }
38
+ if (head.length === 0 || tail.length === 0)
39
+ return null;
40
+ return { head, tail };
41
+ }
42
+ /**
43
+ * Flatten messages into a plain transcript for the summariser, clipping each
44
+ * message so a single huge tool result cannot dominate the summary request.
45
+ */
46
+ export function transcriptFor(messages, maxCharsPerMessage = COMPACT_MAX_CHARS_PER_MESSAGE) {
47
+ const clip = (s) => {
48
+ const t = s.replace(/\r\n?/g, "\n").trim();
49
+ return t.length > maxCharsPerMessage ? t.slice(0, maxCharsPerMessage) + "\n…[truncated]" : t;
50
+ };
51
+ const parts = [];
52
+ for (const m of messages) {
53
+ const body = clip(m.content ?? "");
54
+ if (m.role === "tool") {
55
+ parts.push(`TOOL ${m.name ?? ""} RESULT:\n${body}`);
56
+ }
57
+ else if (m.role === "assistant") {
58
+ parts.push(`ASSISTANT:\n${body}`);
59
+ }
60
+ else if (m.role === "user") {
61
+ parts.push(`USER:\n${body}`);
62
+ }
63
+ else {
64
+ parts.push(`${m.role.toUpperCase()}:\n${body}`);
65
+ }
66
+ }
67
+ return parts.join("\n\n");
68
+ }
package/dist/config.js CHANGED
@@ -36,6 +36,9 @@ export const DEFAULT_CONFIG = {
36
36
  language: "en",
37
37
  theme: "default",
38
38
  default_intensity: "High",
39
+ // Estimated prompt tokens above which the conversation is auto-compressed.
40
+ // 0 disables auto-compression.
41
+ context_compress_threshold: 32000,
39
42
  };
40
43
  export const INTENSITY_LEVELS = ["Low", "Medium", "High", "Max", "Ultra"];
41
44
  export const INTENSITY_PARAMS = {
@@ -45,6 +48,18 @@ export const INTENSITY_PARAMS = {
45
48
  Max: { temperature: 1.0, top_p: 1.0 },
46
49
  Ultra: { temperature: 1.5, top_p: 1.0 },
47
50
  };
51
+ /**
52
+ * Selectable auto-compression thresholds, in estimated prompt tokens.
53
+ * `0` turns the feature off. Rendered as-is in the settings screen.
54
+ */
55
+ export const COMPRESS_THRESHOLD_OPTIONS = [
56
+ { value: 0, label: "Off" },
57
+ { value: 8000, label: "8K tokens" },
58
+ { value: 16000, label: "16K tokens" },
59
+ { value: 32000, label: "32K tokens" },
60
+ { value: 64000, label: "64K tokens" },
61
+ { value: 128000, label: "128K tokens" },
62
+ ];
48
63
  export function ensureDirs() {
49
64
  fs.mkdirSync(NEUTRINO_DIR, { recursive: true });
50
65
  fs.mkdirSync(SESSION_DIR, { recursive: true });
@@ -1,4 +1,138 @@
1
1
  import { INTENSITY_PARAMS } from "../config.js";
2
+ /** How many extra times a request is retried when the connection drops. */
3
+ const NETWORK_RETRIES = 2;
4
+ /** Give up if the server has not started to respond within this window. */
5
+ const CONNECT_TIMEOUT_MS = 120_000;
6
+ /**
7
+ * System prompt used when compressing a long conversation. The instruction
8
+ * spells out exactly what must survive compression (goals, decisions, files,
9
+ * commands) so the summary stays useful for continuing the work.
10
+ */
11
+ const SUMMARY_SYSTEM_PROMPT = "You compress the conversation history of an AI coding assistant so it fits in a smaller context window. " +
12
+ "Write a dense, factual summary in the same language as the conversation. Preserve: the user's goals and " +
13
+ "constraints, decisions that were made, files created or modified (with their paths), commands that were run " +
14
+ "and their meaningful results, and any unresolved problems or next steps. Drop pleasantries, restated " +
15
+ "instructions and redundant tool output. Output only the summary, with no preface.";
16
+ /**
17
+ * Terse transport messages that all mean "the connection dropped mid-flight".
18
+ * The \b keeps "terminated" from matching unrelated text such as
19
+ * "Unterminated string constant".
20
+ */
21
+ const DROP_HINT_RE = /\bterminated\b|other side closed|socket hang up|premature close/i;
22
+ /**
23
+ * Flatten an error and its `cause` chain into one readable line.
24
+ *
25
+ * `fetch` reports almost every transport problem as a bare
26
+ * `TypeError: fetch failed`; the actual reason (`ECONNRESET`, `EMFILE`,
27
+ * `ENOTFOUND`, `UND_ERR_…`) is tucked away in `error.cause`. Without walking
28
+ * that chain a failure is impossible to diagnose, so this always reports it.
29
+ */
30
+ export function describeError(e) {
31
+ if (!(e instanceof Error))
32
+ return String(e);
33
+ const parts = [];
34
+ const push = (msg, code) => {
35
+ const text = code && !msg.includes(code) ? `${msg} (${code})` : msg;
36
+ if (text && !parts.includes(text))
37
+ parts.push(text);
38
+ };
39
+ let cur = e;
40
+ const seen = new Set();
41
+ while (cur != null && !seen.has(cur)) {
42
+ if (typeof cur === "string") {
43
+ // fetch/undici sometimes stash the real reason in a plain string
44
+ // (`cause: "other side closed"`), which an object-only walk would miss.
45
+ push(cur);
46
+ break;
47
+ }
48
+ if (typeof cur !== "object")
49
+ break;
50
+ seen.add(cur);
51
+ const err = cur;
52
+ const message = typeof err.message === "string" && err.message ? err.message : String(err.name ?? "Error");
53
+ const code = typeof err.code === "string" ? err.code : undefined;
54
+ push(message, code);
55
+ cur = err.cause;
56
+ }
57
+ const text = parts.join(": ") || e.message;
58
+ // A dropped mid-stream connection surfaces as a bare `TypeError: terminated`,
59
+ // which on its own tells the user nothing worth reading.
60
+ return DROP_HINT_RE.test(text) ? `${text} — the connection was closed before the response finished` : text;
61
+ }
62
+ /**
63
+ * Whether a failed request is worth retrying: a transient connection problem
64
+ * (reset / refused / DNS / timeout) rather than a real response from the API.
65
+ */
66
+ function isRetriableNetworkError(e) {
67
+ const transient = new Set([
68
+ "ECONNRESET",
69
+ "ECONNREFUSED",
70
+ "ECONNABORTED",
71
+ "EPIPE",
72
+ "ETIMEDOUT",
73
+ "ENETUNREACH",
74
+ "ENETDOWN",
75
+ "EAI_AGAIN",
76
+ "ERR_STREAM_PREMATURE_CLOSE",
77
+ "UND_ERR_SOCKET",
78
+ "UND_ERR_CONNECT_TIMEOUT",
79
+ ]);
80
+ let cur = e;
81
+ const seen = new Set();
82
+ while (cur != null && !seen.has(cur)) {
83
+ if (typeof cur === "string") {
84
+ if (DROP_HINT_RE.test(cur))
85
+ return true;
86
+ break;
87
+ }
88
+ if (typeof cur !== "object")
89
+ break;
90
+ seen.add(cur);
91
+ const err = cur;
92
+ if (typeof err.code === "string" && transient.has(err.code))
93
+ return true;
94
+ // Our own connect timeout aborts with a plain AbortError; undici wraps a
95
+ // dropped socket in a SocketError.
96
+ if (err.name === "AbortError" || err.name === "SocketError")
97
+ return true;
98
+ // A stream that dies mid-flight: "terminated", "socket hang up", …
99
+ if (typeof err.message === "string" && DROP_HINT_RE.test(err.message))
100
+ return true;
101
+ cur = err.cause;
102
+ }
103
+ // Undici's generic wrapper: assume the socket dropped and try again.
104
+ return e instanceof Error && /fetch failed/i.test(e.message);
105
+ }
106
+ function sleep(ms) {
107
+ return new Promise((resolve) => setTimeout(resolve, ms));
108
+ }
109
+ /**
110
+ * Issue a streaming request, retrying a few times on transient network errors
111
+ * and giving up with a clear error if the server never starts responding.
112
+ *
113
+ * Retries are safe here because they only ever happen *before* a response is
114
+ * returned; once bytes start flowing the caller is responsible for the stream.
115
+ */
116
+ async function requestStream(url, init) {
117
+ let lastErr;
118
+ for (let attempt = 0; attempt <= NETWORK_RETRIES; attempt++) {
119
+ const controller = new AbortController();
120
+ const timer = setTimeout(() => controller.abort(new Error(`request timed out after ${CONNECT_TIMEOUT_MS / 1000}s`)), CONNECT_TIMEOUT_MS);
121
+ try {
122
+ const resp = await fetch(url, { ...init, signal: controller.signal });
123
+ clearTimeout(timer);
124
+ return resp;
125
+ }
126
+ catch (e) {
127
+ clearTimeout(timer);
128
+ lastErr = e;
129
+ if (attempt === NETWORK_RETRIES || !isRetriableNetworkError(e))
130
+ throw e;
131
+ await sleep(300 * (attempt + 1));
132
+ }
133
+ }
134
+ throw lastErr;
135
+ }
2
136
  export class LLMClient {
3
137
  model;
4
138
  provider;
@@ -15,6 +149,25 @@ export class LLMClient {
15
149
  this.temperature = params.temperature;
16
150
  this.top_p = params.top_p;
17
151
  }
152
+ /**
153
+ * Ask the model to compress a transcript into a short summary. Returns an
154
+ * empty string on any failure so the caller can fall back to simply dropping
155
+ * the old messages. Uses a throw-away request (no tools) via the normal
156
+ * streaming path, so provider differences are handled for free.
157
+ */
158
+ async summarize(transcript) {
159
+ const messages = [
160
+ { role: "system", content: SUMMARY_SYSTEM_PROMPT },
161
+ { role: "user", content: transcript },
162
+ ];
163
+ let out = "";
164
+ for await (const chunk of this.streamTurn(messages))
165
+ out += chunk;
166
+ const text = out.trim();
167
+ if (!text || text.startsWith("[Error"))
168
+ return "";
169
+ return text;
170
+ }
18
171
  async *streamTurn(messages, tools) {
19
172
  this.toolCalls = [];
20
173
  this.usage = null;
@@ -26,10 +179,38 @@ export class LLMClient {
26
179
  yield* this.streamOpenAI(messages, tools);
27
180
  }
28
181
  catch (e) {
29
- yield `\n[Error: ${e instanceof Error ? e.message : String(e)}]`;
182
+ yield `\n[Error: ${describeError(e)}]`;
30
183
  }
31
184
  }
185
+ /**
186
+ * Drive one request, retrying if the stream dies before it produced anything.
187
+ *
188
+ * A response can be cut off on the very first read — "terminated" when a
189
+ * proxy or the server closes the connection mid-flight. Nothing has reached
190
+ * the user yet, so re-issuing the request is invisible (and cannot duplicate
191
+ * text). Once a token has been emitted a retry is unsafe, so the error is
192
+ * surfaced instead.
193
+ */
32
194
  async *streamOpenAI(messages, tools) {
195
+ for (let attempt = 0;; attempt++) {
196
+ this.toolCalls = [];
197
+ let produced = false;
198
+ try {
199
+ for await (const token of this.openAIAttempt(messages, tools)) {
200
+ produced = true;
201
+ yield token;
202
+ }
203
+ return;
204
+ }
205
+ catch (e) {
206
+ if (produced || attempt >= NETWORK_RETRIES || !isRetriableNetworkError(e))
207
+ throw e;
208
+ await sleep(400 * (attempt + 1));
209
+ }
210
+ }
211
+ }
212
+ /** A single streaming attempt; see `streamOpenAI` for the retry policy. */
213
+ async *openAIAttempt(messages, tools) {
33
214
  const url = this.model.url.replace(/\/+$/, "") + "/chat/completions";
34
215
  const headers = {
35
216
  "Content-Type": "application/json",
@@ -45,7 +226,7 @@ export class LLMClient {
45
226
  };
46
227
  if (tools && tools.length)
47
228
  payload.tools = tools;
48
- const resp = await fetch(url, {
229
+ const resp = await requestStream(url, {
49
230
  method: "POST",
50
231
  headers,
51
232
  body: JSON.stringify(payload),
@@ -64,66 +245,100 @@ export class LLMClient {
64
245
  const decoder = new TextDecoder();
65
246
  let buf = "";
66
247
  let stopped = false;
67
- while (!stopped) {
68
- const { done, value } = await reader.read();
69
- if (done)
70
- break;
71
- buf += decoder.decode(value, { stream: true });
72
- let idx;
73
- while ((idx = buf.indexOf("\n")) >= 0) {
74
- const raw = buf.slice(0, idx).replace(/\r$/, "");
75
- buf = buf.slice(idx + 1);
76
- const line = raw.trim();
77
- if (!line)
78
- continue;
79
- if (!line.startsWith("data:"))
80
- continue;
81
- const data = line.slice(5).trim();
82
- if (data === "[DONE]") {
83
- stopped = true;
248
+ try {
249
+ while (!stopped) {
250
+ const { done, value } = await reader.read();
251
+ if (done)
84
252
  break;
85
- }
86
- let chunk;
87
- try {
88
- chunk = JSON.parse(data);
89
- }
90
- catch {
91
- continue;
92
- }
93
- const usage = chunk.usage;
94
- if (usage) {
95
- this.usage = {
96
- prompt: usage.prompt_tokens ?? 0,
97
- completion: usage.completion_tokens ?? 0,
98
- total: usage.total_tokens ?? (usage.prompt_tokens ?? 0) + (usage.completion_tokens ?? 0),
99
- };
100
- }
101
- const choice = chunk.choices?.[0];
102
- const delta = choice?.delta ?? {};
103
- const content = delta.content;
104
- if (typeof content === "string" && content)
105
- yield content;
106
- const toolCalls = delta.tool_calls;
107
- if (toolCalls) {
108
- for (const tc of toolCalls) {
109
- const i = tc.index ?? 0;
110
- const entry = (toolCallsByIndex[i] ??= { id: "", name: "", arguments: "" });
111
- if (tc.id)
112
- entry.id = tc.id;
113
- if (tc.function?.name)
114
- entry.name = tc.function.name;
115
- if (tc.function?.arguments)
116
- entry.arguments += tc.function.arguments;
253
+ buf += decoder.decode(value, { stream: true });
254
+ let idx;
255
+ while ((idx = buf.indexOf("\n")) >= 0) {
256
+ const raw = buf.slice(0, idx).replace(/\r$/, "");
257
+ buf = buf.slice(idx + 1);
258
+ const line = raw.trim();
259
+ if (!line)
260
+ continue;
261
+ if (!line.startsWith("data:"))
262
+ continue;
263
+ const data = line.slice(5).trim();
264
+ if (data === "[DONE]") {
265
+ stopped = true;
266
+ break;
267
+ }
268
+ let chunk;
269
+ try {
270
+ chunk = JSON.parse(data);
271
+ }
272
+ catch {
273
+ continue;
274
+ }
275
+ const usage = chunk.usage;
276
+ if (usage) {
277
+ this.usage = {
278
+ prompt: usage.prompt_tokens ?? 0,
279
+ completion: usage.completion_tokens ?? 0,
280
+ total: usage.total_tokens ?? (usage.prompt_tokens ?? 0) + (usage.completion_tokens ?? 0),
281
+ };
282
+ }
283
+ const choice = chunk.choices?.[0];
284
+ const delta = choice?.delta ?? {};
285
+ const content = delta.content;
286
+ if (typeof content === "string" && content)
287
+ yield content;
288
+ const toolCalls = delta.tool_calls;
289
+ if (toolCalls) {
290
+ for (const tc of toolCalls) {
291
+ const i = tc.index ?? 0;
292
+ const entry = (toolCallsByIndex[i] ??= { id: "", name: "", arguments: "" });
293
+ if (tc.id)
294
+ entry.id = tc.id;
295
+ if (tc.function?.name)
296
+ entry.name = tc.function.name;
297
+ if (tc.function?.arguments)
298
+ entry.arguments += tc.function.arguments;
299
+ }
117
300
  }
118
301
  }
119
302
  }
120
303
  }
304
+ finally {
305
+ // The consumer may walk away mid-stream (pause, a newer turn taking over,
306
+ // or `[DONE]` arriving before the body ends). Always release the reader so
307
+ // the socket is torn down instead of lingering — leaked streaming bodies
308
+ // are what eventually make every request fail with "fetch failed".
309
+ try {
310
+ await reader.cancel();
311
+ }
312
+ catch {
313
+ // Already closed / errored; nothing left to release.
314
+ }
315
+ }
121
316
  this.toolCalls = Object.keys(toolCallsByIndex)
122
317
  .map((k) => Number(k))
123
318
  .sort((a, b) => a - b)
124
319
  .map((k) => toolCallsByIndex[k]);
125
320
  }
321
+ /** Anthropic counterpart of `streamOpenAI`, with the same retry policy. */
126
322
  async *streamAnthropic(messages) {
323
+ for (let attempt = 0;; attempt++) {
324
+ this.toolCalls = [];
325
+ let produced = false;
326
+ try {
327
+ for await (const token of this.anthropicAttempt(messages)) {
328
+ produced = true;
329
+ yield token;
330
+ }
331
+ return;
332
+ }
333
+ catch (e) {
334
+ if (produced || attempt >= NETWORK_RETRIES || !isRetriableNetworkError(e))
335
+ throw e;
336
+ await sleep(400 * (attempt + 1));
337
+ }
338
+ }
339
+ }
340
+ /** A single streaming attempt; see `streamAnthropic` for the retry policy. */
341
+ async *anthropicAttempt(messages) {
127
342
  const url = this.model.url.replace(/\/+$/, "") + "/messages";
128
343
  const headers = {
129
344
  "Content-Type": "application/json",
@@ -147,7 +362,7 @@ export class LLMClient {
147
362
  };
148
363
  if (systemMsg)
149
364
  payload.system = systemMsg;
150
- const resp = await fetch(url, {
365
+ const resp = await requestStream(url, {
151
366
  method: "POST",
152
367
  headers,
153
368
  body: JSON.stringify(payload),
@@ -165,41 +380,51 @@ export class LLMClient {
165
380
  const decoder = new TextDecoder();
166
381
  let buf = "";
167
382
  let stopped = false;
168
- while (!stopped) {
169
- const { done, value } = await reader.read();
170
- if (done)
171
- break;
172
- buf += decoder.decode(value, { stream: true });
173
- let idx;
174
- while ((idx = buf.indexOf("\n")) >= 0) {
175
- const raw = buf.slice(0, idx).replace(/\r$/, "");
176
- buf = buf.slice(idx + 1);
177
- const line = raw.trim();
178
- if (!line || !line.startsWith("data:"))
179
- continue;
180
- const data = line.slice(5).trim();
181
- if (data === "[DONE]") {
182
- stopped = true;
183
- break;
184
- }
185
- let chunk;
186
- try {
187
- chunk = JSON.parse(data);
188
- }
189
- catch {
190
- continue;
191
- }
192
- const type = chunk.type;
193
- if (type === "content_block_delta") {
194
- const text = chunk.delta?.text;
195
- if (text)
196
- yield text;
197
- }
198
- else if (type === "message_stop") {
199
- stopped = true;
383
+ try {
384
+ while (!stopped) {
385
+ const { done, value } = await reader.read();
386
+ if (done)
200
387
  break;
388
+ buf += decoder.decode(value, { stream: true });
389
+ let idx;
390
+ while ((idx = buf.indexOf("\n")) >= 0) {
391
+ const raw = buf.slice(0, idx).replace(/\r$/, "");
392
+ buf = buf.slice(idx + 1);
393
+ const line = raw.trim();
394
+ if (!line || !line.startsWith("data:"))
395
+ continue;
396
+ const data = line.slice(5).trim();
397
+ if (data === "[DONE]") {
398
+ stopped = true;
399
+ break;
400
+ }
401
+ let chunk;
402
+ try {
403
+ chunk = JSON.parse(data);
404
+ }
405
+ catch {
406
+ continue;
407
+ }
408
+ const type = chunk.type;
409
+ if (type === "content_block_delta") {
410
+ const text = chunk.delta?.text;
411
+ if (text)
412
+ yield text;
413
+ }
414
+ else if (type === "message_stop") {
415
+ stopped = true;
416
+ break;
417
+ }
201
418
  }
202
419
  }
203
420
  }
421
+ finally {
422
+ try {
423
+ await reader.cancel();
424
+ }
425
+ catch {
426
+ // Already closed / errored; nothing left to release.
427
+ }
428
+ }
204
429
  }
205
430
  }