jeo-code 0.8.9 → 0.8.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.ja.md +1 -1
- package/README.ko.md +1 -1
- package/README.md +1 -1
- package/README.zh.md +1 -1
- package/package.json +1 -1
- package/src/agent/engine.ts +18 -10
- package/src/ai/providers/errors.ts +27 -0
- package/src/ai/providers/openai-responses.ts +11 -6
- package/src/util/retry.ts +48 -0
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
6
6
|
|
|
7
7
|
The README mirrors the latest 5 entries — regenerate with `bun run changelog:sync`.
|
|
8
8
|
|
|
9
|
+
## [0.8.10] - 2026-07-08
|
|
10
|
+
_"지피티 모델 연결이 안되는데 원인파악해서 개선해" — OpenAI's Codex/Responses backend (used by every `gpt-5.5`/`gpt-5.4` OAuth call, and by API-key reasoning models) can emit an in-band `response.failed`/`error` SSE EVENT on an otherwise-200 stream (documented codes: `server_error`, `rate_limit_exceeded` — OpenAI's own guidance is "retry with exponential backoff"). This was thrown as an unclassified bare `Error`, which propagated straight out of the engine's model call with NO retry — a transient OpenAI backend hiccup killed the whole turn outright, surfacing as "GPT doesn't connect" during real interactive use even though `jeo doctor` and a fresh one-shot call both looked healthy (the failure only manifests mid-stream, after the connection has already succeeded)._
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
- **In-band SSE error now retries instead of hard-failing** (`src/ai/providers/errors.ts`, `src/ai/providers/openai-responses.ts`, `src/util/retry.ts`, `src/agent/engine.ts`) — new `ProviderStreamError` class carries a synthetic `.status` (429 for `rate_limit_exceeded`, 500 otherwise) so existing retry classifiers treat it like the equivalent HTTP failure; `codexResponsesCall`/`codexResponsesStream` now throw it instead of a bare `Error`; the engine's mid-stream transient-network retry ladder (bounded, capped exponential backoff) now also catches it, sharing ONE retry budget with the existing socket-drop recovery path rather than two independent budgets.
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
- **In-band SSE error retry regression tests** (`test/codex-responses.test.ts`, `test/transient-network-recovery.test.ts`) — cover error-code capture on both SSE error shapes, `ProviderStreamError` retry classification, engine-level recovery from a mid-stream fault, and the shared retry-budget invariant across BOTH failure classes crossing the total cap (not just staying under a per-class cap).
|
|
17
|
+
|
|
9
18
|
## [0.8.9] - 2026-07-08
|
|
10
19
|
_"프롬프트 라우팅 속도 문제가 있는거같은데 근본원인 알려줘" — prompt routing incurred significant latency due to a design oversight in LLM-based escalation: when heuristic confidence falls below threshold (standard conceptual questions with no code blocks/file paths conflict-trigger 0.35 confidence), it makes a blocking, synchronous LLM classifier call. If the user's global `thinkingLevel` is enabled (medium/high/xhigh), the cheap classifier model (Haiku 4.5 / GPT-4o-mini) also ran with reasoning enabled, wasting 500ms–1500ms on internal thoughts for a simple 1-word JSON response. The same reasoning-latency leak existed on all other background/internal LLM calls (compaction summarizer, goal verifier, memory distiller), blocking user turns on compaction boundaries._
|
|
11
20
|
|
package/README.ja.md
CHANGED
|
@@ -262,11 +262,11 @@ CI は `.github/workflows/npm-publish.yml` で公開します — GitHub リリ
|
|
|
262
262
|
## 変更履歴 (Changelog)
|
|
263
263
|
|
|
264
264
|
<!-- CHANGELOG:START (auto-generated from CHANGELOG.md — run `bun run changelog:sync`) -->
|
|
265
|
+
- **[0.8.10]** (2026-07-08) — "지피티 모델 연결이 안되는데 원인파악해서 개선해" — OpenAI's Codex/Responses backend (used by every `gpt-5.5`/`gpt-5.4` OAuth call, and by API-key reasoning models) can emit an in-band `response.failed`/`error` SSE EVENT on an otherwise-200 stream (documented codes: `server_error`, `rate_limit_exceeded` — OpenAI's own guidance is "retry with exponential backoff"). This was thrown as an unclassified bare `Error`, which propagated straight out of the engine's model call with NO retry — a transient OpenAI backend hiccup killed the whole turn outright, surfacing as "GPT doesn't connect" during real interactive use even though `jeo doctor` and a fresh one-shot call both looked healthy (the failure only manifests mid-stream, after the connection has already succeeded).
|
|
265
266
|
- **[0.8.9]** (2026-07-08) — "프롬프트 라우팅 속도 문제가 있는거같은데 근본원인 알려줘" — prompt routing incurred significant latency due to a design oversight in LLM-based escalation: when heuristic confidence falls below threshold (standard conceptual questions with no code blocks/file paths conflict-trigger 0.35 confidence), it makes a blocking, synchronous LLM classifier call. If the user's global `thinkingLevel` is enabled (medium/high/xhigh), the cheap classifier model (Haiku 4.5 / GPT-4o-mini) also ran with reasoning enabled, wasting 500ms–1500ms on internal thoughts for a simple 1-word JSON response. The same reasoning-latency leak existed on all other background/internal LLM calls (compaction summarizer, goal verifier, memory distiller), blocking user turns on compaction boundaries.
|
|
266
267
|
- **[0.8.8]** (2026-07-08) — "실동작검증을 통해서 유효성 평가하고 개선해줘" — live-verified the v0.8.5 fallback mechanism using active Claude Code OAuth credentials on this machine (confirmed 200 OK for both Fable 5 and Opus 4.8 via the proxy). Identified two critical improvement vectors during verification: (1) server-side fallback was previously scoped strictly to API-key credentials, leaving OAuth users (who also hit `reasoning_extraction` refusals) with today's reactive-only recovery; (2) the sequential `postAnthropic` retry ladder could throw immediately on a combination of different error types (e.g. temperature error first, then fallback error, then success).
|
|
267
268
|
- **[0.8.7]** (2026-07-08) — Fix-forward for a CI-caught release failure in v0.8.6 (this same session): `test/launch-prompt-routing.test.ts` shipped with a dropped closing brace (a git 3-way merge misaligned on a duplicate `});` line during this session's own working-tree isolation, then a subsequent `git stash push --keep-index` silently reset the working tree to the pre-fix staged content — the fix was applied once, verified once, then invisibly reverted before the commit that actually shipped). The v0.8.6 GitHub Actions release workflow correctly caught the syntax error at `bun test` and failed BEFORE the npm publish step ran; `npm view jeo-code version` confirmed 0.8.5 remained latest throughout — nothing broken ever reached the registry. Per this repo's established pattern for a same-day CI-caught regression (v0.8.0 → v0.8.1), shipping forward rather than retargeting the already-publicly-failed v0.8.6 tag.
|
|
268
269
|
- **[0.8.6]** (2026-07-08) — Explicit `/model` pin always winning over PromptRouter was correct but opaque: `/route status` showed "routing: on" even while a session pin blocked routing from ever evaluating a prompt, and there was no way back to routed mode short of restarting the session. Also closed a related escalation gap: `routing.enabled` with `roles.smol` unset made ambiguous-prompt LLM escalation silently skip every turn, even when a cheaper credentialed model existed to run the classifier.
|
|
269
|
-
- **[0.8.5]** (2026-07-08) — "딥리서치 다음과같이 페이블5 모델이 응답하게 되는경우를 피하기 위해" — a claude-fable-5 turn hit `Error: Refusal (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service.` jeo already had a full reactive recovery ladder (context reset → thinking-artifact strip → guidance strip → fail-fast on a category-shaped refusal), but every rung still resends to the SAME model — a deterministic classifier trip re-refuses identically no matter how much context gets stripped. Deep-researched Anthropic's current docs and found the actual fix: a documented, dedicated **server-side fallback** beta released alongside Fable 5 that retries a safety-classifier decline against a different model inside the SAME request, before the caller ever sees an error.
|
|
270
270
|
|
|
271
271
|
See [CHANGELOG.md](CHANGELOG.md) for the full history.
|
|
272
272
|
<!-- CHANGELOG:END -->
|
package/README.ko.md
CHANGED
|
@@ -263,11 +263,11 @@ CI는 `.github/workflows/npm-publish.yml`로 배포합니다 — GitHub 릴리
|
|
|
263
263
|
## 변경 이력 (Changelog)
|
|
264
264
|
|
|
265
265
|
<!-- CHANGELOG:START (auto-generated from CHANGELOG.md — run `bun run changelog:sync`) -->
|
|
266
|
+
- **[0.8.10]** (2026-07-08) — "지피티 모델 연결이 안되는데 원인파악해서 개선해" — OpenAI's Codex/Responses backend (used by every `gpt-5.5`/`gpt-5.4` OAuth call, and by API-key reasoning models) can emit an in-band `response.failed`/`error` SSE EVENT on an otherwise-200 stream (documented codes: `server_error`, `rate_limit_exceeded` — OpenAI's own guidance is "retry with exponential backoff"). This was thrown as an unclassified bare `Error`, which propagated straight out of the engine's model call with NO retry — a transient OpenAI backend hiccup killed the whole turn outright, surfacing as "GPT doesn't connect" during real interactive use even though `jeo doctor` and a fresh one-shot call both looked healthy (the failure only manifests mid-stream, after the connection has already succeeded).
|
|
266
267
|
- **[0.8.9]** (2026-07-08) — "프롬프트 라우팅 속도 문제가 있는거같은데 근본원인 알려줘" — prompt routing incurred significant latency due to a design oversight in LLM-based escalation: when heuristic confidence falls below threshold (standard conceptual questions with no code blocks/file paths conflict-trigger 0.35 confidence), it makes a blocking, synchronous LLM classifier call. If the user's global `thinkingLevel` is enabled (medium/high/xhigh), the cheap classifier model (Haiku 4.5 / GPT-4o-mini) also ran with reasoning enabled, wasting 500ms–1500ms on internal thoughts for a simple 1-word JSON response. The same reasoning-latency leak existed on all other background/internal LLM calls (compaction summarizer, goal verifier, memory distiller), blocking user turns on compaction boundaries.
|
|
267
268
|
- **[0.8.8]** (2026-07-08) — "실동작검증을 통해서 유효성 평가하고 개선해줘" — live-verified the v0.8.5 fallback mechanism using active Claude Code OAuth credentials on this machine (confirmed 200 OK for both Fable 5 and Opus 4.8 via the proxy). Identified two critical improvement vectors during verification: (1) server-side fallback was previously scoped strictly to API-key credentials, leaving OAuth users (who also hit `reasoning_extraction` refusals) with today's reactive-only recovery; (2) the sequential `postAnthropic` retry ladder could throw immediately on a combination of different error types (e.g. temperature error first, then fallback error, then success).
|
|
268
269
|
- **[0.8.7]** (2026-07-08) — Fix-forward for a CI-caught release failure in v0.8.6 (this same session): `test/launch-prompt-routing.test.ts` shipped with a dropped closing brace (a git 3-way merge misaligned on a duplicate `});` line during this session's own working-tree isolation, then a subsequent `git stash push --keep-index` silently reset the working tree to the pre-fix staged content — the fix was applied once, verified once, then invisibly reverted before the commit that actually shipped). The v0.8.6 GitHub Actions release workflow correctly caught the syntax error at `bun test` and failed BEFORE the npm publish step ran; `npm view jeo-code version` confirmed 0.8.5 remained latest throughout — nothing broken ever reached the registry. Per this repo's established pattern for a same-day CI-caught regression (v0.8.0 → v0.8.1), shipping forward rather than retargeting the already-publicly-failed v0.8.6 tag.
|
|
269
270
|
- **[0.8.6]** (2026-07-08) — Explicit `/model` pin always winning over PromptRouter was correct but opaque: `/route status` showed "routing: on" even while a session pin blocked routing from ever evaluating a prompt, and there was no way back to routed mode short of restarting the session. Also closed a related escalation gap: `routing.enabled` with `roles.smol` unset made ambiguous-prompt LLM escalation silently skip every turn, even when a cheaper credentialed model existed to run the classifier.
|
|
270
|
-
- **[0.8.5]** (2026-07-08) — "딥리서치 다음과같이 페이블5 모델이 응답하게 되는경우를 피하기 위해" — a claude-fable-5 turn hit `Error: Refusal (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service.` jeo already had a full reactive recovery ladder (context reset → thinking-artifact strip → guidance strip → fail-fast on a category-shaped refusal), but every rung still resends to the SAME model — a deterministic classifier trip re-refuses identically no matter how much context gets stripped. Deep-researched Anthropic's current docs and found the actual fix: a documented, dedicated **server-side fallback** beta released alongside Fable 5 that retries a safety-classifier decline against a different model inside the SAME request, before the caller ever sees an error.
|
|
271
271
|
|
|
272
272
|
See [CHANGELOG.md](CHANGELOG.md) for the full history.
|
|
273
273
|
<!-- CHANGELOG:END -->
|
package/README.md
CHANGED
|
@@ -265,11 +265,11 @@ Huge thanks to [gajae-code](https://github.com/Yeachan-Heo/gajae-code) for the i
|
|
|
265
265
|
## Changelog
|
|
266
266
|
|
|
267
267
|
<!-- CHANGELOG:START (auto-generated from CHANGELOG.md — run `bun run changelog:sync`) -->
|
|
268
|
+
- **[0.8.10]** (2026-07-08) — "지피티 모델 연결이 안되는데 원인파악해서 개선해" — OpenAI's Codex/Responses backend (used by every `gpt-5.5`/`gpt-5.4` OAuth call, and by API-key reasoning models) can emit an in-band `response.failed`/`error` SSE EVENT on an otherwise-200 stream (documented codes: `server_error`, `rate_limit_exceeded` — OpenAI's own guidance is "retry with exponential backoff"). This was thrown as an unclassified bare `Error`, which propagated straight out of the engine's model call with NO retry — a transient OpenAI backend hiccup killed the whole turn outright, surfacing as "GPT doesn't connect" during real interactive use even though `jeo doctor` and a fresh one-shot call both looked healthy (the failure only manifests mid-stream, after the connection has already succeeded).
|
|
268
269
|
- **[0.8.9]** (2026-07-08) — "프롬프트 라우팅 속도 문제가 있는거같은데 근본원인 알려줘" — prompt routing incurred significant latency due to a design oversight in LLM-based escalation: when heuristic confidence falls below threshold (standard conceptual questions with no code blocks/file paths conflict-trigger 0.35 confidence), it makes a blocking, synchronous LLM classifier call. If the user's global `thinkingLevel` is enabled (medium/high/xhigh), the cheap classifier model (Haiku 4.5 / GPT-4o-mini) also ran with reasoning enabled, wasting 500ms–1500ms on internal thoughts for a simple 1-word JSON response. The same reasoning-latency leak existed on all other background/internal LLM calls (compaction summarizer, goal verifier, memory distiller), blocking user turns on compaction boundaries.
|
|
269
270
|
- **[0.8.8]** (2026-07-08) — "실동작검증을 통해서 유효성 평가하고 개선해줘" — live-verified the v0.8.5 fallback mechanism using active Claude Code OAuth credentials on this machine (confirmed 200 OK for both Fable 5 and Opus 4.8 via the proxy). Identified two critical improvement vectors during verification: (1) server-side fallback was previously scoped strictly to API-key credentials, leaving OAuth users (who also hit `reasoning_extraction` refusals) with today's reactive-only recovery; (2) the sequential `postAnthropic` retry ladder could throw immediately on a combination of different error types (e.g. temperature error first, then fallback error, then success).
|
|
270
271
|
- **[0.8.7]** (2026-07-08) — Fix-forward for a CI-caught release failure in v0.8.6 (this same session): `test/launch-prompt-routing.test.ts` shipped with a dropped closing brace (a git 3-way merge misaligned on a duplicate `});` line during this session's own working-tree isolation, then a subsequent `git stash push --keep-index` silently reset the working tree to the pre-fix staged content — the fix was applied once, verified once, then invisibly reverted before the commit that actually shipped). The v0.8.6 GitHub Actions release workflow correctly caught the syntax error at `bun test` and failed BEFORE the npm publish step ran; `npm view jeo-code version` confirmed 0.8.5 remained latest throughout — nothing broken ever reached the registry. Per this repo's established pattern for a same-day CI-caught regression (v0.8.0 → v0.8.1), shipping forward rather than retargeting the already-publicly-failed v0.8.6 tag.
|
|
271
272
|
- **[0.8.6]** (2026-07-08) — Explicit `/model` pin always winning over PromptRouter was correct but opaque: `/route status` showed "routing: on" even while a session pin blocked routing from ever evaluating a prompt, and there was no way back to routed mode short of restarting the session. Also closed a related escalation gap: `routing.enabled` with `roles.smol` unset made ambiguous-prompt LLM escalation silently skip every turn, even when a cheaper credentialed model existed to run the classifier.
|
|
272
|
-
- **[0.8.5]** (2026-07-08) — "딥리서치 다음과같이 페이블5 모델이 응답하게 되는경우를 피하기 위해" — a claude-fable-5 turn hit `Error: Refusal (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service.` jeo already had a full reactive recovery ladder (context reset → thinking-artifact strip → guidance strip → fail-fast on a category-shaped refusal), but every rung still resends to the SAME model — a deterministic classifier trip re-refuses identically no matter how much context gets stripped. Deep-researched Anthropic's current docs and found the actual fix: a documented, dedicated **server-side fallback** beta released alongside Fable 5 that retries a safety-classifier decline against a different model inside the SAME request, before the caller ever sees an error.
|
|
273
273
|
|
|
274
274
|
See [CHANGELOG.md](CHANGELOG.md) for the full history.
|
|
275
275
|
<!-- CHANGELOG:END -->
|
package/README.zh.md
CHANGED
|
@@ -262,11 +262,11 @@ CI 通过 `.github/workflows/npm-publish.yml` 发布 — GitHub 发布 release
|
|
|
262
262
|
## 更新日志 (Changelog)
|
|
263
263
|
|
|
264
264
|
<!-- CHANGELOG:START (auto-generated from CHANGELOG.md — run `bun run changelog:sync`) -->
|
|
265
|
+
- **[0.8.10]** (2026-07-08) — "지피티 모델 연결이 안되는데 원인파악해서 개선해" — OpenAI's Codex/Responses backend (used by every `gpt-5.5`/`gpt-5.4` OAuth call, and by API-key reasoning models) can emit an in-band `response.failed`/`error` SSE EVENT on an otherwise-200 stream (documented codes: `server_error`, `rate_limit_exceeded` — OpenAI's own guidance is "retry with exponential backoff"). This was thrown as an unclassified bare `Error`, which propagated straight out of the engine's model call with NO retry — a transient OpenAI backend hiccup killed the whole turn outright, surfacing as "GPT doesn't connect" during real interactive use even though `jeo doctor` and a fresh one-shot call both looked healthy (the failure only manifests mid-stream, after the connection has already succeeded).
|
|
265
266
|
- **[0.8.9]** (2026-07-08) — "프롬프트 라우팅 속도 문제가 있는거같은데 근본원인 알려줘" — prompt routing incurred significant latency due to a design oversight in LLM-based escalation: when heuristic confidence falls below threshold (standard conceptual questions with no code blocks/file paths conflict-trigger 0.35 confidence), it makes a blocking, synchronous LLM classifier call. If the user's global `thinkingLevel` is enabled (medium/high/xhigh), the cheap classifier model (Haiku 4.5 / GPT-4o-mini) also ran with reasoning enabled, wasting 500ms–1500ms on internal thoughts for a simple 1-word JSON response. The same reasoning-latency leak existed on all other background/internal LLM calls (compaction summarizer, goal verifier, memory distiller), blocking user turns on compaction boundaries.
|
|
266
267
|
- **[0.8.8]** (2026-07-08) — "실동작검증을 통해서 유효성 평가하고 개선해줘" — live-verified the v0.8.5 fallback mechanism using active Claude Code OAuth credentials on this machine (confirmed 200 OK for both Fable 5 and Opus 4.8 via the proxy). Identified two critical improvement vectors during verification: (1) server-side fallback was previously scoped strictly to API-key credentials, leaving OAuth users (who also hit `reasoning_extraction` refusals) with today's reactive-only recovery; (2) the sequential `postAnthropic` retry ladder could throw immediately on a combination of different error types (e.g. temperature error first, then fallback error, then success).
|
|
267
268
|
- **[0.8.7]** (2026-07-08) — Fix-forward for a CI-caught release failure in v0.8.6 (this same session): `test/launch-prompt-routing.test.ts` shipped with a dropped closing brace (a git 3-way merge misaligned on a duplicate `});` line during this session's own working-tree isolation, then a subsequent `git stash push --keep-index` silently reset the working tree to the pre-fix staged content — the fix was applied once, verified once, then invisibly reverted before the commit that actually shipped). The v0.8.6 GitHub Actions release workflow correctly caught the syntax error at `bun test` and failed BEFORE the npm publish step ran; `npm view jeo-code version` confirmed 0.8.5 remained latest throughout — nothing broken ever reached the registry. Per this repo's established pattern for a same-day CI-caught regression (v0.8.0 → v0.8.1), shipping forward rather than retargeting the already-publicly-failed v0.8.6 tag.
|
|
268
269
|
- **[0.8.6]** (2026-07-08) — Explicit `/model` pin always winning over PromptRouter was correct but opaque: `/route status` showed "routing: on" even while a session pin blocked routing from ever evaluating a prompt, and there was no way back to routed mode short of restarting the session. Also closed a related escalation gap: `routing.enabled` with `roles.smol` unset made ambiguous-prompt LLM escalation silently skip every turn, even when a cheaper credentialed model existed to run the classifier.
|
|
269
|
-
- **[0.8.5]** (2026-07-08) — "딥리서치 다음과같이 페이블5 모델이 응답하게 되는경우를 피하기 위해" — a claude-fable-5 turn hit `Error: Refusal (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service.` jeo already had a full reactive recovery ladder (context reset → thinking-artifact strip → guidance strip → fail-fast on a category-shaped refusal), but every rung still resends to the SAME model — a deterministic classifier trip re-refuses identically no matter how much context gets stripped. Deep-researched Anthropic's current docs and found the actual fix: a documented, dedicated **server-side fallback** beta released alongside Fable 5 that retries a safety-classifier decline against a different model inside the SAME request, before the caller ever sees an error.
|
|
270
270
|
|
|
271
271
|
See [CHANGELOG.md](CHANGELOG.md) for the full history.
|
|
272
272
|
<!-- CHANGELOG:END -->
|
package/package.json
CHANGED
package/src/agent/engine.ts
CHANGED
|
@@ -22,7 +22,7 @@ import { createLspTool } from "./lsp-tool";
|
|
|
22
22
|
import { createLspRenameTool } from "./lsp-rename-tool";
|
|
23
23
|
import { createDebugTool } from "./debug-tool";
|
|
24
24
|
import { createBrowserTool } from "./browser-tool";
|
|
25
|
-
import { isRateLimitError, isTransientStreamDropError, waitAbortable } from "../util/retry";
|
|
25
|
+
import { isRateLimitError, isTransientStreamDropError, isProviderStreamError, waitAbortable } from "../util/retry";
|
|
26
26
|
import { isContextOverflowError, isRefusalError, friendlyProviderError } from "../util/provider-error";
|
|
27
27
|
import { runPreToolHooks, runPostTurnHooksForBatch } from "./hooks";
|
|
28
28
|
import { truncateToolOutput, formatToolResultBody } from "./tool-output";
|
|
@@ -818,14 +818,22 @@ export async function runAgentLoop(history: Message[], opts: AgentLoopOptions):
|
|
|
818
818
|
await waitAbortable(delayMs, opts.signal);
|
|
819
819
|
continue; // loop top surfaces "Cancelled." when the wait was aborted
|
|
820
820
|
}
|
|
821
|
-
// Mid-stream transient
|
|
822
|
-
//
|
|
823
|
-
//
|
|
824
|
-
//
|
|
825
|
-
//
|
|
826
|
-
//
|
|
827
|
-
//
|
|
828
|
-
|
|
821
|
+
// Mid-stream transient fault (see `transientNetworkRetries` above): bounded resend
|
|
822
|
+
// with capped exponential backoff, covering TWO distinct failure classes that both
|
|
823
|
+
// land here unretried:
|
|
824
|
+
// - `isTransientStreamDropError` — a live TCP/TLS socket dying mid-stream.
|
|
825
|
+
// - `isProviderStreamError` — an in-band SSE error EVENT on an otherwise-200
|
|
826
|
+
// connection (OpenAI Responses `response.failed`/`error`, e.g. `server_error`).
|
|
827
|
+
// Deliberately narrower than a general `defaultRetryable` check — a rate-limit/5xx/
|
|
828
|
+
// timeout error already ran the FULL model-manager `withRetry` budget (visible as
|
|
829
|
+
// its own "auto-retry #N" notices) before reaching here, so re-retrying it in this
|
|
830
|
+
// ladder too would silently double that budget. Both classes above are structurally
|
|
831
|
+
// exempt from that budget: `retryableStream` (model-manager.ts) only auto-retries
|
|
832
|
+
// losing the FIRST chunk — once any chunk has streamed it deliberately stops (a full
|
|
833
|
+
// re-call would replay already-emitted content) — so a fault arriving AFTER the
|
|
834
|
+
// first chunk never got a model-manager retry at all and needs this engine-level
|
|
835
|
+
// recovery instead.
|
|
836
|
+
if (isTransientStreamDropError(err) || isProviderStreamError(err)) {
|
|
829
837
|
transientNetworkRetries++;
|
|
830
838
|
if (transientNetworkRetries <= GUARD_LIMITS.MAX_TRANSIENT_NETWORK_RETRIES) {
|
|
831
839
|
const baseRaw = Number(jeoEnv("TRANSIENT_NETWORK_BACKOFF_BASE_MS") ?? NaN);
|
|
@@ -833,7 +841,7 @@ export async function runAgentLoop(history: Message[], opts: AgentLoopOptions):
|
|
|
833
841
|
const delayMs = Math.min(baseMs * 2 ** (transientNetworkRetries - 1), GUARD_LIMITS.TRANSIENT_NETWORK_BACKOFF_MAX_MS);
|
|
834
842
|
const escHint = ev.onModelStream ? " (Esc to cancel)" : "";
|
|
835
843
|
ev.onNotice?.(
|
|
836
|
-
`
|
|
844
|
+
`mid-response provider fault (${friendlyProviderError(err)}) — auto-retry #${transientNetworkRetries} in ${Math.max(1, Math.round(delayMs / 1000))}s${escHint}`,
|
|
837
845
|
);
|
|
838
846
|
await waitAbortable(delayMs, opts.signal);
|
|
839
847
|
continue; // loop top surfaces "Cancelled." when the wait was aborted
|
|
@@ -21,6 +21,33 @@ export class ProviderHttpError extends Error {
|
|
|
21
21
|
}
|
|
22
22
|
}
|
|
23
23
|
|
|
24
|
+
/**
|
|
25
|
+
* In-band stream failure: the HTTP response itself was 200 (a live SSE
|
|
26
|
+
* connection), but the provider emitted a terminal error EVENT instead of
|
|
27
|
+
* completing normally (OpenAI Responses `response.failed`/`error`: `{"error":
|
|
28
|
+
* {"code":"server_error","message":"…"}}`). No `Response` object survives to
|
|
29
|
+
* build a `ProviderHttpError` from, so this carries a SYNTHETIC `.status` —
|
|
30
|
+
* 429 for a rate-limit code, 500 otherwise — purely so the existing
|
|
31
|
+
* status-based retry checks (`defaultRetryable`, `isRateLimitError`) treat it
|
|
32
|
+
* exactly like the equivalent HTTP failure instead of falling through as an
|
|
33
|
+
* unclassified bare Error (OpenAI's own guidance for `server_error`/
|
|
34
|
+
* `rate_limit_exceeded` is "retry with exponential backoff" — every other
|
|
35
|
+
* code is unenumerated/rare enough that defaulting to retryable is safer than
|
|
36
|
+
* hard-failing a whole turn on a transient backend hiccup).
|
|
37
|
+
*/
|
|
38
|
+
export class ProviderStreamError extends Error {
|
|
39
|
+
readonly status: number;
|
|
40
|
+
readonly provider: string;
|
|
41
|
+
readonly code?: string;
|
|
42
|
+
constructor(provider: string, message: string, code?: string) {
|
|
43
|
+
super(`${provider} stream failed${code ? ` (${code})` : ""}: ${message}`);
|
|
44
|
+
this.name = "ProviderStreamError";
|
|
45
|
+
this.status = code === "rate_limit_exceeded" ? 429 : 500;
|
|
46
|
+
this.provider = provider;
|
|
47
|
+
this.code = code;
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
|
|
24
51
|
/**
|
|
25
52
|
* Parse a `Retry-After` header into ms. Supports the delta-seconds form
|
|
26
53
|
* (`"5"`) and the HTTP-date form (`"Wed, 21 Oct 2025 07:28:00 GMT"`).
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
import type { Credential } from "../../auth";
|
|
14
14
|
import type { CallOptions, Message } from "../types";
|
|
15
15
|
import { readSse } from "../sse";
|
|
16
|
-
import { providerHttpError, fetchWithArtifactFailSafe } from "./errors";
|
|
16
|
+
import { providerHttpError, fetchWithArtifactFailSafe, ProviderStreamError } from "./errors";
|
|
17
17
|
import { serializeAccumulatedToolCalls } from "../../agent/tool-schemas";
|
|
18
18
|
import { sanitizeJsonStrings } from "../../util/sanitize-json";
|
|
19
19
|
import os from "node:os";
|
|
@@ -181,6 +181,10 @@ export interface ResponsesEvent {
|
|
|
181
181
|
reasoningDelta?: string;
|
|
182
182
|
usage?: { inputTokens?: number; outputTokens?: number };
|
|
183
183
|
error?: string;
|
|
184
|
+
/** OpenAI's structured error `code` (e.g. `server_error`, `rate_limit_exceeded`) —
|
|
185
|
+
* carried separately from the human-readable message so callers can classify
|
|
186
|
+
* retryability without parsing prose (round: in-band SSE error retry fix). */
|
|
187
|
+
errorCode?: string;
|
|
184
188
|
/** `response.incomplete` cause (e.g. max_output_tokens) — surfaced when the
|
|
185
189
|
* whole response produced no text (round-5 #1). */
|
|
186
190
|
incompleteReason?: string;
|
|
@@ -201,10 +205,10 @@ export function parseResponsesEvent(data: string): ResponsesEvent {
|
|
|
201
205
|
output_index?: number;
|
|
202
206
|
response?: {
|
|
203
207
|
usage?: { input_tokens?: number; output_tokens?: number };
|
|
204
|
-
error?: { message?: string };
|
|
208
|
+
error?: { message?: string; code?: string };
|
|
205
209
|
incomplete_details?: { reason?: string };
|
|
206
210
|
};
|
|
207
|
-
error?: { message?: string };
|
|
211
|
+
error?: { message?: string; code?: string };
|
|
208
212
|
};
|
|
209
213
|
try {
|
|
210
214
|
o = JSON.parse(data);
|
|
@@ -237,7 +241,8 @@ export function parseResponsesEvent(data: string): ResponsesEvent {
|
|
|
237
241
|
};
|
|
238
242
|
}
|
|
239
243
|
if (o.type === "response.failed" || o.type === "error") {
|
|
240
|
-
|
|
244
|
+
const e = o.response?.error ?? o.error;
|
|
245
|
+
return { error: e?.message ?? "Codex response failed", errorCode: e?.code };
|
|
241
246
|
}
|
|
242
247
|
return {};
|
|
243
248
|
}
|
|
@@ -294,7 +299,7 @@ export async function codexResponsesCall(messages: Message[], options: CallOptio
|
|
|
294
299
|
accumulateResponsesToolCall(toolAcc, ev);
|
|
295
300
|
if (ev.usage) options.onUsage?.(ev.usage);
|
|
296
301
|
if (ev.incompleteReason) incompleteReason = ev.incompleteReason;
|
|
297
|
-
if (ev.error) throw new
|
|
302
|
+
if (ev.error) throw new ProviderStreamError("OpenAI Codex", ev.error, ev.errorCode);
|
|
298
303
|
}
|
|
299
304
|
// Prefer a native tool call (re-serialized to canonical JSON) over any stray text.
|
|
300
305
|
const envelope = serializeAccumulatedToolCalls(toolAcc);
|
|
@@ -326,7 +331,7 @@ export async function* codexResponsesStream(
|
|
|
326
331
|
accumulateResponsesToolCall(toolAcc, ev);
|
|
327
332
|
if (ev.usage) options.onUsage?.(ev.usage);
|
|
328
333
|
if (ev.incompleteReason) incompleteReason = ev.incompleteReason;
|
|
329
|
-
if (ev.error) throw new
|
|
334
|
+
if (ev.error) throw new ProviderStreamError("OpenAI Codex", ev.error, ev.errorCode);
|
|
330
335
|
}
|
|
331
336
|
// Native tool calls have no output_text deltas — yield the re-serialized envelope once.
|
|
332
337
|
const envelope = serializeAccumulatedToolCalls(toolAcc);
|
package/src/util/retry.ts
CHANGED
|
@@ -56,6 +56,14 @@ export function defaultRetryable(err: unknown): boolean {
|
|
|
56
56
|
if (isRefusalError(err)) {
|
|
57
57
|
return false;
|
|
58
58
|
}
|
|
59
|
+
// A pre-response connection failure (refused / unresolved host) — retryable: a
|
|
60
|
+
// local provider that is mid-restart, or a transient DNS blip, can clear within
|
|
61
|
+
// the backoff budget. A provider that is simply never running exhausts the
|
|
62
|
+
// budget and surfaces friendlyProviderError's actionable message instead of a
|
|
63
|
+
// bare "Unable to connect" (see isConnectionError's docs for the full rationale).
|
|
64
|
+
if (isConnectionError(err)) {
|
|
65
|
+
return true;
|
|
66
|
+
}
|
|
59
67
|
|
|
60
68
|
let message = "";
|
|
61
69
|
if (err instanceof Error) {
|
|
@@ -312,3 +320,43 @@ export function isRefusalError(err: unknown): boolean {
|
|
|
312
320
|
export function isTransientStreamDropError(err: unknown): boolean {
|
|
313
321
|
return /socket connection was closed|socket hang up|other side closed|und_err_socket/i.test(errorMessageOf(err));
|
|
314
322
|
}
|
|
323
|
+
|
|
324
|
+
/** True for a pre-response TCP-level connection failure — the request never reached
|
|
325
|
+
* a server at all (refused, DNS unresolved, host unreachable), as opposed to a
|
|
326
|
+
* server responding with an error status or a mid-stream drop (see
|
|
327
|
+
* `isTransientStreamDropError` for the latter). Bun's fetch/undici throws a bare
|
|
328
|
+
* `Error("Unable to connect. Is the computer able to access the url?")` with
|
|
329
|
+
* `.code === "ConnectionRefused"` for BOTH a refused connection and an unresolvable
|
|
330
|
+
* host — no HTTP status is ever attached, so `defaultRetryable`'s status-based
|
|
331
|
+
* checks never catch it. Node's equivalent carries a `.code` of ECONNREFUSED /
|
|
332
|
+
* ENOTFOUND / EHOSTUNREACH / EAI_AGAIN. Most commonly hit when a local provider
|
|
333
|
+
* (ollama/lmstudio) isn't running, or a custom base URL is misconfigured — see
|
|
334
|
+
* `friendlyProviderError`, which turns this into an actionable message naming the
|
|
335
|
+
* provider and base URL (attached by model-manager's `resolveCall` callers). */
|
|
336
|
+
export function isConnectionError(err: unknown): boolean {
|
|
337
|
+
let code: string | undefined;
|
|
338
|
+
if (err && typeof err === "object" && "code" in err) code = String(err.code);
|
|
339
|
+
if (code && ["ConnectionRefused", "ECONNREFUSED", "ENOTFOUND", "EHOSTUNREACH", "EAI_AGAIN"].includes(code)) return true;
|
|
340
|
+
return /unable to connect\. is the computer able to access the url\?/i.test(errorMessageOf(err));
|
|
341
|
+
}
|
|
342
|
+
|
|
343
|
+
/** True for an in-band provider stream failure: the HTTP response itself was 200 (a
|
|
344
|
+
* live SSE connection), but the provider emitted a terminal error EVENT instead of
|
|
345
|
+
* completing (OpenAI Responses `response.failed`/`error`:
|
|
346
|
+
* `{"error":{"code":"server_error",...}}`) — see `ProviderStreamError` in
|
|
347
|
+
* `ai/providers/errors.ts`. Narrowed via `instanceof Error` + `.name` (not an import
|
|
348
|
+
* of the concrete class) to keep this generic util free of a provider-layer
|
|
349
|
+
* dependency, mirroring the existing `.name === "TimeoutError"` check in
|
|
350
|
+
* `provider-error.ts`. Every instance is treated as retryable here: OpenAI's own
|
|
351
|
+
* guidance for the two documented codes (`server_error`, `rate_limit_exceeded`) is
|
|
352
|
+
* "retry with exponential backoff", and any other/unenumerated code is rare enough
|
|
353
|
+
* that defaulting to retryable beats hard-failing a whole turn on one backend
|
|
354
|
+
* hiccup. Note `ProviderStreamError` ALSO carries a numeric `.status` (429/500), so
|
|
355
|
+
* `defaultRetryable`'s existing status check already retries it in the
|
|
356
|
+
* pre-first-chunk/non-streaming path; this predicate exists for the SEPARATE
|
|
357
|
+
* engine-level mid-stream ladder (see `transientNetworkRetries` in engine.ts),
|
|
358
|
+
* which — like `isTransientStreamDropError` — deliberately does not call
|
|
359
|
+
* `defaultRetryable` there to avoid double-spending that budget. */
|
|
360
|
+
export function isProviderStreamError(err: unknown): boolean {
|
|
361
|
+
return err instanceof Error && err.name === "ProviderStreamError";
|
|
362
|
+
}
|