pi-openai-toolkit 0.21.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -0
- package/README.zh.md +8 -0
- package/docs/configuration.md +6 -2
- package/docs/internals.md +15 -0
- package/package.json +5 -1
- package/src/codex-fast/extension.ts +119 -0
- package/src/codex-fast/payload.ts +36 -0
- package/src/codex-fast/state.ts +25 -0
- package/src/responses-websocket/transport.ts +136 -65
package/README.md
CHANGED
|
@@ -60,6 +60,14 @@ Change `mode` in the example above to `remote-windows`. Eligible sessions expose
|
|
|
60
60
|
|
|
61
61
|
Remote windows use alpha endpoints that may change upstream. When switching models, the remote window remains persisted and reusable; see the [window lifecycle](docs/internals.md#rollover-lifecycle).
|
|
62
62
|
|
|
63
|
+
## Codex Fast
|
|
64
|
+
|
|
65
|
+
Fast is OFF by default. In the terminal, run `/fast` to turn it ON, then run `/fast` again to turn it OFF. The status bar shows `Fast ON`, `Fast OFF`, or `Fast N/A` for an ineligible route. No Fast configuration is needed.
|
|
66
|
+
|
|
67
|
+
Supports native Codex and recognizable GPT/Codex models on compatible Responses gateways. The selection is saved only in the current session: reload/resume and `new_context` retain it; new or forked sessions start OFF. ON requests Fast, while OFF requests Standard, overriding an existing service-tier preference on eligible conversation requests without changing reasoning effort or configuration files.
|
|
68
|
+
|
|
69
|
+
Fast may increase usage/cost. The indicator shows the requested mode, not confirmed backend acceleration; a gateway may reject, ignore, or downgrade it. Opaque gateway model aliases need an existing Codex gateway identity. See the [Fast contract](docs/internals.md#codex-fast-session-control) for scope and lifecycle details.
|
|
70
|
+
|
|
63
71
|
## Search, images, and transport
|
|
64
72
|
|
|
65
73
|
### Web search
|
package/README.zh.md
CHANGED
|
@@ -60,6 +60,14 @@ Toolkit 只读取一个全局配置文件,即使扩展安装在项目内也一
|
|
|
60
60
|
|
|
61
61
|
远程窗口依赖 alpha 接口,上游可能随时调整。切换模型时远程窗口状态仍会持久化并可复用,详见[窗口生命周期](docs/internals.md#rollover-lifecycle)。
|
|
62
62
|
|
|
63
|
+
## Codex Fast
|
|
64
|
+
|
|
65
|
+
Fast 默认关闭。在终端输入 `/fast` 开启,再输入一次 `/fast` 关闭。状态栏显示 `Fast ON`、`Fast OFF`,不适用的路由显示 `Fast N/A`。无需添加 Fast 配置。
|
|
66
|
+
|
|
67
|
+
支持原生 Codex,以及兼容 Responses 网关中可识别的 GPT/Codex 模型。开关只保存在当前会话:reload、续聊和 `new_context` 保留状态,新建或 fork 的会话默认关闭。开启时请求 Fast,关闭时请求标准档,会覆盖符合条件的对话请求中原有的服务档位,但不改变 reasoning effort 或配置文件。
|
|
68
|
+
|
|
69
|
+
Fast 可能增加额度消耗或费用。状态栏表示请求模式,不代表后端已确认加速;网关可能拒绝、忽略或降级请求。无法识别的网关模型别名需要已有的 Codex 网关路由标识。适用范围和生命周期见 [Fast 规则](docs/internals.md#codex-fast-session-control)。
|
|
70
|
+
|
|
63
71
|
## 搜索、生图与传输
|
|
64
72
|
|
|
65
73
|
### 联网搜索
|
package/docs/configuration.md
CHANGED
|
@@ -93,19 +93,23 @@ This selects an existing Toolkit protocol profile, independently of context mode
|
|
|
93
93
|
|
|
94
94
|
The Responses WebSocket extension is disabled by default. Enable it globally with `defaults.responsesWebSocket.enabled: true`; it is intentionally not a model-scoped override. Restart or reload Pi after changing this setting. An invalid or legacy configuration keeps the feature disabled.
|
|
95
95
|
|
|
96
|
-
The extension is an API-adapter override, not a provider registration.
|
|
96
|
+
The extension is an API-adapter override, not a provider registration. For requests dispatched through this API adapter, it applies to GPT model IDs whose Pi API is `openai-responses`, regardless of provider name. Providers that bypass the compat API adapter keep their own transport behavior. The active model's `baseUrl`, `apiKey`, configured headers, session id, and payload remain the source of truth.
|
|
97
97
|
|
|
98
98
|
The extension honors Pi's global `transport` setting:
|
|
99
99
|
|
|
100
100
|
| Pi `transport` | Behavior |
|
|
101
101
|
| --- | --- |
|
|
102
|
-
| `auto` | Try `wss://.../responses?model=...` first;
|
|
102
|
+
| `auto` | Try `wss://.../responses?model=...` first; allow one original HTTP/SSE fallback per fetch call for a recoverable WS failure before the first valid event is exposed. With a stable session id, reuse the WebSocket and send a continuation delta when possible. |
|
|
103
103
|
| `websocket` | Require the WebSocket path; do not fall back to SSE. This mode opens a request-scoped connection. |
|
|
104
104
|
| `websocket-cached` | Require the WebSocket path and keep a session-scoped connection for later turns. |
|
|
105
105
|
| `sse` | Bypass the extension and use the original HTTP/SSE adapter. |
|
|
106
106
|
|
|
107
107
|
The endpoint is derived from the model's base URL by appending `/responses` when needed and switching `https:`/`http:` to `wss:`/`ws:`. The first frame is a standard `response.create` object and incoming frames are passed through Pi's existing Responses event normalizer. For `auto` and `websocket-cached`, Toolkit caches an open socket by session/model/endpoint/auth identity for up to five minutes idle or 55 minutes of age. When the current transcript has the cached request and response as a prefix, the next frame uses `previous_response_id` and only the new input items; if it cannot prove that relationship, it sends the full payload instead. A missing session id or `cacheRetention: "none"` disables reuse. The upstream must support the standard Responses WebSocket contract, including continuation semantics for `previous_response_id`; this does not rewrite a server that only exposes Codex's `/codex/responses` path or Codex-only authentication.
|
|
108
108
|
|
|
109
|
+
The first valid exposed event commits the WS attempt, including `response.created` even before text or tool arguments arrive. After that boundary, Toolkit never switches transport or splices a new generation into the same stream. Local transient WS failures and compatible WS error envelopes containing `stream closed before response.completed` retain their diagnostics with a Pi-recognized transport indication; Pi's native retry settings and budget decide whether to retry the failed step. Authentication, quota/billing, context/invalid-request/policy errors, malformed frames, and cancellation do not authorize Toolkit fallback. An SDK request deadline can cancel startup before a fallback is possible.
|
|
110
|
+
|
|
111
|
+
A failed connection is evicted. Any permitted fallback receives the original complete effective request, not a cached input delta; a new WS connection also starts from the full request. Pi remains responsible for effective context, completed tool results, failed-attempt history, tool execution, and usage. This prevents Toolkit from replaying completed tools as part of recovery, but does not guarantee idempotency if the model explicitly requests the same operation again. Direct SSE and errors from the original fallback SSE path are unchanged. Recovery depends on backend health and Pi's native retry policy; it is not token-level resumption or an unlimited retry loop.
|
|
112
|
+
|
|
109
113
|
This feature does not write Pi settings, model files, auth files, or global extensions. To keep the Toolkit adapter disabled, set `defaults.responsesWebSocket.enabled` to `false` or omit it. Once enabled, Pi's global `transport: "sse"` still bypasses the WebSocket path for a request.
|
|
110
114
|
|
|
111
115
|
---
|
package/docs/internals.md
CHANGED
|
@@ -7,6 +7,7 @@ Developer-facing reference for [pi-openai-toolkit](../README.md). The README kee
|
|
|
7
7
|
### Table of Contents
|
|
8
8
|
|
|
9
9
|
- [Configuration resolution](#configuration-resolution)
|
|
10
|
+
- [Codex Fast session control](#codex-fast-session-control)
|
|
10
11
|
- [Remote Context protocol](#remote-context-protocol)
|
|
11
12
|
- [Rollover lifecycle](#rollover-lifecycle)
|
|
12
13
|
- [Managed manual compact](#managed-manual-compact)
|
|
@@ -26,6 +27,20 @@ Selected invalid policy blocks dependent operations; unrelated model errors rema
|
|
|
26
27
|
|
|
27
28
|
---
|
|
28
29
|
|
|
30
|
+
### Codex Fast session control
|
|
31
|
+
|
|
32
|
+
[src/codex-fast/extension.ts](../src/codex-fast/extension.ts) registers the human-only, no-argument `/fast` toggle and an owned `ctx.ui.setStatus()` item. Default OFF requests `service_tier: "default"`; ON requests `"priority"`, the Codex Fast wire value. The status is the local request preference, not verified server acceptance or latency. Fast can increase usage/cost. No config setting, model tool, credential probe, retry, new header, reasoning-effort change or custom footer is added.
|
|
33
|
+
|
|
34
|
+
Native `openai-codex-responses` routes qualify. Non-official HTTP(S) `openai-responses` gateways qualify for recognizable GPT/Codex IDs or an existing exact Codex gateway identity, including a resolved virtual route. Unknown opaque aliases without that identity, Azure, other APIs and ordinary `api.openai.com` API-key routes remain untouched. A capability marker is not backend entitlement.
|
|
35
|
+
|
|
36
|
+
Version-1 `pi-openai-toolkit:codex-fast` custom entries contain only the owning Pi `sessionId` and `enabled` boolean. They are excluded from model context. Reload/resume and active-branch navigation reconstruct the latest valid owned preference; malformed latest metadata defaults OFF. Independent new/forked sessions ignore copied foreign preferences and start OFF. Remote window UUID changes do not reset the Pi session selection. A model switch shows N/A without discarding it. Missing identity or a failed append does not report a successful toggle.
|
|
37
|
+
|
|
38
|
+
The payload hook takes one selection snapshot and returns a new object only when a matching ordinary Responses request needs a tier change. Explicit OFF also overrides an existing Fast/Flex tier to Standard. Dedicated compaction shapes and malformed/foreign payloads are left unchanged. Fast loads after Web Search/image registration and before the final Astra effort pass, so its tier does not alter the pre-search cached extras used by separate search/image/compaction clients.
|
|
39
|
+
|
|
40
|
+
Native Pi and Toolkit gateway WebSocket adapters already compare non-input body fields before incremental reuse. A changed `service_tier` invalidates the incompatible `previous_response_id` delta; the existing transport/retry logic is retained. Offline official-Pi fixtures in [test/pi-codex-fast.test.ts](../test/pi-codex-fast.test.ts) cover terminal command/status, actual native/gateway HTTP/WebSocket bodies, saved reopen/reload, fork isolation and continuation transitions. They do not establish live Fast availability.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
29
44
|
### Remote Context protocol
|
|
30
45
|
|
|
31
46
|
Internal alpha endpoints, adapted from `@howaboua/pi-codex-conversion`:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-openai-toolkit",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.22.0",
|
|
4
4
|
"private": false,
|
|
5
5
|
"type": "module",
|
|
6
6
|
"engines": {
|
|
@@ -36,6 +36,7 @@
|
|
|
36
36
|
"./src/extension-runtime.ts",
|
|
37
37
|
"./src/web-search/extension.ts",
|
|
38
38
|
"./src/image-generation/extension.ts",
|
|
39
|
+
"./src/codex-fast/extension.ts",
|
|
39
40
|
"./src/codex-astra/extension.ts",
|
|
40
41
|
"./src/responses-websocket/extension.ts"
|
|
41
42
|
]
|
|
@@ -78,6 +79,9 @@
|
|
|
78
79
|
"src/image-generation/render.ts",
|
|
79
80
|
"src/image-generation/service.ts",
|
|
80
81
|
"src/image-generation/types.ts",
|
|
82
|
+
"src/codex-fast/extension.ts",
|
|
83
|
+
"src/codex-fast/payload.ts",
|
|
84
|
+
"src/codex-fast/state.ts",
|
|
81
85
|
"src/codex-astra/effort-planner.ts",
|
|
82
86
|
"src/codex-astra/extension.ts",
|
|
83
87
|
"src/responses-websocket/extension.ts",
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { loadToolkitConfig, resolveToolkitConfig } from "../config";
|
|
3
|
+
import { resolveEffectiveModel } from "../runtime";
|
|
4
|
+
import { isCodexFastModel, rewriteCodexFastPayload } from "./payload";
|
|
5
|
+
import { CODEX_FAST_ENTRY_TYPE, readCodexFastState, type CodexFastState } from "./state";
|
|
6
|
+
|
|
7
|
+
export const CODEX_FAST_STATUS_KEY = "pi-openai-toolkit:codex-fast";
|
|
8
|
+
|
|
9
|
+
function sessionId(ctx: ExtensionContext): string | undefined {
|
|
10
|
+
try {
|
|
11
|
+
const id = ctx.sessionManager.getSessionId();
|
|
12
|
+
return typeof id === "string" && id.trim() ? id : undefined;
|
|
13
|
+
} catch {
|
|
14
|
+
return undefined;
|
|
15
|
+
}
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export function registerCodexFastExtension(
|
|
19
|
+
pi: ExtensionAPI,
|
|
20
|
+
loadConfig: typeof loadToolkitConfig = loadToolkitConfig,
|
|
21
|
+
resolveModel: typeof resolveEffectiveModel = resolveEffectiveModel,
|
|
22
|
+
): void {
|
|
23
|
+
let currentSessionId: string | undefined;
|
|
24
|
+
let enabled = false;
|
|
25
|
+
let statusEpoch = 0;
|
|
26
|
+
|
|
27
|
+
function restore(ctx: ExtensionContext): void {
|
|
28
|
+
currentSessionId = sessionId(ctx);
|
|
29
|
+
try {
|
|
30
|
+
enabled = readCodexFastState(ctx.sessionManager.getBranch(), currentSessionId);
|
|
31
|
+
} catch {
|
|
32
|
+
enabled = false;
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
function synchronize(ctx: ExtensionContext): void {
|
|
36
|
+
if (sessionId(ctx) !== currentSessionId) restore(ctx);
|
|
37
|
+
}
|
|
38
|
+
function notify(ctx: ExtensionContext, text: string, level: "info" | "warning" = "info"): void {
|
|
39
|
+
if (ctx.hasUI) ctx.ui.notify(text, level);
|
|
40
|
+
}
|
|
41
|
+
async function eligibleModel(ctx: ExtensionContext, selected = ctx.model): Promise<ExtensionContext["model"]> {
|
|
42
|
+
try {
|
|
43
|
+
const effective = await resolveModel(ctx, selected);
|
|
44
|
+
if (!effective.ok) return undefined;
|
|
45
|
+
if (isCodexFastModel(effective.model)) return effective.model;
|
|
46
|
+
// Only opaque aliases need the existing compatibility marker. Known
|
|
47
|
+
// Codex/GPT routes work without loading or adding Toolkit settings.
|
|
48
|
+
if (effective.model.api !== "openai-responses") return undefined;
|
|
49
|
+
const resolved = resolveToolkitConfig(loadConfig(), selected);
|
|
50
|
+
if (resolved.invalidFeatures.includes("compatibility")) return undefined;
|
|
51
|
+
return isCodexFastModel(effective.model, resolved.gatewayModelKeys, selected) ? effective.model : undefined;
|
|
52
|
+
} catch {
|
|
53
|
+
return undefined;
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
async function refreshStatus(ctx: ExtensionContext, model = ctx.model): Promise<void> {
|
|
57
|
+
synchronize(ctx);
|
|
58
|
+
const epoch = ++statusEpoch;
|
|
59
|
+
const id = currentSessionId;
|
|
60
|
+
const eligible = id && await eligibleModel(ctx, model);
|
|
61
|
+
if (epoch !== statusEpoch || sessionId(ctx) !== id) return;
|
|
62
|
+
if (ctx.hasUI) ctx.ui.setStatus(CODEX_FAST_STATUS_KEY, eligible ? "Fast " + (enabled ? "ON" : "OFF") : "Fast N/A");
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
pi.on("session_start", async (_event, ctx) => { restore(ctx); await refreshStatus(ctx); });
|
|
66
|
+
pi.on("session_tree", async (_event, ctx) => { restore(ctx); await refreshStatus(ctx); });
|
|
67
|
+
pi.on("model_select", async (event, ctx) => { await refreshStatus(ctx, event.model); });
|
|
68
|
+
pi.on("session_shutdown", (_event, ctx) => {
|
|
69
|
+
statusEpoch++;
|
|
70
|
+
if (ctx.hasUI) ctx.ui.setStatus(CODEX_FAST_STATUS_KEY, undefined);
|
|
71
|
+
});
|
|
72
|
+
pi.registerCommand("fast", {
|
|
73
|
+
description: "Toggle Codex Fast for this session (default OFF; may increase usage/cost).",
|
|
74
|
+
handler: async (args, ctx) => {
|
|
75
|
+
if (!ctx.hasUI || (ctx.mode !== undefined && ctx.mode !== "tui")) {
|
|
76
|
+
notify(ctx, "/fast requires an interactive terminal.", "warning");
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
if (args.trim()) { notify(ctx, "Usage: /fast (toggle ON/OFF)", "warning"); return; }
|
|
80
|
+
synchronize(ctx);
|
|
81
|
+
const id = currentSessionId;
|
|
82
|
+
const selected = ctx.model;
|
|
83
|
+
const model = await eligibleModel(ctx, selected);
|
|
84
|
+
if (!id || sessionId(ctx) !== id || ctx.model !== selected) {
|
|
85
|
+
notify(ctx, "Fast unchanged: session/model unavailable or changed.", "warning");
|
|
86
|
+
return;
|
|
87
|
+
}
|
|
88
|
+
if (!model) { notify(ctx, "Fast unavailable on this model/route.", "warning"); return; }
|
|
89
|
+
const next = !enabled;
|
|
90
|
+
const data: CodexFastState = { version: 1, sessionId: id, enabled: next };
|
|
91
|
+
try {
|
|
92
|
+
pi.appendEntry(CODEX_FAST_ENTRY_TYPE, data);
|
|
93
|
+
} catch {
|
|
94
|
+
notify(ctx, "Fast unchanged: could not save the session preference.", "warning");
|
|
95
|
+
return;
|
|
96
|
+
}
|
|
97
|
+
enabled = next;
|
|
98
|
+
statusEpoch++;
|
|
99
|
+
ctx.ui.setStatus(CODEX_FAST_STATUS_KEY, "Fast " + (next ? "ON" : "OFF"));
|
|
100
|
+
notify(ctx, next
|
|
101
|
+
? "Fast ON for this session's next Codex requests; may increase usage/cost. Backend acceptance is not verified."
|
|
102
|
+
: "Fast OFF for this session's next Codex requests (Standard). No configuration file changed.");
|
|
103
|
+
},
|
|
104
|
+
});
|
|
105
|
+
pi.on("before_provider_request", async (event, ctx) => {
|
|
106
|
+
synchronize(ctx);
|
|
107
|
+
const id = currentSessionId;
|
|
108
|
+
const selection = enabled;
|
|
109
|
+
const selected = ctx.model;
|
|
110
|
+
if (!id) return undefined;
|
|
111
|
+
const model = await eligibleModel(ctx, selected);
|
|
112
|
+
if (!model || sessionId(ctx) !== id || ctx.model !== selected) return undefined;
|
|
113
|
+
return rewriteCodexFastPayload(event.payload, model.id, selection);
|
|
114
|
+
});
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
export default function codexFastExtension(pi: ExtensionAPI): void {
|
|
118
|
+
registerCodexFastExtension(pi);
|
|
119
|
+
}
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { isExactModelAllowed } from "../model-scope";
|
|
3
|
+
|
|
4
|
+
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
5
|
+
return !!value && typeof value === "object" && !Array.isArray(value);
|
|
6
|
+
}
|
|
7
|
+
|
|
8
|
+
/** This checks route identity, never backend entitlement or actual speed. */
|
|
9
|
+
export function isCodexFastModel(
|
|
10
|
+
model: ExtensionContext["model"],
|
|
11
|
+
gatewayModels: readonly string[] = [],
|
|
12
|
+
logicalModel?: ExtensionContext["model"],
|
|
13
|
+
): boolean {
|
|
14
|
+
if (!model || typeof model.id !== "string" || !model.id.trim()) return false;
|
|
15
|
+
if (model.api === "openai-codex-responses") return true;
|
|
16
|
+
if (model.api !== "openai-responses") return false;
|
|
17
|
+
try {
|
|
18
|
+
const url = new URL(model.baseUrl);
|
|
19
|
+
if (!["http:", "https:"].includes(url.protocol) || url.hostname.toLowerCase() === "api.openai.com") return false;
|
|
20
|
+
} catch {
|
|
21
|
+
return false;
|
|
22
|
+
}
|
|
23
|
+
return /^(?:gpt|codex)(?:-|$)/iu.test(model.id)
|
|
24
|
+
|| isExactModelAllowed(model, gatewayModels)
|
|
25
|
+
|| (logicalModel?.api === "pi-virtual" && isExactModelAllowed(logicalModel, gatewayModels));
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/** Only ordinary matching conversation payloads are owned by the session switch. */
|
|
29
|
+
export function rewriteCodexFastPayload(payload: unknown, modelId: string, enabled: boolean): Record<string, unknown> | undefined {
|
|
30
|
+
if (!isRecord(payload) || payload.model !== modelId || !Array.isArray(payload.input)) return undefined;
|
|
31
|
+
if (!payload.input.every(isRecord)) return undefined;
|
|
32
|
+
if (payload.input.some((item) => item.type === "compaction_trigger")) return undefined;
|
|
33
|
+
if (Array.isArray(payload.tools) && payload.tools.some((tool) => isRecord(tool) && tool.type === "remote_compaction_v2")) return undefined;
|
|
34
|
+
const tier = enabled ? "priority" : "default";
|
|
35
|
+
return payload.service_tier === tier ? undefined : { ...payload, service_tier: tier };
|
|
36
|
+
}
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
export const CODEX_FAST_ENTRY_TYPE = "pi-openai-toolkit:codex-fast";
|
|
2
|
+
|
|
3
|
+
export type CodexFastState = {
|
|
4
|
+
version: 1;
|
|
5
|
+
sessionId: string;
|
|
6
|
+
enabled: boolean;
|
|
7
|
+
};
|
|
8
|
+
|
|
9
|
+
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
10
|
+
return !!value && typeof value === "object" && !Array.isArray(value);
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
/** Session metadata, not model input. Foreign fork/copy entries never apply. */
|
|
14
|
+
export function readCodexFastState(entries: readonly unknown[], sessionId: string | undefined): boolean {
|
|
15
|
+
if (!sessionId) return false;
|
|
16
|
+
for (let index = entries.length - 1; index >= 0; index--) {
|
|
17
|
+
const entry = entries[index];
|
|
18
|
+
if (!isRecord(entry) || entry.type !== "custom" || entry.customType !== CODEX_FAST_ENTRY_TYPE) continue;
|
|
19
|
+
const data = entry.data;
|
|
20
|
+
// A malformed latest preference must not resurrect an older ON.
|
|
21
|
+
if (!isRecord(data) || data.version !== 1 || typeof data.sessionId !== "string" || typeof data.enabled !== "boolean") return false;
|
|
22
|
+
if (data.sessionId === sessionId) return data.enabled;
|
|
23
|
+
}
|
|
24
|
+
return false;
|
|
25
|
+
}
|
|
@@ -199,10 +199,69 @@ function buildCacheKey(model: Model<Api>, options: StreamOptions, requestHeaders
|
|
|
199
199
|
return [model.provider, model.id, cleanBaseUrl(model.baseUrl), options.sessionId, stableHash(JSON.stringify(identity))].join("\u0000");
|
|
200
200
|
}
|
|
201
201
|
|
|
202
|
+
// Only transport failures (not arbitrary caught errors) authorize auto fallback.
|
|
203
|
+
class RecoverableWebSocketError extends Error {}
|
|
204
|
+
|
|
205
|
+
const STREAM_INTERRUPTION = /\bstream closed before response\.completed\b/iu;
|
|
206
|
+
const TERMINAL_FAILURE_TEXT = /quota|billing|budget|usage.?limit|api.?key|unauthori[sz]ed|forbidden|authenticat|permission|invalid|malformed|unsupported|policy|context.?length|context.?overflow|\babort(?:ed)?\b|\bcancell?ed\b/iu;
|
|
207
|
+
const TRANSIENT_ERROR_CODES = new Set(["stream_error", "server_error", "request_timeout"]);
|
|
208
|
+
|
|
209
|
+
function transportError(error: unknown): Error {
|
|
210
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
211
|
+
if ((error instanceof Error && error.name === "AbortError") || TERMINAL_FAILURE_TEXT.test(message)) {
|
|
212
|
+
return error instanceof Error ? error : new Error(message);
|
|
213
|
+
}
|
|
214
|
+
return new RecoverableWebSocketError(/^WebSocket (?:error|closed|.*timeout)/iu.test(message)
|
|
215
|
+
? message : `WebSocket error: ${message}`);
|
|
216
|
+
}
|
|
217
|
+
|
|
202
218
|
function socketError(event: WebSocketEvent): Error {
|
|
203
|
-
if (typeof event.message === "string" && event.message.length > 0) return
|
|
204
|
-
if (event.error instanceof Error) return event.error;
|
|
205
|
-
return
|
|
219
|
+
if (typeof event.message === "string" && event.message.length > 0) return transportError(event.message);
|
|
220
|
+
if (event.error instanceof Error) return transportError(event.error);
|
|
221
|
+
return transportError("WebSocket transport failed");
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
function socketCloseError(event: WebSocketEvent, streaming: boolean): Error {
|
|
225
|
+
const suffix = event.reason ? `: ${event.reason}` : "";
|
|
226
|
+
if ([1002, 1003, 1007, 1008].includes(event.code ?? 0) || TERMINAL_FAILURE_TEXT.test(event.reason ?? "")) {
|
|
227
|
+
return new Error(`Responses connection rejected${event.code ? ` (${event.code})` : ""}${suffix}`);
|
|
228
|
+
}
|
|
229
|
+
return transportError(streaming
|
|
230
|
+
? `WebSocket closed before response completion${suffix}`
|
|
231
|
+
: `WebSocket closed${event.code ? ` ${event.code}` : ""}${suffix}`);
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
function compatibleFailureMetadata(error: Record<string, unknown>): boolean {
|
|
235
|
+
if (typeof error.message === "string" && TERMINAL_FAILURE_TEXT.test(error.message)) return false;
|
|
236
|
+
for (const field of ["code", "type"]) {
|
|
237
|
+
const value = error[field];
|
|
238
|
+
if (value !== undefined && value !== null && value !== "" &&
|
|
239
|
+
(typeof value !== "string" || !TRANSIENT_ERROR_CODES.has(value.toLowerCase()))) return false;
|
|
240
|
+
}
|
|
241
|
+
for (const field of ["status", "status_code"]) {
|
|
242
|
+
const value = error[field];
|
|
243
|
+
if (value !== undefined && value !== null &&
|
|
244
|
+
(typeof value !== "number" || ![408, 429, 500, 502, 503, 504].includes(value))) return false;
|
|
245
|
+
}
|
|
246
|
+
return true;
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
function normalizeStreamInterruption(event: Record<string, unknown>): Record<string, unknown> | undefined {
|
|
250
|
+
const response = isRecord(event.response) ? event.response : undefined;
|
|
251
|
+
const error = event.type === "error" ? event : event.type === "response.failed" && isRecord(response?.error) ? response.error : undefined;
|
|
252
|
+
if (!error || typeof error.message !== "string" || !STREAM_INTERRUPTION.test(error.message) ||
|
|
253
|
+
TERMINAL_FAILURE_TEXT.test(error.message)) return undefined;
|
|
254
|
+
// Envelope type/status describe the event/response, not its failure category.
|
|
255
|
+
// Still check their code/HTTP status so a nested transient message cannot
|
|
256
|
+
// override explicit fatal metadata on an enclosing envelope.
|
|
257
|
+
const metadata: Record<string, unknown>[] = [{ ...event, type: undefined }];
|
|
258
|
+
if (event.type === "response.failed") {
|
|
259
|
+
metadata.push({ ...response, type: undefined, status: response?.status === "failed" ? undefined : response?.status }, error);
|
|
260
|
+
}
|
|
261
|
+
if (isRecord(error.error)) metadata.push(error.error);
|
|
262
|
+
if (metadata.some((value) => !compatibleFailureMetadata(value))) return undefined;
|
|
263
|
+
const normalized = { ...error, message: `WebSocket error: ${error.message}` };
|
|
264
|
+
return event.type === "error" ? normalized : { ...event, response: { ...response, error: normalized } };
|
|
206
265
|
}
|
|
207
266
|
|
|
208
267
|
function closeSocket(socket: WebSocketLike | undefined, reason = "done"): void {
|
|
@@ -247,8 +306,9 @@ export function closeResponsesWebSocketSessions(sessionId?: string): void {
|
|
|
247
306
|
}
|
|
248
307
|
|
|
249
308
|
|
|
250
|
-
function liveSignals(options: StreamOptions, init?: RequestInit): AbortSignal[] {
|
|
251
|
-
return [options.signal, init?.signal
|
|
309
|
+
function liveSignals(options: StreamOptions, input: RequestInfo | URL, init?: RequestInit): AbortSignal[] {
|
|
310
|
+
return [...new Set([options.signal, init?.signal, input instanceof Request ? input.signal : undefined]
|
|
311
|
+
.filter((signal): signal is AbortSignal => signal !== undefined && signal !== null))];
|
|
252
312
|
}
|
|
253
313
|
|
|
254
314
|
async function connectWebSocket(
|
|
@@ -258,7 +318,7 @@ async function connectWebSocket(
|
|
|
258
318
|
timeoutMs: number,
|
|
259
319
|
): Promise<WebSocketLike> {
|
|
260
320
|
const WebSocketCtor = globalThis.WebSocket as unknown as WebSocketConstructor | undefined;
|
|
261
|
-
if (typeof WebSocketCtor !== "function") throw
|
|
321
|
+
if (typeof WebSocketCtor !== "function") throw transportError("WebSocket transport is not available in this runtime");
|
|
262
322
|
if (signals.some((signal) => signal.aborted)) throw new Error("Request was aborted");
|
|
263
323
|
|
|
264
324
|
return new Promise((resolve, reject) => {
|
|
@@ -286,9 +346,7 @@ async function connectWebSocket(
|
|
|
286
346
|
resolve(socket!);
|
|
287
347
|
};
|
|
288
348
|
const onError = (event: WebSocketEvent) => fail(socketError(event));
|
|
289
|
-
const onClose = (event: WebSocketEvent) => fail(
|
|
290
|
-
`WebSocket closed${typeof event.code === "number" ? ` ${event.code}` : ""}${event.reason ? ` ${event.reason}` : ""}`,
|
|
291
|
-
));
|
|
349
|
+
const onClose = (event: WebSocketEvent) => fail(socketCloseError(event, false));
|
|
292
350
|
const onAbort = () => fail(new Error("Request was aborted"));
|
|
293
351
|
|
|
294
352
|
try {
|
|
@@ -301,7 +359,8 @@ async function connectWebSocket(
|
|
|
301
359
|
socket.addEventListener("error", onError);
|
|
302
360
|
socket.addEventListener("close", onClose);
|
|
303
361
|
for (const signal of signals) signal.addEventListener("abort", onAbort, { once: true });
|
|
304
|
-
if (
|
|
362
|
+
if (signals.some((signal) => signal.aborted)) { onAbort(); return; }
|
|
363
|
+
if (timeoutMs > 0) timeout = setTimeout(() => fail(transportError(`WebSocket connect timeout after ${timeoutMs}ms`)), timeoutMs);
|
|
305
364
|
});
|
|
306
365
|
}
|
|
307
366
|
|
|
@@ -439,22 +498,26 @@ function completedContinuation(entry: CachedSocketEntry | undefined, fullPayload
|
|
|
439
498
|
return true;
|
|
440
499
|
}
|
|
441
500
|
|
|
442
|
-
function createSseResponse(
|
|
501
|
+
async function createSseResponse(
|
|
443
502
|
socket: WebSocketLike,
|
|
444
503
|
signals: AbortSignal[],
|
|
445
504
|
idleTimeoutMs: number,
|
|
446
505
|
requestFrame: string,
|
|
447
506
|
onTerminal: (event: Record<string, unknown>) => boolean,
|
|
448
507
|
onRelease: (keep: boolean) => void,
|
|
449
|
-
|
|
508
|
+
allowBootstrapFallback: boolean,
|
|
509
|
+
): Promise<Response> {
|
|
450
510
|
const encoder = new TextEncoder();
|
|
451
|
-
let
|
|
511
|
+
let resolveReady!: () => void;
|
|
512
|
+
let rejectReady!: (error: unknown) => void;
|
|
513
|
+
const ready = new Promise<void>((resolve, reject) => { resolveReady = resolve; rejectReady = reject; });
|
|
514
|
+
let cancelBody: (reason?: unknown) => void = () => {};
|
|
452
515
|
const body = new ReadableStream<Uint8Array>({
|
|
453
516
|
start(controller) {
|
|
454
517
|
let closed = false;
|
|
455
|
-
let
|
|
518
|
+
let committed = false;
|
|
456
519
|
let idleTimer: ReturnType<typeof setTimeout> | undefined;
|
|
457
|
-
let
|
|
520
|
+
let pending = Promise.resolve();
|
|
458
521
|
const cleanup = () => {
|
|
459
522
|
if (idleTimer) clearTimeout(idleTimer);
|
|
460
523
|
socket.removeEventListener("message", onMessage);
|
|
@@ -462,73 +525,75 @@ function createSseResponse(
|
|
|
462
525
|
socket.removeEventListener("close", onClose);
|
|
463
526
|
for (const signal of signals) signal.removeEventListener("abort", onAbort);
|
|
464
527
|
};
|
|
465
|
-
const release = (keep: boolean) => {
|
|
466
|
-
if (released) return;
|
|
467
|
-
released = true;
|
|
468
|
-
onRelease(keep);
|
|
469
|
-
};
|
|
470
|
-
const finish = () => {
|
|
471
|
-
if (closed) return;
|
|
472
|
-
closed = true;
|
|
473
|
-
cleanup();
|
|
474
|
-
controller.close();
|
|
475
|
-
};
|
|
476
528
|
const fail = (error: unknown) => {
|
|
477
529
|
if (closed) return;
|
|
478
530
|
closed = true;
|
|
479
531
|
cleanup();
|
|
480
|
-
|
|
481
|
-
|
|
532
|
+
onRelease(false);
|
|
533
|
+
if (!committed) {
|
|
534
|
+
if (signals.some((signal) => signal.aborted) || (allowBootstrapFallback && error instanceof RecoverableWebSocketError)) {
|
|
535
|
+
rejectReady(error);
|
|
536
|
+
} else {
|
|
537
|
+
// Return an errored body for deterministic failures/explicit WS.
|
|
538
|
+
// Rejecting fetch would make the SDK recast these as connection errors.
|
|
539
|
+
resolveReady();
|
|
540
|
+
}
|
|
541
|
+
}
|
|
482
542
|
controller.error(error);
|
|
483
543
|
};
|
|
484
544
|
const armIdleTimer = () => {
|
|
485
545
|
if (idleTimeoutMs <= 0) return;
|
|
486
546
|
if (idleTimer) clearTimeout(idleTimer);
|
|
487
|
-
idleTimer = setTimeout(() => fail(
|
|
488
|
-
};
|
|
489
|
-
const onMessage = (event: WebSocketEvent) => {
|
|
490
|
-
void (async () => {
|
|
491
|
-
try {
|
|
492
|
-
const text = await decodeSocketData(event.data);
|
|
493
|
-
if (!text) return;
|
|
494
|
-
const parsed: unknown = JSON.parse(text);
|
|
495
|
-
if (!isRecord(parsed) || typeof parsed.type !== "string") throw new Error("Invalid Responses WebSocket event");
|
|
496
|
-
controller.enqueue(encoder.encode(`event: ${parsed.type}\ndata: ${JSON.stringify(parsed)}\n\n`));
|
|
497
|
-
armIdleTimer();
|
|
498
|
-
if (TERMINAL_EVENTS.has(parsed.type)) {
|
|
499
|
-
sawTerminal = true;
|
|
500
|
-
const keep = onTerminal(parsed);
|
|
501
|
-
release(keep);
|
|
502
|
-
finish();
|
|
503
|
-
}
|
|
504
|
-
} catch (error) {
|
|
505
|
-
fail(error);
|
|
506
|
-
}
|
|
507
|
-
})();
|
|
547
|
+
idleTimer = setTimeout(() => fail(transportError(`WebSocket idle timeout after ${idleTimeoutMs}ms`)), idleTimeoutMs);
|
|
508
548
|
};
|
|
509
|
-
const
|
|
510
|
-
|
|
511
|
-
if (sawTerminal) finish();
|
|
512
|
-
else fail(new Error(`WebSocket closed before response completion${event.reason ? `: ${event.reason}` : ""}`));
|
|
549
|
+
const enqueue = (operation: () => Promise<void> | void) => {
|
|
550
|
+
pending = pending.then(() => { if (!closed) return operation(); }).catch(fail);
|
|
513
551
|
};
|
|
552
|
+
const onMessage = (event: WebSocketEvent) => enqueue(async () => {
|
|
553
|
+
const text = await decodeSocketData(event.data);
|
|
554
|
+
if (closed) return;
|
|
555
|
+
if (text === undefined) throw new Error("Invalid Responses WebSocket frame");
|
|
556
|
+
const parsed: unknown = JSON.parse(text);
|
|
557
|
+
if (!isRecord(parsed) || typeof parsed.type !== "string" || !parsed.type) throw new Error("Invalid Responses WebSocket event");
|
|
558
|
+
const normalized = normalizeStreamInterruption(parsed);
|
|
559
|
+
if (!committed && allowBootstrapFallback && normalized) {
|
|
560
|
+
fail(transportError("stream closed before response.completed"));
|
|
561
|
+
return;
|
|
562
|
+
}
|
|
563
|
+
const forwarded = normalized ?? parsed;
|
|
564
|
+
controller.enqueue(encoder.encode(`event: ${forwarded.type}\ndata: ${JSON.stringify(forwarded)}\n\n`));
|
|
565
|
+
committed = true;
|
|
566
|
+
resolveReady();
|
|
567
|
+
armIdleTimer();
|
|
568
|
+
if (TERMINAL_EVENTS.has(parsed.type)) {
|
|
569
|
+
closed = true;
|
|
570
|
+
cleanup();
|
|
571
|
+
const keep = onTerminal(parsed);
|
|
572
|
+
onRelease(keep);
|
|
573
|
+
controller.close();
|
|
574
|
+
}
|
|
575
|
+
});
|
|
576
|
+
const onError = (event: WebSocketEvent) => enqueue(() => fail(socketError(event)));
|
|
577
|
+
const onClose = (event: WebSocketEvent) => enqueue(() => fail(socketCloseError(event, true)));
|
|
578
|
+
// Abort/timeout deliberately bypass decoding: a stuck decoder must not
|
|
579
|
+
// prevent cancellation. Its late result sees closed and is discarded.
|
|
514
580
|
const onAbort = () => fail(new Error("Request was aborted"));
|
|
515
|
-
|
|
516
581
|
cancelBody = (reason) => fail(reason instanceof Error ? reason : new Error("Response stream was cancelled"));
|
|
517
582
|
socket.addEventListener("message", onMessage);
|
|
518
583
|
socket.addEventListener("error", onError);
|
|
519
584
|
socket.addEventListener("close", onClose);
|
|
520
585
|
for (const signal of signals) signal.addEventListener("abort", onAbort, { once: true });
|
|
586
|
+
if (signals.some((signal) => signal.aborted)) { onAbort(); return; }
|
|
521
587
|
armIdleTimer();
|
|
522
588
|
try {
|
|
523
589
|
socket.send(requestFrame);
|
|
524
590
|
} catch (error) {
|
|
525
|
-
fail(error);
|
|
591
|
+
fail(transportError(error));
|
|
526
592
|
}
|
|
527
593
|
},
|
|
528
|
-
cancel(reason) {
|
|
529
|
-
cancelBody(reason);
|
|
530
|
-
},
|
|
594
|
+
cancel(reason) { cancelBody(reason); },
|
|
531
595
|
});
|
|
596
|
+
await ready;
|
|
532
597
|
return new Response(body, {
|
|
533
598
|
status: 200,
|
|
534
599
|
headers: { "content-type": "text/event-stream", connection: "keep-alive" },
|
|
@@ -573,12 +638,12 @@ export function createResponsesWebSocketFetch(model: Model<Api>, options: Stream
|
|
|
573
638
|
}
|
|
574
639
|
if (!isRecord(payload)) return fallback(input, init);
|
|
575
640
|
|
|
576
|
-
const signals = liveSignals(options, init);
|
|
641
|
+
const signals = liveSignals(options, input, init);
|
|
577
642
|
const requestId = options.sessionId?.slice(0, 64) || crypto.randomUUID();
|
|
578
643
|
const fetchHeaders = requestHeaders(input, init);
|
|
579
644
|
const headers = buildWebSocketHeaders(model, options, fetchHeaders, requestId);
|
|
580
645
|
const cacheKey = shouldCacheSocket(options) ? buildCacheKey(model, options, fetchHeaders) : undefined;
|
|
581
|
-
let
|
|
646
|
+
let release: ((keep: boolean) => void) | undefined;
|
|
582
647
|
try {
|
|
583
648
|
const lease = await acquireWebSocket(
|
|
584
649
|
buildResponsesWebSocketUrl(model),
|
|
@@ -588,21 +653,27 @@ export function createResponsesWebSocketFetch(model: Model<Api>, options: Stream
|
|
|
588
653
|
cacheKey,
|
|
589
654
|
options.sessionId,
|
|
590
655
|
);
|
|
591
|
-
|
|
656
|
+
let released = false;
|
|
657
|
+
release = (keep) => {
|
|
658
|
+
if (released) return;
|
|
659
|
+
released = true;
|
|
660
|
+
lease.release(keep);
|
|
661
|
+
};
|
|
592
662
|
const fullPayload = payload;
|
|
593
663
|
const requestPayload = lease.entry ? buildCachedRequestBody(lease.entry, fullPayload) : fullPayload;
|
|
594
664
|
const requestFrame = JSON.stringify({ type: "response.create", ...requestPayload });
|
|
595
|
-
return createSseResponse(
|
|
665
|
+
return await createSseResponse(
|
|
596
666
|
lease.socket,
|
|
597
667
|
signals,
|
|
598
668
|
options.timeoutMs ?? 0,
|
|
599
669
|
requestFrame,
|
|
600
670
|
(event) => completedContinuation(lease.entry, fullPayload, event),
|
|
601
|
-
|
|
671
|
+
release,
|
|
672
|
+
transportMode(options) === "auto",
|
|
602
673
|
);
|
|
603
674
|
} catch (error) {
|
|
604
|
-
|
|
605
|
-
if (transportMode(options) === "auto" && !signals.some((signal) => signal.aborted)) {
|
|
675
|
+
release?.(false);
|
|
676
|
+
if (error instanceof RecoverableWebSocketError && transportMode(options) === "auto" && !signals.some((signal) => signal.aborted)) {
|
|
606
677
|
return fallback(input, init);
|
|
607
678
|
}
|
|
608
679
|
throw error;
|