codex-copilot-dx 0.8.6 → 0.8.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/package.json +1 -1
- package/src/adapter.mjs +1 -1
- package/src/cli-status.mjs +9 -1
- package/src/response-history.mjs +24 -1
- package/src/responses-bridge.mjs +54 -16
- package/src/responses-handler.mjs +21 -4
- package/src/responses-proxy.mjs +33 -1
- package/src/stream-performance.mjs +69 -1
package/README.md
CHANGED
|
@@ -12,7 +12,7 @@ One in-process adapter listens on `127.0.0.1:2026` by default and exposes:
|
|
|
12
12
|
- `POST /v1/responses/compact`
|
|
13
13
|
- `GET /v1/models`
|
|
14
14
|
|
|
15
|
-
Responses-only models and compaction use Copilot's Responses transport directly. For
|
|
15
|
+
Responses-only models and compaction use Copilot's Responses transport directly. For requests routed through Chat Completions, CCDX converts the supported text, image, and function-tool subset and rejects incompatible message content rather than turning it into JSON text. Chat `length` and `content_filter` finish reasons become incomplete Responses in both streaming and non-streaming modes. Both streaming SSE and non-streaming responses are supported.
|
|
16
16
|
|
|
17
17
|
The adapter handles response history, encrypted reasoning state, function/custom tools, images, compaction, cancellation, retries, and usage metadata within the compatibility boundaries below. Routing prefers endpoint metadata from Copilot's live or cached catalog; when metadata is unavailable, a small built-in set identifies known Responses-only GPT models.
|
|
18
18
|
|
|
@@ -259,7 +259,7 @@ For the complete machine-readable status payload:
|
|
|
259
259
|
curl -s http://127.0.0.1:2026/_ccdx/status | jq
|
|
260
260
|
```
|
|
261
261
|
|
|
262
|
-
The status endpoint is restricted to the socket's real loopback address even when LAN binding is explicitly enabled. It reports fixed-size request counters, bounded TTFT/TPOT histograms, admission pressure, memory use, response-history size, image queue/cache and adaptive-history state, model-cache counts, token expiry state, and the ten most recent upstream `response.failed` diagnostics retained by the running process. Failure diagnostics include the event type, model, error code/message, response/request IDs when supplied upstream, and retry outcome. Common credential forms, long opaque encrypted values, and explicitly labelled input/prompt/tool-content echoes are redacted. They are cleared when the adapter restarts. No request/response bodies, credential fields, or account profiles are copied into status. Arbitrary upstream prose cannot be guaranteed free of sensitive content; review diagnostics before sharing them.
|
|
262
|
+
The status endpoint is restricted to the socket's real loopback address even when LAN binding is explicitly enabled. It reports fixed-size HTTP request counters separately from bounded `completed` / `incomplete` / `failed` / `cancelled` response outcomes by model and origin, plus bounded incomplete-reason counts, TTFT/TPOT histograms, admission pressure, memory use, response-history size/tree concentration/lookup misses, image queue/cache and adaptive-history state, model-cache counts, token expiry state, and the ten most recent upstream `response.failed` diagnostics retained by the running process. HTTP errors without a model terminal remain "without terminal" and retain their known origin; they are not counted as model failures. History eviction markers are not failed-request counts; `ccdx status` warns near the cache limit without deleting or compressing history. Failure diagnostics include the event type, model, error code/message, response/request IDs when supplied upstream, and retry outcome. Common credential forms, long opaque encrypted values, and explicitly labelled input/prompt/tool-content echoes are redacted. They are cleared when the adapter restarts. No request/response bodies, credential fields, or account profiles are copied into status. Arbitrary upstream prose cannot be guaranteed free of sensitive content; review diagnostics before sharing them.
|
|
263
263
|
|
|
264
264
|
`image_mcp` has its own HTTP route counter. After the image handler has been used, `image_generation` separately reports active, succeeded, failed, busy, cancelled, and chat-file delivery-failure counts, including tool errors returned with HTTP 200. A delivery failure can accompany a successful generation; it never causes another generation. Before handler initialization this field is `null`, meaning not observed, not necessarily disabled. Status never initializes that handler, reads its provider credential, installs guidance, or probes the provider. Input-image optimization metrics remain separate. `logging` reports pending records/bytes, queue caps, overflow drops, and write failures for usage and enabled debug-file logging.
|
|
265
265
|
|
|
@@ -367,7 +367,7 @@ Set `CCDX_LOG_PATH=1` to mirror terminal logs to `~/.local/share/codex-copilot-d
|
|
|
367
367
|
|
|
368
368
|
Long computer-use sessions can accumulate screenshots that are sent again on later turns. This can increase local preparation time, upstream handshake latency, and the chance of a `413 Payload Too Large` response.
|
|
369
369
|
|
|
370
|
-
The response-history cache is process-local and byte/entry bounded. Restarting the adapter discards it; a `previous_response_id` stored only in that process becomes unavailable. CCDX does not persist conversation history to disk. Clients can still submit explicit history in the request body, subject to the same limits.
|
|
370
|
+
The response-history cache is process-local and byte/entry bounded. Restarting the adapter discards it; a `previous_response_id` stored only in that process becomes unavailable. CCDX does not persist conversation history to disk. Near-limit status diagnostics report the largest tree and actual lookup misses, but do not recover an already evicted ID or change the cache budget. Clients can still submit explicit history in the request body, subject to the same limits.
|
|
371
371
|
|
|
372
372
|
CCDX attempts to downsample inline base64 images to model-appropriate pixel bounds, with a 2048 px long-edge fallback, and initially encodes them as WebP quality 82. It never replaces an image with a larger encoding and uses one global concurrency limit across direct images and function/custom-tool outputs. GIFs and non-inline references are not re-encoded. An encoding/decode failure keeps the original image; the final request-byte limit still applies.
|
|
373
373
|
|
package/package.json
CHANGED
package/src/adapter.mjs
CHANGED
|
@@ -389,7 +389,7 @@ export function createAdapterHandler(options = {}) {
|
|
|
389
389
|
if (requestFinished) return;
|
|
390
390
|
requestFinished = true;
|
|
391
391
|
try { complete({ statusCode, aborted }); } catch {}
|
|
392
|
-
try { streamPerformance?.finish({ failed: aborted || statusCode >= 400 }); } catch {}
|
|
392
|
+
try { streamPerformance?.finish({ failed: aborted || statusCode >= 400, aborted }); } catch {}
|
|
393
393
|
try { finishTerminalActivity(); } catch {}
|
|
394
394
|
};
|
|
395
395
|
const containUnexpectedError = (error) => {
|
package/src/cli-status.mjs
CHANGED
|
@@ -132,6 +132,7 @@ export async function readAdapterStatus({
|
|
|
132
132
|
export function formatAdapterStatus({ baseUrl, data }, { commandName = "ccdx", cliVersion = data.version } = {}) {
|
|
133
133
|
const requests = data.requests || {};
|
|
134
134
|
const performance = data.stream_performance?.by_route?.responses || {};
|
|
135
|
+
const terminalOutcomes = performance.terminal_outcomes?.totals;
|
|
135
136
|
const admission = data.admission || {};
|
|
136
137
|
const history = data.response_history || {};
|
|
137
138
|
const images = data.image_optimization || {};
|
|
@@ -150,11 +151,18 @@ export function formatAdapterStatus({ baseUrl, data }, { commandName = "ccdx", c
|
|
|
150
151
|
adapterLine,
|
|
151
152
|
status("info", `Requests: ${count(requests.completed)}/${count(requests.total)} completed, ${count(requests.active)} active, ${count(requests.status_4xx)} 4xx, ${count(requests.status_5xx)} 5xx, ${count(requests.aborted)} aborted`),
|
|
152
153
|
status("info", `Responses stream: TTFT avg ${latency(performance.ttft_ms?.avg)} (${count(performance.ttft_ms?.samples)} samples), TPOT avg ${tpot(performance.tpot_us)}`),
|
|
154
|
+
...(terminalOutcomes ? [status("info", `Model outcomes: ${count(terminalOutcomes.completed)} completed, ${count(terminalOutcomes.incomplete)} incomplete, ${count(terminalOutcomes.failed)} failed, ${count(terminalOutcomes.cancelled)} cancelled, ${count(terminalOutcomes.unknown)} without terminal`)] : []),
|
|
153
155
|
status("info", `Admission: ${count(admission.activeRequests)} active, ${count(admission.queued)} queued, ${count(admission.rejected)} rejected, ${count(admission.timedOut)} timed out, wait avg ${latency(admission.waitMsAvg)}`),
|
|
154
156
|
status("info", `Memory: RSS ${mebibytes(processStats.rss_bytes)}, heap ${mebibytes(processStats.heap_used_bytes)}`),
|
|
155
|
-
status("info", `History: ${mebibytes(history.bytes)} / ${mebibytes(limits.response_history_max_bytes)}, ${count(history.entries)} entries, ${count(history.evicted)} evicted`),
|
|
157
|
+
status("info", `History: ${mebibytes(history.bytes)} / ${mebibytes(limits.response_history_max_bytes)}, ${count(history.entries)} entries, ${count(history.evicted)} evicted ID markers`),
|
|
156
158
|
status("info", `Image cache: ${count(images.cache_entries)} entries, ${cacheHitRate(images)} hits, ${mebibytes(images.cache_bytes)} / ${mebibytes(images.cache_max_bytes)}`),
|
|
157
159
|
];
|
|
160
|
+
if (Number.isFinite(history.tree_count)) {
|
|
161
|
+
const limit = Number(limits.response_history_max_bytes || history.maxBytes);
|
|
162
|
+
const nearLimit = Number.isFinite(limit) && limit > 0 && history.bytes / limit >= 0.9;
|
|
163
|
+
lines.push(status(nearLimit ? "warn" : "info",
|
|
164
|
+
`${nearLimit ? "History near limit" : "History diagnostics"}: largest tree ${mebibytes(history.largest_tree_bytes)} across ${count(history.tree_count)} trees; ${count(history.lookup_misses)} lookup misses, ${count(history.evicted_lookup_misses)} for evicted IDs`));
|
|
165
|
+
}
|
|
158
166
|
if (imageHistory && typeof imageHistory === "object") {
|
|
159
167
|
lines.push(status("info", `Visual history: ${count(imageHistory.active_recovery_trees)} recovery trees, ${count(imageHistory.adapted_requests)} adapted requests, ${count(imageHistory.historical_images_omitted)} older images omitted, ${count(imageHistory.timeouts_recorded)} timeouts`));
|
|
160
168
|
}
|
package/src/response-history.mjs
CHANGED
|
@@ -81,6 +81,9 @@ const treeLru = new Map();
|
|
|
81
81
|
const pinnedTrees = new Map();
|
|
82
82
|
const evictedIds = new Set();
|
|
83
83
|
let totalBytes = 0;
|
|
84
|
+
let evictedEntriesTotal = 0;
|
|
85
|
+
let lookupMisses = 0;
|
|
86
|
+
let evictedLookupMisses = 0;
|
|
84
87
|
const HISTORY_RUNTIME_CONFIG = loadRuntimeConfig();
|
|
85
88
|
let maxBytes = HISTORY_RUNTIME_CONFIG.responseHistoryMaxBytes;
|
|
86
89
|
let maxEntries = HISTORY_RUNTIME_CONFIG.responseHistoryMaxEntries;
|
|
@@ -133,6 +136,7 @@ function removeSubtree(rootId) {
|
|
|
133
136
|
const pending = [rootId];
|
|
134
137
|
const removed = new Set();
|
|
135
138
|
const affectedRoots = new Set();
|
|
139
|
+
let removedEntries = 0;
|
|
136
140
|
while (pending.length) {
|
|
137
141
|
const id = pending.pop();
|
|
138
142
|
if (removed.has(id)) continue;
|
|
@@ -149,7 +153,9 @@ function removeSubtree(rootId) {
|
|
|
149
153
|
totalBytes -= entry.bytes;
|
|
150
154
|
histories.delete(id);
|
|
151
155
|
rememberEvictedId(id);
|
|
156
|
+
removedEntries += 1;
|
|
152
157
|
}
|
|
158
|
+
evictedEntriesTotal += removedEntries;
|
|
153
159
|
for (const affectedRoot of affectedRoots) {
|
|
154
160
|
const entry = histories.get(affectedRoot);
|
|
155
161
|
if (!entry || entry.rootId !== affectedRoot) treeLru.delete(affectedRoot);
|
|
@@ -223,6 +229,9 @@ export function clearResponseHistory() {
|
|
|
223
229
|
pinnedTrees.clear();
|
|
224
230
|
evictedIds.clear();
|
|
225
231
|
totalBytes = 0;
|
|
232
|
+
evictedEntriesTotal = 0;
|
|
233
|
+
lookupMisses = 0;
|
|
234
|
+
evictedLookupMisses = 0;
|
|
226
235
|
return removed;
|
|
227
236
|
}
|
|
228
237
|
|
|
@@ -253,7 +262,18 @@ export function configureResponseHistoryForTests({ maxBytes: nextMaxBytes, maxEn
|
|
|
253
262
|
}
|
|
254
263
|
|
|
255
264
|
export function responseHistoryStats() {
|
|
265
|
+
const trees = new Map();
|
|
266
|
+
for (const entry of histories.values()) {
|
|
267
|
+
trees.set(entry.rootId, (trees.get(entry.rootId) || 0) + entry.bytes);
|
|
268
|
+
}
|
|
269
|
+
let largestTreeBytes = 0;
|
|
270
|
+
for (const bytes of trees.values()) largestTreeBytes = Math.max(largestTreeBytes, bytes);
|
|
256
271
|
return { entries: histories.size, bytes: totalBytes, evicted: evictedIds.size,
|
|
272
|
+
evicted_entries_total: evictedEntriesTotal,
|
|
273
|
+
lookup_misses: lookupMisses,
|
|
274
|
+
evicted_lookup_misses: evictedLookupMisses,
|
|
275
|
+
tree_count: trees.size,
|
|
276
|
+
largest_tree_bytes: largestTreeBytes,
|
|
257
277
|
pinnedTrees: pinnedTrees.size, maxBytes, maxEntries };
|
|
258
278
|
}
|
|
259
279
|
|
|
@@ -270,7 +290,10 @@ function responseHistoryChain(responseId) {
|
|
|
270
290
|
seen.add(currentId);
|
|
271
291
|
const entry = histories.get(currentId);
|
|
272
292
|
if (!entry) {
|
|
273
|
-
|
|
293
|
+
lookupMisses += 1;
|
|
294
|
+
const wasEvicted = evictedIds.has(currentId);
|
|
295
|
+
if (wasEvicted) evictedLookupMisses += 1;
|
|
296
|
+
const reason = wasEvicted ? " was evicted after reaching the local history limit" : " is not available";
|
|
274
297
|
throw httpError(`previous_response_id${reason}: ${currentId}`, 400);
|
|
275
298
|
}
|
|
276
299
|
chain.push(entry);
|
package/src/responses-bridge.mjs
CHANGED
|
@@ -15,7 +15,14 @@ import {
|
|
|
15
15
|
upstreamChatStreamError,
|
|
16
16
|
withChatStreamUsage,
|
|
17
17
|
} from "./stream-contract.mjs";
|
|
18
|
-
import {
|
|
18
|
+
import {
|
|
19
|
+
isChatOutputDelta,
|
|
20
|
+
markFirstOutput,
|
|
21
|
+
markOutputTokens,
|
|
22
|
+
markResponseErrorOrigin,
|
|
23
|
+
markResponseTerminal,
|
|
24
|
+
markStreamFailure,
|
|
25
|
+
} from "./stream-performance.mjs";
|
|
19
26
|
|
|
20
27
|
function cloneJson(value) {
|
|
21
28
|
return value === undefined ? undefined : structuredClone(value);
|
|
@@ -66,18 +73,20 @@ export function responsesToChat(body) {
|
|
|
66
73
|
|
|
67
74
|
const messageContent = (content) => {
|
|
68
75
|
if (typeof content === "string") return content;
|
|
69
|
-
if (
|
|
76
|
+
if (content === null) return null;
|
|
77
|
+
if (!Array.isArray(content)) throw chatCompatibilityError("message content", typeof content);
|
|
70
78
|
const parts = [];
|
|
71
79
|
for (const part of content) {
|
|
72
|
-
if (!part || typeof part !== "object")
|
|
80
|
+
if (!part || typeof part !== "object") throw chatCompatibilityError("message content", part?.type);
|
|
73
81
|
if (["input_text", "output_text", "text"].includes(part.type)) {
|
|
74
|
-
|
|
82
|
+
if (typeof part.text !== "string") throw chatCompatibilityError("message content", part.type);
|
|
83
|
+
parts.push({ type: "text", text: part.text });
|
|
75
84
|
} else if (["input_image", "image_url", "image"].includes(part.type)) {
|
|
76
85
|
const imagePart = chatImagePart(part);
|
|
77
86
|
if (imagePart) parts.push(imagePart);
|
|
78
|
-
else
|
|
87
|
+
else throw chatCompatibilityError("message content", part.type);
|
|
79
88
|
} else {
|
|
80
|
-
|
|
89
|
+
throw chatCompatibilityError("message content", part.type);
|
|
81
90
|
}
|
|
82
91
|
}
|
|
83
92
|
if (parts.length === 1 && parts[0].type === "text") return parts[0].text;
|
|
@@ -176,11 +185,20 @@ export function responsesToChat(body) {
|
|
|
176
185
|
|
|
177
186
|
function uid() { return randomUUID().replace(/-/g, ""); }
|
|
178
187
|
|
|
188
|
+
function chatTerminal(finishReason) {
|
|
189
|
+
const reason = finishReason === "length" ? "max_output_tokens"
|
|
190
|
+
: finishReason === "content_filter" ? "content_filter" : null;
|
|
191
|
+
return reason
|
|
192
|
+
? { status: "incomplete", incomplete_details: { reason } }
|
|
193
|
+
: { status: "completed" };
|
|
194
|
+
}
|
|
195
|
+
|
|
179
196
|
export function chatToResponses(chatResp, model) {
|
|
180
197
|
const id = `resp_${uid()}`, choice = chatResp.choices?.[0], msg = choice?.message, output = [];
|
|
181
|
-
|
|
182
|
-
if (msg?.
|
|
183
|
-
|
|
198
|
+
const terminal = chatTerminal(choice?.finish_reason);
|
|
199
|
+
if (msg?.content) output.push({ type: "message", id: `msg_${uid()}`, role: "assistant", status: terminal.status, content: [{ type: "output_text", text: msg.content }] });
|
|
200
|
+
if (msg?.tool_calls) for (const tc of msg.tool_calls) output.push({ type: "function_call", id: tc.id, call_id: tc.id, name: tc.function.name, arguments: tc.function.arguments, status: terminal.status });
|
|
201
|
+
return { id, object: "response", ...terminal, model: chatResp.model || model, output, usage: chatResp.usage ? { input_tokens: chatResp.usage.prompt_tokens || 0, output_tokens: chatResp.usage.completion_tokens || 0, total_tokens: chatResp.usage.total_tokens || 0 } : undefined };
|
|
184
202
|
}
|
|
185
203
|
|
|
186
204
|
export async function forwardToChat(chatReq, emitEvent, onDone, onError, options = {}) {
|
|
@@ -216,11 +234,15 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
216
234
|
releaseRequest?.();
|
|
217
235
|
}
|
|
218
236
|
} catch (e) {
|
|
237
|
+
markResponseErrorOrigin("transport");
|
|
238
|
+
markStreamFailure();
|
|
219
239
|
const statusCode = isAbortLikeError(e) ? abortErrorStatusCode(abort?.reason) : 502;
|
|
220
240
|
await onError(statusCode, e.message);
|
|
221
241
|
return false;
|
|
222
242
|
}
|
|
223
243
|
if (!resp.ok) {
|
|
244
|
+
markResponseErrorOrigin("upstream_http");
|
|
245
|
+
markStreamFailure();
|
|
224
246
|
await onError(resp.status, await readBoundedResponseText(resp, {
|
|
225
247
|
maxBytes: MAX_UPSTREAM_ERROR_BODY_BYTES,
|
|
226
248
|
label: "Copilot Chat error body",
|
|
@@ -230,6 +252,8 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
230
252
|
try {
|
|
231
253
|
await requireUpstreamEventStream(resp);
|
|
232
254
|
} catch (error) {
|
|
255
|
+
markResponseErrorOrigin("transport");
|
|
256
|
+
markStreamFailure();
|
|
233
257
|
await onError(error.statusCode, error.message, error);
|
|
234
258
|
return false;
|
|
235
259
|
}
|
|
@@ -237,6 +261,7 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
237
261
|
abort?.setTimeout(streamIdleTimeoutMs, "stream_idle_timeout");
|
|
238
262
|
const respId = `resp_${uid()}`;
|
|
239
263
|
let actualModel = requestModel;
|
|
264
|
+
let finishReason = null;
|
|
240
265
|
let fullText = "";
|
|
241
266
|
let messageItem = null;
|
|
242
267
|
let nextOutputIndex = 0;
|
|
@@ -298,7 +323,8 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
298
323
|
return { tool, created: true };
|
|
299
324
|
};
|
|
300
325
|
|
|
301
|
-
const
|
|
326
|
+
const emitTerminal = async () => {
|
|
327
|
+
const terminal = chatTerminal(finishReason);
|
|
302
328
|
if (!messageItem && toolCalls.size === 0) await ensureMessageItem();
|
|
303
329
|
const output = [];
|
|
304
330
|
if (messageItem) {
|
|
@@ -306,7 +332,7 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
306
332
|
type: "message",
|
|
307
333
|
id: messageItem.id,
|
|
308
334
|
role: "assistant",
|
|
309
|
-
status:
|
|
335
|
+
status: terminal.status,
|
|
310
336
|
content: [{ type: "output_text", text: fullText }],
|
|
311
337
|
};
|
|
312
338
|
await emitEvent("response.output_text.done", { output_index: messageItem.outputIndex, content_index: 0, text: fullText });
|
|
@@ -321,7 +347,7 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
321
347
|
call_id: tool.callId,
|
|
322
348
|
name: tool.name,
|
|
323
349
|
arguments: tool.arguments,
|
|
324
|
-
status:
|
|
350
|
+
status: terminal.status,
|
|
325
351
|
};
|
|
326
352
|
await emitEvent("response.function_call_arguments.done", {
|
|
327
353
|
output_index: tool.outputIndex,
|
|
@@ -331,16 +357,21 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
331
357
|
await emitEvent("response.output_item.done", { output_index: tool.outputIndex, item });
|
|
332
358
|
output[tool.outputIndex] = item;
|
|
333
359
|
}
|
|
334
|
-
|
|
360
|
+
if (terminal.status === "incomplete") markStreamFailure();
|
|
361
|
+
await emitEvent(`response.${terminal.status}`, {
|
|
335
362
|
response: {
|
|
336
363
|
id: respId,
|
|
337
364
|
object: "response",
|
|
338
|
-
|
|
365
|
+
...terminal,
|
|
339
366
|
model: actualModel,
|
|
340
367
|
output: output.filter(Boolean),
|
|
341
368
|
usage,
|
|
342
369
|
},
|
|
343
370
|
});
|
|
371
|
+
markResponseTerminal(terminal.status, {
|
|
372
|
+
model: actualModel, origin: "chat_completion", reason: terminal.incomplete_details?.reason,
|
|
373
|
+
});
|
|
374
|
+
return terminal.status === "completed";
|
|
344
375
|
};
|
|
345
376
|
|
|
346
377
|
try {
|
|
@@ -350,7 +381,7 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
350
381
|
})) {
|
|
351
382
|
if (!line.startsWith("data: ")) continue;
|
|
352
383
|
const data = line.slice(6).trim();
|
|
353
|
-
if (data === "[DONE]") { await
|
|
384
|
+
if (data === "[DONE]") { const completed = await emitTerminal(); onDone(); return completed; }
|
|
354
385
|
let parsed;
|
|
355
386
|
try {
|
|
356
387
|
parsed = JSON.parse(data);
|
|
@@ -373,7 +404,9 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
373
404
|
if (cached) usage.input_tokens_details = { cached_tokens: cached };
|
|
374
405
|
markOutputTokens(usage.output_tokens);
|
|
375
406
|
}
|
|
376
|
-
const
|
|
407
|
+
const choice = parsed.choices?.[0];
|
|
408
|
+
if (choice?.finish_reason) finishReason = choice.finish_reason;
|
|
409
|
+
const delta = choice?.delta;
|
|
377
410
|
if (!delta) continue;
|
|
378
411
|
if (isChatOutputDelta(delta)) markPerformanceOutput();
|
|
379
412
|
if (delta.content) {
|
|
@@ -401,14 +434,19 @@ export async function forwardToChat(chatReq, emitEvent, onDone, onError, options
|
|
|
401
434
|
}
|
|
402
435
|
} catch (e) {
|
|
403
436
|
if (e === downstreamClosed) {
|
|
437
|
+
markResponseTerminal("cancelled", { model: actualModel, origin: "client_disconnect" });
|
|
404
438
|
await resp.body?.cancel?.().catch?.(() => {});
|
|
405
439
|
return false;
|
|
406
440
|
}
|
|
441
|
+
markResponseErrorOrigin("transport");
|
|
442
|
+
markStreamFailure();
|
|
407
443
|
const statusCode = isAbortLikeError(e) ? abortErrorStatusCode(abort?.reason) : (e?.statusCode || 502);
|
|
408
444
|
await onError(statusCode, e?.message || "upstream stream error", e);
|
|
409
445
|
return false;
|
|
410
446
|
}
|
|
411
447
|
const error = incompleteUpstreamStream("[DONE]");
|
|
448
|
+
markResponseErrorOrigin("transport");
|
|
449
|
+
markStreamFailure();
|
|
412
450
|
await onError(error.statusCode, error.message, error);
|
|
413
451
|
return false;
|
|
414
452
|
}
|
|
@@ -37,7 +37,13 @@ import { status } from "./status.mjs";
|
|
|
37
37
|
import { endStreamWithError } from "./stream-errors.mjs";
|
|
38
38
|
import { safeUpstreamResponseHeaders } from "./upstream-headers.mjs";
|
|
39
39
|
import { recordResponsesUsage } from "./usage.mjs";
|
|
40
|
-
import {
|
|
40
|
+
import {
|
|
41
|
+
markResponseErrorOrigin,
|
|
42
|
+
markResponseModel,
|
|
43
|
+
markResponseTerminal,
|
|
44
|
+
measureRequestStage,
|
|
45
|
+
measureRequestStageAsync,
|
|
46
|
+
} from "./stream-performance.mjs";
|
|
41
47
|
|
|
42
48
|
const RESPONSES_ONLY_FALLBACK = new Set([
|
|
43
49
|
"gpt-5.6-luna",
|
|
@@ -267,6 +273,7 @@ export function createResponsesHandler(options) {
|
|
|
267
273
|
stripUnsupportedGpt6ServiceTier(prepared.body, requestedModel, upstreamModel, priorityTierModel);
|
|
268
274
|
if (requestedModel === CODEX_AUTO_REVIEW_MODEL) delete prepared.body.service_tier;
|
|
269
275
|
if (upstreamModel !== requestedModel) prepared.body.model = upstreamModel;
|
|
276
|
+
markResponseModel(upstreamModel);
|
|
270
277
|
const upstreamLog = upstreamModel === requestedModel ? "" : ` upstream_model=${upstreamModel}`;
|
|
271
278
|
console.log(status("info", `responses model=${requestedModel}${upstreamLog} stream=${streaming}`));
|
|
272
279
|
const usesCustomTools = responsesBodyUsesCustomTools(prepared.body);
|
|
@@ -322,19 +329,21 @@ export function createResponsesHandler(options) {
|
|
|
322
329
|
});
|
|
323
330
|
const written = await writeOrDrain(res, `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`, { signal: abort.signal });
|
|
324
331
|
if (!written) return false;
|
|
325
|
-
if (event === "response.completed") {
|
|
326
|
-
rememberResponseHistory(prepared, data.response);
|
|
332
|
+
if (event === "response.completed" || event === "response.incomplete") {
|
|
333
|
+
if (event === "response.completed") rememberResponseHistory(prepared, data.response);
|
|
327
334
|
recordResponsesUsage({ surface: prepared.surface, mode: "stream", model, response: data.response, event: data });
|
|
328
335
|
}
|
|
329
336
|
return true;
|
|
330
337
|
}, () => { if (!res.writableEnded) res.end(); }, async (statusCode, errMsg, error, upstreamResponse) => {
|
|
331
338
|
if (!res.headersSent) {
|
|
332
339
|
if (upstreamResponse) {
|
|
340
|
+
markResponseErrorOrigin("upstream_http");
|
|
333
341
|
markAdaptiveHttpTimeout(upstreamResponse.status);
|
|
334
342
|
sendUpstreamError(res, upstreamResponse, errMsg);
|
|
335
343
|
return;
|
|
336
344
|
}
|
|
337
345
|
const responseError = applyAdaptiveTimeout(error || Object.assign(new Error(errMsg), { statusCode }));
|
|
346
|
+
markResponseErrorOrigin("transport");
|
|
338
347
|
sendJsonError(res, responseError, statusCode || 500);
|
|
339
348
|
return;
|
|
340
349
|
}
|
|
@@ -374,13 +383,17 @@ export function createResponsesHandler(options) {
|
|
|
374
383
|
label: "Copilot Chat error body",
|
|
375
384
|
});
|
|
376
385
|
if (!upstream.ok) {
|
|
386
|
+
markResponseErrorOrigin("upstream_http");
|
|
377
387
|
markAdaptiveHttpTimeout(upstream.status);
|
|
378
388
|
sendUpstreamError(res, upstream, data);
|
|
379
389
|
return;
|
|
380
390
|
}
|
|
381
391
|
const response = chatToResponses(JSON.parse(data), model);
|
|
392
|
+
markResponseTerminal(response.status, {
|
|
393
|
+
model: response.model || model, origin: "chat_completion", reason: response.incomplete_details?.reason,
|
|
394
|
+
});
|
|
382
395
|
rememberResponseHistory(prepared, response);
|
|
383
|
-
imagePressure?.markSuccess?.(responseHistoryPressureRootId(prepared));
|
|
396
|
+
if (response.status === "completed") imagePressure?.markSuccess?.(responseHistoryPressureRootId(prepared));
|
|
384
397
|
recordResponsesUsage({ surface: prepared.surface, mode: "json", model, response, event: response });
|
|
385
398
|
res.writeHead(200, safeUpstreamResponseHeaders(upstream.headers, {
|
|
386
399
|
contentType: "application/json",
|
|
@@ -388,6 +401,8 @@ export function createResponsesHandler(options) {
|
|
|
388
401
|
res.end(JSON.stringify(response));
|
|
389
402
|
} catch (error) {
|
|
390
403
|
const responseError = applyAdaptiveTimeout(error);
|
|
404
|
+
markResponseErrorOrigin(responseError?.statusCode >= 400 && responseError.statusCode < 500
|
|
405
|
+
? "client_validation" : "transport");
|
|
391
406
|
logRequestFailure("Responses", responseError, abort);
|
|
392
407
|
sendJsonError(res, responseError, 502);
|
|
393
408
|
}
|
|
@@ -397,6 +412,8 @@ export function createResponsesHandler(options) {
|
|
|
397
412
|
}
|
|
398
413
|
} catch (error) {
|
|
399
414
|
const responseError = applyAdaptiveTimeout(error);
|
|
415
|
+
markResponseErrorOrigin(responseError?.statusCode >= 400 && responseError.statusCode < 500
|
|
416
|
+
? "client_validation" : "transport");
|
|
400
417
|
logRequestFailure("Responses", responseError, abort);
|
|
401
418
|
sendJsonError(res, responseError, 502);
|
|
402
419
|
} finally {
|
package/src/responses-proxy.mjs
CHANGED
|
@@ -40,6 +40,8 @@ import {
|
|
|
40
40
|
isResponsesOutputEvent,
|
|
41
41
|
markFirstOutput,
|
|
42
42
|
markOutputTokens,
|
|
43
|
+
markResponseErrorOrigin,
|
|
44
|
+
markResponseTerminal,
|
|
43
45
|
markStreamFailure,
|
|
44
46
|
} from "./stream-performance.mjs";
|
|
45
47
|
import { safeUpstreamResponseHeaders } from "./upstream-headers.mjs";
|
|
@@ -146,6 +148,8 @@ function inspectResponseSseEvent(state, eventName, data) {
|
|
|
146
148
|
if (eventType === "response.completed") {
|
|
147
149
|
const response = event.response;
|
|
148
150
|
state.completed = { response, event };
|
|
151
|
+
} else if (eventType === "response.incomplete") {
|
|
152
|
+
state.incomplete = { response: event.response, event };
|
|
149
153
|
}
|
|
150
154
|
return changed ? event : null;
|
|
151
155
|
}
|
|
@@ -370,6 +374,7 @@ async function proxyStreamingResponses(opened, res, upstream, options) {
|
|
|
370
374
|
usedRetryPolicies = new Set(),
|
|
371
375
|
} = opened;
|
|
372
376
|
if (errorText !== undefined) {
|
|
377
|
+
markResponseErrorOrigin("upstream_http");
|
|
373
378
|
sendUpstreamError(res, resp, errorText);
|
|
374
379
|
return { successful: false, compacted: false, upstreamStatus: resp.status };
|
|
375
380
|
}
|
|
@@ -386,6 +391,7 @@ async function proxyStreamingResponses(opened, res, upstream, options) {
|
|
|
386
391
|
const reader = resp.body.getReader();
|
|
387
392
|
const streamState = {
|
|
388
393
|
completed: null,
|
|
394
|
+
incomplete: null,
|
|
389
395
|
failure: null,
|
|
390
396
|
sawTerminal: false,
|
|
391
397
|
sawOutput: false,
|
|
@@ -467,12 +473,28 @@ async function proxyStreamingResponses(opened, res, upstream, options) {
|
|
|
467
473
|
}
|
|
468
474
|
continue;
|
|
469
475
|
}
|
|
470
|
-
if (streamState.failure) markStreamFailure();
|
|
471
476
|
if (holdPrelude && !await flushPrelude()) return { successful: false, compacted: false };
|
|
477
|
+
if (streamState.completed) {
|
|
478
|
+
markResponseTerminal("completed", {
|
|
479
|
+
model: streamState.completed.response.model || reqContext.body?.model,
|
|
480
|
+
origin: "upstream_response",
|
|
481
|
+
});
|
|
482
|
+
} else if (streamState.incomplete) {
|
|
483
|
+
markResponseTerminal("incomplete", {
|
|
484
|
+
model: streamState.incomplete.response.model || reqContext.body?.model,
|
|
485
|
+
origin: "upstream_response",
|
|
486
|
+
reason: streamState.incomplete.response.incomplete_details?.reason,
|
|
487
|
+
});
|
|
488
|
+
} else if (streamState.failure) {
|
|
489
|
+
markStreamFailure();
|
|
490
|
+
markResponseTerminal("failed", { model: reqContext.body?.model, origin: "upstream_event" });
|
|
491
|
+
}
|
|
472
492
|
storeCompletedResponse(reqContext, streamState.completed);
|
|
473
493
|
if (!res.writableEnded) res.end();
|
|
474
494
|
return { successful: Boolean(streamState.completed), compacted: false };
|
|
475
495
|
} catch (error) {
|
|
496
|
+
markResponseErrorOrigin("transport");
|
|
497
|
+
markStreamFailure();
|
|
476
498
|
logRequestFailure("Responses", error, options.abort);
|
|
477
499
|
writeHeaders();
|
|
478
500
|
await endStreamWithError(res, error, options.abort);
|
|
@@ -502,6 +524,7 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
|
|
|
502
524
|
const { resp, errorText, usedRetryPolicies = new Set() } = opened;
|
|
503
525
|
reqContext = opened.reqContext;
|
|
504
526
|
if (errorText !== undefined) {
|
|
527
|
+
markResponseErrorOrigin("upstream_http");
|
|
505
528
|
sendUpstreamError(res, resp, errorText);
|
|
506
529
|
return { successful: false, compacted: false, upstreamStatus: resp.status };
|
|
507
530
|
}
|
|
@@ -528,6 +551,7 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
|
|
|
528
551
|
response,
|
|
529
552
|
event: response,
|
|
530
553
|
});
|
|
554
|
+
markResponseTerminal("completed", { model: response.model || reqContext.body?.model, origin: "upstream_response" });
|
|
531
555
|
res.writeHead(resp.status, safeUpstreamResponseHeaders(resp.headers, {
|
|
532
556
|
contentType: "application/json",
|
|
533
557
|
}));
|
|
@@ -572,6 +596,13 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
|
|
|
572
596
|
response,
|
|
573
597
|
event: response,
|
|
574
598
|
});
|
|
599
|
+
if (["completed", "incomplete", "failed"].includes(response.status)) {
|
|
600
|
+
markResponseTerminal(response.status, {
|
|
601
|
+
model: response.model || reqContext.body?.model,
|
|
602
|
+
origin: "upstream_response",
|
|
603
|
+
reason: response.incomplete_details?.reason,
|
|
604
|
+
});
|
|
605
|
+
}
|
|
575
606
|
res.writeHead(resp.status, safeUpstreamResponseHeaders(resp.headers, {
|
|
576
607
|
contentType: "application/json",
|
|
577
608
|
}));
|
|
@@ -581,6 +612,7 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
|
|
|
581
612
|
res.writeHead(resp.status, safeUpstreamResponseHeaders(resp.headers, {
|
|
582
613
|
contentType: "application/json",
|
|
583
614
|
}));
|
|
615
|
+
markResponseErrorOrigin("upstream_http");
|
|
584
616
|
res.end(data);
|
|
585
617
|
return { successful: false, compacted: false };
|
|
586
618
|
}
|
|
@@ -3,6 +3,13 @@ import { currentRequestContext } from "./request-context.mjs";
|
|
|
3
3
|
|
|
4
4
|
const PERFORMANCE_ROUTES = Object.freeze(["responses", "responses_compact"]);
|
|
5
5
|
const PREPARATION_STAGES = Object.freeze(["admission", "body", "history", "images", "serialization"]);
|
|
6
|
+
const TERMINAL_OUTCOMES = Object.freeze(["completed", "incomplete", "failed", "cancelled", "unknown"]);
|
|
7
|
+
const INCOMPLETE_REASONS = new Set(["max_output_tokens", "max_messages", "content_filter", "steered"]);
|
|
8
|
+
const TERMINAL_ORIGINS = new Set([
|
|
9
|
+
"upstream_response", "upstream_event", "chat_completion", "client_validation",
|
|
10
|
+
"upstream_http", "transport", "client_disconnect", "unclassified",
|
|
11
|
+
]);
|
|
12
|
+
const MAX_MODEL_OUTCOME_LABELS = 32;
|
|
6
13
|
const PREPARATION_EDGES_MS = Object.freeze([1, 2, 5, 10, 20, 50, 100, 250, 500, 1_000, 5_000, 30_000, 120_000]);
|
|
7
14
|
const TTFT_EDGES_MS = Object.freeze([
|
|
8
15
|
100, 200, 300, 500, 700, 1_000, 1_400, 2_000, 2_800, 4_000,
|
|
@@ -78,6 +85,16 @@ function histogramSnapshot(histogram, unit) {
|
|
|
78
85
|
};
|
|
79
86
|
}
|
|
80
87
|
|
|
88
|
+
function emptyOutcomes() {
|
|
89
|
+
return Object.fromEntries(TERMINAL_OUTCOMES.map((outcome) => [outcome, 0]));
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
function modelLabel(value) {
|
|
93
|
+
const model = String(value || "").trim();
|
|
94
|
+
return model === "codex-auto-review" || /^gpt-[A-Za-z0-9._-]{1,72}$/.test(model)
|
|
95
|
+
? model : "other";
|
|
96
|
+
}
|
|
97
|
+
|
|
81
98
|
function createRoutePerformance() {
|
|
82
99
|
return {
|
|
83
100
|
ttft: createHistogram(TTFT_EDGES_MS),
|
|
@@ -88,6 +105,10 @@ function createRoutePerformance() {
|
|
|
88
105
|
errors_with_output: 0,
|
|
89
106
|
zero_output_errors: 0,
|
|
90
107
|
neutral: 0,
|
|
108
|
+
terminalTotals: emptyOutcomes(),
|
|
109
|
+
incompleteReasons: Object.create(null),
|
|
110
|
+
terminalByModel: new Map(),
|
|
111
|
+
terminalByOrigin: Object.create(null),
|
|
91
112
|
};
|
|
92
113
|
}
|
|
93
114
|
|
|
@@ -97,6 +118,12 @@ function routeSnapshot(route) {
|
|
|
97
118
|
errors_with_output: route.errors_with_output,
|
|
98
119
|
zero_output_errors: route.zero_output_errors,
|
|
99
120
|
neutral: route.neutral,
|
|
121
|
+
terminal_outcomes: {
|
|
122
|
+
totals: { ...route.terminalTotals },
|
|
123
|
+
incomplete_reasons: { ...route.incompleteReasons },
|
|
124
|
+
by_model: Object.fromEntries([...route.terminalByModel].map(([model, counts]) => [model, { ...counts }])),
|
|
125
|
+
by_origin: Object.fromEntries(Object.entries(route.terminalByOrigin).map(([origin, counts]) => [origin, { ...counts }])),
|
|
126
|
+
},
|
|
100
127
|
ttft_ms: histogramSnapshot(route.ttft, "ms"),
|
|
101
128
|
tpot_us: histogramSnapshot(route.tpot, "us"),
|
|
102
129
|
request_ttft_ms: histogramSnapshot(route.requestTtft, "ms"),
|
|
@@ -117,6 +144,11 @@ export function createStreamPerformanceMetrics({ now = () => performance.now() }
|
|
|
117
144
|
let outputTokens = null;
|
|
118
145
|
let failed = false;
|
|
119
146
|
let finished = false;
|
|
147
|
+
let model = "other";
|
|
148
|
+
let terminalOutcome = null;
|
|
149
|
+
let terminalOrigin = null;
|
|
150
|
+
let incompleteReason = null;
|
|
151
|
+
let errorOrigin = null;
|
|
120
152
|
|
|
121
153
|
return {
|
|
122
154
|
beginStage(stage) {
|
|
@@ -140,13 +172,37 @@ export function createStreamPerformanceMetrics({ now = () => performance.now() }
|
|
|
140
172
|
const tokens = Number(value);
|
|
141
173
|
if (Number.isFinite(tokens) && tokens >= 0) outputTokens = tokens;
|
|
142
174
|
},
|
|
175
|
+
setModel(value) {
|
|
176
|
+
model = modelLabel(value);
|
|
177
|
+
},
|
|
178
|
+
setErrorOrigin(value) {
|
|
179
|
+
if (TERMINAL_ORIGINS.has(value)) errorOrigin = value;
|
|
180
|
+
},
|
|
181
|
+
terminal(outcome, { model: terminalModel, origin, reason } = {}) {
|
|
182
|
+
if (finished || terminalOutcome || !TERMINAL_OUTCOMES.includes(outcome)) return;
|
|
183
|
+
terminalOutcome = outcome;
|
|
184
|
+
if (terminalModel !== undefined) model = modelLabel(terminalModel);
|
|
185
|
+
if (TERMINAL_ORIGINS.has(origin)) terminalOrigin = origin;
|
|
186
|
+
if (outcome === "incomplete") incompleteReason = INCOMPLETE_REASONS.has(reason) ? reason : "other";
|
|
187
|
+
if (outcome !== "completed") failed = true;
|
|
188
|
+
},
|
|
143
189
|
fail() {
|
|
144
190
|
failed = true;
|
|
145
191
|
},
|
|
146
|
-
finish({ failed: finishFailed = false } = {}) {
|
|
192
|
+
finish({ failed: finishFailed = false, aborted = false } = {}) {
|
|
147
193
|
if (finished) return;
|
|
148
194
|
finished = true;
|
|
149
195
|
failed ||= finishFailed;
|
|
196
|
+
const outcome = terminalOutcome || (aborted ? "cancelled" : "unknown");
|
|
197
|
+
const origin = terminalOrigin || errorOrigin || (aborted ? "client_disconnect" : "unclassified");
|
|
198
|
+
route.terminalTotals[outcome] += 1;
|
|
199
|
+
if (incompleteReason) route.incompleteReasons[incompleteReason] = (route.incompleteReasons[incompleteReason] || 0) + 1;
|
|
200
|
+
const label = route.terminalByModel.has(model) || route.terminalByModel.size < MAX_MODEL_OUTCOME_LABELS
|
|
201
|
+
? model : "other";
|
|
202
|
+
if (!route.terminalByModel.has(label)) route.terminalByModel.set(label, emptyOutcomes());
|
|
203
|
+
route.terminalByModel.get(label)[outcome] += 1;
|
|
204
|
+
route.terminalByOrigin[origin] ||= emptyOutcomes();
|
|
205
|
+
route.terminalByOrigin[origin][outcome] += 1;
|
|
150
206
|
const finishedAt = now();
|
|
151
207
|
if (firstOutputAt !== null) observe(route.requestTtft, Math.max(0, firstOutputAt - requestStartedAt));
|
|
152
208
|
if (upstreamStartedAt === null) {
|
|
@@ -212,3 +268,15 @@ export function markOutputTokens(value) {
|
|
|
212
268
|
export function markStreamFailure() {
|
|
213
269
|
tracker()?.fail();
|
|
214
270
|
}
|
|
271
|
+
|
|
272
|
+
export function markResponseModel(model) {
|
|
273
|
+
tracker()?.setModel?.(model);
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
export function markResponseErrorOrigin(origin) {
|
|
277
|
+
tracker()?.setErrorOrigin?.(origin);
|
|
278
|
+
}
|
|
279
|
+
|
|
280
|
+
export function markResponseTerminal(outcome, options) {
|
|
281
|
+
tracker()?.terminal?.(outcome, options);
|
|
282
|
+
}
|