codex-copilot-dx 0.7.2 → 0.7.4

package/README.md

@@ -172,6 +172,8 @@ curl -s http://127.0.0.1:2026/_ccdx/status | jq
 
 The status endpoint is restricted to the socket's real loopback address even when LAN binding is explicitly enabled. It reports fixed-size request counters, bounded TTFT/TPOT histograms, admission pressure, memory use, response-history size, image queue/cache and adaptive-history state, model-cache counts, and token expiry state. It never retains metric samples or includes prompts, completions, tool arguments, image content, account names, or token values.
 
+`stream_performance.by_route` also reports `request_ttft_ms`, measured from adapter entry to the first observed output delta, including local preparation. The existing `ttft_ms` continues to start at upstream dispatch. Neither timestamp means the client has drained or displayed the output. `preparation_ms` contains bounded histograms for admission (including history reservations), body reading/decompression/JSON parsing, history materialization/compact preparation, image optimization, and request serialization. Each sample measures one operation, including failed operations; retries can add samples. These are diagnostic intervals rather than a complete breakdown of request time: body reading includes internal decoded-body admission waits; authentication, routing, and other preparation are not individually timed. Non-streaming requests contribute preparation samples without inventing a first-token measurement.
+
 Every API response includes a safe `X-Request-Id` for correlation. To include the same ID in related terminal and file log lines, start with:
 
 ```bash
@@ -283,6 +285,8 @@ For `/v1/responses/compact`, CCDX sends one terminal `compaction_trigger`, valid
 
 Successful streaming responses are accepted only when the upstream body is SSE and reaches its protocol terminal event. Unexpected EOF becomes a protocol-native stream error, and repeated blank tool-argument deltas are stopped per tool call. Safe quota, retry, model, trace, and upstream request-ID metadata are forwarded; cookies, authorization, body-length, and encoding headers are not.
 
+CCDX 0.7.3 keeps each native Responses message on its first upstream ID when Copilot changes that ID during streaming. Message deltas, completion events, and cached history stay consistent, preventing duplicate live messages in Codex App. Tool identities, reasoning/encrypted content, message phases, and speed-tier routing are unchanged. Stable events retain their original SSE bytes; changed events are rewritten with bounded, backpressured streaming.
+
 Newer Codex App builds can advertise an `image_gen` namespace that already exists upstream. CCDX removes that exact conflicting client tool before forwarding and retries once only when Copilot explicitly reports an image namespace collision. Image input and screenshot optimization remain enabled.
 
 ## Development
@@ -293,8 +297,12 @@ npm run verify
 npm run bench:payload
 ```
 
+`bench:check` covers both SSE parsers, including native Responses identity rewriting, 1/4/8 MiB fragmented events, write batching, and slow-client backpressure. It also repeats four concurrent image preparations across six rounds using a desktop-sized synthetic screenshot and a public-domain photograph. Checks enforce content preservation, single-pass JSON parsing, bounded copying, cache reuse, settled work, and bounded retained heap/ArrayBuffers after warm-up and GC. Sampled RSS and absolute timings remain report-only because allocators, runtimes, and machines differ; the fixtures do not establish production latency or visual model accuracy.
+
 `npm test` runs unit and handler-level tests. `npm run test:smoke` starts a real local adapter with fully injected offline upstreams. `npm run bench:check` enforces linear SSE scanning, image/tool processing, and request-admission resource limits. `npm run pack:check` verifies npm tarball contents without publishing. `npm run bench:payload` is a report-only isolated-process benchmark for 5–60 MiB image payloads and does not contact Copilot. CI runs verification on the supported Node.js release lines.
 
+From a source checkout, `node scripts/codex-message-id-replay.mjs --codex /absolute/path/to/codex` checks message identity against an installed Codex runtime using only synthetic SSE, a local mock server, and automatically cleaned temporary Codex directories. It does not use the real client configuration or call Copilot.
+
 ## License
 
 MIT

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "codex-copilot-dx",
-  "version": "0.7.2",
+  "version": "0.7.4",
   "description": "Use Codex App with GitHub Copilot",
   "author": "Dale Xiao",
   "type": "module",
@@ -12,6 +12,8 @@
     "bin/",
     "src/",
     "scripts/benchmark.mjs",
+    "scripts/benchmark-probes.mjs",
+    "scripts/fixtures/",
     "README.md"
   ],
   "keywords": [

package/scripts/benchmark-probes.mjs

@@ -0,0 +1,204 @@
+import { createHash } from "node:crypto";
+import { EventEmitter } from "node:events";
+import fs from "node:fs";
+import { setImmediate as immediate } from "node:timers/promises";
+import sharp from "sharp";
+import { proxyCopilotResponses } from "../src/responses-proxy.mjs";
+import { clearResponseHistoryForTests } from "../src/response-history.mjs";
+import { imageOptimizationStats, prepareResponsesPayload, resetImageOptimizationCacheForTests } from "../src/image-optimization.mjs";
+import { createStreamPerformanceMetrics } from "../src/stream-performance.mjs";
+import { runWithRequestContext } from "../src/request-context.mjs";
+
+const digest = (value) => createHash("sha256").update(value).digest("hex");
+const round = (value) => +value.toFixed(3);
+
+function nativeWire(count, deltaBytes, drifting) {
+  const delta = "x".repeat(deltaBytes);
+  const id = drifting ? "message_changed" : "message_initial";
+  const item = { type: "message", id: "message_initial", role: "assistant", content: [] };
+  const events = [
+    { type: "response.output_item.added", output_index: 0, item },
+    ...Array.from({ length: count }, () => ({ type: "response.output_text.delta", output_index: 0, item_id: id, delta })),
+    { type: "response.completed", response: {
+      id: "resp_benchmark", status: "completed",
+      output: [{ ...item, id, content: [{ type: "output_text", text: delta.repeat(count) }] }],
+    } },
+  ];
+  return Buffer.from(events.map((event) => `event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`).join(""));
+}
+
+export async function nativeResponsesProbe() {
+  process.env.CCDX_DISABLE_USAGE = "1";
+  const cases = [
+    { name: "stable-small", count: 5000, deltaBytes: 256 },
+    { name: "drifting-small", count: 5000, deltaBytes: 256, drifting: true },
+    ...[1, 4, 8].flatMap((mib) => [false, true].map((drifting) => ({
+      name: `${drifting ? "drifting" : "stable"}-${mib}MiB`, count: 1, deltaBytes: mib * 1048576 - 1024, drifting,
+    }))),
+    { name: "slow-client", count: 5000, deltaBytes: 256, drifting: true, slow: true },
+    { name: "slow-large-event", count: 1, deltaBytes: 8 * 1048576 - 1024, slow: true },
+  ];
+  const results = [];
+  for (const scenario of cases) {
+    clearResponseHistoryForTests();
+    const wire = nativeWire(scenario.count, scenario.deltaBytes, scenario.drifting);
+    const expectedHash = digest(nativeWire(scenario.count, scenario.deltaBytes, false));
+    const outputHash = createHash("sha256");
+    let offset = 0;
+    let blocked = false;
+    let readsWhileBlocked = 0;
+    let writes = 0;
+    let drains = 0;
+    let cancelled = false;
+    const response = new Response(new ReadableStream({
+      pull(controller) {
+        if (blocked) readsWhileBlocked += 1;
+        if (offset === wire.length) return controller.close();
+        const end = Math.min(wire.length, offset + 16 * 1024);
+        controller.enqueue(wire.subarray(offset, end));
+        offset = end;
+      },
+      cancel() { cancelled = true; },
+    }, { highWaterMark: 0 }), { headers: { "Content-Type": "text/event-stream" } });
+    const res = new EventEmitter();
+    Object.assign(res, { destroyed: false, writableEnded: false, headersSent: false });
+    res.writeHead = () => { res.headersSent = true; };
+    res.end = () => { res.writableEnded = true; };
+    res.write = (value) => {
+      writes += 1;
+      if (scenario.slow) {
+        blocked = true;
+        setImmediate(() => {
+          // Retain the actual Buffer until drain to detect buffer reuse/corruption.
+          outputHash.update(value);
+          blocked = false;
+          drains += 1;
+          res.emit("drain");
+        });
+        return false;
+      }
+      outputHash.update(value);
+      return true;
+    };
+    const original = { parse: JSON.parse, concat: Buffer.concat, copy: Buffer.prototype.copy, set: Buffer.prototype.set };
+    let parseCalls = 0;
+    let parsedBytes = 0;
+    let copiedBytes = 0;
+    JSON.parse = function (value, ...args) {
+      parseCalls += 1;
+      parsedBytes += Buffer.byteLength(value);
+      return original.parse(value, ...args);
+    };
+    Buffer.concat = function (values, size) {
+      copiedBytes += size ?? values.reduce((total, value) => total + value.length, 0);
+      return original.concat(values, size);
+    };
+    Buffer.prototype.copy = function (...args) {
+      const copied = original.copy.apply(this, args);
+      copiedBytes += copied;
+      return copied;
+    };
+    Buffer.prototype.set = function (value, offset) {
+      copiedBytes += value.length;
+      return original.set.call(this, value, offset);
+    };
+    const started = performance.now();
+    let result;
+    try {
+      result = await proxyCopilotResponses({ body: { model: "gpt-benchmark", input: [], stream: true } }, {}, res, async () => response);
+    } finally {
+      JSON.parse = original.parse;
+      Buffer.concat = original.concat;
+      Buffer.prototype.copy = original.copy;
+      Buffer.prototype.set = original.set;
+      clearResponseHistoryForTests();
+    }
+    results.push({
+      name: scenario.name, input_bytes: wire.length, expected_events: scenario.count + 2,
+      parse_calls: parseCalls, parse_ratio: round(parsedBytes / wire.length), copy_ratio: round(copiedBytes / wire.length),
+      writes, drains, reads_while_blocked: readsWhileBlocked, cancelled,
+      content_matches: outputHash.digest("hex") === expectedHash,
+      successful: result?.successful === true, elapsed_ms: round(performance.now() - started),
+    });
+  }
+  return results;
+}
+
+export async function imageLifecycleProbe() {
+  // Public-domain photograph plus a deterministic desktop-sized UI fixture.
+  const photo = await sharp(fs.readFileSync(new URL("./fixtures/astronaut.jpg", import.meta.url))).png().toBuffer();
+  const rows = Array.from({ length: 65 }, (_, index) => (
+    `<text x="48" y="${55 + index * 23}" font-family="sans-serif" font-size="17" fill="#b8d7ef">${index}: const request = await prepareResponsesPayload(input); // screenshot benchmark</text>`
+  )).join("");
+  const screenshot = await sharp(Buffer.from(`<svg width="2560" height="1600"><rect width="2560" height="1600" fill="#182332"/>${rows}</svg>`)).png().toBuffer();
+  const sourceUrls = [photo, screenshot].map((bytes) => `data:image/png;base64,${bytes.toString("base64")}`);
+  const concurrency = 4;
+  const metrics = createStreamPerformanceMetrics();
+  resetImageOptimizationCacheForTests();
+  const samples = [];
+  let bodyBytes = 0;
+  let expectedOutputHash;
+  let allOutputsMatch = true;
+  let workSettled = true;
+  let previousHits = 0;
+  const originalLog = console.log;
+  const originalWarn = console.warn;
+  console.log = () => {};
+  console.warn = () => {};
+  try {
+    for (let iteration = 0; iteration < 6; iteration += 1) {
+      const started = performance.now();
+      let peakRss = process.memoryUsage().rss;
+      const samplePeak = () => { peakRss = Math.max(peakRss, process.memoryUsage().rss); };
+      const timer = setInterval(samplePeak, 5);
+      let results;
+      try {
+        results = await Promise.all(Array.from({ length: concurrency }, async () => {
+          const body = { model: "gpt-5.6-sol", input: [{ type: "message", role: "user", content:
+            Array.from({ length: 6 }, (_, index) => ({ type: "input_image", image_url: sourceUrls[index % sourceUrls.length], detail: "high" })),
+          }] };
+          bodyBytes = Buffer.byteLength(JSON.stringify(body));
+          const tracker = metrics.begin("responses");
+          return runWithRequestContext({ streamPerformance: tracker }, async () => {
+            try {
+              const payload = await prepareResponsesPayload(body);
+              samplePeak();
+              return { bytes: payload.bodyBytes, hash: digest(payload.bodyText) };
+            } finally {
+              tracker.finish();
+            }
+          });
+        }));
+      } finally {
+        clearInterval(timer);
+      }
+      expectedOutputHash ??= results[0].hash;
+      allOutputsMatch &&= results.every((result) => result.hash === expectedOutputHash);
+      await immediate();
+      globalThis.gc?.();
+      const memory = process.memoryUsage();
+      const cache = imageOptimizationStats();
+      workSettled &&= cache.active === 0 && cache.queued === 0 && cache.cache_inflight === 0;
+      samples.push({
+        iteration, elapsed_ms: round(performance.now() - started), peak_rss_bytes: peakRss,
+        retained_rss_bytes: memory.rss, retained_heap_bytes: memory.heapUsed, retained_array_buffer_bytes: memory.arrayBuffers,
+        cache_bytes: cache.cache_bytes, cache_hits_added: cache.cache_hits - previousHits, output_bytes: results[0].bytes,
+      });
+      previousHits = cache.cache_hits;
+    }
+  } finally {
+    console.log = originalLog;
+    console.warn = originalWarn;
+    resetImageOptimizationCacheForTests();
+  }
+  const warm = samples[1];
+  const late = samples.slice(2);
+  return {
+    concurrency, input_bytes: bodyBytes, photo_dimensions: [512, 512], screenshot_dimensions: [2560, 1600],
+    all_outputs_match: allOutputsMatch, work_settled: workSettled,
+    retained_heap_growth_bytes: Math.max(0, ...late.map((sample) => sample.retained_heap_bytes - warm.retained_heap_bytes)),
+    retained_array_buffer_growth_bytes: Math.max(0, ...late.map((sample) => sample.retained_array_buffer_bytes - warm.retained_array_buffer_bytes)),
+    retention_budget_bytes: Math.max(8 * 1048576, bodyBytes * concurrency * 2),
+    samples,
+  };
+}

package/scripts/benchmark.mjs

@@ -395,6 +395,14 @@ if (checkMode) {
     largePayloadProbe(30, 2),
   ];
   report.visual_history_pressure_36x4 = visualHistoryPressureProbe();
+  report.native_responses = runProbe(`
+    import { nativeResponsesProbe } from "./scripts/benchmark-probes.mjs";
+    process.stdout.write(JSON.stringify(await nativeResponsesProbe()));
+  `);
+  report.image_lifecycle = runProbe(`
+    import { imageLifecycleProbe } from "./scripts/benchmark-probes.mjs";
+    process.stdout.write(JSON.stringify(await imageLifecycleProbe()));
+  `);
 } else if (process.argv.includes("--large-payload")) {
   report.large_payload_peak = [
     largePayloadProbe(5, 1),
@@ -437,6 +445,29 @@ if (checkMode) {
     || !visualHistory.output_body_mib.every((size, index) => size < visualHistory.input_body_mib[index])) {
     failures.push("36-image x4 visual history pressure did not preserve the expected 16-image upstream window");
   }
+  for (const sample of report.native_responses) {
+    if (!sample.successful || !sample.content_matches || !sample.cancelled) {
+      failures.push(`native Responses changed content or failed to close its reader: ${sample.name}`);
+    }
+    if (sample.parse_calls !== sample.expected_events || sample.parse_ratio > 1.01 || sample.copy_ratio > 8) {
+      failures.push(`native Responses repeated parsing or copying: ${sample.name}`);
+    }
+    if (sample.writes > Math.ceil(sample.input_bytes / (16 * 1024)) + 2) {
+      failures.push(`native Responses lost per-read write batching: ${sample.name}`);
+    }
+    if (sample.name.startsWith("slow") && (sample.drains !== sample.writes || !sample.drains || sample.reads_while_blocked)) {
+      failures.push(`native Responses read ahead of downstream drain: ${sample.name}`);
+    }
+  }
+  const lifecycle = report.image_lifecycle;
+  if (!lifecycle.all_outputs_match || !lifecycle.work_settled
+    || lifecycle.samples.length !== 6 || lifecycle.samples.slice(1).some((sample) => sample.cache_hits_added === 0)) {
+    failures.push("repeated image requests did not reuse stable results or settle image work");
+  }
+  if (lifecycle.retained_heap_growth_bytes > lifecycle.retention_budget_bytes
+    || lifecycle.retained_array_buffer_growth_bytes > lifecycle.retention_budget_bytes) {
+    failures.push("repeated image requests retained more than two batches after warm-up and GC");
+  }
   if (failures.length) {
     for (const failure of failures) console.error(`[FAIL] ${failure}`);
     process.exitCode = 1;

package/scripts/fixtures/README.md

@@ -0,0 +1,13 @@
+# Benchmark photograph
+
+`astronaut.jpg` is the 512 x 512 photograph of NASA astronaut Eileen Collins
+distributed by scikit-image, converted from PNG to JPEG at quality 90 without
+resizing. The original is credited to NASA and documented as public domain.
+
+- [Original image](https://github.com/scikit-image/scikit-image/blob/v0.19.3/skimage/data/astronaut.png)
+- [Source and public-domain statement](https://github.com/scikit-image/scikit-image/blob/v0.19.3/skimage/data/_fetchers.py#L405-L424)
+- Original PNG SHA-256: `88431cd9653ccd539741b555fb0a46b61558b301d4110412b5bc28b5e3ea6cb5`
+
+The benchmark decodes this fixture to PNG before request preparation to exercise
+real image encoding. Its 2560 x 1600 screenshot fixture is generated locally
+from SVG. Neither fixture contains user data or requires a network request.

package/src/image-optimization.mjs

@@ -6,6 +6,7 @@ import {
 } from "./responses-content.mjs";
 import { createBoundedResultCache } from "./bounded-result-cache.mjs";
 import { status } from "./status.mjs";
+import { measureRequestStage, measureRequestStageAsync } from "./stream-performance.mjs";
 
 const DEFAULT_IMG_CONCURRENCY = 2;
 const IMG_MAX_CONCURRENCY = 12;
@@ -430,6 +431,15 @@ export async function optimizeImagesInBody(reqBody, {
   return reqBody;
 }
 
+function serializePayload(reqBody, assertActive) {
+  return measureRequestStage("serialization", () => {
+    assertActive?.();
+    const bodyText = JSON.stringify(reqBody);
+    assertActive?.();
+    return { bodyText, bodyBytes: Buffer.byteLength(bodyText) };
+  });
+}
+
 export async function prepareResponsesPayload(reqBody, {
   assertActive,
   currentInputStart = 0,
@@ -444,18 +454,15 @@ export async function prepareResponsesPayload(reqBody, {
   const collection = shouldOptimizeImages ? collectImageReferences(reqBody, assertActive) : null;
   const originalImages = collection ? uniqueOriginalImages(collection.references, assertActive) : [];
   if (collection) {
-    await optimizeImageReferences(collection, {
+    await measureRequestStageAsync("images", () => optimizeImageReferences(collection, {
       assertActive,
       concurrency: IMG_CONCURRENCY,
       model: reqBody.model,
       optimizeImage: (dataUrl, options) => optimizeImage(dataUrl, { ...options, quality: IMG_QUALITY }),
       signal,
-    });
+    }));
   }
-  assertActive?.();
-  let bodyText = JSON.stringify(reqBody);
-  assertActive?.();
-  let bodyBytes = Buffer.byteLength(bodyText);
+  let { bodyText, bodyBytes } = serializePayload(reqBody, assertActive);
   const summary = summarizeReqBody(reqBody);
   const targetBytes = positiveInt(maxBytes, MAX_UPSTREAM_BODY_BYTES);
   let adapted = false;
@@ -467,37 +474,34 @@ export async function prepareResponsesPayload(reqBody, {
       assertActive?.();
       const beforeBytes = bodyBytes;
       let processed = 0;
+      let bodyTextDirty = false;
       stage = `q${positiveInt(profile.quality, IMG_QUALITY, 100)}`;
       const orderedImages = orderImagesByCurrentWireBytes(originalImages);
       for (let index = 0; index < orderedImages.length && bodyBytes > targetBytes; index += batchSize) {
         assertActive?.();
         const batch = orderedImages.slice(index, index + batchSize);
-        const candidates = await Promise.all(batch.map((image) => optimizeImage(image.dataUrl, {
+        const candidates = await measureRequestStageAsync("images", () => Promise.all(batch.map((image) => optimizeImage(image.dataUrl, {
           ...profile,
           force: true,
           model: reqBody.model,
           signal,
-        })));
+        }))));
         assertActive?.();
         for (let offset = 0; offset < batch.length && bodyBytes > targetBytes; offset += 1) {
           processed += 1;
           const applied = applyImageCandidate(collection, batch[offset], candidates[offset], bodyBytes);
           bodyBytes = applied.bodyBytes;
           adapted ||= applied.changed;
+          bodyTextDirty ||= applied.changed;
           if (bodyBytes <= targetBytes) {
             collection.commitDirty();
-            assertActive?.();
-            bodyText = JSON.stringify(reqBody);
-            assertActive?.();
-            bodyBytes = Buffer.byteLength(bodyText);
+            ({ bodyText, bodyBytes } = serializePayload(reqBody, assertActive));
+            bodyTextDirty = false;
           }
         }
       }
       collection.commitDirty();
-      assertActive?.();
-      bodyText = JSON.stringify(reqBody);
-      assertActive?.();
-      bodyBytes = Buffer.byteLength(bodyText);
+      if (bodyTextDirty) ({ bodyText, bodyBytes } = serializePayload(reqBody, assertActive));
       console.warn(status("warn", `responses payload ${beforeBytes}b exceeds ${targetBytes}b; image profile max_dim=${profile.maxDim} quality=${profile.quality} processed=${processed}/${orderedImages.length} -> ${bodyBytes}b`));
       if (bodyBytes <= targetBytes) break;
     }

package/src/responses-byte-budget.mjs

@@ -3,14 +3,17 @@ import {
   readResponsesImagePart,
   readResponsesToolOutputParts,
 } from "./responses-content.mjs";
+import { measureRequestStage } from "./stream-performance.mjs";
 
 const IMAGE_OMISSION_TEXT = "[CCDX: earlier image omitted to fit the upstream request byte budget.]";
 
 function bodyPayload(reqBody, assertActive) {
-  assertActive?.();
-  const bodyText = JSON.stringify(reqBody);
-  assertActive?.();
-  return { bodyText, bodyBytes: Buffer.byteLength(bodyText) };
+  return measureRequestStage("serialization", () => {
+    assertActive?.();
+    const bodyText = JSON.stringify(reqBody);
+    assertActive?.();
+    return { bodyText, bodyBytes: Buffer.byteLength(bodyText) };
+  });
 }
 
 function inlineImageIdentity(part) {
@@ -264,7 +267,7 @@ export function trimResponsesHistoryToByteBudget(reqBody, {
   initialBodyBytes,
 } = {}) {
   assertActive?.();
-  let bodyText = initialBodyText ?? JSON.stringify(reqBody);
+  let bodyText = initialBodyText ?? measureRequestStage("serialization", () => JSON.stringify(reqBody));
   assertActive?.();
   let bodyBytes = initialBodyBytes ?? Buffer.byteLength(bodyText);
   const limit = Number.isFinite(targetBytes) && targetBytes > 0 ? Math.floor(targetBytes) : bodyBytes;
@@ -322,7 +325,7 @@ export function trimResponsesHistoricalImages(reqBody, {
   initialBodyBytes,
 } = {}) {
   assertActive?.();
-  let bodyText = initialBodyText ?? JSON.stringify(reqBody);
+  let bodyText = initialBodyText ?? measureRequestStage("serialization", () => JSON.stringify(reqBody));
   assertActive?.();
   let bodyBytes = initialBodyBytes ?? Buffer.byteLength(bodyText);
   const byteLimit = Number.isFinite(targetBytes) && targetBytes > 0

package/src/responses-chat-payload.mjs

@@ -6,6 +6,7 @@ import {
 import { responsesToChat } from "./responses-bridge.mjs";
 import { readResponsesImagePart, readResponsesToolOutputParts } from "./responses-content.mjs";
 import { withChatStreamUsage } from "./stream-contract.mjs";
+import { measureRequestStage } from "./stream-performance.mjs";
 
 const DEFAULT_MAX_UPSTREAM_BODY_BYTES = 30 * 1024 * 1024;
 
@@ -23,10 +24,12 @@ function configuredBodyLimit(payloadOptions) {
 }
 
 function jsonPayload(value, assertActive) {
-  assertActive?.();
-  const bodyText = JSON.stringify(value);
-  assertActive?.();
-  return { bodyText, bodyBytes: Buffer.byteLength(bodyText) };
+  return measureRequestStage("serialization", () => {
+    assertActive?.();
+    const bodyText = JSON.stringify(value);
+    assertActive?.();
+    return { bodyText, bodyBytes: Buffer.byteLength(bodyText) };
+  });
 }
 
 function chatRequest(responsesBody, stream) {

package/src/responses-handler.mjs

@@ -37,6 +37,7 @@ import { status } from "./status.mjs";
 import { endStreamWithError } from "./stream-errors.mjs";
 import { safeUpstreamResponseHeaders } from "./upstream-headers.mjs";
 import { recordResponsesUsage } from "./usage.mjs";
+import { measureRequestStage, measureRequestStageAsync } from "./stream-performance.mjs";
 
 const RESPONSES_ONLY_FALLBACK = new Set([
   "gpt-5.6-luna",
@@ -183,28 +184,28 @@ export function createResponsesHandler(options) {
       releaseRequest();
     };
     try {
-      releaseRequest = await acquireRequest(req, { signal: abort.signal });
+      releaseRequest = await measureRequestStageAsync("admission", () => acquireRequest(req, { signal: abort.signal }));
       abort.setTimeout(requestBodyTimeoutMs, "request_body_timeout");
-      parsed = await readJsonBody(req, { admission: releaseRequest, signal: abort.signal });
+      parsed = await measureRequestStageAsync("body", () => readJsonBody(req, { admission: releaseRequest, signal: abort.signal }));
       activeDeadline = now() + upstreamTimeoutMs;
       activeTimeoutReason = "responses_prepare_timeout";
       abort.setTimeout(upstreamTimeoutMs, "responses_prepare_timeout");
       if (parsed.previous_response_id !== undefined && parsed.previous_response_id !== null) {
-        historySnapshot = acquireResponseHistorySnapshot(parsed.previous_response_id, {
+        historySnapshot = measureRequestStage("history", () => acquireResponseHistorySnapshot(parsed.previous_response_id, {
           assertActive: assertPrepareActive,
           signal: abort.signal,
-        });
+        }));
         if (historySnapshot.bytes > 0) {
-          await releaseRequest.reserveResponseHistory?.(historySnapshot.bytes, { signal: abort.signal });
+          await measureRequestStageAsync("admission", () => releaseRequest.reserveResponseHistory?.(historySnapshot.bytes, { signal: abort.signal }));
         }
         assertPrepareActive();
       }
-      prepared = prepareResponsesRequest(parsed, {
+      prepared = measureRequestStage("history", () => prepareResponsesRequest(parsed, {
         assertActive: assertPrepareActive,
         copilotBoundary: false,
         historySnapshot,
         mutate: true,
-      });
+      }));
       parsed = null;
       assertPrepareActive();
       imagePressureResult = imagePressure?.apply?.(prepared, { assertActive: assertPrepareActive }) || null;
@@ -435,33 +436,33 @@ export function createResponsesCompactHandler(options) {
       releaseRequest();
     };
     try {
-      releaseRequest = await acquireRequest(req, { signal: abort.signal });
+      releaseRequest = await measureRequestStageAsync("admission", () => acquireRequest(req, { signal: abort.signal }));
       abort.setTimeout(requestBodyTimeoutMs, "request_body_timeout");
-      const parsed = await readJsonBody(req, { admission: releaseRequest, signal: abort.signal });
+      const parsed = await measureRequestStageAsync("body", () => readJsonBody(req, { admission: releaseRequest, signal: abort.signal }));
       activeDeadline = now() + upstreamTimeoutMs;
       activeTimeoutReason = "responses_prepare_timeout";
       abort.setTimeout(upstreamTimeoutMs, "responses_prepare_timeout");
       if (parsed.previous_response_id !== undefined && parsed.previous_response_id !== null) {
-        historySnapshot = acquireResponseHistorySnapshot(parsed.previous_response_id, {
+        historySnapshot = measureRequestStage("history", () => acquireResponseHistorySnapshot(parsed.previous_response_id, {
           assertActive: assertPrepareActive,
           signal: abort.signal,
-        });
+        }));
         if (historySnapshot.bytes > 0) {
-          await releaseRequest.reserveResponseHistory?.(historySnapshot.bytes, { signal: abort.signal });
+          await measureRequestStageAsync("admission", () => releaseRequest.reserveResponseHistory?.(historySnapshot.bytes, { signal: abort.signal }));
         }
         assertPrepareActive();
       }
-      prepared = prepareResponsesRequest(parsed, {
+      prepared = measureRequestStage("history", () => prepareResponsesRequest(parsed, {
         assertActive: assertPrepareActive,
         copilotBoundary: false,
         historySnapshot,
         mutate: true,
-      });
+      }));
       imagePressureResult = imagePressure?.applyRecovery?.(prepared, { assertActive: assertPrepareActive }) || null;
       if (imagePressureResult?.adapted) {
         console.warn(status("warn", `responses compact visual history mode=recovery historical_images=${imagePressureResult.initialHistoricalImages}->${imagePressureResult.historicalImages} omitted=${imagePressureResult.imagesOmitted} bytes=${imagePressureResult.initialBodyBytes}->${imagePressureResult.bodyBytes}`));
       }
-      prepared = prepareResponsesCompactionRequest(prepared);
+      prepared = measureRequestStage("history", () => prepareResponsesCompactionRequest(prepared));
       assertPrepareActive();
       prepared.surface = "responses_compact";
       const model = parsed.model || "unknown";

package/src/responses-proxy.mjs

@@ -32,12 +32,56 @@ import { safeUpstreamResponseHeaders } from "./upstream-headers.mjs";
 import { recordResponsesUsage } from "./usage.mjs";
 
 const MAX_SSE_BUFFER_BYTES = loadRuntimeConfig().maxSseBufferBytes;
+const SSE_WRITE_BATCH_BYTES = 64 * 1024;
 const RESPONSES_TERMINAL_EVENT_TYPES = new Set([
   "response.completed",
   "response.incomplete",
   "response.failed",
   "error",
 ]);
+const MESSAGE_CONTENT_EVENT_TYPES = new Set([
+  "response.content_part.added",
+  "response.content_part.done",
+  "response.output_text.delta",
+  "response.output_text.done",
+  "response.output_text.annotation.added",
+  "response.refusal.delta",
+  "response.refusal.done",
+]);
+
+function normalizeMessageIds(state, event, eventType) {
+  let changed = false;
+  const replaceId = (object, field, id) => {
+    if (id && typeof object?.[field] === "string" && object[field] !== id) {
+      object[field] = id;
+      changed = true;
+    }
+  };
+  const index = event.output_index;
+  if (Number.isSafeInteger(index) && index >= 0) {
+    // Bind only confirmed message items, using the full output index (including tools).
+    // IDs are opaque: retain the first upstream ID without decoding or synthesizing one.
+    if (eventType === "response.output_item.added" && event.item?.type === "message"
+      && typeof event.item.id === "string" && event.item.id.length > 0
+      && !state.messageIds.has(index)) {
+      state.messageIds.set(index, event.item.id);
+    }
+    const id = state.messageIds.get(index);
+    if (["response.output_item.added", "response.output_item.done"].includes(eventType)
+      && event.item?.type === "message") {
+      replaceId(event.item, "id", id);
+    } else if (MESSAGE_CONTENT_EVENT_TYPES.has(eventType)) {
+      replaceId(event, "item_id", id);
+    }
+  }
+  if (RESPONSES_TERMINAL_EVENT_TYPES.has(eventType) && Array.isArray(event.response?.output)) {
+    for (let index = 0; index < event.response.output.length; index += 1) {
+      const item = event.response.output[index];
+      if (item?.type === "message") replaceId(item, "id", state.messageIds.get(index));
+    }
+  }
+  return changed;
+}
 
 function inspectResponseSseEvent(state, eventName, data) {
   if (!data || state.sawTerminal) return;
@@ -57,6 +101,7 @@ function inspectResponseSseEvent(state, eventName, data) {
     );
   }
   const eventType = event.type || eventName;
+  const changed = normalizeMessageIds(state, event, eventType);
   const outputTokens = event.response?.usage?.output_tokens ?? event.usage?.output_tokens;
   if (outputTokens !== undefined) markOutputTokens(outputTokens);
   if (!state.sawOutput && isResponsesOutputEvent(event, eventType)) {
@@ -66,7 +111,7 @@ function inspectResponseSseEvent(state, eventName, data) {
   if (["response.function_call_arguments.delta", "response.custom_tool_call_input.delta"].includes(eventType)) {
     state.toolArgumentGuard.observe(event.output_index ?? event.item_id ?? event.call_id, event.delta);
   }
-  if (!RESPONSES_TERMINAL_EVENT_TYPES.has(eventType)) return;
+  if (!RESPONSES_TERMINAL_EVENT_TYPES.has(eventType)) return changed ? event : null;
   const expectedStatus = {
     "response.completed": "completed",
     "response.incomplete": "incomplete",
@@ -80,12 +125,14 @@ function inspectResponseSseEvent(state, eventName, data) {
   }
   state.sawTerminal = true;
   if (eventType === "response.failed" || eventType === "error") markStreamFailure();
-  if (eventType !== "response.completed") return;
-  const response = event.response;
-  state.completed = { response, event };
+  if (eventType === "response.completed") {
+    const response = event.response;
+    state.completed = { response, event };
+  }
+  return changed ? event : null;
 }
 
-function createResponseSseInspector(state) {
+function createResponseSseTransformer(state) {
   let eventBuffer = Buffer.allocUnsafe(Math.min(4096, MAX_SSE_BUFFER_BYTES + 4));
   let eventLength = 0;
   let previous1 = -1;
@@ -111,12 +158,12 @@ function createResponseSseInspector(state) {
     eventLength += bytes.byteLength;
   };
 
-  const inspectEvent = (delimiterLength) => {
-    const contentLength = eventLength - delimiterLength;
+  const inspectEvent = (frame, delimiterLength) => {
+    const contentLength = frame.byteLength - delimiterLength;
     if (contentLength > MAX_SSE_BUFFER_BYTES) {
       throw httpError(`Upstream SSE buffer exceeds ${MAX_SSE_BUFFER_BYTES} bytes`, 502);
     }
-    const chunk = eventBuffer.subarray(0, contentLength).toString("utf8");
+    const chunk = frame.subarray(0, contentLength).toString("utf8");
     const lines = chunk.split(/\r?\n/);
     const eventName = lines
       .find((line) => line.startsWith("event:"))
@@ -126,15 +173,31 @@ function createResponseSseInspector(state) {
       .filter((line) => line.startsWith("data:"))
       .map((line) => line.slice(5).trimStart())
       .join("\n");
-    if (data) inspectResponseSseEvent(state, eventName, data);
-    eventLength = 0;
-    previous1 = -1;
-    previous2 = -1;
-    previous3 = -1;
+    const changed = data && inspectResponseSseEvent(state, eventName, data);
+    if (!changed) return frame;
+    let wroteData = false;
+    const rewritten = lines.filter((line) => {
+      if (!line.startsWith("data:")) return true;
+      if (wroteData) return false;
+      wroteData = true;
+      return true;
+    }).map((line) => line.startsWith("data:") ? `data: ${JSON.stringify(changed)}` : line);
+    return Buffer.from(rewritten.join(chunk.includes("\r\n") ? "\r\n" : "\n")
+      + frame.subarray(contentLength).toString("ascii"));
   };
 
   return {
     push(value) {
+      // Fetch exposes Uint8Array chunks; use a zero-copy Buffer view for UTF-8 decoding.
+      if (!Buffer.isBuffer(value)) value = Buffer.from(value.buffer, value.byteOffset, value.byteLength);
+      let pending = [];
+      let pendingBytes = 0;
+      const flush = () => {
+        const output = pending.length === 1 ? pending[0] : Buffer.concat(pending, pendingBytes);
+        pending = [];
+        pendingBytes = 0;
+        return output;
+      };
       let segmentStart = 0;
       for (let index = 0; index < value.byteLength; index += 1) {
         const byte = value[index];
@@ -145,10 +208,32 @@ function createResponseSseInspector(state) {
           delimiterLength = previous3 === 0x0d ? 4 : 3;
         }
         if (delimiterLength) {
-          append(value.subarray(segmentStart, index + 1));
-          inspectEvent(delimiterLength);
+          let frame = value.subarray(segmentStart, index + 1);
+          if (eventLength > 0) {
+            append(frame);
+            // Writes may retain a Buffer after returning; never expose the reusable buffer.
+            if (eventLength > SSE_WRITE_BATCH_BYTES) {
+              frame = eventBuffer.subarray(0, eventLength);
+              eventBuffer = Buffer.allocUnsafe(Math.min(4096, MAX_SSE_BUFFER_BYTES + 4));
+            } else {
+              frame = Buffer.from(eventBuffer.subarray(0, eventLength));
+            }
+          }
+          eventLength = 0;
+          previous1 = -1;
+          previous2 = -1;
+          previous3 = -1;
           segmentStart = index + 1;
-          if (state.sawTerminal) return index + 1;
+          const forwarded = inspectEvent(frame, delimiterLength);
+          const last = pending.at(-1);
+          if (last && last.buffer === forwarded.buffer
+            && last.byteOffset + last.byteLength === forwarded.byteOffset) {
+            pending[pending.length - 1] = Buffer.from(last.buffer, last.byteOffset, last.byteLength + forwarded.byteLength);
+          } else {
+            pending.push(forwarded);
+          }
+          pendingBytes += forwarded.byteLength;
+          if (state.sawTerminal) return flush();
           continue;
         }
         previous3 = previous2;
@@ -159,7 +244,7 @@ function createResponseSseInspector(state) {
       if (eventLength > MAX_SSE_BUFFER_BYTES + 3) {
         throw httpError(`Upstream SSE buffer exceeds ${MAX_SSE_BUFFER_BYTES} bytes`, 502);
       }
-      return null;
+      return pendingBytes > 0 ? flush() : null;
     },
   };
 }
@@ -235,8 +320,9 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
       sawTerminal: false,
       sawOutput: false,
       toolArgumentGuard: new ToolArgumentDeltaGuard(),
+      messageIds: new Map(),
     };
-    const inspector = createResponseSseInspector(streamState);
+    const transformer = createResponseSseTransformer(streamState);
     try {
       while (true) {
         const { done, value } = await reader.read();
@@ -247,9 +333,12 @@ export async function proxyCopilotResponses(reqContext, req, res, upstream = cop
           return { successful: Boolean(streamState.completed), compacted: false };
         }
         options.abort?.setTimeout(options.streamIdleTimeoutMs, "stream_idle_timeout");
-        const terminalOffset = inspector.push(value);
-        const forwarded = terminalOffset === null ? value : value.subarray(0, terminalOffset);
-        if (forwarded.byteLength > 0 && !await writeOrDrain(res, forwarded)) return;
+        // Batch within this read, not across reads, and await drain before processing more.
+        for (let offset = 0; offset < value.byteLength; offset += SSE_WRITE_BATCH_BYTES) {
+          const frame = transformer.push(value.subarray(offset, offset + SSE_WRITE_BATCH_BYTES));
+          if (frame && !await writeOrDrain(res, frame)) return;
+          if (streamState.sawTerminal) break;
+        }
         if (streamState.sawTerminal) {
           storeCompletedResponse(reqContext, streamState.completed);
           res.end();

package/src/stream-performance.mjs

@@ -2,6 +2,8 @@ import { performance } from "node:perf_hooks";
 import { currentRequestContext } from "./request-context.mjs";
 
 const PERFORMANCE_ROUTES = Object.freeze(["responses", "responses_compact"]);
+const PREPARATION_STAGES = Object.freeze(["admission", "body", "history", "images", "serialization"]);
+const PREPARATION_EDGES_MS = Object.freeze([1, 2, 5, 10, 20, 50, 100, 250, 500, 1_000, 5_000, 30_000, 120_000]);
 const TTFT_EDGES_MS = Object.freeze([
   100, 200, 300, 500, 700, 1_000, 1_400, 2_000, 2_800, 4_000,
   5_500, 8_000, 12_000, 16_000, 24_000, 36_000, 60_000, 120_000, 300_000,
@@ -80,6 +82,8 @@ function createRoutePerformance() {
   return {
     ttft: createHistogram(TTFT_EDGES_MS),
     tpot: createHistogram(TPOT_EDGES_US),
+    requestTtft: createHistogram(TTFT_EDGES_MS),
+    preparation: Object.fromEntries(PREPARATION_STAGES.map((stage) => [stage, createHistogram(PREPARATION_EDGES_MS)])),
     success_with_output: 0,
     errors_with_output: 0,
     zero_output_errors: 0,
@@ -95,6 +99,8 @@ function routeSnapshot(route) {
     neutral: route.neutral,
     ttft_ms: histogramSnapshot(route.ttft, "ms"),
     tpot_us: histogramSnapshot(route.tpot, "us"),
+    request_ttft_ms: histogramSnapshot(route.requestTtft, "ms"),
+    preparation_ms: Object.fromEntries(PREPARATION_STAGES.map((stage) => [stage, histogramSnapshot(route.preparation[stage], "ms")])),
   };
 }
 
@@ -105,6 +111,7 @@ export function createStreamPerformanceMetrics({ now = () => performance.now() }
     begin(routeName) {
       const route = routes[routeName];
       if (!route) return null;
+      const requestStartedAt = now();
       let upstreamStartedAt = null;
       let firstOutputAt = null;
       let outputTokens = null;
@@ -112,6 +119,17 @@ export function createStreamPerformanceMetrics({ now = () => performance.now() }
       let finished = false;
 
       return {
+        beginStage(stage) {
+          const histogram = route.preparation[stage];
+          if (!histogram || finished) return null;
+          const startedAt = now();
+          let ended = false;
+          return () => {
+            if (ended) return;
+            ended = true;
+            observe(histogram, Math.max(0, now() - startedAt));
+          };
+        },
         upstreamStarted() {
           if (upstreamStartedAt === null) upstreamStartedAt = now();
         },
@@ -130,6 +148,7 @@ export function createStreamPerformanceMetrics({ now = () => performance.now() }
           finished = true;
           failed ||= finishFailed;
           const finishedAt = now();
+          if (firstOutputAt !== null) observe(route.requestTtft, Math.max(0, firstOutputAt - requestStartedAt));
           if (upstreamStartedAt === null) {
             route.neutral += 1;
             return;
@@ -160,6 +179,24 @@ function tracker() {
   return currentRequestContext()?.streamPerformance || null;
 }
 
+export function measureRequestStage(stage, operation) {
+  const finish = tracker()?.beginStage?.(stage);
+  try {
+    return operation();
+  } finally {
+    finish?.();
+  }
+}
+
+export async function measureRequestStageAsync(stage, operation) {
+  const finish = tracker()?.beginStage?.(stage);
+  try {
+    return await operation();
+  } finally {
+    finish?.();
+  }
+}
+
 export function markUpstreamStarted() {
   tracker()?.upstreamStarted();
 }