codex-grok-bridge 1.6.1 → 1.7.1

package/CHANGELOG.md

@@ -1,6 +1,14 @@
 # Changelog
 
-## Unreleased
+## 1.7.1 — 2026-09-30
+
+- The bridge holds the upstream reply and sends it to Codex only after `response.completed`. A reset before that (`ECONNRESET`, `EPIPE`, `UND_ERR_SOCKET`, or a close before the reply finishes) is retried up to two more times, and only if Codex has not been sent a byte. This provider's Codex `request_max_retries` and `stream_max_retries` are 0. A retry sends the prompt again, so that attempt's input tokens can be billed again.
+- When `grok --version` fails or does not report a version, the client version is `unknown` instead of the stale `1.0.24`.
+
+## 1.7.0 — 2026-09-29
+
+- One `x-grok-conv-id` per Codex thread, and the forwarded transcript prefix stays byte-stable so prompt cache can hit. The full transcript is still sent.
+- Upstream `cached_prompt_tokens` and `cache_read_input_tokens` are copied onto `response.completed` usage and the diagnostics log when the proxy sends them, including `0`. Missing counters are not invented.
 
 ## 1.6.1 — 2026-09-28
 

package/README.md

@@ -72,7 +72,7 @@ starts, and tears the provider down with that process.
 ```sh
 git clone https://github.com/deximple/codex-grok-bridge.git
 cd codex-grok-bridge
-npm test                        # 160 tests, no network, no inference
+npm test                        # 174 tests, no network, no inference
 node scripts/codex-grok.mjs
 ```
 
@@ -120,18 +120,38 @@ address when one exists, so a 5–20 s tool gap does not force a fresh name
 lookup every time. `GROK_BRIDGE_TRANSPORT=fetch` restores the older `fetch`
 path.
 
-A request that dies **before** the first SSE block is written to Codex is
-retried once. After the first block, the bridge never retries — Codex already
-saw bytes, so a replay would duplicate them. Deterministic refusals (422) and
-user aborts are not retried either.
+The bridge holds Grok’s reply until it finishes (`response.completed`). No
+byte of that reply is sent to Codex before then. If the connection drops
+first — a reset (`ECONNRESET`, `EPIPE`, `UND_ERR_SOCKET`) or a close before
+the reply finishes — the bridge sends the same request again, up to two more
+times. DNS and connection failures use that same budget. This provider sets
+Codex `request_max_retries` and `stream_max_retries` to 0, so Codex does not
+send the prompt again as well. If any byte of the reply was already sent to
+Codex, the bridge does not resubmit. A retry sends the prompt to Grok again,
+so that attempt’s input tokens can be billed again. This replaces Codex
+resending the prompt up to two extra times. User aborts and deterministic 422
+responses are not retried. Codex still gives the stream five minutes of
+silence (`stream_idle_timeout_ms`); a reply that takes longer can be cut off
+before any of it is sent.
+
+### In short
+
+Codex keeps the conversation and sends the whole turn each time. Grok can
+treat the unchanged beginning as a cache. If the connection drops before
+Codex has been sent any bytes, the bridge tries again. Codex does not send
+that same request again. Voice, cloud tasks, and video are not in this
+release.
 
 If the Responses path misbehaves, `GROK_BRIDGE_INFERENCE=cli` falls back to the
 older CLI envelope. That path pastes the whole JSON into a prompt each turn, so
 it is slower and more expensive, has no token-by-token streaming, and is capped
 at three minutes per request.
 
-The default Responses path streams. Codex `prompt_cache_key` is forwarded as
-`x-grok-conv-id`.
+The default Responses path streams. One Codex thread keeps a single
+`x-grok-conv-id`: the `thread-id` header when Codex sends it, otherwise
+`prompt_cache_key`. That same id is written into `prompt_cache_key`. An
+unchanged transcript prefix is resent byte-for-byte; the new items are
+appended, not substituted for the history.
 
 ## What works and what does not
 
@@ -236,15 +256,15 @@ ephemeral threads and child agents refuse a switch. Other subscribers can block
 the reload; if they do, no inference is sent. Pick the model you want when
 starting a new thread, before the first turn is saved.
 
-### Not guaranteed
+### Not in this release
 
-Voice, cloud tasks, and video generation are not promised.
+Voice, cloud tasks, and video generation are not in this release.
 
 Upstream sometimes resets the connection mid-response (three measured cases:
-25 s / 27 s / 253 s, 726 KB–22 MB). The bridge cannot retry after the first
-SSE byte. The provider sets Codex `request_max_retries` / `stream_max_retries`
-to 2 so **Codex** can resend the same request. Only the side that owns the
-conversation can retry safely.
+25 s / 27 s / 253 s, 726 KB–22 MB). The bridge holds the reply and, if that
+drop happens before Codex has been sent any of it, tries the same request up
+to two more times. Codex does not also resend it. A retry can bill the input
+tokens for that attempt again.
 
 ## Diagnostics
 
@@ -288,7 +308,7 @@ Start here when something breaks. Do not open `~/.grok/auth.json` or
 ## Verify
 
 ```sh
-npm test                    # 160 tests, no remote inference
+npm test                    # 174 tests, no remote inference
 npm run test:coverage       # 80% line / branch / function gate
 npm run verify:app-server   # real app-server routing; also runs against an installed bundle
 npm audit --omit=dev
@@ -399,7 +419,7 @@ codex-grok exec --skip-git-repo-check --sandbox workspace-write '작업 내용'
 ```sh
 git clone https://github.com/deximple/codex-grok-bridge.git
 cd codex-grok-bridge
-npm test                        # 160건, 네트워크·추론 없음
+npm test                        # 174건, 네트워크·추론 없음
 node scripts/codex-grok.mjs
 ```
 
@@ -444,17 +464,34 @@ Codex 창에도 보일 수 있습니다.
 실행으로 5–20초가 비어도 매번 이름을 다시 찾지 않기 위해서입니다.
 `GROK_BRIDGE_TRANSPORT=fetch`로 이전 `fetch` 경로로 되돌릴 수 있습니다.
 
-Codex에 첫 SSE 블록을 쓰기 **전**에 죽은 요청은 한 번 다시 보냅니다. 아직
-아무것도 전달하지 않았으므로 재전송이 대화를 오염시키지 않습니다. 첫 블록을
-쓴 뒤에는 절대 재시도하지 않습니다. 422 같은 결정적 거절과 사용자 중단도
-재시도하지 않습니다.
+브리지는 Grok의 응답이 끝날 때까지(`response.completed`) 들고 있습니다.
+그 전에 Codex로 응답 바이트를 보내지 않습니다. 그 전에 연결이 끊기면 —
+리셋(`ECONNRESET`, `EPIPE`, `UND_ERR_SOCKET`)이거나, 응답이 끝나기 전에
+닫힌 경우 — 브리지가 같은 요청을 최대 두 번 더 보냅니다. DNS·연결 실패도
+같은 횟수를 씁니다. 이 provider는 Codex `request_max_retries`와
+`stream_max_retries`를 0으로 두어, Codex가 그 프롬프트를 또 보내지 않게
+합니다. 응답 바이트를 Codex에 이미 보냈다면 브리지는 다시 보내지 않습니다.
+재시도는 프롬프트를 Grok에 다시 보내므로, 그 시도의 입력 토큰이 다시 과금될
+수 있습니다. Codex가 프롬프트를 최대 두 번 더 보내던 것을 이것으로 대신합니다.
+사용자 중단과 422 같은 결정적 거절은 재시도하지 않습니다. Codex는 응답
+바이트 없이 5분(`stream_idle_timeout_ms`)을 기다립니다. 그보다 오래 걸리면
+보내기 전에 끊길 수 있습니다.
+
+### 쉽게 말하면
+
+Codex가 대화를 갖고 있고, 턴마다 그 턴 전체를 보냅니다. 앞부분이 그대로면
+Grok는 그 부분을 캐시로 볼 수 있습니다. Codex에 바이트를 보내기 전에 연결이
+끊기면 브리지가 다시 시도합니다. Codex는 그 요청을 또 보내지 않습니다. 음성,
+클라우드 작업, 영상은 이번 릴리스에 없습니다.
 
 Responses 경로가 이상하면 `GROK_BRIDGE_INFERENCE=cli`로 이전 CLI 봉투 경로를
 씁니다. 매 턴 전체 JSON을 프롬프트로 넣으므로 더 느리고 비싸며, 토큰 단위
 실시간 출력이 없고, 요청당 3분 제한입니다.
 
-기본 Responses 경로는 스트림을 전달합니다. Codex `prompt_cache_key`는
-`x-grok-conv-id`로 넘깁니다.
+기본 Responses 경로는 스트림을 전달합니다. Codex 스레드마다 `x-grok-conv-id`는
+하나입니다. `thread-id` 헤더가 있으면 그 값이고, 없으면 `prompt_cache_key`입니다.
+같은 id를 `prompt_cache_key`에도 넣습니다. 바뀌지 않은 트랜스크립트 접두는
+바이트 그대로 다시 보내고, 새 항목은 그 뒤에 붙입니다.
 
 ## 확인된 동작과 제한
 
@@ -553,15 +590,14 @@ OpenAI 경로가 맞습니다.
 구독자가 재로드를 막으면 추론을 보내지 않습니다. 첫 턴이 저장되기 전에 새
 작업을 시작할 때 원하는 모델을 고르세요.
 
-### 아직 보장하지 않는 것
+### 이번 릴리스에 없는 것
 
-음성, 클라우드 작업, 영상 생성은 약속하지 않습니다.
+음성, 클라우드 작업, 영상 생성은 이번 릴리스에 없습니다.
 
 상류가 응답 중간에 연결을 리셋하는 경우가 있습니다(실측 3건: 25초 / 27초 /
-253초, 726 KB–22 MB). 첫 SSE 바이트 이후에는 브리지가 재시도할 수 없습니다.
-provider에 Codex `request_max_retries` / `stream_max_retries`를 2로 두어
-**Codex**가 같은 요청을 다시 보내게 했습니다. 대화를 소유한 쪽만 안전하게
-재시도할 수 있습니다.
+253초, 726 KB–22 MB). 브리지는 응답을 들고 있다가, Codex에 그 응답을 보내기
+전에 끊기면 같은 요청을 최대 두 번 더 시도합니다. Codex가 또 보내지는
+않습니다. 재시도하면 그 시도의 입력 토큰이 다시 과금될 수 있습니다.
 
 ## 진단 로그
 
@@ -604,7 +640,7 @@ provider에 Codex `request_max_retries` / `stream_max_retries`를 2로 두어
 ## 검증
 
 ```sh
-npm test                    # 160건, 외부 추론 없음
+npm test                    # 174건, 외부 추론 없음
 npm run test:coverage       # line/branch/function 80% 게이트
 npm run verify:app-server   # 실제 app-server 라우팅. 설치된 앱 번들에서도 실행
 npm audit --omit=dev

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "codex-grok-bridge",
-  "version": "1.6.1",
+  "version": "1.7.1",
   "description": "Run Grok 4.7 as the model inside Codex, with Codex still owning tools, permissions, history and MCP. Uses the grok login session, not an API key.",
   "type": "module",
   "license": "MIT",

package/src/bridge.mjs

@@ -8,7 +8,13 @@ import {
   parseGrokResult,
   runGrok,
 } from "./cli-inference.mjs";
-import { openProxyStreamWithRetry, pipeProxySse } from "./proxy.mjs";
+import { relayProxySse } from "./proxy.mjs";
+import {
+  applyCacheUsage,
+  createPrefixMemory,
+  readCacheUsage,
+  stableConvId,
+} from "./prefix.mjs";
 import { toProxyRequest } from "./tools.mjs";
 import { classifyBridgeError, errorSignature, bridgeErrorMessage } from "./errors.mjs";
 import { createDiagnostics } from "./diagnostics.mjs";
@@ -36,6 +42,7 @@ export function createBridgeServer(options = {}) {
     limit: options.maxConcurrentInference,
     queueLimit: options.maxQueuedInference,
   });
+  const prefixes = createPrefixMemory();
   return http.createServer(async (req, res) => {
     const json = (status, data) => {
       res.writeHead(status, { "content-type": "application/json" });
@@ -94,8 +101,14 @@ export function createBridgeServer(options = {}) {
       );
     const id = "resp_" + randomUUID();
     const startedAt = Date.now();
+    // A keepalive comment is an SSE byte. Writing one before the reply finishes
+    // would make a later upstream reset unsafe to retry, so the proxy path
+    // stays quiet until the first real event is handed to Codex. The CLI path
+    // writes its own events and may keep the socket warm while it runs.
+    let responseStarted = false;
     const keepalive = setInterval(() => {
-      if (!res.destroyed && !res.writableEnded) res.write(": keepalive\n\n");
+      if (!responseStarted || res.destroyed || res.writableEnded) return;
+      res.write(": keepalive\n\n");
     }, 10000);
     const useCli =
       Boolean(options.runGrok) ||
@@ -112,11 +125,13 @@ export function createBridgeServer(options = {}) {
     const queuedAt = Date.now();
     let acquired = false;
     let queuedMs = 0;
+    let cacheUsage = null;
     try {
       await slots.acquire(controller.signal);
       acquired = true;
       queuedMs = Date.now() - queuedAt;
       if (useCli) {
+        responseStarted = true;
         event("response.created", { response: { id } });
         const invocation = buildGrokInvocation(body, options);
         invocation.threadId = req.headers["thread-id"];
@@ -135,14 +150,18 @@ export function createBridgeServer(options = {}) {
             parsed.usage?.input_tokens ?? parsed.usage?.inputTokens ?? 0,
           outputTokens =
             parsed.usage?.output_tokens ?? parsed.usage?.outputTokens ?? 0;
+        const usage = {
+          input_tokens: inputTokens,
+          output_tokens: outputTokens,
+          total_tokens: inputTokens + outputTokens,
+        };
+        cacheUsage = readCacheUsage(parsed.usage);
+        if (cacheUsage) Object.assign(usage, cacheUsage);
+        applyCacheUsage(usage);
         event("response.completed", {
           response: {
             id,
-            usage: {
-              input_tokens: inputTokens,
-              output_tokens: outputTokens,
-              total_tokens: inputTokens + outputTokens,
-            },
+            usage,
           },
         });
       } else {
@@ -150,29 +169,49 @@ export function createBridgeServer(options = {}) {
         // Grok takes input_image blocks natively; only unusable attachments
         // are swapped for an explanation, so one bad image cannot make the
         // upstream reject the whole conversation.
-        const { request, map } = toProxyRequest(sanitizeImages(body));
-        const proxy = await openProxyStreamWithRetry({
-          token: session.token,
-          userId: session.userId,
-          body: request,
-          signal: controller.signal,
-          fetchImpl: options.proxyFetch,
-          baseUrl: options.proxyBaseUrl,
-          convId: body.prompt_cache_key,
-          sessionId: req.headers["thread-id"],
-          onRetry: ({ attempt, kind }) =>
-            diagnostics.record({
-              event: "turn_retried",
-              kind,
-              attempt,
-              elapsedMs: Date.now() - startedAt,
-            }),
-        });
-        await pipeProxySse(proxy.body, res, map);
+        const threadId = req.headers["thread-id"];
+        const convId = stableConvId(
+          Array.isArray(threadId) ? threadId[0] : threadId,
+          body.prompt_cache_key,
+        );
+        const { request, map, projected } = toProxyRequest(sanitizeImages(body));
+        if (convId) request.prompt_cache_key = convId;
+        request.input = prefixes.reuse(convId, projected);
+        const usageBox = {};
+        try {
+          await relayProxySse(
+            {
+              token: session.token,
+              userId: session.userId,
+              body: request,
+              signal: controller.signal,
+              fetchImpl: options.proxyFetch,
+              baseUrl: options.proxyBaseUrl,
+              convId,
+              sessionId: Array.isArray(threadId) ? threadId[0] : threadId,
+              onClientByte: () => {
+                responseStarted = true;
+              },
+              onRetry: ({ attempt, kind }) =>
+                diagnostics.record({
+                  event: "turn_retried",
+                  kind,
+                  attempt,
+                  elapsedMs: Date.now() - startedAt,
+                }),
+            },
+            res,
+            map,
+            usageBox,
+          );
+        } finally {
+          cacheUsage = usageBox.cacheUsage ?? null;
+        }
       }
       diagnostics.record({
         event: "turn_ok",
         ...shape,
+        ...(cacheUsage ?? {}),
         queuedMs,
         elapsedMs: Date.now() - startedAt,
       });
@@ -183,6 +222,7 @@ export function createBridgeServer(options = {}) {
         kind,
         signature: errorSignature(error),
         ...shape,
+        ...(cacheUsage ?? {}),
         queuedMs,
         elapsedMs: Date.now() - startedAt,
         detail: error?.message,

package/src/errors.mjs

@@ -40,7 +40,14 @@ const CONNECT_CODES = new Set([
   "DEPTH_ZERO_SELF_SIGNED_CERT", "ERR_TLS_CERT_ALTNAME_INVALID",
 ]);
 const TIMEOUT_CODES = new Set(["UND_ERR_HEADERS_TIMEOUT", "UND_ERR_BODY_TIMEOUT", "ETIMEDOUT"]);
-const CLOSED_CODES = new Set(["UND_ERR_SOCKET", "ECONNRESET", "EPIPE"]);
+const CLOSED_CODES = new Set([
+  "UND_ERR_SOCKET",
+  "ECONNRESET",
+  "EPIPE",
+  // The body ended without response.completed. Same class as a socket reset:
+  // the reply never finished, and Codex has not been given a partial one.
+  "ERR_UPSTREAM_PREMATURE_CLOSE",
+]);
 const ABORT_CODES = new Set([
   "ABORT_ERR", "ERR_STREAM_DESTROYED", "ERR_STREAM_WRITE_AFTER_END", "ERR_STREAM_PREMATURE_CLOSE",
 ]);

package/src/prefix.mjs

@@ -0,0 +1,163 @@
+import { createHash } from "node:crypto";
+
+// xAI prompt cache hits only when later requests repeat the same prefix
+// unchanged, on the same x-grok-conv-id. Codex resends the whole transcript,
+// but a fresh rewrite (new ids, encrypted reasoning, tool index names) makes
+// that prefix differ. This module pins the conv id and reuses the exact
+// items already forwarded.
+
+const CACHE_USAGE_FIELDS = [
+  "cached_prompt_tokens",
+  "cache_read_input_tokens",
+  "cache_creation_input_tokens",
+];
+
+const VOLATILE_KEYS = new Set([
+  "id",
+  "status",
+  "encrypted_content",
+  "encrypted_function_args",
+]);
+
+export function stableConvId(threadId, promptCacheKey) {
+  const thread = typeof threadId === "string" ? threadId.trim() : "";
+  if (thread) return thread;
+  const cache = typeof promptCacheKey === "string" ? promptCacheKey.trim() : "";
+  return cache || null;
+}
+
+export function stableProxyName(spec, sanitizedName) {
+  const basis = `${spec.kind}\0${spec.namespace ?? ""}\0${spec.name ?? ""}`;
+  const hash = createHash("sha256").update(basis).digest("hex").slice(0, 8);
+  return `codex_${hash}_${sanitizedName}`;
+}
+
+function normalizeForFingerprint(value) {
+  if (Array.isArray(value)) return value.map(normalizeForFingerprint);
+  if (!value || typeof value !== "object") return value;
+  if (
+    value.type === "input_text" ||
+    value.type === "output_text" ||
+    value.type === "summary_text"
+  ) {
+    return {
+      type: value.type,
+      text: typeof value.text === "string" ? value.text : "",
+    };
+  }
+  if (value.type === "input_image") {
+    const url =
+      value.image_url && typeof value.image_url === "object"
+        ? value.image_url.url
+        : value.image_url;
+    const image = { type: "input_image" };
+    if (url !== undefined) image.image_url = url;
+    if (value.detail !== undefined) image.detail = value.detail;
+    return image;
+  }
+  const out = {};
+  for (const key of Object.keys(value).sort()) {
+    if (VOLATILE_KEYS.has(key) || key.startsWith("internal_")) continue;
+    const child = value[key];
+    if (child === undefined || child === null) continue;
+    out[key] = normalizeForFingerprint(child);
+  }
+  if (
+    out.role &&
+    out.content !== undefined &&
+    (out.type == null || out.type === "message")
+  ) {
+    out.type = "message";
+    if (typeof out.content === "string")
+      out.content = [{ type: "input_text", text: out.content }];
+  }
+  return out;
+}
+
+export function logicalFingerprint(item) {
+  return JSON.stringify(normalizeForFingerprint(item));
+}
+
+// Object key order is part of the cached prefix. Codex does not promise it.
+export function stableJsonValue(value) {
+  if (Array.isArray(value)) return value.map(stableJsonValue);
+  if (!value || typeof value !== "object") return value;
+  const out = {};
+  for (const key of Object.keys(value).sort())
+    out[key] = stableJsonValue(value[key]);
+  return out;
+}
+
+const MAX_THREADS = 64;
+
+export function createPrefixMemory() {
+  const threads = new Map();
+  return {
+    reuse(convId, pairs) {
+      const fresh = pairs.map((pair) => pair.item);
+      if (!convId || pairs.length === 0) return fresh;
+      const previous = threads.get(convId);
+      let reused = 0;
+      if (previous) {
+        const limit = Math.min(previous.length, pairs.length);
+        while (
+          reused < limit &&
+          previous[reused].fingerprint === pairs[reused].fingerprint
+        )
+          reused += 1;
+      }
+      const stored =
+        reused > 0
+          ? [...previous.slice(0, reused), ...pairs.slice(reused)]
+          : pairs.slice();
+      if (threads.has(convId)) threads.delete(convId);
+      threads.set(
+        convId,
+        stored.map((pair) => ({
+          fingerprint: pair.fingerprint,
+          item: structuredClone(pair.item),
+        })),
+      );
+      while (threads.size > MAX_THREADS) {
+        const oldest = threads.keys().next().value;
+        threads.delete(oldest);
+      }
+      if (!previous || reused === 0) return fresh;
+      return [
+        ...previous.slice(0, reused).map((pair) => structuredClone(pair.item)),
+        ...pairs.slice(reused).map((pair) => pair.item),
+      ];
+    },
+  };
+}
+
+// Copy only cache counters the upstream actually sent. Zero is a real miss.
+// Absence is not a miss, so it must not become 0.
+export function readCacheUsage(usage) {
+  if (!usage || typeof usage !== "object") return null;
+  const out = {};
+  for (const key of CACHE_USAGE_FIELDS) {
+    const value = usage[key];
+    if (typeof value === "number" && Number.isFinite(value)) out[key] = value;
+  }
+  return Object.keys(out).length ? out : null;
+}
+
+export function applyCacheUsage(usage) {
+  const cache = readCacheUsage(usage);
+  if (!cache) return usage;
+  const cached =
+    typeof cache.cached_prompt_tokens === "number"
+      ? cache.cached_prompt_tokens
+      : cache.cache_read_input_tokens;
+  if (typeof cached !== "number") return usage;
+  const details =
+    usage.input_tokens_details && typeof usage.input_tokens_details === "object"
+      ? usage.input_tokens_details
+      : {};
+  if (typeof details.cached_tokens !== "number") {
+    details.cached_tokens = cached;
+    usage.input_tokens_details = details;
+  }
+  return usage;
+}

package/src/proxy.mjs

@@ -8,8 +8,13 @@ import { BRIDGE_ERROR, classifyBridgeError } from "./errors.mjs";
 export const DEFAULT_PROXY_BASE = "https://cli-chat-proxy.grok.com/v1";
 export const DEFAULT_CLIENT_IDENTIFIER = "grok-shell";
 
+const UNKNOWN_CLIENT_VERSION = "unknown";
 let cachedClientVersion;
 
+export function clearClientVersionCache() {
+  cachedClientVersion = undefined;
+}
+
 export function detectGrokClientVersion(home = homedir()) {
   if (cachedClientVersion) return cachedClientVersion;
   try {
@@ -20,9 +25,9 @@ export function detectGrokClientVersion(home = homedir()) {
       timeout: 5000,
     });
     const match = output.match(/grok\s+(\S+(?:\s+\([^)]+\))?)/i);
-    cachedClientVersion = match ? match[1].trim() : "1.0.24";
+    cachedClientVersion = match ? match[1].trim() : UNKNOWN_CLIENT_VERSION;
   } catch {
-    cachedClientVersion = "1.0.24";
+    cachedClientVersion = UNKNOWN_CLIENT_VERSION;
   }
   return cachedClientVersion;
 }
@@ -74,9 +79,9 @@ export async function openProxyStream(options) {
   return response;
 }
 
-// Resolve when the consumer has drained, reject if it goes away first. Without
-// this, a slow client makes the bridge buffer the whole Grok response in memory,
-// and a client that vanishes mid-stream is never noticed.
+// The upstream reply is already buffered until it finishes. This wait applies
+// only while that buffer is flushed: a slow Codex exerts backpressure, and a
+// client that vanishes is noticed instead of being written forever.
 function drained(output) {
   return new Promise((resolve, reject) => {
     const settle = (fn, value) => {
@@ -100,24 +105,68 @@ function drained(output) {
   });
 }
 
-async function writeBlock(output, block) {
-  if (output.destroyed || output.writableEnded)
-    throw Object.assign(new Error("The client closed the stream"), {
-      code: "ERR_STREAM_PREMATURE_CLOSE",
-    });
-  if (!output.write(block)) await drained(output);
-}
+// First try, then two more. That is the budget Codex used to spend on
+// request_max_retries / stream_max_retries. A retry sends the prompt again, so
+// the input tokens for that attempt can be billed again.
+export const PROXY_ATTEMPTS = 3;
 
-// Nothing has reached Codex until the first SSE block is written, so a stream
-// that dies before it opens is safe to send again. After that it never is:
-// re-sending would duplicate items Codex has already recorded.
+// DNS and connect failures are included. A socket reset (ECONNRESET, EPIPE,
+// UND_ERR_SOCKET) or a body that ends before response.completed is included
+// only while Codex has not been sent a byte. User aborts and deterministic
+// 422s are not in this set.
 const RETRYABLE = new Set([
   BRIDGE_ERROR.DNS,
   BRIDGE_ERROR.CONNECT,
   BRIDGE_ERROR.UPSTREAM_CLOSED,
 ]);
 
-export async function openProxyStreamWithRetry(options, attempts = 2) {
+// response.incomplete is the other usage event this code already treats as the
+// end of a response. response.failed here is a finished upstream event, not a
+// dropped socket, so it is forwarded instead of retried.
+const TERMINAL_EVENTS = new Set([
+  "response.completed",
+  "response.incomplete",
+  "response.failed",
+]);
+
+export function isTerminalSseBlock(part) {
+  for (const line of part.split("\n")) {
+    if (line.startsWith("event:") && TERMINAL_EVENTS.has(line.slice(6).trim()))
+      return true;
+    if (!line.startsWith("data:")) continue;
+    const payload = line.slice(5).trim();
+    if (!payload || payload === "[DONE]") continue;
+    try {
+      const value = JSON.parse(payload);
+      if (TERMINAL_EVENTS.has(value?.type)) return true;
+    } catch {
+      // A non-JSON data line cannot be the terminal event.
+    }
+  }
+  return false;
+}
+
+function prematureUpstreamClose() {
+  return Object.assign(new Error("Grok closed the connection before finishing"), {
+    code: "ERR_UPSTREAM_PREMATURE_CLOSE",
+  });
+}
+
+function clientClosed() {
+  return Object.assign(new Error("The client closed the stream"), {
+    code: "ERR_STREAM_PREMATURE_CLOSE",
+  });
+}
+
+// Returns the retry kind, or null when this failure must surface as-is.
+function retryKind(error, options, attempt, rounds, clientBytes) {
+  if (clientBytes > 0 || options.signal?.aborted || attempt === rounds)
+    return null;
+  const kind = classifyBridgeError(error);
+  return RETRYABLE.has(kind) ? kind : null;
+}
+
+export async function openProxyStreamWithRetry(options, attempts = PROXY_ATTEMPTS) {
   const rounds = Math.max(1, attempts);
   let lastError;
   for (let attempt = 1; attempt <= rounds; attempt += 1) {
@@ -125,46 +174,113 @@ export async function openProxyStreamWithRetry(options, attempts = 2) {
       return await openProxyStream(options);
     } catch (error) {
       lastError = error;
-      const kind = classifyBridgeError(error);
-      if (
-        !RETRYABLE.has(kind) ||
-        options.signal?.aborted ||
-        attempt === rounds
-      )
-        break;
+      const kind = retryKind(error, options, attempt, rounds, 0);
+      if (!kind) break;
       options.onRetry?.({ attempt, kind });
     }
   }
   throw lastError;
 }
 
-export async function pipeProxySse(stream, output, map) {
+function completeBlocks(buffer) {
+  const parts = buffer.split("\n\n");
+  const rest = parts.pop() ?? "";
+  return { parts: parts.filter((part) => part.trim()), rest };
+}
+
+// Read the upstream body to memory. Write to Codex only after a terminal
+// event, so a reset before that has not shown Codex a partial reply.
+export async function pipeProxySse(stream, output, map, usageBox, onClientByte) {
   const reader = stream.getReader();
   const decoder = new TextDecoder();
   let buffer = "";
   const rewrite = createSseRewriter(map);
+  const pending = [];
+  let clientBytes = 0;
+  let finished = false;
+  let failed = false;
+
+  const flush = async (parts) => {
+    for (const part of parts) {
+      const rewritten = rewrite(part);
+      if (rewritten === null) continue;
+      const block = `${rewritten}\n\n`;
+      if (output.destroyed || output.writableEnded) throw clientClosed();
+      // Count the byte as soon as the client stream accepts it. A later drain
+      // error must not look like "Codex saw nothing" and resubmit the prompt.
+      const accepted = output.write(block);
+      clientBytes += Buffer.byteLength(block);
+      onClientByte?.();
+      if (!accepted) await drained(output);
+    }
+  };
+
+  const accept = async (parts) => {
+    if (!parts.length) return;
+    if (finished) {
+      await flush(parts);
+      return;
+    }
+    const terminalAt = parts.findIndex((part) => isTerminalSseBlock(part));
+    if (terminalAt === -1) {
+      pending.push(...parts);
+      return;
+    }
+    pending.push(...parts.slice(0, terminalAt + 1));
+    await flush(pending);
+    pending.length = 0;
+    finished = true;
+    await flush(parts.slice(terminalAt + 1));
+  };
+
   try {
     while (true) {
+      if (output.destroyed || output.writableEnded) throw clientClosed();
       const { done, value } = await reader.read();
       if (done) break;
       buffer += decoder.decode(value, { stream: true });
-      const parts = buffer.split("\n\n");
-      buffer = parts.pop();
-      for (const part of parts) {
-        if (!part.trim()) continue;
-        const rewritten = rewrite(part);
-        if (rewritten !== null) await writeBlock(output, rewritten + "\n\n");
-      }
+      const split = completeBlocks(buffer);
+      buffer = split.rest;
+      await accept(split.parts);
     }
     buffer += decoder.decode();
-    if (buffer.trim()) {
-      const rewritten = rewrite(buffer);
-      if (rewritten !== null) await writeBlock(output, rewritten + "\n\n");
-    }
+    if (buffer.trim()) await accept([buffer]);
+    if (!finished) throw prematureUpstreamClose();
   } catch (error) {
-    // Stop pulling from Grok the moment the client is gone; leaving the body
+    failed = true;
+    // Stop pulling from Grok the moment this attempt is over; leaving the body
     // unread holds the upstream socket open for the rest of the response.
     await reader.cancel(error).catch(() => {});
+    if (error && typeof error === "object") error.clientBytes = clientBytes;
     throw error;
+  } finally {
+    if (!failed) reader.releaseLock();
+    if (usageBox) usageBox.cacheUsage = rewrite.cacheUsage ?? null;
   }
 }
+
+// Open and read one attempt. If the socket dies before Codex has a byte, send
+// the same request again. Once a byte has been written, never resubmit.
+export async function relayProxySse(options, output, map, usageBox, attempts = PROXY_ATTEMPTS) {
+  const rounds = Math.max(1, attempts);
+  let lastError;
+  for (let attempt = 1; attempt <= rounds; attempt += 1) {
+    try {
+      const response = await openProxyStream(options);
+      await pipeProxySse(response.body, output, map, usageBox, options.onClientByte);
+      return response;
+    } catch (error) {
+      lastError = error;
+      const kind = retryKind(
+        error,
+        options,
+        attempt,
+        rounds,
+        error?.clientBytes ?? 0,
+      );
+      if (!kind) break;
+      options.onRetry?.({ attempt, kind });
+    }
+  }
+  throw lastError;
+}

package/src/runtime.mjs

@@ -29,14 +29,11 @@ export async function startRuntime(options = {}) {
     env_key: "CODEX_GROK_BRIDGE_TOKEN",
     wire_api: "responses",
     requires_openai_auth: false,
-    // The upstream resets a response mid-stream now and then — measured three
-    // times as ECONNRESET at 25s, 27s and 253s, on requests from 726 KB to
-    // 22 MB. The bridge cannot retry that: Codex has already received part of
-    // the answer, so re-sending would duplicate it. Codex can, because it owns
-    // the conversation and re-sends the same request. These were both 0, which
-    // turned every such reset into a dead turn.
-    request_max_retries: 2,
-    stream_max_retries: 2,
+    // The bridge buffers the upstream reply and retries a reset itself, up to
+    // two more attempts, before Codex has seen any SSE byte. Codex must not
+    // also resend the full prompt.
+    request_max_retries: 0,
+    stream_max_retries: 0,
     // Do not inherit whatever Codex's default happens to be across upgrades.
     stream_idle_timeout_ms: 300000,
   };

package/src/tools.mjs

@@ -5,6 +5,13 @@ import {
   saveGeneratedImage,
 } from "./imagegen.mjs";
 import { DEFAULT_GROK_MODEL, resolveGrokModel } from "./models.mjs";
+import {
+  applyCacheUsage,
+  logicalFingerprint,
+  readCacheUsage,
+  stableJsonValue,
+  stableProxyName,
+} from "./prefix.mjs";
 
 const OBJECT_SCHEMA = { type: "object", properties: {} };
 
@@ -227,7 +234,14 @@ function decodeCustomInput(argumentsValue) {
 }
 
 function addFunction(flattened, map, spec) {
-  const proxyName = `codex_${flattened.length}_${sanitize(spec.name)}`;
+  // Index names change when Codex reorders tools, which rewrites every
+  // earlier function_call and breaks the prompt-cache prefix.
+  let proxyName = stableProxyName(spec, sanitize(spec.name));
+  if (map.has(proxyName)) {
+    let n = 2;
+    while (map.has(`${proxyName}_${n}`)) n += 1;
+    proxyName = `${proxyName}_${n}`;
+  }
   map.set(proxyName, {
     kind: spec.kind,
     namespace: spec.namespace,
@@ -239,10 +253,11 @@ function addFunction(flattened, map, spec) {
     description: spec.namespace
       ? `[${spec.namespace}] ${spec.description || spec.name}`
       : spec.description || spec.name,
-    parameters:
+    parameters: stableJsonValue(
       spec.kind === "custom"
         ? structuredClone(CUSTOM_INPUT_SCHEMA)
         : usableParameters(spec.parameters),
+    ),
   });
 }
 
@@ -284,7 +299,7 @@ export function flattenCodexTools(tools = []) {
       }
       continue;
     }
-    flattened.push(structuredClone(tool));
+    flattened.push(stableJsonValue(structuredClone(tool)));
   }
   return { tools: flattened, map };
 }
@@ -392,17 +407,36 @@ function toProxyInputNode(node, map, state) {
   return whitelistInputNode(next);
 }
 
-function toProxyInput(input, map) {
-  const items = toProxyInputNode(input, map, { callIds: new Map() });
-  if (!Array.isArray(items)) return items;
-  return items.filter(
-    (item) =>
-      item &&
-      typeof item === "object" &&
-      (item.type == null || GROK_INPUT_ITEM_TYPES.has(item.type)),
+function isForwardedItem(item) {
+  return (
+    item &&
+    item !== DROP &&
+    typeof item === "object" &&
+    !Array.isArray(item) &&
+    (item.type == null || GROK_INPUT_ITEM_TYPES.has(item.type))
   );
 }
 
+function projectInput(input, map) {
+  if (!Array.isArray(input)) {
+    const items = toProxyInputNode(input, map, { callIds: new Map() });
+    return { items, pairs: [] };
+  }
+  const state = { callIds: new Map() };
+  const pairs = [];
+  for (const item of input) {
+    const fingerprint = logicalFingerprint(item);
+    const next = toProxyInputNode(item, map, state);
+    if (!isForwardedItem(next)) continue;
+    pairs.push({ fingerprint, item: next });
+  }
+  return { items: pairs.map((pair) => pair.item), pairs };
+}
+
+function toProxyInput(input, map) {
+  return projectInput(input, map).items;
+}
+
 // Only the bridge knows a request is being served by the bridge. Codex does not
 // put the provider or model into the prompt, so without this line the model has
 // no way to answer "is Grok actually attached?" and either hedges or goes
@@ -432,9 +466,10 @@ export const IMAGE_GENERATION_PROVENANCE =
 export function toProxyRequest(body) {
   const { tools, map } = flattenCodexTools(body.tools ?? []);
   const effort = EFFORT[body.reasoning?.effort] ?? "high";
+  const projected = projectInput(body.input, map);
   const request = {
     model: resolveGrokModel(body.model),
-    input: toProxyInput(body.input, map),
+    input: projected.items,
     tools,
     reasoning: { effort },
     stream: true,
@@ -463,7 +498,7 @@ export function toProxyRequest(body) {
       : provenance;
   if (typeof body.prompt_cache_key === "string" && body.prompt_cache_key)
     request.prompt_cache_key = body.prompt_cache_key;
-  return { request, map };
+  return { request, map, projected: projected.pairs };
 }
 
 function rememberProxyItem(item, origin, state) {
@@ -508,8 +543,19 @@ function absorbGeneratedImage(item, state) {
   };
 }
 
+const USAGE_EVENTS = new Set(["response.completed", "response.incomplete"]);
+
 function rewriteResponseEvent(value, map, state) {
   if (!value || typeof value !== "object") return value;
+  if (
+    USAGE_EVENTS.has(value.type) &&
+    value.response &&
+    typeof value.response.usage === "object"
+  ) {
+    applyCacheUsage(value.response.usage);
+    const cache = readCacheUsage(value.response.usage);
+    if (cache) state.cacheUsage = cache;
+  }
   if (
     typeof value.item === "object" &&
     value.item &&
@@ -569,6 +615,13 @@ export function createSseRewriter(map, options = {}) {
     callIds: new Map(),
     itemIds: new Map(),
     imageOptions: options.imageOptions,
+    cacheUsage: null,
   };
-  return (block) => rewriteSseBlock(block, map, state);
+  const rewrite = (block) => rewriteSseBlock(block, map, state);
+  Object.defineProperty(rewrite, "cacheUsage", {
+    get() {
+      return state.cacheUsage;
+    },
+  });
+  return rewrite;
 }