@timo972/cc-router 0.9.0 → 0.10.0-rc.0

package/CHANGELOG.md

@@ -6,9 +6,95 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ---
 
-## [Unreleased]
+## [0.10.0] — 2026-08-17
 
-Nothing yet.
+### Added
+
+- OpenAI/Codex sticky session routing: sessions pin to one account for prompt-cache
+  locality (`session_id` → `x-claude-code-session-id` → `prompt_cache_key`), with
+  load- and headroom-aware selection for new sessions.
+- Codex usage tracking from `x-codex-*` response headers: default 5h/weekly windows
+  plus dynamically discovered model-scoped metered buckets, credits, and plan.
+- Scoped cooldowns on upstream failures: bucket-scoped via `x-codex-active-limit`,
+  account-global otherwise; local 429/503 responses when no account is eligible.
+- Dashboard: OpenAI accounts now show 5h/weekly bars, per-bucket rows, credits,
+  plan, request/error/in-flight/session counts, and cooldown state.
+
+### Changed
+
+- OpenAI account records persist `scopes`, `sessionLimitPercent`, and
+  `weeklyLimitPercent`.
+- The stateless OpenAI round-robin picker was removed in favor of
+  `OpenAITokenPool`.
+
+### Fixed
+
+- An unexpected failure partway through an OpenAI request — an upstream
+  connection error, a rejected token refresh, a mid-stream abort — no longer
+  takes down the proxy. Both `/v1/responses` and the `/v1/messages` OpenAI
+  branch awaited the upstream call without catching a rejection, so a single
+  network blip could kill the daemon and lose every account's routing state.
+- `/v1/messages` no longer reports an upstream OpenAI failure as a success. A
+  stream ending in `response.failed`, an `error` event, a JSON error body, or
+  no completion event at all — a stream that stopped mid-flight, or an
+  event-stream response with no body — was translated into an empty Anthropic
+  message with HTTP 200; each now surfaces as an error response, so a rate
+  limit reads as a rate limit instead of an empty assistant turn. A non-2xx
+  upstream response (401, 429, 5xx) is now relayed with its real status, error
+  message, and safe headers — `Retry-After` included, so a client can honor the
+  backoff the server asked for — instead of being parsed as an event stream and
+  reported as a success or a generic failure; non-2xx Codex responses also keep
+  their real content type instead of being rewritten to `text/event-stream`.
+- A terminal event that carries no response object is no longer treated as a
+  successful result. `{"type":"response.completed","response":null}` — or any
+  other non-object payload — satisfied the completion check, so a
+  non-streaming request got HTTP 200 with a `null` body, and `/v1/messages` got
+  a fabricated empty assistant turn; a streamed `/v1/messages` turn was closed
+  with `message_stop` and `end_turn`, telling the client a truncated answer had
+  finished normally. The collected paths now report the `502` that a stream
+  ending without a terminal event already did, and the streamed path ends
+  without `message_stop`, which is what clients already surface as a
+  truncation.
+- A client that disconnects mid-response no longer leaves the upstream Codex
+  request running. Nothing propagated the disconnect, so the relay drained the
+  whole upstream body into a closed socket and held that connection open for a
+  response nobody would receive; the request is now cancelled as soon as the
+  client goes away.
+- A single malformed SSE frame no longer truncates a `/v1/messages` stream.
+  Parsing a chunk was all-or-nothing, so one bad frame discarded the valid
+  events beside it and ended the response as a clean `200` the client could
+  not tell apart from a complete answer.
+- OpenAI credentials are written back to the accounts file the proxy was
+  started with. Under `--accounts <path>` accounts were read from that file
+  but every refresh, add, delete, and update wrote the default
+  `accounts.json` — discarding the change and copying OAuth tokens into an
+  unrelated file.
+- OpenAI token refresh survives a malformed token response. A payload missing
+  `expires_in` produced a `NaN` expiry that read as "never needs refreshing",
+  so the account kept presenting a stale token indefinitely — as did a lifetime
+  large enough to overflow into an infinite expiry, while a zero or negative
+  one reported success on a token that was already due for another refresh.
+  Each is now treated as the failed refresh it is; a failure to
+  persist rotated credentials no longer fails the request that triggered the
+  refresh, and the write is now retried on subsequent requests (and the
+  background refresh loop) until it succeeds, so a rotated refresh token
+  still reaches disk.
+- `PATCH /cc-router/accounts/:id` works for OpenAI accounts instead of
+  returning `404`, so a single OpenAI account can be enabled, disabled, or
+  capped without toggling the whole provider. `POST /cc-router/accounts` now
+  rejects an out-of-range percentage cap the same way `PATCH` does, rather
+  than silently coercing it.
+- A Codex response that ends as `response.incomplete` — e.g. hitting the
+  output-token ceiling — is now delivered with its partial content and token
+  usage instead of being discarded. `/v1/responses` treated only
+  `response.completed` as a terminal event, so `response.incomplete` looked
+  identical to a stream that stopped mid-flight and turned a usable partial
+  answer into a `502 upstream_error`. A streamed `/v1/messages` turn that ends
+  incomplete now closes properly too — the Anthropic translation emitted no
+  `message_stop` for it, leaving the client waiting on a turn that was already
+  over. Both `/v1/messages` paths, streamed or collected, now report
+  `max_tokens` as the stop reason when the output-token ceiling was the cause,
+  instead of an `end_turn` that made a truncated answer look deliberate.
 
 ---
 

package/README.md

@@ -273,6 +273,8 @@ See [docs/litellm-setup.md](docs/litellm-setup.md) for details.
 
 CC-Router exposes an OpenAI Responses-compatible endpoint for Codex CLI at `/v1/responses`. This lets Codex use OpenAI ChatGPT/Codex subscription accounts through the same local router that Claude Code uses for Claude subscriptions.
 
+**Features:** Sticky sessions pin each Codex conversation to one account for prompt-cache locality. Load- and headroom-aware account selection spreads new sessions across available capacity. Usage tracking from response headers reports account-level 5-hour and 7-day windows, dynamically discovered model-scoped metered buckets, credits, and plan. User caps (`sessionLimitPercent`/`weeklyLimitPercent`) apply to the default Codex bucket. The dashboard shows per-bucket rows, usage bars, credits, plan, and cooldown state for OpenAI accounts.
+
 Configure Codex:
 
 ```bash

package/dist/config/manager.js

@@ -137,11 +137,17 @@ export function loadOpenAIAccounts(path) {
         refreshToken: a.refreshToken,
         expiresAt: a.expiresAt,
         enabled: a.enabled !== false,
+        ...(Array.isArray(a.scopes) ? { scopes: a.scopes } : {}),
+        ...(a.sessionLimitPercent !== undefined ? { sessionLimitPercent: a.sessionLimitPercent } : {}),
+        ...(a.weeklyLimitPercent !== undefined ? { weeklyLimitPercent: a.weeklyLimitPercent } : {}),
     }));
 }
-export function saveOpenAIAccounts(accounts) {
+/** Persist OpenAI subscription accounts to an explicit accounts file, preserving
+ *  every other provider's records already in that file. Shared by `saveOpenAIAccounts`
+ *  (default path) and any caller bound to a custom `--accounts <path>`. */
+export function saveOpenAIAccountsToPath(accounts, path) {
     ensureConfigDir();
-    const existing = readAccountsRaw();
+    const existing = readRawFromPath(path);
     const nonOpenAI = existing.filter(a => a.provider !== "openai_subscription");
     const records = accounts.map(a => ({
         id: a.id,
@@ -149,10 +155,15 @@ export function saveOpenAIAccounts(accounts) {
         accessToken: a.accessToken,
         refreshToken: a.refreshToken,
         expiresAt: a.expiresAt,
-        scopes: ["openid", "profile", "email", "offline_access"],
+        scopes: a.scopes ?? ["openid", "profile", "email", "offline_access"],
         enabled: a.enabled,
+        ...(a.sessionLimitPercent !== undefined ? { sessionLimitPercent: a.sessionLimitPercent } : {}),
+        ...(a.weeklyLimitPercent !== undefined ? { weeklyLimitPercent: a.weeklyLimitPercent } : {}),
     }));
-    writeAccountsAtomicToPath(ACCOUNTS_PATH, [...nonOpenAI, ...records]);
+    writeAccountsAtomicToPath(path, [...nonOpenAI, ...records]);
+}
+export function saveOpenAIAccounts(accounts) {
+    saveOpenAIAccountsToPath(accounts, ACCOUNTS_PATH);
 }
 function parseProxyConfig(raw) {
     const parsed = JSON.parse(raw);

package/dist/protocol/openai-response-to-anthropic.js

@@ -1,3 +1,18 @@
+/**
+ * Anthropic stop reason for a terminal Responses payload. Shared by both
+ * translation paths — this module for a collected response, and the streaming
+ * normalizer for a terminal SSE event — so a turn that ends the same way is
+ * reported the same way whether or not the client asked for a stream.
+ *
+ * Keys off `incomplete_details` rather than the event type or `status`: a
+ * completed response carries none, so the same call is correct for both, and
+ * the output-token ceiling is the one reason that maps onto an Anthropic stop
+ * reason of its own. Any other incomplete reason still delivered content, so
+ * `end_turn` stays the honest default.
+ */
+export function anthropicStopReasonForResponse(response) {
+    return response?.incomplete_details?.reason === "max_output_tokens" ? "max_tokens" : "end_turn";
+}
 export function openAIResponseToAnthropicMessage(response) {
     const content = (response.output ?? [])
         .filter(item => item.type === "message")
@@ -10,7 +25,7 @@ export function openAIResponseToAnthropicMessage(response) {
         role: "assistant",
         model: response.model ?? "",
         content,
-        stop_reason: "end_turn",
+        stop_reason: anthropicStopReasonForResponse(response),
         stop_sequence: null,
         usage: {
             input_tokens: response.usage?.input_tokens ?? 0,

package/dist/protocol/openai-responses-collect.js

@@ -2,14 +2,81 @@ import { parseSseLines } from "./sse.js";
 function upstreamError(message) {
     return { kind: "json", status: 502, body: { error: { type: "upstream_error", message } } };
 }
+/**
+ * SSE event types that represent a Responses stream reaching a terminal
+ * *result*, as opposed to a transport/backend failure.
+ *
+ * `response.completed` is the ordinary success terminal event.
+ * `response.incomplete` is also terminal: Codex/OpenAI emit it when
+ * generation stops without completing — most commonly hitting
+ * `max_output_tokens`, or a content filter — but the event still carries a
+ * full response object with `usage` and `incomplete_details.reason`. It is a
+ * *result* to relay, not an error, so it belongs here rather than alongside
+ * `response.failed`.
+ *
+ * `response.failed` and the bare `error` event are failures, not results:
+ * they carry no usable response body and are handled separately by every
+ * caller below.
+ */
+const TERMINAL_RESPONSE_EVENT_TYPES = new Set([
+    "response.completed",
+    "response.incomplete",
+]);
+/**
+ * Returns the `.response` payload carried by a terminal Responses SSE event
+ * (`response.completed` or `response.incomplete`), or `undefined` for any
+ * other event — including `response.failed`/`error`, which are failures and
+ * carry no usable response to return. Shared by every ingress that needs to
+ * recognize "the stream produced a result", so that notion cannot drift
+ * apart between the `/v1/responses` and `/v1/messages` paths.
+ *
+ * The event type alone does not make a result: it has to carry an actual
+ * response object. Upstream can emit `{"type":"response.incomplete",
+ * "response":null}` — or a string, a number, an array — and every consumer
+ * here asks whether the payload is `undefined`, so anything else would count
+ * as a terminal success. That would hand a `200` with a `null` body to a
+ * non-streaming caller and mark an observed stream complete, which is exactly
+ * what the terminal-event checks exist to prevent. A payload nothing can be
+ * read out of is not a result.
+ *
+ * `{}` is the same problem wearing an object's clothes: it satisfies a bare
+ * typeof check and then produces a `200` whose body is `{}`, or an empty
+ * assistant turn on the Messages path. `id` is the field that separates a
+ * Responses object from an empty husk — upstream stamps it from
+ * `response.created` onward — so requiring it is what makes "is this an
+ * object" mean "is this a response".
+ */
+export function terminalResponsePayload(event) {
+    if (typeof event !== "object" || event === null)
+        return undefined;
+    const typed = event;
+    if (typeof typed.type !== "string" || !TERMINAL_RESPONSE_EVENT_TYPES.has(typed.type)) {
+        return undefined;
+    }
+    const payload = typed.response;
+    if (typeof payload !== "object" || payload === null || Array.isArray(payload))
+        return undefined;
+    const { id } = payload;
+    if (typeof id !== "string" || id.length === 0)
+        return undefined;
+    return payload;
+}
 /**
  * Collapse the Codex backend's forced SSE stream into a single Responses
  * object for callers that did not ask to stream. The backend's terminal
- * `response.completed` payload is returned verbatim, preserving tool calls,
- * reasoning, and usage.
+ * `response.completed` or `response.incomplete` payload is returned
+ * verbatim, preserving tool calls, reasoning, and usage — including for a
+ * response that stopped early (e.g. hitting the output-token ceiling), which
+ * is a usable partial answer, not a transport failure.
  */
-export async function collectCodexResponseStream(upstream) {
+export async function collectCodexResponseStream(upstream, 
+/** Invoked the moment upstream announces a failure, so the caller keeps that
+ *  verdict even when the read is later cut short — the catch below turns any
+ *  such interruption into a generic "malformed stream" and would otherwise
+ *  bury it. */
+onUpstreamFailure) {
     if (!upstream.ok) {
+        onUpstreamFailure?.();
         const contentType = upstream.headers.get("content-type") ?? undefined;
         return { kind: "text", status: upstream.status, contentType, body: await upstream.text() };
     }
@@ -27,21 +94,25 @@ export async function collectCodexResponseStream(upstream) {
         return upstreamError("Empty upstream body");
     const decoder = new TextDecoder();
     let remainder = "";
-    let completed;
+    let terminalResponse;
     let failure;
     const applyEvent = (event) => {
+        const payload = terminalResponsePayload(event);
+        if (payload !== undefined) {
+            terminalResponse = payload;
+            return;
+        }
         if (typeof event !== "object" || event === null)
             return;
         const e = event;
-        if (e.type === "response.completed") {
-            completed = e.response;
-        }
-        else if (e.type === "response.failed") {
+        if (e.type === "response.failed") {
             const err = e.response?.error;
             failure = err?.message ?? "Response failed";
+            onUpstreamFailure?.();
         }
         else if (e.type === "error") {
             failure = e.error?.message ?? "Upstream error event";
+            onUpstreamFailure?.();
         }
     };
     try {
@@ -65,7 +136,115 @@ export async function collectCodexResponseStream(upstream) {
     }
     if (failure !== undefined)
         return upstreamError(failure);
-    if (completed === undefined)
-        return upstreamError("Stream ended before response.completed");
-    return { kind: "json", status: upstream.status, body: completed };
+    if (terminalResponse === undefined)
+        return upstreamError("Stream ended before any terminal response event");
+    return { kind: "json", status: upstream.status, body: terminalResponse };
+}
+function usageNumber(value) {
+    return typeof value === "number" && Number.isFinite(value) && value > 0 ? Math.floor(value) : 0;
+}
+/**
+ * Extract usage totals from a response-shaped object (i.e. something with a
+ * `.usage` field directly — the Responses `response.completed`/
+ * `response.incomplete` payload, or an object wrapping one). Shared by every
+ * ingress that needs to report Codex token usage from a fully-materialized
+ * body.
+ */
+export function usageFromResponseBody(body) {
+    if (typeof body !== "object" || body === null)
+        return undefined;
+    const usage = body.usage;
+    if (usage === undefined || usage === null || typeof usage !== "object")
+        return undefined;
+    return {
+        inputTokens: usageNumber(usage.input_tokens),
+        cachedInputTokens: usageNumber(usage.input_tokens_details?.cached_tokens),
+        outputTokens: usageNumber(usage.output_tokens),
+    };
+}
+/**
+ * Extract usage totals from a terminal Responses SSE event —
+ * `response.completed` or `response.incomplete`, see
+ * `TERMINAL_RESPONSE_EVENT_TYPES` above — or `undefined` for any other
+ * event. Single definition shared by every streaming ingress so
+ * `/v1/responses` and `/v1/messages` can never report different token totals
+ * for the same stream.
+ */
+export function usageFromTerminalEvent(event) {
+    return usageFromResponseBody(terminalResponsePayload(event));
+}
+/**
+ * Passive usage reader for the byte-transparent streaming path: it only
+ * observes chunks that are already being piped downstream unchanged. It also
+ * watches for a `response.failed`/`error` event — the same terminal-failure
+ * signal `collectCodexResponseStream` above already detects for the
+ * non-streaming path — so a stream that upstream answered with `200` but
+ * ended in failure can still be reported (for stats/activity only) as the
+ * failure it was, without altering a single byte written to the client.
+ */
+export function createCodexUsageObserver() {
+    const decoder = new TextDecoder();
+    let remainder = "";
+    let totals;
+    let failure;
+    let completed = false;
+    const applyEvent = (event) => {
+        totals = usageFromTerminalEvent(event) ?? totals;
+        if (typeof event !== "object" || event === null)
+            return;
+        const e = event;
+        if (e.type === "response.failed") {
+            const err = e.response?.error;
+            failure = err?.message ?? "Response failed";
+        }
+        else if (e.type === "error") {
+            failure = e.error?.message ?? "Upstream error event";
+        }
+        else if (terminalResponsePayload(event) !== undefined) {
+            completed = true;
+        }
+    };
+    return {
+        push(chunk) {
+            // Best-effort: a malformed SSE frame from upstream must never throw
+            // here. This observer only watches bytes that are already being
+            // relayed to the client verbatim — a parse failure just means that one
+            // frame goes uncaptured, never that the response breaks. Tolerant
+            // parsing keeps the rest of the chunk's valid events.
+            try {
+                const parsed = parseSseLines(remainder + decoder.decode(chunk, { stream: true }), { tolerant: true });
+                remainder = parsed.remainder;
+                parsed.events.forEach(applyEvent);
+            }
+            catch {
+                // swallow — passive observer, see comment above
+            }
+        },
+        finish() {
+            try {
+                const tail = decoder.decode();
+                if (tail || remainder) {
+                    parseSseLines(remainder + tail + "\n", { tolerant: true }).events.forEach(applyEvent);
+                }
+            }
+            catch {
+                // swallow — passive observer, see comment above
+            }
+            remainder = "";
+            return totals;
+        },
+        failure() {
+            // Tolerant parsing drops a malformed frame instead of aborting, which
+            // also means a malformed *terminal* response event (`response.completed`
+            // or `response.incomplete`) frame vanishes silently. Without an
+            // observed terminal event the stream never actually produced a result,
+            // so — mirroring collectCodexResponseStream's non-streaming check —
+            // that is reported as a failure too, unless an explicit
+            // response.failed/error already said more about what went wrong.
+            return failure ?? (completed ? undefined : "Upstream stream ended before any terminal response event");
+        },
+        explicitFailure() {
+            return failure;
+        },
+    };
 }

package/dist/protocol/openai-stream-to-anthropic.js

@@ -1,3 +1,5 @@
+import { anthropicStopReasonForResponse } from "./openai-response-to-anthropic.js";
+import { terminalResponsePayload } from "./openai-responses-collect.js";
 export function createOpenAIStreamToAnthropicNormalizer() {
     let textBlockStarted = false;
     const ensureTextBlockStarted = () => {
@@ -46,7 +48,21 @@ export function createOpenAIStreamToAnthropicNormalizer() {
                     },
                 ];
             }
-            if (event.type === "response.completed") {
+            // Both terminal Responses events must close the Anthropic message.
+            // Emitting nothing for `response.incomplete` would end the HTTP stream
+            // without `message_stop`, leaving the client waiting on a turn that is
+            // already over.
+            //
+            // The same predicate the collector and the usage observer use, not the
+            // event type alone: a terminal frame carrying no response object
+            // (`"response":null`, an array, a string) is not a result, and those two
+            // already treat it as a failed stream. Closing the message here anyway
+            // would emit `stop_reason: end_turn` — telling the client a truncated
+            // turn ended normally, the one outcome worse than a truncated stream.
+            // Emitting nothing ends the body without `message_stop`, which is what
+            // a stream that never reached a terminal event looks like, and what
+            // clients already detect and surface as an error.
+            if (terminalResponsePayload(event) !== undefined) {
                 const usage = event.response?.usage ?? {};
                 const prefix = textBlockStarted
                     ? [{ type: "content_block_stop", index: 0 }]
@@ -56,7 +72,9 @@ export function createOpenAIStreamToAnthropicNormalizer() {
                     ...prefix,
                     {
                         type: "message_delta",
-                        delta: { stop_reason: "end_turn", stop_sequence: null },
+                        // Same helper the collected-response translator uses, so an
+                        // incomplete turn reports the same stop reason on both paths.
+                        delta: { stop_reason: anthropicStopReasonForResponse(event.response), stop_sequence: null },
                         usage: { output_tokens: usage.output_tokens ?? 0 },
                     },
                     { type: "message_stop" },

package/dist/protocol/sse.js

@@ -1,4 +1,4 @@
-export function parseSseLines(input) {
+export function parseSseLines(input, options = {}) {
     const lines = input.split("\n");
     const remainder = lines.pop() ?? "";
     const events = [];
@@ -8,6 +8,15 @@ export function parseSseLines(input) {
         const payload = line.slice(6).trim();
         if (!payload || payload === "[DONE]")
             continue;
+        if (options.tolerant) {
+            try {
+                events.push(JSON.parse(payload));
+            }
+            catch {
+                // Advisory parse — drop just this frame, keep the rest of the chunk.
+            }
+            continue;
+        }
         events.push(JSON.parse(payload));
     }
     return { events, remainder };