@timo972/cc-router 0.10.0 → 0.11.0-rc.0

package/CHANGELOG.md

@@ -10,6 +10,138 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Added
 
+- Automatic upstream failover and retry on both providers. A 429 or 5xx
+  received before any response byte is relayed no longer passes straight
+  through to the client: the router applies the existing cooldown/affinity
+  bookkeeping and retries the request itself, up to 3 upstream attempts
+  per request. A 429 (or an overload the provider cools down: Anthropic
+  529; Codex 503/529) always fails over to a *different* account; any
+  other 5xx keeps a session-bound request on its own account, retrying
+  after a short pause, while a session-less request re-routes the way a
+  fresh request would.
+  Covers Claude `/v1/messages`, Codex `/v1/responses`, and cross-routed
+  `/v1/messages`. When nothing is eligible or the budget is exhausted, the
+  last failed upstream response is relayed unchanged, exactly as before;
+  401s still pass through with a background token refresh, and the router
+  still never retries after response bytes have started. Failed attempts
+  show up in the activity log with a `:will-retry` suffix. On by
+  default — set `"autoFailover": false` in `~/.cc-router/config.json`
+  (restart required) to opt out and restore pure pass-through behavior.
+  Note that current Claude Code builds no longer retry 429s themselves,
+  so with failover off a rate limit surfaces directly in the session.
+
+### Changed
+
+- Claude-bound POST `/v1/messages` moved from the generic proxy middleware
+  to a dedicated transport (same byte-transparent relay contract: verbatim
+  status/headers, raw body bytes, no synthesized events) so the router can
+  decide to retry at upstream response headers. Every other `/v1` endpoint
+  stays on the generic proxy. Claude activity rows now record the requested
+  model and the full `/v1/messages` path.
+
+### Fixed
+
+- An account whose quota refills early — upgrading a Claude plan being the
+  common case — is returned to rotation as soon as the usage endpoint says
+  so, instead of staying benched for the rest of the pre-upgrade window. A
+  429 records a cooldown whose expiry comes from the reset timestamps on
+  that response, and both that cooldown and the header-derived
+  `rate_limited` flag were released only by the wall clock. Nothing
+  reconnected them to the usage refresher, so a plan upgrade produced an
+  account reporting `0%` on every window, `usage fresh`, and `busy` with a
+  multi-hour cooldown — and because it was benched, no new response could
+  ever arrive to correct it. In the reported case one stale cooldown on the
+  only account with capacity left the whole pool answering
+  `429 no-eligible`.
+
+  A usage snapshot now supersedes both blockers, under two conditions that
+  keep it from unbenching an account that is still limited.
+
+  The refresh must have been *initiated* after the block was recorded.
+  `fetchedAt` cannot answer that — it is stamped after the response body is
+  parsed, so a refresh already on the wire when a 429 lands completes
+  afterwards while describing the account as it was before. Neither can
+  wall-clock milliseconds: the 429, the headers taken from it, and the
+  refresh the router starts in response all happen in one event-loop turn
+  and read the same millisecond (measured at 199 ties in 200 runs), which
+  would have made that immediate refresh useless. Ordering now runs on a
+  process-wide monotonic sequence, with tokens on the usage snapshot, the
+  header snapshot, and each cooldown entry.
+
+  And the snapshot must report on the scope that caused the block: only the
+  claimed window releases a global cooldown, and only the matching family
+  releases a model cooldown. Blocks for limits no snapshot describes — an
+  upstream 529 overload, the `seven_day_oauth_apps` quota, an unattributed
+  claim — stay purely time-based. Cooldowns are grouped by scope — global by
+  limiting window, model by family — and within each scope every expiry keeps
+  the sequence of the event that produced it, so overlapping blocks neither
+  merge nor cancel each other: releasing a quota cooldown leaves a concurrent
+  overload running, a brief overload does not make a multi-hour cooldown
+  permanent, and a later shorter 429 cannot revive an expiry a refresh had
+  already retired.
+
+  Relatedly, a usage window with no usable figure — `five_hour: {}`, a
+  non-numeric utilization — no longer parses as `0`. It now arrives with the
+  figure absent, so missing data can never read as proof of capacity and
+  retire a live cooldown. Rolling a spent window over past its reset likewise
+  clears the reading instead of writing a `0` nobody reported. Blocking
+  decisions still treat both as `0`, and the dashboard still displays `0`,
+  exactly as before.
+
+  Which of the response headers and the usage snapshot describes an account's
+  current capacity is now decided on the same event order, rather than on
+  `fetchedAt` against `lastUpdated`. A refresh that starts before a response
+  and finishes after it holds the older picture despite the later clock
+  reading, and preferring it hid a fresher exhaustion signal behind a snapshot
+  that never saw it — while cooldown release, already running on the event
+  order, disagreed about which source was current. Snapshots predating the
+  ordering tokens still fall back to the timestamp comparison.
+
+  Within scope, releasing opens no hole: the same snapshot feeds the
+  exhausted-window check, so an account with no real capacity stays blocked
+  on its own merits.
+
+- The activity log's "cooldown expired — rate limit cleared" entry now marks
+  the moment an account is actually routable again. It hung off the
+  header `rate_limited` flag alone, which both over- and under-reports as soon
+  as anything else can block the account: a 429 overlapping a 529 announced
+  recovery while the overload cooldown still kept the account out of rotation,
+  and because that flag only flips once, the moment it genuinely came back
+  passed unannounced. The entry is now emitted when the last account-wide
+  blocker clears — header status, global cooldown, or a spent account-wide
+  window — whichever that turns out to be, including a cooldown that lapses
+  during an idle stretch with nothing routing or polling in the meantime. A
+  model-scoped limit never emits one, since the account kept serving every
+  other family and so never left the rotation to rejoin.
+
+---
+
+## [0.10.1] — 2026-08-20
+
+### Fixed
+
+- A passed-through upstream 5xx on the Anthropic path is logged and counted.
+  The proxy is byte-transparent and only special-cased 401/429/529, so a
+  plain upstream 500 left no trace: an overnight Anthropic 500 stopped an
+  unattended Claude session while the daemon log showed nothing and the
+  stats reported a clean night. It now produces an `[ERROR]` line, an
+  activity entry (`upstream-error`), and error counts — with no cooldown,
+  since a plain 5xx says nothing about the account's capacity and can even
+  be request-specific.
+- A client-cancelled stream abort is no longer logged as an error on the
+  OpenAI path. The Codex CLI aborts streams routinely, and each abort
+  rejects the relay's body read — the log printed an `[ERROR] ... relay
+  failed` line for every one (eight hours of them in one overnight
+  session) while the stats correctly classified them as cancellations. The
+  log line now waits for the cancellation check, so it fires only for real
+  relay failures.
+
+---
+
+## [0.10.0] — 2026-08-19
+
+### Added
+
 - OpenAI/Codex account usage is fetched proactively, so `cc-router status`
   shows the 5h/weekly bars immediately after a restart — matching how
   Anthropic accounts already behaved. Codex usage previously arrived only on
@@ -52,6 +184,29 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   one row when the selection steps past its bottom or top edge, stays put while
   the selection moves inside it, and re-clamps when new entries push the
   selected row (which is timestamp-anchored) out of the stored window.
+- OpenAI/Codex sticky session routing: sessions pin to one account for prompt-cache
+  locality (`session_id` → `x-claude-code-session-id` → `prompt_cache_key`), with
+  load- and headroom-aware selection for new sessions.
+- Codex usage tracking from `x-codex-*` response headers: default 5h/weekly windows
+  plus dynamically discovered model-scoped metered buckets, credits, and plan.
+- Scoped cooldowns on upstream failures: bucket-scoped via `x-codex-active-limit`,
+  account-global otherwise; local 429/503 responses when no account is eligible.
+- Dashboard: OpenAI accounts now show 5h/weekly bars, per-bucket rows, credits,
+  plan, request/error/in-flight/session counts, and cooldown state.
+- Unprefixed `gpt-*` models route to OpenAI. The Codex CLI writes the bare slug
+  from its own registry — `model = "gpt-5.6-sol"` in `config.toml`, or whatever
+  its `/model` picker selects — and an unprefixed name went to the Claude path,
+  where `/v1/responses` answers `501 Not Implemented`. No configuration could
+  redirect it, because `openAIAliases` is only consulted for names that are
+  already prefixed; those aliases now apply to the bare form as well. Every
+  other unprefixed model still routes to Claude.
+
+### Changed
+
+- OpenAI account records persist `scopes`, `sessionLimitPercent`, and
+  `weeklyLimitPercent`.
+- The stateless OpenAI round-robin picker was removed in favor of
+  `OpenAITokenPool`.
 
 ### Fixed
 
@@ -143,39 +298,6 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   on `CC_ROUTER_DAEMON`, which the LaunchAgent and systemd unit never set —
   they set `CC_ROUTER_SERVICE` — so every service-managed instance left no PID
   behind and took the weaker port-based stop path.
-
----
-
-## [0.10.0] — 2026-08-18
-
-### Added
-
-- OpenAI/Codex sticky session routing: sessions pin to one account for prompt-cache
-  locality (`session_id` → `x-claude-code-session-id` → `prompt_cache_key`), with
-  load- and headroom-aware selection for new sessions.
-- Codex usage tracking from `x-codex-*` response headers: default 5h/weekly windows
-  plus dynamically discovered model-scoped metered buckets, credits, and plan.
-- Scoped cooldowns on upstream failures: bucket-scoped via `x-codex-active-limit`,
-  account-global otherwise; local 429/503 responses when no account is eligible.
-- Dashboard: OpenAI accounts now show 5h/weekly bars, per-bucket rows, credits,
-  plan, request/error/in-flight/session counts, and cooldown state.
-- Unprefixed `gpt-*` models route to OpenAI. The Codex CLI writes the bare slug
-  from its own registry — `model = "gpt-5.6-sol"` in `config.toml`, or whatever
-  its `/model` picker selects — and an unprefixed name went to the Claude path,
-  where `/v1/responses` answers `501 Not Implemented`. No configuration could
-  redirect it, because `openAIAliases` is only consulted for names that are
-  already prefixed; those aliases now apply to the bare form as well. Every
-  other unprefixed model still routes to Claude.
-
-### Changed
-
-- OpenAI account records persist `scopes`, `sessionLimitPercent`, and
-  `weeklyLimitPercent`.
-- The stateless OpenAI round-robin picker was removed in favor of
-  `OpenAITokenPool`.
-
-### Fixed
-
 - A refresh token the OAuth server rejects as terminally expired
   (`400 invalid_grant`) is no longer retried forever. Every rejection was
   treated as transient, so the five-minute refresh loop kept re-POSTing a token

package/README.md

@@ -23,6 +23,7 @@ Distribute Claude Code requests across Claude subscriptions, and expose an OpenA
 - **Codex CLI support** — configure Codex to use CC-Router as a Responses-compatible provider
 - **Automatic token refresh** — OAuth tokens are refreshed before they expire, saved atomically to disk
 - **Model-aware rate limits** — avoids accounts whose requested-model or global allowance is exhausted, and respects scoped cooldowns
+- **Automatic failover & retry** — a 429 fails over to another account and a 5xx is retried inside the router, before any response byte is relayed, on both the Claude and Codex routes; on by default, opt out with `"autoFailover": false`
 - **Client mode** — connect another device you own to your private CC-Router (`cc-router client connect <url>`)
 - **Claude Desktop support** — route Cowork / Agent-mode traffic through CC-Router via mitmproxy interception (macOS, Windows, Linux)
 - **Guided setup wizard** — interactive `cc-router setup` extracts tokens from Keychain or credentials file, configures everything
@@ -75,7 +76,7 @@ CC-Router keeps requests from one Claude Code session on the same eligible Anthr
 
 Anthropic cooldowns, effective global or requested-model quota exhaustion, disabled accounts, invalid authentication, and unhealthy accounts are hard exclusions. The configured per-account percentage caps are softer policy controls: when at least one account is otherwise usable but every usable account is over a configured cap, CC-Router may explicitly fall back to the least-loaded capped account. It never uses that fallback to bypass an Anthropic cooldown or exhausted effective quota.
 
-If an upstream account returns 401, 429, or 529, CC-Router passes that response through unchanged and invalidates the session's affinity. The client's next retry can then select another usable account; the router never retries after response bytes have started. If no account is usable before forwarding begins, the router instead returns a local Anthropic-shaped 429 whenever any account is blocked by a rate limit or exhausted quota. That 429 includes `Retry-After` only when a trustworthy unblock time is known. A local 503 is reserved for entirely non-rate-limit unavailability, such as all accounts being disabled or unhealthy. Either local response makes no Anthropic Messages request. Affinity mappings exist only in process memory, expire after one hour of inactivity, and are capped in size. Session IDs are never persisted or logged.
+If an upstream account returns 429 or any 5xx before a single response byte has been relayed, CC-Router applies the failure's cooldown and affinity bookkeeping and then retries the request itself, up to 3 upstream attempts per request. A 429 (or an overload the provider cools down: Anthropic 529; Codex 503/529) always fails over to a *different* account. Any other 5xx keeps a session-bound request on its own account, retrying after a short pause; a session-less request re-routes the way a fresh request would — typically an idle other account, exactly where the client's own retry used to land. The failover is on by default; set `"autoFailover": false` in `~/.cc-router/config.json` (and restart the router) to opt out — every upstream failure then passes through unchanged and clients own all retries, as before. Be aware that current Claude Code builds no longer retry 429s themselves, so with failover off a rate limit surfaces directly in the session as an error. One trade-off worth knowing: once the router commits to a retry it abandons the original failure response, so a network error on the retry attempt surfaces as a local 502 rather than the original 429. When no other account is eligible or the budget is exhausted, the last failed upstream response is passed through unchanged, exactly as before. A 401 is always passed through (with a background token refresh), and the router never retries after response bytes have started — mid-stream failures reach the client untouched. If no account is usable before forwarding begins, the router instead returns a local Anthropic-shaped 429 whenever any account is blocked by a rate limit or exhausted quota. That 429 includes `Retry-After` only when a trustworthy unblock time is known. A local 503 is reserved for entirely non-rate-limit unavailability, such as all accounts being disabled or unhealthy. Either local response makes no Anthropic Messages request. Affinity mappings exist only in process memory, expire after one hour of inactivity, and are capped in size. Session IDs are never persisted or logged.
 
 Streaming remains byte-transparent. In particular, CC-Router never appends a synthetic `message_stop` event. `proxyRequestTimeoutMs` protects only the phase before Anthropic response headers arrive; once a response starts, its body continues through the native byte-exact proxy pipe. Automatic `cc-router configure` setup manages Claude Code's event-level and byte-level stream idle watchdogs at 30 minutes. Restart any existing Claude Code process after configuration so it inherits those values.
 
@@ -92,8 +93,8 @@ Claude Max has rate limits per account. If you hit them regularly mid-session
 With two accounts you double your effective rate limit. With three, you triple it. The proxy distributes requests automatically; you don't change how you use Claude Code at all.
 
 ```text
-1 account  →  hit limit, wait 60s, continue
-3 accounts →  new sessions spread across all three; each session stays cache-local
+1 account  →  hit limit, session errors out (current Claude Code no longer retries 429s)
+3 accounts →  sessions spread across all three; a rate-limited request fails over mid-flight
 ```
 
 ---

package/dist/config/manager.js

@@ -232,6 +232,15 @@ export function getProxyRequestTimeoutMs() {
         ? timeoutMs
         : DEFAULT_PROXY_REQUEST_TIMEOUT_MS;
 }
+/**
+ * Whether the router may retry upstream 429/5xx failures itself. Enabled
+ * unless the config explicitly says `"autoFailover": false` — a missing or
+ * malformed value keeps the default on, matching how the other optional
+ * proxy settings degrade.
+ */
+export function getAutoFailoverEnabled() {
+    return readConfig().autoFailover !== false;
+}
 function normalizeProxyConfig(cfg) {
     const { proxyRequesTime, ...normalized } = cfg;
     const timeoutMs = normalized.proxyRequestTimeoutMs ?? proxyRequesTime;

package/dist/providers/anthropic/rate-limit-headers.js

@@ -0,0 +1,44 @@
+import { nextEventSequence } from "../../proxy/event-sequence.js";
+/**
+ * Rate-limit extraction from Anthropic's unified response headers. Lives
+ * apart from the server so both Anthropic transports (the generic /v1 proxy
+ * and the retrying /v1/messages route) capture the same snapshot.
+ */
+function inferPlan(requestsLimit) {
+    if (requestsLimit <= 0)
+        return "";
+    if (requestsLimit <= 100)
+        return "Pro";
+    if (requestsLimit <= 500)
+        return "Max 5x";
+    return "Max 20x";
+}
+function extractRateLimits(headers) {
+    const h = (name) => String(headers[name] ?? "");
+    const status = h("anthropic-ratelimit-unified-status");
+    if (!status)
+        return null; // No unified headers in this response
+    const requestsLimit = parseInt(h("anthropic-ratelimit-requests-limit"), 10) || 0;
+    return {
+        status: status === "rate_limited" ? "rate_limited" : "allowed",
+        fiveHourUtil: parseFloat(h("anthropic-ratelimit-unified-5h-utilization")) || 0,
+        fiveHourReset: parseInt(h("anthropic-ratelimit-unified-5h-reset"), 10) || 0,
+        sevenDayUtil: parseFloat(h("anthropic-ratelimit-unified-7d-utilization")) || 0,
+        sevenDayReset: parseInt(h("anthropic-ratelimit-unified-7d-reset"), 10) || 0,
+        claim: h("anthropic-ratelimit-unified-representative-claim"),
+        plan: inferPlan(requestsLimit),
+        requestsLimit,
+        lastUpdated: Date.now(),
+        // Wall-clock ms ties with the usage refresh the router starts from this
+        // same response, so the ordering token is what makes them comparable.
+        lastUpdatedSeq: nextEventSequence(),
+    };
+}
+/** Apply upstream rate-limit headers without discarding the usage snapshot. */
+export function applyRateLimitHeaders(account, headers) {
+    const rateLimits = extractRateLimits(headers);
+    if (!rateLimits)
+        return false;
+    account.rateLimits = { ...account.rateLimits, ...rateLimits };
+    return true;
+}

package/dist/providers/anthropic/usage.js

@@ -1,3 +1,4 @@
+import { nextEventSequence } from "../../proxy/event-sequence.js";
 const ANTHROPIC_USAGE_ENDPOINT = "https://api.anthropic.com/api/oauth/usage";
 const OAUTH_BETA_HEADER = "oauth-2025-04-20";
 const DEFAULT_USAGE_TIMEOUT_MS = 5_000;
@@ -18,10 +19,20 @@ function stringValue(value) {
 function numberValue(value) {
     return typeof value === "number" && Number.isFinite(value) ? value : undefined;
 }
+/**
+ * Normalize a reported percentage into a 0–1 fraction, or undefined when the
+ * provider reported nothing usable.
+ *
+ * The distinction matters downstream: a *reported* 0 is proof of capacity and
+ * can retire a cooldown, while a missing or non-numeric figure is only absence
+ * of information. Collapsing the two to 0 would let a malformed payload —
+ * `five_hour: {}`, `utilization: null` — unbench an account that is still
+ * being rate limited.
+ */
 function utilization(value) {
     const number = numberValue(value);
     if (number === undefined)
-        return 0;
+        return undefined;
     return Math.max(0, Math.min(1, number / 100));
 }
 function resetAt(value) {
@@ -43,8 +54,9 @@ function getFirst(record, keys) {
 function parseWindow(value) {
     if (!isRecord(value))
         return undefined;
+    const reported = utilization(getFirst(value, ["utilization", "percentage", "percent"]));
     return {
-        utilization: utilization(getFirst(value, ["utilization", "percentage", "percent"])),
+        ...(reported === undefined ? {} : { utilization: reported }),
         resetAt: resetAt(getFirst(value, ["resets_at", "reset_at", "resetAt"])),
     };
 }
@@ -88,11 +100,12 @@ function parseModelLimit(value) {
     if (!model)
         return undefined;
     const active = getFirst(value, ["active", "is_active"]);
+    const reported = utilization(getFirst(value, ["utilization", "percentage", "percent"]));
     return {
         kind: "weekly_scoped",
         group: stringValue(value.group) ?? "weekly",
         ...model,
-        utilization: utilization(getFirst(value, ["utilization", "percentage", "percent"])),
+        ...(reported === undefined ? {} : { utilization: reported }),
         resetAt: resetAt(getFirst(value, ["resets_at", "reset_at", "resetAt"])),
         active: typeof active === "boolean" ? active : true,
         severity: stringValue(value.severity) ?? "",
@@ -138,7 +151,7 @@ function legacyModelLimit(family, value) {
     };
 }
 /** Parse the OAuth usage endpoint without retaining its provider-specific payload. */
-export function parseAnthropicUsage(value, fetchedAt) {
+export function parseAnthropicUsage(value, fetchedAt, requestedSeq) {
     if (!isRecord(value) || !Object.keys(value).some((key) => USAGE_FIELDS.has(key)))
         return null;
     const limits = Array.isArray(value.limits) ? value.limits : undefined;
@@ -153,6 +166,8 @@ export function parseAnthropicUsage(value, fetchedAt) {
         fetchedAt,
         fetchStatus: "fresh",
     };
+    if (requestedSeq !== undefined)
+        snapshot.requestedSeq = requestedSeq;
     const fiveHour = parseWindow(value.five_hour);
     const sevenDay = parseWindow(value.seven_day);
     const extraUsage = parseExtraUsage(value.extra_usage);
@@ -183,6 +198,10 @@ export async function fetchAnthropicUsage(account, options = {}) {
     const timeoutMs = Math.max(0, options.timeoutMs ?? DEFAULT_USAGE_TIMEOUT_MS);
     const controller = new AbortController();
     const timeout = setTimeout(() => controller.abort(), timeoutMs);
+    // Claimed before the request goes out: the response describes the account no
+    // earlier than this point in the event order, which is what lets a caller
+    // order the snapshot against events that happened while it was in flight.
+    const requestedSeq = options.nextSequence?.() ?? nextEventSequence();
     try {
         const response = await request(ANTHROPIC_USAGE_ENDPOINT, {
             method: "GET",
@@ -201,7 +220,7 @@ export async function fetchAnthropicUsage(account, options = {}) {
         catch {
             return { ok: false, reason: "invalid_json" };
         }
-        const snapshot = parseAnthropicUsage(body, now());
+        const snapshot = parseAnthropicUsage(body, now(), requestedSeq);
         return snapshot
             ? { ok: true, snapshot }
             : { ok: false, reason: "invalid_schema" };