ldrouter 1.17.4 → 1.17.6

package/CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to this project are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/) and the project adheres to
 [Semantic Versioning](https://semver.org/).
 
+## [1.17.6] - 2026-09-20
+
+### Fixed
+
+- **Every Qoder request logged `Tokens in 0 · out 0 · cache 0`.** Two independent causes. Streaming: the Qoder branch pipes OpenAI-shaped chunks through the runner's shared chunk handler, which only parsed usage under `cfg.type === 'openai'`, so the accumulator stayed at zero for Qoder — live data confirmed 128 of 128 streaming Qoder requests carried no tokens. Non-streaming: a call asking for no stream can be answered with the whole completion as one plain JSON body, which the SSE-only envelope reader left in its unparsed buffer, so `usage` came back `undefined`. The reader now keeps non-`data:` lines rather than dropping them and the non-streaming path folds a plain completion in (a no-op when the origin did frame its answer as SSE). The usage shapes Qoder actually sends are read too: the object nested one level down as `{statusCodeValue, body}`, and OpenAI-compatible `prompt_tokens_details.cached_tokens`.
+- **A fallback that succeeded was logged as a failure.** The failed first attempt's error was never cleared once a later candidate answered, so the row kept the first attempt's status: 17 requests were recorded `success=0` with `http_status` 502/504/429 while the client had received a 200 answer. The success rate and every per-model figure derived from the request log inherited that error.
+- **The cache-hit rate double-counted the cache.** OpenAI-compatible providers report `cached_tokens` as a *subset* of `prompt_tokens`, so `input` already contains the cached prefix and the blanket `cacheRead / (input + cacheRead)` inflated the denominator — measured at 21.75% where the truth was 27.80% over real traffic. Anthropic is the exception (`input_tokens` *excludes* the cached prefix), so each request is now normalised on its own provider's semantics, on the server and in the live feed alike. The live feed also derives its rates from running totals instead of incrementing a ratio, which is meaningless.
+- **Codex reported zero cached and zero reasoning tokens.** The Responses API nests them under `usage.input_tokens_details.cached_tokens` and `usage.output_tokens_details.reasoning_tokens`; only the flat names were read, so 1,381 requests (46% of traffic) contributed nothing to the cache-hit rate. Flat names still work for upstreams that send them.
+- **A Codex account was retired on a single flaky refresh.** Every account kept turning `degraded` with `last_error=oauth_refresh_failed`, and because the gateway reads that code as "these credentials are dead" it also evicted the account (`enabled=0`). One failure code covered five unrelated things — a request timeout, a DNS error, a token-endpoint 5xx, a 200 carrying no usable token, and a rolled-back database write — and only a grant the endpoint actually rejected is a verdict. The outcomes are now split the way 9router splits them: `oauth_refresh_failed`, `credential_unavailable` and `account_not_found` are final, while `oauth_refresh_unavailable` and `invalid_refresh_response` are transient. A transient failure still moves the request to a sibling account but leaves this one enabled, keeps its health, and writes no `last_error`, so a status the operator reads always means what it says. The rejection is read out of the response body and only on a 4xx — a 5xx or a gateway page echoing `invalid_grant` is still transient — and the body is read whole, because OpenAI nests the code under `error.code` where a top-level read sees nothing. Live data: every account held `consecutive_failures = 0`, the counter meant to require a pattern before a verdict was never reached.
+- **`+ Add` in the New combo dialog never added the model.** One handler served both dialogs and wrote to the *edit* form, so adding in the create dialog mutated the edit state while the create member list stayed empty — the button read as dead. The edit dialog worked, which is why the report was "sometimes". Each dialog now owns its member list.
+
+## [1.17.5] - 2026-09-19
+
+### Fixed
+
+- **An account whose session was revoked kept being selected instead of leaving the Codex pool.** Auth0 answers `refresh_token_invalidated` ("Your session has ended. Please log in again.") for an account whose login was revoked; the account cannot serve anything again until it is re-imported. The credential layer surfaced that as a bare `oauth_refresh_failed`, which the router classified `unknown` — and `unknown` is not a routing decision, so `shouldFallback` answered false, the retry never advanced to a sibling account, and the request answered `502`. Live production data confirmed the shape: two accounts holding invalidated refresh tokens sat at the head of the pool, so every request landed on one of them, and 226 of 241 failed requests carried `attempts_count=1` while healthy accounts sat idle behind them. Credential failure is now its own failure class, checked before the error-type switch, and it always advances to the next candidate — with or without a combo plan, since a direct `codex1/...` model has no fallback triggers to consult.
+- **The account with the dead credential is now disabled, not merely degraded.** Marking down *and* disabling (`enabled=0`) is what actually stops the churn, mirroring quota exhaustion: every selection path — `expandCodexAccountCandidates`, `getCodexAccountForProvider` — filters on `enabled`. Deliberately kept out of `isUpstreamHealthFailure`, because one re-imported account must not open the provider circuit breaker for its healthy siblings.
+- **`codexCredentialError` discarded the raw error code.** Its wrapping message is deliberately generic, so without the cause the routing layer could not distinguish a dead account from any other `authentication_error`. The raw error is now carried as `cause`.
+
 ## [1.17.4] - 2026-09-18
 
 ### Fixed

package/dist/server/gateway/runner.js

@@ -261,6 +261,9 @@ export class GatewayRunner {
                     });
                     attempt.statusCode = out.statusCode ?? null;
                     attempt.success = true;
+                    // A fallback that succeeded answered the client 200: the earlier failed
+                    // attempt must not keep the request logged (and reported) as a failure.
+                    lastError = null;
                     attempt.latencyMs = Date.now() - attemptStart;
                     attempt.ttftMs = out.ttftMs;
                     attempt.usage = out.usage;
@@ -296,6 +299,9 @@ export class GatewayRunner {
                 catch (e) {
                     const err = e instanceof GatewayError ? e : new GatewayError('upstream_error', e.message, { cause: e });
                     const quotaFailure = isQuotaFailure(err);
+                    // Only a final verdict retires the account (below); a transient refresh failure still moves
+                    // the request to a sibling account but must leave this one enabled.
+                    const credentialFailure = isPermanentCredentialFailure(err);
                     debugUpstream(ctx.requestId, 'ATTEMPT ERROR', [
                         `attempt=${i + 1}`,
                         `provider=${provider.name}`,
@@ -329,6 +335,11 @@ export class GatewayRunner {
                     // dead account on every request for hours (verified live) and answered "usage limited".
                     if (quotaFailure)
                         markQuotaExhausted(candidate, err.message);
+                    // A dead refresh token also leaves the pool outright: the account cannot serve anything
+                    // again until it is re-imported, so a retry may only be spent on a sibling account. Without
+                    // this the retry landed on the same dead account and the request answered 502.
+                    if (credentialFailure)
+                        markCredentialsDead(candidate, err.message);
                     if (isUpstreamHealthFailure(err)) {
                         recordFailure(provider.id, provider.cbFailureThreshold, provider.cbCooldownSeconds);
                         if (candidate.codexAccountId)
@@ -611,7 +622,9 @@ export class GatewayRunner {
             }
             try {
                 const obj = JSON.parse(chunk.data);
-                if (cfg.type === 'openai') {
+                // Qoder emits OpenAI-shaped chunks and pipes them through this same handler
+                // (see the qoder branch below), so its usage block is parsed here too.
+                if (cfg.type === 'openai' || cfg.type === 'qoder') {
                     const choice = obj.choices?.[0];
                     if (choice?.delta?.content)
                         textBuf += choice.delta.content;
@@ -1032,6 +1045,53 @@ export function markQuotaExhausted(candidate, message) {
     if (candidate.qoderAccountId)
         setQoderAccountHealth(candidate.qoderAccountId, 'down', reason, false);
 }
+/**
+ * A credential failure is a durable, per-account verdict — the same class as quota exhaustion.
+ *
+ * `withCodexCredentials` throws these as bare codes when the account's refresh token is dead
+ * (Auth0 answers `refresh_token_invalidated`, "Your session has ended. Please log in again.").
+ * The account cannot serve anything again until it is re-imported, so retrying it is pure waste
+ * and leaving it in the pool answers 502 to every client.
+ *
+ * Live evidence (production, 2026-09-19): two accounts holding invalidated refresh tokens sat at
+ * the head of the pool, so every request landed on one of them — `oauth_refresh_failed`,
+ * classified `unknown`, `attempts_count=1`, 502 straight to the client (226 of 241 failed
+ * requests had exactly one attempt while healthy accounts sat idle behind them).
+ *
+ * Deliberately NOT part of `isUpstreamHealthFailure`: one re-imported account must not open the
+ * provider circuit breaker for its healthy siblings. The verdict is applied per account by
+ * `markCredentialsDead` instead.
+ *
+ * Both spellings are checked because the code travels either raw (from `withCodexCredentials`
+ * inside the gateway) or wrapped by `codexCredentialError`, which now keeps it as `cause`.
+ */
+export function isCredentialFailure(err) {
+    const codes = /^(oauth_refresh_failed|oauth_refresh_unavailable|invalid_refresh_response|credential_unavailable|account_not_found)$/;
+    const cause = err.cause;
+    return codes.test(err.message) || (typeof cause?.message === 'string' && codes.test(cause.message));
+}
+/**
+ * Whether the verdict is final, i.e. the account cannot serve again until an operator re-imports it.
+ *
+ * A transient refresh failure (timeout, network error, token-endpoint 5xx, rolled-back write) is
+ * still a credential failure for routing — the request should move to a sibling account — but it
+ * must NOT retire this one. Treating both classes as final is what degraded healthy accounts and
+ * disabled them (`enabled=0`) on a single flaky moment; every live account carried
+ * `consecutive_failures = 0`, so no pattern was ever required before the verdict.
+ */
+export function isPermanentCredentialFailure(err) {
+    const codes = /^(oauth_refresh_failed|credential_unavailable|account_not_found)$/;
+    const cause = err.cause;
+    return codes.test(err.message) || (typeof cause?.message === 'string' && codes.test(cause.message));
+}
+/** Takes an account with a dead refresh token out of the pool. `enabled=0` is what stops the churn. */
+export function markCredentialsDead(candidate, message) {
+    const reason = redactString(message);
+    if (candidate.codexAccountId)
+        setCodexAccountHealth(candidate.codexAccountId, 'down', reason, false);
+    if (candidate.qoderAccountId)
+        setQoderAccountHealth(candidate.qoderAccountId, 'down', reason, false);
+}
 /**
  * Whether a failed attempt should advance to the next candidate.
  *
@@ -1043,11 +1103,17 @@ export function markQuotaExhausted(candidate, message) {
  * plan means no fallback.
  */
 export function shouldRetryAttempt(comboPlan, err) {
+    if (isCredentialFailure(err))
+        return true;
     if (isQuotaFailure(err))
         return true;
     return comboPlan ? shouldFallback(comboPlan, { type: classifyFailure(err), status: err.status }) : false;
 }
 export function classifyFailure(err) {
+    // Checked before the type switch for the same reason as quota: a dead account is not a routing
+    // decision the combo's triggers were meant to gate, and `unknown` is not a routing decision at all.
+    if (isCredentialFailure(err))
+        return 'credential';
     // Checked first, before the type switch: a quota refusal arrives as `upstream_rate_limit` (a
     // 429) or `upstream_error` (Codex rewraps it as a 502 with a 429 cause; Qoder uses code 112) and
     // must not fall through to `http_status`/`unknown`, neither of which is a routing decision.

package/dist/server/providers/codex-refresh.js

@@ -10,11 +10,24 @@ export function codexCredentialError(error) {
     const code = error instanceof Error ? error.message : '';
     if (code === 'account_not_found')
         return new GatewayError('invalid_request_error', 'Codex account not found', { status: 404 });
-    if (code === 'oauth_refresh_failed' || code === 'invalid_refresh_response' || code === 'credential_unavailable') {
-        return new GatewayError('authentication_error', 'Codex credentials could not be refreshed — re-import the account', { status: 401 });
+    if (code === 'oauth_refresh_unavailable' || code === 'invalid_refresh_response') {
+        // Transient: the token is probably still good, so the operator should retry, not re-import.
+        // Same wrapping as the final verdict (the routing layer sees one credential class either way)
+        // but the message must not send them to the import dialog over one flaky attempt.
+        return new GatewayError('authentication_error', 'Codex credentials could not be refreshed — try again', { status: 401, cause: error });
+    }
+    if (code === 'oauth_refresh_failed' || code === 'credential_unavailable') {
+        // `cause` carries the raw credential code: the wrapping message is deliberately generic, so
+        // without it the routing layer cannot tell a dead account from any other authentication_error.
+        return new GatewayError('authentication_error', 'Codex credentials could not be refreshed — re-import the account', { status: 401, cause: error });
     }
     return error;
 }
+/** Only a final verdict may retire an account; a transient failure leaves it in the pool. */
+const PERMANENT_REFRESH_FAILURES = new Set(['oauth_refresh_failed', 'credential_unavailable', 'account_not_found']);
+export function isPermanentRefreshFailure(error) {
+    return PERMANENT_REFRESH_FAILURES.has(error);
+}
 const REFRESH_LEAD_MS = 5 * 60 * 1000;
 const REFRESH_TIMEOUT_MS = 10_000;
 const flights = new Map();
@@ -37,8 +50,12 @@ async function defaultRefreshClient(input) {
         body: new URLSearchParams({ grant_type: 'refresh_token', refresh_token: input.refreshToken, client_id: CODEX_OAUTH.clientId }),
         signal: input.signal,
     });
-    if (!response.ok)
-        throw new Error(`oauth refresh http ${response.status}`);
+    if (!response.ok) {
+        // The RFC 6749 error body is what separates a dead grant from a bad moment; `safeError` classifies
+        // on it. Only the resulting code is ever persisted, never this text.
+        const detail = await response.text().catch(() => '');
+        throw Object.assign(new Error(`oauth refresh http ${response.status}`), { status: response.status, oauthError: detail });
+    }
     const value = await response.json();
     if (typeof value.access_token !== 'string' || !value.access_token)
         throw new Error('invalid refresh response');
@@ -50,9 +67,25 @@ async function defaultRefreshClient(input) {
         expiresAt: typeof value.expires_at === 'string' ? value.expires_at : undefined,
     };
 }
-function safeError(_error) {
-    return 'oauth_refresh_failed';
+/** Grant rejections the endpoint answers with; only these mean the refresh token is dead. */
+const REJECTED_GRANT_MARKERS = ['invalid_grant', 'refresh_token_invalidated', 'refresh_token_reused', 'refresh_token_expired', 'session has ended'];
+/**
+ * Decides whether a failed refresh is a verdict or just a failed attempt.
+ *
+ * A marker only counts when the endpoint itself rejected the grant (4xx): a 5xx or a gateway error
+ * page that happens to echo `invalid_grant` is still transient. Anything without a marker — a
+ * timeout, a DNS error, an aborted request, a rolled-back write — is transient by default, so an
+ * account is never retired on evidence this layer does not actually have.
+ */
+function safeError(error) {
+    const status = error?.status;
+    if (typeof status === 'number' && status >= 500)
+        return 'oauth_refresh_unavailable';
+    const oauthError = error?.oauthError;
+    const text = `${error?.message ?? ''} ${typeof oauthError === 'string' ? oauthError : ''}`.toLowerCase();
+    return REJECTED_GRANT_MARKERS.some((marker) => text.includes(marker)) ? 'oauth_refresh_failed' : 'oauth_refresh_unavailable';
 }
+/** Degrades an account for a final verdict only — a transient failure must leave health alone. */
 function markDegraded(accountId, error) {
     try {
         setCodexAccountHealth(accountId, 'degraded', error);
@@ -64,9 +97,11 @@ async function refreshOnce(accountId, now, force) {
     try {
         state = getCodexAccountRefreshState(accountId);
     }
-    catch (error) {
-        markDegraded(accountId, safeError(error));
-        return { ok: false, error: 'oauth_refresh_failed' };
+    catch {
+        // A row that cannot be read is an operator problem, not a failed refresh: it needs a re-import,
+        // so the account leaves the pool instead of burning an attempt on every request.
+        markDegraded(accountId, 'credential_unavailable');
+        return { ok: false, error: 'credential_unavailable' };
     }
     if (!state)
         return { ok: false, error: 'account_not_found' };
@@ -77,7 +112,8 @@ async function refreshOnce(accountId, now, force) {
     try {
         const result = await refreshClient({ refreshToken: state.refreshToken, signal: controller.signal });
         if (typeof result.accessToken !== 'string' || !result.accessToken) {
-            markDegraded(accountId, 'invalid_refresh_response');
+            // A 200 with nothing usable is suspicious, not proof the grant is dead: the next attempt may
+            // succeed, so the account keeps its health and stays in the pool.
             return { ok: false, error: 'invalid_refresh_response' };
         }
         const expiresAt = result.expiresAt ?? (typeof result.expiresIn === 'number' && Number.isFinite(result.expiresIn) && result.expiresIn > 0
@@ -85,7 +121,6 @@ async function refreshOnce(accountId, now, force) {
             : '');
         const expiresMs = Date.parse(expiresAt);
         if (!Number.isFinite(expiresMs) || expiresMs <= now.getTime()) {
-            markDegraded(accountId, 'invalid_refresh_response');
             return { ok: false, error: 'invalid_refresh_response' };
         }
         try {
@@ -96,15 +131,17 @@ async function refreshOnce(accountId, now, force) {
                 expiresAt,
             });
         }
-        catch (error) {
-            markDegraded(accountId, safeError(error));
-            return { ok: false, error: 'oauth_refresh_failed' };
+        catch {
+            // The transaction rolled back, so the stored token is unchanged and still usable.
+            return { ok: false, error: 'oauth_refresh_unavailable' };
         }
         return { ok: true, expiresAt };
     }
     catch (error) {
-        markDegraded(accountId, safeError(error));
-        return { ok: false, error: 'oauth_refresh_failed' };
+        const failure = safeError(error);
+        if (isPermanentRefreshFailure(failure))
+            markDegraded(accountId, failure);
+        return { ok: false, error: failure };
     }
     finally {
         clearTimeout(timer);
@@ -127,8 +164,8 @@ export async function withCodexCredentials(accountId, fn, now = new Date()) {
     try {
         state = getCodexAccountRefreshState(accountId);
     }
-    catch (error) {
-        markDegraded(accountId, safeError(error));
+    catch {
+        markDegraded(accountId, 'credential_unavailable');
         throw new Error('credential_unavailable', { cause: new Error('credential_unavailable') });
     }
     if (!state)
@@ -142,8 +179,8 @@ export async function withCodexCredentials(accountId, fn, now = new Date()) {
     try {
         credentials = getCodexCredentials(accountId);
     }
-    catch (error) {
-        markDegraded(accountId, safeError(error));
+    catch {
+        markDegraded(accountId, 'credential_unavailable');
         throw new Error('credential_unavailable', { cause: new Error('credential_unavailable') });
     }
     try {
@@ -158,8 +195,8 @@ export async function withCodexCredentials(accountId, fn, now = new Date()) {
         try {
             credentials = getCodexCredentials(accountId);
         }
-        catch (credentialError) {
-            markDegraded(accountId, safeError(credentialError));
+        catch {
+            markDegraded(accountId, 'credential_unavailable');
             throw new Error('credential_unavailable', { cause: new Error('credential_unavailable') });
         }
         return fn(credentials);

package/dist/server/providers/codex.js

@@ -151,7 +151,18 @@ function usage(value) {
     const u = (value && typeof value === 'object' ? value : {});
     const input = typeof u.input_tokens === 'number' ? u.input_tokens : 0;
     const output = typeof u.output_tokens === 'number' ? u.output_tokens : 0;
-    return { input, output, total: input + output, cacheRead: typeof u.cached_input_tokens === 'number' ? u.cached_input_tokens : 0, cacheWrite: 0, reasoning: typeof u.reasoning_tokens === 'number' ? u.reasoning_tokens : 0 };
+    // The Responses API nests both of these (`usage.input_tokens_details.cached_tokens`,
+    // `usage.output_tokens_details.reasoning_tokens`). Reading only the flat names returned
+    // 0 for every Codex request — 1,381 of them, 46% of all traffic — so the statistics page
+    // reported a cache-hit rate missing its largest contributor. Flat names stay supported
+    // for compatibility with OpenAI-compatible upstreams that use them.
+    const inDetails = (u.input_tokens_details && typeof u.input_tokens_details === 'object' ? u.input_tokens_details : {});
+    const outDetails = (u.output_tokens_details && typeof u.output_tokens_details === 'object' ? u.output_tokens_details : {});
+    const cacheRead = typeof inDetails.cached_tokens === 'number' ? inDetails.cached_tokens
+        : typeof u.cached_input_tokens === 'number' ? u.cached_input_tokens : 0;
+    const reasoning = typeof outDetails.reasoning_tokens === 'number' ? outDetails.reasoning_tokens
+        : typeof u.reasoning_tokens === 'number' ? u.reasoning_tokens : 0;
+    return { input, output, total: input + output, cacheRead, cacheWrite: 0, reasoning };
 }
 export function codexResponseToCanonical(body, requestedModel) {
     let text = '';

package/dist/server/providers/qoder/client.js

@@ -209,6 +209,23 @@ async function drainEnvelope(reader, response, onChunk) {
         await upstream.cancel().catch(() => { });
     }
 }
+/**
+ * Interpret a non-SSE body as a plain OpenAI chat completion and fold it into the reader.
+ * A no-op when the origin framed its answer as SSE frames, which is the usual case.
+ */
+function applyPlainCompletion(reader) {
+    const raw = reader.takeUnparsed();
+    if (!raw)
+        return;
+    let body;
+    try {
+        body = JSON.parse(raw);
+    }
+    catch {
+        return;
+    }
+    reader.applyCompletion(body);
+}
 /** Streaming: emit OpenAI-shaped chunks as they arrive, stop at the terminal frame. */
 export async function callQoderStreaming(cfg, req, onChunk, deps = {}) {
     const { response, reader, status, upstreamRequestId } = await openSignedStream(cfg, { ...req, stream: true }, deps);
@@ -219,6 +236,12 @@ export async function callQoderStreaming(cfg, req, onChunk, deps = {}) {
 export async function callQoderNonStreaming(cfg, req, deps = {}) {
     const { response, reader, status, upstreamRequestId } = await openSignedStream(cfg, { ...req, stream: false }, deps);
     await drainEnvelope(reader, response);
+    // The caller invoked this with stream disabled, so the origin may honour the stream
+    // preference and answer with the full completion in one body rather than SSE frames.
+    // The reader buffers those bytes as an unparsed trailing line (it only understands
+    // `data:` framing), which is why a non-streaming Qoder call logged zero tokens:
+    // `usage` came back `undefined` and the runner reported `in 0 · out 0 · cache 0`.
+    applyPlainCompletion(reader);
     return { status, upstreamRequestId, text: reader.text, toolCalls: reader.toolCalls, finishReason: reader.finishReason, usage: reader.usage ?? emptyUsage() };
 }
 export async function probeQoder(cfg) {

package/dist/server/providers/qoder/sse.js

@@ -8,13 +8,18 @@ function numberOr(value, fallback) {
 }
 export function qoderUsageOf(value) {
     const usage = (value && typeof value === 'object' ? value : {});
-    const details = (usage.prompt_tokens_details && typeof usage.prompt_tokens_details === 'object' ? usage.prompt_tokens_details : {});
-    const input = numberOr(usage.prompt_tokens ?? usage.input_tokens, 0);
-    const output = numberOr(usage.completion_tokens ?? usage.output_tokens, 0);
+    // The envelope nests the real OpenAI usage object one level down under `body`, so a
+    // few shapes are worth reading; the flat one stays authoritative when present.
+    const nested = (usage.usage && typeof usage.usage === 'object' ? usage.usage : {});
+    const asObject = (v) => (v && typeof v === 'object' ? v : {});
+    const details = asObject(usage.prompt_tokens_details ?? nested.prompt_tokens_details);
+    const input = numberOr(usage.prompt_tokens ?? usage.input_tokens ?? nested.prompt_tokens ?? nested.input_tokens, 0);
+    const output = numberOr(usage.completion_tokens ?? usage.output_tokens ?? nested.completion_tokens ?? nested.output_tokens, 0);
     return {
         input, output,
-        total: numberOr(usage.total_tokens, input + output),
-        cacheRead: numberOr(details.cached_tokens ?? usage.cached_tokens ?? usage.cache_read_input_tokens, 0),
+        total: numberOr(usage.total_tokens ?? nested.total_tokens, input + output),
+        cacheRead: numberOr(details.cached_tokens ?? usage.cached_tokens ?? usage.cache_read_input_tokens
+            ?? nested.cached_tokens, 0),
         cacheWrite: numberOr(details.cache_creation_tokens ?? usage.cache_creation_input_tokens, 0),
         reasoning: numberOr(usage.reasoning_tokens ?? usage.completion_tokens_details?.reasoning_tokens, 0),
     };
@@ -28,6 +33,8 @@ export class QoderEnvelopeReader {
     usage = null;
     error = null;
     buffer = '';
+    // Non-`data:` lines, in arrival order: a plain (unframed) JSON body. See takeUnparsed().
+    plain = [];
     ended = false;
     pendingFinish = null;
     pendingUsage = null;
@@ -39,6 +46,50 @@ export class QoderEnvelopeReader {
     }
     terminal() { return this.ended; }
     errorEnvelope() { return this.error; }
+    /**
+     * Take the buffered, never-`data:`-framed bytes (if any) and clear them. An origin
+     * answering a non-streaming call with a plain JSON completion leaves the whole
+     * response here — the SSE line scanner only recognises `data:` frames.
+     */
+    takeUnparsed() {
+        const raw = [...this.plain, this.buffer].join('\n').trim();
+        this.plain = [];
+        this.buffer = '';
+        return raw;
+    }
+    /**
+     * Fold an OpenAI-shaped non-streaming completion into this reader, so the caller
+     * gets text/toolCalls/finishReason/usage from one accessor set regardless of how
+     * the origin framed the answer. `usage` is only written when the body carries one:
+     * a zero would be indistinguishable from "the provider reported nothing".
+     */
+    applyCompletion(body) {
+        const choice = (Array.isArray(body.choices) ? body.choices[0] : null);
+        const message = (choice?.message ?? {});
+        if (typeof message.content === 'string')
+            this.text += message.content;
+        const calls = Array.isArray(message.tool_calls) ? message.tool_calls : [];
+        for (const call of calls) {
+            let input = {};
+            if (typeof call.function?.arguments === 'string' && call.function.arguments) {
+                try {
+                    input = JSON.parse(call.function.arguments);
+                }
+                catch {
+                    input = {};
+                }
+            }
+            this.toolCalls.push({ id: call.id ?? `call-${this.toolCalls.length}`, name: call.function?.name ?? 'unknown', input });
+        }
+        // Assigned directly, not via the pending machinery: this body is complete, and the
+        // stream has already ended (finish() set `ended`), which those paths return early on.
+        const finish = (choice?.finish_reason ?? (typeof body.status === 'string' ? body.status : null));
+        if (finish)
+            this.finishReason = finish;
+        if (body.usage)
+            this.usage = qoderUsageOf(body.usage);
+        this.ended = true;
+    }
     push(text) {
         if (this.ended)
             return;
@@ -63,9 +114,17 @@ export class QoderEnvelopeReader {
     }
     cancel() { this.ended = true; this.buffer = ''; }
     line(raw) {
-        const trimmed = raw.replace(/\r$/, '').trim();
-        if (!trimmed.startsWith('data:'))
+        // trim() also drops the trailing CR of a CRLF frame, so no separate strip is needed.
+        const trimmed = raw.trim();
+        if (!trimmed.startsWith('data:')) {
+            // An origin answering a non-streaming call may send the whole completion as one
+            // plain JSON body instead of SSE frames. Hold those lines rather than dropping
+            // them: finish() consumes the buffer, so this is the only place they survive.
+            // Blank lines and `:comment` keepalives are not content.
+            if (trimmed && !trimmed.startsWith(':'))
+                this.plain.push(trimmed);
             return;
+        }
         const data = trimmed.slice(5).trimStart();
         if (data === '[DONE]') {
             this.flushPending();

package/dist/server/routes/admin/requests.js

@@ -35,6 +35,7 @@ export function toSummary(r, maps) {
         finalModelPublicId: finalModel?.publicModelId ?? null,
         providerId: provider ? finalModel.providerId : null,
         providerName: provider?.name ?? null,
+        providerType: provider?.type ?? null,
         streaming: Boolean(r.streaming),
         httpStatus: r.httpStatus,
         success: Boolean(r.success),

package/dist/server/routes/admin/stats.js

@@ -18,6 +18,34 @@ const PRESETS = {
         return { from: new Date(now.getTime() - 30 * 24 * 3600 * 1000), to: now, bucket: 'day' };
     },
 };
+/**
+ * Provider-cache hit rate for a window.
+ *
+ * The blanket `cacheRead / (input + cacheRead)` double-counts the cache: OpenAI-compatible
+ * providers report `cached_tokens` as a SUBSET of `prompt_tokens`, so the cached prefix is
+ * already inside `input` and adding it again inflates the denominator — understating the
+ * rate (measured on real traffic: 21.75% instead of 27.80%).
+ *
+ * Anthropic is the exception: its `input_tokens` EXCLUDES the cached prefix, so there the
+ * prompt really is `input + cache`. Each row is weighed by its provider's own `type` in a
+ * single pass.
+ */
+function cacheHitRateFor(db, conds) {
+    const anthropic = sql `${schema.providers.type} = 'anthropic'`;
+    const r = db
+        .select({
+        cacheRead: sql `COALESCE(SUM(${schema.requests.cacheReadTokens}),0)`,
+        anthropicPrompt: sql `COALESCE(SUM(CASE WHEN ${anthropic} THEN ${schema.requests.inputTokens} + ${schema.requests.cacheReadTokens} ELSE 0 END),0)`,
+        otherPrompt: sql `COALESCE(SUM(CASE WHEN ${anthropic} THEN 0 ELSE ${schema.requests.inputTokens} END),0)`,
+    })
+        .from(schema.requests)
+        .leftJoin(schema.models, eq(schema.models.id, schema.requests.finalModelId))
+        .leftJoin(schema.providers, eq(schema.providers.id, schema.models.providerId))
+        .where(and(...conds))
+        .get();
+    const prompt = Number(r?.anthropicPrompt ?? 0) + Number(r?.otherPrompt ?? 0);
+    return prompt > 0 ? Number(r?.cacheRead ?? 0) / prompt : 0;
+}
 function rangeFromQuery(q) {
     if (q.preset && PRESETS[q.preset])
         return PRESETS[q.preset]();
@@ -50,6 +78,7 @@ export async function registerStatsRoutes(app) {
             .from(schema.requests)
             .where(and(...conds))
             .get();
+        const cacheHitRate = cacheHitRateFor(db, conds);
         const latRows = db
             .select({ v: schema.requests.totalLatencyMs })
             .from(schema.requests)
@@ -84,7 +113,7 @@ export async function registerStatsRoutes(app) {
             p95LatencyMs: p95,
             averageTtftMs: avgTtft,
             p95TtftMs: p95Ttft,
-            cacheHitRate: success ? Number(summary?.cacheRead ?? 0) > 0 ? Number(summary?.cacheRead ?? 0) / Math.max(1, Number(summary?.inputTokens ?? 0) + Number(summary?.cacheRead ?? 0)) : 0 : 0,
+            cacheHitRate,
             gatewayCacheHitRate: total ? Number(summary?.gatewayCacheHits ?? 0) / total : 0,
             fallbackRate: total ? Number(summary?.fallbacks ?? 0) / total : 0,
         };
@@ -241,6 +270,7 @@ function buildSummary(db, fromIso, toIso) {
     const success = Number(r?.success ?? 0);
     const cacheRead = Number(r?.cacheRead ?? 0);
     const inputTokens = Number(r?.inputTokens ?? 0);
+    const cacheHitRate = cacheHitRateFor(db, conds);
     return {
         totalRequests: total,
         successfulRequests: success,
@@ -256,7 +286,7 @@ function buildSummary(db, fromIso, toIso) {
         p95LatencyMs: 0, // caller fills via percentile query
         averageTtftMs: (r?.avgTtft ?? null),
         p95TtftMs: null,
-        cacheHitRate: success ? cacheRead > 0 ? cacheRead / Math.max(1, inputTokens + cacheRead) : 0 : 0,
+        cacheHitRate,
         gatewayCacheHitRate: total ? Number(r?.gatewayCacheHits ?? 0) / total : 0,
         fallbackRate: total ? Number(r?.fallbacks ?? 0) / total : 0,
     };