@timo972/cc-router 0.10.0-rc.2 → 0.10.0-rc.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +63 -0
- package/dist/cli/cmd-accounts.js +103 -1
- package/dist/cli/cmd-start.js +15 -4
- package/dist/config/manager.js +19 -0
- package/dist/daemon/launcher.js +35 -20
- package/dist/providers/anthropic/usage-refresher.js +24 -188
- package/dist/providers/openai/token-pool.js +22 -0
- package/dist/providers/openai/usage-fetch.js +77 -0
- package/dist/providers/openai/usage.js +116 -0
- package/dist/proxy/account-patch.js +10 -0
- package/dist/proxy/account-rename.js +48 -0
- package/dist/proxy/server.js +66 -1
- package/dist/proxy/session-router.js +23 -0
- package/dist/proxy/token-pool.js +22 -0
- package/dist/proxy/usage-refresher.js +188 -0
- package/dist/ui/Dashboard.js +39 -3
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -8,8 +8,71 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- OpenAI/Codex account usage is fetched proactively, so `cc-router status`
|
|
14
|
+
shows the 5h/weekly bars immediately after a restart — matching how
|
|
15
|
+
Anthropic accounts already behaved. Codex usage previously arrived only on
|
|
16
|
+
`x-codex-*` response headers, so a freshly restarted daemon rendered empty
|
|
17
|
+
OpenAI bars until the first request happened to route there. The daemon now
|
|
18
|
+
polls the usage endpoint the Codex CLI itself reads
|
|
19
|
+
(`GET chatgpt.com/backend-api/wham/usage`) on the same bounded scheduler the
|
|
20
|
+
Anthropic usage refresher uses (staggered startup, 5-minute cadence, failure
|
|
21
|
+
backoff, identity-owned application), feeding the JSON — the payload twin of
|
|
22
|
+
the response headers, parsed under the same trust rules — through the exact
|
|
23
|
+
merge the headers go through. A failed poll keeps whatever the account
|
|
24
|
+
already knew; an expired access token is refreshed before the first fetch
|
|
25
|
+
rather than 401-ing until traffic happens to fix it.
|
|
26
|
+
- `cc-router accounts rename <id> <new-id>`. An account's id is the key its
|
|
27
|
+
routing state hangs off — in-flight counters and sticky session bindings are
|
|
28
|
+
both id-keyed — so on a running proxy the rename runs as a transaction
|
|
29
|
+
(`PATCH /cc-router/accounts/:id` with `{"id": ...}`): pool, session router,
|
|
30
|
+
and accounts.json move together, and are rolled back together if persistence
|
|
31
|
+
fails. Sticky sessions keep their prompt-cache affinity through the rename.
|
|
32
|
+
With no proxy running the record is renamed on disk. A proxy from before
|
|
33
|
+
this feature answers the PATCH with 200 having silently ignored the field —
|
|
34
|
+
that is detected and reported as an error rather than falling back to a disk
|
|
35
|
+
write its refresh loop would overwrite. The id namespace is shared across
|
|
36
|
+
both providers, so a rename onto any existing account name is refused (409).
|
|
37
|
+
|
|
38
|
+
- The health endpoint reports the daemon's version, and the dashboard shows a
|
|
39
|
+
version-mismatch banner when the daemon runs a different build than the CLI
|
|
40
|
+
rendering it. A service manager can keep an old build alive long after an
|
|
41
|
+
upgrade — launchd pins the versioned pnpm store path in its plist, so the
|
|
42
|
+
daemon silently stays on the old version across upgrades and even reboots —
|
|
43
|
+
and every log row and account view on the dashboard comes from that build.
|
|
44
|
+
Until now nothing surfaced this: a fix could ship, the package could update,
|
|
45
|
+
and the dashboard would still render the old daemon's output as if the new
|
|
46
|
+
version were broken. A daemon that reports no version at all predates the
|
|
47
|
+
field, which is itself proof it is outdated, and banners the same way.
|
|
48
|
+
- The activity list scrolls to follow its selection. The dashboard renders the
|
|
49
|
+
newest 20 of up to 50 entries, but the arrow keys would walk the selection
|
|
50
|
+
through all 50 — past row 20 the highlight left the screen while the detail
|
|
51
|
+
panel kept updating for rows that were not visible. The window now shifts by
|
|
52
|
+
one row when the selection steps past its bottom or top edge, stays put while
|
|
53
|
+
the selection moves inside it, and re-clamps when new entries push the
|
|
54
|
+
selected row (which is timestamp-anchored) out of the stored window.
|
|
55
|
+
|
|
11
56
|
### Fixed
|
|
12
57
|
|
|
58
|
+
- `cc-router stop` no longer terminates itself — or your editor sessions —
|
|
59
|
+
when it falls back to killing by port. The fallback listed every process
|
|
60
|
+
with a socket on the proxy port (`lsof -ti :port` reports both ends of
|
|
61
|
+
every connection), which included the stop CLI itself (its health-check
|
|
62
|
+
fetch leaves a keep-alive socket open) and any live Claude Code or Codex
|
|
63
|
+
session talking to the proxy. All of them received SIGTERM alongside the
|
|
64
|
+
daemon; the shell reported the stop command as `zsh: terminated`. The
|
|
65
|
+
listing now selects only the process LISTENING on the port, and killing by
|
|
66
|
+
port additionally never targets the process doing the killing.
|
|
67
|
+
- Service-mode `cc-router start` no longer reports "nothing is answering" for
|
|
68
|
+
a service that comes up moments later. The post-bootstrap health wait was
|
|
69
|
+
10 seconds, but a stop→start restart re-bootstraps a label whose process
|
|
70
|
+
exited moments earlier, and launchd throttles that spawn by up to ~10s
|
|
71
|
+
(default ThrottleInterval) before the daemon even begins booting — so the
|
|
72
|
+
window regularly closed right as throttled spawns landed, and the very
|
|
73
|
+
next `cc-router status` connected fine. The wait now outlasts the throttle
|
|
74
|
+
(30s), says what it is waiting for, and a genuine timeout points to
|
|
75
|
+
`cc-router status` before suggesting the logs.
|
|
13
76
|
- OpenAI activity rows carry the same columns as Claude ones. The OpenAI ingress
|
|
14
77
|
recorded a path but no method and no client, and the dashboard needs both
|
|
15
78
|
`method` and `path` to render the request — so those rows fell back to the
|
package/dist/cli/cmd-accounts.js
CHANGED
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
import chalk from "chalk";
|
|
2
|
-
import { loadAccounts, loadOpenAIAccounts, accountsFileExists, upsertAccountRecord, removeAccountRecordById, readConfig, serialize } from "../config/manager.js";
|
|
2
|
+
import { loadAccounts, loadOpenAIAccounts, accountsFileExists, upsertAccountRecord, removeAccountRecordById, renameAccountRecordById, readConfig, serialize } from "../config/manager.js";
|
|
3
3
|
import { saveAccounts } from "../proxy/token-refresher.js";
|
|
4
4
|
import { formatExpiry, redactToken } from "../utils/token-extractor.js";
|
|
5
5
|
import { PROXY_PORT } from "../config/paths.js";
|
|
6
6
|
import { createOpenAIAccountRecord } from "../providers/openai/account-record.js";
|
|
7
7
|
import { loginOpenAIWithDeviceCode } from "../providers/openai/device-oauth.js";
|
|
8
|
+
import { isValidAccountId } from "../proxy/account-rename.js";
|
|
8
9
|
export function registerAccounts(program) {
|
|
9
10
|
const accounts = program
|
|
10
11
|
.command("accounts")
|
|
@@ -233,6 +234,47 @@ export function registerAccounts(program) {
|
|
|
233
234
|
console.log(chalk.yellow(" No accounts left. Run: cc-router setup"));
|
|
234
235
|
}
|
|
235
236
|
});
|
|
237
|
+
accounts
|
|
238
|
+
.command("rename <id> <new-id>")
|
|
239
|
+
.description("Rename an account — its routing state and sticky sessions follow the new name")
|
|
240
|
+
.action(async (id, newId) => {
|
|
241
|
+
if (!accountsFileExists()) {
|
|
242
|
+
console.log(chalk.yellow("No accounts configured."));
|
|
243
|
+
return;
|
|
244
|
+
}
|
|
245
|
+
if (!isValidAccountId(newId)) {
|
|
246
|
+
console.log(chalk.red(`✗ "${newId}" is not a valid account name.`));
|
|
247
|
+
console.log(chalk.gray(" 1-64 characters: alphanumeric start, then letters, digits, dots, underscores, or dashes."));
|
|
248
|
+
process.exit(1);
|
|
249
|
+
}
|
|
250
|
+
const { ids: existingIds } = mergeAccountInventory(loadAccounts().map(a => a.id), loadOpenAIAccounts().map(a => a.id), await fetchLiveStats());
|
|
251
|
+
if (!existingIds.includes(id)) {
|
|
252
|
+
console.log(chalk.red(`✗ Account "${id}" not found.`));
|
|
253
|
+
console.log(chalk.gray(` Available: ${existingIds.join(", ")}`));
|
|
254
|
+
process.exit(1);
|
|
255
|
+
}
|
|
256
|
+
if (id === newId) {
|
|
257
|
+
console.log(chalk.gray(`Account is already named "${newId}".`));
|
|
258
|
+
return;
|
|
259
|
+
}
|
|
260
|
+
if (existingIds.includes(newId)) {
|
|
261
|
+
console.log(chalk.red(`✗ An account named "${newId}" already exists.`));
|
|
262
|
+
process.exit(1);
|
|
263
|
+
}
|
|
264
|
+
let result;
|
|
265
|
+
try {
|
|
266
|
+
result = await renameAccountRuntimeAware(id, newId);
|
|
267
|
+
}
|
|
268
|
+
catch (err) {
|
|
269
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
270
|
+
console.log(chalk.red(`✗ Could not rename "${id}": ${message}`));
|
|
271
|
+
process.exit(1);
|
|
272
|
+
}
|
|
273
|
+
console.log(chalk.green(`✓ Renamed "${id}" → "${newId}".`));
|
|
274
|
+
console.log(result.mode === "live"
|
|
275
|
+
? chalk.gray(" Applied to the running proxy — in-flight requests and sticky sessions follow the new name.")
|
|
276
|
+
: chalk.gray(" Saved to accounts.json — loads on next start: cc-router start"));
|
|
277
|
+
});
|
|
236
278
|
}
|
|
237
279
|
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
|
238
280
|
/** Tell the user whether the new account is already live or needs a restart. */
|
|
@@ -326,6 +368,66 @@ export async function tryRemoveAccountFromRunningProxy(id, options = {}) {
|
|
|
326
368
|
}
|
|
327
369
|
return true;
|
|
328
370
|
}
|
|
371
|
+
/**
|
|
372
|
+
* Rename an account on a running proxy. Returns false only when no proxy can
|
|
373
|
+
* be reached (the caller then renames on disk); HTTP errors are authoritative
|
|
374
|
+
* and thrown. A 200 whose returned account still carries the old id means the
|
|
375
|
+
* proxy predates rename support — its patch validation drops unknown fields
|
|
376
|
+
* and reports success having done nothing — and MUST be an error, not a
|
|
377
|
+
* fallthrough to disk: that proxy's refresh loop persists its own snapshot
|
|
378
|
+
* over accounts.json and would silently undo a disk-side rename.
|
|
379
|
+
*/
|
|
380
|
+
export async function tryRenameAccountOnRunningProxy(id, newId, options = {}) {
|
|
381
|
+
const fetchImpl = options.fetch ?? globalThis.fetch;
|
|
382
|
+
const baseUrl = (options.baseUrl ?? `http://localhost:${PROXY_PORT}`).replace(/\/+$/, "");
|
|
383
|
+
const authToken = options.authToken ?? readConfig().proxySecret;
|
|
384
|
+
let response;
|
|
385
|
+
try {
|
|
386
|
+
response = await fetchImpl(`${baseUrl}/cc-router/accounts/${encodeURIComponent(id)}`, {
|
|
387
|
+
method: "PATCH",
|
|
388
|
+
headers: {
|
|
389
|
+
"content-type": "application/json",
|
|
390
|
+
...(authToken ? { authorization: `Bearer ${authToken}` } : {}),
|
|
391
|
+
},
|
|
392
|
+
body: JSON.stringify({ id: newId }),
|
|
393
|
+
signal: AbortSignal.timeout(3_000),
|
|
394
|
+
});
|
|
395
|
+
}
|
|
396
|
+
catch {
|
|
397
|
+
return false;
|
|
398
|
+
}
|
|
399
|
+
if (!response.ok) {
|
|
400
|
+
let detail = "";
|
|
401
|
+
try {
|
|
402
|
+
const payload = await response.json();
|
|
403
|
+
if (typeof payload.error === "string")
|
|
404
|
+
detail = `: ${payload.error}`;
|
|
405
|
+
}
|
|
406
|
+
catch { /* best effort */ }
|
|
407
|
+
throw new Error(`HTTP ${response.status}${detail}`);
|
|
408
|
+
}
|
|
409
|
+
let renamedId;
|
|
410
|
+
try {
|
|
411
|
+
const payload = await response.json();
|
|
412
|
+
renamedId = payload.account?.id;
|
|
413
|
+
}
|
|
414
|
+
catch { /* fall through to the mismatch error below */ }
|
|
415
|
+
if (renamedId !== newId) {
|
|
416
|
+
throw new Error("the running proxy does not support rename (older version) — update and restart it first: cc-router stop --keep-config && cc-router start");
|
|
417
|
+
}
|
|
418
|
+
return true;
|
|
419
|
+
}
|
|
420
|
+
export async function renameAccountRuntimeAware(id, newId, dependencies = {
|
|
421
|
+
tryRenameLive: tryRenameAccountOnRunningProxy,
|
|
422
|
+
renameStored: renameAccountRecordById,
|
|
423
|
+
}) {
|
|
424
|
+
if (await dependencies.tryRenameLive(id, newId))
|
|
425
|
+
return { mode: "live" };
|
|
426
|
+
const renamed = dependencies.renameStored(id, newId);
|
|
427
|
+
if (!renamed)
|
|
428
|
+
throw new Error(`Account "${id}" disappeared before it could be renamed`);
|
|
429
|
+
return { mode: "stored", renamed };
|
|
430
|
+
}
|
|
329
431
|
export async function removeAccountRuntimeAware(id, dependencies = {
|
|
330
432
|
tryRemoveLive: tryRemoveAccountFromRunningProxy,
|
|
331
433
|
removeStored: removeAccountRecordById,
|
package/dist/cli/cmd-start.js
CHANGED
|
@@ -7,6 +7,16 @@ import { checkForUpdate, performUpdate, PKG_NAME } from "../utils/self-update.js
|
|
|
7
7
|
import { launchDaemon, waitForHealth } from "../daemon/launcher.js";
|
|
8
8
|
import { installService } from "../daemon/service.js";
|
|
9
9
|
import { getLocalIPs } from "../utils/network.js";
|
|
10
|
+
/**
|
|
11
|
+
* How long service-mode start waits for the proxy to answer after the
|
|
12
|
+
* LaunchAgent/systemd unit is loaded. This must outlast more than daemon
|
|
13
|
+
* startup: a stop→start restart re-bootstraps a label whose process exited
|
|
14
|
+
* moments earlier, and launchd throttles that spawn by up to ~10s (default
|
|
15
|
+
* ThrottleInterval) before the daemon even begins booting. The previous 10s
|
|
16
|
+
* budget ended right as throttled spawns typically landed, reporting
|
|
17
|
+
* "nothing answering" for a service that came up seconds later.
|
|
18
|
+
*/
|
|
19
|
+
export const SERVICE_HEALTH_TIMEOUT_MS = 30_000;
|
|
10
20
|
export function registerStart(program) {
|
|
11
21
|
program
|
|
12
22
|
.command("start")
|
|
@@ -82,13 +92,14 @@ export function registerStart(program) {
|
|
|
82
92
|
// Installing the service is not the same as the proxy being up: if
|
|
83
93
|
// launchd rejects the load, `installService` only warns. Verify, so a
|
|
84
94
|
// failed start is not reported as a success.
|
|
85
|
-
|
|
95
|
+
console.log(chalk.gray(` Waiting for the proxy on port ${port} (a quick restart can be throttled by the service manager)...`));
|
|
96
|
+
if (await waitForHealth(port, SERVICE_HEALTH_TIMEOUT_MS)) {
|
|
86
97
|
console.log(chalk.green(`✓ CC-Router running on port ${port}`));
|
|
87
98
|
}
|
|
88
99
|
else {
|
|
89
|
-
console.log(chalk.yellow(`\n⚠ Service configured, but nothing
|
|
90
|
-
console.log(chalk.gray(`
|
|
91
|
-
console.log(chalk.gray(`
|
|
100
|
+
console.log(chalk.yellow(`\n⚠ Service configured, but nothing answered on port ${port} within ${Math.round(SERVICE_HEALTH_TIMEOUT_MS / 1000)}s.`));
|
|
101
|
+
console.log(chalk.gray(` It may still come up — check: cc-router status`));
|
|
102
|
+
console.log(chalk.gray(` If it does not: cc-router logs, then try again: cc-router start`));
|
|
92
103
|
process.exitCode = 1;
|
|
93
104
|
return;
|
|
94
105
|
}
|
package/dist/config/manager.js
CHANGED
|
@@ -90,6 +90,25 @@ export function removeAccountRecordById(id) {
|
|
|
90
90
|
writeAccountsAtomicToPath(ACCOUNTS_PATH, existing.filter(a => a.id !== id));
|
|
91
91
|
return removed;
|
|
92
92
|
}
|
|
93
|
+
/**
|
|
94
|
+
* Rename a stored account record in place, keeping every other field. The
|
|
95
|
+
* uniqueness check spans ALL providers — both live in one accounts.json and
|
|
96
|
+
* one URL namespace, so two records sharing an id would be unaddressable.
|
|
97
|
+
* Returns the renamed record, or null if no record has `oldId`.
|
|
98
|
+
*/
|
|
99
|
+
export function renameAccountRecordById(oldId, newId) {
|
|
100
|
+
ensureConfigDir();
|
|
101
|
+
const existing = readAccountsRaw();
|
|
102
|
+
const target = existing.find(a => a.id === oldId) ?? null;
|
|
103
|
+
if (!target)
|
|
104
|
+
return null;
|
|
105
|
+
if (newId !== oldId && existing.some(a => a.id === newId)) {
|
|
106
|
+
throw new Error(`An account named "${newId}" already exists`);
|
|
107
|
+
}
|
|
108
|
+
target.id = newId;
|
|
109
|
+
writeAccountsAtomicToPath(ACCOUNTS_PATH, existing);
|
|
110
|
+
return target;
|
|
111
|
+
}
|
|
93
112
|
function normalizeAccountProvider(record) {
|
|
94
113
|
return record.provider === "openai_subscription"
|
|
95
114
|
? "openai_subscription"
|
package/dist/daemon/launcher.js
CHANGED
|
@@ -158,32 +158,47 @@ export async function killPortAndWait(port, deps, timeoutMs = 5_000) {
|
|
|
158
158
|
}
|
|
159
159
|
const DEATH_POLL_MS = 200;
|
|
160
160
|
const POST_KILL_GRACE_MS = 500;
|
|
161
|
-
|
|
161
|
+
/**
|
|
162
|
+
* PIDs that should die when the proxy on `port` is killed by port: the
|
|
163
|
+
* process LISTENING on it — never connected clients. A bare `lsof -ti :port`
|
|
164
|
+
* lists BOTH ends of every connection, which included the stop CLI itself
|
|
165
|
+
* (its health-check fetch leaves a keep-alive socket open) and any live
|
|
166
|
+
* Claude Code / Codex session talking to the proxy; `cc-router stop` then
|
|
167
|
+
* SIGTERMed all of them, itself included (`zsh: terminated`).
|
|
168
|
+
*/
|
|
169
|
+
export async function listListeningPids(port) {
|
|
162
170
|
const { execFile } = await import("child_process");
|
|
163
171
|
const { promisify } = await import("util");
|
|
164
172
|
const execFileAsync = promisify(execFile);
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
return Number.isNaN(pid) ? [] : [pid];
|
|
176
|
-
}
|
|
177
|
-
const { stdout } = await execFileAsync("lsof", ["-ti", `:${p}`]);
|
|
178
|
-
return stdout.trim().split("\n").filter(Boolean).map(Number).filter(n => !Number.isNaN(n));
|
|
179
|
-
}
|
|
180
|
-
catch {
|
|
181
|
-
return [];
|
|
173
|
+
try {
|
|
174
|
+
if (isWindows()) {
|
|
175
|
+
const { stdout } = await execFileAsync("netstat", ["-ano"]);
|
|
176
|
+
const match = stdout
|
|
177
|
+
.split("\n")
|
|
178
|
+
.find(line => line.includes(`:${port}`) && line.includes("LISTENING"));
|
|
179
|
+
if (!match)
|
|
180
|
+
return [];
|
|
181
|
+
const pid = Number(match.trim().split(/\s+/).at(-1));
|
|
182
|
+
return Number.isNaN(pid) ? [] : [pid];
|
|
182
183
|
}
|
|
183
|
-
|
|
184
|
+
const { stdout } = await execFileAsync("lsof", ["-ti", `tcp:${port}`, "-sTCP:LISTEN"]);
|
|
185
|
+
return stdout.trim().split("\n").filter(Boolean).map(Number)
|
|
186
|
+
.filter(n => !Number.isNaN(n))
|
|
187
|
+
// Belt and braces: whatever the listing says, killing by port must
|
|
188
|
+
// never target the process doing the killing.
|
|
189
|
+
.filter(pid => pid !== process.pid);
|
|
190
|
+
}
|
|
191
|
+
catch {
|
|
192
|
+
return [];
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
async function killByPort(port) {
|
|
196
|
+
const { execFile } = await import("child_process");
|
|
197
|
+
const { promisify } = await import("util");
|
|
198
|
+
const execFileAsync = promisify(execFile);
|
|
184
199
|
try {
|
|
185
200
|
return await killPortAndWait(port, {
|
|
186
|
-
listPids,
|
|
201
|
+
listPids: listListeningPids,
|
|
187
202
|
// Windows has no signals: taskkill /F is the only lever, so both the
|
|
188
203
|
// graceful and forced step map onto it.
|
|
189
204
|
kill: (pid, signal) => {
|
|
@@ -1,195 +1,31 @@
|
|
|
1
|
+
import { UsageRefresher } from "../../proxy/usage-refresher.js";
|
|
1
2
|
import { fetchAnthropicUsage } from "./usage.js";
|
|
2
|
-
const SUCCESS_REFRESH_MS = 5 * 60_000;
|
|
3
|
-
const FAILURE_BACKOFF_MS = [60_000, 2 * 60_000, 5 * 60_000, 15 * 60_000];
|
|
4
|
-
const DEFAULT_STARTUP_STAGGER_MS = 250;
|
|
5
|
-
const MAX_CONCURRENT_REFRESHES = 2;
|
|
6
|
-
const RECONCILE_INTERVAL_MS = 60_000;
|
|
7
3
|
/**
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
4
|
+
* The Anthropic instantiation of the shared usage scheduler (see
|
|
5
|
+
* proxy/usage-refresher.ts for the timing/identity guarantees): fetches the
|
|
6
|
+
* OAuth usage endpoint and lands snapshots on `account.rateLimits.usage`,
|
|
7
|
+
* downgrading prior data to "stale" (or marking "unavailable") on failure.
|
|
11
8
|
*/
|
|
12
|
-
export class AnthropicUsageRefresher {
|
|
13
|
-
pool;
|
|
14
|
-
fetchUsage;
|
|
15
|
-
now;
|
|
16
|
-
startupStaggerMs;
|
|
17
|
-
maxConcurrent;
|
|
18
|
-
timers = new Map();
|
|
19
|
-
inFlight = new Map();
|
|
20
|
-
resolvers = new Map();
|
|
21
|
-
queued = new Set();
|
|
22
|
-
failures = new Map();
|
|
23
|
-
reconcileTimer;
|
|
24
|
-
active = 0;
|
|
25
|
-
started = false;
|
|
26
|
-
stopped = false;
|
|
9
|
+
export class AnthropicUsageRefresher extends UsageRefresher {
|
|
27
10
|
constructor(pool, options = {}) {
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
return;
|
|
47
|
-
this.started = false;
|
|
48
|
-
this.stopped = true;
|
|
49
|
-
if (this.reconcileTimer)
|
|
50
|
-
clearInterval(this.reconcileTimer);
|
|
51
|
-
this.reconcileTimer = undefined;
|
|
52
|
-
for (const timer of this.timers.values())
|
|
53
|
-
clearTimeout(timer);
|
|
54
|
-
this.timers.clear();
|
|
55
|
-
for (const account of this.queued) {
|
|
56
|
-
this.resolvers.get(account)?.({ ok: false, reason: "network" });
|
|
57
|
-
this.resolvers.delete(account);
|
|
58
|
-
this.inFlight.delete(account);
|
|
59
|
-
}
|
|
60
|
-
this.queued.clear();
|
|
61
|
-
}
|
|
62
|
-
/** Join or initiate the one usage refresh owned by this exact account object. */
|
|
63
|
-
refreshNow(account) {
|
|
64
|
-
const existing = this.inFlight.get(account);
|
|
65
|
-
if (existing)
|
|
66
|
-
return existing;
|
|
67
|
-
if (this.stopped)
|
|
68
|
-
return Promise.resolve({ ok: false, reason: "network" });
|
|
69
|
-
const scheduled = this.timers.get(account);
|
|
70
|
-
if (scheduled) {
|
|
71
|
-
clearTimeout(scheduled);
|
|
72
|
-
this.timers.delete(account);
|
|
73
|
-
}
|
|
74
|
-
let resolve;
|
|
75
|
-
const pending = new Promise(done => { resolve = done; });
|
|
76
|
-
this.inFlight.set(account, pending);
|
|
77
|
-
this.resolvers.set(account, resolve);
|
|
78
|
-
this.queued.add(account);
|
|
79
|
-
this.runQueued();
|
|
80
|
-
return pending;
|
|
81
|
-
}
|
|
82
|
-
/** Start a refresh after any request that was already in flight at call time. */
|
|
83
|
-
refreshAfterCurrent(account) {
|
|
84
|
-
const existing = this.inFlight.get(account);
|
|
85
|
-
return existing
|
|
86
|
-
? existing.then(() => this.refreshNow(account))
|
|
87
|
-
: this.refreshNow(account);
|
|
88
|
-
}
|
|
89
|
-
reconcile(startup = false) {
|
|
90
|
-
if (!this.started)
|
|
91
|
-
return;
|
|
92
|
-
const accounts = this.pool.getAll();
|
|
93
|
-
const current = new Set(accounts);
|
|
94
|
-
for (const [account, timer] of this.timers) {
|
|
95
|
-
if (!current.has(account)) {
|
|
96
|
-
clearTimeout(timer);
|
|
97
|
-
this.timers.delete(account);
|
|
98
|
-
}
|
|
99
|
-
}
|
|
100
|
-
for (const account of [...this.queued]) {
|
|
101
|
-
if (!current.has(account)) {
|
|
102
|
-
this.queued.delete(account);
|
|
103
|
-
this.resolvers.get(account)?.({ ok: false, reason: "network" });
|
|
104
|
-
this.resolvers.delete(account);
|
|
105
|
-
this.inFlight.delete(account);
|
|
106
|
-
}
|
|
107
|
-
}
|
|
108
|
-
for (const account of this.failures.keys()) {
|
|
109
|
-
if (!current.has(account))
|
|
110
|
-
this.failures.delete(account);
|
|
111
|
-
}
|
|
112
|
-
accounts.forEach((account, index) => {
|
|
113
|
-
if (!this.timers.has(account) && !this.inFlight.has(account) && !this.queued.has(account)) {
|
|
114
|
-
if (startup)
|
|
115
|
-
this.schedule(account, index * this.startupStaggerMs);
|
|
116
|
-
else
|
|
117
|
-
void this.refreshNow(account);
|
|
118
|
-
}
|
|
11
|
+
const now = options.now ?? Date.now;
|
|
12
|
+
super(pool, {
|
|
13
|
+
fetchUsage: options.fetchUsage ?? fetchAnthropicUsage,
|
|
14
|
+
cancelledResult: () => ({ ok: false, reason: "network" }),
|
|
15
|
+
applyResult: (account, result) => {
|
|
16
|
+
if (result.ok) {
|
|
17
|
+
account.rateLimits = { ...account.rateLimits, usage: result.snapshot };
|
|
18
|
+
return;
|
|
19
|
+
}
|
|
20
|
+
const prior = account.rateLimits.usage;
|
|
21
|
+
const usage = prior
|
|
22
|
+
? { ...prior, fetchStatus: "stale" }
|
|
23
|
+
: { modelLimits: [], fetchedAt: now(), fetchStatus: "unavailable" };
|
|
24
|
+
account.rateLimits = { ...account.rateLimits, usage };
|
|
25
|
+
},
|
|
26
|
+
...(options.now !== undefined ? { now: options.now } : {}),
|
|
27
|
+
...(options.startupStaggerMs !== undefined ? { startupStaggerMs: options.startupStaggerMs } : {}),
|
|
28
|
+
...(options.maxConcurrent !== undefined ? { maxConcurrent: options.maxConcurrent } : {}),
|
|
119
29
|
});
|
|
120
30
|
}
|
|
121
|
-
schedule(account, delayMs) {
|
|
122
|
-
if (!this.started || this.pool.findById(account.id) !== account)
|
|
123
|
-
return;
|
|
124
|
-
const timer = setTimeout(() => {
|
|
125
|
-
this.timers.delete(account);
|
|
126
|
-
this.reconcile();
|
|
127
|
-
void this.refreshNow(account);
|
|
128
|
-
}, delayMs);
|
|
129
|
-
this.timers.set(account, timer);
|
|
130
|
-
}
|
|
131
|
-
runQueued() {
|
|
132
|
-
while (this.active < this.maxConcurrent) {
|
|
133
|
-
const account = this.queued.values().next().value;
|
|
134
|
-
if (!account)
|
|
135
|
-
return;
|
|
136
|
-
this.queued.delete(account);
|
|
137
|
-
if (this.pool.findById(account.id) !== account) {
|
|
138
|
-
this.resolvers.get(account)?.({ ok: false, reason: "network" });
|
|
139
|
-
this.resolvers.delete(account);
|
|
140
|
-
this.inFlight.delete(account);
|
|
141
|
-
continue;
|
|
142
|
-
}
|
|
143
|
-
this.startRequest(account);
|
|
144
|
-
}
|
|
145
|
-
}
|
|
146
|
-
startRequest(account) {
|
|
147
|
-
this.active++;
|
|
148
|
-
const operation = this.inFlight.get(account);
|
|
149
|
-
const resolve = this.resolvers.get(account);
|
|
150
|
-
if (!operation || !resolve) {
|
|
151
|
-
this.active--;
|
|
152
|
-
return;
|
|
153
|
-
}
|
|
154
|
-
void (async () => {
|
|
155
|
-
let result;
|
|
156
|
-
try {
|
|
157
|
-
result = await this.fetchUsage(account);
|
|
158
|
-
}
|
|
159
|
-
catch {
|
|
160
|
-
result = { ok: false, reason: "network" };
|
|
161
|
-
}
|
|
162
|
-
if (this.pool.findById(account.id) === account) {
|
|
163
|
-
this.apply(account, result);
|
|
164
|
-
if (this.started)
|
|
165
|
-
this.schedule(account, this.nextDelay(account, result));
|
|
166
|
-
}
|
|
167
|
-
if (this.inFlight.get(account) === operation)
|
|
168
|
-
this.inFlight.delete(account);
|
|
169
|
-
this.resolvers.delete(account);
|
|
170
|
-
this.active--;
|
|
171
|
-
resolve(result);
|
|
172
|
-
this.reconcile();
|
|
173
|
-
this.runQueued();
|
|
174
|
-
})();
|
|
175
|
-
}
|
|
176
|
-
apply(account, result) {
|
|
177
|
-
if (result.ok) {
|
|
178
|
-
this.failures.delete(account);
|
|
179
|
-
account.rateLimits = { ...account.rateLimits, usage: result.snapshot };
|
|
180
|
-
return;
|
|
181
|
-
}
|
|
182
|
-
this.failures.set(account, (this.failures.get(account) ?? 0) + 1);
|
|
183
|
-
const prior = account.rateLimits.usage;
|
|
184
|
-
const usage = prior
|
|
185
|
-
? { ...prior, fetchStatus: "stale" }
|
|
186
|
-
: { modelLimits: [], fetchedAt: this.now(), fetchStatus: "unavailable" };
|
|
187
|
-
account.rateLimits = { ...account.rateLimits, usage };
|
|
188
|
-
}
|
|
189
|
-
nextDelay(account, result) {
|
|
190
|
-
if (result.ok)
|
|
191
|
-
return SUCCESS_REFRESH_MS;
|
|
192
|
-
const failures = this.failures.get(account) ?? 1;
|
|
193
|
-
return FAILURE_BACKOFF_MS[Math.min(failures - 1, FAILURE_BACKOFF_MS.length - 1)];
|
|
194
|
-
}
|
|
195
31
|
}
|
|
@@ -175,6 +175,28 @@ export class OpenAITokenPool {
|
|
|
175
175
|
findById(id) {
|
|
176
176
|
return this.accounts.find(account => account.id === id) ?? null;
|
|
177
177
|
}
|
|
178
|
+
/**
|
|
179
|
+
* Change an account's id in place. The in-flight counter is keyed by id and
|
|
180
|
+
* must move with it — an open lease's release() re-reads `account.id`, so
|
|
181
|
+
* after a rename it decrements the NEW key, which would otherwise never
|
|
182
|
+
* have been incremented. Cooldowns are keyed by the account object and
|
|
183
|
+
* follow the rename untouched. Returns the renamed account, or null if the
|
|
184
|
+
* id was not found. Callers are responsible for id-uniqueness and for
|
|
185
|
+
* session-binding migration. Mirrors `TokenPool.renameAccount`.
|
|
186
|
+
*/
|
|
187
|
+
renameAccount(oldId, newId) {
|
|
188
|
+
const account = this.findById(oldId);
|
|
189
|
+
if (!account)
|
|
190
|
+
return null;
|
|
191
|
+
if (newId !== oldId) {
|
|
192
|
+
const load = this.inFlight.get(oldId);
|
|
193
|
+
this.inFlight.delete(oldId);
|
|
194
|
+
if (load !== undefined)
|
|
195
|
+
this.inFlight.set(newId, load);
|
|
196
|
+
account.id = newId;
|
|
197
|
+
}
|
|
198
|
+
return account;
|
|
199
|
+
}
|
|
178
200
|
/**
|
|
179
201
|
* Drop every piece of per-account routing state after the account has been
|
|
180
202
|
* removed from the shared `accounts` array. The array splice itself is owned
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
import { applyCodexRateLimits } from "./account-state.js";
|
|
2
|
+
import { parseCodexUsagePayload } from "./usage.js";
|
|
3
|
+
import { UsageRefresher } from "../../proxy/usage-refresher.js";
|
|
4
|
+
/**
|
|
5
|
+
* The endpoint the Codex CLI's own backend client reads rate limits from
|
|
6
|
+
* (codex-rs/backend-client, ChatGPT path style). Answers a bearer-only GET
|
|
7
|
+
* with the JSON twin of the `x-codex-*` response headers — which is what
|
|
8
|
+
* makes usage visible without burning a request through /codex/responses.
|
|
9
|
+
*/
|
|
10
|
+
export const CODEX_USAGE_ENDPOINT = "https://chatgpt.com/backend-api/wham/usage";
|
|
11
|
+
export async function fetchCodexUsage(account, options = {}) {
|
|
12
|
+
const fetchImpl = options.fetch ?? globalThis.fetch;
|
|
13
|
+
const now = options.now ?? Date.now;
|
|
14
|
+
let response;
|
|
15
|
+
try {
|
|
16
|
+
response = await fetchImpl(CODEX_USAGE_ENDPOINT, {
|
|
17
|
+
headers: { authorization: `Bearer ${account.accessToken}` },
|
|
18
|
+
signal: AbortSignal.timeout(10_000),
|
|
19
|
+
});
|
|
20
|
+
}
|
|
21
|
+
catch {
|
|
22
|
+
return { ok: false, reason: "network" };
|
|
23
|
+
}
|
|
24
|
+
if (response.status === 401 || response.status === 403)
|
|
25
|
+
return { ok: false, reason: "auth" };
|
|
26
|
+
if (!response.ok)
|
|
27
|
+
return { ok: false, reason: "http" };
|
|
28
|
+
let body;
|
|
29
|
+
try {
|
|
30
|
+
body = await response.json();
|
|
31
|
+
}
|
|
32
|
+
catch {
|
|
33
|
+
return { ok: false, reason: "malformed" };
|
|
34
|
+
}
|
|
35
|
+
const update = parseCodexUsagePayload(body, now());
|
|
36
|
+
return update ? { ok: true, update } : { ok: false, reason: "malformed" };
|
|
37
|
+
}
|
|
38
|
+
/**
|
|
39
|
+
* The OpenAI/Codex instantiation of the shared usage scheduler: polls the
|
|
40
|
+
* usage endpoint and feeds the result through `applyCodexRateLimits` — the
|
|
41
|
+
* same merge the response headers go through — so the dashboard shows an
|
|
42
|
+
* account's windows right after startup instead of only after its first
|
|
43
|
+
* routed request. A failed fetch keeps whatever the account already knew;
|
|
44
|
+
* unlike the Anthropic snapshot there is no per-fetch staleness field, and
|
|
45
|
+
* header-fed data must not be erased by a flaky poll.
|
|
46
|
+
*/
|
|
47
|
+
export class OpenAIUsageRefresher extends UsageRefresher {
|
|
48
|
+
constructor(pool, options = {}) {
|
|
49
|
+
const now = options.now ?? Date.now;
|
|
50
|
+
const fetchUsage = options.fetchUsage ?? ((account) => fetchCodexUsage(account, { now }));
|
|
51
|
+
const prepare = options.prepare;
|
|
52
|
+
super(pool, {
|
|
53
|
+
fetchUsage: async (account) => {
|
|
54
|
+
if (prepare) {
|
|
55
|
+
let ready;
|
|
56
|
+
try {
|
|
57
|
+
ready = await prepare(account);
|
|
58
|
+
}
|
|
59
|
+
catch {
|
|
60
|
+
ready = false;
|
|
61
|
+
}
|
|
62
|
+
if (!ready)
|
|
63
|
+
return { ok: false, reason: "auth" };
|
|
64
|
+
}
|
|
65
|
+
return fetchUsage(account);
|
|
66
|
+
},
|
|
67
|
+
cancelledResult: () => ({ ok: false, reason: "network" }),
|
|
68
|
+
applyResult: (account, result) => {
|
|
69
|
+
if (result.ok)
|
|
70
|
+
applyCodexRateLimits(account, result.update, now());
|
|
71
|
+
},
|
|
72
|
+
...(options.now !== undefined ? { now: options.now } : {}),
|
|
73
|
+
...(options.startupStaggerMs !== undefined ? { startupStaggerMs: options.startupStaggerMs } : {}),
|
|
74
|
+
...(options.maxConcurrent !== undefined ? { maxConcurrent: options.maxConcurrent } : {}),
|
|
75
|
+
});
|
|
76
|
+
}
|
|
77
|
+
}
|