ldrouter 1.17.9 → 1.18.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@ Lightweight self-hosted LLM gateway with a polished admin UI. Presents stable Op
6
6
 
7
7
  - OpenAI-compatible and Anthropic-compatible gateways (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/models`)
8
8
  - Selective model discovery (Fetch → select → import) with **Select All**
9
- - Virtual **combos** (fallback or weighted round-robin) and one-hop **aliases**
9
+ - Ordered **combos** with fallback routing and one-hop **aliases**
10
10
  - Provider API keys encrypted at rest with AES-256-GCM
11
11
  - Per-key `ld-` bearer tokens (SHA-256 digest storage, displayed once)
12
12
  - IP allow/deny (IPv4 + IPv6 CIDR), trusted-proxy configuration
@@ -19,8 +19,6 @@ Lightweight self-hosted LLM gateway with a polished admin UI. Presents stable Op
19
19
  - Consistent backup / restore (SQLite snapshot + checksum + schema validation)
20
20
  - Prometheus `/metrics`, structured logs, graceful shutdown
21
21
  - One distributable npm package, multi-stage Dockerfile, Docker Compose
22
- - Native Codex OAuth account pools with encrypted JSON/JSONL import, refresh/rotation, account-aware routing, and admin management
23
- - Native Qoder account pools driven by personal access tokens, with encrypted storage, live model-catalog discovery, and account-aware routing
24
22
 
25
23
  ## Quick start
26
24
 
@@ -44,69 +42,6 @@ latedev-router --host 0.0.0.0 --port 8787
44
42
 
45
43
  The data directory defaults to `~/.latedev-router/` and can be overridden via `LATEDEV_DATA_DIR` or `--data-dir`.
46
44
 
47
- ### Codex OAuth setup
48
-
49
- 1. Set `LATEDEV_MASTER_KEY` before creating or importing credentials. Use a strong random base64 key; it encrypts Codex access, refresh, and ID tokens at rest.
50
- 2. In **Providers**, create a provider with type **Codex**. Codex providers do not use the generic API-key field.
51
- 3. Open the Codex account panel and import a redacted copy of a Codex OAuth `auth.json`, a JSON array, an `{ "accounts": [...] }` wrapper, or JSONL with one record per line. Records may contain a `tokens` object with access/refresh/ID tokens. For example:
52
-
53
- `{ "accountId": "acct-…masked", "email": "admin@example.invalid", "tokens": { "accessToken": "[REDACTED]", "refreshToken": "[REDACTED]" } }`
54
-
55
- JSONL stores one similarly redacted object per line; an array uses the same records: `[ { "accountId": "acct-…masked", "tokens": { "accessToken": "[REDACTED]" } } ]`.
56
-
57
- Alternatively, use **Connect Codex** to authorize a ChatGPT account in the browser. The dialog shows the Codex CLI PKCE authorize URL, waits for the loopback callback, and also accepts a pasted callback URL or bare authorization code. The PKCE verifier and the authorization code stay server-side and never appear in the UI. The Codex CLI callback (`http://localhost:1455/auth/callback`) is captured by `GET /oauth/codex/callback`; because browsers block that cross-origin redirect, paste the address bar URL into step 2 when the auto-capture page fails to load.
58
- 4. Review the preview and import only the records you want. Account/workspace identity is preferred for deduplication; email alone never merges unrelated accounts. Re-importing the same identity updates its encrypted tokens while preserving its enabled state.
59
-
60
- Raw tokens are accepted only by the authenticated import pipeline and are never returned in previews, API responses, UI state, audit logs, request logs, errors, or database backups. JWT claims are decoded for metadata only; token signatures are not verified locally. Tokens are refreshed proactively near expiry and once after an upstream 401/403, with rotated values persisted atomically.
61
-
62
- The import endpoint requires the normal admin session and CSRF token (`x-csrf-token`) for mutations, including multipart uploads. Use HTTPS for remote administration and protect the master key like any encryption key. Back up the SQLite data directory consistently; restoring encrypted credentials requires the matching master key, otherwise re-save/re-import credentials after restore.
63
-
64
- ZIP upload and automatic Codex CLI config-file generation/mutation are not included in this release. LateDev Router does not modify Codex CLI files.
65
-
66
- The account panel shows the 5-hour and weekly quota windows with reset countdowns, per-account and bulk usage refresh, weekly reset credits, an opt-in 5-hour window auto-start, and a **Test** control that probes the upstream account without exposing tokens. Routing order is set by dragging rows; the saved order is the fallback order the router uses. **Delete** is permanent and erases the stored encrypted credentials — past request logs are kept but lose the account reference.
67
-
68
- **A quota-exhausted account leaves the pool on its own.** When an account refuses for quota reasons — Codex `429 The usage limit has been reached`, or Qoder `out of credits` — it is marked `down` *and disabled*, so the very next request picks a different account in the pool rather than retrying the spent one. This applies to direct `codex/...` and `qoder/...` models too, not only combos. Re-enable the account in the panel once its window resets. Both Codex usage and Qoder credits are also refreshed in the background after a successful request (throttled, and never on the response path), so the panel figures track real usage without pressing **Refresh**.
69
-
70
- ### Qoder setup
71
-
72
- Qoder providers use a personal access token (PAT) instead of an API key. Create one at `https://qoder.com/account/integrations`; it starts with `pt-`.
73
-
74
- 1. Set `LATEDEV_MASTER_KEY` before adding a token. It encrypts both the PAT and the derived job token at rest.
75
- 2. In **Providers**, click **Add Qoder**. The server creates the provider with its fixed endpoint — there is no base URL or API key to fill in.
76
- 3. In the Qoder account panel, click **Add token** and paste the PAT, or **Import tokens** to paste/upload a list: one token per line, a JSON array of strings, an `{ "accounts": [...] }` wrapper, or JSONL of strings/objects (`token`, `pt_token`, `personal_token`, `access_token`; optional `label`/`name`/`email`). Only `pt-` tokens are accepted — a `dt-` device token or a `jt-` job token is rejected with that reason rather than failing later upstream.
77
-
78
- Adding a token exchanges it immediately for a short-lived job token and fetches the model list; inference then runs against `api2.qoder.sh` using that job token. The PAT is kept so a new job token can always be minted — the operator never has to touch it again. If the token works but the model list cannot be fetched, the account is still stored (health `unknown`, with the error recorded) so you can retry with **Catalog**.
79
-
80
- Because the gateway serves model discovery from that live catalog, a new account has no routable models until its first successful fetch. Use **Import models** on the provider row to pull the discovered models in.
81
-
82
- **Operational caveat:** a revoked or expired PAT is a durable failure. The account is marked `down` with "personal access token rejected — replace it" and routing skips it; nothing retries it forever. Re-add the account with a fresh token. Deleting an account is permanent and erases the stored encrypted PAT — past request logs are kept but lose the account reference.
83
-
84
- The panel shows the masked user id, job-token expiry, when the catalog was last fetched, health, and an enabled toggle. Routing order is set by dragging rows, exactly as with Codex.
85
-
86
- ### Qoder usage, Credits and promotions
87
-
88
- Qoder does not meter models in tokens: its chat stream carries no usage block at all, so request logs
89
- show zero tokens for every Qoder model and the panel's **Credits** column is the account's real
90
- usage. It is read from Qoder's quota API (`GET /api/v2/quota/usage` on `openapi.qoder.sh`) and shows
91
- the plan / add-on / org buckets, the percentage used, and whether the account is exhausted.
92
-
93
- **Credits and free models are different things, and the panel shows both.** Qoder keeps promoting a
94
- model — currently `qmodel_38max`, the Qwen3.8-Max route — by marking its catalog entry `is_free`.
95
- Such a model spends no Credits, so an account at **zero Credits** still answers on it while every
96
- other model returns an envelope `403 code 112` with a `pricingUrl`. That pairing is what makes an
97
- "out of credits" account look partly broken: the panel labels the column **No plan credits** next to
98
- **Free now: qmodel_38max** so the contradiction is visible instead of looking like a router fault.
99
-
100
- There is an upstream edge worth knowing when reading logs: a quota refusal arrives as **HTTP 200**
101
- with a `403` envelope inside the streamed body, so it is not an HTTP error at the transport layer.
102
-
103
- Use **Refresh credits** to re-read the snapshot for every account. It deliberately reuses the live
104
- `job token` instead of exchanging the PAT again — a fresh exchange invalidates the token a
105
- concurrent request may be using.
106
-
107
- **Test** sends one real message, because a catalog probe cannot see a quota refusal. It picks a
108
- promotion-covered model when the account has one, so testing never spends Credits.
109
-
110
45
  ## Environment variables
111
46
 
112
47
  | Variable | Description | Default |
@@ -198,8 +198,6 @@ export async function startApp() {
198
198
  process.once('SIGTERM', () => void close('SIGTERM'));
199
199
  process.once('SIGINT', () => void close('SIGINT'));
200
200
  await app.listen({ host: cfg.host, port: cfg.port });
201
- const { startCodexAutostart } = await import('./providers/codex-autostart.js');
202
- startCodexAutostart();
203
201
  return app;
204
202
  }
205
203
  // re-export for convenience
@@ -188,7 +188,7 @@ function buildInitialSchemaSql() {
188
188
  name TEXT NOT NULL,
189
189
  slug TEXT NOT NULL UNIQUE,
190
190
  public_model_id TEXT NOT NULL UNIQUE,
191
- mode TEXT NOT NULL CHECK (mode IN ('fallback','weighted_round_robin')),
191
+ mode TEXT NOT NULL CHECK (mode IN ('fallback')),
192
192
  enabled INTEGER NOT NULL DEFAULT 1,
193
193
  max_total_attempts INTEGER NOT NULL DEFAULT 3,
194
194
  fallback_on_connection INTEGER NOT NULL DEFAULT 1,
@@ -208,7 +208,6 @@ function buildInitialSchemaSql() {
208
208
  combo_id TEXT NOT NULL REFERENCES combos(id) ON DELETE CASCADE,
209
209
  model_id TEXT NOT NULL REFERENCES models(id) ON DELETE RESTRICT,
210
210
  position INTEGER NOT NULL,
211
- weight INTEGER NOT NULL DEFAULT 1,
212
211
  enabled INTEGER NOT NULL DEFAULT 1
213
212
  );
214
213
  CREATE UNIQUE INDEX IF NOT EXISTS uniq_combo_model ON combo_members(combo_id, model_id);
@@ -209,7 +209,7 @@ export const combos = sqliteTable('combos', {
209
209
  name: text('name').notNull(),
210
210
  slug: text('slug').notNull().unique(),
211
211
  publicModelId: text('public_model_id').notNull().unique(),
212
- mode: text('mode', { enum: ['fallback', 'weighted_round_robin'] }).notNull(),
212
+ mode: text('mode', { enum: ['fallback'] }).notNull(),
213
213
  enabled: integer('enabled', { mode: 'boolean' }).notNull().notNull().default(true),
214
214
  maxTotalAttempts: integer('max_total_attempts').notNull().default(3),
215
215
  fallbackOnConnection: integer('fallback_on_connection', { mode: 'boolean' }).notNull().notNull().default(true),
@@ -235,7 +235,6 @@ export const comboMembers = sqliteTable('combo_members', {
235
235
  .notNull()
236
236
  .references(() => models.id, { onDelete: 'restrict' }),
237
237
  position: integer('position').notNull(),
238
- weight: integer('weight').notNull().default(1),
239
238
  enabled: integer('enabled', { mode: 'boolean' }).notNull().notNull().default(true),
240
239
  }, (t) => ({
241
240
  uniqMember: uniqueIndex('uniq_combo_model').on(t.comboId, t.modelId),
@@ -432,6 +432,7 @@ export class GatewayRunner {
432
432
  const candidate = {
433
433
  modelId: m.id,
434
434
  publicModelId: m.publicModelId,
435
+ upstreamModelId: m.upstreamModelId,
435
436
  providerId: m.providerId,
436
437
  enabled: m.enabled,
437
438
  providerEnabled: p.enabled,
@@ -489,6 +490,7 @@ export class GatewayRunner {
489
490
  return models.map((m) => ({
490
491
  modelId: m.id,
491
492
  publicModelId: m.publicModelId,
493
+ upstreamModelId: m.upstreamModelId,
492
494
  providerId: m.providerId,
493
495
  enabled: m.enabled,
494
496
  providerEnabled: providerEnabled.get(m.providerId),
@@ -508,7 +510,7 @@ export class GatewayRunner {
508
510
  return this.runNonStreamingAttempt(req, candidate, cfg, ctx);
509
511
  }
510
512
  async runNonStreamingAttempt(req, candidate, cfg, ctx) {
511
- const upstreamModel = candidate.publicModelId.split('/').slice(1).join('/');
513
+ const upstreamModel = candidate.upstreamModelId;
512
514
  if (cfg.type === 'codex') {
513
515
  const out = await callCodexNonStreaming({ baseUrl: cfg.baseUrl, accountId: cfg.codexAccountId ?? '', accountRecordId: cfg.accountRecordId, customHeaders: cfg.customHeaders, totalTimeoutMs: cfg.totalTimeoutMs }, { ...req.canonical, model: upstreamModel });
514
516
  return { statusCode: out.status, ttftMs: null, upstreamRequestId: out.upstreamRequestId, usage: out.usage, result: { text: out.text, toolCalls: out.toolCalls, finishReason: out.finishReason } };
@@ -569,7 +571,7 @@ export class GatewayRunner {
569
571
  return { statusCode: call.status, ttftMs: call.ttftMs, upstreamRequestId: call.upstreamRequestId, usage, result };
570
572
  }
571
573
  async runStreamingAttempt(req, ctx, candidate, cfg, onStreamStart, onFirstToken) {
572
- const upstreamModel = candidate.publicModelId.split('/').slice(1).join('/');
574
+ const upstreamModel = candidate.upstreamModelId;
573
575
  const encoder = req.protocol === 'openai' ? openaiStreamEncoder : anthropicStreamEncoder;
574
576
  // Hard streaming invariant: the client SSE head is NOT written until the
575
577
  // upstream confirms by delivering its first chunk (or completes). Before
@@ -6,11 +6,13 @@ import { requireAdminAuth } from '../../auth/middleware.js';
6
6
  import { recordAudit } from '../../db/repositories/audit.js';
7
7
  import { uuid } from '../../auth/ids.js';
8
8
  import { GatewayError } from '../../errors.js';
9
- const MemberSpec = z.object({ modelId: z.string(), position: z.number().int().min(0), weight: z.number().int().min(1).default(1), enabled: z.boolean().default(true) });
9
+ import { circuitBlocks } from '../../routing/circuit.js';
10
+ const MemberSpec = z.object({ modelId: z.string(), position: z.number().int().min(0), enabled: z.boolean().default(true) });
11
+ const OptionalSlug = z.preprocess((value) => typeof value === 'string' && value.trim() === '' ? undefined : value, z.string().min(1).max(64).optional());
10
12
  const ComboCreate = z.object({
11
13
  name: z.string().min(1).max(128),
12
- slug: z.string().min(1).max(64).optional(),
13
- mode: z.enum(['fallback', 'weighted_round_robin']),
14
+ slug: OptionalSlug,
15
+ mode: z.literal('fallback').default('fallback'),
14
16
  maxTotalAttempts: z.number().int().min(1).max(8).optional(),
15
17
  enabled: z.boolean().optional(),
16
18
  fallbackOnConnection: z.boolean().optional(),
@@ -84,6 +86,45 @@ function assertModelsExist(db, members) {
84
86
  throw new GatewayError('invalid_request_error', 'One or more members are not valid physical models', { status: 400 });
85
87
  }
86
88
  }
89
+ const capabilityKeys = ['chat', 'responses', 'streaming', 'tools', 'structured_output', 'image_input', 'audio_input', 'reasoning', 'embeddings'];
90
+ function modelCapabilities(model) {
91
+ try {
92
+ return JSON.parse(model.capabilitiesJson);
93
+ }
94
+ catch {
95
+ return {};
96
+ }
97
+ }
98
+ function memberStatus(model, provider, enabled) {
99
+ if (!enabled)
100
+ return { state: 'unavailable', reason: 'member_disabled' };
101
+ if (!model)
102
+ return { state: 'unavailable', reason: 'model_missing' };
103
+ if (!provider?.enabled)
104
+ return { state: 'unavailable', reason: 'provider_disabled' };
105
+ if (provider.healthState === 'down')
106
+ return { state: 'unavailable', reason: 'provider_down' };
107
+ if (provider.healthState === 'circuit_open')
108
+ return { state: 'unavailable', reason: 'circuit_open' };
109
+ if (!model.enabled)
110
+ return { state: 'unavailable', reason: 'model_disabled' };
111
+ if (!model.upstreamAvailable)
112
+ return { state: 'unavailable', reason: 'upstream_unavailable' };
113
+ if (circuitBlocks(model.providerId, provider.cbCooldownSeconds))
114
+ return { state: 'unavailable', reason: 'circuit_open' };
115
+ if (modelCapabilities(model).chat === false)
116
+ return { state: 'unavailable', reason: 'chat_capability_unavailable' };
117
+ return { state: 'ready', reason: provider.healthState === 'degraded' ? 'provider_degraded' : 'ready' };
118
+ }
119
+ function capabilityWarnings(models) {
120
+ return capabilityKeys.filter((key) => {
121
+ const values = new Set(models.map((model) => {
122
+ const value = modelCapabilities(model)[key];
123
+ return value === true ? 'yes' : value === false ? 'no' : 'unknown';
124
+ }));
125
+ return values.size > 1;
126
+ });
127
+ }
87
128
  export async function registerComboRoutes(app) {
88
129
  app.addHook('preHandler', requireAdminAuth);
89
130
  app.get('/api/admin/combos', async () => {
@@ -92,13 +133,14 @@ export async function registerComboRoutes(app) {
92
133
  const allMembers = db.select().from(schema.comboMembers).all();
93
134
  const models = db.select().from(schema.models).all();
94
135
  const modelMap = new Map(models.map((m) => [m.id, m]));
136
+ const providerMap = new Map(db.select().from(schema.providers).all().map((p) => [p.id, p]));
95
137
  return {
96
138
  combos: combos.map((c) => {
97
139
  const members = allMembers.filter((m) => m.comboId === c.id);
98
- const healthy = members.filter((m) => {
99
- const model = modelMap.get(m.modelId);
100
- return model && model.enabled && model.upstreamAvailable;
101
- });
140
+ const statuses = members.map((m) => memberStatus(modelMap.get(m.modelId), providerMap.get(modelMap.get(m.modelId)?.providerId ?? ''), m.enabled));
141
+ const readyCount = statuses.filter((status) => status.state === 'ready').length;
142
+ const reasons = [...new Set(statuses.filter((status) => status.state === 'unavailable').map((status) => status.reason))];
143
+ const comboModels = members.map((m) => modelMap.get(m.modelId)).filter((m) => Boolean(m));
102
144
  return {
103
145
  id: c.id,
104
146
  name: c.name,
@@ -107,7 +149,10 @@ export async function registerComboRoutes(app) {
107
149
  mode: c.mode,
108
150
  enabled: c.enabled,
109
151
  memberCount: members.length,
110
- healthyMemberCount: healthy.length,
152
+ healthyMemberCount: readyCount,
153
+ status: !c.enabled ? 'disabled' : readyCount > 0 ? 'ready' : 'unavailable',
154
+ unavailableReasons: reasons,
155
+ capabilityWarnings: capabilityWarnings(comboModels),
111
156
  };
112
157
  }),
113
158
  };
@@ -124,6 +169,9 @@ export async function registerComboRoutes(app) {
124
169
  .sort((a, b) => a.position - b.position);
125
170
  const models = db.select().from(schema.models).all();
126
171
  const modelMap = new Map(models.map((m) => [m.id, m]));
172
+ const providers = db.select().from(schema.providers).all();
173
+ const providerMap = new Map(providers.map((p) => [p.id, p]));
174
+ const comboModels = members.map((m) => modelMap.get(m.modelId));
127
175
  return {
128
176
  combo: {
129
177
  ...c,
@@ -131,15 +179,85 @@ export async function registerComboRoutes(app) {
131
179
  id: m.id,
132
180
  modelId: m.modelId,
133
181
  publicModelId: modelMap.get(m.modelId)?.publicModelId ?? '',
182
+ upstreamModelId: modelMap.get(m.modelId)?.upstreamModelId ?? '',
134
183
  displayName: modelMap.get(m.modelId)?.displayName ?? '',
135
- providerSlug: '',
184
+ providerSlug: providerMap.get(modelMap.get(m.modelId)?.providerId ?? '')?.slug ?? '',
136
185
  position: m.position,
137
- weight: m.weight,
138
186
  enabled: m.enabled,
187
+ upstreamAvailable: modelMap.get(m.modelId)?.upstreamAvailable ?? false,
188
+ providerEnabled: providerMap.get(modelMap.get(m.modelId)?.providerId ?? '')?.enabled ?? false,
189
+ providerHealth: providerMap.get(modelMap.get(m.modelId)?.providerId ?? '')?.healthState ?? 'unknown',
190
+ circuitOpen: circuitBlocks(modelMap.get(m.modelId)?.providerId ?? '', providerMap.get(modelMap.get(m.modelId)?.providerId ?? '')?.cbCooldownSeconds ?? 0),
191
+ capabilities: modelMap.get(m.modelId) ? modelCapabilities(modelMap.get(m.modelId)) : {},
192
+ status: memberStatus(modelMap.get(m.modelId), providerMap.get(modelMap.get(m.modelId)?.providerId ?? ''), m.enabled),
139
193
  })),
194
+ capabilityWarnings: capabilityWarnings(comboModels.filter((m) => Boolean(m))),
140
195
  },
141
196
  };
142
197
  });
198
+ app.post('/api/admin/combos/:id/test', async (req) => {
199
+ const { id } = req.params;
200
+ const db = getDb();
201
+ const combo = db.select().from(schema.combos).where(eq(schema.combos.id, id)).get();
202
+ if (!combo)
203
+ throw new GatewayError('invalid_request_error', 'Combo not found', { status: 404 });
204
+ const { GatewayRunner } = await import('../../gateway/runner.js');
205
+ const runner = new GatewayRunner();
206
+ const canonicalReq = {
207
+ model: combo.publicModelId,
208
+ messages: [{ role: 'user', content: [{ type: 'text', text: 'Bạn là model gì?' }] }],
209
+ stream: false,
210
+ maxOutputTokens: 256,
211
+ temperature: 0.7,
212
+ };
213
+ const ctx = {
214
+ requestId: 'combo-test-' + uuid(),
215
+ clientIp: req.ip,
216
+ protocol: 'openai',
217
+ endpoint: 'chat/completions',
218
+ requestedModel: combo.publicModelId,
219
+ key: null,
220
+ reply: { raw: {} },
221
+ };
222
+ const gatewayReq = { canonical: canonicalReq, protocol: 'openai', endpoint: 'chat/completions' };
223
+ let outcome;
224
+ try {
225
+ outcome = await runner.execute(gatewayReq, ctx);
226
+ }
227
+ catch (e) {
228
+ if (e instanceof GatewayError)
229
+ throw e;
230
+ throw new GatewayError('gateway_error', e.message, { cause: e });
231
+ }
232
+ const models = db.select().from(schema.models).all();
233
+ const providers = db.select().from(schema.providers).all();
234
+ const modelMap = new Map(models.map((m) => [m.id, m]));
235
+ const providerMap = new Map(providers.map((p) => [p.id, p]));
236
+ const selected = outcome.attempts.find((attempt) => attempt.success);
237
+ const selectedModel = selected ? modelMap.get(selected.modelId) : undefined;
238
+ recordAudit({ action: 'combo.test', success: outcome.success, targetType: 'combo', targetId: id, targetName: combo.publicModelId, ip: req.ip });
239
+ return {
240
+ success: outcome.success,
241
+ text: outcome.text ?? '',
242
+ latencyMs: outcome.latencyMs,
243
+ selectedModel: selectedModel ? {
244
+ publicModelId: selectedModel.publicModelId,
245
+ upstreamModelId: selectedModel.upstreamModelId,
246
+ providerSlug: providerMap.get(selectedModel.providerId)?.slug ?? '',
247
+ } : null,
248
+ attempts: outcome.attempts.map((attempt) => {
249
+ const model = modelMap.get(attempt.modelId);
250
+ return {
251
+ publicModelId: model?.publicModelId ?? attempt.modelId,
252
+ upstreamModelId: model?.upstreamModelId ?? '',
253
+ providerName: attempt.providerName,
254
+ latencyMs: attempt.latencyMs,
255
+ success: attempt.success,
256
+ failureReason: attempt.failureReason,
257
+ };
258
+ }),
259
+ };
260
+ });
143
261
  app.post('/api/admin/combos', async (req) => {
144
262
  const body = ComboCreate.parse(req.body);
145
263
  const db = getDb();
@@ -170,7 +288,7 @@ export async function registerComboRoutes(app) {
170
288
  configVersion: 1,
171
289
  }).run();
172
290
  for (const m of body.members) {
173
- tx.insert(schema.comboMembers).values({ id: uuid(), comboId: id, modelId: m.modelId, position: m.position, weight: m.weight ?? 1, enabled: m.enabled ?? true }).run();
291
+ tx.insert(schema.comboMembers).values({ id: uuid(), comboId: id, modelId: m.modelId, position: m.position, enabled: m.enabled ?? true }).run();
174
292
  }
175
293
  });
176
294
  recordAudit({ action: 'combo.create', success: true, targetType: 'combo', targetId: id, targetName: body.name, ip: req.ip, metadata: { members: body.members.length, mode: body.mode } });
@@ -223,7 +341,7 @@ export async function registerComboRoutes(app) {
223
341
  tx.update(schema.combos).set(update).where(eq(schema.combos.id, body.id)).run();
224
342
  tx.delete(schema.comboMembers).where(eq(schema.comboMembers.comboId, body.id)).run();
225
343
  for (const m of body.members) {
226
- tx.insert(schema.comboMembers).values({ id: uuid(), comboId: body.id, modelId: m.modelId, position: m.position, weight: m.weight ?? 1, enabled: m.enabled ?? true }).run();
344
+ tx.insert(schema.comboMembers).values({ id: uuid(), comboId: body.id, modelId: m.modelId, position: m.position, enabled: m.enabled ?? true }).run();
227
345
  }
228
346
  });
229
347
  }
@@ -2,6 +2,7 @@
2
2
  import { and, desc, gte, sql } from 'drizzle-orm';
3
3
  import { getDb, schema } from '../../db/index.js';
4
4
  import { requireAdminAuth } from '../../auth/middleware.js';
5
+ import { circuitBlocks } from '../../routing/circuit.js';
5
6
  export async function registerDashboardRoutes(app) {
6
7
  app.addHook('preHandler', requireAdminAuth);
7
8
  app.get('/api/admin/dashboard', async () => {
@@ -27,6 +28,27 @@ export async function registerDashboardRoutes(app) {
27
28
  .orderBy(desc(schema.requests.createdAt))
28
29
  .limit(5)
29
30
  .all();
31
+ const recentAttempts = db.select().from(schema.requestAttempts).where(gte(schema.requestAttempts.startedAt, new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString())).orderBy(desc(schema.requestAttempts.startedAt)).limit(500).all();
32
+ const models = db.select().from(schema.models).all();
33
+ const providers = db.select().from(schema.providers).all();
34
+ const providerMap = new Map(providers.map((p) => [p.id, p]));
35
+ const modelHealth = models.map((m) => {
36
+ const provider = providerMap.get(m.providerId);
37
+ const attempts = recentAttempts.filter((a) => a.modelId === m.id);
38
+ const lastFailure = attempts.find((a) => !a.success);
39
+ const circuitOpen = circuitBlocks(m.providerId, provider?.cbCooldownSeconds ?? 0);
40
+ const state = !provider?.enabled ? 'disabled' : provider.healthState === 'down' ? 'provider_down' : provider.healthState === 'circuit_open' || circuitOpen ? 'circuit_open' : !m.enabled ? 'disabled' : !m.upstreamAvailable ? 'unavailable' : 'ready';
41
+ return { id: m.id, publicModelId: m.publicModelId, upstreamModelId: m.upstreamModelId, providerId: m.providerId, providerName: provider?.name ?? null, state, attempts: attempts.length, failures: attempts.filter((a) => !a.success).length, lastFailureAt: lastFailure?.startedAt ?? null, lastFailureReason: lastFailure?.failureReason ?? null };
42
+ });
43
+ const modelHealthMap = new Map(modelHealth.map((m) => [m.id, m]));
44
+ const combos = db.select().from(schema.combos).all();
45
+ const comboMembers = db.select().from(schema.comboMembers).all();
46
+ const comboHealth = combos.map((combo) => {
47
+ const members = comboMembers.filter((member) => member.comboId === combo.id);
48
+ const readyMembers = members.filter((member) => member.enabled && modelHealthMap.get(member.modelId)?.state === 'ready');
49
+ const reasons = [...new Set(members.filter((member) => !member.enabled || modelHealthMap.get(member.modelId)?.state !== 'ready').map((member) => modelHealthMap.get(member.modelId)?.state ?? 'model_missing'))];
50
+ return { id: combo.id, publicModelId: combo.publicModelId, enabled: combo.enabled, state: !combo.enabled ? 'disabled' : readyMembers.length > 0 ? 'ready' : 'unavailable', readyMembers: readyMembers.length, memberCount: members.length, reasons };
51
+ });
30
52
  return {
31
53
  today: {
32
54
  total: Number(today?.total ?? 0),
@@ -50,6 +72,8 @@ export async function registerDashboardRoutes(app) {
50
72
  errorMessage: r.errorMessage,
51
73
  httpStatus: r.httpStatus,
52
74
  })),
75
+ models: modelHealth,
76
+ combos: comboHealth,
53
77
  };
54
78
  });
55
79
  }
@@ -5,6 +5,7 @@ import { requireAdminAuth } from '../../auth/middleware.js';
5
5
  import { recordAudit } from '../../db/repositories/audit.js';
6
6
  import { uuid } from '../../auth/ids.js';
7
7
  import { GatewayError } from '../../errors.js';
8
+ import { circuitBlocks } from '../../routing/circuit.js';
8
9
  const ImportModelsBody = z.object({
9
10
  providerId: z.string(),
10
11
  modelIds: z.array(z.string().min(1)).min(1),
@@ -66,6 +67,9 @@ export async function registerModelRoutes(app) {
66
67
  displayName: m.displayName,
67
68
  enabled: m.enabled,
68
69
  upstreamAvailable: m.upstreamAvailable,
70
+ providerEnabled: providerMap.get(m.providerId)?.enabled ?? false,
71
+ providerHealth: providerMap.get(m.providerId)?.healthState ?? 'unknown',
72
+ circuitOpen: circuitBlocks(m.providerId, providerMap.get(m.providerId)?.cbCooldownSeconds ?? 0),
69
73
  capabilities: safeJson(m.capabilitiesJson),
70
74
  discoveredCapabilities: m.discoveredMetadataJson ? safeJson(m.discoveredMetadataJson) : null,
71
75
  maxContextTokens: m.maxContextTokens,
@@ -1,20 +1,17 @@
1
1
  import { z } from 'zod';
2
2
  import { sql, eq } from 'drizzle-orm';
3
- import { getDb, getRawDb, schema } from '../../db/index.js';
3
+ import { getDb, schema } from '../../db/index.js';
4
4
  import { requireAdminAuth } from '../../auth/middleware.js';
5
5
  import { recordAudit } from '../../db/repositories/audit.js';
6
6
  import { encryptSecret, decryptSecret, encryptCustomHeaders, decryptCustomHeaders, isMasterKeyConfigured } from '../../auth/crypto.js';
7
7
  import { uuid, slugify } from '../../auth/ids.js';
8
8
  import { GatewayError } from '../../errors.js';
9
9
  import { probeProvider, discoverProviderModels } from '../../providers/index.js';
10
- import { getPooledProvider } from '../../providers/pooled.js';
11
- import { redactString } from '../../security/redact.js';
12
10
  const ProviderCreate = z.object({
13
- // A pool provider is nothing but a group: the admin clicks once and the server owns every field.
14
- // Compatible providers still need name, endpoint, and credential up front (checked in the handler).
11
+ // Providers use one encrypted API key and a compatible HTTP endpoint.
15
12
  name: z.string().min(1).max(128).optional(),
16
13
  slug: z.string().min(1).max(64).optional(),
17
- type: z.enum(['openai', 'anthropic', 'codex', 'qoder']),
14
+ type: z.enum(['openai', 'anthropic']),
18
15
  baseUrl: z.string().url().max(512).optional(),
19
16
  apiKey: z.string().min(1).max(20000).optional(),
20
17
  customHeaders: z.record(z.string(), z.string()).optional(),
@@ -51,7 +48,7 @@ export async function registerProviderRoutes(app) {
51
48
  app.addHook('preHandler', requireAdminAuth);
52
49
  app.get('/api/admin/providers', async () => {
53
50
  const db = getDb();
54
- const providers = db.select().from(schema.providers).all();
51
+ const providers = db.select().from(schema.providers).all().filter((provider) => provider.type === 'openai' || provider.type === 'anthropic');
55
52
  const modelCounts = db
56
53
  .select({ providerId: schema.models.providerId, c: sql `COUNT(*)` })
57
54
  .from(schema.models)
@@ -64,17 +61,13 @@ export async function registerProviderRoutes(app) {
64
61
  });
65
62
  app.post('/api/admin/providers', async (req) => {
66
63
  const body = ProviderCreate.parse(req.body);
67
- const pool = getPooledProvider(body.type);
68
- const defaults = pool?.defaults;
69
- const name = body.name ?? defaults?.name;
70
- // Pool defaults win over the request: a stored Codex base URL pointing anywhere but
71
- // chatgpt.com makes every upstream call 404 (see CODEX_BASE_URL).
72
- const baseUrl = defaults?.baseUrl ?? body.baseUrl;
64
+ const name = body.name;
65
+ const baseUrl = body.baseUrl;
73
66
  if (!name)
74
67
  throw new GatewayError('invalid_request_error', 'Provider name is required', { status: 400 });
75
68
  // apiKey before baseUrl: a compatible provider created without either must be told the
76
69
  // credential is missing (the UI always supplies baseUrl from its dialog default).
77
- if (!pool && !body.apiKey) {
70
+ if (!body.apiKey) {
78
71
  throw new GatewayError('invalid_request_error', 'API key is required for this provider type', { status: 400 });
79
72
  }
80
73
  if (!baseUrl)
@@ -84,14 +77,9 @@ export async function registerProviderRoutes(app) {
84
77
  const slug = slugify(body.slug ?? name);
85
78
  const dup = db.select().from(schema.providers).where(eq(schema.providers.slug, slug)).get();
86
79
  if (dup) {
87
- // Pool path carries the code spec §5.7 names, so the UI can render "already added".
88
- if (pool)
89
- throw new GatewayError('invalid_request_error', `A ${dup.name} provider already exists ('${dup.slug}')`, { status: 400, code: 'slug_taken' });
90
80
  throw new GatewayError('invalid_request_error', `Provider slug '${slug}' is already in use`, { status: 400 });
91
81
  }
92
- // A pool type has no API-key field (spec §5.7): a key sent with `{type}` must not be encrypted
93
- // onto the row.
94
- const enc = !pool && body.apiKey ? encryptSecret(body.apiKey) : null;
82
+ const enc = encryptSecret(body.apiKey);
95
83
  const headersEnc = body.customHeaders ? encryptCustomHeaders(body.customHeaders) : null;
96
84
  const id = uuid();
97
85
  db.insert(schema.providers).values({
@@ -125,15 +113,12 @@ export async function registerProviderRoutes(app) {
125
113
  if (!p)
126
114
  throw new GatewayError('invalid_request_error', 'Provider not found', { status: 404 });
127
115
  const update = { updatedAt: new Date().toISOString() };
128
- // A pool type has no API-key field (spec §5.7). Same rule as the create path: skip the whole
129
- // encryption block, so an existing key on the row is not disturbed either.
130
- const pool = getPooledProvider(p.type);
131
116
  if (body.name)
132
117
  update.name = body.name;
133
118
  if (body.slug)
134
119
  update.slug = slugify(body.slug);
135
120
  if (body.baseUrl)
136
- update.baseUrl = pool?.defaults.baseUrl ?? body.baseUrl;
121
+ update.baseUrl = body.baseUrl;
137
122
  if (body.enabled !== undefined)
138
123
  update.enabled = body.enabled;
139
124
  if (body.connectTimeoutMs !== undefined)
@@ -150,7 +135,7 @@ export async function registerProviderRoutes(app) {
150
135
  update.cbFailureThreshold = body.cbFailureThreshold;
151
136
  if (body.cbCooldownSeconds !== undefined)
152
137
  update.cbCooldownSeconds = body.cbCooldownSeconds;
153
- if (body.apiKey && !pool) {
138
+ if (body.apiKey) {
154
139
  const enc = encryptSecret(body.apiKey);
155
140
  update.encryptedApiKey = enc.ciphertext;
156
141
  update.apiKeyNonce = enc.nonce;
@@ -178,25 +163,14 @@ export async function registerProviderRoutes(app) {
178
163
  recordAudit({ action: 'provider.soft_disable', success: true, targetType: 'provider', targetId: id, targetName: p.name, ip: req.ip });
179
164
  return { ok: true, softDisabled: true };
180
165
  }
181
- // A pool provider owns its account pool. <table>.provider_id is ON DELETE RESTRICT, so the
182
- // accounts must go in the same transaction or the delete fails with a raw SQLite constraint
183
- // error (which surfaces as an opaque 500 "Gateway error").
184
- const pool = getPooledProvider(p.type);
185
- let pooledAccountsDeleted = 0;
186
166
  try {
187
- getRawDb().transaction(() => {
188
- // pool.accountsTable is a fixed registry literal, never user input.
189
- if (pool)
190
- pooledAccountsDeleted = getRawDb().prepare(`DELETE FROM ${pool.accountsTable} WHERE provider_id=?`).run(id).changes;
191
- db.delete(schema.providers).where(eq(schema.providers.id, id)).run();
192
- })();
167
+ db.delete(schema.providers).where(eq(schema.providers.id, id)).run();
193
168
  }
194
169
  catch (error) {
195
170
  throw new GatewayError('invalid_request_error', 'Provider is still referenced and cannot be deleted', { status: 409, cause: error });
196
171
  }
197
- recordAudit({ action: 'provider.delete', success: true, targetType: 'provider', targetId: id, targetName: p.name, ip: req.ip, metadata: { pooledAccountsDeleted } });
198
- // codexAccountsDeleted kept: pre-existing clients and tests read that field name.
199
- return { ok: true, pooledAccountsDeleted, codexAccountsDeleted: pooledAccountsDeleted };
172
+ recordAudit({ action: 'provider.delete', success: true, targetType: 'provider', targetId: id, targetName: p.name, ip: req.ip });
173
+ return { ok: true };
200
174
  });
201
175
  app.post('/api/admin/providers/:id/test', async (req) => {
202
176
  const { id } = req.params;
@@ -204,13 +178,6 @@ export async function registerProviderRoutes(app) {
204
178
  const p = db.select().from(schema.providers).where(eq(schema.providers.id, id)).get();
205
179
  if (!p)
206
180
  throw new GatewayError('invalid_request_error', 'Provider not found', { status: 404 });
207
- const pool = getPooledProvider(p.type);
208
- if (pool) {
209
- const result = await pool.probe({ id: p.id, baseUrl: p.baseUrl, totalTimeoutMs: p.totalTimeoutMs });
210
- db.update(schema.providers).set({ healthState: result.ok ? 'healthy' : 'down', updatedAt: new Date().toISOString() }).where(eq(schema.providers.id, id)).run();
211
- recordAudit({ action: 'provider.test', success: result.ok, targetType: 'provider', targetId: id, targetName: p.name, ip: req.ip, metadata: { detail: redactString(result.detail) } });
212
- return { ...result, detail: redactString(result.detail) };
213
- }
214
181
  if (!p.encryptedApiKey || !p.apiKeyNonce)
215
182
  throw new GatewayError('invalid_request_error', 'Provider credentials are missing', { status: 501 });
216
183
  const apiKey = decryptSecret({ ciphertext: p.encryptedApiKey, nonce: p.apiKeyNonce, version: p.apiKeyVersion });
@@ -239,24 +206,18 @@ export async function registerProviderRoutes(app) {
239
206
  if (!p)
240
207
  throw new GatewayError('invalid_request_error', 'Provider not found', { status: 404 });
241
208
  let discovered;
242
- const pool = getPooledProvider(p.type);
243
- if (pool) {
244
- discovered = await pool.discover({ id: p.id, baseUrl: p.baseUrl, totalTimeoutMs: p.totalTimeoutMs });
245
- }
246
- else {
247
- if (!p.encryptedApiKey || !p.apiKeyNonce)
248
- throw new GatewayError('invalid_request_error', 'Provider credentials are missing', { status: 501 });
249
- const apiKey = decryptSecret({ ciphertext: p.encryptedApiKey, nonce: p.apiKeyNonce, version: p.apiKeyVersion });
250
- const headers = decryptCustomHeaders(p.customHeadersEncrypted && p.customHeadersNonce ? { ciphertext: p.customHeadersEncrypted, nonce: p.customHeadersNonce, version: 1 } : null);
251
- discovered = await discoverProviderModels({
252
- type: p.type,
253
- baseUrl: p.baseUrl,
254
- apiKey,
255
- customHeaders: headers,
256
- connectTimeoutMs: 5000,
257
- totalTimeoutMs: 30000,
258
- });
259
- }
209
+ if (!p.encryptedApiKey || !p.apiKeyNonce)
210
+ throw new GatewayError('invalid_request_error', 'Provider credentials are missing', { status: 501 });
211
+ const apiKey = decryptSecret({ ciphertext: p.encryptedApiKey, nonce: p.apiKeyNonce, version: p.apiKeyVersion });
212
+ const headers = decryptCustomHeaders(p.customHeadersEncrypted && p.customHeadersNonce ? { ciphertext: p.customHeadersEncrypted, nonce: p.customHeadersNonce, version: 1 } : null);
213
+ discovered = await discoverProviderModels({
214
+ type: p.type,
215
+ baseUrl: p.baseUrl,
216
+ apiKey,
217
+ customHeaders: headers,
218
+ connectTimeoutMs: 5000,
219
+ totalTimeoutMs: 30000,
220
+ });
260
221
  const existing = db
261
222
  .select()
262
223
  .from(schema.models)