llm-relay 0.12.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -295,6 +295,25 @@ Claude Code subagent frontmatter (`model:`), which accepts a full model id but n
295
295
  list. A pool is the indirection that gives those callers ranking and failover. `pool` is a
296
296
  reserved provider name; configuring a provider called `pool` fails at load.
297
297
 
298
+ **What failover actually does** (both `/v1/messages` and `/v1/chat/completions`):
299
+
300
+ - **429 / 5xx / 400 / 404** → the candidate is recorded as a breaker failure and the next one is
301
+ tried. A `Retry-After` sets that candidate's cooldown for exactly as long as the provider asked.
302
+ - **401 / 403** → the next candidate is tried, but the fault is recorded on its own axis rather
303
+ than as ill health, so `llm-relay candidates` shows it as `AUTH 401` instead of hiding it. It
304
+ expires after 5 minutes, so a rotated key recovers with no restart.
305
+ - **A genuine client 4xx** (413, 422, …) → returned as-is. Every other candidate would reject it
306
+ identically.
307
+ - **Every candidate failed** → the last real upstream error, not a synthesized one.
308
+
309
+ Responses carry **`x-llm-relay-served-by`**: the deployment that served, or on an error every
310
+ deployment that was tried, in order.
311
+
312
+ ⚠ **A pool routes to fewer members than it lists** when some declare an `authEnv` that is unset —
313
+ those are dropped before ranking, so a 14-member pool can resolve to 7 and the config's *tenth*
314
+ entry can legitimately be the one that answers. `llm-relay candidates` reports the count.
315
+ Background: [docs/pool-failover.md](docs/pool-failover.md).
316
+
298
317
  ### Quieting the onboarding nudge (`leave_me_alone`)
299
318
 
300
319
  `llm-relay onboard` walks every known provider and prompts for the keys you are missing. For a
@@ -518,7 +537,7 @@ silently at request time.
518
537
 
519
538
  ### Discovery endpoint (`GET /registry`) — for a dispatcher
520
539
 
521
- For a caller that does its own selection (e.g. audit-tools dispatch, which weighs
540
+ For a caller that does its own selection (an external dispatcher weighing
522
541
  quota / rate limits / token budget), `GET http://127.0.0.1:8791/registry` returns one
523
542
  coherent JSON view:
524
543
 
@@ -629,7 +648,7 @@ headroom's backend-agnostic memory/learn layer. So stack it for context-fit, not
629
648
 
630
649
  ## Design
631
650
 
632
- Consumers (audit-tools dispatch, plain `claude` CLI) point `ANTHROPIC_BASE_URL` at this proxy; it validates one backend per request. Target *selection* / token-prediction is a separate concern (the router/auditor), deliberately not here. For architecture, invariants, and the script inventory, see [CLAUDE.md](CLAUDE.md).
651
+ Consumers (an external dispatcher, plain `claude` CLI) point `ANTHROPIC_BASE_URL` at this proxy; it validates one backend per request. Target *selection* / token-prediction is a separate concern (the router/auditor), deliberately not here. For architecture, invariants, and the script inventory, see [CLAUDE.md](CLAUDE.md).
633
652
 
634
653
  ## Dev
635
654
 
package/dist/backend.d.ts CHANGED
@@ -13,6 +13,25 @@ export declare const ERROR_ORIGIN_HEADER = "x-llm-relay-error-origin";
13
13
  export type ErrorOrigin = "upstream" | "local";
14
14
  /** Read the origin marker off a Response, when it carries one. */
15
15
  export declare function errorOrigin(res: Response): ErrorOrigin | null;
16
+ /**
17
+ * Response header naming the deployment that actually answered — the (provider, model) left
18
+ * standing after pool expansion, benchmark ranking, breaker demotion and failover.
19
+ *
20
+ * A pool request's most basic debugging question is "who served this?", and until now the only
21
+ * way to answer it was to correlate timestamps against the proxy's own log. When every candidate
22
+ * fails it carries the list that was tried instead, so an exhausted pool is self-describing.
23
+ */
24
+ export declare const SERVED_BY_HEADER = "x-llm-relay-served-by";
25
+ /**
26
+ * The provider's `Retry-After` in milliseconds, or null.
27
+ *
28
+ * Accepts both RFC 9110 forms — delta-seconds and an HTTP-date — because providers use both
29
+ * (groq sends seconds, some CDNs in front of a provider send a date). A date in the past, a
30
+ * negative delta or an unparseable value yields null rather than 0: "the provider said nothing
31
+ * usable" and "the provider said retry immediately" call for different cooldowns, and treating
32
+ * garbage as 0 would silently disable the backoff this exists to honour.
33
+ */
34
+ export declare function parseRetryAfterMs(value: string | null | undefined, now?: number): number | null;
16
35
  /**
17
36
  * Fetch the resolved provider target and return an ANTHROPIC-shaped `Response`,
18
37
  * regardless of the backend's native wire format. For kind="anthropic" this is a
@@ -32,13 +51,34 @@ export declare function fetchBackend(target: ResolvedTarget, args: {
32
51
  }, fetchFn?: typeof fetch): Promise<Response>;
33
52
  /** Map a non-streaming OpenAI chat completion into an Anthropic message. */
34
53
  export declare function openAiResponseToAnthropic(j: Record<string, unknown>, model: string): object;
54
+ /**
55
+ * Coerce an upstream error body into the OpenAI error envelope — WITHOUT rewriting one that
56
+ * already conforms.
57
+ *
58
+ * The OpenAI front promises "OpenAI in, OpenAI out", but on the error path it returned whatever
59
+ * shape the provider chose. Gemini wraps its error in a JSON ARRAY (`[{"error":{…}}]`); a client
60
+ * reading `response.choices[0]` gets `undefined` from that and reports a malformed completion,
61
+ * so a plain 429 surfaces as "the model returned garbage" — which is exactly how a rate limit
62
+ * cost two days of debugging on the caller's side.
63
+ *
64
+ * Rules, in order:
65
+ * - already `{error:{…}}` → returned BYTE-EXACT. A conforming provider's message, code and
66
+ * type are its own to state, and rewriting them would lose detail.
67
+ * - `[{error:{…}}, …]` → unwrapped to the element. Same fields, now at the top level.
68
+ * - anything else (HTML, text,
69
+ * a bare string, empty) → wrapped, with the original preserved as the message.
70
+ *
71
+ * Returns null when the body is already conforming, so the caller can stream the original bytes
72
+ * rather than re-serialize them.
73
+ */
74
+ export declare function normalizeOpenAiErrorBody(body: string, status: number): string | null;
35
75
  /**
36
76
  * OpenAI-compatible FRONT: an OpenAI `/chat/completions` request comes in, its `model`
37
77
  * has already been resolved to a provider target by namespace/tier routing. For an
38
78
  * openai-kind target this is a routing reverse-proxy — rewrite `model` to the backend
39
79
  * id, inject the backend key, and stream the upstream OpenAI response straight back
40
80
  * (OpenAI in, OpenAI out — no translation). This is the transport a dispatcher (e.g.
41
- * audit-tools) consumes to reach many backends behind one endpoint.
81
+ * an external dispatcher) consumes to reach many backends behind one endpoint.
42
82
  *
43
83
  * anthropic-kind targets are not served on the OpenAI front (they need OpenAI↔Anthropic
44
84
  * translation and are not the dispatcher use case) — a clean 400, never a mistranslation.
package/dist/backend.js CHANGED
@@ -15,6 +15,41 @@ export function errorOrigin(res) {
15
15
  const v = res.headers.get(ERROR_ORIGIN_HEADER);
16
16
  return v === "upstream" || v === "local" ? v : null;
17
17
  }
18
+ /**
19
+ * Response header naming the deployment that actually answered — the (provider, model) left
20
+ * standing after pool expansion, benchmark ranking, breaker demotion and failover.
21
+ *
22
+ * A pool request's most basic debugging question is "who served this?", and until now the only
23
+ * way to answer it was to correlate timestamps against the proxy's own log. When every candidate
24
+ * fails it carries the list that was tried instead, so an exhausted pool is self-describing.
25
+ */
26
+ export const SERVED_BY_HEADER = "x-llm-relay-served-by";
27
+ /**
28
+ * The provider's `Retry-After` in milliseconds, or null.
29
+ *
30
+ * Accepts both RFC 9110 forms — delta-seconds and an HTTP-date — because providers use both
31
+ * (groq sends seconds, some CDNs in front of a provider send a date). A date in the past, a
32
+ * negative delta or an unparseable value yields null rather than 0: "the provider said nothing
33
+ * usable" and "the provider said retry immediately" call for different cooldowns, and treating
34
+ * garbage as 0 would silently disable the backoff this exists to honour.
35
+ */
36
+ export function parseRetryAfterMs(value, now = Date.now()) {
37
+ if (typeof value !== "string")
38
+ return null;
39
+ const raw = value.trim();
40
+ if (raw.length === 0)
41
+ return null;
42
+ // delta-seconds — integer per the RFC, but providers do emit fractions ("20.45").
43
+ if (/^\d+(\.\d+)?$/.test(raw)) {
44
+ const ms = Math.round(Number(raw) * 1000);
45
+ return Number.isFinite(ms) && ms >= 0 ? ms : null;
46
+ }
47
+ const at = Date.parse(raw);
48
+ if (Number.isNaN(at))
49
+ return null;
50
+ const delta = at - now;
51
+ return delta > 0 ? delta : null;
52
+ }
18
53
  /**
19
54
  * Fetch the resolved provider target and return an ANTHROPIC-shaped `Response`,
20
55
  * regardless of the backend's native wire format. For kind="anthropic" this is a
@@ -88,7 +123,12 @@ export async function fetchBackend(target, args, fetchFn = fetch) {
88
123
  ? ` — model "${target.model}" is not served by provider "${target.provider}" (a model can be listed in /models and still 404 here)`
89
124
  : "";
90
125
  // The provider really answered with this status — the body is reworded, the origin is not.
91
- return anthropicError(res.status, `openai backend HTTP ${res.status}${hint}: ${body.slice(0, 300)}`, "upstream");
126
+ // `Retry-After` is carried onto the synthesized error: this response is a NEW Response, so
127
+ // without this the one header stating when the provider will serve again was destroyed here,
128
+ // and neither the breaker's cooldown nor the client's backoff could ever honour it.
129
+ return anthropicError(res.status, `openai backend HTTP ${res.status}${hint}: ${body.slice(0, 300)}`, "upstream", {
130
+ ...retryAfterHeader(res.headers),
131
+ });
92
132
  }
93
133
  if (args.wantsStream && res.body) {
94
134
  const anthStream = handleUniversalStreamRequest(res.body, "openai", "anthropic");
@@ -138,10 +178,15 @@ export function openAiResponseToAnthropic(j, model) {
138
178
  usage: { input_tokens: usage.prompt_tokens ?? 0, output_tokens: usage.completion_tokens ?? 0 },
139
179
  };
140
180
  }
141
- function anthropicError(status, message, origin) {
181
+ /** The upstream's `Retry-After`, as a header object to spread, or `{}` when it sent none. */
182
+ function retryAfterHeader(hh) {
183
+ const v = hh.get("retry-after");
184
+ return v ? { "retry-after": v } : {};
185
+ }
186
+ function anthropicError(status, message, origin, extra = {}) {
142
187
  return new Response(JSON.stringify({ type: "error", error: { type: "api_error", message } }), {
143
188
  status,
144
- headers: { "content-type": "application/json", [ERROR_ORIGIN_HEADER]: origin },
189
+ headers: { "content-type": "application/json", [ERROR_ORIGIN_HEADER]: origin, ...extra },
145
190
  });
146
191
  }
147
192
  function openaiError(status, message, origin) {
@@ -150,13 +195,58 @@ function openaiError(status, message, origin) {
150
195
  headers: { "content-type": "application/json", [ERROR_ORIGIN_HEADER]: origin },
151
196
  });
152
197
  }
198
+ /**
199
+ * Coerce an upstream error body into the OpenAI error envelope — WITHOUT rewriting one that
200
+ * already conforms.
201
+ *
202
+ * The OpenAI front promises "OpenAI in, OpenAI out", but on the error path it returned whatever
203
+ * shape the provider chose. Gemini wraps its error in a JSON ARRAY (`[{"error":{…}}]`); a client
204
+ * reading `response.choices[0]` gets `undefined` from that and reports a malformed completion,
205
+ * so a plain 429 surfaces as "the model returned garbage" — which is exactly how a rate limit
206
+ * cost two days of debugging on the caller's side.
207
+ *
208
+ * Rules, in order:
209
+ * - already `{error:{…}}` → returned BYTE-EXACT. A conforming provider's message, code and
210
+ * type are its own to state, and rewriting them would lose detail.
211
+ * - `[{error:{…}}, …]` → unwrapped to the element. Same fields, now at the top level.
212
+ * - anything else (HTML, text,
213
+ * a bare string, empty) → wrapped, with the original preserved as the message.
214
+ *
215
+ * Returns null when the body is already conforming, so the caller can stream the original bytes
216
+ * rather than re-serialize them.
217
+ */
218
+ export function normalizeOpenAiErrorBody(body, status) {
219
+ let parsed;
220
+ try {
221
+ parsed = JSON.parse(body);
222
+ }
223
+ catch {
224
+ const message = body.trim().slice(0, 2000) || `upstream returned HTTP ${status} with an empty body`;
225
+ return JSON.stringify({ error: { message, type: "upstream_error", code: status } });
226
+ }
227
+ const hasError = (v) => typeof v === "object" && v !== null && typeof v.error === "object" && v.error !== null;
228
+ if (hasError(parsed))
229
+ return null; // conforms — do not touch it
230
+ if (Array.isArray(parsed)) {
231
+ const wrapped = parsed.find(hasError);
232
+ if (wrapped)
233
+ return JSON.stringify(wrapped);
234
+ }
235
+ return JSON.stringify({
236
+ error: {
237
+ message: body.trim().slice(0, 2000) || `upstream returned HTTP ${status} with an empty body`,
238
+ type: "upstream_error",
239
+ code: status,
240
+ },
241
+ });
242
+ }
153
243
  /**
154
244
  * OpenAI-compatible FRONT: an OpenAI `/chat/completions` request comes in, its `model`
155
245
  * has already been resolved to a provider target by namespace/tier routing. For an
156
246
  * openai-kind target this is a routing reverse-proxy — rewrite `model` to the backend
157
247
  * id, inject the backend key, and stream the upstream OpenAI response straight back
158
248
  * (OpenAI in, OpenAI out — no translation). This is the transport a dispatcher (e.g.
159
- * audit-tools) consumes to reach many backends behind one endpoint.
249
+ * an external dispatcher) consumes to reach many backends behind one endpoint.
160
250
  *
161
251
  * anthropic-kind targets are not served on the OpenAI front (they need OpenAI↔Anthropic
162
252
  * translation and are not the dispatcher use case) — a clean 400, never a mistranslation.
@@ -1 +1 @@
1
- {"version":3,"file":"backend.js","sourceRoot":"","sources":["../src/backend.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,yBAAyB,EAAE,4BAA4B,EAAE,MAAM,YAAY,CAAC;AAErF,OAAO,EAAE,aAAa,EAAE,kBAAkB,EAAE,MAAM,gBAAgB,CAAC;AAEnE;;;;;;;;GAQG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,0BAA0B,CAAC;AAK9D,kEAAkE;AAClE,MAAM,UAAU,WAAW,CAAC,GAAa;IACvC,MAAM,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,mBAAmB,CAAC,CAAC;IAC/C,OAAO,CAAC,KAAK,UAAU,IAAI,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AACtD,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,MAAsB,EACtB,IAQC,EACD,UAAwB,KAAK;IAE7B,IAAI,MAAM,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;QAChC,MAAM,IAAI,GAAgB,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,OAAO,EAAE,IAAI,CAAC,gBAAgB,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC;QACvG,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM;YAAE,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC;QAChD,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAChD,CAAC;IAED,oBAAoB;IACpB,sFAAsF;IACtF,sFAAsF;IACtF,IAAI,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC;IAC3B,IAAI,CAAC;QACH,OAAO,GAAG,MAAM,kBAAkB,CAAC,OAAO,CAAC,CAAC;IAC9C,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,sFAAsF;QACtF,sFAAsF;QACtF,IAAI,CAAC,YAAY,aAAa;YAAE,OAAO,cAAc,CAAC,GAAG,EAAE,cAAc,CAAC,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;QAC/F,OAAO,cAAc,CAAC,GAAG,EAAE,+BAAgC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC7F,CAAC;IAED,IAAI,UAAmC,CAAC;IACxC,IAAI,CAAC;QACH,UAAU,GAAG,yBAAyB,CAAC,WAAW,EAAE,QAAQ,EAAE,CAAC,OAAO,IAAI,EAAE,CAAU,CAA4B,CAAC;IACrH,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,OAAO,cAAc,CAAC,GAAG,EAAE,+BAAgC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC7F,CAAC;IACD,UAAU,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC;IAChC,UAAU,CAAC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC;IACrC,sFAAsF;IACtF,qFAAqF;IACrF,mFAAmF;IACnF,IAAI,IAAI,CAAC,WAAW;QAAE,UAAU,CAAC,cAAc,GAAG,EAAE,aAAa,EAAE,IAAI,EAAE,CAAC;IAE1E,MAAM,OAAO,GAA2B,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC;IAC/E,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7E,IAAI,GAAG,EAAE,CAAC;QACR,IAAI,MAAM,CAAC,UAAU,KAAK,eAAe;YAAE,OAAO,CAAC,eAAe,CAAC,GAAG,UAAU,GAAG,EAAE,CAAC;;YACjF,OAAO,CAAC,WAAW,CAAC,GAAG,GAAG,CAAC;IAClC,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,IAA6B,EAAE,EAAE,CAC7C,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,mBAAmB,EAAE;QACzC,MAAM,EAAE,MAAM;QACd,OAAO;QACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;QAC1B,MAAM,EAAE,IAAI,CAAC,MAAM;KACpB,CAAC,CAAC;IAEL,IAAI,GAAG,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,CAAC;IAEjC,oFAAoF;IACpF,6EAA6E;IAC7E,IAAI,CAAC,GAAG,CAAC,EAAE,IAAI,UAAU,CAAC,cAAc,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,CAAC,EAAE,CAAC;QACvF,MAAM,EAAE,cAAc,EAAE,KAAK,EAAE,GAAG,YAAY,EAAE,GAAG,UAAU,CAAC;QAC9D,GAAG,GAAG,MAAM,IAAI,CAAC,YAAY,CAAC,CAAC;IACjC,CAAC;IAED,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;QAC9B,6EAA6E;QAC7E,sEAAsE;QACtE,2DAA2D;QAC3D,MAAM,IAAI,GACR,GAAG,CAAC,MAAM,KAAK,GAAG;YAChB,CAAC,CAAC,aAAa,MAAM,CAAC,KAAK,gCAAgC,MAAM,CAAC,QAAQ,yDAAyD;YACnI,CAAC,CAAC,EAAE,CAAC;QACT,2FAA2F;QAC3F,OAAO,cAAc,CAAC,GAAG,CAAC,MAAM,EAAE,uBAAuB,GAAG,CAAC,MAAM,GAAG,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,EAAE,UAAU,CAAC,CAAC;IACnH,CAAC;IAED,IAAI,IAAI,CAAC,WAAW,IAAI,GAAG,CAAC,IAAI,EAAE,CAAC;QACjC,MAAM,UAAU,GAAG,4BAA4B,CAAC,GAAG,CAAC,IAAI,EAAE,QAAQ,EAAE,WAAW,CAAC,CAAC;QACjF,OAAO,IAAI,QAAQ,CAAC,UAAU,EAAE,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,EAAE,mBAAmB,EAAE,EAAE,CAAC,CAAC;IAC5G,CAAC;IAED,IAAI,aAAqB,CAAC;IAC1B,IAAI,CAAC;QACH,aAAa,GAAG,yBAAyB,CAAC,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAA4B,EAAE,MAAM,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;IAC/G,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,kFAAkF;QAClF,kFAAkF;QAClF,OAAO,cAAc,CAAC,GAAG,EAAE,gCAAiC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC9F,CAAC;IACD,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,aAAa,CAAC,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,EAAE,CAAC,CAAC;AACvH,CAAC;AAED,4EAA4E;AAC5E,MAAM,UAAU,yBAAyB,CAAC,CAA0B,EAAE,KAAa;IACjF,MAAM,MAAM,GAAI,CAAC,CAAC,OAAsD,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACpF,MAAM,GAAG,GAAI,MAAM,CAAC,OAA+C,IAAI,EAAE,CAAC;IAC1E,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,IAAI,OAAO,GAAG,CAAC,OAAO,KAAK,QAAQ,IAAI,GAAG,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC;QAAE,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC;IACjH,MAAM,SAAS,GAAI,GAAG,CAAC,UAAyD,IAAI,EAAE,CAAC;IACvF,KAAK,MAAM,EAAE,IAAI,SAAS,EAAE,CAAC;QAC3B,MAAM,EAAE,GAAI,EAAE,CAAC,QAAgD,IAAI,EAAE,CAAC;QACtE,IAAI,KAAc,CAAC;QACnB,IAAI,CAAC;YAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAE,EAAE,CAAC,SAAoB,IAAI,IAAI,CAAC,CAAC;QAAC,CAAC;QAAC,MAAM,CAAC;YAAC,KAAK,GAAG,EAAE,CAAC,SAAS,IAAI,EAAE,CAAC;QAAC,CAAC;QACnG,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAG,EAAE,CAAC,EAAa,IAAI,IAAI,EAAE,IAAI,EAAG,EAAE,CAAC,IAAe,IAAI,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC;IAC5G,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,aAAmC,CAAC;IAC1D,MAAM,UAAU,GACd,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,MAAM,IAAI,UAAU,CAAC;IACjI,MAAM,KAAK,GAAI,CAAC,CAAC,KAA4C,IAAI,EAAE,CAAC;IACpE,OAAO;QACL,EAAE,EAAG,CAAC,CAAC,EAAa,IAAI,gBAAgB;QACxC,IAAI,EAAE,SAAS;QACf,IAAI,EAAE,WAAW;QACjB,KAAK,EAAE,KAAK,IAAI,CAAE,CAAC,CAAC,KAAgB,IAAI,EAAE,CAAC;QAC3C,OAAO;QACP,WAAW,EAAE,UAAU;QACvB,aAAa,EAAE,IAAI;QACnB,KAAK,EAAE,EAAE,YAAY,EAAE,KAAK,CAAC,aAAa,IAAI,CAAC,EAAE,aAAa,EAAE,KAAK,CAAC,iBAAiB,IAAI,CAAC,EAAE;KAC/F,CAAC;AACJ,CAAC;AAED,SAAS,cAAc,CAAC,MAAc,EAAE,OAAe,EAAE,MAAmB;IAC1E,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,WAAW,EAAE,OAAO,EAAE,EAAE,CAAC,EAAE;QAC5F,MAAM;QACN,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,mBAAmB,CAAC,EAAE,MAAM,EAAE;KAC/E,CAAC,CAAC;AACL,CAAC;AAED,SAAS,WAAW,CAAC,MAAc,EAAE,OAAe,EAAE,MAAmB;IACvE,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,EAAE,OAAO,EAAE,IAAI,EAAE,uBAAuB,EAAE,EAAE,CAAC,EAAE;QACzF,MAAM;QACN,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,mBAAmB,CAAC,EAAE,MAAM,EAAE;KAC/E,CAAC,CAAC;AACL,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,MAAsB,EACtB,IAAqE,EACrE,UAAwB,KAAK;IAE7B,IAAI,MAAM,CAAC,IAAI,KAAK,QAAQ,EAAE,CAAC;QAC7B,OAAO,WAAW,CAChB,GAAG,EACH,8DAA8D,MAAM,CAAC,QAAQ,QAAQ,MAAM,CAAC,IAAI,EAAE,EAClG,OAAO,CACR,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAA4B,CAAC;IAC7D,MAAM,IAAI,GAAG,EAAE,GAAG,IAAI,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,CAAC,WAAW,EAAE,CAAC;IACxE,MAAM,OAAO,GAA2B,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC;IAC/E,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7E,IAAI,GAAG,EAAE,CAAC;QACR,IAAI,MAAM,CAAC,UAAU,KAAK,eAAe;YAAE,OAAO,CAAC,eAAe,CAAC,GAAG,UAAU,GAAG,EAAE,CAAC;;YACjF,OAAO,CAAC,WAAW,CAAC,GAAG,GAAG,CAAC;IAClC,CAAC;IACD,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,mBAAmB,EAAE;QAChD,MAAM,EAAE,MAAM;QACd,OAAO;QACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;QAC1B,MAAM,EAAE,IAAI,CAAC,MAAM;KACpB,CAAC,CAAC;AACL,CAAC"}
1
+ {"version":3,"file":"backend.js","sourceRoot":"","sources":["../src/backend.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,yBAAyB,EAAE,4BAA4B,EAAE,MAAM,YAAY,CAAC;AAErF,OAAO,EAAE,aAAa,EAAE,kBAAkB,EAAE,MAAM,gBAAgB,CAAC;AAEnE;;;;;;;;GAQG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,0BAA0B,CAAC;AAK9D,kEAAkE;AAClE,MAAM,UAAU,WAAW,CAAC,GAAa;IACvC,MAAM,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,mBAAmB,CAAC,CAAC;IAC/C,OAAO,CAAC,KAAK,UAAU,IAAI,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AACtD,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,uBAAuB,CAAC;AAExD;;;;;;;;GAQG;AACH,MAAM,UAAU,iBAAiB,CAAC,KAAgC,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;IAClF,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,IAAI,CAAC;IAC3C,MAAM,GAAG,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IACzB,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IAElC,kFAAkF;IAClF,IAAI,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;QAC9B,MAAM,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,CAAC;QAC1C,OAAO,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;IACpD,CAAC;IAED,MAAM,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAC3B,IAAI,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC;QAAE,OAAO,IAAI,CAAC;IAClC,MAAM,KAAK,GAAG,EAAE,GAAG,GAAG,CAAC;IACvB,OAAO,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC;AAClC,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,MAAsB,EACtB,IAQC,EACD,UAAwB,KAAK;IAE7B,IAAI,MAAM,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;QAChC,MAAM,IAAI,GAAgB,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,OAAO,EAAE,IAAI,CAAC,gBAAgB,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC;QACvG,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM;YAAE,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC;QAChD,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAChD,CAAC;IAED,oBAAoB;IACpB,sFAAsF;IACtF,sFAAsF;IACtF,IAAI,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC;IAC3B,IAAI,CAAC;QACH,OAAO,GAAG,MAAM,kBAAkB,CAAC,OAAO,CAAC,CAAC;IAC9C,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,sFAAsF;QACtF,sFAAsF;QACtF,IAAI,CAAC,YAAY,aAAa;YAAE,OAAO,cAAc,CAAC,GAAG,EAAE,cAAc,CAAC,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;QAC/F,OAAO,cAAc,CAAC,GAAG,EAAE,+BAAgC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC7F,CAAC;IAED,IAAI,UAAmC,CAAC;IACxC,IAAI,CAAC;QACH,UAAU,GAAG,yBAAyB,CAAC,WAAW,EAAE,QAAQ,EAAE,CAAC,OAAO,IAAI,EAAE,CAAU,CAA4B,CAAC;IACrH,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,OAAO,cAAc,CAAC,GAAG,EAAE,+BAAgC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC7F,CAAC;IACD,UAAU,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC;IAChC,UAAU,CAAC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC;IACrC,sFAAsF;IACtF,qFAAqF;IACrF,mFAAmF;IACnF,IAAI,IAAI,CAAC,WAAW;QAAE,UAAU,CAAC,cAAc,GAAG,EAAE,aAAa,EAAE,IAAI,EAAE,CAAC;IAE1E,MAAM,OAAO,GAA2B,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC;IAC/E,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7E,IAAI,GAAG,EAAE,CAAC;QACR,IAAI,MAAM,CAAC,UAAU,KAAK,eAAe;YAAE,OAAO,CAAC,eAAe,CAAC,GAAG,UAAU,GAAG,EAAE,CAAC;;YACjF,OAAO,CAAC,WAAW,CAAC,GAAG,GAAG,CAAC;IAClC,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,IAA6B,EAAE,EAAE,CAC7C,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,mBAAmB,EAAE;QACzC,MAAM,EAAE,MAAM;QACd,OAAO;QACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;QAC1B,MAAM,EAAE,IAAI,CAAC,MAAM;KACpB,CAAC,CAAC;IAEL,IAAI,GAAG,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,CAAC;IAEjC,oFAAoF;IACpF,6EAA6E;IAC7E,IAAI,CAAC,GAAG,CAAC,EAAE,IAAI,UAAU,CAAC,cAAc,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,CAAC,EAAE,CAAC;QACvF,MAAM,EAAE,cAAc,EAAE,KAAK,EAAE,GAAG,YAAY,EAAE,GAAG,UAAU,CAAC;QAC9D,GAAG,GAAG,MAAM,IAAI,CAAC,YAAY,CAAC,CAAC;IACjC,CAAC;IAED,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;QAC9B,6EAA6E;QAC7E,sEAAsE;QACtE,2DAA2D;QAC3D,MAAM,IAAI,GACR,GAAG,CAAC,MAAM,KAAK,GAAG;YAChB,CAAC,CAAC,aAAa,MAAM,CAAC,KAAK,gCAAgC,MAAM,CAAC,QAAQ,yDAAyD;YACnI,CAAC,CAAC,EAAE,CAAC;QACT,2FAA2F;QAC3F,2FAA2F;QAC3F,6FAA6F;QAC7F,oFAAoF;QACpF,OAAO,cAAc,CAAC,GAAG,CAAC,MAAM,EAAE,uBAAuB,GAAG,CAAC,MAAM,GAAG,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,EAAE,UAAU,EAAE;YAC/G,GAAG,gBAAgB,CAAC,GAAG,CAAC,OAAO,CAAC;SACjC,CAAC,CAAC;IACL,CAAC;IAED,IAAI,IAAI,CAAC,WAAW,IAAI,GAAG,CAAC,IAAI,EAAE,CAAC;QACjC,MAAM,UAAU,GAAG,4BAA4B,CAAC,GAAG,CAAC,IAAI,EAAE,QAAQ,EAAE,WAAW,CAAC,CAAC;QACjF,OAAO,IAAI,QAAQ,CAAC,UAAU,EAAE,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,EAAE,mBAAmB,EAAE,EAAE,CAAC,CAAC;IAC5G,CAAC;IAED,IAAI,aAAqB,CAAC;IAC1B,IAAI,CAAC;QACH,aAAa,GAAG,yBAAyB,CAAC,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAA4B,EAAE,MAAM,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;IAC/G,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,kFAAkF;QAClF,kFAAkF;QAClF,OAAO,cAAc,CAAC,GAAG,EAAE,gCAAiC,CAAW,CAAC,OAAO,EAAE,EAAE,OAAO,CAAC,CAAC;IAC9F,CAAC;IACD,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,aAAa,CAAC,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,EAAE,CAAC,CAAC;AACvH,CAAC;AAED,4EAA4E;AAC5E,MAAM,UAAU,yBAAyB,CAAC,CAA0B,EAAE,KAAa;IACjF,MAAM,MAAM,GAAI,CAAC,CAAC,OAAsD,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACpF,MAAM,GAAG,GAAI,MAAM,CAAC,OAA+C,IAAI,EAAE,CAAC;IAC1E,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,IAAI,OAAO,GAAG,CAAC,OAAO,KAAK,QAAQ,IAAI,GAAG,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC;QAAE,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC;IACjH,MAAM,SAAS,GAAI,GAAG,CAAC,UAAyD,IAAI,EAAE,CAAC;IACvF,KAAK,MAAM,EAAE,IAAI,SAAS,EAAE,CAAC;QAC3B,MAAM,EAAE,GAAI,EAAE,CAAC,QAAgD,IAAI,EAAE,CAAC;QACtE,IAAI,KAAc,CAAC;QACnB,IAAI,CAAC;YAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAE,EAAE,CAAC,SAAoB,IAAI,IAAI,CAAC,CAAC;QAAC,CAAC;QAAC,MAAM,CAAC;YAAC,KAAK,GAAG,EAAE,CAAC,SAAS,IAAI,EAAE,CAAC;QAAC,CAAC;QACnG,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,EAAE,EAAG,EAAE,CAAC,EAAa,IAAI,IAAI,EAAE,IAAI,EAAG,EAAE,CAAC,IAAe,IAAI,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC;IAC5G,CAAC;IACD,MAAM,MAAM,GAAG,MAAM,CAAC,aAAmC,CAAC;IAC1D,MAAM,UAAU,GACd,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,MAAM,IAAI,UAAU,CAAC;IACjI,MAAM,KAAK,GAAI,CAAC,CAAC,KAA4C,IAAI,EAAE,CAAC;IACpE,OAAO;QACL,EAAE,EAAG,CAAC,CAAC,EAAa,IAAI,gBAAgB;QACxC,IAAI,EAAE,SAAS;QACf,IAAI,EAAE,WAAW;QACjB,KAAK,EAAE,KAAK,IAAI,CAAE,CAAC,CAAC,KAAgB,IAAI,EAAE,CAAC;QAC3C,OAAO;QACP,WAAW,EAAE,UAAU;QACvB,aAAa,EAAE,IAAI;QACnB,KAAK,EAAE,EAAE,YAAY,EAAE,KAAK,CAAC,aAAa,IAAI,CAAC,EAAE,aAAa,EAAE,KAAK,CAAC,iBAAiB,IAAI,CAAC,EAAE;KAC/F,CAAC;AACJ,CAAC;AAED,6FAA6F;AAC7F,SAAS,gBAAgB,CAAC,EAAW;IACnC,MAAM,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,aAAa,CAAC,CAAC;IAChC,OAAO,CAAC,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;AACvC,CAAC;AAED,SAAS,cAAc,CACrB,MAAc,EACd,OAAe,EACf,MAAmB,EACnB,QAAgC,EAAE;IAElC,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,WAAW,EAAE,OAAO,EAAE,EAAE,CAAC,EAAE;QAC5F,MAAM;QACN,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,mBAAmB,CAAC,EAAE,MAAM,EAAE,GAAG,KAAK,EAAE;KACzF,CAAC,CAAC;AACL,CAAC;AAED,SAAS,WAAW,CAAC,MAAc,EAAE,OAAe,EAAE,MAAmB;IACvE,OAAO,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,EAAE,OAAO,EAAE,IAAI,EAAE,uBAAuB,EAAE,EAAE,CAAC,EAAE;QACzF,MAAM;QACN,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC,mBAAmB,CAAC,EAAE,MAAM,EAAE;KAC/E,CAAC,CAAC;AACL,CAAC;AAED;;;;;;;;;;;;;;;;;;;GAmBG;AACH,MAAM,UAAU,wBAAwB,CAAC,IAAY,EAAE,MAAc;IACnE,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC5B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,0BAA0B,MAAM,qBAAqB,CAAC;QACpG,OAAO,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,EAAE,OAAO,EAAE,IAAI,EAAE,gBAAgB,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,CAAC,CAAC;IACtF,CAAC;IAED,MAAM,QAAQ,GAAG,CAAC,CAAU,EAA2C,EAAE,CACvE,OAAO,CAAC,KAAK,QAAQ,IAAI,CAAC,KAAK,IAAI,IAAI,OAAQ,CAAyB,CAAC,KAAK,KAAK,QAAQ,IAAK,CAAyB,CAAC,KAAK,KAAK,IAAI,CAAC;IAE3I,IAAI,QAAQ,CAAC,MAAM,CAAC;QAAE,OAAO,IAAI,CAAC,CAAC,6BAA6B;IAEhE,IAAI,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;QAC1B,MAAM,OAAO,GAAG,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QACtC,IAAI,OAAO;YAAE,OAAO,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC;IAC9C,CAAC;IAED,OAAO,IAAI,CAAC,SAAS,CAAC;QACpB,KAAK,EAAE;YACL,OAAO,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,0BAA0B,MAAM,qBAAqB;YAC5F,IAAI,EAAE,gBAAgB;YACtB,IAAI,EAAE,MAAM;SACb;KACF,CAAC,CAAC;AACL,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,MAAsB,EACtB,IAAqE,EACrE,UAAwB,KAAK;IAE7B,IAAI,MAAM,CAAC,IAAI,KAAK,QAAQ,EAAE,CAAC;QAC7B,OAAO,WAAW,CAChB,GAAG,EACH,8DAA8D,MAAM,CAAC,QAAQ,QAAQ,MAAM,CAAC,IAAI,EAAE,EAClG,OAAO,CACR,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAA4B,CAAC;IAC7D,MAAM,IAAI,GAAG,EAAE,GAAG,IAAI,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,CAAC,WAAW,EAAE,CAAC;IACxE,MAAM,OAAO,GAA2B,EAAE,cAAc,EAAE,kBAAkB,EAAE,CAAC;IAC/E,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;IAC7E,IAAI,GAAG,EAAE,CAAC;QACR,IAAI,MAAM,CAAC,UAAU,KAAK,eAAe;YAAE,OAAO,CAAC,eAAe,CAAC,GAAG,UAAU,GAAG,EAAE,CAAC;;YACjF,OAAO,CAAC,WAAW,CAAC,GAAG,GAAG,CAAC;IAClC,CAAC;IACD,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,GAAG,mBAAmB,EAAE;QAChD,MAAM,EAAE,MAAM;QACd,OAAO;QACP,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;QAC1B,MAAM,EAAE,IAAI,CAAC,MAAM;KACpB,CAAC,CAAC;AACL,CAAC"}
@@ -47,6 +47,19 @@ export interface Candidate {
47
47
  consecutiveFailures: number;
48
48
  lastStatus: number | null;
49
49
  cooldownRemainingMs: number;
50
+ /**
51
+ * Credential faults observed on real traffic, on their own axis.
52
+ *
53
+ * A member answering 401 on every call used to be indistinguishable here from a healthy
54
+ * one — verdict `Pending`, breaker `closed`, no failure count — because a 401 is
55
+ * deliberately not health data and so reached none of the fields above. It is still not
56
+ * health data; it is reported as what it is. `credentialFault` true means the router is
57
+ * currently DEMOTING this member (it is tried only after every other candidate fails),
58
+ * and it expires, so a rotated key recovers without a restart.
59
+ */
60
+ credentialFailures: number;
61
+ lastCredentialStatus: number | null;
62
+ credentialFault: boolean;
50
63
  };
51
64
  /** Observed real traffic through this proxy (not synthetic probes). */
52
65
  observed: {
@@ -127,6 +127,9 @@ export async function buildCandidates(cfg, opts = {}) {
127
127
  consecutiveFailures: state?.consecutiveFailures ?? 0,
128
128
  lastStatus: state?.lastStatus ?? null,
129
129
  cooldownRemainingMs: Math.max(0, (state?.cooldownUntil ?? 0) - nowMs),
130
+ credentialFailures: state?.credentialFailures ?? 0,
131
+ lastCredentialStatus: state?.lastCredentialStatus ?? null,
132
+ credentialFault: breaker.hasCredentialFault(spec, nowMs),
130
133
  },
131
134
  observed: obs
132
135
  ? {
@@ -1 +1 @@
1
- {"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG1C,OAAO,EAAE,WAAW,EAAsB,MAAM,iBAAiB,CAAC;AAClE,OAAO,EAAE,aAAa,EAAiB,MAAM,gBAAgB,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,eAAe,EAAuB,MAAM,eAAe,CAAC;AACrE,OAAO,EAAE,oBAAoB,EAAuB,MAAM,sBAAsB,CAAC;AACjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAmH7C,MAAM,IAAI,GACR,8FAA8F;IAC9F,yFAAyF;IACzF,0FAA0F;IAC1F,wFAAwF,CAAC;AAE3F,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,GAAW;IAC3C,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,IAAI,IAAI,KAAK,WAAW;QAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IACxC,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/C,OAAO,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;AACzC,CAAC;AAED;;;;GAIG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,GAAG,GAAG,IAAI,GAAG,EAAwD,CAAC;IAC5E,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE;QAC7B,IAAI,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACtB,IAAI,CAAC,CAAC,EAAE,CAAC;YACP,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;YACrC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACnB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC,EAAE,CAAC;QACpE,KAAK,MAAM,CAAC,IAAI,KAAK;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACnD,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,IAAI,EAAE,CAAC,EAAE,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;AAClG,CAAC;AAED,+DAA+D;AAC/D,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,GAAW,EACX,OAWI,EAAE;IAEN,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,oBAAoB,CAAC;IACrD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;IACvC,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,IAAI,EAAE,CAAC;IAC5F,MAAM,SAAS,GAAG,oBAAoB,EAAE,CAAC;IAEzC,MAAM,UAAU,GAAgB,EAAE,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,IAAI,CAAC,QAAQ;YAAE,SAAS;QAC1D,MAAM,CAAC,GAA+B,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QAE9D,IAAI,MAAM,GAAmB,IAAI,CAAC;QAClC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACtD,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC;YACtD,CAAC;YAAC,MAAM,CAAC;gBACP,MAAM,GAAG,IAAI,CAAC;YAChB,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAC/F,MAAM,KAAK,GAAG,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrC,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,MAAM,CAAC,GAAG,QAAQ,IAAI,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACzE,MAAM,QAAQ,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,OAAO,GAAG,aAAa,CAAC,KAAK,IAAI,IAAI,EAAE,MAAM,CAAC,CAAC;QACrD,MAAM,IAAI,GAAG,OAAO,EAAE,GAAG,CAAC;QAC1B,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAE,IAAI,CAAC,CAAC,CAAY,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAExF,uFAAuF;QACvF,4FAA4F;QAC5F,2FAA2F;QAC3F,oCAAoC;QACpC,MAAM,cAAc,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO;YACtE,CAAC,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC;YACjE,CAAC,CAAC,IAAI,CAAC;QACT,qFAAqF;QACrF,kFAAkF;QAClF,mFAAmF;QACnF,sFAAsF;QACtF,uFAAuF;QACvF,wBAAwB;QACxB,MAAM,aAAa,GACjB,OAAO,IAAI,OAAO,CAAC,KAAK,KAAK,OAAO,CAAC,CAAC,CAAC,cAAc,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,YAAY,CAAC;QACzF,MAAM,IAAI,GAAG,eAAe,CAAC,KAAK,IAAI,QAAQ,EAAE;YAC9C,cAAc;YACd,SAAS,EAAE;gBACT,aAAa,EAAE,GAAG,CAAC,gBAAgB,CAAC;gBACpC,mBAAmB,EAAE,GAAG,CAAC,cAAc,CAAC;gBACxC,uBAAuB,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBAChD,IAAI,EAAE,aAAa;aACpB;SACF,CAAC,CAAC;QAEH,UAAU,CAAC,IAAI,CAAC;YACd,IAAI;YACJ,QAAQ;YACR,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,UAAU,CAAC,KAAK;YACvB,aAAa,EAAE,UAAU,CAAC,aAAa;YACvC,qFAAqF;YACrF,oFAAoF;YACpF,MAAM,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI;YAChE,MAAM;YACN,eAAe,EAAE,OAAO,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI;YAClF,MAAM,EAAE,OAAO;gBACb,CAAC,CAAC;oBACE,OAAO,EAAE,OAAO,CAAC,OAAO;oBACxB,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,YAAY,EAAE,OAAO,CAAC,YAAY;oBAClC,UAAU,EAAE,OAAO,CAAC,UAAU;iBAC/B;gBACH,CAAC,CAAC,IAAI;YACR,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;YAC7E,OAAO,EAAE;gBACP,IAAI,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,CAAC;gBACrC,mBAAmB,EAAE,KAAK,EAAE,mBAAmB,IAAI,CAAC;gBACpD,UAAU,EAAE,KAAK,EAAE,UAAU,IAAI,IAAI;gBACrC,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,IAAI,CAAC,CAAC,GAAG,KAAK,CAAC;aACtE;YACD,QAAQ,EAAE,GAAG;gBACX,CAAC,CAAC;oBACE,UAAU,EAAE,GAAG,CAAC,UAAU;oBAC1B,YAAY,EAAE,GAAG,CAAC,YAAY;oBAC9B,YAAY,EAAE,GAAG,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,cAAc,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI;oBACzF,YAAY,EAAE,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,IAAI;iBACjF;gBACH,CAAC,CAAC,IAAI;YACR,aAAa,EAAE,IAAI,CAAC,aAAa;YACjC,mBAAmB,EAAE,IAAI,CAAC,mBAAmB;YAC7C,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,qBAAqB,EAAE,IAAI,CAAC,qBAAqB;YACjD,GAAG,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,qBAAqB,EAAE,IAAI,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC5E,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,aAAa,EAAE,OAAO,IAAI,EAAE,cAAc,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI;YACrF,iBAAiB,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC,CAAC,CAAE,IAAI,CAAC,OAAoB,CAAC,CAAC,CAAC,EAAE;YACjF,MAAM,EAAE;gBACN,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBACxC,cAAc,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACtC,QAAQ,EAAE,GAAG,CAAC,WAAW,CAAC;gBAC1B,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;gBAC5B,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,mBAAmB,CAAC;gBACzC,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;aAC7B;YACD,UAAU,EAAE;gBACV,QAAQ,EAAE,QAAQ,CAAC,KAAK;gBACxB,aAAa,EAAE,QAAQ,CAAC,KAAK;gBAC7B,eAAe,EAAE,QAAQ,CAAC,OAAO,IAAI,EAAE;gBACvC,gBAAgB,EAAE,OAAO,CAAC,oBAAoB,CAAC,IAAI,CAAC;aACrD;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO;QACL,YAAY,EAAE,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,EAAE;QACvD,eAAe,EAAE,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,IAAI;QAC7C,IAAI,EAAE,IAAI;QACV,UAAU;KACX,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG1C,OAAO,EAAE,WAAW,EAAsB,MAAM,iBAAiB,CAAC;AAClE,OAAO,EAAE,aAAa,EAAiB,MAAM,gBAAgB,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,eAAe,EAAuB,MAAM,eAAe,CAAC;AACrE,OAAO,EAAE,oBAAoB,EAAuB,MAAM,sBAAsB,CAAC;AACjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAgI7C,MAAM,IAAI,GACR,8FAA8F;IAC9F,yFAAyF;IACzF,0FAA0F;IAC1F,wFAAwF,CAAC;AAE3F,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,GAAW;IAC3C,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,IAAI,IAAI,KAAK,WAAW;QAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IACxC,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/C,OAAO,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;AACzC,CAAC;AAED;;;;GAIG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,GAAG,GAAG,IAAI,GAAG,EAAwD,CAAC;IAC5E,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE;QAC7B,IAAI,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACtB,IAAI,CAAC,CAAC,EAAE,CAAC;YACP,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;YACrC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACnB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC,EAAE,CAAC;QACpE,KAAK,MAAM,CAAC,IAAI,KAAK;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACnD,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,IAAI,EAAE,CAAC,EAAE,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;AAClG,CAAC;AAED,+DAA+D;AAC/D,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,GAAW,EACX,OAWI,EAAE;IAEN,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,oBAAoB,CAAC;IACrD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;IACvC,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,IAAI,EAAE,CAAC;IAC5F,MAAM,SAAS,GAAG,oBAAoB,EAAE,CAAC;IAEzC,MAAM,UAAU,GAAgB,EAAE,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,IAAI,CAAC,QAAQ;YAAE,SAAS;QAC1D,MAAM,CAAC,GAA+B,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QAE9D,IAAI,MAAM,GAAmB,IAAI,CAAC;QAClC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACtD,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC;YACtD,CAAC;YAAC,MAAM,CAAC;gBACP,MAAM,GAAG,IAAI,CAAC;YAChB,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAC/F,MAAM,KAAK,GAAG,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrC,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,MAAM,CAAC,GAAG,QAAQ,IAAI,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACzE,MAAM,QAAQ,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,OAAO,GAAG,aAAa,CAAC,KAAK,IAAI,IAAI,EAAE,MAAM,CAAC,CAAC;QACrD,MAAM,IAAI,GAAG,OAAO,EAAE,GAAG,CAAC;QAC1B,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAE,IAAI,CAAC,CAAC,CAAY,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAExF,uFAAuF;QACvF,4FAA4F;QAC5F,2FAA2F;QAC3F,oCAAoC;QACpC,MAAM,cAAc,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO;YACtE,CAAC,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC;YACjE,CAAC,CAAC,IAAI,CAAC;QACT,qFAAqF;QACrF,kFAAkF;QAClF,mFAAmF;QACnF,sFAAsF;QACtF,uFAAuF;QACvF,wBAAwB;QACxB,MAAM,aAAa,GACjB,OAAO,IAAI,OAAO,CAAC,KAAK,KAAK,OAAO,CAAC,CAAC,CAAC,cAAc,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,YAAY,CAAC;QACzF,MAAM,IAAI,GAAG,eAAe,CAAC,KAAK,IAAI,QAAQ,EAAE;YAC9C,cAAc;YACd,SAAS,EAAE;gBACT,aAAa,EAAE,GAAG,CAAC,gBAAgB,CAAC;gBACpC,mBAAmB,EAAE,GAAG,CAAC,cAAc,CAAC;gBACxC,uBAAuB,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBAChD,IAAI,EAAE,aAAa;aACpB;SACF,CAAC,CAAC;QAEH,UAAU,CAAC,IAAI,CAAC;YACd,IAAI;YACJ,QAAQ;YACR,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,UAAU,CAAC,KAAK;YACvB,aAAa,EAAE,UAAU,CAAC,aAAa;YACvC,qFAAqF;YACrF,oFAAoF;YACpF,MAAM,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI;YAChE,MAAM;YACN,eAAe,EAAE,OAAO,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI;YAClF,MAAM,EAAE,OAAO;gBACb,CAAC,CAAC;oBACE,OAAO,EAAE,OAAO,CAAC,OAAO;oBACxB,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,YAAY,EAAE,OAAO,CAAC,YAAY;oBAClC,UAAU,EAAE,OAAO,CAAC,UAAU;iBAC/B;gBACH,CAAC,CAAC,IAAI;YACR,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;YAC7E,OAAO,EAAE;gBACP,IAAI,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,CAAC;gBACrC,mBAAmB,EAAE,KAAK,EAAE,mBAAmB,IAAI,CAAC;gBACpD,UAAU,EAAE,KAAK,EAAE,UAAU,IAAI,IAAI;gBACrC,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,IAAI,CAAC,CAAC,GAAG,KAAK,CAAC;gBACrE,kBAAkB,EAAE,KAAK,EAAE,kBAAkB,IAAI,CAAC;gBAClD,oBAAoB,EAAE,KAAK,EAAE,oBAAoB,IAAI,IAAI;gBACzD,eAAe,EAAE,OAAO,CAAC,kBAAkB,CAAC,IAAI,EAAE,KAAK,CAAC;aACzD;YACD,QAAQ,EAAE,GAAG;gBACX,CAAC,CAAC;oBACE,UAAU,EAAE,GAAG,CAAC,UAAU;oBAC1B,YAAY,EAAE,GAAG,CAAC,YAAY;oBAC9B,YAAY,EAAE,GAAG,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,cAAc,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI;oBACzF,YAAY,EAAE,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,IAAI;iBACjF;gBACH,CAAC,CAAC,IAAI;YACR,aAAa,EAAE,IAAI,CAAC,aAAa;YACjC,mBAAmB,EAAE,IAAI,CAAC,mBAAmB;YAC7C,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,qBAAqB,EAAE,IAAI,CAAC,qBAAqB;YACjD,GAAG,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,qBAAqB,EAAE,IAAI,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC5E,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,aAAa,EAAE,OAAO,IAAI,EAAE,cAAc,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI;YACrF,iBAAiB,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC,CAAC,CAAE,IAAI,CAAC,OAAoB,CAAC,CAAC,CAAC,EAAE;YACjF,MAAM,EAAE;gBACN,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBACxC,cAAc,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACtC,QAAQ,EAAE,GAAG,CAAC,WAAW,CAAC;gBAC1B,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;gBAC5B,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,mBAAmB,CAAC;gBACzC,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;aAC7B;YACD,UAAU,EAAE;gBACV,QAAQ,EAAE,QAAQ,CAAC,KAAK;gBACxB,aAAa,EAAE,QAAQ,CAAC,KAAK;gBAC7B,eAAe,EAAE,QAAQ,CAAC,OAAO,IAAI,EAAE;gBACvC,gBAAgB,EAAE,OAAO,CAAC,oBAAoB,CAAC,IAAI,CAAC;aACrD;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO;QACL,YAAY,EAAE,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,EAAE;QACvD,eAAe,EAAE,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,IAAI;QAC7C,IAAI,EAAE,IAAI;QACV,UAAU;KACX,CAAC;AACJ,CAAC"}
@@ -7,6 +7,19 @@ export interface CircuitState {
7
7
  lastStatus?: number | undefined;
8
8
  pings: PingRecord[];
9
9
  quotaPercent?: number | null | undefined;
10
+ /**
11
+ * Credential faults (401/403), counted SEPARATELY from health failures.
12
+ *
13
+ * A revoked key is not a sick backend: recording it as a failure would open the breaker on
14
+ * a configuration problem and hide the 401 the operator has to see behind a "target
15
+ * unhealthy" skip, and recording it as a success would launder a permanently broken
16
+ * candidate into a healthy one. So it is neither — it is its own dimension, which is what
17
+ * lets a pool step over the candidate while `/candidates` still reports exactly why.
18
+ */
19
+ credentialFailures: number;
20
+ lastCredentialStatus?: number | undefined;
21
+ /** While in the future, this target is demoted (never dropped) as unusable-for-now. */
22
+ credentialFaultUntil: number;
10
23
  }
11
24
  /**
12
25
  * The ordering value for a target nothing has been measured about.
@@ -45,7 +58,27 @@ export declare class CircuitBreaker {
45
58
  status?: number;
46
59
  quotaPercent?: number | null;
47
60
  at?: number;
61
+ /**
62
+ * The provider's own `Retry-After`, in ms, when it published one. It replaces the flat
63
+ * cooldown guess: the provider is the only party that knows when it will serve again,
64
+ * and 2 minutes is as likely to be far too long (a 20-second groq TPM window) as far
65
+ * too short. Clamped, because the value is attacker-adjacent input.
66
+ */
67
+ retryAfterMs?: number | undefined;
48
68
  }): void;
69
+ /**
70
+ * Record a 401/403 — a credential fault, which is NOT health data.
71
+ *
72
+ * This deliberately does not touch `consecutiveFailures`, `cooldownUntil` or the ping
73
+ * history, so `isHealthy()` keeps answering the question it is named for and a revoked key
74
+ * never masquerades as a flaky backend (or vice versa). What it does do is mark the target
75
+ * unusable-for-now, which `hasCredentialFault()` reports and the router uses to DEMOTE it —
76
+ * so a pool stops paying a round-trip per request for a member that cannot authenticate,
77
+ * while `/candidates` still shows the operator the 401 and its count.
78
+ */
79
+ recordCredentialFault(target: ResolvedTarget | string, status: number, at?: number): void;
80
+ /** True while a recent 401/403 makes this target a last resort. Expires, so a fixed key recovers. */
81
+ hasCredentialFault(target: ResolvedTarget | string, now?: number): boolean;
49
82
  /**
50
83
  * Stability for a target whose behaviour has actually been observed, or
51
84
  * `null` when NOTHING has been measured.
@@ -3,6 +3,17 @@ const DEFAULT_COOLDOWN_MS = 60000; // 1 minute cooldown after consecutive failur
3
3
  const RATE_LIMIT_COOLDOWN_MS = 120000; // 2 minutes cooldown on 429
4
4
  const MAX_FAILURES_BEFORE_TRIP = 2;
5
5
  const MAX_PING_HISTORY = 10;
6
+ /**
7
+ * How long a 401/403 demotes a candidate before it is tried again.
8
+ *
9
+ * Deliberately finite: a rotated key must recover without restarting the proxy. Deliberately
10
+ * not zero: without it, a pool holding seven keyless members pays seven round-trips of 401
11
+ * latency on EVERY request before reaching a live one.
12
+ */
13
+ const CREDENTIAL_FAULT_TTL_MS = 300000; // 5 minutes
14
+ /** Clamp for a provider-supplied Retry-After, so a hostile or absurd value cannot park a target. */
15
+ const MIN_RETRY_AFTER_MS = 1000;
16
+ const MAX_RETRY_AFTER_MS = 900000; // 15 minutes
6
17
  /**
7
18
  * The ordering value for a target nothing has been measured about.
8
19
  *
@@ -49,6 +60,8 @@ export class CircuitBreaker {
49
60
  lastFailureTime: 0,
50
61
  cooldownUntil: 0,
51
62
  pings: [],
63
+ credentialFailures: 0,
64
+ credentialFaultUntil: 0,
52
65
  };
53
66
  this.states.set(key, fresh);
54
67
  return fresh;
@@ -87,18 +100,52 @@ export class CircuitBreaker {
87
100
  if (outcome.ok) {
88
101
  state.consecutiveFailures = 0;
89
102
  state.cooldownUntil = 0;
103
+ // A call that actually succeeded is proof the credential works now — that is the
104
+ // recovery path for a key fixed while the proxy is running.
105
+ state.credentialFailures = 0;
106
+ state.credentialFaultUntil = 0;
90
107
  return;
91
108
  }
92
109
  state.consecutiveFailures += 1;
93
110
  state.lastFailureTime = now;
94
- // HTTP 429 (Rate Limit) trips immediately for 2 minutes
111
+ const asked = outcome.retryAfterMs !== undefined
112
+ ? Math.min(MAX_RETRY_AFTER_MS, Math.max(MIN_RETRY_AFTER_MS, outcome.retryAfterMs))
113
+ : null;
114
+ // HTTP 429 (Rate Limit) trips immediately — for exactly as long as the provider asked,
115
+ // or 2 minutes when it did not say.
95
116
  if (outcome.status === 429) {
96
- state.cooldownUntil = now + RATE_LIMIT_COOLDOWN_MS;
117
+ state.cooldownUntil = now + (asked ?? RATE_LIMIT_COOLDOWN_MS);
118
+ }
119
+ else if (asked !== null) {
120
+ // A 503 with a Retry-After is the provider scheduling us; honour it on the first
121
+ // failure rather than waiting for a second one to trip the generic cooldown.
122
+ state.cooldownUntil = now + asked;
97
123
  }
98
124
  else if (state.consecutiveFailures >= MAX_FAILURES_BEFORE_TRIP) {
99
125
  state.cooldownUntil = now + DEFAULT_COOLDOWN_MS;
100
126
  }
101
127
  }
128
+ /**
129
+ * Record a 401/403 — a credential fault, which is NOT health data.
130
+ *
131
+ * This deliberately does not touch `consecutiveFailures`, `cooldownUntil` or the ping
132
+ * history, so `isHealthy()` keeps answering the question it is named for and a revoked key
133
+ * never masquerades as a flaky backend (or vice versa). What it does do is mark the target
134
+ * unusable-for-now, which `hasCredentialFault()` reports and the router uses to DEMOTE it —
135
+ * so a pool stops paying a round-trip per request for a member that cannot authenticate,
136
+ * while `/candidates` still shows the operator the 401 and its count.
137
+ */
138
+ recordCredentialFault(target, status, at = Date.now()) {
139
+ const state = this.getOrCreate(this.getKey(target));
140
+ state.credentialFailures += 1;
141
+ state.lastCredentialStatus = status;
142
+ state.credentialFaultUntil = at + CREDENTIAL_FAULT_TTL_MS;
143
+ }
144
+ /** True while a recent 401/403 makes this target a last resort. Expires, so a fixed key recovers. */
145
+ hasCredentialFault(target, now = Date.now()) {
146
+ const state = this.states.get(this.getKey(target));
147
+ return !!state && state.credentialFaultUntil > now;
148
+ }
102
149
  /**
103
150
  * Stability for a target whose behaviour has actually been observed, or
104
151
  * `null` when NOTHING has been measured.
@@ -1 +1 @@
1
- {"version":3,"file":"circuit-breaker.js","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,iBAAiB,EAAmB,MAAM,mBAAmB,CAAC;AAWvE,MAAM,mBAAmB,GAAG,KAAK,CAAC,CAAC,+CAA+C;AAClF,MAAM,sBAAsB,GAAG,MAAM,CAAC,CAAC,4BAA4B;AACnE,MAAM,wBAAwB,GAAG,CAAC,CAAC;AACnC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AAE5B;;;;;;;;;;;;GAYG;AACH,MAAM,CAAC,MAAM,oBAAoB,GAAG,EAAE,CAAC;AAEvC;;;;;;;;GAQG;AACH,SAAS,WAAW,CAAC,OAAyC;IAC5D,IAAI,OAAO,CAAC,MAAM,KAAK,SAAS;QAAE,OAAO,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC;IACpE,IAAI,OAAO,CAAC,MAAM,IAAI,GAAG,IAAI,OAAO,CAAC,MAAM,GAAG,GAAG;QAAE,OAAO,KAAK,CAAC;IAChE,OAAO,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,OAAO,cAAc;IACjB,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAC;IAEzC,MAAM,CAAC,MAA+B;QAC5C,IAAI,OAAO,MAAM,KAAK,QAAQ;YAAE,OAAO,MAAM,CAAC;QAC9C,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,QAAQ,IAAI,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC;IAC/E,CAAC;IAEO,WAAW,CAAC,GAAW;QAC7B,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACtC,IAAI,QAAQ;YAAE,OAAO,QAAQ,CAAC;QAC9B,MAAM,KAAK,GAAiB;YAC1B,mBAAmB,EAAE,CAAC;YACtB,eAAe,EAAE,CAAC;YAClB,aAAa,EAAE,CAAC;YAChB,KAAK,EAAE,EAAE;SACV,CAAC;QACF,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QAC5B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,8EAA8E;IAC9E,SAAS,CAAC,MAA+B,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;QACzD,MAAM,GAAG,GAAG,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAChC,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACnC,IAAI,CAAC,KAAK;YAAE,OAAO,IAAI,CAAC;QAExB,IAAI,KAAK,CAAC,aAAa,GAAG,GAAG,EAAE,CAAC;YAC9B,OAAO,KAAK,CAAC,CAAC,8BAA8B;QAC9C,CAAC;QAED,OAAO,IAAI,CAAC,CAAC,8BAA8B;IAC7C,CAAC;IAED;;;;;;;;;;OAUG;IACH,aAAa,CACX,MAA+B,EAC/B,OAAuG;QAEvG,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;QACrC,MAAM,KAAK,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QAEpD,IAAI,OAAO,CAAC,YAAY,KAAK,SAAS;YAAE,KAAK,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,CAAC;QAClF,KAAK,CAAC,UAAU,GAAG,OAAO,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;QACpE,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,OAAO,CAAC,SAAS,EAAE,IAAI,EAAE,WAAW,CAAC,OAAO,CAAC,EAAE,SAAS,EAAE,GAAG,EAAE,CAAC,CAAC;QACxF,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,gBAAgB;YAAE,KAAK,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC;QAE/D,IAAI,OAAO,CAAC,EAAE,EAAE,CAAC;YACf,KAAK,CAAC,mBAAmB,GAAG,CAAC,CAAC;YAC9B,KAAK,CAAC,aAAa,GAAG,CAAC,CAAC;YACxB,OAAO;QACT,CAAC;QAED,KAAK,CAAC,mBAAmB,IAAI,CAAC,CAAC;QAC/B,KAAK,CAAC,eAAe,GAAG,GAAG,CAAC;QAE5B,wDAAwD;QACxD,IAAI,OAAO,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;YAC3B,KAAK,CAAC,aAAa,GAAG,GAAG,GAAG,sBAAsB,CAAC;QACrD,CAAC;aAAM,IAAI,KAAK,CAAC,mBAAmB,IAAI,wBAAwB,EAAE,CAAC;YACjE,KAAK,CAAC,aAAa,GAAG,GAAG,GAAG,mBAAmB,CAAC;QAClD,CAAC;IACH,CAAC;IAED;;;;;;;;OAQG;IACH,oBAAoB,CAAC,MAA+B;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACnD,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC;QACpD,MAAM,KAAK,GAAG,iBAAiB,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QAC7C,OAAO,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAChC,CAAC;IAED,wEAAwE;IACxE,eAAe,CAAC,MAA+B;QAC7C,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACnD,OAAO,CAAC,CAAC,KAAK,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC;IAC3C,CAAC;IAED,qFAAqF;IACrF,2FAA2F;IAC3F,wFAAwF;IACxF,4FAA4F;IAC5F,2FAA2F;IAC3F,4FAA4F;IAC5F,kFAAkF;IAElF,mCAAmC;IACnC,QAAQ,CAAC,MAA+B;QACtC,OAAO,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;IAC9C,CAAC;IAED,sCAAsC;IACtC,YAAY;QACV,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;IAED;;;;;;;;;;;OAWG;IACH,iBAAiB,CAAC,OAAyB,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;QAC3D,MAAM,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;QAC9D,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;QAC1D,OAAO,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YACnC,MAAM,EAAE,GAAG,IAAI,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC;YACxC,MAAM,EAAE,GAAG,IAAI,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC;YACxC,MAAM,EAAE,GAAG,EAAE,IAAI,oBAAoB,CAAC;YACtC,MAAM,EAAE,GAAG,EAAE,IAAI,oBAAoB,CAAC;YACtC,IAAI,EAAE,KAAK,EAAE;gBAAE,OAAO,EAAE,GAAG,EAAE,CAAC;YAC9B,IAAI,CAAC,EAAE,KAAK,IAAI,CAAC,KAAK,CAAC,EAAE,KAAK,IAAI,CAAC;gBAAE,OAAO,EAAE,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACjE,OAAO,CAAC,CAAC;QACX,CAAC,CAAC,CAAC;IACL,CAAC;IAED,gCAAgC;IAChC,KAAK;QACH,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;IACtB,CAAC;CACF;AAED,MAAM,CAAC,MAAM,oBAAoB,GAAG,IAAI,cAAc,EAAE,CAAC"}
1
+ {"version":3,"file":"circuit-breaker.js","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,iBAAiB,EAAmB,MAAM,mBAAmB,CAAC;AAwBvE,MAAM,mBAAmB,GAAG,KAAK,CAAC,CAAC,+CAA+C;AAClF,MAAM,sBAAsB,GAAG,MAAM,CAAC,CAAC,4BAA4B;AACnE,MAAM,wBAAwB,GAAG,CAAC,CAAC;AACnC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AAE5B;;;;;;GAMG;AACH,MAAM,uBAAuB,GAAG,MAAM,CAAC,CAAC,YAAY;AAEpD,oGAAoG;AACpG,MAAM,kBAAkB,GAAG,IAAI,CAAC;AAChC,MAAM,kBAAkB,GAAG,MAAM,CAAC,CAAC,aAAa;AAEhD;;;;;;;;;;;;GAYG;AACH,MAAM,CAAC,MAAM,oBAAoB,GAAG,EAAE,CAAC;AAEvC;;;;;;;;GAQG;AACH,SAAS,WAAW,CAAC,OAAyC;IAC5D,IAAI,OAAO,CAAC,MAAM,KAAK,SAAS;QAAE,OAAO,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC;IACpE,IAAI,OAAO,CAAC,MAAM,IAAI,GAAG,IAAI,OAAO,CAAC,MAAM,GAAG,GAAG;QAAE,OAAO,KAAK,CAAC;IAChE,OAAO,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,OAAO,cAAc;IACjB,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAC;IAEzC,MAAM,CAAC,MAA+B;QAC5C,IAAI,OAAO,MAAM,KAAK,QAAQ;YAAE,OAAO,MAAM,CAAC;QAC9C,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,QAAQ,IAAI,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC;IAC/E,CAAC;IAEO,WAAW,CAAC,GAAW;QAC7B,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACtC,IAAI,QAAQ;YAAE,OAAO,QAAQ,CAAC;QAC9B,MAAM,KAAK,GAAiB;YAC1B,mBAAmB,EAAE,CAAC;YACtB,eAAe,EAAE,CAAC;YAClB,aAAa,EAAE,CAAC;YAChB,KAAK,EAAE,EAAE;YACT,kBAAkB,EAAE,CAAC;YACrB,oBAAoB,EAAE,CAAC;SACxB,CAAC;QACF,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QAC5B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,8EAA8E;IAC9E,SAAS,CAAC,MAA+B,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;QACzD,MAAM,GAAG,GAAG,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAChC,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACnC,IAAI,CAAC,KAAK;YAAE,OAAO,IAAI,CAAC;QAExB,IAAI,KAAK,CAAC,aAAa,GAAG,GAAG,EAAE,CAAC;YAC9B,OAAO,KAAK,CAAC,CAAC,8BAA8B;QAC9C,CAAC;QAED,OAAO,IAAI,CAAC,CAAC,8BAA8B;IAC7C,CAAC;IAED;;;;;;;;;;OAUG;IACH,aAAa,CACX,MAA+B,EAC/B,OAaC;QAED,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;QACrC,MAAM,KAAK,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QAEpD,IAAI,OAAO,CAAC,YAAY,KAAK,SAAS;YAAE,KAAK,CAAC,YAAY,GAAG,OAAO,CAAC,YAAY,CAAC;QAClF,KAAK,CAAC,UAAU,GAAG,OAAO,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;QACpE,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,OAAO,CAAC,SAAS,EAAE,IAAI,EAAE,WAAW,CAAC,OAAO,CAAC,EAAE,SAAS,EAAE,GAAG,EAAE,CAAC,CAAC;QACxF,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,gBAAgB;YAAE,KAAK,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC;QAE/D,IAAI,OAAO,CAAC,EAAE,EAAE,CAAC;YACf,KAAK,CAAC,mBAAmB,GAAG,CAAC,CAAC;YAC9B,KAAK,CAAC,aAAa,GAAG,CAAC,CAAC;YACxB,iFAAiF;YACjF,4DAA4D;YAC5D,KAAK,CAAC,kBAAkB,GAAG,CAAC,CAAC;YAC7B,KAAK,CAAC,oBAAoB,GAAG,CAAC,CAAC;YAC/B,OAAO;QACT,CAAC;QAED,KAAK,CAAC,mBAAmB,IAAI,CAAC,CAAC;QAC/B,KAAK,CAAC,eAAe,GAAG,GAAG,CAAC;QAE5B,MAAM,KAAK,GACT,OAAO,CAAC,YAAY,KAAK,SAAS;YAChC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,kBAAkB,EAAE,IAAI,CAAC,GAAG,CAAC,kBAAkB,EAAE,OAAO,CAAC,YAAY,CAAC,CAAC;YAClF,CAAC,CAAC,IAAI,CAAC;QAEX,uFAAuF;QACvF,oCAAoC;QACpC,IAAI,OAAO,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;YAC3B,KAAK,CAAC,aAAa,GAAG,GAAG,GAAG,CAAC,KAAK,IAAI,sBAAsB,CAAC,CAAC;QAChE,CAAC;aAAM,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;YAC1B,iFAAiF;YACjF,6EAA6E;YAC7E,KAAK,CAAC,aAAa,GAAG,GAAG,GAAG,KAAK,CAAC;QACpC,CAAC;aAAM,IAAI,KAAK,CAAC,mBAAmB,IAAI,wBAAwB,EAAE,CAAC;YACjE,KAAK,CAAC,aAAa,GAAG,GAAG,GAAG,mBAAmB,CAAC;QAClD,CAAC;IACH,CAAC;IAED;;;;;;;;;OASG;IACH,qBAAqB,CAAC,MAA+B,EAAE,MAAc,EAAE,EAAE,GAAG,IAAI,CAAC,GAAG,EAAE;QACpF,MAAM,KAAK,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACpD,KAAK,CAAC,kBAAkB,IAAI,CAAC,CAAC;QAC9B,KAAK,CAAC,oBAAoB,GAAG,MAAM,CAAC;QACpC,KAAK,CAAC,oBAAoB,GAAG,EAAE,GAAG,uBAAuB,CAAC;IAC5D,CAAC;IAED,qGAAqG;IACrG,kBAAkB,CAAC,MAA+B,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;QAClE,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACnD,OAAO,CAAC,CAAC,KAAK,IAAI,KAAK,CAAC,oBAAoB,GAAG,GAAG,CAAC;IACrD,CAAC;IAED;;;;;;;;OAQG;IACH,oBAAoB,CAAC,MAA+B;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACnD,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC;QACpD,MAAM,KAAK,GAAG,iBAAiB,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QAC7C,OAAO,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAChC,CAAC;IAED,wEAAwE;IACxE,eAAe,CAAC,MAA+B;QAC7C,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QACnD,OAAO,CAAC,CAAC,KAAK,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC;IAC3C,CAAC;IAED,qFAAqF;IACrF,2FAA2F;IAC3F,wFAAwF;IACxF,4FAA4F;IAC5F,2FAA2F;IAC3F,4FAA4F;IAC5F,kFAAkF;IAElF,mCAAmC;IACnC,QAAQ,CAAC,MAA+B;QACtC,OAAO,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;IAC9C,CAAC;IAED,sCAAsC;IACtC,YAAY;QACV,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;IAED;;;;;;;;;;;OAWG;IACH,iBAAiB,CAAC,OAAyB,EAAE,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE;QAC3D,MAAM,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;QAC9D,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;QAC1D,OAAO,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YACnC,MAAM,EAAE,GAAG,IAAI,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC;YACxC,MAAM,EAAE,GAAG,IAAI,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAC;YACxC,MAAM,EAAE,GAAG,EAAE,IAAI,oBAAoB,CAAC;YACtC,MAAM,EAAE,GAAG,EAAE,IAAI,oBAAoB,CAAC;YACtC,IAAI,EAAE,KAAK,EAAE;gBAAE,OAAO,EAAE,GAAG,EAAE,CAAC;YAC9B,IAAI,CAAC,EAAE,KAAK,IAAI,CAAC,KAAK,CAAC,EAAE,KAAK,IAAI,CAAC;gBAAE,OAAO,EAAE,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACjE,OAAO,CAAC,CAAC;QACX,CAAC,CAAC,CAAC;IACL,CAAC;IAED,gCAAgC;IAChC,KAAK;QACH,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;IACtB,CAAC;CACF;AAED,MAAM,CAAC,MAAM,oBAAoB,GAAG,IAAI,cAAc,EAAE,CAAC"}
package/dist/cli.d.ts CHANGED
@@ -11,8 +11,15 @@ export declare function splitSpec(spec: string): {
11
11
  };
12
12
  export declare function resolveConfigPath(): string;
13
13
  /**
14
- * Merge `~/.llm-relay/.env` into the environment, once per process, before anything reads
15
- * a key or expands a `${ENV}` in the config. Already-set variables win.
14
+ * Merge every credential source into the environment, once per process, before anything reads a
15
+ * key or expands a `${ENV}` in the config. Already-set variables always win, at every layer.
16
+ *
17
+ * Order is least-explicit first, and both layers only ever FILL GAPS:
18
+ * 1. Windows User/Machine registry scopes — recovers variables the OS could not deliver to an
19
+ * already-running process. A long-lived relay launched at logon otherwise never sees a key
20
+ * added afterwards, while every shell the user opens does; that mismatch made six working
21
+ * credentials look like `401 Wrong API Key`.
22
+ * 2. `~/.llm-relay/.env` — what the onboarding wizard writes.
16
23
  */
17
24
  export declare function ensureEnvFileLoaded(): void;
18
25
  export declare function loadOrExit(): Config;