llm-relay 0.14.5 → 0.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +114 -34
- package/config.example.json +4 -1
- package/dist/candidates.d.ts +4 -1
- package/dist/candidates.js +5 -1
- package/dist/candidates.js.map +1 -1
- package/dist/cli.d.ts +2 -2
- package/dist/cli.js +260 -139
- package/dist/cli.js.map +1 -1
- package/dist/config.d.ts +41 -15
- package/dist/config.js +102 -16
- package/dist/config.js.map +1 -1
- package/dist/dispatch.d.ts +9 -5
- package/dist/dispatch.js +16 -5
- package/dist/dispatch.js.map +1 -1
- package/dist/offload.d.ts +15 -19
- package/dist/offload.js +87 -12
- package/dist/offload.js.map +1 -1
- package/dist/routes/admin.js +111 -4
- package/dist/routes/admin.js.map +1 -1
- package/dist/server.js +8 -5
- package/dist/server.js.map +1 -1
- package/package.json +2 -2
- package/scripts/install-skill.mjs +65 -3
- package/skills/llm-relay/SKILL.md +72 -19
package/README.md
CHANGED
|
@@ -64,7 +64,7 @@ llm-relay
|
|
|
64
64
|
- **Multi-Candidate Failover**: `routing.default` and every `routing.tiers` entry accept an **array** of target specs (e.g. `["nim/z-ai/glm-5.2", "groq/llama-3.3-70b"]`) for continuous fallback. ⚠ Ranking and failover both require **more than one** candidate — a single pinned model silently disables both, and on providers where a listed model may not actually be servable that turns one dead backend into a dead relay. Prefer arrays.
|
|
65
65
|
- **Named pools** (`routing.pools`, addressed as `model: "pool/<name>"`): the same ranked-candidate behaviour for callers that can only send **one model string** — notably Claude Code subagent frontmatter. Lets an agent ask for *the best available coding model* instead of naming one. An unknown pool is a loud 400, never a silent fall-through.
|
|
66
66
|
- **Passthrough targets**: a provider with `kind:"anthropic"` and **no `authEnv`** forwards the caller's own credentials untouched, so real Claude traffic stays on real Anthropic while `pool/*` requests route elsewhere — from the same proxy.
|
|
67
|
-
- **
|
|
67
|
+
- **Granular offload** (`routing.offload` + `routing.subagents`): independently route Claude, Codex, and future client requests to other providers. Each client can be limited to subagents or set to `scope: "all"` to reroute its main conversation too. **Off by default**; `llm-relay candidates` shows what to point it at. See below.
|
|
68
68
|
|
|
69
69
|
### 3. Prompt Token & Context Length Guardrails
|
|
70
70
|
- Estimates the request's prompt token count (`estimateRequestTokens`) against the target model's context limit, read from the warm catalog cache (`cachedLimits()` — it never fetches, so a cold cache costs no round-trip on the request path).
|
|
@@ -89,7 +89,7 @@ llm-relay
|
|
|
89
89
|
|
|
90
90
|
### 6. Programmatic Telemetry & Quota Access for Claude
|
|
91
91
|
- **Read-only HTTP endpoints**: `GET /telemetry` (live JSON metrics), `GET /registry` (full provider/routing/model catalog with quality scores), `GET /ping` (trigger health probe pass & mode summary), `GET /health` (diagnostic status), `GET /candidates` (the un-blended offload decision table).
|
|
92
|
-
- **Mutating HTTP endpoints**: `GET|POST /offload` (read/
|
|
92
|
+
- **Mutating HTTP endpoints**: `GET|POST /offload` (read/set per-client offload rules), `GET|POST /dispatch` (the dispatch ladder). ⚠ **Loopback is not authorization** — any page you visit can POST cross-origin to a loopback listener without a preflight, and these two write your `config.json` and steer lane order. They therefore reject a present-but-non-loopback `Origin` with 403, require `content-type: application/json` on a mutating request, and require a loopback `Host` (closing DNS rebinding). An **absent** `Origin` is allowed on purpose — that is what a CLI sends, and it is what keeps targeted offload changes working against a running proxy with no restart.
|
|
93
93
|
- **CLI Commands**: `llm-relay telemetry` outputs live telemetry metrics; `llm-relay models` lists live model catalogs with SWE-bench & quality scores; `llm-relay ping` performs live health & latency probes.
|
|
94
94
|
- **Response Headers**: Proxy responses include `x-llm-relay-quota-percent`, `x-llm-relay-stability-score`, and `x-llm-relay-target`.
|
|
95
95
|
|
|
@@ -108,9 +108,9 @@ llm-relay
|
|
|
108
108
|
| `llm-relay telemetry` | Output live JSON telemetry, stability scores, and quota metrics |
|
|
109
109
|
| `llm-relay models [-p <name>] [-r]` | Query live `/models` catalog per provider (`-p` filter, `-r` force refresh) |
|
|
110
110
|
| `llm-relay ping [-p <name>]` | Perform live health, latency & quota probe across providers |
|
|
111
|
-
| `llm-relay offload [on\|off\|status]` | Read or
|
|
111
|
+
| `llm-relay offload [client] [on\|off\|status] [--scope subagents\|all]` | Read or change one client's offload rule; changes apply to the next request, no restart |
|
|
112
112
|
| `llm-relay candidates [-p <name>]` | The un-blended offload decision table (capability, cost, live health, quota, breaker state) |
|
|
113
|
-
| `llm-relay dispatch [lane] [-t <task>]` | Which lane to hand a whole delegated task to next; it returns the command, **you** run it (`-x <lane>` reports one spent) |
|
|
113
|
+
| `llm-relay dispatch [lane] [-t <task>] [--client <name>]` | Which lane to hand a whole delegated task to next; it returns the command, **you** run it (`-x <lane>` reports one spent) |
|
|
114
114
|
|
|
115
115
|
---
|
|
116
116
|
|
|
@@ -180,8 +180,14 @@ directories: `~/.claude/skills/llm-relay/SKILL.md` for Claude Code and
|
|
|
180
180
|
switch, `@relay:` directives, reading the candidates table, failure modes) cannot drift between
|
|
181
181
|
hosts. Both refresh automatically on every upgrade; local/dev installs touch neither directory.
|
|
182
182
|
|
|
183
|
+
The same global install also provisions local Codex: it adds the `llm-relay` Responses provider to
|
|
184
|
+
`~/.codex/config.toml` and creates relay-backed `default` and `relay_coding` child agents under
|
|
185
|
+
`~/.codex/agents/` when those files are absent. Existing Codex config and agent files are preserved.
|
|
186
|
+
This keeps the parent on its normal provider while making generic or named child dispatches use the
|
|
187
|
+
relay automatically.
|
|
188
|
+
|
|
183
189
|
If your npm blocks unknown install scripts (`npm warn install-scripts … blocked`), allow this one —
|
|
184
|
-
`npm config set allow-scripts=llm-relay --location=user` — or install the
|
|
190
|
+
`npm config set allow-scripts=llm-relay --location=user` — or install the host integrations by hand:
|
|
185
191
|
`node "$(npm root -g)/llm-relay/scripts/install-skill.mjs" --force`.
|
|
186
192
|
|
|
187
193
|
### Staying current
|
|
@@ -400,43 +406,51 @@ refusing safe calls. So:
|
|
|
400
406
|
There is no built-in list inside the proxy: an empty `repair.destructiveTools` refuses nothing, so
|
|
401
407
|
coverage is always traceable to your config.
|
|
402
408
|
|
|
403
|
-
###
|
|
404
|
-
|
|
405
|
-
Send Claude Code **subagents** to other providers while the human's own conversation stays on the
|
|
406
|
-
Anthropic passthrough — without writing agent files and without naming a model.
|
|
407
|
-
|
|
408
|
-
**Off by default.** Until you turn it on, subagents route exactly like everything else:
|
|
409
|
-
|
|
410
|
-
```bash
|
|
411
|
-
llm-relay offload on
|
|
412
|
-
```
|
|
409
|
+
### Granular offload (`routing.offload` + `routing.subagents`)
|
|
413
410
|
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
411
|
+
Offload rules are keyed by the originating harness. Claude requests use the `/v1/messages` front
|
|
412
|
+
door; Codex requests use `/v1/responses`. Each rule is independent and chooses whether it applies
|
|
413
|
+
to marked subagents only (the current behavior) or to the whole conversation:
|
|
417
414
|
|
|
418
415
|
```jsonc
|
|
419
416
|
"routing": {
|
|
420
|
-
"offload": false, // master switch (default). `subagents` is inert until this is true.
|
|
421
417
|
"tiers": { "opus": "anthropic", "sonnet": "anthropic", "haiku": "anthropic", "fable": "anthropic" },
|
|
422
|
-
"subagents": { "opus": "pool/reasoning", "haiku": "pool/fast", "default": "pool/coding" }
|
|
418
|
+
"subagents": { "opus": "pool/reasoning", "haiku": "pool/fast", "default": "pool/coding" },
|
|
419
|
+
"offload": {
|
|
420
|
+
"claude": { "enabled": true, "scope": "subagents" },
|
|
421
|
+
"codex": { "enabled": false, "scope": "all" }
|
|
422
|
+
}
|
|
423
423
|
}
|
|
424
424
|
```
|
|
425
425
|
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
426
|
+
`scope: "subagents"` preserves the existing topology. `scope: "all"` also applies the same
|
|
427
|
+
`routing.subagents` tier/default map to the client's main conversation, which is useful when a
|
|
428
|
+
Claude or Codex quota is exhausted. Rules may use any future client name; an explicit `default`
|
|
429
|
+
rule is the opt-in catch-all for otherwise unnamed front doors. All rules are off by default.
|
|
430
|
+
|
|
431
|
+
The CLI changes one client without restarting the proxy:
|
|
432
|
+
|
|
433
|
+
```bash
|
|
434
|
+
llm-relay offload status
|
|
435
|
+
llm-relay offload claude on --scope subagents
|
|
436
|
+
llm-relay offload codex on --scope all
|
|
437
|
+
llm-relay offload claude off
|
|
438
|
+
```
|
|
439
|
+
|
|
440
|
+
The old `llm-relay offload on|off` command remains a global compatibility switch for configs that
|
|
441
|
+
still use the boolean form (`"offload": false`). `GET /offload?client=claude` reads one rule;
|
|
442
|
+
`POST /offload` accepts `{"client":"claude","enabled":true,"scope":"all"}`. Changes are
|
|
443
|
+
persisted and take effect on the next request.
|
|
429
444
|
|
|
430
445
|
Claude Code stamps `cc_is_subagent=true` into the `system` block of subagent requests (built-in
|
|
431
|
-
agents like Explore included — verified on the wire, Claude Code 2.1.220).
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
446
|
+
agents like Explore included — verified on the wire, Claude Code 2.1.220). Local Codex stamps
|
|
447
|
+
`x-codex-turn-metadata: {"request_kind":"subagent",...}` on child-agent turns. A subagents-only
|
|
448
|
+
rule requires that marker; an all-scope rule also accepts ordinary main-conversation requests.
|
|
449
|
+
An explicit `@relay:` directive remains a subagent-only per-call opt-in, even when a client has an
|
|
450
|
+
all-scope rule, so text in a human conversation cannot self-reroute it.
|
|
436
451
|
|
|
437
|
-
A dispatcher
|
|
438
|
-
|
|
439
|
-
`subagents.default` applies and the pool's ranking picks the model.
|
|
452
|
+
A dispatcher chooses a destination with the Agent tool's `model` parameter (`sonnet|opus|haiku|fable`)
|
|
453
|
+
or, when it does not choose, `subagents.default` applies and the pool's ranking picks the model.
|
|
440
454
|
|
|
441
455
|
**To pin an exact model for one call**, put a directive on its own line in the subagent's prompt:
|
|
442
456
|
|
|
@@ -454,9 +468,75 @@ authored prompt. Block 0 is Claude Code's injected `<system-reminder>` (your CLA
|
|
|
454
468
|
…), and later messages carry tool results, i.e. file contents. Reading either would let any file a
|
|
455
469
|
subagent happens to read redirect its own routing. Both cases are covered by tests.
|
|
456
470
|
|
|
457
|
-
Precedence for a subagent request: `@relay:` directive → `subagents[<tier>]` →
|
|
458
|
-
`subagents.default` → normal routing. The
|
|
459
|
-
`routing.subagents` entirely and nothing changes
|
|
471
|
+
Precedence for a marked subagent request: `@relay:` directive → `subagents[<tier>]` →
|
|
472
|
+
`subagents.default` → normal routing. The map applies only when that request's client rule is
|
|
473
|
+
enabled and its scope admits the request; omit `routing.subagents` entirely and nothing changes.
|
|
474
|
+
|
|
475
|
+
#### Local Codex setup
|
|
476
|
+
|
|
477
|
+
For the intended split, keep the parent Codex session on its normal provider and define a named
|
|
478
|
+
child agent whose own Responses requests use llm-relay. A global `llm-relay` install creates the
|
|
479
|
+
provider and agents below automatically. If npm lifecycle scripts were blocked, run the bundled
|
|
480
|
+
installer manually with `--force`, or create the files yourself as follows.
|
|
481
|
+
|
|
482
|
+
```toml
|
|
483
|
+
[model_providers.llm-relay]
|
|
484
|
+
name = "llm-relay"
|
|
485
|
+
base_url = "http://127.0.0.1:8791/v1"
|
|
486
|
+
wire_api = "responses"
|
|
487
|
+
requires_openai_auth = true
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
Then create `~/.codex/agents/relay_coding.toml`:
|
|
491
|
+
|
|
492
|
+
```toml
|
|
493
|
+
name = "relay_coding"
|
|
494
|
+
description = "Read-only coding child routed through llm-relay."
|
|
495
|
+
developer_instructions = "Work read-only. Return a concise result to the parent and do not modify files."
|
|
496
|
+
|
|
497
|
+
model_provider = "llm-relay"
|
|
498
|
+
model = "pool/coding"
|
|
499
|
+
model_reasoning_effort = "medium"
|
|
500
|
+
```
|
|
501
|
+
|
|
502
|
+
To make an unqualified child dispatch use the relay automatically, override Codex's built-in
|
|
503
|
+
`default` agent with `~/.codex/agents/default.toml`:
|
|
504
|
+
|
|
505
|
+
```toml
|
|
506
|
+
name = "default"
|
|
507
|
+
description = "General-purpose read-only child routed through llm-relay."
|
|
508
|
+
developer_instructions = "Work read-only. Return a concise result to the parent and do not modify files."
|
|
509
|
+
|
|
510
|
+
model_provider = "llm-relay"
|
|
511
|
+
model = "pool/coding"
|
|
512
|
+
model_reasoning_effort = "medium"
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
With that override, a normal “use a subagent” request keeps the parent native while the generic child
|
|
516
|
+
goes through `pool/coding`; named agents can still select a different pool explicitly.
|
|
517
|
+
|
|
518
|
+
Run Codex normally, without the `llm-relay` profile. Ask the parent to use exactly one subagent of
|
|
519
|
+
type `relay_coding`; Codex keeps the parent on its normal provider and starts the child through the
|
|
520
|
+
relay. The relay pool then chooses the configured provider and can fail over normally.
|
|
521
|
+
|
|
522
|
+
Enable only Codex child offload in `~/.llm-relay/config.json`:
|
|
523
|
+
|
|
524
|
+
```bash
|
|
525
|
+
llm-relay offload codex on --scope subagents
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
To redirect the parent Codex conversation through the same relay as well, use
|
|
529
|
+
`llm-relay offload codex on --scope all`. Claude's rule is unaffected.
|
|
530
|
+
|
|
531
|
+
The `llm-relay` profile remains available as an explicit all-relay mode, but it routes the parent
|
|
532
|
+
through the relay too and is not the split setup described above. The automatic
|
|
533
|
+
`x-codex-turn-metadata` marker is still recognized when a Codex client sends it; using a relay pool
|
|
534
|
+
as the named child model keeps the split setup reliable even when a custom-agent request omits that
|
|
535
|
+
private marker.
|
|
536
|
+
|
|
537
|
+
This applies to local Codex clients that can reach `127.0.0.1`. Hosted ChatGPT/Cloud tasks cannot
|
|
538
|
+
reach a loopback relay, and the relay cannot spend a ChatGPT subscription on behalf of an upstream
|
|
539
|
+
request; those remain separate CLI/client-bound dispatch lanes.
|
|
460
540
|
|
|
461
541
|
Whole-task CLI dispatch can likewise vary by tier with `routing.ladders.{reasoning,coding,fast}`.
|
|
462
542
|
Use `llm-relay dispatch --tier reasoning -t "..."`; without `--tier`, the ladder matching
|
package/config.example.json
CHANGED
|
@@ -88,7 +88,10 @@
|
|
|
88
88
|
"fable": "pool/fast",
|
|
89
89
|
"default": "pool/coding"
|
|
90
90
|
},
|
|
91
|
-
"offload":
|
|
91
|
+
"offload": {
|
|
92
|
+
"claude": { "enabled": false, "scope": "subagents" },
|
|
93
|
+
"codex": { "enabled": false, "scope": "subagents" }
|
|
94
|
+
}
|
|
92
95
|
},
|
|
93
96
|
"mode": "repair",
|
|
94
97
|
"repair": {
|
package/dist/candidates.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import type
|
|
1
|
+
import { type Config, type OffloadRule } from "./config.js";
|
|
2
2
|
import type { ModelCatalog } from "./catalog.js";
|
|
3
3
|
import type { PingLoop } from "./ping/cadence.js";
|
|
4
4
|
import { type StrengthBasis } from "./benchmarks.js";
|
|
@@ -128,7 +128,10 @@ export interface Candidate {
|
|
|
128
128
|
}
|
|
129
129
|
export interface CandidatesView {
|
|
130
130
|
generated_at: string;
|
|
131
|
+
/** True when at least one client-specific rule is enabled. */
|
|
131
132
|
offload_enabled: boolean;
|
|
133
|
+
/** The per-originating-client rules; empty for the legacy boolean form. */
|
|
134
|
+
offload_clients: Record<string, OffloadRule>;
|
|
132
135
|
note: string;
|
|
133
136
|
candidates: Candidate[];
|
|
134
137
|
}
|
package/dist/candidates.js
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { anyOffloadEnabled } from "./config.js";
|
|
1
2
|
import { POOL_PREFIX } from "./config.js";
|
|
2
3
|
import { getStrength } from "./benchmarks.js";
|
|
3
4
|
import { findTierModel } from "./tier-data.js";
|
|
@@ -176,7 +177,10 @@ export async function buildCandidates(cfg, opts = {}) {
|
|
|
176
177
|
}
|
|
177
178
|
return {
|
|
178
179
|
generated_at: opts.now ?? new Date(nowMs).toISOString(),
|
|
179
|
-
offload_enabled: cfg
|
|
180
|
+
offload_enabled: anyOffloadEnabled(cfg),
|
|
181
|
+
offload_clients: typeof cfg.routing.offload === "object" && cfg.routing.offload !== null
|
|
182
|
+
? cfg.routing.offload
|
|
183
|
+
: {},
|
|
180
184
|
note: NOTE,
|
|
181
185
|
candidates,
|
|
182
186
|
};
|
package/dist/candidates.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,iBAAiB,EAAsD,MAAM,aAAa,CAAC;AACpG,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG1C,OAAO,EAAE,WAAW,EAAsB,MAAM,iBAAiB,CAAC;AAClE,OAAO,EAAE,aAAa,EAAiB,MAAM,gBAAgB,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,eAAe,EAAuB,MAAM,eAAe,CAAC;AACrE,OAAO,EAAE,oBAAoB,EAAuB,MAAM,sBAAsB,CAAC;AACjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC7C,OAAO,EAAE,uBAAuB,EAAE,MAAM,oBAAoB,CAAC;AAmI7D,MAAM,IAAI,GACR,8FAA8F;IAC9F,0HAA0H;IAC1H,0FAA0F;IAC1F,wFAAwF,CAAC;AAE3F,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,GAAW;IAC3C,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,IAAI,IAAI,KAAK,WAAW;QAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IACxC,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/C,OAAO,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;AACzC,CAAC;AAED;;;;GAIG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,GAAG,GAAG,IAAI,GAAG,EAAwD,CAAC;IAC5E,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE;QAC7B,IAAI,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACtB,IAAI,CAAC,CAAC,EAAE,CAAC;YACP,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;YACrC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACnB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC,EAAE,CAAC;QACpE,KAAK,MAAM,CAAC,IAAI,KAAK;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACnD,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,IAAI,EAAE,CAAC,EAAE,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;AAClG,CAAC;AAED,+DAA+D;AAC/D,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,GAAW,EACX,OAWI,EAAE;IAEN,IAAI,IAAI,CAAC,OAAO;QAAE,uBAAuB,CAAC,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;IAC7D,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,oBAAoB,CAAC;IACrD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;IACvC,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,MAAM,IAAI,EAAE,CAAC;IAC5F,MAAM,SAAS,GAAG,oBAAoB,EAAE,CAAC;IAEzC,MAAM,UAAU,GAAgB,EAAE,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,IAAI,CAAC,QAAQ;YAAE,SAAS;QAC1D,MAAM,CAAC,GAA+B,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QAE9D,IAAI,MAAM,GAAmB,IAAI,CAAC;QAClC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACtD,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC;YACtD,CAAC;YAAC,MAAM,CAAC;gBACP,MAAM,GAAG,IAAI,CAAC;YAChB,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAC/F,MAAM,KAAK,GAAG,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrC,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,MAAM,CAAC,GAAG,QAAQ,IAAI,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACzE,MAAM,QAAQ,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,OAAO,GAAG,aAAa,CAAC,KAAK,IAAI,IAAI,EAAE,MAAM,CAAC,CAAC;QACrD,MAAM,IAAI,GAAG,OAAO,EAAE,GAAG,CAAC;QAC1B,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAE,IAAI,CAAC,CAAC,CAAY,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAExF,uFAAuF;QACvF,4FAA4F;QAC5F,2FAA2F;QAC3F,oCAAoC;QACpC,MAAM,cAAc,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO;YACtE,CAAC,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC;YACjE,CAAC,CAAC,IAAI,CAAC;QACT,qFAAqF;QACrF,kFAAkF;QAClF,mFAAmF;QACnF,sFAAsF;QACtF,uFAAuF;QACvF,wBAAwB;QACxB,MAAM,aAAa,GACjB,OAAO,IAAI,OAAO,CAAC,KAAK,KAAK,OAAO,CAAC,CAAC,CAAC,cAAc,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,YAAY,CAAC;QACzF,MAAM,IAAI,GAAG,eAAe,CAAC,KAAK,IAAI,QAAQ,EAAE;YAC9C,cAAc;YACd,SAAS,EAAE;gBACT,aAAa,EAAE,GAAG,CAAC,gBAAgB,CAAC;gBACpC,mBAAmB,EAAE,GAAG,CAAC,cAAc,CAAC;gBACxC,uBAAuB,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBAChD,IAAI,EAAE,aAAa;aACpB;SACF,CAAC,CAAC;QAEH,UAAU,CAAC,IAAI,CAAC;YACd,IAAI;YACJ,QAAQ;YACR,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,UAAU,CAAC,KAAK;YACvB,aAAa,EAAE,UAAU,CAAC,aAAa;YACvC,qFAAqF;YACrF,oFAAoF;YACpF,MAAM,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI;YAChE,MAAM;YACN,eAAe,EAAE,OAAO,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI;YAClF,MAAM,EAAE,OAAO;gBACb,CAAC,CAAC;oBACE,OAAO,EAAE,OAAO,CAAC,OAAO;oBACxB,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,YAAY,EAAE,OAAO,CAAC,YAAY;oBAClC,UAAU,EAAE,OAAO,CAAC,UAAU;iBAC/B;gBACH,CAAC,CAAC,IAAI;YACR,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;YAC7E,OAAO,EAAE;gBACP,IAAI,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,CAAC;gBACrC,mBAAmB,EAAE,KAAK,EAAE,mBAAmB,IAAI,CAAC;gBACpD,UAAU,EAAE,KAAK,EAAE,UAAU,IAAI,IAAI;gBACrC,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,IAAI,CAAC,CAAC,GAAG,KAAK,CAAC;gBACrE,kBAAkB,EAAE,KAAK,EAAE,kBAAkB,IAAI,CAAC;gBAClD,oBAAoB,EAAE,KAAK,EAAE,oBAAoB,IAAI,IAAI;gBACzD,eAAe,EAAE,OAAO,CAAC,kBAAkB,CAAC,IAAI,EAAE,KAAK,CAAC;aACzD;YACD,QAAQ,EAAE,GAAG;gBACX,CAAC,CAAC;oBACE,UAAU,EAAE,GAAG,CAAC,UAAU;oBAC1B,YAAY,EAAE,GAAG,CAAC,YAAY;oBAC9B,YAAY,EAAE,GAAG,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,cAAc,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI;oBACzF,YAAY,EAAE,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,IAAI;iBACjF;gBACH,CAAC,CAAC,IAAI;YACR,aAAa,EAAE,IAAI,CAAC,aAAa;YACjC,mBAAmB,EAAE,IAAI,CAAC,mBAAmB;YAC7C,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,qBAAqB,EAAE,IAAI,CAAC,qBAAqB;YACjD,GAAG,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,qBAAqB,EAAE,IAAI,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC5E,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,aAAa,EAAE,OAAO,IAAI,EAAE,cAAc,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI;YACrF,iBAAiB,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC,CAAC,CAAE,IAAI,CAAC,OAAoB,CAAC,CAAC,CAAC,EAAE;YACjF,MAAM,EAAE;gBACN,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBACxC,cAAc,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACtC,QAAQ,EAAE,GAAG,CAAC,WAAW,CAAC;gBAC1B,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;gBAC5B,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,mBAAmB,CAAC;gBACzC,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;aAC7B;YACD,UAAU,EAAE;gBACV,QAAQ,EAAE,QAAQ,CAAC,KAAK;gBACxB,aAAa,EAAE,QAAQ,CAAC,KAAK;gBAC7B,eAAe,EAAE,QAAQ,CAAC,OAAO,IAAI,EAAE;gBACvC,gBAAgB,EAAE,OAAO,CAAC,oBAAoB,CAAC,IAAI,CAAC;aACrD;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO;QACL,YAAY,EAAE,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,EAAE;QACvD,eAAe,EAAE,iBAAiB,CAAC,GAAG,CAAC;QACvC,eAAe,EAAE,OAAO,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,QAAQ,IAAI,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,IAAI;YACtF,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,OAAO;YACrB,CAAC,CAAC,EAAE;QACN,IAAI,EAAE,IAAI;QACV,UAAU;KACX,CAAC;AACJ,CAAC"}
|
package/dist/cli.d.ts
CHANGED
|
@@ -94,8 +94,8 @@ export declare function renderCommand(invoke: {
|
|
|
94
94
|
* order and configuration but knows nothing about what is currently spent.
|
|
95
95
|
*/
|
|
96
96
|
export declare function runDispatch(arg: string | undefined): Promise<void>;
|
|
97
|
-
/** `llm-relay offload [on|off|status]` — the
|
|
98
|
-
export declare function runOffload(arg: string | undefined): Promise<void>;
|
|
97
|
+
/** `llm-relay offload [client] [on|off|status]` — configure one client or the legacy global rule. */
|
|
98
|
+
export declare function runOffload(arg: string | undefined, nextArg?: string): Promise<void>;
|
|
99
99
|
/** `llm-relay candidates` — every dimension of every offload target, side by side, unranked. */
|
|
100
100
|
export declare function runCandidates(): Promise<void>;
|
|
101
101
|
/**
|