llm-relay 0.63.0 → 0.64.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/backend.d.ts +16 -0
- package/dist/backend.js +1 -0
- package/dist/backend.js.map +1 -1
- package/dist/config.d.ts +30 -0
- package/dist/config.js +33 -0
- package/dist/config.js.map +1 -1
- package/dist/latency-demotion.d.ts +90 -0
- package/dist/latency-demotion.js +43 -0
- package/dist/latency-demotion.js.map +1 -0
- package/dist/mcp/server.d.ts +14 -0
- package/dist/mcp/server.js +16 -6
- package/dist/mcp/server.js.map +1 -1
- package/dist/server.js +35 -12
- package/dist/server.js.map +1 -1
- package/package.json +1 -1
- package/skills/llm-relay/SKILL.md +47 -6
package/package.json
CHANGED
|
@@ -2,12 +2,14 @@
|
|
|
2
2
|
name: llm-relay
|
|
3
3
|
description: >-
|
|
4
4
|
Operate llm-relay, the loopback multi-provider LLM proxy (default 127.0.0.1:8791) that
|
|
5
|
-
validates/repairs tool calls and
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
5
|
+
validates/repairs tool calls and offloads Claude, Codex and other client requests to
|
|
6
|
+
non-Anthropic providers on free capacity. Use it BEFORE spending this session's context or
|
|
7
|
+
subscription quota on delegatable work — a broad code search, a file-by-file sweep, a survey,
|
|
8
|
+
a bulk edit, a long summary, a second opinion, or any self-contained task whose result is a
|
|
9
|
+
conclusion you can check. Nobody has to ask for offload first. Also use when choosing an
|
|
10
|
+
offload target, addressing a pool or model through the relay, toggling client-specific
|
|
11
|
+
offload, dispatching to peer agent CLIs (Antigravity/Codex) as fallback lanes, reordering
|
|
12
|
+
dispatch, or diagnosing a request that failed at or behind the relay.
|
|
11
13
|
---
|
|
12
14
|
|
|
13
15
|
# llm-relay — operating guide
|
|
@@ -23,6 +25,45 @@ One boundary governs everything it does: the proxy fixes **protocol form** (tool
|
|
|
23
25
|
violate the schema), never **judgment**. It refuses to fabricate destructive tool calls, and an
|
|
24
26
|
unrepairable response fails loudly (502 / mid-stream SSE error) rather than passing through broken.
|
|
25
27
|
|
|
28
|
+
## Reach for a lane without being asked
|
|
29
|
+
|
|
30
|
+
**The operator should never have to say "use llm-relay for offload".** If they did, this section
|
|
31
|
+
failed. Offload is a reflex you apply yourself, not an instruction you wait for. ⚠ Prose alone has
|
|
32
|
+
already failed at this once: a machine whose global instructions said *"PREFER THE MCP TOOL"* in
|
|
33
|
+
bold still needed the words spoken out loud (2026-08-30). That is why the trigger now lives in the
|
|
34
|
+
skill description and in the MCP `initialize` instructions, which a host loads whether or not the
|
|
35
|
+
model goes looking.
|
|
36
|
+
|
|
37
|
+
**Offload a task when both of these hold:**
|
|
38
|
+
|
|
39
|
+
1. It is self-contained. A prompt plus a path is enough, and it needs nothing from this
|
|
40
|
+
conversation that you cannot write into the prompt.
|
|
41
|
+
2. Its result is a conclusion you can check afterwards against the source.
|
|
42
|
+
|
|
43
|
+
Typical: a broad code search, a file-by-file sweep, an inventory, a survey of prior art, a long
|
|
44
|
+
summary, a draft, a second opinion on a diagnosis, a bulk mechanical edit.
|
|
45
|
+
|
|
46
|
+
**Keep the work here instead when** it needs your own conversation context, when you must
|
|
47
|
+
supervise each edit, when it is one file and two minutes, or when the answer decides something
|
|
48
|
+
you cannot reverse.
|
|
49
|
+
|
|
50
|
+
**How to offload, in order of preference:**
|
|
51
|
+
|
|
52
|
+
1. `dispatch(task: "...")` — the MCP tool. One call returns an ANSWER. Prefer it whenever the
|
|
53
|
+
tools are present, because building and running a lane command correctly is the part that keeps
|
|
54
|
+
going wrong; the server handles the working directory, the environment and the idle timeouts.
|
|
55
|
+
2. `llm-relay dispatch --next-command -t "<task>"` — when the MCP tools are absent. Exit 0 prints
|
|
56
|
+
one runnable command line. Exit 2 means the rung is a relay target, so address the named spec
|
|
57
|
+
as an ordinary subagent.
|
|
58
|
+
|
|
59
|
+
⚠ Free capacity is spent before any metered or subscription lane, so an offloaded task normally
|
|
60
|
+
costs no subscription quota.
|
|
61
|
+
|
|
62
|
+
⚠ **Lane output is advisory.** Verify every claim against the source before you act on it. For a
|
|
63
|
+
lane that WRITES, inspect the tree yourself (`git status --porcelain`, `git diff`,
|
|
64
|
+
`git show --stat HEAD`) rather than trusting the lane's own report, and give every writing lane its
|
|
65
|
+
own worktree.
|
|
66
|
+
|
|
26
67
|
## First use on a machine — ASK, do not assume
|
|
27
68
|
|
|
28
69
|
A fresh install writes a config that **changes nothing**: every Claude model id reaches real
|