@lawaty/lacode 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +117 -0
- package/dist/agents.js +1 -0
- package/dist/index.js +143 -28
- package/dist/pool.js +78 -8
- package/docs/USER-STORIES.md +5 -5
- package/package.json +1 -1
- package/presets/README.md +18 -7
- package/presets/free-tier.json +20 -6
package/README.md
CHANGED
|
@@ -337,6 +337,9 @@ as an opencode provider for you. It is **off by default**: if you do not set
|
|
|
337
337
|
disabled for that run, and the integration is per-run.
|
|
338
338
|
2. Add the plugin block to your opencode config:
|
|
339
339
|
`["@lawaty/lacode", { "omniroute": { "enabled": true, "apiKey": "<your omniroute key>", "combo": "auto", "models": [] } }]`
|
|
340
|
+
`auto` is the zero-config default, but on a free tier it often resolves to a
|
|
341
|
+
slow chain. [A tested fast combo](#a-tested-fast-combo-free-mix) below is
|
|
342
|
+
ordered for latency and is worth applying on a new client.
|
|
340
343
|
3. Optionally, to route pooled work through it, make `omniroute/auto 3` the
|
|
341
344
|
**first** line of `~/.config/lacode/pool.txt`. `pool.txt` is optional user
|
|
342
345
|
config; the bundled default preset does **not** include OmniRoute.
|
|
@@ -426,6 +429,98 @@ the only thing that changes when the gateway is down is that `omniroute/<id>`
|
|
|
426
429
|
fails; every other slot keeps working. See
|
|
427
430
|
[docs/USER-STORIES.md](docs/USER-STORIES.md) US-28.
|
|
428
431
|
|
|
432
|
+
### A tested fast combo: `free-mix`
|
|
433
|
+
|
|
434
|
+
`combo: "auto"` is the zero-configuration default, but `auto` inherits whatever
|
|
435
|
+
the gateway chooses, and on a busy free tier that is frequently the slowest
|
|
436
|
+
chain available. The combo below was **measured** against a live free-tier
|
|
437
|
+
gateway and is ordered for latency rather than for breadth. Apply it during the
|
|
438
|
+
"Setting it up on a client machine" steps above and pooled work is usable
|
|
439
|
+
straight away, instead of starting from a gateway that queues for half a minute
|
|
440
|
+
per call.
|
|
441
|
+
|
|
442
|
+
```jsonc
|
|
443
|
+
{
|
|
444
|
+
"name": "free-mix",
|
|
445
|
+
// Order decides. Do NOT use "weighted" here — weights let the gateway
|
|
446
|
+
// re-shuffle a model that is returning 429 into first position again.
|
|
447
|
+
"strategy": "priority",
|
|
448
|
+
"models": [
|
|
449
|
+
{ "model": "gemini/gemini-3.1-flash-lite", "providerId": "gemini" },
|
|
450
|
+
{ "model": "gemini/gemini-3.8-flash", "providerId": "gemini" },
|
|
451
|
+
{ "model": "gemini/gemini-3.7-flash", "providerId": "gemini" },
|
|
452
|
+
{ "model": "gemini/gemini-3-flash-preview", "providerId": "gemini" },
|
|
453
|
+
{ "model": "openrouter/nvidia/nemotron-3-super-120b-a12b:free", "providerId": "openrouter" },
|
|
454
|
+
{ "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free", "providerId": "openrouter" },
|
|
455
|
+
{ "model": "openrouter/nvidia/nemotron-3.5-lightning:free", "providerId": "openrouter" },
|
|
456
|
+
{ "model": "openrouter/cohere/north-mini-code:free", "providerId": "openrouter" },
|
|
457
|
+
{ "model": "openrouter/poolside/laguna-s-2.1:free", "providerId": "openrouter" },
|
|
458
|
+
{ "model": "openrouter/dots-studio/dots-3-note-preview:free", "providerId": "openrouter" }
|
|
459
|
+
]
|
|
460
|
+
}
|
|
461
|
+
```
|
|
462
|
+
|
|
463
|
+
Why this order, from 859 calls over a single day against that gateway:
|
|
464
|
+
|
|
465
|
+
| Model | Observed behaviour | Position |
|
|
466
|
+
| --- | --- | --- |
|
|
467
|
+
| `gemini/gemini-3.1-flash-lite` | **165/172 succeeded (96%)**, ~4.5s median | **first** |
|
|
468
|
+
| `gemini-3.8-flash` / `3.7-flash` / `3-flash-preview` | succeed intermittently, but **burn exactly 30.0s** on a `503` queue-budget timeout when they do not | next |
|
|
469
|
+
| `openrouter/*:free` | **~90% `429`**, but fails *fast* (~1.8s) and occasionally succeeds | last, as a cheap final fallback |
|
|
470
|
+
| `gemini-2.5-flash`, `gemini-2.5-flash-lite` | **`404` on every attempt** — dead ids | **omitted entirely** |
|
|
471
|
+
|
|
472
|
+
The ordering matters because a combo pays for *every* step it walks past. Put the
|
|
473
|
+
96%-success model first and the common case is a single hop; the 30s models sit
|
|
474
|
+
behind it where they cost nothing, and the 429s sit last where they give up in
|
|
475
|
+
under two seconds.
|
|
476
|
+
|
|
477
|
+
Measured on the same gateway, same day, `max_tokens: 10`:
|
|
478
|
+
|
|
479
|
+
| | Before | After |
|
|
480
|
+
| --- | --- | --- |
|
|
481
|
+
| `free-mix` response | 42.7s / 41.8s / 34.7s | 4.1s / 3.1s / 3.2s / 4.0s / 3.2s |
|
|
482
|
+
| Median | **~41.8s** | **~3.2s** |
|
|
483
|
+
| Provider time burned on calls >25s | **56.9 of 102.4 min (56%)** | — |
|
|
484
|
+
|
|
485
|
+
Before the change, the real call sequence was a `429` after 2.7s, then a `503`
|
|
486
|
+
after 30.0s, then the actual work in 5.7s — roughly **33 seconds of dead time in
|
|
487
|
+
front of every successful request**, and a p90 of 100s when three 30s steps
|
|
488
|
+
chained.
|
|
489
|
+
|
|
490
|
+
These free-tier endpoints change availability constantly, so treat the ordering
|
|
491
|
+
as a **starting point to re-measure, not a permanent truth**. Re-check it with
|
|
492
|
+
OmniRoute's own call logs before assuming it still holds:
|
|
493
|
+
|
|
494
|
+
```bash
|
|
495
|
+
# one timed request through the gateway
|
|
496
|
+
curl -o /dev/null -w '%{http_code} %{time_starttransfer}s\n' \
|
|
497
|
+
-X POST http://localhost:20128/v1/chat/completions \
|
|
498
|
+
-H "Authorization: Bearer $OMNIROUTE_API_KEY" -H 'Content-Type: application/json' \
|
|
499
|
+
-d '{"model":"free-mix","messages":[{"role":"user","content":"Say OK"}],"max_tokens":10}'
|
|
500
|
+
|
|
501
|
+
# per-model status + duration, straight from OmniRoute's logs
|
|
502
|
+
python3 - <<'PY'
|
|
503
|
+
import json, glob, collections
|
|
504
|
+
d = "/home/<user>/.omniroute/call_logs/<YYYY-MM-DD>"
|
|
505
|
+
g = collections.defaultdict(collections.Counter)
|
|
506
|
+
for f in glob.glob(d + "/*.json"):
|
|
507
|
+
try: s = json.load(open(f))["summary"]
|
|
508
|
+
except Exception: continue
|
|
509
|
+
g[s.get("comboStepId")][s.get("status")] += 1
|
|
510
|
+
for k, v in sorted(g.items(), key=lambda x: str(x[0])):
|
|
511
|
+
print(f"{str(k)[:44]:44} {dict(v)}")
|
|
512
|
+
PY
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
Apply it with the bundled CLI (`PUT /api/combos/<id>` updates in place, so the
|
|
516
|
+
combo is never briefly missing):
|
|
517
|
+
|
|
518
|
+
```bash
|
|
519
|
+
omniroute api combos get-api-combos-id- --id <combo-id> --json > free-mix.json
|
|
520
|
+
# edit name/strategy/models in free-mix.json, then:
|
|
521
|
+
omniroute api combos put-api-combos-id- --id <combo-id> --body @free-mix.json
|
|
522
|
+
```
|
|
523
|
+
|
|
429
524
|
**A note on mid-session switching.** opencode 1.18.34 exposes no hook to change
|
|
430
525
|
an agent's main model mid-request, so LaCode cannot swap models inside a running
|
|
431
526
|
session; use an OmniRoute `combo` (or `auto`) if you want per-request fallback.
|
|
@@ -570,6 +665,28 @@ published, because none has been independently measured. The mechanism, the
|
|
|
570
665
|
permission rule that makes one writer possible, and what the log lines mean are
|
|
571
666
|
in [docs/CONTEXT.md](docs/CONTEXT.md).
|
|
572
667
|
|
|
668
|
+
## Managerial directions
|
|
669
|
+
|
|
670
|
+
You can optionally add `manager.md` to `.opencode/context/`. This file is owned
|
|
671
|
+
by you alone and is never written, edited, pruned, or promoted by any agent. It
|
|
672
|
+
outranks the five generated files on focus and settled decisions, but it is
|
|
673
|
+
secondary to the source code: if the code disagrees, the code wins and the
|
|
674
|
+
conflict is reported back to you.
|
|
675
|
+
|
|
676
|
+
The file is per-clone, not committed, and entirely optional: if it is missing,
|
|
677
|
+
agents behave exactly as before. When present, it should contain these three
|
|
678
|
+
sections:
|
|
679
|
+
|
|
680
|
+
- `## Stage`: A snapshot of your current work-in-progress. Rewrite this section
|
|
681
|
+
rather than appending to it to keep it concise. If this file passes roughly 150
|
|
682
|
+
lines, run `/context-review`.
|
|
683
|
+
- `## Confirmed`: Facts and decisions you have personally ratified. Treat these
|
|
684
|
+
as settled, but not as permission to skip reading the code.
|
|
685
|
+
- `## Instructions`: Standing managerial directions for all agents.
|
|
686
|
+
|
|
687
|
+
No agent may write this file at all: the context agent's edit permission denies
|
|
688
|
+
`manager.md` explicitly, so ratification can only ever come from your hand.
|
|
689
|
+
|
|
573
690
|
## Cost
|
|
574
691
|
|
|
575
692
|
The mechanism is specific and small: bulk exploration, implementation and
|
package/dist/agents.js
CHANGED
|
@@ -43,6 +43,7 @@ var DEFAULT_PRESET = join2(PRESETS_DIR, "free-tier.json");
|
|
|
43
43
|
var USER_CONFIG_DIR = join2(homedir(), ".config", "lacode");
|
|
44
44
|
var USER_MODELS_PATH = join2(USER_CONFIG_DIR, "pool.json");
|
|
45
45
|
var USER_MODELS_TXT = join2(USER_CONFIG_DIR, "pool.txt");
|
|
46
|
+
var CHILD_MEMORY_MS = 30 * 6e4;
|
|
46
47
|
|
|
47
48
|
// src/agents.ts
|
|
48
49
|
var POOLED_BASES = ["explore", "implement", "context"];
|
package/dist/index.js
CHANGED
|
@@ -243,6 +243,7 @@ function state(opts = {}) {
|
|
|
243
243
|
cooling: /* @__PURE__ */ new Map(),
|
|
244
244
|
strikes: /* @__PURE__ */ new Map(),
|
|
245
245
|
activity: /* @__PURE__ */ new Map(),
|
|
246
|
+
children: /* @__PURE__ */ new Map(),
|
|
246
247
|
reaped: 0,
|
|
247
248
|
reaping: false,
|
|
248
249
|
lastSweep: 0,
|
|
@@ -284,7 +285,7 @@ function init(s, onWarn) {
|
|
|
284
285
|
}
|
|
285
286
|
function pruneOwn(s) {
|
|
286
287
|
const now = Date.now();
|
|
287
|
-
for (const [key, claim] of s.own) if (now - claim.t > CLAIM_TTL_MS) s.own.delete(key);
|
|
288
|
+
for (const [key, claim] of s.own) if (now - (claim.hb ?? claim.t) > CLAIM_TTL_MS) s.own.delete(key);
|
|
288
289
|
}
|
|
289
290
|
function publish(s) {
|
|
290
291
|
if (!s.dirReady) return;
|
|
@@ -349,7 +350,7 @@ function readSiblings(s) {
|
|
|
349
350
|
try {
|
|
350
351
|
const parsed = JSON.parse(readFileSync2(path4, "utf8"));
|
|
351
352
|
const claims = Object.values(parsed).filter(
|
|
352
|
-
(c) => c && typeof c.v === "string" && typeof c.m === "string" && now - c.t <= CLAIM_TTL_MS
|
|
353
|
+
(c) => c && typeof c.v === "string" && typeof c.m === "string" && now - (c.hb ?? c.t) <= CLAIM_TTL_MS
|
|
353
354
|
);
|
|
354
355
|
siblings.push({ id: match[1], claims });
|
|
355
356
|
} catch {
|
|
@@ -463,26 +464,72 @@ function snapshot(s) {
|
|
|
463
464
|
}
|
|
464
465
|
var loadOf = (counts, model) => counts.get(model) ?? 0;
|
|
465
466
|
var underCeiling = (slot, counts) => loadOf(counts, slot.model) < slot.weight;
|
|
466
|
-
function
|
|
467
|
+
function roundRobinSequence(slots) {
|
|
468
|
+
const seq = [];
|
|
469
|
+
for (const slot of slots) for (let turn = 0; turn < slot.weight; turn++) seq.push(slot.index);
|
|
470
|
+
return seq;
|
|
471
|
+
}
|
|
472
|
+
var CURSOR_FILE = "cursor.json";
|
|
473
|
+
function readCursor(s) {
|
|
474
|
+
try {
|
|
475
|
+
const parsed = JSON.parse(readFileSync2(join2(s.dir, CURSOR_FILE), "utf8"));
|
|
476
|
+
if (typeof parsed.seq === "number" && Number.isInteger(parsed.seq) && parsed.seq >= 0) return parsed.seq;
|
|
477
|
+
} catch {
|
|
478
|
+
}
|
|
479
|
+
return 0;
|
|
480
|
+
}
|
|
481
|
+
function writeCursor(s, seq) {
|
|
482
|
+
if (!s.dirReady) return;
|
|
483
|
+
const path4 = join2(s.dir, CURSOR_FILE);
|
|
484
|
+
try {
|
|
485
|
+
writeFileSync(`${path4}.tmp`, JSON.stringify({ seq }));
|
|
486
|
+
renameSync(`${path4}.tmp`, path4);
|
|
487
|
+
} catch {
|
|
488
|
+
}
|
|
489
|
+
}
|
|
490
|
+
function pickSlot(s, snap, options) {
|
|
491
|
+
const advance = options?.advance !== false;
|
|
467
492
|
const slots = s.slots.length > 0 ? s.slots : FALLBACK_SLOTS;
|
|
493
|
+
const seq = roundRobinSequence(slots);
|
|
494
|
+
if (seq.length > 0) {
|
|
495
|
+
const byIndex = new Map(slots.map((slot) => [slot.index, slot]));
|
|
496
|
+
const start = readCursor(s) % seq.length;
|
|
497
|
+
for (let step = 0; step < seq.length; step++) {
|
|
498
|
+
const at = (start + step) % seq.length;
|
|
499
|
+
const slot = byIndex.get(seq[at]);
|
|
500
|
+
if (slot && isAvailable(s, slot.model, snap.cooling) && underCeiling(slot, snap.counts)) {
|
|
501
|
+
if (advance) writeCursor(s, at + 1);
|
|
502
|
+
return slot;
|
|
503
|
+
}
|
|
504
|
+
}
|
|
505
|
+
}
|
|
468
506
|
const usable = slots.filter((slot) => isAvailable(s, slot.model, snap.cooling));
|
|
469
507
|
const pool = usable.length > 0 ? usable : slots;
|
|
470
|
-
const open = pool.filter((slot) => underCeiling(slot, snap.counts));
|
|
471
|
-
if (open.length > 0) return open.reduce((best, slot) => slot.weight > best.weight ? slot : best, open[0]);
|
|
472
508
|
return pool.reduce((best, slot) => loadOf(snap.counts, slot.model) < loadOf(snap.counts, best.model) ? slot : best, pool[0]);
|
|
473
509
|
}
|
|
474
510
|
function agentFor(base, slot) {
|
|
475
511
|
return `${base}-${slot}`;
|
|
476
512
|
}
|
|
477
513
|
function acquire(s, base, slot, ownerSession, key, target) {
|
|
478
|
-
|
|
479
|
-
|
|
514
|
+
const now = Date.now();
|
|
515
|
+
s.own.set(key, { v: base, k: slot.index, m: slot.model, t: now, hb: now, s: ownerSession, g: target });
|
|
516
|
+
if (target) s.activity.set(target, now);
|
|
480
517
|
publish(s);
|
|
481
518
|
}
|
|
482
519
|
function release(s, key) {
|
|
483
520
|
if (s.own.delete(key)) publish(s);
|
|
484
521
|
}
|
|
522
|
+
var CHILD_RING = 8;
|
|
523
|
+
var CHILD_MEMORY_MS = 30 * 6e4;
|
|
524
|
+
var CHILD_SLACK_MS = 1e3;
|
|
525
|
+
function noteChild(s, parentID, childID) {
|
|
526
|
+
const ring = s.children.get(parentID) ?? [];
|
|
527
|
+
ring.push({ id: childID, t: Date.now() });
|
|
528
|
+
while (ring.length > CHILD_RING) ring.shift();
|
|
529
|
+
s.children.set(parentID, ring);
|
|
530
|
+
}
|
|
485
531
|
function bindChild(s, parentID, childID) {
|
|
532
|
+
noteChild(s, parentID, childID);
|
|
486
533
|
const candidates = [...s.own.entries()].filter(([, c]) => c.s === parentID && !c.g).sort((a, b) => a[1].t - b[1].t);
|
|
487
534
|
if (candidates.length === 0) return false;
|
|
488
535
|
candidates[0][1].g = childID;
|
|
@@ -490,8 +537,29 @@ function bindChild(s, parentID, childID) {
|
|
|
490
537
|
publish(s);
|
|
491
538
|
return true;
|
|
492
539
|
}
|
|
540
|
+
function backfillChild(s, claim) {
|
|
541
|
+
if (claim.g || !claim.s) return false;
|
|
542
|
+
const now = Date.now();
|
|
543
|
+
const taken = new Set([...s.own.values()].map((c) => c.g).filter((g) => typeof g === "string"));
|
|
544
|
+
const ring = s.children.get(claim.s) ?? [];
|
|
545
|
+
for (let i = ring.length - 1; i >= 0; i--) {
|
|
546
|
+
const child = ring[i];
|
|
547
|
+
if (now - child.t > CHILD_MEMORY_MS) break;
|
|
548
|
+
if (child.t < claim.t - CHILD_SLACK_MS) continue;
|
|
549
|
+
if (taken.has(child.id)) continue;
|
|
550
|
+
claim.g = child.id;
|
|
551
|
+
s.activity.set(child.id, now);
|
|
552
|
+
publish(s);
|
|
553
|
+
return true;
|
|
554
|
+
}
|
|
555
|
+
return false;
|
|
556
|
+
}
|
|
493
557
|
function noteActivity(s, sessionID) {
|
|
494
|
-
|
|
558
|
+
const now = Date.now();
|
|
559
|
+
s.activity.set(sessionID, now);
|
|
560
|
+
let touched = false;
|
|
561
|
+
for (const claim of s.own.values()) if (claim.g === sessionID) claim.hb = now, touched = true;
|
|
562
|
+
if (touched) publish(s);
|
|
495
563
|
}
|
|
496
564
|
function idleFor(s, claim, now) {
|
|
497
565
|
const target = claim.g;
|
|
@@ -1342,6 +1410,7 @@ import { tool } from "@opencode-ai/plugin";
|
|
|
1342
1410
|
var SERVICE = "agent-pool";
|
|
1343
1411
|
var REAP_INTERVAL_MS = 3e4;
|
|
1344
1412
|
var HANG_FALLBACK_MS = 10 * 60 * 1e3;
|
|
1413
|
+
var ABORT_TIMEOUT_MS = 1e4;
|
|
1345
1414
|
var agentPoolHooks = (async ({ client, serverUrl }, options) => {
|
|
1346
1415
|
const opts = options ?? {};
|
|
1347
1416
|
const s = state(opts);
|
|
@@ -1355,14 +1424,28 @@ var agentPoolHooks = (async ({ client, serverUrl }, options) => {
|
|
|
1355
1424
|
};
|
|
1356
1425
|
init(s, (m) => void warnOnce(m));
|
|
1357
1426
|
const abortSession = async (sessionID) => {
|
|
1358
|
-
if (!serverUrl) return false;
|
|
1427
|
+
if (!serverUrl) return { ok: false, why: "no serverUrl: this process has no HTTP server to abort through" };
|
|
1359
1428
|
try {
|
|
1360
|
-
await fetch(new URL(`/session/${sessionID}/abort`, serverUrl).toString(), {
|
|
1361
|
-
|
|
1362
|
-
|
|
1363
|
-
|
|
1429
|
+
const res = await fetch(new URL(`/session/${sessionID}/abort`, serverUrl).toString(), {
|
|
1430
|
+
method: "POST",
|
|
1431
|
+
signal: AbortSignal.timeout(ABORT_TIMEOUT_MS)
|
|
1432
|
+
});
|
|
1433
|
+
if (!res.ok) return { ok: false, why: `gateway answered HTTP ${res.status}` };
|
|
1434
|
+
const body = (await res.text()).trim();
|
|
1435
|
+
if (body === "false") return { ok: false, why: "server reported the session was not aborted" };
|
|
1436
|
+
return { ok: true, why: "" };
|
|
1437
|
+
} catch (error) {
|
|
1438
|
+
return { ok: false, why: String(error) };
|
|
1364
1439
|
}
|
|
1365
1440
|
};
|
|
1441
|
+
const MAX_ABORT_ATTEMPTS = 3;
|
|
1442
|
+
const coolForHang = async (claim, log2) => {
|
|
1443
|
+
const now = Date.now();
|
|
1444
|
+
const prior = s.cooling.get(claim.m);
|
|
1445
|
+
if (prior && prior.until > now) return;
|
|
1446
|
+
const strike = recordStrike(s, claim.m, "hang", HANG_FALLBACK_MS);
|
|
1447
|
+
if (strike) await log2("warn", `cooling ${claim.m} for ${Math.round((strike.until - now) / 1e3)}s (hang)`);
|
|
1448
|
+
};
|
|
1366
1449
|
const reapStuck = async () => {
|
|
1367
1450
|
if (s.reaping) return;
|
|
1368
1451
|
s.reaping = true;
|
|
@@ -1370,21 +1453,53 @@ var agentPoolHooks = (async ({ client, serverUrl }, options) => {
|
|
|
1370
1453
|
try {
|
|
1371
1454
|
const stuck = findStuck(s);
|
|
1372
1455
|
if (stuck.length === 0) return;
|
|
1373
|
-
|
|
1374
|
-
for (const { claim } of stuck) {
|
|
1375
|
-
|
|
1456
|
+
const released = [];
|
|
1457
|
+
for (const { key, claim } of stuck) {
|
|
1458
|
+
if (!claim.g) backfillChild(s, claim);
|
|
1459
|
+
const idle = idleFor(s, claim, Date.now());
|
|
1460
|
+
const since = Number.isFinite(idle) ? `${Math.round(idle / 1e3)}s of silence` : "no child session ever bound";
|
|
1461
|
+
const age = Math.round((Date.now() - claim.t) / 1e3);
|
|
1462
|
+
if (claim.g) {
|
|
1463
|
+
const kill = await abortSession(claim.g);
|
|
1464
|
+
if (kill.ok) {
|
|
1465
|
+
released.push(key);
|
|
1466
|
+
s.reaped++;
|
|
1467
|
+
await log(
|
|
1468
|
+
"warn",
|
|
1469
|
+
`hung ${claim.v} on ${claim.m} after ${age}s with ${since}; aborted session ${claim.g}`,
|
|
1470
|
+
{ base: claim.v, model: claim.m, slot: claim.k, session: claim.g, aborted: true }
|
|
1471
|
+
);
|
|
1472
|
+
await coolForHang(claim, log);
|
|
1473
|
+
continue;
|
|
1474
|
+
}
|
|
1475
|
+
claim.n = (claim.n ?? 0) + 1;
|
|
1476
|
+
if (claim.n < MAX_ABORT_ATTEMPTS) {
|
|
1477
|
+
await log(
|
|
1478
|
+
"warn",
|
|
1479
|
+
`could not abort hung ${claim.v} session ${claim.g} on ${claim.m} (${kill.why}); claim held, retry ${claim.n}/${MAX_ABORT_ATTEMPTS}`,
|
|
1480
|
+
{ base: claim.v, model: claim.m, slot: claim.k, session: claim.g, aborted: false }
|
|
1481
|
+
);
|
|
1482
|
+
continue;
|
|
1483
|
+
}
|
|
1484
|
+
released.push(key);
|
|
1485
|
+
s.reaped++;
|
|
1486
|
+
await log(
|
|
1487
|
+
"error",
|
|
1488
|
+
`GIVING UP on hung ${claim.v} session ${claim.g} on ${claim.m}: ${kill.why} after ${claim.n} attempts. The claim is released but the session is still running and must be killed by hand.`,
|
|
1489
|
+
{ base: claim.v, model: claim.m, slot: claim.k, session: claim.g, aborted: false }
|
|
1490
|
+
);
|
|
1491
|
+
continue;
|
|
1492
|
+
}
|
|
1493
|
+
released.push(key);
|
|
1376
1494
|
s.reaped++;
|
|
1377
|
-
await log(
|
|
1378
|
-
|
|
1379
|
-
|
|
1380
|
-
|
|
1381
|
-
|
|
1382
|
-
|
|
1383
|
-
const prior = s.cooling.get(claim.m);
|
|
1384
|
-
if (prior && prior.until > now) continue;
|
|
1385
|
-
const strike = recordStrike(s, claim.m, "hang", HANG_FALLBACK_MS);
|
|
1386
|
-
if (strike) await log("warn", `cooling ${claim.m} for ${Math.round((strike.until - now) / 1e3)}s (hang)`);
|
|
1495
|
+
await log("warn", `released unbound ${claim.v} claim on ${claim.m} after ${age}s (${since}); model not cooled`, {
|
|
1496
|
+
base: claim.v,
|
|
1497
|
+
model: claim.m,
|
|
1498
|
+
slot: claim.k,
|
|
1499
|
+
aborted: false
|
|
1500
|
+
});
|
|
1387
1501
|
}
|
|
1502
|
+
if (released.length > 0) releaseStuck(s, released);
|
|
1388
1503
|
} catch (error) {
|
|
1389
1504
|
await log("error", `reaper failed: ${String(error)}`);
|
|
1390
1505
|
} finally {
|
|
@@ -1549,7 +1664,7 @@ var agentPoolHooks = (async ({ client, serverUrl }, options) => {
|
|
|
1549
1664
|
};
|
|
1550
1665
|
for (const claim of snap.own.values()) add(claim.m, claim.v);
|
|
1551
1666
|
for (const sibling of snap.siblings) for (const claim of sibling.claims) add(claim.m, claim.v);
|
|
1552
|
-
const next = pickSlot(s, snap);
|
|
1667
|
+
const next = pickSlot(s, snap, { advance: false });
|
|
1553
1668
|
const line = (slot) => {
|
|
1554
1669
|
const entry = byModel.get(slot.model);
|
|
1555
1670
|
const users = entry && entry.bases.size > 0 ? [...entry.bases].map(([b, n]) => `${b}x${n}`).join(" ") : "idle";
|
|
@@ -1571,7 +1686,7 @@ var agentPoolHooks = (async ({ client, serverUrl }, options) => {
|
|
|
1571
1686
|
title: `agent-pool: ${total} task(s) in flight, ${coolingNow.length} model(s) cooling`,
|
|
1572
1687
|
output: [
|
|
1573
1688
|
`shared pool over ${slots.length} model(s) from ${s.origin}; bases: ${BASES.join(", ")}
|
|
1574
|
-
|
|
1689
|
+
round robin in slot order, each slot taking as many turns as its weight says, so no slot is starved while another drains (FULL = at ceiling, used only when every slot is)`,
|
|
1575
1690
|
...slots.map(line),
|
|
1576
1691
|
"",
|
|
1577
1692
|
"by process:",
|
package/dist/pool.js
CHANGED
|
@@ -243,6 +243,7 @@ function state(opts = {}) {
|
|
|
243
243
|
cooling: /* @__PURE__ */ new Map(),
|
|
244
244
|
strikes: /* @__PURE__ */ new Map(),
|
|
245
245
|
activity: /* @__PURE__ */ new Map(),
|
|
246
|
+
children: /* @__PURE__ */ new Map(),
|
|
246
247
|
reaped: 0,
|
|
247
248
|
reaping: false,
|
|
248
249
|
lastSweep: 0,
|
|
@@ -284,7 +285,7 @@ function init(s, onWarn) {
|
|
|
284
285
|
}
|
|
285
286
|
function pruneOwn(s) {
|
|
286
287
|
const now = Date.now();
|
|
287
|
-
for (const [key, claim] of s.own) if (now - claim.t > CLAIM_TTL_MS) s.own.delete(key);
|
|
288
|
+
for (const [key, claim] of s.own) if (now - (claim.hb ?? claim.t) > CLAIM_TTL_MS) s.own.delete(key);
|
|
288
289
|
}
|
|
289
290
|
function publish(s) {
|
|
290
291
|
if (!s.dirReady) return;
|
|
@@ -349,7 +350,7 @@ function readSiblings(s) {
|
|
|
349
350
|
try {
|
|
350
351
|
const parsed = JSON.parse(readFileSync2(path, "utf8"));
|
|
351
352
|
const claims = Object.values(parsed).filter(
|
|
352
|
-
(c) => c && typeof c.v === "string" && typeof c.m === "string" && now - c.t <= CLAIM_TTL_MS
|
|
353
|
+
(c) => c && typeof c.v === "string" && typeof c.m === "string" && now - (c.hb ?? c.t) <= CLAIM_TTL_MS
|
|
353
354
|
);
|
|
354
355
|
siblings.push({ id: match[1], claims });
|
|
355
356
|
} catch {
|
|
@@ -463,26 +464,72 @@ function snapshot(s) {
|
|
|
463
464
|
}
|
|
464
465
|
var loadOf = (counts, model) => counts.get(model) ?? 0;
|
|
465
466
|
var underCeiling = (slot, counts) => loadOf(counts, slot.model) < slot.weight;
|
|
466
|
-
function
|
|
467
|
+
function roundRobinSequence(slots) {
|
|
468
|
+
const seq = [];
|
|
469
|
+
for (const slot of slots) for (let turn = 0; turn < slot.weight; turn++) seq.push(slot.index);
|
|
470
|
+
return seq;
|
|
471
|
+
}
|
|
472
|
+
var CURSOR_FILE = "cursor.json";
|
|
473
|
+
function readCursor(s) {
|
|
474
|
+
try {
|
|
475
|
+
const parsed = JSON.parse(readFileSync2(join2(s.dir, CURSOR_FILE), "utf8"));
|
|
476
|
+
if (typeof parsed.seq === "number" && Number.isInteger(parsed.seq) && parsed.seq >= 0) return parsed.seq;
|
|
477
|
+
} catch {
|
|
478
|
+
}
|
|
479
|
+
return 0;
|
|
480
|
+
}
|
|
481
|
+
function writeCursor(s, seq) {
|
|
482
|
+
if (!s.dirReady) return;
|
|
483
|
+
const path = join2(s.dir, CURSOR_FILE);
|
|
484
|
+
try {
|
|
485
|
+
writeFileSync(`${path}.tmp`, JSON.stringify({ seq }));
|
|
486
|
+
renameSync(`${path}.tmp`, path);
|
|
487
|
+
} catch {
|
|
488
|
+
}
|
|
489
|
+
}
|
|
490
|
+
function pickSlot(s, snap, options) {
|
|
491
|
+
const advance = options?.advance !== false;
|
|
467
492
|
const slots = s.slots.length > 0 ? s.slots : FALLBACK_SLOTS;
|
|
493
|
+
const seq = roundRobinSequence(slots);
|
|
494
|
+
if (seq.length > 0) {
|
|
495
|
+
const byIndex = new Map(slots.map((slot) => [slot.index, slot]));
|
|
496
|
+
const start = readCursor(s) % seq.length;
|
|
497
|
+
for (let step = 0; step < seq.length; step++) {
|
|
498
|
+
const at = (start + step) % seq.length;
|
|
499
|
+
const slot = byIndex.get(seq[at]);
|
|
500
|
+
if (slot && isAvailable(s, slot.model, snap.cooling) && underCeiling(slot, snap.counts)) {
|
|
501
|
+
if (advance) writeCursor(s, at + 1);
|
|
502
|
+
return slot;
|
|
503
|
+
}
|
|
504
|
+
}
|
|
505
|
+
}
|
|
468
506
|
const usable = slots.filter((slot) => isAvailable(s, slot.model, snap.cooling));
|
|
469
507
|
const pool = usable.length > 0 ? usable : slots;
|
|
470
|
-
const open = pool.filter((slot) => underCeiling(slot, snap.counts));
|
|
471
|
-
if (open.length > 0) return open.reduce((best, slot) => slot.weight > best.weight ? slot : best, open[0]);
|
|
472
508
|
return pool.reduce((best, slot) => loadOf(snap.counts, slot.model) < loadOf(snap.counts, best.model) ? slot : best, pool[0]);
|
|
473
509
|
}
|
|
474
510
|
function agentFor(base, slot) {
|
|
475
511
|
return `${base}-${slot}`;
|
|
476
512
|
}
|
|
477
513
|
function acquire(s, base, slot, ownerSession, key, target) {
|
|
478
|
-
|
|
479
|
-
|
|
514
|
+
const now = Date.now();
|
|
515
|
+
s.own.set(key, { v: base, k: slot.index, m: slot.model, t: now, hb: now, s: ownerSession, g: target });
|
|
516
|
+
if (target) s.activity.set(target, now);
|
|
480
517
|
publish(s);
|
|
481
518
|
}
|
|
482
519
|
function release(s, key) {
|
|
483
520
|
if (s.own.delete(key)) publish(s);
|
|
484
521
|
}
|
|
522
|
+
var CHILD_RING = 8;
|
|
523
|
+
var CHILD_MEMORY_MS = 30 * 6e4;
|
|
524
|
+
var CHILD_SLACK_MS = 1e3;
|
|
525
|
+
function noteChild(s, parentID, childID) {
|
|
526
|
+
const ring = s.children.get(parentID) ?? [];
|
|
527
|
+
ring.push({ id: childID, t: Date.now() });
|
|
528
|
+
while (ring.length > CHILD_RING) ring.shift();
|
|
529
|
+
s.children.set(parentID, ring);
|
|
530
|
+
}
|
|
485
531
|
function bindChild(s, parentID, childID) {
|
|
532
|
+
noteChild(s, parentID, childID);
|
|
486
533
|
const candidates = [...s.own.entries()].filter(([, c]) => c.s === parentID && !c.g).sort((a, b) => a[1].t - b[1].t);
|
|
487
534
|
if (candidates.length === 0) return false;
|
|
488
535
|
candidates[0][1].g = childID;
|
|
@@ -490,8 +537,29 @@ function bindChild(s, parentID, childID) {
|
|
|
490
537
|
publish(s);
|
|
491
538
|
return true;
|
|
492
539
|
}
|
|
540
|
+
function backfillChild(s, claim) {
|
|
541
|
+
if (claim.g || !claim.s) return false;
|
|
542
|
+
const now = Date.now();
|
|
543
|
+
const taken = new Set([...s.own.values()].map((c) => c.g).filter((g) => typeof g === "string"));
|
|
544
|
+
const ring = s.children.get(claim.s) ?? [];
|
|
545
|
+
for (let i = ring.length - 1; i >= 0; i--) {
|
|
546
|
+
const child = ring[i];
|
|
547
|
+
if (now - child.t > CHILD_MEMORY_MS) break;
|
|
548
|
+
if (child.t < claim.t - CHILD_SLACK_MS) continue;
|
|
549
|
+
if (taken.has(child.id)) continue;
|
|
550
|
+
claim.g = child.id;
|
|
551
|
+
s.activity.set(child.id, now);
|
|
552
|
+
publish(s);
|
|
553
|
+
return true;
|
|
554
|
+
}
|
|
555
|
+
return false;
|
|
556
|
+
}
|
|
493
557
|
function noteActivity(s, sessionID) {
|
|
494
|
-
|
|
558
|
+
const now = Date.now();
|
|
559
|
+
s.activity.set(sessionID, now);
|
|
560
|
+
let touched = false;
|
|
561
|
+
for (const claim of s.own.values()) if (claim.g === sessionID) claim.hb = now, touched = true;
|
|
562
|
+
if (touched) publish(s);
|
|
495
563
|
}
|
|
496
564
|
function idleFor(s, claim, now) {
|
|
497
565
|
const target = claim.g;
|
|
@@ -555,6 +623,7 @@ export {
|
|
|
555
623
|
USER_MODELS_TXT,
|
|
556
624
|
acquire,
|
|
557
625
|
agentFor,
|
|
626
|
+
backfillChild,
|
|
558
627
|
bindChild,
|
|
559
628
|
claimKey,
|
|
560
629
|
coolProvider,
|
|
@@ -585,6 +654,7 @@ export {
|
|
|
585
654
|
releaseStuck,
|
|
586
655
|
resolveModels,
|
|
587
656
|
retryAfterMs,
|
|
657
|
+
roundRobinSequence,
|
|
588
658
|
selfFile,
|
|
589
659
|
slotFor,
|
|
590
660
|
snapshot,
|
package/docs/USER-STORIES.md
CHANGED
|
@@ -96,13 +96,13 @@ The reconciliation with US-7 is that **acceptance and endorsement are different
|
|
|
96
96
|
- [ ] Given every model is cooling, When a spawn is routed, Then the least-loaded slot is chosen anyway (degrade, never block).
|
|
97
97
|
- [ ] Given a limit error names a reset time, When the cooldown is recorded, Then the stated reset is used instead of a guessed backoff.
|
|
98
98
|
|
|
99
|
-
### US-5: Weighted
|
|
100
|
-
**As a** user, **I want** to
|
|
99
|
+
### US-5: Weighted round-robin gives every slot a turn per cycle
|
|
100
|
+
**As a** user, **I want** weights to say how many turns each slot takes per rotation cycle, **so that** no provider is drained to its limit while the others sit idle.
|
|
101
101
|
|
|
102
102
|
**Story ID(s) for tests:** US-5
|
|
103
103
|
**Acceptance criteria:**
|
|
104
|
-
- [ ] Given a heavy slot and a light slot, When spawns are routed below both ceilings, Then the
|
|
105
|
-
- [ ] Given a slot is at its weight ceiling, When
|
|
104
|
+
- [ ] Given a heavy slot and a light slot, When spawns are routed below both ceilings, Then the rotation walks the list in order with each slot taking as many consecutive turns as its weight says, rather than the heavy slot being filled and refilled first.
|
|
105
|
+
- [ ] Given a slot is at its weight ceiling or cooling, When its turn comes up in the rotation, Then the turn is skipped and the rotation resumes at the following turn.
|
|
106
106
|
- [ ] Given every slot is at its ceiling, When another spawn is routed, Then it goes to the least-loaded slot as overflow.
|
|
107
107
|
- [ ] Given all slots carry equal weight, When spawns are routed, Then the picks round-robin in declaration order.
|
|
108
108
|
- [ ] Given per-slot load is read, When it is asserted, Then a slot's load never exceeds its weight ceiling except by overflow.
|
|
@@ -115,7 +115,7 @@ The reconciliation with US-7 is that **acceptance and endorsement are different
|
|
|
115
115
|
- [ ] Given claims are held by several processes, When the status view is requested, Then it shows every process's claims and the aggregate load per slot.
|
|
116
116
|
- [ ] Given a model is cooling with a stated reason, When the status view is requested, Then the model is listed as cooling together with that reason.
|
|
117
117
|
- [ ] Given a claim file is older than the dead-file threshold, When the status view is computed, Then the stale claim is pruned rather than counted.
|
|
118
|
-
- [ ] Given a slot is at its ceiling, When the status view is requested, Then it is marked as the slot the next spawn would take.
|
|
118
|
+
- [ ] Given a slot is at its ceiling, When the status view is requested, Then it is not marked as the slot the next spawn would take, because the rotation's turns are skipped for a full slot until it frees a claim.
|
|
119
119
|
|
|
120
120
|
### US-7: Non-free models still work
|
|
121
121
|
**As a** user who wants a paid model in the pool, **I want** to supply any model list, **so that** the pool is not restricted to free tiers.
|
package/package.json
CHANGED
package/presets/README.md
CHANGED
|
@@ -62,7 +62,7 @@ else means the flat form.
|
|
|
62
62
|
| Field | Required | Meaning |
|
|
63
63
|
|-------|----------|---------|
|
|
64
64
|
| `model` | yes | `provider/model-id`, split on the **first** slash only |
|
|
65
|
-
| `weight` | no | soft ceiling on concurrent claims, and
|
|
65
|
+
| `weight` | no | soft ceiling on concurrent claims, and turns taken per rotation. Absent = 1 |
|
|
66
66
|
|
|
67
67
|
`$comment` is a JSON-legal array of strings and is ignored by the loader; the
|
|
68
68
|
bundled preset uses it to explain itself in place.
|
|
@@ -91,17 +91,28 @@ because the pool requires it.
|
|
|
91
91
|
|
|
92
92
|
## Weight, precisely
|
|
93
93
|
|
|
94
|
-
A weight is a count of concurrent sessions
|
|
95
|
-
|
|
94
|
+
A weight is a count of concurrent sessions — a soft ceiling — and the number
|
|
95
|
+
of turns a slot takes per rotation. Routing is **order-first weighted
|
|
96
|
+
round-robin**:
|
|
96
97
|
|
|
97
|
-
1. the
|
|
98
|
-
|
|
98
|
+
1. the slot list is expanded into a run in which each slot appears once per
|
|
99
|
+
unit of its weight — `[3,2,1,1]` becomes `1,1,1,2,2,3,4` — and the next
|
|
100
|
+
spawn takes the turn a persistent cursor (a file in the pool directory,
|
|
101
|
+
shared by every opencode process) points at
|
|
102
|
+
2. a turn whose slot is cooling or already at its ceiling is **skipped**, and
|
|
103
|
+
the rotation resumes at the following turn
|
|
99
104
|
3. once **every** slot is at its ceiling, overflow goes to the least loaded slot —
|
|
100
105
|
a burst past the summed ceilings degrades to even spreading and never blocks
|
|
101
106
|
|
|
107
|
+
Weight is no longer a priority: a heavy slot is not drained to its ceiling
|
|
108
|
+
before the others see any work. Every slot gets a turn each cycle, so a fast
|
|
109
|
+
provider with a large usage window is used early in the cycle while it is
|
|
110
|
+
still fast — instead of one provider being refilled until it hits its limit
|
|
111
|
+
and its 429s displace every agent type at once.
|
|
112
|
+
|
|
102
113
|
Equal weights give a plain round robin in list order. List order is the
|
|
103
|
-
|
|
104
|
-
|
|
114
|
+
rotation itself, so put the models you want taking the earlier turns of a
|
|
115
|
+
cycle at the top and the ones you want taking the later turns at the bottom.
|
|
105
116
|
|
|
106
117
|
Spreading at least two providers across the slots is worth doing deliberately: a
|
|
107
118
|
provider-wide rate limit takes out every slot on that provider at once.
|
package/presets/free-tier.json
CHANGED
|
@@ -14,20 +14,34 @@
|
|
|
14
14
|
"route at an agent that does not exist. Extra entries are dropped with a warning.",
|
|
15
15
|
"",
|
|
16
16
|
"WEIGHT IS THE ONLY KNOWN KNOB, and it is a SOFT CEILING on how many concurrent claims a",
|
|
17
|
-
"slot holds -- not a share of the work
|
|
18
|
-
"
|
|
19
|
-
"
|
|
17
|
+
"slot holds -- not a share of the work and not a priority. Routing is ORDER-FIRST",
|
|
18
|
+
"WEIGHTED ROUND-ROBIN: the slot list is walked IN ORDER and each slot takes as many",
|
|
19
|
+
"consecutive turns as its weight says, so weights 3,2,1,1 produce the run 1,1,1,2,2,3,4,",
|
|
20
|
+
"which then repeats. The rule is:",
|
|
21
|
+
" 1. a persistent cursor in the pool directory marks where in that run the next spawn",
|
|
22
|
+
" lands, so every opencode process on the machine shares one rotation",
|
|
23
|
+
" 2. a turn is SKIPPED only when its slot is cooling or already at its ceiling, and the",
|
|
24
|
+
" rotation resumes at the following turn",
|
|
20
25
|
" 3. once EVERY slot is at its ceiling, the overflow goes to the least loaded slot,",
|
|
21
26
|
" so a burst past the summed ceilings degrades to even spreading and never blocks",
|
|
22
27
|
"",
|
|
23
|
-
"With this file: space-bunny takes 3
|
|
28
|
+
"With this file: space-bunny takes 3 consecutive turns, then step-5-preview",
|
|
24
29
|
"(opencode-go) takes 2, then big-pickle and step-5-preview (Zen) take 1 each -- sequence 1,1,1,2,2,3,4,...",
|
|
25
30
|
"Equal weights give a plain round robin in list order. 2,2,1,1 reproduces the old",
|
|
26
31
|
"primary/overflow cycle exactly, which is why the old tier values still work.",
|
|
27
32
|
"",
|
|
33
|
+
"Round-robin rather than heaviest-first because re-prioritising the heaviest slot on",
|
|
34
|
+
"EVERY spawn drained the fastest provider's free-tier window early: one provider",
|
|
35
|
+
"absorbed all work up to its ceiling and kept being refilled the moment a claim",
|
|
36
|
+
"completed, while the other slots sat idle -- until the fast provider hit its limit and",
|
|
37
|
+
"its 429s displaced every agent type at once. Every slot a turn per cycle puts a fast",
|
|
38
|
+
"slot with a large usage window to work early in the cycle while it is still fast, and",
|
|
39
|
+
"no provider's quota sits unused while another's is drained.",
|
|
40
|
+
"",
|
|
28
41
|
"Absent means weight 1. Must be a number in (0, 100]; anything else warns and uses 1.",
|
|
29
|
-
"List order
|
|
30
|
-
"
|
|
42
|
+
"List order sets the rotation for every slot, not just a tie-break: earlier positions",
|
|
43
|
+
"take their turns earlier in each cycle, so put the slots you want worked first at the",
|
|
44
|
+
"top of the list.",
|
|
31
45
|
"",
|
|
32
46
|
"COST IS THE DEFAULT, NOT A CONSTRAINT. These four slots are free because that is what",
|
|
33
47
|
"the pool is FOR: mechanical work -- reading files, grepping, mechanical edits, cartography.",
|