loki-mode 11.1.0 → 11.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/dashboard/__init__.py +1 -1
- package/docs/CLI-REFERENCE.md +1 -1
- package/docs/INSTALLATION.md +1 -1
- package/docs/WANG-PRINCIPLES-PLAN.md +1 -1
- package/docs/v10/DECISIONS.md +2 -0
- package/docs/v10/GUIDE.md +85 -0
- package/docs/v11/ROUTER-1.md +15 -13
- package/loki-ts/dist/loki.js +659 -655
- package/mcp/__init__.py +1 -1
- package/mcp/cp_tools.py +333 -0
- package/mcp/server.py +21 -0
- package/mcp/tests/_read_only_wall_probe.py +10 -0
- package/mcp/tests/fixtures/cp_api/artifact_report.md +3 -0
- package/mcp/tests/fixtures/cp_api/cost.json +1 -0
- package/mcp/tests/fixtures/cp_api/repos.json +1 -0
- package/mcp/tests/fixtures/cp_api/run_alpha.json +1 -0
- package/mcp/tests/fixtures/cp_api/run_beta.json +1 -0
- package/mcp/tests/fixtures/cp_api/run_events.json +1 -0
- package/mcp/tests/fixtures/cp_api/runs_list.json +1 -0
- package/mcp/tests/fixtures/cp_api/stats.json +1 -0
- package/mcp/tests/test_cp_tools.py +290 -0
- package/mcp/tests/test_read_only.py +6 -3
- package/package.json +1 -1
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
- package/web-app/dist/assets/{AdminPage-Bqx993m_.js → AdminPage-Bs4Vyd9R.js} +1 -1
- package/web-app/dist/assets/{Avatar-BHjV5Csv.js → Avatar-Dmf_rtmv.js} +1 -1
- package/web-app/dist/assets/{Badge-DGbSCV8L.js → Badge-DCEdgEN8.js} +1 -1
- package/web-app/dist/assets/{Button-CnCZf-0w.js → Button-BpGXXEPk.js} +1 -1
- package/web-app/dist/assets/{CockpitPage-CyZC0Q4e.js → CockpitPage-CzssVmim.js} +1 -1
- package/web-app/dist/assets/{ComparePage-REwWeDu9.js → ComparePage-BU4XtB9F.js} +1 -1
- package/web-app/dist/assets/{ErrorBoundary-C0sG0fj4.js → ErrorBoundary-Bkjf4KP2.js} +1 -1
- package/web-app/dist/assets/{EvidenceReceiptPanel-nXcOrxiA.js → EvidenceReceiptPanel-CcGMfaq7.js} +1 -1
- package/web-app/dist/assets/{GitHubIssuesPanel-D8CUAA7b.js → GitHubIssuesPanel-D53leEFJ.js} +1 -1
- package/web-app/dist/assets/{GitHubPRsPanel-B68Bf309.js → GitHubPRsPanel-UVsPtV3-.js} +1 -1
- package/web-app/dist/assets/{HomePage-mgJEeXH_.js → HomePage-D9wO-XTX.js} +2 -2
- package/web-app/dist/assets/{LoginPage-TM_L3-ws.js → LoginPage-ChxBSele.js} +1 -1
- package/web-app/dist/assets/{MagicPage-CoE5Yz73.js → MagicPage-BqQDKX2L.js} +1 -1
- package/web-app/dist/assets/{MetricsPage-BubxRsx4.js → MetricsPage-CA22NkPl.js} +1 -1
- package/web-app/dist/assets/{NotFoundPage-udeE-1w6.js → NotFoundPage-oYt5qdHQ.js} +1 -1
- package/web-app/dist/assets/{ProjectPage-BuYFdDIy.js → ProjectPage-tSAdI6A6.js} +3 -3
- package/web-app/dist/assets/{ProjectsPage-DP4T7hAW.js → ProjectsPage-Dk22bicF.js} +1 -1
- package/web-app/dist/assets/{SettingsPage-fIgNrA9e.js → SettingsPage-CwDSvMn6.js} +1 -1
- package/web-app/dist/assets/{ShowcasePage-9lILj-je.js → ShowcasePage-By3IMy9k.js} +1 -1
- package/web-app/dist/assets/{SystemSettingsPage-BYBBdIyC.js → SystemSettingsPage-DcWNR28_.js} +1 -1
- package/web-app/dist/assets/{TeamsPage-qhHtMpzb.js → TeamsPage-CFawyYQg.js} +1 -1
- package/web-app/dist/assets/{TemplatesPage-DIClsS3d.js → TemplatesPage-D7pCD-63.js} +1 -1
- package/web-app/dist/assets/{TerminalOutput-DKczDGbC.js → TerminalOutput-CT2v1Vm2.js} +1 -1
- package/web-app/dist/assets/{activity-CU5PK4CB.js → activity-DW6jRbjj.js} +1 -1
- package/web-app/dist/assets/{bell-Czwm7c23.js → bell-CJOu9IIM.js} +1 -1
- package/web-app/dist/assets/{bot-BNB-olYj.js → bot-dVt0Xw_X.js} +1 -1
- package/web-app/dist/assets/{check-DEmFNYuC.js → check-BVC4A8YK.js} +1 -1
- package/web-app/dist/assets/{chevron-left-DcSPJ4KZ.js → chevron-left-Djgr5pg-.js} +1 -1
- package/web-app/dist/assets/{circle-alert-CQ0-gHsx.js → circle-alert-OE2reifn.js} +1 -1
- package/web-app/dist/assets/{clock-CvLXmhWQ.js → clock-BUUN5rWU.js} +1 -1
- package/web-app/dist/assets/{cloud-6BuQFMTx.js → cloud-CzPIksNz.js} +1 -1
- package/web-app/dist/assets/{code-xml-BMf3egg0.js → code-xml-DlE7vlGY.js} +1 -1
- package/web-app/dist/assets/{database-BBTbPQOn.js → database-DUauMcIC.js} +1 -1
- package/web-app/dist/assets/{dollar-sign-B8vHgGQG.js → dollar-sign-CBx9DWbM.js} +1 -1
- package/web-app/dist/assets/{file-code-corner-DwAQcHOd.js → file-code-corner-hTmaP9Mz.js} +1 -1
- package/web-app/dist/assets/{file-plus-BlhGOB-I.js → file-plus-DimFEqtO.js} +1 -1
- package/web-app/dist/assets/{globe-6a-K3mZa.js → globe-BV7yxcQ_.js} +1 -1
- package/web-app/dist/assets/{hammer-CU_u8tsP.js → hammer-CW6Ol_ju.js} +1 -1
- package/web-app/dist/assets/{index-BN6SdA1r.js → index-83w75NC1.js} +3 -3
- package/web-app/dist/assets/{layers-VJW1LqHm.js → layers-DMAqCHxh.js} +1 -1
- package/web-app/dist/assets/{loader-circle-DfL0cxR3.js → loader-circle-B-TJNPo5.js} +1 -1
- package/web-app/dist/assets/{lock-D2LRX42k.js → lock-DbvZW7dV.js} +1 -1
- package/web-app/dist/assets/{package-_JNG_ljQ.js → package-BWFcV2kS.js} +1 -1
- package/web-app/dist/assets/{plus-qYicH17Y.js → plus-CjPzASFv.js} +1 -1
- package/web-app/dist/assets/{refresh-cw-Bqb0Vbp-.js → refresh-cw-DJL2nFq0.js} +1 -1
- package/web-app/dist/assets/{rotate-ccw-Blheo-H1.js → rotate-ccw-Bq3vK-Tu.js} +1 -1
- package/web-app/dist/assets/{scroll-text-3OO4nVP7.js → scroll-text-CAMgtDYH.js} +1 -1
- package/web-app/dist/assets/{server-34C8ObzM.js → server-D71_Bywf.js} +1 -1
- package/web-app/dist/assets/{shield-alert-D7-nkuq3.js → shield-alert-DqQ5wzQ6.js} +1 -1
- package/web-app/dist/assets/{thumbs-up-CvfrKMma.js → thumbs-up-DXE_IT36.js} +1 -1
- package/web-app/dist/assets/{trash-2-4bt3fZC1.js → trash-2-VYAPKajq.js} +1 -1
- package/web-app/dist/assets/{trending-down-C7Uu-rJL.js → trending-down-CuE-J2LH.js} +1 -1
- package/web-app/dist/assets/{trending-up-B7TF-ynr.js → trending-up-BvdD9XmQ.js} +1 -1
- package/web-app/dist/assets/{upload-YlZG_g5B.js → upload-OM2YFziE.js} +1 -1
- package/web-app/dist/assets/{usePolling-BDZJZdnI.js → usePolling-Bk39fp_d.js} +1 -1
- package/web-app/dist/assets/{user-C2Z3KTt0.js → user-CWmKQURs.js} +1 -1
- package/web-app/dist/index.html +1 -1
package/README.md
CHANGED
|
@@ -17,7 +17,7 @@ CLI reference: [docs/CLI-REFERENCE.md](docs/CLI-REFERENCE.md) (generated from th
|
|
|
17
17
|
|
|
18
18
|
Loki Mode is a free, source-available autonomous coding agent by [Autonomi](https://www.autonomi.dev/). Hand it a PRD, GitHub issue, OpenAPI doc or one-line brief. It derives a delivery contract (the acceptance criteria), builds against it, and ends with a signed Evidence Receipt that states what was proven and what was not. If the contract cannot be derived, it stops and asks one question instead of guessing.
|
|
19
19
|
|
|
20
|
-
Before a build counts as done, a review council selects reviewers from a specialist pool (`agents/types.json`, scored by `run.sh:FOCUS_KEYWORDS`). The bundled MCP server exposes
|
|
20
|
+
Before a build counts as done, a review council selects reviewers from a specialist pool (`agents/types.json`, scored by `run.sh:FOCUS_KEYWORDS`). The bundled MCP server exposes 47 tools over stdio (`mcp/server.py`), including `loki_v10_run`, `loki_v10_status` and `loki_v10_verify` to start a Loki 10 run in the background, read its phase, verdict and cost, and verify its receipt (`BLOCKED` runs report their question). Add it with `claude mcp add loki-mode -- python3 -m mcp.server` from the install directory.
|
|
21
21
|
|
|
22
22
|
## Contents
|
|
23
23
|
|
package/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: loki-mode
|
|
|
3
3
|
description: An autonomous software factory that knows what it is supposed to deliver, and proves it did. Use when the user says Loki Mode or asks to build, fix or verify software autonomously.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Loki Mode v11.
|
|
6
|
+
# Loki Mode v11.2.0
|
|
7
7
|
|
|
8
8
|
**Current entry points (Loki 10 engine):** `loki "<task>"`, `loki owner/repo#N` and `loki quick "<task>"` run the Loki 10 engine and end with a signed receipt (`loki verify`). `loki backlog owner/repo --all` runs many issues in parallel. Guide: `docs/v10/GUIDE.md`. The skill instructions below, and `loki start`, describe the legacy engine, which still ships and is being removed (`loki start owner/repo#N` still routes to it; prefer `loki owner/repo#N`).
|
|
9
9
|
|
|
@@ -483,4 +483,4 @@ See `CHANGELOG.md` entries [7.5.7], [7.5.8], [7.5.13] for the per-fix list and r
|
|
|
483
483
|
|
|
484
484
|
---
|
|
485
485
|
|
|
486
|
-
**v11.
|
|
486
|
+
**v11.2.0 | [Autonomi](https://www.autonomi.dev/) flagship product | core skill**
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
11.
|
|
1
|
+
11.2.0
|
package/dashboard/__init__.py
CHANGED
package/docs/CLI-REFERENCE.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# CLI reference
|
|
2
2
|
|
|
3
|
-
Generated from `loki-ts/src/cli/registry.ts` by `scripts/generate-stale-zero.sh` for v11.
|
|
3
|
+
Generated from `loki-ts/src/cli/registry.ts` by `scripts/generate-stale-zero.sh` for v11.2.0. Do not edit by hand.
|
|
4
4
|
Hidden and legacy-only commands are omitted; see `docs/v10/CLI-MODERN.md` for the full inventory.
|
|
5
5
|
|
|
6
6
|
| Command | Description |
|
package/docs/INSTALLATION.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The flagship product of [Autonomi](https://www.autonomi.dev/). Loki Mode is a spec-driven autonomous builder with a built-in trust layer that takes any spec to a deployed product and verifies completion with evidence (quality gates plus a completion council), not just a "done" claim. Complete installation instructions for all platforms and use cases.
|
|
4
4
|
|
|
5
|
-
**Version:** v11.
|
|
5
|
+
**Version:** v11.2.0
|
|
6
6
|
|
|
7
7
|
**Engine note:** `loki "<task>"`, `loki owner/repo#N` and `loki quick` run the Loki 10 engine. `loki start` still routes to the legacy engine, which is being removed (planned work resumes 2026-10-07); prefer `loki owner/repo#N`. See [docs/v10/GUIDE.md](v10/GUIDE.md).
|
|
8
8
|
|
|
@@ -53,7 +53,7 @@ Measured on this machine, not asserted.
|
|
|
53
53
|
| **1 Systems Thinking** | STRONG | 8 quality gates, RARV-C loop, council, Evidence Receipt, dual-route parity enforced by test |
|
|
54
54
|
| **2 Speed** | MEASURED, UNOPTIMISED | agent call = **980s = 96%** of iteration; all gates together = 44s |
|
|
55
55
|
| **3 Reliability** | STRONG, newly so | 73 mutation-proven trust invariants; four gates were shipping broken until v8.38.0 |
|
|
56
|
-
| **4 Extensibility** | STRONG | 5 providers, agent type registry, MCP (
|
|
56
|
+
| **4 Extensibility** | STRONG | 5 providers, agent type registry, MCP (47 tools), plugin marketplace |
|
|
57
57
|
| **5 Feedback Loops / Evals** | **BROKEN** | `cost_usd == 0` on **3 of 5** efficiency records; no eval score for our own harness |
|
|
58
58
|
|
|
59
59
|
**Principle 5 is the gap, and it is the one Wang weights highest.** He states the
|
package/docs/v10/DECISIONS.md
CHANGED
|
@@ -729,3 +729,5 @@ PROGRESS.md (the FC-25 P9 entry) cites a D88 CI waiver that was never recorded i
|
|
|
729
729
|
8. Implementation: docs/v11/ROUTER-1.md (slices R1-01 to R1-21). This slice is R1-04.
|
|
730
730
|
9. Default-on only after parity: while slices are built, `LOKI_ROUTER` unset means off. R1-19 flips the default to on only when, on the release candidate SHA, the FireLater#17 gate is green (VERIFIED, ready PR, time within raw + 30s, cost <= raw x 1.2) and the B9 corpus shows correctness parity with raw `claude -p` on every shape not listed in the seeded router-shape-defaults.json. After the flip, `LOKI_ROUTER=0` is the opt-out and restores the exact pre-router model behavior, pins included (R1-21 golden).
|
|
731
731
|
10. Accepted risk (CTO, 2026-10-07): B9 measures about 5 of the 14 or more Repo Shape Matrix shapes, so an unmeasured shape starts on Haiku without shape-level parity evidence; the backstops are the in-run escalation ladder (Haiku -> Sonnet 5.5 before STALLED, so a miss costs time and money, not correctness) and the H4 history floor, the receipt records shape parity as NOT PROVEN for any shape with no B9 row, and each matrix shape gains a B9 row as it becomes runnable.
|
|
732
|
+
|
|
733
|
+
Amendment to D89 (2026-10-07 21:40Z, CTO decision, refined by the founder at 21:45Z, CoS records). This amends only R1-19, the default-on flip that follows the CTO gate; 11.1.0 and 11.2.0 still ship the router OFF. (1) The default executor is Sonnet 5.5; Haiku 5.5 per work unit when Opus assigns it. (2) Roles: Opus 5.5 is the architect, router and advisor, plans and assigns each unit, and is consulted on hard calls. Sonnet is the senior engineer and the default for implementation and fixes. Haiku does tests, docs and small low-risk changes only when Opus assigns them in the plan. Haiku is never the blanket default. (3) L0 holds: the harness never routes by task type or path regex; it only executes, verifies and escalates. (4) The route schema is `units: [{id, kind, executor, reason}]` plus a run default of sonnet; a missing or invalid unit route goes to sonnet with NOT PROVEN. A haiku unit that fails a code-owned check is redone on sonnet, then escalates to opus; a Wall assigned to haiku gets an Opus review before implement. The receipt lists the executor, assigner and reason per unit. The per-unit schema lands in 11.2.0 or 11.2.1 depending on what is merged. (5) Escalation stays evidence-gated (haiku -> sonnet -> opus, and sonnet -> opus), and `LOKI_ROUTER=0` equals pre-router behavior exactly (the R1-21 golden). (6) The R1-16 shape-defaults semantics are inverted (a shape is listed as `haiku` once it has earned it; an absent listing means Sonnet), and R1-16 history is an input to Opus, not a harness rule. Rationale: a wrong PR costs the user far more than the token savings, B9 covers only about 5 of 14 shapes (item 10), and under L0 a model earns the cheaper tier with evidence. Items 3, 5, 7 and 9 above are read with this amendment; their text is kept as recorded. See docs/v11/ROUTER-1.md sections 4.1, 4.3 and 4.6.
|
package/docs/v10/GUIDE.md
CHANGED
|
@@ -233,6 +233,91 @@ NOT PROVEN line adds the literal entry `kill blocking not enforced`
|
|
|
233
233
|
(`loki-ts/src/engine10/stages/seal.ts`). opencode is not one of engine10's
|
|
234
234
|
provider names (`claude`, `codex`, `cline`, `aider`); it is not supported.
|
|
235
235
|
|
|
236
|
+
## Model routing
|
|
237
|
+
|
|
238
|
+
Model routing picks which Claude model does each stage of a run. It is OFF
|
|
239
|
+
by default in 11.2.0. Set `LOKI_ROUTER=1` to opt in. `LOKI_ROUTER=0` is the
|
|
240
|
+
explicit opt-out and keeps the exact pre-router model behavior, pins
|
|
241
|
+
included (guarded by the opt-out golden test in
|
|
242
|
+
`loki-ts/tests/engine10/router_optout_golden.test.ts`). Explicit
|
|
243
|
+
`LOKI_MODEL_OVERRIDE` and `LOKI_CLAUDE_MODEL_DEVELOPMENT` still win and
|
|
244
|
+
bypass the router. The other knobs are `LOKI_ROUTER_EXECUTOR` (post-flip
|
|
245
|
+
default sonnet; haiku only by earned evidence), `LOKI_ROUTER_ESCALATE` (default sonnet) and `LOKI_ROUTER_ADVISOR`
|
|
246
|
+
(default opus; `off` disables the advisor). The router engine itself lands in
|
|
247
|
+
11.2.0; the pieces below marked "shipped in 11.1.0" are already in the code.
|
|
248
|
+
|
|
249
|
+
### Executor and advisor
|
|
250
|
+
|
|
251
|
+
When the router is on, the post-flip default executor is Sonnet 5.5;
|
|
252
|
+
Haiku 5.5 per work unit when Opus assigns it. Opus 5.5 is the architect,
|
|
253
|
+
router and advisor: it plans, assigns each work unit and is consulted on
|
|
254
|
+
hard calls. Sonnet is the senior engineer and the default for
|
|
255
|
+
implementation and fixes. Haiku does tests, docs and small low-risk changes
|
|
256
|
+
only when Opus assigns them in the plan, and is never the blanket default.
|
|
257
|
+
The harness never routes by task type or path regex; it only executes,
|
|
258
|
+
verifies and escalates. Per-repo history is an input to Opus, not a harness
|
|
259
|
+
rule. The per-unit route (`units: [{id, kind, executor, reason}]` plus a run
|
|
260
|
+
default of sonnet) lands in 11.2.0 or 11.2.1 depending on what is merged. A
|
|
261
|
+
missing or invalid unit route goes to Sonnet and is reported as NOT PROVEN.
|
|
262
|
+
The receipt lists the executor, assigner and reason per unit.
|
|
263
|
+
The advisor needs Claude Code 2.1.293 or later,
|
|
264
|
+
which the Agent SDK 0.3.293 bundles (shipped in 11.1.0). If the advisor is
|
|
265
|
+
unavailable, the executor is Sonnet 5.5 and the run is never failed for it.
|
|
266
|
+
The advisor is unavailable when the provider is not
|
|
267
|
+
claude, when `CLAUDE_CODE_USE_BEDROCK`, `CLAUDE_CODE_USE_VERTEX` or
|
|
268
|
+
`CLAUDE_CODE_USE_FOUNDRY` is set, when `ANTHROPIC_BASE_URL` points at a
|
|
269
|
+
non-Anthropic host, when Claude Code is older than 2.1.293, when
|
|
270
|
+
`LOKI_ROUTER_ADVISOR=off`, or after a session reports an advisor tool error
|
|
271
|
+
(remembered for the rest of the run). The probe is
|
|
272
|
+
`loki-ts/src/runner/router/advisor_probe.ts` (shipped in 11.1.0). The
|
|
273
|
+
default-on flip happens after the CTO gate; 11.1.0 and 11.2.0 still ship the
|
|
274
|
+
router off.
|
|
275
|
+
|
|
276
|
+
### Escalation
|
|
277
|
+
|
|
278
|
+
Escalation is gated on evidence and only moves up: haiku to sonnet to opus,
|
|
279
|
+
and sonnet to opus. A haiku unit that fails a code-owned check is redone on
|
|
280
|
+
Sonnet, then escalates to Opus. A Wall assigned to Haiku gets an Opus review
|
|
281
|
+
before implement. A repeated failure on Sonnet gets a fix round on Opus
|
|
282
|
+
before the run can stall. A failure owned by the harness, the environment or
|
|
283
|
+
the provider causes no transition and is reported as NOT PROVEN.
|
|
284
|
+
|
|
285
|
+
### Per-repo-shape outcome memory
|
|
286
|
+
|
|
287
|
+
Loki remembers how each kind of repo went. The shape key is the Project
|
|
288
|
+
Model `workspaceKind` plus the sorted runners, for example
|
|
289
|
+
`multi-root:pytest+vitest`; a package with no runner contributes `none`.
|
|
290
|
+
Outcomes are stored in the per-repo cache. The haiku floor moves a shape to
|
|
291
|
+
Sonnet after 2 code-owned losses in the last 3 Haiku runs. Only code-owned
|
|
292
|
+
FAILs and escalations count; harness, environment and provider errors and
|
|
293
|
+
NOT PROVEN outcomes never do. A missing or corrupt history file is a cold
|
|
294
|
+
read, never a crash (shipped in 11.1.0, `loki-ts/src/runner/router/history.ts`).
|
|
295
|
+
The shipped `loki-ts/data/router-shape-defaults.json` is empty in 11.1.0.
|
|
296
|
+
After the flip a shape is listed as `haiku` once it has earned it with a B9
|
|
297
|
+
row, and an absent listing means Sonnet. Shape history and the defaults file
|
|
298
|
+
are inputs to Opus, not harness routing rules.
|
|
299
|
+
|
|
300
|
+
### Pricing
|
|
301
|
+
|
|
302
|
+
Haiku 5.5 is priced on its exact model id: $0.10 in and $0.50 out per MTok
|
|
303
|
+
for prompts up to 100K tokens, and $0.50 in and $2.50 out over 100K, with
|
|
304
|
+
the cache rates in `loki-ts/data/model-pricing.json`. The `haiku` family
|
|
305
|
+
alias row in that file is still the Haiku 4.5 price of $1 and $5, because
|
|
306
|
+
the exact-id key is matched first (`loki-ts/src/runner/budget.ts`).
|
|
307
|
+
|
|
308
|
+
### What you see
|
|
309
|
+
|
|
310
|
+
In 11.2.0 the receipt carries a `route` block (initial decision, each
|
|
311
|
+
escalation with its trigger, advisor availability and the share of requests
|
|
312
|
+
above 100K), and `loki doctor` prints an advisor line. The executor and any
|
|
313
|
+
fallback reason are also printed on the start line.
|
|
314
|
+
|
|
315
|
+
### Comparing models
|
|
316
|
+
|
|
317
|
+
`scripts/b9-scoreboard.sh` is the comparison harness for routed against raw
|
|
318
|
+
runs on the B9 corpus, and it can emit shape defaults from those runs
|
|
319
|
+
(lands in 11.2.0).
|
|
320
|
+
|
|
236
321
|
## After the flip
|
|
237
322
|
|
|
238
323
|
D48 made v10 the default for `loki "<task>"`, `loki owner/repo#N` and
|
package/docs/v11/ROUTER-1.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
# ROUTER-1:
|
|
1
|
+
# ROUTER-1: Sonnet executor, Opus advisor, evidence-earned Haiku, evidence-gated escalation
|
|
2
2
|
|
|
3
|
-
Status: plan (Architect, 2026-10-07; default-on adjustment from steering the same day).
|
|
3
|
+
Status: plan (Architect, 2026-10-07; default-on adjustment from steering the same day). CTO decision 2026-10-07 21:40Z, refined by the founder at 21:45Z (D89 amendment), governs R1-19 only: when the default-on flip lands after the CTO gate, the default executor is Sonnet 5.5; Haiku 5.5 per work unit when Opus assigns it. Opus 5.5 is the architect, router and advisor: it plans, assigns each work unit and is consulted on hard calls. Sonnet is the senior engineer and the default for implementation and fixes. Haiku does tests, docs and small low-risk changes only when Opus assigns them in the plan. Haiku is never the blanket default. The per-unit schema (section 4.1) lands in 11.2.0 or 11.2.1 depending on what is merged; the earlier whole-run schema below is superseded. 11.1.0 and 11.2.0 still ship the router OFF. Earlier text below that calls Haiku the default executor describes the superseded plan. While slices are built they merge behind `LOKI_ROUTER` (unset = off). R1-19, the final slice, makes it default ON, gated on steering's FireLater#17 gate plus the B9 scoreboard on the release candidate. After the flip `LOKI_ROUTER=0` is the opt-out and restores the exact pre-router model behavior, pins included (R1-21 test).
|
|
4
4
|
Source: founder directive 2026-10-07 ("Opus is the advisor, Haiku does all engineering work; Sonnet is optional, routed by Opus when Haiku is not performing well enough; model decides").
|
|
5
5
|
Scope: the engine10 path (`loki start <issue|text>`, `loki-ts/src/engine10`). The legacy RARV route (`autonomy/run.sh`, `runner/autonomous.ts`, `runner/capability_router.ts`) keeps its tiers; only its pricing table mirror changes (R1-02).
|
|
6
6
|
|
|
7
7
|
## 1. Goal
|
|
8
|
-
|
|
8
|
+
After the R1-19 flip, every engine10 session runs on Claude Sonnet 5.5 as the default executor with Opus 5.5 attached through the native Claude Code advisor tool, so the model itself decides when it needs stronger judgment. Opus picks the executor at plan time with a recorded reason. Opus may route a task to Haiku 5.5 only when the R1-16 history and a B9 row show Haiku at outcome parity on that repo shape; a shape with no evidence stays on Sonnet. Escalation stays evidence-gated (haiku -> sonnet -> opus, and sonnet -> opus). The outcome (solve rate, time, cost) is never below raw `claude -p`, proven on the B9 scoreboard.
|
|
9
9
|
|
|
10
10
|
## 2. What already exists (read before building; cite, do not rebuild)
|
|
11
11
|
- Session model plumbing: `engine10/session.ts:25` resolveModel, `:31-60` childEnv (tier pins via `LOKI_CLAUDE_MODEL_<TIER>`, `LOKI_E10_MODEL_DEFAULT` omits `--model`).
|
|
@@ -32,15 +32,17 @@ Every engine10 session runs on Claude Haiku 5.5 as the executor with Opus attach
|
|
|
32
32
|
## 4. Architecture
|
|
33
33
|
### 4.1 Routing decision schema (written into plan-scope.json by the plan session; parsed by `runner/router/decision.ts`)
|
|
34
34
|
```
|
|
35
|
-
route: {
|
|
36
|
-
|
|
35
|
+
route: { default: "sonnet",
|
|
36
|
+
units: [ { id: string, kind: string, executor: "haiku" | "sonnet", reason: string (<= 200 chars) } ],
|
|
37
|
+
source: "opus-plan" | "advisor" | "default" }
|
|
37
38
|
```
|
|
38
|
-
|
|
39
|
+
Per-unit schema (founder refinement 2026-10-07 21:45Z; lands in 11.2.0 or 11.2.1 depending on what is merged). Opus writes the units into plan-scope.json at plan time, with a run default of sonnet. The harness never routes by task type or path regex (L0): it only executes, verifies and escalates. R1-16 history is an input to the Opus planning prompt, not a harness rule. The receipt lists the executor, the assigner and the reason per unit.
|
|
40
|
+
A missing or invalid unit route goes to sonnet (`{executor: "sonnet", source: "default", reason: "no route returned"}`; the superseded plan returned haiku here) plus a NOT PROVEN line (L2: fail open on work). Aliases resolve through `providers/model_catalog.json` (haiku -> claude-haiku-5-5, sonnet -> claude-sonnet-5-5, opus -> claude-opus-5-5).
|
|
39
41
|
|
|
40
42
|
### 4.2 Where the decision is made
|
|
41
|
-
1. Run start: advisor availability is probed (R1-06). Start line: `executor
|
|
43
|
+
1. Run start: advisor availability is probed (R1-06). Start line: `executor sonnet-5.5, advisor opus` (or `executor haiku-5.5, advisor opus` for a shape with earned Haiku evidence; or `executor sonnet-5.5, advisor unavailable: <reason>`).
|
|
42
44
|
2. Plan stage runs (normal tasks): the plan session runs on the executor with the Opus advisor attached; its brief asks it to consult the advisor and write `route` into plan-scope.json. With no advisor, the plan session itself is pinned to Opus (the single cheap Opus call) and writes the same field; a `haiku` route is then raised to Sonnet (section 4.4).
|
|
43
|
-
3. Plan skipped (small task, `plan.ts:55`): no extra call. Executor = history floor or shape default if one applies, else
|
|
45
|
+
3. Plan skipped (small task, `plan.ts:55`): no extra call. Executor = history floor or shape default if one applies, else Sonnet with the advisor (post-flip default; Haiku only for a shape listed `haiku` in section 4.6), `source: default`.
|
|
44
46
|
4. Recorded: a `route` event (implement), run.started `route_initial`, receipt `route` block (initial decision, every escalation with trigger and evidence pointer, advisor availability, share of requests above 100K).
|
|
45
47
|
|
|
46
48
|
### 4.3 Escalation state machine (pure function in `runner/router/decision.ts`)
|
|
@@ -51,7 +53,7 @@ EXEC(haiku) --c: SPEC_CONFLICT exit | stall | implement limit kill------> EXEC(s
|
|
|
51
53
|
EXEC(sonnet) --a or stall------------------------------------------------> fix round on opus (existing L1 path) -> STALLED
|
|
52
54
|
any state --harness/env/provider-owned ERROR-----------------------------> no transition (NOT PROVEN path)
|
|
53
55
|
```
|
|
54
|
-
At most one Haiku -> Sonnet transition per run. A SPEC_CONFLICT confirmed by Sonnet stays BLOCKED. Resume across the swap: the fix session resumes the prior session id with the new model when the SDK honors `model` on `resume` (verified in R1-12); otherwise it starts fresh with the full brief plus the prior diagnosis and records `fix_resume_why`.
|
|
56
|
+
Per-unit rule (founder refinement 2026-10-07 21:45Z): a haiku unit that fails a code-owned check is redone on sonnet, then escalates to opus; a Wall assigned to haiku gets an Opus review before implement. The state machine above is the whole-run form and is read per unit. At most one Haiku -> Sonnet transition per run (whole-run form). A SPEC_CONFLICT confirmed by Sonnet stays BLOCKED. Resume across the swap: the fix session resumes the prior session id with the new model when the SDK honors `model` on `resume` (verified in R1-12); otherwise it starts fresh with the full brief plus the prior diagnosis and records `fix_resume_why`.
|
|
55
57
|
|
|
56
58
|
### 4.4 Advisor wiring and fallback detection
|
|
57
59
|
- Wiring: SDK route passes `settings: { advisorModel }` (in SDK 0.3.288 `advisorModel` is a field of `interface Settings`, `sdk.d.ts:6674,8987`, not a top-level Option; R1-07 confirms the 0.3.293 shape). CLI route merges `advisorModel` into the existing `--settings` JSON or uses `--advisor` when `claudeFlagSupported` says so.
|
|
@@ -62,13 +64,13 @@ At most one Haiku -> Sonnet transition per run. A SPEC_CONFLICT confirmed by Son
|
|
|
62
64
|
Lean briefs and the cache-stable prefix already exist (`LOKI_E10_PREFIX=lean`, `providers.ts:524`). R1-07 adds `autoCompactWindow: 100000` (the SDK minimum, `sdk.d.ts` Settings) for routed sessions. R1-08 records per request `input+cache_read+cache_creation` and writes `requests_total`, `requests_over_100k` and advisor tokens into each cost event; the receipt shows the share.
|
|
63
65
|
|
|
64
66
|
### 4.6 Per-shape defaults (a shape that fails parity routes to Sonnet instead of holding the release)
|
|
65
|
-
- Where it lives: `loki-ts/data/router-shape-defaults.json`, shipped in the package next to model-pricing.json. Schema `{"$schema_version":1, "_source":"<B9 run ids + RC version + date>", "shapes":{"<shape_key>":{"executor":"
|
|
67
|
+
- Where it lives: `loki-ts/data/router-shape-defaults.json`, shipped in the package next to model-pricing.json. Schema `{"$schema_version":1, "_source":"<B9 run ids + RC version + date>", "shapes":{"<shape_key>":{"executor":"haiku"|"prior-default","evidence":"<METRICS.md row ref>"}}}`. Semantics inverted by the 2026-10-07 21:40Z CTO decision and refined by the founder at 21:45Z (D89 amendment): the default executor is Sonnet 5.5 and Haiku 5.5 is used per work unit when Opus assigns it. A shape is listed as `haiku` once it has earned it with a B9 row showing Haiku at outcome parity on that shape, and an absent listing means Sonnet 5.5. The shipped file and the R1-16 history are inputs to the Opus planning prompt, not harness routing rules; the harness never routes by task type or path regex (L0). `prior-default` (the `LOKI_ROUTER=0` model for that stage) stays legal for a shape where Sonnet also loses. The shipped file is empty (R1-16), so every shape starts on Sonnet.
|
|
66
68
|
- Shape key: Project Model `workspaceKind` plus the sorted, lowercased, de-duplicated package `runner` labels from `.loki/project.json` (both model-derived, L4; the schema has no language field, `project_model/schema.ts:20-30`, decided 2026-10-07 by the CoS after R1-16 found the gap), for example `multi-root:pytest+vitest`. A package with no runner contributes `none`. The same function (`shapeKey()` in history.ts) computes it at runtime and in the receipt (R1-15 writes `route.shape_key`), so B9 rows and runtime agree by construction. No file-pattern regex (L0).
|
|
67
69
|
- Seeding: `scripts/b9-scoreboard.sh --emit-shape-defaults <out.json>` (R1-17) reads each run's receipt `route.shape_key`, compares arm 2 (router) with arm 1 (raw) per shape on correctness, and lists every shape where router solve count < raw. A shape where Sonnet also loses on the confirming rerun is emitted as `prior-default`. R1-19 commits that output as the shipped file, citing the METRICS.md rows. Later B9 runs reseed it in a normal slice; removing a shape, or lowering it from `prior-default` to `sonnet`, needs a B9 row showing the lower executor at parity on it.
|
|
68
|
-
- Read path: `decision.ts` resolves the executor as the strongest of: Opus route, shipped shape default, per-repo history floor (R1-16, local runs; Haiku
|
|
70
|
+
- Read path: `decision.ts` resolves the executor as the strongest of: Opus route, shipped shape default, per-repo history floor (R1-16, local runs; under the inverted semantics Haiku is reachable only when the shape is listed `haiku` and the history does not show Haiku losing 2 of the last 3 by code-owned FAIL or escalation -> Sonnet, cleared only by a B9 row showing Haiku at parity on that shape), the no-advisor floor (section 4.4), then Sonnet unless the shape is listed `haiku`. `prior-default` resolves to the exact `LOKI_ROUTER=0` model for that stage (R1-21 fixture semantics). Explicit `LOKI_MODEL_OVERRIDE`, `LOKI_CLAUDE_MODEL_DEVELOPMENT` or `LOKI_ROUTER_EXECUTOR` still win. Evidence rungs never lower a model: a shape default can only move the executor up. The shape key is a lookup into executed outcome evidence and may only move the executor up (ENGINE-LAWS H4); this is the documented exception to the L0 review rule. The receipt records `source: shape-default` with the evidence ref. Missing or corrupt file = empty map plus a NOT PROVEN line, never a crash.
|
|
69
71
|
|
|
70
72
|
## 5. Knobs
|
|
71
|
-
`LOKI_ROUTER` (unset = off while building; unset = on after R1-19; `0` = opt-out to pre-router behavior), `LOKI_ROUTER_EXECUTOR` (default haiku), `LOKI_ROUTER_ESCALATE` (default sonnet), `LOKI_ROUTER_ADVISOR` (default opus; `off` disables). `LOKI_MODEL_OVERRIDE` and explicit `LOKI_CLAUDE_MODEL_DEVELOPMENT` still win and bypass the router (recorded `source: override`).
|
|
73
|
+
`LOKI_ROUTER` (unset = off while building; unset = on after R1-19; `0` = opt-out to pre-router behavior), `LOKI_ROUTER_EXECUTOR` (post-flip default sonnet; haiku only by earned evidence), `LOKI_ROUTER_ESCALATE` (default sonnet), `LOKI_ROUTER_ADVISOR` (default opus; `off` disables). `LOKI_MODEL_OVERRIDE` and explicit `LOKI_CLAUDE_MODEL_DEVELOPMENT` still win and bypass the router (recorded `source: override`).
|
|
72
74
|
|
|
73
75
|
## 6. Slices
|
|
74
76
|
Tier sets the review (D12): LOW/MEDIUM by Tech Leads, HIGH by opus reviewers, unanimous. Every TS slice rebuilds dist in the main checkout before its train (memory: symlinked node_modules), runs `bash scripts/local-ci.sh`, and keeps `LOKI_ROUTER` unset behavior byte-identical (existing suites green). File sets do not overlap; `runner/router/flag.ts` (created by R1-05) and `loki-ts/data/router-shape-defaults.json` (created empty by R1-16) are edited only by R1-19, after every other slice has merged, so no two slices touch them concurrently.
|
|
@@ -93,7 +95,7 @@ Tier sets the review (D12): LOW/MEDIUM by Tech Leads, HIGH by opus reviewers, un
|
|
|
93
95
|
| R1-16 | Per-shape outcome history (H4) | loki-ts/src/runner/router/history.ts, loki-ts/src/e10ext/repomemory.ts, loki-ts/tests/runner/router_history.test.ts | MEDIUM | haiku | append/read in the per-repo cache dir; shape key from Project Model workspaceKind + sorted runner labels (model-derived, L4), never from file regex; floor: haiku lost 2 of last 3 on the shape (code-owned FAIL or escalation; harness, env and provider ERRORs do not count) -> sonnet, cleared only by a B9 row showing Haiku at parity on that shape; fixture: 3 harness-owned FAILs on haiku do NOT trigger the floor. Also creates the empty shipped `loki-ts/data/router-shape-defaults.json` and its reader `shapeDefault(key)` (section 4.6), which accepts `sonnet` and `prior-default` and rejects any other value (cold read + NOT PROVEN) | corrupt file is a cold read, never a crash | none |
|
|
94
96
|
| R1-17 | B9 scoreboard harness | scripts/b9-scoreboard.sh, tests/test-b9-scoreboard.sh, tests/run-all-tests.sh (registration + shard row) | MEDIUM | sonnet | dry-run with a stub `claude` produces one METRICS row per arm with solve, wall seconds, usd, shape_key; `--emit-shape-defaults` on a fixture where the router loses one shape lists exactly that shape as `sonnet`; a fixture where the Sonnet confirming rerun also loses emits it as `prior-default` | shellcheck, `timeout -k`, runner registration guard (memory: new test file structural guards) | none |
|
|
95
97
|
| R1-18 | Docs: GUIDE model-routing section, CHANGELOG | docs/v10/GUIDE.md, CHANGELOG.md | LOW | haiku | doc lint; GUIDE names every knob in section 5 and the fallback | Release Manager reads it against the receipt | R1-15 |
|
|
96
|
-
| R1-19 | Final flip: LOKI_ROUTER default ON
|
|
98
|
+
| R1-19 | Final flip after the CTO gate: LOKI_ROUTER default ON with Sonnet 5.5 + Opus 5.5 advisor as the default executor (11.1.0 and 11.2.0 still ship OFF); Haiku 5.5 per work unit only when Opus assigns it in the plan, per-unit schema `units: [{id, kind, executor, reason}]` (11.2.0 or 11.2.1); ship seeded shape defaults (`haiku` only for earned shapes) | loki-ts/src/runner/router/flag.ts, loki-ts/data/router-shape-defaults.json, loki-ts/tests/runner/router_flag_default.test.ts | HIGH | sonnet | red: unset env resolves off; green: unset resolves on, `0`/`off`/`false` resolve off; R1-21 opt-out golden still green with `LOKI_ROUTER=0` | GATE (all on the release candidate SHA): steering's FireLater#17 gate green (VERIFIED, ready PR, time within raw + 30s, cost <= raw x 1.2); B9 correctness parity with raw `claude -p` on the Sonnet default and on every shape listed `haiku` (a shape with no evidence stays on Sonnet); arm 4 correctness parity, or the Sonnet no-advisor fallback in place (R1-07, R1-09); every shape listed as `prior-default` runs the exact LOKI_ROUTER=0 models (R1-21 golden per listed shape); the seeded file committed with its METRICS.md refs | all |
|
|
97
99
|
| R1-21 | Opt-out golden: `LOKI_ROUTER=0` equals pre-router behavior exactly | loki-ts/tests/engine10/router_optout_golden.test.ts, loki-ts/tests/engine10/fixtures/pre_router_models.json | HIGH | sonnet | fixture captured on 11.0.3 code BEFORE any router slice merges: for an env matrix (default; LOKI_E10_CASCADE=1; LOKI_E10_WALL_TIER=haiku; LOKI_ALLOW_HAIKU=true; LOKI_MODEL_OVERRIDE=sonnet; LOKI_CLAUDE_MODEL_DEVELOPMENT=opus; LOKI_E10_PLAN=always) it records, per stage (plan, wall, already_done, implement, fix round 1 and escalated round 2), the model passed to sessions.run, the child-env LOKI_CLAUDE_MODEL_* values, the SDK query options model/fallbackModel/settings, the downgrade list, the start line and the run.started keys. The test replays the matrix through fake sessions with `LOKI_ROUTER=0` (and unset, until R1-19) and requires deep equality. Mutation: changing wallModel's default or adding `settings.advisorModel` under `0` turns it red | runs in the fast tier; must be green on every slice's train | none (lands first in W1) |
|
|
98
100
|
| R1-20 | `loki doctor` shows advisor availability and bundled CC version | loki-ts/src/commands/doctor.ts, loki-ts/tests/commands/doctor_router.test.ts | LOW | haiku | doctor prints `advisor: available` or the reason from R1-06 | doctor suites | R1-06 |
|
|
99
101
|
|