loki-mode 11.74.2 → 11.75.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/SKILL.md +2 -3
- package/VERSION +1 -1
- package/autonomy/completion-council.sh +10 -10
- package/autonomy/council-v2.sh +4 -4
- package/autonomy/grill.sh +1 -1
- package/autonomy/lib/ablation.py +22 -2
- package/autonomy/lib/browser-open.sh +1 -1
- package/autonomy/lib/chat.sh +1 -1
- package/autonomy/lib/cockpit-render.sh +1 -1
- package/autonomy/lib/done-recognition.sh +4 -4
- package/autonomy/lib/efficiency_cost.py +1 -1
- package/autonomy/lib/engine10-push.sh +2 -2
- package/autonomy/lib/prd-enrich.sh +1 -1
- package/autonomy/lib/proof-check.sh +1 -1
- package/autonomy/lib/proof-generator.py +3 -3
- package/autonomy/lib/proof-pr.sh +1 -1
- package/autonomy/lib/skill-link-heal.sh +1 -1
- package/autonomy/lib/voter-agents.sh +1 -1
- package/autonomy/loki +26 -37
- package/autonomy/prd-checklist.sh +1 -1
- package/autonomy/receipt_jwt.py +1 -1
- package/autonomy/run.sh +28 -28
- package/autonomy/trigger-schedule.py +6 -414
- package/autonomy/tui.sh +1 -1
- package/autonomy/verify.sh +4 -4
- package/completions/loki.bash +0 -5
- package/dashboard/__init__.py +1 -1
- package/docs/ARCHITECTURE.md +1 -2
- package/docs/CLI-REFERENCE.md +1 -1
- package/docs/COMPONENTS.md +1 -1
- package/docs/INSTALLATION.md +1 -1
- package/docs/ROUTING-EVIDENCE.md +1 -2
- package/docs/V10-RESEARCH.md +1 -1
- package/docs/dev/architecture-reference.md +3 -7
- package/docs/dev/project-structure.md +1 -4
- package/docs/dev/release-checklist.md +13 -31
- package/docs/research-2026-07/OBSERVABILITY-TOOL-SPEC.md +1 -1
- package/docs/v11/MARKET-SLICES.md +1 -1
- package/docs/v11/REGISTRIES.md +1 -1
- package/docs/v11/ROUTER-1.md +2 -2
- package/loki-ts/dist/loki.js +2 -2
- package/mcp/__init__.py +1 -1
- package/package.json +2 -2
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
- package/skills/extending.md +2 -3
- package/skills/factory-operations.md +11 -18
- package/skills/release-cadence.md +2 -2
- package/web-app/dist/assets/{AdminPage-jkjilfbl.js → AdminPage-CTwi8SUX.js} +1 -1
- package/web-app/dist/assets/{Avatar-bd5JWkSJ.js → Avatar-DVtGtLkJ.js} +1 -1
- package/web-app/dist/assets/{Badge-DVRit2Xw.js → Badge-rwKzz0Pe.js} +1 -1
- package/web-app/dist/assets/{Button-BNAB-pVZ.js → Button-nIsPDfsQ.js} +1 -1
- package/web-app/dist/assets/{CockpitPage-DpG29h_w.js → CockpitPage-BCeDpoWa.js} +1 -1
- package/web-app/dist/assets/{ComparePage-Dasg5V41.js → ComparePage-CYIYanVq.js} +1 -1
- package/web-app/dist/assets/{ErrorBoundary-DJojse6b.js → ErrorBoundary-Bkz6HqWp.js} +1 -1
- package/web-app/dist/assets/{EvidenceReceiptPanel-Blxw7_u1.js → EvidenceReceiptPanel-DZT2Np4F.js} +1 -1
- package/web-app/dist/assets/{GitHubIssuesPanel-BBvSFj80.js → GitHubIssuesPanel-Bzla91Lu.js} +1 -1
- package/web-app/dist/assets/{GitHubPRsPanel-BCIKeSBg.js → GitHubPRsPanel-FjT-Xn5t.js} +1 -1
- package/web-app/dist/assets/{HomePage-C7d__oNG.js → HomePage-pXgmRGLt.js} +2 -2
- package/web-app/dist/assets/{LoginPage-BXljL3yY.js → LoginPage-CnmpE28B.js} +1 -1
- package/web-app/dist/assets/{MagicPage-Xjkts_yE.js → MagicPage-CuSeGsLq.js} +1 -1
- package/web-app/dist/assets/{MetricsPage-kQvVp-j2.js → MetricsPage-Bg3PkBOU.js} +1 -1
- package/web-app/dist/assets/{NotFoundPage-D-4ZlEaO.js → NotFoundPage-uzBtZNV-.js} +1 -1
- package/web-app/dist/assets/{ProjectPage-DfzR5RXP.js → ProjectPage-Dacki8Xr.js} +3 -3
- package/web-app/dist/assets/{ProjectsPage-DFDsWzkV.js → ProjectsPage-BRGjksXG.js} +1 -1
- package/web-app/dist/assets/{SettingsPage-CLPIoYT3.js → SettingsPage-BbNf3fEJ.js} +1 -1
- package/web-app/dist/assets/{ShowcasePage-PZKkyGeY.js → ShowcasePage-XX4830sX.js} +1 -1
- package/web-app/dist/assets/{SystemSettingsPage-BsBlcnzo.js → SystemSettingsPage-DOhQeXa6.js} +1 -1
- package/web-app/dist/assets/{TeamsPage-BSjgNhoU.js → TeamsPage-2reIRUp_.js} +1 -1
- package/web-app/dist/assets/{TemplatesPage-W2KeAzbg.js → TemplatesPage-BUsLEWHW.js} +1 -1
- package/web-app/dist/assets/{TerminalOutput-BjroTiCC.js → TerminalOutput-BrBDcgzC.js} +1 -1
- package/web-app/dist/assets/{activity-BXZ5YYes.js → activity-B6AZpgmb.js} +1 -1
- package/web-app/dist/assets/{bell-D1urp5pa.js → bell-CB792A7S.js} +1 -1
- package/web-app/dist/assets/{bot-Chedf2bF.js → bot-uQZwOFAn.js} +1 -1
- package/web-app/dist/assets/{check-DhaCuOCV.js → check-DrtG0lIZ.js} +1 -1
- package/web-app/dist/assets/{chevron-left-D7BpQyA6.js → chevron-left-CrsJSAqQ.js} +1 -1
- package/web-app/dist/assets/{circle-alert-DnGvQ_AQ.js → circle-alert-BUCly4p3.js} +1 -1
- package/web-app/dist/assets/{clock-CqfxWhHM.js → clock-B2IT9SnE.js} +1 -1
- package/web-app/dist/assets/{cloud-BKxp5AWB.js → cloud-CN6Ux7Q7.js} +1 -1
- package/web-app/dist/assets/{code-xml-BDQ6GBtt.js → code-xml-BEm8PWnK.js} +1 -1
- package/web-app/dist/assets/{database-BmpEx1y_.js → database-cHe5Fgu8.js} +1 -1
- package/web-app/dist/assets/{dollar-sign-DYNY9R_C.js → dollar-sign-BVmP4YrE.js} +1 -1
- package/web-app/dist/assets/{file-code-corner-BJi0IBFd.js → file-code-corner-D-m-NrwO.js} +1 -1
- package/web-app/dist/assets/{file-plus-Dqyb2bcQ.js → file-plus-6mCjIzAB.js} +1 -1
- package/web-app/dist/assets/{globe-DPh4RWfM.js → globe-vcWOujwU.js} +1 -1
- package/web-app/dist/assets/{hammer-B3xnuoA1.js → hammer-D4nZFpO2.js} +1 -1
- package/web-app/dist/assets/{index-BpX8DcTr.js → index-u8toJ43r.js} +3 -3
- package/web-app/dist/assets/{layers-BRCmyCW8.js → layers-CNhYLop8.js} +1 -1
- package/web-app/dist/assets/{loader-circle-n07Pky0M.js → loader-circle-Vl7XIMV6.js} +1 -1
- package/web-app/dist/assets/{lock-BWqOuQmS.js → lock-SW_KwAvf.js} +1 -1
- package/web-app/dist/assets/{package-BJlB_Z-Z.js → package-QPK-7Ej1.js} +1 -1
- package/web-app/dist/assets/{plus-oH17BZu2.js → plus-DUgAlgCB.js} +1 -1
- package/web-app/dist/assets/{refresh-cw-CPQ476LL.js → refresh-cw-LfUksPHq.js} +1 -1
- package/web-app/dist/assets/{rotate-ccw-7hLvOcnS.js → rotate-ccw-CMbuouVq.js} +1 -1
- package/web-app/dist/assets/{scroll-text-BCmLauQn.js → scroll-text-C_FxBmNQ.js} +1 -1
- package/web-app/dist/assets/{server-DfTfvW84.js → server-7UJe9Jqt.js} +1 -1
- package/web-app/dist/assets/{shield-alert-CaeGRq19.js → shield-alert-BMuu6fYB.js} +1 -1
- package/web-app/dist/assets/{thumbs-up-Dz-l2GPC.js → thumbs-up-Bhs0d2Gp.js} +1 -1
- package/web-app/dist/assets/{trash-2-B-OsDmSo.js → trash-2-BZKxv49w.js} +1 -1
- package/web-app/dist/assets/{trending-down-CXTFOxzq.js → trending-down-DKeCMZek.js} +1 -1
- package/web-app/dist/assets/{trending-up-FQEdqr3v.js → trending-up-43_9fShg.js} +1 -1
- package/web-app/dist/assets/{upload-CacWSNTg.js → upload-pV3irPEW.js} +1 -1
- package/web-app/dist/assets/{usePolling-cIjZ9qd2.js → usePolling-CoZQM-WL.js} +1 -1
- package/web-app/dist/assets/{user-DAouWFHi.js → user-CMl7dXvC.js} +1 -1
- package/web-app/dist/index.html +1 -1
- package/autonomy/company-seats.json +0 -82
- package/docs/COMPETITIVE-INTEL-2026-09.md +0 -444
- package/docs/FOUNDER-DECISIONS-TRUST-SEMANTICS-2026-08-01.md +0 -145
- package/docs/FOUNDER-STATUS-AND-DECISIONS-2026-07-01.md +0 -132
- package/docs/LOKI-10-BUILD-PROMPT.md +0 -350
- package/docs/STRATEGY-2026-2028.md +0 -124
- package/docs/V10-VISION.md +0 -161
- package/docs/v10/ADOPTION-PLAN.md +0 -375
- package/docs/v10/ARCHITECT-CUTS.md +0 -744
- package/docs/v10/BACKLOG.md +0 -231
- package/docs/v10/BOARD.md +0 -1403
- package/docs/v10/CONTROL-PLANE.md +0 -120
- package/docs/v10/CONTROL.md +0 -36
- package/docs/v10/CP-ASK-PLAN.md +0 -85
- package/docs/v10/CP-ENTERPRISE-UI.md +0 -537
- package/docs/v10/D51-PHASE-B.md +0 -120
- package/docs/v10/D61-SPEED.md +0 -105
- package/docs/v10/D63-CARDS.md +0 -354
- package/docs/v10/DECISIONS.md +0 -915
- package/docs/v10/E2E-D65.md +0 -123
- package/docs/v10/ENGINE-LAWS.md +0 -163
- package/docs/v10/ENGINE-MEASURE.md +0 -69
- package/docs/v10/ENGINE.md +0 -743
- package/docs/v10/FAILURE-CLASSES.md +0 -2026
- package/docs/v10/FC21B-FC22-PLAN.md +0 -107
- package/docs/v10/FIRST-RUN-GATE.md +0 -39
- package/docs/v10/FOUNDER-QUEUE.md +0 -29
- package/docs/v10/GAP-RESEARCH-1003.md +0 -32
- package/docs/v10/GUARDS.md +0 -883
- package/docs/v10/L0-WAVE1.md +0 -81
- package/docs/v10/LEGACY-REMOVAL.md +0 -408
- package/docs/v10/MEDIUM-ANALYSIS.md +0 -346
- package/docs/v10/METRICS.md +0 -645
- package/docs/v10/MODERNIZE.md +0 -437
- package/docs/v10/OPENCODE-PARITY-PLAN.md +0 -84
- package/docs/v10/OPERATING-MODEL.md +0 -60
- package/docs/v10/PLAN-V12.md +0 -896
- package/docs/v10/PROGRESS.md +0 -2735
- package/docs/v10/RELEASE-11.md +0 -77
- package/docs/v10/RETRO-2026.md +0 -650
- package/docs/v10/ROADMAP-D63.md +0 -35
- package/docs/v10/SCALE.md +0 -140
- package/docs/v10/SCORECARD-PLAN.md +0 -250
- package/docs/v10/SWARM-PROMPT.md +0 -98
- package/docs/v10/SWARM.md +0 -293
- package/docs/v10/cp-redesign/PARITY.md +0 -188
- package/docs/v10/cp-redesign/legacy/INDEX.md +0 -1167
- package/docs/v10/roadmap-10/01-measurement.md +0 -113
- package/docs/v10/roadmap-10/02-cost.md +0 -75
- package/docs/v10/roadmap-10/03-speed.md +0 -65
- package/docs/v10/roadmap-10/04-first-run.md +0 -61
- package/docs/v10/roadmap-10/05-deploy.md +0 -61
- package/docs/v10/roadmap-10/06-stability.md +0 -64
- package/docs/v10/roadmap-10/07-quality.md +0 -57
- package/docs/v10/roadmap-10/08-proof.md +0 -48
- package/docs/v10/roadmap-10/09-safety.md +0 -74
- package/docs/v10/roadmap-10/README.md +0 -1464
- package/skills/sdlc-fleet.md +0 -170
- package/tools/cockpit/SPEC.md +0 -275
- package/tools/cockpit/index.html +0 -146
- package/tools/cockpit/roi-basis.json +0 -33
- package/tools/cockpit/server.py +0 -946
package/docs/v10/E2E-D65.md
DELETED
|
@@ -1,123 +0,0 @@
|
|
|
1
|
-
# D65 Phase 2 end-to-end pass: plan, stub-leg results, bugs, operator commands
|
|
2
|
-
|
|
3
|
-
Status: stub legs run 2026-10-02 (loki-mode 10.6.7 from npm `latest`, macOS arm64, bun 1.x on PATH).
|
|
4
|
-
Real-provider legs are PENDING: they need `~/.config/loki/e2e.env`, which this document's author
|
|
5
|
-
never read or loaded. No leg here used real provider auth.
|
|
6
|
-
|
|
7
|
-
## 1. Plan
|
|
8
|
-
|
|
9
|
-
Every leg starts from a clean install and never touches the operator's real HOME or any remote.
|
|
10
|
-
|
|
11
|
-
1. Clean install: `npm i -g --prefix <run>/p loki-mode@<version>` with `HOME=<run>/home`,
|
|
12
|
-
`npm_config_cache=<run>/npmcache`. Record install seconds and `loki version`.
|
|
13
|
-
2. Three repos:
|
|
14
|
-
- `asklokesh/augmentiq#52` (private, expected ALREADY_SATISFIED, needs gh token and provider auth: PENDING auth).
|
|
15
|
-
- `octocat/Hello-World` (tiny public repo, stub leg 1).
|
|
16
|
-
- `sindresorhus/is-plain-obj` (small public repo, stub leg 2).
|
|
17
|
-
- The gate proper (`scripts/real-repo-gate.sh`) also runs the two corpus rows (humanize-333, click) from `scripts/real-repo-gate.repos`.
|
|
18
|
-
3. Push disabled: shallow clone (`--depth 1`), `git remote set-url --push origin no-push`, `GIT_CONFIG_GLOBAL=/dev/null`, `--no-pr`. Gate clones additionally set `remote.origin.pushurl=invalid://push-disabled-by-real-repo-gate`.
|
|
19
|
-
4. Stub legs: a stub `claude` on PATH, `LOKI_NO_BROWSER=1`, `LOKI_DASHBOARD=false`, no API key variables. Real legs: credentials only through `LOKI_E2E_ENV_FILE` (below).
|
|
20
|
-
5. Evidence per leg: exit codes, wall seconds, receipt presence, `loki verify` verdict, `loki proof verify`, diff.
|
|
21
|
-
|
|
22
|
-
## 2. Stub-leg results (loki-mode 10.6.7)
|
|
23
|
-
|
|
24
|
-
Install: `npm i -g --prefix ... loki-mode@latest` rc=0, 5 s (warm npm cache directory was empty, so a cold fetch), `loki version` = `Loki Mode v10.6.7`.
|
|
25
|
-
|
|
26
|
-
| Leg | Repo @ SHA | Command | Exit | Wall | Receipt | `loki verify` | Notes |
|
|
27
|
-
|---|---|---|---|---|---|---|---|
|
|
28
|
-
| A | octocat/Hello-World @ 7fd1a60b01f91b314f59955a4e4d4e80d8edf11d | `loki start ./task.md --no-pr` (file task, exactly what the gate runs for corpus rows) | 1 | 1 s | none | rc=1, CONCERNS (empty diff) | `Unknown option: --no-pr` (BUG-1) |
|
|
29
|
-
| B | octocat/Hello-World @ 7fd1a60b | `loki start "Add a short Contributing note to the README" --no-pr` (v10 route) | 1 | 4 s | yes, `.loki/runs/e10-20261003T020630Z-9ab3/receipt.json` (sha256:321a4232...) | rc=4, `NOT VERIFIED (run outcome FAILED; receipt integrity intact)` | `Not logged in` from the SDK loop; stub `claude` on PATH ignored by the v10 engine (BUG-2, BUG-3). Outcome FAILED, no diff |
|
|
30
|
-
| C | sindresorhus/is-plain-obj @ 666df7c10035f7e26f27ec214fe5ae3173435f34 | `LOKI_ENGINE=legacy LOKI_MAX_ITERATIONS=1 LOKI_SDK_MODE=off LOKI_SKIP_AUTH_PREFLIGHT=1 loki start ./task.md` | 0 | 71 s | yes, `.loki/proofs/run-20261003020703-25030-19911` | rc=0, `VERDICT: VERIFIED` (2 files, +122) | `loki proof verify` rc=0 (hash_ok, headline_consistent); receipt body says `quality_gate:code_review: failed` (BUG-4) |
|
|
31
|
-
| C0 | is-plain-obj @ 666df7c1 | same as C without `LOKI_SKIP_AUTH_PREFLIGHT=1` | 1 | 1 s | none | CONCERNS | Legacy auth preflight stops with `Claude Code is installed but not logged in`; correct behaviour |
|
|
32
|
-
|
|
33
|
-
Push check: after every leg `git remote get-url --push origin` = `no-push`; nothing left the machine.
|
|
34
|
-
|
|
35
|
-
## 3. Bugs
|
|
36
|
-
|
|
37
|
-
BUG-1 (gate blocker, P1): `loki start <file> --no-pr` is rejected by the legacy parser.
|
|
38
|
-
`bin/loki` routes `start` to the v10 engine only for issue refs, issue URLs and multi-word tasks (`bin/loki` start case); a path to a file stays legacy, and the legacy `cmd_start` has no `--no-pr`. `scripts/real-repo-gate.sh` runs corpus rows as `loki start <T>/issue-NAME.md --no-pr`, so both corpus rows (humanize-333, click) die in 1 s with `Unknown option: --no-pr` on a real install. The stub-`loki` test hides it because the stub accepts anything.
|
|
39
|
-
Repro: `loki start ./task.md --no-pr` in any git clone; exit 1.
|
|
40
|
-
Fix options: route `start <file>.md --no-pr` to v10, or accept and ignore `--no-pr` in legacy `cmd_start`, or have the gate pass the issue text as a multi-word task. Add a gate test that uses the real routing.
|
|
41
|
-
|
|
42
|
-
BUG-2 (P2): the v10 engine does not use a `claude` found on PATH, so a stub provider cannot drive it.
|
|
43
|
-
The SDK loop spawns its own Claude Code and reports `Not logged in` even with a stub first on PATH. The legacy engine does honor PATH `claude`. Result: v10 stub legs are impossible, so no hermetic v10 e2e exists beyond unit tests. Suggest an env override (an executable path for the SDK loop) for tests.
|
|
44
|
-
Repro: Leg B.
|
|
45
|
-
|
|
46
|
-
BUG-3 (P2): an unauthenticated provider is reported as a generic failure by v10.
|
|
47
|
-
`loki-ts/src/engine10/machine.ts` matches only `Failed to authenticate|API key is invalid` and the credit-balance wording. The SDK's `Not logged in - Please run /login` is not matched, so the run ends `Outcome: FAILED Reason: exit 1 (general error)` plus four NOT PROVEN items, with no pointer to the login fix. Legacy prints an actionable message.
|
|
48
|
-
Repro: Leg B stderr log `.loki/runs/*/..-impl.stderr.log` contains `[sdk-loop error: Claude Code returned an error result: Not logged in ...]`.
|
|
49
|
-
|
|
50
|
-
BUG-4 (P3, wording): after a legacy run whose receipt lists `quality_gate:code_review: failed` under "Not yet verified", bare `loki verify` prints `VERDICT: VERIFIED`, because it is the diff-scope deterministic verify, while the receipt headline is not a full pass. Two commands, two meanings of VERIFIED. Operators should read the receipt headline, not only `loki verify`. Repro: Leg C.
|
|
51
|
-
|
|
52
|
-
Observation, not a Loki bug: my first stub appended to `README.md` on every invocation, including the auth probe, so the stub must only write on a real `-p` call.
|
|
53
|
-
|
|
54
|
-
## 4. Gate script change (this commit)
|
|
55
|
-
|
|
56
|
-
`scripts/real-repo-gate.sh` accepts `LOKI_E2E_ENV_FILE` (default unset, behaviour unchanged when unset). When set it requires a regular non-symlink file owned by the current user with mode 0600 (refuses with exit 2 otherwise), then loads it with `set -a; . file; set +a` inside the gate process only, output discarded, never echoed. Tested in `tests/test-real-repo-gate.sh` (0644 refused, 0600 loads, dummy value absent from stdout, stderr and logs). Caveat: the gate's per-repo log captures `loki` output, so a provider that echoes its own key would land in the log; the gate itself never writes values.
|
|
57
|
-
|
|
58
|
-
## 5. Operator commands once `~/.config/loki/e2e.env` exists
|
|
59
|
-
|
|
60
|
-
Create it with `umask 077` and put only the provider variables in it (`ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`; `GH_TOKEN` for the private augmentiq repo, or rely on `gh auth login`).
|
|
61
|
-
|
|
62
|
-
```bash
|
|
63
|
-
chmod 600 ~/.config/loki/e2e.env
|
|
64
|
-
stat -f '%Lp %Su' ~/.config/loki/e2e.env # expect: 600 <you>
|
|
65
|
-
|
|
66
|
-
# 0. Do not run until BUG-1 is fixed, or the two corpus rows fail on the file-task route.
|
|
67
|
-
cd <loki-mode checkout>
|
|
68
|
-
|
|
69
|
-
# 1. Local gate on the published latest (clean install, throwaway HOME, push disabled, all three repos)
|
|
70
|
-
LOKI_E2E_ENV_FILE=$HOME/.config/loki/e2e.env \
|
|
71
|
-
bash scripts/real-repo-gate.sh --version "$(npm view loki-mode version)" \
|
|
72
|
-
--results "$HOME/loki-ci-logs/real-repo-gate-e2e.results"
|
|
73
|
-
echo "gate exit: $?" # 0 pass, 1 repo failed or nothing ran, 2 setup error
|
|
74
|
-
|
|
75
|
-
# 2. augmentiq#52 alone, for a clearer log
|
|
76
|
-
LOKI_E2E_ENV_FILE=$HOME/.config/loki/e2e.env \
|
|
77
|
-
bash scripts/real-repo-gate.sh --version "$(npm view loki-mode version)" --repo 'asklokesh/augmentiq#52'
|
|
78
|
-
|
|
79
|
-
# 3. Inspect: results table and per-repo logs (logs are in the .logs dir next to the results file)
|
|
80
|
-
cat "$HOME/loki-ci-logs/real-repo-gate-e2e.results"
|
|
81
|
-
ls "$HOME/loki-ci-logs/real-repo-gate-e2e.logs"
|
|
82
|
-
|
|
83
|
-
# 4. Confirm no push happened and nothing leaked
|
|
84
|
-
grep -rIl -E 'sk-ant-|oat01-' "$HOME/loki-ci-logs/real-repo-gate-e2e.logs" || echo "no key material in logs"
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Pass criteria per repo are in `scripts/real-repo-gate.repos` (expected outcome, allowed diff paths, max seconds); evidence to record for the log: exit code, wall seconds, Outcome line, receipt path, `loki verify` verdict.
|
|
88
|
-
|
|
89
|
-
## 6. Stub E2E (`scripts/e2e-d65.sh`)
|
|
90
|
-
|
|
91
|
-
Hermetic run on loki-mode 10.6.8 (tree 158d977ba plus the C11b legs), 2026-10-03. The script runs `npm pack`, installs the tarball into a run-owned temp prefix (throwaway `HOME` and npm cache), puts a stub `claude` on PATH and drives the INSTALLED `loki` under `env -i` with an allowlist: no `*_API_KEY`, no `*_TOKEN` (only fake values such as `JIRA_API_TOKEN=e2e-fake-token`). Command: `LOKI_NO_BROWSER=1 timeout 600 bash scripts/e2e-d65.sh`; total 19 s, script rc=1 (two FAIL legs, both named product bugs below).
|
|
92
|
-
|
|
93
|
-
BUG-2 (the v10 engine ignored the stub `claude`) is resolved by `LOKI_E10_INVOKER=cli` plus `LOKI_CLAUDE_CLI=<stub>`, which the harness sets for every leg: legs 1, 2, 5 and 6 each produced a receipt with run rc=0 and no `Not logged in`.
|
|
94
|
-
|
|
95
|
-
| Leg | Result | Wall | rc line (from the run) |
|
|
96
|
-
|---|---|---|---|
|
|
97
|
-
| 1 D61 speed | PASS | 1 s | `loki quick rc=0 run=e10-20261003T053414Z-f543` |
|
|
98
|
-
| 2 Visual evidence | PASS | 1 s | `loki quick rc=0 run=e10-20261003T053416Z-13df`; NOT PROVEN line present |
|
|
99
|
-
| 3 Control Plane | PASS | 1 s | `blocked run rc=4 run=e10-20261003T053417Z-a324`; `POST answer ... -> 200` |
|
|
100
|
-
| 4 Slack | PASS | 1 s | signed `url_verification` returned the challenge; bad signature 401; no outbound connection |
|
|
101
|
-
| 5 Jira | PASS | 1 s | `loki quick jira:ABC-1 rc=0 run=e10-20261003T053418Z-d497`, `issue.json has source=jira`; `no-credentials rc=2 in 0s`, names `JIRA_EMAIL`/`JIRA_API_TOKEN` |
|
|
102
|
-
| 6 Spec to contract | PASS | 1 s | `loki quick rc=0 run=e10-20261003T053419Z-7afd`; receipt has the `contract` section with AC-1 |
|
|
103
|
-
| 7 Multi-repo | FAIL | 1 s | `loki workspace run rc=2`, `receipts=0 integration.json=none` (BUG-5) |
|
|
104
|
-
| 8 MCP | PASS | 0 s | `python rc=0`; `v10_status` done/VERIFIED, `v10_verify` `exit_code` 0 on leg 6's repo |
|
|
105
|
-
| 9 Containers | FAIL | 3 s | deploy files present; docker daemon answered; `docker build` failed (BUG-6) |
|
|
106
|
-
|
|
107
|
-
SKIP legs: none in this run. Leg 8 would SKIP if leg 6 had not produced a run, and leg 9 SKIPs with `SKIP(docker unavailable)` when `docker` is missing or `docker info` does not answer within 10 s; it never passes without a real `/ready` 200 probe on a random host port.
|
|
108
|
-
|
|
109
|
-
Positive control: `E2E_LEGS=6 E2E_FORCE_FAIL=6 bash scripts/e2e-d65.sh` prints `FAIL leg 6 ... forced by E2E_FORCE_FAIL=6 (positive control)` and exits 1. After each run the recorded PIDs (control-plane servers, the fake Jira `http.server`, the Slack proxy trap) are stopped by PID and the run directory is removed; nothing outside the run dir was touched.
|
|
110
|
-
|
|
111
|
-
New bugs found by this run (product code is not changed by this slice):
|
|
112
|
-
|
|
113
|
-
BUG-5 (P1, packaging): the npm package omits `schemas/`, so `loki workspace` fails on every installed copy. `package.json` `files` has no `schemas/` entry, while `autonomy/lib/loki_yaml.py:23` reads `schemas/loki-yaml.schema.json` relative to the package root. Repro: install the tarball, put a `workspaces:` block in `loki.yaml`, run `loki workspace run <name> <task>`; output `workspace: cannot read loki.yaml: [Errno 2] No such file or directory: '.../node_modules/loki-mode/autonomy/lib/../../schemas/loki-yaml.schema.json'`, rc=2. Leg 7 stays FAIL until `schemas/loki-yaml.schema.json` ships; the two-runs and `integration.json` assertions were therefore not reached.
|
|
114
|
-
|
|
115
|
-
BUG-6 (P1, container): `Dockerfile.control-plane` cannot build. Its build stage copies only `packages/control-plane/`, but `packages/control-plane/src/server/ingest.ts:3` and `:5` import `../../../../loki-ts/src/engine10/events.ts` and `../../../../loki-ts/src/util/redact.ts`. Repro: `docker build -f Dockerfile.control-plane .` fails at `RUN bun run build:all` with `error: Could not resolve: "../../../../loki-ts/src/engine10/events.ts"` and `.../util/redact.ts`. The `/ready` probe was therefore not reached and leg 9 is FAIL, not PASS.
|
|
116
|
-
|
|
117
|
-
### 6.1 Rerun after BUG-5, BUG-6 and BUG-7 fixes (2026-10-03)
|
|
118
|
-
|
|
119
|
-
The 07:56Z rerun on the main tree that includes the BUG-5 and BUG-6 fixes passed 8 of 9 legs. Legs 8 (MCP) and 9 (containers) PASS: `docker build` now succeeds and `/ready` answers 200. Leg 7 still failed with `workspace run rc=1, expected 0; found 0 receipts; integration.json written`.
|
|
120
|
-
|
|
121
|
-
BUG-7 (P1, engine): `excludeLokiDir` in `loki-ts/src/engine10/stages/intake.ts` hard-coded `<repo>/.git/info/exclude`. Workspace runs execute in linked git worktrees, where `.git` is a file, so `mkdir` threw `ENOTDIR: not a directory, mkdir '.../worktrees/e2e__api/.git/info'`. The first repo's run exited 1 and the second was skipped. Fix: resolve the path with `git rev-parse --git-path info/exclude` (main b5a0085a0). A new intake test goes red on the old code (15 pass, 1 fail) and green on the fix (16/0).
|
|
122
|
-
|
|
123
|
-
Result after the fix: `PASS leg 7 (2s) leg_07_workspaces: two runs (2 receipts) plus integration.json, rc=0`; `e2e-d65: legs=9 fails=0 total=19s`. BUG-5, BUG-6 and BUG-7 are resolved; all 9 stub legs pass from a clean npm install.
|
package/docs/v10/ENGINE-LAWS.md
DELETED
|
@@ -1,163 +0,0 @@
|
|
|
1
|
-
<!-- Engine constitution. Source: founder plan 2026-10-03 (autonomi-dev research/2026-10-03-engine-laws/PLAN.md), adopted as D86. -->
|
|
2
|
-
|
|
3
|
-
# Engine Laws: from one-off fixes to an enterprise-grade engine
|
|
4
|
-
|
|
5
|
-
Founder directive, 2026-10-03: "Use all intelligence in any session from the model and provider as needed, so our issues in Loki Mode won't impact users. Plan and ultrathink to make it enterprise grade. Not just one-offs: every issue and learning should be thought about at 100k feet."
|
|
6
|
-
|
|
7
|
-
## 1. The trigger and the diagnosis
|
|
8
|
-
|
|
9
|
-
On FireLater#17, plain Claude Code fixed the issue and opened a good PR in about 2 minutes. Loki 10.6.14 took 11 minutes and $2.66, ended STALLED and opened a FAILED draft PR (#25). Six defects showed up in that one run. Read from the code, they come from four root classes. None of them is specific to FireLater.
|
|
10
|
-
|
|
11
|
-
| Symptom on FireLater#17 | Code root | Root class |
|
|
12
|
-
|---|---|---|
|
|
13
|
-
| Verify "Failed Suites 1" three times; the code passed 43/43 from backend/ | testmap.ts: fixed `npx vitest run <files>` strings, always run from repoDir | A. The harness assumes a repo shape instead of learning it |
|
|
14
|
-
| "visual evidence skipped: no package.json" | root-only package.json probe | A |
|
|
15
|
-
| Two fix rounds plus STALLED spent on a non-bug | verify does not separate "runner could not load" from "test failed"; fix.ts treats every vitest signature as a test failure | B. Failures are not attributed to an owner (code, env, harness, provider) |
|
|
16
|
-
| Route edits the issue required were reverted | e10ext/scope.ts: keyword overlap of plan and task text decides "unrelated", then `git restore` | C. A heuristic holds destructive authority |
|
|
17
|
-
| Implement ran below the user's raw intelligence | sizing.ts: cascadeImplementModel = wallModel = sonnet by default; implement brief and limitS 480 replace the agent's own exploration | D. Loki silently gives the user less intelligence than raw |
|
|
18
|
-
| PR body: "not recorded" for issue, why, scope and tests | pr_body.ts output not checked against real run data | E. Output contracts are untested on real runs |
|
|
19
|
-
| Control Plane stuck "in progress" after completion | no guarantee that every sink reaches a terminal state | F. No run-lifecycle invariant |
|
|
20
|
-
|
|
21
|
-
Each class has a law. Each law has one mechanism, plus one enforcement check that fails CI if the law is broken anywhere, not only in the stage the bug report named.
|
|
22
|
-
|
|
23
|
-
## 2. The Engine Laws (constitution for every stage, present and future)
|
|
24
|
-
|
|
25
|
-
### L0. The model decides, the harness proves (above L1 to L7; founder directive 2026-10-03, permanent)
|
|
26
|
-
- The model owns all knowledge and judgment about the user's repo and task: layout and package roots, the runner, command and cwd, whether there is a UI and how to boot it, files in scope, impacted tests, task size, failure ownership, and what the issue asks.
|
|
27
|
-
- The harness owns only mechanism: running what the model proposed and recording cwd, exit code, output and counts; sandboxing, budgets and process control; signing, receipts and integrity; terminal I/O; deterministic tamper checks, because the model never grades its own work.
|
|
28
|
-
- When execution contradicts the model, the output goes back to the model once. The verdict comes from execution evidence plus a separate reviewer call, never from the author.
|
|
29
|
-
- Review rule: reject any change that adds an `if` or a regex about the user's repo shape, language, framework or task wording. The fix is a prompt or schema change plus an execution check. Hardcoded tables are deleted once the model path is green on the Repo Shape Matrix. Slices: docs/v10/L0-WAVE1.md.
|
|
30
|
-
|
|
31
|
-
### L1. Never below raw
|
|
32
|
-
A Loki run never gives the user a worse result, or a meaningfully slower or costlier one, than the same provider run raw, as measured by the Parity Gate thresholds in section 4 (correctness at least raw; time within raw + 30s for small tasks and raw x 1.3 otherwise; cost at most raw x 1.2). "Never below raw" means OUTCOME parity (solve rate, wall time and cost on the B9 corpus and the Parity Gate). It does not mean model identity (amended by D89, 2026-10-07).
|
|
33
|
-
- On the Claude engine10 path with the router on, the default implement model and effort are chosen by the router (Opus decides, per ROUTER-1). Everywhere else, and with `LOKI_ROUTER=0`, they are the provider's best default for the user's account (what the bare CLI would use), or higher.
|
|
34
|
-
- A Haiku executor runs only in a configuration with Parity Gate evidence: with the Opus advisor attached, or without it only if the no-advisor arm also meets the Parity Gate. Otherwise the executor is Sonnet 5.5 and the reason is printed on the start line and recorded in the receipt. A shape where Haiku fails the Parity Gate runs on Sonnet 5.5. A shape where Sonnet also fails runs on the pre-router default (the `LOKI_ROUTER=0` behavior) until a Parity Gate row shows the routed executor at parity.
|
|
35
|
-
- Any downgrade (model, effort, context, time limit) is printed on the start line and recorded in the receipt, including the routed executor and the advisor status. Nothing is silently reduced.
|
|
36
|
-
- Escalation goes UP, always before STALLED, for every provider: a repeated code-owned failure escalates to a stronger model, ending at the strongest model and effort the provider offers, with the full failure output and the agent's own diagnosis. On the router path the ladder is Haiku -> Sonnet 5.5 for the rest of the run -> Opus for a repeated failure on Sonnet.
|
|
37
|
-
- Enforcement: the Parity Gate (section 4) blocks promotion to `latest` when Loki is below raw on the corpus. The router is default-on only after this gate passes on the release candidate (D89 item 9).
|
|
38
|
-
|
|
39
|
-
### L2. Fail closed on trust, fail open on work
|
|
40
|
-
- Trust surfaces fail CLOSED and may block: signing, receipt integrity, secrets, sandbox and permission boundaries, policy the user configured, and tests or CI config being deleted or weakened.
|
|
41
|
-
- Work surfaces fail OPEN: scope, the Wall, impacted-test selection, sizing, visual evidence, repo detection. A harness heuristic that is unsure keeps the agent's work, annotates it, and lowers the claim (NOT PROVEN). It never reverts, stalls or drafts.
|
|
42
|
-
- Enforcement: a static test lists every code path that calls `git restore`, `git checkout --`, `git reset` or file deletion on user files, or sets outcome STALLED, FAILED or draft. Each one must be classified trust or work, and work paths may not do it. A new destructive call fails CI until it is classified.
|
|
43
|
-
|
|
44
|
-
### L3. Evidence outranks heuristics (authority ladder)
|
|
45
|
-
1. Executed evidence: a command Loki ran, with its output, cwd and exit code.
|
|
46
|
-
2. The agent's executed evidence: commands the agent ran in its session, taken from the transcript or tool log.
|
|
47
|
-
3. Model judgment: the agent or a reviewer.
|
|
48
|
-
4. Deterministic heuristics: keyword scope, sizing, repo-map overlap.
|
|
49
|
-
|
|
50
|
-
A lower rung may never override a higher one. A keyword heuristic cannot revert what a passing test run supports. A Loki check that ERRORED (not failed) yields to the agent's successful run of the same suite.
|
|
51
|
-
|
|
52
|
-
### L4. Know the repo before judging it (Project Model)
|
|
53
|
-
At intake, one model discovery session reads the repo like a senior engineer and returns a schema-checked Project Model that cites the files behind each answer (L0). Every stage consumes it. No stage may re-derive repo shape on its own, and no harness table or probe is authoritative; the items below are what the model reads, not code the harness runs.
|
|
54
|
-
- Packages: every package root (package.json, pyproject, go.mod, Cargo.toml, pom or gradle, composer, Gemfile), its runner, and its exact test, lint, build and start commands.
|
|
55
|
-
- Workspace kind: single, npm/pnpm/yarn workspaces, turbo, nx, multi-root with no root manifest (FireLater), polyglot.
|
|
56
|
-
- Sources of truth, in order:
|
|
57
|
-
1. loki.yaml overrides;
|
|
58
|
-
2. AGENTS.md, CLAUDE.md, CONTRIBUTING and README test instructions;
|
|
59
|
-
3. CI workflow files (.github/workflows: how the project's own CI actually runs tests);
|
|
60
|
-
4. manifests and config files;
|
|
61
|
-
5. conventions.
|
|
62
|
-
- Self-calibration: run each package's test command once as a baseline (this is also the Wall base run). If the runner fails to LOAD (0 tests collected, config not found, module resolution error), ask the agent once, cheaply: "how are tests run in this package?". Store the answer, rerun, and cache it per repo in .loki/project.json, keyed by manifest hashes. Users can edit it.
|
|
63
|
-
- Every command records its cwd. A test path is resolved to its nearest package root and made relative to it.
|
|
64
|
-
- Enforcement: no stage may spawn a test, lint or build command except through the Project Model API (a grep check in CI).
|
|
65
|
-
|
|
66
|
-
### L5. Every failure has an owner
|
|
67
|
-
Every check result is one of PASS, FAIL, ERROR or SKIP, with an owner: code, env, harness or provider.
|
|
68
|
-
- Only FAIL owned by code drives fix rounds and counts toward STALLED.
|
|
69
|
-
- ERROR owned by harness or env triggers one self-repair (recalibrate the Project Model). If that does not fix it, the check is NOT PROVEN with a plain reason ("test runner could not load in backend/: <first error line>"). No paid fix rounds.
|
|
70
|
-
- ERROR owned by provider (billing, quota, auth, overload) goes to the existing fatal classifier.
|
|
71
|
-
- The receipt and PR list every NOT PROVEN item with its owner, so a user can tell "your code is broken" from "Loki could not check".
|
|
72
|
-
- Enforcement: a fixture set of runner outputs (load error, 0 collected, missing config, real assertion failure, timeout, OOM) for vitest, jest, node:test, pytest, go and cargo. The classifier must label each correctly.
|
|
73
|
-
|
|
74
|
-
### L6. Every run terminates everywhere
|
|
75
|
-
Every sink reaches the same terminal state: events, receipt, PR, Control Plane, Slack and the CLI summary.
|
|
76
|
-
- The supervisor writes run.completed on every exit path, including crash and kill (signal handlers, plus a sweeper for dead PIDs).
|
|
77
|
-
- Control Plane ingest is idempotent and reconciles: a run with no live worker PID and no terminal event becomes ABORTED (with a reason) after a grace period.
|
|
78
|
-
- Enforcement: an e2e test kills a run at each stage and asserts terminal state in every sink.
|
|
79
|
-
|
|
80
|
-
### L7. Outputs are contracts
|
|
81
|
-
The PR body, receipt, `--json` envelope and CLI summary have schemas. "not recorded" when the data exists is a test failure.
|
|
82
|
-
- Golden tests are built from real recorded runs (FireLater#17 included), not hand-made stubs.
|
|
83
|
-
|
|
84
|
-
### H4. Self-improving routing (per repo, per shape; D89)
|
|
85
|
-
An H-law is a harness learning rule: it lets recorded outcome evidence improve a decision over time, and it never overrides L0 to L7.
|
|
86
|
-
- Every sealed run writes a per-repo, per-shape outcome record: executor, verdict, escalated (yes or no), cost in USD and wall time. Records are executed outcome data (L3 rung 1), never a heuristic about repo shape or task wording (L0).
|
|
87
|
-
- The shape key comes from the Project Model (workspaceKind plus the package runner labels, L4), never from a file pattern or regex (L0). It is used only as a lookup key into executed outcome evidence (the per-repo history, or the shipped router-shape-defaults.json seeded from B9 rows), and it may only move the executor up, never down.
|
|
88
|
-
- The history is fed to the Opus routing prompt. As an evidence floor, a shape where Haiku lost 2 of the last 3 runs (a code-owned FAIL or an escalation; harness, env and provider ERRORs and NOT PROVEN outcomes do not count, L5) defaults to Sonnet 5.5 until a B9 row shows Haiku at parity on that shape.
|
|
89
|
-
- History only changes the starting executor. It never removes the advisor requirement, never disables escalation, and a missing or unreadable history yields the router default with a NOT PROVEN line (L2, fail open on work).
|
|
90
|
-
|
|
91
|
-
## 3. The learning system (how every future issue is handled at 100k feet)
|
|
92
|
-
|
|
93
|
-
No bug becomes a fix slice until the CoS has written its row in docs/v10/FAILURE-CLASSES.md, answering five questions:
|
|
94
|
-
|
|
95
|
-
1. What did the user see, and how does it compare with raw provider output on the same task?
|
|
96
|
-
2. Which law did it break (L1 to L7)? If none fits, propose a new law. That is a founder-level decision; record it in DECISIONS.
|
|
97
|
-
3. Sibling sweep: which other stages, commands or providers share the same assumption? Grep and list them. The fix covers all of them, at the shared layer.
|
|
98
|
-
4. Mechanism: the one shared place the fix lives (Project Model, the result classifier, the authority ladder, the lifecycle reconciler, the output schema).
|
|
99
|
-
5. Regression: the corpus fixture or real-run golden that would have caught it, added in the same PR.
|
|
100
|
-
|
|
101
|
-
A one-off patch that answers question 4 with "this stage only" when siblings exist is rejected in review.
|
|
102
|
-
|
|
103
|
-
## 4. Proof: the Parity Gate and the Repo Shape Matrix
|
|
104
|
-
|
|
105
|
-
- Repo Shape Matrix (the corpus). At least one real public repo or faithful fixture per shape:
|
|
106
|
-
- single npm;
|
|
107
|
-
- npm, pnpm and yarn workspaces;
|
|
108
|
-
- turbo;
|
|
109
|
-
- nx;
|
|
110
|
-
- multi-root with no root manifest (FireLater);
|
|
111
|
-
- python with uv, poetry and plain pip;
|
|
112
|
-
- go multi-module;
|
|
113
|
-
- rust workspace;
|
|
114
|
-
- java gradle and maven;
|
|
115
|
-
- polyglot (TS frontend plus Python backend).
|
|
116
|
-
|
|
117
|
-
Each shape has one small, one medium and one cross-cutting issue (FireLater#17 is the cross-cutting monorepo case).
|
|
118
|
-
- Parity Gate (extends the D49 first-run gate for `latest` promotion). Loki is compared with raw `claude -p` on the same issues, with hidden tests judging correctness.
|
|
119
|
-
- Correctness: Loki is at least raw on every shape.
|
|
120
|
-
- Time: small tasks within raw + 30s; medium and large tasks within raw x 1.3.
|
|
121
|
-
- Cost: at most raw x 1.2, unless the user turned on deeper checks.
|
|
122
|
-
- Zero harness-owned FAILED or STALLED outcomes.
|
|
123
|
-
- Results go to METRICS.md. A regression blocks `latest`; `next` keeps shipping.
|
|
124
|
-
- Field signal, which stays free and private: `loki report <run-id>` builds a redacted bundle the user can attach to an issue. Nothing is sent automatically.
|
|
125
|
-
|
|
126
|
-
## 5. Enterprise posture (what a platform team needs before adopting)
|
|
127
|
-
|
|
128
|
-
- Predictability: published SLOs, the Parity Gate numbers per release, and the same result on the same input (models are pinned and recorded in the receipt).
|
|
129
|
-
- Explainability: every outcome has one plain sentence of cause and an owner. Nothing is "not recorded".
|
|
130
|
-
- Policy without friction: loki.yaml `scope: advisory|enforce`, where advisory is the default and enforce is opt-in for regulated repos. Enforce runs on the Project Model and the contract, never on keywords alone, and its reverts are listed with a one-command restore.
|
|
131
|
-
- Audit: the receipt carries the Project Model hash, the commands with their cwd, the owner attribution and any model or effort changes.
|
|
132
|
-
- No silent degradation, anywhere.
|
|
133
|
-
|
|
134
|
-
## 6. Implementation order (delete first, then build the shared layer)
|
|
135
|
-
|
|
136
|
-
Wave 0, same day. Flips and deletions, each small:
|
|
137
|
-
1. The scope default becomes advisory: annotate, never revert (L2). Keep enforce behind loki.yaml.
|
|
138
|
-
2. Cascade default off: implement runs on the provider's best default model and effort (L1). Fix-round escalation goes up to the strongest model.
|
|
139
|
-
3. A load-error or 0-collected verify result is ERROR with owner harness. It never feeds fix rounds or STALLED (L5, minimal form).
|
|
140
|
-
4. PR body fields come from intake, plan, diff and evidence, with a golden test from the FireLater#17 recording (L7).
|
|
141
|
-
5. Control Plane: ingest run.completed, plus a dead-PID reconciler (L6, minimal form).
|
|
142
|
-
|
|
143
|
-
Wave 1. Shared layers, built in parallel by sonnet builders with opus review for anything touching trust:
|
|
144
|
-
6. Project Model module (L4) with AGENTS.md, CLAUDE.md and CI-workflow parsing, a baseline calibration run and the .loki/project.json cache. Testmap, verify, the Wall, visual evidence, the first-run gate and deep all consume it.
|
|
145
|
-
7. Result classifier with owner attribution, plus the runner fixture set (L5).
|
|
146
|
-
8. Authority ladder in seal: the agent's own passing test commands count as evidence when Loki's check errored (L3).
|
|
147
|
-
9. Lifecycle invariant across all sinks, plus the kill-at-each-stage e2e (L6).
|
|
148
|
-
10. The static checks from L2 and L4.
|
|
149
|
-
|
|
150
|
-
Wave 2. Proof:
|
|
151
|
-
11. The Repo Shape Matrix corpus and the Parity Gate wired into promotion to `latest`.
|
|
152
|
-
12. FAILURE-CLASSES.md seeded with the FireLater#17 rows and every past class from MEDIUM-ANALYSIS.md.
|
|
153
|
-
|
|
154
|
-
## 7. Acceptance
|
|
155
|
-
|
|
156
|
-
- FireLater#17 rerun on the new build:
|
|
157
|
-
- VERIFIED, with a ready (not draft) PR;
|
|
158
|
-
- the route edits kept;
|
|
159
|
-
- visual evidence found in backend/ or frontend/;
|
|
160
|
-
- time within raw + 30s and cost at most raw x 1.2.
|
|
161
|
-
- The Parity Gate is green on the matrix, and the table is published in METRICS.md.
|
|
162
|
-
- Every law has its enforcement check in CI.
|
|
163
|
-
- The CoS steering rule (section 3) is written into OPERATING-MODEL.md, and new bug reports follow it.
|
|
@@ -1,69 +0,0 @@
|
|
|
1
|
-
# Loki 10 engine: measured stage times (2026-09-27)
|
|
2
|
-
|
|
3
|
-
Source: read-only measurement of four real runs from their own logs. Every
|
|
4
|
-
duration comes from a logged timestamp; rows marked (inferred) come from gaps
|
|
5
|
-
between logged events.
|
|
6
|
-
|
|
7
|
-
## Runs
|
|
8
|
-
|
|
9
|
-
| Run | Log directory | Wall time | Tasks completed |
|
|
10
|
-
|---|---|---|---|
|
|
11
|
-
| R1 augmentiq #52 "add a search bar" | /Users/lokesh/git/augmentiq/.loki/ | 73.7+ min, still running at 21:46:12Z | 0 (`orchestrator.json` tasksCompleted 0) |
|
|
12
|
-
| R2 anonima | /Users/lokesh/git/anonima/.loki/ (2026-09-12 14:15Z) | 20.5 min | 1 (reached COMPLETED although code review returned fail) |
|
|
13
|
-
| R3 lokimode-anthropic, empty PRD | /Users/lokesh/git/lokimode-anthropic/.loki/ (2026-09-12 01:16Z) | 6.0+ min, log stops mid provider call | 0 |
|
|
14
|
-
| R4 fizzbuzz toy | /Users/lokesh/loki-council-1789161950/.loki/ (2026-09-11 21:25Z) | 3.8 min | 1 |
|
|
15
|
-
|
|
16
|
-
## Where the minutes go (minutes, logged durations only)
|
|
17
|
-
|
|
18
|
-
| Stage | R1 augmentiq #52 | R2 anonima | R3 lokimode | R4 fizzbuzz |
|
|
19
|
-
|---|---|---|---|---|
|
|
20
|
-
| Boot, intake, spec interrogation, PRD parsing | 1.3 | 0.5 | 0.6 | 0.3 |
|
|
21
|
-
| Agent provider calls | 3.9 (iterations 1-2) + 57.7+ (iteration 3, still running) | 3.9 | 5.4+ (cut off) | 0.6 |
|
|
22
|
-
| Of which: agent running the full E2E suite inside iteration 3 | about 41 (17:05-17:46 EDT, inferred from log file times) | not measured | not measured | not measured |
|
|
23
|
-
| App runner / docker | 1.8 | 0 | 0 | 0 |
|
|
24
|
-
| Gates (static through LSP) plus unlabelled gaps | 2.7 | 0.1 | 0 | 0 |
|
|
25
|
-
| Code review | 0.6 | 4.4 | 0 | 0.8 |
|
|
26
|
-
| Doc generation | 5.0 (timed out at 300s) | 4.3 | 0 | 0 |
|
|
27
|
-
| Checklist, evidence gate, council | 0.6 (council never voted) | 7.2 | 0 | 1.8 |
|
|
28
|
-
| Total wall time | 73.7+ | 20.5 | 6.0+ | 3.8 |
|
|
29
|
-
|
|
30
|
-
Agent share of wall time where the run finished: R2 19% (234s of 1231s), R4
|
|
31
|
-
15% (35s of 228s). The rest is harness stages around the agent.
|
|
32
|
-
|
|
33
|
-
## R1 detail (augmentiq #52)
|
|
34
|
-
|
|
35
|
-
| Stage | Start (Z) | End (Z) | Duration | Evidence |
|
|
36
|
-
|---|---|---|---|---|
|
|
37
|
-
| Issue fetch + boot | 20:32:30 | 20:32:43 | 13s | issue-context.json captured_at 20:32:30 |
|
|
38
|
-
| Spec interrogation | 20:32:44 | 20:33:09 | 24s | 20 assumptions recorded |
|
|
39
|
-
| PRD to tasks, LLM enrichment, complexity | 20:33:09 | 20:33:48 | 39s (inferred) | "Enriched 3 PRD task(s) via LLM"; complexity "complex" (9584 files) |
|
|
40
|
-
| Iteration 1 provider call | 20:33:48 | 20:36:34 | 166s | agent found search already built; COMPLETION_REQUESTED |
|
|
41
|
-
| App runner + checkpoint | 20:36:34 | 20:37:48 | 74s | "docker: unknown command: docker compose" |
|
|
42
|
-
| Gates | 20:37:48 | 20:38:01 | 13s | test_suite not_run |
|
|
43
|
-
| LSP, semantic, invariant gates, review diff | 20:38:01 | 20:39:10 | 69s (inferred) | |
|
|
44
|
-
| Code review (6 reviewers) | 20:39:10 | 20:39:43 | 33s | 5 pass, 1 fail (non-blocking) |
|
|
45
|
-
| Auto-documentation | 20:39:43 | 20:44:43 | 300s | "Auto-documentation: timed out after 300s" |
|
|
46
|
-
| Wiki + evidence gate | 20:44:43 | 20:45:19 | 36s | evidence gate BLOCKED |
|
|
47
|
-
| Iteration 2 provider call | 20:45:22 | 20:46:31 | 69s | claimed done again |
|
|
48
|
-
| App runner restart | 20:46:31 | 20:47:07 | 36s | "crash limit reached (5)" |
|
|
49
|
-
| Gates + diff | 20:47:07 | 20:48:26 | 79s | |
|
|
50
|
-
| Code review | 20:48:26 | 20:48:26 | 0s | refused: context 431361 bytes over 400000 limit |
|
|
51
|
-
| Iteration 3 provider call | 20:48:29 | running at 21:46:12 | 57m43s+ | 340+ tool calls; full vitest, pytest (2847 passed), Playwright 639 tests (88 failures in unrelated specs) |
|
|
52
|
-
|
|
53
|
-
## Five proven causes of non-completion (R1)
|
|
54
|
-
|
|
55
|
-
1. The app runner's docker invocation fails ("docker: unknown command: docker compose", then "unknown shorthand flag: 'd' in -d") while the containers report healthy, so the evidence gate records app_boot_failed (metrics/trust-events.jsonl 20:45:21Z; app-runner/state.json crash_count 5).
|
|
56
|
-
2. A secret-leak match in the changed file SETUP.md blocks completion ("Evidence gate BLOCKED: a secret/credential was detected in the changed files").
|
|
57
|
-
3. The test gate finds no test runner ("no test runner detected, recording inconclusive"; tests_ok=inconclusive runner=none) while the agent's own runs show pytest 2847 passed and vitest 1552 passed.
|
|
58
|
-
4. Code review refuses a 431361-byte context (committed USAGE.md, TESTING.md, SETUP.md), and completion is then rejected "because code review is BLOCKED (Critical/High findings)" although no review ran.
|
|
59
|
-
5. Iteration 3 is one provider call with no time limit and no progress events: the last events.jsonl entry is 20:48:28Z iteration_start 3, and dashboard-state.json still shows iteration 0 and council total_votes 0 at 21:45:53Z. Inside it the agent drifted into the unrelated full E2E suite at 94-100% CPU and killed processes with pkill -f and kill -9.
|
|
60
|
-
|
|
61
|
-
Also: the feature already existed before the run (iteration 1: "The search feature already appears fully built across many prior commits"), and both iteration 1 and 2 completion claims were rejected by the gates.
|
|
62
|
-
|
|
63
|
-
## Design consequences for ENGINE.md
|
|
64
|
-
|
|
65
|
-
- No PRD generation, spec interrogation, doc generation or council on the fast path (R1: 7+ min before and around the agent; R2: 16 of 20.5 min).
|
|
66
|
-
- The implement session needs a wall cap and progress events; the run state must come from one event log, never a separate dashboard-state file that goes stale.
|
|
67
|
-
- Verification runs impacted tests only on the fast path; full suites, app boot and council move to async deep verify after the PR.
|
|
68
|
-
- Test runner detection must read the repo's real runners (pytest, vitest, npm scripts), or the gate says inconclusive while the suites pass.
|
|
69
|
-
- Review context must exclude generated docs, and a refused review must never be reported as a blocking finding.
|