loki-mode 11.2.2 → 11.3.1

package/docs/FEAT-PRDREUSE-DOCKER-PLAN.md

@@ -1,144 +0,0 @@
-# FEAT-PRDREUSE-DOCKER-PLAN
-
-Implementation plan for the PRD-reuse + Docker feature batch. Anchored to verified file:line.
-Designed for 4-5 parallel dev agents with zero file overlap. Founder scope locks:
-- Docker dashboard: publish on a HOST PORT and AUTO-OPEN (like local loki start); show BOTH local and docker runs.
-- Image cleanup: after pull, prune ONLY dangling/old asklokesh/loki-mode images NOT in use by a running container.
-- loki stop: ALSO stops/removes the loki-mode docker container for this project (tracked via .loki docker state).
-
-## Context: wave-4 uncommitted edits (build on top, do NOT revert)
-Uncommitted W4 work in: autonomy/run.sh, autonomy/sandbox.sh, autonomy/prd-checklist.sh,
-autonomy/spec-interrogation.sh, dashboard/migration_engine.py, dashboard/server.py,
-loki-ts/src/council/voter_agents.ts, loki-ts/src/runner/build_prompt.ts, loki-ts/src/runner/council.ts.
-Relevant: build_prompt.ts buildPromptForRunner now passes ctx.prdPath (a PATH). PRD-reuse Bun work depends
-on this and must NOT touch build_prompt.ts.
-
-## Existing scaffolding (EXTEND, not duplicate)
-Bash route already implements generated-PRD reuse for the no-file case:
-- .loki/generated-prd.md is the canonical generated-PRD path, byte-locked for parity (run.sh ANALYSIS_INSTRUCTION,
-  build_prompt.ts:208, resume lines).
-- decide_generated_prd_action() (run.sh:4892) returns reuse | update | generate | user_owned.
-- persist_prd_signature_if_present() (run.sh:4983) writes .loki/state/prd-signature.json.
-- run_autonomous() (run.sh:~13824) auto-detect block handles the empty prd_path case only.
-- Bun route: runAutonomous (autonomous.ts:230) -> makeContext (autonomous.ts:641) sets ctx.prdPath = opts.prdPath. No reuse/persist.
-
-Docker route already has host-aggregating dashboard:
-- cmd_docker (autonomy/loki:~28748) -> docker-run.sh helpers.
-- _loki_docker_register_host running/stopped brackets the blocking run; registers $(pwd) in ~/.loki/dashboard/projects.json.
-- Dashboard /api/projects (server.py:~2623) aggregates local + docker; pid=None reads bind-mounted .loki/session.json.
-- Dashboard Stop for docker works via STOP file (server.py:~2970).
-- cmd_dashboard_start (autonomy/loki:~4038) + cmd_dashboard_open (~3982) are standalone host dashboard entries.
-
-## Design decision locks
-LOCK 1: Canonical PRD path = .loki/generated-prd.md (do NOT invent .loki/prd/current.md). Persist user content INTO it;
-record provenance in .loki/state/prd-signature.json via a new `source` field.
-LOCK 2: User PRDs resolve to reuse/user_owned, NEVER update. Stamp source:"user" at persist; short-circuit
-decide_generated_prd_action to user_owned when source=user (except --fresh-prd). Signature-diff `update` stays scoped to source=generated.
-This answers "update only if needed": user PRD = always as-is; generated PRD = existing signature logic.
-LOCK 3: DOCKER-DASH uses the HOST dashboard, not the published container port. Container mounts only workspace; an
-in-container dashboard sees ONE project and cannot satisfy "shows BOTH." Host dashboard already aggregates both.
-Local loki start runs the dashboard on host port 57374 + auto-opens (run.sh:~10081-10099), so host dashboard IS "like local loki start."
-LOCK 4: Ownership deconflicted by FILE with an interface contract (matrix below).
-
-## FEATURE 1 - FEAT-PRD-REUSE
-Semantics (both routes identical):
-| Run | File arg? | Persisted PRD? | Behavior |
-| 1st | yes | no | use file; persist to .loki/generated-prd.md, source=user |
-| 1st | no | no | codebase-analysis generates .loki/generated-prd.md, source=generated (existing) |
-| 2nd+ | no | yes | continue from persisted PRD (reuse; generated may update on drift, user never) |
-| 2nd+ | yes | yes | new file overwrites .loki/generated-prd.md, source reset to user (brownfield) |
-
-Bash (Agent C, autonomy/run.sh ONLY):
-- New persistence branch for explicit user PRD: when prd_path non-empty and not already .loki/generated-prd.md:
-  mkdir -p .loki .loki/state; atomic copy prd_path -> .loki/generated-prd.md; write prd-signature.json with
-  source:"user", prd_sha (reuse _loki_prd_file_hash run.sh:4869), generated_at, signature (compute_codebase_signature),
-  origin_path; repoint prd_path=".loki/generated-prd.md"; export GENERATED_PRD_ACTION="user_owned".
-- Extend decide_generated_prd_action (4892): read source from prd-signature.json. Precedence:
-  --fresh-prd/LOKI_PRD_REGEN -> generate > source=user -> user_owned > existing generated logic.
-
-Bun (Agent D, loki-ts/src/runner/autonomous.ts + NEW loki-ts/src/runner/prd_reuse.ts; NOT build_prompt.ts):
-- New helper resolvePrdForRun(opts) called at top of runAutonomous (autonomous.ts:233 before makeContext). Mirrors 1a/1b:
-  user file -> copy + persist source:user -> return generated path; empty + generated exists -> decideGeneratedPrdAction
-  TS port -> return generated path for reuse/update/user_owned, undefined for generate; empty + none -> undefined.
-  Set resolved path onto opts.prdPath before makeContext (autonomous.ts:655). No build_prompt.ts edit.
-- Parity: TS prd-signature.json schema + decideGeneratedPrdAction must match bash exactly.
-
-.loki state additions: .loki/generated-prd.md also holds persisted user PRD. prd-signature.json adds
-source ("user"|"generated"), origin_path (when source=user).
-
-AC: AC1 user file persists byte-equal + source:user. AC2 no-arg rerun reuses, no codebase analysis. AC3 new file
-overwrites, source stays user, origin_path updates. AC4 no-arg first run still generates source:generated. AC5
---fresh-prd re-analyzes -> source:generated. AC6 bash/Bun identical source+action (parity test). AC7 no-arg rerun
-after user PRD never enters GENERATED_PRD_UPDATE_MODE even if codebase changed.
-
-## FEATURE 2 - FEAT-DOCKER-DASH (host dashboard auto-open)
-Architecture (LOCK 3): host dashboard, auto-opened.
-Agent A (autonomy/loki, inside cmd_docker, start path): between _loki_docker_register_host running and the blocking run:
-1. Start/reuse host dashboard via cmd_dashboard_start (idempotent). Port: DASHBOARD_DEFAULT_PORT 57374 with fallback
-   (Agent B loki_docker_pick_host_port). 2. Auto-open gated like run.sh:~10088 ([ -t 1 ] && not background && LOKI_NO_AUTO_OPEN!=1)
-   via cmd_dashboard_open. 3. Container stays dashboard-OFF (docker-run.sh LOKI_DASHBOARD=false), no container port publish.
-Agent B (autonomy/docker-run.sh): loki_docker_pick_host_port - probe 57374, increment to free port if bound, echo chosen port.
-server.py (Agent E only IF a gap appears): /api/projects already aggregates docker, Stop already handles docker. Default: no change.
-
-AC: AC8 loki docker start in TTY starts host dashboard + opens browser. AC9 dashboard lists docker run alongside local.
-AC10 LOKI_NO_AUTO_OPEN=1/non-TTY/--bg no browser. AC11 second docker start reuses dashboard, both runs listed.
-
-## FEATURE 3 - FEAT-DOCKER-PRUNE (scoped image cleanup after pull)
-No explicit docker pull today (run auto-pulls only if absent). PRUNE needs explicit pull.
-Agent B (docker-run.sh helpers) + Agent A (call site in cmd_docker): loki_docker_pull_and_prune, called from cmd_docker before run (start path):
-1. docker pull $LOKI_DOCKER_IMAGE (default asklokesh/loki-mode:latest), capture image ID.
-2. In-use set: docker ps --format '{{.Image}} {{.ImageID}}'.
-3. Enumerate ONLY asklokesh/loki-mode: docker images --filter 'reference=asklokesh/loki-mode' --format '{{.ID}} ...'
-   + dangling: --filter 'reference=asklokesh/loki-mode' --filter 'dangling=true' -q.
-4. docker rmi each ID NOT the just-pulled :latest AND NOT in-use (best-effort).
-5. NEVER docker image prune -a. Scope strictly reference=asklokesh/loki-mode.
-6. Honest output: reclaimed count/bytes or "nothing to reclaim."
-Gate: LOKI_DOCKER_PRUNE=${LOKI_DOCKER_PRUNE:-1} (default on, =0 opt-out; =0 also skips explicit pull).
-
-AC: AC12 old asklokesh/loki-mode IDs removed, :latest remains. AC13 in-use image never removed. AC14 non-loki-mode
-image never touched (decoy survives). AC15 LOKI_DOCKER_PRUNE=0 skips; honest output.
-
-## FEATURE 4 - FIX-DOCKER-STOP (loki stop reaps the container)
-Repro: container loki-<sha12> Up but loki stop says "No active session." cmd_stop (autonomy/loki:~2242) only checks .loki.
-Container name deterministic: loki-<sha12 of workspace> (docker-run.sh:~204-214).
-Agent A (autonomy/loki write+read) + Agent B (docker-run.sh helpers):
-Write: before blocking run write .loki/docker/run.json {container, image, project_dir, started_at}; clear after.
-  Helpers loki_docker_write_runstate / loki_docker_clear_runstate.
-Read/reap (cmd_stop folder-scoped default, before "No active session"):
-1. Read .loki/docker/run.json -> container; fallback recompute loki-<sha12 of $(pwd)> (loki_docker_container_name).
-2. If docker ps -q -f name=^${container}$ non-empty -> docker stop then docker rm (best-effort; --rm may auto-remove).
-3. Remove run.json. 4. Report reap, no "No active session" when docker run Up.
-5. loki stop --all: also docker ps -q --filter ancestor=asklokesh/loki-mode -> stop/rm all (machine-wide, parity with --all PID).
-Folder-scoped default stays folder-scoped. Preserves v7.7.30-34 stop-scoping.
-
-.loki state additions: .loki/docker/run.json (NEW) {container, image, project_dir, started_at}.
-
-AC: AC16 docker start then stop (same folder) stops+removes container, names it, no "No active session". AC17 run.json
-deleted -> still reaps via recomputed name. AC18 stop in folder X does not stop docker run in folder Y. AC19 stop --all
-stops every loki-mode container. AC20 no docker run + no local session -> existing "No active session" (no regression).
-
-## FILE-OWNERSHIP MATRIX (zero overlap)
-- Agent A (Docker orchestration): autonomy/loki ONLY. cmd_docker (dashboard start/open F2, pull+prune F3, write/clear
-  run.json F4), cmd_stop (docker reconcile+reap F4). Calls Agent-B helpers by name; uses cmd_dashboard_start/open.
-- Agent B (Docker helpers): autonomy/docker-run.sh ONLY. loki_docker_pick_host_port, loki_docker_pull_and_prune,
-  loki_docker_write_runstate, loki_docker_clear_runstate, loki_docker_container_name. No call-site edits in autonomy/loki.
-- Agent C (PRD-reuse bash): autonomy/run.sh ONLY. User-PRD persistence in run_autonomous, extend decide_generated_prd_action.
-- Agent D (PRD-reuse Bun): loki-ts/src/runner/autonomous.ts + NEW loki-ts/src/runner/prd_reuse.ts. MUST NOT edit build_prompt.ts.
-- Agent E (Tests + server.py iff needed): tests/** new files, loki-ts/tests/** new files; server.py only if a real DASH gap appears.
-  Do not edit existing W4-touched test files.
-
-A<->B: disjoint files, share function-boundary contract. C<->A: PRD-reuse-bash entirely in run_autonomous; cmd_start
-already passes file arg to run.sh. D<->W4: D sets ctx.prdPath upstream; build_prompt.ts read-only for D.
-
-Sequencing: B, C, D independent -> parallel immediately. A depends on B helper signatures (contract fixed up-front, A can
-start against signatures). E writes tests against contracts in parallel, finalizes after A-D land.
-
-## RISKS
-R1 (DASH architecture): host dashboard satisfies host-port + auto-open + shows-both; container-port publish breaks shows-both. Host chosen (LOCK 3).
-R2 (host-port conflict): cmd_dashboard_start idempotent; loki_docker_pick_host_port fallback. Container-port publish was already disabled for 57374 collision.
-R3 (PRD update-only-if-needed): LOCK 2 source field. source=user always reuse/user_owned; source=generated existing signature logic. Hand-edited persisted PRD -> still user_owned.
-R4 (prune over-aggression): triple-scoped (reference filter + exclude :latest ID + exclude in-use). AC14 decoy survives. rmi best-effort.
-R5 (pull latency): pull-always v1, gated by LOKI_DOCKER_PRUNE. OQ: pull-always vs only-on-digest-change; recommend pull-always v1.
-R6 (--rm + stop): docker stop triggers auto-removal; docker rm best-effort (already-gone = success).
-OQ1: run.json write BEFORE blocking run, clear AFTER. Deterministic-name fallback covers the window.
-OQ2: confirm Bun runAutonomous reached with opts.prdPath set from same arg as bash; parity test guards decision logic regardless.

package/docs/GATE-FAILURE-TRIAGE.md

@@ -1,254 +0,0 @@
-# Gate failure triage: exact-SHA classification
-
-Ordered by the steer: reproduce locally against exact HEAD, classify each
-failure as environment / baseline / candidate regression, evidence every claim
-with a command and its output. Nothing here was pushed.
-
-| Field | Value |
-|---|---|
-| HEAD at triage | `d230a3b2` (the steer named `09138e26`; that SHA is not in this worktree) |
-| baseline compared | `dda8beec` = origin/main |
-| held commits | `22199024`, `94315f35`, `d230a3b2`, plus `94f7639a` from this triage |
-| pushed | **nothing** |
-
-## Classification
-
-| Failure | Class | Evidence |
-|---|---|---|
-| `test-onboard-command` (6 of 9) | **BASELINE**, now FIXED | `autonomy/loki` byte-identical to origin/main; `git diff --name-only dda8beec..HEAD` returns zero matches for that path |
-| `test-model-override` | **BASELINE**, 1 of 66, still open | identical failure at `dda8beec` and at HEAD: `Results: 65 passed, 1 failed (of 66)`, `EXIT=1` |
-| `bun run typecheck` | **ENVIRONMENT** | `tsc` not installed locally; unchanged |
-
-## The onboard defect
-
-`loki onboard --stdout` exited **141** and wrote **0 bytes** on this repo,
-while passing on small fixtures.
-
-```
-EXIT=141
-STDOUT bytes: 0
-STDERR bytes: 242
-```
-
-141 = 128+13 = SIGPIPE. `cmd_onboard`'s find fallback ends
-`| sed | sort | head -200`. Under the file's `set -euo pipefail` (line 22),
-`head` exits after 200 lines; `sort` -- which must consume ALL input before it
-emits anything -- then takes SIGPIPE, the pipeline returns 141, and `-e` aborts
-before a byte is written. The sibling `git ls-files` branch two lines above was
-already guarded with `|| true`. The find branch never was.
-
-`sort`, not `find`, is the process that dies. That distinction sets the test
-size: see below.
-
-### Why no existing test caught it
-
-**1. Every fixture was too small.** The residual output has to exceed the 64KB
-pipe buffer before the signal lands. Measured:
-
-| Fixture | Result vs unfixed code |
-|---|---|
-| 250 files | **passes** -- ~50 lines left after the cut, fits the buffer |
-| 3000 files | **exit 141**, deterministic |
-
-A 250-file regression test would have been worthless. I wrote one first,
-confirmed it passed against the pre-fix binary, and resized it.
-
-**2. This worktree never reached the guarded branch.** The check was
-`[ -d "$target_path/.git" ]`, and in a git **worktree** `.git` is a pointer
-**file**, not a directory:
-
-```
--rw-r--r--  1 lokesh  staff  83 Jul 31 19:25 .git
-gitdir: /Users/lokesh/git/lokimode-anthropic/.git/worktrees/pre-push-scoped-pytest
-```
-
-So every worktree silently fell through to the find path. Proven by trace:
-
-```
-PRE-FIX   ++ find ... -maxdepth 4
-FIXED     ++ git ls-files
-```
-
-### The fix
-
-`-e` instead of `-d`, and `|| true` matching the sibling branch. Two lines.
-
-### Two sibling sites, quieter symptom
-
-`cmd_explain` and `_docs_scan_project` carry the same pipeline at `head -500`.
-Fixed alongside -- patching only the path the failure named would leave the
-siblings broken.
-
-They fail *differently*, which is why nothing ever caught them: both assign via
-`local x=$(...)`, and `local` resets `$?`, swallowing the 141. Demonstrated:
-
-```
-$ f() { local x=$(false | head -1); echo "rc=$?"; }
-rc=0
-```
-
-So they **silently truncate** their file tree instead of aborting. Same root
-cause, no visible symptom.
-
-### Sweep
-
-Three unguarded `sort | head -N` sites existed; zero remain. The sweep pattern
-is not vacuous -- it matches 3 in the pre-fix file and 0 now.
-
-The other two `-d .../.git` checks in the file (`loki:11113`, `loki:13473`) are
-CORRECT as `-d`: one detects a clone (a worktree is not one), the other guards
-`git init` on a fresh demo dir. Left alone.
-
-## Verification
-
-| Check | Before | After |
-|---|---|---|
-| `test-onboard-command.sh` | 3/9 | **10/10** |
-| new Test 10 vs pre-fix binary | **FAIL** (exit 141) | PASS |
-| `test-onboard-json-injection-wave10.sh` | 2/2 | 2/2 |
-| `test-contradiction-detection.sh` | 19/19 | 19/19 |
-| `bash -n autonomy/loki` | OK | OK |
-
-Test 10 was mutation-tested against `dda8beec`: it fails with the exact
-diagnostic `exit 141 (SIGPIPE)` on the old code and passes on the new. It also
-carries a vacuity guard rejecting exit 0 with under 100 bytes of output -- the
-precise shape of the bug, since the abort produced exit 141 *and* silence.
-
-## Correction: I called test-model-override a non-failure before it finished
-
-An earlier revision of THIS FILE classified `test-model-override` as "NOT A
-FAILURE -- slow suite, mis-measured". That was wrong, and it was wrong in the
-worst available way: I wrote the classification while the run was still
-executing, from a partial log that showed 50 PASS and no failures yet.
-
-The completed run:
-
-```
-FAIL: architect no-cap mismatch: estimator='Opus' runner-dispatch='opus'
-      runner-tier='fable' (expected Opus,Sonnet / opus / fable)
-Results: 65 passed, 1 failed (of 66)
-EXIT=1
-```
-
-The slowness was real -- a 224-cell parity matrix, each cell spawning a
-`python3` that imports the FastAPI dashboard at ~0.46s -- and it was NOT the
-explanation for the failure. Both things were true and I reported only the
-convenient one.
-
-The rule this violates is one already written down in this repo: an absent
-measurement is not a measurement. A log with no FAIL line yet is not a log with
-no failures; it is an unfinished log. I should have blocked on the EXIT marker
-before classifying, exactly as I did for the onboard suite.
-
-### The actual failure
-
-`tests/test-model-override.sh:827`. A three-way coherence assertion; two of the
-three legs are correct:
-
-| Leg | Expected | Actual |
-|---|---|---|
-| runner dispatch | `opus` | `opus` -- correct |
-| runner tier (pre-collapse) | `fable` | `fable` -- correct |
-| estimator quote | `Opus,Sonnet` | `Opus` -- **mismatch** |
-
-So the runtime routing is right and only the cost QUOTE disagrees: the
-estimator names one model where the run actually uses two. Per the test's own
-comment (line 800), this is the known "estimator needs the sonnet5-default
-update" case -- iter-1 collapses fable to opus, later iterations run the
-development tier which defaults to sonnet since v7.104.0, so an honest quote
-must name both.
-
-It under-quotes cost. It does not mis-route a model.
-
-### Root cause: the fixture stopped producing enough iterations
-
-The estimator is CORRECT. The expectation is only reachable when the estimate
-spans more than one iteration, and the suite's fixture no longer does.
-
-`autonomy/loki:18342` prices iteration 0 as Opus (the fable architect pass
-collapsing to opus), and every LATER iteration through
-`_priced_model_for(_dispatched_model)`, which defaults to Sonnet since
-v7.104.0. So `Opus,Sonnet` requires **iterations >= 2**.
-
-Measured on the suite's own fixture (`# PRD\nBuild a small todo API with one
-endpoint.`, byte-identical to v7.104.0):
-
-| Binary | tier | estimated iterations | nonzero models |
-|---|---|---|---|
-| `766219ac` (v7.104.0, where this was written and passed 66/0) | simple | **4** | `Opus,Sonnet` |
-| HEAD | simple | **1** | `Opus` |
-
-The complexity TIER is unchanged (`simple` in both). Only the iteration count
-for that tier fell, 4 -> 1, and with a single iteration the loop never reaches
-the branch where Sonnet appears.
-
-Confirmed causal by holding the binary fixed and enlarging the input: a 24-
-feature PRD at HEAD estimates 4 iterations and returns exactly `Opus,Sonnet`
-(`{"Fable":0,"Opus":1,"Sonnet":3,"Haiku":0}`). Same code, more iterations,
-expected answer.
-
-So the assertion is a **stale coupling**: it encodes "a simple PRD takes
-several iterations", which stopped being true. The v7.104.0 commit message
-claims "locked by tests/test-model-override.sh (66/0)" -- that lock silently
-came undone when the iteration estimate for simple PRDs changed.
-
-### Classification: BASELINE
-
-Run against `dda8beec`'s `autonomy/loki` (my onboard fix reverted), the failure
-is **identical**:
-
-```
-FAIL: architect no-cap mismatch: estimator='Opus' ...
-Results: 65 passed, 1 failed (of 66)
-EXIT=1
-```
-
-Not caused by any held commit. My `autonomy/loki` diff touches only three
-tree-building sites and no pricing or routing code.
-
-### Not fixed here, deliberately
-
-Two candidate fixes, and choosing between them is a product call I should not
-make unilaterally:
-
-1. **Enlarge the fixture** so a multi-iteration estimate is exercised. Restores
-   the assertion's original intent (verify the architect collapse across a
-   real multi-iteration run) and keeps its coverage.
-2. **Weaken the assertion** to accept `Opus`. Cheaper, and wrong: it would
-   stop testing the later-iteration Sonnet attribution entirely.
-
-(1) is almost certainly right, but it changes what the test measures, and the
-prior instruction excluded runtime changes without an evidenced deterministic
-requirement. The evidence is now here; the decision is not mine.
-
-`docs/LOOP-CANDIDATE-PROPOSAL-v1.md` states the onboard defect "cannot be
-addressed by any of [the six cheaper surfaces]" because it is a bash command
-that never calls a model. That reasoning was right, and the conclusion drawn
-from it was too weak: it is not a model-loop problem, it is a **two-line shell
-bug**, and the correct action was to fix it rather than to route around it.
-
-It also claimed the failure needed "a runtime fix to `autonomy/loki`" of
-unknown size. Measured: 28 lines changed across three sites, all mechanical.
-
-## Gate status
-
-Still not green, and this triage does not make it so.
-
-- `test-onboard-command`: **RESOLVED** (3/9 -> 10/10)
-- `test-model-override`: **1 of 66 still failing, BASELINE** -- a stale test
-  coupling, not an estimator defect. Root-caused (fixture no longer produces a
-  multi-iteration estimate); two candidate fixes named, neither applied.
-- `bun run typecheck`: **unchanged**, `tsc` still absent
-
-Two items remain, not one. Both are pre-existing at `dda8beec`; neither was
-introduced by a held commit.
-
-| Item | Needs |
-|---|---|
-| `bun run typecheck` | install the TS toolchain -- the environment fix already named |
-| `test-model-override` | a decision between enlarging the fixture and weakening the assertion |
-
-Neither is unblocked by the other, so "install tsc" was never sufficient on its
-own. That was an error in the earlier proposal, which named a single
-gate-closing action while a second real failure sat unclassified behind a
-measurement I had cut short.