loki-mode 11.3.0 → 11.3.2

package/docs/KNOWN-ISSUES.md

@@ -1,41 +1,17 @@
+# Known issues
 
-## doctor --json writes 108 bytes to stderr on a provider-less host
+Issues in the current release that are known and not yet fixed. Report new ones with `loki crash` or at https://github.com/asklokesh/loki-mode/issues.
 
-`bash autonomy/loki doctor --json` on a host with NO provider CLI emits:
+## `loki verify` exit codes differ from the target table in three cases
 
-```
-autonomy/loki: line 14066: required: command not found
-autonomy/loki: line 14066: status: command not found
-```
+| Input | Command | Exit today | Target |
+|---|---|---|---|
+| Empty diff (a git repo with no changes vs base) | `loki verify` | 1 (CONCERNS) | 3 |
+| Not a git directory | `loki verify` | 1 (CONCERNS) | 2 |
+| Unknown flag | `loki verify` with a flag it does not know | 3 | 64 |
 
-**Not user-affecting, and not a JSON defect.** stdout is 5176 bytes of valid
-JSON; only stderr is polluted. Every `doctor --json` assertion in the repo
-discards stderr (`tests/test-airgap-commands.sh:50`,
-`tests/test-doctor-ci-gateable.sh:46,84,86`).
-`tests/test-e2e-features.sh:104` was the lone outlier using `2>&1` and has been
-brought into line.
+Changing these is a breaking change for CI gates, so it waits for a planned release. Details: [exit-codes.md](exit-codes.md#known-gaps).
 
-**Attribution:** introduced this session. `git show 0416810d:autonomy/loki`
-(v9.48.2) emits 0 bytes on the same host; HEAD emits 108. The region was last
-touched by `56e03374 feat(doctor): show provider availability`.
-
-**Six hypotheses tested and REFUTED, so the next person does not repeat them:**
-
-1. Word-splitting of `$_provider_availability` at `:14066` -- the assignment is
-   correctly quoted; reproduced in isolation as OK.
-2. `provider_availability_json` emitting to stderr -- 0 bytes, clean 353-byte
-   JSON in isolation.
-3. Its callees `auto_detect_provider` / `check_provider_installed` -- both
-   silent.
-4. The JSON value's content -- `required` and `status` appear 0 times in it.
-5. The value being multi-line and breaking the env-assignment chain -- forcing
-   `LOKI_PROVIDER_AVAILABILITY=""` as a literal STILL reproduces.
-6. `render_provider_availability` -- silent in isolation, and disabling the call
-   still reproduces.
-
-**Also known:** `bash -n` parses the file clean; sourcing the file and calling
-`cmd_doctor_json` directly emits 0 bytes, while executing the file emits 108.
-That difference (streaming read vs single parse) is the strongest remaining
-lead. The words `required` and `status` are python identifiers at `:14085-14098`
-inside the `python3 -c "` string that opens at `:14069`.
+## `loki start` uses the older engine
 
+`loki "<task>"`, `loki owner/repo#N` and `loki quick` run the Loki 10 engine. `loki start ./prd.md` still routes to the older engine, which is being removed. Prefer `loki owner/repo#N` or `loki "<task>"`.

package/docs/OTEL-COLLECTOR.md

@@ -0,0 +1,24 @@
+# Loki traces in Grafana, Honeycomb and Datadog
+
+Loki exports OTLP/HTTP JSON traces to `LOKI_OTEL_ENDPOINT` (path `/v1/traces`). With `LOKI_OTEL_GENAI=1` the engine10 events map to `gen_ai` spans: `invoke_agent loki` for the run, `stage <name>` per stage, `execute_tool <name>` per tool call. Prompt and completion content never reach a span, and unknown token usage is absent, not 0.
+
+The collector config `config/otel-collector/loki-collector.yaml` fans those spans out to three backends.
+
+## Use it
+
+1. Set the credentials for the vendors you use (the config reads them from the environment, nothing is stored in the file):
+   - Grafana Cloud: `GRAFANA_OTLP_ENDPOINT`, `GRAFANA_OTLP_AUTH` (the full header value, `Basic <base64 instanceId:token>`).
+   - Honeycomb: `HONEYCOMB_API_KEY` (ingest key).
+   - Datadog: `DD_API_KEY`, `DD_SITE` (for example `datadoghq.com`). Needs a collector build that includes the Datadog exporter (collector-contrib).
+2. Start the collector: `otelcol-contrib --config config/otel-collector/loki-collector.yaml`.
+3. Point Loki at it: `LOKI_OTEL_ENDPOINT=http://localhost:4318 LOKI_OTEL_GENAI=1 loki start ...`.
+
+To use a subset of vendors, edit only `service.pipelines.traces.exporters` and list the vendors you use. The collector validates every defined exporter even when it is not listed, so the required fields of the other vendors carry inert defaults (`https://unset.invalid` for the Grafana endpoint, `unset` for the Datadog key, `datadoghq.com` for the Datadog site). Those defaults are never used by a vendor that is not in the list, and are not credentials. A listed vendor with its variables unset sends to the inert default and fails at export time, so set the variables for every vendor you list.
+
+## Receipt link
+
+When tracing was on, the receipt carries a signed `trace_id` (see `docs/AGENT-CHANGE-RECEIPT.md`). Search for that id in the backend to open the run's trace.
+
+## Offline check
+
+`bash tests/test-otel-collector-config.sh` parses the config, asserts the receiver, processor, exporters and traces pipeline, asserts every credential is an `${env:NAME}` placeholder (an optional `:-default` is allowed and must not be key-shaped), asserts that every required field still resolves to a non-empty default when a vendor's variables are unset, and maps the deterministic fixture in `tests/fixtures/otel/` (fixed run id and timestamps) to the golden span list. It makes no network call. When `otelcol-contrib` is on PATH it also runs `otelcol-contrib validate` with only `HONEYCOMB_API_KEY` set; that step is skipped, not passed, when the binary is absent. It does not contact a vendor; import into each vendor UI is not tested.

package/docs/ROUTING-EVIDENCE.md

@@ -0,0 +1,32 @@
+# Routing evidence
+
+Measurements cited by provider and routing code. Each entry records the number,
+its source and the date it was taken. Code comments point here instead of
+carrying the figures.
+
+## Cost parity: MiniMax M2.5 vs Claude Opus 4.6
+
+Status: a SINGLE internal run, not re-measured. Do not use it to drive routing
+or in any external claim until it is re-derived (issue 210).
+
+- Benchmark: SWE-bench, uniform scaffold.
+- Harness: mini-swe-agent v2.0.0.
+- Date: 2026-02-17.
+- MiniMax M2.5: resolved 75.8 at $36.64 total.
+- Claude Opus 4.6: resolved 75.6 at $275.76 total.
+- Derived: an open model matching frontier accuracy at ~7.5x lower cost.
+- Source: cited from the cost-context comment that previously lived in
+  providers/opencode.sh; competitive context in docs/COMPETITIVE-INTEL-2026-09.md
+  item 3.
+
+## opencode viability
+
+Date verified: 2026-07-29/30.
+
+- Source opencode.ai/docs/providers documents 75+ providers and custom
+  OpenAI-compatible endpoints.
+- Source anomalyco/opencode: 190,871 stars, release v1.18.9 (2026-07-28),
+  pushed the same week.
+- Source aider: last release v0.86.0 from 2025-08-09.
+- Roo Code is archived; Continue is read-only.
+- CLI surface checked against the installed opencode v1.17.9.

package/docs/TESTING.md

@@ -0,0 +1,161 @@
+# Testing
+
+This document describes the test strategy, test types, and how to run and configure tests for Loki Mode.
+
+Loki Mode is a polyglot codebase. Tests span three languages and runtimes:
+
+- Bash (orchestration engine, CLI, councils, hooks)
+- Python (dashboard API, memory system, benchmarks, MCP server)
+- TypeScript / JavaScript (the Bun runner under `loki-ts/`, the Deno/Node API under `api/`, dashboard UI checks)
+
+Because the product's central promise is verified completion ("Loki does not lie about done"), the test suite is treated as a trust layer, not a formality. Trust-surface code (completion councils, quality gates, evidence receipts, the Bun runner, memory integrity) is held to a higher bar: mutation testing, parity checks, and adversarial detectors back the standard unit and integration tests.
+
+## Test Strategy
+
+1. Test the trust surface hardest. Completion detection, override councils, the verified-completion evidence gate, and the RARV-C closure loop are the highest-risk paths. They get unit tests plus mutation testing plus cross-route parity checks.
+2. Cross-route parity is a first-class concern. Behavior must match between the legacy Bash route and the Bun (`loki-ts`) route. Parity is checked by `tests/test-bash-bun-parity.sh` and the per-command parity suites under `tests/` (for example, doctor-output text drift); there is no continuous parity workflow.
+3. Prefer real verification over mocked assertions. Dedicated detectors (`tests/detect-mock-problems.sh`, `tests/detect-semantic-test-problems.sh`, `tests/detect-test-mutations.sh`, `tests/detect-invariant-violations.sh`) flag tests that pass without exercising real logic.
+4. Local CI is the canonical pre-push gate. `scripts/local-ci.sh` mirrors the GitHub Actions workflows so failures are discovered on the developer machine, not after push. GitHub Actions is the post-push verifier.
+5. Run tests in matrices. Python runs across 3.10 through 3.13; the Node suite runs across the supported Node versions. Version-specific breakage (for example, a Python 3.13-only inode race) must be caught by the matrix.
+
+## Test Types
+
+### Unit tests
+
+- Bash unit tests live under `tests/` (for example `tests/cli/`, `tests/council/`, `tests/memory/`) and are orchestrated by `tests/run-all-tests.sh`. They cover bootstrap, task queue, circuit breaker, agent timeout, state recovery, and the memory engine, among others. Several use function extraction: a helper is sourced out of `run.sh` and exercised in isolation. When a helper gains a new dependency, the extraction test must extract that dependency too, or the logic is silently skipped.
+- Python unit tests are `tests/test_*.py` files (120+ modules) covering the classifier, composer, calibration, embeddings edge cases, event bus, compose port selection, app-runner PID recycling, and more. They run under `pytest`.
+- TypeScript unit tests live under `loki-ts/tests/` (70+ `*.test.ts` files) and run under `bun test`. They cover the runner internals: state, prompt building, providers, budget, checkpoint, and shell utilities.
+- Node/Deno unit tests are `*_test.ts` and `*.test.js` files under `api/` and `tests/` (protocols, observability, policies, audit, integrations), run with `node --test`.
+
+### Integration tests
+
+- `tests/integration/` holds the integration suite, run via `bash tests/integration/run_integration_suite.sh` (also exposed as `npm run test:integration`).
+- `tests/integrations/` covers external system integrations (GitHub, Jira, Slack), run with `node --test`.
+- Managed-memory integration is gated behind flags and covered by `tests/managed_memory/` (flag matrix, SDK isolation, kill switch, shadow-write, retrieve).
+
+### End-to-end tests
+
+- `tests/e2e/` and `tests/live/` contain end-to-end and live-path scenarios.
+- Control Plane UI E2E lives in `packages/control-plane/test/e2e/` (Playwright, headless, loopback stub API).
+- Docker E2E lives under `tests/docker/` with the `docker/Dockerfile.sandbox` image.
+- Post-release distribution E2E is run after a release ships: install from the npm tarball, pull the Docker image, and exercise both the Bun and legacy-Bash routes on each channel.
+
+### Specialized test layers
+
+- Mutation testing (`loki-ts`): Stryker mutates trust-surface modules (`src/runner/state.ts`, `build_prompt.ts`, `providers.ts`, `budget.ts`, `checkpoint.ts`, `src/util/shell.ts`). Config: `loki-ts/stryker.config.json`. Thresholds: break 50%, low 60%, high 80%.
+- Parity tests: `tests/test-bash-bun-parity.sh` and the per-command parity suites verify the Bun and Bash routes agree.
+- Integrity and quality detectors: shell scripts under `tests/` that detect mock-only tests, semantic test problems, test mutations, and invariant violations.
+
+## How to Run Tests
+
+### Quick start
+
+```bash
+# Full default suite (bash syntax checks + node --test suites + managed-memory)
+npm test
+
+# Pre-push gate: mirrors every CI workflow. Run this before every git push.
+bash scripts/local-ci.sh
+```
+
+If `local-ci.sh` reports "DO NOT PUSH", do not push. Fix the failures and re-run.
+
+### By language and layer
+
+```bash
+# Bash shell test suite
+bash tests/run-all-tests.sh
+
+# Python tests (all)
+python3 -m pytest
+
+# Python tests (single module)
+python3 -m pytest tests/test_classifier.py -v
+
+# TypeScript / Bun runner tests
+cd loki-ts && bun test
+
+# Bun runner type check
+cd loki-ts && bun run typecheck
+
+# Integration suite
+npm run test:integration
+# or
+bash tests/integration/run_integration_suite.sh
+
+# Control Plane tests
+cd packages/control-plane && bun test ./test/
+```
+
+### Mutation testing (local)
+
+```bash
+cd loki-ts && npx stryker run
+```
+
+### Syntax-only validation (fast smoke)
+
+```bash
+bash -n autonomy/run.sh
+bash -n autonomy/loki
+bash -n autonomy/completion-council.sh
+python3 -c "import ast, os; [ast.parse(open(f'dashboard/{f}').read()) for f in os.listdir('dashboard') if f.endswith('.py')]"
+```
+
+### Cleanup after test runs (mandatory)
+
+Spawned processes and temp files must be cleaned up after any run that starts a server or test runner:
+
+```bash
+lsof -ti:57374 | xargs kill -9 2>/dev/null || true
+pkill -f "loki-run-" 2>/dev/null || true
+rm -rf /tmp/loki-* /tmp/test-* /tmp/package /tmp/*.tgz 2>/dev/null || true
+
+# Verify
+ps -ef | grep -E "(loki|test)" | grep -v grep || echo "Clean"
+```
+
+Note: never run `rm -rf /tmp/loki-*` while a live `loki` run is in progress; the orchestrator stages its run script at `/tmp/loki-run-*.sh`. Check `pgrep -f loki-run-` first.
+
+## Test Configuration
+
+| Area | Tool | Config / entry point |
+|---|---|---|
+| Default test command | mixed (bash, node) | `package.json` `test` script |
+| Bash suite | bash | `tests/run-all-tests.sh` |
+| Python | pytest | invoked via `python3 -m pytest`; deps: `pytest`, `pytest-asyncio` |
+| Bun runner | bun test | `loki-ts/package.json`, `loki-ts/bunfig.toml` |
+| Node/Deno | `node --test` | `*_test.ts`, `*.test.js` |
+| Mutation | Stryker | `loki-ts/stryker.config.json` |
+| Control Plane UI E2E | Playwright | `packages/control-plane/test/e2e/` |
+| Pre-push gate | bash | `scripts/local-ci.sh` |
+
+Python test dependencies commonly required: `fastapi`, `httpx`, `pydantic`, `sqlalchemy[asyncio]`, `aiosqlite`, `uvicorn`. Install with pip before running the Python suite.
+
+## Coverage Goals
+
+- No CI workflow collects or enforces a numeric line-coverage percentage. `bun test --coverage` can be run locally in `loki-ts`.
+- Mutation-score thresholds for trust-surface modules: break at 50%, low at 60%, high at 80% (`loki-ts/stryker.config.json`).
+- The Bash, Python and Bun suites do not currently enforce a numeric coverage percentage. Coverage of trust-surface logic is asserted through targeted unit tests, parity checks, and the mock/semantic/mutation detectors rather than a global percentage.
+
+## CI Integration
+
+GitHub Actions workflows (under `.github/workflows/`) run on push and pull request:
+
+| Workflow | File | What it runs |
+|---|---|---|
+| Tests | `test.yml` | Node suite (`npm test`), Python suite (`pytest`) across 3.10-3.13, shell suite (`tests/run-all-tests.sh`), Helm lint |
+| Integrity audit | `integrity-audit.yml` | Repository and artifact integrity checks |
+| Post-release smoke | `post-release-smoke.yml` | Smoke tests against published artifacts after a release |
+| Security audit | `security-audit.yml` | Dependency and security scanning |
+| SBOM | `sbom.yml` | Software bill of materials generation |
+
+### Local CI mirrors GitHub Actions
+
+`scripts/local-ci.sh` is the canonical pre-push gate. It mirrors every GitHub Actions workflow: bun typecheck/test, the full `tests/run-all-tests.sh` shell suite (matching CI exactly, not a cherry-picked subset), npm pack contents, SBOM, license audit, npm audit, shellcheck, YAML parse, the no-emoji check, the no-`git add -A` check, and a cleanup probe.
+
+Run it before every push. The Mac (developer machine) is the discovery channel; GitHub Actions is the post-push verifier. Distinguish real failures from local-environment false alarms (for example a missing `pytest-asyncio` in a Homebrew Python) before acting on a red result.
+
+### Release-time validation
+
+Before a release is committed, the npm tarball is verified to contain the built dashboard and web-app artifacts, and a fresh global install is exercised end to end (`loki --version`, `loki web`, `/api/status`). After a release ships, post-release distribution validation runs across npm, Docker, and Homebrew on both the Bun and legacy-Bash routes.

package/docs/air-gapped.md

@@ -90,8 +90,7 @@ your network, but the audit still counts it as required egress.
 Telemetry is ON by default for an individual interactive install and auto-off
 in CI, non-interactive, `LOKI_ENTERPRISE=true` and `LOKI_AIRGAP=true`
 contexts. The audit shows its real state. Every opt-out wins (`DO_NOT_TRACK=1`,
-`LOKI_TELEMETRY=off`, `loki telemetry off`). The adoption instrumentation added
-in v8.6.0 requires a second explicit opt-in on top of that -- see
+`LOKI_TELEMETRY=off`, `loki telemetry off`). The adoption instrumentation requires a second explicit opt-in on top of that -- see
 [PRIVACY.md](./PRIVACY.md).
 
 ## Why `unknown` is the right answer offline

package/docs/alternative-installations.md

@@ -1,6 +1,6 @@
 # Alternative Installation Methods
 
-The recommended installation is via npm or Homebrew (see [README](../README.md#get-started-in-30-seconds)). These alternatives serve specific use cases.
+The recommended installation is via npm or Homebrew (see [INSTALLATION.md](INSTALLATION.md)). These alternatives serve specific use cases.
 
 ---
 
@@ -16,7 +16,7 @@ git clone https://github.com/asklokesh/loki-mode.git ~/.claude/skills/loki-mode
 ln -sf ~/.claude/skills/loki-mode ~/.codex/skills/loki-mode
 
 # Optionally add CLI to PATH:
-ln -sf ~/.claude/skills/loki-mode/autonomy/loki /usr/local/bin/loki
+ln -sf ~/.claude/skills/loki-mode/bin/loki /usr/local/bin/loki
 ```
 
 **Update:** `cd ~/.claude/skills/loki-mode && git pull`
@@ -27,21 +27,21 @@ ln -sf ~/.claude/skills/loki-mode/autonomy/loki /usr/local/bin/loki
 
 ## Docker
 
-**Status:** Image exists on Docker Hub. Tags: `latest`, version-specific (e.g., `9.8.0`).
+**Status:** Image exists on Docker Hub. Tags: `latest`, one per release version (for example `X.Y.Z`, matching the npm version).
 
 ```bash
 docker pull asklokesh/loki-mode:latest
 ```
 
-**Limitation:** Claude Code is an interactive CLI that requires API keys and terminal access. Running it inside a Docker container is not the standard workflow. Docker is useful for:
+**Limitation:** The image ships the Claude Code CLI. Docker is useful for:
 
 - CI/CD sandbox execution (running `loki` in isolated environments)
 - Testing Loki Mode without modifying your local system
 - Air-gapped environments with pre-built images
 
-**Not recommended for:** Interactive Claude Code sessions. Use npm or Homebrew instead.
+`loki docker <command>` runs any loki command in the image with the current folder mounted, so `.loki/` state and resume behave like the local CLI.
 
-See [DOCKER_README.md](../DOCKER_README.md) for Docker-specific usage instructions.
+See [DOCKER.md](DOCKER.md) for Docker-specific usage instructions.
 
 ---
 
@@ -63,7 +63,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: asklokesh/loki-mode@v8.0.0
+      - uses: asklokesh/loki-mode@vX.Y.Z   # pin a release tag
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           mode: review
@@ -75,7 +75,7 @@ jobs:
 ```
 
 **Prerequisites:**
-- API key for your provider (set as repository secret): `ANTHROPIC_API_KEY` (Claude, recommended) or `OPENAI_API_KEY` (Codex). `GOOGLE_API_KEY` was for the Gemini provider, which is DEPRECATED as of v7.5.18.
+- API key for your provider (set as repository secret): `ANTHROPIC_API_KEY` (Claude, recommended) or `OPENAI_API_KEY` (Codex).
 - The action auto-installs `loki-mode` and `@anthropic-ai/claude-code`
 
 **Action Inputs:**
@@ -83,13 +83,13 @@ jobs:
 | Input | Default | Description |
 |-------|---------|-------------|
 | `mode` | `review` | `review`, `fix`, or `test` |
-| `provider` | `claude` | `claude` (recommended), `cline`, `codex`, `aider`, or `opencode`. `gemini` was REMOVED in v7.5.18. |
+| `provider` | `claude` | `claude` (recommended), `cline`, `codex`, `aider`, or `opencode`. |
 | `budget_limit` | `5.00` | Max cost in USD (passed to the Loki 10 engine as `--max-cost`) |
 | `max_iterations` | `3` | Max RARV cycles |
 | `github_token` | (required) | GitHub token for PR comments |
 | `prd_file` | | Path to PRD file (for fix/test modes) |
 
-**Engine:** every mode runs the Loki 10 engine through `loki start "<task>" --no-pr --provider <name> --max-cost <usd>` (the same path as `loki start owner/repo#N`). The PRD file or PR diff is folded into the task text. The action never opens a PR itself (Rule of Two). A run that crosses the cap ends BUDGET_STOP (exit 3). The legacy `--simple` path is no longer used.
+**Engine:** every mode runs the Loki 10 engine through `loki start "<task>" --no-pr --provider <name> --max-cost <usd>` (the same path as `loki start owner/repo#N`). The PRD file or PR diff is folded into the task text. The action never opens a PR itself (Rule of Two). A run that crosses the cap ends BUDGET_STOP (exit 3).
 
 **Modes:**
 
@@ -108,20 +108,10 @@ jobs:
 **Status:** Working. Release assets available for each version.
 
 ```bash
-# Download and extract to skills directory
-curl -sL https://github.com/asklokesh/loki-mode/archive/refs/tags/v8.0.0.tar.gz | tar xz
-mv loki-mode-8.0.0 ~/.claude/skills/loki-mode
+# Download the latest release and extract to the skills directory
+VERSION=$(npm view loki-mode version)
+curl -sL https://github.com/asklokesh/loki-mode/archive/refs/tags/v${VERSION}.tar.gz | tar xz
+mv loki-mode-${VERSION} ~/.claude/skills/loki-mode
 ```
 
 **Best for:** Offline or air-gapped environments, pinned version deployments.
-
----
-
-## VS Code Extension (Deprecated)
-
-> **DEPRECATED as of v7.2.0.** The Loki Mode VS Code extension is no longer
-> maintained and is no longer published to the VS Code Marketplace. Use the
-> Control Plane instead (`loki dashboard`).
-> The `vscode-extension/` source remains in the repository for contributors who
-> want to build it locally. See
-> [INSTALLATION.md](INSTALLATION.md#vs-code-extension-deprecated) for details.