loki-mode 7.25.0 → 7.27.0

package/README.md

@@ -18,7 +18,7 @@
 
 ---
 
-> **How it works:** Drop a spec -- a PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief. Loki Mode classifies complexity (`run.sh:detect_complexity()`), assembles an agent team from 41 specialized types across 8 swarms, and runs autonomous RARV cycles (Reason - Act - Reflect - Verify, see `run.sh:run_autonomous()`) with 11 quality gates (see `skills/quality-gates.md`). Code is not "done" until it passes automated verification. Output is a Git repo with source, tests, configs, and audit logs.
+> **How it works:** Drop a spec -- a PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief. Loki Mode classifies complexity (`run.sh:detect_complexity()`), assembles an agent team from 41 specialized agent roles across 8 domains - prompt-defined specifications the orchestrator adopts per phase, with parallel review (blind council) and optional worktree streams on Claude Code, sequential on other providers - and runs autonomous RARV cycles (Reason - Act - Reflect - Verify, see `run.sh:run_autonomous()`) with 11 quality gates (see `skills/quality-gates.md`). Code is not "done" until it passes automated verification. Output is a Git repo with source, tests, configs, and audit logs.
 
 ---
 
@@ -26,13 +26,15 @@
 
 - **Spec-driven, autonomous, with a built-in trust layer** -- Hand Loki a spec, walk away, come back to working code with tests. The full RARV-C closure loop (Reason - Act - Reflect - Verify - Close) runs until the work is actually done, not just attempted. The verified-completion evidence gate (`skills/quality-gates.md`) refuses any "done" claim on an empty git diff against the run-start commit, and blocks completion when tests run red, so "complete" means proven, not promised.
 - **Production quality built in** -- 11 quality gates (`skills/quality-gates.md`), blind 3-reviewer code review (`run.sh:run_code_review()`), anti-sycophancy checks
+- **Standalone verification: `loki verify`** -- Run Loki's deterministic gates (build, tests, static analysis, secret scan, dependency audit) against any branch or PR diff, including code written by other agents or humans. CI-ready exit codes (0 VERIFIED, 1 CONCERNS, 2 BLOCKED), machine-readable evidence at `.loki/verify/evidence.json`. Inconclusive evidence is never reported as VERIFIED (v7.27.0).
 - **Live App Preview** -- The dashboard embeds the locally-running app in an iframe so you can interact with it immediately during a build. Use `loki preview` (alias `loki open`) to print the URL and open it in your browser. Local-first: no hosted service, no vendor lock (v7.24.0).
+- **Compose-first fullstack** -- When a spec needs more than one service (web + database + cache) Loki generates a 12-factor `docker-compose.yml` with healthchecks, `depends_on` wiring, env-var config, and a `.env.example`. The Live App Preview surfaces the web service URL (not a database port), and health reflects the web service's Docker healthcheck so a crashed app shows as crashed even when the database stays up. Single-service apps stay on a plain run command. All local-first, no hosted service (v7.26.0).
 - **Intelligent `loki start`** -- For interactive foreground runs the dashboard auto-opens in the browser (cross-platform; skipped in CI, SSH-without-TTY, and piped runs; opt out with `LOKI_NO_AUTO_OPEN=1`). The completion summary shows "Your app is live at <url>" so you know exactly where to try what Loki just built. The autonomous loop passes Claude Code's `--effort`, `--max-budget-usd`, and `--fallback-model` on every iteration (each gated on CLI support and individual opt-out env vars) for better long-run unattended execution (v7.25.0).
 - **Cross-project memory** -- Episodic/semantic/procedural memory with vector search; knowledge learned on one project surfaces on the next (v5.15.0+, see `memory/engine.py`)
 - **Self-hosted and private** -- Your keys, your infrastructure, no data leaves your network
 - **Legacy system healing** -- `loki heal` archaeology/stabilize/isolate/modernize/validate phases (v6.67.0, see `skills/healing.md`)
 - **MCP server** -- 34 tools (including ChromaDB code search) plus 3 resources and 2 prompts (`mcp/server.py`, with managed-memory and magic tools registered from `mcp/managed_tools.py` and `mcp/magic_tools.py`)
-- **Full-stack output** -- Source code, tests, Docker configs, CI/CD pipelines, audit logs
+- **Full-stack output** -- Source code, tests, Docker Compose stacks (multi-service with healthchecks), CI/CD pipelines, audit logs
 - **Provider-agnostic** -- runs on Claude, Codex, Cline, or Aider with automatic failover (`loki-ts/src/runner/providers.ts`); no vendor lock-in. Gemini CLI deprecated v7.5.18; Antigravity CLI coming soon.
 - **Open source** -- Free for personal, internal, and academic use.
 
@@ -46,7 +48,7 @@ Loki drives a coding agent CLI and orchestrates real builds, so it needs a few t
 
 Required:
 
-- An agent provider CLI (at least one): [Claude Code](https://docs.claude.com/en/docs/claude-code) (`claude`, Tier 1, recommended), or OpenAI Codex CLI (`codex`), Cline, or Aider.
+- An agent provider CLI: [Claude Code](https://docs.claude.com/en/docs/claude-code) (`claude`, Tier 1, recommended and E2E-verified - the provider Loki Mode is built for). Codex, Cline, and Aider are supported as experimental providers (wiring in place; not yet E2E-verified by us).
 - Python 3.10+ (`python3`) for the dashboard, memory system, and orchestration helpers.
 - Git 2.x (`git`) for checkpoints and worktrees.
 - `curl` for installation and network calls.
@@ -86,7 +88,7 @@ loki quick "build a landing page with a signup form"
 
 | Method | Command | Notes |
 |--------|---------|-------|
-| **Bun (recommended)** | `bun install -g loki-mode` | Fastest. v8 will be Bun-only. |
+| **Bun (recommended)** | `bun install -g loki-mode` | Fastest startup for CLI commands. |
 | **Homebrew** | `brew tap asklokesh/tap && brew install loki-mode` | Auto-installs Bun as a dep |
 | **Docker** | `docker pull asklokesh/loki-mode:7.7.31 && docker run --rm asklokesh/loki-mode:7.7.31 start prd.md` | Bun pre-installed in image |
 | **npm (compat)** | `npm install -g loki-mode` | Works without Bun (bash fallback). Migrate any time with `loki self-update --to bun`. |
@@ -107,7 +109,7 @@ See the [Installation Guide](docs/INSTALLATION.md) for the long form.
 
 ## Runtime Architecture
 
-Loki Mode is in a phased migration from a Bash-based runtime to a TypeScript/Bun runtime. The migration has merged to `main` and ships incrementally with each release.
+Loki Mode runs a dual runtime by deliberate design: the battle-tested Bash engine is the stable core (the autonomous loop, quality gates, and completion council stay on it; it receives bug fixes and hardening), and new product surfaces are built TypeScript/Bun-first as modules that wrap the engine rather than reimplement it. An earlier plan to make v8 Bun-only has been superseded by this stable-engine approach: rewriting the verified trust layer would risk the exact guarantees this product exists to provide, for no capability gain. Bash support is not going away.
 
 **What ships today:**
 
@@ -226,8 +228,8 @@ Every iteration: **Reason** (read state) - **Act** (execute, commit) - **Reflect
 </td>
 <td width="33%" valign="top">
 
-### 41 Agent Types
-8 swarms: engineering, operations, business, data, product, growth, review, orchestration. Auto-composed by PRD complexity.
+### 41 Agent Roles
+8 domains: engineering, operations, business, data, product, growth, review, orchestration. These are prompt-defined role specifications the orchestrator adopts per phase, auto-composed by PRD complexity; parallelism comes from the blind review council, the adversarial reviewer, and optional git-worktree streams on Claude Code, sequential on other providers.
 
 [Agent Types](references/agent-types.md)
 
@@ -330,14 +332,14 @@ Loki's autonomy and quality loop are the product; the underlying coding CLI is s
 
 | Provider | Status | Autonomous Flag | Parallel Agents | Install |
 |----------|--------|:-:|:-:|---------|
-| **Claude Code** | Active (Tier 1) | `--dangerously-skip-permissions` | Yes (10+) | `npm i -g @anthropic-ai/claude-code` |
-| **Codex CLI** | Active (Tier 3) | `--full-auto` | Sequential | `npm i -g @openai/codex` |
-| **Cline CLI** | Active (Tier 2) | `--auto-approve` | Sequential | `npm i -g @anthropic-ai/cline` |
-| **Aider** | Active (Tier 3) | `--yes-always` | Sequential | `pip install aider-chat` |
+| **Claude Code** | Active (Tier 1, E2E-verified) | `--dangerously-skip-permissions` | Yes (10+) | `npm i -g @anthropic-ai/claude-code` |
+| **Codex CLI** | Experimental (Tier 3) | `--full-auto --skip-git-repo-check` | Sequential | `npm i -g @openai/codex` |
+| **Cline CLI** | Experimental (Tier 2) | `-y` | Sequential | `npm i -g @anthropic-ai/cline` |
+| **Aider** | Experimental (Tier 3) | `--yes-always` | Sequential | `pip install aider-chat` |
 | **Google Gemini CLI** | DEPRECATED v7.5.18 | -- | -- | Upstream deprecated; runtime removed. `LOKI_PROVIDER=gemini` exits with migration message. |
 | **Anthropic Antigravity CLI** | Coming soon | -- | -- | Integration planned. |
 
-Claude gets full features (subagents, parallelization, MCP, Task tool). Other active providers run sequentially. Auto-failover switches providers when rate-limited. See [Provider Guide](skills/providers.md).
+Status legend: "E2E-verified" means we run real spec-to-code builds on it ourselves. Claude Code is the primary, fully supported provider and the one Loki Mode is built for; it gets full features (subagents, parallelization, MCP, Task tool). "Experimental" means the wiring is in place but we have not produced an end-to-end verified build ourselves; treat as community-tested. Experimental providers run sequentially. Auto-failover switches providers when rate-limited. See [Provider Guide](skills/providers.md).
 
 ---
 

package/SKILL.md

@@ -3,7 +3,7 @@ name: loki-mode
 description: Autonomous spec-driven build system with a built-in trust layer. It does not call work done until it is verified (RARV-C closure loop, 11 quality gates, completion council, verified-completion evidence gate). Triggers on "Loki Mode". Takes a spec (PRD, GitHub issue, OpenAPI doc, etc.) to deployed product with minimal human intervention. Provider-agnostic. Requires --dangerously-skip-permissions flag.
 ---
 
-# Loki Mode v7.25.0
+# Loki Mode v7.27.0
 
 **You are an autonomous agent. You make decisions. You do not ask questions. You do not stop.**
 
@@ -383,4 +383,4 @@ See `CHANGELOG.md` entries [7.5.7], [7.5.8], [7.5.13] for the per-fix list and r
 
 ---
 
-**v7.25.0 | [Autonomi](https://www.autonomi.dev/) flagship product | ~260 lines core**
+**v7.27.0 | [Autonomi](https://www.autonomi.dev/) flagship product | ~260 lines core**

package/VERSION

@@ -1 +1 @@
-7.25.0
+7.27.0

package/autonomy/app-runner.sh

@@ -43,6 +43,10 @@ _APP_RUNNER_PID=""
 _APP_RUNNER_URL=""
 _APP_RUNNER_IS_DOCKER=false
 _APP_RUNNER_DOCKER_CONTAINER=""
+# v7.26.0 (Phase 4): the identified primary web service of a compose project,
+# used for service-aware health checks and the preview URL. Empty for
+# non-compose runs or when identification falls back to legacy port parsing.
+_APP_RUNNER_WEB_SERVICE=""
 _APP_RUNNER_HAS_SETSID=false
 _APP_RUNNER_CRASH_COUNT=0
 _APP_RUNNER_RESTART_COUNT=0
@@ -146,6 +150,77 @@ _rotate_app_log() {
     fi
 }
 
+# Identify the primary web service of a docker compose project and its
+# published host port. Uses `docker compose config --format json` (fully
+# resolved: env-interpolated, overrides merged) parsed with python3, so we do
+# NOT hand-parse YAML. Precedence MATCHES the contract in COMPOSE_INSTRUCTION
+# (run.sh build_prompt): (1) label loki.primary=true, (2) service named
+# web/app, (3) service publishing a common web port, (4) first service with any
+# published port. Echoes "service_name|published_port" on success, nothing on
+# failure (caller falls back to legacy behavior). Never hard-fails.
+# v7.26.0 (Phase 4): fixes the multi-service URL/health gaps (GAP #1-4).
+_identify_compose_web_service() {
+    local base="${1:-${TARGET_DIR:-.}}"
+    local compose_dir
+    compose_dir=$(_app_runner_compose_dir "$base")
+    command -v docker >/dev/null 2>&1 || return 0
+    command -v python3 >/dev/null 2>&1 || return 0
+    local cfg
+    cfg=$(cd "$compose_dir" && docker compose config --format json 2>/dev/null) || return 0
+    [ -n "$cfg" ] || return 0
+    printf '%s' "$cfg" | python3 -c '
+import json, sys
+COMMON = ["3000", "8000", "8080", "5000", "4200", "5173", "80"]
+try:
+    d = json.load(sys.stdin)
+except Exception:
+    sys.exit(0)
+services = d.get("services", {})
+if not isinstance(services, dict) or not services:
+    sys.exit(0)
+
+def published_ports(svc):
+    out = []
+    for p in (svc.get("ports") or []):
+        if isinstance(p, dict):
+            pub = p.get("published")
+        else:
+            pub = None
+        if pub is not None and str(pub).strip():
+            out.append(str(pub).strip())
+    return out
+
+# (1) label loki.primary=true
+for name, svc in services.items():
+    labels = svc.get("labels") or {}
+    if isinstance(labels, list):
+        labels = dict(x.split("=", 1) for x in labels if "=" in x)
+    if str(labels.get("loki.primary", "")).lower() == "true":
+        pp = published_ports(svc)
+        if pp:
+            print(name + "|" + pp[0]); sys.exit(0)
+# (2) service named web/app
+for cand in ("web", "app"):
+    svc = services.get(cand)
+    if svc:
+        pp = published_ports(svc)
+        if pp:
+            print(cand + "|" + pp[0]); sys.exit(0)
+# (3) service publishing a common web port
+for name, svc in services.items():
+    pp = published_ports(svc)
+    for cp in COMMON:
+        if cp in pp:
+            print(name + "|" + cp); sys.exit(0)
+# (4) first service with any published port
+for name, svc in services.items():
+    pp = published_ports(svc)
+    if pp:
+        print(name + "|" + pp[0]); sys.exit(0)
+sys.exit(0)
+' 2>/dev/null || return 0
+}
+
 # Detect port from project files
 _detect_port() {
     local method="$1"
@@ -158,18 +233,34 @@ _detect_port() {
 
     case "$method" in
         *docker\ compose*)
-            # Parse port from compose file
-            local compose_file
-            if [ -f "${TARGET_DIR:-.}/docker-compose.yml" ]; then
-                compose_file="${TARGET_DIR:-.}/docker-compose.yml"
+            # v7.26.0: identify the PRIMARY WEB service and ITS published port
+            # via docker compose config (resolved JSON), so the preview URL and
+            # health check target the web service, not whichever port (e.g. a
+            # db/cache) appears first in the file. Falls back to the legacy
+            # first-port grep when docker/python is unavailable or no web
+            # service is found.
+            local web_info web_port
+            web_info=$(_identify_compose_web_service "${TARGET_DIR:-.}")
+            if [ -n "$web_info" ]; then
+                _APP_RUNNER_WEB_SERVICE="${web_info%%|*}"
+                web_port="${web_info##*|}"
+            fi
+            if [ -n "${web_port:-}" ] && [[ "$web_port" =~ ^[0-9]+$ ]]; then
+                _APP_RUNNER_PORT="$web_port"
             else
-                compose_file="${TARGET_DIR:-.}/compose.yml"
+                # Legacy fallback: first published port from the compose file.
+                local compose_file
+                if [ -f "${TARGET_DIR:-.}/docker-compose.yml" ]; then
+                    compose_file="${TARGET_DIR:-.}/docker-compose.yml"
+                else
+                    compose_file="${TARGET_DIR:-.}/compose.yml"
+                fi
+                local port
+                # Handle both simple (HOST:CONTAINER) and IP-bound (IP:HOST:CONTAINER) port formats
+                # Also handle port ranges like "8080-8090:8080-8090" by taking the first port
+                port=$(grep -E '^\s*-\s*"?[0-9]' "$compose_file" 2>/dev/null | head -1 | sed 's/.*- *"*//;s/".*//;' | awk -F: '{print $(NF-1)}' | awk -F- '{print $1}')
+                _APP_RUNNER_PORT="${port:-8080}"
             fi
-            local port
-            # Handle both simple (HOST:CONTAINER) and IP-bound (IP:HOST:CONTAINER) port formats
-            # Also handle port ranges like "8080-8090:8080-8090" by taking the first port
-            port=$(grep -E '^\s*-\s*"?[0-9]' "$compose_file" 2>/dev/null | head -1 | sed 's/.*- *"*//;s/".*//;' | awk -F: '{print $(NF-1)}' | awk -F- '{print $1}')
-            _APP_RUNNER_PORT="${port:-8080}"
             ;;
         *docker\ build*)
             local port
@@ -694,14 +785,64 @@ app_runner_health_check() {
         # retries are handled in app_runner_start.
         local running_containers
         running_containers=$(LOKI_COMPOSE_HEALTH_TIMEOUT=1 _app_runner_compose_running_count "${TARGET_DIR:-.}")
-        if [ "${running_containers:-0}" -gt 0 ]; then
+        if [ "${running_containers:-0}" -le 0 ]; then
+            # Nothing running at all.
+            _write_health "false"
+            _write_app_state "crashed"
+            return 1
+        fi
+        # v7.26.0 (Phase 4) GAP #4 fix: "some container is up" is NOT health for
+        # a multi-service stack. If we identified a primary web service, health
+        # keys on THAT service, not on whether any container (e.g. a db/cache) is
+        # up. Two signals, in order: (1) the web service's docker HEALTHCHECK
+        # result when one is declared (COMPOSE_INSTRUCTION mandates an HTTP
+        # healthcheck on the web service) -- "healthy" means actually serving,
+        # "unhealthy"/"starting" do not; (2) when no healthcheck is declared,
+        # fall back to the container lifecycle State (running), matching the
+        # codebase convention. Reads both fields from one `docker compose ps`.
+        if [ -n "${_APP_RUNNER_WEB_SERVICE:-}" ]; then
+            local _web_line _web_state _web_health
+            _web_line=$(cd "$(_app_runner_compose_dir "${TARGET_DIR:-.}")" \
+                && docker compose ps --format '{{.Service}}|{{.State}}|{{.Health}}' 2>/dev/null \
+                | tr -d '\r' | awk -F'|' -v s="$_APP_RUNNER_WEB_SERVICE" '$1==s {print; exit}')
+            _web_state=$(printf '%s' "$_web_line" | awk -F'|' '{print $2}')
+            _web_health=$(printf '%s' "$_web_line" | awk -F'|' '{print $3}')
+            if [ "$_web_state" != "running" ]; then
+                # Container not running -> definitively down.
+                _write_health "false"
+                _write_app_state "crashed"
+                return 1
+            fi
+            if [ -n "$_web_health" ]; then
+                # A healthcheck is declared: it is authoritative.
+                if [ "$_web_health" = "healthy" ]; then
+                    _write_health "true"
+                    _write_app_state "running"
+                    return 0
+                fi
+                if [ "$_web_health" = "unhealthy" ]; then
+                    # Container up but failing its own healthcheck (not serving).
+                    _write_health "false"
+                    _write_app_state "crashed"
+                    return 1
+                fi
+                # "starting" (within start_period): up, not yet healthy. Report
+                # running so the watchdog gives it time instead of restarting,
+                # but do not yet claim a passing health.
+                _write_health "false"
+                _write_app_state "running"
+                return 0
+            fi
+            # No healthcheck declared: container running is the signal.
             _write_health "true"
             _write_app_state "running"
             return 0
-        else
-            _write_health "false"
-            return 1
         fi
+        # No web service identified (legacy/degraded): fall back to the
+        # original "any container running" signal.
+        _write_health "true"
+        _write_app_state "running"
+        return 0
     fi
 
     # Check PID is alive (non-docker-compose methods)
@@ -769,6 +910,36 @@ app_runner_should_restart() {
 app_runner_watchdog() {
     _app_runner_dir
 
+    # v7.26.0 (Phase 4): docker compose runs detached (`up -d` exits immediately),
+    # so the captured PID is a short-lived subshell and `kill -0` is the wrong
+    # liveness signal for a compose stack. For compose, delegate to
+    # app_runner_health_check, whose compose branch keys on the primary web
+    # SERVICE container running (GAP #4) and writes health.json + state.json.
+    # This is what makes the service-aware health logic actually fire in the
+    # live monitoring loop (not just in isolation). On an unhealthy web service
+    # it restarts the stack under the same crash-count circuit breaker.
+    if [ "$_APP_RUNNER_IS_DOCKER" = true ] && echo "$_APP_RUNNER_METHOD" | grep -q "docker compose"; then
+        if app_runner_health_check; then
+            return 0
+        fi
+        _APP_RUNNER_CRASH_COUNT=$(( _APP_RUNNER_CRASH_COUNT + 1 ))
+        log_warn "App Runner: compose web service unhealthy (crash #$_APP_RUNNER_CRASH_COUNT)"
+        if [ "$_APP_RUNNER_CRASH_COUNT" -ge 5 ]; then
+            log_error "App Runner: crash limit reached (5), marking as crashed"
+            tail -20 "$_APP_RUNNER_DIR/app.log" 2>/dev/null | while IFS= read -r line; do
+                log_error "  $line"
+            done
+            _write_app_state "crashed"
+            return 1
+        fi
+        local _c_backoff=$(( 1 << _APP_RUNNER_CRASH_COUNT ))
+        [ "$_c_backoff" -gt 30 ] && _c_backoff=30
+        log_info "App Runner: restarting compose stack in ${_c_backoff}s..."
+        sleep "$_c_backoff"
+        app_runner_start || log_warn "App Runner: compose auto-restart failed"
+        return 0
+    fi
+
     if [ -z "$_APP_RUNNER_PID" ] && [ -f "$_APP_RUNNER_DIR/app.pid" ]; then
         _APP_RUNNER_PID=$(cat "$_APP_RUNNER_DIR/app.pid" 2>/dev/null)
     fi

package/autonomy/completion-council.sh

@@ -752,6 +752,18 @@ with open(state_file, 'w') as f:
         "threshold=$effective_threshold" \
         "result=$([ $approve_count -ge $effective_threshold ] && echo 'APPROVED' || echo 'REJECTED')" 2>/dev/null || true
 
+    # Trust-metrics: durable per-vote record for the council rejection / split
+    # rate. The council state.json verdicts[] array is per-run only; this log is
+    # the cross-run corpus. Additive, best-effort, stdout-silent.
+    if type record_trust_event_bash &>/dev/null; then
+        record_trust_event_bash "council_vote" \
+            "approve=$approve_count" \
+            "reject=$reject_count" \
+            "threshold=$effective_threshold" \
+            "result=$([ $approve_count -ge $effective_threshold ] && echo 'APPROVED' || echo 'REJECTED')" \
+            >/dev/null 2>&1 || true
+    fi
+
     # Write transcript for this council round (Path A: council_vote path)
     local _ct_outcome
     _ct_outcome=$([ $approve_count -ge $effective_threshold ] && echo "APPROVED" || echo "REJECTED")
@@ -1366,6 +1378,19 @@ print(json.dumps(items[:5]))
 }
 EVIDENCE_EOF
     mv "$ev_tmp" "$ev_file"
+
+    # Trust-metrics: durable per-block record. evidence-block.json is a single
+    # state file that is DELETED the moment the gate next passes, so it cannot
+    # be the cross-run corpus for the block rate. Append an event here, where a
+    # block is definitely happening. Additive, best-effort, stdout-silent.
+    if type record_trust_event_bash &>/dev/null; then
+        record_trust_event_bash "evidence_block" \
+            "reason=$reason" \
+            "diff_ok=$diff_ok" \
+            "tests_ok=$tests_ok" \
+            >/dev/null 2>&1 || true
+    fi
+
     return 1
 }