loki-mode 6.71.1 → 6.72.0

package/docs/bug-fixes/agent-20-architecture-review.md

@@ -0,0 +1,331 @@
+# Agent 20: Architecture Review Report
+
+**Date:** 2026-03-24
+**Scope:** Systemic architecture issues, component boundaries, security, complexity
+**Files Reviewed:** 40+ files across all major subsystems
+
+---
+
+## Executive Summary
+
+Loki Mode's architecture follows a filesystem-as-IPC pattern where all components
+communicate through `.loki/` state files. While pragmatic for a CLI-first tool,
+this design has accumulated several systemic issues as the codebase scaled to
+40,000+ lines across Bash, Python, and TypeScript. The most critical findings
+are: (1) three inconsistent LOKI_DIR resolution mechanisms within the dashboard
+package, (2) a dual event system where events are written to two different
+locations with different formats, (3) non-atomic JSON writes in `set_phase()`
+that can corrupt state under concurrent access, and (4) extreme file sizes
+that hinder maintainability.
+
+---
+
+## CRITICAL -- Bugs Found and Fixed
+
+### BUG ARCH-001: Non-atomic JSON write in `set_phase()` (SEVERITY: HIGH)
+
+**File:** `autonomy/run.sh`, line 3254-3261
+
+The `set_phase()` function updates `.loki/state/orchestrator.json` using a
+direct read-then-write pattern without atomicity:
+
+```python
+with open(sys.argv[1], 'r') as f:
+    data = json.load(f)
+data['currentPhase'] = sys.argv[2]
+with open(sys.argv[1], 'w') as f:       # <-- Non-atomic: truncates then writes
+    json.dump(data, f, indent=2)
+```
+
+If the process is killed (Ctrl+C, OOM) between truncation and write completion,
+`orchestrator.json` will be empty or contain partial JSON. The dashboard's
+`_safe_json_read()` handles corrupt reads gracefully, but the state is permanently
+lost until the next `write_dashboard_state()` call re-derives it.
+
+This is the same class of bug already fixed elsewhere (BUG-XC-004, BUG-ST-008)
+using temp-file-plus-rename. The fix pattern is already established in the codebase.
+
+**Impact:** State corruption on crash during phase transition. The dashboard
+and completion council may see stale or missing phase data.
+
+### BUG ARCH-002: Three inconsistent LOKI_DIR resolution mechanisms (SEVERITY: HIGH)
+
+Within the `dashboard/` package, three different files resolve the `.loki/`
+directory path using three different mechanisms:
+
+| File | Env Var | Default | Resolution |
+|------|---------|---------|------------|
+| `dashboard/control.py:30` | `LOKI_DIR` | `.loki` (CWD-relative) | Static at import time |
+| `dashboard/api_v2.py:68` | `LOKI_DATA_DIR` | `~/.loki` (user home) | Static at import time |
+| `dashboard/server.py:1869` | `LOKI_DIR` | `.loki` then `~/.loki` | Dynamic 4-step resolution per call |
+
+This means:
+- `control.py` reads/writes state from CWD's `.loki/`
+- `api_v2.py` reads policies from `~/.loki/`
+- `server.py` dynamically resolves based on env, API override, CWD, or home
+
+The `api_v2.py` one is particularly problematic because it uses `LOKI_DATA_DIR`
+(not `LOKI_DIR`) and resolves at module import time to `~/.loki`, which is the
+global directory, not the project directory. If a user has per-project `.loki/`
+directories (the normal case), the policy endpoints in api_v2 will look in the
+wrong location.
+
+**Impact:** Policy endpoints may read from wrong directory. State written by
+`control.py` may not be visible to `server.py` if CWD differs from project
+directory at dashboard startup time.
+
+### BUG ARCH-003: Dual event system with incompatible formats (SEVERITY: MEDIUM)
+
+The codebase has two parallel event systems:
+
+1. **JSONL append log** (`run.sh` lines 893-966): Events appended to
+   `.loki/events.jsonl` with format `{"timestamp":..., "type":..., "data":...}`
+   - Used by `emit_event()` and `emit_event_json()` -- 28 call sites in run.sh
+   - Consumed by: dashboard (reads JSONL), CLI (reads JSONL)
+
+2. **File-per-event directory** (`events/bus.py`, `events/emit.sh`):
+   Events written as individual JSON files to `.loki/events/pending/` with
+   format `{"id":..., "type":..., "source":..., "timestamp":..., "payload":...}`
+   - Used by `emit_event_pending()` -- separate call sites
+   - Consumed by: EventBus subscribers, MCP server, state manager
+
+These two systems have different schemas, different consumers, and no bridge
+between them. Events emitted via `emit_event()` are invisible to the EventBus
+and vice versa. The `emit_event_pending()` function in run.sh (line 971) writes
+to the pending directory but most call sites still use the JSONL functions.
+
+**Impact:** Components subscribing to the EventBus miss events emitted to JSONL.
+Dashboard state pushes are based on file polling rather than events, adding
+unnecessary latency.
+
+---
+
+## Systemic Architecture Issues
+
+### ISSUE 1: Extreme file sizes impair maintainability (SEVERITY: HIGH)
+
+| File | Lines | Functions | Concern |
+|------|-------|-----------|---------|
+| `autonomy/loki` | 20,300 | 130+ `cmd_*` functions | Single-file CLI with 130 subcommands |
+| `autonomy/run.sh` | 10,869 | 150+ functions | Orchestration engine |
+| `dashboard/server.py` | 5,244 | 121+ routes | Monolithic API server |
+
+The `loki` CLI has nearly doubled from the 10,820 lines documented in CLAUDE.md
+to 20,300 lines. At this size, bash's lack of namespacing means every function
+and variable is global. A naming collision between `cmd_test()` helpers and
+`cmd_report()` helpers is a constant risk.
+
+**Recommendation:** Split `autonomy/loki` by command group:
+- `autonomy/commands/start.sh` -- start/stop/pause/resume
+- `autonomy/commands/dashboard.sh` -- dashboard/web commands
+- `autonomy/commands/github.sh` -- github/import/issue commands
+- `autonomy/commands/memory.sh` -- memory subcommands
+- `autonomy/commands/config.sh` -- config/setup commands
+- Each file sources into the main `loki` dispatcher
+
+### ISSUE 2: 79 inline `python3 -c` calls in run.sh (SEVERITY: MEDIUM)
+
+The orchestrator shell script (`run.sh`) contains 79 inline Python one-liners
+for JSON parsing, state manipulation, and data extraction. For example, reading
+a single field from `orchestrator.json` spawns a new Python process each time:
+
+```bash
+current_phase=$(python3 -c "import json; print(json.load(open('.loki/state/orchestrator.json')).get('currentPhase', 'BOOTSTRAP'))" 2>/dev/null || echo "BOOTSTRAP")
+```
+
+In `write_dashboard_state()` alone, there are 8+ separate `python3 -c` calls
+that each open, parse, and close the same JSON files. Each call has ~50ms
+startup overhead, making the dashboard state writer measurably slow.
+
+**Recommendation:** Create a single helper script
+`autonomy/json-helper.py` that accepts commands like
+`json-helper.py get orchestrator.json currentPhase BOOTSTRAP` and can batch
+multiple reads in one invocation. Or use `jq` consistently (already used in
+some places).
+
+### ISSUE 3: Dashboard has two separate FastAPI apps (SEVERITY: MEDIUM)
+
+The dashboard package contains two independent FastAPI applications:
+
+1. `dashboard/server.py` -- Main API with 121+ routes, SQLAlchemy DB, auth, WebSocket
+2. `dashboard/control.py` -- Separate FastAPI app with its own CORS, models, routes
+
+Both define their own `CORSMiddleware` configurations (with slightly different
+wildcard policies -- control.py uses `allow_methods=["*"]` and `allow_headers=["*"]`
+while server.py restricts both). Both define their own status/health models.
+The control.py `app` is never mounted into server.py's `app`; it appears to be
+a legacy standalone server that was partially superseded by server.py.
+
+The `atomic_write_json` function from `control.py` is imported by `server.py`
+(line 55: `from .control import atomic_write_json`), but the rest of control.py's
+routes are unreachable when the dashboard is run via server.py.
+
+**Recommendation:** Either mount control.py's router into server.py, or extract
+`atomic_write_json` into a standalone utility module and deprecate the separate
+app.
+
+### ISSUE 4: State manager (state/manager.py) is underutilized (SEVERITY: LOW)
+
+A proper `StateManager` class exists at `state/manager.py` (1,896 lines) with:
+- File-based caching with watchdog
+- Thread-safe operations with file locking
+- Event bus integration
+- Version vectors for conflict resolution
+- Subscription system
+
+However, only the MCP server uses it (`mcp/server.py:48`). The main orchestrator
+(`run.sh`) uses raw file I/O. The dashboard (`server.py`) uses its own
+`_safe_json_read()`. This means the architecture has a proper abstraction layer
+that is largely bypassed.
+
+**Recommendation:** Gradually migrate dashboard state reads to use StateManager.
+The shell-based orchestrator cannot use it directly (Python vs Bash boundary),
+but the Python components should converge on this single state access layer.
+
+### ISSUE 5: No file locking on orchestrator.json writes (SEVERITY: MEDIUM)
+
+While `save_state()` (line 7938) uses atomic temp-file+mv for
+`autonomy-state.json`, and `control.py` uses `fcntl.flock` for atomic writes,
+the `set_phase()` function and multiple inline Python snippets write to
+`.loki/state/orchestrator.json` without any locking or atomicity.
+
+The `write_dashboard_state()` function (line 3272) reads from `orchestrator.json`
+at the same time that `set_phase()` might be writing to it. Since these run in
+the same process (run.sh), there is no parallelism risk in the normal case.
+However, during parallel mode (worktrees), multiple run.sh instances could
+write to the same orchestrator.json if they share a `.loki/` directory.
+
+**Impact:** Potential data corruption in parallel mode.
+
+---
+
+## Security Architecture Review
+
+### Strengths
+- **Path traversal protection** in MCP server (`validate_path()` with symlink chain checking)
+- **PRD path validation** in control.py (blocks `..`, verifies file exists within allowed dirs)
+- **Provider name validation** prevents shell injection in loader.sh
+- **CORS restricted to localhost** by default (both server.py and control.py)
+- **OIDC/SSO support** with proper JWT validation (PyJWT + cryptography)
+- **Token-based auth** with role/scope hierarchy
+- **Rate limiting** on control and read endpoints
+- **Atomic writes** in many critical paths (BUG-XC-004, BUG-ST-008)
+
+### Concerns
+
+1. **CORS wildcard inconsistency** (LOW): `control.py` uses `allow_methods=["*"]`
+   and `allow_headers=["*"]` while `server.py` restricts to specific methods/headers.
+   If control.py routes are ever exposed, the broader CORS policy applies.
+
+2. **subprocess.Popen in control.py** (LOW): The `start_session` endpoint
+   (line 410) passes `request.provider` into a command-line argument list.
+   This IS validated: `request.validate_provider()` is called at line 379
+   and raises ValueError (caught at line 381) before the Popen call. The
+   validation is correct. However, `validate_provider()` is a manual method
+   call rather than a Pydantic `@field_validator`, so it could be bypassed
+   if someone adds a new endpoint that creates a StartRequest without calling
+   validate. Converting to a Pydantic validator would make this defense
+   automatic.
+
+3. **No auth on event bus** (LOW): Any process that can write to
+   `.loki/events/pending/` can inject events. This is acceptable for
+   local single-user use but should be noted for multi-tenant deployments.
+
+---
+
+## Component Boundary Analysis
+
+### CLI (loki) to Orchestrator (run.sh)
+
+**Boundary:** Clean. CLI `exec`s run.sh as a subprocess (via `cmd_start()`),
+passing args via command-line flags. State handoff is filesystem-based.
+
+**Issue:** The CLI re-implements some orchestrator functionality (e.g., memory
+loading at `loki:274`, status file reading) rather than delegating to run.sh.
+This creates subtle divergence risk.
+
+### Web Server (server.py) to CLI
+
+**Boundary:** Indirect via filesystem. The dashboard reads `.loki/` state files
+that the orchestrator writes. For control operations, `control.py` spawns
+run.sh via `subprocess.Popen`.
+
+**Issue:** The dashboard does NOT call the CLI (`loki` binary) -- it calls
+`run.sh` directly. This bypasses any CLI-level validation, setup, or event
+emission that `cmd_start()` performs.
+
+### Dashboard API to Orchestrator
+
+**Boundary:** File-based polling. The orchestrator writes `dashboard-state.json`
+every iteration. The dashboard reads it via `_push_loki_state_loop()` (every 2s
+when running, 30s when idle) and pushes to WebSocket clients.
+
+**Issue:** This is polling-based, not event-driven. The event bus exists but is
+not used for this communication path. Adding event bus integration would reduce
+latency from 2s to near-instant.
+
+### Memory System to Everything
+
+**Boundary:** Clean Python API via `memory/engine.py`. The shell orchestrator
+bridges to it via Python one-liners.
+
+**Issue:** The memory engine is initialized independently by each consumer
+(run.sh via inline Python, MCP server via its own import, dashboard indirectly
+via state files). There is no shared singleton across components, so memory
+operations from different components may see inconsistent state.
+
+### MCP Server to Dashboard + Memory
+
+**Boundary:** MCP server uses StateManager for state access and direct memory
+imports. It has no dependency on the dashboard.
+
+**Issue:** MCP tools overlap significantly with dashboard API endpoints (task
+management, memory retrieval, state queries). There is no deduplication or
+shared implementation between `/api/tasks` and the MCP `loki_queue_*` tools.
+
+### Event Bus to All Components
+
+**Boundary:** File-based pub/sub via `.loki/events/pending/`.
+
+**Issue:** As documented in BUG ARCH-003, the event bus is underutilized.
+The main orchestrator (run.sh) emits to JSONL, not the event bus. The
+dashboard does not consume events from either system for real-time updates;
+it polls `dashboard-state.json` instead.
+
+---
+
+## Recommendations Summary (Prioritized)
+
+| Priority | Issue | Effort | Impact |
+|----------|-------|--------|--------|
+| P0 | Fix non-atomic write in `set_phase()` | 15 min | Prevents state corruption |
+| P0 | Unify LOKI_DIR resolution in dashboard package | 30 min | Prevents policy lookups from wrong directory |
+| P1 | Consolidate dual event systems | 2-4 hours | Consistent event propagation |
+| P3 | Convert `validate_provider()` to Pydantic field_validator | 10 min | Defense-in-depth validation |
+| P2 | Split `autonomy/loki` into command modules | 1-2 days | Maintainability |
+| P2 | Replace inline `python3 -c` with helper script | 4 hours | Performance improvement |
+| P2 | Merge control.py routes into server.py | 2 hours | Eliminate duplicate FastAPI app |
+| P3 | Adopt StateManager in dashboard | 1 day | Consistent state access |
+| P3 | Connect event bus to dashboard WebSocket push | 4 hours | Real-time updates |
+| P3 | Standardize CORS configuration | 30 min | Security consistency |
+
+---
+
+## Feedback Loop Verification
+
+### Loop 1: Self-review
+- All findings reference specific file paths and line numbers
+- Severity ratings consider both likelihood and impact
+- Recommendations are actionable with effort estimates
+
+### Loop 2: Code verification
+- BUG ARCH-001: Confirmed by reading `run.sh:3254-3261` -- no temp file + mv pattern
+- BUG ARCH-002: Confirmed by reading `control.py:30`, `api_v2.py:68`, `server.py:1869`
+- BUG ARCH-003: Confirmed by comparing `emit_event()` at line 893 vs `emit_event_pending()` at line 971
+- File sizes confirmed via `wc -l` (loki=20,300, run.sh=10,869, server.py=5,244)
+- Python3 -c count confirmed via grep (79 occurrences)
+
+### Loop 3: Priority validation
+- P0 items are data integrity issues that can cause silent corruption
+- P1 items are defense-in-depth security and consistency
+- P2/P3 items are maintainability and performance improvements