loki-mode 6.71.1 → 6.72.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -1
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/autonomy/hooks/migration-hooks.sh +26 -0
- package/autonomy/loki +429 -92
- package/autonomy/run.sh +219 -38
- package/dashboard/__init__.py +1 -1
- package/dashboard/server.py +101 -19
- package/docs/INSTALLATION.md +20 -11
- package/docs/bug-fixes/agent-01-cli-fixes.md +101 -0
- package/docs/bug-fixes/agent-02-purplelab-fixes.md +88 -0
- package/docs/bug-fixes/agent-03-dashboard-fixes.md +119 -0
- package/docs/bug-fixes/agent-04-memory-fixes.md +105 -0
- package/docs/bug-fixes/agent-05-provider-fixes.md +86 -0
- package/docs/bug-fixes/agent-06-integration-fixes.md +101 -0
- package/docs/bug-fixes/agent-07-dash-run-fixes.md +101 -0
- package/docs/bug-fixes/agent-08-docker-fixes.md +164 -0
- package/docs/bug-fixes/agent-09-e2e-build-fixes.md +69 -0
- package/docs/bug-fixes/agent-10-e2e-fullstack-fixes.md +102 -0
- package/docs/bug-fixes/agent-11-e2e-session-fixes.md +70 -0
- package/docs/bug-fixes/agent-12-scenario-fixes.md +120 -0
- package/docs/bug-fixes/agent-13-enterprise-fixes.md +143 -0
- package/docs/bug-fixes/agent-14-uat-newuser-fixes.md +88 -0
- package/docs/bug-fixes/agent-15-uat-poweruser-fixes.md +132 -0
- package/docs/bug-fixes/agent-19-code-review.md +316 -0
- package/docs/bug-fixes/agent-20-architecture-review.md +331 -0
- package/docs/competitive/bolt-new-analysis.md +579 -0
- package/docs/competitive/emergence-others-analysis.md +605 -0
- package/docs/competitive/replit-lovable-analysis.md +622 -0
- package/docs/test-scenarios/edge-cases.md +813 -0
- package/docs/test-scenarios/enterprise-scenarios.md +732 -0
- package/mcp/__init__.py +1 -1
- package/mcp/server.py +49 -5
- package/memory/consolidation.py +33 -0
- package/memory/embeddings.py +10 -1
- package/memory/engine.py +83 -38
- package/memory/retrieval.py +36 -0
- package/memory/storage.py +56 -4
- package/memory/token_economics.py +14 -2
- package/memory/vector_index.py +36 -7
- package/package.json +1 -1
- package/providers/gemini.sh +89 -2
- package/templates/README.md +1 -1
- package/templates/cli-tool.md +30 -0
- package/templates/dashboard.md +4 -0
- package/templates/data-pipeline.md +4 -0
- package/templates/discord-bot.md +47 -0
- package/templates/game.md +4 -0
- package/templates/microservice.md +4 -0
- package/templates/npm-library.md +4 -0
- package/templates/rest-api-auth.md +50 -20
- package/templates/rest-api.md +15 -0
- package/templates/saas-starter.md +1 -1
- package/templates/slack-bot.md +36 -0
- package/templates/static-landing-page.md +9 -1
- package/templates/web-scraper.md +4 -0
- package/web-app/dist/assets/Badge-CeBkFjo6.js +1 -0
- package/web-app/dist/assets/Button-yuhqo8Fq.js +1 -0
- package/web-app/dist/assets/{Card-B1bV4syB.js → Card-BG17vsX0.js} +1 -1
- package/web-app/dist/assets/{HomePage-CZTV6Nea.js → HomePage-BMSQ7Apj.js} +3 -3
- package/web-app/dist/assets/{LoginPage-D4UdURJc.js → LoginPage-aH_6iolg.js} +1 -1
- package/web-app/dist/assets/{NotFoundPage-CCLSeL6j.js → NotFoundPage-Di8cNtB1.js} +1 -1
- package/web-app/dist/assets/ProjectPage-BtRssmw9.js +285 -0
- package/web-app/dist/assets/ProjectsPage-B-FTFagc.js +6 -0
- package/web-app/dist/assets/{SettingsPage-Xuv8EfAg.js → SettingsPage-DIJPBla4.js} +1 -1
- package/web-app/dist/assets/TeamsPage--19fNX7w.js +36 -0
- package/web-app/dist/assets/TemplatesPage-ChUQNOOv.js +11 -0
- package/web-app/dist/assets/TerminalOutput-Dwrzecyl.js +31 -0
- package/web-app/dist/assets/activity-BNRWeu9N.js +6 -0
- package/web-app/dist/assets/{arrow-left-CaGtolHc.js → arrow-left-Ce6g1_YE.js} +1 -1
- package/web-app/dist/assets/circle-alert-LIndawHL.js +11 -0
- package/web-app/dist/assets/clock-Bpj4VPlP.js +6 -0
- package/web-app/dist/assets/{external-link-CazyUyav.js → external-link-BhhdF0iQ.js} +1 -1
- package/web-app/dist/assets/folder-open-CM2LgfxI.js +11 -0
- package/web-app/dist/assets/index-8-KpWWq7.css +1 -0
- package/web-app/dist/assets/index-kPDW4e_b.js +236 -0
- package/web-app/dist/assets/lock-sAk3Xe54.js +16 -0
- package/web-app/dist/assets/search-CR-2i9by.js +6 -0
- package/web-app/dist/assets/server-DuFh4ymA.js +26 -0
- package/web-app/dist/assets/trash-2-BmkkT8V_.js +11 -0
- package/web-app/dist/index.html +2 -2
- package/web-app/server.py +1321 -53
- package/web-app/dist/assets/Badge-CBUx2PjL.js +0 -6
- package/web-app/dist/assets/Button-DsRiznlh.js +0 -21
- package/web-app/dist/assets/ProjectPage-D0w_X9tG.js +0 -237
- package/web-app/dist/assets/ProjectsPage-ByYxDlKC.js +0 -16
- package/web-app/dist/assets/TemplatesPage-BKWN07mc.js +0 -1
- package/web-app/dist/assets/TerminalOutput-Dj98V8Z-.js +0 -51
- package/web-app/dist/assets/clock-C_CDmobx.js +0 -11
- package/web-app/dist/assets/index-D452pFGl.css +0 -1
- package/web-app/dist/assets/index-Df4_kgLY.js +0 -196
|
@@ -0,0 +1,813 @@
|
|
|
1
|
+
# Loki Mode Edge Case Test Scenarios
|
|
2
|
+
|
|
3
|
+
Agent 12 - Scenario Writing: Edge Cases, Error Paths, Concurrent Usage
|
|
4
|
+
Version: v6.71.1 | Date: 2026-03-24
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Table of Contents
|
|
9
|
+
|
|
10
|
+
1. [Error Path Scenarios (EP-001 to EP-015)](#error-path-scenarios)
|
|
11
|
+
2. [Concurrent Usage Scenarios (CU-001 to CU-010)](#concurrent-usage-scenarios)
|
|
12
|
+
3. [Edge Case Scenarios (EC-001 to EC-015)](#edge-case-scenarios)
|
|
13
|
+
4. [Boundary Scenarios (BD-001 to BD-010)](#boundary-scenarios)
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Error Path Scenarios
|
|
18
|
+
|
|
19
|
+
### EP-001: Empty PRD File Input
|
|
20
|
+
|
|
21
|
+
**Given** a user has an empty file at `./empty.md` (0 bytes)
|
|
22
|
+
**When** they run `loki start ./empty.md`
|
|
23
|
+
**Then** the system should:
|
|
24
|
+
- Detect the PRD has 0 words in `detect_complexity()` (run.sh:1268)
|
|
25
|
+
- Classify complexity as "simple" (words < 200, features < 5, sections < 3)
|
|
26
|
+
- Enter codebase analysis mode since the PRD has no content
|
|
27
|
+
- Log a warning that the PRD file is empty and switch to analysis mode
|
|
28
|
+
|
|
29
|
+
**Expected behavior:** Graceful degradation to codebase analysis mode. No crash.
|
|
30
|
+
|
|
31
|
+
**Current risk:** `wc -w < "$prd_path"` returns 0, `grep -c` returns 0 -- both handle empty files correctly. The system proceeds but generates a trivial "simple" classification, which may be incorrect if the project itself is complex.
|
|
32
|
+
|
|
33
|
+
**Suggested improvement:** Add an explicit empty-file check before complexity detection.
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
### EP-002: Malformed JSON PRD Input
|
|
38
|
+
|
|
39
|
+
**Given** a user provides a PRD file `bad.json` containing `{invalid json`
|
|
40
|
+
**When** they run `loki start bad.json`
|
|
41
|
+
**Then** the system should:
|
|
42
|
+
- Attempt JSON parsing in `detect_complexity()` (run.sh:1276)
|
|
43
|
+
- The `jq` command fails silently (`2>/dev/null || echo "0"`)
|
|
44
|
+
- Fall back to `grep -c` which counts 0 features
|
|
45
|
+
- Classify as "simple" complexity
|
|
46
|
+
|
|
47
|
+
**Expected behavior:** No crash. Falls through to standard complexity classification.
|
|
48
|
+
|
|
49
|
+
**Current risk:** If `jq` is not installed, the fallback grep-based counting works but produces inaccurate results for malformed JSON. The PRD content is still passed to the AI provider as-is, which will likely fail at interpretation.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
### EP-003: Network Failure Mid-Build
|
|
54
|
+
|
|
55
|
+
**Given** an autonomous session is running at iteration 15 of 1000
|
|
56
|
+
**When** the network connection drops (DNS resolution fails, API timeout)
|
|
57
|
+
**Then** the system should:
|
|
58
|
+
- The provider CLI (claude/codex/gemini) exits with a non-zero exit code
|
|
59
|
+
- `run_autonomous()` captures the exit code in `$exit_code` (run.sh:9406)
|
|
60
|
+
- Rate limit detection runs via `is_rate_limited()` (checks for common rate limit strings)
|
|
61
|
+
- If failover is enabled (`LOKI_FAILOVER=true`), attempt `attempt_provider_failover()`
|
|
62
|
+
- Otherwise, apply exponential backoff via `calculate_wait()` (run.sh:6721)
|
|
63
|
+
- State is saved with `save_state $retry "running" $exit_code`
|
|
64
|
+
- Next iteration retries
|
|
65
|
+
|
|
66
|
+
**Expected behavior:** Retry with exponential backoff (capped at MAX_WAIT=3600s). Session survives.
|
|
67
|
+
|
|
68
|
+
**Current risk:** The backoff caps at 1 hour. A prolonged network outage lasting multiple hours will burn through all MAX_RETRIES (default 50) entries, each waiting up to 1 hour. After 50 retries, the session terminates. No automatic resume after connectivity returns.
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
### EP-004: API Key Expired During Autonomous Run
|
|
73
|
+
|
|
74
|
+
**Given** an autonomous session is running with a valid API key
|
|
75
|
+
**When** the API key expires or is revoked mid-session
|
|
76
|
+
**Then** the system should:
|
|
77
|
+
- The provider CLI returns an authentication error (exit code non-zero)
|
|
78
|
+
- The error output is captured in `$iter_output` and `$agent_log`
|
|
79
|
+
- `is_rate_limited()` scans for rate-limit patterns but may not detect auth errors
|
|
80
|
+
- If failover is enabled, the system checks `check_provider_health()` which validates `$key_var` is set (but not that it is valid)
|
|
81
|
+
- The system retries with exponential backoff
|
|
82
|
+
|
|
83
|
+
**Expected behavior:** Session retries until MAX_RETRIES is exhausted.
|
|
84
|
+
|
|
85
|
+
**Bug found (BUG-EP-004):** `check_provider_health()` (run.sh:6864) only checks if the API key environment variable is non-empty (`[ -n "${ANTHROPIC_API_KEY:-}" ]`). It does not validate that the key is actually functional. Failover logic may skip to a healthy-looking but equally broken provider if all keys are from the same account. The system should distinguish auth errors from rate limits and surface them differently.
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
### EP-005: Disk Full During Code Generation
|
|
90
|
+
|
|
91
|
+
**Given** the filesystem has less than 1MB free space
|
|
92
|
+
**When** the AI provider attempts to write generated code to disk
|
|
93
|
+
**Then** the system should:
|
|
94
|
+
- The provider CLI writes to `$iter_output` (a temp file in .loki/logs/)
|
|
95
|
+
- `mktemp` fails when disk is full, causing the provider output to be lost
|
|
96
|
+
- `save_state()` writes to `.loki/autonomy-state.json.tmp.$$` which also fails
|
|
97
|
+
- `create_checkpoint()` fails to write checkpoint metadata
|
|
98
|
+
- The entire iteration is lost with no recovery
|
|
99
|
+
|
|
100
|
+
**Expected behavior:** Session should detect disk space issues and pause gracefully.
|
|
101
|
+
|
|
102
|
+
**Current risk:** No disk space checks exist. `set -e` is active in run.sh, but errors in subshells (piped commands, background processes) may not propagate. The temp file approach for `save_state()` (BUG-XC-004 fix) will fail silently when `cat > "$state_tmp"` fails, then `mv` will fail, leaving the system in an inconsistent state.
|
|
103
|
+
|
|
104
|
+
**Suggested improvement:** Add a pre-iteration disk space check. If free space is below a threshold (e.g., 100MB), pause the session.
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
### EP-006: Provider CLI Not Installed
|
|
109
|
+
|
|
110
|
+
**Given** a user specifies `--provider codex` but `codex` is not in PATH
|
|
111
|
+
**When** they run `loki start --provider codex ./prd.md`
|
|
112
|
+
**Then** the system should:
|
|
113
|
+
- `load_provider()` (loader.sh:25) sources `codex.sh`
|
|
114
|
+
- `validate_provider_config()` checks required variables
|
|
115
|
+
- The `provider_detect()` function (called by `check_provider_installed()`) fails
|
|
116
|
+
- run.sh's `check_prerequisites()` function validates the API key for the provider
|
|
117
|
+
- If codex CLI is missing: `command -v codex &>/dev/null` returns 1 (but this check is in health checking, not in the startup path for non-failover mode)
|
|
118
|
+
|
|
119
|
+
**Expected behavior:** Clear error message indicating the CLI is not installed, with installation instructions.
|
|
120
|
+
|
|
121
|
+
**Current risk:** The check at run.sh:1173 (`command -v cline &>/dev/null`) is only for cline/aider. For codex, the check is whether the API key is set. If OPENAI_API_KEY is set but codex CLI is missing, the session will start and fail on the first provider invocation. The error from the shell will be `codex: command not found` which is captured in the log but not surfaced cleanly.
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
### EP-007: Invalid Template Selection
|
|
126
|
+
|
|
127
|
+
**Given** a user runs `loki init -t nonexistent-template`
|
|
128
|
+
**When** the init command processes the template flag
|
|
129
|
+
**Then** the system should:
|
|
130
|
+
- Check the `templates/` directory for a matching template
|
|
131
|
+
- If not found, display available templates
|
|
132
|
+
- Exit with a clear error message
|
|
133
|
+
|
|
134
|
+
**Expected behavior:** List available templates and exit with error code 1.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
### EP-008: Corrupted session.json State File
|
|
139
|
+
|
|
140
|
+
**Given** a previous session crash left a partially-written `.loki/session.json` containing `{"status": "runn`
|
|
141
|
+
**When** a user runs `loki status` or `loki start`
|
|
142
|
+
**Then** the system should:
|
|
143
|
+
- `load_state()` (run.sh:7956) calls `python3 -c "import json; print(json.load(...))"` which raises JSONDecodeError
|
|
144
|
+
- The `2>/dev/null || echo "unknown"` fallback returns "unknown"
|
|
145
|
+
- RETRY_COUNT defaults to 0, ITERATION_COUNT defaults to 0
|
|
146
|
+
- The session starts fresh
|
|
147
|
+
|
|
148
|
+
**Expected behavior:** Graceful recovery. Starts a new session with a warning.
|
|
149
|
+
|
|
150
|
+
**Bug found (BUG-EP-008):** `_safe_json_read()` in dashboard/server.py (line 79) retries once with a 0.1s sleep on JSONDecodeError, then returns `default`. But the CLI's `load_state()` in run.sh does NOT use `_safe_json_read()` -- it uses raw `python3 -c "import json; ..."` with a fallback. This means the CLI recovery path is less robust. A corrupted file causes a fresh start, which loses all progress (iteration count, retry count). The system should attempt to reconstruct state from checkpoints if session.json is corrupted.
|
|
151
|
+
|
|
152
|
+
---
|
|
153
|
+
|
|
154
|
+
### EP-009: WebSocket Disconnect During Build
|
|
155
|
+
|
|
156
|
+
**Given** a user has the dashboard open with an active WebSocket connection
|
|
157
|
+
**When** the network blips and the WebSocket disconnects
|
|
158
|
+
**Then** the system should:
|
|
159
|
+
- The `websocket_endpoint()` (server.py:1378) catches `WebSocketDisconnect`
|
|
160
|
+
- `manager.disconnect(websocket)` removes the connection from `active_connections`
|
|
161
|
+
- The autonomous build continues unaffected (it does not depend on WebSocket)
|
|
162
|
+
- When the client reconnects, it gets a fresh "connected" message
|
|
163
|
+
- The background `_push_loki_state_loop()` skips broadcasting when no clients are connected
|
|
164
|
+
|
|
165
|
+
**Expected behavior:** Build continues. Client reconnects and gets current state.
|
|
166
|
+
|
|
167
|
+
**Current risk:** The client must handle reconnection logic. If the client does not implement reconnection with backoff, the user sees a stale dashboard. The server sends pings every 30s and closes idle connections after 2 missed pongs (60s total). This is working correctly.
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
### EP-010: Concurrent Chat Messages During Build
|
|
172
|
+
|
|
173
|
+
**Given** an autonomous build is running
|
|
174
|
+
**When** a user submits a chat message via the dashboard while the AI provider is processing
|
|
175
|
+
**Then** the system should:
|
|
176
|
+
- The chat message is written to `.loki/HUMAN_INPUT.md` (via dashboard control API)
|
|
177
|
+
- `check_human_intervention()` (run.sh:~9292) detects the file at the start of the next iteration
|
|
178
|
+
- The `LOKI_HUMAN_INPUT` variable is set with the content
|
|
179
|
+
- `build_prompt()` injects it as `HUMAN_DIRECTIVE (PRIORITY)` in the prompt
|
|
180
|
+
- After consumption, the parent shell clears `LOKI_HUMAN_INPUT` (run.sh:9327)
|
|
181
|
+
|
|
182
|
+
**Expected behavior:** Message is queued and injected in the next iteration's prompt.
|
|
183
|
+
|
|
184
|
+
**Current risk:** If the user sends multiple messages before the next iteration starts, only the last one will be processed (each write to HUMAN_INPUT.md overwrites the previous). Earlier messages are lost.
|
|
185
|
+
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
### EP-011: Rate Limit Hit With No Fallback Provider
|
|
189
|
+
|
|
190
|
+
**Given** `LOKI_FAILOVER=false` (default) and the primary provider hits a rate limit
|
|
191
|
+
**When** the rate limit error is detected via `is_rate_limited()`
|
|
192
|
+
**Then** the system should:
|
|
193
|
+
- Detect rate limit patterns in the output
|
|
194
|
+
- Apply exponential backoff via `calculate_wait()` (base 60s, up to 3600s max)
|
|
195
|
+
- Retry after the backoff period
|
|
196
|
+
- Continue until MAX_RETRIES is exhausted
|
|
197
|
+
|
|
198
|
+
**Expected behavior:** Exponential backoff with jitter, eventual recovery.
|
|
199
|
+
|
|
200
|
+
**Current risk:** With base wait of 60s and exponential growth, after ~6 retries the backoff hits the 3600s (1 hour) cap. The system waits an hour between each attempt. If the rate limit lasts 24 hours, the session burns through ~24 retries at 1 hour each. This is correct but slow recovery.
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
### EP-012: Memory System Corruption Recovery
|
|
205
|
+
|
|
206
|
+
**Given** the `.loki/memory/index.json` file is corrupted (invalid JSON)
|
|
207
|
+
**When** the next memory operation attempts to load the index
|
|
208
|
+
**Then** the system should:
|
|
209
|
+
- `MemoryStorage._load_json()` (storage.py:276) catches `json.JSONDecodeError` and returns `None`
|
|
210
|
+
- The caller receives None and must handle it
|
|
211
|
+
- `_ensure_index()` only runs on initialization (when index does not exist)
|
|
212
|
+
- Subsequent memory operations that depend on the index silently fail
|
|
213
|
+
|
|
214
|
+
**Expected behavior:** Memory operations degrade gracefully. Build continues.
|
|
215
|
+
|
|
216
|
+
**Bug found (BUG-EP-012):** When `_load_json()` returns None for a corrupted index.json, the index is NOT automatically recreated. `_ensure_index()` checks `if not index_path.exists()` which returns False (the file exists, just corrupted). The system should detect corruption and recreate the index from individual episode/pattern files. Currently, all memory operations silently fail until a human manually deletes the corrupted file.
|
|
217
|
+
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
### EP-013: Git Conflicts During Parallel Worktree Merge
|
|
221
|
+
|
|
222
|
+
**Given** parallel mode is enabled with multiple worktrees working on different features
|
|
223
|
+
**When** two worktrees modify the same file and auto-merge is attempted
|
|
224
|
+
**Then** the system should:
|
|
225
|
+
- The auto-merge process detects the conflict
|
|
226
|
+
- Mark the conflicting merge as failed
|
|
227
|
+
- Log the conflicting files
|
|
228
|
+
- Continue the main session without the conflicted changes
|
|
229
|
+
|
|
230
|
+
**Expected behavior:** Merge failure is logged. Non-conflicting changes are merged. Conflicting changes require manual resolution.
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
### EP-014: Browser Closed During Long Build
|
|
235
|
+
|
|
236
|
+
**Given** a user starts a build via the web dashboard and closes the browser tab
|
|
237
|
+
**When** the build is at iteration 50 of 1000
|
|
238
|
+
**Then** the system should:
|
|
239
|
+
- The WebSocket connection drops (handled by EP-009 above)
|
|
240
|
+
- The autonomous build continues in the background (it runs as a shell process)
|
|
241
|
+
- The dashboard server continues running
|
|
242
|
+
- When the user reopens the browser, the dashboard reconnects and shows current state
|
|
243
|
+
|
|
244
|
+
**Expected behavior:** Build is completely unaffected. Dashboard reconnects on reopen.
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
### EP-015: kill -9 During Checkpoint Save
|
|
249
|
+
|
|
250
|
+
**Given** the system is mid-write in `create_checkpoint()` (run.sh:6283)
|
|
251
|
+
**When** `kill -9` is sent to the Loki process
|
|
252
|
+
**Then** the system should:
|
|
253
|
+
- The process terminates immediately (SIGKILL cannot be caught)
|
|
254
|
+
- The `trap cleanup INT TERM` handler does NOT run (SIGKILL bypasses traps)
|
|
255
|
+
- Partial files may exist: `.loki/state/checkpoints/cp-*/metadata.json` may be incomplete
|
|
256
|
+
- The temp file for `save_state()` (`.loki/autonomy-state.json.tmp.$$`) is orphaned
|
|
257
|
+
- The `.loki/loki.pid` file is stale (points to dead process)
|
|
258
|
+
|
|
259
|
+
**Expected behavior on next start:**
|
|
260
|
+
- `load_state()` reads the last good `autonomy-state.json` (the temp file is never renamed)
|
|
261
|
+
- The stale PID file is detected by `kill -0 "$dpid"` returning false
|
|
262
|
+
- Checkpoint index may be inconsistent with actual checkpoint directories
|
|
263
|
+
|
|
264
|
+
**Bug found (BUG-EP-015):** Orphaned temp files `.loki/autonomy-state.json.tmp.*` accumulate on repeated kill -9. No cleanup code removes these. The `_cleanup_stale_locks()` in storage.py only handles `.lock` files, not `.tmp` files.
|
|
265
|
+
|
|
266
|
+
**Suggested improvement:** On startup, clean up `*.tmp.*` files in `.loki/` that are older than 5 minutes.
|
|
267
|
+
|
|
268
|
+
---
|
|
269
|
+
|
|
270
|
+
## Concurrent Usage Scenarios
|
|
271
|
+
|
|
272
|
+
### CU-001: Two Users Same Project Simultaneously
|
|
273
|
+
|
|
274
|
+
**Given** User A and User B both run `loki start` in the same project directory
|
|
275
|
+
**When** both processes attempt to write to `.loki/autonomy-state.json` and `.loki/loki.pid`
|
|
276
|
+
**Then** the system should:
|
|
277
|
+
- The second invocation should detect an existing `loki.pid` and warn
|
|
278
|
+
- If the PID in `loki.pid` is alive, refuse to start (or require `--force`)
|
|
279
|
+
- If the PID is stale, overwrite and proceed
|
|
280
|
+
|
|
281
|
+
**Expected behavior:** Only one active session per project directory.
|
|
282
|
+
|
|
283
|
+
**Current risk:** The PID file check exists, but there is a TOCTOU race between reading the PID file and starting the new session. Two processes starting within milliseconds could both pass the check and run concurrently, causing interleaved writes to shared state files.
|
|
284
|
+
|
|
285
|
+
---
|
|
286
|
+
|
|
287
|
+
### CU-002: Multiple Builds Running At Once (Different Projects)
|
|
288
|
+
|
|
289
|
+
**Given** User runs `loki start` in `/project-a/` and `/project-b/` simultaneously
|
|
290
|
+
**When** both builds are running
|
|
291
|
+
**Then** the system should:
|
|
292
|
+
- Each project has its own `.loki/` directory
|
|
293
|
+
- Dashboard server instances may conflict on port 57374 (the default)
|
|
294
|
+
- The second build should either use a different port or detect the conflict
|
|
295
|
+
|
|
296
|
+
**Expected behavior:** Both builds run independently. Port conflict is handled.
|
|
297
|
+
|
|
298
|
+
**Bug found (BUG-CU-002):** The dashboard port is hardcoded as default `57374` (LOKI_DASHBOARD_PORT). If two projects both start dashboards, the second `uvicorn` instance fails to bind the port. The error is logged but the build proceeds without a dashboard. There is no automatic port increment or discovery of an alternative port.
|
|
299
|
+
|
|
300
|
+
---
|
|
301
|
+
|
|
302
|
+
### CU-003: Dashboard + CLI Controlling Same Session
|
|
303
|
+
|
|
304
|
+
**Given** a build is running via `loki start`
|
|
305
|
+
**When** the user simultaneously:
|
|
306
|
+
- Sends a "pause" command via the dashboard web UI
|
|
307
|
+
- Runs `loki pause` in the terminal
|
|
308
|
+
**Then** the system should:
|
|
309
|
+
- Both commands write `.loki/PAUSE` (creating it if it does not exist)
|
|
310
|
+
- `check_human_intervention()` detects the PAUSE file on the next iteration
|
|
311
|
+
- The system enters pause mode once
|
|
312
|
+
- No conflict arises since PAUSE is a boolean flag (file exists or not)
|
|
313
|
+
|
|
314
|
+
**Expected behavior:** Session pauses once. No double-pause issues.
|
|
315
|
+
|
|
316
|
+
---
|
|
317
|
+
|
|
318
|
+
### CU-004: Pause From CLI While Chat Active in Web
|
|
319
|
+
|
|
320
|
+
**Given** a user has typed a chat message in the web dashboard
|
|
321
|
+
**When** another terminal runs `loki pause`
|
|
322
|
+
**Then** the system should:
|
|
323
|
+
- The PAUSE file is created
|
|
324
|
+
- On the next iteration, the system pauses
|
|
325
|
+
- The chat message (if not yet submitted) remains in the browser's input field
|
|
326
|
+
- If the chat message was submitted (written to HUMAN_INPUT.md), it will be consumed when the session resumes
|
|
327
|
+
|
|
328
|
+
**Expected behavior:** Pause takes priority. Chat message is preserved for after resume.
|
|
329
|
+
|
|
330
|
+
---
|
|
331
|
+
|
|
332
|
+
### CU-005: Export While Build in Progress
|
|
333
|
+
|
|
334
|
+
**Given** an autonomous build is running at iteration 42
|
|
335
|
+
**When** the user runs `loki export json` in another terminal
|
|
336
|
+
**Then** the system should:
|
|
337
|
+
- `cmd_export()` reads from `.loki/` state files
|
|
338
|
+
- It reads `state/orchestrator.json`, `queue/*.json`, etc.
|
|
339
|
+
- These files may be mid-write by the running build
|
|
340
|
+
- The export uses Python's `json.load()` which may fail on partial files
|
|
341
|
+
|
|
342
|
+
**Expected behavior:** Export succeeds with the latest complete state snapshot.
|
|
343
|
+
|
|
344
|
+
**Bug found (BUG-CU-005):** The export function `_export_json()` (loki:5034) reads multiple state files without any locking. If the build process is mid-write (between temp file creation and atomic rename), the export may read an incomplete file. While atomic rename prevents partial reads, there is no guarantee of a consistent snapshot across multiple files. The export may capture `queue/pending.json` from iteration 42 and `state/orchestrator.json` from iteration 41.
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
### CU-006: Template Creation During Template Listing
|
|
349
|
+
|
|
350
|
+
**Given** User A runs `loki init -t` to list templates
|
|
351
|
+
**When** User B simultaneously saves a new template to the `templates/` directory
|
|
352
|
+
**Then** the system should:
|
|
353
|
+
- The listing reads the directory contents at a point in time
|
|
354
|
+
- The new template may or may not appear depending on timing
|
|
355
|
+
- No crash or corruption occurs
|
|
356
|
+
|
|
357
|
+
**Expected behavior:** Listing shows a consistent snapshot. New template appears on next listing.
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
### CU-007: Memory Consolidation During Retrieval
|
|
362
|
+
|
|
363
|
+
**Given** the memory consolidation pipeline is running (`run_memory_consolidation()`)
|
|
364
|
+
**When** simultaneously, `retrieve_memory_context()` attempts to read memory
|
|
365
|
+
**Then** the system should:
|
|
366
|
+
- `MemoryStorage` uses `fcntl.flock()` for file-level locking (storage.py:198)
|
|
367
|
+
- Retrieval acquires a shared lock (exclusive=False)
|
|
368
|
+
- Consolidation acquires an exclusive lock for writes
|
|
369
|
+
- Retrieval blocks until consolidation releases the lock for the specific file
|
|
370
|
+
|
|
371
|
+
**Expected behavior:** Correct results. Retrieval may be delayed but not corrupted.
|
|
372
|
+
|
|
373
|
+
**Current risk:** Lock granularity is per-file. Consolidation may write a new pattern file while retrieval is iterating over pattern IDs from the index. The index read happens before the lock on individual files. A pattern referenced in the index may not yet exist (or may have just been deleted during merge). This results in a None return from `_load_json()`, which the consolidation pipeline handles gracefully.
|
|
374
|
+
|
|
375
|
+
---
|
|
376
|
+
|
|
377
|
+
### CU-008: WebSocket Reconnect With Stale Token
|
|
378
|
+
|
|
379
|
+
**Given** a user's auth token has expired
|
|
380
|
+
**When** the WebSocket client attempts to reconnect with the expired token
|
|
381
|
+
**Then** the system should:
|
|
382
|
+
- `websocket_endpoint()` (server.py:1386) validates the token
|
|
383
|
+
- `auth.validate_token(ws_token)` returns None for expired tokens
|
|
384
|
+
- The server accepts the WebSocket, then immediately closes with code 1008 (Policy Violation)
|
|
385
|
+
- The client must obtain a new token before reconnecting
|
|
386
|
+
|
|
387
|
+
**Expected behavior:** Clean rejection with 1008 close code. No data leakage.
|
|
388
|
+
|
|
389
|
+
---
|
|
390
|
+
|
|
391
|
+
### CU-009: Provider Switch Mid-Build
|
|
392
|
+
|
|
393
|
+
**Given** a build is running with `--provider claude`
|
|
394
|
+
**When** the user modifies the LOKI_PROVIDER environment variable and signals a config reload
|
|
395
|
+
**Then** the system should:
|
|
396
|
+
- The running process has already loaded provider config
|
|
397
|
+
- Environment variable changes do NOT affect the running process
|
|
398
|
+
- The provider can only be changed by stopping and restarting the session
|
|
399
|
+
|
|
400
|
+
**Expected behavior:** Provider change requires session restart. No mid-build switching.
|
|
401
|
+
|
|
402
|
+
**Current risk:** There is no documented way to switch providers mid-build. If a user force-writes to `.loki/state/failover.json` to change the `currentProvider`, the failover code may pick it up, but this is an undocumented and untested code path.
|
|
403
|
+
|
|
404
|
+
---
|
|
405
|
+
|
|
406
|
+
### CU-010: Config Change During Active Session
|
|
407
|
+
|
|
408
|
+
**Given** a build is running
|
|
409
|
+
**When** the user runs `loki config set maxTier sonnet` in another terminal
|
|
410
|
+
**Then** the system should:
|
|
411
|
+
- The config command writes to `.loki/config.json`
|
|
412
|
+
- The running build re-reads config at certain checkpoints (e.g., each iteration)
|
|
413
|
+
- If the running build does NOT re-read config, the change takes effect on next session
|
|
414
|
+
|
|
415
|
+
**Expected behavior:** Config changes may not apply until the next iteration or session restart.
|
|
416
|
+
|
|
417
|
+
**Current risk:** Most config values are read once at startup and cached in shell variables (e.g., MAX_ITERATIONS, BUDGET_LIMIT). Changing them mid-session via `loki config set` has no effect on the running process. Only file-based signals (PAUSE, STOP, HUMAN_INPUT.md) are checked per-iteration.
|
|
418
|
+
|
|
419
|
+
---
|
|
420
|
+
|
|
421
|
+
## Edge Case Scenarios
|
|
422
|
+
|
|
423
|
+
### EC-001: Unicode in Project Name/Path
|
|
424
|
+
|
|
425
|
+
**Given** a project directory is at `/home/user/projects/cafe-app` (contains accented e)
|
|
426
|
+
**When** the user runs `loki start ./prd.md` from that directory
|
|
427
|
+
**Then** the system should:
|
|
428
|
+
- Bash handles UTF-8 paths correctly on modern systems
|
|
429
|
+
- Python's `Path()` and `open()` handle UTF-8 paths
|
|
430
|
+
- The `.loki/` directory is created inside the Unicode-named directory
|
|
431
|
+
- JSON serialization of paths uses UTF-8 encoding
|
|
432
|
+
|
|
433
|
+
**Expected behavior:** Full functionality. No encoding errors.
|
|
434
|
+
|
|
435
|
+
**Current risk:** The `save_state()` function (run.sh:7947) uses `printf '%s' "${PRD_PATH:-}" | sed 's/\\/\\\\/g; s/"/\\"/g'` for JSON escaping. This does not escape Unicode characters that may break JSON (e.g., null bytes). Normal Unicode characters like accented letters are valid JSON and work correctly.
|
|
436
|
+
|
|
437
|
+
---
|
|
438
|
+
|
|
439
|
+
### EC-002: Very Long PRD (100K+ Characters)
|
|
440
|
+
|
|
441
|
+
**Given** a PRD file is 100,000+ characters (approximately 150 pages)
|
|
442
|
+
**When** `loki start very-long-prd.md` is run
|
|
443
|
+
**Then** the system should:
|
|
444
|
+
- `detect_complexity()` reads the full file for `wc -w` and `grep -c` (run.sh:1268)
|
|
445
|
+
- The PRD content is injected into the prompt via `build_prompt()`
|
|
446
|
+
- The prompt may exceed the provider's context window
|
|
447
|
+
|
|
448
|
+
**Expected behavior:** Provider handles context overflow with truncation or error.
|
|
449
|
+
|
|
450
|
+
**Bug found (BUG-EC-002):** `build_prompt()` does not truncate the PRD content before injection. The full PRD path is referenced, and the AI provider reads the entire file. For a 100K+ character PRD with a 200K context window, the PRD alone could consume half the context, leaving insufficient room for codebase context, memory, and instructions. There is no PRD size limit or truncation logic.
|
|
451
|
+
|
|
452
|
+
---
|
|
453
|
+
|
|
454
|
+
### EC-003: Project With 1000+ Files
|
|
455
|
+
|
|
456
|
+
**Given** a large monorepo with 5000+ source files
|
|
457
|
+
**When** `detect_complexity()` runs
|
|
458
|
+
**Then** the system should:
|
|
459
|
+
- `find` command (run.sh:1240) counts all source files matching the pattern
|
|
460
|
+
- With 5000+ files, the find command may take several seconds
|
|
461
|
+
- The system classifies as "complex" since file_count > 50
|
|
462
|
+
- All 8 SDLC phases are activated
|
|
463
|
+
|
|
464
|
+
**Expected behavior:** Correct "complex" classification. Possible delay on first run.
|
|
465
|
+
|
|
466
|
+
---
|
|
467
|
+
|
|
468
|
+
### EC-004: Binary Files in Project (Images, Fonts)
|
|
469
|
+
|
|
470
|
+
**Given** a project contains .png, .woff2, and .mp4 files
|
|
471
|
+
**When** the AI provider attempts to read the project structure
|
|
472
|
+
**Then** the system should:
|
|
473
|
+
- `detect_complexity()` only counts source code files by extension
|
|
474
|
+
- Binary files are excluded from the file count
|
|
475
|
+
- The AI provider's file reading may attempt to read binary files
|
|
476
|
+
- Git diff operations may show binary file changes
|
|
477
|
+
|
|
478
|
+
**Expected behavior:** Binary files are safely ignored in complexity detection. The AI provider handles binary files according to its own capabilities.
|
|
479
|
+
|
|
480
|
+
---
|
|
481
|
+
|
|
482
|
+
### EC-005: Symlinks in Project Directory
|
|
483
|
+
|
|
484
|
+
**Given** a project has symlinks pointing to files outside the project directory
|
|
485
|
+
**When** `detect_complexity()` runs `find` to count files
|
|
486
|
+
**Then** the system should:
|
|
487
|
+
- `find "$target_dir" -type f` follows symlinks by default on some systems
|
|
488
|
+
- Symlinks pointing outside the project may inflate the file count
|
|
489
|
+
- Circular symlinks could cause `find` to loop indefinitely
|
|
490
|
+
|
|
491
|
+
**Expected behavior:** Symlinks should be handled safely without infinite loops.
|
|
492
|
+
|
|
493
|
+
**Suggested improvement:** Add `-not -type l` or `find -P` (do not follow symlinks) to the find command in `detect_complexity()` to avoid counting symlinked files and prevent circular reference issues.
|
|
494
|
+
|
|
495
|
+
---
|
|
496
|
+
|
|
497
|
+
### EC-006: .git Directory Handling
|
|
498
|
+
|
|
499
|
+
**Given** a project has a large `.git/` directory (multiple GB)
|
|
500
|
+
**When** operations scan the project directory
|
|
501
|
+
**Then** the system should:
|
|
502
|
+
- `detect_complexity()` excludes `*/.git/*` in its find command (run.sh:1245)
|
|
503
|
+
- Checkpoint operations use git commands, not direct .git/ access
|
|
504
|
+
- Export operations skip .git/ contents
|
|
505
|
+
|
|
506
|
+
**Expected behavior:** .git/ is correctly excluded from all scanning operations.
|
|
507
|
+
|
|
508
|
+
---
|
|
509
|
+
|
|
510
|
+
### EC-007: Empty Project Directory
|
|
511
|
+
|
|
512
|
+
**Given** a user runs `loki start` in a completely empty directory (no files, no .git)
|
|
513
|
+
**When** the system initializes
|
|
514
|
+
**Then** the system should:
|
|
515
|
+
- `detect_complexity()` returns file_count=0
|
|
516
|
+
- No PRD is provided, so codebase analysis mode activates
|
|
517
|
+
- git commands fail since there is no git repository
|
|
518
|
+
- `git rev-parse HEAD` returns error, caught by `2>/dev/null || echo "no-git"`
|
|
519
|
+
- The `.loki/` directory is created
|
|
520
|
+
|
|
521
|
+
**Expected behavior:** System starts in analysis mode. Git-dependent features degrade gracefully.
|
|
522
|
+
|
|
523
|
+
---
|
|
524
|
+
|
|
525
|
+
### EC-008: PRD With Code Blocks That Look Like Commands
|
|
526
|
+
|
|
527
|
+
**Given** a PRD contains markdown code blocks with shell commands:
|
|
528
|
+
```markdown
|
|
529
|
+
## Setup
|
|
530
|
+
Run: `rm -rf / --no-preserve-root`
|
|
531
|
+
```
|
|
532
|
+
**When** the AI provider interprets the PRD
|
|
533
|
+
**Then** the system should:
|
|
534
|
+
- The PRD content is passed as text to the AI provider
|
|
535
|
+
- The AI provider may interpret code blocks as instructions
|
|
536
|
+
- The `LOKI_BLOCKED_COMMANDS` env var (run.sh:44) should block dangerous commands
|
|
537
|
+
|
|
538
|
+
**Expected behavior:** Blocked commands are enforced. The AI provider does not execute dangerous commands from PRD content.
|
|
539
|
+
|
|
540
|
+
**Current risk:** The `LOKI_BLOCKED_COMMANDS` mechanism's enforcement point is unclear. It is documented as an environment variable but its actual enforcement depends on the AI provider's own safety mechanisms. Loki does not pre-scan PRD content for dangerous commands.
|
|
541
|
+
|
|
542
|
+
---
|
|
543
|
+
|
|
544
|
+
### EC-009: Session That Runs For 24+ Hours
|
|
545
|
+
|
|
546
|
+
**Given** an autonomous session is running in perpetual mode
|
|
547
|
+
**When** the session exceeds 24 hours of continuous operation
|
|
548
|
+
**Then** the system should:
|
|
549
|
+
- Log files accumulate in `.loki/logs/` (one per day: autonomy-YYYYMMDD.log)
|
|
550
|
+
- Agent log is trimmed at 1MB (run.sh:9347)
|
|
551
|
+
- Memory consolidation runs periodically
|
|
552
|
+
- Checkpoints are created every iteration (capped at 50 via retention)
|
|
553
|
+
- Token economics tracking accumulates
|
|
554
|
+
|
|
555
|
+
**Expected behavior:** Stable long-running operation. Log rotation prevents disk exhaustion.
|
|
556
|
+
|
|
557
|
+
**Current risk:** The agent log trimming (tail -c 500000) runs only when the file exceeds 1MB. For a 24+ hour session with verbose output, the daily log file (`autonomy-YYYYMMDD.log`) is NOT trimmed and can grow indefinitely. Only `agent.log` has size management.
|
|
558
|
+
|
|
559
|
+
---
|
|
560
|
+
|
|
561
|
+
### EC-010: 100+ Iterations Before Completion
|
|
562
|
+
|
|
563
|
+
**Given** LOKI_MAX_ITERATIONS=1000 and the build is at iteration 150
|
|
564
|
+
**When** the build continues running
|
|
565
|
+
**Then** the system should:
|
|
566
|
+
- Completion Council checks every `LOKI_COUNCIL_CHECK_INTERVAL` iterations (default 5)
|
|
567
|
+
- Stagnation detector triggers after `LOKI_COUNCIL_STAGNATION_LIMIT` iterations without git changes (default 5)
|
|
568
|
+
- Memory consolidation runs periodically
|
|
569
|
+
- Checkpoint count is capped at 50 (older ones pruned)
|
|
570
|
+
|
|
571
|
+
**Expected behavior:** System operates normally at high iteration counts.
|
|
572
|
+
|
|
573
|
+
**Current risk:** The `calculate_wait()` function (run.sh:6721) uses `2 ** exp` where exp is capped at 30 (BUG-RUN-004 fix). At iteration 100+ with many retries, the backoff works correctly. However, iteration count and retry count are independent counters. The iteration count tracks successful + failed iterations, while retry count tracks consecutive failures.
|
|
574
|
+
|
|
575
|
+
---
|
|
576
|
+
|
|
577
|
+
### EC-011: Nested PRD References
|
|
578
|
+
|
|
579
|
+
**Given** a PRD at `./prd.md` contains `See requirements in ./sub-prd.md` and `./sub-prd.md` references `./sub-sub-prd.md`
|
|
580
|
+
**When** the AI provider processes the PRD
|
|
581
|
+
**Then** the system should:
|
|
582
|
+
- Loki passes only the top-level PRD path to the AI provider
|
|
583
|
+
- The AI provider may or may not follow references to other files
|
|
584
|
+
- Loki does not pre-resolve or inline nested references
|
|
585
|
+
|
|
586
|
+
**Expected behavior:** The AI provider decides whether to read referenced files. Loki does not recursively resolve PRD references.
|
|
587
|
+
|
|
588
|
+
---
|
|
589
|
+
|
|
590
|
+
### EC-012: Template With Missing Required Fields
|
|
591
|
+
|
|
592
|
+
**Given** a template file is missing required sections (e.g., no "## Requirements" heading)
|
|
593
|
+
**When** `loki init -t broken-template` is used
|
|
594
|
+
**Then** the system should:
|
|
595
|
+
- The template content is loaded from `templates/`
|
|
596
|
+
- Missing fields result in empty sections in the generated PRD
|
|
597
|
+
- The generated PRD may be classified as "simple" due to low word/section counts
|
|
598
|
+
|
|
599
|
+
**Expected behavior:** Template generates a PRD with empty sections for missing fields. No crash.
|
|
600
|
+
|
|
601
|
+
---
|
|
602
|
+
|
|
603
|
+
### EC-013: Provider Returns Empty Response
|
|
604
|
+
|
|
605
|
+
**Given** the AI provider executes successfully (exit code 0) but returns no output
|
|
606
|
+
**When** the iteration completes
|
|
607
|
+
**Then** the system should:
|
|
608
|
+
- `$iter_output` is empty (0 bytes)
|
|
609
|
+
- `check_completion_promise()` finds no completion text -> returns 1
|
|
610
|
+
- `is_rate_limited()` finds no rate limit patterns -> returns false
|
|
611
|
+
- The iteration is counted as successful but produces no changes
|
|
612
|
+
- Stagnation detector may trigger if no git changes occur for consecutive iterations
|
|
613
|
+
|
|
614
|
+
**Expected behavior:** Silent iteration with no changes. Stagnation detection eventually triggers.
|
|
615
|
+
|
|
616
|
+
**Bug found (BUG-EC-013):** There is no explicit check for empty provider output. An empty response is treated as a successful iteration that did nothing. If the provider consistently returns empty responses (e.g., due to a broken prompt or API issue), the system will waste iterations until stagnation detection kicks in (default 5 consecutive iterations with no git changes). The system should detect empty provider output and treat it as an error.
|
|
617
|
+
|
|
618
|
+
---
|
|
619
|
+
|
|
620
|
+
### EC-014: Quality Gate Timeout
|
|
621
|
+
|
|
622
|
+
**Given** a quality gate (e.g., test coverage check) hangs indefinitely
|
|
623
|
+
**When** `enforce_test_coverage()` runs `npx vitest run` which never completes
|
|
624
|
+
**Then** the system should:
|
|
625
|
+
- The test runner process runs as a child of the main shell
|
|
626
|
+
- There is no explicit timeout on quality gate execution
|
|
627
|
+
- The entire iteration blocks until the test runner exits
|
|
628
|
+
|
|
629
|
+
**Expected behavior:** Quality gates should have a timeout (e.g., 5 minutes).
|
|
630
|
+
|
|
631
|
+
**Bug found (BUG-EC-014):** Quality gate functions like `enforce_test_coverage()` (run.sh:5516) and `run_code_review()` do not have explicit timeouts. A hung test runner or unresponsive AI reviewer will block the iteration indefinitely. The `timeout` command should wrap quality gate subprocess invocations.
|
|
632
|
+
|
|
633
|
+
---
|
|
634
|
+
|
|
635
|
+
### EC-015: Checkpoint Restore To Different Version
|
|
636
|
+
|
|
637
|
+
**Given** a checkpoint was created at version 6.70.0 of Loki
|
|
638
|
+
**When** the user upgrades to 6.71.1 and runs `loki checkpoint restore cp-15-1711234567`
|
|
639
|
+
**Then** the system should:
|
|
640
|
+
- `rollback_to_checkpoint()` (run.sh:6383) restores state files from the checkpoint
|
|
641
|
+
- The checkpoint contains `state/orchestrator.json` from v6.70.0
|
|
642
|
+
- The new version may expect different fields in the state files
|
|
643
|
+
- The git SHA from the checkpoint may reference a commit that no longer exists
|
|
644
|
+
|
|
645
|
+
**Expected behavior:** State files are restored. Missing fields use defaults. Stale git SHAs are logged as warnings.
|
|
646
|
+
|
|
647
|
+
**Current risk:** No version compatibility check exists in `rollback_to_checkpoint()`. If the state file schema changed between versions, the restored state may cause errors.
|
|
648
|
+
|
|
649
|
+
---
|
|
650
|
+
|
|
651
|
+
## Boundary Scenarios
|
|
652
|
+
|
|
653
|
+
### BD-001: Max File Size Limits
|
|
654
|
+
|
|
655
|
+
**Given** a project contains a single source file that is 50MB
|
|
656
|
+
**When** the AI provider attempts to read it
|
|
657
|
+
**Then** the system should:
|
|
658
|
+
- `detect_complexity()` counts it as one file regardless of size
|
|
659
|
+
- The AI provider's file reading depends on its own context window limits
|
|
660
|
+
- Git operations (diff, commit) handle large files but may be slow
|
|
661
|
+
|
|
662
|
+
**Expected behavior:** Large files are handled by the provider's own limits. Loki does not impose file size limits.
|
|
663
|
+
|
|
664
|
+
---
|
|
665
|
+
|
|
666
|
+
### BD-002: Max Project Count (Cross-Project Registry)
|
|
667
|
+
|
|
668
|
+
**Given** 1000+ projects are registered in the cross-project registry
|
|
669
|
+
**When** `GET /api/registry/projects` is called
|
|
670
|
+
**Then** the system should:
|
|
671
|
+
- The registry API returns all projects
|
|
672
|
+
- No pagination is implemented for the registry endpoint
|
|
673
|
+
- Large responses may cause client-side rendering issues
|
|
674
|
+
|
|
675
|
+
**Expected behavior:** All projects are returned. May be slow with many projects.
|
|
676
|
+
|
|
677
|
+
**Suggested improvement:** Add pagination support to the registry API.
|
|
678
|
+
|
|
679
|
+
---
|
|
680
|
+
|
|
681
|
+
### BD-003: Max Concurrent WebSocket Sessions
|
|
682
|
+
|
|
683
|
+
**Given** LOKI_MAX_WS_CONNECTIONS is set to 100 (default)
|
|
684
|
+
**When** the 101st WebSocket client attempts to connect
|
|
685
|
+
**Then** the system should:
|
|
686
|
+
- `ConnectionManager.connect()` (server.py:292) checks `len(self.active_connections) >= MAX_CONNECTIONS`
|
|
687
|
+
- Returns False, accepts the WebSocket, then immediately closes with code 1013
|
|
688
|
+
- Logs a warning about the connection limit
|
|
689
|
+
|
|
690
|
+
**Expected behavior:** Clean rejection with appropriate close code and message.
|
|
691
|
+
|
|
692
|
+
---
|
|
693
|
+
|
|
694
|
+
### BD-004: Token Budget Exactly At Limit
|
|
695
|
+
|
|
696
|
+
**Given** LOKI_BUDGET_LIMIT="10.00" and current estimated cost is $9.99
|
|
697
|
+
**When** the next iteration costs $0.02 (bringing total to $10.01)
|
|
698
|
+
**Then** the system should:
|
|
699
|
+
- `check_budget_limit()` (run.sh:7204) runs before each iteration
|
|
700
|
+
- At $9.99, the check passes (9.99 < 10.00)
|
|
701
|
+
- The iteration runs (costing $0.02)
|
|
702
|
+
- At the NEXT iteration, the check finds $10.01 >= $10.00
|
|
703
|
+
- The system creates `.loki/PAUSE` and `.loki/signals/BUDGET_EXCEEDED`
|
|
704
|
+
- The budget overshoot is $0.01 (one iteration's worth)
|
|
705
|
+
|
|
706
|
+
**Expected behavior:** Budget is enforced post-iteration, so overshoot by one iteration's cost is expected.
|
|
707
|
+
|
|
708
|
+
**Current risk:** The budget check happens BEFORE the iteration, using the accumulated cost from previous iterations. The current iteration's cost is not pre-calculated. This means the actual spend can exceed the budget by up to one iteration's cost. For expensive operations (e.g., an Opus planning phase), this overshoot could be significant ($5-25).
|
|
709
|
+
|
|
710
|
+
---
|
|
711
|
+
|
|
712
|
+
### BD-005: Iteration Count At Max
|
|
713
|
+
|
|
714
|
+
**Given** LOKI_MAX_ITERATIONS=1000 and ITERATION_COUNT is at 999
|
|
715
|
+
**When** the next iteration begins
|
|
716
|
+
**Then** the system should:
|
|
717
|
+
- `ITERATION_COUNT` increments to 1000 (run.sh:9284)
|
|
718
|
+
- `check_max_iterations()` (run.sh:7399) checks `$ITERATION_COUNT -ge $MAX_ITERATIONS`
|
|
719
|
+
- 1000 >= 1000 is true
|
|
720
|
+
- `save_state $retry "max_iterations_reached" 0` is called
|
|
721
|
+
- The session exits with return code 0
|
|
722
|
+
|
|
723
|
+
**Expected behavior:** Session stops cleanly at exactly MAX_ITERATIONS.
|
|
724
|
+
|
|
725
|
+
---
|
|
726
|
+
|
|
727
|
+
### BD-006: WebSocket Message Size Limit
|
|
728
|
+
|
|
729
|
+
**Given** the dashboard sends a very large state update (e.g., 10MB of task data)
|
|
730
|
+
**When** `manager.broadcast()` sends the message to all WebSocket clients
|
|
731
|
+
**Then** the system should:
|
|
732
|
+
- FastAPI/Starlette WebSocket has no default message size limit for sending
|
|
733
|
+
- The client (browser) has a default incoming message size limit (varies by browser)
|
|
734
|
+
- Very large messages may cause client disconnection
|
|
735
|
+
- The `send_json()` call may raise an exception caught by the `except Exception` block
|
|
736
|
+
|
|
737
|
+
**Expected behavior:** Large messages may cause client disconnection, handled by the broadcast cleanup logic.
|
|
738
|
+
|
|
739
|
+
---
|
|
740
|
+
|
|
741
|
+
### BD-007: API Response Payload Size
|
|
742
|
+
|
|
743
|
+
**Given** a project has 10,000+ tasks in the database
|
|
744
|
+
**When** `GET /api/projects/{id}/tasks` is called without pagination
|
|
745
|
+
**Then** the system should:
|
|
746
|
+
- All tasks are loaded from SQLAlchemy
|
|
747
|
+
- All tasks are serialized to JSON
|
|
748
|
+
- The response may be several MB
|
|
749
|
+
|
|
750
|
+
**Expected behavior:** Response succeeds but may be slow. No OOM expected for typical task counts.
|
|
751
|
+
|
|
752
|
+
**Suggested improvement:** Add pagination with `limit` and `offset` query parameters to task listing endpoints.
|
|
753
|
+
|
|
754
|
+
---
|
|
755
|
+
|
|
756
|
+
### BD-008: Template Name Length Limit
|
|
757
|
+
|
|
758
|
+
**Given** a user creates a template with a 500-character name
|
|
759
|
+
**When** `loki init -t <very-long-name>` is used
|
|
760
|
+
**Then** the system should:
|
|
761
|
+
- The template name is used as a filename in `templates/`
|
|
762
|
+
- Most filesystems limit filenames to 255 bytes
|
|
763
|
+
- A 500-character name would fail at file creation
|
|
764
|
+
|
|
765
|
+
**Expected behavior:** Error message about invalid template name.
|
|
766
|
+
|
|
767
|
+
**Current risk:** No template name length validation exists. The filesystem error propagates as an unhandled exception.
|
|
768
|
+
|
|
769
|
+
---
|
|
770
|
+
|
|
771
|
+
### BD-009: Log File Rotation At Boundary
|
|
772
|
+
|
|
773
|
+
**Given** an autonomous session runs across midnight (UTC)
|
|
774
|
+
**When** the log filename changes from `autonomy-20260324.log` to `autonomy-20260325.log`
|
|
775
|
+
**Then** the system should:
|
|
776
|
+
- `log_file=".loki/logs/autonomy-$(date +%Y%m%d).log"` is evaluated each iteration
|
|
777
|
+
- The new iteration writes to the new day's log file
|
|
778
|
+
- The old log file remains intact
|
|
779
|
+
- No data is lost at the boundary
|
|
780
|
+
|
|
781
|
+
**Expected behavior:** Smooth log rotation at midnight. Both files are preserved.
|
|
782
|
+
|
|
783
|
+
---
|
|
784
|
+
|
|
785
|
+
### BD-010: Memory Episode Count At Consolidation Threshold
|
|
786
|
+
|
|
787
|
+
**Given** 1000 episodes have accumulated (the `limit=1000` in consolidation.py:145)
|
|
788
|
+
**When** `consolidate(since_hours=24)` runs
|
|
789
|
+
**Then** the system should:
|
|
790
|
+
- Load up to 1000 episode IDs
|
|
791
|
+
- Load each episode from disk (1000 file reads)
|
|
792
|
+
- Cluster episodes by task type or embedding similarity
|
|
793
|
+
- Create patterns from clusters
|
|
794
|
+
|
|
795
|
+
**Expected behavior:** Consolidation completes successfully but may take several minutes.
|
|
796
|
+
|
|
797
|
+
**Current risk:** Loading 1000 episodes requires 1000 file reads with per-file locking. This could take 10+ seconds on spinning disks or network filesystems. The consolidation runs synchronously in the shell pipeline, blocking the next iteration until complete. For very active projects that generate many episodes, the 1000 limit prevents unbounded memory usage but the consolidation time could delay the next iteration.
|
|
798
|
+
|
|
799
|
+
---
|
|
800
|
+
|
|
801
|
+
## Appendix: Bug Summary
|
|
802
|
+
|
|
803
|
+
| Bug ID | Severity | Location | Description |
|
|
804
|
+
|--------|----------|----------|-------------|
|
|
805
|
+
| BUG-EP-004 | Medium | run.sh:6864 | `check_provider_health()` validates key exists, not that it works |
|
|
806
|
+
| BUG-EP-008 | Low | run.sh:7956 | CLI `load_state()` does not attempt checkpoint-based recovery on corruption |
|
|
807
|
+
| BUG-EP-012 | Medium | storage.py:170 | Corrupted index.json is not auto-recreated; silently breaks all memory |
|
|
808
|
+
| BUG-EP-015 | Low | run.sh:7939 | Orphaned `.tmp.*` files accumulate on kill -9; no startup cleanup |
|
|
809
|
+
| BUG-CU-002 | Medium | run.sh (dashboard) | No automatic port increment when default dashboard port is in use |
|
|
810
|
+
| BUG-CU-005 | Low | loki:5034 | Export reads multiple state files without cross-file consistency |
|
|
811
|
+
| BUG-EC-002 | Medium | run.sh (build_prompt) | No PRD size limit or truncation before context injection |
|
|
812
|
+
| BUG-EC-013 | Medium | run.sh (iteration) | Empty provider output treated as success; wastes iterations |
|
|
813
|
+
| BUG-EC-014 | High | run.sh:5516 | Quality gate subprocesses have no timeout; can hang indefinitely |
|