loki-mode 6.6.0 → 6.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1767 @@
1
+ # Loki Mode - State Machine Reference
2
+
3
+ Comprehensive state machine diagrams for every stateful component in the codebase.
4
+ Intended for Claude agents to reference in context for navigation and understanding.
5
+
6
+ **Format**: Each section contains an ASCII state diagram, a state table with
7
+ triggers and persistence, and source file references (file:line).
8
+
9
+ ---
10
+
11
+ ## Table of Contents
12
+
13
+ 1. [Master State File Index](#1-master-state-file-index)
14
+ 2. [Core Orchestration](#2-core-orchestration)
15
+ 3. [Completion Council](#3-completion-council)
16
+ 4. [Provider System](#4-provider-system)
17
+ 5. [Memory System](#5-memory-system)
18
+ 6. [Task Queue](#6-task-queue)
19
+ 7. [Quality Gates](#7-quality-gates)
20
+ 8. [Dashboard](#8-dashboard)
21
+ 9. [Event System](#9-event-system)
22
+ 10. [MCP Server](#10-mcp-server)
23
+ 11. [Autonomy Utilities](#11-autonomy-utilities)
24
+ 12. [Parallel Workflows](#12-parallel-workflows)
25
+ 13. [Checkpoint System](#13-checkpoint-system)
26
+ 14. [Complexity Detection](#14-complexity-detection)
27
+
28
+ ---
29
+
30
+ ## 1. Master State File Index
31
+
32
+ All runtime state lives under `.loki/` in the project root. This is the
33
+ filesystem-based communication bus between all components.
34
+
35
+ ```
36
+ .loki/
37
+ state.json # Runner state (iteration, status, exit_code)
38
+ session.json # Active session metadata
39
+ dashboard-state.json # Dashboard polling state (written every 2s)
40
+ generated-prd.md # Auto-generated PRD when none provided
41
+ continuity.md # Cross-iteration context for agent handoff
42
+
43
+ queue/
44
+ pending.json # Task queue entries
45
+ .bmad-populated # Marker: BMAD queue already loaded
46
+
47
+ council/
48
+ convergence.log # Git diff hashes per iteration
49
+ votes/
50
+ round-N.json # Vote results per round
51
+ prd-requirements.json # Extracted PRD requirements
52
+ verdicts.jsonl # Historical council verdicts
53
+
54
+ memory/
55
+ episodic/ # Episode trace JSON files
56
+ semantic/
57
+ patterns/ # Extracted patterns
58
+ skills/ # Procedural skills
59
+ index.json # Memory index (L1 layer)
60
+ timeline.json # Memory timeline (L2 layer)
61
+
62
+ events/
63
+ pending/ # Unprocessed event JSON files
64
+ archive/ # Processed events (for replay)
65
+ processed.json # Set of processed event IDs
66
+
67
+ context/
68
+ tracking.json # Token usage per iteration
69
+ last_offset.txt # Session JSONL parse offset
70
+
71
+ notifications/
72
+ triggers.json # Notification trigger definitions
73
+ active.json # Fired notifications
74
+
75
+ checkpoints/
76
+ checkpoint-NAME/ # Snapshot directories
77
+
78
+ quality/
79
+ reviews/ # Code review results
80
+
81
+ metrics/
82
+ efficiency/ # Tool efficiency data
83
+ rewards/ # Outcome/efficiency signals
84
+
85
+ app-runner/
86
+ state.json # App runner status
87
+ restart-signal # Signal file: trigger restart
88
+ stop-signal # Signal file: trigger stop
89
+
90
+ PAUSE # Signal file: pause execution
91
+ STOP # Signal file: stop execution
92
+ INPUT # Signal file: human input pending
93
+ RESUME # Signal file: resume from pause
94
+ ```
95
+
96
+ ---
97
+
98
+ ## 2. Core Orchestration
99
+
100
+ ### 2.1 Autonomous Iteration Loop
101
+
102
+ The main execution loop in `run.sh`. Each iteration invokes the LLM provider,
103
+ then runs post-iteration checks (checklist, app runner, playwright, code review,
104
+ council).
105
+
106
+ Source: `autonomy/run.sh:7380-8047`
107
+
108
+ ```
109
+ +------------------+
110
+ | INITIALIZED |
111
+ | (run_autonomous) |
112
+ +--------+---------+
113
+ |
114
+ build_prompt(), invoke provider
115
+ |
116
+ v
117
+ +------------+-------------+
118
+ | RUNNING |
119
+ | (provider process active)|
120
+ +------------+-------------+
121
+ |
122
+ provider exits (exit_code)
123
+ |
124
+ +--------------------+--------------------+
125
+ | |
126
+ exit_code == 0 exit_code != 0
127
+ | |
128
+ v v
129
+ +--------+---------+ +----------+---------+
130
+ | ITERATION_SUCCESS| | ITERATION_FAIL |
131
+ | save_state | | store_episode_trace|
132
+ | "exited" exit=0 | | ("failure") |
133
+ +--------+---------+ +----------+---------+
134
+ | |
135
+ +--------+--------+--------+ detect_rate_limit()
136
+ | | | | |
137
+ v v v v +--------+--------+
138
+ perp. council promise default | |
139
+ mode check check continue rate_limited backoff
140
+ | | | | | |
141
+ v v v v v v
142
+ [CONT] [COUNCIL [PROMISE [CONT] [WAIT_RATE [WAIT_BACKOFF
143
+ APPROVED] FULFILLED] _LIMIT] retry++]
144
+ | |
145
+ +---------+-------+
146
+ |
147
+ v
148
+ [CONT]
149
+ |
150
+ (retry > MAX_RETRIES?)
151
+ | |
152
+ yes no
153
+ | |
154
+ v v
155
+ [FAILED] [RUNNING]
156
+ max retries next iteration
157
+ exceeded
158
+ ```
159
+
160
+ | State | Value in state.json | Trigger In | Trigger Out | Source |
161
+ |-------|---------------------|------------|-------------|--------|
162
+ | running | `"running"` | Provider invoked | Provider exits | `run.sh:7380` |
163
+ | exited | `"exited"` | Provider exit | Post-iteration checks | `run.sh:7380` |
164
+ | council_approved | `"council_approved"` | Council votes COMPLETE | Loop returns 0 | `run.sh:7380` |
165
+ | completion_promise_fulfilled | `"completion_promise_fulfilled"` | Promise text found in output | Loop returns 0 | `run.sh:7380` |
166
+ | failed | `"failed"` | MAX_RETRIES exceeded | Loop returns 1 | `run.sh:8047` |
167
+
168
+ Persistence: `.loki/state.json` via `save_state()` at `run.sh:6911-6952`
169
+
170
+ ### 2.2 RARV Cycle
171
+
172
+ Every iteration maps to a phase based on `iteration % 4`.
173
+
174
+ Source: `autonomy/run.sh:1325-1359`
175
+
176
+ ```
177
+ iteration % 4:
178
+
179
+ 0: REASON ──> 1: ACT ──> 2: REFLECT ──> 3: VERIFY ──> 0: REASON ...
180
+ | | | |
181
+ v v v v
182
+ planning development development fast
183
+ (opus) (sonnet) (sonnet) (haiku)
184
+ ```
185
+
186
+ | Phase | iteration % 4 | Model Tier | Purpose |
187
+ |-------|---------------|------------|---------|
188
+ | REASON | 0 | planning (opus) | Architecture, system design, high-level decisions |
189
+ | ACT | 1 | development (sonnet) | Implementation, writing code |
190
+ | REFLECT | 2 | development (sonnet) | Code review, analysis |
191
+ | VERIFY | 3 | fast (haiku) | Unit tests, validation, monitoring |
192
+
193
+ Source: `get_rarv_tier()` at `run.sh:1325-1347`, `get_rarv_phase_name()` at `run.sh:1349-1359`
194
+
195
+ ### 2.3 CLI Session Control
196
+
197
+ The `loki` CLI manages session lifecycle through signal files.
198
+
199
+ Source: `autonomy/loki:485` (cmd_start), `autonomy/run.sh:8059` (check_human_intervention)
200
+
201
+ ```
202
+ +----------+
203
+ | IDLE |
204
+ | (no .loki|
205
+ | session)|
206
+ +----+-----+
207
+ |
208
+ loki start
209
+ |
210
+ v
211
+ +----+-----+
212
+ | RUNNING |<─────────────────────+
213
+ | session | |
214
+ | active | |
215
+ +----+-----+ RESUME file
216
+ | detected
217
+ +----+----+----+ |
218
+ | | | |
219
+ PAUSE file STOP file Ctrl+C |
220
+ detected detected (SIGINT) |
221
+ | | | |
222
+ v v v |
223
+ +----+----+ +-+------+ +--+------+--+
224
+ | PAUSED | |STOPPED | |INTERRUPTED |
225
+ | waiting | |cleanup | | 1st=pause |
226
+ | RESUME | |exit | | 2nd=stop |
227
+ +----+----+ +--------+ +-----+------+
228
+ | |
229
+ +-------------------------+
230
+ (RESUME file or re-run)
231
+ ```
232
+
233
+ Signal files (checked by `check_human_intervention` at `run.sh:8059`):
234
+
235
+ | File | Effect | Created By |
236
+ |------|--------|------------|
237
+ | `.loki/PAUSE` | Pause after current iteration | Dashboard, user, `loki pause` |
238
+ | `.loki/STOP` | Stop execution entirely | Dashboard, user, `loki stop` |
239
+ | `.loki/INPUT` | Wait for human input | Dashboard (contains directive text) |
240
+ | `.loki/RESUME` | Resume from pause | Dashboard, user, `loki resume` |
241
+
242
+ Interrupt handling (Ctrl+C): `run.sh:8261` (cleanup function)
243
+ - `INTERRUPT_COUNT=0`, first Ctrl+C sets `PAUSED=true`
244
+ - Second Ctrl+C within window triggers full stop
245
+
246
+ ### 2.4 Human Intervention
247
+
248
+ Source: `autonomy/run.sh:8059` (check_human_intervention)
249
+
250
+ ```
251
+ check_human_intervention()
252
+ |
253
+ +──> PAUSE file exists? ──yes──> handle_pause() ──> return 1 (paused)
254
+ | (in perpetual mode: auto-clear unless budget-triggered)
255
+ |
256
+ +──> PAUSE_AT_CHECKPOINT file exists? ──yes──> (checkpoint mode) pause ──> return 1
257
+ |
258
+ +──> HUMAN_INPUT.md exists? ──yes──> read directive ──> inject into prompt
259
+ | (security: no symlinks, 1MB limit, LOKI_PROMPT_INJECTION must be true)
260
+ |
261
+ +──> signals/COUNCIL_REVIEW_REQUESTED? ──yes──> force council vote
262
+ |
263
+ +──> STOP file exists? ──yes──> return 2 (stop) ──> exit loop
264
+ |
265
+ +──> (none) ──> return 0 ──> continue normally
266
+ ```
267
+
268
+ ---
269
+
270
+ ## 3. Completion Council
271
+
272
+ Multi-agent voting system that decides when a project is truly complete.
273
+ Prevents infinite loops and premature stops.
274
+
275
+ Source: `autonomy/completion-council.sh`
276
+
277
+ ### 3.1 Council Voting Pipeline
278
+
279
+ Source: `completion-council.sh:1311` (council_should_stop), `completion-council.sh:1260` (council_evaluate)
280
+
281
+ ```
282
+ council_should_stop() [line 1311]
283
+ |
284
+ +──> council disabled? ──yes──> return 1 (CONTINUE)
285
+ |
286
+ +──> iteration < MIN_ITERATIONS? ──yes──> return 1
287
+ |
288
+ +──> council_circuit_breaker_triggered()? [line 198]
289
+ | (stagnation detection, checked every time)
290
+ | sets circuit_triggered flag
291
+ |
292
+ +──> circuit_triggered OR iteration % CHECK_INTERVAL == 0?
293
+ | neither? ──> return 1
294
+ |
295
+ v
296
+ council_evaluate() [line 1260]
297
+ |
298
+ Phase 1: Reverify Checklist
299
+ +──> council_reverify_checklist() [line 550]
300
+ | (refresh checklist data before evaluation)
301
+ |
302
+ Phase 2: Checklist Hard Gate
303
+ +──> council_checklist_gate() [line 563]
304
+ | critical checklist items failing? ──yes──> return 1 (CONTINUE)
305
+ |
306
+ Phase 3: Council Voting
307
+ +──> council_aggregate_votes() [line 1057]
308
+ |
309
+ +──> result == "COMPLETE"?
310
+ | |
311
+ yes no ──> return 1 (CONTINUE)
312
+ |
313
+ v
314
+ unanimous? (complete_count == COUNCIL_SIZE && COUNCIL_SIZE >= 3)
315
+ | |
316
+ yes no ──> return 0 (COMPLETE)
317
+ |
318
+ v
319
+ Phase 4: Devil's Advocate (anti-sycophancy)
320
+ council_devils_advocate_review() [line 1156]
321
+ |
322
+ +────+────+
323
+ | |
324
+ OVERRIDE ALLOW
325
+ CONTINUE COMPLETE
326
+ | |
327
+ v v
328
+ return 1 return 0
329
+ (CONTINUE) (COMPLETE)
330
+
331
+ (back in council_should_stop)
332
+ |
333
+ council_evaluate returned 0 (COMPLETE)?
334
+ yes──> write COMPLETED marker, council_write_report ──> return 0 (STOP)
335
+ no ──> if circuit_triggered: log warning
336
+ if stagnation >= 2x STAGNATION_LIMIT: FORCE STOP (safety valve)
337
+ else ──> return 1 (CONTINUE)
338
+ ```
339
+
340
+ | State | Persistence | Source |
341
+ |-------|-------------|--------|
342
+ | convergence.log | `.loki/council/convergence.log` | Git diff hashes per iteration |
343
+ | votes/round-N.json | `.loki/council/votes/round-N.json` | Per-round vote tallies |
344
+ | prd-requirements.json | `.loki/council/prd-requirements.json` | Extracted requirements |
345
+ | verdicts.jsonl | `.loki/council/verdicts.jsonl` | Historical verdicts |
346
+
347
+ ### 3.2 Circuit Breaker
348
+
349
+ Detects stagnation (no git changes across iterations).
350
+
351
+ Source: `completion-council.sh` (council_circuit_breaker, council_track_iteration)
352
+
353
+ ```
354
+ council_track_iteration()
355
+ |
356
+ v
357
+ compute git diff hash (staged + unstaged)
358
+ |
359
+ +──> same as LAST_DIFF_HASH?
360
+ | |
361
+ yes no
362
+ | |
363
+ v v
364
+ CONSECUTIVE_NO_CHANGE++ CONSECUTIVE_NO_CHANGE=0
365
+ | |
366
+ v v
367
+ append "no_change" to append hash to
368
+ convergence.log convergence.log
369
+ |
370
+ v
371
+ count > STAGNATION_LIMIT (default 5)?
372
+ | |
373
+ yes no
374
+ | |
375
+ v v
376
+ CIRCUIT_BREAK CONTINUE
377
+ (force stop)
378
+ ```
379
+
380
+ ### 3.3 Devil's Advocate (Anti-Sycophancy)
381
+
382
+ Based on CONSENSAGENT (ACL 2025). Runs when council votes are unanimous.
383
+
384
+ Source: `completion-council.sh:1156` (council_devils_advocate_review)
385
+
386
+ ```
387
+ Unanimous COMPLETE vote detected
388
+ |
389
+ v
390
+ council_devils_advocate_review(iteration)
391
+ |
392
+ Spawn adversarial review agent:
393
+ "Challenge this completion claim.
394
+ Find what's missing or broken."
395
+ |
396
+ +──> Finds issues? ──yes──> return "OVERRIDE_CONTINUE"
397
+ |
398
+ +──> No issues ──> return "ALLOW_COMPLETE"
399
+ ```
400
+
401
+ Configuration:
402
+ - `COUNCIL_SIZE`: Number of members (default 3)
403
+ - `COUNCIL_THRESHOLD`: Votes for completion (default 2)
404
+ - `COUNCIL_CHECK_INTERVAL`: Check every N iterations (default 5)
405
+ - `COUNCIL_MIN_ITERATIONS`: Minimum before checking (default 3)
406
+ - `COUNCIL_STAGNATION_LIMIT`: Max no-change iterations (default 5)
407
+ - `COUNCIL_SEVERITY_THRESHOLD`: Blocking severity level (default "low")
408
+ - `COUNCIL_ERROR_BUDGET`: Fraction of issues tolerated (default 0.0)
409
+
410
+ ---
411
+
412
+ ## 4. Provider System
413
+
414
+ ### 4.1 Provider Lifecycle
415
+
416
+ Source: `providers/loader.sh:14-62`
417
+
418
+ ```
419
+ [UNINITIALIZED]
420
+ |
421
+ load_provider(name)
422
+ |
423
+ v
424
+ validate_provider(name)
425
+ |
426
+ +----+----+
427
+ | |
428
+ valid invalid
429
+ | |
430
+ v v
431
+ [VALIDATED] [ERROR: Unknown provider]
432
+ |
433
+ v
434
+ bash -n config_file (syntax check)
435
+ |
436
+ +──> syntax error? ──> [ERROR: Syntax error]
437
+ |
438
+ v
439
+ source config_file
440
+ |
441
+ +──> source failed? ──> [ERROR: Failed to source]
442
+ |
443
+ v
444
+ validate_provider_config()
445
+ |
446
+ +──> missing vars? ──> [ERROR: Config incomplete]
447
+ |
448
+ v
449
+ [READY]
450
+ (8 required vars set)
451
+ ```
452
+
453
+ Required provider variables (`loader.sh:65-83`):
454
+
455
+ | Variable | Example (Claude) |
456
+ |----------|------------------|
457
+ | PROVIDER_NAME | `"claude"` |
458
+ | PROVIDER_DISPLAY_NAME | `"Claude Code"` |
459
+ | PROVIDER_CLI | `"claude"` |
460
+ | PROVIDER_AUTONOMOUS_FLAG | `"-p"` |
461
+ | PROVIDER_PROMPT_POSITIONAL | `"false"` |
462
+ | PROVIDER_HAS_SUBAGENTS | `"true"` |
463
+ | PROVIDER_HAS_PARALLEL | `"true"` |
464
+ | PROVIDER_DEGRADED | `"false"` |
465
+
466
+ Supported providers: `claude`, `codex`, `gemini`, `cline`, `aider`
467
+
468
+ ### 4.2 Model Tier Selection
469
+
470
+ Each provider maps the 3 RARV tiers to provider-specific model settings via
471
+ `provider_get_tier_param()`.
472
+
473
+ Source: `providers/claude.sh:101`, `providers/codex.sh:106`, `providers/gemini.sh:132`,
474
+ `providers/cline.sh:102`, `providers/aider.sh:109`
475
+
476
+ ```
477
+ RARV Tier Claude (model) Codex (effort) Gemini (thinking) Cline Aider
478
+ --------- -------------- -------------- ----------------- ----- -----
479
+ planning opus xhigh high single model* single model**
480
+ development opus (upgraded) high medium single model* single model**
481
+ fast sonnet (upgraded) low low single model* single model**
482
+
483
+ * Cline: returns LOKI_CLINE_MODEL (default: "default"), single externally-configured model
484
+ ** Aider: returns LOKI_AIDER_MODEL (default: "claude-3.7-sonnet"), single externally-configured model
485
+
486
+ Note: Claude default tier mapping upgrades development->opus and fast->sonnet.
487
+ With LOKI_ALLOW_HAIKU=true: planning=opus, development=sonnet, fast=haiku (original mapping).
488
+ ```
489
+
490
+ ### 4.3 Degradation States
491
+
492
+ Source: `providers/claude.sh`, `providers/codex.sh`, `providers/gemini.sh`,
493
+ `providers/cline.sh`, `providers/aider.sh`
494
+
495
+ ```
496
+ Tier 1: Full Tier 2: Partial Tier 3: Degraded
497
+ +------------------+ +-------------------+ +-------------------+
498
+ | Claude (Full) | | Cline | | Codex |
499
+ | - Subagents | | - Subagents | | (Degraded) |
500
+ | - Parallel exec | | - MCP support | | - Sequential only |
501
+ | - Task tool | | - No parallel | | - MCP support |
502
+ | - MCP support | | - Not degraded | | - No subagents |
503
+ | - -p flag prompt | | - Single model | | - Positional prompt|
504
+ +------------------+ +-------------------+ +-------------------+
505
+
506
+ +-------------------+
507
+ | Gemini |
508
+ | (Degraded) |
509
+ | - Sequential only |
510
+ | - No MCP |
511
+ | - No subagents |
512
+ | - Positional prompt|
513
+ +-------------------+
514
+
515
+ +-------------------+
516
+ | Aider |
517
+ | (Degraded) |
518
+ | - Sequential only |
519
+ | - No MCP |
520
+ | - No subagents |
521
+ | - Single model |
522
+ +-------------------+
523
+ ```
524
+
525
+ Provider capability matrix:
526
+
527
+ | Provider | Subagents | Parallel | MCP | Degraded |
528
+ |----------|-----------|----------|-----|----------|
529
+ | Claude | true | true | true| false |
530
+ | Cline | true | false | true| false |
531
+ | Codex | false | false | true| true |
532
+ | Gemini | false | false | false| true |
533
+ | Aider | false | false | false| true |
534
+
535
+ When `PROVIDER_DEGRADED=true`:
536
+ - `build_prompt()` generates simplified prompts (no RARV/SDLC injection)
537
+ - Sequential execution only (no parallel worktrees)
538
+ - No subagent dispatch
539
+ - Prompt size limited to ~4000 chars of PRD content
540
+
541
+ ### 4.4 Rate Limit Recovery
542
+
543
+ Source: `autonomy/run.sh:6134-6173` (parse_retry_after, calculate_rate_limit_backoff, detect_rate_limit)
544
+
545
+ ```
546
+ Provider returns error
547
+ |
548
+ v
549
+ detect_rate_limit(log_file)
550
+ |
551
+ +----+----+
552
+ | |
553
+ detected not detected
554
+ | |
555
+ v v
556
+ parse_retry_after() calculate_wait(retry)
557
+ or calculate_rate_ (exponential backoff)
558
+ limit_backoff()
559
+ |
560
+ v
561
+ [WAIT_RATE_LIMIT]
562
+ countdown with progress
563
+ |
564
+ v
565
+ [RETRY]
566
+ ```
567
+
568
+ ---
569
+
570
+ ## 5. Memory System
571
+
572
+ ### 5.1 Episode Lifecycle
573
+
574
+ Source: `memory/schemas.py:292-319` (EpisodeTrace)
575
+
576
+ ```
577
+ [CREATE]
578
+ EpisodeTrace.create()
579
+ |
580
+ Set fields:
581
+ - id: "ep-YYYY-MM-DD-NNN"
582
+ - timestamp: UTC ISO 8601
583
+ - phase: REASON|ACT|REFLECT|VERIFY
584
+ - outcome: (pending)
585
+ |
586
+ v
587
+ [ACTIVE]
588
+ Agent executes task
589
+ - action_log appended
590
+ - errors_encountered appended
591
+ - files_read/modified tracked
592
+ |
593
+ v
594
+ [COMPLETE]
595
+ outcome set:
596
+ |
597
+ +----+----+----+
598
+ | | |
599
+ success failure partial
600
+ | | |
601
+ v v v
602
+ [STORED]
603
+ engine.store_episode(trace)
604
+ -> storage.save_episode()
605
+ -> .loki/memory/episodic/ep-ID.json
606
+ |
607
+ v
608
+ [INDEXED]
609
+ Importance score assigned (0.0-1.0)
610
+ last_accessed updated on retrieval
611
+ access_count incremented on retrieval
612
+ |
613
+ v
614
+ (Eventually consolidated -> semantic pattern)
615
+ ```
616
+
617
+ Episode outcome values: `success`, `failure`, `partial`
618
+ RARV phases: `REASON`, `ACT`, `REFLECT`, `VERIFY` (defined at `schemas.py:254`)
619
+ Link relations: `derived_from`, `related_to`, `contradicts`, `elaborates`,
620
+ `example_of`, `supersedes`, `superseded_by`, `supports` (defined at `schemas.py:159-168`)
621
+
622
+ ### 5.2 Progressive Disclosure Layers
623
+
624
+ Source: `memory/layers/` directory, `memory/retrieval.py`
625
+
626
+ ```
627
+ Memory Request
628
+ |
629
+ v
630
+ +------+------+
631
+ | L1: INDEX | ~100 tokens
632
+ | Topic names | .loki/memory/index.json
633
+ | + counts |
634
+ +------+------+
635
+ |
636
+ Is topic relevant?
637
+ (keyword/embedding match)
638
+ |
639
+ +----+----+
640
+ | |
641
+ yes no ──> skip topic
642
+ |
643
+ v
644
+ +------+-------+
645
+ | L2: TIMELINE | ~500 tokens
646
+ | Summaries | .loki/memory/timeline.json
647
+ | + dates |
648
+ | + outcomes |
649
+ +------+-------+
650
+ |
651
+ Need full detail?
652
+ (high relevance score)
653
+ |
654
+ +----+----+
655
+ | |
656
+ yes no ──> return L2 summary
657
+ |
658
+ v
659
+ +------+------+
660
+ | L3: FULL | variable tokens
661
+ | Complete | .loki/memory/episodic/*.json
662
+ | episode | .loki/memory/semantic/patterns/*.json
663
+ | traces |
664
+ +-------------+
665
+ ```
666
+
667
+ Token budget: L1 is always loaded. L2 loaded if topic relevant. L3 loaded
668
+ only if high relevance and token budget allows.
669
+
670
+ ### 5.3 Consolidation Pipeline
671
+
672
+ Transforms episodic memories into semantic patterns.
673
+
674
+ Source: `memory/consolidation.py`
675
+
676
+ ```
677
+ consolidate(since_hours=24)
678
+ |
679
+ v
680
+ +------+---------+
681
+ | LOAD_EPISODES |
682
+ | storage.list_ |
683
+ | episodes() |
684
+ | filter by time |
685
+ +------+---------+
686
+ |
687
+ v
688
+ +------+---------+
689
+ | CLUSTER |
690
+ | Group similar |
691
+ | episodes by: |
692
+ | - task type |
693
+ | - outcome |
694
+ | - files touched|
695
+ | (numpy if avail|
696
+ | else keyword) |
697
+ +------+---------+
698
+ |
699
+ v
700
+ +------+----------+
701
+ | EXTRACT_PATTERNS|
702
+ | Per cluster: |
703
+ | - success ratio |
704
+ | - common actions|
705
+ | - error types |
706
+ | Create Semantic |
707
+ | Pattern objects |
708
+ +------+----------+
709
+ |
710
+ v
711
+ +------+------+
712
+ | MERGE |
713
+ | Deduplicate |
714
+ | with existing|
715
+ | patterns |
716
+ | Update |
717
+ | confidence |
718
+ | Create links|
719
+ +------+------+
720
+ |
721
+ v
722
+ ConsolidationResult {
723
+ patterns_created,
724
+ patterns_merged,
725
+ anti_patterns_created,
726
+ links_created,
727
+ episodes_processed,
728
+ duration_seconds
729
+ }
730
+ ```
731
+
732
+ Persistence:
733
+ - Input: `.loki/memory/episodic/*.json`
734
+ - Output: `.loki/memory/semantic/patterns/*.json`
735
+ - Links: Zettelkasten-style (stored inline in pattern JSON)
736
+
737
+ ### 5.4 Token Economics
738
+
739
+ Tracks memory access efficiency to optimize retrieval.
740
+
741
+ Source: `memory/token_economics.py:29-58`
742
+
743
+ ```
744
+ Memory Access
745
+ |
746
+ Track tokens:
747
+ - discovery_tokens (indexing new memories)
748
+ - read_tokens (retrieving existing)
749
+ - layer loads (L1, L2, L3 counts)
750
+ - cache hits/misses
751
+ |
752
+ v
753
+ Evaluate Thresholds
754
+ |
755
+ +----+----+----+----+
756
+ | | | |
757
+ v v v v
758
+ ratio>0.15 savings L3>3 discovery
759
+ (discovery/ <50% loads tokens>200
760
+ read)
761
+ | | | |
762
+ v v v v
763
+ compress_ review_ create_ reorganize_
764
+ layer3 topic_ special topic_
765
+ relevance ized_ index
766
+ index
767
+ ```
768
+
769
+ | Threshold | Metric | Operator | Value | Action | Priority |
770
+ |-----------|--------|----------|-------|--------|----------|
771
+ | 1 | ratio (discovery/read) | > | 0.15 | compress_layer3 | 1 |
772
+ | 2 | savings_percent | < | 50 | review_topic_relevance | 2 |
773
+ | 3 | layer3_loads | > | 3 | create_specialized_index | 3 |
774
+ | 4 | discovery_tokens | > | 200 | reorganize_topic_index | 4 |
775
+
776
+ Persistence: `.loki/memory/` (tracked inline with access metadata)
777
+
778
+ ### 5.5 Storage Locking
779
+
780
+ Source: `memory/storage.py`
781
+
782
+ ```
783
+ File Operation Request
784
+ |
785
+ v
786
+ Acquire Lock
787
+ +----+----+
788
+ | |
789
+ read op write op
790
+ | |
791
+ v v
792
+ LOCK_SH LOCK_EX
793
+ (shared) (exclusive)
794
+ | |
795
+ v v
796
+ Read file Write to temp file
797
+ | |
798
+ v v
799
+ Release Atomic rename
800
+ lock (temp -> target)
801
+ |
802
+ v
803
+ Release lock
804
+ ```
805
+
806
+ Locking uses `fcntl.flock()` (POSIX file locking).
807
+ Atomic writes: write to tempfile, then `os.rename()` to target path.
808
+ Namespace isolation: each memory type has its own subdirectory.
809
+
810
+ ### 5.6 Retrieval Strategies
811
+
812
+ Task-aware memory retrieval with weighted strategies.
813
+
814
+ Source: `memory/retrieval.py:1-16`
815
+
816
+ ```
817
+ Retrieval Request (task_type, query)
818
+ |
819
+ v
820
+ Select Strategy Weights
821
+ |
822
+ +----+----+----+----+----+
823
+ | | | | | |
824
+ v v v v v |
825
+ explor impl debug review refactor
826
+ | | | | |
827
+ v v v v v
828
+
829
+ exploration: episodic=0.6 semantic=0.3 skills=0.1
830
+ implementation: episodic=0.15 semantic=0.5 skills=0.35
831
+ debugging: episodic=0.4 semantic=0.2 skills=0.0 anti_patterns=0.4
832
+ review: episodic=0.3 semantic=0.5 skills=0.2
833
+ refactoring: episodic=0.25 semantic=0.45 skills=0.3
834
+ |
835
+ v
836
+ Weighted merge of results
837
+ -> Score, rank, token-budget trim
838
+ -> Return context block
839
+ ```
840
+
841
+ ---
842
+
843
+ ## 6. Task Queue
844
+
845
+ ### 6.1 Task Lifecycle
846
+
847
+ Source: `.loki/queue/pending.json`
848
+
849
+ ```
850
+ [PENDING]
851
+ Task created (from PRD, BMAD, dashboard, or MCP)
852
+ |
853
+ v
854
+ [IN_PROGRESS]
855
+ Agent claims task
856
+ |
857
+ +────────+────────+
858
+ | | |
859
+ success failure timeout
860
+ | | |
861
+ v v v
862
+ [COMPLETED] [FAILED] [DEAD_LETTER]
863
+ (after max
864
+ retries)
865
+ ```
866
+
867
+ Queue entry format:
868
+ ```json
869
+ {
870
+ "id": "task-NNN",
871
+ "title": "...",
872
+ "description": "...",
873
+ "status": "pending|in_progress|completed|failed",
874
+ "priority": "low|medium|high|critical",
875
+ "created_at": "ISO 8601",
876
+ "epic": "..." (optional, from BMAD)
877
+ }
878
+ ```
879
+
880
+ BMAD population: `run.sh:7270-7372` -- runs once, writes `.loki/queue/.bmad-populated` marker.
881
+
882
+ ### 6.2 Checklist Verification
883
+
884
+ PRD requirements are extracted and verified against the codebase on interval.
885
+
886
+ Source: `run.sh:7880-7881` (checklist_should_verify, checklist_verify)
887
+
888
+ ```
889
+ [UNINITIALIZED]
890
+ No checklist extracted yet
891
+ |
892
+ PRD provided at start
893
+ |
894
+ v
895
+ [INITIALIZED]
896
+ Requirements extracted to
897
+ .loki/council/prd-requirements.json
898
+ |
899
+ checklist_should_verify()
900
+ returns true (on interval)
901
+ |
902
+ v
903
+ [VERIFYING]
904
+ Each requirement checked
905
+ against codebase/tests
906
+ |
907
+ v
908
+ [VERIFIED]
909
+ Results stored, injected
910
+ into next prompt as
911
+ $checklist_status
912
+ ```
913
+
914
+ ---
915
+
916
+ ## 7. Quality Gates
917
+
918
+ ### 7.1 Nine-Gate Pipeline
919
+
920
+ Source: `skills/quality-gates.md`
921
+
922
+ ```
923
+ Code Change
924
+ |
925
+ v
926
+ Gate 1: Static Analysis (CodeQL, ESLint)
927
+ |──BLOCK (critical findings)──> [REJECTED]
928
+ v
929
+ Gate 2: Type Check (tsc --noEmit)
930
+ |──BLOCK──> [REJECTED]
931
+ v
932
+ Gate 3: Unit Tests (>80% coverage, 100% pass)
933
+ |──BLOCK──> [REJECTED]
934
+ v
935
+ Gate 4: Integration Tests
936
+ |──BLOCK──> [REJECTED]
937
+ v
938
+ Gate 5: 3-Reviewer Blind Review (see 7.3)
939
+ |──BLOCK (Critical/High severity)──> [REJECTED]
940
+ v
941
+ Gate 6: Anti-Sycophancy Check
942
+ |──BLOCK (devil's advocate finds issues)──> [REJECTED]
943
+ v
944
+ Gate 7: Security Scan
945
+ |──BLOCK──> [REJECTED]
946
+ v
947
+ Gate 8: Performance Check
948
+ |──BLOCK──> [REJECTED]
949
+ v
950
+ Gate 9: E2E / Playwright
951
+ |──BLOCK──> [REJECTED]
952
+ v
953
+ [APPROVED]
954
+ ```
955
+
956
+ Gate status values: `passed`, `failed`, `skipped`
957
+ Persistence: `.loki/dashboard-state.json` field `qualityGates`
958
+ Severity levels: `critical`, `high`, `medium`, `low`
959
+ Blocking threshold: Critical and High always block; Medium blocks by default.
960
+
961
+ ### 7.2 Model Escalation
962
+
963
+ Source: `skills/model-selection.md`
964
+
965
+ ```
966
+ Issue Detected
967
+ |
968
+ v
969
+ [LOW]
970
+ Haiku handles
971
+ (unit tests, simple fixes)
972
+ |
973
+ Cannot resolve?
974
+ |
975
+ v
976
+ [MEDIUM]
977
+ Sonnet handles
978
+ (implementation, integration)
979
+ |
980
+ Cannot resolve?
981
+ |
982
+ v
983
+ [HIGH]
984
+ Opus handles
985
+ (architecture, complex bugs)
986
+ |
987
+ Cannot resolve?
988
+ |
989
+ v
990
+ [HUMAN]
991
+ Human intervention required
992
+ (.loki/INPUT signal file)
993
+ ```
994
+
995
+ ### 7.3 Code Review (3-Reviewer Blind)
996
+
997
+ Source: `run.sh:5039` (run_code_review)
998
+
999
+ ```
1000
+ Code changes committed
1001
+ |
1002
+ v
1003
+ Spawn 3 independent reviewers (parallel)
1004
+ Each reviews blindly (no access to other reviews)
1005
+ |
1006
+ +----+----+----+
1007
+ | | | |
1008
+ R1 R2 R3
1009
+ | | | |
1010
+ v v v v
1011
+ Collect verdicts
1012
+ |
1013
+ +----+----+
1014
+ | |
1015
+ 2/3 APPROVE <2/3 APPROVE
1016
+ | |
1017
+ v v
1018
+ [APPROVED] [REJECTED]
1019
+ Issues logged to
1020
+ .loki/quality/reviews/
1021
+ ```
1022
+
1023
+ ---
1024
+
1025
+ ## 8. Dashboard
1026
+
1027
+ ### 8.1 Task Status State Machine
1028
+
1029
+ Source: `dashboard/models.py:29-35`
1030
+
1031
+ ```
1032
+ [BACKLOG] ──promote──> [PENDING] ──claim──> [IN_PROGRESS]
1033
+ |
1034
+ +----+----+
1035
+ | |
1036
+ complete stuck
1037
+ | |
1038
+ v v
1039
+ [REVIEW] (stays
1040
+ | IN_PROGRESS)
1041
+ +----+----+
1042
+ | |
1043
+ approve reject
1044
+ | |
1045
+ v v
1046
+ [DONE] [IN_PROGRESS]
1047
+ ```
1048
+
1049
+ Values: `backlog`, `pending`, `in_progress`, `review`, `done`
1050
+ DB column: `tasks.status` (SQLAlchemy Enum)
1051
+
1052
+ ### 8.2 Agent Status State Machine
1053
+
1054
+ Source: `dashboard/models.py:46-51`
1055
+
1056
+ ```
1057
+ [IDLE] ──task assigned──> [RUNNING]
1058
+ ^ |
1059
+ | +----+----+
1060
+ | | |
1061
+ | pause exception
1062
+ | | |
1063
+ | v v
1064
+ | [PAUSED] [ERROR]
1065
+ | | |
1066
+ +──resume─────────────+ +----+
1067
+ +──error cleared───────────+
1068
+ ```
1069
+
1070
+ Values: `idle`, `running`, `paused`, `error`
1071
+ DB column: `agents.status` (SQLAlchemy Enum)
1072
+
1073
+ ### 8.3 Session Status State Machine
1074
+
1075
+ Source: `dashboard/models.py:54-59`
1076
+
1077
+ ```
1078
+ [ACTIVE] ──────────────────+──────────────+
1079
+ | | |
1080
+ /api/control/pause RARV complete council_should_stop
1081
+ | | returns failure
1082
+ v v |
1083
+ [PAUSED] [COMPLETED] v
1084
+ | [FAILED]
1085
+ /api/control/resume
1086
+ |
1087
+ v
1088
+ [ACTIVE]
1089
+ ```
1090
+
1091
+ Values: `active`, `completed`, `failed`, `paused`
1092
+ DB column: `sessions.status` (SQLAlchemy Enum)
1093
+ API endpoints: `POST /api/control/pause`, `POST /api/control/resume`, `POST /api/control/stop`
1094
+
1095
+ ### 8.4 WebSocket Connection Lifecycle
1096
+
1097
+ Source: `dashboard/server.py:1070-1145`
1098
+
1099
+ ```
1100
+ Client connects to /ws
1101
+ |
1102
+ v
1103
+ [RATE_CHECK]
1104
+ _read_limiter.check()
1105
+ |
1106
+ +----+----+
1107
+ | |
1108
+ pass fail
1109
+ | |
1110
+ v v
1111
+ [AUTH] [REJECTED]
1112
+ (if OIDC close code 1008
1113
+ enabled)
1114
+ |
1115
+ +──> token invalid? ──> [REJECTED] close 1008
1116
+ |
1117
+ v
1118
+ [CONNECTED]
1119
+ manager.connect(ws)
1120
+ send {"type": "connected"}
1121
+ |
1122
+ v
1123
+ [LISTENING] <─────────────+
1124
+ wait for message (30s) |
1125
+ | |
1126
+ +----+----+ |
1127
+ | | | |
1128
+ timeout "ping" "subscribe" |
1129
+ | | | |
1130
+ v v v |
1131
+ send send send |
1132
+ ping pong subscribed |
1133
+ | | | |
1134
+ +----+----+--------------+
1135
+ |
1136
+ WebSocketDisconnect
1137
+ |
1138
+ v
1139
+ [DISCONNECTED]
1140
+ manager.disconnect(ws)
1141
+ ```
1142
+
1143
+ Max connections: 100 (configurable via `LOKI_MAX_WS_CONNECTIONS`)
1144
+ Keep-alive timeout: 30 seconds
1145
+ Message types: `connected`, `ping`, `pong`, `subscribe`, `subscribed`
1146
+
1147
+ ### 8.5 Dashboard State File
1148
+
1149
+ Written atomically every 2 seconds by `run.sh` to `.loki/dashboard-state.json`.
1150
+
1151
+ ```json
1152
+ {
1153
+ "status": "running|paused|stopped",
1154
+ "phase": "understand|guardrail|migrate|verify|...",
1155
+ "iteration": 0,
1156
+ "complexity": "simple|standard|complex",
1157
+ "mode": "autonomous|interactive",
1158
+ "provider": "claude|codex|gemini|cline|aider",
1159
+ "current_task": "...",
1160
+ "budget": {"limit": 0.0, "used": 0.0, "remaining": 0.0},
1161
+ "qualityGates": {"gate_name": {"status": "passed|failed"}}
1162
+ }
1163
+ ```
1164
+
1165
+ ### 8.6 Dashboard Crash Recovery
1166
+
1167
+ Source: `autonomy/run.sh:5949` (handle_dashboard_crash)
1168
+
1169
+ ```
1170
+ Dashboard process exits unexpectedly
1171
+ |
1172
+ v
1173
+ handle_dashboard_crash()
1174
+ |
1175
+ +----+----+
1176
+ | |
1177
+ _DASHBOARD dashboard
1178
+ _RESTARTING disabled
1179
+ == true (guard)
1180
+ | |
1181
+ v v
1182
+ return 0 return 0
1183
+ (prevent
1184
+ recursive
1185
+ restart)
1186
+ |
1187
+ (normal path)
1188
+ |
1189
+ v
1190
+ Check PID file exists + process gone
1191
+ |
1192
+ v
1193
+ Restart dashboard silently
1194
+ (no pause handler trigger)
1195
+ ```
1196
+
1197
+ ---
1198
+
1199
+ ## 9. Event System
1200
+
1201
+ ### 9.1 Event Lifecycle
1202
+
1203
+ Source: `events/bus.py:86-415`, `events/bus.ts:118-408`, `events/emit.sh`
1204
+
1205
+ ```
1206
+ Event Created
1207
+ (emit() or emit_simple())
1208
+ |
1209
+ v
1210
+ [PENDING]
1211
+ Written to .loki/events/pending/TIMESTAMP_ID.json
1212
+ (atomic write with fcntl.flock / file lock)
1213
+ |
1214
+ v
1215
+ Background processor polls (every 0.5s)
1216
+ get_pending_events()
1217
+ |
1218
+ v
1219
+ [PROCESSING]
1220
+ For each subscriber:
1221
+ if types match: callback(event)
1222
+ (errors caught, don't break chain)
1223
+ |
1224
+ v
1225
+ mark_processed(event)
1226
+ |
1227
+ +----+----+
1228
+ | |
1229
+ archive no archive
1230
+ =true =false
1231
+ | |
1232
+ v v
1233
+ [ARCHIVED] [DELETED]
1234
+ Moved to Removed from
1235
+ archive/ pending/
1236
+ |
1237
+ v
1238
+ event.id added to processed_ids set
1239
+ (max 1000, LRU prune)
1240
+ ```
1241
+
1242
+ File format: `TIMESTAMP_ID.json` (e.g., `2026-03-02T14-30-45.123Z_a1b2c3d4.json`)
1243
+
1244
+ ### 9.2 Event Types and Sources
1245
+
1246
+ Source: `events/bus.py:21-44`
1247
+
1248
+ Event Types:
1249
+
1250
+ | Type | Value | Description |
1251
+ |------|-------|-------------|
1252
+ | STATE | `"state"` | Phase changes, status updates |
1253
+ | MEMORY | `"memory"` | Memory store/retrieve operations |
1254
+ | TASK | `"task"` | Task lifecycle (claim, complete, fail) |
1255
+ | METRIC | `"metric"` | Token usage, timing data |
1256
+ | ERROR | `"error"` | Errors and failures |
1257
+ | SESSION | `"session"` | Session start/stop/pause |
1258
+ | COMMAND | `"command"` | CLI command execution |
1259
+ | USER | `"user"` | User actions (VS Code, dashboard) |
1260
+
1261
+ Event Sources:
1262
+
1263
+ | Source | Value | Description |
1264
+ |--------|-------|-------------|
1265
+ | CLI | `"cli"` | `loki` CLI commands |
1266
+ | API | `"api"` | Dashboard REST API |
1267
+ | VSCODE | `"vscode"` | VS Code extension |
1268
+ | MCP | `"mcp"` | MCP server tools |
1269
+ | SKILL | `"skill"` | Skill module execution |
1270
+ | HOOK | `"hook"` | Git/lifecycle hooks |
1271
+ | DASHBOARD | `"dashboard"` | Dashboard UI actions |
1272
+ | MEMORY | `"memory"` | Memory system operations |
1273
+ | RUNNER | `"runner"` | run.sh orchestrator |
1274
+
1275
+ ### 9.3 Event Data Structure
1276
+
1277
+ Source: `events/bus.py:46-84`
1278
+
1279
+ ```json
1280
+ {
1281
+ "id": "a1b2c3d4",
1282
+ "type": "state|memory|task|metric|error|session|command|user",
1283
+ "source": "cli|api|vscode|mcp|skill|hook|dashboard|memory|runner",
1284
+ "timestamp": "2026-03-02T14:30:45.123Z",
1285
+ "payload": {"action": "...", ...},
1286
+ "version": "1.0"
1287
+ }
1288
+ ```
1289
+
1290
+ ### 9.4 Background Processing States
1291
+
1292
+ Source: `events/bus.py` (_running flag), `events/bus.ts` (running + pollInterval)
1293
+
1294
+ ```
1295
+ EventBus created
1296
+ |
1297
+ v
1298
+ [IDLE]
1299
+ _running = false
1300
+ No polling
1301
+ |
1302
+ start_background_processing(interval=0.5)
1303
+ |
1304
+ v
1305
+ [PROCESSING]
1306
+ _running = true
1307
+ Daemon thread/setInterval polls pending/
1308
+ |
1309
+ stop_background_processing()
1310
+ |
1311
+ v
1312
+ [STOPPED]
1313
+ _running = false
1314
+ Thread joined (2s timeout) / clearInterval
1315
+ ```
1316
+
1317
+ ### 9.5 Bash Event Emission
1318
+
1319
+ Source: `events/emit.sh:1-93`
1320
+
1321
+ ```
1322
+ ./emit.sh <type> <source> <action> [key=value ...]
1323
+ |
1324
+ v
1325
+ Generate EVENT_ID (8 hex chars from /dev/urandom)
1326
+ |
1327
+ v
1328
+ Generate TIMESTAMP (ISO 8601 UTC)
1329
+ Try: GNU date -> python3 fallback -> basic date
1330
+ |
1331
+ v
1332
+ Build payload JSON
1333
+ {"action": "<action>", "key1": "val1", ...}
1334
+ (json_escape for \, ", \t, \r)
1335
+ |
1336
+ v
1337
+ Write to $EVENTS_DIR/TIMESTAMP_ID.json
1338
+ |
1339
+ v
1340
+ Rotate events.jsonl if > 50MB
1341
+ (rename to events.jsonl.1, keep 1 backup)
1342
+ |
1343
+ v
1344
+ Output event ID to stdout
1345
+ ```
1346
+
1347
+ ---
1348
+
1349
+ ## 10. MCP Server
1350
+
1351
+ ### 10.1 Tool Call Lifecycle
1352
+
1353
+ Source: `mcp/server.py:473-1403`
1354
+
1355
+ ```
1356
+ MCP client sends tool invocation
1357
+ |
1358
+ v
1359
+ [RECEIVED]
1360
+ Parse tool name + arguments
1361
+ |
1362
+ v
1363
+ _emit_tool_event_async(tool, "start")
1364
+ Record start_time in _tool_call_start_times
1365
+ |
1366
+ v
1367
+ [EXECUTING]
1368
+ Tool function runs:
1369
+ - Validate paths (security check)
1370
+ - Read/write .loki/ state files
1371
+ - Return result
1372
+ |
1373
+ +----+----+
1374
+ | |
1375
+ success error
1376
+ | |
1377
+ v v
1378
+ [COMPLETE] [ERROR]
1379
+ _emit_ Return error
1380
+ learning_ message
1381
+ signal()
1382
+ |
1383
+ v
1384
+ _emit_tool_event_async(tool, "end")
1385
+ ```
1386
+
1387
+ 14 registered tools + 3 resources:
1388
+
1389
+ **Tools** (`@mcp.tool()`):
1390
+
1391
+ | Tool | Purpose | State Read | State Write | Line |
1392
+ |------|---------|------------|-------------|------|
1393
+ | loki_memory_retrieve | Retrieve memories | episodic/, semantic/ | -- | 472 |
1394
+ | loki_memory_store_pattern | Store semantic pattern | -- | semantic/patterns/ | 537 |
1395
+ | loki_task_queue_list | List tasks | queue.json | -- | 597 |
1396
+ | loki_task_queue_add | Add task | -- | queue.json | 641 |
1397
+ | loki_task_queue_update | Update task status | queue.json | queue.json | 717 |
1398
+ | loki_state_get | Get autonomy state | state.json | -- | 788 |
1399
+ | loki_metrics_efficiency | Get efficiency data | metrics/efficiency/ | -- | 842 |
1400
+ | loki_consolidate_memory | Run consolidation | episodic/ | semantic/ | 888 |
1401
+ | loki_start_project | Start RARV execution | -- | spawns run.sh | 992 |
1402
+ | loki_project_status | Get project status | state.json, dashboard-state | -- | 1044 |
1403
+ | loki_agent_metrics | Agent performance | Dashboard DB | -- | 1094 |
1404
+ | loki_checkpoint_restore | Restore checkpoint | checkpoints/ | (various) | 1132 |
1405
+ | loki_quality_report | Quality gate status | council/verdicts.jsonl | -- | 1182 |
1406
+ | loki_code_search | Search codebase | (ChromaDB) | -- | 1255 |
1407
+ | loki_code_search_stats | ChromaDB index stats | (ChromaDB) | -- | 1343 |
1408
+
1409
+ **Resources** (`@mcp.resource()`):
1410
+
1411
+ | Resource URI | Function | State Read | Line |
1412
+ |-------------|----------|------------|------|
1413
+ | loki://state/continuity | get_continuity | continuity.md | 928 |
1414
+ | loki://memory/index | get_memory_index | memory/index.json | 941 |
1415
+ | loki://queue/pending | get_pending_tasks | queue.json | 963 |
1416
+
1417
+ ### 10.2 Path Security Validation
1418
+
1419
+ Source: `mcp/server.py:109-150`
1420
+
1421
+ ```
1422
+ Path received from MCP client
1423
+ |
1424
+ v
1425
+ Canonicalize (resolve symlinks)
1426
+ |
1427
+ v
1428
+ Check: is path within allowed base?
1429
+ Allowed bases: [".loki", "memory"]
1430
+ |
1431
+ +----+----+
1432
+ | |
1433
+ within outside
1434
+ | |
1435
+ v v
1436
+ [SAFE] [BLOCKED]
1437
+ Proceed Raise
1438
+ with op PathTraversalError
1439
+ ```
1440
+
1441
+ Safe functions: `safe_path_join()`, `safe_open()`, `safe_makedirs()`, `safe_exists()`
1442
+
1443
+ ---
1444
+
1445
+ ## 11. Autonomy Utilities
1446
+
1447
+ ### 11.1 Context Window Tracking
1448
+
1449
+ Source: `autonomy/context-tracker.py`
1450
+
1451
+ ```
1452
+ update_tracking() called after each iteration
1453
+ |
1454
+ v
1455
+ Load .loki/context/tracking.json
1456
+ (or initialize empty)
1457
+ |
1458
+ v
1459
+ Find session file
1460
+ Claude: ~/.claude/projects/<slug>/*.jsonl
1461
+ Codex/Gemini: --tokens-input/--tokens-output args
1462
+ |
1463
+ v
1464
+ Parse new entries from last_offset
1465
+ (stored in .loki/context/last_offset.txt)
1466
+ |
1467
+ v
1468
+ Detect compactions
1469
+ (message contains "being continued from a previous conversation")
1470
+ |
1471
+ v
1472
+ Calculate iteration cost (provider-specific pricing)
1473
+ |
1474
+ v
1475
+ Evaluate context_window_pct
1476
+ |
1477
+ +----+----+----+
1478
+ | | |
1479
+ 0-80% 80-90% 90%+
1480
+ | | |
1481
+ v v v
1482
+ [NORMAL] [WARNING] [CRITICAL]
1483
+ Triggers notification
1484
+ ```
1485
+
1486
+ Provider pricing (USD per million tokens):
1487
+
1488
+ | Provider | Input | Output | Cache Read | Cache Creation |
1489
+ |----------|-------|--------|------------|----------------|
1490
+ | Claude | $3.00 | $15.00 | $0.30 | $3.75 |
1491
+ | Codex | $2.00 | $8.00 | -- | -- |
1492
+ | Gemini | $1.25 | $5.00 | -- | -- |
1493
+
1494
+ Context window sizes: Claude=200K, Codex=200K, Gemini=1M
1495
+
1496
+ Persistence: `.loki/context/tracking.json` (atomic write via temp+rename)
1497
+
1498
+ ### 11.2 Notification Triggers
1499
+
1500
+ Source: `autonomy/notification-checker.py:24-72`
1501
+
1502
+ ```
1503
+ check_triggers(loki_dir, iteration)
1504
+ |
1505
+ v
1506
+ Load triggers from .loki/notifications/triggers.json
1507
+ (create with defaults if missing)
1508
+ |
1509
+ v
1510
+ Load active notifications from .loki/notifications/active.json
1511
+ |
1512
+ v
1513
+ For each trigger (if enabled):
1514
+ |
1515
+ +----+----+
1516
+ | |
1517
+ already not fired
1518
+ fired for this iteration
1519
+ this iter
1520
+ | |
1521
+ v v
1522
+ [SKIP] Evaluate condition
1523
+ |
1524
+ +----+----+
1525
+ | |
1526
+ fired not fired
1527
+ | |
1528
+ v v
1529
+ Append to [SKIP]
1530
+ notifications[]
1531
+ |
1532
+ v
1533
+ Save .loki/notifications/active.json
1534
+ (max 100 notifications, prune oldest)
1535
+ ```
1536
+
1537
+ 6 trigger types:
1538
+
1539
+ | Trigger ID | Type | Condition | Severity | Default |
1540
+ |------------|------|-----------|----------|---------|
1541
+ | budget-80pct | budget_threshold | budget.used/limit >= 80% | warning | enabled |
1542
+ | context-90pct | context_threshold | context_window_pct >= 90% | critical | enabled |
1543
+ | sensitive-file | file_access | regex match on .env/.pem/.key/secret | critical | enabled |
1544
+ | quality-gate-fail | quality_gate | qualityGates[gate].status == "failed" | warning | enabled |
1545
+ | stuck-iteration | stagnation | 3+ consecutive "no_change" in convergence.log | warning | enabled |
1546
+ | compaction-freq | compaction_frequency | 3+ compactions in last hour | warning | enabled |
1547
+
1548
+ Notification data structure:
1549
+ ```json
1550
+ {
1551
+ "id": "notif-TIMESTAMP-TRIGGER_ID",
1552
+ "trigger_id": "...",
1553
+ "severity": "critical|warning|info",
1554
+ "message": "...",
1555
+ "timestamp": "ISO 8601",
1556
+ "iteration": 0,
1557
+ "acknowledged": false,
1558
+ "data": {}
1559
+ }
1560
+ ```
1561
+
1562
+ Duplicate prevention: `already_fired(trigger_id, iteration)` -- won't re-fire
1563
+ same trigger for same iteration.
1564
+
1565
+ ### 11.3 Budget Limit
1566
+
1567
+ Source: `autonomy/run.sh:6249` (check_budget_limit)
1568
+
1569
+ ```
1570
+ Before each iteration
1571
+ |
1572
+ v
1573
+ check_budget_limit()
1574
+ |
1575
+ Read budget from .loki/dashboard-state.json
1576
+ or environment LOKI_BUDGET_LIMIT
1577
+ |
1578
+ +----+----+
1579
+ | |
1580
+ within exceeded
1581
+ budget limit
1582
+ | |
1583
+ v v
1584
+ [CONTINUE] [BUDGET_EXCEEDED]
1585
+ save_state "budget_exceeded"
1586
+ return 1 (stop loop)
1587
+ ```
1588
+
1589
+ ---
1590
+
1591
+ ## 12. Parallel Workflows
1592
+
1593
+ ### 12.1 Git Worktree Lifecycle
1594
+
1595
+ Source: `skills/parallel-workflows.md`
1596
+
1597
+ ```
1598
+ Main branch (running)
1599
+ |
1600
+ Task requires parallel execution
1601
+ (Claude provider only, PROVIDER_HAS_PARALLEL=true)
1602
+ |
1603
+ v
1604
+ [CREATE_WORKTREE]
1605
+ git worktree add .loki/worktrees/<stream-id> -b <branch>
1606
+ |
1607
+ v
1608
+ [ACTIVE]
1609
+ Agent works in isolated worktree
1610
+ Independent file system, shared git history
1611
+ |
1612
+ +----+----+
1613
+ | |
1614
+ complete error
1615
+ | |
1616
+ v v
1617
+ [MERGE] [CLEANUP]
1618
+ Auto-merge Remove worktree
1619
+ to main git worktree remove
1620
+ |
1621
+ +──> conflict?
1622
+ | |
1623
+ yes no
1624
+ | |
1625
+ v v
1626
+ [CONFLICT] [MERGED]
1627
+ Signal to git worktree remove
1628
+ main agent cleanup
1629
+ for resolution
1630
+ ```
1631
+
1632
+ ### 12.2 Inter-Stream Signal Protocol
1633
+
1634
+ ```
1635
+ Stream A Stream B
1636
+ | |
1637
+ +──write SIGNAL file──> |
1638
+ | |
1639
+ | <──read SIGNAL file────+
1640
+ | |
1641
+ Signal files in .loki/worktrees/<id>/signals/:
1642
+ - COMPLETE: stream finished
1643
+ - BLOCKED: stream waiting on dependency
1644
+ - ERROR: stream encountered error
1645
+ - MERGE_READY: stream ready for merge
1646
+ ```
1647
+
1648
+ ---
1649
+
1650
+ ## 13. Checkpoint System
1651
+
1652
+ ### 13.1 Checkpoint Creation/Rollback
1653
+
1654
+ Source: `autonomy/run.sh:5607` (create_checkpoint)
1655
+
1656
+ ```
1657
+ create_checkpoint(description, tag)
1658
+ |
1659
+ v
1660
+ [SNAPSHOT]
1661
+ Copy current state files:
1662
+ - .loki/state.json
1663
+ - .loki/queue/
1664
+ - .loki/council/
1665
+ - .loki/context/
1666
+ - git stash (if uncommitted changes)
1667
+ |
1668
+ v
1669
+ Write to .loki/checkpoints/checkpoint-TAG/
1670
+ Include metadata:
1671
+ - timestamp
1672
+ - iteration
1673
+ - description
1674
+ - git_ref
1675
+ |
1676
+ v
1677
+ [STORED]
1678
+ ```
1679
+
1680
+ Checkpoints are created:
1681
+ - After each successful iteration (inside `run_autonomous()`)
1682
+ - After each failed iteration (inside `run_autonomous()`)
1683
+
1684
+ Rollback via MCP tool `loki_checkpoint_restore` or CLI.
1685
+
1686
+ ---
1687
+
1688
+ ## 14. Complexity Detection
1689
+
1690
+ ### 14.1 Auto-Detected Tiers
1691
+
1692
+ Source: `autonomy/run.sh:1196` (detect_complexity), `run.sh:1275` (get_complexity_phases),
1693
+ `run.sh:1293` (get_phase_names)
1694
+
1695
+ ```
1696
+ detect_complexity(prd_path) [line 1196]
1697
+ |
1698
+ v
1699
+ Count source files (excluding node_modules, .git, vendor, dist, build, __pycache__)
1700
+ Check for external integrations (OAuth, SAML, OIDC, Stripe, Twilio, AWS, Google Cloud, Azure)
1701
+ Check for microservices (docker-compose.yml/yaml, k8s/ directory)
1702
+ Analyze PRD word count + feature count (markdown headers/checkboxes or JSON arrays)
1703
+ |
1704
+ +----+----+----+
1705
+ | | |
1706
+ v v v
1707
+
1708
+ [SIMPLE] [STANDARD] [COMPLEX]
1709
+ <=5 files 6-50 files >50 files
1710
+ No external (default tier) External integrations
1711
+ integrations OR microservices
1712
+ PRD <200 words OR PRD complex
1713
+ + <5 features
1714
+ | | |
1715
+ v v v
1716
+ Phases (3): Phases (6): Phases (8):
1717
+ IMPLEMENT RESEARCH RESEARCH
1718
+ TEST DESIGN ARCHITECTURE
1719
+ DEPLOY IMPLEMENT DESIGN
1720
+ TEST IMPLEMENT
1721
+ REVIEW TEST
1722
+ DEPLOY REVIEW
1723
+ SECURITY
1724
+ DEPLOY
1725
+ ```
1726
+
1727
+ | Tier | File Count | External Deps | Microservices | PRD | Phase Count |
1728
+ |------|-----------|---------------|---------------|-----|-------------|
1729
+ | simple | <=5 | no | no | <200 words + <5 features | 3 |
1730
+ | standard | 6-50 | (default) | no | 200-1000 words | 6 |
1731
+ | complex | >50 | yes | yes | >1000 words or >15 features | 8 |
1732
+
1733
+ Persistence: `.loki/dashboard-state.json` field `complexity`
1734
+ Override: `COMPLEXITY_TIER` environment variable (bypasses auto-detection)
1735
+
1736
+ ---
1737
+
1738
+ ## Cross-Reference: Component Communication
1739
+
1740
+ ```
1741
+ loki CLI ──exec──> run.sh ──source──> completion-council.sh
1742
+ | | |
1743
+ | +──source──> providers/loader.sh
1744
+ | | +──source──> claude.sh|codex.sh|gemini.sh|cline.sh|aider.sh
1745
+ | |
1746
+ | +──python3──> memory/{engine,storage,retrieval,consolidation}.py
1747
+ | |
1748
+ | +──python3──> autonomy/context-tracker.py
1749
+ | |
1750
+ | +──python3──> autonomy/notification-checker.py
1751
+ | |
1752
+ | +──bash──> events/emit.sh
1753
+ | |
1754
+ | +──spawn──> dashboard/server.py (FastAPI)
1755
+ | |
1756
+ | +──import──> dashboard/models.py
1757
+ | |
1758
+ | +──WebSocket──> VS Code / Browser
1759
+ |
1760
+ +──────────────────> mcp/server.py (MCP protocol)
1761
+ |
1762
+ +──import──> memory/*.py
1763
+ +──import──> events/bus.py
1764
+
1765
+ All components communicate via .loki/ filesystem state files.
1766
+ Events provide async notification; filesystem provides state persistence.
1767
+ ```