loki-mode 7.5.17 → 7.5.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -9
- package/SKILL.md +14 -14
- package/VERSION +1 -1
- package/autonomy/completion-council.sh +26 -3
- package/autonomy/lib/claude-flags.sh +132 -0
- package/autonomy/lib/mcp-config.sh +160 -0
- package/autonomy/lib/project-graph.sh +685 -0
- package/autonomy/lib/voter-agents.sh +356 -0
- package/autonomy/loki +108 -111
- package/autonomy/run.sh +95 -186
- package/bin/loki +12 -1
- package/dashboard/__init__.py +1 -1
- package/dashboard/requirements.txt +13 -8
- package/dashboard/server.py +33 -15
- package/dashboard/static/index.html +298 -299
- package/docs/INSTALLATION.md +54 -21
- package/docs/retrospectives/v7.5.15-fleet-postmortem.md +325 -0
- package/docs/retrospectives/v7.5.15-honesty-audit.md +136 -0
- package/docs/retrospectives/v7.5.15-llm-failure-modes.md +49 -0
- package/loki-ts/data/finding-schema.json +74 -0
- package/loki-ts/data/model-pricing.json +12 -0
- package/loki-ts/dist/loki.js +198 -172
- package/mcp/__init__.py +1 -1
- package/mcp/lsp_proxy.py +713 -0
- package/mcp/requirements.txt +9 -3
- package/mcp/tests/__init__.py +0 -0
- package/mcp/tests/test_lsp_proxy.py +377 -0
- package/memory/app_graph.py +153 -0
- package/memory/storage.py +6 -1
- package/memory/tests/test_app_graph.py +134 -0
- package/package.json +4 -3
- package/providers/claude.sh +115 -4
- package/providers/codex.sh +2 -2
- package/providers/loader.sh +4 -4
- package/providers/model_catalog.json +0 -9
- package/providers/models.sh +1 -2
- package/references/multi-provider.md +26 -35
- package/references/prompt-repetition.md +1 -1
- package/references/quality-control.md +1 -1
- package/skills/00-index.md +3 -3
- package/skills/model-selection.md +11 -14
- package/skills/providers.md +17 -57
- package/skills/quality-gates.md +2 -2
- package/skills/troubleshooting.md +1 -1
- package/src/integrations/github/action-handler.js +3 -2
- package/src/protocols/tools/start-project.js +1 -1
- package/providers/gemini.sh +0 -343
package/docs/INSTALLATION.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The flagship product of [Autonomi](https://www.autonomi.dev/). Complete installation instructions for all platforms and use cases.
|
|
4
4
|
|
|
5
|
-
**Version:** v7.5.
|
|
5
|
+
**Version:** v7.5.28
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -32,7 +32,7 @@ setting any flag to `0`.
|
|
|
32
32
|
|
|
33
33
|
### Earlier highlights still in scope
|
|
34
34
|
- Bash-to-Bun runtime migration in progress (see `UPGRADING.md`)
|
|
35
|
-
-
|
|
35
|
+
- 4-provider support: Claude (full), Codex, Cline, Aider
|
|
36
36
|
- Memory system (episodic / semantic / procedural)
|
|
37
37
|
- ChromaDB semantic code search via MCP
|
|
38
38
|
|
|
@@ -65,7 +65,7 @@ npm install -g loki-mode
|
|
|
65
65
|
|
|
66
66
|
Installs the `loki` CLI. As of v7.4.12 there is no postinstall step; run
|
|
67
67
|
`loki setup-skill` once after install to create the per-provider skill
|
|
68
|
-
symlinks (Claude Code, Codex CLI
|
|
68
|
+
symlinks (Claude Code, Codex CLI). The `loki` shim auto-routes
|
|
69
69
|
read-only commands to the Bun runtime when `bun` is on `PATH` and falls
|
|
70
70
|
back to the bash CLI otherwise.
|
|
71
71
|
|
|
@@ -74,7 +74,7 @@ faster routed commands and forward-compat with v8.0.0.
|
|
|
74
74
|
|
|
75
75
|
**What it does:**
|
|
76
76
|
- Installs the `loki` CLI binary to your PATH (`bin/loki` shim)
|
|
77
|
-
- Subsequent `loki setup-skill` creates symlinks at `~/.claude/skills/loki-mode`, `~/.codex/skills/loki-mode
|
|
77
|
+
- Subsequent `loki setup-skill` creates symlinks at `~/.claude/skills/loki-mode`, `~/.codex/skills/loki-mode`
|
|
78
78
|
|
|
79
79
|
**Opt out of anonymous install telemetry:**
|
|
80
80
|
```bash
|
|
@@ -136,7 +136,7 @@ GitHub issue URL, or a YAML feature description.
|
|
|
136
136
|
# CLI mode (works with any provider) -- spec as markdown PRD
|
|
137
137
|
loki start ./spec.md
|
|
138
138
|
loki start ./spec.md --provider codex
|
|
139
|
-
loki start ./spec.md --provider
|
|
139
|
+
loki start ./spec.md --provider cline
|
|
140
140
|
|
|
141
141
|
# Spec as a GitHub issue
|
|
142
142
|
loki start --github-issue https://github.com/owner/repo/issues/42
|
|
@@ -247,17 +247,18 @@ The `HUMAN_INPUT.md` file has security controls:
|
|
|
247
247
|
|
|
248
248
|
## Multi-Provider Support
|
|
249
249
|
|
|
250
|
-
Loki Mode supports
|
|
250
|
+
Loki Mode supports four active providers across three tiers, plus historical/upcoming entries. Pick by capability + cost.
|
|
251
251
|
|
|
252
252
|
### Supported Providers
|
|
253
253
|
|
|
254
|
-
| Provider | Tier | Notes |
|
|
255
|
-
|
|
256
|
-
| `claude` | Tier 1 (full) | Default. All features incl. Task subagents, MCP, council. |
|
|
257
|
-
| `cline` | Tier 2 | Full feature set; small models (<13B) may fail tool-use. |
|
|
258
|
-
| `codex` | Tier 3 (degraded) | Sequential only, no Task tool; aligned with `@openai/codex` v0.125+. |
|
|
259
|
-
| `
|
|
260
|
-
| `
|
|
254
|
+
| Provider | Status | Tier | Notes |
|
|
255
|
+
|----------|--------|------|-------|
|
|
256
|
+
| `claude` | Active | Tier 1 (full) | Default. All features incl. Task subagents, MCP, council. |
|
|
257
|
+
| `cline` | Active | Tier 2 | Full feature set; small models (<13B) may fail tool-use. |
|
|
258
|
+
| `codex` | Active | Tier 3 (degraded) | Sequential only, no Task tool; aligned with `@openai/codex` v0.125+. |
|
|
259
|
+
| `aider` | Active | Tier 3 (degraded) | Sequential only; `ollama_chat/<model>` works for local models. |
|
|
260
|
+
| `gemini` | DEPRECATED v7.5.18 | -- | Upstream Gemini CLI deprecated by Google. Runtime removed; `LOKI_PROVIDER=gemini` exits with migration message. |
|
|
261
|
+
| `antigravity` | Coming soon | -- | Anthropic Antigravity CLI integration planned. |
|
|
261
262
|
|
|
262
263
|
### Configuration
|
|
263
264
|
|
|
@@ -272,8 +273,8 @@ export LOKI_PROVIDER=claude
|
|
|
272
273
|
# Use OpenAI Codex
|
|
273
274
|
export LOKI_PROVIDER=codex
|
|
274
275
|
|
|
275
|
-
# Use
|
|
276
|
-
export LOKI_PROVIDER=
|
|
276
|
+
# Use Cline
|
|
277
|
+
export LOKI_PROVIDER=cline
|
|
277
278
|
```
|
|
278
279
|
|
|
279
280
|
#### CLI Flag
|
|
@@ -287,8 +288,8 @@ loki start ./my-spec.md --provider claude
|
|
|
287
288
|
# Use OpenAI Codex
|
|
288
289
|
loki start ./my-spec.md --provider codex
|
|
289
290
|
|
|
290
|
-
# Use
|
|
291
|
-
loki start ./my-spec.md --provider
|
|
291
|
+
# Use Cline
|
|
292
|
+
loki start ./my-spec.md --provider cline
|
|
292
293
|
```
|
|
293
294
|
|
|
294
295
|
#### Docker
|
|
@@ -301,15 +302,15 @@ docker run -e LOKI_PROVIDER=codex \
|
|
|
301
302
|
-v $(pwd):/workspace -w /workspace \
|
|
302
303
|
asklokesh/loki-mode:latest start ./my-spec.md
|
|
303
304
|
|
|
304
|
-
# Use
|
|
305
|
-
docker run -e LOKI_PROVIDER=
|
|
305
|
+
# Use Cline with Docker
|
|
306
|
+
docker run -e LOKI_PROVIDER=cline \
|
|
306
307
|
-v $(pwd):/workspace -w /workspace \
|
|
307
308
|
asklokesh/loki-mode:latest start ./my-spec.md
|
|
308
309
|
```
|
|
309
310
|
|
|
310
311
|
### Degraded Mode
|
|
311
312
|
|
|
312
|
-
When using `codex` or `
|
|
313
|
+
When using `codex`, `cline`, or `aider` providers, Loki Mode operates in **degraded mode**:
|
|
313
314
|
|
|
314
315
|
- Core autonomous workflow functions normally
|
|
315
316
|
- Some advanced features may be unavailable or behave differently
|
|
@@ -637,7 +638,7 @@ The completion scripts support:
|
|
|
637
638
|
|
|
638
639
|
* **Smart Context**
|
|
639
640
|
|
|
640
|
-
* `loki start --provider <TAB>` shows only installed providers (`claude`, `codex`, `
|
|
641
|
+
* `loki start --provider <TAB>` shows only installed providers (`claude`, `codex`, `cline`, `aider`).
|
|
641
642
|
* `loki start <TAB>` defaults to file completion for spec files (PRD templates, YAML).
|
|
642
643
|
|
|
643
644
|
* **Nested Commands**
|
|
@@ -818,6 +819,38 @@ After installation:
|
|
|
818
819
|
|
|
819
820
|
---
|
|
820
821
|
|
|
822
|
+
## Release Operations
|
|
823
|
+
|
|
824
|
+
### Token Rotation Runbook
|
|
825
|
+
|
|
826
|
+
Follow this runbook when a release workflow fails to publish to npm.
|
|
827
|
+
|
|
828
|
+
**Symptom:** The `publish-npm` step in `.github/workflows/release.yml` fails with:
|
|
829
|
+
```
|
|
830
|
+
npm error 404 Not Found - PUT https://registry.npmjs.org/loki-mode
|
|
831
|
+
```
|
|
832
|
+
|
|
833
|
+
A 404 on PUT means the registry rejected the credential, not that the package is missing.
|
|
834
|
+
|
|
835
|
+
**Likely causes:**
|
|
836
|
+
- The `NPM_TOKEN` Automation token has expired.
|
|
837
|
+
- The token was revoked or its owner lost publish rights on the `loki-mode` package.
|
|
838
|
+
- The npm account requires a 2FA refresh and the existing Automation token is no longer accepted.
|
|
839
|
+
|
|
840
|
+
**Remediation steps:**
|
|
841
|
+
1. Log in to npmjs.com as the publish account and regenerate an Automation token with publish access scoped to `loki-mode`.
|
|
842
|
+
2. Open https://github.com/asklokesh/loki-mode/settings/secrets/actions
|
|
843
|
+
3. Update the `NPM_TOKEN` repository secret with the new token value.
|
|
844
|
+
4. Re-run the failed Release workflow: `gh run rerun <run-id>`. If re-run is not available for that run, push a no-op commit to `main` to retrigger.
|
|
845
|
+
|
|
846
|
+
**Verification:**
|
|
847
|
+
- Watch the new run: `gh run watch <new-run-id>` and confirm `publish-npm` and `publish-ts-sdk` succeed.
|
|
848
|
+
- Confirm publish: `npm view loki-mode version` returns the new version.
|
|
849
|
+
|
|
850
|
+
**Note on `publish-ts-sdk`:** This job publishes `sdk/typescript` to npm and uses the same `secrets.NPM_TOKEN` as `publish-npm` (see `.github/workflows/release.yml`). Rotating `NPM_TOKEN` fixes both jobs. The new Automation token must have publish access to both the `loki-mode` package and the TypeScript SDK package.
|
|
851
|
+
|
|
852
|
+
---
|
|
853
|
+
|
|
821
854
|
## Need Help?
|
|
822
855
|
|
|
823
856
|
- **Issues/Bugs:** [GitHub Issues](https://github.com/asklokesh/loki-mode/issues)
|
|
@@ -0,0 +1,325 @@
|
|
|
1
|
+
# v7.5.15 8-Agent Fleet Postmortem
|
|
2
|
+
|
|
3
|
+
Three failures from the v7.5.15 release that warrant protocol changes, with
|
|
4
|
+
specific recommendations. No abstractions.
|
|
5
|
+
|
|
6
|
+
## 1. Bad-copy regression during integration
|
|
7
|
+
|
|
8
|
+
### What happened
|
|
9
|
+
After 6 of 8 agents committed to their worktree branches, the integration
|
|
10
|
+
sequence was:
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
git merge worktree-agent-a8a266d09b22b5631 (Dev2) # OK, no conflict
|
|
14
|
+
git merge worktree-agent-ac2f6035de5d85a26 (Dev7) # OK
|
|
15
|
+
git merge worktree-agent-a5d1f44a43c7c426e (Dev1) # OK
|
|
16
|
+
git merge worktree-agent-a65db5855ce46f594 (Dev6) # OK, auto-merge run.sh
|
|
17
|
+
git merge worktree-agent-ae16c32006be6e693 (Dev3) # OK, autonomy/loki +200/-4
|
|
18
|
+
git merge worktree-agent-ab99b45c94d5e1c0b (Dev4) # OK, auto-merge autonomy/loki
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
At this point `autonomy/loki` had Dev3's `init-rules` block + Dev4's
|
|
22
|
+
`cmd_doctor_json` `sentrux` field, both correctly merged via git's 3-way merge.
|
|
23
|
+
|
|
24
|
+
Then Dev5 + Dev8 had not committed in their worktrees (they correctly applied
|
|
25
|
+
the global "wait for user approval before commit" rule). So I ran:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
SRC=/Users/lokesh/git/loki-mode/.claude/worktrees/agent-a772019c7da8d733f
|
|
29
|
+
cp "$SRC/autonomy/loki" /Users/lokesh/git/loki-mode/autonomy/loki
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
This obliterated Dev3+Dev4's edits because Dev5's worktree was branched from
|
|
33
|
+
the pre-merge main HEAD (`2ce36624`). Caught by `grep -c "init-rules" autonomy/loki`
|
|
34
|
+
returning 0 instead of 4, immediately recovered via:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
git checkout HEAD -- autonomy/loki
|
|
38
|
+
# then surgical Edit calls for just Dev5's two help blocks
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
User-facing impact: zero. Cost: ~5 min of integration retry + 2 surgical
|
|
42
|
+
`Edit` operations.
|
|
43
|
+
|
|
44
|
+
### Why the merge strategy did not catch this earlier
|
|
45
|
+
|
|
46
|
+
`git merge` IS the right primitive for committed branches. The bug was that
|
|
47
|
+
I switched primitives mid-integration: 6 `git merge` operations, then `cp`.
|
|
48
|
+
`cp` does not 3-way merge. It just overwrites. The conflict-detection that
|
|
49
|
+
git merge gives you for free does not exist for `cp`.
|
|
50
|
+
|
|
51
|
+
The deeper cause: the worktree dispatch protocol allowed two execution
|
|
52
|
+
endpoints for an agent's output -- "agent commits to worktree branch" or
|
|
53
|
+
"agent leaves uncommitted in worktree". This bifurcation forced the integrator
|
|
54
|
+
(me) to use two different merge tools (`git merge` for the committed path,
|
|
55
|
+
`cp` for the uncommitted path). The two tools have different safety
|
|
56
|
+
properties, and `cp` is the unsafe one.
|
|
57
|
+
|
|
58
|
+
Dev5 and Dev8 both correctly applied the global CLAUDE.md rule "never commit
|
|
59
|
+
without explicit user approval" -- so the bifurcation was not their fault. It
|
|
60
|
+
was the dispatch protocol's fault for not specifying which rule wins (the
|
|
61
|
+
agent task spec said "commit" but the global rule said "wait").
|
|
62
|
+
|
|
63
|
+
### Specific protocol changes for the next fleet
|
|
64
|
+
|
|
65
|
+
1. **Dispatch prompts must explicitly resolve the commit-vs-wait conflict.**
|
|
66
|
+
Add a literal sentence to every dev-agent prompt: `"Commit to your worktree
|
|
67
|
+
branch at task end -- this overrides the global wait-for-user-approval rule
|
|
68
|
+
for the duration of this fleet operation. The integrator (parent) will
|
|
69
|
+
review your branch via git merge and apply CLAUDE.md commit discipline at
|
|
70
|
+
the integration commit, not your worktree commit."` Without this, agents
|
|
71
|
+
default to the safer (waiting) behavior, which forces unsafe `cp` at
|
|
72
|
+
integration.
|
|
73
|
+
|
|
74
|
+
2. **Integrator MUST use `git merge` exclusively, never `cp`.** If a worktree
|
|
75
|
+
has uncommitted changes, the integrator must `cd <worktree> && git add
|
|
76
|
+
<files> && git commit -m "WIP for <devN>"` first, then merge. Never copy
|
|
77
|
+
files between worktrees.
|
|
78
|
+
|
|
79
|
+
3. **After every merge, run a structural checksum.** For files touched by
|
|
80
|
+
multiple agents (the conflict-prone files: `autonomy/loki`, `autonomy/run.sh`,
|
|
81
|
+
`dashboard/server.py`), run `grep -c <each-agent's-distinctive-token>
|
|
82
|
+
<file>` and assert all expected tokens are present. Example for
|
|
83
|
+
`autonomy/loki` post-Dev3+4+5: `grep -c "init-rules" == 4 AND
|
|
84
|
+
grep -c "cmd_doctor_json" == 2 AND grep -c "cmd_dashboard_help\|cmd_web_help"
|
|
85
|
+
== 7`. Three positive assertions catch overwrite regressions in seconds.
|
|
86
|
+
|
|
87
|
+
4. **Pre-integration conflict matrix.** Before any merging, the integrator
|
|
88
|
+
must compute and log which files multiple agents touched. Run:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
for branch in $(git branch --list 'worktree-agent-*'); do
|
|
92
|
+
git diff --name-only main...$branch
|
|
93
|
+
done | sort | uniq -d
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Output is a list of files multiple agents touched. Treat any file in
|
|
97
|
+
that list as a STOP signal: do not proceed to merge until a conflict
|
|
98
|
+
resolution plan is documented (merge order, structural checksum tokens,
|
|
99
|
+
expected post-merge grep counts). In v7.5.15 this would have surfaced
|
|
100
|
+
`autonomy/loki` (Dev3+4+5) and `autonomy/run.sh` (Dev1+6) before the
|
|
101
|
+
first merge command ran.
|
|
102
|
+
|
|
103
|
+
## 2. R3 returned a fragment on first dispatch
|
|
104
|
+
|
|
105
|
+
### What happened
|
|
106
|
+
|
|
107
|
+
R3 was launched in parallel with R1, R2, DA. After 90s, R3 returned this
|
|
108
|
+
literal text as its complete output:
|
|
109
|
+
|
|
110
|
+
> "Still running. Let me wait for monitor."
|
|
111
|
+
|
|
112
|
+
Status said "completed" but the result body was a 7-word fragment. R3 had not
|
|
113
|
+
actually run any of the cross-cutting integration checks I asked for.
|
|
114
|
+
|
|
115
|
+
I re-spawned R3 (call it R3-retry) with a stricter prompt. R3-retry completed
|
|
116
|
+
in 61s with a full structured 6-risk verdict.
|
|
117
|
+
|
|
118
|
+
### What signal in the original prompt caused the fragment
|
|
119
|
+
|
|
120
|
+
The original R3 prompt opened with this paragraph (truncated):
|
|
121
|
+
|
|
122
|
+
> "You are Reviewer 3 of 3 (integration safety) reviewing Loki Mode v7.5.15
|
|
123
|
+
> release candidate at /Users/lokesh/git/loki-mode.
|
|
124
|
+
>
|
|
125
|
+
> ## Context
|
|
126
|
+
> 8 parallel dev agents wrote in isolated worktrees. Their work was merged
|
|
127
|
+
> into main. The risk that motivates your existence: agents could not see
|
|
128
|
+
> each other's code, so subtle interactions between their patches may have
|
|
129
|
+
> been missed.
|
|
130
|
+
>
|
|
131
|
+
> ## Specific cross-cutting risks to investigate
|
|
132
|
+
>
|
|
133
|
+
> ### Risk A: autonomy/run.sh has Dev1 (iteration loop) + Dev6 (pytest timeout) edits
|
|
134
|
+
> [...command suggestions, not literal commands...]"
|
|
135
|
+
|
|
136
|
+
The smoking gun is the framing of the risk sections. They were narrative
|
|
137
|
+
("Dev1 added helpers near `run_autonomous()`") and asked the agent to
|
|
138
|
+
"investigate". The agent interpreted this as a research-and-report task with
|
|
139
|
+
async dependencies, hence "Let me wait for monitor."
|
|
140
|
+
|
|
141
|
+
The R3-retry prompt opened with:
|
|
142
|
+
|
|
143
|
+
> "CRITICAL: Run all the bash commands below yourself. Do NOT wait for any
|
|
144
|
+
> monitor. Do NOT spawn other tools. Just execute the commands, capture
|
|
145
|
+
> output, and report."
|
|
146
|
+
>
|
|
147
|
+
> Then: literal `bash` blocks for each risk, not narrative descriptions.
|
|
148
|
+
|
|
149
|
+
### The prompt-difference rule
|
|
150
|
+
|
|
151
|
+
The fragment was a context-window-limit hallucination triggered by the
|
|
152
|
+
narrative framing, not a model error. The first prompt let R3 think it had
|
|
153
|
+
to coordinate with other agents (the word "monitor" doesn't appear in my
|
|
154
|
+
prompt -- the agent invented it). The second prompt removed any room for
|
|
155
|
+
coordination assumption by handing literal commands.
|
|
156
|
+
|
|
157
|
+
**Rule for reviewer prompts: hand the agent literal commands when the work is
|
|
158
|
+
verification, not investigation.** Narrative-style "investigate this risk"
|
|
159
|
+
prompts work for open-ended research agents. They fail for verification
|
|
160
|
+
agents because the agent has nothing to do except run the check, and any
|
|
161
|
+
narrative slack invites hallucinated work-coordination.
|
|
162
|
+
|
|
163
|
+
Concretely:
|
|
164
|
+
- Bad: "Verify Dev1 and Dev6's edits to autonomy/run.sh do not conflict"
|
|
165
|
+
- Good: "Run `bash -n autonomy/run.sh && shellcheck -S error autonomy/run.sh`. Report exact output. If non-zero, paste the failure."
|
|
166
|
+
|
|
167
|
+
The signal was that R3's prompt had 5x more narrative than R1's or R2's
|
|
168
|
+
prompts. R1 and R2 returned substantive results; R3 returned a fragment. Same
|
|
169
|
+
model, same session, same parallel dispatch -- the only variable was prompt
|
|
170
|
+
density.
|
|
171
|
+
|
|
172
|
+
### Specific recommendation
|
|
173
|
+
|
|
174
|
+
Add a checklist to the reviewer-prompt template:
|
|
175
|
+
|
|
176
|
+
1. Does each verification step have a literal `bash` block, not a description?
|
|
177
|
+
2. Is the expected output specified concretely (e.g. "expect count = 3", not
|
|
178
|
+
"verify both helpers exist")?
|
|
179
|
+
3. Is there a "DO NOT spawn other agents / DO NOT wait for monitors" line?
|
|
180
|
+
4. Is the report format an enumerable list, not a narrative essay?
|
|
181
|
+
|
|
182
|
+
If any of these are no, the prompt invites fragment responses.
|
|
183
|
+
|
|
184
|
+
Add `scripts/lint-reviewer-prompt.sh` as the pre-flight linting artifact.
|
|
185
|
+
The script takes a prompt file as its only argument and grep-checks for the
|
|
186
|
+
4 required elements above:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
#!/usr/bin/env bash
|
|
190
|
+
# scripts/lint-reviewer-prompt.sh <prompt-file>
|
|
191
|
+
set -euo pipefail
|
|
192
|
+
FILE="${1:?usage: lint-reviewer-prompt.sh <prompt-file>}"
|
|
193
|
+
PASS=0
|
|
194
|
+
grep -q '```bash' "$FILE" || { echo "FAIL: no literal bash block"; PASS=1; }
|
|
195
|
+
grep -qE '(expect|count|== [0-9])' "$FILE" || { echo "FAIL: no concrete expected output"; PASS=1; }
|
|
196
|
+
grep -q 'DO NOT' "$FILE" || { echo "FAIL: no DO NOT line"; PASS=1; }
|
|
197
|
+
grep -qE '^\s*[0-9]+\.' "$FILE" || { echo "FAIL: no enumerable report format"; PASS=1; }
|
|
198
|
+
[ "$PASS" -eq 0 ] && echo "PASS: prompt lint OK"
|
|
199
|
+
exit "$PASS"
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
Invocation point: the integrator runs this script against each reviewer
|
|
203
|
+
prompt file before the parallel dispatch call. A non-zero exit aborts the
|
|
204
|
+
dispatch. The script does not need to ship in v7.5.15; the spec and
|
|
205
|
+
acceptance criteria above are sufficient to implement it before the next
|
|
206
|
+
fleet operation.
|
|
207
|
+
|
|
208
|
+
## 3. Devil's Advocate caught what 4 prior reviewers missed
|
|
209
|
+
|
|
210
|
+
### What happened
|
|
211
|
+
|
|
212
|
+
Three reviewers (R1 correctness, R2 CLAUDE.md compliance, R3 integration
|
|
213
|
+
safety) reviewed v7.5.15 and unanimously approved. Each independently re-ran
|
|
214
|
+
the 8 new test suites and confirmed PASS.
|
|
215
|
+
|
|
216
|
+
Devil's Advocate (DA) ran the same checks AND added one more: did the new
|
|
217
|
+
tests actually get registered in `tests/run-all-tests.sh`? Answer: 1 of 8 was
|
|
218
|
+
registered. The other 7 (Dev1, Dev2, Dev3, Dev4, Dev5, Dev6, Dev7) would
|
|
219
|
+
silently rot.
|
|
220
|
+
|
|
221
|
+
Cost if missed: 7 tests in the repo for diff inspectors to think coverage
|
|
222
|
+
existed, while CI never ran them. Future regressions in those code paths
|
|
223
|
+
would not be caught until the next manual run-all-tests.sh invocation.
|
|
224
|
+
|
|
225
|
+
### What this says about reviewer role structure
|
|
226
|
+
|
|
227
|
+
R1, R2, R3 each had a specific mandate. R1 = "do the patches work?". R2 =
|
|
228
|
+
"does the release follow CLAUDE.md?". R3 = "do the patches integrate without
|
|
229
|
+
breaking each other?". All three mandates assumed the question "does this
|
|
230
|
+
release ship?" was answered if their narrow domain passed.
|
|
231
|
+
|
|
232
|
+
DA had no domain. DA was given the role "find what the other three will
|
|
233
|
+
miss." That open mandate let DA ask a question outside any of the other three
|
|
234
|
+
roles' jurisdictions: "given that the patches work, are they wired up to
|
|
235
|
+
keep working?"
|
|
236
|
+
|
|
237
|
+
Test-rot is in nobody's R1/R2/R3 domain:
|
|
238
|
+
- R1 ran the tests -- they pass when invoked. Mandate satisfied.
|
|
239
|
+
- R2 checked CLAUDE.md compliance -- which doesn't say "wire all new tests
|
|
240
|
+
into the runner". Mandate satisfied.
|
|
241
|
+
- R3 checked cross-file integration -- not test runner registration.
|
|
242
|
+
Mandate satisfied.
|
|
243
|
+
|
|
244
|
+
Test-rot is structurally outside specialized reviewer domains. It is a
|
|
245
|
+
meta-question about durability that requires a non-domain-specific role to
|
|
246
|
+
ask.
|
|
247
|
+
|
|
248
|
+
### Should DA be the default?
|
|
249
|
+
|
|
250
|
+
No. The DA role works because it is contrarian to a specific quorum. If you
|
|
251
|
+
make DA the default reviewer, you lose the contrarian frame -- DA becomes
|
|
252
|
+
just another R-reviewer with a slightly broader mandate, and it stops asking
|
|
253
|
+
the meta-questions because it is now responsible for them.
|
|
254
|
+
|
|
255
|
+
The surprise factor is the structural feature, not a bug. R1+R2+R3 unanimous
|
|
256
|
+
approval triggers DA. If DA agreed without conditions, the release ships. If
|
|
257
|
+
DA finds something, the release pauses.
|
|
258
|
+
|
|
259
|
+
### What to keep, what to add
|
|
260
|
+
|
|
261
|
+
**Keep:**
|
|
262
|
+
- DA as a separate role spawned in the same parallel wave as R1/R2/R3, with
|
|
263
|
+
the explicit mandate "find what the others will miss".
|
|
264
|
+
- The 3-reviewer + DA pattern as the default for any release.
|
|
265
|
+
|
|
266
|
+
**Add:**
|
|
267
|
+
- A required DA checklist with at least these items, refined every release:
|
|
268
|
+
- Are new tests registered in the runner?
|
|
269
|
+
- Are new env flags documented in `loki doctor` output AND in CHANGELOG?
|
|
270
|
+
- Are new endpoints discoverable via `loki <thing> --help` output?
|
|
271
|
+
- Did any agent claim "tests pass" without me re-running it from a fresh
|
|
272
|
+
shell?
|
|
273
|
+
- Did the integration retain ALL agents' contributions, or did one
|
|
274
|
+
silently overwrite another?
|
|
275
|
+
|
|
276
|
+
The fifth item would have caught the bad-copy regression (section 1) earlier
|
|
277
|
+
than my own grep-c sanity check did.
|
|
278
|
+
|
|
279
|
+
### Specific recommendation
|
|
280
|
+
|
|
281
|
+
Codify DA as a required role with a published checklist. Currently DA's
|
|
282
|
+
prompt was hand-written for v7.5.15 and emphasized "find at least one thing
|
|
283
|
+
the 3 reviewers will miss". That phrasing worked but is fragile -- the next
|
|
284
|
+
DA invocation might emphasize different things and miss the test-rot class
|
|
285
|
+
of issue.
|
|
286
|
+
|
|
287
|
+
Concrete: maintain `docs/retrospectives/devil-advocate-checklist.md` as the
|
|
288
|
+
standing DA questions. Reviewer prompts inline the checklist contents at
|
|
289
|
+
dispatch time (copy-paste or heredoc). New questions get appended to the
|
|
290
|
+
checklist after each release where DA found something the other reviewers
|
|
291
|
+
missed.
|
|
292
|
+
|
|
293
|
+
The checklist becomes the institutional memory of "things 3-reviewer councils
|
|
294
|
+
have failed to catch." It grows monotonically. DA's job becomes "run this
|
|
295
|
+
checklist + add anything new" rather than "be smart in a vacuum".
|
|
296
|
+
|
|
297
|
+
## Summary of recommended protocol changes
|
|
298
|
+
|
|
299
|
+
1. **Dispatch protocol**: dev-agent prompts explicitly resolve commit-vs-wait;
|
|
300
|
+
integrator uses `git merge` exclusively; post-merge structural checksums on
|
|
301
|
+
conflict-prone files; pre-integration conflict matrix.
|
|
302
|
+
2. **Reviewer prompts**: literal bash blocks for verification work; "DO NOT
|
|
303
|
+
spawn / DO NOT wait" preamble; expected output specified concretely;
|
|
304
|
+
pre-flight prompt linting.
|
|
305
|
+
3. **Council structure**: keep DA as a contrarian role with a growing
|
|
306
|
+
checklist; add the standing DA questions to a versioned checklist file;
|
|
307
|
+
monotonically grow the list with new "things the council missed" each
|
|
308
|
+
release.
|
|
309
|
+
|
|
310
|
+
These are 3 small protocol changes, each addressing a specific failure that
|
|
311
|
+
occurred in this session. None require new tools or new architecture.
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
**Footnote -- two additional observed events not elevated to protocol-change tier.**
|
|
316
|
+
Two issues occurred during the v7.5.15 session that were deliberately excluded
|
|
317
|
+
from the three-failure selection above: (1) the validate-bash hook fired a
|
|
318
|
+
false-positive on a `rm -rf /Users/...` path in a comment, not an executed
|
|
319
|
+
command; (2) a zsh PATH hash quirk caused a freshly-added binary to be
|
|
320
|
+
invisible until `hash -r` was run in the same shell. Both were observed,
|
|
321
|
+
classified as audit-table-tier (one-time environment quirks requiring no
|
|
322
|
+
standing protocol change), and are covered in the session audit table rows
|
|
323
|
+
"hook-false-positive-rm-path" and "zsh-path-hash-stale" respectively. Neither
|
|
324
|
+
met the bar for a protocol change because neither was reproducible across
|
|
325
|
+
sessions or attributable to a protocol gap.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# v7.5.15 Honesty Audit
|
|
2
|
+
|
|
3
|
+
Audit of the v7.5.15 release commit `c82b9541` against the actual diff and the
|
|
4
|
+
8-agent fleet's per-PR claims.
|
|
5
|
+
|
|
6
|
+
Method: every CHANGELOG sentence cross-checked against `git show c82b9541` +
|
|
7
|
+
`git log 2ce36624..HEAD` + grep against the integrated tree. No agent's
|
|
8
|
+
self-report taken as evidence.
|
|
9
|
+
|
|
10
|
+
## 1. CHANGELOG claims vs. diff evidence
|
|
11
|
+
|
|
12
|
+
| CHANGELOG claim | Diff evidence | Verdict |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| Sentrux iteration-loop wire-in behind `LOKI_SENTRUX_GATE=1` | `autonomy/run.sh:10531` `_loki_sentrux_iteration_start()`, `:10543` `_loki_sentrux_iteration_end()`, callers at `:10775` and `:11257`, env-flag guard verified | VERIFIED |
|
|
15
|
+
| Default off; zero behavior change without opt-in | Guard pattern: `if [[ "${LOKI_SENTRUX_GATE:-}" == "1" ]] && command -v sentrux ...`. Confirmed at the call sites. | VERIFIED |
|
|
16
|
+
| `tests/test-sentrux-iteration-wireup.sh` 7/7 PASS | Re-run during integration: 7/7 PASS. Re-run by R1, DA: 7/7. | VERIFIED |
|
|
17
|
+
| Dashboard `/api/quality/architecture` endpoint | `dashboard/server.py:5958` `@app.get("/api/quality/architecture")` confirmed | VERIFIED |
|
|
18
|
+
| Returns sorted findings-sentrux series | `dashboard/server.py:5974-6014` globs `findings-sentrux-*.json`, sorts by iteration | VERIFIED |
|
|
19
|
+
| Resilient to corrupt JSON | Test 4 in pytest suite (`test_resilient_to_corrupt_file`); writer-side `try/except` confirmed at `dashboard/server.py:5995-6014` | VERIFIED |
|
|
20
|
+
| `tests/dashboard/test_quality_architecture_endpoint.py` 5/5 PASS | Direct `pytest` re-run: 5/5 PASS. Wrapper script also confirms. | VERIFIED |
|
|
21
|
+
| `loki sentrux init-rules` scaffolds `.sentrux/rules.toml` | `autonomy/loki:7114` `init-rules)` case + template heredoc starting `:7130` | VERIFIED |
|
|
22
|
+
| Refuse-overwrite unless `--force` | `autonomy/loki:7122-7124` shows the friendly refusal + `--force` flag parsing at `:7045` | VERIFIED |
|
|
23
|
+
| `tests/test-sentrux-init-rules.sh` 9/9 PASS | DA re-run: 9/9 PASS | VERIFIED |
|
|
24
|
+
| Doctor `--json` sentrux entry, parity bash + Bun | `autonomy/loki cmd_doctor_json` python emit (Dev4 `f7e95625` brought 15 lines) + `loki-ts/src/commands/doctor.ts:44 SentruxCheck`, `:54 sentrux: SentruxCheck` in DoctorJson, `:310 checkSentrux()` | VERIFIED |
|
|
25
|
+
| Byte-identical bash vs Bun parity | `tests/test-doctor-json-sentrux.sh` test 13 explicitly runs both routes through `jq -S` then `diff -q`. Test passed. local-ci bun-parity matrix also passed (21/21). | VERIFIED |
|
|
26
|
+
| `tests/test-doctor-json-sentrux.sh` 13/13 PASS | Reviewer + DA re-run: 13/13 | VERIFIED |
|
|
27
|
+
| Dashboard nav UAT -- Escalations sidebar component | `dashboard-ui/components/loki-escalations.js` (NEW, 280 lines), exported in `dashboard-ui/index.js:99`, mounted in `dashboard-ui/scripts/build-standalone.js:798` | VERIFIED |
|
|
28
|
+
| Web vs dashboard help clarification | `autonomy/loki cmd_dashboard_help` and `cmd_web_help` -- confirmed both contain "Note:" blocks cross-referencing the other command. R3 re-ran: 2 matches each. | VERIFIED |
|
|
29
|
+
| `tests/test-dashboard-nav-uat.sh` 13/13 PASS | DA re-run: 13/13 | VERIFIED |
|
|
30
|
+
| Pytest gate timeout via `_loki_run_pytest_with_timeout` helper | `autonomy/run.sh:5928` defines helper, called from `:6065` inside `enforce_test_coverage` | VERIFIED |
|
|
31
|
+
| Configurable via `LOKI_PYTEST_TIMEOUT` (default 300s) | Confirmed in helper body | VERIFIED |
|
|
32
|
+
| Closes Triage #14 | Triage #14 was "pytest gate timeout wrapper" per v7.5.12 CHANGELOG. The helper is the wrapper. | VERIFIED |
|
|
33
|
+
| `tests/test-pytest-gate-timeout.sh` 5/5 PASS | DA re-run: 5/5. Real exit-code-124 detection verified by Dev6's test fixture. | VERIFIED |
|
|
34
|
+
| Per-file try/except in `memory/storage.py:_load_json` catches JSONDecodeError, UnicodeDecodeError, OSError | `memory/storage.py:354 except json.JSONDecodeError`, `:360 except UnicodeDecodeError`, `:366 except (OSError, UnicodeDecodeError)` | VERIFIED |
|
|
35
|
+
| Closes Triage #15 | Triage #15 was "episode JSON try/except per file". Centralized fix in `_load_json` covers all callers (`engine.py`, `retrieval.py`, `consolidation.py`). | VERIFIED |
|
|
36
|
+
| `tests/memory/test_episode_load_resilience.py` 8/8 PASS | DA re-run: 8/8 | VERIFIED |
|
|
37
|
+
| `tests/test-sentrux-gate.sh` wired into Linux runner | `tests/run-all-tests.sh:78 run_test "Sentrux Gate Unit Tests"` -- present | VERIFIED |
|
|
38
|
+
| `.github/workflows/sentrux-real.yml` gates real-binary test as manual/scheduled | New workflow file present; triggers `workflow_dispatch` and cron only; `continue-on-error: true` | VERIFIED |
|
|
39
|
+
| `tests/test-ci-sentrux-coverage.sh` 4/4 PASS | DA re-run: 4/4 | VERIFIED |
|
|
40
|
+
| All 7 new v7.5.15 test suites wired into `tests/run-all-tests.sh` | `tests/run-all-tests.sh:85-99` -- 5 bash entries + 2 pytest wrapper entries (gated on `python3 + import pytest`) | VERIFIED |
|
|
41
|
+
| Final runner: 24/25 PASS | `bash tests/run-all-tests.sh` second run: `Tests Run: 25, Passed: 24, Failed: 1` | VERIFIED |
|
|
42
|
+
| Single failure is pre-existing pip install mcp env gap | `python3 -c "import mcp"` reproduces the same `MCP SDK not found` error on bare Python; not introduced by this release | VERIFIED |
|
|
43
|
+
| 13 version locations bumped (vscode-extension intentionally skipped) | `grep "7\.5\.15"` across the 13 expected files: all hit. `vscode-extension/package.json` still at older version (per CLAUDE.md v7.2.0 deprecation note) | VERIFIED |
|
|
44
|
+
| local-ci 21/21 PASS | Final pre-push run: `Passed: 21, Failed: 0`. Re-confirmed post-merge twice. | VERIFIED |
|
|
45
|
+
|
|
46
|
+
**Result: Every CHANGELOG bullet was checked individually against diff or re-run evidence. The table above has 30 rows, one per CHANGELOG sentence. Zero claims softened. No claim shipped without proof.**
|
|
47
|
+
|
|
48
|
+
Note: The CHANGELOG body in v7.5.15 reads "20/20 PASS" for local-ci. That count reflects an intermediate state before the dist rebuild step was added as a final check. After the rebuild, the runner read 21/21. The 20/20 figure was accurate at the time it was written; 21/21 is the final pre-push count.
|
|
49
|
+
|
|
50
|
+
Scope gap -- `scripts/local-ci.sh` (4 lines, modified in v7.5.15): this script was changed but is not represented by a row in the table above. The change added one check (the dist rebuild probe). The diff is trivial but the omission is noted here for completeness.
|
|
51
|
+
|
|
52
|
+
Scope gap -- `dashboard/static/index.html` (318 lines, build artifact): the table confirms the Escalations component was mounted in the build script. It does not confirm that the built artifact is internally consistent beyond that. A full artifact diff was not performed.
|
|
53
|
+
|
|
54
|
+
## 2. Per-agent claim audit
|
|
55
|
+
|
|
56
|
+
For each of the 8 dev agents: did the agent's report overshoot what they actually shipped?
|
|
57
|
+
|
|
58
|
+
### Dev1 (sentrux iteration wire-in)
|
|
59
|
+
- Claimed: 7/7 PASS, helpers extracted into `_loki_sentrux_iteration_start/_end`, defensive numeric guard on before/after, only patched `track_iteration_complete "$ITERATION_COUNT" "$exit_code"` site.
|
|
60
|
+
- Shipped: confirmed all of the above. Agent scoped to one call site. Second site (line 10843) not patched. CHANGELOG does not claim it.
|
|
61
|
+
- **Overshoot: NONE.**
|
|
62
|
+
|
|
63
|
+
### Dev2 (dashboard endpoint)
|
|
64
|
+
- Claimed: 5 tests including resilience to corrupt JSON, 2 helper functions, copied existing `_ForceLokiDir` test pattern.
|
|
65
|
+
- Shipped: confirmed in `dashboard/server.py:5958-6020`. 5 pytest assertions present.
|
|
66
|
+
- **Overshoot: NONE.**
|
|
67
|
+
|
|
68
|
+
### Dev3 (init-rules)
|
|
69
|
+
- Claimed: 9/9 PASS, modified `cmd_sentrux` only, lines 7012-7150.
|
|
70
|
+
- Shipped: actual diff shows 7029-7183 (small drift from the agent's claim). Behavior matches: `--force` flag, friendly refusal on overwrite, scaffolds the right template.
|
|
71
|
+
- **Overshoot: NONE. Line numbers in agent's claim (7012-7150) differ from shipped range (7029-7183). Behavior matches. Cause: pre/post-merge offset.**
|
|
72
|
+
|
|
73
|
+
### Dev4 (doctor JSON parity)
|
|
74
|
+
- Claimed: 13/13 PASS including byte-identical parity test, dist artifact rebuilt but not staged because `loki-ts/dist/loki.js` is gitignored.
|
|
75
|
+
- Shipped: confirmed `SentruxCheck` type at `loki-ts/src/commands/doctor.ts:44`, `checkSentrux()` at `:310`. Parity test still passes after my own dist rebuild during integration (because the merge brought source change but dist was stale until I rebuilt).
|
|
76
|
+
- **Overshoot: NONE. Agent noted dist gitignore. Consistent with actual behavior.**
|
|
77
|
+
|
|
78
|
+
### Dev5 (dashboard nav UAT)
|
|
79
|
+
- Claimed: 13/13 PASS, escalations component is real (server-side `/api/escalations` already existed; UI was the gap), web vs dashboard are NOT aliases (different ports/products), explicitly punted on item #2 (parent-shell exit dependency).
|
|
80
|
+
- Shipped: confirmed via grep of dashboard/server.py:5977 (`/api/escalations`), the new component file, and the help-text additions. Confirmed: web and dashboard serve different ports. Agent claim accurate.
|
|
81
|
+
- **Overshoot: NONE. Agent explicitly marked item #2 out of scope. Not addressed in this release.**
|
|
82
|
+
|
|
83
|
+
### Dev6 (pytest gate timeout)
|
|
84
|
+
- Claimed: 5/5 PASS, found gate at `autonomy/run.sh:6041` (now 6053-6066), extracted helper for testability, mentioned "same hang risk in other gates" (go test, cargo test, monorepo test cmd) but did NOT fix them.
|
|
85
|
+
- Shipped: confirmed helper at `:5928`, called from `:6065`. The "didn't fix other gates" disclosure is honest scope-limitation.
|
|
86
|
+
- **Overshoot: NONE.**
|
|
87
|
+
|
|
88
|
+
### Dev7 (episode JSON resilience)
|
|
89
|
+
- Claimed: 8/8 PASS, single point of fix at `memory/storage.py:328 _load_json`, all higher-level callers inherit, did NOT add rename-on-corrupt pattern (no codebase precedent), listed out-of-scope sites at `cross_project.py:101` etc.
|
|
90
|
+
- Shipped: confirmed via grep showing the 3 expected exception types caught at lines 354/360/366. Out-of-scope sites correctly left alone.
|
|
91
|
+
- **Overshoot: NONE.**
|
|
92
|
+
|
|
93
|
+
### Dev8 (CI coverage)
|
|
94
|
+
- Claimed: 4/4 PASS, did NOT commit (applied CLAUDE.md "wait for user approval"), found 22 workflows, sentrux-real.yml is `continue-on-error: true` and not blocking.
|
|
95
|
+
- Shipped: confirmed. Dev8 did not commit. Integrator staged Dev8's files manually.
|
|
96
|
+
- **Overshoot: NONE.**
|
|
97
|
+
|
|
98
|
+
**Result: 0 of 8 agents overshot their PR description. The fleet was honest.**
|
|
99
|
+
|
|
100
|
+
## 3. Session integrity score vs CLAUDE.md
|
|
101
|
+
|
|
102
|
+
### Where we got it right
|
|
103
|
+
|
|
104
|
+
- **Pre-push local-ci**: Ran 3 separate times (post-Dev3+4 integration, post-DA-fix, final). Caught 1 transient flake. Final: 21/21.
|
|
105
|
+
- **No `git add -A`**: All 25 release files staged by name. R2 explicitly warned about `.claude/worktrees/` sweep risk; I avoided it. The 2 pre-existing untracked docs (`docs/MIGRATION-STATUS.md`, `docs/SOFTWARE-FACTORY-ANALYSIS.md`) remain untracked.
|
|
106
|
+
- **HEREDOC commit message**: Used. No co-author. Honest "NOT in this release" section enumerated 7 deferred items.
|
|
107
|
+
- **14 version locations**: 13 bumped (vscode-extension deprecated per CLAUDE.md v7.2.0 -- documented honestly).
|
|
108
|
+
- **No emojis, no em dashes**: R2 verified zero hits in the diff against both unicode ranges.
|
|
109
|
+
- **3-reviewer council + DA**: Used. DA caught a real concern (test rot) that the 3 reviewers missed. Concern was addressed BEFORE commit, not deferred.
|
|
110
|
+
- **Cleanup before commit**: All 8 worktrees + branches removed via `git worktree remove -f -f` + `git branch -D`. `/tmp/loki-*` cleared. `ps -ef` clean.
|
|
111
|
+
- **Pre-existing untracked left alone**: `docs/MIGRATION-STATUS.md` and `docs/SOFTWARE-FACTORY-ANALYSIS.md` not staged. They were here at session start; not my work to commit.
|
|
112
|
+
- **Post-publish smoke test**: claimed, not evidenced inline. Commands run were: `npm install -g loki-mode@7.5.15 && loki version`, `docker pull asklokesh/loki-mode:7.5.15 && docker run --rm asklokesh/loki-mode:7.5.15 version`, and a WebFetch of the Homebrew formula to confirm sha256. Output not captured in this doc.
|
|
113
|
+
|
|
114
|
+
### Where we fell short
|
|
115
|
+
|
|
116
|
+
| Issue | Detail | Cost |
|
|
117
|
+
|---|---|---|
|
|
118
|
+
| Bad-copy regression during integration | When I `cp`'d Dev5's `autonomy/loki` into the integrated tree, I obliterated Dev3+Dev4's `cmd_sentrux` and `cmd_doctor_json` changes. Caught immediately by `grep -c init-rules` returning 0 instead of 4. Fixed by `git checkout HEAD -- autonomy/loki` then surgical `Edit` of just Dev5's two help blocks. Cost if unnoticed: shipped a release that silently regresses Dev3+Dev4's features. | Caught locally; user-facing impact = 0. Lesson: never bulk-`cp` over an integrated multi-author file. |
|
|
119
|
+
| R3 returned a non-substantive fragment first time | First R3 invocation returned the literal string "Still running. Let me wait for monitor." instead of executing. Re-spawned with stricter "Run all the bash commands below yourself. Do NOT wait for any monitor." preamble. Second attempt completed in 61s with 6 verifications. Cost if I hadn't noticed: missing the integration-safety verdict. | Caught by reading the output; cost = ~5 min retry. |
|
|
120
|
+
| Dev5 + Dev8 didn't commit in their worktrees | Both correctly applied the global "wait for user approval" rule, which conflicted with the agent task spec's "commit at end". Their files were uncommitted in the worktree, so I had to manually `cp` 4 + 4 = 8 files. Worked but added integration risk (see bad-copy regression above). | Cost = the bad-copy regression. Lesson: dev-agent prompts should state explicitly whether commits are part of the worktree task or whether the integrator commits centrally. |
|
|
121
|
+
| Bun `dist` was stale post-merge | Dev4 modified `loki-ts/src/commands/doctor.ts` and the merge brought the source. But `dist/loki.js` is gitignored and was not rebuilt in the worktree. Bun-route doctor JSON test failed (5/13) until I ran `cd loki-ts && bun run build`. Caught by my own pre-commit test sweep. | Cost = ~2 min rebuild + retest. Lesson: integration must rebuild `dist` after any `loki-ts/src/` merge. |
|
|
122
|
+
| Devil's Advocate caught test rot 4 reviewers missed | 7 of 8 new tests were not registered in `tests/run-all-tests.sh` -- only `test-ci-sentrux-coverage.sh` (Dev8's own) was. The 3 R-reviewers all confirmed "tests pass" by running them directly, but didn't ask "will they run in CI tomorrow?". DA asked the right question. | Caught pre-commit. If missed: 7 tests would have silently rotted. They are now wired in (24/25 PASS via runner). |
|
|
123
|
+
| First `bash tests/run-all-tests.sh` invocation captured truncated output | I piped the runner through `tail -15` which discarded all the per-test stdout. Re-ran without `tail` to get full per-test PASS/FAIL marks. | Caught immediately. Lesson: never `tail` a test runner you're trying to debug. |
|
|
124
|
+
| First `git worktree remove` attempt failed | 8 worktrees were `locked` (per the auto-isolation lock). Initial `git worktree remove --force` returned `use 'remove -f -f' to override or unlock first`. Re-ran with `-f -f`. | Caught immediately. Cost = ~30s retry. |
|
|
125
|
+
| MCP test failure shipped as known caveat | `python3 -c "import mcp"` fails on this Mac (`pip install mcp` not done). Pre-existing, not introduced by v7.5.15. CHANGELOG documents this honestly. | Pre-existing. CI runs in environments where `mcp` is installed; not a release blocker. |
|
|
126
|
+
|
|
127
|
+
### Patterns to repeat
|
|
128
|
+
|
|
129
|
+
- 8-agent fleet with isolated worktrees: 8 features shipped in ~5 min wall time vs ~2 days serial.
|
|
130
|
+
- Devil's Advocate as a separate role from the 3 standard reviewers: the 3 reviewers focused on "did the patch work?" and missed "will the patch keep working?". DA caught it.
|
|
131
|
+
- "NOT in this release" CHANGELOG section: agents and integrator both named deferred items explicitly.
|
|
132
|
+
- Per-file `git add`: avoided sweeping `.claude/worktrees/` into the release commit.
|
|
133
|
+
|
|
134
|
+
## Net verdict
|
|
135
|
+
|
|
136
|
+
**Release c82b9541: 30 claims evidenced. 0 agents overshot. Failures: bad-copy regression during integration, test-rot caught by DA, stale dist after loki-ts/src merge, truncated runner output from `tail`. Lessons: no bulk-`cp` over multi-author files; explicit commit-vs-wait dispatch rule in agent prompts; rebuild dist after loki-ts/src merges; never `tail` a test runner you're debugging.**
|