loki-mode 10.5.17 → 10.5.19

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (125) hide show
  1. package/README.md +877 -30
  2. package/SKILL.md +3 -3
  3. package/VERSION +1 -1
  4. package/autonomy/lib/proof-generator.py +251 -0
  5. package/autonomy/lib/proof-verify.py +1 -0
  6. package/autonomy/loki +52 -6
  7. package/autonomy/run.sh +27 -2
  8. package/dashboard/__init__.py +1 -1
  9. package/dashboard/api_start.py +463 -0
  10. package/dashboard/server.py +31 -0
  11. package/dashboard/static/start.html +100 -0
  12. package/docs/COMPARISON.md +6 -19
  13. package/docs/COMPETITIVE-ANALYSIS.md +6 -41
  14. package/docs/CRASH-REPORTING-PLAN.md +1 -1
  15. package/docs/ENTERPRISE-IDENTITY-ROADMAP.md +1 -1
  16. package/docs/INSTALLATION.md +1 -1
  17. package/docs/PRIVACY.md +1 -1
  18. package/docs/R8-SHAREABLE-TEAM-ASSETS-PLAN.md +1 -1
  19. package/docs/WANG-PRINCIPLES-PLAN.md +1 -1
  20. package/docs/architecture/STATE-MACHINES.md +18 -27
  21. package/docs/architecture/bmad-integration-validation.md +2 -2
  22. package/docs/audit-logging.md +12 -86
  23. package/docs/authentication.md +4 -4
  24. package/docs/authorization.md +15 -47
  25. package/docs/auto-claude-comparison.md +1 -33
  26. package/docs/certification/01-core-concepts/lab.md +3 -3
  27. package/docs/certification/01-core-concepts/lesson.md +11 -22
  28. package/docs/certification/01-core-concepts/quiz.md +7 -7
  29. package/docs/certification/02-enterprise-features/lab.md +0 -2
  30. package/docs/certification/02-enterprise-features/lesson.md +0 -10
  31. package/docs/certification/04-production-deployment/lab.md +1 -1
  32. package/docs/certification/05-troubleshooting/lab.md +2 -2
  33. package/docs/certification/README.md +1 -1
  34. package/docs/certification/answer-key.md +37 -20
  35. package/docs/certification/certification-exam.md +7 -7
  36. package/docs/competitive/emergence-others-analysis.md +3 -3
  37. package/docs/competitive/replit-lovable-analysis.md +2 -2
  38. package/docs/cursor-comparison.md +2 -2
  39. package/docs/dashboard-guide.md +14 -2
  40. package/docs/dev/architecture-reference.md +2 -2
  41. package/docs/dev/project-structure.md +1 -1
  42. package/docs/enterprise/architecture.md +11 -8
  43. package/docs/enterprise/integration-cookbook.md +23 -14
  44. package/docs/enterprise/migration.md +5 -9
  45. package/docs/enterprise/performance.md +3 -6
  46. package/docs/enterprise/sdk-guide.md +1 -1
  47. package/docs/enterprise/security.md +8 -17
  48. package/docs/exit-codes.md +14 -0
  49. package/docs/git-workflow.md +4 -62
  50. package/docs/network-security.md +3 -13
  51. package/docs/plans/ULTRAPLAN-FORGE-BAAS.md +1 -1
  52. package/docs/prd-purple-lab-platform-v2.md +1 -1
  53. package/docs/prd-purple-lab-platform.md +1 -3
  54. package/docs/research-2026-07/GAP-ANALYSIS-BACKLOG.md +1 -1
  55. package/docs/show-hn-post.md +3 -4
  56. package/docs/siem-integration.md +8 -10
  57. package/docs/test-scenarios/edge-cases.md +2 -2
  58. package/docs/test-scenarios/enterprise-scenarios.md +4 -4
  59. package/docs/v10/BOARD.md +2 -0
  60. package/docs/v10/GUIDE.md +5 -3
  61. package/docs/v10/METRICS.md +36 -0
  62. package/docs/v10/PROGRESS.md +35 -0
  63. package/docs/walkthrough/README.md +1 -7
  64. package/loki-ts/dist/loki.js +370 -370
  65. package/mcp/__init__.py +1 -1
  66. package/package.json +1 -1
  67. package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
  68. package/web-app/dist/assets/{AdminPage-DxGZSpFg.js → AdminPage-BH_7fpoz.js} +1 -1
  69. package/web-app/dist/assets/{Avatar-BZEp23D8.js → Avatar-BzyhrX0s.js} +1 -1
  70. package/web-app/dist/assets/{Badge-DnDwqDLu.js → Badge-CL05xwgC.js} +1 -1
  71. package/web-app/dist/assets/{Button-C9KDP4c1.js → Button-BVHrGLGH.js} +1 -1
  72. package/web-app/dist/assets/{CockpitPage-DRlh3LV2.js → CockpitPage-B9BAEqD-.js} +1 -1
  73. package/web-app/dist/assets/{ComparePage-C_gE45dQ.js → ComparePage-D4grckhY.js} +1 -1
  74. package/web-app/dist/assets/{ErrorBoundary-CL-J9-Cs.js → ErrorBoundary-Bg52u7eg.js} +1 -1
  75. package/web-app/dist/assets/{EvidenceReceiptPanel-DABd176Y.js → EvidenceReceiptPanel-B15kRZbJ.js} +1 -1
  76. package/web-app/dist/assets/{GitHubIssuesPanel-C4lYbm6O.js → GitHubIssuesPanel-DP_BhJSC.js} +1 -1
  77. package/web-app/dist/assets/{GitHubPRsPanel-CgiUixwo.js → GitHubPRsPanel-DJBu83u9.js} +1 -1
  78. package/web-app/dist/assets/{HomePage-DC99RIrz.js → HomePage-6mw4uvLs.js} +2 -2
  79. package/web-app/dist/assets/{LoginPage-Bd72QWNk.js → LoginPage-BWWzFHif.js} +1 -1
  80. package/web-app/dist/assets/{MagicPage-CBouFs2K.js → MagicPage-CSyo7k5t.js} +1 -1
  81. package/web-app/dist/assets/{MetricsPage-c9MyDcBp.js → MetricsPage-Buq03cch.js} +1 -1
  82. package/web-app/dist/assets/{NotFoundPage-ZPhiebiW.js → NotFoundPage-BG8vuV4g.js} +1 -1
  83. package/web-app/dist/assets/{ProjectPage-DKwrALUb.js → ProjectPage-D7eJn--6.js} +3 -3
  84. package/web-app/dist/assets/{ProjectsPage-X1uZKZC_.js → ProjectsPage-DXPycvjA.js} +1 -1
  85. package/web-app/dist/assets/{SettingsPage-CiZKZRto.js → SettingsPage-Blvav87U.js} +1 -1
  86. package/web-app/dist/assets/{ShowcasePage-k_imKfeh.js → ShowcasePage-ZCAKRsNN.js} +1 -1
  87. package/web-app/dist/assets/{SystemSettingsPage-Wx9ESLll.js → SystemSettingsPage-ClzjKfI4.js} +1 -1
  88. package/web-app/dist/assets/{TeamsPage-B7upn-I7.js → TeamsPage-CKp1-WGX.js} +1 -1
  89. package/web-app/dist/assets/{TemplatesPage-oJf_VKgh.js → TemplatesPage-C0yAp2Yl.js} +1 -1
  90. package/web-app/dist/assets/{TerminalOutput-BdfPlgLI.js → TerminalOutput-C3KbV8E7.js} +1 -1
  91. package/web-app/dist/assets/{activity-BcZTsgWH.js → activity-CG5PtpVI.js} +1 -1
  92. package/web-app/dist/assets/{bell-JNEkjWQ2.js → bell-CyLp--NR.js} +1 -1
  93. package/web-app/dist/assets/{bot-DMAJobP_.js → bot-Dp0zY9t7.js} +1 -1
  94. package/web-app/dist/assets/{check-DPTry-z2.js → check-ipranz8L.js} +1 -1
  95. package/web-app/dist/assets/{chevron-left-DSqOEtTV.js → chevron-left-DhYs9D3t.js} +1 -1
  96. package/web-app/dist/assets/{circle-alert-BbL8Vysv.js → circle-alert-BLAxBNgz.js} +1 -1
  97. package/web-app/dist/assets/{clock-BHEr5lGW.js → clock-DzocnapF.js} +1 -1
  98. package/web-app/dist/assets/{cloud-C48PunYo.js → cloud-B3YsIUwq.js} +1 -1
  99. package/web-app/dist/assets/{code-xml-Boa7kRqT.js → code-xml-BPcy_Eqm.js} +1 -1
  100. package/web-app/dist/assets/{database-Eu35Gv7z.js → database-DOuuclZf.js} +1 -1
  101. package/web-app/dist/assets/{dollar-sign-BO9fCnHW.js → dollar-sign-o77cj-xB.js} +1 -1
  102. package/web-app/dist/assets/{file-code-corner-DHsSDIL_.js → file-code-corner-BZ2VMnYA.js} +1 -1
  103. package/web-app/dist/assets/{file-plus-BYiQLe5l.js → file-plus-DeHfGncj.js} +1 -1
  104. package/web-app/dist/assets/{globe-BPZ5sQI2.js → globe-DmIW-phf.js} +1 -1
  105. package/web-app/dist/assets/{hammer-DlOLTsH3.js → hammer-CoW8R8V5.js} +1 -1
  106. package/web-app/dist/assets/{index-BrgkuqSA.js → index-5rCTAnsF.js} +3 -3
  107. package/web-app/dist/assets/{layers-DMNdn1GR.js → layers-DGEk-2RQ.js} +1 -1
  108. package/web-app/dist/assets/{loader-circle-_e9fypHL.js → loader-circle-DXn-F-H5.js} +1 -1
  109. package/web-app/dist/assets/{lock-D_elxMZb.js → lock-D5nAjeUu.js} +1 -1
  110. package/web-app/dist/assets/{package-Bw-7g3JI.js → package-D8Drpzi8.js} +1 -1
  111. package/web-app/dist/assets/{plus-CZeq0pgJ.js → plus-C-0T9dAu.js} +1 -1
  112. package/web-app/dist/assets/{refresh-cw-D0WMyffs.js → refresh-cw-AyYQQPZG.js} +1 -1
  113. package/web-app/dist/assets/{rotate-ccw-CcKyNwXG.js → rotate-ccw-DYbslLfT.js} +1 -1
  114. package/web-app/dist/assets/{scroll-text-C0L9Vqpm.js → scroll-text-CQFQhF_6.js} +1 -1
  115. package/web-app/dist/assets/{server-SgwFv6cd.js → server-DxNytQEJ.js} +1 -1
  116. package/web-app/dist/assets/{shield-alert-D1tESlfI.js → shield-alert-Xj5wGUSK.js} +1 -1
  117. package/web-app/dist/assets/{thumbs-up-bUtUA15O.js → thumbs-up-DW0j9Yxz.js} +1 -1
  118. package/web-app/dist/assets/{trash-2-C9Nv1kKk.js → trash-2-VOdpiJ1e.js} +1 -1
  119. package/web-app/dist/assets/{trending-down-Bh107AJe.js → trending-down-eTizH4_b.js} +1 -1
  120. package/web-app/dist/assets/{trending-up-25BqhLVq.js → trending-up-agYVl7bU.js} +1 -1
  121. package/web-app/dist/assets/{upload-DDYtMbN5.js → upload-DQ3NfXA_.js} +1 -1
  122. package/web-app/dist/assets/{usePolling-BSDT0il_.js → usePolling-BKETnkDT.js} +1 -1
  123. package/web-app/dist/assets/{user-CKpUEUhK.js → user-Cv6BNmOT.js} +1 -1
  124. package/web-app/dist/index.html +1 -1
  125. package/docs/README-FULL.md +0 -977
@@ -1,4 +1,4 @@
1
- # Loki Mode v5.51.0 -- Integration Cookbook
1
+ # Loki Mode Integration Cookbook
2
2
 
3
3
  Step-by-step guides for connecting Loki Mode to external services. Each integration is opt-in and configured via environment variables.
4
4
 
@@ -173,9 +173,16 @@ curl -X POST "$LOKI_TEAMS_WEBHOOK_URL" \
173
173
  | Variable | Required | Description |
174
174
  |----------|----------|-------------|
175
175
  | `LOKI_JIRA_URL` | Yes | Jira Cloud base URL |
176
- | `LOKI_JIRA_EMAIL` | Yes | Jira user email for authentication |
177
176
  | `LOKI_JIRA_TOKEN` | Yes | Jira API token |
178
- | `LOKI_JIRA_PROJECT_KEY` | No | Default Jira project key |
177
+ | `LOKI_JIRA_EPIC_KEY` | No | Epic that run events are synced to |
178
+
179
+ These are read by `src/integrations/sync-subscriber.js`. The Jira user email is
180
+ passed to `JiraApiClient` in code (see below); no environment variable carries it.
181
+
182
+ Known issue on main: `src/integrations/sync-subscriber.js` constructs `JiraApiClient`
183
+ without the `email` and `apiToken` it requires, so Jira auto-sync from these
184
+ variables does not start. Construct `JiraApiClient({ baseUrl, email, apiToken })`
185
+ in code until that is fixed.
179
186
 
180
187
  ### Configuration
181
188
 
@@ -187,9 +194,7 @@ curl -X POST "$LOKI_TEAMS_WEBHOOK_URL" \
187
194
 
188
195
  ```bash
189
196
  export LOKI_JIRA_URL="https://company.atlassian.net"
190
- export LOKI_JIRA_EMAIL="user@company.com"
191
197
  export LOKI_JIRA_TOKEN="your-api-token"
192
- export LOKI_JIRA_PROJECT_KEY="PROJ"
193
198
  ```
194
199
 
195
200
  3. **Epic Sync:**
@@ -202,7 +207,7 @@ curl -X POST "$LOKI_TEAMS_WEBHOOK_URL" \
202
207
 
203
208
  const api = new JiraApiClient({
204
209
  baseUrl: process.env.LOKI_JIRA_URL,
205
- email: process.env.LOKI_JIRA_EMAIL,
210
+ email: 'user@company.com',
206
211
  apiToken: process.env.LOKI_JIRA_TOKEN,
207
212
  });
208
213
 
@@ -250,11 +255,11 @@ curl -X POST "$LOKI_TEAMS_WEBHOOK_URL" \
250
255
 
251
256
  ```bash
252
257
  # Test Jira API access
253
- curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
258
+ curl -u "user@company.com:$LOKI_JIRA_TOKEN" \
254
259
  "$LOKI_JIRA_URL/rest/api/3/myself" | python3 -m json.tool
255
260
 
256
261
  # Test epic fetch
257
- curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
262
+ curl -u "user@company.com:$LOKI_JIRA_TOKEN" \
258
263
  "$LOKI_JIRA_URL/rest/api/3/issue/PROJ-123" | python3 -m json.tool
259
264
  ```
260
265
 
@@ -282,9 +287,12 @@ curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
282
287
 
283
288
  | Variable | Required | Description |
284
289
  |----------|----------|-------------|
285
- | `LOKI_LINEAR_API_KEY` | Yes | Linear personal API key |
290
+ | `LOKI_LINEAR_TOKEN` | Yes | Linear personal API key |
286
291
  | `LOKI_LINEAR_TEAM_ID` | No | Default team ID for issue creation |
287
- | `LOKI_LINEAR_WEBHOOK_SECRET` | No | Webhook signing secret for inbound events |
292
+ | `LOKI_LINEAR_PROJECT_ID` | No | Project that run events are synced to |
293
+
294
+ The webhook signing secret is not an environment variable; set `webhook_secret`
295
+ under `integrations.linear` in `.loki/config.yaml`.
288
296
 
289
297
  ### Configuration
290
298
 
@@ -295,7 +303,7 @@ curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
295
303
  2. **Set Environment Variables:**
296
304
 
297
305
  ```bash
298
- export LOKI_LINEAR_API_KEY="lin_api_your_key_here"
306
+ export LOKI_LINEAR_TOKEN="lin_api_your_key_here"
299
307
  export LOKI_LINEAR_TEAM_ID="your-team-id"
300
308
  ```
301
309
 
@@ -304,7 +312,7 @@ curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
304
312
  ```javascript
305
313
  const { LinearClient } = require('./src/integrations/linear/client');
306
314
 
307
- const client = new LinearClient(process.env.LOKI_LINEAR_API_KEY);
315
+ const client = new LinearClient(process.env.LOKI_LINEAR_TOKEN);
308
316
 
309
317
  // Fetch a project with all issues
310
318
  const project = await client.getProject('project-id');
@@ -338,14 +346,15 @@ curl -u "$LOKI_JIRA_EMAIL:$LOKI_JIRA_TOKEN" \
338
346
 
339
347
  5. **Webhook Handler (optional):**
340
348
 
341
- Configure Linear to send webhooks to your Loki Mode instance for bidirectional sync.
349
+ Configure Linear to send webhooks to your Loki Mode instance. Signatures are
350
+ verified when `webhook_secret` is set (`src/integrations/linear/sync.js`).
342
351
 
343
352
  ### Verification
344
353
 
345
354
  ```bash
346
355
  # Test Linear API access
347
356
  curl -X POST https://api.linear.app/graphql \
348
- -H "Authorization: $LOKI_LINEAR_API_KEY" \
357
+ -H "Authorization: $LOKI_LINEAR_TOKEN" \
349
358
  -H "Content-Type: application/json" \
350
359
  -d '{"query": "{ viewer { id name } }"}'
351
360
  ```
@@ -1,4 +1,4 @@
1
- # Loki Mode v5.51.0 -- Migration Guide
1
+ # Loki Mode Migration Guide
2
2
 
3
3
  ## Upgrading from v5.50.0 to v5.51.0
4
4
 
@@ -76,10 +76,7 @@ See `docs/enterprise/sdk-guide.md` for installation and usage.
76
76
  | `LOKI_TEAMS_WEBHOOK_SECRET` | (unset) | Teams webhook shared secret |
77
77
  | `LOKI_OIDC_ISSUER` | (unset) | OIDC identity provider URL |
78
78
  | `LOKI_OIDC_CLIENT_ID` | (unset) | OIDC client ID |
79
- | `LOKI_OIDC_CLIENT_SECRET` | (unset) | OIDC client secret |
80
- | `LOKI_OIDC_REDIRECT_URI` | (unset) | OIDC redirect URI |
81
- | `LOKI_CORS_ORIGINS` | localhost | Allowed CORS origins |
82
- | `LOKI_API_RATE_LIMIT` | 100 | API rate limit (requests/min) |
79
+ | `LOKI_DASHBOARD_CORS` | localhost | Allowed CORS origins |
83
80
 
84
81
  ### Existing Environment Variables (Unchanged)
85
82
 
@@ -101,12 +98,11 @@ These variables from v5.50.0 continue to work identically:
101
98
  | `LOKI_TLS_CERT` | (unset) | TLS certificate path |
102
99
  | `LOKI_TLS_KEY` | (unset) | TLS key path |
103
100
  | `LOKI_JIRA_URL` | (unset) | Jira Cloud base URL |
104
- | `LOKI_JIRA_EMAIL` | (unset) | Jira user email |
105
101
  | `LOKI_JIRA_TOKEN` | (unset) | Jira API token |
106
- | `LOKI_JIRA_PROJECT_KEY` | (unset) | Default Jira project key |
107
- | `LOKI_LINEAR_API_KEY` | (unset) | Linear API key |
102
+ | `LOKI_JIRA_EPIC_KEY` | (unset) | Jira epic that run events sync to |
103
+ | `LOKI_LINEAR_TOKEN` | (unset) | Linear API key |
108
104
  | `LOKI_LINEAR_TEAM_ID` | (unset) | Linear team ID |
109
- | `LOKI_LINEAR_WEBHOOK_SECRET` | (unset) | Linear webhook secret |
105
+ | `LOKI_LINEAR_PROJECT_ID` | (unset) | Linear project that run events sync to |
110
106
  | `LOKI_GITHUB_SYNC` | (unset) | Enable GitHub sync |
111
107
 
112
108
  ### API Changes
@@ -1,4 +1,4 @@
1
- # Loki Mode v5.51.0 -- Enterprise Performance Tuning
1
+ # Loki Mode Enterprise Performance Tuning
2
2
 
3
3
  ## Overview
4
4
 
@@ -233,11 +233,8 @@ uvicorn dashboard.server:app --workers 4 --host 0.0.0.0 --port 57374
233
233
 
234
234
  Dashboard API rate limits protect against abuse:
235
235
 
236
- ```bash
237
- # Default rate limits are built into the API layer
238
- # Override via env vars if needed
239
- export LOKI_API_RATE_LIMIT="100" # requests per minute
240
- ```
236
+ Rate limits are built into the API layer (10 requests/min on control endpoints,
237
+ 60 on rate-limited read endpoints) and are not configurable by environment variable.
241
238
 
242
239
  ### Static Asset Caching
243
240
 
@@ -1,4 +1,4 @@
1
- # Loki Mode v5.51.0 -- SDK Guide
1
+ # Loki Mode SDK Guide
2
2
 
3
3
  Official Python and TypeScript SDKs for the Autonomi Control Plane API. Both SDKs have zero external dependencies -- they use only standard library features.
4
4
 
@@ -1,4 +1,4 @@
1
- # Loki Mode v5.51.0 -- Enterprise Security
1
+ # Loki Mode Enterprise Security
2
2
 
3
3
  ## Overview
4
4
 
@@ -167,27 +167,18 @@ When both variables are set, the dashboard server binds on HTTPS only. Self-sign
167
167
 
168
168
  ### Rate Limiting
169
169
 
170
- The API enforces rate limits per token/IP:
171
-
172
- | Endpoint Group | Default Limit |
173
- |----------------|---------------|
174
- | Read operations | 100 req/min |
175
- | Write operations | 30 req/min |
176
- | Auth operations | 10 req/min |
177
-
178
- Rate limit headers are included in every response:
179
- ```
180
- X-RateLimit-Limit: 100
181
- X-RateLimit-Remaining: 95
182
- X-RateLimit-Reset: 1708520460
183
- ```
170
+ The dashboard API applies in-process rate limits (`_RateLimiter` in
171
+ `dashboard/server.py`): 10 requests/min on control endpoints and 60
172
+ requests/min on rate-limited read endpoints. Exceeding them returns HTTP 429.
173
+ The limits are not configurable by environment variable, and no
174
+ `X-RateLimit-*` headers are sent.
184
175
 
185
176
  ### CORS
186
177
 
187
178
  CORS is configured via environment variables:
188
179
 
189
180
  ```bash
190
- export LOKI_CORS_ORIGINS="https://dashboard.company.com,https://admin.company.com"
181
+ export LOKI_DASHBOARD_CORS="https://dashboard.company.com,https://admin.company.com"
191
182
  ```
192
183
 
193
184
  Default behavior: `localhost` origins are allowed for development. In production, explicitly set allowed origins.
@@ -228,7 +219,7 @@ This is a deliberate fail-closed design. Invalid or missing signatures always re
228
219
 
229
220
  ## Audit Logging
230
221
 
231
- ### Hash-Chained Tamper-Evident Logs
222
+ ### Hash-Chained Audit Logs
232
223
 
233
224
  Audit logs use SHA-256 hash chains for tamper evidence. Each entry's hash depends on the previous entry's hash, creating an immutable chain from the genesis record.
234
225
 
@@ -50,6 +50,20 @@ immediately instead of burning `backoffLimit`. Requires Kubernetes 1.31+.
50
50
  Both the bash runner and the Bun runner (`LOKI_SDK_LOOP`) implement this
51
51
  identically; a parity test asserts they agree status for status.
52
52
 
53
+ ## `loki quick`
54
+
55
+ | Code | Meaning |
56
+ |---|---|
57
+ | 0 | The run completed and its diff did not weaken the tests |
58
+ | 3 | The diff weakened the tests (a skip marker added in a test file, a test runner config changed, or an existing test file deleted or renamed). The receipt headline is NOT VERIFIED and names what was weakened, for example `NOT VERIFIED (tests weakened: skip added in sum.test.js)` |
59
+ | other nonzero | The run itself failed; this code is passed through unchanged |
60
+
61
+ Code 3 only ever raises an exit of 0, in quiet and `LOKI_VERBOSE=1` modes alike;
62
+ it never lowers another nonzero code. Edited assertion lines are disclosed in
63
+ the receipt (`tests_integrity:assertions_edited`) and keep exit 0. A NOT VERIFIED
64
+ headline caused by unproven gates alone also keeps exit 0. The signal is the
65
+ `tests_integrity` item in the proof's `honesty.degraded`, not the headline text.
66
+
53
67
  ## `loki verify`
54
68
 
55
69
  | Code | Verdict |
@@ -19,7 +19,7 @@ When enabled, Loki Mode automatically:
19
19
 
20
20
  1. Creates a feature branch: `loki/session-<timestamp>-<pid>`
21
21
  2. Performs all agent work on the feature branch
22
- 3. Creates a PR at session end (if GitHub CLI is available)
22
+ 3. Creates a PR at session end when `LOKI_AUTO_PR=1` (and GitHub CLI is available)
23
23
  4. Requires manual review and merge to main
24
24
 
25
25
  ### Feature Branch Naming
@@ -62,25 +62,8 @@ Delete feature branch
62
62
 
63
63
  | Variable | Default | Description |
64
64
  |----------|---------|-------------|
65
- | `LOKI_BRANCH_PROTECTION` | `false` | Enable automatic feature branch workflow |
66
- | `LOKI_BASE_BRANCH` | `main` | Target branch for PRs (or `master` if detected) |
67
- | `LOKI_BRANCH_PREFIX` | `loki/session-` | Prefix for auto-created branches |
68
- | `LOKI_AUTO_PR` | `true` | Automatically create PR at session end |
69
- | `LOKI_PR_TEMPLATE` | - | Path to PR description template |
70
-
71
- ### Configuration File
72
-
73
- ```yaml
74
- # .loki/config.yaml
75
- git:
76
- branch_protection:
77
- enabled: true
78
- base_branch: main
79
- branch_prefix: loki/session-
80
- auto_pr: true
81
- squash_merge: true
82
- delete_after_merge: true
83
- ```
65
+ | `LOKI_BRANCH_PROTECTION` | `true` | Run agent work on an auto-created `loki/session-<timestamp>-<pid>` branch |
66
+ | `LOKI_AUTO_PR` | off | Set to `1` to push the session branch and open a PR at session end |
84
67
 
85
68
  ## Manual Git Workflow
86
69
 
@@ -126,7 +109,7 @@ git push origin feature/my-feature
126
109
 
127
110
  ### Automatic PR Creation
128
111
 
129
- When `LOKI_AUTO_PR=true` and GitHub CLI is installed:
112
+ When `LOKI_AUTO_PR=1` and GitHub CLI is installed:
130
113
 
131
114
  ```bash
132
115
  # Loki Mode automatically runs at session end
@@ -137,47 +120,6 @@ gh pr create \
137
120
  --head loki/session-20260215-143022-12345
138
121
  ```
139
122
 
140
- ### PR Description Template
141
-
142
- Create a template for consistent PR descriptions:
143
-
144
- ```markdown
145
- # .loki/pr-template.md
146
-
147
- ## Changes
148
-
149
- <!-- Auto-generated summary of changes -->
150
-
151
- ## Tasks Completed
152
-
153
- <!-- List of completed tasks from task queue -->
154
-
155
- ## Quality Gates
156
-
157
- - [ ] All tests passing
158
- - [ ] Code review completed
159
- - [ ] No security vulnerabilities
160
- - [ ] Documentation updated
161
-
162
- ## Cost
163
-
164
- Estimated cost: $X.XX USD
165
-
166
- ## Session Info
167
-
168
- - Start: YYYY-MM-DD HH:MM:SS
169
- - End: YYYY-MM-DD HH:MM:SS
170
- - Duration: X hours
171
- - Iterations: X
172
- - Provider: claude/cline/codex/aider/opencode
173
- ```
174
-
175
- Set template path:
176
-
177
- ```bash
178
- export LOKI_PR_TEMPLATE=.loki/pr-template.md
179
- ```
180
-
181
123
  ## Agent Action Audit
182
124
 
183
125
  All Git operations performed by agents are logged to `.loki/logs/agent-audit.jsonl`:
@@ -6,19 +6,9 @@ Network egress control and isolation for Loki Mode deployments.
6
6
 
7
7
  This guide covers network-level security controls for restricting outbound network access from Loki Mode containers and pods to only the AI API endpoints required for operation.
8
8
 
9
- ## Environment Variables
10
-
11
- **None of the variables below are implemented. Do not rely on them.** They are
12
- reserved names for a planned application-level enforcement layer, and no
13
- runtime code reads any of them today. Network security is enforced entirely at
14
- the infrastructure level, using the Docker network and Kubernetes NetworkPolicy
15
- recipes in the rest of this guide.
16
-
17
- | Variable | Default | Description | Status |
18
- |----------|---------|-------------|--------|
19
- | `LOKI_NETWORK_EGRESS_POLICY` | `unrestricted` | `unrestricted` (default), `ai-only` (restrict to AI APIs), `none` (block all outbound) | Planned, target TBD |
20
- | `LOKI_ALLOWED_HOSTS` | (empty) | Comma-separated list of additional hostnames to allow when egress policy is `ai-only` | Planned, target TBD |
21
- | `LOKI_BLOCK_METADATA_ENDPOINT` | `false` | Block cloud metadata endpoint (169.254.169.254) from within the application | Planned, target TBD |
9
+ Loki Mode has no application-level egress setting: no runtime code restricts
10
+ outbound hosts. Network security is enforced entirely at the infrastructure
11
+ level, using the Docker network and Kubernetes NetworkPolicy recipes below.
22
12
 
23
13
  ## Docker Network Isolation
24
14
 
@@ -25,7 +25,7 @@ Loki Mode today is a **multi-agent autonomous orchestrator**, not a BaaS. It own
25
25
  - An auth layer (`dashboard/auth.py`, `web-app/auth.py`) supporting Bearer tokens, OIDC (Google/Azure/Okta), bcrypt password hashing, RBAC scopes
26
26
  - A working completion-council with anti-sycophancy, 3-reviewer parallel review, devil's-advocate fallback
27
27
  - A healing-mode for legacy systems (v6.67+) with friction-as-semantics
28
- - 41 specialized agent types and a queue system
28
+ - Specialized agent types and a queue system
29
29
 
30
30
  **The gap:** every backend primitive Loki owns is for orchestrating the agent. None of it is for the *application the agent is building*. When `loki start ./prd.md` builds "a Twitter clone with login and image uploads", the agent has to invent the auth flow, choose a DB, set up storage, wire Stripe, etc. - and the user has to run a parallel set of tools (Supabase, Stripe, Cloudflare R2) to support the resulting app.
31
31
 
@@ -165,7 +165,7 @@ URL bar showing current preview path, Refresh button, Open in New Tab button.
165
165
  Based on feedback:
166
166
  - Memory System Viewer (power-user debug tool, not customer-facing)
167
167
  - Full Integrations page (keep GitHub only, defer Jira/Linear/Slack/Teams)
168
- - 41-agent grid (most sessions have 1-3 agents active; show compact list instead)
168
+ - Agent grid (most sessions have 1-3 agents active; show compact list instead)
169
169
  - Code Review panel (let quality gates surface findings inline)
170
170
  - Collaboration/multiplayer (acknowledged as future need, not v1)
171
171
  - Database provisioning (out of scope for v1)
@@ -137,8 +137,6 @@ Each card:
137
137
 
138
138
  Integrations available:
139
139
  - **GitHub**: Connect repo, import issues, create PRs (`loki github`, `loki run`)
140
- - **Jira**: Bidirectional sync (`LOKI_JIRA_*` env vars)
141
- - **Linear**: Issue sync (`LOKI_LINEAR_*` env vars)
142
140
  - **Slack**: Notifications + slash commands (`LOKI_SLACK_*`)
143
141
  - **Teams**: Adaptive Cards notifications (`LOKI_TEAMS_*`)
144
142
  - **OpenTelemetry**: Trace export (`loki telemetry`)
@@ -197,7 +195,7 @@ Each gate: name, status (pass/fail/pending), details expandable
197
195
 
198
196
  ### Agent Activity
199
197
 
200
- Real-time grid showing which of the 41 agent types are active:
198
+ Real-time grid showing which agent types are active:
201
199
  - Agent name, type, domain, model tier (Opus/Sonnet/Haiku)
202
200
  - Current task
203
201
  - Status (working/idle/completed)
@@ -65,7 +65,7 @@ Acceptance criteria:
65
65
  Validation: Fixture repos: (A) spec names endpoint /orders that exists -> no finding; (B) spec names /orders but no handler -> High finding; (C) plaintext password store -> invariant finding. Measure false-positive rate on 3 clean real repos (must be zero). Prove accuracy lift on the FrontierCode-style benchmark (rank 12): source-grounded checklists should catch wrong-but-internally-consistent implementations that prose-only checklists pass.
66
66
 
67
67
  ### #3 Annotate-before-act: pre-committed expected-outcome ledger, diffed against reality at verify time (loki-mode, high/L)
68
- Before running verification for a change/checklist-item, the runtime writes a tamper-evident ledger of EXPECTED observable outcomes (e.g. 'GET /health -> 200 {status:ok}', 'test X fails before fix, passes after', 'endpoint Y should now exist'). At verify time, actual results are compared to the pre-committed prediction; any expectation silently dropped or contradicted becomes a finding. Reuse the proof-verify canonical-hash + drift pattern so the ledger is tamper-evident and edits-after-the-fact are detectable. Forces a falsifiable prediction, cutting self-serving post-hoc rationalization.
68
+ Before running verification for a change/checklist-item, the runtime writes a hash-chained ledger of EXPECTED observable outcomes (e.g. 'GET /health -> 200 {status:ok}', 'test X fails before fix, passes after', 'endpoint Y should now exist'). At verify time, actual results are compared to the pre-committed prediction; any expectation silently dropped or contradicted becomes a finding. Reuse the proof-verify canonical-hash + drift pattern so the ledger is tamper-evident and edits-after-the-fact are detectable. Forces a falsifiable prediction, cutting self-serving post-hoc rationalization.
69
69
  Acceptance criteria:
70
70
  - .loki/expectations/<iter>.json written BEFORE the verify/checklist run, each entry {id, statement, check_ref, expected}, canonicalized and hashed like proof-generator
71
71
  - loki verify (or the completion gate) reads the ledger and emits a finding for any expectation with no matching executed check, or whose actual contradicts expected
@@ -2,13 +2,13 @@
2
2
 
3
3
  ## Title
4
4
 
5
- Show HN: Loki Mode - PRD in, tested code out (41 agent roles, 8 quality gates, RARV self-verification)
5
+ Show HN: Loki Mode - PRD in, tested code out (8 quality gates, RARV self-verification)
6
6
 
7
7
  ## Body
8
8
 
9
9
  I built Loki Mode because I got tired of the copy-paste loop between AI coding assistants and my terminal. I wanted to hand over a PRD and get back a working, tested codebase -- not perfect, but a solid starting point.
10
10
 
11
- **What it does:** You give it a Product Requirements Document. It breaks the work into tasks across 41 specialized agent roles organized into 8 domains (engineering, operations, business, data, product, growth, review, orchestration) -- prompt-defined specifications the orchestrator adopts per phase, with parallel review (blind council) and optional worktree streams on Claude Code, sequential on other providers -- and runs every iteration through a self-verification loop called RARV: Reason, Act, Reflect, Verify. The idea is that the system catches its own mistakes before you have to.
11
+ **What it does:** You give it a Product Requirements Document. It breaks the work into tasks across specialized agent roles (engineering, operations, business, data, product, growth, review, orchestration) -- prompt-defined specifications the orchestrator adopts per phase, with parallel review (blind council) and optional worktree streams on Claude Code, sequential on other providers -- and runs every iteration through a self-verification loop called RARV: Reason, Act, Reflect, Verify. The idea is that the system catches its own mistakes before you have to.
12
12
 
13
13
  **Quality gates:** 8 automated gates including 3-reviewer blind review (agents review each other's work without seeing prior reviews), anti-sycophancy checks (a devil's advocate pass on unanimous approvals), and mock/mutation detection. These are not foolproof, but they catch a surprising number of issues that single-pass generation misses.
14
14
 
@@ -20,9 +20,8 @@ I built Loki Mode because I got tired of the copy-paste loop between AI coding a
20
20
 
21
21
  **Research foundation:** The architecture draws from Anthropic's Constitutional AI (principles-based self-critique), DeepMind's Scalable Oversight via Debate, and OpenAI's Agents SDK patterns (guardrails, tripwires, handoffs). References to specific papers are in the repo.
22
22
 
23
- **What it does NOT do:** It does not deploy anything. It generates deployment configs (Helm, Docker Compose, Terraform), but a human deploys. Complex domain logic will need human review. The system can and does make mistakes, especially on novel problems. Token costs scale with project complexity. Our SWE-bench numbers (299/300 patches generated) measure output, not resolution -- the official evaluator has not been run, so the actual fix rate is unknown and likely significantly lower than the generation rate. The HumanEval score is self-reported with max 3 retries per problem.
23
+ **What it does NOT do:** It does not deploy anything. It generates deployment configs (Helm, Docker Compose, Terraform), but a human deploys. Complex domain logic will need human review. The system can and does make mistakes, especially on novel problems. Token costs scale with project complexity. There is no published benchmark score.
24
24
 
25
- **Test suite:** 683 npm tests, 631 pytest tests, 16 shell tests. Self-reported HumanEval score of 162/164 (98.78%).
26
25
 
27
26
  Built solo. BUSL-1.1 source-available.
28
27
 
@@ -8,7 +8,7 @@ Loki Mode supports integration with enterprise SIEM systems for:
8
8
 
9
9
  - Centralized security monitoring
10
10
  - Real-time threat detection
11
- - Compliance reporting (SOC2, HIPAA, PCI-DSS)
11
+ - Audit evidence for your own compliance reporting
12
12
  - Incident response
13
13
  - Forensic analysis
14
14
 
@@ -385,7 +385,6 @@ logs:
385
385
  tags:
386
386
  - env:production
387
387
  - team:security
388
- - compliance:soc2
389
388
 
390
389
  # Process JSON logs
391
390
  logs_config:
@@ -487,14 +486,13 @@ in your SIEM, where the correlation and retention rules already live.
487
486
  To pull an evidence slice locally for an auditor:
488
487
 
489
488
  ```bash
490
- AUDIT=~/.loki/dashboard/audit/audit.jsonl
489
+ AUDIT=~/.loki/dashboard/audit # one audit-YYYY-MM-DD.jsonl file per day
491
490
 
492
- # Access events in a date range (SOC2 / HIPAA evidence)
493
- jq -s 'map(select(.timestamp >= "2026-01-01" and .timestamp <= "2026-12-31"))' \
494
- "$AUDIT" > audit-evidence.json
491
+ # Events in a date range
492
+ cat "$AUDIT"/audit-*.jsonl | jq -s 'map(select(.timestamp >= "2026-01-01" and .timestamp <= "2026-12-31"))' > audit-evidence.json
495
493
 
496
- # Token lifecycle events (PCI-DSS Requirement 8)
497
- jq -c 'select(.event | startswith("auth.token."))' "$AUDIT"
494
+ # Token lifecycle events
495
+ cat "$AUDIT"/audit-*.jsonl | jq -c 'select(.resource_type == "token")'
498
496
  ```
499
497
 
500
498
  Verify the hash chain before handing the file over. There is no
@@ -592,8 +590,8 @@ logger -n syslog.example.com -P 514 "Test from Loki Mode"
592
590
  echo $LOKI_AUDIT_SYSLOG_HOST
593
591
  loki syslog test
594
592
 
595
- # Check for forwarding errors
596
- loki enterprise audit tail --event syslog.error
593
+ # Check recent audit entries
594
+ loki enterprise audit tail
597
595
  ```
598
596
 
599
597
  ### Format Issues
@@ -1,7 +1,7 @@
1
1
  # Loki Mode Edge Case Test Scenarios
2
2
 
3
3
  Agent 12 - Scenario Writing: Edge Cases, Error Paths, Concurrent Usage
4
- Version: v6.71.1 | Date: 2026-03-24
4
+ Date: 2026-03-24
5
5
 
6
6
  ---
7
7
 
@@ -55,7 +55,7 @@ Version: v6.71.1 | Date: 2026-03-24
55
55
  **Given** an autonomous session is running at iteration 15 of 1000
56
56
  **When** the network connection drops (DNS resolution fails, API timeout)
57
57
  **Then** the system should:
58
- - The provider CLI (claude/codex/gemini) exits with a non-zero exit code
58
+ - The provider CLI (claude/codex/cline/aider/opencode) exits with a non-zero exit code
59
59
  - `run_autonomous()` captures the exit code in `$exit_code` (run.sh:9406)
60
60
  - Rate limit detection runs via `is_rate_limited()` (checks for common rate limit strings)
61
61
  - If failover is enabled (`LOKI_FAILOVER=true`), attempt `attempt_provider_failover()`
@@ -52,7 +52,7 @@ Source: `.github/workflows/loki-ci-example.yml:41-44`
52
52
  ### TW-04: Handoff Between Human and AI Coding
53
53
 
54
54
  ```
55
- GIVEN a loki autonomous session is running (iteration 5 of 20)
55
+ GIVEN a Loki autonomous session is running (iteration 5 of 20)
56
56
  AND the human developer notices a design issue
57
57
  WHEN the developer creates .loki/PAUSE signal file
58
58
  THEN the session pauses after the current RARV iteration completes
@@ -66,7 +66,7 @@ Source: `autonomy/run.sh:7897` (check_human_intervention)
66
66
  ### TW-05: Project Template Sharing Across Team
67
67
 
68
68
  ```
69
- GIVEN the team uses loki templates (13 types: saas, cli, discord-bot, etc.)
69
+ GIVEN the team uses Loki PRD templates (saas, cli, discord-bot, etc.)
70
70
  AND the team lead wants to standardize on a custom template
71
71
  WHEN the lead creates a PRD template at templates/custom-team.md
72
72
  AND commits it to the shared repository
@@ -195,7 +195,7 @@ Source: `skills/parallel-workflows.md`
195
195
  ### GI-02: Meaningful AI Commit Messages
196
196
 
197
197
  ```
198
- GIVEN a loki session has completed a RARV iteration
198
+ GIVEN a Loki session has completed a RARV iteration
199
199
  AND the iteration modified 4 files
200
200
  WHEN the autonomous system commits changes
201
201
  THEN the commit message follows the format: "<type>: <description>"
@@ -437,7 +437,7 @@ Source: `.github/workflows/release.yml:238-496`
437
437
 
438
438
  ```
439
439
  GIVEN the dashboard WebSocket endpoint is available at /ws
440
- AND a CI pipeline is running a loki session
440
+ AND a CI pipeline is running a Loki session
441
441
  WHEN the WebSocket client connects
442
442
  THEN real-time events are streamed including:
443
443
  - iteration_start / iteration_complete
package/docs/v10/BOARD.md CHANGED
@@ -630,6 +630,8 @@ Source: ~/git/autonomi-dev/research/2026-09-30-adoption/SWARM-PROMPT-ADOPTION.md
630
630
  | E-160 | D46 runner contention (peer steer 02:11Z): a green train SHA fast-forwarded to main re-runs Tests (8 shards), Bun Parity, Security Audit and First-run gate on the identical SHA, doubling runner load and queueing the next train. Proposal: on push to main, a first job checks for a successful run of the same workflow at the same SHA from a train/** push and, if found, exits success recording the reused run id (D28: same SHA = same tree); any SHA not train-tested runs in full. CTO call FIRST: OPERATING-MODEL says "Tier B full verification on every main commit (never cancelled on main)"; confirm reuse does not weaken Release required-ci, branch-conditional steps (train vs main), or the Security Audit range (base tag config). Sequence after E-157 r4 (same files) | .github/workflows/test.yml, bun-parity.yml, security-audit.yml, tests/test_release_required_ci.py | HIGH | main push of a train-green SHA records reuse and finishes in under 1 min; an untested SHA runs in full; a train-red SHA is not reused | ready@2026-10-01T02:10Z | CTO decision first; after E-157 |
631
631
  | E-161 | INCIDENT 02:46Z: an opus reviewer ran `git checkout --detach <tag>` x7 plus a `.gitleaksignore` overwrite in the SHARED main checkout (reflog 02:46:04Z to 02:46:38Z). Cause: its run-owned clone dir was deleted, the next `cd` into a new clone failed, and the shell cwd fell back to the main checkout; every later command ran there. Restored by the Chief of Staff (`git checkout main`, fafe50dd, clean). Guard: v10-guard.sh (PreToolUse) blocks `git checkout`/`git switch`/`git reset`/`git worktree remove` and writes to tracked files when cwd resolves to the main checkout and the caller is an agent (not the leader session); agent prompts use `cd DIR` followed by an exit-on-failure check. Test: a hook test with cwd=main checkout blocks a detached checkout and allows it in a worktree | scripts/v10-guard.sh, tests | HIGH | detached checkout in main checkout blocked; same in a worktree allowed | ready@2026-10-01T02:47Z | GUARDS.md entry |
632
632
  | E-162 | Founder "faster" (03:05Z): slice CI runs only Tier A plus diff-selected suites (the existing Tier A selection), Bun Parity and the structural guards; the full 8-shard Tests stays on train/** and main, and the train stays the release verdict. Frees runners and targets about 2 min slice CI | .github/workflows/test.yml (slice-* job matrix only), tests | HIGH | slice-* push runs no 8-shard matrix; train/** and main still run all 8 shards; a slice touching a suite runs that suite | ready@2026-10-01T03:01Z | after E-157 r4 (same file) |
633
+ | E-165 | tests/test-mirofish-integration.sh line 413 runs `autonomy/loki start --no-mirofish` in the repo root: a live autonomous build that ran about 45 min, switched a worktree branch to loki/session-* and committed a docs change (found by DOC-02). Guard: the test must run against a run-owned fixture repo with a stub provider and LOKI_NO_BROWSER=1, never the repo root; add a structural guard that no test launches start/run against REPO_ROOT | tests/test-mirofish-integration.sh, a structural guard | HIGH | the test never runs loki start in the repo root; guard catches a planted case | ready@2026-10-01T04:07Z | P0 guard (D26) |
634
+ | E-166 | Jira auto-sync never starts: src/integrations/sync-subscriber.js builds JiraApiClient without the required email and apiToken, so LOKI_JIRA_URL plus LOKI_JIRA_TOKEN do nothing (DOC-02 added a known-issue note to the setup docs). Fix the wiring or remove the advertised auto-sync | src/integrations/sync-subscriber.js, tests | MEDIUM | with env set, the subscriber constructs a working client (mocked API) | ready@2026-10-01T04:07Z | none |
633
635
  | D49-auto | Auto-promote to latest on Post-Release Smoke success (founder D49) | .github/workflows/promote.yml, tests/test_release_next_tag.py | HIGH | smoke-success from Release promotes; manual smoke never promotes; not-lower guard | merged@2026-10-01T03:03Z | opus APPROVE 5510b8c8. Follow-ups (D49-auto-b): not-lower guard fails OPEN on an npm error (make it fail closed); loki-mode-sdk latest has no not-lower guard; a newer queued promote can replace a pending one in a release burst |
634
636
  | D51-A3 | Headless `loki backlog owner/repo --all/--label/--issues` plus loki.yaml (schema, validator, env overrides) | autonomy/lib/backlog.py, autonomy/lib/loki_yaml.py, autonomy/loki, schemas/loki-yaml.schema.json, tests/test-backlog.sh | HIGH | backlog 47/0 | merged@2026-10-01T03:11Z | sonnet APPROVE 14ff9ae2; follow-up D51-A3b: `loki_yaml.py show` echoes a token pasted into git.token_env (redact x-secret-ref fields); no-parser test case; re-run of a failed issue discards its kept worktree |
635
637
  | D51-A4 | Slack on the v10 path (PR opened, BLOCKED, finished), webhook from the env var named by LOKI_SLACK_WEBHOOK_ENV, URL redacted, 5s bounded post | loki-ts/src/e10ext/slack_events.ts, engine10/supervisor.ts, tests | MEDIUM | slack tests 40/0 | merged@2026-10-01T03:11Z | sonnet APPROVE adb838b2; one labelled test message after it ships |
package/docs/v10/GUIDE.md CHANGED
@@ -140,9 +140,11 @@ provider names (`claude`, `codex`, `cline`, `aider`); it is not supported.
140
140
 
141
141
  Once v10 becomes the default (docs/v10/DECISIONS.md, D29), `loki`, `loki
142
142
  status`, `loki verify` and `loki dashboard` with no `LOKI_ENGINE` set will
143
- mean the v10 commands described above. The previous engine stays fully
144
- reachable: `loki legacy <args>` and `LOKI_ENGINE=legacy` both route to it,
145
- unchanged. Nothing is removed at the flip.
143
+ mean the v10 commands described above. The previous engine stays
144
+ reachable through `LOKI_ENGINE=legacy` (bin/loki sends every value other
145
+ than `v10` to it), unchanged. Nothing is removed at the flip. A separate
146
+ legacy alias command is planned (docs/v10/ENGINE.md, E-31) but is not on
147
+ main.
146
148
 
147
149
  That flip has not happened on main as of this guide. The opt-in line near
148
150
  the top of this file and of README.md's section is the one place that
@@ -534,3 +534,39 @@ profiling implement's own internal turn-by-turn timing (not available from
534
534
 
535
535
  Command: `python3 eval/loki10/stage-profile.py
536
536
  ~/loki-ci-logs/eval/e98f-engine/{default,nocascade,nowall}-r{1,2,3}`
537
+
538
+ ## D50 model-lift baseline (small tier)
539
+
540
+ Measured 2026-09-30 on main c5eaddb0 (harness_sha c5eaddb0, clean; loki-ts deps via `bun install --frozen-lockfile`), claude 2.1.286, 1 rep, 900s cap, EV-3 isolation, provider-reported cost only, hidden tests decide completion. Models from providers/model_catalog.json: claude-haiku-4-5 and claude-sonnet-5. Opus arms not run. The Loki arm is `v10` at default knobs (LOKI_ENGINE=v10, same model pinned via LOKI_EVAL_MODEL).
541
+
542
+ Subset: the small tier has 29 tasks; 4 arms x 29 would be 116 runs, over the 40-run budget, so the first 10 small-tier tasks by id are used (40 runs total): aiq-52-searchbar, pub-click-2877, pub-click-3059, pub-click-3487, pub-click-3572, pub-humanize-152, pub-humanize-174, pub-humanize-333, pub-jsonschema-1389, pub-markupsafe-417. aiq-52-searchbar is an `expected_outcome: no_change_needed` task. n=10 per arm, 1 rep: every rate has a wide interval (one task is 10 points); this is a baseline, not a gate. Machine load average was 24-37 during the run (more than 2 concurrent arms never ran); wall times are inflated and noisy.
543
+
544
+ | arm | attempted | completed | rate | median wall, all runs | median wall, completed | total cost | cost per completed | timed out |
545
+ |---|---|---|---|---|---|---|---|---|
546
+ | raw haiku-4-5 | 10 | 4 | 40% | 103.2s | 100.5s | $2.3124 | $0.5781 | 0 |
547
+ | Loki+haiku (v10) | 10 | 7 | 70% | 123.0s | 127.7s | $3.4649 (1 run unmeasured) | $0.4950 | 0 |
548
+ | raw sonnet-5 | 10 | 9 | 90% | 111.6s | 118.0s | $4.9239 | $0.5471 | 0 |
549
+ | Loki+sonnet (v10) | 10 | 5 | 50% | 92.9s | 128.8s | $3.7278 (1 run unmeasured) | $0.7456 | 0 |
550
+
551
+ Cost per completed = total spend of all 10 runs (failures included) / completed. The 2 unmeasured v10 runs are the aiq-52-searchbar runs that exited in about 1s before any model call (no spend, no provider cost record).
552
+
553
+ Model lift (D50 targets):
554
+
555
+ | comparison | completion | cost per completed | time (median, all runs) | verdict |
556
+ |---|---|---|---|---|
557
+ | Loki+haiku vs raw haiku | 70% vs 40% (+30 points, 3 tasks) | $0.4950 vs $0.5781 (-14%) | 123.0s vs 103.2s (1.19x) | lift, within 1.2x |
558
+ | Loki+sonnet vs raw sonnet | 50% vs 90% (-40 points, 4 tasks) | $0.7456 vs $0.5471 (+36%) | 92.9s vs 111.6s (0.83x) | LOSS: below raw X |
559
+ | Loki+haiku vs raw sonnet (cross) | 70% vs 90% (-20 points) | $0.4950 vs $0.5471 (-10%); total $3.46 vs $4.92 (-30%) | 123.0s vs 111.6s | target (>= raw sonnet) NOT met |
560
+ | Loki+sonnet vs raw opus | not run | | | |
561
+
562
+ Failures by category (every failed or non-completed cell listed; no run timed out or was capped; all rows status ok):
563
+ - raw haiku (6): hidden tests failed with a PR pushed: pub-click-2877, pub-click-3059, pub-humanize-174, pub-humanize-333, pub-jsonschema-1389. aiq-52-searchbar: hidden regression test passed but it pushed a PR on a no-change task (wrong outcome).
564
+ - raw sonnet (1): hidden tests failed with a PR: pub-click-3059.
565
+ - Loki+haiku (3): aiq-52-searchbar: engine refused to start in 1.3s, "dirty tracked tree: M frontend/package-lock.json" (task `setup` npm install rewrote a tracked file; no model call). pub-humanize-174: outcome BLOCKED (spec conflict claimed against existing tests), no PR. pub-humanize-333: PR opened, hidden tests failed.
566
+ - Loki+sonnet (5): aiq-52-searchbar: same dirty-tree refusal in 0.7s. pub-click-2877: outcome ALREADY_SATISFIED claimed, no PR, hidden tests fail (false already-done). pub-humanize-174: FAILED "empty diff without an already_done marker", no PR. pub-click-3059 and pub-humanize-333: PR opened, hidden tests failed.
567
+
568
+ Findings: (1) the v10 dirty-tree refusal is a harness-vs-engine defect that costs both Loki arms the aiq task (the `no_change_needed` outcome is unreachable when `setup` dirties a tracked file); not fixed in this slice. (2) Loki+sonnet loses to raw sonnet mainly through no-PR outcomes (ALREADY_SATISFIED false positive, empty diff) and BLOCKED, not through slow runs. (3) Per D50 a slice that lowers lift is dropped; this baseline is the bar.
569
+
570
+ Reproduce (per arm; a run-owned temp is created by run.sh; needs `cd loki-ts && bun install --frozen-lockfile` for the v10 arm):
571
+ `LOKI_NO_BROWSER=1 LOKI_EVAL_MAX_LOAD=80 LOKI_EVAL_MODEL=<claude-haiku-4-5|claude-sonnet-5> eval/loki10/run.sh --arm <raw-claude|v10> --tasks aiq-52-searchbar,pub-click-2877,pub-click-3059,pub-click-3487,pub-click-3572,pub-humanize-152,pub-humanize-174,pub-humanize-333,pub-jsonschema-1389,pub-markupsafe-417 --parallel 1 --out ~/loki-ci-logs/d50-<arm>-<model>`
572
+ Raw rows: ~/loki-ci-logs/d50-{raw-haiku,raw-sonnet,v10-haiku,v10-sonnet}/results.jsonl.