loki-mode 7.49.0 → 7.51.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/autonomy/prd-analyzer.py +215 -1
- package/autonomy/prd-checklist.sh +315 -0
- package/autonomy/run.sh +350 -3
- package/autonomy/spec-interrogation.sh +224 -4
- package/autonomy/spec.sh +25 -16
- package/autonomy/verify.sh +108 -26
- package/dashboard/__init__.py +1 -1
- package/dashboard/audit.py +202 -21
- package/docs/INSTALLATION.md +2 -2
- package/docs/siem-integration.md +102 -0
- package/loki-ts/dist/loki.js +231 -230
- package/mcp/__init__.py +1 -1
- package/mcp/lsp_proxy.py +203 -0
- package/mcp/tests/test_lsp_proxy.py +169 -0
- package/package.json +1 -1
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
- package/references/invariant-checks.md +109 -0
- package/src/audit/crosslink.js +413 -0
- package/src/audit/index.js +32 -0
- package/src/observability/siem-export.js +424 -0
- package/src/policies/cost.js +270 -1
package/README.md
CHANGED
|
@@ -106,7 +106,7 @@ loki quick "build a landing page with a signup form"
|
|
|
106
106
|
| **Bun (recommended)** | `bun install -g loki-mode` | Fastest startup for CLI commands. |
|
|
107
107
|
| **Homebrew** | `brew tap asklokesh/tap && brew install loki-mode` | Auto-installs Bun as a dep |
|
|
108
108
|
| **Docker (easiest)** | `loki docker start prd.md` | Host wrapper: runs loki in the published image with zero config. Bind-mounts the current folder so `.loki` state, resume, and continuity work exactly like local. Auto-detects auth (`ANTHROPIC_API_KEY`, else your host Claude Code login). Needs loki + Docker on the host. See DOCKER_README.md |
|
|
109
|
-
| **Docker (raw)** | `docker pull asklokesh/loki-mode:7.
|
|
109
|
+
| **Docker (raw)** | `docker pull asklokesh/loki-mode:7.50.0 && docker run --rm -e ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY" asklokesh/loki-mode:7.50.0 start prd.md` | Bun + Claude CLI pre-installed; needs an API key, or use docker compose with a .env file, see DOCKER_README.md |
|
|
110
110
|
| **npm (compat)** | `npm install -g loki-mode` | Works without Bun (bash fallback). Migrate any time with `loki self-update --to bun`. |
|
|
111
111
|
|
|
112
112
|
**Upgrading:**
|
|
@@ -166,7 +166,7 @@ The next major release sunsets the Bash runtime entirely. There is no firm calen
|
|
|
166
166
|
| Method | Command |
|
|
167
167
|
|--------|---------|
|
|
168
168
|
| **Homebrew** | `brew tap asklokesh/tap && brew install loki-mode` |
|
|
169
|
-
| **Docker** | `docker pull asklokesh/loki-mode:7.
|
|
169
|
+
| **Docker** | `docker pull asklokesh/loki-mode:7.50.0` |
|
|
170
170
|
| **Inside Claude Code** | `claude --dangerously-skip-permissions` then type "Loki Mode" |
|
|
171
171
|
| **Git clone** | `git clone https://github.com/asklokesh/loki-mode.git` |
|
|
172
172
|
|
package/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: loki-mode
|
|
|
3
3
|
description: Autonomous spec-driven build system with a built-in trust layer. It does not call work done until it is verified (RARV-C closure loop, 8 quality gates, completion council, verified-completion evidence gate). Triggers on "Loki Mode". Takes a spec (PRD, GitHub issue, OpenAPI doc, etc.) to deployed product with minimal human intervention. Provider-agnostic. Requires --dangerously-skip-permissions flag.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Loki Mode v7.
|
|
6
|
+
# Loki Mode v7.51.0
|
|
7
7
|
|
|
8
8
|
**You are an autonomous agent. You make decisions. You do not ask questions. You do not stop.**
|
|
9
9
|
|
|
@@ -407,4 +407,4 @@ See `CHANGELOG.md` entries [7.5.7], [7.5.8], [7.5.13] for the per-fix list and r
|
|
|
407
407
|
|
|
408
408
|
---
|
|
409
409
|
|
|
410
|
-
**v7.
|
|
410
|
+
**v7.51.0 | [Autonomi](https://www.autonomi.dev/) flagship product | ~260 lines core**
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
7.
|
|
1
|
+
7.51.0
|
package/autonomy/prd-analyzer.py
CHANGED
|
@@ -170,6 +170,14 @@ class PrdAnalyzer:
|
|
|
170
170
|
self.feature_count = 0
|
|
171
171
|
self.scope = "unknown"
|
|
172
172
|
self.score = 0.0
|
|
173
|
+
# Deterministic structural validation result (P2-5). Populated by
|
|
174
|
+
# validate_structure() during analyze(). Shape:
|
|
175
|
+
# {"ok": bool, "issues": [str, ...], "warnings": [str, ...]}
|
|
176
|
+
# issues = structural problems that would likely yield a garbage
|
|
177
|
+
# checklist (no headings, unparseable, basic contradictions).
|
|
178
|
+
# warnings = lower-confidence findings worth surfacing but not blocking
|
|
179
|
+
# (e.g. a referenced local doc that does not exist).
|
|
180
|
+
self.structure = {"ok": True, "issues": [], "warnings": []}
|
|
173
181
|
|
|
174
182
|
def load(self):
|
|
175
183
|
"""Load and validate the PRD file, optionally appending architecture doc."""
|
|
@@ -190,6 +198,11 @@ class PrdAnalyzer:
|
|
|
190
198
|
def analyze(self):
|
|
191
199
|
"""Run all analysis dimensions and compute score."""
|
|
192
200
|
self.load()
|
|
201
|
+
# P2-5: deterministic structural validation runs BEFORE the rest of the
|
|
202
|
+
# analysis (and therefore before the checklist is extracted downstream)
|
|
203
|
+
# so a malformed/contradictory spec is flagged early with an actionable
|
|
204
|
+
# message instead of silently producing a garbage checklist.
|
|
205
|
+
self.validate_structure()
|
|
193
206
|
total_weight = 0.0
|
|
194
207
|
earned_weight = 0.0
|
|
195
208
|
|
|
@@ -281,6 +294,157 @@ class PrdAnalyzer:
|
|
|
281
294
|
elif word_count > 500 and self.scope == "small":
|
|
282
295
|
self.scope = "medium"
|
|
283
296
|
|
|
297
|
+
def validate_structure(self):
|
|
298
|
+
"""Deterministic structural validation of the spec (P2-5).
|
|
299
|
+
|
|
300
|
+
Runs before checklist extraction so a malformed/contradictory spec is
|
|
301
|
+
caught early with an actionable message rather than producing a garbage
|
|
302
|
+
checklist. All checks are regex/stdlib based and deterministic.
|
|
303
|
+
|
|
304
|
+
Severity policy: only a TRULY UNUSABLE spec (no readable text / binary
|
|
305
|
+
garbage) is an ISSUE (Status FAIL). Everything else is a WARNING, so a
|
|
306
|
+
shallow heuristic never marks a valid spec FAIL. This is deliberate:
|
|
307
|
+
the one-line-brief input mode is supported, and nothing downstream
|
|
308
|
+
currently blocks on FAIL anyway (see deferral note below), so WARNING
|
|
309
|
+
is the honest severity for "structure-thin but possibly valid input".
|
|
310
|
+
|
|
311
|
+
ISSUE (high confidence, Status FAIL):
|
|
312
|
+
1. Parseable / decodable text -- must contain readable word
|
|
313
|
+
characters and not be majority-undecodable bytes. A file of pure
|
|
314
|
+
punctuation or binary content cannot yield a real checklist.
|
|
315
|
+
|
|
316
|
+
WARNINGS (surfaced early, do not flip Status to FAIL):
|
|
317
|
+
2. Headings present -- at least one Markdown heading
|
|
318
|
+
(``# ...``) so sections can be located. An "all prose" spec with
|
|
319
|
+
zero structure yields a less reliable checklist, but a one-line
|
|
320
|
+
brief is still valid input -> WARNING, not FAIL.
|
|
321
|
+
3. Referenced LOCAL docs exist -- only explicit Markdown links to
|
|
322
|
+
LOCAL, RELATIVE files (``[text](./relative.md)``), resolved
|
|
323
|
+
against the PRD's parent directory. Specs legitimately describe
|
|
324
|
+
files to be BUILT, so a missing path is a WARNING; only docs the
|
|
325
|
+
author claims already exist are flagged, and only as a warning.
|
|
326
|
+
4. Trivial self-contradiction -- the same requirement phrased as
|
|
327
|
+
both "must X" and "must not X" on an identical short predicate.
|
|
328
|
+
This is a shallow LEXICAL heuristic only: it has no notion of the
|
|
329
|
+
subject, so "all data must be encrypted" + "public assets must
|
|
330
|
+
not be encrypted" collide on the predicate "be encrypted" even
|
|
331
|
+
though they do not actually conflict. Because of that
|
|
332
|
+
false-positive risk it is a WARNING, never an ISSUE. It does NOT
|
|
333
|
+
do semantic contradiction detection, cross-section reasoning, or
|
|
334
|
+
circular dependency analysis -- that deeper work lives in the
|
|
335
|
+
spec-interrogation pipeline, not here.
|
|
336
|
+
|
|
337
|
+
Populates ``self.structure`` = {"ok", "issues", "warnings"}. Empty/
|
|
338
|
+
missing-file cases are already raised by ``load()`` before this runs.
|
|
339
|
+
Never raises; never changes the process exit code (callers such as
|
|
340
|
+
run.sh invoke the analyzer best-effort and gate on the observations
|
|
341
|
+
file, not the exit status).
|
|
342
|
+
"""
|
|
343
|
+
issues = []
|
|
344
|
+
warnings = []
|
|
345
|
+
|
|
346
|
+
text = self.content or ""
|
|
347
|
+
|
|
348
|
+
# --- Check 1: parseable / decodable -------------------------------
|
|
349
|
+
# load() already replaced undecodable bytes with U+FFFD and rejected
|
|
350
|
+
# empty content. A spec that is overwhelmingly replacement characters
|
|
351
|
+
# or has no word characters at all is effectively unparseable.
|
|
352
|
+
word_chars = len(re.findall(r"\w", text))
|
|
353
|
+
replacement_chars = text.count("�")
|
|
354
|
+
if word_chars == 0:
|
|
355
|
+
issues.append(
|
|
356
|
+
"Spec contains no readable text (no word characters found). "
|
|
357
|
+
"Provide a Markdown/plain-text spec with actual requirements."
|
|
358
|
+
)
|
|
359
|
+
elif replacement_chars > 0 and replacement_chars > word_chars:
|
|
360
|
+
issues.append(
|
|
361
|
+
"Spec appears to be binary or wrong-encoding content "
|
|
362
|
+
f"({replacement_chars} undecodable bytes vs {word_chars} text "
|
|
363
|
+
"characters). Provide a UTF-8 Markdown/plain-text spec."
|
|
364
|
+
)
|
|
365
|
+
|
|
366
|
+
# --- Check 2: headings present ------------------------------------
|
|
367
|
+
# Use self.lines so the (optional) architecture doc counts too.
|
|
368
|
+
heading_count = sum(1 for ln in self.lines if re.match(r"^\s{0,3}#{1,6}\s+\S", ln))
|
|
369
|
+
if heading_count == 0:
|
|
370
|
+
warnings.append(
|
|
371
|
+
"Spec has no Markdown headings (no '# ...' lines). The checklist "
|
|
372
|
+
"will be guessed from unstructured prose, which is less reliable. "
|
|
373
|
+
"Add section headings (e.g. ## Features, ## Acceptance Criteria) "
|
|
374
|
+
"if this is more than a one-line brief."
|
|
375
|
+
)
|
|
376
|
+
|
|
377
|
+
# --- Check 3: referenced LOCAL relative docs exist ----------------
|
|
378
|
+
# Only flag explicit Markdown links to local, relative paths. URLs,
|
|
379
|
+
# anchors, mailto, and absolute paths are skipped. A PRD describes
|
|
380
|
+
# files to be BUILT, so a missing path is a WARNING, not a hard issue.
|
|
381
|
+
base_dir = self.prd_path.parent if self.prd_path.parent != Path("") else Path(".")
|
|
382
|
+
seen_targets = set()
|
|
383
|
+
for m in re.finditer(r"\[[^\]]+\]\(([^)]+)\)", text):
|
|
384
|
+
target = m.group(1).strip()
|
|
385
|
+
# Strip an optional title: [t](path "title")
|
|
386
|
+
target = target.split()[0] if target else target
|
|
387
|
+
if not target or target in seen_targets:
|
|
388
|
+
continue
|
|
389
|
+
seen_targets.add(target)
|
|
390
|
+
low = target.lower()
|
|
391
|
+
# Skip non-local references.
|
|
392
|
+
if (
|
|
393
|
+
"://" in target
|
|
394
|
+
or low.startswith(("http:", "https:", "ftp:", "mailto:", "tel:", "#", "data:"))
|
|
395
|
+
or target.startswith("/")
|
|
396
|
+
or target.startswith("~")
|
|
397
|
+
):
|
|
398
|
+
continue
|
|
399
|
+
# Only consider links that look like a doc/asset reference, i.e.
|
|
400
|
+
# they have a file extension. A bare word in parens is more likely
|
|
401
|
+
# to be incidental than a real file reference.
|
|
402
|
+
stem = target.split("#")[0].split("?")[0]
|
|
403
|
+
if "." not in os.path.basename(stem):
|
|
404
|
+
continue
|
|
405
|
+
candidate = (base_dir / stem)
|
|
406
|
+
try:
|
|
407
|
+
exists = candidate.exists()
|
|
408
|
+
except OSError:
|
|
409
|
+
exists = False
|
|
410
|
+
if not exists:
|
|
411
|
+
warnings.append(
|
|
412
|
+
f"Referenced local file not found: '{target}' "
|
|
413
|
+
f"(resolved to '{candidate}'). If this doc is supposed to "
|
|
414
|
+
"already exist, add it or fix the link; if it describes "
|
|
415
|
+
"something to be built, this can be ignored."
|
|
416
|
+
)
|
|
417
|
+
|
|
418
|
+
# --- Check 4: trivial self-contradiction (BASIC, shallow) ---------
|
|
419
|
+
# Catch only the most obvious lexical case: identical short predicate
|
|
420
|
+
# appearing as both "must <p>" and "must not <p>". This is a deliberate
|
|
421
|
+
# shallow heuristic. Real contradiction/circularity detection is out of
|
|
422
|
+
# scope here (see spec-interrogation pipeline).
|
|
423
|
+
must_pos = {}
|
|
424
|
+
must_neg = set()
|
|
425
|
+
for ln in self.lines:
|
|
426
|
+
low = ln.lower()
|
|
427
|
+
for mm in re.finditer(r"\bmust\s+not\s+([a-z][a-z0-9 _-]{2,40}?)(?=[.,;:)]|$)", low):
|
|
428
|
+
must_neg.add(mm.group(1).strip())
|
|
429
|
+
for mm in re.finditer(r"\bmust\s+(?!not\b)([a-z][a-z0-9 _-]{2,40}?)(?=[.,;:)]|$)", low):
|
|
430
|
+
pred = mm.group(1).strip()
|
|
431
|
+
must_pos.setdefault(pred, ln.strip()[:120])
|
|
432
|
+
contradictions = sorted(set(must_pos) & must_neg)
|
|
433
|
+
for pred in contradictions[:5]:
|
|
434
|
+
warnings.append(
|
|
435
|
+
f"Possible self-contradiction: the spec says both 'must {pred}' "
|
|
436
|
+
f"and 'must not {pred}'. If these apply to the same subject, "
|
|
437
|
+
"resolve the conflict. (Basic lexical check, ignores subject; "
|
|
438
|
+
"may be a false positive -- review manually.)"
|
|
439
|
+
)
|
|
440
|
+
|
|
441
|
+
self.structure = {
|
|
442
|
+
"ok": len(issues) == 0,
|
|
443
|
+
"issues": issues,
|
|
444
|
+
"warnings": warnings,
|
|
445
|
+
}
|
|
446
|
+
return self.structure
|
|
447
|
+
|
|
284
448
|
def generate_observations(self):
|
|
285
449
|
"""Generate the observations markdown content."""
|
|
286
450
|
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
@@ -309,9 +473,40 @@ class PrdAnalyzer:
|
|
|
309
473
|
f"**Quality Score:** {self.score}/10",
|
|
310
474
|
f"**Estimated Scope:** {self.scope} (~{self.feature_count} items detected)",
|
|
311
475
|
f"",
|
|
476
|
+
]
|
|
477
|
+
|
|
478
|
+
# P2-5: structural validation section. This is the durable, visible
|
|
479
|
+
# channel: run.sh invokes the analyzer with stderr discarded and the
|
|
480
|
+
# exit code swallowed, so the observations file is what downstream
|
|
481
|
+
# readers (and the operator) actually see.
|
|
482
|
+
struct = getattr(self, "structure", {"ok": True, "issues": [], "warnings": []})
|
|
483
|
+
status = "PASS" if struct.get("ok", True) else "FAIL"
|
|
484
|
+
lines.append("## Structural Validation")
|
|
485
|
+
lines.append("")
|
|
486
|
+
lines.append(f"**Status:** {status}")
|
|
487
|
+
lines.append("")
|
|
488
|
+
if struct.get("issues"):
|
|
489
|
+
lines.append("Structural issues detected (fix these before relying "
|
|
490
|
+
"on the generated checklist):")
|
|
491
|
+
lines.append("")
|
|
492
|
+
for issue in struct["issues"]:
|
|
493
|
+
lines.append(f"- {issue}")
|
|
494
|
+
lines.append("")
|
|
495
|
+
if struct.get("warnings"):
|
|
496
|
+
lines.append("Warnings (lower confidence, review manually):")
|
|
497
|
+
lines.append("")
|
|
498
|
+
for warn in struct["warnings"]:
|
|
499
|
+
lines.append(f"- {warn}")
|
|
500
|
+
lines.append("")
|
|
501
|
+
if struct.get("ok") and not struct.get("warnings"):
|
|
502
|
+
lines.append("- Spec is parseable, has headings, and has no "
|
|
503
|
+
"obvious self-contradictions.")
|
|
504
|
+
lines.append("")
|
|
505
|
+
|
|
506
|
+
lines.extend([
|
|
312
507
|
f"## Strengths",
|
|
313
508
|
f"",
|
|
314
|
-
]
|
|
509
|
+
])
|
|
315
510
|
if strengths:
|
|
316
511
|
lines.extend(strengths)
|
|
317
512
|
else:
|
|
@@ -475,6 +670,25 @@ def main():
|
|
|
475
670
|
|
|
476
671
|
write_atomic(args.output, observations)
|
|
477
672
|
print(f"PRD analysis complete: score={analyzer.score}/10 scope={analyzer.scope}")
|
|
673
|
+
# P2-5: surface structural validation on stdout (run.sh keeps stdout in
|
|
674
|
+
# its log; only stderr is discarded). Exit code intentionally stays 0
|
|
675
|
+
# for a structurally-suspect-but-non-empty spec to match the
|
|
676
|
+
# best-effort, never-blocks contract other callers rely on.
|
|
677
|
+
struct = getattr(analyzer, "structure", {"ok": True, "issues": [], "warnings": []})
|
|
678
|
+
if not struct.get("ok", True):
|
|
679
|
+
print(
|
|
680
|
+
"PRD structure check: FAIL ("
|
|
681
|
+
+ f"{len(struct.get('issues', []))} issue(s)) -- "
|
|
682
|
+
+ "see Structural Validation section in observations"
|
|
683
|
+
)
|
|
684
|
+
elif struct.get("warnings"):
|
|
685
|
+
print(
|
|
686
|
+
"PRD structure check: PASS with "
|
|
687
|
+
+ f"{len(struct['warnings'])} warning(s) -- "
|
|
688
|
+
+ "see Structural Validation section in observations"
|
|
689
|
+
)
|
|
690
|
+
else:
|
|
691
|
+
print("PRD structure check: PASS")
|
|
478
692
|
print(f"Observations written to: {args.output}")
|
|
479
693
|
|
|
480
694
|
except FileNotFoundError as e:
|
|
@@ -48,6 +48,13 @@ CHECKLIST_DIR=""
|
|
|
48
48
|
CHECKLIST_FILE=""
|
|
49
49
|
CHECKLIST_RESULTS_FILE=""
|
|
50
50
|
CHECKLIST_LAST_VERIFY_ITERATION=0
|
|
51
|
+
# Path to the spec/PRD that drove this checklist. Used by the acceptance-oracle
|
|
52
|
+
# triangulation (checklist_oracle_triangulate) to compare what the spec ASSERTS
|
|
53
|
+
# against actual codebase reality. Empty in codebase-analysis mode (no PRD).
|
|
54
|
+
CHECKLIST_PRD_PATH=""
|
|
55
|
+
|
|
56
|
+
# Acceptance-oracle triangulation toggle (default-on, opt-out).
|
|
57
|
+
CHECKLIST_ORACLE_ENABLED=${LOKI_CHECKLIST_ORACLE:-1}
|
|
51
58
|
|
|
52
59
|
#===============================================================================
|
|
53
60
|
# Initialization
|
|
@@ -66,13 +73,310 @@ checklist_init() {
|
|
|
66
73
|
|
|
67
74
|
mkdir -p "$CHECKLIST_DIR"
|
|
68
75
|
|
|
76
|
+
# Remember the spec path so verify-time triangulation can read what the spec
|
|
77
|
+
# asserts. Only store a real, readable file (codebase-analysis mode has none).
|
|
69
78
|
if [ -n "$prd_path" ] && [ -f "$prd_path" ]; then
|
|
79
|
+
CHECKLIST_PRD_PATH="$prd_path"
|
|
70
80
|
log_info "PRD checklist initialized for: $prd_path"
|
|
71
81
|
fi
|
|
72
82
|
|
|
73
83
|
return 0
|
|
74
84
|
}
|
|
75
85
|
|
|
86
|
+
#===============================================================================
|
|
87
|
+
# Acceptance-Oracle Triangulation (P2-3)
|
|
88
|
+
#===============================================================================
|
|
89
|
+
# The PRD checklist derives acceptance criteria from the SPEC alone. That is a
|
|
90
|
+
# single oracle: if the spec is wrong, every check can pass while the product is
|
|
91
|
+
# wrong. This function adds a SECOND independent oracle -- actual codebase
|
|
92
|
+
# reality -- and triangulates the two:
|
|
93
|
+
#
|
|
94
|
+
# (1) spec : what the PRD/spec ASSERTS (e.g. "uses PostgreSQL")
|
|
95
|
+
# (2) reality : what the codebase actually wires (deps + import/usage signals)
|
|
96
|
+
#
|
|
97
|
+
# When the spec asserts a stack the codebase CONTRADICTS (reality shows a
|
|
98
|
+
# DIFFERENT, mutually-exclusive choice and NOT the spec's), we EMIT A FINDING
|
|
99
|
+
# rather than silently letting the spec win. The finding is written to
|
|
100
|
+
# .loki/checklist/oracle-findings.json and surfaced to the completion council via
|
|
101
|
+
# checklist_as_evidence(), and a best-effort trust event is recorded.
|
|
102
|
+
#
|
|
103
|
+
# Honest scope (v7.x, P2-3 first slice):
|
|
104
|
+
# IMPLEMENTED: the spec-vs-reality CONFLICT dimension for the database/datastore
|
|
105
|
+
# choice, which is the highest-signal, lowest-false-positive triangulation
|
|
106
|
+
# (the choices are mutually exclusive and have unambiguous dependency markers).
|
|
107
|
+
#
|
|
108
|
+
# DEFERRED (documented follow-up, NOT faked): the third leg of full
|
|
109
|
+
# triangulation -- DOMAIN INVARIANTS (auth must hash, money must not be float,
|
|
110
|
+
# PII must be encrypted) -- and additional stack axes (web framework, language
|
|
111
|
+
# runtime). These need an invariant catalog plus per-language AST-aware
|
|
112
|
+
# detection to avoid false positives; a regex grep would be noisy. They are
|
|
113
|
+
# intentionally left out of this slice. Backlog: P2-3 follow-up "domain-
|
|
114
|
+
# invariant oracle". See the conflict-detection design here as the template.
|
|
115
|
+
#
|
|
116
|
+
# Non-conflict cases that must NOT fire (guarded + tested):
|
|
117
|
+
# - spec asserts DB X, codebase has nothing wired yet (greenfield): ABSENT is
|
|
118
|
+
# not a contradiction -> no finding.
|
|
119
|
+
# - spec asserts DB X, codebase wires DB X: agreement -> no finding.
|
|
120
|
+
# - spec asserts nothing about a datastore: no spec oracle -> no finding.
|
|
121
|
+
#
|
|
122
|
+
# Opt-out: LOKI_CHECKLIST_ORACLE=0 (default on). When off, this is a no-op.
|
|
123
|
+
|
|
124
|
+
# Emit oracle findings JSON for the current project. Pure detection in python3
|
|
125
|
+
# (env-var inputs only, never positional args inside the heredoc), so the bash
|
|
126
|
+
# wrapper just decides whether anything was written.
|
|
127
|
+
checklist_oracle_triangulate() {
|
|
128
|
+
if [ "$CHECKLIST_ORACLE_ENABLED" != "1" ]; then
|
|
129
|
+
return 0
|
|
130
|
+
fi
|
|
131
|
+
|
|
132
|
+
# Need a spec oracle to triangulate against. No spec -> nothing to do.
|
|
133
|
+
if [ -z "$CHECKLIST_PRD_PATH" ] || [ ! -f "$CHECKLIST_PRD_PATH" ]; then
|
|
134
|
+
return 0
|
|
135
|
+
fi
|
|
136
|
+
|
|
137
|
+
local findings_file="${CHECKLIST_DIR:-".loki/checklist"}/oracle-findings.json"
|
|
138
|
+
local project_dir
|
|
139
|
+
project_dir="$(pwd)"
|
|
140
|
+
|
|
141
|
+
# Feed the detector to python3 via a quoted heredoc (delimiter in quotes) so
|
|
142
|
+
# bash performs NO interpolation: no dollar-digit footgun, no quote-escaping
|
|
143
|
+
# hazards. All inputs arrive through _ORACLE_* env vars.
|
|
144
|
+
local status_token
|
|
145
|
+
status_token=$(_ORACLE_SPEC="$CHECKLIST_PRD_PATH" \
|
|
146
|
+
_ORACLE_OUT="$findings_file" \
|
|
147
|
+
_ORACLE_PROJECT="$project_dir" \
|
|
148
|
+
python3 - <<'ORACLE_PY' 2>/dev/null || echo "NOOP"
|
|
149
|
+
import json, os, re, sys, tempfile, glob
|
|
150
|
+
|
|
151
|
+
spec_path = os.environ["_ORACLE_SPEC"]
|
|
152
|
+
out_path = os.environ["_ORACLE_OUT"]
|
|
153
|
+
project = os.environ["_ORACLE_PROJECT"]
|
|
154
|
+
|
|
155
|
+
# --- Datastore catalog: canonical name -> (spec regex, reality dependency/usage
|
|
156
|
+
# markers). Choices are mutually exclusive enough that a different one being
|
|
157
|
+
# wired while the spec names another is a genuine contradiction. ---
|
|
158
|
+
DATASTORES = {
|
|
159
|
+
"postgresql": {
|
|
160
|
+
"spec": r"(?i)\b(postgres(?:ql)?|psql)\b",
|
|
161
|
+
# python deps / node deps / connection markers
|
|
162
|
+
"deps": [r"(?i)\bpsycopg2?\b", r"(?i)\basyncpg\b",
|
|
163
|
+
r'(?i)"pg"\s*:', r"(?i)\bpg-promise\b", r"(?i)\bpostgres\b",
|
|
164
|
+
r"(?i)\bsequelize\b.*postgres", r"(?i)postgresql://"],
|
|
165
|
+
},
|
|
166
|
+
"mysql": {
|
|
167
|
+
"spec": r"(?i)\b(mysql|mariadb)\b",
|
|
168
|
+
"deps": [r"(?i)\bpymysql\b", r"(?i)\bmysqlclient\b",
|
|
169
|
+
r'(?i)"mysql2?"\s*:', r"(?i)mysql://"],
|
|
170
|
+
},
|
|
171
|
+
"mongodb": {
|
|
172
|
+
"spec": r"(?i)\b(mongo(?:db)?)\b",
|
|
173
|
+
"deps": [r"(?i)\bpymongo\b", r"(?i)\bmongoengine\b", r"(?i)\bmotor\b",
|
|
174
|
+
r'(?i)"mongoose"\s*:', r'(?i)"mongodb"\s*:', r"(?i)mongodb(?:\+srv)?://"],
|
|
175
|
+
},
|
|
176
|
+
"sqlite": {
|
|
177
|
+
"spec": r"(?i)\bsqlite\b",
|
|
178
|
+
"deps": [r"(?i)\bsqlite3\b", r"(?i)\baiosqlite\b",
|
|
179
|
+
r'(?i)"better-sqlite3"\s*:', r"(?i)sqlite://"],
|
|
180
|
+
},
|
|
181
|
+
"redis": {
|
|
182
|
+
"spec": r"(?i)\bredis\b",
|
|
183
|
+
"deps": [r"(?i)\bredis\b", r"(?i)\bioredis\b", r"(?i)redis://"],
|
|
184
|
+
},
|
|
185
|
+
"dynamodb": {
|
|
186
|
+
"spec": r"(?i)\bdynamo(?:db)?\b",
|
|
187
|
+
"deps": [r"(?i)\bboto3\b.*dynamo", r"(?i)dynamodb", r'(?i)"@aws-sdk/client-dynamodb"'],
|
|
188
|
+
},
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
# Datastores that are caches/secondary by nature: their presence alongside a
|
|
192
|
+
# primary DB is NOT a contradiction (apps routinely use both). Treat them as
|
|
193
|
+
# spec-only signals but never as the "reality contradicts" side.
|
|
194
|
+
SECONDARY = {"redis"}
|
|
195
|
+
|
|
196
|
+
def read_text(path, limit=400000):
|
|
197
|
+
try:
|
|
198
|
+
with open(path, "r", errors="replace") as f:
|
|
199
|
+
return f.read(limit)
|
|
200
|
+
except Exception:
|
|
201
|
+
return ""
|
|
202
|
+
|
|
203
|
+
spec_text = read_text(spec_path)
|
|
204
|
+
if not spec_text.strip():
|
|
205
|
+
print("NOOP")
|
|
206
|
+
sys.exit(0)
|
|
207
|
+
|
|
208
|
+
# --- Spec oracle: which datastores does the spec ASSERT? ---
|
|
209
|
+
spec_asserts = set()
|
|
210
|
+
for name, cfg in DATASTORES.items():
|
|
211
|
+
if re.search(cfg["spec"], spec_text):
|
|
212
|
+
spec_asserts.add(name)
|
|
213
|
+
|
|
214
|
+
# If the spec names exactly one PRIMARY datastore, we can triangulate it. If it
|
|
215
|
+
# names several primaries, the spec itself is ambiguous about the datastore;
|
|
216
|
+
# triangulation of "the" choice is unsound, so skip (spec-interrogation owns
|
|
217
|
+
# spec-internal ambiguity, not this oracle).
|
|
218
|
+
spec_primary = sorted(spec_asserts - SECONDARY)
|
|
219
|
+
if len(spec_primary) != 1:
|
|
220
|
+
print("NO_SPEC_DB")
|
|
221
|
+
sys.exit(0)
|
|
222
|
+
spec_db = spec_primary[0]
|
|
223
|
+
|
|
224
|
+
# --- Reality oracle: scan a bounded set of dependency/lock/source manifests for
|
|
225
|
+
# datastore markers. We scan manifests first (highest signal, lowest noise). ---
|
|
226
|
+
MANIFEST_GLOBS = [
|
|
227
|
+
"package.json", "requirements.txt", "requirements/*.txt", "pyproject.toml",
|
|
228
|
+
"Pipfile", "poetry.lock", "go.mod", "Gemfile", "pom.xml", "build.gradle",
|
|
229
|
+
"composer.json", "Cargo.toml",
|
|
230
|
+
".env.example", ".env.sample", "docker-compose.yml", "docker-compose.yaml",
|
|
231
|
+
]
|
|
232
|
+
SKIP_DIRS = {".git", "node_modules", ".loki", "__pycache__", ".venv", "venv",
|
|
233
|
+
"dist", "build", ".next", "vendor"}
|
|
234
|
+
|
|
235
|
+
manifest_text_parts = []
|
|
236
|
+
for pat in MANIFEST_GLOBS:
|
|
237
|
+
for p in glob.glob(os.path.join(project, pat)):
|
|
238
|
+
if os.path.isfile(p):
|
|
239
|
+
manifest_text_parts.append(read_text(p, 200000))
|
|
240
|
+
manifest_text = "\n".join(manifest_text_parts)
|
|
241
|
+
|
|
242
|
+
# Light source scan for connection-string usage as a secondary reality signal,
|
|
243
|
+
# bounded to a small number of files to stay fast and avoid scanning artifacts.
|
|
244
|
+
def source_scan_hit(markers, cap_files=400):
|
|
245
|
+
seen = 0
|
|
246
|
+
for root, dirs, files in os.walk(project):
|
|
247
|
+
dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
|
|
248
|
+
for fn in files:
|
|
249
|
+
if not fn.endswith((".py", ".js", ".ts", ".tsx", ".jsx", ".go",
|
|
250
|
+
".rb", ".java", ".rs", ".php", ".env")):
|
|
251
|
+
continue
|
|
252
|
+
seen += 1
|
|
253
|
+
if seen > cap_files:
|
|
254
|
+
return False
|
|
255
|
+
txt = read_text(os.path.join(root, fn), 80000)
|
|
256
|
+
for m in markers:
|
|
257
|
+
if re.search(m, txt):
|
|
258
|
+
return True
|
|
259
|
+
return False
|
|
260
|
+
|
|
261
|
+
def reality_has(name):
|
|
262
|
+
cfg = DATASTORES[name]
|
|
263
|
+
for m in cfg["deps"]:
|
|
264
|
+
if re.search(m, manifest_text):
|
|
265
|
+
return True
|
|
266
|
+
# Only fall back to a source scan for the cheap connection-string style
|
|
267
|
+
# markers (URLs); dependency-name markers belong in manifests.
|
|
268
|
+
url_markers = [m for m in cfg["deps"] if "://" in m]
|
|
269
|
+
if url_markers and source_scan_hit(url_markers):
|
|
270
|
+
return True
|
|
271
|
+
return False
|
|
272
|
+
|
|
273
|
+
reality_dbs = set()
|
|
274
|
+
for name in DATASTORES:
|
|
275
|
+
if name in SECONDARY:
|
|
276
|
+
continue
|
|
277
|
+
if reality_has(name):
|
|
278
|
+
reality_dbs.add(name)
|
|
279
|
+
|
|
280
|
+
findings = []
|
|
281
|
+
|
|
282
|
+
# Conflict iff: spec asserts spec_db, AND reality shows a DIFFERENT primary db,
|
|
283
|
+
# AND reality does NOT also show the spec db. Absent reality (greenfield) and
|
|
284
|
+
# agreement both yield no finding.
|
|
285
|
+
contradicting = sorted(d for d in reality_dbs if d != spec_db)
|
|
286
|
+
if contradicting and spec_db not in reality_dbs:
|
|
287
|
+
findings.append({
|
|
288
|
+
"id": "oracle-datastore-conflict",
|
|
289
|
+
"dimension": "spec_vs_reality",
|
|
290
|
+
"axis": "datastore",
|
|
291
|
+
"severity": "high",
|
|
292
|
+
"spec_asserts": spec_db,
|
|
293
|
+
"codebase_reality": contradicting,
|
|
294
|
+
"title": "Spec asserts %s but codebase wires %s" % (
|
|
295
|
+
spec_db, ", ".join(contradicting)),
|
|
296
|
+
"detail": ("The spec/PRD names %s as the datastore, but the codebase "
|
|
297
|
+
"dependencies/usage indicate %s is wired and %s is not. "
|
|
298
|
+
"Acceptance criteria derived from the spec alone would pass "
|
|
299
|
+
"against the wrong datastore. Resolve which oracle is correct "
|
|
300
|
+
"(update the spec or the implementation) before completion."
|
|
301
|
+
% (spec_db, ", ".join(contradicting), spec_db)),
|
|
302
|
+
})
|
|
303
|
+
|
|
304
|
+
result = {
|
|
305
|
+
"version": 1,
|
|
306
|
+
"spec_path": spec_path,
|
|
307
|
+
"spec_datastore": spec_db,
|
|
308
|
+
"reality_datastores": sorted(reality_dbs),
|
|
309
|
+
"findings": findings,
|
|
310
|
+
"deferred": [
|
|
311
|
+
"domain-invariant oracle (auth-hashing, money-not-float, PII-encryption)",
|
|
312
|
+
"additional stack axes (web framework, language runtime)",
|
|
313
|
+
],
|
|
314
|
+
}
|
|
315
|
+
|
|
316
|
+
# Always write the result so evidence/council can read both findings AND a clean
|
|
317
|
+
# "checked, no conflict" record (honest: absence of a finding is itself signal).
|
|
318
|
+
d = os.path.dirname(out_path) or "."
|
|
319
|
+
os.makedirs(d, exist_ok=True)
|
|
320
|
+
fd, tmp = tempfile.mkstemp(dir=d, suffix=".tmp")
|
|
321
|
+
with os.fdopen(fd, "w") as f:
|
|
322
|
+
json.dump(result, f, indent=2)
|
|
323
|
+
f.write("\n")
|
|
324
|
+
os.replace(tmp, out_path)
|
|
325
|
+
|
|
326
|
+
if findings:
|
|
327
|
+
print("CONFLICT %d" % len(findings))
|
|
328
|
+
else:
|
|
329
|
+
print("CLEAN")
|
|
330
|
+
ORACLE_PY
|
|
331
|
+
)
|
|
332
|
+
|
|
333
|
+
# Honest logging + best-effort trust event on a real conflict only.
|
|
334
|
+
local tok rest
|
|
335
|
+
read -r tok rest <<< "$status_token"
|
|
336
|
+
case "$tok" in
|
|
337
|
+
CONFLICT)
|
|
338
|
+
log_warn "[checklist] acceptance-oracle conflict: spec disagrees with codebase reality (${rest:-1} finding(s)); see ${findings_file}"
|
|
339
|
+
if type record_trust_event_bash &>/dev/null; then
|
|
340
|
+
record_trust_event_bash "oracle_conflict" \
|
|
341
|
+
"dimension=spec_vs_reality" \
|
|
342
|
+
"axis=datastore" \
|
|
343
|
+
"findings=${rest:-1}" \
|
|
344
|
+
>/dev/null 2>&1 || true
|
|
345
|
+
fi
|
|
346
|
+
;;
|
|
347
|
+
esac
|
|
348
|
+
|
|
349
|
+
return 0
|
|
350
|
+
}
|
|
351
|
+
|
|
352
|
+
# Echo the oracle findings as a formatted block for council evidence. Empty when
|
|
353
|
+
# no findings (or oracle disabled / not run).
|
|
354
|
+
checklist_oracle_evidence() {
|
|
355
|
+
local findings_file="${CHECKLIST_DIR:-".loki/checklist"}/oracle-findings.json"
|
|
356
|
+
if [ ! -f "$findings_file" ]; then
|
|
357
|
+
return 0
|
|
358
|
+
fi
|
|
359
|
+
_ORACLE_OUT="$findings_file" python3 -c '
|
|
360
|
+
import json, os
|
|
361
|
+
try:
|
|
362
|
+
with open(os.environ["_ORACLE_OUT"]) as f:
|
|
363
|
+
data = json.load(f)
|
|
364
|
+
except Exception:
|
|
365
|
+
raise SystemExit(0)
|
|
366
|
+
findings = data.get("findings", [])
|
|
367
|
+
if not findings:
|
|
368
|
+
raise SystemExit(0)
|
|
369
|
+
print("")
|
|
370
|
+
print("### Acceptance-Oracle Triangulation (spec vs codebase reality)")
|
|
371
|
+
for fnd in findings:
|
|
372
|
+
sev = str(fnd.get("severity", "high")).upper()
|
|
373
|
+
print(" [FAIL] [%s] %s" % (sev, fnd.get("title", fnd.get("id", "?"))))
|
|
374
|
+
detail = fnd.get("detail", "")
|
|
375
|
+
if detail:
|
|
376
|
+
print(" %s" % detail)
|
|
377
|
+
' 2>/dev/null || true
|
|
378
|
+
}
|
|
379
|
+
|
|
76
380
|
#===============================================================================
|
|
77
381
|
# Interval Control
|
|
78
382
|
#===============================================================================
|
|
@@ -314,6 +618,12 @@ checklist_verify() {
|
|
|
314
618
|
# first verification-results.json summary already excludes held-out items.
|
|
315
619
|
checklist_select_heldout
|
|
316
620
|
|
|
621
|
+
# Acceptance-oracle triangulation: compare what the spec ASSERTS against
|
|
622
|
+
# actual codebase reality and emit a finding on conflict. Runs at verify-time
|
|
623
|
+
# (not init) so codebase reality actually exists by now even on a greenfield
|
|
624
|
+
# build. Best-effort, default-on, never blocks verification itself.
|
|
625
|
+
checklist_oracle_triangulate 2>/dev/null || true
|
|
626
|
+
|
|
317
627
|
local script_dir
|
|
318
628
|
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
319
629
|
local verify_script="${script_dir}/checklist-verify.py"
|
|
@@ -477,6 +787,11 @@ try:
|
|
|
477
787
|
except Exception:
|
|
478
788
|
print('Checklist data unavailable')
|
|
479
789
|
" 2>/dev/null || echo "Checklist data unavailable"
|
|
790
|
+
|
|
791
|
+
# Append acceptance-oracle triangulation findings (spec vs codebase
|
|
792
|
+
# reality) so the completion council sees a spec-vs-reality conflict as
|
|
793
|
+
# first-class evidence. Emits nothing when there are no findings.
|
|
794
|
+
checklist_oracle_evidence
|
|
480
795
|
} >> "${evidence_file:-/dev/stdout}"
|
|
481
796
|
}
|
|
482
797
|
|