loki-mode 7.107.0 → 7.109.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/autonomy/lib/expectation-ledger.py +366 -0
- package/autonomy/run.sh +12 -3
- package/autonomy/verify.sh +718 -0
- package/dashboard/__init__.py +1 -1
- package/docs/INSTALLATION.md +1 -1
- package/docs/TOP-100-BACKLOG.md +37 -0
- package/loki-ts/dist/loki.js +2 -2
- package/mcp/__init__.py +1 -1
- package/package.json +1 -1
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
package/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: loki-mode
|
|
|
3
3
|
description: Autonomous spec-driven build system with a built-in trust layer. It does not call work done until it is verified (RARV-C closure loop, 8 quality gates, completion council, verified-completion evidence gate). Triggers on "Loki Mode". Takes a spec (PRD, GitHub issue, OpenAPI doc, etc.) to deployed product with minimal human intervention. Provider-agnostic. Requires --dangerously-skip-permissions flag.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Loki Mode v7.
|
|
6
|
+
# Loki Mode v7.109.0
|
|
7
7
|
|
|
8
8
|
**You are an autonomous agent. You make decisions. You do not ask questions. You do not stop.**
|
|
9
9
|
|
|
@@ -408,4 +408,4 @@ See `CHANGELOG.md` entries [7.5.7], [7.5.8], [7.5.13] for the per-fix list and r
|
|
|
408
408
|
|
|
409
409
|
---
|
|
410
410
|
|
|
411
|
-
**v7.
|
|
411
|
+
**v7.109.0 | [Autonomi](https://www.autonomi.dev/) flagship product | ~260 lines core**
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
7.
|
|
1
|
+
7.109.0
|
|
@@ -0,0 +1,366 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Annotate-before-act expected-outcome ledger for Loki Mode.
|
|
3
|
+
|
|
4
|
+
Companion to proof-generator.py / proof-verify.py. Before running verification
|
|
5
|
+
for a change or checklist item, the runtime writes a tamper-evident ledger of
|
|
6
|
+
EXPECTED observable outcomes (e.g. "GET /health -> 200 {status:ok}", "test X
|
|
7
|
+
fails before the fix and passes after", "endpoint Y should now exist"). At
|
|
8
|
+
verify time the actual results are compared to this PRE-COMMITTED prediction:
|
|
9
|
+
any expectation that was silently dropped (never executed) or contradicted
|
|
10
|
+
(actual != expected) becomes a finding. An expectation that cannot be evaluated
|
|
11
|
+
maps to INCONCLUSIVE, never VERIFIED.
|
|
12
|
+
|
|
13
|
+
Why this is trustworthy:
|
|
14
|
+
- The ledger is written BEFORE the act/verify, so it is a genuine prediction,
|
|
15
|
+
not a post-hoc rationalization of whatever happened.
|
|
16
|
+
- It is TAMPER-EVIDENT: the whole ledger is canonicalized and hashed the SAME
|
|
17
|
+
way proof-generator.py hashes proof.json (sha256 over the compact, sorted
|
|
18
|
+
JSON form). The hash is recorded in the ledger file AND embedded into
|
|
19
|
+
evidence.json / proof.json, so an expectation edited after the fact breaks
|
|
20
|
+
the embedded hash and is detectable.
|
|
21
|
+
|
|
22
|
+
Design rules (mirror proof-generator.py):
|
|
23
|
+
- Canonicalization MUST match proof-generator._canonical EXACTLY
|
|
24
|
+
(json.dumps sort_keys=True, compact separators, default ensure_ascii). The
|
|
25
|
+
ledger hash and the proof hash therefore use one identical scheme.
|
|
26
|
+
- Additive + fail-open: nothing writes a ledger yet in the common case, so
|
|
27
|
+
the read/compare side must no-op cleanly when the ledger is absent.
|
|
28
|
+
- Catch broadly on the write path; never raise into the run loop.
|
|
29
|
+
|
|
30
|
+
CLI (bash-callable, mirrors proof-generator.py / proof-verify.py CLIs):
|
|
31
|
+
expectation-ledger.py write --loki-dir .loki --iter 3 \
|
|
32
|
+
--id health-200 --statement "GET /health -> 200" \
|
|
33
|
+
--check-ref tests --expected '{"status":200}'
|
|
34
|
+
expectation-ledger.py hash --loki-dir .loki --iter 3
|
|
35
|
+
expectation-ledger.py verify-hash --loki-dir .loki --iter 3
|
|
36
|
+
"""
|
|
37
|
+
|
|
38
|
+
import argparse
|
|
39
|
+
import hashlib
|
|
40
|
+
import json
|
|
41
|
+
import os
|
|
42
|
+
import sys
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
# ---------------------------------------------------------------------------
|
|
46
|
+
# canonicalization (MUST match proof-generator._canonical / proof-verify._canonical)
|
|
47
|
+
# ---------------------------------------------------------------------------
|
|
48
|
+
|
|
49
|
+
def _canonical(obj):
|
|
50
|
+
"""Canonical JSON form used for the tamper-evident hash.
|
|
51
|
+
|
|
52
|
+
Identical to proof-generator.py _canonical() and proof-verify.py
|
|
53
|
+
_canonical(): json.dumps with sort_keys=True and compact separators, and no
|
|
54
|
+
explicit ensure_ascii (defaults to True). Keeping this byte-for-byte
|
|
55
|
+
identical is what lets the ledger hash and the proof hash share one scheme,
|
|
56
|
+
so a verifier recomputes it the same way for both artifacts.
|
|
57
|
+
"""
|
|
58
|
+
return json.dumps(obj, sort_keys=True, separators=(",", ":"))
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
# ---------------------------------------------------------------------------
|
|
62
|
+
# ledger paths + shape
|
|
63
|
+
# ---------------------------------------------------------------------------
|
|
64
|
+
|
|
65
|
+
def ledger_path(loki_dir, iteration):
|
|
66
|
+
"""Path to the per-iteration ledger: .loki/expectations/<iter>.json."""
|
|
67
|
+
return os.path.join(loki_dir, "expectations", "%s.json" % iteration)
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def _normalize_entry(entry):
|
|
71
|
+
"""Coerce one expectation into the frozen entry shape.
|
|
72
|
+
|
|
73
|
+
Shape: {id, statement, check_ref, expected}. Unknown keys are dropped so the
|
|
74
|
+
canonical hash is stable regardless of caller-supplied extras. `expected` is
|
|
75
|
+
kept as-is (it may be a scalar, dict, or list): the compare step interprets
|
|
76
|
+
it, the ledger only records it verbatim.
|
|
77
|
+
"""
|
|
78
|
+
if not isinstance(entry, dict):
|
|
79
|
+
return None
|
|
80
|
+
eid = entry.get("id")
|
|
81
|
+
if eid is None or str(eid).strip() == "":
|
|
82
|
+
return None
|
|
83
|
+
return {
|
|
84
|
+
"id": str(eid),
|
|
85
|
+
"statement": str(entry.get("statement") or ""),
|
|
86
|
+
"check_ref": str(entry.get("check_ref") or ""),
|
|
87
|
+
"expected": entry.get("expected"),
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def _sorted_entries(entries):
|
|
92
|
+
"""Entries sorted by id so the ledger hash is order-independent.
|
|
93
|
+
|
|
94
|
+
Two runs that record the same expectations in a different order produce the
|
|
95
|
+
same ledger and the same hash -- the prediction is a SET, not a sequence.
|
|
96
|
+
"""
|
|
97
|
+
clean = [e for e in (_normalize_entry(x) for x in entries) if e is not None]
|
|
98
|
+
return sorted(clean, key=lambda e: e["id"])
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def compute_ledger_hash(entries):
|
|
102
|
+
"""sha256 over the canonical form of the sorted entries.
|
|
103
|
+
|
|
104
|
+
Same algorithm proof-generator.py uses for proof.json's integrity hash
|
|
105
|
+
(sha256 of _canonical(...)), so the value embedded into evidence/proof is
|
|
106
|
+
recomputable by the same verifier code path.
|
|
107
|
+
"""
|
|
108
|
+
canonical = _canonical(_sorted_entries(entries)).encode("utf-8")
|
|
109
|
+
return hashlib.sha256(canonical).hexdigest()
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
# ---------------------------------------------------------------------------
|
|
113
|
+
# read / write
|
|
114
|
+
# ---------------------------------------------------------------------------
|
|
115
|
+
|
|
116
|
+
def read_ledger(loki_dir, iteration):
|
|
117
|
+
"""Return the ledger dict, or None when absent / unreadable.
|
|
118
|
+
|
|
119
|
+
Absent ledger is the common case today; callers treat None as "no ledger,
|
|
120
|
+
no-op". Never raises: a malformed file returns None so the read side stays
|
|
121
|
+
fail-open.
|
|
122
|
+
"""
|
|
123
|
+
path = ledger_path(loki_dir, iteration)
|
|
124
|
+
if not os.path.isfile(path):
|
|
125
|
+
return None
|
|
126
|
+
try:
|
|
127
|
+
with open(path, "r") as f:
|
|
128
|
+
data = json.load(f)
|
|
129
|
+
except Exception:
|
|
130
|
+
return None
|
|
131
|
+
if not isinstance(data, dict):
|
|
132
|
+
return None
|
|
133
|
+
return data
|
|
134
|
+
|
|
135
|
+
|
|
136
|
+
def write_entry(loki_dir, iteration, entry):
|
|
137
|
+
"""Append one expectation to the iteration ledger and re-seal the hash.
|
|
138
|
+
|
|
139
|
+
Idempotent per id: writing the same id again REPLACES the prior entry (a
|
|
140
|
+
prediction is refined, not duplicated). Recomputes ledger_sha256 over the
|
|
141
|
+
full entry set after each write so the on-disk file is always self-consistent
|
|
142
|
+
(the recorded hash matches its own entries). Returns the new ledger_sha256,
|
|
143
|
+
or None on failure (never raises into the run loop).
|
|
144
|
+
"""
|
|
145
|
+
norm = _normalize_entry(entry)
|
|
146
|
+
if norm is None:
|
|
147
|
+
return None
|
|
148
|
+
try:
|
|
149
|
+
existing = read_ledger(loki_dir, iteration) or {}
|
|
150
|
+
entries = existing.get("entries")
|
|
151
|
+
if not isinstance(entries, list):
|
|
152
|
+
entries = []
|
|
153
|
+
# Replace-by-id so refining a prediction does not create a duplicate.
|
|
154
|
+
entries = [e for e in entries
|
|
155
|
+
if not (isinstance(e, dict) and str(e.get("id")) == norm["id"])]
|
|
156
|
+
entries.append(norm)
|
|
157
|
+
entries = _sorted_entries(entries)
|
|
158
|
+
|
|
159
|
+
ledger = {
|
|
160
|
+
"schema_version": "1.0",
|
|
161
|
+
"iteration": str(iteration),
|
|
162
|
+
"entries": entries,
|
|
163
|
+
"ledger_sha256": compute_ledger_hash(entries),
|
|
164
|
+
}
|
|
165
|
+
path = ledger_path(loki_dir, iteration)
|
|
166
|
+
os.makedirs(os.path.dirname(path), exist_ok=True)
|
|
167
|
+
tmp = path + ".tmp"
|
|
168
|
+
with open(tmp, "w") as f:
|
|
169
|
+
json.dump(ledger, f, indent=2)
|
|
170
|
+
f.write("\n")
|
|
171
|
+
os.replace(tmp, path)
|
|
172
|
+
return ledger["ledger_sha256"]
|
|
173
|
+
except Exception:
|
|
174
|
+
return None
|
|
175
|
+
|
|
176
|
+
|
|
177
|
+
def ledger_hash_ok(loki_dir, iteration):
|
|
178
|
+
"""True iff the recorded ledger_sha256 matches its own entries.
|
|
179
|
+
|
|
180
|
+
This is the tamper check: an expectation edited after write (without
|
|
181
|
+
re-sealing) makes the stored hash disagree with the recomputed one. Returns
|
|
182
|
+
(ok, recorded, recomputed); ok is None when there is no ledger.
|
|
183
|
+
"""
|
|
184
|
+
ledger = read_ledger(loki_dir, iteration)
|
|
185
|
+
if ledger is None:
|
|
186
|
+
return None, None, None
|
|
187
|
+
recorded = str(ledger.get("ledger_sha256") or "")
|
|
188
|
+
entries = ledger.get("entries")
|
|
189
|
+
if not isinstance(entries, list):
|
|
190
|
+
entries = []
|
|
191
|
+
recomputed = compute_ledger_hash(entries)
|
|
192
|
+
return (recorded == recomputed and bool(recorded)), recorded, recomputed
|
|
193
|
+
|
|
194
|
+
|
|
195
|
+
# ---------------------------------------------------------------------------
|
|
196
|
+
# compare (READ + COMPARE)
|
|
197
|
+
# ---------------------------------------------------------------------------
|
|
198
|
+
|
|
199
|
+
def _stringify(v):
|
|
200
|
+
"""Stable string form for equality comparison of an expected/actual value.
|
|
201
|
+
|
|
202
|
+
Uses the canonical form for dicts/lists so key order does not matter, and a
|
|
203
|
+
plain str() for scalars. Comparison is intentionally exact after this
|
|
204
|
+
normalization -- an expectation is a precise prediction, not a fuzzy match.
|
|
205
|
+
"""
|
|
206
|
+
if isinstance(v, (dict, list)):
|
|
207
|
+
return _canonical(v)
|
|
208
|
+
return str(v)
|
|
209
|
+
|
|
210
|
+
|
|
211
|
+
def compare(loki_dir, iteration, observed):
|
|
212
|
+
"""Compare the pre-committed ledger against observed results.
|
|
213
|
+
|
|
214
|
+
`observed` maps expectation id -> actual observation. Each observation is
|
|
215
|
+
either:
|
|
216
|
+
- a value (scalar/dict/list): compared directly to `expected`; OR
|
|
217
|
+
- a dict {"actual": <value>} for the same; OR
|
|
218
|
+
- a dict {"evaluable": False, ...} to mark an expectation that ran but
|
|
219
|
+
could not be evaluated (-> inconclusive).
|
|
220
|
+
An id NOT present in `observed` is a DROPPED expectation (never executed).
|
|
221
|
+
|
|
222
|
+
Returns a dict:
|
|
223
|
+
{
|
|
224
|
+
"ledger_present": bool,
|
|
225
|
+
"ledger_hash_ok": bool | None, # None when no ledger
|
|
226
|
+
"ledger_sha256": str | None,
|
|
227
|
+
"results": [ {id, statement, check_ref, expected, actual,
|
|
228
|
+
outcome} ], # outcome in met|contradicted|dropped|inconclusive
|
|
229
|
+
"met": int, "contradicted": int, "dropped": int, "inconclusive": int,
|
|
230
|
+
}
|
|
231
|
+
Fully no-op when the ledger is absent: ledger_present False, empty results.
|
|
232
|
+
"""
|
|
233
|
+
out = {
|
|
234
|
+
"ledger_present": False,
|
|
235
|
+
"ledger_hash_ok": None,
|
|
236
|
+
"ledger_sha256": None,
|
|
237
|
+
"results": [],
|
|
238
|
+
"met": 0, "contradicted": 0, "dropped": 0, "inconclusive": 0,
|
|
239
|
+
}
|
|
240
|
+
ledger = read_ledger(loki_dir, iteration)
|
|
241
|
+
if ledger is None:
|
|
242
|
+
return out
|
|
243
|
+
|
|
244
|
+
out["ledger_present"] = True
|
|
245
|
+
out["ledger_sha256"] = str(ledger.get("ledger_sha256") or "") or None
|
|
246
|
+
ok, _rec, _recomp = ledger_hash_ok(loki_dir, iteration)
|
|
247
|
+
out["ledger_hash_ok"] = ok
|
|
248
|
+
|
|
249
|
+
observed = observed if isinstance(observed, dict) else {}
|
|
250
|
+
entries = ledger.get("entries")
|
|
251
|
+
if not isinstance(entries, list):
|
|
252
|
+
entries = []
|
|
253
|
+
|
|
254
|
+
for e in entries:
|
|
255
|
+
if not isinstance(e, dict):
|
|
256
|
+
continue
|
|
257
|
+
eid = str(e.get("id") or "")
|
|
258
|
+
expected = e.get("expected")
|
|
259
|
+
row = {
|
|
260
|
+
"id": eid,
|
|
261
|
+
"statement": str(e.get("statement") or ""),
|
|
262
|
+
"check_ref": str(e.get("check_ref") or ""),
|
|
263
|
+
"expected": expected,
|
|
264
|
+
"actual": None,
|
|
265
|
+
"outcome": "dropped",
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
if eid not in observed:
|
|
269
|
+
# Not executed at all -> dropped. A predicted-and-forgotten check is
|
|
270
|
+
# exactly the silent-drop this ledger exists to catch.
|
|
271
|
+
row["outcome"] = "dropped"
|
|
272
|
+
else:
|
|
273
|
+
obs = observed[eid]
|
|
274
|
+
if isinstance(obs, dict) and obs.get("evaluable") is False:
|
|
275
|
+
row["outcome"] = "inconclusive"
|
|
276
|
+
row["actual"] = obs.get("actual")
|
|
277
|
+
else:
|
|
278
|
+
actual = obs.get("actual") if isinstance(obs, dict) and "actual" in obs else obs
|
|
279
|
+
row["actual"] = actual
|
|
280
|
+
if _stringify(actual) == _stringify(expected):
|
|
281
|
+
row["outcome"] = "met"
|
|
282
|
+
else:
|
|
283
|
+
row["outcome"] = "contradicted"
|
|
284
|
+
|
|
285
|
+
out[row["outcome"]] += 1
|
|
286
|
+
out["results"].append(row)
|
|
287
|
+
|
|
288
|
+
return out
|
|
289
|
+
|
|
290
|
+
|
|
291
|
+
# ---------------------------------------------------------------------------
|
|
292
|
+
# CLI
|
|
293
|
+
# ---------------------------------------------------------------------------
|
|
294
|
+
|
|
295
|
+
def _cmd_write(args):
|
|
296
|
+
try:
|
|
297
|
+
expected = json.loads(args.expected) if args.expected else None
|
|
298
|
+
except Exception:
|
|
299
|
+
# A bare string that is not JSON is recorded verbatim as the expected
|
|
300
|
+
# value (e.g. --expected "200"): honest passthrough, never a crash.
|
|
301
|
+
expected = args.expected
|
|
302
|
+
h = write_entry(args.loki_dir, args.iter, {
|
|
303
|
+
"id": args.id,
|
|
304
|
+
"statement": args.statement,
|
|
305
|
+
"check_ref": args.check_ref,
|
|
306
|
+
"expected": expected,
|
|
307
|
+
})
|
|
308
|
+
if h is None:
|
|
309
|
+
sys.stderr.write("warn: expectation-ledger write failed\n")
|
|
310
|
+
return 1
|
|
311
|
+
print(h)
|
|
312
|
+
return 0
|
|
313
|
+
|
|
314
|
+
|
|
315
|
+
def _cmd_hash(args):
|
|
316
|
+
ledger = read_ledger(args.loki_dir, args.iter)
|
|
317
|
+
if ledger is None:
|
|
318
|
+
sys.stderr.write("no ledger for iteration %s\n" % args.iter)
|
|
319
|
+
return 2
|
|
320
|
+
print(str(ledger.get("ledger_sha256") or ""))
|
|
321
|
+
return 0
|
|
322
|
+
|
|
323
|
+
|
|
324
|
+
def _cmd_verify_hash(args):
|
|
325
|
+
ok, recorded, recomputed = ledger_hash_ok(args.loki_dir, args.iter)
|
|
326
|
+
if ok is None:
|
|
327
|
+
print(json.dumps({"ok": False, "error": "no ledger"}))
|
|
328
|
+
return 2
|
|
329
|
+
print(json.dumps({"ok": bool(ok), "recorded": recorded,
|
|
330
|
+
"recomputed": recomputed}))
|
|
331
|
+
return 0 if ok else 1
|
|
332
|
+
|
|
333
|
+
|
|
334
|
+
def main(argv=None):
|
|
335
|
+
parser = argparse.ArgumentParser(
|
|
336
|
+
description="Loki Mode annotate-before-act expectation ledger")
|
|
337
|
+
sub = parser.add_subparsers(dest="cmd")
|
|
338
|
+
|
|
339
|
+
w = sub.add_parser("write", help="append/replace one expectation")
|
|
340
|
+
w.add_argument("--loki-dir", default=".loki")
|
|
341
|
+
w.add_argument("--iter", required=True)
|
|
342
|
+
w.add_argument("--id", required=True)
|
|
343
|
+
w.add_argument("--statement", default="")
|
|
344
|
+
w.add_argument("--check-ref", default="", dest="check_ref")
|
|
345
|
+
w.add_argument("--expected", default="")
|
|
346
|
+
w.set_defaults(func=_cmd_write)
|
|
347
|
+
|
|
348
|
+
h = sub.add_parser("hash", help="print the recorded ledger_sha256")
|
|
349
|
+
h.add_argument("--loki-dir", default=".loki")
|
|
350
|
+
h.add_argument("--iter", required=True)
|
|
351
|
+
h.set_defaults(func=_cmd_hash)
|
|
352
|
+
|
|
353
|
+
v = sub.add_parser("verify-hash", help="check ledger_sha256 vs its entries")
|
|
354
|
+
v.add_argument("--loki-dir", default=".loki")
|
|
355
|
+
v.add_argument("--iter", required=True)
|
|
356
|
+
v.set_defaults(func=_cmd_verify_hash)
|
|
357
|
+
|
|
358
|
+
args = parser.parse_args(argv)
|
|
359
|
+
if not getattr(args, "func", None):
|
|
360
|
+
parser.print_help()
|
|
361
|
+
return 2
|
|
362
|
+
return args.func(args)
|
|
363
|
+
|
|
364
|
+
|
|
365
|
+
if __name__ == "__main__":
|
|
366
|
+
sys.exit(main())
|
package/autonomy/run.sh
CHANGED
|
@@ -12200,8 +12200,11 @@ parse_claude_reset_time() {
|
|
|
12200
12200
|
local current_min=$(date +%M)
|
|
12201
12201
|
local current_sec=$(date +%S)
|
|
12202
12202
|
|
|
12203
|
-
# Calculate seconds until reset
|
|
12204
|
-
|
|
12203
|
+
# Calculate seconds until reset. Force base-10 (10#) on the zero-padded
|
|
12204
|
+
# date values: 08/09 are invalid octal, so bare arithmetic aborts ("value
|
|
12205
|
+
# too great for base") during those clock windows and silently discards the
|
|
12206
|
+
# real rate-limit reset wait, falling back to a too-short generic backoff.
|
|
12207
|
+
local current_secs=$((10#$current_hour * 3600 + 10#$current_min * 60 + 10#$current_sec))
|
|
12205
12208
|
local reset_secs=$((hour * 3600))
|
|
12206
12209
|
|
|
12207
12210
|
local wait_secs=$((reset_secs - current_secs))
|
|
@@ -12634,8 +12637,14 @@ except Exception:
|
|
|
12634
12637
|
# Return the payload on stdout
|
|
12635
12638
|
printf '%s\n' "$payload"
|
|
12636
12639
|
|
|
12637
|
-
# Consume the signal (next iteration would otherwise re-trigger)
|
|
12640
|
+
# Consume the signal (next iteration would otherwise re-trigger).
|
|
12641
|
+
# Also remove the fallback if it coexists: TASK_COMPLETION_CLAIMED and
|
|
12642
|
+
# COMPLETION_REQUESTED are both valid, non-exclusive completion mechanisms, so
|
|
12643
|
+
# a belt-and-suspenders agent can leave both present. Removing only the active
|
|
12644
|
+
# one orphans the other, which then reads as a phantom claim on a later
|
|
12645
|
+
# iteration and forces every-iteration council evaluation. Consume both.
|
|
12638
12646
|
rm -f "$signal_file" 2>/dev/null
|
|
12647
|
+
rm -f "$fallback_file" 2>/dev/null
|
|
12639
12648
|
return 0
|
|
12640
12649
|
}
|
|
12641
12650
|
|
package/autonomy/verify.sh
CHANGED
|
@@ -723,6 +723,502 @@ print(n)
|
|
|
723
723
|
return 0
|
|
724
724
|
}
|
|
725
725
|
|
|
726
|
+
# ---------------------------------------------------------------------------
|
|
727
|
+
# Gate: runtime boot smoke (NET-NEW).
|
|
728
|
+
#
|
|
729
|
+
# "It actually runs." A shipped change should not only build and pass tests --
|
|
730
|
+
# for an app that serves HTTP, the strongest cheap evidence is that the app
|
|
731
|
+
# BOOTS and answers a request. This gate detects a start command, boots the app
|
|
732
|
+
# with a hard timeout, probes a health/root path, records the HTTP status (and
|
|
733
|
+
# a screenshot artifact when playwright is available), and tears the app down.
|
|
734
|
+
#
|
|
735
|
+
# REUSE (deterministic port of run.sh smoke machinery, source cited):
|
|
736
|
+
# - start-command detection order: app_runner_init (app-runner.sh:757-912).
|
|
737
|
+
# Faithfully ported here (compose/Dockerfile EXCLUDED: docker boots are slow,
|
|
738
|
+
# need a running daemon, and are out of scope for a fast verify gate).
|
|
739
|
+
# - default-port map: _detect_port (app-runner.sh:641-731).
|
|
740
|
+
# - optional screenshot: the inline chromium smoke script from
|
|
741
|
+
# playwright_verify_app (playwright-verify.sh:118-224), same arg contract.
|
|
742
|
+
# - bounded-run / liveness discipline: _loki_test_provenance
|
|
743
|
+
# (completion-council.sh:1587-1600) -- if no timeout binary exists we do NOT
|
|
744
|
+
# boot unbounded; we mark inconclusive and never hang.
|
|
745
|
+
#
|
|
746
|
+
# VERDICT MAPPING (mirrors this module's inconclusive->CONCERNS policy):
|
|
747
|
+
# no start command / library / CLI -> NO gate row emitted (byte-identical to
|
|
748
|
+
# pre-gate output; see BYTE-IDENTITY below).
|
|
749
|
+
# detected + booted + health 2xx/3xx -> pass, reproducible=true, status+artifact.
|
|
750
|
+
# detected + won't start / health 5xx/no-answer -> High finding -> not VERIFIED.
|
|
751
|
+
# detected but boot could not be ATTEMPTED (no timeout binary, no HTTP port
|
|
752
|
+
# mapping) -> inconclusive -> at-least-CONCERNS (never a silent pass).
|
|
753
|
+
#
|
|
754
|
+
# BYTE-IDENTITY (the critical safety property): when no start command is
|
|
755
|
+
# detectable the gate returns WITHOUT calling _verify_add_gate, so evidence.json
|
|
756
|
+
# and report.md are byte-for-byte identical to a tree without this gate. Library
|
|
757
|
+
# repos, pure CLIs, and any repo with no server therefore see zero change. This
|
|
758
|
+
# is why the gate self-suppresses its row instead of emitting a "skipped" row
|
|
759
|
+
# like the other gates: a skipped row would change the bytes.
|
|
760
|
+
#
|
|
761
|
+
# OPT-OUT: LOKI_RUNTIME_GATE=0 disables the gate entirely (consistent with the
|
|
762
|
+
# other opt-out knobs). Default is on. Disabled -> no row -> byte-identical.
|
|
763
|
+
#
|
|
764
|
+
# NO NETWORK beyond localhost; no secrets. The boot inherits the caller's env.
|
|
765
|
+
# ---------------------------------------------------------------------------
|
|
766
|
+
|
|
767
|
+
# Resolve a bounded-exec wrapper (GNU timeout, then gtimeout). Echoes the binary
|
|
768
|
+
# name, or nothing if neither exists. Mirrors the fallback chain used across the
|
|
769
|
+
# codebase (run.sh:8814, completion-council.sh:1595).
|
|
770
|
+
_verify_runtime_timeout_bin() {
|
|
771
|
+
if command -v timeout >/dev/null 2>&1; then
|
|
772
|
+
echo "timeout"
|
|
773
|
+
elif command -v gtimeout >/dev/null 2>&1; then
|
|
774
|
+
echo "gtimeout"
|
|
775
|
+
fi
|
|
776
|
+
}
|
|
777
|
+
|
|
778
|
+
# Detect the app's start command + HTTP port from the tree. Echoes two
|
|
779
|
+
# TAB-separated fields "method<TAB>port" when a startable HTTP app is found, or
|
|
780
|
+
# nothing when none is detectable. Faithful port of app_runner_init's cascade
|
|
781
|
+
# (app-runner.sh:757) MINUS docker (compose/Dockerfile), which is intentionally
|
|
782
|
+
# out of scope for a fast verify boot. Honors LOKI_APP_COMMAND / LOKI_APP_PORT
|
|
783
|
+
# overrides exactly as the build loop does. When rank 9's setup recipe exists at
|
|
784
|
+
# .loki/setup-recipe.json it is consulted first; its absence is handled (it is
|
|
785
|
+
# not built yet), so this is purely opportunistic.
|
|
786
|
+
_verify_runtime_detect() {
|
|
787
|
+
local dir="$1"
|
|
788
|
+
local method="" port=""
|
|
789
|
+
|
|
790
|
+
# 0. Operator override (same env vars the app runner honors).
|
|
791
|
+
if [ -n "${LOKI_APP_COMMAND:-}" ]; then
|
|
792
|
+
method="$LOKI_APP_COMMAND"
|
|
793
|
+
fi
|
|
794
|
+
|
|
795
|
+
# 0b. Rank 9 setup recipe (opportunistic; NOT yet built -- absence is normal).
|
|
796
|
+
if [ -z "$method" ] && [ -f "$dir/.loki/setup-recipe.json" ] && command -v python3 >/dev/null 2>&1; then
|
|
797
|
+
local recipe
|
|
798
|
+
recipe="$(python3 - "$dir/.loki/setup-recipe.json" <<'PYEOF' 2>/dev/null || true
|
|
799
|
+
import json, sys
|
|
800
|
+
try:
|
|
801
|
+
with open(sys.argv[1]) as f:
|
|
802
|
+
d = json.load(f)
|
|
803
|
+
except Exception:
|
|
804
|
+
sys.exit(0)
|
|
805
|
+
if not isinstance(d, dict):
|
|
806
|
+
sys.exit(0)
|
|
807
|
+
cmd = d.get("start") or d.get("start_command") or d.get("run") or ""
|
|
808
|
+
port = d.get("port") or ""
|
|
809
|
+
if isinstance(cmd, str) and cmd.strip():
|
|
810
|
+
print("%s\t%s" % (cmd.strip(), str(port).strip()))
|
|
811
|
+
PYEOF
|
|
812
|
+
)"
|
|
813
|
+
if [ -n "$recipe" ]; then
|
|
814
|
+
method="$(printf '%s' "$recipe" | cut -f1)"
|
|
815
|
+
port="$(printf '%s' "$recipe" | cut -f2)"
|
|
816
|
+
fi
|
|
817
|
+
fi
|
|
818
|
+
|
|
819
|
+
# 1. package.json: dev preferred, then start (app-runner.sh:814-828).
|
|
820
|
+
# HTTP-SIGNAL REQUIRED (mirrors the Python discipline below): a Node
|
|
821
|
+
# "start"/"dev" script is NOT assumed to be an HTTP server. Many legitimate
|
|
822
|
+
# CLIs/libraries ship "start":"node cli.js" or "dev":"tsc --watch". We only
|
|
823
|
+
# treat the project as bootable-HTTP when there is a POSITIVE HTTP signal:
|
|
824
|
+
# an HTTP framework in package.json deps, OR a listen/createServer call in
|
|
825
|
+
# the source. Without a signal we leave method empty -> byte-identity path
|
|
826
|
+
# (no gate row), so a Node CLI never gets a false-RED boot failure.
|
|
827
|
+
if [ -z "$method" ] && [ -f "$dir/package.json" ]; then
|
|
828
|
+
local _node_http_signal=""
|
|
829
|
+
# (a) HTTP framework or server dep declared in package.json.
|
|
830
|
+
if grep -qE '"(express|fastify|koa|hapi|@hapi/hapi|next|nuxt|@nestjs/core|@nestjs/platform-express|http-server|connect|restify|polka|@sveltejs/kit|vite)"[[:space:]]*:' "$dir/package.json" 2>/dev/null; then
|
|
831
|
+
_node_http_signal="dep"
|
|
832
|
+
fi
|
|
833
|
+
# (b) a STRONG server-creation call in a shallow scan of JS/TS sources.
|
|
834
|
+
# "Strong" means a module-qualified server constructor (http/https/
|
|
835
|
+
# http2/net .createServer, or Bun.serve/Deno.serve) -- NOT a bare
|
|
836
|
+
# `.listen(` which is common in tests (`http.createServer().listen(0)`)
|
|
837
|
+
# and unrelated code. Test/example/build dirs are excluded so an
|
|
838
|
+
# incidental server in a test never marks a CLI as bootable-HTTP.
|
|
839
|
+
# Uses grep -q (a boolean, no pipe) so this is safe under
|
|
840
|
+
# set -o pipefail (a piped `| head` would SIGPIPE grep and, under
|
|
841
|
+
# pipefail, drop the result).
|
|
842
|
+
if [ -z "$_node_http_signal" ]; then
|
|
843
|
+
if grep -rqE 'https?\.createServer|http2\.createServer|net\.createServer|Bun\.serve\(|Deno\.serve\(' \
|
|
844
|
+
"$dir" --include='*.js' --include='*.ts' --include='*.mjs' --include='*.cjs' \
|
|
845
|
+
--exclude-dir=node_modules --exclude-dir=.git \
|
|
846
|
+
--exclude-dir=test --exclude-dir=tests --exclude-dir=__tests__ \
|
|
847
|
+
--exclude-dir=examples --exclude-dir=example \
|
|
848
|
+
--exclude-dir=dist --exclude-dir=build --exclude-dir=spec 2>/dev/null; then
|
|
849
|
+
_node_http_signal="src"
|
|
850
|
+
fi
|
|
851
|
+
fi
|
|
852
|
+
if [ -n "$_node_http_signal" ]; then
|
|
853
|
+
if grep -q '"dev"[[:space:]]*:' "$dir/package.json" 2>/dev/null; then
|
|
854
|
+
method="npm run dev"
|
|
855
|
+
elif grep -q '"start"[[:space:]]*:' "$dir/package.json" 2>/dev/null; then
|
|
856
|
+
method="npm start"
|
|
857
|
+
fi
|
|
858
|
+
fi
|
|
859
|
+
fi
|
|
860
|
+
|
|
861
|
+
# 2. Procfile (12-factor web: process). Grep the web line's command.
|
|
862
|
+
if [ -z "$method" ] && [ -f "$dir/Procfile" ]; then
|
|
863
|
+
local proc_cmd
|
|
864
|
+
proc_cmd="$(grep -E '^web:' "$dir/Procfile" 2>/dev/null | head -1 | sed -E 's/^web:[[:space:]]*//')"
|
|
865
|
+
if [ -n "$proc_cmd" ]; then
|
|
866
|
+
method="$proc_cmd"
|
|
867
|
+
fi
|
|
868
|
+
fi
|
|
869
|
+
|
|
870
|
+
# 3. Makefile run/serve target (app-runner.sh:831-847).
|
|
871
|
+
if [ -z "$method" ] && [ -f "$dir/Makefile" ]; then
|
|
872
|
+
if grep -qE '^run:' "$dir/Makefile" 2>/dev/null; then
|
|
873
|
+
method="make run"
|
|
874
|
+
elif grep -qE '^serve:' "$dir/Makefile" 2>/dev/null; then
|
|
875
|
+
method="make serve"
|
|
876
|
+
fi
|
|
877
|
+
fi
|
|
878
|
+
|
|
879
|
+
# 4. Python web entrypoints (app-runner.sh:849-892). Only treated as a
|
|
880
|
+
# startable HTTP app when a web framework import is present -- a bare
|
|
881
|
+
# script (CLI) is NOT booted.
|
|
882
|
+
if [ -z "$method" ] && [ -f "$dir/manage.py" ]; then
|
|
883
|
+
method="python manage.py runserver"
|
|
884
|
+
fi
|
|
885
|
+
if [ -z "$method" ] && [ -f "$dir/app.py" ]; then
|
|
886
|
+
if grep -qE 'from[[:space:]]+fastapi|import[[:space:]]+FastAPI' "$dir/app.py" 2>/dev/null; then
|
|
887
|
+
method="uvicorn app:app --host 127.0.0.1 --port 8000"
|
|
888
|
+
elif grep -qE 'from[[:space:]]+flask|import[[:space:]]+Flask' "$dir/app.py" 2>/dev/null; then
|
|
889
|
+
method="flask run --host 127.0.0.1 --port 5000"
|
|
890
|
+
fi
|
|
891
|
+
fi
|
|
892
|
+
if [ -z "$method" ] && [ -f "$dir/main.py" ]; then
|
|
893
|
+
if grep -qE 'from[[:space:]]+fastapi|import[[:space:]]+FastAPI' "$dir/main.py" 2>/dev/null; then
|
|
894
|
+
method="uvicorn main:app --host 127.0.0.1 --port 8000"
|
|
895
|
+
fi
|
|
896
|
+
fi
|
|
897
|
+
|
|
898
|
+
# No startable HTTP app detected -> echo nothing (byte-identity path).
|
|
899
|
+
[ -z "$method" ] && return 0
|
|
900
|
+
|
|
901
|
+
# Resolve the port when detection did not already set one. Mirrors the
|
|
902
|
+
# _detect_port default map (app-runner.sh:641-731). LOKI_APP_PORT wins.
|
|
903
|
+
if [ -n "${LOKI_APP_PORT:-}" ]; then
|
|
904
|
+
port="$LOKI_APP_PORT"
|
|
905
|
+
elif [ -z "$port" ]; then
|
|
906
|
+
case "$method" in
|
|
907
|
+
*manage.py*) port=8000 ;;
|
|
908
|
+
*flask*) port=5000 ;;
|
|
909
|
+
*uvicorn*|*fastapi*|*main.py*) port=8000 ;;
|
|
910
|
+
*npm*|*next*|*node*) port=3000 ;;
|
|
911
|
+
*) port=8080 ;;
|
|
912
|
+
esac
|
|
913
|
+
fi
|
|
914
|
+
|
|
915
|
+
printf '%s\t%s\n' "$method" "$port"
|
|
916
|
+
return 0
|
|
917
|
+
}
|
|
918
|
+
|
|
919
|
+
verify_gate_runtime() {
|
|
920
|
+
local tree="$1"
|
|
921
|
+
# Opt-out (default on). Disabled -> emit no row -> byte-identical.
|
|
922
|
+
[ "${LOKI_RUNTIME_GATE:-1}" = "0" ] && return 0
|
|
923
|
+
|
|
924
|
+
local detected
|
|
925
|
+
detected="$(_verify_runtime_detect "$tree")"
|
|
926
|
+
# No startable HTTP app -> byte-identity path: no gate row, no findings.
|
|
927
|
+
[ -z "$detected" ] && return 0
|
|
928
|
+
|
|
929
|
+
local method port
|
|
930
|
+
method="$(printf '%s' "$detected" | cut -f1)"
|
|
931
|
+
port="$(printf '%s' "$detected" | cut -f2)"
|
|
932
|
+
|
|
933
|
+
# Bounded-boot discipline (liveness): without a timeout binary we do NOT boot
|
|
934
|
+
# unbounded. Mark inconclusive -> CONCERNS, never hang (mirror
|
|
935
|
+
# _loki_test_provenance's timeout-absent branch, completion-council.sh:1597).
|
|
936
|
+
local timeout_bin
|
|
937
|
+
timeout_bin="$(_verify_runtime_timeout_bin)"
|
|
938
|
+
if [ -z "$timeout_bin" ]; then
|
|
939
|
+
_verify_add_gate "runtime" "inconclusive" "boot" \
|
|
940
|
+
"start command detected ('$method') but no timeout binary (install coreutils); boot not attempted" "true"
|
|
941
|
+
return 0
|
|
942
|
+
fi
|
|
943
|
+
|
|
944
|
+
local boot_timeout="${LOKI_RUNTIME_BOOT_TIMEOUT:-45}"
|
|
945
|
+
local health_path="${LOKI_RUNTIME_HEALTH_PATH:-/}"
|
|
946
|
+
local url="http://127.0.0.1:${port}${health_path}"
|
|
947
|
+
# Artifacts land under the resolved --out dir (VERIFY_OUT_DIR set by
|
|
948
|
+
# verify_main); default to .loki/verify when the gate is exercised directly.
|
|
949
|
+
local out_dir="${VERIFY_OUT_DIR:-.loki/verify}"
|
|
950
|
+
local artifact_dir="$out_dir/runtime"
|
|
951
|
+
mkdir -p "$artifact_dir" 2>/dev/null || true
|
|
952
|
+
local boot_log="$artifact_dir/boot.log"
|
|
953
|
+
|
|
954
|
+
# Boot the app in the background, bounded by the timeout wrapper. The whole
|
|
955
|
+
# process group is killed on teardown. Env is inherited (no secrets injected).
|
|
956
|
+
# PORT is exported to the resolved port so 12-factor apps (node, many python
|
|
957
|
+
# frameworks) listen where the probe looks; apps that hardcode a port ignore
|
|
958
|
+
# it harmlessly. This keeps boot-port and probe-port consistent.
|
|
959
|
+
local app_pid=""
|
|
960
|
+
(
|
|
961
|
+
cd "$tree" || exit 127
|
|
962
|
+
export PORT="$port"
|
|
963
|
+
exec "$timeout_bin" "$boot_timeout" sh -c "$method"
|
|
964
|
+
) >"$boot_log" 2>&1 &
|
|
965
|
+
app_pid=$!
|
|
966
|
+
|
|
967
|
+
# Poll the health endpoint until it answers or the boot budget elapses.
|
|
968
|
+
# Prefer curl; fall back to a bash /dev/tcp connect + minimal GET when curl
|
|
969
|
+
# is absent (status then unknown -> treated as "answered" only on a real
|
|
970
|
+
# HTTP status line). Never blocks past boot_timeout.
|
|
971
|
+
local http_status="" answered="false"
|
|
972
|
+
local deadline=$(( $(date +%s) + boot_timeout ))
|
|
973
|
+
local have_curl="false"
|
|
974
|
+
command -v curl >/dev/null 2>&1 && have_curl="true"
|
|
975
|
+
# The port we actually probe. Starts at the guessed default; if the boot log
|
|
976
|
+
# announces a different bound port (e.g. Vite on 5173, which ignores PORT and
|
|
977
|
+
# the default map cannot know), we re-point the probe THERE. scraped_port is
|
|
978
|
+
# stashed so teardown can also reclaim it (fixes the different-port leak).
|
|
979
|
+
local probe_port="$port" scraped_port=""
|
|
980
|
+
|
|
981
|
+
while [ "$(date +%s)" -lt "$deadline" ]; do
|
|
982
|
+
# If the app process already exited, stop polling (it failed to stay up).
|
|
983
|
+
if ! kill -0 "$app_pid" 2>/dev/null; then
|
|
984
|
+
# Give one last probe in case it forked a daemon and exited.
|
|
985
|
+
:
|
|
986
|
+
fi
|
|
987
|
+
# Scrape the actually-bound port from the boot log (progressively filled).
|
|
988
|
+
# Only re-point when it differs from the current probe target.
|
|
989
|
+
if [ -z "$scraped_port" ]; then
|
|
990
|
+
scraped_port="$(_verify_runtime_scrape_port "$boot_log" 2>/dev/null || true)"
|
|
991
|
+
if [ -n "$scraped_port" ] && [ "$scraped_port" != "$probe_port" ]; then
|
|
992
|
+
probe_port="$scraped_port"
|
|
993
|
+
url="http://127.0.0.1:${probe_port}${health_path}"
|
|
994
|
+
fi
|
|
995
|
+
fi
|
|
996
|
+
if [ "$have_curl" = "true" ]; then
|
|
997
|
+
http_status="$(curl -s -o /dev/null -w '%{http_code}' --max-time 3 "$url" 2>/dev/null || echo "")"
|
|
998
|
+
if [ -n "$http_status" ] && [ "$http_status" != "000" ]; then
|
|
999
|
+
answered="true"
|
|
1000
|
+
break
|
|
1001
|
+
fi
|
|
1002
|
+
else
|
|
1003
|
+
# Portable fallback: raw TCP GET via bash /dev/tcp (best effort).
|
|
1004
|
+
local resp=""
|
|
1005
|
+
resp="$(_verify_runtime_raw_probe "$probe_port" "$health_path" 2>/dev/null || true)"
|
|
1006
|
+
if [ -n "$resp" ]; then
|
|
1007
|
+
http_status="$resp"
|
|
1008
|
+
answered="true"
|
|
1009
|
+
break
|
|
1010
|
+
fi
|
|
1011
|
+
fi
|
|
1012
|
+
# Stop early if the launcher died AND nothing is listening yet.
|
|
1013
|
+
if ! kill -0 "$app_pid" 2>/dev/null; then
|
|
1014
|
+
break
|
|
1015
|
+
fi
|
|
1016
|
+
sleep 1
|
|
1017
|
+
done
|
|
1018
|
+
|
|
1019
|
+
# Optional screenshot artifact (only when playwright + node are available and
|
|
1020
|
+
# the app answered). Reuses the inline chromium smoke script contract from
|
|
1021
|
+
# playwright_verify_app (playwright-verify.sh). Best effort: absence or
|
|
1022
|
+
# failure never changes the gate outcome.
|
|
1023
|
+
local artifact="" artifact_field=""
|
|
1024
|
+
if [ "$answered" = "true" ] && command -v node >/dev/null 2>&1 \
|
|
1025
|
+
&& npx playwright --version >/dev/null 2>&1; then
|
|
1026
|
+
artifact="$artifact_dir/boot-$(date -u +%Y%m%dT%H%M%SZ).png"
|
|
1027
|
+
_verify_runtime_screenshot "$url" "$artifact" "$timeout_bin" || artifact=""
|
|
1028
|
+
[ -f "$artifact" ] && artifact_field="$artifact"
|
|
1029
|
+
fi
|
|
1030
|
+
|
|
1031
|
+
# Teardown: kill the launcher and any child it spawned. Best effort; bounded.
|
|
1032
|
+
# Reclaim BOTH the detected port and the actually-bound port (scraped from the
|
|
1033
|
+
# boot log) so a server that daemonized onto a different port than we guessed
|
|
1034
|
+
# does not leak. scraped_port may be empty (no banner) -- teardown handles it.
|
|
1035
|
+
_verify_runtime_teardown "$app_pid" "$port" "$scraped_port"
|
|
1036
|
+
|
|
1037
|
+
# Interpret the result.
|
|
1038
|
+
if [ "$answered" != "true" ]; then
|
|
1039
|
+
_verify_add_gate "runtime" "fail" "boot" \
|
|
1040
|
+
"app detected ('$method') but did not answer $url within ${boot_timeout}s (see $boot_log)" "true"
|
|
1041
|
+
_verify_add_finding "High" "runtime" "deterministic:runtime-boot" "" "null" \
|
|
1042
|
+
"Runtime boot smoke failed: '$method' did not serve $url within ${boot_timeout}s. $(tail -2 "$boot_log" 2>/dev/null | tr '\n' ' ')"
|
|
1043
|
+
return 0
|
|
1044
|
+
fi
|
|
1045
|
+
|
|
1046
|
+
# Answered. 5xx is a boot-level failure; 2xx/3xx/4xx means the server is up
|
|
1047
|
+
# and routing (4xx on '/' is common for APIs with no root route -- the server
|
|
1048
|
+
# IS running, so that is a PASS: the gate attests "it runs", not "route X
|
|
1049
|
+
# exists"). Only 5xx (server error) fails.
|
|
1050
|
+
local summary="booted; GET $health_path -> HTTP $http_status"
|
|
1051
|
+
[ -n "$artifact_field" ] && summary="$summary; screenshot $artifact_field"
|
|
1052
|
+
if [ "$http_status" -ge 500 ] 2>/dev/null; then
|
|
1053
|
+
_verify_add_gate "runtime" "fail" "boot" "$summary (server error)" "true"
|
|
1054
|
+
_verify_add_finding "High" "runtime" "deterministic:runtime-boot" "" "null" \
|
|
1055
|
+
"Runtime boot smoke failed: '$method' booted but returned HTTP $http_status on $url (server error)."
|
|
1056
|
+
else
|
|
1057
|
+
_verify_add_gate "runtime" "pass" "boot" "$summary" "true"
|
|
1058
|
+
fi
|
|
1059
|
+
|
|
1060
|
+
# Record a structured runtime artifact alongside the gate row so the boot is
|
|
1061
|
+
# reproducible (command + url + status + artifact are all captured).
|
|
1062
|
+
_VR_DIR="$artifact_dir" _VR_METHOD="$method" _VR_URL="$url" \
|
|
1063
|
+
_VR_STATUS="$http_status" _VR_ART="$artifact_field" _VR_TO="$boot_timeout" \
|
|
1064
|
+
python3 - <<'PYEOF' 2>/dev/null || true
|
|
1065
|
+
import json, os
|
|
1066
|
+
rec = {
|
|
1067
|
+
"start_command": os.environ.get("_VR_METHOD", ""),
|
|
1068
|
+
"url": os.environ.get("_VR_URL", ""),
|
|
1069
|
+
"http_status": os.environ.get("_VR_STATUS", ""),
|
|
1070
|
+
"screenshot": os.environ.get("_VR_ART") or None,
|
|
1071
|
+
"boot_timeout_s": int(os.environ.get("_VR_TO", "0") or 0),
|
|
1072
|
+
"reproducible": True,
|
|
1073
|
+
}
|
|
1074
|
+
d = os.environ["_VR_DIR"]
|
|
1075
|
+
os.makedirs(d, exist_ok=True)
|
|
1076
|
+
with open(os.path.join(d, "runtime.json"), "w") as f:
|
|
1077
|
+
json.dump(rec, f, indent=2)
|
|
1078
|
+
f.write("\n")
|
|
1079
|
+
PYEOF
|
|
1080
|
+
return 0
|
|
1081
|
+
}
|
|
1082
|
+
|
|
1083
|
+
# Raw TCP probe fallback (no curl). Sends a minimal HTTP/1.0 GET via bash
|
|
1084
|
+
# /dev/tcp and echoes the numeric status code on success, nothing on failure.
|
|
1085
|
+
# Best effort; used only when curl is absent.
|
|
1086
|
+
_verify_runtime_raw_probe() {
|
|
1087
|
+
local port="$1" path="$2"
|
|
1088
|
+
# /dev/tcp is a bash builtin; guard against sh-only environments.
|
|
1089
|
+
( exec 3<>"/dev/tcp/127.0.0.1/${port}" 2>/dev/null ) || return 1
|
|
1090
|
+
local line
|
|
1091
|
+
{
|
|
1092
|
+
exec 3<>"/dev/tcp/127.0.0.1/${port}" || return 1
|
|
1093
|
+
printf 'GET %s HTTP/1.0\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n' "$path" >&3
|
|
1094
|
+
IFS= read -r line <&3
|
|
1095
|
+
exec 3>&- 3<&-
|
|
1096
|
+
} 2>/dev/null
|
|
1097
|
+
# line looks like: HTTP/1.0 200 OK
|
|
1098
|
+
printf '%s' "$line" | grep -oE 'HTTP/[0-9.]+ [0-9]{3}' | grep -oE '[0-9]{3}$' || return 1
|
|
1099
|
+
}
|
|
1100
|
+
|
|
1101
|
+
# Scrape the actually-bound localhost port from a dev server's boot log. Dev
|
|
1102
|
+
# servers (Vite, SvelteKit, Nuxt, Next, CRA, ...) print a "Local:" / "listening
|
|
1103
|
+
# on" banner with their real port, which is frequently NOT the guessed default
|
|
1104
|
+
# (bare Vite binds 5173 and ignores PORT). We parse ONLY a localhost/127.0.0.1
|
|
1105
|
+
# bind announcement so we never lock onto an unrelated outbound URL the app might
|
|
1106
|
+
# have logged (which some other live process could answer -> false green). ANSI
|
|
1107
|
+
# color codes are stripped first (dev banners are colorized). Echoes the first
|
|
1108
|
+
# matched port, or nothing. Bounded: reads only the given log file, no network.
|
|
1109
|
+
_verify_runtime_scrape_port() {
|
|
1110
|
+
local log="$1"
|
|
1111
|
+
[ -f "$log" ] || return 1
|
|
1112
|
+
# Strip ANSI escapes first (dev banners are colorized). Then, line by line,
|
|
1113
|
+
# match a localhost/127.0.0.1 bind announcement and extract the PORT that
|
|
1114
|
+
# sits at the end of the match (the number after the final colon / after
|
|
1115
|
+
# "port"), never an IP octet. Emit the first port found.
|
|
1116
|
+
local line p
|
|
1117
|
+
while IFS= read -r line; do
|
|
1118
|
+
# A URL like http://localhost:5173 or http://127.0.0.1:5173/ .
|
|
1119
|
+
p="$(printf '%s' "$line" | grep -oiE 'https?://(localhost|127\.0\.0\.1):[0-9]{2,5}' | grep -oE ':[0-9]{2,5}' | grep -oE '[0-9]{2,5}' | head -1)"
|
|
1120
|
+
# Or a "listening on [127.0.0.1:]5173" / "listening on port 5173" phrase.
|
|
1121
|
+
if [ -z "$p" ]; then
|
|
1122
|
+
p="$(printf '%s' "$line" | grep -oiE 'listening on( port)?[[:space:]:]*([0-9.]+:)?[0-9]{2,5}' | grep -oE '[0-9]{2,5}$' | head -1)"
|
|
1123
|
+
fi
|
|
1124
|
+
if [ -n "$p" ]; then
|
|
1125
|
+
printf '%s' "$p"
|
|
1126
|
+
return 0
|
|
1127
|
+
fi
|
|
1128
|
+
done < <(sed -E $'s/\033\\[[0-9;?]*[A-Za-z]//g' "$log" 2>/dev/null)
|
|
1129
|
+
return 1
|
|
1130
|
+
}
|
|
1131
|
+
|
|
1132
|
+
# Optional screenshot via the playwright inline smoke script (reused contract:
|
|
1133
|
+
# url screenshot results timeout). Bounded by the same timeout wrapper. Returns
|
|
1134
|
+
# 0 if the screenshot file was produced, nonzero otherwise. Never fatal.
|
|
1135
|
+
_verify_runtime_screenshot() {
|
|
1136
|
+
local url="$1" out_png="$2" timeout_bin="$3"
|
|
1137
|
+
local tmp_js results_json
|
|
1138
|
+
tmp_js="$(mktemp -t loki-verify-smoke.XXXXXX.js 2>/dev/null)" || return 1
|
|
1139
|
+
results_json="$(mktemp -t loki-verify-smoke.XXXXXX.json 2>/dev/null)" || { rm -f "$tmp_js"; return 1; }
|
|
1140
|
+
cat >"$tmp_js" <<'SMOKE_SCRIPT'
|
|
1141
|
+
const { chromium } = require('playwright');
|
|
1142
|
+
(async () => {
|
|
1143
|
+
const url = process.argv[2];
|
|
1144
|
+
const screenshotPath = process.argv[3];
|
|
1145
|
+
const pageTimeout = parseInt(process.argv[4] || '15000', 10);
|
|
1146
|
+
let browser;
|
|
1147
|
+
try {
|
|
1148
|
+
browser = await chromium.launch({ headless: true });
|
|
1149
|
+
const page = await browser.newPage();
|
|
1150
|
+
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: pageTimeout });
|
|
1151
|
+
await page.screenshot({ path: screenshotPath, fullPage: false });
|
|
1152
|
+
} catch (err) {
|
|
1153
|
+
process.exit(1);
|
|
1154
|
+
} finally {
|
|
1155
|
+
if (browser) await browser.close();
|
|
1156
|
+
}
|
|
1157
|
+
process.exit(0);
|
|
1158
|
+
})();
|
|
1159
|
+
SMOKE_SCRIPT
|
|
1160
|
+
"$timeout_bin" 30 node "$tmp_js" "$url" "$out_png" 15000 >/dev/null 2>&1
|
|
1161
|
+
local rc=$?
|
|
1162
|
+
rm -f "$tmp_js" "$results_json"
|
|
1163
|
+
[ "$rc" -eq 0 ] && [ -f "$out_png" ]
|
|
1164
|
+
}
|
|
1165
|
+
|
|
1166
|
+
# Teardown: terminate the boot launcher and any process still holding the port.
|
|
1167
|
+
# Bounded and best effort; never blocks the gate.
|
|
1168
|
+
_verify_runtime_teardown() {
|
|
1169
|
+
local app_pid="$1" port="$2" scraped_port="${3:-}"
|
|
1170
|
+
if [ -n "$app_pid" ]; then
|
|
1171
|
+
# Best: kill the launcher's whole process GROUP so a server it spawned in
|
|
1172
|
+
# a subshell (e.g. `vite &`) dies too. GUARDED against self-suicide: in a
|
|
1173
|
+
# non-interactive script job control is off, so a background job can share
|
|
1174
|
+
# the verifier's OWN process group; a negative-PID kill there would kill
|
|
1175
|
+
# us (and our parent). Only group-kill when the child's PGID differs from
|
|
1176
|
+
# ours AND equals the child PID (i.e. it is a real group leader).
|
|
1177
|
+
local child_pgid="" self_pgid=""
|
|
1178
|
+
child_pgid="$(ps -o pgid= -p "$app_pid" 2>/dev/null | tr -d ' ' || true)"
|
|
1179
|
+
self_pgid="$(ps -o pgid= -p $$ 2>/dev/null | tr -d ' ' || true)"
|
|
1180
|
+
if [ -n "$child_pgid" ] && [ "$child_pgid" != "$self_pgid" ] \
|
|
1181
|
+
&& [ "$child_pgid" = "$app_pid" ]; then
|
|
1182
|
+
kill -- -"$child_pgid" 2>/dev/null || true
|
|
1183
|
+
fi
|
|
1184
|
+
# Always TERM the launcher PID itself.
|
|
1185
|
+
kill "$app_pid" 2>/dev/null || true
|
|
1186
|
+
# Kill any direct children (the timeout wrapper spawns sh -c "$method").
|
|
1187
|
+
if command -v pkill >/dev/null 2>&1; then
|
|
1188
|
+
pkill -P "$app_pid" 2>/dev/null || true
|
|
1189
|
+
fi
|
|
1190
|
+
# Give it a moment, then hard-kill (PID, and group again when safe).
|
|
1191
|
+
local i=0
|
|
1192
|
+
while [ "$i" -lt 3 ] && kill -0 "$app_pid" 2>/dev/null; do
|
|
1193
|
+
sleep 1; i=$((i + 1))
|
|
1194
|
+
done
|
|
1195
|
+
if [ -n "$child_pgid" ] && [ "$child_pgid" != "$self_pgid" ] \
|
|
1196
|
+
&& [ "$child_pgid" = "$app_pid" ]; then
|
|
1197
|
+
kill -9 -- -"$child_pgid" 2>/dev/null || true
|
|
1198
|
+
fi
|
|
1199
|
+
kill -9 "$app_pid" 2>/dev/null || true
|
|
1200
|
+
fi
|
|
1201
|
+
# Reclaim BOTH the detected port and the actually-bound (scraped) port from
|
|
1202
|
+
# any orphan that outlived the launcher -- a daemonized server that bound a
|
|
1203
|
+
# different port than we guessed would otherwise leak. Bounded, best effort.
|
|
1204
|
+
if command -v lsof >/dev/null 2>&1; then
|
|
1205
|
+
# Build a unique, non-empty port list (scraped only if it differs).
|
|
1206
|
+
local _ports="$port"
|
|
1207
|
+
[ -n "$scraped_port" ] && [ "$scraped_port" != "$port" ] && _ports="$_ports $scraped_port"
|
|
1208
|
+
local _rp
|
|
1209
|
+
for _rp in $_ports; do
|
|
1210
|
+
[ -z "$_rp" ] && continue
|
|
1211
|
+
local holders
|
|
1212
|
+
holders="$(lsof -ti tcp:"$_rp" 2>/dev/null || true)"
|
|
1213
|
+
if [ -n "$holders" ]; then
|
|
1214
|
+
printf '%s\n' "$holders" | while IFS= read -r pid; do
|
|
1215
|
+
[ -n "$pid" ] && kill -9 "$pid" 2>/dev/null || true
|
|
1216
|
+
done
|
|
1217
|
+
fi
|
|
1218
|
+
done
|
|
1219
|
+
fi
|
|
1220
|
+
}
|
|
1221
|
+
|
|
726
1222
|
# ---------------------------------------------------------------------------
|
|
727
1223
|
# Verdict computation (Entanglement 2: inconclusive -> CONCERNS, never VERIFIED).
|
|
728
1224
|
#
|
|
@@ -841,6 +1337,8 @@ verify_emit_evidence() {
|
|
|
841
1337
|
_V_START="$started_at" \
|
|
842
1338
|
_V_DONE="$completed_at" \
|
|
843
1339
|
_V_BLOCKON="$block_on" \
|
|
1340
|
+
_V_LEDGER_SHA="${_VERIFY_EXPECT_LEDGER_SHA:-}" \
|
|
1341
|
+
_V_LEDGER_HASHOK="${_VERIFY_EXPECT_LEDGER_HASH_OK:-}" \
|
|
844
1342
|
python3 - <<'PYEOF'
|
|
845
1343
|
import json, os, hashlib
|
|
846
1344
|
|
|
@@ -922,6 +1420,18 @@ doc = {
|
|
|
922
1420
|
"suppressed": [],
|
|
923
1421
|
}
|
|
924
1422
|
|
|
1423
|
+
# Annotate-before-act ledger hash embed (additive; ONLY when a ledger was
|
|
1424
|
+
# present this run). Embedding the ledger_sha256 into evidence.json makes an
|
|
1425
|
+
# expectation edited after the fact detectable from the evidence document, the
|
|
1426
|
+
# same tamper-evidence proof.json carries. When no ledger existed the env var is
|
|
1427
|
+
# empty and this key is omitted, so evidence.json stays byte-identical to today.
|
|
1428
|
+
_ledger_sha = os.environ.get("_V_LEDGER_SHA") or ""
|
|
1429
|
+
if _ledger_sha:
|
|
1430
|
+
doc["expectation_ledger"] = {
|
|
1431
|
+
"ledger_sha256": _ledger_sha,
|
|
1432
|
+
"hash_ok": (os.environ.get("_V_LEDGER_HASHOK") == "true"),
|
|
1433
|
+
}
|
|
1434
|
+
|
|
925
1435
|
os.makedirs(out_dir, exist_ok=True)
|
|
926
1436
|
ev_path = os.path.join(out_dir, "evidence.json")
|
|
927
1437
|
with open(ev_path, "w") as f:
|
|
@@ -1116,6 +1626,200 @@ verify_spec_drift_gate() {
|
|
|
1116
1626
|
return 0
|
|
1117
1627
|
}
|
|
1118
1628
|
|
|
1629
|
+
# ---------------------------------------------------------------------------
|
|
1630
|
+
# Annotate-before-act expectation-ledger gate (autonomy/lib/expectation-ledger.py).
|
|
1631
|
+
#
|
|
1632
|
+
# When a pre-committed expectation ledger exists at .loki/expectations/<iter>.json
|
|
1633
|
+
# (written BEFORE the act/verify), diff its predicted outcomes against the actual
|
|
1634
|
+
# observations recorded at .loki/expectations/<iter>.observed.json. Any predicted
|
|
1635
|
+
# check that was silently DROPPED (never executed) or CONTRADICTED (actual !=
|
|
1636
|
+
# expected) becomes a finding; an UNEVALUABLE prediction -> inconclusive ->
|
|
1637
|
+
# CONCERNS (never VERIFIED). A ledger whose recorded hash no longer matches its
|
|
1638
|
+
# own entries (edited after the fact) is a tamper finding.
|
|
1639
|
+
#
|
|
1640
|
+
# ADDITIVE + fail-open. Nothing writes a ledger yet in the common case, so:
|
|
1641
|
+
# - No ledger file present -> this returns immediately, adds NO gate row and
|
|
1642
|
+
# NO finding, so evidence.json is BYTE-IDENTICAL to today.
|
|
1643
|
+
# - LOKI_EXPECTATION_LEDGER=0 -> disabled entirely (same byte-identical result).
|
|
1644
|
+
# - python3 missing, module missing, or any error -> a single inconclusive
|
|
1645
|
+
# gate row at most, never an abort, never a silent pass.
|
|
1646
|
+
#
|
|
1647
|
+
# The embedded ledger_sha256 is folded into evidence.json (verify_emit_evidence
|
|
1648
|
+
# reads _VERIFY_EXPECT_LEDGER_SHA / _VERIFY_EXPECT_LEDGER_HASH_OK) so an
|
|
1649
|
+
# expectation edited after write is detectable from the evidence document too.
|
|
1650
|
+
# ---------------------------------------------------------------------------
|
|
1651
|
+
verify_expectation_ledger_gate() {
|
|
1652
|
+
local tree="${1:-.}"
|
|
1653
|
+
|
|
1654
|
+
# Opt-out knob, consistent with other verify env knobs. Default ON, but the
|
|
1655
|
+
# gate is a total no-op unless a ledger file actually exists, so default-ON is
|
|
1656
|
+
# byte-identical to today for every run that has no ledger.
|
|
1657
|
+
if [ "${LOKI_EXPECTATION_LEDGER:-1}" = "0" ]; then
|
|
1658
|
+
return 0
|
|
1659
|
+
fi
|
|
1660
|
+
|
|
1661
|
+
local expect_dir="$tree/.loki/expectations"
|
|
1662
|
+
expect_dir="${expect_dir#./}"
|
|
1663
|
+
# No ledger directory at all -> byte-identical no-op (no gate row emitted).
|
|
1664
|
+
[ -d "$expect_dir" ] || return 0
|
|
1665
|
+
|
|
1666
|
+
# Pick the most recent ledger (highest-numbered / newest .json that is not an
|
|
1667
|
+
# .observed.json sidecar). No ledger file -> byte-identical no-op.
|
|
1668
|
+
local ledger_file=""
|
|
1669
|
+
local _f
|
|
1670
|
+
for _f in "$expect_dir"/*.json; do
|
|
1671
|
+
[ -f "$_f" ] || continue
|
|
1672
|
+
case "$_f" in
|
|
1673
|
+
*.observed.json) continue ;;
|
|
1674
|
+
esac
|
|
1675
|
+
if [ -z "$ledger_file" ] || [ "$_f" -nt "$ledger_file" ]; then
|
|
1676
|
+
ledger_file="$_f"
|
|
1677
|
+
fi
|
|
1678
|
+
done
|
|
1679
|
+
[ -n "$ledger_file" ] || return 0
|
|
1680
|
+
|
|
1681
|
+
# Derive the iteration token from the ledger filename (<iter>.json).
|
|
1682
|
+
local iter
|
|
1683
|
+
iter="$(basename "$ledger_file" .json)"
|
|
1684
|
+
local observed_file="$expect_dir/$iter.observed.json"
|
|
1685
|
+
[ -f "$observed_file" ] || observed_file=""
|
|
1686
|
+
|
|
1687
|
+
local mod
|
|
1688
|
+
mod="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/expectation-ledger.py"
|
|
1689
|
+
if [ ! -f "$mod" ]; then
|
|
1690
|
+
_verify_add_gate "expectation_ledger" "inconclusive" "loki-ledger" "ledger present but expectation-ledger module not found" "true"
|
|
1691
|
+
return 0
|
|
1692
|
+
fi
|
|
1693
|
+
if ! command -v python3 >/dev/null 2>&1; then
|
|
1694
|
+
_verify_add_gate "expectation_ledger" "inconclusive" "loki-ledger" "ledger present but python3 not on PATH" "true"
|
|
1695
|
+
return 0
|
|
1696
|
+
fi
|
|
1697
|
+
|
|
1698
|
+
# Compare via the module. It prints TAB-separated finding lines (verify TSV
|
|
1699
|
+
# shape) on stdout for contradicted/dropped/tamper, a GATE:<status>:<summary>
|
|
1700
|
+
# control line for the gate row, and SHA:/HASHOK: control lines for the
|
|
1701
|
+
# evidence embed. Any failure degrades to a single inconclusive gate.
|
|
1702
|
+
local compare_out
|
|
1703
|
+
compare_out="$(
|
|
1704
|
+
_LK_LOKI_DIR="$expect_dir/.." \
|
|
1705
|
+
_LK_ITER="$iter" \
|
|
1706
|
+
_LK_OBSERVED="$observed_file" \
|
|
1707
|
+
_LK_MOD="$mod" \
|
|
1708
|
+
python3 - <<'PYEOF' 2>/dev/null
|
|
1709
|
+
import importlib.util, json, os, sys
|
|
1710
|
+
|
|
1711
|
+
mod_path = os.environ["_LK_MOD"]
|
|
1712
|
+
loki_dir = os.environ["_LK_LOKI_DIR"]
|
|
1713
|
+
iteration = os.environ["_LK_ITER"]
|
|
1714
|
+
observed_path = os.environ.get("_LK_OBSERVED") or ""
|
|
1715
|
+
|
|
1716
|
+
spec = importlib.util.spec_from_file_location("expectation_ledger", mod_path)
|
|
1717
|
+
mod = importlib.util.module_from_spec(spec)
|
|
1718
|
+
spec.loader.exec_module(mod)
|
|
1719
|
+
|
|
1720
|
+
observed = {}
|
|
1721
|
+
if observed_path and os.path.isfile(observed_path):
|
|
1722
|
+
try:
|
|
1723
|
+
with open(observed_path) as f:
|
|
1724
|
+
observed = json.load(f)
|
|
1725
|
+
except Exception:
|
|
1726
|
+
observed = {}
|
|
1727
|
+
if not isinstance(observed, dict):
|
|
1728
|
+
observed = {}
|
|
1729
|
+
|
|
1730
|
+
res = mod.compare(loki_dir, iteration, observed)
|
|
1731
|
+
|
|
1732
|
+
def emit_finding(sev, source, fpath, msg):
|
|
1733
|
+
# verify TSV: severity \t category \t source \t file \t line \t message
|
|
1734
|
+
line = str(msg).replace("\t", " ").replace("\n", " ")
|
|
1735
|
+
print("\t".join([sev, "expectation_ledger", source, fpath or "", "null", line]))
|
|
1736
|
+
|
|
1737
|
+
# Tamper check: a broken ledger hash means an expectation was edited after it
|
|
1738
|
+
# was sealed. That is a High finding regardless of the compare outcome.
|
|
1739
|
+
if res.get("ledger_hash_ok") is False:
|
|
1740
|
+
emit_finding("High", "deterministic:ledger-tamper",
|
|
1741
|
+
".loki/expectations/%s.json" % iteration,
|
|
1742
|
+
"expectation ledger hash mismatch (ledger edited after it was sealed)")
|
|
1743
|
+
|
|
1744
|
+
for row in res.get("results", []):
|
|
1745
|
+
outcome = row.get("outcome")
|
|
1746
|
+
stmt = row.get("statement") or row.get("id")
|
|
1747
|
+
if outcome == "contradicted":
|
|
1748
|
+
emit_finding("High", "deterministic:ledger-contradicted", "",
|
|
1749
|
+
"predicted outcome contradicted: %s (expected=%r actual=%r)"
|
|
1750
|
+
% (stmt, row.get("expected"), row.get("actual")))
|
|
1751
|
+
elif outcome == "dropped":
|
|
1752
|
+
# Medium (CONCERNS, not BLOCKED): a predicted-but-unobserved check is a
|
|
1753
|
+
# gap, but until observed-capture is wired into the run loop (follow-up)
|
|
1754
|
+
# every un-observed expectation would land here -- blocking on it would
|
|
1755
|
+
# be a footgun. Record-and-surface, do not hard-block. A contradicted
|
|
1756
|
+
# prediction (actively disproven) stays High above.
|
|
1757
|
+
emit_finding("Medium", "deterministic:ledger-dropped", "",
|
|
1758
|
+
"predicted check never executed (silently dropped): %s" % stmt)
|
|
1759
|
+
|
|
1760
|
+
# Gate control line. An inconclusive prediction, OR a tamper, forces
|
|
1761
|
+
# at-least-CONCERNS via an inconclusive gate. Contradicted/dropped already emit
|
|
1762
|
+
# findings above (which drive BLOCKED/CONCERNS through the finding severity).
|
|
1763
|
+
n_c = res.get("contradicted", 0)
|
|
1764
|
+
n_d = res.get("dropped", 0)
|
|
1765
|
+
n_i = res.get("inconclusive", 0)
|
|
1766
|
+
n_m = res.get("met", 0)
|
|
1767
|
+
tamper = res.get("ledger_hash_ok") is False
|
|
1768
|
+
|
|
1769
|
+
if tamper:
|
|
1770
|
+
gate_status = "fail"
|
|
1771
|
+
summary = "ledger tampered (hash mismatch); %d met / %d contradicted / %d dropped / %d inconclusive" % (n_m, n_c, n_d, n_i)
|
|
1772
|
+
elif n_c or n_d:
|
|
1773
|
+
gate_status = "fail"
|
|
1774
|
+
summary = "%d met / %d contradicted / %d dropped / %d inconclusive" % (n_m, n_c, n_d, n_i)
|
|
1775
|
+
elif n_i:
|
|
1776
|
+
# Unevaluable prediction -> inconclusive -> CONCERNS (never VERIFIED).
|
|
1777
|
+
gate_status = "inconclusive"
|
|
1778
|
+
summary = "%d met / %d inconclusive (unevaluable predictions)" % (n_m, n_i)
|
|
1779
|
+
else:
|
|
1780
|
+
gate_status = "pass"
|
|
1781
|
+
summary = "all %d predicted outcomes met" % n_m
|
|
1782
|
+
|
|
1783
|
+
print("GATE:%s:%s" % (gate_status, summary))
|
|
1784
|
+
print("SHA:%s" % (res.get("ledger_sha256") or ""))
|
|
1785
|
+
print("HASHOK:%s" % ("true" if res.get("ledger_hash_ok") else "false"))
|
|
1786
|
+
PYEOF
|
|
1787
|
+
)"
|
|
1788
|
+
|
|
1789
|
+
if [ -z "$compare_out" ]; then
|
|
1790
|
+
_verify_add_gate "expectation_ledger" "inconclusive" "loki-ledger" "ledger present but comparison could not run" "true"
|
|
1791
|
+
return 0
|
|
1792
|
+
fi
|
|
1793
|
+
|
|
1794
|
+
# Parse the module output: finding TSV lines, then GATE:/SHA:/HASHOK: controls.
|
|
1795
|
+
local gate_status="inconclusive"
|
|
1796
|
+
local gate_summary="ledger comparison produced no gate verdict"
|
|
1797
|
+
local line
|
|
1798
|
+
while IFS= read -r line; do
|
|
1799
|
+
case "$line" in
|
|
1800
|
+
GATE:*)
|
|
1801
|
+
# GATE:<status>:<summary>
|
|
1802
|
+
local rest="${line#GATE:}"
|
|
1803
|
+
gate_status="${rest%%:*}"
|
|
1804
|
+
gate_summary="${rest#*:}"
|
|
1805
|
+
;;
|
|
1806
|
+
SHA:*)
|
|
1807
|
+
_VERIFY_EXPECT_LEDGER_SHA="${line#SHA:}"
|
|
1808
|
+
;;
|
|
1809
|
+
HASHOK:*)
|
|
1810
|
+
_VERIFY_EXPECT_LEDGER_HASH_OK="${line#HASHOK:}"
|
|
1811
|
+
;;
|
|
1812
|
+
*)
|
|
1813
|
+
# A finding TSV line (severity has no leading control prefix).
|
|
1814
|
+
[ -n "$line" ] && printf '%s\n' "$line" >>"$_VERIFY_FINDINGS_FILE"
|
|
1815
|
+
;;
|
|
1816
|
+
esac
|
|
1817
|
+
done <<<"$compare_out"
|
|
1818
|
+
|
|
1819
|
+
_verify_add_gate "expectation_ledger" "$gate_status" "loki-ledger" "$gate_summary" "true"
|
|
1820
|
+
return 0
|
|
1821
|
+
}
|
|
1822
|
+
|
|
1119
1823
|
# ---------------------------------------------------------------------------
|
|
1120
1824
|
# Hosted-engine enrichment (--hosted opt-in).
|
|
1121
1825
|
#
|
|
@@ -1356,6 +2060,11 @@ verify_main() {
|
|
|
1356
2060
|
local started_at completed_at
|
|
1357
2061
|
started_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
1358
2062
|
|
|
2063
|
+
# Expose the resolved output dir to gates that write artifacts (runtime boot
|
|
2064
|
+
# log / screenshot / runtime.json) so those land alongside evidence.json even
|
|
2065
|
+
# when --out overrides the default.
|
|
2066
|
+
VERIFY_OUT_DIR="$out_dir"
|
|
2067
|
+
|
|
1359
2068
|
_verify_log "loki verify (deterministic-only MVP) starting; base=$base_ref out=$out_dir"
|
|
1360
2069
|
|
|
1361
2070
|
verify_diff_base "$base_ref"
|
|
@@ -1382,6 +2091,10 @@ verify_main() {
|
|
|
1382
2091
|
verify_gate_static "$tree"
|
|
1383
2092
|
verify_gate_secret_scan "$tree"
|
|
1384
2093
|
verify_gate_dependency_audit "$tree"
|
|
2094
|
+
# Runtime boot smoke (NET-NEW). Self-suppresses (no gate row) when no
|
|
2095
|
+
# startable HTTP app is detectable, so library/CLI repos stay byte-
|
|
2096
|
+
# identical. Opt-out via LOKI_RUNTIME_GATE=0.
|
|
2097
|
+
verify_gate_runtime "$tree"
|
|
1385
2098
|
fi
|
|
1386
2099
|
|
|
1387
2100
|
# Living-spec integration: when a spec lock exists, fold a SPEC_DRIFT
|
|
@@ -1390,6 +2103,11 @@ verify_main() {
|
|
|
1390
2103
|
# because the optional spec module is missing.
|
|
1391
2104
|
verify_spec_drift_gate "$tree"
|
|
1392
2105
|
|
|
2106
|
+
# Annotate-before-act ledger: diff pre-committed expected outcomes against
|
|
2107
|
+
# observed results. Total no-op (no gate row, no finding, byte-identical
|
|
2108
|
+
# evidence) when no ledger exists -- the common case today.
|
|
2109
|
+
verify_expectation_ledger_gate "$tree"
|
|
2110
|
+
|
|
1393
2111
|
verify_compute_verdict "$block_on"
|
|
1394
2112
|
|
|
1395
2113
|
completed_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
package/dashboard/__init__.py
CHANGED
package/docs/INSTALLATION.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The flagship product of [Autonomi](https://www.autonomi.dev/). Loki Mode is a spec-driven autonomous builder with a built-in trust layer that takes any spec to a deployed product and verifies completion with evidence (quality gates plus a completion council), not just a "done" claim. Complete installation instructions for all platforms and use cases.
|
|
4
4
|
|
|
5
|
-
**Version:** v7.
|
|
5
|
+
**Version:** v7.109.0
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
package/docs/TOP-100-BACKLOG.md
CHANGED
|
@@ -120,3 +120,40 @@ Generated 2026-07-01 by the top100-backlog-enumeration ultracode workflow (7 age
|
|
|
120
120
|
| 98 | low | S | autonomous | dashboard | Web-app docs: fill video placeholder (coming-soon text) |
|
|
121
121
|
| 99 | low | M | autonomous | cli | C# provider support (roslyn analyzers + dotnet build, deferred) |
|
|
122
122
|
| 100 | low | XL | founder-gated | engine | BMAD-METHOD v6 integration (net-new, needs founder planning) |
|
|
123
|
+
|
|
124
|
+
## Gate split (what can actually close autonomously) - 2026-07-01
|
|
125
|
+
|
|
126
|
+
The /goal asks to work through all 100. The honest structural reality: only a
|
|
127
|
+
subset is autonomously completable. The rest need a founder decision or are
|
|
128
|
+
community/CI-infra work that cannot be forced from this session.
|
|
129
|
+
|
|
130
|
+
- **Autonomous (I can execute end-to-end): ~25 items** (the `gate=autonomous` rows).
|
|
131
|
+
- **Founder-gated (blocked on a decision only the founder can make): ~35 items.**
|
|
132
|
+
Licensing (#5), hosted deploy target (#6, #23, #36), enterprise RBAC/SSO (#15),
|
|
133
|
+
sandbox backend choice (#14, #42), spend (#49 SWE-bench, #18 live-model), npm
|
|
134
|
+
publish + brand (#4, #13, #61), telemetry (#62), GitHub App (#46). Prepped to
|
|
135
|
+
one-approval-away where possible; CANNOT close without the founder.
|
|
136
|
+
- **Community / CI-infra (~40 items):** flake root-causes, coverage thresholds,
|
|
137
|
+
platform runners (ARM64/Windows), distribution resilience. Environmental
|
|
138
|
+
hardening, not single-session feature work.
|
|
139
|
+
|
|
140
|
+
### Shipped this session (accuracy-moat, from the competitive gap-analysis)
|
|
141
|
+
|
|
142
|
+
Net-new accuracy work driven from docs/research-2026-07/gap-analysis-backlog.json
|
|
143
|
+
(a companion list), NOT retroactive Top-100 claims:
|
|
144
|
+
|
|
145
|
+
- v7.105.0 - convergence: council evaluates on completion-claim (4.0x faster, n=3). commit b8a368a0.
|
|
146
|
+
- v7.106.0 - reverse-classical test provenance (tautological tests downgraded). commit 71299faa.
|
|
147
|
+
- v7.107.0 - loki mcp --transport http loopback bind + bearer auth (+ latent crash fix). commit f3aa523c.
|
|
148
|
+
- v7.108.0 - runtime boot smoke gate + annotate-before-act expectation ledger. commit 806fc374.
|
|
149
|
+
|
|
150
|
+
### Top-100 items with a concrete shipped commit
|
|
151
|
+
|
|
152
|
+
- #40 (dashboard memory summary on JSON-backed projects) - commit 330de52d.
|
|
153
|
+
- #22 (3 parallel code reviewers) - already present in the engine (council path).
|
|
154
|
+
|
|
155
|
+
### Next autonomous batch (by value, unblocked, user-visible)
|
|
156
|
+
|
|
157
|
+
Picked from the `autonomous` rows by value + real-user impact (not primitives that
|
|
158
|
+
ship dormant). Verify each before starting; several may overlap the already-built
|
|
159
|
+
private autonomi-verify TS engine (e.g. #7/#8 - check before rebuilding).
|
package/loki-ts/dist/loki.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// @bun
|
|
2
|
-
var X8=Object.defineProperty;var K8=($)=>$;function q8($,Q){this[$]=K8.bind(null,Q)}var b=($,Q)=>{for(var Z in Q)X8($,Z,{get:Q[Z],enumerable:!0,configurable:!0,set:q8.bind(Q,Z)})};var L=($,Q)=>()=>($&&(Q=$($=0)),Q);var J$=import.meta.require;var v1={};b(v1,{lokiDir:()=>P,homeLokiDir:()=>T$,findRepoRootForVersion:()=>e$,REPO_ROOT:()=>h});import{resolve as r,dirname as i$}from"path";import{fileURLToPath as J8}from"url";import{existsSync as x$}from"fs";import{homedir as V8}from"os";function W8(){let $=y1;for(let Q=0;Q<6;Q++){if(x$(r($,"VERSION"))&&x$(r($,"autonomy/run.sh")))return $;let Z=i$($);if(Z===$)break;$=Z}return r(y1,"..","..","..")}function e$($){let Q=$;for(let Z=0;Z<6;Z++){if(x$(r(Q,"VERSION"))&&x$(r(Q,"autonomy/run.sh")))return Q;let z=i$(Q);if(z===Q)break;Q=z}return r($,"..","..","..")}function P(){return process.env.LOKI_DIR??r(process.cwd(),".loki")}function T$(){return r(V8(),".loki")}var y1,h;var C=L(()=>{y1=i$(J8(import.meta.url));h=W8()});import{readFileSync as U8}from"fs";import{resolve as H8,dirname as G8}from"path";import{fileURLToPath as B8}from"url";function N$(){if(Q$!==null)return Q$;let $="7.
|
|
2
|
+
var X8=Object.defineProperty;var K8=($)=>$;function q8($,Q){this[$]=K8.bind(null,Q)}var b=($,Q)=>{for(var Z in Q)X8($,Z,{get:Q[Z],enumerable:!0,configurable:!0,set:q8.bind(Q,Z)})};var L=($,Q)=>()=>($&&(Q=$($=0)),Q);var J$=import.meta.require;var v1={};b(v1,{lokiDir:()=>P,homeLokiDir:()=>T$,findRepoRootForVersion:()=>e$,REPO_ROOT:()=>h});import{resolve as r,dirname as i$}from"path";import{fileURLToPath as J8}from"url";import{existsSync as x$}from"fs";import{homedir as V8}from"os";function W8(){let $=y1;for(let Q=0;Q<6;Q++){if(x$(r($,"VERSION"))&&x$(r($,"autonomy/run.sh")))return $;let Z=i$($);if(Z===$)break;$=Z}return r(y1,"..","..","..")}function e$($){let Q=$;for(let Z=0;Z<6;Z++){if(x$(r(Q,"VERSION"))&&x$(r(Q,"autonomy/run.sh")))return Q;let z=i$(Q);if(z===Q)break;Q=z}return r($,"..","..","..")}function P(){return process.env.LOKI_DIR??r(process.cwd(),".loki")}function T$(){return r(V8(),".loki")}var y1,h;var C=L(()=>{y1=i$(J8(import.meta.url));h=W8()});import{readFileSync as U8}from"fs";import{resolve as H8,dirname as G8}from"path";import{fileURLToPath as B8}from"url";function N$(){if(Q$!==null)return Q$;let $="7.109.0";if(typeof $==="string"&&$.length>0)return Q$=$,Q$;try{let Q=G8(B8(import.meta.url)),Z=e$(Q);Q$=U8(H8(Z,"VERSION"),"utf-8").trim()}catch{Q$="unknown"}return Q$}var Q$=null;var $1=L(()=>{C()});var c1={};b(c1,{runOrThrow:()=>x8,run:()=>F,readStreamCapped:()=>u1,commandVersion:()=>S8,commandExists:()=>f,ShellError:()=>Q1,MAX_STDOUT_BYTES:()=>f1});async function u1($,Q=f1){let Z=$.getReader(),z=new TextDecoder,X="",q=0;try{while(q<Q){let{done:K,value:W}=await Z.read();if(K)break;if(!W)continue;if(q+=W.byteLength,q>Q){let V=W.byteLength-(q-Q);X+=z.decode(W.subarray(0,V),{stream:!0});break}X+=z.decode(W,{stream:!0})}X+=z.decode()}finally{try{await Z.cancel()}catch{}Z.releaseLock()}return X}async function F($,Q={}){let Z=Bun.spawn({cmd:[...$],stdout:"pipe",stderr:"pipe",env:Q.env?{...process.env,...Q.env}:process.env,cwd:Q.cwd}),z,X;if(Q.timeoutMs&&Q.timeoutMs>0)z=setTimeout(()=>{try{Z.kill("SIGTERM")}catch{}X=setTimeout(()=>{try{Z.kill("SIGKILL")}catch{}},2000)},Q.timeoutMs);try{let[q,K,W]=await Promise.all([u1(Z.stdout),new Response(Z.stderr).text(),Z.exited]);return{stdout:q,stderr:K,exitCode:W}}finally{if(z)clearTimeout(z);if(X)clearTimeout(X)}}async function x8($,Q={}){let Z=await F($,Q);if(Z.exitCode!==0)throw new Q1(`command failed (${Z.exitCode}): ${$.join(" ")}`,Z.exitCode,Z.stdout,Z.stderr);return Z}async function f($){let Q=N8($),Z=await F(["sh","-c",`command -v ${Q}`],{timeoutMs:5000});if(Z.exitCode===0)return Z.stdout.trim()||null;return null}function N8($){if(!/^[A-Za-z0-9._/-]+$/.test($))throw Error(`refused to shell-escape suspect token: ${$}`);return $}async function S8($,Q="--version"){if(!await f($))return null;let z=await F([$,Q],{timeoutMs:5000});if(z.exitCode!==0)return null;return((z.stdout||z.stderr).split(/\r?\n/)[0]?.trim()??"")||null}var f1=16777216,Q1;var d=L(()=>{Q1=class Q1 extends Error{message;exitCode;stdout;stderr;constructor($,Q,Z,z){super($);this.message=$;this.exitCode=Q;this.stdout=Z;this.stderr=z;this.name="ShellError"}}});function t($){return D8?"":$}var D8,T,S,_,nZ,I,k,y,J;var c=L(()=>{D8=(process.env.NO_COLOR??"").length>0;T=t("\x1B[0;31m"),S=t("\x1B[0;32m"),_=t("\x1B[1;33m"),nZ=t("\x1B[0;34m"),I=t("\x1B[0;36m"),k=t("\x1B[1m"),y=t("\x1B[2m"),J=t("\x1B[0m")});import{existsSync as c8}from"fs";async function Z$(){if(A$!==void 0)return A$;let $="/opt/homebrew/bin/python3.12";if(c8($))return A$=$,$;let Q=await f("python3.12");if(Q)return A$=Q,Q;let Z=await f("python3");return A$=Z,Z}async function z$($,Q={}){let Z=await Z$();if(!Z)return{stdout:"",stderr:"python3 not found",exitCode:127};return F([Z,"-c",$],Q)}var A$;var V$=L(()=>{d()});var J0={};b(J0,{runStatus:()=>U3});import{existsSync as v,readFileSync as U$,readdirSync as e1,statSync as $0}from"fs";import{resolve as D,basename as Q3}from"path";import{homedir as Z3}from"os";function Q0($){let Q=Math.trunc($);if(Q>=1e6)return`${(Math.trunc(Q/1e6*10)/10).toFixed(1)}M`;if(Q>=1000)return`${(Math.trunc(Q/1000*10)/10).toFixed(1)}K`;return String(Q)}function Z0($,Q,Z){if(Q===0)return null;let z=Math.trunc($*100/Q),X=Math.trunc($*D$/Q);if(X>D$)X=D$;let q=D$-X,K=S;if(z>=80)K=T;else if(z>=50)K=_;let W="=".repeat(Math.max(0,X))+" ".repeat(Math.max(0,q)),V=Q0($),U=Q0(Q);return` ${k}${Z}${J} ${K}[${W}]${J} ${z}% (${V} / ${U})`}async function X3(){if(await f("jq"))return!0;return process.stdout.write(`${T}Error: jq is required but not installed.${J}
|
|
3
3
|
`),process.stdout.write(`Install with:
|
|
4
4
|
`),process.stdout.write(` brew install jq (macOS)
|
|
5
5
|
`),process.stdout.write(` apt install jq (Debian/Ubuntu)
|
|
@@ -805,4 +805,4 @@ Set LOKI_LEGACY_BASH=1 to force the bash CLI for every command.
|
|
|
805
805
|
`),2}default:return process.stderr.write(`Unknown command: ${Q}
|
|
806
806
|
`),process.stderr.write(z8),2}}i1();process.on("SIGINT",()=>process.exit(130));process.on("SIGTERM",()=>process.exit(143));var EZ=await RZ(Bun.argv.slice(2));process.exit(EZ);
|
|
807
807
|
|
|
808
|
-
//# debugId=
|
|
808
|
+
//# debugId=68DF9A47D3B818A364756E2164756E21
|
package/mcp/__init__.py
CHANGED
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "loki-mode",
|
|
3
3
|
"mcpName": "io.github.asklokesh/loki-mode",
|
|
4
|
-
"version": "7.
|
|
4
|
+
"version": "7.109.0",
|
|
5
5
|
"description": "Loki Mode by Autonomi. Autonomous spec-to-product system: takes a PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief to a deployed app via the RARV-C closure loop with 8 quality gates. Provider-agnostic (Claude Code, OpenAI Codex, Cline, Aider).",
|
|
6
6
|
"keywords": [
|
|
7
7
|
"agent",
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
|
|
3
3
|
"name": "loki-mode",
|
|
4
4
|
"displayName": "Loki Mode",
|
|
5
|
-
"version": "7.
|
|
5
|
+
"version": "7.109.0",
|
|
6
6
|
"description": "Autonomous spec-to-product build system with a built-in trust layer (RARV-C closure loop, 8 quality gates, completion council). Ships Loki's spec-hardening, drift-detection, and deterministic PR verification commands plus the Loki MCP server.",
|
|
7
7
|
"author": {
|
|
8
8
|
"name": "Autonomi",
|