@andresmassello/uscha 1.62.0 → 1.64.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +1 -1
- package/uscha-kit/.claude/skills/uscha-devloop/qa_ledger.py +457 -0
- package/uscha-kit/.claude/skills/uscha-reverse-discovery/SKILL.md +65 -24
- package/uscha-kit/.claude-plugin/plugin.json +2 -2
- package/uscha-kit/.codex-plugin/plugin.json +1 -1
- package/uscha-kit/README.md +49 -1
- package/uscha-kit/VERSION +1 -1
- package/uscha-kit/reports/junit/.cleanroom-cases.json +1 -0
- package/uscha-kit/reports/junit/.curation-cases.json +1 -0
- package/uscha-kit/skills/uscha-devloop/qa_ledger.py +457 -0
- package/uscha-kit/skills/uscha-reverse-discovery/SKILL.md +65 -24
- package/uscha-kit/templates/BEHAVIOR-LEDGER.md +13 -0
- package/uscha-kit/uscha.config.json +5 -1
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
|
|
3
3
|
"name": "uscha",
|
|
4
|
-
"version": "1.
|
|
4
|
+
"version": "1.64.0",
|
|
5
5
|
"displayName": "Uscha",
|
|
6
|
-
"description": "Spec-driven development for LLM coding agents: 9 skills (discovery, adr-refine, reverse-discovery, characterize, devloop, sysdoc, rubric, mirador, status) + a stdlib measurement engine (qa_ledger.py,
|
|
6
|
+
"description": "Spec-driven development for LLM coding agents: 9 skills (discovery, adr-refine, reverse-discovery, characterize, devloop, sysdoc, rubric, mirador, status) + a stdlib measurement engine (qa_ledger.py, 34 subcommands + universal installer + npm/npx router). Facts block, guesses advise; the human approves.",
|
|
7
7
|
"author": {
|
|
8
8
|
"name": "Andres Massello",
|
|
9
9
|
"url": "https://github.com/andresmassello"
|
package/uscha-kit/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# uscha-kit
|
|
2
2
|
|
|
3
|
-
**Kit version:** v1.
|
|
3
|
+
**Kit version:** v1.64.0 <!-- uscha:version --> · **[uscha.dev](https://uscha.dev)**
|
|
4
4
|
|
|
5
5
|
Spec-driven orchestrator + multi-repo QA for Claude Code, with a deterministic ledger.
|
|
6
6
|
**Nine skills** (`uscha-discovery`, `uscha-adr-refine`, `uscha-devloop`, `uscha-sysdoc`, `uscha-reverse-discovery`,
|
|
@@ -149,6 +149,54 @@ knowing a tree was dirty does not tell you the evidence is wrong, only that it w
|
|
|
149
149
|
produced from a commit alone. The git-worktree clean-room that would answer the stronger
|
|
150
150
|
question is deliberately deferred; ADR-007 records why.
|
|
151
151
|
|
|
152
|
+
## Clean-room (ADR-008) - verify the commit, not your tree
|
|
153
|
+
|
|
154
|
+
A suite can pass in your working tree *because of uncommitted state* and fail against the
|
|
155
|
+
commit alone. The ledger records that as measured, with provenance true of the TREE and not of
|
|
156
|
+
the COMMIT that will merge.
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
python qa_ledger.py cleanroom --repo <name> --run "<your test command>" [--setup "npm ci"]
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
It creates a detached `git worktree` of the commit, verifies it is clean by construction, runs
|
|
163
|
+
what you gave it, records `ref` / `worktree_sha` / `status` / wall-clock, and removes the
|
|
164
|
+
worktree unconditionally (a zombie worktree is a defect). Status is specific:
|
|
165
|
+
`GREEN`, `RED`, `SETUP_FAILED` (a bootstrap failure is not a failing suite),
|
|
166
|
+
`WORKTREE_FAILED`, `WORKTREE_DIRTY`.
|
|
167
|
+
|
|
168
|
+
**The engine never decides what to run.** The command arrives via `--run`, the same contract
|
|
169
|
+
`golden-coverage` uses for `--harness`: the engine owns isolation, the SHA binding and
|
|
170
|
+
cleanup, and never turns a config file into a code-execution surface. That is also why it
|
|
171
|
+
works for any stack - it never had to know the stack.
|
|
172
|
+
|
|
173
|
+
**Opt-in gate:** `defaults.clean_room.mode` absent or `"off"` and nothing changes. `"final"`
|
|
174
|
+
makes `pr-ready` require a GREEN run pinned to the current HEAD; a new commit makes the
|
|
175
|
+
previous run stale for the gate.
|
|
176
|
+
|
|
177
|
+
Honest limit: this is **not** a substitute for CI. A local worktree runs on your machine, your
|
|
178
|
+
OS, your shell - environment variance is invisible to it. Different failure class, different
|
|
179
|
+
instrument.
|
|
180
|
+
|
|
181
|
+
## Curation (ADR-009/010) - candidates in quarantine, verdicts on the record
|
|
182
|
+
|
|
183
|
+
Reverse discovery's brownfield entry: the agent may author CANDIDATE specs of a legacy
|
|
184
|
+
system - in `discovery/`, with mandatory `evidence` (`test|code|inference`) and
|
|
185
|
+
`confidence` frontmatter, refs the engine resolves against real files - and **nothing is
|
|
186
|
+
promoted without a human verdict**:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
python qa_ledger.py curation-check --repo <name> [--json]
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Verdicts live in `BEHAVIOR-LEDGER.md` (`preserve` / `fix` / `undefined`, one ADR each) -
|
|
193
|
+
a human-readable table with machine-enforced rules: strict shape (`exit 2` on malformed,
|
|
194
|
+
because under this gate a silent "no verdicts" would UNBLOCK what it guards), append-only
|
|
195
|
+
verified against git (revert = new row + new ADR, never an edit; latest row wins). While
|
|
196
|
+
any candidate lacks a verdict, `pr-ready` is blocked naming it (INV-CURATION-01) - the
|
|
197
|
+
quarantine is measured, not promised. No `discovery/` directory -> the feature does not
|
|
198
|
+
exist and nothing changes.
|
|
199
|
+
|
|
152
200
|
## End-to-end flow
|
|
153
201
|
|
|
154
202
|
`uscha-discovery` is the front for something new (you only have the idea); `uscha-adr-refine` is the front
|
package/uscha-kit/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
uscha-kit 1.
|
|
1
|
+
uscha-kit 1.64.0
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"AC-CR-06": true, "AC-CR-02": true, "AC-CR-04": true, "AC-CR-05": true, "AC-CR-03": true, "AC-CR-01": true, "AC-CR-07": true, "AC-CR-08": true}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"AC-RD-07": true, "AC-RD-01": true, "AC-RD-02": true, "AC-RD-03": true, "AC-RD-06": true, "AC-RD-04": true, "AC-RD-05": true}
|
|
@@ -2629,6 +2629,58 @@ def _derive_phase(ledger, name, node, k, qa_order):
|
|
|
2629
2629
|
reasons.append("la evidencia medida no contiene tests ejecutados")
|
|
2630
2630
|
_go, sev = _gate_open_and_sev(node)
|
|
2631
2631
|
blk = sev.get("BLOCKER", 0) + sev.get("CRITICAL", 0)
|
|
2632
|
+
# clean-room gate (ADR-008). OPT-IN: no clean_room block, or mode "off", and this does not
|
|
2633
|
+
# exist -- behavior identical to earlier releases. Declared "final": pr-ready additionally
|
|
2634
|
+
# requires clean-room evidence that is GREEN and pinned to the CURRENT HEAD. A new commit
|
|
2635
|
+
# makes the previous run stale for the gate, the same staleness posture the rest of the
|
|
2636
|
+
# engine takes: evidence certifies the thing it was measured against, nothing later.
|
|
2637
|
+
# curation gate (ADR-009, INV-CURATION-01). In use the moment discovery/ exists --
|
|
2638
|
+
# creating candidates IS the opt-in. Fail-closed both ways: an unjudged candidate blocks,
|
|
2639
|
+
# and a malformation blocks too, because "could not validate" must never read as judged.
|
|
2640
|
+
_cu_cfg = (_repo_cfg(ledger, name) if name != "integration"
|
|
2641
|
+
else {"path": "."}) # synthetic scope: never in config["repos"]
|
|
2642
|
+
_cu = _curation_state(_cu_cfg.get("path", "."))
|
|
2643
|
+
if _cu is not None:
|
|
2644
|
+
if _cu["malformed"] or _cu["ledger_errors"] or _cu["append_only"] == "violation":
|
|
2645
|
+
_bits = [m["candidate"] for m in _cu["malformed"][:3]]
|
|
2646
|
+
if _cu["ledger_errors"]:
|
|
2647
|
+
_bits.append(BEHAVIOR_LEDGER_FILE + " malformado")
|
|
2648
|
+
if _cu["append_only"] == "violation":
|
|
2649
|
+
_bits.append(BEHAVIOR_LEDGER_FILE + " editado (append-only)")
|
|
2650
|
+
reasons.append("curation invalida: " + "; ".join(_bits)
|
|
2651
|
+
+ " -- corregir antes de avanzar")
|
|
2652
|
+
conv = False
|
|
2653
|
+
elif _cu["unjudged"]:
|
|
2654
|
+
reasons.append("candidata(s) sin veredicto humano: "
|
|
2655
|
+
+ ", ".join(_cu["unjudged"][:3])
|
|
2656
|
+
+ (" (+%d)" % (len(_cu["unjudged"]) - 3)
|
|
2657
|
+
if len(_cu["unjudged"]) > 3 else "")
|
|
2658
|
+
+ " -- INV-CURATION-01: sin juicio no hay promocion")
|
|
2659
|
+
conv = False
|
|
2660
|
+
_cr = _cr_cfg(ledger)
|
|
2661
|
+
if _cr and _cr.get("mode") == "final":
|
|
2662
|
+
_head = None
|
|
2663
|
+
_hr = None
|
|
2664
|
+
_crcfg = (_repo_cfg(ledger, name) if name != "integration"
|
|
2665
|
+
else {"path": "."}) # synthetic scope: never in config["repos"]
|
|
2666
|
+
try:
|
|
2667
|
+
_hr = subprocess.run(["git", "rev-parse", "HEAD"],
|
|
2668
|
+
cwd=_crcfg.get("path", "."),
|
|
2669
|
+
capture_output=True, text=True, encoding="utf-8",
|
|
2670
|
+
errors="replace")
|
|
2671
|
+
except OSError:
|
|
2672
|
+
_hr = None
|
|
2673
|
+
if _hr is not None and _hr.returncode == 0:
|
|
2674
|
+
_head = _hr.stdout.strip()
|
|
2675
|
+
_run = _cr_latest(ledger, name, _head) if _head else None
|
|
2676
|
+
if not _head:
|
|
2677
|
+
reasons.append("clean-room declarado pero no se pudo resolver HEAD "
|
|
2678
|
+
"(no se mide, no se aprueba)")
|
|
2679
|
+
conv = False
|
|
2680
|
+
elif not _run or not _run.get("ok"):
|
|
2681
|
+
reasons.append("falta clean-room verde para %s (evidencia del arbol no "
|
|
2682
|
+
"certifica el commit)" % _head[:8])
|
|
2683
|
+
conv = False
|
|
2632
2684
|
if conv and tests_measured_green and not tests_red and blk == 0:
|
|
2633
2685
|
evidence = ["ciclo de agente limpio", "tests verdes (medidos)",
|
|
2634
2686
|
"0 BLOCKER/CRITICAL abiertos"]
|
|
@@ -3355,6 +3407,390 @@ def cmd_golden_coverage(args):
|
|
|
3355
3407
|
|
|
3356
3408
|
|
|
3357
3409
|
|
|
3410
|
+
# --------------------------------------------------------------------------- #
|
|
3411
|
+
# cleanroom (ADR-008: verify the COMMIT, not the tree -- opt-in, human-supplied command)
|
|
3412
|
+
# --------------------------------------------------------------------------- #
|
|
3413
|
+
|
|
3414
|
+
CLEAN_ROOM_KEY = "clean_room"
|
|
3415
|
+
|
|
3416
|
+
|
|
3417
|
+
def _cr_cfg(ledger):
|
|
3418
|
+
cfg = ledger["config"].get("defaults", {}).get(CLEAN_ROOM_KEY)
|
|
3419
|
+
return cfg if isinstance(cfg, dict) else None
|
|
3420
|
+
|
|
3421
|
+
|
|
3422
|
+
def _cr_latest(ledger, repo, ref=None):
|
|
3423
|
+
"""Latest clean-room record for a repo, optionally pinned to one ref."""
|
|
3424
|
+
runs = [e for e in ledger.get(CLEAN_ROOM_KEY, [])
|
|
3425
|
+
if e.get("repo") == repo and (ref is None or e.get("ref") == ref)]
|
|
3426
|
+
return runs[-1] if runs else None
|
|
3427
|
+
|
|
3428
|
+
|
|
3429
|
+
def cmd_cleanroom(args):
|
|
3430
|
+
"""Run a command against a CLEAN CHECKOUT of one commit, in a throwaway worktree.
|
|
3431
|
+
|
|
3432
|
+
Evidence produced in the maker's tree is true of the TREE; this produces evidence true
|
|
3433
|
+
of the COMMIT. As a side effect maker != checker becomes physical: the worktree cannot
|
|
3434
|
+
see uncommitted state.
|
|
3435
|
+
|
|
3436
|
+
The engine does NOT decide what to run. The command arrives explicitly via --run, the
|
|
3437
|
+
same contract golden-coverage uses for --harness: the engine owns what it can guarantee
|
|
3438
|
+
(isolation, the SHA binding, cleanup) and never guesses what a project's suite is.
|
|
3439
|
+
Reading test_command_* from config and executing it would make the engine an executor
|
|
3440
|
+
of config-supplied shell, which it is not (ADR-008)."""
|
|
3441
|
+
import time
|
|
3442
|
+
ledger = _load(args.ledger)
|
|
3443
|
+
_repo_node(ledger, args.repo)
|
|
3444
|
+
repo_path = _repo_cfg(ledger, args.repo).get("path", ".")
|
|
3445
|
+
|
|
3446
|
+
def git(*a, **kw):
|
|
3447
|
+
cwd = kw.pop("cwd", repo_path)
|
|
3448
|
+
try:
|
|
3449
|
+
return subprocess.run(["git"] + list(a), cwd=cwd, capture_output=True,
|
|
3450
|
+
text=True, encoding="utf-8", errors="replace")
|
|
3451
|
+
except OSError:
|
|
3452
|
+
return None
|
|
3453
|
+
|
|
3454
|
+
rev = git("rev-parse", "--verify", (args.ref or "HEAD") + "^{commit}")
|
|
3455
|
+
if rev is None or rev.returncode != 0 or not rev.stdout.strip():
|
|
3456
|
+
print("[qa_ledger] cannot resolve ref %r in %s - nothing to verify."
|
|
3457
|
+
% (args.ref or "HEAD", repo_path), file=sys.stderr)
|
|
3458
|
+
sys.exit(2)
|
|
3459
|
+
sha = rev.stdout.strip()
|
|
3460
|
+
|
|
3461
|
+
wt = tempfile.mkdtemp(prefix="uscha-cleanroom-")
|
|
3462
|
+
target = os.path.join(wt, "tree")
|
|
3463
|
+
started = time.time()
|
|
3464
|
+
record = {"repo": args.repo, "ref": sha, "at": _now(), "ok": False,
|
|
3465
|
+
"status": None, "wall_ms": None, "worktree_sha": None}
|
|
3466
|
+
keep = bool((_cr_cfg(ledger) or {}).get("keep_worktree_on_failure"))
|
|
3467
|
+
try:
|
|
3468
|
+
add = git("worktree", "add", "--detach", target, sha)
|
|
3469
|
+
if add is None or add.returncode != 0:
|
|
3470
|
+
record["status"] = "WORKTREE_FAILED"
|
|
3471
|
+
print("[qa_ledger] git worktree add failed:\n%s"
|
|
3472
|
+
% ((add.stderr if add else "git unavailable") or "")[-800:], file=sys.stderr)
|
|
3473
|
+
else:
|
|
3474
|
+
# a worktree of a commit must be clean by construction; if it is not, something
|
|
3475
|
+
# (a smudge filter, a hook, a stale index) intervened and the isolation claim is
|
|
3476
|
+
# already false. Say so rather than measure in it.
|
|
3477
|
+
st = git("status", "--porcelain", cwd=target)
|
|
3478
|
+
if st is None or st.returncode != 0 or st.stdout.strip():
|
|
3479
|
+
record["status"] = "WORKTREE_DIRTY"
|
|
3480
|
+
else:
|
|
3481
|
+
record["worktree_sha"] = sha
|
|
3482
|
+
if args.setup:
|
|
3483
|
+
r = subprocess.run(args.setup, cwd=target, shell=True,
|
|
3484
|
+
capture_output=True, text=True,
|
|
3485
|
+
encoding="utf-8", errors="replace")
|
|
3486
|
+
if r.returncode != 0:
|
|
3487
|
+
record["status"] = "SETUP_FAILED"
|
|
3488
|
+
print((r.stderr or "")[-800:], file=sys.stderr)
|
|
3489
|
+
if record["status"] is None:
|
|
3490
|
+
r = subprocess.run(args.run, cwd=target, shell=True,
|
|
3491
|
+
capture_output=True, text=True,
|
|
3492
|
+
encoding="utf-8", errors="replace")
|
|
3493
|
+
record["exit_code"] = r.returncode
|
|
3494
|
+
record["status"] = "GREEN" if r.returncode == 0 else "RED"
|
|
3495
|
+
record["ok"] = r.returncode == 0
|
|
3496
|
+
if r.returncode != 0:
|
|
3497
|
+
print((r.stdout or "")[-1500:], file=sys.stderr)
|
|
3498
|
+
finally:
|
|
3499
|
+
record["wall_ms"] = int((time.time() - started) * 1000)
|
|
3500
|
+
# Cleanup is unconditional unless the human asked to inspect a FAILURE: a zombie
|
|
3501
|
+
# worktree is a defect, and `git worktree remove` alone leaves the admin entry behind.
|
|
3502
|
+
if record["ok"] or not keep:
|
|
3503
|
+
git("worktree", "remove", "--force", target)
|
|
3504
|
+
git("worktree", "prune")
|
|
3505
|
+
shutil.rmtree(wt, ignore_errors=True)
|
|
3506
|
+
# VERIFY the removal instead of assuming it. rmtree(ignore_errors=True) and a
|
|
3507
|
+
# discarded git return code can both fail in silence -- typically on Windows,
|
|
3508
|
+
# where a handle the caller's command left open blocks removal.
|
|
3509
|
+
if os.path.exists(target):
|
|
3510
|
+
record["cleanup_failed"] = True
|
|
3511
|
+
record["worktree_kept_at"] = target
|
|
3512
|
+
print("[qa_ledger] WARNING: the clean-room worktree could not be removed and "
|
|
3513
|
+
"is still at %s (a process may still hold a handle). Remove it with: "
|
|
3514
|
+
"git worktree remove --force %s && git worktree prune"
|
|
3515
|
+
% (target, target), file=sys.stderr)
|
|
3516
|
+
else:
|
|
3517
|
+
record["worktree_kept_at"] = target
|
|
3518
|
+
|
|
3519
|
+
ledger.setdefault(CLEAN_ROOM_KEY, []).append(record)
|
|
3520
|
+
ledger["step_counter"] += 1
|
|
3521
|
+
ledger["steps"].append({"n": ledger["step_counter"], "at": _now(),
|
|
3522
|
+
"kind": "cleanroom", "repo": args.repo})
|
|
3523
|
+
_save(args.ledger, ledger)
|
|
3524
|
+
|
|
3525
|
+
if args.json:
|
|
3526
|
+
print(json.dumps(record, indent=2, ensure_ascii=False))
|
|
3527
|
+
else:
|
|
3528
|
+
print("CLEANROOM %s @ %s: %s (%.1fs)"
|
|
3529
|
+
% (args.repo, sha[:8], record["status"], (record["wall_ms"] or 0) / 1000.0))
|
|
3530
|
+
if record.get("worktree_kept_at"):
|
|
3531
|
+
print(" worktree kept for inspection: " + record["worktree_kept_at"])
|
|
3532
|
+
sys.exit(0 if record["ok"] else 1)
|
|
3533
|
+
|
|
3534
|
+
|
|
3535
|
+
|
|
3536
|
+
# --------------------------------------------------------------------------- #
|
|
3537
|
+
# curation (ADR-009/010: candidates in quarantine, verdicts in the behavior
|
|
3538
|
+
# ledger, and a promotion gate the ENGINE measures -- INV-CURATION-01)
|
|
3539
|
+
# --------------------------------------------------------------------------- #
|
|
3540
|
+
|
|
3541
|
+
BEHAVIOR_LEDGER_FILE = "BEHAVIOR-LEDGER.md"
|
|
3542
|
+
CANDIDATE_DIR = "discovery"
|
|
3543
|
+
_BL_VERDICTS = ("preserve", "fix", "undefined")
|
|
3544
|
+
|
|
3545
|
+
|
|
3546
|
+
def _parse_candidate(path):
|
|
3547
|
+
"""Parse one candidate's frontmatter. Returns (data, errors); a candidate with errors
|
|
3548
|
+
is INVALID and named -- never silently skipped, because a skipped candidate would walk
|
|
3549
|
+
past the promotion gate unjudged."""
|
|
3550
|
+
errors = []
|
|
3551
|
+
try:
|
|
3552
|
+
# utf-8-sig: a BOM-adding editor must not turn a well-formed candidate into a
|
|
3553
|
+
# false "no frontmatter" (fresh-review LOW)
|
|
3554
|
+
with open(path, encoding="utf-8-sig", errors="replace") as fh:
|
|
3555
|
+
lines = fh.read().splitlines()
|
|
3556
|
+
except OSError as exc:
|
|
3557
|
+
return None, ["unreadable: %s" % exc]
|
|
3558
|
+
if not lines or lines[0].strip() != "---":
|
|
3559
|
+
return None, ["no frontmatter (evidence/confidence are mandatory, ADR-009)"]
|
|
3560
|
+
fm, i = [], 1
|
|
3561
|
+
while i < len(lines) and lines[i].strip() != "---":
|
|
3562
|
+
fm.append(lines[i]); i += 1
|
|
3563
|
+
if i >= len(lines):
|
|
3564
|
+
return None, ["frontmatter never closes"]
|
|
3565
|
+
etype, refs, conf, in_refs, in_evidence = None, [], None, False, False
|
|
3566
|
+
for ln in fm:
|
|
3567
|
+
s = ln.strip()
|
|
3568
|
+
indented = ln.startswith((" ", "\t"))
|
|
3569
|
+
if s.startswith("evidence:") and not indented:
|
|
3570
|
+
in_evidence = True; in_refs = False
|
|
3571
|
+
elif s.startswith("type:"):
|
|
3572
|
+
# scope + duplicates are MALFORMATION, not last-value-wins: a stray top-level
|
|
3573
|
+
# type:/confidence: after the evidence block silently overrode the nested one
|
|
3574
|
+
# and walked straight past the inference->low invariant (fresh-review HIGH).
|
|
3575
|
+
if not (in_evidence and indented):
|
|
3576
|
+
errors.append("type: outside the evidence block")
|
|
3577
|
+
elif etype is not None:
|
|
3578
|
+
errors.append("duplicate type: declaration")
|
|
3579
|
+
else:
|
|
3580
|
+
etype = s[len("type:"):].strip().strip("\x27\x22")
|
|
3581
|
+
in_refs = False
|
|
3582
|
+
elif s.startswith("refs:"):
|
|
3583
|
+
if not (in_evidence and indented):
|
|
3584
|
+
errors.append("refs: outside the evidence block")
|
|
3585
|
+
in_refs = in_evidence and indented
|
|
3586
|
+
elif s.startswith("confidence:") and not indented:
|
|
3587
|
+
if conf is not None:
|
|
3588
|
+
errors.append("duplicate confidence: declaration")
|
|
3589
|
+
else:
|
|
3590
|
+
conf = s[len("confidence:"):].strip().strip("\x27\x22")
|
|
3591
|
+
in_refs = False; in_evidence = False
|
|
3592
|
+
elif in_refs and s.startswith("- "):
|
|
3593
|
+
refs.append(s[2:].strip().strip("\x27\x22"))
|
|
3594
|
+
elif s and not indented:
|
|
3595
|
+
in_refs = False; in_evidence = False
|
|
3596
|
+
if etype not in ("test", "code", "inference"):
|
|
3597
|
+
errors.append("evidence.type %r (expected test|code|inference)" % etype)
|
|
3598
|
+
if not refs:
|
|
3599
|
+
errors.append("evidence.refs is empty (a candidate without evidence is a guess)")
|
|
3600
|
+
if conf not in ("high", "medium", "low"):
|
|
3601
|
+
errors.append("confidence %r (expected high|medium|low)" % conf)
|
|
3602
|
+
if etype == "inference" and conf != "low":
|
|
3603
|
+
errors.append("inference is ALWAYS low confidence (ADR-009); %r declared" % conf)
|
|
3604
|
+
return {"type": etype, "refs": refs, "confidence": conf}, errors
|
|
3605
|
+
|
|
3606
|
+
|
|
3607
|
+
def _resolve_ref(repo_path, ref):
|
|
3608
|
+
"""A ref must point at something REAL: `path`, `path:N`, `path:N-M` or `path#name`.
|
|
3609
|
+
Returns None when it resolves, else the reason."""
|
|
3610
|
+
frag = None
|
|
3611
|
+
if "#" in ref:
|
|
3612
|
+
ref, frag = ref.split("#", 1)
|
|
3613
|
+
span = None
|
|
3614
|
+
m = re.match(r"^(.*?):(\d+)(?:-(\d+))?$", ref)
|
|
3615
|
+
if m:
|
|
3616
|
+
ref = m.group(1)
|
|
3617
|
+
span = (int(m.group(2)), int(m.group(3) or m.group(2)))
|
|
3618
|
+
full = os.path.join(repo_path, ref.replace("/", os.sep))
|
|
3619
|
+
if _gc_rel(full, repo_path) is None:
|
|
3620
|
+
# an absolute path makes os.path.join DISCARD repo_path entirely, and ../ walks
|
|
3621
|
+
# out -- either way the "evidence" would point outside the legacy tree it claims
|
|
3622
|
+
# to evidence (fresh-review HIGH). Confinement is part of resolution.
|
|
3623
|
+
return "ref escapes the repo tree: %s" % ref
|
|
3624
|
+
if not os.path.isfile(full):
|
|
3625
|
+
return "file not found: %s" % ref
|
|
3626
|
+
if span or frag:
|
|
3627
|
+
try:
|
|
3628
|
+
with open(full, encoding="utf-8", errors="replace") as fh:
|
|
3629
|
+
body = fh.read()
|
|
3630
|
+
except OSError as exc:
|
|
3631
|
+
return "unreadable: %s" % exc
|
|
3632
|
+
if span:
|
|
3633
|
+
n = body.count("\n") + 1
|
|
3634
|
+
if span[0] < 1 or span[1] > n or span[0] > span[1]:
|
|
3635
|
+
return "lines %d-%d out of range (file has %d)" % (span[0], span[1], n)
|
|
3636
|
+
if frag and frag not in body:
|
|
3637
|
+
return "fragment %r not found in %s" % (frag, ref)
|
|
3638
|
+
return None
|
|
3639
|
+
|
|
3640
|
+
|
|
3641
|
+
def _load_behavior_ledger(path):
|
|
3642
|
+
"""Strict parse of the verdict table. Returns (rows, errors). Malformed is an ERROR,
|
|
3643
|
+
never a degrade: under the promotion gate, a silent "no verdicts" would UNBLOCK exactly
|
|
3644
|
+
what the gate guards (same posture as golden.scrub.json)."""
|
|
3645
|
+
rows, errors = [], []
|
|
3646
|
+
try:
|
|
3647
|
+
with open(path, encoding="utf-8", errors="replace") as fh:
|
|
3648
|
+
lines = fh.read().splitlines()
|
|
3649
|
+
except OSError as exc:
|
|
3650
|
+
return [], ["unreadable: %s" % exc]
|
|
3651
|
+
for n, ln in enumerate(lines, 1):
|
|
3652
|
+
s = ln.strip()
|
|
3653
|
+
if not s.startswith("|"):
|
|
3654
|
+
continue
|
|
3655
|
+
cells = [c.strip() for c in s.strip("|").split("|")]
|
|
3656
|
+
if cells and all(c and set(c) <= set("-: ") for c in cells):
|
|
3657
|
+
continue # separator row (empty cells are NOT)
|
|
3658
|
+
low = [c.lower() for c in cells]
|
|
3659
|
+
if "candidate" in low and "verdict" in low:
|
|
3660
|
+
continue # header row
|
|
3661
|
+
if len(cells) != 6:
|
|
3662
|
+
errors.append("line %d: %d cells, expected 6 (# | candidate | evidence | "
|
|
3663
|
+
"confidence | verdict | adr)" % (n, len(cells)))
|
|
3664
|
+
continue
|
|
3665
|
+
_, cand, _ev, _conf, verdict, adr = cells
|
|
3666
|
+
if verdict not in _BL_VERDICTS:
|
|
3667
|
+
errors.append("line %d: verdict %r is not one of %s -- a fourth state is "
|
|
3668
|
+
"malformation, not an option" % (n, verdict, "/".join(_BL_VERDICTS)))
|
|
3669
|
+
if not re.match(r"^ADR-\S+$", adr):
|
|
3670
|
+
errors.append("line %d: adr ref %r -- no verdict without its why (ADR-010)"
|
|
3671
|
+
% (n, adr))
|
|
3672
|
+
if not cand:
|
|
3673
|
+
errors.append("line %d: empty candidate" % n)
|
|
3674
|
+
rows.append({"candidate": cand, "verdict": verdict, "adr": adr, "line": n})
|
|
3675
|
+
return rows, errors
|
|
3676
|
+
|
|
3677
|
+
|
|
3678
|
+
def _bl_append_only(repo_path, rel):
|
|
3679
|
+
"""The rows in HEAD must be a byte-identical prefix of the working file. Deliberately
|
|
3680
|
+
blunt (ADR-010): an audit trail that tolerates rewriting is not an audit trail.
|
|
3681
|
+
Returns "ok" | "new" | "violation" | "unmeasured"."""
|
|
3682
|
+
try:
|
|
3683
|
+
# probe the repo FIRST: "git failed entirely" and "file not in HEAD yet" are
|
|
3684
|
+
# different answers, and conflating them turned no-git into a silent "new"
|
|
3685
|
+
# (caught by T120's AC-RD-05 -- unmeasured must never be mistaken for anything).
|
|
3686
|
+
probe = subprocess.run(["git", "rev-parse", "HEAD"], cwd=repo_path,
|
|
3687
|
+
capture_output=True)
|
|
3688
|
+
if probe.returncode != 0:
|
|
3689
|
+
return "unmeasured"
|
|
3690
|
+
r = subprocess.run(["git", "show", "HEAD:" + rel.replace(os.sep, "/")],
|
|
3691
|
+
cwd=repo_path, capture_output=True)
|
|
3692
|
+
except OSError:
|
|
3693
|
+
return "unmeasured"
|
|
3694
|
+
if r.returncode != 0:
|
|
3695
|
+
return "new" # not in HEAD yet
|
|
3696
|
+
try:
|
|
3697
|
+
with open(os.path.join(repo_path, rel), "rb") as fh:
|
|
3698
|
+
cur = fh.read()
|
|
3699
|
+
except OSError:
|
|
3700
|
+
return "violation" # in HEAD but gone from the tree
|
|
3701
|
+
# normalize line endings on BOTH sides: with core.autocrlf, git stores LF and checks out
|
|
3702
|
+
# CRLF, so a raw byte compare reads that translation as tampering on every Windows box
|
|
3703
|
+
# (found by the first probe). Git itself treats line endings as non-content; so do we.
|
|
3704
|
+
# A verdict edit still cannot hide in a CRLF flip.
|
|
3705
|
+
cur_n = cur.replace(b"\r\n", b"\n")
|
|
3706
|
+
head_n = r.stdout.replace(b"\r\n", b"\n")
|
|
3707
|
+
return "ok" if cur_n.startswith(head_n) else "violation"
|
|
3708
|
+
|
|
3709
|
+
|
|
3710
|
+
def _curation_state(repo_path):
|
|
3711
|
+
"""Everything the gate needs, from one scan. None = feature unused (no discovery/):
|
|
3712
|
+
behavior identical to a release where this code does not exist (AC-RD-07)."""
|
|
3713
|
+
disc = os.path.join(repo_path, CANDIDATE_DIR)
|
|
3714
|
+
if not os.path.isdir(disc):
|
|
3715
|
+
return None
|
|
3716
|
+
cands = sorted(f for f in os.listdir(disc) if f.lower().endswith(".md"))
|
|
3717
|
+
state = {"candidates": [], "malformed": [], "ledger_errors": [],
|
|
3718
|
+
"append_only": None, "unjudged": [], "promote_as_is": [],
|
|
3719
|
+
"promote_with_declared_divergence": [], "excluded": []}
|
|
3720
|
+
for f in cands:
|
|
3721
|
+
data, errs = _parse_candidate(os.path.join(disc, f))
|
|
3722
|
+
if data:
|
|
3723
|
+
for ref in data["refs"]:
|
|
3724
|
+
bad = _resolve_ref(repo_path, ref)
|
|
3725
|
+
if bad:
|
|
3726
|
+
errs.append("ref %r: %s" % (ref, bad))
|
|
3727
|
+
if errs:
|
|
3728
|
+
state["malformed"].append({"candidate": f, "errors": errs})
|
|
3729
|
+
else:
|
|
3730
|
+
state["candidates"].append(f)
|
|
3731
|
+
lpath = os.path.join(repo_path, BEHAVIOR_LEDGER_FILE)
|
|
3732
|
+
verdicts = {}
|
|
3733
|
+
if os.path.isfile(lpath):
|
|
3734
|
+
rows, lerrs = _load_behavior_ledger(lpath)
|
|
3735
|
+
state["ledger_errors"] = lerrs
|
|
3736
|
+
state["append_only"] = _bl_append_only(repo_path, BEHAVIOR_LEDGER_FILE)
|
|
3737
|
+
for row in rows:
|
|
3738
|
+
verdicts[row["candidate"]] = row["verdict"] # append-only: the LATEST row wins
|
|
3739
|
+
for f in state["candidates"]:
|
|
3740
|
+
v = verdicts.get(f)
|
|
3741
|
+
if v is None:
|
|
3742
|
+
state["unjudged"].append(f)
|
|
3743
|
+
elif v == "preserve":
|
|
3744
|
+
state["promote_as_is"].append(f)
|
|
3745
|
+
elif v == "fix":
|
|
3746
|
+
state["promote_with_declared_divergence"].append(f)
|
|
3747
|
+
else:
|
|
3748
|
+
state["excluded"].append(f)
|
|
3749
|
+
return state
|
|
3750
|
+
|
|
3751
|
+
|
|
3752
|
+
def cmd_curation_check(args):
|
|
3753
|
+
"""The INV-CURATION-01 gate, measured. Exit 2: malformation or tampering (config-error
|
|
3754
|
+
class -- candidates that cannot be validated, a ledger that cannot be trusted). Exit 1:
|
|
3755
|
+
valid candidates awaiting a human verdict (the quarantine holding). Exit 0: every
|
|
3756
|
+
candidate judged, or the feature unused."""
|
|
3757
|
+
ledger = _load(args.ledger)
|
|
3758
|
+
_repo_node(ledger, args.repo)
|
|
3759
|
+
repo_path = _repo_cfg(ledger, args.repo).get("path", ".")
|
|
3760
|
+
st = _curation_state(repo_path)
|
|
3761
|
+
if st is None:
|
|
3762
|
+
if args.json:
|
|
3763
|
+
print(json.dumps({"repo": args.repo, "in_use": False}))
|
|
3764
|
+
else:
|
|
3765
|
+
print("CURATION %s: no %s/ directory -- feature unused, nothing to gate."
|
|
3766
|
+
% (args.repo, CANDIDATE_DIR))
|
|
3767
|
+
sys.exit(0)
|
|
3768
|
+
out = dict(st)
|
|
3769
|
+
out.update({"repo": args.repo, "in_use": True})
|
|
3770
|
+
hard = bool(st["malformed"] or st["ledger_errors"]
|
|
3771
|
+
or st["append_only"] == "violation")
|
|
3772
|
+
if args.json:
|
|
3773
|
+
print(json.dumps(out, indent=2, ensure_ascii=False))
|
|
3774
|
+
else:
|
|
3775
|
+
print("CURATION %s: %d candidate(s), %d judged, %d awaiting verdict"
|
|
3776
|
+
% (args.repo, len(st["candidates"]) + len(st["malformed"]),
|
|
3777
|
+
len(st["promote_as_is"]) + len(st["promote_with_declared_divergence"])
|
|
3778
|
+
+ len(st["excluded"]), len(st["unjudged"])))
|
|
3779
|
+
for m in st["malformed"]:
|
|
3780
|
+
print(" !! %s: %s" % (m["candidate"], "; ".join(m["errors"])))
|
|
3781
|
+
for e in st["ledger_errors"]:
|
|
3782
|
+
print(" !! %s: %s" % (BEHAVIOR_LEDGER_FILE, e))
|
|
3783
|
+
if st["append_only"] == "violation":
|
|
3784
|
+
print(" !! %s: existing rows were EDITED -- append-only violated; revert and "
|
|
3785
|
+
"add a new row + ADR instead" % BEHAVIOR_LEDGER_FILE)
|
|
3786
|
+
elif st["append_only"] == "unmeasured":
|
|
3787
|
+
print(" -- append-only: UNMEASURED (no git) -- reported, never claimed as pass")
|
|
3788
|
+
for f in st["unjudged"]:
|
|
3789
|
+
print(" .. %s: awaiting human verdict (blocks forward)" % f)
|
|
3790
|
+
sys.exit(2 if hard else (1 if st["unjudged"] else 0))
|
|
3791
|
+
|
|
3792
|
+
|
|
3793
|
+
|
|
3358
3794
|
def cmd_escalate(args):
|
|
3359
3795
|
ledger = _load(args.ledger)
|
|
3360
3796
|
_repo_node(ledger, args.repo)
|
|
@@ -4401,6 +4837,9 @@ def cmd_dashboard(args):
|
|
|
4401
4837
|
_org[_rn] = _snaps[-1]["origin"]
|
|
4402
4838
|
if _org:
|
|
4403
4839
|
out["evidence_origin"] = _org
|
|
4840
|
+
if ledger.get(CLEAN_ROOM_KEY):
|
|
4841
|
+
out["clean_room"] = {r: [e for e in ledger[CLEAN_ROOM_KEY] if e.get("repo") == r][-1]
|
|
4842
|
+
for r in {e.get("repo") for e in ledger[CLEAN_ROOM_KEY]}}
|
|
4404
4843
|
if getattr(args, "json", False):
|
|
4405
4844
|
print(json.dumps(out, indent=2, ensure_ascii=False))
|
|
4406
4845
|
return
|
|
@@ -7105,6 +7544,24 @@ def build_parser():
|
|
|
7105
7544
|
pfp.add_argument("--json", action="store_true")
|
|
7106
7545
|
pfp.set_defaults(func=cmd_fastpath_eval)
|
|
7107
7546
|
|
|
7547
|
+
pcu = sub.add_parser("curation-check",
|
|
7548
|
+
help="the INV-CURATION-01 gate: candidates, verdicts, append-only ledger (ADR-009/010)")
|
|
7549
|
+
pcu.add_argument("--ledger", default="QA-LEDGER.json")
|
|
7550
|
+
pcu.add_argument("--repo", required=True)
|
|
7551
|
+
pcu.add_argument("--json", action="store_true")
|
|
7552
|
+
pcu.set_defaults(func=cmd_curation_check)
|
|
7553
|
+
|
|
7554
|
+
pcr = sub.add_parser("cleanroom",
|
|
7555
|
+
help="run a command against a CLEAN checkout of one commit in a throwaway worktree (ADR-008)")
|
|
7556
|
+
pcr.add_argument("--ledger", default="QA-LEDGER.json")
|
|
7557
|
+
pcr.add_argument("--repo", required=True)
|
|
7558
|
+
pcr.add_argument("--ref", default=None, help="commit to verify; default HEAD")
|
|
7559
|
+
pcr.add_argument("--run", required=True,
|
|
7560
|
+
help="the command to run inside the worktree; the engine never guesses it")
|
|
7561
|
+
pcr.add_argument("--setup", default=None, help="optional bootstrap before --run (e.g. npm ci)")
|
|
7562
|
+
pcr.add_argument("--json", action="store_true")
|
|
7563
|
+
pcr.set_defaults(func=cmd_cleanroom)
|
|
7564
|
+
|
|
7108
7565
|
pgc = sub.add_parser("golden-coverage",
|
|
7109
7566
|
help="record the MEASURED source files a golden's harness exercises (ADR-006)")
|
|
7110
7567
|
pgc.add_argument("--harness", required=True, help="script that drives the subject")
|