@codyswann/lisa 2.261.0 → 2.262.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/upstream-evidence-manifest.d.ts.map +1 -1
- package/dist/core/upstream-evidence-manifest.js +6 -2
- package/dist/core/upstream-evidence-manifest.js.map +1 -1
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-implement/SKILL.md +33 -3
- package/plugins/lisa/hooks/enforce-verification-gate.sh +168 -8
- package/plugins/lisa/skills/lisa-implement/SKILL.md +33 -3
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-implement/SKILL.md +33 -3
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/hooks/enforce-verification-gate.sh +168 -8
- package/plugins/lisa-copilot/skills/lisa-implement/SKILL.md +33 -3
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/hooks/enforce-verification-gate.sh +168 -8
- package/plugins/lisa-cursor/skills/lisa-implement/SKILL.md +33 -3
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/hooks/enforce-verification-gate.sh +168 -8
- package/plugins/src/base/skills/lisa-implement/SKILL.md +33 -3
package/package.json
CHANGED
|
@@ -105,7 +105,7 @@
|
|
|
105
105
|
"form-data": ">=4.0.6"
|
|
106
106
|
},
|
|
107
107
|
"name": "@codyswann/lisa",
|
|
108
|
-
"version": "2.
|
|
108
|
+
"version": "2.262.0",
|
|
109
109
|
"description": "Claude Code governance framework that applies guardrails, guidance, and automated enforcement to projects",
|
|
110
110
|
"main": "dist/index.js",
|
|
111
111
|
"exports": {
|
|
@@ -228,22 +228,52 @@ Before shutting down the team, execute the Verify flow:
|
|
|
228
228
|
|
|
229
229
|
1. Run quality gates: lint, typecheck, tests — all must pass. These are prerequisites, NOT verification.
|
|
230
230
|
2. `verification-specialist`: verify locally by running the actual system and observing results (empirical proof that the change works). This is the real verification step. For UI-surface bugs, the proof must observe the UI surface with browser/device automation against the target environment whenever such a harness exists; unit-level or API-only proof cannot satisfy the empirical verification contract for a UI-surface defect.
|
|
231
|
-
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json`
|
|
231
|
+
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json` in **schema v2**, which binds every claim to the *boundary* it asserts and to the evidence *kinds* that reach that boundary, per the `claim-evidence-mapping` rule:
|
|
232
232
|
|
|
233
233
|
```json
|
|
234
234
|
{
|
|
235
|
+
"schema_version": 2,
|
|
235
236
|
"plan": "<plan-name>",
|
|
236
|
-
"
|
|
237
|
+
"artifact": {
|
|
238
|
+
"repository": "<owner/repo>", "base_sha": "<sha>", "head_sha": "<sha of what will ship>",
|
|
239
|
+
"build_id": "<build/run id>", "environment": "<where it was observed>", "observed_at": "<ISO8601 UTC>"
|
|
240
|
+
},
|
|
241
|
+
"claims": [
|
|
242
|
+
{
|
|
243
|
+
"claim_id": "AC-1",
|
|
244
|
+
"statement": "<the claim, in the operator's language>",
|
|
245
|
+
"boundary": "code-unit | browser | http-api | cli | data | deploy-health | performance | standards-compat",
|
|
246
|
+
"required_for_gate": true,
|
|
247
|
+
"required_evidence_kinds": ["<kinds that reach this boundary, e.g. screenshot, recording>"],
|
|
248
|
+
"status": "established | not-established",
|
|
249
|
+
"evidence_refs": ["EV-1"],
|
|
250
|
+
"not_established": ["<what this claim does NOT cover>"]
|
|
251
|
+
}
|
|
252
|
+
],
|
|
253
|
+
"evidence": [
|
|
254
|
+
{
|
|
255
|
+
"evidence_id": "EV-1",
|
|
256
|
+
"kind": "screenshot | recording | http-transcript | cli-output | log-snippet | db-query-output | perf-trace | test-run-log | deploy-log | state-dump",
|
|
257
|
+
"locator": "evidence/<ticket>/<file>", "sha256": "<hash>",
|
|
258
|
+
"captured_at": "<ISO8601 UTC>", "artifact_head_sha": "<sha the artifact was captured at>"
|
|
259
|
+
}
|
|
260
|
+
],
|
|
261
|
+
"not_established_reviewed": true,
|
|
237
262
|
"criteria": [
|
|
238
263
|
{ "task": "<task id or title>", "criterion": "<the completion condition>", "status": "pass | fail | blocked", "evidence": "<the proof command run and the observed result; for a blocked criterion, the blocker diagnosis (e.g. the missing access and the probe that must pass)>" }
|
|
239
264
|
],
|
|
265
|
+
"status": "pass | fail | blocked | in_progress",
|
|
240
266
|
"updated_at": "<ISO8601 UTC>"
|
|
241
267
|
}
|
|
242
268
|
```
|
|
243
269
|
|
|
270
|
+
Rules for v2: a claim is established **only** by evidence whose `kind` reaches its `boundary` — a unit `test-run-log` reaches only `code-unit` and can never establish a `browser`, `http-api`, or `deploy-health` claim. `not_established_reviewed` must always be present (the `not_established` list may be empty, but the flag may never be omitted). `artifact.head_sha` names what will ship, and each evidence entry's `artifact_head_sha` must match it. The legacy `criteria[]` array is retained and still read, but under v2 it is **display-only** — it can never establish a v2 claim.
|
|
271
|
+
|
|
272
|
+
**v1 is still accepted during the compatibility window.** A verdict that omits `schema_version` (or sets it to `1`) carries only `plan` / `status` / `criteria[]` / `updated_at` and is judged exactly as before: terminal `status` plus no failing criterion plus freshness. Write v2 for new work; nothing in flight breaks.
|
|
273
|
+
|
|
244
274
|
Set `status: "pass"` only when every criterion is `pass` with real evidence (output from running the system, not a claim). The verdict must be judged by an agent that did NOT implement the change (the `verification-specialist`), never self-certified by the implementer. This is runtime scratch — it is gitignored and MUST NOT be committed (treat it like the secrets exclusion in the commit step).
|
|
245
275
|
|
|
246
|
-
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file and **will not let the flow stop** until it shows a terminal, all-`pass` verdict — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
276
|
+
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file — both v1 and v2 — and **will not let the flow stop** until it shows a terminal, all-`pass` verdict. The v2 claim/evidence checks are **advisory-first**: a boundary or identity violation is reported to stderr but does not block until `verification.gate.enforceBoundaries` is set to `true` in `.lisa.config.json` (default `false`, promoted via the threshold ratchet). Treat an advisory warning as a defect to fix now, not a warning to ignore — it becomes blocking on the ratchet. The gate — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
247
277
|
|
|
248
278
|
- **Actionable blocker** — an unresolved dependency or fixable technical gap that some team or repository could build (a missing or changed schema field, an unbuilt sibling work item, a required upstream fix), **including cross-repo dependencies**. Before writing the blocked verdict you MUST (1) file a build-ready fix/dependency ticket capturing the diagnosis — in the dependency's own repository/tracker when it is cross-repo (e.g. a `[<repo>] …` ticket in the shared project, or the sibling tracker) — and (2) link the current work item to it as `is blocked by`. Only then write the verdict. This is the same discipline as the regression-spec blocker and the remote-verification-fail exits above, and it is what makes the block machine-recoverable: `repair-intake` re-dispatches a blocked item once its linked `is blocked by` dependency closes, but it cannot act on a prose-only comment. Recommending the ticket "as a human follow-up" without filing and linking it is **not** a permitted exit.
|
|
249
279
|
- **Human-only blocker** — an input the agent genuinely cannot obtain or produce no matter what it does: credentials, secrets, or **tool access** it does not have (AWS/CloudWatch, Figma, Jam, Sentry, SonarCloud, a database, a protected deploy target, …), or a product/design decision only a human can make. For missing tool access, follow the `tool-access-gate` rule's break-out protocol: post the "Access Needed" comment naming the exact credential/role/env var to grant and the probe that must pass — never work around the gap by substituting weaker verification, mocking the inaccessible system, or narrowing scope. Record the blocked verdict, mark it `human_needed` (the marker `repair-intake` recognizes, so it won't churn re-dispatching it), and surface or reassign to a human; do **not** fabricate a build-ready ticket, because there is no build-ready work.
|
|
@@ -27,7 +27,33 @@
|
|
|
27
27
|
# by a per-session block counter so a genuinely-stuck flow
|
|
28
28
|
# ESCALATES instead of looping forever.
|
|
29
29
|
#
|
|
30
|
-
# The verdict artifact lives at "$CLAUDE_PROJECT_DIR/.lisa/verification-status.json"
|
|
30
|
+
# The verdict artifact lives at "$CLAUDE_PROJECT_DIR/.lisa/verification-status.json".
|
|
31
|
+
#
|
|
32
|
+
# SCHEMA v2 (current) binds every claim to a boundary and to the evidence kinds
|
|
33
|
+
# that reach it, per the claim-evidence-mapping contract:
|
|
34
|
+
# {
|
|
35
|
+
# "schema_version": 2,
|
|
36
|
+
# "plan": "<plan-name>",
|
|
37
|
+
# "artifact": { "repository": "owner/repo", "base_sha": "...", "head_sha": "...",
|
|
38
|
+
# "build_id": "...", "environment": "...", "observed_at": "<ISO8601 UTC>" },
|
|
39
|
+
# "claims": [
|
|
40
|
+
# { "claim_id": "AC-1", "statement": "...", "boundary": "browser",
|
|
41
|
+
# "required_for_gate": true, "required_evidence_kinds": ["screenshot","recording"],
|
|
42
|
+
# "status": "established" | "not-established", "evidence_refs": ["EV-1"],
|
|
43
|
+
# "not_established": ["<what this claim does NOT cover>"] }
|
|
44
|
+
# ],
|
|
45
|
+
# "evidence": [
|
|
46
|
+
# { "evidence_id": "EV-1", "kind": "screenshot", "locator": "evidence/1836/x.png",
|
|
47
|
+
# "sha256": "...", "captured_at": "<ISO8601 UTC>", "artifact_head_sha": "..." }
|
|
48
|
+
# ],
|
|
49
|
+
# "not_established_reviewed": true,
|
|
50
|
+
# "criteria": [ ... legacy, display-only ... ],
|
|
51
|
+
# "status": "pass" | "fail" | "blocked" | "in_progress",
|
|
52
|
+
# "updated_at": "<ISO8601 UTC>"
|
|
53
|
+
# }
|
|
54
|
+
#
|
|
55
|
+
# SCHEMA v1 (legacy, still accepted during the compatibility window) omits
|
|
56
|
+
# "schema_version" or sets it to 1, and carries only:
|
|
31
57
|
# {
|
|
32
58
|
# "plan": "<plan-name>",
|
|
33
59
|
# "status": "pass" | "fail" | "blocked" | "in_progress",
|
|
@@ -36,15 +62,39 @@
|
|
|
36
62
|
# ],
|
|
37
63
|
# "updated_at": "<ISO8601 UTC>"
|
|
38
64
|
# }
|
|
39
|
-
#
|
|
40
|
-
#
|
|
41
|
-
#
|
|
65
|
+
#
|
|
66
|
+
# In BOTH schemas, status "pass" (all criteria pass) or "blocked" (flow recorded
|
|
67
|
+
# a blocker and is stopping deliberately) are terminal and release the gate.
|
|
68
|
+
# "fail"/"in_progress" or a missing/stale file keep it closed.
|
|
69
|
+
#
|
|
70
|
+
# Compatibility window: the gate branches on "schema_version". Absent or 1 takes
|
|
71
|
+
# the v1 path unchanged — byte-for-byte the pre-v2 decision. 2 takes the v1 path
|
|
72
|
+
# PLUS the claim->evidence checks below. Legacy "criteria" is display-only under
|
|
73
|
+
# v2 and can never establish a v2 claim.
|
|
74
|
+
#
|
|
75
|
+
# v2 claim checks (only ever applied to an overall "pass"; a deliberate
|
|
76
|
+
# "blocked" stop is terminal on the v1 conditions alone):
|
|
77
|
+
# - every claim with "required_for_gate": true is "status": "established"
|
|
78
|
+
# - each such claim's "evidence_refs" resolve to "evidence[]" entries, at
|
|
79
|
+
# least one of whose "kind" is in the claim's "required_evidence_kinds"
|
|
80
|
+
# - "not_established_reviewed": true is present (the list may be empty, but
|
|
81
|
+
# the flag may never be omitted)
|
|
82
|
+
# - "artifact.head_sha" exists and no evidence entry declares a different
|
|
83
|
+
# "artifact_head_sha" (reconciliation with the MERGED head is BCE-4)
|
|
84
|
+
#
|
|
85
|
+
# ADVISORY-FIRST: those v2 checks report to stderr but do NOT block unless
|
|
86
|
+
# "verification.gate.enforceBoundaries" is true in .lisa.config.json (default
|
|
87
|
+
# false at ship, promoted via the threshold ratchet). While it is false, a v2
|
|
88
|
+
# verdict releases on exactly the v1-equivalent conditions.
|
|
42
89
|
#
|
|
43
90
|
# Per-session state lives under "$STATE_DIR" as flag files keyed by session_id.
|
|
44
91
|
# Stale state (>24h) is cleaned on each invocation.
|
|
45
92
|
#
|
|
46
|
-
# Fail-open: any unexpected jq parse failure or missing field
|
|
47
|
-
#
|
|
93
|
+
# Fail-open: any unexpected jq parse failure or missing field degrades to the
|
|
94
|
+
# LESS strict outcome rather than inventing a new hard failure, and the
|
|
95
|
+
# MAX_BLOCKS escalation below guarantees the gate always releases eventually. A
|
|
96
|
+
# broken gate must never brick a session. In particular, a v2 verdict whose
|
|
97
|
+
# claim structure cannot be evaluated is judged on the v1 conditions alone.
|
|
48
98
|
|
|
49
99
|
set -uo pipefail
|
|
50
100
|
|
|
@@ -137,9 +187,14 @@ fi
|
|
|
137
187
|
PROJECT_DIR="${CLAUDE_PROJECT_DIR:-.}"
|
|
138
188
|
VERDICT_FILE="${PROJECT_DIR}/.lisa/verification-status.json"
|
|
139
189
|
|
|
190
|
+
# Set by the v2 path when a claim/evidence violation is what closed the gate,
|
|
191
|
+
# so the block message can state the real reason instead of the v1 fallback.
|
|
192
|
+
V2_BLOCK_REASON=""
|
|
193
|
+
|
|
140
194
|
# A terminal verdict (pass or blocked) with no failing criterion, written AFTER
|
|
141
|
-
# the flow was armed, releases the gate.
|
|
142
|
-
|
|
195
|
+
# the flow was armed, releases the gate. This is the v1 decision, unchanged, and
|
|
196
|
+
# it remains the floor for v2 as well.
|
|
197
|
+
verdict_is_terminal_v1() {
|
|
143
198
|
[ -f "$VERDICT_FILE" ] || return 1
|
|
144
199
|
|
|
145
200
|
local status fails
|
|
@@ -160,6 +215,101 @@ verdict_is_terminal() {
|
|
|
160
215
|
return 0
|
|
161
216
|
}
|
|
162
217
|
|
|
218
|
+
# True when the project has ratcheted the v2 claim checks from advisory to
|
|
219
|
+
# blocking. Default false: a missing/unreadable config is advisory-only.
|
|
220
|
+
boundary_enforcement_enabled() {
|
|
221
|
+
local config_file value
|
|
222
|
+
config_file="${PROJECT_DIR}/.lisa.config.json"
|
|
223
|
+
[ -f "$config_file" ] || return 1
|
|
224
|
+
value=$(jq -r '.verification.gate.enforceBoundaries // false' "$config_file" 2>/dev/null || printf 'false')
|
|
225
|
+
[ "$value" = "true" ]
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
# Emits one line per v2 claim->evidence contract violation. Empty output means
|
|
229
|
+
# the verdict satisfies the contract (or could not be evaluated, which degrades
|
|
230
|
+
# to the v1 decision rather than to a new hard failure).
|
|
231
|
+
v2_contract_violations() {
|
|
232
|
+
jq -r '
|
|
233
|
+
. as $v
|
|
234
|
+
| ($v.evidence // []) as $ev
|
|
235
|
+
| (($v.artifact // {}).head_sha // "") as $head
|
|
236
|
+
| [
|
|
237
|
+
(if ($v.not_established_reviewed == true) then empty
|
|
238
|
+
else "not_established_reviewed is absent or not true - the flag may never be omitted" end),
|
|
239
|
+
(if ($head | length) > 0 then empty
|
|
240
|
+
else "artifact.head_sha is missing - required for a v2 pass" end),
|
|
241
|
+
( $ev[]
|
|
242
|
+
| select(($head | length) > 0)
|
|
243
|
+
| select(((.artifact_head_sha // "") | length) > 0)
|
|
244
|
+
| select(.artifact_head_sha != $head)
|
|
245
|
+
| "evidence \(.evidence_id // "?") was captured at \(.artifact_head_sha) but artifact.head_sha is \($head)" ),
|
|
246
|
+
( ($v.claims // [])[]
|
|
247
|
+
| select((.required_for_gate // false) == true)
|
|
248
|
+
| . as $c
|
|
249
|
+
| ($c.claim_id // "?") as $cid
|
|
250
|
+
| ($c.boundary // "?") as $bnd
|
|
251
|
+
| ($c.required_evidence_kinds // []) as $req
|
|
252
|
+
| ($req | join(", ")) as $reqs
|
|
253
|
+
| ([ ($c.evidence_refs // [])[] as $r | $ev[] | select((.evidence_id // "") == $r) ]) as $res
|
|
254
|
+
| (
|
|
255
|
+
(if (($c.status // "") == "established") then empty
|
|
256
|
+
else "claim \($cid) [boundary \($bnd)] is not established (status: \($c.status // "missing"); required kinds: \($reqs))" end),
|
|
257
|
+
(if ($res | length) == 0
|
|
258
|
+
then "claim \($cid) [boundary \($bnd)] cites no resolvable evidence (required kinds: \($reqs))"
|
|
259
|
+
elif ([ $res[].kind // "" ] | map(select(. as $k | $req | index($k))) | length) == 0
|
|
260
|
+
then "claim \($cid) [boundary \($bnd)] cited evidence kinds [\([$res[].kind // "?"] | join(", "))] do not reach the boundary (required kinds: \($reqs))"
|
|
261
|
+
else empty end)
|
|
262
|
+
)
|
|
263
|
+
)
|
|
264
|
+
]
|
|
265
|
+
| .[]
|
|
266
|
+
' "$VERDICT_FILE" 2>/dev/null || true
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
# v2 = the v1 decision PLUS the claim->evidence contract, the latter advisory
|
|
270
|
+
# until verification.gate.enforceBoundaries is ratcheted on.
|
|
271
|
+
verdict_is_terminal_v2() {
|
|
272
|
+
verdict_is_terminal_v1 || return 1
|
|
273
|
+
|
|
274
|
+
# Only a "pass" asserts that claims are established. A deliberate "blocked"
|
|
275
|
+
# stop records an outcome and is terminal on the v1 conditions alone.
|
|
276
|
+
local status violations
|
|
277
|
+
status=$(jq -r '.status // empty' "$VERDICT_FILE" 2>/dev/null || true)
|
|
278
|
+
[ "$status" = "pass" ] || return 0
|
|
279
|
+
|
|
280
|
+
violations=$(v2_contract_violations)
|
|
281
|
+
[ -n "$violations" ] || return 0
|
|
282
|
+
|
|
283
|
+
if boundary_enforcement_enabled; then
|
|
284
|
+
# Hand the diagnosis to the block message below so the operator reads one
|
|
285
|
+
# coherent reason instead of this plus a generic v1-shaped fallback.
|
|
286
|
+
V2_BLOCK_REASON=$(printf '%s\n' "$violations" | sed 's/^/ - /')
|
|
287
|
+
return 1
|
|
288
|
+
fi
|
|
289
|
+
|
|
290
|
+
{
|
|
291
|
+
echo "Verification gate (advisory): the v2 claim/evidence contract is not"
|
|
292
|
+
echo "satisfied. Releasing anyway because verification.gate.enforceBoundaries"
|
|
293
|
+
echo "is false. These become blocking when the flag is ratcheted on:"
|
|
294
|
+
printf '%s\n' "$violations" | sed 's/^/ - /'
|
|
295
|
+
} >&2
|
|
296
|
+
return 0
|
|
297
|
+
}
|
|
298
|
+
|
|
299
|
+
# Compatibility window: branch on schema_version. Absent or 1 -> the v1 decision
|
|
300
|
+
# unchanged; 2 -> v1 plus the claim->evidence contract. An unrecognized or
|
|
301
|
+
# unparseable value degrades to v1 rather than to a new failure mode.
|
|
302
|
+
verdict_is_terminal() {
|
|
303
|
+
[ -f "$VERDICT_FILE" ] || return 1
|
|
304
|
+
|
|
305
|
+
local schema_version
|
|
306
|
+
schema_version=$(jq -r '.schema_version // empty' "$VERDICT_FILE" 2>/dev/null || true)
|
|
307
|
+
case "$schema_version" in
|
|
308
|
+
2) verdict_is_terminal_v2 ;;
|
|
309
|
+
*) verdict_is_terminal_v1 ;;
|
|
310
|
+
esac
|
|
311
|
+
}
|
|
312
|
+
|
|
163
313
|
if verdict_is_terminal; then
|
|
164
314
|
# Gate satisfied — disarm so a follow-up stop in the same session is not
|
|
165
315
|
# re-gated against the now-consumed verdict, and allow the stop.
|
|
@@ -205,6 +355,16 @@ fi
|
|
|
205
355
|
echo "failed, base branch missing, unresolved dependency), write the verdict"
|
|
206
356
|
echo "with status \"blocked\" and the reason instead. That records the"
|
|
207
357
|
echo "outcome and releases this gate."
|
|
358
|
+
elif [ -n "$V2_BLOCK_REASON" ]; then
|
|
359
|
+
echo "The verdict claims to pass, but its evidence does not establish every"
|
|
360
|
+
echo "claim the gate requires:"
|
|
361
|
+
printf '%s\n' "$V2_BLOCK_REASON"
|
|
362
|
+
echo
|
|
363
|
+
echo "A claim counts only when the evidence cited for it is of a kind that"
|
|
364
|
+
echo "reaches that claim's boundary — a unit test log does not prove a"
|
|
365
|
+
echo "button works in a browser. Capture the reaching evidence and"
|
|
366
|
+
echo "re-verify, or — if genuinely blocked — set status \"blocked\" with the"
|
|
367
|
+
echo "reason."
|
|
208
368
|
else
|
|
209
369
|
echo "The verification verdict is not terminal-and-passing. Outstanding:"
|
|
210
370
|
if [ -n "$REASON_DETAIL" ]; then
|
|
@@ -228,22 +228,52 @@ Before shutting down the team, execute the Verify flow:
|
|
|
228
228
|
|
|
229
229
|
1. Run quality gates: lint, typecheck, tests — all must pass. These are prerequisites, NOT verification.
|
|
230
230
|
2. `verification-specialist`: verify locally by running the actual system and observing results (empirical proof that the change works). This is the real verification step. For UI-surface bugs, the proof must observe the UI surface with browser/device automation against the target environment whenever such a harness exists; unit-level or API-only proof cannot satisfy the empirical verification contract for a UI-surface defect.
|
|
231
|
-
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json`
|
|
231
|
+
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json` in **schema v2**, which binds every claim to the *boundary* it asserts and to the evidence *kinds* that reach that boundary, per the `claim-evidence-mapping` rule:
|
|
232
232
|
|
|
233
233
|
```json
|
|
234
234
|
{
|
|
235
|
+
"schema_version": 2,
|
|
235
236
|
"plan": "<plan-name>",
|
|
236
|
-
"
|
|
237
|
+
"artifact": {
|
|
238
|
+
"repository": "<owner/repo>", "base_sha": "<sha>", "head_sha": "<sha of what will ship>",
|
|
239
|
+
"build_id": "<build/run id>", "environment": "<where it was observed>", "observed_at": "<ISO8601 UTC>"
|
|
240
|
+
},
|
|
241
|
+
"claims": [
|
|
242
|
+
{
|
|
243
|
+
"claim_id": "AC-1",
|
|
244
|
+
"statement": "<the claim, in the operator's language>",
|
|
245
|
+
"boundary": "code-unit | browser | http-api | cli | data | deploy-health | performance | standards-compat",
|
|
246
|
+
"required_for_gate": true,
|
|
247
|
+
"required_evidence_kinds": ["<kinds that reach this boundary, e.g. screenshot, recording>"],
|
|
248
|
+
"status": "established | not-established",
|
|
249
|
+
"evidence_refs": ["EV-1"],
|
|
250
|
+
"not_established": ["<what this claim does NOT cover>"]
|
|
251
|
+
}
|
|
252
|
+
],
|
|
253
|
+
"evidence": [
|
|
254
|
+
{
|
|
255
|
+
"evidence_id": "EV-1",
|
|
256
|
+
"kind": "screenshot | recording | http-transcript | cli-output | log-snippet | db-query-output | perf-trace | test-run-log | deploy-log | state-dump",
|
|
257
|
+
"locator": "evidence/<ticket>/<file>", "sha256": "<hash>",
|
|
258
|
+
"captured_at": "<ISO8601 UTC>", "artifact_head_sha": "<sha the artifact was captured at>"
|
|
259
|
+
}
|
|
260
|
+
],
|
|
261
|
+
"not_established_reviewed": true,
|
|
237
262
|
"criteria": [
|
|
238
263
|
{ "task": "<task id or title>", "criterion": "<the completion condition>", "status": "pass | fail | blocked", "evidence": "<the proof command run and the observed result; for a blocked criterion, the blocker diagnosis (e.g. the missing access and the probe that must pass)>" }
|
|
239
264
|
],
|
|
265
|
+
"status": "pass | fail | blocked | in_progress",
|
|
240
266
|
"updated_at": "<ISO8601 UTC>"
|
|
241
267
|
}
|
|
242
268
|
```
|
|
243
269
|
|
|
270
|
+
Rules for v2: a claim is established **only** by evidence whose `kind` reaches its `boundary` — a unit `test-run-log` reaches only `code-unit` and can never establish a `browser`, `http-api`, or `deploy-health` claim. `not_established_reviewed` must always be present (the `not_established` list may be empty, but the flag may never be omitted). `artifact.head_sha` names what will ship, and each evidence entry's `artifact_head_sha` must match it. The legacy `criteria[]` array is retained and still read, but under v2 it is **display-only** — it can never establish a v2 claim.
|
|
271
|
+
|
|
272
|
+
**v1 is still accepted during the compatibility window.** A verdict that omits `schema_version` (or sets it to `1`) carries only `plan` / `status` / `criteria[]` / `updated_at` and is judged exactly as before: terminal `status` plus no failing criterion plus freshness. Write v2 for new work; nothing in flight breaks.
|
|
273
|
+
|
|
244
274
|
Set `status: "pass"` only when every criterion is `pass` with real evidence (output from running the system, not a claim). The verdict must be judged by an agent that did NOT implement the change (the `verification-specialist`), never self-certified by the implementer. This is runtime scratch — it is gitignored and MUST NOT be committed (treat it like the secrets exclusion in the commit step).
|
|
245
275
|
|
|
246
|
-
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file and **will not let the flow stop** until it shows a terminal, all-`pass` verdict — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
276
|
+
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file — both v1 and v2 — and **will not let the flow stop** until it shows a terminal, all-`pass` verdict. The v2 claim/evidence checks are **advisory-first**: a boundary or identity violation is reported to stderr but does not block until `verification.gate.enforceBoundaries` is set to `true` in `.lisa.config.json` (default `false`, promoted via the threshold ratchet). Treat an advisory warning as a defect to fix now, not a warning to ignore — it becomes blocking on the ratchet. The gate — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
247
277
|
|
|
248
278
|
- **Actionable blocker** — an unresolved dependency or fixable technical gap that some team or repository could build (a missing or changed schema field, an unbuilt sibling work item, a required upstream fix), **including cross-repo dependencies**. Before writing the blocked verdict you MUST (1) file a build-ready fix/dependency ticket capturing the diagnosis — in the dependency's own repository/tracker when it is cross-repo (e.g. a `[<repo>] …` ticket in the shared project, or the sibling tracker) — and (2) link the current work item to it as `is blocked by`. Only then write the verdict. This is the same discipline as the regression-spec blocker and the remote-verification-fail exits above, and it is what makes the block machine-recoverable: `repair-intake` re-dispatches a blocked item once its linked `is blocked by` dependency closes, but it cannot act on a prose-only comment. Recommending the ticket "as a human follow-up" without filing and linking it is **not** a permitted exit.
|
|
249
279
|
- **Human-only blocker** — an input the agent genuinely cannot obtain or produce no matter what it does: credentials, secrets, or **tool access** it does not have (AWS/CloudWatch, Figma, Jam, Sentry, SonarCloud, a database, a protected deploy target, …), or a product/design decision only a human can make. For missing tool access, follow the `tool-access-gate` rule's break-out protocol: post the "Access Needed" comment naming the exact credential/role/env var to grant and the probe that must pass — never work around the gap by substituting weaker verification, mocking the inaccessible system, or narrowing scope. Record the blocked verdict, mark it `human_needed` (the marker `repair-intake` recognizes, so it won't churn re-dispatching it), and surface or reassign to a human; do **not** fabricate a build-ready ticket, because there is no build-ready work.
|
|
@@ -228,22 +228,52 @@ Before shutting down the team, execute the Verify flow:
|
|
|
228
228
|
|
|
229
229
|
1. Run quality gates: lint, typecheck, tests — all must pass. These are prerequisites, NOT verification.
|
|
230
230
|
2. `verification-specialist`: verify locally by running the actual system and observing results (empirical proof that the change works). This is the real verification step. For UI-surface bugs, the proof must observe the UI surface with browser/device automation against the target environment whenever such a harness exists; unit-level or API-only proof cannot satisfy the empirical verification contract for a UI-surface defect.
|
|
231
|
-
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json`
|
|
231
|
+
2a. **Record the verification verdict** — the independent, machine-readable proof that gates completion. The `verification-specialist` writes `${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json` in **schema v2**, which binds every claim to the *boundary* it asserts and to the evidence *kinds* that reach that boundary, per the `claim-evidence-mapping` rule:
|
|
232
232
|
|
|
233
233
|
```json
|
|
234
234
|
{
|
|
235
|
+
"schema_version": 2,
|
|
235
236
|
"plan": "<plan-name>",
|
|
236
|
-
"
|
|
237
|
+
"artifact": {
|
|
238
|
+
"repository": "<owner/repo>", "base_sha": "<sha>", "head_sha": "<sha of what will ship>",
|
|
239
|
+
"build_id": "<build/run id>", "environment": "<where it was observed>", "observed_at": "<ISO8601 UTC>"
|
|
240
|
+
},
|
|
241
|
+
"claims": [
|
|
242
|
+
{
|
|
243
|
+
"claim_id": "AC-1",
|
|
244
|
+
"statement": "<the claim, in the operator's language>",
|
|
245
|
+
"boundary": "code-unit | browser | http-api | cli | data | deploy-health | performance | standards-compat",
|
|
246
|
+
"required_for_gate": true,
|
|
247
|
+
"required_evidence_kinds": ["<kinds that reach this boundary, e.g. screenshot, recording>"],
|
|
248
|
+
"status": "established | not-established",
|
|
249
|
+
"evidence_refs": ["EV-1"],
|
|
250
|
+
"not_established": ["<what this claim does NOT cover>"]
|
|
251
|
+
}
|
|
252
|
+
],
|
|
253
|
+
"evidence": [
|
|
254
|
+
{
|
|
255
|
+
"evidence_id": "EV-1",
|
|
256
|
+
"kind": "screenshot | recording | http-transcript | cli-output | log-snippet | db-query-output | perf-trace | test-run-log | deploy-log | state-dump",
|
|
257
|
+
"locator": "evidence/<ticket>/<file>", "sha256": "<hash>",
|
|
258
|
+
"captured_at": "<ISO8601 UTC>", "artifact_head_sha": "<sha the artifact was captured at>"
|
|
259
|
+
}
|
|
260
|
+
],
|
|
261
|
+
"not_established_reviewed": true,
|
|
237
262
|
"criteria": [
|
|
238
263
|
{ "task": "<task id or title>", "criterion": "<the completion condition>", "status": "pass | fail | blocked", "evidence": "<the proof command run and the observed result; for a blocked criterion, the blocker diagnosis (e.g. the missing access and the probe that must pass)>" }
|
|
239
264
|
],
|
|
265
|
+
"status": "pass | fail | blocked | in_progress",
|
|
240
266
|
"updated_at": "<ISO8601 UTC>"
|
|
241
267
|
}
|
|
242
268
|
```
|
|
243
269
|
|
|
270
|
+
Rules for v2: a claim is established **only** by evidence whose `kind` reaches its `boundary` — a unit `test-run-log` reaches only `code-unit` and can never establish a `browser`, `http-api`, or `deploy-health` claim. `not_established_reviewed` must always be present (the `not_established` list may be empty, but the flag may never be omitted). `artifact.head_sha` names what will ship, and each evidence entry's `artifact_head_sha` must match it. The legacy `criteria[]` array is retained and still read, but under v2 it is **display-only** — it can never establish a v2 claim.
|
|
271
|
+
|
|
272
|
+
**v1 is still accepted during the compatibility window.** A verdict that omits `schema_version` (or sets it to `1`) carries only `plan` / `status` / `criteria[]` / `updated_at` and is judged exactly as before: terminal `status` plus no failing criterion plus freshness. Write v2 for new work; nothing in flight breaks.
|
|
273
|
+
|
|
244
274
|
Set `status: "pass"` only when every criterion is `pass` with real evidence (output from running the system, not a claim). The verdict must be judged by an agent that did NOT implement the change (the `verification-specialist`), never self-certified by the implementer. This is runtime scratch — it is gitignored and MUST NOT be committed (treat it like the secrets exclusion in the commit step).
|
|
245
275
|
|
|
246
|
-
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file and **will not let the flow stop** until it shows a terminal, all-`pass` verdict — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
276
|
+
On Claude, the `enforce-verification-gate.sh` Stop hook reads this file — both v1 and v2 — and **will not let the flow stop** until it shows a terminal, all-`pass` verdict. The v2 claim/evidence checks are **advisory-first**: a boundary or identity violation is reported to stderr but does not block until `verification.gate.enforceBoundaries` is set to `true` in `.lisa.config.json` (default `false`, promoted via the threshold ratchet). Treat an advisory warning as a defect to fix now, not a warning to ignore — it becomes blocking on the ratchet. The gate — carrying over the non-bypassable completion gate of the `/goal` primitive, but checked deterministically against real evidence rather than by a transcript-only evaluator model. If you must stop before completion, write the verdict with `status: "blocked"` and the reason — marking each criterion whose proof is blocked as `status: "blocked"` with the blocker diagnosis as its `evidence`, while unaffected criteria keep their real `pass`/`fail` result — that records the outcome and releases the gate instead of leaving it to spin. But a `blocked` verdict is a last resort, not a shortcut around fillable work: **first resolve every gap you can resolve yourself.** If the work item is thin — missing its Validation Journey, acceptance criteria, or other derivable detail — enrich it: derive the missing detail from the ticket context and the codebase, write it back, and proceed. Do **not** block on a gap you could have filled. Only a blocker that survives that attempt is real, and it is one of two kinds:
|
|
247
277
|
|
|
248
278
|
- **Actionable blocker** — an unresolved dependency or fixable technical gap that some team or repository could build (a missing or changed schema field, an unbuilt sibling work item, a required upstream fix), **including cross-repo dependencies**. Before writing the blocked verdict you MUST (1) file a build-ready fix/dependency ticket capturing the diagnosis — in the dependency's own repository/tracker when it is cross-repo (e.g. a `[<repo>] …` ticket in the shared project, or the sibling tracker) — and (2) link the current work item to it as `is blocked by`. Only then write the verdict. This is the same discipline as the regression-spec blocker and the remote-verification-fail exits above, and it is what makes the block machine-recoverable: `repair-intake` re-dispatches a blocked item once its linked `is blocked by` dependency closes, but it cannot act on a prose-only comment. Recommending the ticket "as a human follow-up" without filing and linking it is **not** a permitted exit.
|
|
249
279
|
- **Human-only blocker** — an input the agent genuinely cannot obtain or produce no matter what it does: credentials, secrets, or **tool access** it does not have (AWS/CloudWatch, Figma, Jam, Sentry, SonarCloud, a database, a protected deploy target, …), or a product/design decision only a human can make. For missing tool access, follow the `tool-access-gate` rule's break-out protocol: post the "Access Needed" comment naming the exact credential/role/env var to grant and the probe that must pass — never work around the gap by substituting weaker verification, mocking the inaccessible system, or narrowing scope. Record the blocked verdict, mark it `human_needed` (the marker `repair-intake` recognizes, so it won't churn re-dispatching it), and surface or reassign to a human; do **not** fabricate a build-ready ticket, because there is no build-ready work.
|