@codyswann/lisa 2.266.0 → 2.268.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/upstream-evidence-manifest.d.ts.map +1 -1
- package/dist/core/upstream-evidence-manifest.js +29 -6
- package/dist/core/upstream-evidence-manifest.js.map +1 -1
- package/package.json +5 -3
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-spec-conformance/SKILL.md +43 -11
- package/plugins/lisa/agents/spec-conformance-specialist.md +5 -2
- package/plugins/lisa/agents/verification-specialist.md +1 -1
- package/plugins/lisa/hooks/enforce-verification-gate.sh +83 -9
- package/plugins/lisa/rules/eager/readiness-rubric.md +58 -0
- package/plugins/lisa/rules/reference/readiness-rubric.md +179 -0
- package/plugins/lisa/rules/reference/verification.md +2 -0
- package/plugins/lisa/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/lisa/skills/lisa-spec-conformance/SKILL.md +43 -11
- package/plugins/lisa-agy/agents/spec-conformance-specialist.md +5 -2
- package/plugins/lisa-agy/agents/verification-specialist.md +1 -1
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/lisa-agy/skills/lisa-spec-conformance/SKILL.md +43 -11
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/agents/spec-conformance-specialist.agent.md +5 -2
- package/plugins/lisa-copilot/agents/verification-specialist.agent.md +1 -1
- package/plugins/lisa-copilot/hooks/enforce-verification-gate.sh +83 -9
- package/plugins/lisa-copilot/rules/eager/readiness-rubric.md +58 -0
- package/plugins/lisa-copilot/rules/reference/readiness-rubric.md +179 -0
- package/plugins/lisa-copilot/rules/reference/verification.md +2 -0
- package/plugins/lisa-copilot/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/lisa-copilot/skills/lisa-spec-conformance/SKILL.md +43 -11
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/agents/spec-conformance-specialist.md +5 -2
- package/plugins/lisa-cursor/agents/verification-specialist.md +1 -1
- package/plugins/lisa-cursor/hooks/enforce-verification-gate.sh +83 -9
- package/plugins/lisa-cursor/rules/readiness-rubric-reference.mdc +184 -0
- package/plugins/lisa-cursor/rules/readiness-rubric.mdc +63 -0
- package/plugins/lisa-cursor/rules/verification-reference.mdc +2 -0
- package/plugins/lisa-cursor/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/lisa-cursor/skills/lisa-spec-conformance/SKILL.md +43 -11
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/agents/spec-conformance-specialist.md +5 -2
- package/plugins/src/base/agents/verification-specialist.md +1 -1
- package/plugins/src/base/hooks/enforce-verification-gate.sh +83 -9
- package/plugins/src/base/rules/eager/readiness-rubric.md +58 -0
- package/plugins/src/base/rules/reference/readiness-rubric.md +179 -0
- package/plugins/src/base/rules/reference/verification.md +2 -0
- package/plugins/src/base/skills/lisa-implement/SKILL.md +22 -1
- package/plugins/src/base/skills/lisa-spec-conformance/SKILL.md +43 -11
|
@@ -30,8 +30,9 @@ Follow the `spec-conformance` skill end-to-end:
|
|
|
30
30
|
|
|
31
31
|
1. Resolve the spec source (plan file, JIRA key, Linear, GitHub issue, PRD).
|
|
32
32
|
2. Extract every requirement into a structured list — acceptance criteria, Out of Scope, technical commitments, Validation Journey assertions, deliverables.
|
|
33
|
-
3. Inspect shipped work (diff, tests, PR body, verification-specialist evidence).
|
|
34
|
-
|
|
33
|
+
3. Inspect shipped work (diff, tests, PR body, verification-specialist evidence) **and load the machine-readable verdict** at `.lisa/verification-status.json`.
|
|
34
|
+
3b. Cross-check every claim against its **boundary** — the `claim-evidence-mapping` contract's taxonomy — plus artifact identity and the Not-established review.
|
|
35
|
+
4. Build the coverage matrix — every requirement gets a row with a boundary, an evidence kind, and a status.
|
|
35
36
|
5. Detect scope creep and untraceable changes separately.
|
|
36
37
|
6. Produce the verdict.
|
|
37
38
|
|
|
@@ -42,6 +43,8 @@ Return the structured report defined in the skill. Never summarize or drop rows.
|
|
|
42
43
|
## Rules
|
|
43
44
|
|
|
44
45
|
- **Require empirical evidence.** A requirement is not `MATCH` because code exists. It is `MATCH` only when there is a test AND runtime observation (captured by verification-specialist).
|
|
46
|
+
- **A cited-evidence-boundary mismatch is a conformance finding.** The `claim-evidence-mapping` rule binds every claim to a boundary and every boundary to the evidence kinds that reach it. When a v2 verdict cites evidence whose `kind` does not reach the claim's `boundary` — a unit `test-run-log` for a `browser` claim — or whose `artifact_head_sha` does not match `artifact.head_sha`, or when the verdict omits the Not-established review, the row is `BOUNDARY_MISMATCH` and the verdict is `DIVERGES`. Name the boundary, the kind cited, and the kind(s) required. This is not the same failure as a miss: the work may be correct and the proof still not establish it.
|
|
47
|
+
- **Degrade, never block, on an absent or v1 verdict.** If no v2 verdict exists, say so in the report, cap affected rows at `PARTIAL`, and do not invent a mismatch you could not check.
|
|
45
48
|
- **Scope creep is a distinct failure.** Do not fold `SCOPE_CREEP_VIOLATION` into "missing" or "untraceable." Scope creep means Out of Scope was violated — it blocks shipping.
|
|
46
49
|
- **Untraceable changes get surfaced, not judged.** Refactors and test helpers often land here. Surface them so the human can confirm intent; do not automatically fail.
|
|
47
50
|
- **If the spec itself is inadequate** (no acceptance criteria, no Out of Scope, no Validation Journey for runtime changes), the verdict is `DIVERGES` until the spec is tightened. Do not paper over an ambiguous spec with a generous match.
|
|
@@ -12,7 +12,7 @@ skills:
|
|
|
12
12
|
|
|
13
13
|
You are a verification specialist. Your job is to **prove empirically** that work is done -- not by reading code, but by running the actual system and observing the results.
|
|
14
14
|
|
|
15
|
-
Read `.claude/rules/verification.md` at the start of every investigation for the full verification framework, types, and lifecycle.
|
|
15
|
+
Read `.claude/rules/verification.md` at the start of every investigation for the full verification framework, types, and lifecycle. Read `.claude/rules/claim-evidence-mapping.md` alongside it: it binds every claim to the **boundary** it asserts and every boundary to the evidence **kinds** that reach it. The verdict you write is what `spec-conformance-specialist` cross-checks — record each claim's `boundary`, its `required_evidence_kinds`, its `evidence_refs`, and its `not_established` list so a boundary mismatch is catchable rather than invisible.
|
|
16
16
|
|
|
17
17
|
## Core Philosophy
|
|
18
18
|
|
|
@@ -81,6 +81,20 @@
|
|
|
81
81
|
# the flag may never be omitted)
|
|
82
82
|
# - "artifact.head_sha" exists and no evidence entry declares a different
|
|
83
83
|
# "artifact_head_sha" (reconciliation with the MERGED head is BCE-4)
|
|
84
|
+
# - every evidence entry that records BOTH a "sha256" and a "locator"
|
|
85
|
+
# resolving to a file on disk still hashes to that digest
|
|
86
|
+
# ("evidence_digest_mismatch"). A locator that is not on disk at stop time
|
|
87
|
+
# is NOT judged here: the Stop hook sees only the working tree, and evidence
|
|
88
|
+
# may legitimately live outside it. The absent-artifact arm of that check
|
|
89
|
+
# belongs to the read-side review surfaces, which see the committed
|
|
90
|
+
# evidence directory.
|
|
91
|
+
# - the claim/evidence structure is EVALUABLE at all: a v2 verdict whose
|
|
92
|
+
# "claims"/"evidence"/"artifact" are not the shapes the schema defines
|
|
93
|
+
# (claims as a string, evidence as a scalar) is reported as
|
|
94
|
+
# could-not-evaluate. Could-not-evaluate is NOT the same as no-violations:
|
|
95
|
+
# a verdict the gate cannot read may never be treated as a clean one, or a
|
|
96
|
+
# structurally-wrong verdict would sail past the gate the moment
|
|
97
|
+
# enforcement is ratcheted on.
|
|
84
98
|
#
|
|
85
99
|
# ADVISORY-FIRST: those v2 checks report to stderr but do NOT block unless
|
|
86
100
|
# "verification.gate.enforceBoundaries" is true in .lisa.config.json (default
|
|
@@ -90,11 +104,15 @@
|
|
|
90
104
|
# Per-session state lives under "$STATE_DIR" as flag files keyed by session_id.
|
|
91
105
|
# Stale state (>24h) is cleaned on each invocation.
|
|
92
106
|
#
|
|
93
|
-
# Fail-open:
|
|
94
|
-
#
|
|
95
|
-
#
|
|
96
|
-
#
|
|
97
|
-
#
|
|
107
|
+
# Fail-open: a missing field degrades to the LESS strict outcome rather than
|
|
108
|
+
# inventing a new hard failure, and the MAX_BLOCKS escalation below guarantees
|
|
109
|
+
# the gate always releases eventually. A broken gate must never brick a session.
|
|
110
|
+
#
|
|
111
|
+
# That fail-open posture bounds the BLAST RADIUS of a gate failure; it is not a
|
|
112
|
+
# licence to read an unreadable verdict as a clean one. So a v2 verdict whose
|
|
113
|
+
# claim structure cannot be evaluated is reported like any other violation —
|
|
114
|
+
# advisory while the ratchet is off, blocking (still MAX_BLOCKS-bounded, so it
|
|
115
|
+
# can never hard-wedge a session) once it is on.
|
|
98
116
|
|
|
99
117
|
set -uo pipefail
|
|
100
118
|
|
|
@@ -225,9 +243,39 @@ boundary_enforcement_enabled() {
|
|
|
225
243
|
[ "$value" = "true" ]
|
|
226
244
|
}
|
|
227
245
|
|
|
246
|
+
# Prints the sha256 of a file using whichever tool this machine has. Prints
|
|
247
|
+
# nothing when neither exists — an unrecomputable digest is not a violation.
|
|
248
|
+
sha256_of() {
|
|
249
|
+
if command -v shasum >/dev/null 2>&1; then
|
|
250
|
+
shasum -a 256 "$1" 2>/dev/null | awk '{print $1}'
|
|
251
|
+
elif command -v sha256sum >/dev/null 2>&1; then
|
|
252
|
+
sha256sum "$1" 2>/dev/null | awk '{print $1}'
|
|
253
|
+
fi
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
# Emits one line per evidence entry whose bytes no longer hash to the digest the
|
|
257
|
+
# verdict recorded. Only entries recording BOTH a sha256 and a locator that
|
|
258
|
+
# resolves to a file on disk are judged; see the header for why an absent
|
|
259
|
+
# locator is out of this hook's scope.
|
|
260
|
+
v2_digest_violations() {
|
|
261
|
+
jq -r '
|
|
262
|
+
(.evidence // [])[]
|
|
263
|
+
| select(((.sha256 // "") | length) > 0)
|
|
264
|
+
| select(((.locator // "") | length) > 0)
|
|
265
|
+
| "\(.evidence_id // "?")\t\(.locator)\t\(.sha256)"
|
|
266
|
+
' "$VERDICT_FILE" 2>/dev/null | while IFS=$'\t' read -r eid locator recorded; do
|
|
267
|
+
evidence_path="${PROJECT_DIR}/${locator}"
|
|
268
|
+
[ -f "$evidence_path" ] || continue
|
|
269
|
+
actual=$(sha256_of "$evidence_path")
|
|
270
|
+
[ -n "$actual" ] || continue
|
|
271
|
+
[ "$actual" = "$recorded" ] && continue
|
|
272
|
+
echo "evidence ${eid} (${locator}) no longer matches its recorded digest: recorded ${recorded}, bytes now hash to ${actual}"
|
|
273
|
+
done
|
|
274
|
+
}
|
|
275
|
+
|
|
228
276
|
# Emits one line per v2 claim->evidence contract violation. Empty output means
|
|
229
|
-
# the verdict satisfies the contract
|
|
230
|
-
#
|
|
277
|
+
# the verdict satisfies the contract. A jq evaluation failure is NOT empty
|
|
278
|
+
# output — see v2_evaluate_contract, which turns it into its own violation.
|
|
231
279
|
v2_contract_violations() {
|
|
232
280
|
jq -r '
|
|
233
281
|
. as $v
|
|
@@ -263,7 +311,32 @@ v2_contract_violations() {
|
|
|
263
311
|
)
|
|
264
312
|
]
|
|
265
313
|
| .[]
|
|
266
|
-
' "$VERDICT_FILE" 2>/dev/null
|
|
314
|
+
' "$VERDICT_FILE" 2>/dev/null
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
# Set by v2_evaluate_contract: every violation line, or empty when the verdict
|
|
318
|
+
# satisfies the contract. A global rather than a return value because a command
|
|
319
|
+
# substitution would swallow the could-not-evaluate signal along with it.
|
|
320
|
+
V2_VIOLATIONS=""
|
|
321
|
+
|
|
322
|
+
# Evaluates the full v2 contract into V2_VIOLATIONS.
|
|
323
|
+
#
|
|
324
|
+
# The load-bearing distinction: jq returns NOTHING on an evaluation error, which
|
|
325
|
+
# is byte-identical to "this verdict is clean". A parseable v2 verdict whose
|
|
326
|
+
# claims/evidence are the wrong SHAPE (claims as a string, evidence as a scalar)
|
|
327
|
+
# therefore used to read as violation-free — harmless while advisory, a genuine
|
|
328
|
+
# bypass of the gate the moment enforcement is ratcheted on. So a non-zero jq
|
|
329
|
+
# exit becomes its own, named violation.
|
|
330
|
+
v2_evaluate_contract() {
|
|
331
|
+
local structural digests
|
|
332
|
+
structural=$(v2_contract_violations)
|
|
333
|
+
if [ "$?" -ne 0 ]; then
|
|
334
|
+
V2_VIOLATIONS="the v2 claim/evidence structure could not be evaluated - \"claims\" and \"evidence\" must be arrays of objects and \"artifact\" an object. A verdict the gate cannot read is not a verdict with no violations"
|
|
335
|
+
return 0
|
|
336
|
+
fi
|
|
337
|
+
|
|
338
|
+
digests=$(v2_digest_violations)
|
|
339
|
+
V2_VIOLATIONS=$(printf '%s\n%s' "$structural" "$digests" | sed '/^[[:space:]]*$/d')
|
|
267
340
|
}
|
|
268
341
|
|
|
269
342
|
# v2 = the v1 decision PLUS the claim->evidence contract, the latter advisory
|
|
@@ -277,7 +350,8 @@ verdict_is_terminal_v2() {
|
|
|
277
350
|
status=$(jq -r '.status // empty' "$VERDICT_FILE" 2>/dev/null || true)
|
|
278
351
|
[ "$status" = "pass" ] || return 0
|
|
279
352
|
|
|
280
|
-
|
|
353
|
+
v2_evaluate_contract
|
|
354
|
+
violations="$V2_VIOLATIONS"
|
|
281
355
|
[ -n "$violations" ] || return 0
|
|
282
356
|
|
|
283
357
|
if boundary_enforcement_enabled; then
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# Repository Readiness Rubric (load-bearing)
|
|
2
|
+
|
|
3
|
+
**"Lisa is installed correctly" and "an agent fleet may run here unattended" are two different
|
|
4
|
+
questions.** Doctor's shipped grouped checks answer the first — *installation readiness*. This rubric
|
|
5
|
+
answers the second — *repository readiness* — and it is the only place that question is written down.
|
|
6
|
+
A green test suite is not an answer to it.
|
|
7
|
+
|
|
8
|
+
## Eight ownership dimensions
|
|
9
|
+
|
|
10
|
+
Repository readiness is assessed across exactly eight dimensions, never fewer, and a dimension with
|
|
11
|
+
no applicable evidence renders `SKIP` **with a reason** rather than a blank:
|
|
12
|
+
|
|
13
|
+
1. **context/routing** — can an agent recover the real job from what is written down?
|
|
14
|
+
2. **capabilities/tools** — is every tool the work needs *provably* reachable, not merely installed?
|
|
15
|
+
3. **domain ownership** — are the business rules, glossary, and danger zones owned and written down?
|
|
16
|
+
4. **execution/proof** — can the claimed user-visible outcome be proved by running the system?
|
|
17
|
+
5. **feedback/guardrails** — does a failing loop produce a named outcome and a runbook?
|
|
18
|
+
6. **dependencies/supply chain** — is there a confidence model for what the repo depends on?
|
|
19
|
+
7. **delivery/authority** — does the thing that ships equal the thing that was validated, and does
|
|
20
|
+
the credential that ships it carry only the authority it needs?
|
|
21
|
+
8. **proportionality** — is the machinery proportional to the job, or is there scaffolding to
|
|
22
|
+
subtract?
|
|
23
|
+
|
|
24
|
+
## Seven ship blockers (closed set, v1)
|
|
25
|
+
|
|
26
|
+
A **ship blocker** is a condition that, standing alone, means the answer to "may an unattended fleet
|
|
27
|
+
run here?" is no. The set is closed in v1 — seven, no more — and extended only by editing this rule:
|
|
28
|
+
|
|
29
|
+
- **B1** a realistic path causes **silent data loss**
|
|
30
|
+
- **B2** a **release path bypasses the validated artifact**
|
|
31
|
+
- **B3** **credentials carry material unintended authority**
|
|
32
|
+
- **B4** a **consequential operation has no gate and no recovery**
|
|
33
|
+
- **B5** an **owned compatibility or security surface has no confidence model**
|
|
34
|
+
- **B6** **documentation overstates enforced guarantees**
|
|
35
|
+
- **B7** there is **no way to prove the claimed user-visible outcome**
|
|
36
|
+
|
|
37
|
+
## Verdict and narrowed claim
|
|
38
|
+
|
|
39
|
+
The verdict reuses the shipped `READY` / `READY_WITH_WARNINGS` / `NOT_READY` ladder — cite doctor's
|
|
40
|
+
ladder, never fork a parallel enum. **A standing blocker is `NOT_READY`**, and the report must also
|
|
41
|
+
state the **narrowed claim**: what the repository *is* ready for, in operator language ("ready for
|
|
42
|
+
supervised single-ticket work; not ready for unattended fleet operation, because …").
|
|
43
|
+
|
|
44
|
+
## Ordering
|
|
45
|
+
|
|
46
|
+
**Report section order stays stable and never silently omits a section; findings are ordered by
|
|
47
|
+
consequence** — highest-consequence first — within and across sections. The two contracts do not
|
|
48
|
+
collide: sections are fixed, findings are ranked.
|
|
49
|
+
|
|
50
|
+
## Warn-only
|
|
51
|
+
|
|
52
|
+
This rubric **gates a claim, not a process**. No Lisa surface hard-blocks on the verdict: a standing
|
|
53
|
+
blocker narrows what may be claimed and files tracker work; it never stops `lisa apply`, intake
|
|
54
|
+
dispatch, or cron registration. Where a surface this rule names is not installed in a given branch,
|
|
55
|
+
name what you can and continue — degrade, never block. Written to be read by someone who does not
|
|
56
|
+
code (`factory-model` rule 5).
|
|
57
|
+
|
|
58
|
+
Full rubric (eight-dimension table, seven ship blockers, consequence ordering, worked example): [reference/readiness-rubric.md](../reference/readiness-rubric.md).
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Repository Readiness Rubric
|
|
2
|
+
|
|
3
|
+
Lisa can already tell you whether it is *installed* correctly. It cannot tell you whether a
|
|
4
|
+
repository is somewhere an agent fleet can safely operate **unattended**. Those are two different
|
|
5
|
+
questions, and conflating them is how a brownfield onboarding ends in "we built a wiki, looks good"
|
|
6
|
+
instead of a verdict someone can act on.
|
|
7
|
+
|
|
8
|
+
This document writes the second question down once: **eight ownership dimensions** with concrete
|
|
9
|
+
warning signs, and **seven ship blockers** that, if any one of them stands, mean the answer is no.
|
|
10
|
+
It is documentation. Nothing here executes — the later tickets of PRD #1739 instantiate this rubric
|
|
11
|
+
rather than redefine it.
|
|
12
|
+
|
|
13
|
+
## Two readinesses, one report
|
|
14
|
+
|
|
15
|
+
| | Installation readiness | Repository readiness |
|
|
16
|
+
|---|---|---|
|
|
17
|
+
| Question | is Lisa installed and wired correctly here? | may an agent fleet operate here unattended? |
|
|
18
|
+
| Owner | `lisa doctor`'s shipped grouped checks (project detection, config, distribution, tracker preflight, automation prerequisites, …) | this rubric's eight dimensions |
|
|
19
|
+
| Answer shape | the shipped verdict ladder | the same shipped verdict ladder, plus a narrowed claim |
|
|
20
|
+
|
|
21
|
+
They are **orthogonal** — a repository can be perfectly installed and completely unready, or ready in
|
|
22
|
+
substance while Lisa's own wiring is incomplete. Both render in the same report, under separately
|
|
23
|
+
titled sections, so a reader is never left guessing which question a verdict answered. The doctor
|
|
24
|
+
mode that renders the repository-readiness group ships with **RRR-3 (#1855)**; do not assume that
|
|
25
|
+
surface is present in this branch.
|
|
26
|
+
|
|
27
|
+
## The eight ownership dimensions
|
|
28
|
+
|
|
29
|
+
Exactly eight, always reported, never silently omitted. Each row names what the dimension asks,
|
|
30
|
+
concrete warning signs an assessor can look for, and the **existing** Lisa rule or wiki page that
|
|
31
|
+
supplies its evidence. Cite those slugs; do not restate them, and do not invent a parallel vocabulary
|
|
32
|
+
for a concept another rule already owns.
|
|
33
|
+
|
|
34
|
+
| # | Dimension | The question | Warning signs | Evidence source (existing) |
|
|
35
|
+
|---|---|---|---|---|
|
|
36
|
+
| 1 | context/routing | Can an agent recover the real job from what is written down? | no canonical entry document; routing decided by tribal knowledge; ambiguous config resolution; a README that describes a system that no longer exists | `integration-access-layer`, `wiki-knowledge-source`, `config-resolution` |
|
|
37
|
+
| 2 | capabilities/tools | Is every tool the work needs *provably* reachable, not merely installed? | presence-on-PATH treated as access; no read-only probe before the work starts; agents inventing workarounds instead of breaking out | `tool-access-gate` |
|
|
38
|
+
| 3 | domain ownership | Are the business rules, glossary, and danger zones owned and written down? | undocumented money paths; migrations with no owner; irreversible jobs nobody has described | the wiki pages agent-ready's domain phase already produces |
|
|
39
|
+
| 4 | execution/proof | Can the claimed user-visible outcome be proved by running the system? | claims backed only by unit tests; no representative end-to-end journey; "verified" that names no boundary | `verification`, `empirical-inquiry`, `claim-evidence-mapping` |
|
|
40
|
+
| 5 | feedback/guardrails | Does a failing loop produce a named outcome and a runbook? | silent failures; no run-outcome vocabulary; no observability on the loops that run unattended | `automation-runbook-contract`, `observability-audit` |
|
|
41
|
+
| 6 | dependencies/supply chain | Is there a confidence model for what the repo depends on? | unpinned or unowned dependencies; no trust class; no decision record for why a risky dependency stays | `security-audit-handling` |
|
|
42
|
+
| 7 | delivery/authority | Does the thing that ships equal the thing that was validated, and does the credential that ships it carry only the authority it needs? | a release path that bypasses the validated artifact; broad-scope tokens; deploy credentials shared across environments | `claim-archaeology`, `security-audit-handling` |
|
|
43
|
+
| 8 | proportionality | Is the machinery proportional to the job, or is there scaffolding to subtract? | redundant checks that assert the same thing twice; abandoned harnesses still running in CI | `repo-scope-split`, and the scaffolding-subtraction candidates the journey work of **#1742** already surfaces |
|
|
44
|
+
|
|
45
|
+
**`SKIP` carries a reason and is never blank.** A dimension with no applicable evidence renders
|
|
46
|
+
`SKIP with a reason` — "no deployment target configured, so delivery/authority was not assessed" —
|
|
47
|
+
never an empty cell and never a silent omission. An unassessed dimension is a known unknown, and the
|
|
48
|
+
report says so.
|
|
49
|
+
|
|
50
|
+
## The seven ship blockers
|
|
51
|
+
|
|
52
|
+
A **ship blocker** is a condition that, standing alone, makes the claim "an unattended fleet may run
|
|
53
|
+
here" false. Each has a one-line test an assessor can apply and the dimension that owns it.
|
|
54
|
+
|
|
55
|
+
| # | Blocker | The test an assessor applies | Owning dimension |
|
|
56
|
+
|---|---|---|---|
|
|
57
|
+
| B1 | A realistic path causes **silent data loss** | can you name a plausible sequence that destroys or corrupts data with no error surfaced and no recovery path? | 3 |
|
|
58
|
+
| B2 | A **release path bypasses the validated artifact** | can something reach production that is not the artifact CI actually validated? | 7 |
|
|
59
|
+
| B3 | **Credentials carry material unintended authority** | does any credential used by automation grant authority materially beyond the job it does? | 7 |
|
|
60
|
+
| B4 | A **consequential operation has no gate and no recovery** | is there an irreversible or expensive operation reachable without a gate, with no way back? | 3, 5 |
|
|
61
|
+
| B5 | An **owned compatibility or security surface has no confidence model** | for a surface this repo owns, is there any basis beyond hope for believing it still works? | 6 |
|
|
62
|
+
| B6 | **Documentation overstates enforced guarantees** | does the written word claim something is enforced that nothing actually enforces? | 1 |
|
|
63
|
+
| B7 | There is **no way to prove the claimed user-visible outcome** | can the headline user-visible claim be demonstrated by running the system, at the boundary it claims? | 4 |
|
|
64
|
+
|
|
65
|
+
### The set is a closed set in v1
|
|
66
|
+
|
|
67
|
+
Seven, and only seven, in v1. The set is **not configurable** — there is no config key, no
|
|
68
|
+
host-defined blockers, no severity dial. Extending it is a deliberate **rule edit** here plus a
|
|
69
|
+
version bump, not a configuration surface to design, migrate, and support. Host-specific concerns
|
|
70
|
+
surface as *findings within a dimension*, which is enough to make them visible without fragmenting
|
|
71
|
+
the vocabulary.
|
|
72
|
+
|
|
73
|
+
### "Ship blocker" is net-new vocabulary
|
|
74
|
+
|
|
75
|
+
It is deliberately distinct from three terms other rules already own, and it never replaces them:
|
|
76
|
+
|
|
77
|
+
- `convergent-review`'s **blocking finding** — a review-level judgment about one change.
|
|
78
|
+
- `tool-access-gate`'s **break-out** — the escalation when a required tool is not provably reachable.
|
|
79
|
+
- `leaf-only-lifecycle`'s **safe-block** — a lifecycle-state repair on a work item.
|
|
80
|
+
|
|
81
|
+
A ship blocker is none of these: it is a property of the *repository*, asserted about *unattended
|
|
82
|
+
operation*, and it gates a claim rather than a change, a tool call, or a ticket.
|
|
83
|
+
|
|
84
|
+
## Verdict: the shipped ladder, plus a narrowed claim
|
|
85
|
+
|
|
86
|
+
The rubric reuses the shipped `READY` / `READY_WITH_WARNINGS` / `NOT_READY` verdict ladder that
|
|
87
|
+
`lisa doctor` already emits. There is **no new verdict** value and no new severity level — inventing
|
|
88
|
+
a parallel enum would make two reports disagree about what "ready" means.
|
|
89
|
+
|
|
90
|
+
- **`READY`** — eight dimensions assessed, no blocker stands, no warnings material to unattended
|
|
91
|
+
operation.
|
|
92
|
+
- **`READY_WITH_WARNINGS`** — no blocker stands, but findings exist that a human should see.
|
|
93
|
+
- **`NOT_READY`** — **at least one ship blocker stands.**
|
|
94
|
+
|
|
95
|
+
**The narrowed claim is the net-new field.** When the verdict is `NOT_READY`, the report must also
|
|
96
|
+
state, in operator language, **what the repository IS ready for** — never just what it is not. A
|
|
97
|
+
verdict that only says no is unactionable at the gate; a narrowed claim tells the operator exactly
|
|
98
|
+
which mode of operation remains available and what would widen it.
|
|
99
|
+
|
|
100
|
+
## Consequence ordering
|
|
101
|
+
|
|
102
|
+
Two ordering contracts meet here, and they do not collide:
|
|
103
|
+
|
|
104
|
+
- **Section order stays stable.** The report renders its sections in a fixed order and **never
|
|
105
|
+
silently omits** one — the discipline `lisa doctor`'s grouped output already guarantees.
|
|
106
|
+
- **Findings are ordered by consequence.** Within and across sections, the finding with the largest
|
|
107
|
+
consequence if left standing comes first. Alphabetical, chronological, and discovery order are all
|
|
108
|
+
wrong: the reader at the gate has limited attention, and it belongs on the worst thing first.
|
|
109
|
+
|
|
110
|
+
### The five fields a readiness finding carries
|
|
111
|
+
|
|
112
|
+
On top of the severity / blocking / failure-scenario / evidence / smallest-fix fields
|
|
113
|
+
`convergent-review` already requires, a readiness finding names:
|
|
114
|
+
|
|
115
|
+
| Field | What it states |
|
|
116
|
+
|---|---|
|
|
117
|
+
| `invariant_violated` | the invariant actually at risk, stated as a property of the system |
|
|
118
|
+
| `evidence` | what was observed that establishes the finding, at a boundary that reaches the claim (`claim-evidence-mapping`) |
|
|
119
|
+
| `why_proof_missed` | why the existing proof machinery did not catch this |
|
|
120
|
+
| `root_correction` | the correction at the **owning boundary**, not a patch at the symptom |
|
|
121
|
+
| `machinery_to_remove` | redundant machinery the correction makes unnecessary, if any |
|
|
122
|
+
|
|
123
|
+
Two of these — `invariant_violated` and `machinery_to_remove` — are folded into the shared
|
|
124
|
+
`convergent-review` finding shape by **RRR-2 (#1854)**; that extension ships with that ticket, so do
|
|
125
|
+
not assume the shared shape carries them in this branch.
|
|
126
|
+
|
|
127
|
+
### Worked example
|
|
128
|
+
|
|
129
|
+
```text
|
|
130
|
+
Repository: acme/checkout-service
|
|
131
|
+
|
|
132
|
+
Verdict NOT_READY
|
|
133
|
+
Standing blocker B2 — a release path bypasses the validated artifact
|
|
134
|
+
Owning dimension 7 (delivery/authority)
|
|
135
|
+
|
|
136
|
+
invariant_violated What ships to production is the artifact CI validated.
|
|
137
|
+
evidence The deploy job rebuilds from source at deploy time rather than
|
|
138
|
+
promoting the CI-built image; the deployed digest never matches
|
|
139
|
+
the one the test job signed off on.
|
|
140
|
+
why_proof_missed Every check is green — they all ran against a different artifact
|
|
141
|
+
than the one that shipped. Nothing compared the two.
|
|
142
|
+
root_correction Promote the validated image by digest at the delivery boundary.
|
|
143
|
+
machinery_to_remove The duplicate deploy-time build step.
|
|
144
|
+
|
|
145
|
+
Narrowed claim This repository IS ready for supervised, single-ticket agent work with a
|
|
146
|
+
human approving each release. It is NOT ready for unattended fleet
|
|
147
|
+
operation, because a release can ship code no check ever ran against.
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
## Where the evidence comes from
|
|
151
|
+
|
|
152
|
+
The rubric consumes evidence that already exists; it commissions no second harness.
|
|
153
|
+
|
|
154
|
+
- **execution/proof** consumes the existing qualification evidence recorded by the worker-epoch
|
|
155
|
+
requalification path and, when that evidence is absent or stale, triggers a journey run through the
|
|
156
|
+
shipped `lisa-use-the-product` skill — the machinery **#1742** already ships. There is no second
|
|
157
|
+
journey harness, and the evidence is recorded in the shape `claim-evidence-mapping` defines. That
|
|
158
|
+
wiring ships with **RRR-6 (#1858)**.
|
|
159
|
+
- **domain ownership** sources its findings from the danger-zone wiki pages agent-ready's domain
|
|
160
|
+
phase already produces; the readiness assessment that reads them, and files standing blockers as
|
|
161
|
+
tracker work rather than in-session questions, ships with **RRR-4 (#1856)**.
|
|
162
|
+
- The persisted report at `.lisa/readiness.json` (schema-versioned, read through a single resolver)
|
|
163
|
+
and the doctor render group ship with **RRR-3 (#1855)**; the blocker gate that emits the narrowed
|
|
164
|
+
claim ships with **RRR-5 (#1857)**; the `setup-automations` warning, six-agent parity fan-out, and
|
|
165
|
+
vocabulary documentation ship with **RRR-7 (#1859)**. Each of those surfaces may not yet be present
|
|
166
|
+
in a given branch — name what you can and continue.
|
|
167
|
+
|
|
168
|
+
## Warn-only, always
|
|
169
|
+
|
|
170
|
+
This rubric **gates a claim, not a process**. It is **warn-only** everywhere: no Lisa surface
|
|
171
|
+
hard-blocks on the readiness verdict. `lisa apply`, intake dispatch, and cron registration are
|
|
172
|
+
unaffected; the automation setup flow warns with the standing blocker count and the narrowed claim
|
|
173
|
+
and still completes, consistent with the shipped never-block-always-degrade posture. Where a surface
|
|
174
|
+
named here is not installed, **degrade, never block**: state what was assessed, state what was not,
|
|
175
|
+
and continue.
|
|
176
|
+
|
|
177
|
+
Read the eight dimension titles and the seven blockers to someone who has never seen Lisa. They
|
|
178
|
+
should be able to say, unprompted, why "the tests pass" is not the same as "an agent fleet can run
|
|
179
|
+
here unattended" — that is the bar this document is written to (`factory-model` rule 5).
|
|
@@ -177,6 +177,8 @@ The list may be empty; the flag may not be missing. An absent `not_established_r
|
|
|
177
177
|
|
|
178
178
|
The boundary each artifact type reaches — and therefore which claim a captured artifact can discharge — is the `claim-evidence-mapping` rule's taxonomy; the type table above is its evidence-kind source.
|
|
179
179
|
|
|
180
|
+
The verdict is read twice. The Claude-only `enforce-verification-gate.sh` Stop hook reads it to decide whether the flow may stop; `lisa-spec-conformance` (run by `spec-conformance-specialist` in the verification phase) reads it to decide whether each shipped requirement's proof actually reaches its boundary — a cited-evidence-boundary mismatch is a `BOUNDARY_MISMATCH` conformance finding there, caught alongside empirical verification rather than after it. On harnesses without a Stop hook, `lisa-implement`'s prose gate carries the same v2 expectations by convention. The whole system, operator-readable end to end, is written up as the Lisa wiki's **Bounded-Claims Evidence System** concept page (`wiki/concepts/bounded-claims-evidence-system.md` upstream).
|
|
181
|
+
|
|
180
182
|
### Cross-work-item evidence references are non-claiming
|
|
181
183
|
|
|
182
184
|
When prose needs to point at evidence declared by another work item, use the dedicated reference form:
|
|
@@ -278,7 +278,28 @@ Before shutting down the team, execute the Verify flow:
|
|
|
278
278
|
- **Actionable blocker** — an unresolved dependency or fixable technical gap that some team or repository could build (a missing or changed schema field, an unbuilt sibling work item, a required upstream fix), **including cross-repo dependencies**. Before writing the blocked verdict you MUST (1) file a build-ready fix/dependency ticket capturing the diagnosis — in the dependency's own repository/tracker when it is cross-repo (e.g. a `[<repo>] …` ticket in the shared project, or the sibling tracker) — and (2) link the current work item to it as `is blocked by`. Only then write the verdict. This is the same discipline as the regression-spec blocker and the remote-verification-fail exits above, and it is what makes the block machine-recoverable: `repair-intake` re-dispatches a blocked item once its linked `is blocked by` dependency closes, but it cannot act on a prose-only comment. Recommending the ticket "as a human follow-up" without filing and linking it is **not** a permitted exit.
|
|
279
279
|
- **Human-only blocker** — an input the agent genuinely cannot obtain or produce no matter what it does: credentials, secrets, or **tool access** it does not have (AWS/CloudWatch, Figma, Jam, Sentry, SonarCloud, a database, a protected deploy target, …), or a product/design decision only a human can make. For missing tool access, follow the `tool-access-gate` rule's break-out protocol: post the "Access Needed" comment naming the exact credential/role/env var to grant and the probe that must pass — never work around the gap by substituting weaker verification, mocking the inaccessible system, or narrowing scope. Record the blocked verdict, mark it `human_needed` (the marker `repair-intake` recognizes, so it won't churn re-dispatching it), and surface or reassign to a human; do **not** fabricate a build-ready ticket, because there is no build-ready work.
|
|
280
280
|
|
|
281
|
-
|
|
281
|
+
**Harnesses that do not fire a Stop hook enforce the same discipline by convention.** The
|
|
282
|
+
`enforce-verification-gate.sh` Stop hook is a **Claude-only** surface — Codex, Cursor, Antigravity,
|
|
283
|
+
Copilot, and OpenCode carry the skills, agents, and (where the runtime has a rules surface) the
|
|
284
|
+
`claim-evidence-mapping` rule, but nothing on those runtimes can refuse to stop. That is a known
|
|
285
|
+
**representation gap**, documented here rather than dropped: on those harnesses this prose gate
|
|
286
|
+
*is* the gate, and the flow may not declare completion until it has written the same v2 verdict
|
|
287
|
+
and self-checked it against the same expectations the hook would have applied:
|
|
288
|
+
|
|
289
|
+
- `schema_version: 2` is written, with `plan`, `status`, and `updated_at` terminal and fresh.
|
|
290
|
+
- Every claim carries a `claim_id`, a `boundary` from the closed set, and the
|
|
291
|
+
`required_evidence_kinds` that reach that boundary — and its `evidence_refs` resolve to evidence
|
|
292
|
+
whose `kind` is one of them. A unit `test-run-log` cited for a `browser`, `http-api`, or
|
|
293
|
+
`deploy-health` claim is the failure this check exists to catch.
|
|
294
|
+
- `artifact.head_sha` names what will ship, and every evidence entry's `artifact_head_sha` matches
|
|
295
|
+
it, with a `sha256` digest and `captured_at` recorded as values, never placeholders.
|
|
296
|
+
- The `not_established` list is present on every claim and `not_established_reviewed` is `true` —
|
|
297
|
+
an empty list is fine, an omitted flag is not.
|
|
298
|
+
|
|
299
|
+
Record the self-check in the completion summary the way the hook would have reported it: name the
|
|
300
|
+
boundary each claim reached, or name the violation. Where the runtime lacks the rules surface (the
|
|
301
|
+
agy artifacts carry no rules tree), the obligation still travels in this skill — cite the
|
|
302
|
+
`claim-evidence-mapping` contract by slug and continue; never block on the absent surface.
|
|
282
303
|
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified, codification into **every supported UI runner**: a Playwright spec in the Playwright runner AND a Maestro flow when the project supports Maestro, per `codify-verification`. Prove the new spec actually executed and passed in PR CI by recording a named spec log/reporter line or equivalent execution record; green CI without that named evidence does not satisfy this step.
|
|
283
304
|
4. Record Implement usage on the originating work artifact via `lisa-usage-accounting` so the work item (or other implementation-owned artifact) gains a direct `lisa-implement` usage entry in the canonical `## Lisa Usage` section. If the parent / child graph is already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise still write the direct entry, and if runtime usage is unavailable, use `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
284
305
|
5. Commit ALL outstanding changes in logical batches on the branch (minus sensitive data/information) — not just changes made by the agent team. This includes pre-existing uncommitted changes that were on the branch before the plan started. Do NOT filter commits to only "task-related" files. If it shows up in git status, it gets committed (unless it contains secrets).
|