@codyswann/lisa 2.159.2 → 2.159.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/rules/eager/verification.md +2 -0
- package/plugins/lisa/rules/reference/verification.md +14 -0
- package/plugins/lisa/skills/verification-lifecycle/SKILL.md +16 -0
- package/plugins/lisa/skills/verify/SKILL.md +3 -0
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/verification-lifecycle/SKILL.md +16 -0
- package/plugins/lisa-agy/skills/verify/SKILL.md +3 -0
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/rules/eager/verification.md +2 -0
- package/plugins/lisa-copilot/rules/reference/verification.md +14 -0
- package/plugins/lisa-copilot/skills/verification-lifecycle/SKILL.md +16 -0
- package/plugins/lisa-copilot/skills/verify/SKILL.md +3 -0
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/rules/verification-reference.mdc +14 -0
- package/plugins/lisa-cursor/rules/verification.mdc +2 -0
- package/plugins/lisa-cursor/skills/verification-lifecycle/SKILL.md +16 -0
- package/plugins/lisa-cursor/skills/verify/SKILL.md +3 -0
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/rules/eager/verification.md +2 -0
- package/plugins/src/base/rules/reference/verification.md +14 -0
- package/plugins/src/base/skills/verification-lifecycle/SKILL.md +16 -0
- package/plugins/src/base/skills/verify/SKILL.md +3 -0
package/package.json
CHANGED
|
@@ -84,7 +84,7 @@
|
|
|
84
84
|
"lodash": ">=4.18.1"
|
|
85
85
|
},
|
|
86
86
|
"name": "@codyswann/lisa",
|
|
87
|
-
"version": "2.159.
|
|
87
|
+
"version": "2.159.4",
|
|
88
88
|
"description": "Claude Code governance framework that applies guardrails, guidance, and automated enforcement to projects",
|
|
89
89
|
"main": "dist/index.js",
|
|
90
90
|
"exports": {
|
|
@@ -11,6 +11,8 @@
|
|
|
11
11
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
12
12
|
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes.
|
|
13
13
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
14
|
+
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
15
|
+
- **Never mark required runtime verification done on artifact-only evidence** — if credentials are genuinely unavailable after all sources are checked, post the blocker, transition the item to the configured blocked state, apply the configured `needs-human` / `human-review` label, and label the evidence as `artifact-only / verification deferred`.
|
|
14
16
|
|
|
15
17
|
## Roles
|
|
16
18
|
|
|
@@ -121,4 +121,18 @@ Both levels use the same verification types table above. The difference is the e
|
|
|
121
121
|
|
|
122
122
|
---
|
|
123
123
|
|
|
124
|
+
## Credential-Gated Verification
|
|
125
|
+
|
|
126
|
+
Some runtime verification requires signing in to a deployed or local app. Agents must exhaust credential sources before declaring verification blocked:
|
|
127
|
+
|
|
128
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
129
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
130
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
131
|
+
|
|
132
|
+
If credentials are genuinely unavailable after all three source classes are checked, the item is blocked, not done. The agent must post a tracker comment explaining what could not be verified and which sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating the label if the tracker supports label creation and it is missing.
|
|
133
|
+
|
|
134
|
+
Evidence and summaries must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`. Artifact-only evidence can explain what was checked before escalation, but it cannot complete a work item that requires runtime verification.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
124
138
|
For the full verification lifecycle (classify, check tooling, plan, execute, loop), surfaces, escalation protocol, and proof artifact requirements, see the `verification-lifecycle` skill loaded by the `verification-specialist` agent.
|
|
@@ -35,10 +35,20 @@ For each required verification type, discover what tools are available in the pr
|
|
|
35
35
|
|
|
36
36
|
Report what is available for each required type. If a required type has no available tool, proceed to step 4.
|
|
37
37
|
|
|
38
|
+
If a required verification type needs sign-in or other credentials, exhaust credential sources before declaring the verification blocked. Check credential sources in this order:
|
|
39
|
+
|
|
40
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
41
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
42
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
43
|
+
|
|
44
|
+
Report which sources were checked. Do not say credentials are unavailable until all three source classes have been checked or proven absent.
|
|
45
|
+
|
|
38
46
|
### 4. Fail Fast
|
|
39
47
|
|
|
40
48
|
If a required verification type has no available tool and no reasonable alternative, escalate immediately using the Escalation Protocol. Do not begin implementation without a verification plan for every required type.
|
|
41
49
|
|
|
50
|
+
If credentials are genuinely unavailable after the credential lookup order above is exhausted, treat the work item as blocked rather than done. Post a clear tracker comment stating exactly what runtime behavior could not be verified and which credential sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating that label if the tracker supports label creation and it is missing.
|
|
51
|
+
|
|
42
52
|
### 5. Plan
|
|
43
53
|
|
|
44
54
|
For each verification type, state:
|
|
@@ -52,6 +62,8 @@ A verification plan that only lists `bun run test`, `bun run typecheck`, or `bun
|
|
|
52
62
|
|
|
53
63
|
After implementation, run the verification plan. Execute each verification type in order.
|
|
54
64
|
|
|
65
|
+
Evidence output must explicitly label each verification result as either `verified empirically` or `artifact-only / verification deferred`. Artifact-only evidence can support a blocked escalation packet, but it cannot mark a required runtime verification complete.
|
|
66
|
+
|
|
55
67
|
### 7. Codify
|
|
56
68
|
|
|
57
69
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
@@ -229,6 +241,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
229
241
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
230
242
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
231
243
|
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or empty artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
244
|
+
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
232
245
|
|
|
233
246
|
---
|
|
234
247
|
|
|
@@ -245,6 +258,7 @@ Common blockers:
|
|
|
245
258
|
- Missing documentation on how to access required tooling
|
|
246
259
|
- Production-only access gates
|
|
247
260
|
- Compliance restrictions
|
|
261
|
+
- Missing runtime credentials after checking project e2e / Playwright config and fixtures, `.lisa.config.local.json` / environment, and documented ticket credentials
|
|
248
262
|
|
|
249
263
|
When blocked, agents must do the following:
|
|
250
264
|
|
|
@@ -255,6 +269,8 @@ When blocked, agents must do the following:
|
|
|
255
269
|
5. Produce a Human Action Packet.
|
|
256
270
|
6. Pause until explicit human confirmation or tooling is provided.
|
|
257
271
|
|
|
272
|
+
For tracker-backed work items, also post the packet to the tracker, transition to the configured blocked state, and apply the configured `needs-human` / `human-review` label. If the label is missing and the tracker supports label creation, create it before applying it.
|
|
273
|
+
|
|
258
274
|
Agents must never proceed past an unverified boundary without surfacing it to the human overseer.
|
|
259
275
|
|
|
260
276
|
### Human Action Packet Format
|
|
@@ -39,6 +39,9 @@ Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via
|
|
|
39
39
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa:pull-request-review`
|
|
40
40
|
5. **Merge** when CI is green
|
|
41
41
|
6. **Remote verification** — invoke the `lisa:monitor` skill against the target environment to confirm the deploy actually works (health endpoints, recent logs/errors, Validation Journey replay if defined). If remote verification surfaces a behavioral gap that the existing codified tests do not guard, invoke `codify-verification` to add coverage and open a follow-up PR.
|
|
42
|
+
- When remote verification needs credentials, follow the shared `verification-lifecycle` credential lookup order before declaring them missing: project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as a `Sign-in Required` section.
|
|
43
|
+
- Should credentials remain genuinely unavailable after those sources are exhausted, do not complete the item on artifact-only evidence. Post a tracker comment stating what could not be verified and why, transition the work item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating it if the tracker supports label creation and it is missing.
|
|
44
|
+
- Evidence must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`.
|
|
42
45
|
7. **Evidence usage** — before posting, route the generated evidence artifact through `lisa:usage-accounting` so the comment body / PR evidence section / markdown proof carries a direct `verify` usage entry in the canonical `## Lisa Usage` section. If the originating work item or PRD parentage is known, prefer `record_and_rollup` so ancestor totals refresh in the same pass. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
43
46
|
8. **Evidence** — post results to the originating ticket via `lisa:tracker-evidence` (vendor-neutral; dispatches to `lisa:jira-evidence`, `lisa:github-evidence`, or `lisa:linear-evidence` per `.lisa.config.json` `tracker`), including the list of codified tests added on this branch. **If the work is UI-visible** (any verification step ran in a browser, or the change touches a user-facing surface), author `evidence/comment.md` per the **UI Evidence Checklist** in `lisa:tracker-evidence` — numbered live-session steps, one Playwright-MCP screenshot per step uploaded to the GitHub `pr-assets` release as plain URLs, and an explicit invitation to be corrected.
|
|
44
47
|
|
|
@@ -35,10 +35,20 @@ For each required verification type, discover what tools are available in the pr
|
|
|
35
35
|
|
|
36
36
|
Report what is available for each required type. If a required type has no available tool, proceed to step 4.
|
|
37
37
|
|
|
38
|
+
If a required verification type needs sign-in or other credentials, exhaust credential sources before declaring the verification blocked. Check credential sources in this order:
|
|
39
|
+
|
|
40
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
41
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
42
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
43
|
+
|
|
44
|
+
Report which sources were checked. Do not say credentials are unavailable until all three source classes have been checked or proven absent.
|
|
45
|
+
|
|
38
46
|
### 4. Fail Fast
|
|
39
47
|
|
|
40
48
|
If a required verification type has no available tool and no reasonable alternative, escalate immediately using the Escalation Protocol. Do not begin implementation without a verification plan for every required type.
|
|
41
49
|
|
|
50
|
+
If credentials are genuinely unavailable after the credential lookup order above is exhausted, treat the work item as blocked rather than done. Post a clear tracker comment stating exactly what runtime behavior could not be verified and which credential sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating that label if the tracker supports label creation and it is missing.
|
|
51
|
+
|
|
42
52
|
### 5. Plan
|
|
43
53
|
|
|
44
54
|
For each verification type, state:
|
|
@@ -52,6 +62,8 @@ A verification plan that only lists `bun run test`, `bun run typecheck`, or `bun
|
|
|
52
62
|
|
|
53
63
|
After implementation, run the verification plan. Execute each verification type in order.
|
|
54
64
|
|
|
65
|
+
Evidence output must explicitly label each verification result as either `verified empirically` or `artifact-only / verification deferred`. Artifact-only evidence can support a blocked escalation packet, but it cannot mark a required runtime verification complete.
|
|
66
|
+
|
|
55
67
|
### 7. Codify
|
|
56
68
|
|
|
57
69
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
@@ -229,6 +241,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
229
241
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
230
242
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
231
243
|
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or empty artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
244
|
+
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
232
245
|
|
|
233
246
|
---
|
|
234
247
|
|
|
@@ -245,6 +258,7 @@ Common blockers:
|
|
|
245
258
|
- Missing documentation on how to access required tooling
|
|
246
259
|
- Production-only access gates
|
|
247
260
|
- Compliance restrictions
|
|
261
|
+
- Missing runtime credentials after checking project e2e / Playwright config and fixtures, `.lisa.config.local.json` / environment, and documented ticket credentials
|
|
248
262
|
|
|
249
263
|
When blocked, agents must do the following:
|
|
250
264
|
|
|
@@ -255,6 +269,8 @@ When blocked, agents must do the following:
|
|
|
255
269
|
5. Produce a Human Action Packet.
|
|
256
270
|
6. Pause until explicit human confirmation or tooling is provided.
|
|
257
271
|
|
|
272
|
+
For tracker-backed work items, also post the packet to the tracker, transition to the configured blocked state, and apply the configured `needs-human` / `human-review` label. If the label is missing and the tracker supports label creation, create it before applying it.
|
|
273
|
+
|
|
258
274
|
Agents must never proceed past an unverified boundary without surfacing it to the human overseer.
|
|
259
275
|
|
|
260
276
|
### Human Action Packet Format
|
|
@@ -39,6 +39,9 @@ Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via
|
|
|
39
39
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa:pull-request-review`
|
|
40
40
|
5. **Merge** when CI is green
|
|
41
41
|
6. **Remote verification** — invoke the `lisa:monitor` skill against the target environment to confirm the deploy actually works (health endpoints, recent logs/errors, Validation Journey replay if defined). If remote verification surfaces a behavioral gap that the existing codified tests do not guard, invoke `codify-verification` to add coverage and open a follow-up PR.
|
|
42
|
+
- When remote verification needs credentials, follow the shared `verification-lifecycle` credential lookup order before declaring them missing: project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as a `Sign-in Required` section.
|
|
43
|
+
- Should credentials remain genuinely unavailable after those sources are exhausted, do not complete the item on artifact-only evidence. Post a tracker comment stating what could not be verified and why, transition the work item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating it if the tracker supports label creation and it is missing.
|
|
44
|
+
- Evidence must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`.
|
|
42
45
|
7. **Evidence usage** — before posting, route the generated evidence artifact through `lisa:usage-accounting` so the comment body / PR evidence section / markdown proof carries a direct `verify` usage entry in the canonical `## Lisa Usage` section. If the originating work item or PRD parentage is known, prefer `record_and_rollup` so ancestor totals refresh in the same pass. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
43
46
|
8. **Evidence** — post results to the originating ticket via `lisa:tracker-evidence` (vendor-neutral; dispatches to `lisa:jira-evidence`, `lisa:github-evidence`, or `lisa:linear-evidence` per `.lisa.config.json` `tracker`), including the list of codified tests added on this branch. **If the work is UI-visible** (any verification step ran in a browser, or the change touches a user-facing surface), author `evidence/comment.md` per the **UI Evidence Checklist** in `lisa:tracker-evidence` — numbered live-session steps, one Playwright-MCP screenshot per step uploaded to the GitHub `pr-assets` release as plain URLs, and an explicit invitation to be corrected.
|
|
44
47
|
|
|
@@ -11,6 +11,8 @@
|
|
|
11
11
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
12
12
|
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes.
|
|
13
13
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
14
|
+
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
15
|
+
- **Never mark required runtime verification done on artifact-only evidence** — if credentials are genuinely unavailable after all sources are checked, post the blocker, transition the item to the configured blocked state, apply the configured `needs-human` / `human-review` label, and label the evidence as `artifact-only / verification deferred`.
|
|
14
16
|
|
|
15
17
|
## Roles
|
|
16
18
|
|
|
@@ -121,4 +121,18 @@ Both levels use the same verification types table above. The difference is the e
|
|
|
121
121
|
|
|
122
122
|
---
|
|
123
123
|
|
|
124
|
+
## Credential-Gated Verification
|
|
125
|
+
|
|
126
|
+
Some runtime verification requires signing in to a deployed or local app. Agents must exhaust credential sources before declaring verification blocked:
|
|
127
|
+
|
|
128
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
129
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
130
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
131
|
+
|
|
132
|
+
If credentials are genuinely unavailable after all three source classes are checked, the item is blocked, not done. The agent must post a tracker comment explaining what could not be verified and which sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating the label if the tracker supports label creation and it is missing.
|
|
133
|
+
|
|
134
|
+
Evidence and summaries must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`. Artifact-only evidence can explain what was checked before escalation, but it cannot complete a work item that requires runtime verification.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
124
138
|
For the full verification lifecycle (classify, check tooling, plan, execute, loop), surfaces, escalation protocol, and proof artifact requirements, see the `verification-lifecycle` skill loaded by the `verification-specialist` agent.
|
|
@@ -35,10 +35,20 @@ For each required verification type, discover what tools are available in the pr
|
|
|
35
35
|
|
|
36
36
|
Report what is available for each required type. If a required type has no available tool, proceed to step 4.
|
|
37
37
|
|
|
38
|
+
If a required verification type needs sign-in or other credentials, exhaust credential sources before declaring the verification blocked. Check credential sources in this order:
|
|
39
|
+
|
|
40
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
41
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
42
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
43
|
+
|
|
44
|
+
Report which sources were checked. Do not say credentials are unavailable until all three source classes have been checked or proven absent.
|
|
45
|
+
|
|
38
46
|
### 4. Fail Fast
|
|
39
47
|
|
|
40
48
|
If a required verification type has no available tool and no reasonable alternative, escalate immediately using the Escalation Protocol. Do not begin implementation without a verification plan for every required type.
|
|
41
49
|
|
|
50
|
+
If credentials are genuinely unavailable after the credential lookup order above is exhausted, treat the work item as blocked rather than done. Post a clear tracker comment stating exactly what runtime behavior could not be verified and which credential sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating that label if the tracker supports label creation and it is missing.
|
|
51
|
+
|
|
42
52
|
### 5. Plan
|
|
43
53
|
|
|
44
54
|
For each verification type, state:
|
|
@@ -52,6 +62,8 @@ A verification plan that only lists `bun run test`, `bun run typecheck`, or `bun
|
|
|
52
62
|
|
|
53
63
|
After implementation, run the verification plan. Execute each verification type in order.
|
|
54
64
|
|
|
65
|
+
Evidence output must explicitly label each verification result as either `verified empirically` or `artifact-only / verification deferred`. Artifact-only evidence can support a blocked escalation packet, but it cannot mark a required runtime verification complete.
|
|
66
|
+
|
|
55
67
|
### 7. Codify
|
|
56
68
|
|
|
57
69
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
@@ -229,6 +241,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
229
241
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
230
242
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
231
243
|
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or empty artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
244
|
+
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
232
245
|
|
|
233
246
|
---
|
|
234
247
|
|
|
@@ -245,6 +258,7 @@ Common blockers:
|
|
|
245
258
|
- Missing documentation on how to access required tooling
|
|
246
259
|
- Production-only access gates
|
|
247
260
|
- Compliance restrictions
|
|
261
|
+
- Missing runtime credentials after checking project e2e / Playwright config and fixtures, `.lisa.config.local.json` / environment, and documented ticket credentials
|
|
248
262
|
|
|
249
263
|
When blocked, agents must do the following:
|
|
250
264
|
|
|
@@ -255,6 +269,8 @@ When blocked, agents must do the following:
|
|
|
255
269
|
5. Produce a Human Action Packet.
|
|
256
270
|
6. Pause until explicit human confirmation or tooling is provided.
|
|
257
271
|
|
|
272
|
+
For tracker-backed work items, also post the packet to the tracker, transition to the configured blocked state, and apply the configured `needs-human` / `human-review` label. If the label is missing and the tracker supports label creation, create it before applying it.
|
|
273
|
+
|
|
258
274
|
Agents must never proceed past an unverified boundary without surfacing it to the human overseer.
|
|
259
275
|
|
|
260
276
|
### Human Action Packet Format
|
|
@@ -39,6 +39,9 @@ Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via
|
|
|
39
39
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa:pull-request-review`
|
|
40
40
|
5. **Merge** when CI is green
|
|
41
41
|
6. **Remote verification** — invoke the `lisa:monitor` skill against the target environment to confirm the deploy actually works (health endpoints, recent logs/errors, Validation Journey replay if defined). If remote verification surfaces a behavioral gap that the existing codified tests do not guard, invoke `codify-verification` to add coverage and open a follow-up PR.
|
|
42
|
+
- When remote verification needs credentials, follow the shared `verification-lifecycle` credential lookup order before declaring them missing: project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as a `Sign-in Required` section.
|
|
43
|
+
- Should credentials remain genuinely unavailable after those sources are exhausted, do not complete the item on artifact-only evidence. Post a tracker comment stating what could not be verified and why, transition the work item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating it if the tracker supports label creation and it is missing.
|
|
44
|
+
- Evidence must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`.
|
|
42
45
|
7. **Evidence usage** — before posting, route the generated evidence artifact through `lisa:usage-accounting` so the comment body / PR evidence section / markdown proof carries a direct `verify` usage entry in the canonical `## Lisa Usage` section. If the originating work item or PRD parentage is known, prefer `record_and_rollup` so ancestor totals refresh in the same pass. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
43
46
|
8. **Evidence** — post results to the originating ticket via `lisa:tracker-evidence` (vendor-neutral; dispatches to `lisa:jira-evidence`, `lisa:github-evidence`, or `lisa:linear-evidence` per `.lisa.config.json` `tracker`), including the list of codified tests added on this branch. **If the work is UI-visible** (any verification step ran in a browser, or the change touches a user-facing surface), author `evidence/comment.md` per the **UI Evidence Checklist** in `lisa:tracker-evidence` — numbered live-session steps, one Playwright-MCP screenshot per step uploaded to the GitHub `pr-assets` release as plain URLs, and an explicit invitation to be corrected.
|
|
44
47
|
|
|
@@ -126,4 +126,18 @@ Both levels use the same verification types table above. The difference is the e
|
|
|
126
126
|
|
|
127
127
|
---
|
|
128
128
|
|
|
129
|
+
## Credential-Gated Verification
|
|
130
|
+
|
|
131
|
+
Some runtime verification requires signing in to a deployed or local app. Agents must exhaust credential sources before declaring verification blocked:
|
|
132
|
+
|
|
133
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
134
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
135
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
136
|
+
|
|
137
|
+
If credentials are genuinely unavailable after all three source classes are checked, the item is blocked, not done. The agent must post a tracker comment explaining what could not be verified and which sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating the label if the tracker supports label creation and it is missing.
|
|
138
|
+
|
|
139
|
+
Evidence and summaries must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`. Artifact-only evidence can explain what was checked before escalation, but it cannot complete a work item that requires runtime verification.
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
129
143
|
For the full verification lifecycle (classify, check tooling, plan, execute, loop), surfaces, escalation protocol, and proof artifact requirements, see the `verification-lifecycle` skill loaded by the `verification-specialist` agent.
|
|
@@ -16,6 +16,8 @@ alwaysApply: true
|
|
|
16
16
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
17
17
|
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes.
|
|
18
18
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
19
|
+
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
20
|
+
- **Never mark required runtime verification done on artifact-only evidence** — if credentials are genuinely unavailable after all sources are checked, post the blocker, transition the item to the configured blocked state, apply the configured `needs-human` / `human-review` label, and label the evidence as `artifact-only / verification deferred`.
|
|
19
21
|
|
|
20
22
|
## Roles
|
|
21
23
|
|
|
@@ -35,10 +35,20 @@ For each required verification type, discover what tools are available in the pr
|
|
|
35
35
|
|
|
36
36
|
Report what is available for each required type. If a required type has no available tool, proceed to step 4.
|
|
37
37
|
|
|
38
|
+
If a required verification type needs sign-in or other credentials, exhaust credential sources before declaring the verification blocked. Check credential sources in this order:
|
|
39
|
+
|
|
40
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
41
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
42
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
43
|
+
|
|
44
|
+
Report which sources were checked. Do not say credentials are unavailable until all three source classes have been checked or proven absent.
|
|
45
|
+
|
|
38
46
|
### 4. Fail Fast
|
|
39
47
|
|
|
40
48
|
If a required verification type has no available tool and no reasonable alternative, escalate immediately using the Escalation Protocol. Do not begin implementation without a verification plan for every required type.
|
|
41
49
|
|
|
50
|
+
If credentials are genuinely unavailable after the credential lookup order above is exhausted, treat the work item as blocked rather than done. Post a clear tracker comment stating exactly what runtime behavior could not be verified and which credential sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating that label if the tracker supports label creation and it is missing.
|
|
51
|
+
|
|
42
52
|
### 5. Plan
|
|
43
53
|
|
|
44
54
|
For each verification type, state:
|
|
@@ -52,6 +62,8 @@ A verification plan that only lists `bun run test`, `bun run typecheck`, or `bun
|
|
|
52
62
|
|
|
53
63
|
After implementation, run the verification plan. Execute each verification type in order.
|
|
54
64
|
|
|
65
|
+
Evidence output must explicitly label each verification result as either `verified empirically` or `artifact-only / verification deferred`. Artifact-only evidence can support a blocked escalation packet, but it cannot mark a required runtime verification complete.
|
|
66
|
+
|
|
55
67
|
### 7. Codify
|
|
56
68
|
|
|
57
69
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
@@ -229,6 +241,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
229
241
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
230
242
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
231
243
|
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or empty artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
244
|
+
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
232
245
|
|
|
233
246
|
---
|
|
234
247
|
|
|
@@ -245,6 +258,7 @@ Common blockers:
|
|
|
245
258
|
- Missing documentation on how to access required tooling
|
|
246
259
|
- Production-only access gates
|
|
247
260
|
- Compliance restrictions
|
|
261
|
+
- Missing runtime credentials after checking project e2e / Playwright config and fixtures, `.lisa.config.local.json` / environment, and documented ticket credentials
|
|
248
262
|
|
|
249
263
|
When blocked, agents must do the following:
|
|
250
264
|
|
|
@@ -255,6 +269,8 @@ When blocked, agents must do the following:
|
|
|
255
269
|
5. Produce a Human Action Packet.
|
|
256
270
|
6. Pause until explicit human confirmation or tooling is provided.
|
|
257
271
|
|
|
272
|
+
For tracker-backed work items, also post the packet to the tracker, transition to the configured blocked state, and apply the configured `needs-human` / `human-review` label. If the label is missing and the tracker supports label creation, create it before applying it.
|
|
273
|
+
|
|
258
274
|
Agents must never proceed past an unverified boundary without surfacing it to the human overseer.
|
|
259
275
|
|
|
260
276
|
### Human Action Packet Format
|
|
@@ -39,6 +39,9 @@ Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via
|
|
|
39
39
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa:pull-request-review`
|
|
40
40
|
5. **Merge** when CI is green
|
|
41
41
|
6. **Remote verification** — invoke the `lisa:monitor` skill against the target environment to confirm the deploy actually works (health endpoints, recent logs/errors, Validation Journey replay if defined). If remote verification surfaces a behavioral gap that the existing codified tests do not guard, invoke `codify-verification` to add coverage and open a follow-up PR.
|
|
42
|
+
- When remote verification needs credentials, follow the shared `verification-lifecycle` credential lookup order before declaring them missing: project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as a `Sign-in Required` section.
|
|
43
|
+
- Should credentials remain genuinely unavailable after those sources are exhausted, do not complete the item on artifact-only evidence. Post a tracker comment stating what could not be verified and why, transition the work item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating it if the tracker supports label creation and it is missing.
|
|
44
|
+
- Evidence must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`.
|
|
42
45
|
7. **Evidence usage** — before posting, route the generated evidence artifact through `lisa:usage-accounting` so the comment body / PR evidence section / markdown proof carries a direct `verify` usage entry in the canonical `## Lisa Usage` section. If the originating work item or PRD parentage is known, prefer `record_and_rollup` so ancestor totals refresh in the same pass. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
43
46
|
8. **Evidence** — post results to the originating ticket via `lisa:tracker-evidence` (vendor-neutral; dispatches to `lisa:jira-evidence`, `lisa:github-evidence`, or `lisa:linear-evidence` per `.lisa.config.json` `tracker`), including the list of codified tests added on this branch. **If the work is UI-visible** (any verification step ran in a browser, or the change touches a user-facing surface), author `evidence/comment.md` per the **UI Evidence Checklist** in `lisa:tracker-evidence` — numbered live-session steps, one Playwright-MCP screenshot per step uploaded to the GitHub `pr-assets` release as plain URLs, and an explicit invitation to be corrected.
|
|
44
47
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-openclaw",
|
|
3
|
-
"version": "2.159.
|
|
3
|
+
"version": "2.159.4",
|
|
4
4
|
"description": "Connect staff roles to Telegram or Slack via OpenClaw — facilitator/specialist hub-and-spoke routing and repo-coding topics, for Claude Code and Codex",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-openclaw",
|
|
3
|
-
"version": "2.159.
|
|
3
|
+
"version": "2.159.4",
|
|
4
4
|
"description": "Connect staff roles to Telegram or Slack via OpenClaw — facilitator/specialist hub-and-spoke routing and repo-coding topics, across Claude and Codex.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-openclaw",
|
|
3
|
-
"version": "2.159.
|
|
3
|
+
"version": "2.159.4",
|
|
4
4
|
"description": "Connect staff roles to Telegram or Slack via OpenClaw — facilitator/specialist hub-and-spoke routing and repo-coding topics, for Claude Code and Codex",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-openclaw",
|
|
3
|
-
"version": "2.159.
|
|
3
|
+
"version": "2.159.4",
|
|
4
4
|
"description": "Connect staff roles to Telegram or Slack via OpenClaw — facilitator/specialist hub-and-spoke routing and repo-coding topics, for Claude Code and Codex",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-openclaw",
|
|
3
|
-
"version": "2.159.
|
|
3
|
+
"version": "2.159.4",
|
|
4
4
|
"description": "Connect staff roles to Telegram or Slack via OpenClaw — facilitator/specialist hub-and-spoke routing and repo-coding topics, for Claude Code and Codex",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -11,6 +11,8 @@
|
|
|
11
11
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
12
12
|
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes.
|
|
13
13
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
14
|
+
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
15
|
+
- **Never mark required runtime verification done on artifact-only evidence** — if credentials are genuinely unavailable after all sources are checked, post the blocker, transition the item to the configured blocked state, apply the configured `needs-human` / `human-review` label, and label the evidence as `artifact-only / verification deferred`.
|
|
14
16
|
|
|
15
17
|
## Roles
|
|
16
18
|
|
|
@@ -121,4 +121,18 @@ Both levels use the same verification types table above. The difference is the e
|
|
|
121
121
|
|
|
122
122
|
---
|
|
123
123
|
|
|
124
|
+
## Credential-Gated Verification
|
|
125
|
+
|
|
126
|
+
Some runtime verification requires signing in to a deployed or local app. Agents must exhaust credential sources before declaring verification blocked:
|
|
127
|
+
|
|
128
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
129
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
130
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
131
|
+
|
|
132
|
+
If credentials are genuinely unavailable after all three source classes are checked, the item is blocked, not done. The agent must post a tracker comment explaining what could not be verified and which sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating the label if the tracker supports label creation and it is missing.
|
|
133
|
+
|
|
134
|
+
Evidence and summaries must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`. Artifact-only evidence can explain what was checked before escalation, but it cannot complete a work item that requires runtime verification.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
124
138
|
For the full verification lifecycle (classify, check tooling, plan, execute, loop), surfaces, escalation protocol, and proof artifact requirements, see the `verification-lifecycle` skill loaded by the `verification-specialist` agent.
|
|
@@ -35,10 +35,20 @@ For each required verification type, discover what tools are available in the pr
|
|
|
35
35
|
|
|
36
36
|
Report what is available for each required type. If a required type has no available tool, proceed to step 4.
|
|
37
37
|
|
|
38
|
+
If a required verification type needs sign-in or other credentials, exhaust credential sources before declaring the verification blocked. Check credential sources in this order:
|
|
39
|
+
|
|
40
|
+
1. Project e2e / Playwright config and fixtures, including files such as `e2e/constants.ts`, `e2e/fixtures/api-login.ts`, seeded test users, and OTP-bypass patterns such as `555555`.
|
|
41
|
+
2. `.lisa.config.local.json` and environment variables.
|
|
42
|
+
3. Documented ticket credentials, including a `Sign-in Required` or equivalent section in the issue, ticket, PRD, or linked implementation notes.
|
|
43
|
+
|
|
44
|
+
Report which sources were checked. Do not say credentials are unavailable until all three source classes have been checked or proven absent.
|
|
45
|
+
|
|
38
46
|
### 4. Fail Fast
|
|
39
47
|
|
|
40
48
|
If a required verification type has no available tool and no reasonable alternative, escalate immediately using the Escalation Protocol. Do not begin implementation without a verification plan for every required type.
|
|
41
49
|
|
|
50
|
+
If credentials are genuinely unavailable after the credential lookup order above is exhausted, treat the work item as blocked rather than done. Post a clear tracker comment stating exactly what runtime behavior could not be verified and which credential sources were checked, transition the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating that label if the tracker supports label creation and it is missing.
|
|
51
|
+
|
|
42
52
|
### 5. Plan
|
|
43
53
|
|
|
44
54
|
For each verification type, state:
|
|
@@ -52,6 +62,8 @@ A verification plan that only lists `bun run test`, `bun run typecheck`, or `bun
|
|
|
52
62
|
|
|
53
63
|
After implementation, run the verification plan. Execute each verification type in order.
|
|
54
64
|
|
|
65
|
+
Evidence output must explicitly label each verification result as either `verified empirically` or `artifact-only / verification deferred`. Artifact-only evidence can support a blocked escalation packet, but it cannot mark a required runtime verification complete.
|
|
66
|
+
|
|
55
67
|
### 7. Codify
|
|
56
68
|
|
|
57
69
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
@@ -229,6 +241,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
229
241
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
230
242
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
231
243
|
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or empty artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
244
|
+
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
232
245
|
|
|
233
246
|
---
|
|
234
247
|
|
|
@@ -245,6 +258,7 @@ Common blockers:
|
|
|
245
258
|
- Missing documentation on how to access required tooling
|
|
246
259
|
- Production-only access gates
|
|
247
260
|
- Compliance restrictions
|
|
261
|
+
- Missing runtime credentials after checking project e2e / Playwright config and fixtures, `.lisa.config.local.json` / environment, and documented ticket credentials
|
|
248
262
|
|
|
249
263
|
When blocked, agents must do the following:
|
|
250
264
|
|
|
@@ -255,6 +269,8 @@ When blocked, agents must do the following:
|
|
|
255
269
|
5. Produce a Human Action Packet.
|
|
256
270
|
6. Pause until explicit human confirmation or tooling is provided.
|
|
257
271
|
|
|
272
|
+
For tracker-backed work items, also post the packet to the tracker, transition to the configured blocked state, and apply the configured `needs-human` / `human-review` label. If the label is missing and the tracker supports label creation, create it before applying it.
|
|
273
|
+
|
|
258
274
|
Agents must never proceed past an unverified boundary without surfacing it to the human overseer.
|
|
259
275
|
|
|
260
276
|
### Human Action Packet Format
|
|
@@ -39,6 +39,9 @@ Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via
|
|
|
39
39
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa:pull-request-review`
|
|
40
40
|
5. **Merge** when CI is green
|
|
41
41
|
6. **Remote verification** — invoke the `lisa:monitor` skill against the target environment to confirm the deploy actually works (health endpoints, recent logs/errors, Validation Journey replay if defined). If remote verification surfaces a behavioral gap that the existing codified tests do not guard, invoke `codify-verification` to add coverage and open a follow-up PR.
|
|
42
|
+
- When remote verification needs credentials, follow the shared `verification-lifecycle` credential lookup order before declaring them missing: project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as a `Sign-in Required` section.
|
|
43
|
+
- Should credentials remain genuinely unavailable after those sources are exhausted, do not complete the item on artifact-only evidence. Post a tracker comment stating what could not be verified and why, transition the work item to the configured blocked state, and apply the configured `needs-human` / `human-review` label, creating it if the tracker supports label creation and it is missing.
|
|
44
|
+
- Evidence must explicitly distinguish `verified empirically` from `artifact-only / verification deferred`.
|
|
42
45
|
7. **Evidence usage** — before posting, route the generated evidence artifact through `lisa:usage-accounting` so the comment body / PR evidence section / markdown proof carries a direct `verify` usage entry in the canonical `## Lisa Usage` section. If the originating work item or PRD parentage is known, prefer `record_and_rollup` so ancestor totals refresh in the same pass. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
43
46
|
8. **Evidence** — post results to the originating ticket via `lisa:tracker-evidence` (vendor-neutral; dispatches to `lisa:jira-evidence`, `lisa:github-evidence`, or `lisa:linear-evidence` per `.lisa.config.json` `tracker`), including the list of codified tests added on this branch. **If the work is UI-visible** (any verification step ran in a browser, or the change touches a user-facing surface), author `evidence/comment.md` per the **UI Evidence Checklist** in `lisa:tracker-evidence` — numbered live-session steps, one Playwright-MCP screenshot per step uploaded to the GitHub `pr-assets` release as plain URLs, and an explicit invitation to be corrected.
|
|
44
47
|
|