@codyswann/lisa 2.212.0 → 2.214.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/lisa/.codex-plugin/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/lisa/.codex-plugin/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/lisa/.codex-plugin/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/lisa/.codex-plugin/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/lisa/.codex-plugin/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/lisa/.codex-plugin/skills/lisa-linear-journey/SKILL.md +15 -4
- package/plugins/lisa/.codex-plugin/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-research/SKILL.md +4 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/lisa/.codex-plugin/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa/rules/eager/observability-audit.md +1 -1
- package/plugins/lisa/rules/eager/verification.md +2 -1
- package/plugins/lisa/rules/reference/intent-routing.md +1 -1
- package/plugins/lisa/rules/reference/observability-audit.md +1 -1
- package/plugins/lisa/rules/reference/verification.md +43 -5
- package/plugins/lisa/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/lisa/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/lisa/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/lisa/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/lisa/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/lisa/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/lisa/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/lisa/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/lisa/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/lisa/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/lisa/skills/lisa-linear-journey/SKILL.md +16 -5
- package/plugins/lisa/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/lisa/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/lisa/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/lisa/skills/lisa-research/SKILL.md +4 -2
- package/plugins/lisa/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/lisa/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/lisa/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/lisa/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/lisa-agy/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/lisa-agy/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/lisa-agy/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-agy/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-agy/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/lisa-agy/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/lisa-agy/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/lisa-agy/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/lisa-agy/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/lisa-agy/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/lisa-agy/skills/lisa-linear-journey/SKILL.md +16 -5
- package/plugins/lisa-agy/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-agy/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/lisa-agy/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/lisa-agy/skills/lisa-research/SKILL.md +4 -2
- package/plugins/lisa-agy/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/lisa-agy/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/lisa-agy/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/lisa-agy/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/rules/eager/observability-audit.md +1 -1
- package/plugins/lisa-copilot/rules/eager/verification.md +2 -1
- package/plugins/lisa-copilot/rules/reference/intent-routing.md +1 -1
- package/plugins/lisa-copilot/rules/reference/observability-audit.md +1 -1
- package/plugins/lisa-copilot/rules/reference/verification.md +43 -5
- package/plugins/lisa-copilot/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/lisa-copilot/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/lisa-copilot/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/lisa-copilot/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-copilot/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-copilot/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/lisa-copilot/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/lisa-copilot/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/lisa-copilot/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/lisa-copilot/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/lisa-copilot/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/lisa-copilot/skills/lisa-linear-journey/SKILL.md +16 -5
- package/plugins/lisa-copilot/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-copilot/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/lisa-copilot/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/lisa-copilot/skills/lisa-research/SKILL.md +4 -2
- package/plugins/lisa-copilot/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/lisa-copilot/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/lisa-copilot/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/lisa-copilot/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/rules/intent-routing-reference.mdc +1 -1
- package/plugins/lisa-cursor/rules/observability-audit-reference.mdc +1 -1
- package/plugins/lisa-cursor/rules/observability-audit.mdc +1 -1
- package/plugins/lisa-cursor/rules/verification-reference.mdc +43 -5
- package/plugins/lisa-cursor/rules/verification.mdc +2 -1
- package/plugins/lisa-cursor/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/lisa-cursor/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/lisa-cursor/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/lisa-cursor/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-cursor/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-cursor/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/lisa-cursor/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/lisa-cursor/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/lisa-cursor/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/lisa-cursor/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/lisa-cursor/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/lisa-cursor/skills/lisa-linear-journey/SKILL.md +16 -5
- package/plugins/lisa-cursor/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/lisa-cursor/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/lisa-cursor/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/lisa-cursor/skills/lisa-research/SKILL.md +4 -2
- package/plugins/lisa-cursor/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/lisa-cursor/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/lisa-cursor/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/lisa-cursor/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/skills/jira-journey/SKILL.md +1 -1
- package/plugins/lisa-rails/skills/jira-journey/SKILL.md +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-agy/skills/jira-journey/SKILL.md +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/skills/jira-journey/SKILL.md +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/skills/jira-journey/SKILL.md +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/rules/eager/observability-audit.md +1 -1
- package/plugins/src/base/rules/eager/verification.md +2 -1
- package/plugins/src/base/rules/reference/intent-routing.md +1 -1
- package/plugins/src/base/rules/reference/observability-audit.md +1 -1
- package/plugins/src/base/rules/reference/verification.md +43 -5
- package/plugins/src/base/skills/lisa-codify-verification/SKILL.md +17 -0
- package/plugins/src/base/skills/lisa-github-add-journey/SKILL.md +6 -6
- package/plugins/src/base/skills/lisa-github-journey/SKILL.md +13 -2
- package/plugins/src/base/skills/lisa-github-validate-issue/SKILL.md +9 -2
- package/plugins/src/base/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/src/base/skills/lisa-jira-add-journey/SKILL.md +6 -6
- package/plugins/src/base/skills/lisa-jira-create/SKILL.md +5 -5
- package/plugins/src/base/skills/lisa-jira-journey/SKILL.md +15 -4
- package/plugins/src/base/skills/lisa-jira-validate-ticket/SKILL.md +9 -2
- package/plugins/src/base/skills/lisa-linear-add-journey/SKILL.md +7 -5
- package/plugins/src/base/skills/lisa-linear-create/SKILL.md +4 -4
- package/plugins/src/base/skills/lisa-linear-journey/SKILL.md +16 -5
- package/plugins/src/base/skills/lisa-linear-validate-issue/SKILL.md +9 -2
- package/plugins/src/base/skills/lisa-monitor/SKILL.md +1 -1
- package/plugins/src/base/skills/lisa-project-ideation/examples/idempotency-verification-harness.md +2 -2
- package/plugins/src/base/skills/lisa-research/SKILL.md +4 -2
- package/plugins/src/base/skills/lisa-tracker-add-journey/SKILL.md +1 -1
- package/plugins/src/base/skills/lisa-tracker-evidence/SKILL.md +1 -1
- package/plugins/src/base/skills/lisa-verification-lifecycle/SKILL.md +4 -4
- package/plugins/src/base/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/src/rails/skills/jira-journey/SKILL.md +1 -1
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: lisa-linear-journey
|
|
3
|
-
description: "Parse a Linear Issue's Validation Journey section, execute the verification steps using appropriate tools (curl, test commands, database queries, Playwright), capture evidence at each [EVIDENCE: name] marker, and post to Linear + GitHub PR using the linear-evidence skill. Linear counterpart of lisa-jira-journey."
|
|
3
|
+
description: "Parse a Linear Issue's Validation Journey section, execute the verification steps using appropriate tools (curl, test commands, database queries, Playwright), capture evidence at each typed [EVIDENCE: <artifact-type>: <name>] marker, and post to Linear + GitHub PR using the linear-evidence skill. Linear counterpart of lisa-jira-journey."
|
|
4
4
|
allowed-tools: ["Bash", "Read", "Glob", "Grep", "Skill"]
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Linear Validation Journey
|
|
8
8
|
|
|
9
|
-
Parse a Linear Issue's Validation Journey, execute the verification steps using the appropriate tools for the change type, capture evidence at each `[EVIDENCE: name]` marker, and post to Linear + GitHub PR.
|
|
9
|
+
Parse a Linear Issue's Validation Journey, execute the verification steps using the appropriate tools for the change type, capture evidence at each typed `[EVIDENCE: <artifact-type>: <name>]` marker, and post to Linear + GitHub PR.
|
|
10
10
|
|
|
11
11
|
This skill is the destination of the `lisa-tracker-journey` shim when `tracker = "linear"`.
|
|
12
12
|
|
|
@@ -34,7 +34,7 @@ Reads `linear.workspace`, `linear.teamKey` from `.lisa.config.json` (with `.loca
|
|
|
34
34
|
Fetch the Issue via `lisa-linear-access operation: get-issue` and extract the `## Validation Journey` section from the markdown description. Parse:
|
|
35
35
|
|
|
36
36
|
- `### Prerequisites` — list of required services / env / setup
|
|
37
|
-
- `### Steps` — numbered steps, each potentially containing `[EVIDENCE: name]` markers
|
|
37
|
+
- `### Steps` — numbered steps, each potentially containing typed `[EVIDENCE: <artifact-type>: <name>]` markers
|
|
38
38
|
- `### Assertions` — what must be true after verification
|
|
39
39
|
|
|
40
40
|
If the section is missing or has no steps, report `"No Validation Journey on <IDENTIFIER>. Run /linear-add-journey first."` and stop.
|
|
@@ -61,14 +61,25 @@ Execute each step sequentially. Determine the verification approach based on the
|
|
|
61
61
|
- **Security fixes** → Reproduce exploit attempt, verify fix
|
|
62
62
|
- **UI / frontend** → Playwright browser flow, capture screenshots / DOM state
|
|
63
63
|
|
|
64
|
-
At each `[EVIDENCE: name]` marker, capture
|
|
64
|
+
At each typed `[EVIDENCE: <artifact-type>: <name>]` marker, capture an artifact **of the declared type** — the type is the contract, not a suggestion:
|
|
65
|
+
|
|
66
|
+
- `screenshot` / `recording` → an actual image/video file from the driven UI (Playwright, simulator), never a text description of what was seen
|
|
67
|
+
- `http-transcript` → the exact request (curl command or client call) plus the full response
|
|
68
|
+
- `cli-output` → the command plus stdout/stderr and exit code
|
|
69
|
+
- `log-snippet` → the correlated log lines pulled from the running system
|
|
70
|
+
- `db-query-output` → the query plus returned rows
|
|
71
|
+
- `perf-trace` → the benchmark/frame-timing/profiler output with methodology (device profile, dataset size)
|
|
72
|
+
- `test-run-log` → reporter output naming the spec and showing it ran and passed
|
|
73
|
+
- `deploy-log` / `state-dump` → the deployment/health-check output or observed-state JSON
|
|
74
|
+
|
|
75
|
+
A prose claim ("the error state rendered gracefully") satisfies no marker. Legacy untyped markers: infer the type from the step's action, capture accordingly, and note the inference. Write each artifact to a numbered file:
|
|
65
76
|
|
|
66
77
|
#### Evidence Naming Convention
|
|
67
78
|
|
|
68
79
|
`{NN}-{evidence-name}.{ext}`
|
|
69
80
|
|
|
70
81
|
- `NN`: zero-padded sequential number (`01`, `02`, `03`...)
|
|
71
|
-
- `evidence-name`: the
|
|
82
|
+
- `evidence-name`: the `<name>` part of the typed marker
|
|
72
83
|
- `ext`: `.txt` for plain output, `.json` for structured data
|
|
73
84
|
|
|
74
85
|
Example:
|
|
@@ -200,9 +200,16 @@ An item with zero relations and no documented search: FAIL.
|
|
|
200
200
|
|
|
201
201
|
#### S14 — Evidence manifest binding (leaf work units)
|
|
202
202
|
|
|
203
|
-
When `issue_type ∈ {Bug, Task, Sub-task, Improvement}` AND `runtime_behavior_change = true`, the `## Validation Journey` must declare at least one `[EVIDENCE: name]` marker.
|
|
203
|
+
When `issue_type ∈ {Bug, Task, Sub-task, Improvement}` AND `runtime_behavior_change = true`, the `## Validation Journey` must declare at least one **typed** `[EVIDENCE: <artifact-type>: <name>]` marker. These markers are the work unit's **evidence manifest** — the exact, enumerated set of artifacts that must be captured and attached before the item may be closed (see the "Per-Work-Unit Evidence Contract" section of the `verification` rule, the Definition of Done in `verification-lifecycle`, and the evidence-manifest gate in `tracker-evidence`).
|
|
204
204
|
|
|
205
|
-
|
|
205
|
+
Each marker must satisfy ALL of:
|
|
206
|
+
|
|
207
|
+
- `<artifact-type>` is one of the fixed taxonomy: `screenshot`, `recording`, `http-transcript`, `cli-output`, `log-snippet`, `db-query-output`, `perf-trace`, `test-run-log`, `deploy-log`, `state-dump`. (The legacy `[SCREENSHOT: name]` form is accepted as `screenshot`.)
|
|
208
|
+
- `<name>` is kebab-case and unique within the item.
|
|
209
|
+
|
|
210
|
+
**A marker names an artifact, not an assertion.** An untyped marker (`[EVIDENCE: load-failure-handled-gracefully]`) is an assertion label with nothing to capture and must FAIL, with a remediation that shows the typed transformation (e.g. → `[EVIDENCE: screenshot: load-failure-error-state]`, `[EVIDENCE: perf-trace: pipeline-load-tti]`).
|
|
211
|
+
|
|
212
|
+
FAIL when the Validation Journey is present but declares zero markers, when any marker is untyped or uses a type outside the taxonomy, or when any name is empty, duplicated, or not kebab-case. A behavior-changing work unit SHOULD declare both a success marker and an error/edge marker; a journey with only one marker passes but the remediation should recommend adding the error/edge case.
|
|
206
213
|
|
|
207
214
|
This gate depends on S11. It is `N/A` for containers — a **Project** (the Epic equivalent), or any item with open child work (coordination containers, not work units) — and for leaf units with `runtime_behavior_change = false` (doc-only / config-only / type-only). If S11 fails because the Validation Journey is absent, S14 also FAILs (there is no manifest to bind) with remediation pointing back to `lisa-linear-add-journey`.
|
|
208
215
|
|
|
@@ -59,7 +59,7 @@ The **Rails** ops-specialist composes a different subset (`ops-run-local`, `ops-
|
|
|
59
59
|
After report, file what was found — **only when run standalone**, never under `--report-only`/`--dry-run` and never when nested inside `lisa-verify` (which passes `--report-only`):
|
|
60
60
|
|
|
61
61
|
- **Anomalies** (live signals over the conservative bar) → `Bug` leaves. **Gaps** (in-scope MISSING rubric dimensions) → `Task`/`Improvement` leaves.
|
|
62
|
-
- Every ticket is filed via the vendor-neutral `lisa-tracker-write` shim with `build_ready: true` (never a vendor write skill directly), as a **single-repo leaf** stamped `repo:<current>`, with a real three-audience description, Gherkin AC, Target Backend Environment, and a Validation Journey + `EVIDENCE
|
|
62
|
+
- Every ticket is filed via the vendor-neutral `lisa-tracker-write` shim with `build_ready: true` (never a vendor write skill directly), as a **single-repo leaf** stamped `repo:<current>`, with a real three-audience description, Gherkin AC, Target Backend Environment, and a Validation Journey + typed `[EVIDENCE: <artifact-type>: <name>]` marker (e.g. `[EVIDENCE: log-snippet: alert-cleared]`) so it passes the `tracker-validate` gates.
|
|
63
63
|
- **Idempotent:** embed the `<!-- lisa:monitor-finding: <fingerprint> -->` sentinel and search-before-create; never duplicate a live or just-resolved finding.
|
|
64
64
|
- **Capped** at `max_candidates` (default 20), `core`/high-severity first; report how many were filed vs dropped.
|
|
65
65
|
- **`--dry-run`** previews would-file tickets and creates nothing. **`--all-gaps`** widens gap filing to `recommended` tiers.
|
|
@@ -50,8 +50,8 @@ The harness performs the acceptance check in three phases:
|
|
|
50
50
|
|
|
51
51
|
Capture the harness JSON output as:
|
|
52
52
|
|
|
53
|
-
- `[EVIDENCE: marker-count-one]` from the first and second marker-count checks.
|
|
54
|
-
- `[EVIDENCE: memory-recreated-after-rerun]` from the missing-memory variant.
|
|
53
|
+
- `[EVIDENCE: cli-output: marker-count-one]` from the first and second marker-count checks.
|
|
54
|
+
- `[EVIDENCE: cli-output: memory-recreated-after-rerun]` from the missing-memory variant.
|
|
55
55
|
|
|
56
56
|
The run passes only when all reported counts are `1`, the issue URL is the same across phases, and
|
|
57
57
|
`memoryRecreated` and `memoryFieldsRecorded` are `true` when `--memory-file` is supplied.
|
|
@@ -53,8 +53,10 @@ Execute the **Research** flow as defined in the `intent-routing` rule (loaded vi
|
|
|
53
53
|
## Output
|
|
54
54
|
|
|
55
55
|
A PRD **created in the configured PRD source** (per the intent-routing rule's Research flow
|
|
56
|
-
definition)
|
|
57
|
-
|
|
56
|
+
definition) structured as: problem statement, high-level solution description, links (if needed),
|
|
57
|
+
user stories (each with its own functional/non-functional requirements and, only for stories with
|
|
58
|
+
new UI/visual work, a design-file pointer), overall acceptance criteria, open questions, and the
|
|
59
|
+
"Recommended Tooling for Plan Phase" section. The final
|
|
58
60
|
flow step invokes `lisa-prd-source-write`, which creates the PRD in the configured `source` (Notion
|
|
59
61
|
page in the PRD database, Confluence page under the lifecycle parent, GitHub issue, or Linear
|
|
60
62
|
project) in the `draft` role by default or `ready` when `prd_ready=true`. **The PRD lives in the
|
|
@@ -23,5 +23,5 @@ See the `config-resolution` rule for configuration and dispatch table.
|
|
|
23
23
|
|
|
24
24
|
## Rules
|
|
25
25
|
|
|
26
|
-
- The Validation Journey content format is identical across all vendors (markdown sections with `[EVIDENCE: name]` markers). The only difference is how the section is appended — JIRA via `editJiraIssue` (Jira wiki markup), GitHub via `gh issue edit --body-file` (markdown), Linear via `save_issue` (markdown).
|
|
26
|
+
- The Validation Journey content format is identical across all vendors (markdown sections with typed `[EVIDENCE: <artifact-type>: <name>]` markers per the `verification` rule taxonomy). The only difference is how the section is appended — JIRA via `editJiraIssue` (Jira wiki markup), GitHub via `gh issue edit --body-file` (markdown), Linear via `save_issue` (markdown).
|
|
27
27
|
- If the ticket already has a Validation Journey, the vendor skill reports it and stops. This shim does not retry.
|
|
@@ -27,7 +27,7 @@ See the `config-resolution` rule for configuration and dispatch table.
|
|
|
27
27
|
- The GitHub `pr-assets` release lives on the implementation repo (the one with the PR), regardless of which tracker hosts the ticket/issue. All vendor skills upload there.
|
|
28
28
|
- Never post evidence to a different ticket than the one named — `$ARGUMENTS` is the source of truth.
|
|
29
29
|
- Never invent a verify-specific usage footer. Evidence artifact usage must flow through `lisa-usage-accounting`, preserve the canonical `## Lisa Usage` section, and surface `source: unavailable` explicitly when the runtime cannot provide trustworthy numbers.
|
|
30
|
-
- **Evidence-manifest gate (leaf work units).** Before dispatching to a vendor skill that transitions the ticket, confirm `EVIDENCE_DIR` contains a non-empty artifact for every `[EVIDENCE: name]` marker declared in the ticket's Validation Journey. If any declared marker has no captured artifact
|
|
30
|
+
- **Evidence-manifest gate (leaf work units).** Before dispatching to a vendor skill that transitions the ticket, confirm `EVIDENCE_DIR` contains a non-empty artifact **of the declared type** for every typed `[EVIDENCE: <artifact-type>: <name>]` marker declared in the ticket's Validation Journey — a `screenshot` marker needs an actual image, an `http-transcript` marker needs the request + response text, a `perf-trace` marker needs measured numbers; a prose claim satisfies nothing. If any declared marker has no captured artifact, an empty one, or one whose content/extension does not match its declared type, stop and report the offending markers by name instead of posting — a leaf work unit (Bug / Task / Sub-task / Improvement) may not advance to its review/Done state with an unsatisfied manifest (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Epics / Stories / Spikes, and leaf units without a Validation Journey, are exempt.
|
|
31
31
|
|
|
32
32
|
## UI Evidence Checklist (when work is UI-visible)
|
|
33
33
|
|
|
@@ -76,7 +76,7 @@ If auto-merge is enabled while the regression spec is still in flight, disable a
|
|
|
76
76
|
|
|
77
77
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
78
78
|
|
|
79
|
-
The `codify-verification` skill maps the verification type to the appropriate framework (Playwright for browser/UI, integration test for API/DB/auth, benchmark for performance, etc.), generates a deterministic test that asserts the same observable outcome the verification just confirmed, runs it in isolation to confirm PASS, and commits it in the same PR as the change.
|
|
79
|
+
The `codify-verification` skill maps the verification type to the appropriate framework (Playwright for browser/UI, integration test for API/DB/auth, benchmark for performance, etc.), generates a deterministic test that asserts the same observable outcome the verification just confirmed, runs it in isolation to confirm PASS, and commits it in the same PR as the change. For **frontend work**, codification is dual-runner: a Playwright spec in the project's Playwright test runner AND a Maestro flow in the Maestro test runner whenever the project supports Maestro (`.maestro/` directory, `maestro:test` script, or Maestro CI workflow) — both encoding the same verified journey, neither a substitute for the other.
|
|
80
80
|
|
|
81
81
|
Codification is mandatory for every empirical verification type with one exception set: PR, Documentation, Deploy, and Investigate-Only spikes — those have inherently non-behavioral proof. For every other type, skipping codification is not allowed; if codification is genuinely impossible (e.g., the test framework does not exist and cannot be installed in scope), escalate via the Escalation Protocol rather than silently skipping.
|
|
82
82
|
|
|
@@ -236,7 +236,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
236
236
|
8. Implement the change.
|
|
237
237
|
9. Execute verification plan — run the actual system and observe results.
|
|
238
238
|
10. Collect proof artifacts.
|
|
239
|
-
11. Codify — for each passing empirical verification, invoke `codify-verification` to encode it as a regression test (Playwright for UI, integration test for API/DB/auth, benchmark for performance, etc.) and commit the test in the same PR.
|
|
239
|
+
11. Codify — for each passing empirical verification, invoke `codify-verification` to encode it as a regression test (Playwright for UI, integration test for API/DB/auth, benchmark for performance, etc.) and commit the test in the same PR. Frontend work codifies into every supported UI runner: Playwright spec + Maestro flow when the project supports Maestro (see the dual-runner section of `codify-verification`).
|
|
240
240
|
12. Run spec conformance — build coverage matrix against the spec source (plan/ticket/issue), flag scope creep and untraceable changes, produce verdict.
|
|
241
241
|
13. Summarize what changed, what was verified, what was codified, conformance verdict, and remaining risk.
|
|
242
242
|
14. Label the result with a verification level.
|
|
@@ -250,7 +250,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
250
250
|
3. **If verification fails**: Fix and re-run, don't mark complete
|
|
251
251
|
4. **If verification blocked** (missing tools, services, etc.): Mark as blocked, not complete
|
|
252
252
|
5. **Must not be dependent on CI/CD** if necessary, you may use local deploy methods found in the project manifest, but the verification methods must be listed in the pull request and therefore cannot be dependent on CI/CD completing
|
|
253
|
-
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every `[EVIDENCE: name]` marker declared on the ticket has a corresponding captured, non-empty artifact attached to the ticket. A missing or
|
|
253
|
+
6. **Evidence manifest satisfied (leaf work units)**: For a leaf work unit (Bug / Task / Sub-task / Improvement) whose ticket carries a Validation Journey, do not mark the ticket complete or transition it out of in-progress until every typed `[EVIDENCE: <artifact-type>: <name>]` marker declared on the ticket has a corresponding captured, non-empty artifact **of the declared type** attached to the ticket (an image for `screenshot`, request + response for `http-transcript`, measured output for `perf-trace`, …). A missing, empty, or wrong-type artifact for any declared marker blocks completion exactly like a failed verification — fix and re-capture, or escalate; never close with an unsatisfied manifest. Epics / Stories / Spikes are exempt (coordination containers, not work units).
|
|
254
254
|
7. **No artifact-only completion for required runtime verification**: If empirical verification is required and cannot run because credentials are missing, do not mark the item done on artifact-only evidence. Exhaust the credential lookup order first; if still blocked, post the blocker comment, move the item to the configured blocked state, and apply the configured `needs-human` / `human-review` label.
|
|
255
255
|
|
|
256
256
|
---
|
|
@@ -357,7 +357,7 @@ A task is done only when:
|
|
|
357
357
|
- Required verification surfaces and tooling surfaces are used or explicitly unavailable
|
|
358
358
|
- Proof artifacts are captured
|
|
359
359
|
- Every passing empirical verification is codified as a regression test (or has an explicit, documented skip reason from the allowed set)
|
|
360
|
-
- For a leaf work unit, every `[EVIDENCE: name]` marker declared in its Validation Journey has a captured, non-empty artifact attached to the ticket (the evidence manifest is fully satisfied)
|
|
360
|
+
- For a leaf work unit, every typed `[EVIDENCE: <artifact-type>: <name>]` marker declared in its Validation Journey has a captured, non-empty artifact of the declared type attached to the ticket (the evidence manifest is fully satisfied)
|
|
361
361
|
- Spec conformance verdict is `CONFORMS` (not `PARTIAL`, not `DIVERGES`)
|
|
362
362
|
- Verification level is declared
|
|
363
363
|
- Risks and gaps are documented
|
|
@@ -35,7 +35,7 @@ Treat the first successful lead-spawn request (or, on the Codex fallback, the fi
|
|
|
35
35
|
|
|
36
36
|
Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via the lisa plugin). The flow includes:
|
|
37
37
|
|
|
38
|
-
1. **Pre-flight: codification gate** — confirm that every passing local empirical verification on this branch was codified as a regression test (the Implement flow's codify step). If any verification has no committed test and no allowed skip reason (PR / Documentation / Deploy / Investigate-Only), invoke `codify-verification` now and amend the PR before shipping. A change cannot ship until its verifications are guarded.
|
|
38
|
+
1. **Pre-flight: codification gate** — confirm that every passing local empirical verification on this branch was codified as a regression test (the Implement flow's codify step). If any verification has no committed test and no allowed skip reason (PR / Documentation / Deploy / Investigate-Only), invoke `codify-verification` now and amend the PR before shipping. For frontend work the gate is dual-runner: a Playwright spec AND, when the project supports Maestro (`.maestro/`, `maestro:test` script, or Maestro CI workflow), a Maestro flow for the same journey — a missing runner needs a recorded absence or a linked build-ready follow-up ticket, never a silent skip. A change cannot ship until its verifications are guarded.
|
|
39
39
|
2. **Commit** any pending changes via `lisa-git-commit`
|
|
40
40
|
3. **Push and PR** via `lisa-git-submit-pr`
|
|
41
41
|
4. **Review loop** — handle CodeRabbit / human review comments via `lisa-pull-request-review`
|
|
@@ -72,7 +72,7 @@ Sequence:
|
|
|
72
72
|
2. `product-specialist` -- define user goals, user flows (Gherkin), acceptance criteria, error states, UX concerns, and out-of-scope items
|
|
73
73
|
3. **Edge Case Brainstorm sub-flow** -- run the PRD candidate through the edge-case checklist; fold accepted cases into acceptance criteria, out-of-scope, or open questions
|
|
74
74
|
4. `architecture-specialist` -- assess technical feasibility, identify constraints, map existing system boundaries
|
|
75
|
-
5. Synthesize findings into a PRD
|
|
75
|
+
5. Synthesize findings into a PRD structured as: (1) problem statement, (2) high-level solution description, (3) links to design files/docs if needed, (4) user stories -- each carrying its own functional requirements, non-functional requirements, and a pointer to a design file (only when that story introduces new UI/visual work; omit the pointer otherwise rather than leaving it as a blank required field), (5) overall acceptance criteria, and (6) open questions/decisions. Nest requirements under each story rather than flattening them into global lists -- this keeps the context an agent needs to implement or ticket one story colocated, instead of requiring it to infer which global requirement applies to which story. Any technically viable but genuinely unresolved choice discovered during drafting (for example, a library, framework, or architecture decision with more than one live candidate) MUST be captured as an entry under open questions -- never written into any other section, including the "Recommended Tooling for Plan Phase" section in step 6, as though it were already decided. Every open-questions entry MUST include the drafter's own recommended resolution, with a one-sentence rationale, alongside the question; an open question must never be left bare.
|
|
76
76
|
6. **Plan Phase Tooling** -- review all available skills and agents (project-defined, plugin-provided, and built-in) and determine which ones the Plan phase will need. For each recommended skill or agent, state why it is needed. If no skills or agents beyond the defaults are identified, explicitly justify why the standard set is sufficient. Include this as a "Recommended Tooling for Plan Phase" section in the PRD. This section documents settled recommendations for how to run the Plan phase, meaning which skills or agents to use; it MUST NOT be used to record an unresolved product or technical decision, such as "use library X" when X vs. Y was never actually decided. If step 5 surfaced an unresolved technical choice, it belongs in open questions with a recommendation, not here.
|
|
77
77
|
7. **Create the PRD in the configured source** -- invoke `lisa-prd-source-write` with the synthesized PRD (`title`, `body`, `initial_role` resolved from the caller's `prd_ready` flag — `draft` by default, `ready` when `prd_ready=true`, plus any `dedupe_key`/`marker`/`source_ref` the caller passed). The PRD **lives in the source** (Notion page / Confluence page / GitHub issue / Linear project per `.lisa.config.json` `source`); there is no separate document artifact. A `source` must be configured — if it is not, stop and report it. `prd-source-write` dedupes by marker, so re-running against the same idea references the existing PRD instead of creating a duplicate.
|
|
78
78
|
8. **Record Research usage on the PRD artifact** -- invoke `lisa-usage-accounting` against the created PRD/source artifact so it gains a direct `research` usage entry in the canonical `## Lisa Usage` section at creation time. If the runtime cannot provide trustworthy usage, still write the row with `source: unavailable` and nullable token/cost fields; missing usage is never treated as zero or silently omitted.
|
|
@@ -78,7 +78,7 @@ Both finding types are filed through the vendor-neutral `lisa-tracker-write` shi
|
|
|
78
78
|
- **Three-audience description** (S3): coding-assistant (technical: stack trace / Sentry link / occurrence count / the missing dimension and how to wire it), developer (where it surfaces, suspected cause/affected files, the fix skill to reach for), stakeholder (user/SLO impact).
|
|
79
79
|
- **Gherkin acceptance criteria** (S4).
|
|
80
80
|
- **Single repo** (S10): stamped `repo:<CURRENT_REPO>`; scope the description to this repo only.
|
|
81
|
-
- **Target Backend Environment** (S8) + **Validation Journey** carrying at least one
|
|
81
|
+
- **Target Backend Environment** (S8) + **Validation Journey** carrying at least one typed `[EVIDENCE: <artifact-type>: <name>]` marker (S11/S14) — both anomaly fixes and gap wiring are runtime changes, so both need an env and a journey that proves the fix. The bracketed typed form is what the validator scans for; a bare `EVIDENCE:` line or an untyped assertion label (`[EVIDENCE: alert-fixed]`) fails S14 — for observability work the natural types are `log-snippet` and `state-dump`. Prefer two markers (a success and an error/edge case).
|
|
82
82
|
- **Relationship search before write** (S13) — doubles as the dedup guard (next section).
|
|
83
83
|
- **No parent/Epic required.** These are build-ready standalone leaves; S7's parent requirement is waived for a `build_ready: true` leaf (the leaf carve-out in `leaf-only-lifecycle` / `tracker-validate`). Do not fabricate an Epic parent.
|
|
84
84
|
- A **priority** ordered by tier and severity: `core` gap / high-event anomaly → higher; `recommended` gap → lower.
|
|
@@ -14,7 +14,7 @@ The **audit + file** arm of `lisa-monitor`. On top of its existing live-signal s
|
|
|
14
14
|
- **Conservative by default.** Only high-signal anomalies (over the documented thresholds) and `core` missing dimensions are filed. `--all-gaps` widens gap filing to `recommended` tiers; nothing lowers the anomaly bar.
|
|
15
15
|
- **Idempotent.** Every ticket carries a `<!-- lisa:monitor-finding: <fingerprint> -->` sentinel; search-before-create (including closed tickets) means a re-run never duplicates a live or just-resolved finding.
|
|
16
16
|
- **Capped.** At most `monitor.maxCandidates` tickets per run (default 20), highest-severity first; report filed-vs-dropped (and list the dropped) — never silently truncate.
|
|
17
|
-
- **Gate-passing.** Each ticket is a real authored artifact: three-audience description, Gherkin AC, single-repo, Target Backend Environment, and a Validation Journey with a
|
|
17
|
+
- **Gate-passing.** Each ticket is a real authored artifact: three-audience description, Gherkin AC, single-repo, Target Backend Environment, and a Validation Journey with a typed `[EVIDENCE: <artifact-type>: <name>]` marker (unique kebab-case name) — so `tracker-validate` (S1–S15) accepts it. A finding that cannot be made into a credible ticket is reported, not filed.
|
|
18
18
|
- **Verify guard.** When `monitor` is the post-deploy step of `lisa-verify` it runs **report-only** — Verify invokes it as `lisa-monitor <env> --report-only`, so it never files there. Filing is a standalone-only action.
|
|
19
19
|
|
|
20
20
|
Full reference (profile detection, the rubric table, anomaly thresholds, ticket templates, idempotency contract, the cap, dry-run/report-only semantics): [reference/observability-audit.md](observability-audit-reference.mdc).
|
|
@@ -105,12 +105,48 @@ Every change requires one or more verification types. Classify the change first,
|
|
|
105
105
|
|
|
106
106
|
Every **leaf work unit** — an individually implementable ticket with no child tickets (issue types Bug, Task, Sub-task, Improvement) — that changes runtime behavior must declare, at creation time, the exact evidence that proves it is done. Epics, Stories, and Spikes are coordination containers, not work units: their evidence is the rollup of their children, so this contract does not apply to them.
|
|
107
107
|
|
|
108
|
-
The declaration is not a separate field — it is the set of `[EVIDENCE: name]` markers in the work unit's **Validation Journey**. Those markers are the work unit's **evidence manifest**: an enumerated,
|
|
108
|
+
The declaration is not a separate field — it is the set of `[EVIDENCE: <artifact-type>: <name>]` markers in the work unit's **Validation Journey**. Those markers are the work unit's **evidence manifest**: an enumerated, typed list of the artifacts a verifier must capture. The manifest binds both ends of the ticket lifecycle:
|
|
109
109
|
|
|
110
|
-
- **At creation** — the work unit cannot be written without a Validation Journey that
|
|
111
|
-
- **At completion** — the work unit cannot be marked complete, nor transitioned to its review/Done state, until every
|
|
110
|
+
- **At creation** — the work unit cannot be written without a Validation Journey that declares at least one typed `[EVIDENCE: <artifact-type>: <name>]` artifact (enforced by gate S14 in `tracker-validate` and the vendor `*-validate-*` skills). A behavior-changing unit should declare both a success artifact and an error/edge artifact.
|
|
111
|
+
- **At completion** — the work unit cannot be marked complete, nor transitioned to its review/Done state, until every marker in its manifest has a captured, non-empty artifact **of the declared type** attached to the ticket (enforced by the Task Completion Rules and Definition of Done in `verification-lifecycle`, and by the evidence-manifest gate in `tracker-evidence`). A manifest with a missing, empty, or wrong-type artifact blocks completion exactly like a failed verification.
|
|
112
112
|
|
|
113
|
-
|
|
113
|
+
### Marker grammar: the type is the evidence
|
|
114
|
+
|
|
115
|
+
**A marker names an artifact, not an assertion.** `[EVIDENCE: load-failure-handled-gracefully]` is a claim — there is nothing to capture, so it degenerates into a checkbox someone ticks. Evidence is empirical: a screenshot, a curl transcript, a log snippet, a frame-timing trace. The marker therefore carries two parts:
|
|
116
|
+
|
|
117
|
+
```text
|
|
118
|
+
[EVIDENCE: <artifact-type>: <kebab-case-name>]
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
- `<artifact-type>` — HOW the proof is captured, from the fixed taxonomy below.
|
|
122
|
+
- `<kebab-case-name>` — WHAT it proves, unique within the ticket.
|
|
123
|
+
|
|
124
|
+
Example transformation (the failure mode this grammar exists to prevent):
|
|
125
|
+
|
|
126
|
+
| Assertion label (invalid) | Typed artifact (valid) |
|
|
127
|
+
|---|---|
|
|
128
|
+
| `[EVIDENCE: pipeline-load-under-3s]` | `[EVIDENCE: perf-trace: pipeline-load-tti]` |
|
|
129
|
+
| `[EVIDENCE: long-column-virtualized]` | `[EVIDENCE: perf-trace: long-column-frame-timing]` |
|
|
130
|
+
| `[EVIDENCE: load-failure-handled-gracefully]` | `[EVIDENCE: screenshot: load-failure-error-state]` |
|
|
131
|
+
|
|
132
|
+
### Artifact-type taxonomy (fixed set)
|
|
133
|
+
|
|
134
|
+
| Type | The captured artifact is |
|
|
135
|
+
|---|---|
|
|
136
|
+
| `screenshot` | an image of the observed UI/state (Playwright, simulator, device) |
|
|
137
|
+
| `recording` | a video of the playthrough |
|
|
138
|
+
| `http-transcript` | the exact request (curl command or client call) plus the full response — status, headers of interest, body |
|
|
139
|
+
| `cli-output` | the command plus its stdout/stderr and exit code |
|
|
140
|
+
| `log-snippet` | correlated log lines captured from the running system |
|
|
141
|
+
| `db-query-output` | the query plus the returned rows |
|
|
142
|
+
| `perf-trace` | benchmark / frame-timing / profiler output, with the methodology (device profile, dataset size) noted |
|
|
143
|
+
| `test-run-log` | reporter output naming the spec and showing it ran and passed |
|
|
144
|
+
| `deploy-log` | deployment output or a health-check response from the target environment |
|
|
145
|
+
| `state-dump` | machine-readable observed state (e.g. the `state.json` asserted against) |
|
|
146
|
+
|
|
147
|
+
Do not invent types inline; if none fits, propose extending this table. The legacy `[SCREENSHOT: name]` marker is equivalent to `[EVIDENCE: screenshot: name]`.
|
|
148
|
+
|
|
149
|
+
The manifest is the single source of truth for "what evidence is required": authored once in the Validation Journey, enforced at write time, replayed during `tracker-journey` (which captures each artifact **in its declared type**), and checked again before the ticket closes. There is no second list to keep in sync.
|
|
114
150
|
|
|
115
151
|
---
|
|
116
152
|
|
|
@@ -146,7 +182,7 @@ Verification **is** UAT — one gate, not two. This section makes the single
|
|
|
146
182
|
verification process concrete so "an agent actually exercised the running
|
|
147
183
|
software against the acceptance criteria" is durable and re-checkable, not a
|
|
148
184
|
one-off claim. It builds on the **Per-Work-Unit Evidence Contract** above (the
|
|
149
|
-
`[EVIDENCE: name]` manifest), adding three concrete requirements:
|
|
185
|
+
typed `[EVIDENCE: <artifact-type>: <name>]` manifest), adding three concrete requirements:
|
|
150
186
|
|
|
151
187
|
**1. The codified proof re-runs in CI.** After local verification passes,
|
|
152
188
|
`codify-verification` encodes it where the project's e2e/Playwright tests live
|
|
@@ -154,6 +190,8 @@ one-off claim. It builds on the **Per-Work-Unit Evidence Contract** above (the
|
|
|
154
190
|
project type with verification **enforced**, it is a required check — so the proof
|
|
155
191
|
re-runs on every PR instead of being proven once by hand.
|
|
156
192
|
|
|
193
|
+
For **frontend work**, codification is dual-runner: a Playwright spec in the project's Playwright test runner AND a Maestro flow in the Maestro test runner whenever the project supports Maestro (`.maestro/` directory, `maestro:test` script, or Maestro CI workflow) — both encoding the same verified journey, neither a substitute for the other. The dual-runner requirement is non-demotable: a missing runner is either a recorded absence (the project genuinely has no such harness) or a linked build-ready follow-up ticket — never a silent skip (see "Frontend dual-runner codification" in `codify-verification`).
|
|
194
|
+
|
|
157
195
|
**2. Evidence is committed to the repo.** The named artifacts from the work
|
|
158
196
|
unit's evidence manifest are committed under `evidence/<ticket>/` (in addition to
|
|
159
197
|
any tracker attachment), so the proof lives with the code and is reviewable in the
|
|
@@ -16,9 +16,10 @@ alwaysApply: true
|
|
|
16
16
|
- **Never claim success without runtime evidence.** "The code looks correct" is not evidence.
|
|
17
17
|
- **If all you did was run tests, typecheck, and lint — you have NOT verified.**
|
|
18
18
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
19
|
-
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes.
|
|
19
|
+
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes. For **frontend work**, codification is dual-runner: a Playwright spec in the project's Playwright test runner AND a Maestro flow in the Maestro test runner whenever the project supports Maestro (`.maestro/` directory, `maestro:test` script, or Maestro CI workflow) — both encoding the same verified journey, neither a substitute for the other.
|
|
20
20
|
- **The codified proof re-runs in CI.** Codified runtime verification lives where the project's e2e/Playwright tests live (`tests/e2e/**`) and runs as a required CI check for types with verification enforced — so the proof re-runs on every PR, not once by hand.
|
|
21
21
|
- **Commit the evidence.** Write a durable artifact to `evidence/<ticket>/` — the acceptance criteria with per-criterion pass/fail + note, the observed state, screenshots/recording, and a `verdict.json`. The transient `.lisa/verification-status.json` is the session gate; `evidence/<ticket>/` is the committed proof.
|
|
22
|
+
- **Evidence markers are typed artifacts, not assertion labels.** A Validation Journey marker is `[EVIDENCE: <artifact-type>: <name>]` where the type (screenshot, recording, http-transcript, cli-output, log-snippet, db-query-output, perf-trace, test-run-log, deploy-log, state-dump) says HOW the proof is captured and the name says WHAT it proves. `[EVIDENCE: works-gracefully]` is a claim, not evidence — write `[EVIDENCE: screenshot: load-failure-error-state]`. Completion requires a captured artifact **of the declared type** per marker.
|
|
22
23
|
- **Per-change verification is mandatory.** Every `feat`/`fix` adds or extends a verification (e2e) spec mapped to its acceptance criteria. The only exception is a genuinely non-behavioral change explicitly marked with the **logged** `verification-exempt` label — never a silent skip.
|
|
23
24
|
- **Drive the playthrough with `/lisa:product-walkthrough`** (it walks the live product through a real browser); each project type plugs in its own drive mechanism (e.g. a Phaser game is driven through Playwright + an in-game verification test bridge that seeds RNG, reads state, injects input, and steps frames).
|
|
24
25
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
@@ -57,6 +57,7 @@ Do NOT install a new framework if one already exists for the verification type.
|
|
|
57
57
|
|---|---|
|
|
58
58
|
| UI (web) | Playwright > Cypress > Selenium |
|
|
59
59
|
| UI (mobile) | Maestro > Detox > Playwright (mobile emulation) |
|
|
60
|
+
| UI (frontend, project supports multiple runners) | **ALL supported UI runners** — see "Frontend dual-runner codification" below |
|
|
60
61
|
| API | project's integration test runner (Vitest / Jest / RSpec / pytest) with HTTP client (supertest / fetch / faraday) |
|
|
61
62
|
| Database | integration test with real DB + migrations applied |
|
|
62
63
|
| Auth | API or UI test asserting role-gated access (multi-role coverage) |
|
|
@@ -71,6 +72,20 @@ Do NOT install a new framework if one already exists for the verification type.
|
|
|
71
72
|
|
|
72
73
|
If the project lacks the preferred framework AND no acceptable substitute exists, escalate.
|
|
73
74
|
|
|
75
|
+
### 2a. Frontend dual-runner codification (non-demotable)
|
|
76
|
+
|
|
77
|
+
For **frontend work** — any verification whose validation journey exercised a user-facing UI surface — codification is not one-runner-or-the-other. After the validation journey is complete and verified, the verified behavior MUST be codified in **every UI runner the project supports**:
|
|
78
|
+
|
|
79
|
+
1. **A Playwright spec in the project's Playwright test runner** (where its web e2e tests live, e.g. `tests/e2e/**` / `e2e/**`) — required whenever the project has a Playwright (or equivalent web e2e) harness.
|
|
80
|
+
2. **A Maestro flow in the project's Maestro test runner** — required whenever the project supports Maestro. Detect support by any of: a `.maestro/` directory (flows live in `.maestro/flows/`), a `maestro:test` script in `package.json`, or a Maestro CI workflow (e.g. `maestro-native-e2e`). Wire the new flow where the runner picks it up (`maestro test .maestro/flows`), tagging per the project's tier convention (e.g. `smoke`) when one exists.
|
|
81
|
+
|
|
82
|
+
Both artifacts encode the SAME verified journey — the Playwright spec drives the web surface, the Maestro flow drives the native surface. One is not a substitute for the other: they guard different platforms of the same behavior.
|
|
83
|
+
|
|
84
|
+
Permitted exits, mirroring the regression-spec rule in `lisa-implement` (never a silent skip, never "optional"):
|
|
85
|
+
|
|
86
|
+
- The project genuinely has no runner of that kind (no web e2e harness, or no Maestro support by the detection above) → record the checked locations and the absence in the codification evidence; that runner is N/A.
|
|
87
|
+
- A runner is supported but the flow/spec cannot be added or executed in this PR (genuine technical blocker) → create a linked build-ready follow-up ticket before merge, reference it from the PR and work item, and record the blocker — the same follow-up path as the regression-spec blocker.
|
|
88
|
+
|
|
74
89
|
### 3. Generate the test
|
|
75
90
|
|
|
76
91
|
The generated test must:
|
|
@@ -103,6 +118,7 @@ requires a verification-spec delta on every behavioral change. See the
|
|
|
103
118
|
Run only the new test, using whatever per-test invocation the project supports:
|
|
104
119
|
|
|
105
120
|
- Playwright: `npx playwright test path/to/new.spec.ts`
|
|
121
|
+
- Maestro: `maestro test .maestro/flows/new-flow.yaml`
|
|
106
122
|
- Vitest: `npx vitest run path/to/new.spec.ts`
|
|
107
123
|
- Jest: `npx jest path/to/new.test.ts`
|
|
108
124
|
- RSpec: `bundle exec rspec path/to/new_spec.rb`
|
|
@@ -139,6 +155,7 @@ Append to the verification report (or PR description):
|
|
|
139
155
|
| # | Verification | Framework | Test file | Status |
|
|
140
156
|
|---|--------------|-----------|-----------|--------|
|
|
141
157
|
| 1 | <description> | Playwright | `e2e/checkout.spec.ts::displays order confirmation after checkout` | PASS |
|
|
158
|
+
| 2 | <same journey, native surface> | Maestro | `.maestro/flows/checkout-confirmation.yaml` | PASS |
|
|
142
159
|
```
|
|
143
160
|
|
|
144
161
|
This evidence shows the verification is now guarded.
|
|
@@ -58,7 +58,7 @@ Use Explore agents or read the codebase directly to understand which files are a
|
|
|
58
58
|
|
|
59
59
|
### Step 5: Draft the Validation Journey
|
|
60
60
|
|
|
61
|
-
Compose the journey with `[EVIDENCE: name]` markers at key verification points:
|
|
61
|
+
Compose the journey with typed `[EVIDENCE: <artifact-type>: <name>]` markers at key verification points. The type says HOW the proof is captured (`screenshot`, `recording`, `http-transcript`, `cli-output`, `log-snippet`, `db-query-output`, `perf-trace`, `test-run-log`, `deploy-log`, `state-dump` — the fixed taxonomy in the `verification` rule); the name says WHAT it proves:
|
|
62
62
|
|
|
63
63
|
```markdown
|
|
64
64
|
## Validation Journey
|
|
@@ -69,9 +69,9 @@ Compose the journey with `[EVIDENCE: name]` markers at key verification points:
|
|
|
69
69
|
### Steps
|
|
70
70
|
1. Verify current state before changes
|
|
71
71
|
2. Apply the change
|
|
72
|
-
3. Verify expected new state [EVIDENCE:
|
|
73
|
-
4. Test error/edge cases [EVIDENCE: error-
|
|
74
|
-
5. Verify rollback if applicable [EVIDENCE: rollback]
|
|
72
|
+
3. Verify expected new state [EVIDENCE: http-transcript: health-endpoint-200]
|
|
73
|
+
4. Test error/edge cases [EVIDENCE: screenshot: invalid-input-error-state]
|
|
74
|
+
5. Verify rollback if applicable [EVIDENCE: db-query-output: rows-restored-after-rollback]
|
|
75
75
|
|
|
76
76
|
### Assertions
|
|
77
77
|
- Describe what must be true after verification
|
|
@@ -82,10 +82,10 @@ Compose the journey with `[EVIDENCE: name]` markers at key verification points:
|
|
|
82
82
|
1. **2–5 evidence markers** — Focus on proving the change works and handles errors.
|
|
83
83
|
2. **Concrete, runnable steps** — `Run \`curl -s localhost:3000/health | jq .status\`` not "Check the endpoint".
|
|
84
84
|
3. **Include environment setup** — Database connection, running services, env vars.
|
|
85
|
-
4. **
|
|
85
|
+
4. **Markers are typed artifacts, not assertion labels** — `[EVIDENCE: <artifact-type>: <kebab-name>]`. `[EVIDENCE: load-failure-handled-gracefully]` names a claim with nothing to capture; write `[EVIDENCE: screenshot: load-failure-error-state]` or `[EVIDENCE: perf-trace: pipeline-load-tti]`. Names are kebab-case and unique within the ticket.
|
|
86
86
|
5. **Assertions are measurable** — `Returns 200 with {status: ok}` not "API works correctly".
|
|
87
87
|
6. **Cover happy path AND error path** — At minimum, one success and one failure marker.
|
|
88
|
-
7. **On a leaf work unit, the markers are binding** — For a Bug / Task / Sub-task / Improvement, every `[EVIDENCE: name]` here is the issue's evidence manifest: validation gate S14 requires at least one, and the issue cannot be closed until each named artifact is captured and attached (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Name only evidence you intend to capture — and name all of it.
|
|
88
|
+
7. **On a leaf work unit, the markers are binding** — For a Bug / Task / Sub-task / Improvement, every typed `[EVIDENCE: <artifact-type>: <name>]` here is the issue's evidence manifest: validation gate S14 requires at least one, and the issue cannot be closed until each named artifact is captured **in its declared type** and attached (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Name only evidence you intend to capture — and name all of it.
|
|
89
89
|
|
|
90
90
|
### Step 6: Present to User for Approval
|
|
91
91
|
|
|
@@ -51,14 +51,25 @@ Execute each step sequentially. Determine the verification approach based on the
|
|
|
51
51
|
- **Library/utility changes** → Run tests, capture output.
|
|
52
52
|
- **Security fixes** → Reproduce exploit attempt, verify fix, capture output.
|
|
53
53
|
|
|
54
|
-
At each `[EVIDENCE: name]` marker, capture
|
|
54
|
+
At each typed `[EVIDENCE: <artifact-type>: <name>]` marker, capture an artifact **of the declared type** — the type is the contract, not a suggestion:
|
|
55
|
+
|
|
56
|
+
- `screenshot` / `recording` → an actual image/video file from the driven UI (Playwright, simulator), never a text description of what was seen
|
|
57
|
+
- `http-transcript` → the exact request (curl command or client call) plus the full response
|
|
58
|
+
- `cli-output` → the command plus stdout/stderr and exit code
|
|
59
|
+
- `log-snippet` → the correlated log lines pulled from the running system
|
|
60
|
+
- `db-query-output` → the query plus returned rows
|
|
61
|
+
- `perf-trace` → the benchmark/frame-timing/profiler output with methodology (device profile, dataset size)
|
|
62
|
+
- `test-run-log` → reporter output naming the spec and showing it ran and passed
|
|
63
|
+
- `deploy-log` / `state-dump` → the deployment/health-check output or observed-state JSON
|
|
64
|
+
|
|
65
|
+
A prose claim ("the error state rendered gracefully") satisfies no marker. Legacy untyped markers: infer the type from the step's action, capture accordingly, and note the inference. Write each artifact to a numbered file:
|
|
55
66
|
|
|
56
67
|
#### Evidence Naming Convention
|
|
57
68
|
|
|
58
69
|
`{NN}-{evidence-name}.txt` (or `.json` for structured data):
|
|
59
70
|
|
|
60
71
|
- `NN`: zero-padded sequential number (01, 02, 03...).
|
|
61
|
-
- `evidence-name`: the
|
|
72
|
+
- `evidence-name`: the `<name>` part of the typed marker (kebab-case).
|
|
62
73
|
|
|
63
74
|
Example:
|
|
64
75
|
|
|
@@ -196,9 +196,16 @@ An issue with zero links and no documented search: FAIL.
|
|
|
196
196
|
|
|
197
197
|
#### S14 — Evidence manifest binding (leaf work units)
|
|
198
198
|
|
|
199
|
-
When `issue_type ∈ {Bug, Task, Sub-task, Improvement}` AND `runtime_behavior_change = true`, the `## Validation Journey` must declare at least one `[EVIDENCE: name]` marker.
|
|
199
|
+
When `issue_type ∈ {Bug, Task, Sub-task, Improvement}` AND `runtime_behavior_change = true`, the `## Validation Journey` must declare at least one **typed** `[EVIDENCE: <artifact-type>: <name>]` marker. These markers are the work unit's **evidence manifest** — the exact, enumerated set of artifacts that must be captured and attached before the issue may be closed (see the "Per-Work-Unit Evidence Contract" section of the `verification` rule, the Definition of Done in `verification-lifecycle`, and the evidence-manifest gate in `tracker-evidence`).
|
|
200
200
|
|
|
201
|
-
|
|
201
|
+
Each marker must satisfy ALL of:
|
|
202
|
+
|
|
203
|
+
- `<artifact-type>` is one of the fixed taxonomy: `screenshot`, `recording`, `http-transcript`, `cli-output`, `log-snippet`, `db-query-output`, `perf-trace`, `test-run-log`, `deploy-log`, `state-dump`. (The legacy `[SCREENSHOT: name]` form is accepted as `screenshot`.)
|
|
204
|
+
- `<name>` is kebab-case and unique within the issue.
|
|
205
|
+
|
|
206
|
+
**A marker names an artifact, not an assertion.** An untyped marker (`[EVIDENCE: load-failure-handled-gracefully]`) is an assertion label with nothing to capture and must FAIL, with a remediation that shows the typed transformation (e.g. → `[EVIDENCE: screenshot: load-failure-error-state]`, `[EVIDENCE: perf-trace: pipeline-load-tti]`).
|
|
207
|
+
|
|
208
|
+
FAIL when the Validation Journey is present but declares zero markers, when any marker is untyped or uses a type outside the taxonomy, or when any name is empty, duplicated, or not kebab-case. A behavior-changing work unit SHOULD declare both a success marker and an error/edge marker; a journey with only one marker passes but the remediation should recommend adding the error/edge case.
|
|
202
209
|
|
|
203
210
|
This gate depends on S11. It is `N/A` for containers — an **Epic**, or any item with open child work (coordination containers, not work units) — and for leaf units with `runtime_behavior_change = false` (doc-only / config-only / type-only). If S11 fails because the Validation Journey is absent, S14 also FAILs (there is no manifest to bind) with remediation pointing back to `lisa-github-add-journey`.
|
|
204
211
|
|
|
@@ -115,7 +115,7 @@ IF it is a Fix (bug), execute the Reproduce sub-flow FIRST:
|
|
|
115
115
|
1. Write a simple API client and call the offending API
|
|
116
116
|
2. Start the server on localhost and use the Playwright CLI or Chrome DevTools
|
|
117
117
|
|
|
118
|
-
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it.
|
|
118
|
+
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it. For frontend work this deliverable is **dual-runner** whenever the project supports more than one UI runner: a Playwright spec in the Playwright test runner AND a Maestro flow in the Maestro test runner when the project supports Maestro (`.maestro/` directory, `maestro:test` script, or Maestro CI workflow) — both encoding the same verified journey; neither substitutes for the other (see "Frontend dual-runner codification" in `codify-verification`).
|
|
119
119
|
|
|
120
120
|
The team lead may not waive, defer, demote, or phrase this regression spec as "optional", "if cheap", "nice to have", or equivalent. The only permitted exits are:
|
|
121
121
|
|
|
@@ -203,7 +203,7 @@ Before shutting down the team, execute the Verify flow:
|
|
|
203
203
|
- **Human-only blocker** — an input the agent genuinely cannot obtain or produce no matter what it does: credentials, secrets, or **tool access** it does not have (AWS/CloudWatch, Figma, Jam, Sentry, SonarCloud, a database, a protected deploy target, …), or a product/design decision only a human can make. For missing tool access, follow the `tool-access-gate` rule's break-out protocol: post the "Access Needed" comment naming the exact credential/role/env var to grant and the probe that must pass — never work around the gap by substituting weaker verification, mocking the inaccessible system, or narrowing scope. Record the blocked verdict, mark it `human_needed` (the marker `repair-intake` recognizes, so it won't churn re-dispatching it), and surface or reassign to a human; do **not** fabricate a build-ready ticket, because there is no build-ready work.
|
|
204
204
|
|
|
205
205
|
Other harnesses fall back to this prose obligation.
|
|
206
|
-
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface
|
|
206
|
+
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified, codification into **every supported UI runner**: a Playwright spec in the Playwright runner AND a Maestro flow when the project supports Maestro, per `codify-verification`. Prove the new spec actually executed and passed in PR CI by recording a named spec log/reporter line or equivalent execution record; green CI without that named evidence does not satisfy this step.
|
|
207
207
|
4. Record Implement usage on the originating work artifact via `lisa-usage-accounting` so the work item (or other implementation-owned artifact) gains a direct `lisa-implement` usage entry in the canonical `## Lisa Usage` section. If the parent / child graph is already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise still write the direct entry, and if runtime usage is unavailable, use `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
208
208
|
5. Commit ALL outstanding changes in logical batches on the branch (minus sensitive data/information) — not just changes made by the agent team. This includes pre-existing uncommitted changes that were on the branch before the plan started. Do NOT filter commits to only "task-related" files. If it shows up in git status, it gets committed (unless it contains secrets).
|
|
209
209
|
6. Push the changes - if any pre-push hook blocks you, create a task for the agent team to fix the error/problem whether it was pre-existing or not
|
|
@@ -68,7 +68,7 @@ Based on the change type, generate verification steps using patterns from `verfi
|
|
|
68
68
|
|
|
69
69
|
### Step 5: Draft the Validation Journey
|
|
70
70
|
|
|
71
|
-
Compose the journey with `[EVIDENCE: name]` markers at key verification points:
|
|
71
|
+
Compose the journey with typed `[EVIDENCE: <artifact-type>: <name>]` markers at key verification points. The type says HOW the proof is captured (`screenshot`, `recording`, `http-transcript`, `cli-output`, `log-snippet`, `db-query-output`, `perf-trace`, `test-run-log`, `deploy-log`, `state-dump` — the fixed taxonomy in the `verification` rule); the name says WHAT it proves:
|
|
72
72
|
|
|
73
73
|
```text
|
|
74
74
|
h2. Validation Journey
|
|
@@ -79,9 +79,9 @@ h3. Prerequisites
|
|
|
79
79
|
h3. Steps
|
|
80
80
|
1. Verify current state before changes
|
|
81
81
|
2. Apply the change
|
|
82
|
-
3. Verify expected new state [EVIDENCE:
|
|
83
|
-
4. Test error/edge cases [EVIDENCE: error-
|
|
84
|
-
5. Verify rollback if applicable [EVIDENCE: rollback]
|
|
82
|
+
3. Verify expected new state [EVIDENCE: http-transcript: health-endpoint-200]
|
|
83
|
+
4. Test error/edge cases [EVIDENCE: screenshot: invalid-input-error-state]
|
|
84
|
+
5. Verify rollback if applicable [EVIDENCE: db-query-output: rows-restored-after-rollback]
|
|
85
85
|
|
|
86
86
|
h3. Assertions
|
|
87
87
|
- Describe what must be true after verification
|
|
@@ -92,10 +92,10 @@ h3. Assertions
|
|
|
92
92
|
1. **2-5 evidence markers** — Focus on proving the change works and handles errors
|
|
93
93
|
2. **Concrete, runnable steps** — "Run `curl -s localhost:3000/health | jq .status`" not "Check the endpoint"
|
|
94
94
|
3. **Include environment setup** — Database connection, running services, env vars
|
|
95
|
-
4. **
|
|
95
|
+
4. **Markers are typed artifacts, not assertion labels** — `[EVIDENCE: <artifact-type>: <kebab-name>]`. `[EVIDENCE: load-failure-handled-gracefully]` names a claim with nothing to capture; write `[EVIDENCE: screenshot: load-failure-error-state]` or `[EVIDENCE: perf-trace: pipeline-load-tti]`. Names are kebab-case and unique within the ticket.
|
|
96
96
|
5. **Assertions are measurable** — "Returns 200 with `{status: ok}`" not "API works correctly"
|
|
97
97
|
6. **Cover happy path and error path** — At minimum, one success and one failure evidence marker
|
|
98
|
-
7. **On a leaf work unit, the markers are binding** — For a Bug / Task / Sub-task / Improvement, every `[EVIDENCE: name]` here is the ticket's evidence manifest: validation gate S14 requires at least one, and the ticket cannot be closed until each named artifact is captured and attached (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Name only evidence you intend to capture — and name all of it.
|
|
98
|
+
7. **On a leaf work unit, the markers are binding** — For a Bug / Task / Sub-task / Improvement, every typed `[EVIDENCE: <artifact-type>: <name>]` here is the ticket's evidence manifest: validation gate S14 requires at least one, and the ticket cannot be closed until each named artifact is captured **in its declared type** and attached (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Name only evidence you intend to capture — and name all of it.
|
|
99
99
|
|
|
100
100
|
### Step 6: Present to User for Approval
|
|
101
101
|
|
|
@@ -55,7 +55,7 @@ Skip the Validation Journey for:
|
|
|
55
55
|
|
|
56
56
|
### How to Write
|
|
57
57
|
|
|
58
|
-
Design the journey based on the **change type**. The agent executing the journey determines how to verify each step using patterns from the project's `verfication.md`. Place `[EVIDENCE: name]` markers at key verification points.
|
|
58
|
+
Design the journey based on the **change type**. The agent executing the journey determines how to verify each step using patterns from the project's `verfication.md`. Place typed `[EVIDENCE: <artifact-type>: <name>]` markers at key verification points (types: `screenshot`, `recording`, `http-transcript`, `cli-output`, `log-snippet`, `db-query-output`, `perf-trace`, `test-run-log`, `deploy-log`, `state-dump` — see the `verification` rule).
|
|
59
59
|
|
|
60
60
|
Add this section to the ticket description:
|
|
61
61
|
|
|
@@ -70,9 +70,9 @@ h3. Prerequisites
|
|
|
70
70
|
h3. Steps
|
|
71
71
|
1. Verify the current state before changes
|
|
72
72
|
2. Apply the change (run migration, deploy, etc.)
|
|
73
|
-
3. Verify the expected new state [EVIDENCE: state-after-change]
|
|
74
|
-
4. Test error/edge cases [EVIDENCE: error-
|
|
75
|
-
5. Verify rollback or cleanup if applicable [EVIDENCE: rollback
|
|
73
|
+
3. Verify the expected new state [EVIDENCE: http-transcript: state-after-change]
|
|
74
|
+
4. Test error/edge cases [EVIDENCE: screenshot: error-state-rendered]
|
|
75
|
+
5. Verify rollback or cleanup if applicable [EVIDENCE: db-query-output: rows-restored-after-rollback]
|
|
76
76
|
|
|
77
77
|
h3. Assertions
|
|
78
78
|
- Describe what must be true after verification
|
|
@@ -82,7 +82,7 @@ h3. Assertions
|
|
|
82
82
|
### Guidelines
|
|
83
83
|
|
|
84
84
|
1. **Steps must be concrete and verifiable** — "Run `curl -s localhost:3000/health`" not "Check the API"
|
|
85
|
-
2. **Evidence markers at verification points** — Place `[EVIDENCE: name]` at states that prove the change works.
|
|
85
|
+
2. **Evidence markers at verification points** — Place typed `[EVIDENCE: <artifact-type>: <name>]` at states that prove the change works. The type names HOW the proof is captured, the kebab-case name WHAT it proves (e.g., `[EVIDENCE: http-transcript: api-response-200]`, `[EVIDENCE: screenshot: error-state-rendered]`). An untyped assertion label like `[EVIDENCE: works-gracefully]` fails validation gate S14
|
|
86
86
|
3. **Include 2-5 evidence markers** — Enough to prove the change works across happy path and error cases
|
|
87
87
|
4. **Assertions are testable statements** — "Health check returns 200 with status ok" not "API works"
|
|
88
88
|
5. **Prerequisites include environment setup** — Database connection, env vars, running services
|