@codyswann/lisa 2.342.7 → 2.343.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/upstream-evidence-manifest.d.ts.map +1 -1
- package/dist/core/upstream-evidence-manifest.js +24 -13
- package/dist/core/upstream-evidence-manifest.js.map +1 -1
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa/.codex-plugin/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/lisa/rules/eager/verification.md +3 -2
- package/plugins/lisa/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/lisa/rules/reference/intent-routing.md +7 -6
- package/plugins/lisa/rules/reference/verification.md +7 -6
- package/plugins/lisa/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-agy/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-agy/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-agy/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-agy/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-agy/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-agy/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-agy/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-agy/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-agy/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/lisa-copilot/rules/eager/verification.md +3 -2
- package/plugins/lisa-copilot/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/lisa-copilot/rules/reference/intent-routing.md +7 -6
- package/plugins/lisa-copilot/rules/reference/verification.md +7 -6
- package/plugins/lisa-copilot/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-copilot/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-copilot/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-copilot/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-copilot/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-copilot/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-copilot/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-copilot/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-copilot/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-copilot/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/rules/bdd-e2e-coverage-reference.mdc +187 -0
- package/plugins/lisa-cursor/rules/bdd-e2e-coverage.mdc +54 -0
- package/plugins/lisa-cursor/rules/intent-routing-reference.mdc +7 -6
- package/plugins/lisa-cursor/rules/verification-reference.mdc +7 -6
- package/plugins/lisa-cursor/rules/verification.mdc +3 -2
- package/plugins/lisa-cursor/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-cursor/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-cursor/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-cursor/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-cursor/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-cursor/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-cursor/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-cursor/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-cursor/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-cursor/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/src/base/rules/eager/verification.md +3 -2
- package/plugins/src/base/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/src/base/rules/reference/intent-routing.md +7 -6
- package/plugins/src/base/rules/reference/verification.md +7 -6
- package/plugins/src/base/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/src/base/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/src/base/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/src/base/skills/lisa-research/SKILL.md +9 -1
- package/plugins/src/base/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/src/base/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/src/base/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/src/base/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/src/base/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/src/base/skills/lisa-verify/SKILL.md +1 -1
package/package.json
CHANGED
|
@@ -120,7 +120,7 @@
|
|
|
120
120
|
}
|
|
121
121
|
},
|
|
122
122
|
"name": "@codyswann/lisa",
|
|
123
|
-
"version": "2.
|
|
123
|
+
"version": "2.343.0",
|
|
124
124
|
"description": "Claude Code governance framework that applies guardrails, guidance, and automated enforcement to projects",
|
|
125
125
|
"main": "dist/index.js",
|
|
126
126
|
"exports": {
|
|
@@ -64,6 +64,7 @@ For each acceptance criterion:
|
|
|
64
64
|
- Write acceptance criteria from the user's perspective, not the developer's
|
|
65
65
|
- Every user flow must include at least one error path
|
|
66
66
|
- Use Gherkin format (Given/When/Then) for user flows to enable direct translation into test cases
|
|
67
|
+
- For frontend work, the `bdd-e2e-coverage` rule applies: name the scenario IDs the item will add or change in the project's behavior contract and the platforms each must hold on, and list the contract update plus the aligned e2e automation as explicit acceptance criteria -- they are deliverables, not implied work. List the passing coverage gate and the regenerated matrix and burndown as explicit acceptance criteria too -- the item is not done on a code-complete mapping alone
|
|
67
68
|
- When verifying, always run the feature -- never review by only reading code
|
|
68
69
|
- If you cannot run the feature (missing dependencies, services unavailable), report as a blocker -- do not guess
|
|
69
70
|
- If the changes are purely internal (refactoring, config, tooling), report "No user-facing impact" and explain why
|
|
@@ -57,9 +57,7 @@ If the empirical proof came from Kane, consume its exact objective, observable a
|
|
|
57
57
|
|
|
58
58
|
| Verification type | Preferred framework (use whichever the project already has) |
|
|
59
59
|
|---|---|
|
|
60
|
-
| UI (web) | Playwright
|
|
61
|
-
| UI (mobile) | Maestro > Detox > Playwright (mobile emulation) |
|
|
62
|
-
| UI (frontend, project supports multiple runners) | **ALL supported UI runners** — see "Frontend dual-runner codification" below |
|
|
60
|
+
| UI (web, mobile, or any frontend surface) | **The project's configured runner for every platform the behavior requires** — see "Frontend multi-runner codification" below. This row is authoritative for any UI work covered by a `bdd-e2e-coverage` scenario (in practice, essentially all user-facing UI work); it supersedes any generic runner preference — a project's web runner might be Playwright, Cypress, or Selenium, and its device runner might be Maestro, Detox, or a Playwright mobile-emulation profile, but the choice is read from `runnerPlatforms`, never assumed |
|
|
63
61
|
| API | project's integration test runner (Vitest / Jest / RSpec / pytest) with HTTP client (supertest / fetch / faraday) |
|
|
64
62
|
| Database | integration test with real DB + migrations applied |
|
|
65
63
|
| Auth | API or UI test asserting role-gated access (multi-role coverage) |
|
|
@@ -74,19 +72,24 @@ If the empirical proof came from Kane, consume its exact objective, observable a
|
|
|
74
72
|
|
|
75
73
|
If the project lacks the preferred framework AND no acceptable substitute exists, escalate.
|
|
76
74
|
|
|
77
|
-
### 2a. Frontend
|
|
75
|
+
### 2a. Frontend multi-runner codification (non-demotable)
|
|
78
76
|
|
|
79
|
-
For **frontend work** — any verification whose validation journey exercised a user-facing UI surface — codification is not one-runner-or-the-other.
|
|
77
|
+
For **frontend work** — any verification whose validation journey exercised a user-facing UI surface — codification is not one-runner-or-the-other. The `bdd-e2e-coverage` rule is the contract; this section is only how codification satisfies it. After the validation journey is complete and verified:
|
|
80
78
|
|
|
81
|
-
1. **
|
|
82
|
-
2. **
|
|
79
|
+
1. **Locate the behavior's scenario** in the project's behavior contract (`bdd/features/**` by default) — its stable `@BDD-*` ID and the platforms it declares. If the verified behavior has no scenario yet, write it now; that is part of codification, not a separate task. If the project has no contract yet, take the rule's bootstrap path, scoped to this behavior only.
|
|
80
|
+
2. **Codify into the project's configured runner for EVERY platform that scenario requires.** The runner→platform mapping is project configuration, declared in `bdd/coverage-map.json` under `runnerPlatforms` — read it rather than assuming a tool. Wire each new spec/flow where its runner already picks work up, following the project's existing directory and tagging conventions.
|
|
81
|
+
3. **Record the mapping.** Add one `mappings` entry per scenario-platform obligation naming the runner, platforms, file, and an `evidence` string that actually appears in that file, then regenerate the matrix and burndown so the gate reflects the new coverage.
|
|
82
|
+
4. **Run the coverage gate.** Invoke the project's configured `bdd-e2e-coverage` check command (the same one wired into CI) and confirm it passes. Regenerating the matrix and burndown only recomputes the report; it does not itself prove the gate is green. Record the command and its result in the codification evidence — a regenerated matrix with no observed gate run is not proof of coverage.
|
|
83
83
|
|
|
84
|
-
|
|
84
|
+
Every artifact encodes the SAME verified journey against a different platform. One is never a substitute for another, and a passing test on one platform never seals another platform's obligation.
|
|
85
85
|
|
|
86
|
-
Permitted exits, mirroring the regression-spec rule in `lisa-implement` (never a silent skip, never "optional"):
|
|
86
|
+
Permitted exits, mirroring the regression-spec rule in `lisa-implement` (never a silent skip, never "optional", and never a bare `N/A`):
|
|
87
87
|
|
|
88
|
-
- The project genuinely has no runner
|
|
89
|
-
-
|
|
88
|
+
- The project genuinely has no runner configured for that platform → record a dated `platformWaivers` entry naming the locations checked and "no runner configured" as the reason, exactly like any other unsealable obligation, per the rule. This is never left as a bare `N/A` — an undated absence has no forcing function to ever get revisited.
|
|
89
|
+
- The runner exists but genuinely cannot decide this behavior on that platform (no camera on the simulator, no request interception, an unprovisioned provider credential) → record a dated `platformWaivers` entry with the reason, per the rule. A waiver is an IOU, never coverage.
|
|
90
|
+
- A runner is configured and capable but the spec cannot be added or executed in this PR (genuine technical blocker) → create a linked build-ready follow-up ticket before merge, reference it from the PR and work item, and record the blocker — the same follow-up path as the regression-spec blocker.
|
|
91
|
+
|
|
92
|
+
Either of the first two exits also gets a linked build-ready follow-up ticket, referenced from the waiver's reason, whenever the runner could reasonably be added or the limitation could reasonably be lifted — the waiver records the IOU, the ticket is what pays it down.
|
|
90
93
|
|
|
91
94
|
### 3. Generate the test
|
|
92
95
|
|
|
@@ -107,7 +110,8 @@ For Playwright UI tests specifically:
|
|
|
107
110
|
|
|
108
111
|
**Concrete verification (UAT) contract.** Verification *is* UAT — codifying it is
|
|
109
112
|
how the playthrough becomes durable. For a runtime/behavioral `feat`/`fix`: place
|
|
110
|
-
the codified test
|
|
113
|
+
the codified test wherever the project's own configured e2e runner(s) already look
|
|
114
|
+
for tests — its own directory conventions, never a Lisa-assumed path or tool —
|
|
111
115
|
so CI re-runs it, and commit the evidence artifact to `evidence/<ticket>/`
|
|
112
116
|
(`verdict.json` + state + screenshots). For a Phaser game, drive the canvas
|
|
113
117
|
through the in-game verification test bridge (seed RNG, read state, inject input,
|
|
@@ -147,7 +147,7 @@ IF it is a Fix (bug), execute the Reproduce sub-flow FIRST:
|
|
|
147
147
|
1. Write a simple API client and call the offending API
|
|
148
148
|
2. Start the server on localhost and use the Playwright CLI or Chrome DevTools
|
|
149
149
|
|
|
150
|
-
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it. For frontend work
|
|
150
|
+
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it. For **frontend work** the deliverable is defined by the `bdd-e2e-coverage` rule and has two halves, both landing in this PR: the Gherkin scenario (stable ID, required platforms) added or updated in the project's behavior contract, and aligned automation in the project's configured runner for **every** platform that scenario requires — no runner substitutes for another, because they guard different platforms of the same behavior. The coverage gate must pass and the matrix and burndown be regenerated before the item is done. Cite the rule; do not restate its scenario, waiver, or bootstrap mechanics here.
|
|
151
151
|
|
|
152
152
|
The team lead may not waive, defer, demote, or phrase this regression spec as "optional", "if cheap", "nice to have", or equivalent. The only permitted exits are:
|
|
153
153
|
|
|
@@ -326,7 +326,7 @@ Before shutting down the team, execute the Verify flow:
|
|
|
326
326
|
boundary each claim reached, or name the violation. Where the runtime lacks the rules surface (the
|
|
327
327
|
agy artifacts carry no rules tree), the obligation still travels in this skill — cite the
|
|
328
328
|
`claim-evidence-mapping` contract by slug and continue; never block on the absent surface.
|
|
329
|
-
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified,
|
|
329
|
+
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified, the scenario and its aligned automation for **every platform the scenario requires**, per `codify-verification` and the `bdd-e2e-coverage` rule. Prove the new spec actually executed and passed in PR CI by recording a named spec log/reporter line or equivalent execution record; green CI without that named evidence does not satisfy this step.
|
|
330
330
|
4. Record Implement usage on the originating work artifact via `lisa-usage-accounting` so the work item (or other implementation-owned artifact) gains a direct `lisa-implement` usage entry in the canonical `## Lisa Usage` section. If the parent / child graph is already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise still write the direct entry, and if runtime usage is unavailable, use `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
331
331
|
5. Commit ALL outstanding changes in logical batches on the branch (minus sensitive data/information) — not just changes made by the agent team. This includes pre-existing uncommitted changes that were on the branch before the plan started. Do NOT filter commits to only "task-related" files. If it shows up in git status, it gets committed (unless it contains secrets).
|
|
332
332
|
6. Push the changes - if any pre-push hook blocks you, create a task for the agent team to fix the error/problem whether it was pre-existing or not
|
|
@@ -58,7 +58,15 @@ user stories (each with its own functional/non-functional requirements and, only
|
|
|
58
58
|
new UI/visual work, a design-file pointer), overall acceptance criteria, open questions, and the
|
|
59
59
|
"Recommended Tooling for Plan Phase" section. Requirements MUST conform to the
|
|
60
60
|
`prd-definition-of-ready` rule: identified atoms (`R1`, `R2`, …), one behavior each in an
|
|
61
|
-
EARS-pattern shape, each with a measurable fit criterion, plus the non-functional checklist.
|
|
61
|
+
EARS-pattern shape, each with a measurable fit criterion, plus the non-functional checklist. For any
|
|
62
|
+
story with frontend scope, its user-facing behavior MUST be expressed as — or in a shape directly
|
|
63
|
+
convertible to — Given/When/Then scenarios, naming the platforms each behavior must hold on, per the
|
|
64
|
+
`bdd-e2e-coverage` rule; narrative-only frontend behavior forces that shape to be invented later,
|
|
65
|
+
inconsistently, by whoever picks up the ticket. Each scenario stub carries its originating requirement
|
|
66
|
+
atom (`R1`, `R2`, …) even though the stable `@BDD-<DOMAIN>-<NNN>` ID is not minted until Plan or
|
|
67
|
+
Implement — this is what lets a later stage's provenance tag, and `spec-conformance`'s traceability
|
|
68
|
+
check, walk back from a shipped scenario to the PRD requirement it was written to satisfy, instead of
|
|
69
|
+
accepting any scenario with a stable ID as proof of this requirement. The final
|
|
62
70
|
flow step invokes `lisa-prd-source-write`, which creates the PRD in the configured `source` (Notion
|
|
63
71
|
page in the PRD database, Confluence page under the lifecycle parent, GitHub issue, or Linear
|
|
64
72
|
project) in the `draft` role by default or `ready` when `prd_ready=true`. **The PRD lives in the
|
|
@@ -42,6 +42,7 @@ Sections to extract:
|
|
|
42
42
|
| Deliverables | Each explicit deliverable (migration, doc, endpoint, script) | `deliverable` |
|
|
43
43
|
| Plan file tasks | Each task marked complete in the plan | `task` |
|
|
44
44
|
| Linked blocker resolutions | Each `is blocked by` that required work in this ticket | `blocker` |
|
|
45
|
+
| Behavior contract (frontend only) | Each Gherkin scenario the item was to add or change, and each required scenario-platform obligation it introduced | `behavior` |
|
|
45
46
|
|
|
46
47
|
If an acceptance criterion is not in Gherkin, still extract it as a requirement — but flag it as `LOW_SPECIFICITY` so the verdict downgrades.
|
|
47
48
|
Downgrade rule: if any `LOW_SPECIFICITY` requirement exists, the maximum possible verdict is `PARTIAL` unless the spec is tightened and re-evaluated.
|
|
@@ -96,7 +97,7 @@ For every requirement extracted in Phase 2, produce one row:
|
|
|
96
97
|
| Column | Value |
|
|
97
98
|
|--------|-------|
|
|
98
99
|
| Requirement ID | Stable identifier (e.g. `AC-1`, `OOS-2`, `ASSERT-3`) |
|
|
99
|
-
| Classification | `acceptance` / `excluded` / `technical` / `assertion` / `deliverable` / `task` / `blocker` |
|
|
100
|
+
| Classification | `acceptance` / `excluded` / `technical` / `assertion` / `deliverable` / `task` / `blocker` / `behavior` |
|
|
100
101
|
| Requirement Text | Verbatim from spec |
|
|
101
102
|
| Evidence | Specific pointer — file:line, test name, verification report section, PR file, screenshot name. When a v2 verdict exists, also name the `claim_id` and `evidence_id` that discharge it |
|
|
102
103
|
| Boundary | The claim's `boundary` from the v2 verdict (`code-unit` / `browser` / `http-api` / `cli` / `data` / `deploy-health` / `performance` / `standards-compat`), or `—` when no v2 claim maps to this row |
|
|
@@ -112,6 +113,19 @@ For every requirement extracted in Phase 2, produce one row:
|
|
|
112
113
|
- **`BOUNDARY_MISMATCH`** — the requirement was implemented and evidence was cited, but the evidence does not *reach* the claim's boundary (a unit `test-run-log` offered for a `browser` claim), or its `artifact_head_sha` does not match `artifact.head_sha`, or the verdict omits the required Not-established review. This is a distinct failure from a miss: the work may be right and the proof still does not establish it. A `BOUNDARY_MISMATCH` row forces the verdict to `DIVERGES` — it can never render as `CONFORMS` or `PARTIAL`. Name the boundary, the kind cited, and the kind(s) required, citing the `claim-evidence-mapping` taxonomy.
|
|
113
114
|
- **`SCOPE_CREEP_VIOLATION`** — used for `excluded` classification only. An Out-of-Scope item appears to have been shipped anyway. This is a different failure than a miss — it means the agent exceeded the spec.
|
|
114
115
|
|
|
116
|
+
### Behavior-contract rows (frontend work)
|
|
117
|
+
|
|
118
|
+
For `behavior` rows, evidence is the contract itself, not prose: the scenario exists in the project's behavior contract with its stable ID, and each required scenario-platform obligation resolves to a mapping in the coverage map (or a dated waiver). Read the generated coverage matrix and run the project's coverage gate rather than inferring from the diff.
|
|
119
|
+
|
|
120
|
+
- Shipped frontend behavior with **no scenario** → `MISSING`. It is a spec-conformance failure, not a documentation nit.
|
|
121
|
+
- A scenario satisfies this row only when its provenance tag (tracker item reference) names the current work item, or the PRD atom this item implements. A same-ID or coincidentally-matching scenario with no provenance link to this item does not satisfy the row — treat it as `MISSING` and flag the traceability gap; a passing gate elsewhere in the contract proves nothing about *this* requirement.
|
|
122
|
+
- Scenario present but a required scenario-platform obligation has neither a mapping nor a waiver, or the coverage gate fails (stale mapping, invalid waiver, floor regression) → `MISSING`.
|
|
123
|
+
- Scenario, mapping, and a passing coverage gate present, but the mapped test's execution has not been observed to pass (no verification-specialist evidence, no named passing CI run) → `PARTIAL`. The gate only proves the mapping is statically valid — the file exists and the `evidence` string is still present — not that the test currently passes; that is a separate signal per the `bdd-e2e-coverage` rule, and `MATCH` requires both.
|
|
124
|
+
- Scenario and mappings present but the gate was not re-run and the matrix not regenerated → `PARTIAL`.
|
|
125
|
+
- Scenario, mapping, passing gate, AND observed execution evidence that the mapped test passes → `MATCH`.
|
|
126
|
+
|
|
127
|
+
Per the `bdd-e2e-coverage` rule, a waiver is a dated IOU and never counts as coverage — a row backed only by a waiver is reported as `MISSING`, never `MATCH`, with the waiver and its `recordedAt` date named in the Notes column so the IOU stays visible in the matrix instead of disappearing into a passing status. (There is no separate `WAIVED` status; the existing enum is reused deliberately so a waiver is never mistaken for coverage.) A waiver satisfies the `bdd-e2e-coverage` rule's *definition of done* — it does not block the work item from shipping — but spec-conformance is a stricter accounting lens than that gate: reporting the row as `MISSING` caps this item below `CONFORMS` until the waiver is cleared. That is intentional, not a false failure — it is the mechanism that keeps a waiver from quietly becoming permanent.
|
|
128
|
+
|
|
115
129
|
### Scope creep detection
|
|
116
130
|
|
|
117
131
|
Separately from the matrix, scan the diff for work NOT traceable to any requirement. For each such change:
|
|
@@ -54,6 +54,8 @@ For each task, define what "done" looks like:
|
|
|
54
54
|
- Reference exact behavior: error messages, status codes, output format, performance thresholds
|
|
55
55
|
- If a task modifies existing behavior, state both the before and after
|
|
56
56
|
|
|
57
|
+
For a **frontend task** -- one that adds or changes a user-observable surface -- the `bdd-e2e-coverage` rule makes two further criteria mandatory on the item itself, never left implied: (a) the Gherkin scenarios it adds or changes in the project's behavior contract, with their stable IDs and required platforms, and (b) aligned e2e automation in the project's configured runner for each of those platforms, with the coverage gate passing and the matrix and burndown regenerated. Carry both into the item's Validation Journey. A project with no behavior contract yet does not get an exemption -- the first such task carries the bootstrap scaffolding as a deliverable, scoped to its own behavior (cite the rule; do not restate its bootstrap steps).
|
|
58
|
+
|
|
57
59
|
### 3. Assign Verification Type
|
|
58
60
|
|
|
59
61
|
Each task must have a verification method. Choose the most appropriate:
|
|
@@ -64,6 +64,7 @@ TDD Cycle:
|
|
|
64
64
|
- If the imported module doesn't exist, Jest reports 0 tests found (not N failed) — this is expected RED behavior
|
|
65
65
|
- For a Fix task, or a Build task that changes user-visible behavior, include a regression test at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the RED test plan must include a deterministic spec against the reported surface, using mocked or seeded data where needed.
|
|
66
66
|
- The team lead may not waive, defer, or mark that user-visible regression spec as optional, "if cheap", or equivalent. The only exits are a recorded absence of an end-to-end harness for the affected platform, or a genuine technical blocker with a linked build-ready follow-up ticket created before merge and referenced from the PR and source work item.
|
|
67
|
+
- For frontend work, the `bdd-e2e-coverage` rule governs what that spec is sealing: before writing it, add or update the Gherkin scenario with its stable ID in the project's behavior contract, then write aligned automation in the project's configured runner for each platform the scenario requires. The scenario is the specification the RED test encodes — write the scenario first, in the same PR.
|
|
67
68
|
- A regression spec is not complete merely because it exists. Completion evidence must prove the spec actually ran and passed in PR CI with a named log line, reporter output, or equivalent execution record. Guard against `test.skip`, suite-level environment gates, shard filters, and "0 tests" passes.
|
|
68
69
|
|
|
69
70
|
### GREEN Phase
|
|
@@ -12,8 +12,9 @@ Design test strategies, write tests, and review test quality.
|
|
|
12
12
|
1. **Read existing tests** -- understand the project's test conventions (describe/it structure, naming, helpers)
|
|
13
13
|
2. **Identify test types needed** -- unit, integration, E2E based on the scope of changes
|
|
14
14
|
3. **Map edge cases** -- boundary values, empty inputs, error states, concurrency scenarios
|
|
15
|
-
4. **
|
|
16
|
-
5. **
|
|
15
|
+
4. **For frontend scope, start from the behavior contract** -- read the project's Gherkin scenarios and coverage map, identify which scenario IDs this work adds or changes, and treat each required scenario-platform obligation as a strategy line item per the `bdd-e2e-coverage` rule
|
|
16
|
+
5. **Check coverage gaps** -- run existing tests to understand current coverage of affected files
|
|
17
|
+
6. **Design verification commands** -- proof commands that empirically demonstrate the code works
|
|
17
18
|
|
|
18
19
|
## Test Writing Process
|
|
19
20
|
|
|
@@ -56,6 +57,7 @@ Structure findings as:
|
|
|
56
57
|
|
|
57
58
|
- Always run `bun run test` to understand current test state before recommending or writing new tests
|
|
58
59
|
- Match existing test conventions -- do not introduce new test patterns
|
|
60
|
+
- For frontend work, the strategy is not complete until every required scenario-platform obligation has aligned e2e automation in the project's configured runner for that platform, or a dated waiver naming the runner limitation (`bdd-e2e-coverage`). A unit test, a route boot, or a passing test on a different platform never seals an obligation
|
|
59
61
|
- Every test must have a clear "why" -- no tests for testing's sake
|
|
60
62
|
- Focus on testing behavior, not implementation details
|
|
61
63
|
- Verification commands must be runnable locally (no CI/CD dependencies)
|
|
@@ -91,7 +91,7 @@ If auto-merge is enabled while the regression spec is still in flight, disable a
|
|
|
91
91
|
|
|
92
92
|
After each empirical verification produces PASS evidence, invoke the `codify-verification` skill to encode the verification as an automated regression test. The manual proof becomes a repeatable check that catches future regressions.
|
|
93
93
|
|
|
94
|
-
The `codify-verification` skill maps the verification type to the appropriate framework (Playwright for browser/UI, integration test for API/DB/auth, benchmark for performance, etc.), generates a deterministic test that asserts the same observable outcome the verification just confirmed, runs it in isolation to confirm PASS, and commits it in the same PR as the change. For **frontend work**, codification is
|
|
94
|
+
The `codify-verification` skill maps the verification type to the appropriate framework (Playwright for browser/UI, integration test for API/DB/auth, benchmark for performance, etc.), generates a deterministic test that asserts the same observable outcome the verification just confirmed, runs it in isolation to confirm PASS, and commits it in the same PR as the change. For **frontend work**, codification is multi-runner and governed by the `bdd-e2e-coverage` rule: the behavior exists as a Gherkin scenario with a stable ID in the project's behavior contract, and the same verified journey is encoded in the project's configured runner for every platform that scenario requires — neither a substitute for the other, since they guard different platforms.
|
|
95
95
|
|
|
96
96
|
Codification is mandatory for every empirical verification type with one exception set: PR, Documentation, Deploy, and Investigate-Only spikes — those have inherently non-behavioral proof. For every other type, skipping codification is not allowed; if codification is genuinely impossible (e.g., the test framework does not exist and cannot be installed in scope), escalate via the Escalation Protocol rather than silently skipping.
|
|
97
97
|
|
|
@@ -251,7 +251,7 @@ Agents must follow this sequence unless explicitly instructed otherwise:
|
|
|
251
251
|
8. Implement the change.
|
|
252
252
|
9. Execute verification plan — run the actual system and observe results.
|
|
253
253
|
10. Collect proof artifacts.
|
|
254
|
-
11. Codify — for each passing empirical verification, invoke `codify-verification` to encode it as a regression test (Playwright for UI, integration test for API/DB/auth, benchmark for performance, etc.) and commit the test in the same PR. Frontend work codifies into
|
|
254
|
+
11. Codify — for each passing empirical verification, invoke `codify-verification` to encode it as a regression test (Playwright for UI, integration test for API/DB/auth, benchmark for performance, etc.) and commit the test in the same PR. Frontend work also updates the behavior contract and codifies into the project's configured runner for every platform the scenario requires, then regenerates the coverage matrix (`bdd-e2e-coverage`; see the frontend multi-runner section of `codify-verification`).
|
|
255
255
|
12. Run spec conformance — build coverage matrix against the spec source (plan/ticket/issue), flag scope creep and untraceable changes, produce verdict.
|
|
256
256
|
13. Summarize what changed, what was verified, what was codified, conformance verdict, and remaining risk.
|
|
257
257
|
14. Label the result with a verification level.
|
|
@@ -373,6 +373,7 @@ A task is done only when:
|
|
|
373
373
|
- Required verification surfaces and tooling surfaces are used or explicitly unavailable
|
|
374
374
|
- Proof artifacts are captured
|
|
375
375
|
- Every passing empirical verification is codified as a regression test (or has an explicit, documented skip reason from the allowed set)
|
|
376
|
+
- For frontend work, the `bdd-e2e-coverage` contract is satisfied: every user-facing behavior added or changed exists as a Gherkin scenario with a stable ID traceable to the work item, every required scenario-platform obligation is mapped to aligned e2e automation in the project's configured runner for that platform or carries a dated waiver, and the coverage gate passes with the matrix and burndown regenerated and committed. Missing BDD coverage is a verification failure, not a warning
|
|
376
377
|
- For a leaf work unit, every typed `[EVIDENCE: <artifact-type>: <name>]` marker declared in its Validation Journey has a captured, non-empty artifact of the declared type attached to the ticket (the evidence manifest is fully satisfied)
|
|
377
378
|
- Cross-work-item `EVIDENCE-REF` pointers were excluded from the local manifest and did not satisfy S14 or completion; a runtime-changing leaf has at least one local claiming marker
|
|
378
379
|
- Spec conformance verdict is `CONFORMS` (not `PARTIAL`, not `DIVERGES`)
|
|
@@ -35,7 +35,7 @@ Treat the first successful lead-spawn request (or, on the Codex fallback, the fi
|
|
|
35
35
|
|
|
36
36
|
Execute the **Verify** flow as defined in the `intent-routing` rule (loaded via the lisa plugin). The flow includes:
|
|
37
37
|
|
|
38
|
-
1. **Pre-flight: codification gate** — confirm that every passing local empirical verification on this branch was codified as a regression test (the Implement flow's codify step). If any verification has no committed test and no allowed skip reason (PR / Documentation / Deploy / Investigate-Only), invoke `codify-verification` now and amend the PR before shipping. For frontend work the gate
|
|
38
|
+
1. **Pre-flight: codification gate** — confirm that every passing local empirical verification on this branch was codified as a regression test (the Implement flow's codify step). If any verification has no committed test and no allowed skip reason (PR / Documentation / Deploy / Investigate-Only), invoke `codify-verification` now and amend the PR before shipping. For frontend work the gate additionally enforces the `bdd-e2e-coverage` rule: the shipped behavior exists as a Gherkin scenario with a stable ID, every required scenario-platform obligation has aligned automation in the project's configured runner for that platform or a dated waiver, and the project's coverage gate passes with the matrix regenerated. A missing runner is never a bare recorded absence — it needs a dated `platformWaivers` entry (per the rule) and, where the runner could reasonably be added, a linked build-ready follow-up ticket referenced from it; never a silent skip. Missing BDD coverage is a verification failure that blocks the ship, not a warning. A change cannot ship until its verifications are guarded.
|
|
39
39
|
2. **Commit** any pending changes via `lisa-git-commit`
|
|
40
40
|
3. **Push and PR** via `lisa-git-submit-pr`
|
|
41
41
|
4. **PR Watch Loop** — drive the PR to MERGED via `lisa-drive-pr-to-merge`, the single source of truth for clearing every blocker: auto-merge with direct-merge fallback, `BEHIND` re-sync, conflict resolution, failing-check fixes, human + bot review-comment handling with thread resolution (it invokes `lisa-pull-request-review` itself), stale `CHANGES_REQUESTED` dismissal, and post-merge ancestry verification. Do not re-implement the loop or its terminal conditions.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# BDD Behavior Contract & E2E Coverage (load-bearing)
|
|
2
|
+
|
|
3
|
+
**Every frontend work item ships its behavior as Gherkin scenarios with stable IDs, and seals each required scenario-platform obligation with aligned e2e automation in the project's configured runner for that platform.** A frontend change that adds no scenario, or a required scenario-platform obligation with neither a mapped test nor a dated waiver, is a contract violation.
|
|
4
|
+
|
|
5
|
+
**One vendor-neutral contract, cited by** `lisa-research`, `lisa-acceptance-criteria`, `lisa-task-decomposition`, `lisa-test-strategy`, `lisa-tdd-implementation`, `lisa-implement`, `lisa-codify-verification`, `lisa-verification-lifecycle`, `lisa-spec-conformance`, and `lisa-verify` (the `leaf-only-lifecycle` / `repo-scope-split` precedent: one shared slug, never divergent per-skill prose).
|
|
6
|
+
|
|
7
|
+
## Membership
|
|
8
|
+
|
|
9
|
+
Membership is **surface, not repo name or ticket label**: a work item is frontend-type the moment it adds or changes something a user can observe — a screen, route, component, interaction, visual state, empty/error/loading state, navigation path, or copy a user reads. It applies on every platform the project ships. Backend-only, infrastructure, docs, tooling, and refactors with no user-observable delta are out; a refactor that changes what the user sees is in.
|
|
10
|
+
|
|
11
|
+
## The contract lives in the repo
|
|
12
|
+
|
|
13
|
+
Default locations (a project that already has an equivalent contract keeps its own paths and records them in `.lisa.config.json`):
|
|
14
|
+
|
|
15
|
+
| Artifact | Default path | What it is |
|
|
16
|
+
|---|---|---|
|
|
17
|
+
| Scenarios | `bdd/features/*.feature` | Gherkin, the product-level behavior contract |
|
|
18
|
+
| Coverage map | `bdd/coverage-map.json` | scenario → runner/platform/file/evidence mappings, plus waivers |
|
|
19
|
+
| Generated matrix | `docs/bdd-scenario-matrix.md` | regenerated, never hand-edited |
|
|
20
|
+
| Burndown | `docs/e2e-bdd-coverage.md` | current coverage per platform |
|
|
21
|
+
|
|
22
|
+
## Scenario shape
|
|
23
|
+
|
|
24
|
+
Every scenario carries **one stable `@BDD-<DOMAIN>-<NNN>` ID**, the platforms it requires (`@web`, `@ios`, `@android`, … as the project declares them), provenance (design node, ratified decision), and a tracker tag tying it to the work item that introduced or changed it. IDs are never renumbered or reused: behavior that stays user-observably the same keeps its ID through a rewrite; genuinely different behavior gets a new ID and the old one is marked `@superseded`.
|
|
25
|
+
|
|
26
|
+
Lifecycle tag, exactly one or none:
|
|
27
|
+
|
|
28
|
+
- **no tag** — ratified and required; it is in the coverage denominator.
|
|
29
|
+
- **`@blocked`** — specified but awaiting a decision or dependency; the *product* does not do this yet.
|
|
30
|
+
- **`@reference-only`** — intentionally not authoritative.
|
|
31
|
+
- **`@superseded`** — replaced by a ratified rule or a newer scenario.
|
|
32
|
+
|
|
33
|
+
The last three are out of the denominator. Deleting a committed capability to make coverage look better is a violation; mark it, do not drop it.
|
|
34
|
+
|
|
35
|
+
## What seals a scenario
|
|
36
|
+
|
|
37
|
+
Coverage is counted per **required scenario-platform obligation**, and an obligation is sealed only by **aligned e2e automation in the project's configured runner for that platform** — the runner→platform mapping is project configuration (e.g. a web runner covering `web`, a device runner covering `ios`/`android`), never named by this contract. A unit test, a route boot, a screenshot, a fixture, an API test, or a passing test on a *different* platform never seals an obligation: they prove something else. Each mapping names the runner, the platforms, the file, and the evidence string inside it, so the gate can prove the mapping still resolves.
|
|
38
|
+
|
|
39
|
+
**Waivers are dated IOUs, never coverage.** An obligation whose runner genuinely cannot decide it today (no camera on the simulator, no request interception, no provisioned provider credential) is recorded with scenario, platforms, reason, and `recordedAt`. It leaves the denominator; it never counts as covered, and nothing about it asserts the behavior works. A waiver is invalid — and fails the gate — when it names a nonexistent scenario or an undeclared platform, covers an already-excluded scenario, or masks a scenario-platform that already has a mapping. Revisit a waiver when its reason dies.
|
|
40
|
+
|
|
41
|
+
## Definition of done
|
|
42
|
+
|
|
43
|
+
A frontend work item is not done until: (1) every user-facing behavior it adds or changes exists as a Gherkin scenario with a stable ID in the project's behavior contract, traceable to the work item; (2) every required scenario-platform obligation it introduces is either mapped to aligned e2e automation in the project's configured runner for that platform, or carries a dated waiver naming the runner limitation; and (3) the coverage gate passes and the generated matrix and burndown are regenerated and committed in the same PR. **Missing BDD coverage is a verification failure, not a warning** — it blocks completion exactly as a `PARTIAL` spec-conformance verdict does, and it is never demoted to "optional", "if cheap", or a follow-up without a linked build-ready ticket created before merge.
|
|
44
|
+
|
|
45
|
+
## Bootstrap and degradation
|
|
46
|
+
|
|
47
|
+
A project with no `bdd/` contract yet is not exempt — the first frontend work item scaffolds the minimum (features dir, coverage map with the project's detected runner→platform mapping, gate + matrix scripts) and covers **its own** behavior. It never backfills the whole app: pre-existing uncovered behavior is recorded as burndown, not treated as this item's blocker. If no e2e runner exists for a required platform at all, record the checked locations and the absence as the reason, exactly as the regression-spec absence path in `verification` does — a silent skip is never one of the exits.
|
|
48
|
+
|
|
49
|
+
Full contract (coverage-map schema, gate semantics, bootstrap procedure, per-flow obligations): [reference/bdd-e2e-coverage.md](../reference/bdd-e2e-coverage.md).
|
|
@@ -13,11 +13,12 @@
|
|
|
13
13
|
- **A check that cannot fail is not evidence either.** Every gate, probe, sweep, and codified spec you author is subject to the `falsifiable-checks` rule: break the guarded property, observe the check fail and name the location, then restore — and report that falsification with the result. "Mentally reverting" does not count, and an unfalsified gate is reported as *unvalidated*, never as passing.
|
|
14
14
|
- **Browser-controller neutrality.** For UI work, control a live browser and perform the Validation Journey as a human would. An in-app Browser/Chrome tool, interactive Playwright control (MCP, API, or ad hoc script), CDP, computer use, the optional Lisa-owned Kane adapter, or an equivalent controller is acceptable. Kane requires explicit upload approval, a passing `lisa kane probe`, an allow-listed non-production environment, and mutation policy `full`; its provider failure is not a product failure. Do not block merely because one preferred backend is unavailable when another interactive controller can drive the browser. Running an automated Playwright or Maestro test alone is still a quality gate, not the initial empirical evidence; after the live journey passes, codify it in the applicable native runner(s). Kane never replaces those regression gates.
|
|
15
15
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
16
|
-
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes. For **frontend work**, codification is
|
|
17
|
-
- **The codified proof re-runs in CI.** Codified runtime verification lives
|
|
16
|
+
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes. For **frontend work**, codification is multi-runner and governed by the `bdd-e2e-coverage` rule: the verified journey is encoded in the project's configured runner for **every** platform the behavior's scenario requires — one runner is never a substitute for another, because they guard different platforms of the same behavior. Cite that rule; do not restate its obligations or name a runner as though the contract mandated one.
|
|
17
|
+
- **The codified proof re-runs in CI.** Codified runtime verification lives wherever the project's own configured e2e runner(s) already look for tests — its own config and conventions, never a Lisa-assumed path or tool — and runs as a required CI check for types with verification enforced, so the proof re-runs on every PR, not once by hand. For **frontend work** specifically, where that runner lives and how many runners are required is governed by the `bdd-e2e-coverage` rule, not this bullet.
|
|
18
18
|
- **Commit the evidence.** Write a durable artifact to `evidence/<ticket>/` — the acceptance criteria with per-criterion pass/fail + note, the observed state, screenshots/recording, and a `verdict.json`. The transient `.lisa/verification-status.json` is the session gate; `evidence/<ticket>/` is the committed proof.
|
|
19
19
|
- **Evidence markers are typed artifacts, not assertion labels.** A Validation Journey marker is `[EVIDENCE: <artifact-type>: <name>]` where the type (screenshot, recording, http-transcript, cli-output, log-snippet, db-query-output, perf-trace, test-run-log, deploy-log, state-dump) says HOW the proof is captured and the name says WHAT it proves. `[EVIDENCE: works-gracefully]` is a claim, not evidence — write `[EVIDENCE: screenshot: load-failure-error-state]`. Completion requires a captured artifact **of the declared type** per marker. To cite another work item's artifact without claiming it, write `[EVIDENCE-REF: <work-item-ref> | <artifact-type>: <kebab-case-name>]`. The Lisa 2.223.0 colon form `[EVIDENCE-REF: <tracker-ref>: <artifact-type>: <kebab-case-name>]` remains accepted as a legacy non-claiming alias but must not be newly authored. A reference never enters or satisfies the local S14 manifest; malformed references fail S14, and agents must not quote sibling `[EVIDENCE: ...]` markers directly.
|
|
20
20
|
- **Per-change verification is mandatory.** Every `feat`/`fix` adds or extends a verification (e2e) spec mapped to its acceptance criteria. The only exception is a genuinely non-behavioral change explicitly marked with the **logged** `verification-exempt` label — never a silent skip.
|
|
21
|
+
- **Frontend work also seals its behavior contract.** Per the `bdd-e2e-coverage` rule, a work item that adds or changes a user-observable surface is not done until its behavior exists as Gherkin scenarios with stable IDs, every required scenario-platform obligation is mapped to aligned e2e automation (or carries a dated waiver), and the coverage gate passes with the matrix regenerated. Missing BDD coverage is a verification failure, not a warning.
|
|
21
22
|
- **Drive the playthrough with `/lisa:product-walkthrough`** (it walks the live product through a real browser); each project type plugs in its own drive mechanism (e.g. a Phaser game is driven through Playwright + an in-game verification test bridge that seeds RNG, reads state, injects input, and steps frames).
|
|
22
23
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
23
24
|
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
# BDD Behavior Contract & E2E Coverage
|
|
2
|
+
|
|
3
|
+
Frontend behavior has historically been specified twice and owned nowhere: once as prose acceptance
|
|
4
|
+
criteria on a ticket that closes and disappears, and once as whatever e2e tests someone had time to
|
|
5
|
+
write. Neither survives the work item. The cost lands at verification time — nobody can answer "what
|
|
6
|
+
is this app supposed to do?" without re-reading a year of closed tickets, and "is that behavior
|
|
7
|
+
guarded?" has a different answer shape per platform and per runner. This contract writes both
|
|
8
|
+
answers down once, in the repo, next to the code.
|
|
9
|
+
|
|
10
|
+
It is a **single vendor-neutral contract** consumed by `lisa-research` (PRDs express frontend
|
|
11
|
+
behavior in a BDD-convertible shape), `lisa-acceptance-criteria` and `lisa-task-decomposition`
|
|
12
|
+
(the contract update and the e2e sealing are explicit obligations on the work item, not implied
|
|
13
|
+
work), `lisa-test-strategy` and `lisa-tdd-implementation` and `lisa-implement` (building a frontend
|
|
14
|
+
item includes both), `lisa-codify-verification` (the verified journey is codified into every runner
|
|
15
|
+
the scenario's platforms require), and `lisa-verification-lifecycle` / `lisa-spec-conformance` /
|
|
16
|
+
`lisa-verify` (the gate is checked, and a miss fails the item). Those skills cite this slug; they do
|
|
17
|
+
not restate it.
|
|
18
|
+
|
|
19
|
+
## Vendor neutrality is the point
|
|
20
|
+
|
|
21
|
+
This contract never names a test runner. Projects use Playwright, Cypress, Maestro, Detox, Appium,
|
|
22
|
+
WebdriverIO, XCUITest, Espresso, an in-house harness, or several at once, and that choice is
|
|
23
|
+
project configuration that changes without touching this contract. What the contract fixes is the
|
|
24
|
+
*obligation*: a required scenario-platform pair is sealed by aligned automation **in whatever runner
|
|
25
|
+
the project has configured for that platform**. Any tool named anywhere downstream is an example,
|
|
26
|
+
never a mandate.
|
|
27
|
+
|
|
28
|
+
The runner→platform mapping is declared once, in the coverage map, and is the only place a tool name
|
|
29
|
+
is authoritative:
|
|
30
|
+
|
|
31
|
+
```json
|
|
32
|
+
"runnerPlatforms": {
|
|
33
|
+
"<web-runner>": ["web"],
|
|
34
|
+
"<device-runner>": ["ios", "android"]
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Platform vocabulary is the project's own — a project that ships only web declares only `web`; one
|
|
39
|
+
that ships a desktop or TV surface declares those. The contract cares that each scenario names the
|
|
40
|
+
platforms it requires and that each named platform has a configured runner.
|
|
41
|
+
|
|
42
|
+
## Coverage map schema
|
|
43
|
+
|
|
44
|
+
`bdd/coverage-map.json` is the machine-readable half of the contract. Minimum shape:
|
|
45
|
+
|
|
46
|
+
```json
|
|
47
|
+
{
|
|
48
|
+
"schemaVersion": 1,
|
|
49
|
+
"asOf": "<ISO date>",
|
|
50
|
+
"runnerPlatforms": { "<runner>": ["<platform>", "..."] },
|
|
51
|
+
"coverageFloor": { "<platform>": "<0-100, the committed floor — may rise, never falls>" },
|
|
52
|
+
"platformWaivers": [
|
|
53
|
+
{
|
|
54
|
+
"scenario": "BDD-DOMAIN-NNN",
|
|
55
|
+
"platforms": ["<platform>"],
|
|
56
|
+
"runner": "<runner — required when more than one runner covers a waived platform, so the waiver names which one cannot decide it>",
|
|
57
|
+
"reason": "why this runner cannot decide this behavior today",
|
|
58
|
+
"recordedAt": "<ISO date>"
|
|
59
|
+
}
|
|
60
|
+
],
|
|
61
|
+
"mappings": [
|
|
62
|
+
{
|
|
63
|
+
"scenario": "BDD-DOMAIN-NNN",
|
|
64
|
+
"runner": "<runner>",
|
|
65
|
+
"platforms": ["<platform>", "..."],
|
|
66
|
+
"file": "<path to the spec/flow>",
|
|
67
|
+
"evidence": "<exact string inside that file — the test title or flow name>",
|
|
68
|
+
"level": "behavioral"
|
|
69
|
+
}
|
|
70
|
+
],
|
|
71
|
+
"exclusions": [
|
|
72
|
+
{ "file": "<path>", "reason": "why this test aligns to no product behavior" }
|
|
73
|
+
]
|
|
74
|
+
}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
`evidence` is what makes a mapping falsifiable: the gate reads the mapped file and confirms the
|
|
78
|
+
string is still there, so renaming or deleting a test breaks the map loudly instead of leaving a
|
|
79
|
+
scenario silently unguarded. `exclusions` records tests that deliberately map to nothing — starter
|
|
80
|
+
templates, source-level corroboration — so "unmapped test" stays a meaningful signal.
|
|
81
|
+
|
|
82
|
+
## The gate
|
|
83
|
+
|
|
84
|
+
Two commands, wired into the project's script surface and into CI:
|
|
85
|
+
|
|
86
|
+
1. **Regenerate** — recompute coverage from features + map, rewrite the generated matrix
|
|
87
|
+
(`docs/bdd-scenario-matrix.md`) and the burndown (`docs/e2e-bdd-coverage.md`), and refresh the
|
|
88
|
+
machine-readable report. Low coverage is reported honestly and does **not** fail this command.
|
|
89
|
+
2. **Check** — the CI gate. It fails on contract defects, not on ambition:
|
|
90
|
+
- a duplicate or malformed scenario ID;
|
|
91
|
+
- a scenario declaring a platform with no configured runner;
|
|
92
|
+
- a mapping whose file is missing or whose `evidence` string is gone (stale mapping);
|
|
93
|
+
- a mapping claiming a platform the scenario does not declare;
|
|
94
|
+
- an invalid waiver (nonexistent scenario, undeclared platform, already-excluded scenario, a
|
|
95
|
+
`runner` that does not cover the waived platform under `runnerPlatforms`, or one that masks an
|
|
96
|
+
existing mapping);
|
|
97
|
+
- a regression against the project's committed `coverageFloor` per platform.
|
|
98
|
+
|
|
99
|
+
The percentage measures **aligned automation inventory, not the latest run result**. A mapped test
|
|
100
|
+
that currently fails is a red CI check, a separate signal; the map only asserts the automation
|
|
101
|
+
exists and still says what it claimed. Both facts are required — a green gate over a red suite is
|
|
102
|
+
not coverage.
|
|
103
|
+
|
|
104
|
+
### Coverage floor, not coverage target
|
|
105
|
+
|
|
106
|
+
Projects adopting this contract mid-life start below 100% and must not be blocked by that. The gate
|
|
107
|
+
enforces a **ratchet**: the committed floor per platform may rise and may never fall. Recorded in
|
|
108
|
+
`coverageFloor`, it is bumped by regenerating the matrix after a frontend work item raises it by the
|
|
109
|
+
obligations it seals; it is never asked to clear the whole backlog.
|
|
110
|
+
|
|
111
|
+
## Waivers versus `@blocked`
|
|
112
|
+
|
|
113
|
+
These are different failures and must never be conflated:
|
|
114
|
+
|
|
115
|
+
| | Means | Recorded as |
|
|
116
|
+
|---|---|---|
|
|
117
|
+
| `@blocked` | The **product** does not do this yet — undesigned, undecided, or unbuilt. | Lifecycle tag on the scenario |
|
|
118
|
+
| Waiver | The product may well do it, but this **runner** cannot decide it on this platform today. | Dated entry in `platformWaivers` |
|
|
119
|
+
|
|
120
|
+
Both leave the denominator, and neither is ever counted as covered: a waiver is a dated IOU, never
|
|
121
|
+
coverage. A waiver names a runner limitation — no camera on
|
|
122
|
+
the simulator, no request interception, a provider sign-in needing a real credential nobody
|
|
123
|
+
provisioned, a backend with no seedable row for the required state — and it carries a date so it can
|
|
124
|
+
be audited for age. When the limitation dies, the waiver is deleted and the automation is written;
|
|
125
|
+
a waiver that outlives its reason is a defect the burndown should surface.
|
|
126
|
+
|
|
127
|
+
## Scenario identity and change
|
|
128
|
+
|
|
129
|
+
A stable ID is a promise about *behavior*, not about text:
|
|
130
|
+
|
|
131
|
+
- The user-observable behavior is unchanged → **keep the ID**, however much the wording, the
|
|
132
|
+
implementation, or the design node changed.
|
|
133
|
+
- The behavior is genuinely different → **new ID**; mark the old one `@superseded` and leave it in
|
|
134
|
+
the file. Superseded scenarios are the audit trail for why the product changed.
|
|
135
|
+
- The capability is committed but not yet designed or built → write it now and tag it `@blocked`.
|
|
136
|
+
Omitting a committed capability makes coverage look better by describing less of the product,
|
|
137
|
+
which is exactly the failure this contract exists to prevent.
|
|
138
|
+
- IDs are never renumbered or reused. Gaps in the sequence are normal and harmless.
|
|
139
|
+
|
|
140
|
+
Provenance tags tie a scenario back to its authority (design node, ratified decision, wiki
|
|
141
|
+
requirement) and to the tracker item that introduced or changed it, so `spec-conformance` can walk
|
|
142
|
+
from a requirement to a scenario to a test without guessing.
|
|
143
|
+
|
|
144
|
+
## Per-flow obligations
|
|
145
|
+
|
|
146
|
+
| Flow | Obligation |
|
|
147
|
+
|---|---|
|
|
148
|
+
| **Research / PRD** | For frontend scope, user-facing behavior is written as — or in a shape directly convertible to — Given/When/Then scenarios, with the platforms each behavior must hold on. A PRD that describes frontend behavior only as narrative forces the shape to be invented later, inconsistently. |
|
|
149
|
+
| **Plan / decomposition / acceptance criteria** | Each frontend work item's acceptance criteria and Validation Journey name the contract update *and* the e2e sealing as explicit deliverables, with the scenario IDs it will add or change once known. They are line items, not implied work. |
|
|
150
|
+
| **Implement / TDD / test strategy** | Building the item updates `bdd/features` and the coverage map and writes the aligned automation, in the same PR as the behavior. The scenario is the RED test's specification for user-facing behavior. |
|
|
151
|
+
| **Codify verification** | The verified journey is codified into **every runner the scenario's platforms require** — one runner is never a substitute for another, because they guard different platforms of the same behavior. |
|
|
152
|
+
| **Verify / verification-lifecycle / spec-conformance** | Verification confirms scenarios exist for the shipped behavior and the coverage gate passes. A shipped frontend behavior with no scenario, or a required obligation with neither mapping nor waiver, is a verification **failure** — not a warning, not a nit. |
|
|
153
|
+
|
|
154
|
+
## Bootstrap
|
|
155
|
+
|
|
156
|
+
A repo with no contract yet, taking its first frontend work item:
|
|
157
|
+
|
|
158
|
+
1. **Detect runners.** Find the e2e harnesses the project actually has, per the Tool Discovery
|
|
159
|
+
Process in `verification-lifecycle`. Record which platforms each covers.
|
|
160
|
+
2. **Scaffold the minimum** — `bdd/features/`, `bdd/coverage-map.json` (with `runnerPlatforms` from
|
|
161
|
+
step 1 and a coverage floor of the current, honest number), the regenerate + check scripts wired
|
|
162
|
+
into the project's script surface, and the CI invocation of the check.
|
|
163
|
+
3. **Write only this item's scenarios.** The first item is not a backfill project. Pre-existing
|
|
164
|
+
uncovered behavior becomes burndown in `docs/e2e-bdd-coverage.md`, and the floor starts where the
|
|
165
|
+
repo actually is.
|
|
166
|
+
4. **Seal this item's obligations** and commit the regenerated matrix and burndown with the change.
|
|
167
|
+
|
|
168
|
+
If a required platform has **no** e2e runner at all, that obligation is never left as a bare `N/A` —
|
|
169
|
+
record it exactly like any other unsealable obligation: a dated `platformWaivers` entry naming the
|
|
170
|
+
locations checked and "no runner configured" as the reason, so the absence is dated, audited for age,
|
|
171
|
+
and visible in the same place every other IOU lives — the same recorded-absence exit the
|
|
172
|
+
regression-spec rule in `verification` allows, made falsifiable. Where the runner could reasonably be
|
|
173
|
+
added, additionally file a linked build-ready follow-up ticket before merge and reference it from the
|
|
174
|
+
waiver's reason. Silence is never an exit.
|
|
175
|
+
|
|
176
|
+
## Degradation
|
|
177
|
+
|
|
178
|
+
Missing scaffolding degrades the run, it does not crash it: a project with no contract gets the
|
|
179
|
+
bootstrap path, an unreadable coverage map is reported as a blocker on the work item rather than
|
|
180
|
+
swallowed, and an absent optional runner is recorded as an absence with the locations checked. What
|
|
181
|
+
never degrades is the reporting — an agent that could not evaluate the contract says so explicitly
|
|
182
|
+
and does not report the item as done.
|