@codyswann/lisa 2.342.7 → 2.344.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/upstream-evidence-manifest.d.ts.map +1 -1
- package/dist/core/upstream-evidence-manifest.js +25 -14
- package/dist/core/upstream-evidence-manifest.js.map +1 -1
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa/.codex-plugin/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa/.codex-plugin/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/lisa/rules/eager/verification.md +3 -2
- package/plugins/lisa/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/lisa/rules/reference/intent-routing.md +7 -6
- package/plugins/lisa/rules/reference/verification.md +7 -6
- package/plugins/lisa/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/lisa/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-agy/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-agy/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-agy/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/lisa-agy/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-agy/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-agy/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-agy/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-agy/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-agy/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-agy/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/lisa-copilot/rules/eager/verification.md +3 -2
- package/plugins/lisa-copilot/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/lisa-copilot/rules/reference/intent-routing.md +7 -6
- package/plugins/lisa-copilot/rules/reference/verification.md +7 -6
- package/plugins/lisa-copilot/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-copilot/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-copilot/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-copilot/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/lisa-copilot/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-copilot/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-copilot/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-copilot/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-copilot/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-copilot/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-copilot/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/rules/bdd-e2e-coverage-reference.mdc +187 -0
- package/plugins/lisa-cursor/rules/bdd-e2e-coverage.mdc +54 -0
- package/plugins/lisa-cursor/rules/intent-routing-reference.mdc +7 -6
- package/plugins/lisa-cursor/rules/verification-reference.mdc +7 -6
- package/plugins/lisa-cursor/rules/verification.mdc +3 -2
- package/plugins/lisa-cursor/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/lisa-cursor/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/lisa-cursor/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/lisa-cursor/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/lisa-cursor/skills/lisa-research/SKILL.md +9 -1
- package/plugins/lisa-cursor/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/lisa-cursor/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/lisa-cursor/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/lisa-cursor/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/lisa-cursor/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/lisa-cursor/skills/lisa-verify/SKILL.md +1 -1
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/rules/eager/bdd-e2e-coverage.md +49 -0
- package/plugins/src/base/rules/eager/verification.md +3 -2
- package/plugins/src/base/rules/reference/bdd-e2e-coverage.md +182 -0
- package/plugins/src/base/rules/reference/intent-routing.md +7 -6
- package/plugins/src/base/rules/reference/verification.md +7 -6
- package/plugins/src/base/skills/lisa-acceptance-criteria/SKILL.md +1 -0
- package/plugins/src/base/skills/lisa-codify-verification/SKILL.md +16 -12
- package/plugins/src/base/skills/lisa-implement/SKILL.md +2 -2
- package/plugins/src/base/skills/lisa-parity-safety-net-rules/SKILL.md +13 -2
- package/plugins/src/base/skills/lisa-research/SKILL.md +9 -1
- package/plugins/src/base/skills/lisa-spec-conformance/SKILL.md +15 -1
- package/plugins/src/base/skills/lisa-task-decomposition/SKILL.md +2 -0
- package/plugins/src/base/skills/lisa-tdd-implementation/SKILL.md +1 -0
- package/plugins/src/base/skills/lisa-test-strategy/SKILL.md +4 -2
- package/plugins/src/base/skills/lisa-verification-lifecycle/SKILL.md +3 -2
- package/plugins/src/base/skills/lisa-verify/SKILL.md +1 -1
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# BDD Behavior Contract & E2E Coverage (load-bearing)
|
|
2
|
+
|
|
3
|
+
**Every frontend work item ships its behavior as Gherkin scenarios with stable IDs, and seals each required scenario-platform obligation with aligned e2e automation in the project's configured runner for that platform.** A frontend change that adds no scenario, or a required scenario-platform obligation with neither a mapped test nor a dated waiver, is a contract violation.
|
|
4
|
+
|
|
5
|
+
**One vendor-neutral contract, cited by** `lisa-research`, `lisa-acceptance-criteria`, `lisa-task-decomposition`, `lisa-test-strategy`, `lisa-tdd-implementation`, `lisa-implement`, `lisa-codify-verification`, `lisa-verification-lifecycle`, `lisa-spec-conformance`, and `lisa-verify` (the `leaf-only-lifecycle` / `repo-scope-split` precedent: one shared slug, never divergent per-skill prose).
|
|
6
|
+
|
|
7
|
+
## Membership
|
|
8
|
+
|
|
9
|
+
Membership is **surface, not repo name or ticket label**: a work item is frontend-type the moment it adds or changes something a user can observe — a screen, route, component, interaction, visual state, empty/error/loading state, navigation path, or copy a user reads. It applies on every platform the project ships. Backend-only, infrastructure, docs, tooling, and refactors with no user-observable delta are out; a refactor that changes what the user sees is in.
|
|
10
|
+
|
|
11
|
+
## The contract lives in the repo
|
|
12
|
+
|
|
13
|
+
Default locations (a project that already has an equivalent contract keeps its own paths and records them in `.lisa.config.json`):
|
|
14
|
+
|
|
15
|
+
| Artifact | Default path | What it is |
|
|
16
|
+
|---|---|---|
|
|
17
|
+
| Scenarios | `bdd/features/*.feature` | Gherkin, the product-level behavior contract |
|
|
18
|
+
| Coverage map | `bdd/coverage-map.json` | scenario → runner/platform/file/evidence mappings, plus waivers |
|
|
19
|
+
| Generated matrix | `docs/bdd-scenario-matrix.md` | regenerated, never hand-edited |
|
|
20
|
+
| Burndown | `docs/e2e-bdd-coverage.md` | current coverage per platform |
|
|
21
|
+
|
|
22
|
+
## Scenario shape
|
|
23
|
+
|
|
24
|
+
Every scenario carries **one stable `@BDD-<DOMAIN>-<NNN>` ID**, the platforms it requires (`@web`, `@ios`, `@android`, … as the project declares them), provenance (design node, ratified decision), and a tracker tag tying it to the work item that introduced or changed it. IDs are never renumbered or reused: behavior that stays user-observably the same keeps its ID through a rewrite; genuinely different behavior gets a new ID and the old one is marked `@superseded`.
|
|
25
|
+
|
|
26
|
+
Lifecycle tag, exactly one or none:
|
|
27
|
+
|
|
28
|
+
- **no tag** — ratified and required; it is in the coverage denominator.
|
|
29
|
+
- **`@blocked`** — specified but awaiting a decision or dependency; the *product* does not do this yet.
|
|
30
|
+
- **`@reference-only`** — intentionally not authoritative.
|
|
31
|
+
- **`@superseded`** — replaced by a ratified rule or a newer scenario.
|
|
32
|
+
|
|
33
|
+
The last three are out of the denominator. Deleting a committed capability to make coverage look better is a violation; mark it, do not drop it.
|
|
34
|
+
|
|
35
|
+
## What seals a scenario
|
|
36
|
+
|
|
37
|
+
Coverage is counted per **required scenario-platform obligation**, and an obligation is sealed only by **aligned e2e automation in the project's configured runner for that platform** — the runner→platform mapping is project configuration (e.g. a web runner covering `web`, a device runner covering `ios`/`android`), never named by this contract. A unit test, a route boot, a screenshot, a fixture, an API test, or a passing test on a *different* platform never seals an obligation: they prove something else. Each mapping names the runner, the platforms, the file, and the evidence string inside it, so the gate can prove the mapping still resolves.
|
|
38
|
+
|
|
39
|
+
**Waivers are dated IOUs, never coverage.** An obligation whose runner genuinely cannot decide it today (no camera on the simulator, no request interception, no provisioned provider credential) is recorded with scenario, platforms, reason, and `recordedAt`. It leaves the denominator; it never counts as covered, and nothing about it asserts the behavior works. A waiver is invalid — and fails the gate — when it names a nonexistent scenario or an undeclared platform, covers an already-excluded scenario, or masks a scenario-platform that already has a mapping. Revisit a waiver when its reason dies.
|
|
40
|
+
|
|
41
|
+
## Definition of done
|
|
42
|
+
|
|
43
|
+
A frontend work item is not done until: (1) every user-facing behavior it adds or changes exists as a Gherkin scenario with a stable ID in the project's behavior contract, traceable to the work item; (2) every required scenario-platform obligation it introduces is either mapped to aligned e2e automation in the project's configured runner for that platform, or carries a dated waiver naming the runner limitation; and (3) the coverage gate passes and the generated matrix and burndown are regenerated and committed in the same PR. **Missing BDD coverage is a verification failure, not a warning** — it blocks completion exactly as a `PARTIAL` spec-conformance verdict does, and it is never demoted to "optional", "if cheap", or a follow-up without a linked build-ready ticket created before merge.
|
|
44
|
+
|
|
45
|
+
## Bootstrap and degradation
|
|
46
|
+
|
|
47
|
+
A project with no `bdd/` contract yet is not exempt — the first frontend work item scaffolds the minimum (features dir, coverage map with the project's detected runner→platform mapping, gate + matrix scripts) and covers **its own** behavior. It never backfills the whole app: pre-existing uncovered behavior is recorded as burndown, not treated as this item's blocker. If no e2e runner exists for a required platform at all, record the checked locations and the absence as the reason, exactly as the regression-spec absence path in `verification` does — a silent skip is never one of the exits.
|
|
48
|
+
|
|
49
|
+
Full contract (coverage-map schema, gate semantics, bootstrap procedure, per-flow obligations): [reference/bdd-e2e-coverage.md](../reference/bdd-e2e-coverage.md).
|
|
@@ -13,11 +13,12 @@
|
|
|
13
13
|
- **A check that cannot fail is not evidence either.** Every gate, probe, sweep, and codified spec you author is subject to the `falsifiable-checks` rule: break the guarded property, observe the check fail and name the location, then restore — and report that falsification with the result. "Mentally reverting" does not count, and an unfalsified gate is reported as *unvalidated*, never as passing.
|
|
14
14
|
- **Browser-controller neutrality.** For UI work, control a live browser and perform the Validation Journey as a human would. An in-app Browser/Chrome tool, interactive Playwright control (MCP, API, or ad hoc script), CDP, computer use, the optional Lisa-owned Kane adapter, or an equivalent controller is acceptable. Kane requires explicit upload approval, a passing `lisa kane probe`, an allow-listed non-production environment, and mutation policy `full`; its provider failure is not a product failure. Do not block merely because one preferred backend is unavailable when another interactive controller can drive the browser. Running an automated Playwright or Maestro test alone is still a quality gate, not the initial empirical evidence; after the live journey passes, codify it in the applicable native runner(s). Kane never replaces those regression gates.
|
|
15
15
|
- **Before starting implementation, state your verification plan** — how you will USE the resulting software to prove it works. A plan that only lists `test`/`typecheck`/`lint` commands is not a plan. Do not begin until confirmed.
|
|
16
|
-
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes. For **frontend work**, codification is
|
|
17
|
-
- **The codified proof re-runs in CI.** Codified runtime verification lives
|
|
16
|
+
- **After verifying empirically, codify it as a regression test** via the `codify-verification` skill — Playwright for UI, integration test for API/DB/auth, benchmark for performance. Codification is mandatory for every verification type except PR/Documentation/Deploy and Investigate-Only spikes. For **frontend work**, codification is multi-runner and governed by the `bdd-e2e-coverage` rule: the verified journey is encoded in the project's configured runner for **every** platform the behavior's scenario requires — one runner is never a substitute for another, because they guard different platforms of the same behavior. Cite that rule; do not restate its obligations or name a runner as though the contract mandated one.
|
|
17
|
+
- **The codified proof re-runs in CI.** Codified runtime verification lives wherever the project's own configured e2e runner(s) already look for tests — its own config and conventions, never a Lisa-assumed path or tool — and runs as a required CI check for types with verification enforced, so the proof re-runs on every PR, not once by hand. For **frontend work** specifically, where that runner lives and how many runners are required is governed by the `bdd-e2e-coverage` rule, not this bullet.
|
|
18
18
|
- **Commit the evidence.** Write a durable artifact to `evidence/<ticket>/` — the acceptance criteria with per-criterion pass/fail + note, the observed state, screenshots/recording, and a `verdict.json`. The transient `.lisa/verification-status.json` is the session gate; `evidence/<ticket>/` is the committed proof.
|
|
19
19
|
- **Evidence markers are typed artifacts, not assertion labels.** A Validation Journey marker is `[EVIDENCE: <artifact-type>: <name>]` where the type (screenshot, recording, http-transcript, cli-output, log-snippet, db-query-output, perf-trace, test-run-log, deploy-log, state-dump) says HOW the proof is captured and the name says WHAT it proves. `[EVIDENCE: works-gracefully]` is a claim, not evidence — write `[EVIDENCE: screenshot: load-failure-error-state]`. Completion requires a captured artifact **of the declared type** per marker. To cite another work item's artifact without claiming it, write `[EVIDENCE-REF: <work-item-ref> | <artifact-type>: <kebab-case-name>]`. The Lisa 2.223.0 colon form `[EVIDENCE-REF: <tracker-ref>: <artifact-type>: <kebab-case-name>]` remains accepted as a legacy non-claiming alias but must not be newly authored. A reference never enters or satisfies the local S14 manifest; malformed references fail S14, and agents must not quote sibling `[EVIDENCE: ...]` markers directly.
|
|
20
20
|
- **Per-change verification is mandatory.** Every `feat`/`fix` adds or extends a verification (e2e) spec mapped to its acceptance criteria. The only exception is a genuinely non-behavioral change explicitly marked with the **logged** `verification-exempt` label — never a silent skip.
|
|
21
|
+
- **Frontend work also seals its behavior contract.** Per the `bdd-e2e-coverage` rule, a work item that adds or changes a user-observable surface is not done until its behavior exists as Gherkin scenarios with stable IDs, every required scenario-platform obligation is mapped to aligned e2e automation (or carries a dated waiver), and the coverage gate passes with the matrix regenerated. Missing BDD coverage is a verification failure, not a warning.
|
|
21
22
|
- **Drive the playthrough with `/lisa:product-walkthrough`** (it walks the live product through a real browser); each project type plugs in its own drive mechanism (e.g. a Phaser game is driven through Playwright + an in-game verification test bridge that seeds RNG, reads state, injects input, and steps frames).
|
|
22
23
|
- **Every PR must include reviewer replay steps** — the exact human steps to use the software and confirm the change works. Not test commands. If a reviewer can't reproduce from the PR description alone, the PR is incomplete.
|
|
23
24
|
- **Exhaust credential sources before deferring runtime verification** — check project e2e / Playwright config and fixtures first, then `.lisa.config.local.json` / environment variables, then documented ticket credentials such as `Sign-in Required`.
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
# BDD Behavior Contract & E2E Coverage
|
|
2
|
+
|
|
3
|
+
Frontend behavior has historically been specified twice and owned nowhere: once as prose acceptance
|
|
4
|
+
criteria on a ticket that closes and disappears, and once as whatever e2e tests someone had time to
|
|
5
|
+
write. Neither survives the work item. The cost lands at verification time — nobody can answer "what
|
|
6
|
+
is this app supposed to do?" without re-reading a year of closed tickets, and "is that behavior
|
|
7
|
+
guarded?" has a different answer shape per platform and per runner. This contract writes both
|
|
8
|
+
answers down once, in the repo, next to the code.
|
|
9
|
+
|
|
10
|
+
It is a **single vendor-neutral contract** consumed by `lisa-research` (PRDs express frontend
|
|
11
|
+
behavior in a BDD-convertible shape), `lisa-acceptance-criteria` and `lisa-task-decomposition`
|
|
12
|
+
(the contract update and the e2e sealing are explicit obligations on the work item, not implied
|
|
13
|
+
work), `lisa-test-strategy` and `lisa-tdd-implementation` and `lisa-implement` (building a frontend
|
|
14
|
+
item includes both), `lisa-codify-verification` (the verified journey is codified into every runner
|
|
15
|
+
the scenario's platforms require), and `lisa-verification-lifecycle` / `lisa-spec-conformance` /
|
|
16
|
+
`lisa-verify` (the gate is checked, and a miss fails the item). Those skills cite this slug; they do
|
|
17
|
+
not restate it.
|
|
18
|
+
|
|
19
|
+
## Vendor neutrality is the point
|
|
20
|
+
|
|
21
|
+
This contract never names a test runner. Projects use Playwright, Cypress, Maestro, Detox, Appium,
|
|
22
|
+
WebdriverIO, XCUITest, Espresso, an in-house harness, or several at once, and that choice is
|
|
23
|
+
project configuration that changes without touching this contract. What the contract fixes is the
|
|
24
|
+
*obligation*: a required scenario-platform pair is sealed by aligned automation **in whatever runner
|
|
25
|
+
the project has configured for that platform**. Any tool named anywhere downstream is an example,
|
|
26
|
+
never a mandate.
|
|
27
|
+
|
|
28
|
+
The runner→platform mapping is declared once, in the coverage map, and is the only place a tool name
|
|
29
|
+
is authoritative:
|
|
30
|
+
|
|
31
|
+
```json
|
|
32
|
+
"runnerPlatforms": {
|
|
33
|
+
"<web-runner>": ["web"],
|
|
34
|
+
"<device-runner>": ["ios", "android"]
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Platform vocabulary is the project's own — a project that ships only web declares only `web`; one
|
|
39
|
+
that ships a desktop or TV surface declares those. The contract cares that each scenario names the
|
|
40
|
+
platforms it requires and that each named platform has a configured runner.
|
|
41
|
+
|
|
42
|
+
## Coverage map schema
|
|
43
|
+
|
|
44
|
+
`bdd/coverage-map.json` is the machine-readable half of the contract. Minimum shape:
|
|
45
|
+
|
|
46
|
+
```json
|
|
47
|
+
{
|
|
48
|
+
"schemaVersion": 1,
|
|
49
|
+
"asOf": "<ISO date>",
|
|
50
|
+
"runnerPlatforms": { "<runner>": ["<platform>", "..."] },
|
|
51
|
+
"coverageFloor": { "<platform>": "<0-100, the committed floor — may rise, never falls>" },
|
|
52
|
+
"platformWaivers": [
|
|
53
|
+
{
|
|
54
|
+
"scenario": "BDD-DOMAIN-NNN",
|
|
55
|
+
"platforms": ["<platform>"],
|
|
56
|
+
"runner": "<runner — required when more than one runner covers a waived platform, so the waiver names which one cannot decide it>",
|
|
57
|
+
"reason": "why this runner cannot decide this behavior today",
|
|
58
|
+
"recordedAt": "<ISO date>"
|
|
59
|
+
}
|
|
60
|
+
],
|
|
61
|
+
"mappings": [
|
|
62
|
+
{
|
|
63
|
+
"scenario": "BDD-DOMAIN-NNN",
|
|
64
|
+
"runner": "<runner>",
|
|
65
|
+
"platforms": ["<platform>", "..."],
|
|
66
|
+
"file": "<path to the spec/flow>",
|
|
67
|
+
"evidence": "<exact string inside that file — the test title or flow name>",
|
|
68
|
+
"level": "behavioral"
|
|
69
|
+
}
|
|
70
|
+
],
|
|
71
|
+
"exclusions": [
|
|
72
|
+
{ "file": "<path>", "reason": "why this test aligns to no product behavior" }
|
|
73
|
+
]
|
|
74
|
+
}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
`evidence` is what makes a mapping falsifiable: the gate reads the mapped file and confirms the
|
|
78
|
+
string is still there, so renaming or deleting a test breaks the map loudly instead of leaving a
|
|
79
|
+
scenario silently unguarded. `exclusions` records tests that deliberately map to nothing — starter
|
|
80
|
+
templates, source-level corroboration — so "unmapped test" stays a meaningful signal.
|
|
81
|
+
|
|
82
|
+
## The gate
|
|
83
|
+
|
|
84
|
+
Two commands, wired into the project's script surface and into CI:
|
|
85
|
+
|
|
86
|
+
1. **Regenerate** — recompute coverage from features + map, rewrite the generated matrix
|
|
87
|
+
(`docs/bdd-scenario-matrix.md`) and the burndown (`docs/e2e-bdd-coverage.md`), and refresh the
|
|
88
|
+
machine-readable report. Low coverage is reported honestly and does **not** fail this command.
|
|
89
|
+
2. **Check** — the CI gate. It fails on contract defects, not on ambition:
|
|
90
|
+
- a duplicate or malformed scenario ID;
|
|
91
|
+
- a scenario declaring a platform with no configured runner;
|
|
92
|
+
- a mapping whose file is missing or whose `evidence` string is gone (stale mapping);
|
|
93
|
+
- a mapping claiming a platform the scenario does not declare;
|
|
94
|
+
- an invalid waiver (nonexistent scenario, undeclared platform, already-excluded scenario, a
|
|
95
|
+
`runner` that does not cover the waived platform under `runnerPlatforms`, or one that masks an
|
|
96
|
+
existing mapping);
|
|
97
|
+
- a regression against the project's committed `coverageFloor` per platform.
|
|
98
|
+
|
|
99
|
+
The percentage measures **aligned automation inventory, not the latest run result**. A mapped test
|
|
100
|
+
that currently fails is a red CI check, a separate signal; the map only asserts the automation
|
|
101
|
+
exists and still says what it claimed. Both facts are required — a green gate over a red suite is
|
|
102
|
+
not coverage.
|
|
103
|
+
|
|
104
|
+
### Coverage floor, not coverage target
|
|
105
|
+
|
|
106
|
+
Projects adopting this contract mid-life start below 100% and must not be blocked by that. The gate
|
|
107
|
+
enforces a **ratchet**: the committed floor per platform may rise and may never fall. Recorded in
|
|
108
|
+
`coverageFloor`, it is bumped by regenerating the matrix after a frontend work item raises it by the
|
|
109
|
+
obligations it seals; it is never asked to clear the whole backlog.
|
|
110
|
+
|
|
111
|
+
## Waivers versus `@blocked`
|
|
112
|
+
|
|
113
|
+
These are different failures and must never be conflated:
|
|
114
|
+
|
|
115
|
+
| | Means | Recorded as |
|
|
116
|
+
|---|---|---|
|
|
117
|
+
| `@blocked` | The **product** does not do this yet — undesigned, undecided, or unbuilt. | Lifecycle tag on the scenario |
|
|
118
|
+
| Waiver | The product may well do it, but this **runner** cannot decide it on this platform today. | Dated entry in `platformWaivers` |
|
|
119
|
+
|
|
120
|
+
Both leave the denominator, and neither is ever counted as covered: a waiver is a dated IOU, never
|
|
121
|
+
coverage. A waiver names a runner limitation — no camera on
|
|
122
|
+
the simulator, no request interception, a provider sign-in needing a real credential nobody
|
|
123
|
+
provisioned, a backend with no seedable row for the required state — and it carries a date so it can
|
|
124
|
+
be audited for age. When the limitation dies, the waiver is deleted and the automation is written;
|
|
125
|
+
a waiver that outlives its reason is a defect the burndown should surface.
|
|
126
|
+
|
|
127
|
+
## Scenario identity and change
|
|
128
|
+
|
|
129
|
+
A stable ID is a promise about *behavior*, not about text:
|
|
130
|
+
|
|
131
|
+
- The user-observable behavior is unchanged → **keep the ID**, however much the wording, the
|
|
132
|
+
implementation, or the design node changed.
|
|
133
|
+
- The behavior is genuinely different → **new ID**; mark the old one `@superseded` and leave it in
|
|
134
|
+
the file. Superseded scenarios are the audit trail for why the product changed.
|
|
135
|
+
- The capability is committed but not yet designed or built → write it now and tag it `@blocked`.
|
|
136
|
+
Omitting a committed capability makes coverage look better by describing less of the product,
|
|
137
|
+
which is exactly the failure this contract exists to prevent.
|
|
138
|
+
- IDs are never renumbered or reused. Gaps in the sequence are normal and harmless.
|
|
139
|
+
|
|
140
|
+
Provenance tags tie a scenario back to its authority (design node, ratified decision, wiki
|
|
141
|
+
requirement) and to the tracker item that introduced or changed it, so `spec-conformance` can walk
|
|
142
|
+
from a requirement to a scenario to a test without guessing.
|
|
143
|
+
|
|
144
|
+
## Per-flow obligations
|
|
145
|
+
|
|
146
|
+
| Flow | Obligation |
|
|
147
|
+
|---|---|
|
|
148
|
+
| **Research / PRD** | For frontend scope, user-facing behavior is written as — or in a shape directly convertible to — Given/When/Then scenarios, with the platforms each behavior must hold on. A PRD that describes frontend behavior only as narrative forces the shape to be invented later, inconsistently. |
|
|
149
|
+
| **Plan / decomposition / acceptance criteria** | Each frontend work item's acceptance criteria and Validation Journey name the contract update *and* the e2e sealing as explicit deliverables, with the scenario IDs it will add or change once known. They are line items, not implied work. |
|
|
150
|
+
| **Implement / TDD / test strategy** | Building the item updates `bdd/features` and the coverage map and writes the aligned automation, in the same PR as the behavior. The scenario is the RED test's specification for user-facing behavior. |
|
|
151
|
+
| **Codify verification** | The verified journey is codified into **every runner the scenario's platforms require** — one runner is never a substitute for another, because they guard different platforms of the same behavior. |
|
|
152
|
+
| **Verify / verification-lifecycle / spec-conformance** | Verification confirms scenarios exist for the shipped behavior and the coverage gate passes. A shipped frontend behavior with no scenario, or a required obligation with neither mapping nor waiver, is a verification **failure** — not a warning, not a nit. |
|
|
153
|
+
|
|
154
|
+
## Bootstrap
|
|
155
|
+
|
|
156
|
+
A repo with no contract yet, taking its first frontend work item:
|
|
157
|
+
|
|
158
|
+
1. **Detect runners.** Find the e2e harnesses the project actually has, per the Tool Discovery
|
|
159
|
+
Process in `verification-lifecycle`. Record which platforms each covers.
|
|
160
|
+
2. **Scaffold the minimum** — `bdd/features/`, `bdd/coverage-map.json` (with `runnerPlatforms` from
|
|
161
|
+
step 1 and a coverage floor of the current, honest number), the regenerate + check scripts wired
|
|
162
|
+
into the project's script surface, and the CI invocation of the check.
|
|
163
|
+
3. **Write only this item's scenarios.** The first item is not a backfill project. Pre-existing
|
|
164
|
+
uncovered behavior becomes burndown in `docs/e2e-bdd-coverage.md`, and the floor starts where the
|
|
165
|
+
repo actually is.
|
|
166
|
+
4. **Seal this item's obligations** and commit the regenerated matrix and burndown with the change.
|
|
167
|
+
|
|
168
|
+
If a required platform has **no** e2e runner at all, that obligation is never left as a bare `N/A` —
|
|
169
|
+
record it exactly like any other unsealable obligation: a dated `platformWaivers` entry naming the
|
|
170
|
+
locations checked and "no runner configured" as the reason, so the absence is dated, audited for age,
|
|
171
|
+
and visible in the same place every other IOU lives — the same recorded-absence exit the
|
|
172
|
+
regression-spec rule in `verification` allows, made falsifiable. Where the runner could reasonably be
|
|
173
|
+
added, additionally file a linked build-ready follow-up ticket before merge and reference it from the
|
|
174
|
+
waiver's reason. Silence is never an exit.
|
|
175
|
+
|
|
176
|
+
## Degradation
|
|
177
|
+
|
|
178
|
+
Missing scaffolding degrades the run, it does not crash it: a project with no contract gets the
|
|
179
|
+
bootstrap path, an unreadable coverage map is reported as a blocker on the work item rather than
|
|
180
|
+
swallowed, and an absent optional runner is recorded as an absence with the locations checked. What
|
|
181
|
+
never degrades is the reporting — an agent that could not evaluate the contract says so explicitly
|
|
182
|
+
and does not report the item as done.
|
|
@@ -64,7 +64,7 @@ Gate:
|
|
|
64
64
|
|
|
65
65
|
Sequence:
|
|
66
66
|
1. **Investigate sub-flow** -- gather context from codebase, git history, existing behavior, and external sources
|
|
67
|
-
2. `product-specialist` -- define user goals, user flows (Gherkin), acceptance criteria, error states, UX concerns, and out-of-scope items
|
|
67
|
+
2. `product-specialist` -- define user goals, user flows (Gherkin), acceptance criteria, error states, UX concerns, and out-of-scope items. For frontend scope, the Gherkin flows must satisfy the `bdd-e2e-coverage` rule's scenario shape (behavior stated as Given/When/Then, with the platforms each behavior must hold on) so the PRD converts to behavior-contract scenarios without re-specification
|
|
68
68
|
3. **Edge Case Brainstorm sub-flow** -- run the PRD candidate through the edge-case checklist; fold accepted cases into acceptance criteria, out-of-scope, or open questions
|
|
69
69
|
4. `architecture-specialist` -- assess technical feasibility, identify constraints, map existing system boundaries
|
|
70
70
|
5. Synthesize findings into a PRD structured as: (1) problem statement, (2) high-level solution description, (3) links to design files/docs if needed, (4) user stories -- each carrying its own functional requirements, non-functional requirements, and a pointer to a design file (only when that story introduces new UI/visual work; omit the pointer otherwise rather than leaving it as a blank required field), (5) overall acceptance criteria, and (6) open questions/decisions. Nest requirements under each story rather than flattening them into global lists -- this keeps the context an agent needs to implement or ticket one story colocated, instead of requiring it to infer which global requirement applies to which story. Any technically viable but genuinely unresolved choice discovered during drafting (for example, a library, framework, or architecture decision with more than one live candidate) MUST be captured as an entry under open questions -- never written into any other section, including the "Recommended Tooling for Plan Phase" section in step 6, as though it were already decided. Every open-questions entry MUST include the drafter's own recommended resolution, with a one-sentence rationale, alongside the question; an open question must never be left bare.
|
|
@@ -93,8 +93,8 @@ Sequence:
|
|
|
93
93
|
5. **Implement/Verify Phase Tooling** -- review all available skills and agents (project-defined, plugin-provided, and built-in) and determine which ones the Implement and Verify phases will need for each work item. For each recommended skill or agent, state why it is needed and which work items it applies to. If no skills or agents beyond the defaults are identified for a work item, explicitly justify why the standard set is sufficient.
|
|
94
94
|
6. Decompose into ordered work items (epics, stories, tasks, spikes, bugs). For each item, run the **Edge Case Brainstorm sub-flow** scoped to that item — accepted cases become additional acceptance criteria or sub-tasks; rejected ones are noted with a one-line reason. Each item carries:
|
|
95
95
|
- Type (epic, story, task, spike, bug)
|
|
96
|
-
- Acceptance criteria (including any added by the per-item brainstorm)
|
|
97
|
-
- Verification method
|
|
96
|
+
- Acceptance criteria (including any added by the per-item brainstorm). For a frontend item, these MUST name the behavior-contract update and the aligned e2e automation as explicit deliverables per the `bdd-e2e-coverage` rule -- never left implied
|
|
97
|
+
- Verification method -- becomes the item's Validation Journey. For a frontend item this MUST name, symmetrically with the acceptance criteria above: the scenario IDs the item will add or change (once known), the platforms each requires, the contract update, and the e2e-sealing evidence markers per the `bdd-e2e-coverage` rule -- a generic "manual QA" or "run the e2e suite" description does not satisfy it
|
|
98
98
|
- Dependencies
|
|
99
99
|
- Skills and agents required (from step 5)
|
|
100
100
|
7. Create work items in the tracker (JIRA, Linear, GitHub) with acceptance criteria, dependencies, and recommended skills/agents
|
|
@@ -126,7 +126,7 @@ Determine the work type and execute the matching variant:
|
|
|
126
126
|
5. `builder` -- implement via TDD (acceptance criteria become tests)
|
|
127
127
|
6. Run quality gates: lint, typecheck, tests (these are prerequisites, NOT verification)
|
|
128
128
|
7. `verification-specialist` -- verify locally (run the software, observe behavior)
|
|
129
|
-
8. `verification-specialist` -- invoke `codify-verification` skill per passing verification (
|
|
129
|
+
8. `verification-specialist` -- invoke `codify-verification` skill per passing verification (integration test for API/DB/auth, benchmark for performance, the project's e2e runner for UI); commit each test in the same PR. For frontend work this also means the `bdd-e2e-coverage` obligations: the Gherkin scenario added/updated with its stable ID, aligned automation in the project's configured runner for every platform that scenario requires, and the coverage gate re-run with the matrix regenerated
|
|
130
130
|
9. **Record Implement usage on the work artifact** -- invoke `lisa-usage-accounting` against the originating work item or implementation artifact so it gains a direct `implement` usage entry in the canonical `## Lisa Usage` section. If the hierarchy / parent refs are already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise record the direct entry and leave rollup for the next caller that has the child refs. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of omitting the row.
|
|
131
131
|
10. **Review sub-flow**
|
|
132
132
|
11. `learner` -- capture discoveries
|
|
@@ -141,7 +141,7 @@ Determine the work type and execute the matching variant:
|
|
|
141
141
|
6. `bug-fixer` -- implement fix via TDD (reproduction becomes failing test)
|
|
142
142
|
7. Run quality gates: lint, typecheck, tests (these are prerequisites, NOT verification)
|
|
143
143
|
8. `verification-specialist` -- verify locally (prove the bug is fixed)
|
|
144
|
-
9. `verification-specialist` -- invoke `codify-verification` skill to encode the fix as a regression test (mandatory for bug fixes — the test must fail against the pre-fix commit and pass against the fix); commit in the same PR
|
|
144
|
+
9. `verification-specialist` -- invoke `codify-verification` skill to encode the fix as a regression test (mandatory for bug fixes — the test must fail against the pre-fix commit and pass against the fix); commit in the same PR. For a user-visible frontend fix, the `bdd-e2e-coverage` obligations apply as in the Build flow: the scenario the fix restores or changes, aligned automation per required platform, and a passing coverage gate
|
|
145
145
|
10. **Record Implement usage on the work artifact** -- invoke `lisa-usage-accounting` against the originating work item or implementation artifact so it gains a direct `implement` usage entry in the canonical `## Lisa Usage` section. If the hierarchy / parent refs are already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise record the direct entry and leave rollup for the next caller that has the child refs. If runtime usage is unavailable, still write `source: unavailable` with nullable token/cost fields instead of omitting the row.
|
|
146
146
|
11. **Review sub-flow**
|
|
147
147
|
12. `learner` -- capture discoveries
|
|
@@ -168,7 +168,7 @@ Determine the work type and execute the matching variant:
|
|
|
168
168
|
|
|
169
169
|
In every work type above, before a task completes -- immediately ahead of the closing `learner` step -- the implementing agent records concise kind-tagged MLD (Mistakes / Learnings / Desires) into that task's `metadata.learnings`: one line per item, empty is valid, never re-prompted or scored. See the `lisa-implement` skill for the full `{ kind, note, evidence? }` schema and routing (change the schema there, not here).
|
|
170
170
|
|
|
171
|
-
Output: Code passing all quality gates + local empirical verification + codified regression test for each verification (except for spikes, which produce findings only, and non-behavioral verification types — PR / Documentation / Deploy — which carry their own proof).
|
|
171
|
+
Output: Code passing all quality gates + local empirical verification + codified regression test for each verification (except for spikes, which produce findings only, and non-behavioral verification types — PR / Documentation / Deploy — which carry their own proof). For frontend work the output additionally includes the updated behavior contract, coverage-map mappings for every required scenario-platform obligation, and a regenerated matrix with a passing coverage gate (`bdd-e2e-coverage`).
|
|
172
172
|
|
|
173
173
|
### Verify
|
|
174
174
|
|
|
@@ -178,6 +178,7 @@ Gate:
|
|
|
178
178
|
- Code must pass quality gates (lint, typecheck, tests)
|
|
179
179
|
- Local empirical verification must be complete
|
|
180
180
|
- Each passing local verification must be codified as a regression test (or carry a documented skip from the allowed set: PR / Documentation / Deploy / Investigate-Only). If verifications are not codified, return to the Implement flow's codify step before shipping
|
|
181
|
+
- For frontend work, the `bdd-e2e-coverage` contract must be satisfied: scenarios exist for the shipped behavior, every required scenario-platform obligation is mapped or dated-waived, and the coverage gate passes with the matrix regenerated. A miss returns to Implement — it is a verification failure, not a warning
|
|
181
182
|
- If quality gates fail, go back to **Implement**
|
|
182
183
|
- If no code changes exist, there is nothing to verify
|
|
183
184
|
|
|
@@ -26,7 +26,7 @@ For UI work, the agent must control a live browser and use the product the way a
|
|
|
26
26
|
|
|
27
27
|
Kane is a guarded provider, not a global browser override. It is eligible only when the project explicitly enables it, acknowledges TestMu cloud upload, pins Lisa's contract-tested version, selects an allow-listed non-production environment, and resolves mutation policy `full`; `lisa kane probe` must prove installation, authentication, Test Manager targeting, and usable credits before a factory starts. Invoke it only through `lisa-kane-browser`. Treat auth, Chrome, upload, schema, and control-plane failures as tooling failures. Persist the local evidence pack through Lisa's evidence pipeline because a Test Manager share link is secondary and expiring.
|
|
28
28
|
|
|
29
|
-
Running an automated Playwright or Maestro test is still a quality gate, **not the initial empirical verification evidence**. The distinction is how the browser is used, not the library name: interactive Playwright control or a guarded Kane objective that performs and observes the Validation Journey is valid empirical verification; invoking a prewritten test and reporting its green result alone is not. After the live journey passes, codify that observed behavior in the project's
|
|
29
|
+
Running an automated Playwright or Maestro test is still a quality gate, **not the initial empirical verification evidence**. The distinction is how the browser is used, not the library name: interactive Playwright control or a guarded Kane objective that performs and observes the Validation Journey is valid empirical verification; invoking a prewritten test and reporting its green result alone is not. After the live journey passes, codify that observed behavior in the project's configured runner(s) for the platforms it covers, as required below, so CI can prevent regressions. Kane `_test.md`, code export, and auto-healed recordings are not authoritative substitutes during the initial rollout.
|
|
30
30
|
|
|
31
31
|
Verification is mandatory. Never skip it, defer it, or claim it was unnecessary. Every task must be verified before claiming completion.
|
|
32
32
|
|
|
@@ -234,12 +234,13 @@ one-off claim. It builds on the **Per-Work-Unit Evidence Contract** above (the
|
|
|
234
234
|
typed `[EVIDENCE: <artifact-type>: <name>]` manifest), adding three concrete requirements:
|
|
235
235
|
|
|
236
236
|
**1. The codified proof re-runs in CI.** After local verification passes,
|
|
237
|
-
`codify-verification` encodes it
|
|
238
|
-
|
|
239
|
-
project
|
|
240
|
-
re-runs on every PR instead of being
|
|
237
|
+
`codify-verification` encodes it wherever the project's own configured e2e runner(s) already look
|
|
238
|
+
for tests — its own directory conventions and config, never a Lisa-assumed path or tool. That suite
|
|
239
|
+
runs in CI (the project's existing e2e job, whatever runner backs it); for a project type with
|
|
240
|
+
verification **enforced**, it is a required check — so the proof re-runs on every PR instead of being
|
|
241
|
+
proven once by hand.
|
|
241
242
|
|
|
242
|
-
For **frontend work**, codification is
|
|
243
|
+
For **frontend work**, codification is multi-runner and governed by the `bdd-e2e-coverage` rule: the same verified journey is encoded in the project's configured runner for every platform the behavior's scenario requires, and no runner substitutes for another because they guard different platforms. The requirement is non-demotable: a missing runner is either a recorded absence (the project genuinely has no such harness for that platform) or a linked build-ready follow-up ticket — never a silent skip. That rule also owns the behavior contract the codified tests are sealing (Gherkin scenarios with stable IDs, the coverage map, waivers, and the gate); cite it rather than restating it here.
|
|
243
244
|
|
|
244
245
|
**2. Evidence is committed to the repo.** The named artifacts from the work
|
|
245
246
|
unit's evidence manifest are committed under `evidence/<ticket>/` (in addition to
|
|
@@ -64,6 +64,7 @@ For each acceptance criterion:
|
|
|
64
64
|
- Write acceptance criteria from the user's perspective, not the developer's
|
|
65
65
|
- Every user flow must include at least one error path
|
|
66
66
|
- Use Gherkin format (Given/When/Then) for user flows to enable direct translation into test cases
|
|
67
|
+
- For frontend work, the `bdd-e2e-coverage` rule applies: name the scenario IDs the item will add or change in the project's behavior contract and the platforms each must hold on, and list the contract update plus the aligned e2e automation as explicit acceptance criteria -- they are deliverables, not implied work. List the passing coverage gate and the regenerated matrix and burndown as explicit acceptance criteria too -- the item is not done on a code-complete mapping alone
|
|
67
68
|
- When verifying, always run the feature -- never review by only reading code
|
|
68
69
|
- If you cannot run the feature (missing dependencies, services unavailable), report as a blocker -- do not guess
|
|
69
70
|
- If the changes are purely internal (refactoring, config, tooling), report "No user-facing impact" and explain why
|
|
@@ -57,9 +57,7 @@ If the empirical proof came from Kane, consume its exact objective, observable a
|
|
|
57
57
|
|
|
58
58
|
| Verification type | Preferred framework (use whichever the project already has) |
|
|
59
59
|
|---|---|
|
|
60
|
-
| UI (web) | Playwright
|
|
61
|
-
| UI (mobile) | Maestro > Detox > Playwright (mobile emulation) |
|
|
62
|
-
| UI (frontend, project supports multiple runners) | **ALL supported UI runners** — see "Frontend dual-runner codification" below |
|
|
60
|
+
| UI (web, mobile, or any frontend surface) | **The project's configured runner for every platform the behavior requires** — see "Frontend multi-runner codification" below. This row is authoritative for any UI work covered by a `bdd-e2e-coverage` scenario (in practice, essentially all user-facing UI work); it supersedes any generic runner preference — a project's web runner might be Playwright, Cypress, or Selenium, and its device runner might be Maestro, Detox, or a Playwright mobile-emulation profile, but the choice is read from `runnerPlatforms`, never assumed |
|
|
63
61
|
| API | project's integration test runner (Vitest / Jest / RSpec / pytest) with HTTP client (supertest / fetch / faraday) |
|
|
64
62
|
| Database | integration test with real DB + migrations applied |
|
|
65
63
|
| Auth | API or UI test asserting role-gated access (multi-role coverage) |
|
|
@@ -74,19 +72,24 @@ If the empirical proof came from Kane, consume its exact objective, observable a
|
|
|
74
72
|
|
|
75
73
|
If the project lacks the preferred framework AND no acceptable substitute exists, escalate.
|
|
76
74
|
|
|
77
|
-
### 2a. Frontend
|
|
75
|
+
### 2a. Frontend multi-runner codification (non-demotable)
|
|
78
76
|
|
|
79
|
-
For **frontend work** — any verification whose validation journey exercised a user-facing UI surface — codification is not one-runner-or-the-other.
|
|
77
|
+
For **frontend work** — any verification whose validation journey exercised a user-facing UI surface — codification is not one-runner-or-the-other. The `bdd-e2e-coverage` rule is the contract; this section is only how codification satisfies it. After the validation journey is complete and verified:
|
|
80
78
|
|
|
81
|
-
1. **
|
|
82
|
-
2. **
|
|
79
|
+
1. **Locate the behavior's scenario** in the project's behavior contract (`bdd/features/**` by default) — its stable `@BDD-*` ID and the platforms it declares. If the verified behavior has no scenario yet, write it now; that is part of codification, not a separate task. If the project has no contract yet, take the rule's bootstrap path, scoped to this behavior only.
|
|
80
|
+
2. **Codify into the project's configured runner for EVERY platform that scenario requires.** The runner→platform mapping is project configuration, declared in `bdd/coverage-map.json` under `runnerPlatforms` — read it rather than assuming a tool. Wire each new spec/flow where its runner already picks work up, following the project's existing directory and tagging conventions.
|
|
81
|
+
3. **Record the mapping.** Add one `mappings` entry per scenario-platform obligation naming the runner, platforms, file, and an `evidence` string that actually appears in that file, then regenerate the matrix and burndown so the gate reflects the new coverage.
|
|
82
|
+
4. **Run the coverage gate.** Invoke the project's configured `bdd-e2e-coverage` check command (the same one wired into CI) and confirm it passes. Regenerating the matrix and burndown only recomputes the report; it does not itself prove the gate is green. Record the command and its result in the codification evidence — a regenerated matrix with no observed gate run is not proof of coverage.
|
|
83
83
|
|
|
84
|
-
|
|
84
|
+
Every artifact encodes the SAME verified journey against a different platform. One is never a substitute for another, and a passing test on one platform never seals another platform's obligation.
|
|
85
85
|
|
|
86
|
-
Permitted exits, mirroring the regression-spec rule in `lisa-implement` (never a silent skip, never "optional"):
|
|
86
|
+
Permitted exits, mirroring the regression-spec rule in `lisa-implement` (never a silent skip, never "optional", and never a bare `N/A`):
|
|
87
87
|
|
|
88
|
-
- The project genuinely has no runner
|
|
89
|
-
-
|
|
88
|
+
- The project genuinely has no runner configured for that platform → record a dated `platformWaivers` entry naming the locations checked and "no runner configured" as the reason, exactly like any other unsealable obligation, per the rule. This is never left as a bare `N/A` — an undated absence has no forcing function to ever get revisited.
|
|
89
|
+
- The runner exists but genuinely cannot decide this behavior on that platform (no camera on the simulator, no request interception, an unprovisioned provider credential) → record a dated `platformWaivers` entry with the reason, per the rule. A waiver is an IOU, never coverage.
|
|
90
|
+
- A runner is configured and capable but the spec cannot be added or executed in this PR (genuine technical blocker) → create a linked build-ready follow-up ticket before merge, reference it from the PR and work item, and record the blocker — the same follow-up path as the regression-spec blocker.
|
|
91
|
+
|
|
92
|
+
Either of the first two exits also gets a linked build-ready follow-up ticket, referenced from the waiver's reason, whenever the runner could reasonably be added or the limitation could reasonably be lifted — the waiver records the IOU, the ticket is what pays it down.
|
|
90
93
|
|
|
91
94
|
### 3. Generate the test
|
|
92
95
|
|
|
@@ -107,7 +110,8 @@ For Playwright UI tests specifically:
|
|
|
107
110
|
|
|
108
111
|
**Concrete verification (UAT) contract.** Verification *is* UAT — codifying it is
|
|
109
112
|
how the playthrough becomes durable. For a runtime/behavioral `feat`/`fix`: place
|
|
110
|
-
the codified test
|
|
113
|
+
the codified test wherever the project's own configured e2e runner(s) already look
|
|
114
|
+
for tests — its own directory conventions, never a Lisa-assumed path or tool —
|
|
111
115
|
so CI re-runs it, and commit the evidence artifact to `evidence/<ticket>/`
|
|
112
116
|
(`verdict.json` + state + screenshots). For a Phaser game, drive the canvas
|
|
113
117
|
through the in-game verification test bridge (seed RNG, read state, inject input,
|
|
@@ -147,7 +147,7 @@ IF it is a Fix (bug), execute the Reproduce sub-flow FIRST:
|
|
|
147
147
|
1. Write a simple API client and call the offending API
|
|
148
148
|
2. Start the server on localhost and use the Playwright CLI or Chrome DevTools
|
|
149
149
|
|
|
150
|
-
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it. For frontend work
|
|
150
|
+
For any Fix flow, and for any Build flow that changes user-visible behavior, regression coverage is a required deliverable at the highest practical observation level for the reported surface. If the project has a browser, device, or end-to-end harness for that platform (for example Playwright, Maestro, Detox, Cypress, or an equivalent runtime), the task plan and definition of done MUST include a deterministic regression spec against the reported surface, using mocked or seeded data where needed. This is alongside unit or integration coverage, not a substitute for it. For **frontend work** the deliverable is defined by the `bdd-e2e-coverage` rule and has two halves, both landing in this PR: the Gherkin scenario (stable ID, required platforms) added or updated in the project's behavior contract, and aligned automation in the project's configured runner for **every** platform that scenario requires — no runner substitutes for another, because they guard different platforms of the same behavior. The coverage gate must pass and the matrix and burndown be regenerated before the item is done. Cite the rule; do not restate its scenario, waiver, or bootstrap mechanics here.
|
|
151
151
|
|
|
152
152
|
The team lead may not waive, defer, demote, or phrase this regression spec as "optional", "if cheap", "nice to have", or equivalent. The only permitted exits are:
|
|
153
153
|
|
|
@@ -326,7 +326,7 @@ Before shutting down the team, execute the Verify flow:
|
|
|
326
326
|
boundary each claim reached, or name the violation. Where the runtime lacks the rules surface (the
|
|
327
327
|
agy artifacts carry no rules tree), the obligation still travels in this skill — cite the
|
|
328
328
|
`claim-evidence-mapping` contract by slug and continue; never block on the absent surface.
|
|
329
|
-
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified,
|
|
329
|
+
3. Write the highest-practical-observation regression test encoding the verification. For user-visible bugs or user-visible Build changes with an available browser/device/e2e harness, this means a deterministic spec on the reported surface — and for frontend work, once the validation journey is verified, the scenario and its aligned automation for **every platform the scenario requires**, per `codify-verification` and the `bdd-e2e-coverage` rule. Prove the new spec actually executed and passed in PR CI by recording a named spec log/reporter line or equivalent execution record; green CI without that named evidence does not satisfy this step.
|
|
330
330
|
4. Record Implement usage on the originating work artifact via `lisa-usage-accounting` so the work item (or other implementation-owned artifact) gains a direct `lisa-implement` usage entry in the canonical `## Lisa Usage` section. If the parent / child graph is already known, prefer `record_and_rollup` so ancestor totals refresh in the same write; otherwise still write the direct entry, and if runtime usage is unavailable, use `source: unavailable` with nullable token/cost fields instead of skipping the row.
|
|
331
331
|
5. Commit ALL outstanding changes in logical batches on the branch (minus sensitive data/information) — not just changes made by the agent team. This includes pre-existing uncommitted changes that were on the branch before the plan started. Do NOT filter commits to only "task-related" files. If it shows up in git status, it gets committed (unless it contains secrets).
|
|
332
332
|
6. Push the changes - if any pre-push hook blocks you, create a task for the agent team to fix the error/problem whether it was pre-existing or not
|