@codyswann/lisa 2.232.0 → 2.233.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/migrations/ensure-lisa-postinstall.d.ts.map +1 -1
- package/dist/migrations/ensure-lisa-postinstall.js +9 -17
- package/dist/migrations/ensure-lisa-postinstall.js.map +1 -1
- package/dist/strategies/copy-contents.d.ts.map +1 -1
- package/dist/strategies/copy-contents.js +7 -0
- package/dist/strategies/copy-contents.js.map +1 -1
- package/dist/strategies/copy-overwrite.d.ts +0 -1
- package/dist/strategies/copy-overwrite.d.ts.map +1 -1
- package/dist/strategies/copy-overwrite.js +1 -7
- package/dist/strategies/copy-overwrite.js.map +1 -1
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-checklist/agents/openai.yaml +4 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-clear/agents/openai.yaml +4 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-fail/agents/openai.yaml +4 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-qa-queue/agents/openai.yaml +4 -0
- package/plugins/lisa/.codex-plugin/skills/lisa-rework-triage/SKILL.md +3 -3
- package/plugins/lisa/commands/qa-checklist.md +6 -0
- package/plugins/lisa/commands/qa-clear.md +6 -0
- package/plugins/lisa/commands/qa-fail.md +6 -0
- package/plugins/lisa/commands/qa-queue.md +6 -0
- package/plugins/lisa/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/lisa/skills/lisa-qa-checklist/agents/openai.yaml +4 -0
- package/plugins/lisa/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/lisa/skills/lisa-qa-clear/agents/openai.yaml +4 -0
- package/plugins/lisa/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/lisa/skills/lisa-qa-fail/agents/openai.yaml +4 -0
- package/plugins/lisa/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/lisa/skills/lisa-qa-queue/agents/openai.yaml +4 -0
- package/plugins/lisa/skills/lisa-rework-triage/SKILL.md +3 -3
- package/plugins/lisa-agy/commands/lisa/qa-checklist.md +6 -0
- package/plugins/lisa-agy/commands/lisa/qa-clear.md +6 -0
- package/plugins/lisa-agy/commands/lisa/qa-fail.md +6 -0
- package/plugins/lisa-agy/commands/lisa/qa-queue.md +6 -0
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/lisa-agy/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/lisa-agy/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/lisa-agy/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/lisa-agy/skills/lisa-rework-triage/SKILL.md +3 -3
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/commands/lisa/qa-checklist.md +6 -0
- package/plugins/lisa-copilot/commands/lisa/qa-clear.md +6 -0
- package/plugins/lisa-copilot/commands/lisa/qa-fail.md +6 -0
- package/plugins/lisa-copilot/commands/lisa/qa-queue.md +6 -0
- package/plugins/lisa-copilot/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/lisa-copilot/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/lisa-copilot/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/lisa-copilot/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/lisa-copilot/skills/lisa-rework-triage/SKILL.md +3 -3
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/commands/lisa/qa-checklist.md +6 -0
- package/plugins/lisa-cursor/commands/lisa/qa-clear.md +6 -0
- package/plugins/lisa-cursor/commands/lisa/qa-fail.md +6 -0
- package/plugins/lisa-cursor/commands/lisa/qa-queue.md +6 -0
- package/plugins/lisa-cursor/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/lisa-cursor/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/lisa-cursor/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/lisa-cursor/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/lisa-cursor/skills/lisa-rework-triage/SKILL.md +3 -3
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/commands/qa-checklist.md +6 -0
- package/plugins/src/base/commands/qa-clear.md +6 -0
- package/plugins/src/base/commands/qa-fail.md +6 -0
- package/plugins/src/base/commands/qa-queue.md +6 -0
- package/plugins/src/base/skills/lisa-qa-checklist/SKILL.md +70 -0
- package/plugins/src/base/skills/lisa-qa-clear/SKILL.md +74 -0
- package/plugins/src/base/skills/lisa-qa-fail/SKILL.md +92 -0
- package/plugins/src/base/skills/lisa-qa-queue/SKILL.md +69 -0
- package/plugins/src/base/skills/lisa-rework-triage/SKILL.md +3 -3
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-checklist
|
|
3
|
+
description: "Serve the current manual regression checklist to a human QA tester. Reads the project's curated journey list, cross-references it against what the automated suites (Playwright web E2E, Maestro native E2E) actually cover by scanning the spec files, and serves only the journeys that still need human eyes — each as plain-language steps. Keeps a single source of truth so testers never work from a stale personal copy, and shrinks automatically as automated coverage grows."
|
|
4
|
+
allowed-tools: ["Bash", "Read", "Glob", "Grep", "Write", "Edit"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Checklist: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
The manual regression sweep exists to catch what automation does not. Its checklist must
|
|
10
|
+
therefore be computed, not remembered: curated journeys minus automated coverage, at the
|
|
11
|
+
moment the tester asks.
|
|
12
|
+
|
|
13
|
+
## Sources
|
|
14
|
+
|
|
15
|
+
1. **Curated journey list** — `qa.checklistFile` in `.lisa.config.json`, default
|
|
16
|
+
`.lisa/qa-checklist.md`. Format: one `## Journey: <name>` section per user journey,
|
|
17
|
+
with plain-language steps and an optional `automation:` line naming the covering spec
|
|
18
|
+
file(s) once one exists. If the file does not exist, offer to bootstrap it: derive
|
|
19
|
+
candidate journeys from the app's route map and the existing E2E suites' describe
|
|
20
|
+
blocks, write the draft, and ask the operator to curate it once. Never invent
|
|
21
|
+
journeys silently.
|
|
22
|
+
2. **Automated coverage** — scan the repo's E2E suites (Playwright specs, Maestro flows;
|
|
23
|
+
locate via the project's e2e/test directories). An `automation:` line must name BOTH
|
|
24
|
+
the spec file and the specific test within it (the Playwright `describe`/`test` title
|
|
25
|
+
or Maestro flow name): `automation: <spec-path> :: <test-or-flow-name>`. A journey
|
|
26
|
+
counts as covered only when that spec file exists, the named test/flow is present in
|
|
27
|
+
it, and it is not skipped (`test.skip`, commented-out flow, or excluded from CI). A
|
|
28
|
+
file-only line, a named test that no longer matches, or any ambiguous mapping is
|
|
29
|
+
treated as **uncovered** — a live filename proves nothing about what the spec
|
|
30
|
+
exercises. A deleted, renamed, or skipped test silently un-covers its journey — the
|
|
31
|
+
very regression this check exists to catch; call it out loudly.
|
|
32
|
+
|
|
33
|
+
## Serving the sweep
|
|
34
|
+
|
|
35
|
+
Present two lists, human-first:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
## Manual sweep — <n> journeys need your eyes
|
|
39
|
+
1. <Journey name> — <plain-language steps>
|
|
40
|
+
Why manual: <no automation | automation skipped/stale: <spec>>
|
|
41
|
+
...
|
|
42
|
+
|
|
43
|
+
## Covered by automation — <n> journeys (skim, don't re-test)
|
|
44
|
+
- <Journey name> — <spec file> (<playwright|maestro>)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Rules for the served text: steps an intern can follow, no spec-file jargon in the manual
|
|
48
|
+
section beyond the "why manual" line, and stable journey ordering (file order) so testers
|
|
49
|
+
can resume mid-sweep.
|
|
50
|
+
|
|
51
|
+
## Maintaining the list
|
|
52
|
+
|
|
53
|
+
- Tester or operator adds/edits journeys in plain language → edit the checklist file
|
|
54
|
+
(this skill may apply the edit on request; it is a repo file, so changes ride normal
|
|
55
|
+
review).
|
|
56
|
+
- When a journey gains automation (e.g. `lisa-codify-verification` lands a spec), add its
|
|
57
|
+
`automation: <spec-path> :: <test-or-flow-name>` line — on request this skill locates
|
|
58
|
+
the covering spec and test, confirms the named test actually drives the journey's
|
|
59
|
+
steps, and writes the line itself.
|
|
60
|
+
- Never maintain per-tester copies; the file is the single source of truth and the
|
|
61
|
+
computed view is always derived fresh.
|
|
62
|
+
|
|
63
|
+
## Rules
|
|
64
|
+
|
|
65
|
+
- Coverage claims must point at an existing, non-skipped spec — "there's a test somewhere"
|
|
66
|
+
does not count.
|
|
67
|
+
- Serving is read-only by default; file edits happen only on explicit request.
|
|
68
|
+
- If the automated-coverage scan finds specs exercising a journey that is missing from
|
|
69
|
+
the curated list entirely, surface it as a suggested addition — the human curates, the
|
|
70
|
+
skill proposes.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-clear
|
|
3
|
+
description: "Bulk-clear tickets a human QA tester cannot verify. Finds tickets in the configured QA queue status that are scoped entirely to non-user-facing repos (API/backend services, infrastructure) — where QA has only end-user access and nothing observable to test — and transitions them to the configured certified status, reporting the moved list for the record. Runnable from any coding agent as a single instruction like 'move all backend- and infrastructure-only tickets from the QA queue to certified'."
|
|
4
|
+
allowed-tools: ["Skill", "Bash", "Read", "Glob", "Grep"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Clear: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
Human QA can only judge what a user can see. Tickets whose entire scope is a
|
|
10
|
+
non-user-facing repo were already verified by the automated lifecycle before reaching the
|
|
11
|
+
QA queue; holding them for a human who cannot observe them is pure queue noise. Clear
|
|
12
|
+
them in one auditable batch.
|
|
13
|
+
|
|
14
|
+
## Config resolution
|
|
15
|
+
|
|
16
|
+
Read `.lisa.config.json`:
|
|
17
|
+
|
|
18
|
+
- **QA queue status** — `jira.workflow.qa.queue`, falling back to
|
|
19
|
+
`jira.workflow.done.staging`.
|
|
20
|
+
- **Certified status** — `jira.workflow.qa.certified`. Required; if missing, stop and
|
|
21
|
+
instruct the operator — never guess a terminal status.
|
|
22
|
+
- **Tracker dispatch** — `tracker` decides the surface as everywhere; on GitHub or
|
|
23
|
+
Linear the status names above map to the equivalent labels/states.
|
|
24
|
+
- **Non-user-facing repos** — `qa.nonUserFacingRepos` (array of repo names, e.g.
|
|
25
|
+
`["api", "infrastructure"]`). If the key is missing, derive a proposal from
|
|
26
|
+
the project registry (repos with no UI framework signals — no expo/react/native/web
|
|
27
|
+
app surface) and **present it for operator confirmation before moving anything**; never
|
|
28
|
+
bulk-transition on an unconfirmed inference. Recommend persisting the confirmed list to
|
|
29
|
+
the config.
|
|
30
|
+
|
|
31
|
+
## Procedure
|
|
32
|
+
|
|
33
|
+
1. Query all tickets in the QA queue status.
|
|
34
|
+
2. Classify each by repo scope, in order:
|
|
35
|
+
- `repo:<name>` label or matching component → that repo.
|
|
36
|
+
- Unlabeled → determine the repo from the ticket content (description, AC, technical
|
|
37
|
+
approach) exactly as build-intake's repo-scope gate does, and stamp the
|
|
38
|
+
`repo:<name>` label while you're there so the next sweep is cheap.
|
|
39
|
+
- Undeterminable → leave in place; flag for the operator.
|
|
40
|
+
3. Partition:
|
|
41
|
+
- **Every** touched repo is non-user-facing → eligible to clear.
|
|
42
|
+
- Any user-facing repo in scope (including mixed-scope) → stays in the queue for the
|
|
43
|
+
human pass via `lisa-qa-queue`.
|
|
44
|
+
4. For each eligible ticket, in this order — the pairing must be failure-safe:
|
|
45
|
+
1. Post the audit comment first:
|
|
46
|
+
`[lisa-qa-clear] Certified without human QA: scope is <repo(s)>, not observable
|
|
47
|
+
with end-user access. Verified by the automated lifecycle pre-promotion.`
|
|
48
|
+
2. Then transition to the certified status.
|
|
49
|
+
3. Verify both halves landed before counting the ticket as moved. A comment without
|
|
50
|
+
the transition, or a transition without the comment, is a **partial** — report it
|
|
51
|
+
as such, never as completed.
|
|
52
|
+
5. Report the batch:
|
|
53
|
+
|
|
54
|
+
```text
|
|
55
|
+
## QA Clear — <date>
|
|
56
|
+
Moved to <certified>: <n> (<KEY-1>, <KEY-2>, …) — repo per ticket
|
|
57
|
+
Left in queue (user-facing): <n>
|
|
58
|
+
Left in queue (undeterminable repo — needs operator): <keys or none>
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The operator (or tester) reviews the moved list; anything that "sounds user-facing" can
|
|
62
|
+
be pulled back with a single instruction — the transition is reversible and the comment
|
|
63
|
+
marks exactly what was auto-cleared.
|
|
64
|
+
|
|
65
|
+
## Rules
|
|
66
|
+
|
|
67
|
+
- Never clear a mixed-scope ticket — partial human-verifiability means human QA.
|
|
68
|
+
- Never bulk-move on an inferred repo list without explicit operator confirmation.
|
|
69
|
+
- Every cleared ticket carries the audit comment; a transition without the comment is a
|
|
70
|
+
bug in this procedure.
|
|
71
|
+
- Idempotent by repair, not by skip: a ticket is complete only when it has BOTH the
|
|
72
|
+
`[lisa-qa-clear]` comment AND the certified status. Re-runs finish partials — comment
|
|
73
|
+
present but not certified → transition it; certified but no comment → post the
|
|
74
|
+
comment. Only fully-complete tickets are skipped silently.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-fail
|
|
3
|
+
description: "QA failure front door. Takes a human tester's plain-language failure description, finds the right ticket (the served ticket, or the original via duplicate-discovery when the report arrives untied), writes the structured failure report (repro / expected vs. actual / which acceptance criterion failed), diagnoses the EXPECTATION GAP — why the implementing agent's done-evidence said done when QA says it isn't — applies the qa-fail label that makes lisa-rework-triage detection deterministic, and transitions the ticket back to the build-ready status. QA never words a ticket again; agents never guess what QA meant."
|
|
4
|
+
allowed-tools: ["Skill", "Bash", "Read", "Glob", "Grep"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Fail: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
Convert a human failure observation into the exact artifact the fixing agent needs. Two
|
|
10
|
+
inputs arrive together or the report cannot be filed: a **target ticket** and the
|
|
11
|
+
**tester's verbatim description** (what they did, what they expected, what happened,
|
|
12
|
+
plus any screenshots).
|
|
13
|
+
|
|
14
|
+
## Phase 1 — Resolve the target ticket
|
|
15
|
+
|
|
16
|
+
- **Tied report** (invoked from `lisa-qa-queue` with a key): use it.
|
|
17
|
+
- **Untied report** ("I found something broken: …"): run duplicate-discovery BEFORE any
|
|
18
|
+
write, reusing the mandatory relationship-discovery searches defined by the configured
|
|
19
|
+
tracker's write skill (dispatched through `lisa-tracker-write`): search open and
|
|
20
|
+
recently-closed tickets in the affected area, and the current QA-queue set. If an existing ticket
|
|
21
|
+
covers it, that ticket is the target — update it, never file a twin. Only when the
|
|
22
|
+
search documents no match, create a new Bug via `lisa-tracker-write` (which enforces
|
|
23
|
+
the full quality gates) and treat it as the target. Report which path was taken.
|
|
24
|
+
|
|
25
|
+
## Phase 2 — Structured failure report
|
|
26
|
+
|
|
27
|
+
Fetch the bundle via `lisa-tracker-read`. Post one comment:
|
|
28
|
+
|
|
29
|
+
```text
|
|
30
|
+
[lisa-qa-fail] QA failure — <one-line summary>
|
|
31
|
+
Reported by: <tester> on <date>, against <environment>
|
|
32
|
+
Steps to reproduce: <numbered, from the tester's words>
|
|
33
|
+
Expected: <what the ticket/AC promised, quoted or paraphrased faithfully>
|
|
34
|
+
Actual: <what the tester observed, verbatim where possible>
|
|
35
|
+
Failed acceptance criterion: <AC number/text from the ticket — or "not covered by any AC" (see gap)>
|
|
36
|
+
Attachments: <screenshots if provided>
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Preserve the tester's language in `Actual` — their words are the evidence; your job is
|
|
40
|
+
structure, not rewording.
|
|
41
|
+
|
|
42
|
+
## Phase 3 — Expectation-gap diagnosis
|
|
43
|
+
|
|
44
|
+
Answer explicitly: **why did the implementing agent believe this was done?** Locate the
|
|
45
|
+
prior cycle's done-evidence — the build-evidence comment (`lisa-tracker-evidence` output),
|
|
46
|
+
the merged PR's verification section, and any codified regression test — and compare it
|
|
47
|
+
against the failure. Classify the gap (exactly one):
|
|
48
|
+
|
|
49
|
+
| Gap | Meaning |
|
|
50
|
+
|-----|---------|
|
|
51
|
+
| `ac-mismatch` | The evidence verified a different reading of the AC than QA's — the criterion is ambiguous or the ticket distorted it. |
|
|
52
|
+
| `verification-weakness` | The evidence never actually exercised the failing path (asserted adjacent behavior, mocked the surface, or skipped the step). |
|
|
53
|
+
| `environment-difference` | Verified where it works (e.g. dev) but fails on the QA environment — config, data, or deployment drift. |
|
|
54
|
+
| `data-difference` | Behavior depends on data present in one environment and absent in the other. |
|
|
55
|
+
| `regression-since-merge` | The evidence was genuinely valid when produced; later merges broke it. |
|
|
56
|
+
| `not-covered-by-ac` | QA's expectation is real but no AC promises it — a spec gap, not an implementation failure. |
|
|
57
|
+
|
|
58
|
+
Append to the same comment:
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
Expectation gap: <classification>
|
|
62
|
+
Why the agent thought it was done: <2-3 lines citing the specific evidence artifact>
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
If no done-evidence exists at all, say so — `Expectation gap: no-evidence-found` is
|
|
66
|
+
itself a serious finding (the verification lifecycle was skipped) and must be surfaced,
|
|
67
|
+
not smoothed over.
|
|
68
|
+
|
|
69
|
+
For `not-covered-by-ac`, do NOT transition the ticket — the spec question goes to the
|
|
70
|
+
human product gate. Flag it in the response and stop after posting.
|
|
71
|
+
|
|
72
|
+
## Phase 4 — Label and transition
|
|
73
|
+
|
|
74
|
+
1. Apply the `qa-fail` label — this is the deterministic rework signal `lisa-rework-triage`
|
|
75
|
+
keys on at next claim, and the gap classification above becomes its primary evidence.
|
|
76
|
+
2. Transition the ticket to the build-ready status (`jira.workflow.ready`, or the
|
|
77
|
+
configured tracker's equivalent ready label/state when `tracker` is GitHub or
|
|
78
|
+
Linear). The rework loop takes it from here: intake claims it, `lisa-ticket-triage` Phase 2.5 runs
|
|
79
|
+
`lisa-rework-triage`, and the fix proceeds with full context.
|
|
80
|
+
3. Confirm to the tester in one plain sentence: what was recorded, and that the fix is
|
|
81
|
+
queued — no further action needed from them.
|
|
82
|
+
|
|
83
|
+
## Rules
|
|
84
|
+
|
|
85
|
+
- Never file a new ticket without the documented duplicate search (Phase 1).
|
|
86
|
+
- Never paraphrase away the tester's observation; structure around it.
|
|
87
|
+
- One `[lisa-qa-fail]` comment per failure event — if the same tester reports the same
|
|
88
|
+
failure again before a fix ships, add a short "seen again <date>" line to the existing
|
|
89
|
+
comment instead of a new block.
|
|
90
|
+
- The expectation gap must cite a specific evidence artifact or say `no-evidence-found` —
|
|
91
|
+
a gap classification without a citation is a guess, and guesses are worse than
|
|
92
|
+
`no-evidence-found`.
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-queue
|
|
3
|
+
description: "QA acceptance queue for human testers. Serves the next ticket awaiting QA verdict from the configured QA queue status as a plain-language acceptance brief — what the ticket promises, how to exercise it on the QA environment, in words a non-technical tester can follow. On 'pass', transitions the ticket to the configured certified status. On 'fail', delegates to lisa-qa-fail for the structured failure report, expectation-gap diagnosis, and return to the build-ready status. The conversational front door for human QA acceptance on any coding agent: testers say 'what's the next item?' / 'pass' / 'fail — here's what I saw'."
|
|
4
|
+
allowed-tools: ["Skill", "Bash", "Read", "Glob", "Grep"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Queue: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
Serve one ticket at a time to a human QA tester and record their verdict. The tester is
|
|
10
|
+
assumed **non-technical**: everything you present must be readable by an intern on their
|
|
11
|
+
first day — no stack traces, no jargon, no internal identifiers without explanation.
|
|
12
|
+
|
|
13
|
+
## Config resolution
|
|
14
|
+
|
|
15
|
+
Read `.lisa.config.json`:
|
|
16
|
+
|
|
17
|
+
- **QA queue status** — `jira.workflow.qa.queue`, falling back to `jira.workflow.done.staging`
|
|
18
|
+
(whatever status the project uses for "deployed to the environment QA tests against").
|
|
19
|
+
If neither exists, stop and report the missing config.
|
|
20
|
+
- **Certified status** — `jira.workflow.qa.certified` (the project's "passed QA, ships
|
|
21
|
+
with the next release" status). Required for the pass path; if missing, stop and
|
|
22
|
+
instruct the operator to add it — never guess a terminal status.
|
|
23
|
+
- **Build-ready status** — `jira.workflow.ready` (fail path, via `lisa-qa-fail`).
|
|
24
|
+
- **Tracker dispatch** — as everywhere: `tracker` decides JIRA / GitHub / Linear surfaces;
|
|
25
|
+
status names above map to labels/states on non-JIRA trackers.
|
|
26
|
+
|
|
27
|
+
## Serving the next item
|
|
28
|
+
|
|
29
|
+
1. Query the QA queue: tickets in the queue status, oldest first, excluding tickets
|
|
30
|
+
already carrying an unresolved `[lisa-qa-fail]` verdict from this sweep. Skip tickets
|
|
31
|
+
whose repo is listed in `qa.nonUserFacingRepos` — those belong to `lisa-qa-clear`,
|
|
32
|
+
not a human tester; note any encountered so the operator knows to run the clear.
|
|
33
|
+
2. Fetch the full context bundle via `lisa-tracker-read` (never serve from a bare summary).
|
|
34
|
+
3. Present ONE ticket as an **acceptance brief**:
|
|
35
|
+
- **What changed, in user terms** — one or two sentences, translated from the ticket's
|
|
36
|
+
stakeholder section.
|
|
37
|
+
- **How to try it** — concrete steps on the QA environment (the `exploration` /
|
|
38
|
+
Validation Journey config supplies the URL and test credentials): where to go, what
|
|
39
|
+
to click or type, which account to use.
|
|
40
|
+
- **What success looks like** — each acceptance criterion rewritten as an observable
|
|
41
|
+
check ("you should see …"), numbered so the tester can cite one on failure.
|
|
42
|
+
- **Worth poking at** — up to three edge probes drawn from the ticket's edge-case
|
|
43
|
+
triage findings, phrased as actions ("try it with an empty search box").
|
|
44
|
+
4. End with: "Say **pass**, or describe what you saw if it failed."
|
|
45
|
+
|
|
46
|
+
## Recording the verdict
|
|
47
|
+
|
|
48
|
+
- **Pass** — transition the ticket to the certified status via the tracker access layer,
|
|
49
|
+
post a brief `[lisa-qa-queue] QA pass` comment naming who verified and when, and offer
|
|
50
|
+
the next item.
|
|
51
|
+
- **Fail** — invoke `lisa-qa-fail` with the ticket key and the tester's own words
|
|
52
|
+
(verbatim — do not paraphrase away detail; attach any screenshots they provided). That
|
|
53
|
+
skill owns the failure report, the expectation-gap diagnosis, the `qa-fail` label, and
|
|
54
|
+
the transition back to build-ready. When it completes, confirm to the tester in one
|
|
55
|
+
plain sentence what was recorded and offer the next item.
|
|
56
|
+
- **Unclear / can't test** — if the tester cannot exercise the ticket (missing access,
|
|
57
|
+
feature flag off, no test data), do NOT guess a verdict. Post a
|
|
58
|
+
`[lisa-qa-queue] QA blocked: <reason>` comment, leave the status untouched, flag it in
|
|
59
|
+
the session summary for the operator, and serve the next item.
|
|
60
|
+
|
|
61
|
+
## Rules
|
|
62
|
+
|
|
63
|
+
- One ticket at a time — never dump the queue on the tester.
|
|
64
|
+
- The tester's verbatim description is evidence; preserve it exactly in whatever is posted.
|
|
65
|
+
- Never transition to certified without an explicit "pass" from the tester.
|
|
66
|
+
- All tracker writes go through the access layer / `lisa-qa-fail` — this skill never
|
|
67
|
+
hand-crafts tracker mutations beyond the pass transition and its comment.
|
|
68
|
+
- Session summary on request ("how did we do?"): counts of passed / failed / blocked /
|
|
69
|
+
remaining in queue.
|
|
@@ -24,9 +24,9 @@ If none confirm, output `## Verdict: NOT_REWORK` and stop — this skill takes n
|
|
|
24
24
|
first-attempt work.
|
|
25
25
|
|
|
26
26
|
1. **Status history shows a post-implementation regression transition.** The ticket's
|
|
27
|
-
changelog contains a transition from a post-build state (the configured `claimed`,
|
|
28
|
-
`done.*` environment state
|
|
29
|
-
|
|
27
|
+
changelog contains a transition from a post-build state (the configured `claimed`,
|
|
28
|
+
any configured `done.*` environment state, or a review state) back to the configured
|
|
29
|
+
`ready` state.
|
|
30
30
|
- JIRA: fetch the changelog via the `lisa-atlassian-access` REST substrate —
|
|
31
31
|
`GET /rest/api/3/issue/<KEY>/changelog` — and scan `items[].field == "status"` entries.
|
|
32
32
|
- GitHub: `gh issue view <n> --json timelineItems` (or the timeline API) — scan for the
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Bulk-move QA-queue tickets scoped entirely to non-user-facing repos to the configured certified status, with an auditable moved-list report."
|
|
3
|
+
argument-hint: ""
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Use the /lisa-qa-clear skill to clear non-human-verifiable tickets from the QA queue. $ARGUMENTS
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "File a QA failure: find the right ticket (never a duplicate), write the structured failure report, diagnose the expectation gap, label qa-fail, and return the ticket to build-ready."
|
|
3
|
+
argument-hint: "[ticket-key] <what you saw, in your own words>"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Use the /lisa-qa-fail skill to file this QA failure. $ARGUMENTS
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Serve the next ticket awaiting QA verdict as a plain-language acceptance brief; record pass (→ certified) or fail (→ structured report via lisa-qa-fail)."
|
|
3
|
+
argument-hint: "[pass | fail <description> | next]"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Use the /lisa-qa-queue skill to serve the QA acceptance queue. $ARGUMENTS
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-checklist
|
|
3
|
+
description: "Serve the current manual regression checklist to a human QA tester. Reads the project's curated journey list, cross-references it against what the automated suites (Playwright web E2E, Maestro native E2E) actually cover by scanning the spec files, and serves only the journeys that still need human eyes — each as plain-language steps. Keeps a single source of truth so testers never work from a stale personal copy, and shrinks automatically as automated coverage grows."
|
|
4
|
+
allowed-tools: ["Bash", "Read", "Glob", "Grep", "Write", "Edit"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Checklist: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
The manual regression sweep exists to catch what automation does not. Its checklist must
|
|
10
|
+
therefore be computed, not remembered: curated journeys minus automated coverage, at the
|
|
11
|
+
moment the tester asks.
|
|
12
|
+
|
|
13
|
+
## Sources
|
|
14
|
+
|
|
15
|
+
1. **Curated journey list** — `qa.checklistFile` in `.lisa.config.json`, default
|
|
16
|
+
`.lisa/qa-checklist.md`. Format: one `## Journey: <name>` section per user journey,
|
|
17
|
+
with plain-language steps and an optional `automation:` line naming the covering spec
|
|
18
|
+
file(s) once one exists. If the file does not exist, offer to bootstrap it: derive
|
|
19
|
+
candidate journeys from the app's route map and the existing E2E suites' describe
|
|
20
|
+
blocks, write the draft, and ask the operator to curate it once. Never invent
|
|
21
|
+
journeys silently.
|
|
22
|
+
2. **Automated coverage** — scan the repo's E2E suites (Playwright specs, Maestro flows;
|
|
23
|
+
locate via the project's e2e/test directories). An `automation:` line must name BOTH
|
|
24
|
+
the spec file and the specific test within it (the Playwright `describe`/`test` title
|
|
25
|
+
or Maestro flow name): `automation: <spec-path> :: <test-or-flow-name>`. A journey
|
|
26
|
+
counts as covered only when that spec file exists, the named test/flow is present in
|
|
27
|
+
it, and it is not skipped (`test.skip`, commented-out flow, or excluded from CI). A
|
|
28
|
+
file-only line, a named test that no longer matches, or any ambiguous mapping is
|
|
29
|
+
treated as **uncovered** — a live filename proves nothing about what the spec
|
|
30
|
+
exercises. A deleted, renamed, or skipped test silently un-covers its journey — the
|
|
31
|
+
very regression this check exists to catch; call it out loudly.
|
|
32
|
+
|
|
33
|
+
## Serving the sweep
|
|
34
|
+
|
|
35
|
+
Present two lists, human-first:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
## Manual sweep — <n> journeys need your eyes
|
|
39
|
+
1. <Journey name> — <plain-language steps>
|
|
40
|
+
Why manual: <no automation | automation skipped/stale: <spec>>
|
|
41
|
+
...
|
|
42
|
+
|
|
43
|
+
## Covered by automation — <n> journeys (skim, don't re-test)
|
|
44
|
+
- <Journey name> — <spec file> (<playwright|maestro>)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Rules for the served text: steps an intern can follow, no spec-file jargon in the manual
|
|
48
|
+
section beyond the "why manual" line, and stable journey ordering (file order) so testers
|
|
49
|
+
can resume mid-sweep.
|
|
50
|
+
|
|
51
|
+
## Maintaining the list
|
|
52
|
+
|
|
53
|
+
- Tester or operator adds/edits journeys in plain language → edit the checklist file
|
|
54
|
+
(this skill may apply the edit on request; it is a repo file, so changes ride normal
|
|
55
|
+
review).
|
|
56
|
+
- When a journey gains automation (e.g. `lisa-codify-verification` lands a spec), add its
|
|
57
|
+
`automation: <spec-path> :: <test-or-flow-name>` line — on request this skill locates
|
|
58
|
+
the covering spec and test, confirms the named test actually drives the journey's
|
|
59
|
+
steps, and writes the line itself.
|
|
60
|
+
- Never maintain per-tester copies; the file is the single source of truth and the
|
|
61
|
+
computed view is always derived fresh.
|
|
62
|
+
|
|
63
|
+
## Rules
|
|
64
|
+
|
|
65
|
+
- Coverage claims must point at an existing, non-skipped spec — "there's a test somewhere"
|
|
66
|
+
does not count.
|
|
67
|
+
- Serving is read-only by default; file edits happen only on explicit request.
|
|
68
|
+
- If the automated-coverage scan finds specs exercising a journey that is missing from
|
|
69
|
+
the curated list entirely, surface it as a suggested addition — the human curates, the
|
|
70
|
+
skill proposes.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-clear
|
|
3
|
+
description: "Bulk-clear tickets a human QA tester cannot verify. Finds tickets in the configured QA queue status that are scoped entirely to non-user-facing repos (API/backend services, infrastructure) — where QA has only end-user access and nothing observable to test — and transitions them to the configured certified status, reporting the moved list for the record. Runnable from any coding agent as a single instruction like 'move all backend- and infrastructure-only tickets from the QA queue to certified'."
|
|
4
|
+
allowed-tools: ["Skill", "Bash", "Read", "Glob", "Grep"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Clear: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
Human QA can only judge what a user can see. Tickets whose entire scope is a
|
|
10
|
+
non-user-facing repo were already verified by the automated lifecycle before reaching the
|
|
11
|
+
QA queue; holding them for a human who cannot observe them is pure queue noise. Clear
|
|
12
|
+
them in one auditable batch.
|
|
13
|
+
|
|
14
|
+
## Config resolution
|
|
15
|
+
|
|
16
|
+
Read `.lisa.config.json`:
|
|
17
|
+
|
|
18
|
+
- **QA queue status** — `jira.workflow.qa.queue`, falling back to
|
|
19
|
+
`jira.workflow.done.staging`.
|
|
20
|
+
- **Certified status** — `jira.workflow.qa.certified`. Required; if missing, stop and
|
|
21
|
+
instruct the operator — never guess a terminal status.
|
|
22
|
+
- **Tracker dispatch** — `tracker` decides the surface as everywhere; on GitHub or
|
|
23
|
+
Linear the status names above map to the equivalent labels/states.
|
|
24
|
+
- **Non-user-facing repos** — `qa.nonUserFacingRepos` (array of repo names, e.g.
|
|
25
|
+
`["api", "infrastructure"]`). If the key is missing, derive a proposal from
|
|
26
|
+
the project registry (repos with no UI framework signals — no expo/react/native/web
|
|
27
|
+
app surface) and **present it for operator confirmation before moving anything**; never
|
|
28
|
+
bulk-transition on an unconfirmed inference. Recommend persisting the confirmed list to
|
|
29
|
+
the config.
|
|
30
|
+
|
|
31
|
+
## Procedure
|
|
32
|
+
|
|
33
|
+
1. Query all tickets in the QA queue status.
|
|
34
|
+
2. Classify each by repo scope, in order:
|
|
35
|
+
- `repo:<name>` label or matching component → that repo.
|
|
36
|
+
- Unlabeled → determine the repo from the ticket content (description, AC, technical
|
|
37
|
+
approach) exactly as build-intake's repo-scope gate does, and stamp the
|
|
38
|
+
`repo:<name>` label while you're there so the next sweep is cheap.
|
|
39
|
+
- Undeterminable → leave in place; flag for the operator.
|
|
40
|
+
3. Partition:
|
|
41
|
+
- **Every** touched repo is non-user-facing → eligible to clear.
|
|
42
|
+
- Any user-facing repo in scope (including mixed-scope) → stays in the queue for the
|
|
43
|
+
human pass via `lisa-qa-queue`.
|
|
44
|
+
4. For each eligible ticket, in this order — the pairing must be failure-safe:
|
|
45
|
+
1. Post the audit comment first:
|
|
46
|
+
`[lisa-qa-clear] Certified without human QA: scope is <repo(s)>, not observable
|
|
47
|
+
with end-user access. Verified by the automated lifecycle pre-promotion.`
|
|
48
|
+
2. Then transition to the certified status.
|
|
49
|
+
3. Verify both halves landed before counting the ticket as moved. A comment without
|
|
50
|
+
the transition, or a transition without the comment, is a **partial** — report it
|
|
51
|
+
as such, never as completed.
|
|
52
|
+
5. Report the batch:
|
|
53
|
+
|
|
54
|
+
```text
|
|
55
|
+
## QA Clear — <date>
|
|
56
|
+
Moved to <certified>: <n> (<KEY-1>, <KEY-2>, …) — repo per ticket
|
|
57
|
+
Left in queue (user-facing): <n>
|
|
58
|
+
Left in queue (undeterminable repo — needs operator): <keys or none>
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The operator (or tester) reviews the moved list; anything that "sounds user-facing" can
|
|
62
|
+
be pulled back with a single instruction — the transition is reversible and the comment
|
|
63
|
+
marks exactly what was auto-cleared.
|
|
64
|
+
|
|
65
|
+
## Rules
|
|
66
|
+
|
|
67
|
+
- Never clear a mixed-scope ticket — partial human-verifiability means human QA.
|
|
68
|
+
- Never bulk-move on an inferred repo list without explicit operator confirmation.
|
|
69
|
+
- Every cleared ticket carries the audit comment; a transition without the comment is a
|
|
70
|
+
bug in this procedure.
|
|
71
|
+
- Idempotent by repair, not by skip: a ticket is complete only when it has BOTH the
|
|
72
|
+
`[lisa-qa-clear]` comment AND the certified status. Re-runs finish partials — comment
|
|
73
|
+
present but not certified → transition it; certified but no comment → post the
|
|
74
|
+
comment. Only fully-complete tickets are skipped silently.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lisa-qa-fail
|
|
3
|
+
description: "QA failure front door. Takes a human tester's plain-language failure description, finds the right ticket (the served ticket, or the original via duplicate-discovery when the report arrives untied), writes the structured failure report (repro / expected vs. actual / which acceptance criterion failed), diagnoses the EXPECTATION GAP — why the implementing agent's done-evidence said done when QA says it isn't — applies the qa-fail label that makes lisa-rework-triage detection deterministic, and transitions the ticket back to the build-ready status. QA never words a ticket again; agents never guess what QA meant."
|
|
4
|
+
allowed-tools: ["Skill", "Bash", "Read", "Glob", "Grep"]
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# QA Fail: $ARGUMENTS
|
|
8
|
+
|
|
9
|
+
Convert a human failure observation into the exact artifact the fixing agent needs. Two
|
|
10
|
+
inputs arrive together or the report cannot be filed: a **target ticket** and the
|
|
11
|
+
**tester's verbatim description** (what they did, what they expected, what happened,
|
|
12
|
+
plus any screenshots).
|
|
13
|
+
|
|
14
|
+
## Phase 1 — Resolve the target ticket
|
|
15
|
+
|
|
16
|
+
- **Tied report** (invoked from `lisa-qa-queue` with a key): use it.
|
|
17
|
+
- **Untied report** ("I found something broken: …"): run duplicate-discovery BEFORE any
|
|
18
|
+
write, reusing the mandatory relationship-discovery searches defined by the configured
|
|
19
|
+
tracker's write skill (dispatched through `lisa-tracker-write`): search open and
|
|
20
|
+
recently-closed tickets in the affected area, and the current QA-queue set. If an existing ticket
|
|
21
|
+
covers it, that ticket is the target — update it, never file a twin. Only when the
|
|
22
|
+
search documents no match, create a new Bug via `lisa-tracker-write` (which enforces
|
|
23
|
+
the full quality gates) and treat it as the target. Report which path was taken.
|
|
24
|
+
|
|
25
|
+
## Phase 2 — Structured failure report
|
|
26
|
+
|
|
27
|
+
Fetch the bundle via `lisa-tracker-read`. Post one comment:
|
|
28
|
+
|
|
29
|
+
```text
|
|
30
|
+
[lisa-qa-fail] QA failure — <one-line summary>
|
|
31
|
+
Reported by: <tester> on <date>, against <environment>
|
|
32
|
+
Steps to reproduce: <numbered, from the tester's words>
|
|
33
|
+
Expected: <what the ticket/AC promised, quoted or paraphrased faithfully>
|
|
34
|
+
Actual: <what the tester observed, verbatim where possible>
|
|
35
|
+
Failed acceptance criterion: <AC number/text from the ticket — or "not covered by any AC" (see gap)>
|
|
36
|
+
Attachments: <screenshots if provided>
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Preserve the tester's language in `Actual` — their words are the evidence; your job is
|
|
40
|
+
structure, not rewording.
|
|
41
|
+
|
|
42
|
+
## Phase 3 — Expectation-gap diagnosis
|
|
43
|
+
|
|
44
|
+
Answer explicitly: **why did the implementing agent believe this was done?** Locate the
|
|
45
|
+
prior cycle's done-evidence — the build-evidence comment (`lisa-tracker-evidence` output),
|
|
46
|
+
the merged PR's verification section, and any codified regression test — and compare it
|
|
47
|
+
against the failure. Classify the gap (exactly one):
|
|
48
|
+
|
|
49
|
+
| Gap | Meaning |
|
|
50
|
+
|-----|---------|
|
|
51
|
+
| `ac-mismatch` | The evidence verified a different reading of the AC than QA's — the criterion is ambiguous or the ticket distorted it. |
|
|
52
|
+
| `verification-weakness` | The evidence never actually exercised the failing path (asserted adjacent behavior, mocked the surface, or skipped the step). |
|
|
53
|
+
| `environment-difference` | Verified where it works (e.g. dev) but fails on the QA environment — config, data, or deployment drift. |
|
|
54
|
+
| `data-difference` | Behavior depends on data present in one environment and absent in the other. |
|
|
55
|
+
| `regression-since-merge` | The evidence was genuinely valid when produced; later merges broke it. |
|
|
56
|
+
| `not-covered-by-ac` | QA's expectation is real but no AC promises it — a spec gap, not an implementation failure. |
|
|
57
|
+
|
|
58
|
+
Append to the same comment:
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
Expectation gap: <classification>
|
|
62
|
+
Why the agent thought it was done: <2-3 lines citing the specific evidence artifact>
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
If no done-evidence exists at all, say so — `Expectation gap: no-evidence-found` is
|
|
66
|
+
itself a serious finding (the verification lifecycle was skipped) and must be surfaced,
|
|
67
|
+
not smoothed over.
|
|
68
|
+
|
|
69
|
+
For `not-covered-by-ac`, do NOT transition the ticket — the spec question goes to the
|
|
70
|
+
human product gate. Flag it in the response and stop after posting.
|
|
71
|
+
|
|
72
|
+
## Phase 4 — Label and transition
|
|
73
|
+
|
|
74
|
+
1. Apply the `qa-fail` label — this is the deterministic rework signal `lisa-rework-triage`
|
|
75
|
+
keys on at next claim, and the gap classification above becomes its primary evidence.
|
|
76
|
+
2. Transition the ticket to the build-ready status (`jira.workflow.ready`, or the
|
|
77
|
+
configured tracker's equivalent ready label/state when `tracker` is GitHub or
|
|
78
|
+
Linear). The rework loop takes it from here: intake claims it, `lisa-ticket-triage` Phase 2.5 runs
|
|
79
|
+
`lisa-rework-triage`, and the fix proceeds with full context.
|
|
80
|
+
3. Confirm to the tester in one plain sentence: what was recorded, and that the fix is
|
|
81
|
+
queued — no further action needed from them.
|
|
82
|
+
|
|
83
|
+
## Rules
|
|
84
|
+
|
|
85
|
+
- Never file a new ticket without the documented duplicate search (Phase 1).
|
|
86
|
+
- Never paraphrase away the tester's observation; structure around it.
|
|
87
|
+
- One `[lisa-qa-fail]` comment per failure event — if the same tester reports the same
|
|
88
|
+
failure again before a fix ships, add a short "seen again <date>" line to the existing
|
|
89
|
+
comment instead of a new block.
|
|
90
|
+
- The expectation gap must cite a specific evidence artifact or say `no-evidence-found` —
|
|
91
|
+
a gap classification without a citation is a guess, and guesses are worse than
|
|
92
|
+
`no-evidence-found`.
|