@codyswann/lisa 2.25.5 → 2.26.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/codex/hooks-installer.d.ts +7 -0
- package/dist/codex/hooks-installer.d.ts.map +1 -1
- package/dist/codex/hooks-installer.js +45 -3
- package/dist/codex/hooks-installer.js.map +1 -1
- package/dist/codex/scripts/_extract-edit-paths.sh +49 -0
- package/dist/codex/scripts/block-migration-edits.sh +15 -49
- package/dist/codex/scripts/format-on-edit.sh +26 -21
- package/dist/codex/scripts/lint-on-edit.sh +23 -14
- package/dist/codex/scripts/rubocop-on-edit.sh +23 -14
- package/dist/codex/scripts/sg-scan-on-edit.sh +22 -12
- package/package.json +1 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -2
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/commands/exploratory-qa.md +7 -0
- package/plugins/lisa-expo/skills/exploratory-qa/SKILL.md +108 -0
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/commands/exploratory-qa.md +7 -0
- package/plugins/lisa-harper-fabric/skills/exploratory-qa/SKILL.md +108 -0
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -2
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -2
- package/plugins/lisa-rails/commands/exploratory-qa.md +7 -0
- package/plugins/lisa-rails/skills/exploratory-qa/SKILL.md +108 -0
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -2
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/src/expo/commands/exploratory-qa.md +7 -0
- package/plugins/src/expo/skills/exploratory-qa/SKILL.md +108 -0
- package/plugins/src/harper-fabric/commands/exploratory-qa.md +7 -0
- package/plugins/src/harper-fabric/skills/exploratory-qa/SKILL.md +108 -0
- package/plugins/src/rails/commands/exploratory-qa.md +7 -0
- package/plugins/src/rails/skills/exploratory-qa/SKILL.md +108 -0
- package/scripts/generate-codex-plugin-artifacts.mjs +17 -129
- package/plugins/lisa/hooks/hooks.json +0 -104
- package/plugins/lisa-nestjs/hooks/hooks.json +0 -15
- package/plugins/lisa-rails/hooks/hooks.json +0 -19
- package/plugins/lisa-typescript/hooks/hooks.json +0 -23
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: exploratory-qa
|
|
3
|
+
description: Playwright-backed exploratory QA workflow for web apps. Use when asked to audit an app or project with Playwright/e2e tests, find human-noticeable bugs, identify gaps in automated test coverage, test responsive breakpoints, observe slow or unclear load states, exercise mutable workflows with cleanup, or produce a QA gaps report.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploratory QA
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
Run a human-style exploratory QA pass informed by the existing Playwright suite. The goal is to find issues users notice and machines often miss, then translate those observations into actionable Playwright coverage gaps.
|
|
11
|
+
|
|
12
|
+
## Core Workflow
|
|
13
|
+
|
|
14
|
+
### 1. Establish Scope
|
|
15
|
+
|
|
16
|
+
- Identify the target environment, account type, browser requirement, and requested report path.
|
|
17
|
+
- If credentials, tenant, seed data, or mutation boundaries are missing and cannot be discovered safely, ask a concise clarifying question.
|
|
18
|
+
- Treat production-like environments conservatively. Do not mutate production data unless the user explicitly approves it.
|
|
19
|
+
- Prefer a test user, dev/staging environment, or isolated seeded account for mutation testing.
|
|
20
|
+
|
|
21
|
+
### 2. Research Existing Playwright Coverage
|
|
22
|
+
|
|
23
|
+
- Inspect Playwright config, auth/setup projects, fixtures, constants, selectors, test directories, skipped tests, retries, and viewport/device projects.
|
|
24
|
+
- Summarize:
|
|
25
|
+
- Number and shape of specs/tests.
|
|
26
|
+
- Major workflows covered well.
|
|
27
|
+
- Skipped/flaky/TODO coverage.
|
|
28
|
+
- Viewports and browser projects used.
|
|
29
|
+
- Mutating tests and their cleanup strategy.
|
|
30
|
+
- Areas where tests assert presence instead of behavior.
|
|
31
|
+
- Use existing test helpers/selectors when exploring the app. They reveal intended flows and stable hooks.
|
|
32
|
+
|
|
33
|
+
### 3. Choose Browser Tools
|
|
34
|
+
|
|
35
|
+
- Use the user's requested browser/tool when specified.
|
|
36
|
+
- Use a real profile browser when the task depends on existing cookies, SSO, extensions, or a logged-in session.
|
|
37
|
+
- Use Playwright automation when exact viewport sizing, repeatable navigation, screenshots, console logs, traces, or route timing are needed.
|
|
38
|
+
- Do not let automation hide human observations. Screenshots, visible text, layout, latency, scrollability, and affordance clarity matter.
|
|
39
|
+
|
|
40
|
+
### 4. Explore Like A Human
|
|
41
|
+
|
|
42
|
+
Cover at least these dimensions unless the user narrows scope:
|
|
43
|
+
|
|
44
|
+
- **Navigation:** direct URLs and visible click paths.
|
|
45
|
+
- **Responsive behavior:** supported breakpoints, plus boundary widths where layout changes.
|
|
46
|
+
- **Visual/layout quality:** cutoff text, overlap, offscreen controls, accidental horizontal scroll, empty space, awkward density, hidden stale content.
|
|
47
|
+
- **Load states:** blanks, skeletons, spinners, `Loading...`, `Connecting...`, delayed hydration, and whether delays exceed a human-acceptable threshold.
|
|
48
|
+
- **Behavior correctness:** sorting, filtering, saving, deleting, disabled states, permissions, tab persistence, URL canonicalization.
|
|
49
|
+
- **Accessibility/testability:** hidden or off-canvas content exposed to accessibility/text locators, duplicate inactive route content, zero-sized interactive elements.
|
|
50
|
+
- **Console/network health:** errors, missing assets, failed requests, noisy expected errors that could bury real failures.
|
|
51
|
+
- **Data hygiene:** repeated test artifacts, stale seeded content, polluted shared accounts.
|
|
52
|
+
|
|
53
|
+
## Mutation Discipline
|
|
54
|
+
|
|
55
|
+
Exploratory QA should exercise mutable workflows when the environment is safe.
|
|
56
|
+
|
|
57
|
+
- Prefer high-value user workflows: create/edit/delete records, lists, boards, tags, notes, comments, scenarios, uploads, messages, settings, invitations, or assignments.
|
|
58
|
+
- Use unique names with a clear prefix such as `qa-`, `pw-`, or `codex-`.
|
|
59
|
+
- Before mutating, identify the cleanup path. After mutating, make a best effort to clean up through the UI, then verify cleanup.
|
|
60
|
+
- If UI cleanup is unavailable, record that as a product/test gap. Use documented API cleanup only if appropriate for the project and account.
|
|
61
|
+
- Record all mutations performed, cleanup attempts, and residue left behind.
|
|
62
|
+
- Avoid destructive bulk actions unless the user explicitly asks or the test account is clearly disposable.
|
|
63
|
+
|
|
64
|
+
## Breakpoint Strategy
|
|
65
|
+
|
|
66
|
+
- Discover breakpoints from the codebase, design tokens, CSS, docs, or Playwright constants when possible.
|
|
67
|
+
- Test each named breakpoint and the boundary on both sides of important cutoffs.
|
|
68
|
+
- If breakpoints are unknown, use a practical baseline: phone, tablet/narrow, desktop, and any app-specific cutoff discovered during research.
|
|
69
|
+
- For each breakpoint, assert both user-visible behavior and automation-relevant state:
|
|
70
|
+
- Expected shell/navigation variant.
|
|
71
|
+
- Route-specific loaded content.
|
|
72
|
+
- Critical controls visible and reachable.
|
|
73
|
+
- No unintended document-level horizontal overflow.
|
|
74
|
+
- Intentional scroll containers remain usable.
|
|
75
|
+
- Hidden/off-canvas UI is not exposed as active content.
|
|
76
|
+
|
|
77
|
+
## Load-State Strategy
|
|
78
|
+
|
|
79
|
+
Measure perceived latency, not just eventual success.
|
|
80
|
+
|
|
81
|
+
- Track time to shell, time to meaningful route content, and time spent in blank/loading/spinner/connecting states.
|
|
82
|
+
- Note when a page is technically interactive but still visually incomplete.
|
|
83
|
+
- Treat long waits without clear progress, error, retry, or cancellation as bugs or test gaps.
|
|
84
|
+
- Do not overfit exact milliseconds unless the project has defined budgets. Use practical labels such as noticeable, slow, or unacceptable and include observed durations when available.
|
|
85
|
+
|
|
86
|
+
## Report Structure
|
|
87
|
+
|
|
88
|
+
Write the report to the user's requested path. If none is specified, use a clear local name such as `PLAYWRIGHT_GAPS.md` or `EXPLORATORY_QA_REPORT.md`.
|
|
89
|
+
|
|
90
|
+
Include:
|
|
91
|
+
|
|
92
|
+
- Scope: target URL/env, browser/tool, account type, build/version if visible, date.
|
|
93
|
+
- Existing Playwright coverage: strengths, thin areas, skipped tests, viewport/browser matrix, mutation coverage.
|
|
94
|
+
- Exploratory findings: steps, observed behavior, impact, why existing tests miss it, recommended test.
|
|
95
|
+
- Breakpoint findings: exact viewports tested and boundary behavior.
|
|
96
|
+
- Load-state findings: observed delays, blank/spinner/connecting states, suggested budgets/tests.
|
|
97
|
+
- Mutation findings: data created, behavior observed, cleanup attempt, cleanup result, residue.
|
|
98
|
+
- Missing Playwright tests to add: prioritized, concrete, and automatable.
|
|
99
|
+
- Maintenance notes: selector scoping, fixture cleanup, flaky/stale-state risks.
|
|
100
|
+
|
|
101
|
+
## Quality Bar
|
|
102
|
+
|
|
103
|
+
- Be honest about what was and was not tested.
|
|
104
|
+
- Distinguish user-visible bugs from missing automated coverage.
|
|
105
|
+
- Prefer concrete examples over vague recommendations.
|
|
106
|
+
- Do not claim cleanup succeeded unless verified.
|
|
107
|
+
- Do not let broad locators pass against hidden/inactive content.
|
|
108
|
+
- Preserve unrelated repo changes and keep report edits scoped.
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Run a Playwright-backed exploratory QA pass: audit the app like a human, find user-noticeable bugs and gaps in automated test coverage, and produce a QA gaps report."
|
|
3
|
+
allowed-tools: ["Skill"]
|
|
4
|
+
argument-hint: "[target-url | env] [report-path]"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Use the /lisa-harper-fabric:exploratory-qa skill to run a human-style exploratory QA pass informed by the existing Playwright suite, then produce an actionable coverage-gaps report. $ARGUMENTS
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: exploratory-qa
|
|
3
|
+
description: Playwright-backed exploratory QA workflow for web apps. Use when asked to audit an app or project with Playwright/e2e tests, find human-noticeable bugs, identify gaps in automated test coverage, test responsive breakpoints, observe slow or unclear load states, exercise mutable workflows with cleanup, or produce a QA gaps report.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploratory QA
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
Run a human-style exploratory QA pass informed by the existing Playwright suite. The goal is to find issues users notice and machines often miss, then translate those observations into actionable Playwright coverage gaps.
|
|
11
|
+
|
|
12
|
+
## Core Workflow
|
|
13
|
+
|
|
14
|
+
### 1. Establish Scope
|
|
15
|
+
|
|
16
|
+
- Identify the target environment, account type, browser requirement, and requested report path.
|
|
17
|
+
- If credentials, tenant, seed data, or mutation boundaries are missing and cannot be discovered safely, ask a concise clarifying question.
|
|
18
|
+
- Treat production-like environments conservatively. Do not mutate production data unless the user explicitly approves it.
|
|
19
|
+
- Prefer a test user, dev/staging environment, or isolated seeded account for mutation testing.
|
|
20
|
+
|
|
21
|
+
### 2. Research Existing Playwright Coverage
|
|
22
|
+
|
|
23
|
+
- Inspect Playwright config, auth/setup projects, fixtures, constants, selectors, test directories, skipped tests, retries, and viewport/device projects.
|
|
24
|
+
- Summarize:
|
|
25
|
+
- Number and shape of specs/tests.
|
|
26
|
+
- Major workflows covered well.
|
|
27
|
+
- Skipped/flaky/TODO coverage.
|
|
28
|
+
- Viewports and browser projects used.
|
|
29
|
+
- Mutating tests and their cleanup strategy.
|
|
30
|
+
- Areas where tests assert presence instead of behavior.
|
|
31
|
+
- Use existing test helpers/selectors when exploring the app. They reveal intended flows and stable hooks.
|
|
32
|
+
|
|
33
|
+
### 3. Choose Browser Tools
|
|
34
|
+
|
|
35
|
+
- Use the user's requested browser/tool when specified.
|
|
36
|
+
- Use a real profile browser when the task depends on existing cookies, SSO, extensions, or a logged-in session.
|
|
37
|
+
- Use Playwright automation when exact viewport sizing, repeatable navigation, screenshots, console logs, traces, or route timing are needed.
|
|
38
|
+
- Do not let automation hide human observations. Screenshots, visible text, layout, latency, scrollability, and affordance clarity matter.
|
|
39
|
+
|
|
40
|
+
### 4. Explore Like A Human
|
|
41
|
+
|
|
42
|
+
Cover at least these dimensions unless the user narrows scope:
|
|
43
|
+
|
|
44
|
+
- **Navigation:** direct URLs and visible click paths.
|
|
45
|
+
- **Responsive behavior:** supported breakpoints, plus boundary widths where layout changes.
|
|
46
|
+
- **Visual/layout quality:** cutoff text, overlap, offscreen controls, accidental horizontal scroll, empty space, awkward density, hidden stale content.
|
|
47
|
+
- **Load states:** blanks, skeletons, spinners, `Loading...`, `Connecting...`, delayed hydration, and whether delays exceed a human-acceptable threshold.
|
|
48
|
+
- **Behavior correctness:** sorting, filtering, saving, deleting, disabled states, permissions, tab persistence, URL canonicalization.
|
|
49
|
+
- **Accessibility/testability:** hidden or off-canvas content exposed to accessibility/text locators, duplicate inactive route content, zero-sized interactive elements.
|
|
50
|
+
- **Console/network health:** errors, missing assets, failed requests, noisy expected errors that could bury real failures.
|
|
51
|
+
- **Data hygiene:** repeated test artifacts, stale seeded content, polluted shared accounts.
|
|
52
|
+
|
|
53
|
+
## Mutation Discipline
|
|
54
|
+
|
|
55
|
+
Exploratory QA should exercise mutable workflows when the environment is safe.
|
|
56
|
+
|
|
57
|
+
- Prefer high-value user workflows: create/edit/delete records, lists, boards, tags, notes, comments, scenarios, uploads, messages, settings, invitations, or assignments.
|
|
58
|
+
- Use unique names with a clear prefix such as `qa-`, `pw-`, or `codex-`.
|
|
59
|
+
- Before mutating, identify the cleanup path. After mutating, make a best effort to clean up through the UI, then verify cleanup.
|
|
60
|
+
- If UI cleanup is unavailable, record that as a product/test gap. Use documented API cleanup only if appropriate for the project and account.
|
|
61
|
+
- Record all mutations performed, cleanup attempts, and residue left behind.
|
|
62
|
+
- Avoid destructive bulk actions unless the user explicitly asks or the test account is clearly disposable.
|
|
63
|
+
|
|
64
|
+
## Breakpoint Strategy
|
|
65
|
+
|
|
66
|
+
- Discover breakpoints from the codebase, design tokens, CSS, docs, or Playwright constants when possible.
|
|
67
|
+
- Test each named breakpoint and the boundary on both sides of important cutoffs.
|
|
68
|
+
- If breakpoints are unknown, use a practical baseline: phone, tablet/narrow, desktop, and any app-specific cutoff discovered during research.
|
|
69
|
+
- For each breakpoint, assert both user-visible behavior and automation-relevant state:
|
|
70
|
+
- Expected shell/navigation variant.
|
|
71
|
+
- Route-specific loaded content.
|
|
72
|
+
- Critical controls visible and reachable.
|
|
73
|
+
- No unintended document-level horizontal overflow.
|
|
74
|
+
- Intentional scroll containers remain usable.
|
|
75
|
+
- Hidden/off-canvas UI is not exposed as active content.
|
|
76
|
+
|
|
77
|
+
## Load-State Strategy
|
|
78
|
+
|
|
79
|
+
Measure perceived latency, not just eventual success.
|
|
80
|
+
|
|
81
|
+
- Track time to shell, time to meaningful route content, and time spent in blank/loading/spinner/connecting states.
|
|
82
|
+
- Note when a page is technically interactive but still visually incomplete.
|
|
83
|
+
- Treat long waits without clear progress, error, retry, or cancellation as bugs or test gaps.
|
|
84
|
+
- Do not overfit exact milliseconds unless the project has defined budgets. Use practical labels such as noticeable, slow, or unacceptable and include observed durations when available.
|
|
85
|
+
|
|
86
|
+
## Report Structure
|
|
87
|
+
|
|
88
|
+
Write the report to the user's requested path. If none is specified, use a clear local name such as `PLAYWRIGHT_GAPS.md` or `EXPLORATORY_QA_REPORT.md`.
|
|
89
|
+
|
|
90
|
+
Include:
|
|
91
|
+
|
|
92
|
+
- Scope: target URL/env, browser/tool, account type, build/version if visible, date.
|
|
93
|
+
- Existing Playwright coverage: strengths, thin areas, skipped tests, viewport/browser matrix, mutation coverage.
|
|
94
|
+
- Exploratory findings: steps, observed behavior, impact, why existing tests miss it, recommended test.
|
|
95
|
+
- Breakpoint findings: exact viewports tested and boundary behavior.
|
|
96
|
+
- Load-state findings: observed delays, blank/spinner/connecting states, suggested budgets/tests.
|
|
97
|
+
- Mutation findings: data created, behavior observed, cleanup attempt, cleanup result, residue.
|
|
98
|
+
- Missing Playwright tests to add: prioritized, concrete, and automatable.
|
|
99
|
+
- Maintenance notes: selector scoping, fixture cleanup, flaky/stale-state risks.
|
|
100
|
+
|
|
101
|
+
## Quality Bar
|
|
102
|
+
|
|
103
|
+
- Be honest about what was and was not tested.
|
|
104
|
+
- Distinguish user-visible bugs from missing automated coverage.
|
|
105
|
+
- Prefer concrete examples over vague recommendations.
|
|
106
|
+
- Do not claim cleanup succeeded unless verified.
|
|
107
|
+
- Do not let broad locators pass against hidden/inactive content.
|
|
108
|
+
- Preserve unrelated repo changes and keep report edits scoped.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-nestjs",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.26.1",
|
|
4
4
|
"description": "NestJS-specific skills and migration write-protection hooks.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -15,7 +15,6 @@
|
|
|
15
15
|
"lisa-typescript"
|
|
16
16
|
],
|
|
17
17
|
"skills": "./skills/",
|
|
18
|
-
"hooks": "./hooks/hooks.json",
|
|
19
18
|
"interface": {
|
|
20
19
|
"displayName": "Lisa NestJS",
|
|
21
20
|
"shortDescription": "NestJS workflow guidance",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-rails",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.26.1",
|
|
4
4
|
"description": "Ruby on Rails-specific skills and hooks for RuboCop and ast-grep scanning on edit.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -15,7 +15,6 @@
|
|
|
15
15
|
"lisa"
|
|
16
16
|
],
|
|
17
17
|
"skills": "./skills/",
|
|
18
|
-
"hooks": "./hooks/hooks.json",
|
|
19
18
|
"interface": {
|
|
20
19
|
"displayName": "Lisa Rails",
|
|
21
20
|
"shortDescription": "Ruby on Rails workflows",
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Run a Playwright-backed exploratory QA pass: audit the app like a human, find user-noticeable bugs and gaps in automated test coverage, and produce a QA gaps report."
|
|
3
|
+
allowed-tools: ["Skill"]
|
|
4
|
+
argument-hint: "[target-url | env] [report-path]"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Use the /lisa-rails:exploratory-qa skill to run a human-style exploratory QA pass informed by the existing Playwright suite, then produce an actionable coverage-gaps report. $ARGUMENTS
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: exploratory-qa
|
|
3
|
+
description: Playwright-backed exploratory QA workflow for web apps. Use when asked to audit an app or project with Playwright/e2e tests, find human-noticeable bugs, identify gaps in automated test coverage, test responsive breakpoints, observe slow or unclear load states, exercise mutable workflows with cleanup, or produce a QA gaps report.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploratory QA
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
Run a human-style exploratory QA pass informed by the existing Playwright suite. The goal is to find issues users notice and machines often miss, then translate those observations into actionable Playwright coverage gaps.
|
|
11
|
+
|
|
12
|
+
## Core Workflow
|
|
13
|
+
|
|
14
|
+
### 1. Establish Scope
|
|
15
|
+
|
|
16
|
+
- Identify the target environment, account type, browser requirement, and requested report path.
|
|
17
|
+
- If credentials, tenant, seed data, or mutation boundaries are missing and cannot be discovered safely, ask a concise clarifying question.
|
|
18
|
+
- Treat production-like environments conservatively. Do not mutate production data unless the user explicitly approves it.
|
|
19
|
+
- Prefer a test user, dev/staging environment, or isolated seeded account for mutation testing.
|
|
20
|
+
|
|
21
|
+
### 2. Research Existing Playwright Coverage
|
|
22
|
+
|
|
23
|
+
- Inspect Playwright config, auth/setup projects, fixtures, constants, selectors, test directories, skipped tests, retries, and viewport/device projects.
|
|
24
|
+
- Summarize:
|
|
25
|
+
- Number and shape of specs/tests.
|
|
26
|
+
- Major workflows covered well.
|
|
27
|
+
- Skipped/flaky/TODO coverage.
|
|
28
|
+
- Viewports and browser projects used.
|
|
29
|
+
- Mutating tests and their cleanup strategy.
|
|
30
|
+
- Areas where tests assert presence instead of behavior.
|
|
31
|
+
- Use existing test helpers/selectors when exploring the app. They reveal intended flows and stable hooks.
|
|
32
|
+
|
|
33
|
+
### 3. Choose Browser Tools
|
|
34
|
+
|
|
35
|
+
- Use the user's requested browser/tool when specified.
|
|
36
|
+
- Use a real profile browser when the task depends on existing cookies, SSO, extensions, or a logged-in session.
|
|
37
|
+
- Use Playwright automation when exact viewport sizing, repeatable navigation, screenshots, console logs, traces, or route timing are needed.
|
|
38
|
+
- Do not let automation hide human observations. Screenshots, visible text, layout, latency, scrollability, and affordance clarity matter.
|
|
39
|
+
|
|
40
|
+
### 4. Explore Like A Human
|
|
41
|
+
|
|
42
|
+
Cover at least these dimensions unless the user narrows scope:
|
|
43
|
+
|
|
44
|
+
- **Navigation:** direct URLs and visible click paths.
|
|
45
|
+
- **Responsive behavior:** supported breakpoints, plus boundary widths where layout changes.
|
|
46
|
+
- **Visual/layout quality:** cutoff text, overlap, offscreen controls, accidental horizontal scroll, empty space, awkward density, hidden stale content.
|
|
47
|
+
- **Load states:** blanks, skeletons, spinners, `Loading...`, `Connecting...`, delayed hydration, and whether delays exceed a human-acceptable threshold.
|
|
48
|
+
- **Behavior correctness:** sorting, filtering, saving, deleting, disabled states, permissions, tab persistence, URL canonicalization.
|
|
49
|
+
- **Accessibility/testability:** hidden or off-canvas content exposed to accessibility/text locators, duplicate inactive route content, zero-sized interactive elements.
|
|
50
|
+
- **Console/network health:** errors, missing assets, failed requests, noisy expected errors that could bury real failures.
|
|
51
|
+
- **Data hygiene:** repeated test artifacts, stale seeded content, polluted shared accounts.
|
|
52
|
+
|
|
53
|
+
## Mutation Discipline
|
|
54
|
+
|
|
55
|
+
Exploratory QA should exercise mutable workflows when the environment is safe.
|
|
56
|
+
|
|
57
|
+
- Prefer high-value user workflows: create/edit/delete records, lists, boards, tags, notes, comments, scenarios, uploads, messages, settings, invitations, or assignments.
|
|
58
|
+
- Use unique names with a clear prefix such as `qa-`, `pw-`, or `codex-`.
|
|
59
|
+
- Before mutating, identify the cleanup path. After mutating, make a best effort to clean up through the UI, then verify cleanup.
|
|
60
|
+
- If UI cleanup is unavailable, record that as a product/test gap. Use documented API cleanup only if appropriate for the project and account.
|
|
61
|
+
- Record all mutations performed, cleanup attempts, and residue left behind.
|
|
62
|
+
- Avoid destructive bulk actions unless the user explicitly asks or the test account is clearly disposable.
|
|
63
|
+
|
|
64
|
+
## Breakpoint Strategy
|
|
65
|
+
|
|
66
|
+
- Discover breakpoints from the codebase, design tokens, CSS, docs, or Playwright constants when possible.
|
|
67
|
+
- Test each named breakpoint and the boundary on both sides of important cutoffs.
|
|
68
|
+
- If breakpoints are unknown, use a practical baseline: phone, tablet/narrow, desktop, and any app-specific cutoff discovered during research.
|
|
69
|
+
- For each breakpoint, assert both user-visible behavior and automation-relevant state:
|
|
70
|
+
- Expected shell/navigation variant.
|
|
71
|
+
- Route-specific loaded content.
|
|
72
|
+
- Critical controls visible and reachable.
|
|
73
|
+
- No unintended document-level horizontal overflow.
|
|
74
|
+
- Intentional scroll containers remain usable.
|
|
75
|
+
- Hidden/off-canvas UI is not exposed as active content.
|
|
76
|
+
|
|
77
|
+
## Load-State Strategy
|
|
78
|
+
|
|
79
|
+
Measure perceived latency, not just eventual success.
|
|
80
|
+
|
|
81
|
+
- Track time to shell, time to meaningful route content, and time spent in blank/loading/spinner/connecting states.
|
|
82
|
+
- Note when a page is technically interactive but still visually incomplete.
|
|
83
|
+
- Treat long waits without clear progress, error, retry, or cancellation as bugs or test gaps.
|
|
84
|
+
- Do not overfit exact milliseconds unless the project has defined budgets. Use practical labels such as noticeable, slow, or unacceptable and include observed durations when available.
|
|
85
|
+
|
|
86
|
+
## Report Structure
|
|
87
|
+
|
|
88
|
+
Write the report to the user's requested path. If none is specified, use a clear local name such as `PLAYWRIGHT_GAPS.md` or `EXPLORATORY_QA_REPORT.md`.
|
|
89
|
+
|
|
90
|
+
Include:
|
|
91
|
+
|
|
92
|
+
- Scope: target URL/env, browser/tool, account type, build/version if visible, date.
|
|
93
|
+
- Existing Playwright coverage: strengths, thin areas, skipped tests, viewport/browser matrix, mutation coverage.
|
|
94
|
+
- Exploratory findings: steps, observed behavior, impact, why existing tests miss it, recommended test.
|
|
95
|
+
- Breakpoint findings: exact viewports tested and boundary behavior.
|
|
96
|
+
- Load-state findings: observed delays, blank/spinner/connecting states, suggested budgets/tests.
|
|
97
|
+
- Mutation findings: data created, behavior observed, cleanup attempt, cleanup result, residue.
|
|
98
|
+
- Missing Playwright tests to add: prioritized, concrete, and automatable.
|
|
99
|
+
- Maintenance notes: selector scoping, fixture cleanup, flaky/stale-state risks.
|
|
100
|
+
|
|
101
|
+
## Quality Bar
|
|
102
|
+
|
|
103
|
+
- Be honest about what was and was not tested.
|
|
104
|
+
- Distinguish user-visible bugs from missing automated coverage.
|
|
105
|
+
- Prefer concrete examples over vague recommendations.
|
|
106
|
+
- Do not claim cleanup succeeded unless verified.
|
|
107
|
+
- Do not let broad locators pass against hidden/inactive content.
|
|
108
|
+
- Preserve unrelated repo changes and keep report edits scoped.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lisa-typescript",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.26.1",
|
|
4
4
|
"description": "TypeScript-specific hooks for formatting, linting, and ast-grep scanning on edit.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Cody Swann"
|
|
@@ -14,7 +14,6 @@
|
|
|
14
14
|
"dependencies": [
|
|
15
15
|
"lisa"
|
|
16
16
|
],
|
|
17
|
-
"hooks": "./hooks/hooks.json",
|
|
18
17
|
"interface": {
|
|
19
18
|
"displayName": "Lisa TypeScript",
|
|
20
19
|
"shortDescription": "TypeScript lifecycle checks",
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Run a Playwright-backed exploratory QA pass: audit the app like a human, find user-noticeable bugs and gaps in automated test coverage, and produce a QA gaps report."
|
|
3
|
+
allowed-tools: ["Skill"]
|
|
4
|
+
argument-hint: "[target-url | env] [report-path]"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Use the /lisa-expo:exploratory-qa skill to run a human-style exploratory QA pass informed by the existing Playwright suite, then produce an actionable coverage-gaps report. $ARGUMENTS
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: exploratory-qa
|
|
3
|
+
description: Playwright-backed exploratory QA workflow for web apps. Use when asked to audit an app or project with Playwright/e2e tests, find human-noticeable bugs, identify gaps in automated test coverage, test responsive breakpoints, observe slow or unclear load states, exercise mutable workflows with cleanup, or produce a QA gaps report.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Exploratory QA
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
Run a human-style exploratory QA pass informed by the existing Playwright suite. The goal is to find issues users notice and machines often miss, then translate those observations into actionable Playwright coverage gaps.
|
|
11
|
+
|
|
12
|
+
## Core Workflow
|
|
13
|
+
|
|
14
|
+
### 1. Establish Scope
|
|
15
|
+
|
|
16
|
+
- Identify the target environment, account type, browser requirement, and requested report path.
|
|
17
|
+
- If credentials, tenant, seed data, or mutation boundaries are missing and cannot be discovered safely, ask a concise clarifying question.
|
|
18
|
+
- Treat production-like environments conservatively. Do not mutate production data unless the user explicitly approves it.
|
|
19
|
+
- Prefer a test user, dev/staging environment, or isolated seeded account for mutation testing.
|
|
20
|
+
|
|
21
|
+
### 2. Research Existing Playwright Coverage
|
|
22
|
+
|
|
23
|
+
- Inspect Playwright config, auth/setup projects, fixtures, constants, selectors, test directories, skipped tests, retries, and viewport/device projects.
|
|
24
|
+
- Summarize:
|
|
25
|
+
- Number and shape of specs/tests.
|
|
26
|
+
- Major workflows covered well.
|
|
27
|
+
- Skipped/flaky/TODO coverage.
|
|
28
|
+
- Viewports and browser projects used.
|
|
29
|
+
- Mutating tests and their cleanup strategy.
|
|
30
|
+
- Areas where tests assert presence instead of behavior.
|
|
31
|
+
- Use existing test helpers/selectors when exploring the app. They reveal intended flows and stable hooks.
|
|
32
|
+
|
|
33
|
+
### 3. Choose Browser Tools
|
|
34
|
+
|
|
35
|
+
- Use the user's requested browser/tool when specified.
|
|
36
|
+
- Use a real profile browser when the task depends on existing cookies, SSO, extensions, or a logged-in session.
|
|
37
|
+
- Use Playwright automation when exact viewport sizing, repeatable navigation, screenshots, console logs, traces, or route timing are needed.
|
|
38
|
+
- Do not let automation hide human observations. Screenshots, visible text, layout, latency, scrollability, and affordance clarity matter.
|
|
39
|
+
|
|
40
|
+
### 4. Explore Like A Human
|
|
41
|
+
|
|
42
|
+
Cover at least these dimensions unless the user narrows scope:
|
|
43
|
+
|
|
44
|
+
- **Navigation:** direct URLs and visible click paths.
|
|
45
|
+
- **Responsive behavior:** supported breakpoints, plus boundary widths where layout changes.
|
|
46
|
+
- **Visual/layout quality:** cutoff text, overlap, offscreen controls, accidental horizontal scroll, empty space, awkward density, hidden stale content.
|
|
47
|
+
- **Load states:** blanks, skeletons, spinners, `Loading...`, `Connecting...`, delayed hydration, and whether delays exceed a human-acceptable threshold.
|
|
48
|
+
- **Behavior correctness:** sorting, filtering, saving, deleting, disabled states, permissions, tab persistence, URL canonicalization.
|
|
49
|
+
- **Accessibility/testability:** hidden or off-canvas content exposed to accessibility/text locators, duplicate inactive route content, zero-sized interactive elements.
|
|
50
|
+
- **Console/network health:** errors, missing assets, failed requests, noisy expected errors that could bury real failures.
|
|
51
|
+
- **Data hygiene:** repeated test artifacts, stale seeded content, polluted shared accounts.
|
|
52
|
+
|
|
53
|
+
## Mutation Discipline
|
|
54
|
+
|
|
55
|
+
Exploratory QA should exercise mutable workflows when the environment is safe.
|
|
56
|
+
|
|
57
|
+
- Prefer high-value user workflows: create/edit/delete records, lists, boards, tags, notes, comments, scenarios, uploads, messages, settings, invitations, or assignments.
|
|
58
|
+
- Use unique names with a clear prefix such as `qa-`, `pw-`, or `codex-`.
|
|
59
|
+
- Before mutating, identify the cleanup path. After mutating, make a best effort to clean up through the UI, then verify cleanup.
|
|
60
|
+
- If UI cleanup is unavailable, record that as a product/test gap. Use documented API cleanup only if appropriate for the project and account.
|
|
61
|
+
- Record all mutations performed, cleanup attempts, and residue left behind.
|
|
62
|
+
- Avoid destructive bulk actions unless the user explicitly asks or the test account is clearly disposable.
|
|
63
|
+
|
|
64
|
+
## Breakpoint Strategy
|
|
65
|
+
|
|
66
|
+
- Discover breakpoints from the codebase, design tokens, CSS, docs, or Playwright constants when possible.
|
|
67
|
+
- Test each named breakpoint and the boundary on both sides of important cutoffs.
|
|
68
|
+
- If breakpoints are unknown, use a practical baseline: phone, tablet/narrow, desktop, and any app-specific cutoff discovered during research.
|
|
69
|
+
- For each breakpoint, assert both user-visible behavior and automation-relevant state:
|
|
70
|
+
- Expected shell/navigation variant.
|
|
71
|
+
- Route-specific loaded content.
|
|
72
|
+
- Critical controls visible and reachable.
|
|
73
|
+
- No unintended document-level horizontal overflow.
|
|
74
|
+
- Intentional scroll containers remain usable.
|
|
75
|
+
- Hidden/off-canvas UI is not exposed as active content.
|
|
76
|
+
|
|
77
|
+
## Load-State Strategy
|
|
78
|
+
|
|
79
|
+
Measure perceived latency, not just eventual success.
|
|
80
|
+
|
|
81
|
+
- Track time to shell, time to meaningful route content, and time spent in blank/loading/spinner/connecting states.
|
|
82
|
+
- Note when a page is technically interactive but still visually incomplete.
|
|
83
|
+
- Treat long waits without clear progress, error, retry, or cancellation as bugs or test gaps.
|
|
84
|
+
- Do not overfit exact milliseconds unless the project has defined budgets. Use practical labels such as noticeable, slow, or unacceptable and include observed durations when available.
|
|
85
|
+
|
|
86
|
+
## Report Structure
|
|
87
|
+
|
|
88
|
+
Write the report to the user's requested path. If none is specified, use a clear local name such as `PLAYWRIGHT_GAPS.md` or `EXPLORATORY_QA_REPORT.md`.
|
|
89
|
+
|
|
90
|
+
Include:
|
|
91
|
+
|
|
92
|
+
- Scope: target URL/env, browser/tool, account type, build/version if visible, date.
|
|
93
|
+
- Existing Playwright coverage: strengths, thin areas, skipped tests, viewport/browser matrix, mutation coverage.
|
|
94
|
+
- Exploratory findings: steps, observed behavior, impact, why existing tests miss it, recommended test.
|
|
95
|
+
- Breakpoint findings: exact viewports tested and boundary behavior.
|
|
96
|
+
- Load-state findings: observed delays, blank/spinner/connecting states, suggested budgets/tests.
|
|
97
|
+
- Mutation findings: data created, behavior observed, cleanup attempt, cleanup result, residue.
|
|
98
|
+
- Missing Playwright tests to add: prioritized, concrete, and automatable.
|
|
99
|
+
- Maintenance notes: selector scoping, fixture cleanup, flaky/stale-state risks.
|
|
100
|
+
|
|
101
|
+
## Quality Bar
|
|
102
|
+
|
|
103
|
+
- Be honest about what was and was not tested.
|
|
104
|
+
- Distinguish user-visible bugs from missing automated coverage.
|
|
105
|
+
- Prefer concrete examples over vague recommendations.
|
|
106
|
+
- Do not claim cleanup succeeded unless verified.
|
|
107
|
+
- Do not let broad locators pass against hidden/inactive content.
|
|
108
|
+
- Preserve unrelated repo changes and keep report edits scoped.
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Run a Playwright-backed exploratory QA pass: audit the app like a human, find user-noticeable bugs and gaps in automated test coverage, and produce a QA gaps report."
|
|
3
|
+
allowed-tools: ["Skill"]
|
|
4
|
+
argument-hint: "[target-url | env] [report-path]"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Use the /lisa-harper-fabric:exploratory-qa skill to run a human-style exploratory QA pass informed by the existing Playwright suite, then produce an actionable coverage-gaps report. $ARGUMENTS
|