qualiow-exploratory-testing 1.3.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +26 -0
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +259 -0
- package/README.md +472 -204
- package/agents/qa-diff-indexer-agent.md +81 -0
- package/agents/qa-gather-agent.md +41 -0
- package/agents/qa-page-mapper-agent.md +74 -0
- package/agents/qa-reporting-agent.md +103 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/bin/qualiow +30 -0
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/knowledge/changelog.yml +30 -12
- package/data/knowledge/manifest.yml +355 -127
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +3 -3
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +9 -6
- package/data/templates/expected-behaviour.md +3 -2
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +2286 -701
- package/dist/index.d.ts +637 -81
- package/dist/index.js +2058 -228
- package/docs/BACKEND-VERIFICATION.md +39 -9
- package/docs/GETTING-STARTED.md +300 -121
- package/docs/MOBILE-SETUP.md +149 -18
- package/hooks/hooks.json +27 -0
- package/hooks/scripts/read-guard.mjs +95 -0
- package/hooks/scripts/secret-patterns.d.mts +8 -0
- package/hooks/scripts/secret-patterns.mjs +136 -0
- package/hooks/scripts/write-guard.mjs +87 -0
- package/package.json +33 -18
- package/skills/qa-explore/SKILL.md +32 -25
- package/skills/qa-explore/phases/00-setup.md +76 -42
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +93 -27
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +102 -142
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/delegation-rules.md +82 -0
- package/skills/qa-explore/references/output-contract.md +192 -0
- package/skills/qa-explore/references/paths.md +108 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +70 -25
- package/skills/qa-explore/references/session-rules.md +2 -1
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +51 -15
- package/skills/qa-explore-feedback/SKILL.md +23 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +108 -60
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +71 -26
- package/skills/qa-explore-report/SKILL.md +65 -35
- package/skills/qa-gather/SKILL.md +44 -8
- package/skills/qa-knowledge-add/SKILL.md +25 -9
- package/skills/qa-knowledge-list/SKILL.md +37 -40
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +31 -19
- package/skills/qa-verify-backend/phases/00-setup.md +57 -12
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
- package/skills/qa-verify-backend/phases/02-static-review.md +11 -0
- package/skills/qa-verify-backend/phases/03-live-verification.md +5 -3
- package/skills/qa-verify-backend/phases/03b-api-verification.md +24 -9
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
- package/skills/qa-verify-backend/phases/05-reporting.md +72 -12
- package/skills/qa-verify-backend/references/api-probes.md +28 -1
- package/skills/qa-verify-backend/references/payload-verification.md +1 -1
- package/skills/qa-verify-backend/references/release-readiness.md +1 -1
- package/skills/qa-verify-backend/references/safety-rules.md +13 -8
- package/data/domains/_default.md +0 -181
- package/data/domains/ecommerce.md +0 -192
- package/data/domains/fintech.md +0 -200
- package/data/domains/identity.md +0 -88
- package/data/domains/marketing.md +0 -194
- package/data/domains/saas.md +0 -197
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
package/package.json
CHANGED
|
@@ -1,65 +1,80 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "qualiow-exploratory-testing",
|
|
3
|
-
"version": "
|
|
4
|
-
"description": "qualiow — AI-powered exploratory testing skills
|
|
3
|
+
"version": "2.2.0",
|
|
4
|
+
"description": "qualiow — AI-powered exploratory testing skills for Claude Code: web via playwright-cli, iOS/Android simulators via Maestro/mobile-cli, and backend/API acceptance-criteria verification",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
7
7
|
"types": "dist/index.d.ts",
|
|
8
8
|
"bin": {
|
|
9
|
-
"qualiow": "dist/cli/index.js"
|
|
9
|
+
"qualiow": "dist/cli/index.js",
|
|
10
|
+
"mcli": "bin/mcli",
|
|
11
|
+
"wadb": "bin/wadb",
|
|
12
|
+
"wk-ios": "bin/wk-ios",
|
|
13
|
+
"qualiow-doctor-mobile": "bin/doctor-mobile.sh",
|
|
14
|
+
"qualiow-setup-mobile": "bin/setup-mobile.sh"
|
|
10
15
|
},
|
|
11
16
|
"files": [
|
|
12
17
|
"dist/",
|
|
13
18
|
"skills/",
|
|
19
|
+
"agents/",
|
|
14
20
|
"bin/",
|
|
15
|
-
"
|
|
16
|
-
"
|
|
17
|
-
"data/domains
|
|
21
|
+
"hooks/",
|
|
22
|
+
".claude-plugin/",
|
|
23
|
+
"data/domains/*.yml",
|
|
18
24
|
"data/knowledge/",
|
|
19
25
|
"data/templates/",
|
|
20
|
-
"data/
|
|
21
|
-
"data/targets/
|
|
22
|
-
"data/targets/_example-sim-ios-safari.yml",
|
|
23
|
-
"data/targets/_example-sim-android-chrome.yml",
|
|
24
|
-
"data/targets/_example-mobile-emulation.yml",
|
|
25
|
-
"data/targets/_example-backend.yml",
|
|
26
|
+
"data/security/",
|
|
27
|
+
"data/targets/_*.yml",
|
|
26
28
|
"data/targets/testers-ai.yml",
|
|
27
29
|
"docs/GETTING-STARTED.md",
|
|
28
30
|
"docs/MOBILE-SETUP.md",
|
|
29
|
-
"docs/BACKEND-VERIFICATION.md"
|
|
31
|
+
"docs/BACKEND-VERIFICATION.md",
|
|
32
|
+
".env.example",
|
|
33
|
+
"CHANGELOG.md"
|
|
30
34
|
],
|
|
31
35
|
"scripts": {
|
|
32
|
-
"sync:
|
|
33
|
-
"prebuild": "npm run sync:
|
|
36
|
+
"sync:plugin": "rm -rf skills agents && cp -R .claude/skills skills && cp -R .claude/agents agents",
|
|
37
|
+
"prebuild": "npm run sync:plugin",
|
|
34
38
|
"build": "tsup",
|
|
35
39
|
"dev": "tsup --watch",
|
|
36
40
|
"test": "vitest run",
|
|
37
41
|
"test:watch": "vitest",
|
|
38
42
|
"test:coverage": "vitest run --coverage",
|
|
39
43
|
"validate": "node dist/cli/index.js validate --all",
|
|
44
|
+
"kb:sync": "node dist/cli/index.js kb sync",
|
|
45
|
+
"kb:check": "node dist/cli/index.js kb check",
|
|
46
|
+
"check:mirror": "diff -rq .claude/skills skills && diff -rq .claude/agents agents",
|
|
47
|
+
"check:pack": "node scripts/check-pack.mjs",
|
|
40
48
|
"lint": "tsc --noEmit",
|
|
41
|
-
"
|
|
49
|
+
"version": "node scripts/sync-version.mjs && git add .claude-plugin/plugin.json .claude-plugin/marketplace.json",
|
|
50
|
+
"prepublishOnly": "npm run build && npm run lint && npm test && npm run check:pack"
|
|
42
51
|
},
|
|
43
52
|
"keywords": [
|
|
44
53
|
"exploratory-testing",
|
|
45
54
|
"qa",
|
|
46
55
|
"testing",
|
|
47
56
|
"playwright",
|
|
57
|
+
"playwright-cli",
|
|
48
58
|
"maestro",
|
|
49
59
|
"mobile-testing",
|
|
50
60
|
"ios-simulator",
|
|
51
61
|
"android-emulator",
|
|
62
|
+
"api-testing",
|
|
63
|
+
"acceptance-criteria",
|
|
52
64
|
"ai",
|
|
53
65
|
"claude",
|
|
66
|
+
"claude-code",
|
|
54
67
|
"skills",
|
|
68
|
+
"plugin",
|
|
55
69
|
"heuristics",
|
|
56
70
|
"bug-detection"
|
|
57
71
|
],
|
|
58
72
|
"author": "William Oliveira",
|
|
59
73
|
"license": "MIT",
|
|
60
74
|
"engines": {
|
|
61
|
-
"node": ">=
|
|
75
|
+
"node": ">=22.4.0"
|
|
62
76
|
},
|
|
77
|
+
"packageManager": "npm@10.9.2",
|
|
63
78
|
"repository": {
|
|
64
79
|
"type": "git",
|
|
65
80
|
"url": "git+https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git"
|
|
@@ -69,7 +84,7 @@
|
|
|
69
84
|
"url": "https://github.com/willcoliveira/qualiow-exploratory-testing-skills/issues"
|
|
70
85
|
},
|
|
71
86
|
"dependencies": {
|
|
72
|
-
"@playwright/cli": "^0.1.
|
|
87
|
+
"@playwright/cli": "^0.1.19",
|
|
73
88
|
"chalk": "^5.3.0",
|
|
74
89
|
"commander": "^15.0.0",
|
|
75
90
|
"glob": "^13.0.6",
|
|
@@ -7,10 +7,11 @@ description: >
|
|
|
7
7
|
and writes bug reports that executives act on.
|
|
8
8
|
Use when user says: "explore", "test this site", "find bugs", "QA check", "exploratory session",
|
|
9
9
|
or provides a URL to test.
|
|
10
|
-
|
|
10
|
+
argument-hint: "<url> [--target <id>] [--context <file>] [--focus <area>] [--session <dir>]"
|
|
11
|
+
allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Bash(qualiow:*), Bash(npx:*), Bash(wc:*), Bash(diff:*), Read, Write, Glob, Grep
|
|
11
12
|
---
|
|
12
13
|
|
|
13
|
-
# Exploratory Testing Session
|
|
14
|
+
# Exploratory Testing Session
|
|
14
15
|
|
|
15
16
|
## Your Identity
|
|
16
17
|
|
|
@@ -29,43 +30,49 @@ You are a **Principal QA Engineer** with 20+ years of experience. You don't just
|
|
|
29
30
|
# Mode 1: Blind exploration (default — always works, no prior info needed)
|
|
30
31
|
/qa-explore https://example.com
|
|
31
32
|
|
|
32
|
-
# Mode 2: Context-enriched (
|
|
33
|
-
/qa-explore https://example.com --context
|
|
33
|
+
# Mode 2: Context-enriched (a /qa-gather context file or any requirements document)
|
|
34
|
+
/qa-explore https://example.com --context output/context/checkout-context.md
|
|
34
35
|
|
|
35
36
|
# Mode 3: Inline context (user provides context in their message)
|
|
36
37
|
/qa-explore https://example.com
|
|
37
38
|
> "This is a new checkout flow. Should support Visa/MC, require delivery address,
|
|
38
39
|
> send confirmation email. Team just refactored the payment service."
|
|
39
40
|
|
|
40
|
-
# With target config
|
|
41
|
-
/qa-explore --target company-staging --context
|
|
41
|
+
# With a saved target config, or reusing a `qualiow explore` pre-flight directory
|
|
42
|
+
/qa-explore --target company-staging --context output/context/SI-2305-context.md
|
|
43
|
+
/qa-explore https://example.com --session output/sessions/2026-09-08-1420-explore-example-com
|
|
42
44
|
```
|
|
43
45
|
|
|
44
|
-
|
|
46
|
+
## Before You Start
|
|
45
47
|
|
|
46
|
-
|
|
48
|
+
- Paths, target resolution, the session-directory scheme and the `-s=<sid>` session id are defined once in **`${CLAUDE_SKILL_DIR}/references/paths.md`**. Read it first.
|
|
49
|
+
- Every artefact you write follows **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** (bug report, session report, `stats.json`, confidentiality header). `qualiow report` and the feedback skills only understand that format.
|
|
50
|
+
- **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** is absolute: production detection, redaction before disk, session isolation, output classification.
|
|
47
51
|
|
|
48
|
-
|
|
52
|
+
## Session Phases
|
|
49
53
|
|
|
50
|
-
|
|
51
|
-
|-------|------|---------|
|
|
52
|
-
| **Setup** | Read and follow: `phases/00-setup.md` | Parse input, resolve target, load context, load knowledge, create session directory |
|
|
53
|
-
| **Auth** | Read and follow: `phases/01-auth.md` | Authenticate via storage state, credentials, or token; verify login |
|
|
54
|
-
| **Charter** | Read and follow: `phases/02-charter.md` | Use app as real user, identify journeys, risk-rank features, select heuristics, write charter |
|
|
55
|
-
| **Discovery** | Read and follow: `phases/03-discovery.md` | Map the site, check console/network, enforce scope, find what's MISSING |
|
|
56
|
-
| **Journeys** | Read and follow: `phases/04-journeys.md` | End-to-end user journeys, data integrity verification, cross-page consistency |
|
|
57
|
-
| **Features** | Read and follow: `phases/05-features.md` | Deep feature testing with SFDIPOT, business logic stress, negative space, FEW HICCUPPS |
|
|
58
|
-
| **Edge Cases** | Read and follow: `phases/06-edge-cases.md` | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
|
|
59
|
-
| **Reporting** | Read and follow: `phases/07-reporting.md` | Stop recording, reflect, write bug reports with business impact, session report, coverage map |
|
|
54
|
+
Execute each phase in order. Read and follow the linked file. The budget sums to 45 minutes; write to disk after each phase and carry only summaries in context.
|
|
60
55
|
|
|
61
|
-
|
|
56
|
+
| Phase | File | Budget | Summary |
|
|
57
|
+
|-------|------|--------|---------|
|
|
58
|
+
| **Setup** | `${CLAUDE_SKILL_DIR}/phases/00-setup.md` | 2 min | Parse input, resolve target and data paths, load context and knowledge, create the session directory |
|
|
59
|
+
| **Auth** | `${CLAUDE_SKILL_DIR}/phases/01-auth.md` | 2 min | Authenticate via storage state, credentials, or token; verify login |
|
|
60
|
+
| **Charter** | `${CLAUDE_SKILL_DIR}/phases/02-charter.md` | 5 min | Use the app as a real user, identify journeys, risk-rank features, select heuristics, write the charter |
|
|
61
|
+
| **Discovery** | `${CLAUDE_SKILL_DIR}/phases/03-discovery.md` | 6 min | Map the site, check console and requests, enforce scope, find what's MISSING |
|
|
62
|
+
| **Journeys** | `${CLAUDE_SKILL_DIR}/phases/04-journeys.md` | 10 min | End-to-end user journeys, data-integrity verification, cross-page consistency |
|
|
63
|
+
| **Features** | `${CLAUDE_SKILL_DIR}/phases/05-features.md` | 10 min | Deep feature testing with SFDIPOT, business-logic stress, negative space, FEW HICCUPPS |
|
|
64
|
+
| **Edge Cases** | `${CLAUDE_SKILL_DIR}/phases/06-edge-cases.md` | 6 min | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
|
|
65
|
+
| **Reporting** | `${CLAUDE_SKILL_DIR}/phases/07-reporting.md` | 4 min | Stop recording, reflect, write bug reports with business impact, session report, stats, close the browser session |
|
|
62
66
|
|
|
63
67
|
## References
|
|
64
68
|
|
|
65
69
|
Consult these as needed throughout the session:
|
|
66
70
|
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
-
|
|
71
|
-
-
|
|
71
|
+
- **`${CLAUDE_SKILL_DIR}/references/paths.md`** — resolution order for targets, data, credentials and output; session-directory scheme; session id; index rows
|
|
72
|
+
- **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** — the one bug-report and session-report format, `stats.json`, confidentiality header
|
|
73
|
+
- **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** — prompt-injection resistance, production rule, redaction list, session isolation, output classification
|
|
74
|
+
- **`${CLAUDE_SKILL_DIR}/references/severity-guide.md`** — severity definitions, the "when in doubt go LOWER" rule, risk matrix, priority table
|
|
75
|
+
- **`${CLAUDE_SKILL_DIR}/references/session-rules.md`** — 20 testing-discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI-bias avoidance
|
|
76
|
+
- **`${CLAUDE_SKILL_DIR}/references/delegation-rules.md`** — what the `qualiow` CLI does instead of you, the never-delegate list (severity, impact, bug reports, verdicts, reflection) and the read thresholds
|
|
77
|
+
- **`${CLAUDE_SKILL_DIR}/references/context-integration.md`** — how `--context` changes the session (context file, inline context, blind mode)
|
|
78
|
+
- **`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md`** — opt-in handoff to Playwright Test Agents (planner / generator / healer, Playwright 1.56+) when a reproducible bug should become a regression test; requires the optional `@playwright/test` peer dependency
|
|
@@ -1,44 +1,53 @@
|
|
|
1
|
-
# Phase 0: Session Setup
|
|
1
|
+
# Phase 0: Session Setup (2 min)
|
|
2
2
|
|
|
3
3
|
## Step 1: Parse Input & Resolve Target
|
|
4
4
|
|
|
5
|
-
Extract from user message:
|
|
6
|
-
- **target_url**:
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
5
|
+
Extract from the user message:
|
|
6
|
+
- **target_url**: the URL to test (required unless `--target` is given)
|
|
7
|
+
- **--target <id>**: a saved target config (optional)
|
|
8
|
+
- **--context <file>** or inline context text (optional)
|
|
9
|
+
- **--focus <area>**: a specific area or feature (optional)
|
|
10
|
+
- **--session <dir>**: an existing pre-flight directory created by `qualiow explore` (optional)
|
|
11
|
+
- **domain**: domain type (optional; auto-detect from the app if not given)
|
|
12
|
+
- **time_box**: 45 min (default and maximum)
|
|
13
|
+
|
|
14
|
+
Resolve the target config, the data directory (`<data>`) and the credentials file in the
|
|
15
|
+
order defined in `${CLAUDE_SKILL_DIR}/references/paths.md`: `--target` → `data/targets/<id>.yml`
|
|
16
|
+
in the project, else the plugin's copy; no `--target` → `qa/target.yml` in the project, else
|
|
17
|
+
`_default.yml`; credentials from `qa/.env`, else `.env`. Then read
|
|
18
|
+
`<data>/domains/<domain>.yml` for the domain's completeness checklist, data-integrity checks
|
|
19
|
+
and journeys.
|
|
20
|
+
|
|
21
|
+
Apply the **production rule** from `${CLAUDE_SKILL_DIR}/references/security-rules.md` to the
|
|
22
|
+
resolved hostname now. If it fires, log `[SAFETY] Production detected -- running in read-only mode`
|
|
23
|
+
and carry read-only mode through every later phase.
|
|
20
24
|
|
|
21
25
|
## Step 2: Load Context (if provided)
|
|
22
26
|
|
|
23
|
-
**If `--context <file>` provided:**
|
|
27
|
+
**If `--context <file>` provided:** read the file. It may be a `/qa-gather` context file
|
|
28
|
+
(structured markdown with ticket data, specs, MR diffs, design notes) or any document
|
|
29
|
+
describing what to test.
|
|
24
30
|
|
|
25
|
-
**If inline context provided:**
|
|
31
|
+
**If inline context provided:** extract from the user's message any descriptions of: what
|
|
32
|
+
the feature does, acceptance criteria, what changed, what the design should look like,
|
|
33
|
+
known risks, user personas.
|
|
26
34
|
|
|
27
|
-
**If no context provided:**
|
|
35
|
+
**If no context provided:** that's fine; run in blind exploration mode.
|
|
28
36
|
|
|
29
37
|
**When context IS available, extract and use:**
|
|
30
38
|
|
|
31
39
|
| Context Element | How It Changes the Session |
|
|
32
40
|
|----------------|---------------------------|
|
|
33
|
-
| **Acceptance criteria** | Add as explicit checkpoints
|
|
34
|
-
| **What changed** (MR diff, commits) | Focus testing on changed areas
|
|
35
|
-
| **Design specs** (Figma, mockups) | Compare live app against intended design
|
|
41
|
+
| **Acceptance criteria** | Add as explicit checkpoints; verify each AC during testing |
|
|
42
|
+
| **What changed** (MR diff, commits) | Focus testing on changed areas; highest regression risk |
|
|
43
|
+
| **Design specs** (Figma, mockups) | Compare the live app against the intended design |
|
|
36
44
|
| **User stories / requirements** | Test against INTENT, not just what's visible |
|
|
37
|
-
| **Known risks**
|
|
38
|
-
| **API contracts** (OpenAPI, endpoints) | Verify UI matches API
|
|
39
|
-
| **Previous test results** | Don't re-test what passed
|
|
45
|
+
| **Known risks** | Prioritize testing on flagged risk areas |
|
|
46
|
+
| **API contracts** (OpenAPI, endpoints) | Verify the UI matches API behaviour |
|
|
47
|
+
| **Previous test results** | Don't re-test what passed; focus on gaps |
|
|
40
48
|
|
|
41
|
-
**Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
|
|
49
|
+
**Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
|
|
50
|
+
with zero documentation should still get a full, valuable session.
|
|
42
51
|
|
|
43
52
|
When context is loaded, note in the charter:
|
|
44
53
|
```
|
|
@@ -49,17 +58,32 @@ CHANGE SCOPE: [what changed or "unknown -- full exploration"]
|
|
|
49
58
|
|
|
50
59
|
## Step 3: Load Knowledge
|
|
51
60
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
61
|
+
Run the digest — it selects the always-loaded heuristics, the domain entries, the tagged
|
|
62
|
+
techniques and the learned-pattern lead-ins, and prints a summary of each:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
qualiow kb digest --for explore --domain <domain>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Add `--tag security` when the target has authentication, and `--tag data-integrity` when the
|
|
69
|
+
app has transactions or any state change. Resolve the `qualiow` prefix first (`${CLAUDE_SKILL_DIR}/references/paths.md`)
|
|
70
|
+
and write it literally.
|
|
71
|
+
|
|
72
|
+
Read **only the digest**. When you need a heuristic's full question list, fetch that one entry
|
|
73
|
+
— `qualiow list knowledge --entry <id>`, or `Grep` the release file for the heading. Never
|
|
74
|
+
`Read` `<data>/knowledge/manifest.yml` or a release entry whole: see
|
|
75
|
+
`${CLAUDE_SKILL_DIR}/references/delegation-rules.md`.
|
|
57
76
|
|
|
58
|
-
|
|
77
|
+
## Step 4: Create the Session Directory
|
|
59
78
|
|
|
60
|
-
|
|
79
|
+
Unless `--session <dir>` was given, create
|
|
80
|
+
`output/sessions/<YYYY-MM-DD-HHmm>-explore-<slug>/` (scheme in `paths.md`; `<slug>` is the
|
|
81
|
+
target id without a leading `_`, or the hostname with dots replaced by dashes) containing
|
|
82
|
+
`screenshots/`, `bugs/`, `videos/`, `snapshots/`, and an empty `session-log.md`. With
|
|
83
|
+
`--session`, reuse that directory and append to its `session-log.md`.
|
|
61
84
|
|
|
62
|
-
|
|
85
|
+
Choose the playwright-cli session id `-s=explore-<HHmm>-<slug>`. **Every `playwright-cli`
|
|
86
|
+
call in this session carries `-s=<sid>`; the examples in the phase files omit the prefix.**
|
|
63
87
|
|
|
64
88
|
## Step 5: Initialize Progress Tracking
|
|
65
89
|
|
|
@@ -67,6 +91,7 @@ Write `output/sessions/<session-dir>/progress.json`:
|
|
|
67
91
|
```json
|
|
68
92
|
{
|
|
69
93
|
"session_id": "<session-dir>",
|
|
94
|
+
"kind": "explore",
|
|
70
95
|
"target": "<target_url>",
|
|
71
96
|
"domain": "<domain>",
|
|
72
97
|
"started_at": "<ISO timestamp>",
|
|
@@ -91,27 +116,36 @@ Write `output/sessions/<session-dir>/progress.json`:
|
|
|
91
116
|
|
|
92
117
|
Append to `session-log.md`:
|
|
93
118
|
```
|
|
94
|
-
[<timestamp>] [PHASE] Setup complete — session
|
|
119
|
+
[<timestamp>] [PHASE] Setup complete — session <session-dir>, sid <sid>, read_only: <true|false>
|
|
95
120
|
```
|
|
96
121
|
|
|
97
122
|
## Step 6: Optional — Bootstrap Playwright Test Agents (opt-in)
|
|
98
123
|
|
|
99
|
-
**Skip this step entirely if `@playwright/test` is not installed
|
|
124
|
+
**Skip this step entirely if `@playwright/test` is not installed in the consumer project OR
|
|
125
|
+
no `playwright.config.*` is present.** This is an additive capability for teams that want to
|
|
126
|
+
turn reproducible findings into regression tests with the Playwright Test Agents framework
|
|
127
|
+
(planner / generator / healer, Playwright 1.56+).
|
|
100
128
|
|
|
101
|
-
If both prerequisites are present AND the user has opted in (via target config or
|
|
129
|
+
If both prerequisites are present AND the user has opted in (via the target config or the
|
|
130
|
+
context), you MAY offer to scaffold the agent workspace:
|
|
102
131
|
|
|
103
132
|
```bash
|
|
104
133
|
npx playwright init-agents --loop=claude
|
|
105
134
|
```
|
|
106
135
|
|
|
107
|
-
|
|
136
|
+
With `--loop=claude` this writes the three agent definitions into `.claude/agents/`, a
|
|
137
|
+
`.mcp.json` registering the `playwright-test` server, and a `specs/` scaffold with a seed
|
|
138
|
+
spec if the project has none; other loops: `codex`, `copilot`, `opencode`, `vscode`,
|
|
139
|
+
`vscode-legacy`. It does not touch `playwright.config.*`. See
|
|
140
|
+
`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md` for the hand-off after the
|
|
141
|
+
session.
|
|
108
142
|
|
|
109
|
-
**Never block the session on this step.** If the user declines,
|
|
143
|
+
**Never block the session on this step.** If the user declines, the prerequisites are
|
|
144
|
+
missing, or the command fails, log a note in `session-log.md` and continue.
|
|
110
145
|
|
|
111
|
-
**IMPORTANT: After EVERY phase completion, update progress.json and append to
|
|
146
|
+
**IMPORTANT: After EVERY phase completion, update `progress.json` and append to
|
|
147
|
+
`session-log.md`.** This is how the user monitors progress:
|
|
112
148
|
|
|
113
149
|
```
|
|
114
|
-
# Update progress.json — set current phase complete, next phase as current
|
|
115
|
-
# Append to session-log.md:
|
|
116
150
|
[<timestamp>] [PHASE] <phase_name> complete — <pages> pages, <bugs> bugs, <key finding summary>
|
|
117
151
|
```
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Phase 1: Authentication
|
|
1
|
+
# Phase 1: Authentication (2 min)
|
|
2
2
|
|
|
3
|
-
Skip if `auth.strategy: none`.
|
|
3
|
+
Skip if `auth.strategy: none`. Every command below carries the session id `-s=<sid>` (omitted here).
|
|
4
4
|
|
|
5
5
|
## Storage State (fastest)
|
|
6
6
|
|
|
@@ -9,30 +9,47 @@ playwright-cli state-load .auth/<target>.json
|
|
|
9
9
|
playwright-cli goto <base_url>
|
|
10
10
|
```
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
If the state file is older than 24 hours, expect it to have expired and fall through to
|
|
13
|
+
credentials when the login page appears.
|
|
13
14
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
15
|
+
## Credentials (adaptive: uses snapshot to find form fields)
|
|
16
|
+
|
|
17
|
+
Resolve the values named by `auth.credentials.username` / `auth.credentials.password` in the
|
|
18
|
+
order from `${CLAUDE_SKILL_DIR}/references/paths.md` (`qa/.env`, then `.env`). Read the value,
|
|
19
|
+
pass it to the command, and never echo it into the log or the report.
|
|
17
20
|
|
|
18
21
|
```bash
|
|
19
22
|
playwright-cli open <login_url>
|
|
20
23
|
playwright-cli snapshot
|
|
21
|
-
playwright-cli fill <email_ref> "
|
|
22
|
-
playwright-cli fill <pass_ref> "
|
|
24
|
+
playwright-cli fill <email_ref> "<username value>"
|
|
25
|
+
playwright-cli fill <pass_ref> "<password value>"
|
|
23
26
|
playwright-cli click <submit_ref>
|
|
24
27
|
playwright-cli state-save .auth/<target>.json
|
|
25
28
|
```
|
|
26
29
|
|
|
30
|
+
## Token (`auth.strategy: token`)
|
|
31
|
+
|
|
32
|
+
Inject the token named by `auth.token` where the app expects it, then reload:
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
playwright-cli open <base_url>
|
|
36
|
+
playwright-cli cookie-set <cookie-name> "<token value>" --domain=<host> --httpOnly --secure # cookie-based
|
|
37
|
+
playwright-cli localstorage-set <key> "<token value>" # localStorage-based
|
|
38
|
+
playwright-cli reload
|
|
39
|
+
```
|
|
40
|
+
|
|
27
41
|
## Verify Login
|
|
28
42
|
|
|
29
|
-
After authentication,
|
|
43
|
+
After authentication, `playwright-cli snapshot` and confirm user indicators are present
|
|
44
|
+
(username display, avatar, dashboard access).
|
|
30
45
|
|
|
31
46
|
If login fails:
|
|
32
|
-
1. Check console
|
|
33
|
-
2.
|
|
34
|
-
3.
|
|
35
|
-
4.
|
|
36
|
-
|
|
37
|
-
|
|
47
|
+
1. Check the console: `playwright-cli console error`
|
|
48
|
+
2. Check the last requests: `playwright-cli requests --filter="login|auth|session"`
|
|
49
|
+
3. Verify the credentials are the ones the target names
|
|
50
|
+
4. Check for MFA. Never automate an MFA challenge; hand it to the user in a headed browser
|
|
51
|
+
(`playwright-cli open <login_url> --headed`) and continue once they confirm
|
|
52
|
+
5. Log the failure and attempt an alternative strategy if the target defines one
|
|
53
|
+
|
|
54
|
+
**Update progress:** set the auth phase complete (or failed) in `progress.json`. Append to `session-log.md`:
|
|
38
55
|
`[<timestamp>] [PHASE] Auth complete — strategy: <strategy>, result: <success/failed>, user: <redacted>`
|
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Phase 2: Business Context & Charter (
|
|
1
|
+
# Phase 2: Business Context & Charter (5 min)
|
|
2
2
|
|
|
3
3
|
**This is the most important phase. Skip it and everything else is shallow.**
|
|
4
4
|
|
|
5
5
|
## Step 1: Use the App as a Real User
|
|
6
6
|
|
|
7
|
-
Before any testing, spend 2
|
|
7
|
+
Before any testing, spend 2 minutes USING the app the way a customer would. Not testing -- USING.
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
10
|
playwright-cli open <url>
|
|
@@ -62,9 +62,12 @@ Also select 2 Test Tours:
|
|
|
62
62
|
|
|
63
63
|
## Step 5: Write Charter
|
|
64
64
|
|
|
65
|
-
Save to `output/sessions/<session-dir>/charter.md
|
|
65
|
+
Save to `output/sessions/<session-dir>/charter.md` (confidentiality header first, per `${CLAUDE_SKILL_DIR}/references/output-contract.md`):
|
|
66
66
|
|
|
67
67
|
```markdown
|
|
68
|
+
> CONFIDENTIAL: This report may contain internal URLs, security vulnerabilities,
|
|
69
|
+
> and application details. Do not share outside your organization without review.
|
|
70
|
+
|
|
68
71
|
# Session Charter
|
|
69
72
|
|
|
70
73
|
## Context Source
|
|
@@ -101,12 +104,20 @@ Save to `output/sessions/<session-dir>/charter.md`:
|
|
|
101
104
|
- FEW HICCUPPS oracles: Focus on [Users, Claims, Product]
|
|
102
105
|
|
|
103
106
|
## Session Parameters
|
|
104
|
-
- Time box:
|
|
107
|
+
- Time box: 45 min
|
|
105
108
|
- Domain: [domain]
|
|
106
109
|
- Focus: [focus area or "full exploration"]
|
|
110
|
+
- Read-only: [true | false]
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Start evidence capture (both carry `-s=<sid>`):
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
playwright-cli tracing-start
|
|
117
|
+
playwright-cli video-start output/sessions/<session-dir>/videos/session.webm
|
|
107
118
|
```
|
|
108
119
|
|
|
109
|
-
|
|
120
|
+
Add `playwright-cli video-chapter "<phase or feature>"` at each phase boundary so the recording is navigable.
|
|
110
121
|
|
|
111
122
|
**Update progress:** Set charter phase complete in progress.json. Append to session-log.md:
|
|
112
123
|
`[<timestamp>] [PHASE] Charter complete — risk ranking defined, <N> journeys planned, heuristics: <selected>`
|