qualiow-exploratory-testing 1.3.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (93) hide show
  1. package/.claude-plugin/plugin.json +22 -0
  2. package/.env.example +27 -0
  3. package/CHANGELOG.md +194 -0
  4. package/README.md +362 -205
  5. package/agents/qa-gather-agent.md +24 -0
  6. package/{scripts → bin}/doctor-mobile.sh +57 -18
  7. package/bin/mcli +15 -9
  8. package/bin/mobile-cli.mjs +574 -286
  9. package/{scripts → bin}/setup-mobile.sh +56 -19
  10. package/bin/wadb +10 -4
  11. package/bin/wk-ios +29 -17
  12. package/bin/wkeval.mjs +7 -1
  13. package/data/domains/_default.yml +1 -1
  14. package/data/knowledge/changelog.yml +30 -12
  15. package/data/knowledge/manifest.yml +333 -127
  16. package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
  17. package/data/knowledge/releases/v0.1.0/release.yml +8 -1
  18. package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
  19. package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
  20. package/data/knowledge/releases/v0.6.0/release.yml +30 -0
  21. package/data/security/SECURITY-POLICY.md +273 -0
  22. package/data/targets/_default.yml +1 -1
  23. package/data/targets/_example-api-only.yml +80 -0
  24. package/data/targets/_example-mobile-emulation.yml +2 -2
  25. package/data/targets/_example-native-mobile.yml +2 -2
  26. package/data/targets/_example-sim-android-chrome.yml +1 -1
  27. package/data/targets/_example-sim-ios-safari.yml +1 -1
  28. package/data/templates/api-probe-matrix.md +3 -3
  29. package/data/templates/bug-report.md +29 -151
  30. package/data/templates/coverage-map.md +9 -6
  31. package/data/templates/expected-behaviour.md +3 -2
  32. package/data/templates/session-charter.md +3 -0
  33. package/data/templates/session-report.md +57 -80
  34. package/dist/cli/index.js +1452 -700
  35. package/dist/index.d.ts +589 -81
  36. package/dist/index.js +1612 -227
  37. package/docs/BACKEND-VERIFICATION.md +39 -9
  38. package/docs/GETTING-STARTED.md +188 -128
  39. package/docs/MOBILE-SETUP.md +149 -18
  40. package/package.json +32 -18
  41. package/skills/qa-explore/SKILL.md +31 -25
  42. package/skills/qa-explore/phases/00-setup.md +67 -41
  43. package/skills/qa-explore/phases/01-auth.md +32 -15
  44. package/skills/qa-explore/phases/02-charter.md +16 -5
  45. package/skills/qa-explore/phases/03-discovery.md +50 -26
  46. package/skills/qa-explore/phases/04-journeys.md +2 -2
  47. package/skills/qa-explore/phases/05-features.md +39 -24
  48. package/skills/qa-explore/phases/06-edge-cases.md +24 -13
  49. package/skills/qa-explore/phases/07-reporting.md +79 -140
  50. package/skills/qa-explore/references/context-integration.md +1 -1
  51. package/skills/qa-explore/references/output-contract.md +187 -0
  52. package/skills/qa-explore/references/paths.md +84 -0
  53. package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
  54. package/skills/qa-explore/references/security-rules.md +66 -25
  55. package/skills/qa-explore/references/severity-guide.md +31 -4
  56. package/skills/qa-explore-cleanup/SKILL.md +17 -6
  57. package/skills/qa-explore-feedback/SKILL.md +18 -5
  58. package/skills/qa-explore-mobile/SKILL.md +86 -64
  59. package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
  60. package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
  61. package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
  62. package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
  63. package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
  64. package/skills/qa-explore-mobile/phases/05-features.md +9 -9
  65. package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
  66. package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
  67. package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
  68. package/skills/qa-explore-quick/SKILL.md +60 -25
  69. package/skills/qa-explore-report/SKILL.md +40 -34
  70. package/skills/qa-gather/SKILL.md +17 -2
  71. package/skills/qa-knowledge-add/SKILL.md +20 -9
  72. package/skills/qa-knowledge-list/SKILL.md +12 -3
  73. package/skills/qa-target-setup/SKILL.md +63 -41
  74. package/skills/qa-verify-backend/SKILL.md +26 -17
  75. package/skills/qa-verify-backend/phases/00-setup.md +57 -12
  76. package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
  77. package/skills/qa-verify-backend/phases/03-live-verification.md +3 -3
  78. package/skills/qa-verify-backend/phases/03b-api-verification.md +22 -9
  79. package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
  80. package/skills/qa-verify-backend/phases/05-reporting.md +38 -10
  81. package/skills/qa-verify-backend/references/api-probes.md +28 -1
  82. package/skills/qa-verify-backend/references/payload-verification.md +1 -1
  83. package/skills/qa-verify-backend/references/release-readiness.md +1 -1
  84. package/skills/qa-verify-backend/references/safety-rules.md +13 -8
  85. package/data/domains/_default.md +0 -181
  86. package/data/domains/ecommerce.md +0 -192
  87. package/data/domains/fintech.md +0 -200
  88. package/data/domains/identity.md +0 -88
  89. package/data/domains/marketing.md +0 -194
  90. package/data/domains/saas.md +0 -197
  91. package/dist/cli/index.d.ts +0 -1
  92. package/dist/cli/index.js.map +0 -1
  93. package/dist/index.js.map +0 -1
@@ -6,32 +6,35 @@ The mobile skill drives **real** simulator/emulator browsers and native apps via
6
6
  and/or Android toolchains being installed on the machine. This guide gets a clean Mac to
7
7
  the point where a session can start.
8
8
 
9
+ Paths below are written as `bin/…` (a git checkout or a plugin install). In a project you
10
+ ran `qualiow init` in, the same files are at `qa/bin/…`; a global npm install also puts
11
+ `mcli`, `wadb`, `wk-ios`, `qualiow-doctor-mobile` and `qualiow-setup-mobile` on your `PATH`.
12
+
9
13
  ## TL;DR
10
14
 
11
15
  ```bash
12
16
  # 1. Install everything scriptable + create standard devices (idempotent, re-runnable)
13
- scripts/setup-mobile.sh # or --android / --ios, add --yes for non-interactive
17
+ bin/setup-mobile.sh # or --android / --ios; --yes for non-interactive
14
18
 
15
19
  # 2. Confirm the machine is ready (read-only; installs nothing)
16
- scripts/doctor-mobile.sh
20
+ bin/doctor-mobile.sh # add --quiet to print only failures + the verdict
17
21
 
18
22
  # 3. Run a mobile session (once doctor says READY)
19
- # e.g. iOS Safari against your mobile web app:
20
- # drive via bin/mcli — see the target config notes in data/targets/_example-sim-*.yml
23
+ /qa-explore-mobile --target my-mobile-target
21
24
  ```
22
25
 
23
- `scripts/setup-mobile.sh` automates everything it can and **prints clear MANUAL steps**
24
- for the few things it can't (Xcode + the iOS runtime). It finishes by running the doctor.
26
+ `bin/setup-mobile.sh` automates everything it can and **prints clear MANUAL steps** for the
27
+ few things it can't (Xcode + the iOS runtime). It finishes by running the doctor.
25
28
 
26
29
  ## What gets installed / created
27
30
 
28
31
  | Component | Needed for | Auto by `setup-mobile.sh`? |
29
32
  |-----------|-----------|----------------------------|
30
- | Homebrew | base | ✅ (offers to install) |
31
- | Node ≥ 18 | runs `mobile-cli.mjs` | ✅ |
32
- | **Maestro** | the driver (hierarchy/tap/input) | ✅ (`get.maestro.mobile.dev`) |
33
- | **JDK 17** | Android SDK tooling | ✅ `brew install openjdk@17` |
34
- | Android cmdline-tools | `sdkmanager`/`avdmanager` | ✅ `brew install --cask android-commandline-tools` |
33
+ | Homebrew | base | ✅ (offers to install; `curl \| bash`) |
34
+ | **Node ≥ 22.4** | runs `mobile-cli.mjs`, `wkeval.mjs` | ✅ |
35
+ | **Maestro** (pinned, default 2.10.0) | the driver (hierarchy/tap/input) | ✅ (`get.maestro.mobile.dev`, `curl \| bash`) |
36
+ | **JDK 17** | Maestro and the Android SDK tooling | ✅ `brew install openjdk@17` |
37
+ | Android cmdline-tools | `sdkmanager`/`avdmanager` | ✅ `brew install --cask android-commandlinetools` |
35
38
  | platform-tools (`adb`), `emulator` | drive Android | ✅ `sdkmanager` |
36
39
  | Google **Play** system image (API 35) | **Chrome must be present** | ✅ `sdkmanager` |
37
40
  | SDK licenses | unblock installs | ✅ `sdkmanager --licenses` |
@@ -39,10 +42,31 @@ for the few things it can't (Xcode + the iOS runtime). It finishes by running th
39
42
  | **Xcode** + CLI tools | iOS simulators at all | ⚠️ **manual** (App Store, multi-GB) |
40
43
  | iOS runtime | a simulator OS to run | ⚠️ **manual** if none installed (Xcode ▸ Settings ▸ Components) |
41
44
  | iOS simulator device `qa-iphone` | a device to boot | ✅ `simctl create` (once Xcode + a runtime exist) |
45
+ | `ios-webkit-debug-proxy` + `python3` | the `bin/wk-ios` DOM bridge (iOS Safari) | ✅ `brew install ios-webkit-debug-proxy` (python3 warned if absent) |
42
46
  | Safari / Chrome (the browsers) | web-mode SUT | ✅ preinstalled (Safari always; Chrome via the Play image) |
43
47
 
48
+ ### Flags
49
+
50
+ ```bash
51
+ bin/setup-mobile.sh # both platforms, interactive
52
+ bin/setup-mobile.sh --android # scope to one platform
53
+ bin/setup-mobile.sh --ios
54
+ bin/setup-mobile.sh --yes # non-interactive: assume yes to prompts
55
+ bin/setup-mobile.sh --yes --allow-curl-bash # ...and let the curl|bash installers run too
56
+ MAESTRO_VERSION=2.10.0 bin/setup-mobile.sh # pin a specific Maestro release
57
+ ```
58
+
59
+ The two `curl | bash` installers (Homebrew and Maestro) **always ask first**. Under `--yes`
60
+ they are *skipped* with the exact command printed, unless `--allow-curl-bash` is also given —
61
+ so an unattended run never pipes a remote script into a shell behind your back.
62
+
63
+ Other environment overrides: `ANDROID_API` (default 35), `ANDROID_AVD_NAME`, `IOS_DEVICE_NAME`,
64
+ `ANDROID_HOME`.
65
+
44
66
  ### The two genuinely manual iOS steps
67
+
45
68
  Xcode can't be reliably scripted:
69
+
46
70
  ```bash
47
71
  # After installing Xcode from the App Store:
48
72
  sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
@@ -50,9 +74,32 @@ sudo xcodebuild -license accept
50
74
  # If no iOS runtime is installed:
51
75
  xcodebuild -downloadPlatform iOS # or Xcode ▸ Settings ▸ Components
52
76
  ```
53
- Then re-run `scripts/setup-mobile.sh` — it will create the `qa-iphone` device.
77
+
78
+ Then re-run `bin/setup-mobile.sh` — it will create the `qa-iphone` device.
79
+
80
+ ## The preflight
81
+
82
+ ```bash
83
+ bin/doctor-mobile.sh # check both platforms
84
+ bin/doctor-mobile.sh --ios # iOS only
85
+ bin/doctor-mobile.sh --android # Android only
86
+ bin/doctor-mobile.sh --quiet # only failures + the final verdict
87
+ ```
88
+
89
+ Read-only: it installs nothing and prints `✓`/`✗` with the exact fix command for each gap.
90
+ The mobile session's `00-setup` phase runs it first, so a fresh machine fails fast with
91
+ actionable guidance instead of a cryptic error deep in a session. Exit code 0 means every
92
+ REQUIRED check for the requested platform passed.
93
+
94
+ It checks macOS, Homebrew, **Node ≥ 22.4**, **Maestro ≥ 2.6.0** (`MAESTRO_MIN_VERSION` —
95
+ `mobile-cli` parses the JSON-only `maestro hierarchy` introduced in 2.6), then per platform:
96
+ JDK 17, the SDK root, `adb`, `emulator`, `sdkmanager`/`avdmanager` and an AVD for Android;
97
+ Xcode CLT + full Xcode, an iOS runtime and an iPhone simulator for iOS. `ios-webkit-debug-proxy`
98
+ and `python3` are reported as **warnings** — without them the `bin/wk-ios` DOM bridge is
99
+ unavailable, but a session can still run.
54
100
 
55
101
  ## Standard device names (referenced by the target configs)
102
+
56
103
  - Android AVD: **`qa_pixel_api35`** (Google-Play API 35) — set `ANDROID_AVD_NAME` to override.
57
104
  - iOS sim: **`qa-iphone`** — set `IOS_DEVICE_NAME` to override.
58
105
 
@@ -60,15 +107,92 @@ The `data/targets/_example-sim-*.yml` configs resolve the device **by name** (no
60
107
  hardcoded UDID) so they're portable across machines. `00-setup` boots the named device,
61
108
  creating it if missing.
62
109
 
110
+ ## Driving the device
111
+
112
+ ```bash
113
+ bin/mcli boot qa_pixel_api35 # boots an AVD or an iOS sim by name, waits, set-devices it
114
+ bin/mcli set-device emulator-5554 --platform android # adb serial, iOS UDID, or iOS sim NAME
115
+ bin/mcli set-app com.android.chrome # or com.apple.mobilesafari, or your app id
116
+ bin/mcli launch
117
+ bin/mcli open-url https://staging.m.example.com # web mode (alias of deep-link)
118
+ bin/mcli snapshot # a11y tree as refs e1, e2, … — refs expire after 60 s
119
+ bin/mcli fill e4 my-username
120
+ bin/mcli snapshot # ALWAYS re-snapshot after the keyboard appears
121
+ bin/mcli click e7
122
+ bin/mcli info # resolved device/app + which state file is in use
123
+ bin/mcli --help # the full command surface
124
+ ```
125
+
126
+ `set-device` resolves the platform by matching the id against `xcrun simctl list devices` and
127
+ `adb devices`; pass `--platform ios|android` when it cannot tell. `boot` handles the
128
+ `emulator -avd …` / `simctl boot && open -a Simulator` dance for you.
129
+
130
+ **State.** Device and app selection is stored in a small JSON file. `--state <file>` (or
131
+ `$MOBILE_CLI_STATE`) picks it explicitly; otherwise `set-device` chooses
132
+ `~/.mobile-cli/<device>.json`. Give each concurrent session its own state file:
133
+
134
+ ```bash
135
+ bin/mcli --state /tmp/run-a.json set-device qa-iphone
136
+ bin/mcli --state /tmp/run-a.json set-app com.apple.mobilesafari
137
+ ```
138
+
139
+ **Exit codes.** `0` ok · `1` command failed · `2` usage · `3` stale or unknown ref (re-run
140
+ `snapshot`) · `4` no device or app set. A stale ref is not a flake: the tree is renumbered on
141
+ every snapshot, so a ref older than `MCLI_REF_TTL_MS` (default 60 000) is rejected rather than
142
+ tapped at a coordinate that has since moved.
143
+
144
+ **Evidence.** `screenshot <path>` needs no app. `logs [--since <sec>] [--errors]` reads logcat
145
+ or `simctl log`; `logs-clear` empties the Android buffer. `record-start <path>.mp4` /
146
+ `record-stop` produce mp4 on both platforms (Android caps a single recording at 180 s).
147
+
148
+ ## The helper wrappers
149
+
150
+ `bin/mcli`, `bin/wadb` and `bin/wk-ios` exist because Claude Code permission rules match on
151
+ the **first token** of a Bash command — a compound `export JAVA_HOME=…; node …` line can never
152
+ be pre-approved. Each wrapper resolves the environment internally and then execs the real tool.
153
+
154
+ ### `bin/wadb` — raw adb, environment resolved
155
+
156
+ ```bash
157
+ bin/wadb devices
158
+ bin/wadb logcat -d | tail -50
159
+ bin/wadb shell pm list packages | grep chrome
160
+ ```
161
+
162
+ It uses the `adb` already on `PATH` when there is one, else the SDK copy under `ANDROID_HOME`
163
+ (default `~/Library/Android/sdk`).
164
+
165
+ ### `bin/wk-ios` — real JS/DOM in iOS Simulator Safari
166
+
167
+ Maestro reads the native XCUITest accessibility tree, which collapses Safari's web content
168
+ into unlabeled nodes. The DOM itself is fully addressable through WebKit's Remote Inspector,
169
+ and `wk-ios` wires `ios-webkit-debug-proxy` up to the running Simulator so you can evaluate JS
170
+ in the frontmost page — the same power `playwright-cli eval` gives you on desktop, but on the
171
+ real iOS engine.
172
+
173
+ ```bash
174
+ bin/wk-ios 'document.title'
175
+ bin/wk-ios 'document.querySelectorAll(".cart_item").length'
176
+ bin/wk-ios --url # print the page WebSocket URL
177
+ bin/wk-ios --stop # stop the proxy this script started (never anyone else's)
178
+ ```
179
+
180
+ Requires `ios-webkit-debug-proxy`, `python3`, Node ≥ 22.4 (`wkeval.mjs` uses the global
181
+ `WebSocket`), and a booted Simulator with Safari on a page. `WK_IOS_PORT` (default 9221) sets
182
+ the iwdp device-list port; pages are served on `WK_IOS_PORT + 1`.
183
+
63
184
  ## Why a Google **Play** (or `google_apis`) Android image
185
+
64
186
  A stock **AOSP** system image ships **no `com.android.chrome`** — only a basic AOSP
65
187
  browser or none. Web-mode Android targets need Chrome, so `setup-mobile.sh` installs a
66
188
  `google_apis_playstore` image. Verify on any AVD with:
189
+
67
190
  ```bash
68
- adb shell pm list packages | grep chrome
191
+ bin/wadb shell pm list packages | grep chrome
69
192
  ```
70
193
 
71
194
  ## Auth reality (web targets)
195
+
72
196
  There is **no transferable `storage_state`** for a real device browser (unlike Playwright).
73
197
  SSO + MFA logins are done **once, interactively, by a human** in the sim/emulator browser;
74
198
  the device persists the browser profile across launches. Never script MFA. Never
@@ -76,17 +200,24 @@ the device persists the browser profile across launches. Never script MFA. Never
76
200
  `notes:` blocks in `data/targets/_example-sim-*.yml`.
77
201
 
78
202
  ## Troubleshooting
203
+
79
204
  - **`doctor-mobile.sh` says NOT READY** → fix each `✗` with the printed command, or run
80
205
  `setup-mobile.sh`, then re-run the doctor.
81
206
  - **`JAVA_HOME` / `adb` not found mid-session** → `bin/mcli` resolves JDK 17 via
82
- `/usr/libexec/java_home -v 17` and defaults `ANDROID_HOME` to `~/Library/Android/sdk`.
83
- Export them explicitly if your SDK lives elsewhere.
207
+ `/usr/libexec/java_home -v 17` (falling back to the Homebrew `openjdk@17` prefixes) and
208
+ defaults `ANDROID_HOME` to `~/Library/Android/sdk`. Export them explicitly if your SDK lives
209
+ elsewhere.
210
+ - **Maestro too old** → `mobile-cli` needs the JSON-only `maestro hierarchy` of 2.6.0+.
211
+ Reinstall pinned: `curl -Ls https://get.maestro.mobile.dev | MAESTRO_VERSION=2.10.0 bash`.
212
+ - **Every tap hits the wrong element** → your refs expired. Re-run `snapshot`; exit code 3
213
+ means exactly this.
84
214
  - **Intel Mac** → `setup-mobile.sh` selects `x86_64` system images automatically (`arm64`
85
215
  Macs get `arm64-v8a`).
86
216
  - **Emulator slow/flaky to boot or Chrome absent** → fall back to the iOS Safari target;
87
217
  both engines are independent.
88
218
 
89
219
  ## Scope
90
- This covers iOS Simulator + Android Emulator (browser **and** native modes). It does **not**
91
- provision physical devices, CI runners, or the desktop web path (`/qa-explore`, which only
92
- needs `playwright-cli`).
220
+
221
+ This covers iOS Simulator + Android Emulator (browser **and** native modes) on macOS. It does
222
+ **not** provision physical devices, CI runners, Windows/Linux hosts, or the desktop web path
223
+ (`/qa-explore`, which only needs `playwright-cli`).
package/package.json CHANGED
@@ -1,65 +1,79 @@
1
1
  {
2
2
  "name": "qualiow-exploratory-testing",
3
- "version": "1.3.0",
4
- "description": "qualiow — AI-powered exploratory testing skills using Claude Code + Playwright CLI (web) and Maestro/mobile-cli (iOS/Android simulators)",
3
+ "version": "2.0.0",
4
+ "description": "qualiow — AI-powered exploratory testing skills for Claude Code: web via playwright-cli, iOS/Android simulators via Maestro/mobile-cli, and backend/API acceptance-criteria verification",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
7
7
  "types": "dist/index.d.ts",
8
8
  "bin": {
9
- "qualiow": "dist/cli/index.js"
9
+ "qualiow": "dist/cli/index.js",
10
+ "mcli": "bin/mcli",
11
+ "wadb": "bin/wadb",
12
+ "wk-ios": "bin/wk-ios",
13
+ "qualiow-doctor-mobile": "bin/doctor-mobile.sh",
14
+ "qualiow-setup-mobile": "bin/setup-mobile.sh"
10
15
  },
11
16
  "files": [
12
17
  "dist/",
13
18
  "skills/",
19
+ "agents/",
14
20
  "bin/",
15
- "scripts/setup-mobile.sh",
16
- "scripts/doctor-mobile.sh",
17
- "data/domains/",
21
+ ".claude-plugin/",
22
+ "data/domains/*.yml",
18
23
  "data/knowledge/",
19
24
  "data/templates/",
20
- "data/targets/_default.yml",
21
- "data/targets/_example-native-mobile.yml",
22
- "data/targets/_example-sim-ios-safari.yml",
23
- "data/targets/_example-sim-android-chrome.yml",
24
- "data/targets/_example-mobile-emulation.yml",
25
- "data/targets/_example-backend.yml",
25
+ "data/security/",
26
+ "data/targets/_*.yml",
26
27
  "data/targets/testers-ai.yml",
27
28
  "docs/GETTING-STARTED.md",
28
29
  "docs/MOBILE-SETUP.md",
29
- "docs/BACKEND-VERIFICATION.md"
30
+ "docs/BACKEND-VERIFICATION.md",
31
+ ".env.example",
32
+ "CHANGELOG.md"
30
33
  ],
31
34
  "scripts": {
32
- "sync:skills": "rm -rf skills && cp -R .claude/skills skills",
33
- "prebuild": "npm run sync:skills",
35
+ "sync:plugin": "rm -rf skills agents && cp -R .claude/skills skills && cp -R .claude/agents agents",
36
+ "prebuild": "npm run sync:plugin",
34
37
  "build": "tsup",
35
38
  "dev": "tsup --watch",
36
39
  "test": "vitest run",
37
40
  "test:watch": "vitest",
38
41
  "test:coverage": "vitest run --coverage",
39
42
  "validate": "node dist/cli/index.js validate --all",
43
+ "kb:sync": "node dist/cli/index.js kb sync",
44
+ "kb:check": "node dist/cli/index.js kb check",
45
+ "check:mirror": "diff -rq .claude/skills skills && diff -rq .claude/agents agents",
46
+ "check:pack": "node scripts/check-pack.mjs",
40
47
  "lint": "tsc --noEmit",
41
- "prepublishOnly": "npm run build && npm test"
48
+ "version": "node scripts/sync-version.mjs && git add .claude-plugin/plugin.json",
49
+ "prepublishOnly": "npm run build && npm run lint && npm test && npm run check:pack"
42
50
  },
43
51
  "keywords": [
44
52
  "exploratory-testing",
45
53
  "qa",
46
54
  "testing",
47
55
  "playwright",
56
+ "playwright-cli",
48
57
  "maestro",
49
58
  "mobile-testing",
50
59
  "ios-simulator",
51
60
  "android-emulator",
61
+ "api-testing",
62
+ "acceptance-criteria",
52
63
  "ai",
53
64
  "claude",
65
+ "claude-code",
54
66
  "skills",
67
+ "plugin",
55
68
  "heuristics",
56
69
  "bug-detection"
57
70
  ],
58
71
  "author": "William Oliveira",
59
72
  "license": "MIT",
60
73
  "engines": {
61
- "node": ">=18.0.0"
74
+ "node": ">=22.4.0"
62
75
  },
76
+ "packageManager": "npm@10.9.2",
63
77
  "repository": {
64
78
  "type": "git",
65
79
  "url": "git+https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git"
@@ -69,7 +83,7 @@
69
83
  "url": "https://github.com/willcoliveira/qualiow-exploratory-testing-skills/issues"
70
84
  },
71
85
  "dependencies": {
72
- "@playwright/cli": "^0.1.15",
86
+ "@playwright/cli": "^0.1.19",
73
87
  "chalk": "^5.3.0",
74
88
  "commander": "^15.0.0",
75
89
  "glob": "^13.0.6",
@@ -7,10 +7,11 @@ description: >
7
7
  and writes bug reports that executives act on.
8
8
  Use when user says: "explore", "test this site", "find bugs", "QA check", "exploratory session",
9
9
  or provides a URL to test.
10
- allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Read, Write, Glob, Grep
10
+ argument-hint: "<url> [--target <id>] [--context <file>] [--focus <area>] [--session <dir>]"
11
+ allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Bash(diff:*), Read, Write, Glob, Grep
11
12
  ---
12
13
 
13
- # Exploratory Testing Session (v2)
14
+ # Exploratory Testing Session
14
15
 
15
16
  ## Your Identity
16
17
 
@@ -29,43 +30,48 @@ You are a **Principal QA Engineer** with 20+ years of experience. You don't just
29
30
  # Mode 1: Blind exploration (default — always works, no prior info needed)
30
31
  /qa-explore https://example.com
31
32
 
32
- # Mode 2: Context-enriched (requirements file from tool-qa-workflow or manual)
33
- /qa-explore https://example.com --context requirements-SI-2305.md
33
+ # Mode 2: Context-enriched (a /qa-gather context file or any requirements document)
34
+ /qa-explore https://example.com --context output/context/checkout-context.md
34
35
 
35
36
  # Mode 3: Inline context (user provides context in their message)
36
37
  /qa-explore https://example.com
37
38
  > "This is a new checkout flow. Should support Visa/MC, require delivery address,
38
39
  > send confirmation email. Team just refactored the payment service."
39
40
 
40
- # With target config
41
- /qa-explore --target company-staging --context requirements-SI-2305.md
41
+ # With a saved target config, or reusing a `qualiow explore` pre-flight directory
42
+ /qa-explore --target company-staging --context output/context/SI-2305-context.md
43
+ /qa-explore https://example.com --session output/sessions/2026-09-08-1420-explore-example-com
42
44
  ```
43
45
 
44
- ---
46
+ ## Before You Start
45
47
 
46
- ## Session Phases
48
+ - Paths, target resolution, the session-directory scheme and the `-s=<sid>` session id are defined once in **`${CLAUDE_SKILL_DIR}/references/paths.md`**. Read it first.
49
+ - Every artefact you write follows **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** (bug report, session report, `stats.json`, confidentiality header). `qualiow report` and the feedback skills only understand that format.
50
+ - **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** is absolute: production detection, redaction before disk, session isolation, output classification.
47
51
 
48
- Execute each phase in order. Read and follow the linked file for full instructions.
52
+ ## Session Phases
49
53
 
50
- | Phase | File | Summary |
51
- |-------|------|---------|
52
- | **Setup** | Read and follow: `phases/00-setup.md` | Parse input, resolve target, load context, load knowledge, create session directory |
53
- | **Auth** | Read and follow: `phases/01-auth.md` | Authenticate via storage state, credentials, or token; verify login |
54
- | **Charter** | Read and follow: `phases/02-charter.md` | Use app as real user, identify journeys, risk-rank features, select heuristics, write charter |
55
- | **Discovery** | Read and follow: `phases/03-discovery.md` | Map the site, check console/network, enforce scope, find what's MISSING |
56
- | **Journeys** | Read and follow: `phases/04-journeys.md` | End-to-end user journeys, data integrity verification, cross-page consistency |
57
- | **Features** | Read and follow: `phases/05-features.md` | Deep feature testing with SFDIPOT, business logic stress, negative space, FEW HICCUPPS |
58
- | **Edge Cases** | Read and follow: `phases/06-edge-cases.md` | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
59
- | **Reporting** | Read and follow: `phases/07-reporting.md` | Stop recording, reflect, write bug reports with business impact, session report, coverage map |
54
+ Execute each phase in order. Read and follow the linked file. The budget sums to 45 minutes; write to disk after each phase and carry only summaries in context.
60
55
 
61
- ---
56
+ | Phase | File | Budget | Summary |
57
+ |-------|------|--------|---------|
58
+ | **Setup** | `${CLAUDE_SKILL_DIR}/phases/00-setup.md` | 2 min | Parse input, resolve target and data paths, load context and knowledge, create the session directory |
59
+ | **Auth** | `${CLAUDE_SKILL_DIR}/phases/01-auth.md` | 2 min | Authenticate via storage state, credentials, or token; verify login |
60
+ | **Charter** | `${CLAUDE_SKILL_DIR}/phases/02-charter.md` | 5 min | Use the app as a real user, identify journeys, risk-rank features, select heuristics, write the charter |
61
+ | **Discovery** | `${CLAUDE_SKILL_DIR}/phases/03-discovery.md` | 6 min | Map the site, check console and requests, enforce scope, find what's MISSING |
62
+ | **Journeys** | `${CLAUDE_SKILL_DIR}/phases/04-journeys.md` | 10 min | End-to-end user journeys, data-integrity verification, cross-page consistency |
63
+ | **Features** | `${CLAUDE_SKILL_DIR}/phases/05-features.md` | 10 min | Deep feature testing with SFDIPOT, business-logic stress, negative space, FEW HICCUPPS |
64
+ | **Edge Cases** | `${CLAUDE_SKILL_DIR}/phases/06-edge-cases.md` | 6 min | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
65
+ | **Reporting** | `${CLAUDE_SKILL_DIR}/phases/07-reporting.md` | 4 min | Stop recording, reflect, write bug reports with business impact, session report, stats, close the browser session |
62
66
 
63
67
  ## References
64
68
 
65
69
  Consult these as needed throughout the session:
66
70
 
67
- - **`references/security-rules.md`** — Prompt injection resistance, credential protection, session isolation, production safety, output classification
68
- - **`references/severity-guide.md`** — Severity definitions (Critical/High/Medium/Low) and the "when in doubt go LOWER" rule
69
- - **`references/session-rules.md`** — 20 testing discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI bias avoidance
70
- - **`references/context-integration.md`** — How `--context` changes the session (context file, inline context, blind mode)
71
- - **`references/playwright-agents-integration.md`** — Opt-in handoff to Playwright Test Agents (planner / generator / healer, 1.56+) when a reproducible bug should become a regression test. Requires the `@playwright/test` peer dependency.
71
+ - **`${CLAUDE_SKILL_DIR}/references/paths.md`** — resolution order for targets, data, credentials and output; session-directory scheme; session id; index rows
72
+ - **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** — the one bug-report and session-report format, `stats.json`, confidentiality header
73
+ - **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** — prompt-injection resistance, production rule, redaction list, session isolation, output classification
74
+ - **`${CLAUDE_SKILL_DIR}/references/severity-guide.md`** — severity definitions, the "when in doubt go LOWER" rule, risk matrix, priority table
75
+ - **`${CLAUDE_SKILL_DIR}/references/session-rules.md`** — 20 testing-discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI-bias avoidance
76
+ - **`${CLAUDE_SKILL_DIR}/references/context-integration.md`** — how `--context` changes the session (context file, inline context, blind mode)
77
+ - **`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md`** — opt-in handoff to Playwright Test Agents (planner / generator / healer, Playwright 1.56+) when a reproducible bug should become a regression test; requires the optional `@playwright/test` peer dependency
@@ -1,44 +1,53 @@
1
- # Phase 0: Session Setup
1
+ # Phase 0: Session Setup (2 min)
2
2
 
3
3
  ## Step 1: Parse Input & Resolve Target
4
4
 
5
- Extract from user message:
6
- - **target_url**: The URL to test (required)
7
- - **domain**: Domain type (optional -- auto-detect if not specified)
8
- - **focus**: Specific area or feature (optional)
9
- - **time_box**: Session duration (default: 45 min, max: 45 min)
10
- - **context**: Path to requirements file OR inline context text (optional)
11
-
12
- Resolve target config in this order:
13
- 1. If `--target <name>` given → load from skill `data/targets/<name>.yml`
14
- 2. Else if CWD contains `qa/target.yml` → load it as the target config
15
- 3. Else fall back to `data/targets/_default.yml`
16
-
17
- If loaded from `qa/target.yml`, also check `qa/.env` in CWD for credential env var values before falling back to the skill root `.env`.
18
-
19
- Read domain config from `data/domains/`.
5
+ Extract from the user message:
6
+ - **target_url**: the URL to test (required unless `--target` is given)
7
+ - **--target <id>**: a saved target config (optional)
8
+ - **--context <file>** or inline context text (optional)
9
+ - **--focus <area>**: a specific area or feature (optional)
10
+ - **--session <dir>**: an existing pre-flight directory created by `qualiow explore` (optional)
11
+ - **domain**: domain type (optional; auto-detect from the app if not given)
12
+ - **time_box**: 45 min (default and maximum)
13
+
14
+ Resolve the target config, the data directory (`<data>`) and the credentials file in the
15
+ order defined in `${CLAUDE_SKILL_DIR}/references/paths.md`: `--target` → `data/targets/<id>.yml`
16
+ in the project, else the plugin's copy; no `--target` → `qa/target.yml` in the project, else
17
+ `_default.yml`; credentials from `qa/.env`, else `.env`. Then read
18
+ `<data>/domains/<domain>.yml` for the domain's completeness checklist, data-integrity checks
19
+ and journeys.
20
+
21
+ Apply the **production rule** from `${CLAUDE_SKILL_DIR}/references/security-rules.md` to the
22
+ resolved hostname now. If it fires, log `[SAFETY] Production detected -- running in read-only mode`
23
+ and carry read-only mode through every later phase.
20
24
 
21
25
  ## Step 2: Load Context (if provided)
22
26
 
23
- **If `--context <file>` provided:** Read the file. It may be a `tool-qa-workflow` output (structured markdown with Jira data, Confluence specs, GitLab MR diffs, Figma design notes) or any document describing what to test.
27
+ **If `--context <file>` provided:** read the file. It may be a `/qa-gather` context file
28
+ (structured markdown with ticket data, specs, MR diffs, design notes) or any document
29
+ describing what to test.
24
30
 
25
- **If inline context provided:** Extract from the user's message any descriptions of: what the feature does, acceptance criteria, what changed, what the design should look like, known risks, user personas.
31
+ **If inline context provided:** extract from the user's message any descriptions of: what
32
+ the feature does, acceptance criteria, what changed, what the design should look like,
33
+ known risks, user personas.
26
34
 
27
- **If no context provided:** That's fine -- run in blind exploration mode. The agent discovers everything on its own.
35
+ **If no context provided:** that's fine; run in blind exploration mode.
28
36
 
29
37
  **When context IS available, extract and use:**
30
38
 
31
39
  | Context Element | How It Changes the Session |
32
40
  |----------------|---------------------------|
33
- | **Acceptance criteria** | Add as explicit checkpoints -- verify each AC during testing |
34
- | **What changed** (MR diff, commits) | Focus testing on changed areas -- highest regression risk |
35
- | **Design specs** (Figma, mockups) | Compare live app against intended design -- spot deviations |
41
+ | **Acceptance criteria** | Add as explicit checkpoints; verify each AC during testing |
42
+ | **What changed** (MR diff, commits) | Focus testing on changed areas; highest regression risk |
43
+ | **Design specs** (Figma, mockups) | Compare the live app against the intended design |
36
44
  | **User stories / requirements** | Test against INTENT, not just what's visible |
37
- | **Known risks** (from risk analysis) | Prioritize testing on flagged risk areas |
38
- | **API contracts** (OpenAPI, endpoints) | Verify UI matches API behavior |
39
- | **Previous test results** | Don't re-test what passed -- focus on gaps |
45
+ | **Known risks** | Prioritize testing on flagged risk areas |
46
+ | **API contracts** (OpenAPI, endpoints) | Verify the UI matches API behaviour |
47
+ | **Previous test results** | Don't re-test what passed; focus on gaps |
40
48
 
41
- **Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app with zero documentation should still get a full, valuable session. Context is a boost, not a gate.
49
+ **Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
50
+ with zero documentation should still get a full, valuable session.
42
51
 
43
52
  When context is loaded, note in the charter:
44
53
  ```
@@ -49,17 +58,24 @@ CHANGE SCOPE: [what changed or "unknown -- full exploration"]
49
58
 
50
59
  ## Step 3: Load Knowledge
51
60
 
52
- Read `data/knowledge/manifest.yml`. Load:
61
+ Read `<data>/knowledge/manifest.yml`. Load:
53
62
  1. Core heuristics (SFDIPOT, FEW HICCUPPS, Test Tours)
54
- 2. Domain-specific entries
55
- 3. `data/knowledge/learned-patterns.md` -- false-positive skip list + must-check patterns
56
- 4. Business logic & race condition techniques (if app has transactions/state changes)
63
+ 2. Domain-specific entries from `loading_strategy.by_domain`
64
+ 3. `<data>/knowledge/learned-patterns.md`: the false-positive skip list and must-check patterns
65
+ 4. Business-logic and race-condition techniques (if the app has transactions or state changes)
66
+
67
+ Keep brief summaries only; don't fill context with full entries.
57
68
 
58
- Keep brief summaries only -- don't fill context with full entries.
69
+ ## Step 4: Create the Session Directory
59
70
 
60
- ## Step 4: Create Session Directory
71
+ Unless `--session <dir>` was given, create
72
+ `output/sessions/<YYYY-MM-DD-HHmm>-explore-<slug>/` (scheme in `paths.md`; `<slug>` is the
73
+ target id without a leading `_`, or the hostname with dots replaced by dashes) containing
74
+ `screenshots/`, `bugs/`, `videos/`, and an empty `session-log.md`. With `--session`, reuse
75
+ that directory and append to its `session-log.md`.
61
76
 
62
- Create `output/sessions/<YYYY-MM-DD-HHmm-target>/` with: `charter.md`, `session-log.md`, `screenshots/`, `bugs/`
77
+ Choose the playwright-cli session id `-s=explore-<HHmm>-<slug>`. **Every `playwright-cli`
78
+ call in this session carries `-s=<sid>`; the examples in the phase files omit the prefix.**
63
79
 
64
80
  ## Step 5: Initialize Progress Tracking
65
81
 
@@ -67,6 +83,7 @@ Write `output/sessions/<session-dir>/progress.json`:
67
83
  ```json
68
84
  {
69
85
  "session_id": "<session-dir>",
86
+ "kind": "explore",
70
87
  "target": "<target_url>",
71
88
  "domain": "<domain>",
72
89
  "started_at": "<ISO timestamp>",
@@ -91,27 +108,36 @@ Write `output/sessions/<session-dir>/progress.json`:
91
108
 
92
109
  Append to `session-log.md`:
93
110
  ```
94
- [<timestamp>] [PHASE] Setup complete — session initialized
111
+ [<timestamp>] [PHASE] Setup complete — session <session-dir>, sid <sid>, read_only: <true|false>
95
112
  ```
96
113
 
97
114
  ## Step 6: Optional — Bootstrap Playwright Test Agents (opt-in)
98
115
 
99
- **Skip this step entirely if `@playwright/test` is not installed as a peer dep OR `playwright.config.*` is not present.** This is an additive capability for consumers who want to pair an exploratory session with the Playwright Test Agents framework (planner / generator / healer, Playwright 1.56+).
116
+ **Skip this step entirely if `@playwright/test` is not installed in the consumer project OR
117
+ no `playwright.config.*` is present.** This is an additive capability for teams that want to
118
+ turn reproducible findings into regression tests with the Playwright Test Agents framework
119
+ (planner / generator / healer, Playwright 1.56+).
100
120
 
101
- If both prerequisites are present AND the user has opted in (via target config or inline context), you MAY offer to scaffold the agent workspace:
121
+ If both prerequisites are present AND the user has opted in (via the target config or the
122
+ context), you MAY offer to scaffold the agent workspace:
102
123
 
103
124
  ```bash
104
125
  npx playwright init-agents --loop=claude
105
126
  ```
106
127
 
107
- This generates agent definition files that later phases can hand off to — most usefully in phase 5 (features), where a reproducible bug can be converted into a regression test by the generator agent. See `references/playwright-agents-integration.md` for the full workflow.
128
+ With `--loop=claude` this writes the three agent definitions into `.claude/agents/`, a
129
+ `.mcp.json` registering the `playwright-test` server, and a `specs/` scaffold with a seed
130
+ spec if the project has none; other loops: `codex`, `copilot`, `opencode`, `vscode`,
131
+ `vscode-legacy`. It does not touch `playwright.config.*`. See
132
+ `${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md` for the hand-off after the
133
+ session.
108
134
 
109
- **Never block the session on this step.** If the user declines, if the prerequisites are missing, or if the command fails, log a note in `session-log.md` and continue to Phase 1 normally.
135
+ **Never block the session on this step.** If the user declines, the prerequisites are
136
+ missing, or the command fails, log a note in `session-log.md` and continue.
110
137
 
111
- **IMPORTANT: After EVERY phase completion, update progress.json and append to session-log.md.** This is how the user monitors session progress. Use this pattern at the end of each phase:
138
+ **IMPORTANT: After EVERY phase completion, update `progress.json` and append to
139
+ `session-log.md`.** This is how the user monitors progress:
112
140
 
113
141
  ```
114
- # Update progress.json — set current phase complete, next phase as current
115
- # Append to session-log.md:
116
142
  [<timestamp>] [PHASE] <phase_name> complete — <pages> pages, <bugs> bugs, <key finding summary>
117
143
  ```