qualiow-exploratory-testing 1.3.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +194 -0
- package/README.md +362 -205
- package/agents/qa-gather-agent.md +24 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/knowledge/changelog.yml +30 -12
- package/data/knowledge/manifest.yml +333 -127
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +3 -3
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +9 -6
- package/data/templates/expected-behaviour.md +3 -2
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +1452 -700
- package/dist/index.d.ts +589 -81
- package/dist/index.js +1612 -227
- package/docs/BACKEND-VERIFICATION.md +39 -9
- package/docs/GETTING-STARTED.md +188 -128
- package/docs/MOBILE-SETUP.md +149 -18
- package/package.json +32 -18
- package/skills/qa-explore/SKILL.md +31 -25
- package/skills/qa-explore/phases/00-setup.md +67 -41
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +50 -26
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +79 -140
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/output-contract.md +187 -0
- package/skills/qa-explore/references/paths.md +84 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +66 -25
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +17 -6
- package/skills/qa-explore-feedback/SKILL.md +18 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +60 -25
- package/skills/qa-explore-report/SKILL.md +40 -34
- package/skills/qa-gather/SKILL.md +17 -2
- package/skills/qa-knowledge-add/SKILL.md +20 -9
- package/skills/qa-knowledge-list/SKILL.md +12 -3
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +26 -17
- package/skills/qa-verify-backend/phases/00-setup.md +57 -12
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
- package/skills/qa-verify-backend/phases/03-live-verification.md +3 -3
- package/skills/qa-verify-backend/phases/03b-api-verification.md +22 -9
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
- package/skills/qa-verify-backend/phases/05-reporting.md +38 -10
- package/skills/qa-verify-backend/references/api-probes.md +28 -1
- package/skills/qa-verify-backend/references/payload-verification.md +1 -1
- package/skills/qa-verify-backend/references/release-readiness.md +1 -1
- package/skills/qa-verify-backend/references/safety-rules.md +13 -8
- package/data/domains/_default.md +0 -181
- package/data/domains/ecommerce.md +0 -192
- package/data/domains/fintech.md +0 -200
- package/data/domains/identity.md +0 -88
- package/data/domains/marketing.md +0 -194
- package/data/domains/saas.md +0 -197
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
package/docs/MOBILE-SETUP.md
CHANGED
|
@@ -6,32 +6,35 @@ The mobile skill drives **real** simulator/emulator browsers and native apps via
|
|
|
6
6
|
and/or Android toolchains being installed on the machine. This guide gets a clean Mac to
|
|
7
7
|
the point where a session can start.
|
|
8
8
|
|
|
9
|
+
Paths below are written as `bin/…` (a git checkout or a plugin install). In a project you
|
|
10
|
+
ran `qualiow init` in, the same files are at `qa/bin/…`; a global npm install also puts
|
|
11
|
+
`mcli`, `wadb`, `wk-ios`, `qualiow-doctor-mobile` and `qualiow-setup-mobile` on your `PATH`.
|
|
12
|
+
|
|
9
13
|
## TL;DR
|
|
10
14
|
|
|
11
15
|
```bash
|
|
12
16
|
# 1. Install everything scriptable + create standard devices (idempotent, re-runnable)
|
|
13
|
-
|
|
17
|
+
bin/setup-mobile.sh # or --android / --ios; --yes for non-interactive
|
|
14
18
|
|
|
15
19
|
# 2. Confirm the machine is ready (read-only; installs nothing)
|
|
16
|
-
|
|
20
|
+
bin/doctor-mobile.sh # add --quiet to print only failures + the verdict
|
|
17
21
|
|
|
18
22
|
# 3. Run a mobile session (once doctor says READY)
|
|
19
|
-
|
|
20
|
-
# drive via bin/mcli — see the target config notes in data/targets/_example-sim-*.yml
|
|
23
|
+
/qa-explore-mobile --target my-mobile-target
|
|
21
24
|
```
|
|
22
25
|
|
|
23
|
-
`
|
|
24
|
-
|
|
26
|
+
`bin/setup-mobile.sh` automates everything it can and **prints clear MANUAL steps** for the
|
|
27
|
+
few things it can't (Xcode + the iOS runtime). It finishes by running the doctor.
|
|
25
28
|
|
|
26
29
|
## What gets installed / created
|
|
27
30
|
|
|
28
31
|
| Component | Needed for | Auto by `setup-mobile.sh`? |
|
|
29
32
|
|-----------|-----------|----------------------------|
|
|
30
|
-
| Homebrew | base | ✅ (offers to install) |
|
|
31
|
-
| Node ≥
|
|
32
|
-
| **Maestro** | the driver (hierarchy/tap/input) | ✅ (`get.maestro.mobile.dev`) |
|
|
33
|
-
| **JDK 17** | Android SDK tooling | ✅ `brew install openjdk@17` |
|
|
34
|
-
| Android cmdline-tools | `sdkmanager`/`avdmanager` | ✅ `brew install --cask android-
|
|
33
|
+
| Homebrew | base | ✅ (offers to install; `curl \| bash`) |
|
|
34
|
+
| **Node ≥ 22.4** | runs `mobile-cli.mjs`, `wkeval.mjs` | ✅ |
|
|
35
|
+
| **Maestro** (pinned, default 2.10.0) | the driver (hierarchy/tap/input) | ✅ (`get.maestro.mobile.dev`, `curl \| bash`) |
|
|
36
|
+
| **JDK 17** | Maestro and the Android SDK tooling | ✅ `brew install openjdk@17` |
|
|
37
|
+
| Android cmdline-tools | `sdkmanager`/`avdmanager` | ✅ `brew install --cask android-commandlinetools` |
|
|
35
38
|
| platform-tools (`adb`), `emulator` | drive Android | ✅ `sdkmanager` |
|
|
36
39
|
| Google **Play** system image (API 35) | **Chrome must be present** | ✅ `sdkmanager` |
|
|
37
40
|
| SDK licenses | unblock installs | ✅ `sdkmanager --licenses` |
|
|
@@ -39,10 +42,31 @@ for the few things it can't (Xcode + the iOS runtime). It finishes by running th
|
|
|
39
42
|
| **Xcode** + CLI tools | iOS simulators at all | ⚠️ **manual** (App Store, multi-GB) |
|
|
40
43
|
| iOS runtime | a simulator OS to run | ⚠️ **manual** if none installed (Xcode ▸ Settings ▸ Components) |
|
|
41
44
|
| iOS simulator device `qa-iphone` | a device to boot | ✅ `simctl create` (once Xcode + a runtime exist) |
|
|
45
|
+
| `ios-webkit-debug-proxy` + `python3` | the `bin/wk-ios` DOM bridge (iOS Safari) | ✅ `brew install ios-webkit-debug-proxy` (python3 warned if absent) |
|
|
42
46
|
| Safari / Chrome (the browsers) | web-mode SUT | ✅ preinstalled (Safari always; Chrome via the Play image) |
|
|
43
47
|
|
|
48
|
+
### Flags
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
bin/setup-mobile.sh # both platforms, interactive
|
|
52
|
+
bin/setup-mobile.sh --android # scope to one platform
|
|
53
|
+
bin/setup-mobile.sh --ios
|
|
54
|
+
bin/setup-mobile.sh --yes # non-interactive: assume yes to prompts
|
|
55
|
+
bin/setup-mobile.sh --yes --allow-curl-bash # ...and let the curl|bash installers run too
|
|
56
|
+
MAESTRO_VERSION=2.10.0 bin/setup-mobile.sh # pin a specific Maestro release
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The two `curl | bash` installers (Homebrew and Maestro) **always ask first**. Under `--yes`
|
|
60
|
+
they are *skipped* with the exact command printed, unless `--allow-curl-bash` is also given —
|
|
61
|
+
so an unattended run never pipes a remote script into a shell behind your back.
|
|
62
|
+
|
|
63
|
+
Other environment overrides: `ANDROID_API` (default 35), `ANDROID_AVD_NAME`, `IOS_DEVICE_NAME`,
|
|
64
|
+
`ANDROID_HOME`.
|
|
65
|
+
|
|
44
66
|
### The two genuinely manual iOS steps
|
|
67
|
+
|
|
45
68
|
Xcode can't be reliably scripted:
|
|
69
|
+
|
|
46
70
|
```bash
|
|
47
71
|
# After installing Xcode from the App Store:
|
|
48
72
|
sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
|
|
@@ -50,9 +74,32 @@ sudo xcodebuild -license accept
|
|
|
50
74
|
# If no iOS runtime is installed:
|
|
51
75
|
xcodebuild -downloadPlatform iOS # or Xcode ▸ Settings ▸ Components
|
|
52
76
|
```
|
|
53
|
-
|
|
77
|
+
|
|
78
|
+
Then re-run `bin/setup-mobile.sh` — it will create the `qa-iphone` device.
|
|
79
|
+
|
|
80
|
+
## The preflight
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
bin/doctor-mobile.sh # check both platforms
|
|
84
|
+
bin/doctor-mobile.sh --ios # iOS only
|
|
85
|
+
bin/doctor-mobile.sh --android # Android only
|
|
86
|
+
bin/doctor-mobile.sh --quiet # only failures + the final verdict
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Read-only: it installs nothing and prints `✓`/`✗` with the exact fix command for each gap.
|
|
90
|
+
The mobile session's `00-setup` phase runs it first, so a fresh machine fails fast with
|
|
91
|
+
actionable guidance instead of a cryptic error deep in a session. Exit code 0 means every
|
|
92
|
+
REQUIRED check for the requested platform passed.
|
|
93
|
+
|
|
94
|
+
It checks macOS, Homebrew, **Node ≥ 22.4**, **Maestro ≥ 2.6.0** (`MAESTRO_MIN_VERSION` —
|
|
95
|
+
`mobile-cli` parses the JSON-only `maestro hierarchy` introduced in 2.6), then per platform:
|
|
96
|
+
JDK 17, the SDK root, `adb`, `emulator`, `sdkmanager`/`avdmanager` and an AVD for Android;
|
|
97
|
+
Xcode CLT + full Xcode, an iOS runtime and an iPhone simulator for iOS. `ios-webkit-debug-proxy`
|
|
98
|
+
and `python3` are reported as **warnings** — without them the `bin/wk-ios` DOM bridge is
|
|
99
|
+
unavailable, but a session can still run.
|
|
54
100
|
|
|
55
101
|
## Standard device names (referenced by the target configs)
|
|
102
|
+
|
|
56
103
|
- Android AVD: **`qa_pixel_api35`** (Google-Play API 35) — set `ANDROID_AVD_NAME` to override.
|
|
57
104
|
- iOS sim: **`qa-iphone`** — set `IOS_DEVICE_NAME` to override.
|
|
58
105
|
|
|
@@ -60,15 +107,92 @@ The `data/targets/_example-sim-*.yml` configs resolve the device **by name** (no
|
|
|
60
107
|
hardcoded UDID) so they're portable across machines. `00-setup` boots the named device,
|
|
61
108
|
creating it if missing.
|
|
62
109
|
|
|
110
|
+
## Driving the device
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
bin/mcli boot qa_pixel_api35 # boots an AVD or an iOS sim by name, waits, set-devices it
|
|
114
|
+
bin/mcli set-device emulator-5554 --platform android # adb serial, iOS UDID, or iOS sim NAME
|
|
115
|
+
bin/mcli set-app com.android.chrome # or com.apple.mobilesafari, or your app id
|
|
116
|
+
bin/mcli launch
|
|
117
|
+
bin/mcli open-url https://staging.m.example.com # web mode (alias of deep-link)
|
|
118
|
+
bin/mcli snapshot # a11y tree as refs e1, e2, … — refs expire after 60 s
|
|
119
|
+
bin/mcli fill e4 my-username
|
|
120
|
+
bin/mcli snapshot # ALWAYS re-snapshot after the keyboard appears
|
|
121
|
+
bin/mcli click e7
|
|
122
|
+
bin/mcli info # resolved device/app + which state file is in use
|
|
123
|
+
bin/mcli --help # the full command surface
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
`set-device` resolves the platform by matching the id against `xcrun simctl list devices` and
|
|
127
|
+
`adb devices`; pass `--platform ios|android` when it cannot tell. `boot` handles the
|
|
128
|
+
`emulator -avd …` / `simctl boot && open -a Simulator` dance for you.
|
|
129
|
+
|
|
130
|
+
**State.** Device and app selection is stored in a small JSON file. `--state <file>` (or
|
|
131
|
+
`$MOBILE_CLI_STATE`) picks it explicitly; otherwise `set-device` chooses
|
|
132
|
+
`~/.mobile-cli/<device>.json`. Give each concurrent session its own state file:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
bin/mcli --state /tmp/run-a.json set-device qa-iphone
|
|
136
|
+
bin/mcli --state /tmp/run-a.json set-app com.apple.mobilesafari
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
**Exit codes.** `0` ok · `1` command failed · `2` usage · `3` stale or unknown ref (re-run
|
|
140
|
+
`snapshot`) · `4` no device or app set. A stale ref is not a flake: the tree is renumbered on
|
|
141
|
+
every snapshot, so a ref older than `MCLI_REF_TTL_MS` (default 60 000) is rejected rather than
|
|
142
|
+
tapped at a coordinate that has since moved.
|
|
143
|
+
|
|
144
|
+
**Evidence.** `screenshot <path>` needs no app. `logs [--since <sec>] [--errors]` reads logcat
|
|
145
|
+
or `simctl log`; `logs-clear` empties the Android buffer. `record-start <path>.mp4` /
|
|
146
|
+
`record-stop` produce mp4 on both platforms (Android caps a single recording at 180 s).
|
|
147
|
+
|
|
148
|
+
## The helper wrappers
|
|
149
|
+
|
|
150
|
+
`bin/mcli`, `bin/wadb` and `bin/wk-ios` exist because Claude Code permission rules match on
|
|
151
|
+
the **first token** of a Bash command — a compound `export JAVA_HOME=…; node …` line can never
|
|
152
|
+
be pre-approved. Each wrapper resolves the environment internally and then execs the real tool.
|
|
153
|
+
|
|
154
|
+
### `bin/wadb` — raw adb, environment resolved
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
bin/wadb devices
|
|
158
|
+
bin/wadb logcat -d | tail -50
|
|
159
|
+
bin/wadb shell pm list packages | grep chrome
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
It uses the `adb` already on `PATH` when there is one, else the SDK copy under `ANDROID_HOME`
|
|
163
|
+
(default `~/Library/Android/sdk`).
|
|
164
|
+
|
|
165
|
+
### `bin/wk-ios` — real JS/DOM in iOS Simulator Safari
|
|
166
|
+
|
|
167
|
+
Maestro reads the native XCUITest accessibility tree, which collapses Safari's web content
|
|
168
|
+
into unlabeled nodes. The DOM itself is fully addressable through WebKit's Remote Inspector,
|
|
169
|
+
and `wk-ios` wires `ios-webkit-debug-proxy` up to the running Simulator so you can evaluate JS
|
|
170
|
+
in the frontmost page — the same power `playwright-cli eval` gives you on desktop, but on the
|
|
171
|
+
real iOS engine.
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
bin/wk-ios 'document.title'
|
|
175
|
+
bin/wk-ios 'document.querySelectorAll(".cart_item").length'
|
|
176
|
+
bin/wk-ios --url # print the page WebSocket URL
|
|
177
|
+
bin/wk-ios --stop # stop the proxy this script started (never anyone else's)
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Requires `ios-webkit-debug-proxy`, `python3`, Node ≥ 22.4 (`wkeval.mjs` uses the global
|
|
181
|
+
`WebSocket`), and a booted Simulator with Safari on a page. `WK_IOS_PORT` (default 9221) sets
|
|
182
|
+
the iwdp device-list port; pages are served on `WK_IOS_PORT + 1`.
|
|
183
|
+
|
|
63
184
|
## Why a Google **Play** (or `google_apis`) Android image
|
|
185
|
+
|
|
64
186
|
A stock **AOSP** system image ships **no `com.android.chrome`** — only a basic AOSP
|
|
65
187
|
browser or none. Web-mode Android targets need Chrome, so `setup-mobile.sh` installs a
|
|
66
188
|
`google_apis_playstore` image. Verify on any AVD with:
|
|
189
|
+
|
|
67
190
|
```bash
|
|
68
|
-
|
|
191
|
+
bin/wadb shell pm list packages | grep chrome
|
|
69
192
|
```
|
|
70
193
|
|
|
71
194
|
## Auth reality (web targets)
|
|
195
|
+
|
|
72
196
|
There is **no transferable `storage_state`** for a real device browser (unlike Playwright).
|
|
73
197
|
SSO + MFA logins are done **once, interactively, by a human** in the sim/emulator browser;
|
|
74
198
|
the device persists the browser profile across launches. Never script MFA. Never
|
|
@@ -76,17 +200,24 @@ the device persists the browser profile across launches. Never script MFA. Never
|
|
|
76
200
|
`notes:` blocks in `data/targets/_example-sim-*.yml`.
|
|
77
201
|
|
|
78
202
|
## Troubleshooting
|
|
203
|
+
|
|
79
204
|
- **`doctor-mobile.sh` says NOT READY** → fix each `✗` with the printed command, or run
|
|
80
205
|
`setup-mobile.sh`, then re-run the doctor.
|
|
81
206
|
- **`JAVA_HOME` / `adb` not found mid-session** → `bin/mcli` resolves JDK 17 via
|
|
82
|
-
`/usr/libexec/java_home -v 17`
|
|
83
|
-
Export them explicitly if your SDK lives
|
|
207
|
+
`/usr/libexec/java_home -v 17` (falling back to the Homebrew `openjdk@17` prefixes) and
|
|
208
|
+
defaults `ANDROID_HOME` to `~/Library/Android/sdk`. Export them explicitly if your SDK lives
|
|
209
|
+
elsewhere.
|
|
210
|
+
- **Maestro too old** → `mobile-cli` needs the JSON-only `maestro hierarchy` of 2.6.0+.
|
|
211
|
+
Reinstall pinned: `curl -Ls https://get.maestro.mobile.dev | MAESTRO_VERSION=2.10.0 bash`.
|
|
212
|
+
- **Every tap hits the wrong element** → your refs expired. Re-run `snapshot`; exit code 3
|
|
213
|
+
means exactly this.
|
|
84
214
|
- **Intel Mac** → `setup-mobile.sh` selects `x86_64` system images automatically (`arm64`
|
|
85
215
|
Macs get `arm64-v8a`).
|
|
86
216
|
- **Emulator slow/flaky to boot or Chrome absent** → fall back to the iOS Safari target;
|
|
87
217
|
both engines are independent.
|
|
88
218
|
|
|
89
219
|
## Scope
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
220
|
+
|
|
221
|
+
This covers iOS Simulator + Android Emulator (browser **and** native modes) on macOS. It does
|
|
222
|
+
**not** provision physical devices, CI runners, Windows/Linux hosts, or the desktop web path
|
|
223
|
+
(`/qa-explore`, which only needs `playwright-cli`).
|
package/package.json
CHANGED
|
@@ -1,65 +1,79 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "qualiow-exploratory-testing",
|
|
3
|
-
"version": "
|
|
4
|
-
"description": "qualiow — AI-powered exploratory testing skills
|
|
3
|
+
"version": "2.0.0",
|
|
4
|
+
"description": "qualiow — AI-powered exploratory testing skills for Claude Code: web via playwright-cli, iOS/Android simulators via Maestro/mobile-cli, and backend/API acceptance-criteria verification",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
7
7
|
"types": "dist/index.d.ts",
|
|
8
8
|
"bin": {
|
|
9
|
-
"qualiow": "dist/cli/index.js"
|
|
9
|
+
"qualiow": "dist/cli/index.js",
|
|
10
|
+
"mcli": "bin/mcli",
|
|
11
|
+
"wadb": "bin/wadb",
|
|
12
|
+
"wk-ios": "bin/wk-ios",
|
|
13
|
+
"qualiow-doctor-mobile": "bin/doctor-mobile.sh",
|
|
14
|
+
"qualiow-setup-mobile": "bin/setup-mobile.sh"
|
|
10
15
|
},
|
|
11
16
|
"files": [
|
|
12
17
|
"dist/",
|
|
13
18
|
"skills/",
|
|
19
|
+
"agents/",
|
|
14
20
|
"bin/",
|
|
15
|
-
"
|
|
16
|
-
"
|
|
17
|
-
"data/domains/",
|
|
21
|
+
".claude-plugin/",
|
|
22
|
+
"data/domains/*.yml",
|
|
18
23
|
"data/knowledge/",
|
|
19
24
|
"data/templates/",
|
|
20
|
-
"data/
|
|
21
|
-
"data/targets/
|
|
22
|
-
"data/targets/_example-sim-ios-safari.yml",
|
|
23
|
-
"data/targets/_example-sim-android-chrome.yml",
|
|
24
|
-
"data/targets/_example-mobile-emulation.yml",
|
|
25
|
-
"data/targets/_example-backend.yml",
|
|
25
|
+
"data/security/",
|
|
26
|
+
"data/targets/_*.yml",
|
|
26
27
|
"data/targets/testers-ai.yml",
|
|
27
28
|
"docs/GETTING-STARTED.md",
|
|
28
29
|
"docs/MOBILE-SETUP.md",
|
|
29
|
-
"docs/BACKEND-VERIFICATION.md"
|
|
30
|
+
"docs/BACKEND-VERIFICATION.md",
|
|
31
|
+
".env.example",
|
|
32
|
+
"CHANGELOG.md"
|
|
30
33
|
],
|
|
31
34
|
"scripts": {
|
|
32
|
-
"sync:
|
|
33
|
-
"prebuild": "npm run sync:
|
|
35
|
+
"sync:plugin": "rm -rf skills agents && cp -R .claude/skills skills && cp -R .claude/agents agents",
|
|
36
|
+
"prebuild": "npm run sync:plugin",
|
|
34
37
|
"build": "tsup",
|
|
35
38
|
"dev": "tsup --watch",
|
|
36
39
|
"test": "vitest run",
|
|
37
40
|
"test:watch": "vitest",
|
|
38
41
|
"test:coverage": "vitest run --coverage",
|
|
39
42
|
"validate": "node dist/cli/index.js validate --all",
|
|
43
|
+
"kb:sync": "node dist/cli/index.js kb sync",
|
|
44
|
+
"kb:check": "node dist/cli/index.js kb check",
|
|
45
|
+
"check:mirror": "diff -rq .claude/skills skills && diff -rq .claude/agents agents",
|
|
46
|
+
"check:pack": "node scripts/check-pack.mjs",
|
|
40
47
|
"lint": "tsc --noEmit",
|
|
41
|
-
"
|
|
48
|
+
"version": "node scripts/sync-version.mjs && git add .claude-plugin/plugin.json",
|
|
49
|
+
"prepublishOnly": "npm run build && npm run lint && npm test && npm run check:pack"
|
|
42
50
|
},
|
|
43
51
|
"keywords": [
|
|
44
52
|
"exploratory-testing",
|
|
45
53
|
"qa",
|
|
46
54
|
"testing",
|
|
47
55
|
"playwright",
|
|
56
|
+
"playwright-cli",
|
|
48
57
|
"maestro",
|
|
49
58
|
"mobile-testing",
|
|
50
59
|
"ios-simulator",
|
|
51
60
|
"android-emulator",
|
|
61
|
+
"api-testing",
|
|
62
|
+
"acceptance-criteria",
|
|
52
63
|
"ai",
|
|
53
64
|
"claude",
|
|
65
|
+
"claude-code",
|
|
54
66
|
"skills",
|
|
67
|
+
"plugin",
|
|
55
68
|
"heuristics",
|
|
56
69
|
"bug-detection"
|
|
57
70
|
],
|
|
58
71
|
"author": "William Oliveira",
|
|
59
72
|
"license": "MIT",
|
|
60
73
|
"engines": {
|
|
61
|
-
"node": ">=
|
|
74
|
+
"node": ">=22.4.0"
|
|
62
75
|
},
|
|
76
|
+
"packageManager": "npm@10.9.2",
|
|
63
77
|
"repository": {
|
|
64
78
|
"type": "git",
|
|
65
79
|
"url": "git+https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git"
|
|
@@ -69,7 +83,7 @@
|
|
|
69
83
|
"url": "https://github.com/willcoliveira/qualiow-exploratory-testing-skills/issues"
|
|
70
84
|
},
|
|
71
85
|
"dependencies": {
|
|
72
|
-
"@playwright/cli": "^0.1.
|
|
86
|
+
"@playwright/cli": "^0.1.19",
|
|
73
87
|
"chalk": "^5.3.0",
|
|
74
88
|
"commander": "^15.0.0",
|
|
75
89
|
"glob": "^13.0.6",
|
|
@@ -7,10 +7,11 @@ description: >
|
|
|
7
7
|
and writes bug reports that executives act on.
|
|
8
8
|
Use when user says: "explore", "test this site", "find bugs", "QA check", "exploratory session",
|
|
9
9
|
or provides a URL to test.
|
|
10
|
-
|
|
10
|
+
argument-hint: "<url> [--target <id>] [--context <file>] [--focus <area>] [--session <dir>]"
|
|
11
|
+
allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Bash(diff:*), Read, Write, Glob, Grep
|
|
11
12
|
---
|
|
12
13
|
|
|
13
|
-
# Exploratory Testing Session
|
|
14
|
+
# Exploratory Testing Session
|
|
14
15
|
|
|
15
16
|
## Your Identity
|
|
16
17
|
|
|
@@ -29,43 +30,48 @@ You are a **Principal QA Engineer** with 20+ years of experience. You don't just
|
|
|
29
30
|
# Mode 1: Blind exploration (default — always works, no prior info needed)
|
|
30
31
|
/qa-explore https://example.com
|
|
31
32
|
|
|
32
|
-
# Mode 2: Context-enriched (
|
|
33
|
-
/qa-explore https://example.com --context
|
|
33
|
+
# Mode 2: Context-enriched (a /qa-gather context file or any requirements document)
|
|
34
|
+
/qa-explore https://example.com --context output/context/checkout-context.md
|
|
34
35
|
|
|
35
36
|
# Mode 3: Inline context (user provides context in their message)
|
|
36
37
|
/qa-explore https://example.com
|
|
37
38
|
> "This is a new checkout flow. Should support Visa/MC, require delivery address,
|
|
38
39
|
> send confirmation email. Team just refactored the payment service."
|
|
39
40
|
|
|
40
|
-
# With target config
|
|
41
|
-
/qa-explore --target company-staging --context
|
|
41
|
+
# With a saved target config, or reusing a `qualiow explore` pre-flight directory
|
|
42
|
+
/qa-explore --target company-staging --context output/context/SI-2305-context.md
|
|
43
|
+
/qa-explore https://example.com --session output/sessions/2026-09-08-1420-explore-example-com
|
|
42
44
|
```
|
|
43
45
|
|
|
44
|
-
|
|
46
|
+
## Before You Start
|
|
45
47
|
|
|
46
|
-
|
|
48
|
+
- Paths, target resolution, the session-directory scheme and the `-s=<sid>` session id are defined once in **`${CLAUDE_SKILL_DIR}/references/paths.md`**. Read it first.
|
|
49
|
+
- Every artefact you write follows **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** (bug report, session report, `stats.json`, confidentiality header). `qualiow report` and the feedback skills only understand that format.
|
|
50
|
+
- **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** is absolute: production detection, redaction before disk, session isolation, output classification.
|
|
47
51
|
|
|
48
|
-
|
|
52
|
+
## Session Phases
|
|
49
53
|
|
|
50
|
-
|
|
51
|
-
|-------|------|---------|
|
|
52
|
-
| **Setup** | Read and follow: `phases/00-setup.md` | Parse input, resolve target, load context, load knowledge, create session directory |
|
|
53
|
-
| **Auth** | Read and follow: `phases/01-auth.md` | Authenticate via storage state, credentials, or token; verify login |
|
|
54
|
-
| **Charter** | Read and follow: `phases/02-charter.md` | Use app as real user, identify journeys, risk-rank features, select heuristics, write charter |
|
|
55
|
-
| **Discovery** | Read and follow: `phases/03-discovery.md` | Map the site, check console/network, enforce scope, find what's MISSING |
|
|
56
|
-
| **Journeys** | Read and follow: `phases/04-journeys.md` | End-to-end user journeys, data integrity verification, cross-page consistency |
|
|
57
|
-
| **Features** | Read and follow: `phases/05-features.md` | Deep feature testing with SFDIPOT, business logic stress, negative space, FEW HICCUPPS |
|
|
58
|
-
| **Edge Cases** | Read and follow: `phases/06-edge-cases.md` | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
|
|
59
|
-
| **Reporting** | Read and follow: `phases/07-reporting.md` | Stop recording, reflect, write bug reports with business impact, session report, coverage map |
|
|
54
|
+
Execute each phase in order. Read and follow the linked file. The budget sums to 45 minutes; write to disk after each phase and carry only summaries in context.
|
|
60
55
|
|
|
61
|
-
|
|
56
|
+
| Phase | File | Budget | Summary |
|
|
57
|
+
|-------|------|--------|---------|
|
|
58
|
+
| **Setup** | `${CLAUDE_SKILL_DIR}/phases/00-setup.md` | 2 min | Parse input, resolve target and data paths, load context and knowledge, create the session directory |
|
|
59
|
+
| **Auth** | `${CLAUDE_SKILL_DIR}/phases/01-auth.md` | 2 min | Authenticate via storage state, credentials, or token; verify login |
|
|
60
|
+
| **Charter** | `${CLAUDE_SKILL_DIR}/phases/02-charter.md` | 5 min | Use the app as a real user, identify journeys, risk-rank features, select heuristics, write the charter |
|
|
61
|
+
| **Discovery** | `${CLAUDE_SKILL_DIR}/phases/03-discovery.md` | 6 min | Map the site, check console and requests, enforce scope, find what's MISSING |
|
|
62
|
+
| **Journeys** | `${CLAUDE_SKILL_DIR}/phases/04-journeys.md` | 10 min | End-to-end user journeys, data-integrity verification, cross-page consistency |
|
|
63
|
+
| **Features** | `${CLAUDE_SKILL_DIR}/phases/05-features.md` | 10 min | Deep feature testing with SFDIPOT, business-logic stress, negative space, FEW HICCUPPS |
|
|
64
|
+
| **Edge Cases** | `${CLAUDE_SKILL_DIR}/phases/06-edge-cases.md` | 6 min | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
|
|
65
|
+
| **Reporting** | `${CLAUDE_SKILL_DIR}/phases/07-reporting.md` | 4 min | Stop recording, reflect, write bug reports with business impact, session report, stats, close the browser session |
|
|
62
66
|
|
|
63
67
|
## References
|
|
64
68
|
|
|
65
69
|
Consult these as needed throughout the session:
|
|
66
70
|
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
-
|
|
71
|
-
-
|
|
71
|
+
- **`${CLAUDE_SKILL_DIR}/references/paths.md`** — resolution order for targets, data, credentials and output; session-directory scheme; session id; index rows
|
|
72
|
+
- **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** — the one bug-report and session-report format, `stats.json`, confidentiality header
|
|
73
|
+
- **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** — prompt-injection resistance, production rule, redaction list, session isolation, output classification
|
|
74
|
+
- **`${CLAUDE_SKILL_DIR}/references/severity-guide.md`** — severity definitions, the "when in doubt go LOWER" rule, risk matrix, priority table
|
|
75
|
+
- **`${CLAUDE_SKILL_DIR}/references/session-rules.md`** — 20 testing-discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI-bias avoidance
|
|
76
|
+
- **`${CLAUDE_SKILL_DIR}/references/context-integration.md`** — how `--context` changes the session (context file, inline context, blind mode)
|
|
77
|
+
- **`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md`** — opt-in handoff to Playwright Test Agents (planner / generator / healer, Playwright 1.56+) when a reproducible bug should become a regression test; requires the optional `@playwright/test` peer dependency
|
|
@@ -1,44 +1,53 @@
|
|
|
1
|
-
# Phase 0: Session Setup
|
|
1
|
+
# Phase 0: Session Setup (2 min)
|
|
2
2
|
|
|
3
3
|
## Step 1: Parse Input & Resolve Target
|
|
4
4
|
|
|
5
|
-
Extract from user message:
|
|
6
|
-
- **target_url**:
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
5
|
+
Extract from the user message:
|
|
6
|
+
- **target_url**: the URL to test (required unless `--target` is given)
|
|
7
|
+
- **--target <id>**: a saved target config (optional)
|
|
8
|
+
- **--context <file>** or inline context text (optional)
|
|
9
|
+
- **--focus <area>**: a specific area or feature (optional)
|
|
10
|
+
- **--session <dir>**: an existing pre-flight directory created by `qualiow explore` (optional)
|
|
11
|
+
- **domain**: domain type (optional; auto-detect from the app if not given)
|
|
12
|
+
- **time_box**: 45 min (default and maximum)
|
|
13
|
+
|
|
14
|
+
Resolve the target config, the data directory (`<data>`) and the credentials file in the
|
|
15
|
+
order defined in `${CLAUDE_SKILL_DIR}/references/paths.md`: `--target` → `data/targets/<id>.yml`
|
|
16
|
+
in the project, else the plugin's copy; no `--target` → `qa/target.yml` in the project, else
|
|
17
|
+
`_default.yml`; credentials from `qa/.env`, else `.env`. Then read
|
|
18
|
+
`<data>/domains/<domain>.yml` for the domain's completeness checklist, data-integrity checks
|
|
19
|
+
and journeys.
|
|
20
|
+
|
|
21
|
+
Apply the **production rule** from `${CLAUDE_SKILL_DIR}/references/security-rules.md` to the
|
|
22
|
+
resolved hostname now. If it fires, log `[SAFETY] Production detected -- running in read-only mode`
|
|
23
|
+
and carry read-only mode through every later phase.
|
|
20
24
|
|
|
21
25
|
## Step 2: Load Context (if provided)
|
|
22
26
|
|
|
23
|
-
**If `--context <file>` provided:**
|
|
27
|
+
**If `--context <file>` provided:** read the file. It may be a `/qa-gather` context file
|
|
28
|
+
(structured markdown with ticket data, specs, MR diffs, design notes) or any document
|
|
29
|
+
describing what to test.
|
|
24
30
|
|
|
25
|
-
**If inline context provided:**
|
|
31
|
+
**If inline context provided:** extract from the user's message any descriptions of: what
|
|
32
|
+
the feature does, acceptance criteria, what changed, what the design should look like,
|
|
33
|
+
known risks, user personas.
|
|
26
34
|
|
|
27
|
-
**If no context provided:**
|
|
35
|
+
**If no context provided:** that's fine; run in blind exploration mode.
|
|
28
36
|
|
|
29
37
|
**When context IS available, extract and use:**
|
|
30
38
|
|
|
31
39
|
| Context Element | How It Changes the Session |
|
|
32
40
|
|----------------|---------------------------|
|
|
33
|
-
| **Acceptance criteria** | Add as explicit checkpoints
|
|
34
|
-
| **What changed** (MR diff, commits) | Focus testing on changed areas
|
|
35
|
-
| **Design specs** (Figma, mockups) | Compare live app against intended design
|
|
41
|
+
| **Acceptance criteria** | Add as explicit checkpoints; verify each AC during testing |
|
|
42
|
+
| **What changed** (MR diff, commits) | Focus testing on changed areas; highest regression risk |
|
|
43
|
+
| **Design specs** (Figma, mockups) | Compare the live app against the intended design |
|
|
36
44
|
| **User stories / requirements** | Test against INTENT, not just what's visible |
|
|
37
|
-
| **Known risks**
|
|
38
|
-
| **API contracts** (OpenAPI, endpoints) | Verify UI matches API
|
|
39
|
-
| **Previous test results** | Don't re-test what passed
|
|
45
|
+
| **Known risks** | Prioritize testing on flagged risk areas |
|
|
46
|
+
| **API contracts** (OpenAPI, endpoints) | Verify the UI matches API behaviour |
|
|
47
|
+
| **Previous test results** | Don't re-test what passed; focus on gaps |
|
|
40
48
|
|
|
41
|
-
**Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
|
|
49
|
+
**Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
|
|
50
|
+
with zero documentation should still get a full, valuable session.
|
|
42
51
|
|
|
43
52
|
When context is loaded, note in the charter:
|
|
44
53
|
```
|
|
@@ -49,17 +58,24 @@ CHANGE SCOPE: [what changed or "unknown -- full exploration"]
|
|
|
49
58
|
|
|
50
59
|
## Step 3: Load Knowledge
|
|
51
60
|
|
|
52
|
-
Read
|
|
61
|
+
Read `<data>/knowledge/manifest.yml`. Load:
|
|
53
62
|
1. Core heuristics (SFDIPOT, FEW HICCUPPS, Test Tours)
|
|
54
|
-
2. Domain-specific entries
|
|
55
|
-
3.
|
|
56
|
-
4. Business
|
|
63
|
+
2. Domain-specific entries from `loading_strategy.by_domain`
|
|
64
|
+
3. `<data>/knowledge/learned-patterns.md`: the false-positive skip list and must-check patterns
|
|
65
|
+
4. Business-logic and race-condition techniques (if the app has transactions or state changes)
|
|
66
|
+
|
|
67
|
+
Keep brief summaries only; don't fill context with full entries.
|
|
57
68
|
|
|
58
|
-
|
|
69
|
+
## Step 4: Create the Session Directory
|
|
59
70
|
|
|
60
|
-
|
|
71
|
+
Unless `--session <dir>` was given, create
|
|
72
|
+
`output/sessions/<YYYY-MM-DD-HHmm>-explore-<slug>/` (scheme in `paths.md`; `<slug>` is the
|
|
73
|
+
target id without a leading `_`, or the hostname with dots replaced by dashes) containing
|
|
74
|
+
`screenshots/`, `bugs/`, `videos/`, and an empty `session-log.md`. With `--session`, reuse
|
|
75
|
+
that directory and append to its `session-log.md`.
|
|
61
76
|
|
|
62
|
-
|
|
77
|
+
Choose the playwright-cli session id `-s=explore-<HHmm>-<slug>`. **Every `playwright-cli`
|
|
78
|
+
call in this session carries `-s=<sid>`; the examples in the phase files omit the prefix.**
|
|
63
79
|
|
|
64
80
|
## Step 5: Initialize Progress Tracking
|
|
65
81
|
|
|
@@ -67,6 +83,7 @@ Write `output/sessions/<session-dir>/progress.json`:
|
|
|
67
83
|
```json
|
|
68
84
|
{
|
|
69
85
|
"session_id": "<session-dir>",
|
|
86
|
+
"kind": "explore",
|
|
70
87
|
"target": "<target_url>",
|
|
71
88
|
"domain": "<domain>",
|
|
72
89
|
"started_at": "<ISO timestamp>",
|
|
@@ -91,27 +108,36 @@ Write `output/sessions/<session-dir>/progress.json`:
|
|
|
91
108
|
|
|
92
109
|
Append to `session-log.md`:
|
|
93
110
|
```
|
|
94
|
-
[<timestamp>] [PHASE] Setup complete — session
|
|
111
|
+
[<timestamp>] [PHASE] Setup complete — session <session-dir>, sid <sid>, read_only: <true|false>
|
|
95
112
|
```
|
|
96
113
|
|
|
97
114
|
## Step 6: Optional — Bootstrap Playwright Test Agents (opt-in)
|
|
98
115
|
|
|
99
|
-
**Skip this step entirely if `@playwright/test` is not installed
|
|
116
|
+
**Skip this step entirely if `@playwright/test` is not installed in the consumer project OR
|
|
117
|
+
no `playwright.config.*` is present.** This is an additive capability for teams that want to
|
|
118
|
+
turn reproducible findings into regression tests with the Playwright Test Agents framework
|
|
119
|
+
(planner / generator / healer, Playwright 1.56+).
|
|
100
120
|
|
|
101
|
-
If both prerequisites are present AND the user has opted in (via target config or
|
|
121
|
+
If both prerequisites are present AND the user has opted in (via the target config or the
|
|
122
|
+
context), you MAY offer to scaffold the agent workspace:
|
|
102
123
|
|
|
103
124
|
```bash
|
|
104
125
|
npx playwright init-agents --loop=claude
|
|
105
126
|
```
|
|
106
127
|
|
|
107
|
-
|
|
128
|
+
With `--loop=claude` this writes the three agent definitions into `.claude/agents/`, a
|
|
129
|
+
`.mcp.json` registering the `playwright-test` server, and a `specs/` scaffold with a seed
|
|
130
|
+
spec if the project has none; other loops: `codex`, `copilot`, `opencode`, `vscode`,
|
|
131
|
+
`vscode-legacy`. It does not touch `playwright.config.*`. See
|
|
132
|
+
`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md` for the hand-off after the
|
|
133
|
+
session.
|
|
108
134
|
|
|
109
|
-
**Never block the session on this step.** If the user declines,
|
|
135
|
+
**Never block the session on this step.** If the user declines, the prerequisites are
|
|
136
|
+
missing, or the command fails, log a note in `session-log.md` and continue.
|
|
110
137
|
|
|
111
|
-
**IMPORTANT: After EVERY phase completion, update progress.json and append to
|
|
138
|
+
**IMPORTANT: After EVERY phase completion, update `progress.json` and append to
|
|
139
|
+
`session-log.md`.** This is how the user monitors progress:
|
|
112
140
|
|
|
113
141
|
```
|
|
114
|
-
# Update progress.json — set current phase complete, next phase as current
|
|
115
|
-
# Append to session-log.md:
|
|
116
142
|
[<timestamp>] [PHASE] <phase_name> complete — <pages> pages, <bugs> bugs, <key finding summary>
|
|
117
143
|
```
|