qualiow-exploratory-testing 1.3.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +26 -0
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +259 -0
- package/README.md +472 -204
- package/agents/qa-diff-indexer-agent.md +81 -0
- package/agents/qa-gather-agent.md +41 -0
- package/agents/qa-page-mapper-agent.md +74 -0
- package/agents/qa-reporting-agent.md +103 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/bin/qualiow +30 -0
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/knowledge/changelog.yml +30 -12
- package/data/knowledge/manifest.yml +355 -127
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +3 -3
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +9 -6
- package/data/templates/expected-behaviour.md +3 -2
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +2286 -701
- package/dist/index.d.ts +637 -81
- package/dist/index.js +2058 -228
- package/docs/BACKEND-VERIFICATION.md +39 -9
- package/docs/GETTING-STARTED.md +300 -121
- package/docs/MOBILE-SETUP.md +149 -18
- package/hooks/hooks.json +27 -0
- package/hooks/scripts/read-guard.mjs +95 -0
- package/hooks/scripts/secret-patterns.d.mts +8 -0
- package/hooks/scripts/secret-patterns.mjs +136 -0
- package/hooks/scripts/write-guard.mjs +87 -0
- package/package.json +33 -18
- package/skills/qa-explore/SKILL.md +32 -25
- package/skills/qa-explore/phases/00-setup.md +76 -42
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +93 -27
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +102 -142
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/delegation-rules.md +82 -0
- package/skills/qa-explore/references/output-contract.md +192 -0
- package/skills/qa-explore/references/paths.md +108 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +70 -25
- package/skills/qa-explore/references/session-rules.md +2 -1
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +51 -15
- package/skills/qa-explore-feedback/SKILL.md +23 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +108 -60
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +71 -26
- package/skills/qa-explore-report/SKILL.md +65 -35
- package/skills/qa-gather/SKILL.md +44 -8
- package/skills/qa-knowledge-add/SKILL.md +25 -9
- package/skills/qa-knowledge-list/SKILL.md +37 -40
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +31 -19
- package/skills/qa-verify-backend/phases/00-setup.md +57 -12
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
- package/skills/qa-verify-backend/phases/02-static-review.md +11 -0
- package/skills/qa-verify-backend/phases/03-live-verification.md +5 -3
- package/skills/qa-verify-backend/phases/03b-api-verification.md +24 -9
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
- package/skills/qa-verify-backend/phases/05-reporting.md +72 -12
- package/skills/qa-verify-backend/references/api-probes.md +28 -1
- package/skills/qa-verify-backend/references/payload-verification.md +1 -1
- package/skills/qa-verify-backend/references/release-readiness.md +1 -1
- package/skills/qa-verify-backend/references/safety-rules.md +13 -8
- package/data/domains/_default.md +0 -181
- package/data/domains/ecommerce.md +0 -192
- package/data/domains/fintech.md +0 -200
- package/data/domains/identity.md +0 -88
- package/data/domains/marketing.md +0 -194
- package/data/domains/saas.md +0 -197
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
|
@@ -1,79 +1,83 @@
|
|
|
1
1
|
# Phase 0: Mobile Session Setup (mode-aware)
|
|
2
2
|
|
|
3
|
-
Follow
|
|
4
|
-
context, load knowledge, create session dir, init progress.json). The mobile
|
|
5
|
-
and the native-vs-web branch, are below.
|
|
3
|
+
Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/00-setup.md` for the general structure (parse
|
|
4
|
+
input, load context, load knowledge, create the session dir, init progress.json). The mobile
|
|
5
|
+
differences, and the native-vs-web branch, are below. There is no playwright-cli session and
|
|
6
|
+
therefore no `-s=<sid>` on mobile.
|
|
6
7
|
|
|
7
|
-
## Step 0 —
|
|
8
|
+
## Step 0 — Resolve the driver, then preflight (fail fast on a fresh machine)
|
|
8
9
|
|
|
9
|
-
|
|
10
|
-
|
|
10
|
+
Resolve the driver prefix once, in the order from
|
|
11
|
+
`${CLAUDE_SKILL_DIR}/../qa-explore/references/paths.md`: `qa/bin/mcli` (after `qualiow init`)
|
|
12
|
+
→ `bin/mcli` (a checkout of this repository) → `mcli` on PATH → the plugin's `bin/mcli`.
|
|
13
|
+
Write that literal prefix in every command for the rest of the session. **This and every
|
|
14
|
+
other phase file show `qa/bin/mcli` and `qa/bin/wadb`; substitute yours.**
|
|
15
|
+
|
|
16
|
+
Then verify the toolchain exists (Maestro plus the iOS and/or Android SDKs, which are NOT
|
|
17
|
+
installed by default and are easy to assume present):
|
|
11
18
|
|
|
12
19
|
```bash
|
|
13
|
-
|
|
20
|
+
qa/bin/doctor-mobile.sh # or --ios / --android to scope to one platform
|
|
14
21
|
```
|
|
15
22
|
|
|
16
23
|
If it prints **NOT READY**, STOP: surface the `✗` items to the user and point them at
|
|
17
|
-
`
|
|
18
|
-
|
|
19
|
-
`
|
|
24
|
+
`qa/bin/setup-mobile.sh` (same prefix you resolved above; auto-installs the scriptable prerequisites; Xcode + the iOS runtime
|
|
25
|
+
are guided-manual). Do NOT start a session on a half-provisioned machine; a missing `adb` /
|
|
26
|
+
`maestro` / runtime fails cryptically deep in the run. Full guide: `docs/MOBILE-SETUP.md`.
|
|
20
27
|
|
|
21
28
|
## Step 1 — Parse target + DETECT MODE
|
|
22
29
|
|
|
23
|
-
|
|
30
|
+
Resolve the target as `paths.md` says (`--target <id>` → `data/targets/<id>.yml`; no
|
|
31
|
+
`--target` → `qa/target.yml`), then extract:
|
|
24
32
|
|
|
25
33
|
- **platform**: `android` | `ios` (required).
|
|
26
34
|
- **device**: emulator/sim identifier.
|
|
27
35
|
- Android: AVD name (from `device.avd`) or `emulator-5554` if already booted.
|
|
28
|
-
- iOS: simulator name (e.g. `
|
|
36
|
+
- iOS: simulator name (e.g. `qa-iphone`) or UDID (`device.name` / `device.udid`).
|
|
29
37
|
- **app id**: `app.bundle_id` (iOS) / `app.package` (Android).
|
|
30
|
-
- **mode
|
|
38
|
+
- **mode**, derived:
|
|
31
39
|
- **WEB mode** if the app id is a known browser bundle (`com.apple.mobilesafari` /
|
|
32
40
|
`com.android.chrome`) AND the config has a `web.base_url`. The browser is the driver; the
|
|
33
41
|
web URL is the SUT. Templates: `data/targets/_example-sim-ios-safari.yml`,
|
|
34
|
-
`data/targets/_example-sim-android-chrome.yml` (read the matching one
|
|
42
|
+
`data/targets/_example-sim-android-chrome.yml` (read the matching one; their `notes:`
|
|
35
43
|
blocks document the web setup).
|
|
36
|
-
- **NATIVE mode** otherwise
|
|
44
|
+
- **NATIVE mode** otherwise: a non-browser app id with installable artifacts
|
|
37
45
|
(`app.app_paths` / `app.apk_paths`), optionally a `source_repo` with `build_commands`.
|
|
38
46
|
The installed app is the SUT. Template: `data/targets/_example-native-mobile.yml`.
|
|
39
47
|
- **WEB only**: `web.base_url` / `web.start_url`.
|
|
40
48
|
- **NATIVE only**: `app.app_paths`/`apk_paths` (artifact to verify/install), and optional
|
|
41
49
|
`source_repo.path` + `source_repo.build_commands`.
|
|
42
50
|
|
|
43
|
-
Record `mode
|
|
51
|
+
Record `mode`; every later step branches on it. Apply the **production rule** from
|
|
52
|
+
`security-rules.md` to `web.base_url` (WEB) or to the target's `environment`/`notes` (NATIVE)
|
|
53
|
+
and carry read-only mode through the session if it fires.
|
|
44
54
|
|
|
45
55
|
## Step 2 — Boot the simulator/emulator if needed (shared)
|
|
46
56
|
|
|
47
57
|
```bash
|
|
48
58
|
# Android — check first, only boot if no device attached
|
|
49
|
-
bin/wadb devices
|
|
50
|
-
#
|
|
51
|
-
# export ANDROID_HOME=$HOME/Library/Android/sdk
|
|
52
|
-
# export PATH=$ANDROID_HOME/platform-tools:$ANDROID_HOME/emulator:$PATH
|
|
53
|
-
# nohup emulator -avd <avd-name> -no-snapshot -no-boot-anim -memory 4096 -gpu auto & disown
|
|
54
|
-
# adb wait-for-device
|
|
59
|
+
qa/bin/wadb devices
|
|
60
|
+
qa/bin/mcli boot <avd-name> # ONLY if the line above listed no device
|
|
55
61
|
|
|
56
62
|
# iOS — check first, only boot if no sim booted
|
|
57
63
|
xcrun simctl list devices booted
|
|
58
|
-
#
|
|
59
|
-
# xcrun simctl boot <UDID-or-name>
|
|
60
|
-
# open -a Simulator
|
|
64
|
+
qa/bin/mcli boot <sim-name-or-UDID> # boots and opens Simulator.app
|
|
61
65
|
```
|
|
62
66
|
|
|
63
|
-
**Do NOT reboot a sim/emulator that's already running
|
|
64
|
-
mode) the persisted browser profile that holds the logged-in session.
|
|
67
|
+
**Do NOT reboot a sim/emulator that's already running**: preserve installed apps and (WEB
|
|
68
|
+
mode) the persisted browser profile that holds the logged-in session. On iOS `boot` is safe
|
|
69
|
+
to re-run (an already-booted sim is accepted); on **Android it always spawns a NEW emulator
|
|
70
|
+
process** and, if that AVD is already up, blocks for 180 s and then fails. Check first.
|
|
65
71
|
|
|
66
|
-
|
|
72
|
+
Point the driver at the device (this also picks a **per-device** state file,
|
|
73
|
+
`~/.mobile-cli/<device>.json`). If another session is driving the same device, pass your own
|
|
74
|
+
`--state <file>` on every command or the two will overwrite each other's refs:
|
|
67
75
|
|
|
68
76
|
```bash
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
$MCLI set-device <emulator-5554 OR ios-udid>
|
|
77
|
+
qa/bin/mcli set-device <emulator-5554 | ios sim name or UDID> --platform <android|ios>
|
|
78
|
+
qa/bin/mcli info
|
|
72
79
|
```
|
|
73
80
|
|
|
74
|
-
Use a dedicated `MOBILE_CLI_STATE` file per target so concurrent/sequential sessions don't
|
|
75
|
-
clobber each other's device/app state.
|
|
76
|
-
|
|
77
81
|
---
|
|
78
82
|
|
|
79
83
|
## Step 3 — NATIVE branch: verify / install / (optional rebuild) / launch
|
|
@@ -84,7 +88,7 @@ Skip this whole step in WEB mode.
|
|
|
84
88
|
|
|
85
89
|
```bash
|
|
86
90
|
# Android
|
|
87
|
-
bin/wadb shell pm list packages | grep <app.package>
|
|
91
|
+
qa/bin/wadb shell pm list packages | grep <app.package>
|
|
88
92
|
# iOS
|
|
89
93
|
xcrun simctl get_app_container <udid> <app.bundle_id> 2>/dev/null
|
|
90
94
|
```
|
|
@@ -92,7 +96,7 @@ xcrun simctl get_app_container <udid> <app.bundle_id> 2>/dev/null
|
|
|
92
96
|
### 3b. Install if missing
|
|
93
97
|
|
|
94
98
|
If not installed and the config provides an artifact path, install from it:
|
|
95
|
-
- Android: `bin/wadb install -r <app.apk_paths[...]>`
|
|
99
|
+
- Android: `qa/bin/wadb install -r <app.apk_paths[...]>`
|
|
96
100
|
- iOS: `xcrun simctl install <udid> <app.app_paths[...]>`
|
|
97
101
|
|
|
98
102
|
If not installed and no artifact path is configured, **stop and ask the user** how to obtain
|
|
@@ -103,13 +107,12 @@ build command.
|
|
|
103
107
|
|
|
104
108
|
**Skip unless the user passed `--rebuild` OR the session goal explicitly mentions "latest" /
|
|
105
109
|
"fresh" / "newest" build.** Otherwise run against whatever is installed (faster, preserves
|
|
106
|
-
state). This skill does **not** assume any build framework
|
|
110
|
+
state). This skill does **not** assume any build framework; it runs the EXACT commands the
|
|
107
111
|
target config provides:
|
|
108
112
|
|
|
109
113
|
```bash
|
|
110
114
|
# 1. Drift check (using the target's source_repo.build_commands.refresh, if defined)
|
|
111
|
-
|
|
112
|
-
git status -sb && git fetch && git log HEAD..@{u} --oneline
|
|
115
|
+
git -C <source_repo.path> status -sb && git -C <source_repo.path> fetch && git -C <source_repo.path> log HEAD..@{u} --oneline
|
|
113
116
|
|
|
114
117
|
# 2. Decide WITH the user before pulling:
|
|
115
118
|
# clean + behind → safe to pull then rebuild; dirty → ask (stash/commit/build-as-is);
|
|
@@ -117,12 +120,10 @@ git status -sb && git fetch && git log HEAD..@{u} --oneline
|
|
|
117
120
|
|
|
118
121
|
# 3. Run the platform build command verbatim from source_repo.build_commands
|
|
119
122
|
# (e.g. build_commands.android_release / build_commands.ios_debug). These are
|
|
120
|
-
# project-defined; the skill does not hardcode them.
|
|
121
|
-
# background and
|
|
122
|
-
# nohup <the configured build command> > /tmp/mobile-build.log 2>&1 &
|
|
123
|
-
# BUILD_PID=$!; until ! kill -0 $BUILD_PID 2>/dev/null; do sleep 15; done
|
|
123
|
+
# project-defined; the skill does not hardcode them. Run long builds in the
|
|
124
|
+
# background and wait for completion before installing.
|
|
124
125
|
|
|
125
|
-
# 4. After install, cold-start fresh:
|
|
126
|
+
# 4. After install, cold-start fresh: qa/bin/mcli relaunch-clean (Android) or stop + launch (iOS)
|
|
126
127
|
```
|
|
127
128
|
|
|
128
129
|
**Always log the source state in the report** (branch + sha + clean/dirty) so readers know
|
|
@@ -131,9 +132,9 @@ what code the session ran against.
|
|
|
131
132
|
### 3d. Launch the app
|
|
132
133
|
|
|
133
134
|
```bash
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
135
|
+
qa/bin/mcli set-app <app.bundle_id OR app.package>
|
|
136
|
+
qa/bin/mcli launch
|
|
137
|
+
qa/bin/mcli snapshot # sanity check — first screen renders
|
|
137
138
|
```
|
|
138
139
|
|
|
139
140
|
---
|
|
@@ -143,23 +144,24 @@ $MCLI snapshot # sanity check — first scre
|
|
|
143
144
|
Skip this step in NATIVE mode.
|
|
144
145
|
|
|
145
146
|
On Android, confirm the browser exists first (Safari ships with every iOS runtime; Chrome is
|
|
146
|
-
only guaranteed on **Google-APIs/Play** images
|
|
147
|
+
only guaranteed on **Google-APIs/Play** images; stock AOSP images may lack it):
|
|
147
148
|
|
|
148
149
|
```bash
|
|
149
|
-
bin/wadb shell pm list packages | grep chrome # expect: package:com.android.chrome
|
|
150
|
+
qa/bin/wadb shell pm list packages | grep chrome # expect: package:com.android.chrome
|
|
150
151
|
```
|
|
151
152
|
|
|
152
153
|
If `com.android.chrome` is absent, use a Play-image AVD (or fall back to the iOS Safari
|
|
153
154
|
target) rather than assuming Chrome is installed. Do not attempt to install a browser.
|
|
154
155
|
|
|
155
156
|
```bash
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
157
|
+
qa/bin/mcli set-app <com.apple.mobilesafari OR com.android.chrome>
|
|
158
|
+
qa/bin/mcli launch
|
|
159
|
+
qa/bin/mcli open-url <web.base_url or web.start_url>
|
|
160
|
+
qa/bin/mcli wait-text "<a word from the landing page>" 15
|
|
161
|
+
qa/bin/mcli snapshot # sanity check — the page renders
|
|
160
162
|
```
|
|
161
163
|
|
|
162
|
-
On a fresh Chrome profile you may see a one-time first-run / "Open tabs" promo bubble
|
|
164
|
+
On a fresh Chrome profile you may see a one-time first-run / "Open tabs" promo bubble;
|
|
163
165
|
`mobile-cli` filters the routine promo out of snapshots, but if a full-screen
|
|
164
166
|
first-run/sign-in-to-Chrome interstitial appears, it must be dismissed once by hand.
|
|
165
167
|
|
|
@@ -167,20 +169,27 @@ first-run/sign-in-to-Chrome interstitial appears, it must be dismissed once by h
|
|
|
167
169
|
|
|
168
170
|
## Step 5 — Session directory (shared)
|
|
169
171
|
|
|
170
|
-
|
|
172
|
+
Per the scheme in `paths.md`, with the platform in the slug:
|
|
171
173
|
|
|
172
174
|
```
|
|
173
|
-
output/sessions/<YYYY-MM-DD-HHmm
|
|
175
|
+
output/sessions/<YYYY-MM-DD-HHmm>-mobile-<target>-<android|ios>/
|
|
174
176
|
charter.md
|
|
175
177
|
session-log.md
|
|
178
|
+
progress.json
|
|
176
179
|
screenshots/
|
|
177
180
|
bugs/
|
|
178
|
-
videos/ #
|
|
181
|
+
videos/ # record-start output (.mp4)
|
|
182
|
+
logs/ # process-log excerpts per bug (logs/BUG-NNN.log)
|
|
179
183
|
```
|
|
180
184
|
|
|
185
|
+
Create all four subdirectories now — `screenshots/`, `bugs/`, `videos/` **and `logs/`**.
|
|
186
|
+
Phase 7 redirects log output into `logs/` with `>`, which fails outright if the directory
|
|
187
|
+
does not exist.
|
|
188
|
+
|
|
181
189
|
## Step 6 — progress.json additions (shared)
|
|
182
190
|
|
|
183
|
-
Same base schema as qa-explore
|
|
191
|
+
Same base schema as qa-explore with `"kind": "mobile"`. Add a `platform` key including the
|
|
192
|
+
resolved `mode`:
|
|
184
193
|
|
|
185
194
|
```json
|
|
186
195
|
"platform": {
|
|
@@ -188,16 +197,16 @@ Same base schema as qa-explore. Add a `platform` key including the resolved `mod
|
|
|
188
197
|
"os": "android",
|
|
189
198
|
"device": "emulator-5554",
|
|
190
199
|
"app_id": "com.example.myapp",
|
|
191
|
-
"app_version": "<from `
|
|
200
|
+
"app_version": "<from `qa/bin/wadb shell dumpsys package <pkg> | grep versionName`>",
|
|
192
201
|
"target_url": null
|
|
193
202
|
}
|
|
194
203
|
```
|
|
195
204
|
|
|
196
205
|
For WEB mode, set `"mode": "web"`, `app_id` to the browser bundle, and `target_url` to the
|
|
197
|
-
`base_url` (leave `app_version` null
|
|
206
|
+
`base_url` (leave `app_version` null; it's the browser's, not the SUT's).
|
|
198
207
|
|
|
199
208
|
The `console_errors` counter from qa-explore becomes `app_log_errors` (app/browser-process
|
|
200
|
-
ERROR-level log matches
|
|
209
|
+
ERROR-level log matches; coarser than a real JS console, see Phase 3).
|
|
201
210
|
|
|
202
|
-
**Skip qa-explore's Step 6 (Playwright Test Agents bootstrap) entirely
|
|
211
|
+
**Skip qa-explore's Step 6 (Playwright Test Agents bootstrap) entirely; not applicable when
|
|
203
212
|
the driver is mobile-cli/Maestro.**
|
|
@@ -8,12 +8,7 @@ the device. Pick the branch by the target's `auth.strategy`:
|
|
|
8
8
|
human login the browser profile persists).
|
|
9
9
|
- `auth.strategy: none` → skip this phase.
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
```bash
|
|
14
|
-
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
15
|
-
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
16
|
-
```
|
|
11
|
+
Credential values come from `qa/.env`, else `.env` (see `paths.md`); never echo them.
|
|
17
12
|
|
|
18
13
|
---
|
|
19
14
|
|
|
@@ -27,14 +22,14 @@ the app each run (or reuse an existing account already on the device).
|
|
|
27
22
|
|
|
28
23
|
```bash
|
|
29
24
|
# Optional cold start (Android): relaunch-clean wipes app data, then cold-starts.
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
25
|
+
qa/bin/mcli relaunch-clean
|
|
26
|
+
|
|
27
|
+
qa/bin/mcli snapshot # welcome / landing
|
|
28
|
+
qa/bin/mcli click <create-account / sign-up ref>
|
|
29
|
+
qa/bin/mcli snapshot # email / details screen
|
|
30
|
+
qa/bin/mcli fill <email input ref> "<config test_email_pattern, e.g. qa+$(date +%s)@example.com>"
|
|
31
|
+
qa/bin/mcli click <continue ref>
|
|
32
|
+
qa/bin/mcli snapshot # OTP / verification screen
|
|
38
33
|
```
|
|
39
34
|
|
|
40
35
|
**OTP on staging:** if the target config records a fixed code (`auth.static_otp`), use it
|
|
@@ -42,21 +37,23 @@ instead of waiting for a real OTP. Multi-cell OTP inputs often **do not accept a
|
|
|
42
37
|
`fill`** — tap each cell and send one digit at a time:
|
|
43
38
|
|
|
44
39
|
```bash
|
|
45
|
-
$MCLI snapshot # find the per-cell OTP refs
|
|
46
40
|
for d in <digits of static_otp>; do
|
|
47
|
-
|
|
48
|
-
|
|
41
|
+
qa/bin/mcli snapshot # refs expire after 60 s — re-read per cell
|
|
42
|
+
qa/bin/mcli fill <next-cell-ref> "$d"
|
|
49
43
|
done
|
|
50
44
|
```
|
|
51
45
|
|
|
46
|
+
Each `fill` spawns a Maestro flow (~3-5 s), so six digits can outlast the 60 s ref TTL and
|
|
47
|
+
abort with exit 3. Re-`snapshot` inside the loop as above, or raise `MCLI_REF_TTL_MS`.
|
|
48
|
+
|
|
52
49
|
### Existing-user login (faster for post-auth flows)
|
|
53
50
|
|
|
54
51
|
```bash
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
52
|
+
qa/bin/mcli launch # do NOT relaunch-clean — preserves the existing account
|
|
53
|
+
qa/bin/mcli snapshot # if on landing, tap Log In
|
|
54
|
+
qa/bin/mcli click <login ref>
|
|
55
|
+
qa/bin/mcli fill <email/username input ref> "<existing test account>"
|
|
56
|
+
qa/bin/mcli click <continue ref>
|
|
60
57
|
# OTP same as above (static_otp on staging)
|
|
61
58
|
```
|
|
62
59
|
|
|
@@ -75,10 +72,10 @@ Google Workspace, Auth0, …) — see the web target templates (`_example-sim-io
|
|
|
75
72
|
`_example-sim-android-chrome.yml`).
|
|
76
73
|
|
|
77
74
|
```bash
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
75
|
+
qa/bin/mcli launch # do NOT relaunch-clean — that wipes the profile
|
|
76
|
+
qa/bin/mcli open-url <web.start_url e.g. .../auth/login>
|
|
77
|
+
qa/bin/mcli snapshot # login surface / SSO button
|
|
78
|
+
qa/bin/mcli click <sign-in / SSO button ref>
|
|
82
79
|
# → hands off to the identity provider (e.g. login.microsoftonline.com, your Okta org, …).
|
|
83
80
|
# A HUMAN completes username + password + MFA by hand in the sim/emulator UI.
|
|
84
81
|
```
|
|
@@ -90,9 +87,9 @@ reappears mid-session, the session lapsed — hand back to a human; do not fill
|
|
|
90
87
|
### Returning session (already logged in)
|
|
91
88
|
|
|
92
89
|
```bash
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
90
|
+
qa/bin/mcli launch
|
|
91
|
+
qa/bin/mcli open-url <web.base_url>
|
|
92
|
+
qa/bin/mcli snapshot # expect an authenticated landing view, not login
|
|
96
93
|
```
|
|
97
94
|
|
|
98
95
|
If you see the login screen or IdP domain, re-do the interactive login above.
|
|
@@ -107,14 +104,14 @@ If you see the login screen or IdP domain, re-do the interactive login above.
|
|
|
107
104
|
|
|
108
105
|
## Verify Auth (shared)
|
|
109
106
|
|
|
110
|
-
After auth,
|
|
107
|
+
After auth, `qa/bin/mcli snapshot` and confirm authenticated indicators are present:
|
|
111
108
|
- NATIVE: the app's post-login home/landing controls.
|
|
112
109
|
- WEB: a known post-login heading, nav, or the user's name.
|
|
113
110
|
|
|
114
111
|
If stuck on a login/IdP screen, capture the log:
|
|
115
112
|
|
|
116
113
|
```bash
|
|
117
|
-
|
|
114
|
+
qa/bin/mcli logs --errors --since 60
|
|
118
115
|
```
|
|
119
116
|
|
|
120
117
|
**Auth issues to flag as bugs (not session blockers):**
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# Phase 2: Business Context & Charter (Mobile)
|
|
2
2
|
|
|
3
|
-
Follow
|
|
3
|
+
Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/02-charter.md` for the structure (use the app as a user,
|
|
4
4
|
identify journeys, risk-rank, select heuristics, write charter). The mobile additions below
|
|
5
5
|
apply to both modes.
|
|
6
6
|
|
|
7
7
|
## Use the app as a real user — for ~3 min
|
|
8
8
|
|
|
9
|
-
Drive the system under test via
|
|
9
|
+
Drive the system under test via `qa/bin/mcli snapshot` → `click` → `snapshot` loops to reach entry
|
|
10
10
|
points (NATIVE: navigate the app's screens; WEB: `open-url` plus in-page links). Don't test
|
|
11
11
|
yet — just observe what it exists to do, on a phone-sized screen, the way a real mobile user
|
|
12
12
|
would. Note the primary task, where money/data/trust is at stake, and what a frustrated user
|
|
@@ -44,13 +44,17 @@ dimension** appropriate to the feature under charter, e.g.:
|
|
|
44
44
|
Replace qa-explore's `tracing-start` with a video:
|
|
45
45
|
|
|
46
46
|
```bash
|
|
47
|
-
|
|
47
|
+
qa/bin/mcli record-start output/sessions/<session-dir>/videos/session.mp4 # a real .mp4 on both platforms
|
|
48
48
|
```
|
|
49
49
|
|
|
50
|
+
On **Android** one recording is capped at 180 s and everything after it is lost with no
|
|
51
|
+
error: `record-stop` / `record-start` per journey instead of one session-long clip. iOS has
|
|
52
|
+
no cap.
|
|
53
|
+
|
|
50
54
|
If recording fails or is too heavy, take frequent screenshots instead. Recording is
|
|
51
55
|
nice-to-have, not required.
|
|
52
56
|
|
|
53
|
-
## Charter additions (append to the qa-explore charter template)
|
|
57
|
+
## Charter additions (append to the qa-explore charter template; confidentiality header first)
|
|
54
58
|
|
|
55
59
|
```markdown
|
|
56
60
|
## Mobile Context
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Phase 3: Discovery & Mapping (Mobile)
|
|
2
2
|
|
|
3
|
-
Follow
|
|
3
|
+
Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/03-discovery.md` for goal and method (map structure,
|
|
4
4
|
refine risk). The substitutions below apply to both modes.
|
|
5
5
|
|
|
6
6
|
## Map screens/views by name + nav path (not URLs)
|
|
@@ -10,9 +10,9 @@ the nav path through the app; WEB: the URL that `open-url`/a link loaded, record
|
|
|
10
10
|
For each reachable screen/view:
|
|
11
11
|
|
|
12
12
|
```bash
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
13
|
+
qa/bin/mcli snapshot # refs expire after 60 s — re-snapshot before acting
|
|
14
|
+
qa/bin/mcli screenshot output/sessions/<session-dir>/screenshots/discovery-<view>.png
|
|
15
|
+
qa/bin/mcli logs --errors --since 30
|
|
16
16
|
```
|
|
17
17
|
|
|
18
18
|
Log each: `[DISCOVERED] <view-name> — <one-line description> — Risk: <P0/P1/P2/P3> — entry: <how-to-reach>`
|
|
@@ -22,18 +22,23 @@ Log each: `[DISCOVERED] <view-name> — <one-line description> — Risk: <P0/P1/
|
|
|
22
22
|
`snapshot` reads the device **accessibility tree, not the DOM**, so you identify elements by
|
|
23
23
|
their **visible text and accessibility labels** (headings, links, buttons, inputs), not by
|
|
24
24
|
CSS selectors or a fixed testID map. Read the live snapshot tree on each view. When a control
|
|
25
|
-
has no usable accessible label, fall back to
|
|
25
|
+
has no usable accessible label, fall back to `snapshot --full`, which assigns a ref to every
|
|
26
|
+
bounded node — `click <ref>` taps that node's centre on both platforms. A raw coordinate tap
|
|
27
|
+
exists only on Android (`qa/bin/wadb shell input tap <x> <y>`); the driver has no iOS
|
|
28
|
+
equivalent.
|
|
26
29
|
|
|
27
30
|
## App/browser log instead of console
|
|
28
31
|
|
|
29
32
|
Replace `playwright-cli console error` with:
|
|
30
33
|
|
|
31
34
|
```bash
|
|
32
|
-
|
|
35
|
+
qa/bin/mcli logs --errors --since 60
|
|
33
36
|
```
|
|
34
37
|
|
|
35
|
-
Caveat: this is the **process log
|
|
36
|
-
app/browser process
|
|
38
|
+
Caveat: this is the **process log**, and only iOS is process-scoped: `simctl spawn log show`
|
|
39
|
+
filters on the app/browser process, while Android's `logcat` is the **whole device** at `*:E`
|
|
40
|
+
/ `*:W` (last 400 lines), so lines from other packages appear — correlate by tag/pid before
|
|
41
|
+
attributing one to the app. In WEB mode it is the BROWSER's log, NOT the page's JS console, so it
|
|
37
42
|
catches gross failures (renderer crashes) but misses most page-level `console.error`. In
|
|
38
43
|
NATIVE mode it is the app's own log (crashes, native exceptions, bridge errors). Treat a clean
|
|
39
44
|
log as "no gross crash", not "no errors". Still watch for:
|
|
@@ -65,12 +70,13 @@ Beyond the qa-explore "what's not here" check, ask:
|
|
|
65
70
|
On iOS, some views **aggregate child elements into one parent accessibility label** (e.g. a
|
|
66
71
|
form returns one big comma-joined label of several fields). When you see a single ref with a
|
|
67
72
|
comma-joined label of many controls, that's an iOS a11y aggregation issue — file as a Medium
|
|
68
|
-
bug (it blocks automation and hurts VoiceOver users) and
|
|
69
|
-
|
|
73
|
+
bug (it blocks automation and hurts VoiceOver users) and drive that view with
|
|
74
|
+
`snapshot --full` refs for the rest of the session (on Android you can also fall back to
|
|
75
|
+
`qa/bin/wadb shell input tap <x> <y>`; iOS has no raw-coordinate tap).
|
|
70
76
|
|
|
71
77
|
## After Discovery
|
|
72
78
|
|
|
73
|
-
Write findings to `phase-
|
|
79
|
+
Write findings to `phase-3-discovery.md` (confidentiality header first) including:
|
|
74
80
|
- View map (table: view → key visible labels/controls → entry path → URL if known)
|
|
75
81
|
- Log error summary (count by level) with the browser-log caveat noted
|
|
76
82
|
- iOS a11y aggregation hits (if any)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Phase 4: End-to-End User Journeys (Mobile)
|
|
2
2
|
|
|
3
|
-
Follow
|
|
3
|
+
Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/04-journeys.md` for goal and the data-integrity
|
|
4
4
|
discipline. Build the journey templates from the charter's top user goals. Generic
|
|
5
5
|
mobile journey shapes to instantiate (apply to both modes):
|
|
6
6
|
|
|
@@ -10,7 +10,7 @@ mobile journey shapes to instantiate (apply to both modes):
|
|
|
10
10
|
2. **Create / edit / save** — create or edit a record, save it, navigate away and back, confirm the change is reflected (not just an optimistic UI that drops on reload/relaunch).
|
|
11
11
|
3. **Multi-step flow / wizard** — go through a multi-step flow start to finish; then go partway, back out, and resume — does it preserve or correctly reset state?
|
|
12
12
|
4. **Cross-screen consistency** — a value shown on one screen (a count, total, status, balance, name) matches everywhere else it appears.
|
|
13
|
-
5. **Deep entry** — reach a deep screen directly (NATIVE: deep link via
|
|
13
|
+
5. **Deep entry** — reach a deep screen directly (NATIVE: deep link via `qa/bin/mcli deep-link`; WEB: `open-url` a deep view) rather than navigating from home — does it load with auth/context intact, or break?
|
|
14
14
|
|
|
15
15
|
Concrete examples:
|
|
16
16
|
- **WEB:** login/SSO → landing; the core transactional journey end-to-end (e.g. search → product → basket → checkout); any onboarding wizard; editing settings and confirming persistence; exercising the mobile form elements (pickers, toggles, date/number inputs).
|
|
@@ -31,16 +31,16 @@ Log every check: `[DATA CHECK] <oracle> — expected: <X>, actual: <Y> — MATCH
|
|
|
31
31
|
## Lifecycle interleaving (mobile-only addition)
|
|
32
32
|
|
|
33
33
|
For at least ONE journey, interrupt mid-flow and verify recovery:
|
|
34
|
-
- Background the app/browser via
|
|
35
|
-
- Kill via
|
|
36
|
-
- Toggle airplane mode mid-submit (`bin/wadb shell svc wifi disable; bin/wadb shell svc data disable`) — does the app surface the failure cleanly or hang/silently drop the action? Re-enable to undo.
|
|
34
|
+
- Background the app/browser via `qa/bin/mcli press HOME` (both platforms — iOS goes through Maestro `pressKey: Home`), wait 30s, return — does state restore, or reload-and-lose-input?
|
|
35
|
+
- Kill via `qa/bin/mcli stop`, relaunch via `qa/bin/mcli launch` (WEB: + `open-url`) — does in-progress work survive? Does auth survive?
|
|
36
|
+
- Toggle airplane mode mid-submit (`qa/bin/wadb shell svc wifi disable; qa/bin/wadb shell svc data disable`) — does the app surface the failure cleanly or hang/silently drop the action? Re-enable to undo.
|
|
37
37
|
|
|
38
38
|
These lifecycle results often surface the most impactful bugs (data loss, stuck states,
|
|
39
39
|
silent failures, duplicate submissions).
|
|
40
40
|
|
|
41
41
|
## After Journeys
|
|
42
42
|
|
|
43
|
-
Write findings to `phase-
|
|
43
|
+
Write findings to `phase-4-journeys.md` (confidentiality header first) with:
|
|
44
44
|
- Journey-by-journey result (passed / failed / partial — and the WHY)
|
|
45
45
|
- Data integrity table (oracle, expected, actual, pass/fail)
|
|
46
46
|
- Lifecycle test results
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Phase 5: Deep Feature Testing (Mobile)
|
|
2
2
|
|
|
3
|
-
Follow
|
|
3
|
+
Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/05-features.md` for the SFDIPOT structure, boundary
|
|
4
4
|
values, business-logic stress, and FEW HICCUPPS oracles. The mobile changes (both modes):
|
|
5
5
|
|
|
6
6
|
## Substitutions
|
|
@@ -11,15 +11,15 @@ values, business-logic stress, and FEW HICCUPPS oracles. The mobile changes (bot
|
|
|
11
11
|
tooling (mitmproxy)"**. Do NOT invent a result. (You CAN still tamper values that travel in
|
|
12
12
|
a deep link / URL via `deep-link`/`open-url` — see Phase 6.)
|
|
13
13
|
|
|
14
|
-
- **`playwright-cli fill <ref> <long-string>`** — use the same patterns
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
- **`playwright-cli fill <ref> <long-string>`** — use the same patterns (`fill` types through a
|
|
15
|
+
Maestro `inputText` flow on both platforms), but very long strings can still be truncated by
|
|
16
|
+
the platform IME, so a "field accepted N chars" result may be a driver limit, not the app's.
|
|
17
17
|
Note the truncation when reporting, and for true very-long tests prefer pasting via the
|
|
18
18
|
clipboard:
|
|
19
19
|
```bash
|
|
20
|
-
bin/wadb shell "echo '<very long string>' | xargs -I {} am broadcast -a clipper.set -e text '{}'"
|
|
21
|
-
|
|
22
|
-
# then long-press to paste — or use
|
|
20
|
+
qa/bin/wadb shell "echo '<very long string>' | xargs -I {} am broadcast -a clipper.set -e text '{}'"
|
|
21
|
+
qa/bin/mcli click <input ref>
|
|
22
|
+
# then long-press to paste — or use qa/bin/mcli fill with the actual string and note truncation
|
|
23
23
|
```
|
|
24
24
|
|
|
25
25
|
## Generic mobile feature stress recipes
|
|
@@ -51,7 +51,7 @@ screens and web views alike — they replace the desktop-only Playwright specifi
|
|
|
51
51
|
### Navigation & routing
|
|
52
52
|
|
|
53
53
|
- Hardware/gesture back (Android `press BACK`) vs in-screen back control — same destination?
|
|
54
|
-
- Deep-link straight into a mid-flow screen (NATIVE:
|
|
54
|
+
- Deep-link straight into a mid-flow screen (NATIVE: `qa/bin/mcli deep-link <app-scheme>`; WEB: `open-url`) — does it load with context or break?
|
|
55
55
|
- After a successful action, does back land somewhere sane (not re-submitting)?
|
|
56
56
|
|
|
57
57
|
### Submit / save actions
|
|
@@ -65,7 +65,7 @@ Concrete high-value targets:
|
|
|
65
65
|
|
|
66
66
|
## After Feature Testing
|
|
67
67
|
|
|
68
|
-
Write `phase-
|
|
68
|
+
Write `phase-5-features.md` (confidentiality header first) with feature-by-feature notes. For each deferred test
|
|
69
69
|
(proxy-blocked, multi-device-blocked), record it explicitly in the coverage map so it shows
|
|
70
70
|
up as a known gap in the report.
|
|
71
71
|
|