qualiow-exploratory-testing 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -0
- package/bin/mcli +16 -0
- package/bin/mobile-cli.mjs +573 -0
- package/bin/wadb +6 -0
- package/bin/wk-ios +58 -0
- package/bin/wkeval.mjs +44 -0
- package/data/knowledge/learned-patterns.md +85 -0
- package/data/knowledge/manifest.yml +2 -1
- package/data/targets/_example-mobile-emulation.yml +52 -0
- package/data/targets/_example-native-mobile.yml +61 -0
- package/data/targets/_example-sim-android-chrome.yml +81 -0
- package/data/targets/_example-sim-ios-safari.yml +80 -0
- package/dist/cli/index.js +156 -33
- package/dist/cli/index.js.map +1 -1
- package/dist/index.d.ts +260 -527
- package/dist/index.js +142 -12
- package/dist/index.js.map +1 -1
- package/docs/MOBILE-SETUP.md +92 -0
- package/package.json +23 -11
- package/scripts/doctor-mobile.sh +143 -0
- package/scripts/setup-mobile.sh +213 -0
- package/skills/qa-explore/phases/00-setup.md +8 -1
- package/skills/qa-explore/phases/01-auth.md +4 -0
- package/skills/qa-explore-mobile/SKILL.md +146 -0
- package/skills/qa-explore-mobile/phases/00-setup.md +203 -0
- package/skills/qa-explore-mobile/phases/01-auth.md +127 -0
- package/skills/qa-explore-mobile/phases/02-charter.md +70 -0
- package/skills/qa-explore-mobile/phases/03-discovery.md +79 -0
- package/skills/qa-explore-mobile/phases/04-journeys.md +49 -0
- package/skills/qa-explore-mobile/phases/05-features.md +72 -0
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +157 -0
- package/skills/qa-explore-mobile/phases/07-reporting.md +123 -0
- package/skills/qa-explore-mobile/references/mobile-edges.md +117 -0
- package/skills/qa-explore-quick/SKILL.md +6 -0
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa-explore-mobile
|
|
3
|
+
description: >
|
|
4
|
+
Run a full AI-driven exploratory testing session on a simulator/emulator — works for
|
|
5
|
+
BOTH a NATIVE app (the installed app is the system under test) and a WEB app loaded in
|
|
6
|
+
the real device browser (iOS Simulator Safari / Android emulator Chrome). The mode is
|
|
7
|
+
selected from the target config. Same Principal-QA mindset as /qa-explore, but the driver
|
|
8
|
+
is Maestro (via the mobile-cli shim) instead of Playwright, and the phases are adapted for
|
|
9
|
+
mobile concerns (app/browser lifecycle, rotation, on-screen keyboard, deep links, offline,
|
|
10
|
+
permissions, native a11y tree). Use when user says: "test on the simulator/emulator",
|
|
11
|
+
"explore the app on iOS/Android", "find bugs on mobile", "QA the mobile build", or provides
|
|
12
|
+
a mobile-sim target (native or web).
|
|
13
|
+
allowed-tools: Bash(node:*), Bash(adb:*), Bash(xcrun:*), Bash(maestro:*), Read, Write, Glob, Grep
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Mobile Exploratory Testing Session
|
|
17
|
+
|
|
18
|
+
You are a **Principal QA Engineer** running an exploratory session on a simulator/emulator.
|
|
19
|
+
This skill works in **two modes, both co-equal, selected from the target config**:
|
|
20
|
+
|
|
21
|
+
- **NATIVE mode** — the target declares an installable app (`app.bundle_id`/`app.package` +
|
|
22
|
+
`app.app_paths`/`apk_paths`, optionally a `source_repo` with `build_commands`). The
|
|
23
|
+
installed app is the system under test; you drive it directly.
|
|
24
|
+
- **WEB mode** — the target declares a browser as the app (`app: com.apple.mobilesafari` /
|
|
25
|
+
`com.android.chrome`) plus a `web.base_url`. The browser is just the driver; the web URL
|
|
26
|
+
(the app's mobile site, e.g. `https://staging.m.example.com`) is the system under test.
|
|
27
|
+
|
|
28
|
+
The mindset and reporting standards are identical to `/qa-explore` — what changes is the
|
|
29
|
+
driver (Maestro via `mobile-cli`) and the failure modes (app/browser lifecycle, on-screen
|
|
30
|
+
keyboard, rotation, permissions, native a11y tree instead of console/network).
|
|
31
|
+
|
|
32
|
+
**Mode detection** is done in `phases/00-setup.md`: if the target's `app` id is a known
|
|
33
|
+
browser bundle AND a `web.base_url` is present → WEB mode; otherwise (non-browser app id
|
|
34
|
+
with installable artifacts) → NATIVE mode. Everything downstream (the snapshot→click→fill
|
|
35
|
+
loop, charter, discovery, journeys, features, edges, reporting) is shared and generic; each
|
|
36
|
+
phase notes the few places native vs web diverge.
|
|
37
|
+
|
|
38
|
+
Why Maestro/`mobile-cli` instead of Playwright? Playwright **cannot** drive a native app at
|
|
39
|
+
all, and it **cannot** drive iOS Simulator Safari (its "webkit" is desktop macOS WebKit, not
|
|
40
|
+
the simulator's engine). The Maestro/simctl/adb route used by `mobile-cli` is the only way to
|
|
41
|
+
exercise either a real native app or the genuine mobile-browser engine.
|
|
42
|
+
|
|
43
|
+
## The Driver — mobile-cli
|
|
44
|
+
|
|
45
|
+
A single shim wraps Maestro and `adb` / `xcrun simctl` to present a playwright-cli–shaped
|
|
46
|
+
command surface. Call it through the `bin/mcli` wrapper (it sets `JAVA_HOME` / `ANDROID_HOME`
|
|
47
|
+
/ `PATH` for you, so you don't need an export block):
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Isolate this session's device/app state so you don't clobber another target's state.
|
|
51
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
52
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
53
|
+
# Then:
|
|
54
|
+
$MCLI snapshot
|
|
55
|
+
$MCLI click e7
|
|
56
|
+
$MCLI fill e9 "user@example.com"
|
|
57
|
+
# For raw adb (logcat, am start, pm) that mcli doesn't wrap, use the bin/wadb wrapper.
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Command surface (mirrors playwright-cli where it makes sense):
|
|
61
|
+
|
|
62
|
+
| Command | Purpose |
|
|
63
|
+
|---|---|
|
|
64
|
+
| `set-device <udid>` | `emulator-5554` for Android, iOS sim UDID for iOS — sets platform |
|
|
65
|
+
| `set-app <id>` | NATIVE: the app's bundle id / package. WEB: the browser bundle `com.apple.mobilesafari` / `com.android.chrome`. |
|
|
66
|
+
| `launch` / `stop` / `relaunch-clean` | App/browser lifecycle. `relaunch-clean` is Android-only (force-stop + pm clear) — fine for a cold native start, but **avoid it in WEB mode, it wipes the logged-in browser profile.** |
|
|
67
|
+
| `open-url <url>` / `deep-link <url>` | `simctl openurl` / `am start -a VIEW -d`. WEB mode: your "address bar" to load `base_url`. NATIVE mode: fire the app's deep links. |
|
|
68
|
+
| `snapshot [--full]` | Parses Maestro's a11y hierarchy, assigns refs `e1…eN`, prints a tree. In WEB mode the browser chrome (URL bar, tabs, toolbar, keyboard) is auto-filtered. `--full` includes non-interactive nodes. KNOWN QUIRK (Android Chrome): after keyboard show/dismiss the a11y export can go stale — the WebView shows as one empty "Web View" group. Recover with `screenshot` + a coordinate tap (`bin/wadb shell input tap <x> <y>`), or `open-url` the page again to refresh the tree. |
|
|
69
|
+
| `click <ref>` | Tap by ref. Uses `adb input tap x y` on Android, `maestro tap --point` on iOS. |
|
|
70
|
+
| `fill <ref> <text>` | Tap then `adb input text` / `maestro input`. Appends to existing text — `clear` first if the field is pre-filled. Filling opens the on-screen keyboard, which SHIFTS the layout — refs cached before the keyboard are stale afterwards. Always re-`snapshot` after the last `fill` before clicking a submit control (and `press BACK` on Android first to dismiss the keyboard — with the keyboard closed BACK becomes page-back/history, so only use it while the keyboard is up). |
|
|
71
|
+
| `clear <ref>` | Tap then erase the field's existing text (Maestro `eraseText`). |
|
|
72
|
+
| `press <BACK\|ENTER\|HOME\|TAB\|ESCAPE>` | Android keyevents. iOS supports ENTER + HOME (via Maestro `pressKey`); iOS has no hardware BACK — use an in-app/on-page back control via snapshot. |
|
|
73
|
+
| `screenshot <path>` | Save PNG. |
|
|
74
|
+
| `logs [--since <sec>] [--errors]` | logcat tail (Android) / `simctl spawn log show` (iOS), filtered to the app/browser process. Replaces playwright-cli `console error`. In WEB mode it is the BROWSER's process log, not the page's JS console. |
|
|
75
|
+
| `record-start <path>` / `record-stop` | Video. Replaces playwright-cli tracing. |
|
|
76
|
+
| `running` | Foreground app — sanity check the app/browser is frontmost. |
|
|
77
|
+
|
|
78
|
+
### Safari DOM bridge — `bin/wk-ios` (iOS WEB mode only)
|
|
79
|
+
|
|
80
|
+
For iOS Safari web targets, `bin/wk-ios` restores real JS/DOM access via WebKit Remote
|
|
81
|
+
Inspector (through `ios-webkit-debug-proxy`, installed by `scripts/setup-mobile.sh`):
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
bin/wk-ios 'document.title' # eval any JS in the sim's Safari page
|
|
85
|
+
bin/wk-ios 'document.querySelectorAll(".cart_item").length' # exact-DOM assertions
|
|
86
|
+
bin/wk-ios --url # print the page's WebSocket debugger URL
|
|
87
|
+
bin/wk-ios --stop # stop the background proxy when the session ends
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Use it whenever the a11y snapshot is not enough: exact-DOM verification (counts, attribute
|
|
91
|
+
values, hidden state), reading values the a11y tree collapses, or checking a suspected
|
|
92
|
+
rendering/JS failure. It is the same power as `playwright-cli eval`, but on the REAL iOS
|
|
93
|
+
engine. Requirements: a booted simulator with Safari on a page (`open-url` first). There is
|
|
94
|
+
no Android equivalent wired up in v1 — on Android, drive by a11y refs and `logs`.
|
|
95
|
+
|
|
96
|
+
## What's Dropped vs Playwright
|
|
97
|
+
|
|
98
|
+
Both modes are driven through the device **accessibility tree, not a DOM** — native views and
|
|
99
|
+
web content alike are surfaced to a11y (headings, links, buttons, inputs and visible text
|
|
100
|
+
become tappable refs). Versus Playwright you LOSE, in BOTH modes:
|
|
101
|
+
|
|
102
|
+
- **CSS / DOM selectors** — you drive purely by visible/accessible labels and tap coordinates (native has no DOM at all; WEB mode has one but `snapshot` doesn't read it). EXCEPTION: iOS Safari web targets can READ the DOM via `bin/wk-ios` (see above) — driving/tapping still goes through refs.
|
|
103
|
+
- **eval / JS injection** — no script context on native or Android Chrome. EXCEPTION: iOS Safari web targets get full JS eval via `bin/wk-ios`.
|
|
104
|
+
- **console & network introspection** — `logs` is the app/browser process log, not a page console; there is no network panel. (iOS WEB: `bin/wk-ios` can read `window`-level state, but there is still no console/network history.)
|
|
105
|
+
- **route / network mocking** — would need mitmproxy. Out of scope for v1; flag in the report if a test required it.
|
|
106
|
+
- **state-save / state-load (storage_state)** — no transferable cookie jar. Auth happens on the device (NATIVE: in-app login/signup; WEB: a one-time interactive login the browser profile persists — see Phase 1).
|
|
107
|
+
- **resize** — sim viewport is fixed. Use rotation instead.
|
|
108
|
+
- **multi-session race conditions** — would need two emulators. Defer to a future enhancement.
|
|
109
|
+
- **XSS payload assertions (WEB)** — without DOM/console you can't reliably confirm execution; you can still observe gross misbehavior (crash, broken render). Generic input edges (long strings, emoji, paste, RTL, control chars) still apply in both modes.
|
|
110
|
+
|
|
111
|
+
## Mobile-Specific Heuristics (added on top of FEW HICCUPPS / SFDIPOT)
|
|
112
|
+
|
|
113
|
+
When charter-selecting heuristics in Phase 2, consider these mobile dimensions (most apply to
|
|
114
|
+
both modes; mode-specific ones are tagged):
|
|
115
|
+
|
|
116
|
+
- **Lifecycle:** background the app/browser → foreground after 10 min, kill → relaunch, OS-initiated suspend mid-flow. Does state survive? Is the session still valid? (WEB: does the page reload and lose input?)
|
|
117
|
+
- **Permissions (mostly NATIVE):** deny camera / notifications / contacts / location / biometrics on first prompt; revoke and re-grant. In WEB mode this surfaces as the browser's permission prompts (getUserMedia, geolocation, notifications).
|
|
118
|
+
- **Connectivity:** airplane mode mid-action, slow 3G, captive-portal Wi-Fi, IPv6-only.
|
|
119
|
+
- **Hardware / display:** rotation, dark mode, dynamic type / large font, low-battery (≤15%), on-screen keyboard show/hide jank and input occlusion.
|
|
120
|
+
- **Deep links / URLs:** legitimate inbound link/intent, malformed scheme or query, foreign-app intent injection, very long URL.
|
|
121
|
+
- **Native a11y:** VoiceOver/TalkBack focus order, contrast, dynamic type, RTL layout — driven through the device a11y tree.
|
|
122
|
+
- **Storage / profile:** NATIVE — fresh install vs upgrade, force-clear app data mid-flow; WEB — fresh profile vs returning profile, cleared cookies, first-run interstitials.
|
|
123
|
+
|
|
124
|
+
## Session Phases
|
|
125
|
+
|
|
126
|
+
Execute each phase in order. Each phase doc points at the platform-agnostic guidance in
|
|
127
|
+
`../qa-explore/phases/` and lists the mobile-specific replacements/additions.
|
|
128
|
+
|
|
129
|
+
| Phase | File | Summary |
|
|
130
|
+
|-------|------|---------|
|
|
131
|
+
| **Setup** | `phases/00-setup.md` | Resolve target + **detect mode**, boot sim; NATIVE: verify/install/launch the app; WEB: set browser, open base_url; init session dir |
|
|
132
|
+
| **Auth** | `phases/01-auth.md` | NATIVE: in-app login/signup (static OTP from config on staging). WEB: one-time interactive SSO/MFA the profile persists — never automate MFA |
|
|
133
|
+
| **Charter** | `phases/02-charter.md` | Business context + journeys + risk ranking + heuristic selection |
|
|
134
|
+
| **Discovery** | `phases/03-discovery.md` | Map screens/views (not raw DOM); app/browser logs instead of console; flag a11y aggregation (iOS) |
|
|
135
|
+
| **Journeys** | `phases/04-journeys.md` | E2E user journeys with data verification across views |
|
|
136
|
+
| **Features** | `phases/05-features.md` | Deep feature testing — same SFDIPOT, no eval-based tampering |
|
|
137
|
+
| **Edge Cases** | `phases/06-edge-cases.md` | Mobile edges: lifecycle, permissions, rotation, keyboard, deep links, offline (mode-tagged) |
|
|
138
|
+
| **Reporting** | `phases/07-reporting.md` | Same report format. Use video instead of trace. |
|
|
139
|
+
|
|
140
|
+
## References
|
|
141
|
+
|
|
142
|
+
- `references/mobile-edges.md` — checklist of mobile-only edge cases, tagged by mode (rotation, keyboard, deep links, offline, permissions, biometrics, a11y).
|
|
143
|
+
- Content is matched by **visible text / accessibility labels** from `snapshot` (and tap coordinates when a11y aggregates nodes). There is no fixed selector/testID catalog — read the live snapshot tree each time.
|
|
144
|
+
- `../qa-explore/references/security-rules.md` — applies as-is.
|
|
145
|
+
- `../qa-explore/references/severity-guide.md` — applies as-is.
|
|
146
|
+
- `../qa-explore/references/session-rules.md` — applies as-is (the 45-min cap matters even more on mobile because emulator/sim latency adds up).
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
# Phase 0: Mobile Session Setup (mode-aware)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/00-setup.md` for the general structure (parse input, load
|
|
4
|
+
context, load knowledge, create session dir, init progress.json). The mobile differences,
|
|
5
|
+
and the native-vs-web branch, are below.
|
|
6
|
+
|
|
7
|
+
## Step 0 — Preflight (fail fast on a fresh machine)
|
|
8
|
+
|
|
9
|
+
Before anything else, verify the toolchain exists — the mobile path needs Maestro plus the
|
|
10
|
+
iOS and/or Android SDKs, which are NOT installed by default and are easy to assume present:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
scripts/doctor-mobile.sh # or --ios / --android to scope to one platform
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
If it prints **NOT READY**, STOP: surface the `✗` items to the user and point them at
|
|
17
|
+
`scripts/setup-mobile.sh` (auto-installs the scriptable prerequisites; Xcode + the iOS
|
|
18
|
+
runtime are guided-manual). Do NOT start a session on a half-provisioned machine — a missing
|
|
19
|
+
`adb` / `maestro` / runtime fails cryptically deep in the run. Full guide: `docs/MOBILE-SETUP.md`.
|
|
20
|
+
|
|
21
|
+
## Step 1 — Parse target + DETECT MODE
|
|
22
|
+
|
|
23
|
+
From the user message or the target config in `data/targets/`, extract:
|
|
24
|
+
|
|
25
|
+
- **platform**: `android` | `ios` (required).
|
|
26
|
+
- **device**: emulator/sim identifier.
|
|
27
|
+
- Android: AVD name (from `device.avd`) or `emulator-5554` if already booted.
|
|
28
|
+
- iOS: simulator name (e.g. `iPhone 17`) or UDID.
|
|
29
|
+
- **app id**: `app.bundle_id` (iOS) / `app.package` (Android).
|
|
30
|
+
- **mode** — derive it:
|
|
31
|
+
- **WEB mode** if the app id is a known browser bundle (`com.apple.mobilesafari` /
|
|
32
|
+
`com.android.chrome`) AND the config has a `web.base_url`. The browser is the driver; the
|
|
33
|
+
web URL is the SUT. Templates: `data/targets/_example-sim-ios-safari.yml`,
|
|
34
|
+
`data/targets/_example-sim-android-chrome.yml` (read the matching one — their `notes:`
|
|
35
|
+
blocks document the web setup).
|
|
36
|
+
- **NATIVE mode** otherwise — i.e. a non-browser app id with installable artifacts
|
|
37
|
+
(`app.app_paths` / `app.apk_paths`), optionally a `source_repo` with `build_commands`.
|
|
38
|
+
The installed app is the SUT. Template: `data/targets/_example-native-mobile.yml`.
|
|
39
|
+
- **WEB only**: `web.base_url` / `web.start_url`.
|
|
40
|
+
- **NATIVE only**: `app.app_paths`/`apk_paths` (artifact to verify/install), and optional
|
|
41
|
+
`source_repo.path` + `source_repo.build_commands`.
|
|
42
|
+
|
|
43
|
+
Record `mode` — every later step branches on it.
|
|
44
|
+
|
|
45
|
+
## Step 2 — Boot the simulator/emulator if needed (shared)
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# Android — check first, only boot if no device attached
|
|
49
|
+
bin/wadb devices
|
|
50
|
+
# If empty:
|
|
51
|
+
# export ANDROID_HOME=$HOME/Library/Android/sdk
|
|
52
|
+
# export PATH=$ANDROID_HOME/platform-tools:$ANDROID_HOME/emulator:$PATH
|
|
53
|
+
# nohup emulator -avd <avd-name> -no-snapshot -no-boot-anim -memory 4096 -gpu auto & disown
|
|
54
|
+
# adb wait-for-device
|
|
55
|
+
|
|
56
|
+
# iOS — check first, only boot if no sim booted
|
|
57
|
+
xcrun simctl list devices booted
|
|
58
|
+
# If empty:
|
|
59
|
+
# xcrun simctl boot <UDID-or-name>
|
|
60
|
+
# open -a Simulator
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
**Do NOT reboot a sim/emulator that's already running** — preserve installed apps and (WEB
|
|
64
|
+
mode) the persisted browser profile that holds the logged-in session.
|
|
65
|
+
|
|
66
|
+
Set the driver state for the rest of the phase:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json # isolate this session's device/app state
|
|
70
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
71
|
+
$MCLI set-device <emulator-5554 OR ios-udid>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Use a dedicated `MOBILE_CLI_STATE` file per target so concurrent/sequential sessions don't
|
|
75
|
+
clobber each other's device/app state.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## Step 3 — NATIVE branch: verify / install / (optional rebuild) / launch
|
|
80
|
+
|
|
81
|
+
Skip this whole step in WEB mode.
|
|
82
|
+
|
|
83
|
+
### 3a. Verify the app is installed
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
# Android
|
|
87
|
+
bin/wadb shell pm list packages | grep <app.package>
|
|
88
|
+
# iOS
|
|
89
|
+
xcrun simctl get_app_container <udid> <app.bundle_id> 2>/dev/null
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### 3b. Install if missing
|
|
93
|
+
|
|
94
|
+
If not installed and the config provides an artifact path, install from it:
|
|
95
|
+
- Android: `bin/wadb install -r <app.apk_paths[...]>`
|
|
96
|
+
- iOS: `xcrun simctl install <udid> <app.app_paths[...]>`
|
|
97
|
+
|
|
98
|
+
If not installed and no artifact path is configured, **stop and ask the user** how to obtain
|
|
99
|
+
a build (point them at the target's `source_repo.build_commands` if defined). Do NOT invent a
|
|
100
|
+
build command.
|
|
101
|
+
|
|
102
|
+
### 3c. OPTIONAL refresh against latest source
|
|
103
|
+
|
|
104
|
+
**Skip unless the user passed `--rebuild` OR the session goal explicitly mentions "latest" /
|
|
105
|
+
"fresh" / "newest" build.** Otherwise run against whatever is installed (faster, preserves
|
|
106
|
+
state). This skill does **not** assume any build framework — it runs the EXACT commands the
|
|
107
|
+
target config provides:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
# 1. Drift check (using the target's source_repo.build_commands.refresh, if defined)
|
|
111
|
+
cd <source_repo.path>
|
|
112
|
+
git status -sb && git fetch && git log HEAD..@{u} --oneline
|
|
113
|
+
|
|
114
|
+
# 2. Decide WITH the user before pulling:
|
|
115
|
+
# clean + behind → safe to pull then rebuild; dirty → ask (stash/commit/build-as-is);
|
|
116
|
+
# intentionally pinned branch → do NOT switch branches.
|
|
117
|
+
|
|
118
|
+
# 3. Run the platform build command verbatim from source_repo.build_commands
|
|
119
|
+
# (e.g. build_commands.android_release / build_commands.ios_debug). These are
|
|
120
|
+
# project-defined; the skill does not hardcode them. Long builds: run in the
|
|
121
|
+
# background and PID-wait so you get notified on completion:
|
|
122
|
+
# nohup <the configured build command> > /tmp/mobile-build.log 2>&1 &
|
|
123
|
+
# BUILD_PID=$!; until ! kill -0 $BUILD_PID 2>/dev/null; do sleep 15; done
|
|
124
|
+
|
|
125
|
+
# 4. After install, cold-start fresh: $MCLI relaunch-clean (Android) or stop+launch (iOS)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
**Always log the source state in the report** (branch + sha + clean/dirty) so readers know
|
|
129
|
+
what code the session ran against.
|
|
130
|
+
|
|
131
|
+
### 3d. Launch the app
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
$MCLI set-app <app.bundle_id OR app.package>
|
|
135
|
+
$MCLI launch
|
|
136
|
+
$MCLI snapshot # sanity check — first screen renders
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Step 4 — WEB branch: set browser, open URL, snapshot
|
|
142
|
+
|
|
143
|
+
Skip this step in NATIVE mode.
|
|
144
|
+
|
|
145
|
+
On Android, confirm the browser exists first (Safari ships with every iOS runtime; Chrome is
|
|
146
|
+
only guaranteed on **Google-APIs/Play** images — stock AOSP images may lack it):
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
bin/wadb shell pm list packages | grep chrome # expect: package:com.android.chrome
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
If `com.android.chrome` is absent, use a Play-image AVD (or fall back to the iOS Safari
|
|
153
|
+
target) rather than assuming Chrome is installed. Do not attempt to install a browser.
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
$MCLI set-app <com.apple.mobilesafari OR com.android.chrome>
|
|
157
|
+
$MCLI launch
|
|
158
|
+
$MCLI open-url <web.base_url or web.start_url>
|
|
159
|
+
$MCLI snapshot # sanity check — the page renders
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
On a fresh Chrome profile you may see a one-time first-run / "Open tabs" promo bubble —
|
|
163
|
+
`mobile-cli` filters the routine promo out of snapshots, but if a full-screen
|
|
164
|
+
first-run/sign-in-to-Chrome interstitial appears, it must be dismissed once by hand.
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## Step 5 — Session directory (shared)
|
|
169
|
+
|
|
170
|
+
Same as qa-explore but include `platform` in the dir name:
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
output/sessions/<YYYY-MM-DD-HHmm>-<target>-<platform>/
|
|
174
|
+
charter.md
|
|
175
|
+
session-log.md
|
|
176
|
+
screenshots/
|
|
177
|
+
bugs/
|
|
178
|
+
videos/ # mobile-only — for record-start output
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Step 6 — progress.json additions (shared)
|
|
182
|
+
|
|
183
|
+
Same base schema as qa-explore. Add a `platform` key including the resolved `mode`:
|
|
184
|
+
|
|
185
|
+
```json
|
|
186
|
+
"platform": {
|
|
187
|
+
"mode": "native",
|
|
188
|
+
"os": "android",
|
|
189
|
+
"device": "emulator-5554",
|
|
190
|
+
"app_id": "com.example.myapp",
|
|
191
|
+
"app_version": "<from `adb shell dumpsys package <pkg> | grep versionName`>",
|
|
192
|
+
"target_url": null
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
For WEB mode, set `"mode": "web"`, `app_id` to the browser bundle, and `target_url` to the
|
|
197
|
+
`base_url` (leave `app_version` null — it's the browser's, not the SUT's).
|
|
198
|
+
|
|
199
|
+
The `console_errors` counter from qa-explore becomes `app_log_errors` (app/browser-process
|
|
200
|
+
ERROR-level log matches — coarser than a real JS console, see Phase 3).
|
|
201
|
+
|
|
202
|
+
**Skip qa-explore's Step 6 (Playwright Test Agents bootstrap) entirely — not applicable when
|
|
203
|
+
the driver is mobile-cli/Maestro.**
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# Phase 1: Authentication (mode-aware)
|
|
2
|
+
|
|
3
|
+
In both modes there is **NO transferable `storage_state` you can inject** — auth happens on
|
|
4
|
+
the device. Pick the branch by the target's `auth.strategy`:
|
|
5
|
+
|
|
6
|
+
- `auth.strategy: in-app` → **NATIVE branch** (account creation / login inside the app).
|
|
7
|
+
- `auth.strategy: interactive-sso` (or any interactive web SSO) → **WEB branch** (one-time
|
|
8
|
+
human login the browser profile persists).
|
|
9
|
+
- `auth.strategy: none` → skip this phase.
|
|
10
|
+
|
|
11
|
+
Set up once per Bash block:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
15
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## NATIVE branch — in-app login / signup
|
|
21
|
+
|
|
22
|
+
Drive the app's own auth screens via the snapshot→click→fill loop, following the steps the
|
|
23
|
+
target's `auth.strategy` describes. There is no shared session, so you authenticate inside
|
|
24
|
+
the app each run (or reuse an existing account already on the device).
|
|
25
|
+
|
|
26
|
+
### Account creation (when the charter targets signup, or as one-time setup)
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
# Optional cold start (Android): relaunch-clean wipes app data, then cold-starts.
|
|
30
|
+
$MCLI relaunch-clean
|
|
31
|
+
|
|
32
|
+
$MCLI snapshot # welcome / landing
|
|
33
|
+
$MCLI click <create-account / sign-up ref>
|
|
34
|
+
$MCLI snapshot # email / details screen
|
|
35
|
+
$MCLI fill <email input ref> "<config test_email_pattern, e.g. qa+$(date +%s)@example.com>"
|
|
36
|
+
$MCLI click <continue ref>
|
|
37
|
+
$MCLI snapshot # OTP / verification screen
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
**OTP on staging:** if the target config records a fixed code (`auth.static_otp`), use it
|
|
41
|
+
instead of waiting for a real OTP. Multi-cell OTP inputs often **do not accept a single
|
|
42
|
+
`fill`** — tap each cell and send one digit at a time:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
$MCLI snapshot # find the per-cell OTP refs
|
|
46
|
+
for d in <digits of static_otp>; do
|
|
47
|
+
$MCLI click <next-cell-ref>
|
|
48
|
+
$MCLI fill <that-ref> "$d"
|
|
49
|
+
done
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
### Existing-user login (faster for post-auth flows)
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
$MCLI launch # do NOT relaunch-clean — preserves the existing account
|
|
56
|
+
$MCLI snapshot # if on landing, tap Log In
|
|
57
|
+
$MCLI click <login ref>
|
|
58
|
+
$MCLI fill <email/username input ref> "<existing test account>"
|
|
59
|
+
$MCLI click <continue ref>
|
|
60
|
+
# OTP same as above (static_otp on staging)
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Generalize email/validation rules from the target config or prior findings (e.g. some backends
|
|
64
|
+
reject disposable-email domains, or client vs server validation diverges) — do not assume any
|
|
65
|
+
specific provider's quirks.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## WEB branch — one-time interactive SSO/MFA login
|
|
70
|
+
|
|
71
|
+
The browser keeps its own profile (cookies/session) on the device, persisted across `launch`
|
|
72
|
+
/ `open-url`. Do the login **once per device per session-lifetime; a HUMAN performs the MFA
|
|
73
|
+
step — never script it.** This applies to any interactive IdP (Microsoft Entra, Okta,
|
|
74
|
+
Google Workspace, Auth0, …) — see the web target templates (`_example-sim-ios-safari.yml`,
|
|
75
|
+
`_example-sim-android-chrome.yml`).
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
$MCLI launch # do NOT relaunch-clean — that wipes the profile
|
|
79
|
+
$MCLI open-url <web.start_url e.g. .../auth/login>
|
|
80
|
+
$MCLI snapshot # login surface / SSO button
|
|
81
|
+
$MCLI click <sign-in / SSO button ref>
|
|
82
|
+
# → hands off to the identity provider (e.g. login.microsoftonline.com, your Okta org, …).
|
|
83
|
+
# A HUMAN completes username + password + MFA by hand in the sim/emulator UI.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Once login completes, subsequent `launch` / `open-url` / exploration reuse the session until
|
|
87
|
+
it expires or the profile is wiped. **Never automate the MFA challenge.** If the IdP domain
|
|
88
|
+
reappears mid-session, the session lapsed — hand back to a human; do not fill credentials.
|
|
89
|
+
|
|
90
|
+
### Returning session (already logged in)
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
$MCLI launch
|
|
94
|
+
$MCLI open-url <web.base_url>
|
|
95
|
+
$MCLI snapshot # expect an authenticated landing view, not login
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
If you see the login screen or IdP domain, re-do the interactive login above.
|
|
99
|
+
|
|
100
|
+
### Profile hygiene (WEB) — what NOT to do
|
|
101
|
+
|
|
102
|
+
- **Do not run `relaunch-clean`** on a browser target — on Android it `pm clear`s the browser and forces a fresh SSO+MFA login.
|
|
103
|
+
- **Do not** inject cookies / storage_state — no supported path on a real sim browser.
|
|
104
|
+
- Boot, don't erase, the device — erasing the sim wipes the profile.
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## Verify Auth (shared)
|
|
109
|
+
|
|
110
|
+
After auth, `$MCLI snapshot` and confirm authenticated indicators are present:
|
|
111
|
+
- NATIVE: the app's post-login home/landing controls.
|
|
112
|
+
- WEB: a known post-login heading, nav, or the user's name.
|
|
113
|
+
|
|
114
|
+
If stuck on a login/IdP screen, capture the log:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
$MCLI logs --errors --since 60
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
**Auth issues to flag as bugs (not session blockers):**
|
|
121
|
+
- OTP rejected even with the documented static/staging code (NATIVE).
|
|
122
|
+
- A valid session dropped far sooner than expected (forced re-login mid-flow).
|
|
123
|
+
- A broken/blank state instead of redirecting to login when the session expires.
|
|
124
|
+
- Login surface renders but inputs / SSO button are unreachable in the a11y tree (blocks automation AND assistive tech).
|
|
125
|
+
- Client-side input validation (email format, required fields) diverges from server-side behavior.
|
|
126
|
+
|
|
127
|
+
**Update progress:** mark auth phase complete with `branch: native|web`, `user: <redacted>`.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Phase 2: Business Context & Charter (Mobile)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/02-charter.md` for the structure (use the app as a user,
|
|
4
|
+
identify journeys, risk-rank, select heuristics, write charter). The mobile additions below
|
|
5
|
+
apply to both modes.
|
|
6
|
+
|
|
7
|
+
## Use the app as a real user — for ~3 min
|
|
8
|
+
|
|
9
|
+
Drive the system under test via `$MCLI snapshot` → `click` → `snapshot` loops to reach entry
|
|
10
|
+
points (NATIVE: navigate the app's screens; WEB: `open-url` plus in-page links). Don't test
|
|
11
|
+
yet — just observe what it exists to do, on a phone-sized screen, the way a real mobile user
|
|
12
|
+
would. Note the primary task, where money/data/trust is at stake, and what a frustrated user
|
|
13
|
+
on a flaky connection would feel.
|
|
14
|
+
|
|
15
|
+
Concrete examples:
|
|
16
|
+
- **WEB mode:** the SUT is the target's mobile web app (an SPA or responsive site used on
|
|
17
|
+
phones in the field) — high-value areas are typically the login/SSO flow, checkout or the
|
|
18
|
+
core transactional flow, settings, any onboarding wizard, and the mobile form elements.
|
|
19
|
+
- **NATIVE mode:** treat the installed app under test the same way — open it, walk its primary
|
|
20
|
+
task (onboarding, the core create/transact/save flow, settings) and note the money/data/
|
|
21
|
+
trust-bearing screens before risk-ranking.
|
|
22
|
+
|
|
23
|
+
## Optional — load gathered requirements as charter input
|
|
24
|
+
|
|
25
|
+
If requirements/context were prepared (e.g. via `/qa-gather`, or the user pasted tickets /
|
|
26
|
+
acceptance criteria), use them as **charter inputs, not a checklist** — exploratory testing
|
|
27
|
+
is not test execution. Requirements tell you what was *deliberately designed*; your job is to
|
|
28
|
+
also find what *wasn't* (missing states, unhandled errors, mobile-only gaps). Risk-rank the
|
|
29
|
+
feature areas yourself based on blast radius (data loss, money, security, trust) and adjust
|
|
30
|
+
from any provided priority order if your observation says otherwise.
|
|
31
|
+
|
|
32
|
+
## Heuristics — mobile-flavored picks
|
|
33
|
+
|
|
34
|
+
In addition to qa-explore's SFDIPOT picks, **always select at least 1 mobile-specific
|
|
35
|
+
dimension** appropriate to the feature under charter, e.g.:
|
|
36
|
+
|
|
37
|
+
- Submitting / paying / saving → **Lifecycle** (background mid-submit; airplane mode at submit — does it surface failure or hang?)
|
|
38
|
+
- Long forms / multi-step wizards → **Keyboard & rotation** (does the on-screen keyboard occlude the active input? does rotation reflow or break layout?)
|
|
39
|
+
- Auth / session-sensitive areas → **Lifecycle / storage** (background 10 min → return: still authenticated? page reloaded or restored?)
|
|
40
|
+
- Settings / content-heavy views → **Native a11y** (VoiceOver/TalkBack focus order, dynamic type, contrast)
|
|
41
|
+
|
|
42
|
+
## Start recording
|
|
43
|
+
|
|
44
|
+
Replace qa-explore's `tracing-start` with a video:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
$MCLI record-start output/sessions/<session-dir>/videos/session.mp4
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
If recording fails or is too heavy, take frequent screenshots instead. Recording is
|
|
51
|
+
nice-to-have, not required.
|
|
52
|
+
|
|
53
|
+
## Charter additions (append to the qa-explore charter template)
|
|
54
|
+
|
|
55
|
+
```markdown
|
|
56
|
+
## Mobile Context
|
|
57
|
+
**Mode:** [native | web]
|
|
58
|
+
**Platform:** [android | ios]
|
|
59
|
+
**Device:** [device label]
|
|
60
|
+
**App under test:** [native: app id + version | web: browser bundle]
|
|
61
|
+
**Target URL:** [web mode only: base_url]
|
|
62
|
+
**Environment:** [preprod | staging | production]
|
|
63
|
+
|
|
64
|
+
## Requirements Input (if provided)
|
|
65
|
+
**Source:** [gathered context / ticket / acceptance criteria / none — blind]
|
|
66
|
+
**Areas referenced:** [list]
|
|
67
|
+
**Areas excluded from this session (and why):** [e.g. requires hardware not on sim, biometrics, etc.]
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
**Update progress:** charter complete + platform info recorded.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Phase 3: Discovery & Mapping (Mobile)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/03-discovery.md` for goal and method (map structure,
|
|
4
|
+
refine risk). The substitutions below apply to both modes.
|
|
5
|
+
|
|
6
|
+
## Map screens/views by name + nav path (not URLs)
|
|
7
|
+
|
|
8
|
+
There is no address bar to map by. Map by **screen/view name + how you reached it** (NATIVE:
|
|
9
|
+
the nav path through the app; WEB: the URL that `open-url`/a link loaded, recorded as a hint).
|
|
10
|
+
For each reachable screen/view:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
$MCLI snapshot
|
|
14
|
+
$MCLI screenshot output/sessions/<session-dir>/screenshots/discovery-<view>.png
|
|
15
|
+
$MCLI logs --errors --since 30
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Log each: `[DISCOVERED] <view-name> — <one-line description> — Risk: <P0/P1/P2/P3> — entry: <how-to-reach>`
|
|
19
|
+
|
|
20
|
+
## Match content by visible text / a11y label — there is no testID catalog
|
|
21
|
+
|
|
22
|
+
`snapshot` reads the device **accessibility tree, not the DOM**, so you identify elements by
|
|
23
|
+
their **visible text and accessibility labels** (headings, links, buttons, inputs), not by
|
|
24
|
+
CSS selectors or a fixed testID map. Read the live snapshot tree on each view. When a control
|
|
25
|
+
has no usable accessible label, fall back to tapping its coordinates from `snapshot --full`.
|
|
26
|
+
|
|
27
|
+
## App/browser log instead of console
|
|
28
|
+
|
|
29
|
+
Replace `playwright-cli console error` with:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
$MCLI logs --errors --since 60
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Caveat: this is the **process log** (logcat / `simctl spawn log show` scoped to the
|
|
36
|
+
app/browser process). In WEB mode it is the BROWSER's log, NOT the page's JS console, so it
|
|
37
|
+
catches gross failures (renderer crashes) but misses most page-level `console.error`. In
|
|
38
|
+
NATIVE mode it is the app's own log (crashes, native exceptions, bridge errors). Treat a clean
|
|
39
|
+
log as "no gross crash", not "no errors". Still watch for:
|
|
40
|
+
|
|
41
|
+
- Process crashes, native exceptions, or repeated reloads/restarts
|
|
42
|
+
- Any log line leaking PII or credentials (tokens, emails, secrets) — always file as a high-severity finding
|
|
43
|
+
- Mixed-content / TLS warnings (WEB)
|
|
44
|
+
|
|
45
|
+
## Network — limited
|
|
46
|
+
|
|
47
|
+
`mobile-cli` does not proxy traffic in v1, and there is no network panel. If a view seems to
|
|
48
|
+
make a request and the result is wrong (stale data, missing item), note "needs proxy capture"
|
|
49
|
+
in the bug report rather than guessing. Don't invent network details.
|
|
50
|
+
|
|
51
|
+
## What's MISSING — mobile-specific
|
|
52
|
+
|
|
53
|
+
Beyond the qa-explore "what's not here" check, ask:
|
|
54
|
+
- **Permission prompts (mostly NATIVE)** — does the app explain WHY before requesting camera/notifications/location/contacts? (WEB: the same for browser permission prompts.)
|
|
55
|
+
- **Offline state** — what happens with no network? Is there a UI for it or just a hang/blank screen?
|
|
56
|
+
- **Empty states** — first launch/visit, no data: are zero-item lists helpful or just blank?
|
|
57
|
+
- **Pull-to-refresh / reload** — present where a user would expect it?
|
|
58
|
+
- **Keyboard handling** — does the on-screen keyboard cover the active input? Does "Done"/return dismiss it?
|
|
59
|
+
- **Rotation** — does the layout reflow, or break/clip in landscape?
|
|
60
|
+
- **Background→foreground** — return after 10 min: still authenticated? Does state restore (WEB: page reload or restore)?
|
|
61
|
+
- **Touch targets** — finger-sized, or too small/desktop-sized and hard to hit?
|
|
62
|
+
|
|
63
|
+
## iOS-specific note — a11y aggregation
|
|
64
|
+
|
|
65
|
+
On iOS, some views **aggregate child elements into one parent accessibility label** (e.g. a
|
|
66
|
+
form returns one big comma-joined label of several fields). When you see a single ref with a
|
|
67
|
+
comma-joined label of many controls, that's an iOS a11y aggregation issue — file as a Medium
|
|
68
|
+
bug (it blocks automation and hurts VoiceOver users) and tap by approximate coordinates from
|
|
69
|
+
the snapshot bounds for the rest of the session on that view.
|
|
70
|
+
|
|
71
|
+
## After Discovery
|
|
72
|
+
|
|
73
|
+
Write findings to `phase-1-discovery.md` including:
|
|
74
|
+
- View map (table: view → key visible labels/controls → entry path → URL if known)
|
|
75
|
+
- Log error summary (count by level) with the browser-log caveat noted
|
|
76
|
+
- iOS a11y aggregation hits (if any)
|
|
77
|
+
- Updated risk ranking
|
|
78
|
+
|
|
79
|
+
**Update progress:** discovery complete with `pages_explored` (views), `app_log_errors`, `missing_checklist_items`, `bugs_so_far`.
|