qualiow-exploratory-testing 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -0
- package/bin/mcli +16 -0
- package/bin/mobile-cli.mjs +573 -0
- package/bin/wadb +6 -0
- package/bin/wk-ios +58 -0
- package/bin/wkeval.mjs +44 -0
- package/data/knowledge/learned-patterns.md +85 -0
- package/data/knowledge/manifest.yml +2 -1
- package/data/targets/_example-mobile-emulation.yml +52 -0
- package/data/targets/_example-native-mobile.yml +61 -0
- package/data/targets/_example-sim-android-chrome.yml +81 -0
- package/data/targets/_example-sim-ios-safari.yml +80 -0
- package/dist/cli/index.js +156 -33
- package/dist/cli/index.js.map +1 -1
- package/dist/index.d.ts +260 -527
- package/dist/index.js +142 -12
- package/dist/index.js.map +1 -1
- package/docs/MOBILE-SETUP.md +92 -0
- package/package.json +23 -11
- package/scripts/doctor-mobile.sh +143 -0
- package/scripts/setup-mobile.sh +213 -0
- package/skills/qa-explore/phases/00-setup.md +8 -1
- package/skills/qa-explore/phases/01-auth.md +4 -0
- package/skills/qa-explore-mobile/SKILL.md +146 -0
- package/skills/qa-explore-mobile/phases/00-setup.md +203 -0
- package/skills/qa-explore-mobile/phases/01-auth.md +127 -0
- package/skills/qa-explore-mobile/phases/02-charter.md +70 -0
- package/skills/qa-explore-mobile/phases/03-discovery.md +79 -0
- package/skills/qa-explore-mobile/phases/04-journeys.md +49 -0
- package/skills/qa-explore-mobile/phases/05-features.md +72 -0
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +157 -0
- package/skills/qa-explore-mobile/phases/07-reporting.md +123 -0
- package/skills/qa-explore-mobile/references/mobile-edges.md +117 -0
- package/skills/qa-explore-quick/SKILL.md +6 -0
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Phase 4: End-to-End User Journeys (Mobile)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/04-journeys.md` for goal and the data-integrity
|
|
4
|
+
discipline. Build the journey templates from the charter's top user goals. Generic
|
|
5
|
+
mobile journey shapes to instantiate (apply to both modes):
|
|
6
|
+
|
|
7
|
+
## Standard mobile journeys
|
|
8
|
+
|
|
9
|
+
1. **First-run / entry → primary task** — launch the app (NATIVE) or open the start URL (WEB), complete the main thing it exists for (submit a form, complete a wizard, make a transaction, change a setting), confirm the success state actually persists.
|
|
10
|
+
2. **Create / edit / save** — create or edit a record, save it, navigate away and back, confirm the change is reflected (not just an optimistic UI that drops on reload/relaunch).
|
|
11
|
+
3. **Multi-step flow / wizard** — go through a multi-step flow start to finish; then go partway, back out, and resume — does it preserve or correctly reset state?
|
|
12
|
+
4. **Cross-screen consistency** — a value shown on one screen (a count, total, status, balance, name) matches everywhere else it appears.
|
|
13
|
+
5. **Deep entry** — reach a deep screen directly (NATIVE: deep link via `$MCLI deep-link`; WEB: `open-url` a deep view) rather than navigating from home — does it load with auth/context intact, or break?
|
|
14
|
+
|
|
15
|
+
Concrete examples:
|
|
16
|
+
- **WEB:** login/SSO → landing; the core transactional journey end-to-end (e.g. search → product → basket → checkout); any onboarding wizard; editing settings and confirming persistence; exercising the mobile form elements (pickers, toggles, date/number inputs).
|
|
17
|
+
- **NATIVE:** onboarding/account-creation → home; the app's core create/transact/save flow with verification on a related screen; a settings change that must persist across relaunch.
|
|
18
|
+
|
|
19
|
+
## Data integrity (MANDATORY)
|
|
20
|
+
|
|
21
|
+
After every state-changing action, verify on a related view. Generic correctness oracles:
|
|
22
|
+
|
|
23
|
+
- **After a save/edit:** the new value is shown back correctly on the detail screen AND survives a reload (NATIVE: navigate away and back, or relaunch; WEB: `open-url` the same view again).
|
|
24
|
+
- **After a create:** the new item appears in its list/index with the right attributes.
|
|
25
|
+
- **After a delete:** the item is gone from every view that listed it.
|
|
26
|
+
- **Cross-view:** a total/count/status equals the sum or state of its parts across views.
|
|
27
|
+
- **Formatting:** numbers, dates, currencies, and localized text render correctly at mobile width (no truncation, no `NaN`, no `undefined`, correct locale).
|
|
28
|
+
|
|
29
|
+
Log every check: `[DATA CHECK] <oracle> — expected: <X>, actual: <Y> — MATCH/MISMATCH`
|
|
30
|
+
|
|
31
|
+
## Lifecycle interleaving (mobile-only addition)
|
|
32
|
+
|
|
33
|
+
For at least ONE journey, interrupt mid-flow and verify recovery:
|
|
34
|
+
- Background the app/browser via `$MCLI press HOME` (Android) / home gesture (iOS), wait 30s, return — does state restore, or reload-and-lose-input?
|
|
35
|
+
- Kill via `$MCLI stop`, relaunch via `$MCLI launch` (WEB: + `open-url`) — does in-progress work survive? Does auth survive?
|
|
36
|
+
- Toggle airplane mode mid-submit (`bin/wadb shell svc wifi disable; bin/wadb shell svc data disable`) — does the app surface the failure cleanly or hang/silently drop the action? Re-enable to undo.
|
|
37
|
+
|
|
38
|
+
These lifecycle results often surface the most impactful bugs (data loss, stuck states,
|
|
39
|
+
silent failures, duplicate submissions).
|
|
40
|
+
|
|
41
|
+
## After Journeys
|
|
42
|
+
|
|
43
|
+
Write findings to `phase-2-user-journeys.md` with:
|
|
44
|
+
- Journey-by-journey result (passed / failed / partial — and the WHY)
|
|
45
|
+
- Data integrity table (oracle, expected, actual, pass/fail)
|
|
46
|
+
- Lifecycle test results
|
|
47
|
+
- Cross-view inconsistencies (e.g., "Step 3 shows value X, the summary shows Y")
|
|
48
|
+
|
|
49
|
+
**Update progress:** journeys complete, data integrity counts.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Phase 5: Deep Feature Testing (Mobile)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/05-features.md` for the SFDIPOT structure, boundary
|
|
4
|
+
values, business-logic stress, and FEW HICCUPPS oracles. The mobile changes (both modes):
|
|
5
|
+
|
|
6
|
+
## Substitutions
|
|
7
|
+
|
|
8
|
+
- **`playwright-cli eval` (parameter tampering)** — **DROPPED**. No script context in either
|
|
9
|
+
mode. If a test *would* benefit from tampering server-controlled values (hidden fields,
|
|
10
|
+
computed totals, query params), document the scenario and mark **"deferred — needs proxy
|
|
11
|
+
tooling (mitmproxy)"**. Do NOT invent a result. (You CAN still tamper values that travel in
|
|
12
|
+
a deep link / URL via `deep-link`/`open-url` — see Phase 6.)
|
|
13
|
+
|
|
14
|
+
- **`playwright-cli fill <ref> <long-string>`** — use the same patterns, but be aware the
|
|
15
|
+
Android `input text` command silently truncates very long strings (~5000 chars max on most
|
|
16
|
+
emulators), so a "field accepted N chars" result may be a driver limit, not the app's.
|
|
17
|
+
Note the truncation when reporting, and for true very-long tests prefer pasting via the
|
|
18
|
+
clipboard:
|
|
19
|
+
```bash
|
|
20
|
+
bin/wadb shell "echo '<very long string>' | xargs -I {} am broadcast -a clipper.set -e text '{}'"
|
|
21
|
+
$MCLI click <input ref>
|
|
22
|
+
# then long-press to paste — or use $MCLI fill with the actual string and note truncation
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Generic mobile feature stress recipes
|
|
26
|
+
|
|
27
|
+
Instantiate these for the feature under test (pick the ones that fit; they apply to native
|
|
28
|
+
screens and web views alike — they replace the desktop-only Playwright specifics).
|
|
29
|
+
|
|
30
|
+
### Form fields & inputs
|
|
31
|
+
|
|
32
|
+
- **Boundary values:** empty, single char, max length, max+1, leading/trailing whitespace, whitespace-only.
|
|
33
|
+
- **Character classes:** unicode/emoji (`naïve café 🤑 العربية`), RTL text, control chars, paste with formatting.
|
|
34
|
+
- **Client vs server validation:** does the field reject client-side what the server would also reject — and accept what the server actually accepts? Try values just outside the visible UI constraint.
|
|
35
|
+
- **Keyboard occlusion:** when the input is near the bottom, does the on-screen keyboard cover it or the submit button? Does the view scroll the input into view?
|
|
36
|
+
- **Keyboard type:** does a numeric/email field bring up the right mobile keyboard?
|
|
37
|
+
|
|
38
|
+
### Pickers, toggles, steppers, date/number controls
|
|
39
|
+
|
|
40
|
+
- Min/max/step boundaries; rapid toggling; selecting then dismissing without choosing.
|
|
41
|
+
- Native vs custom picker behavior on a touch device — is the custom control reachable and operable by tap (and by a11y)?
|
|
42
|
+
- Does the chosen value persist on reload and reflect everywhere it's shown?
|
|
43
|
+
|
|
44
|
+
### Multi-step flows / wizards
|
|
45
|
+
|
|
46
|
+
- Forward-only vs back-navigable: go back mid-wizard — is prior input preserved?
|
|
47
|
+
- Skip an optional step then return — consistent state?
|
|
48
|
+
- Submit twice / double-tap the final action — duplicate submission? Idempotency?
|
|
49
|
+
- Reload mid-wizard (NATIVE: background/relaunch; WEB: `open-url` the current step) — resume or lose everything?
|
|
50
|
+
|
|
51
|
+
### Navigation & routing
|
|
52
|
+
|
|
53
|
+
- Hardware/gesture back (Android `press BACK`) vs in-screen back control — same destination?
|
|
54
|
+
- Deep-link straight into a mid-flow screen (NATIVE: `$MCLI deep-link <app-scheme>`; WEB: `open-url`) — does it load with context or break?
|
|
55
|
+
- After a successful action, does back land somewhere sane (not re-submitting)?
|
|
56
|
+
|
|
57
|
+
### Submit / save actions
|
|
58
|
+
|
|
59
|
+
- Slow network at submit: does it show progress, prevent double-submit, and confirm success?
|
|
60
|
+
- Background the app/browser on a confirm/submit screen → return → submit: stale data? re-validated?
|
|
61
|
+
|
|
62
|
+
Concrete high-value targets:
|
|
63
|
+
- **WEB:** the **mobile form elements** (pickers, toggles, date/number inputs — do they work by touch and persist?), any **multi-step wizard/checkout** (back/resume/skip), and **settings** (do changes save and reflect app-wide?).
|
|
64
|
+
- **NATIVE:** the app's primary transactional/create flow under boundary + replay stress, its custom input controls, and any setting that must persist across relaunch.
|
|
65
|
+
|
|
66
|
+
## After Feature Testing
|
|
67
|
+
|
|
68
|
+
Write `phase-3-feature-testing.md` with feature-by-feature notes. For each deferred test
|
|
69
|
+
(proxy-blocked, multi-device-blocked), record it explicitly in the coverage map so it shows
|
|
70
|
+
up as a known gap in the report.
|
|
71
|
+
|
|
72
|
+
**Update progress.**
|
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
# Phase 6: Edge Cases & Mobile-Specific (mode-aware)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/06-edge-cases.md` for the saboteur mindset. Edges are tagged
|
|
4
|
+
**[both]**, **[native]**, or **[web]** — run the ones that fit the resolved mode.
|
|
5
|
+
|
|
6
|
+
Set up once per Bash block:
|
|
7
|
+
```bash
|
|
8
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
9
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Dropped / limited (no DOM, no JS console)
|
|
13
|
+
|
|
14
|
+
- **XSS execution assertions via `fill` [web]** — without DOM/console you can't confirm execution. You can still submit payloads and watch for gross breakage (crash, broken render, reflected unescaped text in the a11y tree).
|
|
15
|
+
- **`route ... --status=500` network simulation [both]** — needs mitmproxy. **Deferred** — log scenarios that would benefit, run them once a proxy exists.
|
|
16
|
+
- **Multi-session race [both]** — needs two emulators. **Deferred for v1.**
|
|
17
|
+
- **`resize` viewport tests** — sim viewport is fixed. Use **rotation** instead (below).
|
|
18
|
+
|
|
19
|
+
## Kept (still useful on mobile)
|
|
20
|
+
|
|
21
|
+
- **Unicode / emoji [both]** in inputs: `$MCLI fill <ref> "naïve café 🤑 العربية"`
|
|
22
|
+
- **Whitespace-only [both]:** `$MCLI fill <ref> " "`
|
|
23
|
+
- **Extremely long [both]:** see the truncation caveat in `05-features.md`
|
|
24
|
+
- **State manipulation via deep link / URL [both]:** NATIVE `$MCLI deep-link "<app-scheme>://<path>?<tampered>"`; WEB `$MCLI open-url "<base_url>/<deep view>?<tampered params>"`
|
|
25
|
+
- **Replay [both]:** double-tap submit; submit then back then re-submit
|
|
26
|
+
- **Brute force / rate limits [both]:** repeated failed submissions/logins — lockout / cooldown?
|
|
27
|
+
|
|
28
|
+
## Mobile-device edges (the high-value additions)
|
|
29
|
+
|
|
30
|
+
### Lifecycle [both]
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
# Background mid-flow
|
|
34
|
+
$MCLI press HOME # Android
|
|
35
|
+
# (iOS: use the home gesture/button via the Simulator; press HOME is Android-only)
|
|
36
|
+
sleep 30
|
|
37
|
+
$MCLI launch # foreground — did state restore or reload-and-lose-state?
|
|
38
|
+
|
|
39
|
+
# Kill mid-flow
|
|
40
|
+
$MCLI stop
|
|
41
|
+
$MCLI launch
|
|
42
|
+
# WEB only: re-open the view you were on
|
|
43
|
+
# $MCLI open-url <current view>
|
|
44
|
+
|
|
45
|
+
# OS-initiated suspend (Android)
|
|
46
|
+
bin/wadb shell am broadcast -a android.intent.action.SCREEN_OFF
|
|
47
|
+
sleep 5
|
|
48
|
+
bin/wadb shell am broadcast -a android.intent.action.SCREEN_ON
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
For each: did state survive? Was auth still valid? Did an in-progress action complete or
|
|
52
|
+
duplicate?
|
|
53
|
+
|
|
54
|
+
### Permissions [native] (browser prompts in [web])
|
|
55
|
+
|
|
56
|
+
NATIVE — deny / revoke / re-grant runtime permissions and check graceful handling:
|
|
57
|
+
```bash
|
|
58
|
+
bin/wadb shell pm revoke <app.package> android.permission.CAMERA # then try the QR/camera feature
|
|
59
|
+
bin/wadb shell pm grant <app.package> android.permission.CAMERA
|
|
60
|
+
# iOS sim: xcrun simctl privacy <udid> reset|grant camera <app.bundle_id>
|
|
61
|
+
```
|
|
62
|
+
Test camera / notifications / contacts / location. In WEB mode the same applies to the
|
|
63
|
+
**browser** package (controls page getUserMedia/geolocation/notifications). Ask: does denial
|
|
64
|
+
crash or degrade gracefully? Does the app explain WHY before prompting? Can the user re-grant?
|
|
65
|
+
|
|
66
|
+
### Biometrics [native]
|
|
67
|
+
|
|
68
|
+
If the app uses biometric auth: approve → succeeds; deny/cancel → password fallback works;
|
|
69
|
+
disable biometrics in OS mid-flow → graceful fallback. Android emulator can simulate a
|
|
70
|
+
fingerprint touch:
|
|
71
|
+
```bash
|
|
72
|
+
bin/wadb -e emu finger touch 1
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Push notifications [native]
|
|
76
|
+
|
|
77
|
+
If the app registers for push: deny the notification permission and confirm the app still
|
|
78
|
+
works; trigger a notification and tap it — does it deep-link to the right screen with context?
|
|
79
|
+
|
|
80
|
+
### Connectivity [both]
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
# Android — toggle network
|
|
84
|
+
bin/wadb shell svc wifi disable && bin/wadb shell svc data disable
|
|
85
|
+
# Run a submit/save flow — does it surface "offline" cleanly or hang/blank?
|
|
86
|
+
bin/wadb shell svc wifi enable && bin/wadb shell svc data enable
|
|
87
|
+
|
|
88
|
+
# Slow network: use the emulator console — telnet localhost 5554 → network speed gsm / network delay umts
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Ask: does the app surface offline cleanly, queue or fail loudly, auto-retry, or just spin?
|
|
92
|
+
|
|
93
|
+
### Rotation [both]
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
# Android
|
|
97
|
+
bin/wadb shell settings put system accelerometer_rotation 0
|
|
98
|
+
bin/wadb shell settings put system user_rotation 1 # landscape
|
|
99
|
+
$MCLI snapshot # does the layout reflow? keyboard cover input?
|
|
100
|
+
bin/wadb shell settings put system user_rotation 0 # portrait
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
iOS sim: Cmd+Right/Left Arrow in the Simulator to rotate. Check reflow, clipping, and that
|
|
104
|
+
no content becomes unreachable in landscape.
|
|
105
|
+
|
|
106
|
+
### On-screen keyboard [both]
|
|
107
|
+
|
|
108
|
+
Focus an input near the bottom of the screen and snapshot — does the keyboard occlude the
|
|
109
|
+
input or the submit button? Does the view scroll it into view? Does return/"Done" dismiss it
|
|
110
|
+
and submit correctly?
|
|
111
|
+
|
|
112
|
+
### Deep links / URL attack surface [both]
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
# NATIVE — exercise the app's own scheme/intent:
|
|
116
|
+
$MCLI deep-link "<app-scheme>://<a real path>"
|
|
117
|
+
$MCLI deep-link "<app-scheme>://<path>?id=&n=NaN"
|
|
118
|
+
bin/wadb shell am start -a android.intent.action.VIEW -d "<app-scheme>://<path>?x=$(python3 -c 'print("A"*5000)')"
|
|
119
|
+
|
|
120
|
+
# WEB — exercise the target URL:
|
|
121
|
+
$MCLI open-url "<base_url>/<a real deep view>"
|
|
122
|
+
$MCLI open-url "<base_url>/<view>?id=&n=NaN"
|
|
123
|
+
$MCLI open-url "<base_url>/<view>?next=javascript:alert(1)"
|
|
124
|
+
$MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Crash, freeze, blank screen, open-redirect, or accept-as-valid are all findings.
|
|
128
|
+
|
|
129
|
+
### Privacy in app-switcher / Recents [both]
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
# Navigate to a view with sensitive data, then open Recents and screenshot the preview
|
|
133
|
+
bin/wadb shell input keyevent KEYCODE_APP_SWITCH # Android Recents
|
|
134
|
+
$MCLI screenshot output/sessions/<session-dir>/screenshots/recents-preview.png
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Is sensitive data (account info, tokens-in-URL, personal data) visible in the Recents preview?
|
|
138
|
+
A native app can mask its preview (FLAG_SECURE); web apps generally can't — note the leak as a
|
|
139
|
+
finding where it matters.
|
|
140
|
+
|
|
141
|
+
### Storage / data wipe
|
|
142
|
+
|
|
143
|
+
- **NATIVE:** force-clear app data mid-flow (Android `$MCLI relaunch-clean` = force-stop + `pm clear`) — does it land on a clean welcome state or get stuck on an intermediate screen? Also test fresh-install vs upgrade if you have both builds.
|
|
144
|
+
- **WEB:** **do NOT use `relaunch-clean`** — on Android it `pm clear`s the *browser* and wipes the logged-in profile, forcing a fresh SSO+MFA login. If you need a cleared-cookie test, do it knowingly and expect to re-login.
|
|
145
|
+
- Returning vs fresh state [both]: correct first-ever behavior (empty states, first-run interstitials) AND returning behavior.
|
|
146
|
+
|
|
147
|
+
### Empty / boundary states [both]
|
|
148
|
+
|
|
149
|
+
For each list / feed: zero items, exactly one item, many items. Check formatters at the
|
|
150
|
+
extremes (very large numbers, long names, long localized strings at mobile width).
|
|
151
|
+
|
|
152
|
+
## After Edge Cases
|
|
153
|
+
|
|
154
|
+
Write `phase-4-edge-cases.md` with one section per category. Each deferred test gets a single
|
|
155
|
+
line so it's visible.
|
|
156
|
+
|
|
157
|
+
**Update progress.**
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# Phase 7: Reflection & Reporting (Mobile)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/07-reporting.md` for the report structure, reflection
|
|
4
|
+
questions, bug-report template, session-report template, and stats.json schema. The
|
|
5
|
+
mobile additions (both modes):
|
|
6
|
+
|
|
7
|
+
## Stop recording
|
|
8
|
+
|
|
9
|
+
Replace `tracing-stop` with:
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
13
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
14
|
+
$MCLI record-stop # closes the video (if record-start was used)
|
|
15
|
+
$MCLI stop # close the app/browser — leaves the sim running for next time
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Do NOT `relaunch-clean` / wipe the device at session end — in WEB mode this preserves the
|
|
19
|
+
persisted login; in NATIVE mode it preserves any account/state created this session.
|
|
20
|
+
|
|
21
|
+
## Bug-report — mobile fields
|
|
22
|
+
|
|
23
|
+
In each `bugs/BUG-NNN.md`, replace the qa-explore `Environment` line with:
|
|
24
|
+
|
|
25
|
+
```markdown
|
|
26
|
+
**Environment:**
|
|
27
|
+
- Mode: [native / web]
|
|
28
|
+
- Platform: [android / ios]
|
|
29
|
+
- Device: [emulator-5554 / iPhone 17 sim UDID]
|
|
30
|
+
- OS version: [from `bin/wadb shell getprop ro.build.version.release` / `xcrun simctl`]
|
|
31
|
+
- App under test: [native: app id + version + build type | web: browser bundle + version]
|
|
32
|
+
- Target URL: [web mode only: base_url]
|
|
33
|
+
- Environment: [preprod / staging / production]
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
And add an **Evidence** block enriched for mobile:
|
|
37
|
+
|
|
38
|
+
```markdown
|
|
39
|
+
### Evidence
|
|
40
|
+
- Screenshot: `screenshots/bug-NNN.png`
|
|
41
|
+
- Video clip: `videos/bug-NNN.mp4` (if recorded)
|
|
42
|
+
- Process log: `logs/bug-NNN.log` (logcat / simctl excerpt around the time of failure — app process in native mode, browser process in web mode, not a page JS console)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
When you file a bug, capture the log at the moment of failure:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
$MCLI logs --since 30 --errors > output/sessions/<session-dir>/logs/bug-NNN.log
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Session report — mobile additions
|
|
52
|
+
|
|
53
|
+
Append a "Mobile context" section to `session-report.md`:
|
|
54
|
+
|
|
55
|
+
```markdown
|
|
56
|
+
## Mobile Context
|
|
57
|
+
|
|
58
|
+
| Field | Value |
|
|
59
|
+
|---|---|
|
|
60
|
+
| Mode | [native / web] |
|
|
61
|
+
| Platform | [android / ios] |
|
|
62
|
+
| Device | [device label] |
|
|
63
|
+
| OS version | [version] |
|
|
64
|
+
| App under test | [native: app id + version + build type | web: browser bundle + version] |
|
|
65
|
+
| Target URL | [web mode only: base_url + environment] |
|
|
66
|
+
| mobile-cli version | [git sha of this repo] |
|
|
67
|
+
|
|
68
|
+
## Deferred Tests (require additional tooling)
|
|
69
|
+
|
|
70
|
+
| Test | Why deferred | What's needed |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| Network failure simulation | mobile-cli v1 has no proxy | mitmproxy + cert install |
|
|
73
|
+
| Multi-device race conditions | mobile-cli v1 single-device | Second emulator + parallel sessions |
|
|
74
|
+
| Server-side parameter tampering (non-deep-link) | No script eval on device | Proxy + request modification |
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Requirements coverage cross-check (optional)
|
|
78
|
+
|
|
79
|
+
If the session was scoped to gathered requirements / tickets, add a cross-check table — this
|
|
80
|
+
is **not** test execution, it's "did exploration touch the areas those requirements
|
|
81
|
+
designed?":
|
|
82
|
+
|
|
83
|
+
```markdown
|
|
84
|
+
## Requirements Cross-Check
|
|
85
|
+
|
|
86
|
+
Source: <ticket / acceptance criteria / gathered context>
|
|
87
|
+
|
|
88
|
+
| Requirement / Area | Covered? | Found bug? | Notes |
|
|
89
|
+
|---|---|---|---|
|
|
90
|
+
| ... | Yes/No/Partial | BUG-NNN or — | ... |
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
The point is to surface coverage gaps the team can decide to fill manually or with automation.
|
|
94
|
+
|
|
95
|
+
## stats.json — mobile fields
|
|
96
|
+
|
|
97
|
+
Append a `mobile` block to the qa-explore stats schema:
|
|
98
|
+
|
|
99
|
+
```json
|
|
100
|
+
"mobile": {
|
|
101
|
+
"mode": "web",
|
|
102
|
+
"platform": "ios",
|
|
103
|
+
"device": "iPhone 17",
|
|
104
|
+
"app_under_test": "com.apple.mobilesafari",
|
|
105
|
+
"target_url": "https://staging.m.example.com",
|
|
106
|
+
"environment": "staging",
|
|
107
|
+
"videos_recorded": 1,
|
|
108
|
+
"log_errors_observed": 0,
|
|
109
|
+
"lifecycle_tests_run": 0,
|
|
110
|
+
"rotation_tests_run": 0,
|
|
111
|
+
"deferred_tests": []
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
(For native mode set `"mode": "native"`, `app_under_test` to the app id, and `target_url` to null.)
|
|
116
|
+
|
|
117
|
+
## Update progress.json — final
|
|
118
|
+
|
|
119
|
+
Same as qa-explore. Set `status: complete`, all phases to final status.
|
|
120
|
+
|
|
121
|
+
## Update indexes
|
|
122
|
+
|
|
123
|
+
Append the session to `output/sessions/INDEX.md`.
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Mobile Edge Cases — Checklist (native + web)
|
|
2
|
+
|
|
3
|
+
Treat as a thinking tool, not a checkbox sheet. Pick what matches the feature and the resolved
|
|
4
|
+
**mode**. Each entry is tagged **[both]**, **[native]**, or **[web]**, with the adb / simctl
|
|
5
|
+
trigger where possible. `<app>` = the app under test's package/bundle in NATIVE mode, or the
|
|
6
|
+
browser bundle (`com.android.chrome` / `com.apple.mobilesafari`) in WEB mode. `$MCLI` is the
|
|
7
|
+
`bin/mcli` wrapper; use `bin/wadb` for raw adb.
|
|
8
|
+
|
|
9
|
+
## Lifecycle [both]
|
|
10
|
+
|
|
11
|
+
| Edge | Android trigger | iOS sim trigger |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| Background app/browser | `bin/wadb shell input keyevent KEYCODE_HOME` | home gesture/button in Simulator |
|
|
14
|
+
| Kill app/browser | `bin/wadb shell am force-stop <app>` | `xcrun simctl terminate <udid> <app>` |
|
|
15
|
+
| Cold relaunch | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) |
|
|
16
|
+
| Clear data | NATIVE: `$MCLI relaunch-clean` (= force-stop + pm clear) — clean-state test. WEB: same command **wipes the logged-in profile; avoid unless intended** | NATIVE: `xcrun simctl uninstall` + reinstall. WEB: clear Safari data in sim Settings (drops the login) |
|
|
17
|
+
| Screen off / on | `bin/wadb shell input keyevent KEYCODE_POWER` (toggle) | n/a (sim doesn't lock realistically) |
|
|
18
|
+
|
|
19
|
+
**Ask:** does state survive? Does an in-progress action complete? Does auth/session persist, or is everything reloaded?
|
|
20
|
+
|
|
21
|
+
## Permissions [native] (and browser prompts in [web])
|
|
22
|
+
|
|
23
|
+
NATIVE: the app requests OS runtime permissions directly. WEB: the *page* requests them via
|
|
24
|
+
the BROWSER, so grant/revoke on the browser package controls `getUserMedia` camera,
|
|
25
|
+
geolocation, notifications. Use `<app>` = app package (native) or browser (web).
|
|
26
|
+
|
|
27
|
+
| Permission | Android revoke | Android grant | iOS sim |
|
|
28
|
+
|---|---|---|---|
|
|
29
|
+
| Camera | `bin/wadb shell pm revoke <app> android.permission.CAMERA` | `pm grant` | `xcrun simctl privacy <udid> reset camera <app>` (then re-prompt) |
|
|
30
|
+
| Location | `pm revoke <app> android.permission.ACCESS_FINE_LOCATION` | `pm grant` | `xcrun simctl privacy <udid> reset location <app>` |
|
|
31
|
+
| Notifications | `pm revoke <app> android.permission.POST_NOTIFICATIONS` | `pm grant` | `xcrun simctl privacy <udid> reset notifications <app>` |
|
|
32
|
+
| Contacts [native] | `pm revoke <app> android.permission.READ_CONTACTS` | `pm grant` | `xcrun simctl privacy <udid> reset contacts <app>` |
|
|
33
|
+
|
|
34
|
+
**Ask:** does the app explain WHY before prompting? Does denial crash or degrade gracefully? Can the user recover/re-grant?
|
|
35
|
+
|
|
36
|
+
## Biometrics [native]
|
|
37
|
+
|
|
38
|
+
App-level biometric auth only (a real sim browser doesn't expose a biometric flow to web).
|
|
39
|
+
Approve → succeeds; deny/cancel → password fallback; disable in OS mid-flow → graceful
|
|
40
|
+
fallback. Android emulator can simulate a fingerprint: `bin/wadb -e emu finger touch 1`.
|
|
41
|
+
|
|
42
|
+
## Push notifications [native]
|
|
43
|
+
|
|
44
|
+
If the app registers for push: deny the prompt and confirm the app still works; deliver a
|
|
45
|
+
notification and tap it — does it deep-link to the right screen with context?
|
|
46
|
+
|
|
47
|
+
## Connectivity [both]
|
|
48
|
+
|
|
49
|
+
| Edge | Android | iOS sim |
|
|
50
|
+
|---|---|---|
|
|
51
|
+
| Airplane mode | `bin/wadb shell svc wifi disable && bin/wadb shell svc data disable` (re-enable to undo) | Network Link Conditioner (manual) |
|
|
52
|
+
| Slow network | telnet to emulator console: `telnet localhost 5554`, then `network speed gsm` | Network Link Conditioner |
|
|
53
|
+
| Captive portal | hard — skip for v1 | hard — skip |
|
|
54
|
+
|
|
55
|
+
**Ask:** does the app surface offline cleanly? Does it queue actions or fail loudly? Does it
|
|
56
|
+
auto-retry? Does it leak retries (battery drain)?
|
|
57
|
+
|
|
58
|
+
## Hardware / display [both]
|
|
59
|
+
|
|
60
|
+
| Edge | Android | iOS sim |
|
|
61
|
+
|---|---|---|
|
|
62
|
+
| Rotation portrait/landscape | `bin/wadb shell settings put system user_rotation 0/1/2/3` | Cmd+Right/Left Arrow in Simulator |
|
|
63
|
+
| Dark mode | `bin/wadb shell "cmd uimode night yes"` (and `no`) | `xcrun simctl ui <udid> appearance dark/light` |
|
|
64
|
+
| Dynamic type / large font | `bin/wadb shell settings put system font_scale 1.5` | Settings app inside sim |
|
|
65
|
+
| Low battery | `bin/wadb shell dumpsys battery set level 10` (and `reset`) | n/a |
|
|
66
|
+
| Split-screen | manual via Recents | n/a |
|
|
67
|
+
|
|
68
|
+
**Ask:** does the layout reflow, clip, or hide content? Does it honor the OS dark-mode /
|
|
69
|
+
font-scale, or break? **[web]** also: address-bar show/hide on scroll changing the viewport
|
|
70
|
+
height (`100vh` jumps), momentum/overscroll bounce, and whether the page respects the
|
|
71
|
+
`viewport` meta (no desktop-width zoom-out, no horizontal scroll).
|
|
72
|
+
|
|
73
|
+
## Deep links / URL handling [both]
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
# NATIVE — the app's own scheme/intent
|
|
77
|
+
$MCLI deep-link "<app-scheme>://<a real path>"
|
|
78
|
+
$MCLI deep-link "<app-scheme>://<path>?id=&n=NaN"
|
|
79
|
+
bin/wadb shell am start -W -a android.intent.action.VIEW -d "<app-scheme>://<path>" -f 0x10000000
|
|
80
|
+
|
|
81
|
+
# WEB — the target URL
|
|
82
|
+
$MCLI open-url "<base_url>/<a real deep view>"
|
|
83
|
+
$MCLI open-url "<base_url>/<view>?id=&n=NaN"
|
|
84
|
+
$MCLI open-url "<base_url>/<view>?next=javascript:alert(1)"
|
|
85
|
+
$MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Ask:** crash, freeze, blank screen, open-redirect, or unsanitized reflection?
|
|
89
|
+
|
|
90
|
+
## Privacy / leak surface [both]
|
|
91
|
+
|
|
92
|
+
- **Recents preview:** background while on a sensitive screen, screencap Recents — is personal data / a token-in-URL visible? A native app can mask its preview (FLAG_SECURE); web pages generally can't.
|
|
93
|
+
- **Process log:** `$MCLI logs --errors` — grep for PII or credentials (emails, tokens, secrets) leaking into the app/browser process log.
|
|
94
|
+
- **URL leakage [web]:** sensitive values in query strings end up in history and Recents — flag where they shouldn't be.
|
|
95
|
+
- **Clipboard:** if the app copies sensitive text, does it auto-clear/expire?
|
|
96
|
+
|
|
97
|
+
## Storage / state [both]
|
|
98
|
+
|
|
99
|
+
- NATIVE: force-clear app data mid-flow (`pm clear`) — clean restart or stuck intermediate screen? Fresh-install vs upgrade: did state migrate?
|
|
100
|
+
- WEB: clearing browser cookies/data drops the login session — test the cleared-state behavior knowingly, not by accident.
|
|
101
|
+
- Returning vs fresh state: correct first-run interstitials, empty states, onboarding?
|
|
102
|
+
- Long-lived session: does a session that expires while the app/page is open prompt a graceful re-auth, or break into a blank/broken state?
|
|
103
|
+
|
|
104
|
+
## Native a11y [both] (driven through the device a11y tree)
|
|
105
|
+
|
|
106
|
+
- TalkBack (Android): `bin/wadb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/.TalkBackService`
|
|
107
|
+
- VoiceOver (iOS sim): mostly limited on sim — manual via the accessibility menu
|
|
108
|
+
- **Focus order:** swipe right repeatedly — does focus land on every interactive element in a sensible order?
|
|
109
|
+
- **Aggregation:** does each control announce its own label, or does the view dump many fields into one comma-joined label? (Known iOS issue — see `phases/03-discovery.md`.)
|
|
110
|
+
- **Contrast:** spot-check critical text — sufficient contrast in both light and dark mode?
|
|
111
|
+
- **Touch targets:** are tap targets finger-sized, or desktop-sized and hard to hit on a phone?
|
|
112
|
+
|
|
113
|
+
## Empty / boundary states [both]
|
|
114
|
+
|
|
115
|
+
- Zero items / one item / many items in lists.
|
|
116
|
+
- First-ever visit vs returning visit.
|
|
117
|
+
- Formatter extremes: very large numbers, long names, long localized strings at mobile width — truncation, overflow, `NaN`/`undefined`?
|
|
@@ -19,6 +19,12 @@ You are a **Principal QA Engineer** doing a focused 15-minute exploratory sessio
|
|
|
19
19
|
## Flow
|
|
20
20
|
|
|
21
21
|
### 1. Setup (~1 min)
|
|
22
|
+
|
|
23
|
+
Before opening the browser, check for project-local config:
|
|
24
|
+
1. If `qa/target.yml` exists in CWD, read it for base URL, auth strategy, and scope hints
|
|
25
|
+
2. If `qa/.env` exists in CWD, load credentials from it (never log these values)
|
|
26
|
+
3. If the target requires auth (storage_state or token strategy), apply it before navigating
|
|
27
|
+
|
|
22
28
|
```bash
|
|
23
29
|
playwright-cli open <url>
|
|
24
30
|
playwright-cli snapshot
|