qualiow-exploratory-testing 1.3.0 → 2.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (106) hide show
  1. package/.claude-plugin/marketplace.json +26 -0
  2. package/.claude-plugin/plugin.json +22 -0
  3. package/.env.example +27 -0
  4. package/CHANGELOG.md +259 -0
  5. package/README.md +472 -204
  6. package/agents/qa-diff-indexer-agent.md +81 -0
  7. package/agents/qa-gather-agent.md +41 -0
  8. package/agents/qa-page-mapper-agent.md +74 -0
  9. package/agents/qa-reporting-agent.md +103 -0
  10. package/{scripts → bin}/doctor-mobile.sh +57 -18
  11. package/bin/mcli +15 -9
  12. package/bin/mobile-cli.mjs +574 -286
  13. package/bin/qualiow +30 -0
  14. package/{scripts → bin}/setup-mobile.sh +56 -19
  15. package/bin/wadb +10 -4
  16. package/bin/wk-ios +29 -17
  17. package/bin/wkeval.mjs +7 -1
  18. package/data/domains/_default.yml +1 -1
  19. package/data/knowledge/changelog.yml +30 -12
  20. package/data/knowledge/manifest.yml +355 -127
  21. package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
  22. package/data/knowledge/releases/v0.1.0/release.yml +8 -1
  23. package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
  24. package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
  25. package/data/knowledge/releases/v0.6.0/release.yml +30 -0
  26. package/data/security/SECURITY-POLICY.md +273 -0
  27. package/data/targets/_default.yml +1 -1
  28. package/data/targets/_example-api-only.yml +80 -0
  29. package/data/targets/_example-mobile-emulation.yml +2 -2
  30. package/data/targets/_example-native-mobile.yml +2 -2
  31. package/data/targets/_example-sim-android-chrome.yml +1 -1
  32. package/data/targets/_example-sim-ios-safari.yml +1 -1
  33. package/data/templates/api-probe-matrix.md +3 -3
  34. package/data/templates/bug-report.md +29 -151
  35. package/data/templates/coverage-map.md +9 -6
  36. package/data/templates/expected-behaviour.md +3 -2
  37. package/data/templates/session-charter.md +3 -0
  38. package/data/templates/session-report.md +57 -80
  39. package/dist/cli/index.js +2286 -701
  40. package/dist/index.d.ts +637 -81
  41. package/dist/index.js +2058 -228
  42. package/docs/BACKEND-VERIFICATION.md +39 -9
  43. package/docs/GETTING-STARTED.md +300 -121
  44. package/docs/MOBILE-SETUP.md +149 -18
  45. package/hooks/hooks.json +27 -0
  46. package/hooks/scripts/read-guard.mjs +95 -0
  47. package/hooks/scripts/secret-patterns.d.mts +8 -0
  48. package/hooks/scripts/secret-patterns.mjs +136 -0
  49. package/hooks/scripts/write-guard.mjs +87 -0
  50. package/package.json +33 -18
  51. package/skills/qa-explore/SKILL.md +32 -25
  52. package/skills/qa-explore/phases/00-setup.md +76 -42
  53. package/skills/qa-explore/phases/01-auth.md +32 -15
  54. package/skills/qa-explore/phases/02-charter.md +16 -5
  55. package/skills/qa-explore/phases/03-discovery.md +93 -27
  56. package/skills/qa-explore/phases/04-journeys.md +2 -2
  57. package/skills/qa-explore/phases/05-features.md +39 -24
  58. package/skills/qa-explore/phases/06-edge-cases.md +24 -13
  59. package/skills/qa-explore/phases/07-reporting.md +102 -142
  60. package/skills/qa-explore/references/context-integration.md +1 -1
  61. package/skills/qa-explore/references/delegation-rules.md +82 -0
  62. package/skills/qa-explore/references/output-contract.md +192 -0
  63. package/skills/qa-explore/references/paths.md +108 -0
  64. package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
  65. package/skills/qa-explore/references/security-rules.md +70 -25
  66. package/skills/qa-explore/references/session-rules.md +2 -1
  67. package/skills/qa-explore/references/severity-guide.md +31 -4
  68. package/skills/qa-explore-cleanup/SKILL.md +51 -15
  69. package/skills/qa-explore-feedback/SKILL.md +23 -5
  70. package/skills/qa-explore-mobile/SKILL.md +86 -64
  71. package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
  72. package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
  73. package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
  74. package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
  75. package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
  76. package/skills/qa-explore-mobile/phases/05-features.md +9 -9
  77. package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
  78. package/skills/qa-explore-mobile/phases/07-reporting.md +108 -60
  79. package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
  80. package/skills/qa-explore-quick/SKILL.md +71 -26
  81. package/skills/qa-explore-report/SKILL.md +65 -35
  82. package/skills/qa-gather/SKILL.md +44 -8
  83. package/skills/qa-knowledge-add/SKILL.md +25 -9
  84. package/skills/qa-knowledge-list/SKILL.md +37 -40
  85. package/skills/qa-target-setup/SKILL.md +63 -41
  86. package/skills/qa-verify-backend/SKILL.md +31 -19
  87. package/skills/qa-verify-backend/phases/00-setup.md +57 -12
  88. package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
  89. package/skills/qa-verify-backend/phases/02-static-review.md +11 -0
  90. package/skills/qa-verify-backend/phases/03-live-verification.md +5 -3
  91. package/skills/qa-verify-backend/phases/03b-api-verification.md +24 -9
  92. package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
  93. package/skills/qa-verify-backend/phases/05-reporting.md +72 -12
  94. package/skills/qa-verify-backend/references/api-probes.md +28 -1
  95. package/skills/qa-verify-backend/references/payload-verification.md +1 -1
  96. package/skills/qa-verify-backend/references/release-readiness.md +1 -1
  97. package/skills/qa-verify-backend/references/safety-rules.md +13 -8
  98. package/data/domains/_default.md +0 -181
  99. package/data/domains/ecommerce.md +0 -192
  100. package/data/domains/fintech.md +0 -200
  101. package/data/domains/identity.md +0 -88
  102. package/data/domains/marketing.md +0 -194
  103. package/data/domains/saas.md +0 -197
  104. package/dist/cli/index.d.ts +0 -1
  105. package/dist/cli/index.js.map +0 -1
  106. package/dist/index.js.map +0 -1
@@ -1,27 +1,21 @@
1
1
  # Phase 6: Edge Cases & Mobile-Specific (mode-aware)
2
2
 
3
- Follow `../../qa-explore/phases/06-edge-cases.md` for the saboteur mindset. Edges are tagged
3
+ Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/06-edge-cases.md` for the saboteur mindset. Edges are tagged
4
4
  **[both]**, **[native]**, or **[web]** — run the ones that fit the resolved mode.
5
5
 
6
- Set up once per Bash block:
7
- ```bash
8
- export MOBILE_CLI_STATE=/tmp/<target>-state.json
9
- MCLI="$PWD/bin/mcli" # run from the repo root
10
- ```
11
-
12
6
  ## Dropped / limited (no DOM, no JS console)
13
7
 
14
8
  - **XSS execution assertions via `fill` [web]** — without DOM/console you can't confirm execution. You can still submit payloads and watch for gross breakage (crash, broken render, reflected unescaped text in the a11y tree).
15
9
  - **`route ... --status=500` network simulation [both]** — needs mitmproxy. **Deferred** — log scenarios that would benefit, run them once a proxy exists.
16
- - **Multi-session race [both]** — needs two emulators. **Deferred for v1.**
10
+ - **Multi-session race [both]** — needs two emulators. **Deferred.**
17
11
  - **`resize` viewport tests** — sim viewport is fixed. Use **rotation** instead (below).
18
12
 
19
13
  ## Kept (still useful on mobile)
20
14
 
21
- - **Unicode / emoji [both]** in inputs: `$MCLI fill <ref> "naïve café 🤑 العربية"`
22
- - **Whitespace-only [both]:** `$MCLI fill <ref> " "`
15
+ - **Unicode / emoji [both]** in inputs: `qa/bin/mcli fill <ref> "naïve café 🤑 العربية"`
16
+ - **Whitespace-only [both]:** `qa/bin/mcli fill <ref> " "`
23
17
  - **Extremely long [both]:** see the truncation caveat in `05-features.md`
24
- - **State manipulation via deep link / URL [both]:** NATIVE `$MCLI deep-link "<app-scheme>://<path>?<tampered>"`; WEB `$MCLI open-url "<base_url>/<deep view>?<tampered params>"`
18
+ - **State manipulation via deep link / URL [both]:** NATIVE `qa/bin/mcli deep-link "<app-scheme>://<path>?<tampered>"`; WEB `qa/bin/mcli open-url "<base_url>/<deep view>?<tampered params>"`
25
19
  - **Replay [both]:** double-tap submit; submit then back then re-submit
26
20
  - **Brute force / rate limits [both]:** repeated failed submissions/logins — lockout / cooldown?
27
21
 
@@ -31,21 +25,20 @@ MCLI="$PWD/bin/mcli" # run from the repo root
31
25
 
32
26
  ```bash
33
27
  # Background mid-flow
34
- $MCLI press HOME # Android
35
- # (iOS: use the home gesture/button via the Simulator; press HOME is Android-only)
28
+ qa/bin/mcli press HOME # both platforms (iOS via Maestro pressKey Home)
36
29
  sleep 30
37
- $MCLI launch # foreground — did state restore or reload-and-lose-state?
30
+ qa/bin/mcli launch # foreground — did state restore or reload-and-lose-state?
38
31
 
39
32
  # Kill mid-flow
40
- $MCLI stop
41
- $MCLI launch
33
+ qa/bin/mcli stop
34
+ qa/bin/mcli launch
42
35
  # WEB only: re-open the view you were on
43
- # $MCLI open-url <current view>
36
+ # qa/bin/mcli open-url <current view>
44
37
 
45
38
  # OS-initiated suspend (Android)
46
- bin/wadb shell am broadcast -a android.intent.action.SCREEN_OFF
39
+ qa/bin/wadb shell am broadcast -a android.intent.action.SCREEN_OFF
47
40
  sleep 5
48
- bin/wadb shell am broadcast -a android.intent.action.SCREEN_ON
41
+ qa/bin/wadb shell am broadcast -a android.intent.action.SCREEN_ON
49
42
  ```
50
43
 
51
44
  For each: did state survive? Was auth still valid? Did an in-progress action complete or
@@ -55,8 +48,8 @@ duplicate?
55
48
 
56
49
  NATIVE — deny / revoke / re-grant runtime permissions and check graceful handling:
57
50
  ```bash
58
- bin/wadb shell pm revoke <app.package> android.permission.CAMERA # then try the QR/camera feature
59
- bin/wadb shell pm grant <app.package> android.permission.CAMERA
51
+ qa/bin/wadb shell pm revoke <app.package> android.permission.CAMERA # then try the QR/camera feature
52
+ qa/bin/wadb shell pm grant <app.package> android.permission.CAMERA
60
53
  # iOS sim: xcrun simctl privacy <udid> reset|grant camera <app.bundle_id>
61
54
  ```
62
55
  Test camera / notifications / contacts / location. In WEB mode the same applies to the
@@ -69,7 +62,7 @@ If the app uses biometric auth: approve → succeeds; deny/cancel → password f
69
62
  disable biometrics in OS mid-flow → graceful fallback. Android emulator can simulate a
70
63
  fingerprint touch:
71
64
  ```bash
72
- bin/wadb -e emu finger touch 1
65
+ qa/bin/wadb -e emu finger touch 1
73
66
  ```
74
67
 
75
68
  ### Push notifications [native]
@@ -81,11 +74,13 @@ works; trigger a notification and tap it — does it deep-link to the right scre
81
74
 
82
75
  ```bash
83
76
  # Android — toggle network
84
- bin/wadb shell svc wifi disable && bin/wadb shell svc data disable
77
+ qa/bin/wadb shell svc wifi disable && qa/bin/wadb shell svc data disable
85
78
  # Run a submit/save flow — does it surface "offline" cleanly or hang/blank?
86
- bin/wadb shell svc wifi enable && bin/wadb shell svc data enable
79
+ qa/bin/wadb shell svc wifi enable && qa/bin/wadb shell svc data enable
87
80
 
88
- # Slow network: use the emulator console — telnet localhost 5554 → network speed gsm / network delay umts
81
+ # Slow network (Android emulator console via adb, no telnet):
82
+ qa/bin/wadb emu network speed gsm && qa/bin/wadb emu network delay umts
83
+ # restore: qa/bin/wadb emu network speed full && qa/bin/wadb emu network delay none
89
84
  ```
90
85
 
91
86
  Ask: does the app surface offline cleanly, queue or fail loudly, auto-retry, or just spin?
@@ -94,10 +89,10 @@ Ask: does the app surface offline cleanly, queue or fail loudly, auto-retry, or
94
89
 
95
90
  ```bash
96
91
  # Android
97
- bin/wadb shell settings put system accelerometer_rotation 0
98
- bin/wadb shell settings put system user_rotation 1 # landscape
99
- $MCLI snapshot # does the layout reflow? keyboard cover input?
100
- bin/wadb shell settings put system user_rotation 0 # portrait
92
+ qa/bin/wadb shell settings put system accelerometer_rotation 0
93
+ qa/bin/wadb shell settings put system user_rotation 1 # landscape
94
+ qa/bin/mcli snapshot # does the layout reflow? keyboard cover input?
95
+ qa/bin/wadb shell settings put system user_rotation 0 # portrait
101
96
  ```
102
97
 
103
98
  iOS sim: Cmd+Right/Left Arrow in the Simulator to rotate. Check reflow, clipping, and that
@@ -113,15 +108,15 @@ and submit correctly?
113
108
 
114
109
  ```bash
115
110
  # NATIVE — exercise the app's own scheme/intent:
116
- $MCLI deep-link "<app-scheme>://<a real path>"
117
- $MCLI deep-link "<app-scheme>://<path>?id=&n=NaN"
118
- bin/wadb shell am start -a android.intent.action.VIEW -d "<app-scheme>://<path>?x=$(python3 -c 'print("A"*5000)')"
111
+ qa/bin/mcli deep-link "<app-scheme>://<a real path>"
112
+ qa/bin/mcli deep-link "<app-scheme>://<path>?id=&n=NaN"
113
+ qa/bin/wadb shell am start -a android.intent.action.VIEW -d "<app-scheme>://<path>?x=$(python3 -c 'print("A"*5000)')"
119
114
 
120
115
  # WEB — exercise the target URL:
121
- $MCLI open-url "<base_url>/<a real deep view>"
122
- $MCLI open-url "<base_url>/<view>?id=&n=NaN"
123
- $MCLI open-url "<base_url>/<view>?next=javascript:alert(1)"
124
- $MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
116
+ qa/bin/mcli open-url "<base_url>/<a real deep view>"
117
+ qa/bin/mcli open-url "<base_url>/<view>?id=&n=NaN"
118
+ qa/bin/mcli open-url "<base_url>/<view>?next=javascript:alert(1)"
119
+ qa/bin/mcli open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
125
120
  ```
126
121
 
127
122
  Crash, freeze, blank screen, open-redirect, or accept-as-valid are all findings.
@@ -130,8 +125,8 @@ Crash, freeze, blank screen, open-redirect, or accept-as-valid are all findings.
130
125
 
131
126
  ```bash
132
127
  # Navigate to a view with sensitive data, then open Recents and screenshot the preview
133
- bin/wadb shell input keyevent KEYCODE_APP_SWITCH # Android Recents
134
- $MCLI screenshot output/sessions/<session-dir>/screenshots/recents-preview.png
128
+ qa/bin/wadb shell input keyevent KEYCODE_APP_SWITCH # Android Recents
129
+ qa/bin/mcli screenshot output/sessions/<session-dir>/screenshots/recents-preview.png
135
130
  ```
136
131
 
137
132
  Is sensitive data (account info, tokens-in-URL, personal data) visible in the Recents preview?
@@ -140,7 +135,7 @@ finding where it matters.
140
135
 
141
136
  ### Storage / data wipe
142
137
 
143
- - **NATIVE:** force-clear app data mid-flow (Android `$MCLI relaunch-clean` = force-stop + `pm clear`) — does it land on a clean welcome state or get stuck on an intermediate screen? Also test fresh-install vs upgrade if you have both builds.
138
+ - **NATIVE:** force-clear app data mid-flow (Android `qa/bin/mcli relaunch-clean` = force-stop + `pm clear` + launch) — does it land on a clean welcome state or get stuck on an intermediate screen? Also test fresh-install vs upgrade if you have both builds.
144
139
  - **WEB:** **do NOT use `relaunch-clean`** — on Android it `pm clear`s the *browser* and wipes the logged-in profile, forcing a fresh SSO+MFA login. If you need a cleared-cookie test, do it knowingly and expect to re-login.
145
140
  - Returning vs fresh state [both]: correct first-ever behavior (empty states, first-run interstitials) AND returning behavior.
146
141
 
@@ -151,7 +146,7 @@ extremes (very large numbers, long names, long localized strings at mobile width
151
146
 
152
147
  ## After Edge Cases
153
148
 
154
- Write `phase-4-edge-cases.md` with one section per category. Each deferred test gets a single
149
+ Write `phase-6-edge-cases.md` (confidentiality header first) with one section per category. Each deferred test gets a single
155
150
  line so it's visible.
156
151
 
157
152
  **Update progress.**
@@ -1,84 +1,112 @@
1
1
  # Phase 7: Reflection & Reporting (Mobile)
2
2
 
3
- Follow `../../qa-explore/phases/07-reporting.md` for the report structure, reflection
4
- questions, bug-report template, session-report template, and stats.json schema. The
5
- mobile additions (both modes):
3
+ Follow `${CLAUDE_SKILL_DIR}/../qa-explore/phases/07-reporting.md` for the reflection
4
+ questions, the `phase-7-notes.md` sections and the hand-off to the reporting agent; every
5
+ artefact uses the format in
6
+ `${CLAUDE_SKILL_DIR}/../qa-explore/references/output-contract.md` (confidentiality header
7
+ first). The mobile differences (both modes):
6
8
 
7
9
  ## Stop recording
8
10
 
9
- Replace `tracing-stop` with:
11
+ Replace `video-stop` / `tracing-stop` with:
10
12
 
11
13
  ```bash
12
- export MOBILE_CLI_STATE=/tmp/<target>-state.json
13
- MCLI="$PWD/bin/mcli" # run from the repo root
14
- $MCLI record-stop # closes the video (if record-start was used)
15
- $MCLI stop # close the app/browser — leaves the sim running for next time
14
+ qa/bin/mcli record-stop # closes the .mp4 (if record-start was used)
15
+ qa/bin/mcli stop # close the app/browser — leaves the sim running for next time
16
+ qa/bin/wk-ios --stop # iOS WEB mode only, if the DOM bridge was used
16
17
  ```
17
18
 
18
- Do NOT `relaunch-clean` / wipe the device at session end — in WEB mode this preserves the
19
- persisted login; in NATIVE mode it preserves any account/state created this session.
19
+ Do NOT `relaunch-clean` / wipe the device at session end: in WEB mode this preserves the
20
+ persisted login; in NATIVE mode it preserves any account/state created this session. There
21
+ is no playwright-cli session to `close` / `delete-data` on mobile.
20
22
 
21
- ## Bug-report — mobile fields
23
+ ## Bug report — mobile fields
22
24
 
23
- In each `bugs/BUG-NNN.md`, replace the qa-explore `Environment` line with:
25
+ Same file and headings as the contract — `bugs/BUG-NNN.md`, starting with the two-line
26
+ header, verbatim:
27
+
28
+ ```
29
+ > CONFIDENTIAL: This report may contain internal URLs, security vulnerabilities,
30
+ > and application details. Do not share outside your organization without review.
31
+ ```
32
+
33
+ then `# BUG-NNN: [Component] fails [Condition] causing [Impact]`, `**Severity:**`,
34
+ `**Priority:**`, `**Component:**`, `**URL:**`, `**Environment:**`,
35
+ `**Reproduction rate:**`, then `## Summary`, `## Expected Behavior`, `## Actual Behavior`,
36
+ `## Steps to Reproduce`, `## Business Impact`, `## Evidence` and
37
+ `## Recommended Fix Priority`. Two fields change on mobile:
24
38
 
25
39
  ```markdown
40
+ **URL:** <WEB: the page URL loaded via open-url | NATIVE: the screen name or deep link>
26
41
  **Environment:**
27
- - Mode: [native / web]
28
- - Platform: [android / ios]
29
- - Device: [emulator-5554 / iPhone 17 sim UDID]
30
- - OS version: [from `bin/wadb shell getprop ro.build.version.release` / `xcrun simctl`]
31
- - App under test: [native: app id + version + build type | web: browser bundle + version]
32
- - Target URL: [web mode only: base_url]
33
- - Environment: [preprod / staging / production]
42
+ - Mode: native | web
43
+ - Platform: android | ios
44
+ - Device: emulator-5554 | qa-iphone (UDID …)
45
+ - OS version: <android: qa/bin/wadb shell getprop ro.build.version.release — ios: xcrun simctl list runtimes>
46
+ - App under test: <native: app id + version + build type | web: browser bundle + version>
47
+ - Target URL: <web mode only: base_url>
48
+ - Environment: <preprod | staging | production>
34
49
  ```
35
50
 
36
- And add an **Evidence** block enriched for mobile:
51
+ And `## Evidence` carries the mobile artefacts:
37
52
 
38
53
  ```markdown
39
- ### Evidence
40
- - Screenshot: `screenshots/bug-NNN.png`
41
- - Video clip: `videos/bug-NNN.mp4` (if recorded)
42
- - Process log: `logs/bug-NNN.log` (logcat / simctl excerpt around the time of failure — app process in native mode, browser process in web mode, not a page JS console)
54
+ ## Evidence
55
+ - Screenshot: `screenshots/BUG-NNN.png`
56
+ - Video: `videos/BUG-NNN.mp4` (if recorded)
57
+ - Log: `logs/BUG-NNN.log` (logcat / simctl excerpt around the failure — iOS is scoped to the app/browser process, Android is whole-device logcat, and neither is a page JS console)
58
+ - Console errors: n/a on mobile (or the wk-ios observation, iOS WEB)
59
+ - Network failures: n/a (no proxy) — or "needs proxy capture"
43
60
  ```
44
61
 
45
- When you file a bug, capture the log at the moment of failure:
62
+ Capture the log at the moment of failure (the `logs/` directory was created in setup):
46
63
 
47
64
  ```bash
48
- $MCLI logs --since 30 --errors > output/sessions/<session-dir>/logs/bug-NNN.log
65
+ qa/bin/mcli screenshot output/sessions/<session-dir>/screenshots/BUG-NNN.png
66
+ qa/bin/mcli logs --since 30 --errors > output/sessions/<session-dir>/logs/BUG-NNN.log
49
67
  ```
50
68
 
51
- ## Session report — mobile additions
69
+ Redact the log excerpt per `security-rules.md` before it is saved: process logs routinely
70
+ carry tokens and emails.
71
+
72
+ ## phase-7-notes.md — mobile additions
52
73
 
53
- Append a "Mobile context" section to `session-report.md`:
74
+ Write `phase-7-notes.md` with the sections the web phase lists (`## Executive Summary`,
75
+ `## Coverage Map`, `## Observations`, `## Areas Not Tested`, `## Recommendations`,
76
+ `## Reflection`), then these two — they are copied into `session-report.md` after
77
+ `## Session Stats`:
54
78
 
55
79
  ```markdown
56
80
  ## Mobile Context
57
81
 
58
82
  | Field | Value |
59
83
  |---|---|
60
- | Mode | [native / web] |
61
- | Platform | [android / ios] |
62
- | Device | [device label] |
63
- | OS version | [version] |
64
- | App under test | [native: app id + version + build type | web: browser bundle + version] |
65
- | Target URL | [web mode only: base_url + environment] |
66
- | mobile-cli version | [git sha of this repo] |
84
+ | Mode | native / web |
85
+ | Platform | android / ios |
86
+ | Device | <device label> |
87
+ | OS version | <version> |
88
+ | App under test | <native: app id + version + build type | web: browser bundle + version> |
89
+ | Target URL | <web mode only: base_url + environment> |
90
+ | Source state | <native: branch + sha + clean/dirty, if rebuilt> |
91
+ | Driver version | <output of `qa/bin/mcli version`> |
67
92
 
68
93
  ## Deferred Tests (require additional tooling)
69
94
 
70
95
  | Test | Why deferred | What's needed |
71
96
  |---|---|---|
72
- | Network failure simulation | mobile-cli v1 has no proxy | mitmproxy + cert install |
73
- | Multi-device race conditions | mobile-cli v1 single-device | Second emulator + parallel sessions |
74
- | Server-side parameter tampering (non-deep-link) | No script eval on device | Proxy + request modification |
97
+ | Network failure simulation | mobile-cli has no proxy | mitmproxy + cert install |
98
+ | Multi-device race conditions | single device | second emulator + parallel sessions |
99
+ | Server-side parameter tampering (non-deep-link) | no script eval on device | proxy + request modification |
75
100
  ```
76
101
 
102
+ Every deferred test also appears in the coverage map as `not-tested` with the reason, so it
103
+ shows up as a known gap.
104
+
77
105
  ## Requirements coverage cross-check (optional)
78
106
 
79
- If the session was scoped to gathered requirements / tickets, add a cross-check table — this
80
- is **not** test execution, it's "did exploration touch the areas those requirements
81
- designed?":
107
+ If the session was scoped to gathered requirements / tickets, add a cross-check table to the
108
+ notes; it is carried into the report with the other sections. This is **not** test execution;
109
+ it is "did exploration touch the areas those requirements designed?":
82
110
 
83
111
  ```markdown
84
112
  ## Requirements Cross-Check
@@ -90,34 +118,54 @@ Source: <ticket / acceptance criteria / gathered context>
90
118
  | ... | Yes/No/Partial | BUG-NNN or — | ... |
91
119
  ```
92
120
 
93
- The point is to surface coverage gaps the team can decide to fill manually or with automation.
94
-
95
121
  ## stats.json — mobile fields
96
122
 
97
- Append a `mobile` block to the qa-explore stats schema:
123
+ `stats.json` exactly as the contract defines it, with `"kind": "mobile"`, and the device
124
+ block under `coverage.mobile` (the schema is strict; no other top-level keys):
98
125
 
99
126
  ```json
100
- "mobile": {
101
- "mode": "web",
102
- "platform": "ios",
103
- "device": "iPhone 17",
104
- "app_under_test": "com.apple.mobilesafari",
105
- "target_url": "https://staging.m.example.com",
106
- "environment": "staging",
107
- "videos_recorded": 1,
108
- "log_errors_observed": 0,
109
- "lifecycle_tests_run": 0,
110
- "rotation_tests_run": 0,
111
- "deferred_tests": []
127
+ "coverage": {
128
+ "mobile": {
129
+ "mode": "web",
130
+ "platform": "ios",
131
+ "device": "qa-iphone",
132
+ "app_under_test": "com.apple.mobilesafari",
133
+ "target_url": "https://staging.m.example.com",
134
+ "environment": "staging",
135
+ "videos_recorded": 1,
136
+ "log_errors_observed": 0,
137
+ "lifecycle_tests_run": 0,
138
+ "rotation_tests_run": 0,
139
+ "deferred_tests": []
140
+ }
112
141
  }
113
142
  ```
114
143
 
115
144
  (For native mode set `"mode": "native"`, `app_under_test` to the app id, and `target_url` to null.)
116
145
 
117
- ## Update progress.json — final
146
+ ## Assemble and Finalize
147
+
148
+ Invoke the `qa-reporting-agent` sub-agent (`qualiow:qa-reporting-agent` under a plugin
149
+ install) with the session directory and `kind: mobile`. It assembles `session-report.md` from
150
+ `phase-7-notes.md`, the bugs, `stats.json` and the phase files — keeping `## Mobile Context`
151
+ and `## Deferred Tests` after `## Session Stats` — and then runs
152
+ `qualiow session finalize`, which validates `stats.json` against the strict schema, checks the
153
+ confidentiality header on every artefact and scans the directory against the redaction list —
154
+ the logcat / simctl excerpts under `logs/` are scanned like everything else — then appends the
155
+ session row to `output/sessions/INDEX.md` (`Kind` = `mobile`) and one row per bug to
156
+ `output/bugs/all-bugs.md`, records the session metrics and sets `progress.json` to
157
+ `complete`. It is idempotent.
158
+
159
+ Read its return (at most 8 lines). If it reports a missing notes section or a violation it
160
+ could not fix, fix that yourself and re-run the check:
118
161
 
119
- Same as qa-explore. Set `status: complete`, all phases to final status.
162
+ ```bash
163
+ qualiow session finalize output/sessions/<session-dir> --check
164
+ ```
120
165
 
121
- ## Update indexes
166
+ `--check` writes nothing and prints a numbered list of violations naming each file. Resolve
167
+ the `qualiow` prefix per `${CLAUDE_SKILL_DIR}/../qa-explore/references/paths.md`; if the CLI
168
+ is unavailable, write the rows by hand in the column order defined there.
122
169
 
123
- Append the session to `output/sessions/INDEX.md`.
170
+ If sub-agents are unavailable, do the step yourself as in 2.1.0: write `session-report.md`
171
+ from the notes in the contract's format and run `qualiow session finalize output/sessions/<session-dir>`.
@@ -3,18 +3,18 @@
3
3
  Treat as a thinking tool, not a checkbox sheet. Pick what matches the feature and the resolved
4
4
  **mode**. Each entry is tagged **[both]**, **[native]**, or **[web]**, with the adb / simctl
5
5
  trigger where possible. `<app>` = the app under test's package/bundle in NATIVE mode, or the
6
- browser bundle (`com.android.chrome` / `com.apple.mobilesafari`) in WEB mode. `$MCLI` is the
7
- `bin/mcli` wrapper; use `bin/wadb` for raw adb.
6
+ browser bundle (`com.android.chrome` / `com.apple.mobilesafari`) in WEB mode. `qa/bin/mcli` is the
7
+ driver prefix resolved in phase 0 (`bin/mcli` in a checkout); use `qa/bin/wadb` for raw adb.
8
8
 
9
9
  ## Lifecycle [both]
10
10
 
11
11
  | Edge | Android trigger | iOS sim trigger |
12
12
  |---|---|---|
13
- | Background app/browser | `bin/wadb shell input keyevent KEYCODE_HOME` | home gesture/button in Simulator |
14
- | Kill app/browser | `bin/wadb shell am force-stop <app>` | `xcrun simctl terminate <udid> <app>` |
15
- | Cold relaunch | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) |
16
- | Clear data | NATIVE: `$MCLI relaunch-clean` (= force-stop + pm clear) — clean-state test. WEB: same command **wipes the logged-in profile; avoid unless intended** | NATIVE: `xcrun simctl uninstall` + reinstall. WEB: clear Safari data in sim Settings (drops the login) |
17
- | Screen off / on | `bin/wadb shell input keyevent KEYCODE_POWER` (toggle) | n/a (sim doesn't lock realistically) |
13
+ | Background app/browser | `qa/bin/mcli press HOME` | `qa/bin/mcli press HOME` (Maestro `pressKey: Home`) |
14
+ | Kill app/browser | `qa/bin/wadb shell am force-stop <app>` | `xcrun simctl terminate <udid> <app>` |
15
+ | Cold relaunch | `qa/bin/mcli launch` (WEB: then `qa/bin/mcli open-url <url>`) | `qa/bin/mcli launch` (WEB: then `qa/bin/mcli open-url <url>`) |
16
+ | Clear data | NATIVE: `qa/bin/mcli relaunch-clean` (= force-stop + pm clear + launch) — clean-state test. WEB: same command **wipes the logged-in profile; avoid unless intended** | NATIVE: `xcrun simctl uninstall` + reinstall. WEB: clear Safari data in sim Settings (drops the login) |
17
+ | Screen off / on | `qa/bin/wadb shell input keyevent KEYCODE_POWER` (toggle) | n/a (sim doesn't lock realistically) |
18
18
 
19
19
  **Ask:** does state survive? Does an in-progress action complete? Does auth/session persist, or is everything reloaded?
20
20
 
@@ -26,7 +26,7 @@ geolocation, notifications. Use `<app>` = app package (native) or browser (web).
26
26
 
27
27
  | Permission | Android revoke | Android grant | iOS sim |
28
28
  |---|---|---|---|
29
- | Camera | `bin/wadb shell pm revoke <app> android.permission.CAMERA` | `pm grant` | `xcrun simctl privacy <udid> reset camera <app>` (then re-prompt) |
29
+ | Camera | `qa/bin/wadb shell pm revoke <app> android.permission.CAMERA` | `pm grant` | `xcrun simctl privacy <udid> reset camera <app>` (then re-prompt) |
30
30
  | Location | `pm revoke <app> android.permission.ACCESS_FINE_LOCATION` | `pm grant` | `xcrun simctl privacy <udid> reset location <app>` |
31
31
  | Notifications | `pm revoke <app> android.permission.POST_NOTIFICATIONS` | `pm grant` | `xcrun simctl privacy <udid> reset notifications <app>` |
32
32
  | Contacts [native] | `pm revoke <app> android.permission.READ_CONTACTS` | `pm grant` | `xcrun simctl privacy <udid> reset contacts <app>` |
@@ -37,7 +37,7 @@ geolocation, notifications. Use `<app>` = app package (native) or browser (web).
37
37
 
38
38
  App-level biometric auth only (a real sim browser doesn't expose a biometric flow to web).
39
39
  Approve → succeeds; deny/cancel → password fallback; disable in OS mid-flow → graceful
40
- fallback. Android emulator can simulate a fingerprint: `bin/wadb -e emu finger touch 1`.
40
+ fallback. Android emulator can simulate a fingerprint: `qa/bin/wadb -e emu finger touch 1`.
41
41
 
42
42
  ## Push notifications [native]
43
43
 
@@ -48,9 +48,9 @@ notification and tap it — does it deep-link to the right screen with context?
48
48
 
49
49
  | Edge | Android | iOS sim |
50
50
  |---|---|---|
51
- | Airplane mode | `bin/wadb shell svc wifi disable && bin/wadb shell svc data disable` (re-enable to undo) | Network Link Conditioner (manual) |
52
- | Slow network | telnet to emulator console: `telnet localhost 5554`, then `network speed gsm` | Network Link Conditioner |
53
- | Captive portal | hard — skip for v1 | hard — skip |
51
+ | Airplane mode | `qa/bin/wadb shell svc wifi disable && qa/bin/wadb shell svc data disable` (re-enable to undo) | Network Link Conditioner (manual) |
52
+ | Slow network | `qa/bin/wadb emu network speed gsm` (restore with `speed full`) | Network Link Conditioner |
53
+ | Captive portal | hard — skip | hard — skip |
54
54
 
55
55
  **Ask:** does the app surface offline cleanly? Does it queue actions or fail loudly? Does it
56
56
  auto-retry? Does it leak retries (battery drain)?
@@ -59,10 +59,10 @@ auto-retry? Does it leak retries (battery drain)?
59
59
 
60
60
  | Edge | Android | iOS sim |
61
61
  |---|---|---|
62
- | Rotation portrait/landscape | `bin/wadb shell settings put system user_rotation 0/1/2/3` | Cmd+Right/Left Arrow in Simulator |
63
- | Dark mode | `bin/wadb shell "cmd uimode night yes"` (and `no`) | `xcrun simctl ui <udid> appearance dark/light` |
64
- | Dynamic type / large font | `bin/wadb shell settings put system font_scale 1.5` | Settings app inside sim |
65
- | Low battery | `bin/wadb shell dumpsys battery set level 10` (and `reset`) | n/a |
62
+ | Rotation portrait/landscape | `qa/bin/wadb shell settings put system user_rotation 0/1/2/3` | Cmd+Right/Left Arrow in Simulator |
63
+ | Dark mode | `qa/bin/wadb shell "cmd uimode night yes"` (and `no`) | `xcrun simctl ui <udid> appearance dark/light` |
64
+ | Dynamic type / large font | `qa/bin/wadb shell settings put system font_scale 1.5` | Settings app inside sim |
65
+ | Low battery | `qa/bin/wadb shell dumpsys battery set level 10` (and `reset`) | n/a |
66
66
  | Split-screen | manual via Recents | n/a |
67
67
 
68
68
  **Ask:** does the layout reflow, clip, or hide content? Does it honor the OS dark-mode /
@@ -74,15 +74,15 @@ height (`100vh` jumps), momentum/overscroll bounce, and whether the page respect
74
74
 
75
75
  ```bash
76
76
  # NATIVE — the app's own scheme/intent
77
- $MCLI deep-link "<app-scheme>://<a real path>"
78
- $MCLI deep-link "<app-scheme>://<path>?id=&n=NaN"
79
- bin/wadb shell am start -W -a android.intent.action.VIEW -d "<app-scheme>://<path>" -f 0x10000000
77
+ qa/bin/mcli deep-link "<app-scheme>://<a real path>"
78
+ qa/bin/mcli deep-link "<app-scheme>://<path>?id=&n=NaN"
79
+ qa/bin/wadb shell am start -W -a android.intent.action.VIEW -d "<app-scheme>://<path>" -f 0x10000000
80
80
 
81
81
  # WEB — the target URL
82
- $MCLI open-url "<base_url>/<a real deep view>"
83
- $MCLI open-url "<base_url>/<view>?id=&n=NaN"
84
- $MCLI open-url "<base_url>/<view>?next=javascript:alert(1)"
85
- $MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
82
+ qa/bin/mcli open-url "<base_url>/<a real deep view>"
83
+ qa/bin/mcli open-url "<base_url>/<view>?id=&n=NaN"
84
+ qa/bin/mcli open-url "<base_url>/<view>?next=javascript:alert(1)"
85
+ qa/bin/mcli open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
86
86
  ```
87
87
 
88
88
  **Ask:** crash, freeze, blank screen, open-redirect, or unsanitized reflection?
@@ -90,7 +90,7 @@ $MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
90
90
  ## Privacy / leak surface [both]
91
91
 
92
92
  - **Recents preview:** background while on a sensitive screen, screencap Recents — is personal data / a token-in-URL visible? A native app can mask its preview (FLAG_SECURE); web pages generally can't.
93
- - **Process log:** `$MCLI logs --errors` — grep for PII or credentials (emails, tokens, secrets) leaking into the app/browser process log.
93
+ - **Process log:** `qa/bin/mcli logs --errors` — grep for PII or credentials (emails, tokens, secrets) leaking into the app/browser process log.
94
94
  - **URL leakage [web]:** sensitive values in query strings end up in history and Recents — flag where they shouldn't be.
95
95
  - **Clipboard:** if the app copies sensitive text, does it auto-clear/expire?
96
96
 
@@ -103,10 +103,10 @@ $MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
103
103
 
104
104
  ## Native a11y [both] (driven through the device a11y tree)
105
105
 
106
- - TalkBack (Android): `bin/wadb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/.TalkBackService`
106
+ - TalkBack (Android): `qa/bin/wadb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/.TalkBackService`
107
107
  - VoiceOver (iOS sim): mostly limited on sim — manual via the accessibility menu
108
108
  - **Focus order:** swipe right repeatedly — does focus land on every interactive element in a sensible order?
109
- - **Aggregation:** does each control announce its own label, or does the view dump many fields into one comma-joined label? (Known iOS issue — see `phases/03-discovery.md`.)
109
+ - **Aggregation:** does each control announce its own label, or does the view dump many fields into one comma-joined label? (Known iOS issue — see `${CLAUDE_SKILL_DIR}/phases/03-discovery.md`.)
110
110
  - **Contrast:** spot-check critical text — sufficient contrast in both light and dark mode?
111
111
  - **Touch targets:** are tap targets finger-sized, or desktop-sized and hard to hit on a phone?
112
112