prowl-tools 0.1.5 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,12 +1,53 @@
1
1
  # Prowl
2
2
 
3
- CLI-first QA testing tool for deterministic web testing with Playwright.
3
+ End-to-end testing for **native macOS apps and web apps** from the same
4
+ declarative YAML.
4
5
 
5
6
  <!-- ILLUSTRATION: Prowl raccoon mascot hero image — cyan raccoon with terminal window showing pass/fail output -->
6
7
 
7
- Write tests in YAML. Run them from the terminal. Get screenshots, traces, and reports automatically.
8
+ Write a test (a "hunt") in YAML, run it from the terminal, and get screenshots,
9
+ traces, and reports automatically. The same step vocabulary — `click`, `fill`,
10
+ `assert` — drives a macOS app through Apple's Accessibility API and a web app
11
+ through Playwright. One tool, one file format, both targets.
12
+
13
+ **Desktop-first.** The macOS target — menu-bar extras (`NSStatusItem`) included —
14
+ is the gap no other tool fills: [Maestro](https://maestro.mobile.dev) targets
15
+ mobile and web, [Playwright](https://playwright.dev) is web-only, and
16
+ [XCUITest](https://developer.apple.com/documentation/xctest) means Swift and
17
+ Xcode. Web is Prowl's second first-class target; iOS and Android are
18
+ experimental. The macOS target is **experimental today** — install its signed
19
+ helper with `prowl macdriver install` (see
20
+ [macOS Target](#macos-target-experimental)) — but it's where Prowl leads.
21
+
22
+ **Native macOS app** — driven through the Accessibility API:
23
+
24
+ ```yaml
25
+ # .prowl/config.yml → target: { type: macos, app: "com.example.App" }
26
+ # .prowl/hunts/save-note.yml
27
+ name: save-note
28
+ steps:
29
+ - click: "id=newNote"
30
+ - type: "Buy milk"
31
+ - click: "id=saveButton"
32
+ - assert:
33
+ visible: "Saved"
34
+ ```
35
+
36
+ ```text
37
+ ● Running hunt: save-note
38
+ ✓ click "id=newNote" (90ms)
39
+ ✓ type "Buy milk" (40ms)
40
+ ✓ click "id=saveButton" (110ms)
41
+ ✓ assert visible "Saved" (12ms)
42
+
43
+ PASS save-note (252ms) 4/4 steps
44
+ Artifacts: .prowl/runs/2026-02-09_10-30-45
45
+ ```
46
+
47
+ **Web app** — driven through Playwright:
8
48
 
9
49
  ```yaml
50
+ # .prowl/config.yml → target: { type: web, url: "http://localhost:3000" }
10
51
  # .prowl/hunts/login-flow.yml
11
52
  name: login-flow
12
53
  steps:
@@ -20,7 +61,7 @@ steps:
20
61
  visible: "Dashboard"
21
62
  ```
22
63
 
23
- ```
64
+ ```text
24
65
  ● Running hunt: login-flow
25
66
  ✓ navigate "/login" (120ms)
26
67
  ✓ fill "Email" (85ms)
@@ -32,6 +73,23 @@ steps:
32
73
  Artifacts: .prowl/runs/2026-02-09_10-30-45
33
74
  ```
34
75
 
76
+ ## Why Prowl
77
+
78
+ - **Desktop-first, from the same YAML.** Native macOS apps (Accessibility API)
79
+ and web apps (Playwright) share one step vocabulary, one config, one report
80
+ format. No incumbent covers the Mac desktop the way Prowl aims to.
81
+ - **Deterministic.** Explicit scripted steps — no natural-language guessing.
82
+ - **Developer-ready artifacts.** Every run writes screenshots, a Playwright
83
+ trace, a console log, and both human- (`summary.md`) and machine-readable
84
+ (`result.json`) reports.
85
+ - **Self-sovereign & file-based.** Hunts, run history, and baselines live in
86
+ your repo. No database, no account, no service to sign up for.
87
+ - **Guardrails built in.** Allowed domains/apps, forbidden selectors, and step
88
+ caps keep runs scoped and safe.
89
+
90
+ Additional experimental targets — iOS Simulator and Android — follow the same
91
+ hunt format; see their sections below.
92
+
35
93
  ---
36
94
 
37
95
  ## Getting Started
@@ -87,20 +145,23 @@ prowl init
87
145
 
88
146
  <!-- ILLUSTRATION: Terminal screenshot showing `prowl init` output with raccoon mascot and file listing -->
89
147
 
90
- This creates a `.prowl/` directory with a config file and 8 example hunts:
148
+ This creates a `.prowl/` directory with a config file and two starter hunts:
91
149
 
92
150
  ```text
93
151
  .prowl/
94
- ├── config.yml # Target URL, browser settings, guardrails
152
+ ├── config.yml # Target URL, browser settings, guardrails
153
+ ├── .gitignore # Keeps runs/, auth-state.json, and .env out of git
95
154
  └── hunts/
96
- ├── homepage.yml # Basic page load smoke test
97
- ├── login-flow.yml # Email/password authentication
98
- ├── signup-flow.yml # Registration with validation
99
- ├── form-submit.yml # Form fill and submit
100
- ├── form-validation.yml # Validation errors and resubmit
101
- ├── crud-cycle.yml # Create, read, update, delete lifecycle
102
- ├── checkout-flow.yml # E-commerce checkout
103
- └── onboarding-wizard.yml # Multi-step SaaS onboarding
155
+ ├── hello.yml # Minimal smoke test verifies the app loads
156
+ └── login-flow.yml # Fuller example — auth, secrets, assertions
157
+ ```
158
+
159
+ `prowl init` finishes by pointing you at the first hunt:
160
+
161
+ ```text
162
+ Initialized .prowl directory.
163
+ Run prowl run hello to get started.
164
+ See .prowl/hunts/login-flow.yml for a fuller example.
104
165
  ```
105
166
 
106
167
  ### 3. Configure
@@ -112,11 +173,26 @@ target:
112
173
  url: "http://localhost:3000"
113
174
  ```
114
175
 
115
- ### 4. Write Your First Hunt
176
+ ### 4. Run the starter hunt
177
+
178
+ Run the bundled `hello` hunt to confirm your app is reachable:
179
+
180
+ ```bash
181
+ prowl run hello
182
+ ```
183
+
184
+ <!-- ILLUSTRATION: Terminal screenshot showing colorized pass/fail output with step timings -->
185
+
186
+ A hunt's name is its **file name** under `.prowl/hunts/` — `hello` runs
187
+ `.prowl/hunts/hello.yml`.
188
+
189
+ ### 5. Write your own hunt
116
190
 
117
- Edit `.prowl/hunts/homepage.yml` or create a new file:
191
+ Create a new file `.prowl/hunts/smoke-test.yml`. The file name *is* the hunt
192
+ name, so this hunt runs as `smoke-test`:
118
193
 
119
194
  ```yaml
195
+ # .prowl/hunts/smoke-test.yml
120
196
  name: smoke-test
121
197
  steps:
122
198
  - navigate: "/"
@@ -127,18 +203,226 @@ assertions:
127
203
  - noConsoleErrors: true
128
204
  ```
129
205
 
130
- ### 5. Run
206
+ Run it by file name:
131
207
 
132
208
  ```bash
133
209
  prowl run smoke-test
134
210
  ```
135
211
 
136
- <!-- ILLUSTRATION: Terminal screenshot showing colorized pass/fail output with step timings -->
137
-
138
212
  That's it. You're testing.
139
213
 
140
214
  ---
141
215
 
216
+ ## macOS Target (Experimental)
217
+
218
+ > **Experimental (PROWL-048).** Prowl can drive **native macOS apps** — including
219
+ > menu bar extras (`NSStatusItem` + `NSMenu`) — through Apple's Accessibility API,
220
+ > in addition to the web. The API, selector dialect, and step coverage may change.
221
+
222
+ ### Enabling it
223
+
224
+ 1. **Install the helper** (recommended — no Xcode, no Swift toolchain):
225
+
226
+ ```bash
227
+ prowl macdriver install
228
+ ```
229
+
230
+ This downloads the pinned, **signed and notarized** `prowl-macdriver` binary
231
+ from GitHub Releases, verifies its SHA-256 against the released checksum, and
232
+ installs it to `~/.prowl/macdriver/<version>/prowl-macdriver`. Check what's
233
+ resolved at any time with `prowl macdriver status`.
234
+
235
+ > **Until the first signed release is cut, `prowl macdriver install` returns a
236
+ > 404** (the maintainer publishes the first `macdriver-v*` release and
237
+ > verifies the flow before this becomes the default path). In the meantime,
238
+ > build from source as below.
239
+
240
+ **Contributors / pre-release — build from source** (requires the Swift
241
+ toolchain / Xcode CLT):
242
+
243
+ ```bash
244
+ cd macdriver
245
+ swift build -c release
246
+ ```
247
+
248
+ **Binary search order.** Prowl resolves the helper via, in order:
249
+ 1. `$PROWL_MACDRIVER_BIN` (absolute path to a binary), then
250
+ 2. the user-level install at `~/.prowl/macdriver/<version>/prowl-macdriver`
251
+ (what `prowl macdriver install` writes), then
252
+ 3. the repo-local source build at `macdriver/.build/release/prowl-macdriver`
253
+ (then `.../debug/...`).
254
+
255
+ If none is found, Prowl fails with a clear message pointing at
256
+ `prowl macdriver install` (with the source build as the contributor fallback)
257
+ rather than crashing.
258
+
259
+ 2. **Point your config at a macOS target:**
260
+
261
+ ```yaml
262
+ target:
263
+ type: macos
264
+ app: "com.example.App" # bundle id, or an absolute /path/to/App.app
265
+ guardrails:
266
+ allowedApps: # optional scope; empty = allow the target app
267
+ - "com.example.App"
268
+ ```
269
+
270
+ When `target.app` is an app path, `allowedApps` may list the exact `.app`
271
+ path, the app bundle name (`Example` for `Example.app`), or the bundle id
272
+ from `Contents/Info.plist` when that file is readable.
273
+
274
+ 3. **Grant Accessibility permission** (see below), then run a hunt as usual:
275
+ `prowl run my-macos-hunt`.
276
+
277
+ ### Accessibility & Screen Recording permission
278
+
279
+ The **process that hosts** Prowl (your terminal — Terminal, iTerm, VS Code, or a CI
280
+ agent) must be granted **Accessibility** permission: **System Settings → Privacy &
281
+ Security → Accessibility**, then enable that app. macOS attributes the grant to the
282
+ hosting app, not to `prowl-macdriver`. `prowl macdriver status` prints the
283
+ resolved binary path, installed versions, and this permission guidance. Preflight
284
+ the Accessibility grant directly from the helper (use the path `status` reports,
285
+ or the source build):
286
+
287
+ ```bash
288
+ MACDRIVER_VERSION=0.1.0 # replace with the version shown by `prowl macdriver status`
289
+ ~/.prowl/macdriver/$MACDRIVER_VERSION/prowl-macdriver check
290
+ # or: macdriver/.build/release/prowl-macdriver check
291
+ # prints {"trusted": <bool>}; prompts on first run
292
+ ```
293
+
294
+ The `screenshot`/`assertScreenshot` steps additionally need **Screen Recording**
295
+ permission for the hosting app.
296
+
297
+ **CI notes (macOS runners):** headless CI cannot click "Allow" in a dialog, so grant
298
+ the permissions non-interactively before the run. On a self-hosted runner you can
299
+ pre-authorize the agent's host app with a TCC profile via MDM, or (on ephemeral
300
+ runners where it's acceptable) seed the TCC database, e.g.:
301
+
302
+ ```bash
303
+ sudo sqlite3 "/Library/Application Support/com.apple.TCC/TCC.db" \
304
+ "INSERT OR REPLACE INTO access VALUES('kTCCServiceAccessibility','<runner-app-bundle-id>',0,2,2,1,NULL,NULL,NULL,'UNUSED',NULL,0,1,NULL,NULL,NULL);"
305
+ ```
306
+
307
+ GitHub-hosted macOS runners do not grant Accessibility, so the macOS target is aimed
308
+ at self-hosted / MDM-managed runners for now.
309
+
310
+ ### Selector dialect (macOS)
311
+
312
+ See the [Native Selector Dialect matrix](#native-selector-dialect-compatibility-matrix)
313
+ for how these compare across native targets (and the `label=` exact-match trap).
314
+ Native selectors address accessibility identifiers, roles, and labels:
315
+
316
+ | Selector | Matches |
317
+ |---|---|
318
+ | `id=openSettings` | element whose `AXIdentifier` equals `openSettings` |
319
+ | `role=button[name="Save"]` | an `AXButton` whose title/description/value contains `Save` |
320
+ | `label="Email"` | element whose accessibility label equals `Email` |
321
+ | `text="Save"` or bare `Save` | element whose title/description/value contains the text |
322
+ | `statusItem` | opens the app's menu bar status-item menu |
323
+ | `menu=Preferences…` | opens the status-item menu and clicks that item |
324
+
325
+ `forbiddenSelectors` still applies (text patterns match via the same substring
326
+ semantics as the web target). Prefer `id=` (accessibility identifiers) — the native
327
+ analog of `data-testid`.
328
+
329
+ ### Finding selectors
330
+
331
+ Don't guess selectors — dump them. `prowl analyze` works on the macOS target the
332
+ same way it does on the web: it launches/attaches to the app, walks the
333
+ Accessibility tree, and prints every interactive element with **ranked selector
334
+ candidates** (best first) plus the app's windows and status-item menu contents.
335
+ It is read-only (the only interaction is opening and closing the status menu),
336
+ honors `guardrails.allowedApps`, and leaves the app running when done.
337
+
338
+ ```bash
339
+ # Uses the macOS target from .prowl/config.yml:
340
+ prowl analyze
341
+
342
+ # …or point it at any app without a config:
343
+ prowl analyze --app com.example.App
344
+ prowl analyze --app "/Applications/Example.app"
345
+
346
+ # Machine-readable output for agents:
347
+ prowl analyze --app com.example.App --json
348
+ ```
349
+
350
+ Example (human-readable) output:
351
+
352
+ ```text
353
+ App Analysis: com.example.App
354
+
355
+ Windows:
356
+ "Main Window" id=mainWindow
357
+
358
+ Interactive Elements:
359
+ AXButton id=saveButton "Save"
360
+ AXTextField label="Email" "Email"
361
+ AXCheckBox label="Remember me" "Remember me" (disabled)
362
+
363
+ Menu Bar:
364
+ AXMenuItem id=preferences "Preferences…"
365
+ AXMenuItem label="Quit" "Quit"
366
+
367
+ 3 elements, 1 windows, 2 menu items
368
+ ```
369
+
370
+ Selectors are ranked `id=` > `label=` > `role=…[name="…"]` > `text=` — copy the
371
+ first (most durable) candidate into your hunt. Status-item menu identifiers
372
+ (`id=preferences` above) are especially valuable, since menu titles often carry
373
+ ellipses or localized text that are awkward to match by substring.
374
+
375
+ ### Step compatibility
376
+
377
+ Portable steps run on **both** targets; web-only steps are rejected up front on the
378
+ macOS target (with a clear error), and `prowl login` / URL guardrails do not apply.
379
+
380
+ | Portable (web + macOS) | Web-only (rejected on macOS) |
381
+ |---|---|
382
+ | `click`, `fill`, `type`, `press` | `navigate`, `waitForUrl`, `waitForNetworkIdle` |
383
+ | `wait`, `waitForSelector` | `mockRoute` / `unmockRoute` |
384
+ | `assert: visible` / `notVisible` | `evalScript`, `runScript` |
385
+ | `screenshot`, `assertScreenshot` | `onDialog`, `select` / `selectOption` |
386
+ | `hover`, `scrollTo` | `setInputFiles`, `waitForDownload` |
387
+ | `repeat`, `if`, `runHunt`, `copyText` | `scroll` (directional), `assert: urlIncludes` / `urlEquals` |
388
+
389
+ Notes: `press` accepts the same key vocabulary as the web target — single printable
390
+ characters, `Enter`/`Return`/`Space`, `Tab`, `Escape`, `Backspace`, `Delete`, `Home`,
391
+ `End`, `PageUp`, `PageDown`, arrows (`ArrowUp`/`ArrowDown`/`ArrowLeft`/`ArrowRight`),
392
+ `F1`–`F12`, and `+`-joined modifier combos (`Control`, `Shift`, `Alt`, `Meta`,
393
+ `ControlOrMeta`; aliases: `Ctrl`, `Option`, `Cmd`, `Command`; `ControlOrMeta` maps
394
+ to Command on macOS), e.g. `Shift+Tab`, `ControlOrMeta+a`, or `Meta+a`; unknown keys
395
+ error clearly. A bare `Enter`/`Return`/`Space` uses the element's activate action when
396
+ available, otherwise keystrokes are synthesized and posted to the target app (activated
397
+ first) so they never land elsewhere — the existing Accessibility grant already covers
398
+ this, no new permission prompt. `type` fills the focused control; app teardown quits the
399
+ target app after the run.
400
+
401
+ #### Hunt-level assertion compatibility
402
+
403
+ Hunt-level `assertions:` are evaluated after the steps complete (even when a step
404
+ failed, matching the web path). Selector assertions run on every target; URL, console,
405
+ and network assertions are web-only and are reported as **`skipped`** on a native
406
+ target — visible in `result.json` / `summary.md` / JUnit, never silently dropped and
407
+ never a hard error.
408
+
409
+ | Assertion | Web | macOS | Android | iOS |
410
+ |---|---|---|---|---|
411
+ | `selectorExists`, `selectorNotExists` | ✅ runs | ✅ runs | ✅ runs | ✅ runs |
412
+ | `urlIncludes`, `urlEquals` | ✅ runs | ⏭️ skipped (web-only) | ⏭️ skipped | ⏭️ skipped |
413
+ | `noConsoleErrors`, `noNetworkErrors` | ✅ runs | ⏭️ skipped (web-only) | ⏭️ skipped | ⏭️ skipped |
414
+
415
+ A web-only assertion a hunt explicitly authored also prints a console warning naming
416
+ the target; the `noConsoleErrors` / `noNetworkErrors` config defaults are surfaced as
417
+ `skipped` but do not warn on every run. For per-step checks on a native target, use
418
+ inline `assert: visible` / `notVisible` steps.
419
+
420
+ > Docs follow-up: the customer-facing docs site (`prowl-docs`) should gain a "macOS
421
+ > target" page mirroring this section (target type + step-compatibility matrix +
422
+ > assertion-compatibility matrix + permission setup); tracked separately from this repo.
423
+
424
+ ---
425
+
142
426
  ## Step Type Reference
143
427
 
144
428
  Prowl supports both **shorthand** and **explicit** syntax for most step types. Shorthand is concise and readable. Explicit gives you full control over selectors.
@@ -200,6 +484,14 @@ Press a keyboard key on a specific element.
200
484
  key: "Enter"
201
485
  ```
202
486
 
487
+ Key names follow the web (Playwright) vocabulary — e.g. `Enter`, `Escape`, `Tab`,
488
+ `Backspace`, `Delete`, arrows, `F1`–`F12`, single characters, and modifier combos like
489
+ `Control+a`, `ControlOrMeta+a`, or `Shift+Tab`. The macOS target accepts the same names
490
+ (see the macOS target notes above); `ControlOrMeta` maps to Command, and
491
+ `Cmd`/`Command`/`Option` are also accepted there as aliases for `Meta`/`Alt`.
492
+ On macOS, a shortcut letter in a combo (e.g. `Meta+s`) is synthesized against the US/ANSI
493
+ physical keyboard layout — the standard trade-off for synthesized keystrokes.
494
+
203
495
  ### select / selectOption
204
496
 
205
497
  Select a dropdown value. Shorthand finds by label, explicit uses a selector.
@@ -438,6 +730,10 @@ assertions:
438
730
  - noNetworkErrors: true # No HTTP responses >= 400
439
731
  ```
440
732
 
733
+ On native targets (macOS / Android / iOS) the selector assertions run and the
734
+ URL/console/network ones are reported as `skipped` — see the
735
+ [hunt-level assertion compatibility matrix](#hunt-level-assertion-compatibility).
736
+
441
737
  ---
442
738
 
443
739
  ## Config Reference
@@ -699,9 +995,17 @@ spans. When the app emits no trace headers, nothing is recorded (no noise).
699
995
 
700
996
  ## CLI Reference
701
997
 
998
+ A `<hunt-name>` is a hunt's file name under `.prowl/hunts/` — `homepage` for
999
+ `.prowl/hunts/homepage.yml`, or `admin/users` for a nested
1000
+ `.prowl/hunts/admin/users.yml`. `run`, `watch`, and `history` also accept
1001
+ supported `.yml` path forms (`.prowl/hunts/homepage.yml` or
1002
+ `hunts/homepage.yml`) and a bare `.yml` file name (`homepage.yml`); they resolve
1003
+ to the same hunt.
1004
+
702
1005
  ```bash
703
1006
  # Run a hunt
704
1007
  prowl run <hunt-name>
1008
+ prowl run .prowl/hunts/homepage.yml # A literal path resolves to `homepage`
705
1009
  prowl run <hunt-name> --headed # Show browser window
706
1010
  prowl run <hunt-name> --trace # Capture Playwright trace
707
1011
  prowl run <hunt-name> --slow-mo 500 # Slow down actions (ms)
@@ -927,164 +1231,40 @@ CLI Commands
927
1231
 
928
1232
  ---
929
1233
 
930
- ## Community Hub
931
-
932
- Browse and contribute hunt templates through the internal community registry (contact ops for access).
933
-
934
- Templates cover auth flows (OAuth, 2FA), e-commerce (Stripe), admin panels, SaaS patterns, and more. Each template is heavily commented and ready to customize.
935
-
936
- ---
937
-
938
- ## macOS Target (Experimental)
939
-
940
- > **Experimental (PROWL-048).** Prowl can drive **native macOS apps** including
941
- > menu bar extras (`NSStatusItem` + `NSMenu`) — through Apple's Accessibility API,
942
- > in addition to the web. The API, selector dialect, and step coverage may change.
943
- > **Distribution is deferred:** the required helper binary is **not** shipped in the
944
- > npm package; you build it locally (below).
945
-
946
- ### Enabling it
947
-
948
- 1. **Build the helper** (one time; requires the Swift toolchain / Xcode CLT):
949
-
950
- ```bash
951
- cd macdriver
952
- swift build -c release
953
- ```
954
-
955
- Prowl finds the binary at `macdriver/.build/release/prowl-macdriver`, or at
956
- `$PROWL_MACDRIVER_BIN` if set. If it is missing, Prowl fails with a clear
957
- "build the helper" message rather than crashing.
958
-
959
- 2. **Point your config at a macOS target:**
960
-
961
- ```yaml
962
- target:
963
- type: macos
964
- app: "com.example.App" # bundle id, or an absolute /path/to/App.app
965
- guardrails:
966
- allowedApps: # optional scope; empty = allow the target app
967
- - "com.example.App"
968
- ```
969
-
970
- When `target.app` is an app path, `allowedApps` may list the exact `.app`
971
- path, the app bundle name (`Example` for `Example.app`), or the bundle id
972
- from `Contents/Info.plist` when that file is readable.
973
-
974
- 3. **Grant Accessibility permission** (see below), then run a hunt as usual:
975
- `prowl run my-macos-hunt`.
976
-
977
- ### Accessibility & Screen Recording permission
978
-
979
- The **process that hosts** Prowl (your terminal — Terminal, iTerm, VS Code, or a CI
980
- agent) must be granted **Accessibility** permission: **System Settings → Privacy &
981
- Security → Accessibility**, then enable that app. macOS attributes the grant to the
982
- hosting app, not to `prowl-macdriver`. Preflight from the helper:
983
-
984
- ```bash
985
- macdriver/.build/release/prowl-macdriver check # prints {"trusted": <bool>}; prompts on first run
986
- ```
987
-
988
- The `screenshot`/`assertScreenshot` steps additionally need **Screen Recording**
989
- permission for the hosting app.
990
-
991
- **CI notes (macOS runners):** headless CI cannot click "Allow" in a dialog, so grant
992
- the permissions non-interactively before the run. On a self-hosted runner you can
993
- pre-authorize the agent's host app with a TCC profile via MDM, or (on ephemeral
994
- runners where it's acceptable) seed the TCC database, e.g.:
995
-
996
- ```bash
997
- sudo sqlite3 "/Library/Application Support/com.apple.TCC/TCC.db" \
998
- "INSERT OR REPLACE INTO access VALUES('kTCCServiceAccessibility','<runner-app-bundle-id>',0,2,2,1,NULL,NULL,NULL,'UNUSED',NULL,0,1,NULL,NULL,NULL);"
999
- ```
1000
-
1001
- GitHub-hosted macOS runners do not grant Accessibility, so the macOS target is aimed
1002
- at self-hosted / MDM-managed runners for now.
1003
-
1004
- ### Selector dialect (macOS)
1005
-
1006
- Native selectors address accessibility identifiers, roles, and labels:
1007
-
1008
- | Selector | Matches |
1009
- |---|---|
1010
- | `id=openSettings` | element whose `AXIdentifier` equals `openSettings` |
1011
- | `role=button[name="Save"]` | an `AXButton` whose title/description/value contains `Save` |
1012
- | `label="Email"` | element whose accessibility label equals `Email` |
1013
- | `text="Save"` or bare `Save` | element whose title/description/value contains the text |
1014
- | `statusItem` | opens the app's menu bar status-item menu |
1015
- | `menu=Preferences…` | opens the status-item menu and clicks that item |
1016
-
1017
- `forbiddenSelectors` still applies (text patterns match via the same substring
1018
- semantics as the web target). Prefer `id=` (accessibility identifiers) — the native
1019
- analog of `data-testid`.
1020
-
1021
- ### Finding selectors
1022
-
1023
- Don't guess selectors — dump them. `prowl analyze` works on the macOS target the
1024
- same way it does on the web: it launches/attaches to the app, walks the
1025
- Accessibility tree, and prints every interactive element with **ranked selector
1026
- candidates** (best first) plus the app's windows and status-item menu contents.
1027
- It is read-only (the only interaction is opening and closing the status menu),
1028
- honors `guardrails.allowedApps`, and leaves the app running when done.
1029
-
1030
- ```bash
1031
- # Uses the macOS target from .prowl/config.yml:
1032
- prowl analyze
1033
-
1034
- # …or point it at any app without a config:
1035
- prowl analyze --app com.example.App
1036
- prowl analyze --app "/Applications/Example.app"
1037
-
1038
- # Machine-readable output for agents:
1039
- prowl analyze --app com.example.App --json
1040
- ```
1041
-
1042
- Example (human-readable) output:
1043
-
1044
- ```text
1045
- App Analysis: com.example.App
1046
-
1047
- Windows:
1048
- "Main Window" id=mainWindow
1049
-
1050
- Interactive Elements:
1051
- AXButton id=saveButton "Save"
1052
- AXTextField label="Email" "Email"
1053
- AXCheckBox label="Remember me" "Remember me" (disabled)
1054
-
1055
- Menu Bar:
1056
- AXMenuItem id=preferences "Preferences…"
1057
- AXMenuItem label="Quit" "Quit"
1058
-
1059
- 3 elements, 1 windows, 2 menu items
1060
- ```
1061
-
1062
- Selectors are ranked `id=` > `label=` > `role=…[name="…"]` > `text=` — copy the
1063
- first (most durable) candidate into your hunt. Status-item menu identifiers
1064
- (`id=preferences` above) are especially valuable, since menu titles often carry
1065
- ellipses or localized text that are awkward to match by substring.
1066
-
1067
- ### Step compatibility
1068
-
1069
- Portable steps run on **both** targets; web-only steps are rejected up front on the
1070
- macOS target (with a clear error), and `prowl login` / URL guardrails do not apply.
1071
-
1072
- | Portable (web + macOS) | Web-only (rejected on macOS) |
1073
- |---|---|
1074
- | `click`, `fill`, `type`, `press` | `navigate`, `waitForUrl`, `waitForNetworkIdle` |
1075
- | `wait`, `waitForSelector` | `mockRoute` / `unmockRoute` |
1076
- | `assert: visible` / `notVisible` | `evalScript`, `runScript` |
1077
- | `screenshot`, `assertScreenshot` | `onDialog`, `select` / `selectOption` |
1078
- | `hover`, `scrollTo` | `setInputFiles`, `waitForDownload` |
1079
- | `repeat`, `if`, `runHunt`, `copyText` | `scroll` (directional), `assert: urlIncludes` / `urlEquals` |
1080
-
1081
- Notes: `press` maps Enter/Return/Space onto the element's activate action (other keys
1082
- are unsupported); `type` fills the focused control; app teardown quits the target app
1083
- after the run.
1084
-
1085
- > Docs follow-up: the customer-facing docs site (`prowl-docs`) should gain a "macOS
1086
- > target" page mirroring this section (target type + step-compatibility matrix +
1087
- > permission setup); tracked separately from this repo.
1234
+ ## Native Selector Dialect (compatibility matrix)
1235
+
1236
+ Android and iOS now consume one shared selector dialect implementation, so `id=`
1237
+ / `label=` / `text=` / `role=` mean the same *shape* of thing on both mobile
1238
+ targets. That shared grammar, per-platform attribute mapping, ranking order, and
1239
+ host-side matching live in `src/selector/native.ts`. macOS remains on its existing
1240
+ driver/analyzer implementation for now, with migration deferred, but follows the
1241
+ same documented selector shape. The web target speaks Playwright's own selector
1242
+ engines and is shown for contrast.
1243
+
1244
+ | Kind | Web (Playwright) | macOS (AX) | Android (uiautomator2) | iOS (WebDriverAgent) |
1245
+ |---|---|---|---|---|
1246
+ | `id=` | use CSS `#id` / `[data-testid]` | `AXIdentifier`, exact | `resource-id`, exact (bare names are package-qualified: `save` → `<pkg>:id/save`) | accessibility id (the `name` attribute), exact |
1247
+ | `label=` | *(no native kind; analyzer surfaces the associated `<label>` text)* | `title`/`description`, **exact** | `content-desc`, **exact** | `accessibilityLabel`, **exact** |
1248
+ | `text=` (or bare) | text engine, substring, case-insensitive | `title`/`description`/`value`, substring | visible `text`, substring | `label` **or** `value`, substring |
1249
+ | `role=` | ARIA role engine | AX role (e.g. `AXButton`) | widget class (e.g. `android.widget.Button`) | element type (`XCUIElementType…`; shorthand `Button` accepted) |
1250
+ | `role=X[name="Y"]` | role + accessible name (substring) | role + name (substring) | class + visible-text (substring) | type + (`label` or `value`) substring |
1251
+ | `:focus` | *(n/a)* | focused element | `UiSelector().focused(true)` | `hasKeyboardFocus == 1` |
1252
+
1253
+ **The `label=`-in-assertions trap.** On every native target `label=` is an **exact**
1254
+ match on the accessibility label — unlike `text=`, which is a substring match, and
1255
+ unlike the web, where text matching is forgiving. So `assert: selectorExists:
1256
+ label="Save"` will **not** match an element whose real label is "Save changes"; it
1257
+ silently fails rather than partially matching. Use `text=` when you want substring
1258
+ behavior in an assertion, and keep `label=` for the exact accessibility label. On
1259
+ iOS there is a second trap: WDA's page source exposes a single `name` attribute that
1260
+ is the `accessibilityIdentifier` when one is set and otherwise the label. Prowl's
1261
+ analyzer only recommends `id=` when `name` differs from `label`, so it does not emit
1262
+ label-shaped ids, but runtime and host-side matching still resolve `id=` against
1263
+ WDA's `name`.
1264
+
1265
+ Prefer `id=` on every native target — the native analog of `data-testid`. Per-target
1266
+ specifics (escaping, `statusItem`/`menu=` on macOS above, the Compose
1267
+ `testTagsAsResourceId` caveat on Android below) live in each target's own section.
1088
1268
 
1089
1269
  ---
1090
1270
 
@@ -1139,8 +1319,10 @@ validates it before installing.
1139
1319
  ### Selector dialect (Android)
1140
1320
 
1141
1321
  Native selectors address `resource-id`, `content-desc`, visible text, and widget
1142
- class. Semantics match the macOS target so a selector means the same thing on both
1143
- native targets:
1322
+ class. Semantics match the macOS and iOS targets so a selector means the same thing
1323
+ across native targets — see the
1324
+ [Native Selector Dialect matrix](#native-selector-dialect-compatibility-matrix)
1325
+ (and the `label=` exact-match trap):
1144
1326
 
1145
1327
  | Selector | Matches |
1146
1328
  |---|---|
@@ -1178,9 +1360,57 @@ equivalent yet and are rejected with a clear message; scroll-gesture support is
1178
1360
  follow-up. A degraded pure-`adb` fallback (`uiautomator dump` + `input tap`) is a
1179
1361
  possible future diagnostic mode, not the primary path.
1180
1362
 
1181
- > Out of scope for PROWL-058 (tracked separately): `prowl analyze` for Android and
1182
- > CI recipes (PROWL-061), the unified native selector engine (PROWL-060), and real
1183
- > iOS devices (PROWL-062). The iOS **simulator** target ships below.
1363
+ Hunt-level `assertions:` behave as on macOS `selectorExists` / `selectorNotExists`
1364
+ run against the device; `urlIncludes` / `urlEquals` / `noConsoleErrors` /
1365
+ `noNetworkErrors` are web-only and reported as `skipped`. See the
1366
+ [hunt-level assertion compatibility matrix](#hunt-level-assertion-compatibility).
1367
+
1368
+ ### Finding selectors (Android)
1369
+
1370
+ Don't guess selectors — dump them. `prowl analyze` works on the Android target the
1371
+ same way it does on the web and macOS: it attaches to the running app, reads the
1372
+ uiautomator UI hierarchy, and prints every interactive element with **ranked
1373
+ selector candidates** (best first). It is read-only, honors
1374
+ `guardrails.allowedApps`, and leaves the app running when done. Point it at a
1375
+ booted emulator/device:
1376
+
1377
+ ```bash
1378
+ # Uses the Android target from .prowl/config.yml:
1379
+ prowl analyze
1380
+
1381
+ # …or force the Android target explicitly:
1382
+ prowl analyze --app com.android.settings --platform android
1383
+ prowl analyze --app ./app-debug.apk # an .apk implies Android
1384
+ prowl analyze --app com.example.app --device emulator-5556 # pick a device
1385
+
1386
+ # Machine-readable output for agents:
1387
+ prowl analyze --app com.android.settings --platform android --json
1388
+ ```
1389
+
1390
+ Ranking (best → last resort): `id=` (the package-qualified `resource-id`, the
1391
+ native `data-testid`) > `label=` (content-desc) > `role=<class>[name="<text>"]` >
1392
+ `text=`. Example (human-readable) output:
1393
+
1394
+ ```text
1395
+ App Analysis: com.android.settings
1396
+
1397
+ Interactive Elements:
1398
+ android.widget.EditText id=com.android.settings:id/search_src_text "Search settings"
1399
+ android.widget.LinearLayout text="Network & internet" "Network & internet"
1400
+ android.widget.Switch id=com.android.settings:id/switch_widget (disabled)
1401
+
1402
+ 3 elements
1403
+ ```
1404
+
1405
+ > Platform selection for `--app`: an `.apk` implies Android; otherwise pass
1406
+ > `--platform android` (a bare bundle-id / package is ambiguous with the macOS and
1407
+ > iOS targets, which default to macOS unless a config `target.type` or `--platform`
1408
+ > says otherwise). With an Android `target.type` in `.prowl/config.yml`, a bare
1409
+ > `prowl analyze` needs no flag.
1410
+ >
1411
+ > Out of scope for PROWL-058 (tracked separately): the unified native selector
1412
+ > engine (PROWL-060) and real iOS devices (PROWL-062). `prowl analyze` for Android
1413
+ > and the CI recipes shipped in PROWL-061 (above, and see "Mobile targets in CI").
1184
1414
 
1185
1415
  ---
1186
1416
 
@@ -1242,7 +1472,9 @@ treated as a **bundle id** unless a directory of that name exists, so bundle ids
1242
1472
 
1243
1473
  Native selectors address accessibility ids, labels, visible text, and element type.
1244
1474
  Semantics match the macOS/Android targets so a selector means the same thing across
1245
- native targets:
1475
+ native targets — see the
1476
+ [Native Selector Dialect matrix](#native-selector-dialect-compatibility-matrix)
1477
+ (and the `label=` exact-match trap):
1246
1478
 
1247
1479
  | Selector | Matches |
1248
1480
  |---|---|
@@ -1278,9 +1510,170 @@ message. Screenshots are captured with `simctl` (not WDA), so artifacts still wo
1278
1510
  even if the agent wedges. `hover` and `scrollTo` have no touch equivalent yet and are
1279
1511
  rejected with a clear message; scroll-gesture support is a follow-up.
1280
1512
 
1281
- > Out of scope for PROWL-059 (tracked separately): `prowl analyze` for iOS and CI
1282
- > recipes (PROWL-061), the unified native selector engine (PROWL-060), and real iOS
1283
- > devices (PROWL-062).
1513
+ Hunt-level `assertions:` behave as on macOS `selectorExists` / `selectorNotExists`
1514
+ run against the simulator; `urlIncludes` / `urlEquals` / `noConsoleErrors` /
1515
+ `noNetworkErrors` are web-only and reported as `skipped`. See the
1516
+ [hunt-level assertion compatibility matrix](#hunt-level-assertion-compatibility).
1517
+
1518
+ ### Finding selectors (iOS)
1519
+
1520
+ Don't guess selectors — dump them. `prowl analyze` works on the iOS target the
1521
+ same way it does on the web, macOS, and Android: it attaches to the running app on
1522
+ a booted simulator, reads WebDriverAgent's UI hierarchy, and prints every
1523
+ interactive element (plus the app's windows) with **ranked selector candidates**
1524
+ (best first). It is read-only, honors `guardrails.allowedApps`, and leaves the app
1525
+ running when done:
1526
+
1527
+ ```bash
1528
+ # Uses the iOS target from .prowl/config.yml:
1529
+ prowl analyze
1530
+
1531
+ # …or force the iOS target explicitly:
1532
+ prowl analyze --app com.apple.Preferences --platform ios
1533
+ prowl analyze --app com.example.App --platform ios --udid <SIM-UDID>
1534
+
1535
+ # Machine-readable output for agents:
1536
+ prowl analyze --app com.apple.Preferences --platform ios --json
1537
+ ```
1538
+
1539
+ Ranking (best → last resort): `id=` (accessibility id) > `label=` > `role=<Type>
1540
+ [name="<text>"]` > `text=`. Because WDA's page source exposes only a single `name`
1541
+ attribute — the accessibility identifier when set, otherwise the label — `id=` is
1542
+ offered only when that `name` differs from the element's label. Example output:
1543
+
1544
+ ```text
1545
+ App Analysis: com.apple.Preferences
1546
+
1547
+ Windows:
1548
+ (untitled) role=Window
1549
+
1550
+ Interactive Elements:
1551
+ XCUIElementTypeButton id=general_button "General"
1552
+ XCUIElementTypeCell label="Wi-Fi" "Wi-Fi"
1553
+ XCUIElementTypeSwitch label="Airplane Mode" "Airplane Mode" (disabled)
1554
+
1555
+ 3 elements, 1 windows
1556
+ ```
1557
+
1558
+ > A bare bundle-id `--app` is ambiguous with the macOS target (which is the
1559
+ > default), so pass `--platform ios`. With an iOS `target.type` in
1560
+ > `.prowl/config.yml`, a bare `prowl analyze` needs no flag.
1561
+ >
1562
+ > Out of scope for PROWL-059 (tracked separately): the unified native selector
1563
+ > engine (PROWL-060) and real iOS devices (PROWL-062). `prowl analyze` for iOS and
1564
+ > the CI recipes shipped in PROWL-061 (above, and see "Mobile targets in CI").
1565
+
1566
+ ---
1567
+
1568
+ ## Mobile targets in CI
1569
+
1570
+ The Android and iOS targets run in continuous integration, either on GitHub-hosted
1571
+ runners or on a self-hosted Mac. Every recipe runs the real `prowl` CLI against a
1572
+ booted emulator/simulator and uploads run artifacts (screenshots, JUnit, reports).
1573
+
1574
+ ### Android on `ubuntu-latest` (GitHub-hosted)
1575
+
1576
+ GitHub's Linux runners support KVM, so a hardware-accelerated emulator boots in the
1577
+ job via [`reactivecircus/android-emulator-runner`](https://github.com/ReactiveCircus/android-emulator-runner).
1578
+ Prowl installs the uiautomator2 agent APKs from its optional dependency automatically.
1579
+
1580
+ ```yaml
1581
+ name: Android E2E
1582
+ on: [push, pull_request]
1583
+ jobs:
1584
+ android:
1585
+ runs-on: ubuntu-latest
1586
+ steps:
1587
+ - uses: actions/checkout@v4
1588
+ - uses: actions/setup-node@v4
1589
+ with:
1590
+ node-version: 20
1591
+ - run: npm ci
1592
+ - run: npm run build
1593
+
1594
+ # KVM must be accessible for a fast emulator.
1595
+ - name: Enable KVM
1596
+ run: |
1597
+ echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
1598
+ | sudo tee /etc/udev/rules.d/99-kvm4all.rules
1599
+ sudo udevadm control --reload-rules
1600
+ sudo udevadm trigger --name-match=kvm
1601
+
1602
+ - name: Run hunts against the emulator
1603
+ uses: reactivecircus/android-emulator-runner@v2
1604
+ with:
1605
+ api-level: 34
1606
+ arch: x86_64
1607
+ force-avd-creation: false
1608
+ emulator-options: -no-window -no-audio -no-boot-anim -no-snapshot -gpu swiftshader_indirect
1609
+ disable-animations: true
1610
+ # `prowl` is your installed CLI (e.g. `npx prowl` or a global install);
1611
+ # the emulator is booted and on adb by the time this runs.
1612
+ script: npx prowl ci --junit
1613
+
1614
+ - name: Upload artifacts
1615
+ if: always()
1616
+ uses: actions/upload-artifact@v4
1617
+ with:
1618
+ name: android-artifacts
1619
+ path: .prowl/runs/**
1620
+ if-no-files-found: ignore
1621
+ ```
1622
+
1623
+ ### iOS simulators on `macos-*` (GitHub-hosted)
1624
+
1625
+ macOS runners ship Xcode and the iOS simulator runtimes. Boot a simulator with
1626
+ `xcrun simctl`, and cache the one-time WebDriverAgent build (`~/.prowl/wda/`, keyed
1627
+ on the WDA + Xcode versions) so subsequent runs skip the ~2-minute `xcodebuild`.
1628
+
1629
+ ```yaml
1630
+ name: iOS E2E
1631
+ on: [push, pull_request]
1632
+ jobs:
1633
+ ios:
1634
+ runs-on: macos-15
1635
+ steps:
1636
+ - uses: actions/checkout@v4
1637
+ - uses: actions/setup-node@v4
1638
+ with:
1639
+ node-version: 20
1640
+ - run: npm ci
1641
+ - run: npm run build
1642
+
1643
+ # Cache the built WebDriverAgent runner across runs.
1644
+ - name: Cache WebDriverAgent
1645
+ uses: actions/cache@v4
1646
+ with:
1647
+ path: ~/.prowl/wda
1648
+ key: prowl-wda-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
1649
+
1650
+ - name: Boot a simulator
1651
+ run: |
1652
+ xcrun simctl boot "iPhone 16" || true
1653
+ xcrun simctl bootstatus "iPhone 16"
1654
+
1655
+ - name: Run hunts against the simulator
1656
+ run: npx prowl ci --junit
1657
+
1658
+ - name: Upload artifacts
1659
+ if: always()
1660
+ uses: actions/upload-artifact@v4
1661
+ with:
1662
+ name: ios-artifacts
1663
+ path: .prowl/runs/**
1664
+ if-no-files-found: ignore
1665
+ ```
1666
+
1667
+ ### Self-hosted device-verification gate
1668
+
1669
+ This repo also ships `.github/workflows/mobile-e2e.yml`, a real end-to-end gate on
1670
+ the Prowl Tools self-hosted Mac (labels `self-hosted, macOS, prowl-mobile`) that
1671
+ boots both a headless emulator and a simulator and drives Settings on each through
1672
+ the real CLI. It runs on `workflow_dispatch` (the owner's post-merge verification)
1673
+ and on same-repo pull requests, skipping cleanly (green) for forks/outside PRs that
1674
+ can't reach the runner, and uses its own `concurrency` group so it never collides
1675
+ with other jobs on the shared box. It is the machine-run version of the manual
1676
+ smoke tests that caught the uiautomator2 wire-shape bug and the WDA readiness hang.
1284
1677
 
1285
1678
  ---
1286
1679