prowl-tools 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +190 -0
- package/NOTICE +63 -0
- package/README.md +945 -0
- package/dist/chunk-NXXGJOBG.js +546 -0
- package/dist/chunk-NXXGJOBG.js.map +1 -0
- package/dist/chunk-T7YLXF6X.js +3158 -0
- package/dist/chunk-T7YLXF6X.js.map +1 -0
- package/dist/index.cjs +5143 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.js +1366 -0
- package/dist/index.js.map +1 -0
- package/dist/lib.cjs +3765 -0
- package/dist/lib.cjs.map +1 -0
- package/dist/lib.d.cts +935 -0
- package/dist/lib.d.ts +935 -0
- package/dist/lib.js +55 -0
- package/dist/lib.js.map +1 -0
- package/dist/loader-5RDNTJHH.js +27 -0
- package/dist/loader-5RDNTJHH.js.map +1 -0
- package/dist/visual-FSARM2JS.js +46 -0
- package/dist/visual-FSARM2JS.js.map +1 -0
- package/examples/config.yml +41 -0
- package/examples/hunts/hello.yml +20 -0
- package/package.json +83 -0
package/README.md
ADDED
|
@@ -0,0 +1,945 @@
|
|
|
1
|
+
# Prowl
|
|
2
|
+
|
|
3
|
+
CLI-first QA testing tool for deterministic web testing with Playwright.
|
|
4
|
+
|
|
5
|
+
<!-- ILLUSTRATION: Prowl raccoon mascot hero image — cyan raccoon with terminal window showing pass/fail output -->
|
|
6
|
+
|
|
7
|
+
Write tests in YAML. Run them from the terminal. Get screenshots, traces, and reports automatically.
|
|
8
|
+
|
|
9
|
+
```yaml
|
|
10
|
+
# .prowl/hunts/login-flow.yml
|
|
11
|
+
name: login-flow
|
|
12
|
+
steps:
|
|
13
|
+
- navigate: "/login"
|
|
14
|
+
- fill:
|
|
15
|
+
"Email": "{{TEST_EMAIL}}"
|
|
16
|
+
- fill:
|
|
17
|
+
"Password": "{{TEST_PASSWORD}}"
|
|
18
|
+
- click: "Sign In"
|
|
19
|
+
- assert:
|
|
20
|
+
visible: "Dashboard"
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
● Running hunt: login-flow
|
|
25
|
+
✓ navigate "/login" (120ms)
|
|
26
|
+
✓ fill "Email" (85ms)
|
|
27
|
+
✓ fill "Password" (62ms)
|
|
28
|
+
✓ click "Sign In" (340ms)
|
|
29
|
+
✓ assert visible "Dashboard" (15ms)
|
|
30
|
+
|
|
31
|
+
PASS login-flow (622ms) 5/5 steps
|
|
32
|
+
Artifacts: .prowl/runs/2026-02-09_10-30-45
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Getting Started
|
|
38
|
+
|
|
39
|
+
### 1. Install
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
npm install -g prowl-tools
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Or with Homebrew:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
brew tap prowl-tools/tap
|
|
49
|
+
brew install prowl
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Prowl uses Playwright under the hood. Install the browser:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
npx playwright install chromium
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### 2. Initialize
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
cd your-project
|
|
62
|
+
prowl init
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
<!-- ILLUSTRATION: Terminal screenshot showing `prowl init` output with raccoon mascot and file listing -->
|
|
66
|
+
|
|
67
|
+
This creates a `.prowl/` directory with a config file and 8 example hunts:
|
|
68
|
+
|
|
69
|
+
```text
|
|
70
|
+
.prowl/
|
|
71
|
+
├── config.yml # Target URL, browser settings, guardrails
|
|
72
|
+
└── hunts/
|
|
73
|
+
├── homepage.yml # Basic page load smoke test
|
|
74
|
+
├── login-flow.yml # Email/password authentication
|
|
75
|
+
├── signup-flow.yml # Registration with validation
|
|
76
|
+
├── form-submit.yml # Form fill and submit
|
|
77
|
+
├── form-validation.yml # Validation errors and resubmit
|
|
78
|
+
├── crud-cycle.yml # Create, read, update, delete lifecycle
|
|
79
|
+
├── checkout-flow.yml # E-commerce checkout
|
|
80
|
+
└── onboarding-wizard.yml # Multi-step SaaS onboarding
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### 3. Configure
|
|
84
|
+
|
|
85
|
+
Edit `.prowl/config.yml` to point at your app:
|
|
86
|
+
|
|
87
|
+
```yaml
|
|
88
|
+
target:
|
|
89
|
+
url: "http://localhost:3000"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### 4. Write Your First Hunt
|
|
93
|
+
|
|
94
|
+
Edit `.prowl/hunts/homepage.yml` or create a new file:
|
|
95
|
+
|
|
96
|
+
```yaml
|
|
97
|
+
name: smoke-test
|
|
98
|
+
steps:
|
|
99
|
+
- navigate: "/"
|
|
100
|
+
- wait: "Welcome"
|
|
101
|
+
- assert:
|
|
102
|
+
visible: "Sign In"
|
|
103
|
+
assertions:
|
|
104
|
+
- noConsoleErrors: true
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### 5. Run
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
prowl run smoke-test
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
<!-- ILLUSTRATION: Terminal screenshot showing colorized pass/fail output with step timings -->
|
|
114
|
+
|
|
115
|
+
That's it. You're testing.
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Step Type Reference
|
|
120
|
+
|
|
121
|
+
Prowl supports both **shorthand** and **explicit** syntax for most step types. Shorthand is concise and readable. Explicit gives you full control over selectors.
|
|
122
|
+
|
|
123
|
+
### navigate
|
|
124
|
+
|
|
125
|
+
Navigate to a URL (relative to your `target.url` or absolute).
|
|
126
|
+
|
|
127
|
+
```yaml
|
|
128
|
+
- navigate: "/"
|
|
129
|
+
- navigate: "/login"
|
|
130
|
+
- navigate: "https://example.com/page"
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
### click
|
|
134
|
+
|
|
135
|
+
Click an element. Shorthand finds buttons by text, then falls back to any matching text.
|
|
136
|
+
|
|
137
|
+
```yaml
|
|
138
|
+
# Shorthand — finds by button role, then text
|
|
139
|
+
- click: "Sign In"
|
|
140
|
+
|
|
141
|
+
# Explicit — use any Playwright selector
|
|
142
|
+
- click:
|
|
143
|
+
selector: "[data-testid='submit-btn']"
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
### fill
|
|
147
|
+
|
|
148
|
+
Fill an input field. Shorthand finds inputs by label or placeholder text.
|
|
149
|
+
|
|
150
|
+
```yaml
|
|
151
|
+
# Shorthand — finds by label, then placeholder
|
|
152
|
+
- fill:
|
|
153
|
+
"Email": "user@example.com"
|
|
154
|
+
|
|
155
|
+
# Explicit — use any selector
|
|
156
|
+
- fill:
|
|
157
|
+
selector: "input[name='email']"
|
|
158
|
+
value: "user@example.com"
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
### type
|
|
162
|
+
|
|
163
|
+
Type into the currently focused element. Useful after clicking into a field.
|
|
164
|
+
|
|
165
|
+
```yaml
|
|
166
|
+
- click: "Message"
|
|
167
|
+
- type: "Hello, I have a question."
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### press
|
|
171
|
+
|
|
172
|
+
Press a keyboard key on a specific element.
|
|
173
|
+
|
|
174
|
+
```yaml
|
|
175
|
+
- press:
|
|
176
|
+
selector: "input[name='search']"
|
|
177
|
+
key: "Enter"
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
### select / selectOption
|
|
181
|
+
|
|
182
|
+
Select a dropdown value. Shorthand finds by label, explicit uses a selector.
|
|
183
|
+
|
|
184
|
+
```yaml
|
|
185
|
+
# Shorthand — finds <select> by label, aria-label, or placeholder
|
|
186
|
+
- select:
|
|
187
|
+
"State": "FL"
|
|
188
|
+
|
|
189
|
+
# Explicit
|
|
190
|
+
- selectOption:
|
|
191
|
+
selector: "select[name='state']"
|
|
192
|
+
value: "FL"
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
### assert
|
|
196
|
+
|
|
197
|
+
Mid-flow assertions. Fails the hunt immediately if the assertion fails.
|
|
198
|
+
|
|
199
|
+
```yaml
|
|
200
|
+
- assert:
|
|
201
|
+
visible: "Welcome back"
|
|
202
|
+
|
|
203
|
+
- assert:
|
|
204
|
+
notVisible: "Error"
|
|
205
|
+
|
|
206
|
+
- assert:
|
|
207
|
+
urlIncludes: "/dashboard"
|
|
208
|
+
|
|
209
|
+
- assert:
|
|
210
|
+
urlEquals: "https://example.com/dashboard"
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
### if
|
|
214
|
+
|
|
215
|
+
Conditionally execute steps based on whether a selector is visible or not visible. This step is explicit-only.
|
|
216
|
+
|
|
217
|
+
Key fields:
|
|
218
|
+
- `visible` or `notVisible` (exactly one)
|
|
219
|
+
- `then` (required array of steps)
|
|
220
|
+
- `else` (optional array of steps)
|
|
221
|
+
|
|
222
|
+
```yaml
|
|
223
|
+
- if:
|
|
224
|
+
visible: ".cookie-banner"
|
|
225
|
+
then:
|
|
226
|
+
- click: ".accept"
|
|
227
|
+
else:
|
|
228
|
+
- wait: "Welcome back"
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
### repeat
|
|
232
|
+
|
|
233
|
+
Repeat a block of steps either a fixed number of times or while a selector condition is true. This step is explicit-only.
|
|
234
|
+
|
|
235
|
+
Key fields:
|
|
236
|
+
- `times` (fixed count) or `while` (condition), exactly one
|
|
237
|
+
- `while.visible` or `while.notVisible` (when using `while`)
|
|
238
|
+
- `maxIterations` (required with `while`)
|
|
239
|
+
- `steps` (required array of steps to execute each iteration)
|
|
240
|
+
|
|
241
|
+
```yaml
|
|
242
|
+
# Fixed count
|
|
243
|
+
- repeat:
|
|
244
|
+
times: 3
|
|
245
|
+
steps:
|
|
246
|
+
- click: ".load-more"
|
|
247
|
+
|
|
248
|
+
# Condition-based loop
|
|
249
|
+
- repeat:
|
|
250
|
+
while:
|
|
251
|
+
visible: ".load-more"
|
|
252
|
+
maxIterations: 10
|
|
253
|
+
steps:
|
|
254
|
+
- click: ".load-more"
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
### mockRoute
|
|
258
|
+
|
|
259
|
+
Mock network responses for a URL pattern. This step is explicit-only.
|
|
260
|
+
|
|
261
|
+
Key fields:
|
|
262
|
+
- `url` (Playwright route pattern, e.g. `**/api/users`)
|
|
263
|
+
- `response.status`
|
|
264
|
+
- exactly one of `response.body` or `response.file`
|
|
265
|
+
- optional `response.contentType` (defaults to `application/json`)
|
|
266
|
+
|
|
267
|
+
```yaml
|
|
268
|
+
- mockRoute:
|
|
269
|
+
url: "**/api/users"
|
|
270
|
+
response:
|
|
271
|
+
status: 200
|
|
272
|
+
body: '{"users":[{"id":1}]}'
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
### unmockRoute
|
|
276
|
+
|
|
277
|
+
Remove a previously registered route mock. This step is explicit-only.
|
|
278
|
+
|
|
279
|
+
Key fields:
|
|
280
|
+
- `url` (must match the mocked route URL pattern)
|
|
281
|
+
|
|
282
|
+
```yaml
|
|
283
|
+
- unmockRoute:
|
|
284
|
+
url: "**/api/users"
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
### wait
|
|
288
|
+
|
|
289
|
+
Wait for text to appear on the page. Shorthand for `waitForSelector` with text matching.
|
|
290
|
+
|
|
291
|
+
```yaml
|
|
292
|
+
# Simple — wait for text with default timeout
|
|
293
|
+
- wait: "Loading complete"
|
|
294
|
+
|
|
295
|
+
# With custom timeout
|
|
296
|
+
- wait:
|
|
297
|
+
for: "Loading complete"
|
|
298
|
+
timeout: 10000
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
### waitForSelector
|
|
302
|
+
|
|
303
|
+
Wait for any Playwright selector to appear.
|
|
304
|
+
|
|
305
|
+
```yaml
|
|
306
|
+
- waitForSelector:
|
|
307
|
+
selector: "[data-testid='results-table']"
|
|
308
|
+
timeout: 5000
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
### waitForUrl
|
|
312
|
+
|
|
313
|
+
Wait for the URL to contain a substring.
|
|
314
|
+
|
|
315
|
+
```yaml
|
|
316
|
+
- waitForUrl:
|
|
317
|
+
value: "/dashboard"
|
|
318
|
+
timeout: 10000
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
### waitForNetworkIdle
|
|
322
|
+
|
|
323
|
+
Wait for all network requests to complete.
|
|
324
|
+
|
|
325
|
+
```yaml
|
|
326
|
+
- waitForNetworkIdle:
|
|
327
|
+
timeout: 5000
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
### onDialog
|
|
331
|
+
|
|
332
|
+
Handle browser-native dialogs (alert, confirm, prompt). Register the handler **before** the action that triggers the dialog.
|
|
333
|
+
|
|
334
|
+
```yaml
|
|
335
|
+
- onDialog:
|
|
336
|
+
action: accept # or "dismiss"
|
|
337
|
+
- click: "Delete" # this triggers the confirm dialog
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
### setInputFiles
|
|
341
|
+
|
|
342
|
+
Set files on `<input type="file">` elements. Paths are relative to `.prowl/`.
|
|
343
|
+
|
|
344
|
+
```yaml
|
|
345
|
+
# Single file
|
|
346
|
+
- setInputFiles:
|
|
347
|
+
selector: "[data-testid='avatar-upload']"
|
|
348
|
+
files: "fixtures/avatar.png"
|
|
349
|
+
|
|
350
|
+
# Multiple files
|
|
351
|
+
- setInputFiles:
|
|
352
|
+
selector: "[data-testid='attachments']"
|
|
353
|
+
files:
|
|
354
|
+
- "fixtures/doc1.pdf"
|
|
355
|
+
- "fixtures/doc2.pdf"
|
|
356
|
+
```
|
|
357
|
+
|
|
358
|
+
### runHunt
|
|
359
|
+
|
|
360
|
+
Execute another hunt file inline. Enables reusable sub-flows like login.
|
|
361
|
+
|
|
362
|
+
```yaml
|
|
363
|
+
# Simple — run the hunt as-is
|
|
364
|
+
- runHunt: "login-flow"
|
|
365
|
+
|
|
366
|
+
# With variable overrides
|
|
367
|
+
- runHunt:
|
|
368
|
+
name: "login-flow"
|
|
369
|
+
vars:
|
|
370
|
+
EMAIL: "admin@test.com"
|
|
371
|
+
PASSWORD: "{{ADMIN_PASSWORD}}"
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
Circular dependencies are detected automatically (`A → B → A` will error).
|
|
375
|
+
|
|
376
|
+
### screenshot
|
|
377
|
+
|
|
378
|
+
Capture a screenshot at any point.
|
|
379
|
+
|
|
380
|
+
```yaml
|
|
381
|
+
- screenshot:
|
|
382
|
+
name: "after-login"
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
---
|
|
386
|
+
|
|
387
|
+
## Assertion Reference
|
|
388
|
+
|
|
389
|
+
### Inline Assertions (step-level)
|
|
390
|
+
|
|
391
|
+
Use `assert` steps anywhere in your hunt for mid-flow checks:
|
|
392
|
+
|
|
393
|
+
```yaml
|
|
394
|
+
- assert:
|
|
395
|
+
visible: "Welcome" # Text must be visible on page
|
|
396
|
+
- assert:
|
|
397
|
+
notVisible: "Error" # Text must NOT be visible
|
|
398
|
+
- assert:
|
|
399
|
+
urlIncludes: "/dashboard" # Current URL must contain string
|
|
400
|
+
- assert:
|
|
401
|
+
urlEquals: "https://..." # Current URL must match exactly
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
### Hunt-Level Assertions
|
|
405
|
+
|
|
406
|
+
Run after all steps complete:
|
|
407
|
+
|
|
408
|
+
```yaml
|
|
409
|
+
assertions:
|
|
410
|
+
- selectorExists: "h1" # Element must exist
|
|
411
|
+
- selectorNotExists: ".error-banner" # Element must NOT exist
|
|
412
|
+
- urlIncludes: "/dashboard"
|
|
413
|
+
- urlEquals: "https://example.com/"
|
|
414
|
+
- noConsoleErrors: true # No console.error messages
|
|
415
|
+
- noNetworkErrors: true # No HTTP responses >= 400
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
---
|
|
419
|
+
|
|
420
|
+
## Config Reference
|
|
421
|
+
|
|
422
|
+
Config lives at `.prowl/config.yml`. All options with defaults:
|
|
423
|
+
|
|
424
|
+
```yaml
|
|
425
|
+
# The base URL for all hunt navigation
|
|
426
|
+
target:
|
|
427
|
+
url: "http://localhost:3000" # Required
|
|
428
|
+
|
|
429
|
+
# Browser settings
|
|
430
|
+
browser:
|
|
431
|
+
headless: true # false = show the browser window
|
|
432
|
+
slowMo: 0 # ms delay between actions (debugging)
|
|
433
|
+
timeout: 30000 # default page operation timeout
|
|
434
|
+
|
|
435
|
+
# What gets saved per run
|
|
436
|
+
artifacts:
|
|
437
|
+
screenshots: "on-failure" # "on-failure" or "all"
|
|
438
|
+
networkHar: false # save network activity as HAR
|
|
439
|
+
console: true # save browser console output
|
|
440
|
+
|
|
441
|
+
# Hunt-level assertions (applied to every hunt)
|
|
442
|
+
assertions:
|
|
443
|
+
noConsoleErrors: true # fail on console.error
|
|
444
|
+
noNetworkErrors: true # fail on HTTP >= 400
|
|
445
|
+
maxTotalTimeMs: 30000 # max total time for all steps
|
|
446
|
+
networkIgnorePatterns: [] # URL substrings to ignore
|
|
447
|
+
|
|
448
|
+
# Safety guardrails
|
|
449
|
+
guardrails:
|
|
450
|
+
maxSteps: 50 # max steps per hunt
|
|
451
|
+
allowedDomains: # only navigate to these domains
|
|
452
|
+
- "localhost"
|
|
453
|
+
- "127.0.0.1"
|
|
454
|
+
forbiddenSelectors: # selectors that steps cannot use
|
|
455
|
+
- "[data-danger]"
|
|
456
|
+
- ".delete-btn"
|
|
457
|
+
|
|
458
|
+
# Auth state from `prowl login`
|
|
459
|
+
auth:
|
|
460
|
+
storageStatePath: ".prowl/auth-state.json"
|
|
461
|
+
|
|
462
|
+
# Run history retention
|
|
463
|
+
history:
|
|
464
|
+
maxRuns: 100 # keep last N runs per hunt
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
### Guardrails Matching Semantics
|
|
468
|
+
|
|
469
|
+
- **`forbiddenSelectors`** and **`assertions.networkIgnorePatterns`** both use JavaScript `includes()` for case-sensitive substring matching. A pattern of `"Delete"` matches `"Delete History"`, but `"delete"` does not. Specific selectors like `".delete-btn"` also match `".undelete-btn"` because the substring is present, so prefer exact-enough patterns instead of broad fragments.
|
|
470
|
+
- **`allowedDomains`** is enforced only for `http:` and `https:` navigations. The `about:` and `data:` protocols (for example, `about:blank`) bypass the allowlist by design so hunts can interact with browser-internal pages.
|
|
471
|
+
- **Migration note:** If an older config relied on lowercase patterns like `"delete"` matching uppercase text such as `"Delete History"`, update the pattern to the exact case present in the selector or URL. Apply the same review to `forbiddenSelectors`, `assertions.networkIgnorePatterns`, and any `allowedDomains` assumptions about `about:` or `data:` URLs.
|
|
472
|
+
|
|
473
|
+
<!-- ILLUSTRATION: Annotated diagram showing each config section's purpose and how it maps to runtime behavior -->
|
|
474
|
+
|
|
475
|
+
---
|
|
476
|
+
|
|
477
|
+
### Self-Healing Selectors
|
|
478
|
+
|
|
479
|
+
Set `guardrails.selfHealing: true` (default `false`) to let Prowl recover when an
|
|
480
|
+
**explicit** selector stops matching — for example after a markup change renames
|
|
481
|
+
`#sign-in-btn`. When such a selector matches nothing, Prowl derives the intent from
|
|
482
|
+
the selector text and tries, in order:
|
|
483
|
+
|
|
484
|
+
1. **Fuzzy text** — an element containing the selector's words (e.g. "sign in")
|
|
485
|
+
2. **ARIA label** — an element whose `aria-label` contains those words
|
|
486
|
+
3. **Structural** — an interactive element (`button`, `a`, `input`, …) containing the text
|
|
487
|
+
|
|
488
|
+
It heals **only** to a candidate that matches exactly one element — it never guesses
|
|
489
|
+
among multiple. A heal is logged as a warning and recorded in the run report:
|
|
490
|
+
|
|
491
|
+
- `result.json`: the step gains a `healedFrom` field
|
|
492
|
+
- `summary.md`: a **Self-Healed Selectors** section lists `original → healed`
|
|
493
|
+
|
|
494
|
+
Healing applies to action steps (`click`, `fill`, `selectOption`, `setInputFiles`,
|
|
495
|
+
`press`, `hover`, `scrollTo`) and is meant as a safety net — update your hunt to a stable
|
|
496
|
+
selector (ideally a `data-testid`) when you see a heal. `waitForSelector` is excluded,
|
|
497
|
+
since a not-yet-present element is its normal state.
|
|
498
|
+
|
|
499
|
+
## Variable Interpolation
|
|
500
|
+
|
|
501
|
+
Use `{{VAR_NAME}}` to inject dynamic values into your hunts.
|
|
502
|
+
|
|
503
|
+
### Variable Sources (precedence order)
|
|
504
|
+
|
|
505
|
+
1. **Hunt vars** — defined in the hunt's `vars:` block (highest priority)
|
|
506
|
+
2. **Environment variables** — from `process.env`
|
|
507
|
+
3. **`.env` file** — from `.prowl/.env` (loaded automatically)
|
|
508
|
+
|
|
509
|
+
```yaml
|
|
510
|
+
# .prowl/hunts/login-flow.yml
|
|
511
|
+
vars:
|
|
512
|
+
EMAIL: "{{TEST_EMAIL}}" # References env var TEST_EMAIL
|
|
513
|
+
TIMEOUT: "5000" # Static value
|
|
514
|
+
|
|
515
|
+
steps:
|
|
516
|
+
- fill:
|
|
517
|
+
"Email": "{{EMAIL}}" # Resolves to the value of TEST_EMAIL
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
### .env File
|
|
521
|
+
|
|
522
|
+
Create `.prowl/.env` for secrets:
|
|
523
|
+
|
|
524
|
+
```env
|
|
525
|
+
TEST_EMAIL=user@example.com
|
|
526
|
+
TEST_PASSWORD=secret123
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
### Automatic Redaction
|
|
530
|
+
|
|
531
|
+
Any `fill` or `type` step whose value came from a `{{VAR}}` interpolation is automatically redacted in reports:
|
|
532
|
+
|
|
533
|
+
```
|
|
534
|
+
# In summary.md and result.json:
|
|
535
|
+
fill "[data-testid='email']" → [REDACTED]
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
This prevents credentials from leaking into artifacts, CI logs, or screenshots.
|
|
539
|
+
|
|
540
|
+
---
|
|
541
|
+
|
|
542
|
+
## Shorthand vs Explicit Syntax
|
|
543
|
+
|
|
544
|
+
Every shorthand has an explicit equivalent. Use shorthand for readability, explicit for precision.
|
|
545
|
+
|
|
546
|
+
| Shorthand | Explicit Equivalent |
|
|
547
|
+
|-----------|-------------------|
|
|
548
|
+
| `click: "Sign In"` | `click: { selector: 'button:has-text("Sign In")' }` |
|
|
549
|
+
| `fill: { "Email": "val" }` | `fill: { selector: 'input[placeholder="Email"]', value: "val" }` |
|
|
550
|
+
| `type: "text"` | `fill: { selector: ':focus', value: "text" }` |
|
|
551
|
+
| `select: { "State": "FL" }` | `selectOption: { selector: 'select[name="state"]', value: "FL" }` |
|
|
552
|
+
| `wait: "Welcome"` | `waitForSelector: { selector: 'text="Welcome"' }` |
|
|
553
|
+
| `runHunt: "login"` | `runHunt: { name: "login" }` |
|
|
554
|
+
|
|
555
|
+
---
|
|
556
|
+
|
|
557
|
+
## Selector Best Practices
|
|
558
|
+
|
|
559
|
+
Prowl uses Playwright's selector engine. For stable, maintainable selectors:
|
|
560
|
+
|
|
561
|
+
1. **`data-testid`** (best) — explicit test hooks that don't change with UI refactors
|
|
562
|
+
```yaml
|
|
563
|
+
- click: { selector: "[data-testid='submit']" }
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
2. **Accessible roles** — semantic and resilient to styling changes
|
|
567
|
+
```yaml
|
|
568
|
+
- click: { selector: "role=button[name='Submit']" }
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
3. **Labels/placeholders** — via shorthand, Prowl resolves these automatically
|
|
572
|
+
```yaml
|
|
573
|
+
- fill: { "Email": "user@test.com" }
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
4. **Text content** — via shorthand click, good for buttons and links
|
|
577
|
+
```yaml
|
|
578
|
+
- click: "Sign In"
|
|
579
|
+
```
|
|
580
|
+
|
|
581
|
+
5. **CSS selectors** (last resort) — fragile, avoid class names that change
|
|
582
|
+
```yaml
|
|
583
|
+
- click: { selector: ".btn-primary" } # Avoid if possible
|
|
584
|
+
```
|
|
585
|
+
|
|
586
|
+
---
|
|
587
|
+
|
|
588
|
+
## Auth Setup
|
|
589
|
+
|
|
590
|
+
For hunts that require authentication, use `prowl login` to capture browser state:
|
|
591
|
+
|
|
592
|
+
```bash
|
|
593
|
+
prowl login
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
This opens a headed Chromium window. Log in manually, then close the browser. Prowl saves cookies, localStorage, and sessionStorage to `.prowl/auth-state.json`.
|
|
597
|
+
|
|
598
|
+
All subsequent `prowl run` commands will load this auth state, so your hunts start already logged in.
|
|
599
|
+
|
|
600
|
+
### Using Auth State in Hunts
|
|
601
|
+
|
|
602
|
+
No changes needed — auth state is loaded automatically from the path in `config.yml`:
|
|
603
|
+
|
|
604
|
+
```yaml
|
|
605
|
+
auth:
|
|
606
|
+
storageStatePath: ".prowl/auth-state.json"
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
### Refreshing Auth
|
|
610
|
+
|
|
611
|
+
If your session expires, run `prowl login` again to re-capture.
|
|
612
|
+
|
|
613
|
+
---
|
|
614
|
+
|
|
615
|
+
## Artifacts
|
|
616
|
+
|
|
617
|
+
Every hunt run generates artifacts in `.prowl/runs/<timestamp>/`:
|
|
618
|
+
|
|
619
|
+
```text
|
|
620
|
+
.prowl/runs/2026-02-09_10-30-45/
|
|
621
|
+
├── summary.md # Human-readable report
|
|
622
|
+
├── result.json # Machine-readable results
|
|
623
|
+
├── console.log # Browser console output
|
|
624
|
+
├── screenshots/
|
|
625
|
+
│ ├── final.png # Final page state
|
|
626
|
+
│ └── failure_step_3.png # Screenshot on failure (if any)
|
|
627
|
+
├── trace.zip # Playwright trace (if --trace)
|
|
628
|
+
└── network.har # Network activity (if networkHar: true)
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
<!-- ILLUSTRATION: Screenshot of a run directory in Finder/terminal showing the artifact files -->
|
|
632
|
+
|
|
633
|
+
### Viewing Traces
|
|
634
|
+
|
|
635
|
+
```bash
|
|
636
|
+
npx playwright show-trace .prowl/runs/2026-02-09_10-30-45/trace.zip
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
### Trace Correlation (link failures to your app's traces)
|
|
640
|
+
|
|
641
|
+
When a hunt hits a failing request (HTTP status ≥ 400), Prowl reads the response's
|
|
642
|
+
`traceparent` header, extracts the W3C trace ID, and records it. This lets you pivot
|
|
643
|
+
straight from a hunt failure to the matching distributed trace in your own
|
|
644
|
+
observability stack (Datadog, Grafana/Tempo, Jaeger, etc.).
|
|
645
|
+
|
|
646
|
+
The trace IDs appear in:
|
|
647
|
+
- `result.json` under a `traceCorrelations` array (`url`, `status`, `traceId`, `header`)
|
|
648
|
+
- `summary.md` under a **Trace Correlations** section
|
|
649
|
+
|
|
650
|
+
If your app uses a non-standard header, configure it in `.prowl/config.yml`:
|
|
651
|
+
|
|
652
|
+
```yaml
|
|
653
|
+
tracing:
|
|
654
|
+
header: "x-request-id" # default: "traceparent"
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
This is a correlation bridge only — Prowl does not generate or propagate its own
|
|
658
|
+
spans. When the app emits no trace headers, nothing is recorded (no noise).
|
|
659
|
+
|
|
660
|
+
---
|
|
661
|
+
|
|
662
|
+
## CLI Reference
|
|
663
|
+
|
|
664
|
+
```bash
|
|
665
|
+
# Run a hunt
|
|
666
|
+
prowl run <hunt-name>
|
|
667
|
+
prowl run <hunt-name> --headed # Show browser window
|
|
668
|
+
prowl run <hunt-name> --trace # Capture Playwright trace
|
|
669
|
+
prowl run <hunt-name> --slow-mo 500 # Slow down actions (ms)
|
|
670
|
+
prowl run <hunt-name> --url <override> # Override target URL
|
|
671
|
+
prowl run <hunt-name> --config <path> # Custom config path
|
|
672
|
+
|
|
673
|
+
# Watch mode — re-runs on file changes
|
|
674
|
+
prowl watch <hunt-name>
|
|
675
|
+
|
|
676
|
+
# Auth — capture login state interactively
|
|
677
|
+
prowl login
|
|
678
|
+
|
|
679
|
+
# Initialize — create .prowl directory with examples
|
|
680
|
+
prowl init
|
|
681
|
+
prowl init --force # Overwrite existing
|
|
682
|
+
|
|
683
|
+
# List available hunts
|
|
684
|
+
prowl list
|
|
685
|
+
|
|
686
|
+
# CI mode — run all hunts with aggregate status
|
|
687
|
+
prowl ci
|
|
688
|
+
prowl ci --json # Machine-readable CI output
|
|
689
|
+
prowl ci --parallel 4 # Run hunts with 4 workers
|
|
690
|
+
|
|
691
|
+
# History — show past runs of a hunt
|
|
692
|
+
prowl history <hunt-name>
|
|
693
|
+
prowl history <hunt-name> --limit 50 # Show the last 50 runs (default: 20)
|
|
694
|
+
prowl history <hunt-name> --json # Machine-readable history output
|
|
695
|
+
|
|
696
|
+
# MCP server — expose Prowl to AI agents over stdio
|
|
697
|
+
prowl mcp
|
|
698
|
+
prowl mcp --projects ~/.prowl/projects.yml # Drive multiple repos via a registry
|
|
699
|
+
```
|
|
700
|
+
|
|
701
|
+
`--parallel <count>` details:
|
|
702
|
+
- Runs hunts in parallel with `count` workers.
|
|
703
|
+
- Must be a positive integer (`>= 1`).
|
|
704
|
+
- Invalid values (for example `0` or `1.5`) fail fast with an argument error.
|
|
705
|
+
|
|
706
|
+
### Run History
|
|
707
|
+
|
|
708
|
+
Every `prowl run` and `prowl ci` appends an entry to `.prowl/history.json`
|
|
709
|
+
with the hunt name, status, start time, duration, and run directory. Retention
|
|
710
|
+
is capped per hunt by `history.maxRuns` (default 100) — once a hunt exceeds the
|
|
711
|
+
cap, its oldest entries are dropped on the next write. Other hunts are not
|
|
712
|
+
affected.
|
|
713
|
+
|
|
714
|
+
```yaml
|
|
715
|
+
# In .prowl/config.yml
|
|
716
|
+
history:
|
|
717
|
+
maxRuns: 50 # keep the last 50 runs per hunt (default: 100)
|
|
718
|
+
```
|
|
719
|
+
|
|
720
|
+
Use `prowl history <hunt-name>` for a quick status/duration table, or
|
|
721
|
+
`--json` to feed the entries into dashboards, flake detectors, or agents.
|
|
722
|
+
|
|
723
|
+
### Failure Clustering
|
|
724
|
+
|
|
725
|
+
When a `prowl ci` run has multiple failures that share a common cause — the same
|
|
726
|
+
step type, selector, and (normalized) error — Prowl groups them into a single
|
|
727
|
+
**failure cluster**. Instead of triaging five separate failures, you see one root
|
|
728
|
+
cause (for example, a renamed `#submit` selector that broke five hunts).
|
|
729
|
+
|
|
730
|
+
Clusters appear in:
|
|
731
|
+
- the **Failure clusters** section of the CI summary, with the cause and affected hunts
|
|
732
|
+
- a `clusters` array in `ci-result.json` (and `prowl ci --json`), each entry with
|
|
733
|
+
`cause`, `stepType`, `selector`, `error`, `count`, and `hunts`
|
|
734
|
+
|
|
735
|
+
Only causes shared by more than one hunt are reported as clusters. This pairs well
|
|
736
|
+
with self-healing selectors and flake detection to cut triage time on large suites.
|
|
737
|
+
|
|
738
|
+
---
|
|
739
|
+
|
|
740
|
+
## MCP Server (AI Agent Integration)
|
|
741
|
+
|
|
742
|
+
Prowl can run as an [MCP](https://modelcontextprotocol.io) server, exposing QA
|
|
743
|
+
as a small set of named tools that any MCP-capable agent can call over stdio. The
|
|
744
|
+
agent triggers runs and reads structured results through these tools — it never
|
|
745
|
+
needs shell access to your repo.
|
|
746
|
+
|
|
747
|
+
**Prerequisites:**
|
|
748
|
+
|
|
749
|
+
- **`prowl` must be on your `PATH`.** Install it globally with `npm install -g prowl-tools`, or launch it through `npx` (use `"command": "npx", "args": ["prowl-tools", "mcp"]` in the client config below). If the binary can't be found, the MCP client fails to start the server with no hunt-specific error.
|
|
750
|
+
- **The target project must be initialized** — a `.prowl/` directory with a valid config and hunts. Run `prowl init` and author hunts first. Pointed at an uninitialized repo, MCP tool calls fail with a missing `.prowl/config.yml` error.
|
|
751
|
+
|
|
752
|
+
```bash
|
|
753
|
+
prowl mcp
|
|
754
|
+
```
|
|
755
|
+
|
|
756
|
+
This starts a stdio server for the current project (it discovers `.prowl/` from
|
|
757
|
+
the working directory, exactly like the other commands). Point your MCP client at
|
|
758
|
+
it — for example:
|
|
759
|
+
|
|
760
|
+
```json
|
|
761
|
+
{
|
|
762
|
+
"mcpServers": {
|
|
763
|
+
"prowl": {
|
|
764
|
+
"command": "prowl",
|
|
765
|
+
"args": ["mcp"],
|
|
766
|
+
"cwd": "/path/to/your/project"
|
|
767
|
+
}
|
|
768
|
+
}
|
|
769
|
+
}
|
|
770
|
+
```
|
|
771
|
+
|
|
772
|
+
### Tools
|
|
773
|
+
|
|
774
|
+
| Tool | Arguments | Returns |
|
|
775
|
+
|------|-----------|---------|
|
|
776
|
+
| `list_hunts` | `project?` | Hunt names in run order |
|
|
777
|
+
| `run_hunt` | `hunt`, `project?` | The full `RunResult` for a single hunt |
|
|
778
|
+
| `run_suite` | `includeTags?`, `excludeTags?`, `parallel?`, `logBugs?`, `project?` | Pass/fail/skip counts, the `ci-result.json` path, and the bug tickets created |
|
|
779
|
+
| `list_projects` | — | Registered projects (empty unless a registry is configured) |
|
|
780
|
+
|
|
781
|
+
Existing guardrails (`allowedDomains`, `forbiddenSelectors`, `maxSteps`,
|
|
782
|
+
`maxTotalTimeMs`) apply to every run the server triggers.
|
|
783
|
+
|
|
784
|
+
**Controlling what the agent can do:** Prowl exposes only these four tools and
|
|
785
|
+
never runs arbitrary shell. To restrict the agent further, allow-list tool names
|
|
786
|
+
in your MCP client (e.g. OpenClaw) config — for example, allow `list_hunts` and
|
|
787
|
+
`run_suite` but withhold `run_hunt`. That allow-listing is configured on the
|
|
788
|
+
agent/client side, not in Prowl.
|
|
789
|
+
|
|
790
|
+
### Logging bugs automatically
|
|
791
|
+
|
|
792
|
+
`run_suite` runs every hunt and, by default, logs each failure as a deduplicated
|
|
793
|
+
bug ticket in the project's `docs/backlog.md`, under a `## QA Findings (automated)`
|
|
794
|
+
section that stays separate from your hand-written items. A bug is identified by
|
|
795
|
+
hunt + failing step + normalized error, so:
|
|
796
|
+
|
|
797
|
+
- a brand-new failure creates a `QA-NNN` ticket with the hunt, failing step, error, and a link to the run artifacts;
|
|
798
|
+
- a failure that already has an open ticket is left alone (no duplicates);
|
|
799
|
+
- a failure matching something already in `docs/resolved.md` is logged as a **regression** that references the old ticket id.
|
|
800
|
+
|
|
801
|
+
Pass `logBugs: false` to run without touching the backlog. A `run_suite` response
|
|
802
|
+
looks like this:
|
|
803
|
+
|
|
804
|
+
```json
|
|
805
|
+
{
|
|
806
|
+
"status": "fail",
|
|
807
|
+
"totalHunts": 8,
|
|
808
|
+
"passed": 6,
|
|
809
|
+
"failed": 2,
|
|
810
|
+
"skipped": 0,
|
|
811
|
+
"resultPath": "/path/to/project/.prowl/runs/ci-2026-05-26_09-12-03-456/ci-result.json",
|
|
812
|
+
"bugs": {
|
|
813
|
+
"created": ["QA-014"],
|
|
814
|
+
"regressions": ["QA-015"],
|
|
815
|
+
"alreadyOpen": ["QA-009"],
|
|
816
|
+
"backlogPath": "/path/to/project/docs/backlog.md"
|
|
817
|
+
}
|
|
818
|
+
}
|
|
819
|
+
```
|
|
820
|
+
|
|
821
|
+
`status` is one of `pass`, `fail`, `no-hunts`, or `all-skipped`. When
|
|
822
|
+
`logBugs` is `false`, `bugs` arrays are empty and `backlogPath` is `null`.
|
|
823
|
+
|
|
824
|
+
### Driving multiple projects
|
|
825
|
+
|
|
826
|
+
By default the server acts on the current directory. To drive several repos from a
|
|
827
|
+
single server, give it a **project registry** — one YAML file that maps project
|
|
828
|
+
names to repo roots. This file lives *outside* any repo (it spans many), not
|
|
829
|
+
inside a target project:
|
|
830
|
+
|
|
831
|
+
```yaml
|
|
832
|
+
# ~/.prowl/projects.yml
|
|
833
|
+
projects:
|
|
834
|
+
coupe:
|
|
835
|
+
root: /Users/you/projects/coupe
|
|
836
|
+
storefront:
|
|
837
|
+
root: /Users/you/projects/storefront
|
|
838
|
+
configPath: /custom/.prowl/config.yml # optional; defaults to <root>/.prowl/config.yml
|
|
839
|
+
```
|
|
840
|
+
|
|
841
|
+
The registry is resolved in priority order:
|
|
842
|
+
|
|
843
|
+
1. `prowl mcp --projects <path>`
|
|
844
|
+
2. the `PROWL_PROJECTS` environment variable
|
|
845
|
+
3. `~/.prowl/projects.yml`
|
|
846
|
+
|
|
847
|
+
With a registry loaded, every tool accepts an optional `project` argument that
|
|
848
|
+
selects which repo to act on, and `list_projects` enumerates what's available.
|
|
849
|
+
For example, these `run_suite` arguments run the smoke suite for `coupe` and log
|
|
850
|
+
any failures to `coupe/docs/backlog.md`:
|
|
851
|
+
|
|
852
|
+
```jsonc
|
|
853
|
+
{ "project": "coupe", "includeTags": ["smoke"] }
|
|
854
|
+
```
|
|
855
|
+
|
|
856
|
+
Omit `project` and the tool falls back to the current directory. Naming a project
|
|
857
|
+
that isn't registered — or naming one when no registry is configured — returns a
|
|
858
|
+
clear error.
|
|
859
|
+
|
|
860
|
+
---
|
|
861
|
+
|
|
862
|
+
## Architecture
|
|
863
|
+
|
|
864
|
+
<!-- ILLUSTRATION: Architecture diagram showing: CLI (Commander) → Config (YAML + Zod) → Runner (Step Execution) → Browser (Playwright) → Reporter (summary.md + result.json) -->
|
|
865
|
+
|
|
866
|
+
```
|
|
867
|
+
CLI Commands
|
|
868
|
+
│
|
|
869
|
+
├── Config Loader (YAML → Zod validation → merged defaults)
|
|
870
|
+
│ │
|
|
871
|
+
│ ├── Hunt Loader (YAML → schema validation → interpolation)
|
|
872
|
+
│ │
|
|
873
|
+
│ └── .env Loader (dotenv)
|
|
874
|
+
│
|
|
875
|
+
├── Runner
|
|
876
|
+
│ │
|
|
877
|
+
│ ├── Step Executor (16 step types, guardrail checks)
|
|
878
|
+
│ │
|
|
879
|
+
│ ├── Assertion Evaluator (6 assertion types)
|
|
880
|
+
│ │
|
|
881
|
+
│ └── Browser Controller (Playwright launch/close)
|
|
882
|
+
│
|
|
883
|
+
└── Reporter
|
|
884
|
+
│
|
|
885
|
+
├── summary.md (human-readable, redacted)
|
|
886
|
+
│
|
|
887
|
+
└── result.json (machine-readable)
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
---
|
|
891
|
+
|
|
892
|
+
## Community Hub
|
|
893
|
+
|
|
894
|
+
Browse and contribute hunt templates through the internal community registry (contact ops for access).
|
|
895
|
+
|
|
896
|
+
Templates cover auth flows (OAuth, 2FA), e-commerce (Stripe), admin panels, SaaS patterns, and more. Each template is heavily commented and ready to customize.
|
|
897
|
+
|
|
898
|
+
---
|
|
899
|
+
|
|
900
|
+
## Troubleshooting
|
|
901
|
+
|
|
902
|
+
### "Could not find .prowl/config.yml"
|
|
903
|
+
|
|
904
|
+
Run `prowl init` in your project root to create the `.prowl/` directory.
|
|
905
|
+
|
|
906
|
+
### "Navigation to disallowed domain"
|
|
907
|
+
|
|
908
|
+
Add the domain to `guardrails.allowedDomains` in your config:
|
|
909
|
+
|
|
910
|
+
```yaml
|
|
911
|
+
guardrails:
|
|
912
|
+
allowedDomains:
|
|
913
|
+
- "localhost"
|
|
914
|
+
- "your-domain.com"
|
|
915
|
+
```
|
|
916
|
+
|
|
917
|
+
### "Forbidden selector"
|
|
918
|
+
|
|
919
|
+
The selector matches a pattern in `guardrails.forbiddenSelectors`. Either change the selector or update the guardrails config.
|
|
920
|
+
|
|
921
|
+
### "Missing variable: VAR_NAME"
|
|
922
|
+
|
|
923
|
+
The `{{VAR_NAME}}` in your hunt couldn't be resolved. Check:
|
|
924
|
+
1. Is it defined in the hunt's `vars:` block?
|
|
925
|
+
2. Is it set in your `.prowl/.env` file?
|
|
926
|
+
3. Is it set as an environment variable?
|
|
927
|
+
|
|
928
|
+
### Selectors not finding elements
|
|
929
|
+
|
|
930
|
+
- Use `--headed` and `--slow-mo 1000` to watch the browser in real time
|
|
931
|
+
- Check if the element is inside an iframe
|
|
932
|
+
- Check if the element appears after a network request (add `waitForNetworkIdle` before)
|
|
933
|
+
- Use `--trace` and view with `npx playwright show-trace` for detailed diagnostics
|
|
934
|
+
|
|
935
|
+
### Hunt running slowly
|
|
936
|
+
|
|
937
|
+
- Check `browser.timeout` in your config — lower it for faster failures
|
|
938
|
+
- Add `waitForNetworkIdle` only where needed (it waits for ALL requests)
|
|
939
|
+
- Use `waitForSelector` with a specific element instead of `waitForNetworkIdle`
|
|
940
|
+
|
|
941
|
+
---
|
|
942
|
+
|
|
943
|
+
## License
|
|
944
|
+
|
|
945
|
+
Apache 2.0 — see [LICENSE](LICENSE)
|