prompttest-mobile 1.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/CONTRIBUTING.md +80 -0
  2. package/LICENSE +42 -0
  3. package/LICENSES.md +81 -0
  4. package/README.md +526 -0
  5. package/SECURITY.md +56 -0
  6. package/TERMS.md +138 -0
  7. package/dist/bin/prompttest.d.ts +2 -0
  8. package/dist/bin/prompttest.js +1418 -0
  9. package/dist/bin/server.d.ts +1 -0
  10. package/dist/constants/commands.d.ts +125 -0
  11. package/dist/engine.bundle.js +1039 -0
  12. package/dist/index.d.ts +143 -0
  13. package/dist/index.js +1039 -0
  14. package/dist/lib/adaptive-timing.d.ts +55 -0
  15. package/dist/lib/adb-provisioner.d.ts +55 -0
  16. package/dist/lib/adb.d.ts +448 -0
  17. package/dist/lib/ai/heuristic-resolver.d.ts +19 -0
  18. package/dist/lib/ai/index.d.ts +16 -0
  19. package/dist/lib/ai/llm-provider.d.ts +40 -0
  20. package/dist/lib/ai/types.d.ts +64 -0
  21. package/dist/lib/baseline.d.ts +159 -0
  22. package/dist/lib/benchmark.d.ts +100 -0
  23. package/dist/lib/checkpoint.d.ts +61 -0
  24. package/dist/lib/ci.d.ts +49 -0
  25. package/dist/lib/config-loader.d.ts +87 -0
  26. package/dist/lib/config.d.ts +136 -0
  27. package/dist/lib/crawler.d.ts +335 -0
  28. package/dist/lib/data-loader.d.ts +53 -0
  29. package/dist/lib/dfs-engine.d.ts +149 -0
  30. package/dist/lib/dictionary.d.ts +41 -0
  31. package/dist/lib/doctor.d.ts +45 -0
  32. package/dist/lib/driver-interface.d.ts +50 -0
  33. package/dist/lib/enterprise.d.ts +71 -0
  34. package/dist/lib/errors.d.ts +68 -0
  35. package/dist/lib/explorer.d.ts +165 -0
  36. package/dist/lib/feedback.d.ts +35 -0
  37. package/dist/lib/form-filler.d.ts +101 -0
  38. package/dist/lib/ios-driver.d.ts +38 -0
  39. package/dist/lib/jail-guard.d.ts +59 -0
  40. package/dist/lib/license.d.ts +51 -0
  41. package/dist/lib/live-server.d.ts +52 -0
  42. package/dist/lib/lock.d.ts +30 -0
  43. package/dist/lib/logger.d.ts +68 -0
  44. package/dist/lib/memory.d.ts +259 -0
  45. package/dist/lib/patterns.d.ts +202 -0
  46. package/dist/lib/profiler.d.ts +75 -0
  47. package/dist/lib/prompt-runner.d.ts +101 -0
  48. package/dist/lib/quiescence.d.ts +44 -0
  49. package/dist/lib/recorder.d.ts +155 -0
  50. package/dist/lib/repl.d.ts +29 -0
  51. package/dist/lib/reporter.d.ts +269 -0
  52. package/dist/lib/runner-utils.d.ts +316 -0
  53. package/dist/lib/scaffold.d.ts +53 -0
  54. package/dist/lib/step-handlers.d.ts +390 -0
  55. package/dist/lib/triage.d.ts +50 -0
  56. package/dist/lib/wizard.d.ts +42 -0
  57. package/dist/lib/zip-util.d.ts +36 -0
  58. package/docs/ARCHITECTURE.md +154 -0
  59. package/docs/CLI_CONTRACT.md +131 -0
  60. package/docs/CLI_STUDIO_CONTRACT.md +123 -0
  61. package/docs/PERFORMANCE_BASELINE.md +71 -0
  62. package/docs/PRODUCT_STATUS.md +56 -0
  63. package/docs/USER_MANUAL.md +764 -0
  64. package/package.json +66 -0
@@ -0,0 +1,764 @@
1
+ # 📖 PromptTest — Comprehensive User Manual & Reference Guide
2
+
3
+ Welcome to the **PromptTest** User Manual! This guide provides detailed documentation, syntax references, real-world examples, and best practices for writing and executing autonomous mobile QA tests in plain English.
4
+
5
+ ---
6
+
7
+ ## 📑 Table of Contents
8
+
9
+ 1. [Introduction & Architecture](#1-introduction--architecture)
10
+ 2. [Installation & Setup](#2-installation--setup)
11
+ 3. [CLI Command Reference](#3-cli-command-reference)
12
+ 4. [PromptTest Syntax Specification](#4-prompttest-syntax-specification)
13
+ - [Tapping & Gestures](#tapping--gestures)
14
+ - [Text Entry & Form Inputs](#text-entry--form-inputs)
15
+ - [Assertions & Verifications](#assertions--verifications)
16
+ - [Navigation, Scrolling & Swiping](#navigation-scrolling--swiping)
17
+ - [App Lifecycle & Permissions](#app-lifecycle--permissions)
18
+ 5. [Dynamic & Conditional Testing](#5-dynamic--conditional-testing)
19
+ - [Regular Expression / Pattern Matching](#regular-expression--pattern-matching)
20
+ - [Multi-Alternative (OR) Matching](#multi-alternative-or-matching)
21
+ - [Optional / Conditional Actions](#optional--conditional-actions)
22
+ - [Dynamic Variables & Generators](#dynamic-variables--generators)
23
+ 6. [Autonomous Record & Replay](#6-autonomous-record--replay)
24
+ 7. [Autonomous Exploration & State-Graph Crawler (Mode 2)](#7-autonomous-exploration--state-graph-crawler-mode-2)
25
+ 8. [Self-Healing & Resilience Engine](#8-self-healing--resilience-engine)
26
+ 9. [Multi-Device Matrix & CI/CD Pipelines](#9-multi-device-matrix--cicd-pipelines)
27
+ 10. [Advanced Features & SDK](#10-advanced-features--sdk)
28
+
29
+ - [Checkpointing & Resume](#checkpointing--resume)
30
+ - [Visual Baselines](#visual-baselines)
31
+ - [Custom Reporters](#custom-reporters)
32
+ - [APK Auto-Install](#apk-auto-install)
33
+ - [Device Locking](#device-locking)
34
+ - [Programmatic SDK](#programmatic-sdk)
35
+
36
+ 11. [Real-World Domain Recipes](#11-real-world-domain-recipes)
37
+
38
+ - [EdTech & Coaching Portals](#edtech--coaching-portals)
39
+ - [E-Commerce & Food Delivery](#e-commerce--food-delivery)
40
+ - [FinTech & Banking](#fintech--banking)
41
+ - [Social & Messaging](#social--messaging)
42
+
43
+ 12. [Mode Expectations, Capabilities & Limitations](#12-mode-expectations-capabilities--limitations)
44
+
45
+ ---
46
+
47
+ ## 1. Introduction & Architecture
48
+
49
+ **PromptTest** is an AI-native autonomous mobile testing framework for Android. It enables developers, QA engineers, and product teams to write executable test specs in plain English without writing fragile XPath selectors or maintaining complex Appium setups.
50
+
51
+ ### Key Architecture Components:
52
+
53
+ - **`AndroidDriver` (ADB Layer):** Direct, zero-latency communication with physical Android devices and emulators over USB or Wi-Fi.
54
+ - **`HierarchyCrawler` (Visual UI Engine):** Extracts, cleans, and indexes on-screen Android view trees.
55
+ - **`PromptRunner` (Execution Brain):** Parses plain-English instructions, performs label-to-input pairing, auto-scrolls viewports, and executes resilient assertions.
56
+ - **`MemoryEngine` (Autonomous Knowledge Graph):** Learns screen aliases and component identifiers across test runs for instant 0ms lookups.
57
+ - **`AutonomousRecorder` (Interactive Record-and-Play):** Live-monitors screen interactions and auto-generates test specs.
58
+
59
+ ---
60
+
61
+ ## 2. Installation & Setup
62
+
63
+ ### Requirements
64
+
65
+ - **Node.js**: v18.0.0 or higher
66
+ - **Android SDK Platform Tools**: `adb` must be available in your system `PATH`.
67
+ - **Target Device**: Physical Android device (with USB debugging enabled) or Android Emulator.
68
+
69
+ ### Running via npx (Zero Installation)
70
+
71
+ ```bash
72
+ npx prompttest devices
73
+ npx prompttest run specs/my_test.txt
74
+ ```
75
+
76
+ ### Installing Locally in Your Repository
77
+
78
+ ```bash
79
+ npm install --save-dev prompttest
80
+ ```
81
+
82
+ Add scripts to your `package.json`:
83
+
84
+ ```json
85
+ {
86
+ "scripts": {
87
+ "test:mobile": "prompttest run specs/regression_suite.txt",
88
+ "test:fresh": "prompttest run specs/regression_suite.txt --fresh",
89
+ "test:matrix": "prompttest run specs/regression_suite.txt --all-devices"
90
+ }
91
+ }
92
+ ```
93
+
94
+ ---
95
+
96
+ ## 3. CLI Command Reference
97
+
98
+ | Command | Description | Example |
99
+ | --------------------------------- | ----------------------------------------------- | ------------------------------------------ |
100
+ | `prompttest devices` | Scans and lists all connected Android devices | `npx prompttest devices` |
101
+ | `prompttest wifi [ip[:port]]` | Connects to a device wirelessly over Wi-Fi | `npx prompttest wifi 192.168.1.50` |
102
+ | `prompttest disconnect [ip]` | Disconnects wireless device | `npx prompttest disconnect` |
103
+ | `prompttest status` | Inspects foreground application & UI components | `npx prompttest status` |
104
+ | `prompttest record [spec.txt]` | Interactive record-and-play session | `npx prompttest record specs/login.txt` |
105
+ | `prompttest run <spec.txt>` | Executes plain-English test spec (Fail-Fast) | `npx prompttest run specs/teacher.txt` |
106
+ | `prompttest watch <spec.txt>` | Continuously re-runs test upon spec file save | `npx prompttest watch specs/login.txt` |
107
+ | `prompttest doctor` | Runs full workstation & ADB readiness diagnosis | `npx prompttest doctor` |
108
+ | `prompttest repl` | Launches interactive live testing shell | `npx prompttest repl` |
109
+ | `prompttest benchmark` | Profiles ADB hierarchy, screencap, & matching | `npx prompttest benchmark` |
110
+ | `prompttest merge-reports [dir]` | Merges multi-device results into Atto dashboard | `npx prompttest merge-reports output` |
111
+ | `prompttest explore [pkg]` | Autonomous crawler exploration (forms/tabs) | `npx prompttest explore host.exp.exponent` |
112
+ | `prompttest memory [pkg]` | Views learned component aliases & confidence | `npx prompttest memory host.exp.exponent` |
113
+ | `prompttest memory stats [pkg]` | Displays learned knowledge health statistics | `npx prompttest memory stats` |
114
+ | `prompttest memory export [pkg]` | Exports learned memory to a JSON file | `npx prompttest memory export host.exp` |
115
+ | `prompttest memory import <file>` | Imports/merges learned memory from JSON file | `npx prompttest memory import mem.json` |
116
+ | `prompttest memory clear [pkg]` | Clears learned knowledge cache | `npx prompttest memory clear` |
117
+ | `prompttest init-ci` | Generates GitHub Actions CI workflows | `npx prompttest init-ci` |
118
+
119
+ ### CLI Execution Flags
120
+
121
+ #### Filtering, Flow Control & Debugging
122
+
123
+ - `--dry-run`: Validates and parses test spec syntax without device interaction.
124
+ - `--list-steps`: Prints a structured table of the parsed step AST to stdout and exits.
125
+ - `--only="<range>"`: Executes only matching step numbers or ranges (e.g. `--only="1-3,5"`).
126
+ - `--tags="<tag>"`: Filters execution to specs matching tagged directives (e.g. `--tags="smoke"`).
127
+ - `--continue` or `--continue-on-error`: Continues running remaining steps even if an upstream step fails.
128
+ - `--fresh` or `--cold-start`: Force-stops and restarts target app before test begins.
129
+ - `--resume`: Resumes test execution from the last recorded checkpoint upon failure.
130
+ - `--var:KEY=VALUE`: Injects inline dynamic variables (e.g. `--var:ROLE=TEACHER`).
131
+
132
+ #### Observability, Reporting & Live Monitoring
133
+
134
+ - `--json`: Outputs machine-readable execution report JSON directly to stdout.
135
+ - `--serve [port]`: Streams live test execution events over HTTP/SSE dashboard (default: `http://localhost:4040`).
136
+ - `--embed-screenshots`: Inlines base64 screenshots into standalone HTML report for zero-asset portability.
137
+ - `--screenshots` or `--all-screenshots`: Captures full screenshots on every step (default: on-failure only).
138
+ - `--video`: Records the entire test session to high-definition MP4 video via native ADB screenrecord.
139
+ - `--reporters=json,html,junit,md`: Generates specific output report formats.
140
+ - `--reporter=<path>`: Loads a custom reporter module implementing the `Reporter` interface.
141
+
142
+ #### Data-Driven & Localization
143
+
144
+ - `--data=<path.csv|json>`: Executes data-driven tests across multiple rows/parameter sets.
145
+ - `--iterations=N`: Repeats the test suite across N iterations or data rows.
146
+ - `--only-row=N`: Executes only the specified 1-indexed data row.
147
+ - `--locale=<code>`: Sets active taxonomy locale dictionary (e.g. `'en'`, `'es'`, `'hi'`).
148
+
149
+ #### AI, Resilience & Visual Regression
150
+
151
+ - `--heal`: Enables automatic element locator self-healing via offline synonyms and heuristic memory.
152
+ - `--no-ai`: Disables third-party LLM providers, forcing offline-only heuristic matching.
153
+ - `--expand`: Previews AI natural-language macro intent expansion without modifying the file.
154
+ - `--expand-apply`: Expands high-level macro steps and writes concrete actions directly to the spec file.
155
+ - `--save-baseline`: Saves screen layout screenshots as visual baseline.
156
+ - `--compare-baseline`: Compares live screens against saved baseline using perceptual diffing.
157
+ - `--baseline-threshold=N`: Sets perceptual diff variance threshold (0.0–1.0, default: `0.02`).
158
+ - `--apk=<path>`: Automatically installs or updates the target APK before test execution if absent.
159
+ - `--all-devices`: Runs test suite concurrently across all connected online devices.
160
+ - `-s <id>` or `--serial=<id>`: Targets a specific connected Android device by serial.
161
+
162
+ ### Environment Diagnostic Tool (`prompttest doctor`)
163
+
164
+ Run `prompttest doctor` whenever setting up a new workstation, troubleshooting device connectivity, or configuring a CI runner:
165
+
166
+ - Validates the `adb` binary presence and version in system `PATH`.
167
+ - Confirms USB/Wi-Fi debugging authorization and platform-tools integrity.
168
+ - Verifies physical screen dimensions, orientation, and battery readiness.
169
+
170
+ ### Interactive Live Sandbox (`prompttest repl`)
171
+
172
+ Use `prompttest repl` to prototype actions, debug ambiguous locators, or interactively explore screens without writing full spec files:
173
+
174
+ - `inspect`: Dumps actionable buttons, text fields, and tabs on the current screen.
175
+ - `tap '<label>'`: Taps element matching target text or descriptor.
176
+ - `type '<value>' into '<target>'`: Enters text into an input field.
177
+ - `screenshot [path]`: Captures screen evidence instantly.
178
+
179
+ ---
180
+
181
+ ## 4. PromptTest Syntax Specification
182
+
183
+ PromptTest specs are plain-text files (e.g. `specs/checkout.txt`). Steps can be numbered or written as bullet points. Lines starting with `#` are treated as comments.
184
+
185
+ ### Spec Tagging & Categorization
186
+
187
+ You can annotate test specs with tags using `# @tag: <name>` directives at the top of the file:
188
+
189
+ ```txt
190
+ # @tag: smoke
191
+ # @tag: regression
192
+ # @tag: payments
193
+
194
+ 1. Tap 'Cart'
195
+ 2. Tap 'Checkout'
196
+ ```
197
+
198
+ Run only specs matching your tag filter:
199
+
200
+ ```bash
201
+ npx prompttest run specs/checkout.txt --tags="smoke"
202
+ ```
203
+
204
+ ### Tapping & Gestures
205
+
206
+ ```txt
207
+ # Basic Tap / Click
208
+ Tap 'Sign In'
209
+ Click "Add to Cart"
210
+ Press 'Submit'
211
+
212
+ # Ordinal Tapping (when multiple matching elements exist)
213
+ Tap 1st 'View Details'
214
+ Click 2nd 'Export Report'
215
+ Tap last 'Delete'
216
+
217
+ # Proximity Tapping (targets element near an anchor)
218
+ Tap 'Delete' next to 'Rohan Gupta'
219
+ Click 'Edit' near 'Batch Grade 10'
220
+
221
+ # Long Press
222
+ Long press 'Mid-Term Examination Schedule'
223
+ Press and hold 'Voice Message'
224
+ ```
225
+
226
+ ### Text Entry & Form Inputs
227
+
228
+ PromptTest automatically associates input fields with their preceding labels and distinguishes between password and email/text fields.
229
+
230
+ ```txt
231
+ # Entering Credentials
232
+ Type 'ramesh@coachconnect.app' into 'Email Address'
233
+ Type 'Teacher@1234' into 'Password'
234
+
235
+ # Entering Search Queries
236
+ Type 'Mathematics' into 'Search by batch, subject, or teacher...'
237
+
238
+ # Clearing and Re-entering
239
+ Type 'New York' into 'City'
240
+ ```
241
+
242
+ ### Assertions & Verifications
243
+
244
+ ```txt
245
+ # Visibility Assertions
246
+ Verify 'Apex Coaching Academy' is visible
247
+ Ensure 'Dr. Ramesh Sharma' appears
248
+ Assert 'Order Confirmed' exists
249
+
250
+ # Component State Assertions
251
+ Verify 'Submit' is disabled
252
+ Verify 'Pay Now' is enabled
253
+ ```
254
+
255
+ ### Navigation, Scrolling & Swiping
256
+
257
+ ```txt
258
+ # System Back Navigation
259
+ Press back
260
+ Go back
261
+
262
+ # Vertical Scrolling
263
+ Scroll down
264
+ Scroll up
265
+
266
+ # Directional Swipes (for Carousels, Tabs, Bottom Sheets)
267
+ Swipe left
268
+ Swipe right
269
+ Swipe left on 'Featured Courses'
270
+ Swipe down on 'Modal Sheet'
271
+ ```
272
+
273
+ ### App Lifecycle & Permissions
274
+
275
+ ```txt
276
+ # Explicit App Management
277
+ Launch app 'host.exp.exponent'
278
+ Restart app 'com.example.myapp'
279
+ Terminate app
280
+ Clear app data for 'com.example.myapp'
281
+
282
+ # System Permissions
283
+ Grant permission 'android.permission.CAMERA'
284
+ Grant permission 'android.permission.ACCESS_FINE_LOCATION' to 'com.example.myapp'
285
+
286
+ # Deep Links
287
+ Open deep link 'myapp://batches/grade-10'
288
+ ```
289
+
290
+ ---
291
+
292
+ ## 5. Dynamic & Conditional Testing
293
+
294
+ ### Regular Expression / Pattern Matching
295
+
296
+ When asserting dynamic numbers, timestamps, currencies, or roll numbers that vary at runtime:
297
+
298
+ ```txt
299
+ # Percentages (e.g. 40%, 75%, 100%)
300
+ Verify pattern '\d+%' is visible
301
+
302
+ # Counters and Ratios (e.g. "5 Students", "5 / 30")
303
+ Verify pattern '\d+\s*(Students|Batches)' is visible
304
+ Verify pattern '\d+\s*/\s*\d+' is visible
305
+
306
+ # Currencies (e.g. "$49.99", "₹1,499")
307
+ Verify pattern '[$₹€]\s*\d+([.,]\d{2})?' is visible
308
+
309
+ # Formatted IDs / Roll Numbers
310
+ Verify pattern 'STD-\d{4}-\d{4}' is visible
311
+ Verify pattern 'INV-[A-Z0-9]+' is visible
312
+ ```
313
+
314
+ ### Multi-Alternative (OR) Matching
315
+
316
+ ```txt
317
+ Verify '100%|90%|80%|75%' is visible
318
+ Verify 'Active|Present|Enrolled' is visible
319
+ Verify 'In Stock|Limited Stock' is visible
320
+ ```
321
+
322
+ ### Optional / Conditional Actions
323
+
324
+ Steps marked `(optional)` or `(if present)` will execute if the element is on screen, but will **never fail the test** if absent:
325
+
326
+ ```txt
327
+ Tap 'Dismiss' (optional)
328
+ Tap 'Allow' (if present)
329
+ Tap 'Skip Tour' (optional)
330
+ ```
331
+
332
+ ### Dynamic Variables & Generators
333
+
334
+ PromptTest supports built-in generators and custom runtime variable injection.
335
+
336
+ #### Built-in Dynamic Generators
337
+
338
+ ```txt
339
+ Type '$RANDOM_EMAIL' into 'Email Address' # Generates: test_492019@prompttest.io
340
+ Type '$RANDOM_NAME' into 'Full Name' # Generates: User_8302
341
+ Type '$RANDOM_PHONE' into 'Phone Number' # Generates: 9849201832
342
+ Type '$TIMESTAMP' into 'Reference Note' # Generates: 1788839201
343
+ Type '$UUID' into 'Transaction ID' # Generates: e9a2c-f901
344
+ ```
345
+
346
+ #### Custom CLI Variables
347
+
348
+ In your spec file:
349
+
350
+ ```txt
351
+ Type '$USER_EMAIL' into 'Email Address'
352
+ Type '$USER_PASSWORD' into 'Password'
353
+ Verify '$EXPECTED_ROLE' is visible
354
+ ```
355
+
356
+ Execute with values:
357
+
358
+ ```bash
359
+ npx prompttest run spec.txt --var:USER_EMAIL=ramesh@coachconnect.app --var:USER_PASSWORD=Teacher@1234 --var:EXPECTED_ROLE=TEACHER
360
+ ```
361
+
362
+ ---
363
+
364
+ ## 6. Autonomous Record & Replay
365
+
366
+ Record complex end-to-end user journeys interactively on your device with enterprise-grade safety:
367
+
368
+ ### Basic Recording
369
+
370
+ ```bash
371
+ npx prompttest record specs/recorded_journey.txt
372
+ ```
373
+
374
+ - **Auto-Package Tagging:** Automatically detects the foreground app and writes `# Package: <package>` to the spec header for zero-config replay.
375
+ - **Auto-Backup Protection:** If the file already exists, PromptTest automatically preserves a safety backup as `specs/recorded_journey.txt.bak` before saving.
376
+ - **Auto-Increment Defaults:** Running `npx prompttest record` without a filename will automatically generate `specs/recorded_suite_1.txt`, `specs/recorded_suite_2.txt`, etc., preventing collisions.
377
+
378
+ ### Appending to Existing Flows (`--append`)
379
+
380
+ To extend an existing test suite (e.g., adding sidebar FAQ checks to an existing 25-step test):
381
+
382
+ ```bash
383
+ npx prompttest record specs/recorded_journey.txt --append
384
+ ```
385
+
386
+ 1. PromptTest reads existing steps (e.g. 25 steps) and creates a backup.
387
+ 2. It starts numbering new recorded steps continuously from `#26`.
388
+ 3. It appends the new session with clear timestamped divider comments upon saving.
389
+
390
+ ### Intelligent Live Filtering
391
+
392
+ - **Password Bullet Filtering:** Automatically strips security masking characters (`••••`).
393
+ - **Parent Label Association:** Links input fields to their visual labels rather than duplicating input contents.
394
+ - **Keystroke Debouncing:** Captures the final entered value instead of recording each character keystroke.
395
+ - **Tap Jitter Deduplication:** Filters out accidental rapid double-taps on the same UI element.
396
+
397
+ ### Replay
398
+
399
+ ```bash
400
+ npx prompttest run specs/recorded_journey.txt
401
+ ```
402
+
403
+ - Automatically checks device app installation, launches the app if not in foreground, and executes the suite.
404
+
405
+ ---
406
+
407
+ ## 7. Autonomous Exploration & State-Graph Crawler (Mode 2)
408
+
409
+ PromptTest's **Autonomous Explorer v3** is a zero-script state-graph crawler designed to traverse, test, and map your mobile application automatically without writing test specs.
410
+
411
+ ### Basic Autonomous Exploration
412
+
413
+ ```bash
414
+ # Auto-detects the active foreground app and starts autonomous crawling
415
+ npx prompttest explore
416
+
417
+ # Target a specific installed package on a specific connected device
418
+ npx prompttest explore host.exp.exponent --serial=GEVKDEUWOJWC89OR
419
+ ```
420
+
421
+ ### The 5-Step State-Graph Exploration Pipeline
422
+
423
+ 1. **Foreground Verification & Auto-Launch**: Ensures the target application is running and in the foreground. If absent, launches the app and verifies launch readiness.
424
+ 2. **Semantic Form Detection & Auto-Filling**: Identifies username, email, password, and phone input fields, auto-populates them with realistic test data, dismisses the soft keyboard (`KEYCODE_ESCAPE`), and submits the form.
425
+ 3. **Action Buttons & Toggle Testing**: Taps standalone action buttons, flips toggle switches and checkboxes, opens side navigation drawers, and records observable UI outcomes (`NAVIGATED`, `STATE_MUTATED`, or `NO_OP` warning).
426
+ 4. **Master-Detail List Exploration & Vertical Scrolling**: Identifies scrollable lists and feeds (`RecyclerView`, `ScrollView`), scrolls vertically to discover off-screen elements below the fold, dives into detail views, and unwinds back via `KEYCODE_BACK`.
427
+ 5. **Bottom Navigation Tab Traversal & Sub-Crawling**: Traverses root-level navigation tabs with bottom gesture bar tap-clamping, recursively testing sub-actions under each tab while tracking visited screens in a state graph.
428
+
429
+ ---
430
+
431
+ ### Safety Policy Engine (`--safety-mode`)
432
+
433
+ To protect production data, shared testing environments, and user accounts from accidental deletion or modification, PromptTest enforces a multi-tiered safety policy:
434
+
435
+ ```bash
436
+ # Strict mode (default): Blocks auth drop, checkout/payments, and all deletions
437
+ npx prompttest explore host.exp.exponent --safety-mode=strict
438
+
439
+ # Moderate mode: Blocks account loss & auth, allows local in-app draft/item deletions
440
+ npx prompttest explore host.exp.exponent --safety-mode=moderate
441
+
442
+ # Interactive mode: Prompts user in console before executing risky actions
443
+ npx prompttest explore host.exp.exponent --safety-mode=interactive
444
+
445
+ # Disabled mode: Unrestricted sandbox exploration (isolated environments only)
446
+ npx prompttest explore host.exp.exponent --safety-mode=disabled
447
+ ```
448
+
449
+ | Policy Mode | Description | Default Protections |
450
+ | :------------------- | :----------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------- |
451
+ | `strict` _(Default)_ | Maximum safety for production or shared test backends. | Blocks logout, signout, account deletion, data wipe, payments, checkout, and generic item deletions. |
452
+ | `moderate` | Recommended for active development and staging environments. | Shields account deletion, user wipe, and auth termination, but permits localized draft and test item deletion. |
453
+ | `interactive` | Hybrid human-in-the-loop testing. | Prompts `[y/N]` before touching destructive elements. Failsafe: automatically falls back to skip in headless CI. |
454
+ | `disabled` | Isolated sandboxes and ephemeral emulators only. | Bypasses all safety pattern checks. |
455
+
456
+ ---
457
+
458
+ ### Configurable Safety Blacklists (`--safety-blacklist`)
459
+
460
+ You can define custom blacklist keywords to protect domain-specific critical actions (e.g. "Publish", "Transfer", "Revoke"):
461
+
462
+ ```bash
463
+ # Via CLI argument (comma-separated keywords)
464
+ npx prompttest explore host.exp.exponent --safety-blacklist="Transfer,Revoke,Publish,Notice"
465
+ ```
466
+
467
+ Or configure permanent safety rules in your `.prompttestrc.json`:
468
+
469
+ ```json
470
+ {
471
+ "safety": {
472
+ "mode": "strict",
473
+ "customBlacklist": ["Transfer Funds", "Revoke Access", "Publish Live"]
474
+ }
475
+ }
476
+ ```
477
+
478
+ ---
479
+
480
+ ### External Intent Filtering & Package Jailing (`packageJailGuard`)
481
+
482
+ Mobile applications often contain links that trigger external apps (e.g. "Rate Us" opening Google Play Store, "Terms of Service" opening Chrome, or external OAuth portals).
483
+
484
+ - **Proactive Intent Filter**: The crawler automatically detects and bypasses elements matching `EXTERNAL_INTENT_PATTERNS` (`Rate Us`, `Play Store`, `Privacy Policy`, `https://`, `share via`).
485
+ - **Package Jail Guard**: If an unclassified action causes the device to escape the target app into Chrome or Google Play, PromptTest detects the foreign package, executes `pressBack()`, and if needed, force re-launches the target app to restore containment immediately.
486
+
487
+ ---
488
+
489
+ ### Development Console & Overlay Auto-Triage
490
+
491
+ In React Native and hybrid apps, runtime warnings or unhandled promise rejections often pop up overlays (LogBox, RedBox, Flutter exception banners, Android ANR alerts) that block underlying UI buttons and stall ordinary automation:
492
+
493
+ 1. **Auto-Detection**: Scans the view hierarchy for LogBox overlays and ANR dialogs.
494
+ 2. **Stack Trace Extraction**: Captures the exact error message and logs it as a critical defect in the executive report.
495
+ 3. **Auto-Dismissal**: Taps the "Dismiss" or "Close" button so autonomous exploration can continue testing the rest of the application.
496
+
497
+ ---
498
+
499
+ ### Budgets & Performance Controls
500
+
501
+ ```bash
502
+ # Deep crawl: up to 50 unique screens, 5 levels deep, 100 interaction steps
503
+ npx prompttest explore host.exp.exponent --max-screens=50 --max-depth=5 --step-budget=100
504
+
505
+ # Fast crawl with lazy failure-only screenshots (slashes execution time by 3-5x)
506
+ npx prompttest explore host.exp.exponent --screenshots=failure-only
507
+ ```
508
+
509
+ | Flag | Default | Description |
510
+ | :--------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------ |
511
+ | `--max-screens=N` | `20` | Maximum unique screen states to index in the state graph. |
512
+ | `--max-depth=N` | `3` | Maximum recursion depth for nested screen diving. |
513
+ | `--step-budget=N` | `40` | Total interaction steps before completing exploration. |
514
+ | `--screenshots=<mode>` | `state-change` | Visual capture policy: `state-change` (only when screen updates), `failure-only` (only on errors), or `all` (every step). |
515
+
516
+ ---
517
+
518
+ ## 8. Self-Healing & Resilience Engine
519
+
520
+ PromptTest is built with multi-layered self-healing mechanisms:
521
+
522
+ 1. **Pre-Authentication Fast-Forwarding:**
523
+ - If a test starts with a login verification (`Verify 'Welcome Back'`), but the app is already authenticated on the dashboard, PromptTest scans ahead and **fast-forwards** past the redundant login steps automatically.
524
+ 2. **Cursor-Independent Input Flushing:**
525
+ - Uses `MOVE_END` + `DEL` + `MOVE_HOME` + `FORWARD_DEL` keycodes to reliably clear inputs regardless of initial cursor position or password visibility toggle icons.
526
+ 3. **Viewport Auto-Scroll:**
527
+ - Detects if an asserted element is clipped above or below screen boundaries and smoothly scrolls it into view.
528
+ 4. **Bottom Tab Bar Coordinate Normalization:**
529
+ - Automatically clamps vertical tap coordinates strictly within the element's actual physical bounds (`[bounds.y1 + 2, bounds.y2 - 2]`), preventing taps from landing outside the viewport or inadvertently triggering Android system gesture navigation bars.
530
+
531
+ ---
532
+
533
+ ## 9. Multi-Device Matrix & CI/CD Pipelines
534
+
535
+ ### Concurrent Multi-Device Execution
536
+
537
+ Test your app across multiple Android models, resolutions, and OS versions concurrently with a single command:
538
+
539
+ ```bash
540
+ npx prompttest run specs/smoke_suite.txt --all-devices
541
+ ```
542
+
543
+ ### GitHub Actions CI Workflow Setup
544
+
545
+ Run `npx prompttest init-ci` to automatically generate two production-ready GitHub Actions workflows under `.github/workflows/`:
546
+
547
+ - **`unit.yml`**: Headless Node.js 18 & 20 quality gates (Prettier check, ESLint, TypeScript typecheck, Vitest headless test suite with code coverage, and npm audit).
548
+ - **`device.yml`**: Android Emulator API 33 live spec execution with hardware KVM acceleration, automated JUnit XML publishing, and artifact capture.
549
+
550
+ ```bash
551
+ npx prompttest init-ci
552
+ ```
553
+
554
+ ### Central Multi-Device Report Aggregation & Atto Quality Scoring
555
+
556
+ When running tests across multiple physical devices or emulators using `--all-devices`, individual result JSON files (`*-results.json`) are produced per device. PromptTest can merge these results into a unified matrix dashboard:
557
+
558
+ ```bash
559
+ npx prompttest merge-reports output
560
+ ```
561
+
562
+ The generated `output/aggregated-dashboard.html` and `output/aggregated-results.json` feature:
563
+
564
+ - **Atto-Inspired Release Confidence Score (0–100%)**: An enterprise quality index calculated from weighted pass rates, critical-path assertion coverage, and device diversity.
565
+ - **Cross-Device Comparison Matrix**: Detailed side-by-side grid comparing execution status and timing for each step across every device model.
566
+ - **Consolidated Defect Summary**: Highlighting visual or functional regressions that appear on specific device form factors.
567
+
568
+ ---
569
+
570
+ ## 10. Advanced Features & SDK
571
+
572
+ ### Checkpointing & Resume
573
+
574
+ Long test suites can fail halfway due to network or environment issues. PromptTest automatically checkpoints progress after each successful step.
575
+ You can use the `--resume` flag to skip previously passed steps and continue execution from the exact point of failure:
576
+
577
+ ```bash
578
+ npx prompttest run specs/regression_suite.txt --resume
579
+ ```
580
+
581
+ ### Visual Baselines
582
+
583
+ PromptTest can capture visual baselines and perform perceptual diffing to detect unintended UI changes.
584
+
585
+ - `--save-baseline`: Saves the current screen layout as a visual baseline for future comparisons.
586
+ - `--compare-baseline`: Compares the current screen layout against the saved baseline and fails if differences exceed the threshold.
587
+ - `--baseline-threshold=N`: Adjusts the diffing tolerance as a ratio from `0` to `1` (e.g., `--baseline-threshold=0.05` for 5%).
588
+
589
+ ### Custom Reporters
590
+
591
+ You can integrate custom test reporters by supplying a path to your reporter module:
592
+
593
+ ```bash
594
+ npx prompttest run specs/suite.txt --reporter=./reporters/my-custom-reporter.js
595
+ ```
596
+
597
+ Custom reporters must implement the `Reporter` interface for pluggable report formats (e.g., Slack, DataDog, HTML).
598
+
599
+ ### APK Auto-Install
600
+
601
+ For CI environments, you can automatically install a specific APK before running tests:
602
+
603
+ ```bash
604
+ npx prompttest run specs/suite.txt --apk=./build/app-release.apk
605
+ ```
606
+
607
+ PromptTest will verify the device architecture and seamlessly deploy the build prior to execution.
608
+
609
+ ### Device Locking
610
+
611
+ PromptTest includes a cross-process, file-based mutex lock (`lib/lock.ts`) that guarantees concurrent safety. If multiple test processes try to control the same physical device or emulator simultaneously, they will wait in a queue until the lock is released.
612
+
613
+ ### Programmatic SDK & Cross-Platform Drivers
614
+
615
+ PromptTest exposes a clean programmatic API using factory functions and a unified `DriverInterface`. Both Android and iOS drivers share the identical driver abstraction:
616
+
617
+ ```typescript
618
+ import { createPromptRunner, createAndroidDriver, IosDriver } from 'prompttest';
619
+
620
+ async function runCrossPlatformTests() {
621
+ // 🤖 Android Execution
622
+ const androidDriver = createAndroidDriver('DEVICE_SERIAL');
623
+ const androidRunner = createPromptRunner(androidDriver);
624
+ const androidReport = await androidRunner.runSpec('specs/login.txt', 'com.example.app');
625
+
626
+ // 🍎 iOS Execution (Simulator or Physical Device)
627
+ const iosDriver = new IosDriver({
628
+ udid: '00008101-001234567890',
629
+ });
630
+ const iosRunner = createPromptRunner(iosDriver);
631
+ const iosReport = await iosRunner.runSpec('specs/login.txt', 'com.example.app');
632
+
633
+ console.log(`Android passed: ${androidReport.passed}, iOS passed: ${iosReport.passed}`);
634
+ }
635
+
636
+ runCrossPlatformTests();
637
+ ```
638
+
639
+ ### Cross-Platform Driver Architecture (`DriverInterface`)
640
+
641
+ PromptTest decouples test execution from OS-level automation protocols. Custom mobile environments (e.g. Flutter desktop, HarmonyOS, or cloud device farms) can implement the minimal `DriverInterface`:
642
+
643
+ - `getUiHierarchy(serial?)`: Returns the XML / accessibility view hierarchy.
644
+ - `tap(x, y, serial?)`: Dispatches a tap event at screen coordinates.
645
+ - `inputText(text, serial?)`: Inputs text into the active field.
646
+ - `pressKey(keyCode, serial?)`: Sends hardware key codes (e.g., BACK, ENTER).
647
+ - `takeScreenshot(outputPath, serial?)`: Captures screen image artifact.
648
+
649
+ ---
650
+
651
+ ## 11. Real-World Domain Recipes
652
+
653
+ ### EdTech & Coaching Portals
654
+
655
+ ```txt
656
+ # 1. Sign In as Faculty
657
+ 1. Verify 'Welcome Back' is visible
658
+ 2. Type 'ramesh@coachconnect.app' into 'Email Address'
659
+ 3. Type 'Teacher@1234' into 'Password'
660
+ 4. Tap 'Sign In'
661
+ 5. Wait 2s
662
+
663
+ # 2. Verify Batches & Class Schedulers
664
+ 6. Tap 'Batches'
665
+ 7. Verify 'Batches & Schedules' is visible
666
+ 8. Verify 'Grade 10 Mathematics' is visible
667
+ 9. Verify pattern '\d+\s*/\s*30' is visible
668
+
669
+ # 3. Attendance Hub
670
+ 10. Tap 'Attendance'
671
+ 11. Verify 'Attendance Hub' is visible
672
+ 12. Verify 'Rohan Gupta' is visible
673
+ 13. Verify pattern '\d+%' is visible
674
+ 14. Tap 'Export Report'
675
+ 15. Verify 'Export Attendance Report' is visible
676
+ 16. Press back
677
+
678
+ # 4. Sign Out
679
+ 17. Tap 'Profile'
680
+ 18. Tap 'Sign Out'
681
+ 19. Tap 'Sign Out'
682
+ 20. Verify 'Welcome Back' is visible
683
+ ```
684
+
685
+ ### E-Commerce & Food Delivery
686
+
687
+ ```txt
688
+ # 1. Search & Browse
689
+ 1. Type 'Wireless Headphones' into 'Search products...'
690
+ 2. Tap 1st 'Add to Cart'
691
+ 3. Tap 'Cart'
692
+ 4. Verify 'Shopping Cart' is visible
693
+ 5. Verify pattern 'Total:\s*₹\d+' is visible
694
+
695
+ # 2. Checkout & Address
696
+ 6. Tap 'Proceed to Checkout'
697
+ 7. Type '42 HSR Layout, Sector 2' into 'Delivery Address'
698
+ 8. Type '560102' into 'Pincode'
699
+ 9. Tap 'Cash on Delivery|UPI Payment'
700
+ 10. Tap 'Place Order'
701
+ 11. Verify 'Order Confirmed' is visible
702
+ ```
703
+
704
+ ### FinTech & Banking
705
+
706
+ ```txt
707
+ # 1. Biometric / PIN Auth
708
+ 1. Verify 'Secure Login' is visible
709
+ 2. Type '1234' into 'Enter 4-digit PIN'
710
+ 3. Tap 'Continue'
711
+
712
+ # 2. Balance & Transactions
713
+ 4. Verify 'Account Overview' is visible
714
+ 5. Verify pattern 'Balance:\s*₹[\d,]+(\.\d{2})?' is visible
715
+ 6. Tap 'Transaction History'
716
+ 7. Verify 'Recent Transactions' is visible
717
+ 8. Verify pattern 'TXN-\d{8}' is visible
718
+ ```
719
+
720
+ ---
721
+
722
+ ## 🎯 12. Mode Expectations, Capabilities & Limitations
723
+
724
+ > **Read first:** Full details in **[`docs/MODE_EXPECTATIONS.md`](MODE_EXPECTATIONS.md)**. This section is the honest contract for what each mode can and cannot do.
725
+
726
+ PromptTest is a **fast UI-tree automation engine over ADB**. It "sees" what `uiautomator dump` exposes — not pixels. This one fact drives every limit below.
727
+
728
+ ### Which mode should I use?
729
+
730
+ | Mode | Command | Best for | Not for |
731
+ | ----------------------------- | -------------------------------------- | ------------------------------------------------------------ | ----------------------------------- |
732
+ | **Instructed (`run`)** | `prompttest run specs/foo.txt <pkg>` | Regression gates, CI, verifying a known flow with assertions | Exploratory discovery |
733
+ | **Autonomous (`explore`)** | `prompttest explore <pkg>` | Unattended state-graph discovery and safety-guarded sweeps | Replacing asserted regression tests |
734
+ | **Hybrid (`record` → `run`)** | `prompttest record` → `prompttest run` | Turning a manual walkthrough into a repeatable script | — (review the generated spec first) |
735
+
736
+ ### What you should honestly expect
737
+
738
+ - ✅ **`run`**: executes plain-English steps, polls for elements, asserts visibility/state, self-heals broken locators, reuses offline memory to speed repeat runs. Deterministic per step, CI-friendly.
739
+ - ✅ **`explore`**: **State-Graph Crawler v3** — systematically traverses forms, toggles switches/checkboxes, inspects side navigation drawers, discovers off-screen content via vertical scrolling, dives into master-detail lists, and traverses bottom navigation tabs with sub-crawling. Protected by configurable safety blacklists, external package jailing, and dev overlay auto-dismissal.
740
+ - ⚠️ **Performance**: "sub-second" = per **step**, not per whole suite.
741
+
742
+ ### Autonomous `explore` — capabilities and architecture
743
+
744
+ - **State-Graph Crawling**: Uses BFS/DFS graph indexing via hierarchy hashing (`computeHierarchyHash`) and offline `MemoryEngine` caching to track visited screens, detect loops, and fast-forward past redundant paths.
745
+ - **Safety Policy Enforcement**: Hierarchical safety modes (`strict`, `moderate`, `interactive`, `disabled`) ensure dangerous buttons (delete account, checkout, logout) are blocked and reported in the audit hit-list.
746
+ - **Domain Containment**: `packageJailGuard` prevents the crawler from leaving the app when tapping external intent links (Rate Us, Store URLs, Web browsers).
747
+ - **Overlay Triage**: Automatically extracts errors from React Native LogBox, RedBox, Flutter exception views, and Android ANRs, logging them as defects and dismissing the overlay.
748
+ - **Zero third-party AI required**: 100% heuristic and offline execution using the in-repo `HeuristicResolver` + `MemoryEngine`. Runs in air-gapped CI environments without network latency.
749
+
750
+ ### Honest limitations (apply to every mode)
751
+
752
+ 1. **Accessibility-tree dependency.** An element with no `text`, `content-desc`, `resource-id`, or clickable bounds is invisible or only tappable "blind."
753
+ 2. **Games / canvas / fully-drawn content** may expose no hierarchy → PromptTest cannot inspect or assert it.
754
+ 3. **OS overlays & Secure/FLAG_SECURE screens** can block dumps/capture.
755
+ 4. **Determinism ceiling.** Timers, real services, and animations can cause flakiness; wait for a stable state in your spec.
756
+ 5. **Zero third-party AI required.** All capabilities — including autonomous crawling — work fully offline via heuristics + memory. LLM is an optional, gated enhancement, never a prerequisite.
757
+
758
+ ---
759
+
760
+ ## 💡 Support & Contribution
761
+
762
+ - **Community Repository**: [github.com/shriramsingh/prompttest-community](https://github.com/shriramsingh/prompttest-community)
763
+ - **Issues & Requests**: [File an Issue](https://github.com/shriramsingh/prompttest-community/issues)
764
+ - **License**: Business Source License 1.1 (BSL 1.1)