prompttest 1.3.1 → 1.3.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. package/README.md +196 -76
  2. package/dist/bin/prompttest.js +142 -142
  3. package/dist/index.js +122 -122
  4. package/dist/lib/recorder.d.ts +19 -3
  5. package/package.json +1 -2
package/README.md CHANGED
@@ -4,14 +4,26 @@
4
4
  > The zero-setup, zero-instrumentation alternative to Appium & Detox for React Native, Expo, Flutter, and Native Android.
5
5
 
6
6
  [![npm version](https://img.shields.io/npm/v/prompttest.svg?color=cb3837)](https://www.npmjs.com/package/prompttest)
7
- [![License: BSL 1.1](https://img.shields.io/badge/License-BSL%201.1-blue.svg)](LICENSE)
7
+ [![License: BSL 1.1](https://img.shields.io/badge/License-BSL%201.1-blue.svg)](#-license)
8
8
  [![Node.js](https://img.shields.io/badge/Node.js-18%2B-green.svg)](https://nodejs.org/)
9
9
  [![Android ADB](https://img.shields.io/badge/Android-ADB%20Native-orange.svg)](https://developer.android.com/tools/adb)
10
10
  [![TypeScript](https://img.shields.io/badge/TypeScript-5.7-blue.svg)](https://www.typescriptlang.org/)
11
11
 
12
12
  **PromptTest** enables QA engineers, mobile developers, and autonomous agents to test Android applications using **plain-English natural language scripts**. It completely bypasses heavy server frameworks (Appium, Selenium, WebDriver) in favor of high-speed native ADB streams, expectation-driven polling, and an autonomous state-graph exploration engine.
13
13
 
14
- 📖 **[Read the Full User Manual](docs/USER_MANUAL.md)** • 🏛 **[System Architecture](docs/ARCHITECTURE.md)** • 📜 **[License (BSL 1.1)](LICENSE)** • 🤝 **[Contributing](CONTRIBUTING.md)**
14
+ ---
15
+
16
+ ### 📑 Quick Navigation
17
+ - [🚀 Quick Start & CLI Workflows](#-quick-start--cli-workflows)
18
+ - [📂 Outputs, Reports & Screenshots](#-outputs-reports--screenshots)
19
+ - [✍️ Writing Plain-English Tests](#️-writing-plain-english-tests)
20
+ - [📖 Syntax Reference](#-syntax-reference)
21
+ - [🛠 CLI Command Reference](#-cli-command-reference)
22
+ - [🎯 Mode Expectations & Limits](#-mode-expectations--limits)
23
+ - [💻 Programmatic TypeScript SDK](#-programmatic-typescript-sdk)
24
+ - [🐳 Docker Deployment](#-docker-container-deployment)
25
+ - [📊 Real-Device Benchmarks](#-proven-real-device-benchmarks)
26
+ - [📄 License](#-license)
15
27
 
16
28
  ---
17
29
 
@@ -27,33 +39,120 @@
27
39
 
28
40
  ---
29
41
 
30
- ## 🚀 30-Second Quick Start
42
+ ## 🚀 Quick Start & CLI Workflows
31
43
 
32
44
  ### Prerequisites
33
45
  - **Node.js 18.0.0+**
34
46
  - **Android ADB** installed and accessible in your system `PATH`
35
- - An Android device or emulator with **USB Debugging** enabled
47
+ - An Android device (USB or Wi-Fi) or emulator with **USB Debugging** enabled
48
+
49
+ ### Install in Your Mobile Project
50
+ You can run PromptTest instantly with `npx` or install it as a dev dependency:
51
+ ```bash
52
+ # In your React Native, Expo, Flutter, or Android project:
53
+ npm install --save-dev prompttest
54
+ ```
55
+
56
+ ---
57
+
58
+ ### 5 Core CLI Workflows
36
59
 
37
- ### 1. Run Instant Diagnostics (Zero Install Needed)
38
- Verify your environment and connected devices with zero configuration:
60
+ #### 1. Environment Diagnostic Check
61
+ Before running tests, verify your ADB connectivity, connected devices, and permissions:
39
62
  ```bash
40
63
  npx prompttest doctor
41
64
  ```
42
65
 
43
- ### 2. Autonomous Exploration (0 Lines of Code)
44
- Autonomously crawl, discover navigation tabs, and stress-test your app:
66
+ #### 2. Zero-Code Autonomous Exploration (AI App Crawl)
67
+ Crawl bottom tabs, lists, and forms automatically to discover bugs, crashes, or React Native red-screens without writing a single line of test code:
45
68
  ```bash
46
- npx prompttest explore <your.app.package>
69
+ npx prompttest explore com.yourcompany.app
47
70
  ```
71
+ *Tip: `explore` is its own top-level command. Pass `--max-screens=30` or `--safety-mode=strict` to customize.*
48
72
 
49
- ### 3. Install in Your Project
50
- Add PromptTest as a development dependency in your mobile project:
73
+ #### 3. Interactive Record & Replay
74
+ Record your natural interactions on a physical phone directly into a reusable test spec:
51
75
  ```bash
52
- npm install --save-dev prompttest
76
+ npx prompttest record specs/login.txt
77
+ ```
78
+ - Tap buttons and type in input fields on your phone; PromptTest auto-generates conversational English test steps.
79
+ - **Append mode**: Add `--append` to add more steps to an existing spec without overwriting.
80
+ - **Safety backup**: If the target file already exists, PromptTest automatically creates a `.bak` backup before modifying.
81
+
82
+ #### 4. Run Plain-English Test Specs
83
+ Execute test specifications with fail-fast validation and locator self-healing:
84
+ ```bash
85
+ npx prompttest run specs/login.txt com.yourcompany.app
86
+ ```
87
+ Power flags:
88
+ - `--heal`: Automatically self-heal altered counts and dynamic locators.
89
+ - `--video`: Record an MP4 video of the execution session.
90
+ - `--screenshots`: Capture high-resolution visual evidence at every step.
91
+ - `--fresh`: Cold-restart the target app before execution begins.
92
+ - `--embed-screenshots`: Embed screenshots directly into a standalone, shareable HTML report.
93
+
94
+ #### 5. Interactive Live REPL Playground
95
+ Experiment with commands live in your terminal against your connected device:
96
+ ```bash
97
+ npx prompttest repl com.yourcompany.app
53
98
  ```
54
99
 
55
100
  ---
56
101
 
102
+ ### ⚠️ CLI Command Disambiguation Table
103
+
104
+ To avoid common syntax errors, remember that `explore`, `record`, and `repl` are **standalone commands**:
105
+
106
+ | Goal | ✅ Correct CLI Command | ❌ Common Mistake |
107
+ | :--- | :--- | :--- |
108
+ | **Autonomous App Crawling** | `npx prompttest explore <pkg>` | `npx prompttest run explore` |
109
+ | **Record from Phone Touches** | `npx prompttest record <spec.txt>` | `npx prompttest run record` |
110
+ | **Run Existing Spec File** | `npx prompttest run <spec.txt> [pkg]` | `npx prompttest <spec.txt>` |
111
+ | **Interactive Live Terminal** | `npx prompttest repl [pkg]` | `npx prompttest run repl` |
112
+ | **Diagnostic Health Check** | `npx prompttest doctor` | `npx prompttest run doctor` |
113
+
114
+ ---
115
+
116
+ ### 📦 Recommended `package.json` Scripts
117
+
118
+ Add these convenient shortcuts to your project's `package.json`:
119
+ ```json
120
+ "scripts": {
121
+ "test:mobile": "prompttest run specs/smoke.txt com.yourcompany.app --heal",
122
+ "test:explore": "prompttest explore com.yourcompany.app --max-screens=25",
123
+ "test:record": "prompttest record specs/new_flow.txt",
124
+ "test:doctor": "prompttest doctor"
125
+ }
126
+ ```
127
+
128
+ ---
129
+
130
+ ## 📂 Outputs, Reports & Screenshots
131
+
132
+ Whenever PromptTest runs (`run`, `explore`, or `record`), all outputs are automatically organized inside an **`output/`** folder at your project root:
133
+
134
+ ```
135
+ your-mobile-project/
136
+ ├── node_modules/
137
+ ├── specs/
138
+ │ └── login.txt
139
+ ├── output/ <-- 📂 Created automatically
140
+ │ ├── login-report.html <-- 🌐 Interactive visual HTML report
141
+ │ ├── login-junit.xml <-- 🤖 CI/CD JUnit test results
142
+ │ ├── login-report.md <-- 📝 Markdown summary for PR comments
143
+ │ ├── login-results.json <-- 📊 Structured raw JSON execution metrics
144
+ │ ├── step_1_tap_sign_in.png <-- 📸 High-res visual screenshots
145
+ │ ├── login-recording.mp4 <-- 🎥 Full MP4 video (when using --video)
146
+ │ └── screenshots/ <-- 📸 Screen transition photos from record sessions
147
+ ```
148
+
149
+ ### Viewing & Sharing Reports:
150
+ - **Interactive HTML Dashboard (`output/<spec>-report.html`)**: Double-click to open in any browser. Features step-by-step audit timelines, latency timings, and failure triage bundles.
151
+ - **CI/CD Integration (`output/<spec>-junit.xml`)**: Standard JUnit format natively recognized by GitHub Actions, GitLab CI, Jenkins, and CircleCI.
152
+ - **Zero-Dependency Sharing (`--embed-screenshots`)**: Generates a single standalone HTML report with all images inlined via Base64 data URIs. Email or Slack it directly to teammates without missing image links!
153
+
154
+ ---
155
+
57
156
  ## ✍️ Writing Plain-English Tests
58
157
 
59
158
  Test specifications are simple text files containing numbered steps written in conversational English.
@@ -61,7 +160,7 @@ Test specifications are simple text files containing numbered steps written in c
61
160
  ### Example Spec (`specs/login_and_dashboard.txt`)
62
161
 
63
162
  ```text
64
- # 1. Authentication
163
+ # 1. Authentication Flow
65
164
  1. Type 'ramesh@coachconnect.app' into 'Email Address'
66
165
  2. Type 'Teacher@1234' into 'Password'
67
166
  3. Tap 'Sign In'
@@ -72,7 +171,7 @@ Test specifications are simple text files containing numbered steps written in c
72
171
  6. Verify 'Dr. Ramesh Sharma' is visible
73
172
  7. Verify 'My Batches' is visible
74
173
 
75
- # 3. Batches Module
174
+ # 3. Batches Module & Navigation
76
175
  8. Tap 'Batches'
77
176
  9. Verify 'Batches & Schedules' is visible
78
177
  10. Tap 'Grade 10 Mathematics'
@@ -93,86 +192,107 @@ npx prompttest run specs/login_and_dashboard.txt com.yourcompany.app
93
192
 
94
193
  ---
95
194
 
96
- ## 🛠 Core Commands & Capabilities
195
+ ## 📖 Syntax Reference
196
+
197
+ | Category | Command Syntax | Description |
198
+ | :--- | :--- | :--- |
199
+ | **Tap / Click** | `Tap 'Sign In'` | Taps button, icon, link, or tab matching label or text. |
200
+ | **Input Fields** | `Type 'user@test.com' into 'Email'` | Auto-focuses field, clears existing text, and enters string safely. |
201
+ | **Assertions** | `Verify 'Dashboard' is visible` | Polls until the element appears on screen (sub-second resolution). |
202
+ | **Absence Check** | `Verify 'Loading...' is not visible` | Confirms an element, dialog, or spinner has dismissed. |
203
+ | **Spatial Tap** | `Tap 'Delete' next to 'Order #12'` | Disambiguates duplicate elements using directional proximity. |
204
+ | **Gestures** | `Scroll down`, `Scroll up`, `Swipe left` | Performs viewport-proportional touch flings. |
205
+ | **Hardware Keys** | `Press back`, `Press home` | Dispatches physical Android keycodes (`KEYCODE_BACK`, etc.). |
206
+ | **Delays** | `Wait 2s` or `Wait 1500ms` | Pauses execution for custom animation settling. |
207
+ | **Conditionals** | `Tap 'Dismiss' (if present)` | Executes step only if element exists, without failing the suite. |
208
+ | **Generators** | `$random.email`, `$date.now`, `$uuid` | Inlines dynamic synthetic data into input fields. |
209
+
210
+ ---
211
+
212
+ ## 🛠 CLI Command Reference
213
+
214
+ ### Primary Execution Modes
97
215
 
98
- ### 1. Spec Runner (`prompttest run`)
99
- Executes deterministic plain-English test specs with fail-fast execution and self-healing locators:
100
216
  ```bash
101
- # Basic run
102
- npx prompttest run specs/flow.txt com.example.app
217
+ # 1. Deterministic Spec Runner
218
+ npx prompttest run specs/flow.txt <package>
103
219
 
104
- # Cold-restart app before testing begins
105
- npx prompttest run specs/flow.txt com.example.app --fresh
220
+ # Cold-restart app before suite begins
221
+ npx prompttest run specs/flow.txt <package> --fresh
106
222
 
107
- # Capture full screenshots on every step
108
- npx prompttest run specs/flow.txt com.example.app --screenshots
223
+ # Capture high-resolution visual screenshots on every step
224
+ npx prompttest run specs/flow.txt <package> --screenshots
109
225
 
110
- # Dry-run validation (validates grammar without touching device)
226
+ # Dry-run validation (checks syntax without touching device)
111
227
  npx prompttest run specs/flow.txt --dry-run
112
228
 
113
229
  # Display formatted table of parsed steps
114
230
  npx prompttest run specs/flow.txt --list-steps
115
231
 
116
232
  # Run data-driven iterations
117
- npx prompttest run specs/flow.txt com.example.app --iterations 5
233
+ npx prompttest run specs/flow.txt <package> --iterations 5
118
234
 
119
- # Run against a managed device pool
120
- npx prompttest run specs/flow.txt com.example.app --device-pool emulator-5554,emulator-5556
235
+ # Run against a managed device pool with concurrency leasing
236
+ npx prompttest run specs/flow.txt <package> --device-pool emulator-5554,emulator-5556
121
237
 
122
238
  # Visual regression testing against gold baseline images
123
239
  npx prompttest run specs/flow.txt --save-baseline
124
240
  npx prompttest run specs/flow.txt --compare-baseline --baseline-threshold=0.02
125
241
  ```
126
242
 
127
- ### 2. Autonomous State-Graph Explorer (`prompttest explore`)
128
- Autonomous crawling engine with bottom-tab detection, smart form filling, and safety policies:
243
+ ### Autonomous State-Graph Explorer (`explore`)
244
+
129
245
  ```bash
130
- # Standard autonomous crawl (Strict safety mode)
131
- npx prompttest explore com.example.app
246
+ # Autonomous exploration with Strict safety policy
247
+ npx prompttest explore <package>
132
248
 
133
- # Set custom screen and step interaction budgets
134
- npx prompttest explore com.example.app --max-screens=30 --step-budget=60
249
+ # Set custom screen discovery and step interaction limits
250
+ npx prompttest explore <package> --max-screens=30 --step-budget=60
135
251
 
136
- # Add custom protected blacklist keywords
137
- npx prompttest explore com.example.app --safety-blacklist="Wipe,Revoke,Transfer"
252
+ # Protect custom sensitive action keywords from being clicked
253
+ npx prompttest explore <package> --safety-blacklist="Wipe,Revoke,Transfer"
138
254
  ```
139
255
 
140
- ### 3. Interactive Record & Replay (`prompttest record`)
141
- Record your real interactions on the phone and auto-generate clean test specs:
256
+ ### Interactive Record & Replay (`record`)
257
+
142
258
  ```bash
259
+ # Record gestures, taps, and inputs directly on device into a spec
143
260
  npx prompttest record specs/recorded_flow.txt
144
261
  ```
145
262
 
146
- ### 4. Device Management & Wi-Fi Debugging
147
- Connect to physical phones wirelessly without keeping USB cables attached:
263
+ ### Device Management & Wi-Fi Debugging
264
+
148
265
  ```bash
149
- # List connected devices and serials
266
+ # List all connected devices, emulators, and serial numbers
150
267
  npx prompttest devices
151
268
 
152
- # Connect to device wirelessly over Wi-Fi
269
+ # Connect to physical device wirelessly over Wi-Fi
153
270
  npx prompttest wifi 192.168.1.50
154
271
 
155
272
  # Inspect active screen hierarchy and detected components
156
273
  npx prompttest status
274
+
275
+ # Run comprehensive environment diagnostic check
276
+ npx prompttest doctor
277
+
278
+ # View offline learned component memory graph
279
+ npx prompttest memory <package>
157
280
  ```
158
281
 
159
282
  ---
160
283
 
161
- ## 📖 Plain-English Syntax Reference
284
+ ## 🎯 Mode Expectations & Limits
162
285
 
163
- | Interaction | Example Syntax | Description |
164
- | :--- | :--- | :--- |
165
- | **Tap / Click** | `Tap 'Sign In'` | Clicks buttons, icons, tabs, or links by text or accessibility label. |
166
- | **Text Input** | `Type 'test@example.com' into 'Email'` | Focuses input, clears existing text, and types value safely. |
167
- | **Assertions** | `Verify 'Dashboard' is visible` | Polls until the element appears on screen. |
168
- | **Negative Check** | `Verify 'Loading...' is not visible` | Confirms an element or spinner has dismissed. |
169
- | **Relative Tap** | `Tap 'Delete' next to 'Order #12'` | Disambiguates duplicate elements using spatial proximity. |
170
- | **Gestures** | `Scroll down`, `Scroll up`, `Swipe left` | Responsive proportional swipe gestures across the viewport. |
171
- | **System Keys** | `Press back`, `Press home` | Emits hardware key events (`KEYCODE_BACK`, etc.). |
172
- | **Pauses** | `Wait 2s` or `Wait 1500ms` | Pauses execution for custom animation delays. |
173
- | **Conditionals** | `Tap 'Dismiss' (if present)` | Executes step only if target element is visible without failing. |
174
-
175
- *For advanced syntax (loops, regex assertions, data-driven iterations, subflow `#include`), check the **[Comprehensive User Manual](docs/USER_MANUAL.md)**.*
286
+ To maintain honest expectations, here is what PromptTest excels at and its architectural boundaries:
287
+
288
+ | Capability | Supported? | Notes |
289
+ | :--- | :---: | :--- |
290
+ | **Standard UI Apps** | ✅ Full | React Native, Expo, Flutter, Native Views, Jetpack Compose. |
291
+ | **Zero Code Changes** | ✅ Full | Operates 100% via native ADB hierarchy stream (`uiautomator dump`). |
292
+ | **Self-Healing** | ✅ Full | Auto-resolves modified counts and labels dynamically. |
293
+ | **Bottom-Tab Discovery** | ✅ Full | Explores hubs, nested master-detail views, and backtracks cleanly. |
294
+ | **Game Engines & Canvas**| ⚠️ Limited | Fully custom OpenGL/Vulkan/Unity games lack accessible UI nodes. |
295
+ | **Biometrics / OS Dialogs**| ⚠️ Limited | System-level biometric prompts require hardware-level mocks. |
176
296
 
177
297
  ---
178
298
 
@@ -183,7 +303,7 @@ PromptTest exports a full programmatic SDK for embedding into Node.js test runne
183
303
  ```typescript
184
304
  import { createPromptRunner, createAndroidDriver } from 'prompttest';
185
305
 
186
- // Initialize native ADB driver for device
306
+ // Initialize native ADB driver for connected device
187
307
  const driver = createAndroidDriver('DEVICE_SERIAL');
188
308
  const runner = createPromptRunner(driver);
189
309
 
@@ -193,40 +313,40 @@ const report = await runner.runSpec('specs/login.txt', 'com.example.app', {
193
313
  screenshots: 'failure-only',
194
314
  });
195
315
 
196
- console.log(`Passed: ${report.passedSteps}/${report.totalSteps}`);
316
+ console.log(`Execution complete: ${report.passedSteps}/${report.totalSteps} passed.`);
197
317
  ```
198
318
 
199
319
  ---
200
320
 
201
- ## 📊 Proven Real-Device Benchmarks
202
-
203
- Tested and verified against live physical Android devices running complex multi-role enterprise apps:
204
-
205
- | Test Suite | Total Steps | Passed | Failed | Pass Rate | Execution Time |
206
- | :--- | :---: | :---: | :---: | :---: | :---: |
207
- | **CoachConnect Enterprise Suite** | 52 | 52 | 0 | **100%** | 47.8s |
208
- | **Govindam Multilingual Suite** | 8 | 8 | 0 | **100%** | 35.1s |
209
- | **Calculator Operations Suite** | 5 | 5 | 0 | **100%** | 19.6s |
210
- | **Automated Vitest Test Matrix** | 294 | 294 | 0 | **100%** | 14.8s |
211
-
212
321
  ## 🐳 Docker Container Deployment
213
322
 
214
323
  PromptTest is containerized with Android platform-tools and headless runtime support:
215
324
 
216
325
  ```bash
326
+ # Build container image
217
327
  docker build -t prompttest .
218
- docker run --rm --net=host -v $(pwd)/specs:/app/specs prompttest run specs/flow.txt com.example.app
328
+
329
+ # Execute test spec inside container sharing host ADB daemon
330
+ docker run --rm --net=host \
331
+ -v $(pwd)/specs:/app/specs \
332
+ -v $(pwd)/output:/app/output \
333
+ prompttest run specs/flow.txt com.example.app
219
334
  ```
220
335
 
221
336
  ---
222
337
 
223
- ## 📁 Packaged Artifacts & Documentation
338
+ ## 📊 Proven Real-Device Benchmarks
339
+
340
+ Tested and verified against live physical Android devices running complex multi-role enterprise apps:
341
+
342
+ | Test Suite | Total Steps | Passed | Failed | Pass Rate | Execution Time |
343
+ | :--- | :---: | :---: | :---: | :---: | :---: |
344
+ | **CoachConnect Enterprise Suite** | 52 | 52 | 0 | **100%** | 47.8s |
345
+ | **Govindam Multilingual Suite** | 8 | 8 | 0 | **100%** | 35.1s |
346
+ | **Calculator Operations Suite** | 5 | 5 | 0 | **100%** | 19.6s |
347
+ | **Automated Vitest Test Matrix** | 294 | 294 | 0 | **100%** | 14.8s |
224
348
 
225
- - **[docs/USER_MANUAL.md](docs/USER_MANUAL.md)**: Exhaustive reference manual covering all keywords, configurations, edge-case triage, and real-world recipes.
226
- - **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)**: Deep dive into the ADB streaming driver, state-graph crawler, and memory indexing engine.
227
- - **[LICENSE](LICENSE)**: Legal terms under the Business Source License (BSL 1.1).
228
- - **[CONTRIBUTING.md](CONTRIBUTING.md)**: Guidelines for contributing bug fixes and enhancements.
229
- - **[SECURITY.md](SECURITY.md)**: Security vulnerability disclosure policy.
349
+ *For deep architectural details and driver design, see `docs/ARCHITECTURE.md` included in the package.*
230
350
 
231
351
  ---
232
352
 
@@ -236,4 +356,4 @@ PromptTest is licensed under the **Business Source License 1.1 (BSL 1.1)**:
236
356
 
237
357
  - **Free for Community Use**: 100% free for educational use, individual developers, open-source projects, and startups/businesses with under $100,000 in annual revenue.
238
358
  - **Commercial License**: Required for organizations exceeding the annual revenue threshold or companies wrapping PromptTest into hosted cloud services.
239
- - See the full terms in **[LICENSE](LICENSE)**. For commercial licensing inquiries, contact `jairam.singh9@gmail.com`.
359
+ - Commercial inquiries and enterprise licensing: `jairam.singh9@gmail.com`.