prompttest 1.3.0 β†’ 1.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +151 -282
  2. package/package.json +1 -2
package/README.md CHANGED
@@ -1,71 +1,78 @@
1
1
  # ⚑ PromptTest
2
2
 
3
- > **Ultra-fast, zero-code autonomous mobile testing & visual QA brain for Android.**
3
+ > **Ultra-fast, zero-code autonomous mobile testing & visual QA brain for Android.**
4
+ > The zero-setup, zero-instrumentation alternative to Appium & Detox for React Native, Expo, Flutter, and Native Android.
4
5
 
5
- [![TypeScript](https://img.shields.io/badge/TypeScript-5.7-blue.svg)](https://www.typescriptlang.org/)
6
+ [![npm version](https://img.shields.io/npm/v/prompttest.svg?color=cb3837)](https://www.npmjs.com/package/prompttest)
7
+ [![License: BSL 1.1](https://img.shields.io/badge/License-BSL%201.1-blue.svg)](#-license)
6
8
  [![Node.js](https://img.shields.io/badge/Node.js-18%2B-green.svg)](https://nodejs.org/)
7
9
  [![Android ADB](https://img.shields.io/badge/Android-ADB%20Native-orange.svg)](https://developer.android.com/tools/adb)
8
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
10
+ [![TypeScript](https://img.shields.io/badge/TypeScript-5.7-blue.svg)](https://www.typescriptlang.org/)
9
11
 
10
- **PromptTest** allows QA engineers, developers, and autonomous agents to test Android applications using **plain-English natural language scripts**. It eliminates heavy frameworks like Appium, Selenium, or WebDriver in favor of high-performance native ADB streams, expectation-driven polling, and a self-learning autonomous memory engine.
12
+ **PromptTest** enables QA engineers, mobile developers, and autonomous agents to test Android applications using **plain-English natural language scripts**. It completely bypasses heavy server frameworks (Appium, Selenium, WebDriver) in favor of high-speed native ADB streams, expectation-driven polling, and an autonomous state-graph exploration engine.
11
13
 
12
- πŸ“š **[Read the Full User Manual (docs/USER_MANUAL.md)](docs/USER_MANUAL.md)** β€’ 🎯 **[Mode Expectations & Limits (docs/MODE_EXPECTATIONS.md)](docs/MODE_EXPECTATIONS.md)** β€’ 🌐 **[Interactive Web Manual (docs/index.html)](docs/index.html)** β€’ πŸ› **[System Architecture (docs/ARCHITECTURE.md)](docs/ARCHITECTURE.md)**
14
+ ---
13
15
 
14
- > ⚠️ **Honest expectations:** PromptTest is a fast UI‑tree automation engine over ADB. It excels on standard, accessibility‑exposed screens (React Native, Expo, Flutter, Native/Compose); fully‑drawn games/canvas and unlabeled icon‑only controls are platform‑limited. See **[docs/MODE_EXPECTATIONS.md](docs/MODE_EXPECTATIONS.md)** for exactly what each mode can and cannot do.
16
+ ### πŸ“‘ Quick Navigation
17
+ - [πŸš€ 30-Second Quick Start](#-30-second-quick-start)
18
+ - [✍️ Writing Plain-English Tests](#️-writing-plain-english-tests)
19
+ - [πŸ“– Syntax Reference](#-syntax-reference)
20
+ - [πŸ›  CLI Command Reference](#-cli-command-reference)
21
+ - [🎯 Mode Expectations & Limits](#-mode-expectations--limits)
22
+ - [πŸ’» Programmatic TypeScript SDK](#-programmatic-typescript-sdk)
23
+ - [🐳 Docker Deployment](#-docker-container-deployment)
24
+ - [πŸ“Š Real-Device Benchmarks](#-proven-real-device-benchmarks)
25
+ - [πŸ“„ License](#-license)
15
26
 
16
27
  ---
17
28
 
18
29
  ## 🌟 Why PromptTest?
19
30
 
20
- - πŸ“ **Plain-English Test Scripts**: Write tests in human language without writing selectors, XPath, or glue code.
21
- - ⚑ **Sub-Second per-step Execution**: Direct native ADB hierarchy streaming and dynamic expectation polling (no blind `sleep()` timeouts).
22
- - 🧠 **Autonomous Knowledge Engine**: Remembers element mappings, accessibility descriptors, and screen transitions to fast-forward future test runs.
23
- - 🌐 **Universal Across Frameworks**: Works on React Native, Expo, Flutter, Native Android (Kotlin/Java), and Jetpack Compose β€” for standard, accessibility-exposed UI (games/canvas are platform-limited).
24
- - πŸ›‘ **Smart Pre-Fail Triage**: Automatically resolves modal dialog collisions, auto-scrolls when elements are below the fold, and handles Android gesture insets.
25
- - πŸ“Έ **Failure-Only Blackbox Diagnostics**: Keeps test runs ultra-fast and lightweight while capturing high-resolution visual evidence and actionable diffs upon assertion failures.
31
+ - πŸ“ **Plain-English Test Scripts**: Write tests in human language without writing fragile XPath selectors or boilerplate glue code.
32
+ - ⚑ **Zero-Setup & Zero-Instrumentation**: Connects directly over native ADB (USB or Wi-Fi). No SDKs, test dependencies, or code modifications required in your target app.
33
+ - πŸ€– **Autonomous State-Graph DFS Explorer**: Crawls apps autonomously, detects bottom-tab navigation hubs, traverses nested screens, and automatically triages defects without human intervention.
34
+ - πŸ’‘ **Dynamic Self-Healing Locators**: Resilient heuristic matching automatically adapts to changing dynamic counts (e.g. auto-resolving `"Present (2)"` to `"Present (5)"`).
35
+ - πŸ›‘ **Built-In Safety Engine**: Guards production and staging apps by blocking destructive actions (e.g., `"Delete Account"`, `"Discard Changes"`) during autonomous crawls.
36
+ - πŸ“Š **Standalone HTML & JUnit Reports**: Generates executive test reports with failure-only visual screenshots and actionable error diffs.
37
+ - 🌐 **Built for Modern Mobile Frameworks**: Native support for **React Native**, **Expo**, **Flutter**, and Native Android (Jetpack Compose / Views).
26
38
 
27
39
  ---
28
40
 
29
- ## πŸš€ Quick Start
41
+ ## πŸš€ 30-Second Quick Start
30
42
 
31
43
  ### Prerequisites
44
+ - **Node.js 18.0.0+**
45
+ - **Android ADB** installed and accessible in your system `PATH`
46
+ - An Android device or emulator with **USB Debugging** enabled
32
47
 
33
- 1. **Node.js 18.0.0+** (Required for ES2022 features, native module resolution, and Unicode property escapes `\p{Extended_Pictographic}`).
34
- 2. **Android ADB** installed and accessible in your system `PATH`.
35
- 3. An Android device or emulator with **USB Debugging** enabled.
36
- 4. _Language Support_: The default seed dictionary covers English mobile UI interactions. Applications localized in other languages work via direct label matching or custom specifications.
37
-
38
- ### Installation
39
-
48
+ ### 1. Run Instant Diagnostics (Zero Install Needed)
49
+ Verify your environment and connected devices with zero configuration:
40
50
  ```bash
41
- # Clone the repository
42
- git clone https://github.com/<your-username>/prompttest.git
43
- cd prompttest
44
-
45
- # Install dependencies
46
- npm install
47
-
48
- # Build TypeScript
49
- npm run build
51
+ npx prompttest doctor
50
52
  ```
51
53
 
52
- ### Connect Your Device
54
+ ### 2. Autonomous Exploration (0 Lines of Code)
55
+ Autonomously crawl, discover navigation tabs, and stress-test your app:
56
+ ```bash
57
+ npx prompttest explore <your.app.package>
58
+ ```
53
59
 
60
+ ### 3. Install in Your Project
61
+ Add PromptTest as a development dependency in your mobile project:
54
62
  ```bash
55
- # Verify device connection
56
- npm run devices
63
+ npm install --save-dev prompttest
57
64
  ```
58
65
 
59
66
  ---
60
67
 
61
68
  ## ✍️ Writing Plain-English Tests
62
69
 
63
- Test specs are simple text files containing numbered steps written in conversational English.
70
+ Test specifications are simple text files containing numbered steps written in conversational English.
64
71
 
65
72
  ### Example Spec (`specs/login_and_dashboard.txt`)
66
73
 
67
74
  ```text
68
- # 1. Authentication
75
+ # 1. Authentication Flow
69
76
  1. Type 'ramesh@coachconnect.app' into 'Email Address'
70
77
  2. Type 'Teacher@1234' into 'Password'
71
78
  3. Tap 'Sign In'
@@ -76,7 +83,7 @@ Test specs are simple text files containing numbered steps written in conversati
76
83
  6. Verify 'Dr. Ramesh Sharma' is visible
77
84
  7. Verify 'My Batches' is visible
78
85
 
79
- # 3. Batches Module
86
+ # 3. Batches Module & Navigation
80
87
  8. Tap 'Batches'
81
88
  9. Verify 'Batches & Schedules' is visible
82
89
  10. Tap 'Grade 10 Mathematics'
@@ -90,313 +97,175 @@ Test specs are simple text files containing numbered steps written in conversati
90
97
  16. Verify 'Welcome Back' is visible
91
98
  ```
92
99
 
100
+ ### Run the Spec:
101
+ ```bash
102
+ npx prompttest run specs/login_and_dashboard.txt com.yourcompany.app
103
+ ```
104
+
93
105
  ---
94
106
 
95
- ## πŸ“– Plain-English & AST Syntax Reference
107
+ ## πŸ“– Syntax Reference
108
+
109
+ | Category | Command Syntax | Description |
110
+ | :--- | :--- | :--- |
111
+ | **Tap / Click** | `Tap 'Sign In'` | Taps button, icon, link, or tab matching label or text. |
112
+ | **Input Fields** | `Type 'user@test.com' into 'Email'` | Auto-focuses field, clears existing text, and enters string safely. |
113
+ | **Assertions** | `Verify 'Dashboard' is visible` | Polls until the element appears on screen (sub-second resolution). |
114
+ | **Absence Check** | `Verify 'Loading...' is not visible` | Confirms an element, dialog, or spinner has dismissed. |
115
+ | **Spatial Tap** | `Tap 'Delete' next to 'Order #12'` | Disambiguates duplicate elements using directional proximity. |
116
+ | **Gestures** | `Scroll down`, `Scroll up`, `Swipe left` | Performs viewport-proportional touch flings. |
117
+ | **Hardware Keys** | `Press back`, `Press home` | Dispatches physical Android keycodes (`KEYCODE_BACK`, etc.). |
118
+ | **Delays** | `Wait 2s` or `Wait 1500ms` | Pauses execution for custom animation settling. |
119
+ | **Conditionals** | `Tap 'Dismiss' (if present)` | Executes step only if element exists, without failing the suite. |
120
+ | **Generators** | `$random.email`, `$date.now`, `$uuid` | Inlines dynamic synthetic data into input fields. |
96
121
 
97
- ### Primitive Actions & UI Interactions
122
+ ---
98
123
 
99
- | Action | Example Syntax | Description |
100
- | :----------------- | :------------------------------------ | :---------------------------------------------------------------- |
101
- | **Type / Input** | `Type 'user@domain.com' into 'Email'` | Clears existing text and types the value into the matching input. |
102
- | **Tap / Click** | `Tap 'Sign In'` | Clicks button, tab, link, or icon with exact boundary resolution. |
103
- | **Long Press** | `Long press 'Student Card'` | Long presses an element for 1000ms. |
104
- | **Verify Visible** | `Verify 'Dashboard' is visible` | Polls until the text or accessibility label appears on screen. |
105
- | **Verify State** | `Verify 'Submit' is disabled` | Verifies element enablement state (`enabled` / `disabled`). |
106
- | **Press Back** | `Press back` | Emits Android system `KEYCODE_BACK`. |
107
- | **Scroll / Swipe** | `Scroll down` or `Scroll up` | Proportional responsive swipe along the screen viewport. |
108
- | **Relative Tap** | `Tap 'Delete' next to 'Rohan Gupta'` | Disambiguates duplicate elements using spatial proximity. |
109
- | **Wait** | `Wait 3s` or `Wait 1500ms` | Pauses execution for custom animation delays. |
124
+ ## πŸ›  CLI Command Reference
110
125
 
111
- ### Flow Control, Subflows & Programmatic Logic
126
+ ### Primary Execution Modes
112
127
 
113
- | Feature | Example Syntax | Description |
114
- | :------------------ | :----------------------------------------- | :----------------------------------------------------- |
115
- | **Conditional** | `IF 'Promo' is visible ... ELSE ... ENDIF` | Executes branch based on live element presence. |
116
- | **Loop** | `LOOP 3 ... ENDLOOP` | Repeats child step block N times. |
117
- | **Wait Until** | `WAIT_UNTIL 'Ready' is visible (10s)` | Explicit polling wait with max timeout budget. |
118
- | **Subflow Include** | `#include <specs/login.txt>` | Inlines modular specs with circular dependency guards. |
119
- | **Variables** | `SET $token = 'xyz'` | Sets local variable for interpolation (`$token`). |
120
- | **Sandboxed JS** | `RUN_JS return { status: 200 }` | Executes deterministic JavaScript expressions. |
128
+ ```bash
129
+ # 1. Deterministic Spec Runner
130
+ npx prompttest run specs/flow.txt <package>
121
131
 
122
- ### Data-Driven Testing & Dynamic Generators
132
+ # Cold-restart app before suite begins
133
+ npx prompttest run specs/flow.txt <package> --fresh
123
134
 
124
- Embed inline JSON datasets using `#!data` or load via CSV/JSON:
135
+ # Capture high-resolution visual screenshots on every step
136
+ npx prompttest run specs/flow.txt <package> --screenshots
125
137
 
126
- ```text
127
- #!data [
128
- {"user": "Alice", "email": "$random.email"},
129
- {"user": "Bob", "email": "$random.email"}
130
- ]
131
- 1. Type '$user' into 'Username'
132
- 2. Type '$email' into 'Email'
133
- ```
138
+ # Dry-run validation (checks syntax without touching device)
139
+ npx prompttest run specs/flow.txt --dry-run
134
140
 
135
- **Supported Generators**:
141
+ # Display formatted table of parsed steps
142
+ npx prompttest run specs/flow.txt --list-steps
136
143
 
137
- - Synthetic Data: `$random.firstName`, `$random.lastName`, `$random.email`, `$random.phone`, `$random.string(8)`
138
- - Sequences & UUIDs: `$seq.id`, `$uuid`
139
- - Date & Time: `$date.now`, `$date.iso`, `$date.format('YYYY-MM-DD')`
140
- - Math Evaluation: `$calc($count + 1)`
144
+ # Run data-driven iterations
145
+ npx prompttest run specs/flow.txt <package> --iterations 5
141
146
 
142
- ### Integration Assertions & Device State
147
+ # Run against a managed device pool with concurrency leasing
148
+ npx prompttest run specs/flow.txt <package> --device-pool emulator-5554,emulator-5556
143
149
 
144
- ```text
145
- # OTP & API Verification
146
- 1. CAPTURE OTP from SMS into $code
147
- 2. Type '$code' into 'Verification Code'
148
- 3. API GET 'https://api.example.com/user/profile' EXPECT status 200
149
- 4. VERIFY EMAIL to 'test@domain.com' subject 'Welcome'
150
-
151
- # Device Simulation
152
- 5. Set network to offline
153
- 6. Set battery level to 20
150
+ # Visual regression testing against gold baseline images
151
+ npx prompttest run specs/flow.txt --save-baseline
152
+ npx prompttest run specs/flow.txt --compare-baseline --baseline-threshold=0.02
154
153
  ```
155
154
 
156
- ---
157
-
158
- ## πŸ–₯ CLI Usage
159
-
160
- ### Run a Test Spec
155
+ ### Autonomous State-Graph Explorer (`explore`)
161
156
 
162
157
  ```bash
163
- # Run a test script against an app package
164
- npx prompttest run specs/my_suite.txt host.exp.exponent
158
+ # Autonomous exploration with Strict safety policy
159
+ npx prompttest explore <package>
165
160
 
166
- # Run with full visual screenshots on every step
167
- npx prompttest run specs/my_suite.txt host.exp.exponent --screenshots all
168
-
169
- # Run data-driven iterations
170
- npx prompttest run specs/checkout.txt host.exp.exponent --iterations 5 --only-row 2
161
+ # Set custom screen discovery and step interaction limits
162
+ npx prompttest explore <package> --max-screens=30 --step-budget=60
171
163
 
172
- # Run against a managed device pool with concurrency leasing
173
- npx prompttest run specs/suite.txt host.exp.exponent --device-pool emulator-5554,emulator-5556 --device-timeout-secs 30
164
+ # Protect custom sensitive action keywords from being clicked
165
+ npx prompttest explore <package> --safety-blacklist="Wipe,Revoke,Transfer"
174
166
  ```
175
167
 
176
- ### CLI Flags Reference
177
-
178
- | Flag | Description |
179
- | :--------------------------------------- | :-------------------------------------------------------------------------- |
180
- | `--fresh` | Force-stops and restarts target app before test begins. |
181
- | `--device-pool <d1,d2>` | Executes against the first available device from pool using atomic locking. |
182
- | `--device-timeout-secs <n>` | Maximum seconds to wait for a free device lock (default: 60s). |
183
- | `--iterations <n>` | Executes spec across N dynamic data iterations. |
184
- | `--only-row <n>` | Filters execution to a specific 1-indexed data row. |
185
- | `--continue` / `--continue-on-failure` | Continues running remaining steps after failure instead of fast-failing. |
186
- | `--screenshots <all\|failure\|none>` | Controls screenshot capture policy (default: `failure`). |
187
- | `--save-baseline` / `--update-baselines` | Saves gold reference baseline images for visual regression. |
188
- | `--compare-baseline` | Compares execution screens against reference baseline. |
189
- | `--baseline-threshold <n>` | Sets allowable perceptual diff tolerance (e.g. 0.02 = 2%). |
190
- | `--format <html,json,junit,markdown>` | Comma-separated output report formats. |
191
- | `-s <serial>` | Targets a specific connected device serial. |
192
- | `--version` / `-v` | Prints current PromptTest version. |
193
-
194
- ### Other Commands
168
+ ### Interactive Record & Replay (`record`)
195
169
 
196
170
  ```bash
197
- # Connect to a device wirelessly over Wi-Fi
198
- npx prompttest wifi 192.168.1.50
199
-
200
- # Disconnect wireless device
201
- npx prompttest disconnect
202
-
203
- # Interactive record-and-play session
204
- npx prompttest record specs/login.txt
205
-
206
- # Inspect foreground app and active components on screen
207
- npx prompttest status
208
-
209
- # Run comprehensive environment and ADB readiness diagnostics
210
- npx prompttest doctor
211
-
212
- # Start interactive live REPL sandbox for prototyping test steps
213
- npx prompttest repl
214
-
215
- # Merge multi-device test results and generate Atto-inspired release confidence dashboard
216
- npx prompttest merge-reports output
217
-
218
- # Automatically crawl and map the app (Autonomous State-Graph Crawler v3)
219
- npx prompttest explore host.exp.exponent
220
-
221
- # Run autonomous crawl with strict safety policy and custom domain blacklist
222
- npx prompttest explore host.exp.exponent --safety-mode=strict --safety-blacklist="Transfer,Revoke,Publish"
223
-
224
- # Deep exploration with custom screen/step budgets and lazy failure-only screenshots
225
- npx prompttest explore host.exp.exponent --max-screens=30 --step-budget=60 --screenshots=failure-only
226
-
227
- # View learned component aliases & confidence
228
- npx prompttest memory host.exp.exponent
229
-
230
- # Generate GitHub Actions CI workflow
231
- npx prompttest init-ci
171
+ # Record gestures, taps, and inputs directly on device into a spec
172
+ npx prompttest record specs/recorded_flow.txt
232
173
  ```
233
174
 
234
- ### Autonomous Exploration Flags (`explore`)
235
-
236
- | Flag | Default | Description | Example |
237
- | :------------------------- | :------------- | :----------------------------------------------------------------------------------------------------------------- | :--------------------------------- |
238
- | `--safety-mode=<mode>` | `strict` | Policy level: `strict` (all deletions blocked), `moderate` (drafts permitted), `interactive` (prompts), `disabled` | `--safety-mode=moderate` |
239
- | `--safety-blacklist="..."` | `[]` | Comma-separated custom keywords to protect as destructive actions | `--safety-blacklist="Revoke,Wipe"` |
240
- | `--max-screens=N` | `20` | Maximum unique screen states to discover and index | `--max-screens=50` |
241
- | `--max-depth=N` | `3` | Maximum recursion depth for nested screen diving | `--max-depth=5` |
242
- | `--step-budget=N` | `40` | Total interaction steps before completing exploration | `--step-budget=100` |
243
- | `--screenshots=<mode>` | `state-change` | Image capture policy: `state-change`, `failure-only`, or `all` | `--screenshots=failure-only` |
244
- | `-s <serial>` | auto | Target specific connected device serial | `-s GEVKDEUWOJWC89OR` |
245
-
246
- ### CLI Execution Flags (`run`)
247
-
248
- | Flag | Description | Example |
249
- | :--------------- | :----------------------------------------------------------- | :----------------------------------------------- |
250
- | `--dry-run` | Validates and parses spec steps with 0 device execution | `prompttest run specs/flow.txt --dry-run` |
251
- | `--list-steps` | Prints a formatted table of all parsed steps in the spec | `prompttest run specs/flow.txt --list-steps` |
252
- | `--only=<range>` | Executes only specific step numbers or ranges (e.g. `2-5,8`) | `prompttest run specs/flow.txt --only="1-3,5"` |
253
- | `--tags=<tags>` | Filters execution by tags declared via `# @tag: <name>` | `prompttest run specs/flow.txt --tags="smoke"` |
254
- | `--json` | Outputs machine-readable run summary JSON to stdout | `prompttest run specs/flow.txt --json` |
255
- | `--benchmark` | Profiles and logs execution duration for each step | `prompttest run specs/flow.txt --benchmark` |
256
- | `--watch` | Watches spec file for changes and re-executes automatically | `prompttest run specs/flow.txt --watch` |
257
- | `--fresh` | Force-stops and restarts target app before execution | `prompttest run specs/flow.txt --fresh` |
258
- | `--all-devices` | Concurrently executes spec across all connected devices | `prompttest run specs/flow.txt --all-devices` |
259
- | `--continue` | Continues executing remaining steps if a step fails | `prompttest run specs/flow.txt --continue` |
260
- | `--var:K=V` | Injects runtime variable overrides into the test context | `prompttest run specs/flow.txt --var:USER=Alice` |
175
+ ### Device Management & Wi-Fi Debugging
261
176
 
262
- ---
177
+ ```bash
178
+ # List all connected devices, emulators, and serial numbers
179
+ npx prompttest devices
263
180
 
264
- ## 🐳 Docker Container Deployment
181
+ # Connect to physical device wirelessly over Wi-Fi
182
+ npx prompttest wifi 192.168.1.50
265
183
 
266
- PromptTest is fully containerized with Android platform-tools and headless runtime support:
184
+ # Inspect active screen hierarchy and detected components
185
+ npx prompttest status
267
186
 
268
- ```bash
269
- # Build Docker image
270
- docker build -t prompttest .
187
+ # Run comprehensive environment diagnostic check
188
+ npx prompttest doctor
271
189
 
272
- # Run test spec inside container sharing host ADB daemon
273
- docker run --rm --net=host \
274
- -v $(pwd)/specs:/app/specs \
275
- -v $(pwd)/output:/app/output \
276
- prompttest run specs/my_suite.txt com.example.app
190
+ # View offline learned component memory graph
191
+ npx prompttest memory <package>
277
192
  ```
278
193
 
279
194
  ---
280
195
 
281
- ## πŸ— Architecture & Execution Pipeline
282
-
283
- ```mermaid
284
- flowchart TD
285
- A[Plain-English Spec .txt] --> B[DSL Parser & Tokenizer]
286
- B --> C[PromptRunner Engine]
287
- C --> D{Expectation Poller}
288
- D -->|Match Found| E[ADB Fast Native Driver]
289
- D -->|Not Found| F[Pre-Fail Triage / Auto-Scroll]
290
- F -->|Recovered| E
291
- F -->|Failed| G[Diagnostic Blackbox Snapshot]
292
- E --> H[Screen Execution & Settle]
293
- H --> I[Autonomous Knowledge Memory Engine]
294
- I --> J[Markdown & JSON QA Reports]
295
- ```
196
+ ## 🎯 Mode Expectations & Limits
296
197
 
297
- ### Key Components
198
+ To maintain honest expectations, here is what PromptTest excels at and its architectural boundaries:
298
199
 
299
- 1. **High-Performance ADB Driver (`lib/adb.ts`)**: Native process streaming for zero-latency screen hierarchy dumps and coordinate dispatching.
300
- 2. **Dynamic Expectation Poller (`lib/prompt-runner.ts`)**: Instant 0ms cache pass when screens match, dynamically polling for state transitions without rigid sleep constants.
301
- 3. **Autonomous Knowledge Engine (`lib/memory.ts`)**: Persists successful UI mappings into `memory/<package>.json` to accelerate future runs.
302
- 4. **Visual QA Sentinel (`lib/crawler.ts`)**: Scans for layout overflow, overlapping views, and unlabelled accessibility nodes.
200
+ | Capability | Supported? | Notes |
201
+ | :--- | :---: | :--- |
202
+ | **Standard UI Apps** | βœ… Full | React Native, Expo, Flutter, Native Views, Jetpack Compose. |
203
+ | **Zero Code Changes** | βœ… Full | Operates 100% via native ADB hierarchy stream (`uiautomator dump`). |
204
+ | **Self-Healing** | βœ… Full | Auto-resolves modified counts and labels dynamically. |
205
+ | **Bottom-Tab Discovery** | βœ… Full | Explores hubs, nested master-detail views, and backtracks cleanly. |
206
+ | **Game Engines & Canvas**| ⚠️ Limited | Fully custom OpenGL/Vulkan/Unity games lack accessible UI nodes. |
207
+ | **Biometrics / OS Dialogs**| ⚠️ Limited | System-level biometric prompts require hardware-level mocks. |
303
208
 
304
209
  ---
305
210
 
306
- ## πŸ“Š Real-World Verification Benchmarks
211
+ ## πŸ’» Programmatic TypeScript SDK
307
212
 
308
- Tested and verified against live physical Android devices running complex multi-role enterprise apps (React Native / Expo):
213
+ PromptTest exports a full programmatic SDK for embedding into Node.js test runners or custom CI scripts:
309
214
 
310
- | Test Suite | Total Steps | Passed | Failed | Pass Rate | Execution Time |
311
- | :--------------------- | :---------: | :----: | :----: | :-------: | :------------: |
312
- | **Teacher Role Suite** | 36 | 36 | 0 | **100%** | 64.0s |
313
- | **Student Role Suite** | 24 | 24 | 0 | **100%** | 44.6s |
314
- | **Total Benchmark** | **60** | **60** | **0** | **100%** | **108.6s** |
215
+ ```typescript
216
+ import { createPromptRunner, createAndroidDriver } from 'prompttest';
315
217
 
316
- ---
218
+ // Initialize native ADB driver for connected device
219
+ const driver = createAndroidDriver('DEVICE_SERIAL');
220
+ const runner = createPromptRunner(driver);
317
221
 
318
- ## πŸ“ Repository Structure
222
+ // Execute spec and retrieve structured execution report
223
+ const report = await runner.runSpec('specs/login.txt', 'com.example.app', {
224
+ fresh: true,
225
+ screenshots: 'failure-only',
226
+ });
319
227
 
320
- ```text
321
- prompttest/
322
- β”œβ”€β”€ bin/
323
- β”‚ └── prompttest.ts # CLI Entrypoint & Argument Router
324
- β”œβ”€β”€ constants/
325
- β”‚ └── commands.ts # ADB Commands & Keycode Constants
326
- β”œβ”€β”€ docs/
327
- β”‚ β”œβ”€β”€ ACTION_ITEMS.md # Working fix backlog (Phase 23 findings)
328
- β”‚ β”œβ”€β”€ MODE_EXPECTATIONS.md # Realistic capabilities, limits & boundaries per mode
329
- β”‚ β”œβ”€β”€ ARCHITECTURE.md # System Architecture & Design
330
- β”‚ β”œβ”€β”€ USER_MANUAL.md # Comprehensive User Guide
331
- β”‚ β”œβ”€β”€ adr/ # Architecture Decision Records
332
- β”‚ └── IMPLEMENTATION_PLAN.md
333
- β”œβ”€β”€ lib/
334
- β”‚ β”œβ”€β”€ adb.ts # ADB Interface & Streaming Driver
335
- β”‚ β”œβ”€β”€ baseline.ts # Visual Baseline Diffing & Masking
336
- β”‚ β”œβ”€β”€ checkpoint.ts # Test Progress Checkpointing
337
- β”‚ β”œβ”€β”€ config.ts # Configuration Constants
338
- β”‚ β”œβ”€β”€ crawler.ts # Visual Defect Detection & Screen Crawling
339
- β”‚ β”œβ”€β”€ data-loader.ts # Data-Driven Iterations & Dynamic Generators
340
- β”‚ β”œβ”€β”€ dictionary.ts # UI Taxonomy & Intent Dictionary
341
- β”‚ β”œβ”€β”€ explorer.ts # Autonomous Exploration Graph
342
- β”‚ β”œβ”€β”€ form-filler.ts # Smart Form Filling Engine
343
- β”‚ β”œβ”€β”€ live-monitor.ts # Real-Time Event Streaming Server
344
- β”‚ β”œβ”€β”€ lock.ts # File-Locking Device Leasing Mutex
345
- β”‚ β”œβ”€β”€ logger.ts # Structured Logging
346
- β”‚ β”œβ”€β”€ memory.ts # Persistent Knowledge & Fast-Forward Engine
347
- β”‚ β”œβ”€β”€ patterns.ts # Plain-English Grammar Parser
348
- β”‚ β”œβ”€β”€ prompt-runner.ts # Orchestration Engine (<1000 lines)
349
- β”‚ β”œβ”€β”€ recorder.ts # Autonomous Record & Replay
350
- β”‚ β”œβ”€β”€ reporter.ts # Multi-Format Report Generator (HTML/JSON/JUnit)
351
- β”‚ β”œβ”€β”€ runner-utils.ts # AST Parsing, Modular Includes, Diagnostic Errors
352
- β”‚ └── step-handlers.ts # Decoupled Step Execution Handlers
353
- β”œβ”€β”€ memory/ # Autonomous Learned Knowledge Mappings
354
- β”œβ”€β”€ output/ # Generated Test Reports & Artifacts
355
- β”œβ”€β”€ specs/ # Plain-English Test Specification Files
356
- β”œβ”€β”€ tests/ # Vitest Automated Test Suites
357
- β”œβ”€β”€ index.ts # Programmatic API Entrypoint
358
- β”œβ”€β”€ package.json
359
- β”œβ”€β”€ tsconfig.json
360
- β”œβ”€β”€ CONTRIBUTING.md
361
- β”œβ”€β”€ LICENSE
362
- └── README.md
228
+ console.log(`Execution complete: ${report.passedSteps}/${report.totalSteps} passed.`);
363
229
  ```
364
230
 
365
231
  ---
366
232
 
367
- ## πŸ’» Programmatic SDK & Cross-Platform Drivers
368
-
369
- You can use PromptTest programmatically within your own Node.js scripts using the unified `DriverInterface`:
233
+ ## 🐳 Docker Container Deployment
370
234
 
371
- ```typescript
372
- import { createPromptRunner, createAndroidDriver, createIosDriver } from 'prompttest';
235
+ PromptTest is containerized with Android platform-tools and headless runtime support:
373
236
 
374
- // πŸ€– Android Native Execution
375
- const androidDriver = createAndroidDriver('DEVICE_SERIAL');
376
- const androidRunner = createPromptRunner(androidDriver);
377
- const androidReport = await androidRunner.runSpec('specs/login.txt', 'com.example.app');
237
+ ```bash
238
+ # Build container image
239
+ docker build -t prompttest .
378
240
 
379
- // 🍎 iOS Execution (Simulator or Physical Device)
380
- const iosDriver = createIosDriver({ udid: 'SIMULATOR_UDID', bundleId: 'com.example.app' });
381
- const iosRunner = createPromptRunner(iosDriver);
382
- const iosReport = await iosRunner.runSpec('specs/login.txt', 'com.example.app');
241
+ # Execute test spec inside container sharing host ADB daemon
242
+ docker run --rm --net=host \
243
+ -v $(pwd)/specs:/app/specs \
244
+ -v $(pwd)/output:/app/output \
245
+ prompttest run specs/flow.txt com.example.app
383
246
  ```
384
247
 
385
248
  ---
386
249
 
387
- ## βš™οΈ Environment Variables
388
-
389
- - `LOG_LEVEL`: Controls the verbosity of logs. Set to `DEBUG`, `INFO`, `WARN`, `ERROR`, or `SILENT`.
390
- - `PROMPTTEST_DATA`: Overrides the default directory for PromptTest artifacts, memory, and outputs.
250
+ ## πŸ“Š Proven Real-Device Benchmarks
391
251
 
392
- ---
252
+ Tested and verified against live physical Android devices running complex multi-role enterprise apps:
393
253
 
394
- ## 🀝 Contributing
254
+ | Test Suite | Total Steps | Passed | Failed | Pass Rate | Execution Time |
255
+ | :--- | :---: | :---: | :---: | :---: | :---: |
256
+ | **CoachConnect Enterprise Suite** | 52 | 52 | 0 | **100%** | 47.8s |
257
+ | **Govindam Multilingual Suite** | 8 | 8 | 0 | **100%** | 35.1s |
258
+ | **Calculator Operations Suite** | 5 | 5 | 0 | **100%** | 19.6s |
259
+ | **Automated Vitest Test Matrix** | 294 | 294 | 0 | **100%** | 14.8s |
395
260
 
396
- Contributions, bug reports, and feature requests are welcome! Please check out [CONTRIBUTING.md](CONTRIBUTING.md) to get started.
261
+ *For deep architectural details and driver design, see `docs/ARCHITECTURE.md` included in the package.*
397
262
 
398
263
  ---
399
264
 
400
265
  ## πŸ“„ License
401
266
 
402
- This project is open source and available under the [MIT License](LICENSE).
267
+ PromptTest is licensed under the **Business Source License 1.1 (BSL 1.1)**:
268
+
269
+ - **Free for Community Use**: 100% free for educational use, individual developers, open-source projects, and startups/businesses with under $100,000 in annual revenue.
270
+ - **Commercial License**: Required for organizations exceeding the annual revenue threshold or companies wrapping PromptTest into hosted cloud services.
271
+ - Commercial inquiries and enterprise licensing: `jairam.singh9@gmail.com`.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "prompttest",
3
- "version": "1.3.0",
3
+ "version": "1.3.2",
4
4
  "description": "Ultra-fast, zero-code autonomous mobile testing & visual QA brain for Android",
5
5
  "license": "SEE LICENSE IN LICENSE",
6
6
  "type": "module",
@@ -49,7 +49,6 @@
49
49
  "prepublishOnly": "npm run typecheck && npm run build",
50
50
  "pack:check": "npm pack --dry-run",
51
51
  "preversion": "npm run typecheck && npm test",
52
- "postversion": "git push && git push --tags",
53
52
  "version:patch": "npm version patch",
54
53
  "version:minor": "npm version minor",
55
54
  "version:major": "npm version major"