assuremind 1.2.0 → 1.2.3

package/docs/CLI-REFERENCE.md

@@ -252,10 +252,11 @@ npx assuremind apply-healing --from results/healing/run-2026-03-23.json
 
 ## `validate`
 
-Validate the config file, environment variables, and test file structure.
+Check config health and consistency.
 
 ```bash
-npx assuremind validate
+npx assuremind validate                # check config
+npx assuremind validate --strict       # exit code 1 if errors exist
 ```
 
 Checks:
@@ -264,6 +265,21 @@ Checks:
 - All `*.test.json` files can be parsed
 - No duplicate suite or case names
 
+| Flag | Description |
+|------|------------|
+| `--strict` | Exit code 1 if any validation errors exist |
+
+---
+
+## `ci`
+
+CI/CD pipeline utilities.
+
+```bash
+npx assuremind ci gate               # evaluate quality gate for latest run
+npx assuremind ci report             # post PR comment for latest run
+```
+
 ---
 
 ## `doctor`
@@ -372,7 +388,7 @@ The Test Recorder is available through the Studio UI (Test Editor → Record but
 
 | Endpoint | Method | Description |
 |----------|--------|-------------|
-| `/api/recorder/start` | POST | Launch a headed browser and begin recording. Body: `{ url?: string }` |
+| `/api/recorder/start` | POST | Launch a headed browser and begin recording (UI & Audit suites only). Body: `{ url?: string }` |
 | `/api/recorder/stop` | POST | Stop recording and return all captured actions |
 | `/api/recorder/status` | GET | Check if a recording session is active |
 

package/docs/GETTING-STARTED.md

@@ -174,6 +174,10 @@ Start the Studio and use the built-in **Test Recorder** to create tests by click
 2. Click the red **Record** button in the step editor
 3. A headed Chromium browser opens your app — interact naturally:
    - Click buttons, fill forms, navigate pages
+   - Elements **inside iframes** are captured automatically — generates correct `frameLocator()` code
+   - Elements **inside shadow roots** (Web Components) are captured via `composedPath()` — generates `>>` pierce locators
+   - **JS alerts/confirms** are auto-accepted and a `page.once('dialog', ...)` handler is prepended to the triggering step
+   - **Keyboard** — Tab, Shift+Tab, arrow keys, Ctrl+A, Enter, Escape, and Space on buttons are all recorded
    - **Shift+Click** any element to assert it's visible (hard assertion — test stops on failure)
    - **Ctrl+Shift+Click** any element for a **soft assertion** (test continues, failures collected at end)
    - **Ctrl+Shift+U** to assert the current URL
@@ -181,7 +185,7 @@ Start the Studio and use the built-in **Test Recorder** to create tests by click
 4. Click **Stop Recording** — all actions become steps with pre-generated Playwright code
 5. Click **Run** to execute immediately
 
-**No AI calls, no API keys needed** — the recorder resolves locators against Playwright's accessibility tree in real time.
+**No AI calls, no API keys needed** — the recorder resolves locators against Playwright's accessibility tree in real time. The recorder is available for **UI** and **Audit** suites only (not API suites).
 
 ### Option C — CLI generate command
 
@@ -431,6 +435,43 @@ Most users **never need to touch RAG settings**. The Settings → RAG Memory car
 
 ---
 
+## Advanced Step Patterns
+
+The AI handles complex browser scenarios from plain English — no special syntax needed:
+
+| Scenario | Example instruction |
+|----------|-------------------|
+| **Accept a JS alert** | `Accept the alert dialog` |
+| **Dismiss a confirm popup** | `Dismiss the confirmation popup` |
+| **Enter text in a prompt** | `Enter "yes" in the prompt and accept` |
+| **Shadow DOM click** | `Click the submit button inside the my-form component` |
+| **Shadow DOM fill** | `Fill the email input inside custom-input with "user@example.com"` |
+| **Iframe click** | `Click the Submit button inside the payment iframe` |
+| **Iframe fill** | `Enter "4111..." in the card number field inside the checkout frame` |
+| **Press a key** | `Press Enter` / `Press Escape to close the modal` |
+| **Keyboard shortcut** | `Press Ctrl+A to select all` / `Press Shift+Tab` |
+| **Type via keyboard** | `Type "Hello World" using the keyboard` |
+
+### Stopping a run
+
+While a run is in progress in the Studio, a red **Stop** button appears in the execution header. Click it to immediately signal the server to stop — remaining test cases are skipped and the run exits cleanly.
+
+---
+
+## What's New — Enterprise Features
+
+Assuremind v1.3 adds 6 enterprise-grade features, all accessible from the Studio sidebar:
+
+| Feature | Studio Page | What it does |
+|---------|------------|-------------|
+| **Faker Data** | `/data-templates` | 100+ generators with multi-select — pick multiple tokens at once, inserted comma-separated, auto-refresh on every run |
+| **Visual Regression** | Settings + Reports | Configure in Settings → Visual Regression; review diffs and baselines in Reports tabs |
+| **CI/CD Integration** | `/ci-integration` | Quality gates, PR comments, Slack/Teams notifications |
+
+All features are configured from the Studio UI — no code, no config files, no terminal required.
+
+---
+
 ## 13. Next Steps
 
 | Topic | Where to look |

package/docs/STUDIO.md

@@ -150,6 +150,8 @@ The Test Recorder lets you create tests by interacting with your application in
 
 #### Recording a test
 
+> **Note:** The recorder is available for **UI** and **Audit** suites only. API suites do not show the Record button.
+
 1. Open a test case in the editor
 2. Click the red **Record** button (between "Add Step" and "Generate All")
 3. A headed Chromium browser opens your app's base URL
@@ -202,6 +204,8 @@ The recorder does **not** use simple CSS selectors or XPath. Instead, it:
    - CSS fallback (last resort)
 4. The first strategy that uniquely identifies the element is used
 
+For elements **inside a shadow root**, the recorder uses `composedPath()[0]` to pierce the shadow boundary and builds a `host >> inner` pierce path automatically — no special action needed.
+
 This produces the most resilient locators possible — the same quality as hand-written Playwright tests.
 
 #### What makes it stand out vs other recorders
@@ -210,7 +214,10 @@ This produces the most resilient locators possible — the same quality as hand-
 |---------|-------------|-------------------|---------------------|
 | Locator quality | CSS/XPath | Good | Best — 6 strategies, verified against live page |
 | Accessibility tree | No | Partial | Full — every locator checked via Playwright API |
-| **Iframe support** | Partial | Manual | **Auto** — detects iframes, generates `frameLocator()` code |
+| **Iframe support** | Partial | Manual | **Auto** — detects iframes, generates `frameLocator()` code, both click & fill |
+| **Shadow DOM** | No | Partial | **Auto** — `composedPath()` pierces shadow roots, generates `>>` locators |
+| **JS dialogs** | No | No | **Auto** — dialogs auto-accepted, `page.once('dialog', ...)` prepended to triggering step |
+| **Keyboard capture** | Partial | Partial | **Full** — Tab, Shift+Tab, arrows, Enter, Escape, Ctrl+A, Space |
 | Assertions | Manual | Manual | Shift+Click (hard), Ctrl+Shift+Click (soft), URL & title shortcuts |
 | Plain-English steps | No | No | Yes — human-readable instructions auto-generated |
 | Self-healing after | No | No | Yes — 5-level AI healing cascade |
@@ -228,21 +235,65 @@ This produces the most resilient locators possible — the same quality as hand-
 | Assertions are hard to write | Shift+Click (hard), Ctrl+Shift+Click (soft), Ctrl+Shift+U for URL, Ctrl+Shift+T for title |
 | Recorded tests are fragile | 6-strategy locator resolution + post-run 5-level self-healing |
 | Apps use iframes (SAP, Salesforce) | Auto-detects iframe context, generates `frameLocator()` chains |
+| Apps use Web Components / Shadow DOM | Automatic pierce-path detection using `composedPath()` + `>>` combinator |
+| App pops up a JS alert mid-recording | Dialog auto-accepted, `page.once('dialog', ...)` handler prepended to the triggering step |
+| Need to record Tab/arrow key navigation | Tab, Shift+Tab, arrow keys, Ctrl+A and Space captured automatically |
 
 #### Iframe support
 
-The recorder automatically handles **same-origin iframes** — common in enterprise apps like SAP, Salesforce, and embedded widgets:
+The recorder and AI code generator both fully support **same-origin iframes** — common in enterprise apps like SAP, Salesforce, and embedded widgets:
 
 - The capture script is injected into **all frames** (main page + every iframe), not just the top-level page
 - When an element is inside an iframe, the recorder detects the frame context and computes a selector for the iframe element (using `id`, `name`, `data-testid`, or `src`)
 - Locators are resolved against the correct frame using `page.frameLocator('...')` — producing code like:
   ```typescript
   await page.frameLocator('#content-frame').getByRole('button', { name: 'Submit' }).click();
+  await page.frameLocator('iframe[src*="payment"]').getByPlaceholder('Card number').fill('4111...');
   ```
+- Both **click** and **fill** (text input) actions work inside iframes
 - Dynamically added iframes are detected via `frameattached` events and injected automatically
 - The recording banner only appears in the main frame — iframes display cleanly
 - **Limitation:** Cross-origin iframes (different domain) cannot be instrumented due to browser security. The main frame and same-origin iframes work fully.
 
+#### Shadow DOM support
+
+The recorder automatically handles Web Components and any element nested inside a shadow root:
+
+- Uses `composedPath()[0]` on every event to get the real inner element, not the shadow host that the browser re-targets events to
+- Detects shadow root context via `getRootNode() instanceof ShadowRoot`
+- Builds a **pierce path** (`host-selector >> inner-selector`) and uses Playwright's `>>` combinator in generated code:
+  ```typescript
+  await page.locator('my-login-form >> button[type="submit"]').click();
+  await page.locator('custom-input >> input').fill('user@example.com');
+  ```
+- Works transparently — just interact with the element normally, the recorder handles the rest
+
+#### JavaScript alerts & dialogs
+
+When your app shows a JS `alert()`, `confirm()`, or `prompt()` during recording:
+
+- The dialog is **automatically accepted** so it doesn't block the session
+- A `page.once('dialog', dialog => dialog.accept())` handler is **prepended** to the triggering step's generated code, ensuring correct execution order during replay:
+  ```typescript
+  page.once('dialog', dialog => dialog.accept());
+  await page.getByRole('button', { name: 'Delete' }).click();
+  ```
+- The dialog type and message are stored on the step for reference
+
+#### Keyboard capture
+
+Beyond click and fill, the recorder captures keyboard interactions:
+
+| Key | When captured | Generated code |
+|-----|--------------|----------------|
+| **Enter** | While focused on a form field | `await page.getByLabel('...').press('Enter');` |
+| **Escape** | Any time | `await page.keyboard.press('Escape');` |
+| **Tab** | While focused on a form field | `await page.getByLabel('...').press('Tab');` |
+| **Shift+Tab** | While focused on a form field | `await page.getByLabel('...').press('Shift+Tab');` |
+| **Arrow keys** | When focused on a non-input element | `await page.keyboard.press('ArrowDown');` |
+| **Ctrl+A** | Any time | `await page.keyboard.press('Control+A');` |
+| **Space** | On a button or role=button element | `await page.getByRole('button', { name: '...' }).press(' ');` |
+
 #### Technical details
 
 - The recorder uses `context.exposeBinding()` for browser↔Node.js communication that survives page navigations — works across all frames
@@ -251,6 +302,67 @@ The recorder automatically handles **same-origin iframes** — common in enterpr
 - Recorded steps use the `recorder` strategy tag, distinguishing them from AI-generated code
 - All recorded steps are immediately usable — no "Generate Code" step required
 
+### Advanced Step Patterns
+
+The AI code generator handles several advanced browser scenarios automatically — just describe them in plain English.
+
+#### JavaScript Alerts & Dialogs
+
+Mention "alert", "confirm dialog", "popup", "click OK", or "dismiss" and the AI generates a `page.once('dialog', ...)` handler registered before the triggering action:
+
+| Instruction | Generated pattern |
+|-------------|------------------|
+| `Accept the alert dialog` | `page.once('dialog', d => d.accept())` |
+| `Dismiss the confirmation popup` | `page.once('dialog', d => d.dismiss())` |
+| `Enter "yes" in the prompt dialog` | `page.once('dialog', d => d.accept('yes'))` |
+| `Verify the alert says "Are you sure?" and accept it` | Handler that checks `dialog.message()` then accepts |
+
+#### Shadow DOM
+
+For elements inside a Web Component's shadow root, use language like "inside the `<component-name>` component" or "in the shadow root" — the AI uses Playwright's `>>` pierce combinator:
+
+```
+Click the submit button inside the my-form component
+Fill the email input inside custom-input
+```
+
+Generated code:
+```typescript
+await page.locator('my-form >> button[type="submit"]').click();
+await page.locator('custom-input >> input').fill('user@example.com');
+```
+
+#### iFrames — Click and Fill
+
+Both clicking and typing inside iframes work. Describe the action naturally, referencing the iframe if known:
+
+```
+Click the Submit button inside the payment iframe
+Enter "4111111111111111" in the card number field inside the checkout frame
+```
+
+Generated code:
+```typescript
+await page.frameLocator('iframe[src*="payment"]').getByRole('button', { name: 'Submit' }).click();
+await page.frameLocator('iframe[src*="checkout"]').getByPlaceholder('Card number').fill('4111111111111111');
+```
+
+#### Keyboard Actions
+
+Describe key presses directly — the AI uses `page.keyboard.press()` or `page.keyboard.type()`:
+
+| Instruction | Generated code |
+|-------------|----------------|
+| `Press Enter` | `await page.keyboard.press('Enter')` |
+| `Press Escape to close the modal` | `await page.keyboard.press('Escape')` |
+| `Press Tab to move to the next field` | `await page.keyboard.press('Tab')` |
+| `Press Ctrl+A to select all text` | `await page.keyboard.press('Control+A')` |
+| `Press Ctrl+Z to undo` | `await page.keyboard.press('Control+Z')` |
+| `Type "Hello World" using the keyboard` | `await page.keyboard.type('Hello World')` |
+| `Press Enter on the search field` | `await page.getByRole('textbox').press('Enter')` |
+
+---
+
 ### Generating Code
 
 After adding steps, click **Generate Code** (or **Generate** on an individual step) to have the AI write the Playwright code.
@@ -275,7 +387,7 @@ Deletions are permanent. There is no undo in the Studio — use Git to recover.
 
 ---
 
-## Story Generator
+## Story Generator (Smart Tests)
 
 Generate an entire test suite from a plain-English user story:
 
@@ -301,6 +413,10 @@ The AI creates:
 
 The generated files appear in the Test Editor immediately.
 
+### Jira Integration Warning
+
+If Jira credentials are not configured and you try to load a Jira issue, a **Configuration Required** modal appears. This modal includes a **"Don't show this again"** checkbox at the bottom — checking it and clicking Close suppresses the modal permanently for that integration (stored in browser localStorage). You can re-enable it by clearing site data or browser storage.
+
 ---
 
 ## Run Config
@@ -322,6 +438,10 @@ Configure and start a test run without the command line:
 
 A live progress panel shows test results as they complete. The WebSocket connection streams updates in real time — no page refresh needed.
 
+### Stopping a run
+
+While a run is in progress, a **Stop** button (red square icon) appears in the execution header. Clicking it sends an immediate stop signal to the server — the current step finishes, then all remaining test cases are skipped and the run exits cleanly. The UI transitions out of the running state automatically once the stop is confirmed.
+
 When the run finishes, you are offered a link to the full report.
 
 ### Device Emulation
@@ -574,13 +694,95 @@ Go to **Settings → RAG Memory** to configure:
 
 ---
 
+## Faker Data
+
+The Faker Data modal (accessible from the Test Editor) lets you browse 100+ dynamic data generators across 16 categories. Search or browse by category, then select one or more tokens — the modal supports **multi-select**, showing selected tokens as removable chips in a chip bar. Click **"Insert N tokens"** to insert all selected tokens comma-separated into your step (e.g. `{{FAKE_PERSON_FIRSTNAME}}, {{FAKE_PERSON_LASTNAME}}, {{FAKE_INTERNET_EMAIL}}`). This is especially useful for form-filling steps where you need first name, last name, email, and phone all at once. Tokens resolve to fresh random values on every test run.
+
+---
+
+## Visual Regression
+
+Visual regression is configured in **Settings** → Visual Regression and reviewed in **Reports** → Visual Diffs / Baselines tabs.
+
+**Configuration** (Settings → Visual Regression):
+| Setting | Description | Default |
+|---------|------------|---------|
+| Enabled | Master toggle for visual regression | Off |
+| Threshold | Pixel diff percentage to fail (0.1% - 10%) | 0.5% |
+| Full Page | Capture entire scrollable page | Off |
+| Mask Selectors | CSS selectors to mask (ignore dynamic content like timestamps) | Empty |
+
+**Per-step toggle** — In the Test Editor, click the **Eye icon** on any step to enable visual comparison for that step only. Only steps with the Eye icon active are compared during runs.
+
+**Reports → Visual Diffs tab** — after runs with visual checks:
+- Select a run from the dropdown
+- Summary: passed, failed, new baseline counts
+- Each diff shows three images: Baseline (expected), Actual (current), Diff (red highlights)
+- **Approve** — update baseline to the new screenshot (intentional change)
+- **Reject** — keep current baseline (it's a real bug, dev needs to fix)
+- **Approve All** — bulk approve all diffs in a run
+
+**Reports → Baselines tab** — gallery of all current baseline screenshots:
+- Organized by suite → case → step
+- Browser and viewport info per baseline
+- Delete individual or bulk-delete baselines
+
+**File structure:**
+```
+baselines/                     ← Committed to git
+├── login-suite/valid-login/
+│   ├── step-2-chromium-1920x1080.png
+│   └── step-2-firefox-768x1024.png
+results/visual-diffs/{runId}/  ← Gitignored, per-run
+├── step-2-baseline.png
+├── step-2-actual.png
+└── step-2-diff.png
+```
+
+---
+
+## CI/CD Integration
+
+The **CI/CD Integration** page (`/ci-integration`) configures pipeline behavior — quality gates, PR comments, and notifications.
+
+**Quality Gates** — define pass/fail thresholds:
+| Gate | Description | Default |
+|------|-----------|---------|
+| Min Pass Rate | Minimum percentage of tests that must pass | 95% |
+| Max Duration | Maximum run duration in seconds (0 = unlimited) | 0 |
+| Required Tags | Tags where ALL tests must pass (e.g., smoke, critical) | Empty |
+
+Quality gate history shows last 5 results as green/red badges.
+
+**PR Comments** — auto-post test results:
+- Supports GitHub (uses `GITHUB_TOKEN`) and GitLab (uses `GITLAB_TOKEN`)
+- Markdown table with pass rate, failed count, duration, quality gate status
+- Optional: healing summary, visual diff summary
+- Posted automatically when `--ci` flag is used
+
+**Notifications** — push results to team channels:
+| Channel | Configuration |
+|---------|--------------|
+| Slack | Webhook URL — sends block kit message with pass/fail, failures list, healing summary |
+| Microsoft Teams | Webhook URL — sends adaptive card |
+| Email | Recipient list (SMTP placeholder) |
+| Custom Webhook | Any URL + headers — sends full JSON run result |
+
+Each channel has trigger options: On every run, On failure only, On healing events.
+
+---
+
 ## WebSocket Live Updates
 
 The Studio uses a WebSocket connection (`ws://localhost:<port>/ws`) to receive live updates:
 
 | Event | Description |
 |-------|-------------|
+| `run:start` | A run has started — UI spinner activates |
+| `run:case` | A test case has started executing |
+| `run:step` | A step within a case has started |
 | `run:complete` | A run has finished — results available |
+| `run:stopped` | A run was stopped early via the Stop button |
 | `run:error` | A run failed to start |
 | `recorder:action` | A new action was recorded (click, fill, navigate, assert) |
 | `recorder:stopped` | Recording session ended |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "assuremind",
-  "version": "1.2.0",
+  "version": "1.2.3",
   "description": "AI-powered codeless UI & API automation framework",
   "author": "Deepak Hiremath",
   "license": "MIT",
@@ -67,6 +67,7 @@
   "dependencies": {
     "@anthropic-ai/sdk": "^0.30.0",
     "@aws-sdk/client-bedrock-runtime": "^3.679.0",
+    "@faker-js/faker": "^10.4.0",
     "@fastify/cors": "^11.2.0",
     "@fastify/static": "^9.0.0",
     "@fastify/websocket": "^11.2.0",
@@ -95,7 +96,9 @@
     "ora": "^8.0.1",
     "pino": "^9.4.0",
     "pino-pretty": "^11.2.0",
+    "pixelmatch": "^7.1.0",
     "playwright": "^1.47.0",
+    "pngjs": "^7.0.0",
     "simple-git": "^3.33.0",
     "uuid": "^10.0.0",
     "zod": "^3.23.0"
@@ -105,6 +108,7 @@
     "@types/js-yaml": "^4.0.9",
     "@types/lodash-es": "^4.17.0",
     "@types/node": "^22.7.0",
+    "@types/pngjs": "^6.0.5",
     "@types/uuid": "^10.0.0",
     "@types/ws": "^8.5.0",
     "@typescript-eslint/eslint-plugin": "^8.8.0",

package/templates/ASSUREMIND.md

@@ -112,6 +112,13 @@ npx assuremind doctor
 | Wait | `Wait for the loading spinner to disappear` |
 | Upload | `Upload the file "fixtures/data/avatar.png" to the photo field` |
 | Variable | `Enter "{{ADMIN_EMAIL}}" in the email field` |
+| **Accept JS alert** | `Accept the alert dialog` |
+| **Dismiss confirm popup** | `Dismiss the confirmation popup` |
+| **Shadow DOM** | `Click the submit button inside the my-form component` |
+| **iFrame click** | `Click the Submit button inside the payment iframe` |
+| **iFrame fill** | `Enter "4111..." in the card field inside the checkout iframe` |
+| **Key press** | `Press Enter` / `Press Escape` / `Press Ctrl+A` |
+| **Keyboard type** | `Type "Hello World" using the keyboard` |
 
 ---
 

package/templates/docs/GETTING-STARTED.md

@@ -216,6 +216,18 @@ A suite is a collection of related test cases. For example:
 - Assertions start with: `Verify`, `Assert`, `Check`, `Confirm`
 - Navigation: `Go to`, `Navigate to`, `Open`
 
+**Advanced patterns — the AI handles these automatically:**
+
+| Scenario | Example instruction |
+|----------|-------------------|
+| Accept JS alert | `Accept the alert dialog` |
+| Dismiss confirm popup | `Dismiss the confirmation popup` |
+| Shadow DOM | `Click the submit button inside the my-form component` |
+| iFrame click | `Click the Submit button inside the payment iframe` |
+| iFrame fill | `Enter "4111..." in the card field inside the checkout iframe` |
+| Key press / shortcut | `Press Enter` / `Press Ctrl+A to select all` |
+| Keyboard type | `Type "Hello World" using the keyboard` |
+
 ---
 
 ## 9. Generate Code with AI
@@ -250,6 +262,7 @@ You can also:
 **From the Studio:**
 - Open the case, click the **▶ Run** button in the toolbar
 - Watch real-time progress in the run panel — each step lights up green/red
+- To stop mid-run, click the red **Stop** button in the execution header — remaining cases are skipped cleanly
 
 **From the CLI:**
 ```bash

package/templates/docs/STUDIO.md

@@ -217,8 +217,14 @@ Each step row shows:
 | Wait | `Wait for the loading spinner to disappear` |
 | Hover | `Hover over the user avatar` |
 | Upload file | `Upload "fixtures/data/resume.pdf" to the file input` |
-| Press key | `Press the Escape key` |
+| Press key | `Press Enter` / `Press Escape` / `Press Ctrl+A` |
 | Scroll | `Scroll down to the footer` |
+| **Accept JS alert** | `Accept the alert dialog` |
+| **Dismiss confirm** | `Dismiss the confirmation popup` |
+| **Shadow DOM** | `Click the submit button inside the my-form component` |
+| **iFrame click** | `Click the Submit button inside the payment iframe` |
+| **iFrame fill** | `Enter "4111..." in the card field inside the checkout iframe` |
+| **Keyboard type** | `Type "Hello World" using the keyboard` |
 
 ### Editing a Step Instruction
 
@@ -310,6 +316,10 @@ A **Run Panel** appears at the bottom of the editor showing:
 | ⬜ | Step not yet reached |
 | ⚡ | Step was healed (self-healing kicked in) |
 
+### Stopping a Run
+
+While a run is active, a red **Stop** button (square icon) appears in the execution header bar. Click it to send an immediate stop signal — the current step completes, then all remaining cases are skipped and the run exits cleanly.
+
 ### After a Run
 
 - Click any **failed step** to see the error message and screenshot