assuremind 1.2.1 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +133 -9
- package/dist/cli/index.js +999 -17105
- package/dist/index.d.mts +27 -1
- package/dist/index.d.ts +27 -1
- package/dist/index.js +56 -1709
- package/dist/index.mjs +56 -1606
- package/docs/CLI-REFERENCE.md +19 -3
- package/docs/GETTING-STARTED.md +42 -1
- package/docs/STUDIO.md +205 -3
- package/package.json +9 -5
- package/templates/ASSUREMIND.md +7 -0
- package/templates/docs/GETTING-STARTED.md +13 -0
- package/templates/docs/STUDIO.md +11 -1
- package/ui/dist/assets/index-Bkr40QZS.js +942 -0
- package/ui/dist/assets/index-Bz5b82ao.css +1 -0
- package/ui/dist/index.html +2 -2
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
- package/dist/index.mjs.map +0 -1
- package/ui/dist/assets/index-DTtYd1hD.js +0 -837
- package/ui/dist/assets/index-lOAh29q9.css +0 -1
package/docs/CLI-REFERENCE.md
CHANGED
|
@@ -252,10 +252,11 @@ npx assuremind apply-healing --from results/healing/run-2026-03-23.json
|
|
|
252
252
|
|
|
253
253
|
## `validate`
|
|
254
254
|
|
|
255
|
-
|
|
255
|
+
Check config health and consistency.
|
|
256
256
|
|
|
257
257
|
```bash
|
|
258
|
-
npx assuremind validate
|
|
258
|
+
npx assuremind validate # check config
|
|
259
|
+
npx assuremind validate --strict # exit code 1 if errors exist
|
|
259
260
|
```
|
|
260
261
|
|
|
261
262
|
Checks:
|
|
@@ -264,6 +265,21 @@ Checks:
|
|
|
264
265
|
- All `*.test.json` files can be parsed
|
|
265
266
|
- No duplicate suite or case names
|
|
266
267
|
|
|
268
|
+
| Flag | Description |
|
|
269
|
+
|------|------------|
|
|
270
|
+
| `--strict` | Exit code 1 if any validation errors exist |
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
## `ci`
|
|
275
|
+
|
|
276
|
+
CI/CD pipeline utilities.
|
|
277
|
+
|
|
278
|
+
```bash
|
|
279
|
+
npx assuremind ci gate # evaluate quality gate for latest run
|
|
280
|
+
npx assuremind ci report # post PR comment for latest run
|
|
281
|
+
```
|
|
282
|
+
|
|
267
283
|
---
|
|
268
284
|
|
|
269
285
|
## `doctor`
|
|
@@ -372,7 +388,7 @@ The Test Recorder is available through the Studio UI (Test Editor → Record but
|
|
|
372
388
|
|
|
373
389
|
| Endpoint | Method | Description |
|
|
374
390
|
|----------|--------|-------------|
|
|
375
|
-
| `/api/recorder/start` | POST | Launch a headed browser and begin recording. Body: `{ url?: string }` |
|
|
391
|
+
| `/api/recorder/start` | POST | Launch a headed browser and begin recording (UI & Audit suites only). Body: `{ url?: string }` |
|
|
376
392
|
| `/api/recorder/stop` | POST | Stop recording and return all captured actions |
|
|
377
393
|
| `/api/recorder/status` | GET | Check if a recording session is active |
|
|
378
394
|
|
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -174,6 +174,10 @@ Start the Studio and use the built-in **Test Recorder** to create tests by click
|
|
|
174
174
|
2. Click the red **Record** button in the step editor
|
|
175
175
|
3. A headed Chromium browser opens your app — interact naturally:
|
|
176
176
|
- Click buttons, fill forms, navigate pages
|
|
177
|
+
- Elements **inside iframes** are captured automatically — generates correct `frameLocator()` code
|
|
178
|
+
- Elements **inside shadow roots** (Web Components) are captured via `composedPath()` — generates `>>` pierce locators
|
|
179
|
+
- **JS alerts/confirms** are auto-accepted and a `page.once('dialog', ...)` handler is prepended to the triggering step
|
|
180
|
+
- **Keyboard** — Tab, Shift+Tab, arrow keys, Ctrl+A, Enter, Escape, and Space on buttons are all recorded
|
|
177
181
|
- **Shift+Click** any element to assert it's visible (hard assertion — test stops on failure)
|
|
178
182
|
- **Ctrl+Shift+Click** any element for a **soft assertion** (test continues, failures collected at end)
|
|
179
183
|
- **Ctrl+Shift+U** to assert the current URL
|
|
@@ -181,7 +185,7 @@ Start the Studio and use the built-in **Test Recorder** to create tests by click
|
|
|
181
185
|
4. Click **Stop Recording** — all actions become steps with pre-generated Playwright code
|
|
182
186
|
5. Click **Run** to execute immediately
|
|
183
187
|
|
|
184
|
-
**No AI calls, no API keys needed** — the recorder resolves locators against Playwright's accessibility tree in real time.
|
|
188
|
+
**No AI calls, no API keys needed** — the recorder resolves locators against Playwright's accessibility tree in real time. The recorder is available for **UI** and **Audit** suites only (not API suites).
|
|
185
189
|
|
|
186
190
|
### Option C — CLI generate command
|
|
187
191
|
|
|
@@ -431,6 +435,43 @@ Most users **never need to touch RAG settings**. The Settings → RAG Memory car
|
|
|
431
435
|
|
|
432
436
|
---
|
|
433
437
|
|
|
438
|
+
## Advanced Step Patterns
|
|
439
|
+
|
|
440
|
+
The AI handles complex browser scenarios from plain English — no special syntax needed:
|
|
441
|
+
|
|
442
|
+
| Scenario | Example instruction |
|
|
443
|
+
|----------|-------------------|
|
|
444
|
+
| **Accept a JS alert** | `Accept the alert dialog` |
|
|
445
|
+
| **Dismiss a confirm popup** | `Dismiss the confirmation popup` |
|
|
446
|
+
| **Enter text in a prompt** | `Enter "yes" in the prompt and accept` |
|
|
447
|
+
| **Shadow DOM click** | `Click the submit button inside the my-form component` |
|
|
448
|
+
| **Shadow DOM fill** | `Fill the email input inside custom-input with "user@example.com"` |
|
|
449
|
+
| **Iframe click** | `Click the Submit button inside the payment iframe` |
|
|
450
|
+
| **Iframe fill** | `Enter "4111..." in the card number field inside the checkout frame` |
|
|
451
|
+
| **Press a key** | `Press Enter` / `Press Escape to close the modal` |
|
|
452
|
+
| **Keyboard shortcut** | `Press Ctrl+A to select all` / `Press Shift+Tab` |
|
|
453
|
+
| **Type via keyboard** | `Type "Hello World" using the keyboard` |
|
|
454
|
+
|
|
455
|
+
### Stopping a run
|
|
456
|
+
|
|
457
|
+
While a run is in progress in the Studio, a red **Stop** button appears in the execution header. Click it to immediately signal the server to stop — remaining test cases are skipped and the run exits cleanly.
|
|
458
|
+
|
|
459
|
+
---
|
|
460
|
+
|
|
461
|
+
## What's New — Enterprise Features
|
|
462
|
+
|
|
463
|
+
Assuremind v1.3 adds 6 enterprise-grade features, all accessible from the Studio sidebar:
|
|
464
|
+
|
|
465
|
+
| Feature | Studio Page | What it does |
|
|
466
|
+
|---------|------------|-------------|
|
|
467
|
+
| **Faker Data** | `/data-templates` | 100+ generators with multi-select — pick multiple tokens at once, inserted comma-separated, auto-refresh on every run |
|
|
468
|
+
| **Visual Regression** | Settings + Reports | Configure in Settings → Visual Regression; review diffs and baselines in Reports tabs |
|
|
469
|
+
| **CI/CD Integration** | `/ci-integration` | Quality gates, PR comments, Slack/Teams notifications |
|
|
470
|
+
|
|
471
|
+
All features are configured from the Studio UI — no code, no config files, no terminal required.
|
|
472
|
+
|
|
473
|
+
---
|
|
474
|
+
|
|
434
475
|
## 13. Next Steps
|
|
435
476
|
|
|
436
477
|
| Topic | Where to look |
|
package/docs/STUDIO.md
CHANGED
|
@@ -150,6 +150,8 @@ The Test Recorder lets you create tests by interacting with your application in
|
|
|
150
150
|
|
|
151
151
|
#### Recording a test
|
|
152
152
|
|
|
153
|
+
> **Note:** The recorder is available for **UI** and **Audit** suites only. API suites do not show the Record button.
|
|
154
|
+
|
|
153
155
|
1. Open a test case in the editor
|
|
154
156
|
2. Click the red **Record** button (between "Add Step" and "Generate All")
|
|
155
157
|
3. A headed Chromium browser opens your app's base URL
|
|
@@ -202,6 +204,8 @@ The recorder does **not** use simple CSS selectors or XPath. Instead, it:
|
|
|
202
204
|
- CSS fallback (last resort)
|
|
203
205
|
4. The first strategy that uniquely identifies the element is used
|
|
204
206
|
|
|
207
|
+
For elements **inside a shadow root**, the recorder uses `composedPath()[0]` to pierce the shadow boundary and builds a `host >> inner` pierce path automatically — no special action needed.
|
|
208
|
+
|
|
205
209
|
This produces the most resilient locators possible — the same quality as hand-written Playwright tests.
|
|
206
210
|
|
|
207
211
|
#### What makes it stand out vs other recorders
|
|
@@ -210,7 +214,10 @@ This produces the most resilient locators possible — the same quality as hand-
|
|
|
210
214
|
|---------|-------------|-------------------|---------------------|
|
|
211
215
|
| Locator quality | CSS/XPath | Good | Best — 6 strategies, verified against live page |
|
|
212
216
|
| Accessibility tree | No | Partial | Full — every locator checked via Playwright API |
|
|
213
|
-
| **Iframe support** | Partial | Manual | **Auto** — detects iframes, generates `frameLocator()` code |
|
|
217
|
+
| **Iframe support** | Partial | Manual | **Auto** — detects iframes, generates `frameLocator()` code, both click & fill |
|
|
218
|
+
| **Shadow DOM** | No | Partial | **Auto** — `composedPath()` pierces shadow roots, generates `>>` locators |
|
|
219
|
+
| **JS dialogs** | No | No | **Auto** — dialogs auto-accepted, `page.once('dialog', ...)` prepended to triggering step |
|
|
220
|
+
| **Keyboard capture** | Partial | Partial | **Full** — Tab, Shift+Tab, arrows, Enter, Escape, Ctrl+A, Space |
|
|
214
221
|
| Assertions | Manual | Manual | Shift+Click (hard), Ctrl+Shift+Click (soft), URL & title shortcuts |
|
|
215
222
|
| Plain-English steps | No | No | Yes — human-readable instructions auto-generated |
|
|
216
223
|
| Self-healing after | No | No | Yes — 5-level AI healing cascade |
|
|
@@ -228,21 +235,65 @@ This produces the most resilient locators possible — the same quality as hand-
|
|
|
228
235
|
| Assertions are hard to write | Shift+Click (hard), Ctrl+Shift+Click (soft), Ctrl+Shift+U for URL, Ctrl+Shift+T for title |
|
|
229
236
|
| Recorded tests are fragile | 6-strategy locator resolution + post-run 5-level self-healing |
|
|
230
237
|
| Apps use iframes (SAP, Salesforce) | Auto-detects iframe context, generates `frameLocator()` chains |
|
|
238
|
+
| Apps use Web Components / Shadow DOM | Automatic pierce-path detection using `composedPath()` + `>>` combinator |
|
|
239
|
+
| App pops up a JS alert mid-recording | Dialog auto-accepted, `page.once('dialog', ...)` handler prepended to the triggering step |
|
|
240
|
+
| Need to record Tab/arrow key navigation | Tab, Shift+Tab, arrow keys, Ctrl+A and Space captured automatically |
|
|
231
241
|
|
|
232
242
|
#### Iframe support
|
|
233
243
|
|
|
234
|
-
The recorder
|
|
244
|
+
The recorder and AI code generator both fully support **same-origin iframes** — common in enterprise apps like SAP, Salesforce, and embedded widgets:
|
|
235
245
|
|
|
236
246
|
- The capture script is injected into **all frames** (main page + every iframe), not just the top-level page
|
|
237
247
|
- When an element is inside an iframe, the recorder detects the frame context and computes a selector for the iframe element (using `id`, `name`, `data-testid`, or `src`)
|
|
238
248
|
- Locators are resolved against the correct frame using `page.frameLocator('...')` — producing code like:
|
|
239
249
|
```typescript
|
|
240
250
|
await page.frameLocator('#content-frame').getByRole('button', { name: 'Submit' }).click();
|
|
251
|
+
await page.frameLocator('iframe[src*="payment"]').getByPlaceholder('Card number').fill('4111...');
|
|
241
252
|
```
|
|
253
|
+
- Both **click** and **fill** (text input) actions work inside iframes
|
|
242
254
|
- Dynamically added iframes are detected via `frameattached` events and injected automatically
|
|
243
255
|
- The recording banner only appears in the main frame — iframes display cleanly
|
|
244
256
|
- **Limitation:** Cross-origin iframes (different domain) cannot be instrumented due to browser security. The main frame and same-origin iframes work fully.
|
|
245
257
|
|
|
258
|
+
#### Shadow DOM support
|
|
259
|
+
|
|
260
|
+
The recorder automatically handles Web Components and any element nested inside a shadow root:
|
|
261
|
+
|
|
262
|
+
- Uses `composedPath()[0]` on every event to get the real inner element, not the shadow host that the browser re-targets events to
|
|
263
|
+
- Detects shadow root context via `getRootNode() instanceof ShadowRoot`
|
|
264
|
+
- Builds a **pierce path** (`host-selector >> inner-selector`) and uses Playwright's `>>` combinator in generated code:
|
|
265
|
+
```typescript
|
|
266
|
+
await page.locator('my-login-form >> button[type="submit"]').click();
|
|
267
|
+
await page.locator('custom-input >> input').fill('user@example.com');
|
|
268
|
+
```
|
|
269
|
+
- Works transparently — just interact with the element normally, the recorder handles the rest
|
|
270
|
+
|
|
271
|
+
#### JavaScript alerts & dialogs
|
|
272
|
+
|
|
273
|
+
When your app shows a JS `alert()`, `confirm()`, or `prompt()` during recording:
|
|
274
|
+
|
|
275
|
+
- The dialog is **automatically accepted** so it doesn't block the session
|
|
276
|
+
- A `page.once('dialog', dialog => dialog.accept())` handler is **prepended** to the triggering step's generated code, ensuring correct execution order during replay:
|
|
277
|
+
```typescript
|
|
278
|
+
page.once('dialog', dialog => dialog.accept());
|
|
279
|
+
await page.getByRole('button', { name: 'Delete' }).click();
|
|
280
|
+
```
|
|
281
|
+
- The dialog type and message are stored on the step for reference
|
|
282
|
+
|
|
283
|
+
#### Keyboard capture
|
|
284
|
+
|
|
285
|
+
Beyond click and fill, the recorder captures keyboard interactions:
|
|
286
|
+
|
|
287
|
+
| Key | When captured | Generated code |
|
|
288
|
+
|-----|--------------|----------------|
|
|
289
|
+
| **Enter** | While focused on a form field | `await page.getByLabel('...').press('Enter');` |
|
|
290
|
+
| **Escape** | Any time | `await page.keyboard.press('Escape');` |
|
|
291
|
+
| **Tab** | While focused on a form field | `await page.getByLabel('...').press('Tab');` |
|
|
292
|
+
| **Shift+Tab** | While focused on a form field | `await page.getByLabel('...').press('Shift+Tab');` |
|
|
293
|
+
| **Arrow keys** | When focused on a non-input element | `await page.keyboard.press('ArrowDown');` |
|
|
294
|
+
| **Ctrl+A** | Any time | `await page.keyboard.press('Control+A');` |
|
|
295
|
+
| **Space** | On a button or role=button element | `await page.getByRole('button', { name: '...' }).press(' ');` |
|
|
296
|
+
|
|
246
297
|
#### Technical details
|
|
247
298
|
|
|
248
299
|
- The recorder uses `context.exposeBinding()` for browser↔Node.js communication that survives page navigations — works across all frames
|
|
@@ -251,6 +302,67 @@ The recorder automatically handles **same-origin iframes** — common in enterpr
|
|
|
251
302
|
- Recorded steps use the `recorder` strategy tag, distinguishing them from AI-generated code
|
|
252
303
|
- All recorded steps are immediately usable — no "Generate Code" step required
|
|
253
304
|
|
|
305
|
+
### Advanced Step Patterns
|
|
306
|
+
|
|
307
|
+
The AI code generator handles several advanced browser scenarios automatically — just describe them in plain English.
|
|
308
|
+
|
|
309
|
+
#### JavaScript Alerts & Dialogs
|
|
310
|
+
|
|
311
|
+
Mention "alert", "confirm dialog", "popup", "click OK", or "dismiss" and the AI generates a `page.once('dialog', ...)` handler registered before the triggering action:
|
|
312
|
+
|
|
313
|
+
| Instruction | Generated pattern |
|
|
314
|
+
|-------------|------------------|
|
|
315
|
+
| `Accept the alert dialog` | `page.once('dialog', d => d.accept())` |
|
|
316
|
+
| `Dismiss the confirmation popup` | `page.once('dialog', d => d.dismiss())` |
|
|
317
|
+
| `Enter "yes" in the prompt dialog` | `page.once('dialog', d => d.accept('yes'))` |
|
|
318
|
+
| `Verify the alert says "Are you sure?" and accept it` | Handler that checks `dialog.message()` then accepts |
|
|
319
|
+
|
|
320
|
+
#### Shadow DOM
|
|
321
|
+
|
|
322
|
+
For elements inside a Web Component's shadow root, use language like "inside the `<component-name>` component" or "in the shadow root" — the AI uses Playwright's `>>` pierce combinator:
|
|
323
|
+
|
|
324
|
+
```
|
|
325
|
+
Click the submit button inside the my-form component
|
|
326
|
+
Fill the email input inside custom-input
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
Generated code:
|
|
330
|
+
```typescript
|
|
331
|
+
await page.locator('my-form >> button[type="submit"]').click();
|
|
332
|
+
await page.locator('custom-input >> input').fill('user@example.com');
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
#### iFrames — Click and Fill
|
|
336
|
+
|
|
337
|
+
Both clicking and typing inside iframes work. Describe the action naturally, referencing the iframe if known:
|
|
338
|
+
|
|
339
|
+
```
|
|
340
|
+
Click the Submit button inside the payment iframe
|
|
341
|
+
Enter "4111111111111111" in the card number field inside the checkout frame
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
Generated code:
|
|
345
|
+
```typescript
|
|
346
|
+
await page.frameLocator('iframe[src*="payment"]').getByRole('button', { name: 'Submit' }).click();
|
|
347
|
+
await page.frameLocator('iframe[src*="checkout"]').getByPlaceholder('Card number').fill('4111111111111111');
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
#### Keyboard Actions
|
|
351
|
+
|
|
352
|
+
Describe key presses directly — the AI uses `page.keyboard.press()` or `page.keyboard.type()`:
|
|
353
|
+
|
|
354
|
+
| Instruction | Generated code |
|
|
355
|
+
|-------------|----------------|
|
|
356
|
+
| `Press Enter` | `await page.keyboard.press('Enter')` |
|
|
357
|
+
| `Press Escape to close the modal` | `await page.keyboard.press('Escape')` |
|
|
358
|
+
| `Press Tab to move to the next field` | `await page.keyboard.press('Tab')` |
|
|
359
|
+
| `Press Ctrl+A to select all text` | `await page.keyboard.press('Control+A')` |
|
|
360
|
+
| `Press Ctrl+Z to undo` | `await page.keyboard.press('Control+Z')` |
|
|
361
|
+
| `Type "Hello World" using the keyboard` | `await page.keyboard.type('Hello World')` |
|
|
362
|
+
| `Press Enter on the search field` | `await page.getByRole('textbox').press('Enter')` |
|
|
363
|
+
|
|
364
|
+
---
|
|
365
|
+
|
|
254
366
|
### Generating Code
|
|
255
367
|
|
|
256
368
|
After adding steps, click **Generate Code** (or **Generate** on an individual step) to have the AI write the Playwright code.
|
|
@@ -275,7 +387,7 @@ Deletions are permanent. There is no undo in the Studio — use Git to recover.
|
|
|
275
387
|
|
|
276
388
|
---
|
|
277
389
|
|
|
278
|
-
## Story Generator
|
|
390
|
+
## Story Generator (Smart Tests)
|
|
279
391
|
|
|
280
392
|
Generate an entire test suite from a plain-English user story:
|
|
281
393
|
|
|
@@ -301,6 +413,10 @@ The AI creates:
|
|
|
301
413
|
|
|
302
414
|
The generated files appear in the Test Editor immediately.
|
|
303
415
|
|
|
416
|
+
### Jira Integration Warning
|
|
417
|
+
|
|
418
|
+
If Jira credentials are not configured and you try to load a Jira issue, a **Configuration Required** modal appears. This modal includes a **"Don't show this again"** checkbox at the bottom — checking it and clicking Close suppresses the modal permanently for that integration (stored in browser localStorage). You can re-enable it by clearing site data or browser storage.
|
|
419
|
+
|
|
304
420
|
---
|
|
305
421
|
|
|
306
422
|
## Run Config
|
|
@@ -322,6 +438,10 @@ Configure and start a test run without the command line:
|
|
|
322
438
|
|
|
323
439
|
A live progress panel shows test results as they complete. The WebSocket connection streams updates in real time — no page refresh needed.
|
|
324
440
|
|
|
441
|
+
### Stopping a run
|
|
442
|
+
|
|
443
|
+
While a run is in progress, a **Stop** button (red square icon) appears in the execution header. Clicking it sends an immediate stop signal to the server — the current step finishes, then all remaining test cases are skipped and the run exits cleanly. The UI transitions out of the running state automatically once the stop is confirmed.
|
|
444
|
+
|
|
325
445
|
When the run finishes, you are offered a link to the full report.
|
|
326
446
|
|
|
327
447
|
### Device Emulation
|
|
@@ -574,13 +694,95 @@ Go to **Settings → RAG Memory** to configure:
|
|
|
574
694
|
|
|
575
695
|
---
|
|
576
696
|
|
|
697
|
+
## Faker Data
|
|
698
|
+
|
|
699
|
+
The Faker Data modal (accessible from the Test Editor) lets you browse 100+ dynamic data generators across 16 categories. Search or browse by category, then select one or more tokens — the modal supports **multi-select**, showing selected tokens as removable chips in a chip bar. Click **"Insert N tokens"** to insert all selected tokens comma-separated into your step (e.g. `{{FAKE_PERSON_FIRSTNAME}}, {{FAKE_PERSON_LASTNAME}}, {{FAKE_INTERNET_EMAIL}}`). This is especially useful for form-filling steps where you need first name, last name, email, and phone all at once. Tokens resolve to fresh random values on every test run.
|
|
700
|
+
|
|
701
|
+
---
|
|
702
|
+
|
|
703
|
+
## Visual Regression
|
|
704
|
+
|
|
705
|
+
Visual regression is configured in **Settings** → Visual Regression and reviewed in **Reports** → Visual Diffs / Baselines tabs.
|
|
706
|
+
|
|
707
|
+
**Configuration** (Settings → Visual Regression):
|
|
708
|
+
| Setting | Description | Default |
|
|
709
|
+
|---------|------------|---------|
|
|
710
|
+
| Enabled | Master toggle for visual regression | Off |
|
|
711
|
+
| Threshold | Pixel diff percentage to fail (0.1% - 10%) | 0.5% |
|
|
712
|
+
| Full Page | Capture entire scrollable page | Off |
|
|
713
|
+
| Mask Selectors | CSS selectors to mask (ignore dynamic content like timestamps) | Empty |
|
|
714
|
+
|
|
715
|
+
**Per-step toggle** — In the Test Editor, click the **Eye icon** on any step to enable visual comparison for that step only. Only steps with the Eye icon active are compared during runs.
|
|
716
|
+
|
|
717
|
+
**Reports → Visual Diffs tab** — after runs with visual checks:
|
|
718
|
+
- Select a run from the dropdown
|
|
719
|
+
- Summary: passed, failed, new baseline counts
|
|
720
|
+
- Each diff shows three images: Baseline (expected), Actual (current), Diff (red highlights)
|
|
721
|
+
- **Approve** — update baseline to the new screenshot (intentional change)
|
|
722
|
+
- **Reject** — keep current baseline (it's a real bug, dev needs to fix)
|
|
723
|
+
- **Approve All** — bulk approve all diffs in a run
|
|
724
|
+
|
|
725
|
+
**Reports → Baselines tab** — gallery of all current baseline screenshots:
|
|
726
|
+
- Organized by suite → case → step
|
|
727
|
+
- Browser and viewport info per baseline
|
|
728
|
+
- Delete individual or bulk-delete baselines
|
|
729
|
+
|
|
730
|
+
**File structure:**
|
|
731
|
+
```
|
|
732
|
+
baselines/ ← Committed to git
|
|
733
|
+
├── login-suite/valid-login/
|
|
734
|
+
│ ├── step-2-chromium-1920x1080.png
|
|
735
|
+
│ └── step-2-firefox-768x1024.png
|
|
736
|
+
results/visual-diffs/{runId}/ ← Gitignored, per-run
|
|
737
|
+
├── step-2-baseline.png
|
|
738
|
+
├── step-2-actual.png
|
|
739
|
+
└── step-2-diff.png
|
|
740
|
+
```
|
|
741
|
+
|
|
742
|
+
---
|
|
743
|
+
|
|
744
|
+
## CI/CD Integration
|
|
745
|
+
|
|
746
|
+
The **CI/CD Integration** page (`/ci-integration`) configures pipeline behavior — quality gates, PR comments, and notifications.
|
|
747
|
+
|
|
748
|
+
**Quality Gates** — define pass/fail thresholds:
|
|
749
|
+
| Gate | Description | Default |
|
|
750
|
+
|------|-----------|---------|
|
|
751
|
+
| Min Pass Rate | Minimum percentage of tests that must pass | 95% |
|
|
752
|
+
| Max Duration | Maximum run duration in seconds (0 = unlimited) | 0 |
|
|
753
|
+
| Required Tags | Tags where ALL tests must pass (e.g., smoke, critical) | Empty |
|
|
754
|
+
|
|
755
|
+
Quality gate history shows last 5 results as green/red badges.
|
|
756
|
+
|
|
757
|
+
**PR Comments** — auto-post test results:
|
|
758
|
+
- Supports GitHub (uses `GITHUB_TOKEN`) and GitLab (uses `GITLAB_TOKEN`)
|
|
759
|
+
- Markdown table with pass rate, failed count, duration, quality gate status
|
|
760
|
+
- Optional: healing summary, visual diff summary
|
|
761
|
+
- Posted automatically when `--ci` flag is used
|
|
762
|
+
|
|
763
|
+
**Notifications** — push results to team channels:
|
|
764
|
+
| Channel | Configuration |
|
|
765
|
+
|---------|--------------|
|
|
766
|
+
| Slack | Webhook URL — sends block kit message with pass/fail, failures list, healing summary |
|
|
767
|
+
| Microsoft Teams | Webhook URL — sends adaptive card |
|
|
768
|
+
| Email | Recipient list (SMTP placeholder) |
|
|
769
|
+
| Custom Webhook | Any URL + headers — sends full JSON run result |
|
|
770
|
+
|
|
771
|
+
Each channel has trigger options: On every run, On failure only, On healing events.
|
|
772
|
+
|
|
773
|
+
---
|
|
774
|
+
|
|
577
775
|
## WebSocket Live Updates
|
|
578
776
|
|
|
579
777
|
The Studio uses a WebSocket connection (`ws://localhost:<port>/ws`) to receive live updates:
|
|
580
778
|
|
|
581
779
|
| Event | Description |
|
|
582
780
|
|-------|-------------|
|
|
781
|
+
| `run:start` | A run has started — UI spinner activates |
|
|
782
|
+
| `run:case` | A test case has started executing |
|
|
783
|
+
| `run:step` | A step within a case has started |
|
|
583
784
|
| `run:complete` | A run has finished — results available |
|
|
785
|
+
| `run:stopped` | A run was stopped early via the Stop button |
|
|
584
786
|
| `run:error` | A run failed to start |
|
|
585
787
|
| `recorder:action` | A new action was recorded (click, fill, navigate, assert) |
|
|
586
788
|
| `recorder:stopped` | Recording session ended |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "assuremind",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.3.0",
|
|
4
4
|
"description": "AI-powered codeless UI & API automation framework",
|
|
5
5
|
"author": "Deepak Hiremath",
|
|
6
6
|
"license": "MIT",
|
|
@@ -67,6 +67,7 @@
|
|
|
67
67
|
"dependencies": {
|
|
68
68
|
"@anthropic-ai/sdk": "^0.30.0",
|
|
69
69
|
"@aws-sdk/client-bedrock-runtime": "^3.679.0",
|
|
70
|
+
"@faker-js/faker": "^10.4.0",
|
|
70
71
|
"@fastify/cors": "^11.2.0",
|
|
71
72
|
"@fastify/static": "^9.0.0",
|
|
72
73
|
"@fastify/websocket": "^11.2.0",
|
|
@@ -95,9 +96,11 @@
|
|
|
95
96
|
"ora": "^8.0.1",
|
|
96
97
|
"pino": "^9.4.0",
|
|
97
98
|
"pino-pretty": "^11.2.0",
|
|
98
|
-
"
|
|
99
|
+
"pixelmatch": "^7.1.0",
|
|
100
|
+
"playwright": "^1.60.0",
|
|
101
|
+
"pngjs": "^7.0.0",
|
|
99
102
|
"simple-git": "^3.33.0",
|
|
100
|
-
"uuid": "
|
|
103
|
+
"uuid": "11.1.1",
|
|
101
104
|
"zod": "^3.23.0"
|
|
102
105
|
},
|
|
103
106
|
"devDependencies": {
|
|
@@ -105,16 +108,17 @@
|
|
|
105
108
|
"@types/js-yaml": "^4.0.9",
|
|
106
109
|
"@types/lodash-es": "^4.17.0",
|
|
107
110
|
"@types/node": "^22.7.0",
|
|
111
|
+
"@types/pngjs": "^6.0.5",
|
|
108
112
|
"@types/uuid": "^10.0.0",
|
|
109
113
|
"@types/ws": "^8.5.0",
|
|
110
114
|
"@typescript-eslint/eslint-plugin": "^8.8.0",
|
|
111
115
|
"@typescript-eslint/parser": "^8.8.0",
|
|
112
|
-
"@vitest/coverage-v8": "
|
|
116
|
+
"@vitest/coverage-v8": "3.2.6",
|
|
113
117
|
"docx": "^9.6.1",
|
|
114
118
|
"eslint": "^9.12.0",
|
|
115
119
|
"prettier": "^3.3.0",
|
|
116
120
|
"tsup": "^8.3.0",
|
|
117
121
|
"typescript": "^5.6.0",
|
|
118
|
-
"vitest": "
|
|
122
|
+
"vitest": "3.2.6"
|
|
119
123
|
}
|
|
120
124
|
}
|
package/templates/ASSUREMIND.md
CHANGED
|
@@ -112,6 +112,13 @@ npx assuremind doctor
|
|
|
112
112
|
| Wait | `Wait for the loading spinner to disappear` |
|
|
113
113
|
| Upload | `Upload the file "fixtures/data/avatar.png" to the photo field` |
|
|
114
114
|
| Variable | `Enter "{{ADMIN_EMAIL}}" in the email field` |
|
|
115
|
+
| **Accept JS alert** | `Accept the alert dialog` |
|
|
116
|
+
| **Dismiss confirm popup** | `Dismiss the confirmation popup` |
|
|
117
|
+
| **Shadow DOM** | `Click the submit button inside the my-form component` |
|
|
118
|
+
| **iFrame click** | `Click the Submit button inside the payment iframe` |
|
|
119
|
+
| **iFrame fill** | `Enter "4111..." in the card field inside the checkout iframe` |
|
|
120
|
+
| **Key press** | `Press Enter` / `Press Escape` / `Press Ctrl+A` |
|
|
121
|
+
| **Keyboard type** | `Type "Hello World" using the keyboard` |
|
|
115
122
|
|
|
116
123
|
---
|
|
117
124
|
|
|
@@ -216,6 +216,18 @@ A suite is a collection of related test cases. For example:
|
|
|
216
216
|
- Assertions start with: `Verify`, `Assert`, `Check`, `Confirm`
|
|
217
217
|
- Navigation: `Go to`, `Navigate to`, `Open`
|
|
218
218
|
|
|
219
|
+
**Advanced patterns — the AI handles these automatically:**
|
|
220
|
+
|
|
221
|
+
| Scenario | Example instruction |
|
|
222
|
+
|----------|-------------------|
|
|
223
|
+
| Accept JS alert | `Accept the alert dialog` |
|
|
224
|
+
| Dismiss confirm popup | `Dismiss the confirmation popup` |
|
|
225
|
+
| Shadow DOM | `Click the submit button inside the my-form component` |
|
|
226
|
+
| iFrame click | `Click the Submit button inside the payment iframe` |
|
|
227
|
+
| iFrame fill | `Enter "4111..." in the card field inside the checkout iframe` |
|
|
228
|
+
| Key press / shortcut | `Press Enter` / `Press Ctrl+A to select all` |
|
|
229
|
+
| Keyboard type | `Type "Hello World" using the keyboard` |
|
|
230
|
+
|
|
219
231
|
---
|
|
220
232
|
|
|
221
233
|
## 9. Generate Code with AI
|
|
@@ -250,6 +262,7 @@ You can also:
|
|
|
250
262
|
**From the Studio:**
|
|
251
263
|
- Open the case, click the **▶ Run** button in the toolbar
|
|
252
264
|
- Watch real-time progress in the run panel — each step lights up green/red
|
|
265
|
+
- To stop mid-run, click the red **Stop** button in the execution header — remaining cases are skipped cleanly
|
|
253
266
|
|
|
254
267
|
**From the CLI:**
|
|
255
268
|
```bash
|
package/templates/docs/STUDIO.md
CHANGED
|
@@ -217,8 +217,14 @@ Each step row shows:
|
|
|
217
217
|
| Wait | `Wait for the loading spinner to disappear` |
|
|
218
218
|
| Hover | `Hover over the user avatar` |
|
|
219
219
|
| Upload file | `Upload "fixtures/data/resume.pdf" to the file input` |
|
|
220
|
-
| Press key | `Press
|
|
220
|
+
| Press key | `Press Enter` / `Press Escape` / `Press Ctrl+A` |
|
|
221
221
|
| Scroll | `Scroll down to the footer` |
|
|
222
|
+
| **Accept JS alert** | `Accept the alert dialog` |
|
|
223
|
+
| **Dismiss confirm** | `Dismiss the confirmation popup` |
|
|
224
|
+
| **Shadow DOM** | `Click the submit button inside the my-form component` |
|
|
225
|
+
| **iFrame click** | `Click the Submit button inside the payment iframe` |
|
|
226
|
+
| **iFrame fill** | `Enter "4111..." in the card field inside the checkout iframe` |
|
|
227
|
+
| **Keyboard type** | `Type "Hello World" using the keyboard` |
|
|
222
228
|
|
|
223
229
|
### Editing a Step Instruction
|
|
224
230
|
|
|
@@ -310,6 +316,10 @@ A **Run Panel** appears at the bottom of the editor showing:
|
|
|
310
316
|
| ⬜ | Step not yet reached |
|
|
311
317
|
| ⚡ | Step was healed (self-healing kicked in) |
|
|
312
318
|
|
|
319
|
+
### Stopping a Run
|
|
320
|
+
|
|
321
|
+
While a run is active, a red **Stop** button (square icon) appears in the execution header bar. Click it to send an immediate stop signal — the current step completes, then all remaining cases are skipped and the run exits cleanly.
|
|
322
|
+
|
|
313
323
|
### After a Run
|
|
314
324
|
|
|
315
325
|
- Click any **failed step** to see the error message and screenshot
|