assuremind 1.1.2 → 1.2.0

package/docs/CLI-REFERENCE.md

@@ -327,3 +327,107 @@ MCP (Model Context Protocol) settings are configured in `autotest.config.json` (
 | `mcp.idleTimeout` | `30000` | Auto-disconnect MCP browser after idle period in ms |
 
 > **Note:** MCP is used only during code generation. Test execution (`npx assuremind run`) is never affected by MCP settings.
+
+---
+
+## RAG Configuration
+
+RAG (Retrieval-Augmented Generation) is **ON by default — zero setup required**. The AI automatically learns from every test run, retrieving similar past steps and healing fixes to improve accuracy over time. No database, no external service — uses local TF-IDF embeddings and file-based JSON storage.
+
+**Most users never need to change these settings.** They exist for power-user scenarios like debugging flaky tests, resetting memory after a major app redesign, or disabling RAG for deterministic CI runs.
+
+Settings are configured in `autotest.config.json` (or via Settings → RAG Memory in Studio).
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `rag.enabled` | `true` | Master switch for semantic memory from past runs |
+| `rag.codeCorpus.enabled` | `true` | Remember instruction-to-code mappings from successful runs |
+| `rag.codeCorpus.maxEntries` | `500` | Maximum entries in the code corpus |
+| `rag.codeCorpus.similarityThreshold` | `0.65` | Minimum similarity score to include as AI prompt example |
+| `rag.codeCorpus.directUseThreshold` | `0.90` | Minimum similarity score to use directly as fuzzy cache hit ($0) |
+| `rag.healingCorpus.enabled` | `true` | Remember past healing fixes for smarter self-repair |
+| `rag.healingCorpus.maxEntries` | `300` | Maximum entries in the healing corpus |
+| `rag.healingCorpus.similarityThreshold` | `0.60` | Minimum similarity score for healing retrieval |
+| `rag.errorCatalog.enabled` | `true` | Track recurring error patterns per URL |
+| `rag.errorCatalog.maxEntries` | `200` | Maximum entries in the error catalog |
+| `rag.embedder` | `'tfidf'` | Embedding strategy: `tfidf` (local, free, offline) |
+
+> **Note:** RAG data is stored in `results/.rag/` as JSON files. It persists across runs and is fully Git-friendly. Delete the directory to reset all memory.
+
+### When to change RAG settings
+
+| Scenario | What to do |
+|----------|------------|
+| Debugging a flaky test | Turn OFF `rag.codeCorpus.enabled` — forces fresh AI generation instead of reusing a possibly stale mapping |
+| Healing keeps suggesting a bad fix | Turn OFF `rag.healingCorpus.enabled` — clears bad fix influence |
+| Major app redesign (UI overhaul) | Set `rag.enabled: false` — old memory from the previous UI is now misleading |
+| Want deterministic CI runs | Disable RAG in CI config, keep ON for local development |
+| Error warnings are outdated | Turn OFF `rag.errorCatalog.enabled` — stops AI from avoiding selectors that are fine now |
+
+---
+
+## Recorder API
+
+The Test Recorder is available through the Studio UI (Test Editor → Record button) and also exposes REST endpoints for programmatic access:
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/recorder/start` | POST | Launch a headed browser and begin recording. Body: `{ url?: string }` |
+| `/api/recorder/stop` | POST | Stop recording and return all captured actions |
+| `/api/recorder/status` | GET | Check if a recording session is active |
+
+Recorded actions are broadcast in real time via WebSocket (`recorder:action` event). When recording stops, a `recorder:stopped` event is sent.
+
+### Soft Assertions
+
+The recorder supports **soft assertions** via `Ctrl+Shift+Click` — these use Playwright's `expect.soft()` so the test continues executing even when the assertion fails. All failures are collected and reported at the end.
+
+For plain-English steps, prefix with "Soft" to generate soft assertion code:
+- `Soft verify "Dashboard" text is visible` → `await expect.soft(page.getByText('Dashboard')).toBeVisible();`
+- `Soft check the URL contains "/dashboard"` → `await expect.soft(page).toHaveURL(/dashboard/);`
+
+You can also toggle any assertion step between hard and soft using the **Hard/Soft** badge in the Test Editor.
+
+### Recorder locator strategies
+
+The recorder resolves locators against Playwright's live accessibility tree using 6 strategies in priority order:
+
+| Priority | Strategy | Example |
+|----------|----------|---------|
+| 1 | `data-testid` | `page.getByTestId('login-btn')` |
+| 2 | `getByRole()` exact + level | `page.getByRole('heading', { name: 'Dashboard', level: 1 })` |
+| 3 | `getByRole()` exact | `page.getByRole('button', { name: 'Login' })` |
+| 4 | `getByLabel()` | `page.getByLabel('Email')` |
+| 5 | `getByPlaceholder()` | `page.getByPlaceholder('Enter email')` |
+| 6 | `getByText()` | `page.getByText('Welcome')` |
+| 7 | CSS fallback | `page.locator('#login-btn')` |
+
+Each candidate is verified with `count() === 1` — only uniquely-matching locators are used.
+
+### Iframe support
+
+The recorder automatically handles **same-origin iframes** — essential for enterprise apps like SAP and Salesforce:
+
+- The capture script is injected into **all frames**, not just the main page
+- Elements inside iframes produce `frameLocator()` chains:
+  ```typescript
+  // Element inside an iframe with id="content-frame"
+  await page.frameLocator('#content-frame').getByRole('button', { name: 'Submit' }).click();
+  ```
+- The iframe selector is computed from the iframe's `id`, `name`, `data-testid`, or `src` attribute
+- Dynamically loaded iframes are detected and instrumented automatically
+- **Limitation:** Cross-origin iframes cannot be instrumented due to browser security restrictions
+
+---
+
+### CI/CD tips
+
+```yaml
+# Cache RAG memory between CI runs for persistent learning
+- name: Cache RAG memory
+  uses: actions/cache@v4
+  with:
+    path: results/.rag
+    key: rag-memory-${{ github.ref }}
+    restore-keys: rag-memory-
+```

package/docs/GETTING-STARTED.md

@@ -128,6 +128,13 @@ export default defineConfig({
     actThenScript: false,  // Two-phase generation (higher accuracy, slower)
     proactiveHealing: false, // Pre-run selector validation
   },
+  rag: {
+    enabled: true,         // AI learns from past runs (semantic memory)
+    codeCorpus: { enabled: true, maxEntries: 500, similarityThreshold: 0.65, directUseThreshold: 0.90 },
+    healingCorpus: { enabled: true, maxEntries: 300, similarityThreshold: 0.60 },
+    errorCatalog: { enabled: true, maxEntries: 200 },
+    embedder: 'tfidf',     // local, free, offline — no API calls
+  },
 });
 ```
 
@@ -159,7 +166,24 @@ Your browser opens at `http://localhost:4400`. From there:
 
 See [STUDIO.md](./STUDIO.md) for a full Studio walkthrough.
 
-### Option B — CLI generate command
+### Option B — Record a test (fastest, zero AI)
+
+Start the Studio and use the built-in **Test Recorder** to create tests by clicking through your app:
+
+1. Click **Test Editor** → select a suite and case (or create new ones)
+2. Click the red **Record** button in the step editor
+3. A headed Chromium browser opens your app — interact naturally:
+   - Click buttons, fill forms, navigate pages
+   - **Shift+Click** any element to assert it's visible (hard assertion — test stops on failure)
+   - **Ctrl+Shift+Click** any element for a **soft assertion** (test continues, failures collected at end)
+   - **Ctrl+Shift+U** to assert the current URL
+   - **Ctrl+Shift+T** to assert the page title
+4. Click **Stop Recording** — all actions become steps with pre-generated Playwright code
+5. Click **Run** to execute immediately
+
+**No AI calls, no API keys needed** — the recorder resolves locators against Playwright's accessibility tree in real time.
+
+### Option C — CLI generate command
 
 Generate a full test suite from a user story in one command:
 
@@ -171,7 +195,7 @@ npx assuremind generate \
 
 This creates the suite file structure under `tests/login-tests/` and generates Playwright code for every step.
 
-### Option C — Write JSON directly
+### Option D — Write JSON directly
 
 Create `tests/login-tests/suite.json`:
 
@@ -372,14 +396,51 @@ Or manage variables in the Studio → **Variables** page.
 
 ---
 
-## 12. Next Steps
+## 12. RAG Memory — AI Gets Smarter Over Time
+
+RAG (Retrieval-Augmented Generation) is **ON by default — zero setup required**. The AI automatically learns from every test run:
+
+- **Run 1** — memory is empty, AI generates code normally
+- **Run 2+** — similar instructions are retrieved from past runs instead of making API calls (free + faster)
+- **Run 10+** — most common steps served from memory at zero cost, self-healing resolves issues on the first attempt
+
+### What happens automatically
+
+| Event | What RAG does |
+|-------|---------------|
+| Step passes | Remembers the instruction → code mapping |
+| Step fails | Records the error pattern for that URL |
+| Healing fixes a step | Remembers the error → fix pair |
+| Next similar step | Retrieves past code instead of calling AI ($0) |
+| Next similar error | Uses proven past fix in the healing prompt |
+
+### Storage
+
+RAG data lives in `results/.rag/` as plain JSON files. To share memory across your team, commit it to Git. To reset, delete the folder.
+
+### When to change RAG settings
+
+Most users **never need to touch RAG settings**. The Settings → RAG Memory card exists for edge cases:
+
+| Scenario | Action |
+|----------|--------|
+| Debugging a flaky test | Turn OFF Code Corpus — forces fresh AI generation |
+| Healing keeps suggesting a bad fix | Turn OFF Healing Corpus |
+| Major app redesign | Turn OFF RAG entirely — old memory is misleading |
+| Want deterministic CI runs | Disable RAG in CI, keep ON locally |
+
+---
+
+## 13. Next Steps
 
 | Topic | Where to look |
 |-------|---------------|
+| Test Recorder | [Studio Guide → Test Recorder](./STUDIO.md#test-recorder) |
 | All CLI flags | [CLI Reference](./CLI-REFERENCE.md) |
 | Studio UI walkthrough | [Studio Guide](./STUDIO.md) |
 | Build the package from source | [CONTRIBUTING.md](../CONTRIBUTING.md) |
 | Config options | `autotest.config.ts` comments |
 | MCP integration | Settings page → MCP Integration |
+| RAG memory | Settings page → RAG Memory |
 | All supported AI providers | `.env.example` |
 | Quick daily reference | `ASSUREMIND.md` in your project root |

package/docs/STUDIO.md

@@ -144,6 +144,113 @@ If all three are unchecked, Lighthouse is skipped for that case. A warning is sh
 - Reference variables with double-braces: `Enter "{{ADMIN_EMAIL}}" in the email field`
 - Include assertions: `Verify that the success banner is visible`
 
+### Test Recorder
+
+The Test Recorder lets you create tests by interacting with your application in a real browser — **no coding, no AI, no cost**.
+
+#### Recording a test
+
+1. Open a test case in the editor
+2. Click the red **Record** button (between "Add Step" and "Generate All")
+3. A headed Chromium browser opens your app's base URL
+4. Interact with the page naturally — click buttons, fill forms, navigate
+5. **Iframes are handled automatically** — the recorder injects into all frames and generates correct `frameLocator()` code
+6. Each action appears in the live preview panel in real time
+7. Click **Stop Recording** when done
+
+Every recorded action becomes a test step with **pre-generated Playwright code** — no AI generation needed.
+
+#### Assertion shortcuts
+
+While recording, use these keyboard shortcuts to add assertions:
+
+| Shortcut | Assertion Type | Behavior | Example Output |
+|----------|---------------|----------|----------------|
+| **Shift+Click** | Hard assertion — element visible | Test **stops** on failure | `await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();` |
+| **Ctrl+Shift+Click** | Soft assertion — element visible | Test **continues**, failures collected at end | `await expect.soft(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();` |
+| **Ctrl+Shift+U** | Current URL matches | Hard assertion | `await expect(page).toHaveURL(/dashboard/);` |
+| **Ctrl+Shift+T** | Page title matches | Hard assertion | `await expect(page).toHaveTitle(/Dashboard/);` |
+
+Assertion steps show a green badge in the live preview. Soft assertions show a blue **Soft** badge in the step editor.
+
+#### Hard vs Soft Assertions
+
+| Type | Keyword | Behavior | When to use |
+|------|---------|----------|-------------|
+| **Hard** | `Verify ...` | Test stops immediately on failure | Critical checks — login worked, page loaded |
+| **Soft** | `Soft verify ...` | Test continues, all failures reported at end | Multiple checks on one page — verify 5 fields are correct |
+
+**Three ways to use soft assertions:**
+
+1. **Recorder** — Ctrl+Shift+Click instead of Shift+Click
+2. **Plain English** — Start instruction with "Soft verify..." or "Soft check..." or "Soft assert..."
+3. **Step toggle** — Click the **Hard/Soft** badge on any assertion step in the editor to switch
+
+#### How locators are resolved
+
+The recorder does **not** use simple CSS selectors or XPath. Instead, it:
+
+1. Captures raw element metadata from the DOM (tag, attributes, text, ARIA properties)
+2. Sends the metadata to Node.js where Playwright's accessibility tree is queried
+3. Tries **6 locator strategies** in priority order, verifying each with `count() === 1`:
+   - `data-testid` attribute
+   - `getByRole()` with exact name + heading level
+   - `getByRole()` with exact name
+   - `getByLabel()`
+   - `getByPlaceholder()`
+   - `getByText()`
+   - CSS fallback (last resort)
+4. The first strategy that uniquely identifies the element is used
+
+This produces the most resilient locators possible — the same quality as hand-written Playwright tests.
+
+#### What makes it stand out vs other recorders
+
+| Feature | Selenium IDE | Playwright Codegen | Assuremind Recorder |
+|---------|-------------|-------------------|---------------------|
+| Locator quality | CSS/XPath | Good | Best — 6 strategies, verified against live page |
+| Accessibility tree | No | Partial | Full — every locator checked via Playwright API |
+| **Iframe support** | Partial | Manual | **Auto** — detects iframes, generates `frameLocator()` code |
+| Assertions | Manual | Manual | Shift+Click (hard), Ctrl+Shift+Click (soft), URL & title shortcuts |
+| Plain-English steps | No | No | Yes — human-readable instructions auto-generated |
+| Self-healing after | No | No | Yes — 5-level AI healing cascade |
+| RAG memory | No | No | Yes — recorded steps feed the learning loop |
+| Cost | Free | Free | Free |
+
+#### Biggest pain points solved
+
+| Pain Point | How the Recorder Solves It |
+|-----------|---------------------------|
+| Writing tests is slow | Record a full test in 30 seconds |
+| Selectors break constantly | Locators verified against Playwright's accessibility tree in real time |
+| AI costs money | Recording + code generation = $0, zero AI calls |
+| Non-technical testers can't write tests | Anyone who can click a browser can create tests |
+| Assertions are hard to write | Shift+Click (hard), Ctrl+Shift+Click (soft), Ctrl+Shift+U for URL, Ctrl+Shift+T for title |
+| Recorded tests are fragile | 6-strategy locator resolution + post-run 5-level self-healing |
+| Apps use iframes (SAP, Salesforce) | Auto-detects iframe context, generates `frameLocator()` chains |
+
+#### Iframe support
+
+The recorder automatically handles **same-origin iframes** — common in enterprise apps like SAP, Salesforce, and embedded widgets:
+
+- The capture script is injected into **all frames** (main page + every iframe), not just the top-level page
+- When an element is inside an iframe, the recorder detects the frame context and computes a selector for the iframe element (using `id`, `name`, `data-testid`, or `src`)
+- Locators are resolved against the correct frame using `page.frameLocator('...')` — producing code like:
+  ```typescript
+  await page.frameLocator('#content-frame').getByRole('button', { name: 'Submit' }).click();
+  ```
+- Dynamically added iframes are detected via `frameattached` events and injected automatically
+- The recording banner only appears in the main frame — iframes display cleanly
+- **Limitation:** Cross-origin iframes (different domain) cannot be instrumented due to browser security. The main frame and same-origin iframes work fully.
+
+#### Technical details
+
+- The recorder uses `context.exposeBinding()` for browser↔Node.js communication that survives page navigations — works across all frames
+- Input values are captured on blur (when the user leaves the field), not on keystrokes — no duplicate steps
+- The capture script is re-injected on every page load via `page.on('load')` and into all child frames via `page.on('frameattached')` + `page.on('framenavigated')`
+- Recorded steps use the `recorder` strategy tag, distinguishing them from AI-generated code
+- All recorded steps are immediately usable — no "Generate Code" step required
+
 ### Generating Code
 
 After adding steps, click **Generate Code** (or **Generate** on an individual step) to have the AI write the Playwright code.
@@ -355,6 +462,11 @@ Edit `autotest.config.ts` from the browser:
 | MCP headless | Run MCP browser without visible window |
 | MCP act-then-script | Two-phase generation: execute via MCP first, convert to code |
 | MCP proactive healing | Validate selectors against live page before test runs |
+| RAG enabled | Master switch for semantic memory from past runs (ON by default) |
+| RAG Code Corpus | Remember instruction-to-code mappings from successful runs |
+| RAG Healing Corpus | Remember past healing fixes for smarter self-repair |
+| RAG Error Catalog | Track recurring error patterns to avoid known-bad selectors |
+| RAG Embedder | Similarity engine: TF-IDF (local, free, offline) |
 
 Changes are saved immediately to both `autotest.config.json` and `autotest.config.ts`.
 
@@ -390,6 +502,78 @@ If MCP fails at any point (browser crash, timeout, network issue), generation si
 
 ---
 
+## RAG Memory (Retrieval-Augmented Generation)
+
+RAG gives the AI **semantic memory** from past test runs. Every successful step, every healing fix, and every recurring error is indexed so the AI can retrieve similar experiences and generate better code over time.
+
+### Three Corpora
+
+| Corpus | What it stores | When it's used |
+|--------|---------------|----------------|
+| **Code Corpus** | Instruction-to-code mappings from successful runs | During generation — similar past steps retrieved as examples (score 0.65-0.90) or used directly as fuzzy cache hits (score >= 0.90) |
+| **Healing Corpus** | Past healing events (instruction + error + fix) | During self-healing — proven past fixes injected into Level 2 repair prompt |
+| **Error Catalog** | Recurring error patterns per URL with fix history | During generation — AI warned about known-bad selectors/patterns to avoid |
+
+### How It Works
+
+1. **Step passes** → the instruction + code are ingested into the Code Corpus.
+2. **Step fails** → the error pattern is recorded in the Error Catalog.
+3. **Healing succeeds** → the fix is ingested into the Healing Corpus, and the Error Catalog is updated with the fix.
+4. **Next generation** → the SmartRouter queries the Code Corpus. High-confidence matches (>= 90%) are used directly ($0, instant). Lower matches (65-90%) are passed as examples in the AI prompt.
+5. **Next healing** → Level 2 queries the Healing Corpus for similar past fixes and appends them to the repair instruction.
+
+### Storage
+
+All RAG data is stored as JSON files in `results/.rag/`:
+- `idf-vocab.json` — TF-IDF learned vocabulary
+- `code-corpus.json` — instruction → code mappings
+- `healing-corpus.json` — failure → fix mappings
+- `error-catalog.json` — recurring error patterns
+
+### Consumer Experience
+
+**RAG is ON by default — zero setup required.** It works automatically from the very first run.
+
+- **Run 1** — memory is empty, AI generates code normally
+- **Run 2+** — RAG kicks in silently: similar instructions are retrieved instead of making API calls (free + faster), healing uses proven past fixes
+- **Run 10+** — most common steps are served from RAG memory at zero cost
+
+RAG data is stored in `results/.rag/` as plain JSON — fully Git-friendly. Commit it to share memory across your team, or cache it in CI to persist between runs.
+
+### FAQ
+
+| Question | Answer |
+|----------|--------|
+| Do I need to configure anything? | No. RAG is ON by default with zero setup. |
+| Does it cost anything? | No. TF-IDF embedder runs locally. RAG hits replace paid AI calls. |
+| Does it slow down tests? | No. RAG lookup is <1ms. It speeds up generation. |
+| Works in CI/CD? | Yes. Cache `results/.rag/` between runs to persist memory. |
+| Share memory across team? | Commit `results/.rag/` to Git or use a CI cache step. |
+| How to reset? | Delete the `results/.rag/` folder. |
+
+### When to Use the Settings Card
+
+Most users never need to touch RAG settings. The **Settings → RAG Memory** card exists for power-user scenarios:
+
+| Scenario | Action |
+|----------|--------|
+| Debugging a flaky test | Turn OFF Code Corpus — forces fresh AI generation instead of reusing a possibly stale mapping |
+| Healing keeps suggesting a bad fix | Turn OFF Healing Corpus — clears the influence of a bad past fix |
+| Major app redesign (UI overhaul) | Turn OFF RAG entirely — old memory from the previous UI is now misleading |
+| Error warnings are outdated | Turn OFF Error Catalog — stops avoiding selectors that are fine now |
+| Want deterministic CI runs | Disable RAG in CI config, keep ON for local development |
+
+### Settings Reference
+
+Go to **Settings → RAG Memory** to configure:
+- **Enable RAG** — Master switch (ON by default)
+- **Code Corpus** — Toggle instruction-to-code memory
+- **Healing Corpus** — Toggle healing fix memory
+- **Error Catalog** — Toggle error pattern tracking
+- **Embedder** — TF-IDF (local, free, offline) — no API calls needed
+
+---
+
 ## WebSocket Live Updates
 
 The Studio uses a WebSocket connection (`ws://localhost:<port>/ws`) to receive live updates:
@@ -398,6 +582,8 @@ The Studio uses a WebSocket connection (`ws://localhost:<port>/ws`) to receive l
 |-------|-------------|
 | `run:complete` | A run has finished — results available |
 | `run:error` | A run failed to start |
+| `recorder:action` | A new action was recorded (click, fill, navigate, assert) |
+| `recorder:stopped` | Recording session ended |
 
 The connection indicator in the top bar shows:
 - **Green dot** — connected

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "assuremind",
-  "version": "1.1.2",
+  "version": "1.2.0",
   "description": "AI-powered codeless UI & API automation framework",
   "author": "Deepak Hiremath",
   "license": "MIT",