elementus-ai 1.2.0 → 1.5.0

package/CHANGELOG.md

@@ -2,6 +2,49 @@
 
 All notable changes are documented here. This project adheres to [Semantic Versioning](https://semver.org).
 
+## 1.5.0
+
+### Added
+
+- `HealEvent.suggestion` — a ready-to-use, framework-native, properly-escaped suggested replacement
+  locator (`getByRole('link', { name: 'Save' })` on Playwright, `$('aria/Save')` on WDIO, `$('~Save')`
+  on Appium). Surface it verbatim. This moves suggestion-building into the library so consumers stop
+  hand-rolling it from `resolved` — hand-rolling re-introduced quote-escaping bugs (text with
+  apostrophes) and bad role mapping (form controls suggested as `getByText`). `resolved` stays for
+  consumers that want the raw facts.
+
+### Changed
+
+- Additive on the existing `onHeal` event; no behavior change.
+
+## 1.4.0
+
+### Added
+
+- `HealEvent.resolved` — `{ tag, text, role }` of the element a heal resolved to (when known), so a
+  consumer can compose a **suggested replacement locator** (e.g. `getByRole(role, { name: text })`) and
+  show "selector X failed → replace with Y" in reports. Absent on coordinate-only vision heals.
+
+### Changed
+
+- Additive on the existing `onHeal` event; no behavior change.
+
+## 1.3.0
+
+### Added
+
+- `onHeal(event)` config callback — fires once per heal with `{ description, selector, method, framework }`
+  so consumers can surface healed locators (report annotations, CI fail-on-heal). Framework-agnostic:
+  elementus emits the event; wiring it to a reporter is consumer-side (README "Detecting heals" has
+  Playwright + WebdriverIO/Appium recipes).
+- `HealEvent` exported type (`index.d.ts`); `onHeal` added to `ElementusOptions`.
+- Test `T47` covering onHeal; docs in README, `docs/API.md`, `docs/ARCHITECTURE.md`.
+
+### Changed
+
+- No change to existing behavior. `onHeal` is opt-in; unset ⇒ byte-for-byte the same as 1.2.0
+  (zero-overhead path untouched, single export preserved).
+
 ## 1.2.0
 
 ### Added

package/README.md

@@ -34,11 +34,24 @@ I just installed the npm package "elementus-ai" — a self-healing element resol
 
 3. INTEGRATE BASED ON MY FRAMEWORK
 
+   First create the elementus instance using the provider chosen in step 2:
+     const { createElementus } = require('elementus-ai')   // ESM/TS: import { createElementus } from 'elementus-ai'
+     const el = createElementus({ provider: 'gemini', geminiApiKey: process.env.GEMINI_API_KEY })
+     // LM Studio instead: createElementus({ provider: 'lmstudio', lmStudioUrl: 'http://localhost:1234/v1/chat/completions', model: 'holo-3.1-9b' })
+   For exact, copy-pasteable code (TypeScript fixture, onHeal recipe), read the installed package's
+   node_modules/elementus-ai/README.md ("Quick Start", "TypeScript", "Detecting heals") and docs/API.md.
+
    For Playwright:
-   - Create or update a fixtures file that wraps page with el.wrapPage(page)
+   - Create or update a fixtures file that overrides the page fixture with el.wrapPage(page)
    - Make sure all tests import from the fixtures file instead of @playwright/test
    - TypeScript projects: use import/export and type the override as base.extend<{ page: ElementusPage }>({ ... }) (import ElementusPage from elementus-ai) so { ai } is autocompleted and documented. Types are bundled — do NOT add @types or a "declare module" shim
    - Set actionTimeout: 10000 in playwright config (Elementus respects framework timeouts)
+   - Optional heal visibility: only if the user wants healed locators surfaced, wire onHeal to the
+     framework's reporter. Use e.suggestion — a ready-to-use, escaped, framework-native replacement
+     locator — verbatim. Playwright: push a testInfo "healed" annotation (HTML report) AND
+     testInfo.attach (so it also reaches Allure + custom reporters); optionally fail CI on heal via an
+     env-gated afterEach/fixture. WebdriverIO/Appium: collect in onHeal, fail in afterTest. See
+     "Detecting heals" in the README. Default off
 
    For WebDriverIO:
    - In wdio.conf.js before hook, wrap browser and override global $:
@@ -320,9 +333,85 @@ createElementus({
 
   // Custom stop words
   stopWords: null,          // Set of words to ignore in descriptions
+
+  // Heal telemetry (opt-in) — called once per heal with { description, selector, method, framework, resolved, suggestion }.
+  // Best-effort & isolated: a throwing callback never breaks a test. See "Detecting heals" below.
+  onHeal: null,             // e.g. (e) => console.log('healed', e.selector, '→', e.description)
 })
 ```
 
+## Detecting heals (reporting & CI)
+
+A heal is silent by default — the test still passes. To surface it, pass an `onHeal(event)` callback;
+elementus calls it **once per heal** with `{ description, selector, method, framework, resolved, suggestion }`. The library only
+*emits* the event — turning it into a report annotation or a CI failure is framework-specific (a few lines
+on your side). It fires when the AI resolves a replacement element, so it signals "the selector drifted and
+AI stepped in", not "the action ultimately passed".
+
+**Playwright** — annotate the test (shows in the Playwright HTML report), and optionally fail
+CI on heal:
+
+```javascript
+// fixtures.js
+const { test: base } = require('@playwright/test')
+const { createElementus } = require('elementus-ai')
+
+const el = createElementus({
+  provider: 'gemini',
+  geminiApiKey: process.env.GEMINI_API_KEY,
+  onHeal: (e) => {
+    // e.suggestion is a ready-to-use, escaped, framework-native replacement locator — use it verbatim.
+    const line = `${e.selector}  →  ${e.suggestion ?? '(located visually — no DOM suggestion)'}`
+    try {
+      const info = base.info()
+      info.annotations.push({ type: 'healed', description: line })          // Playwright HTML report
+      void info.attach('healed', { body: line, contentType: 'text/plain' }) // Allure + custom reporters
+    } catch {}  // ignored outside a running test
+  },
+})
+
+const test = base.extend({
+  page: async ({ page }, use) => { await use(el.wrapPage(page)) },
+  // Optional: fail CI when a selector only passed via self-healing (drift you should fix).
+  failOnHeal: [async ({}, use, testInfo) => {
+    await use()
+    const heals = testInfo.annotations.filter(a => a.type === 'healed')
+    if (process.env.ELEMENTUS_FAIL_ON_HEAL && heals.length) {
+      throw new Error('Passed only via self-healing — fix the selector(s):\n  ' + heals.map(h => h.description).join('\n  '))
+    }
+  }, { auto: true }],
+})
+module.exports = { test, expect: require('@playwright/test').expect }
+```
+
+(TypeScript: import the `HealEvent` type from `elementus-ai`; the annotation/fixture code is identical.)
+
+**WebdriverIO / Appium** — collect in `onHeal`, fail (or report) in `afterTest`:
+
+```javascript
+// wdio.conf.js
+const { createElementus } = require('elementus-ai')
+let heals = []
+const el = createElementus({ provider: 'gemini', geminiApiKey: '...', onHeal: (e) => heals.push(e) })
+
+exports.config = {
+  beforeTest() { heals = [] },
+  afterTest()  {
+    if (process.env.ELEMENTUS_FAIL_ON_HEAL && heals.length) {
+      throw new Error('healed: ' + heals.map(h => h.description).join(', '))
+    }
+  },
+}
+```
+
+**Where each report shows it:** the annotation appears in the **Playwright HTML report** (test → _Annotations_).
+In **Allure**, the heal shows in the test's **step tree** (the broken locator → the healed `[data-elementus=…]`
+element) and, with fail-on-heal on, the **failure message** — note `allure-playwright` does not convert
+runtime annotations into labels/steps, so for a dedicated Allure marker also
+`testInfo.attach('healed', { body: \`${e.selector} → ${e.description}\`, contentType: 'text/plain' })`
+(attachments surface in Allure and most custom reporters). A custom HTML reporter surfaces the heal via the
+failure message and captured stdout.
+
 ## Security Notes
 
 - **Debug screenshots** capture the full page — including any sensitive data visible on it. Keep `debugDir` out of version control.

package/elementus.js

@@ -377,6 +377,7 @@ const REGION_LABELS = [
  * @param {number} [userConfig.visionMaxWidth=1280] - max screenshot width (px) sent to vision LLM
  * @param {string|null} [userConfig.cacheFile=null] - opt-in fingerprint cache file (e.g. './elementus-cache.json')
  * @param {string|null} [userConfig.embeddingModel=null] - opt-in embedding model for semantic matching
+ * @param {(event: { description: string, selector: string, method: string, framework: string, resolved?: { tag: string, text: string, role: string|null }, suggestion?: string }) => void} [userConfig.onHeal] - called after a heal resolves a replacement element (original selector failed, AI found it). `resolved` is the element it healed to; `suggestion` is a ready-to-use, framework-native replacement locator (use it verbatim). Best-effort; throwing never breaks a test.
  * @returns {{ wrap, wrapPage, wrapBrowser, locate, find, click }}
  */
 function createElementus(userConfig = {}) {
@@ -1988,6 +1989,64 @@ function createElementus(userConfig = {}) {
     return { record: null, somCandidates: out.somCandidates || null }
   }
 
+  // Facts about the element the last heal resolved to (tag/text/role) — set during resolution,
+  // read once by _notifyHeal. Framework-agnostic; consumers format a suggested replacement locator.
+  let _lastResolved = null
+  function _resolvedFacts(record) {
+    if (!record) return null
+    return { tag: record.tag || '', text: (record.text || '').trim(), role: record.role || null }
+  }
+
+  // Implicit ARIA roles for the common interactive tags (used when the element has no explicit
+  // role attribute). `input` is deliberately omitted — its role depends on its type.
+  const ROLE_BY_TAG = { a: 'link', button: 'button', select: 'combobox', textarea: 'textbox' }
+  // Escape a value for a single-quoted string literal in the suggested locator.
+  function _escSingle(s) {
+    return String(s).replace(/\\/g, '\\\\').replace(/'/g, "\\'")
+  }
+  // Collapse whitespace and cap length so the suggested locator stays one readable line.
+  function _suggestName(s) {
+    return String(s || '').replace(/\s+/g, ' ').trim().slice(0, 80)
+  }
+  // Build a ready-to-use, framework-native, properly-escaped suggested replacement locator from
+  // the resolved element. Consumers should surface event.suggestion verbatim — do NOT hand-roll it
+  // (raw text interpolation re-introduces quote-escaping and role-mapping bugs).
+  function _suggestLocator(framework, resolved) {
+    if (!resolved) return undefined
+    const tag = resolved.tag || ''
+    const name = _suggestName(resolved.text)
+    const role = resolved.role || ROLE_BY_TAG[tag] || null
+    if (framework === 'playwright') {
+      if (role && name) return `getByRole('${_escSingle(role)}', { name: '${_escSingle(name)}' })`
+      if (name) return `getByText('${_escSingle(name)}')`
+      return `locator('${_escSingle(tag || '*')}')`
+    }
+    if (framework === 'appium') {
+      return name ? `$('~${_escSingle(name)}')` : `$('${_escSingle(tag || '*')}')`
+    }
+    return name ? `$('aria/${_escSingle(name)}')` : `$('${_escSingle(tag || '*')}')`
+  }
+
+  // Fire the user's onHeal(event) after a real heal — the original selector failed and the AI
+  // pipeline resolved a replacement. Best-effort and fully isolated: a throwing or rejecting
+  // callback must never break a test. No-op with zero overhead when onHeal is not configured.
+  function _notifyHeal(ctx, description, selectorKey, method) {
+    const resolved = _lastResolved
+    _lastResolved = null
+    const cb = config.onHeal
+    if (typeof cb !== 'function') return
+    const framework = _isNative(ctx) ? 'appium' : _isPlaywright(ctx) ? 'playwright' : 'wdio'
+    try {
+      const event = { description, selector: selectorKey, method, framework }
+      if (resolved) {
+        event.resolved = resolved
+        event.suggestion = _suggestLocator(framework, resolved)
+      }
+      const r = cb(event)
+      if (r && typeof r.then === 'function') r.then(undefined, () => {})
+    } catch {}
+  }
+
   async function _findByDescription(ctx, description, selectorKey = '') {
     const { record, somCandidates } = await _resolveElement(ctx, description, selectorKey)
     if (record) {
@@ -1995,6 +2054,7 @@ function createElementus(userConfig = {}) {
         const mark = {}
         const locator = record._locator || await markByElement(ctx, record, mark)
         await _cacheStore(ctx, description, selectorKey, record, record._uid || mark.uid || null)
+        _lastResolved = _resolvedFacts(record)
         return locator
       } catch (err) {
         console.log(`[Resolve] Mark failed (${err.message}) — trying vision`)
@@ -2006,11 +2066,13 @@ function createElementus(userConfig = {}) {
         const mark = {}
         const locator = await markByElement(ctx, result.element, mark)
         await _cacheStore(ctx, description, selectorKey, result.element, mark.uid || null)
+        _lastResolved = _resolvedFacts(result.element)
         return locator
       }
       const mark = {}
       const locator = await markAtCoordinates(ctx, result.coords.docX, result.coords.docY, mark)
       await _cacheStore(ctx, description, selectorKey, result.coords, mark.uid || null)
+      _lastResolved = null
       return locator
     } catch (err) {
       throw new Error(`All fallback paths exhausted for "${description}": ${err.message}`)
@@ -2038,7 +2100,10 @@ function createElementus(userConfig = {}) {
     } catch {
       console.log(`\u2717 Locator failed \u2014 searching for: "${description}"`)
     }
-    return _findByDescription(ctx, description, _selectorKey(locator))
+    const selectorKey = _selectorKey(locator)
+    const resolved = await _findByDescription(ctx, description, selectorKey)
+    _notifyHeal(ctx, description, selectorKey, 'locate')
+    return resolved
   }
 
   /**
@@ -2088,6 +2153,8 @@ function createElementus(userConfig = {}) {
       // Store before clicking \u2014 the click may navigate away from the page
       await _cacheStore(ctx, description, selectorKey, record)
       await scrollAndClick(ctx, record)
+      _lastResolved = _resolvedFacts(record)
+      _notifyHeal(ctx, description, selectorKey, 'click')
       return
     }
     try {
@@ -2095,9 +2162,13 @@ function createElementus(userConfig = {}) {
       if (result.element) {
         await _cacheStore(ctx, description, selectorKey, result.element)
         await scrollAndClick(ctx, result.element)
+        _lastResolved = _resolvedFacts(result.element)
+        _notifyHeal(ctx, description, selectorKey, 'click')
         return
       }
       await clickAtCoords(ctx, result.coords)
+      _lastResolved = null
+      _notifyHeal(ctx, description, selectorKey, 'click')
     } catch (err) {
       throw new Error(`All fallback paths exhausted for "${description}": ${err.message}`)
     }
@@ -2160,6 +2231,7 @@ function createElementus(userConfig = {}) {
             if (!_resolved) {
               console.log(`[AI] ${prop}() \u2014 resolving via AI first for "${description}"`)
               _resolved = await _findByDescription(driverContext, description, wrapSelectorKey)
+              _notifyHeal(driverContext, description, wrapSelectorKey, String(prop))
             }
             return _resolved[prop](...args)
           }
@@ -2168,7 +2240,10 @@ function createElementus(userConfig = {}) {
             return await original.apply(target, args)
           } catch (firstError) {
             console.log(`[AI] ${String(prop)}() failed \u2014 AI fallback for "${description}"`)
-            if (!_resolved) _resolved = await _findByDescription(driverContext, description, wrapSelectorKey)
+            if (!_resolved) {
+              _resolved = await _findByDescription(driverContext, description, wrapSelectorKey)
+              _notifyHeal(driverContext, description, wrapSelectorKey, String(prop))
+            }
 
             const resolvedMethod = _resolved[prop]
             if (typeof resolvedMethod !== 'function') {

package/index.d.ts

@@ -36,6 +36,33 @@ export interface ElementusOptions {
   cacheFile?: string | null
   /** Opt-in embedding model for semantic paraphrase matching. @default null */
   embeddingModel?: string | null
+  /**
+   * Called after a heal resolves a replacement element (the original selector failed and the AI
+   * pipeline found a match). Best-effort and isolated — a throwing/rejecting callback never breaks
+   * a test. Fires at resolution time, so it does NOT guarantee the subsequent action succeeded.
+   */
+  onHeal?: (event: HealEvent) => void | Promise<void>
+}
+
+/** Event passed to `onHeal` when a broken selector is healed. */
+export interface HealEvent {
+  /** The natural-language `{ ai }` description that resolved the element. */
+  description: string
+  /** The original selector that failed (selector key). */
+  selector: string
+  /** The action that triggered the heal: 'click', 'fill', 'isVisible', 'locate', etc. */
+  method: string
+  /** Which driver the heal happened on. */
+  framework: 'playwright' | 'wdio' | 'appium'
+  /** The element elementus resolved to (when known — absent on coordinate-only vision heals). */
+  resolved?: { tag: string; text: string; role: string | null }
+  /**
+   * Ready-to-use, framework-native, properly-escaped suggested replacement locator for the failed
+   * selector (e.g. `getByRole('link', { name: 'Save' })` on Playwright, `$('aria/Save')` on WDIO).
+   * Surface it verbatim — don't hand-build one from `resolved` (that re-introduces quote-escaping
+   * and role-mapping bugs). Absent on coordinate-only vision heals.
+   */
+  suggestion?: string
 }
 
 /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "elementus-ai",
-  "version": "1.2.0",
+  "version": "1.5.0",
   "description": "Self-healing element resolution for Playwright, WDIO & Appium. AI-powered fallback when selectors break.",
   "main": "elementus.js",
   "types": "index.d.ts",