elementus-ai 1.1.0 → 1.2.0

package/CHANGELOG.md

@@ -0,0 +1,21 @@
+# Changelog
+
+All notable changes are documented here. This project adheres to [Semantic Versioning](https://semver.org).
+
+## 1.2.0
+
+### Added
+
+- Bundled TypeScript type definitions (`index.d.ts`), exposed via the package `types` field — no `@types/...` package or `declare module` shim required.
+- Exported types: `createElementus`, `ElementusOptions`, `Elementus`, `ElementusPage`, `AiLocatorOptions`.
+- `ElementusPage` types `page.locator(selector, { ai })` by extending Playwright's own locator options, so the `{ ai }` hint type-checks while plain locators and native options keep working.
+- `@playwright/test` types resolve as an optional peer — WDIO/Appium-only projects don't need Playwright installed.
+- README: new **TypeScript** section (typed fixture + Page Object Model patterns); the One-Prompt Setup is now TypeScript-aware.
+
+### Changed
+
+- No runtime changes. `elementus.js` is untouched; this release is type definitions and documentation only.
+
+## 1.1.1
+
+- Previous release.

package/README.md

@@ -24,18 +24,20 @@ I just installed the npm package "elementus-ai" — a self-healing element resol
    - Search for: playwright.config, wdio.conf, appium config files
    - Check package.json for: @playwright/test, playwright, webdriverio, wdio, appium
    - Read a few existing test files to understand the test structure
+   - Note whether the project is TypeScript (tsconfig.json or .ts test files) — this changes the fixture syntax (see step 3)
    - If none found, tell me you can't detect a supported framework and stop
 
 2. CHOOSE THE LLM PROVIDER
-   - Ask me: "Do you want to use a local LLM (LM Studio, free, private) or Google Gemini (cloud, fast, ~$0.01/500 tests)?"
+   - Ask me: "Do you want to use a local LLM (LM Studio, free, private) or Google Gemini (cloud, fast, ~$0.001 per AI-healed selector on gemini-3.5-flash; selectors that still work cost nothing)?"
    - If Gemini: ask for API key or check for GEMINI_API_KEY env var
-   - If LM Studio: use defaults (localhost:1234, gemma model)
+   - If LM Studio: use defaults (localhost:1234) with a vision/grounding model loaded (recommended: holo-3.1-9b)
 
 3. INTEGRATE BASED ON MY FRAMEWORK
 
    For Playwright:
    - Create or update a fixtures file that wraps page with el.wrapPage(page)
    - Make sure all tests import from the fixtures file instead of @playwright/test
+   - TypeScript projects: use import/export and type the override as base.extend<{ page: ElementusPage }>({ ... }) (import ElementusPage from elementus-ai) so { ai } is autocompleted and documented. Types are bundled — do NOT add @types or a "declare module" shim
    - Set actionTimeout: 10000 in playwright config (Elementus respects framework timeouts)
 
    For WebDriverIO:
@@ -90,24 +92,26 @@ await p.locator('#submit-btn', { ai: 'Submit order button' }).click()
 await p.locator('#stable-element').click()
 ```
 
+> **Using TypeScript or ESM?** `import { createElementus } from 'elementus-ai'` — type definitions are bundled. See [TypeScript](#typescript) for the typed fixture pattern.
+
 ## LLM Provider Setup
 
 ### Option A: Local LLM via LM Studio (free, private)
 
 1. Download [LM Studio](https://lmstudio.ai)
-2. Load a vision-capable model (e.g., `gemma-4-26b-a4b-it`)
+2. Load a vision-capable model. Recommended: **`holo-3.1-9b`** — a GUI-grounding model that locates on-screen elements far better than general chat VLMs, and it's small (9B). Any vision model works, but grounding models earn their keep on the vision-fallback path.
 3. Start the local server (default: `http://localhost:1234`)
 
 ```javascript
 const el = createElementus({
   provider: 'lmstudio',
   lmStudioUrl: 'http://localhost:1234/v1/chat/completions',
-  model: 'gemma-4-26b-a4b-it',
+  model: 'holo-3.1-9b',
 })
 ```
 
 Tips for the local setup:
-- **Vision accuracy:** a dedicated GUI-grounding model (e.g. `Holo2-8B`, Apache-2.0 GGUF on Hugging Face) typically grounds screen coordinates better than general chat VLMs — benchmark numbers are vendor-reported (Nov 2025), verify it loads in your LM Studio version before switching.
+- **Context length:** set it to 16k+ in LM Studio — the ARIA-snapshot grounding step can send large prompts, and the default 4k will silently truncate.
 - **Semantic matching:** load an embedding model (e.g. `text-embedding-nomic-embed-text-v1.5`) and set `embeddingModel` to let paraphrased descriptions ("sign in" vs "log in") resolve without vision.
 
 ### Option B: Google Gemini API (cloud, fast, better vision)
@@ -193,6 +197,55 @@ await d.$('~emailField', { ai: 'Email input' }).setValue('test@test.com')
 
 Works with Flutter, React Native, native Android/iOS — any Appium driver.
 
+## TypeScript
+
+Type definitions are bundled — there is no `@types/elementus-ai` package to install and no `declare module` shim to write. Because `@playwright/test` is an *optional* peer, WDIO/Appium-only projects can use the types without installing Playwright.
+
+```ts
+import { createElementus, type ElementusPage } from 'elementus-ai'
+```
+
+**Typed Playwright fixture.** `wrapPage` changes the page's runtime value but not its static type, so override the `page` fixture's type with `ElementusPage` — then `{ ai }` is recognized and autocompleted (with docs) in your tests:
+
+```ts
+// fixtures.ts
+import { test as base, expect } from '@playwright/test'
+import { createElementus, type ElementusPage } from 'elementus-ai'
+
+const el = createElementus({ provider: 'gemini', geminiApiKey: process.env.GEMINI_API_KEY })
+
+export const test = base.extend<{ page: ElementusPage }>({
+  page: async ({ page }, use) => {
+    await use(el.wrapPage(page))
+  },
+})
+export { expect }
+
+// In tests — page is already wrapped and typed:
+test('example', async ({ page }) => {
+  await page.locator('#btn', { ai: 'Submit button' }).click() // { ai } type-checks
+  await page.locator('#btn').click()                          // plain locator, zero overhead
+})
+```
+
+> The override is for editor support — IntelliSense and inline docs for `{ ai }`. It heals at runtime either way, and because Playwright's `locator()` options are permissive, `{ ai }` compiles with or without the override; the override just surfaces it as a documented option.
+
+**Page Object Model.** Type the page your objects receive as `ElementusPage`:
+
+```ts
+import { type ElementusPage } from 'elementus-ai'
+
+abstract class BasePage {
+  constructor(protected readonly page: ElementusPage) {}
+}
+
+class LoginPage extends BasePage {
+  readonly submit = this.page.locator('#submit', { ai: 'Submit button' })
+}
+```
+
+**Exported types:** `ElementusOptions`, `Elementus`, `ElementusPage`, `AiLocatorOptions`. `AiLocatorOptions` is Playwright's own `locator()` option type plus `ai?: string`, derived from the installed Playwright version so it never drifts.
+
 ## API Reference
 
 ### `el.wrapPage(page)`
@@ -243,7 +296,7 @@ createElementus({
 
   // LM Studio
   lmStudioUrl: 'http://localhost:1234/v1/chat/completions',
-  model: 'gemma-4-26b-a4b-it',
+  model: 'holo-3.1-9b',
 
   // Gemini
   geminiApiKey: null,       // or GEMINI_API_KEY env var

package/elementus.js

@@ -20,13 +20,13 @@
  *
  * Option A — Local LLM via LM Studio (free, private, no API key):
  *   1. Download LM Studio from https://lmstudio.ai
- *   2. Load a vision-capable model (e.g., gemma-4-26b-a4b-it)
+ *   2. Load a vision-capable model (recommended: holo-3.1-9b, a GUI-grounding model)
  *   3. Start the local server (default: http://localhost:1234)
  *   4. Configure:
  *        const el = createElementus({
  *          provider: 'lmstudio',
  *          lmStudioUrl: 'http://localhost:1234/v1/chat/completions',
- *          model: 'gemma-4-26b-a4b-it',
+ *          model: 'holo-3.1-9b',
  *        })
  *
  * Option B — Google Gemini API (cloud, fast, better vision):
@@ -158,7 +158,7 @@
  *
  *   // LM Studio (when provider = 'lmstudio')
  *   lmStudioUrl: 'http://localhost:1234/v1/chat/completions',
- *   model: 'gemma-4-26b-a4b-it',
+ *   model: 'holo-3.1-9b',
  *
  *   // Gemini (when provider = 'gemini')
  *   geminiApiKey: null,       // or GEMINI_API_KEY env var
@@ -287,7 +287,7 @@ const path = require('path')
 const DEFAULTS = {
   provider: 'lmstudio',
   lmStudioUrl: 'http://localhost:1234/v1/chat/completions',
-  model: 'gemma-4-26b-a4b-it',
+  model: 'holo-3.1-9b',
   geminiApiKey: null,
   geminiModel: 'gemini-3.5-flash',
   maxCandidates: 20,
@@ -367,7 +367,7 @@ const REGION_LABELS = [
  * @param {Object} userConfig
  * @param {'lmstudio'|'gemini'} [userConfig.provider='lmstudio'] - LLM provider
  * @param {string} [userConfig.lmStudioUrl='http://localhost:1234/v1/chat/completions'] - LM Studio endpoint
- * @param {string} [userConfig.model='gemma-4-26b-a4b-it'] - LM Studio model name
+ * @param {string} [userConfig.model='holo-3.1-9b'] - LM Studio model name
  * @param {string|null} [userConfig.geminiApiKey=null] - Google Gemini API key (or GEMINI_API_KEY env var)
  * @param {string} [userConfig.geminiModel='gemini-3.5-flash'] - Gemini model ID
  * @param {number} [userConfig.maxCandidates=20] - max elements sent to LLM for disambiguation

package/index.d.ts

@@ -0,0 +1,81 @@
+// Type definitions for elementus-ai
+// Project: https://github.com/Morph93/elementus
+//
+// Self-healing element resolution for Playwright, WebdriverIO & Appium.
+// These types describe the Playwright/core API. WebdriverIO's global `$`
+// augmentation lives in the separate, opt-in `wdio.d.ts`.
+//
+// `@playwright/test` is an OPTIONAL peer dependency. The `@ts-ignore` below lets
+// WDIO/Appium-only consumers (who have no Playwright installed) fall back to
+// `any` for these types instead of failing module resolution.
+// @ts-ignore -- optional peer dependency
+import type { Page, Locator } from '@playwright/test'
+
+export interface ElementusOptions {
+  /** LLM provider. @default 'lmstudio' */
+  provider?: 'lmstudio' | 'gemini'
+  /** LM Studio chat-completions endpoint. @default 'http://localhost:1234/v1/chat/completions' */
+  lmStudioUrl?: string
+  /** LM Studio model name. @default 'holo-3.1-9b' */
+  model?: string
+  /** Google Gemini API key (or set the GEMINI_API_KEY env var). @default null */
+  geminiApiKey?: string | null
+  /** Gemini model id. @default 'gemini-3.5-flash' */
+  geminiModel?: string
+  /** Max elements sent to the LLM for disambiguation. @default 20 */
+  maxCandidates?: number
+  /** Save debug screenshots to `debugDir`. @default false */
+  debug?: boolean
+  /** Directory for debug screenshots (required when `debug` is true). @default null */
+  debugDir?: string | null
+  /** Custom stop words to ignore in descriptions (replaces the defaults). @default null */
+  stopWords?: Set<string> | null
+  /** Max screenshot width (px) sent to the vision LLM. @default 1280 */
+  visionMaxWidth?: number
+  /** Opt-in fingerprint cache file, e.g. './elementus-cache.json'. @default null */
+  cacheFile?: string | null
+  /** Opt-in embedding model for semantic paraphrase matching. @default null */
+  embeddingModel?: string | null
+}
+
+/**
+ * Playwright's own `locator()` options, plus the Elementus `ai` hint.
+ * Derived from the installed Playwright types so it never drifts.
+ */
+export type AiLocatorOptions = NonNullable<Parameters<Page['locator']>[1]> & {
+  /** Natural-language description; the self-healing fallback used when `selector` breaks. */
+  ai?: string
+}
+
+/**
+ * A Playwright Page whose `locator()` also accepts `{ ai }`. Locators created
+ * with an `ai` hint self-heal when the selector breaks; locators without it are
+ * returned unchanged (zero overhead).
+ */
+export type ElementusPage = Page & {
+  locator(selector: string, options?: AiLocatorOptions): Locator
+}
+
+export interface Elementus {
+  /**
+   * Wrap a Playwright Page so `page.locator(selector, { ai })` self-heals.
+   * Call once per test, or in a fixture for the whole suite.
+   */
+  wrapPage(page: Page): ElementusPage
+  /**
+   * Wrap a WebdriverIO/Appium browser so `$(selector, { ai })` self-heals.
+   * Returns the same object it was given (now AI-aware).
+   */
+  wrapBrowser<T>(browser: T): T
+  /** Try `locator` first; fall back to AI resolution if it fails. */
+  locate(ctx: Page, locator: Locator, description: string): Promise<Locator>
+  /** Resolve an element from a natural-language description alone. */
+  find(ctx: Page, description: string): Promise<Locator>
+  /** Click with an optimized fallback (goto for links, JS click for buttons). */
+  click(ctx: Page, locator: Locator, description: string): Promise<void>
+  /** Low-level: wrap a single locator with AI fallback. Prefer wrapPage(). */
+  wrap(ctx: Page, locator: Locator, description: string): Locator
+}
+
+/** Create an Elementus instance with the given configuration. */
+export function createElementus(options?: ElementusOptions): Elementus

package/package.json

@@ -1,8 +1,9 @@
 {
   "name": "elementus-ai",
-  "version": "1.1.0",
+  "version": "1.2.0",
   "description": "Self-healing element resolution for Playwright, WDIO & Appium. AI-powered fallback when selectors break.",
   "main": "elementus.js",
+  "types": "index.d.ts",
   "scripts": {
     "test": "playwright test test/playwright.spec.js",
     "test:smoke": "playwright test test/playwright.spec.js -g \"T01 |T02 |T09 |T17 |T23 \""
@@ -51,8 +52,10 @@
   },
   "files": [
     "elementus.js",
+    "index.d.ts",
     "wdio.d.ts",
     "README.md",
+    "CHANGELOG.md",
     "LICENSE"
   ]
 }