semantic-playwright 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,220 @@
1
+ # semantic-playwright
2
+
3
+ **Click by description, not selectors.**
4
+
5
+ A Playwright extension for semantic element targeting. Write resilient tests that survive website redesigns.
6
+
7
+ ## Installation
8
+
9
+ ```bash
10
+ npm install semantic-playwright playwright
11
+ ```
12
+
13
+ ## Quick Start
14
+
15
+ ```typescript
16
+ import { test } from '@playwright/test';
17
+ import { semantic } from 'semantic-playwright';
18
+
19
+ test('login flow', async ({ page }) => {
20
+ await page.goto('https://example.com/login');
21
+
22
+ const s = semantic(page);
23
+
24
+ // Click and fill by description - no CSS selectors needed
25
+ await s.fill('email input', 'user@example.com');
26
+ await s.fill('password field', 'secret123');
27
+ await s.click('sign in button');
28
+
29
+ // Wait for success
30
+ await s.waitFor('welcome message');
31
+ });
32
+ ```
33
+
34
+ ## Why?
35
+
36
+ Modern web UIs change constantly. CSS selectors like `#main-nav > ul > li:nth-child(3) > a.btn-primary` break the moment a designer moves a button. This library provides **self-healing locators** that target elements by their semantic purpose.
37
+
38
+ ```typescript
39
+ // Fragile - breaks on redesign
40
+ await page.click('#nav-signin-btn');
41
+
42
+ // Resilient - survives redesign
43
+ await s.click('sign in button');
44
+ ```
45
+
46
+ ## How It Works
47
+
48
+ The semantic engine classifies DOM elements using multiple signals:
49
+ - **ARIA roles**: `role="button"`, `role="navigation"`
50
+ - **Semantic HTML**: `<nav>`, `<main>`, `<button>`, `<form>`
51
+ - **Text content**: "Sign In", "Submit", "Search"
52
+ - **Class patterns**: `btn`, `nav`, `form-input`
53
+ - **Context**: Element inside `<nav>` likely navigation-related
54
+
55
+ **No LLM required.** Pure heuristic matching. Fast, local, privacy-preserving.
56
+
57
+ ## API
58
+
59
+ ### `semantic(page)`
60
+
61
+ Wrap a Playwright page with semantic methods:
62
+
63
+ ```typescript
64
+ const s = semantic(page);
65
+ ```
66
+
67
+ ### `s.click(description, options?)`
68
+
69
+ Click an element by description:
70
+
71
+ ```typescript
72
+ await s.click('submit button');
73
+ await s.click('login', { purposeHint: 'action' });
74
+ ```
75
+
76
+ ### `s.fill(description, value, options?)`
77
+
78
+ Fill a form field by description:
79
+
80
+ ```typescript
81
+ await s.fill('email input', 'test@example.com');
82
+ await s.fill('search box', 'playwright');
83
+ ```
84
+
85
+ ### `s.locator(description, options?)`
86
+
87
+ Get a Playwright Locator for chaining:
88
+
89
+ ```typescript
90
+ const btn = await s.locator('checkout button');
91
+ await btn.hover();
92
+ await btn.click();
93
+ ```
94
+
95
+ ### `s.analyze(includeUnknown?)`
96
+
97
+ Analyze page structure:
98
+
99
+ ```typescript
100
+ const { elements, summary } = await s.analyze();
101
+ console.log(summary);
102
+ // { navigation: 12, action: 8, form: 5, content: 20, data: 3, unknown: 0 }
103
+ ```
104
+
105
+ ### `s.query(category, textQuery?)`
106
+
107
+ Find elements by category:
108
+
109
+ ```typescript
110
+ const buttons = await s.query('action');
111
+ const searchInputs = await s.query('form', 'search');
112
+ ```
113
+
114
+ ### `s.findAll(description, options?)`
115
+
116
+ Find all matching elements with scores:
117
+
118
+ ```typescript
119
+ const matches = await s.findAll('submit');
120
+ for (const { locator, score, text } of matches) {
121
+ console.log(`${text}: ${score}`);
122
+ }
123
+ ```
124
+
125
+ ### `s.waitFor(description, options?)`
126
+
127
+ Wait for element to appear:
128
+
129
+ ```typescript
130
+ await s.waitFor('success notification', { timeout: 10000 });
131
+ ```
132
+
133
+ ## Categories
134
+
135
+ | Category | Description | Examples |
136
+ |----------|-------------|----------|
137
+ | `navigation` | Navigation elements | Nav bars, menus, breadcrumbs |
138
+ | `action` | Interactive actions | Buttons, CTAs, links styled as buttons |
139
+ | `form` | Form inputs | Text fields, selects, checkboxes |
140
+ | `content` | Main content | Articles, headings, paragraphs |
141
+ | `data` | Data display | Tables, lists, grids |
142
+
143
+ ## Options
144
+
145
+ ```typescript
146
+ interface SemanticOptions {
147
+ purposeHint?: 'navigation' | 'action' | 'form' | 'content' | 'data';
148
+ timeout?: number; // Default: 5000ms
149
+ minScore?: number; // Default: 20
150
+ }
151
+ ```
152
+
153
+ ## Architecture
154
+
155
+ ```
156
+ src/
157
+ ├── index.ts # Main API - SemanticPage wrapper
158
+ ├── semantic-mapper.ts # Classification engine & JS generators
159
+ ```
160
+
161
+ **Key design decisions:**
162
+ - Scripts execute in browser context via `page.evaluate()`
163
+ - Classification is entirely heuristic-based (no external calls)
164
+ - Selectors generated are full CSS paths for reliability
165
+ - Scoring system prefers exact matches, falls back gracefully
166
+
167
+ ## Development
168
+
169
+ ```bash
170
+ npm install
171
+ npm run build
172
+ npm test
173
+ ```
174
+
175
+ ### Testing Philosophy
176
+
177
+ Tests should use real websites to validate resilience:
178
+ 1. Test against sites that redesign frequently
179
+ 2. Snapshot selectors, verify semantic descriptions still work after updates
180
+ 3. Compare failure rates: CSS selectors vs semantic descriptions
181
+
182
+ ## Roadmap
183
+
184
+ ### v0.1.0 (Current)
185
+ - [x] Core semantic classification engine
186
+ - [x] `semantic()` page wrapper
187
+ - [x] `click()`, `fill()`, `locator()` methods
188
+ - [x] `analyze()` and `query()` methods
189
+ - [x] `findAll()` and `waitFor()` methods
190
+
191
+ ### v0.2.0
192
+ - [ ] Playwright Test fixtures (`useSemanticPage`)
193
+ - [ ] Custom matchers (`expect(s).toHaveElement('login button')`)
194
+ - [ ] Confidence thresholds configuration
195
+ - [ ] Debug mode with match explanations
196
+
197
+ ### v0.3.0
198
+ - [ ] Learning/correction persistence
199
+ - [ ] Domain-specific overrides
200
+ - [ ] Screenshot-on-failure with element highlighting
201
+ - [ ] Playwright Reporter integration
202
+
203
+ ### v1.0.0
204
+ - [ ] Comprehensive test suite
205
+ - [ ] Performance benchmarks
206
+ - [ ] Documentation site
207
+ - [ ] VS Code extension for selector conversion
208
+
209
+ ## Prior Art
210
+
211
+ This library is inspired by and aims to improve upon:
212
+ - **Browser-Use**: AI agent browser automation (requires LLM)
213
+ - **Healenium**: Self-healing Selenium (Java-focused)
214
+ - **Test.ai**: AI-powered testing (SaaS, expensive)
215
+
216
+ Our differentiator: **No LLM required**. Fast, local, free.
217
+
218
+ ## License
219
+
220
+ MIT
@@ -0,0 +1,130 @@
1
+ /**
2
+ * Semantic Playwright - Click by description, not selectors
3
+ *
4
+ * Extends Playwright with semantic element targeting that survives
5
+ * website redesigns. No LLM required - pure heuristic matching.
6
+ *
7
+ * @example
8
+ * ```typescript
9
+ * import { semantic } from 'semantic-playwright';
10
+ *
11
+ * // Wrap your page
12
+ * const s = semantic(page);
13
+ *
14
+ * // Click by description instead of selector
15
+ * await s.click('login button');
16
+ * await s.fill('email input', 'user@example.com');
17
+ *
18
+ * // Or use locators
19
+ * const loginBtn = await s.locator('login button');
20
+ * await loginBtn.click();
21
+ * ```
22
+ */
23
+ import type { Page, Locator } from 'playwright';
24
+ import { SemanticCategory, SemanticElement, ClickResult, QueryResult } from './semantic-mapper';
25
+ export { SemanticCategory, SemanticElement, ClickResult, QueryResult, } from './semantic-mapper';
26
+ /**
27
+ * Options for semantic operations
28
+ */
29
+ export interface SemanticOptions {
30
+ /** Hint about element purpose: navigation, action, form, content, data */
31
+ purposeHint?: SemanticCategory;
32
+ /** Timeout in milliseconds (default: 5000) */
33
+ timeout?: number;
34
+ /** Minimum score required for a match (default: 20) */
35
+ minScore?: number;
36
+ }
37
+ /**
38
+ * Options for semantic fill operations
39
+ */
40
+ export interface FillOptions extends SemanticOptions {
41
+ /** Clear existing content before filling (default: true) */
42
+ clear?: boolean;
43
+ }
44
+ /**
45
+ * Result of a semantic analysis
46
+ */
47
+ export interface AnalysisResult {
48
+ elements: SemanticElement[];
49
+ summary: {
50
+ navigation: number;
51
+ action: number;
52
+ form: number;
53
+ content: number;
54
+ data: number;
55
+ unknown: number;
56
+ };
57
+ }
58
+ /**
59
+ * Semantic page wrapper providing resilient element targeting
60
+ */
61
+ export interface SemanticPage {
62
+ /** The underlying Playwright page */
63
+ readonly page: Page;
64
+ /**
65
+ * Analyze the page and classify all elements by semantic purpose
66
+ * @param includeUnknown - Include elements with unknown category
67
+ * @returns Analysis result with elements and summary counts
68
+ */
69
+ analyze(includeUnknown?: boolean): Promise<AnalysisResult>;
70
+ /**
71
+ * Query elements by semantic category
72
+ * @param category - Element category to find
73
+ * @param textQuery - Optional text to filter by
74
+ * @returns Array of matching elements
75
+ */
76
+ query(category: SemanticCategory, textQuery?: string): Promise<QueryResult[]>;
77
+ /**
78
+ * Click an element by semantic description
79
+ * @param description - Natural language description (e.g., "login button")
80
+ * @param options - Optional targeting options
81
+ * @returns Result with clicked status and matched selector
82
+ */
83
+ click(description: string, options?: SemanticOptions): Promise<ClickResult>;
84
+ /**
85
+ * Fill a form field by semantic description
86
+ * @param description - Natural language description (e.g., "email input")
87
+ * @param value - Value to fill
88
+ * @param options - Optional fill options
89
+ * @returns Result with filled status and matched selector
90
+ */
91
+ fill(description: string, value: string, options?: FillOptions): Promise<{
92
+ filled: boolean;
93
+ selector?: string;
94
+ error?: string;
95
+ }>;
96
+ /**
97
+ * Get a Playwright Locator for an element by semantic description
98
+ * Useful when you need to chain Playwright methods
99
+ * @param description - Natural language description
100
+ * @param options - Optional targeting options
101
+ * @returns Playwright Locator for the matched element
102
+ */
103
+ locator(description: string, options?: SemanticOptions): Promise<Locator>;
104
+ /**
105
+ * Find all clickable elements matching a description
106
+ * @param description - Natural language description
107
+ * @param options - Optional targeting options
108
+ * @returns Array of matching locators with scores
109
+ */
110
+ findAll(description: string, options?: SemanticOptions): Promise<Array<{
111
+ locator: Locator;
112
+ score: number;
113
+ text: string | null;
114
+ }>>;
115
+ /**
116
+ * Wait for an element matching the description to appear
117
+ * @param description - Natural language description
118
+ * @param options - Optional targeting options
119
+ * @returns Locator for the appeared element
120
+ */
121
+ waitFor(description: string, options?: SemanticOptions): Promise<Locator>;
122
+ }
123
+ /**
124
+ * Create a semantic wrapper around a Playwright page
125
+ * @param page - Playwright Page instance
126
+ * @returns SemanticPage with semantic targeting methods
127
+ */
128
+ export declare function semantic(page: Page): SemanticPage;
129
+ export default semantic;
130
+ //# sourceMappingURL=index.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAEH,OAAO,KAAK,EAAE,IAAI,EAAE,OAAO,EAAE,MAAM,YAAY,CAAC;AAChD,OAAO,EACL,gBAAgB,EAChB,eAAe,EACf,WAAW,EACX,WAAW,EAKZ,MAAM,mBAAmB,CAAC;AAG3B,OAAO,EACL,gBAAgB,EAChB,eAAe,EACf,WAAW,EACX,WAAW,GACZ,MAAM,mBAAmB,CAAC;AAE3B;;GAEG;AACH,MAAM,WAAW,eAAe;IAC9B,0EAA0E;IAC1E,WAAW,CAAC,EAAE,gBAAgB,CAAC;IAC/B,8CAA8C;IAC9C,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,uDAAuD;IACvD,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;GAEG;AACH,MAAM,WAAW,WAAY,SAAQ,eAAe;IAClD,4DAA4D;IAC5D,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB;AAED;;GAEG;AACH,MAAM,WAAW,cAAc;IAC7B,QAAQ,EAAE,eAAe,EAAE,CAAC;IAC5B,OAAO,EAAE;QACP,UAAU,EAAE,MAAM,CAAC;QACnB,MAAM,EAAE,MAAM,CAAC;QACf,IAAI,EAAE,MAAM,CAAC;QACb,OAAO,EAAE,MAAM,CAAC;QAChB,IAAI,EAAE,MAAM,CAAC;QACb,OAAO,EAAE,MAAM,CAAC;KACjB,CAAC;CACH;AAED;;GAEG;AACH,MAAM,WAAW,YAAY;IAC3B,qCAAqC;IACrC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;IAEpB;;;;OAIG;IACH,OAAO,CAAC,cAAc,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,cAAc,CAAC,CAAC;IAE3D;;;;;OAKG;IACH,KAAK,CAAC,QAAQ,EAAE,gBAAgB,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC,CAAC;IAE9E;;;;;OAKG;IACH,KAAK,CAAC,WAAW,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;IAE5E;;;;;;OAMG;IACH,IAAI,CAAC,WAAW,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC;QAAE,MAAM,EAAE,OAAO,CAAC;QAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAEjI;;;;;;OAMG;IACH,OAAO,CAAC,WAAW,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAE1E;;;;;OAKG;IACH,OAAO,CAAC,WAAW,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,KAAK,CAAC;QAAE,OAAO,EAAE,OAAO,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC,CAAC,CAAC;IAElI;;;;;OAKG;IACH,OAAO,CAAC,WAAW,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;CAC3E;AAED;;;;GAIG;AACH,wBAAgB,QAAQ,CAAC,IAAI,EAAE,IAAI,GAAG,YAAY,CAEjD;AAiSD,eAAe,QAAQ,CAAC"}
package/dist/index.js ADDED
@@ -0,0 +1,296 @@
1
+ "use strict";
2
+ /**
3
+ * Semantic Playwright - Click by description, not selectors
4
+ *
5
+ * Extends Playwright with semantic element targeting that survives
6
+ * website redesigns. No LLM required - pure heuristic matching.
7
+ *
8
+ * @example
9
+ * ```typescript
10
+ * import { semantic } from 'semantic-playwright';
11
+ *
12
+ * // Wrap your page
13
+ * const s = semantic(page);
14
+ *
15
+ * // Click by description instead of selector
16
+ * await s.click('login button');
17
+ * await s.fill('email input', 'user@example.com');
18
+ *
19
+ * // Or use locators
20
+ * const loginBtn = await s.locator('login button');
21
+ * await loginBtn.click();
22
+ * ```
23
+ */
24
+ Object.defineProperty(exports, "__esModule", { value: true });
25
+ exports.semantic = semantic;
26
+ const semantic_mapper_1 = require("./semantic-mapper");
27
+ /**
28
+ * Create a semantic wrapper around a Playwright page
29
+ * @param page - Playwright Page instance
30
+ * @returns SemanticPage with semantic targeting methods
31
+ */
32
+ function semantic(page) {
33
+ return new SemanticPageImpl(page);
34
+ }
35
+ /**
36
+ * Implementation of SemanticPage
37
+ */
38
+ class SemanticPageImpl {
39
+ constructor(page) {
40
+ this.page = page;
41
+ }
42
+ async analyze(includeUnknown = false) {
43
+ const script = (0, semantic_mapper_1.generateAnalysisScript)(includeUnknown);
44
+ const elements = await this.page.evaluate(script);
45
+ const summary = {
46
+ navigation: 0,
47
+ action: 0,
48
+ form: 0,
49
+ content: 0,
50
+ data: 0,
51
+ unknown: 0,
52
+ };
53
+ for (const el of elements) {
54
+ summary[el.category]++;
55
+ }
56
+ return { elements, summary };
57
+ }
58
+ async query(category, textQuery) {
59
+ const script = (0, semantic_mapper_1.generateQueryScript)(category, textQuery);
60
+ return await this.page.evaluate(script);
61
+ }
62
+ async click(description, options = {}) {
63
+ const { purposeHint, timeout = 5000, minScore = 20 } = options;
64
+ // Generate and execute the click script
65
+ const script = (0, semantic_mapper_1.generateClickScript)(description, purposeHint);
66
+ // Wait for page to be ready
67
+ await this.page.waitForLoadState('domcontentloaded', { timeout });
68
+ const result = await this.page.evaluate(script);
69
+ // Validate minimum score
70
+ if (result.clicked && result.score !== undefined && result.score < minScore) {
71
+ return {
72
+ clicked: false,
73
+ error: `Match score ${result.score} below minimum ${minScore}`,
74
+ };
75
+ }
76
+ return result;
77
+ }
78
+ async fill(description, value, options = {}) {
79
+ const { timeout = 5000, clear = true } = options;
80
+ // Wait for page to be ready
81
+ await this.page.waitForLoadState('domcontentloaded', { timeout });
82
+ const script = (0, semantic_mapper_1.generateFillScript)(description, value);
83
+ const result = await this.page.evaluate(script);
84
+ if (result.filled && result.selector && clear) {
85
+ // The script already sets the value, but we can use Playwright's fill
86
+ // for better event handling if needed
87
+ try {
88
+ const element = await this.page.$(result.selector);
89
+ if (element) {
90
+ await element.fill(value);
91
+ }
92
+ }
93
+ catch {
94
+ // Script already filled it, ignore fill errors
95
+ }
96
+ }
97
+ return {
98
+ filled: result.filled,
99
+ selector: result.selector,
100
+ error: result.error,
101
+ };
102
+ }
103
+ async locator(description, options = {}) {
104
+ const { purposeHint, timeout = 5000 } = options;
105
+ // First, find the element using our semantic script
106
+ const findScript = `
107
+ (function() {
108
+ 'use strict';
109
+
110
+ const searchText = ${JSON.stringify(description.toLowerCase())};
111
+ const purposeHint = ${JSON.stringify(purposeHint || '')};
112
+
113
+ function getUniqueSelector(el) {
114
+ if (el.id) return '#' + CSS.escape(el.id);
115
+ let path = [];
116
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
117
+ let selector = el.tagName.toLowerCase();
118
+ if (el.id) { selector = '#' + CSS.escape(el.id); path.unshift(selector); break; }
119
+ let sibling = el, nth = 1;
120
+ while (sibling = sibling.previousElementSibling) { if (sibling.tagName === el.tagName) nth++; }
121
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) selector += ':nth-of-type(' + nth + ')';
122
+ path.unshift(selector);
123
+ el = el.parentElement;
124
+ }
125
+ return path.join(' > ');
126
+ }
127
+
128
+ function classify(el) {
129
+ const tag = el.tagName.toLowerCase();
130
+ const role = el.getAttribute('role');
131
+ if (tag === 'nav' || role === 'navigation') return 'navigation';
132
+ if (tag === 'button' || role === 'button' || el.type === 'submit') return 'action';
133
+ if (['input', 'textarea', 'select', 'form'].includes(tag)) return 'form';
134
+ if (tag === 'table' || role === 'table') return 'data';
135
+ if (['main', 'article', 'h1', 'p'].includes(tag)) return 'content';
136
+ return 'unknown';
137
+ }
138
+
139
+ function scoreMatch(el) {
140
+ const text = (el.textContent || '').toLowerCase();
141
+ const ariaLabel = (el.getAttribute('aria-label') || '').toLowerCase();
142
+ const title = (el.getAttribute('title') || '').toLowerCase();
143
+ const id = (el.id || '').toLowerCase();
144
+ const classes = Array.from(el.classList).join(' ').toLowerCase();
145
+ const placeholder = (el.placeholder || '').toLowerCase();
146
+ const name = (el.name || '').toLowerCase();
147
+
148
+ let score = 0;
149
+
150
+ if (text.trim() === searchText || ariaLabel === searchText) score += 100;
151
+ else if (text.includes(searchText) || ariaLabel.includes(searchText)) score += 50;
152
+ if (title.includes(searchText)) score += 30;
153
+ if (placeholder.includes(searchText)) score += 40;
154
+ if (id.includes(searchText.replace(/\\s+/g, '-'))) score += 25;
155
+ if (name.includes(searchText.replace(/\\s+/g, '_'))) score += 25;
156
+ if (classes.includes(searchText.replace(/\\s+/g, '-'))) score += 20;
157
+
158
+ if (purposeHint && classify(el) === purposeHint) score += 40;
159
+
160
+ return score;
161
+ }
162
+
163
+ const candidates = document.querySelectorAll('a, button, [role="button"], input, textarea, select, [onclick], [tabindex]');
164
+
165
+ let bestMatch = null;
166
+ let bestScore = 0;
167
+
168
+ candidates.forEach(el => {
169
+ const style = getComputedStyle(el);
170
+ if (style.display === 'none' || style.visibility === 'hidden') return;
171
+
172
+ const score = scoreMatch(el);
173
+ if (score > bestScore) {
174
+ bestScore = score;
175
+ bestMatch = el;
176
+ }
177
+ });
178
+
179
+ if (bestMatch && bestScore >= 20) {
180
+ return { found: true, selector: getUniqueSelector(bestMatch), score: bestScore };
181
+ }
182
+
183
+ return { found: false, error: 'No matching element found' };
184
+ })()`;
185
+ await this.page.waitForLoadState('domcontentloaded', { timeout });
186
+ const result = await this.page.evaluate(findScript);
187
+ if (!result.found || !result.selector) {
188
+ throw new Error(`Semantic locator failed: ${result.error || 'No matching element found'}`);
189
+ }
190
+ return this.page.locator(result.selector);
191
+ }
192
+ async findAll(description, options = {}) {
193
+ const { purposeHint, timeout = 5000, minScore = 20 } = options;
194
+ const findAllScript = `
195
+ (function() {
196
+ 'use strict';
197
+
198
+ const searchText = ${JSON.stringify(description.toLowerCase())};
199
+ const purposeHint = ${JSON.stringify(purposeHint || '')};
200
+ const minScore = ${minScore};
201
+
202
+ function getUniqueSelector(el) {
203
+ if (el.id) return '#' + CSS.escape(el.id);
204
+ let path = [];
205
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
206
+ let selector = el.tagName.toLowerCase();
207
+ if (el.id) { selector = '#' + CSS.escape(el.id); path.unshift(selector); break; }
208
+ let sibling = el, nth = 1;
209
+ while (sibling = sibling.previousElementSibling) { if (sibling.tagName === el.tagName) nth++; }
210
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) selector += ':nth-of-type(' + nth + ')';
211
+ path.unshift(selector);
212
+ el = el.parentElement;
213
+ }
214
+ return path.join(' > ');
215
+ }
216
+
217
+ function classify(el) {
218
+ const tag = el.tagName.toLowerCase();
219
+ const role = el.getAttribute('role');
220
+ if (tag === 'nav' || role === 'navigation') return 'navigation';
221
+ if (tag === 'button' || role === 'button' || el.type === 'submit') return 'action';
222
+ if (['input', 'textarea', 'select', 'form'].includes(tag)) return 'form';
223
+ if (tag === 'table' || role === 'table') return 'data';
224
+ if (['main', 'article', 'h1', 'p'].includes(tag)) return 'content';
225
+ return 'unknown';
226
+ }
227
+
228
+ function scoreMatch(el) {
229
+ const text = (el.textContent || '').toLowerCase();
230
+ const ariaLabel = (el.getAttribute('aria-label') || '').toLowerCase();
231
+ const title = (el.getAttribute('title') || '').toLowerCase();
232
+ const id = (el.id || '').toLowerCase();
233
+ const classes = Array.from(el.classList).join(' ').toLowerCase();
234
+
235
+ let score = 0;
236
+
237
+ if (text.trim() === searchText || ariaLabel === searchText) score += 100;
238
+ else if (text.includes(searchText) || ariaLabel.includes(searchText)) score += 50;
239
+ if (title.includes(searchText)) score += 30;
240
+ if (id.includes(searchText.replace(/\\s+/g, '-'))) score += 25;
241
+ if (classes.includes(searchText.replace(/\\s+/g, '-'))) score += 20;
242
+
243
+ if (purposeHint && classify(el) === purposeHint) score += 40;
244
+
245
+ return score;
246
+ }
247
+
248
+ const candidates = document.querySelectorAll('a, button, [role="button"], input, textarea, select, [onclick], [tabindex]');
249
+ const matches = [];
250
+
251
+ candidates.forEach(el => {
252
+ const style = getComputedStyle(el);
253
+ if (style.display === 'none' || style.visibility === 'hidden') return;
254
+
255
+ const score = scoreMatch(el);
256
+ if (score >= minScore) {
257
+ matches.push({
258
+ selector: getUniqueSelector(el),
259
+ score: score,
260
+ text: el.textContent?.trim().substring(0, 100) || null
261
+ });
262
+ }
263
+ });
264
+
265
+ return matches.sort((a, b) => b.score - a.score);
266
+ })()`;
267
+ await this.page.waitForLoadState('domcontentloaded', { timeout });
268
+ const results = await this.page.evaluate(findAllScript);
269
+ return results.map(r => ({
270
+ locator: this.page.locator(r.selector),
271
+ score: r.score,
272
+ text: r.text,
273
+ }));
274
+ }
275
+ async waitFor(description, options = {}) {
276
+ const { timeout = 30000 } = options;
277
+ const startTime = Date.now();
278
+ while (Date.now() - startTime < timeout) {
279
+ try {
280
+ const locator = await this.locator(description, { ...options, timeout: 1000 });
281
+ const isVisible = await locator.isVisible();
282
+ if (isVisible) {
283
+ return locator;
284
+ }
285
+ }
286
+ catch {
287
+ // Element not found yet, continue waiting
288
+ }
289
+ await this.page.waitForTimeout(100);
290
+ }
291
+ throw new Error(`Timeout waiting for element: "${description}"`);
292
+ }
293
+ }
294
+ // Default export for convenience
295
+ exports.default = semantic;
296
+ //# sourceMappingURL=index.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;;AA+HH,4BAEC;AA9HD,uDAS2B;AA8G3B;;;;GAIG;AACH,SAAgB,QAAQ,CAAC,IAAU;IACjC,OAAO,IAAI,gBAAgB,CAAC,IAAI,CAAC,CAAC;AACpC,CAAC;AAED;;GAEG;AACH,MAAM,gBAAgB;IACpB,YAA4B,IAAU;QAAV,SAAI,GAAJ,IAAI,CAAM;IAAG,CAAC;IAE1C,KAAK,CAAC,OAAO,CAAC,iBAA0B,KAAK;QAC3C,MAAM,MAAM,GAAG,IAAA,wCAAsB,EAAC,cAAc,CAAC,CAAC;QACtD,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAsB,CAAC;QAEvE,MAAM,OAAO,GAAG;YACd,UAAU,EAAE,CAAC;YACb,MAAM,EAAE,CAAC;YACT,IAAI,EAAE,CAAC;YACP,OAAO,EAAE,CAAC;YACV,IAAI,EAAE,CAAC;YACP,OAAO,EAAE,CAAC;SACX,CAAC;QAEF,KAAK,MAAM,EAAE,IAAI,QAAQ,EAAE,CAAC;YAC1B,OAAO,CAAC,EAAE,CAAC,QAAQ,CAAC,EAAE,CAAC;QACzB,CAAC;QAED,OAAO,EAAE,QAAQ,EAAE,OAAO,EAAE,CAAC;IAC/B,CAAC;IAED,KAAK,CAAC,KAAK,CAAC,QAA0B,EAAE,SAAkB;QACxD,MAAM,MAAM,GAAG,IAAA,qCAAmB,EAAC,QAAQ,EAAE,SAAS,CAAC,CAAC;QACxD,OAAO,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAkB,CAAC;IAC3D,CAAC;IAED,KAAK,CAAC,KAAK,CAAC,WAAmB,EAAE,UAA2B,EAAE;QAC5D,MAAM,EAAE,WAAW,EAAE,OAAO,GAAG,IAAI,EAAE,QAAQ,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC;QAE/D,wCAAwC;QACxC,MAAM,MAAM,GAAG,IAAA,qCAAmB,EAAC,WAAW,EAAE,WAAW,CAAC,CAAC;QAE7D,4BAA4B;QAC5B,MAAM,IAAI,CAAC,IAAI,CAAC,gBAAgB,CAAC,kBAAkB,EAAE,EAAE,OAAO,EAAE,CAAC,CAAC;QAElE,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAgB,CAAC;QAE/D,yBAAyB;QACzB,IAAI,MAAM,CAAC,OAAO,IAAI,MAAM,CAAC,KAAK,KAAK,SAAS,IAAI,MAAM,CAAC,KAAK,GAAG,QAAQ,EAAE,CAAC;YAC5E,OAAO;gBACL,OAAO,EAAE,KAAK;gBACd,KAAK,EAAE,eAAe,MAAM,CAAC,KAAK,kBAAkB,QAAQ,EAAE;aAC/D,CAAC;QACJ,CAAC;QAED,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,WAAmB,EAAE,KAAa,EAAE,UAAuB,EAAE;QACtE,MAAM,EAAE,OAAO,GAAG,IAAI,EAAE,KAAK,GAAG,IAAI,EAAE,GAAG,OAAO,CAAC;QAEjD,4BAA4B;QAC5B,MAAM,IAAI,CAAC,IAAI,CAAC,gBAAgB,CAAC,kBAAkB,EAAE,EAAE,OAAO,EAAE,CAAC,CAAC;QAElE,MAAM,MAAM,GAAG,IAAA,oCAAkB,EAAC,WAAW,EAAE,KAAK,CAAC,CAAC;QACtD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAA2E,CAAC;QAE1H,IAAI,MAAM,CAAC,MAAM,IAAI,MAAM,CAAC,QAAQ,IAAI,KAAK,EAAE,CAAC;YAC9C,sEAAsE;YACtE,sCAAsC;YACtC,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;gBACnD,IAAI,OAAO,EAAE,CAAC;oBACZ,MAAM,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBAC5B,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,+CAA+C;YACjD,CAAC;QACH,CAAC;QAED,OAAO;YACL,MAAM,EAAE,MAAM,CAAC,MAAM;YACrB,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,KAAK,EAAE,MAAM,CAAC,KAAK;SACpB,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,WAAmB,EAAE,UAA2B,EAAE;QAC9D,MAAM,EAAE,WAAW,EAAE,OAAO,GAAG,IAAI,EAAE,GAAG,OAAO,CAAC;QAEhD,oDAAoD;QACpD,MAAM,UAAU,GAAG;;;;uBAIA,IAAI,CAAC,SAAS,CAAC,WAAW,CAAC,WAAW,EAAE,CAAC;wBACxC,IAAI,CAAC,SAAS,CAAC,WAAW,IAAI,EAAE,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAyEpD,CAAC;QAEF,MAAM,IAAI,CAAC,IAAI,CAAC,gBAAgB,CAAC,kBAAkB,EAAE,EAAE,OAAO,EAAE,CAAC,CAAC;QAElE,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,UAAU,CAA0E,CAAC;QAE7H,IAAI,CAAC,MAAM,CAAC,KAAK,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;YACtC,MAAM,IAAI,KAAK,CAAC,4BAA4B,MAAM,CAAC,KAAK,IAAI,2BAA2B,EAAE,CAAC,CAAC;QAC7F,CAAC;QAED,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;IAC5C,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,WAAmB,EAAE,UAA2B,EAAE;QAC9D,MAAM,EAAE,WAAW,EAAE,OAAO,GAAG,IAAI,EAAE,QAAQ,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC;QAE/D,MAAM,aAAa,GAAG;;;;uBAIH,IAAI,CAAC,SAAS,CAAC,WAAW,CAAC,WAAW,EAAE,CAAC;wBACxC,IAAI,CAAC,SAAS,CAAC,WAAW,IAAI,EAAE,CAAC;qBACpC,QAAQ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAkExB,CAAC;QAEF,MAAM,IAAI,CAAC,IAAI,CAAC,gBAAgB,CAAC,kBAAkB,EAAE,EAAE,OAAO,EAAE,CAAC,CAAC;QAElE,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,aAAa,CAAoE,CAAC;QAE3H,OAAO,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YACvB,OAAO,EAAE,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,QAAQ,CAAC;YACtC,KAAK,EAAE,CAAC,CAAC,KAAK;YACd,IAAI,EAAE,CAAC,CAAC,IAAI;SACb,CAAC,CAAC,CAAC;IACN,CAAC;IAED,KAAK,CAAC,OAAO,CAAC,WAAmB,EAAE,UAA2B,EAAE;QAC9D,MAAM,EAAE,OAAO,GAAG,KAAK,EAAE,GAAG,OAAO,CAAC;QACpC,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QAE7B,OAAO,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,GAAG,OAAO,EAAE,CAAC;YACxC,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,WAAW,EAAE,EAAE,GAAG,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC,CAAC;gBAC/E,MAAM,SAAS,GAAG,MAAM,OAAO,CAAC,SAAS,EAAE,CAAC;gBAC5C,IAAI,SAAS,EAAE,CAAC;oBACd,OAAO,OAAO,CAAC;gBACjB,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,0CAA0C;YAC5C,CAAC;YACD,MAAM,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC,GAAG,CAAC,CAAC;QACtC,CAAC;QAED,MAAM,IAAI,KAAK,CAAC,iCAAiC,WAAW,GAAG,CAAC,CAAC;IACnE,CAAC;CACF;AAED,iCAAiC;AACjC,kBAAe,QAAQ,CAAC"}
@@ -0,0 +1,48 @@
1
+ /**
2
+ * Semantic Mapper - Core classification engine
3
+ *
4
+ * Classifies DOM elements by semantic purpose using heuristics,
5
+ * without requiring LLM calls. Fast, local, privacy-preserving.
6
+ */
7
+ export type SemanticCategory = 'navigation' | 'action' | 'form' | 'content' | 'data' | 'unknown';
8
+ export interface SemanticElement {
9
+ selector: string;
10
+ tagName: string;
11
+ category: SemanticCategory;
12
+ confidence: number;
13
+ textPreview: string | null;
14
+ role: string | null;
15
+ ariaLabel: string | null;
16
+ id: string | null;
17
+ classes: string | null;
18
+ }
19
+ export interface ClickResult {
20
+ clicked: boolean;
21
+ selector?: string;
22
+ text?: string;
23
+ score?: number;
24
+ error?: string;
25
+ }
26
+ export interface QueryResult {
27
+ selector: string;
28
+ tagName: string;
29
+ textPreview: string | null;
30
+ ariaLabel: string | null;
31
+ }
32
+ /**
33
+ * Generate JavaScript to analyze page and classify all elements
34
+ */
35
+ export declare function generateAnalysisScript(includeUnknown?: boolean): string;
36
+ /**
37
+ * Generate JavaScript to find elements by semantic category
38
+ */
39
+ export declare function generateQueryScript(category: SemanticCategory, query?: string): string;
40
+ /**
41
+ * Generate JavaScript to click element by semantic description
42
+ */
43
+ export declare function generateClickScript(description: string, purposeHint?: SemanticCategory): string;
44
+ /**
45
+ * Generate JavaScript to fill a form field by semantic description
46
+ */
47
+ export declare function generateFillScript(description: string, value: string): string;
48
+ //# sourceMappingURL=semantic-mapper.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"semantic-mapper.d.ts","sourceRoot":"","sources":["../src/semantic-mapper.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,MAAM,MAAM,gBAAgB,GAAG,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,SAAS,GAAG,MAAM,GAAG,SAAS,CAAC;AAEjG,MAAM,WAAW,eAAe;IAC9B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE,gBAAgB,CAAC;IAC3B,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3B,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,EAAE,EAAE,MAAM,GAAG,IAAI,CAAC;IAClB,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CACxB;AAED,MAAM,WAAW,WAAW;IAC1B,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,WAAW;IAC1B,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3B,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;CAC1B;AAgBD;;GAEG;AACH,wBAAgB,sBAAsB,CAAC,cAAc,GAAE,OAAe,GAAG,MAAM,CA+J9E;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,QAAQ,EAAE,gBAAgB,EAAE,KAAK,CAAC,EAAE,MAAM,GAAG,MAAM,CAwEtF;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,WAAW,EAAE,MAAM,EAAE,WAAW,CAAC,EAAE,gBAAgB,GAAG,MAAM,CA4F/F;AAED;;GAEG;AACH,wBAAgB,kBAAkB,CAAC,WAAW,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CA8E7E"}
@@ -0,0 +1,440 @@
1
+ "use strict";
2
+ /**
3
+ * Semantic Mapper - Core classification engine
4
+ *
5
+ * Classifies DOM elements by semantic purpose using heuristics,
6
+ * without requiring LLM calls. Fast, local, privacy-preserving.
7
+ */
8
+ Object.defineProperty(exports, "__esModule", { value: true });
9
+ exports.generateAnalysisScript = generateAnalysisScript;
10
+ exports.generateQueryScript = generateQueryScript;
11
+ exports.generateClickScript = generateClickScript;
12
+ exports.generateFillScript = generateFillScript;
13
+ /**
14
+ * Escape string for safe JavaScript injection
15
+ */
16
+ function escapeJS(str) {
17
+ return str
18
+ .replace(/\\/g, '\\\\')
19
+ .replace(/'/g, "\\'")
20
+ .replace(/"/g, '\\"')
21
+ .replace(/`/g, '\\`')
22
+ .replace(/\$/g, '\\$')
23
+ .replace(/\n/g, '\\n')
24
+ .replace(/\r/g, '\\r');
25
+ }
26
+ /**
27
+ * Generate JavaScript to analyze page and classify all elements
28
+ */
29
+ function generateAnalysisScript(includeUnknown = false) {
30
+ return `
31
+ (function() {
32
+ 'use strict';
33
+
34
+ // Generate unique CSS selector for element
35
+ function getUniqueSelector(el) {
36
+ if (el.id) return '#' + CSS.escape(el.id);
37
+
38
+ let path = [];
39
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
40
+ let selector = el.tagName.toLowerCase();
41
+
42
+ if (el.id) {
43
+ selector = '#' + CSS.escape(el.id);
44
+ path.unshift(selector);
45
+ break;
46
+ }
47
+
48
+ let sibling = el;
49
+ let nth = 1;
50
+ while (sibling = sibling.previousElementSibling) {
51
+ if (sibling.tagName === el.tagName) nth++;
52
+ }
53
+
54
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) {
55
+ selector += ':nth-of-type(' + nth + ')';
56
+ }
57
+
58
+ path.unshift(selector);
59
+ el = el.parentElement;
60
+ }
61
+
62
+ return path.join(' > ');
63
+ }
64
+
65
+ // Classify element by semantic purpose
66
+ function classify(el) {
67
+ const tag = el.tagName.toLowerCase();
68
+ const role = el.getAttribute('role');
69
+ const classes = Array.from(el.classList).join(' ').toLowerCase();
70
+ const id = (el.id || '').toLowerCase();
71
+ const type = el.type || '';
72
+
73
+ // Navigation patterns
74
+ if (tag === 'nav' || role === 'navigation' || role === 'menu' || role === 'menubar') {
75
+ return { category: 'navigation', confidence: 0.95 };
76
+ }
77
+ if (classes.match(/\\b(nav|menu|breadcrumb|pagination|sidebar|header-links)\\b/) ||
78
+ id.match(/\\b(nav|menu|navigation)\\b/)) {
79
+ return { category: 'navigation', confidence: 0.85 };
80
+ }
81
+ if (tag === 'a' && el.closest('nav, [role="navigation"]')) {
82
+ return { category: 'navigation', confidence: 0.80 };
83
+ }
84
+
85
+ // Action patterns
86
+ if (tag === 'button' || role === 'button') {
87
+ return { category: 'action', confidence: 0.95 };
88
+ }
89
+ if (type === 'submit' || type === 'button') {
90
+ return { category: 'action', confidence: 0.90 };
91
+ }
92
+ if (classes.match(/\\b(btn|button|cta|action|submit)\\b/)) {
93
+ return { category: 'action', confidence: 0.85 };
94
+ }
95
+ if (tag === 'a' && classes.match(/\\b(btn|button|cta)\\b/)) {
96
+ return { category: 'action', confidence: 0.85 };
97
+ }
98
+
99
+ // Form patterns
100
+ if (tag === 'form' || role === 'form') {
101
+ return { category: 'form', confidence: 0.95 };
102
+ }
103
+ if (['input', 'textarea', 'select'].includes(tag)) {
104
+ return { category: 'form', confidence: 0.95 };
105
+ }
106
+ if (role === 'textbox' || role === 'combobox' || role === 'listbox' || role === 'checkbox' || role === 'radio') {
107
+ return { category: 'form', confidence: 0.90 };
108
+ }
109
+ if (classes.match(/\\b(form|input|field|control)\\b/) && tag !== 'button') {
110
+ return { category: 'form', confidence: 0.75 };
111
+ }
112
+
113
+ // Data patterns
114
+ if (tag === 'table' || role === 'table' || role === 'grid') {
115
+ return { category: 'data', confidence: 0.95 };
116
+ }
117
+ if (role === 'list' || role === 'listitem') {
118
+ return { category: 'data', confidence: 0.85 };
119
+ }
120
+ if (tag === 'dl' || tag === 'ul' || tag === 'ol') {
121
+ if (!el.closest('nav, [role="navigation"]')) {
122
+ return { category: 'data', confidence: 0.70 };
123
+ }
124
+ }
125
+ if (classes.match(/\\b(table|list|grid|card|item|row)\\b/)) {
126
+ return { category: 'data', confidence: 0.70 };
127
+ }
128
+
129
+ // Content patterns
130
+ if (tag === 'main' || tag === 'article' || role === 'main' || role === 'article') {
131
+ return { category: 'content', confidence: 0.95 };
132
+ }
133
+ if (['h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'p', 'section'].includes(tag)) {
134
+ return { category: 'content', confidence: 0.85 };
135
+ }
136
+ if (role === 'heading' || role === 'region') {
137
+ return { category: 'content', confidence: 0.80 };
138
+ }
139
+ if (classes.match(/\\b(content|article|post|text|body|main)\\b/)) {
140
+ return { category: 'content', confidence: 0.75 };
141
+ }
142
+
143
+ return { category: 'unknown', confidence: 0.0 };
144
+ }
145
+
146
+ // Get meaningful elements
147
+ const elements = [];
148
+ const meaningfulTags = new Set([
149
+ 'nav', 'main', 'article', 'section', 'aside', 'header', 'footer',
150
+ 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'p',
151
+ 'a', 'button', 'input', 'select', 'textarea', 'form',
152
+ 'table', 'ul', 'ol', 'dl', 'div'
153
+ ]);
154
+
155
+ const selector = Array.from(meaningfulTags).join(',') + ',[role]';
156
+
157
+ document.querySelectorAll(selector).forEach(el => {
158
+ const style = getComputedStyle(el);
159
+ if (style.display === 'none' || style.visibility === 'hidden') return;
160
+
161
+ const rect = el.getBoundingClientRect();
162
+ if (rect.width < 10 || rect.height < 10) return;
163
+
164
+ const { category, confidence } = classify(el);
165
+
166
+ if (category === 'unknown' && !${includeUnknown}) return;
167
+
168
+ let textPreview = '';
169
+ if (el.textContent) {
170
+ textPreview = el.textContent.trim().replace(/\\s+/g, ' ').substring(0, 100);
171
+ }
172
+
173
+ elements.push({
174
+ selector: getUniqueSelector(el),
175
+ tagName: el.tagName.toLowerCase(),
176
+ category: category,
177
+ confidence: confidence,
178
+ textPreview: textPreview || null,
179
+ role: el.getAttribute('role') || null,
180
+ ariaLabel: el.getAttribute('aria-label') || null,
181
+ id: el.id || null,
182
+ classes: Array.from(el.classList).join(' ') || null
183
+ });
184
+ });
185
+
186
+ return elements;
187
+ })()`;
188
+ }
189
+ /**
190
+ * Generate JavaScript to find elements by semantic category
191
+ */
192
+ function generateQueryScript(category, query) {
193
+ const queryFilter = query?.toLowerCase() ?? '';
194
+ return `
195
+ (function() {
196
+ 'use strict';
197
+
198
+ const targetCategory = '${category}';
199
+ const searchQuery = '${escapeJS(queryFilter)}'.toLowerCase();
200
+
201
+ function getUniqueSelector(el) {
202
+ if (el.id) return '#' + CSS.escape(el.id);
203
+ let path = [];
204
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
205
+ let selector = el.tagName.toLowerCase();
206
+ if (el.id) {
207
+ selector = '#' + CSS.escape(el.id);
208
+ path.unshift(selector);
209
+ break;
210
+ }
211
+ let sibling = el;
212
+ let nth = 1;
213
+ while (sibling = sibling.previousElementSibling) {
214
+ if (sibling.tagName === el.tagName) nth++;
215
+ }
216
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) {
217
+ selector += ':nth-of-type(' + nth + ')';
218
+ }
219
+ path.unshift(selector);
220
+ el = el.parentElement;
221
+ }
222
+ return path.join(' > ');
223
+ }
224
+
225
+ function classify(el) {
226
+ const tag = el.tagName.toLowerCase();
227
+ const role = el.getAttribute('role');
228
+ const classes = Array.from(el.classList).join(' ').toLowerCase();
229
+
230
+ if (tag === 'nav' || role === 'navigation' || classes.match(/\\bnav\\b/)) return 'navigation';
231
+ if (tag === 'button' || role === 'button' || el.type === 'submit' || classes.match(/\\bbtn\\b/)) return 'action';
232
+ if (['input', 'textarea', 'select', 'form'].includes(tag) || role === 'textbox') return 'form';
233
+ if (tag === 'table' || role === 'table' || role === 'list') return 'data';
234
+ if (['main', 'article', 'h1', 'h2', 'h3', 'p', 'section'].includes(tag) || role === 'main') return 'content';
235
+ return 'unknown';
236
+ }
237
+
238
+ const matches = [];
239
+ document.querySelectorAll('*').forEach(el => {
240
+ const style = getComputedStyle(el);
241
+ if (style.display === 'none' || style.visibility === 'hidden') return;
242
+
243
+ const category = classify(el);
244
+ if (category !== targetCategory) return;
245
+
246
+ const text = el.textContent?.trim().toLowerCase() || '';
247
+ const ariaLabel = (el.getAttribute('aria-label') || '').toLowerCase();
248
+
249
+ if (searchQuery && !text.includes(searchQuery) && !ariaLabel.includes(searchQuery)) {
250
+ return;
251
+ }
252
+
253
+ matches.push({
254
+ selector: getUniqueSelector(el),
255
+ tagName: el.tagName.toLowerCase(),
256
+ textPreview: el.textContent?.trim().substring(0, 100) || null,
257
+ ariaLabel: el.getAttribute('aria-label') || null
258
+ });
259
+ });
260
+
261
+ return matches.slice(0, 50);
262
+ })()`;
263
+ }
264
+ /**
265
+ * Generate JavaScript to click element by semantic description
266
+ */
267
+ function generateClickScript(description, purposeHint) {
268
+ const searchText = description.toLowerCase();
269
+ const purposeFilter = purposeHint ?? '';
270
+ return `
271
+ (function() {
272
+ 'use strict';
273
+
274
+ const searchText = '${escapeJS(searchText)}'.toLowerCase();
275
+ const purposeHint = '${purposeFilter}';
276
+
277
+ function getUniqueSelector(el) {
278
+ if (el.id) return '#' + CSS.escape(el.id);
279
+ let path = [];
280
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
281
+ let selector = el.tagName.toLowerCase();
282
+ if (el.id) { selector = '#' + CSS.escape(el.id); path.unshift(selector); break; }
283
+ let sibling = el, nth = 1;
284
+ while (sibling = sibling.previousElementSibling) { if (sibling.tagName === el.tagName) nth++; }
285
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) selector += ':nth-of-type(' + nth + ')';
286
+ path.unshift(selector);
287
+ el = el.parentElement;
288
+ }
289
+ return path.join(' > ');
290
+ }
291
+
292
+ function classify(el) {
293
+ const tag = el.tagName.toLowerCase();
294
+ const role = el.getAttribute('role');
295
+ if (tag === 'nav' || role === 'navigation') return 'navigation';
296
+ if (tag === 'button' || role === 'button' || el.type === 'submit') return 'action';
297
+ if (['input', 'textarea', 'select', 'form'].includes(tag)) return 'form';
298
+ if (tag === 'table' || role === 'table') return 'data';
299
+ if (['main', 'article', 'h1', 'p'].includes(tag)) return 'content';
300
+ return 'unknown';
301
+ }
302
+
303
+ function scoreMatch(el) {
304
+ const text = (el.textContent || '').toLowerCase();
305
+ const ariaLabel = (el.getAttribute('aria-label') || '').toLowerCase();
306
+ const title = (el.getAttribute('title') || '').toLowerCase();
307
+ const id = (el.id || '').toLowerCase();
308
+ const classes = Array.from(el.classList).join(' ').toLowerCase();
309
+
310
+ let score = 0;
311
+
312
+ // Exact matches score highest
313
+ if (text.trim() === searchText || ariaLabel === searchText) score += 100;
314
+ else if (text.includes(searchText) || ariaLabel.includes(searchText)) score += 50;
315
+ if (title.includes(searchText)) score += 30;
316
+ if (id.includes(searchText.replace(/\\s+/g, '-'))) score += 25;
317
+ if (classes.includes(searchText.replace(/\\s+/g, '-'))) score += 20;
318
+
319
+ // Purpose match bonus
320
+ if (purposeHint && classify(el) === purposeHint) score += 40;
321
+
322
+ // Prefer buttons for action-like queries
323
+ if (searchText.match(/button|submit|click|press/) && (el.tagName === 'BUTTON' || el.type === 'submit')) {
324
+ score += 30;
325
+ }
326
+
327
+ return score;
328
+ }
329
+
330
+ const clickables = document.querySelectorAll('a, button, [role="button"], input[type="submit"], [onclick]');
331
+
332
+ let bestMatch = null;
333
+ let bestScore = 0;
334
+
335
+ clickables.forEach(el => {
336
+ const style = getComputedStyle(el);
337
+ if (style.display === 'none' || style.visibility === 'hidden') return;
338
+
339
+ const score = scoreMatch(el);
340
+ if (score > bestScore) {
341
+ bestScore = score;
342
+ bestMatch = el;
343
+ }
344
+ });
345
+
346
+ if (bestMatch && bestScore >= 20) {
347
+ bestMatch.click();
348
+ return {
349
+ clicked: true,
350
+ selector: getUniqueSelector(bestMatch),
351
+ text: bestMatch.textContent?.trim().substring(0, 100) || null,
352
+ score: bestScore
353
+ };
354
+ }
355
+
356
+ return { clicked: false, error: 'No matching element found' };
357
+ })()`;
358
+ }
359
+ /**
360
+ * Generate JavaScript to fill a form field by semantic description
361
+ */
362
+ function generateFillScript(description, value) {
363
+ const searchText = description.toLowerCase();
364
+ const escapedValue = escapeJS(value);
365
+ return `
366
+ (function() {
367
+ 'use strict';
368
+
369
+ const searchText = '${escapeJS(searchText)}'.toLowerCase();
370
+ const fillValue = '${escapedValue}';
371
+
372
+ function getUniqueSelector(el) {
373
+ if (el.id) return '#' + CSS.escape(el.id);
374
+ let path = [];
375
+ while (el && el.nodeType === Node.ELEMENT_NODE) {
376
+ let selector = el.tagName.toLowerCase();
377
+ if (el.id) { selector = '#' + CSS.escape(el.id); path.unshift(selector); break; }
378
+ let sibling = el, nth = 1;
379
+ while (sibling = sibling.previousElementSibling) { if (sibling.tagName === el.tagName) nth++; }
380
+ if (nth > 1 || el.nextElementSibling?.tagName === el.tagName) selector += ':nth-of-type(' + nth + ')';
381
+ path.unshift(selector);
382
+ el = el.parentElement;
383
+ }
384
+ return path.join(' > ');
385
+ }
386
+
387
+ function scoreMatch(el) {
388
+ const tag = el.tagName.toLowerCase();
389
+ if (!['input', 'textarea', 'select'].includes(tag)) return 0;
390
+
391
+ const label = document.querySelector('label[for="' + el.id + '"]');
392
+ const labelText = label ? label.textContent.toLowerCase() : '';
393
+ const placeholder = (el.placeholder || '').toLowerCase();
394
+ const name = (el.name || '').toLowerCase();
395
+ const ariaLabel = (el.getAttribute('aria-label') || '').toLowerCase();
396
+ const id = (el.id || '').toLowerCase();
397
+
398
+ let score = 0;
399
+
400
+ if (labelText.includes(searchText)) score += 100;
401
+ if (ariaLabel.includes(searchText)) score += 90;
402
+ if (placeholder.includes(searchText)) score += 80;
403
+ if (name.includes(searchText.replace(/\\s+/g, '_'))) score += 70;
404
+ if (id.includes(searchText.replace(/\\s+/g, '-'))) score += 60;
405
+
406
+ return score;
407
+ }
408
+
409
+ const inputs = document.querySelectorAll('input, textarea, select');
410
+
411
+ let bestMatch = null;
412
+ let bestScore = 0;
413
+
414
+ inputs.forEach(el => {
415
+ const style = getComputedStyle(el);
416
+ if (style.display === 'none' || style.visibility === 'hidden') return;
417
+ if (el.disabled || el.readOnly) return;
418
+
419
+ const score = scoreMatch(el);
420
+ if (score > bestScore) {
421
+ bestScore = score;
422
+ bestMatch = el;
423
+ }
424
+ });
425
+
426
+ if (bestMatch && bestScore >= 50) {
427
+ bestMatch.value = fillValue;
428
+ bestMatch.dispatchEvent(new Event('input', { bubbles: true }));
429
+ bestMatch.dispatchEvent(new Event('change', { bubbles: true }));
430
+ return {
431
+ filled: true,
432
+ selector: getUniqueSelector(bestMatch),
433
+ score: bestScore
434
+ };
435
+ }
436
+
437
+ return { filled: false, error: 'No matching input found' };
438
+ })()`;
439
+ }
440
+ //# sourceMappingURL=semantic-mapper.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"semantic-mapper.js","sourceRoot":"","sources":["../src/semantic-mapper.ts"],"names":[],"mappings":";AAAA;;;;;GAKG;;AAgDH,wDA+JC;AAKD,kDAwEC;AAKD,kDA4FC;AAKD,gDA8EC;AAjbD;;GAEG;AACH,SAAS,QAAQ,CAAC,GAAW;IAC3B,OAAO,GAAG;SACP,OAAO,CAAC,KAAK,EAAE,MAAM,CAAC;SACtB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;AAC3B,CAAC;AAED;;GAEG;AACH,SAAgB,sBAAsB,CAAC,iBAA0B,KAAK;IACpE,OAAO;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;qCAwI4B,cAAc;;;;;;;;;;;;;;;;;;;;;KAqB9C,CAAC;AACN,CAAC;AAED;;GAEG;AACH,SAAgB,mBAAmB,CAAC,QAA0B,EAAE,KAAc;IAC5E,MAAM,WAAW,GAAG,KAAK,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;IAE/C,OAAO;;;;4BAImB,QAAQ;yBACX,QAAQ,CAAC,WAAW,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KA+DzC,CAAC;AACN,CAAC;AAED;;GAEG;AACH,SAAgB,mBAAmB,CAAC,WAAmB,EAAE,WAA8B;IACrF,MAAM,UAAU,GAAG,WAAW,CAAC,WAAW,EAAE,CAAC;IAC7C,MAAM,aAAa,GAAG,WAAW,IAAI,EAAE,CAAC;IAExC,OAAO;;;;wBAIe,QAAQ,CAAC,UAAU,CAAC;yBACnB,aAAa;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAkFjC,CAAC;AACN,CAAC;AAED;;GAEG;AACH,SAAgB,kBAAkB,CAAC,WAAmB,EAAE,KAAa;IACnE,MAAM,UAAU,GAAG,WAAW,CAAC,WAAW,EAAE,CAAC;IAC7C,MAAM,YAAY,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC;IAErC,OAAO;;;;wBAIe,QAAQ,CAAC,UAAU,CAAC;uBACrB,YAAY;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAoE9B,CAAC;AACN,CAAC"}
package/package.json ADDED
@@ -0,0 +1,42 @@
1
+ {
2
+ "name": "semantic-playwright",
3
+ "version": "0.1.0",
4
+ "description": "Semantic element targeting for Playwright - click by description, not selectors",
5
+ "main": "dist/index.js",
6
+ "types": "dist/index.d.ts",
7
+ "scripts": {
8
+ "build": "tsc",
9
+ "test": "playwright test",
10
+ "lint": "eslint src --ext .ts",
11
+ "prepublishOnly": "npm run build"
12
+ },
13
+ "keywords": [
14
+ "playwright",
15
+ "testing",
16
+ "automation",
17
+ "semantic",
18
+ "self-healing",
19
+ "web-automation",
20
+ "e2e",
21
+ "locators"
22
+ ],
23
+ "author": "",
24
+ "license": "MIT",
25
+ "peerDependencies": {
26
+ "@playwright/test": ">=1.40.0",
27
+ "playwright": ">=1.40.0"
28
+ },
29
+ "devDependencies": {
30
+ "@playwright/test": "^1.40.0",
31
+ "@types/node": "^20.10.0",
32
+ "typescript": "^5.3.0"
33
+ },
34
+ "files": [
35
+ "dist",
36
+ "README.md"
37
+ ],
38
+ "repository": {
39
+ "type": "git",
40
+ "url": "https://github.com/semantic-engine/semantic-playwright"
41
+ }
42
+ }