chrometools-mcp 3.5.5 → 3.5.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +13 -4
- package/index.js +137 -11
- package/package.json +1 -1
- package/pom/apom-tree-converter.js +56 -1
- package/server/tool-definitions.js +9 -4
- package/server/tool-schemas.js +10 -5
- package/specs/SEGM-537-UNBLOCKERS_PROGRESS.md +94 -0
- package/specs/SEGM-537-UNBLOCKERS_SPEC.md +187 -0
- package/utils/actions/click-action.js +33 -1
- package/NUL +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,19 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
|
|
5
|
+
## [3.5.6] - 2026-05-28
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
- **`analyzePage({ includePortals, portalSelectors })`** — Generic React Portal scan beyond framework modals. Default selectors `['#modal-root', '#menu-popup-root', '#tooltip-root', '#popover-root', '[data-portal]']`, opt-out via `includePortals: false`. Without this, action menus / tooltips / popovers rendered outside `#root` are invisible to APOM
|
|
9
|
+
- **In-tree popup detection** — `analyzePage` now also force-includes Popper/Tippy/FloatingUI-style popups: positioned (absolute/fixed) descendants inside a 0-height inline wrapper. Same opt-out flag (`includePortals`). Covers custom contextMenu implementations that don't use real React Portals
|
|
10
|
+
- **`click({ waitForSelector, waitTimeoutMs })`** — Atomic click + wait. After click, waits for a CSS selector to appear (visible). On timeout the click still succeeds but the result text reports `⚠️ WAIT_TIMEOUT`. Designed for dropdowns/popups that race against the next MCP call
|
|
11
|
+
- **`click({ autoAnalyzeAfter })`** — After click, diffs APOM state and appends `+N appeared: id:"text"` / `-N disappeared` delta. New element ids are pre-registered for follow-up `click`/`type` calls — opens a dropdown and clicks one of its items in two MCP calls instead of three
|
|
12
|
+
- **`screenshot()` viewport mode** — Both `id` and `selector` are now optional; without either, captures the viewport (same compression pipeline as element screenshots)
|
|
13
|
+
- **`executeScript` auto-IIFE** — Snippets starting with `return ...` are now auto-wrapped in `(async () => { ... })()`. Skipped when the snippet declares a `function`, to preserve implicit-return behavior
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
- **`ModelRegistry is not defined` after navigation** — Root cause: `quickRegisterElements` (called from `resolveSelector` auto-refresh) didn't inject models code, so `buildAPOMTree` inside it failed with `ReferenceError` when the browser context had been wiped. Models are now always re-injected. If the error still surfaces, it's mapped to a clear `APOM registry stale, call analyzePage()` message instead of leaking the raw ReferenceError
|
|
17
|
+
|
|
5
18
|
## [3.5.5] - 2026-03-27
|
|
6
19
|
|
|
7
20
|
### Added
|
package/README.md
CHANGED
|
@@ -403,7 +403,10 @@ executeScenario({ name: "login_flow", parameters: { email: "user@test.com" } })
|
|
|
403
403
|
- `refresh` (optional): Force refresh cache to get CURRENT state after changes (default: false)
|
|
404
404
|
- `includeAll` (optional): Include ALL page elements, not just interactive ones (default: false). Useful for layout work - find any element, get its selector, then use `getComputedCss` or `setStyles` on it.
|
|
405
405
|
- `useLegacyFormat` (optional): Return legacy format instead of APOM (default: false - APOM is the default)
|
|
406
|
-
- `registerElements` (optional): Auto-register elements for ID-based usage (default: true) - `groupBy` (optional): 'type' or 'flat' - how to group elements (default: 'type')
|
|
406
|
+
- `registerElements` (optional): Auto-register elements for ID-based usage (default: true) - `groupBy` (optional): 'type' or 'flat' - how to group elements (default: 'type')
|
|
407
|
+
- `includePortals` (optional): Include contents of React Portal containers — menus, tooltips, popovers rendered outside the main React root (default: `true`). Without this, items inside dropdown popups (e.g. action menus in MTS-like apps) are invisible to `analyzePage`.
|
|
408
|
+
- `portalSelectors` (optional): Array of CSS selectors for portal root containers. Default: `['#modal-root', '#menu-popup-root', '#tooltip-root', '#popover-root', '[data-portal]']`. Override when the app uses different portal element ids.
|
|
409
|
+
- **In-tree popup heuristic**: when `includePortals` is enabled (default), `analyzePage` also detects "in-tree portal" patterns — popups rendered inside a 0-height inline wrapper and absolute-positioned out of it (Popper, Tippy, FloatingUI, custom contextMenu implementations). Without this, popup items live inside an `offsetHeight: 0` wrapper that `isVisible` drops, making the whole popup subtree invisible to `analyzePage`. - **Why better than screenshot**:
|
|
407
410
|
- Shows actual data (form values, validation errors) not just visual
|
|
408
411
|
- Uses 2-5k tokens vs screenshot 5-10k tokens
|
|
409
412
|
- Returns structured data with **unique element IDs** for easy interaction
|
|
@@ -514,6 +517,9 @@ Click an element with optional result screenshot. **PREFERRED**: Use APOM ID fro
|
|
|
514
517
|
- `timeout` (optional): Max operation time in ms (default: 30000)
|
|
515
518
|
- `skipNetworkWait` (optional): Skip waiting for network requests (default: false). **Use for pages with continuous long-polling to get instant response.**
|
|
516
519
|
- `networkWaitTimeout` (optional): Custom network wait timeout in ms (default: 10000). Only used if skipNetworkWait is false.
|
|
520
|
+
- `waitForSelector` (optional): CSS selector to wait for **after** the click — atomic click+wait. Use for dropdowns/popups that render into a React Portal and otherwise race with the next MCP call. Example: `click({ id: 'button_47', waitForSelector: '#menu-popup-root > div' })`.
|
|
521
|
+
- `waitTimeoutMs` (optional): Timeout for `waitForSelector` in ms (default: 2000). On timeout the click still succeeds but the result text reports `⚠️ WAIT_TIMEOUT`.
|
|
522
|
+
- `autoAnalyzeAfter` (optional): After click, automatically diff APOM and append the delta to the result text (e.g. `+3 appeared: button_42:"Статистика", button_43:"Настройки", link_44:"Удалить"`). New element ids are pre-registered so the next `click({ id })`/`type({ id })` call works **without an extra `analyzePage`**. Designed for the dropdown/menu pattern: one MCP call instead of three.
|
|
517
523
|
- **Use case**: Buttons, links, form submissions, Django admin forms
|
|
518
524
|
- **Returns**: Confirmation text + optional screenshot + network diagnostics
|
|
519
525
|
- **Performance**: 2-10x faster without screenshot, instant with skipNetworkWait
|
|
@@ -676,10 +682,12 @@ Get precise dimensions, positioning, margins, padding, and borders.
|
|
|
676
682
|
- **Returns**: Box model data + metrics
|
|
677
683
|
|
|
678
684
|
#### screenshot
|
|
679
|
-
Capture optimized screenshot of specific element
|
|
685
|
+
Capture optimized screenshot of a specific element, or the full viewport when no `id`/`selector` is given. Smart compression with a 3 MB hard limit.
|
|
680
686
|
- **Parameters**:
|
|
681
|
-
- `
|
|
682
|
-
- `
|
|
687
|
+
- `id` (optional): APOM element ID from `analyzePage`. Mutually exclusive with `selector`.
|
|
688
|
+
- `selector` (optional): CSS selector. Mutually exclusive with `id`.
|
|
689
|
+
- Omit both `id` and `selector` to capture the **full viewport** (no element resolution needed).
|
|
690
|
+
- `padding` (optional): Padding in pixels (default: 0). Ignored for viewport screenshots.
|
|
683
691
|
- `maxWidth` (optional): Max width for auto-scaling (default: 1024, null for original size)
|
|
684
692
|
- `maxHeight` (optional): Max height for auto-scaling (default: 8000, null for original size)
|
|
685
693
|
- `quality` (optional): JPEG quality 1-100 (default: 40)
|
|
@@ -717,6 +725,7 @@ Execute arbitrary JavaScript in page context with optional screenshot.
|
|
|
717
725
|
- **Use case**: Complex interactions, custom manipulations
|
|
718
726
|
- **Returns**: Execution result + optional screenshot
|
|
719
727
|
- **Performance**: 2-10x faster without screenshot
|
|
728
|
+
- **Top-level `return`**: snippets that start with `return ...` (e.g. `return document.title`) are auto-wrapped in an async IIFE — no need to manually wrap in `(() => { ... })()`. Scripts that declare a `function` are left unmodified so implicit-return patterns keep working.
|
|
720
729
|
|
|
721
730
|
#### getConsoleLogs
|
|
722
731
|
Retrieve browser console logs (log, warn, error, etc.).
|
package/index.js
CHANGED
|
@@ -403,8 +403,13 @@ async function executeToolInternal(name, args) {
|
|
|
403
403
|
* Quick element registration - runs APOM analysis and registers elements
|
|
404
404
|
*/
|
|
405
405
|
async function quickRegisterElements(page) {
|
|
406
|
-
await page.evaluate((apomTreeConverterCode, selectorResolverCode) => {
|
|
407
|
-
// Inject utilities
|
|
406
|
+
await page.evaluate((apomTreeConverterCode, selectorResolverCode, modelsCode) => {
|
|
407
|
+
// Inject utilities. Models must be loaded BEFORE buildAPOMTree because
|
|
408
|
+
// initializeModelRegistry() inside it references window.ModelRegistry. After a
|
|
409
|
+
// navigation the browser window is wiped, so we always re-eval if missing.
|
|
410
|
+
if (typeof window.ModelRegistry === 'undefined') {
|
|
411
|
+
eval(modelsCode);
|
|
412
|
+
}
|
|
408
413
|
if (typeof buildAPOMTree === 'undefined') {
|
|
409
414
|
eval(apomTreeConverterCode);
|
|
410
415
|
}
|
|
@@ -441,7 +446,7 @@ async function executeToolInternal(name, args) {
|
|
|
441
446
|
if (typeof registerElements !== 'undefined') {
|
|
442
447
|
registerElements(elementsArray);
|
|
443
448
|
}
|
|
444
|
-
}, apomTreeConverter, selectorResolver);
|
|
449
|
+
}, apomTreeConverter, selectorResolver, elementModelBase + '\n' + elementModels + '\n' + modelRegistry);
|
|
445
450
|
}
|
|
446
451
|
|
|
447
452
|
/**
|
|
@@ -470,15 +475,56 @@ async function executeToolInternal(name, args) {
|
|
|
470
475
|
|
|
471
476
|
let resolved = await Promise.race([tryResolve(), timeoutPromise]);
|
|
472
477
|
|
|
473
|
-
// Auto-refresh: if looks like APOM ID but not found, re-register elements and retry
|
|
478
|
+
// Auto-refresh: if looks like APOM ID but not found, re-register elements and retry.
|
|
479
|
+
// quickRegisterElements re-injects ModelRegistry — if that itself fails (e.g. ReferenceError
|
|
480
|
+
// from a partially-loaded browser context), surface a clear "call analyzePage" message
|
|
481
|
+
// instead of leaking the raw browser error.
|
|
474
482
|
if (!resolved.found && isApomIdPattern(identifier)) {
|
|
475
|
-
|
|
483
|
+
try {
|
|
484
|
+
await quickRegisterElements(page);
|
|
485
|
+
} catch (e) {
|
|
486
|
+
if (/ModelRegistry is not defined/.test(String(e?.message || e))) {
|
|
487
|
+
throw new Error(
|
|
488
|
+
`APOM registry stale after navigation/reload. Call analyzePage() to refresh element ids before reusing "${identifier}".`
|
|
489
|
+
);
|
|
490
|
+
}
|
|
491
|
+
throw e;
|
|
492
|
+
}
|
|
476
493
|
resolved = await Promise.race([tryResolve(), timeoutPromise]);
|
|
477
494
|
}
|
|
478
495
|
|
|
479
496
|
return resolved;
|
|
480
497
|
}
|
|
481
498
|
|
|
499
|
+
/**
|
|
500
|
+
* Snapshot of all APOM ids on the page → { id: { tag, type, text } }
|
|
501
|
+
* Used by click({ autoAnalyzeAfter: true }) to compute pre/post deltas.
|
|
502
|
+
*/
|
|
503
|
+
async function getApomSnapshot(page) {
|
|
504
|
+
return await page.evaluate((apomCode, modelsCode) => {
|
|
505
|
+
if (typeof window.ModelRegistry === 'undefined') eval(modelsCode);
|
|
506
|
+
if (typeof buildAPOMTree === 'undefined') eval(apomCode);
|
|
507
|
+
const apom = buildAPOMTree(true);
|
|
508
|
+
const map = {};
|
|
509
|
+
function walk(node) {
|
|
510
|
+
if (!node || typeof node !== 'object') return;
|
|
511
|
+
if (node.id) {
|
|
512
|
+
map[node.id] = {
|
|
513
|
+
tag: node.tag,
|
|
514
|
+
type: node.type,
|
|
515
|
+
text: (node.metadata && node.metadata.text ? String(node.metadata.text).substring(0, 60) : null)
|
|
516
|
+
};
|
|
517
|
+
}
|
|
518
|
+
if (Array.isArray(node.children)) node.children.forEach(walk);
|
|
519
|
+
for (const k of Object.keys(node)) {
|
|
520
|
+
if (Array.isArray(node[k])) node[k].forEach(walk);
|
|
521
|
+
}
|
|
522
|
+
}
|
|
523
|
+
walk(apom.tree);
|
|
524
|
+
return map;
|
|
525
|
+
}, apomTreeConverter, elementModelBase + '\n' + elementModels + '\n' + modelRegistry);
|
|
526
|
+
}
|
|
527
|
+
|
|
482
528
|
if (name === "click") {
|
|
483
529
|
const validatedArgs = schemas.ClickSchema.parse(args);
|
|
484
530
|
const page = await getLastOpenPage();
|
|
@@ -500,13 +546,61 @@ async function executeToolInternal(name, args) {
|
|
|
500
546
|
throw new Error(`Element not found: ${identifier}`);
|
|
501
547
|
}
|
|
502
548
|
|
|
549
|
+
// Pre-click APOM snapshot (only if delta is requested) — captures every id
|
|
550
|
+
// currently in the tree so we can diff the post-click state.
|
|
551
|
+
let preSnap = null;
|
|
552
|
+
if (validatedArgs.autoAnalyzeAfter) {
|
|
553
|
+
preSnap = await getApomSnapshot(page);
|
|
554
|
+
}
|
|
555
|
+
|
|
503
556
|
// Use shared click action handler
|
|
504
|
-
|
|
557
|
+
const clickResult = await executeClickAction(page, element, {
|
|
505
558
|
identifier,
|
|
506
559
|
screenshot: validatedArgs.screenshot,
|
|
507
560
|
skipNetworkWait: validatedArgs.skipNetworkWait,
|
|
508
|
-
networkWaitTimeout: validatedArgs.networkWaitTimeout
|
|
561
|
+
networkWaitTimeout: validatedArgs.networkWaitTimeout,
|
|
562
|
+
waitForSelector: validatedArgs.waitForSelector,
|
|
563
|
+
waitTimeoutMs: validatedArgs.waitTimeoutMs
|
|
509
564
|
});
|
|
565
|
+
|
|
566
|
+
// Post-click APOM delta: re-register new ids so callers can immediately
|
|
567
|
+
// use them in follow-up click/type calls without an extra analyzePage call.
|
|
568
|
+
if (validatedArgs.autoAnalyzeAfter && preSnap) {
|
|
569
|
+
try {
|
|
570
|
+
await quickRegisterElements(page);
|
|
571
|
+
const postSnap = await getApomSnapshot(page);
|
|
572
|
+
const added = Object.keys(postSnap).filter(id => !preSnap[id]);
|
|
573
|
+
const removed = Object.keys(preSnap).filter(id => !postSnap[id]);
|
|
574
|
+
|
|
575
|
+
let deltaText = '\n\n** APOM DELTA **';
|
|
576
|
+
if (added.length === 0 && removed.length === 0) {
|
|
577
|
+
deltaText += '\nNo APOM changes detected after click';
|
|
578
|
+
} else {
|
|
579
|
+
if (added.length > 0) {
|
|
580
|
+
const sample = added.slice(0, 15).map(id => {
|
|
581
|
+
const m = postSnap[id] || {};
|
|
582
|
+
const lab = m.text ? `"${m.text}"` : (m.type || m.tag || '?');
|
|
583
|
+
return `${id}:${lab}`;
|
|
584
|
+
});
|
|
585
|
+
deltaText += `\n+${added.length} appeared: ${sample.join(', ')}${added.length > 15 ? `, ... (${added.length - 15} more)` : ''}`;
|
|
586
|
+
}
|
|
587
|
+
if (removed.length > 0) {
|
|
588
|
+
deltaText += `\n-${removed.length} disappeared`;
|
|
589
|
+
}
|
|
590
|
+
}
|
|
591
|
+
|
|
592
|
+
if (clickResult && Array.isArray(clickResult.content) && clickResult.content[0]) {
|
|
593
|
+
clickResult.content[0].text += deltaText;
|
|
594
|
+
}
|
|
595
|
+
} catch (e) {
|
|
596
|
+
// Delta is best-effort — never let it fail the actual click result
|
|
597
|
+
if (clickResult && Array.isArray(clickResult.content) && clickResult.content[0]) {
|
|
598
|
+
clickResult.content[0].text += `\n\n** APOM DELTA ** unavailable (${e.message})`;
|
|
599
|
+
}
|
|
600
|
+
}
|
|
601
|
+
}
|
|
602
|
+
|
|
603
|
+
return clickResult;
|
|
510
604
|
};
|
|
511
605
|
|
|
512
606
|
// Execute with timeout
|
|
@@ -786,6 +880,24 @@ async function executeToolInternal(name, args) {
|
|
|
786
880
|
// Get identifier (id or selector)
|
|
787
881
|
const identifier = validatedArgs.id || validatedArgs.selector;
|
|
788
882
|
|
|
883
|
+
// No identifier → full viewport screenshot. Mirrors processSceenshot pipeline used
|
|
884
|
+
// by element screenshots so the output (JPEG, scaled, base64) stays consistent.
|
|
885
|
+
if (!identifier) {
|
|
886
|
+
const buffer = await page.screenshot({ encoding: 'binary', fullPage: false });
|
|
887
|
+
const processed = await processScreenshot(buffer, {
|
|
888
|
+
maxWidth: validatedArgs.maxWidth !== undefined ? validatedArgs.maxWidth : 1024,
|
|
889
|
+
maxHeight: validatedArgs.maxHeight !== undefined ? validatedArgs.maxHeight : 8000,
|
|
890
|
+
quality: validatedArgs.quality || 40,
|
|
891
|
+
format: validatedArgs.format || 'jpeg',
|
|
892
|
+
});
|
|
893
|
+
return {
|
|
894
|
+
content: [
|
|
895
|
+
{ type: 'text', text: 'Viewport screenshot' },
|
|
896
|
+
{ type: 'image', data: processed.buffer.toString('base64'), mimeType: processed.mimeType }
|
|
897
|
+
]
|
|
898
|
+
};
|
|
899
|
+
}
|
|
900
|
+
|
|
789
901
|
// Resolve selector (supports both APOM ID and CSS selector)
|
|
790
902
|
const resolved = await resolveSelector(page, identifier);
|
|
791
903
|
if (!resolved.found) {
|
|
@@ -940,6 +1052,17 @@ async function executeToolInternal(name, args) {
|
|
|
940
1052
|
const page = await getLastOpenPage();
|
|
941
1053
|
const timeout = validatedArgs.timeout || 30000;
|
|
942
1054
|
|
|
1055
|
+
// Auto-wrap top-level `return` into an async IIFE: `eval('return X')` is an
|
|
1056
|
+
// Illegal return statement in script context, so users had to wrap manually.
|
|
1057
|
+
// Heuristic: rewrite only when the snippet starts with `return ...` (most common
|
|
1058
|
+
// case from QA reports). Skip when the snippet declares a function — that signals
|
|
1059
|
+
// an explicit user-defined scope and an implicit-return result is likely intended.
|
|
1060
|
+
let scriptToRun = validatedArgs.script;
|
|
1061
|
+
const trimmedHead = scriptToRun.replace(/^\s+/, '');
|
|
1062
|
+
if (/^return[\s;]/.test(trimmedHead) && !/\bfunction\s*[\w*(]/.test(scriptToRun)) {
|
|
1063
|
+
scriptToRun = `(async () => { ${scriptToRun} })()`;
|
|
1064
|
+
}
|
|
1065
|
+
|
|
943
1066
|
// Wrap operation in timeout
|
|
944
1067
|
const executeOperation = async () => {
|
|
945
1068
|
// Use page.evaluate with async support — if eval returns a Promise,
|
|
@@ -956,7 +1079,7 @@ async function executeToolInternal(name, args) {
|
|
|
956
1079
|
} catch (error) {
|
|
957
1080
|
return { success: false, error: error.message };
|
|
958
1081
|
}
|
|
959
|
-
},
|
|
1082
|
+
}, scriptToRun);
|
|
960
1083
|
|
|
961
1084
|
await new Promise(resolve => setTimeout(resolve, validatedArgs.waitAfter || 500));
|
|
962
1085
|
|
|
@@ -2421,7 +2544,7 @@ Start coding now.`;
|
|
|
2421
2544
|
}
|
|
2422
2545
|
|
|
2423
2546
|
// APOM Tree format (default) - v2 with tree structure and positioning
|
|
2424
|
-
const apomResult = await page.evaluate(async (apomTreeConverterCode, selectorResolverCode, modelsCode, shouldRegister, includeAll, viewportOnly) => {
|
|
2547
|
+
const apomResult = await page.evaluate(async (apomTreeConverterCode, selectorResolverCode, modelsCode, shouldRegister, includeAll, viewportOnly, portalOpts) => {
|
|
2425
2548
|
// Inject utilities if not already loaded
|
|
2426
2549
|
if (typeof buildAPOMTree === 'undefined') {
|
|
2427
2550
|
eval(apomTreeConverterCode);
|
|
@@ -2458,7 +2581,7 @@ Start coding now.`;
|
|
|
2458
2581
|
|
|
2459
2582
|
// Build APOM tree
|
|
2460
2583
|
// interactiveOnly = !includeAll (if includeAll is true, we want ALL elements)
|
|
2461
|
-
const apomData = buildAPOMTree(!includeAll, viewportOnly);
|
|
2584
|
+
const apomData = buildAPOMTree(!includeAll, viewportOnly, portalOpts);
|
|
2462
2585
|
|
|
2463
2586
|
// Register elements in selector resolver if requested
|
|
2464
2587
|
if (shouldRegister) {
|
|
@@ -2491,7 +2614,10 @@ Start coding now.`;
|
|
|
2491
2614
|
}
|
|
2492
2615
|
|
|
2493
2616
|
return apomData;
|
|
2494
|
-
}, apomTreeConverter, selectorResolver, elementModelBase + '\n' + elementModels + '\n' + modelRegistry, validatedArgs.registerElements !== false, validatedArgs.includeAll || false, validatedArgs.viewportOnly || false
|
|
2617
|
+
}, apomTreeConverter, selectorResolver, elementModelBase + '\n' + elementModels + '\n' + modelRegistry, validatedArgs.registerElements !== false, validatedArgs.includeAll || false, validatedArgs.viewportOnly || false, {
|
|
2618
|
+
include: validatedArgs.includePortals !== false,
|
|
2619
|
+
selectors: validatedArgs.portalSelectors || undefined
|
|
2620
|
+
});
|
|
2495
2621
|
|
|
2496
2622
|
// Handle diff mode
|
|
2497
2623
|
if (validatedArgs.diff) {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "chrometools-mcp",
|
|
3
|
-
"version": "3.5.
|
|
3
|
+
"version": "3.5.6",
|
|
4
4
|
"description": "MCP (Model Context Protocol) server for Chrome automation using Puppeteer. Persistent browser sessions, UI framework detection (MUI, Ant Design, etc.), Page Object support, visual testing, Figma comparison. Works seamlessly in WSL, Linux, macOS, and Windows.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
|
@@ -37,9 +37,16 @@ function initializeModelRegistry() {
|
|
|
37
37
|
*
|
|
38
38
|
* @param {boolean} interactiveOnly - Only include interactive elements and their parents
|
|
39
39
|
* @param {boolean} viewportOnly - Only include elements visible in current viewport
|
|
40
|
+
* @param {Object} portalOpts - Portal scan options: { include: boolean, selectors: string[] }
|
|
41
|
+
* When include=true, force-includes contents of React Portal containers
|
|
42
|
+
* (e.g. #menu-popup-root, #tooltip-root) that live outside main React root.
|
|
40
43
|
* @returns {Object} APOM tree structure
|
|
41
44
|
*/
|
|
42
|
-
function buildAPOMTree(interactiveOnly = true, viewportOnly = false) {
|
|
45
|
+
function buildAPOMTree(interactiveOnly = true, viewportOnly = false, portalOpts = undefined) {
|
|
46
|
+
const portalInclude = portalOpts ? portalOpts.include !== false : true;
|
|
47
|
+
const portalSelectors = (portalOpts && Array.isArray(portalOpts.selectors) && portalOpts.selectors.length > 0)
|
|
48
|
+
? portalOpts.selectors
|
|
49
|
+
: ['#modal-root', '#menu-popup-root', '#tooltip-root', '#popover-root', '[data-portal]'];
|
|
43
50
|
const pageId = `page_${btoa(window.location.href).replace(/[^a-zA-Z0-9]/g, '').substring(0, 20)}_${Date.now()}`;
|
|
44
51
|
|
|
45
52
|
// Initialize model registry (Strategy Pattern setup)
|
|
@@ -116,6 +123,54 @@ function buildAPOMTree(interactiveOnly = true, viewportOnly = false) {
|
|
|
116
123
|
forceMarkModalTree(el);
|
|
117
124
|
}
|
|
118
125
|
});
|
|
126
|
+
|
|
127
|
+
// Generic portal containers (menus, tooltips, popovers) — opt-in by selector list.
|
|
128
|
+
// Unlike framework modals, these are app-defined wrappers (e.g. #menu-popup-root from
|
|
129
|
+
// client/index.html). When non-empty, force-include their subtree.
|
|
130
|
+
if (portalInclude && portalSelectors.length > 0) {
|
|
131
|
+
try {
|
|
132
|
+
document.querySelectorAll(portalSelectors.join(',')).forEach(container => {
|
|
133
|
+
if (!container.children || container.children.length === 0) return;
|
|
134
|
+
for (const child of container.children) {
|
|
135
|
+
if (!modalElements.has(child)) {
|
|
136
|
+
forceMarkModalTree(child);
|
|
137
|
+
}
|
|
138
|
+
}
|
|
139
|
+
});
|
|
140
|
+
} catch (e) {
|
|
141
|
+
// Invalid selector in portalSelectors — skip silently, do not break analyzePage
|
|
142
|
+
}
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
// In-tree popups (Popper/Tippy/FloatingUI/custom contextMenu pattern). Some libs
|
|
146
|
+
// render the popup inside a 0-height inline wrapper and then absolute-position it
|
|
147
|
+
// out of the wrapper's box. Default isVisible() drops the wrapper (height: 0) and
|
|
148
|
+
// every popup descendant is lost. Detect: a 0×0 wrapper whose subtree contains a
|
|
149
|
+
// positioned (absolute/fixed) child with real bounds — force-mark that child.
|
|
150
|
+
if (portalInclude) {
|
|
151
|
+
function findPositionedPopup(el, maxDepth) {
|
|
152
|
+
if (maxDepth <= 0) return null;
|
|
153
|
+
for (const child of el.children) {
|
|
154
|
+
if (modalElements.has(child)) continue;
|
|
155
|
+
const cs = window.getComputedStyle(child);
|
|
156
|
+
if ((cs.position === 'absolute' || cs.position === 'fixed') &&
|
|
157
|
+
child.offsetWidth > 0 && child.offsetHeight > 0) {
|
|
158
|
+
return child;
|
|
159
|
+
}
|
|
160
|
+
const deeper = findPositionedPopup(child, maxDepth - 1);
|
|
161
|
+
if (deeper) return deeper;
|
|
162
|
+
}
|
|
163
|
+
return null;
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
const allEls = document.body.querySelectorAll('*');
|
|
167
|
+
for (const el of allEls) {
|
|
168
|
+
// Only zero-sized wrappers with children are candidates — cheap filter
|
|
169
|
+
if ((el.offsetHeight !== 0 && el.offsetWidth !== 0) || el.children.length === 0) continue;
|
|
170
|
+
const popup = findPositionedPopup(el, 3);
|
|
171
|
+
if (popup) forceMarkModalTree(popup);
|
|
172
|
+
}
|
|
173
|
+
}
|
|
119
174
|
}
|
|
120
175
|
|
|
121
176
|
// Build tree from body
|
|
@@ -37,6 +37,9 @@ export const toolDefinitions = [
|
|
|
37
37
|
waitAfter: { type: "number", description: "Wait ms (default: 1500)" },
|
|
38
38
|
screenshot: { type: "boolean", description: "Screenshot (default: false)" },
|
|
39
39
|
timeout: { type: "number", description: "Max wait ms (default: 30000)" },
|
|
40
|
+
waitForSelector: { type: "string", description: "CSS selector to wait for after click (atomic click+wait). Use for dropdowns/popups that render into portals." },
|
|
41
|
+
waitTimeoutMs: { type: "number", description: "Timeout for waitForSelector in ms (default: 2000)." },
|
|
42
|
+
autoAnalyzeAfter: { type: "boolean", description: "After click, diff APOM and append '+N appeared: id:\"text\"' delta to result. New ids are pre-registered for follow-up clicks. Use for dropdowns/menus opening with new options." },
|
|
40
43
|
},
|
|
41
44
|
},
|
|
42
45
|
},
|
|
@@ -92,18 +95,18 @@ export const toolDefinitions = [
|
|
|
92
95
|
},
|
|
93
96
|
{
|
|
94
97
|
name: "screenshot",
|
|
95
|
-
description: "Capture element image (5-10k tokens). Use analyzePage for form data/validation (8-10k tokens).",
|
|
98
|
+
description: "Capture element image (5-10k tokens), or full viewport when no id/selector is given. Use analyzePage for form data/validation (8-10k tokens).",
|
|
96
99
|
inputSchema: {
|
|
97
100
|
type: "object",
|
|
98
101
|
properties: {
|
|
99
|
-
|
|
100
|
-
|
|
102
|
+
id: { type: "string", description: "APOM element ID. Mutually exclusive with selector. Omit both for viewport screenshot." },
|
|
103
|
+
selector: { type: "string", description: "CSS selector. Mutually exclusive with id. Omit both for viewport screenshot." },
|
|
104
|
+
padding: { type: "number", description: "Padding px (default: 0). Ignored for viewport." },
|
|
101
105
|
maxWidth: { type: "number", description: "Max width px (default: 1024, null=original)" },
|
|
102
106
|
maxHeight: { type: "number", description: "Max height px (default: 8000, null=original)" },
|
|
103
107
|
quality: { type: "number", minimum: 1, maximum: 100, description: "JPEG quality (default: 40)" },
|
|
104
108
|
format: { type: "string", enum: ["png", "jpeg", "auto"], description: "Format (default: jpeg)" },
|
|
105
109
|
},
|
|
106
|
-
required: ["selector"],
|
|
107
110
|
},
|
|
108
111
|
},
|
|
109
112
|
{
|
|
@@ -532,6 +535,8 @@ Examples:
|
|
|
532
535
|
groupBy: { type: "string", description: "Group elements: 'type' or 'flat' (default: 'type')", enum: ["type", "flat"] },
|
|
533
536
|
viewportOnly: { type: "boolean", description: "Only analyze elements in current viewport (default: false). Reduces output for long pages." },
|
|
534
537
|
diff: { type: "boolean", description: "Return only changes since last analysis: {added, removed, changed} (default: false)." },
|
|
538
|
+
includePortals: { type: "boolean", description: "Include React Portal contents — menus, tooltips, popovers outside main root (default: true). Without this, dropdown items are invisible." },
|
|
539
|
+
portalSelectors: { type: "array", items: { type: "string" }, description: "Custom portal root CSS selectors. Default: ['#modal-root', '#menu-popup-root', '#tooltip-root', '#popover-root', '[data-portal]']." },
|
|
535
540
|
},
|
|
536
541
|
},
|
|
537
542
|
},
|
package/server/tool-schemas.js
CHANGED
|
@@ -23,6 +23,9 @@ export const ClickSchema = z.object({
|
|
|
23
23
|
timeout: z.number().optional().describe("Maximum time to wait for operation in ms (default: 30000)"),
|
|
24
24
|
skipNetworkWait: z.boolean().optional().describe("Skip waiting for network requests (default: false). Use for forms with long-polling/WebSockets to avoid timeouts."),
|
|
25
25
|
networkWaitTimeout: z.number().optional().describe("Maximum time to wait for network requests in ms (default: 3000). Only used if skipNetworkWait is false."),
|
|
26
|
+
waitForSelector: z.string().optional().describe("CSS selector to wait for after click — atomic click+wait. Useful for dropdowns/popups in portals (e.g. '#menu-popup-root > div') that otherwise race against the next MCP call."),
|
|
27
|
+
waitTimeoutMs: z.number().optional().describe("Timeout for waitForSelector in ms (default: 2000)."),
|
|
28
|
+
autoAnalyzeAfter: z.boolean().optional().describe("After click, automatically diff APOM state and append a delta to the result: '+N appeared: id1:\"text\", id2:\"text\"'. New ids are re-registered so callers can use them directly in the next click/type call without an extra analyzePage. Use for dropdowns and menus that reveal new options on click."),
|
|
26
29
|
}).refine(data => (data.id && !data.selector) || (!data.id && data.selector), {
|
|
27
30
|
message: "Either 'id' or 'selector' must be provided, but not both"
|
|
28
31
|
});
|
|
@@ -110,15 +113,15 @@ export const SetStylesSchema = z.object({
|
|
|
110
113
|
|
|
111
114
|
// Screenshot tools
|
|
112
115
|
export const ScreenshotSchema = z.object({
|
|
113
|
-
id: z.string().optional().describe("APOM element ID from analyzePage (e.g., 'div_20'). Mutually exclusive with selector."),
|
|
114
|
-
selector: z.string().optional().describe("CSS selector for element to screenshot. Mutually exclusive with id."),
|
|
115
|
-
padding: z.number().optional().describe("Padding around element in pixels (default: 0)"),
|
|
116
|
+
id: z.string().optional().describe("APOM element ID from analyzePage (e.g., 'div_20'). Mutually exclusive with selector. If neither id nor selector is provided, captures full viewport."),
|
|
117
|
+
selector: z.string().optional().describe("CSS selector for element to screenshot. Mutually exclusive with id. If neither id nor selector is provided, captures full viewport."),
|
|
118
|
+
padding: z.number().optional().describe("Padding around element in pixels (default: 0). Ignored for viewport screenshot."),
|
|
116
119
|
maxWidth: z.number().nullable().optional().describe("Maximum width in pixels, auto-scales if larger (default: 1024, set to null for original size)"),
|
|
117
120
|
maxHeight: z.number().nullable().optional().describe("Maximum height in pixels, auto-scales if larger (default: 8000 for API limit, set to null for original size)"),
|
|
118
121
|
quality: z.number().min(1).max(100).optional().describe("JPEG quality 1-100 (default: 40)"),
|
|
119
122
|
format: z.enum(['png', 'jpeg', 'auto']).optional().describe("Image format (default: 'jpeg')"),
|
|
120
|
-
}).refine(data =>
|
|
121
|
-
message: "
|
|
123
|
+
}).refine(data => !(data.id && data.selector), {
|
|
124
|
+
message: "Provide only one of 'id' or 'selector' (or neither for a viewport screenshot)"
|
|
122
125
|
});
|
|
123
126
|
|
|
124
127
|
export const SaveScreenshotSchema = z.object({
|
|
@@ -289,6 +292,8 @@ export const AnalyzePageSchema = z.object({
|
|
|
289
292
|
groupBy: z.enum(['type', 'flat']).optional().describe("Group elements by type or return flat structure (default: 'type')"),
|
|
290
293
|
viewportOnly: z.boolean().optional().describe("Only analyze elements visible in current viewport (default: false). Reduces output for long pages."),
|
|
291
294
|
diff: z.boolean().optional().describe("Return only changes since last analysis: {added, removed, changed} (default: false). Useful after clicks to see what changed."),
|
|
295
|
+
includePortals: z.boolean().optional().describe("Include contents of React Portal containers that live outside main React root (default: true). Covers menus, tooltips, popovers rendered via portals — without this, dropdown contents are invisible to analyzePage."),
|
|
296
|
+
portalSelectors: z.array(z.string()).optional().describe("CSS selectors of portal root containers to scan (default: ['#modal-root', '#menu-popup-root', '#tooltip-root', '#popover-root', '[data-portal]']). Provide custom list when the app uses different portal element ids."),
|
|
292
297
|
});
|
|
293
298
|
|
|
294
299
|
export const GetElementDetailsSchema = z.object({
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Progress: SEGM-537 QA Unblockers
|
|
2
|
+
|
|
3
|
+
**Status**: NEEDS APPROVAL
|
|
4
|
+
**Spec**: [SEGM-537-UNBLOCKERS_SPEC.md](./SEGM-537-UNBLOCKERS_SPEC.md)
|
|
5
|
+
**Created**: 2026-05-28
|
|
6
|
+
|
|
7
|
+
## Phase 0 — Approval & alignment
|
|
8
|
+
|
|
9
|
+
- [ ] Пользователь одобрил scope (все 5 блокеров)
|
|
10
|
+
- [ ] Пользователь одобрил дизайн API (имена параметров, дефолты)
|
|
11
|
+
- [ ] Дать команду «погнали, начни с Phase 1»
|
|
12
|
+
|
|
13
|
+
## Phase 1 — Блокер #1: portal scan расширение
|
|
14
|
+
|
|
15
|
+
- [x] Расширить `pom/apom-tree-converter.js` — добавить `portalSelectors` параметр + третий проход (~120)
|
|
16
|
+
- [x] ~~Переименовать `forceMarkModalTree`~~ — переиспользована as-is, переименование избыточно
|
|
17
|
+
- [x] Прокинуть `includePortals` / `portalSelectors` через `page.evaluate` в `index.js`
|
|
18
|
+
- [x] Обновить `server/tool-schemas.js` + `server/tool-definitions.js` для `analyzePage`
|
|
19
|
+
- [x] `npm run build` — Syntax validation passed
|
|
20
|
+
- [ ] Verification: бенчмарк Google (требует запуска MCP — пользователь должен перезапустить и прогнать)
|
|
21
|
+
- [ ] Verification: ручной тест на QA-стенде (требует доступа QA — отдаём после рестарта MCP)
|
|
22
|
+
- [x] Обновить `README.md` — секция `analyzePage`
|
|
23
|
+
|
|
24
|
+
## Phase 2 — Блокер #2: click + waitForSelector
|
|
25
|
+
|
|
26
|
+
- [x] Добавить `waitForSelector` / `waitTimeoutMs` в `executeClickAction` + `click` handler
|
|
27
|
+
- [x] Возвращать `appearedInMs` в результате (или `⚠️ WAIT_TIMEOUT` сообщение)
|
|
28
|
+
- [x] Обновить `server/tool-schemas.js` + `server/tool-definitions.js` для `click`
|
|
29
|
+
- [x] `npm run build` — Syntax validation passed
|
|
30
|
+
- [ ] Verification: на стенде клик «три точки» с `waitForSelector` (требует рестарта MCP)
|
|
31
|
+
- [ ] Verification: несуществующий селектор → таймаут (требует рестарта MCP)
|
|
32
|
+
- [x] Обновить `README.md` — секция `click`
|
|
33
|
+
|
|
34
|
+
## Phase 3 — Блокер #4: ModelRegistry stale error
|
|
35
|
+
|
|
36
|
+
- [x] **Root cause fix**: `quickRegisterElements` теперь инжектит models code (раньше не делал → ReferenceError при auto-refresh после navigation)
|
|
37
|
+
- [x] User-friendly fallback: catch на `ReferenceError: ModelRegistry is not defined` в `resolveSelector` → сообщение «APOM registry stale, call analyzePage()»
|
|
38
|
+
- [x] `npm run build` — Syntax validation passed
|
|
39
|
+
- [ ] Verification: воспроизвести (analyzePage → navigate → click) — требует рестарта MCP
|
|
40
|
+
|
|
41
|
+
## Phase 4 — Блокер #3: screenshot без selector
|
|
42
|
+
|
|
43
|
+
- [x] Ослабить `.refine` в `ScreenshotSchema` (запрещаем только конфликт, не отсутствие)
|
|
44
|
+
- [x] Без id/selector — viewport screenshot через `processScreenshot`
|
|
45
|
+
- [x] Обновить `server/tool-schemas.js` + `server/tool-definitions.js`
|
|
46
|
+
- [x] `npm run build` — Syntax validation passed
|
|
47
|
+
- [ ] Verification: `screenshot()` без аргументов возвращает viewport — требует рестарта MCP
|
|
48
|
+
- [x] Обновить `README.md` — секция `screenshot`
|
|
49
|
+
|
|
50
|
+
## Phase 5 — Блокер #5: executeScript auto-IIFE
|
|
51
|
+
|
|
52
|
+
- [x] Препроцессор скрипта в `executeScript` handler (index.js)
|
|
53
|
+
- [x] Regex: оборачивает только если `^return[\s;]` И код не содержит `function ...`
|
|
54
|
+
- [x] `npm run build` — Syntax validation passed
|
|
55
|
+
- [x] Verification (offline): 5 канонических случаев работают корректно (`return document.title`, ` return 42;`, IIFE, plain expr, `function ... return`)
|
|
56
|
+
- [x] Обновить `README.md` — секция `executeScript`
|
|
57
|
+
|
|
58
|
+
## Phase 7 — click autoAnalyzeAfter
|
|
59
|
+
|
|
60
|
+
- [x] Helper `getApomSnapshot(page)` в index.js
|
|
61
|
+
- [x] Pre/post snapshot в click handler с регистрацией новых id через `quickRegisterElements`
|
|
62
|
+
- [x] Дельта `+N appeared: id:"text"` / `-N disappeared` / `No APOM changes` в результате click
|
|
63
|
+
- [x] Schema + tool-definitions + README
|
|
64
|
+
- [x] `npm run build` — passed
|
|
65
|
+
- [x] Verified на стенде: Phase 7 механика работает (snapshots + diff + content append). На SEGM-стенде дельта пустая из-за Phase 8 проблемы (popup не попадает в дерево).
|
|
66
|
+
|
|
67
|
+
## Phase 8 — In-tree popup detection
|
|
68
|
+
|
|
69
|
+
Открыт **при e2e-тесте Phase 7**: popup-меню стенда (Popper-style) рендерится внутри 0-height wrapper и не попадает в APOM tree, потому что `isVisible` отбрасывает wrapper.
|
|
70
|
+
|
|
71
|
+
- [x] Расширение portal-scan в `pom/apom-tree-converter.js` — новый блок после id-portalSelectors
|
|
72
|
+
- [x] `findPositionedPopup(el, depth=3)` — рекурсивный поиск absolute/fixed-positioned child с реальными bounds внутри 0×0 wrapper
|
|
73
|
+
- [x] force-mark найденного popup через существующий `forceMarkModalTree`
|
|
74
|
+
- [x] Используется тот же flag `portalInclude` (default `true`) — opt-out возможен через `includePortals: false`
|
|
75
|
+
- [x] README обновлён
|
|
76
|
+
- [x] `npm run build` — passed
|
|
77
|
+
- [ ] Verification на SEGM-стенде — требует рестарта MCP
|
|
78
|
+
|
|
79
|
+
## Phase 6 — Финализация сессии
|
|
80
|
+
|
|
81
|
+
- [ ] Прогон всех verification на реальном QA-стенде SEGM-537 (если доступ есть)
|
|
82
|
+
- [ ] Спросить пользователя: bump version + CHANGELOG entry?
|
|
83
|
+
- [ ] Если YES — single CHANGELOG entry для всей сессии, версия += patch (3.5.4 → 3.5.5)
|
|
84
|
+
- [ ] Отдать QA на повтор сценария из отчёта
|
|
85
|
+
|
|
86
|
+
## Verification status: per-phase checks
|
|
87
|
+
|
|
88
|
+
Каждая phase имеет свои verification-шаги выше. Не двигаться к следующей пока текущая не зелёная.
|
|
89
|
+
|
|
90
|
+
## Открытые вопросы (повторяю из spec)
|
|
91
|
+
|
|
92
|
+
1. Дефолтные id-портал-селекторы (`#menu-popup-root`, `#modal-root`, `#tooltip-root`, `#popover-root`) — норм или хотите другой набор?
|
|
93
|
+
2. `keepOpen: true` для `click` — пропускаем, делаем только `waitForSelector`. Если QA увидит что попап всё равно закрывается — добавим в Phase 2.5.
|
|
94
|
+
3. Авто-IIFE для `executeScript` — допустимый риск перехвата `return` в случаях, где `return` намеренно внутри функции? Митигируем regex'ом на top-level.
|