replicas-cli 0.2.354 → 0.2.356

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/dist/index.mjs +1377 -161
  2. package/package.json +1 -1
package/dist/index.mjs CHANGED
@@ -8010,13 +8010,12 @@ replicas computer info
8010
8010
  # 2) Launch a browser on the workspace display.
8011
8011
  replicas computer launch chrome https://news.ycombinator.com
8012
8012
 
8013
- # 3) Observe the settled screen and browser tab state before clicking.
8014
- replicas computer observe /tmp/state.png
8015
- replicas computer browser --snapshot
8013
+ # 3) Capture the settled viewport and native accessibility tree together.
8014
+ replicas computer browser-state /tmp/state.png --full
8016
8015
 
8017
- # 4) Drive the UI.
8018
- replicas computer click 521 700 # click coordinates from the screenshot
8019
- replicas computer scroll down --amount 5
8016
+ # 4) Batch actions that use refs from the same state, then capture the semantic diff.
8017
+ replicas computer browser-batch '[{"action":"fill","ref":17,"value":"hello"},{"action":"click","ref":42}]'
8018
+ replicas computer browser-state /tmp/after-click.png
8020
8019
 
8021
8020
  # 5) (Optional) Record a screencap to share back.
8022
8021
  replicas computer record start /tmp/demo.mp4 --fps 60
@@ -8045,21 +8044,33 @@ Waits briefly for the screen to stop changing, saves a 1:1 screenshot, and print
8045
8044
 
8046
8045
  Use this instead of hand-written \`sleep && screenshot\` loops after clicks, navigation, typing, or page loads. By default it saves a 100px coordinate-grid screenshot and waits up to 3s for 600ms of visual stability. Pass \`--raw\` if you need an unannotated 1:1 screenshot.
8047
8046
 
8048
- ### \`replicas computer browser\`
8049
- Prints JSON for Chrome tabs launched through Replicas, including page titles and URLs. Pass \`--snapshot\` to include visible page text and interactive controls with DOM viewport bounding boxes.
8047
+ ### \`replicas computer browser-state <path> [--full] [--timeout MS] [--stable-ms MS]\`
8048
+ Waits for the selected Chrome page to settle, saves its viewport screenshot, and prints a compact native Chrome accessibility tree in the same coordinate space. Actionable nodes have DOM-backed refs such as \`[ref=42] button "Save"\`; the tree includes accessible roles, names, values, states, visible bounds, and iframe content.
8049
+
8050
+ The first call returns a full tree. Later calls for the same target and document return semantic changes; navigation or reload forces a new full tree. Pass \`--full\` to reset the baseline. Refs are valid for the current document.
8051
+
8052
+ ### \`replicas computer browser [--snapshot]\`
8053
+ Lists Chrome tabs. \`--snapshot\` returns the compatibility accessibility snapshot. Use \`browser-state\` for the primary action/observe loop.
8054
+
8055
+ ### \`replicas computer browser-batch '<actions-json>'\`
8056
+ Runs up to 100 ordered browser actions through one Chrome session and stops at the first failure. Supported actions are \`click\`, \`fill\`, \`key\`, \`type\`, \`scroll\`, and \`wait\`. Prefer this when several actions use refs from the same \`browser-state\`; it avoids repeated process, tab-selection, and connection setup. Do not carry refs across navigation\u2014capture fresh state and start a new batch.
8057
+
8058
+ ### \`replicas computer browser-click --ref ID [--button left|middle|right] [--double]\`
8059
+ Scrolls the referenced node into view and sends trusted Chrome mouse input at its current visible bounds. Prefer refs from \`browser-state\`. \`browser-click <text> [--exact] [--index N]\` remains a fallback when no fresh ref is available.
8050
8060
 
8051
- Use this alongside \`observe\` when testing web apps so you do not infer navigation, page state, or click targets from pixels alone. Snapshot coordinates are DOM viewport coordinates, not desktop click coordinates.
8061
+ ### \`replicas computer browser-fill --ref ID <value>\`
8062
+ Focuses the referenced field, fills it with trusted Chrome input, and verifies the resulting value. Select controls use their semantic option value. \`browser-fill <field> <value> [--exact] [--index N]\` remains a text-matching fallback.
8052
8063
 
8053
- ### \`replicas computer browser-click <text> [--exact] [--index N]\`
8054
- Clicks the first visible Chrome control whose text, label, placeholder, or href matches \`<text>\`. Use this for web buttons and links found via \`browser --snapshot\`; it is faster and less error-prone than converting DOM coordinates to desktop pixels.
8064
+ ### \`replicas computer browser-scroll <up|down|left|right> [--ref ID] [--amount PX]\`
8065
+ Sends trusted wheel input at a referenced element or the center of the viewport.
8055
8066
 
8056
- ### \`replicas computer browser-fill <field> <value> [--exact] [--index N]\`
8057
- Fills the first visible Chrome field whose label, placeholder, name, or text matches \`<field>\`, then dispatches input/change events. Use this for web forms instead of clicking a field and typing through the desktop.
8067
+ ### \`replicas computer browser-key <combo>\` / \`browser-type <text>\`
8068
+ Sends trusted keyboard input to the focused element in the selected Chrome target. Prefer \`browser-fill\` when setting a known field value.
8058
8069
 
8059
8070
  ### \`replicas computer browser-wait <text> [--mode any|text|title|url|control] [--exact] [--timeout MS]\`
8060
8071
  Waits until the active Chrome page matches text in the title, URL, body text, or visible controls. Use this after \`browser-click\` / \`browser-fill\` when you need web app state to settle without screenshot polling.
8061
8072
 
8062
- For \`browser-click\`, \`browser-fill\`, and \`browser-wait\`, pass \`--id <id>\`, \`--title <text>\`, \`--url <text>\`, or \`--page <n>\` when multiple Chrome tabs are open. Run \`replicas computer browser\` first to list tabs. Prefer \`--id\` when a click may change the page title or URL.
8073
+ For browser state and actions, pass \`--target-id <id>\`, \`--title <text>\`, \`--url <text>\`, or \`--page <n>\` when multiple Chrome tabs are open. Prefer \`--target-id\` because titles and URLs can change after an action. The matched tab is brought to the foreground before each action, so driving multiple tabs or recording the screen switches which tab is visible.
8063
8074
 
8064
8075
  ### \`replicas computer click <x> <y> [--button N] [--double] [--modifiers ctrl+shift]\`
8065
8076
  Move to (x, y) and click. Coordinates can be absolute pixels or percentages such as \`50%\` \`50%\`. Default is left-click (button 1); pass \`--button 3\` for right-click. \`--modifiers\` holds keys during the click (e.g. ctrl-click a link to open in a new tab).
@@ -8092,26 +8103,29 @@ Spawns an app on the workspace display. Built-in aliases:
8092
8103
  Anything else gets \`exec\`'d verbatim, so \`replicas computer launch xeyes\` works if xeyes is installed.
8093
8104
  When opening a known page, prefer passing the URL directly: \`replicas computer launch chrome http://localhost:3000/\`.
8094
8105
 
8106
+ Chrome launches do not return until the requested page is controllable. The output includes its page ID; carry that ID with \`--target-id\` into \`browser-state\` and browser actions so unrelated tabs cannot steal task context.
8107
+
8095
8108
  ### \`replicas computer record start <path> [--fps N]\`
8096
- Starts an ffmpeg screen recorder. Output is post-processed when stopped: action windows stay at normal speed, idle gaps accelerate, click moments get eased camera zoom, and a synthetic cursor animates between logged mouse positions. The raw capture is fragmented MP4 while active (still playable if the workspace dies mid-record). Default 60fps; drop to 30 if the workspace is CPU-constrained.
8109
+ Starts an ffmpeg screen recorder and returns only after ffmpeg has written the first bytes. Output is post-processed when stopped: action windows stay at normal speed, idle gaps accelerate, click moments get eased camera zoom, and a synthetic cursor animates between logged mouse positions. The raw capture is fragmented MP4 while active (still playable if the workspace dies mid-record). Default 60fps; drop to 30 if the workspace is CPU-constrained.
8097
8110
 
8098
8111
  Only one recording at a time. Re-running \`start\` while one is active fails - call \`stop\` first.
8099
8112
 
8100
8113
  ### \`replicas computer record stop\`
8101
- SIGINTs ffmpeg, waits for it to finalize the MP4, prints the output path. Upload it with \`replicas media upload <path>\` to share it.
8114
+ SIGINTs ffmpeg, verifies that the process has finalized the MP4, then prints the output path. A finalization timeout preserves the recording state so \`stop\` can be retried safely. Upload it with \`replicas media upload <path>\` to share it.
8102
8115
 
8103
8116
  ## Patterns
8104
8117
 
8105
- ### Action / observe loop
8106
- You are blind between tool calls. After any action that changes the screen, observe before deciding the next coordinate:
8118
+ ### Browser action / state loop
8119
+ For Chrome, use the native accessibility state before interpreting pixels or guessing coordinates:
8107
8120
 
8108
8121
  \`\`\`bash
8109
- replicas computer click 521 700
8110
- replicas computer observe /tmp/after-click.png
8111
- # read /tmp/after-click.png and the JSON output, decide next click
8122
+ replicas computer launch chrome https://example.com
8123
+ replicas computer browser-state /tmp/before.png --full --target-id <page-id>
8124
+ replicas computer browser-batch '[{"action":"fill","ref":17,"value":"hello"},{"action":"click","ref":23}]' --target-id <page-id>
8125
+ replicas computer browser-state /tmp/after.png --target-id <page-id>
8112
8126
  \`\`\`
8113
8127
 
8114
- The JSON \`stable\`, \`frames\`, and \`changes\` fields tell you whether something changed while you were waiting. If \`stable\` is false, observe again or increase \`--timeout\` before acting on coordinates.
8128
+ Read the screenshot and tree as one state. Batch ordered actions whose refs all come from that state, then inspect the next semantic diff. A batch is fail-fast and validates each ref against the cached document. Split at navigation or whenever a later action depends on newly rendered refs. Fall back to \`observe\` plus desktop coordinates for canvas content, browser chrome, non-Chrome apps, or controls missing from the accessibility tree.
8115
8129
 
8116
8130
  ### Typing into an address bar
8117
8131
  \`\`\`bash
@@ -8120,10 +8134,9 @@ replicas computer observe /tmp/browser-open.png
8120
8134
  replicas computer key ctrl+l
8121
8135
  replicas computer type "https://example.com"
8122
8136
  replicas computer key Return
8123
- replicas computer observe /tmp/loaded.png
8124
- replicas computer browser --snapshot
8125
- replicas computer browser-fill "Search" "replicas"
8126
- replicas computer browser-click "More information" --title "Example"
8137
+ replicas computer browser-state /tmp/loaded.png --full
8138
+ replicas computer browser-fill --ref 17 "replicas"
8139
+ replicas computer browser-click --ref 23 --title "Example"
8127
8140
  replicas computer browser-wait "Example Domain" --mode title --title "Example"
8128
8141
  \`\`\`
8129
8142
 
@@ -9283,6 +9296,7 @@ In agent mode the CLI hides commands that don't make sense for in-workspace agen
9283
9296
  | \`replicas automation ...\` | Manage automations (cron + GitHub/GitLab event triggers) |
9284
9297
  | \`replicas preview ...\` | Register / list preview URLs (covered in \`PREVIEWS.md\`) |
9285
9298
  | \`replicas media ...\` | Upload screenshots, videos, audio (covered in \`MEDIA.md\`) |
9299
+ | \`replicas slack thread ...\` | Attach or switch Slack thread routing (covered in \`SLACK.md\`) |
9286
9300
 
9287
9301
  \`replicas <command> --help\` is always the source of truth for flags.
9288
9302
 
@@ -9516,6 +9530,30 @@ curl -s -X POST "https://slack.com/api/chat.postMessage" \\
9516
9530
 
9517
9531
  Omit \`thread_ts\` to post a new message to the channel. Include it to reply in a thread.
9518
9532
 
9533
+ #### Attach every new conversation
9534
+
9535
+ Whenever you send a Slack message outside the thread that already routes to the current workspace, immediately attach the destination thread so future replies are delivered to this workspace:
9536
+
9537
+ \`\`\`bash
9538
+ replicas slack thread attach --channel <channel-id> --thread-ts <thread-root-ts>
9539
+ \`\`\`
9540
+
9541
+ - For a new top-level message or DM, use the \`channel\` and \`ts\` from the successful \`chat.postMessage\` response. That message's \`ts\` is the new thread root.
9542
+ - For a reply in another existing thread, use its root \`thread_ts\`, not the reply's \`ts\`.
9543
+ - Do not attach the originating thread again when it already routes to the current workspace.
9544
+ - Treat sending and attaching as one operation: check that both API calls succeed, and report an attachment failure instead of implying future replies will reach you.
9545
+
9546
+ #### Switch a thread to an existing workspace
9547
+
9548
+ When the user wants a Slack thread to continue in a particular existing workspace, resolve the target and switch the thread:
9549
+
9550
+ \`\`\`bash
9551
+ replicas list
9552
+ replicas slack thread switch <workspace-id-or-name> --channel <channel-id> --thread-ts <thread-root-ts>
9553
+ \`\`\`
9554
+
9555
+ The channel and thread flags default to \`SLACK_CHANNEL_ID\` and \`SLACK_THREAD_TS\` for a Slack-triggered workspace. Workspace-authenticated agents may target only their current workspace; switching to a different workspace requires running the command with user authentication outside agent mode. If that restriction applies, give the user the exact command to run rather than claiming the switch succeeded.
9556
+
9519
9557
  ### Searching Messages
9520
9558
 
9521
9559
  \`\`\`bash
@@ -9654,7 +9692,7 @@ var HOOK_EXEC_MAX_BUFFER_BYTES = 10 * 1024 * 1024;
9654
9692
  var REPLICAS_CONFIG_FILENAMES = ["replicas.json", "replicas.yaml", "replicas.yml"];
9655
9693
 
9656
9694
  // ../shared/src/cli-version.ts
9657
- var CLI_VERSION = "0.2.354";
9695
+ var CLI_VERSION = "0.2.356";
9658
9696
 
9659
9697
  // ../shared/src/version.ts
9660
9698
  function compareVersions(v1, v2) {
@@ -15010,7 +15048,7 @@ var sleep = (ms) => new Promise((resolve2) => setTimeout(resolve2, ms));
15010
15048
 
15011
15049
  // src/commands/computer/recording.ts
15012
15050
  import { spawn as spawn3 } from "child_process";
15013
- import { appendFileSync, existsSync as existsSync2, mkdirSync as mkdirSync2, readFileSync as readFileSync2, rmSync as rmSync2, writeFileSync as writeFileSync2 } from "fs";
15051
+ import { appendFileSync, existsSync as existsSync2, mkdirSync as mkdirSync2, readFileSync as readFileSync2, rmSync as rmSync2, statSync, writeFileSync as writeFileSync2 } from "fs";
15014
15052
  import { dirname as dirname2 } from "path";
15015
15053
 
15016
15054
  // src/commands/computer/recording/render.ts
@@ -15491,6 +15529,18 @@ var RECORD_STARTED_AT_FILE = `${STATE_DIR}/recording-started-at.txt`;
15491
15529
  var RECORD_FPS_FILE = `${STATE_DIR}/recording-fps.txt`;
15492
15530
  var RECORD_DIMENSIONS_FILE = `${STATE_DIR}/recording-dimensions.json`;
15493
15531
  var RECORD_ACTIONS_FILE = `${STATE_DIR}/recording-actions.jsonl`;
15532
+ var RECORD_STATE_FILES = [
15533
+ RECORD_PID_FILE,
15534
+ RECORD_PATH_FILE,
15535
+ RECORD_RAW_PATH_FILE,
15536
+ RECORD_STARTED_AT_FILE,
15537
+ RECORD_FPS_FILE,
15538
+ RECORD_DIMENSIONS_FILE,
15539
+ RECORD_ACTIONS_FILE
15540
+ ];
15541
+ function clearRecordingState() {
15542
+ for (const file of RECORD_STATE_FILES) rmSync2(file, { force: true });
15543
+ }
15494
15544
  function recordingStartedAt() {
15495
15545
  if (!existsSync2(RECORD_STARTED_AT_FILE)) return null;
15496
15546
  const startedAt = Number.parseInt(readFileSync2(RECORD_STARTED_AT_FILE, "utf8").trim(), 10);
@@ -15548,13 +15598,13 @@ async function computerRecordStartCommand(path6, options) {
15548
15598
  if (existsSync2(RECORD_PID_FILE)) {
15549
15599
  const pid = parseInt(readFileSync2(RECORD_PID_FILE, "utf8").trim(), 10);
15550
15600
  if (Number.isFinite(pid)) {
15551
- let alive = false;
15601
+ let alive2 = false;
15552
15602
  try {
15553
15603
  process.kill(pid, 0);
15554
- alive = true;
15604
+ alive2 = true;
15555
15605
  } catch {
15556
15606
  }
15557
- if (alive) fail(`recording already in progress (pid ${pid}). run \`replicas computer record stop\` first.`);
15607
+ if (alive2) fail(`recording already in progress (pid ${pid}). run \`replicas computer record stop\` first.`);
15558
15608
  }
15559
15609
  }
15560
15610
  const target = resolvePath(path6);
@@ -15601,7 +15651,29 @@ async function computerRecordStartCommand(path6, options) {
15601
15651
  writeFileSync2(RECORD_STARTED_AT_FILE, String(Date.now()));
15602
15652
  writeFileSync2(RECORD_FPS_FILE, String(fps));
15603
15653
  writeFileSync2(RECORD_DIMENSIONS_FILE, JSON.stringify({ width, height }));
15604
- console.log(target);
15654
+ const startedAt = Date.now();
15655
+ while (Date.now() - startedAt < 5e3) {
15656
+ try {
15657
+ process.kill(child.pid, 0);
15658
+ if (existsSync2(rawTarget) && statSync(rawTarget).size > 0) {
15659
+ console.log(`${target} (recording ready in ${Date.now() - startedAt}ms)`);
15660
+ return;
15661
+ }
15662
+ } catch {
15663
+ break;
15664
+ }
15665
+ await sleep(100);
15666
+ }
15667
+ let alive = false;
15668
+ try {
15669
+ process.kill(child.pid, 0);
15670
+ alive = true;
15671
+ } catch {
15672
+ }
15673
+ if (alive) fail("ffmpeg is running but did not produce recording output within 5 seconds; run `replicas computer record stop` to finalize or retry");
15674
+ clearRecordingState();
15675
+ rmSync2(rawTarget, { force: true });
15676
+ fail("ffmpeg exited before screen recording became ready");
15605
15677
  }
15606
15678
  async function computerRecordStopCommand() {
15607
15679
  if (!existsSync2(RECORD_PID_FILE) && !existsSync2(RECORD_PATH_FILE)) fail("no recording in progress");
@@ -15612,15 +15684,17 @@ async function computerRecordStopCommand() {
15612
15684
  process.kill(pid, "SIGINT");
15613
15685
  } catch {
15614
15686
  }
15615
- for (let i = 0; i < 30; i++) {
15687
+ let alive = true;
15688
+ for (let i = 0; i < 150; i++) {
15616
15689
  try {
15617
15690
  process.kill(pid, 0);
15618
15691
  } catch {
15692
+ alive = false;
15619
15693
  break;
15620
15694
  }
15621
- await new Promise((r) => setTimeout(r, 200));
15695
+ await sleep(200);
15622
15696
  }
15623
- rmSync2(RECORD_PID_FILE, { force: true });
15697
+ if (alive) fail(`ffmpeg did not finalize recording within 30 seconds (pid ${pid})`);
15624
15698
  }
15625
15699
  if (existsSync2(RECORD_PATH_FILE)) {
15626
15700
  const target = readFileSync2(RECORD_PATH_FILE, "utf8").trim();
@@ -15633,13 +15707,8 @@ async function computerRecordStopCommand() {
15633
15707
  rmSync2(rawPath, { force: true });
15634
15708
  }
15635
15709
  console.log(target);
15636
- rmSync2(RECORD_PATH_FILE, { force: true });
15637
- rmSync2(RECORD_RAW_PATH_FILE, { force: true });
15638
- rmSync2(RECORD_STARTED_AT_FILE, { force: true });
15639
- rmSync2(RECORD_FPS_FILE, { force: true });
15640
- rmSync2(RECORD_DIMENSIONS_FILE, { force: true });
15641
- rmSync2(RECORD_ACTIONS_FILE, { force: true });
15642
15710
  }
15711
+ clearRecordingState();
15643
15712
  }
15644
15713
 
15645
15714
  // src/commands/computer/index.ts
@@ -15978,8 +16047,8 @@ async function getChromePages() {
15978
16047
  async function selectChromePage(options = {}) {
15979
16048
  const pages = await getChromePages();
15980
16049
  let matches = pages;
15981
- if (options.id) {
15982
- matches = matches.filter((page2) => page2.id === options.id);
16050
+ if (options.targetId) {
16051
+ matches = matches.filter((page2) => page2.id === options.targetId);
15983
16052
  }
15984
16053
  if (options.title) {
15985
16054
  const needle = options.title.toLowerCase();
@@ -15998,121 +16067,528 @@ async function selectChromePage(options = {}) {
15998
16067
  `No matching debuggable Chrome page found (${matches.length} match${matches.length === 1 ? "" : "es"}, index ${index}). Use \`replicas computer browser\` to list pages.`
15999
16068
  );
16000
16069
  }
16070
+ await sendChromeCommand(webSocketDebuggerUrl, "Page.bringToFront", {});
16001
16071
  return { ...page, webSocketDebuggerUrl };
16002
16072
  }
16003
- function buildBrowserSnapshotExpression(textLimit, elementLimit) {
16004
- return `(() => {
16005
- const clip = (value, limit) => String(value || '').replace(/\\s+/g, ' ').trim().slice(0, limit);
16006
- const escape = (value) => globalThis.CSS && CSS.escape ? CSS.escape(value) : String(value).replace(/["\\\\]/g, '\\\\$&');
16073
+ var VISIBLE_JS = `
16007
16074
  const visible = (el) => {
16008
16075
  const rect = el.getBoundingClientRect();
16009
16076
  const style = getComputedStyle(el);
16010
16077
  return rect.width > 0 && rect.height > 0 && style.visibility !== 'hidden' && style.display !== 'none';
16011
- };
16012
- const labels = (el) => {
16013
- const values = [];
16014
- const id = el.getAttribute('id');
16015
- if (id) values.push(...Array.from(document.querySelectorAll('label[for="' + escape(id) + '"]')).map((label) => label.innerText));
16016
- const wrappingLabel = el.closest('label');
16017
- if (wrappingLabel) values.push(wrappingLabel.innerText);
16018
- values.push(el.innerText, el.value, el.getAttribute('aria-label'), el.getAttribute('title'), el.getAttribute('placeholder'), el.href, el.id, el.name, el.tagName);
16019
- return values.map((value) => clip(value, 160)).filter(Boolean);
16020
- };
16021
- const name = (el) => labels(el)[0] || clip(el.tagName, 160);
16022
- const selector = 'a,button,input,textarea,select,[role="button"],[role="link"],[role="textbox"],[contenteditable="true"]';
16023
- const controls = Array.from(document.querySelectorAll(selector))
16024
- .filter(visible)
16025
- .slice(0, ${elementLimit})
16026
- .map((el) => {
16027
- const rect = el.getBoundingClientRect();
16028
- return {
16029
- tag: el.tagName.toLowerCase(),
16030
- role: el.getAttribute('role') || null,
16031
- type: el.getAttribute('type') || null,
16032
- text: name(el),
16033
- href: el.href || null,
16034
- disabled: !!el.disabled || el.getAttribute('aria-disabled') === 'true',
16035
- rect: {
16036
- x: Math.round(rect.x),
16037
- y: Math.round(rect.y),
16038
- width: Math.round(rect.width),
16039
- height: Math.round(rect.height),
16040
- centerX: Math.round(rect.x + rect.width / 2),
16041
- centerY: Math.round(rect.y + rect.height / 2),
16042
- },
16043
- };
16044
- });
16045
- return {
16046
- title: document.title,
16047
- url: location.href,
16048
- text: clip(document.body ? document.body.innerText : '', ${textLimit}),
16049
- controls,
16050
- };
16051
- })()`;
16078
+ };`;
16079
+ async function sendChromeSessionCommand(session, method, params) {
16080
+ return chromeResult(await session.send({ method, params }), method);
16052
16081
  }
16053
- async function evaluateChromeTarget(webSocketDebuggerUrl, expression) {
16082
+ async function withChromeSession(webSocketDebuggerUrl, callback) {
16054
16083
  return await new Promise((resolve2, reject) => {
16055
16084
  const ws = new WebSocket(webSocketDebuggerUrl);
16085
+ const pending = /* @__PURE__ */ new Map();
16086
+ let nextId = 1;
16056
16087
  let settled = false;
16057
- const finish = (callback) => {
16088
+ const failSession = (error) => {
16058
16089
  if (settled) return;
16059
16090
  settled = true;
16060
- clearTimeout(timeout);
16091
+ for (const request of pending.values()) {
16092
+ clearTimeout(request.timeout);
16093
+ request.reject(error);
16094
+ }
16095
+ pending.clear();
16061
16096
  ws.close();
16062
- callback();
16097
+ reject(error);
16063
16098
  };
16064
- const timeout = setTimeout(() => {
16065
- finish(() => reject(new Error("Chrome DevTools evaluation timed out")));
16066
- }, 5e3);
16067
- ws.addEventListener("open", () => {
16068
- ws.send(JSON.stringify({
16069
- id: 1,
16070
- method: "Runtime.evaluate",
16071
- params: {
16072
- expression,
16073
- awaitPromise: true,
16074
- returnByValue: true
16075
- }
16076
- }));
16099
+ const finishSession = (value) => {
16100
+ if (settled) return;
16101
+ settled = true;
16102
+ for (const request of pending.values()) clearTimeout(request.timeout);
16103
+ pending.clear();
16104
+ ws.close();
16105
+ resolve2(value);
16106
+ };
16107
+ const session = {
16108
+ send: ({ method, params }) => new Promise((resolveCommand, rejectCommand) => {
16109
+ const id = nextId++;
16110
+ const timeout = setTimeout(() => {
16111
+ pending.delete(id);
16112
+ rejectCommand(new Error(`Chrome DevTools ${method} timed out`));
16113
+ }, 1e4);
16114
+ pending.set(id, { method, resolve: resolveCommand, reject: rejectCommand, timeout });
16115
+ ws.send(JSON.stringify({ id, method, params }));
16116
+ })
16117
+ };
16118
+ ws.addEventListener("open", async () => {
16119
+ try {
16120
+ finishSession(await callback(session));
16121
+ } catch (error) {
16122
+ failSession(error instanceof Error ? error : new Error(String(error)));
16123
+ }
16077
16124
  });
16078
16125
  ws.addEventListener("message", (event) => {
16079
16126
  const data = typeof event.data === "string" ? event.data : Buffer.isBuffer(event.data) ? event.data.toString("utf8") : Buffer.from(event.data).toString("utf8");
16080
- let parsed;
16127
+ let message;
16081
16128
  try {
16082
- parsed = JSON.parse(data);
16129
+ message = JSON.parse(data);
16083
16130
  } catch (error) {
16084
- const reason = error instanceof Error ? error.message : String(error);
16085
- finish(() => reject(new Error(`Chrome DevTools returned invalid WebSocket JSON: ${reason}`)));
16086
- return;
16087
- }
16088
- if (typeof parsed !== "object" || parsed === null) {
16089
- finish(() => reject(new Error("Chrome DevTools returned an invalid WebSocket message")));
16131
+ failSession(new Error(`Chrome DevTools returned invalid WebSocket JSON: ${error instanceof Error ? error.message : String(error)}`));
16090
16132
  return;
16091
16133
  }
16092
- const message = parsed;
16093
- if (message.id !== 1) return;
16094
- if (message.error) {
16095
- finish(() => reject(new Error(message.error?.message || "Chrome DevTools evaluation failed")));
16096
- return;
16097
- }
16098
- if (message.result?.exceptionDetails) {
16099
- finish(() => reject(new Error(message.result.exceptionDetails.text || "Chrome DevTools evaluation threw")));
16100
- return;
16101
- }
16102
- finish(() => resolve2(message.result?.result?.value ?? null));
16134
+ if (!isRecord(message) || typeof message.id !== "number") return;
16135
+ const request = pending.get(message.id);
16136
+ if (!request) return;
16137
+ clearTimeout(request.timeout);
16138
+ pending.delete(message.id);
16139
+ const detail = isRecord(message.error) && typeof message.error.message === "string" ? message.error.message : void 0;
16140
+ request.resolve(message.error ? { error: { message: detail } } : { result: isRecord(message.result) ? message.result : {} });
16103
16141
  });
16104
- ws.addEventListener("error", () => {
16105
- finish(() => reject(new Error("Chrome DevTools WebSocket failed")));
16142
+ ws.addEventListener("error", () => failSession(new Error("Chrome DevTools WebSocket failed")));
16143
+ ws.addEventListener("close", () => {
16144
+ if (!settled) failSession(new Error("Chrome DevTools WebSocket closed unexpectedly"));
16106
16145
  });
16107
16146
  });
16108
16147
  }
16148
+ async function sendChromeCommands(webSocketDebuggerUrl, commands) {
16149
+ return await withChromeSession(webSocketDebuggerUrl, async (session) => await Promise.all(commands.map((command) => session.send(command))));
16150
+ }
16151
+ async function sendChromeCommand(webSocketDebuggerUrl, method, params) {
16152
+ const response = (await sendChromeCommands(webSocketDebuggerUrl, [{ method, params }]))[0];
16153
+ if (response.error) throw new Error(response.error.message || `Chrome DevTools ${method} failed`);
16154
+ return response.result ?? {};
16155
+ }
16156
+ async function callFunctionOnChromeNodeInSession(session, backendNodeId, functionDeclaration, args = []) {
16157
+ const resolved = await sendChromeSessionCommand(session, "DOM.resolveNode", { backendNodeId });
16158
+ const objectId = isRecord(resolved.object) ? resolved.object.objectId : void 0;
16159
+ if (typeof objectId !== "string") throw new Error(`Could not resolve browser element ref ${backendNodeId}`);
16160
+ return await sendChromeSessionCommand(session, "Runtime.callFunctionOn", {
16161
+ objectId,
16162
+ functionDeclaration,
16163
+ arguments: args.map((value) => ({ value })),
16164
+ returnByValue: true
16165
+ });
16166
+ }
16167
+ function chromeCallValue(response) {
16168
+ return isRecord(response.result) ? response.result.value : void 0;
16169
+ }
16170
+ async function evaluateChromeTarget(webSocketDebuggerUrl, expression) {
16171
+ return await withChromeSession(webSocketDebuggerUrl, async (session) => await evaluateChromeSession(session, expression));
16172
+ }
16173
+ async function evaluateChromeSession(session, expression) {
16174
+ const response = await sendChromeSessionCommand(session, "Runtime.evaluate", {
16175
+ expression,
16176
+ awaitPromise: true,
16177
+ returnByValue: true
16178
+ });
16179
+ const exception = response.exceptionDetails;
16180
+ if (exception) {
16181
+ const text = typeof exception === "object" && exception !== null && "text" in exception && typeof exception.text === "string" ? exception.text : "";
16182
+ throw new Error(text || "Chrome DevTools evaluation threw");
16183
+ }
16184
+ const evaluated = response.result;
16185
+ return typeof evaluated === "object" && evaluated !== null && "value" in evaluated ? evaluated.value ?? null : null;
16186
+ }
16187
+ function isRawAXNode(value) {
16188
+ return isRecord(value);
16189
+ }
16190
+ function isDOMSnapshotDocument(value) {
16191
+ return isRecord(value);
16192
+ }
16193
+ function chromeVisualViewport(layout) {
16194
+ const viewport = isRecord(layout.cssVisualViewport) ? layout.cssVisualViewport : {};
16195
+ return {
16196
+ width: typeof viewport.clientWidth === "number" ? viewport.clientWidth : 0,
16197
+ height: typeof viewport.clientHeight === "number" ? viewport.clientHeight : 0
16198
+ };
16199
+ }
16200
+ function roundBrowserRect(rect) {
16201
+ return {
16202
+ x: Math.round(rect.x),
16203
+ y: Math.round(rect.y),
16204
+ width: Math.round(rect.width),
16205
+ height: Math.round(rect.height),
16206
+ centerX: Math.round(rect.centerX),
16207
+ centerY: Math.round(rect.centerY)
16208
+ };
16209
+ }
16210
+ function browserStateProperties(node) {
16211
+ const properties = [];
16212
+ for (const { name, value } of node.properties ?? []) {
16213
+ const state = value?.value;
16214
+ if (!name || !BROWSER_STATE_PROPERTIES.has(name) || typeof state !== "string" && typeof state !== "number" && typeof state !== "boolean") continue;
16215
+ properties.push([name, name === "url" ? clippedText(state, 300) : state]);
16216
+ }
16217
+ return Object.fromEntries(properties);
16218
+ }
16219
+ function readBrowserStateCache(path6) {
16220
+ let value;
16221
+ try {
16222
+ value = JSON.parse(readFileSync3(path6, "utf8"));
16223
+ } catch {
16224
+ return null;
16225
+ }
16226
+ if (!isRecord(value) || !Array.isArray(value.entries)) return null;
16227
+ const entries = [];
16228
+ for (const entry of value.entries) {
16229
+ if (!isRecord(entry) || typeof entry.key !== "string" || typeof entry.semantic !== "string" || typeof entry.line !== "string") return null;
16230
+ entries.push({ key: entry.key, semantic: entry.semantic, line: entry.line });
16231
+ }
16232
+ return {
16233
+ url: typeof value.url === "string" ? value.url : void 0,
16234
+ documentId: typeof value.documentId === "string" || value.documentId === null ? value.documentId : void 0,
16235
+ entries
16236
+ };
16237
+ }
16238
+ function chromeResult(response, method) {
16239
+ if (response.error) throw new Error(response.error.message || `Chrome DevTools ${method} failed`);
16240
+ return response.result ?? {};
16241
+ }
16242
+ function collectFrameIds(frameTree) {
16243
+ const ids = [];
16244
+ const visit = (value) => {
16245
+ if (!isRecord(value)) return;
16246
+ const frame = isRecord(value.frame) ? value.frame : {};
16247
+ if (typeof frame.id === "string") ids.push(frame.id);
16248
+ if (Array.isArray(value.childFrames)) value.childFrames.forEach(visit);
16249
+ };
16250
+ visit(frameTree);
16251
+ return ids;
16252
+ }
16253
+ function browserDocumentId(frameTree) {
16254
+ if (!isRecord(frameTree) || !isRecord(frameTree.frame)) return null;
16255
+ const loaderId = frameTree.frame.loaderId;
16256
+ return typeof loaderId === "string" ? loaderId : null;
16257
+ }
16258
+ function intersectRects(a, b) {
16259
+ const x = Math.max(a.x, b.x);
16260
+ const y = Math.max(a.y, b.y);
16261
+ const right = Math.min(a.x + a.width, b.x + b.width);
16262
+ const bottom = Math.min(a.y + a.height, b.y + b.height);
16263
+ if (right <= x || bottom <= y) return null;
16264
+ return {
16265
+ x,
16266
+ y,
16267
+ width: right - x,
16268
+ height: bottom - y,
16269
+ centerX: x + (right - x) / 2,
16270
+ centerY: y + (bottom - y) / 2
16271
+ };
16272
+ }
16273
+ function browserContentQuadRects(quads, viewportRect) {
16274
+ if (!Array.isArray(quads)) return [];
16275
+ return quads.filter((quad) => Array.isArray(quad) && quad.length >= 8 && quad.every(Number.isFinite)).map((quad) => {
16276
+ const xs = [quad[0], quad[2], quad[4], quad[6]];
16277
+ const ys = [quad[1], quad[3], quad[5], quad[7]];
16278
+ const x = Math.min(...xs);
16279
+ const y = Math.min(...ys);
16280
+ const width = Math.max(...xs) - x;
16281
+ const height = Math.max(...ys) - y;
16282
+ return { x, y, width, height, centerX: x + width / 2, centerY: y + height / 2 };
16283
+ }).map((rect) => intersectRects(rect, viewportRect)).filter((rect) => !!rect).sort((a, b) => b.width * b.height - a.width * a.height);
16284
+ }
16285
+ function buildBrowserBounds(result, viewport, targetId) {
16286
+ const strings = Array.isArray(result.strings) ? result.strings : [];
16287
+ const documents = Array.isArray(result.documents) ? result.documents.filter(isDOMSnapshotDocument) : [];
16288
+ const localRects = documents.map((document) => {
16289
+ const rects = /* @__PURE__ */ new Map();
16290
+ const nodeIndexes = document.layout?.nodeIndex ?? [];
16291
+ const bounds = document.layout?.bounds ?? [];
16292
+ const scrollX = document.scrollOffsetX ?? 0;
16293
+ const scrollY = document.scrollOffsetY ?? 0;
16294
+ nodeIndexes.forEach((nodeIndex, index) => {
16295
+ const bound = bounds[index];
16296
+ if (!Array.isArray(bound) || bound.length < 4 || !bound.every(Number.isFinite)) return;
16297
+ const [rawX, rawY, width, height] = bound;
16298
+ rects.set(nodeIndex, {
16299
+ x: rawX - scrollX,
16300
+ y: rawY - scrollY,
16301
+ width,
16302
+ height,
16303
+ centerX: rawX - scrollX + width / 2,
16304
+ centerY: rawY - scrollY + height / 2
16305
+ });
16306
+ });
16307
+ return rects;
16308
+ });
16309
+ const parents = /* @__PURE__ */ new Map();
16310
+ documents.forEach((document, documentIndex) => {
16311
+ const sparse = document.nodes?.contentDocumentIndex;
16312
+ sparse?.index?.forEach((nodeIndex, index) => {
16313
+ const childDocumentIndex = sparse.value?.[index];
16314
+ if (typeof childDocumentIndex === "number") parents.set(childDocumentIndex, { documentIndex, nodeIndex });
16315
+ });
16316
+ });
16317
+ const targetDocumentIndex = documents.findIndex(
16318
+ (document) => typeof document.frameId === "number" && strings[document.frameId] === targetId
16319
+ );
16320
+ const rootIndex = targetDocumentIndex >= 0 ? targetDocumentIndex : documents.findIndex((_, index) => !parents.has(index));
16321
+ const viewportRect = {
16322
+ x: 0,
16323
+ y: 0,
16324
+ width: viewport.width,
16325
+ height: viewport.height,
16326
+ centerX: viewport.width / 2,
16327
+ centerY: viewport.height / 2
16328
+ };
16329
+ const origins = /* @__PURE__ */ new Map();
16330
+ origins.set(rootIndex < 0 ? 0 : rootIndex, { x: 0, y: 0, clip: viewportRect });
16331
+ for (let pass = 0; pass < documents.length; pass++) {
16332
+ let changed = false;
16333
+ for (const [childIndex, parent] of parents) {
16334
+ if (origins.has(childIndex)) continue;
16335
+ const parentOrigin = origins.get(parent.documentIndex);
16336
+ const frameRect = localRects[parent.documentIndex]?.get(parent.nodeIndex);
16337
+ if (!parentOrigin || !frameRect) continue;
16338
+ const globalFrameRect = {
16339
+ ...frameRect,
16340
+ x: parentOrigin.x + frameRect.x,
16341
+ y: parentOrigin.y + frameRect.y,
16342
+ centerX: parentOrigin.x + frameRect.centerX,
16343
+ centerY: parentOrigin.y + frameRect.centerY
16344
+ };
16345
+ const childDocument = documents[childIndex];
16346
+ const insetX = Math.max(0, (globalFrameRect.width - (childDocument?.contentWidth ?? globalFrameRect.width)) / 2);
16347
+ const insetY = Math.max(0, (globalFrameRect.height - (childDocument?.contentHeight ?? globalFrameRect.height)) / 2);
16348
+ const contentRect = {
16349
+ x: globalFrameRect.x + insetX,
16350
+ y: globalFrameRect.y + insetY,
16351
+ width: Math.max(0, globalFrameRect.width - insetX * 2),
16352
+ height: Math.max(0, globalFrameRect.height - insetY * 2),
16353
+ centerX: globalFrameRect.centerX,
16354
+ centerY: globalFrameRect.centerY
16355
+ };
16356
+ const clip = intersectRects(parentOrigin.clip, contentRect);
16357
+ if (!clip) continue;
16358
+ origins.set(childIndex, { x: contentRect.x, y: contentRect.y, clip });
16359
+ changed = true;
16360
+ }
16361
+ if (!changed) break;
16362
+ }
16363
+ const byBackendNodeId = /* @__PURE__ */ new Map();
16364
+ documents.forEach((document, documentIndex) => {
16365
+ const origin = origins.get(documentIndex);
16366
+ if (!origin) return;
16367
+ const backendNodeIds = document.nodes?.backendNodeId ?? [];
16368
+ for (const [nodeIndex, local] of localRects[documentIndex] ?? []) {
16369
+ const backendNodeId = backendNodeIds[nodeIndex];
16370
+ if (!Number.isInteger(backendNodeId)) continue;
16371
+ const rect = {
16372
+ ...local,
16373
+ x: origin.x + local.x,
16374
+ y: origin.y + local.y,
16375
+ centerX: origin.x + local.centerX,
16376
+ centerY: origin.y + local.centerY
16377
+ };
16378
+ if (intersectRects(rect, origin.clip)) byBackendNodeId.set(backendNodeId, { rect, clip: origin.clip });
16379
+ }
16380
+ });
16381
+ return byBackendNodeId;
16382
+ }
16383
+ var BROWSER_INTERACTIVE_ROLES = /* @__PURE__ */ new Set([
16384
+ "button",
16385
+ "checkbox",
16386
+ "combobox",
16387
+ "link",
16388
+ "listbox",
16389
+ "menuitem",
16390
+ "menuitemcheckbox",
16391
+ "menuitemradio",
16392
+ "option",
16393
+ "radio",
16394
+ "scrollbar",
16395
+ "searchbox",
16396
+ "slider",
16397
+ "spinbutton",
16398
+ "switch",
16399
+ "tab",
16400
+ "textbox",
16401
+ "treeitem"
16402
+ ]);
16403
+ var BROWSER_STRUCTURAL_ROLES = /* @__PURE__ */ new Set([
16404
+ "alert",
16405
+ "cell",
16406
+ "columnheader",
16407
+ "dialog",
16408
+ "figure",
16409
+ "gridcell",
16410
+ "heading",
16411
+ "image",
16412
+ "listitem",
16413
+ "main",
16414
+ "paragraph",
16415
+ "region",
16416
+ "row",
16417
+ "rowheader",
16418
+ "status",
16419
+ "StaticText"
16420
+ ]);
16421
+ var BROWSER_STATE_PROPERTIES = /* @__PURE__ */ new Set([
16422
+ "autocomplete",
16423
+ "checked",
16424
+ "disabled",
16425
+ "editable",
16426
+ "expanded",
16427
+ "focusable",
16428
+ "focused",
16429
+ "haspopup",
16430
+ "invalid",
16431
+ "level",
16432
+ "multiselectable",
16433
+ "orientation",
16434
+ "pressed",
16435
+ "readonly",
16436
+ "protected",
16437
+ "required",
16438
+ "selected",
16439
+ "url"
16440
+ ]);
16441
+ function clippedText(value, limit = 240) {
16442
+ return String(value ?? "").replace(/\s+/g, " ").trim().slice(0, limit);
16443
+ }
16444
+ function browserTargetOutput(page) {
16445
+ return {
16446
+ targetId: page.id ?? null,
16447
+ title: (page.title ?? "").slice(0, 1e3),
16448
+ url: (page.url ?? "").slice(0, 2e3)
16449
+ };
16450
+ }
16451
+ function browserNodeActionable(role, states) {
16452
+ return BROWSER_INTERACTIVE_ROLES.has(role) || states.editable === true || states.focusable === true && role !== "RootWebArea";
16453
+ }
16454
+ async function captureBrowserSnapshot(page, options) {
16455
+ const initialCommands = [
16456
+ { method: "Page.getFrameTree", params: {} },
16457
+ { method: "Page.getLayoutMetrics", params: {} },
16458
+ { method: "DOMSnapshot.captureSnapshot", params: { computedStyles: [], includeDOMRects: true, includePaintOrder: true } }
16459
+ ];
16460
+ const initial = await sendChromeCommands(page.webSocketDebuggerUrl, initialCommands);
16461
+ const frameTree = chromeResult(initial[0], initialCommands[0].method).frameTree;
16462
+ const layout = chromeResult(initial[1], initialCommands[1].method);
16463
+ const domSnapshot = chromeResult(initial[2], initialCommands[2].method);
16464
+ const viewport = chromeVisualViewport(layout);
16465
+ const frameIds = collectFrameIds(frameTree);
16466
+ if (frameIds.length === 0 && page.id) frameIds.push(page.id);
16467
+ const axCommands = frameIds.map((frameId) => ({
16468
+ method: "Accessibility.getFullAXTree",
16469
+ params: { frameId }
16470
+ }));
16471
+ const axResults = await sendChromeCommands(page.webSocketDebuggerUrl, axCommands);
16472
+ const nodes = [];
16473
+ axResults.forEach((response, index) => {
16474
+ if (response.error) return;
16475
+ const responseNodes = response.result?.nodes;
16476
+ if (!Array.isArray(responseNodes)) return;
16477
+ for (const node of responseNodes) {
16478
+ if (!isRawAXNode(node)) continue;
16479
+ nodes.push({ ...node, sourceFrameId: frameIds[index] });
16480
+ }
16481
+ });
16482
+ const bounds = buildBrowserBounds(domSnapshot, viewport, page.id);
16483
+ const nodesByKey = new Map(nodes.map((node) => [`${node.sourceFrameId}:${node.nodeId}`, node]));
16484
+ const propertyMap = browserStateProperties;
16485
+ const roleOf = (node) => clippedText(node.role?.value, 80);
16486
+ const nameOf = (node) => clippedText(node.name?.value);
16487
+ const valueOf = (node) => {
16488
+ if (propertyMap(node).protected === true) return null;
16489
+ const value = node.value?.value;
16490
+ if (typeof value === "number" || typeof value === "boolean") return value;
16491
+ const text = clippedText(value);
16492
+ if (/^[•●*]+$/.test(text)) return "[protected]";
16493
+ return text || null;
16494
+ };
16495
+ const meaningful = nodes.filter((node) => !node.ignored && Number.isInteger(node.backendDOMNodeId)).map((node) => ({
16496
+ node,
16497
+ role: roleOf(node),
16498
+ bound: typeof node.backendDOMNodeId === "number" ? bounds.get(node.backendDOMNodeId) : void 0
16499
+ })).filter(({ node, role, bound }) => {
16500
+ if (!bound || bound.rect.width <= 0 || bound.rect.height <= 0) return false;
16501
+ if (role === "InlineTextBox" || role === "none" || role === "generic") return false;
16502
+ if (BROWSER_INTERACTIVE_ROLES.has(role) || BROWSER_STRUCTURAL_ROLES.has(role)) return true;
16503
+ return !!nameOf(node) || valueOf(node) !== null || propertyMap(node).focusable === true;
16504
+ }).filter(({ node, role }) => {
16505
+ if (role !== "StaticText") return true;
16506
+ const parent = node.parentId ? nodesByKey.get(`${node.sourceFrameId}:${node.parentId}`) : void 0;
16507
+ return !parent || nameOf(parent) !== nameOf(node);
16508
+ }).sort((a, b) => a.bound.rect.y - b.bound.rect.y || a.bound.rect.x - b.bound.rect.x);
16509
+ const elements = [];
16510
+ const entries = [];
16511
+ const meaningfulKeys = new Set(meaningful.map(({ node }) => `${node.sourceFrameId}:${node.nodeId}`));
16512
+ const visibleText = [];
16513
+ let visibleTextLength = 0;
16514
+ let treeLength = 0;
16515
+ const treeLimit = Math.min(5e4, Math.max(4e3, options.textLimit * 2));
16516
+ for (const { node, role, bound } of meaningful) {
16517
+ const name = nameOf(node);
16518
+ const description = clippedText(node.description?.value) || null;
16519
+ const value = valueOf(node);
16520
+ const states = propertyMap(node);
16521
+ const actionable = browserNodeActionable(role, states);
16522
+ const ref = actionable && elements.length < options.elementLimit ? String(node.backendDOMNodeId) : null;
16523
+ const visibleRect = role === "RootWebArea" && node.sourceFrameId === page.id ? { x: 0, y: 0, width: viewport.width, height: viewport.height, centerX: viewport.width / 2, centerY: viewport.height / 2 } : intersectRects(bound.rect, bound.clip) ?? bound.rect;
16524
+ const rect = roundBrowserRect(visibleRect);
16525
+ const actions = [
16526
+ ...actionable ? ["click"] : [],
16527
+ ...["textbox", "searchbox", "combobox", "spinbutton", "slider"].includes(role) || states.editable === true ? ["set_value"] : [],
16528
+ "scroll"
16529
+ ];
16530
+ if (ref) {
16531
+ elements.push({
16532
+ ref,
16533
+ role,
16534
+ name: name || null,
16535
+ description,
16536
+ value,
16537
+ states,
16538
+ actions,
16539
+ rect,
16540
+ frameId: node.sourceFrameId
16541
+ });
16542
+ }
16543
+ const stateText = Object.entries(states).filter(([key]) => key !== "url" && key !== "focusable").map(([key, state]) => `${key}=${JSON.stringify(state)}`);
16544
+ const url = typeof states.url === "string" ? clippedText(states.url, 300) : "";
16545
+ const semantic = [role, name && JSON.stringify(name), value !== null && `value=${JSON.stringify(value)}`, description && `description=${JSON.stringify(description)}`, url && `url=${JSON.stringify(url)}`, ...stateText].filter(Boolean).join(" ");
16546
+ let depth = 0;
16547
+ let parentId = node.parentId;
16548
+ const visited = /* @__PURE__ */ new Set();
16549
+ while (parentId && depth < 8) {
16550
+ const parentKey = `${node.sourceFrameId}:${parentId}`;
16551
+ if (visited.has(parentKey)) break;
16552
+ visited.add(parentKey);
16553
+ if (meaningfulKeys.has(parentKey)) depth++;
16554
+ parentId = nodesByKey.get(parentKey)?.parentId;
16555
+ }
16556
+ const line = `${" ".repeat(depth)}${ref ? `[ref=${ref}]` : "-"} ${semantic} (${rect.x},${rect.y} ${rect.width}x${rect.height})`;
16557
+ if (treeLength + line.length + 1 <= treeLimit) {
16558
+ const key = Number.isInteger(node.backendDOMNodeId) ? `dom:${node.sourceFrameId}:${node.backendDOMNodeId}` : `ax:${node.sourceFrameId}:${node.nodeId}`;
16559
+ entries.push({ key, semantic, line });
16560
+ treeLength += line.length + 1;
16561
+ }
16562
+ if (name && visibleTextLength < options.textLimit && (role === "StaticText" || role === "heading" || BROWSER_INTERACTIVE_ROLES.has(role))) {
16563
+ const remaining = options.textLimit - visibleTextLength;
16564
+ visibleText.push(name.slice(0, remaining));
16565
+ visibleTextLength += Math.min(name.length, remaining) + 1;
16566
+ }
16567
+ }
16568
+ const revision = createHash2("sha256").update(JSON.stringify([page.url, entries.map(({ key, semantic }) => [key, semantic])])).digest("hex").slice(0, 16);
16569
+ const snapshot = {
16570
+ title: (page.title ?? "").slice(0, 1e3),
16571
+ url: (page.url ?? "").slice(0, 2e3),
16572
+ documentId: browserDocumentId(frameTree),
16573
+ revision,
16574
+ viewport,
16575
+ text: visibleText.join(" ").slice(0, options.textLimit),
16576
+ tree: entries.map(({ line }) => line).join("\n"),
16577
+ elements,
16578
+ controls: elements,
16579
+ nodeCount: entries.length,
16580
+ elementCount: elements.length,
16581
+ truncated: entries.length < meaningful.length || elements.length < meaningful.filter(({ node, role }) => browserNodeActionable(role, propertyMap(node))).length
16582
+ };
16583
+ return { snapshot, entries };
16584
+ }
16109
16585
  async function computerBrowserCommand(options = {}) {
16110
16586
  const textLimit = options.limit ? parseCoord(options.limit, "--limit") : 4e3;
16111
16587
  const elementLimit = options.elementLimit ? parseCoord(options.elementLimit, "--element-limit") : 80;
16112
16588
  if (textLimit < 0 || textLimit > 5e4) fail("--limit must be between 0 and 50000");
16113
16589
  if (elementLimit < 0 || elementLimit > 500) fail("--element-limit must be between 0 and 500");
16114
- const expression = buildBrowserSnapshotExpression(textLimit, elementLimit);
16115
- const pages = (await getChromePages()).map((target) => ({
16590
+ const targets = options.targetId || options.page || options.title || options.url ? [await selectChromePage(options)] : await getChromePages();
16591
+ const pages = targets.map((target) => ({
16116
16592
  target,
16117
16593
  page: {
16118
16594
  id: target.id ?? null,
@@ -16125,7 +16601,11 @@ async function computerBrowserCommand(options = {}) {
16125
16601
  for (const { target, page } of pages) {
16126
16602
  if (options.snapshot && target.webSocketDebuggerUrl) {
16127
16603
  try {
16128
- pageResults.push({ ...page, snapshot: await evaluateChromeTarget(target.webSocketDebuggerUrl, expression) });
16604
+ const { snapshot } = await captureBrowserSnapshot(
16605
+ { ...target, webSocketDebuggerUrl: target.webSocketDebuggerUrl },
16606
+ { textLimit, elementLimit }
16607
+ );
16608
+ pageResults.push({ ...page, snapshot });
16129
16609
  } catch (error) {
16130
16610
  pageResults.push({ ...page, snapshotError: error instanceof Error ? error.message : "snapshot failed" });
16131
16611
  }
@@ -16139,6 +16619,188 @@ async function computerBrowserCommand(options = {}) {
16139
16619
  pages: pageResults
16140
16620
  }, null, 2));
16141
16621
  }
16622
+ function buildBrowserStabilityExpression() {
16623
+ return `(() => {
16624
+ const key = '__replicasComputerUseObserver';
16625
+ if (!globalThis[key]) {
16626
+ const state = { revision: 0 };
16627
+ const observer = new MutationObserver(() => state.revision++);
16628
+ globalThis[key] = { state, observer, observing: false };
16629
+ }
16630
+ if (document.documentElement && !globalThis[key].observing) {
16631
+ globalThis[key].observer.observe(document.documentElement, {
16632
+ subtree: true,
16633
+ childList: true,
16634
+ attributes: true,
16635
+ characterData: true,
16636
+ });
16637
+ globalThis[key].observing = true;
16638
+ }
16639
+ const controls = Array.from(document.querySelectorAll('input,textarea,select,[contenteditable="true"],[role="checkbox"],[role="combobox"],[role="slider"],[role="switch"],[role="textbox"]')).slice(0, 200);
16640
+ const active = document.activeElement;
16641
+ const controlState = controls.map((element, index) => {
16642
+ const value = 'value' in element
16643
+ ? element instanceof HTMLInputElement && element.type === 'password'
16644
+ ? String(element.value).length
16645
+ : String(element.value)
16646
+ : element.textContent;
16647
+ return [
16648
+ index,
16649
+ element === active,
16650
+ value,
16651
+ 'checked' in element ? element.checked : null,
16652
+ 'selectedIndex' in element ? element.selectedIndex : null,
16653
+ 'disabled' in element ? element.disabled : null,
16654
+ element.getAttribute('aria-checked'),
16655
+ element.getAttribute('aria-expanded'),
16656
+ element.getAttribute('aria-selected'),
16657
+ ].join('|');
16658
+ }).join('\\u001f');
16659
+ let controlRevision = 2166136261;
16660
+ for (let index = 0; index < controlState.length; index++) {
16661
+ controlRevision ^= controlState.charCodeAt(index);
16662
+ controlRevision = Math.imul(controlRevision, 16777619);
16663
+ }
16664
+ return {
16665
+ revision: globalThis[key].state.revision,
16666
+ controlRevision: (controlRevision >>> 0).toString(16),
16667
+ readyState: document.readyState,
16668
+ busy: !!document.querySelector('[aria-busy="true"]'),
16669
+ url: location.href.slice(0, 2000),
16670
+ scrollX: Math.round(scrollX),
16671
+ scrollY: Math.round(scrollY),
16672
+ innerWidth,
16673
+ innerHeight,
16674
+ devicePixelRatio,
16675
+ };
16676
+ })()`;
16677
+ }
16678
+ async function waitForBrowserStability(page, options) {
16679
+ const startedAt = Date.now();
16680
+ let lastSignature = null;
16681
+ let lastChangeAt = startedAt;
16682
+ let samples = 0;
16683
+ let changes = 0;
16684
+ let state = null;
16685
+ while (Date.now() - startedAt <= options.timeoutMs) {
16686
+ try {
16687
+ state = await evaluateChromeTarget(page.webSocketDebuggerUrl, buildBrowserStabilityExpression());
16688
+ samples++;
16689
+ const signature = JSON.stringify(state);
16690
+ const now = Date.now();
16691
+ if (lastSignature === null || signature !== lastSignature) {
16692
+ if (lastSignature !== null) changes++;
16693
+ lastSignature = signature;
16694
+ lastChangeAt = now;
16695
+ }
16696
+ const ready = typeof state === "object" && state !== null && "readyState" in state && state.readyState !== "loading" && "busy" in state && state.busy !== true;
16697
+ if (samples > 1 && ready && now - lastChangeAt >= options.stableMs) {
16698
+ return { stable: true, elapsedMs: now - startedAt, samples, changes, state };
16699
+ }
16700
+ } catch {
16701
+ }
16702
+ await sleep(options.pollMs);
16703
+ }
16704
+ return { stable: false, elapsedMs: Date.now() - startedAt, samples, changes, state };
16705
+ }
16706
+ function browserStateCachePath(targetId) {
16707
+ return `${STATE_DIR}/browser-state-${targetId.replace(/[^a-zA-Z0-9_-]/g, "_")}.json`;
16708
+ }
16709
+ function browserStateDiff(previous, current) {
16710
+ const before = new Map(previous.map((entry) => [entry.key, entry]));
16711
+ const after = new Map(current.map((entry) => [entry.key, entry]));
16712
+ const lines = [];
16713
+ let added = 0;
16714
+ let changed = 0;
16715
+ let removed = 0;
16716
+ for (const entry of current) {
16717
+ const old = before.get(entry.key);
16718
+ if (!old) {
16719
+ added++;
16720
+ lines.push(`+ ${entry.line}`);
16721
+ } else if (old.semantic !== entry.semantic) {
16722
+ changed++;
16723
+ lines.push(`~ ${old.line}
16724
+ ${entry.line}`);
16725
+ }
16726
+ }
16727
+ for (const entry of previous) {
16728
+ if (after.has(entry.key)) continue;
16729
+ removed++;
16730
+ lines.push(`- ${entry.line}`);
16731
+ }
16732
+ return { tree: lines.join("\n") || "No accessibility changes.", added, changed, removed };
16733
+ }
16734
+ async function computerBrowserStateCommand(path6, options = {}) {
16735
+ const textLimit = options.limit ? parseCoord(options.limit, "--limit") : 8e3;
16736
+ const elementLimit = options.elementLimit ? parseCoord(options.elementLimit, "--element-limit") : 120;
16737
+ const timeoutMs = options.timeout ? parseCoord(options.timeout, "--timeout") : 5e3;
16738
+ const stableMs = options.stableMs ? parseCoord(options.stableMs, "--stable-ms") : 500;
16739
+ const pollMs = options.pollMs ? parseCoord(options.pollMs, "--poll-ms") : 100;
16740
+ if (textLimit < 0 || textLimit > 5e4) fail("--limit must be between 0 and 50000");
16741
+ if (elementLimit < 0 || elementLimit > 500) fail("--element-limit must be between 0 and 500");
16742
+ if (timeoutMs < 0) fail("--timeout must be >= 0");
16743
+ if (stableMs < 0) fail("--stable-ms must be >= 0");
16744
+ if (pollMs < 50 || pollMs > 2e3) fail("--poll-ms must be between 50 and 2000");
16745
+ const page = await selectChromePage(options);
16746
+ const stability = await waitForBrowserStability(page, { timeoutMs, stableMs, pollMs });
16747
+ const target = resolvePath(path6);
16748
+ mkdirSync3(dirname3(target), { recursive: true });
16749
+ const [{ snapshot, entries }, screenshotResult] = await Promise.all([
16750
+ captureBrowserSnapshot(page, { textLimit, elementLimit }),
16751
+ sendChromeCommand(page.webSocketDebuggerUrl, "Page.captureScreenshot", {
16752
+ format: "png",
16753
+ fromSurface: true,
16754
+ captureBeyondViewport: false
16755
+ })
16756
+ ]);
16757
+ const data = screenshotResult.data;
16758
+ if (typeof data !== "string") fail("Chrome did not return screenshot data");
16759
+ writeFileSync3(target, Buffer.from(data, "base64"));
16760
+ const screenshot = readPngDimensions(target);
16761
+ const targetId = page.id ?? "unknown";
16762
+ const cachePath = browserStateCachePath(targetId);
16763
+ let mode = "full";
16764
+ let tree = snapshot.tree;
16765
+ let changes = { added: snapshot.nodeCount, changed: 0, removed: 0 };
16766
+ if (!options.full && existsSync3(cachePath)) {
16767
+ const cached = readBrowserStateCache(cachePath);
16768
+ if (cached?.url === snapshot.url && cached.documentId === snapshot.documentId) {
16769
+ mode = "diff";
16770
+ const diff = browserStateDiff(cached.entries, entries);
16771
+ tree = diff.tree;
16772
+ changes = { added: diff.added, changed: diff.changed, removed: diff.removed };
16773
+ }
16774
+ }
16775
+ mkdirSync3(STATE_DIR, { recursive: true });
16776
+ writeFileSync3(cachePath, JSON.stringify({ url: snapshot.url, documentId: snapshot.documentId, entries }));
16777
+ const stableState = isRecord(stability.state) ? stability.state : {};
16778
+ const state = {
16779
+ title: snapshot.title,
16780
+ url: snapshot.url,
16781
+ documentId: snapshot.documentId,
16782
+ revision: snapshot.revision,
16783
+ viewport: snapshot.viewport,
16784
+ text: snapshot.text,
16785
+ tree,
16786
+ nodeCount: snapshot.nodeCount,
16787
+ elementCount: snapshot.elementCount,
16788
+ truncated: snapshot.truncated,
16789
+ mode,
16790
+ changes
16791
+ };
16792
+ console.log(JSON.stringify({
16793
+ targetId,
16794
+ screenshot: {
16795
+ path: target,
16796
+ width: screenshot.width,
16797
+ height: screenshot.height,
16798
+ devicePixelRatio: typeof stableState.devicePixelRatio === "number" ? stableState.devicePixelRatio : 1
16799
+ },
16800
+ stability,
16801
+ state
16802
+ }, null, 2));
16803
+ }
16142
16804
  var DESKTOP_POINT_JS = `
16143
16805
  const desktopPoint = (rect) => {
16144
16806
  const borderX = Math.max(0, (window.outerWidth - window.innerWidth) / 2);
@@ -16148,17 +16810,95 @@ var DESKTOP_POINT_JS = `
16148
16810
  y: Math.round(window.screenY + topChrome + rect.y + rect.height / 2),
16149
16811
  };
16150
16812
  };`;
16813
+ function parseBrowserRef(value) {
16814
+ const ref = parseCoord(value.replace(/^ref=/, ""), "--ref");
16815
+ if (ref <= 0) fail("--ref must be a positive backend DOM node ID from the latest browser state");
16816
+ return ref;
16817
+ }
16818
+ function readBrowserRefCache(page) {
16819
+ const targetId = page.id;
16820
+ if (!targetId) fail("Chrome target has no stable ID. Capture fresh browser state and retry.");
16821
+ const cachePath = browserStateCachePath(targetId);
16822
+ if (!existsSync3(cachePath)) fail(`No browser state is cached for target ${targetId}. Run replicas computer browser-state <path> --target-id ${targetId} first.`);
16823
+ const cached = readBrowserStateCache(cachePath);
16824
+ if (!cached) fail(`Browser state for target ${targetId} is invalid. Capture fresh browser state and retry.`);
16825
+ if (cached.url !== (page.url ?? "").slice(0, 2e3)) fail("Cached browser state belongs to a different URL. Capture fresh browser state and retry.");
16826
+ return { targetId, cached };
16827
+ }
16828
+ function browserViewportRect(layout) {
16829
+ const viewport = chromeVisualViewport(layout);
16830
+ return {
16831
+ x: 0,
16832
+ y: 0,
16833
+ width: viewport.width,
16834
+ height: viewport.height,
16835
+ centerX: viewport.width / 2,
16836
+ centerY: viewport.height / 2
16837
+ };
16838
+ }
16839
+ async function prepareBrowserElement(page, refValue) {
16840
+ const backendNodeId = parseBrowserRef(refValue);
16841
+ const { cached } = readBrowserRefCache(page);
16842
+ return await withChromeSession(page.webSocketDebuggerUrl, async (session) => {
16843
+ const layout = await sendChromeSessionCommand(session, "Page.getLayoutMetrics", {});
16844
+ const rect = await prepareBrowserRefInSession(session, backendNodeId, cached, browserViewportRect(layout));
16845
+ const axNodes = (await sendChromeSessionCommand(session, "Accessibility.getPartialAXTree", {
16846
+ backendNodeId,
16847
+ fetchRelatives: false
16848
+ })).nodes;
16849
+ const rawNode = Array.isArray(axNodes) && isRawAXNode(axNodes[0]) ? axNodes[0] : null;
16850
+ const states = rawNode ? browserStateProperties(rawNode) : {};
16851
+ const role = clippedText(rawNode?.role?.value, 80);
16852
+ const element = rawNode ? {
16853
+ ref: String(backendNodeId),
16854
+ role,
16855
+ name: clippedText(rawNode.name?.value) || null,
16856
+ description: clippedText(rawNode.description?.value) || null,
16857
+ value: typeof rawNode.value?.value === "string" ? clippedText(rawNode.value.value) : typeof rawNode.value?.value === "number" || typeof rawNode.value?.value === "boolean" ? rawNode.value.value : null,
16858
+ states,
16859
+ actions: [
16860
+ "click",
16861
+ ...["textbox", "searchbox", "combobox", "spinbutton", "slider"].includes(role) || states.editable === true ? ["set_value"] : [],
16862
+ "scroll"
16863
+ ]
16864
+ } : null;
16865
+ const desktopPoint = await evaluateChromeSession(session, `(() => {
16866
+ ${DESKTOP_POINT_JS}
16867
+ return desktopPoint({ x: ${rect.centerX}, y: ${rect.centerY}, width: 0, height: 0 });
16868
+ })()`);
16869
+ return {
16870
+ ref: String(backendNodeId),
16871
+ rect,
16872
+ element,
16873
+ desktopPoint: typeof desktopPoint === "object" && desktopPoint !== null && "x" in desktopPoint && typeof desktopPoint.x === "number" && "y" in desktopPoint && typeof desktopPoint.y === "number" ? { x: desktopPoint.x, y: desktopPoint.y } : null
16874
+ };
16875
+ });
16876
+ }
16877
+ async function prepareBrowserRefInSession(session, backendNodeId, cached, viewportRect) {
16878
+ if (!cached.entries.some(({ line }) => line.trimStart().startsWith(`[ref=${backendNodeId}]`))) {
16879
+ fail(`Browser element ref ${backendNodeId} is not in the latest state. Capture fresh browser state and retry.`);
16880
+ }
16881
+ const frameTree = (await sendChromeSessionCommand(session, "Page.getFrameTree", {})).frameTree;
16882
+ if (cached.documentId !== browserDocumentId(frameTree)) {
16883
+ fail(`Browser element ref ${backendNodeId} belongs to a previous document. Capture fresh browser state and retry.`);
16884
+ }
16885
+ try {
16886
+ await sendChromeSessionCommand(session, "DOM.scrollIntoViewIfNeeded", { backendNodeId });
16887
+ } catch (error) {
16888
+ fail(`Browser element ref ${backendNodeId} is stale or cannot be scrolled into view: ${error instanceof Error ? error.message : String(error)}`);
16889
+ }
16890
+ const quads = (await sendChromeSessionCommand(session, "DOM.getContentQuads", { backendNodeId })).quads;
16891
+ const candidates = browserContentQuadRects(quads, viewportRect);
16892
+ if (!candidates[0]) fail(`Browser element ref ${backendNodeId} has no visible content quad. Fetch fresh browser state and retry.`);
16893
+ return roundBrowserRect(candidates[0]);
16894
+ }
16151
16895
  function buildBrowserClickExpression(query, options) {
16152
16896
  return `(() => {
16153
16897
  const query = ${JSON.stringify(query)};
16154
16898
  const exact = ${JSON.stringify(options.exact)};
16155
16899
  const index = ${options.index};
16156
16900
  const normalize = (value) => String(value || '').replace(/\\s+/g, ' ').trim();
16157
- const visible = (el) => {
16158
- const rect = el.getBoundingClientRect();
16159
- const style = getComputedStyle(el);
16160
- return rect.width > 0 && rect.height > 0 && style.visibility !== 'hidden' && style.display !== 'none';
16161
- };
16901
+ ${VISIBLE_JS}
16162
16902
  const label = (el) => normalize(el.innerText || el.value || el.getAttribute('aria-label') || el.getAttribute('title') || el.getAttribute('placeholder') || el.href || el.id || el.name || el.tagName);
16163
16903
  ${DESKTOP_POINT_JS}
16164
16904
  const matches = (text) => {
@@ -16183,8 +16923,7 @@ ${DESKTOP_POINT_JS}
16183
16923
  const { el, text } = candidate;
16184
16924
  el.scrollIntoView({ block: 'center', inline: 'center' });
16185
16925
  const rect = el.getBoundingClientRect();
16186
- if (typeof el.focus === 'function') el.focus();
16187
- el.click();
16926
+ if (typeof el.focus === 'function') el.focus({ preventScroll: true });
16188
16927
  return {
16189
16928
  clicked: true,
16190
16929
  query,
@@ -16207,6 +16946,15 @@ ${DESKTOP_POINT_JS}
16207
16946
  };
16208
16947
  })()`;
16209
16948
  }
16949
+ function browserActionRectCenter(result) {
16950
+ if (typeof result !== "object" || result === null || !("rect" in result)) return null;
16951
+ const rect = result.rect;
16952
+ if (typeof rect !== "object" || rect === null) return null;
16953
+ const x = "centerX" in rect ? rect.centerX : void 0;
16954
+ const y = "centerY" in rect ? rect.centerY : void 0;
16955
+ if (typeof x !== "number" || typeof y !== "number" || !Number.isFinite(x) || !Number.isFinite(y)) return null;
16956
+ return { x, y };
16957
+ }
16210
16958
  function browserActionPoint(result) {
16211
16959
  if (typeof result !== "object" || result === null) return null;
16212
16960
  const action = "result" in result ? result.result : result;
@@ -16222,19 +16970,149 @@ function browserActionPoint(result) {
16222
16970
  y: clamp(Math.round(y), 0, dimensions.height)
16223
16971
  };
16224
16972
  }
16973
+ async function dispatchChromeClick(page, x, y, button, clicks) {
16974
+ await withChromeSession(page.webSocketDebuggerUrl, async (session) => {
16975
+ await dispatchChromeClickInSession(session, x, y, button, clicks);
16976
+ });
16977
+ }
16978
+ async function dispatchChromeClickInSession(session, x, y, button, clicks) {
16979
+ for (let clickCount = 1; clickCount <= clicks; clickCount++) {
16980
+ await sendChromeSessionCommand(session, "Input.dispatchMouseEvent", {
16981
+ type: "mousePressed",
16982
+ x,
16983
+ y,
16984
+ button,
16985
+ clickCount
16986
+ });
16987
+ await sendChromeSessionCommand(session, "Input.dispatchMouseEvent", {
16988
+ type: "mouseReleased",
16989
+ x,
16990
+ y,
16991
+ button,
16992
+ clickCount
16993
+ });
16994
+ }
16995
+ }
16996
+ async function replaceChromeText(page, text) {
16997
+ await withChromeSession(page.webSocketDebuggerUrl, async (session) => {
16998
+ await replaceChromeTextInSession(session, text);
16999
+ });
17000
+ }
17001
+ async function replaceChromeTextInSession(session, text) {
17002
+ await sendChromeSessionCommand(session, "Input.dispatchKeyEvent", {
17003
+ type: "rawKeyDown",
17004
+ key: "a",
17005
+ code: "KeyA",
17006
+ windowsVirtualKeyCode: 65,
17007
+ nativeVirtualKeyCode: 65,
17008
+ modifiers: 2
17009
+ });
17010
+ await sendChromeSessionCommand(session, "Input.dispatchKeyEvent", {
17011
+ type: "keyUp",
17012
+ key: "a",
17013
+ code: "KeyA",
17014
+ windowsVirtualKeyCode: 65,
17015
+ nativeVirtualKeyCode: 65,
17016
+ modifiers: 2
17017
+ });
17018
+ await sendChromeSessionCommand(session, "Input.insertText", { text });
17019
+ }
17020
+ var BROWSER_DIRECT_VALUE_INPUT_TYPES = /* @__PURE__ */ new Set(["number", "range", "date", "datetime-local", "month", "week", "time", "color"]);
17021
+ function browserFillInvalid(actual, expected, fieldType, semanticSelection = false, selectionMatched = false) {
17022
+ if (semanticSelection) return !selectionMatched;
17023
+ if (["number", "range"].includes(fieldType)) {
17024
+ const actualNumber = typeof actual === "number" ? actual : Number(actual);
17025
+ const expectedNumber = Number(expected);
17026
+ return !Number.isFinite(actualNumber) || !Number.isFinite(expectedNumber) || actualNumber !== expectedNumber;
17027
+ }
17028
+ if (fieldType === "color") return String(actual).toLowerCase() !== expected.toLowerCase();
17029
+ return actual !== expected;
17030
+ }
17031
+ async function fillChromeNodeInSession(session, backendNodeId, value) {
17032
+ await sendChromeSessionCommand(session, "DOM.focus", { backendNodeId });
17033
+ const directResult = await callFunctionOnChromeNodeInSession(
17034
+ session,
17035
+ backendNodeId,
17036
+ `function(value) {
17037
+ const emit = () => {
17038
+ this.dispatchEvent(new Event('input', { bubbles: true }));
17039
+ this.dispatchEvent(new Event('change', { bubbles: true }));
17040
+ };
17041
+ if (this instanceof HTMLSelectElement) {
17042
+ const option = Array.from(this.options).find((item) => item.value === value || item.text.trim() === value);
17043
+ if (option) this.value = option.value;
17044
+ else this.value = value;
17045
+ emit();
17046
+ return { handled: true, semanticSelection: true, matched: !!option, value: this.value };
17047
+ }
17048
+ if (this instanceof HTMLInputElement && ${JSON.stringify([...BROWSER_DIRECT_VALUE_INPUT_TYPES])}.includes(this.type.toLowerCase())) {
17049
+ const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value')?.set;
17050
+ if (!setter) return { handled: false };
17051
+ setter.call(this, value);
17052
+ emit();
17053
+ return { handled: true, semanticSelection: false, matched: true, value: this.value };
17054
+ }
17055
+ return { handled: false };
17056
+ }`,
17057
+ [value]
17058
+ );
17059
+ const directValue = chromeCallValue(directResult);
17060
+ const directHandled = isRecord(directValue) && directValue.handled === true;
17061
+ if (!directHandled) await replaceChromeTextInSession(session, value);
17062
+ const verification = await callFunctionOnChromeNodeInSession(
17063
+ session,
17064
+ backendNodeId,
17065
+ `function() {
17066
+ const value = this instanceof HTMLSelectElement || 'value' in this ? this.value : this.textContent;
17067
+ const type = this instanceof HTMLSelectElement
17068
+ ? 'select'
17069
+ : this instanceof HTMLInputElement
17070
+ ? this.type.toLowerCase()
17071
+ : this.isContentEditable
17072
+ ? 'contenteditable'
17073
+ : 'text';
17074
+ return { value, valueLength: String(value ?? '').length, type };
17075
+ }`
17076
+ );
17077
+ const actual = chromeCallValue(verification);
17078
+ if (!isRecord(actual)) fail(`Chrome did not return a value for browser element ref ${backendNodeId}.`);
17079
+ const fieldType = typeof actual.type === "string" ? actual.type : "";
17080
+ const semanticSelection = isRecord(directValue) && directValue.semanticSelection === true;
17081
+ const selectionMatched = isRecord(directValue) && directValue.matched === true;
17082
+ if (browserFillInvalid(actual.value, value, fieldType, semanticSelection, selectionMatched)) {
17083
+ fail(`Chrome reported that ref ${backendNodeId} contains ${JSON.stringify(actual.value)} after filling, expected ${JSON.stringify(value)}.`);
17084
+ }
17085
+ return { value: actual.value, valueLength: actual.valueLength, type: fieldType };
17086
+ }
16225
17087
  async function computerBrowserClickCommand(query, options = {}) {
16226
17088
  const page = await selectChromePage(options);
17089
+ if (options.ref) {
17090
+ const target = await prepareBrowserElement(page, options.ref);
17091
+ const buttonNames = { "1": "left", "2": "middle", "3": "right", left: "left", middle: "middle", right: "right" };
17092
+ const button = buttonNames[(options.button ?? "left").toLowerCase()];
17093
+ if (!button) fail("--button must be one of left|middle|right|1|2|3");
17094
+ const clicks = options.double ? 2 : 1;
17095
+ await dispatchChromeClick(page, target.rect.centerX, target.rect.centerY, button, clicks);
17096
+ if (target.desktopPoint) logRecordingAction({ type: "click", ...target.desktopPoint });
17097
+ console.log(JSON.stringify({
17098
+ ...browserTargetOutput(page),
17099
+ result: { clicked: true, ref: target.ref, button, clickCount: clicks, rect: target.rect, element: target.element }
17100
+ }, null, 2));
17101
+ return;
17102
+ }
17103
+ if (!query) fail("Provide control text or --ref <id> from the latest browser state");
16227
17104
  const index = options.index ? parseCoord(options.index, "--index") : 0;
16228
17105
  if (index < 0) fail("--index must be >= 0");
16229
17106
  const result = await evaluateChromeTarget(
16230
17107
  page.webSocketDebuggerUrl,
16231
17108
  buildBrowserClickExpression(query, { exact: !!options.exact, index })
16232
17109
  );
17110
+ const center = browserActionRectCenter(result);
17111
+ if (center) await dispatchChromeClick(page, center.x, center.y, "left", 1);
16233
17112
  const point = browserActionPoint(result);
16234
17113
  if (point) logRecordingAction({ type: "click", ...point });
16235
17114
  console.log(JSON.stringify({
16236
- title: page.title ?? "",
16237
- url: page.url ?? "",
17115
+ ...browserTargetOutput(page),
16238
17116
  result
16239
17117
  }, null, 2));
16240
17118
  }
@@ -16246,11 +17124,7 @@ function buildBrowserFillExpression(query, value, options) {
16246
17124
  const index = ${options.index};
16247
17125
  const normalize = (text) => String(text || '').replace(/\\s+/g, ' ').trim();
16248
17126
  const escape = (text) => globalThis.CSS && CSS.escape ? CSS.escape(text) : String(text).replace(/["\\\\]/g, '\\\\$&');
16249
- const visible = (el) => {
16250
- const rect = el.getBoundingClientRect();
16251
- const style = getComputedStyle(el);
16252
- return rect.width > 0 && rect.height > 0 && style.visibility !== 'hidden' && style.display !== 'none';
16253
- };
17127
+ ${VISIBLE_JS}
16254
17128
  const matches = (text) => {
16255
17129
  const haystack = text.toLowerCase();
16256
17130
  const needle = query.toLowerCase();
@@ -16295,18 +17169,42 @@ ${DESKTOP_POINT_JS}
16295
17169
  }
16296
17170
  const { el, text } = candidate;
16297
17171
  el.scrollIntoView({ block: 'center', inline: 'center' });
16298
- if (typeof el.focus === 'function') el.focus();
17172
+ if (typeof el.focus === 'function') el.focus({ preventScroll: true });
17173
+ const fieldType = el.tagName.toLowerCase() === 'select' ? 'select' : String(el.getAttribute('type') || 'text').toLowerCase();
17174
+ let nativeInputRequired = false;
17175
+ let semanticSelection = false;
17176
+ let selectionMatched = false;
16299
17177
  if (el.tagName.toLowerCase() === 'select') {
16300
17178
  const option = Array.from(el.options).find((item) => item.value === value || normalize(item.text) === value);
16301
17179
  if (option) el.value = option.value;
16302
17180
  else el.value = value;
17181
+ semanticSelection = true;
17182
+ selectionMatched = !!option;
17183
+ el.dispatchEvent(new Event('input', { bubbles: true }));
17184
+ el.dispatchEvent(new Event('change', { bubbles: true }));
17185
+ } else if (el instanceof HTMLInputElement && ${JSON.stringify([...BROWSER_DIRECT_VALUE_INPUT_TYPES])}.includes(fieldType)) {
17186
+ const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value')?.set;
17187
+ if (setter) {
17188
+ setter.call(el, value);
17189
+ el.dispatchEvent(new Event('input', { bubbles: true }));
17190
+ el.dispatchEvent(new Event('change', { bubbles: true }));
17191
+ } else {
17192
+ nativeInputRequired = true;
17193
+ }
16303
17194
  } else if (el.isContentEditable) {
16304
- el.textContent = value;
17195
+ const selection = getSelection();
17196
+ const range = document.createRange();
17197
+ range.selectNodeContents(el);
17198
+ selection.removeAllRanges();
17199
+ selection.addRange(range);
17200
+ nativeInputRequired = true;
17201
+ } else if (typeof el.select === 'function') {
17202
+ el.select();
17203
+ nativeInputRequired = true;
16305
17204
  } else {
16306
- el.value = value;
17205
+ el.setSelectionRange(0, String(el.value || '').length);
17206
+ nativeInputRequired = true;
16307
17207
  }
16308
- el.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'insertText', data: value }));
16309
- el.dispatchEvent(new Event('change', { bubbles: true }));
16310
17208
  const rect = el.getBoundingClientRect();
16311
17209
  return {
16312
17210
  filled: true,
@@ -16315,9 +17213,13 @@ ${DESKTOP_POINT_JS}
16315
17213
  index,
16316
17214
  tag: el.tagName.toLowerCase(),
16317
17215
  role: el.getAttribute('role') || null,
16318
- type: el.getAttribute('type') || null,
17216
+ type: fieldType,
16319
17217
  text,
16320
- value: el.isContentEditable ? el.textContent : el.value,
17218
+ actualValue: el.isContentEditable ? el.textContent : 'value' in el ? el.value : null,
17219
+ valueLength: String(el.isContentEditable ? el.textContent : 'value' in el ? el.value : '').length,
17220
+ nativeInputRequired,
17221
+ semanticSelection,
17222
+ selectionMatched,
16321
17223
  desktopPoint: desktopPoint(rect),
16322
17224
  rect: {
16323
17225
  x: Math.round(rect.x),
@@ -16332,6 +17234,18 @@ ${DESKTOP_POINT_JS}
16332
17234
  }
16333
17235
  async function computerBrowserFillCommand(query, value, options = {}) {
16334
17236
  const page = await selectChromePage(options);
17237
+ if (options.ref) {
17238
+ const target = await prepareBrowserElement(page, options.ref);
17239
+ const backendNodeId = parseBrowserRef(options.ref);
17240
+ const actual = await withChromeSession(page.webSocketDebuggerUrl, async (session) => await fillChromeNodeInSession(session, backendNodeId, value));
17241
+ if (target.desktopPoint) logRecordingAction({ type: "type", ...target.desktopPoint });
17242
+ console.log(JSON.stringify({
17243
+ ...browserTargetOutput(page),
17244
+ result: { filled: true, ref: target.ref, valueLength: actual.valueLength, rect: target.rect, element: target.element }
17245
+ }, null, 2));
17246
+ return;
17247
+ }
17248
+ if (!query) fail("Provide field text or --ref <id> from the latest browser state");
16335
17249
  const index = options.index ? parseCoord(options.index, "--index") : 0;
16336
17250
  if (index < 0) fail("--index must be >= 0");
16337
17251
  const result = await evaluateChromeTarget(
@@ -16339,10 +17253,28 @@ async function computerBrowserFillCommand(query, value, options = {}) {
16339
17253
  buildBrowserFillExpression(query, value, { exact: !!options.exact, index })
16340
17254
  );
16341
17255
  const point = browserActionPoint(result);
17256
+ if (typeof result === "object" && result !== null && "nativeInputRequired" in result && result.nativeInputRequired === true) {
17257
+ await replaceChromeText(page, value);
17258
+ const actualValue = await evaluateChromeTarget(page.webSocketDebuggerUrl, `(() => {
17259
+ const el = document.activeElement;
17260
+ return el && el.isContentEditable ? el.textContent : el && 'value' in el ? el.value : null;
17261
+ })()`);
17262
+ const fieldType = "type" in result && typeof result.type === "string" ? result.type.toLowerCase() : "";
17263
+ if (browserFillInvalid(actualValue, value, fieldType)) {
17264
+ fail(`Chrome reported that ${JSON.stringify(query)} contains ${JSON.stringify(actualValue)} after filling, expected ${JSON.stringify(value)}.`);
17265
+ }
17266
+ if (isRecord(result)) {
17267
+ Object.assign(result, { actualValue, valueLength: String(actualValue ?? "").length });
17268
+ }
17269
+ } else if (isRecord(result) && result.filled === true) {
17270
+ const fieldType = typeof result.type === "string" ? result.type : "";
17271
+ if (browserFillInvalid(result.actualValue, value, fieldType, result.semanticSelection === true, result.selectionMatched === true)) {
17272
+ fail(`Chrome reported that ${JSON.stringify(query)} contains ${JSON.stringify(result.actualValue)} after filling, expected ${JSON.stringify(value)}.`);
17273
+ }
17274
+ }
16342
17275
  if (point) logRecordingAction({ type: "type", ...point });
16343
17276
  console.log(JSON.stringify({
16344
- title: page.title ?? "",
16345
- url: page.url ?? "",
17277
+ ...browserTargetOutput(page),
16346
17278
  result
16347
17279
  }, null, 2));
16348
17280
  }
@@ -16352,11 +17284,7 @@ function buildBrowserWaitExpression(query, options) {
16352
17284
  const mode = ${JSON.stringify(options.mode)};
16353
17285
  const exact = ${JSON.stringify(options.exact)};
16354
17286
  const normalize = (text) => String(text || '').replace(/\\s+/g, ' ').trim();
16355
- const visible = (el) => {
16356
- const rect = el.getBoundingClientRect();
16357
- const style = getComputedStyle(el);
16358
- return rect.width > 0 && rect.height > 0 && style.visibility !== 'hidden' && style.display !== 'none';
16359
- };
17287
+ ${VISIBLE_JS}
16360
17288
  const matches = (text) => {
16361
17289
  const haystack = normalize(text).toLowerCase();
16362
17290
  const needle = query.toLowerCase();
@@ -16411,6 +17339,241 @@ async function computerBrowserWaitCommand(query, options = {}) {
16411
17339
  console.log(JSON.stringify({ ok: false, elapsedMs: Date.now() - start, attempts, result: lastResult }, null, 2));
16412
17340
  process.exitCode = 1;
16413
17341
  }
17342
+ async function computerBrowserScrollCommand(direction, options = {}) {
17343
+ const page = await selectChromePage(options);
17344
+ const normalized = direction.toLowerCase();
17345
+ if (!["up", "down", "left", "right"].includes(normalized)) fail("direction must be one of up|down|left|right");
17346
+ const amount = options.amount ? parseCoord(options.amount, "--amount") : 600;
17347
+ if (amount <= 0 || amount > 1e4) fail("--amount must be between 1 and 10000 CSS pixels");
17348
+ const layout = await sendChromeCommand(page.webSocketDebuggerUrl, "Page.getLayoutMetrics", {});
17349
+ const viewport = chromeVisualViewport(layout);
17350
+ const target = options.ref ? await prepareBrowserElement(page, options.ref) : null;
17351
+ const x = target?.rect.centerX ?? viewport.width / 2;
17352
+ const y = target?.rect.centerY ?? viewport.height / 2;
17353
+ const deltaX = normalized === "left" ? -amount : normalized === "right" ? amount : 0;
17354
+ const deltaY = normalized === "up" ? -amount : normalized === "down" ? amount : 0;
17355
+ await sendChromeCommand(page.webSocketDebuggerUrl, "Input.dispatchMouseEvent", {
17356
+ type: "mouseWheel",
17357
+ x,
17358
+ y,
17359
+ deltaX,
17360
+ deltaY
17361
+ });
17362
+ logRecordingAction({ type: "scroll", ...target?.desktopPoint ?? {} });
17363
+ console.log(JSON.stringify({
17364
+ ...browserTargetOutput(page),
17365
+ result: { scrolled: true, direction: normalized, amount, ref: target?.ref ?? null, x: Math.round(x), y: Math.round(y) }
17366
+ }, null, 2));
17367
+ }
17368
+ function browserKeyDetails(combo) {
17369
+ const tokens = combo.split("+").map((token) => token.trim()).filter(Boolean);
17370
+ if (tokens.length === 0) fail("key combination cannot be empty");
17371
+ let modifiers = 0;
17372
+ for (const token of tokens.slice(0, -1)) {
17373
+ const normalized2 = token.toLowerCase();
17374
+ if (["alt", "option"].includes(normalized2)) modifiers |= 1;
17375
+ else if (["ctrl", "control"].includes(normalized2)) modifiers |= 2;
17376
+ else if (["meta", "cmd", "command", "super"].includes(normalized2)) modifiers |= 4;
17377
+ else if (normalized2 === "shift") modifiers |= 8;
17378
+ else fail(`Unsupported browser key modifier ${JSON.stringify(token)}`);
17379
+ }
17380
+ const requested = tokens[tokens.length - 1];
17381
+ const aliases = {
17382
+ enter: { key: "Enter", code: "Enter", windowsVirtualKeyCode: 13 },
17383
+ return: { key: "Enter", code: "Enter", windowsVirtualKeyCode: 13 },
17384
+ tab: { key: "Tab", code: "Tab", windowsVirtualKeyCode: 9 },
17385
+ escape: { key: "Escape", code: "Escape", windowsVirtualKeyCode: 27 },
17386
+ esc: { key: "Escape", code: "Escape", windowsVirtualKeyCode: 27 },
17387
+ backspace: { key: "Backspace", code: "Backspace", windowsVirtualKeyCode: 8 },
17388
+ delete: { key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 },
17389
+ space: { key: " ", code: "Space", windowsVirtualKeyCode: 32 },
17390
+ up: { key: "ArrowUp", code: "ArrowUp", windowsVirtualKeyCode: 38 },
17391
+ arrowup: { key: "ArrowUp", code: "ArrowUp", windowsVirtualKeyCode: 38 },
17392
+ down: { key: "ArrowDown", code: "ArrowDown", windowsVirtualKeyCode: 40 },
17393
+ arrowdown: { key: "ArrowDown", code: "ArrowDown", windowsVirtualKeyCode: 40 },
17394
+ left: { key: "ArrowLeft", code: "ArrowLeft", windowsVirtualKeyCode: 37 },
17395
+ arrowleft: { key: "ArrowLeft", code: "ArrowLeft", windowsVirtualKeyCode: 37 },
17396
+ right: { key: "ArrowRight", code: "ArrowRight", windowsVirtualKeyCode: 39 },
17397
+ arrowright: { key: "ArrowRight", code: "ArrowRight", windowsVirtualKeyCode: 39 },
17398
+ home: { key: "Home", code: "Home", windowsVirtualKeyCode: 36 },
17399
+ end: { key: "End", code: "End", windowsVirtualKeyCode: 35 },
17400
+ pageup: { key: "PageUp", code: "PageUp", windowsVirtualKeyCode: 33 },
17401
+ pagedown: { key: "PageDown", code: "PageDown", windowsVirtualKeyCode: 34 }
17402
+ };
17403
+ const normalized = requested.toLowerCase();
17404
+ const alias = aliases[normalized];
17405
+ if (alias) return { ...alias, modifiers, ...alias.key === " " && modifiers === 0 ? { text: " " } : {} };
17406
+ if (/^[a-z]$/i.test(requested)) {
17407
+ const upper = requested.toUpperCase();
17408
+ const key = modifiers & 8 ? upper : requested.toLowerCase();
17409
+ return { key, code: `Key${upper}`, windowsVirtualKeyCode: upper.charCodeAt(0), modifiers, ...modifiers === 0 ? { text: key } : {} };
17410
+ }
17411
+ if (/^[0-9]$/.test(requested)) {
17412
+ return { key: requested, code: `Digit${requested}`, windowsVirtualKeyCode: requested.charCodeAt(0), modifiers, ...modifiers === 0 ? { text: requested } : {} };
17413
+ }
17414
+ fail(`Unsupported browser key ${JSON.stringify(requested)}`);
17415
+ }
17416
+ async function computerBrowserKeyCommand(combo, options = {}) {
17417
+ const page = await selectChromePage(options);
17418
+ const key = browserKeyDetails(combo);
17419
+ await withChromeSession(page.webSocketDebuggerUrl, async (session) => {
17420
+ await dispatchChromeKeyInSession(session, key);
17421
+ });
17422
+ logRecordingAction({ type: "key" });
17423
+ console.log(JSON.stringify({ targetId: page.id ?? null, result: { pressed: combo } }, null, 2));
17424
+ }
17425
+ async function dispatchChromeKeyInSession(session, key) {
17426
+ await sendChromeSessionCommand(session, "Input.dispatchKeyEvent", {
17427
+ type: "keyDown",
17428
+ ...key,
17429
+ nativeVirtualKeyCode: key.windowsVirtualKeyCode
17430
+ });
17431
+ await sendChromeSessionCommand(session, "Input.dispatchKeyEvent", {
17432
+ type: "keyUp",
17433
+ key: key.key,
17434
+ code: key.code,
17435
+ modifiers: key.modifiers,
17436
+ windowsVirtualKeyCode: key.windowsVirtualKeyCode,
17437
+ nativeVirtualKeyCode: key.windowsVirtualKeyCode
17438
+ });
17439
+ }
17440
+ async function computerBrowserTypeCommand(text, options = {}) {
17441
+ const page = await selectChromePage(options);
17442
+ await sendChromeCommand(page.webSocketDebuggerUrl, "Input.insertText", { text });
17443
+ logRecordingAction({ type: "type" });
17444
+ console.log(JSON.stringify({ targetId: page.id ?? null, result: { typed: true, length: text.length } }, null, 2));
17445
+ }
17446
+ function parseBrowserBatchActions(value) {
17447
+ let input;
17448
+ try {
17449
+ input = JSON.parse(value);
17450
+ } catch (error) {
17451
+ fail(`Batch actions must be valid JSON: ${error instanceof Error ? error.message : String(error)}`);
17452
+ }
17453
+ if (!Array.isArray(input) || input.length === 0 || input.length > 100) {
17454
+ fail("Batch actions must be a JSON array containing 1 to 100 actions");
17455
+ }
17456
+ return input.map((item, index) => {
17457
+ if (!isRecord(item) || typeof item.action !== "string") fail(`Batch action ${index} must be an object with an action`);
17458
+ if (item.action === "click") {
17459
+ if (typeof item.ref !== "string" && typeof item.ref !== "number") fail(`Batch click ${index} requires ref`);
17460
+ const buttons = { "1": "left", "2": "middle", "3": "right", left: "left", middle: "middle", right: "right" };
17461
+ const requestedButton = item.button === void 0 ? "left" : String(item.button).toLowerCase();
17462
+ const button = buttons[requestedButton];
17463
+ if (!button) fail(`Batch click ${index} button must be left, middle, or right`);
17464
+ return { action: "click", ref: String(item.ref), button, clicks: item.double === true ? 2 : 1 };
17465
+ }
17466
+ if (item.action === "fill") {
17467
+ if (typeof item.ref !== "string" && typeof item.ref !== "number" || typeof item.value !== "string") {
17468
+ fail(`Batch fill ${index} requires ref and string value`);
17469
+ }
17470
+ return { action: "fill", ref: String(item.ref), value: item.value };
17471
+ }
17472
+ if (item.action === "key") {
17473
+ if (typeof item.combo !== "string") fail(`Batch key ${index} requires combo`);
17474
+ return { action: "key", combo: item.combo };
17475
+ }
17476
+ if (item.action === "type") {
17477
+ if (typeof item.text !== "string") fail(`Batch type ${index} requires text`);
17478
+ return { action: "type", text: item.text };
17479
+ }
17480
+ if (item.action === "scroll") {
17481
+ const directions = { up: "up", down: "down", left: "left", right: "right" };
17482
+ const direction = typeof item.direction === "string" ? directions[item.direction] : void 0;
17483
+ if (!direction) {
17484
+ fail(`Batch scroll ${index} direction must be up, down, left, or right`);
17485
+ }
17486
+ const amount = item.amount === void 0 ? 600 : Number(item.amount);
17487
+ if (!Number.isFinite(amount) || amount <= 0 || amount > 1e4) fail(`Batch scroll ${index} amount must be between 1 and 10000`);
17488
+ if (item.ref !== void 0 && typeof item.ref !== "string" && typeof item.ref !== "number") fail(`Batch scroll ${index} ref is invalid`);
17489
+ return {
17490
+ action: "scroll",
17491
+ direction,
17492
+ amount,
17493
+ ...item.ref === void 0 ? {} : { ref: String(item.ref) }
17494
+ };
17495
+ }
17496
+ if (item.action === "wait") {
17497
+ if (typeof item.text !== "string") fail(`Batch wait ${index} requires text`);
17498
+ const mode = item.mode === void 0 ? "any" : String(item.mode);
17499
+ if (!["any", "text", "title", "url", "control"].includes(mode)) fail(`Batch wait ${index} mode is invalid`);
17500
+ const timeoutMs = item.timeoutMs === void 0 ? 1e4 : Number(item.timeoutMs);
17501
+ const pollMs = item.pollMs === void 0 ? 250 : Number(item.pollMs);
17502
+ if (!Number.isFinite(timeoutMs) || timeoutMs < 0) fail(`Batch wait ${index} timeoutMs must be >= 0`);
17503
+ if (!Number.isFinite(pollMs) || pollMs < 50 || pollMs > 2e3) fail(`Batch wait ${index} pollMs must be between 50 and 2000`);
17504
+ return { action: "wait", text: item.text, mode, exact: item.exact === true, timeoutMs, pollMs };
17505
+ }
17506
+ fail(`Batch action ${index} has unsupported action ${JSON.stringify(item.action)}`);
17507
+ });
17508
+ }
17509
+ async function computerBrowserBatchCommand(actionsJson, options = {}) {
17510
+ const actions = parseBrowserBatchActions(actionsJson);
17511
+ const page = await selectChromePage(options);
17512
+ const { targetId, cached } = readBrowserRefCache(page);
17513
+ const startedAt = Date.now();
17514
+ const completed = [];
17515
+ try {
17516
+ await withChromeSession(page.webSocketDebuggerUrl, async (session) => {
17517
+ const layout = await sendChromeSessionCommand(session, "Page.getLayoutMetrics", {});
17518
+ const viewportRect = browserViewportRect(layout);
17519
+ for (const [index, action] of actions.entries()) {
17520
+ const actionStartedAt = Date.now();
17521
+ if (action.action === "click") {
17522
+ const ref = parseBrowserRef(action.ref);
17523
+ const rect = await prepareBrowserRefInSession(session, ref, cached, viewportRect);
17524
+ await dispatchChromeClickInSession(session, rect.centerX, rect.centerY, action.button, action.clicks);
17525
+ logRecordingAction({ type: "click" });
17526
+ completed.push({ index, action: action.action, ref: String(ref), elapsedMs: Date.now() - actionStartedAt });
17527
+ } else if (action.action === "fill") {
17528
+ const ref = parseBrowserRef(action.ref);
17529
+ await prepareBrowserRefInSession(session, ref, cached, viewportRect);
17530
+ const result = await fillChromeNodeInSession(session, ref, action.value);
17531
+ logRecordingAction({ type: "type" });
17532
+ completed.push({ index, action: action.action, ref: String(ref), valueLength: result.valueLength, elapsedMs: Date.now() - actionStartedAt });
17533
+ } else if (action.action === "key") {
17534
+ await dispatchChromeKeyInSession(session, browserKeyDetails(action.combo));
17535
+ logRecordingAction({ type: "key" });
17536
+ completed.push({ index, action: action.action, elapsedMs: Date.now() - actionStartedAt });
17537
+ } else if (action.action === "type") {
17538
+ await sendChromeSessionCommand(session, "Input.insertText", { text: action.text });
17539
+ logRecordingAction({ type: "type" });
17540
+ completed.push({ index, action: action.action, length: action.text.length, elapsedMs: Date.now() - actionStartedAt });
17541
+ } else if (action.action === "scroll") {
17542
+ const ref = action.ref ? parseBrowserRef(action.ref) : null;
17543
+ const rect = ref ? await prepareBrowserRefInSession(session, ref, cached, viewportRect) : viewportRect;
17544
+ const deltaX = action.direction === "left" ? -action.amount : action.direction === "right" ? action.amount : 0;
17545
+ const deltaY = action.direction === "up" ? -action.amount : action.direction === "down" ? action.amount : 0;
17546
+ await sendChromeSessionCommand(session, "Input.dispatchMouseEvent", {
17547
+ type: "mouseWheel",
17548
+ x: rect.centerX,
17549
+ y: rect.centerY,
17550
+ deltaX,
17551
+ deltaY
17552
+ });
17553
+ logRecordingAction({ type: "scroll" });
17554
+ completed.push({ index, action: action.action, direction: action.direction, amount: action.amount, elapsedMs: Date.now() - actionStartedAt });
17555
+ } else {
17556
+ const expression = buildBrowserWaitExpression(action.text, { mode: action.mode, exact: action.exact });
17557
+ let attempts = 0;
17558
+ let result = null;
17559
+ while (Date.now() - actionStartedAt <= action.timeoutMs) {
17560
+ attempts++;
17561
+ result = await evaluateChromeSession(session, expression);
17562
+ if (isRecord(result) && result.matched === true) break;
17563
+ await sleep(action.pollMs);
17564
+ }
17565
+ if (!isRecord(result) || result.matched !== true) fail(`Batch wait ${index} timed out after ${Date.now() - actionStartedAt}ms`);
17566
+ completed.push({ index, action: action.action, attempts, elapsedMs: Date.now() - actionStartedAt });
17567
+ }
17568
+ }
17569
+ });
17570
+ } catch (error) {
17571
+ const message = error instanceof Error ? error.message : String(error);
17572
+ console.log(JSON.stringify({ ok: false, targetId, elapsedMs: Date.now() - startedAt, completed, failedIndex: completed.length, error: message }, null, 2));
17573
+ throw error;
17574
+ }
17575
+ console.log(JSON.stringify({ ok: true, targetId, elapsedMs: Date.now() - startedAt, completed }, null, 2));
17576
+ }
16414
17577
  async function computerClickCommand(xStr, yStr, options) {
16415
17578
  const dimensions = getDisplayDimensions();
16416
17579
  const x = parseScreenCoord(xStr, "x", dimensions.width);
@@ -16533,12 +17696,55 @@ async function computerLaunchCommand(app, args) {
16533
17696
  const baseArgs = app === "chrome" && !existsSync3(CHROME_WRAPPER) ? LEGACY_CHROME_ALIAS : APP_ALIASES[app] ?? [app];
16534
17697
  const bin = baseArgs[0];
16535
17698
  const fullArgs = [...baseArgs.slice(1), ...args];
17699
+ const expectsNewChromePage = args.some((arg) => !arg.startsWith("-"));
17700
+ const existingChromePageIds = /* @__PURE__ */ new Set();
17701
+ if (app === "chrome") {
17702
+ try {
17703
+ for (const page of await getChromePages()) {
17704
+ if (page.id) existingChromePageIds.add(page.id);
17705
+ }
17706
+ } catch {
17707
+ }
17708
+ }
16536
17709
  const child = spawn4(bin, fullArgs, {
16537
17710
  env: withDisplay(),
16538
17711
  detached: true,
16539
17712
  stdio: "ignore"
16540
17713
  });
16541
17714
  child.unref();
17715
+ if (app === "chrome") {
17716
+ const startedAt = Date.now();
17717
+ let pageCount = 0;
17718
+ let readyPage = null;
17719
+ while (Date.now() - startedAt < 15e3) {
17720
+ let pages = [];
17721
+ try {
17722
+ pages = await getChromePages();
17723
+ } catch {
17724
+ }
17725
+ pageCount = pages.length;
17726
+ const candidates = expectsNewChromePage && existingChromePageIds.size > 0 ? pages.filter((page) => page.id && !existingChromePageIds.has(page.id)) : pages;
17727
+ for (const page of candidates) {
17728
+ if (!page.webSocketDebuggerUrl || !page.url || page.url === "about:blank") continue;
17729
+ const readyState = await evaluateChromeTarget(page.webSocketDebuggerUrl, "document.readyState");
17730
+ if (readyState === "interactive" || readyState === "complete") {
17731
+ readyPage = page;
17732
+ break;
17733
+ }
17734
+ }
17735
+ if (readyPage) break;
17736
+ try {
17737
+ if (child.pid) process.kill(child.pid, 0);
17738
+ } catch {
17739
+ fail(`Chrome exited before its desktop control channel became ready.`);
17740
+ }
17741
+ await sleep(100);
17742
+ }
17743
+ if (!readyPage?.webSocketDebuggerUrl) fail(`Chrome launched but its requested page was not controllable after 15 seconds.`);
17744
+ await sendChromeCommand(readyPage.webSocketDebuggerUrl, "Page.bringToFront", {});
17745
+ console.log(`launched ${bin} (pid ${child.pid}, ready in ${Date.now() - startedAt}ms, page ${readyPage.id}, ${pageCount} open)`);
17746
+ return;
17747
+ }
16542
17748
  console.log(`launched ${bin} (pid ${child.pid})`);
16543
17749
  }
16544
17750
 
@@ -20965,10 +22171,20 @@ if (isAgentMode()) {
20965
22171
  computer.command("status").description("Show which desktop services are running and the active preview URL").action(wrap(() => computerStatusCommand()));
20966
22172
  computer.command("screenshot <path>").description("Capture the current desktop to a PNG file. Use --raw or --grid for agent click planning; omit both for a branded shareable image.").option("--raw", "Save a 1:1 desktop capture with no branding, padding, or rounded corners").option("--grid [px]", "Overlay a coordinate grid on a 1:1 desktop capture. Default 100px.").action(wrap((path6, options) => computerScreenshotCommand(path6, options)));
20967
22173
  computer.command("observe <path>").description("Wait briefly for the desktop to settle, save a 1:1 screenshot, and print JSON screen context for agents.").option("--raw", "Save a 1:1 desktop capture with no coordinate grid").option("--grid [px]", "Overlay a coordinate grid on the 1:1 desktop capture. Default 100px.").option("--timeout <ms>", "Maximum time to wait for visual stability. Default 3000.").option("--stable-ms <ms>", "Required unchanged time before the screen is considered stable. Default 600.").option("--poll-ms <ms>", "Screenshot polling interval while waiting. Default 200.").action(wrap((path6, options) => computerObserveCommand(path6, options)));
20968
- computer.command("browser").description("Print JSON for Chrome tabs launched through Replicas, including page titles and URLs.").option("--snapshot", "Include visible page text and interactive controls with bounding boxes").option("--limit <chars>", "Maximum body text characters per page snapshot. Default 4000.").option("--element-limit <n>", "Maximum controls per page snapshot. Default 80.").action(wrap((options) => computerBrowserCommand(options)));
20969
- computer.command("browser-click <text>").description("Click the first visible Chrome control whose text, label, placeholder, or href matches <text>.").option("--exact", "Require an exact text match instead of substring matching").option("--index <n>", "When multiple controls match, click the nth match. Default 0.").option("--id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((text, options) => computerBrowserClickCommand(text, options)));
20970
- computer.command("browser-fill <field> <value>").description("Fill the first visible Chrome field whose label, placeholder, name, or text matches <field>.").option("--exact", "Require an exact field match instead of substring matching").option("--index <n>", "When multiple fields match, fill the nth match. Default 0.").option("--id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((field, value, options) => computerBrowserFillCommand(field, value, options)));
20971
- computer.command("browser-wait <text>").description("Wait until Chrome page title, URL, body text, or controls match <text>.").option("--mode <mode>", "Where to match: any, text, title, url, or control. Default any.").option("--exact", "Require an exact match instead of substring matching").option("--timeout <ms>", "Maximum time to wait. Default 10000.").option("--poll-ms <ms>", "Polling interval. Default 250.").option("--id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((text, options) => computerBrowserWaitCommand(text, options)));
22174
+ computer.command("browser").description("Print JSON for Chrome tabs launched through Replicas, including page titles and URLs.").option("--snapshot", "Include visible page text and interactive controls with bounding boxes").option("--limit <chars>", "Maximum body text characters per page snapshot. Default 4000.").option("--element-limit <n>", "Maximum controls per page snapshot. Default 80.").option("--target-id <id>", "Only include the Chrome page with this target id").option("--page <n>", "When multiple pages match, include the nth page. Default 0.").option("--title <text>", "Only include pages whose title contains text").option("--url <text>", "Only include pages whose URL contains text").action(wrap((options) => computerBrowserCommand(options)));
22175
+ computer.command("browser-state <path>").description("Wait for a Chrome page to settle, capture its viewport, and print a bounded accessibility state with actionable refs.").option("--full", "Return the full accessibility tree instead of a diff from the previous state").option("--limit <chars>", "Maximum visible text characters. Default 8000.").option("--element-limit <n>", "Maximum actionable elements. Default 120.").option("--timeout <ms>", "Maximum time to wait for state stability. Default 5000.").option("--stable-ms <ms>", "Required unchanged time before capture. Default 500.").option("--poll-ms <ms>", "State polling interval. Default 100.").option("--target-id <id>", "Target the Chrome page with this stable target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((path6, options) => computerBrowserStateCommand(path6, options)));
22176
+ computer.command("browser-batch <actions>").description("Run an ordered, fail-fast JSON array of browser actions through one Chrome session.").option("--target-id <id>", "Target the Chrome page with this stable target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((actions, options) => computerBrowserBatchCommand(actions, options)));
22177
+ computer.command("browser-click [text]").description("Click a Chrome element by ref from browser-state, or fall back to visible control text.").option("--ref <id>", "Click the backend DOM node ref from the latest browser state").option("--exact", "Require an exact text match instead of substring matching").option("--index <n>", "When multiple controls match, click the nth match. Default 0.").option("--button <button>", "Mouse button for ref clicks: left, middle, or right. Default left.").option("--double", "Double-click a ref target").option("--target-id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((text, options) => computerBrowserClickCommand(text, options)));
22178
+ computer.command("browser-fill [field] [value]").description("Fill a Chrome field by ref from browser-state, or fall back to visible field text.").option("--ref <id>", "Fill the backend DOM node ref from the latest browser state").option("--exact", "Require an exact field match instead of substring matching").option("--index <n>", "When multiple fields match, fill the nth match. Default 0.").option("--target-id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((field, value, options) => {
22179
+ const resolvedValue = value ?? (options.ref ? field : void 0);
22180
+ const resolvedField = value === void 0 && options.ref ? void 0 : field;
22181
+ if (resolvedValue === void 0) throw new Error("Provide <field> <value>, or --ref <id> <value>");
22182
+ return computerBrowserFillCommand(resolvedField, resolvedValue, options);
22183
+ }));
22184
+ computer.command("browser-scroll <direction>").description("Send trusted wheel input to a Chrome element ref or the center of the target page.").option("--ref <id>", "Scroll from the backend DOM node ref from the latest browser state").option("--amount <px>", "Scroll distance in CSS pixels. Default 600.").option("--target-id <id>", "Target the Chrome page with this stable target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((direction, options) => computerBrowserScrollCommand(direction, options)));
22185
+ computer.command("browser-key <combo>").description("Press a key or modifier chord in the target Chrome page with trusted browser input.").option("--target-id <id>", "Target the Chrome page with this stable target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((combo, options) => computerBrowserKeyCommand(combo, options)));
22186
+ computer.command("browser-type <text>").description("Type literal text into the focused field in the target Chrome page with trusted browser input.").option("--target-id <id>", "Target the Chrome page with this stable target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((text, options) => computerBrowserTypeCommand(text, options)));
22187
+ computer.command("browser-wait <text>").description("Wait until Chrome page title, URL, body text, or controls match <text>.").option("--mode <mode>", "Where to match: any, text, title, url, or control. Default any.").option("--exact", "Require an exact match instead of substring matching").option("--timeout <ms>", "Maximum time to wait. Default 10000.").option("--poll-ms <ms>", "Polling interval. Default 250.").option("--target-id <id>", "Only target the Chrome page with this target id").option("--page <n>", "When multiple pages match, target the nth page. Default 0.").option("--title <text>", "Only target pages whose title contains text").option("--url <text>", "Only target pages whose URL contains text").action(wrap((text, options) => computerBrowserWaitCommand(text, options)));
20972
22188
  computer.command("click <x> <y>").description("Move to (x, y) and click. Coordinates are pixels or percentages like 50% 50%.").option("-b, --button <n>", "Mouse button (1=left, 2=middle, 3=right). Default 1.").option("--double", "Double-click instead of single-click").option("--modifiers <mods>", "Hold modifier keys during the click, e.g. ctrl or ctrl+shift").action(wrap((x, y, options) => computerClickCommand(x, y, options)));
20973
22189
  computer.command("move <x> <y>").description("Move the mouse to (x, y) without clicking. Coordinates are pixels or percentages like 50% 50%.").action(wrap((x, y) => computerMoveCommand(x, y)));
20974
22190
  computer.command("type <text>").description("Type a literal string into the focused field. Use `key` for key combos like ctrl+l.").option("--delay <ms>", "Per-character delay in ms (default 12 \u2248 80 wpm)").action(wrap((text, options) => computerTypeCommand(text, options)));