@automatebrowser/mcp 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +17 -0
- package/.claude-plugin/plugin.json +10 -0
- package/CHANGELOG.md +167 -0
- package/LICENSE +202 -0
- package/README.md +2092 -0
- package/dist/chunk-R5JREBXQ.js +211 -0
- package/dist/chunk-WAXVZKT5.js +5363 -0
- package/dist/cli.js +135 -0
- package/dist/index.js +287 -0
- package/dist/relay.js +919 -0
- package/package.json +80 -0
- package/skills/automate-browser/SKILL.md +181 -0
- package/skills/automate-browser/references/capture-and-diagnostics.md +486 -0
- package/skills/automate-browser/references/page-interaction.md +237 -0
- package/skills/automate-browser/references/reading-and-extraction.md +113 -0
- package/skills/automate-browser/references/sessions-and-state.md +186 -0
- package/skills/automate-browser/references/tabs-and-multi-agent.md +156 -0
- package/skills/automate-browser/references/tool-reference.md +236 -0
- package/skills/automate-browser/references/troubleshooting.md +136 -0
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# Tabs, ownership, and sharing a browser
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
| Section | What it answers |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [The one rule](#the-one-rule) | Which tab am I allowed to drive? |
|
|
8
|
+
| [Getting a tab](#getting-a-tab) | `new` vs `select` vs `switch` |
|
|
9
|
+
| [Focus](#focus-who-may-take-it) | What is allowed to interrupt the user |
|
|
10
|
+
| [Finishing](#finishing-what-gets-cleaned-up) | What `browser_release_client` closes |
|
|
11
|
+
| [Two agents, one browser](#two-agents-one-browser) | Claims, leases, `TAB_CLAIMED`, `LEASE_LOST` |
|
|
12
|
+
| [Several browsers](#several-browsers) | `browser_select_client` and friends |
|
|
13
|
+
| [Being a good neighbour](#being-a-good-neighbour) | The habits that keep you welcome |
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## The one rule
|
|
18
|
+
|
|
19
|
+
**You drive a tab you own. Never the user's.**
|
|
20
|
+
|
|
21
|
+
A tab becomes yours in exactly two ways: you **opened** it, or you **adopted** it because the user
|
|
22
|
+
asked you to. There is no third way, and there is no fallback to "whatever tab is in front".
|
|
23
|
+
|
|
24
|
+
You do not have to do anything to get one. On your first action the server opens a background tab and
|
|
25
|
+
adopts it for you. It stays yours until you select another.
|
|
26
|
+
|
|
27
|
+
> **This changed on 2026-08-30, and older guidance says the opposite.** Previously, an agent with no
|
|
28
|
+
> explicit selection inherited the browser's *focused* tab — the one the user was reading. That meant
|
|
29
|
+
> "go and test this URL" could navigate away a tab holding unsaved work. If you have seen advice that
|
|
30
|
+
> says your target follows the user's focus, it is out of date.
|
|
31
|
+
|
|
32
|
+
## Getting a tab
|
|
33
|
+
|
|
34
|
+
| You want | Call | What happens |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| Somewhere to work | *nothing* | A background tab is opened and adopted on your first action |
|
|
37
|
+
| Somewhere to work, at a URL | `browser_new_tab { url }` | Opens **in the background**, adopted automatically |
|
|
38
|
+
| The tab the user already has open | `browser_select_tab { url \| title \| tabId \| index }` | Adopted where it sits, **not** brought forward |
|
|
39
|
+
| To show the user something | `browser_switch_tab { tabId \| index }` | Brings it to the front — **takes their focus** |
|
|
40
|
+
| To see the site logged OUT | `browser_new_tab { url, incognito: true }` | A **private window**, with none of the user's logins |
|
|
41
|
+
|
|
42
|
+
Prefer `url` or `title` over `index` when adopting. An index shifts every time any tab is opened or
|
|
43
|
+
closed; a URL substring does not.
|
|
44
|
+
|
|
45
|
+
```
|
|
46
|
+
browser_select_tab { url: "localhost:3000" }
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
### A logged-out tab, and the setting it needs
|
|
50
|
+
|
|
51
|
+
Every tab you drive is the user's real, signed-in profile, so "what does a first-time visitor see?"
|
|
52
|
+
is normally unanswerable without logging them out for real. `incognito: true` opens a private window
|
|
53
|
+
instead: a clean session with no cookies and no logins. It is the way to check a signup flow, a
|
|
54
|
+
paywall, a cookie banner, or anything that looks different to a stranger.
|
|
55
|
+
|
|
56
|
+
**It needs a one-off setting that only a person can turn on**, on the extension's own details page —
|
|
57
|
+
"Allow in Incognito" in Chrome, "Allow in InPrivate" in Edge. Without it the call fails with
|
|
58
|
+
`INCOGNITO_BLOCKED` and the message spells out where to click. **Ask the user to do it; do not retry.**
|
|
59
|
+
Turning it on restarts the extension, so the connection blinks.
|
|
60
|
+
|
|
61
|
+
Three things behave differently in a private tab, all of them measured rather than assumed:
|
|
62
|
+
|
|
63
|
+
- It is **claimed and released like any other tab**, and shows in `browser_list_tabs` marked
|
|
64
|
+
`(private)`. Check that marker before you trust a session — a private tab that looks ordinary is how
|
|
65
|
+
"why am I logged out?" starts.
|
|
66
|
+
- **Cookies are a separate jar.** `browser_get_cookies` and `browser_set_cookie` read and write the
|
|
67
|
+
private tab's own jar, not the user's. A cookie you set in a private tab is gone when the window
|
|
68
|
+
closes, and the user's real session is neither visible there nor at risk from it.
|
|
69
|
+
- **`localStorage` and `sessionStorage` are the private ones too**, so `browser_storage` sees an empty
|
|
70
|
+
origin rather than the user's saved state.
|
|
71
|
+
|
|
72
|
+
Closing the tab ends the session. There is nothing to clean up, and nothing survives.
|
|
73
|
+
|
|
74
|
+
## Focus: who may take it
|
|
75
|
+
|
|
76
|
+
`browser_switch_tab` is the **only** tool that moves the user's focus, and `browser_new_tab { active:
|
|
77
|
+
true }` is the only argument that does. Both are for one situation: the user asked to be *shown*
|
|
78
|
+
something.
|
|
79
|
+
|
|
80
|
+
Everything else — navigating, clicking, typing, reading, snapshotting, screenshotting — runs on a
|
|
81
|
+
background tab without disturbing them. Assume the user is working in another window the entire time
|
|
82
|
+
you are running, because they usually are.
|
|
83
|
+
|
|
84
|
+
## Finishing: what gets cleaned up
|
|
85
|
+
|
|
86
|
+
`browser_release_client` frees the browser for other agents **and closes every tab you opened**. A tab
|
|
87
|
+
you *adopted* from the user is left exactly where it was.
|
|
88
|
+
|
|
89
|
+
That asymmetry is deliberate and enforced in the server, not a convention you have to remember.
|
|
90
|
+
Closing a tab the user handed you would lose their work just as surely as navigating it away.
|
|
91
|
+
|
|
92
|
+
Three honest limits:
|
|
93
|
+
|
|
94
|
+
- Cleanup happens on an **explicit** release. If the editor simply exits, the tab you opened is left
|
|
95
|
+
behind. Call `browser_release_client` when you are genuinely finished.
|
|
96
|
+
- Nothing cleans up a tab you adopted. That is the user's tab and stays their business.
|
|
97
|
+
- Cleanup only ever goes to the browser that owns the tabs. If that browser has disconnected, or quits
|
|
98
|
+
part-way through the sweep, the rest stay on its books and the next release closes them. So a release
|
|
99
|
+
can legitimately close nothing and still report success — that is not a fault, and those tabs are
|
|
100
|
+
not forgotten.
|
|
101
|
+
|
|
102
|
+
**If one of your tabs survives a release while its browser stayed connected throughout, say so.**
|
|
103
|
+
That should not happen. Until
|
|
104
|
+
2026-09-02 it did, and silently: the extension reconnecting mid-session (the user reloading it, or its
|
|
105
|
+
background worker being evicted and revived) gave the browser a new identity, and the list of tabs you
|
|
106
|
+
had opened was filed under the old one. Release closed nothing. Ownership now follows the extension's
|
|
107
|
+
own stored id, which a reconnect does not change. Worth knowing because the failure leaves no error —
|
|
108
|
+
the release reports success and the tabs simply stay.
|
|
109
|
+
|
|
110
|
+
## Two agents, one browser
|
|
111
|
+
|
|
112
|
+
Every editor's server connects to one shared relay, so **every agent sees every browser**. Ownership
|
|
113
|
+
is per **tab**: two agents drive two tabs of the same browser concurrently, and only same-tab access
|
|
114
|
+
is serialised. Driving takes a soft lease of about a minute, renewed by each action and released on
|
|
115
|
+
idle, on disconnect, or on `browser_release_client`. *Selecting* a tab does not claim it — only
|
|
116
|
+
acting does.
|
|
117
|
+
|
|
118
|
+
Because each agent now opens its own tab, two agents that select nothing **cannot** collide. A
|
|
119
|
+
conflict means you deliberately aimed at the same tab.
|
|
120
|
+
|
|
121
|
+
**`TAB_CLAIMED` — someone else is driving that tab.** The message names them. In order of politeness:
|
|
122
|
+
|
|
123
|
+
1. Work somewhere else — `browser_select_tab { url: "the thing you were sent for" }`, or just open a
|
|
124
|
+
new tab.
|
|
125
|
+
2. `browser_force_claim` — only when that exact tab is the point of the task. The other agent is told
|
|
126
|
+
immediately, mid-task.
|
|
127
|
+
|
|
128
|
+
**`LEASE_LOST` — you were the one displaced.** It arrives as a notice on your next result, whatever
|
|
129
|
+
tool that was, and names the tab and who took it. Stop: your refs belong to a page you no longer
|
|
130
|
+
control. Pick another tab, or take it back if the task demands it.
|
|
131
|
+
|
|
132
|
+
## Several browsers
|
|
133
|
+
|
|
134
|
+
Tools refuse with a list rather than guessing which browser you meant. Choose once:
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
browser_select_client { browser: "chrome" } // or { label: "..." } or { id: "..." }
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
It sticks for your session only and does not change what other agents see. `browser_force_claim` is
|
|
141
|
+
the same selector plus a steal, and takes the **whole browser** — it is the only thing that still
|
|
142
|
+
does, now that a drive always names its tab.
|
|
143
|
+
|
|
144
|
+
`browser_status` is the one call that explains the rest: which relay you are on and whether its
|
|
145
|
+
version matches yours, your own agent name, every connected browser with its live tab and driver, and
|
|
146
|
+
every other agent connected. Call it before guessing.
|
|
147
|
+
|
|
148
|
+
## Being a good neighbour
|
|
149
|
+
|
|
150
|
+
- **Never drive the user's real, logged-in tabs while testing.** You now get a throwaway tab by
|
|
151
|
+
default — do not go out of your way to defeat that.
|
|
152
|
+
- **Do not close tabs you did not open.** The server enforces this on release; do not undo it by
|
|
153
|
+
calling `browser_close_tab` on a tab you adopted.
|
|
154
|
+
- **Release when you are done** with a browser others may want.
|
|
155
|
+
- **Take a name.** `AUTOMATE_BROWSER_CLIENT_NAME` is what a human sees in the extension popup and
|
|
156
|
+
what other agents see in a claim error. `mcp-12345` tells nobody anything.
|
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
# Every tool, and its gotcha
|
|
2
|
+
|
|
3
|
+
The tables below are **generated from the server's own schemas** by `npm run docs:generate`, so the
|
|
4
|
+
tool list here cannot drift from the code. Do not edit between the `AUTO-GENERATED` markers.
|
|
5
|
+
|
|
6
|
+
The **Gotchas** section after them is hand-written, and is where the value is: the table tells you a
|
|
7
|
+
tool exists, the gotcha tells you the thing that will cost you an hour.
|
|
8
|
+
|
|
9
|
+
## Contents
|
|
10
|
+
|
|
11
|
+
| Section | What it covers |
|
|
12
|
+
|---|---|
|
|
13
|
+
| [The tools](#the-tools) | All of them, generated, grouped |
|
|
14
|
+
| [Arguments and gotchas](#arguments-and-gotchas) | What to pass, and what bites |
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## The tools
|
|
19
|
+
|
|
20
|
+
<!-- AUTO-GENERATED:tools START — do not edit by hand; run `npm run docs:generate` -->
|
|
21
|
+
|
|
22
|
+
### Navigation & history
|
|
23
|
+
| Tool | Description |
|
|
24
|
+
|------|-------------|
|
|
25
|
+
| `browser_navigate` | Navigate to a URL in YOUR OWN tab, opened in the background if you have none |
|
|
26
|
+
| `browser_go_back` | Go back to the previous page |
|
|
27
|
+
| `browser_go_forward` | Go forward to the next page |
|
|
28
|
+
|
|
29
|
+
### Snapshot & interaction
|
|
30
|
+
| Tool | Description |
|
|
31
|
+
|------|-------------|
|
|
32
|
+
| `browser_snapshot` | Capture accessibility snapshot of the current page |
|
|
33
|
+
| `browser_click` | Click an element by ref, or a viewport point by x/y |
|
|
34
|
+
| `browser_hover` | Hover over element on page |
|
|
35
|
+
| `browser_type` | Type text into editable element |
|
|
36
|
+
| `browser_select_option` | Select an option in a dropdown |
|
|
37
|
+
| `browser_drag` | Perform drag and drop between two elements |
|
|
38
|
+
|
|
39
|
+
### Input & timing
|
|
40
|
+
| Tool | Description |
|
|
41
|
+
|------|-------------|
|
|
42
|
+
| `browser_press_key` | Press a key or modifier combo (e.g. Enter, Tab, "Control+A", "Shift+Tab") on the focused element |
|
|
43
|
+
| `browser_wait` | Wait for a specified time in seconds |
|
|
44
|
+
| `browser_wait_for` | Wait for a page condition to become true (element appears/disappears, text appears, URL changes) |
|
|
45
|
+
|
|
46
|
+
### Reading content
|
|
47
|
+
| Tool | Description |
|
|
48
|
+
|------|-------------|
|
|
49
|
+
| `browser_read_page` | Read the page's main content as clean text or Markdown (strips nav/scripts/styles) |
|
|
50
|
+
| `browser_get_html` | Get the raw outerHTML of the page (or of a specific element by `ref`) |
|
|
51
|
+
| `browser_find` | Find elements by text, role, and/or CSS selector and return fresh refs WITHOUT a full snapshot |
|
|
52
|
+
|
|
53
|
+
### Page-declared tools
|
|
54
|
+
Actions the PAGE publishes about itself, which an agent can call directly instead of finding and clicking controls for. Forward-looking: the standard is a draft and almost no live site declares anything yet, so `list` normally comes back empty with the reason.
|
|
55
|
+
|
|
56
|
+
| Tool | Description |
|
|
57
|
+
|------|-------------|
|
|
58
|
+
| `browser_page_tools` | List and call actions a page declares about itself (WebMCP) |
|
|
59
|
+
|
|
60
|
+
### Forms & scrolling
|
|
61
|
+
| Tool | Description |
|
|
62
|
+
|------|-------------|
|
|
63
|
+
| `browser_fill_form` | Fill multiple form fields (inputs, textareas, selects, checkboxes, radios, contenteditable) in ONE call |
|
|
64
|
+
| `browser_clear` | Clear the value of an input, textarea, or contenteditable element by `ref` |
|
|
65
|
+
| `browser_scroll` | Scroll the page or an element |
|
|
66
|
+
|
|
67
|
+
### State: cookies, storage, network, downloads, dialogs
|
|
68
|
+
| Tool | Description |
|
|
69
|
+
|------|-------------|
|
|
70
|
+
| `browser_get_cookies` | List cookies for the URL of the tab you are driving (optionally filter by `name`) |
|
|
71
|
+
| `browser_set_cookie` | Set (create/overwrite) a cookie on the URL of the tab you are driving |
|
|
72
|
+
| `browser_storage` | Read or write the page's localStorage/sessionStorage |
|
|
73
|
+
| `browser_network_requests` | List network requests the tab you are driving made on the CURRENT page (method, URL, status, type, timing) — pass includePreserved for the pages before it |
|
|
74
|
+
| `browser_handle_dialog` | Control JS dialogs (alert/confirm/prompt) |
|
|
75
|
+
| `browser_downloads` | Recent downloads: final path on disk, URL, mime, size, state |
|
|
76
|
+
| `browser_proxy` | Route the browser through a proxy |
|
|
77
|
+
|
|
78
|
+
### Performance
|
|
79
|
+
`browser_perf_trace` measures THIS machine on THIS run and needs `browser_advanced_mode` — except `action: "memory"`, which samples the JS heap with no debugger and no banner. `browser_perf_field_data` needs no browser at all - it reads Google's Chrome UX Report for what real visitors experienced, and sends the URL you ask about to that public API.
|
|
80
|
+
|
|
81
|
+
| Tool | Description |
|
|
82
|
+
|------|-------------|
|
|
83
|
+
| `browser_perf_field_data` | Real-user Core Web Vitals (p75 LCP/INP/CLS/FCP/TTFB) for a URL, from Google's Chrome UX Report |
|
|
84
|
+
|
|
85
|
+
### Capture & evaluation
|
|
86
|
+
| Tool | Description |
|
|
87
|
+
|------|-------------|
|
|
88
|
+
| `browser_screenshot` | Capture the visible viewport of the tab you are driving — including a background tab, which is rendered via the debugger (brief banner) rather than refused |
|
|
89
|
+
| `browser_get_console_logs` | Console logs, uncaught errors with stacks, and service-worker lifecycle (register/state/messages) |
|
|
90
|
+
| `browser_issues` | Problems the browser detected that produce NO console error: blocked content (CSP), deprecated API use, browser interventions, and failed or 4xx/5xx network requests |
|
|
91
|
+
| `browser_eval` | Evaluate JavaScript in the tab you are driving and return the result |
|
|
92
|
+
|
|
93
|
+
### Tabs
|
|
94
|
+
| Tool | Description |
|
|
95
|
+
|------|-------------|
|
|
96
|
+
| `browser_list_tabs` | List the connected browser's open tabs |
|
|
97
|
+
| `browser_new_tab` | Open a new tab IN THE BACKGROUND and drive it — no focus stealing |
|
|
98
|
+
| `browser_switch_tab` | STEALS THE USER'S FOCUS: brings a tab to the front and drives it, by `tabId` (preferred) or `index` |
|
|
99
|
+
| `browser_select_tab` | TAKE OVER a tab the user already has open, WITHOUT focusing it — the tool for "pick up the testing I started" |
|
|
100
|
+
| `browser_close_tab` | Close a tab by `tabId` or `index` |
|
|
101
|
+
|
|
102
|
+
### Multi-IDE / clients
|
|
103
|
+
| Tool | Description |
|
|
104
|
+
|------|-------------|
|
|
105
|
+
| `browser_list_clients` | List every browser connected to the shared AutomateBrowser relay (e.g. Chrome and Edge when both have the extension connected), across all IDEs |
|
|
106
|
+
| `browser_select_client` | Choose which connected browser your subsequent tools act on |
|
|
107
|
+
| `browser_force_claim` | Forcibly take over a browser that another agent is currently driving, and make it active for your tools |
|
|
108
|
+
| `browser_release_client` | Release your claim on the browser you are currently driving so another agent can take it |
|
|
109
|
+
| `browser_status` | Diagnostics for the AutomateBrowser relay |
|
|
110
|
+
|
|
111
|
+
### Advanced (opt-in CDP)
|
|
112
|
+
Attach the Chrome debugger only when you need full-fidelity input or network bodies. Enable with
|
|
113
|
+
`browser_advanced_mode` first; a debugging banner shows only while it's attached.
|
|
114
|
+
|
|
115
|
+
| Tool | Description |
|
|
116
|
+
|------|-------------|
|
|
117
|
+
| `browser_advanced_mode` | Enable/disable opt-in debugger (CDP) mode for the tab you are driving |
|
|
118
|
+
| `browser_upload_file` | Set files on a file input (real upload) |
|
|
119
|
+
| `browser_get_network_request` | Get a network request's response BODY, status and headers by URL substring |
|
|
120
|
+
| `browser_perf_trace` | Record a performance trace (requires advanced/debugger mode) |
|
|
121
|
+
| `browser_emulate` | Emulate location, headers, colour scheme, viewport, user agent, network or CPU |
|
|
122
|
+
|
|
123
|
+
<!-- AUTO-GENERATED:tools END -->
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## Arguments and gotchas
|
|
128
|
+
|
|
129
|
+
`*` marks a required argument. Tools that navigate or act on an element also take `timeout` (ms); the
|
|
130
|
+
read-only ones — snapshot, read_page, get_html, find, screenshot, eval — do **not**, and reject it.
|
|
131
|
+
|
|
132
|
+
### Navigation & history
|
|
133
|
+
|
|
134
|
+
| Tool | Arguments | Gotcha |
|
|
135
|
+
|---|---|---|
|
|
136
|
+
| `browser_navigate` | `url`, `reload`, `ignoreCache`, `includeSnapshot`, `include`, `waitUntil`, `settleMs`, `initScript`, `handleBeforeUnload` | Runs in **your** tab, opening one in the background if you have none. `reload` re-requests the current page and takes no `url`; `ignoreCache` makes it a hard reload — the answer to "but I already fixed that". `initScript` and `handleBeforeUnload` need advanced mode, and `initScript` alone is refused with `EVAL_BLOCKED` where the operator has switched off agent-written JavaScript — drop it and the same call goes through. `settled` describes the navigation you asked for, never the page you were leaving; with `initScript` in play, `waitUntil: "none"` still waits for the new document, because a script torn down before then would never run. If the tab is not where you asked it to go, the reply opens with `Did NOT reach <url>` — including inside the snapshot reply, where the snapshot below it is then the OLD page and every ref in it belongs to that page. |
|
|
137
|
+
| `browser_go_back` | `waitUntil`, `settleMs` | History is per tab, so this is your tab's history, not the user's browsing. |
|
|
138
|
+
| `browser_go_forward` | `waitUntil`, `settleMs` | Silently does nothing if there is no forward entry — about a second of waiting, then an unsettled result and the same url. |
|
|
139
|
+
|
|
140
|
+
### Snapshot & interaction
|
|
141
|
+
|
|
142
|
+
| Tool | Arguments | Gotcha |
|
|
143
|
+
|---|---|---|
|
|
144
|
+
| `browser_snapshot` | `verbose`, `filePath` | The **most expensive call in the set**. Read it once to learn the page, then use `browser_find`. `filePath` keeps a big one out of your context. An `<iframe>` in the tree is only a marker — **every** frame's contents come below under their own `- frame <url>` heading with `fN:` refs, same-origin ones included, capped at 10 frames. |
|
|
145
|
+
| `browser_click` | `element`, `ref`, `x`, `y`, `dblClick`, `include`, `includeSnapshot`, `waitUntil`, `settleMs` | Pass a ref **or** coordinates, never both — giving both is refused before it runs. The coordinate form reports what was actually under the point. |
|
|
146
|
+
| `browser_hover` | `element*`, `ref*`, `includeSnapshot` | For menus that only open on hover. Nothing "sticks" — the next action may move the pointer. |
|
|
147
|
+
| `browser_type` | `element*`, `ref*`, `text*`, `submit*`, `include`, `includeSnapshot`, `waitUntil`, `settleMs` | `submit` is **required**: decide explicitly whether Enter is pressed. Use `browser_clear` first rather than typing over existing content. |
|
|
148
|
+
| `browser_select_option` | `element*`, `ref*`, `values*`, `includeSnapshot` | `values` is an array even for a single option, and matches by visible label or value. |
|
|
149
|
+
| `browser_drag` | `startElement*`, `startRef*`, `endElement*`, `endRef*`, `includeSnapshot` | Both ends need a description as well as a ref. Both ends must also be in the **same frame** — a cross-frame drag is refused, not attempted. Some drag libraries need a real pointer sequence — if it does nothing, try advanced mode. |
|
|
150
|
+
|
|
151
|
+
### Input & timing
|
|
152
|
+
|
|
153
|
+
| Tool | Arguments | Gotcha |
|
|
154
|
+
|---|---|---|
|
|
155
|
+
| `browser_press_key` | `key*`, `waitUntil`, `settleMs` | Goes to the page, not to an element — focus something first. Combos are `"Control+A"`, `"Shift+Tab"`. |
|
|
156
|
+
| `browser_wait` | `time*` | A blind sleep, in **seconds**. Last resort: either too short and flaky or too long and slow. Prefer `browser_wait_for`. |
|
|
157
|
+
| `browser_wait_for` | `selector`, `text`, `urlPattern`, `state`, `timeoutMs` | The right tool when a site swaps content **without** navigating. Waits on a real condition instead of a guess. |
|
|
158
|
+
|
|
159
|
+
### Reading content
|
|
160
|
+
|
|
161
|
+
| Tool | Arguments | Gotcha |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| `browser_read_page` | `format`, `maxLength` | The default answer to "what does this page say". `format: "markdown"` keeps headings and links; plain text is cheaper. |
|
|
164
|
+
| `browser_get_html` | `ref`, `maxLength` | **Always pass a `ref`.** Whole-page HTML is almost never what you want and is enormous. |
|
|
165
|
+
| `browser_find` | `text`, `role`, `selector`, `max` | Far cheaper than a snapshot and returns usable refs. Set `max` — an unbounded match on a big page is not the saving you wanted. |
|
|
166
|
+
|
|
167
|
+
### Page-declared tools
|
|
168
|
+
|
|
169
|
+
| Tool | Arguments | Gotcha |
|
|
170
|
+
|---|---|---|
|
|
171
|
+
| `browser_page_tools` | `action*`, `name`, `args` | **Expect an empty list.** Almost no live site declares tools yet, and an empty answer is a fact about the page, not a failure — do not retry it, and do not let it stop you clicking. `args` is a JSON object **string**, not an object. A result that will not encode as JSON comes back as a note instead; reach for `browser_eval` and `window.__dtmcp.executeTool` if you need the live value. |
|
|
172
|
+
|
|
173
|
+
### Forms & scrolling
|
|
174
|
+
|
|
175
|
+
| Tool | Arguments | Gotcha |
|
|
176
|
+
|---|---|---|
|
|
177
|
+
| `browser_fill_form` | `fields*` | Fill **every** field in one call. Field-by-field clicking and typing is slower and far more brittle. Submit separately. **May mix frames** — the only ref-taking tool that may — and fills in the order you pass, so a form spanning an embedded widget goes in one call. Nothing is written unless every ref parses; read `errors` for per-field results, which name the refs you passed. **Some-but-not-all is `outcome: "partial"`, not an error** — `isError` is set only when NOTHING landed, so read `Filled n/total` rather than the flag. Checkboxes and radios take **`"true"` / `"false"` only** — anything else is refused per field rather than guessed at; choose one option of a group by setting that option's own ref to `"true"`. |
|
|
178
|
+
| `browser_clear` | `ref*` | Empties an input properly. Typing over existing content is how you end up with `oldnew`. Follows the same frame rules as `browser_fill_form`, and reaches inside a shadow root. |
|
|
179
|
+
| `browser_scroll` | `ref`, `to`, `dx`, `dy` | On an infinite list, content below the fold may not be in the DOM at all — scroll, read, repeat. |
|
|
180
|
+
|
|
181
|
+
### State: cookies, storage, network, downloads, dialogs
|
|
182
|
+
|
|
183
|
+
| Tool | Arguments | Gotcha |
|
|
184
|
+
|---|---|---|
|
|
185
|
+
| `browser_get_cookies` | `name`, `revealValues` | Values are **redacted by default**. Scoped to your tab's URL, so navigate to the origin first. |
|
|
186
|
+
| `browser_set_cookie` | `name*`, `value*`, `path`, `secure`, `httpOnly`, `expirationDate`, `sameSite` | Only for a session you were explicitly given. Never harvest one. |
|
|
187
|
+
| `browser_storage` | `action*`, `area`, `key`, `value`, `revealValues` | `sessionStorage` is **not** shared with a new tab; `localStorage` is. That difference explains most "why am I logged out in the new tab". |
|
|
188
|
+
| `browser_network_requests` | `limit`, `page`, `resourceTypes`, `includePreserved` | The request **list**, not bodies. **Paged — 50 newest per call**, and `page: 2` is *older*, not newer. `includePreserved` reaches back through a redirect. |
|
|
189
|
+
| `browser_handle_dialog` | `action`, `promptText` | An unanswered dialog **freezes the page**, so your next call hangs to its full timeout. A hang is a dialog until proven otherwise. |
|
|
190
|
+
| `browser_downloads` | `limit`, `wait`, `timeout` | Returns the **path on disk**, so you can read the file. `wait: true` avoids reading a half-written one. |
|
|
191
|
+
| `browser_proxy` | `mode`, `server`, `pacUrl`, `bypass`, `clear` | **Affects the whole browser**, not your tab — it changes the browsing of the human sharing it. Clear it when done. |
|
|
192
|
+
|
|
193
|
+
### Performance
|
|
194
|
+
|
|
195
|
+
| Tool | Arguments | Gotcha |
|
|
196
|
+
|---|---|---|
|
|
197
|
+
| `browser_perf_field_data` | `url*`, `formFactor` | Needs no browser, but **sends the URL to a Google API** and needs `AUTOMATE_BROWSER_CRUX_KEY`. The only outbound call this server makes. |
|
|
198
|
+
|
|
199
|
+
### Capture & evaluation
|
|
200
|
+
|
|
201
|
+
| Tool | Arguments | Gotcha |
|
|
202
|
+
|---|---|---|
|
|
203
|
+
| `browser_screenshot` | `format`, `ref`, `filePath`, `quality`, `fullPage`, `keepEnabled`, `frames`, `intervalMs` | Use `filePath` unless you must see it — an inlined image is one of the costliest things in a reply. On a background tab it attaches the debugger briefly (banner), and **refuses rather than returning the wrong tab's pixels** if it cannot. `frames` writes a strip to disk (needs `filePath`) — but a background tab yields **one frame every ~4s**, so switch to it first if you need motion. An **inline** image is downscaled to fit 1536x4096 device px and says so — never read coordinates off one that was; a `filePath` capture never is. A `ref` from a **same-origin** frame crops fine at any depth; one from a **cross-origin** frame is refused, because the frame's position in the tab cannot be measured from outside it — capture the viewport instead. |
|
|
204
|
+
| `browser_get_console_logs` | `includePreserved`, `page` | **Paged — 50 newest entries per call**, and `page: 2` is *older*, not newer. `includePreserved` returns the previous pages' logs — the answer to "it errored then redirected". |
|
|
205
|
+
| `browser_issues` | `limit`, `audit`, `page` | The **only** tool that sees failures with no console error: CSP, dropped third-party cookies, mixed content, CORS. Reach for it on "works by hand, not here". `audit: "a11y"` switches it to an axe-core accessibility audit instead — snapshot FIRST so findings carry refs, page through with `page` (`limit` sets the page size), and never report "0 violations" as "accessible": automated rules catch about a third of real barriers. |
|
|
206
|
+
| `browser_eval` | `expression`, `function`, `args`, `filePath`, `dialogAction` | Use `function` + `args` of refs for anything structured. Return **JSON-serialisable** values — DOM nodes do not survive. Passing both forms, or `args` without `function`, is refused. An operator can switch this tool off entirely (`EVAL_BLOCKED`, even for a read) — if that happens, read with `browser_snapshot` or `browser_find` and stop looking for a way around it. |
|
|
207
|
+
|
|
208
|
+
### Tabs
|
|
209
|
+
|
|
210
|
+
| Tool | Arguments | Gotcha |
|
|
211
|
+
|---|---|---|
|
|
212
|
+
| `browser_list_tabs` | (none) | Pure discovery — claims nothing, so it never locks another agent out. |
|
|
213
|
+
| `browser_new_tab` | `url`, `active`, `incognito` | Opens in the **background** and is adopted automatically. `active: true` steals the user's focus — only on request. `incognito: true` gives a clean logged-out session, but needs a setting only a person can turn on; on `INCOGNITO_BLOCKED`, ask them rather than retrying. |
|
|
214
|
+
| `browser_switch_tab` | `tabId`, `index` | The **only** tool that takes the user's focus. For "show me", nothing else. |
|
|
215
|
+
| `browser_select_tab` | `tabId`, `index`, `url`, `title` | Adopt a tab **without** focusing it. Prefer `url`/`title` over `index`, which shifts as tabs open and close. |
|
|
216
|
+
| `browser_close_tab` | `tabId`, `index` | Do not close a tab you did not open. Release already closes yours. |
|
|
217
|
+
|
|
218
|
+
### Multi-IDE / clients
|
|
219
|
+
|
|
220
|
+
| Tool | Arguments | Gotcha |
|
|
221
|
+
|---|---|---|
|
|
222
|
+
| `browser_list_clients` | (none) | Shows every connected browser **and who is driving each tab**. |
|
|
223
|
+
| `browser_select_client` | `id`, `browser`, `label`, `force` | Per-agent and sticky; it does not change what other agents see. Selecting does **not** claim — acting does. |
|
|
224
|
+
| `browser_force_claim` | `id`, `browser`, `label`, `force` | Steals at **whole-browser** level and tells the other agent immediately, mid-task. Last resort. |
|
|
225
|
+
| `browser_release_client` | (none) | Frees the browser **and closes every tab you opened**. A tab you adopted from the user is left alone. Call it when genuinely done. |
|
|
226
|
+
| `browser_status` | (none) | The one call that explains everything else. Call it before guessing. Its `link:` line names the connection state in capitals — `CONNECTED`, `WAITING` (on the relay, no browser has joined), `RETRYING` (it recovers on its own, with the countdown), `CONNECTING`, `STOPPED` (nothing is being retried, so waiting is wrong). |
|
|
227
|
+
|
|
228
|
+
### Advanced (opt-in CDP)
|
|
229
|
+
|
|
230
|
+
| Tool | Arguments | Gotcha |
|
|
231
|
+
|---|---|---|
|
|
232
|
+
| `browser_advanced_mode` | `enable` | Attaches the debugger and shows a banner. Turn it off when done. Omit `enable` to just ask whether it is on. **There is no `acceptInsecureCerts` any more** — it never worked (Chrome does not expose the domain it needed to extensions) and was deleted on 2026-09-16; passing it is refused by name. For a bad certificate, ask the user to click through the warning once by hand, or to start the browser with `--ignore-certificate-errors`. |
|
|
233
|
+
| `browser_upload_file` | `ref*`, `filePaths*`, `keepEnabled` | Needs advanced mode. The `ref` must be the file input itself, not a styled wrapper around it. A **cross-origin** frame's input is refused and says the frame is why — no fresh snapshot will help, because the debugger session does not reach into another origin. Same-origin frames (at any depth) and shadow roots work. |
|
|
234
|
+
| `browser_get_network_request` | `url`, `requestId`, `maxLength`, `keepEnabled`, `revealValues` | The response **body**, plus both header sets with credential values **redacted** — `revealValues` opts out. When a URL matches several, it returns the newest and lists the rest so you can pick by `requestId`. |
|
|
235
|
+
| `browser_perf_trace` | `action*`, `categories`, `filePath`, `reload`, `autoStop`, `durationMs` | `action: "analyze"` re-reads a saved trace with **no browser at all**, and `action: "memory"` samples the JS heap with **no advanced mode**. Recording needs it. `durationMs` (memory) is 1000-30000 and is refused, not clamped, outside that. A rising heap is not a leak, and there is **no heap snapshot** — Chrome blocks the domain for extensions. |
|
|
236
|
+
| `browser_emulate` | `geolocation`, `headers`, `colorScheme`, `viewport`, `mobile`, `userAgent`, `network`, `cpuThrottling`, `clear` | Per tab, and gone with the tab. Some options need advanced mode and say so. `clear` takes them back off. |
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# When it will not drive the browser
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
| Section | What it answers |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [Always start here](#always-start-here) | The one call that explains most of this page |
|
|
8
|
+
| [No connection](#no-connection-to-browser-extension) | Nothing connected, empty browser list |
|
|
9
|
+
| [A call that hangs](#a-call-that-hangs-then-times-out) | Timeouts |
|
|
10
|
+
| [Success but nothing happened](#an-action-reports-success-but-nothing-happened) | Silent failures |
|
|
11
|
+
| [Wrong tab](#it-acted-on-the-wrong-tab) | Targeting |
|
|
12
|
+
| [Error codes](#the-error-codes) | What each means and the tool that fixes it |
|
|
13
|
+
| [Not faults](#three-things-that-are-not-faults) | Expected behaviour that looks broken |
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Always start here
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
browser_status
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
One round-trip, and it answers most of this page: whether you are on the relay and whether its version
|
|
24
|
+
matches yours, which browsers are connected, which tab each is on, who is driving them, your own agent
|
|
25
|
+
name, and whether an old self-hosting server is squatting the port and hiding browsers.
|
|
26
|
+
|
|
27
|
+
Call it before guessing. Call it whenever a claim error names an agent you did not expect.
|
|
28
|
+
|
|
29
|
+
### The `link:` line — whether it is coming back
|
|
30
|
+
|
|
31
|
+
Its second line begins `link:` and the first word is the state, in capitals. Read that before you
|
|
32
|
+
read anything else on this page, because it decides whether waiting is the right move at all:
|
|
33
|
+
|
|
34
|
+
| Line starts | Meaning | Do |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| `link: CONNECTED` | on the relay, with a browser | carry on |
|
|
37
|
+
| `link: WAITING` | on the relay, **no browser has joined** | ask the user to open a browser with the extension — the server is fine, and nothing on this page applies |
|
|
38
|
+
| `link: RETRYING` | reconnecting on its own; the line gives the attempt count, the countdown and the last failure | wait and retry the call — do not restart anything |
|
|
39
|
+
| `link: CONNECTING` | first connect in flight | wait |
|
|
40
|
+
| `link: STOPPED` | shutting down, **nothing is being retried** | stop waiting; this needs the server restarted |
|
|
41
|
+
|
|
42
|
+
`WAITING` and `STOPPED` are the two that look like a broken tool and are not. `WAITING` means the
|
|
43
|
+
half that is missing is a browser, not the connection. `STOPPED` is the only state where waiting is
|
|
44
|
+
always wrong.
|
|
45
|
+
|
|
46
|
+
## "No connection to browser extension"
|
|
47
|
+
|
|
48
|
+
In the order worth trying:
|
|
49
|
+
|
|
50
|
+
1. **Is the extension loaded and enabled?** Open the browser's extensions page. After an update that
|
|
51
|
+
added permissions, the browser **disables it until the user re-approves** — the single most common
|
|
52
|
+
cause after an upgrade, and it looks exactly like a crash.
|
|
53
|
+
2. **Is it asleep?** The extension's background worker is evicted after about 30 seconds idle. It is
|
|
54
|
+
built to revive itself — a repeating alarm plus the browser's own startup hooks — and **opening
|
|
55
|
+
the extension's popup wakes it instantly**.
|
|
56
|
+
**Do not assume it always revives.** Measured 2026-09-01: after an idle spell a call failed
|
|
57
|
+
*immediately* with "No connection to browser extension", and `browser_status` then reported no
|
|
58
|
+
browsers at all across five calls spanning several minutes, against a relay that was up and
|
|
59
|
+
healthy the whole time. It came back only when a person clicked the extension icon.
|
|
60
|
+
So: **retry two or three times across about a minute.** If `browser_status` still lists no
|
|
61
|
+
browser, stop and tell the user to click the AutomateBrowser icon in their toolbar, or reload the
|
|
62
|
+
extension. Retrying past that point cannot help — nothing an agent can call reaches a worker that
|
|
63
|
+
is not running.
|
|
64
|
+
**Say what happened, with times.** The relay writes `~/.automate-browser/automate-browser-relay.log`,
|
|
65
|
+
one timestamped line per `browser connected` / `browser removed`. If you can read files, the tail of
|
|
66
|
+
it says exactly when the browser dropped and whether it ever returned by itself — which is the
|
|
67
|
+
difference between "asleep and slow" and a fault worth reporting. It lived in the temp directory
|
|
68
|
+
until 2026-09-02, where it was swept daily, so older sessions have no history to read.
|
|
69
|
+
3. **Is the tab a normal page?** `chrome://`, the extension store and PDF viewer pages refuse
|
|
70
|
+
automation entirely — `RESTRICTED_PAGE`. Open an ordinary `http(s)` page.
|
|
71
|
+
4. **Version mismatch.** If `browser_status` warns the relay is a different build: the first editor to
|
|
72
|
+
start owns the shared relay. Killing that editor alone does not help — the next one respawns
|
|
73
|
+
whichever build asks first. Point every editor at the same build, close them all so the relay exits
|
|
74
|
+
on its own, then reopen.
|
|
75
|
+
|
|
76
|
+
This is a user-visible situation. Say which of the four you think it is rather than retrying silently.
|
|
77
|
+
|
|
78
|
+
## A call that hangs, then times out
|
|
79
|
+
|
|
80
|
+
Almost always a **dialog**. An `alert`, `confirm`, `prompt` or a "Leave site?" prompt freezes the
|
|
81
|
+
page, so anything injected into it never runs and the call waits out its whole budget.
|
|
82
|
+
`browser_handle_dialog` clears it. Tools that touch the page name this cause in the timeout message.
|
|
83
|
+
|
|
84
|
+
## An action reports success but nothing happened
|
|
85
|
+
|
|
86
|
+
The action landed on the wrong thing, or the page refused it silently. In order of cost:
|
|
87
|
+
|
|
88
|
+
- `browser_click { ..., include: "console, network" }` — the new console lines and the request log come
|
|
89
|
+
back with the click itself.
|
|
90
|
+
- `browser_issues` — the **only** tool that sees failures producing no console error at all: blocked
|
|
91
|
+
content-security policy, dropped third-party cookies, mixed content, CORS.
|
|
92
|
+
- A fresh `browser_snapshot`. If the page re-rendered, your ref pointed somewhere else.
|
|
93
|
+
|
|
94
|
+
## It acted on the wrong tab
|
|
95
|
+
|
|
96
|
+
You drive a tab you own — one you opened, or one you adopted. If an action landed somewhere
|
|
97
|
+
unexpected, name your target explicitly:
|
|
98
|
+
|
|
99
|
+
```
|
|
100
|
+
browser_select_tab { url: "localhost:3000" }
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Prefer a `url` or `title` substring over an `index`, which shifts whenever any tab opens or closes.
|
|
104
|
+
See [tabs-and-multi-agent.md](tabs-and-multi-agent.md) for the full ownership model.
|
|
105
|
+
|
|
106
|
+
## The error codes
|
|
107
|
+
|
|
108
|
+
| Code | Meaning | Do this |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| `NO_BROWSER` | Nothing connected, or the link dropped mid-call | The list above. Retryable **unless** the message says the action **may have taken effect** — that means the request went out and the reply was lost, so check the page with `browser_snapshot` before repeating anything that changes it |
|
|
111
|
+
| `TAB_CLAIMED` | Another agent is driving that tab | Use another tab, or `browser_force_claim` |
|
|
112
|
+
| `LEASE_LOST` | You were displaced mid-task | Stop; your refs are stale. Pick a tab again |
|
|
113
|
+
| `TAB_GONE` | The tab closed, or none could be opened | `browser_list_tabs`, then `browser_select_tab` |
|
|
114
|
+
| `STALE_REF` | The page re-rendered under you | `browser_snapshot`, then reuse the new ref |
|
|
115
|
+
| `BAD_ARGS` | The arguments could not be understood, so **nothing ran** | Fix the call and send it again. The commonest cause is refs from two different frames in one call, or a half-written `f3:` prefix — see [page-interaction](./page-interaction.md) |
|
|
116
|
+
| `NOT_ACTIONABLE` | Hidden, disabled, moving, or covered | The message names the failing check and what covered it |
|
|
117
|
+
| `RESTRICTED_PAGE` | Browser-internal page | Open a normal page |
|
|
118
|
+
| `ADVANCED_MODE_REQUIRED` | Needs the debugger | `browser_advanced_mode { enable: true }` |
|
|
119
|
+
| `CAPTURE_STALLED` | Chrome stopped drawing a tab nobody is looking at, so the screenshot got no frame | **Just call it again** — this one is marked retryable, and a repeat usually works. `browser_switch_tab` always captures, at the cost of the user's focus |
|
|
120
|
+
| `ORIGIN_BLOCKED` / `READ_ONLY` | A safety setting refused it — or, for `ORIGIN_BLOCKED` alone, the tab moved between the check and the action | `browser_status` prints the policy; a refusal it explains is the operator's choice, not a bug. A message saying the target **moved between the safety check and the action** is the other case: nothing was sent, so call it again |
|
|
121
|
+
| `EVAL_BLOCKED` | Running JavaScript you wrote is switched off for this server | **Do not retry, and do not look for another way in — there isn't one.** Read the page with `browser_snapshot`, `browser_find` or `browser_read_page` instead, and drive it by clicking. Only `browser_eval` and `browser_navigate`'s `initScript` are refused; everything else works normally |
|
|
122
|
+
| `CSP_BLOCKED`, `CORS_BLOCKED`, `MIXED_CONTENT`, `THIRD_PARTY_COOKIE_BLOCKED`, `DEPRECATED_API` | Browser-detected issues | Surfaced by `browser_issues`; these are page bugs, not tool bugs |
|
|
123
|
+
|
|
124
|
+
Every code arrives as `CODE: message`, with a `Recover: call <tool>` line when there is a next step.
|
|
125
|
+
**Read the code, not the prose** — the prose may be reworded, the code will not.
|
|
126
|
+
|
|
127
|
+
## Three things that are not faults
|
|
128
|
+
|
|
129
|
+
- **A slow first call after idle** is the background worker waking up. Expected — *as long as it
|
|
130
|
+
then works*. A first call that fails, and keeps failing, is a real fault and not patience owed;
|
|
131
|
+
see "No connection to browser extension" above for how long to keep trying.
|
|
132
|
+
- **A "being debugged" banner** appears while advanced mode is attached, for a full-page screenshot,
|
|
133
|
+
or for a screenshot of a background tab. It detaches again afterwards. See
|
|
134
|
+
[capture-and-diagnostics.md](capture-and-diagnostics.md).
|
|
135
|
+
- **A new background tab appearing on your first action** is the server giving you a tab of your own,
|
|
136
|
+
so you never drive the user's. That is the design, not a stray tab.
|