barebrowse 0.25.0 → 0.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +58 -0
- package/README.md +32 -1
- package/barebrowse.context.md +10 -5
- package/cli.js +24 -6
- package/commands/barebrowse/SKILL.md +142 -0
- package/commands/barebrowse.md +141 -0
- package/mcp-server.js +78 -9
- package/package.json +2 -1
- package/src/aria.js +5 -0
- package/src/ax-snapshot.js +1 -0
- package/src/bareagent.js +5 -3
- package/src/blocklist-firefox.js +12 -4
- package/src/daemon.js +7 -4
- package/src/data.js +24 -9
- package/src/firefox-page.js +71 -12
- package/src/index.js +127 -24
- package/src/links.js +46 -0
- package/src/navigation.js +54 -0
- package/src/prune.js +11 -3
- package/src/site-lock.js +184 -0
- package/types/aria.d.ts +0 -6
- package/types/blocklist-firefox.d.ts +2 -1
- package/types/data.d.ts +6 -1
- package/types/firefox-page.d.ts +17 -1
- package/types/index.d.ts +22 -3
- package/types/links.d.ts +28 -0
- package/types/navigation.d.ts +56 -0
- package/types/prune.d.ts +2 -0
- package/types/site-lock.d.ts +70 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,63 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.27.0] - 2026-10-30
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Site lock (`allowHosts`).** `connect({ allowHosts: ['a.com', '*.b.org'] })`
|
|
8
|
+
and `browse(url, { allowHosts })` confine top-level navigation to a host list
|
|
9
|
+
on both engines: `goto()`, redirect hops, clicked links, JS jumps, popups and
|
|
10
|
+
new tabs. A blocked `goto()` rejects with `err.code === 'NAV_BLOCKED'` and
|
|
11
|
+
`err.host`; the page stays where it was. A blocked in-page jump shows as
|
|
12
|
+
`[blocked navigation to <host>]` on the next `snapshot()`. Not locked:
|
|
13
|
+
subresources, iframes, WebSocket. Attach mode (`port`) throws. CLI:
|
|
14
|
+
`--allow-hosts=a.com,*.b.org` on `open` and `browse`.
|
|
15
|
+
- **MCP tool annotations and access tiers.** Every tool has `readOnlyHint`,
|
|
16
|
+
`destructiveHint`, `openWorldHint` (and `idempotentHint` where it applies).
|
|
17
|
+
`BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier, and a
|
|
18
|
+
hidden tool answers `Unknown tool`. `BAREBROWSE_ALLOW=a.com,*.b.org` sets the
|
|
19
|
+
site lock. An invalid value of either stops the server at start. The
|
|
20
|
+
`capabilities` tool and the server instructions report `tier` and
|
|
21
|
+
`allowHosts`.
|
|
22
|
+
|
|
23
|
+
## [0.26.0] - 2026-10-27
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- **Opt-in link URLs in snapshots.** `links: true` adds each link's absolute URL
|
|
28
|
+
after its name, in every mode: `link "name" <absolute-url> [ref=N]`. Headings
|
|
29
|
+
that wrap a link keep its URL, and in data mode so do group items and table
|
|
30
|
+
cells. `javascript:` links and same-page `#fragment` links are skipped, and
|
|
31
|
+
spaces and `<>` in a URL are percent-encoded. It is available as
|
|
32
|
+
`snapshot({ links: true })`, in `browse()` and `connect()`, as the `links`
|
|
33
|
+
argument of the MCP `browse` and `snapshot` tools, as `--links` on
|
|
34
|
+
`barebrowse snapshot` and `barebrowse browse`, and in the bareagent tools.
|
|
35
|
+
Off by default, and the output is unchanged when it is off. The data-mode
|
|
36
|
+
legend gains a links note only when links are on.
|
|
37
|
+
- **`goto()` now reports where it landed.** It resolves `{ url, status }`: the
|
|
38
|
+
final URL after redirects and the main document's HTTP status, on both
|
|
39
|
+
engines. `page.lastNavigation` holds the same value and is reset to `null`
|
|
40
|
+
after `goBack()`, `goForward()`, `reload()` and a tab switch. After a non-2xx
|
|
41
|
+
`goto()`, the next `snapshot()` and the `browse()` output start with
|
|
42
|
+
`[HTTP n — the page may be an error page]`. An HTTP 4xx or 5xx is reported,
|
|
43
|
+
never thrown, including an error response with an empty body. The MCP and
|
|
44
|
+
CLI `goto` print `ok (HTTP n)` for a non-2xx page.
|
|
45
|
+
- **Network failures reject with a typed error.** An unreachable host (DNS
|
|
46
|
+
failure, connection refused, TLS error and the like) rejects with
|
|
47
|
+
`err.code === 'NAV_FAILED'` and the browser's raw message in `err.reason`.
|
|
48
|
+
Behaviour change: on Chromium, `goto()` and `browse()` to an unreachable host
|
|
49
|
+
used to resolve with the browser's own error page; they now reject. Firefox
|
|
50
|
+
already rejected, with an untyped error; it is now the same typed error.
|
|
51
|
+
|
|
52
|
+
### Fixed
|
|
53
|
+
|
|
54
|
+
- **A download no longer makes `goto()` wait 30s and time out on Chromium.**
|
|
55
|
+
Navigating to a URL that turns into a download, or to a 204 response, returns
|
|
56
|
+
at once. A 204 reports status 204 on both engines.
|
|
57
|
+
- **`barebrowse install --skill` works from an npm install.** The skill file
|
|
58
|
+
was missing from the published package, so the command failed outside a
|
|
59
|
+
checkout.
|
|
60
|
+
|
|
3
61
|
## [0.25.0] - 2026-10-26
|
|
4
62
|
|
|
5
63
|
### Added
|
package/README.md
CHANGED
|
@@ -96,7 +96,7 @@ Or manually add to your config (`claude_desktop_config.json`, `.cursor/mcp.json`
|
|
|
96
96
|
}
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (reports the active engine — chromium or firefox — and what the session can do, plus a `cookies` field naming the cookies injected on the last goto — names only, never values; no side effects), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so call `goto` after). Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
|
|
99
|
+
MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (reports the active engine — chromium or firefox — and what the session can do, plus a `cookies` field naming the cookies injected on the last goto — names only, never values; no side effects), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so call `goto` after). Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. `BAREBROWSE_TIER` and `BAREBROWSE_ALLOW` restrict the tool set and the reachable hosts (see Site lock and MCP access tiers below). Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
|
|
100
100
|
|
|
101
101
|
**Idle release.** An MCP server can outlive its conversation, so it closes its browser after `BAREBROWSE_IDLE_MINUTES` (default `15`, `0` = never, blank/invalid = default, fractional allowed, capped at ~24.8 days) with no tool call in flight. The next call opens a fresh browser (a new job, not a resume — a page left open, e.g. a solved captcha in the headed window, is gone). A transient browser error also closes the broken browser rather than orphaning it; an ordinary `wait_for` timeout does not reset the page.
|
|
102
102
|
|
|
@@ -104,6 +104,8 @@ MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `
|
|
|
104
104
|
|
|
105
105
|
`browse` and `snapshot` accept `pruneMode: 'act'|'read'|'data'`. `act` (default) keeps interactive elements — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If act-mode collapses a content-heavy page near-totally, the snapshot includes a `hint: …` line suggesting `pruneMode='read'` so the agent doesn't bail to a separate HTTP fetch. `data` is one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a one-line legend (`# data mode: …`): table rows read `row "<label>": <column>: <value> · …`, groups read `<label>: a, b, c`, and a title line followed by detail lines is one block, up to the next title line. It covers the main landmark only (nav, header and pagination outside `<main>` are dropped): use `act` for menus and page controls.
|
|
106
106
|
|
|
107
|
+
`links: true` (opt-in, `snapshot({ links: true })`, `browse(url, { links: true })`, or the `connect({ links: true })` default; MCP/bareagent `links` param, CLI `snapshot --links` / `browse <url> --links`) adds each link's absolute URL right after its quoted name: `- link "Fix login bug" <https://github.com/o/r/issues/42> [ref=12]`. Works in every mode; a heading that wraps a link keeps the URL on its own line (`heading "title" <url> [level=3] [ref=N]`); data mode also puts it after link cells in table rows and each item of a collapsed group (`Labels: bug <url> [ref=1], docs <url> [ref=2]`) and its legend gains ` Links: "name" <url>.`. `javascript:` and same-page `#fragment` links get no URL, every link line, row cell and group item carries its own URL (nothing is deduped across them, even when two links share a URL), and URLs never contain a raw space, `<` or `>`. Off by default: output is unchanged and costs nothing (about +23% chars on a GitHub issue list in data mode).
|
|
108
|
+
|
|
107
109
|
`npx barebrowse doctor` reports the runtime environment first — default engine/mode, installed Chromium/Firefox binaries, cookie sources, and any live session's engine + capabilities — then scans every known MCP config location and flags scope conflicts. `npx barebrowse install --force` overwrites an existing entry pointing at a different endpoint.
|
|
108
110
|
|
|
109
111
|
### 3. Library -- for agentic automation
|
|
@@ -138,6 +140,25 @@ const snap = await page.snapshot();
|
|
|
138
140
|
await page.close(); // closes only the tab barebrowse opened — your browser keeps running
|
|
139
141
|
```
|
|
140
142
|
|
|
143
|
+
### Did the navigation work?
|
|
144
|
+
|
|
145
|
+
`goto()` resolves to `{ url, status }` — the final URL after redirects and the
|
|
146
|
+
main document's HTTP status (`0` when no HTTP response was observed, e.g. an aborted navigation).
|
|
147
|
+
`page.lastNavigation` returns the same object (`null` before any goto, and reset
|
|
148
|
+
to `null` by `goBack`/`goForward`/`reload`/`switchTab` — status is reported for
|
|
149
|
+
`goto()` only, so those never leave a stale warning). A
|
|
150
|
+
non-2xx page still loads, and its snapshot (and `browse()` output) carries a
|
|
151
|
+
`[HTTP 404 — the page may be an error page]` line. If the browser cannot
|
|
152
|
+
reach a document at all (DNS failure, connection refused, TLS error) `goto()`
|
|
153
|
+
and `browse()` reject with an Error whose `code` is `'NAV_FAILED'` and `reason`
|
|
154
|
+
the browser's raw string (`net::ERR_NAME_NOT_RESOLVED`, `NS_ERROR_UNKNOWN_HOST`).
|
|
155
|
+
A download or a 204 is not a failure. Through MCP and the CLI, `goto` prints
|
|
156
|
+
`ok` for 2xx and `ok (HTTP 404)` otherwise.
|
|
157
|
+
|
|
158
|
+
```js
|
|
159
|
+
const { url, status } = await page.goto('https://example.com/gone');
|
|
160
|
+
```
|
|
161
|
+
|
|
141
162
|
### Firefox (WebDriver BiDi)
|
|
142
163
|
|
|
143
164
|
CDP is deprecated in Firefox, so barebrowse drives it over the W3C WebDriver
|
|
@@ -190,8 +211,18 @@ barebrowse hands an autonomous — and therefore prompt-injectable — agent an
|
|
|
190
211
|
- **CLI daemon authenticated** with a per-session token (loopback alone isn't an authorization boundary); snapshots and saved state are written owner-only (`0600`).
|
|
191
212
|
- **Opt-in hardening** for stricter deployments: `blockPrivateNetwork` (SSRF guard for loopback/RFC-1918/cloud-metadata) and `uploadDir` (confine `upload()` to one directory). Both available on the library, MCP, bareagent, and CLI (`--block-private-network`, `--upload-dir`).
|
|
192
213
|
|
|
214
|
+
- **Site lock** (`allowHosts`): confine top-level navigation to a list of hosts. See below.
|
|
215
|
+
|
|
193
216
|
See `barebrowse.context.md` and the PRD's "Security Model & Safe Defaults" for the full rationale.
|
|
194
217
|
|
|
218
|
+
### Site lock and MCP access tiers
|
|
219
|
+
|
|
220
|
+
`allowHosts: ['example.com', '*.docs.example.org']` on `connect()` and `browse()` (CLI `--allow-hosts=a.com,*.b.org` on `open` and `browse`; MCP `BAREBROWSE_ALLOW=a.com,*.b.org`) locks the top frame to those hosts, on both engines. An entry is `host` (exactly that host) or `*.host` (its subdomains, not `host` itself); a scheme, port or path in an entry is an error. A `goto()` to another host, or one that redirects there, rejects with `err.code === 'NAV_BLOCKED'` (`err.host`) and the page stays where it was. A clicked link, a JS jump or a `window.open` popup to another host never loads; the next `snapshot()` carries `[blocked navigation to <host>]`. New tabs are locked too. `connect({ port, allowHosts })` (attach mode) throws: the lock is browser-wide and would reach your own tabs.
|
|
221
|
+
|
|
222
|
+
Not locked, by design: subresources (images, scripts, CSS, XHR/fetch), iframes and WebSocket. The lock stops an agent being steered to another site, not a page from loading its own assets. The MCP `eval` tool (tier `rwx`) can still `fetch()` anywhere.
|
|
223
|
+
|
|
224
|
+
MCP only: `BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier. `r` reads (`browse`, `goto`, `back`, `forward`, `reload`, `scroll`, `snapshot`, `readable`, `screenshot`, `pdf`, `wait_for`, `tabs`, `capabilities`, `assess`); `rw` adds `click`, `type`, `press`, `select`, `drag`, `hover`, `switch_engine`; `rwx` adds `upload` and `eval`. A hidden tool answers `Unknown tool`. Every tool also carries MCP `annotations` (`readOnlyHint`, `destructiveHint`, `openWorldHint`, `idempotentHint`). An invalid `BAREBROWSE_TIER` or `BAREBROWSE_ALLOW` stops the server at start; it never falls back to `rwx` or to no lock. `capabilities` and the server instructions report the tier and the allowed hosts.
|
|
225
|
+
|
|
195
226
|
## What the agent sees
|
|
196
227
|
|
|
197
228
|
Raw ARIA output from a page is noisy -- decorative wrappers, hidden elements, structural junk. The pruning pipeline (ported from [mcprune](https://github.com/hamr0/mcprune)) strips it down to what matters.
|
package/barebrowse.context.md
CHANGED
|
@@ -50,6 +50,7 @@ const snapshot = await browse('https://example.com', {
|
|
|
50
50
|
browser: 'firefox', // cookie source: 'firefox' | 'chromium' (auto-detected)
|
|
51
51
|
prune: true, // apply ARIA pruning (47-95% token reduction)
|
|
52
52
|
pruneMode: 'act', // 'act' (interactive elements) | 'read' (all content) | 'data' (one line per record)
|
|
53
|
+
links: false, // default for snapshot({ links }): link URLs after each link name
|
|
53
54
|
consent: true, // auto-dismiss cookie consent dialogs
|
|
54
55
|
blockAds: true, // block 128 ad/tracker URL patterns (default on for owned browsers)
|
|
55
56
|
blockUrls: [], // extra URL globs to block (merged with the default)
|
|
@@ -65,11 +66,12 @@ const snapshot = await browse('https://example.com', {
|
|
|
65
66
|
|
|
66
67
|
| Method | Args | Returns | Notes |
|
|
67
68
|
|---|---|---|---|
|
|
68
|
-
| `goto(url, timeout?)` | url: string, timeout: number (default 30000) |
|
|
69
|
+
| `goto(url, timeout?)` | url: string, timeout: number (default 30000) | `{ url, status }` | Navigate + wait for load + dismiss consent. Resolves to the final URL (after redirects) and the main document's HTTP status (`0` = no HTTP response observed, e.g. an aborted navigation). Non-2xx still resolves; `snapshot()` then carries `[HTTP 404 — the page may be an error page]`. Network failure (DNS, refused, TLS) rejects with `err.code === 'NAV_FAILED'` and `err.reason` (raw browser string, e.g. `net::ERR_NAME_NOT_RESOLVED` / `NS_ERROR_UNKNOWN_HOST`). A download or 204 (`ERR_ABORTED` / `NS_BINDING_ABORTED`) is not a failure and returns without waiting for a load event. `browse()` still returns a string (warning line prepended when non-2xx) and rejects with the same `NAV_FAILED`. With `allowHosts`, an off-list URL or redirect hop rejects with `err.code === 'NAV_BLOCKED'` (`err.host`) and the page stays put. |
|
|
70
|
+
| `lastNavigation` | -- | `{ url, status }` or null | Outcome of the latest `goto()`; null before any goto and while a goto is failing. Reset to null by `reload`/`goBack`/`goForward`/`switchTab` (status is reported for `goto()` only) |
|
|
69
71
|
| `goBack()` | -- | void | Navigate back in browser history |
|
|
70
72
|
| `goForward()` | -- | void | Navigate forward in browser history |
|
|
71
73
|
| `reload(opts?)` | { ignoreCache?: boolean, timeout?: number } | void | Reload the current page. Clears refMap (refs from pre-reload reject). |
|
|
72
|
-
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data' } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. |
|
|
74
|
+
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data', links?: boolean } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. `links: true` adds `<absolute-url>` after each link name (see below). |
|
|
73
75
|
| `readable()` | -- | object | Clean article text (Reader-View engine). `{ ok, title, byline, text, length, confidence: 'high'\|'low', readerable, hint? }` or `{ ok: false, hint }`. For *reading*, not interacting — see note below. |
|
|
74
76
|
| `click(ref)` | ref: string | void | Scroll into view + mouse press+release at center |
|
|
75
77
|
| `type(ref, text, opts?)` | ref: string, text: string, opts: { clear?, keyEvents? } | void | Focus + insert text. `clear: true` replaces existing. |
|
|
@@ -110,6 +112,7 @@ const snapshot = await browse('https://example.com', {
|
|
|
110
112
|
- `blockUrls: ['*://foo.com/*', ...]` — Extra glob patterns (CDP `Network.setBlockedURLs` format) to block in addition to the default. Merged with the default unless `blockAds: false`.
|
|
111
113
|
- `allowLocalUrls: true|false` — (v0.11.0) Default `false`: navigation to local-resource / browser-internal schemes (`file:`, `view-source:`, `chrome:`, `filesystem:`, `devtools:`, …) is **blocked** to stop a prompt-injected agent reading local files. `http`/`https`/`data`/`blob`/`about` are always allowed. Set `true` to permit local schemes.
|
|
112
114
|
- `blockPrivateNetwork: true|false` — (v0.11.0) Default `false`. When `true`, `goto()`/`browse()` refuse loopback / RFC-1918 / link-local / cloud-metadata (`169.254.169.254`) / `*.internal` hosts (SSRF guard). Off by default so localhost dev-server browsing works. Hostname-based — does not catch DNS names that resolve to private IPs.
|
|
115
|
+
- `allowHosts: ['example.com', '*.docs.example.org']` — Site lock (both engines, `connect()` and `browse()`; CLI `--allow-hosts=a,b` on `open`/`browse`; MCP `BAREBROWSE_ALLOW=a,b`). Top-level navigations (goto, redirect hops, clicked links, JS jumps, `window.open` popups, new tabs) to a host outside the list are failed in the browser and the page stays put. Entry = `host` (exact) or `*.host` (strict subdomains, not the apex); scheme/port/path/bare `*` throw. `goto()` rejects with `err.code === 'NAV_BLOCKED'` + `err.host`; in-page blocks add `[blocked navigation to <host>]` to the next `snapshot()`. Attach mode (`port`) throws. NOT locked: subresources, iframes, WebSocket (and MCP `eval`, which can `fetch()` anywhere).
|
|
113
116
|
- `uploadDir: '/abs/dir'` — (v0.11.0) Default unset (no restriction). When set, `upload()` rejects any file that does not resolve (symlinks included, via `realpath`) inside this directory — sandboxes the agent's file-upload capability.
|
|
114
117
|
|
|
115
118
|
## Snapshot format
|
|
@@ -266,7 +269,7 @@ barebrowse switch-engine firefox # Swap chromium/firefox in place (page/ta
|
|
|
266
269
|
barebrowse close # Kill daemon + browser
|
|
267
270
|
```
|
|
268
271
|
|
|
269
|
-
**Open flags:** `--mode=headless|headed|hybrid`, `--port=N` (attach to running browser), `--proxy=URL`, `--viewport=WxH`, `--storage-state=FILE`, `--download-path=DIR` (v0.9.0), `--no-cookies`, `--browser=firefox|chromium`, `--timeout=N`, `--block-private-network` (SSRF guard, v0.11.0), `--upload-dir=DIR` (upload sandbox, v0.11.0)
|
|
272
|
+
**Open flags:** `--mode=headless|headed|hybrid`, `--port=N` (attach to running browser), `--proxy=URL`, `--viewport=WxH`, `--storage-state=FILE`, `--download-path=DIR` (v0.9.0), `--no-cookies`, `--browser=firefox|chromium`, `--timeout=N`, `--block-private-network` (SSRF guard, v0.11.0), `--upload-dir=DIR` (upload sandbox, v0.11.0), `--allow-hosts=a.com,*.b.org` (site lock; also on `browse`)
|
|
270
273
|
|
|
271
274
|
Session lifecycle: `open` spawns a background daemon holding a `connect()` session. Subsequent commands POST to the daemon over HTTP (localhost). `close` shuts everything down. JS dialogs (alert/confirm/prompt) are auto-dismissed and logged.
|
|
272
275
|
|
|
@@ -294,7 +297,7 @@ barebrowse ships an MCP server for direct use with Claude Desktop, Cursor, or an
|
|
|
294
297
|
}
|
|
295
298
|
```
|
|
296
299
|
|
|
297
|
-
21 core tools: `browse` (one-shot), `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (engine + feature report, plus a `cookies` field naming the last goto's injected cookies — names only, never values), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so `goto` after). Plus `assess` (privacy scan) if `wearehere` is installed (`npm install wearehere`). Plus the **opt-in `eval` tool** gated by `BAREBROWSE_MCP_EVAL=1` (default OFF) — `Runtime.evaluate` in the user's authenticated session can read cookies/localStorage and hit any same-origin endpoint, so opt-in only.
|
|
300
|
+
21 core tools: `browse` (one-shot), `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (engine + feature report, plus a `cookies` field naming the last goto's injected cookies — names only, never values), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so `goto` after). Plus `assess` (privacy scan) if `wearehere` is installed (`npm install wearehere`). Every tool carries MCP `annotations` (`readOnlyHint`/`destructiveHint`/`openWorldHint`/`idempotentHint`). `BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier (`r` = read the web, `rw` = + click/type/press/select/drag/hover/switch_engine, `rwx` = + upload/eval); `BAREBROWSE_ALLOW=a.com,*.b.org` sets the site lock. Bad values refuse to start. `capabilities` reports `tier` and `allowHosts`. Plus the **opt-in `eval` tool** gated by `BAREBROWSE_MCP_EVAL=1` (default OFF) — `Runtime.evaluate` in the user's authenticated session can read cookies/localStorage and hit any same-origin endpoint, so opt-in only.
|
|
298
301
|
|
|
299
302
|
**Idle release:** the MCP server closes its browser after `BAREBROWSE_IDLE_MINUTES` of no tool calls (default `15`; `0` = off; blank/invalid = default; capped at ~24.8 days; calls in flight postpone it). The next call opens a fresh browser — page state (open tabs, a solved captcha) is not resumed. Transient browser errors close the broken browser instead of orphaning it; a `wait_for` that merely times out keeps the page (its outer deadline is `max(60s, timeout+5s)`).
|
|
300
303
|
|
|
@@ -304,7 +307,9 @@ Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe.
|
|
|
304
307
|
|
|
305
308
|
`browse` and `snapshot` also accept `pruneMode: 'act'|'read'|'data'`. `act` (the default) keeps interactive elements and short labels — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. `data` (opt-in) is one line per record — tables with headers (`row "Label": Header: value · …`), repeated items grouped (`Label: a, b, c`) — for data pages (timetables, listings, results), not articles; every data snapshot starts with ONE legend line (`# data mode: one line per record. Tables: row "<label>": <column>: <value> · … Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode "act" for menus and page controls.`) so you can read it without docs; it keeps `[ref=N]` on interactive nodes (clickable like act) and is *larger* than act on prose pages — use `read`/`readable()` there. Same surface on the bareagent adapter. If act mode collapses a content-heavy page (raw > 5 KB → pruned < 500 chars AND < 5% of raw), the result includes a `hint: act mode dropped most of the page — retry with pruneMode='read' …` line between the stats and the tree so the caller knows to re-snapshot in read mode instead of bailing to a separate HTTP fetch.
|
|
306
309
|
|
|
307
|
-
|
|
310
|
+
`links: true` (opt-in, `snapshot({ links: true })`, `browse(url, { links: true })`, or the `connect({ links: true })` default; MCP/bareagent `links` param, CLI `snapshot --links` / `browse <url> --links`) adds each link's absolute URL right after its quoted name: `- link "Fix login bug" <https://github.com/o/r/issues/42> [ref=12]`. Works in every mode; a heading that wraps a link keeps the URL on its own line (`heading "title" <url> [level=3] [ref=N]`); data mode also puts it after link cells in table rows and each item of a collapsed group (`Labels: bug <url> [ref=1], docs <url> [ref=2]`) and its legend gains ` Links: "name" <url>.`. `javascript:` and same-page `#fragment` links get no URL, every link line, row cell and group item carries its own URL (nothing is deduped across them, even when two links share a URL), and URLs never contain a raw space, `<` or `>`. Off by default: output is unchanged and costs nothing (about +23% chars on a GitHub issue list in data mode).
|
|
311
|
+
|
|
312
|
+
`goto` answers `ok` for 2xx and `ok (HTTP 404)` for other statuses (the next `snapshot` starts with an `[HTTP 404 …]` warning line); an unreachable host is an error. Session runs in hybrid mode (headless with automatic headed fallback on bot detection). `goto` injects cookies from the user's browser before navigation for authenticated access.
|
|
308
313
|
|
|
309
314
|
Session tools share a singleton page, lazy-created on first use. All session tools have auto-retry on transient failures (browser crash, WebSocket close, navigation timeout) on a per-tool deadline (v0.9.0 H5): `goto`/`reload`/`wait_for` 60s, `back`/`forward` 30s, interactive ops (`click`/`type`/`press`/`scroll`/`hover`/`select`/`drag`/`snapshot`/`eval`) 15s, `tabs` 5s, heavy I/O (`pdf`/`screenshot`/`upload`) 45s — replaces the prior blanket 30s. Session resets between attempts. Idempotent tools retry once; mutating tools (`click`/`type`/`upload`/etc.) `{ retry: false }` so partial first attempts don't replay on a fresh page. Scroll accepts `direction: "up"/"down"` in addition to numeric `deltaY`. Click falls back to JS `.click()` when elements have no layout. `browse` has a 60s timeout (no retry — stateless). Assess tries headless first; if bot-blocked, retries headed. Browser OOM/crash auto-recovers (session resets, server stays alive).
|
|
310
315
|
|
package/cli.js
CHANGED
|
@@ -9,6 +9,7 @@ import { existsSync, readFileSync, writeFileSync, mkdirSync, copyFileSync } from
|
|
|
9
9
|
import { join, resolve } from 'node:path';
|
|
10
10
|
import { homedir, platform } from 'node:os';
|
|
11
11
|
import { fileURLToPath } from 'node:url';
|
|
12
|
+
import { parseAllowHosts } from './src/site-lock.js';
|
|
12
13
|
|
|
13
14
|
const args = process.argv.slice(2);
|
|
14
15
|
const cmd = args[0];
|
|
@@ -37,7 +38,7 @@ if (args.includes('--daemon-internal')) {
|
|
|
37
38
|
} else if (cmd === 'goto' && args[1]) {
|
|
38
39
|
await cmdProxy('goto', { url: args[1], timeout: parseFlag('--timeout') });
|
|
39
40
|
} else if (cmd === 'snapshot') {
|
|
40
|
-
await cmdProxy('snapshot', { mode: parseFlag('--mode') });
|
|
41
|
+
await cmdProxy('snapshot', { mode: parseFlag('--mode'), links: hasFlag('--links') || undefined });
|
|
41
42
|
} else if (cmd === 'readable') {
|
|
42
43
|
await cmdProxy('readable');
|
|
43
44
|
} else if (cmd === 'screenshot') {
|
|
@@ -127,6 +128,7 @@ async function cmdOpen() {
|
|
|
127
128
|
blockUrls: parseFlagAll('--block-urls'),
|
|
128
129
|
blockPrivateNetwork: hasFlag('--block-private-network') || undefined,
|
|
129
130
|
uploadDir: parseFlag('--upload-dir') ? resolve(parseFlag('--upload-dir')) : undefined,
|
|
131
|
+
allowHosts: parseAllowHostsFlag(),
|
|
130
132
|
};
|
|
131
133
|
|
|
132
134
|
try {
|
|
@@ -185,7 +187,7 @@ async function cmdProxy(command, cmdArgs) {
|
|
|
185
187
|
try { unlinkSync(sessionPath); } catch { /* already gone */ }
|
|
186
188
|
process.stdout.write('Session closed.\n');
|
|
187
189
|
} else {
|
|
188
|
-
process.stdout.write('ok\n');
|
|
190
|
+
process.stdout.write(result.status ? `ok (HTTP ${result.status})\n` : 'ok\n');
|
|
189
191
|
}
|
|
190
192
|
} catch (err) {
|
|
191
193
|
if (command === 'close') {
|
|
@@ -203,9 +205,9 @@ async function cmdProxy(command, cmdArgs) {
|
|
|
203
205
|
async function oneShot() {
|
|
204
206
|
const { browse } = await import('./src/index.js');
|
|
205
207
|
const url = args[1];
|
|
206
|
-
const mode = args[2]
|
|
208
|
+
const mode = args[2] && !args[2].startsWith('--') ? args[2] : 'headless';
|
|
207
209
|
try {
|
|
208
|
-
const snapshot = await browse(url, { mode, incognito: hasFlag('--incognito') || undefined });
|
|
210
|
+
const snapshot = await browse(url, { mode, incognito: hasFlag('--incognito') || undefined, links: hasFlag('--links') || undefined, allowHosts: parseAllowHostsFlag() });
|
|
209
211
|
process.stdout.write(snapshot + '\n');
|
|
210
212
|
process.exit(0);
|
|
211
213
|
} catch (err) {
|
|
@@ -234,6 +236,7 @@ async function runDaemonInternal() {
|
|
|
234
236
|
blockUrls: parseFlagAll('--block-urls'),
|
|
235
237
|
blockPrivateNetwork: hasFlag('--block-private-network') || undefined,
|
|
236
238
|
uploadDir: parseFlag('--upload-dir'),
|
|
239
|
+
allowHosts: parseAllowHostsFlag(),
|
|
237
240
|
};
|
|
238
241
|
const outputDir = parseFlag('--output-dir') || resolve('.barebrowse');
|
|
239
242
|
const url = parseFlag('--url');
|
|
@@ -252,6 +255,18 @@ function parseFlag(name) {
|
|
|
252
255
|
return undefined;
|
|
253
256
|
}
|
|
254
257
|
|
|
258
|
+
// --allow-hosts=a.com,*.b.org → validated array (site lock), or undefined when
|
|
259
|
+
// absent. A bad entry exits here, before any browser or daemon is started.
|
|
260
|
+
function parseAllowHostsFlag() {
|
|
261
|
+
const raw = parseFlag('--allow-hosts');
|
|
262
|
+
if (raw === undefined) return undefined;
|
|
263
|
+
try { return parseAllowHosts(raw.split(',')); }
|
|
264
|
+
catch (err) {
|
|
265
|
+
process.stderr.write(`Error: --allow-hosts: ${err.message}\n`);
|
|
266
|
+
process.exit(1);
|
|
267
|
+
}
|
|
268
|
+
}
|
|
269
|
+
|
|
255
270
|
function hasFlag(name) {
|
|
256
271
|
return args.includes(name);
|
|
257
272
|
}
|
|
@@ -579,13 +594,16 @@ Session:
|
|
|
579
594
|
cloud-metadata hosts. Off by default so localhost browsing works.
|
|
580
595
|
--upload-dir=DIR Sandbox uploads: reject files outside DIR (symlinks resolved).
|
|
581
596
|
Default: no restriction. (file:/chrome: schemes are always blocked.)
|
|
597
|
+
--allow-hosts=a.com,*.b.org Site lock: top-level navigation (goto, redirects, links, popups) only to these
|
|
598
|
+
hosts; others fail with NAV_BLOCKED. *.host = subdomains only. Subresources and
|
|
599
|
+
iframes are not locked. Also accepted by browse. Not allowed with --port.
|
|
582
600
|
|
|
583
601
|
Navigation:
|
|
584
602
|
barebrowse goto <url> Navigate to URL
|
|
585
603
|
barebrowse back Go back in history
|
|
586
604
|
barebrowse forward Go forward in history
|
|
587
605
|
barebrowse reload [--no-cache] Reload current page
|
|
588
|
-
barebrowse snapshot [--mode=M]
|
|
606
|
+
barebrowse snapshot [--mode=M] [--links] ARIA snapshot -> .barebrowse/page-*.yml (--links: link URLs)
|
|
589
607
|
barebrowse readable Clean article text -> .barebrowse/article-*.txt
|
|
590
608
|
barebrowse screenshot [--format] Screenshot -> .barebrowse/screenshot-*.png
|
|
591
609
|
barebrowse pdf [--landscape] PDF export -> .barebrowse/page-*.pdf
|
|
@@ -620,7 +638,7 @@ Debugging:
|
|
|
620
638
|
barebrowse switch-engine <chromium|firefox> Swap the browser in place (page, tabs, refs are lost)
|
|
621
639
|
|
|
622
640
|
One-shot:
|
|
623
|
-
barebrowse browse <url> [mode]
|
|
641
|
+
barebrowse browse <url> [mode] [--links] [--allow-hosts=...] Browse + print snapshot to stdout
|
|
624
642
|
|
|
625
643
|
MCP:
|
|
626
644
|
barebrowse mcp Start MCP server (JSON-RPC over stdio)
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: barebrowse
|
|
3
|
+
description: Browser automation using the user's real browser with real cookies. Handles consent walls, login sessions, and bot detection automatically.
|
|
4
|
+
allowed-tools: Bash(barebrowse:*)
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# barebrowse CLI — Browser Automation for Agents
|
|
8
|
+
|
|
9
|
+
Browse any URL using the user's real browser with real cookies. Returns pruned ARIA snapshots (40-90% smaller than raw) with `[ref=N]` markers for interaction. Handles cookie consent, login sessions, JS dialogs, and bot detection automatically.
|
|
10
|
+
|
|
11
|
+
## Quick Start
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
barebrowse open https://example.com # Start session + navigate
|
|
15
|
+
barebrowse snapshot # Get ARIA snapshot → .barebrowse/page-*.yml
|
|
16
|
+
barebrowse click 8 # Click element with ref=8
|
|
17
|
+
barebrowse snapshot # See result
|
|
18
|
+
barebrowse close # End session
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
All output files go to `.barebrowse/` in the current directory. Read them with the Read tool when needed. Output files older than 7 days are removed automatically at `open`; saved state files (`state-*.json`) are kept.
|
|
22
|
+
|
|
23
|
+
## Commands
|
|
24
|
+
|
|
25
|
+
### Session Lifecycle
|
|
26
|
+
|
|
27
|
+
| Command | Description |
|
|
28
|
+
|---------|-------------|
|
|
29
|
+
| `barebrowse open [url] [flags]` | Start browser session. Optionally navigate to URL. |
|
|
30
|
+
| `barebrowse close` | Close session and kill browser. |
|
|
31
|
+
| `barebrowse status` | Check if session is running. |
|
|
32
|
+
|
|
33
|
+
**Open flags:**
|
|
34
|
+
- `--mode=headless|headed|hybrid` — Browser mode (default: headless)
|
|
35
|
+
- `--no-cookies` — Skip cookie injection
|
|
36
|
+
- `--browser=firefox|chromium` — Cookie source
|
|
37
|
+
- `--prune-mode=act|read|data` — Default pruning mode
|
|
38
|
+
- `--timeout=N` — Navigation timeout in ms
|
|
39
|
+
- `--proxy=URL` — HTTP/SOCKS proxy server
|
|
40
|
+
- `--viewport=WxH` — Viewport size (e.g. 1280x720)
|
|
41
|
+
- `--storage-state=FILE` — Load cookies/localStorage from JSON file
|
|
42
|
+
- `--block-private-network` — SSRF guard: refuse loopback / RFC-1918 / link-local / cloud-metadata hosts (v0.11.0)
|
|
43
|
+
- `--upload-dir=DIR` — Sandbox uploads to DIR; reject files outside it (v0.11.0)
|
|
44
|
+
- `--allow-hosts=a.com,*.b.org` — Site lock: top-level navigation only to these hosts (`*.host` = subdomains); others fail with `NAV_BLOCKED`. Subresources and iframes are not locked. Also on `browse`.
|
|
45
|
+
|
|
46
|
+
> Security (v0.11.0): `file:`/`chrome:`/etc. navigation is blocked by default, and the daemon requires a per-session token (handled transparently by the CLI). Snapshots and saved state are written owner-only (`0600`).
|
|
47
|
+
|
|
48
|
+
### Navigation
|
|
49
|
+
|
|
50
|
+
| Command | Output |
|
|
51
|
+
|---------|--------|
|
|
52
|
+
| `barebrowse goto <url>` | Navigates, waits for load, dismisses consent. Prints "ok", or "ok (HTTP 404)" for a non-2xx page (the next snapshot starts with an `[HTTP 404 …]` warning). Unreachable host: error. |
|
|
53
|
+
| `barebrowse back` | Go back in browser history. |
|
|
54
|
+
| `barebrowse forward` | Go forward in browser history. |
|
|
55
|
+
| `barebrowse snapshot` | ARIA snapshot → `.barebrowse/page-<timestamp>.yml` |
|
|
56
|
+
| `barebrowse snapshot --mode=read` | Read mode: keeps all text (for content extraction) |
|
|
57
|
+
| `barebrowse snapshot --mode=data` | Data mode: one line per record (table rows with headers, repeated items grouped). For data pages, not articles. Main content only; use `act` for menus and page controls. |
|
|
58
|
+
| `barebrowse snapshot --links` | Add each link's absolute URL after its name: `link "title" <https://…> [ref=N]` (combine with `--mode`; also on `browse <url> --links`). `javascript:` and same-page `#` links get none. |
|
|
59
|
+
| `barebrowse readable` | Clean article text → `.barebrowse/article-<timestamp>.txt`. For reading/summarising articles (news, blogs, docs); falls back to snapshot on non-articles. |
|
|
60
|
+
| `barebrowse screenshot` | Screenshot → `.barebrowse/screenshot-<timestamp>.png` |
|
|
61
|
+
| `barebrowse pdf [--landscape]` | PDF export → `.barebrowse/page-<timestamp>.pdf` |
|
|
62
|
+
|
|
63
|
+
### Interaction
|
|
64
|
+
|
|
65
|
+
| Command | Description |
|
|
66
|
+
|---------|-------------|
|
|
67
|
+
| `barebrowse click <ref>` | Click element (scrolls into view first) |
|
|
68
|
+
| `barebrowse type <ref> <text>` | Type text into element |
|
|
69
|
+
| `barebrowse fill <ref> <text>` | Clear existing content + type new text |
|
|
70
|
+
| `barebrowse press <key>` | Press key: Enter, Tab, Escape, Backspace, Delete, arrows, Space |
|
|
71
|
+
| `barebrowse scroll <deltaY>` | Scroll page (positive=down, negative=up) |
|
|
72
|
+
| `barebrowse hover <ref>` | Hover over element (triggers tooltips) |
|
|
73
|
+
| `barebrowse select <ref> <value>` | Select dropdown option |
|
|
74
|
+
| `barebrowse drag <fromRef> <toRef>` | Drag element to another element |
|
|
75
|
+
| `barebrowse upload <ref> <files..>` | Upload file(s) to a file input element |
|
|
76
|
+
|
|
77
|
+
### Tabs
|
|
78
|
+
|
|
79
|
+
| Command | Description |
|
|
80
|
+
|---------|-------------|
|
|
81
|
+
| `barebrowse tabs` | List open tabs (index, url, title) |
|
|
82
|
+
| `barebrowse tab <index>` | Switch to tab by index |
|
|
83
|
+
|
|
84
|
+
### Debugging
|
|
85
|
+
|
|
86
|
+
| Command | Output |
|
|
87
|
+
|---------|--------|
|
|
88
|
+
| `barebrowse eval <expression>` | Evaluate JS in page, print result |
|
|
89
|
+
| `barebrowse wait-idle` | Wait for network idle (no requests for 500ms) |
|
|
90
|
+
| `barebrowse wait-for [opts]` | Wait for content to appear on page |
|
|
91
|
+
| `barebrowse console-logs` | Console logs → `.barebrowse/console-<timestamp>.json` |
|
|
92
|
+
| `barebrowse network-log` | Network log → `.barebrowse/network-<timestamp>.json` |
|
|
93
|
+
| `barebrowse network-log --failed` | Only failed/4xx/5xx requests |
|
|
94
|
+
| `barebrowse dialog-log` | JS dialog log → `.barebrowse/dialogs-<timestamp>.json` |
|
|
95
|
+
| `barebrowse save-state` | Cookies + localStorage → `.barebrowse/state-<timestamp>.json` |
|
|
96
|
+
| `barebrowse switch-engine <chromium\|firefox>` | Swap the session's browser engine in place; page, tabs and refs are lost, so `goto` again |
|
|
97
|
+
|
|
98
|
+
**wait-for flags:**
|
|
99
|
+
- `--text=STRING` — Wait for text to appear in page body
|
|
100
|
+
- `--selector=CSS` — Wait for CSS selector to match
|
|
101
|
+
- `--timeout=N` — Max wait time in ms (default: 30000)
|
|
102
|
+
|
|
103
|
+
## Snapshot Format
|
|
104
|
+
|
|
105
|
+
The snapshot is a YAML-like ARIA tree. Each line is one node:
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
# https://example.com/
|
|
109
|
+
# 379 chars → 45 chars (88% pruned)
|
|
110
|
+
- heading "Example Domain" [level=1] [ref=3]
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
- `[ref=N]` — Use this number with click, type, fill, hover, select, drag, upload
|
|
114
|
+
- Refs change on every snapshot — always take a fresh snapshot before interacting
|
|
115
|
+
- **act mode** (default): interactive elements + labels — for clicking, typing, navigating
|
|
116
|
+
- **read mode**: all text content — for reading articles, extracting data
|
|
117
|
+
|
|
118
|
+
## Workflow Pattern
|
|
119
|
+
|
|
120
|
+
1. `barebrowse open <url>` — start session
|
|
121
|
+
2. `barebrowse snapshot` — observe page (read the .yml file)
|
|
122
|
+
3. Decide action based on snapshot content
|
|
123
|
+
4. `barebrowse click/type/fill/press/scroll/drag/upload <ref>` — act
|
|
124
|
+
5. `barebrowse snapshot` — observe result (refs are now different!)
|
|
125
|
+
6. Repeat 3-5 until goal achieved
|
|
126
|
+
7. `barebrowse close` — clean up
|
|
127
|
+
|
|
128
|
+
## Tips
|
|
129
|
+
|
|
130
|
+
- **Always snapshot before interacting** — refs are ephemeral and change every time
|
|
131
|
+
- **Use `fill` instead of `type`** when replacing existing text in input fields
|
|
132
|
+
- **Use `--mode=read`** for snapshot when you need to extract article content or data
|
|
133
|
+
- **Use `back`/`forward`** to navigate browser history instead of re-entering URLs
|
|
134
|
+
- **Use `upload`** for file inputs — pass absolute paths to the files
|
|
135
|
+
- **Use `wait-for`** when content loads asynchronously — more reliable than `wait-idle`
|
|
136
|
+
- **Check `dialog-log`** if JS alerts/confirms were auto-dismissed during your session
|
|
137
|
+
- **Use `save-state`** to persist cookies/localStorage for later sessions via `--storage-state`
|
|
138
|
+
- **Check `console-logs`** when page behavior seems wrong — JS errors show up there
|
|
139
|
+
- **Check `network-log --failed`** to debug missing content or broken API calls
|
|
140
|
+
- **Use `eval`** as an escape hatch when ARIA tree doesn't show what you need
|
|
141
|
+
- **One session per project** — `.barebrowse/` is project-scoped
|
|
142
|
+
- For bot-detected sites, use `--mode=headed` (requires browser with `--remote-debugging-port=9222`)
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: barebrowse
|
|
3
|
+
description: Browser automation using the user's real browser with real cookies. Handles consent walls, login sessions, and bot detection automatically.
|
|
4
|
+
allowed-tools: Bash(barebrowse:*)
|
|
5
|
+
---
|
|
6
|
+
# barebrowse CLI — Browser Automation for Agents
|
|
7
|
+
|
|
8
|
+
Browse any URL using the user's real browser with real cookies. Returns pruned ARIA snapshots (40-90% smaller than raw) with `[ref=N]` markers for interaction. Handles cookie consent, login sessions, JS dialogs, and bot detection automatically.
|
|
9
|
+
|
|
10
|
+
## Quick Start
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
barebrowse open https://example.com # Start session + navigate
|
|
14
|
+
barebrowse snapshot # Get ARIA snapshot → .barebrowse/page-*.yml
|
|
15
|
+
barebrowse click 8 # Click element with ref=8
|
|
16
|
+
barebrowse snapshot # See result
|
|
17
|
+
barebrowse close # End session
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
All output files go to `.barebrowse/` in the current directory. Read them with the Read tool when needed. Output files older than 7 days are removed automatically at `open`; saved state files (`state-*.json`) are kept.
|
|
21
|
+
|
|
22
|
+
## Commands
|
|
23
|
+
|
|
24
|
+
### Session Lifecycle
|
|
25
|
+
|
|
26
|
+
| Command | Description |
|
|
27
|
+
|---------|-------------|
|
|
28
|
+
| `barebrowse open [url] [flags]` | Start browser session. Optionally navigate to URL. |
|
|
29
|
+
| `barebrowse close` | Close session and kill browser. |
|
|
30
|
+
| `barebrowse status` | Check if session is running. |
|
|
31
|
+
|
|
32
|
+
**Open flags:**
|
|
33
|
+
- `--mode=headless|headed|hybrid` — Browser mode (default: headless)
|
|
34
|
+
- `--no-cookies` — Skip cookie injection
|
|
35
|
+
- `--browser=firefox|chromium` — Cookie source
|
|
36
|
+
- `--prune-mode=act|read|data` — Default pruning mode
|
|
37
|
+
- `--timeout=N` — Navigation timeout in ms
|
|
38
|
+
- `--proxy=URL` — HTTP/SOCKS proxy server
|
|
39
|
+
- `--viewport=WxH` — Viewport size (e.g. 1280x720)
|
|
40
|
+
- `--storage-state=FILE` — Load cookies/localStorage from JSON file
|
|
41
|
+
- `--block-private-network` — SSRF guard: refuse loopback / RFC-1918 / link-local / cloud-metadata hosts (v0.11.0)
|
|
42
|
+
- `--upload-dir=DIR` — Sandbox uploads to DIR; reject files outside it (v0.11.0)
|
|
43
|
+
- `--allow-hosts=a.com,*.b.org` — Site lock: top-level navigation only to these hosts (`*.host` = subdomains); others fail with `NAV_BLOCKED`. Subresources and iframes are not locked. Also on `browse`.
|
|
44
|
+
|
|
45
|
+
> Security (v0.11.0): `file:`/`chrome:`/etc. navigation is blocked by default, and the daemon requires a per-session token (handled transparently by the CLI). Snapshots and saved state are written owner-only (`0600`).
|
|
46
|
+
|
|
47
|
+
### Navigation
|
|
48
|
+
|
|
49
|
+
| Command | Output |
|
|
50
|
+
|---------|--------|
|
|
51
|
+
| `barebrowse goto <url>` | Navigates, waits for load, dismisses consent. Prints "ok", or "ok (HTTP 404)" for a non-2xx page (the next snapshot starts with an `[HTTP 404 …]` warning). Unreachable host: error. |
|
|
52
|
+
| `barebrowse back` | Go back in browser history. |
|
|
53
|
+
| `barebrowse forward` | Go forward in browser history. |
|
|
54
|
+
| `barebrowse snapshot` | ARIA snapshot → `.barebrowse/page-<timestamp>.yml` |
|
|
55
|
+
| `barebrowse snapshot --mode=read` | Read mode: keeps all text (for content extraction) |
|
|
56
|
+
| `barebrowse snapshot --mode=data` | Data mode: one line per record (table rows with headers, repeated items grouped). For data pages, not articles. Main content only; use `act` for menus and page controls. |
|
|
57
|
+
| `barebrowse snapshot --links` | Add each link's absolute URL after its name: `link "title" <https://…> [ref=N]` (combine with `--mode`; also on `browse <url> --links`). `javascript:` and same-page `#` links get none. |
|
|
58
|
+
| `barebrowse readable` | Clean article text → `.barebrowse/article-<timestamp>.txt`. For reading/summarising articles (news, blogs, docs); falls back to snapshot on non-articles. |
|
|
59
|
+
| `barebrowse screenshot` | Screenshot → `.barebrowse/screenshot-<timestamp>.png` |
|
|
60
|
+
| `barebrowse pdf [--landscape]` | PDF export → `.barebrowse/page-<timestamp>.pdf` |
|
|
61
|
+
|
|
62
|
+
### Interaction
|
|
63
|
+
|
|
64
|
+
| Command | Description |
|
|
65
|
+
|---------|-------------|
|
|
66
|
+
| `barebrowse click <ref>` | Click element (scrolls into view first) |
|
|
67
|
+
| `barebrowse type <ref> <text>` | Type text into element |
|
|
68
|
+
| `barebrowse fill <ref> <text>` | Clear existing content + type new text |
|
|
69
|
+
| `barebrowse press <key>` | Press key: Enter, Tab, Escape, Backspace, Delete, arrows, Space |
|
|
70
|
+
| `barebrowse scroll <deltaY>` | Scroll page (positive=down, negative=up) |
|
|
71
|
+
| `barebrowse hover <ref>` | Hover over element (triggers tooltips) |
|
|
72
|
+
| `barebrowse select <ref> <value>` | Select dropdown option |
|
|
73
|
+
| `barebrowse drag <fromRef> <toRef>` | Drag element to another element |
|
|
74
|
+
| `barebrowse upload <ref> <files..>` | Upload file(s) to a file input element |
|
|
75
|
+
|
|
76
|
+
### Tabs
|
|
77
|
+
|
|
78
|
+
| Command | Description |
|
|
79
|
+
|---------|-------------|
|
|
80
|
+
| `barebrowse tabs` | List open tabs (index, url, title) |
|
|
81
|
+
| `barebrowse tab <index>` | Switch to tab by index |
|
|
82
|
+
|
|
83
|
+
### Debugging
|
|
84
|
+
|
|
85
|
+
| Command | Output |
|
|
86
|
+
|---------|--------|
|
|
87
|
+
| `barebrowse eval <expression>` | Evaluate JS in page, print result |
|
|
88
|
+
| `barebrowse wait-idle` | Wait for network idle (no requests for 500ms) |
|
|
89
|
+
| `barebrowse wait-for [opts]` | Wait for content to appear on page |
|
|
90
|
+
| `barebrowse console-logs` | Console logs → `.barebrowse/console-<timestamp>.json` |
|
|
91
|
+
| `barebrowse network-log` | Network log → `.barebrowse/network-<timestamp>.json` |
|
|
92
|
+
| `barebrowse network-log --failed` | Only failed/4xx/5xx requests |
|
|
93
|
+
| `barebrowse dialog-log` | JS dialog log → `.barebrowse/dialogs-<timestamp>.json` |
|
|
94
|
+
| `barebrowse save-state` | Cookies + localStorage → `.barebrowse/state-<timestamp>.json` |
|
|
95
|
+
| `barebrowse switch-engine <chromium\|firefox>` | Swap the session's browser engine in place; page, tabs and refs are lost, so `goto` again |
|
|
96
|
+
|
|
97
|
+
**wait-for flags:**
|
|
98
|
+
- `--text=STRING` — Wait for text to appear in page body
|
|
99
|
+
- `--selector=CSS` — Wait for CSS selector to match
|
|
100
|
+
- `--timeout=N` — Max wait time in ms (default: 30000)
|
|
101
|
+
|
|
102
|
+
## Snapshot Format
|
|
103
|
+
|
|
104
|
+
The snapshot is a YAML-like ARIA tree. Each line is one node:
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
# https://example.com/
|
|
108
|
+
# 379 chars → 45 chars (88% pruned)
|
|
109
|
+
- heading "Example Domain" [level=1] [ref=3]
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
- `[ref=N]` — Use this number with click, type, fill, hover, select, drag, upload
|
|
113
|
+
- Refs change on every snapshot — always take a fresh snapshot before interacting
|
|
114
|
+
- **act mode** (default): interactive elements + labels — for clicking, typing, navigating
|
|
115
|
+
- **read mode**: all text content — for reading articles, extracting data
|
|
116
|
+
|
|
117
|
+
## Workflow Pattern
|
|
118
|
+
|
|
119
|
+
1. `barebrowse open <url>` — start session
|
|
120
|
+
2. `barebrowse snapshot` — observe page (read the .yml file)
|
|
121
|
+
3. Decide action based on snapshot content
|
|
122
|
+
4. `barebrowse click/type/fill/press/scroll/drag/upload <ref>` — act
|
|
123
|
+
5. `barebrowse snapshot` — observe result (refs are now different!)
|
|
124
|
+
6. Repeat 3-5 until goal achieved
|
|
125
|
+
7. `barebrowse close` — clean up
|
|
126
|
+
|
|
127
|
+
## Tips
|
|
128
|
+
|
|
129
|
+
- **Always snapshot before interacting** — refs are ephemeral and change every time
|
|
130
|
+
- **Use `fill` instead of `type`** when replacing existing text in input fields
|
|
131
|
+
- **Use `--mode=read`** for snapshot when you need to extract article content or data
|
|
132
|
+
- **Use `back`/`forward`** to navigate browser history instead of re-entering URLs
|
|
133
|
+
- **Use `upload`** for file inputs — pass absolute paths to the files
|
|
134
|
+
- **Use `wait-for`** when content loads asynchronously — more reliable than `wait-idle`
|
|
135
|
+
- **Check `dialog-log`** if JS alerts/confirms were auto-dismissed during your session
|
|
136
|
+
- **Use `save-state`** to persist cookies/localStorage for later sessions via `--storage-state`
|
|
137
|
+
- **Check `console-logs`** when page behavior seems wrong — JS errors show up there
|
|
138
|
+
- **Check `network-log --failed`** to debug missing content or broken API calls
|
|
139
|
+
- **Use `eval`** as an escape hatch when ARIA tree doesn't show what you need
|
|
140
|
+
- **One session per project** — `.barebrowse/` is project-scoped
|
|
141
|
+
- For bot-detected sites, use `--mode=headed` (requires browser with `--remote-debugging-port=9222`)
|