barebrowse 0.26.0 → 0.28.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +90 -0
- package/README.md +13 -1
- package/barebrowse.context.md +30 -5
- package/cli.js +39 -3
- package/commands/barebrowse/SKILL.md +6 -0
- package/commands/barebrowse.md +1 -0
- package/mcp-server.js +108 -9
- package/package.json +1 -1
- package/src/auth.js +25 -7
- package/src/ax-snapshot.js +24 -5
- package/src/bidi.js +3 -2
- package/src/blocklist-firefox.js +12 -4
- package/src/daemon.js +27 -2
- package/src/data.js +2 -2
- package/src/firefox-page.js +126 -25
- package/src/index.js +152 -37
- package/src/navigation.js +12 -0
- package/src/prune.js +3 -2
- package/src/site-lock.js +184 -0
- package/src/zoom.js +250 -0
- package/types/auth.d.ts +26 -0
- package/types/blocklist-firefox.d.ts +2 -1
- package/types/data.d.ts +2 -1
- package/types/firefox-page.d.ts +14 -2
- package/types/index.d.ts +16 -3
- package/types/navigation.d.ts +12 -0
- package/types/prune.d.ts +2 -0
- package/types/site-lock.d.ts +70 -0
- package/types/zoom.d.ts +51 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,95 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.28.0] - 2026-11-16
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Map + zoom for `snapshot()`.** `snapshot({ map: true })` returns an outline of
|
|
8
|
+
the page's big boxes (landmarks, lists, tables, grids, feeds, tabpanels,
|
|
9
|
+
dialogs) with a ref and a size each; `snapshot({ within })` returns one box, or
|
|
10
|
+
several in page order, as if it were the whole page: a ref from the map, a
|
|
11
|
+
role, `role "name"`, or `role after "heading"` for an unnamed box. Works in
|
|
12
|
+
every mode and with `links`, on both engines. Two or more matches return the
|
|
13
|
+
map with the matches marked; none, or a ref whose element is gone, throws. The
|
|
14
|
+
library default is unchanged (byte for byte). MCP `snapshot` gets `map`,
|
|
15
|
+
`within` and `full`; a snapshot over `maxChars` now also returns the map
|
|
16
|
+
inline (the full snapshot is still saved to a file). CLI: `snapshot --map`,
|
|
17
|
+
`--within=<box>` (repeatable), `--full`, `--max-chars=N`.
|
|
18
|
+
- **Map lists card groups.** A grid of alike cards (products, listings, videos)
|
|
19
|
+
that no landmark, list or table role marks now shows in the map as
|
|
20
|
+
`- group ×18 (6.3k chars) [ref=N]`, nested where it sits; `within: <ref>` zooms
|
|
21
|
+
to it and `within: 'group'` selects the groups. Found by structure only (4 or
|
|
22
|
+
more siblings of one role with similar shape and real text), never inside
|
|
23
|
+
banner/navigation/contentinfo, never for a list or table. Only the map and
|
|
24
|
+
`within` output change; default, act, read and data snapshots are unchanged.
|
|
25
|
+
- **MCP and CLI settle once before the first snapshot after a goto.** A page's
|
|
26
|
+
load event is not its last request (Airbnb on Firefox was incomplete 5 of 5
|
|
27
|
+
times; Flights about 1 in 5). The first `snapshot` after a `goto` now waits for
|
|
28
|
+
the network to go quiet (idle 500ms, capped at 3s; about 0.5s on a finished
|
|
29
|
+
page), best-effort: a timeout is not an error. A goto followed by a click pays
|
|
30
|
+
nothing. MCP `snapshot` takes `settle: false` and the CLI `snapshot --no-settle`
|
|
31
|
+
to skip it. The library (`connect()`, `browse()`) is unchanged.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- **Chromium: `goto()` to a hash-only change took 30s.** `goto('/ok#a')` then
|
|
36
|
+
`goto('/ok#b')` waited for a load event that never fires; it now returns at once
|
|
37
|
+
with `{ url, status: 0 }`, as Firefox does.
|
|
38
|
+
- **Chromium: refs die when a click or key press navigates to a new document.**
|
|
39
|
+
An old ref after a link click used to hit whatever element now had that number;
|
|
40
|
+
it now throws, like after `goto`. Hash links and `pushState` keep refs valid.
|
|
41
|
+
- **Firefox: clicks on elements below the fold.** `click()`, `hover()` and
|
|
42
|
+
`drag()` threw `move target out of bounds`, so Google's consent interstitial
|
|
43
|
+
was never dismissed on Firefox. The target is scrolled into view first.
|
|
44
|
+
- **Chromium: `goBack()` / `goForward()` took 30s** when the traversal restored
|
|
45
|
+
the page from the back/forward cache or landed on a same-document history
|
|
46
|
+
entry. They now return when the traversal lands.
|
|
47
|
+
- **Chromium: numeric refs.** `click`, `type`, `hover`, `select`, `drag` and
|
|
48
|
+
`upload` accept a ref as a number as well as a string.
|
|
49
|
+
- **Firefox: refs from before a navigation no longer resolve to another
|
|
50
|
+
element.** A stale ref now throws, as on Chromium, including after a page
|
|
51
|
+
script's `history.back()` restores the old document from the bfcache. The
|
|
52
|
+
token that tracks this lives in an isolated BiDi sandbox realm, out of reach
|
|
53
|
+
of page script.
|
|
54
|
+
- **Firefox: table cells and image-only links in act/read snapshots.** Cell,
|
|
55
|
+
column-header and row-header text was dropped, so tables pruned to their name;
|
|
56
|
+
an image-only link had no name (its `alt` was ignored), so data mode dropped
|
|
57
|
+
it. Both now match Chromium.
|
|
58
|
+
- **Firefox: consent accept that navigates.** The snapshot after the click no
|
|
59
|
+
longer reads the consent interstitial before the next page lands (1s settle,
|
|
60
|
+
as on Chromium).
|
|
61
|
+
- **Consent: `goto()` reports the page behind an accept that navigated.** The
|
|
62
|
+
resolved `{ url, status }` is the landed page, not the interstitial.
|
|
63
|
+
- **MCP and CLI: a zoom over `maxChars` says why it came back as a file path**,
|
|
64
|
+
instead of returning the path with no explanation.
|
|
65
|
+
- **Chromium cookie reuse works on KDE.** The KWallet password was
|
|
66
|
+
base64-decoded before key derivation, which made every encrypted (`v11`)
|
|
67
|
+
cookie fail to decrypt and get dropped (0 of 17 on a real jar). It is now used
|
|
68
|
+
as stored, as Chromium does.
|
|
69
|
+
- **Partitioned Chromium cookies are skipped.** A partitioned (CHIPS) cookie
|
|
70
|
+
sharing name, domain and path with the top-level one could shadow it on merge,
|
|
71
|
+
as the Firefox reader already avoided.
|
|
72
|
+
|
|
73
|
+
## [0.27.0] - 2026-10-30
|
|
74
|
+
|
|
75
|
+
### Added
|
|
76
|
+
|
|
77
|
+
- **Site lock (`allowHosts`).** `connect({ allowHosts: ['a.com', '*.b.org'] })`
|
|
78
|
+
and `browse(url, { allowHosts })` confine top-level navigation to a host list
|
|
79
|
+
on both engines: `goto()`, redirect hops, clicked links, JS jumps, popups and
|
|
80
|
+
new tabs. A blocked `goto()` rejects with `err.code === 'NAV_BLOCKED'` and
|
|
81
|
+
`err.host`; the page stays where it was. A blocked in-page jump shows as
|
|
82
|
+
`[blocked navigation to <host>]` on the next `snapshot()`. Not locked:
|
|
83
|
+
subresources, iframes, WebSocket. Attach mode (`port`) throws. CLI:
|
|
84
|
+
`--allow-hosts=a.com,*.b.org` on `open` and `browse`.
|
|
85
|
+
- **MCP tool annotations and access tiers.** Every tool has `readOnlyHint`,
|
|
86
|
+
`destructiveHint`, `openWorldHint` (and `idempotentHint` where it applies).
|
|
87
|
+
`BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier, and a
|
|
88
|
+
hidden tool answers `Unknown tool`. `BAREBROWSE_ALLOW=a.com,*.b.org` sets the
|
|
89
|
+
site lock. An invalid value of either stops the server at start. The
|
|
90
|
+
`capabilities` tool and the server instructions report `tier` and
|
|
91
|
+
`allowHosts`.
|
|
92
|
+
|
|
3
93
|
## [0.26.0] - 2026-10-27
|
|
4
94
|
|
|
5
95
|
### Added
|
package/README.md
CHANGED
|
@@ -96,7 +96,7 @@ Or manually add to your config (`claude_desktop_config.json`, `.cursor/mcp.json`
|
|
|
96
96
|
}
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (reports the active engine — chromium or firefox — and what the session can do, plus a `cookies` field naming the cookies injected on the last goto — names only, never values; no side effects), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so call `goto` after). Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
|
|
99
|
+
MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (reports the active engine — chromium or firefox — and what the session can do, plus a `cookies` field naming the cookies injected on the last goto — names only, never values; no side effects), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so call `goto` after). Plus `assess` (privacy scan) if [wearehere](https://github.com/hamr0/wearehere) is installed. Plus opt-in `eval` (`BAREBROWSE_MCP_EVAL=1`) — runs JS in the authenticated session, off by default because it can read cookies/localStorage. `BAREBROWSE_TIER` and `BAREBROWSE_ALLOW` restrict the tool set and the reachable hosts (see Site lock and MCP access tiers below). Session runs in hybrid mode with automatic cookie injection. Per-tool timeouts (goto/reload/wait_for 60s, back/forward 30s, interactive ops 15s, pdf/screenshot/upload 45s) with auto-retry on transient failures (idempotent only — mutating tools fail loudly to avoid double-submits).
|
|
100
100
|
|
|
101
101
|
**Idle release.** An MCP server can outlive its conversation, so it closes its browser after `BAREBROWSE_IDLE_MINUTES` (default `15`, `0` = never, blank/invalid = default, fractional allowed, capped at ~24.8 days) with no tool call in flight. The next call opens a fresh browser (a new job, not a resume — a page left open, e.g. a solved captcha in the headed window, is gone). A transient browser error also closes the broken browser rather than orphaning it; an ordinary `wait_for` timeout does not reset the page.
|
|
102
102
|
|
|
@@ -106,6 +106,8 @@ MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `
|
|
|
106
106
|
|
|
107
107
|
`links: true` (opt-in, `snapshot({ links: true })`, `browse(url, { links: true })`, or the `connect({ links: true })` default; MCP/bareagent `links` param, CLI `snapshot --links` / `browse <url> --links`) adds each link's absolute URL right after its quoted name: `- link "Fix login bug" <https://github.com/o/r/issues/42> [ref=12]`. Works in every mode; a heading that wraps a link keeps the URL on its own line (`heading "title" <url> [level=3] [ref=N]`); data mode also puts it after link cells in table rows and each item of a collapsed group (`Labels: bug <url> [ref=1], docs <url> [ref=2]`) and its legend gains ` Links: "name" <url>.`. `javascript:` and same-page `#fragment` links get no URL, every link line, row cell and group item carries its own URL (nothing is deduped across them, even when two links share a URL), and URLs never contain a raw space, `<` or `>`. Off by default: output is unchanged and costs nothing (about +23% chars on a GitHub issue list in data mode).
|
|
108
108
|
|
|
109
|
+
**Map + zoom** (big or cluttered pages). `snapshot({ map: true })` returns an outline of the page's big boxes instead of content, one line each, nested, with a ref and the size of its text in data mode: `- main [ref=5] (21.3k chars)`, ` - list "Issues" [ref=40] (14.9k chars)`, ` - list after "Other flights" [ref=88] (3.1k chars)` (an unnamed box shows the heading before it). Boxes: banner, navigation, main, complementary, contentinfo, region, form, search, list, table, grid, treegrid, feed, tablist, tabpanel, dialog; only those of at least 800 chars are listed. Card groups are listed too: a parent whose substantial children are mostly alike (a grid of product, listing or video cards that no role marks; found by structure only: 4 or more cards of one role and similar shape, each with real text) shows as `- group ×18 (6.3k chars) [ref=N]` where it sits (size = the cards' text; ref = their parent's, so `within: N` zooms the cards and anything beside them in that parent; when the cards sit straight in `main` that is main's ref). Not looked for inside banner, navigation or contentinfo, and not for a list or table (already boxes). A bare `within: 'group'` selects the card groups (two or more: the map with `<- match`; with none on the page, it matches real `group` nodes as before). No name or position selector: use the ref. `snapshot({ within })` then returns just one box, as if it were the whole page (prune's landmark step and data mode's main-only scope are off inside it; every mode and `links` work): `within: 40` (a ref from the map), `'table'` (role), `'list "Issues"'` (role + exact accessible name), `'list after "Other flights"'` (an unnamed box by the heading right before it, which survives a reload), or an array of those for several boxes, output in page order. Two or more matches return the map with the matches marked `<- match` (never picks one); none match, or a ref whose element is gone, throws. A ref in `within` is a ref from your previous snapshot (refs renumber on every snapshot and reset on navigation), found again by its element, not its number; refs printed in a zoom are clickable. Refs only hold until the next snapshot or navigation (including one a click or key press caused: Chromium checks the page's document identity at every ref use, so a hash link or `pushState` keeps refs, a new page kills them), and a ref from an older map can point at a different box; for anything you store or hardcode, use names (`list "Issues"`, `list after "Top flights"`), which are stable across reloads. `within` wins over `map`. The library default is unchanged: no `map`/`within` gives today's output byte for byte (so a consumer that freezes the whole page text is unaffected). MCP `snapshot` takes `map`, `within` (string, number or array) and `full`, and a snapshot over `maxChars` returns the map inline while still saving the full snapshot to `.barebrowse/` (`full: true` keeps the path-only answer); MCP and the CLI daemon wait once for the network to go quiet (idle 500ms, up to 3s) before the first snapshot after a `goto`, so late-loading content is in it (`settle: false` / `--no-settle` skip it; the library is unchanged). CLI `snapshot --map`, `--within=<box>` (repeat the flag for several boxes; quote a name: `--within='list "Issues"'`), `--full`, `--max-chars=N` (default 30000: over it, the map is printed after the file path). `browse()` and the bareagent adapter are unchanged. A table keeps its cell text in act/read mode on both engines.
|
|
110
|
+
|
|
109
111
|
`npx barebrowse doctor` reports the runtime environment first — default engine/mode, installed Chromium/Firefox binaries, cookie sources, and any live session's engine + capabilities — then scans every known MCP config location and flags scope conflicts. `npx barebrowse install --force` overwrites an existing entry pointing at a different endpoint.
|
|
110
112
|
|
|
111
113
|
### 3. Library -- for agentic automation
|
|
@@ -211,8 +213,18 @@ barebrowse hands an autonomous — and therefore prompt-injectable — agent an
|
|
|
211
213
|
- **CLI daemon authenticated** with a per-session token (loopback alone isn't an authorization boundary); snapshots and saved state are written owner-only (`0600`).
|
|
212
214
|
- **Opt-in hardening** for stricter deployments: `blockPrivateNetwork` (SSRF guard for loopback/RFC-1918/cloud-metadata) and `uploadDir` (confine `upload()` to one directory). Both available on the library, MCP, bareagent, and CLI (`--block-private-network`, `--upload-dir`).
|
|
213
215
|
|
|
216
|
+
- **Site lock** (`allowHosts`): confine top-level navigation to a list of hosts. See below.
|
|
217
|
+
|
|
214
218
|
See `barebrowse.context.md` and the PRD's "Security Model & Safe Defaults" for the full rationale.
|
|
215
219
|
|
|
220
|
+
### Site lock and MCP access tiers
|
|
221
|
+
|
|
222
|
+
`allowHosts: ['example.com', '*.docs.example.org']` on `connect()` and `browse()` (CLI `--allow-hosts=a.com,*.b.org` on `open` and `browse`; MCP `BAREBROWSE_ALLOW=a.com,*.b.org`) locks the top frame to those hosts, on both engines. An entry is `host` (exactly that host) or `*.host` (its subdomains, not `host` itself); a scheme, port or path in an entry is an error. A `goto()` to another host, or one that redirects there, rejects with `err.code === 'NAV_BLOCKED'` (`err.host`) and the page stays where it was. A clicked link, a JS jump or a `window.open` popup to another host never loads; the next `snapshot()` carries `[blocked navigation to <host>]`. New tabs are locked too. `connect({ port, allowHosts })` (attach mode) throws: the lock is browser-wide and would reach your own tabs.
|
|
223
|
+
|
|
224
|
+
Not locked, by design: subresources (images, scripts, CSS, XHR/fetch), iframes and WebSocket. The lock stops an agent being steered to another site, not a page from loading its own assets. The MCP `eval` tool (tier `rwx`) can still `fetch()` anywhere.
|
|
225
|
+
|
|
226
|
+
MCP only: `BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier. `r` reads (`browse`, `goto`, `back`, `forward`, `reload`, `scroll`, `snapshot`, `readable`, `screenshot`, `pdf`, `wait_for`, `tabs`, `capabilities`, `assess`); `rw` adds `click`, `type`, `press`, `select`, `drag`, `hover`, `switch_engine`; `rwx` adds `upload` and `eval`. A hidden tool answers `Unknown tool`. Every tool also carries MCP `annotations` (`readOnlyHint`, `destructiveHint`, `openWorldHint`, `idempotentHint`). An invalid `BAREBROWSE_TIER` or `BAREBROWSE_ALLOW` stops the server at start; it never falls back to `rwx` or to no lock. `capabilities` and the server instructions report the tier and the allowed hosts.
|
|
227
|
+
|
|
216
228
|
## What the agent sees
|
|
217
229
|
|
|
218
230
|
Raw ARIA output from a page is noisy -- decorative wrappers, hidden elements, structural junk. The pruning pipeline (ported from [mcprune](https://github.com/hamr0/mcprune)) strips it down to what matters.
|
package/barebrowse.context.md
CHANGED
|
@@ -66,12 +66,12 @@ const snapshot = await browse('https://example.com', {
|
|
|
66
66
|
|
|
67
67
|
| Method | Args | Returns | Notes |
|
|
68
68
|
|---|---|---|---|
|
|
69
|
-
| `goto(url, timeout?)` | url: string, timeout: number (default 30000) | `{ url, status }` | Navigate + wait for load + dismiss consent. Resolves to the final URL (after redirects) and the main document's HTTP status (`0` = no HTTP response observed, e.g. an aborted navigation). Non-2xx still resolves; `snapshot()` then carries `[HTTP 404 — the page may be an error page]`. Network failure (DNS, refused, TLS) rejects with `err.code === 'NAV_FAILED'` and `err.reason` (raw browser string, e.g. `net::ERR_NAME_NOT_RESOLVED` / `NS_ERROR_UNKNOWN_HOST`). A download or 204 (`ERR_ABORTED` / `NS_BINDING_ABORTED`) is not a failure and returns without waiting for a load event. `browse()` still returns a string (warning line prepended when non-2xx) and rejects with the same `NAV_FAILED`. |
|
|
69
|
+
| `goto(url, timeout?)` | url: string, timeout: number (default 30000) | `{ url, status }` | Navigate + wait for load + dismiss consent. Resolves to the final URL (after redirects) and the main document's HTTP status (`0` = no HTTP response observed, e.g. an aborted navigation). Non-2xx still resolves; `snapshot()` then carries `[HTTP 404 — the page may be an error page]`. Network failure (DNS, refused, TLS) rejects with `err.code === 'NAV_FAILED'` and `err.reason` (raw browser string, e.g. `net::ERR_NAME_NOT_RESOLVED` / `NS_ERROR_UNKNOWN_HOST`). A download or 204 (`ERR_ABORTED` / `NS_BINDING_ABORTED`) is not a failure and returns without waiting for a load event. `browse()` still returns a string (warning line prepended when non-2xx) and rejects with the same `NAV_FAILED`. With `allowHosts`, an off-list URL or redirect hop rejects with `err.code === 'NAV_BLOCKED'` (`err.host`) and the page stays put. |
|
|
70
70
|
| `lastNavigation` | -- | `{ url, status }` or null | Outcome of the latest `goto()`; null before any goto and while a goto is failing. Reset to null by `reload`/`goBack`/`goForward`/`switchTab` (status is reported for `goto()` only) |
|
|
71
71
|
| `goBack()` | -- | void | Navigate back in browser history |
|
|
72
72
|
| `goForward()` | -- | void | Navigate forward in browser history |
|
|
73
73
|
| `reload(opts?)` | { ignoreCache?: boolean, timeout?: number } | void | Reload the current page. Clears refMap (refs from pre-reload reject). |
|
|
74
|
-
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data', links?: boolean } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. `links: true` adds `<absolute-url>` after each link name (see below). |
|
|
74
|
+
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data', links?: boolean, map?: boolean, within?: string\|number\|Array } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. `links: true` adds `<absolute-url>` after each link name (see below). `map: true` returns an outline of the page's big boxes (ref + size); `within` zooms to one box or several (see Map + zoom below). |
|
|
75
75
|
| `readable()` | -- | object | Clean article text (Reader-View engine). `{ ok, title, byline, text, length, confidence: 'high'\|'low', readerable, hint? }` or `{ ok: false, hint }`. For *reading*, not interacting — see note below. |
|
|
76
76
|
| `click(ref)` | ref: string | void | Scroll into view + mouse press+release at center |
|
|
77
77
|
| `type(ref, text, opts?)` | ref: string, text: string, opts: { clear?, keyEvents? } | void | Focus + insert text. `clear: true` replaces existing. |
|
|
@@ -112,6 +112,7 @@ const snapshot = await browse('https://example.com', {
|
|
|
112
112
|
- `blockUrls: ['*://foo.com/*', ...]` — Extra glob patterns (CDP `Network.setBlockedURLs` format) to block in addition to the default. Merged with the default unless `blockAds: false`.
|
|
113
113
|
- `allowLocalUrls: true|false` — (v0.11.0) Default `false`: navigation to local-resource / browser-internal schemes (`file:`, `view-source:`, `chrome:`, `filesystem:`, `devtools:`, …) is **blocked** to stop a prompt-injected agent reading local files. `http`/`https`/`data`/`blob`/`about` are always allowed. Set `true` to permit local schemes.
|
|
114
114
|
- `blockPrivateNetwork: true|false` — (v0.11.0) Default `false`. When `true`, `goto()`/`browse()` refuse loopback / RFC-1918 / link-local / cloud-metadata (`169.254.169.254`) / `*.internal` hosts (SSRF guard). Off by default so localhost dev-server browsing works. Hostname-based — does not catch DNS names that resolve to private IPs.
|
|
115
|
+
- `allowHosts: ['example.com', '*.docs.example.org']` — Site lock (both engines, `connect()` and `browse()`; CLI `--allow-hosts=a,b` on `open`/`browse`; MCP `BAREBROWSE_ALLOW=a,b`). Top-level navigations (goto, redirect hops, clicked links, JS jumps, `window.open` popups, new tabs) to a host outside the list are failed in the browser and the page stays put. Entry = `host` (exact) or `*.host` (strict subdomains, not the apex); scheme/port/path/bare `*` throw. `goto()` rejects with `err.code === 'NAV_BLOCKED'` + `err.host`; in-page blocks add `[blocked navigation to <host>]` to the next `snapshot()`. Attach mode (`port`) throws. NOT locked: subresources, iframes, WebSocket (and MCP `eval`, which can `fetch()` anywhere).
|
|
115
116
|
- `uploadDir: '/abs/dir'` — (v0.11.0) Default unset (no restriction). When set, `upload()` rejects any file that does not resolve (symlinks included, via `realpath`) inside this directory — sandboxes the agent's file-upload capability.
|
|
116
117
|
|
|
117
118
|
## Snapshot format
|
|
@@ -132,6 +133,28 @@ Key rules:
|
|
|
132
133
|
- `click(ref)` / `type(ref, text)` / `hover(ref)` / `select(ref, value)` use these ref strings
|
|
133
134
|
- Pruning removes noise (~47-95% token reduction) while keeping all interactive elements
|
|
134
135
|
|
|
136
|
+
## Map + zoom — big pages
|
|
137
|
+
|
|
138
|
+
Order for a big or cluttered page: map, then zoom, then mode. The library default is unchanged (no `map`/`within` = today's output, byte for byte).
|
|
139
|
+
|
|
140
|
+
```js
|
|
141
|
+
await page.snapshot({ map: true }); // outline of the big boxes, no content
|
|
142
|
+
await page.snapshot({ within: 40 }); // one box by ref from the map
|
|
143
|
+
await page.snapshot({ within: 'table' }); // by role
|
|
144
|
+
await page.snapshot({ within: 'list "Issues"' }); // role + exact accessible name
|
|
145
|
+
await page.snapshot({ within: 'list after "Other flights"' }); // unnamed box: the heading right before it (survives a reload)
|
|
146
|
+
await page.snapshot({ within: [430, 'table'], mode: 'data', links: true }); // several boxes, page order; any mode/links
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
- Map line: `- role "name" [ref=N] (14.9k chars)`, nested; unnamed boxes show `after "<heading>"`. Size = the box's text in data mode. Boxes: banner, navigation, main, complementary, contentinfo, region, form, search, list, table, grid, treegrid, feed, tablist, tabpanel, dialog; listed from 800 chars.
|
|
150
|
+
- Inside a zoom the box is the whole page: no landmark pruning, no main-only scope in data mode; everything else prunes as usual.
|
|
151
|
+
- Card groups: a grid of alike cards no role marks (products, listings, videos; structural detection, 4+ alike siblings) is listed as `- group ×18 (6.3k chars) [ref=N]`; zoom it by that ref (it is the cards' parent, so cards straight in `main` give main's ref). `within: 'group'` selects the card groups (several: map with `<- match`). Not inside banner/navigation/contentinfo, not for a list or table. No name selector.
|
|
152
|
+
- Match is exact on role and accessible name (or the box's preceding sibling heading text). Two or more matches: the map comes back with `<- match` on them, nothing is picked. Zero: throws, naming what was asked.
|
|
153
|
+
- A ref in `within` is a ref from your previous snapshot; it is found again by its element (refs renumber every snapshot). Gone element or a ref from before a navigation: throws. Refs printed in a zoom work with `click()` etc. Refs only hold until the next snapshot or navigation, and a ref from an older map can point at a different box; for anything you store or hardcode, use names (`list "Issues"`, `list after "Top flights"`), which are stable across reloads. `within` wins over `map`.
|
|
154
|
+
- Firefox act/read keep a table's cells (text under each cell), as Chromium does; `mode: 'data'` still gives one line per row.
|
|
155
|
+
- `browse()` and `createBrowseTools()` (bareagent) have no map/zoom.
|
|
156
|
+
- MCP: `snapshot` takes `map`, `within`, `full`; over `maxChars` it returns the map inline and saves the full snapshot to a file (`full: true` = path only). CLI: `snapshot --map`, `--within=<box>` (repeatable; quote names), `--full`, `--max-chars=N`. MCP and the CLI daemon wait once (network idle 500ms, up to 3s) before the first snapshot after a `goto` so late content is in it; `settle: false` / `--no-settle` skip it; the library is unchanged.
|
|
157
|
+
|
|
135
158
|
## readable() vs snapshot() — which to use
|
|
136
159
|
|
|
137
160
|
Two different jobs:
|
|
@@ -253,6 +276,8 @@ For coding agents (Claude Code, Copilot, Cursor) and quick interactive testing.
|
|
|
253
276
|
```bash
|
|
254
277
|
barebrowse open https://example.com # Start daemon + navigate
|
|
255
278
|
barebrowse snapshot # → .barebrowse/page-<timestamp>.yml
|
|
279
|
+
barebrowse snapshot --map # outline of the page's big boxes (ref + size)
|
|
280
|
+
barebrowse snapshot --within='list "Issues"' # one box only (ref, role, role "name", role after "heading"; repeat for several)
|
|
256
281
|
barebrowse readable # → .barebrowse/article-<timestamp>.txt (clean article text)
|
|
257
282
|
barebrowse click 8 # Click element ref=8
|
|
258
283
|
barebrowse type 12 hello world # Type into element ref=12
|
|
@@ -268,7 +293,7 @@ barebrowse switch-engine firefox # Swap chromium/firefox in place (page/ta
|
|
|
268
293
|
barebrowse close # Kill daemon + browser
|
|
269
294
|
```
|
|
270
295
|
|
|
271
|
-
**Open flags:** `--mode=headless|headed|hybrid`, `--port=N` (attach to running browser), `--proxy=URL`, `--viewport=WxH`, `--storage-state=FILE`, `--download-path=DIR` (v0.9.0), `--no-cookies`, `--browser=firefox|chromium`, `--timeout=N`, `--block-private-network` (SSRF guard, v0.11.0), `--upload-dir=DIR` (upload sandbox, v0.11.0)
|
|
296
|
+
**Open flags:** `--mode=headless|headed|hybrid`, `--port=N` (attach to running browser), `--proxy=URL`, `--viewport=WxH`, `--storage-state=FILE`, `--download-path=DIR` (v0.9.0), `--no-cookies`, `--browser=firefox|chromium`, `--timeout=N`, `--block-private-network` (SSRF guard, v0.11.0), `--upload-dir=DIR` (upload sandbox, v0.11.0), `--allow-hosts=a.com,*.b.org` (site lock; also on `browse`)
|
|
272
297
|
|
|
273
298
|
Session lifecycle: `open` spawns a background daemon holding a `connect()` session. Subsequent commands POST to the daemon over HTTP (localhost). `close` shuts everything down. JS dialogs (alert/confirm/prompt) are auto-dismissed and logged.
|
|
274
299
|
|
|
@@ -296,13 +321,13 @@ barebrowse ships an MCP server for direct use with Claude Desktop, Cursor, or an
|
|
|
296
321
|
}
|
|
297
322
|
```
|
|
298
323
|
|
|
299
|
-
21 core tools: `browse` (one-shot), `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (engine + feature report, plus a `cookies` field naming the last goto's injected cookies — names only, never values), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so `goto` after). Plus `assess` (privacy scan) if `wearehere` is installed (`npm install wearehere`). Plus the **opt-in `eval` tool** gated by `BAREBROWSE_MCP_EVAL=1` (default OFF) — `Runtime.evaluate` in the user's authenticated session can read cookies/localStorage and hit any same-origin endpoint, so opt-in only.
|
|
324
|
+
21 core tools: `browse` (one-shot), `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `scroll`, `hover`, `select`, `back`, `forward`, `reload`, `drag`, `upload`, `pdf`, `screenshot`, `wait_for`, `tabs`, `capabilities` (engine + feature report, plus a `cookies` field naming the last goto's injected cookies — names only, never values), `switch_engine` (swap chromium/firefox without restarting; page, tabs and refs reset, so `goto` after). Plus `assess` (privacy scan) if `wearehere` is installed (`npm install wearehere`). Every tool carries MCP `annotations` (`readOnlyHint`/`destructiveHint`/`openWorldHint`/`idempotentHint`). `BAREBROWSE_TIER=r|rw|rwx` (default `rwx`) hides tools above the tier (`r` = read the web, `rw` = + click/type/press/select/drag/hover/switch_engine, `rwx` = + upload/eval); `BAREBROWSE_ALLOW=a.com,*.b.org` sets the site lock. Bad values refuse to start. `capabilities` reports `tier` and `allowHosts`. Plus the **opt-in `eval` tool** gated by `BAREBROWSE_MCP_EVAL=1` (default OFF) — `Runtime.evaluate` in the user's authenticated session can read cookies/localStorage and hit any same-origin endpoint, so opt-in only.
|
|
300
325
|
|
|
301
326
|
**Idle release:** the MCP server closes its browser after `BAREBROWSE_IDLE_MINUTES` of no tool calls (default `15`; `0` = off; blank/invalid = default; capped at ~24.8 days; calls in flight postpone it). The next call opens a fresh browser — page state (open tabs, a solved captcha) is not resumed. Transient browser errors close the broken browser instead of orphaning it; a `wait_for` that merely times out keeps the page (its outer deadline is `max(60s, timeout+5s)`).
|
|
302
327
|
|
|
303
328
|
Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe. This avoids double-token output since MCP tool calls are cheap to chain.
|
|
304
329
|
|
|
305
|
-
`browse`, `snapshot`, and `readable` accept a `maxChars` param (default 30000). If the output exceeds the limit it's saved to `.barebrowse/` and a short message with the file path is returned instead (`page-<timestamp>.yml` for snapshots, `article-<timestamp>.txt` for `readable`). `screenshot` always saves to `.barebrowse/screenshot-<timestamp>.{png,jpeg,webp}` and returns the file path (raw base64 in a JSON-RPC response would blow `maxChars`). `tabs` returns the JSON array, or with `switchTo: N` it switches and returns `'ok'`. All files MCP writes are owner-only (`0600` in a `0700` dir) — they can hold authenticated page content, so they're not world-readable on a shared host.
|
|
330
|
+
`browse`, `snapshot`, and `readable` accept a `maxChars` param (default 30000). If the output exceeds the limit it's saved to `.barebrowse/` and a short message with the file path is returned instead (`page-<timestamp>.yml` for snapshots, `article-<timestamp>.txt` for `readable`). The exception is `snapshot` with neither `within`, `map` nor `full`: it also returns the page map inline, to zoom from with `within` (see Map + zoom); `full: true` gives the path alone. `browse` is unchanged (one-shot: nothing to zoom afterwards). `screenshot` always saves to `.barebrowse/screenshot-<timestamp>.{png,jpeg,webp}` and returns the file path (raw base64 in a JSON-RPC response would blow `maxChars`). `tabs` returns the JSON array, or with `switchTo: N` it switches and returns `'ok'`. All files MCP writes are owner-only (`0600` in a `0700` dir) — they can hold authenticated page content, so they're not world-readable on a shared host.
|
|
306
331
|
|
|
307
332
|
`browse` and `snapshot` also accept `pruneMode: 'act'|'read'|'data'`. `act` (the default) keeps interactive elements and short labels — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. `data` (opt-in) is one line per record — tables with headers (`row "Label": Header: value · …`), repeated items grouped (`Label: a, b, c`) — for data pages (timetables, listings, results), not articles; every data snapshot starts with ONE legend line (`# data mode: one line per record. Tables: row "<label>": <column>: <value> · … Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode "act" for menus and page controls.`) so you can read it without docs; it keeps `[ref=N]` on interactive nodes (clickable like act) and is *larger* than act on prose pages — use `read`/`readable()` there. Same surface on the bareagent adapter. If act mode collapses a content-heavy page (raw > 5 KB → pruned < 500 chars AND < 5% of raw), the result includes a `hint: act mode dropped most of the page — retry with pruneMode='read' …` line between the stats and the tree so the caller knows to re-snapshot in read mode instead of bailing to a separate HTTP fetch.
|
|
308
333
|
|
package/cli.js
CHANGED
|
@@ -9,6 +9,7 @@ import { existsSync, readFileSync, writeFileSync, mkdirSync, copyFileSync } from
|
|
|
9
9
|
import { join, resolve } from 'node:path';
|
|
10
10
|
import { homedir, platform } from 'node:os';
|
|
11
11
|
import { fileURLToPath } from 'node:url';
|
|
12
|
+
import { parseAllowHosts } from './src/site-lock.js';
|
|
12
13
|
|
|
13
14
|
const args = process.argv.slice(2);
|
|
14
15
|
const cmd = args[0];
|
|
@@ -37,7 +38,14 @@ if (args.includes('--daemon-internal')) {
|
|
|
37
38
|
} else if (cmd === 'goto' && args[1]) {
|
|
38
39
|
await cmdProxy('goto', { url: args[1], timeout: parseFlag('--timeout') });
|
|
39
40
|
} else if (cmd === 'snapshot') {
|
|
40
|
-
|
|
41
|
+
const within = parseFlagAll('--within');
|
|
42
|
+
await cmdProxy('snapshot', {
|
|
43
|
+
mode: parseFlag('--mode'), links: hasFlag('--links') || undefined,
|
|
44
|
+
within: within && (within.length === 1 ? within[0] : within),
|
|
45
|
+
map: hasFlag('--map') || undefined, full: hasFlag('--full') || undefined,
|
|
46
|
+
maxChars: parseFlag('--max-chars') && Number(parseFlag('--max-chars')),
|
|
47
|
+
settle: hasFlag('--no-settle') ? false : undefined,
|
|
48
|
+
});
|
|
41
49
|
} else if (cmd === 'readable') {
|
|
42
50
|
await cmdProxy('readable');
|
|
43
51
|
} else if (cmd === 'screenshot') {
|
|
@@ -127,6 +135,7 @@ async function cmdOpen() {
|
|
|
127
135
|
blockUrls: parseFlagAll('--block-urls'),
|
|
128
136
|
blockPrivateNetwork: hasFlag('--block-private-network') || undefined,
|
|
129
137
|
uploadDir: parseFlag('--upload-dir') ? resolve(parseFlag('--upload-dir')) : undefined,
|
|
138
|
+
allowHosts: parseAllowHostsFlag(),
|
|
130
139
|
};
|
|
131
140
|
|
|
132
141
|
try {
|
|
@@ -175,6 +184,10 @@ async function cmdProxy(command, cmdArgs) {
|
|
|
175
184
|
// Print result
|
|
176
185
|
if (result.file && result.count !== undefined) {
|
|
177
186
|
process.stdout.write(`${result.file} (${result.count} entries)\n`);
|
|
187
|
+
} else if (result.file && result.map) {
|
|
188
|
+
process.stdout.write(`${result.file}\nOver --max-chars, so here is the map of the page; zoom with: snapshot --within=<ref or name> (repeat the flag for several boxes), or --full for the path alone.\n${result.map}\n`);
|
|
189
|
+
} else if (result.file && result.note) {
|
|
190
|
+
process.stdout.write(`${result.file}\n${result.note}\n`);
|
|
178
191
|
} else if (result.file) {
|
|
179
192
|
process.stdout.write(`${result.file}\n`);
|
|
180
193
|
} else if (result.value !== undefined) {
|
|
@@ -205,7 +218,7 @@ async function oneShot() {
|
|
|
205
218
|
const url = args[1];
|
|
206
219
|
const mode = args[2] && !args[2].startsWith('--') ? args[2] : 'headless';
|
|
207
220
|
try {
|
|
208
|
-
const snapshot = await browse(url, { mode, incognito: hasFlag('--incognito') || undefined, links: hasFlag('--links') || undefined });
|
|
221
|
+
const snapshot = await browse(url, { mode, incognito: hasFlag('--incognito') || undefined, links: hasFlag('--links') || undefined, allowHosts: parseAllowHostsFlag() });
|
|
209
222
|
process.stdout.write(snapshot + '\n');
|
|
210
223
|
process.exit(0);
|
|
211
224
|
} catch (err) {
|
|
@@ -234,6 +247,7 @@ async function runDaemonInternal() {
|
|
|
234
247
|
blockUrls: parseFlagAll('--block-urls'),
|
|
235
248
|
blockPrivateNetwork: hasFlag('--block-private-network') || undefined,
|
|
236
249
|
uploadDir: parseFlag('--upload-dir'),
|
|
250
|
+
allowHosts: parseAllowHostsFlag(),
|
|
237
251
|
};
|
|
238
252
|
const outputDir = parseFlag('--output-dir') || resolve('.barebrowse');
|
|
239
253
|
const url = parseFlag('--url');
|
|
@@ -252,6 +266,18 @@ function parseFlag(name) {
|
|
|
252
266
|
return undefined;
|
|
253
267
|
}
|
|
254
268
|
|
|
269
|
+
// --allow-hosts=a.com,*.b.org → validated array (site lock), or undefined when
|
|
270
|
+
// absent. A bad entry exits here, before any browser or daemon is started.
|
|
271
|
+
function parseAllowHostsFlag() {
|
|
272
|
+
const raw = parseFlag('--allow-hosts');
|
|
273
|
+
if (raw === undefined) return undefined;
|
|
274
|
+
try { return parseAllowHosts(raw.split(',')); }
|
|
275
|
+
catch (err) {
|
|
276
|
+
process.stderr.write(`Error: --allow-hosts: ${err.message}\n`);
|
|
277
|
+
process.exit(1);
|
|
278
|
+
}
|
|
279
|
+
}
|
|
280
|
+
|
|
255
281
|
function hasFlag(name) {
|
|
256
282
|
return args.includes(name);
|
|
257
283
|
}
|
|
@@ -579,6 +605,9 @@ Session:
|
|
|
579
605
|
cloud-metadata hosts. Off by default so localhost browsing works.
|
|
580
606
|
--upload-dir=DIR Sandbox uploads: reject files outside DIR (symlinks resolved).
|
|
581
607
|
Default: no restriction. (file:/chrome: schemes are always blocked.)
|
|
608
|
+
--allow-hosts=a.com,*.b.org Site lock: top-level navigation (goto, redirects, links, popups) only to these
|
|
609
|
+
hosts; others fail with NAV_BLOCKED. *.host = subdomains only. Subresources and
|
|
610
|
+
iframes are not locked. Also accepted by browse. Not allowed with --port.
|
|
582
611
|
|
|
583
612
|
Navigation:
|
|
584
613
|
barebrowse goto <url> Navigate to URL
|
|
@@ -586,6 +615,13 @@ Navigation:
|
|
|
586
615
|
barebrowse forward Go forward in history
|
|
587
616
|
barebrowse reload [--no-cache] Reload current page
|
|
588
617
|
barebrowse snapshot [--mode=M] [--links] ARIA snapshot -> .barebrowse/page-*.yml (--links: link URLs)
|
|
618
|
+
--map Outline of the page's big boxes (ref + size) instead of content
|
|
619
|
+
--within=<box> Zoom to one box, as if it were the page: a ref from the map, a role (table),
|
|
620
|
+
role "name" (--within='list "Issues"'), or role after "heading". Repeat the flag
|
|
621
|
+
for several boxes (page order). Works with --mode and --links.
|
|
622
|
+
--full Skip the auto map. A snapshot over --max-chars (default 30000) is saved
|
|
623
|
+
in full and the map is printed after the path, to zoom from.
|
|
624
|
+
--no-settle Don't wait for the network to go quiet (up to 3s) before the first snapshot after a goto
|
|
589
625
|
barebrowse readable Clean article text -> .barebrowse/article-*.txt
|
|
590
626
|
barebrowse screenshot [--format] Screenshot -> .barebrowse/screenshot-*.png
|
|
591
627
|
barebrowse pdf [--landscape] PDF export -> .barebrowse/page-*.pdf
|
|
@@ -620,7 +656,7 @@ Debugging:
|
|
|
620
656
|
barebrowse switch-engine <chromium|firefox> Swap the browser in place (page, tabs, refs are lost)
|
|
621
657
|
|
|
622
658
|
One-shot:
|
|
623
|
-
barebrowse browse <url> [mode] [--links] Browse + print snapshot to stdout
|
|
659
|
+
barebrowse browse <url> [mode] [--links] [--allow-hosts=...] Browse + print snapshot to stdout
|
|
624
660
|
|
|
625
661
|
MCP:
|
|
626
662
|
barebrowse mcp Start MCP server (JSON-RPC over stdio)
|
|
@@ -41,6 +41,7 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
|
|
|
41
41
|
- `--storage-state=FILE` — Load cookies/localStorage from JSON file
|
|
42
42
|
- `--block-private-network` — SSRF guard: refuse loopback / RFC-1918 / link-local / cloud-metadata hosts (v0.11.0)
|
|
43
43
|
- `--upload-dir=DIR` — Sandbox uploads to DIR; reject files outside it (v0.11.0)
|
|
44
|
+
- `--allow-hosts=a.com,*.b.org` — Site lock: top-level navigation only to these hosts (`*.host` = subdomains); others fail with `NAV_BLOCKED`. Subresources and iframes are not locked. Also on `browse`.
|
|
44
45
|
|
|
45
46
|
> Security (v0.11.0): `file:`/`chrome:`/etc. navigation is blocked by default, and the daemon requires a per-session token (handled transparently by the CLI). Snapshots and saved state are written owner-only (`0600`).
|
|
46
47
|
|
|
@@ -55,6 +56,10 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
|
|
|
55
56
|
| `barebrowse snapshot --mode=read` | Read mode: keeps all text (for content extraction) |
|
|
56
57
|
| `barebrowse snapshot --mode=data` | Data mode: one line per record (table rows with headers, repeated items grouped). For data pages, not articles. Main content only; use `act` for menus and page controls. |
|
|
57
58
|
| `barebrowse snapshot --links` | Add each link's absolute URL after its name: `link "title" <https://…> [ref=N]` (combine with `--mode`; also on `browse <url> --links`). `javascript:` and same-page `#` links get none. |
|
|
59
|
+
| `barebrowse snapshot --map` | Outline of the page's big boxes (lists, tables, main, …), each with a ref and a size. Cheap (1-2% of the page): use it on big pages, then zoom. A grid of alike cards (products, listings, videos) shows as `group ×18 (6.3k chars) [ref=N]`: zoom it with `--within=<that ref>`. |
|
|
60
|
+
| `barebrowse snapshot --within=<box>` | Only that box, as if it were the page. `<box>` = a ref from the map, a role (`table`), `role "name"` (`--within='list "Issues"'`), or an unnamed box by its heading (`--within='list after "Other flights"'`). Repeat the flag for several boxes (page order). Combine with `--mode` / `--links`. Several matches: the map comes back with the matches marked; none: error. |
|
|
61
|
+
| `barebrowse snapshot --no-settle` | The first snapshot after a `goto` waits once for the network to go quiet (up to 3s) so late-loading content is in it; this skips the wait. |
|
|
62
|
+
| `barebrowse snapshot --full` | A snapshot over `--max-chars` (default 30000) is saved in full and the map is printed after the path; `--full` prints the path alone. |
|
|
58
63
|
| `barebrowse readable` | Clean article text → `.barebrowse/article-<timestamp>.txt`. For reading/summarising articles (news, blogs, docs); falls back to snapshot on non-articles. |
|
|
59
64
|
| `barebrowse screenshot` | Screenshot → `.barebrowse/screenshot-<timestamp>.png` |
|
|
60
65
|
| `barebrowse pdf [--landscape]` | PDF export → `.barebrowse/page-<timestamp>.pdf` |
|
|
@@ -128,6 +133,7 @@ The snapshot is a YAML-like ARIA tree. Each line is one node:
|
|
|
128
133
|
|
|
129
134
|
- **Always snapshot before interacting** — refs are ephemeral and change every time
|
|
130
135
|
- **Use `fill` instead of `type`** when replacing existing text in input fields
|
|
136
|
+
- **On a big page, `snapshot --map` then `snapshot --within=<ref or name>`** instead of reading the whole file; refs in `--within` come from the last snapshot
|
|
131
137
|
- **Use `--mode=read`** for snapshot when you need to extract article content or data
|
|
132
138
|
- **Use `back`/`forward`** to navigate browser history instead of re-entering URLs
|
|
133
139
|
- **Use `upload`** for file inputs — pass absolute paths to the files
|
package/commands/barebrowse.md
CHANGED
|
@@ -40,6 +40,7 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
|
|
|
40
40
|
- `--storage-state=FILE` — Load cookies/localStorage from JSON file
|
|
41
41
|
- `--block-private-network` — SSRF guard: refuse loopback / RFC-1918 / link-local / cloud-metadata hosts (v0.11.0)
|
|
42
42
|
- `--upload-dir=DIR` — Sandbox uploads to DIR; reject files outside it (v0.11.0)
|
|
43
|
+
- `--allow-hosts=a.com,*.b.org` — Site lock: top-level navigation only to these hosts (`*.host` = subdomains); others fail with `NAV_BLOCKED`. Subresources and iframes are not locked. Also on `browse`.
|
|
43
44
|
|
|
44
45
|
> Security (v0.11.0): `file:`/`chrome:`/etc. navigation is blocked by default, and the daemon requires a per-session token (handled transparently by the CLI). Snapshots and saved state are written owner-only (`0600`).
|
|
45
46
|
|
package/mcp-server.js
CHANGED
|
@@ -15,6 +15,8 @@ import { formatReadable } from './src/readable.js';
|
|
|
15
15
|
import { cleanOld } from './src/output-clean.js';
|
|
16
16
|
import { scopedCookiesForUrl } from './src/auth.js';
|
|
17
17
|
import { isHttpError } from './src/navigation.js';
|
|
18
|
+
import { overNote } from './src/zoom.js';
|
|
19
|
+
import { parseAllowHosts } from './src/site-lock.js';
|
|
18
20
|
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
|
|
19
21
|
import { join, dirname } from 'node:path';
|
|
20
22
|
import { pathToFileURL, fileURLToPath } from 'node:url';
|
|
@@ -121,6 +123,16 @@ export function saveSnapshot(text, { prefix = 'page', ext = 'yml' } = {}) {
|
|
|
121
123
|
}
|
|
122
124
|
|
|
123
125
|
let _page = null;
|
|
126
|
+
// A goto's load event is not the page's last request: the first snapshot after it waits
|
|
127
|
+
// once for the network to go quiet (idle 500ms, capped 3s), like consentRecheck does once
|
|
128
|
+
// per navigation. A goto followed by a click pays nothing. Library timing is unchanged.
|
|
129
|
+
let _settlePending = false;
|
|
130
|
+
async function settleOnce(page, settle) {
|
|
131
|
+
if (!_settlePending) return;
|
|
132
|
+
_settlePending = false;
|
|
133
|
+
if (settle === false) return;
|
|
134
|
+
try { await page.waitForNetworkIdle({ idle: 500, timeout: 3000 }); } catch { /* best-effort: a busy page is snapshotted as it is */ }
|
|
135
|
+
}
|
|
124
136
|
let _pageConnecting = null;
|
|
125
137
|
// Active engine: starts from BAREBROWSE_ENGINE, changed at runtime by switch_engine.
|
|
126
138
|
let _engine = process.env.BAREBROWSE_ENGINE === 'firefox' ? 'firefox' : 'chromium';
|
|
@@ -217,6 +229,58 @@ function engineInfo() {
|
|
|
217
229
|
};
|
|
218
230
|
}
|
|
219
231
|
|
|
232
|
+
// --- Access tier + site lock (env, fixed at server start) ---
|
|
233
|
+
//
|
|
234
|
+
// BAREBROWSE_TIER = r | rw | rwx (default rwx = everything). r: read the web
|
|
235
|
+
// (navigate, observe); rw: + interact (click/type/...); rwx: + upload and eval.
|
|
236
|
+
// Every tool has an explicit tier; a tool missing from the map is rwx, so a
|
|
237
|
+
// new tool is hidden from restricted tiers until someone classifies it.
|
|
238
|
+
const TIER_RANK = { r: 0, rw: 1, rwx: 2 };
|
|
239
|
+
export const TOOL_TIER = {
|
|
240
|
+
browse: 'r', goto: 'r', back: 'r', forward: 'r', reload: 'r', scroll: 'r',
|
|
241
|
+
snapshot: 'r', readable: 'r', screenshot: 'r', pdf: 'r', wait_for: 'r', tabs: 'r',
|
|
242
|
+
capabilities: 'r', assess: 'r',
|
|
243
|
+
click: 'rw', type: 'rw', press: 'rw', select: 'rw', drag: 'rw', hover: 'rw', switch_engine: 'rw',
|
|
244
|
+
upload: 'rwx', eval: 'rwx',
|
|
245
|
+
};
|
|
246
|
+
|
|
247
|
+
// MCP annotation hints, classified next to the tier map: read tools whose repeat call is a no-op,
|
|
248
|
+
// and state-changing tools that are still non-destructive.
|
|
249
|
+
const IDEMPOTENT_READS = new Set(['snapshot', 'readable', 'screenshot', 'pdf', 'tabs', 'capabilities', 'wait_for']);
|
|
250
|
+
const NON_DESTRUCTIVE = new Set(['hover', 'switch_engine']);
|
|
251
|
+
|
|
252
|
+
/** @param {string} name */
|
|
253
|
+
export function toolTier(name) { return TOOL_TIER[name] ?? 'rwx'; }
|
|
254
|
+
|
|
255
|
+
/** Parse BAREBROWSE_TIER. Unset/empty = rwx; anything else must be exact, or it throws (never widen on a typo). */
|
|
256
|
+
export function parseTier(v) {
|
|
257
|
+
if (v === undefined || v === '') return 'rwx';
|
|
258
|
+
if (!Object.hasOwn(TIER_RANK, v)) throw new Error(`BAREBROWSE_TIER must be r, rw or rwx (got ${JSON.stringify(v)})`);
|
|
259
|
+
return v;
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
/** Parse BAREBROWSE_ALLOW (comma list of hosts). Unset/blank = no lock (null); a bad entry throws. */
|
|
263
|
+
export function parseAllowEnv(v) {
|
|
264
|
+
if (v === undefined || !v.trim()) return null;
|
|
265
|
+
try { return parseAllowHosts(v.split(',')); }
|
|
266
|
+
catch (e) { throw new Error(`BAREBROWSE_ALLOW: ${e.message}`); }
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
/** Read + validate the tier/site-lock env. Throws on bad config. */
|
|
270
|
+
function readAccess() {
|
|
271
|
+
return { tier: parseTier(process.env.BAREBROWSE_TIER), allowHosts: parseAllowEnv(process.env.BAREBROWSE_ALLOW) };
|
|
272
|
+
}
|
|
273
|
+
|
|
274
|
+
/** MCP tool hints derived from the tier: r = read-only; rw/rwx change state (hover and switch_engine non-destructively). */
|
|
275
|
+
function annotationsFor(name) {
|
|
276
|
+
if (toolTier(name) === 'r') {
|
|
277
|
+
const a = { readOnlyHint: true, openWorldHint: name !== 'capabilities' };
|
|
278
|
+
if (IDEMPOTENT_READS.has(name)) a.idempotentHint = true;
|
|
279
|
+
return a;
|
|
280
|
+
}
|
|
281
|
+
return { readOnlyHint: false, destructiveHint: !NON_DESTRUCTIVE.has(name), openWorldHint: true };
|
|
282
|
+
}
|
|
283
|
+
|
|
220
284
|
async function getPage() {
|
|
221
285
|
if (_page) return _page;
|
|
222
286
|
if (_pageConnecting) return _pageConnecting;
|
|
@@ -230,9 +294,10 @@ async function getPage() {
|
|
|
230
294
|
// across tool calls, so it's an env var (BAREBROWSE_INCOGNITO=1), not a
|
|
231
295
|
// per-request flag. Skips all auth injection for a clean, logged-out session.
|
|
232
296
|
const incognito = process.env.BAREBROWSE_INCOGNITO === '1';
|
|
297
|
+
const { allowHosts } = readAccess();
|
|
233
298
|
const connectOpts = _engine === 'firefox'
|
|
234
|
-
? { engine: 'firefox', mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito }
|
|
235
|
-
: { mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito };
|
|
299
|
+
? { engine: 'firefox', mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito, allowHosts }
|
|
300
|
+
: { mode: process.env.BAREBROWSE_MODE || 'hybrid', incognito, allowHosts };
|
|
236
301
|
_pageConnecting = connect(connectOpts);
|
|
237
302
|
try {
|
|
238
303
|
_page = await _pageConnecting;
|
|
@@ -288,13 +353,20 @@ export const TOOLS = [
|
|
|
288
353
|
},
|
|
289
354
|
{
|
|
290
355
|
name: 'snapshot',
|
|
291
|
-
description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements.',
|
|
356
|
+
description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements. Big pages: start with map:true (an outline of the page\'s boxes: header, main, lists, tables… each with a ref and a size), then call again with within=<ref or name> to read just the box you need, in any pruneMode. A page over maxChars returns that map for you instead of the content (the full snapshot is also saved to a file); pass full:true to get the file path alone.',
|
|
292
357
|
inputSchema: {
|
|
293
358
|
type: 'object',
|
|
294
359
|
properties: {
|
|
295
360
|
pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If a previous snapshot looked empty on a content-heavy page, retry with "read". "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
|
|
296
361
|
links: { type: 'boolean', description: 'Show each link\'s absolute URL after its name: link "name" <url> [ref=N]. Default off.' },
|
|
297
|
-
|
|
362
|
+
map: { type: 'boolean', description: 'Return an outline of the page\'s big boxes instead of content: `- list "Issues" [ref=40] (14.9k chars)`, nested, one line each. Unnamed boxes show the heading before them: `list after "Other flights"`. Size is the box\'s text in "data" mode. A grid of alike cards (products, listings, videos) shows as `group ×18 (6.3k chars) [ref=N]`; within=<that ref> zooms it. Costs 1-2% of the page; use it to pick a box for within.' },
|
|
363
|
+
within: {
|
|
364
|
+
anyOf: [{ type: 'string' }, { type: 'number' }, { type: 'array', items: { anyOf: [{ type: 'string' }, { type: 'number' }] } }],
|
|
365
|
+
description: 'Zoom: return only this box, as if it were the whole page (works with every pruneMode and links). A ref from the map (e.g. 40; refs change on every snapshot, so use one from the latest map), a role (table), a role and its exact name (list "Issues"), or an unnamed box by the heading right before it (list after "Other flights"). An array returns several boxes, in page order. If two or more boxes match you get the map back with the matches marked; if none match, an error. A ref whose element is gone is an error: take a fresh map. Refs only hold until the next snapshot or navigation, and a ref from an older map can point at a different box; for anything you store or hardcode, use names (list "Issues", list after "Top flights"), which are stable across reloads.',
|
|
366
|
+
},
|
|
367
|
+
settle: { type: 'boolean', description: 'The first snapshot after a goto waits once for the network to go quiet (up to 3s, about 0.5s on a finished page) so late-loading content is in it. Pass false to snapshot immediately. Default: true.' },
|
|
368
|
+
full: { type: 'boolean', description: 'Skip the automatic map: a snapshot over maxChars is saved to a file and only the path is returned (the old behavior). Ignored when within or map is set.' },
|
|
369
|
+
maxChars: { type: 'number', description: 'Max chars to return inline. A larger snapshot is saved to .barebrowse/ in full and the map of the page is returned instead (or just the file path with full:true). Default: 30000.' },
|
|
298
370
|
},
|
|
299
371
|
},
|
|
300
372
|
},
|
|
@@ -517,12 +589,17 @@ if (assessFn) {
|
|
|
517
589
|
});
|
|
518
590
|
}
|
|
519
591
|
|
|
592
|
+
for (const t of TOOLS) t.annotations = annotationsFor(t.name);
|
|
593
|
+
|
|
520
594
|
async function handleToolCall(name, args) {
|
|
521
595
|
switch (name) {
|
|
522
596
|
case 'capabilities': {
|
|
523
597
|
// Introspection only — must NOT launch a browser. Report the configured
|
|
524
598
|
// engine from env; if a page is already connected, attach its live flags.
|
|
525
599
|
const info = engineInfo();
|
|
600
|
+
const access = readAccess();
|
|
601
|
+
info.tier = access.tier;
|
|
602
|
+
info.allowHosts = access.allowHosts;
|
|
526
603
|
if (_page) {
|
|
527
604
|
info.connected = true;
|
|
528
605
|
info.capabilities = _page.capabilities;
|
|
@@ -547,7 +624,7 @@ async function handleToolCall(name, args) {
|
|
|
547
624
|
case 'browse': {
|
|
548
625
|
let timer;
|
|
549
626
|
const text = await Promise.race([
|
|
550
|
-
browse(args.url, { mode: args.mode, pruneMode: args.pruneMode, incognito: args.incognito, links: args.links }),
|
|
627
|
+
browse(args.url, { mode: args.mode, pruneMode: args.pruneMode, incognito: args.incognito, links: args.links, allowHosts: readAccess().allowHosts ?? undefined }),
|
|
551
628
|
new Promise((_, rej) => { timer = setTimeout(() => rej(new Error('browse timed out after 60s')), 60000); }),
|
|
552
629
|
]);
|
|
553
630
|
clearTimeout(timer);
|
|
@@ -570,15 +647,25 @@ async function handleToolCall(name, args) {
|
|
|
570
647
|
_lastCookies = cookieReport(args.url, { cookies, error });
|
|
571
648
|
}
|
|
572
649
|
const { status } = await page.goto(args.url);
|
|
650
|
+
_settlePending = true;
|
|
573
651
|
return isHttpError(status) ? `ok (HTTP ${status})` : 'ok';
|
|
574
652
|
}, TIMEOUTS.goto);
|
|
575
653
|
case 'snapshot': return withRetry(async () => {
|
|
576
654
|
const page = await getPage();
|
|
577
|
-
|
|
655
|
+
await settleOnce(page, args.settle);
|
|
656
|
+
const zoomed = args.map || args.within != null;
|
|
657
|
+
const text = await page.snapshot(args.pruneMode || args.links || zoomed
|
|
658
|
+
? { mode: args.pruneMode, links: args.links, map: args.map, within: args.within } : undefined);
|
|
578
659
|
const limit = args.maxChars ?? MAX_CHARS_DEFAULT;
|
|
579
660
|
if (text.length > limit) {
|
|
580
661
|
const file = saveSnapshot(text);
|
|
581
|
-
|
|
662
|
+
const saved = `Snapshot (${text.length} chars) saved to ${file}`;
|
|
663
|
+
if (zoomed && !args.full) return `${saved}. ${overNote(args.within != null ? 'zoom' : 'map', text.length, limit)}`;
|
|
664
|
+
if (zoomed || args.full) return saved;
|
|
665
|
+
// Over the cap and the agent chose nothing: hand back the map (a few % of the page) to zoom from.
|
|
666
|
+
const map = await page.snapshot({ links: args.links, map: true });
|
|
667
|
+
if (map.length > limit) return saved;
|
|
668
|
+
return `${saved}. Over maxChars (${limit}), so here is the map of the page; call snapshot with within=<ref or name> to read one box (an array takes several), or full:true for the path alone.\n${map}`;
|
|
582
669
|
}
|
|
583
670
|
return text;
|
|
584
671
|
}, TIMEOUTS.snapshot);
|
|
@@ -771,6 +858,10 @@ async function handleMessage(msg) {
|
|
|
771
858
|
|
|
772
859
|
if (method === 'initialize') {
|
|
773
860
|
const e = engineInfo();
|
|
861
|
+
const { tier, allowHosts } = readAccess();
|
|
862
|
+
const access = (tier === 'r' ? ' Access tier: read-only (no clicking, typing or uploading).'
|
|
863
|
+
: tier === 'rw' ? ' Access tier: read/write (no upload or eval).' : '')
|
|
864
|
+
+ (allowHosts ? ` Navigation is restricted to: ${allowHosts.join(', ')}.` : '');
|
|
774
865
|
return jsonrpcResponse(id, {
|
|
775
866
|
protocolVersion: '2024-11-05',
|
|
776
867
|
capabilities: { tools: {} },
|
|
@@ -780,7 +871,7 @@ async function handleMessage(msg) {
|
|
|
780
871
|
instructions:
|
|
781
872
|
`barebrowse drives a real browser. Active engine: ${e.activeEngine} (mode: ${e.mode}). `
|
|
782
873
|
+ `Both chromium and firefox are supported; ${e.switchEngine} `
|
|
783
|
-
+ `Call the "capabilities" tool for the live engine + feature set
|
|
874
|
+
+ `Call the "capabilities" tool for the live engine + feature set.${access}`,
|
|
784
875
|
});
|
|
785
876
|
}
|
|
786
877
|
|
|
@@ -793,7 +884,8 @@ async function handleMessage(msg) {
|
|
|
793
884
|
// active engine (and that both are supported) is visible without a call.
|
|
794
885
|
const e = engineInfo();
|
|
795
886
|
const note = ` [Active engine: ${e.activeEngine}; chromium + firefox both supported — ${e.switchEngine}]`;
|
|
796
|
-
const
|
|
887
|
+
const { tier } = readAccess();
|
|
888
|
+
const tools = TOOLS.filter((t) => TIER_RANK[toolTier(t.name)] <= TIER_RANK[tier]).map((t) =>
|
|
797
889
|
t.name === 'browse' ? { ...t, description: t.description + note } : t);
|
|
798
890
|
return jsonrpcResponse(id, { tools });
|
|
799
891
|
}
|
|
@@ -803,6 +895,8 @@ async function handleMessage(msg) {
|
|
|
803
895
|
_inflight++;
|
|
804
896
|
clearTimeout(_idleTimer);
|
|
805
897
|
try {
|
|
898
|
+
// A tool above the tier answers exactly like one that doesn't exist.
|
|
899
|
+
if (TIER_RANK[toolTier(name)] > TIER_RANK[readAccess().tier]) throw new Error(`Unknown tool: ${name}`);
|
|
806
900
|
const result = await handleToolCall(name, args || {});
|
|
807
901
|
return jsonrpcResponse(id, {
|
|
808
902
|
content: [{ type: 'text', text: typeof result === 'string' ? result : JSON.stringify(result) }],
|
|
@@ -833,6 +927,11 @@ async function handleMessage(msg) {
|
|
|
833
927
|
// runStdio() explicitly. Tests import TIMEOUTS/TOOLS without calling it.
|
|
834
928
|
|
|
835
929
|
export function runStdio() {
|
|
930
|
+
// Fail closed: a bad tier or allow list must stop the server, never fall back to rwx / no lock.
|
|
931
|
+
try { readAccess(); } catch (e) {
|
|
932
|
+
process.stderr.write(`barebrowse mcp: ${e.message}\n`);
|
|
933
|
+
process.exit(1);
|
|
934
|
+
}
|
|
836
935
|
cleanOld(OUTPUT_DIR);
|
|
837
936
|
// One-line startup banner to stderr (stderr because stdout is the JSON-RPC
|
|
838
937
|
// channel — must not contain non-JSON-RPC bytes). Captured by Claude Code's
|