barebrowse 0.27.0 → 0.28.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +70 -0
- package/README.md +2 -0
- package/barebrowse.context.md +26 -2
- package/cli.js +19 -1
- package/commands/barebrowse/SKILL.md +5 -0
- package/mcp-server.js +39 -6
- package/package.json +1 -1
- package/src/auth.js +25 -7
- package/src/ax-snapshot.js +24 -5
- package/src/bidi.js +3 -2
- package/src/daemon.js +25 -2
- package/src/data.js +2 -2
- package/src/firefox-page.js +116 -23
- package/src/index.js +105 -26
- package/src/prune.js +3 -2
- package/src/zoom.js +250 -0
- package/types/auth.d.ts +26 -0
- package/types/data.d.ts +2 -1
- package/types/firefox-page.d.ts +2 -2
- package/types/index.d.ts +12 -3
- package/types/prune.d.ts +2 -0
- package/types/zoom.d.ts +51 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,75 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.28.0] - 2026-11-16
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Map + zoom for `snapshot()`.** `snapshot({ map: true })` returns an outline of
|
|
8
|
+
the page's big boxes (landmarks, lists, tables, grids, feeds, tabpanels,
|
|
9
|
+
dialogs) with a ref and a size each; `snapshot({ within })` returns one box, or
|
|
10
|
+
several in page order, as if it were the whole page: a ref from the map, a
|
|
11
|
+
role, `role "name"`, or `role after "heading"` for an unnamed box. Works in
|
|
12
|
+
every mode and with `links`, on both engines. Two or more matches return the
|
|
13
|
+
map with the matches marked; none, or a ref whose element is gone, throws. The
|
|
14
|
+
library default is unchanged (byte for byte). MCP `snapshot` gets `map`,
|
|
15
|
+
`within` and `full`; a snapshot over `maxChars` now also returns the map
|
|
16
|
+
inline (the full snapshot is still saved to a file). CLI: `snapshot --map`,
|
|
17
|
+
`--within=<box>` (repeatable), `--full`, `--max-chars=N`.
|
|
18
|
+
- **Map lists card groups.** A grid of alike cards (products, listings, videos)
|
|
19
|
+
that no landmark, list or table role marks now shows in the map as
|
|
20
|
+
`- group ×18 (6.3k chars) [ref=N]`, nested where it sits; `within: <ref>` zooms
|
|
21
|
+
to it and `within: 'group'` selects the groups. Found by structure only (4 or
|
|
22
|
+
more siblings of one role with similar shape and real text), never inside
|
|
23
|
+
banner/navigation/contentinfo, never for a list or table. Only the map and
|
|
24
|
+
`within` output change; default, act, read and data snapshots are unchanged.
|
|
25
|
+
- **MCP and CLI settle once before the first snapshot after a goto.** A page's
|
|
26
|
+
load event is not its last request (Airbnb on Firefox was incomplete 5 of 5
|
|
27
|
+
times; Flights about 1 in 5). The first `snapshot` after a `goto` now waits for
|
|
28
|
+
the network to go quiet (idle 500ms, capped at 3s; about 0.5s on a finished
|
|
29
|
+
page), best-effort: a timeout is not an error. A goto followed by a click pays
|
|
30
|
+
nothing. MCP `snapshot` takes `settle: false` and the CLI `snapshot --no-settle`
|
|
31
|
+
to skip it. The library (`connect()`, `browse()`) is unchanged.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- **Chromium: `goto()` to a hash-only change took 30s.** `goto('/ok#a')` then
|
|
36
|
+
`goto('/ok#b')` waited for a load event that never fires; it now returns at once
|
|
37
|
+
with `{ url, status: 0 }`, as Firefox does.
|
|
38
|
+
- **Chromium: refs die when a click or key press navigates to a new document.**
|
|
39
|
+
An old ref after a link click used to hit whatever element now had that number;
|
|
40
|
+
it now throws, like after `goto`. Hash links and `pushState` keep refs valid.
|
|
41
|
+
- **Firefox: clicks on elements below the fold.** `click()`, `hover()` and
|
|
42
|
+
`drag()` threw `move target out of bounds`, so Google's consent interstitial
|
|
43
|
+
was never dismissed on Firefox. The target is scrolled into view first.
|
|
44
|
+
- **Chromium: `goBack()` / `goForward()` took 30s** when the traversal restored
|
|
45
|
+
the page from the back/forward cache or landed on a same-document history
|
|
46
|
+
entry. They now return when the traversal lands.
|
|
47
|
+
- **Chromium: numeric refs.** `click`, `type`, `hover`, `select`, `drag` and
|
|
48
|
+
`upload` accept a ref as a number as well as a string.
|
|
49
|
+
- **Firefox: refs from before a navigation no longer resolve to another
|
|
50
|
+
element.** A stale ref now throws, as on Chromium, including after a page
|
|
51
|
+
script's `history.back()` restores the old document from the bfcache. The
|
|
52
|
+
token that tracks this lives in an isolated BiDi sandbox realm, out of reach
|
|
53
|
+
of page script.
|
|
54
|
+
- **Firefox: table cells and image-only links in act/read snapshots.** Cell,
|
|
55
|
+
column-header and row-header text was dropped, so tables pruned to their name;
|
|
56
|
+
an image-only link had no name (its `alt` was ignored), so data mode dropped
|
|
57
|
+
it. Both now match Chromium.
|
|
58
|
+
- **Firefox: consent accept that navigates.** The snapshot after the click no
|
|
59
|
+
longer reads the consent interstitial before the next page lands (1s settle,
|
|
60
|
+
as on Chromium).
|
|
61
|
+
- **Consent: `goto()` reports the page behind an accept that navigated.** The
|
|
62
|
+
resolved `{ url, status }` is the landed page, not the interstitial.
|
|
63
|
+
- **MCP and CLI: a zoom over `maxChars` says why it came back as a file path**,
|
|
64
|
+
instead of returning the path with no explanation.
|
|
65
|
+
- **Chromium cookie reuse works on KDE.** The KWallet password was
|
|
66
|
+
base64-decoded before key derivation, which made every encrypted (`v11`)
|
|
67
|
+
cookie fail to decrypt and get dropped (0 of 17 on a real jar). It is now used
|
|
68
|
+
as stored, as Chromium does.
|
|
69
|
+
- **Partitioned Chromium cookies are skipped.** A partitioned (CHIPS) cookie
|
|
70
|
+
sharing name, domain and path with the top-level one could shadow it on merge,
|
|
71
|
+
as the Firefox reader already avoided.
|
|
72
|
+
|
|
3
73
|
## [0.27.0] - 2026-10-30
|
|
4
74
|
|
|
5
75
|
### Added
|
package/README.md
CHANGED
|
@@ -106,6 +106,8 @@ MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `
|
|
|
106
106
|
|
|
107
107
|
`links: true` (opt-in, `snapshot({ links: true })`, `browse(url, { links: true })`, or the `connect({ links: true })` default; MCP/bareagent `links` param, CLI `snapshot --links` / `browse <url> --links`) adds each link's absolute URL right after its quoted name: `- link "Fix login bug" <https://github.com/o/r/issues/42> [ref=12]`. Works in every mode; a heading that wraps a link keeps the URL on its own line (`heading "title" <url> [level=3] [ref=N]`); data mode also puts it after link cells in table rows and each item of a collapsed group (`Labels: bug <url> [ref=1], docs <url> [ref=2]`) and its legend gains ` Links: "name" <url>.`. `javascript:` and same-page `#fragment` links get no URL, every link line, row cell and group item carries its own URL (nothing is deduped across them, even when two links share a URL), and URLs never contain a raw space, `<` or `>`. Off by default: output is unchanged and costs nothing (about +23% chars on a GitHub issue list in data mode).
|
|
108
108
|
|
|
109
|
+
**Map + zoom** (big or cluttered pages). `snapshot({ map: true })` returns an outline of the page's big boxes instead of content, one line each, nested, with a ref and the size of its text in data mode: `- main [ref=5] (21.3k chars)`, ` - list "Issues" [ref=40] (14.9k chars)`, ` - list after "Other flights" [ref=88] (3.1k chars)` (an unnamed box shows the heading before it). Boxes: banner, navigation, main, complementary, contentinfo, region, form, search, list, table, grid, treegrid, feed, tablist, tabpanel, dialog; only those of at least 800 chars are listed. Card groups are listed too: a parent whose substantial children are mostly alike (a grid of product, listing or video cards that no role marks; found by structure only: 4 or more cards of one role and similar shape, each with real text) shows as `- group ×18 (6.3k chars) [ref=N]` where it sits (size = the cards' text; ref = their parent's, so `within: N` zooms the cards and anything beside them in that parent; when the cards sit straight in `main` that is main's ref). Not looked for inside banner, navigation or contentinfo, and not for a list or table (already boxes). A bare `within: 'group'` selects the card groups (two or more: the map with `<- match`; with none on the page, it matches real `group` nodes as before). No name or position selector: use the ref. `snapshot({ within })` then returns just one box, as if it were the whole page (prune's landmark step and data mode's main-only scope are off inside it; every mode and `links` work): `within: 40` (a ref from the map), `'table'` (role), `'list "Issues"'` (role + exact accessible name), `'list after "Other flights"'` (an unnamed box by the heading right before it, which survives a reload), or an array of those for several boxes, output in page order. Two or more matches return the map with the matches marked `<- match` (never picks one); none match, or a ref whose element is gone, throws. A ref in `within` is a ref from your previous snapshot (refs renumber on every snapshot and reset on navigation), found again by its element, not its number; refs printed in a zoom are clickable. Refs only hold until the next snapshot or navigation (including one a click or key press caused: Chromium checks the page's document identity at every ref use, so a hash link or `pushState` keeps refs, a new page kills them), and a ref from an older map can point at a different box; for anything you store or hardcode, use names (`list "Issues"`, `list after "Top flights"`), which are stable across reloads. `within` wins over `map`. The library default is unchanged: no `map`/`within` gives today's output byte for byte (so a consumer that freezes the whole page text is unaffected). MCP `snapshot` takes `map`, `within` (string, number or array) and `full`, and a snapshot over `maxChars` returns the map inline while still saving the full snapshot to `.barebrowse/` (`full: true` keeps the path-only answer); MCP and the CLI daemon wait once for the network to go quiet (idle 500ms, up to 3s) before the first snapshot after a `goto`, so late-loading content is in it (`settle: false` / `--no-settle` skip it; the library is unchanged). CLI `snapshot --map`, `--within=<box>` (repeat the flag for several boxes; quote a name: `--within='list "Issues"'`), `--full`, `--max-chars=N` (default 30000: over it, the map is printed after the file path). `browse()` and the bareagent adapter are unchanged. A table keeps its cell text in act/read mode on both engines.
|
|
110
|
+
|
|
109
111
|
`npx barebrowse doctor` reports the runtime environment first — default engine/mode, installed Chromium/Firefox binaries, cookie sources, and any live session's engine + capabilities — then scans every known MCP config location and flags scope conflicts. `npx barebrowse install --force` overwrites an existing entry pointing at a different endpoint.
|
|
110
112
|
|
|
111
113
|
### 3. Library -- for agentic automation
|
package/barebrowse.context.md
CHANGED
|
@@ -71,7 +71,7 @@ const snapshot = await browse('https://example.com', {
|
|
|
71
71
|
| `goBack()` | -- | void | Navigate back in browser history |
|
|
72
72
|
| `goForward()` | -- | void | Navigate forward in browser history |
|
|
73
73
|
| `reload(opts?)` | { ignoreCache?: boolean, timeout?: number } | void | Reload the current page. Clears refMap (refs from pre-reload reject). |
|
|
74
|
-
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data', links?: boolean } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. `links: true` adds `<absolute-url>` after each link name (see below). |
|
|
74
|
+
| `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data', links?: boolean, map?: boolean, within?: string\|number\|Array } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. `links: true` adds `<absolute-url>` after each link name (see below). `map: true` returns an outline of the page's big boxes (ref + size); `within` zooms to one box or several (see Map + zoom below). |
|
|
75
75
|
| `readable()` | -- | object | Clean article text (Reader-View engine). `{ ok, title, byline, text, length, confidence: 'high'\|'low', readerable, hint? }` or `{ ok: false, hint }`. For *reading*, not interacting — see note below. |
|
|
76
76
|
| `click(ref)` | ref: string | void | Scroll into view + mouse press+release at center |
|
|
77
77
|
| `type(ref, text, opts?)` | ref: string, text: string, opts: { clear?, keyEvents? } | void | Focus + insert text. `clear: true` replaces existing. |
|
|
@@ -133,6 +133,28 @@ Key rules:
|
|
|
133
133
|
- `click(ref)` / `type(ref, text)` / `hover(ref)` / `select(ref, value)` use these ref strings
|
|
134
134
|
- Pruning removes noise (~47-95% token reduction) while keeping all interactive elements
|
|
135
135
|
|
|
136
|
+
## Map + zoom — big pages
|
|
137
|
+
|
|
138
|
+
Order for a big or cluttered page: map, then zoom, then mode. The library default is unchanged (no `map`/`within` = today's output, byte for byte).
|
|
139
|
+
|
|
140
|
+
```js
|
|
141
|
+
await page.snapshot({ map: true }); // outline of the big boxes, no content
|
|
142
|
+
await page.snapshot({ within: 40 }); // one box by ref from the map
|
|
143
|
+
await page.snapshot({ within: 'table' }); // by role
|
|
144
|
+
await page.snapshot({ within: 'list "Issues"' }); // role + exact accessible name
|
|
145
|
+
await page.snapshot({ within: 'list after "Other flights"' }); // unnamed box: the heading right before it (survives a reload)
|
|
146
|
+
await page.snapshot({ within: [430, 'table'], mode: 'data', links: true }); // several boxes, page order; any mode/links
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
- Map line: `- role "name" [ref=N] (14.9k chars)`, nested; unnamed boxes show `after "<heading>"`. Size = the box's text in data mode. Boxes: banner, navigation, main, complementary, contentinfo, region, form, search, list, table, grid, treegrid, feed, tablist, tabpanel, dialog; listed from 800 chars.
|
|
150
|
+
- Inside a zoom the box is the whole page: no landmark pruning, no main-only scope in data mode; everything else prunes as usual.
|
|
151
|
+
- Card groups: a grid of alike cards no role marks (products, listings, videos; structural detection, 4+ alike siblings) is listed as `- group ×18 (6.3k chars) [ref=N]`; zoom it by that ref (it is the cards' parent, so cards straight in `main` give main's ref). `within: 'group'` selects the card groups (several: map with `<- match`). Not inside banner/navigation/contentinfo, not for a list or table. No name selector.
|
|
152
|
+
- Match is exact on role and accessible name (or the box's preceding sibling heading text). Two or more matches: the map comes back with `<- match` on them, nothing is picked. Zero: throws, naming what was asked.
|
|
153
|
+
- A ref in `within` is a ref from your previous snapshot; it is found again by its element (refs renumber every snapshot). Gone element or a ref from before a navigation: throws. Refs printed in a zoom work with `click()` etc. Refs only hold until the next snapshot or navigation, and a ref from an older map can point at a different box; for anything you store or hardcode, use names (`list "Issues"`, `list after "Top flights"`), which are stable across reloads. `within` wins over `map`.
|
|
154
|
+
- Firefox act/read keep a table's cells (text under each cell), as Chromium does; `mode: 'data'` still gives one line per row.
|
|
155
|
+
- `browse()` and `createBrowseTools()` (bareagent) have no map/zoom.
|
|
156
|
+
- MCP: `snapshot` takes `map`, `within`, `full`; over `maxChars` it returns the map inline and saves the full snapshot to a file (`full: true` = path only). CLI: `snapshot --map`, `--within=<box>` (repeatable; quote names), `--full`, `--max-chars=N`. MCP and the CLI daemon wait once (network idle 500ms, up to 3s) before the first snapshot after a `goto` so late content is in it; `settle: false` / `--no-settle` skip it; the library is unchanged.
|
|
157
|
+
|
|
136
158
|
## readable() vs snapshot() — which to use
|
|
137
159
|
|
|
138
160
|
Two different jobs:
|
|
@@ -254,6 +276,8 @@ For coding agents (Claude Code, Copilot, Cursor) and quick interactive testing.
|
|
|
254
276
|
```bash
|
|
255
277
|
barebrowse open https://example.com # Start daemon + navigate
|
|
256
278
|
barebrowse snapshot # → .barebrowse/page-<timestamp>.yml
|
|
279
|
+
barebrowse snapshot --map # outline of the page's big boxes (ref + size)
|
|
280
|
+
barebrowse snapshot --within='list "Issues"' # one box only (ref, role, role "name", role after "heading"; repeat for several)
|
|
257
281
|
barebrowse readable # → .barebrowse/article-<timestamp>.txt (clean article text)
|
|
258
282
|
barebrowse click 8 # Click element ref=8
|
|
259
283
|
barebrowse type 12 hello world # Type into element ref=12
|
|
@@ -303,7 +327,7 @@ barebrowse ships an MCP server for direct use with Claude Desktop, Cursor, or an
|
|
|
303
327
|
|
|
304
328
|
Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe. This avoids double-token output since MCP tool calls are cheap to chain.
|
|
305
329
|
|
|
306
|
-
`browse`, `snapshot`, and `readable` accept a `maxChars` param (default 30000). If the output exceeds the limit it's saved to `.barebrowse/` and a short message with the file path is returned instead (`page-<timestamp>.yml` for snapshots, `article-<timestamp>.txt` for `readable`). `screenshot` always saves to `.barebrowse/screenshot-<timestamp>.{png,jpeg,webp}` and returns the file path (raw base64 in a JSON-RPC response would blow `maxChars`). `tabs` returns the JSON array, or with `switchTo: N` it switches and returns `'ok'`. All files MCP writes are owner-only (`0600` in a `0700` dir) — they can hold authenticated page content, so they're not world-readable on a shared host.
|
|
330
|
+
`browse`, `snapshot`, and `readable` accept a `maxChars` param (default 30000). If the output exceeds the limit it's saved to `.barebrowse/` and a short message with the file path is returned instead (`page-<timestamp>.yml` for snapshots, `article-<timestamp>.txt` for `readable`). The exception is `snapshot` with neither `within`, `map` nor `full`: it also returns the page map inline, to zoom from with `within` (see Map + zoom); `full: true` gives the path alone. `browse` is unchanged (one-shot: nothing to zoom afterwards). `screenshot` always saves to `.barebrowse/screenshot-<timestamp>.{png,jpeg,webp}` and returns the file path (raw base64 in a JSON-RPC response would blow `maxChars`). `tabs` returns the JSON array, or with `switchTo: N` it switches and returns `'ok'`. All files MCP writes are owner-only (`0600` in a `0700` dir) — they can hold authenticated page content, so they're not world-readable on a shared host.
|
|
307
331
|
|
|
308
332
|
`browse` and `snapshot` also accept `pruneMode: 'act'|'read'|'data'`. `act` (the default) keeps interactive elements and short labels — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. `data` (opt-in) is one line per record — tables with headers (`row "Label": Header: value · …`), repeated items grouped (`Label: a, b, c`) — for data pages (timetables, listings, results), not articles; every data snapshot starts with ONE legend line (`# data mode: one line per record. Tables: row "<label>": <column>: <value> · … Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode "act" for menus and page controls.`) so you can read it without docs; it keeps `[ref=N]` on interactive nodes (clickable like act) and is *larger* than act on prose pages — use `read`/`readable()` there. Same surface on the bareagent adapter. If act mode collapses a content-heavy page (raw > 5 KB → pruned < 500 chars AND < 5% of raw), the result includes a `hint: act mode dropped most of the page — retry with pruneMode='read' …` line between the stats and the tree so the caller knows to re-snapshot in read mode instead of bailing to a separate HTTP fetch.
|
|
309
333
|
|
package/cli.js
CHANGED
|
@@ -38,7 +38,14 @@ if (args.includes('--daemon-internal')) {
|
|
|
38
38
|
} else if (cmd === 'goto' && args[1]) {
|
|
39
39
|
await cmdProxy('goto', { url: args[1], timeout: parseFlag('--timeout') });
|
|
40
40
|
} else if (cmd === 'snapshot') {
|
|
41
|
-
|
|
41
|
+
const within = parseFlagAll('--within');
|
|
42
|
+
await cmdProxy('snapshot', {
|
|
43
|
+
mode: parseFlag('--mode'), links: hasFlag('--links') || undefined,
|
|
44
|
+
within: within && (within.length === 1 ? within[0] : within),
|
|
45
|
+
map: hasFlag('--map') || undefined, full: hasFlag('--full') || undefined,
|
|
46
|
+
maxChars: parseFlag('--max-chars') && Number(parseFlag('--max-chars')),
|
|
47
|
+
settle: hasFlag('--no-settle') ? false : undefined,
|
|
48
|
+
});
|
|
42
49
|
} else if (cmd === 'readable') {
|
|
43
50
|
await cmdProxy('readable');
|
|
44
51
|
} else if (cmd === 'screenshot') {
|
|
@@ -177,6 +184,10 @@ async function cmdProxy(command, cmdArgs) {
|
|
|
177
184
|
// Print result
|
|
178
185
|
if (result.file && result.count !== undefined) {
|
|
179
186
|
process.stdout.write(`${result.file} (${result.count} entries)\n`);
|
|
187
|
+
} else if (result.file && result.map) {
|
|
188
|
+
process.stdout.write(`${result.file}\nOver --max-chars, so here is the map of the page; zoom with: snapshot --within=<ref or name> (repeat the flag for several boxes), or --full for the path alone.\n${result.map}\n`);
|
|
189
|
+
} else if (result.file && result.note) {
|
|
190
|
+
process.stdout.write(`${result.file}\n${result.note}\n`);
|
|
180
191
|
} else if (result.file) {
|
|
181
192
|
process.stdout.write(`${result.file}\n`);
|
|
182
193
|
} else if (result.value !== undefined) {
|
|
@@ -604,6 +615,13 @@ Navigation:
|
|
|
604
615
|
barebrowse forward Go forward in history
|
|
605
616
|
barebrowse reload [--no-cache] Reload current page
|
|
606
617
|
barebrowse snapshot [--mode=M] [--links] ARIA snapshot -> .barebrowse/page-*.yml (--links: link URLs)
|
|
618
|
+
--map Outline of the page's big boxes (ref + size) instead of content
|
|
619
|
+
--within=<box> Zoom to one box, as if it were the page: a ref from the map, a role (table),
|
|
620
|
+
role "name" (--within='list "Issues"'), or role after "heading". Repeat the flag
|
|
621
|
+
for several boxes (page order). Works with --mode and --links.
|
|
622
|
+
--full Skip the auto map. A snapshot over --max-chars (default 30000) is saved
|
|
623
|
+
in full and the map is printed after the path, to zoom from.
|
|
624
|
+
--no-settle Don't wait for the network to go quiet (up to 3s) before the first snapshot after a goto
|
|
607
625
|
barebrowse readable Clean article text -> .barebrowse/article-*.txt
|
|
608
626
|
barebrowse screenshot [--format] Screenshot -> .barebrowse/screenshot-*.png
|
|
609
627
|
barebrowse pdf [--landscape] PDF export -> .barebrowse/page-*.pdf
|
|
@@ -56,6 +56,10 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
|
|
|
56
56
|
| `barebrowse snapshot --mode=read` | Read mode: keeps all text (for content extraction) |
|
|
57
57
|
| `barebrowse snapshot --mode=data` | Data mode: one line per record (table rows with headers, repeated items grouped). For data pages, not articles. Main content only; use `act` for menus and page controls. |
|
|
58
58
|
| `barebrowse snapshot --links` | Add each link's absolute URL after its name: `link "title" <https://…> [ref=N]` (combine with `--mode`; also on `browse <url> --links`). `javascript:` and same-page `#` links get none. |
|
|
59
|
+
| `barebrowse snapshot --map` | Outline of the page's big boxes (lists, tables, main, …), each with a ref and a size. Cheap (1-2% of the page): use it on big pages, then zoom. A grid of alike cards (products, listings, videos) shows as `group ×18 (6.3k chars) [ref=N]`: zoom it with `--within=<that ref>`. |
|
|
60
|
+
| `barebrowse snapshot --within=<box>` | Only that box, as if it were the page. `<box>` = a ref from the map, a role (`table`), `role "name"` (`--within='list "Issues"'`), or an unnamed box by its heading (`--within='list after "Other flights"'`). Repeat the flag for several boxes (page order). Combine with `--mode` / `--links`. Several matches: the map comes back with the matches marked; none: error. |
|
|
61
|
+
| `barebrowse snapshot --no-settle` | The first snapshot after a `goto` waits once for the network to go quiet (up to 3s) so late-loading content is in it; this skips the wait. |
|
|
62
|
+
| `barebrowse snapshot --full` | A snapshot over `--max-chars` (default 30000) is saved in full and the map is printed after the path; `--full` prints the path alone. |
|
|
59
63
|
| `barebrowse readable` | Clean article text → `.barebrowse/article-<timestamp>.txt`. For reading/summarising articles (news, blogs, docs); falls back to snapshot on non-articles. |
|
|
60
64
|
| `barebrowse screenshot` | Screenshot → `.barebrowse/screenshot-<timestamp>.png` |
|
|
61
65
|
| `barebrowse pdf [--landscape]` | PDF export → `.barebrowse/page-<timestamp>.pdf` |
|
|
@@ -129,6 +133,7 @@ The snapshot is a YAML-like ARIA tree. Each line is one node:
|
|
|
129
133
|
|
|
130
134
|
- **Always snapshot before interacting** — refs are ephemeral and change every time
|
|
131
135
|
- **Use `fill` instead of `type`** when replacing existing text in input fields
|
|
136
|
+
- **On a big page, `snapshot --map` then `snapshot --within=<ref or name>`** instead of reading the whole file; refs in `--within` come from the last snapshot
|
|
132
137
|
- **Use `--mode=read`** for snapshot when you need to extract article content or data
|
|
133
138
|
- **Use `back`/`forward`** to navigate browser history instead of re-entering URLs
|
|
134
139
|
- **Use `upload`** for file inputs — pass absolute paths to the files
|
package/mcp-server.js
CHANGED
|
@@ -15,6 +15,7 @@ import { formatReadable } from './src/readable.js';
|
|
|
15
15
|
import { cleanOld } from './src/output-clean.js';
|
|
16
16
|
import { scopedCookiesForUrl } from './src/auth.js';
|
|
17
17
|
import { isHttpError } from './src/navigation.js';
|
|
18
|
+
import { overNote } from './src/zoom.js';
|
|
18
19
|
import { parseAllowHosts } from './src/site-lock.js';
|
|
19
20
|
import { mkdirSync, writeFileSync, readFileSync } from 'node:fs';
|
|
20
21
|
import { join, dirname } from 'node:path';
|
|
@@ -122,6 +123,16 @@ export function saveSnapshot(text, { prefix = 'page', ext = 'yml' } = {}) {
|
|
|
122
123
|
}
|
|
123
124
|
|
|
124
125
|
let _page = null;
|
|
126
|
+
// A goto's load event is not the page's last request: the first snapshot after it waits
|
|
127
|
+
// once for the network to go quiet (idle 500ms, capped 3s), like consentRecheck does once
|
|
128
|
+
// per navigation. A goto followed by a click pays nothing. Library timing is unchanged.
|
|
129
|
+
let _settlePending = false;
|
|
130
|
+
async function settleOnce(page, settle) {
|
|
131
|
+
if (!_settlePending) return;
|
|
132
|
+
_settlePending = false;
|
|
133
|
+
if (settle === false) return;
|
|
134
|
+
try { await page.waitForNetworkIdle({ idle: 500, timeout: 3000 }); } catch { /* best-effort: a busy page is snapshotted as it is */ }
|
|
135
|
+
}
|
|
125
136
|
let _pageConnecting = null;
|
|
126
137
|
// Active engine: starts from BAREBROWSE_ENGINE, changed at runtime by switch_engine.
|
|
127
138
|
let _engine = process.env.BAREBROWSE_ENGINE === 'firefox' ? 'firefox' : 'chromium';
|
|
@@ -233,6 +244,11 @@ export const TOOL_TIER = {
|
|
|
233
244
|
upload: 'rwx', eval: 'rwx',
|
|
234
245
|
};
|
|
235
246
|
|
|
247
|
+
// MCP annotation hints, classified next to the tier map: read tools whose repeat call is a no-op,
|
|
248
|
+
// and state-changing tools that are still non-destructive.
|
|
249
|
+
const IDEMPOTENT_READS = new Set(['snapshot', 'readable', 'screenshot', 'pdf', 'tabs', 'capabilities', 'wait_for']);
|
|
250
|
+
const NON_DESTRUCTIVE = new Set(['hover', 'switch_engine']);
|
|
251
|
+
|
|
236
252
|
/** @param {string} name */
|
|
237
253
|
export function toolTier(name) { return TOOL_TIER[name] ?? 'rwx'; }
|
|
238
254
|
|
|
@@ -259,10 +275,10 @@ function readAccess() {
|
|
|
259
275
|
function annotationsFor(name) {
|
|
260
276
|
if (toolTier(name) === 'r') {
|
|
261
277
|
const a = { readOnlyHint: true, openWorldHint: name !== 'capabilities' };
|
|
262
|
-
if (
|
|
278
|
+
if (IDEMPOTENT_READS.has(name)) a.idempotentHint = true;
|
|
263
279
|
return a;
|
|
264
280
|
}
|
|
265
|
-
return { readOnlyHint: false, destructiveHint: !
|
|
281
|
+
return { readOnlyHint: false, destructiveHint: !NON_DESTRUCTIVE.has(name), openWorldHint: true };
|
|
266
282
|
}
|
|
267
283
|
|
|
268
284
|
async function getPage() {
|
|
@@ -337,13 +353,20 @@ export const TOOLS = [
|
|
|
337
353
|
},
|
|
338
354
|
{
|
|
339
355
|
name: 'snapshot',
|
|
340
|
-
description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements.',
|
|
356
|
+
description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements. Big pages: start with map:true (an outline of the page\'s boxes: header, main, lists, tables… each with a ref and a size), then call again with within=<ref or name> to read just the box you need, in any pruneMode. A page over maxChars returns that map for you instead of the content (the full snapshot is also saved to a file); pass full:true to get the file path alone.',
|
|
341
357
|
inputSchema: {
|
|
342
358
|
type: 'object',
|
|
343
359
|
properties: {
|
|
344
360
|
pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If a previous snapshot looked empty on a content-heavy page, retry with "read". "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
|
|
345
361
|
links: { type: 'boolean', description: 'Show each link\'s absolute URL after its name: link "name" <url> [ref=N]. Default off.' },
|
|
346
|
-
|
|
362
|
+
map: { type: 'boolean', description: 'Return an outline of the page\'s big boxes instead of content: `- list "Issues" [ref=40] (14.9k chars)`, nested, one line each. Unnamed boxes show the heading before them: `list after "Other flights"`. Size is the box\'s text in "data" mode. A grid of alike cards (products, listings, videos) shows as `group ×18 (6.3k chars) [ref=N]`; within=<that ref> zooms it. Costs 1-2% of the page; use it to pick a box for within.' },
|
|
363
|
+
within: {
|
|
364
|
+
anyOf: [{ type: 'string' }, { type: 'number' }, { type: 'array', items: { anyOf: [{ type: 'string' }, { type: 'number' }] } }],
|
|
365
|
+
description: 'Zoom: return only this box, as if it were the whole page (works with every pruneMode and links). A ref from the map (e.g. 40; refs change on every snapshot, so use one from the latest map), a role (table), a role and its exact name (list "Issues"), or an unnamed box by the heading right before it (list after "Other flights"). An array returns several boxes, in page order. If two or more boxes match you get the map back with the matches marked; if none match, an error. A ref whose element is gone is an error: take a fresh map. Refs only hold until the next snapshot or navigation, and a ref from an older map can point at a different box; for anything you store or hardcode, use names (list "Issues", list after "Top flights"), which are stable across reloads.',
|
|
366
|
+
},
|
|
367
|
+
settle: { type: 'boolean', description: 'The first snapshot after a goto waits once for the network to go quiet (up to 3s, about 0.5s on a finished page) so late-loading content is in it. Pass false to snapshot immediately. Default: true.' },
|
|
368
|
+
full: { type: 'boolean', description: 'Skip the automatic map: a snapshot over maxChars is saved to a file and only the path is returned (the old behavior). Ignored when within or map is set.' },
|
|
369
|
+
maxChars: { type: 'number', description: 'Max chars to return inline. A larger snapshot is saved to .barebrowse/ in full and the map of the page is returned instead (or just the file path with full:true). Default: 30000.' },
|
|
347
370
|
},
|
|
348
371
|
},
|
|
349
372
|
},
|
|
@@ -624,15 +647,25 @@ async function handleToolCall(name, args) {
|
|
|
624
647
|
_lastCookies = cookieReport(args.url, { cookies, error });
|
|
625
648
|
}
|
|
626
649
|
const { status } = await page.goto(args.url);
|
|
650
|
+
_settlePending = true;
|
|
627
651
|
return isHttpError(status) ? `ok (HTTP ${status})` : 'ok';
|
|
628
652
|
}, TIMEOUTS.goto);
|
|
629
653
|
case 'snapshot': return withRetry(async () => {
|
|
630
654
|
const page = await getPage();
|
|
631
|
-
|
|
655
|
+
await settleOnce(page, args.settle);
|
|
656
|
+
const zoomed = args.map || args.within != null;
|
|
657
|
+
const text = await page.snapshot(args.pruneMode || args.links || zoomed
|
|
658
|
+
? { mode: args.pruneMode, links: args.links, map: args.map, within: args.within } : undefined);
|
|
632
659
|
const limit = args.maxChars ?? MAX_CHARS_DEFAULT;
|
|
633
660
|
if (text.length > limit) {
|
|
634
661
|
const file = saveSnapshot(text);
|
|
635
|
-
|
|
662
|
+
const saved = `Snapshot (${text.length} chars) saved to ${file}`;
|
|
663
|
+
if (zoomed && !args.full) return `${saved}. ${overNote(args.within != null ? 'zoom' : 'map', text.length, limit)}`;
|
|
664
|
+
if (zoomed || args.full) return saved;
|
|
665
|
+
// Over the cap and the agent chose nothing: hand back the map (a few % of the page) to zoom from.
|
|
666
|
+
const map = await page.snapshot({ links: args.links, map: true });
|
|
667
|
+
if (map.length > limit) return saved;
|
|
668
|
+
return `${saved}. Over maxChars (${limit}), so here is the map of the page; call snapshot with within=<ref or name> to read one box (an array takes several), or full:true for the path alone.\n${map}`;
|
|
636
669
|
}
|
|
637
670
|
return text;
|
|
638
671
|
}, TIMEOUTS.snapshot);
|
package/package.json
CHANGED
package/src/auth.js
CHANGED
|
@@ -56,6 +56,16 @@ export function findFirefoxCookieDb() {
|
|
|
56
56
|
|
|
57
57
|
// --- Chromium cookie decryption (Linux) ---
|
|
58
58
|
|
|
59
|
+
/**
|
|
60
|
+
* kwallet-query output -> Chromium keyring password. Chromium feeds the stored
|
|
61
|
+
* string to PBKDF2 as-is (it is not base64 to decode).
|
|
62
|
+
* @param {string} stdout
|
|
63
|
+
* @returns {string}
|
|
64
|
+
*/
|
|
65
|
+
export function kwalletPassword(stdout) {
|
|
66
|
+
return stdout.trim();
|
|
67
|
+
}
|
|
68
|
+
|
|
59
69
|
/**
|
|
60
70
|
* Get Chromium encryption password from OS keyring.
|
|
61
71
|
* Tries KWallet (KDE) first, then GNOME keyring, then fallback.
|
|
@@ -63,11 +73,11 @@ export function findFirefoxCookieDb() {
|
|
|
63
73
|
function getChromiumPassword() {
|
|
64
74
|
// KDE / KWallet
|
|
65
75
|
try {
|
|
66
|
-
const
|
|
76
|
+
const pw = kwalletPassword(execSync(
|
|
67
77
|
'kwallet-query -r "Chromium Safe Storage" -f "Chromium Keys" kdewallet',
|
|
68
78
|
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }
|
|
69
|
-
)
|
|
70
|
-
if (
|
|
79
|
+
));
|
|
80
|
+
if (pw) return pw;
|
|
71
81
|
} catch { /* not KDE or no entry */ }
|
|
72
82
|
|
|
73
83
|
// GNOME keyring / libsecret
|
|
@@ -87,7 +97,7 @@ function getChromiumPassword() {
|
|
|
87
97
|
* Derive AES key from Chromium keyring password.
|
|
88
98
|
* Chrome Linux: PBKDF2-SHA1, salt='saltysalt', 1 iteration, 16-byte key.
|
|
89
99
|
*/
|
|
90
|
-
function deriveKey(password) {
|
|
100
|
+
export function deriveKey(password) {
|
|
91
101
|
return pbkdf2Sync(password, 'saltysalt', 1, 16, 'sha1');
|
|
92
102
|
}
|
|
93
103
|
|
|
@@ -97,7 +107,7 @@ function deriveKey(password) {
|
|
|
97
107
|
* @param {Buffer} aesKey - Derived AES key
|
|
98
108
|
* @returns {string} Decrypted cookie value
|
|
99
109
|
*/
|
|
100
|
-
function decryptCookie(encrypted, aesKey) {
|
|
110
|
+
export function decryptCookie(encrypted, aesKey) {
|
|
101
111
|
const buf = Buffer.from(encrypted);
|
|
102
112
|
if (buf.length === 0) return '';
|
|
103
113
|
|
|
@@ -121,7 +131,7 @@ function decryptCookie(encrypted, aesKey) {
|
|
|
121
131
|
* @param {string} [domain] - Filter by domain (e.g. '.github.com')
|
|
122
132
|
* @returns {Array<object>} Cookies in CDP Network.setCookie format
|
|
123
133
|
*/
|
|
124
|
-
function extractChromiumCookies(dbPath, domain) {
|
|
134
|
+
export function extractChromiumCookies(dbPath, domain) {
|
|
125
135
|
const password = getChromiumPassword();
|
|
126
136
|
const aesKey = deriveKey(password);
|
|
127
137
|
|
|
@@ -132,10 +142,18 @@ function extractChromiumCookies(dbPath, domain) {
|
|
|
132
142
|
CAST(expires_utc AS TEXT) AS expires_utc, is_secure, is_httponly, samesite
|
|
133
143
|
FROM cookies`;
|
|
134
144
|
const params = [];
|
|
145
|
+
// Skip partitioned (CHIPS) rows: a non-empty top_frame_site_key marks a cookie
|
|
146
|
+
// scoped to a top-level site, and it can share name+domain+path with the real
|
|
147
|
+
// one. Older Chromium schemas lack the column, so only filter when it exists.
|
|
148
|
+
const where = [];
|
|
149
|
+
if (db.prepare('PRAGMA table_info(cookies)').all().some((c) => c.name === 'top_frame_site_key')) {
|
|
150
|
+
where.push(`top_frame_site_key = ''`);
|
|
151
|
+
}
|
|
135
152
|
if (domain) {
|
|
136
|
-
|
|
153
|
+
where.push(`host_key LIKE ?`);
|
|
137
154
|
params.push(`%${domain}%`);
|
|
138
155
|
}
|
|
156
|
+
if (where.length) sql += ` WHERE ${where.join(' AND ')}`;
|
|
139
157
|
|
|
140
158
|
const stmt = db.prepare(sql);
|
|
141
159
|
const rows = params.length ? stmt.all(...params) : stmt.all();
|
package/src/ax-snapshot.js
CHANGED
|
@@ -49,6 +49,10 @@ const AX_FN = function reconstructAX(base, REF_ATTR) {
|
|
|
49
49
|
'tab', 'menuitem', 'option', 'treeitem', 'switch', 'checkbox', 'radio',
|
|
50
50
|
]);
|
|
51
51
|
|
|
52
|
+
// Table cells name from content like the roles above, but CDP also keeps
|
|
53
|
+
// their text as StaticText children; without them act/read prune the cell away.
|
|
54
|
+
const KEEP_TEXT_CHILDREN = new Set(['cell', 'columnheader', 'rowheader']);
|
|
55
|
+
|
|
52
56
|
// HTML tag → implicit ARIA role (the common, high-value subset).
|
|
53
57
|
const TAG_ROLE = {
|
|
54
58
|
A: (el) => (el.hasAttribute('href') ? 'link' : 'generic'),
|
|
@@ -159,6 +163,23 @@ const AX_FN = function reconstructAX(base, REF_ATTR) {
|
|
|
159
163
|
return '';
|
|
160
164
|
}
|
|
161
165
|
|
|
166
|
+
// Descendant text for name-from-content, with an <img>'s alt standing in for
|
|
167
|
+
// the image (accname step 2F), so an image-only link is named like in CDP.
|
|
168
|
+
function contentName(el) {
|
|
169
|
+
let out = '';
|
|
170
|
+
const add = (n) => {
|
|
171
|
+
for (const c of n.childNodes) {
|
|
172
|
+
if (c.nodeType === 3) out += c.textContent;
|
|
173
|
+
else if (c.nodeType === 1) {
|
|
174
|
+
if (c.tagName === 'IMG') out += ' ' + (c.getAttribute('alt') || '') + ' ';
|
|
175
|
+
else add(c);
|
|
176
|
+
}
|
|
177
|
+
}
|
|
178
|
+
};
|
|
179
|
+
add(el);
|
|
180
|
+
return out.replace(/\s+/g, ' ').trim();
|
|
181
|
+
}
|
|
182
|
+
|
|
162
183
|
function props(el, role) {
|
|
163
184
|
const p = {};
|
|
164
185
|
if (role === 'checkbox' || role === 'radio' || role === 'switch') p.checked = !!el.checked;
|
|
@@ -190,9 +211,7 @@ const AX_FN = function reconstructAX(base, REF_ATTR) {
|
|
|
190
211
|
// text for roles that name from content (button/link/heading/…), matching
|
|
191
212
|
// CDP — otherwise those nodes come back nameless.
|
|
192
213
|
let name = accName(el);
|
|
193
|
-
if (!name && NAME_FROM_CONTENT.has(role))
|
|
194
|
-
name = el.textContent.replace(/\s+/g, ' ').trim();
|
|
195
|
-
}
|
|
214
|
+
if (!name && NAME_FROM_CONTENT.has(role)) name = contentName(el);
|
|
196
215
|
const node = makeNode(role, name, props(el, role));
|
|
197
216
|
el.setAttribute(REF_ATTR, node.nodeId);
|
|
198
217
|
|
|
@@ -201,7 +220,7 @@ const AX_FN = function reconstructAX(base, REF_ATTR) {
|
|
|
201
220
|
// would otherwise flood every snapshot). A multi/size listbox keeps them.
|
|
202
221
|
if (el.tagName === 'SELECT' && !el.multiple) return node;
|
|
203
222
|
|
|
204
|
-
if (!NAME_FROM_CONTENT.has(role)) {
|
|
223
|
+
if (!NAME_FROM_CONTENT.has(role) || KEEP_TEXT_CHILDREN.has(role)) {
|
|
205
224
|
for (const child of kidsOf(el)) {
|
|
206
225
|
if (child.nodeType === 3) {
|
|
207
226
|
const t = child.textContent.replace(/\s+/g, ' ').trim();
|
|
@@ -213,7 +232,7 @@ const AX_FN = function reconstructAX(base, REF_ATTR) {
|
|
|
213
232
|
}
|
|
214
233
|
} else {
|
|
215
234
|
// name-from-content: still recurse into element children for nested
|
|
216
|
-
// interactives (e.g. a link inside a heading), but drop bare text.
|
|
235
|
+
// interactives (e.g. a link inside a heading), but drop bare text (cells excepted, above).
|
|
217
236
|
for (const child of kidsOf(el)) {
|
|
218
237
|
if (child.nodeType !== 1) continue;
|
|
219
238
|
const c = walk(child);
|
package/src/bidi.js
CHANGED
|
@@ -98,11 +98,12 @@ export async function createBiDi(wsUrl) {
|
|
|
98
98
|
* @param {string} context - browsing-context id
|
|
99
99
|
* @param {string} expression - JS source to evaluate
|
|
100
100
|
* @param {boolean} [awaitPromise=true]
|
|
101
|
+
* @param {string} [sandbox] - run in this isolated realm (page script cannot see or alter it)
|
|
101
102
|
* @returns {Promise<*>} the deserialized primitive value (result.value)
|
|
102
103
|
*/
|
|
103
|
-
async evaluate(context, expression, awaitPromise = true) {
|
|
104
|
+
async evaluate(context, expression, awaitPromise = true, sandbox) {
|
|
104
105
|
const res = await client.send('script.evaluate', {
|
|
105
|
-
expression, target: { context }, awaitPromise,
|
|
106
|
+
expression, target: sandbox ? { context, sandbox } : { context }, awaitPromise,
|
|
106
107
|
});
|
|
107
108
|
if (res.type === 'exception') {
|
|
108
109
|
const d = res.exceptionDetails;
|
package/src/daemon.js
CHANGED
|
@@ -11,6 +11,7 @@ import { writeFileSync, mkdirSync, existsSync, readFileSync, unlinkSync } from '
|
|
|
11
11
|
import { randomBytes, timingSafeEqual } from 'node:crypto';
|
|
12
12
|
import { join, resolve } from 'node:path';
|
|
13
13
|
import { connect } from './index.js';
|
|
14
|
+
import { overNote } from './zoom.js';
|
|
14
15
|
import { formatReadable } from './readable.js';
|
|
15
16
|
import { cleanOld } from './output-clean.js';
|
|
16
17
|
import { isHttpError } from './navigation.js';
|
|
@@ -302,12 +303,23 @@ export async function runDaemon(opts, outputDir, initialUrl) {
|
|
|
302
303
|
}
|
|
303
304
|
await attachCapture(page);
|
|
304
305
|
|
|
306
|
+
// The first snapshot after a goto waits once for the network to go quiet (idle 500ms,
|
|
307
|
+
// capped 3s): a page's load event is not its last request. Library timing is unchanged.
|
|
308
|
+
let settlePending = false;
|
|
309
|
+
async function settleOnce(settle) {
|
|
310
|
+
if (!settlePending) return;
|
|
311
|
+
settlePending = false;
|
|
312
|
+
if (settle === false) return;
|
|
313
|
+
try { await page.waitForNetworkIdle({ idle: 500, timeout: 3000 }); } catch { /* best-effort */ }
|
|
314
|
+
}
|
|
315
|
+
|
|
305
316
|
// Navigate to initial URL if provided
|
|
306
317
|
if (initialUrl) {
|
|
307
318
|
if (opts.cookies !== false) {
|
|
308
319
|
try { await page.injectCookies(initialUrl, { browser: opts.browser }); } catch { /* no cookies */ }
|
|
309
320
|
}
|
|
310
321
|
await page.goto(initialUrl, opts.timeout ? Number(opts.timeout) : 30000);
|
|
322
|
+
settlePending = true;
|
|
311
323
|
}
|
|
312
324
|
|
|
313
325
|
// Default prune mode
|
|
@@ -317,15 +329,26 @@ export async function runDaemon(opts, outputDir, initialUrl) {
|
|
|
317
329
|
const handlers = {
|
|
318
330
|
async goto({ url, timeout }) {
|
|
319
331
|
const { status } = await page.goto(url, timeout || 30000);
|
|
332
|
+
settlePending = true;
|
|
320
333
|
return isHttpError(status) ? { ok: true, status } : { ok: true };
|
|
321
334
|
},
|
|
322
335
|
|
|
323
|
-
async snapshot({ mode, links }) {
|
|
336
|
+
async snapshot({ mode, links, within, map, full, maxChars, settle }) {
|
|
337
|
+
await settleOnce(settle);
|
|
324
338
|
const pruneMode = mode || defaultPruneMode;
|
|
325
|
-
const text = await page.snapshot({ mode: pruneMode, links });
|
|
339
|
+
const text = await page.snapshot({ mode: pruneMode, links, within, map });
|
|
326
340
|
const ts = new Date().toISOString().replace(/[:.]/g, '-');
|
|
327
341
|
const file = join(absDir, `page-${ts}.yml`);
|
|
328
342
|
writeFilePrivate(file, text);
|
|
343
|
+
// Over the cap with no zoom asked for: the file stays complete, and the
|
|
344
|
+
// map (a few % of the page) comes back too, to zoom from. Same as MCP.
|
|
345
|
+
if (within == null && !map && !full && text.length > (maxChars ?? 30000)) {
|
|
346
|
+
const outline = await page.snapshot({ links, map: true });
|
|
347
|
+
if (outline.length <= (maxChars ?? 30000)) return { ok: true, file, map: outline };
|
|
348
|
+
}
|
|
349
|
+
if ((within != null || map) && !full && text.length > (maxChars ?? 30000)) {
|
|
350
|
+
return { ok: true, file, note: overNote(within != null ? 'zoom' : 'map', text.length, maxChars ?? 30000, true) };
|
|
351
|
+
}
|
|
329
352
|
return { ok: true, file };
|
|
330
353
|
},
|
|
331
354
|
|
package/src/data.js
CHANGED
|
@@ -88,7 +88,7 @@ export const stripLegend = (s) => (s.startsWith(LEGEND) ? s.slice(s.indexOf('\n'
|
|
|
88
88
|
* Render a raw ARIA tree in data mode.
|
|
89
89
|
*
|
|
90
90
|
* @param {object} root - Root node from buildTree() (CDP vocabulary)
|
|
91
|
-
* @param {{ links?: boolean }} [opts] - links: the tree's link nodes carry `urls` (annotateLinks); adds the legend note
|
|
91
|
+
* @param {{ links?: boolean, whole?: boolean }} [opts] - links: the tree's link nodes carry `urls` (annotateLinks); adds the legend note. whole: root is a zoom box (snapshot `within`), so no main-only scope
|
|
92
92
|
* @returns {string} LEGEND line, then one line per record, YAML-ish like formatTree(); '' when there are no records
|
|
93
93
|
*/
|
|
94
94
|
export function dataSnapshot(root, opts = {}) {
|
|
@@ -283,7 +283,7 @@ export function dataSnapshot(root, opts = {}) {
|
|
|
283
283
|
if (tops.length === 1 && (tops[0].role === 'RootWebArea' || tops[0].role === 'WebArea')) tops = tops[0].children;
|
|
284
284
|
const find = (ns, role) => { for (const n of ns) { if (n.role === role) return n; const f = find(n.children, role); if (f) return f; } return null; };
|
|
285
285
|
const main = find(tops, 'main');
|
|
286
|
-
if (main && main.children.length) tops = [main];
|
|
286
|
+
if (!opts.whole && main && main.children.length) tops = [main];
|
|
287
287
|
const res = xform({ role: 'root', name: '', children: tops, props: {} }, []);
|
|
288
288
|
|
|
289
289
|
const lines = [];
|