barebrowse 0.24.0 → 0.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,67 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.25.0] - 2026-10-26
4
+
5
+ ### Added
6
+
7
+ - **Opt-in data mode for pages made of records.** `snapshot({ mode: 'data' })`
8
+ turns tables, listings and repeated items into one line per record instead of
9
+ a nested tree: table rows are mapped to their column headers, repeated
10
+ siblings are grouped, and noise is dropped. Refs are shared with the default
11
+ mode, so you can still click what you read. It is available as
12
+ `pruneMode: 'data'` in `browse()` and `connect()`, as the `pruneMode`
13
+ argument of the MCP `browse` and `snapshot` tools, as
14
+ `barebrowse open --prune-mode=data` and `barebrowse snapshot --mode=data` in
15
+ the CLI, and in the bareagent tools. The default (`act`) and `read` output
16
+ are unchanged.
17
+ - **Every data-mode snapshot starts with a one-line legend.** An agent that has
18
+ never seen the format learns how to read table rows, groups and
19
+ title-plus-detail blocks from the output itself. Data mode keeps main content
20
+ only, so use `act` for menus and page controls. Buttons and links whose only
21
+ name is an icon-font glyph (private-use characters) are dropped as nameless.
22
+
23
+ ### Fixed
24
+
25
+ - **A consent dialog whose only button says "Consent" is now dismissed.** The
26
+ bare label matched no accept pattern, so a Google Funding Choices dialog
27
+ stayed open. It is matched only inside a consent dialog, never page-wide, so
28
+ "Manage consent", "Consent settings" and "Do not consent" are not clicked.
29
+ - **Consent dialogs that render 1-3s after load are dismissed.** Consent used
30
+ to run once, right after navigation, and never looked again. The first
31
+ `snapshot()` after a `goto()` now checks once more, with no added waiting, and
32
+ rebuilds the snapshot if it dismissed something. It only fires on the same
33
+ document the `goto()` landed on, so a later redirect or click to a different
34
+ page is not auto-accepted, while `pushState` and hash changes on the same
35
+ page do not skip it. This also applies in attach mode (`connect({port})`).
36
+ `createTab().goto()` and the one-shot `browse()` are not covered.
37
+ - **Chromium `snapshot()` now honours the `connect({ pruneMode })` default.** It
38
+ ignored the option and always returned `act` output, while Firefox honoured
39
+ it. An explicit `snapshot({ mode })` still wins, then the `connect()` default,
40
+ then `act`. Behaviour change: Chromium callers who set `pruneMode` (for
41
+ example `read`) will now get that mode from a bare `snapshot()`.
42
+ - **Data mode's "% pruned" no longer counts the legend.** A tiny page could
43
+ report a negative figure because the legend outweighed the content. A page
44
+ with no records now returns an empty string instead of a lone legend line.
45
+
46
+ ## [0.24.1] - 2026-10-25
47
+
48
+ ### Fixed
49
+
50
+ - **Browsers are no longer orphaned when the launching process is hard-killed.**
51
+ `kill -9`, a process-group SIGKILL or a crash used to leave Chromium or
52
+ Firefox running forever. Chromium is now tethered to its launcher through
53
+ `--remote-debugging-pipe` (all launches), so it exits when the launcher dies.
54
+ Firefox is tethered with `setpriv --pdeathsig` on Linux when `setpriv` is
55
+ installed. Where there is no `setpriv`, on non-Linux systems, or with Snap
56
+ Firefox (a setuid wrapper defeats the tether), a hard kill can still orphan
57
+ the browser.
58
+ - **Stale temporary directories of dead launchers are cleaned up.** At the next
59
+ launch, leftover `barebrowse-*` profile and download directories whose
60
+ launcher is gone are removed, in both `/tmp` and the system temp directory. A
61
+ directory whose browser lock is still live is kept. Profile and download
62
+ directory names now carry the launcher's pid to make this possible. A reused
63
+ pid or a changed hostname keeps a directory instead of removing it.
64
+
3
65
  ## [0.24.0] - 2026-10-23
4
66
 
5
67
  ### Added
package/README.md CHANGED
@@ -100,7 +100,9 @@ MCP tools: `browse`, `goto`, `snapshot`, `readable`, `click`, `type`, `press`, `
100
100
 
101
101
  **Idle release.** An MCP server can outlive its conversation, so it closes its browser after `BAREBROWSE_IDLE_MINUTES` (default `15`, `0` = never, blank/invalid = default, fractional allowed, capped at ~24.8 days) with no tool call in flight. The next call opens a fresh browser (a new job, not a resume — a page left open, e.g. a solved captcha in the headed window, is gone). A transient browser error also closes the broken browser rather than orphaning it; an ordinary `wait_for` timeout does not reset the page.
102
102
 
103
- `browse` and `snapshot` accept `pruneMode: 'act'|'read'`. `act` (default) keeps interactive elements — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If act-mode collapses a content-heavy page near-totally, the snapshot includes a `hint: …` line suggesting `pruneMode='read'` so the agent doesn't bail to a separate HTTP fetch.
103
+ **Hard-kill safety (Linux).** If the process that launched a browser is `kill -9`'d or crashes, the browser dies with it: Chromium via a `--remote-debugging-pipe` tether, Firefox via `setpriv --pdeathsig` (needs `setpriv` from util-linux; without it, or off Linux, a hard kill still orphans Firefox, as does a Snap Firefox). The dead launcher's leftover `barebrowse-*` temp dirs are cleaned at the next launch. Known limits: a reused pid keeps its dir, and dirs created under a different hostname are never removed.
104
+
105
+ `browse` and `snapshot` accept `pruneMode: 'act'|'read'|'data'`. `act` (default) keeps interactive elements — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If act-mode collapses a content-heavy page near-totally, the snapshot includes a `hint: …` line suggesting `pruneMode='read'` so the agent doesn't bail to a separate HTTP fetch. `data` is one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a one-line legend (`# data mode: …`): table rows read `row "<label>": <column>: <value> · …`, groups read `<label>: a, b, c`, and a title line followed by detail lines is one block, up to the next title line. It covers the main landmark only (nav, header and pagination outside `<main>` are dropped): use `act` for menus and page controls.
104
106
 
105
107
  `npx barebrowse doctor` reports the runtime environment first — default engine/mode, installed Chromium/Firefox binaries, cookie sources, and any live session's engine + capabilities — then scans every known MCP config location and flags scope conflicts. `npx barebrowse install --force` overwrites an existing entry pointing at a different endpoint.
106
108
 
@@ -201,7 +203,7 @@ Raw ARIA output from a page is noisy -- decorative wrappers, hidden elements, st
201
203
  | Wikipedia (article) | 109,479 chars | 40,566 chars | 63% |
202
204
  | DuckDuckGo | 42,254 chars | 5,407 chars | 87% |
203
205
 
204
- Two pruning modes: **act** (default) keeps interactive elements and visible labels -- for clicking, typing, navigating. **read** keeps all text content -- for reading articles and extracting information.
206
+ Three snapshot modes: **act** (default) keeps interactive elements and visible labels -- for clicking, typing, navigating. **read** keeps all text content -- for reading articles and extracting information. **data** (opt-in) emits one line per record -- tables become `row "Label": Header: value · ...`, repeated items are grouped (`Lines: Red, Green, Blue`) -- for data pages (timetables, listings, results). `[ref=N]` stays on interactive nodes, so click/type work as in act. Every data snapshot starts with a one-line legend explaining the format. It is larger than act on prose pages; use `read` or `readable()` for articles.
205
207
 
206
208
  ## Actions
207
209
 
@@ -211,7 +213,7 @@ Everything the agent can do through barebrowse:
211
213
  |--------|-------------|
212
214
  | **Navigate** | Load a URL, wait for page load, auto-dismiss consent |
213
215
  | **Back / Forward** | Browser history navigation |
214
- | **Snapshot** | Pruned ARIA tree with `[ref=N]` markers. Two modes: `act` (buttons, links, inputs) and `read` (full text). 40-90% token reduction. |
216
+ | **Snapshot** | Pruned ARIA tree with `[ref=N]` markers. Modes: `act` (buttons, links, inputs), `read` (full text), `data` (one line per record, for tables/listings). 40-90% token reduction. |
215
217
  | **Readable** | Clean article text (title + body, chrome stripped — Reader-View engine). For *reading* article-like pages, not interacting. Advisory `confidence`; falls back to snapshot on non-articles. |
216
218
  | **Click** | Scroll into view + mouse click at element center, JS fallback for hidden elements |
217
219
  | **Type** | Focus + insert text, with option to clear existing content first |
@@ -49,7 +49,7 @@ const snapshot = await browse('https://example.com', {
49
49
  incognito: false, // clean, unauthenticated session: skip ALL auth injection
50
50
  browser: 'firefox', // cookie source: 'firefox' | 'chromium' (auto-detected)
51
51
  prune: true, // apply ARIA pruning (47-95% token reduction)
52
- pruneMode: 'act', // 'act' (interactive elements) | 'read' (all content)
52
+ pruneMode: 'act', // 'act' (interactive elements) | 'read' (all content) | 'data' (one line per record)
53
53
  consent: true, // auto-dismiss cookie consent dialogs
54
54
  blockAds: true, // block 128 ad/tracker URL patterns (default on for owned browsers)
55
55
  blockUrls: [], // extra URL globs to block (merged with the default)
@@ -69,7 +69,7 @@ const snapshot = await browse('https://example.com', {
69
69
  | `goBack()` | -- | void | Navigate back in browser history |
70
70
  | `goForward()` | -- | void | Navigate forward in browser history |
71
71
  | `reload(opts?)` | { ignoreCache?: boolean, timeout?: number } | void | Reload the current page. Clears refMap (refs from pre-reload reject). |
72
- | `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read' } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. |
72
+ | `snapshot(pruneOpts?)` | false or { mode: 'act'\|'read'\|'data' } | string | ARIA tree with `[ref=N]` markers. Pass `false` for raw. |
73
73
  | `readable()` | -- | object | Clean article text (Reader-View engine). `{ ok, title, byline, text, length, confidence: 'high'\|'low', readerable, hint? }` or `{ ok: false, hint }`. For *reading*, not interacting — see note below. |
74
74
  | `click(ref)` | ref: string | void | Scroll into view + mouse press+release at center |
75
75
  | `type(ref, text, opts?)` | ref: string, text: string, opts: { clear?, keyEvents? } | void | Focus + insert text. `clear: true` replaces existing. |
@@ -105,7 +105,7 @@ const snapshot = await browse('https://example.com', {
105
105
  - `viewport: '1280x720'` — Set viewport dimensions
106
106
  - `storageState: 'file.json'` — Load cookies/localStorage from saved state
107
107
  - `incognito: true|false` — Default `false`. When `true`, run a clean, unauthenticated session: skips `storageState` loading AND makes `injectCookies()` a no-op, so no auth ever enters the session even though callers (MCP `goto`, daemon) inject unconditionally. The temp profile is already throwaway; this gates the *other* auth source — the user's real browser cookies. Not Chrome's `--incognito` flag. Surfaced everywhere: library, MCP (`browse` tool param + `BAREBROWSE_INCOGNITO=1` env for the session), CLI (`--incognito`), bareagent (via `connect` opts).
108
- - `downloadPath: '/abs/dir'` — Where downloads land. Default: per-session `mkdtemp` under `/tmp/barebrowse-dl-*` that gets removed on `close()`. Caller-supplied paths are not cleaned up — caller owns the lifecycle.
108
+ - `downloadPath: '/abs/dir'` — Where downloads land. Default: per-session `mkdtemp` under `os.tmpdir()` named `barebrowse-dl-<pid>-*` (Firefox: `barebrowse-ff-dl-<pid>-*`) that gets removed on `close()`, or by the next launch's stale-dir sweep if the process was hard-killed. Caller-supplied paths are not cleaned up — caller owns the lifecycle.
109
109
  - `blockAds: true|false` — CDP-level URL blocking of 128 common ad/tracker patterns (Google ads/analytics, FB/Amazon/MS/Adobe ad+analytics, Segment/Amplitude/Mixpanel/Heap/PostHog, Hotjar/FullStory/LogRocket, Criteo/Taboola/Outbrain, the consumer-pixel cluster, AppNexus/Rubicon/PubMatic supply, marketing automation; v0.10.1 added AppsFlyer/Branch/Adjust, Cloudflare Web Analytics, Matomo Cloud). Default `true` for launched browsers, `false` in attach mode (would affect any tab in the user's running browser). Explicit `true` in attach mode is honored and follows the session across `switchTab()` (regression-tested). Shrinks ARIA snapshots and speeds page loads. On legacy Chromium lacking `Network.setBlockedURLs` a one-time `console.warn` surfaces the fallback.
110
110
  - `blockUrls: ['*://foo.com/*', ...]` — Extra glob patterns (CDP `Network.setBlockedURLs` format) to block in addition to the default. Merged with the default unless `blockAds: false`.
111
111
  - `allowLocalUrls: true|false` — (v0.11.0) Default `false`: navigation to local-resource / browser-internal schemes (`file:`, `view-source:`, `chrome:`, `filesystem:`, `devtools:`, …) is **blocked** to stop a prompt-injected agent reading local files. `http`/`https`/`data`/`blob`/`about` are always allowed. Set `true` to permit local schemes.
@@ -193,7 +193,7 @@ barebrowse can inject cookies from the user's real browser sessions, bypassing l
193
193
 
194
194
  | Obstacle | How | Mode |
195
195
  |---|---|---|
196
- | Cookie consent | ARIA scan for the accept button, 29 languages. CDP: `jsClick` (overlay-bypassing) then a real-mouse-click fallback for CMPs that ignore synthetic clicks. Firefox/BiDi: a single real pointer click. A page-wide accept scan runs only for banner-style consent (no dialog) — when a dialog is detected but has no in-dialog accept button, no page-wide scan runs, to avoid clicking an unrelated `Accept all …` control | Both engines |
196
+ | Cookie consent | ARIA scan for the accept button, 29 languages. CDP: `jsClick` (overlay-bypassing) then a real-mouse-click fallback for CMPs that ignore synthetic clicks. Firefox/BiDi: a single real pointer click. A page-wide accept scan runs only for banner-style consent (no dialog) — when a dialog is detected but has no in-dialog accept button, no page-wide scan runs, to avoid clicking an unrelated `Accept all …` control. A bare `Consent` button (Google Funding Choices) is accepted only inside a detected dialog. If `goto()` found no dialog, the first `snapshot()` after it re-checks once, on the same document only, for CMPs that render 1-3s after load (not covered by one-shot `browse()` or `createTab().goto()`) | Both engines |
197
197
  | Consent behind iframes | Scanned/clicked in the owning frame (CDP `DOM.resolveNode`; Firefox resolves the ref in its browsing context) | Both engines |
198
198
  | Permission prompts | Launch flags + CDP Browser.setPermission auto-deny | Both |
199
199
  | Media autoplay blocked | `--autoplay-policy=no-user-gesture-required` | Both |
@@ -206,8 +206,8 @@ barebrowse can inject cookies from the user's real browser sessions, bypassing l
206
206
  | Stealth (headless tells) | v0.9.0 (H4): `Network.setUserAgentOverride` strips "HeadlessChrome" from UA in HTTP headers AND `navigator.userAgent`; JS patches for webdriver, plugins, languages, full `chrome.runtime` enum shape, `Notification` constructor + `permission: 'default'`, `hardwareConcurrency: 8`, `deviceMemory: 8`, WebGL `UNMASKED_VENDOR_WEBGL`/`UNMASKED_RENDERER_WEBGL` spoofed to Intel. v0.10.0: canvas fingerprint noise — `toDataURL`/`getImageData` XOR a per-session `crypto.getRandomValues`-seeded mask into ~1 byte per 64-byte stride (stable within a session, different across sessions; bitmap is restored after encoding so legitimate canvas use is unaffected). | Headless |
207
207
  | Ad / tracker URL blocking | v0.10.0: CDP `Network.setBlockedURLs` with 128 curated patterns (Google/FB/Amazon/MS/Adobe ad+analytics, the major SaaS analytics + session-replay stacks, content-rec, supply-side ad networks, marketing automation). v0.10.1 added long-tail: AppsFlyer/Branch/Adjust, Cloudflare Web Analytics, Matomo Cloud, broader Outbrain (`amplify`/`log`) and PostHog (`/static/array.js`). Default on for launched browsers, off in attach mode. `opts.blockUrls` extends; `opts.blockAds: false` disables. Shrinks ARIA snapshots and speeds loads. v0.10.1: regression-tested across `switchTab()` in attach mode; one-time `console.warn` if Chromium lacks the CDP method. | Launched |
208
208
  | iframe / OOPIF content (Stripe, reCAPTCHA, embedded forms) | v0.9.0 (H2): `Target.setAutoAttach({flatten:true})` registers a CDP session per iframe; `ariaTree()` walks `Page.getFrameTree`, fetches each frame's AX tree on the right session, splices children under iframe placeholders via `DOM.getFrameOwner`. Refs route via `{session, backendNodeId}` so clicks dispatch in the iframe's Input domain. `--site-per-process` launch flag forces every iframe — including same-origin — into OOPIF so coords work. | Both |
209
- | Downloads | v0.9.0 (H7): `Browser.setDownloadBehavior({behavior:'allowAndName', downloadPath, eventsEnabled:true})` + listeners populate `page.downloads`. Files land at `savedPath` (under `--download-path` if supplied, else per-session `/tmp/barebrowse-dl-*`). | Headless + Headed (skipped in attach mode) |
210
- | Profile locking | Unique temp dir per headless instance. A launch that fails (start timeout, early exit, bad endpoint — both engines) SIGKILLs the browser's whole process group and removes that temp dir before rejecting | Headless |
209
+ | Downloads | v0.9.0 (H7): `Browser.setDownloadBehavior({behavior:'allowAndName', downloadPath, eventsEnabled:true})` + listeners populate `page.downloads`. Files land at `savedPath` (under `--download-path` if supplied, else per-session `barebrowse-dl-<pid>-*` under `os.tmpdir()`). | Headless + Headed (skipped in attach mode) |
210
+ | Profile locking | Unique temp dir per headless instance. A launch that fails (start timeout, early exit, bad endpoint — both engines) SIGKILLs the browser's whole process group and removes that temp dir before rejecting. **Hard-kill safety (Linux):** if the launcher itself is `kill -9`'d or crashes (no exit handler can run), the browser still dies with it: Chromium is spawned with `--remote-debugging-pipe` and stdio fds 3/4 as a parent-death tether (the kernel closes the pipe, Chromium exits with its helpers); Firefox is spawned through `setpriv --pdeathsig SIGKILL` when `setpriv` exists (Linux only; without it, or off Linux, a hard kill still orphans Firefox). The dead launcher's leftover `barebrowse-*` temp dirs (Chromium profile, Firefox profile, download dirs, all named with the owner pid) are removed by the next launch's sweep, which keeps any dir whose owner pid or browser lock is still live. Known limits: a reused pid keeps its dir; a dir whose Chromium lock names a different hostname (renamed machine, container sharing `/tmp`) is kept forever; a Snap Firefox (its setuid wrapper clears pdeathsig) can still be orphaned, though its profile is kept while it runs | Headless |
211
211
  | Shared memory crash (Linux) | `--disable-dev-shm-usage` flag prevents `/dev/shm` exhaustion | Headless |
212
212
  | ARIA noise | 9-step pruning: wrapper collapse, noise removal, landmark promotion | Both |
213
213
 
@@ -302,7 +302,7 @@ Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe.
302
302
 
303
303
  `browse`, `snapshot`, and `readable` accept a `maxChars` param (default 30000). If the output exceeds the limit it's saved to `.barebrowse/` and a short message with the file path is returned instead (`page-<timestamp>.yml` for snapshots, `article-<timestamp>.txt` for `readable`). `screenshot` always saves to `.barebrowse/screenshot-<timestamp>.{png,jpeg,webp}` and returns the file path (raw base64 in a JSON-RPC response would blow `maxChars`). `tabs` returns the JSON array, or with `switchTo: N` it switches and returns `'ok'`. All files MCP writes are owner-only (`0600` in a `0700` dir) — they can hold authenticated page content, so they're not world-readable on a shared host.
304
304
 
305
- `browse` and `snapshot` also accept `pruneMode: 'act'|'read'`. `act` (the default) keeps interactive elements and short labels — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. Same surface on the bareagent adapter. If act mode collapses a content-heavy page (raw > 5 KB → pruned < 500 chars AND < 5% of raw), the result includes a `hint: act mode dropped most of the page — retry with pruneMode='read' …` line between the stats and the tree so the caller knows to re-snapshot in read mode instead of bailing to a separate HTTP fetch.
305
+ `browse` and `snapshot` also accept `pruneMode: 'act'|'read'|'data'`. `act` (the default) keeps interactive elements and short labels — best for clicking/filling. `read` keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. `data` (opt-in) is one line per record — tables with headers (`row "Label": Header: value · …`), repeated items grouped (`Label: a, b, c`) — for data pages (timetables, listings, results), not articles; every data snapshot starts with ONE legend line (`# data mode: one line per record. Tables: row "<label>": <column>: <value> · … Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode "act" for menus and page controls.`) so you can read it without docs; it keeps `[ref=N]` on interactive nodes (clickable like act) and is *larger* than act on prose pages — use `read`/`readable()` there. Same surface on the bareagent adapter. If act mode collapses a content-heavy page (raw > 5 KB → pruned < 500 chars AND < 5% of raw), the result includes a `hint: act mode dropped most of the page — retry with pruneMode='read' …` line between the stats and the tree so the caller knows to re-snapshot in read mode instead of bailing to a separate HTTP fetch.
306
306
 
307
307
  Session runs in hybrid mode (headless with automatic headed fallback on bot detection). `goto` injects cookies from the user's browser before navigation for authenticated access.
308
308
 
@@ -373,7 +373,7 @@ Useful for agent threshold decisions: "skip sites above score 40", "warn if term
373
373
 
374
374
  2. **SPA navigation has no loadEventFired.** For single-page apps (React, YouTube, GitHub), use `waitForNetworkIdle()` or a timed wait after click instead of `waitForNavigation()`.
375
375
 
376
- 3. **Pruning modes matter.** `act` mode (default) keeps interactive elements + visible labels. `read` mode keeps all text content. Use `read` for content extraction, `act` for form filling and navigation.
376
+ 3. **Pruning modes matter.** `act` mode (default) keeps interactive elements + visible labels. `read` mode keeps all text content. Use `read` for content extraction, `act` for form filling and navigation, `data` for tables and listings.
377
377
 
378
378
  4. **Headed mode auto-launches Chromium.** No need to start a browser manually — barebrowse launches a headed Chromium instance with CDP enabled automatically.
379
379
 
package/cli.js CHANGED
@@ -563,7 +563,9 @@ Session:
563
563
  --incognito Clean, unauthenticated session (skip all auth injection)
564
564
  --browser=firefox|chromium Cookie source browser
565
565
  --timeout=N Navigation timeout in ms
566
- --prune-mode=act|read Default pruning mode
566
+ --prune-mode=act|read|data Default pruning mode (data: one line per record — tables with headers,
567
+ repeated items grouped; for data pages, not articles;
568
+ output starts with a one-line legend)
567
569
  --no-consent Skip consent dismissal
568
570
  --proxy=URL HTTP/SOCKS proxy server
569
571
  --viewport=WxH Viewport size (e.g. 1280x720)
package/mcp-server.js CHANGED
@@ -266,7 +266,7 @@ export const TOOLS = [
266
266
  properties: {
267
267
  url: { type: 'string', description: 'URL to browse' },
268
268
  mode: { type: 'string', enum: ['headless', 'headed', 'hybrid'], description: 'Browser mode (default: headless)' },
269
- pruneMode: { type: 'string', enum: ['act', 'read'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If the page is content-heavy and act-mode returns mostly empty, retry with "read".' },
269
+ pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If the page is content-heavy and act-mode returns mostly empty, retry with "read". "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
270
270
  incognito: { type: 'boolean', description: 'Clean, unauthenticated session: skip cookie injection so the page loads logged-out (default: false).' },
271
271
  maxChars: { type: 'number', description: 'Max chars to return inline. Larger snapshots are saved to .barebrowse/ and a file path is returned instead. Default: 30000.' },
272
272
  },
@@ -290,7 +290,7 @@ export const TOOLS = [
290
290
  inputSchema: {
291
291
  type: 'object',
292
292
  properties: {
293
- pruneMode: { type: 'string', enum: ['act', 'read'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If a previous snapshot looked empty on a content-heavy page, retry with "read".' },
293
+ pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: 'Pruning mode. "act" (default) keeps interactive elements and short labels — best for clicking/filling. "read" keeps paragraphs, headings, and long text — best for articles, docs, and content extraction. If a previous snapshot looked empty on a content-heavy page, retry with "read". "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
294
294
  maxChars: { type: 'number', description: 'Max chars to return inline. Larger snapshots are saved to .barebrowse/ and a file path is returned instead. Default: 30000.' },
295
295
  },
296
296
  },
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "barebrowse",
3
- "version": "0.24.0",
3
+ "version": "0.25.0",
4
4
  "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
5
5
  "repository": {
6
6
  "type": "git",
package/src/bareagent.js CHANGED
@@ -62,7 +62,7 @@ export function createBrowseTools(opts = {}) {
62
62
  type: 'object',
63
63
  properties: {
64
64
  url: { type: 'string', description: 'URL to browse' },
65
- pruneMode: { type: 'string', enum: ['act', 'read'], description: '"act" (default) for interactive elements only; "read" for paragraphs and long text (articles/docs).' },
65
+ pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: '"act" (default) for interactive elements only; "read" for paragraphs and long text (articles/docs); "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
66
66
  },
67
67
  required: ['url'],
68
68
  },
@@ -86,7 +86,7 @@ export function createBrowseTools(opts = {}) {
86
86
  parameters: {
87
87
  type: 'object',
88
88
  properties: {
89
- pruneMode: { type: 'string', enum: ['act', 'read'], description: '"act" (default) for interactive elements only; "read" for paragraphs and long text (articles/docs).' },
89
+ pruneMode: { type: 'string', enum: ['act', 'read', 'data'], description: '"act" (default) for interactive elements only; "read" for paragraphs and long text (articles/docs); "data": one line per record — tables with headers, repeated items grouped; for data pages, not articles. Output starts with a legend line; read it as: table rows `row "<label>": <column>: <value> · …`, groups `<label>: a, b, c`, and a title line followed by detail lines is one block up to the next title line. Main content only: use "act" for menus and page controls.' },
90
90
  },
91
91
  },
92
92
  execute: async (/** @type {{ pruneMode?: string }} */ { pruneMode } = {}) => {
package/src/chromium.js CHANGED
@@ -6,7 +6,8 @@
6
6
  */
7
7
 
8
8
  import { execFileSync, spawn } from 'node:child_process';
9
- import { existsSync, rmSync } from 'node:fs';
9
+ import { existsSync, rmSync, readdirSync, readlinkSync } from 'node:fs';
10
+ import { tmpdir, hostname } from 'node:os';
10
11
 
11
12
  /** Block the current thread for `ms` without spawning a process. */
12
13
  function sleepSync(ms) {
@@ -121,6 +122,7 @@ export async function launch(opts = {}) {
121
122
 
122
123
  const args = [
123
124
  `--remote-debugging-port=${port}`,
125
+ '--remote-debugging-pipe',
124
126
  '--no-first-run',
125
127
  '--no-default-browser-check',
126
128
  '--disable-background-networking',
@@ -156,6 +158,7 @@ export async function launch(opts = {}) {
156
158
  if (opts.userDataDir) {
157
159
  args.push(`--user-data-dir=${opts.userDataDir}`);
158
160
  } else {
161
+ sweepStaleProfiles();
159
162
  ownedProfileDir = `/tmp/barebrowse-${process.pid}-${Date.now()}`;
160
163
  args.push(`--user-data-dir=${ownedProfileDir}`);
161
164
  }
@@ -176,10 +179,19 @@ export async function launch(opts = {}) {
176
179
  // directly. The SIGINT handler in registerExitHandlers() that calls
177
180
  // reapAllSync() is what actually kills Chromium under Ctrl-C now. Do not
178
181
  // remove that handler without restoring some other path to reap children.
182
+ //
183
+ // Parent-death tether: --remote-debugging-pipe makes Chromium read CDP from
184
+ // fd 3 and write to fd 4. We never use them (CDP still goes over the ws
185
+ // port) — they exist so that when this Node process dies by SIGKILL/crash,
186
+ // where no exit handler can run, the kernel closes our end, Chromium reads
187
+ // EOF and exits itself, helpers included. Without it the browser is
188
+ // orphaned to init with its profile dir. Nothing may write to these fds.
179
189
  const child = spawn(binary, args, {
180
- stdio: ['ignore', 'pipe', 'pipe'],
190
+ stdio: ['ignore', 'pipe', 'pipe', 'pipe', 'pipe'],
181
191
  detached: true,
182
192
  });
193
+ // Drop our ends once Chromium is gone so they can't hold the event loop.
194
+ child.once('exit', () => { child.stdio[3]?.destroy(); child.stdio[4]?.destroy(); });
183
195
 
184
196
  // Not in activeBrowsers yet, so nothing else would reap it: any failure
185
197
  // before registration (start timeout, spawn error, early exit, bad URL)
@@ -194,7 +206,7 @@ export async function launch(opts = {}) {
194
206
  reject(new Error(`Browser failed to start within 10s. stderr: ${stderr}`));
195
207
  }, 10000);
196
208
 
197
- child.stderr.on('data', (chunk) => {
209
+ child.stderr?.on('data', (chunk) => {
198
210
  stderr += chunk.toString();
199
211
  const match = stderr.match(/ws:\/\/[^\s]+/);
200
212
  if (match) {
@@ -234,6 +246,53 @@ export async function launch(opts = {}) {
234
246
  return browser;
235
247
  }
236
248
 
249
+ /**
250
+ * True when a profile dir's browser lock says a browser may still be using it: the
251
+ * lock's pid is alive, or (Chromium) it names another host (shared dir, can't judge).
252
+ * Chromium: SingletonLock -> '<hostname>-<pid>'; Firefox: lock -> '<ip>:+<pid>'.
253
+ * Download dirs have no lock.
254
+ */
255
+ function browserStillUsing(path, host) {
256
+ for (const f of ['SingletonLock', 'lock']) {
257
+ let t;
258
+ try { t = readlinkSync(`${path}/${f}`); } catch { continue; }
259
+ const m = f === 'lock' ? /:\+?(\d+)$/.exec(t) : /^(.*)-(\d+)$/.exec(t);
260
+ if (!m) continue;
261
+ if (f !== 'lock' && m[1] !== host) return true;
262
+ try { process.kill(Number(m.at(-1)), 0); return true; } catch (err) { if (err.code !== 'ESRCH') return true; }
263
+ }
264
+ return false;
265
+ }
266
+
267
+ /**
268
+ * Remove profile dirs left by launchers that were hard-killed (the pipe tether
269
+ * kills their Chromium but cannot delete the dir). Matches exactly
270
+ * barebrowse-<pid>-<ms> (Chromium) or barebrowse-ff-<pid>-<6 random chars>
271
+ * (Firefox), or the dl-/ff-dl- download-dir forms of those, and removes it only when that pid is dead (EPERM means alive), and only if no browser lock in it names a live pid or a foreign host.
272
+ * Dirs only, never processes; old pid-less dirs are never matched. Known limit: a reused pid keeps its dir.
273
+ * Foreign-host rule: if the hostname changed (renamed machine, container with a random hostname sharing /tmp), a dead launcher's dir is kept forever, a deliberate leak-over-delete tradeoff.
274
+ * With no argument sweeps both '/tmp' (Chromium profiles) and tmpdir() (download
275
+ * dirs, Firefox profiles); they differ when TMPDIR is set.
276
+ * @param {string} [dir]
277
+ * @param {string} [host] hostname a Chromium lock must name to be judged by its pid (default: this machine's; injectable for tests)
278
+ */
279
+ export function sweepStaleProfiles(dir, host = hostname()) {
280
+ if (dir === undefined) {
281
+ for (const d of new Set(['/tmp', tmpdir()])) sweepStaleProfiles(d, host);
282
+ return;
283
+ }
284
+ let names = [];
285
+ try { names = readdirSync(dir); } catch {}
286
+ for (const name of names) {
287
+ const m = /^barebrowse-(?:(\d+)-\d+|(?:ff-|dl-|ff-dl-)(\d+)-[A-Za-z0-9]{6})$/.exec(name);
288
+ const pid = Number(m?.[1] ?? m?.[2]);
289
+ if (!m || pid === process.pid) continue;
290
+ try { process.kill(pid, 0); continue; } catch (err) { if (err.code !== 'ESRCH') continue; }
291
+ if (browserStillUsing(`${dir}/${name}`, host)) continue;
292
+ try { rmSync(`${dir}/${name}`, { recursive: true, force: true }); } catch {}
293
+ }
294
+ }
295
+
237
296
  /**
238
297
  * Kill a launched browser and remove its temp profile dir (if we created one).
239
298
  * Waits up to 2s for the process to actually exit before unlinking the dir —
@@ -16,7 +16,7 @@
16
16
  * makes "descendant of dialog" a plain subtree walk — no parentMap needed.
17
17
  */
18
18
 
19
- import { ACCEPT_PATTERNS, DIALOG_ROLES, CONSENT_DIALOG_HINTS } from './consent-patterns.js';
19
+ import { ACCEPT_PATTERNS, DIALOG_ACCEPT_PATTERNS, DIALOG_ROLES, CONSENT_DIALOG_HINTS } from './consent-patterns.js';
20
20
 
21
21
  /** Roles whose text confirms a container is a consent dialog. */
22
22
  const CONSENT_TEXT_ROLES = new Set(['heading', 'StaticText', 'generic']);
@@ -44,7 +44,7 @@ function hasConsentContent(dialog) {
44
44
  * ACCEPT_PATTERNS priority (most specific first).
45
45
  */
46
46
  function findAcceptButton(dialog) {
47
- for (const pattern of ACCEPT_PATTERNS) {
47
+ for (const pattern of DIALOG_ACCEPT_PATTERNS) {
48
48
  for (const node of walk(dialog)) {
49
49
  if (node.role === 'button' && node.name && pattern.test(node.name)) return node;
50
50
  }
@@ -127,6 +127,16 @@ export const ACCEPT_PATTERNS = [
127
127
  /^ok$/i,
128
128
  ];
129
129
 
130
+ // Accept labels that are only safe INSIDE an already-detected consent dialog
131
+ // (e.g. Google Funding Choices' bare "Consent" button). Never used by the
132
+ // page-wide fallback, where a lone "Consent" could be any unrelated control.
133
+ export const DIALOG_ONLY_ACCEPT_PATTERNS = [
134
+ /^consent$/i,
135
+ ];
136
+
137
+ // Priority-ordered list the dialog-scoped search walks, built once.
138
+ export const DIALOG_ACCEPT_PATTERNS = [...ACCEPT_PATTERNS, ...DIALOG_ONLY_ACCEPT_PATTERNS];
139
+
130
140
  // Roles that indicate a consent dialog container.
131
141
  export const DIALOG_ROLES = new Set(['dialog', 'alertdialog']);
132
142
 
package/src/consent.js CHANGED
@@ -10,7 +10,7 @@
10
10
  * tree walk + click.
11
11
  */
12
12
 
13
- import { ACCEPT_PATTERNS, DIALOG_ROLES, CONSENT_DIALOG_HINTS } from './consent-patterns.js';
13
+ import { ACCEPT_PATTERNS, DIALOG_ACCEPT_PATTERNS, DIALOG_ROLES, CONSENT_DIALOG_HINTS } from './consent-patterns.js';
14
14
 
15
15
  /**
16
16
  * Click a node via JavaScript .click() instead of mouse events.
@@ -134,7 +134,7 @@ function hasConsentContent(dialogId, nodes, nodeMap, parentMap) {
134
134
  * Find the best "accept" button inside a dialog subtree.
135
135
  */
136
136
  function findAcceptButton(dialogId, nodes, nodeMap, parentMap) {
137
- for (const pattern of ACCEPT_PATTERNS) {
137
+ for (const pattern of DIALOG_ACCEPT_PATTERNS) {
138
138
  for (const node of nodes) {
139
139
  if (node.role?.value !== 'button') continue;
140
140
  const name = node.name?.value || '';
package/src/data.js ADDED
@@ -0,0 +1,300 @@
1
+ /**
2
+ * data.js — "data" snapshot mode: one line per record.
3
+ *
4
+ * For pages that are data, not prose (tables, listings, feeds, search results).
5
+ * Takes the RAW tree from buildTree() (not a pruned one: prune.js collapse()
6
+ * destroys table structure) and returns text. Never touches prune.js/aria.js.
7
+ *
8
+ * Rules, applied in this order:
9
+ * Clean pass (N1-N6) — per node, bottom-up
10
+ * N1 drop rendering noise (InlineTextBox, LineBreak, ListMarker, images)
11
+ * N2 drop StaticText with no letter/digit (whitespace, punctuation)
12
+ * N3 StaticText keeps only its text (no role word on output)
13
+ * N4 drop children whose text is already inside a content-derived name
14
+ * (unless the child is, or holds, something interactive)
15
+ * N5 unwrap unnamed wrappers; all-text wrappers become one text leaf
16
+ * N6 drop empty/nameless leaves that say nothing and can't be targeted
17
+ * (a name of only icon-font glyphs counts as no name)
18
+ * Repeated siblings (R1-R3)
19
+ * R1 a run of >=3 same-shape leaves collapses to "label: a, b, c"
20
+ * R2 a run of 2 collapses too when a >=3 run of the same shape and
21
+ * lead-in exists in the same sibling list ("weak pair")
22
+ * R3 label = lead-in leaves back to a heading/interactive anchor, else the
23
+ * nearest preceding heading, else the nearest named container
24
+ * Tables (T1-T4)
25
+ * T1 header rows (columnheader-only) become the column names
26
+ * T2 data row -> `row "<row header>": Header: value · Header: value`
27
+ * T3 header/row width mismatch (CDP exposes no colspan): drop the
28
+ * positional mapping, print `columns:` once, rows stay header-less
29
+ * T4 one value filling most cells of a wide row is stated once
30
+ *
31
+ * Scope: the main landmark if there is one, else the whole page.
32
+ * Refs: `[ref=N]` on interactive nodes only (same ids as act/read, so
33
+ * click/type resolve through the same refMap).
34
+ */
35
+
36
+ const SKIP = new Set(['InlineTextBox', 'LineBreak', 'ListMarker', 'image', 'img']);
37
+ const CELLS = new Set(['cell', 'gridcell', 'columnheader', 'rowheader']);
38
+ const TABLES = new Set(['table', 'grid', 'treegrid']);
39
+ const INTERACTIVE = new Set(['button', 'link', 'textbox', 'searchbox', 'checkbox', 'radio', 'combobox', 'listbox', 'menuitem', 'menuitemcheckbox', 'menuitemradio', 'option', 'slider', 'spinbutton', 'switch', 'tab', 'treeitem']);
40
+ // Roles whose accessible name is computed from content (ARIA name-from-content):
41
+ // a child whose text is already inside the name is redundant.
42
+ const NFC = new Set(['link', 'button', 'heading', 'generic', 'cell', 'gridcell', 'columnheader', 'rowheader', 'listitem', 'tab', 'menuitem', 'menuitemcheckbox', 'menuitemradio', 'option', 'treeitem', 'checkbox', 'radio', 'switch', 'StaticText', 'Abbr', 'strong', 'emphasis', 'LabelText', 'caption', 'term', 'definition', 'paragraph', 'LayoutTableCell', 'LayoutTableRow']);
43
+ // Unnamed wrappers that carry no semantics of their own.
44
+ const WRAP = new Set(['generic', 'none', 'group', 'presentation', 'paragraph', 'sectionheader', 'rowgroup', 'LayoutTable', 'LayoutTableRow', 'LayoutTableCell', 'list', 'Section', 'div', 'document']);
45
+ const MIN_RUN = 3;
46
+
47
+ /** First line of every data-mode snapshot: tells an agent with no docs how to read it. */
48
+ export const LEGEND = '# data mode: one line per record. Tables: row "<label>": <column>: <value> · … Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode "act" for menus and page controls.';
49
+
50
+ // A name made only of private-use code points (icon-font glyphs, unreadable to an
51
+ // agent) is no name. Glyphs mixed into real text are left as they are.
52
+ const norm = (s) => (/^[\p{Co}\s]*$/u.test(s || '') ? '' : s.replace(/\s+/g, ' ').trim());
53
+ const hasInteractive = (n) => INTERACTIVE.has(n.role) || (n.children || []).some(hasInteractive);
54
+ const isLeaf = (n) => n.text === undefined && !n.children.length && n.name;
55
+ const showRef = (n) => !!n.ref && INTERACTIVE.has(n.role);
56
+ const withRef = (n) => (showRef(n) ? `${n.name} [ref=${n.ref}]` : n.name);
57
+ // Leaves carrying state (checked, level, value...) never merge into a run.
58
+ const plain = (n) => !(n.props.checked !== undefined || n.props.expanded !== undefined || n.props.selected || n.props.level || n.props.required || (n.props.value !== undefined && n.props.value !== ''));
59
+ // Run key: leaves with the same key are "the same kind of item". `g` = the
60
+ // multi-child wrapper they came from, `sig` = raw child roles (+ '*' when the
61
+ // name overrides the content, e.g. aria-label).
62
+ const kindOf = (n) => (isLeaf(n) && n.role !== 'StaticText' && n.role !== 'heading' && plain(n) && !TABLES.has(n.role) && !CELLS.has(n.role))
63
+ ? n.role + (n.props.disabled ? '!' : '') + '|' + (n.g ?? '') + '|' + n.sig
64
+ : null;
65
+
66
+ // Flat text of a cleaned node; with refs, interactive nodes get their [ref=N].
67
+ function inline(n, refs) {
68
+ if (n.text !== undefined) return n.text;
69
+ const sub = n.children.length ? n.children.map((c) => inline(c, refs)).filter(Boolean).join(' ') : n.name;
70
+ const v = (n.name && !n.children.length) || INTERACTIVE.has(n.role) && n.name && refs ? n.name : sub;
71
+ const out = n.children.length && n.name && n.role !== 'StaticText' && !sub ? n.name : v;
72
+ return refs && n.ref && INTERACTIVE.has(n.role) ? `${out} [ref=${n.ref}]` : out;
73
+ }
74
+
75
+ /** A data-mode result without its legend line, for size stats ('' stays ''). */
76
+ export const stripLegend = (s) => (s.startsWith(LEGEND) ? s.slice(LEGEND.length + 1) : s);
77
+
78
+ /**
79
+ * Render a raw ARIA tree in data mode.
80
+ *
81
+ * @param {object} root - Root node from buildTree() (CDP vocabulary)
82
+ * @returns {string} LEGEND line, then one line per record, YAML-ish like formatTree(); '' when there are no records
83
+ */
84
+ export function dataSnapshot(root) {
85
+ if (!root) return '';
86
+ let gid = 0;
87
+ const rawKids = (n) => n.children.flatMap((c) => (c.ignored ? rawKids(c) : [c]));
88
+
89
+ // ---------- Clean pass (N1-N6) ----------
90
+ function clean(n) {
91
+ if (n.ignored) return n.children.flatMap(clean);
92
+ if (SKIP.has(n.role)) return []; // N1
93
+ if (n.role === 'StaticText') {
94
+ const t = norm(n.name);
95
+ return t && /[\p{L}\p{N}]/u.test(t) ? [{ role: 'StaticText', name: t, children: [], props: {} }] : []; // N2
96
+ }
97
+ let kids = n.children.flatMap(clean);
98
+ const name = norm(n.name);
99
+ const childText = kids.map((k) => inline(k, false)).filter(Boolean).join(' ');
100
+ const sig = rawKids(n).map((c) => c.role).join(',') + (name && childText && childText !== name ? '*' : '');
101
+ const props = n.properties || {};
102
+ // N4: a name that is just the concatenation of interactive children is hidden
103
+ // (children stay, they carry the refs); otherwise redundant children go.
104
+ const nameRedundant = !!(name && NFC.has(n.role) && kids.length && !INTERACTIVE.has(n.role)
105
+ && kids.every((k) => { const t = inline(k, false); return t && name.includes(t); }) && kids.some(hasInteractive));
106
+ if (name && NFC.has(n.role) && !nameRedundant) {
107
+ kids = kids.filter((c) => { const t = inline(c, false); return !(t && !hasInteractive(c) && name.includes(t)); });
108
+ }
109
+ const nm = nameRedundant ? '' : name;
110
+ if (!nm && WRAP.has(n.role)) { // N5
111
+ if (kids.length === 0) return [];
112
+ if (n.role === 'generic' && kids.every((k) => k.role === 'StaticText')) {
113
+ return [{ role: 'generic', name: kids.map((k) => k.name).join(' '), children: [], props, ref: n.nodeId, sig }];
114
+ }
115
+ if (kids.length === 1) return kids;
116
+ const id = ++gid; // multi-child wrapper: promote, tagged so runs don't merge across wrappers
117
+ kids.forEach((k) => { if (k.g === undefined) k.g = id; });
118
+ return kids;
119
+ }
120
+ if (!nm && kids.length === 0 && (n.role === 'link' || n.role === 'button')) return []; // N6
121
+ if (!nm && kids.length === 0 && !INTERACTIVE.has(n.role) && !CELLS.has(n.role) && n.role !== 'row') return [];
122
+ if (n.role === 'listitem' && !nm && kids.length === 1) return kids;
123
+ return [{ role: n.role, name: nm, children: kids, props, ref: n.nodeId, sig }];
124
+ }
125
+
126
+ // ---------- Transforms, bottom-up with the ancestor chain for labels ----------
127
+ function xform(n, chain) {
128
+ if (n.text !== undefined) return n;
129
+ if (TABLES.has(n.role)) return xformTable(n, chain);
130
+ const newKids = n.children.map((k, i) => xform(k, [...chain, { sibs: n.children, i, node: n }]));
131
+ return { ...n, children: collapseRuns({ ...n, children: newKids }, chain) };
132
+ }
133
+ function dedupe(kids) {
134
+ const out = [];
135
+ for (const k of kids) {
136
+ const p = out[out.length - 1];
137
+ if (p && isLeaf(p) && isLeaf(k) && p.role === k.role && p.name === k.name) continue;
138
+ out.push(k);
139
+ }
140
+ return out;
141
+ }
142
+
143
+ // R3: label for the run starting at kids[i]. Returns {label, consume} (consume =
144
+ // preceding siblings absorbed into the label), null for no label, undefined for
145
+ // "don't collapse" (a content-named container already says it).
146
+ function labelFor(parent, kids, i, chain) {
147
+ // lead-in: preceding text leaves back to an anchor (heading / interactive leaf)
148
+ const lead = []; let k = i - 1, found = false;
149
+ while (k >= 0 && lead.length < 3) {
150
+ const s = kids[k];
151
+ if (s.text !== undefined || s.children.length) break;
152
+ if (s.g !== kids[i].g) break;
153
+ if (s.role === 'heading' || INTERACTIVE.has(s.role)) {
154
+ if (!lead.length && s.role === kids[i].role) break; // an adjacent same-role item is a peer, not a title
155
+ lead.unshift(withRef(s)); found = true; break;
156
+ }
157
+ lead.unshift(s.name); k--;
158
+ }
159
+ if (found) {
160
+ let extra = 0; const a = kids[k];
161
+ while (k - 1 - extra >= 0 && isLeaf(kids[k - 1 - extra]) && kids[k - 1 - extra].role === a.role && kids[k - 1 - extra].name === a.name) extra++; // image-link twin of the anchor
162
+ return { label: [...new Set(lead)].join(' · '), consume: lead.length + extra };
163
+ }
164
+ // else: nearest preceding heading at this level, then the level's named container, then climb
165
+ const up = chain.slice().reverse(); let sibs = kids, idx = i, node = parent, m = 0;
166
+ for (;;) {
167
+ for (let j = idx - 1; j >= 0; j--) if (sibs[j].role === 'heading' && sibs[j].name) return { label: sibs[j].name, consume: 0 };
168
+ if (node.name && node.role !== 'root') return NFC.has(node.role) ? undefined : { label: node.name, consume: 0 };
169
+ if (m >= up.length) return null;
170
+ ({ sibs, i: idx, node } = up[m++]);
171
+ }
172
+ }
173
+ // Roles of the lead-in leaves back to the anchor; null if there is no anchor.
174
+ function leadShape(kids, i) {
175
+ const shape = [];
176
+ for (let k = i - 1; k >= 0 && shape.length < 3; k--) {
177
+ const s = kids[k];
178
+ if (s.text !== undefined || s.children.length || s.g !== kids[i].g) return null;
179
+ if (!shape.length && s.role === kids[i].role) return null;
180
+ shape.unshift(s.role + (s.props.disabled ? '!' : ''));
181
+ if (s.role === 'heading' || INTERACTIVE.has(s.role)) return shape.join('+');
182
+ }
183
+ return null;
184
+ }
185
+
186
+ // R1-R3
187
+ function collapseRuns(parent, chain) {
188
+ const kids = parent.children;
189
+ const out = [];
190
+ // R2 evidence: kinds that form a >=MIN_RUN run (with a lead-in shape) in this sibling list
191
+ const strong = new Set();
192
+ for (let i = 0; i < kids.length;) {
193
+ const kd = kindOf(kids[i]); let j = i;
194
+ if (kd) while (j + 1 < kids.length && kindOf(kids[j + 1]) === kd) j++;
195
+ if (kd && j - i + 1 >= MIN_RUN) { const sh = leadShape(kids, i); if (sh) strong.add(kd + '#' + sh); }
196
+ i = j + 1;
197
+ }
198
+ for (let i = 0; i < kids.length;) {
199
+ const kd = kindOf(kids[i]);
200
+ let j = i;
201
+ if (kd) while (j + 1 < kids.length && kindOf(kids[j + 1]) === kd) j++;
202
+ const len = j - i + 1;
203
+ const weak = kd && len === 2 && strong.has(kd + '#' + leadShape(kids, i));
204
+ if (kd && (len >= MIN_RUN || weak)) {
205
+ const items = kids.slice(i, j + 1);
206
+ const lf = labelFor(parent, kids, i, chain);
207
+ if (lf === undefined) { out.push(...items); i = j + 1; continue; }
208
+ if (lf && lf.consume) out.splice(out.length - lf.consume, lf.consume); // lead-in leaves now live in the label
209
+ const dis = items[0].props.disabled ? ' (disabled)' : '';
210
+ const body = items.map(withRef).join(', ');
211
+ out.push({ role: '_line', name: '', children: [], props: {}, text: lf ? `${lf.label}${dis}: ${body}` : body });
212
+ i = j + 1;
213
+ } else { out.push(kids[i]); i++; }
214
+ }
215
+ return dedupe(out);
216
+ }
217
+
218
+ // ---------- Tables (T1-T4) ----------
219
+ function xformTable(t, chain) {
220
+ const rows = [];
221
+ (function walk(n) { for (const c of n.children) { if (c.role === 'row' || c.role === 'LayoutTableRow') rows.push(c); else if (c.text === undefined) walk(c); } })(t);
222
+ const cellsOf = (r) => r.children.filter((c) => CELLS.has(c.role) || c.role === 'LayoutTableCell');
223
+ const isHeaderRow = (r) => { const cs = cellsOf(r); return cs.some((c) => c.role === 'columnheader') && !cs.some((c) => (c.role === 'cell' || c.role === 'gridcell') && (c.name || c.children.length)); };
224
+ const xcell = (c) => xform(c, [...chain, { sibs: [t], i: 0, node: t }]);
225
+ const out = [];
226
+ for (const c of t.children.filter((c) => !rows.includes(c) && c.role !== 'rowgroup')) out.push(xform(c, [...chain, { sibs: t.children, i: 0, node: t }]));
227
+ // T1: leading header rows (multi-row headers are merged per column)
228
+ const hdrRows = []; let hdrDone = false;
229
+ for (const r of rows) { if (!hdrDone && isHeaderRow(r)) hdrRows.push(r); else if (hdrRows.length) hdrDone = true; }
230
+ let headers = null;
231
+ if (hdrRows.length) {
232
+ const w = Math.max(...hdrRows.map((r) => cellsOf(r).length));
233
+ headers = Array.from({ length: w }, (_, i) => [...new Set(hdrRows.map((r) => { const c = cellsOf(r)[i]; return c ? inline(xcell(c), false) : ''; }).filter(Boolean))].join(' / '));
234
+ }
235
+ const dataRows = rows.filter((r) => !hdrRows.includes(r));
236
+ // T3: positional mapping only when every row is exactly as wide as the header
237
+ const useHdr = headers && dataRows.every((r) => cellsOf(r).length === headers.length) ? headers : null;
238
+ if (headers && !useHdr) out.push({ role: '_line', children: [], props: {}, text: `columns: ${headers.filter(Boolean).join(' · ')} [header/row width mismatch: positional mapping unsafe]` });
239
+ const firstRef = (n) => (showRef(n) ? n.ref : (n.children || []).map(firstRef).find(Boolean));
240
+ for (const r of dataRows) {
241
+ // T2
242
+ const cs = cellsOf(r);
243
+ const labelCells = cs.filter((c) => c.role === 'rowheader' || c.role === 'columnheader');
244
+ const lcx = labelCells.map(xcell);
245
+ const label = lcx.map((c) => inline(c, false)).filter(Boolean).join(' / ');
246
+ const labelRef = lcx.map(firstRef).find(Boolean);
247
+ const vals = [];
248
+ cs.forEach((c, idx) => {
249
+ if (labelCells.includes(c)) return;
250
+ const v = inline(xcell(c), true);
251
+ if (v) vals.push({ h: useHdr ? useHdr[idx] : '', v });
252
+ });
253
+ // T4: modal-value compression for wide rows
254
+ let tail = '';
255
+ if (vals.length >= 6) {
256
+ const cnt = new Map(); vals.forEach((x) => cnt.set(x.v, (cnt.get(x.v) || 0) + 1));
257
+ const [mv, mc] = [...cnt.entries()].sort((a, b) => b[1] - a[1])[0];
258
+ if (mc >= 4 && mc >= vals.length * 0.3) {
259
+ tail = ` (other ${mc} columns: ${mv})`;
260
+ for (let i = vals.length - 1; i >= 0; i--) if (vals[i].v === mv) vals.splice(i, 1);
261
+ }
262
+ }
263
+ if (!vals.length && !label) continue; // header-only / empty rows
264
+ const body = vals.map((x) => (x.h ? `${x.h}: ` : '') + x.v).join(' · ');
265
+ out.push({ role: '_line', children: [], props: {}, text: `row${label ? ` "${label}"` : ''}${labelRef ? ` [ref=${labelRef}]` : ''}${tail}${body ? ': ' + body : ''}` });
266
+ }
267
+ return { ...t, children: out };
268
+ }
269
+
270
+ // ---------- Run ----------
271
+ let tops = clean(root);
272
+ if (tops.length === 1 && (tops[0].role === 'RootWebArea' || tops[0].role === 'WebArea')) tops = tops[0].children;
273
+ const find = (ns, role) => { for (const n of ns) { if (n.role === role) return n; const f = find(n.children, role); if (f) return f; } return null; };
274
+ const main = find(tops, 'main');
275
+ if (main && main.children.length) tops = [main];
276
+ const res = xform({ role: 'root', name: '', children: tops, props: {} }, []);
277
+
278
+ const lines = [];
279
+ function emit(c, d) {
280
+ const ind = ' '.repeat(d);
281
+ if (c.text !== undefined) { lines.push(`${ind}- ${c.text}`); return; }
282
+ if (isLeaf(c) && (c.role === 'StaticText' || c.role === 'generic')) { lines.push(`${ind}- ${c.name}`); return; }
283
+ let l = `${ind}- ${c.role}`;
284
+ if (c.name) l += ` "${c.name}"`;
285
+ const p = c.props, pp = [];
286
+ if (p.checked !== undefined) pp.push(`checked=${p.checked}`);
287
+ if (p.disabled) pp.push('disabled');
288
+ if (p.expanded !== undefined) pp.push(`expanded=${p.expanded}`);
289
+ if (p.level) pp.push(`level=${p.level}`);
290
+ if (p.selected) pp.push('selected');
291
+ if (p.required && !CELLS.has(c.role)) pp.push('required');
292
+ if (p.value !== undefined && p.value !== '') pp.push(`value="${p.value}"`);
293
+ if (pp.length) l += ` [${pp.join(', ')}]`;
294
+ if (showRef(c)) l += ` [ref=${c.ref}]`;
295
+ lines.push(l);
296
+ for (const k of c.children) emit(k, d + 1);
297
+ }
298
+ for (const c of res.children) emit(c, 0);
299
+ return lines.length ? [LEGEND, ...lines].join('\n') : '';
300
+ }
@@ -17,6 +17,7 @@
17
17
 
18
18
  import { formatTree } from './aria.js';
19
19
  import { prune as pruneTree } from './prune.js';
20
+ import { dataSnapshot, stripLegend } from './data.js';
20
21
  import { axSnapshotExpression, REF_ATTR } from './ax-snapshot.js';
21
22
  import { EXTRACT_EXPRESSION, finalizeReadable } from './readable.js';
22
23
  import { scopedCookiesForUrl } from './auth.js';
@@ -39,7 +40,7 @@ const BIDI_KEYS = {
39
40
  * Build a Firefox/BiDi-backed page object.
40
41
  * @param {object} bidi - BiDi client from createBiDi()
41
42
  * @param {object} [opts]
42
- * @param {'act'|'browse'|'navigate'|'full'|'read'} [opts.pruneMode='act'] - Default prune mode.
43
+ * @param {'act'|'browse'|'navigate'|'full'|'read'|'data'} [opts.pruneMode='act'] - Default prune mode.
43
44
  * @param {{allowLocalUrls?: boolean, blockPrivateNetwork?: boolean}} [opts.urlGuard] - Navigation safety policy, applied on every goto().
44
45
  * @param {string} [opts.uploadDir] - When set, upload() rejects files outside this dir.
45
46
  * @param {boolean} [opts.incognito=false] - Clean session: injectCookies() is a no-op.
@@ -54,6 +55,15 @@ export async function createFirefoxPage(bidi, opts = {}) {
54
55
  const uploadDir = opts.uploadDir || null;
55
56
  const incognito = !!opts.incognito;
56
57
  const consent = opts.consent !== false;
58
+ // True when the consent pass after the latest goto() found nothing: the first
59
+ // snapshot() re-checks once (late-rendered CMP dialogs).
60
+ // Document identity (tab + performance.timeOrigin) the latest goto() landed on
61
+ // when its consent pass found nothing; null when no re-check is pending.
62
+ let consentRecheck = null;
63
+ async function docKey() {
64
+ try { return `${topContext}:${await bidi.evaluate(topContext, 'performance.timeOrigin', false)}`; }
65
+ catch { return null; }
66
+ }
57
67
  // Hybrid mode: on a bot-challenge page, relaunch headed and retry. The
58
68
  // relaunch tears down this Firefox+BiDi and returns a fresh one, so the
59
69
  // browser lifecycle is owned by connectFirefox, which supplies relaunchHeaded.
@@ -260,8 +270,9 @@ export async function createFirefoxPage(bidi, opts = {}) {
260
270
  // the pre-dismiss tree avoids a second full AX reconstruction per goto.
261
271
  let root = null;
262
272
  try { root = await buildTree(); } catch { /* null → treated as challenge */ }
273
+ consentRecheck = null;
263
274
  if (consent && root) {
264
- try { await dismissConsentFirefox(root, (ref) => pointerClick(ref)); }
275
+ try { if (!(await dismissConsentFirefox(root, (ref) => pointerClick(ref)))) consentRecheck = await docKey(); }
265
276
  catch { /* consent dismissal is best-effort */ }
266
277
  }
267
278
  return isChallengePage(root, countNodes(root));
@@ -302,6 +313,7 @@ export async function createFirefoxPage(bidi, opts = {}) {
302
313
  // view-source: and (optionally) private-network hosts before navigating.
303
314
  assertNavigable(url, urlGuard);
304
315
  refContexts = new Map(); // refs from the previous page are now stale
316
+ consentRecheck = null; // a throwing navigate() must not leave the old page's flag
305
317
  await withTimeout(
306
318
  bidi.send('browsingContext.navigate', { context: topContext, url, wait: 'complete' }),
307
319
  timeout, `goto(${url})`);
@@ -339,7 +351,14 @@ export async function createFirefoxPage(bidi, opts = {}) {
339
351
 
340
352
  /** @param {import('./index.js').SnapshotOptions|false} [pruneOpts] */
341
353
  async snapshot(pruneOpts = {}) {
342
- const root = await buildTree();
354
+ let root = await buildTree();
355
+ if (root && consentRecheck) {
356
+ const key = consentRecheck;
357
+ consentRecheck = null; // once per navigation
358
+ try {
359
+ if (key === await docKey() && await dismissConsentFirefox(root, (ref) => pointerClick(ref))) root = await buildTree();
360
+ } catch { /* best-effort */ }
361
+ }
343
362
  if (!root) return '';
344
363
 
345
364
  const pageUrl = await bidi.evaluate(topContext, 'location.href', false).catch(() => '');
@@ -347,10 +366,12 @@ export async function createFirefoxPage(bidi, opts = {}) {
347
366
  if (pruneOpts === false) return `url: ${pageUrl}\n` + raw;
348
367
 
349
368
  const mode = pruneOpts?.mode || defaultPruneMode;
350
- const pruned = pruneTree(root, { mode });
351
- const out = pruned ? formatTree(pruned) : '';
352
- const stats = `url: ${pageUrl}\n${raw.length.toLocaleString()} chars → ${out.length.toLocaleString()} chars`
353
- + ` (${raw.length ? Math.round((1 - out.length / raw.length) * 100) : 0}% pruned)`;
369
+ let out;
370
+ if (mode === 'data') out = dataSnapshot(root);
371
+ else { const pruned = pruneTree(root, { mode }); out = pruned ? formatTree(pruned) : ''; }
372
+ const sized = mode === 'data' ? stripLegend(out) : out; // the legend is not content
373
+ const stats = `url: ${pageUrl}\n${raw.length.toLocaleString()} chars → ${sized.length.toLocaleString()} chars`
374
+ + ` (${raw.length ? Math.round((1 - sized.length / raw.length) * 100) : 0}% pruned)`;
354
375
  return stats + '\n' + out;
355
376
  },
356
377
 
@@ -492,6 +513,7 @@ export async function createFirefoxPage(bidi, opts = {}) {
492
513
  bidi.send('browsingContext.reload', { context: topContext, wait: 'complete' }),
493
514
  30000, 'reload');
494
515
  refContexts = new Map();
516
+ consentRecheck = null;
495
517
  await new Promise((r) => setTimeout(r, 300));
496
518
  },
497
519
 
@@ -644,6 +666,7 @@ export async function createFirefoxPage(bidi, opts = {}) {
644
666
  bidi.send('browsingContext.traverseHistory', { context: topContext, delta }),
645
667
  30000, 'history navigation');
646
668
  refContexts = new Map();
669
+ consentRecheck = null;
647
670
  await new Promise((r) => setTimeout(r, 500));
648
671
  }
649
672
 
package/src/firefox.js CHANGED
@@ -15,9 +15,10 @@
15
15
  */
16
16
 
17
17
  import { execFileSync, spawn } from 'node:child_process';
18
- import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
18
+ import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
19
19
  import { tmpdir } from 'node:os';
20
20
  import { join } from 'node:path';
21
+ import { sweepStaleProfiles } from './chromium.js';
21
22
 
22
23
  /** Block the current thread for `ms` (sync, for the 'exit' reaper). */
23
24
  function sleepSync(ms) {
@@ -55,6 +56,8 @@ function registerExitHandlers() {
55
56
  }
56
57
  }
57
58
 
59
+ const SETPRIV = ['/usr/bin/setpriv', '/bin/setpriv'];
60
+
58
61
  const CANDIDATES = [
59
62
  'firefox',
60
63
  'firefox-esr',
@@ -103,7 +106,9 @@ export function findFirefox() {
103
106
  export async function launchFirefox(opts = {}) {
104
107
  const binary = opts.binary || findFirefox();
105
108
  const port = opts.port || 0;
106
- const profileDir = mkdtempSync(join(tmpdir(), 'barebrowse-ff-'));
109
+ // Always our own profile: sweep dirs of hard-killed launchers, then pid-tag ours.
110
+ sweepStaleProfiles();
111
+ const profileDir = mkdtempSync(join(tmpdir(), `barebrowse-ff-${process.pid}-`));
107
112
 
108
113
  // Proxy + prompt-suppressing prefs go in user.js (read at profile load).
109
114
  const prefs = [
@@ -167,7 +172,21 @@ export async function launchFirefox(opts = {}) {
167
172
  args.push('about:blank');
168
173
 
169
174
  // detached:true → own process group, so cleanup can signal the whole tree.
170
- const child = spawn(binary, args, { stdio: ['ignore', 'pipe', 'pipe'], detached: true });
175
+ //
176
+ // Parent-death tether: Firefox has no --remote-debugging-pipe, so a SIGKILLed
177
+ // or crashed Node parent (no exit handler runs) would orphan it. On Linux we
178
+ // exec it through util-linux `setpriv --pdeathsig SIGKILL`: setpriv execs
179
+ // Firefox (child.pid stays Firefox's pid) and the kernel SIGKILLs it when this
180
+ // process dies; its children follow. Without setpriv (or off Linux) we spawn
181
+ // exactly as before and a hard kill still orphans it.
182
+ // A setuid wrapper (e.g. Snap's snap-confine) clears pdeathsig on exec, so a Snap
183
+ // Firefox can still be orphaned even with setpriv; the stale-profile sweep keeps
184
+ // its profile dir while it runs (it checks the browser lock).
185
+ const setpriv = process.platform === 'linux' && SETPRIV.find(existsSync);
186
+ const child = spawn(setpriv || binary, setpriv ? ['--pdeathsig', 'SIGKILL', binary, ...args] : args, {
187
+ stdio: ['ignore', 'pipe', 'pipe'],
188
+ detached: true,
189
+ });
171
190
 
172
191
  // Not in activeBrowsers yet, so nothing else would reap it: any failure
173
192
  // before registration must kill the process group and remove the profile.
package/src/index.js CHANGED
@@ -16,6 +16,7 @@ import { createFirefoxPage } from './firefox-page.js';
16
16
  import { formatTree } from './aria.js';
17
17
  import { authenticate } from './auth.js';
18
18
  import { prune as pruneTree } from './prune.js';
19
+ import { dataSnapshot, stripLegend } from './data.js';
19
20
  import { click as cdpClick, type as cdpType, scroll as cdpScroll, press as cdpPress, hover as cdpHover, select as cdpSelect, drag as cdpDrag, upload as cdpUpload } from './interact.js';
20
21
  import { dismissConsent } from './consent.js';
21
22
  import { applyStealth } from './stealth.js';
@@ -58,7 +59,7 @@ import { chmodSync } from 'node:fs';
58
59
  * @param {{width: number, height: number}} [opts.viewport] - Viewport dimensions.
59
60
  * @param {string} [opts.browser] - Source browser for cookie extraction.
60
61
  * @param {boolean} [opts.consent=true] - Auto-dismiss cookie consent dialogs.
61
- * @param {'act'|'browse'|'navigate'|'full'|'read'} [opts.pruneMode='act'] - Pruning mode.
62
+ * @param {'act'|'browse'|'navigate'|'full'|'read'|'data'} [opts.pruneMode='act'] - Pruning mode.
62
63
  * @returns {Promise<string>} ARIA snapshot text
63
64
  */
64
65
  export async function browse(url, opts = {}) {
@@ -141,12 +142,14 @@ export async function browse(url, opts = {}) {
141
142
  const raw = formatTree(tree);
142
143
  let snapshot;
143
144
  if (opts.prune !== false) {
144
- const pruned = pruneTree(tree, { mode: opts.pruneMode || 'act' });
145
- snapshot = formatTree(pruned);
145
+ snapshot = opts.pruneMode === 'data'
146
+ ? dataSnapshot(tree)
147
+ : formatTree(pruneTree(tree, { mode: opts.pruneMode || 'act' }));
146
148
  } else {
147
149
  snapshot = raw;
148
150
  }
149
- const stats = `url: ${url}\n${raw.length.toLocaleString()} chars → ${snapshot.length.toLocaleString()} chars (${Math.round((1 - snapshot.length / raw.length) * 100)}% pruned)`;
151
+ const sized = opts.pruneMode === 'data' ? stripLegend(snapshot) : snapshot; // the legend is not content
152
+ const stats = `url: ${url}\n${raw.length.toLocaleString()} chars → ${sized.length.toLocaleString()} chars (${Math.round((1 - sized.length / raw.length) * 100)}% pruned)`;
150
153
  const actMode = !opts.pruneMode || opts.pruneMode === 'act';
151
154
  const hint = (actMode && raw.length > 5000 && snapshot.length < 500 && snapshot.length < raw.length * 0.05)
152
155
  ? `hint: act mode dropped most of the page — retry with pruneMode='read' for paragraphs and long text\n`
@@ -204,7 +207,7 @@ export async function browse(url, opts = {}) {
204
207
  /**
205
208
  * Options for `page.snapshot()`. Pass `false` instead of an object to skip
206
209
  * pruning and return the raw ARIA tree.
207
- * @typedef {{ mode?: 'act'|'browse'|'navigate'|'full'|'read' }} SnapshotOptions
210
+ * @typedef {{ mode?: 'act'|'browse'|'navigate'|'full'|'read'|'data' }} SnapshotOptions
208
211
  */
209
212
 
210
213
  /**
@@ -257,7 +260,7 @@ export async function browse(url, opts = {}) {
257
260
  * @param {boolean} [opts.incognito=false] - Clean, unauthenticated session:
258
261
  * skip storageState loading and make injectCookies() a no-op, so no auth
259
262
  * ever enters the session even if a caller injects unconditionally.
260
- * @param {'act'|'browse'|'navigate'|'full'|'read'} [opts.pruneMode='act'] - Pruning mode.
263
+ * @param {'act'|'browse'|'navigate'|'full'|'read'|'data'} [opts.pruneMode='act'] - Pruning mode.
261
264
  * @param {'chromium'|'firefox'} [opts.engine='chromium'] - Browser engine.
262
265
  * 'firefox' drives over WebDriver BiDi (a separate transport / page object);
263
266
  * 'chromium' (default) uses CDP.
@@ -364,6 +367,21 @@ export async function connect(opts = {}) {
364
367
  let page = await createPage(cdp, !currentlyHeaded, pageOpts);
365
368
  let refMap = new Map();
366
369
  let botBlocked = false;
370
+ // Set when the consent pass after the latest goto() found nothing: the first
371
+ // snapshot() re-checks once, since some CMPs (Google Funding Choices) render
372
+ // their dialog 1-3s after load. Holds the document identity (tab + main-frame
373
+ // loaderId) the goto landed on, so a snapshot of any other document (click,
374
+ // redirect, switchTab) never re-checks. loaderId changes on every
375
+ // cross-document navigation but not on pushState/replaceState/hash change.
376
+ // Null when no re-check is pending.
377
+ // Also set in attach mode (goto() runs consent there too).
378
+ let consentRecheck = null;
379
+ async function navKey() {
380
+ try {
381
+ const { frameTree } = await page.session.send('Page.getFrameTree');
382
+ return `${page.targetId}:${frameTree.frame.loaderId}`;
383
+ } catch { return null; }
384
+ }
367
385
 
368
386
  // Suppress permission prompts. Skipped in attach mode — Browser.setPermission
369
387
  // is browser-wide (no origin scope here), so flipping permissions to denied
@@ -397,7 +415,7 @@ export async function connect(opts = {}) {
397
415
  if (!downloadPath) {
398
416
  const { mkdtempSync } = await import('node:fs');
399
417
  const { tmpdir } = await import('node:os');
400
- ownedDownloadDir = mkdtempSync(pathJoin(tmpdir(), 'barebrowse-dl-'));
418
+ ownedDownloadDir = mkdtempSync(pathJoin(tmpdir(), `barebrowse-dl-${process.pid}-`));
401
419
  downloadPath = ownedDownloadDir;
402
420
  }
403
421
  // Register listeners BEFORE sending setDownloadBehavior so no
@@ -470,6 +488,7 @@ export async function connect(opts = {}) {
470
488
  // silently resolving to whatever backendNodeId happens to still be in
471
489
  // the map.
472
490
  refMap = new Map();
491
+ consentRecheck = null; // a throwing navigate() must not leave the old page's flag
473
492
  // Switch back to headless if we fell back to headed previously.
474
493
  // Not in attach mode — we never own the browser there, so there's
475
494
  // nothing to rewind.
@@ -488,7 +507,8 @@ export async function connect(opts = {}) {
488
507
 
489
508
  await navigate(page, url, timeout);
490
509
  if (opts.consent !== false) {
491
- await dismissConsent(page.session);
510
+ const dismissed = await dismissConsent(page.session);
511
+ if (!dismissed) consentRecheck = await navKey();
492
512
  }
493
513
 
494
514
  // Check for bot challenge
@@ -510,7 +530,10 @@ export async function connect(opts = {}) {
510
530
  setupDialogHandler(page.session);
511
531
  await suppressPermissions(cdp);
512
532
  await navigate(page, url, timeout);
513
- if (opts.consent !== false) await dismissConsent(page.session);
533
+ consentRecheck = null;
534
+ if (opts.consent !== false && !(await dismissConsent(page.session))) {
535
+ consentRecheck = await navKey();
536
+ }
514
537
 
515
538
  // Re-check after headed fallback
516
539
  const after = await ariaTree(page);
@@ -529,6 +552,7 @@ export async function connect(opts = {}) {
529
552
  await page.session.send('Page.navigateToHistoryEntry', { entryId: entries[currentIndex - 1].id });
530
553
  try { await loadPromise; } catch { await new Promise((r) => setTimeout(r, 500)); }
531
554
  refMap = new Map(); // refs from the previous page are now invalid
555
+ consentRecheck = null;
532
556
  },
533
557
 
534
558
  async goForward() {
@@ -538,6 +562,7 @@ export async function connect(opts = {}) {
538
562
  await page.session.send('Page.navigateToHistoryEntry', { entryId: entries[currentIndex + 1].id });
539
563
  try { await loadPromise; } catch { await new Promise((r) => setTimeout(r, 500)); }
540
564
  refMap = new Map();
565
+ consentRecheck = null;
541
566
  },
542
567
 
543
568
  async reload(reloadOpts = {}) {
@@ -548,6 +573,7 @@ export async function connect(opts = {}) {
548
573
  });
549
574
  try { await loadPromise; } catch { await new Promise((r) => setTimeout(r, 500)); }
550
575
  refMap = new Map(); // refs from the pre-reload page are invalid
576
+ consentRecheck = null;
551
577
  },
552
578
 
553
579
  /** @param {string} url @param {CookieOptions} [cookieOpts] */
@@ -560,6 +586,14 @@ export async function connect(opts = {}) {
560
586
 
561
587
  /** @param {SnapshotOptions|false} [pruneOpts] */
562
588
  async snapshot(pruneOpts = {}) {
589
+ if (consentRecheck) {
590
+ const key = consentRecheck;
591
+ consentRecheck = null; // once per navigation
592
+ // Only if we are still on the document the goto landed on.
593
+ if (key === await navKey()) {
594
+ try { await dismissConsent(page.session); } catch { /* best-effort */ }
595
+ }
596
+ }
563
597
  const result = await ariaTree(page);
564
598
  refMap = result.refMap;
565
599
  const raw = formatTree(result.tree);
@@ -567,10 +601,10 @@ export async function connect(opts = {}) {
567
601
  const pageUrl = entries[currentIndex]?.url || '';
568
602
  const warn = botBlocked ? '[BOT CHALLENGE DETECTED — page content may be incomplete or blocked]\n' : '';
569
603
  if (pruneOpts === false) return `url: ${pageUrl}\n` + warn + raw;
570
- const mode = pruneOpts?.mode || 'act';
571
- const pruned = pruneTree(result.tree, { mode });
572
- const out = formatTree(pruned);
573
- const stats = `url: ${pageUrl}\n${raw.length.toLocaleString()} chars → ${out.length.toLocaleString()} chars (${Math.round((1 - out.length / raw.length) * 100)}% pruned)`;
604
+ const mode = pruneOpts?.mode || opts.pruneMode || 'act';
605
+ const out = mode === 'data' ? dataSnapshot(result.tree) : formatTree(pruneTree(result.tree, { mode }));
606
+ const sized = mode === 'data' ? stripLegend(out) : out; // the legend is not content
607
+ const stats = `url: ${pageUrl}\n${raw.length.toLocaleString()} chars → ${sized.length.toLocaleString()} chars (${Math.round((1 - sized.length / raw.length) * 100)}% pruned)`;
574
608
  const hint = (mode === 'act' && raw.length > 5000 && out.length < 500 && out.length < raw.length * 0.05)
575
609
  ? `hint: act mode dropped most of the page — retry with pruneMode='read' for paragraphs and long text\n`
576
610
  : '';
@@ -888,7 +922,7 @@ async function connectFirefox(opts) {
888
922
  if (!downloadDir) {
889
923
  const { mkdtempSync } = await import('node:fs');
890
924
  const { tmpdir } = await import('node:os');
891
- ownedDownloadDir = mkdtempSync(pathJoin(tmpdir(), 'barebrowse-ff-dl-'));
925
+ ownedDownloadDir = mkdtempSync(pathJoin(tmpdir(), `barebrowse-ff-dl-${process.pid}-`));
892
926
  downloadDir = ownedDownloadDir;
893
927
  }
894
928
 
@@ -25,6 +25,19 @@ export function launch(opts?: {
25
25
  process: import("node:child_process").ChildProcess;
26
26
  port: number;
27
27
  }>;
28
+ /**
29
+ * Remove profile dirs left by launchers that were hard-killed (the pipe tether
30
+ * kills their Chromium but cannot delete the dir). Matches exactly
31
+ * barebrowse-<pid>-<ms> (Chromium) or barebrowse-ff-<pid>-<6 random chars>
32
+ * (Firefox), or the dl-/ff-dl- download-dir forms of those, and removes it only when that pid is dead (EPERM means alive), and only if no browser lock in it names a live pid or a foreign host.
33
+ * Dirs only, never processes; old pid-less dirs are never matched. Known limit: a reused pid keeps its dir.
34
+ * Foreign-host rule: if the hostname changed (renamed machine, container with a random hostname sharing /tmp), a dead launcher's dir is kept forever, a deliberate leak-over-delete tradeoff.
35
+ * With no argument sweeps both '/tmp' (Chromium profiles) and tmpdir() (download
36
+ * dirs, Firefox profiles); they differ when TMPDIR is set.
37
+ * @param {string} [dir]
38
+ * @param {string} [host] hostname a Chromium lock must name to be judged by its pid (default: this machine's; injectable for tests)
39
+ */
40
+ export function sweepStaleProfiles(dir?: string, host?: string): void;
28
41
  /**
29
42
  * Kill a launched browser and remove its temp profile dir (if we created one).
30
43
  * Waits up to 2s for the process to actually exit before unlinking the dir —
@@ -8,5 +8,7 @@
8
8
  * pattern is added once and both engines get it.
9
9
  */
10
10
  export const ACCEPT_PATTERNS: RegExp[];
11
+ export const DIALOG_ONLY_ACCEPT_PATTERNS: RegExp[];
12
+ export const DIALOG_ACCEPT_PATTERNS: RegExp[];
11
13
  export const DIALOG_ROLES: Set<string>;
12
14
  export const CONSENT_DIALOG_HINTS: RegExp[];
@@ -0,0 +1,10 @@
1
+ /**
2
+ * Render a raw ARIA tree in data mode.
3
+ *
4
+ * @param {object} root - Root node from buildTree() (CDP vocabulary)
5
+ * @returns {string} LEGEND line, then one line per record, YAML-ish like formatTree(); '' when there are no records
6
+ */
7
+ export function dataSnapshot(root: object): string;
8
+ /** First line of every data-mode snapshot: tells an agent with no docs how to read it. */
9
+ export const LEGEND: "# data mode: one line per record. Tables: row \"<label>\": <column>: <value> \u00B7 \u2026 Groups: <label>: a, b, c. A title line followed by detail lines is one block, up to the next title line. Main content only; use mode \"act\" for menus and page controls.";
10
+ export function stripLegend(s: any): any;
@@ -2,7 +2,7 @@
2
2
  * Build a Firefox/BiDi-backed page object.
3
3
  * @param {object} bidi - BiDi client from createBiDi()
4
4
  * @param {object} [opts]
5
- * @param {'act'|'browse'|'navigate'|'full'|'read'} [opts.pruneMode='act'] - Default prune mode.
5
+ * @param {'act'|'browse'|'navigate'|'full'|'read'|'data'} [opts.pruneMode='act'] - Default prune mode.
6
6
  * @param {{allowLocalUrls?: boolean, blockPrivateNetwork?: boolean}} [opts.urlGuard] - Navigation safety policy, applied on every goto().
7
7
  * @param {string} [opts.uploadDir] - When set, upload() rejects files outside this dir.
8
8
  * @param {boolean} [opts.incognito=false] - Clean session: injectCookies() is a no-op.
@@ -12,7 +12,7 @@
12
12
  * @param {?function(): Promise<{bidi: object, topContext: string}>} [opts.relaunchHeaded] - Hybrid relaunch hook (from connectFirefox).
13
13
  */
14
14
  export function createFirefoxPage(bidi: object, opts?: {
15
- pruneMode?: "act" | "browse" | "navigate" | "full" | "read" | undefined;
15
+ pruneMode?: "data" | "act" | "browse" | "navigate" | "full" | "read" | undefined;
16
16
  urlGuard?: {
17
17
  allowLocalUrls?: boolean;
18
18
  blockPrivateNetwork?: boolean;
package/types/index.d.ts CHANGED
@@ -25,7 +25,7 @@
25
25
  * @param {{width: number, height: number}} [opts.viewport] - Viewport dimensions.
26
26
  * @param {string} [opts.browser] - Source browser for cookie extraction.
27
27
  * @param {boolean} [opts.consent=true] - Auto-dismiss cookie consent dialogs.
28
- * @param {'act'|'browse'|'navigate'|'full'|'read'} [opts.pruneMode='act'] - Pruning mode.
28
+ * @param {'act'|'browse'|'navigate'|'full'|'read'|'data'} [opts.pruneMode='act'] - Pruning mode.
29
29
  * @returns {Promise<string>} ARIA snapshot text
30
30
  */
31
31
  export function browse(url: string, opts?: {
@@ -47,7 +47,7 @@ export function browse(url: string, opts?: {
47
47
  } | undefined;
48
48
  browser?: string | undefined;
49
49
  consent?: boolean | undefined;
50
- pruneMode?: "act" | "browse" | "navigate" | "full" | "read" | undefined;
50
+ pruneMode?: "data" | "act" | "browse" | "navigate" | "full" | "read" | undefined;
51
51
  }): Promise<string>;
52
52
  /**
53
53
  * Decide which engine connect() should use. Pure — the caller passes lazy
@@ -248,7 +248,7 @@ export type Capabilities = {
248
248
  * pruning and return the raw ARIA tree.
249
249
  */
250
250
  export type SnapshotOptions = {
251
- mode?: "act" | "browse" | "navigate" | "full" | "read";
251
+ mode?: "act" | "browse" | "navigate" | "full" | "read" | "data";
252
252
  };
253
253
  /**
254
254
  * Options for `page.type()`.