barebrowse 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/memory/AGENT_RULES.md +251 -0
- package/.claude/settings.local.json +30 -0
- package/.claude/stash/barebrowse-research-2026-02-22.md +49 -0
- package/.claude/stash/phase3-interactions-complete.md +69 -0
- package/.claude/stash/phase3-prep.md +88 -0
- package/.mcp.json +8 -0
- package/CHANGELOG.md +93 -0
- package/CLAUDE.md +22 -0
- package/README.md +200 -41
- package/barebrowse.context.md +261 -0
- package/cli.js +156 -0
- package/docs/blueprint.md +361 -0
- package/docs/poc-plan.md +230 -0
- package/docs/prd.md +284 -0
- package/docs/testing.md +202 -0
- package/examples/headed-demo.js +157 -0
- package/examples/yt-demo.js +137 -0
- package/mcp-server.js +216 -0
- package/package.json +22 -9
- package/src/aria.js +69 -0
- package/src/auth.js +279 -0
- package/src/bareagent.js +161 -0
- package/src/cdp.js +148 -0
- package/src/chromium.js +148 -0
- package/src/consent.js +210 -0
- package/src/index.js +186 -10
- package/src/interact.js +208 -0
- package/src/prune.js +472 -0
- package/src/stealth.js +51 -0
- package/test/integration/browse.test.js +108 -0
- package/test/integration/interact.test.js +514 -0
- package/test/unit/auth.test.js +66 -0
- package/test/unit/cdp.test.js +110 -0
- package/test/unit/prune.test.js +292 -0
|
@@ -0,0 +1,361 @@
|
|
|
1
|
+
# barebrowse -- Blueprint
|
|
2
|
+
|
|
3
|
+
Vanilla JS library. CDP-direct. URL in, pruned ARIA snapshot out.
|
|
4
|
+
No Playwright, no bundled browser, no build step.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## What It Does
|
|
9
|
+
|
|
10
|
+
Gives autonomous agents authenticated access to the web through the user's own Chromium browser.
|
|
11
|
+
|
|
12
|
+
```js
|
|
13
|
+
import { browse, connect } from 'barebrowse';
|
|
14
|
+
|
|
15
|
+
// One-shot: read a page
|
|
16
|
+
const snapshot = await browse('https://any-page.com');
|
|
17
|
+
|
|
18
|
+
// Session: navigate, interact, observe
|
|
19
|
+
const page = await connect();
|
|
20
|
+
await page.goto('https://any-page.com');
|
|
21
|
+
console.log(await page.snapshot());
|
|
22
|
+
await page.click('8'); // ref from snapshot
|
|
23
|
+
await page.type('3', 'hello');
|
|
24
|
+
await page.scroll(500);
|
|
25
|
+
await page.close();
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Capabilities
|
|
31
|
+
|
|
32
|
+
Every action returns a **pruned ARIA snapshot** -- the agent's view of the page after each move. The snapshot is a YAML-like tree with `[ref=N]` markers on interactive elements. The agent reads the snapshot, picks a ref, acts, then reads the next snapshot. This is the observe-think-act loop.
|
|
33
|
+
|
|
34
|
+
### Actions
|
|
35
|
+
|
|
36
|
+
| Action | Method | What It Does | Status |
|
|
37
|
+
|--------|--------|-------------|--------|
|
|
38
|
+
| Navigate | `page.goto(url)` | Load a URL, wait for page load, dismiss consent | Done |
|
|
39
|
+
| Snapshot | `page.snapshot()` | Pruned ARIA tree (47-95% token reduction) | Done |
|
|
40
|
+
| Click | `page.click(ref)` | Scroll into view, mouse press+release at element center | Done |
|
|
41
|
+
| Type | `page.type(ref, text)` | Focus element, insert text (fast batch mode) | Done |
|
|
42
|
+
| Type (clear) | `page.type(ref, text, { clear: true })` | Select-all + delete, then type (replaces pre-filled content) | Done |
|
|
43
|
+
| Type (key events) | `page.type(ref, text, { keyEvents: true })` | Char-by-char keyDown/keyUp (triggers JS handlers) | Done |
|
|
44
|
+
| Press | `page.press(key)` | Special key: Enter, Tab, Escape, Backspace, Delete, arrows, Home/End, PageUp/Down, Space | Done |
|
|
45
|
+
| Scroll | `page.scroll(deltaY)` | Mouse wheel event (positive=down, negative=up) | Done |
|
|
46
|
+
| Hover | `page.hover(ref)` | Move mouse to element center (triggers hover styles/tooltips) | Done |
|
|
47
|
+
| Select | `page.select(ref, value)` | Set `<select>` value or click custom dropdown option | Done |
|
|
48
|
+
| Screenshot | `page.screenshot(opts)` | `Page.captureScreenshot`, returns base64 string | Done |
|
|
49
|
+
| Wait for nav | `page.waitForNavigation()` | Promise.race of loadEventFired + frameNavigated (SPA-aware) | Done |
|
|
50
|
+
| Wait for idle | `page.waitForNetworkIdle(opts)` | Resolve when no pending requests for N ms (default 500) | Done |
|
|
51
|
+
| Inject cookies | `page.injectCookies(url, opts)` | Extract cookies from Firefox/Chromium, inject via CDP | Done |
|
|
52
|
+
| Raw CDP | `page.cdp.send(method, params)` | Escape hatch for any CDP command | Done |
|
|
53
|
+
| Close | `page.close()` | Close page target, disconnect CDP, kill browser (if headless) | Done |
|
|
54
|
+
|
|
55
|
+
### Obstacle course -- what barebrowse handles automatically
|
|
56
|
+
|
|
57
|
+
| Obstacle | How It's Handled | Mode |
|
|
58
|
+
|----------|-----------------|------|
|
|
59
|
+
| **Cookie consent walls** (GDPR) | ARIA tree scan, jsClick accept button. 7 languages: EN, NL, DE, FR, ES, IT, PT | Both |
|
|
60
|
+
| **Consent in dialog role** | Detect `dialog`/`alertdialog` with consent hints, click accept inside | Both |
|
|
61
|
+
| **Consent outside dialog** (BBC SourcePoint) | Fallback global button scan when dialog has no accept button | Both |
|
|
62
|
+
| **Consent behind iframe overlay** | JS `.click()` via `DOM.resolveNode` bypasses z-index/overlay issues | Both |
|
|
63
|
+
| **Permission prompts** (location, notifications, camera, mic) | Launch flags + CDP `Browser.setPermission` auto-deny | Both |
|
|
64
|
+
| **Media autoplay blocked** | `--autoplay-policy=no-user-gesture-required` | Both |
|
|
65
|
+
| **Login walls** | Firefox cookie extraction, CDP injection (user's real sessions) | Both |
|
|
66
|
+
| **Pre-filled form inputs** | `type({ clear: true })` selects all + deletes before typing | Both |
|
|
67
|
+
| **Off-screen elements** | `DOM.scrollIntoViewIfNeeded` before every click | Both |
|
|
68
|
+
| **Form submission** | `press('Enter')` with proper `text: '\r'` triggers onsubmit | Both |
|
|
69
|
+
| **Tab between fields** | `press('Tab')` with `text: '\t'` moves focus | Both |
|
|
70
|
+
| **SPA navigation** (YouTube, GitHub) | `waitForNavigation()` uses frameNavigated + loadEventFired race | Both |
|
|
71
|
+
| **Bot detection** (Google, Reddit) | Stealth patches (headless) + headed mode with real cookies | Both |
|
|
72
|
+
| **`navigator.webdriver`** | Stealth patches: webdriver, plugins, languages, chrome object | Headless |
|
|
73
|
+
| **Profile locking** | Unique temp dir per headless instance (`/tmp/barebrowse-<pid>-<ts>`) | Headless |
|
|
74
|
+
| **ARIA noise** | 9-step pruning: wrapper collapse, noise removal, landmark promotion | Both |
|
|
75
|
+
|
|
76
|
+
### Not yet handled
|
|
77
|
+
|
|
78
|
+
| Obstacle | What's Needed | Difficulty |
|
|
79
|
+
|----------|--------------|------------|
|
|
80
|
+
| File upload | `Input.setFiles` via CDP | Low |
|
|
81
|
+
| Drag and drop | `Input.dispatchDragEvent` sequence | Medium |
|
|
82
|
+
| Infinite scroll | Scroll + wait for new content strategy | Medium |
|
|
83
|
+
| CAPTCHAs | Cannot solve -- headed mode lets user solve manually | N/A |
|
|
84
|
+
| Cross-origin iframes | Frame tree traversal via CDP | Medium |
|
|
85
|
+
| Canvas/WebGL | Opaque to ARIA -- needs screenshot + vision model | Hard |
|
|
86
|
+
|
|
87
|
+
### Tested sites (16+ sites, 8 countries, all consent dismissed)
|
|
88
|
+
|
|
89
|
+
| Site | Consent | Cookies | Interactions | Notes |
|
|
90
|
+
|------|---------|---------|-------------|-------|
|
|
91
|
+
| google.com | NL dialog dismissed | Firefox injection | Search (combobox + Enter) | Bot-blocks headless |
|
|
92
|
+
| youtube.com | Bypassed via cookies | Firefox injection | Search + video playback | Full e2e demo, SPA nav |
|
|
93
|
+
| bbc.com | SourcePoint dismissed | -- | -- | Button outside dialog |
|
|
94
|
+
| wikipedia.org | -- | -- | Link click + navigation | Clean, no consent |
|
|
95
|
+
| github.com | -- | -- | SPA navigation | Needs settle time |
|
|
96
|
+
| duckduckgo.com | -- | -- | Search + results | Headless-friendly |
|
|
97
|
+
| news.ycombinator.com | -- | -- | Story link click | Clean, simple DOM |
|
|
98
|
+
| amazon.de | Banner dismissed | -- | -- | |
|
|
99
|
+
| theguardian.com | CMP dismissed | -- | -- | |
|
|
100
|
+
| spiegel.de | CMP dismissed | -- | -- | German |
|
|
101
|
+
| lemonde.fr | CMP dismissed | -- | -- | French |
|
|
102
|
+
| elpais.com | CMP dismissed | -- | -- | Spanish |
|
|
103
|
+
| corriere.it | CMP dismissed | -- | -- | Italian |
|
|
104
|
+
| nos.nl | CMP dismissed | -- | -- | Dutch |
|
|
105
|
+
| bild.de | CMP dismissed | -- | -- | German |
|
|
106
|
+
| nu.nl | CMP dismissed | -- | -- | Dutch |
|
|
107
|
+
| booking.com | Banner dismissed | -- | -- | |
|
|
108
|
+
| nytimes.com | -- | -- | -- | No consent wall |
|
|
109
|
+
| stackoverflow.com | Footer link only | -- | -- | Not blocking |
|
|
110
|
+
| cnn.com | -- | -- | -- | No consent wall |
|
|
111
|
+
| reddit.com | -- | -- | Fallback to old.reddit | Bot-blocks headless |
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Architecture
|
|
116
|
+
|
|
117
|
+
### Full pipeline: browse(url) or connect() -> goto(url)
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
1. LAUNCH chromium.js finds installed browser
|
|
121
|
+
Headless: spawn fresh Chromium with permission flags
|
|
122
|
+
Headed: connect to running browser on CDP port
|
|
123
|
+
Hybrid: try headless, detect challenge page, fallback to headed
|
|
124
|
+
|
|
125
|
+
2. CDP CONNECTION cdp.js opens WebSocket to browser
|
|
126
|
+
Creates page target, attaches flattened session
|
|
127
|
+
Enables Page, Network, DOM domains
|
|
128
|
+
|
|
129
|
+
3. STEALTH stealth.js (headless only)
|
|
130
|
+
Page.addScriptToEvaluateOnNewDocument before any page scripts
|
|
131
|
+
Patches: navigator.webdriver, plugins, languages, chrome object
|
|
132
|
+
|
|
133
|
+
4. PERMISSIONS Browser.setPermission denies all prompts
|
|
134
|
+
geo, notifications, camera, mic, midi, sensors, idle
|
|
135
|
+
|
|
136
|
+
5. AUTH auth.js extracts cookies from user's browser
|
|
137
|
+
Firefox: SQLite cookies.sqlite (plaintext)
|
|
138
|
+
Chromium: SQLite Cookies + AES decrypt via keyring
|
|
139
|
+
Injects via Network.setCookie before navigation
|
|
140
|
+
|
|
141
|
+
6. NAVIGATE Page.navigate(url), wait for Page.loadEventFired
|
|
142
|
+
500ms settle for dynamic content
|
|
143
|
+
|
|
144
|
+
7. CONSENT consent.js scans ARIA tree post-load
|
|
145
|
+
Finds dialog/alertdialog with consent hints
|
|
146
|
+
Falls back to global button scan (BBC SourcePoint pattern)
|
|
147
|
+
jsClick via DOM.resolveNode (bypasses iframe overlays)
|
|
148
|
+
|
|
149
|
+
8. SNAPSHOT Accessibility.getFullAXTree -> nested tree (aria.js)
|
|
150
|
+
prune.js: 9-step pipeline (47-95% token reduction)
|
|
151
|
+
Output: YAML-like text with [ref=N] markers
|
|
152
|
+
|
|
153
|
+
9. INTERACT interact.js dispatches real CDP Input events
|
|
154
|
+
click: scrollIntoView -> getBoxModel -> mousePressed/Released
|
|
155
|
+
type: DOM.focus -> insertText or keyDown/keyUp per char
|
|
156
|
+
press: special keys (Enter, Tab, Escape, arrows, etc.)
|
|
157
|
+
scroll: mouseWheel events
|
|
158
|
+
hover: mouseMoved at element center
|
|
159
|
+
select: set <select> value or click custom dropdown option
|
|
160
|
+
|
|
161
|
+
10. OBSERVE AGAIN Back to step 8. Refs are ephemeral -- fresh snapshot needed.
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Module table
|
|
165
|
+
|
|
166
|
+
Eleven modules, 2,396 lines, zero required dependencies.
|
|
167
|
+
|
|
168
|
+
| Module | Lines | Purpose |
|
|
169
|
+
|---|---|---|
|
|
170
|
+
| `src/index.js` | 434 | Public API: `browse()`, `connect()`, screenshot, network idle, hybrid |
|
|
171
|
+
| `src/cdp.js` | 148 | WebSocket CDP client, flattened sessions |
|
|
172
|
+
| `src/chromium.js` | 148 | Find/launch Chromium browsers, permission-suppressing flags |
|
|
173
|
+
| `src/aria.js` | 69 | Format ARIA tree as YAML-like text |
|
|
174
|
+
| `src/auth.js` | 279 | Cookie extraction (Chromium AES + keyring, Firefox), CDP injection |
|
|
175
|
+
| `src/prune.js` | 472 | ARIA pruning pipeline (9-step, ported from mcprune) |
|
|
176
|
+
| `src/interact.js` | 208 | Click, type, press, scroll, hover, select |
|
|
177
|
+
| `src/consent.js` | 210 | Auto-dismiss cookie consent dialogs, 7 languages |
|
|
178
|
+
| `src/stealth.js` | 51 | Navigator patches for headless anti-detection |
|
|
179
|
+
| `src/bareagent.js` | 161 | Tool adapter for bareagent Loop |
|
|
180
|
+
| `mcp-server.js` | 216 | MCP server (JSON-RPC 2.0 over stdio) |
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## What's Built
|
|
185
|
+
|
|
186
|
+
### Headless mode -- done
|
|
187
|
+
Spawn a fresh Chromium, navigate, snapshot, close. Default mode.
|
|
188
|
+
- Cookie extraction from user's Firefox or Chromium profile
|
|
189
|
+
- Cookie injection via `Network.setCookie` before navigation
|
|
190
|
+
- ARIA tree extraction via `Accessibility.getFullAXTree`
|
|
191
|
+
- 9-step pruning: landmarks, noise removal, wrapper collapsing, context filtering
|
|
192
|
+
- 47-95% token reduction depending on page complexity
|
|
193
|
+
- Permission prompts auto-suppressed (notifications, geolocation, camera, mic)
|
|
194
|
+
- Stealth patches: `navigator.webdriver`, plugins, languages, chrome object
|
|
195
|
+
|
|
196
|
+
### Headed mode -- done
|
|
197
|
+
Connect to an already-running browser on a CDP debug port.
|
|
198
|
+
- Same ARIA + prune pipeline
|
|
199
|
+
- Manual cookie injection via `page.injectCookies(url, { browser })` (e.g. inject Firefox cookies into headed Chromium)
|
|
200
|
+
- Permission prompts suppressed via CDP `Browser.setPermission`
|
|
201
|
+
- User must launch browser with `--remote-debugging-port=9222`
|
|
202
|
+
|
|
203
|
+
### Hybrid mode -- done
|
|
204
|
+
Try headless first. If bot-blocked (Cloudflare, etc.), fall back to headed automatically.
|
|
205
|
+
- Detection: heuristic on ARIA tree for challenge phrases ("Just a moment", "Checking your browser")
|
|
206
|
+
- Fallback: kill headless, connect to user's running browser on port 9222, re-navigate
|
|
207
|
+
- One flag: `mode: 'hybrid'`
|
|
208
|
+
|
|
209
|
+
### Interactions -- done, real-world tested
|
|
210
|
+
On `connect()` sessions: `click(ref)`, `type(ref, text, opts)`, `press(key)`, `scroll(deltaY)`, `hover(ref)`, `select(ref, value)`, `screenshot()`, `waitForNavigation()`, `waitForNetworkIdle()`, `injectCookies(url, opts)`.
|
|
211
|
+
- Refs come from ARIA snapshot (`[ref=N]` markers)
|
|
212
|
+
- Click: `DOM.scrollIntoViewIfNeeded` -> `DOM.getBoxModel` -> center -> `Input.dispatchMouseEvent`
|
|
213
|
+
- Type: `DOM.focus` + `Input.insertText` (fast) or `Input.dispatchKeyEvent` (triggers handlers)
|
|
214
|
+
- Type with `{ clear: true }`: select-all (Ctrl+A) + delete before typing
|
|
215
|
+
- Press: special keys (Enter, Tab, Escape, Backspace, arrows) with proper key/code/keyCode
|
|
216
|
+
- Scroll: `Input.dispatchMouseEvent` mouseWheel
|
|
217
|
+
- Hover: `DOM.scrollIntoViewIfNeeded` -> `Input.dispatchMouseEvent` mouseMoved
|
|
218
|
+
- Select: native `<select>` (set value + change event) or custom dropdown (click + find option)
|
|
219
|
+
- Screenshot: `Page.captureScreenshot` -> base64 string (png/jpeg/webp)
|
|
220
|
+
- WaitForNavigation: `Promise.race` of `Page.loadEventFired` + `Page.frameNavigated` (SPA-aware)
|
|
221
|
+
- WaitForNetworkIdle: track pending requests, resolve when 0 for N ms
|
|
222
|
+
|
|
223
|
+
**Real-world tested against:** Google, Wikipedia, GitHub (SPA), Hacker News, DuckDuckGo, YouTube (search + video playback), example.com
|
|
224
|
+
|
|
225
|
+
### Cookie consent auto-dismiss -- done
|
|
226
|
+
Automatically detects and dismisses GDPR/cookie consent dialogs after page load.
|
|
227
|
+
- Scans ARIA tree for `dialog`/`alertdialog` with consent-related content
|
|
228
|
+
- Falls back to global button scan for sites that don't use dialog roles (e.g. BBC SourcePoint)
|
|
229
|
+
- Uses JS `.click()` via `DOM.resolveNode` + `Runtime.callFunctionOn` to bypass iframe overlays
|
|
230
|
+
- Multi-language: EN, NL, DE, FR, ES, IT, PT button text patterns
|
|
231
|
+
- Opt-out via `{ consent: false }`
|
|
232
|
+
- Works in both headless and headed modes
|
|
233
|
+
|
|
234
|
+
**Tested against 16+ sites across 8 countries, 0 consent dialogs remaining.**
|
|
235
|
+
|
|
236
|
+
### Permission suppression -- done
|
|
237
|
+
Chrome permission prompts (location, notifications, camera, mic, etc.) are suppressed automatically.
|
|
238
|
+
- Headless: launch flags (`--disable-notifications`, `--autoplay-policy=no-user-gesture-required`, `--use-fake-device-for-media-stream`, `--use-fake-ui-for-media-stream`, `--disable-features=MediaRouter`)
|
|
239
|
+
- Both modes: CDP `Browser.setPermission` denies geolocation, notifications, midi, audioCapture, videoCapture, sensors, idleDetection, etc.
|
|
240
|
+
- No user prompt ever appears -- agents browse without interruption
|
|
241
|
+
|
|
242
|
+
### Cross-browser cookie injection -- done
|
|
243
|
+
Firefox cookies (user's default browser) extracted from SQLite -> injected into headless or headed Chromium via CDP `Network.setCookie`. No need to use Chromium as daily browser.
|
|
244
|
+
- `browse()`: auto-injects cookies before navigation (opt-out with `{ cookies: false }`)
|
|
245
|
+
- `connect()`: manual injection via `page.injectCookies(url, { browser: 'firefox' })`
|
|
246
|
+
- Proven: YouTube login session transferred from Firefox -> headed Chromium -> video playback
|
|
247
|
+
|
|
248
|
+
### Stealth patches -- done
|
|
249
|
+
Anti-detection for headless mode via `Page.addScriptToEvaluateOnNewDocument` (runs before page scripts).
|
|
250
|
+
- `navigator.webdriver` -> undefined
|
|
251
|
+
- `navigator.plugins` -> fake 3 plugins
|
|
252
|
+
- `navigator.languages` -> `['en-US', 'en']`
|
|
253
|
+
- `window.chrome` -> fake object
|
|
254
|
+
- `Permissions.prototype.query` -> notifications return 'prompt'
|
|
255
|
+
- Applied automatically in headless mode
|
|
256
|
+
|
|
257
|
+
### Tests -- 47+ passing
|
|
258
|
+
- 16 unit tests (pruning logic)
|
|
259
|
+
- 7 unit tests (cookie extraction -- 2 skip when Chromium profile locked)
|
|
260
|
+
- 5 unit tests (CDP client + browser launch)
|
|
261
|
+
- 11 integration tests (end-to-end browse pipeline)
|
|
262
|
+
- 15 integration tests (real-world interactions: data: URL fixture + live sites)
|
|
263
|
+
|
|
264
|
+
---
|
|
265
|
+
|
|
266
|
+
## Integrations
|
|
267
|
+
|
|
268
|
+
### bareagent -- tool adapter
|
|
269
|
+
|
|
270
|
+
`createBrowseTools(opts)` returns bareagent-compatible tools for the Loop:
|
|
271
|
+
|
|
272
|
+
```js
|
|
273
|
+
import { Loop } from 'bare-agent';
|
|
274
|
+
import { Anthropic } from 'bare-agent/providers';
|
|
275
|
+
import { createBrowseTools } from 'barebrowse/src/bareagent.js';
|
|
276
|
+
|
|
277
|
+
const { tools, close } = createBrowseTools();
|
|
278
|
+
const loop = new Loop({ provider: new Anthropic({ apiKey }) });
|
|
279
|
+
const result = await loop.run(messages, tools);
|
|
280
|
+
await close();
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
9 tools: browse, goto, snapshot, click, type, press, scroll, select, screenshot.
|
|
284
|
+
Action tools auto-return snapshot (300ms settle delay). The LLM always sees the result.
|
|
285
|
+
|
|
286
|
+
### MCP server
|
|
287
|
+
|
|
288
|
+
Raw JSON-RPC 2.0 over stdio. Zero SDK dependencies. `npm install barebrowse` then:
|
|
289
|
+
|
|
290
|
+
```json
|
|
291
|
+
{
|
|
292
|
+
"mcpServers": {
|
|
293
|
+
"barebrowse": {
|
|
294
|
+
"command": "npx",
|
|
295
|
+
"args": ["barebrowse", "mcp"]
|
|
296
|
+
}
|
|
297
|
+
}
|
|
298
|
+
}
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
7 tools: browse (one-shot), goto, snapshot, click, type, press, scroll.
|
|
302
|
+
Action tools return `'ok'` -- agent calls `snapshot` explicitly (MCP tool calls are cheap to chain).
|
|
303
|
+
Session tools share a singleton page, lazy-created on first use.
|
|
304
|
+
|
|
305
|
+
---
|
|
306
|
+
|
|
307
|
+
## Ecosystem
|
|
308
|
+
|
|
309
|
+
```
|
|
310
|
+
bareagent = the brain (orchestration, LLM loop, memory, retries)
|
|
311
|
+
barebrowse = the eyes + hands (browse, read, interact with the web)
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
**barebrowse is a library.** bareagent imports it as a capability. barebrowse doesn't know about bareagent. bareagent doesn't know about CDP. Clean boundary. Each ships and tests independently.
|
|
315
|
+
|
|
316
|
+
---
|
|
317
|
+
|
|
318
|
+
## Constraints
|
|
319
|
+
|
|
320
|
+
- **Chromium-only.** CDP protocol. Covers Chrome, Chromium, Edge, Brave, Vivaldi, Arc, Opera (~80% desktop share). Firefox later via WebDriver BiDi.
|
|
321
|
+
- **Linux first.** Tested on Fedora/KDE. macOS/Windows cookie extraction paths exist in auth.js but untested.
|
|
322
|
+
- **Node >= 22.** Built-in WebSocket, built-in SQLite.
|
|
323
|
+
- **Not a server.** Library that agents import. Wrap as MCP (included) or HTTP if needed.
|
|
324
|
+
- **Not cross-platform tested.** Local development only, not published to npm.
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
## File Map
|
|
329
|
+
|
|
330
|
+
```
|
|
331
|
+
barebrowse/
|
|
332
|
+
├── src/
|
|
333
|
+
│ ├── index.js # Public API: browse(), connect(), screenshot, network idle, hybrid
|
|
334
|
+
│ ├── cdp.js # WebSocket CDP client
|
|
335
|
+
│ ├── chromium.js # Find/launch Chromium, permission flags
|
|
336
|
+
│ ├── aria.js # ARIA tree formatting
|
|
337
|
+
│ ├── auth.js # Cookie extraction + injection
|
|
338
|
+
│ ├── prune.js # ARIA pruning (9-step pipeline)
|
|
339
|
+
│ ├── interact.js # Click, type, press, scroll, hover, select
|
|
340
|
+
│ ├── consent.js # Auto-dismiss cookie consent dialogs
|
|
341
|
+
│ ├── stealth.js # Navigator patches for headless anti-detection
|
|
342
|
+
│ └── bareagent.js # Tool adapter for bareagent Loop
|
|
343
|
+
├── test/
|
|
344
|
+
│ ├── unit/ # prune, auth, cdp tests
|
|
345
|
+
│ └── integration/ # browse + interact tests (real sites)
|
|
346
|
+
├── examples/
|
|
347
|
+
│ ├── headed-demo.js # Interactive demo: Wikipedia → DuckDuckGo
|
|
348
|
+
│ └── yt-demo.js # YouTube demo: Firefox cookies → search → play video
|
|
349
|
+
├── docs/
|
|
350
|
+
│ ├── prd.md # Decisions + rationale (reference)
|
|
351
|
+
│ ├── poc-plan.md # Original POC phases + DoD
|
|
352
|
+
│ ├── blueprint.md # This file
|
|
353
|
+
│ └── testing.md # Test guide: pyramid, all 54 tests, CI strategy
|
|
354
|
+
├── mcp-server.js # MCP server (JSON-RPC 2.0 over stdio)
|
|
355
|
+
├── cli.js # CLI entry: `npx barebrowse mcp` or `npx barebrowse browse <url>`
|
|
356
|
+
├── .mcp.json # MCP server config for Claude Desktop / Cursor
|
|
357
|
+
├── barebrowse.context.md # LLM-consumable integration guide
|
|
358
|
+
├── package.json
|
|
359
|
+
├── README.md
|
|
360
|
+
└── CLAUDE.md
|
|
361
|
+
```
|
package/docs/poc-plan.md
ADDED
|
@@ -0,0 +1,230 @@
|
|
|
1
|
+
# barebrowse — POC Plan
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-02-22
|
|
4
|
+
**Goal:** Prove that an autonomous agent can get an authenticated, pruned ARIA snapshot of any web page via CDP — no Playwright, no bundled browser.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Repo Structure
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
barebrowse/
|
|
12
|
+
├── src/
|
|
13
|
+
│ ├── index.js # Public API: browse(), connect()
|
|
14
|
+
│ ├── chromium.js # Find/launch/connect to Chromium browsers
|
|
15
|
+
│ ├── cdp.js # Vanilla WebSocket CDP client
|
|
16
|
+
│ ├── aria.js # Accessibility.getFullAXTree → structured tree
|
|
17
|
+
│ ├── auth.js # Cookie extraction + CDP Network.setCookie injection
|
|
18
|
+
│ ├── prune.js # ARIA tree pruning (ported from mcprune)
|
|
19
|
+
│ ├── interact.js # Click, type, scroll via CDP Input domain
|
|
20
|
+
│ ├── consent.js # Auto-dismiss cookie consent dialogs
|
|
21
|
+
│ └── stealth.js # Anti-detection patches via Runtime.evaluate (Phase 4)
|
|
22
|
+
├── test/
|
|
23
|
+
│ ├── integration/
|
|
24
|
+
│ │ ├── browse.test.js # End-to-end: URL → pruned snapshot
|
|
25
|
+
│ │ ├── auth.test.js # Cookie injection → authenticated page
|
|
26
|
+
│ │ └── interact.test.js # Click/type on a live page
|
|
27
|
+
│ └── unit/
|
|
28
|
+
│ ├── cdp.test.js # CDP client message handling
|
|
29
|
+
│ ├── aria.test.js # ARIA tree formatting
|
|
30
|
+
│ └── prune.test.js # Pruning logic on sample trees
|
|
31
|
+
├── docs/
|
|
32
|
+
│ ├── prd.md # Product requirements (comprehensive)
|
|
33
|
+
│ └── poc-plan.md # This file
|
|
34
|
+
├── package.json
|
|
35
|
+
└── CLAUDE.md
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
**No build step.** Vanilla JS, ES modules, runs directly with Node.js >= 22.
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Phases
|
|
43
|
+
|
|
44
|
+
### Phase 1 — CDP + ARIA Foundation
|
|
45
|
+
|
|
46
|
+
**Prove:** Get an ARIA tree from any page via CDP, no Playwright.
|
|
47
|
+
|
|
48
|
+
**Files:**
|
|
49
|
+
- `src/chromium.js` — Find installed Chromium browsers on the system (Chrome, Chromium, Brave, Edge). Launch headless with `--headless=new --remote-debugging-port=<port>`. Parse CDP WebSocket URL from stderr output.
|
|
50
|
+
- `src/cdp.js` — Vanilla WebSocket client that speaks CDP. Send JSON commands, receive responses and events. Handle command IDs, promises, event subscriptions. ~100 lines.
|
|
51
|
+
- `src/aria.js` — Call `Accessibility.getFullAXTree` via CDP. Transform the raw CDP response (flat array of AXNodes with parentId references) into a nested tree structure. Format as readable output.
|
|
52
|
+
- `src/index.js` — Wire chromium → cdp → aria into `browse(url)` function. Minimal, just the pipeline.
|
|
53
|
+
|
|
54
|
+
**Test:**
|
|
55
|
+
```bash
|
|
56
|
+
node -e "import { browse } from './src/index.js'; console.log(await browse('https://example.com'))"
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
**DoD:**
|
|
60
|
+
- [x] `chromium.js` finds and launches at least one Chromium browser on Fedora Linux
|
|
61
|
+
- [x] `cdp.js` connects via WebSocket, sends commands, receives responses
|
|
62
|
+
- [x] `aria.js` returns a structured ARIA tree for any public page
|
|
63
|
+
- [x] `browse(url)` works end-to-end with zero external dependencies
|
|
64
|
+
- [x] Headless Chrome process is cleaned up on close
|
|
65
|
+
|
|
66
|
+
### Phase 2 — Auth + Prune
|
|
67
|
+
|
|
68
|
+
**Prove:** Authenticated, pruned ARIA snapshot of a Cloudflare-protected page.
|
|
69
|
+
|
|
70
|
+
**Files:**
|
|
71
|
+
- `src/auth.js` — Extract cookies from user's browser profile (use sweet-cookie or implement minimal extraction from Chrome's Cookies SQLite DB + Linux keyring decryption via `secret-tool`). Inject via CDP `Network.setCookie` before navigation.
|
|
72
|
+
- `src/prune.js` — Port mcprune's pruning logic as a pure function. Input: raw ARIA tree. Output: pruned ARIA tree. Role-based: keep landmarks + interactive elements, drop noise/structural wrappers.
|
|
73
|
+
- Update `src/index.js` — Add cookie injection and pruning to the `browse()` pipeline.
|
|
74
|
+
|
|
75
|
+
**Test:**
|
|
76
|
+
```bash
|
|
77
|
+
# Should return authenticated content, not a login wall or CF challenge
|
|
78
|
+
node -e "import { browse } from './src/index.js'; console.log(await browse('https://some-cf-protected-site.com'))"
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
**DoD:**
|
|
82
|
+
- [x] `auth.js` extracts cookies from Firefox profile on Linux (also supports Chromium when installed)
|
|
83
|
+
- [x] Cookies injected via CDP before navigation
|
|
84
|
+
- [ ] CF-protected page returns real content, not challenge page (needs active session to test)
|
|
85
|
+
- [x] `prune.js` reduces ARIA tree by 47%+ on HN (minimal site — heavier sites will see 70%+)
|
|
86
|
+
- [x] Pruned output preserves all interactive elements and landmarks
|
|
87
|
+
- [x] `browse(url)` returns pruned, authenticated snapshot by default
|
|
88
|
+
|
|
89
|
+
### Phase 3 — Headed Mode + Interaction
|
|
90
|
+
|
|
91
|
+
**Prove:** Connect to user's running browser and interact with a logged-in page.
|
|
92
|
+
|
|
93
|
+
**Files:**
|
|
94
|
+
- Update `src/chromium.js` — Add `connect()` mode: connect to an already-running browser's debug port instead of launching a new one. Detect running browsers with debug ports.
|
|
95
|
+
- `src/interact.js` — Click (`Input.dispatchMouseEvent`), type (`Input.dispatchKeyEvent`), scroll. Resolve ARIA node IDs to DOM coordinates for click targets.
|
|
96
|
+
- Update `src/index.js` — Add `connect()` export for long-lived sessions. Add `mode: 'headed'` option.
|
|
97
|
+
|
|
98
|
+
**Prerequisite:** User must launch their browser with `--remote-debugging-port=9222` flag.
|
|
99
|
+
|
|
100
|
+
**Test:**
|
|
101
|
+
```bash
|
|
102
|
+
# User has Chrome open with debug port, logged into GitHub
|
|
103
|
+
node -e "
|
|
104
|
+
import { connect } from './src/index.js';
|
|
105
|
+
const page = await connect({ mode: 'headed' });
|
|
106
|
+
await page.goto('https://github.com/notifications');
|
|
107
|
+
console.log(await page.snapshot());
|
|
108
|
+
"
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
**DoD:**
|
|
112
|
+
- [x] `connect()` attaches to a running Chromium browser via CDP
|
|
113
|
+
- [x] Same ARIA + prune pipeline works on headed browser
|
|
114
|
+
- [x] `click()` and `type()` send real input events via CDP
|
|
115
|
+
- [x] `press()` sends special keys (Enter, Tab, Escape, arrows) — triggers form submit
|
|
116
|
+
- [x] `scrollIntoView` before click ensures off-screen elements are reachable
|
|
117
|
+
- [x] `type({ clear: true })` replaces pre-filled input content
|
|
118
|
+
- [x] `waitForNavigation()` waits for page load after link clicks
|
|
119
|
+
- [x] Interactions tested against real sites: Wikipedia, GitHub, Google, Hacker News, DuckDuckGo, YouTube
|
|
120
|
+
- [x] Browser stays open after barebrowse disconnects
|
|
121
|
+
- [x] Cookie injection via `page.injectCookies()` for headed mode (Firefox → Chromium)
|
|
122
|
+
- [x] Permission prompts suppressed via launch flags + CDP `Browser.setPermission`
|
|
123
|
+
- [x] Cookie consent dialogs auto-dismissed across 16+ sites in 7 languages
|
|
124
|
+
- [x] YouTube end-to-end: Firefox cookies → search → click → video playback in headed mode
|
|
125
|
+
|
|
126
|
+
### Phase 4 — Hybrid + bareagent Integration
|
|
127
|
+
|
|
128
|
+
**Prove:** Agent autonomously browses the web using barebrowse tools.
|
|
129
|
+
|
|
130
|
+
**Files:**
|
|
131
|
+
- Update `src/chromium.js` — Add `mode: 'hybrid'`. Try headless first. If navigation returns a CF challenge or 403, automatically retry in headed mode.
|
|
132
|
+
- `src/stealth.js` — Basic anti-detection: patch `navigator.webdriver`, `navigator.plugins`, `window.chrome`. Applied via `Runtime.evaluate` on new page.
|
|
133
|
+
- Update `src/index.js` — Final API surface: `browse()`, `connect()`.
|
|
134
|
+
|
|
135
|
+
**Test:**
|
|
136
|
+
```js
|
|
137
|
+
import { Loop } from 'bare-agent';
|
|
138
|
+
import { browse } from './src/index.js';
|
|
139
|
+
|
|
140
|
+
const tools = [
|
|
141
|
+
{ name: 'browse', execute: ({ url }) => browse(url) },
|
|
142
|
+
];
|
|
143
|
+
|
|
144
|
+
const loop = new Loop({ provider });
|
|
145
|
+
await loop.run([
|
|
146
|
+
{ role: 'user', content: 'Go to hacker news and tell me the top 3 stories' }
|
|
147
|
+
], tools);
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
**DoD:**
|
|
151
|
+
- [ ] Hybrid mode automatically falls back when headless is blocked
|
|
152
|
+
- [ ] Stealth patches reduce headless detection on common sites
|
|
153
|
+
- [ ] bareagent can use `browse()` as a tool in its think/act/observe loop
|
|
154
|
+
- [ ] Agent successfully completes a multi-page research task autonomously
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## Definition of Done — Full POC
|
|
159
|
+
|
|
160
|
+
The POC is complete when ALL of these are true:
|
|
161
|
+
|
|
162
|
+
1. **`browse(url)` works end-to-end** — URL in, pruned ARIA snapshot out, authenticated as the user
|
|
163
|
+
2. **Zero heavy deps** — no Playwright, no Puppeteer. Only deps: `ws` (WebSocket client, if Node's built-in isn't sufficient) and optionally `sweet-cookie`
|
|
164
|
+
3. **Three modes work** — headless (default), headed (connect to running browser), hybrid (auto-fallback)
|
|
165
|
+
4. **Works on Fedora Linux** — finds Chrome/Chromium/Brave, launches headless, connects headed
|
|
166
|
+
5. **Token-efficient output** — pruned ARIA tree is 70%+ smaller than raw tree
|
|
167
|
+
6. **Clean process management** — headless browser spawned and killed cleanly, no orphan processes
|
|
168
|
+
7. **Under 1,000 lines total** for core src/ (excluding tests)
|
|
169
|
+
8. **Documented** — PRD captures all decisions, this file captures all phases
|
|
170
|
+
|
|
171
|
+
## What the POC is NOT
|
|
172
|
+
|
|
173
|
+
- Not production-ready. No error recovery, no retry logic, no edge case handling beyond happy path.
|
|
174
|
+
- Not cross-platform tested. Linux first (Fedora). macOS/Windows later.
|
|
175
|
+
- Not an MCP server. That's a future wrapper.
|
|
176
|
+
- Not a published npm package. Local development only.
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## Running Tests
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
# All tests (47+ tests)
|
|
184
|
+
node --test test/unit/*.test.js test/integration/*.test.js
|
|
185
|
+
|
|
186
|
+
# Unit tests only (fast, no network)
|
|
187
|
+
node --test test/unit/prune.test.js # 16 tests — pruning logic
|
|
188
|
+
node --test test/unit/auth.test.js # 7 tests — cookie extraction (2 fail when Chromium locked)
|
|
189
|
+
node --test test/unit/cdp.test.js # 5 tests — CDP client + browser launch
|
|
190
|
+
|
|
191
|
+
# Integration tests (needs network + Chromium)
|
|
192
|
+
node --test test/integration/browse.test.js # 11 tests — end-to-end pipeline
|
|
193
|
+
node --test test/integration/interact.test.js # 15 tests — interactions on real sites
|
|
194
|
+
|
|
195
|
+
# Quick smoke test
|
|
196
|
+
node -e "import { browse } from './src/index.js'; console.log(await browse('https://example.com'))"
|
|
197
|
+
|
|
198
|
+
# Headed mode demos (requires: chromium-browser --remote-debugging-port=9222)
|
|
199
|
+
node examples/headed-demo.js # Wikipedia → DuckDuckGo search
|
|
200
|
+
node examples/yt-demo.js # YouTube: Firefox cookies → search → play video
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
## Repos Studied — What We Borrowed vs Built
|
|
206
|
+
|
|
207
|
+
| steipete repo | What we studied | What we used | Why not more |
|
|
208
|
+
|---|---|---|---|
|
|
209
|
+
| **sweet-cookie** | Cookie extraction (SQLite + keyring) | **Concept only** — wrote `auth.js` ourselves | Not on npm (different package). Our version is simpler, tailored, vanilla JS |
|
|
210
|
+
| **sweetlink** | CDP dual-channel, selector discovery, daemon | **CDP-direct concept only** | Daemon + WebSocket bridge + in-page runtime = bloat. CDP direct is 100 lines vs ~2,000 |
|
|
211
|
+
| **canvas** | Stealth/anti-detection patterns | **Noted for Phase 4** `stealth.js` | Not needed yet — headless + real cookies handles most cases |
|
|
212
|
+
| **mcprune (own)** | ARIA pruning pipeline | **Full port** — `prune.js` (472 lines) | Proven code, adapted node format from Playwright YAML to CDP tree objects |
|
|
213
|
+
|
|
214
|
+
### What to explore in later phases
|
|
215
|
+
|
|
216
|
+
- **Selector discovery** (sweetlink) — crawl ARIA tree, score interactive elements, rank action targets. Phase 3/4.
|
|
217
|
+
- **Stealth patches** (canvas) — `navigator.webdriver`, plugins, chrome object spoofing. Phase 4.
|
|
218
|
+
- **In-page JS execution** (sweetlink) — `Runtime.evaluate` for complex interactions. Phase 3.
|
|
219
|
+
- **Screenshot + visual grounding** — `Page.captureScreenshot` for multimodal agents. Post-POC.
|
|
220
|
+
|
|
221
|
+
---
|
|
222
|
+
|
|
223
|
+
## Dev Rules (from AGENT_RULES.md)
|
|
224
|
+
|
|
225
|
+
- **Vanilla JS only.** No TypeScript, no build step, no transpilation.
|
|
226
|
+
- **Dependency hierarchy:** vanilla → stdlib → external. Write it yourself if <50 lines.
|
|
227
|
+
- **Simple > clever.** Readable code a junior can follow.
|
|
228
|
+
- **POC first.** Validate logic before designing. Never ship the POC — rewrite it.
|
|
229
|
+
- **Test behavior, not implementation.** Integration tests over unit tests.
|
|
230
|
+
- **No speculative code.** Every line must have a purpose.
|