barebrowse 0.4.2 → 0.4.4

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## 0.4.4
+
+Snapshot URL prefix and MCP large-snapshot handling.
+
+### Snapshot URL (`src/index.js`)
+- First line of every snapshot is now `# <current-page-url>`
+- Works in both `browse()` (uses the url param) and `connect().snapshot()` (uses `Page.getNavigationHistory`)
+
+### MCP maxChars (`mcp-server.js`)
+- `browse` and `snapshot` tools accept `maxChars` param (default 30000)
+- If snapshot exceeds limit: saved to `.barebrowse/page-<timestamp>.yml`, returns file path message
+- If under limit: returned inline as before
+
+### Docs
+- barebrowse.context.md: snapshot format updated with URL line, maxChars documented
+- commands/barebrowse.md + SKILL.md: snapshot example updated
+- docs/00-context/system-state.md: pipeline step 8 updated, MCP maxChars noted
+
+## 0.4.3
+
+Cookie consent expanded to 29 languages.
+
+- Added: RU, UK, PL, CS, TR, RO, HU, EL, SV, DA, NO, FI, AR, FA, ZH, JA, KO, VI, TH, HI, ID/MS
+- Dialog hints for 11 more languages
+- `.npmignore`: added `.idea/` (leaked in 0.4.2 tarball)
+
 ## 0.4.2
 
 Authenticated browsing improvements. MCP sessions now auto-inject cookies and fall back to headed mode when bot-detected.

package/README.md

@@ -16,7 +16,7 @@
 
 ## What this is
 
-barebrowse is agentic browsing stripped to the bone. It gives your AI agent eyes and hands on the web -- navigate any page, see what's there, click buttons, fill forms, scroll, and move on. It uses your installed Chromium browser (Chrome, Brave, Edge -- whatever you have), reuses your existing login sessions, and handles all the friction automatically: cookie consent walls, permission prompts, bot detection, GDPR dialogs.
+barebrowse is agentic browsing stripped to the bone. It gives your AI agent eyes and hands on the web -- navigate any page, see what's there, click buttons, fill forms, scroll, and move on. It uses your installed Chromium browser (Chrome, Brave, Edge -- whatever you have), reuses your existing login sessions, and handles all the friction automatically: cookie consent walls, permission prompts, and bot detection.
 
 Instead of dumping raw DOM or taking screenshots, barebrowse returns a **pruned ARIA snapshot** -- a compact semantic view of what's on the page and what the agent can interact with. Buttons, links, inputs, headings -- labeled with `[ref=N]` markers the agent uses to act. The pruning pipeline is ported from [mcprune](https://github.com/hamr0/mcprune) and cuts 40-90% of tokens compared to raw page output. Every token your agent reads is meaningful.
 
@@ -109,7 +109,7 @@ This is the obstacle course your agent doesn't have to think about:
 
 | Obstacle | How it's handled | Mode |
 |----------|-----------------|------|
-| **Cookie consent walls** (GDPR) | ARIA tree scan + jsClick accept button, 7 languages (EN, NL, DE, FR, ES, IT, PT) | Both |
+| **Cookie consent walls** | ARIA tree scan + jsClick accept button, 29 languages | Both |
 | **Consent in dialog role** | Detect `dialog`/`alertdialog` with consent hints, click accept inside | Both |
 | **Consent outside dialog** (BBC SourcePoint) | Fallback global button scan when dialog has no accept button | Both |
 | **Consent behind iframe overlay** | JS click via DOM.resolveNode bypasses z-index/overlay issues | Both |

package/barebrowse.context.md

@@ -87,14 +87,13 @@ const snapshot = await browse('https://example.com', {
 
 ## Snapshot format
 
-The snapshot is a YAML-like ARIA tree. Each line is one node:
+The snapshot is a YAML-like ARIA tree. First line is the page URL, second is pruning stats, then the tree:
 
 ```
-- WebArea "Example Domain" [ref=1]
-  - heading "Example Domain" [level=1] [ref=3]
-  - paragraph [ref=5]
-    - StaticText "This domain is for use in illustrative examples." [ref=6]
-  - link "More information..." [ref=8]
+# https://example.com/
+# 379 chars → 45 chars (88% pruned)
+- heading "Example Domain" [level=1] [ref=3]
+- link "More information..." [ref=8]
 ```
 
 Key rules:
@@ -145,7 +144,7 @@ barebrowse can inject cookies from the user's real browser sessions, bypassing l
 
 | Obstacle | How | Mode |
 |---|---|---|
-| Cookie consent (GDPR) | ARIA scan + jsClick accept button, 7 languages | Both |
+| Cookie consent | ARIA scan + jsClick accept button, 29 languages | Both |
 | Consent behind iframes | JS `.click()` via DOM.resolveNode bypasses overlays | Both |
 | Permission prompts | Launch flags + CDP Browser.setPermission auto-deny | Both |
 | Media autoplay blocked | `--autoplay-policy=no-user-gesture-required` | Both |
@@ -238,6 +237,8 @@ barebrowse ships an MCP server for direct use with Claude Desktop, Cursor, or an
 
 Action tools return `'ok'` -- the agent calls `snapshot` explicitly to observe. This avoids double-token output since MCP tool calls are cheap to chain.
 
+`browse` and `snapshot` accept a `maxChars` param (default 30000). If the snapshot exceeds the limit, it's saved to `.barebrowse/page-<timestamp>.yml` and a short message with the file path is returned instead.
+
 Session runs in hybrid mode (headless with automatic headed fallback on bot detection). `goto` injects cookies from the user's browser before navigation for authenticated access.
 
 Session tools share a singleton page, lazy-created on first use.
@@ -288,7 +289,7 @@ URL -> chromium.js (find/launch browser, permission flags)
 
 7. **One page per connect().** Each `connect()` call creates one page. For multiple tabs, call `connect()` multiple times.
 
-8. **Consent dismiss is best-effort.** It handles 16+ tested sites across 7 languages but novel consent implementations may need manual handling. Disable with `{ consent: false }`.
+8. **Consent dismiss is best-effort.** It handles 16+ tested sites across 29 languages but novel consent implementations may need manual handling. Disable with `{ consent: false }`.
 
 9. **Screenshot returns base64.** Write to file with `fs.writeFileSync('shot.png', Buffer.from(base64, 'base64'))` or pass directly to a vision model.
 

package/commands/barebrowse/SKILL.md

@@ -96,11 +96,9 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
 The snapshot is a YAML-like ARIA tree. Each line is one node:
 
 ```
-- WebArea "Example Domain" [ref=1]
-  - heading "Example Domain" [level=1] [ref=3]
-  - paragraph [ref=5]
-    - StaticText "This domain is for use in illustrative examples." [ref=6]
-  - link "More information..." [ref=8]
+# https://example.com/
+# 379 chars → 45 chars (88% pruned)
+- heading "Example Domain" [level=1] [ref=3]
 ```
 
 - `[ref=N]` — Use this number with click, type, fill, hover, select, drag, upload

package/commands/barebrowse.md

@@ -95,11 +95,9 @@ All output files go to `.barebrowse/` in the current directory. Read them with t
 The snapshot is a YAML-like ARIA tree. Each line is one node:
 
 ```
-- WebArea "Example Domain" [ref=1]
-  - heading "Example Domain" [level=1] [ref=3]
-  - paragraph [ref=5]
-    - StaticText "This domain is for use in illustrative examples." [ref=6]
-  - link "More information..." [ref=8]
+# https://example.com/
+# 379 chars → 45 chars (88% pruned)
+- heading "Example Domain" [level=1] [ref=3]
 ```
 
 - `[ref=N]` — Use this number with click, type, fill, hover, select, drag, upload

package/docs/00-context/system-state.md

@@ -64,7 +64,7 @@ Every action returns a **pruned ARIA snapshot** -- the agent's view of the page
 
 | Obstacle | How It's Handled | Mode |
 |----------|-----------------|------|
-| **Cookie consent walls** (GDPR) | ARIA tree scan, jsClick accept button. 7 languages: EN, NL, DE, FR, ES, IT, PT | Both |
+| **Cookie consent walls** | ARIA tree scan, jsClick accept button. 29 languages | Both |
 | **Consent in dialog role** | Detect `dialog`/`alertdialog` with consent hints, click accept inside | Both |
 | **Consent outside dialog** (BBC SourcePoint) | Fallback global button scan when dialog has no accept button | Both |
 | **Consent behind iframe overlay** | JS `.click()` via `DOM.resolveNode` bypasses z-index/overlay issues | Both |
@@ -155,7 +155,7 @@ Every action returns a **pruned ARIA snapshot** -- the agent's view of the page
 
 8. SNAPSHOT          Accessibility.getFullAXTree -> nested tree (aria.js)
                      prune.js: 9-step pipeline (47-95% token reduction)
-                     Output: YAML-like text with [ref=N] markers
+                     Output: URL + pruning stats + YAML-like text with [ref=N] markers
 
 9. INTERACT          interact.js dispatches real CDP Input events
                      click: scrollIntoView -> getBoxModel -> mousePressed/Released
@@ -181,7 +181,7 @@ Thirteen modules, zero required dependencies.
 | `src/auth.js` | 279 | Cookie extraction (Chromium AES + keyring, Firefox), CDP injection |
 | `src/prune.js` | 472 | ARIA pruning pipeline (9-step, ported from mcprune) |
 | `src/interact.js` | 208 | Click, type, press, scroll, hover, select |
-| `src/consent.js` | 210 | Auto-dismiss cookie consent dialogs, 7 languages |
+| `src/consent.js` | ~280 | Auto-dismiss cookie consent dialogs, 29 languages |
 | `src/stealth.js` | 51 | Navigator patches for headless anti-detection |
 | `src/bareagent.js` | 161 | Tool adapter for bareagent Loop |
 | `src/daemon.js` | ~230 | Background HTTP server holding connect() session for CLI mode |
@@ -232,11 +232,11 @@ On `connect()` sessions: `click(ref)`, `type(ref, text, opts)`, `press(key)`, `s
 **Real-world tested against:** Google, Wikipedia, GitHub (SPA), Hacker News, DuckDuckGo, YouTube (search + video playback), example.com
 
 ### Cookie consent auto-dismiss -- done
-Automatically detects and dismisses GDPR/cookie consent dialogs after page load.
+Automatically detects and dismisses cookie consent dialogs after page load.
 - Scans ARIA tree for `dialog`/`alertdialog` with consent-related content
 - Falls back to global button scan for sites that don't use dialog roles (e.g. BBC SourcePoint)
 - Uses JS `.click()` via `DOM.resolveNode` + `Runtime.callFunctionOn` to bypass iframe overlays
-- Multi-language: EN, NL, DE, FR, ES, IT, PT button text patterns
+- 29 languages: EN, NL, DE, FR, ES, IT, PT, RU, UK, PL, CS, TR, RO, HU, EL, SV, DA, NO, FI, AR, FA, ZH, JA, KO, VI, TH, HI, ID/MS
 - Opt-out via `{ consent: false }`
 - Works in both headless and headed modes
 
@@ -311,6 +311,7 @@ Raw JSON-RPC 2.0 over stdio. Zero SDK dependencies. `npm install barebrowse` the
 
 12 tools: browse (one-shot), goto, snapshot, click, type, press, scroll, back, forward, drag, upload, pdf.
 Action tools return `'ok'` -- agent calls `snapshot` explicitly (MCP tool calls are cheap to chain).
+`browse` and `snapshot` accept `maxChars` (default 30000) — large snapshots are saved to `.barebrowse/` and a file path is returned.
 Session runs in hybrid mode (headless + automatic headed fallback on bot detection). `goto` injects cookies from the user's browser before navigation.
 Session tools share a singleton page, lazy-created on first use.
 

package/mcp-server.js

@@ -10,6 +10,19 @@
  */
 
 import { browse, connect } from './src/index.js';
+import { mkdirSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+
+const MAX_CHARS_DEFAULT = 30000;
+const OUTPUT_DIR = join(process.cwd(), '.barebrowse');
+
+function saveSnapshot(text) {
+  mkdirSync(OUTPUT_DIR, { recursive: true });
+  const ts = new Date().toISOString().replace(/[:.]/g, '-');
+  const file = join(OUTPUT_DIR, `page-${ts}.yml`);
+  writeFileSync(file, text);
+  return file;
+}
 
 let _page = null;
 
@@ -27,6 +40,7 @@ const TOOLS = [
       properties: {
         url: { type: 'string', description: 'URL to browse' },
         mode: { type: 'string', enum: ['headless', 'headed', 'hybrid'], description: 'Browser mode (default: headless)' },
+        maxChars: { type: 'number', description: 'Max chars to return inline. Larger snapshots are saved to .barebrowse/ and a file path is returned instead. Default: 30000.' },
       },
       required: ['url'],
     },
@@ -45,7 +59,12 @@ const TOOLS = [
   {
     name: 'snapshot',
     description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements.',
-    inputSchema: { type: 'object', properties: {} },
+    inputSchema: {
+      type: 'object',
+      properties: {
+        maxChars: { type: 'number', description: 'Max chars to return inline. Larger snapshots are saved to .barebrowse/ and a file path is returned instead. Default: 30000.' },
+      },
+    },
   },
   {
     name: 'click',
@@ -141,9 +160,15 @@ const TOOLS = [
 
 async function handleToolCall(name, args) {
   switch (name) {
-    case 'browse':
-      return await browse(args.url, { mode: args.mode });
-
+    case 'browse': {
+      const text = await browse(args.url, { mode: args.mode });
+      const limit = args.maxChars ?? MAX_CHARS_DEFAULT;
+      if (text.length > limit) {
+        const file = saveSnapshot(text);
+        return `Snapshot (${text.length} chars) saved to ${file}`;
+      }
+      return text;
+    }
     case 'goto': {
       const page = await getPage();
       try { await page.injectCookies(args.url); } catch {}
@@ -152,7 +177,13 @@ async function handleToolCall(name, args) {
     }
     case 'snapshot': {
       const page = await getPage();
-      return await page.snapshot();
+      const text = await page.snapshot();
+      const limit = args.maxChars ?? MAX_CHARS_DEFAULT;
+      if (text.length > limit) {
+        const file = saveSnapshot(text);
+        return `Snapshot (${text.length} chars) saved to ${file}`;
+      }
+      return text;
     }
     case 'click': {
       const page = await getPage();
@@ -218,7 +249,7 @@ async function handleMessage(msg) {
     return jsonrpcResponse(id, {
       protocolVersion: '2024-11-05',
       capabilities: { tools: {} },
-      serverInfo: { name: 'barebrowse', version: '0.4.2' },
+      serverInfo: { name: 'barebrowse', version: '0.4.4' },
     });
   }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "barebrowse",
-  "version": "0.4.2",
+  "version": "0.4.4",
   "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "type": "module",
   "main": "src/index.js",

package/src/consent.js

@@ -38,6 +38,86 @@ const ACCEPT_PATTERNS = [
   /\baccetto\b/i,
   // Portuguese
   /\baceitar\s*tudo\b/i,
+  // Russian
+  /принять\s*все/i,
+  /принять/i,
+  /согласен/i,
+  // Ukrainian
+  /прийняти\s*все/i,
+  /прийняти/i,
+  // Polish
+  /zaakceptuj\s*wszystk/i,
+  /akceptuj\s*wszystk/i,
+  /zgadzam\s*się/i,
+  // Czech
+  /přijmout\s*vše/i,
+  /souhlasím/i,
+  // Turkish
+  /tümünü\s*kabul\s*et/i,
+  /kabul\s*et/i,
+  /kabul\s*ediyorum/i,
+  // Romanian
+  /acceptă\s*tot/i,
+  /accept\s*toate/i,
+  // Hungarian
+  /összes\s*elfogadás/i,
+  /elfogad/i,
+  // Greek
+  /αποδοχή\s*όλων/i,
+  /αποδέχομαι/i,
+  // Swedish
+  /acceptera\s*alla/i,
+  /godkänn\s*alla/i,
+  // Danish
+  /accepter\s*alle/i,
+  /acceptér\s*alle/i,
+  // Norwegian
+  /godta\s*alle/i,
+  /aksepter\s*alle/i,
+  // Finnish
+  /hyväksy\s*kaikki/i,
+  /hyväksyn/i,
+  // Arabic
+  /قبول\s*الكل/,
+  /قبول\s*الجميع/,
+  /موافق/,
+  /قبول/,
+  // Persian
+  /پذیرش\s*همه/,
+  /موافقم/,
+  /پذیرش/,
+  // Chinese (Simplified + Traditional)
+  /全部接受/,
+  /接受所有/,
+  /接受全部/,
+  /同意并继续/,
+  /全部接受/,
+  /接受/,
+  /同意/,
+  // Japanese
+  /すべて受け入れ/,
+  /すべて許可/,
+  /同意する/,
+  /同意します/,
+  // Korean
+  /모두\s*수락/,
+  /모두\s*동의/,
+  /동의합니다/,
+  /수락/,
+  // Vietnamese
+  /chấp\s*nhận\s*tất\s*cả/i,
+  /đồng\s*ý\s*tất\s*cả/i,
+  /đồng\s*ý/i,
+  // Thai
+  /ยอมรับทั้งหมด/,
+  /ยอมรับ/,
+  // Hindi
+  /सभी\s*स्वीकार/,
+  /स्वीकार\s*करें/,
+  /सहमत/,
+  // Indonesian / Malay
+  /terima\s*semua/i,
+  /setuju/i,
   // Generic single-word fallbacks (only matched inside dialogs)
   /^accept$/i,
   /^agree$/i,
@@ -52,10 +132,22 @@ const CONSENT_DIALOG_HINTS = [
   /cookie/i,
   /consent/i,
   /privacy/i,
-  /voordat\s*je\s*verdergaat/i,  // Dutch: "Before you continue"
   /before\s*you\s*continue/i,
-  /bevor\s*du\s*fortf/i,         // German: "Before you continue"
-  /avant\s*de\s*continuer/i,     // French: "Before you continue"
+  /voordat\s*je\s*verdergaat/i,  // Dutch
+  /bevor\s*du\s*fortf/i,         // German
+  /avant\s*de\s*continuer/i,     // French
+  /antes\s*de\s*continuar/i,     // Spanish / Portuguese
+  /prima\s*di\s*continuare/i,    // Italian
+  /zanim\s*przejdziesz/i,        // Polish
+  /прежде\s*чем\s*продолжить/i,  // Russian
+  /devam\s*etmeden\s*önce/i,     // Turkish
+  /続行する前に/,                  // Japanese
+  /继续前/,                        // Chinese Simplified
+  /繼續前/,                        // Chinese Traditional
+  /계속하기\s*전에/,                // Korean
+  /trước\s*khi\s*tiếp\s*tục/i,   // Vietnamese
+  /ملفات\s*تعريف\s*الارتباط/,    // Arabic: cookies
+  /คุกกี้/,                        // Thai: cookies
 ];
 
 /**

package/src/index.js

@@ -103,7 +103,7 @@ export async function browse(url, opts = {}) {
     } else {
       snapshot = raw;
     }
-    const stats = `# ${raw.length.toLocaleString()} chars → ${snapshot.length.toLocaleString()} chars (${Math.round((1 - snapshot.length / raw.length) * 100)}% pruned)`;
+    const stats = `# ${url}\n# ${raw.length.toLocaleString()} chars → ${snapshot.length.toLocaleString()} chars (${Math.round((1 - snapshot.length / raw.length) * 100)}% pruned)`;
     snapshot = stats + '\n' + snapshot;
 
     // Step 7: Clean up
@@ -221,10 +221,12 @@ export async function connect(opts = {}) {
       const result = await ariaTree(page);
       refMap = result.refMap;
       const raw = formatTree(result.tree);
-      if (pruneOpts === false) return raw;
+      const { currentIndex, entries } = await page.session.send('Page.getNavigationHistory');
+      const pageUrl = entries[currentIndex]?.url || '';
+      if (pruneOpts === false) return `# ${pageUrl}\n` + raw;
       const pruned = pruneTree(result.tree, { mode: pruneOpts?.mode || 'act' });
       const out = formatTree(pruned);
-      const stats = `# ${raw.length.toLocaleString()} chars → ${out.length.toLocaleString()} chars (${Math.round((1 - out.length / raw.length) * 100)}% pruned)`;
+      const stats = `# ${pageUrl}\n# ${raw.length.toLocaleString()} chars → ${out.length.toLocaleString()} chars (${Math.round((1 - out.length / raw.length) * 100)}% pruned)`;
       return stats + '\n' + out;
     },
 

package/.idea/barebrowse.iml

@@ -1,10 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<module type="PYTHON_MODULE" version="4">
-  <component name="NewModuleRootManager">
-    <content url="file://$MODULE_DIR$">
-      <sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
-    </content>
-    <orderEntry type="jdk" jdkName="Python 3.14" jdkType="Python SDK" />
-    <orderEntry type="sourceFolder" forTests="false" />
-  </component>
-</module>

package/.idea/inspectionProfiles/profiles_settings.xml

@@ -1,6 +0,0 @@
-<component name="InspectionProjectProfileManager">
-  <settings>
-    <option name="USE_PROJECT_PROFILE" value="false" />
-    <version value="1.0" />
-  </settings>
-</component>

package/.idea/misc.xml

@@ -1,4 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<project version="4">
-  <component name="ProjectRootManager" version="2" project-jdk-name="Python 3.14" project-jdk-type="Python SDK" />
-</project>

package/.idea/modules.xml

@@ -1,8 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<project version="4">
-  <component name="ProjectModuleManager">
-    <modules>
-      <module fileurl="file://$PROJECT_DIR$/.idea/barebrowse.iml" filepath="$PROJECT_DIR$/.idea/barebrowse.iml" />
-    </modules>
-  </component>
-</project>

package/.idea/vcs.xml

@@ -1,6 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<project version="4">
-  <component name="VcsDirectoryMappings">
-    <mapping directory="" vcs="Git" />
-  </component>
-</project>