barebrowse 0.1.0 → 0.2.1

package/docs/testing.md

@@ -0,0 +1,202 @@
+# barebrowse -- Testing Guide
+
+## Run all tests
+
+```bash
+node --test test/unit/*.test.js test/integration/*.test.js
+```
+
+54 tests, 5 files, ~45s on a typical machine. No test framework -- uses Node's built-in `node:test` runner.
+
+## Test pyramid
+
+```
+          /  E2E  \         15 tests — real websites (Google, Wikipedia, GitHub, etc.)
+         /----------\
+        / Integration \     11 tests — full browse/connect pipeline against example.com, HN
+       /----------------\
+      /      Unit        \  28 tests — pruning, cookie extraction, CDP client, browser launch
+     /--------------------\
+```
+
+Unit tests are fast and isolated. Integration tests launch a real headless Chromium. E2E tests (part of interact.test.js) hit live websites — they require internet and may be slower or flaky on CI.
+
+---
+
+## Unit tests (28 tests)
+
+### `test/unit/prune.test.js` -- 16 tests
+
+Tests the 9-step ARIA pruning pipeline in isolation. No browser, no network.
+
+| # | Test | What it validates |
+|---|------|-------------------|
+| 1 | returns null for empty tree | prune(null) returns null |
+| 2 | unwraps RootWebArea | Root container node stripped from output |
+| 3 | keeps interactive elements in act mode | Buttons, links, textboxes survive pruning |
+| 4 | drops paragraphs in act mode | Non-interactive text removed in act mode |
+| 5 | keeps paragraphs in browse mode | Text content preserved in browse/read mode |
+| 6 | drops InlineTextBox noise | Low-level rendering nodes always filtered |
+| 7 | keeps headings | h1/h2 headings preserved in browse mode |
+| 8 | drops description headings in act mode | Only primary h1 kept, secondary headings removed |
+| 9 | collapses unnamed structural wrappers | Nested generic divs flattened, children promoted |
+| 10 | keeps named groups | Radiogroup/radio elements preserved |
+| 11 | drops separators | Separator/hr nodes always removed |
+| 12 | drops images in act mode, keeps named in browse | Act strips all images, browse keeps named ones |
+| 13 | trims combobox to just name + selected value | Combobox children (options list) stripped |
+| 14 | uses context keywords to condense non-matching cards | Context filtering collapses irrelevant list items |
+| 15 | extracts main landmark when present | Act mode keeps only main content area |
+| 16 | handles pages without landmarks (HN-style) | Pruning works on flat, landmark-less pages |
+
+### `test/unit/auth.test.js` -- 7 tests
+
+Tests cookie extraction from the local filesystem. Reads real browser cookie databases.
+
+| # | Test | What it validates |
+|---|------|-------------------|
+| 1 | auto-detects a browser and returns cookies | extractCookies() finds Firefox or Chromium and returns array |
+| 2 | returns cookies with correct shape | Each cookie has name, value, domain, path, secure, httpOnly, sameSite, expires |
+| 3 | filters by domain | Domain filter parameter restricts results |
+| 4 | extracts from firefox explicitly | `{ browser: 'firefox' }` parameter works |
+| 5 | throws for non-existent browser | Error thrown for unknown browser string |
+| 6 | cookies have non-empty values | All returned cookies have non-empty value strings |
+| 7 | sameSite is a valid value | sameSite is one of 'None', 'Lax', or 'Strict' |
+
+Note: 2 tests may skip when Chromium profile is locked by a running instance (AES decryption needs keyring access).
+
+### `test/unit/cdp.test.js` -- 5 tests
+
+Tests browser discovery, launch, CDP WebSocket client, and session handling.
+
+| # | Test | What it validates |
+|---|------|-------------------|
+| 1 | finds a Chromium-based browser | findBrowser() returns path to chromium/chrome/brave/edge |
+| 2 | launches headless Chromium and returns WebSocket URL | launch() returns valid ws:// URL, port, and live process |
+| 3 | connects to browser and sends commands | createCDP() connects, Browser.getVersion responds |
+| 4 | creates session-scoped handles | Target.createTarget + session() dispatches to correct target |
+| 5 | gets accessibility tree from a page | Accessibility.getFullAXTree returns nodes with role/name |
+
+---
+
+## Integration tests (11 tests)
+
+### `test/integration/browse.test.js` -- 11 tests
+
+Tests the full `browse()` and `connect()` pipeline end-to-end against real pages.
+
+| # | Suite | Test | What it validates |
+|---|-------|------|-------------------|
+| 1 | browse() | returns ARIA snapshot for a public page | browse('example.com') returns non-empty snapshot with title |
+| 2 | browse() | includes heading and ref markers | Snapshot contains roles and [ref=N] markers |
+| 3 | browse() | prunes by default (act mode) | Pruned output smaller than raw ARIA tree |
+| 4 | browse() | browse mode preserves paragraphs | pruneMode: 'browse' keeps text content |
+| 5 | browse() | act mode drops paragraphs | pruneMode: 'act' removes non-interactive text |
+| 6 | browse() | handles complex pages with significant reduction | Hacker News pruned by at least 20% |
+| 7 | browse() | can disable cookies | cookies: false works without error |
+| 8 | browse() | can disable pruning | prune: false keeps raw RootWebArea |
+| 9 | connect() | creates a long-lived session and navigates | connect() + goto() + snapshot() works |
+| 10 | connect() | supports multiple navigations in one session | Multiple goto() calls on same page |
+| 11 | connect() | snapshot accepts prune: false for raw output | snapshot(false) preserves full tree |
+
+---
+
+## E2E tests (15 tests)
+
+### `test/integration/interact.test.js` -- 15 tests
+
+Tests real interactions: clicking, typing, scrolling, form submission, and navigation. Uses a local `data:` URL fixture for deterministic tests, plus live websites for real-world coverage.
+
+#### Data URL fixture tests (7 tests)
+
+| # | Test | What it validates |
+|---|------|-------------------|
+| 1 | click sets button result text | page.click(ref) triggers onclick handler |
+| 2 | type fills an empty input | page.type(ref, text) fills empty textbox |
+| 3 | type with clear replaces existing text | { clear: true } replaces prefilled input |
+| 4 | click on offscreen element scrolls into view first | Auto-scroll before click on element at 3000px |
+| 5 | press Enter submits a form | page.press('Enter') triggers form onsubmit |
+| 6 | press throws on unknown key | Error thrown for unrecognized key names |
+| 7 | link click + waitForNavigation navigates | Cross-page navigation via click + waitForNavigation |
+
+#### Live website tests (8 tests)
+
+| # | Site | Test | What it validates |
+|---|------|------|-------------------|
+| 1 | Google | search and navigate results | type() + press('Enter') + waitForNavigation() on Google |
+| 2 | Wikipedia | navigate article links | click() + waitForNavigation() on Wikipedia article links |
+| 3 | GitHub | navigate SPA repo links | click() works for SPA navigation (no loadEventFired) |
+| 4 | DuckDuckGo | search query and verify results | type() + press('Enter') + navigation on DDG |
+| 5 | Hacker News | load homepage and navigate to a story | click() + waitForNavigation() on HN story links |
+| 6 | Reddit (old) | load and navigate to a post | Page navigation with fallback to www.reddit.com |
+| 7 | Firefox cookies | extract and inject into CDP session | extractCookies() + injectCookies() workflow |
+| 8 | Firefox cookies | extractCookies with firefox returns array | Explicit browser parameter returns proper array |
+
+---
+
+## Writing new tests
+
+Follow the existing pattern:
+
+```javascript
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { connect } from '../../src/index.js';
+
+describe('my feature', () => {
+  it('does the thing', async () => {
+    const page = await connect();
+    try {
+      await page.goto('https://example.com');
+      const snap = await page.snapshot();
+      assert.ok(snap.includes('Example Domain'));
+    } finally {
+      await page.close();
+    }
+  });
+});
+```
+
+Key conventions:
+- Always `page.close()` in a `finally` block to avoid leaked browser processes
+- Use `data:` URL fixtures for deterministic tests (no network dependency)
+- Real-site tests go in interact.test.js, grouped by site in `describe()` blocks
+- Use `assert.ok()` and `assert.strictEqual()` from `node:assert/strict`
+- No test framework dependencies -- `node:test` only
+
+### Data URL fixture pattern
+
+For testing interactions without network:
+
+```javascript
+const FIXTURE = `data:text/html,${encodeURIComponent(`
+<html><body>
+  <button onclick="document.getElementById('r').textContent='clicked'">Click Me</button>
+  <div id="r"></div>
+</body></html>
+`)}`;
+
+it('clicks the button', async () => {
+  const page = await connect();
+  try {
+    await page.goto(FIXTURE);
+    const snap = await page.snapshot({ mode: 'browse' });
+    const ref = findRef(snap, 'button', 'Click Me');
+    await page.click(ref);
+    const snap2 = await page.snapshot({ mode: 'browse' });
+    assert.ok(snap2.includes('clicked'));
+  } finally {
+    await page.close();
+  }
+});
+```
+
+---
+
+## CI considerations
+
+- Unit tests: fast, no network, always safe to run
+- Integration tests: need Chromium installed, no network (uses example.com/HN but tolerates failures)
+- E2E tests: need internet, may be flaky (sites change, rate limits, geo-blocks)
+- Recommended CI split: run unit + integration always, E2E on manual trigger or nightly
+- Each test launches/kills its own browser instance -- no shared state between tests
+- Auth tests may skip when Chromium profile is locked by a running instance

package/mcp-server.js

@@ -0,0 +1,216 @@
+#!/usr/bin/env node
+/**
+ * mcp-server.js — MCP server for barebrowse.
+ *
+ * Raw JSON-RPC 2.0 over stdio. No SDK dependency.
+ * 7 tools: browse (one-shot), goto, snapshot, click, type, press, scroll.
+ *
+ * Session tools share a singleton page, lazy-created on first use.
+ * Action tools return 'ok' — agent calls snapshot explicitly to observe.
+ */
+
+import { browse, connect } from './src/index.js';
+
+let _page = null;
+
+async function getPage() {
+  if (!_page) _page = await connect();
+  return _page;
+}
+
+const TOOLS = [
+  {
+    name: 'browse',
+    description: 'One-shot: navigate to a URL and return a pruned ARIA snapshot. Stateless — does not use the session page.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        url: { type: 'string', description: 'URL to browse' },
+        mode: { type: 'string', enum: ['headless', 'headed', 'hybrid'], description: 'Browser mode (default: headless)' },
+      },
+      required: ['url'],
+    },
+  },
+  {
+    name: 'goto',
+    description: 'Navigate the session page to a URL. Returns ok — call snapshot to observe.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        url: { type: 'string', description: 'URL to navigate to' },
+      },
+      required: ['url'],
+    },
+  },
+  {
+    name: 'snapshot',
+    description: 'Get the current ARIA snapshot of the session page. Returns a YAML-like tree with [ref=N] markers on interactive elements.',
+    inputSchema: { type: 'object', properties: {} },
+  },
+  {
+    name: 'click',
+    description: 'Click an element by its ref from the snapshot. Returns ok — call snapshot to observe.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        ref: { type: 'string', description: 'Element ref from snapshot (e.g. "8")' },
+      },
+      required: ['ref'],
+    },
+  },
+  {
+    name: 'type',
+    description: 'Type text into an element by its ref. Returns ok — call snapshot to observe.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        ref: { type: 'string', description: 'Element ref from snapshot' },
+        text: { type: 'string', description: 'Text to type' },
+        clear: { type: 'boolean', description: 'Clear existing content first (default: false)' },
+      },
+      required: ['ref', 'text'],
+    },
+  },
+  {
+    name: 'press',
+    description: 'Press a special key (Enter, Tab, Escape, Backspace, Delete, ArrowUp/Down/Left/Right, Home, End, PageUp, PageDown, Space). Returns ok.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        key: { type: 'string', description: 'Key name' },
+      },
+      required: ['key'],
+    },
+  },
+  {
+    name: 'scroll',
+    description: 'Scroll the page. Positive deltaY scrolls down, negative scrolls up. Returns ok.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        deltaY: { type: 'number', description: 'Pixels to scroll (positive=down, negative=up)' },
+      },
+      required: ['deltaY'],
+    },
+  },
+];
+
+async function handleToolCall(name, args) {
+  switch (name) {
+    case 'browse':
+      return await browse(args.url, { mode: args.mode });
+
+    case 'goto': {
+      const page = await getPage();
+      await page.goto(args.url);
+      return 'ok';
+    }
+    case 'snapshot': {
+      const page = await getPage();
+      return await page.snapshot();
+    }
+    case 'click': {
+      const page = await getPage();
+      await page.click(args.ref);
+      return 'ok';
+    }
+    case 'type': {
+      const page = await getPage();
+      await page.type(args.ref, args.text, { clear: args.clear });
+      return 'ok';
+    }
+    case 'press': {
+      const page = await getPage();
+      await page.press(args.key);
+      return 'ok';
+    }
+    case 'scroll': {
+      const page = await getPage();
+      await page.scroll(args.deltaY);
+      return 'ok';
+    }
+    default:
+      throw new Error(`Unknown tool: ${name}`);
+  }
+}
+
+function jsonrpcResponse(id, result) {
+  return JSON.stringify({ jsonrpc: '2.0', id, result });
+}
+
+function jsonrpcError(id, code, message) {
+  return JSON.stringify({ jsonrpc: '2.0', id, error: { code, message } });
+}
+
+async function handleMessage(msg) {
+  const { id, method, params } = msg;
+
+  if (method === 'initialize') {
+    return jsonrpcResponse(id, {
+      protocolVersion: '2024-11-05',
+      capabilities: { tools: {} },
+      serverInfo: { name: 'barebrowse', version: '0.2.1' },
+    });
+  }
+
+  if (method === 'notifications/initialized') {
+    return null; // notification, no response
+  }
+
+  if (method === 'tools/list') {
+    return jsonrpcResponse(id, { tools: TOOLS });
+  }
+
+  if (method === 'tools/call') {
+    const { name, arguments: args } = params;
+    try {
+      const result = await handleToolCall(name, args || {});
+      return jsonrpcResponse(id, {
+        content: [{ type: 'text', text: typeof result === 'string' ? result : JSON.stringify(result) }],
+      });
+    } catch (err) {
+      return jsonrpcResponse(id, {
+        content: [{ type: 'text', text: `Error: ${err.message}` }],
+        isError: true,
+      });
+    }
+  }
+
+  return jsonrpcError(id, -32601, `Method not found: ${method}`);
+}
+
+// --- Stdio transport ---
+
+let buffer = '';
+
+process.stdin.setEncoding('utf8');
+process.stdin.on('data', async (chunk) => {
+  buffer += chunk;
+  let newlineIdx;
+  while ((newlineIdx = buffer.indexOf('\n')) !== -1) {
+    const line = buffer.slice(0, newlineIdx).trim();
+    buffer = buffer.slice(newlineIdx + 1);
+    if (!line) continue;
+
+    try {
+      const msg = JSON.parse(line);
+      const response = await handleMessage(msg);
+      if (response) {
+        process.stdout.write(response + '\n');
+      }
+    } catch (err) {
+      process.stdout.write(jsonrpcError(null, -32700, `Parse error: ${err.message}`) + '\n');
+    }
+  }
+});
+
+// Clean up on exit
+process.on('SIGINT', async () => {
+  if (_page) await _page.close().catch(() => {});
+  process.exit(0);
+});
+
+process.on('SIGTERM', async () => {
+  if (_page) await _page.close().catch(() => {});
+  process.exit(0);
+});

package/package.json

@@ -1,19 +1,32 @@
 {
   "name": "barebrowse",
-  "version": "0.1.0",
-  "description": "Authenticated web browsing for autonomous agents via CDP",
+  "version": "0.2.1",
+  "description": "Authenticated web browsing for autonomous agents via CDP. URL in, pruned ARIA snapshot out.",
   "type": "module",
   "main": "src/index.js",
-  "exports": "./src/index.js",
+  "exports": {
+    ".": "./src/index.js",
+    "./bareagent": "./src/bareagent.js"
+  },
+  "bin": {
+    "barebrowse": "./cli.js"
+  },
   "engines": {
     "node": ">=22"
   },
   "scripts": {
-    "test": "node --test test/**/*.test.js"
+    "test": "node --test test/unit/*.test.js test/integration/*.test.js"
   },
-  "license": "MIT",
-  "files": [
-    "README.md",
-    "LICENSE"
-  ]
+  "keywords": [
+    "browser",
+    "cdp",
+    "chromium",
+    "aria",
+    "accessibility",
+    "web-scraping",
+    "agent",
+    "mcp",
+    "headless"
+  ],
+  "license": "MIT"
 }

package/src/aria.js

@@ -0,0 +1,69 @@
+/**
+ * aria.js — Format ARIA accessibility tree nodes for agent consumption.
+ *
+ * Takes a nested tree (built from CDP's Accessibility.getFullAXTree)
+ * and formats it as readable YAML-like text, similar to Playwright's ariaSnapshot.
+ */
+
+/**
+ * Format a nested ARIA tree as readable text output.
+ *
+ * Output format (one node per line, indented):
+ *   - role "name" [props] [ref=nodeId]
+ *
+ * @param {object} node - Tree node { role, name, properties, children, ignored, nodeId }
+ * @param {number} [depth=0] - Current indentation depth
+ * @returns {string} Formatted ARIA tree text
+ */
+export function formatTree(node, depth = 0) {
+  if (!node) return '';
+
+  // Skip ignored nodes but still process their children
+  if (node.ignored) {
+    return node.children.map((c) => formatTree(c, depth)).join('');
+  }
+
+  // Skip low-level rendering nodes that are noise for agents
+  const SKIP_ROLES = new Set(['InlineTextBox', 'LineBreak']);
+  if (SKIP_ROLES.has(node.role)) return '';
+
+  const indent = '  '.repeat(depth);
+  const lines = [];
+
+  // Build line: "- role "name" [properties] [ref=id]"
+  let line = `${indent}- ${node.role || 'none'}`;
+
+  if (node.name) {
+    line += ` "${node.name}"`;
+  }
+
+  // Notable properties that agents care about
+  const props = node.properties || {};
+  const propParts = [];
+  if (props.checked !== undefined) propParts.push(`checked=${props.checked}`);
+  if (props.disabled) propParts.push('disabled');
+  if (props.expanded !== undefined) propParts.push(`expanded=${props.expanded}`);
+  if (props.level) propParts.push(`level=${props.level}`);
+  if (props.selected) propParts.push('selected');
+  if (props.required) propParts.push('required');
+  if (props.value !== undefined && props.value !== '') propParts.push(`value="${props.value}"`);
+
+  if (propParts.length > 0) {
+    line += ` [${propParts.join(', ')}]`;
+  }
+
+  // Node ID as ref — agents use this to target interactions
+  if (node.nodeId) {
+    line += ` [ref=${node.nodeId}]`;
+  }
+
+  lines.push(line);
+
+  // Recurse into children
+  for (const child of node.children) {
+    const childText = formatTree(child, depth + 1);
+    if (childText) lines.push(childText);
+  }
+
+  return lines.join('\n');
+}