barebrowse 0.2.2 → 0.3.1

package/docs/01-product/prd.md

@@ -0,0 +1,284 @@
+# barebrowse — Product Requirements Document
+
+**Version:** 1.0
+**Date:** 2026-02-22
+**Status:** POC
+
+---
+
+## What barebrowse is
+
+A standalone vanilla JavaScript library that gives autonomous agents authenticated access to the web through the user's own Chromium browser. One package, one import, three modes.
+
+```js
+import { browse } from 'barebrowse';
+const snapshot = await browse('https://any-page.com');
+```
+
+barebrowse handles: finding the browser, connecting via CDP, injecting cookies, navigating, extracting the ARIA accessibility tree, and pruning it down to what an agent actually needs. The output is a clean, token-efficient snapshot of any web page — authenticated as the real user.
+
+## What barebrowse is NOT
+
+- **Not a framework.** No plugin system, no config files, no lifecycle hooks.
+- **Not an MCP server.** But trivially wrappable as one (~30 lines).
+- **Not Playwright.** No bundled browser, no cross-engine abstraction, no 200MB download.
+- **Not an agent.** No LLM, no planning, no orchestration — that's bareagent's job.
+- **Not a scraper.** It browses as the user, not as a bot harvesting data.
+
+---
+
+## The Problem
+
+Every AI agent that needs to read or interact with the web hits the same walls:
+
+1. **Cloudflare / bot detection** — headless browsers get blocked
+2. **Authentication** — sites require login, OAuth, session cookies
+3. **Token bloat** — raw DOM is 100K+ tokens; agents need ~5K
+4. **Two consumers, same need** — research agents (read pages) and personal assistants (click/type) both need an authenticated browser, but existing tools force you to choose one path
+
+Existing solutions (Playwright MCP, sweetlink, open-operator, browser-use) are either too heavy, too opinionated, or solve only half the problem.
+
+## The Insight
+
+The user already has a browser. It's already logged in. It already passes Cloudflare. Instead of fighting the web with headless stealth tricks, **use what's already there**.
+
+CDP (Chrome DevTools Protocol) lets us connect to any Chromium-based browser — the same one the user browses with daily. We get their cookies, their sessions, their anti-detection posture, for free.
+
+---
+
+## Core Architecture
+
+### CDP-Direct (Why No Playwright)
+
+**Decision:** Use CDP over WebSocket directly. No Playwright dependency.
+
+**Why:**
+- Playwright downloads a bundled Chromium (~200MB). barebrowse uses the browser already installed on the user's machine.
+- Playwright abstracts CDP, but we need CDP directly for all three modes (headless, headed, hybrid) against the user's real browser.
+- Every Playwright API call maps 1:1 to a CDP method. The abstraction adds weight without adding capability for our use case.
+- CDP gives us everything: `Accessibility.getFullAXTree`, `Page.navigate`, `Runtime.evaluate`, `Input.dispatch*Event`, `Network.setCookie`, `Page.captureScreenshot`.
+- The CDP WebSocket client is ~100 lines of vanilla JS. Playwright is ~50,000.
+
+**What we lose:** Cross-engine support (Firefox, WebKit). CDP only works with Chromium-family browsers (Chrome, Chromium, Edge, Brave, Vivaldi, Arc, Opera). This covers ~80% of desktop browsers. Firefox support could come later via WebDriver BiDi.
+
+**What we gain:** Zero heavy deps, uses the user's real browser, same code path for headless/headed/hybrid, drastically simpler codebase.
+
+### ARIA-First (Why Not DOM)
+
+**Decision:** Use `Accessibility.getFullAXTree` (ARIA/accessibility tree) as the primary page representation, not DOM.
+
+**Why:**
+- The accessibility tree is the semantic structure of the page — roles, names, states, interactive elements. It's what screen readers see. It's also what agents need.
+- DOM is bloated: wrapper divs, styling, tracking pixels, ad scripts. An agent doesn't need any of that.
+- mcprune already proved this: ARIA snapshots pruned by role achieve 75-95% token reduction on typical pages while preserving all actionable information.
+- CDP's `Accessibility.getFullAXTree` returns the tree directly. No parsing HTML, no building a DOM tree, no traversing nodes.
+- ARIA refs map directly to CDP interaction targets — the agent reads a button in the tree and can click it via the same CDP connection.
+
+**The pipeline:** CDP connect → authenticate → navigate → ARIA tree → prune → agent gets clean snapshot.
+
+### Three Modes (Why All Three)
+
+**Decision:** Headless, headed, and hybrid — not as separate packages or optional features, but as a single flag on the same API.
+
+**Why they're not bloat:** The CDP conversation is identical regardless of mode. The only difference is how you get a browser process with a debug port. It's one code path with a different entry point:
+
+```
+headless: spawn chromium --headless=new --remote-debugging-port=N
+headed:   connect to user's already-running browser on debug port
+hybrid:   try headless → detect failure → fall back to headed
+```
+
+After connection, every CDP command is the same. Three modes = ~20 extra lines in `chromium.js`, not three implementations.
+
+**When to use each:**
+
+| Mode | Use case | Example |
+|---|---|---|
+| `headless` | Agent research, background tasks, CI | "Read this article and summarize it" |
+| `headed` | Personal assistant, interactive tasks, auth flows | "Book me a flight on this page" |
+| `hybrid` | Default for autonomous agents | Try headless; if CF-blocked, fall back to headed |
+
+**Headless is the default.** Most agent tasks are "go read this page." Headed is the escape hatch for when headless fails or the task requires user-visible interaction.
+
+### Cookie Authentication
+
+**Decision:** Extract cookies from the user's browser profile and inject via CDP `Network.setCookie`.
+
+**Why:**
+- The user's browser has active sessions for every site they use. We reuse those sessions instead of building new auth flows.
+- sweet-cookie (npm package) already extracts cookies from Chrome/Firefox/Safari SQLite databases with OS keychain decryption. We use it or vendor the relevant parts.
+- For headed mode, cookies are already present in the browser — no extraction needed.
+- For headless mode, we extract from the user's profile and inject into the headless instance.
+
+**Limitation:** Cookies expire. This works for existing sessions, not new logins. For sites requiring fresh auth, headed mode with user interaction is the fallback.
+
+### Pruning (Absorbed from mcprune)
+
+**Decision:** Port mcprune's role-based ARIA tree pruning into barebrowse as a built-in step, not an optional module.
+
+**Why:**
+- Pruning is not optional for agent consumption. A raw ARIA tree is still too large for most LLM context windows. Pruning is part of the pipeline, not an afterthought.
+- mcprune's pruning logic is a pure function: takes an ARIA tree, returns a smaller ARIA tree. No browser dependency, no Playwright coupling. It's ~300 lines of role-based tree surgery.
+- By absorbing it, barebrowse becomes a complete "URL in, agent-ready snapshot out" solution. No second package needed.
+
+**What we port from mcprune:**
+- Role taxonomy (landmarks, interactive, structural, noise)
+- Landmark extraction (main, nav, banner, etc.)
+- Noise removal (ads, tracking, legal boilerplate)
+- Interactive element preservation (buttons, links, inputs)
+- Wrapper collapsing (nested generics, empty groups)
+- Context-aware filtering (search relevance, dedup)
+
+**What stays in mcprune:** The Playwright MCP proxy architecture. mcprune can continue to exist as a Playwright-based MCP server for users who want that path. But for barebrowse consumers, pruning is built in.
+
+---
+
+## API Design
+
+### Public API
+
+```js
+import { browse, connect } from 'barebrowse';
+
+// One-shot: URL in, pruned ARIA snapshot out
+const tree = await browse('https://example.com');
+
+// With options
+const tree = await browse('https://example.com', {
+  mode: 'hybrid',        // 'headless' (default) | 'headed' | 'hybrid'
+  cookies: true,          // inject user's cookies (default: true)
+  prune: true,            // apply ARIA pruning (default: true)
+  browser: 'chrome',      // which browser profile for cookies
+  timeout: 30000,         // navigation timeout ms
+});
+
+// Long-lived session for interaction
+const page = await connect({ mode: 'headed' });
+await page.goto('https://amazon.com/cart');
+await page.click('[data-action="checkout"]');
+await page.type('#gift-message', 'Happy birthday!');
+const tree = await page.snapshot();  // ARIA + prune
+await page.close();
+```
+
+### Design Principles
+
+1. **One package, one import.** No picking pieces. `browse()` does everything. Power users get `connect()` for long-lived sessions.
+2. **Batteries included.** Cookies, ARIA, pruning — all happen inside by default. Disable with flags if you want raw access.
+3. **Escape hatches.** `connect()` returns an object with the raw CDP connection accessible. If you need something we don't wrap, you can send CDP commands directly.
+4. **Progressive complexity.** `browse(url)` for 90% of use cases. Options object for the rest. `connect()` for interactive sessions.
+
+---
+
+## The bare- Ecosystem
+
+```
+bareagent   = the brain  (orchestration, planning, memory, retries, tool loop)
+barebrowse  = the eyes + hands  (browse, read, interact with the web)
+```
+
+**Integration with bareagent:**
+
+```js
+import { Loop } from 'bare-agent';
+import { browse } from 'barebrowse';
+
+const tools = [
+  { name: 'browse', execute: ({ url }) => browse(url) },
+];
+
+const loop = new Loop({ provider });
+await loop.run([{ role: 'user', content: 'Find the cheapest flight to Tokyo' }], tools);
+```
+
+bareagent handles the think/act/observe loop. barebrowse handles "see the web and act on it." Neither is opinionated about the other. Tools are plain functions.
+
+**Integration with multis:**
+
+multis (personal assistant) uses barebrowse in headed mode for interactive tasks. The multis proxy is already running, providing a desktop session. barebrowse connects to the user's Chrome and drives it on behalf of the assistant.
+
+**MCP server wrapper (future):**
+
+barebrowse is not an MCP server, but wrapping it as one is ~30 lines. This would replace Playwright MCP + mcprune proxy with a single, lighter MCP server.
+
+---
+
+## Decisions Log — Why We Chose Each
+
+This section exists so we don't re-debate settled decisions.
+
+| Decision | Choice | Why | Alternative considered | Why not |
+|---|---|---|---|---|
+| Browser protocol | CDP direct | Uses user's browser, ~100 lines, all 3 modes | Playwright | 200MB download, bundles its own Chromium, abstracts what we need raw |
+| Page representation | ARIA tree | Semantic, token-efficient, what agents need | DOM/HTML | Bloated, noisy, needs heavy parsing |
+| Pruning | Built-in | Agents always need pruned output | Optional/separate | Two deps for one job, pruning isn't optional |
+| Cookie auth | Own auth.js + CDP inject | User's existing sessions (Firefox or Chromium), cross-browser injection into headless Chromium | OAuth/credential storage | Complex, security liability, reinventing what the browser already solved |
+| Three modes | One flag | Same CDP code, ~20 lines difference | Separate packages | Same code, artificial separation |
+| Chromium only | CDP constraint | ~80% browser share, user's real browser | Cross-browser (Playwright) | Requires Playwright, loses "use your own browser" benefit |
+| Anti-detection | Runtime.evaluate patches | Minimal stealth for headless mode | Full stealth framework | Over-engineering; headless + real cookies handles 90% |
+| Daemon/server | None | CDP is direct, no intermediary needed | sweetlink daemon pattern | Unnecessary complexity for local agent→browser |
+| Framework | None (vanilla JS) | Matches bare- philosophy, zero deps | Express/Fastify wrapper | Not a server, not needed |
+| Language | Vanilla JavaScript | Node.js ecosystem, same as bareagent, CDP libs available | TypeScript | Added build step, not needed for POC; can add types later |
+| Naming | chromium.js | Covers all Chromium-family browsers, not just Chrome | chrome.js | Too specific; Brave/Edge/Arc are also targets |
+| mcprune integration | Absorb pruning logic | One package does it all, mcprune pruning is a pure function | Keep separate | Agents shouldn't need two packages to browse |
+| openclaw lesson | Single bridge protocol | One CDP connection vs many API integrations | Direct multi-API | openclaw proved this fails — bloat, maintenance, fragility |
+
+---
+
+## Future Features (Post-POC)
+
+### Near-term
+- **Screenshot capture** — `Page.captureScreenshot` via CDP. Useful for visual verification and multimodal agents.
+- **Network interception** — `Network.requestWillBeSent` / `Network.responseReceived` for monitoring page loads. Detect redirects, blocked resources, API calls.
+- **Wait strategies** — `waitForNavigation()` done (Page.loadEventFired). Still needed: network idle, element presence polling.
+- **Tab management** — Multiple pages in one browser session. CDP `Target.createTarget` / `Target.attachToTarget`.
+- **MCP server wrapper** — Expose browse/click/type as MCP tools. Replaces Playwright MCP + mcprune combo.
+
+### Medium-term
+- **Firefox support** — Via WebDriver BiDi protocol (cross-browser standard, still maturing). Second protocol adapter alongside CDP.
+- **Cookie sync** — In hybrid mode, extract fresh cookies from headed session and cache for future headless use. Self-refreshing auth.
+- **Selector discovery** — Port sweetlink's `discoverSelectors` — crawl ARIA tree, score interactive elements, return ranked action targets.
+- **Form understanding** — Detect forms in ARIA tree, map fields to semantic purposes, enable agents to fill forms intelligently.
+- **Proxy/Tor support** — Route headless browser through proxy for geo-restricted content.
+
+### Long-term
+- **Profile management** — Multiple browser profiles for different identities/accounts.
+- **Session recording/replay** — Record browsing sessions as CDP commands, replay for testing.
+- **Visual grounding** — Combine ARIA tree with screenshot regions for multimodal agents.
+- **Agent memory integration** — Remember visited pages, cache snapshots, track which sites need headed mode.
+
+---
+
+## Repos Studied — What We Borrowed and Why
+
+| Repo | What we took | What we skipped |
+|---|---|---|
+| **steipete/sweet-cookie** | Cookie extraction from browser profiles, OS keychain decryption | Nothing — clean, focused library |
+| **steipete/sweetlink** | CDP dual-channel concept, selector discovery scoring, click/command patterns | Daemon architecture, WebSocket bridge, in-page runtime injection, HMAC auth |
+| **steipete/canvas** | Stealth/anti-detection config patterns | Go implementation (we're JS) |
+| **nichochar/open-operator** | AI agent web automation patterns | Full framework, too opinionated |
+| **AntlerClaw/playwright-mcp** | How to expose browser as MCP tools | Playwright dependency |
+| **AntlerClaw/mcp-browser-use** | MCP-native browser patterns | Heavy deps |
+| **AitchKay/chromancer** | Accessibility tree extraction approach | Different stack |
+| **mcprune (own)** | ARIA pruning logic — role taxonomy, landmark extraction, noise removal, wrapper collapsing | Playwright dependency, MCP proxy architecture |
+| **openclaw (own)** | Lesson learned: multi-API direct integration = bloat. Use a single bridge protocol | Everything — the architecture was the cautionary tale |
+
+### The openclaw lesson
+
+openclaw tried to integrate 10+ messaging APIs directly — each with its own auth, format, quirks. It became a maintenance nightmare. multis solved the same problem by using Beeper/Matrix as a single bridge.
+
+barebrowse applies the same lesson: instead of integrating Playwright + Puppeteer + WebDriver + stealth plugins + cookie libraries + proxy managers, we use **one protocol (CDP) to one browser (the user's)**. Everything else is unnecessary.
+
+---
+
+## Success Criteria
+
+barebrowse succeeds when:
+
+1. `browse(url)` returns a pruned ARIA snapshot of any page, authenticated as the user
+2. Zero heavy dependencies — no Playwright, no Puppeteer, no bundled browser
+3. Works with any installed Chromium-based browser
+4. Headless for research, headed for interaction, hybrid for autonomous agents
+5. Plugs into bareagent as plain tool functions
+6. Total source under 1,000 lines for core functionality
+7. An agent using barebrowse + bareagent can autonomously research the web and act on pages

package/docs/03-logs/bug-log.md

@@ -0,0 +1,16 @@
+# Bug Log
+
+Track bugs: symptom, root cause, fix, regression test.
+
+---
+
+*No bugs logged yet. When one is found, add an entry:*
+
+```
+## [date] Short description
+
+**Symptom:** What the user/test observed
+**Root cause:** Why it happened
+**Fix:** What was changed (file:line)
+**Regression test:** Which test prevents recurrence
+```

package/docs/03-logs/decisions-log.md

@@ -0,0 +1,32 @@
+# Decisions Log
+
+Settled decisions. Don't re-debate these -- see rationale column.
+
+## Founding decisions (v0.1.0)
+
+| # | Decision | Choice | Why | Alternative | Why not |
+|---|----------|--------|-----|-------------|---------|
+| 1 | Browser protocol | CDP direct | Uses user's browser, ~100 lines, all 3 modes | Playwright | 200MB download, bundles its own Chromium, abstracts what we need raw |
+| 2 | Page representation | ARIA tree | Semantic, token-efficient, what agents need | DOM/HTML | Bloated, noisy, needs heavy parsing |
+| 3 | Pruning | Built-in | Agents always need pruned output | Optional/separate | Two deps for one job, pruning isn't optional |
+| 4 | Cookie auth | Own auth.js + CDP inject | User's existing sessions (Firefox or Chromium), cross-browser injection | OAuth/credential storage | Complex, security liability, reinventing what the browser already solved |
+| 5 | Three modes | One flag | Same CDP code, ~20 lines difference | Separate packages | Same code, artificial separation |
+| 6 | Chromium only | CDP constraint | ~80% browser share, user's real browser | Cross-browser (Playwright) | Requires Playwright, loses "use your own browser" benefit |
+| 7 | Framework | None (vanilla JS) | Matches bare- philosophy, zero deps | Express/Fastify wrapper | Not a server, not needed |
+| 8 | Language | Vanilla JavaScript | Node.js ecosystem, same as bareagent, CDP libs available | TypeScript | Added build step, not needed; can add types later |
+| 9 | mcprune integration | Absorb pruning logic | One package does it all, mcprune pruning is a pure function | Keep separate | Agents shouldn't need two packages to browse |
+| 10 | Daemon/server | None | CDP is direct, no intermediary needed | sweetlink daemon pattern | Unnecessary complexity for local agent-to-browser |
+
+## v0.2.0 decisions
+
+| # | Decision | Choice | Why | Alternative | Why not |
+|---|----------|--------|-----|-------------|---------|
+| 11 | Anti-detection | Runtime.evaluate patches | Minimal stealth for headless mode | Full stealth framework | Over-engineering; headless + real cookies handles 90% |
+| 12 | sweet-cookie | Wrote own auth.js | sweet-cookie not on npm (different package). Our version is simpler, tailored, vanilla JS | Use sweet-cookie | Not available as npm package |
+| 13 | MCP server | Raw JSON-RPC, no SDK | Zero deps, ~200 lines. SDK adds weight without capability for stdio | @modelcontextprotocol/sdk | Unnecessary dependency for simple JSON-RPC |
+| 14 | bareagent adapter | Action tools auto-return snapshot | LLM always sees result without extra tool call. 300ms settle for DOM updates | Return 'ok' like MCP | Different tradeoff -- bareagent tool calls are expensive (LLM round-trip) |
+| 15 | MCP action tools | Return 'ok', agent calls snapshot | MCP tool calls are cheap to chain. Avoids double-token output | Auto-return snapshot | Would bloat every action response |
+
+---
+
+*Add new decisions below this line. Include date, context, and rationale.*

package/docs/03-logs/implementation-log.md

@@ -0,0 +1,54 @@
+# Implementation Log
+
+Chronological record of what changed and why. For detailed changelogs, see `/CHANGELOG.md`.
+
+---
+
+## v0.2.1 (2026-02-22)
+
+- README rewritten: no code blocks, obstacle table, two usage paths (MCP vs framework)
+- MCP auto-installer: `npx barebrowse install` detects Claude Desktop, Cursor, Claude Code
+- MCP config uses `npx` instead of local file paths
+
+## v0.2.0 (2026-02-22)
+
+Major release: agent integration layer.
+
+**New modules:**
+- `mcp-server.js` -- JSON-RPC 2.0 over stdio, 7 tools, singleton session
+- `src/bareagent.js` -- tool adapter for bareagent Loop, 9 tools, auto-snapshot
+- `src/stealth.js` -- navigator patches for headless anti-detection
+- `cli.js` -- `npx barebrowse mcp|install|browse`
+
+**New features:**
+- Hybrid mode (try headless, fallback to headed on bot detection)
+- `page.hover(ref)`, `page.select(ref, value)`, `page.screenshot(opts)`
+- `page.waitForNetworkIdle(opts)` -- resolve when no pending requests
+- SPA-aware `waitForNavigation()`
+
+**Docs:**
+- `barebrowse.context.md` -- LLM integration guide
+- `docs/testing.md` -- test pyramid, all 54 tests
+- `docs/blueprint.md` -- full pipeline, module table
+
+**Tests:** 54 passing (was 47)
+
+## v0.1.0 (2026-02-22)
+
+Initial release. CDP-direct browsing with ARIA snapshots.
+
+**Core modules (7):**
+- `src/index.js` -- `browse()`, `connect()` API
+- `src/cdp.js` -- WebSocket CDP client
+- `src/chromium.js` -- browser discovery and launch
+- `src/aria.js` -- ARIA tree formatting
+- `src/auth.js` -- cookie extraction (Firefox SQLite, Chromium AES + keyring)
+- `src/prune.js` -- 9-step pruning pipeline (ported from mcprune)
+- `src/interact.js` -- click, type, press, scroll
+- `src/consent.js` -- cookie consent auto-dismiss (7 languages, 16+ sites)
+
+**Tests:** 47 passing across 5 files
+
+---
+
+*Add new entries at the top. Include version, date, and what changed.*

package/docs/03-logs/insights.md

@@ -0,0 +1,35 @@
+# Insights
+
+Lessons learned, patterns discovered, things to remember.
+
+---
+
+## The openclaw lesson
+
+openclaw tried to integrate 10+ messaging APIs directly -- each with its own auth, format, quirks. It became a maintenance nightmare. multis solved the same problem by using Beeper/Matrix as a single bridge.
+
+barebrowse applies the same lesson: instead of integrating Playwright + Puppeteer + WebDriver + stealth plugins + cookie libraries + proxy managers, we use **one protocol (CDP) to one browser (the user's)**. Everything else is unnecessary.
+
+**Takeaway:** When possible, find a single bridge protocol instead of N direct integrations.
+
+## Repos studied -- what we took and what we skipped
+
+| Repo | What we took | What we skipped | Why |
+|------|-------------|-----------------|-----|
+| **steipete/sweet-cookie** | Cookie extraction concept (SQLite + keyring) | Nothing | Not on npm. Wrote our own auth.js -- simpler, tailored, vanilla JS |
+| **steipete/sweetlink** | CDP-direct concept | Daemon, WebSocket bridge, in-page runtime, HMAC auth | CDP direct is 100 lines vs ~2,000 |
+| **steipete/canvas** | Stealth/anti-detection patterns | Go implementation | Noted for stealth.js |
+| **mcprune (own)** | Full pruning pipeline port | Playwright dependency, MCP proxy | prune.js is 472 lines, adapted from Playwright YAML to CDP tree |
+| **openclaw (own)** | Cautionary tale | Everything | Multi-API direct integration = bloat |
+
+## Key technical insights
+
+- **ARIA tree > DOM** for agent consumption. Semantic, compact, interactive elements are first-class. Token reduction of 47-95% is real.
+- **Cookie consent is solvable** with ARIA tree scanning + a button text corpus in 7 languages. Dialog role detection + global fallback covers >95% of sites.
+- **Headed mode is the ultimate fallback.** When stealth fails, when cookies expire, when CAPTCHAs appear -- connecting to the user's real browser session handles it.
+- **CDP flattened sessions** are the way to go. One WebSocket, multiple targets. The session ID header routes commands to the right tab.
+- **`Page.addScriptToEvaluateOnNewDocument`** runs before any page scripts -- perfect for stealth patches without race conditions.
+
+---
+
+*Add new insights as they emerge. These should be durable lessons, not session notes.*

package/docs/03-logs/validation-log.md

@@ -0,0 +1,123 @@
+# Validation Log
+
+What's been tested against the real world. Updated when new sites or features are validated.
+
+---
+
+## Test suite (64 tests, 6 files)
+
+| File | Tests | Type | What it covers |
+|------|-------|------|----------------|
+| `test/unit/prune.test.js` | 16 | Unit | 9-step pruning pipeline in isolation |
+| `test/unit/auth.test.js` | 7 | Unit | Cookie extraction from Firefox/Chromium |
+| `test/unit/cdp.test.js` | 5 | Unit | Browser discovery, launch, CDP client, sessions |
+| `test/integration/browse.test.js` | 11 | Integration | Full `browse()` and `connect()` pipeline |
+| `test/integration/cli.test.js` | 10 | Integration | CLI session lifecycle: open/snapshot/goto/click/eval/console/network/close |
+| `test/integration/interact.test.js` | 15 | E2E | Real interactions on data: fixtures + live sites |
+
+Run all: `node --test test/unit/*.test.js test/integration/*.test.js`
+
+## Site validation matrix
+
+Tested across 16+ sites, 8 countries, 7 languages.
+
+| Site | Consent | Cookies | Interactions | Notes |
+|------|---------|---------|-------------|-------|
+| google.com | NL dialog dismissed | Firefox injection | Search (combobox + Enter) | Bot-blocks headless |
+| youtube.com | Bypassed via cookies | Firefox injection | Search + video playback | Full e2e demo, SPA nav |
+| bbc.com | SourcePoint dismissed | -- | -- | Button outside dialog |
+| wikipedia.org | -- | -- | Link click + navigation | Clean, no consent |
+| github.com | -- | -- | SPA navigation | Needs settle time |
+| duckduckgo.com | -- | -- | Search + results | Headless-friendly |
+| news.ycombinator.com | -- | -- | Story link click | Clean, simple DOM |
+| amazon.de | Banner dismissed | -- | -- | |
+| theguardian.com | CMP dismissed | -- | -- | |
+| spiegel.de | CMP dismissed | -- | -- | German |
+| lemonde.fr | CMP dismissed | -- | -- | French |
+| elpais.com | CMP dismissed | -- | -- | Spanish |
+| corriere.it | CMP dismissed | -- | -- | Italian |
+| nos.nl | CMP dismissed | -- | -- | Dutch |
+| bild.de | CMP dismissed | -- | -- | German |
+| nu.nl | CMP dismissed | -- | -- | Dutch |
+| booking.com | Banner dismissed | -- | -- | |
+| nytimes.com | -- | -- | -- | No consent wall |
+| stackoverflow.com | Footer link only | -- | -- | Not blocking |
+| cnn.com | -- | -- | -- | No consent wall |
+| reddit.com | -- | -- | Fallback to old.reddit | Bot-blocks headless |
+
+## Token reduction measurements
+
+| Page | Raw ARIA | Pruned | Reduction |
+|------|----------|--------|-----------|
+| example.com | 377 chars | 45 chars | 88% |
+| Hacker News | 51,726 chars | 27,197 chars | 47% |
+| Wikipedia (article) | 109,479 chars | 40,566 chars | 63% |
+| DuckDuckGo | 42,254 chars | 5,407 chars | 87% |
+
+---
+
+## CLI manual validation (v0.3.0)
+
+Full end-to-end validation of every CLI command against real websites.
+
+### Session lifecycle
+
+| Command | Result |
+|---------|--------|
+| `barebrowse open https://example.com` | Session started, pid+port printed, session.json created |
+| `barebrowse status` | Shows running pid, port, start time |
+| `barebrowse close` | "Session closed", session.json removed, daemon exited |
+| `status` after close | "No session found", exit code 1 |
+| `click 5` with no session | "No active session. Run `barebrowse open` first.", exit 1 |
+| double `open` | "Session already running. Use `barebrowse close` first.", exit 1 |
+
+### Navigation + snapshots (example.com, HN)
+
+| Command | Result |
+|---------|--------|
+| `snapshot` (example.com) | `.barebrowse/page-*.yml` created, clean formatting |
+| `snapshot --mode=read` | Read mode includes paragraphs, each node on own line |
+| `goto https://news.ycombinator.com` | "ok" |
+| `snapshot` (HN) | Clean ARIA tree with refs, proper newline separation |
+| `screenshot` | Valid 780x493 PNG file |
+
+### Interactions (DuckDuckGo search)
+
+| Command | Result |
+|---------|--------|
+| `type 12 barebrowse npm` | "ok", multi-word text correctly joined |
+| `press Enter` | "ok", search submitted |
+| `wait-idle` | "ok", waited for network settle |
+| `eval "document.title"` | `"barebrowse npm at DuckDuckGo"` |
+| `snapshot` | Search results page, clean formatting with refs |
+| `fill 2583 hello world` | "ok", cleared search box + typed new text |
+| `hover 2402` | "ok" |
+| `scroll 300` | "ok" |
+
+### Debugging commands
+
+| Command | Result |
+|---------|--------|
+| `eval "1 + 1"` | `2` |
+| `eval "document.location.href"` | `"https://news.ycombinator.com/news"` |
+| `eval "console.log('test'); console.error('err')"` | `ok` (undefined return) |
+| `console-logs` | `.json (2 entries)` — log + error captured with types and timestamps |
+| `network-log` | `.json (15 entries)` — all requests with URL, method, status |
+| `network-log --failed` | `.json (1 entries)` — filtered to failed/4xx+ only |
+
+### Legacy + install commands
+
+| Command | Result |
+|---------|--------|
+| `browse https://example.com` | One-shot snapshot to stdout |
+| `install` | "No MCP clients detected" + Claude Code hint |
+| `install --skill` | SKILL.md copied to `~/.config/claude/skills/barebrowse/` |
+| (no args) | Clean help output with all commands |
+
+### Bug found and fixed during validation
+
+**`src/aria.js` line 23**: ignored nodes joined children with `''` instead of `'\n'`, causing sibling subtrees to concatenate on one line (e.g. `[ref=15]- _promote`). Fixed to `.filter(Boolean).join('\n')`. All 64 tests pass with the fix.
+
+---
+
+*Add new validation entries when testing against new sites or features.*

package/docs/04-process/definition-of-done.md

@@ -0,0 +1,31 @@
+# Definition of Done
+
+A feature or change is "done" when ALL of these are true.
+
+## Code
+
+- [ ] Works end-to-end (not just the happy path)
+- [ ] No heavy dependencies added (vanilla -> stdlib -> external hierarchy respected)
+- [ ] Under reasonable line count -- no bloat
+- [ ] Clean process management -- no orphan browser processes
+- [ ] No security vulnerabilities introduced (command injection, XSS, etc.)
+
+## Tests
+
+- [ ] Existing tests still pass: `node --test test/unit/*.test.js test/integration/*.test.js`
+- [ ] New behavior has test coverage (integration preferred over unit)
+- [ ] Bug fixes include a regression test that fails before the fix
+
+## Documentation
+
+- [ ] `docs/00-context/system-state.md` updated if architecture changed
+- [ ] `docs/03-logs/decisions-log.md` updated if a design decision was made
+- [ ] `barebrowse.context.md` updated if public API changed
+- [ ] `CHANGELOG.md` updated with what changed
+
+## Not required (avoid over-engineering)
+
+- 100% code coverage
+- TypeScript types
+- Cross-platform testing (Linux first, others later)
+- Performance benchmarks (unless performance is the feature)

package/docs/04-process/dev-workflow.md

@@ -0,0 +1,68 @@
+# Development Workflow
+
+## Dev rules
+
+**POC first.** Always validate logic with a ~15min proof-of-concept before building. Cover happy path + common edges. POC works -> design properly -> build with tests. Never ship the POC.
+
+**Build incrementally.** Break work into small independent modules. One piece at a time, each must work on its own before integrating.
+
+**Dependency hierarchy -- follow strictly:**
+1. Vanilla language -- write it yourself if <50 lines and not security-critical
+2. Standard library -- `node:test`, `node:fs`, `node:crypto`, `node:sqlite`
+3. External -- only when stdlib can't do it in <100 lines. Must be maintained, lightweight, widely adopted
+
+**Exception:** Always use vetted libraries for security-critical code (crypto, auth, sanitization).
+
+**Lightweight over complex.** Fewer moving parts, fewer deps, less config. Simple > clever. Readable > elegant.
+
+**Open-source only.** No vendor lock-in. Every line of code must have a purpose -- no speculative code, no premature abstractions.
+
+## Language and runtime
+
+- Vanilla JavaScript, ES modules, no build step
+- Node.js >= 22 (built-in WebSocket, built-in SQLite)
+- No TypeScript -- can add types later if needed
+
+## Running tests
+
+```bash
+# All 54 tests
+node --test test/unit/*.test.js test/integration/*.test.js
+
+# Unit only (fast, no network)
+node --test test/unit/prune.test.js
+node --test test/unit/auth.test.js
+node --test test/unit/cdp.test.js
+
+# Integration (needs Chromium + network)
+node --test test/integration/browse.test.js
+node --test test/integration/interact.test.js
+
+# Quick smoke test
+node -e "import { browse } from './src/index.js'; console.log(await browse('https://example.com'))"
+```
+
+## Testing standards
+
+- **Test behavior, not implementation.** Call the public API, assert on observable output.
+- **Integration tests are the sweet spot.** Real components working together.
+- **No test framework deps.** `node:test` and `node:assert/strict` only.
+- **Always `page.close()` in a `finally` block** to avoid leaked browser processes.
+- **Use `data:` URL fixtures** for deterministic tests (no network dependency).
+- **Real-site tests** go in `interact.test.js`, grouped by site.
+
+See `docs/04-process/testing.md` for the full test guide.
+
+## Git workflow
+
+- Main branch: `main`
+- Commit messages: conventional (`fix:`, `feat:`, `chore:`, `docs:`, `release:`)
+- No force pushes to main
+
+## Environment
+
+- OS: Fedora Linux, KDE Plasma, Wayland
+- Node: 22.22.0
+- Browser: `/usr/bin/chromium-browser`
+- Default browser: Firefox (cookies extracted from `~/.mozilla/firefox/*.default-release/cookies.sqlite`)
+- KWallet has Chromium Safe Storage key