barebrowse 0.2.1 → 0.3.0

package/.claude/memory/AGENT_RULES.md

@@ -0,0 +1,251 @@
+# AI Agent Collaboration Guide
+
+## Table of Contents
+1. [Communication Protocol](#communication-protocol)
+2. [Development Standards](#development-standards)
+3. [Testing Standards](#testing-standards)
+4. [Environment](#environment)
+5. [Development Workflow](#development-workflow)
+6. [Twelve-Factor Reference](#twelve-factor-reference)
+7. [CLAUDE.md Stub](#claudemd-stub)
+8. [AI Agent Instructions](#ai-agent-instructions)
+
+---
+
+## Communication Protocol
+
+### Core Rules
+- **Clarity First**: Always ask clarifying questions when requirements are ambiguous
+- **Fact-Based**: Base all recommendations on verified, current information
+- **Simplicity Advocate**: Call out overcomplications and suggest simpler alternatives
+- **Safety First**: Never modify critical systems without explicit understanding and approval
+
+### User Profile
+- **Technical Level**: Non-coder but technically savvy
+- **Learning Style**: Understands concepts, needs executable instructions
+- **Expects**: Step-by-step guidance with clear explanations
+- **Comfortable with**: Command-line operations and scripts
+
+### Required Safeguards
+- Always identify affected files before making changes
+- Never modify authentication systems without explicit permission
+- Never alter database schema without proper migration files
+- Explain what changes will be made and why
+
+---
+
+## Development Standards
+
+### Validate Before You Build
+
+- **POC everything first.** Before committing to a design, build a quick proof-of-concept (~15 min) that validates the core logic. Keep it stupidly simple — manual steps are fine, hardcoded values are fine, no tests needed yet
+- **POC scope:** Cover the happy path and 2-3 common edge cases. If those work, the idea is sound
+- **Graduation criteria:** POC validates logic and covers most common scenarios → stop, design properly, then build with structure, tests, and error handling. Never ship the POC — rewrite it
+- **Build incrementally.** After POC graduates, break the work into small, independent modules. Focus on one at a time. Each piece must work on its own before integrating with the next
+
+### Dependency Hierarchy
+
+Always exhaust the simpler option before reaching for the next:
+
+1. **Vanilla language** — Write it yourself using only language primitives. If it's <50 lines and not security-critical, this is the answer
+2. **Standard library** — Use built-in modules (`os`, `json`, `pathlib`, `http`, `fs`, `crypto`). The stdlib is tested, maintained, and has zero supply chain risk
+3. **External library** — Only when both vanilla and stdlib are insufficient. Must pass the checklist below
+
+### External Dependency Checklist
+
+Before adding any external dependency, all of these must be true:
+- **Necessity:** Can't reasonably implement this with stdlib in <100 lines
+- **Maintained:** Active commits in the last 6 months, responsive maintainer
+- **Lightweight:** Few transitive dependencies (check the dep tree, not just the top-level)
+- **Established:** Widely used, not a single-maintainer hobby project for production-critical code
+- **Security-aware:** For security-critical domains (crypto, auth, sanitization, parsing untrusted input), a vetted library is *required* — never roll your own
+
+### Language Selection
+
+- **Use widely-adopted languages only** — Python, JavaScript/TypeScript, Go, Rust. No niche languages unless the domain demands it
+- **Pick the lightest language that fits the domain:** shell scripts for automation, Python for data/backend/CLI, TypeScript for web, Go for systems/infra, Rust for performance-critical
+- **Minimize the polyglot tax.** Every language in the stack adds CI config, tooling, and onboarding friction. Do not add a new language for one microservice — use what's already in the stack unless there's a compelling reason
+- **Vanilla over frameworks.** Express over NestJS, Flask over Django, unless the project genuinely needs the framework's structure. Structure can always be added later; removing a framework is painful
+
+### Build Rules
+
+- **Open-source only.** Always use open-source solutions. No vendor lock-in
+- **Lightweight over complex.** If two solutions solve the same problem, use the one with fewer moving parts, fewer dependencies, and less configuration
+- **Every line must have a purpose.** No speculative code, no "might need this later", no abstractions for one use case
+- **Simple > clever.** Readable code that a junior can follow beats elegant code that requires a PhD to debug
+- **Containerize only when necessary.** Start with a virtualenv or bare metal. Docker adds value for deployment parity and isolation — not for running a script
+
+### Red Flags — Stop and Flag These
+- Over-engineering simple problems
+- Adding external dependencies for trivial operations
+- Frameworks where a library or stdlib would suffice
+- Vendor-specific implementations when open alternatives exist
+- Skipping POC validation for unproven ideas
+
+---
+
+## Testing Standards
+
+### Rules
+
+**Test behavior, not implementation.** A test suite must give you confidence to refactor freely. If changing internal code (without changing behavior) breaks tests, those tests are liabilities, not assets.
+
+**Follow the Testing Trophy** (not the Testing Pyramid):
+- Few unit tests — only for pure logic, algorithms, and complex calculations
+- Many integration tests — the sweet spot; test real components working together
+- Some E2E tests — cover critical user journeys end-to-end
+- Static analysis — types and linters catch bugs cheaper than tests
+
+### When to Write Tests
+
+- **After the design stabilizes, not during exploration.** Do not TDD a prototype — you'll write 500 tests for code you delete tomorrow. First make it work (POC), then make it right (refactor + tests), then make it fast
+- **Write tests when the code has users.** If a function is called by other modules or exposed to users, it needs tests. Internal helpers that only serve one caller don't need their own test file
+- **Write tests for bugs.** Every bug fix must include a regression test that fails before the fix and passes after. This is the highest-value test you can write
+- **Write tests before refactoring.** Before changing working code, write characterization tests first to lock in current behavior, then refactor with confidence
+- **Do not write tests for glue code.** Code that just wires components together (calls A then B then C) is tested at the integration level, not unit level
+
+### TDD: When It Works and When It Doesn't
+
+- **TDD works for:** Pure functions, algorithms, parsers, validators, data transformations — anything with clear inputs and outputs
+- **TDD does not work for:** Exploring a design, building a POC, or unstable interfaces. Writing tests for unstable APIs creates churn and false confidence
+- **The rule:** You must understand what you're building before you TDD it. TDD is a design tool for known problems, not a discovery tool for unknown ones
+- **Red-green-refactor discipline:** If you do TDD, follow the cycle strictly. Write a failing test, write minimal code to pass, refactor. Do not write 20 tests then implement — that's front-loading waste
+
+### What Makes a Good Test
+
+- **Tests real behavior.** Call the public API, assert on observable output. Do not reach into internals
+- **Fails for the right reason.** A good test fails when the feature is broken, not when the implementation changes
+- **Reads like a spec.** Someone unfamiliar with the code must understand what the feature does by reading the test
+- **Self-contained.** Each test sets up its own state, runs, and cleans up. No ordering dependencies between tests
+- **Fast and deterministic.** Flaky tests erode trust. If a test depends on timing, network, or global state, fix that dependency
+
+### Anti-Patterns — Do Not Do These
+
+- **Mocking more than 60% of the test.** If most of the test is mock setup, you're testing mocks, not code. Use real implementations with `tmp_path`, `:memory:` SQLite, or test containers
+- **Smoke tests.** `assert result is not None` proves nothing. Assert on specific values, structure, or side effects
+- **Testing private methods.** If you need to test a private method, either it should be public or the public method's tests should cover it
+- **Mirroring implementation.** Tests that replicate the source code line-by-line break on every refactor and catch zero bugs
+- **Test-only production code.** Never add methods, flags, or branches to production code solely for testing. Use dependency injection instead
+
+### Test Organization
+
+- **Co-locate tests with packages:** `packages/<pkg>/tests/` not a root `tests/` directory. Each package owns its tests
+- **Separate by type:**
+  ```
+  packages/<pkg>/tests/
+    unit/           # Fast, isolated, mocked deps, <1s each
+    integration/    # Real DB, filesystem, multi-component, <10s each
+    e2e/            # Full workflows, subprocess calls, <60s each
+    conftest.py     # Shared fixtures for this package
+  ```
+- **One test file per module** (not per function). `test_auth.py` tests the auth module, not `test_login.py` + `test_logout.py` + `test_session.py`
+- **No duplicate test files.** Before creating a new test file, check if one already exists for that module
+
+### Markers and Signals
+
+| Marker | Purpose | CI Behavior |
+|--------|---------|-------------|
+| `@pytest.mark.slow` | Runtime > 5s | Run in full suite, skip in quick checks |
+| `@pytest.mark.ml` | Requires ML deps (torch, etc.) | Skip if deps not installed |
+| `@pytest.mark.real_api` | Calls external APIs | Skip in CI — run manually before release |
+
+**CI runs for fast signals:**
+- `pytest -m "not slow and not ml and not real_api"` — fast gate on every push (~30s)
+- `pytest` — full suite on PR merge or nightly
+- Package-level runs for targeted debugging: `pytest packages/core/tests/`
+
+### Coverage and Ratios
+
+- **Do not chase a coverage number.** 80% coverage with meaningless tests is worse than 40% with behavior-testing integration tests
+- **Cover the critical path first.** Data layer, auth, payment, core business logic — before helper utilities
+- **Coverage tells you what's NOT tested, not what IS tested.** High coverage with bad assertions is false confidence
+- **Delete tests that don't catch bugs.** If a test has never failed (or only fails on refactors), it's not providing value
+
+**Target ratio:** ~20% unit, ~60% integration, ~15% E2E, ~5% manual/exploratory
+
+### Test Tooling Standards
+
+- Use `tmp_path` for filesystem tests, `:memory:` or `tmp_path` SQLite for DB tests
+- Use dependency injection over `@patch` — it's more readable and survives refactors
+- Tests must be self-sufficient — no dependency on project directories, user config, or environment state
+- Use factories or builders for test data, not raw constructors with 15 arguments
+- Keep test fixtures close to where they're used. Shared fixtures in `conftest.py`, not a global test utilities package
+
+---
+
+## Environment
+
+- **OS**: Fedora Linux (use `dnf` for packages, `systemctl` for services)
+- **Testing**: pytest (Python), Jest/Vitest (JS/TS), Playwright (browser automation)
+
+---
+
+## Development Workflow
+
+### Environments
+- **Development**: Local machines
+- **Staging**: VPS with isolated database
+- **Production**: VPS with containerized setup
+
+### Deployment Strategy
+
+**Simple Projects:** `Local → GitHub → VPS (direct deployment)`
+
+**Complex Projects:** `Local → GitHub → GHCR → VPS (containerized)`
+
+---
+
+## Twelve-Factor Reference
+
+The [Twelve-Factor App](https://12factor.net) methodology for modern, scalable applications:
+
+| # | Factor | Rule |
+|---|--------|------|
+| 1 | Codebase | One repo per app, multiple deploys from same codebase |
+| 2 | Dependencies | Explicitly declare and isolate all dependencies |
+| 3 | Config | Store config in environment variables, never in code |
+| 4 | Backing Services | Treat databases, caches, queues as attached resources |
+| 5 | Build, Release, Run | Strict separation between build, release, and run stages |
+| 6 | Processes | Run as stateless processes, persist state externally |
+| 7 | Port Binding | Apps are self-contained, export services via port binding |
+| 8 | Concurrency | Scale out via the process model, not bigger instances |
+| 9 | Disposability | Fast startup, graceful shutdown, idempotent operations |
+| 10 | Dev/Prod Parity | Keep dev, staging, and production as similar as possible |
+| 11 | Logs | Treat logs as event streams to stdout |
+| 12 | Admin Processes | Run admin/maintenance tasks as one-off processes |
+
+---
+
+## CLAUDE.md Stub
+
+Copy this to any project's CLAUDE.md. These are mandatory rules, not suggestions.
+
+```markdown
+## Dev Rules
+
+**POC first.** Always validate logic with a ~15min proof-of-concept before building. Cover happy path + common edges. POC works → design properly → build with tests. Never ship the POC.
+
+**Build incrementally.** Break work into small independent modules. One piece at a time, each must work on its own before integrating.
+
+**Dependency hierarchy — follow strictly:** vanilla language → standard library → external (only when stdlib can't do it in <100 lines). External deps must be maintained, lightweight, and widely adopted. Exception: always use vetted libraries for security-critical code (crypto, auth, sanitization).
+
+**Lightweight over complex.** Fewer moving parts, fewer deps, less config. Express over NestJS, Flask over Django, unless the project genuinely needs the framework. Simple > clever. Readable > elegant.
+
+**Open-source only.** No vendor lock-in. Every line of code must have a purpose — no speculative code, no premature abstractions.
+
+For full development and testing standards, see `.claude/memory/AGENT_RULES.md`.
+```
+
+---
+
+## AI Agent Instructions
+
+When working with this user:
+1. **Always verify** you understand the requirements before proceeding
+2. **Provide step-by-step** instructions with clear explanations
+3. **Include ready-to-run** scripts and commands
+4. **Explain the "why"** behind technical recommendations
+5. **Flag potential issues** before they become problems
+6. **Suggest simpler alternatives** when appropriate
+7. **Never modify** authentication or database schema without explicit permission
+8. **Always identify** which files will be affected by changes

package/.claude/settings.local.json

@@ -0,0 +1,37 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(wc:*)",
+      "Bash(flatpak list)",
+      "WebSearch",
+      "Bash(tail:*)",
+      "Bash(grep:*)",
+      "Bash(rpm -qa)",
+      "Bash(node:*)",
+      "Bash(cp:*)",
+      "Bash(ls:*)",
+      "Bash(head:*)",
+      "Bash(which kwalletcli:*)",
+      "Bash(kwallet-query:*)",
+      "WebFetch(domain:github.com)",
+      "WebFetch(domain:raw.githubusercontent.com)",
+      "Bash(npm view:*)",
+      "Bash(git add:*)",
+      "Bash(git commit:*)",
+      "Bash(npm whoami:*)",
+      "Bash(npm pack:*)",
+      "Bash(npm publish:*)",
+      "Skill(git-commit)",
+      "Bash(python3:*)",
+      "Bash(printf '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{}}\\\\n{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\",\"params\":{}}\\\\n' | timeout 5 node /home/hamr/PycharmProjects/barebrowse/mcp-server.js 2>/dev/null)",
+      "Bash(chmod +x:*)",
+      "Bash(sort:*)",
+      "WebFetch(domain:www.npmjs.com)",
+      "WebFetch(domain:testcollab.com)",
+      "WebFetch(domain:deepwiki.com)",
+      "WebFetch(domain:mikhail.io)",
+      "WebFetch(domain:playbooks.com)",
+      "Bash(echo exit: $?:*)"
+    ]
+  }
+}

package/.claude/skills/barebrowse/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: barebrowse
+description: Browser automation using the user's real browser with real cookies. Handles consent walls, login sessions, and bot detection automatically.
+allowed-tools: Bash(barebrowse:*)
+---
+
+# barebrowse CLI — Browser Automation for Agents
+
+Browse any URL using the user's real browser with real cookies. Returns pruned ARIA snapshots (40-90% smaller than raw) with `[ref=N]` markers for interaction. Handles cookie consent, login sessions, and bot detection automatically.
+
+## Quick Start
+
+```bash
+barebrowse open https://example.com    # Start session + navigate
+barebrowse snapshot                    # Get ARIA snapshot → .barebrowse/page-*.yml
+barebrowse click 8                     # Click element with ref=8
+barebrowse snapshot                    # See result
+barebrowse close                       # End session
+```
+
+All output files go to `.barebrowse/` in the current directory. Read them with the Read tool when needed.
+
+## Commands
+
+### Session Lifecycle
+
+| Command | Description |
+|---------|-------------|
+| `barebrowse open [url] [flags]` | Start browser session. Optionally navigate to URL. |
+| `barebrowse close` | Close session and kill browser. |
+| `barebrowse status` | Check if session is running. |
+
+**Open flags:**
+- `--mode=headless|headed|hybrid` — Browser mode (default: headless)
+- `--no-cookies` — Skip cookie injection
+- `--browser=firefox|chromium` — Cookie source
+- `--prune-mode=act|read` — Default pruning mode
+- `--timeout=N` — Navigation timeout in ms
+
+### Navigation
+
+| Command | Output |
+|---------|--------|
+| `barebrowse goto <url>` | Navigates, waits for load, dismisses consent. Prints "ok". |
+| `barebrowse snapshot` | ARIA snapshot → `.barebrowse/page-<timestamp>.yml` |
+| `barebrowse snapshot --mode=read` | Read mode: keeps all text (for content extraction) |
+| `barebrowse screenshot` | Screenshot → `.barebrowse/screenshot-<timestamp>.png` |
+
+### Interaction
+
+| Command | Description |
+|---------|-------------|
+| `barebrowse click <ref>` | Click element (scrolls into view first) |
+| `barebrowse type <ref> <text>` | Type text into element |
+| `barebrowse fill <ref> <text>` | Clear existing content + type new text |
+| `barebrowse press <key>` | Press key: Enter, Tab, Escape, Backspace, Delete, arrows, Space |
+| `barebrowse scroll <deltaY>` | Scroll page (positive=down, negative=up) |
+| `barebrowse hover <ref>` | Hover over element (triggers tooltips) |
+| `barebrowse select <ref> <value>` | Select dropdown option |
+
+### Debugging
+
+| Command | Output |
+|---------|--------|
+| `barebrowse eval <expression>` | Evaluate JS in page, print result |
+| `barebrowse wait-idle` | Wait for network idle (no requests for 500ms) |
+| `barebrowse console-logs` | Console logs → `.barebrowse/console-<timestamp>.json` |
+| `barebrowse network-log` | Network log → `.barebrowse/network-<timestamp>.json` |
+| `barebrowse network-log --failed` | Only failed/4xx/5xx requests |
+
+## Snapshot Format
+
+The snapshot is a YAML-like ARIA tree. Each line is one node:
+
+```
+- WebArea "Example Domain" [ref=1]
+  - heading "Example Domain" [level=1] [ref=3]
+  - paragraph [ref=5]
+    - StaticText "This domain is for use in illustrative examples." [ref=6]
+  - link "More information..." [ref=8]
+```
+
+- `[ref=N]` — Use this number with click, type, fill, hover, select
+- Refs change on every snapshot — always take a fresh snapshot before interacting
+- **act mode** (default): interactive elements + labels — for clicking, typing, navigating
+- **read mode**: all text content — for reading articles, extracting data
+
+## Workflow Pattern
+
+1. `barebrowse open <url>` — start session
+2. `barebrowse snapshot` — observe page (read the .yml file)
+3. Decide action based on snapshot content
+4. `barebrowse click/type/fill/press/scroll <ref>` — act
+5. `barebrowse snapshot` — observe result (refs are now different!)
+6. Repeat 3-5 until goal achieved
+7. `barebrowse close` — clean up
+
+## Tips
+
+- **Always snapshot before interacting** — refs are ephemeral and change every time
+- **Use `fill` instead of `type`** when replacing existing text in input fields
+- **Use `--mode=read`** for snapshot when you need to extract article content or data
+- **Check `console-logs`** when page behavior seems wrong — JS errors show up there
+- **Check `network-log --failed`** to debug missing content or broken API calls
+- **Use `eval`** as an escape hatch when ARIA tree doesn't show what you need
+- **One session per project** — `.barebrowse/` is project-scoped
+- For bot-detected sites, use `--mode=headed` (requires browser with `--remote-debugging-port=9222`)

package/.claude/stash/barebrowse-research-2026-02-22.md

@@ -0,0 +1,49 @@
+# Stash: barebrowse Research & Repo Creation
+**Timestamp:** 2026-02-22
+**Session focus:** Research steipete repos, design shared browsing layer
+
+## Key Decisions
+
+1. **Researched steipete/sweetlink** — daemon + WebSocket + CDP architecture for driving real browser tabs. Not headless — opposite approach. Good ideas, overcomplicated for our needs.
+
+2. **Researched steipete/sweet-cookie** — TS library extracting cookies from real browser profiles (Chrome/FF/Safari). Direct fit for mcprune's Cloudflare/auth-wall problem.
+
+3. **Identified the parallel** — mcprune (headless DOM reading) and Multis (interactive browsing) share the same underlying need: get a browser context that loads any URL, authenticated as the user, without being blocked.
+
+4. **Designed `barebrowse`** — a unified browsing layer with 3 modes:
+   - `headless` — Playwright + cookie injection + stealth (mcprune)
+   - `headed` — CDP connect to running Chrome (Multis)
+   - `hybrid` — try headless, auto-fallback to headed
+
+5. **Key simplification over sweetlink** — skip daemon entirely, use CDP directly. Playwright wraps CDP already. No WebSocket bridge, no in-page runtime injection.
+
+6. **Named it `barebrowse`** — follows bare- ecosystem (bareagent, barebrowse, mcprune, multis)
+
+## Work Completed
+
+- Created repo: `~/PycharmProjects/barebrowse/` (git init, main branch)
+- Moved research doc: `mcprune/docs/peter-repos.md` → `barebrowse/docs/prd.md`
+- PRD includes: architecture diagrams, API sketches, 4-phase POC plan, repos to study, package structure
+
+## Repos to Borrow From
+
+| Repo | Take |
+|---|---|
+| steipete/sweet-cookie | Cookie extraction (TS) |
+| steipete/sweetlink | Selector discovery, click patterns, CDP dual-channel |
+| steipete/canvas | Stealth/anti-detection config |
+
+## Next Steps
+
+- [ ] Start barebrowse POC Phase 1 (headless + cookies)
+- [ ] Wire mcprune to use barebrowse instead of direct Playwright
+- [ ] Design Multis headed-mode integration
+
+## Bare Ecosystem Map
+
+```
+bareagent  → agent orchestration
+barebrowse → browser access layer (NEW)
+mcprune    → DOM snapshot pruning
+multis     → personal assistant
+```

package/.claude/stash/phase3-interactions-complete.md

@@ -0,0 +1,69 @@
+# Phase 3 Interactions — Complete
+
+**Date:** 2026-02-22
+**Session:** Real-world interaction testing + fixes
+
+## What Was Done
+
+### Fixes to `src/interact.js`
+1. **scrollIntoView before click** — `DOM.scrollIntoViewIfNeeded` in `getCenter()` before `DOM.getBoxModel`
+2. **`press(key)` function** — 14-key KEY_MAP (Enter, Tab, Escape, Backspace, Delete, arrows, Home/End, PageUp/Down, Space). Enter has `text: '\r'`, Tab has `text: '\t'` for proper form submission.
+3. **`type({ clear: true })`** — Ctrl+A select-all + Backspace before typing to replace pre-filled content
+4. **Key event text field** — Without `text: '\r'` on Enter keyDown, form `onsubmit` never fires
+
+### Additions to `src/index.js`
+- `page.press(key)` — wired to `cdpPress`
+- `page.waitForNavigation(timeout)` — `Page.loadEventFired` promise
+
+### Tests: 54/54 passing
+- `test/integration/interact.test.js` (15 tests, 8 suites):
+  - Round 1: data: URL fixture (7 tests — click, type, clear, offscreen scroll, Enter submit, unknown key error, link navigation)
+  - Round 2: Google Search (consent handling, type+Enter+nav — bot-blocked but flow works)
+  - Round 3: Wikipedia (article link click + navigation)
+  - Round 4: GitHub (SPA navigation with settle time)
+  - Round 5: DuckDuckGo (search + results — headless-friendly)
+  - Round 6: Hacker News (story link click + navigation)
+  - Round 7: Reddit/old.reddit.com (with fallback to www.reddit.com)
+  - Round 8: Firefox cookie injection (extractCookies → injectCookies → Network.getAllCookies verification)
+
+### `examples/headed-demo.js`
+10-step demo: Wikipedia → click link → DuckDuckGo → type query → Enter → results. Requires `chromium-browser --remote-debugging-port=9222`.
+
+### YouTube Demo (ad-hoc, in /tmp)
+Proved the full loop: Firefox cookies extracted → injected into headed Chromium → YouTube consent bypassed → searched "Family Portrait Pink" → clicked and played the video. User watched it happen live.
+
+## Key Findings
+
+### Real-world breakage patterns
+- **Cookie consent walls** block everything (Google, YouTube). Need locale-aware button matching or cookie injection to bypass.
+- **Bot detection** (Google, Reddit) blocks headless. Headed mode with real cookies is the fix.
+- **ARIA roles vary wildly** — DuckDuckGo search box is `LabelText` not `textbox`, Google's is `combobox` with `[expanded=false]` between role and ref.
+- **`Page.loadEventFired` doesn't fire** for data: URL → data: URL navigation or SPA transitions.
+- **`Page.frameNavigated` fires too early** — page DOM not ready yet, snapshots return empty.
+- **Example.com link text changed** from "More information..." to "Learn more" — real sites change.
+- **IANA page about example domains** contains "Example Domain" text — can't assert absence.
+- **Act mode prunes links** on simple pages like example.com — need browse mode to find clickable links.
+
+### Firefox → Chromium cookie flow (proven)
+```
+Firefox cookies.sqlite (plaintext) → extractCookies({ browser: 'firefox', domain })
+  → injectCookies(session, cookies) via Network.setCookie
+  → headless/headed Chromium uses user's sessions
+```
+Works for YouTube (8 cookies), Google (35 cookies), GitHub, etc.
+
+## Docs Updated
+- `docs/poc-plan.md` — Phase 3 DoD checkboxes checked, test section expanded
+- `docs/blueprint.md` — interactions section updated, "Real-world testing" marked DONE, wait strategies partially done
+- `docs/prd.md` — wait strategies and cookie auth decision updated
+
+## Commits
+1. `6017113` — feat(interact): add scrollIntoView, press(), clear, waitForNavigation
+2. `41bed12` — test: add real-site rounds + headed demo, update docs
+
+## What's Next (not started)
+- Phase 4: Hybrid mode (headless → headed fallback)
+- Stealth patches (`navigator.webdriver`, etc.)
+- More wait strategies (network idle, element presence)
+- Shopping/checkout flows, login forms, dropdowns
+- Screenshot capture for visual verification

package/.claude/stash/phase3-prep.md

@@ -0,0 +1,88 @@
+# Stash: barebrowse Phase 3 Prep
+**Timestamp:** 2026-02-22
+**Session focus:** Built Phase 1 + Phase 2 of barebrowse POC
+
+## What barebrowse Is
+Vanilla JS library — CDP-direct browsing for autonomous agents. URL in → pruned ARIA snapshot out. No Playwright, no bundled browser. Uses user's installed Chromium.
+
+## Completed Work
+
+### Phase 1 — CDP + ARIA Foundation (DONE)
+- `src/chromium.js` (142 lines) — Find/launch any Chromium browser, parse CDP WebSocket URL from stderr
+- `src/cdp.js` (148 lines) — Vanilla WebSocket CDP client with flattened session support (sessionId at top level)
+- `src/aria.js` (69 lines) — Format ARIA tree as YAML-like text, skip InlineTextBox/LineBreak noise
+- `src/index.js` (223 lines) — `browse(url)` and `connect(opts)` public API
+
+### Phase 2 — Auth + Prune (DONE)
+- `src/auth.js` (279 lines) — Cookie extraction from Chromium (AES-128-CBC + KWallet/GNOME keyring) and Firefox (plaintext). Injection via CDP `Network.setCookie`. Uses `node:sqlite` with `immutable=1` URI to read live DBs.
+- `src/prune.js` (472 lines) — Full port of mcprune's 9-step pruning pipeline adapted for CDP ARIA tree format. Two modes: `act` (actions only) and `browse` (keeps content).
+
+### Tests — 39/39 passing
+- `test/unit/prune.test.js` — 16 tests (pruning logic, pure function)
+- `test/unit/auth.test.js` — 7 tests (cookie extraction from Firefox)
+- `test/unit/cdp.test.js` — 5 tests (CDP client, browser launch, ARIA tree)
+- `test/integration/browse.test.js` — 11 tests (end-to-end pipeline)
+
+### Docs
+- `docs/prd.md` — Comprehensive PRD with all decisions and rationale
+- `docs/poc-plan.md` — 4-phase plan with DoD, test instructions, repo study reference
+- `CLAUDE.md` — Project dev rules stub
+
+## Key Architecture Decisions (settled)
+- CDP direct, not Playwright (no 200MB download)
+- ARIA tree, not DOM (semantic, token-efficient)
+- Pruning built-in from mcprune (not optional)
+- Three modes: headless/headed/hybrid (one flag, same CDP code)
+- Chromium-only (CDP constraint, Firefox later via BiDi)
+- Vanilla JS, ES modules, Node >= 22, zero required deps
+- sweet-cookie not on npm — wrote our own auth.js
+- Unique temp dirs per browser instance (`/tmp/barebrowse-{pid}-{timestamp}`)
+
+## Test Results
+- HN: 51,932 chars raw → 27,312 pruned (47% reduction on minimal site)
+- example.com: browse mode keeps paragraphs, act mode keeps only heading
+- Cookie extraction: 181 Firefox cookies across 54 domains
+- GitHub cookies found but `logged_in=no` (user not logged in via Firefox)
+
+## System Details
+- OS: Fedora Linux, KDE Plasma, Wayland
+- Node: 22.22.0 (built-in WebSocket, experimental node:sqlite)
+- Browser: Firefox default, Chromium installed via `sudo dnf install chromium`
+- Chromium binary: `/usr/bin/chromium-browser`
+- KWallet running with Chromium Safe Storage key available
+
+## What's Next — Phase 3 (Headed + Interaction)
+
+**Goal:** Connect to user's running browser, click/type on pages.
+
+**Files to create/update:**
+- Update `src/chromium.js` — `connect()` mode for running browser on debug port
+- `src/interact.js` — Click (`Input.dispatchMouseEvent`), type (`Input.dispatchKeyEvent`), scroll. Resolve ARIA nodeId to DOM coordinates.
+- Update `src/index.js` — Add click/type to connect() page handle
+
+**Key challenge:** Mapping ARIA nodeId to screen coordinates for click targeting. CDP `DOM.getBoxModel` + `Accessibility.getFullAXTree` node→backendDOMNodeId mapping needed.
+
+**Prerequisite:** User launches browser with `--remote-debugging-port=9222`
+
+## Phase 4 — Hybrid + bareagent Integration
+- `src/stealth.js` — Anti-detection patches via `Runtime.evaluate`
+- Hybrid mode: try headless, detect CF/403, fall back to headed
+- Wire as bareagent tool functions
+
+## Repos Studied
+- steipete/sweet-cookie — concept only (not on npm, wrote our own)
+- steipete/sweetlink — CDP-direct concept, skip daemon/WS bloat
+- steipete/canvas — stealth patterns noted for Phase 4
+- mcprune (own) — pruning logic fully ported
+
+## File Inventory
+```
+src/index.js      223 lines  — Public API
+src/chromium.js   142 lines  — Browser find/launch
+src/cdp.js        148 lines  — CDP WebSocket client
+src/aria.js        69 lines  — ARIA tree formatter
+src/auth.js       279 lines  — Cookie extraction + injection
+src/prune.js      472 lines  — ARIA tree pruning
+                 1333 total source
+                  576 total tests
+```

package/.claude/stash/phase4-complete-2026-02-22.md

@@ -0,0 +1,61 @@
+# Stash: Phase 4 Complete — MCP, bareagent, stealth, hybrid
+
+**Date:** 2026-02-22
+**Version:** 0.2.2 (published to npm)
+
+## What was done this session
+
+### Phase 4 implementation (all done)
+1. **barebrowse.context.md** — LLM-consumable integration guide, full API reference
+2. **mcp-server.js** — raw JSON-RPC 2.0 over stdio, 7 tools, zero SDK deps
+3. **.mcp.json** — uses `npx barebrowse mcp` for npm consumers
+4. **src/bareagent.js** — `createBrowseTools(opts)` → { tools, close }, 9 tools, auto-snapshot
+5. **src/stealth.js** — navigator.webdriver patches via Page.addScriptToEvaluateOnNewDocument
+6. **src/interact.js** — added hover() and select() (native + custom dropdown)
+7. **src/index.js** — screenshot, waitForNetworkIdle, SPA-aware waitForNavigation, hybrid mode, stealth wiring
+8. **cli.js** — `npx barebrowse mcp`, `npx barebrowse install` (auto-configures MCP clients), `npx barebrowse browse <url>`
+9. **docs/blueprint.md** — full 10-step pipeline, module table, integration sections
+10. **docs/testing.md** — test pyramid, all 54 tests documented
+11. **CHANGELOG.md** — 0.1.0 and 0.2.0 entries
+12. **README.md** — ASCII logo, tagline, no code, obstacle table, two usage modes, mcprune credit, measured token savings
+13. **.npmignore** — excludes .claude/, test/, examples/ from tarball
+14. **package.json** — subpath exports (./bareagent), bin entry, keywords
+
+### Key decisions
+- MCP action tools return 'ok', agent calls snapshot explicitly (cheap to chain)
+- bareagent action tools auto-return snapshot (LLM always sees result, 300ms settle)
+- waitForNavigation: catches loadEventFired timeout gracefully for SPA fallback (not Promise.race, which resolved too early on frameNavigated)
+- Hybrid mode: isChallengePage() heuristic on ARIA tree text, kills headless + connects headed
+- `npx barebrowse install` auto-detects Claude Desktop, Cursor, Claude Code and writes config
+- mcprune link: github.com/hamr0/mcprune (not nickvdyck)
+
+### Test results
+- 54/54 passing after all changes
+- YouTube demo verified twice (Firefox cookies → headed Chromium → search → play)
+- MCP server verified: initialize + tools/list over stdin
+- bareagent module verified: imports cleanly, returns 9 tools
+
+### Published versions
+- 0.1.0 — initial (before this session, already on npm)
+- 0.2.0 — phase 4 features (MCP, bareagent, stealth, hybrid, hover/select, screenshot, networkIdle)
+- 0.2.1 — README rewrite, MCP auto-installer, npx-based config
+- 0.2.2 — fix mcprune GitHub link
+
+### Token savings (measured)
+- example.com: 88% reduction
+- Hacker News: 47% reduction
+- Wikipedia: 63% reduction
+- DuckDuckGo: 87% reduction
+- Safe claim: 40-90%
+
+### Pending / not done
+- bareagent integration (user has prompt ready to paste into bareagent session)
+- npm publish of .npmignore improvement (will take effect on next publish)
+- File upload, drag and drop, infinite scroll, CAPTCHAs, cross-origin iframes, canvas/WebGL
+
+### bareagent wiring prompt
+User has the prompt ready. Key points:
+- `optionalDependencies: { "barebrowse": "^0.2.0" }`
+- Dynamic import: `await import('barebrowse/bareagent')` with try/catch
+- Tools already in bareagent format, no adapter needed
+- close() must be called (kills browser)