barebrowse 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,251 @@
1
+ # AI Agent Collaboration Guide
2
+
3
+ ## Table of Contents
4
+ 1. [Communication Protocol](#communication-protocol)
5
+ 2. [Development Standards](#development-standards)
6
+ 3. [Testing Standards](#testing-standards)
7
+ 4. [Environment](#environment)
8
+ 5. [Development Workflow](#development-workflow)
9
+ 6. [Twelve-Factor Reference](#twelve-factor-reference)
10
+ 7. [CLAUDE.md Stub](#claudemd-stub)
11
+ 8. [AI Agent Instructions](#ai-agent-instructions)
12
+
13
+ ---
14
+
15
+ ## Communication Protocol
16
+
17
+ ### Core Rules
18
+ - **Clarity First**: Always ask clarifying questions when requirements are ambiguous
19
+ - **Fact-Based**: Base all recommendations on verified, current information
20
+ - **Simplicity Advocate**: Call out overcomplications and suggest simpler alternatives
21
+ - **Safety First**: Never modify critical systems without explicit understanding and approval
22
+
23
+ ### User Profile
24
+ - **Technical Level**: Non-coder but technically savvy
25
+ - **Learning Style**: Understands concepts, needs executable instructions
26
+ - **Expects**: Step-by-step guidance with clear explanations
27
+ - **Comfortable with**: Command-line operations and scripts
28
+
29
+ ### Required Safeguards
30
+ - Always identify affected files before making changes
31
+ - Never modify authentication systems without explicit permission
32
+ - Never alter database schema without proper migration files
33
+ - Explain what changes will be made and why
34
+
35
+ ---
36
+
37
+ ## Development Standards
38
+
39
+ ### Validate Before You Build
40
+
41
+ - **POC everything first.** Before committing to a design, build a quick proof-of-concept (~15 min) that validates the core logic. Keep it stupidly simple — manual steps are fine, hardcoded values are fine, no tests needed yet
42
+ - **POC scope:** Cover the happy path and 2-3 common edge cases. If those work, the idea is sound
43
+ - **Graduation criteria:** POC validates logic and covers most common scenarios → stop, design properly, then build with structure, tests, and error handling. Never ship the POC — rewrite it
44
+ - **Build incrementally.** After POC graduates, break the work into small, independent modules. Focus on one at a time. Each piece must work on its own before integrating with the next
45
+
46
+ ### Dependency Hierarchy
47
+
48
+ Always exhaust the simpler option before reaching for the next:
49
+
50
+ 1. **Vanilla language** — Write it yourself using only language primitives. If it's <50 lines and not security-critical, this is the answer
51
+ 2. **Standard library** — Use built-in modules (`os`, `json`, `pathlib`, `http`, `fs`, `crypto`). The stdlib is tested, maintained, and has zero supply chain risk
52
+ 3. **External library** — Only when both vanilla and stdlib are insufficient. Must pass the checklist below
53
+
54
+ ### External Dependency Checklist
55
+
56
+ Before adding any external dependency, all of these must be true:
57
+ - **Necessity:** Can't reasonably implement this with stdlib in <100 lines
58
+ - **Maintained:** Active commits in the last 6 months, responsive maintainer
59
+ - **Lightweight:** Few transitive dependencies (check the dep tree, not just the top-level)
60
+ - **Established:** Widely used, not a single-maintainer hobby project for production-critical code
61
+ - **Security-aware:** For security-critical domains (crypto, auth, sanitization, parsing untrusted input), a vetted library is *required* — never roll your own
62
+
63
+ ### Language Selection
64
+
65
+ - **Use widely-adopted languages only** — Python, JavaScript/TypeScript, Go, Rust. No niche languages unless the domain demands it
66
+ - **Pick the lightest language that fits the domain:** shell scripts for automation, Python for data/backend/CLI, TypeScript for web, Go for systems/infra, Rust for performance-critical
67
+ - **Minimize the polyglot tax.** Every language in the stack adds CI config, tooling, and onboarding friction. Do not add a new language for one microservice — use what's already in the stack unless there's a compelling reason
68
+ - **Vanilla over frameworks.** Express over NestJS, Flask over Django, unless the project genuinely needs the framework's structure. Structure can always be added later; removing a framework is painful
69
+
70
+ ### Build Rules
71
+
72
+ - **Open-source only.** Always use open-source solutions. No vendor lock-in
73
+ - **Lightweight over complex.** If two solutions solve the same problem, use the one with fewer moving parts, fewer dependencies, and less configuration
74
+ - **Every line must have a purpose.** No speculative code, no "might need this later", no abstractions for one use case
75
+ - **Simple > clever.** Readable code that a junior can follow beats elegant code that requires a PhD to debug
76
+ - **Containerize only when necessary.** Start with a virtualenv or bare metal. Docker adds value for deployment parity and isolation — not for running a script
77
+
78
+ ### Red Flags — Stop and Flag These
79
+ - Over-engineering simple problems
80
+ - Adding external dependencies for trivial operations
81
+ - Frameworks where a library or stdlib would suffice
82
+ - Vendor-specific implementations when open alternatives exist
83
+ - Skipping POC validation for unproven ideas
84
+
85
+ ---
86
+
87
+ ## Testing Standards
88
+
89
+ ### Rules
90
+
91
+ **Test behavior, not implementation.** A test suite must give you confidence to refactor freely. If changing internal code (without changing behavior) breaks tests, those tests are liabilities, not assets.
92
+
93
+ **Follow the Testing Trophy** (not the Testing Pyramid):
94
+ - Few unit tests — only for pure logic, algorithms, and complex calculations
95
+ - Many integration tests — the sweet spot; test real components working together
96
+ - Some E2E tests — cover critical user journeys end-to-end
97
+ - Static analysis — types and linters catch bugs cheaper than tests
98
+
99
+ ### When to Write Tests
100
+
101
+ - **After the design stabilizes, not during exploration.** Do not TDD a prototype — you'll write 500 tests for code you delete tomorrow. First make it work (POC), then make it right (refactor + tests), then make it fast
102
+ - **Write tests when the code has users.** If a function is called by other modules or exposed to users, it needs tests. Internal helpers that only serve one caller don't need their own test file
103
+ - **Write tests for bugs.** Every bug fix must include a regression test that fails before the fix and passes after. This is the highest-value test you can write
104
+ - **Write tests before refactoring.** Before changing working code, write characterization tests first to lock in current behavior, then refactor with confidence
105
+ - **Do not write tests for glue code.** Code that just wires components together (calls A then B then C) is tested at the integration level, not unit level
106
+
107
+ ### TDD: When It Works and When It Doesn't
108
+
109
+ - **TDD works for:** Pure functions, algorithms, parsers, validators, data transformations — anything with clear inputs and outputs
110
+ - **TDD does not work for:** Exploring a design, building a POC, or unstable interfaces. Writing tests for unstable APIs creates churn and false confidence
111
+ - **The rule:** You must understand what you're building before you TDD it. TDD is a design tool for known problems, not a discovery tool for unknown ones
112
+ - **Red-green-refactor discipline:** If you do TDD, follow the cycle strictly. Write a failing test, write minimal code to pass, refactor. Do not write 20 tests then implement — that's front-loading waste
113
+
114
+ ### What Makes a Good Test
115
+
116
+ - **Tests real behavior.** Call the public API, assert on observable output. Do not reach into internals
117
+ - **Fails for the right reason.** A good test fails when the feature is broken, not when the implementation changes
118
+ - **Reads like a spec.** Someone unfamiliar with the code must understand what the feature does by reading the test
119
+ - **Self-contained.** Each test sets up its own state, runs, and cleans up. No ordering dependencies between tests
120
+ - **Fast and deterministic.** Flaky tests erode trust. If a test depends on timing, network, or global state, fix that dependency
121
+
122
+ ### Anti-Patterns — Do Not Do These
123
+
124
+ - **Mocking more than 60% of the test.** If most of the test is mock setup, you're testing mocks, not code. Use real implementations with `tmp_path`, `:memory:` SQLite, or test containers
125
+ - **Smoke tests.** `assert result is not None` proves nothing. Assert on specific values, structure, or side effects
126
+ - **Testing private methods.** If you need to test a private method, either it should be public or the public method's tests should cover it
127
+ - **Mirroring implementation.** Tests that replicate the source code line-by-line break on every refactor and catch zero bugs
128
+ - **Test-only production code.** Never add methods, flags, or branches to production code solely for testing. Use dependency injection instead
129
+
130
+ ### Test Organization
131
+
132
+ - **Co-locate tests with packages:** `packages/<pkg>/tests/` not a root `tests/` directory. Each package owns its tests
133
+ - **Separate by type:**
134
+ ```
135
+ packages/<pkg>/tests/
136
+ unit/ # Fast, isolated, mocked deps, <1s each
137
+ integration/ # Real DB, filesystem, multi-component, <10s each
138
+ e2e/ # Full workflows, subprocess calls, <60s each
139
+ conftest.py # Shared fixtures for this package
140
+ ```
141
+ - **One test file per module** (not per function). `test_auth.py` tests the auth module, not `test_login.py` + `test_logout.py` + `test_session.py`
142
+ - **No duplicate test files.** Before creating a new test file, check if one already exists for that module
143
+
144
+ ### Markers and Signals
145
+
146
+ | Marker | Purpose | CI Behavior |
147
+ |--------|---------|-------------|
148
+ | `@pytest.mark.slow` | Runtime > 5s | Run in full suite, skip in quick checks |
149
+ | `@pytest.mark.ml` | Requires ML deps (torch, etc.) | Skip if deps not installed |
150
+ | `@pytest.mark.real_api` | Calls external APIs | Skip in CI — run manually before release |
151
+
152
+ **CI runs for fast signals:**
153
+ - `pytest -m "not slow and not ml and not real_api"` — fast gate on every push (~30s)
154
+ - `pytest` — full suite on PR merge or nightly
155
+ - Package-level runs for targeted debugging: `pytest packages/core/tests/`
156
+
157
+ ### Coverage and Ratios
158
+
159
+ - **Do not chase a coverage number.** 80% coverage with meaningless tests is worse than 40% with behavior-testing integration tests
160
+ - **Cover the critical path first.** Data layer, auth, payment, core business logic — before helper utilities
161
+ - **Coverage tells you what's NOT tested, not what IS tested.** High coverage with bad assertions is false confidence
162
+ - **Delete tests that don't catch bugs.** If a test has never failed (or only fails on refactors), it's not providing value
163
+
164
+ **Target ratio:** ~20% unit, ~60% integration, ~15% E2E, ~5% manual/exploratory
165
+
166
+ ### Test Tooling Standards
167
+
168
+ - Use `tmp_path` for filesystem tests, `:memory:` or `tmp_path` SQLite for DB tests
169
+ - Use dependency injection over `@patch` — it's more readable and survives refactors
170
+ - Tests must be self-sufficient — no dependency on project directories, user config, or environment state
171
+ - Use factories or builders for test data, not raw constructors with 15 arguments
172
+ - Keep test fixtures close to where they're used. Shared fixtures in `conftest.py`, not a global test utilities package
173
+
174
+ ---
175
+
176
+ ## Environment
177
+
178
+ - **OS**: Fedora Linux (use `dnf` for packages, `systemctl` for services)
179
+ - **Testing**: pytest (Python), Jest/Vitest (JS/TS), Playwright (browser automation)
180
+
181
+ ---
182
+
183
+ ## Development Workflow
184
+
185
+ ### Environments
186
+ - **Development**: Local machines
187
+ - **Staging**: VPS with isolated database
188
+ - **Production**: VPS with containerized setup
189
+
190
+ ### Deployment Strategy
191
+
192
+ **Simple Projects:** `Local → GitHub → VPS (direct deployment)`
193
+
194
+ **Complex Projects:** `Local → GitHub → GHCR → VPS (containerized)`
195
+
196
+ ---
197
+
198
+ ## Twelve-Factor Reference
199
+
200
+ The [Twelve-Factor App](https://12factor.net) methodology for modern, scalable applications:
201
+
202
+ | # | Factor | Rule |
203
+ |---|--------|------|
204
+ | 1 | Codebase | One repo per app, multiple deploys from same codebase |
205
+ | 2 | Dependencies | Explicitly declare and isolate all dependencies |
206
+ | 3 | Config | Store config in environment variables, never in code |
207
+ | 4 | Backing Services | Treat databases, caches, queues as attached resources |
208
+ | 5 | Build, Release, Run | Strict separation between build, release, and run stages |
209
+ | 6 | Processes | Run as stateless processes, persist state externally |
210
+ | 7 | Port Binding | Apps are self-contained, export services via port binding |
211
+ | 8 | Concurrency | Scale out via the process model, not bigger instances |
212
+ | 9 | Disposability | Fast startup, graceful shutdown, idempotent operations |
213
+ | 10 | Dev/Prod Parity | Keep dev, staging, and production as similar as possible |
214
+ | 11 | Logs | Treat logs as event streams to stdout |
215
+ | 12 | Admin Processes | Run admin/maintenance tasks as one-off processes |
216
+
217
+ ---
218
+
219
+ ## CLAUDE.md Stub
220
+
221
+ Copy this to any project's CLAUDE.md. These are mandatory rules, not suggestions.
222
+
223
+ ```markdown
224
+ ## Dev Rules
225
+
226
+ **POC first.** Always validate logic with a ~15min proof-of-concept before building. Cover happy path + common edges. POC works → design properly → build with tests. Never ship the POC.
227
+
228
+ **Build incrementally.** Break work into small independent modules. One piece at a time, each must work on its own before integrating.
229
+
230
+ **Dependency hierarchy — follow strictly:** vanilla language → standard library → external (only when stdlib can't do it in <100 lines). External deps must be maintained, lightweight, and widely adopted. Exception: always use vetted libraries for security-critical code (crypto, auth, sanitization).
231
+
232
+ **Lightweight over complex.** Fewer moving parts, fewer deps, less config. Express over NestJS, Flask over Django, unless the project genuinely needs the framework. Simple > clever. Readable > elegant.
233
+
234
+ **Open-source only.** No vendor lock-in. Every line of code must have a purpose — no speculative code, no premature abstractions.
235
+
236
+ For full development and testing standards, see `.claude/memory/AGENT_RULES.md`.
237
+ ```
238
+
239
+ ---
240
+
241
+ ## AI Agent Instructions
242
+
243
+ When working with this user:
244
+ 1. **Always verify** you understand the requirements before proceeding
245
+ 2. **Provide step-by-step** instructions with clear explanations
246
+ 3. **Include ready-to-run** scripts and commands
247
+ 4. **Explain the "why"** behind technical recommendations
248
+ 5. **Flag potential issues** before they become problems
249
+ 6. **Suggest simpler alternatives** when appropriate
250
+ 7. **Never modify** authentication or database schema without explicit permission
251
+ 8. **Always identify** which files will be affected by changes
@@ -0,0 +1,30 @@
1
+ {
2
+ "permissions": {
3
+ "allow": [
4
+ "Bash(wc:*)",
5
+ "Bash(flatpak list)",
6
+ "WebSearch",
7
+ "Bash(tail:*)",
8
+ "Bash(grep:*)",
9
+ "Bash(rpm -qa)",
10
+ "Bash(node:*)",
11
+ "Bash(cp:*)",
12
+ "Bash(ls:*)",
13
+ "Bash(head:*)",
14
+ "Bash(which kwalletcli:*)",
15
+ "Bash(kwallet-query:*)",
16
+ "WebFetch(domain:github.com)",
17
+ "WebFetch(domain:raw.githubusercontent.com)",
18
+ "Bash(npm view:*)",
19
+ "Bash(git add:*)",
20
+ "Bash(git commit:*)",
21
+ "Bash(npm whoami:*)",
22
+ "Bash(npm pack:*)",
23
+ "Bash(npm publish:*)",
24
+ "Skill(git-commit)",
25
+ "Bash(python3:*)",
26
+ "Bash(printf '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{}}\\\\n{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\",\"params\":{}}\\\\n' | timeout 5 node /home/hamr/PycharmProjects/barebrowse/mcp-server.js 2>/dev/null)",
27
+ "Bash(chmod +x:*)"
28
+ ]
29
+ }
30
+ }
@@ -0,0 +1,49 @@
1
+ # Stash: barebrowse Research & Repo Creation
2
+ **Timestamp:** 2026-02-22
3
+ **Session focus:** Research steipete repos, design shared browsing layer
4
+
5
+ ## Key Decisions
6
+
7
+ 1. **Researched steipete/sweetlink** — daemon + WebSocket + CDP architecture for driving real browser tabs. Not headless — opposite approach. Good ideas, overcomplicated for our needs.
8
+
9
+ 2. **Researched steipete/sweet-cookie** — TS library extracting cookies from real browser profiles (Chrome/FF/Safari). Direct fit for mcprune's Cloudflare/auth-wall problem.
10
+
11
+ 3. **Identified the parallel** — mcprune (headless DOM reading) and Multis (interactive browsing) share the same underlying need: get a browser context that loads any URL, authenticated as the user, without being blocked.
12
+
13
+ 4. **Designed `barebrowse`** — a unified browsing layer with 3 modes:
14
+ - `headless` — Playwright + cookie injection + stealth (mcprune)
15
+ - `headed` — CDP connect to running Chrome (Multis)
16
+ - `hybrid` — try headless, auto-fallback to headed
17
+
18
+ 5. **Key simplification over sweetlink** — skip daemon entirely, use CDP directly. Playwright wraps CDP already. No WebSocket bridge, no in-page runtime injection.
19
+
20
+ 6. **Named it `barebrowse`** — follows bare- ecosystem (bareagent, barebrowse, mcprune, multis)
21
+
22
+ ## Work Completed
23
+
24
+ - Created repo: `~/PycharmProjects/barebrowse/` (git init, main branch)
25
+ - Moved research doc: `mcprune/docs/peter-repos.md` → `barebrowse/docs/prd.md`
26
+ - PRD includes: architecture diagrams, API sketches, 4-phase POC plan, repos to study, package structure
27
+
28
+ ## Repos to Borrow From
29
+
30
+ | Repo | Take |
31
+ |---|---|
32
+ | steipete/sweet-cookie | Cookie extraction (TS) |
33
+ | steipete/sweetlink | Selector discovery, click patterns, CDP dual-channel |
34
+ | steipete/canvas | Stealth/anti-detection config |
35
+
36
+ ## Next Steps
37
+
38
+ - [ ] Start barebrowse POC Phase 1 (headless + cookies)
39
+ - [ ] Wire mcprune to use barebrowse instead of direct Playwright
40
+ - [ ] Design Multis headed-mode integration
41
+
42
+ ## Bare Ecosystem Map
43
+
44
+ ```
45
+ bareagent → agent orchestration
46
+ barebrowse → browser access layer (NEW)
47
+ mcprune → DOM snapshot pruning
48
+ multis → personal assistant
49
+ ```
@@ -0,0 +1,69 @@
1
+ # Phase 3 Interactions — Complete
2
+
3
+ **Date:** 2026-02-22
4
+ **Session:** Real-world interaction testing + fixes
5
+
6
+ ## What Was Done
7
+
8
+ ### Fixes to `src/interact.js`
9
+ 1. **scrollIntoView before click** — `DOM.scrollIntoViewIfNeeded` in `getCenter()` before `DOM.getBoxModel`
10
+ 2. **`press(key)` function** — 14-key KEY_MAP (Enter, Tab, Escape, Backspace, Delete, arrows, Home/End, PageUp/Down, Space). Enter has `text: '\r'`, Tab has `text: '\t'` for proper form submission.
11
+ 3. **`type({ clear: true })`** — Ctrl+A select-all + Backspace before typing to replace pre-filled content
12
+ 4. **Key event text field** — Without `text: '\r'` on Enter keyDown, form `onsubmit` never fires
13
+
14
+ ### Additions to `src/index.js`
15
+ - `page.press(key)` — wired to `cdpPress`
16
+ - `page.waitForNavigation(timeout)` — `Page.loadEventFired` promise
17
+
18
+ ### Tests: 54/54 passing
19
+ - `test/integration/interact.test.js` (15 tests, 8 suites):
20
+ - Round 1: data: URL fixture (7 tests — click, type, clear, offscreen scroll, Enter submit, unknown key error, link navigation)
21
+ - Round 2: Google Search (consent handling, type+Enter+nav — bot-blocked but flow works)
22
+ - Round 3: Wikipedia (article link click + navigation)
23
+ - Round 4: GitHub (SPA navigation with settle time)
24
+ - Round 5: DuckDuckGo (search + results — headless-friendly)
25
+ - Round 6: Hacker News (story link click + navigation)
26
+ - Round 7: Reddit/old.reddit.com (with fallback to www.reddit.com)
27
+ - Round 8: Firefox cookie injection (extractCookies → injectCookies → Network.getAllCookies verification)
28
+
29
+ ### `examples/headed-demo.js`
30
+ 10-step demo: Wikipedia → click link → DuckDuckGo → type query → Enter → results. Requires `chromium-browser --remote-debugging-port=9222`.
31
+
32
+ ### YouTube Demo (ad-hoc, in /tmp)
33
+ Proved the full loop: Firefox cookies extracted → injected into headed Chromium → YouTube consent bypassed → searched "Family Portrait Pink" → clicked and played the video. User watched it happen live.
34
+
35
+ ## Key Findings
36
+
37
+ ### Real-world breakage patterns
38
+ - **Cookie consent walls** block everything (Google, YouTube). Need locale-aware button matching or cookie injection to bypass.
39
+ - **Bot detection** (Google, Reddit) blocks headless. Headed mode with real cookies is the fix.
40
+ - **ARIA roles vary wildly** — DuckDuckGo search box is `LabelText` not `textbox`, Google's is `combobox` with `[expanded=false]` between role and ref.
41
+ - **`Page.loadEventFired` doesn't fire** for data: URL → data: URL navigation or SPA transitions.
42
+ - **`Page.frameNavigated` fires too early** — page DOM not ready yet, snapshots return empty.
43
+ - **Example.com link text changed** from "More information..." to "Learn more" — real sites change.
44
+ - **IANA page about example domains** contains "Example Domain" text — can't assert absence.
45
+ - **Act mode prunes links** on simple pages like example.com — need browse mode to find clickable links.
46
+
47
+ ### Firefox → Chromium cookie flow (proven)
48
+ ```
49
+ Firefox cookies.sqlite (plaintext) → extractCookies({ browser: 'firefox', domain })
50
+ → injectCookies(session, cookies) via Network.setCookie
51
+ → headless/headed Chromium uses user's sessions
52
+ ```
53
+ Works for YouTube (8 cookies), Google (35 cookies), GitHub, etc.
54
+
55
+ ## Docs Updated
56
+ - `docs/poc-plan.md` — Phase 3 DoD checkboxes checked, test section expanded
57
+ - `docs/blueprint.md` — interactions section updated, "Real-world testing" marked DONE, wait strategies partially done
58
+ - `docs/prd.md` — wait strategies and cookie auth decision updated
59
+
60
+ ## Commits
61
+ 1. `6017113` — feat(interact): add scrollIntoView, press(), clear, waitForNavigation
62
+ 2. `41bed12` — test: add real-site rounds + headed demo, update docs
63
+
64
+ ## What's Next (not started)
65
+ - Phase 4: Hybrid mode (headless → headed fallback)
66
+ - Stealth patches (`navigator.webdriver`, etc.)
67
+ - More wait strategies (network idle, element presence)
68
+ - Shopping/checkout flows, login forms, dropdowns
69
+ - Screenshot capture for visual verification
@@ -0,0 +1,88 @@
1
+ # Stash: barebrowse Phase 3 Prep
2
+ **Timestamp:** 2026-02-22
3
+ **Session focus:** Built Phase 1 + Phase 2 of barebrowse POC
4
+
5
+ ## What barebrowse Is
6
+ Vanilla JS library — CDP-direct browsing for autonomous agents. URL in → pruned ARIA snapshot out. No Playwright, no bundled browser. Uses user's installed Chromium.
7
+
8
+ ## Completed Work
9
+
10
+ ### Phase 1 — CDP + ARIA Foundation (DONE)
11
+ - `src/chromium.js` (142 lines) — Find/launch any Chromium browser, parse CDP WebSocket URL from stderr
12
+ - `src/cdp.js` (148 lines) — Vanilla WebSocket CDP client with flattened session support (sessionId at top level)
13
+ - `src/aria.js` (69 lines) — Format ARIA tree as YAML-like text, skip InlineTextBox/LineBreak noise
14
+ - `src/index.js` (223 lines) — `browse(url)` and `connect(opts)` public API
15
+
16
+ ### Phase 2 — Auth + Prune (DONE)
17
+ - `src/auth.js` (279 lines) — Cookie extraction from Chromium (AES-128-CBC + KWallet/GNOME keyring) and Firefox (plaintext). Injection via CDP `Network.setCookie`. Uses `node:sqlite` with `immutable=1` URI to read live DBs.
18
+ - `src/prune.js` (472 lines) — Full port of mcprune's 9-step pruning pipeline adapted for CDP ARIA tree format. Two modes: `act` (actions only) and `browse` (keeps content).
19
+
20
+ ### Tests — 39/39 passing
21
+ - `test/unit/prune.test.js` — 16 tests (pruning logic, pure function)
22
+ - `test/unit/auth.test.js` — 7 tests (cookie extraction from Firefox)
23
+ - `test/unit/cdp.test.js` — 5 tests (CDP client, browser launch, ARIA tree)
24
+ - `test/integration/browse.test.js` — 11 tests (end-to-end pipeline)
25
+
26
+ ### Docs
27
+ - `docs/prd.md` — Comprehensive PRD with all decisions and rationale
28
+ - `docs/poc-plan.md` — 4-phase plan with DoD, test instructions, repo study reference
29
+ - `CLAUDE.md` — Project dev rules stub
30
+
31
+ ## Key Architecture Decisions (settled)
32
+ - CDP direct, not Playwright (no 200MB download)
33
+ - ARIA tree, not DOM (semantic, token-efficient)
34
+ - Pruning built-in from mcprune (not optional)
35
+ - Three modes: headless/headed/hybrid (one flag, same CDP code)
36
+ - Chromium-only (CDP constraint, Firefox later via BiDi)
37
+ - Vanilla JS, ES modules, Node >= 22, zero required deps
38
+ - sweet-cookie not on npm — wrote our own auth.js
39
+ - Unique temp dirs per browser instance (`/tmp/barebrowse-{pid}-{timestamp}`)
40
+
41
+ ## Test Results
42
+ - HN: 51,932 chars raw → 27,312 pruned (47% reduction on minimal site)
43
+ - example.com: browse mode keeps paragraphs, act mode keeps only heading
44
+ - Cookie extraction: 181 Firefox cookies across 54 domains
45
+ - GitHub cookies found but `logged_in=no` (user not logged in via Firefox)
46
+
47
+ ## System Details
48
+ - OS: Fedora Linux, KDE Plasma, Wayland
49
+ - Node: 22.22.0 (built-in WebSocket, experimental node:sqlite)
50
+ - Browser: Firefox default, Chromium installed via `sudo dnf install chromium`
51
+ - Chromium binary: `/usr/bin/chromium-browser`
52
+ - KWallet running with Chromium Safe Storage key available
53
+
54
+ ## What's Next — Phase 3 (Headed + Interaction)
55
+
56
+ **Goal:** Connect to user's running browser, click/type on pages.
57
+
58
+ **Files to create/update:**
59
+ - Update `src/chromium.js` — `connect()` mode for running browser on debug port
60
+ - `src/interact.js` — Click (`Input.dispatchMouseEvent`), type (`Input.dispatchKeyEvent`), scroll. Resolve ARIA nodeId to DOM coordinates.
61
+ - Update `src/index.js` — Add click/type to connect() page handle
62
+
63
+ **Key challenge:** Mapping ARIA nodeId to screen coordinates for click targeting. CDP `DOM.getBoxModel` + `Accessibility.getFullAXTree` node→backendDOMNodeId mapping needed.
64
+
65
+ **Prerequisite:** User launches browser with `--remote-debugging-port=9222`
66
+
67
+ ## Phase 4 — Hybrid + bareagent Integration
68
+ - `src/stealth.js` — Anti-detection patches via `Runtime.evaluate`
69
+ - Hybrid mode: try headless, detect CF/403, fall back to headed
70
+ - Wire as bareagent tool functions
71
+
72
+ ## Repos Studied
73
+ - steipete/sweet-cookie — concept only (not on npm, wrote our own)
74
+ - steipete/sweetlink — CDP-direct concept, skip daemon/WS bloat
75
+ - steipete/canvas — stealth patterns noted for Phase 4
76
+ - mcprune (own) — pruning logic fully ported
77
+
78
+ ## File Inventory
79
+ ```
80
+ src/index.js 223 lines — Public API
81
+ src/chromium.js 142 lines — Browser find/launch
82
+ src/cdp.js 148 lines — CDP WebSocket client
83
+ src/aria.js 69 lines — ARIA tree formatter
84
+ src/auth.js 279 lines — Cookie extraction + injection
85
+ src/prune.js 472 lines — ARIA tree pruning
86
+ 1333 total source
87
+ 576 total tests
88
+ ```
package/.mcp.json ADDED
@@ -0,0 +1,8 @@
1
+ {
2
+ "mcpServers": {
3
+ "barebrowse": {
4
+ "command": "npx",
5
+ "args": ["barebrowse", "mcp"]
6
+ }
7
+ }
8
+ }
package/CHANGELOG.md ADDED
@@ -0,0 +1,93 @@
1
+ # Changelog
2
+
3
+ ## 0.2.0
4
+
5
+ Agent integration release. MCP server, bareagent adapter, and interaction features that make barebrowse usable as a standalone tool or embedded browsing layer.
6
+
7
+ ### New: MCP server
8
+ - Raw JSON-RPC 2.0 over stdio, zero SDK dependencies
9
+ - 7 tools: `browse`, `goto`, `snapshot`, `click`, `type`, `press`, `scroll`
10
+ - Singleton session page, lazy-created on first session tool call
11
+ - `npx barebrowse mcp` to start, `npx barebrowse install` to auto-configure
12
+
13
+ ### New: MCP auto-installer
14
+ - `npx barebrowse install` detects Claude Desktop, Cursor, and Claude Code
15
+ - Writes MCP config automatically -- no manual JSON editing
16
+ - Reports status for each detected client
17
+
18
+ ### New: bareagent tool adapter
19
+ - `import { createBrowseTools } from 'barebrowse/bareagent'`
20
+ - Returns `{ tools, close }` with 9 bareagent-compatible tools
21
+ - Action tools auto-return fresh snapshot after each action (300ms settle)
22
+ - Tools: browse, goto, snapshot, click, type, press, scroll, select, screenshot
23
+
24
+ ### New: stealth patches
25
+ - `src/stealth.js` -- anti-detection for headless mode
26
+ - Uses `Page.addScriptToEvaluateOnNewDocument` (runs before page scripts)
27
+ - Patches: `navigator.webdriver`, `navigator.plugins`, `navigator.languages`, `window.chrome`, `Permissions.prototype.query`
28
+ - Auto-applied in headless mode
29
+
30
+ ### New: interactions
31
+ - `page.hover(ref)` -- mouse move to element center, triggers hover styles/tooltips
32
+ - `page.select(ref, value)` -- native `<select>` (set value + change event) or custom dropdown (click + find option)
33
+ - `page.screenshot(opts)` -- `Page.captureScreenshot`, returns base64 (png/jpeg/webp)
34
+
35
+ ### New: wait strategies
36
+ - `page.waitForNetworkIdle(opts)` -- resolve when no pending requests for N ms (default 500)
37
+ - `page.waitForNavigation()` now SPA-aware -- falls back gracefully when no `loadEventFired` fires
38
+
39
+ ### New: hybrid mode
40
+ - `mode: 'hybrid'` in `browse()` -- tries headless, detects challenge pages (Cloudflare, etc.), falls back to headed
41
+ - Challenge detection via ARIA tree heuristic ("Just a moment", "Checking your browser", etc.)
42
+
43
+ ### New: CLI
44
+ - `npx barebrowse mcp` -- start MCP server
45
+ - `npx barebrowse install` -- auto-configure MCP clients
46
+ - `npx barebrowse browse <url>` -- one-shot browse, print snapshot to stdout
47
+
48
+ ### New: documentation
49
+ - `README.md` -- complete guide: idea, token savings, modes, library vs MCP, bareagent wiring
50
+ - `barebrowse.context.md` -- LLM-consumable integration guide for AI assistants
51
+ - `docs/testing.md` -- test pyramid, all 54 tests documented, CI guidance
52
+ - `docs/blueprint.md` -- updated with full 10-step pipeline, module table, integration sections
53
+
54
+ ### Changed
55
+ - `package.json` -- subpath exports (`./bareagent`), `bin` entry, keywords
56
+ - `src/index.js` -- stealth auto-applied in headless via `createPage()`, `type()` param renamed to avoid shadowing
57
+ - `src/interact.js` -- `getCenter()` reused by new `hover()` function
58
+
59
+ ### Tests
60
+ - 54 tests passing (was 47 in 0.1.0)
61
+ - All existing tests unchanged and passing
62
+
63
+ ---
64
+
65
+ ## 0.1.0
66
+
67
+ Initial release. CDP-direct browsing with ARIA snapshots.
68
+
69
+ ### Core
70
+ - `browse(url, opts)` -- one-shot: URL in, pruned ARIA snapshot out
71
+ - `connect(opts)` -- session: navigate, interact, observe across pages
72
+ - Three modes: headless (default), headed (connect to running browser)
73
+ - Zero required dependencies, vanilla JS, ES modules, Node >= 22
74
+
75
+ ### Modules
76
+ - `src/cdp.js` -- WebSocket CDP client with flattened session support
77
+ - `src/chromium.js` -- find/launch any installed Chromium browser
78
+ - `src/aria.js` -- format ARIA tree as YAML-like text
79
+ - `src/auth.js` -- cookie extraction from Firefox (SQLite) and Chromium (AES + keyring)
80
+ - `src/prune.js` -- 9-step ARIA pruning pipeline (47-95% token reduction)
81
+ - `src/interact.js` -- click, type (with clear), press (14 special keys), scroll
82
+ - `src/consent.js` -- auto-dismiss cookie consent dialogs (7 languages, 16+ sites tested)
83
+
84
+ ### Features
85
+ - Cookie injection from Firefox/Chromium into headless CDP sessions
86
+ - Permission suppression (notifications, geolocation, camera, mic) via launch flags + CDP
87
+ - Cookie consent auto-dismiss across EN, NL, DE, FR, ES, IT, PT
88
+ - `waitForNavigation()` for post-click page loads
89
+ - Unique temp dirs per headless instance to avoid profile locking
90
+
91
+ ### Tests
92
+ - 47 tests across 5 files (unit: prune, auth, cdp; integration: browse, interact)
93
+ - Real-site testing: Google, Wikipedia, GitHub, DuckDuckGo, YouTube, HN, Reddit
package/CLAUDE.md ADDED
@@ -0,0 +1,22 @@
1
+ ## Dev Rules
2
+
3
+ **POC first.** Always validate logic with a ~15min proof-of-concept before building. Cover happy path + common edges. POC works → design properly → build with tests. Never ship the POC.
4
+
5
+ **Build incrementally.** Break work into small independent modules. One piece at a time, each must work on its own before integrating.
6
+
7
+ **Dependency hierarchy — follow strictly:** vanilla language → standard library → external (only when stdlib can't do it in <100 lines). External deps must be maintained, lightweight, and widely adopted. Exception: always use vetted libraries for security-critical code (crypto, auth, sanitization).
8
+
9
+ **Lightweight over complex.** Fewer moving parts, fewer deps, less config. Simple > clever. Readable > elegant.
10
+
11
+ **Open-source only.** No vendor lock-in. Every line of code must have a purpose — no speculative code, no premature abstractions.
12
+
13
+ ## Project Specifics
14
+
15
+ - **Language:** Vanilla JavaScript, ES modules, no build step
16
+ - **Runtime:** Node.js >= 22 (built-in WebSocket, sqlite)
17
+ - **Protocol:** CDP (Chrome DevTools Protocol) direct — no Playwright
18
+ - **Browser:** Any installed Chromium-based browser (chromium, chrome, brave, edge)
19
+ - **Key files:** `src/index.js` (API), `src/cdp.js` (CDP client), `src/chromium.js` (browser launch), `src/aria.js` (ARIA formatting)
20
+ - **Docs:** `docs/prd.md` (decisions + rationale), `docs/poc-plan.md` (phases + DoD)
21
+
22
+ For full development and testing standards, see `.claude/memory/AGENT_RULES.md`.