cdpilot 0.4.3 → 0.5.0

package/README.md

@@ -37,22 +37,34 @@ AI agents and developers need browser control that **just works**:
 - **AI control warning** — Red toast notification appears when you hover during active automation
 - **Privacy-first** — Everything runs locally. No data leaves your machine
 
-### Why Brave?
+### Browser Selection (Workload-Aware Auto-Pick)
 
-cdpilot uses [Brave Browser](https://brave.com) as its engine. Here's why:
+cdpilot picks the right browser for what you're doing. `auto` (default) is a
+two-axis policy — extension workload × platform stability:
 
-| Feature | Brave | Chrome | Why it matters |
-|---------|-------|--------|---------------|
-| **Built-in ad blocker** | Shields (native) | Extension needed | Pages load faster, less noise in DOM |
-| **Tracker blocking** | Default on | Manual config | Cleaner network logs for debugging |
-| **Fingerprint protection** | Native | None | Better privacy for automated sessions |
-| **Chromium-based** | Full CDP support | Full CDP support | Same DevTools Protocol, same power |
-| **Open source** | Yes (MPL 2.0) | Chromium yes, Chrome no | Transparent, auditable |
-| **Resource usage** | Lower memory | Higher memory | Better for running alongside your work |
+| Your workload | Auto-pick order |
+|---|---|
+| Has extensions registered (`ext-install`) | vivaldi → brave → edge → chromium → chrome |
+| No extensions (pure automation) | chrome → vivaldi → edge → chromium → brave |
 
-**TL;DR:** Brave = Chrome's power + built-in privacy + less bloat. Perfect for automation.
+Override anytime:
 
-> cdpilot also works with Chrome and Chromium as fallback. Brave is recommended, not required.
+```bash
+cdpilot browser            # show current pick + reason
+cdpilot browser vivaldi    # pin to Vivaldi
+cdpilot browser auto       # restore smart default
+```
+
+**Why the split?**
+- **Chrome 147+ silently drops `--load-extension`** for unpacked extensions
+  (no error, no warning). Verified — `chrome://extensions` shows 0 items.
+- **Vivaldi, Brave, Edge, Chromium** honor `--load-extension` (tested).
+- On **macOS 26 (Tahoe)** Brave 1.89 crashes deterministically at ~7min
+  uptime (SIGTRAP in ThreadPoolForegroundWorker). cdpilot detects the OS
+  and demotes Brave automatically until a fixed Brave release ships.
+
+Each browser gets its own isolated profile (`~/.cdpilot/.../profile-vivaldi`
+etc.) so switching never causes prefs corruption.
 
 ## Installation
 
@@ -74,6 +86,39 @@ npx cdpilot launch    # Start browser with CDP enabled
 npx cdpilot status    # Check connection
 ```
 
+### Upgrading from 0.4.x → 0.5.0 — read this first
+
+**One breaking change**, the rest is additive.
+
+**Breaking — Visual feedback default flipped to OFF.** The green glow border,
+animated fake cursor, click ripples, and keystroke display made cdpilot feel
+like an amateur typing on every page. They are now opt-in:
+
+```bash
+cdpilot show on     # restore the old visual feedback layer
+cdpilot show off    # default since 0.5.0
+```
+
+The MCP server's persistent-glow flow (`CDPILOT_MCP_SESSION=1`) is **unchanged**
+— AI agents that rely on visible feedback during a session still see it
+automatically. Only direct-CLI users see the difference.
+
+**New in 0.5.0** (no migration needed):
+
+- `cdpilot dismiss` — heuristic auto-click for "Stay signed out / No thanks"
+  buttons on LLM chat sign-up walls.
+- `cdpilot adaptive on` — auto-escalate to stealth on CAPTCHA-protected
+  hosts, with persistent per-host memory.
+- `cdpilot cookies save/load` — export/import cookies as JSON to replay
+  CF/DataDome clearance across runs.
+- `cdpilot context create/list/close` + `CDPILOT_TARGET` — isolated browser
+  contexts for true parallel automation inside a single browser.
+- `cdpilot fast` / `cdpilot show` — bundled timing + visual toggles.
+- Pure performance: post-load sleep 1500ms → 300ms, `scrollIntoView` instant,
+  WebSocket connection pool, `/json` TTL cache.
+
+See [CHANGELOG.md](CHANGELOG.md) for the full list with rationale.
+
 ## Commands
 
 ### Navigation & Content
@@ -107,8 +152,50 @@ cdpilot network [url]         # Monitor network requests
 cdpilot debug [url]           # Full diagnostic (console+network+perf+shot)
 cdpilot perf                  # Performance metrics
 cdpilot eval <js>             # Execute JavaScript
+cdpilot eval-batch <json>     # Run N JS expressions in 1 roundtrip (5-30x faster)
+```
+
+```bash
+# Example: read 4 DOM values in a single CDP roundtrip instead of 4
+cdpilot eval-batch '["document.title","location.href","document.links.length","document.images.length"]'
+# → [{"ok":true,"value":"..."}, {"ok":true,"value":"..."}, ...]
+```
+
+### Performance
+
+```bash
+cdpilot block                                # Show status
+cdpilot block on                             # Enable (default preset: images+fonts+ads)
+cdpilot block off                            # Disable
+cdpilot block preset images,fonts,ads,media  # Set patterns from named presets
+cdpilot block patterns '*.png' '*.woff2'     # Custom URL patterns
+cdpilot block clear                          # Drop all patterns
 ```
 
+> **Stealth caveat:** `block` changes the fingerprint surface — real browsers fetch
+> images, fonts, and analytics. Cloudflare-class bot detectors notice missing
+> requests. Keep `block` **off** for stealth/anti-bot targets; turn it **on** for
+> known-safe internal sites where speed matters more than blending in.
+
+```bash
+cdpilot fast                       # Show status (effective auto-wait ms)
+cdpilot fast on                    # Auto-wait 5s → 2s, less idle padding
+cdpilot fast off                   # Back to defaults
+CDPILOT_WAIT_MS=1000 cdpilot click # Per-command override (env wins over fast mode)
+```
+
+```bash
+cdpilot show                       # Show status (visual feedback on/off)
+cdpilot show on                    # Re-enable glow border + cursor + ripples + keystrokes
+cdpilot show off                   # Default since 0.4.4 — quiet, professional output
+```
+
+> **Visual feedback default changed in 0.4.4** — the old animations (green glow,
+> moving cursor, click ripples) used to make every action look like an amateur
+> driving the screen. They're now opt-in via `cdpilot show on`. The MCP server's
+> persistent-glow flow (`CDPILOT_MCP_SESSION=1`) is unaffected — AI agents that
+> rely on visible feedback during a session still see it automatically.
+
 ### Tab Management
 
 ```bash
@@ -177,7 +264,10 @@ cdpilot session-close [id]    # Close session
 ### Advanced
 
 ```bash
-cdpilot cookies [domain]      # List cookies
+cdpilot cookies [domain]      # List cookies (filter by domain)
+cdpilot cookies save <file> [<domain>]
+                              # Export cookies as JSON (optional domain filter)
+cdpilot cookies load <file>   # Import cookies (replay CF clearance across runs)
 cdpilot storage               # localStorage contents
 cdpilot upload <sel> <file>   # Upload file to input
 cdpilot multi-eval <js>       # Execute JS in all tabs
@@ -187,6 +277,89 @@ cdpilot dialog auto-accept    # Auto-accept dialogs
 cdpilot permission grant geo  # Grant geolocation
 ```
 
+### Parallel Contexts
+
+```bash
+cdpilot context create [url]  # Make fresh browser context + tab (prints JSON)
+cdpilot context list          # Tree of contexts and their tabs
+cdpilot context close <ctx>   # Destroy a context (refuses 'default')
+```
+
+Address a specific context's tab in subsequent commands via the env pin:
+
+```bash
+ID=$(cdpilot context create https://example.com | jq -r .target_id)
+CDPILOT_TARGET=$ID cdpilot eval 'document.title'
+```
+
+> True isolation — each context has its own cookie/storage jar. Designed for
+> running N AI chat queries in parallel without history pollution, or A/B
+> testing logged-in vs logged-out flows without spinning up multiple browsers.
+
+### Smart Navigation (LLM-aware)
+
+```bash
+cdpilot dismiss               # Click best "Stay signed out / No thanks" button
+cdpilot dismiss aggressive    # Handle chained modals (cookie banner → signup)
+```
+
+> Built-in English + Turkish pattern library. Explicitly excludes destructive
+> lookalikes (Delete account, Sign out, Subscribe) — safe to chain into a
+> query workflow.
+
+### Stealth & CAPTCHA
+
+Zero-dependency anti-fingerprint layer — patches `navigator.webdriver`,
+`chrome.runtime`, plugins (proper `PluginArray` inheritance), WebGL
+vendor/renderer, permissions, hardware concurrency, and the `Worker`
+constructor. Injected via `Page.addScriptToEvaluateOnNewDocument` before
+any page script runs. Disabled by default; opt-in.
+
+```bash
+cdpilot stealth on            # enable fingerprint patches (opt-in)
+cdpilot stealth off           # disable (default)
+cdpilot stealth status        # show which patches are applied
+
+cdpilot captcha-check         # JSON detection of Turnstile/hCaptcha/reCAPTCHA/
+                              # DataDome/PerimeterX/Arkose/GeeTest. Exit 0/3
+cdpilot captcha-wait [sec]    # block until user solves (interactive)
+                              # or poll with JSON stream (non-interactive)
+
+cdpilot adaptive [on|off|status]
+                              # Auto-escalate to stealth on hosts that show
+                              # CAPTCHA. Persistent per-host memory.
+cdpilot adaptive forget <host>
+                              # Remove a hostname from the stealth list
+cdpilot adaptive clear        # Drop the stealth host memory entirely
+```
+
+> **Adaptive mode** is the "run fast, climb walls when seen" automation: cdpilot
+> runs in the open lane by default, detects CAPTCHA after each navigation, and
+> when it sees one — adds the host to a persistent list, retries once with
+> stealth on. Never auto-demotes. Conservative by design.
+
+Verified against public bot-detection panels:
+- **bot.sannysoft.com:** 24/24 PASS (WebDriver, Chrome obj, Plugins as PluginArray, WebGL, PHANTOM_*, HEADCHR_*, SELENIUM_DRIVER)
+- **bot.incolumitas.com** intoli: 6/6 OK — new-tests: 6/7 OK (one FAIL = pure CDP presence, cannot be JS-patched)
+- **nowsecure.nl** (Cloudflare full challenge): passed
+- **arh.antoinevastel.com/areyouheadless:** "You are not Chrome headless"
+
+### Reliability
+
+```bash
+cdpilot browser [name|auto]   # workload-aware browser selection
+cdpilot health                # JSON: alive, port, tabs, browser, today's crashes
+```
+
+`cdpilot health` is designed for shell watchdogs:
+
+```bash
+until cdpilot health >/dev/null; do cdpilot launch; sleep 2; done
+```
+
+Surfaces today's Brave crash count from `~/Library/Logs/DiagnosticReports/`
+on macOS — spot degradation before your automation silently stalls.
+
 ## Use with AI Agents
 
 cdpilot is designed to be called by AI agents as a tool:
@@ -294,6 +467,15 @@ The only browser MCP with built-in test assertions. Here's what we've shipped an
 - [x] **Vision fallback** (`describe`) — a11y + screenshot + text in one call
 - [x] **Annotated screenshots** — @N badge overlays on interactive elements
 - [x] **Auto-wait** — MutationObserver-based, 5s automatic element waiting
+- [x] **`wait-for-text`** — adaptive text-based waiting (subtree + characterData) for streaming AI responses, async toasts, and selector-less synchronization
+- [x] **`eval-batch`** — run N JS expressions in 1 CDP roundtrip (5-30x speedup vs sequential `eval`)
+- [x] **`block`** — request blocking via `Network.setBlockedURLs` with built-in presets (images/fonts/ads/media), 3-10x faster page loads on opt-in
+- [x] **`dismiss`** — heuristic auto-click for LLM chat sign-up walls (EN+TR pattern library, destructive-action guards)
+- [x] **`adaptive`** — auto-escalate to stealth on CAPTCHA-protected hosts, persistent per-host memory ("run fast, climb walls")
+- [x] **`cookies save/load`** — export/import cookies as JSON (replay CF/DataDome clearance across runs)
+- [x] **`context` pool + `CDPILOT_TARGET`** — isolated browser contexts for true parallel automation in a single browser (Playwright's parallel-tabs model)
+- [x] **`fast` / `show`** — bundled timing + visual toggles. Default quiet/fast in 0.5.0
+- [x] **WebSocket pool + `/json` TTL cache** — zero-regression connection reuse for MCP/batch workloads
 - [x] **Batch commands** — pipe JSON arrays via stdin for multi-step automation
 - [x] Visual feedback system (persistent green glow, cursor, ripples, keystroke display)
 - [x] AI control warning toast (red warning when user interacts during automation)

package/bin/cdpilot.js

@@ -318,16 +318,45 @@ function showHelp() {
     network [url]      Monitor network requests
     debug [url]        Full diagnostic
     eval <js>          Execute JavaScript
+    eval-batch <json>  Run N JS expressions in 1 roundtrip (perf)
+
+  PERFORMANCE
+    block [on|off|preset|patterns|clear]
+                       Block requests via Network.setBlockedURLs (perf opt-in,
+                       breaks fingerprint plausibility — not for stealth targets)
+    fast [on|off]      Fast mode — auto-wait 5s→2s (env CDPILOT_WAIT_MS overrides)
+    show [on|off]      Visual feedback (glow + cursor + ripples).
+                       Default OFF since 0.4.4 — opt-in for "see automation" mode.
+
+  SMART NAVIGATION
+    dismiss [N|aggressive]
+                       Click best "Stay signed out / No thanks / Skip" button.
+                       English + Turkish patterns; never clicks destructive
+                       lookalikes (Delete account, Sign out, Subscribe).
+                       Pass N (1-10) or "aggressive" for chained modals.
 
   TABS
     tabs               List open tabs
     new-tab [url]      Open new tab
     close-tab [id]     Close tab
 
+  PARALLEL CONTEXTS (isolated cookies/storage inside one browser)
+    context create [url]     Make a fresh browser context + tab; prints JSON
+    context list             List all browser contexts and their tabs
+    context close <ctx-id>   Destroy a browser context (closes all its tabs)
+    (Address a context's tab in subsequent commands via CDPILOT_TARGET=<tgt-id>)
+
   STEALTH & CAPTCHA
     stealth [on|off]   Toggle anti-fingerprint patches (opt-in)
     captcha-check      Detect CAPTCHA on active page (JSON output)
     captcha-wait [s]   Pause until user solves CAPTCHA (default 300s)
+    adaptive [on|off]  Auto-escalate to stealth on hosts that show CAPTCHA.
+                       Remembers per-host. Use 'adaptive forget <host>' to reset.
+    cookies save <file> [<dom>]
+                       Export cookies (all or scoped). Replay clearance cookies
+                       across cdpilot runs to skip Cloudflare walls.
+    cookies load <file>
+                       Import previously-saved cookies into the current jar.
 
   RELIABILITY
     browser [name]     Show or set preferred browser (chrome|brave|chromium|edge|vivaldi|auto)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cdpilot",
-  "version": "0.4.3",
+  "version": "0.5.0",
   "description": "Zero-dependency browser automation from your terminal. One command, full control.",
   "bin": {
     "cdpilot": "./bin/cdpilot.js",