cdpilot 0.8.0 → 0.9.0

package/README.md

@@ -29,7 +29,7 @@ AI agents and developers need browser control that **just works**:
 
 - **Zero config** — `npx cdpilot launch` starts an isolated browser session
 - **Zero dependency** — No Puppeteer, no Playwright, no Selenium. Pure CDP over HTTP
-- **40+ commands** — Navigate, click, type, screenshot, network, console, accessibility, and more
+- **70+ commands** — Navigate, click, type, screenshot, network, console, accessibility, video understanding, progressive anti-bot resilience, and more
 - **AI-agent friendly** — Designed for Claude, GPT, Gemini, and any LLM tool-use workflow
 - **Isolated sessions** — Your personal browser stays untouched. cdpilot runs in its own profile
 - **Visual feedback** — Green glow overlay, cursor visualization, click ripples, and keystroke display keep you informed during automation
@@ -343,6 +343,42 @@ cdpilot dismiss aggressive    # Handle chained modals (cookie banner → signup)
 > lookalikes (Delete account, Sign out, Subscribe) — safe to chain into a
 > query workflow.
 
+### Video Understanding
+
+A single screenshot can't see motion. `cdpilot watch` runs a continuous
+screencast (`Page.startScreencast`) into a ring buffer of JPEG frames, so an
+AI agent can query a time window and actually *watch* what happened —
+animations, mouse cursor movement, scroll, an explosion effect — instead of
+guessing from one still frame.
+
+```bash
+cdpilot watch start <url|file://...>   # Begin screencast, play the video
+cdpilot watch query --at 1:23 --window 5s  # Frames around a timestamp
+cdpilot watch ask "did the menu animate open?"  # Ask about recent frames
+cdpilot watch status          # Show capture state + buffer size
+cdpilot watch stop            # Stop the screencast
+```
+
+Works on both local files (`file://...`) and online video (YouTube, Vimeo,
+Twitter, Facebook, Instagram). Zero dependency — Pillow is optional and only
+used for motion-detection between frames.
+
+> **DRM limitation:** DRM-protected players (Netflix and similar) render as
+> black frames at the CDP layer — cdpilot cannot capture them. Everything
+> non-DRM works.
+
+MCP exposes this as `browser_watch_*` tools for AI agents.
+
+### Video Understanding (commands)
+
+```bash
+cdpilot watch start <url|file://>     # Start screencast, play video
+cdpilot watch query --at 1:23 --window 5s  # Frames around a timestamp
+cdpilot watch ask "did the modal slide in?"  # Ask about recent frames
+cdpilot watch status                   # Capture state + buffer size
+cdpilot watch stop                     # Stop screencast
+```
+
 ### Stealth & CAPTCHA
 
 Zero-dependency anti-fingerprint layer — patches `navigator.webdriver`,
@@ -351,6 +387,30 @@ vendor/renderer, permissions, hardware concurrency, and the `Worker`
 constructor. Injected via `Page.addScriptToEvaluateOnNewDocument` before
 any page script runs. Disabled by default; opt-in.
 
+At launch, cdpilot also passes `--disable-blink-features=AutomationControlled`,
+which closes the Blink runtime flag that Cloudflare and DataDome probe to detect
+an automated browser.
+
+#### Three-tier stealth mode
+
+`cdpilot mode` is the recommended entry point — one switch that sets how much
+fingerprint surface cdpilot touches, lightest to heaviest:
+
+```bash
+cdpilot mode             # show current tier + what it injects
+cdpilot mode regular     # no fingerprint patch — cleanest, fastest (default)
+cdpilot mode stealth     # light patch: webdriver / chrome.runtime / permissions
+cdpilot mode undetected  # full patch: + plugin array + WebGL + Worker
+```
+
+`regular` is the default because Stealth Bench V1 found the full patch set
+*alone* lowered scores — a synthetic plugin array is itself a tell. The
+`stealth` tier deliberately omits plugin spoofing; escalate to `undetected`
+only for hard targets. The adaptive layer learns the right tier per host and
+escalates on CAPTCHA. Effect applies on the next navigation. Env override:
+`CDPILOT_MODE=<tier>`. The legacy `stealth on/off` toggle still works and stays
+coherent with the tier:
+
 ```bash
 cdpilot stealth on            # enable fingerprint patches (opt-in)
 cdpilot stealth off           # disable (default)
@@ -374,6 +434,47 @@ cdpilot adaptive clear        # Drop the stealth host memory entirely
 > when it sees one — adds the host to a persistent list, retries once with
 > stealth on. Never auto-demotes. Conservative by design.
 
+### Friction Ladder (progressive anti-bot detection)
+
+Real sites don't just throw a CAPTCHA — they stack defenses incrementally.
+`cdpilot friction` reports which rung is currently active so an agent can react
+appropriately instead of guessing. Six levels, lowest to highest:
+
+```
+none → rate_limited → soft_captcha → login_wall → otp_sms → hard_block
+```
+
+```bash
+cdpilot friction              # JSON: current rung + recommended response policy
+```
+
+Bilingual (English + Turkish) DOM heuristics. The detection is **read-only — it
+never bypasses anything.** The response policy is deliberately conservative:
+
+- `rate_limited` → automatic exponential backoff + retry
+- `soft_captcha` → defer to the captcha tools
+- `login_wall` / `otp_sms` / `hard_block` → flagged for **human handoff, not
+  autonomously solved**
+
+That last line is an ethics boundary, not a missing feature: cdpilot will not
+attempt to defeat a login, an OTP/SMS gate, or an outright block on its own.
+MCP exposes this as `browser_friction`.
+
+### Press-and-Hold (PerimeterX / HUMAN behavioral challenge)
+
+PerimeterX's "Press & Hold" is a *behavioral* challenge, not a token — there's
+no provider to call. The only solution is a real press → hold → release
+gesture, which cdpilot emits via the CDP Input domain: a Gaussian-randomized
+~3–7s hold with ±1–2px micro-jitter while the button is held.
+
+```bash
+cdpilot press-hold                       # auto-find the px-captcha target
+cdpilot press-hold "#px-captcha button"  # explicit selector
+```
+
+`captcha-solve` auto-routes here when it detects a `perimeterx` challenge.
+MCP exposes this as `browser_press_hold`.
+
 ### Captcha Solver Plugins (v0.6+)
 
 Optional integration with 2captcha, anti-captcha, and capmonster. Per-solve
@@ -409,6 +510,33 @@ after each navigation. Without auto-on, detection still works but solving is man
 
 > **Expected bench improvement (v0.6):** reCaptcha 2/6 → 5+/6, hCaptcha 2/3 → 3/3
 
+#### Image CAPTCHA + profile warming
+
+`captcha-solve` handles the image-based rate-limit CAPTCHAs that the token
+solvers above don't cover:
+
+```bash
+cdpilot captcha-solve                       # auto-detect + route (incl. press-hold)
+cdpilot captcha-solve --provider amazon-local  # offline OCR (default)
+cdpilot captcha-solve --provider capsolver   # BYOK image-to-text
+cdpilot captcha-solve --provider 2captcha    # BYOK image-to-text
+```
+
+- **Amazon classic image CAPTCHA** (the "Type the characters you see" page) is
+  OCR'd offline via the **optional** `amazoncaptcha` library
+  (`pip install amazoncaptcha` — pure-Python + Pillow, MIT). Not installed = the
+  command reports it and exits cleanly; no hard dependency added.
+- **BYOK providers** (`capsolver`, `2captcha`) use their image-to-text APIs via
+  `CAPSOLVER_API_KEY` / `TWOCAPTCHA_API_KEY`.
+
+```bash
+cdpilot profile warm          # age the profile for reCAPTCHA v3 score
+```
+
+`profile warm` browses a set of low-risk sites to build cookie/history age,
+which nudges reCAPTCHA v3's behavioral score upward over time. Slow by
+design — run it ahead of a session, not inline.
+
 Verified against public bot-detection panels:
 - **bot.sannysoft.com:** 24/24 PASS (WebDriver, Chrome obj, Plugins as PluginArray, WebGL, PHANTOM_*, HEADCHR_*, SELENIUM_DRIVER)
 - **bot.incolumitas.com** intoli: 6/6 OK — new-tests: 6/7 OK (one FAIL = pure CDP presence, cannot be JS-patched)
@@ -431,6 +559,24 @@ until cdpilot health >/dev/null; do cdpilot launch; sleep 2; done
 Surfaces today's Brave crash count from `~/Library/Logs/DiagnosticReports/`
 on macOS — spot degradation before your automation silently stalls.
 
+### Scaling & Workstation Use
+
+```bash
+CDPILOT_POOL_SIZE=4 cdpilot launch   # 4 separate browser processes
+CDPILOT_OFFSCREEN=1 cdpilot launch   # headed, but no window on your screen
+```
+
+- **Multi-instance pool** (`CDPILOT_POOL_SIZE=N`) starts N independent browser
+  processes and dispatches work to the least-loaded one, for `N × per-instance`
+  parallelism. Default is `1` — single instance, no change for existing users.
+- **Off-screen mode** (`CDPILOT_OFFSCREEN=1`) keeps the browser headed (real
+  rendering, no headless fingerprint) but positions the window where it can't
+  steal focus — meant for automating on a workstation you're also using.
+- **Docker + Xvfb harness** (`cdpilot-bench/docker/`) runs headed-in-Xvfb so
+  bench/automation runs never pop a window on the host display. CI-ready. Note
+  that software rendering (no GPU) lowers anti-bot scores versus native — it's
+  an isolated environment for reproducibility, not the headline configuration.
+
 ## Use with AI Agents
 
 cdpilot is designed to be called by AI agents as a tool:
@@ -476,6 +622,9 @@ print(result.stdout)
 | `CHROME_BIN` | Auto-detect | Browser binary path |
 | `CDPILOT_PROFILE` | `~/.cdpilot/profile` | Isolated browser profile |
 | `BROWSER_SESSION` | Auto | Session identifier |
+| `CDPILOT_MODE` | `regular` | Stealth tier override (`regular`/`stealth`/`undetected`) |
+| `CDPILOT_OFFSCREEN` | `0` | Headed but render off-screen — no window steals focus |
+| `CDPILOT_POOL_SIZE` | `1` | N separate browser processes, least-loaded dispatch |
 
 ## How It Works
 
@@ -601,7 +750,7 @@ The only browser MCP with built-in test assertions. Here's what we've shipped an
 
 ### Shipped
 
-- [x] 60+ CLI commands (navigate, click, fill, screenshot, PDF, console, network...)
+- [x] 70+ CLI commands (navigate, click, fill, screenshot, PDF, console, network, video understanding, friction ladder...)
 - [x] MCP server for AI agent integration (Claude Code, Cursor, etc.)
 - [x] **10 built-in test assertions** — assert, assert-url, assert-title, assert-count, assert-value, assert-attr, assert-visible/hidden, wait-for, check (batch), screenshot-diff
 - [x] **Accessibility tree snapshot** (`a11y-snapshot`) — structured data with @ref references, 500x fewer tokens than screenshots
@@ -625,14 +774,20 @@ The only browser MCP with built-in test assertions. Here's what we've shipped an
 - [x] Pre-flight wizard (auto-installs dependencies on first run)
 - [x] Persistent MCP glow (stays on during entire AI session, like Claude's orange glow)
 - [x] DevExtension system (native JS injection without browser store)
-- [x] **Smart commands** — `smart-click`, `smart-fill`, `smart-select` — interact by visible text, no CSS selectors needed, no LLM required
+- [x] **Smart commands** — `smart-click`, `smart-fill`, `smart-select` — interact by visible text, no CSS selectors needed, no LLM required. Now with a disabled-element guard (no more false "clicked" on disabled buttons), Shadow DOM traversal (Lightning, Polymer, lit-element widgets), locale-aware text matching (Turkish İ/i, German ß), and floating-label support for `smart-fill` (Material / Ant / Chakra via `aria-labelledby` and `closest` label resolution)
+- [x] **Video understanding** (`watch`) — continuous screencast into a ring buffer so an AI agent can query a time window and see motion (animation, cursor, scroll), not just one still frame. Local `file://` and online video (YouTube/Vimeo/Twitter/etc.); DRM players (Netflix) excluded
+- [x] **Friction ladder** (`friction`) — 6-level progressive anti-bot detection (none → rate_limited → soft_captcha → login_wall → otp_sms → hard_block), bilingual EN+TR, read-only; rate-limit auto-backoff, login/OTP/hard-block flagged for human handoff (no autonomous bypass)
+- [x] **Three-tier stealth mode** (`mode regular|stealth|undetected`) — single switch over fingerprint surface, with per-host adaptive tier learning
+- [x] **Press-and-hold solver** (`press-hold`) — humanized press → hold → release gesture (Gaussian 3–7s + micro-jitter) for PerimeterX/HUMAN behavioral challenges; `captcha-solve` auto-routes here
+- [x] **Image CAPTCHA + profile warming** — offline Amazon image OCR (optional `amazoncaptcha` lib) + BYOK image-to-text (capsolver/2captcha); `profile warm` ages the profile for reCAPTCHA v3 score
+- [x] **Multi-instance pool + off-screen mode** (`CDPILOT_POOL_SIZE`, `CDPILOT_OFFSCREEN`) — N parallel browser processes; headed rendering without stealing window focus
 - [x] **Data extraction** (`extract`) — structured DOM data in text, JSON, or list format
 - [x] **Page observation** (`observe`) — list all interactive elements with available actions
 - [x] **Script runner** (`run`) — execute `.cdp` script files with pass/fail reporting
 
 ### Coming Soon
 
-- [ ] **iframe & Shadow DOM** support — interact with elements inside iframes and shadow roots
+- [ ] **iframe** support — interact with elements inside iframes (Shadow DOM traversal already shipped in smart commands)
 - [ ] **Session recording & replay** — record browser sessions and replay them deterministically
 - [ ] **Stealth mode** *(Pro)* — human-like mouse/typing, anti-fingerprint, CAPTCHA solving
 - [ ] **cdpilot Cloud** — hosted browser sessions API, REST + WebSocket MCP endpoint

package/bin/cdpilot.js

@@ -69,11 +69,20 @@ function findBrowser() {
 // ── Python Detection ──
 
 function findPython() {
-  for (const cmd of ['python3', 'python']) {
+  // cdpilot.py uses PEP 604 unions (dict | None) → requires Python 3.10+.
+  // Prefer version-specific + Homebrew interpreters so a stale default python3
+  // (e.g. macOS 3.9) is skipped instead of selected and crashing at runtime.
+  const candidates = [
+    process.env.CDPILOT_PYTHON,
+    'python3.13', 'python3.12', 'python3.11', 'python3.10',
+    '/opt/homebrew/bin/python3', '/usr/local/bin/python3',
+    'python3', 'python',
+  ].filter(Boolean);
+  for (const cmd of candidates) {
     try {
       const ver = execSync(`${cmd} --version 2>&1`, { stdio: 'pipe' }).toString().trim();
       const match = ver.match(/(\d+)\.(\d+)/);
-      if (match && parseInt(match[1]) >= 3 && parseInt(match[2]) >= 8) {
+      if (match && (parseInt(match[1]) > 3 || (parseInt(match[1]) === 3 && parseInt(match[2]) >= 10))) {
         return cmd;
       }
     } catch {}
@@ -81,6 +90,21 @@ function findPython() {
   return null;
 }
 
+// ── Python websockets Detection ──
+
+function findWebsockets() {
+  // Import via the interpreter cdpilot actually selects — pip metadata can
+  // point at a different Python than findPython() resolves.
+  const python = findPython();
+  if (!python) return null;
+  try {
+    const out = execSync(`${python} -c "import websockets; print(getattr(websockets, '__version__', 'installed'))"`, { stdio: 'pipe' }).toString().trim();
+    return out || 'installed';
+  } catch {
+    return null;
+  }
+}
+
 // ── Setup Command ──
 
 function runSetup() {
@@ -88,11 +112,12 @@ function runSetup() {
   const config = resolveProjectConfig();
 
   console.log('\n  cdpilot setup\n');
-  console.log(`  Browser:  ${browser || '❌ Not found'}`);
-  console.log(`  Profile:  ${config.profileDir}`);
-  console.log(`  CDP Port: ${config.port === '0' ? 'auto' : config.port}`);
-  console.log(`  Project:  ${config.projectId || 'manual mode'}`);
-  console.log(`  Python:   ${findPython() || '❌ Not found'}`);
+  console.log(`  Browser:    ${browser || '❌ Not found'}`);
+  console.log(`  Profile:    ${config.profileDir}`);
+  console.log(`  CDP Port:   ${config.port === '0' ? 'auto' : config.port}`);
+  console.log(`  Project:    ${config.projectId || 'manual mode'}`);
+  console.log(`  Python:     ${findPython() || '❌ Not found'}`);
+  console.log(`  websockets: ${findWebsockets() || '❌ Not found'}`);
 
   if (!browser) {
     console.log('\n  ❌ No compatible browser found.');
@@ -102,11 +127,17 @@ function runSetup() {
   }
 
   if (!findPython()) {
-    console.log('\n  ❌ Python 3.8+ not found.');
+    console.log('\n  ❌ Python 3.10+ not found.');
     console.log('  Install: https://www.python.org/downloads/\n');
     process.exit(1);
   }
 
+  if (!findWebsockets()) {
+    console.log('\n  ❌ Python websockets not found.');
+    console.log('  Install: pip install websockets\n');
+    process.exit(1);
+  }
+
   // Create profile directory
   if (!fs.existsSync(config.profileDir)) {
     fs.mkdirSync(config.profileDir, { recursive: true });
@@ -147,7 +178,7 @@ function preflight() {
     const ver = execSync(`${python} --version 2>&1`, { stdio: 'pipe' }).toString().trim();
     console.log(`  ✓ ${ver}`);
   } else {
-    console.log('  ✗ Python 3.8+ not found');
+    console.log('  ✗ Python 3.10+ not found');
     console.log('    → Install: https://www.python.org/downloads/\n');
     process.exit(1);
   }
@@ -296,7 +327,9 @@ function showHelp() {
     setup              Auto-detect browser, create isolated profile
     launch             Start browser with CDP enabled
     status             Check browser connection
-    stop               Stop browser
+    stop [--smart]     Stop browser (--smart = close owned tabs, quit if empty)
+    close [--force|--keep]  Smart close: close cdpilot's tabs; quit browser only
+                       if no user tabs remain (--force quits anyway, --keep never quits)
 
   NAVIGATION
     go <url>           Navigate to URL
@@ -347,9 +380,29 @@ function showHelp() {
     (Address a context's tab in subsequent commands via CDPILOT_TARGET=<tgt-id>)
 
   STEALTH & CAPTCHA
-    stealth [on|off]   Toggle anti-fingerprint patches (opt-in)
+    mode [regular|stealth|undetected]
+                       Three-tier stealth (crawl4ai-style). regular = no patch
+                       (cleanest, default); stealth = light patch (webdriver/
+                       chrome.runtime/permissions); undetected = full patch
+                       (+ plugins + WebGL + Worker). Adaptive auto-escalates.
+    stealth [on|off]   Legacy binary toggle (on -> undetected tier)
     captcha-check      Detect CAPTCHA on active page (JSON output)
     captcha-wait [s]   Pause until user solves CAPTCHA (default 300s)
+    captcha-solve [--provider P]
+                       Solve Amazon classic image CAPTCHA (opt-in). amazon-local
+                       (optional amazoncaptcha lib) or BYOK capsolver/2captcha.
+                       Auto-routes PerimeterX 'Press & Hold' to press-hold.
+    press-hold [selector]
+                       Solve a PerimeterX/HUMAN 'Press & Hold' challenge with a
+                       humanized press->hold(jitter)->release gesture (no token,
+                       no provider). Auto-finds #px-captcha if no selector given.
+    friction           Detect highest anti-bot rung (none/rate_limited/
+                       soft_captcha/login_wall/otp_sms/hard_block) + policy.
+                       rate_limit auto-backoff in 'go'; login/OTP/block = human
+                       handoff (no autonomous bypass). Env: CDPILOT_FRICTION_BACKOFF,
+                       CDPILOT_FRICTION_MAX_RETRY.
+    profile warm [--minutes N]
+                       Age cookies/history on safe sites to boost reCAPTCHA v3 score.
     adaptive [on|off]  Auto-escalate to stealth on hosts that show CAPTCHA.
                        Remembers per-host. Use 'adaptive forget <host>' to reset.
     cookies save <file> [<dom>]
@@ -370,6 +423,18 @@ function showHelp() {
   AI AGENT
     mcp                Start MCP server (stdin/stdout JSON-RPC)
 
+  WATCH (continuous screencast for AI video understanding)
+    watch start <url>  Begin JPEG screencast at N fps to a disk ring buffer
+                       (default 10fps, 5min retention, 100MB cap). Background
+                       daemon — command returns immediately.
+    watch query --at MM:SS --window 5s
+                       Return JSON list of frame paths around a video time.
+    watch query --last 5s | --since-last
+                       Recent frames or everything new since the last query.
+    watch status       Daemon state, frame count, disk usage.
+    watch stop         Stop daemon + clean up frames (--keep-frames to retain).
+    watch ask "<q>"    Tiny NL parser: extracts time window from a question.
+
   More: https://github.com/mehmetnadir/cdpilot#commands
 `);
 }
@@ -402,7 +467,7 @@ function runInternalTestRunner(testFile, traceDir, traceMode, grepPattern) {
   const results = { passed: 0, failed: 0, skipped: 0, tests: [] };
   const cdpPort = process.env.CDP_PORT || '9222';
   const SCRIPT = path.join(__dirname, '..', 'src', 'cdpilot.py');
-  const python = 'python3';
+  const python = findPython() || 'python3';
   let stepIdx = 0;
 
   const makeT = () => {

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "cdpilot",
-  "version": "0.8.0",
-  "description": "Zero-dependency browser automation CLI. Raw CDP, 60+ commands, stealth + adaptive escalation, CAPTCHA solvers, proxy pools, TLS probe, MCP server. No Puppeteer/Playwright/Selenium.",
+  "version": "0.9.0",
+  "description": "Zero-dependency browser automation CLI. Raw CDP, 70+ commands, friction ladder, 3-tier stealth, CAPTCHA + press-and-hold solvers, proxy pools, MCP server. No Puppeteer/Playwright/Selenium.",
   "bin": {
     "cdpilot": "./bin/cdpilot.js",
     "bctl": "./bin/cdpilot.js"
@@ -42,7 +42,14 @@
     "adaptive-escalation",
     "proxy-rotation",
     "tls-fingerprint",
-    "cookie-management"
+    "cookie-management",
+    "video-understanding",
+    "screencast",
+    "shadow-dom",
+    "perimeterx",
+    "press-and-hold",
+    "progressive-resilience",
+    "multi-instance"
   ],
   "author": "Mehmet Nadir",
   "license": "MIT",