openchrome-mcp 1.8.11 → 1.8.13

package/README.md

@@ -5,8 +5,8 @@
 <h1 align="center">OpenChrome</h1>
 
 <p align="center">
-  <b>Smart. Fast. Parallel.</b><br>
-  Browser automation MCP server that uses your real Chrome.
+  <b>Harness-Engineered Browser Automation</b><br>
+  The MCP server that guides AI agents instead of just exposing APIs.
 </p>
 
 <p align="center">
@@ -32,12 +32,17 @@
 | **RAM (20 parallel)** | **~300 MB** | ~5 GB+ | impractical | impractical |
 | **Bot detection** | **invisible** (real Chrome) | detected (TLS fingerprint) | detected (CDP signals) | detected (local) / cloud only |
 | **Chrome login reuse** | **built-in** | extension mode only | manual | manual state files |
-| **LLM hang prevention** | **hint engine** (17 rules) | none | none | error rewrite (5 patterns) |
-| **Shadow DOM** | **supported** | invisible | invisible | invisible |
+| **LLM hang prevention** | **hint engine** (30+ rules) | none | none | error rewrite (5 patterns) |
+| **Reliability mechanisms** | **49** (8-layer defense) | ~3 | ~3 | ~5 |
+| **Token compression** | **15x** (DOM serializer) | none | none | none |
+| **Outcome classification** | **yes** (DOM delta) | none | none | none |
+| **Cross-session learning** | **yes** (domain memory) | none | none | none |
+| **Circuit breaker** | **3-level** | none | none | none |
+| **Shadow DOM** | **all types** (open + closed) | open only | invisible | invisible |
 | **MCP native** | **yes** | yes | yes | no (CLI only) |
 | **Parallel sessions** | **1 Chrome, N tabs** | N browsers | manual tabs | N daemons |
 
-> **tl;dr** — OpenChrome talks directly to Chrome via CDP with zero middleware, reuses your real login sessions (making bot detection irrelevant), and is the only tool with a hint engine that stops LLM agents from wandering.
+> **tl;dr** — OpenChrome talks directly to Chrome via CDP with zero middleware, reuses your real login sessions, and is the only browser MCP server with **harness engineering** — 27 intelligent subsystems that guide, protect, and optimize the AI agent at every step.
 
 ---
 
@@ -68,7 +73,9 @@ AI:  [8 parallel workers, all sites simultaneously]
 
 ---
 
-## Guided, Not Guessing
+## Harness-Engineered, Not Just Automated
+
+Traditional browser automation exposes raw APIs. When the AI agent fails, it's on its own — burning tokens guessing, retrying, and wandering. **Harness engineering** means the tool itself wraps intelligence around those APIs: preventing mistakes, recovering from errors, and guiding the agent toward efficient behavior.
 
 The bottleneck in browser automation isn't the browser — it's the **LLM thinking between each step**. Every tool call costs 5–15 seconds of inference time. When an AI agent guesses wrong, it doesn't just fail — it spends another 10 seconds thinking about why, then another 10 seconds trying something else.
 
@@ -105,7 +112,7 @@ OpenChrome agent checking prices on 5 sites:
   = ~20 LLM calls,  ~15 seconds,  ~$0.40
 ```
 
-The hint engine watches every tool call across 6 layers — error recovery, composite suggestions, repetition detection, sequence detection, learned patterns, and success guidance. When it sees the same error→recovery pattern 3+ times, it promotes it to a permanent rule across sessions.
+The hint engine watches every tool call across 9 categories — error recovery, blocking page detection, composite suggestions, repetition detection, sequence detection, pagination detection, learned patterns, success guidance, and setup hints. When it sees the same error→recovery pattern 3+ times, it promotes it to a permanent rule across sessions via the Pattern Learner.
 
 | | Playwright | OpenChrome | Savings |
 |---|---|---|---|
@@ -114,6 +121,35 @@ The hint engine watches every tool call across 6 layers — error recovery, comp
 | **Token cost** | ~$2.00 | ~$0.40 | **5x cheaper** |
 | **Wasted calls** | ~95% | ~0% | |
 
+### 27 Harness Features Across 7 Categories
+
+OpenChrome isn't just a browser API — it's an intelligent harness with 27 subsystems that work together:
+
+| Category | Key Features | What It Does |
+|----------|-------------|--------------|
+| **Guidance** | Hint Engine (30+ rules, 9 types), Progress Tracker, Usage Guide | Prevents mistakes before they cascade |
+| **Resilience** | Ralph Engine (7-strategy waterfall), Auto-Reconnect, Ref Self-Healing | Recovers from failures automatically |
+| **Protection** | 3-Level Circuit Breaker, Rate Limiter, Domain Guard | Stops runaway token waste |
+| **Feedback** | Outcome Classifier, DOM Delta, Visual Summary, Hit Detection | Reports what *actually* happened |
+| **Learning** | Pattern Learner, Strategy Learner, Domain Memory | Gets smarter across sessions |
+| **Optimization** | DOM Mode (15x compression), Adaptive Screenshot, Snapshot Delta | Minimizes token consumption |
+| **Detection** | Auth Redirect Detection, Blocking Page, Pagination Detector | Identifies situations early |
+
+<details>
+<summary>Feature highlights</summary>
+
+**Hint Engine** — 30+ rules across 9 categories (error recovery, blocking page detection, repetition loops, pagination, composite suggestions, sequence optimization, learned patterns, success guidance, setup hints). Escalates from `info` → `warning` → `critical` as patterns repeat. The Progress Tracker detects stuck agents within 3-5 tool calls.
+
+**Ralph Engine** — When an interaction fails, Ralph automatically tries 7 strategies in sequence: AX tree click → CSS discovery → CDP coordinate dispatch → JS injection → Keyboard navigation → Raw CDP mouse events → Human-in-the-loop escalation. Each attempt is classified by the Outcome Classifier (SUCCESS / SILENT_CLICK / WRONG_ELEMENT).
+
+**3-Level Circuit Breaker** — Element level (3 failures → skip, 2min reset), Page level (5 distinct failures → suggest reload), Global level (10 failures in 5min → pause all). Prevents agents from burning tokens on permanently broken elements.
+
+**Pattern Learner** — When a hint rule misses, the learner observes the next 3 tool calls. If a different tool succeeds, it records the error→recovery correlation. After 3 occurrences at 60%+ confidence, it promotes the pattern to a permanent rule that fires in future sessions.
+
+**DOM Mode** — Serializes the full DOM into a compact text format: strips SCRIPT/STYLE/SVG, keeps only 18 actionable attributes, deduplicates repetitive siblings, collapses nested wrapper chains. **Benchmarked: ~12K tokens vs ~180K tokens** for the same page (15x compression).
+
+</details>
+
 ---
 
 ## Quick Start
@@ -184,7 +220,7 @@ oc compare prices for "AirPods Pro" across Amazon, eBay, Walmart, Best Buy
 
 ---
 
-## 45 Tools
+## 46 Tools
 
 | Category | Tools |
 |----------|-------|
@@ -196,7 +232,7 @@ oc compare prices for "AirPods Pro" across Amazon, eBay, Walmart, Best Buy
 | **Memory** | `memory_record`, `memory_query`, `memory_validate` |
 
 <details>
-<summary>Full tool list (45)</summary>
+<summary>Full tool list (46)</summary>
 
 `navigate` `interact` `computer` `read_page` `find` `form_input` `fill_form` `javascript_tool` `page_reload` `page_content` `page_pdf` `wait_for` `user_agent` `geolocation` `emulate_device` `network` `selector_query` `xpath_query` `cookies` `storage` `console_capture` `performance_metrics` `request_intercept` `drag_drop` `file_upload` `http_auth` `worker_create` `worker_list` `worker_update` `worker_complete` `worker_delete` `tabs_create` `tabs_context` `tabs_close` `workflow_init` `workflow_status` `workflow_collect` `workflow_collect_partial` `workflow_cleanup` `execute_plan` `batch_execute` `lightweight_scroll` `memory_record` `memory_query` `memory_validate` `oc_stop`
 
@@ -316,7 +352,7 @@ By default, benchmarks run in **stub mode** — measuring protocol correctness a
 
 ## Server / Headless Deployment
 
-OpenChrome works on servers and in CI/CD pipelines without Chrome login. All 45 tools function with unauthenticated Chrome — navigation, scraping, screenshots, form filling, and parallel workflows all work in clean sessions.
+OpenChrome works on servers and in CI/CD pipelines without Chrome login. All 46 tools function with unauthenticated Chrome — navigation, scraping, screenshots, form filling, and parallel workflows all work in clean sessions.
 
 ### Quick start
 
@@ -389,6 +425,66 @@ openchrome serve \
 
 ---
 
+## Under the Hood: 8-Layer Reliability
+
+OpenChrome has **49 distinct reliability mechanisms** across 8 defense layers — ensuring no single failure can hang the MCP server.
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Layer 7: MCP Gateway                                       │
+│  Rate limiter · Tool timeout (120s) · Error recovery hints  │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 6: Session Management                                │
+│  TTL cleanup · Memory pressure · Target reconciliation      │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 5: Request Queue                                     │
+│  Per-session FIFO · Per-item timeout (120s)                 │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 4: Circuit Breaker                                   │
+│  Element (3 fails) · Page (5 fails) · Global (10/5min)     │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 3: CDP Client                                        │
+│  Adaptive heartbeat · Stale target guard · Page defenses    │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 2: Reconnection Engine                               │
+│  Auto-reconnect (5 retries) · Exponential backoff · Cookie  │
+│  restore · Sleep/wake detection                             │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 1: Self-Healing                                      │
+│  Chrome watchdog · Tab health monitor · Event loop monitor  │
+│  Disk monitor · Health endpoint (/health, /metrics)         │
+├─────────────────────────────────────────────────────────────┤
+│  Layer 0: Process Lifecycle                                 │
+│  Graceful shutdown · Orphan cleanup · Atomic file writes    │
+└─────────────────────────────────────────────────────────────┘
+```
+
+**32 configurable timeouts** cover every operation from CDP commands (15s) to tool execution (120s) to Chrome launch (60s). Every timeout is independently tunable via `src/config/defaults.ts`.
+
+## Element Intelligence
+
+Finding elements by natural language instead of CSS selectors:
+
+```
+"Submit button" → normalizeQuery → parseQueryForAX → AX Tree Resolution
+                                                          │
+                                                     match found?
+                                                     /         \
+                                                   yes          no
+                                                    │            │
+                                              [AX result]   CSS Fallback
+                                                             + Shadow DOM
+                                                             + Scoring
+```
+
+- **AX-first**: Uses Chrome's accessibility tree — framework-agnostic across React, Angular, Vue, Web Components
+- **Cascading filter**: 4-level deterministic priority (exact role+name → role+contains → exact name → partial)
+- **3-tier Shadow DOM**: Open roots (JS) + closed roots (CDP) + user-agent roots
+- **Hit detection**: After clicking, reports what was actually hit + nearest interactive element
+- **i18n**: Korean role keywords built-in (`"버튼"` → button, `"링크"` → link, `"드롭다운"` → combobox)
+
+---
+
 ## Development
 
 ```bash

package/dist/cdp/client.d.ts

@@ -238,7 +238,7 @@ export declare class CDPClient {
      * CDP observer attached. CDP is attached only after the settle window expires.
      *
      * @param url      URL to open in the new tab
-     * @param settleMs Milliseconds to wait before attaching CDP (default 5000, range 1000-30000)
+     * @param settleMs Total settle time in ms — used for navigation timeout and post-nav wait (default 8000)
      * @returns        The Puppeteer Page and its targetId
      */
     createTargetStealth(url: string, settleMs?: number): Promise<{

package/dist/cdp/client.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"client.d.ts","sourceRoot":"","sources":["../../src/cdp/client.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAkB,EAAE,OAAO,EAAE,cAAc,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAwC9F,MAAM,WAAW,gBAAgB;IAC/B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,oEAAoE;IACpE,UAAU,CAAC,EAAE,OAAO,CAAC;CACtB;AAED,MAAM,MAAM,eAAe,GAAG,cAAc,GAAG,YAAY,GAAG,WAAW,GAAG,cAAc,CAAC;AAE3F,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,WAAW,GAAG,cAAc,GAAG,cAAc,GAAG,aAAa,GAAG,kBAAkB,CAAC;IACzF,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAkBD,qBAAa,SAAS;IACpB,OAAO,CAAC,OAAO,CAAwB;IACvC,OAAO,CAAC,QAAQ,CAAsC;IACtD,OAAO,CAAC,IAAI,CAAS;IACrB,OAAO,CAAC,oBAAoB,CAAS;IACrC,OAAO,CAAC,gBAAgB,CAAS;IACjC,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,cAAc,CAA+B;IACrD,OAAO,CAAC,eAAe,CAAmC;IAC1D,OAAO,CAAC,cAAc,CAA4C;IAClE,OAAO,CAAC,wBAAwB,CAAmD;IACnF,OAAO,CAAC,iBAAiB,CAAK;IAC9B,OAAO,CAAC,4BAA4B,CAAK;IACzC,OAAO,CAAC,6BAA6B,CAAK;IAC1C,OAAO,CAAC,uBAAuB,CAAS;IACxC,OAAO,CAAC,UAAU,CAAU;IAC5B,OAAO,CAAC,iBAAiB,CAAmE;IAC5F,OAAO,CAAC,eAAe,CAAyE;IAChG,OAAO,CAAC,aAAa,CAAgC;IACrD,OAAO,CAAC,mBAAmB,CAAkD;IAC7E,wFAAwF;IACxF,OAAO,CAAC,cAAc,CAA8B;IACpD,wFAAwF;IACxF,OAAO,CAAC,cAAc,CAAK;IAG3B,OAAO,CAAC,aAAa,CAAsD;IAC3E,OAAO,CAAC,aAAa,CAAK;IAC1B,OAAO,CAAC,kBAAkB,CAA+B;IAGzD,OAAO,CAAC,cAAc,CAAK;IAC3B,OAAO,CAAC,aAAa,CAAgB;IACrC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,gBAAgB,CAAM;IAE9C,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,gBAAgB,CAAU;IAGlD,OAAO,CAAC,YAAY,CAAS;IAC7B,OAAO,CAAC,mBAAmB,CAAK;IAChC,OAAO,CAAC,oBAAoB,CAAK;gBAErB,OAAO,GAAE,gBAAqB;IAU1C;;OAEG;IACH,kBAAkB,IAAI,eAAe;IAIrC;;OAEG;IACH,cAAc,IAAI,OAAO;IAIzB;;;OAGG;IACH,gBAAgB,IAAI,MAAM;IAO1B;;OAEG;IACH,qBAAqB,CAAC,QAAQ,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,GAAG,IAAI;IAIvE;;OAEG;IACH,wBAAwB,CAAC,QAAQ,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,GAAG,IAAI;IAO1E;;OAEG;IACH,0BAA0B,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,IAAI,KAAK,IAAI,GAAG,IAAI;IAInF;;OAEG;IACH,6BAA6B,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,IAAI,KAAK,IAAI,GAAG,IAAI;IAOtF;;OAEG;IACH,OAAO,CAAC,iBAAiB;IAsBzB;;OAEG;IACH,OAAO,CAAC,mBAAmB;IAU3B;;OAEG;IACH,OAAO,CAAC,cAAc;IA4BtB;;OAEG;IACH,OAAO,CAAC,aAAa;IAOrB;;OAEG;IACH,gBAAgB,CAAC,IAAI,EAAE,MAAM,GAAG,QAAQ,GAAG,OAAO,GAAG,UAAU,GAAG,IAAI;IA2BtE;;OAEG;IACH,OAAO,CAAC,6BAA6B;IASrC;;;;;OAKG;IACH,qBAAqB,IAAI,IAAI;IAO7B;;OAEG;IACH,gBAAgB,IAAI,MAAM,GAAG,QAAQ,GAAG,OAAO,GAAG,UAAU;IAI5D;;;OAGG;IACH,YAAY,IAAI,MAAM,GAAG,IAAI;IAI7B;;OAEG;IACH,oBAAoB,IAAI;QACtB,mBAAmB,EAAE,MAAM,CAAC;QAC5B,cAAc,EAAE,MAAM,CAAC;QACvB,gBAAgB,EAAE,MAAM,CAAC;QACzB,aAAa,EAAE,MAAM,CAAC;QACtB,oBAAoB,EAAE,MAAM,CAAC;QAC7B,cAAc,EAAE,MAAM,CAAC;QACvB,YAAY,EAAE,OAAO,CAAC;QACtB,gBAAgB,EAAE,MAAM,CAAC;QACzB,sBAAsB,EAAE,MAAM,CAAC;KAChC;IAoBD;;;;OAIG;YACW,eAAe;IAyD7B;;OAEG;YACW,gBAAgB;IA2H9B;;OAEG;YACW,eAAe;IAyI7B;;;;;OAKG;IACG,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC;IA4D9B;;;;;;;;OAQG;IACG,cAAc,IAAI,OAAO,CAAC,IAAI,CAAC;IAwDrC;;;;OAIG;YACW,4BAA4B;IAqB1C;;OAEG;IACG,UAAU,IAAI,OAAO,CAAC,IAAI,CAAC;IAwBjC;;OAEG;IACH,UAAU,IAAI,OAAO;IAQrB,MAAM,CAAC,QAAQ,CAAC,gBAAgB;;;MAAoB;IAEpD;;;OAGG;IACG,oBAAoB,IAAI,OAAO,CAAC,cAAc,CAAC;IAOrD;;OAEG;IACG,mBAAmB,CAAC,OAAO,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC;IAUjE;;OAEG;IACH,OAAO,CAAC,WAAW;IASnB;;;OAGG;IACH,OAAO,CAAC,gBAAgB;IAqCxB;;;;;;;;OAQG;IACG,6BAA6B,CAAC,YAAY,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IA0BlF;;;;OAIG;YACW,gCAAgC;IA6G9C;;;;OAIG;IACG,iBAAiB,CAAC,cAAc,EAAE,MAAM,EAAE,QAAQ,EAAE,IAAI,GAAG,OAAO,CAAC,MAAM,CAAC;IA6GhF;;;;;OAKG;IACG,UAAU,CAAC,GAAG,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,cAAc,GAAG,IAAI,EAAE,gBAAgB,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC;IAgF1G;;;;;;;;;OASG;IACG,mBAAmB,CAAC,GAAG,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAa,GAAG,OAAO,CAAC;QAAE,IAAI,EAAE,IAAI,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IA+L1G;;;OAGG;IACH,OAAO,CAAC,qBAAqB;IAwK7B;;OAEG;IACG,QAAQ,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;IAKjC;;;;OAIG;IACG,oBAAoB,IAAI,OAAO,CAAC,MAAM,CAAC;IAsB7C;;OAEG;IACG,iBAAiB,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,GAAG,IAAI,CAAC;IA+B/D;;OAEG;IACG,aAAa,CAAC,IAAI,EAAE,IAAI,GAAG,OAAO,CAAC,UAAU,CAAC;IAuBpD;;;OAGG;IACG,IAAI,CAAC,CAAC,GAAG,OAAO,EACpB,IAAI,EAAE,IAAI,EACV,MAAM,EAAE,MAAM,EACd,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAC/B,OAAO,CAAC,CAAC,CAAC;IAoBb;;OAEG;IACH,UAAU,IAAI,MAAM,EAAE;IAItB;;OAEG;IACH,UAAU,CAAC,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS;IAIhD;;OAEG;IACG,SAAS,CAAC,IAAI,EAAE,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;IAS1C;;OAEG;IACG,SAAS,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAShD;;OAEG;IACH,WAAW,IAAI,OAAO;IAItB;;OAEG;IACH,OAAO,IAAI,MAAM;IAIjB;;OAEG;IACH,MAAM,CAAC,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS;CAG1E;AAKD,wBAAgB,YAAY,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS,CAKlE;AAED;;GAEG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,OAAO,CAAqC;IAEpD;;OAEG;IACH,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS;IAShE;;OAEG;IACH,GAAG,CAAC,IAAI,EAAE,MAAM,GAAG,SAAS,GAAG,SAAS;IAIxC;;OAEG;IACH,MAAM,IAAI,SAAS,EAAE;IAIrB;;OAEG;IACG,aAAa,IAAI,OAAO,CAAC,IAAI,CAAC;CASrC;AAKD,wBAAgB,mBAAmB,IAAI,gBAAgB,CAKtD"}
+{"version":3,"file":"client.d.ts","sourceRoot":"","sources":["../../src/cdp/client.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAkB,EAAE,OAAO,EAAE,cAAc,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAyC9F,MAAM,WAAW,gBAAgB;IAC/B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,oEAAoE;IACpE,UAAU,CAAC,EAAE,OAAO,CAAC;CACtB;AAED,MAAM,MAAM,eAAe,GAAG,cAAc,GAAG,YAAY,GAAG,WAAW,GAAG,cAAc,CAAC;AAE3F,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,WAAW,GAAG,cAAc,GAAG,cAAc,GAAG,aAAa,GAAG,kBAAkB,CAAC;IACzF,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAkBD,qBAAa,SAAS;IACpB,OAAO,CAAC,OAAO,CAAwB;IACvC,OAAO,CAAC,QAAQ,CAAsC;IACtD,OAAO,CAAC,IAAI,CAAS;IACrB,OAAO,CAAC,oBAAoB,CAAS;IACrC,OAAO,CAAC,gBAAgB,CAAS;IACjC,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,cAAc,CAA+B;IACrD,OAAO,CAAC,eAAe,CAAmC;IAC1D,OAAO,CAAC,cAAc,CAA4C;IAClE,OAAO,CAAC,wBAAwB,CAAmD;IACnF,OAAO,CAAC,iBAAiB,CAAK;IAC9B,OAAO,CAAC,4BAA4B,CAAK;IACzC,OAAO,CAAC,6BAA6B,CAAK;IAC1C,OAAO,CAAC,uBAAuB,CAAS;IACxC,OAAO,CAAC,UAAU,CAAU;IAC5B,OAAO,CAAC,iBAAiB,CAAmE;IAC5F,OAAO,CAAC,eAAe,CAAyE;IAChG,OAAO,CAAC,aAAa,CAAgC;IACrD,OAAO,CAAC,mBAAmB,CAAkD;IAC7E,wFAAwF;IACxF,OAAO,CAAC,cAAc,CAA8B;IACpD,wFAAwF;IACxF,OAAO,CAAC,cAAc,CAAK;IAG3B,OAAO,CAAC,aAAa,CAAsD;IAC3E,OAAO,CAAC,aAAa,CAAK;IAC1B,OAAO,CAAC,kBAAkB,CAA+B;IAGzD,OAAO,CAAC,cAAc,CAAK;IAC3B,OAAO,CAAC,aAAa,CAAgB;IACrC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,gBAAgB,CAAM;IAE9C,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,gBAAgB,CAAU;IAGlD,OAAO,CAAC,YAAY,CAAS;IAC7B,OAAO,CAAC,mBAAmB,CAAK;IAChC,OAAO,CAAC,oBAAoB,CAAK;gBAErB,OAAO,GAAE,gBAAqB;IAU1C;;OAEG;IACH,kBAAkB,IAAI,eAAe;IAIrC;;OAEG;IACH,cAAc,IAAI,OAAO;IAIzB;;;OAGG;IACH,gBAAgB,IAAI,MAAM;IAO1B;;OAEG;IACH,qBAAqB,CAAC,QAAQ,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,GAAG,IAAI;IAIvE;;OAEG;IACH,wBAAwB,CAAC,QAAQ,EAAE,CAAC,KAAK,EAAE,eAAe,KAAK,IAAI,GAAG,IAAI;IAO1E;;OAEG;IACH,0BAA0B,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,IAAI,KAAK,IAAI,GAAG,IAAI;IAInF;;OAEG;IACH,6BAA6B,CAAC,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,IAAI,KAAK,IAAI,GAAG,IAAI;IAOtF;;OAEG;IACH,OAAO,CAAC,iBAAiB;IAsBzB;;OAEG;IACH,OAAO,CAAC,mBAAmB;IAU3B;;OAEG;IACH,OAAO,CAAC,cAAc;IA4BtB;;OAEG;IACH,OAAO,CAAC,aAAa;IAOrB;;OAEG;IACH,gBAAgB,CAAC,IAAI,EAAE,MAAM,GAAG,QAAQ,GAAG,OAAO,GAAG,UAAU,GAAG,IAAI;IA2BtE;;OAEG;IACH,OAAO,CAAC,6BAA6B;IASrC;;;;;OAKG;IACH,qBAAqB,IAAI,IAAI;IAO7B;;OAEG;IACH,gBAAgB,IAAI,MAAM,GAAG,QAAQ,GAAG,OAAO,GAAG,UAAU;IAI5D;;;OAGG;IACH,YAAY,IAAI,MAAM,GAAG,IAAI;IAI7B;;OAEG;IACH,oBAAoB,IAAI;QACtB,mBAAmB,EAAE,MAAM,CAAC;QAC5B,cAAc,EAAE,MAAM,CAAC;QACvB,gBAAgB,EAAE,MAAM,CAAC;QACzB,aAAa,EAAE,MAAM,CAAC;QACtB,oBAAoB,EAAE,MAAM,CAAC;QAC7B,cAAc,EAAE,MAAM,CAAC;QACvB,YAAY,EAAE,OAAO,CAAC;QACtB,gBAAgB,EAAE,MAAM,CAAC;QACzB,sBAAsB,EAAE,MAAM,CAAC;KAChC;IAoBD;;;;OAIG;YACW,eAAe;IAyD7B;;OAEG;YACW,gBAAgB;IA2H9B;;OAEG;YACW,eAAe;IAyI7B;;;;;OAKG;IACG,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC;IA4D9B;;;;;;;;OAQG;IACG,cAAc,IAAI,OAAO,CAAC,IAAI,CAAC;IAwDrC;;;;OAIG;YACW,4BAA4B;IAqB1C;;OAEG;IACG,UAAU,IAAI,OAAO,CAAC,IAAI,CAAC;IAwBjC;;OAEG;IACH,UAAU,IAAI,OAAO;IAQrB,MAAM,CAAC,QAAQ,CAAC,gBAAgB;;;MAAoB;IAEpD;;;OAGG;IACG,oBAAoB,IAAI,OAAO,CAAC,cAAc,CAAC;IAOrD;;OAEG;IACG,mBAAmB,CAAC,OAAO,EAAE,cAAc,GAAG,OAAO,CAAC,IAAI,CAAC;IAUjE;;OAEG;IACH,OAAO,CAAC,WAAW;IASnB;;;OAGG;IACH,OAAO,CAAC,gBAAgB;IAqCxB;;;;;;;;OAQG;IACG,6BAA6B,CAAC,YAAY,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IA0BlF;;;;OAIG;YACW,gCAAgC;IA6G9C;;;;OAIG;IACG,iBAAiB,CAAC,cAAc,EAAE,MAAM,EAAE,QAAQ,EAAE,IAAI,GAAG,OAAO,CAAC,MAAM,CAAC;IA6GhF;;;;;OAKG;IACG,UAAU,CAAC,GAAG,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,cAAc,GAAG,IAAI,EAAE,gBAAgB,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC;IAgF1G;;;;;;;;;OASG;IACG,mBAAmB,CAAC,GAAG,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAa,GAAG,OAAO,CAAC;QAAE,IAAI,EAAE,IAAI,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IAyN1G;;;OAGG;IACH,OAAO,CAAC,qBAAqB;IA+K7B;;OAEG;IACG,QAAQ,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;IAKjC;;;;OAIG;IACG,oBAAoB,IAAI,OAAO,CAAC,MAAM,CAAC;IAsB7C;;OAEG;IACG,iBAAiB,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,GAAG,IAAI,CAAC;IA+B/D;;OAEG;IACG,aAAa,CAAC,IAAI,EAAE,IAAI,GAAG,OAAO,CAAC,UAAU,CAAC;IAuBpD;;;OAGG;IACG,IAAI,CAAC,CAAC,GAAG,OAAO,EACpB,IAAI,EAAE,IAAI,EACV,MAAM,EAAE,MAAM,EACd,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAC/B,OAAO,CAAC,CAAC,CAAC;IAoBb;;OAEG;IACH,UAAU,IAAI,MAAM,EAAE;IAItB;;OAEG;IACH,UAAU,CAAC,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS;IAIhD;;OAEG;IACG,SAAS,CAAC,IAAI,EAAE,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;IAS1C;;OAEG;IACG,SAAS,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAShD;;OAEG;IACH,WAAW,IAAI,OAAO;IAItB;;OAEG;IACH,OAAO,IAAI,MAAM;IAIjB;;OAEG;IACH,MAAM,CAAC,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS;CAG1E;AAKD,wBAAgB,YAAY,CAAC,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS,CAKlE;AAED;;GAEG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,OAAO,CAAqC;IAEpD;;OAEG;IACH,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,gBAAgB,GAAG,SAAS;IAShE;;OAEG;IACH,GAAG,CAAC,IAAI,EAAE,MAAM,GAAG,SAAS,GAAG,SAAS;IAIxC;;OAEG;IACH,MAAM,IAAI,SAAS,EAAE;IAIrB;;OAEG;IACG,aAAa,IAAI,OAAO,CAAC,IAAI,CAAC;CASrC;AAKD,wBAAgB,mBAAmB,IAAI,gBAAgB,CAKtD"}

package/dist/cdp/client.js

@@ -52,6 +52,7 @@ const defaults_1 = require("../config/defaults");
 const with_timeout_1 = require("../utils/with-timeout");
 const collector_1 = require("../metrics/collector");
 const connection_1 = require("../errors/connection");
+const fingerprint_defense_1 = require("../stealth/fingerprint-defense");
 function parseEnvInt(name, fallback) {
     const raw = process.env[name];
     if (raw === undefined)
@@ -1214,28 +1215,31 @@ class CDPClient {
      * CDP observer attached. CDP is attached only after the settle window expires.
      *
      * @param url      URL to open in the new tab
-     * @param settleMs Milliseconds to wait before attaching CDP (default 5000, range 1000-30000)
+     * @param settleMs Total settle time in ms — used for navigation timeout and post-nav wait (default 8000)
      * @returns        The Puppeteer Page and its targetId
      */
     async createTargetStealth(url, settleMs = 8000) {
         const browser = this.getBrowser();
-        // Step 1: Create target via CDP Target.createTarget.
-        // Puppeteer's ChromeTargetManager uses Target.setAutoAttach with { exclude: true }
-        // for page targets, so the new tab is added to #discoveredTargetsByTargetId but NOT
-        // #attachedTargetsByTargetId. This means browser.pages() will never find it.
-        // We use a CDP session on the browser target to issue the command directly.
+        // Stealth v3 architecture (#450):
+        // Instead of navigating directly to the target URL (which lets anti-bot scripts
+        // fingerprint the raw browser before defenses are applied), we:
+        //   1. Create tab with about:blank (no anti-bot scripts, no network request)
+        //   2. Attach CDP immediately and register all defenses via evaluateOnNewDocument
+        //   3. Navigate to the real URL — defenses fire at document_start, BEFORE any page JS
+        // This closes the fingerprint timing gap that caused Access Denied on Coupang/Reddit.
+        // Step 1: Create target with about:blank — invisible to anti-bot systems
         const cdp = await browser.target().createCDPSession();
         let targetId;
         try {
-            const result = await cdp.send('Target.createTarget', { url });
+            const result = await cdp.send('Target.createTarget', { url: 'about:blank' });
             targetId = result.targetId;
         }
         catch (createErr) {
             await cdp.detach().catch(() => { });
             throw new Error(`Stealth navigation: failed to create target: ${createErr instanceof Error ? createErr.message : String(createErr)}`);
         }
-        console.error(`[CDPClient] Stealth tab created: ${targetId}, settling for ${settleMs}ms`);
-        // Warn if headless — Turnstile detection is nearly guaranteed in headless mode
+        console.error(`[CDPClient] Stealth tab created: ${targetId} (about:blank), will navigate to ${url}`);
+        // Warn if headless — anti-bot detection is nearly guaranteed in headless mode
         {
             const { headless } = (0, global_1.getGlobalConfig)();
             let isHeadless = !!headless;
@@ -1249,31 +1253,23 @@ class CDPClient {
                 }
             }
             if (isHeadless) {
-                console.error('[CDPClient] WARNING: Stealth mode in headless Chrome is unlikely to bypass Turnstile. Use headed Chrome (--visible) for anti-bot pages.');
+                console.error('[CDPClient] WARNING: Stealth mode in headless Chrome is unlikely to bypass anti-bot systems. Use headed Chrome (--visible) for anti-bot pages.');
             }
         }
-        // Step 2: Wait for the page to load without CDP observation (Turnstile runs here)
-        await new Promise(resolve => setTimeout(resolve, settleMs));
-        // Step 3: Attach to the target via CDP to bring it into Puppeteer's attached set
+        // Step 2: Attach CDP immediately (about:blank has no anti-bot to detect it)
         try {
             await cdp.send('Target.attachToTarget', { targetId, flatten: true });
         }
         catch (attachErr) {
-            // Ghost tab cleanup: close the orphaned Chrome tab before throwing
             await cdp.send('Target.closeTarget', { targetId }).catch(() => { });
             await cdp.detach().catch(() => { });
             throw new Error(`Stealth navigation: failed to attach to target ${targetId}: ${attachErr instanceof Error ? attachErr.message : String(attachErr)}`);
         }
-        // Step 4: Use browser.waitForTarget() to reliably find the target.
-        // After attachToTarget, Puppeteer's internal ChromeTargetManager processes the
-        // attachment asynchronously. browser.targets().find() may miss it due to a race
-        // condition. waitForTarget() listens for the target event and handles timing.
         let target;
         try {
             target = await browser.waitForTarget(t => (0, puppeteer_helpers_1.getTargetId)(t) === targetId, { timeout: 5000 });
         }
         catch {
-            // Ghost tab cleanup: close the orphaned Chrome tab before throwing
             await cdp.send('Target.closeTarget', { targetId }).catch(() => { });
             await cdp.detach().catch(() => { });
             throw new Error(`Stealth navigation: target ${targetId} not found after attach (waitForTarget timed out)`);
@@ -1286,22 +1282,51 @@ class CDPClient {
             page = null;
         }
         if (!page) {
-            // Ghost tab cleanup: close the orphaned Chrome tab before throwing
             await cdp.send('Target.closeTarget', { targetId }).catch(() => { });
             await cdp.detach().catch(() => { });
             throw new Error(`Stealth navigation: could not get page for target ${targetId}`);
         }
-        // Clean up the helper session — the page now has its own CDP session managed by Puppeteer
         await cdp.detach().catch(() => { });
-        // Step 5: Index the page and configure defenses (CDP commands flow from here)
+        // Step 3: Register ALL defense scripts BEFORE navigation.
+        // evaluateOnNewDocument scripts persist on the CDP session and execute at
+        // document_start of every new document — before any <script> tag on the page.
         this.targetIdIndex.set(targetId, page);
         this.configurePageDefenses(page);
-        // Step 6: Apply all stealth defenses immediately to the already-loaded page.
-        // configurePageDefenses() registers evaluateOnNewDocument scripts that only run on
-        // future navigations. The current page loaded without CDP, so we must patch it now.
+        // Stealth-only fingerprint defenses (WebGL, Canvas, Audio, hardware, screen, webdriver)
+        const fpScript = (0, fingerprint_defense_1.getStealthFingerprintDefenseScript)();
+        const stackScript = (0, fingerprint_defense_1.getStealthStackSanitizationScript)();
+        await page.evaluateOnNewDocument(fpScript).catch(() => { });
+        await page.evaluateOnNewDocument(stackScript).catch(() => { });
+        // Step 4: Navigate to the real URL — all defenses now fire at document_start
+        console.error(`[CDPClient] Stealth tab ${targetId}: navigating to ${url} with defenses pre-registered`);
+        try {
+            await page.goto(url, {
+                waitUntil: 'domcontentloaded',
+                timeout: Math.max(settleMs, 30000),
+            });
+        }
+        catch (navErr) {
+            // Navigation timeout is not fatal — the page may still be usable
+            // (e.g., Turnstile challenge pages load slowly but are interactive)
+            console.error(`[CDPClient] Stealth navigation warning: ${navErr instanceof Error ? navErr.message : String(navErr)}`);
+        }
+        // Step 5: Post-navigation settle period for challenge completion (Turnstile, etc.)
+        // The page has loaded with defenses active; this extra wait lets async challenges finish.
+        const postNavSettleMs = Math.max(settleMs - 5000, 2000); // At least 2s, reduced from total settle
+        await new Promise(resolve => setTimeout(resolve, postNavSettleMs));
+        // Step 6: Defense-in-depth — apply patches directly to the current page.
+        // evaluateOnNewDocument should have handled this at document_start, but we
+        // re-apply as fallback in case of edge cases (e.g., SPA soft-navigation).
+        await page.evaluate(fpScript).catch(() => { });
+        await page.evaluate(stackScript).catch(() => { });
         await page.evaluate(() => {
-            // 1. navigator.webdriver
-            Object.defineProperty(navigator, 'webdriver', { get: () => undefined, configurable: true });
+            // 1. navigator.webdriver — prototype-level deletion (less detectable than defineProperty)
+            try {
+                delete Object.getPrototypeOf(navigator).webdriver;
+            }
+            catch {
+                Object.defineProperty(navigator, 'webdriver', { get: () => undefined, configurable: true });
+            }
             // 2. chrome.runtime shape
             if (window.chrome) {
                 const originalChrome = window.chrome;
@@ -1387,7 +1412,7 @@ class CDPClient {
                 }
             }
         }).catch(() => { });
-        console.error(`[CDPClient] Stealth tab ${targetId} attached after settle period`);
+        console.error(`[CDPClient] Stealth tab ${targetId} ready (defenses pre-injected, page loaded)`);
         return { page, targetId };
     }
     /**
@@ -1428,14 +1453,22 @@ class CDPClient {
             window.print = () => { console.warn('[OpenChrome] window.print() suppressed'); };
         }).catch(() => { });
         // Remove navigator.webdriver flag that CDP sets automatically.
-        // Anti-automation systems (e.g., Cloudflare Turnstile) check this flag and refuse
-        // to function even for manual human interaction. Defense-in-depth alongside the
-        // --disable-blink-features=AutomationControlled launch flag. (#247)
+        // The --disable-blink-features=AutomationControlled launch flag prevents Blink from
+        // setting this flag in headed mode. For headless mode (where the flag may not work),
+        // we delete the property from the prototype rather than using Object.defineProperty
+        // on the instance — the defineProperty approach is detectable via
+        // Object.getOwnPropertyDescriptor(navigator, 'webdriver'). (#247, #446)
         page.evaluateOnNewDocument(() => {
-            Object.defineProperty(navigator, 'webdriver', {
-                get: () => undefined,
-                configurable: true,
-            });
+            try {
+                delete Object.getPrototypeOf(navigator).webdriver;
+            }
+            catch {
+                // Fallback: defineProperty if prototype deletion fails
+                Object.defineProperty(navigator, 'webdriver', {
+                    get: () => undefined,
+                    configurable: true,
+                });
+            }
         }).catch(() => { });
         // Mask Chrome automation artifacts that anti-bot systems scan for.
         // These cover the most common fingerprinting vectors after navigator.webdriver.