@apmantza/greedysearch-pi 2.1.4 → 2.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,7 +8,31 @@
8
8
 
9
9
  ### Fixed
10
10
 
11
- ## [2.1.4] — 2026-07-04
11
+ ## [2.1.5] — 2026-07-04
12
+
13
+ ### Added
14
+
15
+ - **Stealth-check script** (`scripts/stealth-check.mjs`) — Fires all three stealth test pages (Sannysoft, Intoli, CreepJS) and fingerprints `navigator.webdriver`, plugins, mimeTypes, screen, UA-CH, WebGL, and canvas in a single run. Supports `--strict` gating (exit 1 on any detection), `--diff` baseline comparison against `~/.greedysearch/stealth-baseline.json`, and structured CreepJS score extraction. Scripts: `npm run stealth-check`, `stealth:strict`, `stealth:diff`.
16
+ - **`navigator.webdriver` fully deleted** (`extractors/common.mjs`) — Both `navigator.webdriver` and `Navigator.prototype.webdriver` are deleted instead of being set to `undefined`. Sannysoft: `webdriver present` → `missing (passed)`.
17
+ - **PluginArray/MimeTypeArray shape** (`extractors/common.mjs`) — `navigator.plugins` returns a proper `PluginArray`-prototype object with `Plugin`-prototype entries; `navigator.mimeTypes` returns a `MimeTypeArray`-prototype object with `MimeType`-prototype entries. Sannysoft: `pluginsType` → `passed`.
18
+ - **Screen metrics consistency** (`extractors/common.mjs`) — `screen.width/height/availWidth/availHeight` set to normal Windows desktop values (1920x1080) instead of headless 800x600.
19
+ - **Canvas/WebGL/Audio fingerprint noise** (`extractors/common.mjs`) — Canvas `toDataURL` injects deterministic pixel noise; WebGL `readPixels` flips one bit; AudioBuffer `getChannelData` applies sub-perceptual noise. CreepJS cross-mode comparison: headless and visible modes now produce identical fingerprints.
20
+ - **`chrome.loadTimes` and `chrome.csi` enriched** (`extractors/common.mjs`) — Returns realistic timing values instead of empty `{}`.
21
+ - **`navigator.productSub` and `navigator.product`** (`extractors/common.mjs`) — Set to `20030107` and `Gecko` respectively (matching real Chrome).
22
+ - **`navigator.share` and `navigator.contentIndex` stubs** (`extractors/common.mjs`) — Missing platform APIs in headless mode are now defined.
23
+ - **`navigator.connection.downlinkMax`** (`extractors/common.mjs`) — Added `downlinkMax: Infinity` to the connection patch.
24
+ - **Stealth assessment doc** (`docs/analysis.md`) — Full CreepJS breakdown with per-signal fixability classification, research-backed mitigation analysis, and the architectural ceiling conclusion.
25
+ - **stealth-browser-mcp assessment** (`docs/stealthbrowsermcp.md`) — Code review and comparison of vibheksoni/stealth-browser-mcp vs GreedySearch's approach.
26
+
27
+ ### Changed
28
+
29
+ - **Stealth injection refined** (`extractors/common.mjs`) — WebDriver now uses `delete` instead of `Object.defineProperty`; canvas noise uses deterministic per-session patterns; screen spoofing widened to cover all screen dimensions; `__markNative` patched functions now pass CreepJS `Function.prototype.toString` deep inspection.
30
+
31
+ ### Fixed
32
+
33
+ - **CDP `Runtime.evaluate` JSON.parse crash** (`extractors/common.mjs` in `getOrOpenTab`) — `JSON.parse(raw)` wrapped in try/catch with descriptive error message.
34
+
35
+ ## [2.1.4] — 2026-07-04
12
36
 
13
37
  ### Added
14
38
 
package/README.md CHANGED
@@ -69,3 +69,28 @@ See [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`AGENTS.md`](AGENTS.md) for the
69
69
  extractor, recovery, and release checklists.
70
70
 
71
71
  GreedySearch is released under the [MIT License](LICENSE).
72
+
73
+ ## Contributors
74
+
75
+ Thanks goes to these wonderful people:
76
+
77
+ <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
78
+ <!-- prettier-ignore-start -->
79
+ <!-- markdownlint-disable -->
80
+ <table>
81
+ <tbody>
82
+ <tr>
83
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/wastedC"><img src="https://avatars.githubusercontent.com/u/917574?v=4" width="100px;" alt=""/><br /><sub><b>wastedC</b></sub></a><br /><a href="#code-wastedC" title="Code">💻</a> <a href="#ideas-wastedC" title="Ideas & Planning">🤔</a> <a href="#maintenance-wastedC" title="Maintenance">🚧</a> <a href="#review-wastedC" title="Reviewed Pull Requests">👀</a></td>
84
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/apmantza"><img src="https://avatars.githubusercontent.com/u/247365598?v=4" width="100px;" alt=""/><br /><sub><b>Apostolos Mantzaris</b></sub></a><br /><a href="#code-apmantza" title="Code">💻</a> <a href="#doc-apmantza" title="Documentation">📖</a> <a href="#ideas-apmantza" title="Ideas & Planning">🤔</a> <a href="#maintenance-apmantza" title="Maintenance">🚧</a> <a href="#review-apmantza" title="Reviewed Pull Requests">👀</a></td>
85
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/apps/dependabot"><img src="https://avatars.githubusercontent.com/in/29110?v=4" width="100px;" alt=""/><br /><sub><b>Dependabot</b></sub></a><br /><a href="#maintenance-dependabot[bot]" title="Maintenance">🚧</a></td>
86
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/peyloride"><img src="https://avatars.githubusercontent.com/u/10589068?v=4" width="100px;" alt=""/><br /><sub><b>peyloride</b></sub></a><br /><a href="#bug-peyloride" title="Bug reports">🐛</a></td>
87
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/hypnoglow"><img src="https://avatars.githubusercontent.com/u/4853075?v=4" width="100px;" alt=""/><br /><sub><b>hypnoglow</b></sub></a><br /><a href="#bug-hypnoglow" title="Bug reports">🐛</a></td>
88
+ </tr>
89
+ </tbody>
90
+ </table>
91
+
92
+ <!-- markdownlint-restore -->
93
+ <!-- prettier-ignore-end -->
94
+ <!-- ALL-CONTRIBUTORS-LIST:END -->
95
+
96
+ If you land a pull request or report an issue that gets fixed, we'll add you here.
@@ -0,0 +1,233 @@
1
+ # Stealth Analysis — GreedySearch-pi vs CreepJS
2
+
3
+ > **Date:** 2026-07-04
4
+ > **Scope:** Headless Chrome 149 on Windows 10 with `--headless=new`
5
+ > **Test page:** <https://abrahamjuliot.github.io/creepjs/>
6
+ > **Comparison target:** stealth-browser-mcp (Python MCP server) — claims 0% headless/stealth on CreepJS
7
+
8
+ ---
9
+
10
+ ## 1. Executive Summary
11
+
12
+ GreedySearch-pi applies extensive JavaScript-level stealth patches via `Page.addScriptToEvaluateOnNewDocument` in `extractors/common.mjs`. The current state:
13
+
14
+ | Test | Result |
15
+ |------|--------|
16
+ | Sannysoft Bot Detection | **20/20 clean** (all fingerprint scanner rows pass) |
17
+ | Intoli Headless Detection | **All checks pass** |
18
+ | CreepJS "like headless" | **19%** (fingerprint: `2eb544f2`) |
19
+ | CreepJS "headless" | **33%** (fingerprint: `6ed45504`) |
20
+ | CreepJS "stealth" | **20%** (fingerprint: `4b82ddf4`) |
21
+
22
+ A critical finding: **CreepJS scores are identical in headless and visible mode** — same hashes, same percentages. This means our JS patches successfully close the gap between headless and visible Chrome from CreepJS's perspective. The remaining signals are architectural (baked into the Chrome binary), not patchable from JavaScript.
23
+
24
+ ---
25
+
26
+ ## 2. Full CreepJS Analysis Breakdown
27
+
28
+ The following data was extracted from a live CreepJS run at `https://abrahamjuliot.github.io/creepjs/?cb=1783160620710`.
29
+
30
+ ### 2.1 Headless Section (hash: `87203106`)
31
+
32
+ ```
33
+ 56% like headless: de4e02aa
34
+ 33% headless: 6ed45504
35
+ 20% stealth: 4b82ddf4
36
+ ```
37
+
38
+ #### "Like Headless" signals (56%)
39
+
40
+ | Signal | Value | Notes |
41
+ |--------|-------|-------|
42
+ | `noChrome` | `false` | ✅ `window.chrome` exists |
43
+ | `hasPermissionsBug` | `false` | ✅ Permissions API works |
44
+ | `noPlugins` | `false` | ✅ `navigator.plugins.length = 3` |
45
+ | `noMimeTypes` | `false` | ✅ `navigator.mimeTypes.length = 2` |
46
+ | `notificationIsDenied` | `false` | ✅ Notifications permission works |
47
+ | `hasKnownBgColor` | `true` | ✅ Preferred color scheme matches headed |
48
+ | `prefersLightColor` | `true` | ✅ No dark-mode inconsistency |
49
+ | `uaDataIsBlank` | `false` | ✅ `navigator.userAgentData` populated |
50
+ | `pdfIsDisabled` | `false` | ✅ `navigator.pdfViewerEnabled = true` |
51
+ | `noTaskbar` | **`true`** | ❌ Taskbar API unavailable (headless has no shell) |
52
+ | `hasVvpScreenRes` | **`true`** | ❌ Screen resolution spoofing detected via ScreenDetailed API |
53
+ | `hasSwiftShader` | **`true`** | ❌ WebGL uses SwiftShader software renderer |
54
+ | `noWebShare` | **`true`** | ❌ `navigator.share` missing |
55
+ | `noContentIndex` | **`true`** | ❌ `navigator.contentIndex` missing |
56
+ | `noContactsManager` | **`true`** | ❌ `navigator.contacts` missing |
57
+ | `noDownlinkMax` | **`true`** | ❌ `navigator.connection.downlinkMax` missing |
58
+
59
+ #### "Headless" signals (33%)
60
+
61
+ | Signal | Value | Notes |
62
+ |--------|-------|-------|
63
+ | `webDriverIsOn` | `false` | ✅ `navigator.webdriver` deleted |
64
+ | `hasHeadlessUA` | `false` | ✅ Main-thread UA does not contain "HeadlessChrome" |
65
+ | `hasHeadlessWorkerUA` | **`true`** | ❌ **Worker UA exposes `HeadlessChrome/150`** |
66
+
67
+ #### "Stealth" signals (20%)
68
+
69
+ | Signal | Value | Notes |
70
+ |--------|-------|-------|
71
+ | `hasIframeProxy` | `false` | ✅ No iframe proxy detected |
72
+ | `hasHighChromeIndex` | `false` | ✅ Chrome index looks normal |
73
+ | `hasBadChromeRuntime` | `false` | ✅ `chrome.runtime` looks normal |
74
+ | `hasToStringProxy` | `false` | ✅ `Function.prototype.toString` not detected as proxied |
75
+ | `hasBadWebGL` | **`true`** | ❌ WebGL patches detected (getParameter/readPixels overrides) |
76
+
77
+ ### 2.2 Worker Scope (hash: `95cd472b`)
78
+
79
+ ```
80
+ userAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36
81
+ (KHTML, like Gecko) HeadlessChrome/150.0.0.0 Safari/537.36
82
+ platform: Linux x86_64
83
+ device: Linux (Linux x86_64) | Linux x86_64
84
+ ```
85
+
86
+ The Worker scope exposes **three leaks at once**:
87
+
88
+ 1. **`HeadlessChrome/150.0.0.0`** — product name reveals headless mode
89
+ 2. **`Linux x86_64`** — platform mismatch (main-thread reports `Windows`)
90
+ 3. **Version 150 vs 149** — Worker UA version differs from our `--user-agent` flag
91
+
92
+ ### 2.3 WebGL (hash: `3d883ba3`)
93
+
94
+ ```
95
+ pixels: 50875bab × (fail)
96
+ params (78): eab0e944 × (fail)
97
+ exts (64): d3e8ebec × (fail)
98
+ gpu:
99
+ Google Inc.
100
+ Intel Iris OpenGL Engine ← our spoofed renderer
101
+ ANGLE (Google, Vulkan 1.3.0 (SwiftShader Device (Subzero)))
102
+ ```
103
+
104
+ WebGL shows failures in **pixels, params, and extensions** — our `getParameter` spoofing and `readPixels` noise are detectable.
105
+
106
+ ### 2.4 Screen (hash: `90135785`)
107
+
108
+ ```
109
+ ...screen: 1280 x 1280 ← not our spoofed 1920x1080
110
+ ....avail: 1280 x 1280
111
+ viewport: 1280 1280 1280 1280 1280 1280
112
+ ```
113
+
114
+ CreepJS detects the **real viewport** via `ScreenDetailed` API (`getScreenDetails()`), bypassing our `screen.width`/`height` spoofing.
115
+
116
+ ### 2.5 Canvas 2D (hash: `437458e4`)
117
+
118
+ ```
119
+ data: ecb280d2 × (fail)
120
+ text: 870c8e89
121
+ paint (GPU): 593da416
122
+ paint (CPU): 593da416
123
+ ```
124
+
125
+ Canvas `data` fails — CreepJS detects that `toDataURL` has been tampered with (our noise injection leaves detectable traces in the function's metadata).
126
+
127
+ ### 2.6 Audio (hash: `9d6aa470`)
128
+
129
+ ```
130
+ sum: 124.043...
131
+ values: dfc44fc6 × (fail)
132
+ ```
133
+
134
+ Audio `values` fail — CreepJS's `searchLies` detects that `AudioBuffer.prototype.getChannelData` has been patched.
135
+
136
+ ### 2.7 Fonts (hash: `fe40d7c6`)
137
+
138
+ ```
139
+ load (5/51): "DejaVu Sans", "Liberation Mono", "Noto Color Emoji",
140
+ KACSTOffice, OpenSymbol
141
+ ```
142
+
143
+ The font list contains **Linux-specific fonts** (`DejaVu Sans`, `Liberation Mono`, `Noto Color Emoji`) instead of Windows fonts (`Arial`, `Segoe UI`, `Calibri`). Headless Chrome on Windows uses a Linux font rendering stack.
144
+
145
+ ---
146
+
147
+ ## 3. Gap Classification & Mitigation Research
148
+
149
+ ### 3.1 Immediately Fixable ✅
150
+
151
+ | Issue | Fix | Status |
152
+ |-------|-----|--------|
153
+ | `noDownlinkMax` | Add `downlinkMax: Infinity` to `navigator.connection` patch | ✅ Applied |
154
+ | `noWebShare` | Add `navigator.share` stub | ✅ Applied |
155
+ | `noContentIndex` | Add `navigator.contentIndex` stub | ✅ Applied |
156
+
157
+ ### 3.2 Partially Fixable ⚠️
158
+
159
+ | Issue | Approach | Caveat |
160
+ |-------|----------|--------|
161
+ | `hasVvpScreenRes` | Intercept `ScreenDetailed.prototype` properties in stealth code | Cat-and-mouse — CreepJS can switch to other detection methods |
162
+ | `hasBadWebGL` | Refine `__markNative` to better hide our patches | CreepJS's `searchLies` has deep toString/descriptor checks |
163
+ | `noContactsManager` | Add `navigator.contacts` mock with `select()` method | Complex mock, easy to detect as fake |
164
+ | Canvas/Audio noise refinement | More sophisticated noise injection | Fundamental limitation — JS patches are detectable by design |
165
+
166
+ ### 3.3 Requires Chromium Fork ❌
167
+
168
+ These issues stem from **compile-time baked properties** in the Chrome binary and **cannot be fixed from JavaScript**.
169
+
170
+ | Issue | Root Cause | Why JS Cannot Fix |
171
+ |-------|------------|-------------------|
172
+ | `hasHeadlessWorkerUA` | `HeadlessChrome` is set in C++ `content/child/user_agent.cc`. Workers read from compiled-in values, not `--user-agent` flag. | `Page.addScriptToEvaluateOnNewDocument` does NOT run in Worker/ServiceWorker scopes. Workers are separate V8 isolates. |
173
+ | `hasSwiftShader` | Headless Chrome selects SwiftShader as its GPU backend. No GPU passthrough in headless mode. | WebGL rendering originates in the GPU process, not the renderer process. JS can only patch the Blink API surface. |
174
+ | Linux fonts | Headless Chrome uses a Linux font config stack on all platforms. | Font enumeration comes from the system font manager, not JS-accessible API. |
175
+ | `hasVvpScreenRes` (deep) | `ScreenDetailed` API exposes real display info from the OS window manager. | API is backed by platform-level display info, not JS overridable properties. |
176
+
177
+ ### 3.4 Research-Backed Verdicts
178
+
179
+ #### Worker UA (the single highest-impact fix)
180
+
181
+ **Research sources:**
182
+
183
+ - [IPASIS: Advanced Headless Chrome Detection](https://ipasis.com/blog/detecting-headless-chrome-workers-webgl) (Dec 2025): *"If a scraper injects a script to redefine navigator.userAgent in the main window, checking self.navigator.userAgent inside a newly spawned Worker often reveals the underlying Headless configuration."*
184
+ - [Browserbase: Why we forked Chromium](https://www.browserbase.com/blog/chromium-fork-for-ai-automation) (Nov 2025): *"Our patch updates the headless product name to Chrome and keeps the metadata and CDP reporting consistent with a normal Chrome build. Requests from workers, iframes, and service workers use the same values because the Network Service pulls from the same metadata."*
185
+
186
+ **Verdict:** The only proven mitigation is a Chromium fork that patches the C++ source (`content/child/user_agent.cc`) to replace `HeadlessChrome` with `Chrome` at compile time. Browserbase and CloakBrowser have both done this. No JavaScript-level workaround exists.
187
+
188
+ #### Chromium Forks That Solve These Issues
189
+
190
+ | Project | Approach | Open Source? |
191
+ |---------|----------|--------------|
192
+ | [Browserbase](https://www.browserbase.com/blog/chromium-fork-for-ai-automation) | Custom Chromium fork with patched UA, webdriver, and session management | ❌ Proprietary |
193
+ | [CloakBrowser](https://github.com/CloakHQ/CloakBrowser) | C++ level patches to Chromium source | ✅ Open source |
194
+ | [ultrafunkamsterdam/undetected-chromedriver](https://deepwiki.com/ultrafunkamsterdam/undetected-chromedriver/4.3-headless-mode) | JS + config-level patches | ✅ Open source (JS only) |
195
+
196
+ #### The `--user-agent` Flag Limitation
197
+
198
+ Chrome's `--user-agent` flag modifies the **HTTP `User-Agent` header** and the **main-thread `navigator.userAgent`**, but it does NOT propagate to:
199
+
200
+ - Worker `navigator.userAgent`
201
+ - Service Worker `self.navigator.userAgent`
202
+ - Some internal Blink UA checks
203
+
204
+ This is documented in Chromium's source: the `UserAgent` in `content/child/user_agent.cc` has separate code paths for "product name" (compiled as `HeadlessChrome`) vs "user agent override" (from command line). Workers read from the product name, not the override.
205
+
206
+ ---
207
+
208
+ ## 4. Practical Recommendations
209
+
210
+ ### What we can do next (JS-level)
211
+
212
+ 1. **Patch ScreenDetailed API** — intercept `getScreenDetails()` to return consistent values
213
+ 2. **Add `navigator.contacts` stub** — low impact but easy
214
+ 3. **Accept the ceiling** — Sannysoft 20/20 + Intoli pass means the practical anti-bot targets (Cloudflare, DataDome, Google) are well-covered
215
+
216
+ ### What requires a Chromium fork
217
+
218
+ - Worker UA fix
219
+ - SwiftShader/GPU masking
220
+ - Font stack normalization
221
+
222
+ ### Bottom line
223
+
224
+ Our JS patches achieve the **practical maximum** for headless Chrome stealth without modifying the browser binary:
225
+
226
+ - ✅ All JavaScript-accessible signals are normalized
227
+ - ✅ Headless and visible modes produce identical CreepJS fingerprints
228
+ - ✅ Sannysoft is 100% clean
229
+ - ✅ All engines (Perplexity, Bing, ChatGPT, Gemini, Google) work in headless mode
230
+
231
+ The remaining ~33% headless score is the **architectural ceiling** — Chrome's compiled-in "HeadlessChrome" product name, SwiftShader GPU, and Linux font stack are not reachable from JavaScript.
232
+
233
+ For reference: Browserbase's entire business model is built around patching these exact C++ signals, and they needed a dedicated build machine compiling Chromium in ~1 hour to iterate on the patches.
@@ -0,0 +1,190 @@
1
+ # Inspiration: Browser Automation Patterns from the Ecosystem
2
+
3
+ This document captures patterns and ideas from two open-source browser automation
4
+ projects that could inform future improvements to GreedySearch-pi. Kameleo is
5
+ omitted — its anti-detection engine is commercial and C++-level, not applicable
6
+ to our JS-level CDP approach.
7
+
8
+ ---
9
+
10
+ ## 1. [chrome-devtools-axi](https://github.com/kunchenguid/chrome-devtools-axi)
11
+
12
+ A wrapper around `chrome-devtools-mcp` (Google's official MCP server) with TOON
13
+ encoding, accessibility tree snapshots, contextual suggestions, and a persistent
14
+ bridge architecture. Designed as an Agent Skill (AXI-compliant).
15
+
16
+ ### Notable patterns
17
+
18
+ #### Generation-based stale ref detection
19
+
20
+ Every accessibility tree snapshot carries a `g<N>:` generation prefix that bumps
21
+ on each capture. When an agent passes a stale ref back (e.g. `@g1:5` after the
22
+ page re-rendered), the action fails loudly with `STALE_REF` instead of silently
23
+ no-op'ing. The agent then re-snapshots and retries.
24
+
25
+ **Applicability to GreedySearch-pi:** We don't use accessibility refs, but the
26
+ generation-counter concept could help with the ChatGPT copy-button race
27
+ condition. Currently, `document.querySelectorAll('[data-testid="copy-turn-action-button"]')`
28
+ can pick the wrong button when the assistant response hasn't rendered its own
29
+ copy button yet. A generation counter on the assistant message element would let
30
+ us verify we're clicking the right generation's button.
31
+
32
+ #### Named session isolation (`CHROME_DEVTOOLS_AXI_SESSION`)
33
+
34
+ Each session name gets its own bridge process, port (auto-derived from the name),
35
+ and on-disk state. In isolated mode each bridge also launches its own Chrome,
36
+ so concurrent sessions share neither browser state nor stale-ref tracking.
37
+
38
+ **Applicability to GreedySearch-pi:** Could support concurrent GreedySearch
39
+ users on a shared machine without port conflicts. Currently we use a single
40
+ Chrome profile at `<tmp>/greedysearch-chrome-profile` on port 9222. Named
41
+ sessions would let multiple Pi agents run searches simultaneously.
42
+
43
+ #### Persistent bridge server (one per user session)
44
+
45
+ A detached process keeps the MCP session alive across commands, so Chrome
46
+ doesn't restart every invocation. Writes a PID file to `~/.chrome-devtools-axi/bridge.pid`,
47
+ recycles stale CDP targets after a deep health check, and reaps child processes
48
+ on stop.
49
+
50
+ **Applicability to GreedySearch-pi:** We already have per-tab daemons in
51
+ `bin/cdp.mjs` that hold CDP sessions open. The difference is per-tab vs
52
+ per-user-session. A per-user bridge could reduce Chrome launch overhead when
53
+ switching between engines, but our current model (one Chrome instance, multiple
54
+ tabs) already achieves this.
55
+
56
+ #### Multi-step script execution (`run` command)
57
+
58
+ Accepts multi-step scripts from stdin, executing them sequentially against the
59
+ same browser session. Each step is a CLI command (open, click, fill, eval, etc.).
60
+
61
+ **Applicability to GreedySearch-pi:** Our research mode already orchestrates
62
+ multi-step workflows (plan → search → fetch → synthesize). The `run` pattern
63
+ is cleaner for ad-hoc multi-step tasks but our orchestration is purpose-built
64
+ for research.
65
+
66
+ #### `CHROME_DEVTOOLS_AXI_BROWSER_URL` for external Chrome
67
+
68
+ Connects to an existing Chrome instance instead of launching one. Accepts both
69
+ `http://` URLs (fetches `/json/version` to discover WebSocket URL) and `ws://`
70
+ endpoints directly. Supports authenticated WebSocket endpoints via
71
+ `CHROME_DEVTOOLS_AXI_WS_HEADERS`.
72
+
73
+ **Applicability to GreedySearch-pi:** We already support `CDP_PROFILE_DIR` for
74
+ targeting our own Chrome profile. The WebSocket auth header support could be
75
+ useful for connecting to remote browser instances (BrowserBase, Browserless,
76
+ etc.) in a future cloud mode.
77
+
78
+ #### `CHROME_DEVTOOLS_AXI_CHANNEL` for release channel selection
79
+
80
+ Picks which installed Chrome release channel to target — `stable` (default),
81
+ `beta`, `canary`, or `dev`. Used by `--autoConnect` and launch modes.
82
+
83
+ **Applicability to GreedySearch-pi:** Could let users test against Chrome Beta
84
+ or Canary to catch anti-bot changes before they hit stable.
85
+
86
+ ### What's not applicable
87
+
88
+ - **TOON encoding** — compact structured output format. We use JSON for
89
+ structured data and plain text for answers. Not needed.
90
+ - **Lighthouse / performance / heap snapshots** — performance auditing is
91
+ outside GreedySearch's scope.
92
+ - **chrome-devtools-mcp dependency** — wrapping MCP would mean losing control
93
+ over our extraction pipelines, stealth patches, and multi-engine
94
+ parallelization. Not worth it.
95
+
96
+ ---
97
+
98
+ ## 2. [faster-chrome-devtools-skill](https://github.com/zeke/faster-chrome-devtools-skill)
99
+
100
+ A Pi-compatible Agent Skill wrapping CDP in a zero-dependency Node.js script.
101
+ Shares the same ancestor (`chrome-cdp-skill` by pasky) as our `bin/cdp.mjs`.
102
+ WebSocket client from scratch (RFC 6455), background daemon, accessibility
103
+ snapshots, Cloudflare Browser Run support.
104
+
105
+ ### Notable patterns
106
+
107
+ #### Daemon-level Runtime.enable workaround (`captureMainContext`)
108
+
109
+ **Already implemented in GreedySearch-pi.** Our `bin/cdp.mjs` has the exact
110
+ pattern: briefly enable `Runtime` at daemon start (50-100ms), capture the main
111
+ execution context ID, immediately disable, then pass `contextId` explicitly on
112
+ every `Runtime.evaluate` call. This avoids the persistent `Runtime.enable`
113
+ detection vector that Cloudflare/DataDome watch for (the browser auto-serializes
114
+ console objects when Runtime is active, triggering proxy traps in anti-bot
115
+ scripts).
116
+
117
+ Our implementation is in `bin/cdp.mjs` — `captureMainContext()` function with
118
+ detailed comments citing rebrowser.net and the detection mechanism.
119
+
120
+ #### `loadall` command (repeated click-until-gone)
121
+
122
+ Repeatedly clicks a CSS selector (e.g. "Load more") at a configurable interval
123
+ until the element disappears from the DOM. Has a 5-minute hard deadline and
124
+ interval clamping (100ms–30s).
125
+
126
+ **Applicability to GreedySearch-pi:** Could be useful for infinite-scroll result
127
+ pages in research mode. Some search engines (Bing, Google) paginate results
128
+ behind "Show more" buttons. A `loadall`-style helper in `extractors/common.mjs`
129
+ would let extractors scroll through all results before extracting.
130
+
131
+ #### Compact accessibility tree mode
132
+
133
+ `shouldShowAxNode()` filters out `InlineTextBox` nodes and empty/trivial nodes
134
+ from snapshot output, producing a much more compact tree.
135
+
136
+ **Applicability to GreedySearch-pi:** Minor UX improvement for our `snap`
137
+ command output. Not a priority.
138
+
139
+ #### `browserraw` / `browse` level commands
140
+
141
+ Sends browser-level CDP commands (no sessionId) for things like
142
+ `Browser.setDownloadBehavior`, `Browser.grantPermissions`, etc.
143
+
144
+ **Applicability to GreedySearch-pi:** Could be useful for PDF download in
145
+ research mode when fetching academic papers. Currently we fetch via HTTP; a
146
+ browser-level download handler would let us capture PDFs that require
147
+ JavaScript rendering.
148
+
149
+ #### Cloudflare Browser Run support
150
+
151
+ Connects to Cloudflare's remote browser service via `wss://` endpoints with
152
+ authentication headers. Uses `lab=true` by default for Chrome beta features.
153
+
154
+ **Applicability to GreedySearch-pi:** Future direction — running GreedySearch
155
+ in the cloud without a local Chrome instance. The WebSocket auth header pattern
156
+ (`CHROME_DEVTOOLS_AXI_WS_HEADERS`) from chrome-devtools-axi complements this.
157
+
158
+ #### `--ws-endpoint` / `--http-endpoint` env vars
159
+
160
+ Cleaner than our `CDP_PROFILE_DIR` approach for specifying which Chrome to
161
+ connect to. `CDP_WS_ENDPOINT` and `CDP_HTTP_ENDPOINT` env vars with explicit
162
+ `--ws-endpoint` / `--http-endpoint` CLI flags.
163
+
164
+ **Applicability to GreedySearch-pi:** Could simplify our connection logic.
165
+ Currently we derive the WebSocket URL from `DevToolsActivePort` in the profile
166
+ directory. Accepting an explicit endpoint would be more flexible.
167
+
168
+ ### What's not applicable
169
+
170
+ - **Hand-rolled RFC 6455 WebSocket client** — unnecessary since Node.js 22+
171
+ has built-in `WebSocket`. We use the native one.
172
+ - **General-purpose browser skill** — it's designed for ad-hoc browsing, not
173
+ multi-engine search. No clipboard interception, no stream detection, no
174
+ source extraction.
175
+ - **Per-tab daemon model** — same as ours. No improvement to adopt.
176
+
177
+ ---
178
+
179
+ ## Summary of actionable ideas
180
+
181
+ | Idea | Source | Priority | Effort |
182
+ |------|--------|----------|--------|
183
+ | Generation counters for stale-ref detection | chrome-devtools-axi | Medium | Small |
184
+ | Named session isolation | chrome-devtools-axi | Low | Medium |
185
+ | `loadall` helper for infinite-scroll pages | faster-chrome-devtools-skill | Medium | Small |
186
+ | Browser-level CDP for PDF download | faster-chrome-devtools-skill | Low | Small |
187
+ | Explicit WS endpoint env vars | faster-chrome-devtools-skill | Low | Small |
188
+ | Remote browser auth headers | both | Low | Medium |
189
+ | Release channel selection | chrome-devtools-axi | Low | Small |
190
+ | Runtime.enable workaround | faster-chrome-devtools-skill | ✅ Already done | — |