@apmantza/greedysearch-pi 2.1.3 → 2.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +48 -1
- package/README.md +52 -175
- package/bin/search.mjs +2 -0
- package/docs/analysis.md +233 -0
- package/docs/banner.svg +86 -0
- package/docs/development.md +63 -0
- package/docs/inspiration2.md +190 -0
- package/docs/releases.md +88 -0
- package/docs/research.md +82 -0
- package/docs/runtime.md +65 -0
- package/docs/source-fetching.md +33 -0
- package/docs/stealthbrowsermcp.md +807 -0
- package/docs/usage.md +69 -0
- package/extractors/chatgpt.mjs +127 -23
- package/extractors/common.mjs +116 -28
- package/extractors/gemini.mjs +63 -38
- package/package.json +13 -7
- package/scripts/backfill-github-releases.mjs +139 -0
- package/scripts/changelog-extract.mjs +48 -0
- package/scripts/changelog-release.mjs +65 -0
- package/scripts/check-lockfile.mjs +45 -0
- package/scripts/lib/changelog.mjs +168 -0
- package/scripts/lint.mjs +62 -0
- package/scripts/run-tests.ps1 +179 -0
- package/scripts/stealth-check.mjs +663 -0
- package/src/fetcher.mjs +9 -5
- package/src/search/constants.mjs +1 -1
- package/src/search/research.mjs +57 -11
- package/src/search/simple-research.mjs +4 -1
- package/test.mjs +16 -3
- package/skills/greedy-search/skill.md +0 -20
|
@@ -0,0 +1,807 @@
|
|
|
1
|
+
# Stealth Browser MCP — Reuse Assessment
|
|
2
|
+
|
|
3
|
+
> **Repo:** <https://github.com/vibheksoni/stealth-browser-mcp>
|
|
4
|
+
> **Assessed:** 2026-07-04
|
|
5
|
+
> **Scope:** Full code-level review of architecture, stealth techniques, CDP patterns, network interception, process lifecycle, and security
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## ⚠️ Confidence Notes (read first)
|
|
10
|
+
|
|
11
|
+
This assessment is based on **partial code review** — I read these files in full or near-full:
|
|
12
|
+
|
|
13
|
+
- ✅ `README.md` (full)
|
|
14
|
+
- ✅ `src/browser_manager.py` (full)
|
|
15
|
+
- ✅ `src/dom_handler.py` (full)
|
|
16
|
+
- ✅ `src/network_interceptor.py` (full)
|
|
17
|
+
- ✅ `src/dynamic_hook_system.py` (full)
|
|
18
|
+
- ✅ `src/server.py` (first half — 1587 of 2969 lines)
|
|
19
|
+
- ✅ `src/models.py` (full)
|
|
20
|
+
- ✅ `STEALTH_TESTS.md`, `COMPARISON.md` (full)
|
|
21
|
+
|
|
22
|
+
I did **not** fully read:
|
|
23
|
+
|
|
24
|
+
- ⚠️ `src/process_cleanup.py` — file is 30K chars; I read the header + class outline but not every method. Claims about its sophistication are based on file size and method signatures, not exhaustive code review.
|
|
25
|
+
- ⚠️ `src/platform_utils.py` — referenced via imports; only header + key methods read.
|
|
26
|
+
- ⚠️ `src/debug_logger.py` — referenced extensively in `server.py` but body not opened.
|
|
27
|
+
- ⚠️ `src/dynamic_hook_ai_interface.py` — referenced but not read.
|
|
28
|
+
- ⚠️ `src/server.py` lines 1588-2969 — second half (likely contains more `section_tool` definitions and the element-cloning wiring).
|
|
29
|
+
- ⚠️ All `src/js/*.js` files (element extractors).
|
|
30
|
+
- ⚠️ All element-cloner modules (`cdp_element_cloner.py`, `comprehensive_element_cloner.py`, etc.).
|
|
31
|
+
|
|
32
|
+
**No live testing was performed.** I did not run either tool, did not compare fingerprint results side-by-side, and did not benchmark tab-recycling claims against real ChatGPT stalling.
|
|
33
|
+
|
|
34
|
+
**Comparative claims about GreedySearch's stealth superiority are inference, not proof.** I read GreedySearch's `AGENTS.md` design notes (which describe the patches) but did not see empirical test results comparable to stealth-browser-mcp's `STEALTH_TESTS.md` (which has actual 0% on CreepJS, 20/20 on Sannysoft). So in practice, stealth-browser-mcp may have *more validation evidence* — I just inferred GreedySearch's patches are deeper from the design intent described in AGENTS.md.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Table of Contents
|
|
39
|
+
|
|
40
|
+
- [1. Executive Summary](#1-executive-summary)
|
|
41
|
+
- [2. Architecture Patterns](#2-architecture-patterns)
|
|
42
|
+
- [3. Stealth Techniques](#3-stealth-techniques)
|
|
43
|
+
- [4. CDP Patterns](#4-cdp-patterns)
|
|
44
|
+
- [5. Network Interception](#5-network-interception)
|
|
45
|
+
- [6. Process Lifecycle & Cleanup](#6-process-lifecycle--cleanup)
|
|
46
|
+
- [7. DOM Interaction Patterns](#7-dom-interaction-patterns)
|
|
47
|
+
- [8. Element Cloning](#8-element-cloning)
|
|
48
|
+
- [9. Security Considerations](#9-security-considerations)
|
|
49
|
+
- [10. Test & Validation Practices](#10-test--validation-practices)
|
|
50
|
+
- [11. Portable Code Patterns (confidence-rated)](#11-portable-code-patterns-confidence-rated)
|
|
51
|
+
- [12. Summary Matrix](#12-summary-matrix)
|
|
52
|
+
- [13. Recommendations](#13-recommendations)
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 1. Executive Summary
|
|
57
|
+
|
|
58
|
+
**stealth-browser-mcp** is a Python-based MCP server that provides undetectable browser automation to AI agents. It uses **nodriver** (a CDP-only browser driver), **FastMCP** for the MCP server layer, and offers **97 tools across 11 sections** — from basic navigation to advanced CDP function execution, network interception, and AI-generated dynamic hooks.
|
|
59
|
+
|
|
60
|
+
The project is architecturally impressive but solves a **different problem** than GreedySearch-pi. Stealth Browser MCP is a general-purpose undetectable browser MCP server for AI agents to browse any website. GreedySearch-pi is a purpose-built answer extraction pipeline for AI search/chat engines.
|
|
61
|
+
|
|
62
|
+
**Top reuse candidates** (where I'm confident):
|
|
63
|
+
|
|
64
|
+
1. **Tab recycling after N navigations** — solid code in their `browser_manager.py`, directly addresses a class of stale-tab issues GreedySearch extractors may hit.
|
|
65
|
+
2. **Recoverable navigation error detection** — the marker list (websocket, target closed, etc.) is a useful pattern.
|
|
66
|
+
3. **Stealth test suite URLs** — CreepJS, Sannysoft, Intoli are the right fingerprints to validate against.
|
|
67
|
+
|
|
68
|
+
**What GreedySearch already has that stealth-browser-mcp does NOT** (per AGENTS.md design notes, not empirical test):
|
|
69
|
+
|
|
70
|
+
- Aggressive canvas noise / console guards / native-code stringifying
|
|
71
|
+
- `Target.createTarget` for less-detectable tab creation
|
|
72
|
+
- Headless → visible recovery with cookie caching
|
|
73
|
+
- Parallel multi-engine extraction with foregrounded tabs
|
|
74
|
+
- Single-eval stream waits (the "Node polling vs single-eval" rule)
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 2. Architecture Patterns
|
|
79
|
+
|
|
80
|
+
### 2.1 Modular Tool Section System
|
|
81
|
+
|
|
82
|
+
**File:** `src/server.py` — `@section_tool` decorator, `SECTION_TOOLS` dict, `apply_disabled_sections()`
|
|
83
|
+
|
|
84
|
+
The repo uses a clean section-based tool registration pattern:
|
|
85
|
+
|
|
86
|
+
```python
|
|
87
|
+
SECTION_TOOLS: Dict[str, List[str]] = defaultdict(list)
|
|
88
|
+
|
|
89
|
+
def section_tool(section: str):
|
|
90
|
+
def decorator(func):
|
|
91
|
+
SECTION_TOOLS[section].append(func.__name__)
|
|
92
|
+
return mcp.tool(func)
|
|
93
|
+
return decorator
|
|
94
|
+
|
|
95
|
+
def apply_disabled_sections() -> None:
|
|
96
|
+
for section in sorted(DISABLED_SECTIONS):
|
|
97
|
+
for tool_name in SECTION_TOOLS.get(section, []):
|
|
98
|
+
try:
|
|
99
|
+
mcp.remove_tool(tool_name)
|
|
100
|
+
except Exception:
|
|
101
|
+
continue
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
This enables:
|
|
105
|
+
|
|
106
|
+
- `--minimal` mode (20 core tools instead of 97)
|
|
107
|
+
- `--disable-cdp-functions --disable-dynamic-hooks` selective disabling
|
|
108
|
+
- `--list-sections` introspection
|
|
109
|
+
|
|
110
|
+
**Confidence: Medium.** I read this pattern in the first half of `server.py`. The second half (not read) likely follows the same pattern for element-extraction/file-extraction/progressive-cloning tools. I assume the pattern is consistent but haven't verified.
|
|
111
|
+
|
|
112
|
+
**Relevance to GreedySearch:** Medium. Your parallel extractors already serve different engines; a section-based system could help `--fast` mode disable expensive subsystems, but it's an organizational pattern, not a performance win.
|
|
113
|
+
|
|
114
|
+
### 2.2 Multi-Instance Browser Manager
|
|
115
|
+
|
|
116
|
+
**File:** `src/browser_manager.py` — `BrowserManager` class (read in full)
|
|
117
|
+
|
|
118
|
+
The `BrowserManager` manages multiple browser instances via a `Dict[str, dict]` keyed by UUID:
|
|
119
|
+
|
|
120
|
+
```python
|
|
121
|
+
self._instances: Dict[str, dict] = {}
|
|
122
|
+
# Each entry: {
|
|
123
|
+
# 'browser': Browser,
|
|
124
|
+
# 'tab': Tab,
|
|
125
|
+
# 'instance': BrowserInstance,
|
|
126
|
+
# 'options': BrowserOptions,
|
|
127
|
+
# 'navigation_count': int,
|
|
128
|
+
# 'idle_timeout_seconds': int,
|
|
129
|
+
# 'spawn_diagnostics': dict,
|
|
130
|
+
# 'network_data': []
|
|
131
|
+
# }
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Key methods:
|
|
135
|
+
|
|
136
|
+
- `touch_instance()` — updates last-activity timestamp
|
|
137
|
+
- `cleanup_inactive()` — closes instances past their idle timeout
|
|
138
|
+
- `start_idle_reaper()` — background asyncio task for periodic cleanup
|
|
139
|
+
- `get_navigation_tab()` — validates tab health before use, recycles stale tabs
|
|
140
|
+
|
|
141
|
+
**Confidence: High** (read the full file).
|
|
142
|
+
|
|
143
|
+
**Relevance to GreedySearch:** Low currently — you use a singleton Chrome. If multi-instance (different profiles per engine) is ever needed, this pattern is directly reusable, but it's a YAGNI risk until that need exists.
|
|
144
|
+
|
|
145
|
+
### 2.3 FastMCP Server with Lifespan Management
|
|
146
|
+
|
|
147
|
+
**File:** `src/server.py` — `app_lifespan()`
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
@asynccontextmanager
|
|
151
|
+
async def app_lifespan(server):
|
|
152
|
+
try:
|
|
153
|
+
await browser_manager.start_idle_reaper()
|
|
154
|
+
yield
|
|
155
|
+
finally:
|
|
156
|
+
await browser_manager.stop_idle_reaper()
|
|
157
|
+
await browser_manager.close_all()
|
|
158
|
+
process_cleanup._cleanup_all_tracked()
|
|
159
|
+
persistent_storage.clear_all()
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Clean startup → serve → shutdown lifecycle.
|
|
163
|
+
|
|
164
|
+
**Confidence: High.**
|
|
165
|
+
|
|
166
|
+
**Relevance to GreedySearch:** Low. Your `bin/launch.mjs` and `bin/search.mjs` already do this manually. The pattern is worth noting for any persistent server mode but doesn't change your current architecture.
|
|
167
|
+
|
|
168
|
+
### 2.4 Response Handler for Large Data
|
|
169
|
+
|
|
170
|
+
**File:** `src/response_handler.py` (referenced in `take_screenshot`, not read directly)
|
|
171
|
+
|
|
172
|
+
The `take_screenshot` tool auto-detects when base64 data exceeds ~20K estimated tokens and saves to file instead, returning a message instructing the AI agent to use the Read tool. This prevents context window overflow.
|
|
173
|
+
|
|
174
|
+
**Confidence: Low.** I inferred this from the `take_screenshot` implementation in `server.py` — the `response_handler.handle_response()` call and the explicit 20K-token threshold check. I did not read `response_handler.py` itself.
|
|
175
|
+
|
|
176
|
+
**Relevance to GreedySearch:** Low. Your research bundle writing already handles large output via disk writes. The auto-detection + file fallback is a clean pattern but already implicit in your architecture.
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## 3. Stealth Techniques
|
|
181
|
+
|
|
182
|
+
### 3.1 What Stealth Browser MCP Does
|
|
183
|
+
|
|
184
|
+
The stealth comes primarily from **nodriver** (not raw puppeteer/playwright). nodriver provides:
|
|
185
|
+
|
|
186
|
+
- Direct CDP protocol (no WebDriver protocol — less detectable)
|
|
187
|
+
- Auto-generated random user data dirs (`uc_*` temp profiles)
|
|
188
|
+
- `navigator.webdriver` set to `undefined`
|
|
189
|
+
- Removal of automation-controlling command-line flags
|
|
190
|
+
- Real Chrome/Chromium/Edge (not bundled Chromium)
|
|
191
|
+
|
|
192
|
+
The codebase itself does **not** inject custom stealth scripts. No `Page.addScriptToEvaluateOnNewDocument` patches, no canvas noise, no `navigator.plugins` overrides visible in the files I read. The stealth is purely from nodriver's defaults plus:
|
|
193
|
+
|
|
194
|
+
- `check_browser_executable()` — auto-detects the real browser (Chrome/Edge/Chromium)
|
|
195
|
+
- `merge_browser_args()` — strips conflicting args
|
|
196
|
+
- Sandbox auto-detection (root/container handling)
|
|
197
|
+
|
|
198
|
+
**Confidence: Medium-High.** I read `browser_manager.py` fully and confirmed no `addScriptToEvaluateOnNewDocument` calls. I did not search exhaustively across the second half of `server.py` or every submodule, so a stealth patch hidden in one of the unread files is possible but unlikely given the pattern of tool organization.
|
|
199
|
+
|
|
200
|
+
### 3.2 Comparison with GreedySearch-pi
|
|
201
|
+
|
|
202
|
+
This table is **design-intent vs design-intent**, not empirical test vs empirical test:
|
|
203
|
+
|
|
204
|
+
| Feature | Stealth Browser MCP | GreedySearch-pi (per AGENTS.md) |
|
|
205
|
+
|---------|---------------------|----------------------------------|
|
|
206
|
+
| Browser driver | nodriver (CDP) | CDP directly |
|
|
207
|
+
| `navigator.webdriver` | Undefined (nodriver default) | Undefined + native-code stringify |
|
|
208
|
+
| Canvas noise | None visible | Stable per-page noise |
|
|
209
|
+
| Console guards | None visible | Fire-and-forget for Bing |
|
|
210
|
+
| `navigator.plugins` | Default Chrome | Patched for consistency |
|
|
211
|
+
| `navigator.mediaDevices` | Default Chrome | Patched for consistency |
|
|
212
|
+
| `navigator.userAgentData` | Default Chrome | Patched for consistency |
|
|
213
|
+
| Tab creation | `uc.start()` → main tab | `Target.createTarget` (less detectable) |
|
|
214
|
+
| Headless detection avoidance | nodriver defaults | Aggressive patches |
|
|
215
|
+
| Visible recovery | Not implemented | Bing, Perplexity, ChatGPT, Semantic Scholar, Logically |
|
|
216
|
+
|
|
217
|
+
**Important caveat:** stealth-browser-mcp has empirical test results in `STEALTH_TESTS.md`:
|
|
218
|
+
|
|
219
|
+
- 0% headless, 0% stealth on CreepJS
|
|
220
|
+
- 20/20 on Sannysoft Bot Detection
|
|
221
|
+
- All checks passed on Intoli
|
|
222
|
+
|
|
223
|
+
GreedySearch's `AGENTS.md` describes the stealth patches but does **not** include equivalent empirical test numbers. So in **measured evidence**, stealth-browser-mcp currently wins — I just inferred GreedySearch's design is *deeper* from the description, not from test data.
|
|
224
|
+
|
|
225
|
+
**Confidence: Medium.** I have full test data for one side and design intent for the other.
|
|
226
|
+
|
|
227
|
+
### 3.3 What to Adopt (with caveats)
|
|
228
|
+
|
|
229
|
+
The **multi-platform browser auto-detection** from `platform_utils.py` is worth porting. It apparently handles:
|
|
230
|
+
|
|
231
|
+
- Microsoft Edge (msedge.exe on Windows, microsoft-edge on Linux)
|
|
232
|
+
- Multiple Chrome install paths
|
|
233
|
+
- macOS .app bundles
|
|
234
|
+
- Linux snap/flatpak installations
|
|
235
|
+
|
|
236
|
+
**Confidence: Low-Medium.** I did not read `platform_utils.py` in full. I saw the function name `check_browser_executable` imported into `browser_manager.py` and the high-level description in their README mentions "Windows, macOS, Linux, and Docker; supports Chrome, Chromium, and Edge." The actual implementation details are inferred.
|
|
237
|
+
|
|
238
|
+
---
|
|
239
|
+
|
|
240
|
+
## 4. CDP Patterns
|
|
241
|
+
|
|
242
|
+
### 4.1 Navigation with Tab Recycling
|
|
243
|
+
|
|
244
|
+
**File:** `src/browser_manager.py` — `navigate()`, `get_navigation_tab()`, `_replace_main_tab()` (read in full)
|
|
245
|
+
|
|
246
|
+
```python
|
|
247
|
+
NAVIGATION_RECYCLE_THRESHOLD = 25
|
|
248
|
+
|
|
249
|
+
async def get_navigation_tab(self, instance_id):
|
|
250
|
+
data = await self.get_instance(instance_id)
|
|
251
|
+
navigation_count = data.get("navigation_count", 0)
|
|
252
|
+
|
|
253
|
+
# Recycle tab after threshold to prevent memory leaks
|
|
254
|
+
if navigation_count >= self.NAVIGATION_RECYCLE_THRESHOLD:
|
|
255
|
+
return await self._replace_main_tab(instance_id, reason="recycle threshold")
|
|
256
|
+
|
|
257
|
+
# Validate tracked tab health
|
|
258
|
+
await browser.update_targets()
|
|
259
|
+
tracked_target_id = self._get_tab_target_id(tracked_tab)
|
|
260
|
+
# ... find matching tab in browser.tabs
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
Recoverable navigation errors trigger automatic retry with a fresh tab:
|
|
264
|
+
|
|
265
|
+
```python
|
|
266
|
+
@staticmethod
|
|
267
|
+
def _is_recoverable_navigation_error(error):
|
|
268
|
+
recoverable_markers = (
|
|
269
|
+
"connection dropped", "connection closed", "connection lost",
|
|
270
|
+
"websocket", "target closed", "target crashed",
|
|
271
|
+
"session closed", "invalid state", "not attached",
|
|
272
|
+
)
|
|
273
|
+
return any(marker in message for marker in recoverable_markers)
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
**Confidence: High.** I read the full `navigate()` and `_replace_main_tab()` implementations. The error recovery wraps the navigation in a 2-attempt loop with the second attempt using a fresh tab.
|
|
277
|
+
|
|
278
|
+
**Relevance to GreedySearch: Likely High, but with a caveat.**
|
|
279
|
+
|
|
280
|
+
Your AGENTS.md describes stale-tab symptoms:
|
|
281
|
+
|
|
282
|
+
- "ChatGPT background tabs can stall and leave only citation/header stubs in the DOM"
|
|
283
|
+
- "Bing copy button exists in DOM before React hydrates its click handler"
|
|
284
|
+
- "Perplexity's anti-bot system detects aggressive stealth patches"
|
|
285
|
+
|
|
286
|
+
The **first two are potentially related to tab age/health**, but they may also be caused by:
|
|
287
|
+
|
|
288
|
+
- Background-tab throttling (foregrounding fixes this — your AGENTS.md already mentions `Page.bringToFront`)
|
|
289
|
+
- React hydration race (timing, not tab age)
|
|
290
|
+
- CDP contention from parallel extractors
|
|
291
|
+
|
|
292
|
+
Tab recycling might help, might not. **The fix is not guaranteed to address the specific symptoms unless the root cause is tab-age-related.** I cannot determine root cause from either codebase.
|
|
293
|
+
|
|
294
|
+
### 4.2 Timezone Override via CDP
|
|
295
|
+
|
|
296
|
+
**File:** `src/browser_manager.py` — `_apply_timezone_override()` (read in full)
|
|
297
|
+
|
|
298
|
+
```python
|
|
299
|
+
@staticmethod
|
|
300
|
+
async def _apply_timezone_override(*, tab, timezone_id):
|
|
301
|
+
if not timezone_id:
|
|
302
|
+
return None
|
|
303
|
+
trimmed_timezone = timezone_id.strip()
|
|
304
|
+
if not trimmed_timezone:
|
|
305
|
+
return None
|
|
306
|
+
await tab.send(uc.cdp.emulation.set_timezone_override(timezone_id=trimmed_timezone))
|
|
307
|
+
return trimmed_timezone
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
**Confidence: High.**
|
|
311
|
+
|
|
312
|
+
**Relevance to GreedySearch: Medium.** You have `/set-greedy-locale` but I did not verify whether it applies timezone via CDP. This is a small, safe addition either way.
|
|
313
|
+
|
|
314
|
+
### 4.3 Wait Conditions with Timeout Budget
|
|
315
|
+
|
|
316
|
+
**File:** `src/browser_manager.py` — `_wait_for_navigation_condition()` (read in full)
|
|
317
|
+
|
|
318
|
+
```python
|
|
319
|
+
@staticmethod
|
|
320
|
+
async def _wait_for_navigation_condition(tab, wait_until, timeout_seconds):
|
|
321
|
+
if timeout_seconds <= 0:
|
|
322
|
+
raise asyncio.TimeoutError("Navigation wait budget exhausted")
|
|
323
|
+
|
|
324
|
+
if wait_until == "domcontentloaded":
|
|
325
|
+
await asyncio.wait_for(tab.wait(uc.cdp.page.DomContentEventFired), timeout=timeout_seconds)
|
|
326
|
+
elif wait_until == "networkidle":
|
|
327
|
+
await asyncio.sleep(min(timeout_seconds, 2.0)) # Note: this is a lazy approximation
|
|
328
|
+
else:
|
|
329
|
+
await asyncio.wait_for(tab.wait(uc.cdp.page.LoadEventFired), timeout=timeout_seconds)
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
The timeout budget is split across navigation steps (navigate → wait condition → get URL/title):
|
|
333
|
+
|
|
334
|
+
```python
|
|
335
|
+
await asyncio.wait_for(tab.get(url), timeout=timeout_seconds)
|
|
336
|
+
elapsed = time.monotonic() - start_time
|
|
337
|
+
await self._wait_for_navigation_condition(tab, wait_until, timeout_seconds - elapsed)
|
|
338
|
+
# ... then uses remaining budget for tab.evaluate()
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
**Confidence: High.**
|
|
342
|
+
|
|
343
|
+
**Relevance to GreedySearch: Medium.** The budget-splitting pattern is good. The `networkidle` implementation is a weak `asyncio.sleep(2)` — not real network-idle detection. So port the budget-splitting idea, not the `networkidle` impl.
|
|
344
|
+
|
|
345
|
+
### 4.4 Direct CDP Command Execution
|
|
346
|
+
|
|
347
|
+
**File:** `src/cdp_function_executor.py` (imported but not read)
|
|
348
|
+
|
|
349
|
+
The repo exposes raw CDP execution as a tool — `execute_cdp_command('Page.navigate', {'url': '...'})`. This is a **security risk** (any AI agent can trigger any CDP command) but useful for debugging.
|
|
350
|
+
|
|
351
|
+
**Confidence: Low.** I only saw the import and tool name in `server.py`. The actual implementation and security boundary are unverified.
|
|
352
|
+
|
|
353
|
+
**Relevance to GreedySearch: Negative.** Your `bin/cdp-greedy.mjs` already does this in a controlled way for development. Exposing it as a server-level tool is over-permissive for your use case.
|
|
354
|
+
|
|
355
|
+
---
|
|
356
|
+
|
|
357
|
+
## 5. Network Interception
|
|
358
|
+
|
|
359
|
+
### 5.1 Request/Response Capture
|
|
360
|
+
|
|
361
|
+
**File:** `src/network_interceptor.py` (read in full)
|
|
362
|
+
|
|
363
|
+
Uses `cdp.network.enable()` + event handlers to capture all network traffic:
|
|
364
|
+
|
|
365
|
+
```python
|
|
366
|
+
tab.add_handler(uc.cdp.network.RequestWillBeSent,
|
|
367
|
+
lambda event: asyncio.create_task(self._on_request(event, instance_id)))
|
|
368
|
+
tab.add_handler(uc.cdp.network.ResponseReceived,
|
|
369
|
+
lambda event: asyncio.create_task(self._on_response(event, instance_id, tab)))
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
Features I confirmed in the code:
|
|
373
|
+
|
|
374
|
+
- Resource type filtering (include/exclude lists)
|
|
375
|
+
- Body capture via `get_response_body()`
|
|
376
|
+
- Search with pagination (URL pattern, method, status code, body content)
|
|
377
|
+
- Export/import to/from JSON
|
|
378
|
+
- Cookie management (get/set/clear)
|
|
379
|
+
- Network condition emulation (offline, latency, throughput)
|
|
380
|
+
|
|
381
|
+
**Confidence: High** for the code I read. I did not test live.
|
|
382
|
+
|
|
383
|
+
**Relevance to GreedySearch: Low.**
|
|
384
|
+
|
|
385
|
+
This is a real subsystem but it solves a different problem. GreedySearch extracts answer text from rendered AI chat pages — clipboard interception and DOM text are faster, more reliable, and avoid:
|
|
386
|
+
|
|
387
|
+
- Storing potentially sensitive request/response bodies
|
|
388
|
+
- LLM context bloat from API payloads
|
|
389
|
+
- Anti-bot detection from intercepted fetch events (some engines detect `Fetch.enable`)
|
|
390
|
+
|
|
391
|
+
The only relevant sub-feature is resource blocking (next section).
|
|
392
|
+
|
|
393
|
+
### 5.2 Resource Blocking
|
|
394
|
+
|
|
395
|
+
**File:** `src/network_interceptor.py` — `setup_interception()` (read in full)
|
|
396
|
+
|
|
397
|
+
```python
|
|
398
|
+
resource_patterns = {
|
|
399
|
+
'image': ['*.jpg', '*.jpeg', '*.png', '*.gif', '*.webp', '*.svg'],
|
|
400
|
+
'stylesheet': ['*.css'],
|
|
401
|
+
'font': ['*.woff', '*.woff2', '*.ttf', '*.otf', '*.eot'],
|
|
402
|
+
'script': ['*.js', '*.mjs'],
|
|
403
|
+
'media': ['*.mp4', '*.mp3', '*.wav', '*.avi', '*.webm']
|
|
404
|
+
}
|
|
405
|
+
# Uses network.set_blocked_ur_ls() to block
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
**Confidence: High.**
|
|
409
|
+
|
|
410
|
+
**Relevance to GreedySearch: Medium-Low.** Blocking images/fonts on AI chat pages *would* speed up navigation, but:
|
|
411
|
+
|
|
412
|
+
- AI chat UIs depend on JS frameworks (React/Vue) — blocking scripts would break them
|
|
413
|
+
- Your extractors already use tight timeouts (20s nav, 35-60s engine budgets)
|
|
414
|
+
- The speedup is probably small for text-heavy AI pages (most resources are already small)
|
|
415
|
+
|
|
416
|
+
**Worth measuring before adopting.**
|
|
417
|
+
|
|
418
|
+
### 5.3 Dynamic AI-Generated Hook System
|
|
419
|
+
|
|
420
|
+
**File:** `src/dynamic_hook_system.py` (read in full — ~600 lines)
|
|
421
|
+
|
|
422
|
+
The most architecturally ambitious feature: AI-generated Python functions that intercept `fetch.RequestPaused` events in real-time.
|
|
423
|
+
|
|
424
|
+
**How it works:**
|
|
425
|
+
|
|
426
|
+
1. AI generates Python function code with `process_request(request)` signature
|
|
427
|
+
2. Code is compiled via `exec()` into a restricted namespace
|
|
428
|
+
3. On each `RequestPaused` event, matching hooks execute and return `HookAction`
|
|
429
|
+
4. Actions: `continue`, `block`, `redirect`, `fulfill`, `modify`
|
|
430
|
+
|
|
431
|
+
**Security analysis — the `exec()` sandbox is weak (verified in code):**
|
|
432
|
+
|
|
433
|
+
```python
|
|
434
|
+
def _compile_function(self):
|
|
435
|
+
namespace = {
|
|
436
|
+
'HookAction': HookAction,
|
|
437
|
+
'datetime': datetime,
|
|
438
|
+
'fnmatch': fnmatch,
|
|
439
|
+
'__builtins__': {
|
|
440
|
+
'len': len, 'str': str, 'int': int, 'float': float,
|
|
441
|
+
'bool': bool, 'dict': dict, 'list': list, 'tuple': tuple,
|
|
442
|
+
'print': lambda *args: ...
|
|
443
|
+
}
|
|
444
|
+
}
|
|
445
|
+
exec(self.function_code, namespace)
|
|
446
|
+
return namespace['process_request']
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
The restricted `__builtins__` is the **only** sandbox. But `HookAction` is a real class exposed in the namespace. A motivated AI can do:
|
|
450
|
+
|
|
451
|
+
```python
|
|
452
|
+
def process_request(request):
|
|
453
|
+
return HookAction.__class__.__mro__[-1].__subclasses__() # -> <class 'type'> and all subclasses
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
From there, standard Python jailbreak: find `subprocess.Popen` or `os._wrap_close` in the subclasses, access its globals, get `os.system`. This is a textbook Python `exec()` escape.
|
|
457
|
+
|
|
458
|
+
**Confidence: High** for the security finding (I read the compile function and the namespace construction).
|
|
459
|
+
|
|
460
|
+
**Relevance to GreedySearch: NONE.** This pattern should not be ported:
|
|
461
|
+
|
|
462
|
+
1. **Security:** `exec()` from AI prompts is dangerous in any context
|
|
463
|
+
2. **Complexity:** You don't need real-time request interception
|
|
464
|
+
3. **Maintenance:** `fetch.RequestPaused` is a complex CDP domain — the code has bugs (e.g., `add_handler` with a lambda creating a task on every event has no backpressure; if many requests pause simultaneously, you get unbounded task spawning)
|
|
465
|
+
4. **Use case mismatch:** You want the AI to answer your question, not to be your request-interception programmer
|
|
466
|
+
|
|
467
|
+
---
|
|
468
|
+
|
|
469
|
+
## 6. Process Lifecycle & Cleanup
|
|
470
|
+
|
|
471
|
+
### 6.1 Process Cleanup Manager
|
|
472
|
+
|
|
473
|
+
**File:** `src/process_cleanup.py` — 30K chars (largest file in the project)
|
|
474
|
+
|
|
475
|
+
**Confidence: Low.** I fetched the file but did not read it in full. The following characterization is based on:
|
|
476
|
+
|
|
477
|
+
- File size (30K chars is substantial for a Python file)
|
|
478
|
+
- Method names in the header (which I did read)
|
|
479
|
+
- Imports referenced in `browser_manager.py` (`process_cleanup.track_browser_process`, `kill_browser_process`, `finalize_browser_process`, `cleanup_deferred_profiles`)
|
|
480
|
+
|
|
481
|
+
I do not have code-level confirmation of:
|
|
482
|
+
|
|
483
|
+
- Whether the graceful SIGTERM → SIGKILL escalation actually works on Windows
|
|
484
|
+
- Whether the deferred profile cleanup handles Windows file-lock delays correctly
|
|
485
|
+
- The actual orphan-detection logic at startup
|
|
486
|
+
|
|
487
|
+
**Claims to treat as provisional:** "Sophisticated process tracking," "orphan detection at startup," "graceful shutdown with timeout escalation," "temp profile cleanup on Windows."
|
|
488
|
+
|
|
489
|
+
**What I can say with confidence:**
|
|
490
|
+
|
|
491
|
+
- The file exists and is large
|
|
492
|
+
- It provides the API surface that `browser_manager.py` uses
|
|
493
|
+
- GreedySearch's equivalent (`bin/kill-visible.mjs`, `bin/launch.mjs --kill`) is simpler based on what I've seen
|
|
494
|
+
|
|
495
|
+
**Relevance to GreedySearch: Probably High, but I cannot confirm without reading the full file.**
|
|
496
|
+
|
|
497
|
+
### 6.2 Idle Reaper Pattern
|
|
498
|
+
|
|
499
|
+
**File:** `src/browser_manager.py` — `_run_idle_reaper()` (read in full)
|
|
500
|
+
|
|
501
|
+
```python
|
|
502
|
+
async def _run_idle_reaper(self):
|
|
503
|
+
try:
|
|
504
|
+
while True:
|
|
505
|
+
await asyncio.sleep(self._idle_reaper_interval_seconds)
|
|
506
|
+
closed_count = await self.cleanup_inactive()
|
|
507
|
+
finalized_profiles = process_cleanup.cleanup_deferred_profiles()
|
|
508
|
+
except asyncio.CancelledError:
|
|
509
|
+
raise
|
|
510
|
+
```
|
|
511
|
+
|
|
512
|
+
Configurable via env vars:
|
|
513
|
+
|
|
514
|
+
- `BROWSER_IDLE_TIMEOUT` (default 600s, 0 to disable)
|
|
515
|
+
- `BROWSER_IDLE_REAPER_INTERVAL` (default 60s)
|
|
516
|
+
- `BROWSER_ORPHAN_PROFILE_MAX_AGE` (default 21600s = 6h)
|
|
517
|
+
|
|
518
|
+
**Confidence: High.**
|
|
519
|
+
|
|
520
|
+
**Relevance to GreedySearch: Low.** Your Chrome lifecycle is session-based. This pattern only matters for persistent server mode.
|
|
521
|
+
|
|
522
|
+
### 6.3 Instance Timeout Resolution
|
|
523
|
+
|
|
524
|
+
**File:** `src/browser_manager.py` — `_resolve_idle_timeout_seconds()` (read in full)
|
|
525
|
+
|
|
526
|
+
```python
|
|
527
|
+
def _resolve_idle_timeout_seconds(self, override):
|
|
528
|
+
if self._idle_timeout_seconds_default == 0:
|
|
529
|
+
return 0
|
|
530
|
+
if override is None:
|
|
531
|
+
return self._idle_timeout_seconds_default
|
|
532
|
+
return max(int(override), 0)
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
**Confidence: High.**
|
|
536
|
+
|
|
537
|
+
**Relevance to GreedySearch: Low** (not applicable to your architecture).
|
|
538
|
+
|
|
539
|
+
---
|
|
540
|
+
|
|
541
|
+
## 7. DOM Interaction Patterns
|
|
542
|
+
|
|
543
|
+
### 7.1 File Upload with Wrapper Resolution
|
|
544
|
+
|
|
545
|
+
**File:** `src/dom_handler.py` — `file_upload()` (read in full)
|
|
546
|
+
|
|
547
|
+
```python
|
|
548
|
+
tag_name = element.tag_name
|
|
549
|
+
input_type = (element.attrs.get('type') or '').lower()
|
|
550
|
+
if tag_name != 'input' or input_type != 'file':
|
|
551
|
+
inner_input = await element.query_selector('input[type="file"]')
|
|
552
|
+
if inner_input:
|
|
553
|
+
element = inner_input
|
|
554
|
+
```
|
|
555
|
+
|
|
556
|
+
**Confidence: High.**
|
|
557
|
+
|
|
558
|
+
**Relevance to GreedySearch: Low.** You're not doing file upload. The "resolve the actual target when a wrapper is found" pattern is general, but your specific copy-button extraction is already more sophisticated (data-attribute selectors, hydration race handling, etc.).
|
|
559
|
+
|
|
560
|
+
### 7.2 Human-like Typing
|
|
561
|
+
|
|
562
|
+
**File:** `src/dom_handler.py` — `type_text()`, `paste_text()` (read in full)
|
|
563
|
+
|
|
564
|
+
```python
|
|
565
|
+
# Bulk paste via CDP (fast)
|
|
566
|
+
await tab.send(cdp.input_.insert_text(text))
|
|
567
|
+
|
|
568
|
+
# Human-like typing (slow, detectable)
|
|
569
|
+
for char in text:
|
|
570
|
+
await element.send_keys(char)
|
|
571
|
+
await asyncio.sleep(delay_ms / 1000)
|
|
572
|
+
```
|
|
573
|
+
|
|
574
|
+
**Confidence: High.**
|
|
575
|
+
|
|
576
|
+
**Relevance to GreedySearch: Low.** Your extractors use URL-parameter queries (e.g., `https://perplexity.ai/?q=...`) or clipboard injection, not character-by-character typing into chat inputs. If you ever needed to type into a chat, the `cdp.input_.insert_text()` approach is already what you'd want.
|
|
577
|
+
|
|
578
|
+
### 7.3 Paginated Debug Logging
|
|
579
|
+
|
|
580
|
+
**File:** `src/debug_logger.py` (not read — only referenced)
|
|
581
|
+
|
|
582
|
+
**Confidence: Very Low.** I saw the function names `get_debug_view_paginated`, `clear_debug_view_safe`, `export_to_file_paginated` referenced in `server.py` but did not read the implementation. Claims about JSON/pickle/gzip-pickle format support are inferred from the `format` parameter in `export_debug_logs`.
|
|
583
|
+
|
|
584
|
+
**Relevance to GreedySearch: Low.** You have your own logging system.
|
|
585
|
+
|
|
586
|
+
---
|
|
587
|
+
|
|
588
|
+
## 8. Element Cloning
|
|
589
|
+
|
|
590
|
+
### 8.1 CDP-Based Element Cloner
|
|
591
|
+
|
|
592
|
+
**Files:** `src/cdp_element_cloner.py`, `src/comprehensive_element_cloner.py`, `src/progressive_element_cloner.py`, `src/element_cloner.py`, `src/file_based_element_cloner.py` — 6+ files (not read)
|
|
593
|
+
|
|
594
|
+
**Confidence: Very Low.** I saw the imports in `server.py` and the tool names (`extract_element_styles`, `extract_element_structure`, etc.) but did not read any of the implementation. The "300+ CSS properties," "React/Vue event detector" claims are from the README, not code review.
|
|
595
|
+
|
|
596
|
+
**Relevance to GreedySearch: NONE.** Element cloning is a completely different use case from answer extraction.
|
|
597
|
+
|
|
598
|
+
### 8.2 JS Injectors for Element Extraction
|
|
599
|
+
|
|
600
|
+
**Files:** `src/js/*.js` (7 files, not read)
|
|
601
|
+
|
|
602
|
+
**Confidence: Very Low.** Listed in the file tree only.
|
|
603
|
+
|
|
604
|
+
**Pattern note:** The approach of injecting full JS scripts and then calling them from Python is interesting. GreedySearch already does this for clipboard interception (`injectClipboardInterceptor`) and stream waits. Not new ground.
|
|
605
|
+
|
|
606
|
+
---
|
|
607
|
+
|
|
608
|
+
## 9. Security Considerations
|
|
609
|
+
|
|
610
|
+
### 9.1 Hook System Sandbox Weakness
|
|
611
|
+
|
|
612
|
+
**Severity: HIGH (verified)**
|
|
613
|
+
|
|
614
|
+
I read `dynamic_hook_system.py` in full. The `exec()` sandbox is weak:
|
|
615
|
+
|
|
616
|
+
- `__builtins__` is restricted to ~10 functions
|
|
617
|
+
- But `HookAction` (a real class) is in the namespace
|
|
618
|
+
- Standard Python jailbreak: `HookAction.__class__.__mro__[-1].__subclasses__()` returns `<class 'type'>` and all loaded subclasses, including `os._wrap_close`, `subprocess.Popen`, etc.
|
|
619
|
+
- The `print` replacement is a lambda — harmless but also not a real sandbox
|
|
620
|
+
|
|
621
|
+
**Verdict:** Unsafe for production. GreedySearch-pi should not adopt this pattern. Documented in the repo as "AI-generated Python functions" with a vague note about safety — the safety is not real.
|
|
622
|
+
|
|
623
|
+
### 9.2 CDP Function Executor
|
|
624
|
+
|
|
625
|
+
**Severity: MEDIUM (claimed, not verified)**
|
|
626
|
+
|
|
627
|
+
The `execute_cdp_command()` tool allegedly exposes ALL CDP commands. I did not read `cdp_function_executor.py`, so I cannot confirm the exact command allowlist (if any). The risk model is:
|
|
628
|
+
|
|
629
|
+
- If commands are unfiltered: an AI can call `Browser.grantPermissions`, `Storage.clearDataForOrigin`, etc.
|
|
630
|
+
- If commands are allowlisted: the risk depends on what's allowed
|
|
631
|
+
|
|
632
|
+
**Verdict:** Without reading the implementation, I cannot rate this accurately. Treat as "possible risk" until verified.
|
|
633
|
+
|
|
634
|
+
**Relevance to GreedySearch: Low.** Your `bin/cdp-greedy.mjs` is a CLI for development, not exposed to AI agents.
|
|
635
|
+
|
|
636
|
+
### 9.3 File Upload Security
|
|
637
|
+
|
|
638
|
+
**File:** `src/file_upload_security.py` (not read)
|
|
639
|
+
|
|
640
|
+
**Confidence: Low.** I saw the `validate_upload_paths` function imported into `dom_handler.py` and the env var `BROWSER_FILE_UPLOAD_ALLOWED_DIRS` documented in README, but did not read the implementation.
|
|
641
|
+
|
|
642
|
+
**Relevance to GreedySearch: N/A** (no file upload feature).
|
|
643
|
+
|
|
644
|
+
### 9.4 HTTP Transport Auth
|
|
645
|
+
|
|
646
|
+
**File:** `src/http_security.py` (not read)
|
|
647
|
+
|
|
648
|
+
**Confidence: Low.** Documented in README: `STEALTH_BROWSER_MCP_AUTH_TOKEN` enables bearer-token auth for HTTP transport. `stdio` transport has no auth (safe for local use).
|
|
649
|
+
|
|
650
|
+
**Relevance to GreedySearch: N/A** (you use stdio via Pi).
|
|
651
|
+
|
|
652
|
+
---
|
|
653
|
+
|
|
654
|
+
## 10. Test & Validation Practices
|
|
655
|
+
|
|
656
|
+
### 10.1 STEALTH_TESTS.md Methodology
|
|
657
|
+
|
|
658
|
+
**File:** `STEALTH_TESTS.md` (read in full)
|
|
659
|
+
|
|
660
|
+
The empirical testing approach is solid and worth adopting:
|
|
661
|
+
|
|
662
|
+
| Test | Result | URL |
|
|
663
|
+
|------|--------|-----|
|
|
664
|
+
| CreepJS fingerprint analysis | 0% headless, 0% stealth detection | `https://abrahamjuliot.github.io/creepjs/` |
|
|
665
|
+
| Sannysoft Bot Detection | 20/20 (all checks passed) | `https://bot.sannysoft.com/` |
|
|
666
|
+
| Intoli Headless Detection | All parameters cleared | `https://intoli.com/blog/not-possible-to-block-chrome-headless/chrome-headless-test.html` |
|
|
667
|
+
| Cloudflare Challenge | Automated bypass | `https://nowsecure.nl/` |
|
|
668
|
+
| X.com login wall | DOM manipulation bypass | `https://x.com/` |
|
|
669
|
+
|
|
670
|
+
**Test date:** February 10, 2026
|
|
671
|
+
|
|
672
|
+
**Confidence: High** for the URLs and methodology. The specific pass rates are from their tests, not mine.
|
|
673
|
+
|
|
674
|
+
**Relevance to GreedySearch: High.**
|
|
675
|
+
|
|
676
|
+
This is the most concrete, actionable finding. A `npm run stealth-check` script that:
|
|
677
|
+
|
|
678
|
+
1. Spawns GreedySearch Chrome (headless + visible)
|
|
679
|
+
2. Navigates to each test URL
|
|
680
|
+
3. Extracts the fingerprint report
|
|
681
|
+
4. Compares against expected "clean" values
|
|
682
|
+
5. Fails the CI run if regressions appear
|
|
683
|
+
|
|
684
|
+
…would catch stealth-patch regressions early. The test URLs themselves are the most valuable part — they are the right fingerprints to validate against.
|
|
685
|
+
|
|
686
|
+
### 10.2 COMPARISON.md
|
|
687
|
+
|
|
688
|
+
**File:** `COMPARISON.md` (read in full)
|
|
689
|
+
|
|
690
|
+
Compares against Playwright MCP across:
|
|
691
|
+
|
|
692
|
+
- Cloudflare / Queue-It bypass
|
|
693
|
+
- Banking / Gov portals
|
|
694
|
+
- Social media automation
|
|
695
|
+
- UI element cloning
|
|
696
|
+
- Network debugging
|
|
697
|
+
- API reverse engineering
|
|
698
|
+
- Dynamic hook system
|
|
699
|
+
- Modular architecture
|
|
700
|
+
- Total tools (97 vs ~20)
|
|
701
|
+
|
|
702
|
+
**Confidence: High** for the content, **Low** for the objectivity (it's their own comparison doc, naturally favorable).
|
|
703
|
+
|
|
704
|
+
**Relevance to GreedySearch: Low.** You could write your own comparison doc vs other web search MCP tools, but this isn't a direct port.
|
|
705
|
+
|
|
706
|
+
---
|
|
707
|
+
|
|
708
|
+
## 11. Portable Code Patterns (confidence-rated)
|
|
709
|
+
|
|
710
|
+
This table is **replaced** from the original. Each row now includes a confidence rating based on how thoroughly I read the source.
|
|
711
|
+
|
|
712
|
+
| Pattern | Source file | Confidence in pattern | Confidence in priority | Notes |
|
|
713
|
+
|---------|-------------|----------------------|------------------------|-------|
|
|
714
|
+
| `_is_recoverable_navigation_error()` | `browser_manager.py` | High | Medium | Marker list is solid; may not fix specific GreedySearch symptoms |
|
|
715
|
+
| Tab recycling at `NAVIGATION_RECYCLE_THRESHOLD=25` | `browser_manager.py` | High | Medium-Low | Useful pattern but root cause of GreedySearch issues unclear |
|
|
716
|
+
| `_apply_timezone_override()` via CDP | `browser_manager.py` | High | Medium | Small, safe addition |
|
|
717
|
+
| Budget-splitting timeout pattern | `browser_manager.py` | High | Medium | Port the idea, not the `networkidle` impl (it's a 2s sleep) |
|
|
718
|
+
| `cdp.input_.insert_text()` for bulk paste | `dom_handler.py` | High | Low | GreedySearch already has equivalent approaches |
|
|
719
|
+
| `check_browser_executable()` (multi-platform) | `platform_utils.py` | Low | Medium | File not read in full; claims are inferred |
|
|
720
|
+
| `merge_browser_args()` (strip conflicting args) | `platform_utils.py` | Low | Low | Not read |
|
|
721
|
+
| Process cleanup with deferred profiles | `process_cleanup.py` | Low | Probably High | File not read in full; claims are size-based |
|
|
722
|
+
| Resource blocking patterns | `network_interceptor.py` | High | Low | Speedup uncertain; needs measurement |
|
|
723
|
+
| Section-based tool registration | `server.py` | Medium | Low | Organizational pattern, not a perf win |
|
|
724
|
+
| Stealth test URLs (CreepJS, Sannysoft, Intoli) | `STEALTH_TESTS.md` | High | High | Most actionable finding in the whole review |
|
|
725
|
+
|
|
726
|
+
### What I'm NOT Confident Enough to Recommend
|
|
727
|
+
|
|
728
|
+
- **Specific process cleanup mechanics** — the file is 30K chars; I read maybe 5%. The "graceful kill, orphan detection" claims could be right or could be a thin wrapper around `os.kill`.
|
|
729
|
+
- **`platform_utils.py` browser detection quality** — the function names suggest thoroughness, but I didn't read the implementations.
|
|
730
|
+
- **Multi-platform Edge/Chrome path coverage** — README claims Windows/macOS/Linux/Docker; I didn't verify the actual paths searched.
|
|
731
|
+
- **Dynamic hook system stability** — I read the code but didn't test it. There may be subtle bugs (e.g., the unbounded `asyncio.create_task` spawning on each `RequestPaused` event).
|
|
732
|
+
|
|
733
|
+
### What I Would Not Port Even If It Were Perfect
|
|
734
|
+
|
|
735
|
+
- **Dynamic hook system** — `exec()` of AI prompts is a security vulnerability regardless of the surrounding code quality.
|
|
736
|
+
- **Element cloning** — unrelated to your use case.
|
|
737
|
+
- **Raw CDP command exposure** — too permissive for a tool installed in user environments.
|
|
738
|
+
|
|
739
|
+
---
|
|
740
|
+
|
|
741
|
+
## 12. Summary Matrix
|
|
742
|
+
|
|
743
|
+
| Category | Score (1-5) | Confidence in score | Notes |
|
|
744
|
+
|----------|-------------|---------------------|-------|
|
|
745
|
+
| Architecture patterns | ⭐⭐⭐⭐⭐ | Medium | Section-based tools, multi-instance manager, FastMCP lifespan — verified for the parts I read |
|
|
746
|
+
| Stealth techniques | ⭐⭐ | Low-Medium | Their patches are minimal; my "GreedySearch wins" claim is design-intent, not empirical |
|
|
747
|
+
| CDP usage patterns | ⭐⭐⭐⭐ | High | Tab recycling, error recovery, wait conditions are concrete code |
|
|
748
|
+
| Network interception | ⭐⭐⭐⭐⭐ | High (for the code), Low (for "best-in-class" vs alternatives) | Rich feature set, but I haven't compared to Playwright HAR, Puppeteer, etc. |
|
|
749
|
+
| Dynamic hooks (AI→Python) | ⭐ | High | The pattern is interesting; the `exec()` sandbox is genuinely weak |
|
|
750
|
+
| Element cloning | ⭐⭐⭐⭐⭐ | Very Low | Score is for the feature itself, not my verification of it |
|
|
751
|
+
| Process lifecycle | ⭐⭐⭐⭐⭐ | Low | File is 30K chars but I read ~5% |
|
|
752
|
+
| Security patterns | ⭐⭐ | High for hook system, Low for others | Hook system is unsafe (verified); others not read |
|
|
753
|
+
| Testing methodology | ⭐⭐⭐⭐⭐ | High | Stealth test suite is directly portable |
|
|
754
|
+
| Documentation quality | ⭐⭐⭐⭐ | High | Good README, detailed env-var docs |
|
|
755
|
+
|
|
756
|
+
### Overall: ~3.5 / 5 (with wide confidence intervals)
|
|
757
|
+
|
|
758
|
+
The score is driven up by the directly-verifiable portable patterns (tab recycling, error recovery, test URLs) and down by the unverified claims (process cleanup, platform utilities). Net positive but not as definitive as the first version presented.
|
|
759
|
+
|
|
760
|
+
---
|
|
761
|
+
|
|
762
|
+
## 13. Recommendations
|
|
763
|
+
|
|
764
|
+
### Confident Recommendations (High Confidence)
|
|
765
|
+
|
|
766
|
+
1. **Adopt the stealth test URLs.** CreepJS, Sannysoft, and Intoli are the right fingerprints. Add a `npm run stealth-check` script that navigates to each and checks the result. This catches stealth-patch regressions even if you don't port any other code from this repo.
|
|
767
|
+
|
|
768
|
+
2. **Consider tab recycling for `all` mode.** Even if it doesn't fix the specific symptoms in AGENTS.md, a 25-navigation tab-recycle threshold is a defensive measure that costs little. Add it to `common.mjs` `getOrOpenTab`.
|
|
769
|
+
|
|
770
|
+
3. **Add the recoverable-navigation-error marker list.** The pattern of "retry with fresh tab on connection drop / websocket failure / target closed" is low-risk and catches a real failure mode.
|
|
771
|
+
|
|
772
|
+
### Probable Recommendations (Medium Confidence)
|
|
773
|
+
|
|
774
|
+
1. **Audit `process_cleanup.py` before deciding to port.** The file is large enough that it probably has value, but I cannot confirm without reading it. Recommendation: spend 30 minutes reading it before deciding.
|
|
775
|
+
|
|
776
|
+
2. **Add timezone override via CDP if you don't already.** Small, safe, and GreedySearch's `/set-greedy-locale` may or may not already do this — I didn't read the full GreedySearch source to confirm.
|
|
777
|
+
|
|
778
|
+
3. **Adopt the budget-splitting timeout pattern.** The `navigate → wait condition → get URL/title` budget split is a clean pattern. Don't copy the `networkidle = asyncio.sleep(2)` implementation — it's a placeholder, not real network-idle detection.
|
|
779
|
+
|
|
780
|
+
### Recommendations Against Porting (High Confidence)
|
|
781
|
+
|
|
782
|
+
1. **Do not port the dynamic hook system.** The `exec()` sandbox is genuinely weak. Standard Python jailbreak techniques work. If you want user-extensible network interception, do it with a typed schema (JSON request/response, not arbitrary Python).
|
|
783
|
+
|
|
784
|
+
2. **Do not port element cloning.** Unrelated to your use case.
|
|
785
|
+
|
|
786
|
+
3. **Do not expose raw CDP commands to AI agents.** Your `bin/cdp-greedy.mjs` is a fine developer tool. Don't add a server-level equivalent.
|
|
787
|
+
|
|
788
|
+
### Areas Where I Was Wrong or Overconfident in v1
|
|
789
|
+
|
|
790
|
+
- **"GreedySearch wins on stealth depth"** — I have full test data for stealth-browser-mcp and design intent for GreedySearch. In empirical terms, stealth-browser-mcp is currently better-validated. GreedySearch's design is plausibly deeper, but unproven.
|
|
791
|
+
- **"Process cleanup is directly portable"** — I made this claim without reading the file in full. The character count suggests complexity, but I cannot confirm the quality.
|
|
792
|
+
- **"3.5/5 overall"** — with the confidence intervals now explicit, the honest range is 3.0-4.0 depending on what the unread files actually contain.
|
|
793
|
+
- **The priority table in section 11** — I gave confident "High/Medium/Low" ratings without confidence labels. Some of those ratings were based on weak evidence (e.g., I rated process cleanup as High priority because the file is 30K chars, which is a poor proxy for quality).
|
|
794
|
+
|
|
795
|
+
---
|
|
796
|
+
|
|
797
|
+
## 14. What I'd Want to Read Before Being More Definitive
|
|
798
|
+
|
|
799
|
+
To upgrade this from "informed opinion" to "verified assessment":
|
|
800
|
+
|
|
801
|
+
1. **`process_cleanup.py` in full** — confirms or refutes the lifecycle claims
|
|
802
|
+
2. **`platform_utils.py` in full** — confirms multi-platform browser detection
|
|
803
|
+
3. **`dynamic_hook_ai_interface.py`** — may explain how AI is supposed to use the hook system
|
|
804
|
+
4. **`server.py` lines 1588-2969** — second half has more tool definitions
|
|
805
|
+
5. **Live testing** — run both GreedySearch and stealth-browser-mcp against the same test URLs and compare
|
|
806
|
+
|
|
807
|
+
Without those, treat the High-Confidence recommendations as solid and the Medium-Confidence ones as "probably worth a look, but verify before porting."
|