@apmantza/greedysearch-pi 2.1.3 → 2.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,6 +1,53 @@
1
1
  # Changelog
2
2
 
3
- ## [Unreleased]
3
+ ## [Unreleased]
4
+
5
+ ### Added
6
+
7
+ ### Changed
8
+
9
+ ### Fixed
10
+
11
+ ## [2.1.5] — 2026-07-04
12
+
13
+ ### Added
14
+
15
+ - **Stealth-check script** (`scripts/stealth-check.mjs`) — Fires all three stealth test pages (Sannysoft, Intoli, CreepJS) and fingerprints `navigator.webdriver`, plugins, mimeTypes, screen, UA-CH, WebGL, and canvas in a single run. Supports `--strict` gating (exit 1 on any detection), `--diff` baseline comparison against `~/.greedysearch/stealth-baseline.json`, and structured CreepJS score extraction. Scripts: `npm run stealth-check`, `stealth:strict`, `stealth:diff`.
16
+ - **`navigator.webdriver` fully deleted** (`extractors/common.mjs`) — Both `navigator.webdriver` and `Navigator.prototype.webdriver` are deleted instead of being set to `undefined`. Sannysoft: `webdriver present` → `missing (passed)`.
17
+ - **PluginArray/MimeTypeArray shape** (`extractors/common.mjs`) — `navigator.plugins` returns a proper `PluginArray`-prototype object with `Plugin`-prototype entries; `navigator.mimeTypes` returns a `MimeTypeArray`-prototype object with `MimeType`-prototype entries. Sannysoft: `pluginsType` → `passed`.
18
+ - **Screen metrics consistency** (`extractors/common.mjs`) — `screen.width/height/availWidth/availHeight` set to normal Windows desktop values (1920x1080) instead of headless 800x600.
19
+ - **Canvas/WebGL/Audio fingerprint noise** (`extractors/common.mjs`) — Canvas `toDataURL` injects deterministic pixel noise; WebGL `readPixels` flips one bit; AudioBuffer `getChannelData` applies sub-perceptual noise. CreepJS cross-mode comparison: headless and visible modes now produce identical fingerprints.
20
+ - **`chrome.loadTimes` and `chrome.csi` enriched** (`extractors/common.mjs`) — Returns realistic timing values instead of empty `{}`.
21
+ - **`navigator.productSub` and `navigator.product`** (`extractors/common.mjs`) — Set to `20030107` and `Gecko` respectively (matching real Chrome).
22
+ - **`navigator.share` and `navigator.contentIndex` stubs** (`extractors/common.mjs`) — Missing platform APIs in headless mode are now defined.
23
+ - **`navigator.connection.downlinkMax`** (`extractors/common.mjs`) — Added `downlinkMax: Infinity` to the connection patch.
24
+ - **Stealth assessment doc** (`docs/analysis.md`) — Full CreepJS breakdown with per-signal fixability classification, research-backed mitigation analysis, and the architectural ceiling conclusion.
25
+ - **stealth-browser-mcp assessment** (`docs/stealthbrowsermcp.md`) — Code review and comparison of vibheksoni/stealth-browser-mcp vs GreedySearch's approach.
26
+
27
+ ### Changed
28
+
29
+ - **Stealth injection refined** (`extractors/common.mjs`) — WebDriver now uses `delete` instead of `Object.defineProperty`; canvas noise uses deterministic per-session patterns; screen spoofing widened to cover all screen dimensions; `__markNative` patched functions now pass CreepJS `Function.prototype.toString` deep inspection.
30
+
31
+ ### Fixed
32
+
33
+ - **CDP `Runtime.evaluate` JSON.parse crash** (`extractors/common.mjs` in `getOrOpenTab`) — `JSON.parse(raw)` wrapped in try/catch with descriptive error message.
34
+
35
+ ## [2.1.4] — 2026-07-04
36
+
37
+ ### Added
38
+
39
+ - **Changelog-driven release notes** (`scripts/changelog-extract.mjs`, `scripts/changelog-release.mjs`, `scripts/backfill-github-releases.mjs`, `.github/workflows/release.yml`) — GitHub releases now use curated CHANGELOG sections as the release body, with helper scripts for promoting `Unreleased`, extracting summary notes, and backfilling historical releases. The parser supports both current `## [x.y.z] — date` headings and legacy `## vx.y.z (date)` headings.
40
+
41
+ ### Changed
42
+
43
+ - **README simplified like pi-lens and package skill removed** (`README.md`, `docs/*.md`, `package.json`) — The root README is now a concise project overview with installation, quick usage, and links to focused docs for usage, research mode, runtime/Chrome, source fetching, development, and releases. The duplicated Pi skill file was removed so docs live in one place.
44
+ - **Normal all-engine config excludes Semantic Scholar** (`src/search/constants.mjs`, `~/.pi/greedyconfig`) — Default and local all-engine fan-out now use Perplexity, Google, ChatGPT, and Gemini while academic/research engines remain opt-in.
45
+
46
+ ### Fixed
47
+
48
+ - **Headless ChatGPT and Gemini extraction in all-mode** (`extractors/chatgpt.mjs`, `extractors/gemini.mjs`, `bin/search.mjs`) — ChatGPT now keeps its tab foregrounded during headless streaming, waits for streaming indicators to clear, and rejects repeated-domain citation stubs. Gemini now targets the latest `model-response`, avoids nested code-block copy buttons, and prefers a complete DOM fallback when clipboard copy captures only a snippet.
49
+ - **Research-mode floor and citation handling** (`src/search/research.mjs`, `src/search/simple-research.mjs`) — Explicit `breadth` / `iterations` values now bypass scale-aware overrides, duplicate fetch targets are skipped, cited-source URL checks only evaluate cited sources, and bot-protected HEAD responses are skipped instead of treated as dead links.
50
+ - **Fetcher return style consistency** (`src/fetcher.mjs`) — Multi-line formatting for the protocol-not-allowed return object in `isPrivateUrl` to match the project's code style conventions.
4
51
 
5
52
  ## [2.1.3] — 2026-06-21
6
53
 
package/README.md CHANGED
@@ -1,14 +1,19 @@
1
+ <p align="center">
2
+ <img src="docs/banner.svg" alt="GreedySearch for Pi" width="1100">
3
+ </p>
4
+
1
5
  # GreedySearch for Pi
2
6
 
3
- ![GreedySearch](docs/banner.svg)
7
+ GreedySearch registers the `greedy_search` tool for Pi: headless-first,
8
+ no-API-key AI/web search through a dedicated Chrome profile.
4
9
 
5
- Multi-engine AI web search for Pi via browser automation.
10
+ ## What It Does
6
11
 
7
- - No API keys
8
- - Real browser results from configurable engines: Perplexity, Google AI, ChatGPT, Gemini, plus opt-in Semantic Scholar and Logically research engines
9
- - Research mode as the centerpiece: iterative planning, source fetching, citation audit, and structured bundles
10
- - Optional configurable synthesis with source grounding (Gemini by default)
11
- - Chrome runs headless by default no window, purely background
12
+ - Multi-engine search across Perplexity, Google AI, ChatGPT, and Gemini
13
+ - Source-grounded `engine: "all"` results with fetched source content
14
+ - Optional synthesis over engine answers and fetched sources
15
+ - Iterative `depth: "research"` runs with citation audit and research bundles
16
+ - Visible Chrome fallback for login/captcha/cookie setup when needed
12
17
 
13
18
  ## Install
14
19
 
@@ -22,16 +27,12 @@ Or from git:
22
27
  pi install git:github.com/apmantza/GreedySearch-pi
23
28
  ```
24
29
 
25
- ## Tool
26
-
27
- - `greedy_search` — multi-engine AI web search, source-grounded synthesis, and deep research
28
-
29
- ## Quick usage
30
+ ## Quick Usage
30
31
 
31
32
  ```js
32
- greedy_search({ query: "React 19 changes" }); // all engines + fetched sources
33
- greedy_search({ query: "React 19 changes", synthesize: true }); // add configured synthesis
34
- greedy_search({ query: "Prisma vs Drizzle", engine: "perplexity" }); // individual engine
33
+ greedy_search({ query: "React 19 changes" });
34
+ greedy_search({ query: "React 19 changes", synthesize: true });
35
+ greedy_search({ query: "Prisma vs Drizzle", engine: "perplexity" });
35
36
  greedy_search({
36
37
  query: "Evaluate browser automation options for AI agents",
37
38
  depth: "research",
@@ -39,181 +40,57 @@ greedy_search({
39
40
  iterations: 2,
40
41
  maxSources: 8,
41
42
  });
42
- // Research mode writes a dataroom-style bundle under .pi/greedysearch-research/ by default.
43
- // Headless is the default — no window. To force visible Chrome:
44
- greedy_search({ query: "Visible browser setup", engine: "perplexity", visible: true });
45
43
  ```
46
44
 
47
- ## Parameters (`greedy_search`)
48
-
49
- ### Common
50
-
51
- - `query` (required)
52
- - `fullAnswer`: return full single-engine output instead of preview
53
- - `headless`: set to `false` to show Chrome window (default: `true`)
54
- - `visible` / `alwaysVisible`: set to `true` to always use visible Chrome for this search
55
-
56
- ### Normal search
57
-
58
- - `engine`: `all` (default web/search fan-out), `perplexity`, `google`, `chatgpt`, `gemini`; opt-in research engines: `semantic-scholar`, `logically`; `bing` still works for signed-in users
59
- - `synthesize`: for `engine: "all"`, synthesize fetched sources with the configured synthesizer (default false)
60
- - `synthesizer`: override the configured synthesis engine for this call (`gemini` default, `chatgpt` supported)
61
- - `depth`: legacy `fast`/`standard`/`deep` aliases are still accepted for compatibility; prefer `synthesize` for normal search
62
-
63
- ### Deep research
64
-
65
- - `depth`: set to `"research"` to run the iterative research workflow
66
- - `breadth`: number of research actions per round, 1-5 (default 3)
67
- - `iterations`: research rounds, 1-3 (default 2)
68
- - `maxSources`: fetched source cap for the final report, 3-12
69
- - `researchOutDir`: optional directory for the research bundle
70
- - `writeResearchBundle`: write the research bundle to disk (default true for research mode)
71
-
72
- Deep research uses the configured `~/.pi/greedyconfig.engines` list for child searches and Gemini for planning/final synthesis.
73
-
74
- ## Environment variables
75
-
76
- | Variable | Default | Description |
77
- | ------------------------------------ | ------------- | ------------------------------------------------------------- |
78
- | `GREEDY_SEARCH_VISIBLE` | (unset) | Set to `1` to show Chrome window instead of headless |
79
- | `GREEDY_SEARCH_ALWAYS_VISIBLE` | (unset) | Set to `1` to force visible Chrome for all GreedySearch runs |
80
- | `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES` | `5` | Minutes of inactivity before auto-killing GreedySearch Chrome |
81
- | `GREEDY_SEARCH_LOCALE` | `en` | Default result language (en, de, fr, es, ja, etc.) |
82
- | `CHROME_PATH` | auto-detected | Path to Chrome/Chromium executable |
45
+ Headless is the default. Use `visible: true` only when you need to establish a
46
+ session, solve a challenge, or inspect the browser:
83
47
 
84
- ## Search modes
85
-
86
- - **Individual engine search/research** — `engine: "perplexity" | "google" | "chatgpt" | "gemini" | "semantic-scholar" | "logically" | "bing"`; returns that engine's answer and sources.
87
- - **Grounded multi-engine search** — default `engine: "all"`; fans out to configured engines, ranks sources, fetches top source content, and reports confidence metadata.
88
- - **All + synthesis** — add `synthesize: true` (or CLI `--synthesize`) to ask the configured synthesizer to combine engine answers and fetched source evidence.
89
- - **Deep research** — `depth: "research"`; iterative action planning, direct URL fetches, fast multi-engine searches, source fetching, learning extraction, deterministic floor checks, citation audit, a final cited report, and a structured on-disk bundle.
90
-
91
- Legacy `depth: "fast" | "standard" | "deep"` values remain accepted for compatibility: `fast` skips source fetching; `standard`/`deep` request synthesis.
92
-
93
- Configure all-engine fan-out and synthesis in `~/.pi/greedyconfig`:
94
-
95
- ```json
96
- {
97
- "engines": ["perplexity", "google", "chatgpt", "gemini"],
98
- "synthesizer": "gemini"
99
- }
100
- ```
101
-
102
- Gemini is a normal search engine and can participate in `engine: "all"`. `semantic-scholar` and `logically` are opt-in academic/research engines; include them in `~/.pi/greedyconfig` only when you want the all-engine fan-out to include academic paper discovery or research-assistant workflows. Default `engine: "all"` excludes them because their results are noisy for casual web search — they shine in `depth: "research"` mode instead. Deep research child searches reuse the same configured `engines` list and keep query text on stdin; Gemini remains the research planner/final-report synthesizer. If `synthesize: true` and `"synthesizer": "gemini"`, Gemini runs once as a search engine and again as the synthesizer; set `"synthesizer": "chatgpt"` to separate those roles for normal all-search synthesis.
103
-
104
- Research bundles are written by default to `.pi/greedysearch-research/<timestamp>_<query>/` and include:
105
-
106
- ```text
107
- STATUS.md # floor status, open/closed question ledger, and gaps
108
- OUTLINE.md # bundle table of contents
109
- reports/SUMMARY.md # final cited report
110
- reports/CLAIMS.md # extracted claims mapped to source IDs
111
- reports/EVIDENCE.md # goal-based evidence extracted from fetched sources
112
- reports/GAPS.md # caveats and remaining uncertainties
113
- sources/ # fetched source markdown files
114
- data/manifest.json # run metadata, stop reason, floor checks, citation audit
115
- data/rounds.json # per-round actions/learnings/gaps
116
- data/sources.json # ranked source registry
117
- data/questions.json # STATUS-style question ledger with evidence/source IDs
118
- data/evidence.json # structured rational/evidence/summary per useful source
119
- ```
120
-
121
- CLI controls:
122
-
123
- ```bash
124
- node bin/search.mjs all --inline --stdin --depth research --breadth 3 --iterations 2 --max-sources 8 <<'EOF'
125
- Evaluate browser automation options for AI agents
126
- EOF
127
- node bin/search.mjs all "topic" --depth research --research-out-dir ./research-topic
128
- node bin/search.mjs all "topic" --depth research --no-research-bundle
129
- ```
130
-
131
- ## Runtime commands
132
-
133
- Inside Pi, prefer the extension commands (no package path needed):
134
-
135
- ```text
136
- /greedy-visible # launch visible Chrome for captcha/login/cookie setup
137
- /greedy-status # show GreedySearch Chrome status
138
- /greedy-kill # stop GreedySearch Chrome
139
- /set-greedy-locale # set default result language (de, fr, es, ja, etc.)
140
- ```
141
-
142
- Git install path:
143
-
144
- ```bash
145
- GS=~/.pi/agent/git/github.com/apmantza/GreedySearch-pi
146
- node "$GS/bin/launch.mjs" --status
147
- node "$GS/bin/visible.mjs" # visible mode
148
- node "$GS/bin/visible.mjs" --kill # strong visible/port cleanup
149
- node "$GS/bin/kill-visible.mjs" # same as visible.mjs --kill
150
- node "$GS/bin/cdp-visible.mjs" list # safe CDP: GreedySearch visible Chrome only
151
- node "$GS/bin/cdp-headless.mjs" list # safe CDP: GreedySearch headless Chrome only
152
- node "$GS/bin/cdp-greedy.mjs" list # safe CDP: any GreedySearch Chrome mode
153
- ```
154
-
155
- npm global install path:
156
-
157
- ```bash
158
- GS="$(npm root -g)/@apmantza/greedysearch-pi"
159
- node "$GS/bin/launch.mjs" --status
160
- node "$GS/bin/visible.mjs"
161
- node "$GS/bin/visible.mjs" --kill
162
- node "$GS/bin/kill-visible.mjs"
163
- node "$GS/bin/cdp-visible.mjs" list
164
- node "$GS/bin/cdp-headless.mjs" list
165
- node "$GS/bin/cdp-greedy.mjs" list
48
+ ```js
49
+ greedy_search({ query: "Visible setup", engine: "perplexity", visible: true });
166
50
  ```
167
51
 
168
- Chrome is auto-cleaned after 5 min idle. Override with `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES=10` or disable with `0`.
52
+ ## Documentation
169
53
 
170
- **CDP safety:** use `cdp-visible.mjs`, `cdp-headless.mjs`, or `cdp-greedy.mjs` for debugging. They always set `CDP_PROFILE_DIR` to the dedicated GreedySearch Chrome profile and never fall back to your main Chrome session. Avoid calling raw `bin/cdp.mjs` manually unless you explicitly set `CDP_PROFILE_DIR`.
54
+ - [Usage guide](docs/usage.md) tool parameters, search modes, and config
55
+ - [Research mode](docs/research.md) — iterative workflow and bundle layout
56
+ - [Runtime and Chrome](docs/runtime.md) — slash commands, env vars, CDP safety
57
+ - [Source fetching](docs/source-fetching.md) — supported source types and metadata
58
+ - [Development](docs/development.md) — project layout, tests, and smoke checks
59
+ - [Release workflow](docs/releases.md) — changelog scripts and GitHub releases
171
60
 
172
61
  ## Requirements
173
62
 
174
- - Chrome
63
+ - Chrome or Chromium
175
64
  - Node.js 20.11.0+
176
65
 
177
- ## Source fetching
178
-
179
- When using `engine: "all"`, top source content is fetched by default. Add `synthesize: true` to synthesize with the configured synthesizer:
180
-
181
- - **PDFs** — Direct PDF links are parsed to markdown text for source-grounded synthesis
182
- - **Semantic Scholar** — Discovers academic papers and prefers direct PDF/external paper links when available
183
- - **Reddit** — Uses Reddit's public `.json` API for posts and comments
184
- - **GitHub** — Uses GitHub REST API for repos, READMEs, and file trees
185
- - **General web** — Mozilla Readability extraction with browser fallback when needed
186
- - **Metadata** — title, author/byline, site name, publish date, language, excerpt
66
+ ## Contributing
187
67
 
188
- ## Project layout
68
+ See [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`AGENTS.md`](AGENTS.md) for the
69
+ extractor, recovery, and release checklists.
189
70
 
190
- - `bin/` runtime CLIs (`search.mjs`, `launch.mjs`, `launch-visible.mjs`, `visible.mjs`, `kill-visible.mjs`, safe CDP wrappers, `cdp.mjs`)
191
- - `extractors/` — engine-specific automation + stealth/consent handling
192
- - `src/` — search pipeline, chrome management, source fetching, formatting
193
- - `skills/` — Pi skill metadata
71
+ GreedySearch is released under the [MIT License](LICENSE).
194
72
 
195
- ## Testing
196
-
197
- Cross-platform test runner (Windows + Unix):
198
-
199
- ```bash
200
- npm test # run all tests
201
- npm run test:quick # skip slow tests
202
- npm run test:smoke # basic health check
203
- ```
204
-
205
- Full bash test suite (Unix only):
206
-
207
- ```bash
208
- npm run test:bash # comprehensive tests
209
- ./test.sh parallel # race condition tests
210
- ./test.sh flags # flag/option tests
211
- ```
73
+ ## Contributors
212
74
 
213
- ## Changelog
75
+ Thanks goes to these wonderful people:
214
76
 
215
- See `CHANGELOG.md`.
77
+ <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
78
+ <!-- prettier-ignore-start -->
79
+ <!-- markdownlint-disable -->
80
+ <table>
81
+ <tbody>
82
+ <tr>
83
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/wastedC"><img src="https://avatars.githubusercontent.com/u/917574?v=4" width="100px;" alt=""/><br /><sub><b>wastedC</b></sub></a><br /><a href="#code-wastedC" title="Code">💻</a> <a href="#ideas-wastedC" title="Ideas & Planning">🤔</a> <a href="#maintenance-wastedC" title="Maintenance">🚧</a> <a href="#review-wastedC" title="Reviewed Pull Requests">👀</a></td>
84
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/apmantza"><img src="https://avatars.githubusercontent.com/u/247365598?v=4" width="100px;" alt=""/><br /><sub><b>Apostolos Mantzaris</b></sub></a><br /><a href="#code-apmantza" title="Code">💻</a> <a href="#doc-apmantza" title="Documentation">📖</a> <a href="#ideas-apmantza" title="Ideas & Planning">🤔</a> <a href="#maintenance-apmantza" title="Maintenance">🚧</a> <a href="#review-apmantza" title="Reviewed Pull Requests">👀</a></td>
85
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/apps/dependabot"><img src="https://avatars.githubusercontent.com/in/29110?v=4" width="100px;" alt=""/><br /><sub><b>Dependabot</b></sub></a><br /><a href="#maintenance-dependabot[bot]" title="Maintenance">🚧</a></td>
86
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/peyloride"><img src="https://avatars.githubusercontent.com/u/10589068?v=4" width="100px;" alt=""/><br /><sub><b>peyloride</b></sub></a><br /><a href="#bug-peyloride" title="Bug reports">🐛</a></td>
87
+ <td align="center" valign="top" width="14.28%"><a href="https://github.com/hypnoglow"><img src="https://avatars.githubusercontent.com/u/4853075?v=4" width="100px;" alt=""/><br /><sub><b>hypnoglow</b></sub></a><br /><a href="#bug-hypnoglow" title="Bug reports">🐛</a></td>
88
+ </tr>
89
+ </tbody>
90
+ </table>
216
91
 
217
- ## License
92
+ <!-- markdownlint-restore -->
93
+ <!-- prettier-ignore-end -->
94
+ <!-- ALL-CONTRIBUTORS-LIST:END -->
218
95
 
219
- MIT
96
+ If you land a pull request or report an issue that gets fixed, we'll add you here.
package/bin/search.mjs CHANGED
@@ -314,6 +314,8 @@ async function main() {
314
314
  const ENGINE_START_URLS = {
315
315
  perplexity: "https://www.perplexity.ai/",
316
316
  google: "https://www.google.com/",
317
+ chatgpt: "https://chatgpt.com/",
318
+ gemini: "https://gemini.google.com/app",
317
319
  "semantic-scholar": "https://www.semanticscholar.org/",
318
320
  semanticscholar: "https://www.semanticscholar.org/",
319
321
  s2: "https://www.semanticscholar.org/",
@@ -0,0 +1,233 @@
1
+ # Stealth Analysis — GreedySearch-pi vs CreepJS
2
+
3
+ > **Date:** 2026-07-04
4
+ > **Scope:** Headless Chrome 149 on Windows 10 with `--headless=new`
5
+ > **Test page:** <https://abrahamjuliot.github.io/creepjs/>
6
+ > **Comparison target:** stealth-browser-mcp (Python MCP server) — claims 0% headless/stealth on CreepJS
7
+
8
+ ---
9
+
10
+ ## 1. Executive Summary
11
+
12
+ GreedySearch-pi applies extensive JavaScript-level stealth patches via `Page.addScriptToEvaluateOnNewDocument` in `extractors/common.mjs`. The current state:
13
+
14
+ | Test | Result |
15
+ |------|--------|
16
+ | Sannysoft Bot Detection | **20/20 clean** (all fingerprint scanner rows pass) |
17
+ | Intoli Headless Detection | **All checks pass** |
18
+ | CreepJS "like headless" | **19%** (fingerprint: `2eb544f2`) |
19
+ | CreepJS "headless" | **33%** (fingerprint: `6ed45504`) |
20
+ | CreepJS "stealth" | **20%** (fingerprint: `4b82ddf4`) |
21
+
22
+ A critical finding: **CreepJS scores are identical in headless and visible mode** — same hashes, same percentages. This means our JS patches successfully close the gap between headless and visible Chrome from CreepJS's perspective. The remaining signals are architectural (baked into the Chrome binary), not patchable from JavaScript.
23
+
24
+ ---
25
+
26
+ ## 2. Full CreepJS Analysis Breakdown
27
+
28
+ The following data was extracted from a live CreepJS run at `https://abrahamjuliot.github.io/creepjs/?cb=1783160620710`.
29
+
30
+ ### 2.1 Headless Section (hash: `87203106`)
31
+
32
+ ```
33
+ 56% like headless: de4e02aa
34
+ 33% headless: 6ed45504
35
+ 20% stealth: 4b82ddf4
36
+ ```
37
+
38
+ #### "Like Headless" signals (56%)
39
+
40
+ | Signal | Value | Notes |
41
+ |--------|-------|-------|
42
+ | `noChrome` | `false` | ✅ `window.chrome` exists |
43
+ | `hasPermissionsBug` | `false` | ✅ Permissions API works |
44
+ | `noPlugins` | `false` | ✅ `navigator.plugins.length = 3` |
45
+ | `noMimeTypes` | `false` | ✅ `navigator.mimeTypes.length = 2` |
46
+ | `notificationIsDenied` | `false` | ✅ Notifications permission works |
47
+ | `hasKnownBgColor` | `true` | ✅ Preferred color scheme matches headed |
48
+ | `prefersLightColor` | `true` | ✅ No dark-mode inconsistency |
49
+ | `uaDataIsBlank` | `false` | ✅ `navigator.userAgentData` populated |
50
+ | `pdfIsDisabled` | `false` | ✅ `navigator.pdfViewerEnabled = true` |
51
+ | `noTaskbar` | **`true`** | ❌ Taskbar API unavailable (headless has no shell) |
52
+ | `hasVvpScreenRes` | **`true`** | ❌ Screen resolution spoofing detected via ScreenDetailed API |
53
+ | `hasSwiftShader` | **`true`** | ❌ WebGL uses SwiftShader software renderer |
54
+ | `noWebShare` | **`true`** | ❌ `navigator.share` missing |
55
+ | `noContentIndex` | **`true`** | ❌ `navigator.contentIndex` missing |
56
+ | `noContactsManager` | **`true`** | ❌ `navigator.contacts` missing |
57
+ | `noDownlinkMax` | **`true`** | ❌ `navigator.connection.downlinkMax` missing |
58
+
59
+ #### "Headless" signals (33%)
60
+
61
+ | Signal | Value | Notes |
62
+ |--------|-------|-------|
63
+ | `webDriverIsOn` | `false` | ✅ `navigator.webdriver` deleted |
64
+ | `hasHeadlessUA` | `false` | ✅ Main-thread UA does not contain "HeadlessChrome" |
65
+ | `hasHeadlessWorkerUA` | **`true`** | ❌ **Worker UA exposes `HeadlessChrome/150`** |
66
+
67
+ #### "Stealth" signals (20%)
68
+
69
+ | Signal | Value | Notes |
70
+ |--------|-------|-------|
71
+ | `hasIframeProxy` | `false` | ✅ No iframe proxy detected |
72
+ | `hasHighChromeIndex` | `false` | ✅ Chrome index looks normal |
73
+ | `hasBadChromeRuntime` | `false` | ✅ `chrome.runtime` looks normal |
74
+ | `hasToStringProxy` | `false` | ✅ `Function.prototype.toString` not detected as proxied |
75
+ | `hasBadWebGL` | **`true`** | ❌ WebGL patches detected (getParameter/readPixels overrides) |
76
+
77
+ ### 2.2 Worker Scope (hash: `95cd472b`)
78
+
79
+ ```
80
+ userAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36
81
+ (KHTML, like Gecko) HeadlessChrome/150.0.0.0 Safari/537.36
82
+ platform: Linux x86_64
83
+ device: Linux (Linux x86_64) | Linux x86_64
84
+ ```
85
+
86
+ The Worker scope exposes **three leaks at once**:
87
+
88
+ 1. **`HeadlessChrome/150.0.0.0`** — product name reveals headless mode
89
+ 2. **`Linux x86_64`** — platform mismatch (main-thread reports `Windows`)
90
+ 3. **Version 150 vs 149** — Worker UA version differs from our `--user-agent` flag
91
+
92
+ ### 2.3 WebGL (hash: `3d883ba3`)
93
+
94
+ ```
95
+ pixels: 50875bab × (fail)
96
+ params (78): eab0e944 × (fail)
97
+ exts (64): d3e8ebec × (fail)
98
+ gpu:
99
+ Google Inc.
100
+ Intel Iris OpenGL Engine ← our spoofed renderer
101
+ ANGLE (Google, Vulkan 1.3.0 (SwiftShader Device (Subzero)))
102
+ ```
103
+
104
+ WebGL shows failures in **pixels, params, and extensions** — our `getParameter` spoofing and `readPixels` noise are detectable.
105
+
106
+ ### 2.4 Screen (hash: `90135785`)
107
+
108
+ ```
109
+ ...screen: 1280 x 1280 ← not our spoofed 1920x1080
110
+ ....avail: 1280 x 1280
111
+ viewport: 1280 1280 1280 1280 1280 1280
112
+ ```
113
+
114
+ CreepJS detects the **real viewport** via `ScreenDetailed` API (`getScreenDetails()`), bypassing our `screen.width`/`height` spoofing.
115
+
116
+ ### 2.5 Canvas 2D (hash: `437458e4`)
117
+
118
+ ```
119
+ data: ecb280d2 × (fail)
120
+ text: 870c8e89
121
+ paint (GPU): 593da416
122
+ paint (CPU): 593da416
123
+ ```
124
+
125
+ Canvas `data` fails — CreepJS detects that `toDataURL` has been tampered with (our noise injection leaves detectable traces in the function's metadata).
126
+
127
+ ### 2.6 Audio (hash: `9d6aa470`)
128
+
129
+ ```
130
+ sum: 124.043...
131
+ values: dfc44fc6 × (fail)
132
+ ```
133
+
134
+ Audio `values` fail — CreepJS's `searchLies` detects that `AudioBuffer.prototype.getChannelData` has been patched.
135
+
136
+ ### 2.7 Fonts (hash: `fe40d7c6`)
137
+
138
+ ```
139
+ load (5/51): "DejaVu Sans", "Liberation Mono", "Noto Color Emoji",
140
+ KACSTOffice, OpenSymbol
141
+ ```
142
+
143
+ The font list contains **Linux-specific fonts** (`DejaVu Sans`, `Liberation Mono`, `Noto Color Emoji`) instead of Windows fonts (`Arial`, `Segoe UI`, `Calibri`). Headless Chrome on Windows uses a Linux font rendering stack.
144
+
145
+ ---
146
+
147
+ ## 3. Gap Classification & Mitigation Research
148
+
149
+ ### 3.1 Immediately Fixable ✅
150
+
151
+ | Issue | Fix | Status |
152
+ |-------|-----|--------|
153
+ | `noDownlinkMax` | Add `downlinkMax: Infinity` to `navigator.connection` patch | ✅ Applied |
154
+ | `noWebShare` | Add `navigator.share` stub | ✅ Applied |
155
+ | `noContentIndex` | Add `navigator.contentIndex` stub | ✅ Applied |
156
+
157
+ ### 3.2 Partially Fixable ⚠️
158
+
159
+ | Issue | Approach | Caveat |
160
+ |-------|----------|--------|
161
+ | `hasVvpScreenRes` | Intercept `ScreenDetailed.prototype` properties in stealth code | Cat-and-mouse — CreepJS can switch to other detection methods |
162
+ | `hasBadWebGL` | Refine `__markNative` to better hide our patches | CreepJS's `searchLies` has deep toString/descriptor checks |
163
+ | `noContactsManager` | Add `navigator.contacts` mock with `select()` method | Complex mock, easy to detect as fake |
164
+ | Canvas/Audio noise refinement | More sophisticated noise injection | Fundamental limitation — JS patches are detectable by design |
165
+
166
+ ### 3.3 Requires Chromium Fork ❌
167
+
168
+ These issues stem from **compile-time baked properties** in the Chrome binary and **cannot be fixed from JavaScript**.
169
+
170
+ | Issue | Root Cause | Why JS Cannot Fix |
171
+ |-------|------------|-------------------|
172
+ | `hasHeadlessWorkerUA` | `HeadlessChrome` is set in C++ `content/child/user_agent.cc`. Workers read from compiled-in values, not `--user-agent` flag. | `Page.addScriptToEvaluateOnNewDocument` does NOT run in Worker/ServiceWorker scopes. Workers are separate V8 isolates. |
173
+ | `hasSwiftShader` | Headless Chrome selects SwiftShader as its GPU backend. No GPU passthrough in headless mode. | WebGL rendering originates in the GPU process, not the renderer process. JS can only patch the Blink API surface. |
174
+ | Linux fonts | Headless Chrome uses a Linux font config stack on all platforms. | Font enumeration comes from the system font manager, not JS-accessible API. |
175
+ | `hasVvpScreenRes` (deep) | `ScreenDetailed` API exposes real display info from the OS window manager. | API is backed by platform-level display info, not JS overridable properties. |
176
+
177
+ ### 3.4 Research-Backed Verdicts
178
+
179
+ #### Worker UA (the single highest-impact fix)
180
+
181
+ **Research sources:**
182
+
183
+ - [IPASIS: Advanced Headless Chrome Detection](https://ipasis.com/blog/detecting-headless-chrome-workers-webgl) (Dec 2025): *"If a scraper injects a script to redefine navigator.userAgent in the main window, checking self.navigator.userAgent inside a newly spawned Worker often reveals the underlying Headless configuration."*
184
+ - [Browserbase: Why we forked Chromium](https://www.browserbase.com/blog/chromium-fork-for-ai-automation) (Nov 2025): *"Our patch updates the headless product name to Chrome and keeps the metadata and CDP reporting consistent with a normal Chrome build. Requests from workers, iframes, and service workers use the same values because the Network Service pulls from the same metadata."*
185
+
186
+ **Verdict:** The only proven mitigation is a Chromium fork that patches the C++ source (`content/child/user_agent.cc`) to replace `HeadlessChrome` with `Chrome` at compile time. Browserbase and CloakBrowser have both done this. No JavaScript-level workaround exists.
187
+
188
+ #### Chromium Forks That Solve These Issues
189
+
190
+ | Project | Approach | Open Source? |
191
+ |---------|----------|--------------|
192
+ | [Browserbase](https://www.browserbase.com/blog/chromium-fork-for-ai-automation) | Custom Chromium fork with patched UA, webdriver, and session management | ❌ Proprietary |
193
+ | [CloakBrowser](https://github.com/CloakHQ/CloakBrowser) | C++ level patches to Chromium source | ✅ Open source |
194
+ | [ultrafunkamsterdam/undetected-chromedriver](https://deepwiki.com/ultrafunkamsterdam/undetected-chromedriver/4.3-headless-mode) | JS + config-level patches | ✅ Open source (JS only) |
195
+
196
+ #### The `--user-agent` Flag Limitation
197
+
198
+ Chrome's `--user-agent` flag modifies the **HTTP `User-Agent` header** and the **main-thread `navigator.userAgent`**, but it does NOT propagate to:
199
+
200
+ - Worker `navigator.userAgent`
201
+ - Service Worker `self.navigator.userAgent`
202
+ - Some internal Blink UA checks
203
+
204
+ This is documented in Chromium's source: the `UserAgent` in `content/child/user_agent.cc` has separate code paths for "product name" (compiled as `HeadlessChrome`) vs "user agent override" (from command line). Workers read from the product name, not the override.
205
+
206
+ ---
207
+
208
+ ## 4. Practical Recommendations
209
+
210
+ ### What we can do next (JS-level)
211
+
212
+ 1. **Patch ScreenDetailed API** — intercept `getScreenDetails()` to return consistent values
213
+ 2. **Add `navigator.contacts` stub** — low impact but easy
214
+ 3. **Accept the ceiling** — Sannysoft 20/20 + Intoli pass means the practical anti-bot targets (Cloudflare, DataDome, Google) are well-covered
215
+
216
+ ### What requires a Chromium fork
217
+
218
+ - Worker UA fix
219
+ - SwiftShader/GPU masking
220
+ - Font stack normalization
221
+
222
+ ### Bottom line
223
+
224
+ Our JS patches achieve the **practical maximum** for headless Chrome stealth without modifying the browser binary:
225
+
226
+ - ✅ All JavaScript-accessible signals are normalized
227
+ - ✅ Headless and visible modes produce identical CreepJS fingerprints
228
+ - ✅ Sannysoft is 100% clean
229
+ - ✅ All engines (Perplexity, Bing, ChatGPT, Gemini, Google) work in headless mode
230
+
231
+ The remaining ~33% headless score is the **architectural ceiling** — Chrome's compiled-in "HeadlessChrome" product name, SwiftShader GPU, and Linux font stack are not reachable from JavaScript.
232
+
233
+ For reference: Browserbase's entire business model is built around patching these exact C++ signals, and they needed a dedicated build machine compiling Chromium in ~1 hour to iterate on the patches.