@apmantza/greedysearch-pi 1.8.5 → 1.8.7

package/CHANGELOG.md

@@ -1,6 +1,180 @@
 # Changelog
 
-## Unreleased
+## [Unreleased]
+
+### Fixed
+
+- **SonarCloud minor vulnerability false positives** — Confirmed both remaining issues are false positives (internal diagnostic logging in `bin/gschrome.mjs` and test debug output in `test/fetcher-cli.mjs`). Verified via full smoke test suite: all 33 unit tests pass, all 4 engines (Perplexity, Bing, Google, Gemini) return results at all depths (fast/standard/deep), CDP safety wrappers correctly enforce mode boundaries.
+
+- **SonarCloud security hotspots** (re-verified) — All previously fixed hotspots remain resolved: replaced `spawn("node", ...)` with `spawn(process.execPath, ...)`, replaced `Math.random()` with `crypto.randomInt()`, 19 remaining hotspots confirmed as false positives (hardcoded `execSync` commands, simple regex patterns).
+
+### Fixed
+
+- **Headless→visible mode switching** (`src/search/chrome.mjs`) — `ensureChrome()` only handled the case where visible was requested but headless Chrome was running. When headless was requested (the default) but visible Chrome was running, it silently kept visible mode — causing env var mismatches that broke extractors like Perplexity. Now properly detects both directions and kills/relaunches in the correct mode.
+
+- **SonarCloud security hotspots** — Replaced `spawn("node", ...)` with `spawn(process.execPath, ...)` in cdp wrapper, `runExtractor`, `synthesizeWithGemini`, and test helper to prevent PATH-based binary substitution. Replaced `Math.random()` with `crypto.randomInt()` in `jitter()` for non-security-sensitive timing variance. Remaining 19 hotspots are verified false positives (hardcoded `execSync` commands, simple regex patterns).
+- **Bing stealth not active on page load** (`src/search/chrome.mjs`) — `injectHeadlessStealth` was fire-and-forget (`.catch(() => {})`). The CDP `Page.addScriptToEvaluateOnNewDocument` command is async — extractors often navigated to Copilot before stealth registered. Cloudflare saw headless fingerprints and blocked the page. Fixed by awaiting stealth for Bing tabs. Perplexity/Google kept fire-and-forget since Perplexity's anti-bot detects the awaited patches.
+- **Bing copy button handler not hydrated** (`extractors/bing-copilot.mjs`) — Copilot's React copy button exists in the DOM before its click handler is bound. `clickCopyAndPollClipboard` clicked too early → clipboard interceptor empty → 13s wasted polling + DOM fallback. Added 800ms hydration delay after `waitForCopyButton`. Solo Bing went from 37-73s → 16s.
+- **Manual verification blocked synthesis** (`bin/search.mjs`) — When Bing/Perplexity needed manual verification after visible recovery, `search.mjs` returned early with `synthesize: false`, discarding all engine results. Now synthesis continues with whichever engines succeeded. Visible Chrome stays open for the user.
+- **Source-fetch crash after visible→headless recovery** (`src/search/fetch-source.mjs`) — After recovery killed/restarted Chrome, stale CDP tab references in parallel source-fetch workers caused "No target matching prefix" crashes. Workers now catch `fetchSourceContent` errors; `fetchSourceContentBrowser` returns error objects instead of throwing.
+- **Progress tracker "🔄 synthesizing" hang** (`src/tools/shared.ts`) — When synthesis was skipped (manual verification), the progress tracker showed "🔄 synthesizing" forever because no `PROGRESS:synthesis:done` was ever emitted. Now handles `done`/`error`/`skipped` synthesis states.
+- **Gemini synthesis eval timeout** (`bin/cdp.mjs`) — CDP daemon `TIMEOUT` was 30s, but `waitForStreamComplete` uses a single `Runtime.evaluate` call that can run 60-90s for long synthesis prompts. Increased to 90s.
+
+### Performance
+
+- **Reduced timeouts across all extractors** — Navigation: 35s→20s, verification retry: 30s→10s (Bing/Perplexity), 60s→10s (Gemini/Google), post-nav settle: 1200ms→600ms (Bing), 1200ms→600ms (Gemini). Turnstile never clears in headless, so 30s of retry loops were pure waste.
+- **Hard per-engine timeouts raised** (`bin/search.mjs`) — Fast: 22s→30s, Standard/Deep: 35s→55s. CDP contention from 3 parallel extractors adds overhead that the old budgets didn't account for.
+- **Tab creation split: Bing gets blank+stealth, others pre-seeded** (`src/search/chrome.mjs`) — `Target.createTarget` navigation is less detectable than CDP `Page.navigate` for Perplexity/Google. Bing needs blank tab + awaited stealth to hide headless fingerprints from Copilot's Cloudflare.
+
+### Performance
+
+- **Hard per-engine timeouts** (`bin/search.mjs`) — Fast mode: 22s per engine. Standard/deep: 35s per engine. Slow engines are skipped instead of stalling the whole batch. Previously a single slow engine could push `all` searches to 60–90s.
+- **Parallel tab creation** (`bin/search.mjs`, `src/search/chrome.mjs`) — All engine tabs open simultaneously instead of sequential 300ms staggered delays. Tabs are pre-seeded to each engine's homepage so extractors skip redundant initial navigation.
+- **Reduced settle delays** (`extractors/common.mjs`) — `postNav` 1500→800ms, `postNavSlow` 2000→1200ms, `postClick`/`postType` 400→300ms, `afterVerify` 3000→1500ms. Safe because tabs now load the target domain before the extractor even starts.
+- **Higher source-fetch concurrency** (`src/search/constants.mjs`) — Default `GREEDY_FETCH_CONCURRENCY` raised from 2 → 4.
+- **Faster HTTP timeouts** (`src/search/fetch-source.mjs`) — HTTP fetch timeout 15s → 10s, browser fallback settle 1500ms → 800ms.
+- **Non-blocking cleanup** (`bin/search.mjs`) — Removed the 1500ms hard sleep at process exit; `minimizeChrome` now fire-and-forget.
+- **Domain-aware navigation skip** (`extractors/bing-copilot.mjs`, `extractors/perplexity.mjs`, `extractors/google-ai.mjs`) — When a tab is already on the engine's domain (pre-seeded by orchestrator), skip the redundant `cdp nav` call and settle delay.
+- **Fast mode keeps short engine budgets** (`bin/search.mjs`) — Fast mode still uses 22s per-engine extraction timeouts and skips source fetch/synthesis work. Verification recovery can now run in fast mode when Bing/Perplexity are blocked, because returning no result is worse than the retry cost.
+
+### Anti-Bot Detection Hardening (Anti-CDP Evasion)
+
+- **Runtime.enable evasion** (`bin/cdp.mjs`) — The primary CDP detection vector (Cloudflare/DataDome watch for `Runtime.consoleAPICalled` timing) has been eliminated. All `Runtime.evaluate` calls now use an explicit `contextId` captured via brief `Runtime.enable` → `Runtime.disable` at daemon startup (~100ms window). No persistent Runtime domain enable for the session. See: rebrowser.net / DataDome research.
+- **Stale PID / ghost Chrome cleanup** (`src/search/chrome.mjs`) — `killChrome()` now uses port-based process detection via `netstat`/`lsof` instead of relying solely on the PID file. Handles ghost processes that hold port 9222 after the tracked PID dies. Old `killHeadlessChrome` kept as backward-compat alias.
+- **Idle cleanup for both modes** (`src/search/chrome.mjs`) — `checkAndKillIdle()` no longer gates on `GREEDY_SEARCH_HEADLESS=1`. Both headless and visible Chrome auto-kill after idle timeout. Disable with `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES=0`.
+- **`--disable-blink-features=AutomationControlled` for visible mode** (`bin/launch.mjs`, `bin/gschrome.mjs`) — Previously headless-only. The flag and `--window-size` now apply to both modes, suppressing `navigator.webdriver` in visible Chrome too.
+- **Stealth injection for visible mode** (`src/search/chrome.mjs`, `extractors/common.mjs`) — Canvas noise, plugin spoofing, `window.chrome.runtime`, and console safening now inject on both headless and visible tabs.
+- **Client Hints consistency** (`src/fetcher.mjs`) — Added `Sec-CH-UA`, `Sec-CH-UA-Mobile`, `Sec-CH-UA-Platform` headers to `DEFAULT_HEADERS`, matching the Chrome 122 user-agent. Inconsistency between UA and Client Hints is a strong bot signal.
+- **Perplexity Cloudflare verification** (`extractors/perplexity.mjs`) — Added `handleVerification` call after navigation. Perplexity was the only engine missing Cloudflare challenge handling (Bing, Gemini, Google AI already had it).
+- **Chrome TLS fetch fallback** (`src/search/fetch-source.mjs`) — New `fetchSourceViaChrome()` uses `Network.loadNetworkResource` (Chrome 124+) to fetch with authentic Chrome TLS/JA3+HTTP/2 fingerprints when Node.js HTTP fails. Zero navigation overhead.
+
+### Added
+
+- **`visible` / `alwaysVisible` search options** (`src/tools/greedy-search-handler.ts`, `src/tools/shared.ts`, `bin/search.mjs`) — Agents can now force visible Chrome per call with `visible: true`, `alwaysVisible: true`, or `headless: false`. CLI aliases: `--visible`, `--always-visible`. Global env: `GREEDY_SEARCH_ALWAYS_VISIBLE=1`.
+- **GreedySearch Chrome commands for Pi** (`index.ts`) — Added `/greedy-visible`, `/greedy-status`, and `/greedy-kill` so users do not need to know package install paths to manage the dedicated Chrome instance.
+- **Safe CDP wrappers** (`bin/cdp-greedy.mjs`, `bin/cdp-visible.mjs`, `bin/cdp-headless.mjs`) — Agents can inspect only the dedicated GreedySearch Chrome profile. The wrappers always set `CDP_PROFILE_DIR` and mode-specific wrappers refuse to attach to the wrong mode, preventing accidental main-Chrome pollution.
+- **`bin/kill-visible.mjs`** — Strong visible/port cleanup helper backed by `launch-visible.mjs`'s PID + port nuke path.
+- **`bin/gschrome.mjs`** — Standalone Chrome lifecycle manager: `launch-headless`, `launch-visible`, `kill`, `status`. Port-based PID detection, forces mode switches, writes `DevToolsActivePort` for CDP.
+
+### Fixed
+
+- **Single-engine visible recovery** (`bin/search.mjs`) — `engine: "bing"` and `engine: "perplexity"` now perform the same headless → visible retry as `engine: "all"` when blocked by Cloudflare, captcha, timeout, missing input, or clipboard failures.
+- **Bing visible clipboard race** (`extractors/bing-copilot.mjs`) — Waits for the assistant copy button, polls clipboard interception after click, retries copy/poll, then falls back to visible DOM text. Fixes cases where Copilot visibly answered but the extractor returned `Clipboard interceptor returned empty text`.
+- **Manual verification flow** (`bin/search.mjs`, `src/formatters/results.ts`) — If visible retry reaches a human verification challenge, GreedySearch leaves visible Chrome open and returns a clear “solve verification, then rerun” result instead of killing the browser and returning no results.
+- **Visible/headless process cleanup** (`bin/launch.mjs`, `bin/visible.mjs`) — Fixed Windows `taskkill` arguments, added port fallback cleanup for `--kill`, and made `visible.mjs --kill` delegate to the stronger `launch-visible.mjs` cleanup path.
+- **README install paths and skill guidance** (`README.md`, `skills/greedy-search/skill.md`) — Corrected Pi git/npm package paths, documented visible mode and safe CDP wrappers, and removed stale `coding_task` guidance from the agent skill.
+
+## [1.8.6] — 2026-05-04
+
+### Bing Copilot: Headless Cloudflare Recovery
+
+- **Auto-retry triggers on all Bing failures** — Error pattern expanded from `input not found|verification` to include `clipboard` failures, so any extraction failure triggers the visible Chrome recovery.
+- **Clipboard retry** — `bing-copilot.mjs` now retries clipboard extraction once with a 2s delay, matching the Perplexity extractor pattern.
+- **Cloudflare detection** — If the clipboard is empty and the AI copy button is hidden, the extractor checks the accessibility tree for Cloudflare challenge text and logs it explicitly for faster diagnosis.
+- **DOM extraction fallback** — If clipboard fails and the copy button is missing (headless anti-bot behavior), attempts direct text extraction from the `copilot.fun` → blob: iframe chain via CDP targets. Falls through to the visible auto-retry if Cloudflare blocks the iframe.
+- **Investigation confirmed** — In headless mode, Copilot renders the AI response inside a `copilot.fun` → blob: iframe sandbox with a Cloudflare Turnstile challenge. The `copy-ai-message-button` (`data-testid`) is hidden. Content is unreachable from both the main frame JS (cross-origin) and CDP iframe traversal (Cloudflare blocks load). The only viable path is visible Chrome recovery — once cookies are cached in the profile, subsequent headless searches pass transparently.
+
+### Visible Chrome Recovery
+
+- **Mode-aware `ensureChrome()`** — `src/search/chrome.mjs` now reads a mode marker file (`greedysearch-chrome-mode`) written by `launch.mjs`. When `GREEDY_SEARCH_VISIBLE=1` and Chrome is running headless, it kills and relaunches in visible mode with a forced relaunch guard (always relaunches after kill, even if port wasn't freed).
+- **`launch.mjs` mode check on reuse** — When Chrome is already running and visible is requested (`GREEDY_SEARCH_VISIBLE=1`), checks the mode file. If headless, kills the running instance and launches visible instead of reusing.
+- **Mode file cleanup** — Mode marker file cleaned on `--kill`, ghost cleanup, and idle timeout kill.
+- **`bin/launch-visible.mjs`** — Standalone visible Chrome launcher. Nukes any process on port 9222 (by PID file + port scan), launches Chrome without `--headless`, and writes `"visible"` to the mode file. No ghost cleanup complexity, no mode switching — fire-and-forget visible Chrome.
+- **`bin/visible.mjs`** — Convenience wrapper: kills headless, then launches visible (delegates to `launch-visible.mjs`).
+- **Progress notification** — When the auto-retry launches visible Chrome for manual Cloudflare verification, a `PROGRESS:bing:needs-human` line is emitted to stderr. The progress tracker renders `🔓 bing needs manual verification` in the Pi UI.
+- **Idle cleanup preserves mode** — Headless idle timeout cleanup now also removes the mode marker file.
+
+### Security & Robustness
+
+- **Chrome process cleanup hardening** — `launch-visible.mjs` uses `taskkill /F /PID X /T` (process tree kill) on Windows to prevent orphan renderer processes. Repeated up to 5s until port 9222 is confirmed free.
+- **Zombie Chrome prevention** — `launch.mjs` and `chrome.mjs` now clean up the mode marker and PID file consistently across all kill paths (--kill, ghost cleanup, idle timeout).
+
+### Added
+
+- **`google-search` engine** — plain Google search extractor (locale-agnostic, `textarea[name="q"]`). Returns title/URL/snippet for traditional 10-blue-link results. Aliases: `gs`, `googlesearch`.
+
+### Headless Mode (default)
+
+- **Chrome now runs headless by default** — no window, no GUI, purely background. Set `GREEDY_SEARCH_VISIBLE=1` to show the browser window.
+- **Anti-detection stealth** — Patches injected via `Page.addScriptToEvaluateOnNewDocument` (runs before any page JS):
+  - `Runtime.enable` / CDP marker deletion (`__REBROWSER_*`, `__nightmare`, `__phantom`, etc.)
+  - `navigator.webdriver` → `false`, `navigator.plugins` → realistic list, `navigator.languages` → `['en-US', 'en']`
+  - `window.chrome` shim, WebGL vendor → Intel Iris, `hardwareConcurrency` → 8, `deviceMemory` → 8
+  - `TrustedTypes` policy, `requestAnimationFrame` keep-alive (prevents headless stall detection)
+  - `--disable-blink-features=AutomationControlled`, realistic `--user-agent`, `--window-size=1920,1080`
+- **Human click simulation** — All verification/clicks now use CDP `Input.dispatchMouseEvent` with multi-event `mouseMoved→pressed→released`, ±3px coordinate jitter, and random delays (80–180ms hover, 30–90ms hold). Detection scripts return element selectors instead of clicking in-page; `handleVerification` performs human clicks via `humanClickElement()`/`humanClickXY()`. Applies to Turnstile iframes, reCAPTCHA, Cloudflare challenges, Microsoft auth, Copilot modals, and all generic verify/continue buttons.
+- **Idle auto-cleanup** — Headless Chrome auto-killed after `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES` (default 5 min) of inactivity. Kills only the PID-tracked instance on port 9222 — never touches the main Chrome session. Activity timestamp written at search start and end.
+
+### Performance
+
+- **Timeouts cut ~40–50%** across all extractors — typical search ~60–90s → ~30–45s:
+  - `TIMING`: postNav 1500→800ms, postNavSlow 2000→1000ms, postClick 400→250ms, postType 400→250ms, inputPoll 400→300ms, copyPoll 600→400ms, afterVerify 3000→2000ms
+  - Defaults: waitForCopyButton 60s→30s, waitForStreamComplete 30s→20s, handleVerification 60s→30s
+  - Per-extractor: Google stream 45s→30s, Gemini copyButton 120s→60s + inputDeadline 10s→8s, Perplexity inputDeadline 8s→5s + stream 30s→20s, Bing verification 90s→30s + copyButton 60s→30s
+  - Engine process timeout: 90s→60s (180s→120s Gemini)
+
+### Security
+
+- **SonarCloud security hotspots fixed** — Two open hotspots resolved:
+  - _Weak cryptography (S2245)_ in `extractors/consent.mjs`: replaced `Math.random()` with `crypto.randomInt()` for the mouse-jitter RNG. Not actually security-sensitive (used only for ±3px jitter and timing delays), but compliant now.
+  - _PATH injection (S4036)_ in `src/search/chrome.mjs`: `spawn("node", ...)` replaced with `spawn(process.execPath, ...)` so the launcher doesn't rely on the `PATH` environment variable.
+- **Query/prompt leakage prevention** — Queries and synthesis prompts no longer appear in OS process tables. All `spawn()` calls now pipe query/prompt through stdin via `--stdin` flag instead of command-line arguments. Affects `runSearch`, `runExtractor`, `synthesizeWithGemini`, and all 5 extractors (`perplexity`, `bing-copilot`, `google-ai`, `google-search`, `gemini`).
+
+### Visual
+
+- **Redesigned banner** — Cleaner SVG layout with pi logo icon, no text, no lens graphic. Gemini Synthesizer pill badge integrated. Three design iterations landed on a minimal icon-only look (`docs/banner.svg`).
+
+### Fixed
+
+- **Gemini & Bing copy button race condition** — Both extractors were capturing the user's query instead of the AI's answer. Root cause: `document.querySelector()` returns the first copy button in DOM order, which is the user's echoed message (above the assistant's response). For short queries this triggers instantly. Fixed by: (1) replacing `waitForCopyButton` with `waitForStreamComplete` to ensure the response finishes streaming before copying, and (2) clicking the **last** copy button (`querySelectorAll` + `[length-1]`) instead of the first — matching Perplexity's proven pattern. Also added periodic scroll-to-bottom alongside stream wait for Gemini to trigger lazy-loaded content.
+- **Progress tracker shows false ✅ for errors** — `makeProgressTracker` in `shared.ts` completely ignored the `status` parameter, always showing `✅ done` for every engine. Now correctly tracks per-engine status and shows `❌ failed` when an engine errors.
+- **Synthesis echoes engine JSON when engines fail** — When Perplexity/Bing fail, Gemini was echoing the engine summary JSON back as its "answer". `synthesis-runner.mjs` now detects this pattern (engine keys without synthesis fields) and treats it as a parse failure, falling back to individual engine results.
+- **`headless=false` parameter ignored** — The `--headless` flag was never checked by `search.mjs` or `launch.mjs`; they only read `GREEDY_SEARCH_VISIBLE`. `shared.ts` now propagates the visibility preference via the env var when `headless=false` is passed.
+
+### Cloudflare / Verification Recovery
+
+- **Auto-recovery from Cloudflare blocks** — When Perplexity (`#ask-input` not found) or Bing (`input not found` / `verification required`) fail in headless mode, `search.mjs` now:
+  1. Detects the Cloudflare/verification error pattern
+  2. Kills headless Chrome, relaunches in visible mode
+  3. Retries the blocked engines — Cloudflare bypasses, cookies stored in Chrome profile
+  4. Kills visible Chrome, relaunches headless
+  5. Continues remaining pipeline (source fetch, synthesis)
+  6. Cookies persist — subsequent headless searches pass transparently
+
+### Removed
+
+- **`coding_task` tool removed** — `bin/coding-task.mjs`, `src/formatters/coding.ts`, registration deleted (644 lines).
+- **`deep_research` tool removed** — handler, test, and `formatDeepResearch` + helpers deleted (521 lines). Use `greedy_search` with `depth: "deep"`.
+- **Minimize debug logs** — Removed 9 verbose `[minimize]` console.log statements from launch.mjs.
+
+### Fixes
+
+- **Code scanning alerts resolved (5 alerts)** — (1) Added `permissions: contents: read` to `sync-to-webaio.yml` workflow (#14). (2) Fixed backslash escaping in `consent.mjs`'s `humanClickElement` selector injection (#10) — selectors containing backslashes (e.g., `\"`) weren't properly escaped before DOM injection. (3) Fixed same backslash escaping in `google-search.mjs`'s `SEARCH_BOX` selector in 3 locations (#11-13).
+- **`cdp.mjs` `getPages()` filter** — Allows `chrome://newtab/` (headless Chrome default initial tab). Prevents "No Chrome tabs found" on cold start.
+
+### Security
+
+- **SonarCloud: Log injection vulnerability (1 alert)** — `bin/launch.mjs` no longer logs the raw WebSocket debugger URL (user-controlled data). Replaced with a static "WebSocket URL received" message to prevent query/URL content from leaking into logs.
+
+### Code Quality
+
+- **SonarCloud batch fixes (~52 issues resolved)** across 16 source files:
+  - `S7781` — Replaced 18 `String#replace()` calls with `String#replaceAll()` for global replacements (regex → literal where applicable).
+  - `S1128` — Removed 15 unused imports (`dirname`, `join`, `relative`, `spawn`, `tmpdir`, `existsSync`, `shouldUseBrowser`, `closeTabs`, `cdp`, `openNewTab`, `closeTab`, `activateTab`, `trimText`).
+  - `S7773` — Migrated 11 `parseInt`/`parseFloat` calls to `Number.parseInt`/`Number.parseFloat`.
+  - `S7780` — Wrapped 8 CDP eval templates containing backslash sequences in `String.raw()` to eliminate double-escaping.
+  - `S7735` — Eliminated 13 negated-condition ternaries by inverting the conditional logic (`!== -1 ? ... : null` → `=== -1 ? null : ...`).
+
+### Security Hotspot Review
+
+- **SonarCloud: 20 security hotspots reviewed and marked Safe** — All outstanding hotspots were assessed and resolved in SonarCloud:
+  - `S4721` OS Command Injection (×2) — Inputs are hardcoded (`port=9222`) or parsed from system output and validated via `Number.parseInt`. Not user-controlled.
+  - `S5852` Regex ReDoS (×10) — Regexes operate on bounded input with negated char classes or short fixed patterns. No practical denial-of-service risk.
+  - `S4036` PATH environment variable (×8) — Local CLI extension spawning package-internal Node scripts. PATH is host-controlled; no untrusted input reaches the command.
+
+### Tooling
+
+- **SonarCloud configuration** — Added `sonar-project.properties` with exclusions for `test/**`, `test.mjs`, `test.sh`, `test_unit.mjs`, and `scripts/**` so test-only code does not skew source quality metrics.
 
 ## v1.8.5 (2026-04-29)
 

package/README.md

@@ -1,98 +1,173 @@
-# GreedySearch for Pi
-
-Multi-engine AI web search for Pi via browser automation.
-
-- No API keys
-- Real browser results (Perplexity, Bing Copilot, Google AI)
-- Optional Gemini synthesis with source grounding
-
-## Install
-
-```bash
-pi install npm:@apmantza/greedysearch-pi
-```
-
-Or from git:
-
-```bash
-pi install git:github.com/apmantza/GreedySearch-pi
-```
-
-## Tools
-
-- `greedy_search` - fast or grounded multi-engine search
-- `coding_task` - browser-routed Gemini/Copilot coding assistance
-
-## Quick usage
-
-```js
-greedy_search({ query: "React 19 changes" })
-greedy_search({ query: "Prisma vs Drizzle", engine: "all", depth: "fast" })
-greedy_search({ query: "Best auth architecture 2026", engine: "all", depth: "deep" })
-```
-
-## Parameters (`greedy_search`)
-
-- `query` (required)
-- `engine`: `all` (default), `perplexity`, `bing`, `google`, `gemini`
-- `depth`: `standard` (default), `fast`, `deep`
-- `fullAnswer`: return full single-engine output instead of preview
-
-## Depth modes
-
-- `fast` - quickest, no synthesis/source fetching
-- `standard` - balanced default for `engine: "all"` (synthesis + fetched sources)
-- `deep` - strongest grounding and confidence metadata
-
-## Runtime commands
-
-```bash
-node ~/.pi/agent/git/GreedySearch-pi/bin/launch.mjs
-node ~/.pi/agent/git/GreedySearch-pi/bin/launch.mjs --status
-node ~/.pi/agent/git/GreedySearch-pi/bin/launch.mjs --kill
-```
-
-## Requirements
-
-- Chrome
-- Node.js 20.11.0+ (22+ recommended)
-
-## Source fetching
-
-When using `depth: "standard"` or `depth: "deep"`, source content is fetched and synthesized:
-
-- **Reddit** — Uses Reddit's public `.json` API for posts and comments (no scraping)
-- **GitHub** — Uses GitHub REST API for repos, READMEs, and file trees
-- **General web** — Mozilla Readability extraction with browser fallback for bot-blocked pages
-- **Metadata** — title, author/byline, site name, publish date, language, excerpt
-
-## Project layout
-
-- `bin/` - runtime CLIs (`search.mjs`, `launch.mjs`, `cdp.mjs`, `coding-task.mjs`)
-- `extractors/` - engine-specific automation
-- `src/` - ranking/fetching/formatting internals (includes `reddit.mjs`, `github.mjs`, `fetcher.mjs`)
-- `skills/` - Pi skill metadata
-
-## Testing
-
-Cross-platform test runner (Windows + Unix):
-```bash
-npm test              # run all tests
-npm run test:quick    # skip slow tests
-npm run test:smoke    # basic health check
-```
-
-Full bash test suite (Unix only):
-```bash
-npm run test:bash           # comprehensive tests
-./test.sh parallel          # race condition tests
-./test.sh flags             # flag/option tests
-```
-
-## Changelog
-
-See `CHANGELOG.md`.
-
-## License
-
-MIT
+# GreedySearch for Pi
+
+![GreedySearch](docs/banner.svg)
+
+Multi-engine AI web search for Pi via browser automation.
+
+- No API keys
+- Real browser results (Perplexity, Bing Copilot, Google AI)
+- Optional Gemini synthesis with source grounding
+- Chrome runs headless by default — no window, purely background
+
+## Install
+
+```bash
+pi install npm:@apmantza/greedysearch-pi
+```
+
+Or from git:
+
+```bash
+pi install git:github.com/apmantza/GreedySearch-pi
+```
+
+## Tools
+
+- `greedy_search` — multi-engine AI web search
+- `websearch` — lightweight DuckDuckGo/Brave search (via pi-webaio)
+- `webfetch` / `webpull` — page fetching and site crawling (via pi-webaio)
+
+## Quick usage
+
+```js
+greedy_search({ query: "React 19 changes" });
+greedy_search({ query: "Prisma vs Drizzle", engine: "all", depth: "fast" });
+greedy_search({
+  query: "Best auth architecture 2026",
+  engine: "all",
+  depth: "deep",
+});
+// Headless is the default — no window. To force visible Chrome:
+greedy_search({ query: "Bing captcha setup", engine: "bing", visible: true });
+```
+
+## Parameters (`greedy_search`)
+
+- `query` (required)
+- `engine`: `all` (default), `perplexity`, `bing`, `google`, `gemini`
+- `depth`: `standard` (default), `fast`, `deep`
+- `fullAnswer`: return full single-engine output instead of preview
+- `headless`: set to `false` to show Chrome window (default: `true`)
+- `visible` / `alwaysVisible`: set to `true` to always use visible Chrome for this search
+
+## Environment variables
+
+| Variable                             | Default       | Description                                                   |
+| ------------------------------------ | ------------- | ------------------------------------------------------------- |
+| `GREEDY_SEARCH_VISIBLE`              | (unset)       | Set to `1` to show Chrome window instead of headless          |
+| `GREEDY_SEARCH_ALWAYS_VISIBLE`       | (unset)       | Set to `1` to force visible Chrome for all GreedySearch runs  |
+| `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES` | `5`           | Minutes of inactivity before auto-killing GreedySearch Chrome |
+| `GREEDY_SEARCH_LOCALE`               | `en`          | Default result language (en, de, fr, es, ja, etc.)            |
+| `CHROME_PATH`                        | auto-detected | Path to Chrome/Chromium executable                            |
+
+## Depth modes
+
+- `fast` - quickest, no synthesis/source fetching
+- `standard` - balanced default for `engine: "all"` (synthesis + fetched sources)
+- `deep` - strongest grounding and confidence metadata
+
+## Runtime commands
+
+Inside Pi, prefer the extension commands (no package path needed):
+
+```text
+/greedy-visible  # launch visible Chrome for captcha/login/cookie setup
+/greedy-status   # show GreedySearch Chrome status
+/greedy-kill     # stop GreedySearch Chrome
+```
+
+Git install path:
+
+```bash
+GS=~/.pi/agent/git/github.com/apmantza/GreedySearch-pi
+node "$GS/bin/launch.mjs" --status
+node "$GS/bin/visible.mjs"          # visible mode
+node "$GS/bin/visible.mjs" --kill   # strong visible/port cleanup
+node "$GS/bin/kill-visible.mjs"     # same as visible.mjs --kill
+node "$GS/bin/cdp-visible.mjs" list # safe CDP: GreedySearch visible Chrome only
+node "$GS/bin/cdp-headless.mjs" list # safe CDP: GreedySearch headless Chrome only
+node "$GS/bin/cdp-greedy.mjs" list  # safe CDP: any GreedySearch Chrome mode
+```
+
+npm global install path:
+
+```bash
+GS="$(npm root -g)/@apmantza/greedysearch-pi"
+node "$GS/bin/launch.mjs" --status
+node "$GS/bin/visible.mjs"
+node "$GS/bin/visible.mjs" --kill
+node "$GS/bin/kill-visible.mjs"
+node "$GS/bin/cdp-visible.mjs" list
+node "$GS/bin/cdp-headless.mjs" list
+node "$GS/bin/cdp-greedy.mjs" list
+```
+
+Chrome is auto-cleaned after 5 min idle. Override with `GREEDY_SEARCH_IDLE_TIMEOUT_MINUTES=10` or disable with `0`.
+
+**CDP safety:** use `cdp-visible.mjs`, `cdp-headless.mjs`, or `cdp-greedy.mjs` for debugging. They always set `CDP_PROFILE_DIR` to the dedicated GreedySearch Chrome profile and never fall back to your main Chrome session. Avoid calling raw `bin/cdp.mjs` manually unless you explicitly set `CDP_PROFILE_DIR`.
+
+## Requirements
+
+- Chrome
+- Node.js 22+
+
+## Known engine quirks
+
+### Bing Copilot
+
+Bing Copilot detects headless Chrome and sandboxes all AI responses inside nested iframes (`copilot.microsoft.com` → `copilot.fun` → `blob:`). In this mode the copy button is hidden and the Cloudflare Turnstile challenge blocks content delivery. The clipboard-based extraction cannot work.
+
+**Auto-recovery:** When Bing or Perplexity fails with a headless-only extraction error (clipboard, verification, timeout, Cloudflare), GreedySearch automatically switches to **visible Chrome** and retries, even in `fast` mode. If manual verification is required, the visible browser is left open and the tool returns instructions to solve the challenge and rerun the same search.
+
+If you prefer to skip the auto-recovery delay, launch visible Chrome ahead of time with `/greedy-visible`, set `GREEDY_SEARCH_ALWAYS_VISIBLE=1`, or pass `visible: true` to `greedy_search`.
+
+## Anti-detection
+
+Headless Chrome auto-injects stealth patches before any page JavaScript runs:
+
+- `navigator.webdriver` hidden, plugins/languages faked, `window.chrome` shimmed
+- WebGL vendor spoofed (Intel Iris), realistic hardware concurrency / memory
+- CDP automation markers deleted, `requestAnimationFrame` kept alive
+- Human-like click simulation with coordinate jitter and variable delays
+
+This bypasses casual bot detection (basic `navigator.webdriver` checks) but does not defeat commercial anti-bot services (DataDome, PerimeterX, Kasada). **Bing Copilot specifically detects headless and sandboxes responses behind Cloudflare Turnstile** — see [Known engine quirks](#known-engine-quirks) for the auto-recovery mechanism.
+
+When using `depth: "standard"` or `depth: "deep"`, source content is fetched and synthesized:
+
+- **Reddit** — Uses Reddit's public `.json` API for posts and comments (no scraping)
+- **GitHub** — Uses GitHub REST API for repos, READMEs, and file trees
+- **General web** — Mozilla Readability extraction with browser fallback for bot-blocked pages
+- **Metadata** — title, author/byline, site name, publish date, language, excerpt
+
+## Project layout
+
+- `bin/` — runtime CLIs (`search.mjs`, `launch.mjs`, `launch-visible.mjs`, `visible.mjs`, `kill-visible.mjs`, safe CDP wrappers, `cdp.mjs`)
+- `extractors/` — engine-specific automation + stealth/consent handling
+- `src/` — search pipeline, chrome management, source fetching, formatting
+- `skills/` — Pi skill metadata
+
+## Testing
+
+Cross-platform test runner (Windows + Unix):
+
+```bash
+npm test              # run all tests
+npm run test:quick    # skip slow tests
+npm run test:smoke    # basic health check
+```
+
+Full bash test suite (Unix only):
+
+```bash
+npm run test:bash           # comprehensive tests
+./test.sh parallel          # race condition tests
+./test.sh flags             # flag/option tests
+```
+
+## Changelog
+
+See `CHANGELOG.md`.
+
+## License
+
+MIT

package/bin/cdp-greedy.mjs

@@ -0,0 +1,62 @@
+#!/usr/bin/env node
+// cdp-greedy.mjs — safe CDP wrapper for the dedicated GreedySearch Chrome.
+//
+// This ALWAYS sets CDP_PROFILE_DIR to the GreedySearch profile so it never
+// falls back to the user's main Chrome DevToolsActivePort.
+//
+// Usage:
+//   node bin/cdp-greedy.mjs list
+//   node bin/cdp-greedy.mjs --mode visible list
+//   node bin/cdp-greedy.mjs --mode headless snap <tab>
+
+import { spawn } from "node:child_process";
+import { existsSync, readFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+
+const tmp = tmpdir().replaceAll("\\", "/");
+const PROFILE_DIR = `${tmp}/greedysearch-chrome-profile`;
+const ACTIVE_PORT = `${PROFILE_DIR}/DevToolsActivePort`;
+const MODE_FILE = `${tmp}/greedysearch-chrome-mode`;
+
+const args = process.argv.slice(2);
+let desiredMode = null;
+const modeIdx = args.indexOf("--mode");
+if (modeIdx !== -1) {
+	desiredMode = args[modeIdx + 1] || null;
+	args.splice(modeIdx, 2);
+}
+
+if (desiredMode && !["visible", "headless"].includes(desiredMode)) {
+	console.error(`Invalid --mode ${desiredMode}. Use visible or headless.`);
+	process.exit(2);
+}
+
+if (!existsSync(ACTIVE_PORT)) {
+	console.error(
+		`GreedySearch Chrome is not running (missing ${ACTIVE_PORT}). Launch with bin/visible.mjs or bin/launch.mjs.`,
+	);
+	process.exit(1);
+}
+
+if (desiredMode) {
+	const actualMode = existsSync(MODE_FILE)
+		? readFileSync(MODE_FILE, "utf8").trim()
+		: "unknown";
+	if (actualMode !== desiredMode) {
+		console.error(
+			`GreedySearch Chrome is ${actualMode}, not ${desiredMode}. Refusing to attach.`,
+		);
+		process.exit(1);
+	}
+}
+
+const cdpBin = new URL("./cdp.mjs", import.meta.url).pathname.replace(
+	/^\/([A-Z]:)/,
+	"$1",
+);
+
+const proc = spawn(process.execPath, [cdpBin, ...args], {
+	stdio: "inherit",
+	env: { ...process.env, CDP_PROFILE_DIR: PROFILE_DIR },
+});
+proc.on("close", (code) => process.exit(code ?? 0));

package/bin/cdp-headless.mjs

@@ -0,0 +1,16 @@
+#!/usr/bin/env node
+// cdp-headless.mjs — safe CDP wrapper for headless GreedySearch Chrome only.
+
+import { spawn } from "node:child_process";
+
+const cdpGreedyBin = new URL(
+	"./cdp-greedy.mjs",
+	import.meta.url,
+).pathname.replace(/^\/([A-Z]:)/, "$1");
+
+const proc = spawn(
+	process.execPath,
+	[cdpGreedyBin, "--mode", "headless", ...process.argv.slice(2)],
+	{ stdio: "inherit" },
+);
+proc.on("close", (code) => process.exit(code ?? 0));

package/bin/cdp-visible.mjs

@@ -0,0 +1,16 @@
+#!/usr/bin/env node
+// cdp-visible.mjs — safe CDP wrapper for visible GreedySearch Chrome only.
+
+import { spawn } from "node:child_process";
+
+const cdpGreedyBin = new URL(
+	"./cdp-greedy.mjs",
+	import.meta.url,
+).pathname.replace(/^\/([A-Z]:)/, "$1");
+
+const proc = spawn(
+	process.execPath,
+	[cdpGreedyBin, "--mode", "visible", ...process.argv.slice(2)],
+	{ stdio: "inherit" },
+);
+proc.on("close", (code) => process.exit(code ?? 0));