@lincoln504/pi-research 1.5.2 → 1.6.0

package/README.md

@@ -27,7 +27,7 @@
 <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-97ca00?style=flat-square" /></a><br />
 <a href="https://github.com/Lincoln504/pi-research"><img alt="GitHub repository" src="https://img.shields.io/badge/GitHub-Lincoln504%2Fpi--research-181717?style=flat-square&logo=github&logoColor=white" /></a>
 
-Search and scraping run locally through a stealth browser with no search provider and no monthly cap. You pay only for the LLM tokens you use.
+Search and scraping run locally through a stealth browser with no search provider and no monthly cap. The only cost is LLM tokens.
 
 ### Install
 
@@ -37,45 +37,45 @@ As a [Pi](https://github.com/earendil-works/pi) Coding Agent extension:
 pi install npm:@lincoln504/pi-research
 ```
 
-Standalone, as an [agent skill](docs/AGENT-SKILL.md) for Claude Code, Codex, and OpenClaw:
+Standalone, as an [agent skill](docs/AGENT-SKILL.md) for Claude Code, Codex, and other skills-compatible agents:
 
 ```bash
 npm install -g @lincoln504/pi-research
 pi-research skill install
 ```
 
-In pi it works out of the box on your session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
+In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
 
 ### How it works
 
 A research run loops through agent teams: a coordinator plans and opens with a search, researcher agents scrape and read in parallel, and a research lead decides whether to go another round — then writes the report from every report collected. The result is one cited Markdown report with findings optionally saved to the knowledge store.
 
-Three depth levels (normal, deep, ultra) set the team size and number of rounds. Ask in natural language and the tool picks the right one:
+Three depth levels (normal, deep, ultra) set the team size and number of rounds. A natural-language request is enough — the tool picks the right one:
 
 ![Prompt-driven multi-round research in the pi TUI](https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/02-prompt-research.gif)
 
 ### Use cases
 
-- Researching for your pi. Plug and play with no API key needed.
-- Researching from Claude Code, Codex, OpenClaw, or another coding agent while a cheaper or local model drives the run so it doesn't spend your main agent's budget.
+- Researching inside pi. Plug and play with no API key needed.
+- Researching from Claude Code, Codex, or another coding agent while a cheaper or local model drives the run, so it doesn't spend the main agent's budget.
 - Keeping a persistent and searchable knowledge store of findings, scoped globally or per project.
-- Building agent systems that find and read web content or populating a dataset of web sources.
+- Building agent systems that find and read web content, or populating a dataset of web sources.
 
 ### Why pi-research
 
 - **No quota and no monthly fee.** Most AI search providers cap free searches and then charge, with results served from their index on their servers. pi-research searches and scrapes locally through DuckDuckGo in a stealth browser.
-- **You own your index.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
-- **Cited reports rather than raw dumps.** You get one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
-- **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything. It can read files, so treat it as you would any read-only agent with access to your machine.
+- **The index stays local.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
+- **Cited reports rather than raw dumps.** Each run returns one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
+- **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything — the right shape for an agent whose whole job is reading untrusted web pages. Prompt injection picked up mid-run has nothing to act on.
 - **Search a little or a lot.** Depth levels span a quick pass to a large-scale investigation.
 
 ### Requirements / limitations
 
 - Node.js >= 22.19.0
-- An LLM with a 100k+ context window (bring your own key or use a local model)
+- An LLM with a 100k+ context window (an API key or a local model)
 - Internet access on a residential IP, since search, scraping, and YouTube transcripts get bot-blocked from datacenter/VPS/cloud IPs
 - pi. The pi extension uses the host's copy while the standalone CLI and agent skill install it as a dependency.
-- Cloudflare and similar systems block scraping on some sites. A run flags pages it can't reach and compensates with a wide pool of search results to cite.
+- Cloudflare and similar systems block scraping on some sites. A run flags unreachable pages and compensates with a wide pool of search results to cite.
 
 ### Channels
 

package/{skills → agent-skill}/pi-research/README.md

@@ -3,8 +3,8 @@
 A portable [Agent Skill](https://agentskills.io/specification) that gives any
 coding agent multi-agent web research by driving the
 [pi-research](https://github.com/Lincoln504/pi-research) engine. It is the same
-engine the `pi` extension uses, packaged as a skill so Claude, Cursor,
-Codex, OpenClaw, and other Agent-Skills-compatible tools can use it.
+engine the `pi` extension uses, packaged as a skill so Claude, Codex, and other
+Agent-Skills-compatible tools can use it.
 
 ### What it exposes
 
@@ -50,9 +50,9 @@ node "<skill_dir>/scripts/run.mjs" status  # verify detection
 ### Install the skill
 
 Recommended. From the pi extension, run `/research-config` and choose
-Install in External Agents. It detects Claude, Codex, and OpenClaw and
-symlinks this skill into each one that is installed. Remove from External Agents
-removes those symlinks.
+Install in External Agents. It detects the supported agents set up on this machine
+and symlinks this skill into each one. Remove from External Agents removes those
+symlinks.
 
 Uninstalling the npm package does **not** remove them: npm 7 and newer do not run a
 package's own `preuninstall` script (verified against npm 11). Use Remove from
@@ -60,9 +60,6 @@ External Agents first, or run `node scripts/cleanup.cjs` from the package direct
 otherwise the symlinks, `~/.pi/research/state` and the `~/.cache/pi-research` model
 cache are all left behind.
 
-Cursor is not auto-installed — it has no global skills directory and only reads
-project-level `.cursor/skills/`. Symlink it per-project instead (see the table).
-
 Manual. Or symlink this `pi-research/` directory into your agent's skills folder:
 
 | Agent | Location |
@@ -71,7 +68,6 @@ Manual. Or symlink this `pi-research/` directory into your agent's skills folder
 | Claude (project)   | `<project>/.claude/skills/pi-research/` |
 | Codex CLI (personal) | `~/.codex/skills/pi-research/` |
 | Codex CLI (project)  | `<project>/.codex/skills/pi-research/` |
-| Cursor (project-only) | `<project>/.cursor/skills/pi-research/` |
 | OpenClaw (personal) | `~/.openclaw/skills/pi-research/` |
 
 Then ask your agent to pi-research something — its skill system matches the

package/{skills → agent-skill}/pi-research/SKILL.md

@@ -2,15 +2,16 @@
 name: pi-research
 description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
 license: MIT
-metadata: { "author": "Lincoln Deen", "version": "1.5.2", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
+metadata: { "author": "Lincoln Deen", "version": "1.6.0", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
 allowed-tools: Bash(node:*)
 ---
 
 # pi-research skill
 
-Run via your shell tool — the **Bash** tool in Claude Code, the **`exec`** tool in
-OpenClaw. `<SKILL_DIR>` is this skill's own directory: substitute the absolute path
-your harness reports for this skill (in OpenClaw, `{baseDir}` resolves to it).
+Run via your shell tool — the **Bash** tool in Claude Code, whatever equivalent
+your harness exposes elsewhere. `<SKILL_DIR>` is this skill's own directory:
+substitute the absolute path your harness reports for this skill (some harnesses
+expose it as `{baseDir}`).
 
 ```
 node "<SKILL_DIR>/scripts/run.mjs" research  "<query>" --depth 1

package/{skills → agent-skill}/pi-research/scripts/run.mjs

@@ -1,6 +1,6 @@
 #!/usr/bin/env node
 
-// skills/pi-research/scripts/run.ts
+// agent-skill/pi-research/scripts/run.ts
 import { spawn } from "node:child_process";
 import { existsSync, readFileSync } from "node:fs";
 import { homedir, constants as osConstants } from "node:os";

package/dist/cli.mjs

@@ -7273,11 +7273,10 @@ function assertValidEmbeddingVectors(vectors, expectedCount, declaredDim) {
   }
 }
 async function addDocumentsToStore(table, docs, embedder, isClosing, workspace, isGlobal) {
-  if (docs.length === 0) return;
+  if (docs.length === 0) return "empty";
   if (isClosing()) {
     logger.warn("[store] Ignoring addDocuments during close");
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "ignored_closing" });
-    return;
+    return "ignored_closing";
   }
   const startTime = Date.now();
   try {
@@ -7297,14 +7296,13 @@ async function addDocumentsToStore(table, docs, embedder, isClosing, workspace,
     await table.add(data);
     const duration = Date.now() - startTime;
     metrics.observe("knowledge_store_add_documents_duration_ms", duration);
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "success" });
     metrics.increment("knowledge_store_chunks_added_total", docs.length);
     logger.log(`[store] Added ${docs.length} chunk(s) for ${docs[0]?.url} [workspace=${workspace}, global=${isGlobal}]`);
+    return "written";
   } catch (err) {
     const duration = Date.now() - startTime;
     metrics.observe("knowledge_store_add_documents_duration_ms", duration, { status: "error" });
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "error" });
-    logger.error("[store] Failed to add documents:", err);
+    logger.debug(`[store] add attempt failed after ${duration}ms: ${err instanceof Error ? err.message : String(err)}`);
     throw err;
   }
 }
@@ -7876,18 +7874,19 @@ var init_store = __esm({
         try {
           if (this.isClosing) {
             logger.warn(`[store] Skipping addDocuments \u2014 store is closing`);
+            metrics.increment("knowledge_store_add_documents_total", 1, { status: "ignored_closing" });
             return;
           }
           let table = await this.getFreshTable();
           const workspace = this.getWorkspace();
-          await this.withEmbedderReconnect(async (embedder) => {
+          const outcome = await this.withEmbedderReconnect(async (embedder) => {
             let retryCount = 0;
             const MAX_RETRIES = 5;
             const BASE_DELAY = 100;
             while (retryCount <= MAX_RETRIES) {
               try {
                 const isGlobal = this.options.knowledgeMode === "global";
-                await addDocumentsToStore(
+                return await addDocumentsToStore(
                   table,
                   docs,
                   embedder,
@@ -7895,7 +7894,6 @@ var init_store = __esm({
                   workspace,
                   isGlobal
                 );
-                return;
               } catch (err) {
                 if (isLanceCommitConflict(err)) {
                   retryCount++;
@@ -7909,7 +7907,17 @@ var init_store = __esm({
                 throw err;
               }
             }
+            throw new Error("[store] addDocuments retry loop exited without result");
           });
+          if (outcome !== "empty") {
+            metrics.increment("knowledge_store_add_documents_total", 1, {
+              status: outcome === "ignored_closing" ? "ignored_closing" : "success"
+            });
+          }
+        } catch (err) {
+          metrics.increment("knowledge_store_add_documents_total", 1, { status: "error" });
+          logger.error("[store] Failed to add documents:", err);
+          throw err;
         } finally {
           this.activeWrites.delete(writePromise);
           writeResolve();
@@ -10157,7 +10165,8 @@ function resolveSkillSourceDir(explicit) {
     if (!start) continue;
     let dir = start;
     for (let i = 0; i < 8; i++) {
-      candidates.push(path18.join(dir, "skills", SKILL_NAME));
+      candidates.push(path18.join(dir, SKILL_SOURCE_DIR, SKILL_NAME));
+      candidates.push(path18.join(dir, LEGACY_SKILL_SOURCE_DIR, SKILL_NAME));
       const parent = path18.dirname(dir);
       if (parent === dir) break;
       dir = parent;
@@ -10169,7 +10178,7 @@ function resolveSkillSourceDir(explicit) {
     } catch {
     }
   }
-  throw new Error("Could not locate the bundled research skill (skills/pi-research/SKILL.md). Set PI_RESEARCH_SKILL_DIR to its path.");
+  throw new Error("Could not locate the bundled research skill (agent-skill/pi-research/SKILL.md). Set PI_RESEARCH_SKILL_DIR to its path.");
 }
 function getManifestPath(opts) {
   return path18.join(homeDir(opts), ".pi", "research", "installed-skills.json");
@@ -10213,7 +10222,7 @@ function isOwnedSymlink(targetPath, expectedSource) {
     if (!st.isSymbolicLink()) return false;
     const dest = path18.resolve(path18.dirname(targetPath), fs16.readlinkSync(targetPath));
     if (expectedSource && path18.resolve(expectedSource) === dest) return true;
-    return /[/\\]pi-research[/\\]skills[/\\]pi-research$/.test(dest);
+    return OWNED_SYMLINK_DEST.test(dest);
   } catch {
     return false;
   }
@@ -10564,20 +10573,28 @@ function copyDir(src, dest) {
     fs16.copyFileSync(src, dest);
   }
 }
-var HARNESSES, SKILL_NAME, PACKAGE_NAME, SKILL_AGENT_TARGETS, manifestTmpSeq, isTarget;
+var HARNESSES, SKILL_NAME, PACKAGE_NAME, SKILL_SOURCE_DIR, LEGACY_SKILL_SOURCE_DIR, OWNED_SYMLINK_DEST, SKILL_AGENT_TARGETS, manifestTmpSeq, isTarget;
 var init_skill_installer = __esm({
   "src/skill-install/skill-installer.ts"() {
     "use strict";
     HARNESSES = [
       { id: "claude", label: "Claude", baseDir: ".claude", skillsDir: path18.join(".claude", "skills"), confidence: "confirmed" },
       { id: "openclaw", label: "OpenClaw", baseDir: ".openclaw", skillsDir: path18.join(".openclaw", "skills"), confidence: "confirmed", note: "OpenClaw reads ~/.openclaw/skills as a managed skill root and accepts symlinked skill folders (docs.openclaw.ai/tools/skills)." },
-      { id: "pi", label: "pi", baseDir: ".pi", skillsDir: path18.join(".pi", "skills"), confidence: "confirmed" },
-      { id: "cursor", label: "Cursor", baseDir: ".cursor", skillsDir: path18.join(".cursor", "skills"), confidence: "partial", note: "Cursor project-level skills are documented; the global ~/.cursor/skills path is community-reported." },
+      // pi's own skill root is under its agent dir (~/.pi/agent/skills), NOT ~/.pi/skills.
+      // Listed for DETECTION only — pi is never an install target (see SKILL_AGENT_TARGETS):
+      // the host already exposes research as a native tool, so a skill there would just
+      // duplicate it through a slower subprocess.
+      { id: "pi", label: "pi", baseDir: ".pi", skillsDir: path18.join(".pi", "agent", "skills"), confidence: "confirmed" },
       { id: "codex", label: "OpenAI Codex CLI", baseDir: ".codex", skillsDir: path18.join(".codex", "skills"), confidence: "unverified", note: "Codex skills support is emerging; path not confirmed by official docs." },
       { id: "agents", label: "Cross-tool (~/.agents/skills)", baseDir: ".agents", skillsDir: path18.join(".agents", "skills"), confidence: "unverified", note: "Proposed Universal Agents convention; not yet a ratified standard." }
     ];
     SKILL_NAME = "pi-research";
     PACKAGE_NAME = "@lincoln504/pi-research";
+    SKILL_SOURCE_DIR = "agent-skill";
+    LEGACY_SKILL_SOURCE_DIR = "skills";
+    OWNED_SYMLINK_DEST = new RegExp(
+      `[/\\\\]${SKILL_NAME}[/\\\\](?:${SKILL_SOURCE_DIR}|${LEGACY_SKILL_SOURCE_DIR})[/\\\\]${SKILL_NAME}$`
+    );
     SKILL_AGENT_TARGETS = ["claude", "codex", "openclaw"];
     manifestTmpSeq = 0;
     isTarget = (id) => SKILL_AGENT_TARGETS.includes(id);
@@ -12427,6 +12444,101 @@ function getClientAgent() {
   return clientAgent;
 }
 
+// src/infrastructure/browser/browser-error-utils.ts
+init_circuit_breaker();
+function isTransientSocketError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("ECONNREFUSED") || err.message.includes("ECONNRESET") || err.message.includes("socket hang up") || err.message.includes("EPIPE") || err.message.includes("ETIMEDOUT") || err.message.includes("timed out") || err.message.includes("pool busy") || err.message.includes("unreachable") || // WorkerPoolManager throws this during the window between forceSchedulerRestart
+  // clearing the scheduler reference and the old pool's 1500ms drain completing.
+  // It is a transient state — retry after a short delay succeeds once drain finishes.
+  err.message.includes("Worker pool is shutting down") || // Poolifier throws this when pool.execute() is called while the pool is being
+  // destroyed (e.g., during scheduler restart). Treat as transient — the pool will
+  // be ready for new tasks once destruction completes and a new pool is initialized.
+  err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // The leader answered 503 because it is shutting down after losing
+  // leadership. Retrying re-elects and reaches the new leader, so this is
+  // transient in exactly the same sense as a pool drain.
+  err.message.includes("Browser pool leader is draining"));
+}
+function isCloudflareBlockError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("Fetch blocked: Cloudflare") || err.message.includes("Cloudflare challenge"));
+}
+function isTaskTimeoutError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("timed out after") || err.message.includes("Search task timed out") || err.message.includes("Scrape task timed out") || err.message.includes("Health check timed out") || err.message.includes("[BrowserClient] Request to"));
+}
+function isServerDrainingError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && err.message.includes("Browser pool leader is draining");
+}
+function isPoolShutdownError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("Worker pool is shutting down") || err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // worker-pool-manager throws this when auto-recovery is still swapping the
+  // pool. Its own text says "please retry", but until it was listed here no
+  // classifier matched it, so the first attempt rethrew instead of retrying —
+  // and auto-recovery fires only after 3 consecutive worker failures, i.e.
+  // exactly when a burst of tasks is in flight and would all die together.
+  err.message.includes("Worker pool is being reset") || // poolifier's wording once destroy() has COMPLETED (vs "destroying pool"
+  // while it is in progress). A queued closure holding a pool captured before
+  // an auto-recovery destroy dispatches into exactly this window; it is the
+  // same transient swap the "being reset" string covers, not a hard fault.
+  err.message.includes("Cannot execute a task on not started pool"));
+}
+var BENIGN_NAVIGATION_FAILURE = /page\.goto: (?:Timeout \d+ *ms exceeded|net::|NS_ERROR_|SEC_ERROR_|MOZILLA_PKIX_ERROR_)/;
+function isBenignScrapeFailure(error) {
+  const msg2 = error instanceof Error ? error.message : String(error);
+  return msg2.startsWith("Fetch blocked:") || // bot-protection interstitial (Cloudflare, DDoS-Guard, …)
+  msg2.startsWith("Fetch returned stub:") || // nav-only / near-empty page
+  msg2.startsWith("Could not extract content from PDF") || // malformed/encrypted/scanned-only PDF
+  msg2.startsWith("PDF too large") || // exceeds the deliberate 100MB policy cap
+  msg2.startsWith("Unsupported download:") || // attachment-served non-PDF (zip/xlsx/…) — nothing to extract
+  BENIGN_NAVIGATION_FAILURE.test(msg2) || // remote TLS/DNS/timeout — the URL could not be loaded
+  msg2.startsWith("HTML response too large") || // fetch-layer 25MB policy cap — same class as the PDF cap
+  msg2.startsWith("Browser HTML too large") || // browser-layer 25MB policy cap
+  /^HTTP \d{3}\b/.test(msg2) || // remote 4xx/5xx status
+  isCloudflareBlockError(error) || isTaskTimeoutError(error) || isPoolShutdownError(error);
+}
+var sessionCircuitBreakers = /* @__PURE__ */ new Map();
+var MAX_SESSION_BREAKERS = 256;
+var DEFAULT_BREAKER_CONFIG = {
+  // Raised from 3→8: parallel search bursts of 20 queries frequently produce
+  // simultaneous timeouts on slow/CF-protected sites. A threshold of 3 caused
+  // the breaker to open mid-burst, blocking all subsequent requests for the round.
+  failureThreshold: 8,
+  // Raised from 15s→45s: enough time for one full search burst + retry window
+  // to complete before the breaker attempts to half-open.
+  resetTimeoutMs: 45e3,
+  name: "BrowserPool",
+  isTransientError: (error) => {
+    if (isPoolShutdownError(error)) return false;
+    if (isServerDrainingError(error)) return false;
+    if (isCloudflareBlockError(error)) return false;
+    if (isTaskTimeoutError(error)) return false;
+    return isTransientSocketError(error);
+  }
+};
+function getBrowserCircuitBreaker(sessionId) {
+  let breaker2 = sessionCircuitBreakers.get(sessionId);
+  if (!breaker2) {
+    if (sessionCircuitBreakers.size >= MAX_SESSION_BREAKERS) {
+      const oldest = sessionCircuitBreakers.keys().next().value;
+      if (oldest !== void 0) sessionCircuitBreakers.delete(oldest);
+    }
+    breaker2 = new CircuitBreaker({ ...DEFAULT_BREAKER_CONFIG, name: `BrowserPool-${sessionId}` });
+    sessionCircuitBreakers.set(sessionId, breaker2);
+  }
+  return breaker2;
+}
+function clearSessionCircuitBreaker(sessionId) {
+  sessionCircuitBreakers.delete(sessionId);
+}
+var browserCircuitBreaker = new CircuitBreaker(DEFAULT_BREAKER_CONFIG);
+
 // src/infrastructure/browser/browser-client.ts
 var CLIENT_PATIENCE_MARGIN_MS = 15e3;
 var CLIENT_TIMEOUT_CAP_MS = 3e5;
@@ -12588,12 +12700,16 @@ var BrowserClient = class {
           errorType = "unknown";
         }
         const error = new Error(errorMsg);
-        logger.error(`[BrowserClient] Request to http://127.0.0.1:${this.port}${path20} failed:`, errorMsg);
-        errorTracker.trackError(error, {
-          component: "browser-manager",
-          operation,
-          errorType
-        });
+        if (isTransientSocketError(error)) {
+          logger.debug(`[BrowserClient] Transient failure talking to http://127.0.0.1:${this.port}${path20} (caller will retry): ${errorMsg}`);
+        } else {
+          logger.error(`[BrowserClient] Request to http://127.0.0.1:${this.port}${path20} failed:`, errorMsg);
+          errorTracker.trackError(error, {
+            component: "browser-manager",
+            operation,
+            errorType
+          });
+        }
         reject(error);
       });
       req.write(JSON.stringify(data));
@@ -12635,89 +12751,6 @@ init_logger();
 init_http_body();
 import * as http5 from "node:http";
 import * as crypto5 from "node:crypto";
-
-// src/infrastructure/browser/browser-error-utils.ts
-init_circuit_breaker();
-function isTransientSocketError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("ECONNREFUSED") || err.message.includes("ECONNRESET") || err.message.includes("socket hang up") || err.message.includes("EPIPE") || err.message.includes("ETIMEDOUT") || err.message.includes("timed out") || err.message.includes("pool busy") || err.message.includes("unreachable") || // WorkerPoolManager throws this during the window between forceSchedulerRestart
-  // clearing the scheduler reference and the old pool's 1500ms drain completing.
-  // It is a transient state — retry after a short delay succeeds once drain finishes.
-  err.message.includes("Worker pool is shutting down") || // Poolifier throws this when pool.execute() is called while the pool is being
-  // destroyed (e.g., during scheduler restart). Treat as transient — the pool will
-  // be ready for new tasks once destruction completes and a new pool is initialized.
-  err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // The leader answered 503 because it is shutting down after losing
-  // leadership. Retrying re-elects and reaches the new leader, so this is
-  // transient in exactly the same sense as a pool drain.
-  err.message.includes("Browser pool leader is draining"));
-}
-function isCloudflareBlockError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("Fetch blocked: Cloudflare") || err.message.includes("Cloudflare challenge"));
-}
-function isTaskTimeoutError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("timed out after") || err.message.includes("Search task timed out") || err.message.includes("Scrape task timed out") || err.message.includes("Health check timed out") || err.message.includes("[BrowserClient] Request to"));
-}
-function isServerDrainingError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && err.message.includes("Browser pool leader is draining");
-}
-function isPoolShutdownError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("Worker pool is shutting down") || err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // worker-pool-manager throws this when auto-recovery is still swapping the
-  // pool. Its own text says "please retry", but until it was listed here no
-  // classifier matched it, so the first attempt rethrew instead of retrying —
-  // and auto-recovery fires only after 3 consecutive worker failures, i.e.
-  // exactly when a burst of tasks is in flight and would all die together.
-  err.message.includes("Worker pool is being reset") || // poolifier's wording once destroy() has COMPLETED (vs "destroying pool"
-  // while it is in progress). A queued closure holding a pool captured before
-  // an auto-recovery destroy dispatches into exactly this window; it is the
-  // same transient swap the "being reset" string covers, not a hard fault.
-  err.message.includes("Cannot execute a task on not started pool"));
-}
-var sessionCircuitBreakers = /* @__PURE__ */ new Map();
-var MAX_SESSION_BREAKERS = 256;
-var DEFAULT_BREAKER_CONFIG = {
-  // Raised from 3→8: parallel search bursts of 20 queries frequently produce
-  // simultaneous timeouts on slow/CF-protected sites. A threshold of 3 caused
-  // the breaker to open mid-burst, blocking all subsequent requests for the round.
-  failureThreshold: 8,
-  // Raised from 15s→45s: enough time for one full search burst + retry window
-  // to complete before the breaker attempts to half-open.
-  resetTimeoutMs: 45e3,
-  name: "BrowserPool",
-  isTransientError: (error) => {
-    if (isPoolShutdownError(error)) return false;
-    if (isServerDrainingError(error)) return false;
-    if (isCloudflareBlockError(error)) return false;
-    if (isTaskTimeoutError(error)) return false;
-    return isTransientSocketError(error);
-  }
-};
-function getBrowserCircuitBreaker(sessionId) {
-  let breaker2 = sessionCircuitBreakers.get(sessionId);
-  if (!breaker2) {
-    if (sessionCircuitBreakers.size >= MAX_SESSION_BREAKERS) {
-      const oldest = sessionCircuitBreakers.keys().next().value;
-      if (oldest !== void 0) sessionCircuitBreakers.delete(oldest);
-    }
-    breaker2 = new CircuitBreaker({ ...DEFAULT_BREAKER_CONFIG, name: `BrowserPool-${sessionId}` });
-    sessionCircuitBreakers.set(sessionId, breaker2);
-  }
-  return breaker2;
-}
-function clearSessionCircuitBreaker(sessionId) {
-  sessionCircuitBreakers.delete(sessionId);
-}
-var browserCircuitBreaker = new CircuitBreaker(DEFAULT_BREAKER_CONFIG);
-
-// src/infrastructure/browser/browser-server.ts
 var _authSecret = null;
 function getOrCreateAuthSecret() {
   if (!_authSecret) {
@@ -12859,6 +12892,8 @@ var BrowserServer = class {
               logger.warn("[BrowserServer] Request blocked by bot protection:", error instanceof Error ? error.message : String(error));
             } else if (isPoolShutdownError(error)) {
               logger.debug("[BrowserServer] Request abandoned during shutdown:", error instanceof Error ? error.message : String(error));
+            } else if (isBenignScrapeFailure(error)) {
+              logger.debug("[BrowserServer] Expected per-URL scrape outcome:", error instanceof Error ? error.message.split("\n")[0] : String(error));
             } else {
               logger.error("[BrowserServer] Error handling request:", error);
             }
@@ -18051,17 +18086,6 @@ import * as dnsCb from "node:dns";
 import * as net3 from "node:net";
 import crypto9 from "node:crypto";
 import { NodeHtmlMarkdown } from "node-html-markdown";
-function isBenignScrapeFailure(error) {
-  const msg2 = error instanceof Error ? error.message : String(error);
-  return msg2.startsWith("Fetch blocked:") || // bot-protection interstitial (Cloudflare, DDoS-Guard, …)
-  msg2.startsWith("Fetch returned stub:") || // nav-only / near-empty page
-  msg2.startsWith("Could not extract content from PDF") || // malformed/encrypted/scanned-only PDF
-  msg2.startsWith("PDF too large") || // exceeds the deliberate 100MB policy cap
-  msg2.startsWith("HTML response too large") || // fetch-layer 25MB policy cap — same class as the PDF cap
-  msg2.startsWith("Browser HTML too large") || // browser-layer 25MB policy cap
-  /^HTTP \d{3}\b/.test(msg2) || // remote 4xx/5xx status
-  isCloudflareBlockError(error) || isTaskTimeoutError(error) || isPoolShutdownError(error);
-}
 function getRandomUserAgent() {
   try {
     const index = crypto9.randomInt(0, USER_AGENTS.length);
@@ -21962,9 +21986,14 @@ async function doMint(identifiers, opts) {
   const requestKey = opts.requestKey || process.env["PI_RESEARCH_YOUTUBE_POTOKEN_REQUEST_KEY"] || DEFAULT_REQUEST_KEY;
   const fetchImpl = opts.fetchImpl ?? fetch;
   const started = Date.now();
-  const bgMod = await import("bgutils-js");
-  const BG = bgMod.BG ?? bgMod.default;
-  const { buildURL: buildURL2, getHeaders } = bgMod;
+  const [bgGuard, bgWebPo, bgUtils] = await Promise.all([
+    import("bgutils-js/botguard"),
+    import("bgutils-js/webpo"),
+    import("bgutils-js/utils")
+  ]);
+  const { getChallenge, BotGuardClient } = bgGuard;
+  const { WebPoMinter } = bgWebPo;
+  const { buildURL: buildURL2, getHeaders } = bgUtils;
   const { JSDOM } = await import("jsdom");
   const dom = new JSDOM('<!DOCTYPE html><html lang="en"><body></body></html>', {
     url: "https://www.youtube.com/"
@@ -21995,24 +22024,24 @@ async function doMint(identifiers, opts) {
   };
   bridge();
   try {
-    const bgConfig = {
-      fetch: (input, init) => fetchImpl(input, {
+    const challenge = await getChallenge({
+      requestKey,
+      fetchFunction: ((input, init) => fetchImpl(input, {
         ...init,
         signal: init?.signal ?? opts.signal
-      }),
-      globalObj: globalThis,
-      identifier: identifiers[0],
-      requestKey
-    };
-    const challenge = await BG.Challenge.create(bgConfig);
+      }))
+    });
     if (!challenge) throw new Error("BotGuard challenge unavailable");
     const interpreter = challenge.interpreterJavascript?.privateDoNotAccessOrElseSafeScriptWrappedValue;
     if (!interpreter) throw new Error("BotGuard interpreter script missing");
     new Function(interpreter)();
-    const botguard = await BG.BotGuardClient.create({
+    if (!challenge.program || !challenge.globalName) {
+      throw new Error("BotGuard challenge missing program/globalName");
+    }
+    const botguard = await BotGuardClient.create({
       program: challenge.program,
       globalName: challenge.globalName,
-      globalObj: globalThis
+      globalObject: globalThis
     });
     const webPoSignalOutput = [];
     const botguardResponse = await botguard.snapshot({ webPoSignalOutput });
@@ -22030,7 +22059,7 @@ async function doMint(identifiers, opts) {
     if (!integrityToken) {
       throw new Error("Integrity token rejected by attestation server (likely a non-residential IP)");
     }
-    const minter = await BG.WebPoMinter.create({ integrityToken }, webPoSignalOutput);
+    const minter = await WebPoMinter.create({ integrityToken }, webPoSignalOutput);
     const tokens = /* @__PURE__ */ new Map();
     for (const id of identifiers) {
       tokens.set(id, await minter.mintAsWebsafeString(id));