@lincoln504/pi-research 1.5.3 → 1.6.1

package/README.md

@@ -27,7 +27,7 @@
 <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-97ca00?style=flat-square" /></a><br />
 <a href="https://github.com/Lincoln504/pi-research"><img alt="GitHub repository" src="https://img.shields.io/badge/GitHub-Lincoln504%2Fpi--research-181717?style=flat-square&logo=github&logoColor=white" /></a>
 
-Search and scraping run locally through a stealth browser with no search provider and no monthly cap. You pay only for the LLM tokens you use.
+Search and scraping run locally through a stealth browser with no search provider and no monthly cap. The only cost is LLM tokens.
 
 ### Install
 
@@ -44,38 +44,38 @@ npm install -g @lincoln504/pi-research
 pi-research skill install
 ```
 
-In pi it works out of the box on your session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
+In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
 
 ### How it works
 
 A research run loops through agent teams: a coordinator plans and opens with a search, researcher agents scrape and read in parallel, and a research lead decides whether to go another round — then writes the report from every report collected. The result is one cited Markdown report with findings optionally saved to the knowledge store.
 
-Three depth levels (normal, deep, ultra) set the team size and number of rounds. Ask in natural language and the tool picks the right one:
+Three depth levels (normal, deep, ultra) set the team size and number of rounds. A natural-language request is enough — the tool picks the right one:
 
 ![Prompt-driven multi-round research in the pi TUI](https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/02-prompt-research.gif)
 
 ### Use cases
 
-- Researching for your pi. Plug and play with no API key needed.
-- Researching from Claude Code, Codex, or another coding agent while a cheaper or local model drives the run so it doesn't spend your main agent's budget.
+- Researching inside pi. Plug and play with no API key needed.
+- Researching from Claude Code, Codex, or another coding agent while a cheaper or local model drives the run, so it doesn't spend the main agent's budget.
 - Keeping a persistent and searchable knowledge store of findings, scoped globally or per project.
-- Building agent systems that find and read web content or populating a dataset of web sources.
+- Building agent systems that find and read web content, or populating a dataset of web sources.
 
 ### Why pi-research
 
 - **No quota and no monthly fee.** Most AI search providers cap free searches and then charge, with results served from their index on their servers. pi-research searches and scrapes locally through DuckDuckGo in a stealth browser.
-- **You own your index.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
-- **Cited reports rather than raw dumps.** You get one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
-- **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything. It can read files, so treat it as you would any read-only agent with access to your machine.
+- **The index stays local.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
+- **Cited reports rather than raw dumps.** Each run returns one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
+- **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything — the right shape for an agent whose whole job is reading untrusted web pages. Prompt injection picked up mid-run has nothing to act on.
 - **Search a little or a lot.** Depth levels span a quick pass to a large-scale investigation.
 
 ### Requirements / limitations
 
 - Node.js >= 22.19.0
-- An LLM with a 100k+ context window (bring your own key or use a local model)
+- An LLM with a 100k+ context window (an API key or a local model)
 - Internet access on a residential IP, since search, scraping, and YouTube transcripts get bot-blocked from datacenter/VPS/cloud IPs
 - pi. The pi extension uses the host's copy while the standalone CLI and agent skill install it as a dependency.
-- Cloudflare and similar systems block scraping on some sites. A run flags pages it can't reach and compensates with a wide pool of search results to cite.
+- Cloudflare and similar systems block scraping on some sites. A run flags unreachable pages and compensates with a wide pool of search results to cite.
 
 ### Channels
 

package/agent-skill/pi-research/SKILL.md

@@ -2,7 +2,7 @@
 name: pi-research
 description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
 license: MIT
-metadata: { "author": "Lincoln Deen", "version": "1.5.3", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
+metadata: { "author": "Lincoln Deen", "version": "1.6.1", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
 allowed-tools: Bash(node:*)
 ---
 

package/dist/cli.mjs

@@ -2216,6 +2216,29 @@ function lastCitedLinksHeader(text) {
   }
   return { lineStart, textStart };
 }
+function stripReportPreamble(text) {
+  const trimmed = text.trimStart();
+  const breakAt = trimmed.indexOf("\n\n");
+  if (breakAt < 0) return { body: text, preamble: "" };
+  const head = trimmed.slice(0, breakAt).trim();
+  const rest = trimmed.slice(breakAt + 2).trimStart();
+  if (head.includes("\n")) return { body: text, preamble: "" };
+  if (!rest) return { body: text, preamble: "" };
+  if (!REPORT_NARRATION_OPENER.test(head)) return { body: text, preamble: "" };
+  if (!REPORT_NARRATION_SUBJECT.test(head)) return { body: text, preamble: "" };
+  return { body: rest, preamble: head };
+}
+function synthesisProseBody(text) {
+  const { lineStart } = lastCitedLinksHeader(text);
+  return (lineStart < 0 ? text : text.slice(0, lineStart)).trim();
+}
+function isAnalysisFreeSynthesis(text, minProseChars) {
+  const prose = synthesisProseBody(text);
+  const stripped = prose.replace(/^[ \t]*(?:#{1,6}[ \t]*|>[ \t]*|\*\*)/gm, "").trim();
+  if (minProseChars > 0 && stripped.length < minProseChars) return true;
+  if (!/[.!?。！？।॥۔؟።]/.test(stripped)) return true;
+  return false;
+}
 function parseCitations(report) {
   const headerIdx = lastCitedLinksHeaderIndex(report);
   if (headerIdx < 0) return [];
@@ -2268,11 +2291,14 @@ function parseCitations(report) {
   }
   return citations;
 }
+var REPORT_NARRATION_OPENER, REPORT_NARRATION_SUBJECT;
 var init_text_utils = __esm({
   "src/utils/text-utils.ts"() {
     "use strict";
     init_logger();
     init_url_utils();
+    REPORT_NARRATION_OPENER = /^(?:I|I'll|I'm|Let me|Now I|Next I|First,? I)\b/i;
+    REPORT_NARRATION_SUBJECT = /\b(?:report|synthesi[sz]\w*|findings|material|sources|scraped|gathered|research\w*|summar\w+)\b/i;
   }
 });
 
@@ -3217,7 +3243,7 @@ function getMaxScrapeBatches(config, cachingActive = false) {
 function resolveExcludedTools(callerExclusions, configDisabled = []) {
   return [.../* @__PURE__ */ new Set([...DEFAULT_EXCLUDED_TOOLS, ...callerExclusions ?? [], ...configDisabled])];
 }
-var MAX_GATHERING_CALLS, MAX_GREP_CALLS, MAX_SCRAPE_URLS, MAX_TEAM_SIZE_LEVEL_1, MAX_TEAM_SIZE_LEVEL_2, MAX_TEAM_SIZE_LEVEL_3, MAX_ROUNDS_LEVEL_1, MAX_ROUNDS_LEVEL_2, MAX_ROUNDS_LEVEL_3, MAX_EXTRA_ROUNDS_WITH_STEERING, OSV_TIMEOUT_MS, WEBGPU_PROBE_TIMEOUT_MS, MAX_QUERY_LENGTH, MAX_FILENAME_QUERY_LENGTH, MIN_QUERY_LENGTH, MAX_EXPORT_RETRIES, DEFAULT_MAX_RETRIES, DEFAULT_INITIAL_DELAY_MS, DEFAULT_MAX_DELAY_MS, DEFAULT_MODEL_CONTEXT_WINDOW, BATCH_2_DEFAULT_CONCURRENCY, MAX_SCRAPE_CONTENT_CHARS_PER_DOC, RESEARCHER_LAUNCH_DELAY_MS, MAX_QUERIES_PER_RESEARCHER_LEVEL_1, MAX_QUERIES_PER_RESEARCHER_LEVEL_2, MAX_QUERIES_PER_RESEARCHER_LEVEL_3, DEFAULT_EXCLUDED_TOOLS, RESEARCH_TOOL_NAMES;
+var MAX_GATHERING_CALLS, MAX_GREP_CALLS, MAX_SCRAPE_URLS, MAX_TEAM_SIZE_LEVEL_1, MAX_TEAM_SIZE_LEVEL_2, MAX_TEAM_SIZE_LEVEL_3, MAX_ROUNDS_LEVEL_1, MAX_ROUNDS_LEVEL_2, MAX_ROUNDS_LEVEL_3, MAX_EXTRA_ROUNDS_WITH_STEERING, OSV_TIMEOUT_MS, WEBGPU_PROBE_TIMEOUT_MS, MAX_QUERY_LENGTH, MAX_FILENAME_QUERY_LENGTH, MIN_QUERY_LENGTH, MAX_EXPORT_RETRIES, DEFAULT_MAX_RETRIES, DEFAULT_INITIAL_DELAY_MS, DEFAULT_MAX_DELAY_MS, DEFAULT_MODEL_CONTEXT_WINDOW, BATCH_2_DEFAULT_CONCURRENCY, MAX_SCRAPE_CONTENT_CHARS_PER_DOC, RESEARCHER_LAUNCH_DELAY_MS, MAX_QUERIES_PER_RESEARCHER_LEVEL_1, MAX_QUERIES_PER_RESEARCHER_LEVEL_2, MAX_QUERIES_PER_RESEARCHER_LEVEL_3, DEFAULT_EXCLUDED_TOOLS, MIN_SYNTHESIS_PROSE_CHARS, QUICK_THIN_REPORT_NOTICE, SYNTHESIS_FALLBACK_NOTICE, RESEARCH_TOOL_NAMES;
 var init_constants = __esm({
   "src/constants.ts"() {
     "use strict";
@@ -3249,6 +3275,9 @@ var init_constants = __esm({
     MAX_QUERIES_PER_RESEARCHER_LEVEL_2 = 15;
     MAX_QUERIES_PER_RESEARCHER_LEVEL_3 = 20;
     DEFAULT_EXCLUDED_TOOLS = ["grep"];
+    MIN_SYNTHESIS_PROSE_CHARS = 800;
+    QUICK_THIN_REPORT_NOTICE = "_Note: this run did not produce a substantive write-up. The sources below were retrieved and are cited accurately, but the summary above should not be treated as a complete answer \u2014 consider re-running, or running at a deeper level._";
+    SYNTHESIS_FALLBACK_NOTICE = "_Note: the synthesis step did not produce a usable summary for this run, so the individual researcher reports are included below in full. The sources cited at the end of this document are unaffected._";
     RESEARCH_TOOL_NAMES = [
       "search",
       "scrape",
@@ -7273,11 +7302,10 @@ function assertValidEmbeddingVectors(vectors, expectedCount, declaredDim) {
   }
 }
 async function addDocumentsToStore(table, docs, embedder, isClosing, workspace, isGlobal) {
-  if (docs.length === 0) return;
+  if (docs.length === 0) return "empty";
   if (isClosing()) {
     logger.warn("[store] Ignoring addDocuments during close");
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "ignored_closing" });
-    return;
+    return "ignored_closing";
   }
   const startTime = Date.now();
   try {
@@ -7297,14 +7325,13 @@ async function addDocumentsToStore(table, docs, embedder, isClosing, workspace,
     await table.add(data);
     const duration = Date.now() - startTime;
     metrics.observe("knowledge_store_add_documents_duration_ms", duration);
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "success" });
     metrics.increment("knowledge_store_chunks_added_total", docs.length);
     logger.log(`[store] Added ${docs.length} chunk(s) for ${docs[0]?.url} [workspace=${workspace}, global=${isGlobal}]`);
+    return "written";
   } catch (err) {
     const duration = Date.now() - startTime;
     metrics.observe("knowledge_store_add_documents_duration_ms", duration, { status: "error" });
-    metrics.increment("knowledge_store_add_documents_total", 1, { status: "error" });
-    logger.error("[store] Failed to add documents:", err);
+    logger.debug(`[store] add attempt failed after ${duration}ms: ${err instanceof Error ? err.message : String(err)}`);
     throw err;
   }
 }
@@ -7876,18 +7903,19 @@ var init_store = __esm({
         try {
           if (this.isClosing) {
             logger.warn(`[store] Skipping addDocuments \u2014 store is closing`);
+            metrics.increment("knowledge_store_add_documents_total", 1, { status: "ignored_closing" });
             return;
           }
           let table = await this.getFreshTable();
           const workspace = this.getWorkspace();
-          await this.withEmbedderReconnect(async (embedder) => {
+          const outcome = await this.withEmbedderReconnect(async (embedder) => {
             let retryCount = 0;
             const MAX_RETRIES = 5;
             const BASE_DELAY = 100;
             while (retryCount <= MAX_RETRIES) {
               try {
                 const isGlobal = this.options.knowledgeMode === "global";
-                await addDocumentsToStore(
+                return await addDocumentsToStore(
                   table,
                   docs,
                   embedder,
@@ -7895,7 +7923,6 @@ var init_store = __esm({
                   workspace,
                   isGlobal
                 );
-                return;
               } catch (err) {
                 if (isLanceCommitConflict(err)) {
                   retryCount++;
@@ -7909,7 +7936,17 @@ var init_store = __esm({
                 throw err;
               }
             }
+            throw new Error("[store] addDocuments retry loop exited without result");
           });
+          if (outcome !== "empty") {
+            metrics.increment("knowledge_store_add_documents_total", 1, {
+              status: outcome === "ignored_closing" ? "ignored_closing" : "success"
+            });
+          }
+        } catch (err) {
+          metrics.increment("knowledge_store_add_documents_total", 1, { status: "error" });
+          logger.error("[store] Failed to add documents:", err);
+          throw err;
         } finally {
           this.activeWrites.delete(writePromise);
           writeResolve();
@@ -12436,6 +12473,101 @@ function getClientAgent() {
   return clientAgent;
 }
 
+// src/infrastructure/browser/browser-error-utils.ts
+init_circuit_breaker();
+function isTransientSocketError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("ECONNREFUSED") || err.message.includes("ECONNRESET") || err.message.includes("socket hang up") || err.message.includes("EPIPE") || err.message.includes("ETIMEDOUT") || err.message.includes("timed out") || err.message.includes("pool busy") || err.message.includes("unreachable") || // WorkerPoolManager throws this during the window between forceSchedulerRestart
+  // clearing the scheduler reference and the old pool's 1500ms drain completing.
+  // It is a transient state — retry after a short delay succeeds once drain finishes.
+  err.message.includes("Worker pool is shutting down") || // Poolifier throws this when pool.execute() is called while the pool is being
+  // destroyed (e.g., during scheduler restart). Treat as transient — the pool will
+  // be ready for new tasks once destruction completes and a new pool is initialized.
+  err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // The leader answered 503 because it is shutting down after losing
+  // leadership. Retrying re-elects and reaches the new leader, so this is
+  // transient in exactly the same sense as a pool drain.
+  err.message.includes("Browser pool leader is draining"));
+}
+function isCloudflareBlockError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("Fetch blocked: Cloudflare") || err.message.includes("Cloudflare challenge"));
+}
+function isTaskTimeoutError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("timed out after") || err.message.includes("Search task timed out") || err.message.includes("Scrape task timed out") || err.message.includes("Health check timed out") || err.message.includes("[BrowserClient] Request to"));
+}
+function isServerDrainingError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && err.message.includes("Browser pool leader is draining");
+}
+function isPoolShutdownError(error) {
+  if (!error || typeof error !== "object") return false;
+  const err = error;
+  return typeof err.message === "string" && (err.message.includes("Worker pool is shutting down") || err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // worker-pool-manager throws this when auto-recovery is still swapping the
+  // pool. Its own text says "please retry", but until it was listed here no
+  // classifier matched it, so the first attempt rethrew instead of retrying —
+  // and auto-recovery fires only after 3 consecutive worker failures, i.e.
+  // exactly when a burst of tasks is in flight and would all die together.
+  err.message.includes("Worker pool is being reset") || // poolifier's wording once destroy() has COMPLETED (vs "destroying pool"
+  // while it is in progress). A queued closure holding a pool captured before
+  // an auto-recovery destroy dispatches into exactly this window; it is the
+  // same transient swap the "being reset" string covers, not a hard fault.
+  err.message.includes("Cannot execute a task on not started pool"));
+}
+var BENIGN_NAVIGATION_FAILURE = /page\.goto: (?:Timeout \d+ *ms exceeded|net::|NS_ERROR_|SEC_ERROR_|MOZILLA_PKIX_ERROR_)/;
+function isBenignScrapeFailure(error) {
+  const msg2 = error instanceof Error ? error.message : String(error);
+  return msg2.startsWith("Fetch blocked:") || // bot-protection interstitial (Cloudflare, DDoS-Guard, …)
+  msg2.startsWith("Fetch returned stub:") || // nav-only / near-empty page
+  msg2.startsWith("Could not extract content from PDF") || // malformed/encrypted/scanned-only PDF
+  msg2.startsWith("PDF too large") || // exceeds the deliberate 100MB policy cap
+  msg2.startsWith("Unsupported download:") || // attachment-served non-PDF (zip/xlsx/…) — nothing to extract
+  BENIGN_NAVIGATION_FAILURE.test(msg2) || // remote TLS/DNS/timeout — the URL could not be loaded
+  msg2.startsWith("HTML response too large") || // fetch-layer 25MB policy cap — same class as the PDF cap
+  msg2.startsWith("Browser HTML too large") || // browser-layer 25MB policy cap
+  /^HTTP \d{3}\b/.test(msg2) || // remote 4xx/5xx status
+  isCloudflareBlockError(error) || isTaskTimeoutError(error) || isPoolShutdownError(error);
+}
+var sessionCircuitBreakers = /* @__PURE__ */ new Map();
+var MAX_SESSION_BREAKERS = 256;
+var DEFAULT_BREAKER_CONFIG = {
+  // Raised from 3→8: parallel search bursts of 20 queries frequently produce
+  // simultaneous timeouts on slow/CF-protected sites. A threshold of 3 caused
+  // the breaker to open mid-burst, blocking all subsequent requests for the round.
+  failureThreshold: 8,
+  // Raised from 15s→45s: enough time for one full search burst + retry window
+  // to complete before the breaker attempts to half-open.
+  resetTimeoutMs: 45e3,
+  name: "BrowserPool",
+  isTransientError: (error) => {
+    if (isPoolShutdownError(error)) return false;
+    if (isServerDrainingError(error)) return false;
+    if (isCloudflareBlockError(error)) return false;
+    if (isTaskTimeoutError(error)) return false;
+    return isTransientSocketError(error);
+  }
+};
+function getBrowserCircuitBreaker(sessionId) {
+  let breaker2 = sessionCircuitBreakers.get(sessionId);
+  if (!breaker2) {
+    if (sessionCircuitBreakers.size >= MAX_SESSION_BREAKERS) {
+      const oldest = sessionCircuitBreakers.keys().next().value;
+      if (oldest !== void 0) sessionCircuitBreakers.delete(oldest);
+    }
+    breaker2 = new CircuitBreaker({ ...DEFAULT_BREAKER_CONFIG, name: `BrowserPool-${sessionId}` });
+    sessionCircuitBreakers.set(sessionId, breaker2);
+  }
+  return breaker2;
+}
+function clearSessionCircuitBreaker(sessionId) {
+  sessionCircuitBreakers.delete(sessionId);
+}
+var browserCircuitBreaker = new CircuitBreaker(DEFAULT_BREAKER_CONFIG);
+
 // src/infrastructure/browser/browser-client.ts
 var CLIENT_PATIENCE_MARGIN_MS = 15e3;
 var CLIENT_TIMEOUT_CAP_MS = 3e5;
@@ -12597,12 +12729,16 @@ var BrowserClient = class {
           errorType = "unknown";
         }
         const error = new Error(errorMsg);
-        logger.error(`[BrowserClient] Request to http://127.0.0.1:${this.port}${path20} failed:`, errorMsg);
-        errorTracker.trackError(error, {
-          component: "browser-manager",
-          operation,
-          errorType
-        });
+        if (isTransientSocketError(error)) {
+          logger.debug(`[BrowserClient] Transient failure talking to http://127.0.0.1:${this.port}${path20} (caller will retry): ${errorMsg}`);
+        } else {
+          logger.error(`[BrowserClient] Request to http://127.0.0.1:${this.port}${path20} failed:`, errorMsg);
+          errorTracker.trackError(error, {
+            component: "browser-manager",
+            operation,
+            errorType
+          });
+        }
         reject(error);
       });
       req.write(JSON.stringify(data));
@@ -12644,89 +12780,6 @@ init_logger();
 init_http_body();
 import * as http5 from "node:http";
 import * as crypto5 from "node:crypto";
-
-// src/infrastructure/browser/browser-error-utils.ts
-init_circuit_breaker();
-function isTransientSocketError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("ECONNREFUSED") || err.message.includes("ECONNRESET") || err.message.includes("socket hang up") || err.message.includes("EPIPE") || err.message.includes("ETIMEDOUT") || err.message.includes("timed out") || err.message.includes("pool busy") || err.message.includes("unreachable") || // WorkerPoolManager throws this during the window between forceSchedulerRestart
-  // clearing the scheduler reference and the old pool's 1500ms drain completing.
-  // It is a transient state — retry after a short delay succeeds once drain finishes.
-  err.message.includes("Worker pool is shutting down") || // Poolifier throws this when pool.execute() is called while the pool is being
-  // destroyed (e.g., during scheduler restart). Treat as transient — the pool will
-  // be ready for new tasks once destruction completes and a new pool is initialized.
-  err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // The leader answered 503 because it is shutting down after losing
-  // leadership. Retrying re-elects and reaches the new leader, so this is
-  // transient in exactly the same sense as a pool drain.
-  err.message.includes("Browser pool leader is draining"));
-}
-function isCloudflareBlockError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("Fetch blocked: Cloudflare") || err.message.includes("Cloudflare challenge"));
-}
-function isTaskTimeoutError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("timed out after") || err.message.includes("Search task timed out") || err.message.includes("Scrape task timed out") || err.message.includes("Health check timed out") || err.message.includes("[BrowserClient] Request to"));
-}
-function isServerDrainingError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && err.message.includes("Browser pool leader is draining");
-}
-function isPoolShutdownError(error) {
-  if (!error || typeof error !== "object") return false;
-  const err = error;
-  return typeof err.message === "string" && (err.message.includes("Worker pool is shutting down") || err.message.includes("Cannot execute a task on destroying pool") || err.message.includes("destroying pool") || // worker-pool-manager throws this when auto-recovery is still swapping the
-  // pool. Its own text says "please retry", but until it was listed here no
-  // classifier matched it, so the first attempt rethrew instead of retrying —
-  // and auto-recovery fires only after 3 consecutive worker failures, i.e.
-  // exactly when a burst of tasks is in flight and would all die together.
-  err.message.includes("Worker pool is being reset") || // poolifier's wording once destroy() has COMPLETED (vs "destroying pool"
-  // while it is in progress). A queued closure holding a pool captured before
-  // an auto-recovery destroy dispatches into exactly this window; it is the
-  // same transient swap the "being reset" string covers, not a hard fault.
-  err.message.includes("Cannot execute a task on not started pool"));
-}
-var sessionCircuitBreakers = /* @__PURE__ */ new Map();
-var MAX_SESSION_BREAKERS = 256;
-var DEFAULT_BREAKER_CONFIG = {
-  // Raised from 3→8: parallel search bursts of 20 queries frequently produce
-  // simultaneous timeouts on slow/CF-protected sites. A threshold of 3 caused
-  // the breaker to open mid-burst, blocking all subsequent requests for the round.
-  failureThreshold: 8,
-  // Raised from 15s→45s: enough time for one full search burst + retry window
-  // to complete before the breaker attempts to half-open.
-  resetTimeoutMs: 45e3,
-  name: "BrowserPool",
-  isTransientError: (error) => {
-    if (isPoolShutdownError(error)) return false;
-    if (isServerDrainingError(error)) return false;
-    if (isCloudflareBlockError(error)) return false;
-    if (isTaskTimeoutError(error)) return false;
-    return isTransientSocketError(error);
-  }
-};
-function getBrowserCircuitBreaker(sessionId) {
-  let breaker2 = sessionCircuitBreakers.get(sessionId);
-  if (!breaker2) {
-    if (sessionCircuitBreakers.size >= MAX_SESSION_BREAKERS) {
-      const oldest = sessionCircuitBreakers.keys().next().value;
-      if (oldest !== void 0) sessionCircuitBreakers.delete(oldest);
-    }
-    breaker2 = new CircuitBreaker({ ...DEFAULT_BREAKER_CONFIG, name: `BrowserPool-${sessionId}` });
-    sessionCircuitBreakers.set(sessionId, breaker2);
-  }
-  return breaker2;
-}
-function clearSessionCircuitBreaker(sessionId) {
-  sessionCircuitBreakers.delete(sessionId);
-}
-var browserCircuitBreaker = new CircuitBreaker(DEFAULT_BREAKER_CONFIG);
-
-// src/infrastructure/browser/browser-server.ts
 var _authSecret = null;
 function getOrCreateAuthSecret() {
   if (!_authSecret) {
@@ -12868,6 +12921,8 @@ var BrowserServer = class {
               logger.warn("[BrowserServer] Request blocked by bot protection:", error instanceof Error ? error.message : String(error));
             } else if (isPoolShutdownError(error)) {
               logger.debug("[BrowserServer] Request abandoned during shutdown:", error instanceof Error ? error.message : String(error));
+            } else if (isBenignScrapeFailure(error)) {
+              logger.debug("[BrowserServer] Expected per-URL scrape outcome:", error instanceof Error ? error.message.split("\n")[0] : String(error));
             } else {
               logger.error("[BrowserServer] Error handling request:", error);
             }
@@ -18060,17 +18115,6 @@ import * as dnsCb from "node:dns";
 import * as net3 from "node:net";
 import crypto9 from "node:crypto";
 import { NodeHtmlMarkdown } from "node-html-markdown";
-function isBenignScrapeFailure(error) {
-  const msg2 = error instanceof Error ? error.message : String(error);
-  return msg2.startsWith("Fetch blocked:") || // bot-protection interstitial (Cloudflare, DDoS-Guard, …)
-  msg2.startsWith("Fetch returned stub:") || // nav-only / near-empty page
-  msg2.startsWith("Could not extract content from PDF") || // malformed/encrypted/scanned-only PDF
-  msg2.startsWith("PDF too large") || // exceeds the deliberate 100MB policy cap
-  msg2.startsWith("HTML response too large") || // fetch-layer 25MB policy cap — same class as the PDF cap
-  msg2.startsWith("Browser HTML too large") || // browser-layer 25MB policy cap
-  /^HTTP \d{3}\b/.test(msg2) || // remote 4xx/5xx status
-  isCloudflareBlockError(error) || isTaskTimeoutError(error) || isPoolShutdownError(error);
-}
 function getRandomUserAgent() {
   try {
     const index = crypto9.randomInt(0, USER_AGENTS.length);
@@ -21971,9 +22015,14 @@ async function doMint(identifiers, opts) {
   const requestKey = opts.requestKey || process.env["PI_RESEARCH_YOUTUBE_POTOKEN_REQUEST_KEY"] || DEFAULT_REQUEST_KEY;
   const fetchImpl = opts.fetchImpl ?? fetch;
   const started = Date.now();
-  const bgMod = await import("bgutils-js");
-  const BG = bgMod.BG ?? bgMod.default;
-  const { buildURL: buildURL2, getHeaders } = bgMod;
+  const [bgGuard, bgWebPo, bgUtils] = await Promise.all([
+    import("bgutils-js/botguard"),
+    import("bgutils-js/webpo"),
+    import("bgutils-js/utils")
+  ]);
+  const { getChallenge, BotGuardClient } = bgGuard;
+  const { WebPoMinter } = bgWebPo;
+  const { buildURL: buildURL2, getHeaders } = bgUtils;
   const { JSDOM } = await import("jsdom");
   const dom = new JSDOM('<!DOCTYPE html><html lang="en"><body></body></html>', {
     url: "https://www.youtube.com/"
@@ -22004,24 +22053,24 @@ async function doMint(identifiers, opts) {
   };
   bridge();
   try {
-    const bgConfig = {
-      fetch: (input, init) => fetchImpl(input, {
+    const challenge = await getChallenge({
+      requestKey,
+      fetchFunction: ((input, init) => fetchImpl(input, {
         ...init,
         signal: init?.signal ?? opts.signal
-      }),
-      globalObj: globalThis,
-      identifier: identifiers[0],
-      requestKey
-    };
-    const challenge = await BG.Challenge.create(bgConfig);
+      }))
+    });
     if (!challenge) throw new Error("BotGuard challenge unavailable");
     const interpreter = challenge.interpreterJavascript?.privateDoNotAccessOrElseSafeScriptWrappedValue;
     if (!interpreter) throw new Error("BotGuard interpreter script missing");
     new Function(interpreter)();
-    const botguard = await BG.BotGuardClient.create({
+    if (!challenge.program || !challenge.globalName) {
+      throw new Error("BotGuard challenge missing program/globalName");
+    }
+    const botguard = await BotGuardClient.create({
       program: challenge.program,
       globalName: challenge.globalName,
-      globalObj: globalThis
+      globalObject: globalThis
     });
     const webPoSignalOutput = [];
     const botguardResponse = await botguard.snapshot({ webPoSignalOutput });
@@ -22039,7 +22088,7 @@ async function doMint(identifiers, opts) {
     if (!integrityToken) {
       throw new Error("Integrity token rejected by attestation server (likely a non-residential IP)");
     }
-    const minter = await BG.WebPoMinter.create({ integrityToken }, webPoSignalOutput);
+    const minter = await WebPoMinter.create({ integrityToken }, webPoSignalOutput);
     const tokens = /* @__PURE__ */ new Map();
     for (const id of identifiers) {
       tokens.set(id, await minter.mintAsWebsafeString(id));
@@ -23846,6 +23895,19 @@ ${userMessage}`);
         const synthesisService = await getService(ServiceNames.RESEARCH_SYNTHESIS_SERVICE, ctx, container);
         synthesisService.storeReport(this.options.researchId, "quick", result);
         result = synthesisService.getReport(this.options.researchId, "quick") ?? result;
+        const { body: withoutPreamble, preamble } = stripReportPreamble(result);
+        if (preamble) {
+          logger.debug(`[QuickOrchestrator] Dropped a ${preamble.length}-char narration preamble from the report: ${preamble.slice(0, 120)}`);
+          metrics.increment("research_report_preamble_stripped_total", 1, { mode: "quick" });
+          result = withoutPreamble;
+        }
+        if (isAnalysisFreeSynthesis(result, 0)) {
+          logger.warn(`[QuickOrchestrator] Report contains no complete sentence (${synthesisProseBody(result).length} char(s) of prose) \u2014 flagging it in the report`);
+          metrics.increment("research_synthesis_analysis_free_total", 1, { mode: "quick" });
+          result = `${result.trim()}
+
+${QUICK_THIN_REPORT_NOTICE}`;
+        }
         result = synthesisService.ensureCitedLinks(this.options.researchId, result);
         const citationSource = result;
         const finalSteeringMessages = getActiveSteeringMessages(this.options.sessionId);
@@ -23956,6 +24018,7 @@ ${result}`);
 init_logger();
 init_metrics();
 init_constants();
+init_text_utils();
 init_service_interfaces();
 init_service_registry();
 init_config();
@@ -24271,6 +24334,13 @@ var DeepResearchOrchestrator = class {
         observer?.onEvaluationDecision?.("synthesize", finalReport, this.currentRound);
       }
       let result = finalReport.content || "";
+      try {
+        const parsed = JSON.parse(result);
+        if (parsed && typeof parsed === "object" && typeof parsed["content"] === "string") {
+          result = parsed["content"];
+        }
+      } catch {
+      }
       if (!result.trim()) {
         if (hasReports) {
           logger.warn(`[DeepOrchestrator] LLM returned empty synthesis for ${researchId}, building fallback from ${synthesisCheckService.getReportCount(researchId)} reports`);
@@ -24278,13 +24348,15 @@ var DeepResearchOrchestrator = class {
         } else {
           throw new Error("Research produced no report \u2014 no source material was collected.");
         }
-      }
-      try {
-        const parsed = JSON.parse(result);
-        if (parsed && typeof parsed === "object" && typeof parsed["content"] === "string") {
-          result = parsed["content"];
-        }
-      } catch {
+      } else if (hasReports && isAnalysisFreeSynthesis(result, MIN_SYNTHESIS_PROSE_CHARS)) {
+        const proseChars = synthesisProseBody(result).length;
+        logger.warn(`[DeepOrchestrator] LLM synthesis for ${researchId} carried no analysis (${proseChars} char(s) of prose, min ${MIN_SYNTHESIS_PROSE_CHARS}) \u2014 appending ${synthesisCheckService.getReportCount(researchId)} researcher report(s) below it`);
+        metrics.increment("research_synthesis_analysis_free_total", 1, { mode: "deep" });
+        result = [
+          result.trim(),
+          SYNTHESIS_FALLBACK_NOTICE,
+          synthesisCheckService.buildFallbackSynthesis(researchId, this.currentRound)
+        ].join("\n\n");
       }
       const synthesisService = await getService(ServiceNames.RESEARCH_SYNTHESIS_SERVICE, ctx, container);
       result = synthesisService.ensureCitedLinks(researchId, result);
@@ -25152,7 +25224,7 @@ var ResearchSynthesisService = class _ResearchSynthesisService {
     if (reportCount === 0) {
       synthesis += "No researcher reports were generated before the process stopped.";
     } else {
-      synthesis += `This is an automated synthesis of ${reportCount} individual researcher report(s) gathered before the process was interrupted.
+      synthesis += `This is an automated compilation of ${reportCount} individual researcher report(s). The synthesis step did not produce a usable summary for this run, so the findings are presented as collected.
 
 `;
       synthesis += Array.from(reports.entries()).map(([id, report]) => `Researcher ${id}

package/dist/prompts/researcher.md

@@ -62,6 +62,8 @@ Write your report immediately after scraping is complete, or as soon as a tool r
 ## Report Format
 
 {{digest_section}}
+Emit the report and nothing else. No preamble, and no narration of what you are about to do — not "I have gathered extensive material…", not "Let me now synthesize my findings…". Begin at the report's own first line.
+
 Use [N] inline citations throughout. Write in plain prose — no markdown headings, no bullet points, no bold or italic text. Use clear paragraphs separated by blank lines. The full CITED LINKS list goes at the very end.
 
 [Topic Title]