@lincoln504/pi-research 1.3.4 → 1.3.6

package/README.md

@@ -24,38 +24,31 @@
 
 <a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
 
-**Free, unlimited web search &amp; scraping — no monthly quota, no subscription.**
-You pay only for the LLM tokens you use, and it builds your own private, searchable
-knowledge store on your machine.
+**Free, unlimited web search &amp; scraping — no monthly quota, no subscription.** You pay only for the LLM tokens you use, and it can build a searchable knowledge store locally.
 
 ### How it works
 
-A research run is a loop of agent teams: a coordinator plans the work and runs an
-opening search, parallel researcher agents scrape and read sources through a fixed,
-sandboxed toolkit (no shell access, and no ability to write or modify files), and an evaluator decides whether
-to run another round or synthesize — returning one cited Markdown report while its
-sources are saved to the knowledge store in the background.
+A research run loops through agent teams: a coordinator plans and opens with a search, researcher agents scrape and read in parallel, and an evaluator decides whether to go another round or synthesize — ending in one cited Markdown report, with findings optionally saved to the knowledge store.
 
-Three depth levels — normal, deep, ultra — set the team size and number of rounds;
-ask in natural language and the tool picks the right one:
+Three depth levels — normal, deep, ultra — set the team size and number of rounds; ask in natural language and the tool picks the right one:
 
 ![Prompt-driven multi-round research in the pi TUI](https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/02-prompt-research.gif)
 
 ### Use cases
 
-- Plug-and-play (no API key needed) research tool for your [pi](https://github.com/earendil-works/pi).
-- Research from Claude Code, Codex, or another coding agent, with a cheaper lightweight or local model driving the research so it doesn't spend your main agent's budget.
-- Populating a dataset or building an index of knowledge sources from the web.
+- Researching for your [Pi](https://github.com/earendil-works/pi) Coding Agent — plug-and-play, no API key needed.
+- Researching from Claude Code, Codex, or another coding agent, with a cheaper lightweight or local model driving the research so it doesn't spend your main agent's budget.
+- Populating a dataset or index of web sources.
 - Holding research in the knowledge store with a configurable scope — project-specific or globally user-scoped, set per directory from the `/research-config` TUI.
 - Using the pi-research agent skill as OpenClaw's web access.
-- Building agent systems that identify and examine web sources.
+- Building agent systems that find and read web content.
 
 ### Why pi-research
 
-- **Free, unlimited search &amp; scrape — no quota, no monthly fee.** Most AI-search providers cap free searches per month and then charge; results run on *their* index on *their* servers. pi-research's search and scraping layer (DuckDuckGo via a stealth browser) is free and unlimited — you pay only for the LLM tokens you use, never per search or per month.
-- **You own your index.** Every finding is written into a local [LanceDB](https://lancedb.com) knowledge store on *your* machine — a persistent, searchable index you control and keep, which also seeds future runs so repeat questions get faster, cheaper answers. No borrowed, server-side index you can't keep or query. Findings can also be kept **globally**, scoped to a **single project**, or **not kept at all** — set per directory, changed whenever you like. The optionality is the point: the store is yours to build, partition, or switch off.
-- **Cited reports, not raw dumps.** It returns one synthesized, sourced Markdown report to the chat instead of dumping raw web content into the conversation — context-efficient and easy to verify.
-- **Read-only by design.** Web access runs inside a specialized, limited research agent that cannot run shell commands or write, edit, or delete anything. It can read and search files, so treat it as you would any read-only agent with access to your machine.
+- **Free, unlimited search &amp; scrape — no quota, no monthly fee.** Most AI-search providers cap free searches per month and then charge; results run on *their* index on *their* servers. pi-research's search and scraping layer (DuckDuckGo via a stealth browser) is free and unlimited — you pay only for the LLM tokens you use.
+- **You own your index.** Every finding is written into a local [LanceDB](https://lancedb.com) knowledge store on *your* machine — a persistent, searchable index you control and keep, which also seeds future runs so repeat questions get faster, cheaper answers. Findings can also be kept globally, scoped to a single project, or not kept at all — set per directory, changed whenever you like.
+- **Cited reports, not raw dumps.** It returns one synthesized, sourced Markdown report instead of pasting raw web content into the conversation — context-efficient and easy to verify.
+- **Read-only by design.** Web access runs inside a limited research agent that cannot run shell commands or write, edit, or delete anything. It can read and search files, so treat it as you would any read-only agent with access to your machine.
 - **Search a little or a lot** — three depth levels in the pi tool (four via the SDK and standalone CLI, which add the quick depth-0 path). Levels 1 and 2 are recommended for everyday workflow use; level 3 is for larger-scale investigations.
 
 ### Requirements / limitations
@@ -63,9 +56,9 @@ ask in natural language and the tool picks the right one:
 - Node.js >= 22.19.0
 - An LLM with a 100k+ context window (bring your own key or use a local model)
 - Internet access
-- A residential IP address — search, scraping, and YouTube transcripts all rely on a residential connection. A datacenter/VPS/cloud IP gets bot-blocked by the providers these features depend on.
+- A residential IP address — search, scraping, and YouTube transcripts all rely on a residential connection. A datacenter/VPS/cloud IP gets bot-blocked by these providers.
 - The pi runtime the engine builds on — `@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`, and `@earendil-works/pi-tui`. The pi extension uses the host's copies; the standalone CLI / agent skill install them as dependencies.
-- Cloudflare and similar anti-automation systems block scraping on some sites, so runs will identify sources they cannot reach. pi-research compensates with volume: the search tool rapidly pulls a large set of results off free DuckDuckGo, giving the model a wide selection of reachable content to cite.
+- Cloudflare and similar anti-automation systems block scraping on some sites, so a run will flag pages it can't reach. pi-research compensates with volume: the search tool pulls a large set of results off free DuckDuckGo, giving the model a wide selection of reachable content to cite.
 
 ### Install
 
@@ -73,21 +66,23 @@ ask in natural language and the tool picks the right one:
 pi install npm:@lincoln504/pi-research
 ```
 
-See [Agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md) for standalone (no pi extension) installation.
+See [Agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md) to install standalone (no pi extension).
 
-The pi extension works out of the box — it runs on your pi session's model and pi's own configuration. Standalone use (agent skill / CLI) requires a configured model: set `PI_RESEARCH_MODEL` (e.g. `openai/gpt-4o`) as an environment variable or in `~/.pi/research/config.env` (the CLI also accepts a per-run `--model`), plus `PI_RESEARCH_API_KEY` if you don't use pi; the SDK takes a `model` option. See [Configuration](docs/CONFIGURATION.md).
+The pi extension works out of the box — it runs on your pi session's model and pi's own configuration. To use it standalone (agent skill / CLI), configure a model: set `PI_RESEARCH_MODEL` (e.g. `openai/gpt-4o`) as an environment variable or in `~/.pi/research/config.env` (the CLI also accepts a per-run `--model`), plus `PI_RESEARCH_API_KEY` if you don't use pi; the SDK takes a `model` option. See [Configuration](docs/CONFIGURATION.md).
 
 The first install pulls the stealth browser engine, which takes a few minutes.
 
 ### Stability (v1.x.x)
 
-v0.1.13 (April 2026) was the last release before a rebuild against current pi APIs
-and the stealth-browser stack. v1.0.0 is the first stable release since. Channels:
+v0.1.13 (April 2026) was the last release before a rebuild against current pi APIs and the stealth-browser stack. v1.0.0 is the first stable release since. Channels:
 
 - npm (`npm:@lincoln504/pi-research`) is the stable channel, kept current with breaking pi changes.
 - A git install is the development channel: latest commits, first to break.
 
 ### License
 
-MIT. Bundled third-party licenses are listed in
-[docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).
+MIT. Bundled third-party licenses are listed in [docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).
+
+### Package identity
+
+This project is always `@lincoln504/pi-research` on npm. The `pi-research` (unscoped) package is unrelated and deprecated.

package/dist/cli.mjs

@@ -215,7 +215,15 @@ function formatArg(arg) {
   return String(arg);
 }
 function redactSecrets(message) {
-  let out = message;
+  const rawLength = message.length;
+  let scanEnd = REDACT_SCAN_LENGTH;
+  if (rawLength > scanEnd) {
+    const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
+    while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
+      scanEnd += 1;
+    }
+  }
+  let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
   out = out.replace(ANSI_PATTERN, "");
   out = out.replace(URL_CREDENTIALS_PATTERN, "$1[REDACTED]@");
   out = out.replace(JWT_PATTERN, "[REDACTED]");
@@ -232,14 +240,14 @@ function redactSecrets(message) {
     if (lastKeptUnit >= 55296 && lastKeptUnit <= 56319) keep -= 1;
   }
   if (out.length > MAX_LOG_MESSAGE_LENGTH) {
-    out = `${out.slice(0, keep)}\u2026[truncated ${out.length - keep} chars]`;
+    out = `${out.slice(0, keep)}\u2026[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
   }
   return out;
 }
 function stripTerminalEscapes(message) {
   return message.replace(TERMINAL_ESCAPE_PATTERN, "").replace(CONTROL_CHARS_PATTERN, " ");
 }
-var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, TERMINAL_ESCAPE_PATTERN;
+var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TOKEN_CHAR_PATTERN, REDACT_SCAN_EXTENSION_MAX, TERMINAL_ESCAPE_PATTERN;
 var init_log_utils = __esm({
   "src/utils/log-utils.ts"() {
     "use strict";
@@ -256,6 +264,9 @@ var init_log_utils = __esm({
     PROVIDER_TOKEN_PATTERN = /\b(AIza[0-9A-Za-z_-]{35}|ya29\.[0-9A-Za-z_-]{20,}|[sp]k_(?:live|test)_[0-9A-Za-z]{16,})\b/g;
     LONG_HEX_SECRET_PATTERN = /\b[0-9a-fA-F]{32,}\b/g;
     CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
+    REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
+    TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
+    REDACT_SCAN_EXTENSION_MAX = 2048;
     TERMINAL_ESCAPE_PATTERN = new RegExp(
       "\\x1b\\[[0-?]*[ -/]*[@-~]?|\\x1b\\][^\\x07\\x1b]*(?:\\x07|\\x1b\\\\)?|\\x1b[PX^_][^\\x1b]*(?:\\x1b\\\\)?|\\x1b.?",
       "g"
@@ -266,10 +277,26 @@ var init_log_utils = __esm({
 // src/utils/log-rotation.ts
 import * as fs from "node:fs";
 import * as path2 from "node:path";
-var LogRotation;
+function touchOwnerOnly(filePath, mode) {
+  let fd;
+  try {
+    fd = fs.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
+    fs.fchmodSync(fd, mode);
+  } catch {
+  } finally {
+    if (fd !== void 0) {
+      try {
+        fs.closeSync(fd);
+      } catch {
+      }
+    }
+  }
+}
+var APPEND_CREATE_NOFOLLOW_FLAGS, LogRotation;
 var init_log_rotation = __esm({
   "src/utils/log-rotation.ts"() {
     "use strict";
+    APPEND_CREATE_NOFOLLOW_FLAGS = fs.constants.O_WRONLY | fs.constants.O_APPEND | fs.constants.O_CREAT | (typeof fs.constants.O_NOFOLLOW === "number" ? fs.constants.O_NOFOLLOW : 0);
     LogRotation = class {
       MAX_LOG_SIZE = 10 * 1024 * 1024;
       // 10MB max file size
@@ -287,14 +314,7 @@ var init_log_rotation = __esm({
         try {
           if (fs.existsSync(logFile)) {
             fs.unlinkSync(logFile);
-            try {
-              fs.closeSync(fs.openSync(logFile, "a", 384));
-            } catch {
-            }
-            try {
-              fs.chmodSync(logFile, 384);
-            } catch {
-            }
+            touchOwnerOnly(logFile, 384);
           }
           const files = fs.readdirSync(logDir);
           const baseName = path2.basename(logFile);
@@ -330,14 +350,7 @@ var init_log_rotation = __esm({
             this.logger.log("[Logger] Log rotation skipped (rename failed; file likely in use):", renameErr);
             return;
           }
-          try {
-            fs.closeSync(fs.openSync(logFile, "a", 384));
-          } catch {
-          }
-          try {
-            fs.chmodSync(logFile, 384);
-          } catch {
-          }
+          touchOwnerOnly(logFile, 384);
           try {
             const files = fs.readdirSync(logDir);
             const logFiles = files.filter((f) => f.startsWith(path2.basename(logFile)) && f !== path2.basename(logFile)).sort();
@@ -9911,19 +9924,19 @@ function reconcileSkillInstalls(opts = {}) {
     if (e.type !== "symlink") {
       const stale2 = currentVersion !== null && e.version !== currentVersion;
       if (stale2 && source !== null && fs16.existsSync(e.path) && isOwnedCopy(e.path)) {
-        const dir = path17.dirname(e.path);
-        const base = path17.basename(e.path);
+        const dir2 = path17.dirname(e.path);
+        const base2 = path17.basename(e.path);
         try {
-          for (const name of fs16.readdirSync(dir)) {
-            if (name.startsWith(`.${base}.new-`) || name.startsWith(`.${base}.old-`)) {
-              fs16.rmSync(path17.join(dir, name), { recursive: true, force: true });
+          for (const name of fs16.readdirSync(dir2)) {
+            if (name.startsWith(`.${base2}.new-`) || name.startsWith(`.${base2}.old-`)) {
+              fs16.rmSync(path17.join(dir2, name), { recursive: true, force: true });
             }
           }
         } catch {
         }
         const suffix = crypto10.randomBytes(4).toString("hex");
-        const fresh = path17.join(dir, `.${base}.new-${suffix}`);
-        const retired = path17.join(dir, `.${base}.old-${suffix}`);
+        const fresh = path17.join(dir2, `.${base2}.new-${suffix}`);
+        const retired = path17.join(dir2, `.${base2}.old-${suffix}`);
         try {
           copyDir(source, fresh);
         } catch {
@@ -9969,6 +9982,16 @@ function reconcileSkillInstalls(opts = {}) {
       }
       continue;
     }
+    const dir = path17.dirname(e.path);
+    const base = path17.basename(e.path);
+    try {
+      for (const name of fs16.readdirSync(dir)) {
+        if (name.startsWith(`.${base}.new-`)) {
+          fs16.rmSync(path17.join(dir, name), { recursive: true, force: true });
+        }
+      }
+    } catch {
+    }
     if (!isSymlinkPresent(e.path)) {
       result.pruned.push(e.path);
       continue;
@@ -9988,11 +10011,29 @@ function reconcileSkillInstalls(opts = {}) {
       if (source !== null) {
         try {
           fs16.unlinkSync(e.path);
+        } catch {
+          keep.push(e);
+          continue;
+        }
+        try {
           fs16.symlinkSync(source, e.path, process.platform === "win32" ? "junction" : "dir");
           keep.push({ ...e, type: "symlink", source });
           result.repointed.push(e.path);
         } catch {
-          keep.push(e);
+          const suffix = crypto10.randomBytes(4).toString("hex");
+          const fresh = path17.join(dir, `.${base}.new-${suffix}`);
+          try {
+            copyDir(source, fresh);
+            fs16.renameSync(fresh, e.path);
+            keep.push({ ...e, type: "copy", source });
+            result.repointed.push(e.path);
+          } catch {
+            try {
+              fs16.rmSync(fresh, { recursive: true, force: true });
+            } catch {
+            }
+            result.pruned.push(e.path);
+          }
         }
       } else if (!targetLive) {
         try {
@@ -11439,6 +11480,7 @@ function isFullMockMode() {
 init_config();
 init_logger();
 init_error_tracker();
+init_log_utils();
 init_http_body();
 import * as http2 from "node:http";
 
@@ -11574,7 +11616,7 @@ var BrowserClient = class {
               resolve9(parsed);
             }
           } catch (_e) {
-            const raw = collected.toString();
+            const raw = redactSecrets(collected.toString());
             const preview = raw.length > 200 ? raw.slice(0, 200) + "..." : raw;
             const error = new Error(`Failed to parse response (status ${res.statusCode}): ${preview}`);
             errorTracker.trackError(error, {
@@ -17545,6 +17587,7 @@ async function scrape(urls, maxConcurrency = 5, signal, config, sessionId, onUrl
 // src/tools/scrape.ts
 init_constants();
 init_text_utils();
+init_log_utils();
 init_config();
 init_service_registry();
 init_service_interfaces();
@@ -17716,18 +17759,19 @@ ${evictedDuplicates.map((u) => `- ${u}`).join("\n")}
       let freshResults;
       const concurrency = p.maxConcurrency || defaultConcurrency;
       try {
-        const results = await scrape(urlsToFetch, concurrency, signal, options.config, getGlobalState().researchId, (result) => {
+        const results = await scrape(urlsToFetch, concurrency, signal, config, getGlobalState().researchId, (result) => {
           options.onUrlScrapeResult?.(result.url, result.success);
         }, container);
         freshResults = Array.isArray(results) ? results : [];
       } catch (error) {
-        logger.error(`[scrape tool] Scrape failed: ${error instanceof Error ? error.message : String(error)}`);
+        const message = error instanceof Error ? error.message : String(error);
+        logger.error(`[scrape tool] Scrape failed: ${message}`);
         metrics.increment("tool_scrape_calls_total", 1, { status: "error" });
         return {
           content: [{ type: "text", text: `# Scrape Failed
 
-${error instanceof Error ? error.message : String(error)}` }],
-          details: { error: String(error) }
+${redactSecrets(message)}` }],
+          details: { error: redactSecrets(String(error)) }
         };
       }
       const successfulFresh = freshResults.filter((r) => r.success);
@@ -18974,7 +19018,7 @@ function mapOsvItemToVulnerability(item) {
       for (const cwe of item.database_specific.cwe) {
         if (typeof cwe === "string") {
           cwes.push(cwe);
-        } else if (typeof cwe === "object" && typeof cwe.id === "string") {
+        } else if (typeof cwe === "object" && cwe !== null && typeof cwe.id === "string") {
           cwes.push(cwe.id);
         }
       }
@@ -19355,6 +19399,10 @@ async function searchSecurityDatabases(params, signal) {
 // src/tools/security.ts
 init_constants();
 init_metrics();
+var MAX_DESCRIPTION_CHARS = 300;
+function truncateDescription(description) {
+  return description.length > MAX_DESCRIPTION_CHARS ? `${description.substring(0, MAX_DESCRIPTION_CHARS)}...` : description;
+}
 function createSecuritySearchTool(options) {
   const SecuritySearchParamsSchema = Type6.Object({
     databases: Type6.Optional(Type6.Array(Type6.String({
@@ -19557,7 +19605,7 @@ Unable to search security databases. This may be a temporary issue - try again l
               }
             }
             const description = vuln.description;
-            markdown += `- **Description:** ${description.length > 300 ? `${description.substring(0, 300)}...` : description}
+            markdown += `- **Description:** ${truncateDescription(description)}
 `;
             if (vuln.knownExploited === true) {
               markdown += "- **[Actively Exploited]**\n";
@@ -19609,7 +19657,7 @@ Unable to search security databases. This may be a temporary issue - try again l
 `;
             }
             const description = vuln.description;
-            markdown += `- **Description:** ${description.length > 300 ? `${description.substring(0, 300)}...` : description}
+            markdown += `- **Description:** ${truncateDescription(description)}
 `;
             if (vuln.dueDate !== void 0) {
               markdown += `- **Due Date:** ${vuln.dueDate}
@@ -19657,7 +19705,7 @@ Unable to search security databases. This may be a temporary issue - try again l
             markdown += `- **Summary:** ${adv.summary}
 `;
             const description = adv.description ?? "";
-            markdown += `- **Description:** ${description.length > 300 ? `${description.substring(0, 300)}...` : description}
+            markdown += `- **Description:** ${truncateDescription(description)}
 `;
             if (adv.published) {
               markdown += `- **Published:** ${adv.published}
@@ -19699,7 +19747,7 @@ Unable to search security databases. This may be a temporary issue - try again l
             markdown += `- **Severity:** ${vuln.severity}
 `;
             const description = vuln.description;
-            markdown += `- **Description:** ${description.length > 300 ? `${description.substring(0, 300)}...` : description}
+            markdown += `- **Description:** ${truncateDescription(description)}
 `;
             if (vuln.affectedProducts && vuln.affectedProducts.length > 0) {
               markdown += `- **Affected:** ${vuln.affectedProducts.join(", ")}
@@ -20628,11 +20676,13 @@ function watchUrl(videoId) {
 init_retry_utils();
 init_circuit_breaker();
 init_logger();
+init_log_utils();
 init_metrics();
 init_http_body();
 
 // src/youtube/potoken.ts
 init_logger();
+init_safe_unref();
 init_metrics();
 var DEFAULT_REQUEST_KEY = "O43z0dpjhgX20SCx4KAo";
 var BRIDGED_GLOBALS = ["window", "document", "location", "navigator", "origin"];
@@ -20645,9 +20695,27 @@ function describeError(error) {
   return String(error);
 }
 var mintChain = Promise.resolve();
+var wedged = false;
+var MINT_HARD_RELEASE_MS = 5 * 60 * 1e3;
 async function mintPoTokens(identifiers, opts = {}) {
+  if (wedged) {
+    throw new Error("PoToken minter is wedged by a prior mint that never settled; refusing to start a new mint until it resolves.");
+  }
   const run = mintChain.then(() => doMint(identifiers, opts), () => doMint(identifiers, opts));
-  mintChain = run.then(() => void 0, () => void 0);
+  const settled = run.then(() => void 0, () => void 0);
+  settled.then(() => {
+    wedged = false;
+  });
+  let hardTimer;
+  const hardRelease = new Promise((resolve9) => {
+    hardTimer = setTimeout(() => {
+      logger.warn("[youtube] potoken mint mutex force-released \u2014 doMint() did not settle within the hard timeout");
+      wedged = true;
+      resolve9();
+    }, MINT_HARD_RELEASE_MS);
+    safeUnref(hardTimer);
+  });
+  mintChain = Promise.race([settled, hardRelease]).finally(() => clearTimeout(hardTimer));
   return run;
 }
 async function doMint(identifiers, opts) {
@@ -20929,7 +20997,7 @@ function parseJson3(body) {
   try {
     parsed = JSON.parse(body);
   } catch {
-    logger.debug(`[youtube] timedtext body was not valid json3 (first 120 chars): ${body.slice(0, 120)}`);
+    logger.debug(`[youtube] timedtext body was not valid json3 (first 120 chars): ${redactSecrets(body).slice(0, 120)}`);
     return "";
   }
   const events = parsed.events;

package/dist/thread-worker.mjs

@@ -45,7 +45,15 @@ function formatArg(arg) {
   return String(arg);
 }
 function redactSecrets(message) {
-  let out = message;
+  const rawLength = message.length;
+  let scanEnd = REDACT_SCAN_LENGTH;
+  if (rawLength > scanEnd) {
+    const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
+    while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
+      scanEnd += 1;
+    }
+  }
+  let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
   out = out.replace(ANSI_PATTERN, "");
   out = out.replace(URL_CREDENTIALS_PATTERN, "$1[REDACTED]@");
   out = out.replace(JWT_PATTERN, "[REDACTED]");
@@ -62,14 +70,14 @@ function redactSecrets(message) {
     if (lastKeptUnit >= 55296 && lastKeptUnit <= 56319) keep -= 1;
   }
   if (out.length > MAX_LOG_MESSAGE_LENGTH) {
-    out = `${out.slice(0, keep)}\u2026[truncated ${out.length - keep} chars]`;
+    out = `${out.slice(0, keep)}\u2026[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
   }
   return out;
 }
 function stripTerminalEscapes(message) {
   return message.replace(TERMINAL_ESCAPE_PATTERN, "").replace(CONTROL_CHARS_PATTERN, " ");
 }
-var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, TERMINAL_ESCAPE_PATTERN;
+var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TOKEN_CHAR_PATTERN, REDACT_SCAN_EXTENSION_MAX, TERMINAL_ESCAPE_PATTERN;
 var init_log_utils = __esm({
   "src/utils/log-utils.ts"() {
     "use strict";
@@ -86,6 +94,9 @@ var init_log_utils = __esm({
     PROVIDER_TOKEN_PATTERN = /\b(AIza[0-9A-Za-z_-]{35}|ya29\.[0-9A-Za-z_-]{20,}|[sp]k_(?:live|test)_[0-9A-Za-z]{16,})\b/g;
     LONG_HEX_SECRET_PATTERN = /\b[0-9a-fA-F]{32,}\b/g;
     CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
+    REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
+    TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
+    REDACT_SCAN_EXTENSION_MAX = 2048;
     TERMINAL_ESCAPE_PATTERN = new RegExp(
       "\\x1b\\[[0-?]*[ -/]*[@-~]?|\\x1b\\][^\\x07\\x1b]*(?:\\x07|\\x1b\\\\)?|\\x1b[PX^_][^\\x1b]*(?:\\x1b\\\\)?|\\x1b.?",
       "g"
@@ -357,10 +368,26 @@ var init_error_tracker = __esm({
 // src/utils/log-rotation.ts
 import * as fs2 from "node:fs";
 import * as path2 from "node:path";
-var LogRotation;
+function touchOwnerOnly(filePath, mode) {
+  let fd;
+  try {
+    fd = fs2.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
+    fs2.fchmodSync(fd, mode);
+  } catch {
+  } finally {
+    if (fd !== void 0) {
+      try {
+        fs2.closeSync(fd);
+      } catch {
+      }
+    }
+  }
+}
+var APPEND_CREATE_NOFOLLOW_FLAGS, LogRotation;
 var init_log_rotation = __esm({
   "src/utils/log-rotation.ts"() {
     "use strict";
+    APPEND_CREATE_NOFOLLOW_FLAGS = fs2.constants.O_WRONLY | fs2.constants.O_APPEND | fs2.constants.O_CREAT | (typeof fs2.constants.O_NOFOLLOW === "number" ? fs2.constants.O_NOFOLLOW : 0);
     LogRotation = class {
       MAX_LOG_SIZE = 10 * 1024 * 1024;
       // 10MB max file size
@@ -378,14 +405,7 @@ var init_log_rotation = __esm({
         try {
           if (fs2.existsSync(logFile)) {
             fs2.unlinkSync(logFile);
-            try {
-              fs2.closeSync(fs2.openSync(logFile, "a", 384));
-            } catch {
-            }
-            try {
-              fs2.chmodSync(logFile, 384);
-            } catch {
-            }
+            touchOwnerOnly(logFile, 384);
           }
           const files = fs2.readdirSync(logDir);
           const baseName = path2.basename(logFile);
@@ -421,14 +441,7 @@ var init_log_rotation = __esm({
             this.logger.log("[Logger] Log rotation skipped (rename failed; file likely in use):", renameErr);
             return;
           }
-          try {
-            fs2.closeSync(fs2.openSync(logFile, "a", 384));
-          } catch {
-          }
-          try {
-            fs2.chmodSync(logFile, 384);
-          } catch {
-          }
+          touchOwnerOnly(logFile, 384);
           try {
             const files = fs2.readdirSync(logDir);
             const logFiles = files.filter((f) => f.startsWith(path2.basename(logFile)) && f !== path2.basename(logFile)).sort();
@@ -1917,10 +1930,21 @@ import { platform as platform2 } from "node:os";
 
 // src/infrastructure/browser/thread-worker-messaging.ts
 init_log_utils();
-init_scraper_types();
 import { appendFileSync as appendFileSync2 } from "node:fs";
+
+// src/utils/safe-unref.ts
+function safeUnref(timer) {
+  if (timer && typeof timer.unref === "function") {
+    timer.unref();
+  }
+}
+
+// src/infrastructure/browser/thread-worker-messaging.ts
+init_scraper_types();
 var workerId2 = "";
 var pageCreationLock = Promise.resolve();
+var PAGE_CREATION_HARD_RELEASE_MS = 5 * 60 * 1e3;
+var wedgedContext = null;
 async function createPageSafe(context2) {
   let release;
   const nextLock = new Promise((resolve) => {
@@ -1929,8 +1953,34 @@ async function createPageSafe(context2) {
   const currentLock = pageCreationLock;
   pageCreationLock = currentLock.then(() => nextLock);
   await currentLock;
+  if (context2 === wedgedContext) {
+    release();
+    throw new Error("Browser context is wedged from a prior page-creation call that never settled; a new browser context is required.");
+  }
   const pagePromise = context2.newPage();
-  pagePromise.then(release, release);
+  let hardReleased = false;
+  const releaseOnce = () => {
+    if (hardReleased) return;
+    hardReleased = true;
+    clearTimeout(hardReleaseTimer);
+    release();
+  };
+  const hardReleaseTimer = setTimeout(() => {
+    logToDebugFile2("WARN", "[ThreadWorker] page-creation mutex force-released \u2014 newPage() did not settle within the hard timeout");
+    wedgedContext = context2;
+    releaseOnce();
+  }, PAGE_CREATION_HARD_RELEASE_MS);
+  safeUnref(hardReleaseTimer);
+  pagePromise.then(
+    () => {
+      if (wedgedContext === context2) wedgedContext = null;
+      releaseOnce();
+    },
+    () => {
+      if (wedgedContext === context2) wedgedContext = null;
+      releaseOnce();
+    }
+  );
   pagePromise.catch((err) => logToDebugFile2("DEBUG", `[ThreadWorker] Background page creation rejection: ${err.message}`));
   let timeoutId;
   try {
@@ -1957,6 +2007,15 @@ async function createPageSafe(context2) {
 function setWorkerId2(id) {
   workerId2 = id;
 }
+function parseTimeoutMs(envVar, def) {
+  if (envVar === void 0 || envVar.trim() === "") return def;
+  const n = Number(envVar.trim());
+  if (!Number.isFinite(n) || n < 0) {
+    logToDebugFile2("WARN", `[ThreadWorker] Invalid timeout value "${envVar}", using default: ${def}ms`);
+    return def;
+  }
+  return n;
+}
 function logToDebugFile2(level, ...args) {
   const logFile = process.env["PI_RESEARCH_LOG_FILE"];
   if (!logFile) return;
@@ -2066,7 +2125,7 @@ async function extractSearchResults(page) {
 }
 async function executeSearchTask(_context, query) {
   const page = await createPageSafe(_context);
-  const SEARCH_TIMEOUT = parseInt(process.env["PI_RESEARCH_SEARCH_TIMEOUT_MS"] || "45000", 10);
+  const SEARCH_TIMEOUT = parseTimeoutMs(process.env["PI_RESEARCH_SEARCH_TIMEOUT_MS"], 45e3);
   page.setDefaultTimeout(SEARCH_TIMEOUT);
   page.setDefaultNavigationTimeout(SEARCH_TIMEOUT);
   try {
@@ -2135,7 +2194,7 @@ function seedSubresourceVerdict(validatedUrl) {
 }
 async function executeScrapeTask(_context, url, signal) {
   const page = await createPageSafe(_context);
-  const SCRAPE_TIMEOUT = parseInt(process.env["PI_RESEARCH_SCRAPE_TIMEOUT_MS"] || "15000", 10);
+  const SCRAPE_TIMEOUT = parseTimeoutMs(process.env["PI_RESEARCH_SCRAPE_TIMEOUT_MS"], 15e3);
   page.setDefaultTimeout(SCRAPE_TIMEOUT);
   page.setDefaultNavigationTimeout(SCRAPE_TIMEOUT);
   const onAbort = () => {
@@ -2301,7 +2360,7 @@ async function executeHealthCheckAttempt(_context, navTimeoutMs, url = HEALTH_PR
   }
 }
 async function executeHealthCheck(_context, initMs) {
-  const configuredMs = parseInt(process.env["PI_RESEARCH_HEALTH_CHECK_TIMEOUT_MS"] || "0", 10);
+  const configuredMs = parseTimeoutMs(process.env["PI_RESEARCH_HEALTH_CHECK_TIMEOUT_MS"], 0);
   const HEALTH_TIMEOUT = configuredMs > 0 ? Math.max(1e4, configuredMs) : 1e4;
   if (initMs > 3e3) {
     logToDebugFile2("WARN", `[Worker-${workerId2}] Browser init was slow: ${initMs}ms \u2014 leaving less headroom for DDG navigation.`);
@@ -2594,6 +2653,35 @@ async function initBrowser() {
               height: 1080,
               colorDepth: 24,
               pixelRatio: 1
+            },
+            // Mute all audio output at the engine level. The general scrape
+            // path drives a REAL Firefox (Camoufox), and a researcher can be
+            // pointed at a youtube.com watch page through the ordinary
+            // `scrape` tool (as opposed to the dedicated `youtube_transcript`
+            // tool, which never touches the browser) — YouTube watch pages
+            // autoplay video with sound, so without an explicit mute the
+            // process plays audible audio through the host's sound device.
+            //
+            // media.volume_scale forces the browser's effective output
+            // volume to 0 regardless of per-tab mute/autoplay state or any
+            // page script re-enabling audio — the most reliable, engine-level
+            // mute available for headless Firefox/Camoufox (the same pref
+            // Selenium/Playwright Firefox test grids use to silence CI runs).
+            // media.autoplay.default=5 additionally blocks autoplay outright
+            // (belt-and-suspenders; some sites route around the volume scale
+            // via Web Audio gain nodes). Neither pref affects rendering — the
+            // page still loads and paints normally, only the audio output
+            // device is silenced.
+            //
+            // Set on every platform/headless-mode combination, not just
+            // Linux: this was only *observed* on Linux (true headless and
+            // Xvfb-backed 'virtual' mode both retain a real PulseAudio/
+            // PipeWire/ALSA sink for the Firefox process to play through),
+            // but headless Firefox on macOS/Windows is not guaranteed to be
+            // silent either, so the mute is applied unconditionally.
+            firefox_user_prefs: {
+              "media.volume_scale": "0.0",
+              "media.autoplay.default": 5
             }
           });
           launchPromise.catch((err) => logToDebugFile3("DEBUG", `[ThreadWorker] Background browser launch rejection: ${err.message}`));