@lincoln504/pi-research 1.3.5 → 1.3.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,9 +1,9 @@
1
1
  <p align="center">
2
- <img src="https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/banner.jpg" alt="pi-research" width="100%" />
2
+ <img src="https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/banner.jpg" alt="pi-research — free, unlimited web search and deep research for coding agents" width="100%" />
3
3
  </p>
4
4
 
5
5
  <p align="center">
6
- <strong>Local web research and knowledge store</strong>
6
+ <strong>Free, unlimited web search &amp; deep research for coding agents — no API key, no quota</strong>
7
7
  </p>
8
8
 
9
9
  <p align="center">
@@ -24,75 +24,55 @@
24
24
 
25
25
  <a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
26
26
 
27
- **Free, unlimited web search &amp; scraping — no monthly quota, no subscription.**
28
- You pay only for the LLM tokens you use, and it builds your own private, searchable
29
- knowledge store on your machine.
27
+ Search and scraping run locally through a stealth browser — no search provider, no monthly cap. You pay only for the LLM tokens you use.
28
+
29
+ ### Install
30
+
31
+ ```bash
32
+ pi install npm:@lincoln504/pi-research
33
+ ```
34
+
35
+ In [pi](https://github.com/earendil-works/pi) it works out of the box, on your session's model and pi's configuration. Standalone use — [agent skill](docs/AGENT-SKILL.md), CLI, or [SDK](docs/SDK.md) — needs a model configured; see [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
30
36
 
31
37
  ### How it works
32
38
 
33
- A research run is a loop of agent teams: a coordinator plans the work and runs an
34
- opening search, parallel researcher agents scrape and read sources through a fixed,
35
- sandboxed toolkit (no shell access, and no ability to write or modify files), and an evaluator decides whether
36
- to run another round or synthesize — returning one cited Markdown report while its
37
- sources are saved to the knowledge store in the background.
39
+ A research run loops through agent teams: a coordinator plans and opens with a search, researcher agents scrape and read in parallel, and an evaluator decides whether to go another round or synthesize. The result is one cited Markdown report, with findings optionally saved to the knowledge store.
38
40
 
39
- Three depth levels — normal, deep, ultra — set the team size and number of rounds;
40
- ask in natural language and the tool picks the right one:
41
+ Three depth levels — normal, deep, ultra — set the team size and number of rounds. Ask in natural language and the tool picks the right one:
41
42
 
42
43
  ![Prompt-driven multi-round research in the pi TUI](https://raw.githubusercontent.com/Lincoln504/pi-research/main/docs/media/02-prompt-research.gif)
43
44
 
44
45
  ### Use cases
45
46
 
46
- - Plug-and-play (no API key needed) research tool for your [Pi](https://github.com/earendil-works/pi) Coding Agent.
47
- - Research from Claude Code, Codex, or another coding agent, with a cheaper lightweight or local model driving the research so it doesn't spend your main agent's budget.
48
- - Populating a dataset or building an index of knowledge sources from the web.
49
- - Holding research in the knowledge store with a configurable scope — project-specific or globally user-scoped, set per directory from the `/research-config` TUI.
50
- - Using the pi-research agent skill as OpenClaw's web access.
51
- - Building agent systems that identify and examine web sources.
47
+ - Researching for your [Pi](https://github.com/earendil-works/pi) Coding Agent — plug-and-play, no API key needed.
48
+ - Researching from Claude Code, Codex, OpenClaw, or another coding agent, with a cheaper or local model driving the run so it doesn't spend your main agent's budget.
49
+ - Keeping a persistent, searchable knowledge store of findings — global or per-project.
50
+ - Building agent systems that find and read web content, or populating a dataset of web sources.
52
51
 
53
52
  ### Why pi-research
54
53
 
55
- - **Free, unlimited search &amp; scrape — no quota, no monthly fee.** Most AI-search providers cap free searches per month and then charge; results run on *their* index on *their* servers. pi-research's search and scraping layer (DuckDuckGo via a stealth browser) is free and unlimited — you pay only for the LLM tokens you use, never per search or per month.
56
- - **You own your index.** Every finding is written into a local [LanceDB](https://lancedb.com) knowledge store on *your* machine — a persistent, searchable index you control and keep, which also seeds future runs so repeat questions get faster, cheaper answers. No borrowed, server-side index you can't keep or query. Findings can also be kept **globally**, scoped to a **single project**, or **not kept at all** — set per directory, changed whenever you like. The optionality is the point: the store is yours to build, partition, or switch off.
57
- - **Cited reports, not raw dumps.** It returns one synthesized, sourced Markdown report to the chat instead of dumping raw web content into the conversation — context-efficient and easy to verify.
58
- - **Read-only by design.** Web access runs inside a specialized, limited research agent that cannot run shell commands or write, edit, or delete anything. It can read and search files, so treat it as you would any read-only agent with access to your machine.
59
- - **Search a little or a lot** — three depth levels in the pi tool (four via the SDK and standalone CLI, which add the quick depth-0 path). Levels 1 and 2 are recommended for everyday workflow use; level 3 is for larger-scale investigations.
54
+ - **No quota, no monthly fee.** Most AI-search providers cap free searches and then charge, with results served from *their* index on *their* servers. pi-research searches and scrapes locally — DuckDuckGo via a stealth browser.
55
+ - **You own your index.** Every finding is written to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster, cheaper answers.
56
+ - **Cited reports, not raw dumps.** One synthesized, sourced Markdown report instead of raw web content pasted into the conversation — context-efficient and easy to verify.
57
+ - **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything. It can read and search files, so treat it as you would any read-only agent with access to your machine.
58
+ - **Search a little or a lot.** Depth levels span a quick pass to a large-scale investigation.
60
59
 
61
60
  ### Requirements / limitations
62
61
 
63
62
  - Node.js >= 22.19.0
64
- - An LLM with a 100k+ context window (bring your own key or use a local model)
65
- - Internet access
66
- - A residential IP address — search, scraping, and YouTube transcripts all rely on a residential connection. A datacenter/VPS/cloud IP gets bot-blocked by the providers these features depend on.
67
- - The pi runtime the engine builds on — `@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`, and `@earendil-works/pi-tui`. The pi extension uses the host's copies; the standalone CLI / agent skill install them as dependencies.
68
- - Cloudflare and similar anti-automation systems block scraping on some sites, so runs will identify sources they cannot reach. pi-research compensates with volume: the search tool rapidly pulls a large set of results off free DuckDuckGo, giving the model a wide selection of reachable content to cite.
69
-
70
- ### Install
71
-
72
- ```bash
73
- pi install npm:@lincoln504/pi-research
74
- ```
75
-
76
- See [Agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md) for standalone (no pi extension) installation.
77
-
78
- The pi extension works out of the box — it runs on your pi session's model and pi's own configuration. Standalone use (agent skill / CLI) requires a configured model: set `PI_RESEARCH_MODEL` (e.g. `openai/gpt-4o`) as an environment variable or in `~/.pi/research/config.env` (the CLI also accepts a per-run `--model`), plus `PI_RESEARCH_API_KEY` if you don't use pi; the SDK takes a `model` option. See [Configuration](docs/CONFIGURATION.md).
79
-
80
- The first install pulls the stealth browser engine, which takes a few minutes.
81
-
82
- ### Stability (v1.x.x)
63
+ - An LLM with a 100k+ context window — bring your own key or use a local model
64
+ - Internet access on a residential IP — search, scraping, and YouTube transcripts rely on a residential connection; a datacenter/VPS/cloud IP gets bot-blocked
65
+ - The [pi runtime](https://github.com/earendil-works/pi) — the pi extension uses the host's copy; the standalone CLI and agent skill install it as a dependency
66
+ - Cloudflare and similar systems block scraping on some sites — a run flags pages it can't reach and compensates with a wide pool of search results to cite
83
67
 
84
- v0.1.13 (April 2026) was the last release before a rebuild against current pi APIs
85
- and the stealth-browser stack. v1.0.0 is the first stable release since. Channels:
68
+ ### Channels
86
69
 
87
- - npm (`npm:@lincoln504/pi-research`) is the stable channel, kept current with breaking pi changes.
88
- - A git install is the development channel: latest commits, first to break.
70
+ npm (`npm:@lincoln504/pi-research`) is the stable channel, kept current with breaking pi changes. A git install is the development channel: latest commits, first to break.
89
71
 
90
72
  ### License
91
73
 
92
- MIT. Bundled third-party licenses are listed in
93
- [docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).
74
+ MIT. Bundled third-party licenses are listed in [docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).
94
75
 
95
76
  ### Package identity
96
77
 
97
- This project is always `@lincoln504/pi-research` on npm. The unscoped `pi-research`
98
- package is unrelated and deprecated.
78
+ This project is always `@lincoln504/pi-research` on npm. The `pi-research` (unscoped) package is unrelated and deprecated.
package/dist/cli.mjs CHANGED
@@ -216,7 +216,14 @@ function formatArg(arg) {
216
216
  }
217
217
  function redactSecrets(message) {
218
218
  const rawLength = message.length;
219
- let out = rawLength > REDACT_SCAN_LENGTH ? message.slice(0, REDACT_SCAN_LENGTH) : message;
219
+ let scanEnd = REDACT_SCAN_LENGTH;
220
+ if (rawLength > scanEnd) {
221
+ const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
222
+ while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
223
+ scanEnd += 1;
224
+ }
225
+ }
226
+ let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
220
227
  out = out.replace(ANSI_PATTERN, "");
221
228
  out = out.replace(URL_CREDENTIALS_PATTERN, "$1[REDACTED]@");
222
229
  out = out.replace(JWT_PATTERN, "[REDACTED]");
@@ -233,14 +240,14 @@ function redactSecrets(message) {
233
240
  if (lastKeptUnit >= 55296 && lastKeptUnit <= 56319) keep -= 1;
234
241
  }
235
242
  if (out.length > MAX_LOG_MESSAGE_LENGTH) {
236
- out = `${out.slice(0, keep)}\u2026[truncated ${rawLength - keep} chars]`;
243
+ out = `${out.slice(0, keep)}\u2026[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
237
244
  }
238
245
  return out;
239
246
  }
240
247
  function stripTerminalEscapes(message) {
241
248
  return message.replace(TERMINAL_ESCAPE_PATTERN, "").replace(CONTROL_CHARS_PATTERN, " ");
242
249
  }
243
- var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TERMINAL_ESCAPE_PATTERN;
250
+ var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TOKEN_CHAR_PATTERN, REDACT_SCAN_EXTENSION_MAX, TERMINAL_ESCAPE_PATTERN;
244
251
  var init_log_utils = __esm({
245
252
  "src/utils/log-utils.ts"() {
246
253
  "use strict";
@@ -258,6 +265,8 @@ var init_log_utils = __esm({
258
265
  LONG_HEX_SECRET_PATTERN = /\b[0-9a-fA-F]{32,}\b/g;
259
266
  CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
260
267
  REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
268
+ TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
269
+ REDACT_SCAN_EXTENSION_MAX = 2048;
261
270
  TERMINAL_ESCAPE_PATTERN = new RegExp(
262
271
  "\\x1b\\[[0-?]*[ -/]*[@-~]?|\\x1b\\][^\\x07\\x1b]*(?:\\x07|\\x1b\\\\)?|\\x1b[PX^_][^\\x1b]*(?:\\x1b\\\\)?|\\x1b.?",
263
272
  "g"
@@ -269,9 +278,18 @@ var init_log_utils = __esm({
269
278
  import * as fs from "node:fs";
270
279
  import * as path2 from "node:path";
271
280
  function touchOwnerOnly(filePath, mode) {
281
+ let fd;
272
282
  try {
273
- fs.closeSync(fs.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode));
283
+ fd = fs.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
284
+ fs.fchmodSync(fd, mode);
274
285
  } catch {
286
+ } finally {
287
+ if (fd !== void 0) {
288
+ try {
289
+ fs.closeSync(fd);
290
+ } catch {
291
+ }
292
+ }
275
293
  }
276
294
  }
277
295
  var APPEND_CREATE_NOFOLLOW_FLAGS, LogRotation;
@@ -297,10 +315,6 @@ var init_log_rotation = __esm({
297
315
  if (fs.existsSync(logFile)) {
298
316
  fs.unlinkSync(logFile);
299
317
  touchOwnerOnly(logFile, 384);
300
- try {
301
- fs.chmodSync(logFile, 384);
302
- } catch {
303
- }
304
318
  }
305
319
  const files = fs.readdirSync(logDir);
306
320
  const baseName = path2.basename(logFile);
@@ -337,10 +351,6 @@ var init_log_rotation = __esm({
337
351
  return;
338
352
  }
339
353
  touchOwnerOnly(logFile, 384);
340
- try {
341
- fs.chmodSync(logFile, 384);
342
- } catch {
343
- }
344
354
  try {
345
355
  const files = fs.readdirSync(logDir);
346
356
  const logFiles = files.filter((f) => f.startsWith(path2.basename(logFile)) && f !== path2.basename(logFile)).sort();
@@ -9914,19 +9924,19 @@ function reconcileSkillInstalls(opts = {}) {
9914
9924
  if (e.type !== "symlink") {
9915
9925
  const stale2 = currentVersion !== null && e.version !== currentVersion;
9916
9926
  if (stale2 && source !== null && fs16.existsSync(e.path) && isOwnedCopy(e.path)) {
9917
- const dir = path17.dirname(e.path);
9918
- const base = path17.basename(e.path);
9927
+ const dir2 = path17.dirname(e.path);
9928
+ const base2 = path17.basename(e.path);
9919
9929
  try {
9920
- for (const name of fs16.readdirSync(dir)) {
9921
- if (name.startsWith(`.${base}.new-`) || name.startsWith(`.${base}.old-`)) {
9922
- fs16.rmSync(path17.join(dir, name), { recursive: true, force: true });
9930
+ for (const name of fs16.readdirSync(dir2)) {
9931
+ if (name.startsWith(`.${base2}.new-`) || name.startsWith(`.${base2}.old-`)) {
9932
+ fs16.rmSync(path17.join(dir2, name), { recursive: true, force: true });
9923
9933
  }
9924
9934
  }
9925
9935
  } catch {
9926
9936
  }
9927
9937
  const suffix = crypto10.randomBytes(4).toString("hex");
9928
- const fresh = path17.join(dir, `.${base}.new-${suffix}`);
9929
- const retired = path17.join(dir, `.${base}.old-${suffix}`);
9938
+ const fresh = path17.join(dir2, `.${base2}.new-${suffix}`);
9939
+ const retired = path17.join(dir2, `.${base2}.old-${suffix}`);
9930
9940
  try {
9931
9941
  copyDir(source, fresh);
9932
9942
  } catch {
@@ -9972,6 +9982,16 @@ function reconcileSkillInstalls(opts = {}) {
9972
9982
  }
9973
9983
  continue;
9974
9984
  }
9985
+ const dir = path17.dirname(e.path);
9986
+ const base = path17.basename(e.path);
9987
+ try {
9988
+ for (const name of fs16.readdirSync(dir)) {
9989
+ if (name.startsWith(`.${base}.new-`)) {
9990
+ fs16.rmSync(path17.join(dir, name), { recursive: true, force: true });
9991
+ }
9992
+ }
9993
+ } catch {
9994
+ }
9975
9995
  if (!isSymlinkPresent(e.path)) {
9976
9996
  result.pruned.push(e.path);
9977
9997
  continue;
@@ -10000,8 +10020,6 @@ function reconcileSkillInstalls(opts = {}) {
10000
10020
  keep.push({ ...e, type: "symlink", source });
10001
10021
  result.repointed.push(e.path);
10002
10022
  } catch {
10003
- const dir = path17.dirname(e.path);
10004
- const base = path17.basename(e.path);
10005
10023
  const suffix = crypto10.randomBytes(4).toString("hex");
10006
10024
  const fresh = path17.join(dir, `.${base}.new-${suffix}`);
10007
10025
  try {
@@ -20677,14 +20695,22 @@ function describeError(error) {
20677
20695
  return String(error);
20678
20696
  }
20679
20697
  var mintChain = Promise.resolve();
20698
+ var wedged = false;
20680
20699
  var MINT_HARD_RELEASE_MS = 5 * 60 * 1e3;
20681
20700
  async function mintPoTokens(identifiers, opts = {}) {
20701
+ if (wedged) {
20702
+ throw new Error("PoToken minter is wedged by a prior mint that never settled; refusing to start a new mint until it resolves.");
20703
+ }
20682
20704
  const run = mintChain.then(() => doMint(identifiers, opts), () => doMint(identifiers, opts));
20683
20705
  const settled = run.then(() => void 0, () => void 0);
20706
+ settled.then(() => {
20707
+ wedged = false;
20708
+ });
20684
20709
  let hardTimer;
20685
20710
  const hardRelease = new Promise((resolve9) => {
20686
20711
  hardTimer = setTimeout(() => {
20687
20712
  logger.warn("[youtube] potoken mint mutex force-released \u2014 doMint() did not settle within the hard timeout");
20713
+ wedged = true;
20688
20714
  resolve9();
20689
20715
  }, MINT_HARD_RELEASE_MS);
20690
20716
  safeUnref(hardTimer);
@@ -46,7 +46,14 @@ function formatArg(arg) {
46
46
  }
47
47
  function redactSecrets(message) {
48
48
  const rawLength = message.length;
49
- let out = rawLength > REDACT_SCAN_LENGTH ? message.slice(0, REDACT_SCAN_LENGTH) : message;
49
+ let scanEnd = REDACT_SCAN_LENGTH;
50
+ if (rawLength > scanEnd) {
51
+ const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
52
+ while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
53
+ scanEnd += 1;
54
+ }
55
+ }
56
+ let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
50
57
  out = out.replace(ANSI_PATTERN, "");
51
58
  out = out.replace(URL_CREDENTIALS_PATTERN, "$1[REDACTED]@");
52
59
  out = out.replace(JWT_PATTERN, "[REDACTED]");
@@ -63,14 +70,14 @@ function redactSecrets(message) {
63
70
  if (lastKeptUnit >= 55296 && lastKeptUnit <= 56319) keep -= 1;
64
71
  }
65
72
  if (out.length > MAX_LOG_MESSAGE_LENGTH) {
66
- out = `${out.slice(0, keep)}\u2026[truncated ${rawLength - keep} chars]`;
73
+ out = `${out.slice(0, keep)}\u2026[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
67
74
  }
68
75
  return out;
69
76
  }
70
77
  function stripTerminalEscapes(message) {
71
78
  return message.replace(TERMINAL_ESCAPE_PATTERN, "").replace(CONTROL_CHARS_PATTERN, " ");
72
79
  }
73
- var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TERMINAL_ESCAPE_PATTERN;
80
+ var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TOKEN_CHAR_PATTERN, REDACT_SCAN_EXTENSION_MAX, TERMINAL_ESCAPE_PATTERN;
74
81
  var init_log_utils = __esm({
75
82
  "src/utils/log-utils.ts"() {
76
83
  "use strict";
@@ -88,6 +95,8 @@ var init_log_utils = __esm({
88
95
  LONG_HEX_SECRET_PATTERN = /\b[0-9a-fA-F]{32,}\b/g;
89
96
  CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
90
97
  REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
98
+ TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
99
+ REDACT_SCAN_EXTENSION_MAX = 2048;
91
100
  TERMINAL_ESCAPE_PATTERN = new RegExp(
92
101
  "\\x1b\\[[0-?]*[ -/]*[@-~]?|\\x1b\\][^\\x07\\x1b]*(?:\\x07|\\x1b\\\\)?|\\x1b[PX^_][^\\x1b]*(?:\\x1b\\\\)?|\\x1b.?",
93
102
  "g"
@@ -360,9 +369,18 @@ var init_error_tracker = __esm({
360
369
  import * as fs2 from "node:fs";
361
370
  import * as path2 from "node:path";
362
371
  function touchOwnerOnly(filePath, mode) {
372
+ let fd;
363
373
  try {
364
- fs2.closeSync(fs2.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode));
374
+ fd = fs2.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
375
+ fs2.fchmodSync(fd, mode);
365
376
  } catch {
377
+ } finally {
378
+ if (fd !== void 0) {
379
+ try {
380
+ fs2.closeSync(fd);
381
+ } catch {
382
+ }
383
+ }
366
384
  }
367
385
  }
368
386
  var APPEND_CREATE_NOFOLLOW_FLAGS, LogRotation;
@@ -388,10 +406,6 @@ var init_log_rotation = __esm({
388
406
  if (fs2.existsSync(logFile)) {
389
407
  fs2.unlinkSync(logFile);
390
408
  touchOwnerOnly(logFile, 384);
391
- try {
392
- fs2.chmodSync(logFile, 384);
393
- } catch {
394
- }
395
409
  }
396
410
  const files = fs2.readdirSync(logDir);
397
411
  const baseName = path2.basename(logFile);
@@ -428,10 +442,6 @@ var init_log_rotation = __esm({
428
442
  return;
429
443
  }
430
444
  touchOwnerOnly(logFile, 384);
431
- try {
432
- fs2.chmodSync(logFile, 384);
433
- } catch {
434
- }
435
445
  try {
436
446
  const files = fs2.readdirSync(logDir);
437
447
  const logFiles = files.filter((f) => f.startsWith(path2.basename(logFile)) && f !== path2.basename(logFile)).sort();
@@ -1920,11 +1930,21 @@ import { platform as platform2 } from "node:os";
1920
1930
 
1921
1931
  // src/infrastructure/browser/thread-worker-messaging.ts
1922
1932
  init_log_utils();
1923
- init_scraper_types();
1924
1933
  import { appendFileSync as appendFileSync2 } from "node:fs";
1934
+
1935
+ // src/utils/safe-unref.ts
1936
+ function safeUnref(timer) {
1937
+ if (timer && typeof timer.unref === "function") {
1938
+ timer.unref();
1939
+ }
1940
+ }
1941
+
1942
+ // src/infrastructure/browser/thread-worker-messaging.ts
1943
+ init_scraper_types();
1925
1944
  var workerId2 = "";
1926
1945
  var pageCreationLock = Promise.resolve();
1927
1946
  var PAGE_CREATION_HARD_RELEASE_MS = 5 * 60 * 1e3;
1947
+ var wedgedContext = null;
1928
1948
  async function createPageSafe(context2) {
1929
1949
  let release;
1930
1950
  const nextLock = new Promise((resolve) => {
@@ -1933,6 +1953,10 @@ async function createPageSafe(context2) {
1933
1953
  const currentLock = pageCreationLock;
1934
1954
  pageCreationLock = currentLock.then(() => nextLock);
1935
1955
  await currentLock;
1956
+ if (context2 === wedgedContext) {
1957
+ release();
1958
+ throw new Error("Browser context is wedged from a prior page-creation call that never settled; a new browser context is required.");
1959
+ }
1936
1960
  const pagePromise = context2.newPage();
1937
1961
  let hardReleased = false;
1938
1962
  const releaseOnce = () => {
@@ -1943,9 +1967,20 @@ async function createPageSafe(context2) {
1943
1967
  };
1944
1968
  const hardReleaseTimer = setTimeout(() => {
1945
1969
  logToDebugFile2("WARN", "[ThreadWorker] page-creation mutex force-released \u2014 newPage() did not settle within the hard timeout");
1970
+ wedgedContext = context2;
1946
1971
  releaseOnce();
1947
1972
  }, PAGE_CREATION_HARD_RELEASE_MS);
1948
- pagePromise.then(releaseOnce, releaseOnce);
1973
+ safeUnref(hardReleaseTimer);
1974
+ pagePromise.then(
1975
+ () => {
1976
+ if (wedgedContext === context2) wedgedContext = null;
1977
+ releaseOnce();
1978
+ },
1979
+ () => {
1980
+ if (wedgedContext === context2) wedgedContext = null;
1981
+ releaseOnce();
1982
+ }
1983
+ );
1949
1984
  pagePromise.catch((err) => logToDebugFile2("DEBUG", `[ThreadWorker] Background page creation rejection: ${err.message}`));
1950
1985
  let timeoutId;
1951
1986
  try {
package/docs/CHANGELOG.md CHANGED
@@ -5,6 +5,32 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [1.3.7] - 2026-08-11
9
+
10
+ Discovery and packaging metadata only — no engine changes.
11
+
12
+ ### Changed
13
+
14
+ - **README rewritten for concision and effectiveness.** The pitch ("free, unlimited web search & deep research for coding agents — no API key, no quota") now leads the page, install moved up to directly under the badges, each claim is stated once, and detail that only matters after adoption moved to `docs/`. 88 → 78 lines.
15
+ - **npm description and keywords rewritten.** The description is now benefit-led and survives catalog truncation; keywords gained `pi-coding-agent`, `web-research`, `deep-research`, `knowledge-store`, `duckduckgo`, and `skills`, and dropped `security` and `stack-exchange`.
16
+
17
+ ## [1.3.6] - 2026-08-10
18
+
19
+ An independent fresh-eyes review of the entire 1.3.4→1.3.5 diff — the previous
20
+ release's own fixes had only ever been reviewed by their author. It found seven
21
+ further defects, several in the exact hard-release-timer and redaction-boundary
22
+ patterns 1.3.5 introduced. Every fix below is covered by a regression test
23
+ confirmed (via a local revert-and-rerun) to fail against the pre-fix code.
24
+
25
+ ### Fixed
26
+
27
+ - **The page-creation and PoToken-mint hard-release timers (new in 1.3.5) could still let two calls race.** Both mutexes force-release after a 5-minute bound so a truly wedged call can't livelock the queue forever — but the bound has no way to tell "permanently hung" apart from "just very slow." A call that resolves a moment after the hard release fires was already let straight through to the next queued caller, who then ran its own `newPage()`/`doMint()` concurrently with the still-in-flight first one: the exact race each mutex exists to prevent (Playwright/Firefox deadlock on concurrent `newPage()`; corrupted `globalThis.window`/`document`/`navigator` from two overlapping bridge/unbridge pairs). Both now mark the wedged resource (the browser context; the whole PoToken minter, since `globalThis` is process-global) and reject further calls against it fast — no second `newPage()`/`doMint()` call — until the original stuck call actually settles.
28
+ - **`redactSecrets()`'s own cost-bounding pre-slice (new in 1.3.5) could itself cut a secret in half.** The 4×`MAX_LOG_MESSAGE_LENGTH` pre-slice added to bound redaction's cost on huge inputs runs before any pattern sees the text — the same class of mid-token truncation the truncate-before-redact fix (1.3.4) closed at the *final* boundary, just reopened at this new earlier one. A secret straddling that cut, in a message dense enough with other redactable content that the cut point's post-redaction position falls inside the final visible window, leaked its un-redacted fragment. The slice boundary now extends forward past any run of token-shaped characters (bounded to 2KB extra) so it never lands mid-secret.
29
+ - **`fs.chmodSync` right after the new `O_NOFOLLOW`-hardened `touchOwnerOnly()` (1.3.5) still followed symlinks.** The O_NOFOLLOW open correctly refused to write through a symlink planted at the log path, but the very next line re-stamped the file's mode with a plain, path-based `chmodSync` — which happily dereferences a symlink and silently chmods whatever it points at, undoing the hardening one line later. `touchOwnerOnly()` now stamps the mode itself via `fchmodSync` on the same already-open, already-verified file descriptor, and the separate path-based chmod calls are gone.
30
+ - **The truncation-count in a redacted log message could go negative.** 1.3.5 switched the reported "truncated N chars" count to the pre-redaction raw length so it would reflect content discarded by the earlier cost-bounding slice — but several redaction patterns *expand* text (`key=1` → `key=[REDACTED]`), so a short, redaction-dense message could end up with a post-redaction length exceeding the raw length, making the subtraction negative and printing the literal `…[truncated -1 chars]`. Now takes the larger of the raw and post-redaction lengths.
31
+ - **`reconcileSkillInstalls()`'s new (1.3.5) symlink-repoint copy fallback never swept its own staging leftovers.** The fallback stages a fresh copy into a dot-prefixed temp dir before an atomic rename into place; a process killed between the copy succeeding and the rename could leave that temp dir behind. The sibling copy-refresh branch already sweeps its own leftovers before every attempt — this branch didn't, and since a crash also frequently makes the manifest entry look deleted (pruning it before the branch runs again), repeated failures could accumulate orphaned staging directories with nothing ever revisiting them. Now sweeps unconditionally on every visit to a symlink entry, independent of whether that entry survives.
32
+ - **The page-creation mutex's hard-release timer (1.3.5) was never unref'd**, unlike the identical PoToken-mint timer in the same release. A bare `setTimeout` keeps the event loop alive for up to 5 minutes; any shutdown path that doesn't route through the cluster worker's forced `process.exit(0)` would hang waiting on it instead of exiting promptly. Now unref'd like its sibling.
33
+
8
34
  ## [1.3.5] - 2026-08-10
9
35
 
10
36
  Two rounds of workflow-backed code review on the clean 1.3.4 tree (high-effort,
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@lincoln504/pi-research",
3
- "version": "1.3.5",
4
- "description": "A set of web research tools (search, scrape, and others) and research agent system that uses them safely and efficiently.",
3
+ "version": "1.3.7",
4
+ "description": "Free, unlimited web search and deep research for the Pi coding agent — no search API key, no quotas. Local knowledge store; also works from Claude Code, Codex, and OpenClaw.",
5
5
  "type": "module",
6
6
  "private": false,
7
7
  "main": "./src/index.ts",
@@ -52,16 +52,20 @@
52
52
  "pi",
53
53
  "pi-package",
54
54
  "pi-extension",
55
+ "pi-coding-agent",
55
56
  "openclaw",
56
57
  "agent-skill",
58
+ "skills",
57
59
  "research",
60
+ "web-research",
61
+ "deep-research",
58
62
  "agent",
59
63
  "web-search",
60
64
  "scraping",
65
+ "knowledge-store",
66
+ "duckduckgo",
61
67
  "stealth-browser",
62
- "camoufox",
63
- "security",
64
- "stack-exchange"
68
+ "camoufox"
65
69
  ],
66
70
  "author": "Lincoln Deen",
67
71
  "contributors": [
@@ -2,7 +2,7 @@
2
2
  name: pi-research
3
3
  description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
4
4
  license: MIT
5
- metadata: { "author": "Lincoln Deen", "version": "1.3.5", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
5
+ metadata: { "author": "Lincoln Deen", "version": "1.3.7", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
6
6
  allowed-tools: Bash(node:*)
7
7
  ---
8
8
 
@@ -8,6 +8,7 @@ let workerId: string = '';
8
8
 
9
9
  import { appendFileSync } from 'node:fs';
10
10
  import { redactSecrets } from '../../utils/log-utils.ts';
11
+ import { safeUnref } from '../../utils/safe-unref.ts';
11
12
  import { MAX_HTML_SIZE, MAX_PDF_SIZE } from '../../web-research/scraper-types.ts';
12
13
 
13
14
  /**
@@ -21,6 +22,13 @@ let pageCreationLock = Promise.resolve();
21
22
  * bound the pathological case where newPage() never settles at all (see comment below). */
22
23
  const PAGE_CREATION_HARD_RELEASE_MS = 5 * 60 * 1000;
23
24
 
25
+ /** Set when a hard-release fires while its newPage() call is still pending —
26
+ * identifies the context that call was creating a page on. A concurrent
27
+ * newPage() on that SAME context is exactly what pageCreationLock exists to
28
+ * prevent, so any call for this context is rejected fast (without ever
29
+ * calling newPage()) until the stuck call actually settles and clears this. */
30
+ let wedgedContext: any = null;
31
+
24
32
  /** Exported for unit testing of the page-creation mutex's timeout behavior. */
25
33
  export async function createPageSafe(context: any): Promise<any> {
26
34
  let release!: () => void;
@@ -29,6 +37,13 @@ export async function createPageSafe(context: any): Promise<any> {
29
37
  pageCreationLock = currentLock.then(() => nextLock);
30
38
 
31
39
  await currentLock;
40
+ if (context === wedgedContext) {
41
+ // Release immediately so the queue keeps flowing for OTHER contexts (or
42
+ // this one, once the stuck call clears wedgedContext) instead of stalling
43
+ // every future caller behind the still-pending call.
44
+ release();
45
+ throw new Error('Browser context is wedged from a prior page-creation call that never settled; a new browser context is required.');
46
+ }
32
47
  const pagePromise = context.newPage();
33
48
  // Release the mutex once newPage() itself has actually settled — NOT when the
34
49
  // timeout race below settles. Releasing in an outer `finally` (the old shape)
@@ -43,7 +58,9 @@ export async function createPageSafe(context: any): Promise<any> {
43
58
  // A hard release timer bounds the case where newPage() never settles at all
44
59
  // (rather than just being slow) — without it, a single wedged newPage() call
45
60
  // would hold this lock forever and livelock every future createPageSafe() call
46
- // on this worker thread.
61
+ // on this worker thread. It marks the context wedged rather than letting the
62
+ // next caller straight through: a merely-slow (not hung) newPage() would
63
+ // otherwise still be in flight when a new one starts on the same context.
47
64
  let hardReleased = false;
48
65
  const releaseOnce = (): void => {
49
66
  if (hardReleased) return;
@@ -53,9 +70,14 @@ export async function createPageSafe(context: any): Promise<any> {
53
70
  };
54
71
  const hardReleaseTimer = setTimeout(() => {
55
72
  logToDebugFile('WARN', '[ThreadWorker] page-creation mutex force-released — newPage() did not settle within the hard timeout');
73
+ wedgedContext = context;
56
74
  releaseOnce();
57
75
  }, PAGE_CREATION_HARD_RELEASE_MS);
58
- pagePromise.then(releaseOnce, releaseOnce);
76
+ safeUnref(hardReleaseTimer);
77
+ pagePromise.then(
78
+ () => { if (wedgedContext === context) wedgedContext = null; releaseOnce(); },
79
+ () => { if (wedgedContext === context) wedgedContext = null; releaseOnce(); },
80
+ );
59
81
  pagePromise.catch((err: Error) => logToDebugFile('DEBUG', `[ThreadWorker] Background page creation rejection: ${err.message}`));
60
82
 
61
83
  let timeoutId: NodeJS.Timeout | undefined;
@@ -46,7 +46,14 @@ function formatArg(arg) {
46
46
  }
47
47
  function redactSecrets(message) {
48
48
  const rawLength = message.length;
49
- let out = rawLength > REDACT_SCAN_LENGTH ? message.slice(0, REDACT_SCAN_LENGTH) : message;
49
+ let scanEnd = REDACT_SCAN_LENGTH;
50
+ if (rawLength > scanEnd) {
51
+ const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
52
+ while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
53
+ scanEnd += 1;
54
+ }
55
+ }
56
+ let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
50
57
  out = out.replace(ANSI_PATTERN, "");
51
58
  out = out.replace(URL_CREDENTIALS_PATTERN, "$1[REDACTED]@");
52
59
  out = out.replace(JWT_PATTERN, "[REDACTED]");
@@ -63,14 +70,14 @@ function redactSecrets(message) {
63
70
  if (lastKeptUnit >= 55296 && lastKeptUnit <= 56319) keep -= 1;
64
71
  }
65
72
  if (out.length > MAX_LOG_MESSAGE_LENGTH) {
66
- out = `${out.slice(0, keep)}\u2026[truncated ${rawLength - keep} chars]`;
73
+ out = `${out.slice(0, keep)}\u2026[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
67
74
  }
68
75
  return out;
69
76
  }
70
77
  function stripTerminalEscapes(message) {
71
78
  return message.replace(TERMINAL_ESCAPE_PATTERN, "").replace(CONTROL_CHARS_PATTERN, " ");
72
79
  }
73
- var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TERMINAL_ESCAPE_PATTERN;
80
+ var logContextStorage, MAX_LOG_MESSAGE_LENGTH, ANSI_PATTERN, URL_CREDENTIALS_PATTERN, COOKIE_HEADER_PATTERN, SENSITIVE_KV_PATTERN, KNOWN_TOKEN_PATTERN, JWT_PATTERN, BASIC_AUTH_PATTERN, BEARER_AUTH_PATTERN, PROVIDER_TOKEN_PATTERN, LONG_HEX_SECRET_PATTERN, CONTROL_CHARS_PATTERN, REDACT_SCAN_LENGTH, TOKEN_CHAR_PATTERN, REDACT_SCAN_EXTENSION_MAX, TERMINAL_ESCAPE_PATTERN;
74
81
  var init_log_utils = __esm({
75
82
  "src/utils/log-utils.ts"() {
76
83
  "use strict";
@@ -88,6 +95,8 @@ var init_log_utils = __esm({
88
95
  LONG_HEX_SECRET_PATTERN = /\b[0-9a-fA-F]{32,}\b/g;
89
96
  CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
90
97
  REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
98
+ TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
99
+ REDACT_SCAN_EXTENSION_MAX = 2048;
91
100
  TERMINAL_ESCAPE_PATTERN = new RegExp(
92
101
  "\\x1b\\[[0-?]*[ -/]*[@-~]?|\\x1b\\][^\\x07\\x1b]*(?:\\x07|\\x1b\\\\)?|\\x1b[PX^_][^\\x1b]*(?:\\x1b\\\\)?|\\x1b.?",
93
102
  "g"
@@ -360,9 +369,18 @@ var init_error_tracker = __esm({
360
369
  import * as fs2 from "node:fs";
361
370
  import * as path2 from "node:path";
362
371
  function touchOwnerOnly(filePath, mode) {
372
+ let fd;
363
373
  try {
364
- fs2.closeSync(fs2.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode));
374
+ fd = fs2.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
375
+ fs2.fchmodSync(fd, mode);
365
376
  } catch {
377
+ } finally {
378
+ if (fd !== void 0) {
379
+ try {
380
+ fs2.closeSync(fd);
381
+ } catch {
382
+ }
383
+ }
366
384
  }
367
385
  }
368
386
  var APPEND_CREATE_NOFOLLOW_FLAGS, LogRotation;
@@ -388,10 +406,6 @@ var init_log_rotation = __esm({
388
406
  if (fs2.existsSync(logFile)) {
389
407
  fs2.unlinkSync(logFile);
390
408
  touchOwnerOnly(logFile, 384);
391
- try {
392
- fs2.chmodSync(logFile, 384);
393
- } catch {
394
- }
395
409
  }
396
410
  const files = fs2.readdirSync(logDir);
397
411
  const baseName = path2.basename(logFile);
@@ -428,10 +442,6 @@ var init_log_rotation = __esm({
428
442
  return;
429
443
  }
430
444
  touchOwnerOnly(logFile, 384);
431
- try {
432
- fs2.chmodSync(logFile, 384);
433
- } catch {
434
- }
435
445
  try {
436
446
  const files = fs2.readdirSync(logDir);
437
447
  const logFiles = files.filter((f) => f.startsWith(path2.basename(logFile)) && f !== path2.basename(logFile)).sort();
@@ -1920,11 +1930,21 @@ import { platform as platform2 } from "node:os";
1920
1930
 
1921
1931
  // src/infrastructure/browser/thread-worker-messaging.ts
1922
1932
  init_log_utils();
1923
- init_scraper_types();
1924
1933
  import { appendFileSync as appendFileSync2 } from "node:fs";
1934
+
1935
+ // src/utils/safe-unref.ts
1936
+ function safeUnref(timer) {
1937
+ if (timer && typeof timer.unref === "function") {
1938
+ timer.unref();
1939
+ }
1940
+ }
1941
+
1942
+ // src/infrastructure/browser/thread-worker-messaging.ts
1943
+ init_scraper_types();
1925
1944
  var workerId2 = "";
1926
1945
  var pageCreationLock = Promise.resolve();
1927
1946
  var PAGE_CREATION_HARD_RELEASE_MS = 5 * 60 * 1e3;
1947
+ var wedgedContext = null;
1928
1948
  async function createPageSafe(context2) {
1929
1949
  let release;
1930
1950
  const nextLock = new Promise((resolve) => {
@@ -1933,6 +1953,10 @@ async function createPageSafe(context2) {
1933
1953
  const currentLock = pageCreationLock;
1934
1954
  pageCreationLock = currentLock.then(() => nextLock);
1935
1955
  await currentLock;
1956
+ if (context2 === wedgedContext) {
1957
+ release();
1958
+ throw new Error("Browser context is wedged from a prior page-creation call that never settled; a new browser context is required.");
1959
+ }
1936
1960
  const pagePromise = context2.newPage();
1937
1961
  let hardReleased = false;
1938
1962
  const releaseOnce = () => {
@@ -1943,9 +1967,20 @@ async function createPageSafe(context2) {
1943
1967
  };
1944
1968
  const hardReleaseTimer = setTimeout(() => {
1945
1969
  logToDebugFile2("WARN", "[ThreadWorker] page-creation mutex force-released \u2014 newPage() did not settle within the hard timeout");
1970
+ wedgedContext = context2;
1946
1971
  releaseOnce();
1947
1972
  }, PAGE_CREATION_HARD_RELEASE_MS);
1948
- pagePromise.then(releaseOnce, releaseOnce);
1973
+ safeUnref(hardReleaseTimer);
1974
+ pagePromise.then(
1975
+ () => {
1976
+ if (wedgedContext === context2) wedgedContext = null;
1977
+ releaseOnce();
1978
+ },
1979
+ () => {
1980
+ if (wedgedContext === context2) wedgedContext = null;
1981
+ releaseOnce();
1982
+ }
1983
+ );
1949
1984
  pagePromise.catch((err) => logToDebugFile2("DEBUG", `[ThreadWorker] Background page creation rejection: ${err.message}`));
1950
1985
  let timeoutId;
1951
1986
  try {
@@ -631,6 +631,22 @@ export function reconcileSkillInstalls(opts: InstallOptions = {}): ReconcileResu
631
631
  continue;
632
632
  }
633
633
 
634
+ // Sweep any `.{base}.new-*` staging leftover from a previous crashed
635
+ // symlink-repoint attempt at this path, unconditionally — a process kill
636
+ // between copyDir succeeding and the final renameSync below can leave one
637
+ // behind, and this branch is not guaranteed to run again for the same
638
+ // entry (a crash also often leaves isSymlinkPresent() false, pruning the
639
+ // entry on the very next line before it would otherwise get a chance to).
640
+ const dir = path.dirname(e.path);
641
+ const base = path.basename(e.path);
642
+ try {
643
+ for (const name of fs.readdirSync(dir)) {
644
+ if (name.startsWith(`.${base}.new-`)) {
645
+ fs.rmSync(path.join(dir, name), { recursive: true, force: true });
646
+ }
647
+ }
648
+ } catch { /* hygiene only */ }
649
+
634
650
  // Link the user deleted by hand — nothing on disk; drop the stale entry.
635
651
  if (!isSymlinkPresent(e.path)) { result.pruned.push(e.path); continue; }
636
652
 
@@ -662,8 +678,6 @@ export function reconcileSkillInstalls(opts: InstallOptions = {}): ReconcileResu
662
678
  // partial, SKILL.md-less directory permanently orphaned on disk
663
679
  // (unrecognized by isOwnedCopy) when copyDir failed partway (disk
664
680
  // full, interrupted process).
665
- const dir = path.dirname(e.path);
666
- const base = path.basename(e.path);
667
681
  const suffix = crypto.randomBytes(4).toString('hex');
668
682
  const fresh = path.join(dir, `.${base}.new-${suffix}`);
669
683
  try {
@@ -21,12 +21,27 @@ const APPEND_CREATE_NOFOLLOW_FLAGS =
21
21
  (typeof fs.constants.O_NOFOLLOW === 'number' ? fs.constants.O_NOFOLLOW : 0);
22
22
 
23
23
  /** Best-effort (re)create `filePath` with `mode`, refusing to follow a symlink
24
- * planted at the path. Failures are swallowed by design — see call sites.
24
+ * planted at the path, and unconditionally stamp `mode` on it. Uses fchmodSync
25
+ * on the already-open (and O_NOFOLLOW-verified) descriptor rather than
26
+ * fs.chmodSync(filePath, ...) — chmodSync operates on the path and follows
27
+ * symlinks, which would silently re-open the exact hole O_NOFOLLOW closes if
28
+ * a symlink appears at this path between the open above and a path-based
29
+ * chmod. mode is only applied by open() when O_CREAT actually creates the
30
+ * file, so callers relying on the file always ending up at `mode` (even when
31
+ * a concurrent writer already recreated it) still need this explicit stamp.
32
+ * Failures are swallowed by design — see call sites.
25
33
  * Exported for unit testing of the symlink-refusal behavior. */
26
34
  export function touchOwnerOnly(filePath: string, mode: number): void {
35
+ let fd: number | undefined;
27
36
  try {
28
- fs.closeSync(fs.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode));
37
+ fd = fs.openSync(filePath, APPEND_CREATE_NOFOLLOW_FLAGS, mode);
38
+ fs.fchmodSync(fd, mode);
29
39
  } catch { /* best-effort — the next append recreates the file */ }
40
+ finally {
41
+ if (fd !== undefined) {
42
+ try { fs.closeSync(fd); } catch { /* best-effort */ }
43
+ }
44
+ }
30
45
  }
31
46
 
32
47
  export class LogRotation {
@@ -48,13 +63,11 @@ export class LogRotation {
48
63
  fs.unlinkSync(logFile);
49
64
  // Same rationale as the post-rotation recreate in rotateLogFile: the
50
65
  // unlink discarded the 0o600 file, and a umask-default recreate by the
51
- // next append would drop the owner-only posture.
66
+ // next append would drop the owner-only posture. touchOwnerOnly stamps
67
+ // the mode unconditionally (via fchmod on its own fd), covering the
68
+ // case where a concurrent append already recreated the file at umask
69
+ // perms between the unlink and this call.
52
70
  touchOwnerOnly(logFile, 0o600);
53
- // And, as there, re-stamp the mode: a concurrent append landing between
54
- // the unlink and the open makes the creation-time 0o600 a no-op.
55
- try {
56
- fs.chmodSync(logFile, 0o600);
57
- } catch { /* best-effort */ }
58
71
  }
59
72
 
60
73
  const files = fs.readdirSync(logDir);
@@ -109,18 +122,12 @@ export class LogRotation {
109
122
  // Rotation moved the 0o600 file aside; without an explicit recreate the next
110
123
  // append materializes it at the umask default (typically 0644) in a
111
124
  // world-traversable tmpdir, silently dropping the owner-only posture Logger
112
- // establishes at construction. Mode applies only at creation, so this is
113
- // harmless where a file already exists or mode semantics differ (win32).
114
- touchOwnerOnly(logFile, 0o600);
115
- // The mode above applies only if THIS open created the file. Several
116
- // processes append to this consolidated log, so a concurrent async append
117
- // can recreate it at umask perms between the rename and the open, turning
118
- // the 0o600 into a no-op — stamp the mode unconditionally to close that
119
- // window for good. ENOENT (nothing recreated it and the open failed) is
125
+ // establishes at construction. touchOwnerOnly stamps the mode
126
+ // unconditionally (via fchmod on its own fd) regardless of whether this
127
+ // open created the file or a concurrent append already recreated it
128
+ // between the rename and this call — ENOENT (open itself failed) is
120
129
  // tolerable: best-effort posture, same as the recreate itself.
121
- try {
122
- fs.chmodSync(logFile, 0o600);
123
- } catch { /* best-effort */ }
130
+ touchOwnerOnly(logFile, 0o600);
124
131
 
125
132
 
126
133
  // Clean up old archives
@@ -143,6 +143,15 @@ const CONTROL_CHARS_PATTERN = /[\x00-\x1f\x7f]/g;
143
143
  // O(input length) on multi-megabyte scraped payloads.
144
144
  const REDACT_SCAN_LENGTH = MAX_LOG_MESSAGE_LENGTH * 4;
145
145
 
146
+ // Characters that can appear inside an unbroken secret token (hex, base64url,
147
+ // JWT segments, API keys). Used to push the REDACT_SCAN_LENGTH cut point past
148
+ // the end of whatever token it would otherwise land in the middle of.
149
+ const TOKEN_CHAR_PATTERN = /[A-Za-z0-9_\-./+=]/;
150
+ // Hard cap on how far the boundary can be pushed past REDACT_SCAN_LENGTH —
151
+ // bounds the extra work to a fixed amount even against an adversarial run of
152
+ // token-like characters.
153
+ const REDACT_SCAN_EXTENSION_MAX = 2048;
154
+
146
155
  export function redactSecrets(message: string): string {
147
156
  // Redact BEFORE truncating. Every pattern below needs to see a secret in
148
157
  // full to match it (e.g. LONG_HEX_SECRET_PATTERN requires 32+ contiguous
@@ -151,7 +160,17 @@ export function redactSecrets(message: string): string {
151
160
  // browser auth secret) too short for any pattern to recognize, so it was
152
161
  // written out in the clear right before the "…[truncated N chars]" marker.
153
162
  const rawLength = message.length;
154
- let out = rawLength > REDACT_SCAN_LENGTH ? message.slice(0, REDACT_SCAN_LENGTH) : message;
163
+ // The REDACT_SCAN_LENGTH cut below is itself a truncation, subject to the
164
+ // exact same mid-token risk — extend it forward past any run of token
165
+ // characters straddling the cut point so it never slices a secret in half.
166
+ let scanEnd = REDACT_SCAN_LENGTH;
167
+ if (rawLength > scanEnd) {
168
+ const extensionLimit = Math.min(rawLength, scanEnd + REDACT_SCAN_EXTENSION_MAX);
169
+ while (scanEnd < extensionLimit && TOKEN_CHAR_PATTERN.test(message.charAt(scanEnd))) {
170
+ scanEnd += 1;
171
+ }
172
+ }
173
+ let out = rawLength > scanEnd ? message.slice(0, scanEnd) : message;
155
174
  out = out.replace(ANSI_PATTERN, '');
156
175
  out = out.replace(URL_CREDENTIALS_PATTERN, '$1[REDACTED]@');
157
176
  // Redact whole-token formats BEFORE the key/value pass: the KV value class
@@ -183,7 +202,12 @@ export function redactSecrets(message: string): string {
183
202
  if (lastKeptUnit >= 0xd800 && lastKeptUnit <= 0xdbff) keep -= 1;
184
203
  }
185
204
  if (out.length > MAX_LOG_MESSAGE_LENGTH) {
186
- out = `${out.slice(0, keep)}…[truncated ${rawLength - keep} chars]`;
205
+ // rawLength can be SMALLER than out.length when redaction net-expands text
206
+ // (e.g. "key=1" → "key=[REDACTED]"), which would make `rawLength - keep`
207
+ // negative. out.length is always > keep in this branch, so the max() is
208
+ // never negative; it still prefers the larger rawLength when the
209
+ // REDACT_SCAN_LENGTH cut discarded raw content redaction then shrank.
210
+ out = `${out.slice(0, keep)}…[truncated ${Math.max(rawLength, out.length) - keep} chars]`;
187
211
  }
188
212
  return out;
189
213
  }
@@ -77,6 +77,13 @@ function describeError(error: unknown): string {
77
77
  // Module-level mutex: serialize all globalThis mutation across concurrent callers.
78
78
  let mintChain: Promise<unknown> = Promise.resolve();
79
79
 
80
+ // Set when a hard-release fires while its doMint() is still bridging globals —
81
+ // globalThis is a single process-global resource, so unlike a per-context lock
82
+ // ANY new bridge while one is still installed is unsafe, not just one from the
83
+ // same caller. Blocks new mints until the stuck call's own doMint() actually
84
+ // settles (success or failure) and unbridges for real.
85
+ let wedged = false;
86
+
80
87
  // How long the mutex waits for a straggling doMint() before force-releasing
81
88
  // the chain for the next caller. doMint() awaits non-fetch operations with no
82
89
  // abort signal (interpreter eval, BotGuard VM execution against a
@@ -98,19 +105,34 @@ export async function mintPoTokens(
98
105
  identifiers: string[],
99
106
  opts: MintOptions = {},
100
107
  ): Promise<Map<string, string>> {
108
+ if (wedged) {
109
+ throw new Error('PoToken minter is wedged by a prior mint that never settled; refusing to start a new mint until it resolves.');
110
+ }
101
111
  // Chain on the mutex so only one bridged section runs at a time.
102
112
  const run = mintChain.then(() => doMint(identifiers, opts), () => doMint(identifiers, opts));
103
113
  // Keep the chain alive regardless of this call's success/failure, but bound
104
114
  // how long the NEXT caller waits on it: if `run` never settles, force the
105
115
  // chain open after the hard timeout rather than wedging every future
106
- // mintPoTokens() call behind it forever. The abandoned `run` may still
107
- // mutate globalThis in the background after this — an accepted trade-off
108
- // in the pathological case, same as the sibling browser-worker fix.
116
+ // mintPoTokens() call behind it forever. Marking `wedged` (rather than
117
+ // letting the next caller straight through to its own doMint()) prevents
118
+ // two calls from bridging globalThis at once — the abandoned `run` may
119
+ // still be mid-bridge in the background, and a second bridge/unbridge pair
120
+ // racing it would restore the WRONG saved descriptors, permanently
121
+ // corrupting globalThis.window/document/navigator for the process.
122
+ //
123
+ // Derived from `settled` (which never rejects) rather than attaching
124
+ // directly to `run` — `run.finally(...)` would return a new, unconsumed
125
+ // promise that adopts `run`'s rejection, becoming an unhandled rejection
126
+ // whenever doMint() throws.
109
127
  const settled = run.then(() => undefined, () => undefined);
128
+ // Once doMint() actually settles (success or failure), its own `finally`
129
+ // always runs unbridge() for real — safe for a new mint to bridge again.
130
+ settled.then(() => { wedged = false; });
110
131
  let hardTimer: NodeJS.Timeout;
111
132
  const hardRelease = new Promise<void>(resolve => {
112
133
  hardTimer = setTimeout(() => {
113
134
  logger.warn('[youtube] potoken mint mutex force-released — doMint() did not settle within the hard timeout');
135
+ wedged = true;
114
136
  resolve();
115
137
  }, MINT_HARD_RELEASE_MS);
116
138
  safeUnref(hardTimer);