@lincoln504/pi-research 1.6.1 → 1.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -44,11 +44,13 @@ npm install -g @lincoln504/pi-research
44
44
  pi-research skill install
45
45
  ```
46
46
 
47
- In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install pulls the stealth browser engine, which takes a few minutes.
47
+ On npm 12 or newer, add `--allow-scripts` to either install above. npm 12 stopped running dependency install scripts by default, and the stealth browser's native dependencies are built by one — without it searches fail with a missing-module error rather than a warning at install time.
48
+
49
+ In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install downloads the stealth browser engine, which takes a few minutes.
48
50
 
49
51
  ### How it works
50
52
 
51
- A research run loops through agent teams: a coordinator plans and opens with a search, researcher agents scrape and read in parallel, and a research lead decides whether to go another round — then writes the report from every report collected. The result is one cited Markdown report with findings optionally saved to the knowledge store.
53
+ A research run loops through agent teams: a coordinator plans and starts with a search, researcher agents scrape and read in parallel, and a research lead decides whether to go another round — then writes the report from every report collected. The result is one cited Markdown report with findings optionally saved to the knowledge store.
52
54
 
53
55
  Three depth levels (normal, deep, ultra) set the team size and number of rounds. A natural-language request is enough — the tool picks the right one:
54
56
 
@@ -61,13 +63,12 @@ Three depth levels (normal, deep, ultra) set the team size and number of rounds.
61
63
  - Keeping a persistent and searchable knowledge store of findings, scoped globally or per project.
62
64
  - Building agent systems that find and read web content, or populating a dataset of web sources.
63
65
 
64
- ### Why pi-research
66
+ ### Why
65
67
 
66
68
  - **No quota and no monthly fee.** Most AI search providers cap free searches and then charge, with results served from their index on their servers. pi-research searches and scrapes locally through DuckDuckGo in a stealth browser.
67
- - **The index stays local.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
68
- - **Cited reports rather than raw dumps.** Each run returns one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
69
+ - **The index is local.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
69
70
  - **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything — the right shape for an agent whose whole job is reading untrusted web pages. Prompt injection picked up mid-run has nothing to act on.
70
- - **Search a little or a lot.** Depth levels span a quick pass to a large-scale investigation.
71
+ - **Search a little or a lot.** Depth levels range from a quick pass to a large-scale investigation.
71
72
 
72
73
  ### Requirements / limitations
73
74
 
@@ -75,7 +76,7 @@ Three depth levels (normal, deep, ultra) set the team size and number of rounds.
75
76
  - An LLM with a 100k+ context window (an API key or a local model)
76
77
  - Internet access on a residential IP, since search, scraping, and YouTube transcripts get bot-blocked from datacenter/VPS/cloud IPs
77
78
  - pi. The pi extension uses the host's copy while the standalone CLI and agent skill install it as a dependency.
78
- - Cloudflare and similar systems block scraping on some sites. A run flags unreachable pages and compensates with a wide pool of search results to cite.
79
+ - Cloudflare Turnstile and similar systems block scraping on some sites. A run compensates with a wide pool of search results to scrape.
79
80
 
80
81
  ### Channels
81
82
 
@@ -2,7 +2,7 @@
2
2
  name: pi-research
3
3
  description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
4
4
  license: MIT
5
- metadata: { "author": "Lincoln Deen", "version": "1.6.1", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
5
+ metadata: { "author": "Lincoln Deen", "version": "1.6.3", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
6
6
  allowed-tools: Bash(node:*)
7
7
  ---
8
8
 
package/dist/cli.mjs CHANGED
@@ -6831,7 +6831,7 @@ var init_file_lock_service = __esm({
6831
6831
  const errnoError = error;
6832
6832
  if (errnoError.code === "EEXIST") {
6833
6833
  contentionCount++;
6834
- let reclaimEligibleAt = null;
6834
+ let awaitingReclaim = false;
6835
6835
  try {
6836
6836
  const rawContent = await fs7.readFile(this.lockFilePath, "utf-8");
6837
6837
  const parsed = this._parseLockContent(rawContent);
@@ -6843,7 +6843,7 @@ var init_file_lock_service = __esm({
6843
6843
  }
6844
6844
  lastSeenUuid = lockUuid;
6845
6845
  lastSeenMtime = stats.mtimeMs;
6846
- reclaimEligibleAt = (parsed?.pid ?? null) === null ? stats.mtimeMs + this._unidentifiedStaleThreshold() : null;
6846
+ awaitingReclaim = this.lockTimeout > 0 && (parsed?.pid ?? null) === null && lockAge <= this._unidentifiedStaleThreshold();
6847
6847
  if (lockUuid === this.lockUuid) {
6848
6848
  try {
6849
6849
  await fs7.unlink(this.lockFilePath);
@@ -6880,7 +6880,6 @@ var init_file_lock_service = __esm({
6880
6880
  }
6881
6881
  const stalledFor = Date.now() - lastProgressAt;
6882
6882
  const waitedFor = Date.now() - startTime;
6883
- const awaitingReclaim = this.lockTimeout > 0 && reclaimEligibleAt !== null && Date.now() < reclaimEligibleAt;
6884
6883
  if (stalledFor >= this.lockTimeout && !awaitingReclaim || waitedFor >= this.acquireCeilingMs) {
6885
6884
  metrics.observe("state_lock_acquire_duration_ms", waitedFor, { status: "timeout" });
6886
6885
  metrics.increment("state_lock_acquire_total", 1, { status: "timeout" });
@@ -10283,9 +10282,17 @@ function detectHarnesses(opts) {
10283
10282
  }
10284
10283
  } catch {
10285
10284
  }
10286
- return { ...h, present, absSkillsDir, absSkillPath, installed };
10285
+ const broken = (installed === "owned-symlink" || installed === "owned-copy") && !isUsableSkillDir(absSkillPath);
10286
+ return { ...h, present, absSkillsDir, absSkillPath, installed, broken };
10287
10287
  });
10288
10288
  }
10289
+ function isUsableSkillDir(p) {
10290
+ try {
10291
+ return fs16.existsSync(path18.join(p, "SKILL.md")) && fs16.existsSync(path18.join(p, "scripts", "run.mjs"));
10292
+ } catch {
10293
+ return false;
10294
+ }
10295
+ }
10289
10296
  function isSymlinkPresent(p) {
10290
10297
  try {
10291
10298
  return fs16.lstatSync(p).isSymbolicLink();
@@ -10455,8 +10462,8 @@ function reconcileSkillInstalls(opts = {}) {
10455
10462
  const keep = [];
10456
10463
  for (const e of manifest.entries) {
10457
10464
  if (e.type !== "symlink") {
10458
- const stale2 = currentVersion !== null && e.version !== currentVersion;
10459
- if (stale2 && source !== null && fs16.existsSync(e.path) && isOwnedCopy(e.path)) {
10465
+ const stale = currentVersion !== null && e.version !== currentVersion;
10466
+ if (stale && source !== null && fs16.existsSync(e.path) && isOwnedCopy(e.path)) {
10460
10467
  const dir2 = path18.dirname(e.path);
10461
10468
  const base2 = path18.basename(e.path);
10462
10469
  try {
@@ -10533,14 +10540,8 @@ function reconcileSkillInstalls(opts = {}) {
10533
10540
  keep.push(e);
10534
10541
  continue;
10535
10542
  }
10536
- const targetLive = fs16.existsSync(e.path);
10537
- let dest = null;
10538
- try {
10539
- dest = path18.resolve(path18.dirname(e.path), fs16.readlinkSync(e.path));
10540
- } catch {
10541
- }
10542
- const stale = source !== null && dest !== null && dest !== path18.resolve(source);
10543
- if (!targetLive || stale) {
10543
+ const targetLive = fs16.existsSync(e.path) && isUsableSkillDir(e.path);
10544
+ if (!targetLive) {
10544
10545
  if (source !== null) {
10545
10546
  try {
10546
10547
  fs16.unlinkSync(e.path);
@@ -12318,6 +12319,7 @@ import * as crypto4 from "node:crypto";
12318
12319
  import { join as join8 } from "node:path";
12319
12320
  import { existsSync as existsSync6, mkdirSync as mkdirSync3 } from "node:fs";
12320
12321
  import { platform, homedir as homedir3 } from "node:os";
12322
+ import { createRequire as createRequire3 } from "node:module";
12321
12323
  function getWindowsCamoufoxDir() {
12322
12324
  return join8(homedir3(), "AppData", "Local", "camoufox", "camoufox", "Cache");
12323
12325
  }
@@ -12426,6 +12428,20 @@ function resolveHeadlessMode() {
12426
12428
  if (process.env["PI_RESEARCH_USE_XVFB"] === "true") return "virtual";
12427
12429
  return true;
12428
12430
  }
12431
+ function probeBrowserNativeDeps(load = defaultNativeDepLoad) {
12432
+ try {
12433
+ load();
12434
+ return { ok: true };
12435
+ } catch (err) {
12436
+ return { ok: false, error: err instanceof Error ? err.message.split("\n")[0].trim() : String(err) };
12437
+ }
12438
+ }
12439
+ function defaultNativeDepLoad() {
12440
+ const require2 = createRequire3(import.meta.url);
12441
+ const Database = require2("better-sqlite3");
12442
+ const db = new Database(":memory:");
12443
+ db.close();
12444
+ }
12429
12445
  function isBrowserAvailable() {
12430
12446
  if (isFullMockMode()) return false;
12431
12447
  try {
@@ -12518,6 +12534,11 @@ function isPoolShutdownError(error) {
12518
12534
  // same transient swap the "being reset" string covers, not a hard fault.
12519
12535
  err.message.includes("Cannot execute a task on not started pool"));
12520
12536
  }
12537
+ var NATIVE_BINDING_FAILURE = /Could not locate the bindings file|ERR_DLOPEN_FAILED|NODE_MODULE_VERSION|was compiled against a different Node\.js version|invalid ELF header|Cannot find module '[^']*\.node'/;
12538
+ function isNativeBindingError(error) {
12539
+ const msg2 = error instanceof Error ? error.message : String(error);
12540
+ return NATIVE_BINDING_FAILURE.test(msg2);
12541
+ }
12521
12542
  var BENIGN_NAVIGATION_FAILURE = /page\.goto: (?:Timeout \d+ *ms exceeded|net::|NS_ERROR_|SEC_ERROR_|MOZILLA_PKIX_ERROR_)/;
12522
12543
  function isBenignScrapeFailure(error) {
12523
12544
  const msg2 = error instanceof Error ? error.message : String(error);
@@ -15749,7 +15770,7 @@ import { FixedClusterPool, WorkerChoiceStrategies } from "poolifier";
15749
15770
  // src/infrastructure/browser/ensure-browser.ts
15750
15771
  init_logger();
15751
15772
  import { spawn as spawn2 } from "node:child_process";
15752
- import { createRequire as createRequire3 } from "node:module";
15773
+ import { createRequire as createRequire4 } from "node:module";
15753
15774
  import { createHash as createHash2 } from "node:crypto";
15754
15775
  import { existsSync as existsSync7, readdirSync as readdirSync3, statSync as statSync3, fstatSync, openSync as openSync4, closeSync as closeSync4, rmSync } from "node:fs";
15755
15776
  import { join as join11, dirname as dirname8 } from "node:path";
@@ -15776,7 +15797,7 @@ function isBrowserBinaryPresent() {
15776
15797
  }
15777
15798
  function resolveCamoufoxBin() {
15778
15799
  try {
15779
- const require2 = createRequire3(import.meta.url);
15800
+ const require2 = createRequire4(import.meta.url);
15780
15801
  const pkgJson = require2.resolve("camoufox-js/package.json");
15781
15802
  const bin = join11(dirname8(pkgJson), "..", ".bin", process.platform === "win32" ? "camoufox-js.cmd" : "camoufox-js");
15782
15803
  if (existsSync7(bin)) return bin;
@@ -17532,6 +17553,12 @@ async function performSearch(queries, config, signal, onProgress, container = ge
17532
17553
  } else if (timeoutCount + errorCount === filteredQueries.length) {
17533
17554
  reason = `${timeoutCount} queries timed out and ${errorCount} queries failed`;
17534
17555
  }
17556
+ if (isNativeBindingError(sampleWorkerError)) {
17557
+ metrics.increment("browser_search_total_failures_total", 1, { cause: "native_binding" });
17558
+ throw new Error(
17559
+ `Search completely failed: ${reason}, all with a missing native module. This is an incomplete install, not a network problem: the browser engine's native dependencies were never built. npm 12 does not run dependency install scripts by default \u2014 reinstall allowing them (npm install @lincoln504/pi-research --allow-scripts, or npm approve-scripts --allow-scripts-pending), then retry.` + (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : "")
17560
+ );
17561
+ }
17535
17562
  throw new Error(
17536
17563
  `Search completely failed: ${reason}. Browser workers may be unavailable, DuckDuckGo is unreachable, or the system is under extreme load.` + (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : "")
17537
17564
  );
@@ -26707,11 +26734,12 @@ async function cmdStatus(json) {
26707
26734
  const paths = resolvedConfigPaths();
26708
26735
  const cfg = getConfig(process.cwd(), "cli");
26709
26736
  const browserInstalled = isBrowserAvailable();
26737
+ const nativeDeps = probeBrowserNativeDeps();
26710
26738
  const summary = {
26711
26739
  package: "@lincoln504/pi-research",
26712
26740
  version: PKG_VERSION,
26713
26741
  cli: `node ${process.argv[1]}`,
26714
- ready: !det.problem,
26742
+ ready: !det.problem && nativeDeps.ok,
26715
26743
  credentials: {
26716
26744
  source: det.source,
26717
26745
  apiKeyConfigured: det.apiKeyConfigured,
@@ -26722,6 +26750,9 @@ async function cmdStatus(json) {
26722
26750
  },
26723
26751
  browser: {
26724
26752
  installed: browserInstalled,
26753
+ nativeDepsOk: nativeDeps.ok,
26754
+ nativeDepsError: nativeDeps.ok ? null : nativeDeps.error,
26755
+ nativeDepsFix: nativeDeps.ok ? null : "Reinstall allowing dependency install scripts: npm install -g @lincoln504/pi-research --allow-scripts (npm 12 turns them off by default).",
26725
26756
  // Same remedy the health check prints, so both surfaces name one command.
26726
26757
  fix: browserInstalled ? null : "npx camoufox-js fetch",
26727
26758
  note: browserInstalled ? null : "Stealth browser not installed \u2014 scraping falls back to plain fetch, which some sites block. npm 11.19+ blocks install scripts by default, which skips this package's postinstall."
@@ -26739,7 +26770,16 @@ async function cmdStatus(json) {
26739
26770
  toStdout(`ready: ${summary.ready ? "yes" : "no"}
26740
26771
 
26741
26772
  `);
26742
- toStdout(configBlock(det, summary.ready ? void 0 : "\u2014 fix the problem above, then re-run.") + "\n");
26773
+ toStdout(configBlock(det, det.problem ? "\u2014 fix the problem above, then re-run." : void 0) + "\n");
26774
+ if (!summary.browser.nativeDepsOk) {
26775
+ toStdout(`
26776
+ browser engine: BROKEN \u2014 native dependencies were never built, so no search can run.
26777
+ `);
26778
+ toStdout(` ${summary.browser.nativeDepsFix}
26779
+ `);
26780
+ toStdout(` Underlying error: ${summary.browser.nativeDepsError}
26781
+ `);
26782
+ }
26743
26783
  if (!summary.browser.installed) {
26744
26784
  toStdout(`
26745
26785
  stealth browser: NOT INSTALLED \u2014 run: ${summary.browser.fix}
@@ -26812,7 +26852,7 @@ async function cmdSkill(s) {
26812
26852
  toStdout(pretty({
26813
26853
  command: "skill",
26814
26854
  action: "status",
26815
- agents: agents.map((d) => ({ id: d.id, label: d.label, present: d.present, installed: d.installed, path: d.absSkillPath }))
26855
+ agents: agents.map((d) => ({ id: d.id, label: d.label, present: d.present, installed: d.installed, broken: d.broken, path: d.absSkillPath }))
26816
26856
  }));
26817
26857
  return EXIT.OK;
26818
26858
  }
@@ -26820,7 +26860,8 @@ async function cmdSkill(s) {
26820
26860
 
26821
26861
  `);
26822
26862
  for (const d of agents) {
26823
- const state = !d.present ? "agent not detected" : d.installed === "none" ? "not installed" : d.installed === "foreign" ? "a different skill occupies this slot \u2014 left untouched" : `installed (${d.installed === "owned-copy" ? "copy" : "symlink"})`;
26863
+ const kind = d.installed === "owned-copy" ? "copy" : "symlink";
26864
+ const state = !d.present ? "agent not detected" : d.installed === "none" ? "not installed" : d.installed === "foreign" ? "a different skill occupies this slot \u2014 left untouched" : d.broken ? `BROKEN ${kind} \u2014 target is missing or incomplete; re-run \`${BINARY_NAME} skill install\`` : `installed (${kind})`;
26824
26865
  const loc = d.installed === "owned-symlink" || d.installed === "owned-copy" ? ` \u2192 ${d.absSkillPath}` : "";
26825
26866
  toStdout(` ${d.label.padEnd(18)} ${state}${loc}
26826
26867
  `);
package/docs/CHANGELOG.md CHANGED
@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [1.6.3] - 2026-08-25
9
+
10
+ A unit test that had already been granted CI-jitter headroom twice went red a third
11
+ time — and this time the timing signature (rejection 10008ms into a 10000ms window)
12
+ identified a real race in the lock acquire loop rather than a slow runner.
13
+
14
+ ### Fixed
15
+
16
+ - **A waiter gave up on a torn lock at the exact moment its reclaim window opened.** The acquire loop's reclaim decision measures the lock's age at `fs.stat` time, but the stall-check exemption that keeps a waiter alive until that reclaim ("never give up on a lock we are about to be allowed to take") compared a fresh `Date.now()` against the window. For the few milliseconds between the stat and the stall check the two clocks could disagree, and a retry tick landing astride the window boundary satisfied neither: not yet reclaimable by its stat, no longer awaited by the clock. Since a torn, pid-less lock never shows progress, the long-expired stall bound then fired, and a wait that had correctly outlasted it was abandoned right as the recovery became permitted — in production, a knowledge-store init failing at the moment it could have recovered. Both judgments now derive from the same observation, so every tick that can read the lock either reclaims it or keeps waiting; the try-take exclusion and the absolute ceiling are unchanged (`src/infrastructure/file-lock-service.ts`).
17
+ - **The test's forensic comment blamed scheduler jitter for all three CI failures.** True for the first two (which failed at the stall bound); wrong for this one (which failed at the window boundary). The record now distinguishes the two mechanisms (`test/unit/infrastructure/file-lock-progress-aware-wait.test.ts`).
18
+ - **A browser-config test deleted the whole suite's shared log directory as its own cleanup.** `unit-env.ts` routes every unit worker's log into `<tmpdir>/pi-research-test/`; the profile-containment test nested its scratch path inside that directory and removed the shared parent recursively — destroying the live log out from under every parallel worker, and racing the lazy logger re-creating the file mid-removal, which is an `ENOTEMPTY` crash of the cleanup itself (seen on macos-latest CI on this release's first tag). The test now creates its own `mkdtemp` directory and removes only that (`test/unit/infrastructure/browser-config.test.ts`).
19
+
20
+ ### Verified
21
+
22
+ - 2938 unit tests over 233 files, lint, type-check, and the full CI matrix green on the fix commit — including the ubuntu-latest job that caught the failure.
23
+
24
+ ## [1.6.2] - 2026-08-24
25
+
26
+ An agent-skill run reported that every search had failed and concluded it had hit an
27
+ unfixable environment problem. It was half right: the install really was broken, but
28
+ nothing in the tool would say so, and one of the two causes was the tool's own doing.
29
+
30
+ ### Fixed
31
+
32
+ - **A second install captured the skill links of the first.** `reconcileSkillInstalls()` runs on every engine-touching CLI invocation and resolves its source from whichever copy is being invoked. It re-pointed any owned link whose destination differed — including a perfectly healthy one — so an ordinary second install captured the user's global Claude/Codex/OpenClaw skill links merely by being run once. That is a documented setup, not an exotic one: `docs/SDK.md` instructs users to `npm install @lincoln504/pi-research` INTO their project so the SDK imports resolve, and anything that then ran the project-local CLI moved the global skill into the project. Moving, cleaning or deleting the project later killed the skill with an opaque `Cannot find module …/scripts/run.mjs`. Reconcile now repairs only a link that is dangling or whose target has been gutted — the case it was written for, which is what an update that relocates the package actually produces. Choosing between two live installs is what the explicit `skill install` is for. Two existing tests asserted the old behaviour and were unfaithful in the same way: each described "package relocated by an update" while leaving the old directory on disk, so the link never dangled (`src/skill-install/skill-installer.ts`).
33
+ - **`skill status` called a dead skill installed.** Detection tested that the link existed, never that it resolved, so a dangling symlink reported as `installed (symlink)` — leaving the harness's later "Cannot find module" as the only signal anything was wrong. A link or copy whose target is missing, or which lacks `SKILL.md` / `scripts/run.mjs`, is now reported as BROKEN with the command that repairs it (`src/skill-install/skill-installer.ts`, `src/cli.ts`).
34
+ - **`status` reported a build that cannot search as ready.** npm 12 turns dependency install scripts off by default. `camoufox-js` needs `better-sqlite3` to launch a browser at all, and its binding is produced by exactly such a script — so the module imports fine and throws only at first use. `ready` was derived from credentials alone and said yes; `isBrowserAvailable()` only stats the camoufox binary and never sees it. Measured across every native dependency this package ships: better-sqlite3 is the ONLY one that fails that way — onnxruntime-node carries its binding inside its own tarball, and lancedb, impit and html-to-markdown resolve prebuilt platform packages, which is why the knowledge store kept answering while search was dead. `status` now opens an in-memory database to settle it, gates `ready` on the result, and prints the remedy. The old comment claiming "nothing else fails" was wrong and is corrected (`src/infrastructure/browser/config.ts`, `src/cli.ts`).
35
+ - **A broken install was reported as a network outage.** With no binding, every browser worker dies identically and the run said browser workers may be unavailable, DuckDuckGo unreachable, or the system under extreme load. All three were false and none is actionable — a calling agent read it and gave up rather than reinstalling. A total search failure whose workers all died on a missing native module now says it is an incomplete install and names the flag that fixes it (`src/infrastructure/browser/browser-error-utils.ts`, `src/web-research/browser-search.ts`).
36
+
37
+ ### Changed
38
+
39
+ - **README documents the npm 12 install flag** on the install commands themselves, and drops a claim that a run flags unreachable pages. Turnstile is named because it is the challenge readers actually hit, and a handful of inflated verbs are now plain ones.
40
+
41
+ ### Verified
42
+
43
+ - 2938 unit tests over 233 files, lint, type-check over source and tests, dependency-cruiser.
44
+ - The failure mode was reproduced and the remedy measured: a scripts-blocked install of 1.6.1 fails `better-sqlite3` at first use and passes every other native dependency. A live agent-skill run through the installed link completes clean with zero binding errors.
45
+
8
46
  ## [1.6.1] - 2026-08-23
9
47
 
10
48
  A report that contained no analysis was being delivered as a completed run, because
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@lincoln504/pi-research",
3
- "version": "1.6.1",
3
+ "version": "1.6.3",
4
4
  "description": "Web research for your agents with smart and safe tooling + knowledge store",
5
5
  "type": "module",
6
6
  "private": false,
package/src/cli.ts CHANGED
@@ -45,7 +45,7 @@ import { getConfig, getGlobalConfigDir, getGlobalEnvFilePath, getInterfaceEnvFil
45
45
  import { getAgentDir } from '@earendil-works/pi-coding-agent';
46
46
  import { buildModelRegistry, safeGetAvailable } from './core/llm/model-registry-factory.ts';
47
47
  import { RESEARCH_TOOL_NAMES } from './constants.ts';
48
- import { isBrowserAvailable } from './infrastructure/browser/config.ts';
48
+ import { isBrowserAvailable, probeBrowserNativeDeps } from './infrastructure/browser/config.ts';
49
49
 
50
50
  // ---------------------------------------------------------------------------
51
51
  // Constants
@@ -796,19 +796,25 @@ async function cmdStatus(json?: boolean): Promise<number> {
796
796
  // (scripts/setup.cjs), which npm 11.19+ BLOCKS BY DEFAULT — it now warns
797
797
  // "packages have install scripts not yet covered by allowScripts" and carries
798
798
  // on, so the browser is simply absent on an otherwise successful install.
799
- // Every other native artifact survives that (onnxruntime, sharp, esbuild and
800
- // webgpu all ship prebuilt platform packages), which is exactly what makes
801
- // this one worth reporting: nothing else fails, and without this the first
802
- // symptom is a scrape falling back mid-run. A missing browser degrades
803
- // capability rather than breaking the tool — the fetch path still serves most
804
- // pages — so it is reported alongside `ready` rather than folded into it.
805
- // isBrowserAvailable() is a resolve + existsSync; it never launches anything.
799
+ // A missing browser DEGRADES capability rather than breaking the tool — the fetch
800
+ // path still serves most pages — so it is reported alongside `ready` rather than
801
+ // folded into it. isBrowserAvailable() is a resolve + existsSync; it never launches
802
+ // anything.
803
+ //
804
+ // The browser's NATIVE DEPENDENCIES are a separate and harsher question, and were
805
+ // previously assumed away here with "nothing else fails". Measured: better-sqlite3,
806
+ // which camoufox-js needs to launch at all, is built by a dependency install script
807
+ // and does NOT survive a scripts-blocked install. When it is missing every browser
808
+ // worker dies and SEARCH IS DEAD — not degraded — while the knowledge store keeps
809
+ // working because onnxruntime and lancedb both ship their binaries. That is not a
810
+ // capability note, so it gates `ready`: a build that cannot search cannot research.
806
811
  const browserInstalled = isBrowserAvailable();
812
+ const nativeDeps = probeBrowserNativeDeps();
807
813
  const summary = {
808
814
  package: '@lincoln504/pi-research',
809
815
  version: PKG_VERSION,
810
816
  cli: `node ${process.argv[1]}`,
811
- ready: !det.problem,
817
+ ready: !det.problem && nativeDeps.ok,
812
818
  credentials: {
813
819
  source: det.source,
814
820
  apiKeyConfigured: det.apiKeyConfigured,
@@ -819,6 +825,11 @@ async function cmdStatus(json?: boolean): Promise<number> {
819
825
  },
820
826
  browser: {
821
827
  installed: browserInstalled,
828
+ nativeDepsOk: nativeDeps.ok,
829
+ nativeDepsError: nativeDeps.ok ? null : nativeDeps.error,
830
+ nativeDepsFix: nativeDeps.ok
831
+ ? null
832
+ : 'Reinstall allowing dependency install scripts: npm install -g @lincoln504/pi-research --allow-scripts (npm 12 turns them off by default).',
822
833
  // Same remedy the health check prints, so both surfaces name one command.
823
834
  fix: browserInstalled ? null : 'npx camoufox-js fetch',
824
835
  note: browserInstalled
@@ -837,7 +848,19 @@ async function cmdStatus(json?: boolean): Promise<number> {
837
848
 
838
849
  toStdout(`pi-research ${PKG_VERSION}\n`);
839
850
  toStdout(`ready: ${summary.ready ? 'yes' : 'no'}\n\n`);
840
- toStdout(configBlock(det, summary.ready ? undefined : '— fix the problem above, then re-run.') + '\n');
851
+ // The hint points at the CONFIG block, so it belongs to a credential problem only.
852
+ // `ready` can now also be false because the browser's native deps are missing, and
853
+ // that is reported below with its own remedy — sending the reader back to a config
854
+ // block that has nothing wrong with it would be its own small lie.
855
+ toStdout(configBlock(det, det.problem ? '— fix the problem above, then re-run.' : undefined) + '\n');
856
+ if (!summary.browser.nativeDepsOk) {
857
+ // Printed BEFORE the browser-download note: a missing binary is a degradation the
858
+ // fetch path covers, while a missing binding means no search runs at all. The more
859
+ // severe problem has to be the one the reader sees first.
860
+ toStdout(`\nbrowser engine: BROKEN — native dependencies were never built, so no search can run.\n`);
861
+ toStdout(` ${summary.browser.nativeDepsFix}\n`);
862
+ toStdout(` Underlying error: ${summary.browser.nativeDepsError}\n`);
863
+ }
841
864
  if (!summary.browser.installed) {
842
865
  toStdout(`\nstealth browser: NOT INSTALLED — run: ${summary.browser.fix}\n`);
843
866
  toStdout(` ${summary.browser.note}\n`);
@@ -940,16 +963,21 @@ async function cmdSkill(s: NonNullable<ParsedArgs['skill']>): Promise<number> {
940
963
  if (s.json) {
941
964
  toStdout(pretty({
942
965
  command: 'skill', action: 'status',
943
- agents: agents.map((d) => ({ id: d.id, label: d.label, present: d.present, installed: d.installed, path: d.absSkillPath })),
966
+ agents: agents.map((d) => ({ id: d.id, label: d.label, present: d.present, installed: d.installed, broken: d.broken, path: d.absSkillPath })),
944
967
  }));
945
968
  return EXIT.OK;
946
969
  }
947
970
  toStdout(`pi-research agent skill — install status\n\n`);
948
971
  for (const d of agents) {
972
+ // A broken install must not read as a working one. It said "installed (symlink)"
973
+ // for a link whose target had been deleted, so the only sign anything was wrong
974
+ // came later, from the harness, as "Cannot find module …/scripts/run.mjs".
975
+ const kind = d.installed === 'owned-copy' ? 'copy' : 'symlink';
949
976
  const state = !d.present ? 'agent not detected'
950
977
  : d.installed === 'none' ? 'not installed'
951
978
  : d.installed === 'foreign' ? 'a different skill occupies this slot — left untouched'
952
- : `installed (${d.installed === 'owned-copy' ? 'copy' : 'symlink'})`;
979
+ : d.broken ? `BROKEN ${kind} — target is missing or incomplete; re-run \`${BINARY_NAME} skill install\``
980
+ : `installed (${kind})`;
953
981
  const loc = d.installed === 'owned-symlink' || d.installed === 'owned-copy' ? ` → ${d.absSkillPath}` : '';
954
982
  toStdout(` ${d.label.padEnd(18)} ${state}${loc}\n`);
955
983
  }
@@ -124,6 +124,27 @@ export function isPoolShutdownError(error: unknown): boolean {
124
124
  );
125
125
  }
126
126
 
127
+ /**
128
+ * A native addon that is missing or unloadable, i.e. a BROKEN INSTALL rather than
129
+ * anything about the page, the network or the load on the machine.
130
+ *
131
+ * The case that produced this: npm 12 turns `allowScripts` OFF by default, so a
132
+ * plain `npm install` no longer runs dependency lifecycle scripts — including the
133
+ * one that fetches/builds `better-sqlite3`, which `camoufox-js` needs to launch a
134
+ * browser at all. Every browser worker then dies with "Could not locate the
135
+ * bindings file", every query fails, and the run reported that DuckDuckGo might be
136
+ * unreachable or the machine under extreme load. Neither was true, and the advice
137
+ * that follows from each is useless. An install fault has to say so.
138
+ */
139
+ const NATIVE_BINDING_FAILURE =
140
+ /Could not locate the bindings file|ERR_DLOPEN_FAILED|NODE_MODULE_VERSION|was compiled against a different Node\.js version|invalid ELF header|Cannot find module '[^']*\.node'/;
141
+
142
+ /** True when the failure is a missing/unloadable native addon — a broken install. */
143
+ export function isNativeBindingError(error: unknown): boolean {
144
+ const msg = error instanceof Error ? error.message : String(error);
145
+ return NATIVE_BINDING_FAILURE.test(msg);
146
+ }
147
+
127
148
  /** Playwright navigation failures whose cause is at the remote end — see below. */
128
149
  const BENIGN_NAVIGATION_FAILURE =
129
150
  /page\.goto: (?:Timeout \d+ *ms exceeded|net::|NS_ERROR_|SEC_ERROR_|MOZILLA_PKIX_ERROR_)/;
@@ -12,6 +12,7 @@ import { platform, homedir } from 'node:os';
12
12
  import type { Config } from '../../config.ts';
13
13
  import { getConfig } from '../../config.ts';
14
14
  import { getLogger } from '../../logger.ts';
15
+ import { createRequire } from 'node:module';
15
16
 
16
17
  // ============================================================================
17
18
  // Binary Cache Management
@@ -310,6 +311,48 @@ export function resolveHeadlessMode(): boolean | 'virtual' {
310
311
  *
311
312
  * Single point of control: all existing skipTests() checks use this function.
312
313
  */
314
+ /**
315
+ * Whether the browser stack's NATIVE dependencies can actually load.
316
+ *
317
+ * `isBrowserAvailable()` answers a different question — is the camoufox binary on
318
+ * disk — and a browser can be fully downloaded and still unable to launch. camoufox-js
319
+ * requires `better-sqlite3`, whose binding is produced by a dependency INSTALL SCRIPT,
320
+ * and npm 12 turns those off by default. The module then imports fine and throws only
321
+ * when first used, so nothing notices until every browser worker dies mid-run with
322
+ * "Could not locate the bindings file" and the run reports a network problem it does
323
+ * not have.
324
+ *
325
+ * Measured, not assumed: of this package's native dependencies, only better-sqlite3
326
+ * fails a scripts-blocked install. onnxruntime-node ships its binding inside its own
327
+ * tarball, and lancedb, impit and html-to-markdown all resolve prebuilt platform
328
+ * packages, so the knowledge store keeps working while search is dead — exactly the
329
+ * split seen in the field.
330
+ *
331
+ * Opening an in-memory database is the only check that settles it: resolving the
332
+ * module or stat-ing a path does not, because the failure is in the binding lookup at
333
+ * first use. It costs a few milliseconds and is confined to diagnostics.
334
+ */
335
+ export function probeBrowserNativeDeps(
336
+ /** Injected for tests. Production callers pass nothing. */
337
+ load: () => void = defaultNativeDepLoad,
338
+ ): { ok: true } | { ok: false; error: string } {
339
+ try {
340
+ load();
341
+ return { ok: true };
342
+ } catch (err) {
343
+ // First line only: the bindings error enumerates every path it tried, which is
344
+ // pages of noise in a status block. The rest is in the log if it is wanted.
345
+ return { ok: false, error: err instanceof Error ? err.message.split('\n')[0]!.trim() : String(err) };
346
+ }
347
+ }
348
+
349
+ function defaultNativeDepLoad(): void {
350
+ const require = createRequire(import.meta.url);
351
+ const Database = require('better-sqlite3') as new (p: string) => { close(): void };
352
+ const db = new Database(':memory:');
353
+ db.close();
354
+ }
355
+
313
356
  export function isBrowserAvailable(): boolean {
314
357
  if (isFullMockMode()) return false;
315
358
  try {
@@ -694,12 +694,13 @@ export class FileLockService implements IService {
694
694
  const errnoError = error as NodeJS.ErrnoException;
695
695
  if (errnoError.code === 'EEXIST') {
696
696
  contentionCount++;
697
- // When the contended lock becomes eligible for reclaim, if this tick can
698
- // know it. Scoped per observation: a tick that could not read the lock
697
+ // Whether THIS tick judged the contended lock not-yet-eligible for the
698
+ // pid-less reclaim — i.e. we are waiting for a recovery we will be allowed
699
+ // to perform. Scoped per observation: a tick that could not read the lock
699
700
  // learned nothing, and an eligibility carried over from an earlier one is
700
701
  // no longer evidence of anything. See the stall check below for why the
701
702
  // stall bound must not fire before this moment.
702
- let reclaimEligibleAt: number | null = null;
703
+ let awaitingReclaim = false;
703
704
  // Read lock content to verify ownership and liveness before considering stale
704
705
  try {
705
706
  const rawContent = await fs.readFile(this.lockFilePath, 'utf-8');
@@ -720,17 +721,26 @@ export class FileLockService implements IService {
720
721
  // A lock with no readable pid is judged purely on age (see
721
722
  // _shouldReclaim), and its mtime never moves again — so it can never
722
723
  // show progress and the stall bound is guaranteed to expire first.
723
- // Record when it becomes reclaimable so the wait below can outlast the
724
- // stall bound rather than abandoning the recovery at 20s.
724
+ // Mark this tick as awaiting that reclaim so the wait below can outlast
725
+ // the stall bound rather than abandoning the recovery at 20s.
726
+ //
727
+ // Judged from the SAME lockAge observation _shouldReclaim will use, not
728
+ // from a fresh Date.now() at the stall check: the two clocks disagreed
729
+ // for the few ms between the stat and that check, and a tick landing
730
+ // astride the window boundary judged the lock not-yet-reclaimable here
731
+ // yet no-longer-awaited there — giving up at the exact moment the
732
+ // window opened (ubuntu CI, 2026-08-24: rejected 10008ms into a
733
+ // 10000ms window). The complement of _shouldReclaim's strict > means
734
+ // one of the two branches now holds on every readable tick.
725
735
  //
726
736
  // Deliberately NOT extended to a live owner: there the heartbeat is the
727
737
  // designed progress signal, its absence past lockTimeout is real evidence
728
738
  // of a wedge, and reporting that to the caller promptly beats waiting out
729
739
  // a two-to-four-minute live-owner threshold on a lock we may well never
730
740
  // be handed.
731
- reclaimEligibleAt = (parsed?.pid ?? null) === null
732
- ? stats.mtimeMs + this._unidentifiedStaleThreshold()
733
- : null;
741
+ awaitingReclaim = this.lockTimeout > 0
742
+ && (parsed?.pid ?? null) === null
743
+ && lockAge <= this._unidentifiedStaleThreshold();
734
744
 
735
745
  if (lockUuid === this.lockUuid) {
736
746
  // This is our own lock — a previous attempt was interrupted. Reclaim.
@@ -802,8 +812,6 @@ export class FileLockService implements IService {
802
812
  // on the spot rather than wait for anything, and the run semaphore's slot
803
813
  // files carry a TEN-MINUTE unidentified window, so honouring eligibility
804
814
  // there would turn "is this slot free right now" into a ten-minute block.
805
- const awaitingReclaim =
806
- this.lockTimeout > 0 && reclaimEligibleAt !== null && Date.now() < reclaimEligibleAt;
807
815
  if ((stalledFor >= this.lockTimeout && !awaitingReclaim) || waitedFor >= this.acquireCeilingMs) {
808
816
  metrics.observe('state_lock_acquire_duration_ms', waitedFor, { status: 'timeout' });
809
817
  metrics.increment('state_lock_acquire_total', 1, { status: 'timeout' });
@@ -325,6 +325,14 @@ export interface DetectedHarness extends HarnessDef {
325
325
  absSkillPath: string;
326
326
  /** Current install state of THIS skill at that path. */
327
327
  installed: 'none' | 'owned-symlink' | 'owned-copy' | 'foreign';
328
+ /**
329
+ * Ours, but not usable: the symlink dangles, or the directory it resolves to is
330
+ * missing SKILL.md / scripts/run.mjs. Reported because the alternative is what
331
+ * actually happened — `skill status` said "installed (symlink)" for a link whose
332
+ * target had been deleted, so the only signal the user got was the harness failing
333
+ * later with "Cannot find module …/scripts/run.mjs".
334
+ */
335
+ broken: boolean;
328
336
  }
329
337
 
330
338
  export function detectHarnesses(opts?: { home?: string }): DetectedHarness[] {
@@ -342,10 +350,28 @@ export function detectHarnesses(opts?: { home?: string }): DetectedHarness[] {
342
350
  else installed = 'foreign';
343
351
  }
344
352
  } catch { /* none */ }
345
- return { ...h, present, absSkillsDir, absSkillPath, installed };
353
+ const broken = (installed === 'owned-symlink' || installed === 'owned-copy')
354
+ && !isUsableSkillDir(absSkillPath);
355
+ return { ...h, present, absSkillsDir, absSkillPath, installed, broken };
346
356
  });
347
357
  }
348
358
 
359
+ /**
360
+ * The two files a skill directory is useless without: the manifest a harness reads to
361
+ * discover the skill, and the launcher it executes. A link or copy can exist while the
362
+ * thing it points at does not — the target was deleted, moved, or partially removed —
363
+ * and the harness then fails at invocation time with "Cannot find module …/run.mjs",
364
+ * which says nothing about the install being the problem. Checked rather than assumed
365
+ * so both `skill status` and the startup reconcile can tell a live install from a
366
+ * hollow one.
367
+ */
368
+ function isUsableSkillDir(p: string): boolean {
369
+ try {
370
+ return fs.existsSync(path.join(p, 'SKILL.md'))
371
+ && fs.existsSync(path.join(p, 'scripts', 'run.mjs'));
372
+ } catch { return false; }
373
+ }
374
+
349
375
  function isSymlinkPresent(p: string): boolean {
350
376
  try { return fs.lstatSync(p).isSymbolicLink(); } catch { return false; }
351
377
  }
@@ -695,12 +721,23 @@ export function reconcileSkillInstalls(opts: InstallOptions = {}): ReconcileResu
695
721
  // Only ever touch links we own; a foreign link at this path stays + tracked.
696
722
  if (!isOwnedSymlink(e.path)) { keep.push(e); continue; }
697
723
 
698
- const targetLive = fs.existsSync(e.path); // existsSync follows the link
699
- let dest: string | null = null;
700
- try { dest = path.resolve(path.dirname(e.path), fs.readlinkSync(e.path)); } catch { /* unreadable */ }
701
-
702
- const stale = source !== null && dest !== null && dest !== path.resolve(source);
703
- if (!targetLive || stale) {
724
+ // A target that resolves but has been gutted (partial delete, interrupted
725
+ // install, pruned tree) is as dead as one that is gone — and it fails later,
726
+ // at invocation, with an error that never mentions the install. Treat both as
727
+ // needing repair.
728
+ const targetLive = fs.existsSync(e.path) && isUsableSkillDir(e.path); // existsSync follows the link
729
+
730
+ // A link pointing at a DIFFERENT but perfectly live pi-research install is left
731
+ // alone. Re-pointing it was a silent hijack: reconcile runs on every
732
+ // engine-touching CLI invocation, from whichever copy of the package is being
733
+ // invoked, so any second install captured the user's skill links just by being
734
+ // run once. docs/SDK.md tells users to `npm install @lincoln504/pi-research` INTO
735
+ // their project, so the second install is a documented, ordinary setup — and when
736
+ // that project was later moved or cleaned, the global skill died with an opaque
737
+ // "Cannot find module .../run.mjs". Self-healing only needs the dangling case
738
+ // (`!targetLive`), which is what an update that relocates the package produces;
739
+ // choosing between two live installs is what the explicit `skill install` is for.
740
+ if (!targetLive) {
704
741
  if (source !== null) {
705
742
  // Re-point to the current package source (handles update/reinstall).
706
743
  try {
@@ -6,7 +6,7 @@
6
6
 
7
7
  import { runWorkerSearch } from '../infrastructure/browser/task-execution-service.ts';
8
8
  import { getMaxWorkers, COLD_START_ALLOWANCE_MS } from '../infrastructure/browser/config.ts';
9
- import { isTaskTimeoutError } from '../infrastructure/browser/browser-error-utils.ts';
9
+ import { isTaskTimeoutError, isNativeBindingError } from '../infrastructure/browser/browser-error-utils.ts';
10
10
  import { logger } from '../logger.ts';
11
11
  import { safeUnref } from '../utils/safe-unref.ts';
12
12
  import { normalizeUrl } from '../utils/url-utils.ts';
@@ -321,6 +321,24 @@ export async function performSearch(
321
321
  reason = `${timeoutCount} queries timed out and ${errorCount} queries failed`;
322
322
  }
323
323
 
324
+ // A missing native addon is a BROKEN INSTALL, and the generic advice below
325
+ // actively misleads: it sends the reader to check their network and their
326
+ // machine load when neither is involved and neither can be acted on. npm 12
327
+ // defaults `allowScripts` to off, so a plain install no longer builds
328
+ // camoufox-js's better-sqlite3 dependency and every browser worker dies
329
+ // identically. Say that instead, and say what fixes it.
330
+ if (isNativeBindingError(sampleWorkerError)) {
331
+ metrics.increment('browser_search_total_failures_total', 1, { cause: 'native_binding' });
332
+ throw new Error(
333
+ `Search completely failed: ${reason}, all with a missing native module. ` +
334
+ `This is an incomplete install, not a network problem: the browser engine's ` +
335
+ `native dependencies were never built. npm 12 does not run dependency install ` +
336
+ `scripts by default — reinstall allowing them (npm install @lincoln504/pi-research ` +
337
+ `--allow-scripts, or npm approve-scripts --allow-scripts-pending), then retry.` +
338
+ (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : '')
339
+ );
340
+ }
341
+
324
342
  throw new Error(
325
343
  `Search completely failed: ${reason}. ` +
326
344
  `Browser workers may be unavailable, DuckDuckGo is unreachable, or the system is under extreme load.` +