@lincoln504/pi-research 1.6.2 → 1.6.4

package/README.md

@@ -44,7 +44,13 @@ npm install -g @lincoln504/pi-research
 pi-research skill install
 ```
 
-On npm 12 or newer, add `--allow-scripts` to either install above. npm 12 stopped running dependency install scripts by default, and the stealth browser's native dependencies are built by one — without it searches fail with a missing-module error rather than a warning at install time.
+On npm 12 or newer, allow the one install script search depends on before installing:
+
+```bash
+npm config set allow-scripts=better-sqlite3 --location=user
+```
+
+npm 12 stopped running dependency install scripts by default, and the stealth browser's SQLite module is built by one. Skipped, the install still succeeds with a warning — but every search fails with a missing-module error. The config covers both installs; for the standalone install, `--allow-scripts=better-sqlite3` on the `npm install -g` line also works. The bare flag allows nothing, and `pi install` accepts only the config form.
 
 In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install downloads the stealth browser engine, which takes a few minutes.
 
@@ -63,11 +69,10 @@ Three depth levels (normal, deep, ultra) set the team size and number of rounds.
 - Keeping a persistent and searchable knowledge store of findings, scoped globally or per project.
 - Building agent systems that find and read web content, or populating a dataset of web sources.
 
-### Why pi-research
+### Why
 
 - **No quota and no monthly fee.** Most AI search providers cap free searches and then charge, with results served from their index on their servers. pi-research searches and scrapes locally through DuckDuckGo in a stealth browser.
 - **The index is local.** Every finding can be saved to a local [LanceDB](https://lancedb.com) knowledge store that seeds future runs, so repeat questions get faster and cheaper answers.
-- **Cited reports rather than raw dumps.** Each run returns one sourced Markdown report instead of raw web content pasted into the conversation. Context efficient and easy to verify.
 - **Read-only by design.** The research agent cannot run shell commands or write, edit, or delete anything — the right shape for an agent whose whole job is reading untrusted web pages. Prompt injection picked up mid-run has nothing to act on.
 - **Search a little or a lot.** Depth levels range from a quick pass to a large-scale investigation.
 

package/agent-skill/pi-research/README.md

@@ -47,6 +47,10 @@ npm install -g @lincoln504/pi-research
 node "<skill_dir>/scripts/run.mjs" status  # verify detection
 ```
 
+On npm 12 or newer, first run `npm config set allow-scripts=better-sqlite3 --location=user`:
+npm 12 blocks the install script that builds the stealth browser's SQLite module, and
+without it every search fails with a missing-module error.
+
 ### Install the skill
 
 Recommended. From the pi extension, run `/research-config` and choose

package/agent-skill/pi-research/SKILL.md

@@ -2,7 +2,7 @@
 name: pi-research
 description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
 license: MIT
-metadata: { "author": "Lincoln Deen", "version": "1.6.2", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
+metadata: { "author": "Lincoln Deen", "version": "1.6.4", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
 allowed-tools: Bash(node:*)
 ---
 

package/dist/cli.mjs

@@ -6831,7 +6831,7 @@ var init_file_lock_service = __esm({
                 const errnoError = error;
                 if (errnoError.code === "EEXIST") {
                   contentionCount++;
-                  let reclaimEligibleAt = null;
+                  let awaitingReclaim = false;
                   try {
                     const rawContent = await fs7.readFile(this.lockFilePath, "utf-8");
                     const parsed = this._parseLockContent(rawContent);
@@ -6843,7 +6843,7 @@ var init_file_lock_service = __esm({
                     }
                     lastSeenUuid = lockUuid;
                     lastSeenMtime = stats.mtimeMs;
-                    reclaimEligibleAt = (parsed?.pid ?? null) === null ? stats.mtimeMs + this._unidentifiedStaleThreshold() : null;
+                    awaitingReclaim = this.lockTimeout > 0 && (parsed?.pid ?? null) === null && lockAge <= this._unidentifiedStaleThreshold();
                     if (lockUuid === this.lockUuid) {
                       try {
                         await fs7.unlink(this.lockFilePath);
@@ -6880,7 +6880,6 @@ var init_file_lock_service = __esm({
                   }
                   const stalledFor = Date.now() - lastProgressAt;
                   const waitedFor = Date.now() - startTime;
-                  const awaitingReclaim = this.lockTimeout > 0 && reclaimEligibleAt !== null && Date.now() < reclaimEligibleAt;
                   if (stalledFor >= this.lockTimeout && !awaitingReclaim || waitedFor >= this.acquireCeilingMs) {
                     metrics.observe("state_lock_acquire_duration_ms", waitedFor, { status: "timeout" });
                     metrics.increment("state_lock_acquire_total", 1, { status: "timeout" });
@@ -17557,7 +17556,7 @@ async function performSearch(queries, config, signal, onProgress, container = ge
     if (isNativeBindingError(sampleWorkerError)) {
       metrics.increment("browser_search_total_failures_total", 1, { cause: "native_binding" });
       throw new Error(
-        `Search completely failed: ${reason}, all with a missing native module. This is an incomplete install, not a network problem: the browser engine's native dependencies were never built. npm 12 does not run dependency install scripts by default \u2014 reinstall allowing them (npm install @lincoln504/pi-research --allow-scripts, or npm approve-scripts --allow-scripts-pending), then retry.` + (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : "")
+        `Search completely failed: ${reason}, all with a missing native module. This is an incomplete install, not a network problem: the browser engine's native dependencies were never built. npm 12 does not run dependency install scripts by default \u2014 reinstall allowing the script (global install: npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3; project install: npm approve-scripts better-sqlite3, then npm rebuild better-sqlite3), then retry.` + (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : "")
       );
     }
     throw new Error(
@@ -26753,10 +26752,14 @@ async function cmdStatus(json) {
       installed: browserInstalled,
       nativeDepsOk: nativeDeps.ok,
       nativeDepsError: nativeDeps.ok ? null : nativeDeps.error,
-      nativeDepsFix: nativeDeps.ok ? null : "Reinstall allowing dependency install scripts: npm install -g @lincoln504/pi-research --allow-scripts (npm 12 turns them off by default).",
+      // --allow-scripts is a package-name LIST, not a boolean: the bare flag is
+      // accepted and silently allows nothing (measured on npm 12.0.2). It is
+      // also global/one-off only — a project-scoped install rejects it with
+      // EALLOWSCRIPTS, where `npm approve-scripts` is the supported route.
+      nativeDepsFix: nativeDeps.ok ? null : "Reinstall allowing the dependency install script: npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3 (npm 12 blocks dependency install scripts by default). For a project-scoped install: npm approve-scripts better-sqlite3, then npm rebuild better-sqlite3.",
       // Same remedy the health check prints, so both surfaces name one command.
       fix: browserInstalled ? null : "npx camoufox-js fetch",
-      note: browserInstalled ? null : "Stealth browser not installed \u2014 scraping falls back to plain fetch, which some sites block. npm 11.19+ blocks install scripts by default, which skips this package's postinstall."
+      note: browserInstalled ? null : "Stealth browser not installed \u2014 scraping falls back to plain fetch, which some sites block. npm 12 blocks install scripts by default, which skips this package's postinstall; the engine is fetched on first use instead."
     },
     knowledgeStoreMode: cfg.KNOWLEDGE_STORE_MODE,
     defaultDepth: cfg.DEFAULT_RESEARCH_DEPTH,

package/docs/AGENT-SKILL.md

@@ -21,6 +21,10 @@ npm install -g @lincoln504/pi-research   # the engine (puts `pi-research` on PAT
 pi-research skill install                # link the skill into every detected agent
 ```
 
+On npm 12 or newer, first run `npm config set allow-scripts=better-sqlite3 --location=user`:
+npm 12 blocks the install script that builds the stealth browser's SQLite module, and
+without it every search fails with a missing-module error (see the [README](../README.md#install)).
+
 `skill install` targets only agents already set up under `$HOME`, never overwrites a
 different skill in the slot, and records what it created so `pi-research skill uninstall`
 removes exactly that. Run `pi-research skill status` to see where it is installed.

package/docs/CHANGELOG.md

@@ -5,6 +5,41 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.6.4] - 2026-08-26
+
+The npm 12 install remedy shipped in 1.6.2 was put through real installs in
+disposable npm 12.0.2 containers and found not to work: the bare `--allow-scripts`
+flag every surface recommended is accepted by npm and allows nothing.
+
+### Fixed
+
+- **`status` and the total-search-failure error printed a repair command that repairs nothing.** Measured per npm version against a scripted tarball: npm ≤11.15 runs dependency install scripts silently, 11.16–11.19 warn but still run them, and npm 12 blocks them — so the 1.6.2 claim that "npm 11.19+ blocks" was wrong on version, and the remedy was worse than wrong on form: `--allow-scripts` is a comma-separated package-name list, so the bare flag is accepted and allows nothing — the install stays broken. The flag is also global-install-only: a project-scoped install (the SDK path, and what `pi install` runs underneath) rejects it with `EALLOWSCRIPTS`. And after a blocked install, approving and rerunning `npm install` does not rebuild the already-extracted package. Both surfaces now print commands measured to work — `--allow-scripts=better-sqlite3` for a global install; `npm approve-scripts better-sqlite3` then `npm rebuild better-sqlite3` for a project-scoped one (`src/cli.ts`, `src/web-research/browser-search.ts`).
+
+### Changed
+
+- **Install docs lead with the one form measured to cover every install path** — `npm config set allow-scripts=better-sqlite3 --location=user`, verified against a fresh `pi install` and a project install — and the npm 12 note now appears on every surface that shows an install command (README, SDK, agent-skill docs, and the shipped skill README) rather than the README alone, with the wording tightened.
+
+### Verified
+
+- Each printed command form measured on real installs in npm 12.0.2 containers: the global flag form builds the binding, the user-level config form covers global, project, and `pi install`, and the approve + rebuild flow repairs an already-blocked install.
+- 2938 unit tests over 233 files, lint, and type-check green.
+
+## [1.6.3] - 2026-08-25
+
+A unit test that had already been granted CI-jitter headroom twice went red a third
+time — and this time the timing signature (rejection 10008ms into a 10000ms window)
+identified a real race in the lock acquire loop rather than a slow runner.
+
+### Fixed
+
+- **A waiter gave up on a torn lock at the exact moment its reclaim window opened.** The acquire loop's reclaim decision measures the lock's age at `fs.stat` time, but the stall-check exemption that keeps a waiter alive until that reclaim ("never give up on a lock we are about to be allowed to take") compared a fresh `Date.now()` against the window. For the few milliseconds between the stat and the stall check the two clocks could disagree, and a retry tick landing astride the window boundary satisfied neither: not yet reclaimable by its stat, no longer awaited by the clock. Since a torn, pid-less lock never shows progress, the long-expired stall bound then fired, and a wait that had correctly outlasted it was abandoned right as the recovery became permitted — in production, a knowledge-store init failing at the moment it could have recovered. Both judgments now derive from the same observation, so every tick that can read the lock either reclaims it or keeps waiting; the try-take exclusion and the absolute ceiling are unchanged (`src/infrastructure/file-lock-service.ts`).
+- **The test's forensic comment blamed scheduler jitter for all three CI failures.** True for the first two (which failed at the stall bound); wrong for this one (which failed at the window boundary). The record now distinguishes the two mechanisms (`test/unit/infrastructure/file-lock-progress-aware-wait.test.ts`).
+- **A browser-config test deleted the whole suite's shared log directory as its own cleanup.** `unit-env.ts` routes every unit worker's log into `<tmpdir>/pi-research-test/`; the profile-containment test nested its scratch path inside that directory and removed the shared parent recursively — destroying the live log out from under every parallel worker, and racing the lazy logger re-creating the file mid-removal, which is an `ENOTEMPTY` crash of the cleanup itself (seen on macos-latest CI on this release's first tag). The test now creates its own `mkdtemp` directory and removes only that (`test/unit/infrastructure/browser-config.test.ts`).
+
+### Verified
+
+- 2938 unit tests over 233 files, lint, type-check, and the full CI matrix green on the fix commit — including the ubuntu-latest job that caught the failure.
+
 ## [1.6.2] - 2026-08-24
 
 An agent-skill run reported that every search had failed and concluded it had hit an

package/docs/SDK.md

@@ -13,6 +13,19 @@ run the `pi` extension, which keeps its own private copy your scripts can't impo
 npm install @lincoln504/pi-research
 ```
 
+On npm 12 or newer, allow the one install script search depends on before installing:
+
+```bash
+npm config set allow-scripts=better-sqlite3 --location=user
+```
+
+npm 12 blocks dependency install scripts by default, and a project-scoped install
+rejects the `--allow-scripts` flag — hence the config form. Skipped, the install
+succeeds but every search fails with a missing-module error. To repair an install
+that already ran blocked: `npm approve-scripts better-sqlite3` (pins the approval
+in your project's `allowScripts` field), then `npm rebuild better-sqlite3` —
+rerunning `npm install` does not rebuild the already-extracted package.
+
 Then pick the model: pass `model` to `initResearchSDK`, or set `PI_RESEARCH_MODEL`
 (env or `~/.pi/research/config.env`). The SDK never follows the model selected
 inside the pi extension; only when neither is set does it fall back to the first

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lincoln504/pi-research",
-  "version": "1.6.2",
+  "version": "1.6.4",
   "description": "Web research for your agents with smart and safe tooling + knowledge store",
   "type": "module",
   "private": false,

package/src/cli.ts

@@ -793,9 +793,10 @@ async function cmdStatus(json?: boolean): Promise<number> {
   const paths = resolvedConfigPaths();
   const cfg = getConfig(process.cwd(), 'cli');
   // The stealth browser is installed by this package's `postinstall`
-  // (scripts/setup.cjs), which npm 11.19+ BLOCKS BY DEFAULT — it now warns
-  // "packages have install scripts not yet covered by allowScripts" and carries
-  // on, so the browser is simply absent on an otherwise successful install.
+  // (scripts/setup.cjs), which npm 12 BLOCKS BY DEFAULT (11.16–11.19 only warn
+  // and still run it — measured per version against a scripted tarball) — npm
+  // prints "packages had install scripts blocked" and carries on, so the
+  // browser is simply absent on an otherwise successful install.
   // A missing browser DEGRADES capability rather than breaking the tool — the fetch
   // path still serves most pages — so it is reported alongside `ready` rather than
   // folded into it. isBrowserAvailable() is a resolve + existsSync; it never launches
@@ -827,14 +828,18 @@ async function cmdStatus(json?: boolean): Promise<number> {
       installed: browserInstalled,
       nativeDepsOk: nativeDeps.ok,
       nativeDepsError: nativeDeps.ok ? null : nativeDeps.error,
+      // --allow-scripts is a package-name LIST, not a boolean: the bare flag is
+      // accepted and silently allows nothing (measured on npm 12.0.2). It is
+      // also global/one-off only — a project-scoped install rejects it with
+      // EALLOWSCRIPTS, where `npm approve-scripts` is the supported route.
       nativeDepsFix: nativeDeps.ok
         ? null
-        : 'Reinstall allowing dependency install scripts: npm install -g @lincoln504/pi-research --allow-scripts (npm 12 turns them off by default).',
+        : 'Reinstall allowing the dependency install script: npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3 (npm 12 blocks dependency install scripts by default). For a project-scoped install: npm approve-scripts better-sqlite3, then npm rebuild better-sqlite3.',
       // Same remedy the health check prints, so both surfaces name one command.
       fix: browserInstalled ? null : 'npx camoufox-js fetch',
       note: browserInstalled
         ? null
-        : 'Stealth browser not installed — scraping falls back to plain fetch, which some sites block. npm 11.19+ blocks install scripts by default, which skips this package\'s postinstall.',
+        : 'Stealth browser not installed — scraping falls back to plain fetch, which some sites block. npm 12 blocks install scripts by default, which skips this package\'s postinstall; the engine is fetched on first use instead.',
     },
     knowledgeStoreMode: cfg.KNOWLEDGE_STORE_MODE,
     defaultDepth: cfg.DEFAULT_RESEARCH_DEPTH,

package/src/infrastructure/file-lock-service.ts

@@ -694,12 +694,13 @@ export class FileLockService implements IService {
             const errnoError = error as NodeJS.ErrnoException;
             if (errnoError.code === 'EEXIST') {
               contentionCount++;
-              // When the contended lock becomes eligible for reclaim, if this tick can
-              // know it. Scoped per observation: a tick that could not read the lock
+              // Whether THIS tick judged the contended lock not-yet-eligible for the
+              // pid-less reclaim — i.e. we are waiting for a recovery we will be allowed
+              // to perform. Scoped per observation: a tick that could not read the lock
               // learned nothing, and an eligibility carried over from an earlier one is
               // no longer evidence of anything. See the stall check below for why the
               // stall bound must not fire before this moment.
-              let reclaimEligibleAt: number | null = null;
+              let awaitingReclaim = false;
               // Read lock content to verify ownership and liveness before considering stale
               try {
                 const rawContent = await fs.readFile(this.lockFilePath, 'utf-8');
@@ -720,17 +721,26 @@ export class FileLockService implements IService {
                 // A lock with no readable pid is judged purely on age (see
                 // _shouldReclaim), and its mtime never moves again — so it can never
                 // show progress and the stall bound is guaranteed to expire first.
-                // Record when it becomes reclaimable so the wait below can outlast the
-                // stall bound rather than abandoning the recovery at 20s.
+                // Mark this tick as awaiting that reclaim so the wait below can outlast
+                // the stall bound rather than abandoning the recovery at 20s.
+                //
+                // Judged from the SAME lockAge observation _shouldReclaim will use, not
+                // from a fresh Date.now() at the stall check: the two clocks disagreed
+                // for the few ms between the stat and that check, and a tick landing
+                // astride the window boundary judged the lock not-yet-reclaimable here
+                // yet no-longer-awaited there — giving up at the exact moment the
+                // window opened (ubuntu CI, 2026-08-24: rejected 10008ms into a
+                // 10000ms window). The complement of _shouldReclaim's strict > means
+                // one of the two branches now holds on every readable tick.
                 //
                 // Deliberately NOT extended to a live owner: there the heartbeat is the
                 // designed progress signal, its absence past lockTimeout is real evidence
                 // of a wedge, and reporting that to the caller promptly beats waiting out
                 // a two-to-four-minute live-owner threshold on a lock we may well never
                 // be handed.
-                reclaimEligibleAt = (parsed?.pid ?? null) === null
-                  ? stats.mtimeMs + this._unidentifiedStaleThreshold()
-                  : null;
+                awaitingReclaim = this.lockTimeout > 0
+                  && (parsed?.pid ?? null) === null
+                  && lockAge <= this._unidentifiedStaleThreshold();
 
                 if (lockUuid === this.lockUuid) {
                   // This is our own lock — a previous attempt was interrupted. Reclaim.
@@ -802,8 +812,6 @@ export class FileLockService implements IService {
               // on the spot rather than wait for anything, and the run semaphore's slot
               // files carry a TEN-MINUTE unidentified window, so honouring eligibility
               // there would turn "is this slot free right now" into a ten-minute block.
-              const awaitingReclaim =
-                this.lockTimeout > 0 && reclaimEligibleAt !== null && Date.now() < reclaimEligibleAt;
               if ((stalledFor >= this.lockTimeout && !awaitingReclaim) || waitedFor >= this.acquireCeilingMs) {
                 metrics.observe('state_lock_acquire_duration_ms', waitedFor, { status: 'timeout' });
                 metrics.increment('state_lock_acquire_total', 1, { status: 'timeout' });

package/src/web-research/browser-search.ts

@@ -333,8 +333,9 @@ export async function performSearch(
                 `Search completely failed: ${reason}, all with a missing native module. ` +
                 `This is an incomplete install, not a network problem: the browser engine's ` +
                 `native dependencies were never built. npm 12 does not run dependency install ` +
-                `scripts by default — reinstall allowing them (npm install @lincoln504/pi-research ` +
-                `--allow-scripts, or npm approve-scripts --allow-scripts-pending), then retry.` +
+                `scripts by default — reinstall allowing the script (global install: ` +
+                `npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3; ` +
+                `project install: npm approve-scripts better-sqlite3, then npm rebuild better-sqlite3), then retry.` +
                 (sampleWorkerError ? ` Last worker error: ${sampleWorkerError}` : '')
             );
         }