@lincoln504/pi-research 1.3.11 → 1.4.1

package/docs/CONFIGURATION.md

@@ -74,12 +74,12 @@ Research
 | `PI_RESEARCH_MAX_RETRIES` | `2` | 0–5 | Retries per researcher request. |
 | `PI_RESEARCH_RETRY_DELAY_MS` | `2000` | 100–10000 | Base delay between retries. |
 | `PI_RESEARCH_MAX_FAILED_RESEARCHERS` | `2` | 1–10 | Unique researcher failures that abort the whole run. Raise to let slower, still-in-flight researchers finish before giving up. |
-| `PI_RESEARCH_WORKER_THREADS` | `4` | 1–10 | Browser worker processes. Higher = more throughput, more CPU/RAM. |
+| `PI_RESEARCH_WORKER_THREADS` | `4` | 1–10 | Browser worker processes. Higher = more throughput, more CPU/RAM. It sets how *fast* a search burst drains, not how large a burst may be — a burst larger than the pool waits its turn rather than being trimmed or timing out. |
 | `PI_RESEARCH_WORKER_CONCURRENCY` | `2` | 1–10 | Tasks per worker process. |
 | `PI_RESEARCH_MAX_CONCURRENT_RUNS` | `3` | ≥1 | Machine-wide cap on research runs executing at the same time, across **every** process (CLI, agent skill, pi extension, SDK). Runs beyond the cap queue rather than fail. All concurrent runs share one leader-elected browser/embedding pool, so oversubscribing it degrades every run at once. |
 | `PI_RESEARCH_RUN_ACQUIRE_TIMEOUT_MS` | `600000` | ≥0 | How long a run queues for a free slot before failing with "maximum concurrent research runs reached" (CLI exit `75`). `0` = fail immediately instead of queueing. |
-| `PI_RESEARCH_MODEL` | _(pi: session model; CLI/skill: required)_ | — | The model research runs on. **Required for the standalone CLI / agent skill** — they use only this configured model (never the model selected inside the pi extension) and refuse to start without one (the CLI's per-run `--model` flag also satisfies this). On the SDK it selects the session model when no `model` option is given. In the pi extension it overrides researcher sub-agents and knowledge synthesis, while the coordinator and evaluator keep using the session model. Accepts `provider/id` or a bare model id. |
-| `PI_RESEARCH_DISABLED_TOOLS` | _(none)_ | — | Comma-separated research tools to disable for a run (`search`, `scrape`, `security_search`, `stackexchange`, `youtube_transcript`, `grep`). Removed from every researcher's toolset and named in the coordinator/evaluator prompt. Strictly additive — it can only remove capabilities, never grant one, and it stacks on top of the default exclusions rather than replacing them. |
+| `PI_RESEARCH_MODEL` | _(pi: session model; CLI/skill: required)_ | — | The model research runs on. **Required for the standalone CLI / agent skill** — they use only this configured model (never the model selected inside the pi extension) and refuse to start without one (the CLI's per-run `--model` flag also satisfies this). On the SDK it selects the session model when no `model` option is given. In the pi extension it overrides researcher sub-agents and knowledge synthesis, while the coordinator and research lead keep using the session model. Accepts `provider/id` or a bare model id. |
+| `PI_RESEARCH_DISABLED_TOOLS` | _(none)_ | — | Comma-separated research tools to disable for a run (`search`, `scrape`, `security_search`, `stackexchange`, `youtube_transcript`, `grep`, `read`). Removed from every researcher's toolset and named in the coordinator and research-lead prompts. Strictly additive — it can only remove capabilities, never grant one, and it stacks on top of the default exclusions rather than replacing them. An unrecognized name excludes nothing and is warned about rather than failing the run. |
 | `PI_RESEARCH_REPORT_EXPORT_ENABLED` (TUI) | `false` | — | Front-ends write a Markdown report to disk and surface its path. |
 | `PI_RESEARCH_REPORT_EXPORT_DIR` | _(smart cwd)_ | — | Pin exported reports to a fixed directory, bypassing the cwd-relative resolution. Useful for the agent skill, which runs from the host agent's arbitrary directory. |
 | `PI_RESEARCH_MAX_SCRAPE_TOKEN_FRACTION_FOR_SCRAPING` | `0.15` | 0.05–1.0 | Max fraction of the context window used for initial scrape context. |
@@ -99,10 +99,10 @@ Timeouts
 
 | Variable | Default | Range | Description |
 |----------|---------|-------|-------------|
-| `PI_RESEARCH_LLM_TIMEOUT_MS` | `300000` | 60000–1800000 | Coordinator / evaluator / repair / knowledge LLM call timeout. |
+| `PI_RESEARCH_LLM_TIMEOUT_MS` | `300000` | 60000–1800000 | Coordinator / research-lead / repair / knowledge LLM call timeout. |
 | `PI_RESEARCH_SCRAPE_TIMEOUT_MS` | `15000` | 5000–120000 | Per-page scrape (page-load) timeout. |
 | `PI_RESEARCH_SEARCH_TIMEOUT_MS` | `45000` | 5000–120000 | Browser search page timeout. |
-| `PI_RESEARCH_BROWSER_TASK_TIMEOUT_MS` | `10000` | 2000–120000 | Queue-wait / overhead margin added to each browser op's own timeout (a search task ceiling is `SEARCH_TIMEOUT_MS` + this; a scrape is `SCRAPE_TIMEOUT_MS` + this). |
+| `PI_RESEARCH_BROWSER_TASK_TIMEOUT_MS` | `10000` | 2000–120000 | Overhead margin added to each browser op's own timeout (a search task ceiling is `SEARCH_TIMEOUT_MS` + this; a scrape is `SCRAPE_TIMEOUT_MS` + this). These ceilings bound **execution**: a task's clock starts when a worker picks it up, so time spent queued behind other work is not charged against it and raising this to cover a busy pool is neither needed nor effective. |
 | `PI_RESEARCH_HEALTH_CHECK_TIMEOUT_MS` | `10000` | 2000–120000 | Pre-flight health-check timeout. |
 
 LLM output & reasoning
@@ -111,8 +111,8 @@ These are advanced, env-only knobs (not in the TUI).
 
 | Variable | Default | Range | Description |
 |----------|---------|-------|-------------|
-| `PI_RESEARCH_LLM_THINKING_LEVEL` | `off` | off · minimal · low · medium · high | Chain-of-thought level for all engine LLM work (coordinator, evaluator, synthesis, JSON-repair, knowledge extraction, and researcher sub-agents). Off by default — these calls emit structured JSON / cited reports, so a thinking block only consumes the output budget and can truncate the answer. Clamped per model by pi. |
-| `PI_RESEARCH_PLANNING_MAX_TOKENS` | `16384` | 1024–131072 | Max output tokens for the plan + mid-round evaluator decision. Clamped to the model's real ceiling. |
+| `PI_RESEARCH_LLM_THINKING_LEVEL` | `off` | off · minimal · low · medium · high | Chain-of-thought level for all engine LLM work (coordinator, router, synthesizer, JSON-repair, knowledge extraction, and researcher sub-agents). Off by default — these calls emit structured JSON / cited reports, so a thinking block only consumes the output budget and can truncate the answer. Clamped per model by pi. |
+| `PI_RESEARCH_PLANNING_MAX_TOKENS` | `16384` | 1024–131072 | Max output tokens for the coordinator's plan. The router's decision has its own, smaller ceiling; the final synthesis uses `PI_RESEARCH_SYNTHESIS_MAX_TOKENS`. Clamped to the model's real ceiling. |
 | `PI_RESEARCH_SYNTHESIS_MAX_TOKENS` | `32768` | 1024–131072 | Max output tokens for the final synthesized report. Clamped to the model's real ceiling. |
 
 Knowledge store
@@ -156,7 +156,8 @@ Diagnostics & platform
 | `PI_RESEARCH_SKILL_DIR` | _(auto)_ | Override the bundled research-skill source directory used by the skill installer. |
 | `PI_RESEARCH_PURGE_BROWSERS` | _(unset)_ | Read by the bundled `scripts/cleanup.cjs`: set `1` to also delete the shared camoufox browser cache (kept by default because other installs may use it). Note npm ≥7 does not run `preuninstall`, so that script does not fire on `npm uninstall` — see [AGENT-SKILL.md](AGENT-SKILL.md). |
 | `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD` | _(unset)_ | Set `1` during `npm install` to skip the camoufox browser download (fetched lazily on first use instead; standard Playwright convention). |
-| `PLAYWRIGHT_BROWSERS_PATH` | _(user cache)_ | Standard Playwright variable, honoured here too: overrides where the camoufox binary is stored and looked up, and is passed through to browser workers. |
+| `CAMOUFOX_INSTALL_DIR` | _(user cache)_ | camoufox-js's own variable, and the only one that can relocate the browser. Sets where pi-research looks the binary up, and is exported to the postinstall fetch and to browser workers. **Not effective on the currently pinned camoufox-js (0.10.x), which hardcodes its cache directory and honours no variable** — so relocating the browser is unsupported until that pin can advance (see ARCHITECTURE.md). |
+| `PLAYWRIGHT_BROWSERS_PATH` | _(user cache)_ | Alias for the above, accepted for compatibility and exported onward as `CAMOUFOX_INSTALL_DIR`. Same caveat: on the pinned camoufox-js it moves only where pi-research looks, not where the download lands, so setting it reports the browser missing from the custom path after a successful install. |
 | `XDG_CACHE_HOME` | `~/.cache` | Standard XDG variable. When set, every `~/.cache/pi-research/...` path below is rooted at `$XDG_CACHE_HOME/pi-research/...` instead. |
 | `PI_RESEARCH_BIN` (alias `PI_RESEARCH_PATH`) | _(auto)_ | Agent-skill launcher only: explicit path to the pi-research engine binary when auto-resolution (PATH → local install → npx) should be bypassed. See [skills/pi-research/references/configuration.md](../skills/pi-research/references/configuration.md). |
 | `PLAYWRIGHT_INSTALL_DEPS` | _(unset)_ | Linux only. Set `true` during `npm install` to also install system libraries via `npx playwright install-deps` (same as `npm run install:system-deps`). |
@@ -238,6 +239,105 @@ just export the variable for that process.
 > `/research-config` TUI (which writes it), export variables in your shell, or use a
 > loader such as direnv. `.env.example` is a reference, not an active config file.
 
+### Prompt caching
+
+A research run re-sends a large, mostly-unchanged prompt many times: a researcher
+re-sends its whole conversation (including every scraped page) on each tool turn, and
+the research lead used to re-send the accumulated findings on every round. Every provider will
+serve that repetition from a prompt cache at a fraction of the input price — but only
+for an **exact prefix** of the request, and only if nothing that varies sits in front
+of it.
+
+What pi-research does about it. The research lead is split into two roles so that the
+repetition is removed rather than merely discounted. The **router** decides each round
+whether to continue, reading each report in full on the one round it arrives and only its
+short coverage digest thereafter, instead of re-reading every report collected so far. The **synthesizer** runs once, at the end, and is the only
+call that reads the reports in full. Before the split, one call did both jobs and re-sent
+the whole corpus every round, so its input grew with the square of the round count. The
+synthesizer's corpus is also budgeted against the model's context window: over budget, the
+reports are reduced in partial passes and merged rather than truncated or rejected.
+
+How often the router runs depends on the round budget. Depth 2 and 3 route on every round
+but the last. Depth 1's base budget is two rounds and the last one is skipped, so a plain
+depth-1 run never routes — but steering raises the budget (up to two extra rounds), and a
+steered depth-1 run does route. That is also the depth-1 case carrying the most findings,
+so it is where routing on digests rather than on the full corpus matters most.
+
+On top of that, both lead prompts are laid out stable-part-first: they interpolate only
+values fixed for the whole run (complexity, team size, query budget, disabled tools), and
+everything that changes between rounds — root query, round number, agenda, executed
+queries, steering, round-phase guidance — is appended afterwards as a `RUN CONTEXT`
+block. Unit tests hold the layout in place; if you edit
+`src/prompts/system-lead-router.md` or `src/prompts/system-lead-synthesizer.md`, keep
+run-varying text out of them.
+
+What your provider does about it depends on which API pi talks to (the provider's `api`
+field in `~/.pi/agent/models.json`) and, on OpenAI-compatible endpoints, on one `compat`
+key:
+
+| Provider API | Behaviour |
+|--------------|-----------|
+| `anthropic-messages` | pi inserts `cache_control` breakpoints automatically — on the system prompt, the last tool definition, and the last user/assistant/tool-result block. Nothing to configure. |
+| `openai-completions` | pi inserts the **same** Anthropic-style breakpoints into the OpenAI-shaped payload whenever `compat.cacheControlFormat` is `"anthropic"`. Auto-detection sets that key for exactly one case — OpenRouter routing to an `anthropic/*` model — so by default no markers are sent and the provider's own implicit prefix caching is relied on. That default is correct for OpenAI, DeepSeek, Gemini and GLM, which all cache implicitly. |
+| `openai-responses` | No markers at any setting; implicit caching only. |
+
+Note that the marker behaviour is gated on `cacheControlFormat`, **not** on the `api`
+field — setting the key makes an `openai-completions` provider emit explicit breakpoints
+exactly like an `anthropic-messages` one. That matters when reading the counters below:
+on a provider you have configured this way, a zero cache read is not evidence that
+markers are missing.
+
+The gap is providers that need explicit markers but are reached over an OpenAI-compatible
+endpoint and are not the one auto-detected case. Anything in that category (Qwen through
+OpenRouter, or any gateway fronting Claude) caches **nothing at all**, silently and with
+no error. Fix it per provider with a `compat` block:
+
+```json
+{
+  "providers": {
+    "openrouter": {
+      "api": "openai-completions",
+      "compat": {
+        "cacheControlFormat": "anthropic",
+        "sessionAffinityFormat": "openrouter",
+        "sendSessionAffinityHeaders": true
+      }
+    }
+  }
+}
+```
+
+`cacheControlFormat: "anthropic"` forces the markers on. It is safe to set for a whole
+provider: models that do not use markers ignore the extra field. The two session-affinity
+keys ask OpenRouter to keep a conversation on the replica that warmed its cache; without
+them OpenRouter falls back to hashing the first messages, which an agentic loop mutates.
+
+`PI_CACHE_RETENTION=long` (a pi variable, not a pi-research one) requests 1-hour
+retention where the provider supports it. It raises the cache-write multiplier from
+1.25x to 2x, so it only pays off when a run's gaps between calls exceed the provider's
+default window — nominally 5 minutes.
+
+Measure before enabling it. On a direct probe against GLM over `anthropic-messages`, an
+entry was still read back unchanged after **seven minutes at the default retention**, and
+`long` produced an identical number — so on that route the doubled write multiplier buys
+nothing. The nominal 5-minute figure is a floor providers are free to exceed, not a
+deadline you can plan around.
+
+Verifying it works. Set `PI_RESEARCH_DEBUG=true` and read the run log: every LLM call
+records `llm_cache_read_tokens_total` and `llm_cache_write_tokens_total` alongside
+`llm_tokens_total`, labelled by component (`coordinator` / `router` / `synthesizer` /
+`researcher`).
+Cache reads that stay at zero on a multi-turn researcher mean the prefix is not being
+matched — usually a provider with a minimum cacheable prefix (commonly 1024–4096 tokens)
+that short prompts never reach, or an explicit-marker provider missing the `compat` block
+above. A counter that is absent rather than zero means the provider does not report
+caching at all.
+
+Cache **writes** are the weaker signal of the two. Several providers report reads and
+omit writes entirely — OpenRouter and Z.ai's Anthropic-compatible endpoint both return
+non-zero `cache_read` alongside a flat zero `cache_write` — so a zero there is normal and
+says nothing about whether caching is working. Judge it by the read counter.
+
 ### Where files live
 
 All pi-research state lives under its own namespace, `~/.pi/research/`:

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@lincoln504/pi-research",
-  "version": "1.3.11",
-  "description": "Free unlimited web search and deep research for the Pi coding agent. Local knowledge store. Also works from Claude Code, Codex, and OpenClaw.",
+  "version": "1.4.1",
+  "description": "Web research for your agents with smart and safe tooling + knowledge store",
   "type": "module",
   "private": false,
   "main": "./src/index.ts",
@@ -53,19 +53,27 @@
     "pi-package",
     "pi-extension",
     "pi-coding-agent",
+    "claude-code",
     "openclaw",
+    "codex",
+    "agent-skills",
     "agent-skill",
     "skills",
-    "research",
+    "web-search",
     "web-research",
     "deep-research",
-    "agent",
-    "web-search",
+    "ai-research",
+    "research",
+    "web-scraping",
     "scraping",
-    "knowledge-store",
-    "duckduckgo",
+    "browser-automation",
     "stealth-browser",
-    "camoufox"
+    "camoufox",
+    "duckduckgo",
+    "knowledge-store",
+    "llm-tools",
+    "web-access",
+    "agent"
   ],
   "author": "Lincoln Deen",
   "contributors": [
@@ -120,7 +128,7 @@
     "@earendil-works/pi-tui": ">=0.84.0 <1",
     "@huggingface/transformers": "4.2.0",
     "@kreuzberg/html-to-markdown-node": "^3.5.5",
-    "@lancedb/lancedb": "0.29.0",
+    "@lancedb/lancedb": "0.37.1",
     "apache-arrow": "21.1.0",
     "bgutils-js": "^3.2.0",
     "camoufox-js": "^0.10.2",
@@ -133,7 +141,7 @@
     "poolifier": "^5.3.2",
     "typebox": "1.3.7",
     "undici": "^8.9.0",
-    "youtubei.js": "^17.2.0"
+    "youtubei.js": "^18.0.0"
   },
   "optionalDependencies": {
     "webgpu": "^0.4.0"
@@ -154,7 +162,7 @@
   },
   "devDependencies": {
     "@eslint/js": "^10.0.1",
-    "@types/jsdom": "^28.0.3",
+    "@types/jsdom": "^30.0.0",
     "@typescript-eslint/eslint-plugin": "^8.65.0",
     "@typescript-eslint/parser": "^8.65.0",
     "@vitest/coverage-v8": "^4.1.10",

package/scripts/build.cjs

@@ -87,9 +87,16 @@ function copyThreadWorker() {
 // 'prompts')), so the templates must sit next to the bundle.
 function copyPrompts() {
   fs.mkdirSync(p('dist', 'prompts'), { recursive: true });
-  for (const f of fs.readdirSync(p('src', 'prompts'))) {
+  const sources = fs.readdirSync(p('src', 'prompts'));
+  for (const f of sources) {
     fs.copyFileSync(p('src', 'prompts', f), p('dist', 'prompts', f));
   }
+  // Prune prompts that no longer exist in src. Copying alone leaves a deleted or renamed
+  // prompt behind in an incremental dist forever; verify-package rejects the orphan at
+  // publish time, which surfaces as a confusing failure long after the rename.
+  for (const f of fs.readdirSync(p('dist', 'prompts'))) {
+    if (!sources.includes(f)) fs.rmSync(p('dist', 'prompts', f));
+  }
 }
 
 const TARGETS = {

package/scripts/cleanup.cjs

@@ -197,13 +197,34 @@ const dirName = rawDirName && !rawDirName.includes('/') && !rawDirName.includes(
  * several tools' state by name); a substring match would treat that shared
  * folder as exclusively ours and wholesale-delete it. Only the exact default
  * layout is something WE constructed and can therefore prove is ours.
+ *
+ * Segment-adjacency is NOT that proof either, and used to be what this did:
+ * it tested whether `<dirName>/research` appeared ANYWHERE in the path, with no
+ * constraint on what followed. That also matched:
+ *   - the research ROOT itself (`~/.pi/research`) — which holds config.env and
+ *     knowledge_db/, the two things this script's header promises to preserve;
+ *   - any unrelated directory beneath it (`/mnt/backup/.pi/research/mystate`);
+ *   - traversal forms (`~/.pi/research/../../..`) that resolve far outside our
+ *     tree entirely.
+ *
+ * The real proof is the full default *shape*: the runtime only ever builds
+ * `<root>/<dirName>/research/state`. Requiring those three trailing segments on
+ * the RESOLVED path keeps a relocated-but-default layout reclaimable while
+ * refusing the research root, sibling directories, and traversal escapes.
  */
+const DEFAULT_STATE_TAIL = [dirName, 'research', 'state'];
+
 function isPiResearchNamespaced(p) {
-  const segs = p.split(/[\\/]+/).filter(Boolean);
-  for (let i = 0; i < segs.length - 1; i++) {
-    if (segs[i] === dirName && segs[i + 1] === 'research') return true;
-  }
-  return false;
+  // Resolve first so `..` cannot smuggle the tail past this check.
+  const resolved = path.resolve(p);
+  // Windows and macOS default to case-insensitive filesystems, so a path
+  // differing only in case still names the directory we built.
+  const fold = (s) =>
+    process.platform === 'win32' || process.platform === 'darwin' ? s.toLowerCase() : s;
+  const segs = resolved.split(/[\\/]+/).filter(Boolean);
+  if (segs.length < DEFAULT_STATE_TAIL.length) return false;
+  const tail = segs.slice(-DEFAULT_STATE_TAIL.length);
+  return DEFAULT_STATE_TAIL.every((want, i) => fold(tail[i]) === fold(want));
 }
 
 /**

package/scripts/setup.cjs

@@ -47,8 +47,25 @@ function resolveCamoufoxBin() {
   return null;
 }
 
+/**
+ * The custom camoufox location, if the user set one. Mirrors
+ * getCustomCamoufoxDir() in src/infrastructure/browser/config.ts — the two must
+ * agree or install and lookup land in different directories.
+ *
+ * CAMOUFOX_INSTALL_DIR first: it is camoufox-js's own variable and the only one
+ * that can relocate the install (camoufox-js <0.12.0 hardcodes
+ * userCacheDir("camoufox") and honours nothing). PLAYWRIGHT_BROWSERS_PATH is
+ * kept as the documented alias and is exported BELOW as CAMOUFOX_INSTALL_DIR.
+ * NOTE: inert on the pinned camoufox-js 0.10.x, which reads neither — this is
+ * forward-compatible plumbing, not a live fix. See src/infrastructure/browser/
+ * config.ts and thread-worker-browser.ts.
+ */
+function customCamoufoxDir() {
+  return process.env.CAMOUFOX_INSTALL_DIR || process.env.PLAYWRIGHT_BROWSERS_PATH || null;
+}
+
 function camoufoxCachePath() {
-  const customPath = process.env.PLAYWRIGHT_BROWSERS_PATH;
+  const customPath = customCamoufoxDir();
   if (customPath) return customPath;
 
   if (isWindows) {
@@ -68,6 +85,13 @@ if (process.env.PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD === '1') {
   console.log('pi-research: skipping browser download (PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1)');
 } else {
   const env = { ...process.env };
+  // Make the fetch honour the custom location. Without this the download always
+  // went to camoufox's default cache while camoufoxCachePath() reported the
+  // custom one: the "already installed" probe never matched, every install
+  // re-fetched, and at runtime the browser was reported missing despite having
+  // downloaded successfully.
+  const customDir = customCamoufoxDir();
+  if (customDir) env.CAMOUFOX_INSTALL_DIR = customDir;
 
   const installDeps = process.argv.includes('--system-deps') || process.env.PLAYWRIGHT_INSTALL_DEPS === 'true';
   if (installDeps && isLinux) {

package/skills/pi-research/SKILL.md

@@ -2,7 +2,7 @@
 name: pi-research
 description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
 license: MIT
-metadata: { "author": "Lincoln Deen", "version": "1.3.11", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
+metadata: { "author": "Lincoln Deen", "version": "1.4.1", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
 allowed-tools: Bash(node:*)
 ---
 

package/src/cli.ts

@@ -45,6 +45,7 @@ import { getConfig, getGlobalConfigDir, getGlobalEnvFilePath, getInterfaceEnvFil
 import { getAgentDir } from '@earendil-works/pi-coding-agent';
 import { buildModelRegistry, safeGetAvailable } from './core/llm/model-registry-factory.ts';
 import { RESEARCH_TOOL_NAMES } from './constants.ts';
+import { isBrowserAvailable } from './infrastructure/browser/config.ts';
 
 // ---------------------------------------------------------------------------
 // Constants
@@ -167,7 +168,11 @@ function makeProgressObserver(): HeadlessObserverOptions {
           toStderr(`  • round ${data?.round}\n`);
           break;
         case 'search_start':
-          toStderr(`  • search burst: ${data?.queries?.length ?? 0} query/${(data?.queries?.length ?? 0) === 1 ? '' : 's'}\n`);
+          // "N queries", not "N query/s" — the old `query/${n===1?'':'s'}` form rendered a
+          // COUNT as what reads unmistakably as a RATE ("45 query/s"), so a burst that was
+          // in fact 45 queries looked like 45 per second and then appeared frozen. It also
+          // emitted a bare trailing slash ("1 query/") in the singular.
+          toStderr(`  • search burst: ${data?.queries?.length ?? 0} ${(data?.queries?.length ?? 0) === 1 ? 'query' : 'queries'}\n`);
           break;
         case 'researcher_start':
           toStderr(`  • researcher ${data?.id}: ${data?.name}\n`);
@@ -770,6 +775,18 @@ async function cmdStatus(json?: boolean): Promise<number> {
   const det = await detectCredentials();
   const paths = resolvedConfigPaths();
   const cfg = getConfig(process.cwd(), 'cli');
+  // The stealth browser is installed by this package's `postinstall`
+  // (scripts/setup.cjs), which npm 11.19+ BLOCKS BY DEFAULT — it now warns
+  // "packages have install scripts not yet covered by allowScripts" and carries
+  // on, so the browser is simply absent on an otherwise successful install.
+  // Every other native artifact survives that (onnxruntime, sharp, esbuild and
+  // webgpu all ship prebuilt platform packages), which is exactly what makes
+  // this one worth reporting: nothing else fails, and without this the first
+  // symptom is a scrape falling back mid-run. A missing browser degrades
+  // capability rather than breaking the tool — the fetch path still serves most
+  // pages — so it is reported alongside `ready` rather than folded into it.
+  // isBrowserAvailable() is a resolve + existsSync; it never launches anything.
+  const browserInstalled = isBrowserAvailable();
   const summary = {
     package: '@lincoln504/pi-research',
     version: PKG_VERSION,
@@ -783,6 +800,14 @@ async function cmdStatus(json?: boolean): Promise<number> {
       modelFrom: det.modelFrom,
       problem: det.problem ?? null,
     },
+    browser: {
+      installed: browserInstalled,
+      // Same remedy the health check prints, so both surfaces name one command.
+      fix: browserInstalled ? null : 'npx camoufox-js fetch',
+      note: browserInstalled
+        ? null
+        : 'Stealth browser not installed — scraping falls back to plain fetch, which some sites block. npm 11.19+ blocks install scripts by default, which skips this package\'s postinstall.',
+    },
     knowledgeStoreMode: cfg.KNOWLEDGE_STORE_MODE,
     defaultDepth: cfg.DEFAULT_RESEARCH_DEPTH,
     paths,
@@ -796,6 +821,10 @@ async function cmdStatus(json?: boolean): Promise<number> {
   toStdout(`pi-research ${PKG_VERSION}\n`);
   toStdout(`ready: ${summary.ready ? 'yes' : 'no'}\n\n`);
   toStdout(configBlock(det, summary.ready ? undefined : '— fix the problem above, then re-run.') + '\n');
+  if (!summary.browser.installed) {
+    toStdout(`\nstealth browser: NOT INSTALLED — run: ${summary.browser.fix}\n`);
+    toStdout(`  ${summary.browser.note}\n`);
+  }
   toStdout(`\nknowledge store mode: ${cfg.KNOWLEDGE_STORE_MODE}   (default depth: ${cfg.DEFAULT_RESEARCH_DEPTH})\n`);
   return EXIT.OK;
 }
@@ -1064,7 +1093,17 @@ export async function reportError(err: unknown, what: string, json?: boolean): P
   // --json: emit a structured error on stdout so a machine consumer never has to
   // special-case plain-text stderr. Mirrors the { ok: true, ... } success shape.
   if (json) {
-    const payload: Record<string, unknown> = { ok: false, error: msg, exitCode };
+    // A cancellation reports itself as a cancellation, not as whatever component
+    // happened to notice the abort first. The internal message is genuinely
+    // misleading here — a Ctrl-C unwinding through the planner surfaces as
+    // "Coordinator failed: Request was aborted", so a consumer relaying `error`
+    // verbatim (which SKILL.md instructs agents to do) tells the user a component
+    // broke when they simply stopped the run. The plain-text branch below already
+    // refuses to say "failed" for a cancel; JSON said it anyway. The original
+    // wording is preserved under `cause` so nothing diagnostic is lost.
+    const payload: Record<string, unknown> = isCancellation
+      ? { ok: false, error: `pi-research ${what} cancelled.`, cause: msg, exitCode }
+      : { ok: false, error: msg, exitCode };
     // Machine-readable "this will succeed later" flag, so a consumer can back off
     // and retry without string-matching the human message. Gated on cancellation:
     // a Ctrl-C landing in the same window a queue-wait expires must never emit

package/src/config.ts

@@ -88,7 +88,7 @@ export const ConfigSchema = Type.Object({
    *  never killed before it has used its full nav budget plus time waiting in the queue.
    *  (default: 10000) */
   BROWSER_TASK_TIMEOUT_MS: Type.Number({ minimum: 2000, maximum: 120000, default: 10000 }),
-  /** Timeout for coordinator/evaluator/repair/knowledge LLM calls in ms (default: 300000 = 5 min, range: 60s-30min).
+  /** Timeout for coordinator/research-lead/repair/knowledge LLM calls in ms (default: 300000 = 5 min, range: 60s-30min).
    *  Not exposed in TUI — controlled via PI_RESEARCH_LLM_TIMEOUT_MS env var.
    *  Ceiling raised to 30min (matching RESEARCHER_TIMEOUT_MS): a slow model on a
    *  large-context synthesis/plan call can legitimately run >10min, and the old
@@ -96,7 +96,7 @@ export const ConfigSchema = Type.Object({
    *  (e.g. 900000), causing those calls to time out mid-generation. */
   LLM_TIMEOUT_MS: Type.Number({ minimum: 60000, maximum: 1800000, default: 300000 }),
   /** Chain-of-thought "thinking" level for the engine's own LLM calls (coordinator,
-   *  evaluator, synthesis, JSON-repair, knowledge extraction) AND the researcher
+   *  router, synthesizer, JSON-repair, knowledge extraction) AND the researcher
    *  sub-agents. Default 'off': these calls emit structured JSON / cited reports, not
    *  open-ended reasoning, so thinking only burns the output-token budget (often
    *  truncating before the JSON/report block) and adds latency for at most a marginal
@@ -111,22 +111,26 @@ export const ConfigSchema = Type.Object({
     Type.Literal('medium'),
     Type.Literal('high'),
   ], { default: 'off' }),
-  /** Max output tokens for the coordinator plan + mid-round evaluator decision (and,
-   *  when the evaluator synthesizes mid-round, the report). Default 16384. The blanket
+  /** Max output tokens for the coordinator's round-1 plan. Default 16384. The routing
+   *  decision has its OWN, much smaller ceiling (it emits a decision, never a report,
+   *  and cannot synthesize mid-round under the split protocol); the final report uses
+   *  SYNTHESIS_MAX_TOKENS below. The blanket
    *  4096 cap previously throttled every call and could be exhausted by a thinking block
    *  before any text was emitted. Capped at the model's real ceiling at call time.
    *  Not exposed in TUI — controlled via PI_RESEARCH_PLANNING_MAX_TOKENS env var. */
   PLANNING_MAX_TOKENS: Type.Number({ minimum: 1024, maximum: 131072, default: 16384 }),
-  /** Max output tokens for the final synthesized research report (the forced-synthesis
-   *  evaluator call). Default 32768 so a full cited report is never truncated. Capped at
+  /** Max output tokens for the final synthesized research report (the synthesizer call,
+   *  and each pass of a reduce when the corpus does not fit in one). Default 32768 so a
+   *  full cited report is never truncated. It is also the reserve subtracted from the
+   *  model's context window when budgeting that corpus. Capped at
    *  the model's real maxTokens at call time.
    *  Not exposed in TUI — controlled via PI_RESEARCH_SYNTHESIS_MAX_TOKENS env var. */
   SYNTHESIS_MAX_TOKENS: Type.Number({ minimum: 1024, maximum: 131072, default: 32768 }),
   /** LLM Model override for researcher sub-agents and knowledge synthesis.
    *  Format: provider/model-id (e.g. google/gemini-2.0-flash-001) or just model-id.
    *  When set, this overrides ctx.model for researcher sub-agents (both deep and quick)
-   *  and the knowledge synthesis background LLM. The coordinator and evaluator always
-   *  use the caller's model (ctx.model).
+   *  and the knowledge synthesis background LLM. The coordinator and the research lead
+   *  always use the caller's model (ctx.model).
    */
   RESEARCH_MODEL: Type.Optional(Type.String()),
   /** Explicit directory for the knowledge store database (overrides default) */

package/src/constants.ts

@@ -69,7 +69,7 @@ export function getUnitsPerResearcher(config?: Config): number {
   return 1 + Math.min(batches, 4);
 }
 
-/** Progress bar weight per lead evaluator round */
+/** Progress bar weight per research-lead round */
 export const LEAD_EVAL_UNITS = 2;
 
 

package/src/core/interfaces/observer-interfaces.ts

@@ -58,9 +58,9 @@ export interface ResearchObserver {
   // Synthesis/Completion
   /**
    * Fired when the run enters FINAL synthesis / stops consuming steering:
-   *   - deep mode: right before the forced (mustSynthesize) evaluator call, and
-   *     on the evaluator-chose-synthesize path (idempotent there — the flag is
-   *     already off via onEvaluationDecision('synthesize')).
+   *   - deep mode: right before the synthesizer call, and on the path where the
+   *     router chose to finish early (idempotent there — the flag is already off
+   *     via onEvaluationDecision('synthesize')).
    *   - quick mode: as soon as the researcher session settles, i.e. the point
    *     the 500ms steering-consume poller stops running.
    * TUI consequence: steeringAcceptable flips false, so a late steer takes the

package/src/core/interfaces/orchestration-interfaces.ts

@@ -59,6 +59,8 @@ export interface RunResearchersOptions {
 export interface IResearchSynthesisService extends IService {
   storeReport(researchId: string, researcherId: string, report: string): void;
   getAllReports(researchId: string): Map<string, string>;
+  /** Coverage digest per stored report, derived when a researcher emitted none. */
+  getAllDigests(researchId: string): Map<string, string>;
   getReport(researchId: string, id: string): string | undefined;
   getReportsForRound(researchId: string, round: number): Map<string, string>;
   getReportCount(researchId: string): number;

package/src/core/interfaces/planning-interfaces.ts

@@ -56,6 +56,12 @@ export interface GenerateQueriesOptions {
 export interface UpdatePlanOptions {
   sessionId: string;
   reports: Map<string, string>;
+  /** Coverage digest per report id. The router reads the FRESH round's reports in full
+   *  (evidence-based routing) and falls back to these digests only for PRIOR rounds — it
+   *  does not read digests instead of the current round's reports. Optional: when omitted,
+   *  digests are derived from the report bodies so a caller that doesn't track them still
+   *  routes on coverage rather than on nothing. */
+  digests?: Map<string, string>;
   round: number;
   query: string;
   complexity: 1 | 2 | 3;
@@ -98,7 +104,7 @@ export interface IPlanningService extends IService {
   getTeamSize(complexity: 1 | 2 | 3): number;
   getQueryBudget(complexity: 1 | 2 | 3): number;
   getComplexityGuidance(complexity: 1 | 2 | 3, maxTeamSize: number, queryBudget: number): string;
-  getEvaluatorComplexityGuidance(complexity: 1 | 2 | 3): string;
+  getEvaluatorComplexityGuidance(complexity: 1 | 2 | 3, researchRounds: number): string;
   getRoundPhaseGuidance(currentRound: number, maxRounds: number, complexity: 1 | 2 | 3, maxTeamSize: number): string;
   capResearcherQueries(plan: ResearchPlan, complexity: 1 | 2 | 3, serviceName: string): ResearchPlan;
   parseJsonPlan(text: string): ResearchPlan;

package/src/core/interfaces/scheduler-interfaces.ts

@@ -70,9 +70,25 @@ export interface ScrapeResult {
 /**
  * Base scheduler interface
  */
+/**
+ * Reports when a task's EXECUTION starts and stops.
+ *
+ * Callers that keep their own guard over a scheduler call arm it on 'dispatched'
+ * rather than at call time, so queue wait never counts against the work's budget,
+ * and disarm it on 'settled' so the guard is not left running over a period in
+ * which nothing is executing — between a failed attempt and its retry, that
+ * period includes a backoff sleep, a re-enqueue and a fresh queue wait, and a
+ * timer left armed across it kills the retry for the predecessor's overrun.
+ *
+ * An implementation that cannot observe dispatch (BrowserClient, which only sees
+ * the leader's final reply) never calls it at all, and the caller's guard stays
+ * disarmed — that implementation bounds the call itself.
+ */
+export type TaskDispatchListener = (event: 'dispatched' | 'settled') => void;
+
 export interface IScheduler {
-  runSearch(query: string, config?: Config, signal?: AbortSignal): Promise<SearchResult[]>;
-  runScrape(url: string, config?: Config, signal?: AbortSignal): Promise<ScrapeResult>;
+  runSearch(query: string, config?: Config, signal?: AbortSignal, onTaskEvent?: TaskDispatchListener): Promise<SearchResult[]>;
+  runScrape(url: string, config?: Config, signal?: AbortSignal, onTaskEvent?: TaskDispatchListener): Promise<ScrapeResult>;
   runHealthCheck(config?: Config, signal?: AbortSignal): Promise<{ success: boolean }>;
   shutdown(): Promise<void>;
   resetIdleTimerOnActivity?(): void;

package/src/core/llm/agentic-repair.ts

@@ -34,6 +34,10 @@ export interface JsonRepairOptions {
   maxTokens?: number;
   /** Thinking level for the repair pass (default 'off' — repair emits JSON, not reasoning). */
   thinkingLevel?: ModelThinkingLevel;
+  /** Research session id, forwarded to the provider as its prompt-cache / session-affinity
+   *  key (see llm-utils.buildSafeOptions). Keeps the repair pass on the same upstream
+   *  replica as the call it is repairing. */
+  sessionId?: string;
   /** Usage sink invoked once per billed LLM attempt with the raw usage object from
    *  the response. Lets callers attribute the repair pass's tokens/cost to the run
    *  (metrics + observer) exactly as they do for the primary call. Every attempt that
@@ -70,7 +74,7 @@ export async function repairJsonWithLlm<T = any>(
   auth: { apiKey: string; headers?: Record<string, string | null> },
   options: JsonRepairOptions
 ): Promise<T | null> {
-  const { model, context, schema, serviceName = 'RepairService', signal, maxTokens = 16384, thinkingLevel = 'off' } = options;
+  const { model, context, schema, serviceName = 'RepairService', signal, maxTokens = 16384, thinkingLevel = 'off', sessionId } = options;
   
   logger.warn(`[${serviceName}] JSON parse failed; attempting agentic salvage`);
 
@@ -118,7 +122,8 @@ Return ONLY the valid JSON object. No prose before or after.`;
           ],
         }, buildSafeOptions(model, {
           ...auth,
-          signal
+          signal,
+          ...(sessionId ? { sessionId } : {}),
         }, maxTokens, thinkingLevel)),
         llmTimeout, `agentic-repair-${serviceName}`,
       );

package/src/core/llm/inject-date.ts

@@ -35,7 +35,10 @@ function getCurrentDateString(): string {
  * training-cutoff defaults). The full date follows for precision. Applies to
  * coordinator, researcher, and evaluator across all modes.
  */
-export function injectCurrentDate(prompt: string, _agentType: 'coordinator' | 'researcher' | 'evaluator'): string {
+export function injectCurrentDate(
+  prompt: string,
+  _agentType: 'coordinator' | 'researcher' | 'router' | 'synthesizer',
+): string {
   const monthYear = getCurrentMonthYearString();
   const fullDate = getCurrentDateString();
   // Derive the stale-year examples from the current year so the guidance stays correct in any

package/src/core/llm/llm-utils.ts

@@ -22,6 +22,13 @@ import { logger } from '../../logger.ts';
  *   thinking block only consumes the output-token budget (often truncating before the
  *   text block is emitted) for at most a marginal quality gain that does not justify the cost.
  *
+ * Callers should also pass the research `sessionId` in `options`. pi-ai forwards it as the
+ * provider's prompt-cache / session-affinity key — OpenAI's `prompt_cache_key`, OpenRouter's
+ * `x-session-id` — which keeps a run's calls on the replica that warmed their cache. Without
+ * it OpenRouter falls back to hashing the first system and user messages, which every round
+ * of a research run mutates, so it re-routes and re-warms. Providers that do not implement
+ * session affinity ignore the field, so passing it is unconditionally safe.
+ *
  * @param model - The model being called
  * @param options - Caller-provided options (an explicit `reasoning` here wins)
  * @param defaultCap - Default maxTokens cap (clamped to model.maxTokens)