@lincoln504/pi-research 1.6.5 → 1.6.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/README.md +14 -9
  2. package/agent-skill/pi-research/README.md +5 -4
  3. package/agent-skill/pi-research/SKILL.md +1 -1
  4. package/dist/cli.mjs +616 -153
  5. package/dist/pdf-extract-worker.mjs +1289 -0
  6. package/dist/prompts/research-tool-usage.md +2 -1
  7. package/dist/prompts/researcher.md +1 -1
  8. package/dist/prompts/system-coordinator.md +2 -2
  9. package/dist/prompts/system-lead-router.md +1 -1
  10. package/docs/AGENT-SKILL.md +6 -6
  11. package/docs/ARCHITECTURE.md +24 -22
  12. package/docs/CHANGELOG.md +73 -0
  13. package/docs/CONFIGURATION.md +3 -2
  14. package/docs/SDK.md +7 -12
  15. package/package.json +8 -5
  16. package/scripts/build.cjs +35 -0
  17. package/scripts/setup.cjs +12 -10
  18. package/src/cleanup/research-cleanup.ts +11 -5
  19. package/src/cli.ts +7 -5
  20. package/src/config.ts +9 -3
  21. package/src/core/interfaces/knowledge-interfaces.ts +8 -0
  22. package/src/core/interfaces/orchestration-interfaces.ts +1 -1
  23. package/src/core/interfaces/research-plan-types.ts +6 -0
  24. package/src/core/llm/agentic-repair.ts +40 -3
  25. package/src/core/llm/llm-utils.ts +93 -5
  26. package/src/core/llm/model-registry-factory.ts +2 -15
  27. package/src/core/pi-version.ts +1 -1
  28. package/src/core/planning-service.ts +110 -15
  29. package/src/core/planning-utils.ts +3 -0
  30. package/src/core/service-registry.ts +8 -6
  31. package/src/index.ts +8 -5
  32. package/src/infrastructure/browser/config.ts +6 -7
  33. package/src/infrastructure/browser/ensure-browser.ts +14 -7
  34. package/src/infrastructure/browser/thread-worker-browser.ts +18 -19
  35. package/src/knowledge/chunker.ts +22 -4
  36. package/src/knowledge/embedder-init.ts +41 -1
  37. package/src/knowledge/embedder-utils.ts +7 -1
  38. package/src/knowledge/store.ts +68 -1
  39. package/src/knowledge/transformers-loader.ts +58 -1
  40. package/src/observers/research-observer-impl.ts +4 -1
  41. package/src/orchestration/deep-research-orchestrator.ts +3 -3
  42. package/src/orchestration/headless-observer.ts +33 -0
  43. package/src/orchestration/quick-research-orchestrator.ts +17 -3
  44. package/src/orchestration/research-orchestration-service.ts +27 -4
  45. package/src/orchestration/session-state.ts +36 -7
  46. package/src/prompts/research-tool-usage.md +2 -1
  47. package/src/prompts/researcher.md +1 -1
  48. package/src/prompts/system-coordinator.md +2 -2
  49. package/src/prompts/system-lead-router.md +1 -1
  50. package/src/sdk.ts +12 -4
  51. package/src/tools/research-knowledge-search.ts +23 -7
  52. package/src/tools/research-tool-definition.ts +1 -1
  53. package/src/tools/scrape.ts +12 -9
  54. package/src/tools/search.ts +12 -6
  55. package/src/tools/security.ts +6 -1
  56. package/src/tools/stackexchange.ts +6 -1
  57. package/src/utils/cancellation.ts +34 -1
  58. package/src/utils/llm-usage.ts +8 -0
  59. package/src/utils/log-utils.ts +1 -1
  60. package/src/web-research/browser-search.ts +6 -5
  61. package/src/web-research/pdf-extract-worker.mjs +1289 -0
  62. package/src/web-research/pdf-extract-worker.ts +42 -0
  63. package/src/web-research/pdf-extraction.ts +392 -0
  64. package/src/web-research/web-scraper.ts +3 -111
package/README.md CHANGED
@@ -40,19 +40,19 @@ pi install npm:@lincoln504/pi-research
40
40
  Standalone, as an [agent skill](docs/AGENT-SKILL.md) for Claude Code, Codex, and other skills-compatible agents:
41
41
 
42
42
  ```bash
43
- npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3
43
+ npm install -g @lincoln504/pi-research
44
44
  pi-research skill install
45
45
  ```
46
46
 
47
- On npm 12 or newer, npm skips dependency install scripts by default; the flag allows the one that builds search's SQLite module — without it, every search fails with a missing-module error (the flag is harmless on older npm). `pi install` passes no flags to npm, so on npm 12 bracket any install or update with the same allowance as config — set, run, delete — because while the setting is in place it can break installs of other projects with git dependencies ([npm/cli#9783](https://github.com/npm/cli/issues/9783)):
47
+ On npm ≥11.19 (and npm 12), dependency install scripts are skipped by default. This
48
+ package needs none of them: better-sqlite3 13 ships prebuilt bindings for every
49
+ supported platform inside its own tarball, and the stealth browser (~500MB)
50
+ self-provisions on its first use instead of at install time — so the first scrape
51
+ takes a few minutes. Nothing needs approving. (On older npm, which still runs
52
+ scripts, a Windows machine without Visual Studio Build Tools can fail while npm
53
+ needlessly recompiles better-sqlite3; upgrading npm fixes that too.)
48
54
 
49
- ```bash
50
- npm config set allow-scripts=better-sqlite3 --location=user
51
- pi install npm:@lincoln504/pi-research
52
- npm config delete allow-scripts --location=user
53
- ```
54
-
55
- In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md). The first install downloads the stealth browser engine, which takes a few minutes.
55
+ In pi it works out of the box on the session's model and pi's configuration. Standalone use ([agent skill](docs/AGENT-SKILL.md) or [SDK](docs/SDK.md)) needs a model configured. See [Configuration](docs/CONFIGURATION.md).
56
56
 
57
57
  ### How it works
58
58
 
@@ -82,6 +82,7 @@ Three depth levels (normal, deep, ultra) set the team size and number of rounds.
82
82
  - An LLM with a 100k+ context window (an API key or a local model)
83
83
  - Internet access on a residential IP, since search, scraping, and YouTube transcripts get bot-blocked from datacenter/VPS/cloud IPs
84
84
  - pi. The pi extension uses the host's copy while the standalone CLI and agent skill install it as a dependency.
85
+ - Local knowledge-store embeddings need `@huggingface/transformers`, which is an **optional dependency**: if its native image chain (sharp) cannot install on your machine, npm skips it with a warning, the install still succeeds, and everything except local embeddings works. The store disables embeddings gracefully with a message telling you how to restore them.
85
86
  - Cloudflare Turnstile and similar systems block scraping on some sites. A run compensates with a wide pool of search results to scrape.
86
87
 
87
88
  ### Channels
@@ -95,3 +96,7 @@ MIT. Bundled third-party licenses are listed in [docs/THIRD-PARTY-NOTICES.md](do
95
96
  ### Package identity
96
97
 
97
98
  This project is always `@lincoln504/pi-research` on npm. The `pi-research` (unscoped) package is unrelated and deprecated.
99
+
100
+ ### Versioning
101
+
102
+ pi-research typically follows the latest pi version and is not actively backwards compatible.
@@ -43,13 +43,14 @@ model/key is missing.
43
43
  model (`PI_RESEARCH_MODEL`) + API key. See [`references/configuration.md`](references/configuration.md).
44
44
 
45
45
  ```bash
46
- npm install -g @lincoln504/pi-research --allow-scripts=better-sqlite3
46
+ npm install -g @lincoln504/pi-research
47
47
  node "<skill_dir>/scripts/run.mjs" status # verify detection
48
48
  ```
49
49
 
50
- On npm 12 or newer the flag is needed: npm blocks the install script that builds the
51
- stealth browser's SQLite module, and without it every search fails with a
52
- missing-module error. It is harmless on older npm.
50
+ On npm ≥11.19 (and npm 12) dependency install scripts are skipped by default — and
51
+ none are needed: better-sqlite3 13 ships prebuilt bindings in its own tarball, and
52
+ the stealth browser self-provisions on its first use (the first scrape takes a few
53
+ minutes). No approval is required.
53
54
 
54
55
  ### Install the skill
55
56
 
@@ -2,7 +2,7 @@
2
2
  name: pi-research
3
3
  description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
4
4
  license: MIT
5
- metadata: { "author": "Lincoln Deen", "version": "1.6.5", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
5
+ metadata: { "author": "Lincoln Deen", "version": "1.6.7", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
6
6
  allowed-tools: Bash(node:*)
7
7
  ---
8
8