@lincoln504/pi-research 0.1.12 → 1.0.0

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2026 Lincoln504
+Copyright (c) 2026 Lincoln Deen
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,147 +1,96 @@
-![pi-research banner](docs/assets/README-banner.jpg)
-
-<a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
-
-Multi-agent web research extension for [pi](https://github.com/badlogic/pi-mono). Runs parallel AI researchers against a self-hosted SearXNG search container, scrapes pages, queries security databases, and tracks everything in a real-time terminal UI.
+<p align="center">
+  <img src="docs/media/banner.jpg" alt="pi-research" width="100%" />
+</p>
+
+<p align="center">
+  <strong>Limitless research and knowledge store — for pi, agent skill, or SDK.</strong>
+</p>
+
+<p align="center">
+  <a href="docs/PI-EXTENSION.md">Pi extension</a>
+  ·
+  <a href="docs/AGENT-SKILL.md">Agent skill</a>
+  ·
+  <a href="docs/SDK.md">SDK</a>
+  ·
+  <a href="docs/KNOWLEDGE-STORE.md">Knowledge store</a>
+  ·
+  <a href="docs/CONFIGURATION.md">Configuration</a>
+  ·
+  <a href="docs/ARCHITECTURE.md">Architecture</a>
+</p>
 
 ---
 
-## Contents
+<a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
 
-- [Capabilities](#capabilities)
-- [Requirements](#requirements)
-- [Install](#install)
-- [Platform Setup](#platform-setup)
-- [Usage](#usage)
-- [Research Depth](#research-depth)
-- [Configuration](#configuration)
-- [Development & Release](#development--release)
-- [Architecture & Design](docs/ARCHITECTURE.md)
+Research is broken into subtasks automatically, with each sub-researcher given a high volume of sources to investigate. Then, evaluator decides whether the answer is complete or an additional round of research is needed. The final result is a synthesized cited report.
 
----
+Ask in natural language — the tool understands the depth needed:
 
-## Capabilities
+![Prompt-driven multi-round research in the pi TUI](docs/media/02-prompt-research.gif)
 
-- **Multi-agent deep research** — AI coordinator decomposes a query into parallel research tracks; an AI lead evaluator synthesizes or delegates further rounds
-- **Quick mode** — single researcher for fast, focused queries with no coordinator overhead
-- **Web search** — SearXNG running in Docker, aggregating Bing, Brave, Yahoo, Qwant, DuckDuckGo, Wikipedia, GitHub, arXiv, and Semantic Scholar (engines can be enabled or disabled via [config](docs/ARCHITECTURE.md#searxng-lifecycle-management))
-- **URL scraping** — fetch-first with Playwright/Chromium fallback for JavaScript-heavy pages
-- **Security databases** — NVD, CISA KEV, GitHub Advisories, OSV
-- **Stack Exchange** — full network search and filtering
-- **Real-time TUI** — per-researcher token and cost tracking
+### Use cases
 
-```
-── Research: 70% ─────────────────────────────────────
-┌───────┬────┐ 1 ┌─────┬────┐ 2 ┌────┬────┐ 3 ┌────┐
-│SearXNG│     18k      │     36k     │     50k     │
-│:55732 │   $0.0056    │   $0.0055   │   $0.0071   │
-└───────┴──────────────┴─────────────┴─────────────┘
-```
+- Plug-and-play (no API key needed) research tool for your [pi](https://github.com/earendil-works/pi).
+- Research from Claude Code, Codex, or another coding agent, with a cheaper lightweight or local model driving the research so it doesn't spend your main agent's budget.
+- Populating a dataset or building an index of knowledge sources from the web.
+- Holding research in the knowledge store with a configurable scope — project-specific or globally user-scoped, set per directory from the `/research-config` TUI.
+- Using the pi-research agent skill as OpenClaw's web access.
+- Building agent systems that identify and examine web sources.
 
+### Advantages
 
----
+- Unlimited search and scrape, for free — you only pay for LLM tokens.
+- Context-efficient — it returns a synthesized, cited report to the chat instead of dumping raw web content into the conversation.
+- Safe by design — web access runs inside a specialized, limited research agent with no filesystem or shell access.
+- Search a little or a lot — three depth levels in the pi tool (four via the SDK and standalone CLI, which add the quick depth-0 path). Levels 1 and 2 are recommended for everyday workflow use; level 3 is for larger-scale investigations.
 
-## Requirements
+### Requirements / limitations
 
-- Node.js `22.13.0` or any `24.x+` release
-- The `pi` CLI installed and configured
-- Docker daemon running (required for SearXNG)
+- Node.js >= 22.19.0
+- An LLM with a 100k+ context window (bring your own key)
 - Internet access
+- A residential IP address — search, scraping, and YouTube transcripts all rely on a residential connection. A datacenter/VPS/cloud IP gets bot-blocked by the providers these features depend on.
+- The pi runtime the engine builds on — `@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`, and `@earendil-works/pi-tui`. The pi extension uses the host's copies; the standalone CLI / agent skill install them as dependencies (`@earendil-works/pi-coding-agent` is the dependency the launcher checks is resolvable, and it will tell you to reinstall if a partial install leaves it out).
+- Cloudflare and similar anti-automation systems block scraping on some sites, so runs will identify sources they cannot reach. pi-research compensates with volume: the search tool rapidly pulls a large set of results off free DuckDuckGo, giving the model a wide selection of reachable content to cite.
 
----
+### Install
 
-## Install
+Install the front-end you want.
 
-```bash
-pi install npm:@lincoln504/pi-research
-```
-
-**From a checkout**
+pi extension
 
 ```bash
-npm install
-pi install .
+pi install npm:@lincoln504/pi-research
 ```
 
----
-
-## Platform Setup
-
-**macOS** — Install Docker Desktop and start it from Applications.
+Standalone CLI / agent skill
 
-**Linux** — Install Docker and add your user to the docker group:
 ```bash
-# Install Docker
-sudo apt-get update && sudo apt-get install -y docker.io
-
-# Start Docker
-sudo systemctl start docker
-sudo systemctl enable docker
-
-# Add your user to docker group (then log out and back in)
-sudo usermod -aG docker $USER
+npm install -g @lincoln504/pi-research
 ```
 
-**Windows** — Install Docker Desktop (Linux containers). Detecting named pipes is automatic.
-
-> **Note**: The first time you use `pi install` on a fresh system, Docker may not be running yet. The extension will check Docker when you actually use the research tool, not during install.
-
----
+Then configure a model and key — set `PI_RESEARCH_MODEL` (e.g. `openai/gpt-4o`) plus `PI_RESEARCH_API_KEY` and `PI_RESEARCH_PROVIDER`, either as environment variables or in `~/.pi/research/config.env`. See [Configuration](docs/CONFIGURATION.md). (The pi extension needs no key — it uses pi's own auth.)
 
-## Usage
+See [Agent skill](docs/AGENT-SKILL.md) for details.
 
-Ask pi for web research. The extension registers a `research` tool.
+The first install pulls the stealth browser engine, which takes a few minutes.
 
-```text
-Research "What is a binary search tree?"
-Do a deep dive on CVE-2024-3094 at depth 2
-```
-
-| Depth | Mode | Researchers | Target rounds |
-|-------|------|-------------|---------------|
-| 0 (default) | Quick | 1 | 1 |
-| 1 | Normal | 2 | 2 |
-| 2 | Deep | 3 | 3 |
-| 3 | Ultra | 5 | 5 |
-
----
+On Windows, run the CLI from `cmd` or use `pi-research.cmd` — stock PowerShell
+execution policy (`Restricted`) blocks npm's `.ps1` shims with a "running
+scripts is disabled" error. Alternatively run
+`Set-ExecutionPolicy -Scope CurrentUser RemoteSigned` once.
 
-## Configuration
+### Stability (v1.x.x)
 
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `BRAVE_SEARCH_API_KEY` | — | Enables `braveapi` engine (official REST API) |
-| `PI_RESEARCH_VERBOSE` | — | Set to `1` to write diagnostic JSONL logs to a temp file (see below) |
-| `PI_RESEARCH_SKIP_HEALTHCHECK` | — | Skip startup health check (set to `1` if engines are blocked) |
-| `PROXY_URL` | — | Proxy for SearXNG outgoing requests (e.g. `socks5://127.0.0.1:9050`) |
-| `STACKEXCHANGE_API_KEY` | — | Stack Exchange API key (increases limit from 300 to 10,000/day) |
-| `PI_RESEARCH_SEARCH_LANGUAGE` | `en-US` | BCP 47 language tag for search result filtering |
-| `PI_RESEARCH_RESEARCHER_TIMEOUT_MS` | `240000` | Per-researcher timeout (30s to 10m) |
-| `PI_RESEARCH_MAX_CONCURRENT_RESEARCHERS` | `3` | Maximum researchers running simultaneously (1 to 10) |
-| `PI_RESEARCH_CONSOLE_RESTORE_DELAY_MS` | `15000` | Delay before TUI clears after research ends |
-| `SEARXNG_IMAGE_TAG` | `latest` | SearXNG Docker image tag — pin to a specific release for reproducibility (e.g. `2024.12.23`) |
+v0.1.13 (April 2026) was the last release before a rebuild against current pi APIs
+and the stealth-browser stack. v1.0.0 is the first stable release since. Channels:
 
-**Verbose logging**
-
-Set `PI_RESEARCH_VERBOSE=1` (or pass `--verbose`) to enable diagnostic output. Logs are written as JSONL to `{os.tmpdir()}/pi-research-debug-{hash}.log` — one file per session, never written to stdout. Each log line includes a session correlation ID so multiple concurrent sessions are distinguishable. The log path is printed to stderr on startup when verbose mode is active.
-
----
-
-## Development & Release
-
-**Commands**
-- `npm run lint` / `npm run lint:fix`: Code quality
-- `npm run type-check`: TypeScript verification
-- `npm run test:unit`: Unit tests — no dependencies
-- `npm run test:integration`: Integration tests — requires Docker; uses Testcontainers to spin up a real SearXNG instance (runs locally and in CI)
-
-**Release**
-1. `npm version patch` (or minor/major)
-2. `git push origin main --tags`
-The GitHub Action will automatically publish to npm.
-
----
+- npm (`npm:@lincoln504/pi-research`) is the stable channel, kept current with breaking pi changes.
+- A git install is the development channel: latest commits, first to break.
 
-## Architecture
+### License
 
-See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed design information.
+MIT. Bundled third-party licenses are listed in
+[docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).