@lincoln504/pi-research 0.1.13 → 1.0.0

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2026 Lincoln504
+Copyright (c) 2026 Lincoln Deen
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,164 +1,96 @@
-![pi-research banner](docs/assets/README-banner.jpg)
-
-<a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
-
-Multi-agent web research extension for [pi](https://github.com/badlogic/pi-mono). Runs parallel AI researchers against a self-hosted SearXNG search container, scrapes pages, queries security databases, and tracks everything in a real-time terminal UI.
-
-There are other ways to connect a `pi` session to the internet. Here is why this one is designed the way it is:
-
-- **SearXNG-based** — free, no API keys required by default. A single Docker container is shared across all concurrent research calls on the same machine; no duplicate instances, no per-session overhead. Point `SEARXNG_URL` at an existing instance to skip container management entirely.
-- **Unified subagent system** — web access, scraping, and synthesis are handled by isolated researcher agents. Research context stays out of the main chat session while still enabling thorough multi-source exploration.
-- **Safety** — researcher agents have no access to file writes, file edits, or shell commands. All web tools are scoped to the subagent system and executed in a structured, tool-limited sequence to keep agents focused and prevent runaway use.
-- **Minimal integration** — a system prompt hook provides light guidance to the host LLM on when and how to invoke the tool. No manual prompt engineering required. The extension does not conflict with other subagent or tool systems, and the unified toolset means no need to wire up or tune multiple separate tools.
+<p align="center">
+  <img src="docs/media/banner.jpg" alt="pi-research" width="100%" />
+</p>
+
+<p align="center">
+  <strong>Limitless research and knowledge store — for pi, agent skill, or SDK.</strong>
+</p>
+
+<p align="center">
+  <a href="docs/PI-EXTENSION.md">Pi extension</a>
+  ·
+  <a href="docs/AGENT-SKILL.md">Agent skill</a>
+  ·
+  <a href="docs/SDK.md">SDK</a>
+  ·
+  <a href="docs/KNOWLEDGE-STORE.md">Knowledge store</a>
+  ·
+  <a href="docs/CONFIGURATION.md">Configuration</a>
+  ·
+  <a href="docs/ARCHITECTURE.md">Architecture</a>
+</p>
 
 ---
 
-## Contents
+<a href="https://github.com/Lincoln504/pi-research/actions/workflows/ci.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/Lincoln504/pi-research/ci.yml?style=flat-square&branch=main" /></a> <a href="https://www.npmjs.com/package/@lincoln504/pi-research"><img alt="npm version" src="https://img.shields.io/npm/v/@lincoln504/pi-research.svg?style=flat-square" /></a>
 
-- [Capabilities](#capabilities)
-- [Requirements](#requirements)
-- [Install](#install)
-- [Platform Setup](#platform-setup)
-- [Usage](#usage)
-- [Research Depth](#research-depth)
-- [Configuration](#configuration)
-- [Development & Release](#development--release)
-- [Architecture & Design](docs/ARCHITECTURE.md)
+Research is broken into subtasks automatically, with each sub-researcher given a high volume of sources to investigate. Then, evaluator decides whether the answer is complete or an additional round of research is needed. The final result is a synthesized cited report.
 
----
+Ask in natural language — the tool understands the depth needed:
 
-## Capabilities
+![Prompt-driven multi-round research in the pi TUI](docs/media/02-prompt-research.gif)
 
-- **Multi-agent deep research** — AI coordinator decomposes a query into parallel research tracks; an AI lead evaluator synthesizes or delegates further rounds
-- **Quick mode** — single researcher for fast, focused queries with no coordinator overhead
-- **Web search** — SearXNG running in Docker, aggregating Bing, Brave, Yahoo, Qwant, DuckDuckGo, Wikipedia, GitHub, arXiv, and Semantic Scholar (engines can be enabled or disabled via [config](docs/ARCHITECTURE.md#searxng-lifecycle-management))
-- **URL scraping** — fetch-first with Playwright/Chromium fallback for JavaScript-heavy pages
-- **Security databases** — NVD, CISA KEV, GitHub Advisories, OSV
-- **Stack Exchange** — full network search and filtering
-- **Real-time TUI** — per-researcher token and cost tracking
+### Use cases
 
-```
-── Research: 70% ─────────────────────────────────────
-┌───────┬────┐ 1 ┌─────┬────┐ 2 ┌────┬────┐ 3 ┌────┐
-│SearXNG│     18k      │     36k     │     50k     │
-│:55732 │   $0.0056    │   $0.0055   │   $0.0071   │
-└───────┴──────────────┴─────────────┴─────────────┘
-```
+- Plug-and-play (no API key needed) research tool for your [pi](https://github.com/earendil-works/pi).
+- Research from Claude Code, Codex, or another coding agent, with a cheaper lightweight or local model driving the research so it doesn't spend your main agent's budget.
+- Populating a dataset or building an index of knowledge sources from the web.
+- Holding research in the knowledge store with a configurable scope — project-specific or globally user-scoped, set per directory from the `/research-config` TUI.
+- Using the pi-research agent skill as OpenClaw's web access.
+- Building agent systems that identify and examine web sources.
 
+### Advantages
 
----
+- Unlimited search and scrape, for free — you only pay for LLM tokens.
+- Context-efficient — it returns a synthesized, cited report to the chat instead of dumping raw web content into the conversation.
+- Safe by design — web access runs inside a specialized, limited research agent with no filesystem or shell access.
+- Search a little or a lot — three depth levels in the pi tool (four via the SDK and standalone CLI, which add the quick depth-0 path). Levels 1 and 2 are recommended for everyday workflow use; level 3 is for larger-scale investigations.
 
-## Requirements
+### Requirements / limitations
 
-- Node.js `22.13.0` or any `24.x+` release
-- The `pi` CLI installed and configured
-- Docker daemon running (required for SearXNG)
+- Node.js >= 22.19.0
+- An LLM with a 100k+ context window (bring your own key)
 - Internet access
-- LLM in pi with minimum 100k context window (optimized for 100k+)  
+- A residential IP address — search, scraping, and YouTube transcripts all rely on a residential connection. A datacenter/VPS/cloud IP gets bot-blocked by the providers these features depend on.
+- The pi runtime the engine builds on — `@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`, and `@earendil-works/pi-tui`. The pi extension uses the host's copies; the standalone CLI / agent skill install them as dependencies (`@earendil-works/pi-coding-agent` is the dependency the launcher checks is resolvable, and it will tell you to reinstall if a partial install leaves it out).
+- Cloudflare and similar anti-automation systems block scraping on some sites, so runs will identify sources they cannot reach. pi-research compensates with volume: the search tool rapidly pulls a large set of results off free DuckDuckGo, giving the model a wide selection of reachable content to cite.
 
----
+### Install
 
-## Install
-
-```bash
-pi install npm:@lincoln504/pi-research
-```
-
-The installation will automatically download and install:
-- Node.js dependencies
-- Playwright browser binaries (Chromium)
-
-This process may take a few minutes on first install. If browser installation fails during postinstall, chromium will be installed automatically when you first use the research tool.
-
-**Manual browser installation (if needed)**
-
-```bash
-npm run install:browsers
-```
+Install the front-end you want.
 
-**From a checkout**
+pi extension
 
 ```bash
-pi install .
+pi install npm:@lincoln504/pi-research
 ```
 
----
-
-## Platform Setup
+Standalone CLI / agent skill
 
-**macOS** — Install Docker Desktop and start it from Applications.
-
-**Linux** — Install Docker for your distribution ([docs.docker.com/engine/install](https://docs.docker.com/engine/install)), then:
 ```bash
-# Start Docker
-sudo systemctl start docker
-sudo systemctl enable docker
-
-# Add your user to docker group (then log out and back in)
-sudo usermod -aG docker $USER
-```
-
-**Windows** — Install Docker Desktop (Linux containers). Detecting named pipes is automatic.
-
-> **Note**: The first time you use `pi install` on a fresh system, Docker may not be running yet. The extension will check Docker when you actually use the research tool, not during install.
-
----
-
-## Usage
-
-Ask pi for web research. The extension registers a `research` tool.
-
-```text
-Research "What is a binary search tree?"
-Do a deep dive on CVE-2024-3094 at depth 2
+npm install -g @lincoln504/pi-research
 ```
 
-| Depth | Mode | Researchers | Target rounds |
-|-------|------|-------------|---------------|
-| 0 (default) | Quick | 1 | 1 |
-| 1 | Normal | 2 | 2 |
-| 2 | Deep | 3 | 3 |
-| 3 | Ultra | 5 | 5 |
-
----
-
-## Configuration
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `SEARXNG_URL` | — | External SearXNG base URL (e.g. `http://localhost:8080`). When set, Docker container management is skipped entirely and this URL is used directly |
-| `BRAVE_SEARCH_API_KEY` | — | Enables `braveapi` engine (official REST API) |
-| `PI_RESEARCH_VERBOSE` | — | Set to `1` to write diagnostic JSONL logs to a temp file (see below) |
-| `PI_RESEARCH_SKIP_HEALTHCHECK` | — | Skip startup health check (set to `1` if engines are blocked) |
-| `PROXY_URL` | — | Proxy for SearXNG outgoing requests (e.g. `socks5://127.0.0.1:9050`) |
-| `STACKEXCHANGE_API_KEY` | — | Stack Exchange API key (increases limit from 300 to 10,000/day) |
-| `PI_RESEARCH_SEARCH_LANGUAGE` | `en-US` | BCP 47 language tag for search result filtering |
-| `PI_RESEARCH_RESEARCHER_TIMEOUT_MS` | `240000` | Per-researcher timeout (30s to 10m) |
-| `PI_RESEARCH_MAX_CONCURRENT_RESEARCHERS` | `3` | Maximum researchers running simultaneously (1 to 10) |
-| `PI_RESEARCH_CONSOLE_RESTORE_DELAY_MS` | `15000` | Delay before TUI clears after research ends |
-| `SEARXNG_IMAGE_TAG` | `latest` | SearXNG Docker image tag — pin to a specific release for reproducibility (e.g. `2024.12.23`) |
+Then configure a model and key — set `PI_RESEARCH_MODEL` (e.g. `openai/gpt-4o`) plus `PI_RESEARCH_API_KEY` and `PI_RESEARCH_PROVIDER`, either as environment variables or in `~/.pi/research/config.env`. See [Configuration](docs/CONFIGURATION.md). (The pi extension needs no key — it uses pi's own auth.)
 
-**Verbose logging**
+See [Agent skill](docs/AGENT-SKILL.md) for details.
 
-Set `PI_RESEARCH_VERBOSE=1` (or pass `--verbose`) to enable diagnostic output. Logs are written as JSONL to `{os.tmpdir()}/pi-research-debug-{hash}.log` — one file per session, never written to stdout. Each log line includes a session correlation ID so multiple concurrent sessions are distinguishable. The log path is printed to stderr on startup when verbose mode is active.
+The first install pulls the stealth browser engine, which takes a few minutes.
 
----
+On Windows, run the CLI from `cmd` or use `pi-research.cmd` — stock PowerShell
+execution policy (`Restricted`) blocks npm's `.ps1` shims with a "running
+scripts is disabled" error. Alternatively run
+`Set-ExecutionPolicy -Scope CurrentUser RemoteSigned` once.
 
-## Development & Release
+### Stability (v1.x.x)
 
-**Commands**
-- `npm run lint` / `npm run lint:fix`: Code quality
-- `npm run type-check`: TypeScript verification
-- `npm run test:unit`: Unit tests — no dependencies
-- `npm run test:integration`: Integration tests — requires Docker; uses Testcontainers to spin up a real SearXNG instance (runs locally and in CI)
+v0.1.13 (April 2026) was the last release before a rebuild against current pi APIs
+and the stealth-browser stack. v1.0.0 is the first stable release since. Channels:
 
-**Release**
-1. `npm version patch` (or minor/major)
-2. `git push origin main --tags`
-The GitHub Action will automatically publish to npm.
-
----
+- npm (`npm:@lincoln504/pi-research`) is the stable channel, kept current with breaking pi changes.
+- A git install is the development channel: latest commits, first to break.
 
-## Architecture
+### License
 
-See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed design information.
+MIT. Bundled third-party licenses are listed in
+[docs/THIRD-PARTY-NOTICES.md](docs/THIRD-PARTY-NOTICES.md).