pi-para 0.5.2 → 0.6.0

package/SETUP.md

@@ -1,171 +1,255 @@
-# pi-para Setup Instructions
+# pi-para Setup
 
-> This file is designed to be read by pi (the AI agent). Ask pi: "read SETUP.md and set up pi-para"
+This is the current setup flow for pi-para. It is cross-platform and does not require `systemd`, a long-running daemon, or a global QMD CLI.
 
 ## Prerequisites
 
-- pi (coding agent) installed and working
-- Node.js available (`node --version`)
-- Internet access for npm
+- Pi installed and working.
+- Node.js/npm available.
+- A restarted Pi session after install/config changes.
 
-## Step 1: Install the extension
+## Standard install
 
 ```bash
-pi install @picassio/pi-para
+pi install pi-para
 ```
 
-Verify it loaded — restart pi and check extensions list. You should see `pi-para` in the `[Extensions]` section.
+Then restart Pi.
 
-## Step 2: Install qmd search engine (recommended)
+Run diagnostics:
 
-qmd provides hybrid search (BM25 + vector + rerank). Without it, wiki search uses BM25 keyword matching only.
+```bash
+pi-para doctor
+```
+
+If needed, apply safe repairs:
+
+```bash
+pi-para doctor --fix
+```
+
+## npx setup flow
+
+For a fresh machine or scripted install:
 
 ```bash
-npm install -g @picassio/qmd
+npx pi-para@latest setup
 ```
 
-Verify: `qmd --version` should print a version number.
+POSIX one-liner:
 
-## Step 3: Configure search providers
+```bash
+curl -fsSL https://raw.githubusercontent.com/picassio/pi-para/main/scripts/install.sh | bash
+```
+
+Windows PowerShell one-liner:
+
+```powershell
+irm https://raw.githubusercontent.com/picassio/pi-para/main/scripts/install.ps1 | iex
+```
+
+Non-interactive defaults:
 
-Create `~/.config/qmd/index.yml` with your API providers. Choose based on what keys you have:
+```bash
+npx pi-para@latest setup --yes
+```
 
-### Option A: OpenRouter (embedding) + MiniMax CN (chat)
+Dry run:
 
-```yaml
-providers:
-  embed:
-    url: https://openrouter.ai/api/v1
-    key: YOUR_OPENROUTER_KEY
-    model: openai/text-embedding-3-small
-    dims: 768
-  chat:
-    url: https://api.minimaxi.com/anthropic
-    key: YOUR_MINIMAX_CN_KEY
-    model: MiniMax-M2.7-highspeed
-    api: anthropic
+```bash
+npx pi-para@latest setup --dry-run
 ```
 
-### Option B: OpenAI only
+Local development install:
 
-```yaml
-providers:
-  embed:
-    url: https://api.openai.com/v1
-    key: YOUR_OPENAI_KEY
-    model: text-embedding-3-small
-    dims: 1536
-  chat:
-    url: https://api.openai.com/v1
-    key: YOUR_OPENAI_KEY
-    model: gpt-4o-mini
+```bash
+cd /path/to/pi-para
+npm install
+npm run build
+node ./dist/cli.js setup --local .
 ```
 
-### Option C: No providers (BM25 only)
+Restart Pi after setup.
+
+## What setup does
+
+`pi-para setup` is idempotent. It may:
 
-Skip this step. Wiki search works with keyword matching. No API keys needed.
+1. locate Pi settings,
+2. register the pi-para extension,
+3. initialize `~/.pi/wiki/`,
+4. create or migrate `~/.pi/para/config.jsonc`,
+5. repair generated-state `.gitignore` entries,
+6. validate the embedded QMD SDK,
+7. print next steps.
 
-## Step 4: Configure daemon LLM (optional)
+It does **not** install a system daemon by default, does **not** require global `qmd`, and does **not** copy secrets unless you explicitly store one.
+
+## Configuration locations
+
+Canonical:
+
+```text
+~/.pi/para/config.jsonc
+~/.pi/para/secrets.json
+~/.pi/wiki/
+~/.pi/wiki/.pi-para.sqlite
+~/.pi/wiki/.qmd.sqlite
+```
 
-Edit `~/.pi/wiki/config.json` (created after first pi session) and set `daemonModel`:
+Legacy migration inputs:
 
-```json
-{
-  "daemonModel": "anthropic/claude-sonnet-4"
-}
+```text
+~/.pi/wiki/config.json
+~/.pi/wiki/.daemon.sqlite
+~/.config/qmd/index.yml
 ```
 
-If null, the daemon auto-detects from:
-1. Pi's API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.)
-2. qmd chat provider from Step 3
+## Provider credentials
 
-## Step 5: Start the background capture daemon
+Credential refs use:
 
-### Linux (systemd — recommended)
+```text
+pi-auth:<provider>
+secret:<name>
+none
+```
 
-Run the setup script from the pi-para install directory:
+Examples:
 
 ```bash
-# Find the install path
-PARA_DIR="$(npm root -g)/@picassio/pi-para"
-# Or if installed via pi:
-# PARA_DIR="$HOME/.pi/agent/packages/npm/@picassio/pi-para"
+pi-para providers
+pi-para providers set-secret embedding sk-...
+pi-para providers remove-secret embedding
+```
+
+Recommended order:
+
+1. Pi AuthStorage (`~/.pi/agent/auth.json`),
+2. pi-para secrets (`~/.pi/para/secrets.json`),
+3. `none` for local/no-auth endpoints.
+
+No env-var setup path is required.
+
+## Configure inside Pi
+
+Run:
+
+```text
+/wiki-settings
+```
+
+The settings menu writes through the same config service as the CLI. It can configure:
 
-cd "$PARA_DIR"
+- context token budget,
+- search limit and graph boost,
+- lint autofix/stale days,
+- capture model,
+- WebWiki host/port/enabled,
+- embedding and rerank provider profiles,
+- local secrets.
 
-# Install tsx if not present
-npm install tsx 2>/dev/null
+## Verify inside Pi
 
-# Create systemd user service
-mkdir -p ~/.config/systemd/user
-NODE_BIN="$(dirname "$(which node)")"
+Useful commands:
+
+```text
+/wiki
+/wiki-search test
+/wiki-settings
+/wiki-scheduler status
+/wiki-lint --report-only
+```
 
-cat > ~/.config/systemd/user/pi-para-daemon.service << EOF
-[Unit]
-Description=pi-para knowledge capture daemon
-After=network.target
+Try a smoke test:
 
-[Service]
-Type=simple
-WorkingDirectory=$PARA_DIR
-ExecStart=$PARA_DIR/node_modules/.bin/tsx src/cli.ts start
-Restart=on-failure
-RestartSec=10
-Environment=HOME=$HOME
-Environment=PATH=$NODE_BIN:$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin
+```text
+Save this to the wiki as a test note: pi-para setup succeeded.
+```
 
-[Install]
-WantedBy=default.target
-EOF
+Then:
 
-# Enable, start, and survive SSH logout
-systemctl --user daemon-reload
-systemctl --user enable pi-para-daemon
-systemctl --user start pi-para-daemon
-sudo loginctl enable-linger $USER
+```text
+/wiki-search setup succeeded
 ```
 
-Verify: `systemctl --user status pi-para-daemon`
+## Background capture
+
+No daemon needs to be started.
 
-### macOS / Manual
+pi-para queues completed/compacted sessions and processes them when a Pi session with pi-para is open. If Pi is closed, queued work resumes on the next startup.
+
+Inspect queue/history:
 
 ```bash
-cd "$PARA_DIR" && npx tsx src/cli.ts start
+pi-para tasks
+pi-para tasks history
+pi-para capture-recent --hours 24
+```
+
+Inside Pi:
+
+```text
+/wiki-scheduler queue
+/wiki-scheduler history
+/wiki-scheduler capture-history
 ```
 
-Keep the terminal open, or use a process manager like pm2.
+`/wiki-daemon` and `pi-para-daemon` remain compatibility aliases for older workflows.
+
+## QMD search
+
+Global QMD CLI is not required. pi-para uses embedded `qmd-engine` SDK.
 
-## Step 6: Verify everything works
+- BM25 search works with no providers.
+- Embeddings/rerank use provider profiles in `~/.pi/para/config.jsonc`.
+- Legacy `~/.config/qmd/index.yml` is read only for compatibility.
+
+Doctor check:
+
+```bash
+pi-para doctor
+```
 
-In pi, run these commands:
+Provider/model diagnostic:
 
-1. `/wiki-settings` — should show config, search engine status, providers, daemon status
-2. `/wiki` — should show "Pages: 0 total" (empty wiki)
-3. Say: "save to wiki: this is a test page" — should create a page
-4. `/wiki` — should show "Pages: 1 total"
-5. Say: "search the wiki for test" — should find the test page
+```bash
+pi-para doctor --test-capture-model
+pi-para status --json
+```
 
 ## Troubleshooting
 
-### Extension not loading
-- Check: `pi list` should show `@picassio/pi-para`
-- Try: `pi -e npm:@picassio/pi-para` for a quick test
+### Extension changed but behavior did not
+
+Restart Pi. Existing sessions keep loaded extension code.
+
+### Missing or stale provider credentials
+
+Run:
+
+```bash
+pi-para doctor
+pi-para providers
+```
+
+Store a local secret only if you do not want to use Pi AuthStorage:
+
+```bash
+pi-para providers set-secret embedding <key>
+```
+
+### Search works but embeddings fail
+
+BM25 search remains available. Check:
 
-### qmd not found
-- Check: `which qmd` and `qmd --version`
-- Install: `npm install -g @picassio/qmd`
+```bash
+pi-para doctor
+/wiki-settings
+```
 
-### Search not working (no results)
-- BM25 works immediately after `wiki_write`
-- Vector search needs providers configured in `~/.config/qmd/index.yml`
-- Check: `/wiki-settings` shows providers
+Then configure or disable embedding.
 
-### Daemon not running
-- Check: `systemctl --user status pi-para-daemon`
-- Logs: `journalctl --user -u pi-para-daemon -n 20`
-- Manual start: `cd <pi-para-dir> && npx tsx src/cli.ts start`
+### Old daemon/systemd service exists
 
-### Vulkan/CMake errors in logs
-- These are from qmd's node-llama-cpp dependency (safe to ignore)
-- They only appear if no API providers configured
-- Fix: configure providers in `~/.config/qmd/index.yml`
+The old service is no longer required. You can leave it stopped while using the in-process scheduler. Do not start both old daemon capture and the scheduler for the same wiki unless you are intentionally testing legacy behavior.

package/STRUCTURE.md

@@ -0,0 +1,140 @@
+# pi-para Repository Structure
+
+## Top level
+
+```text
+README.md                     # user overview
+SETUP.md                      # current setup/onboarding
+ARCHITECTURE.md               # runtime architecture
+CONFIGURATION.md              # config schema and credentials
+NO_DAEMON_REFACTOR_ROADMAP.md # implementation roadmap/history
+PROVIDER_MODEL_SELECTION_PLAN.md
+SETUP_SETTINGS_FLOW_PLAN.md
+package.json
+src/
+test/
+scripts/gepa/
+skills/
+```
+
+## `src/`
+
+### Extension entry and CLI
+
+| Path | Purpose |
+|---|---|
+| `src/index.ts` | Pi extension entry point; registers lifecycle handlers/tools/commands/scheduler |
+| `src/cli.ts` | `pi-para` and compatibility `pi-para-daemon` CLI |
+| `src/commands.ts` | Pi slash commands |
+| `src/tools.ts` | Pi tool definitions |
+
+### Wiki core
+
+| Path | Purpose |
+|---|---|
+| `src/wiki.ts` | Filesystem wiki operations, page writes, index/log maintenance |
+| `src/frontmatter.ts` | Frontmatter parsing/formatting |
+| `src/scope.ts` | Project scope detection and matching |
+| `src/query.ts` | Query formatting and search orchestration |
+| `src/ingest.ts` | Source ingestion |
+| `src/raw.ts` | Raw source storage and session digest helpers |
+| `src/lint.ts` | Wiki health checks/autofix |
+| `src/summarize.ts` | Summary operations |
+| `src/link-utils.ts`, `src/link-discovery.ts`, `src/tag-registry.ts` | Link/tag helpers |
+
+### Runtime/search/state
+
+| Path | Purpose |
+|---|---|
+| `src/store.ts` | QMD SDK store lifecycle, search, reindex/embed helpers |
+| `src/qmd-providers.ts` | Convert pi-para provider profiles into QMD SDK provider config |
+| `src/context.ts` | Wiki context injection into Pi sessions |
+| `src/wiki-tool-guidance.ts` | Shared wiki tool descriptions, prompt snippets, and behavior guidance |
+| `src/state.ts` | Legacy/session capture state DB helpers |
+
+### Config/setup/diagnostics
+
+| Path | Purpose |
+|---|---|
+| `src/paths.ts` | Cross-platform path resolution |
+| `src/config.ts` | JSONC config defaults, migration, save/load |
+| `src/credentials.ts` | Pi AuthStorage/local secret credential refs |
+| `src/settings.ts` | Pure helpers for `/wiki-settings` updates |
+| `src/setup.ts` | CLI setup flow |
+| `src/status.ts` | Lightweight config/wiki/scheduler status summary |
+| `src/doctor.ts` | Diagnostics and safe repairs |
+| `src/repair.ts` | Repair helpers such as generated-state `.gitignore` and secret perms |
+| `src/atomic-write.ts` | Atomic file writes |
+
+### Scheduler
+
+```text
+src/scheduler/
+├── index.ts           # scheduler lifecycle and task execution
+├── state.ts           # queue/history/task SQLite tables
+├── leases.ts          # SQLite lease acquire/release
+├── controls.ts        # CLI/command observability helpers
+└── session-capture.ts # completed-session registry and capture task handler
+```
+
+### GEPA
+
+```text
+src/gepa/index.ts       # TS orchestration/CLI integration
+src/templates/prompts.ts
+scripts/gepa/           # uv/DSPy Python optimizer
+```
+
+### Legacy/compatibility
+
+| Path | Purpose |
+|---|---|
+| `src/daemon.ts` | Legacy daemon foreground mode |
+| `src/watcher.ts` | Legacy `.completed-sessions` watcher |
+| `src/processor.ts` | Legacy session processor |
+| `src/session-tools.ts` | Session exploration tools used by capture |
+| `src/maintainer.ts` | Legacy/maintenance helpers |
+
+New code should avoid adding daemon-only dependencies unless intentionally maintaining compatibility.
+
+## `test/`
+
+Tests are Vitest-based. Important suites:
+
+| Path | Purpose |
+|---|---|
+| `test/config.test.ts` | Config defaults/migration |
+| `test/credentials.test.ts` | Credential refs and secrets |
+| `test/settings.test.ts` | `/wiki-settings` pure helper behavior |
+| `test/qmd-providers.test.ts` | QMD provider profile translation |
+| `test/scheduler*.test.ts` | Scheduler state/controls |
+| `test/session-capture.test.ts` | Completed-session parsing/enqueueing |
+| `test/status.test.ts` | Status summary formatting/counts |
+| `test/doctor.test.ts` | Diagnostics/repairs/provider checks |
+| `test/index.test.ts` | Pi extension entry lifecycle |
+| `test/tools.test.ts`, `test/commands.test.ts` | Tool/command behavior |
+
+Run:
+
+```bash
+npm run check
+npm test
+npm run test:coverage
+npm run build
+```
+
+## Packaging notes
+
+`package.json` exposes public modules under `exports`. When adding a new public module, add it to both `exports` and tests/build expectations.
+
+Primary binary:
+
+```text
+pi-para -> dist/cli.js
+```
+
+Compatibility binary:
+
+```text
+pi-para-daemon -> dist/cli.js
+```

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pi-para",
-  "version": "0.5.2",
-  "description": "PARA Knowledge Base Extension for pi \u2014 persistent, LLM-curated personal wiki",
+  "version": "0.6.0",
+  "description": "PARA Knowledge Base Extension for pi — persistent, LLM-curated personal wiki",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -65,6 +65,62 @@
     "./context": {
       "types": "./dist/context.d.ts",
       "import": "./dist/context.js"
+    },
+    "./config": {
+      "types": "./dist/config.d.ts",
+      "import": "./dist/config.js"
+    },
+    "./credentials": {
+      "types": "./dist/credentials.d.ts",
+      "import": "./dist/credentials.js"
+    },
+    "./paths": {
+      "types": "./dist/paths.d.ts",
+      "import": "./dist/paths.js"
+    },
+    "./scheduler": {
+      "types": "./dist/scheduler/index.d.ts",
+      "import": "./dist/scheduler/index.js"
+    },
+    "./setup": {
+      "types": "./dist/setup.d.ts",
+      "import": "./dist/setup.js"
+    },
+    "./doctor": {
+      "types": "./dist/doctor.d.ts",
+      "import": "./dist/doctor.js"
+    },
+    "./repair": {
+      "types": "./dist/repair.d.ts",
+      "import": "./dist/repair.js"
+    },
+    "./scheduler/session-capture": {
+      "types": "./dist/scheduler/session-capture.d.ts",
+      "import": "./dist/scheduler/session-capture.js"
+    },
+    "./scheduler/controls": {
+      "types": "./dist/scheduler/controls.d.ts",
+      "import": "./dist/scheduler/controls.js"
+    },
+    "./model-resolver": {
+      "types": "./dist/model-resolver.d.ts",
+      "import": "./dist/model-resolver.js"
+    },
+    "./settings": {
+      "types": "./dist/settings.d.ts",
+      "import": "./dist/settings.js"
+    },
+    "./qmd-providers": {
+      "types": "./dist/qmd-providers.d.ts",
+      "import": "./dist/qmd-providers.js"
+    },
+    "./status": {
+      "types": "./dist/status.d.ts",
+      "import": "./dist/status.js"
+    },
+    "./wiki-tool-guidance": {
+      "types": "./dist/wiki-tool-guidance.d.ts",
+      "import": "./dist/wiki-tool-guidance.js"
     }
   },
   "pi": {
@@ -81,45 +137,55 @@
     "scripts",
     "CHANGELOG.md",
     "README.md",
-    "skills",
-    "SETUP.md"
+    "SETUP.md",
+    "ARCHITECTURE.md",
+    "STRUCTURE.md",
+    "CONFIGURATION.md",
+    "skills"
   ],
   "scripts": {
-    "clean": "rm -rf dist",
-    "build": "tsc -p tsconfig.json && mkdir -p dist/webui/client/dist && cp src/webui/client/dist/index.html dist/webui/client/dist/index.html && mkdir -p dist/gepa/optimized && cp -r src/gepa/optimized/*.txt dist/gepa/optimized/ 2>/dev/null || true",
+    "clean": "node -e \"require('node:fs').rmSync('dist',{recursive:true,force:true})\"",
+    "build": "tsc -p tsconfig.json && node scripts/build-assets.mjs",
     "check": "tsc --noEmit",
     "test": "vitest --run",
+    "test:coverage": "vitest --run --coverage",
+    "smoke:install": "npm run build && node scripts/smoke-install.mjs",
     "fix-links": "tsx src/scripts/bulk-link-fix.ts",
     "fix-links:dry": "tsx src/scripts/bulk-link-fix.ts --dry-run"
   },
   "dependencies": {
-    "qmd-engine": "^2.3.3",
     "@types/better-sqlite3": "^7.6.13",
-    "better-sqlite3": "^11.9.1",
+    "better-sqlite3": "^12.11.1",
+    "qmd-engine": "^2.5.0",
     "yaml": "^2.8.2"
   },
+  "engines": {
+    "node": ">=22.0.0"
+  },
   "peerDependencies": {
-    "@mariozechner/pi-agent-core": "^0.70.2",
-    "@mariozechner/pi-ai": "^0.70.2",
-    "@mariozechner/pi-coding-agent": "^0.70.2",
-    "@mariozechner/pi-tui": "^0.70.2",
+    "@earendil-works/pi-agent-core": "^0.80.2",
+    "@earendil-works/pi-ai": "^0.80.2",
+    "@earendil-works/pi-coding-agent": "^0.80.2",
+    "@earendil-works/pi-tui": "^0.80.2",
     "typebox": "^1.1.24"
   },
   "peerDependenciesMeta": {},
   "devDependencies": {
-    "@mariozechner/pi-agent-core": "^0.70.2",
-    "@mariozechner/pi-ai": "^0.70.2",
-    "@mariozechner/pi-coding-agent": "^0.70.2",
-    "@mariozechner/pi-tui": "^0.70.2",
+    "@earendil-works/pi-agent-core": "^0.80.2",
+    "@earendil-works/pi-ai": "^0.80.2",
+    "@earendil-works/pi-coding-agent": "^0.80.2",
+    "@earendil-works/pi-tui": "^0.80.2",
     "@types/node": "^24.3.0",
+    "@vitest/coverage-v8": "^3.2.4",
     "tsx": "^4.21.0",
     "typebox": "^1.1.34",
     "typescript": "^5.7.3",
     "vitest": "^3.2.4"
   },
   "keywords": [
+    "pi-package",
+    "pi-extension",
     "pi",
-    "extension",
     "para",
     "knowledge-base",
     "wiki"
@@ -134,9 +200,11 @@
     "url": "https://github.com/picassio/pi-para/issues"
   },
   "publishConfig": {
+    "registry": "https://registry.npmjs.org/",
     "access": "public"
   },
   "bin": {
+    "pi-para": "./dist/cli.js",
     "pi-para-daemon": "./dist/cli.js"
   }
-}
+}