@kylebrodeur/pi-model-router 0.1.2

package/README.md

@@ -0,0 +1,195 @@
+# pi-model-router
+
+[![npm version](https://img.shields.io/npm/v/@kylebrodeur/pi-model-router.svg)](https://www.npmjs.com/package/@kylebrodeur/pi-model-router)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Pi Version](https://img.shields.io/badge/pi-%3E%3D0.70.2-blue)](https://github.com/mariozechner/pi-coding-agent)
+
+Intelligent per-turn model router extension for the [pi](https://github.com/mariozechner/pi-coding-agent) coding agent. Automatically selects between high, medium, and low-tier LLMs based on task intent, session budget, context size, and custom rules — with automatic fallbacks, phase awareness, Ollama sync, and transparent rate-limit recovery.
+
+## ✨ Features
+
+| Feature | Description |
+|---------|-------------|
+| **Logical Router Provider** | Registers a `router` provider that exposes stable profiles (e.g., `router/auto`) as models. |
+| **Per-Turn Routing** | Intelligently chooses between `high`, `medium`, and `low` tiers for every turn based on task intent and complexity. |
+| **Task-Aware Heuristics** | Detects planning vs. implementation vs. lightweight tasks using keyword analysis, word count, and conversation history. |
+| **LLM Intent Classifier** | Optionally use a fast model to categorize intent (overrides heuristics). |
+| **Custom Rules** | Define keyword-based tier overrides for specific patterns (e.g., `deploy` → `high`). |
+| **Context Trigger** | Automatically upgrade to high-tier when token usage exceeds a threshold. |
+| **Cost Budgeting** | Set a session spend limit; high tier downgrades to medium once exceeded. |
+| **Fallback Chains** | Automatic retry with alternative models if the primary choice fails. |
+| **Phase Memory** | Biased stickiness to keep you in the same tier during multi-turn planning or implementation work. |
+| **Thinking Control** | Full control over reasoning/thinking levels per tier and profile. |
+| **Persistent State** | Pins, profiles, costs, and debug history are remembered across agent restarts and conversation branches. |
+| **Ollama Auto-Sync** | Auto-detect and register local Ollama models. |
+| **Rate-Limit Fallback** | Detect HTTP 402/429/503/529 and transparently fall back to local models. |
+| **Feature Toggles** | Enable/disable features at user or project level. |
+| **Progressive Enhancement** | Detects installed plugins (qmd-ledger, agent-bus) and integrates conditionally. |
+
+## 📦 Installation
+
+### From npm
+
+```bash
+pi install npm:@kylebrodeur/pi-model-router
+```
+
+### From source
+
+```bash
+git clone https://github.com/kylebrodeur/pi-model-router.git
+cd pi-model-router
+pi install .
+```
+
+### Quick test
+
+```bash
+pi -e ./extensions/index.ts
+```
+
+## 🚀 Quick Start
+
+```bash
+# 1. Install the extension
+pi install npm:@kylebrodeur/pi-model-router
+
+# 2. Create default config
+/router init
+
+# 3. Reload to apply
+/reload
+
+# 4. Check status
+/router status
+```
+
+## ⚙️ Configuration
+
+Copy the example config to one of:
+
+- `~/.pi/agent/model-router.json` (Global)
+- `.pi/model-router.json` (Project-specific)
+
+### Basic Config Shape
+
+```json
+{
+  "defaultProfile": "auto",
+  "classifierModel": "google/gemini-flash-latest",
+  "maxSessionBudget": 1.0,
+  "profiles": {
+    "auto": {
+      "high": { "model": "openai/gpt-5.4-pro", "thinking": "high" },
+      "medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
+      "low": { "model": "openai/gpt-5.4-nano", "thinking": "low" }
+    }
+  }
+}
+```
+
+### Configuration Fields
+
+| Field | Description |
+|-------|-------------|
+| `defaultProfile` | The profile to use when starting a new session. |
+| `classifierModel` | (Optional) Model used to categorize intent. If omitted, fast heuristics are used. |
+| `maxSessionBudget` | (Optional) USD budget for the session. Forces `medium` tier once exceeded. |
+| `largeContextThreshold` | (Optional) Token count trigger to force `high` tier for large contexts. |
+| `phaseBias` | (0.0 - 1.0) Stickiness of the current phase. Higher = more stable. Default `0.5`. |
+| `rules` | List of custom keyword rules (e.g. `{ "matches": "deploy", "tier": "high" }`). |
+| `profiles` | Map of profile definitions, each containing `high`, `medium`, and `low` tiers. |
+
+### Feature Toggles
+
+```json
+{
+  "features": {
+    "ollamaSync": true,
+    "rateLimitFallback": true,
+    "scopeShim": true,
+    "perTurnRouting": true,
+    "intentClassifier": false,
+    "costBudgeting": true,
+    "phaseMemory": true,
+    "contextCompression": true,
+    "ledgerIntegration": false,
+    "agentBusIntegration": false
+  }
+}
+```
+
+**Priority:** Project config `.pi/model-router.json` overrides user config `~/.pi/agent/model-router.json`. Both override defaults.
+
+### Progressive Enhancement Configs
+
+After installing optional extensions, copy one of these to `.pi/model-router.json`:
+
+| File | Feature |
+|------|---------|
+| `model-router.ledger.json` | Log routing decisions to qmd-ledger |
+| `model-router.agent-bus.json` | Publish model changes to agent-bus |
+| `model-router.essential.json` | All integrations enabled |
+
+## ⌨️ Commands
+
+### Core Router Commands
+
+| Command | Description |
+|---------|-------------|
+| `/router` | Show detailed status, current profile, spend, and settings. |
+| `/router status` | Alias for `/router` (show current status). |
+| `/router profile [name]` | Switch to a profile or list available ones (enables router if off). |
+| `/router pin [prof] <t\|a>` | Pin a tier (high/medium/low/auto) for the current or specified profile. |
+| `/router fix <tier>` | Correct the *last* decision and pin that tier for the current profile. |
+| `/router thinking ...` | Override thinking levels (e.g., `/router thinking low xhigh`). |
+| `/router disable` | Disable the router and switch back to the last non-router model. |
+| `/router widget <on\|off>` | Toggle the persistent state widget (supports `toggle`). |
+| `/router debug <on\|off>` | Toggle turn-by-turn routing notifications (supports `toggle`, `clear`, `show`). |
+| `/router reload` | Hot-reload the configuration JSON. |
+| `/router help` | Show usage help for all subcommands. |
+
+### Fork-Added Commands
+
+| Command | Feature | Description |
+|---------|---------|-------------|
+| `/router ollama-sync` | ollamaSync | Manually sync Ollama models |
+| `/router fallback` | rateLimitFallback | Switch to fallback model sequence |
+| `/router restore` | rateLimitFallback | Restore cloud model |
+| `/router init` | | Scaffold default config file |
+| `/router scope apply` | scopeShim | Sync router profiles to Pi enabled models |
+| `/router scope reset` | scopeShim | Clear router profiles from Pi enabled list |
+| `/router scope show` | scopeShim | Show current Pi scoped models settings |
+
+## 🔒 Requirements
+
+- **Pi >= 0.70.2** — Required for `after_provider_response` event (rate limit detection)
+- **Ollama** — Running locally for Ollama sync and fallback
+- **Node.js >= 18** — For TypeScript compilation
+
+## 📚 Documentation
+
+- [Architecture Guide](docs/ARCHITECTURE.md): Deep dive into routing logic and modular design.
+- [Testing Guide](TESTING.md): Step-by-step testing checklist.
+- [Contributing Guide](CONTRIBUTING.md): How to contribute back to upstream.
+- [Quick Start](QUICKSTART.md): 6-step install guide.
+- [Sample Configuration](model-router.example.json): Diverse profile examples (`cheap`, `deep`, `balanced`).
+- [Learnings](LEARNINGS.md): Development insights and best practices.
+
+## 🤝 Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for:
+
+- How to split changes into upstream-friendly PRs
+- Commit message conventions
+- Testing requirements
+- What to say in PR descriptions
+
+## 📜 License
+
+MIT — See [LICENSE](LICENSE).
+
+## 🙏 Acknowledgements
+
+- **[yeliu84/pi-model-router](https://github.com/yeliu84/pi-model-router)**: The original author and architecture behind the `router` provider.
+- **[shouvik12/trooper](https://github.com/shouvik12/trooper)**: Inspiration for robust, transparent HTTP rate-limit fallback triggers.

package/TESTING.md

@@ -0,0 +1,374 @@
+# 🧪 Testing Checklist - pi-model-router Fork
+
+## Pre-requisites
+
+- [ ] **Pi updated** to 0.67+ (for `after_provider_response` event support)
+- [ ] **Node.js** 18+ available
+- [ ] **Ollama installed and running**
+- [ ] At least one Ollama model pulled locally
+
+## Installation Steps
+
+```bash
+# Install from github
+pi install git:github.com/kylebrodeur/pi-model-router@main
+
+# Start Pi session
+pi
+
+# Scaffold config
+/router init
+/router reload
+```
+
+## Test 1: Basic Router Functionality (Upstream Baseline)
+
+**Goal:** Verify upstream routing still works with our changes.
+
+```bash
+# Start Pi with router extension
+pi
+```
+
+- [ ] Pi starts without errors
+- [ ] Console shows `[router] Feature sync complete`
+- [ ] Status bar shows router info
+
+```
+# In Pi TUI
+/router status
+```
+
+- [ ] Shows current profile, tier, model
+- [ ] Shows `Router enabled: true`
+
+### Test Routing Decision
+
+```
+# Switch to router profile
+/router profile balanced
+```
+
+- [ ] Model switches to `router/balanced`
+- [ ] Status bar updates
+
+```
+# Ask a simple question (should route to low tier)
+What is 2+2?
+```
+
+- [ ] Request uses low-tier model
+
+```
+# Ask a complex design question (should route to high tier)
+Design a full-stack app architecture with React, Node, and PostgreSQL
+```
+
+- [ ] Request uses high-tier model
+
+## Test 2: Ollama Auto-Sync
+
+**Goal:** Verify new Ollama models are detected and added.
+
+### Step 1: Pull a New Model
+
+```bash
+# In another terminal
+ollama pull llama3.2:3b
+```
+
+### Step 2: Trigger Sync in Pi
+
+```
+# In Pi TUI
+/router ollama-sync
+```
+
+**Expected:**
+
+- [ ] Notification: `[Router] Added 1 model(s)`
+- [ ] Notification: `Run /reload to see: llama3.2:3b`
+
+### Step 3: Reload and Verify
+
+```
+/reload
+```
+
+**Expected:**
+
+- [ ] Extension reloads
+- [ ] New model appears in `/model` selector under "ollama"
+
+### Step 4: Auto-Sync on Session Start
+
+```
+/model  # Switch to a non-router model
+/new     # Start new session
+```
+
+**Expected:**
+
+- [ ] On session start, Ollama sync runs automatically
+- [ ] If new models exist, notifications appear
+- [ ] Console log: `[router] ollama-sync: feature enabled`
+
+### Step 5: Project-Level Config Override
+
+```bash
+# In your project directory
+echo '{ "features": { "ollamaSync": false } }' > .pi/model-router.json
+```
+
+```
+# In Pi TUI (from project directory)
+/router
+```
+
+**Expected:**
+
+- [ ] Auto-sync doesn't run on session start
+- [ ] Manual `/router ollama-sync` still works
+
+## Test 3: Rate Limit Fallback
+
+**Goal:** Verify rate limit detection and manual fallback.
+
+### Step 1: Verify Rate Limit Monitoring
+
+**Note:** Real rate limits are hard to trigger intentionally. This requires actual API usage.
+
+```
+# In Pi TUI, ensure tracking is active
+/router status
+```
+
+**Expected:**
+
+- [ ] Status output includes rate limit or fallback info if relevant.
+
+### Step 2: Manual Fallback
+
+```
+# Switch to a cloud model first
+/model anthropic/claude-sonnet-4
+
+# Then trigger manual fallback
+/router fallback
+```
+
+**Expected:**
+
+- [ ] Switches to fallback model (best available matching sequence)
+- [ ] Status bar shows `🏠 fallback`
+- [ ] Console log shows fallback model name
+
+### Step 3: Restore
+
+```
+/router restore
+```
+
+**Expected:**
+
+- [ ] Restores to original cloud model
+- [ ] Status bar clears `🏠 fallback`
+
+### Step 4: Feature Disabled
+
+```bash
+echo '{ "features": { "rateLimitFallback": false } }' > ~/.pi/agent/model-router.json
+```
+
+```
+/reload
+```
+
+**Expected:**
+
+- [ ] Console shows `[router] rate-limit-fallback: disabled`
+- [ ] Status bar still shows router info but no fallback indicator
+
+## Test 4: Feature Toggles (Config Merging)
+
+### Step 1: User-Level Config
+
+```bash
+cat > ~/.pi/agent/model-router.json << 'EOF'
+{
+  "features": {
+    "ollamaSync": true,
+    "rateLimitFallback": true,
+    "perTurnRouting": true
+  }
+}
+EOF
+```
+
+### Step 2: Project-Level Override
+
+```bash
+mkdir -p .pi
+cat > .pi/model-router.json << 'EOF'
+{
+  "features": {
+    "ollamaSync": false,
+    "rateLimitFallback": false
+  }
+}
+EOF
+```
+
+### Step 3: Verify
+
+```
+# In Pi TUI from project directory
+/router
+```
+
+**Expected:**
+
+- [ ] Console shows `[router] rate-limit-fallback: disabled`
+- [ ] Console shows `[router] ollama-sync: disabled`
+- [ ] Per-turn routing still works (project didn't override it)
+
+## Test 5: Edge Cases
+
+### Missing Ollama
+
+```
+# Stop Ollama
+ollama stop  # or kill process
+
+# Try sync
+/router ollama-sync
+```
+
+**Expected:**
+
+- [ ] Notification: `Ollama not available`
+- [ ] No crash, graceful failure
+
+### Missing models.json
+
+```bash
+mv ~/.pi/agent/models.json ~/.pi/agent/models.json.bak
+```
+
+```
+/router ollama-sync
+```
+
+**Expected:**
+
+- [ ] Notification: `models.json not found`
+
+```bash
+# Restore
+mv ~/.pi/agent/models.json.bak ~/.pi/agent/models.json
+```
+
+### No Ollama Models Configured
+
+```bash
+# Temporarily move models.json ollama section
+```
+
+```
+/router fallback
+```
+
+**Expected:**
+
+- [ ] `No Ollama models available` error notification
+
+## Test 6: Combined Features (Full Workflow)
+
+**Goal:** Verify all features work together.
+
+```bash
+# Pull new model
+ollama pull qwen3.5:7b
+```
+
+```
+# Pi session
+/new
+```
+
+**Expected:**
+
+1. [ ] Session starts
+2. [ ] Auto-sync detects `qwen3.5:7b`
+3. [ ] Notification: `Added 1 model(s)`
+4. [ ] `/reload` prompt shown
+5. [ ] After `/reload`, routing profiles work
+6. [ ] `/router profile balanced` switches to router
+7. [ ] Simple query routes to low tier (possibly Ollama)
+8. [ ] Complex query routes to high tier (cloud)
+
+## Regression Tests (Upstream Feature Checklist)
+
+- [ ] `/router` shows status
+- [ ] `/router profile <name>` switches profiles
+- [ ] `/router pin high` pins tier
+- [ ] `/router pin auto` releases pin
+- [ ] `/router disable` disables router
+- [ ] `/router debug toggle` enables debug
+- [ ] `/router widget toggle` toggles widget
+- [ ] `/router reload` reloads config
+- [ ] Routing decisions shown in UI (if debug enabled)
+- [ ] Cost tracking works (if budgeting enabled)
+- [ ] Cost budget forces downgrade when exceeded
+- [ ] Phase memory works across turns
+- [ ] Custom rules match and override tiers
+- [ ] Context trigger forces high tier on large contexts
+
+## Troubleshooting
+
+### Extension Not Loading
+
+```bash
+# Check file paths
+ls -la ~/.pi/agent/extensions/pi-model-router/
+# Should see: index.ts, config.ts, types.ts, routing.ts, provider.ts, commands.ts, state.ts, ui.ts, constants.ts, ollama-sync.ts, rate-limit.ts, features.ts
+```
+
+### TypeScript Errors After Update
+
+```bash
+cd ~/.pi/agent/extensions/pi-model-router
+npm install  # ensure dependencies match
+./node_modules/.bin/tsc --noEmit
+```
+
+### Config Not Applying
+
+```bash
+# Verify files exist
+cat ~/.pi/agent/model-router.json
+cat .pi/model-router.json 2>/dev/null || echo "No project config"
+```
+
+### Rate Limit Not Detected
+
+```
+# Requires Pi 0.67+ — check version
+pi --version
+
+# If outdated:
+npm install -g @mariozechner/pi-coding-agent
+```
+
+### Fallback Model Not In Registry
+
+```
+/reload
+/model  # Check if new models appear
+```
+
+If Ollama models don't appear after `/reload`:
+
+1. Check `models.json` has ollama provider section
+2. Verify `baseUrl` is correct: `http://127.0.0.1:11434/v1`
+3. Check Ollama is running: `ollama list` in terminal

package/docs/ARCHITECTURE.md

@@ -0,0 +1,54 @@
+# Architecture: Pi Model Router Extension
+
+The `pi-model-router` is an extension-first model router for the `pi` coding agent. It registers a custom logical provider (`router`) that exposes "profiles" as models (e.g., `router/auto`). For every turn, the router intelligently selects an underlying concrete model based on task complexity, conversation phase, and user-defined rules.
+
+## Core Concepts
+
+### 1. Profiles & Tiers
+
+The router is organized into **Profiles** (e.g., `auto`, `cheap`, `deep`). Each profile defines three **Tiers**:
+
+- **High**: Reserved for architecture, design, complex debugging, and planning. Uses high-reasoning models.
+- **Medium**: The default for standard implementation, multi-file edits, and focused fixes.
+- **Low**: Used for summaries, changelogs, formatting, and simple read-only lookups.
+
+### 2. Custom Provider Implementation
+
+The extension uses `pi.registerProvider` to hook into the `pi` model lifecycle. This ensures that the selected model in the `pi` footer remains stable (e.g., `router/auto`) while the underlying model changes transparently turn-by-turn via the `streamSimple` interception.
+
+## Routing Decision Flow
+
+For every request sent to a `router/*` model, the following logic is executed:
+
+1. **Budget Check**: If a `maxSessionBudget` is configured and the session spend exceeds it, the router automatically downgrades `high` tier requests to `medium`.
+2. **Context Trigger**: If `largeContextThreshold` is exceeded (measured in tokens), the router forces the `high` tier to ensure the model can handle the large context.
+3. **Manual Pin**: If the user has pinned a tier via `/router pin` or `/router fix`, that tier is used.
+4. **Custom Rules**: Keyword-based rules defined in the config are checked against the user prompt.
+5. **LLM Classifier (Optional)**: If `classifierModel` is configured, a fast LLM is called to categorize the user's intent.
+6. **Heuristics (Fallback)**: If the classifier is off or fails, a fast local heuristic (keyword/length/tool-use analysis) is used.
+7. **Biased Stickiness**: The `phaseBias` setting modulates thresholds to keep the router in a consistent phase (e.g., staying in `high` tier during a multi-turn planning session).
+
+## Module Architecture
+
+The extension is modularized for maintainability:
+
+- `extensions/index.ts`: Orchestrator. Manages state, hooks into `pi` events, and wires modules together.
+- `extensions/provider.ts`: Implements the `router` provider and the delegation/retry loop.
+- `extensions/routing.ts`: Core decision logic, heuristics, and the LLM classifier.
+- `extensions/config.ts`: Loads, merges, and normalizes the JSON configuration.
+- `extensions/commands.ts`: Registers all `/router` subcommands and their autocompletions.
+- `extensions/ui.ts`: Manages the status line and the optional state widget.
+- `extensions/state.ts`: Handles session-persisted state and snapshots.
+- `extensions/types.ts`: Centralized interface and type definitions.
+
+## State & Persistence
+
+The router state is persisted using `pi.appendEntry` with a custom type `router-state`. This allows the router to:
+
+- Restore the active profile and pins across agent relaunches.
+- Maintain independent pins and state for different conversation branches.
+- Track accumulated session costs safely.
+
+## Reliability: Fallback Chains
+
+Each tier in a profile can define an optional `fallbacks` list. If the primary model fails (e.g., due to rate limits or provider downtime), the router automatically retries the next model in the chain before surfacing an error to the user.