pi-bifrost 0.1.0

package/CONTEXT.md

@@ -0,0 +1,56 @@
+# pi-bifrost
+
+Query-aware model router extension for the pi coding agent. Classifies each user prompt, selects the best model from configured tiers, and switches pi's active model transparently. No native dependencies — uses pi's built-in model registry.
+
+## Language
+
+**Tier**:
+A quality/cost class of models (e.g. `frontier`, `economical`). Each tier maps to one or more model patterns. The classifier or regex rules produce a tier name; model selection then picks a candidate from that tier.
+_Avoid_: category (the code uses both; prefer tier)
+
+**Route rule**:
+A regex pattern paired with a target tier. Rules are matched case-insensitively in order. First match wins. Example: `pattern: "\\b(debug|fix)\\b"` → `model: "frontier"`.
+_Avoid_: routing rule, classification rule
+
+**Classifier**:
+The LLM-based tier selector. Invoked before regex rules. Uses a cheap model to read the prompt and respond with a tier name. Two invocation methods: direct HTTP (fast, OpenAI-compatible) or subprocess (spawns a fresh pi CLI; slower but covers any provider).
+
+**Strategy**:
+How to pick among candidates within a tier. `first` = first available candidate. `cheapest` = lowest combined input+output cost per 1M tokens. Set globally or per-tier via `categoryStrategies`.
+
+**Candidate**:
+A model that matches a tier's pattern list. Resolved from pi's model registry. A tier may list multiple patterns; each pattern may match zero or more models.
+
+**Model pattern**:
+A string that resolves to models in pi's registry. Two forms: exact `provider/id` (e.g. `anthropic/claude-opus-4-5`) or bare substring (e.g. `qwen2.5-coder` matches any provider containing that string).
+
+**Pin**:
+Freeze the current model and stop automatic routing. Triggered automatically when the user runs `/model`, or manually via `/bifrost pin`. Reversed by `/bifrost unpin`.
+
+**Config layer**:
+One of four config sources merged in order: extension default → global (`~/.pi/agent/`) → project-local (`.pi/`) → project root. Later layers override earlier ones. The rules file (`bifrost-routes.json`) is loaded separately and overrides inline `rules` in `bifrost.json`.
+
+**Fuzzy cache**:
+Persisted classification cache using Jaccard token-set similarity. Stores `(normalizedPrompt, tier)` pairs in `.pi/bifrost-cache.jsonl`. On cache hit, skips the classifier entirely. Eviction is LRU by last-use timestamp.
+
+## Example dialogue
+
+**Dev:** "User types 'help me debug this race condition in my Go backend.' What happens?"
+
+**Expert:** "Prompt enters the pipeline. First, fuzzy cache check — has Bifrost seen a similar prompt? If yes, reuse the cached tier, skip the classifier. If no, the LLM classifier reads the prompt and ideally returns `frontier` — 'debug' and 'race condition' are strong frontier signals. If the classifier is down, regex rules catch it — both 'debug' and 'race condition' hit the frontier rule. Once we have the tier, we look up the frontier model patterns, resolve candidates from the registry, apply the strategy — say `cheapest` — and switch pi's model."
+
+**Dev:** "And if nothing matches?"
+
+**Expert:** "Falls through to the default tier, configured as `economical`. Safer to waste a cheap model on a hard prompt than burn frontier credits on 'hello world'."
+
+**Dev:** "What happens when the user manually switches models with /model?"
+
+**Expert:** "Bifrost detects the `model_select` event, pins itself, and logs a warning. It stops routing until the user runs `/bifrost unpin`. The assumption: if you manually picked a model, you had a reason."
+
+**Dev:** "So the classifier costs tokens on every prompt?"
+
+**Expert:** "Only on cache misses. The classifier model is configured to be cheap — typically a local Ollama model or a free-tier cloud model. Max 20 output tokens. And the fuzzy cache means repeat or near-repeat prompts skip it entirely. With threshold 0.85, 'debug the race condition' and 'help me debug a race condition' share a cache entry."
+
+## Flagged ambiguities
+
+- **tier vs category**: The codebase uses both interchangeably. The schema.json and `guessTier()` use "tier"; `classifyPrompt()` and cache entries use "category." The config key `categoryStrategies` retains the old name for backward compatibility. Prefer "tier" in new code and docs.

package/README.md

@@ -0,0 +1,170 @@
+# Bifrost
+
+Automatic model routing for [pi](https://pi.dev). Reads your prompt, picks the right model, switches automatically. No thinking required.
+
+## Install
+
+```bash
+pi install ./pi-bifrost
+```
+
+Or test first:
+
+```bash
+pi -e ./pi-bifrost/index.ts
+```
+
+## Setup
+
+Run this once:
+
+```
+/bifrost init
+```
+
+It probes every model you have access to, finds the ones that actually work, and writes a config. Say yes when asked.
+
+Done. Bifrost is now routing your prompts.
+
+## What it does
+
+You type a prompt. Bifrost classifies it as `frontier` (hard) or `economical` (easy), picks the best available model for that tier, and switches pi's model before the prompt is sent. You just type — the model changes behind the scenes.
+
+If you manually switch models with `/model`, Bifrost pins itself and stops routing. `/bifrost unpin` to resume.
+
+## Commands
+
+| Command | What it does |
+|---------|-------------|
+| `/bifrost` | Show current status |
+| `/bifrost init` | Probe models and generate config |
+| `/bifrost probe` | Test which models actually respond |
+| `/bifrost preview <prompt>` | See what would happen without sending |
+| `/bifrost on` / `off` | Enable / disable routing |
+| `/bifrost pin` / `unpin` | Lock current model / resume routing |
+| `/bifrost reload` | Reload config after editing |
+| `/bifrost cache stats` | Show classification cache |
+| `/bifrost cache clear` | Clear classification cache |
+| `/bifrost classifier status` | Show classifier state |
+
+## Config
+
+Everything lives in `.pi/bifrost.json`. Editor autocomplete works in VS Code, Zed, Cursor.
+
+### Tiers and models
+
+```json
+{
+  "models": {
+    "economical": [
+      "opencode/deepseek-v4-flash-free",
+      "lmstudio/openai/gpt-oss-20b"
+    ],
+    "frontier": [
+      "opencode-go/glm-5.1",
+      "opencode-go/grok-4.5"
+    ]
+  }
+}
+```
+
+Each tier has a list of model patterns. `provider/id` for exact, or just `qwen` for substring match.
+
+### Strategies
+
+How Bifrost picks from the list. Set globally or per-tier:
+
+| Strategy | Picks |
+|----------|-------|
+| `cheapest` | Lowest cost (input + output) |
+| `cheapest_input` | Lowest input cost |
+| `cheapest_output` | Lowest output cost |
+| `largest_context` | Biggest context window |
+| `random` | Random pick |
+| `first` / `fastest` | First in list (init sorts by speed) |
+
+```json
+{
+  "strategy": "first",
+  "categoryStrategies": {
+    "economical": "cheapest"
+  }
+}
+```
+
+### Routing rules
+
+Regex patterns that map prompts to tiers. First match wins. Case insensitive.
+
+```json
+{
+  "rules": [
+    {
+      "pattern": "\\b(debug|fix|implement|refactor|architect)\\b",
+      "model": "frontier"
+    },
+    {
+      "pattern": "\\b(explain|summarize|format|hello)\\b",
+      "model": "economical"
+    }
+  ]
+}
+```
+
+Or use a separate `.pi/bifrost-routes.json` file — it overrides inline rules.
+
+### Classifier
+
+An LLM that reads your prompt and picks a tier. More accurate than regex, costs a few tokens. Uses a cheap model — you configure which one.
+
+```json
+{
+  "classifier": {
+    "enabled": true,
+    "model": "opencode/mimo-v2.5-free"
+  }
+}
+```
+
+If the classifier fails or is disabled, regex rules take over. Both paths cache results so repeat prompts skip the check entirely.
+
+### Debug logging
+
+```json
+{
+  "debug": { "enabled": true }
+}
+```
+
+Writes `.pi/bifrost-debug.jsonl` — one JSON line per event. Timings, tiers, decisions. Useful for understanding what Bifrost is doing.
+
+### Full config reference
+
+```json
+{
+  "$schema": "../pi-bifrost/schema.json",
+  "enabled": true,
+  "default": "economical",
+  "strategy": "first",
+  "categoryStrategies": { "economical": "cheapest" },
+  "models": { ... },
+  "rules": [ ... ],
+  "classifier": {
+    "enabled": true,
+    "model": "...",
+    "method": "auto",
+    "maxTokens": 20,
+    "temperature": 0,
+    "fallbackToRegex": true,
+    "systemPrompt": "..."
+  },
+  "cache": {
+    "enabled": true,
+    "maxEntries": 500,
+    "threshold": 0.85
+  },
+  "debug": { "enabled": false }
+}
+```
+
+Every field is optional. Config merges from: extension default → global (`~/.pi/agent/bifrost.json`) → project (`.pi/bifrost.json`).

package/bifrost-routes.json.example

@@ -0,0 +1,10 @@
+[
+  {
+    "pattern": "\\b(plan|design|architect|refactor|debug|fix|implement|complex|performance|memory leak|race condition)\\b",
+    "model": "frontier"
+  },
+  {
+    "pattern": "\\b(explain|summarize|define|what is|how to|format|lint|convert|hello|idea)\\b",
+    "model": "economical"
+  }
+]

package/bifrost.json

@@ -0,0 +1,45 @@
+{
+  "$schema": "./schema.json",
+  "enabled": true,
+  "default": "economical",
+  "strategy": "first",
+  "categoryStrategies": {
+    "economical": "cheapest"
+  },
+  "classifier": {
+    "enabled": true,
+    "model": "opencode/deepseek-v4-flash-free",
+    "method": "auto",
+    "maxTokens": 20,
+    "temperature": 0,
+    "fallbackToRegex": true
+  },
+  "cache": {
+    "enabled": true,
+    "maxEntries": 500,
+    "threshold": 0.85
+  },
+  "debug": {
+    "enabled": false
+  },
+  "models": {
+    "economical": [
+      "ollama/qwen2.5-coder:latest",
+      "lmstudio/openai/gpt-oss-20b"
+    ],
+    "frontier": [
+      "openai-codex/gpt-5.4",
+      "opencode-go/kimi-k3"
+    ]
+  },
+  "rules": [
+    {
+      "pattern": "\\b(plan|design|architect|refactor|debug|fix|implement|complex|performance|memory leak|race condition)\\b",
+      "model": "frontier"
+    },
+    {
+      "pattern": "\\b(explain|summarize|define|what is|how to|format|lint|convert|hello|idea)\\b",
+      "model": "economical"
+    }
+  ]
+}

package/cache.ts

@@ -0,0 +1,171 @@
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+
+export interface CacheEntry {
+  normalized: string;
+  category: string;
+  lastUsed: number;
+  hits: number;
+  /** Monotonic sequence number for stable eviction ordering. */
+  seq?: number;
+}
+
+export interface CacheOptions {
+  enabled?: boolean;
+  maxEntries?: number;
+  threshold?: number;
+  path?: string;
+}
+
+export const DEFAULT_MAX_ENTRIES = 500;
+export const DEFAULT_THRESHOLD = 0.85;
+
+let nextSeq = 0;
+
+export function normalize(text: string): string {
+  return text
+    .toLowerCase()
+    .replace(/[^\p{L}\p{N}]+/gu, " ")
+    .split(" ")
+    .filter(Boolean)
+    .sort()
+    .join(" ");
+}
+
+function tokenSet(text: string): Set<string> {
+  return new Set(normalize(text).split(" "));
+}
+
+export function loadCache(path: string): CacheEntry[] {
+  if (!existsSync(path)) return [];
+  try {
+    const text = readFileSync(path, "utf-8");
+    const entries = text
+      .split("\n")
+      .filter(Boolean)
+      .map((line: string) => {
+        try {
+          return JSON.parse(line) as CacheEntry;
+        } catch {
+          return undefined;
+        }
+      })
+      .filter((e): e is CacheEntry => e !== undefined);
+
+    // Seed seq counter from loaded entries to avoid collision on restart.
+    const maxSeq = entries.reduce((max: number, e: CacheEntry) => Math.max(max, e.seq ?? 0), 0);
+    if (maxSeq >= nextSeq) nextSeq = maxSeq + 1;
+
+    return entries;
+  } catch (err) {
+    console.error(`[bifrost] failed to load cache: ${err}`);
+    return [];
+  }
+}
+
+export function saveCache(path: string, entries: CacheEntry[]) {
+  try {
+    const dir = dirname(path);
+    if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
+    const lines = entries.map((e) => JSON.stringify(e)).join("\n");
+    writeFileSync(path, lines ? lines + "\n" : "", "utf-8");
+  } catch (err) {
+    console.error(`[bifrost] failed to save cache: ${err}`);
+  }
+}
+
+function evictIfNeeded(entries: CacheEntry[], maxEntries: number): CacheEntry[] {
+  if (entries.length <= maxEntries) return entries;
+  // Sort by lastUsed descending; tie-break by seq (newer entries have higher seq).
+  return [...entries]
+    .sort((a, b) => b.lastUsed - a.lastUsed || (b.seq ?? 0) - (a.seq ?? 0))
+    .slice(0, maxEntries);
+}
+
+/** Pure query — finds matching cache entry without mutation. */
+export function lookupCache(
+  entries: CacheEntry[],
+  prompt: string,
+  threshold: number,
+): CacheEntry | undefined {
+  const normalized = normalize(prompt);
+  const promptTokens = tokenSet(normalized);
+  let best: { entry: CacheEntry; score: number } | undefined;
+
+  for (const entry of entries) {
+    if (entry.normalized === normalized) return entry;
+
+    const entryTokens = tokenSet(entry.normalized);
+    if (promptTokens.size === 0 || entryTokens.size === 0) continue;
+
+    let intersection = 0;
+    for (const token of promptTokens) {
+      if (entryTokens.has(token)) intersection++;
+    }
+    const union = promptTokens.size + entryTokens.size - intersection;
+    const score = union === 0 ? 0 : intersection / union;
+    if (score >= threshold && (!best || score > best.score)) {
+      best = { entry, score };
+    }
+  }
+
+  return best?.entry;
+}
+
+/** Explicit mutation — updates LRU timestamp and hit count. */
+export function touchCacheEntry(entry: CacheEntry): void {
+  entry.lastUsed = Date.now();
+  entry.hits++;
+}
+
+/** Convenience: pure lookup composed with LRU touch.
+ * @deprecated Prefer explicit `lookupCache` + `touchCacheEntry` at call sites. */
+export function findCachedCategory(
+  entries: CacheEntry[],
+  prompt: string,
+  threshold: number,
+): string | undefined {
+  const entry = lookupCache(entries, prompt, threshold);
+  if (entry) {
+    touchCacheEntry(entry);
+    return entry.category;
+  }
+  return undefined;
+}
+
+export function updateCache(
+  entries: CacheEntry[],
+  prompt: string,
+  category: string,
+  maxEntries: number,
+): CacheEntry[] {
+  const normalized = normalize(prompt);
+  const idx = entries.findIndex((e) => e.normalized === normalized);
+
+  if (idx !== -1) {
+    const updated = [...entries];
+    updated[idx] = {
+      ...updated[idx],
+      category,
+      lastUsed: Date.now(),
+      hits: updated[idx].hits + 1,
+    };
+    return evictIfNeeded(updated, maxEntries);
+  }
+
+  return evictIfNeeded(
+    [...entries, { normalized, category, lastUsed: Date.now(), hits: 1, seq: nextSeq++ }],
+    maxEntries,
+  );
+}
+
+export function cachePath(cwd: string, configuredPath?: string): string {
+  if (configuredPath) {
+    if (configuredPath.startsWith("/")) return configuredPath;
+    if (configuredPath.startsWith("~")) {
+      return (process.env.HOME ?? "/tmp") + configuredPath.slice(1);
+    }
+    return join(cwd, configuredPath);
+  }
+  return join(cwd, ".pi", "bifrost-cache.jsonl");
+}

package/classification-pipeline.ts

@@ -0,0 +1,107 @@
+import type { ClassifierModel } from "./classifier.ts";
+import { classify as regexClassify, type RouteRule } from "./routing.ts";
+import { debug, debugMeasure } from "./debug.ts";
+
+// ── ADT result type ────────────────────────────────────────────
+
+export type ClassificationSource = "cache" | "classifier" | "regex";
+
+export type ClassificationResult =
+  | { readonly kind: "classified"; readonly tier: string; readonly source: ClassificationSource }
+  | { readonly kind: "fallback"; readonly tier: string }
+  | { readonly kind: "unclassified" };
+
+// ── Pipeline dependencies ──────────────────────────────────────
+
+/**
+ * Dependencies injected into the pipeline. All are in-process.
+ *
+ * `cacheLookup` may internally mutate its backing store for LRU
+ * tracking — this is an accepted impurity (see ADR candidate #4).
+ */
+export interface PipelineDeps {
+  /** Query cache. Returns tier or undefined. */
+  readonly cacheLookup: (text: string) => string | undefined;
+  /** Classifier models in priority order. Empty array = skip LLM. */
+  readonly classifierModels: readonly ClassifierModel[];
+  /** Invoke the LLM classifier for a single model. Returns tier or undefined. */
+  readonly classifyWithLLM: (
+    model: ClassifierModel,
+    text: string,
+    tiers: readonly string[],
+  ) => Promise<string | undefined>;
+  /** Regex routing rules. First match wins. */
+  readonly regexRules: readonly RouteRule[];
+  /** Default tier when nothing matches. */
+  readonly defaultTier: string | undefined;
+  /** Known tier names, from config.models keys. */
+  readonly tiers: readonly string[];
+}
+
+// ── Pipeline interface ─────────────────────────────────────────
+
+export interface ClassificationPipeline {
+  readonly classify: (text: string) => Promise<ClassificationResult>;
+}
+
+// ── Factory ────────────────────────────────────────────────────
+
+export function createPipeline(deps: PipelineDeps): ClassificationPipeline {
+  const {
+    cacheLookup,
+    classifierModels,
+    classifyWithLLM,
+    regexRules,
+    defaultTier,
+    tiers,
+  } = deps;
+
+  async function classify(text: string): Promise<ClassificationResult> {
+    if (tiers.length === 0) return { kind: "unclassified" };
+
+    // Stage 1: cache lookup
+    const endCache = debugMeasure("pipeline", "cache");
+    const cached = cacheLookup(text);
+    endCache({ hit: !!cached });
+    if (cached && tiers.includes(cached)) {
+      debug("pipeline", "result", { source: "cache", tier: cached });
+      return { kind: "classified", tier: cached, source: "cache" };
+    }
+
+    // Stage 2: LLM classifier — try each model in priority order
+    for (const model of classifierModels) {
+      try {
+        const endLLM = debugMeasure("pipeline", "classifier.attempt");
+        const tier = await classifyWithLLM(model, text, tiers);
+        const modelId = model.kind === "registry" ? model.model.id : model.id;
+        endLLM({ model: modelId, tier });
+        if (tier && tiers.includes(tier)) {
+          debug("pipeline", "result", { source: "classifier", tier });
+          return { kind: "classified", tier, source: "classifier" };
+        }
+      } catch (err) {
+        debug("pipeline", "classifier.error", { error: String(err) });
+        console.error(`[bifrost] classifier model failed: ${err}`);
+      }
+    }
+
+    // Stage 3: regex rules
+    const endRegex = debugMeasure("pipeline", "regex");
+    const regex = regexClassify(text, regexRules);
+    endRegex({ match: !!regex, tier: regex });
+    if (regex && tiers.includes(regex)) {
+      debug("pipeline", "result", { source: "regex", tier: regex });
+      return { kind: "classified", tier: regex, source: "regex" };
+    }
+
+    // Stage 4: default fallback
+    debug("pipeline", "result", { source: "fallback", tier: defaultTier });
+    if (defaultTier) {
+      return { kind: "fallback", tier: defaultTier };
+    }
+
+    return { kind: "unclassified" };
+  }
+
+  return { classify };
+}