@huseynsnmz/pi-suite 0.0.0-stage → 0.2.0

package/packages/router/README.md

@@ -0,0 +1,90 @@
+# pi-router
+
+Part of [pi-suite](../../README.md). Turn it off with `/suite off router`.
+
+Adds `router/auto` to [Pi](https://pi.dev): one choice in `/model` that uses a strong model to plan and a cheaper one to build, and gives the page reader, the permission judge and subagents their own models.
+
+## Quick start
+
+Requires Pi 1.5 or newer.
+
+```bash
+pi install npm:@huseynsnmz/pi-suite@0.1.0
+```
+
+Log in to the providers you want with `/login`, run `/router` to choose a model for each role, then pick `router/auto` in `/model`.
+
+## Roles
+
+| Role | What it does | With none set |
+|---|---|---|
+| `plan` | Reads and plans, up to the first edit, and everything in `plan` mode | Required |
+| `build` | Everything after that, and compaction summaries | Required |
+| `reader` | Reads web pages for [pi-web](https://github.com/huseynsnmz/pi-suite/tree/main/packages/web) | Same as `build` |
+| `judge` | Decides permission asks for `pi-permission-classifier` | See below |
+| `subagent` | Runs subagents that use `router/auto` | Same as `build` |
+| `fallback` | Takes over when a provider is overloaded or rate limited | Retries on the same model |
+
+`/router` saves them in `~/.pi/agent/router.json`, apart from `judge`, which goes in the judge's own config:
+
+```json
+{
+  "plan": "anthropic/claude-opus-5-5",
+  "build": "anthropic/claude-haiku-5-5",
+  "fallback": "openai/gpt-5.6-luna"
+}
+```
+
+There's no setting per project, so a repository can't choose where your code gets sent. The thinking level you pick with `/effort` applies to every role, and Pi lowers it for a model that can't do it.
+
+## How it decides
+
+```
+ new session ──> plan model ──── first successful edit ───> build model
+                     ▲                                          │
+                     └── plan mode on ──────────────────────────┤
+                                                                │
+                     leaving plan mode ─────────────────────────┘
+```
+
+- Moving from `plan` to `build` happens once in a session. It loses the prompt cache one time, as any change of model does.
+- The session stays on `build` after that. For a hard follow-up, switch to `plan` mode, or start a new session.
+- If a request fails because the provider is overloaded, it moves to `fallback` for the rest of that turn. The next message tries the first choice again. Any other failure retries on the same model.
+- A role whose model is missing or has no login stops that request with an error naming it, and so does a broken `router.json`. The router doesn't pick another model for you.
+- Leaving `plan` mode counts as finishing planning, whether you approved a plan or changed mode.
+- The roles only apply while `router/auto` is selected. On any other model, nothing goes to them, and pages are read as if the router weren't there.
+- `/suite off router` moves the session to the model that last answered. If `router/auto` is your saved default, later sessions use another available model, as Pi does for any model that no longer exists.
+
+The phase follows the session tree, so a branch keeps its own.
+
+## With other extensions
+
+**`modes`:** tells the router the mode. With it off, the router only goes by the first edit.
+
+**`web`:** your `readerModel` in `web.json` still wins over the `reader` role.
+
+**`frame`:** shows who answered, such as `Auto → Haiku`. Pi's own footer already does.
+
+**`pi-permission-classifier`:** it calls its model directly, which can't work with a virtual model. With `router/auto` selected and no judge model, every ask comes to you, and the router warns you. Setting `judge` in `/router` writes the model into the judge's own config, changing only its `provider` and `model`, so `/permission-model` shows the same choice. Run `/reload` afterwards. The `judge` role doesn't have to be the cheap model, and I'd choose carefully, because it decides what runs in `auto`. A project's own judge config can still override it.
+
+**`pi-subagents`:** a subagent uses the `subagent` role only if its model is `router/auto`, for example through `model: "inherit"`. A subagent with its own model ignores the router. The router recognises a subagent by `PI_SUBAGENT_CHILD=1`, and that was tested by setting the variable rather than through a real subagent.
+
+**`pi-board`:** its model list includes `router/auto`, but sessions started that way haven't been tested.
+
+## Limits
+
+- **It isn't a cost limit.** It chooses models and doesn't stop spending.
+- **Changing model mid-session can lower quality.** The build model sees the plan model's work, but not its private reasoning.
+- **It doesn't judge how hard a task is.** A one-line change still starts on the plan model.
+- **Another extension that calls the virtual model directly gets the `build` model.**
+
+## Development
+
+```bash
+npm install --ignore-scripts
+npm run check
+npm test
+npm run test:e2e
+```
+
+The end-to-end run drives real Pi in a real terminal with a fake model, using all of pi-suite and the real permission extensions.

package/packages/router/package.json

@@ -0,0 +1,51 @@
+{
+  "name": "@huseynsnmz/pi-router",
+  "version": "0.1.0",
+  "description": "A virtual model for Pi: a strong model to plan, a cheap one to build, and one model each for the page reader, the judge and subagents.",
+  "type": "module",
+  "license": "MIT",
+  "keywords": [
+    "pi-package",
+    "pi-extension",
+    "virtual-model",
+    "model-routing"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/huseynsnmz/pi-suite.git",
+    "directory": "packages/router"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "files": [
+    "src",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "pi": {
+    "extensions": [
+      "./src/index.ts"
+    ]
+  },
+  "engines": {
+    "node": ">=22.18"
+  },
+  "scripts": {
+    "check": "tsc --noEmit",
+    "test": "node --test test/*.test.ts",
+    "test:e2e": "python3 test/e2e/run.py"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-ai": "*",
+    "@earendil-works/pi-coding-agent": "*"
+  },
+  "devDependencies": {
+    "@earendil-works/pi-ai": "1.5.0",
+    "@earendil-works/pi-coding-agent": "1.5.0",
+    "@types/node": "26.6.6",
+    "typescript": "7.0.2"
+  },
+  "private": true
+}

package/packages/router/src/config.ts

@@ -0,0 +1,65 @@
+// ~/.pi/agent/router.json. Global only: a repository must not choose where your code gets sent.
+import { existsSync, readFileSync, realpathSync, renameSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+
+/** The judge's model lives only in the judge's own config, so there's one answer to which model judges. */
+export const ROLES = ["plan", "build", "reader", "subagent", "fallback"] as const;
+export type Role = (typeof ROLES)[number];
+
+export interface RouterConfig {
+  plan: string;
+  build: string;
+  reader?: string;
+  subagent?: string;
+  fallback?: string;
+}
+
+export type SavedConfig = Partial<Record<Role, string>>;
+
+export const VIRTUAL_PROVIDER = "router";
+export const VIRTUAL_ID = "auto";
+
+export function configPath(agentDir: string): string {
+  return join(agentDir, "router.json");
+}
+
+export function parseRef(ref: string): { provider: string; id: string } {
+  const at = ref.indexOf("/");
+  return { provider: ref.slice(0, at), id: ref.slice(at + 1) };
+}
+
+/** Throws on anything it doesn't understand, so a typo can't quietly send work to the wrong model. */
+export function parseConfig(raw: unknown): SavedConfig {
+  if (raw === null || typeof raw !== "object" || Array.isArray(raw)) throw new Error("router.json must be an object");
+  for (const [role, ref] of Object.entries(raw)) {
+    if (!(ROLES as readonly string[]).includes(role)) throw new Error(`router.json: unknown role "${role}". Use ${ROLES.join(", ")}`);
+    if (typeof ref !== "string" || !/^[^/\s]+\/\S+$/.test(ref)) throw new Error(`router.json: ${role} must look like "provider/model"`);
+    if (ref.startsWith(`${VIRTUAL_PROVIDER}/`)) throw new Error(`router.json: ${role} can't be a virtual model`);
+  }
+  return raw as SavedConfig;
+}
+
+export function isComplete(saved: SavedConfig): saved is RouterConfig {
+  return saved.plan !== undefined && saved.build !== undefined;
+}
+
+/** An absent file is a router nobody has set up yet. A broken one throws. */
+export function loadConfig(agentDir: string): SavedConfig {
+  let text: string;
+  try {
+    text = readFileSync(configPath(agentDir), "utf8");
+  } catch (error) {
+    if ((error as NodeJS.ErrnoException).code === "ENOENT") return {};
+    throw error;
+  }
+  return parseConfig(JSON.parse(text));
+}
+
+export function saveConfig(agentDir: string, saved: SavedConfig): void {
+  // A linked file, as in a dotfiles setup, stays linked.
+  const named = configPath(agentDir);
+  const path = existsSync(named) ? realpathSync(named) : named;
+  const temp = `${path}.${process.pid}.tmp`;
+  writeFileSync(temp, `${JSON.stringify(saved, null, 2)}\n`);
+  renameSync(temp, path);
+}

package/packages/router/src/index.ts

@@ -0,0 +1,172 @@
+import { type ExtensionAPI, type ExtensionContext, getAgentDir } from "@earendil-works/pi-coding-agent";
+
+import { isComplete, loadConfig, parseConfig, parseRef, type Role, ROLES, type SavedConfig, saveConfig, VIRTUAL_ID, VIRTUAL_PROVIDER } from "./config.ts";
+import { judgeModel, setJudgeModel } from "./judge.ts";
+import { chooseModel, type RouterState } from "./route.ts";
+
+/** Other extensions emit these with an object, and whoever knows fills it in, whatever the load order. */
+const READER_PROBE = "pi-router:reader-model";
+const MODE_PROBE = "pi-modes:mode";
+
+const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh", "max"] as const;
+const VIRTUAL_API = "pi-virtual";
+
+const JUDGE = "judge";
+const JUDGE_HELP = "decides permission asks for pi-permission-classifier";
+const ROLE_HELP: Record<Role, string> = {
+  plan: "reads and plans, until the first edit, and all of plan mode",
+  build: "the rest of the work, and compaction summaries",
+  reader: "reads web pages for pi-web",
+  subagent: "runs subagents that use this virtual model",
+  fallback: "takes over when a provider is overloaded",
+};
+
+/** What a role shows, and offers to go back to, with no model of its own. */
+const UNSET: Partial<Record<Role, string>> = {
+  reader: "same as build",
+  subagent: "same as build",
+  fallback: "none",
+};
+
+function refOf(model: { provider: string; id: string } | undefined): string | undefined {
+  return model === undefined ? undefined : `${model.provider}/${model.id}`;
+}
+
+export default function piRouter(pi: ExtensionAPI): void {
+  const agentDir = getAgentDir();
+  let saved: SavedConfig = {};
+  let problem: string | undefined;
+  try {
+    saved = loadConfig(agentDir);
+  } catch (error) {
+    problem = `pi-router is off: ${(error as Error).message}`;
+  }
+  let registered = false;
+  let warnedJudge = false;
+
+  pi.events.on(READER_PROBE, (data) => {
+    if (!isComplete(saved)) return;
+    (data as { model?: string }).model = saved.reader ?? saved.build;
+  });
+
+  function currentMode(): string | undefined {
+    const probe: { mode?: string } = {};
+    pi.events.emit(MODE_PROBE, probe);
+    return probe.mode;
+  }
+
+  function register(): void {
+    // A broken file still registers, so router/auto stops with the reason rather than Pi picking another model.
+    if (registered || (!isComplete(saved) && problem === undefined)) return;
+    registered = true;
+    pi.registerVirtualModel<RouterState>({
+      provider: VIRTUAL_PROVIDER,
+      id: VIRTUAL_ID,
+      name: "Auto",
+      thinkingLevels: [...THINKING_LEVELS],
+      route(request, ctx) {
+        if (problem !== undefined) throw new Error(problem);
+        if (!isComplete(saved)) throw new Error("pi-router has no plan and build models. Run /router");
+        const choice = chooseModel(saved, {
+          reason: request.reason,
+          mode: currentMode(),
+          subagent: process.env.PI_SUBAGENT_CHILD === "1",
+          state: request.state,
+          previous: refOf(request.previous?.model),
+          failed: request.failed && { model: refOf(request.failed.model) as string, errorMessage: request.failed.message.errorMessage },
+          messages: request.messages,
+        });
+        const { provider, id } = parseRef(choice.model);
+        const model = ctx.modelRegistry.find(provider, id);
+        if (model === undefined) throw new Error(`pi-router: ${choice.model} isn't a known model. Run /router`);
+        if (!ctx.modelRegistry.hasConfiguredAuth(model)) throw new Error(`pi-router: no login for ${choice.model}. Run /login or /router`);
+        return { model, thinkingLevel: request.thinkingLevel, state: choice.state };
+      },
+    });
+  }
+  register();
+
+  /** With a virtual model selected and no model of its own, the judge fails and every ask comes to you. */
+  function warnAboutJudge(ctx: ExtensionContext): void {
+    if (warnedJudge || !ctx.hasUI || refOf(ctx.model) !== `${VIRTUAL_PROVIDER}/${VIRTUAL_ID}`) return;
+    if (judgeModel(agentDir) !== null) return;
+    warnedJudge = true;
+    ctx.ui.notify("The permission judge has no model of its own, so with router/auto it can't judge and every ask comes to you. Set judge in /router.", "warning");
+  }
+
+  pi.on("session_start", (_event, ctx) => {
+    register();
+    if (problem !== undefined && ctx.hasUI) ctx.ui.notify(problem, "error");
+    warnAboutJudge(ctx);
+  });
+  // Pi keeps virtual models across a reload, so one left here would outlive this extension being turned off.
+  pi.on("session_shutdown", () => {
+    if (!registered) return;
+    registered = false;
+    pi.unregisterVirtualModel(VIRTUAL_PROVIDER, VIRTUAL_ID);
+  });
+  pi.on("model_select", (_event, ctx) => warnAboutJudge(ctx));
+
+  function setRole(role: Role, ref: string | undefined): void {
+    // Another session may have changed it since this one read it.
+    const next: SavedConfig = { ...loadConfig(agentDir) };
+    if (ref === undefined) delete next[role];
+    else next[role] = ref;
+    saved = parseConfig(next);
+    saveConfig(agentDir, saved);
+    register();
+  }
+
+  async function chooseJudge(ctx: ExtensionContext, models: readonly string[]): Promise<void> {
+    const picked = await ctx.ui.select(`${JUDGE} ${JUDGE_HELP}`, ["not set", ...models]);
+    if (picked === undefined) return;
+    try {
+      setJudgeModel(agentDir, picked === "not set" ? undefined : picked);
+      ctx.ui.notify("Judge model set. Run /reload for the judge to use it.", "info");
+    } catch (error) {
+      ctx.ui.notify(`Couldn't set the judge, so nothing changed. Is pi-permission-classifier set up? ${(error as Error).message}`, "error");
+    }
+  }
+
+  async function chooseRoles(ctx: ExtensionContext): Promise<void> {
+    if (problem !== undefined) {
+      // Saving over it would throw away whatever the file held.
+      ctx.ui.notify(`${problem}. Fix or delete ~/.pi/agent/router.json, then /reload.`, "error");
+      return;
+    }
+    const models = ctx.modelRegistry.getAvailable().filter((model) => model.api !== VIRTUAL_API).map((model) => `${model.provider}/${model.id}`);
+    if (models.length === 0) {
+      ctx.ui.notify("No models to choose from. Run /login first.", "warning");
+      return;
+    }
+    for (;;) {
+      const judge = judgeModel(agentDir);
+      const rows = [...ROLES.map((role) => `${role}: ${saved[role] ?? UNSET[role] ?? "not set"}`), `${JUDGE}: ${judge ?? (judge === null ? "not set" : "not installed")}`];
+      const row = await ctx.ui.select("Router: which role? (Esc when done)", rows);
+      if (row === undefined) break;
+      const role = ROLES[rows.indexOf(row)];
+      if (role === undefined) {
+        await chooseJudge(ctx, models);
+        continue;
+      }
+      const unset = UNSET[role];
+      const picked = await ctx.ui.select(`${role} ${ROLE_HELP[role]}`, [...(unset === undefined ? [] : [unset]), ...models]);
+      if (picked === undefined) continue;
+      try {
+        setRole(role, picked === unset ? undefined : picked);
+      } catch (error) {
+        ctx.ui.notify((error as Error).message, "error");
+      }
+    }
+    if (isComplete(saved)) ctx.ui.notify(`Choose ${VIRTUAL_PROVIDER}/${VIRTUAL_ID} in /model to use these.`, "info");
+    else ctx.ui.notify("The router needs both plan and build before router/auto appears.", "warning");
+  }
+
+  pi.registerCommand("router", {
+    description: "Choose which model does what for router/auto",
+    handler: async (_args, ctx) => {
+      if (!ctx.hasUI) return;
+      await chooseRoles(ctx);
+    },
+  });
+}

package/packages/router/src/judge.ts

@@ -0,0 +1,51 @@
+// pi-permission-classifier calls its model directly, which can't work with a virtual model, and it
+// has no way to ask another extension. So its own setting holds the judge's model.
+import { fchmodSync, closeSync, fsyncSync, openSync, readFileSync, realpathSync, renameSync, statSync, writeSync } from "node:fs";
+import { join } from "node:path";
+
+import { parseRef } from "./config.ts";
+
+function judgeConfigPath(agentDir: string): string {
+  return join(agentDir, "extensions", "pi-permission-classifier", "config.json");
+}
+
+function readObject(path: string): Record<string, unknown> {
+  const parsed: unknown = JSON.parse(readFileSync(path, "utf8"));
+  if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) throw new Error(`${path} is not a JSON object`);
+  return parsed as Record<string, unknown>;
+}
+
+/** Undefined when the judge isn't set up here, so there's nothing to wire. */
+export function judgeModel(agentDir: string): string | null | undefined {
+  let config: Record<string, unknown>;
+  try {
+    config = readObject(judgeConfigPath(agentDir));
+  } catch {
+    return undefined;
+  }
+  return typeof config.provider === "string" && typeof config.model === "string" ? `${config.provider}/${config.model}` : null;
+}
+
+/** Changes only these two settings. Throws, touching nothing, if the file is absent or broken. */
+export function setJudgeModel(agentDir: string, ref: string | undefined): void {
+  const target = realpathSync(judgeConfigPath(agentDir));
+  const config = readObject(target);
+  delete config.provider;
+  delete config.model;
+  if (ref !== undefined) {
+    const { provider, id } = parseRef(ref);
+    config.provider = provider;
+    config.model = id;
+  }
+  const mode = statSync(target).mode & 0o777;
+  const temp = `${target}.${process.pid}.tmp`;
+  const file = openSync(temp, "w", mode);
+  try {
+    writeSync(file, `${JSON.stringify(config, null, 2)}\n`);
+    fchmodSync(file, mode);
+    fsyncSync(file);
+  } finally {
+    closeSync(file);
+  }
+  renameSync(temp, target);
+}

package/packages/router/src/route.ts

@@ -0,0 +1,67 @@
+// Which real model answers a request. Pure, so every rule can be tested without Pi.
+import type { RouterConfig } from "./config.ts";
+
+export interface RouterState {
+  phase: "plan" | "build";
+  /** Plan mode was on. Leaving it, by approving a plan or changing mode, starts building. */
+  planned?: true;
+}
+
+export interface RouteInput {
+  reason: "user" | "continuation" | "retry" | "direct";
+  /** From pi-modes, when it is installed. */
+  mode: string | undefined;
+  /** This Pi was started by pi-subagents. */
+  subagent: boolean;
+  state: RouterState | undefined;
+  previous: string | undefined;
+  failed: { model: string; errorMessage: string | undefined } | undefined;
+  messages: readonly { role: string; toolName?: string; isError?: boolean }[];
+}
+
+export interface RouteChoice {
+  model: string;
+  /** Only when it changed, because Pi stores every new object it gets. */
+  state?: RouterState;
+}
+
+const OVERLOADED = /overload|rate.?limit|too many requests|capacity|\b(429|503|529)\b/i;
+/** Includes pi-modes' yolo_edit and yolo_write. */
+const CHANGES_FILES = /(^|_)(edit|write)$/;
+
+/** Anywhere in the conversation, so a message sent right after an edit can't hide it. */
+export function hasEdited(messages: RouteInput["messages"]): boolean {
+  return messages.some((message) => message.role === "toolResult" && !message.isError && CHANGES_FILES.test(message.toolName ?? ""));
+}
+
+function nextState(input: RouteInput): RouterState {
+  const state = input.state ?? { phase: "plan" };
+  if (input.mode === "plan") return state.phase === "plan" && state.planned ? state : { phase: "plan", planned: true };
+  if (state.phase === "plan" && (state.planned || hasEdited(input.messages))) return { phase: "build" };
+  return state;
+}
+
+export function chooseModel(config: RouterConfig, input: RouteInput): RouteChoice {
+  const home = input.subagent ? (config.subagent ?? config.build) : config.build;
+  // Compaction summaries, and any other extension's call that names the virtual model.
+  if (input.reason === "direct") return { model: home };
+
+  // Worked out first, so an edit made on the fallback or during a retry still counts.
+  const state = input.subagent ? input.state : nextState(input);
+  const changed = state === input.state ? {} : { state };
+
+  if (input.reason === "retry" && input.failed) {
+    const overloaded = OVERLOADED.test(input.failed.errorMessage ?? "");
+    const moves = overloaded && config.fallback !== undefined && config.fallback !== input.failed.model;
+    return { model: moves ? (config.fallback as string) : input.failed.model, ...changed };
+  }
+
+  // Back on the first choice at the next message, not on every tool call in between.
+  if (input.reason === "continuation" && input.previous !== undefined && input.previous === config.fallback) {
+    return { model: input.previous, ...changed };
+  }
+
+  if (input.subagent) return { model: home };
+  const model = input.mode === "plan" || state?.phase === "plan" ? config.plan : config.build;
+  return { model, ...changed };
+}

package/packages/web/CHANGELOG.md

@@ -0,0 +1,36 @@
+# Changelog
+
+## [0.1.3]
+
+### Changes
+
+- **Virtual models**: pages can be read when the session's model is a virtual one, and the `/web reader` list leaves them out. With [pi-router](https://github.com/huseynsnmz/pi-router), the reader is the model it chooses, unless you set `readerModel`.
+
+## [0.1.2] - 2026-11-04
+
+### Changes
+
+- **web_fetch**: the agent can follow any link on a page it has read, and write its own page addresses on sites you named or a search returned. It used to be refused anything you hadn't written or the tools hadn't returned, which stopped research. A new site asks you once, as does an address with a query string or encoded-looking text. `trustedDomains` skips the question for sites you choose, and `strictUrls` asks about every URL it wasn't given
+- **site filter**: links in search results and answers are checked as well, so the agent isn't pointed at a blocked site
+
+## [0.1.1] - 2026-11-04
+
+### Features
+
+- **reader**: `/web reader` picks the page reader from your models, saves it to `web.json`, and applies it straight away
+
+## [0.1.0] - 2026-11-04
+
+### Features
+
+- **web_fetch**: finds something out from a page. A separate model with no tools reads it, and the agent only gets the answer
+- **web_search**: searches through your own SearXNG server, or `markdown.new` if `companySearch` is on. It's off until you choose one
+- **confidence**: the reader scores its answer from 1 to 10, and an answer below `minConfidence` is withheld
+- **site filter**: Cloudflare's filtering DNS blocks known malware and adult sites before a fetch. It can be set to malware only, or off
+
+### Security
+
+- The agent can only fetch URLs you wrote or these tools returned, matched exactly
+- Queries holding secrets, paths, email addresses, encoded text or your `privateWords` are refused
+- Local and private addresses are refused, and checked again when connecting
+- A wrong setting in `web.json` turns the web tools off

package/packages/web/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Hüseyin Sönmez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/packages/web/README.md

@@ -0,0 +1,87 @@
+# pi-web
+
+Part of [pi-suite](../../README.md). Turn it off with `/suite off web`.
+
+Web search and fetch for [Pi](https://pi.dev), built so that the agent never reads the web.
+
+A web page can hold instructions aimed at the AI reading it. So a separate model with no tools reads every page and search result, and the agent only gets its answer. If that model isn't available, the page isn't read.
+
+## Install
+
+```bash
+pi install npm:@huseynsnmz/pi-suite@0.1.0
+```
+
+`web_fetch` works straight away. It reads pages with the session's model, unless you set `readerModel` or use [pi-router](https://github.com/huseynsnmz/pi-suite/tree/main/packages/router), which can choose the reader. `web_search` stays off until you choose where searches go.
+
+## Search
+
+Everything you search for leaves your machine, so nothing is chosen for you. Settings go in `~/.pi/agent/web.json`, and are read when Pi starts, so restart it after editing the file.
+
+**Your own SearXNG server** needs no account, but it does pass your searches on to the search engines it uses:
+
+```json
+{ "searxngUrl": "http://localhost:8888" }
+```
+
+SearXNG only answers in JSON if you list it under `search.formats` in its `settings.yml`. Public SearXNG servers almost always refuse this.
+
+**markdown.new** is free and needs no key: 30 searches a minute and 500 a day for your IP, with a shared cap across everyone using it. Your searches go to its operator, then Serper and Google. This is behind a flag:
+
+```json
+{ "companySearch": true }
+```
+
+## Settings
+
+| Setting | What it does |
+| --- | --- |
+| `companySearch` | Allows searching through a company's service. Off by default |
+| `provider` | `searxng` or `markdown.new`, if you've set up both |
+| `searxngUrl` | Your SearXNG server |
+| `readerModel` | The model that reads pages, as `provider/id`. A small one costs less. Defaults to the session's model |
+| `allowDomains` | If set, only these domains and their subdomains can be fetched |
+| `denyDomains` | These domains and their subdomains can't be fetched |
+| `dnsFilter` | `cloudflare-family` blocks malware and adult sites, and is the default. `cloudflare-security` blocks malware only. `off` turns it off |
+| `privateWords` | Words a search can never hold, such as client or project names |
+| `minConfidence` | Answers the reader scores below this, out of 10, are withheld. Defaults to 4 |
+| `trustedDomains` | Sites the agent can write its own addresses on without asking, such as docs you use a lot, and their subdomains. Other sites still work, with a prompt |
+| `strictUrls` | Set to `true` to be asked about every URL the agent wasn't given |
+| `askNewDomains` | Set to `true` to be asked before the first fetch from each site |
+
+`/web` shows what's in use, and `/web reader` lets you pick the page reader from your models. It saves to `web.json` and applies straight away. If `web.json` can't be read or has a wrong value, the web tools stay off and Pi says why.
+
+## What it checks
+
+**Before anything leaves:**
+
+- A query or URL that looks like it holds a secret is refused, as is a query holding a long encoded string.
+- The agent can fetch URLs you wrote, and links from `web_search` and `web_fetch` pages. It can also write its own page addresses on sites you named, sites a search returned, and sites in `trustedDomains`.
+- Any URL can carry data to whoever runs that site, so you get asked once before it writes an address on a new site, and about any address with a query string or encoded-looking text, or after 25 addresses of its own on one site. With nobody to ask, it's refused. `strictUrls` asks about every URL it wasn't given.
+- Local and private addresses are refused, checked again at connection time, so a public name can't point at your network.
+- A redirect to another host is reported, not followed. Fetching it goes through the same checks.
+- Each site's name is checked with Cloudflare's filtering DNS before it's fetched, and links in search results and answers are checked before the agent sees them. If the filter can't answer, the page isn't fetched. With the filter on, bare IP addresses are refused, as they have no name to check.
+
+**Before the reader sees a page:** scripts, images, forms, invisible characters, and elements hidden in the page or its own stylesheets are removed.
+
+**Before the agent sees an answer:**
+
+- The reader scores from 1 to 10 how well the page supports its answer, and the agent sees the score. Below `minConfidence` the answer is withheld. A page that fools the reader can fake this score, so it's a measure of quality, not safety.
+- Links that weren't in the page are removed.
+- An answer that reads like instructions to an AI is withheld.
+- If the page itself holds such wording, the agent is told.
+
+## What it can't do
+
+- It can't stop a page lying. A page can still fool the reader into a false answer, so treat answers as claims from a stranger.
+- It can't catch every attack. The wording checks are patterns, and text hidden by a stylesheet in another file isn't noticed. A page about prompt injection may also get its answer withheld.
+- The site filter only knows the sites on Cloudflare's lists. It says nothing about a new or unlisted site, and every site name you fetch goes to Cloudflare.
+- It can't stop every way of carrying data out. The checks catch secrets, paths and encoded text, but a few plain words in a search or a page address can still say something, on a site you've named or approved. A site you approve is trusted for the rest of the session.
+- It can't tell a private word in a query from an ordinary one unless it's in `privateWords`, so the agent could still search for a project's name.
+- The sandbox's domain list doesn't cover it, as it runs inside Pi. Use `allowDomains` and `denyDomains`.
+- It doesn't run JavaScript or read PDFs, and sites that block bots return an error.
+- It doesn't use a proxy, and it refuses `198.18.0.0/15`, so a VPN that hands out those addresses will block every fetch.
+
+## Cost
+
+Each `web_fetch` is one extra model call with up to 100,000 characters of web text, and each `web_search` one with up to 40,000. The same question on the same page within 15 minutes isn't asked again. Set `readerModel` to something small to keep the rest down.

package/packages/web/SECURITY.md

@@ -0,0 +1,13 @@
+# Security
+
+Please report a vulnerability through [GitHub's private reporting](https://github.com/huseynsnmz/pi-web/security/advisories/new), not in a public issue.
+
+These count as vulnerabilities:
+
+- Web text reaching the agent without passing through the page reader
+- The agent getting a URL it wrote past the fetch check with a query string or encoded data in it
+- Reaching a local or private address
+- Getting a blocked site past the site filter, or the site filter failing open
+- A company receiving searches while `companySearch` is off
+
+A page fooling the reader into a false answer doesn't count, as the README says this can't be prevented.