@mrclrchtr/supi-web 7.1.0 → 7.2.0

package/README.md

@@ -4,11 +4,11 @@
   </a>
 </div>
 
-# @mrclrchtr/supi-web — Web Fetch and Context7 for Pi
+# @mrclrchtr/supi-web — Web Fetch, Search, and Context7 for Pi
 
 [![GitHub stars](https://img.shields.io/github/stars/mrclrchtr/supi)](https://github.com/mrclrchtr/supi/stargazers) [![npm downloads](https://img.shields.io/npm/dm/@mrclrchtr/supi-web)](https://www.npmjs.com/package/@mrclrchtr/supi-web)
 
-Adds web fetch and Context7 documentation tools to the [Pi coding agent](https://github.com/earendil-works/pi), without making you paste sources into chat.
+Adds web fetch, public web search, and Context7 documentation tools to the [Pi coding agent](https://github.com/earendil-works/pi), without making you paste sources into chat.
 
 ## What your agent gets
 
@@ -18,6 +18,7 @@ After installation, keep asking Pi normal questions. The agent can:
 - **Prefer source Markdown when available** — detect Markdown responses and common Markdown siblings before converting HTML.
 - **Handle plain-text and source files** — wrap them in fenced code blocks with a language hint when the URL provides one.
 - **Look up current library docs** — search Context7 for the right project and version, then retrieve documentation focused on the task at hand.
+- **Search public web sources** — return source titles, URLs, and source excerpts without generating an answer.
 - **Protect the context window** — return small results inline, move long pages to temporary files, and preserve full output when model-visible text must be truncated.
 
 Relative page links and images become absolute by default, so the agent can follow them without reconstructing URLs.
@@ -34,16 +35,29 @@ You do not need to learn the tool-call syntax. Try asking Pi:
 
 ## Agent tools
 
-The package adds three tools that Pi selects as needed:
+The package adds four tools that Pi selects as needed:
 
 | Tool | What it lets the agent do |
 |---|---|
 | `web_fetch_md` | Fetch a public HTTP(S) page and return readable Markdown, fenced plain text, or a temporary file |
+| `web_search` | Search the public web and return source titles, URLs, and source excerpts |
 | `web_docs_search` | Find matching Context7 library IDs, versions, trust scores, benchmark scores, and snippet counts |
 | `web_docs_fetch` | Retrieve focused documentation for a chosen Context7 library ID |
 
 Pi searches Context7 first when the library ID is unknown, then fetches docs with the selected ID. Narrow questions produce more useful documentation context than broad requests. Documentation returns as Markdown by default; the agent can request structured JSON snippets when needed.
 
+## Web Search
+
+`web_search` uses the installed `bx` executable. The tool is enabled by default, but it is available only when `bx` is on `PATH`. Search freshness accepts `pd`, `pw`, `pm`, `py`, or an ordered `YYYY-MM-DDtoYYYY-MM-DD` range.
+
+### Set up bx
+
+Install `bx` and configure it with the [Brave Search CLI instructions](https://github.com/brave/brave-search-cli). Start Pi from the shell where `command -v bx` finds the executable, then verify it with `bx --version`. `bx` handles its own credentials and configuration. Do not put credentials in Pi settings, prompts, or repository files.
+
+To enable or disable the tool, open `/supi-settings`, choose **Web Search**, and change **Web Search tool**. Run `/reload` after the change. The setting applies only when the extension starts or reloads. If `bx` was not available, install or configure it, make sure it is on `PATH`, and run `/reload`.
+
+Search results are provider-reported source material. A source date can be a publication or modification date and is not a publication-date guarantee.
+
 ## How page fetching works
 
 `web_fetch_md` chooses the cleanest representation it can find:
@@ -59,7 +73,7 @@ It follows redirects, removes scripts and styles, preserves useful structures su
 
 The default `auto` mode returns content inline up to 15,000 characters and writes larger pages to a temporary Markdown file. You can ask Pi to force inline or file output.
 
-All three tools cap model-visible inline output at Pi's 2,000-line / 50KB limit. If output is truncated, the complete result is saved to a temporary file for follow-up reads.
+All tools cap model-visible inline output at Pi's 2,000-line / 50KB limit. If output is truncated, the complete result is saved to a temporary file for follow-up reads.
 
 Tool rows stay collapsed in Pi's TUI by default. Press `Ctrl+O` to expand their inline output.
 
@@ -91,4 +105,4 @@ pi
 - `web_fetch_md` is for public `http://` and `https://` sources. It does not authenticate to login-protected, private, or paywalled pages.
 - It fetches HTTP responses and does not run a browser, page JavaScript, or interactive flows; client-rendered pages may expose little useful content.
 - When the `gh` CLI is available, Pi is guided to use it instead for GitHub URLs.
-- Fetched pages are external, untrusted content. Treat them as source material, not as repository instructions.
+- Fetched pages and search excerpts are external, untrusted content. Treat them as source material, not as repository instructions.

package/node_modules/@mrclrchtr/supi-core/README.md

@@ -0,0 +1,127 @@
+<div align="center">
+  <a href="https://github.com/mrclrchtr/supi/tree/main/packages/supi-core">
+    <img src="https://raw.githubusercontent.com/mrclrchtr/supi/main/packages/supi-core/assets/social-preview.png" alt="SuPi Core" width="100%">
+  </a>
+</div>
+
+# @mrclrchtr/supi-core
+
+[![GitHub stars](https://img.shields.io/github/stars/mrclrchtr/supi)](https://github.com/mrclrchtr/supi/stargazers)
+
+Shared infrastructure for SuPi extensions.
+
+This is a **pure library** — it does not register any pi commands or tools. The `/supi-settings` command is now available through `@mrclrchtr/supi-settings`.
+
+## Install
+
+```bash
+pnpm add @mrclrchtr/supi-core
+```
+
+## Package surfaces
+
+- `@mrclrchtr/supi-core/api` — reusable helpers for other packages and extensions
+- `@mrclrchtr/supi-core/llm` — PI-owned direct model requests and JSON helpers
+- `@mrclrchtr/supi-core/report` — shared text/report rendering helpers for TUI and plain-text summaries
+
+## What you get from the API
+
+### Config helpers
+
+- `loadSupiConfig()` — merged config with resolution order `defaults <- global <- project`
+- `loadSupiConfigForScope()` — load one scope at a time for settings UIs
+- `writeSupiConfig()` — persist values
+- `replaceSupiConfigSection()` — replace one nested section while preserving other sections
+- `removeSupiConfigKey()` — remove a key or override
+
+Config file locations:
+
+- global: `~/.pi/agent/supi/config.json`
+- project: `.pi/supi/config.json`
+
+### Settings helpers
+
+- `registerSettings(pi, module)` — register one canonical asynchronous settings module
+- `defineConfigSettings(options)` — adapt a fixed SuPi config section to that module interface
+- settings registry helpers and types for the `@mrclrchtr/supi-settings` configuration surface
+
+### Context helpers
+
+- `wrapExtensionContext()` — wrap injected text in SuPi's `<extension-context>` tag
+
+### Model requests
+
+- `completeModelRequest(ctx, model, context, options)` — send API-specific options through PI's model registry.
+- `completeSimpleModelRequest(ctx, model, context, options)` — send provider-neutral options through `streamSimple().result()` on PI 0.86.0 or later. PI supplies model-derived output limits and context guards.
+- `callWithJsonResponse()` — retry a registry request, extract JSON, and validate it with TypeBox.
+
+Both completion helpers require a stable `affinityScope`. PI owns auth and endpoint resolution. The helpers keep cache retention defaults, exclude prompt content from the affinity ID, and add OpenCode headers only when the provider or exact model endpoint matches OpenCode. Use the simple helper when PI should manage output limits; the API-specific helper forwards the caller's limit unchanged.
+
+### Shared registries
+
+- context-provider registry for `/supi-context`
+- debug-event registry and monotonic phase timers for producers that want shared debug capture
+- optional Debug Operation IDs for exact, directly owned public Tool-call correlation; ambient events stay uncorrelated
+- settings registry used by `/supi-settings`
+
+### Project and session helpers
+
+- project-root detection and directory walking helpers such as `findProjectRoot()` and `walkProject()`
+- active-branch session helper: `getActiveBranchEntries()`
+- terminal helpers such as `formatTitle()`, `signalWaiting()`, and `signalDone()`
+
+### Report helpers
+
+- `clampReportWidth()` — enforce a minimum readable report width
+- `formatReportTitle()` / `formatSectionHeader()` — shared themed headers
+- `formatDimLine()` / `formatKeyValueLine()` — common summary rows
+- `formatOverflowHint()` — consistent preview-overflow hints
+- `wrapReportText()` — ANSI-aware wrapped report blocks with optional indentation
+
+## Example
+
+```ts
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import {
+  defineConfigSettings,
+  loadSupiConfig,
+  registerSettings,
+  wrapExtensionContext,
+} from "@mrclrchtr/supi-core/api";
+
+export default function myExtension(pi: ExtensionAPI) {
+  const defaults = { enabled: true };
+  const config = loadSupiConfig("my-extension", process.cwd(), defaults);
+
+  registerSettings(
+    pi,
+    defineConfigSettings({
+      id: "my-extension",
+      label: "My Extension",
+      section: "my-extension",
+      defaults,
+      fields: [
+        {
+          kind: "boolean" as const,
+          key: "enabled",
+          label: "Enabled",
+        },
+      ],
+    }),
+  );
+
+  const message = wrapExtensionContext("my-extension", "hello", {
+    enabled: config.enabled,
+  });
+  void message;
+}
+```
+
+## Source
+
+- `src/api.ts` — exported library surface
+- `src/config.ts` — shared config loading and writing
+- `src/debug-registry.ts` — Debug domain surface, event state, retention, redaction, listeners, and queries
+- `src/debug-timing.ts` — monotonic total and phase timers for Debug Event Producers
+- `src/settings/` — settings registry, schema, scope resolution, and persistence
+- `src/report.ts` — shared text/report rendering helpers

package/node_modules/@mrclrchtr/supi-core/package.json

@@ -0,0 +1,72 @@
+{
+  "name": "@mrclrchtr/supi-core",
+  "version": "7.2.0",
+  "description": "Shared settings, configuration, reporting, and session infrastructure",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/mrclrchtr/supi.git",
+    "directory": "packages/supi-core"
+  },
+  "homepage": "https://github.com/mrclrchtr/supi/tree/main/packages/supi-core#readme",
+  "bugs": {
+    "url": "https://github.com/mrclrchtr/supi/issues"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "keywords": [
+    "pi",
+    "pi-coding-agent",
+    "supi",
+    "extension-utils",
+    "settings",
+    "configuration"
+  ],
+  "type": "module",
+  "files": [
+    "src/**/*.ts",
+    "!__tests__"
+  ],
+  "peerDependencies": {
+    "@earendil-works/pi-ai": "*",
+    "@earendil-works/pi-coding-agent": "*",
+    "@earendil-works/pi-tui": "*",
+    "typebox": "*"
+  },
+  "peerDependenciesMeta": {
+    "@earendil-works/pi-ai": {
+      "optional": true
+    },
+    "@earendil-works/pi-coding-agent": {
+      "optional": true
+    },
+    "@earendil-works/pi-tui": {
+      "optional": true
+    },
+    "typebox": {
+      "optional": true
+    }
+  },
+  "main": "src/api.ts",
+  "exports": {
+    "./api": "./src/api.ts",
+    "./config": "./src/config.ts",
+    "./context": "./src/context.ts",
+    "./debug": "./src/debug.ts",
+    "./evidence-badge": "./src/evidence-badge.ts",
+    "./footer-registry": "./src/footer-registry.ts",
+    "./llm": "./src/llm.ts",
+    "./model-selection": "./src/model-selection.ts",
+    "./package.json": "./package.json",
+    "./path": "./src/path.ts",
+    "./project": "./src/project.ts",
+    "./prompt-surface": "./src/prompt-surface.ts",
+    "./report": "./src/report.ts",
+    "./session": "./src/session.ts",
+    "./settings": "./src/settings.ts",
+    "./spinner-frames": "./src/spinner-frames.ts",
+    "./status-spinner": "./src/status-spinner.ts",
+    "./terminal": "./src/terminal.ts"
+  }
+}

package/node_modules/@mrclrchtr/supi-core/src/api.ts

@@ -0,0 +1,40 @@
+// supi-core — shared infrastructure for SuPi extensions.
+// Provides XML context tag wrapping, unified config, prompt-surface resolution,
+// and shared settings, reporting, and session utilities.
+//
+// Convenience barrel — re-exports all domain entry points.
+// For lighter imports, use one of the domain subpaths directly
+// (e.g. @mrclrchtr/supi-core/config, @mrclrchtr/supi-core/context).
+
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./config.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./context.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./debug.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./evidence-badge.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./footer-registry.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./llm.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./model-selection.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./path.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./project.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./prompt-surface.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./report.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./session.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./settings.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./spinner-frames.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./status-spinner.ts";
+// biome-ignore lint/performance/noReExportAll: intentional convenience barrel
+export * from "./terminal.ts";

package/node_modules/@mrclrchtr/supi-core/src/config/config.ts

@@ -0,0 +1,232 @@
+// Shared config system for SuPi extensions.
+//
+// Global config: ~/.pi/agent/supi/config.json
+// Project config: .pi/supi/config.json (relative to cwd)
+// Resolution: hardcoded defaults ← global ← project
+
+import * as fs from "node:fs";
+import * as os from "node:os";
+import * as path from "node:path";
+import { CONFIG_DIR_NAME } from "@earendil-works/pi-coding-agent";
+
+const GLOBAL_CONFIG_DIR = `${CONFIG_DIR_NAME}/agent/supi`;
+const PROJECT_CONFIG_DIR = `${CONFIG_DIR_NAME}/supi`;
+const CONFIG_FILE = "config.json";
+
+function getGlobalConfigPath(homeDir?: string): string {
+  return path.join(homeDir ?? os.homedir(), GLOBAL_CONFIG_DIR, CONFIG_FILE);
+}
+
+function getProjectConfigPath(cwd: string): string {
+  return path.join(cwd, PROJECT_CONFIG_DIR, CONFIG_FILE);
+}
+
+/** Return the SuPi config file path for one scope. */
+export function getSupiConfigPath(
+  scope: "global" | "project",
+  cwd: string,
+  options?: SupiConfigOptions,
+): string {
+  return scope === "global" ? getGlobalConfigPath(options?.homeDir) : getProjectConfigPath(cwd);
+}
+
+export function readJsonFile(filePath: string): Record<string, unknown> | null {
+  let content: string;
+  try {
+    content = fs.readFileSync(filePath, "utf-8");
+  } catch {
+    // ENOENT or permission error — silent, file may not exist
+    return null;
+  }
+
+  let parsed: unknown;
+  try {
+    parsed = JSON.parse(content);
+  } catch {
+    // biome-ignore lint/suspicious/noConsole: deliberate config parse warning
+    console.warn(`[supi-core] Failed to parse config file, ignoring: ${filePath}`);
+    return null;
+  }
+
+  if (typeof parsed === "object" && parsed !== null && !Array.isArray(parsed)) {
+    return parsed as Record<string, unknown>;
+  }
+
+  // biome-ignore lint/suspicious/noConsole: deliberate config parse warning
+  console.warn(`[supi-core] Config file root is not an object, ignoring: ${filePath}`);
+  return null;
+}
+
+function shallowMerge<T>(base: T, ...overrides: Array<Record<string, unknown> | null>): T {
+  let result = { ...base };
+  for (const override of overrides) {
+    if (!override) continue;
+    result = { ...result, ...override };
+  }
+  return result;
+}
+
+export interface SupiConfigOptions {
+  homeDir?: string;
+}
+
+/**
+ * Load and merge config for a given extension section.
+ *
+ * Resolution order: defaults ← global ← project
+ */
+export function loadSupiConfig<T>(
+  section: string,
+  cwd: string,
+  defaults: T,
+  options?: SupiConfigOptions,
+): T {
+  const globalConfig = readJsonFile(getGlobalConfigPath(options?.homeDir));
+  const projectConfig = readJsonFile(getProjectConfigPath(cwd));
+
+  const globalSection = extractSection(globalConfig, section);
+  const projectSection = extractSection(projectConfig, section);
+
+  return shallowMerge(defaults, globalSection, projectSection);
+}
+
+/**
+ * Load config for a single scope only.
+ *
+ * Resolution order: defaults ← selected scope
+ *
+ * This is useful for settings UIs that need to show the raw values stored in
+ * one scope, rather than the effective merged config.
+ */
+export function loadSupiConfigForScope<T>(
+  section: string,
+  cwd: string,
+  defaults: T,
+  options: { scope: "global" | "project" } & SupiConfigOptions,
+): T {
+  const config = readJsonFile(getSupiConfigPath(options.scope, cwd, { homeDir: options.homeDir }));
+
+  const scopedSection = extractSection(config, section);
+  return shallowMerge(defaults, scopedSection);
+}
+
+/** Load the raw object for one config section and one scope. */
+export function loadSupiConfigSectionForScope(
+  section: string,
+  cwd: string,
+  options: { scope: "global" | "project" } & SupiConfigOptions,
+): Record<string, unknown> | null {
+  const config = readJsonFile(getSupiConfigPath(options.scope, cwd, { homeDir: options.homeDir }));
+  return extractSection(config, section);
+}
+
+export interface SupiConfigLocation {
+  section: string;
+  scope: "global" | "project";
+  cwd: string;
+}
+
+/**
+ * Write config values for a given extension section.
+ */
+export function writeSupiConfig(
+  loc: SupiConfigLocation,
+  value: Record<string, unknown>,
+  options?: SupiConfigOptions,
+): void {
+  const configPath = getSupiConfigPath(loc.scope, loc.cwd, options);
+
+  const dir = path.dirname(configPath);
+  fs.mkdirSync(dir, { recursive: true });
+
+  const existing = readJsonFile(configPath) ?? {};
+  existing[loc.section] = {
+    ...((existing[loc.section] as Record<string, unknown>) ?? {}),
+    ...value,
+  };
+
+  fs.writeFileSync(configPath, `${JSON.stringify(existing, null, 2)}\n`, "utf-8");
+}
+
+/**
+ * Replace one complete config section while preserving other sections.
+ *
+ * This is useful for nested settings that must remove stale keys as part of
+ * one update. An empty section is removed from the config file.
+ */
+export function replaceSupiConfigSection(
+  loc: SupiConfigLocation,
+  value: Record<string, unknown>,
+  options?: SupiConfigOptions,
+): void {
+  const configPath = getSupiConfigPath(loc.scope, loc.cwd, options);
+  const existing = readJsonFile(configPath) ?? {};
+
+  if (Object.keys(value).length > 0) existing[loc.section] = value;
+  else delete existing[loc.section];
+
+  const content = Object.keys(existing).length > 0 ? `${JSON.stringify(existing, null, 2)}\n` : "";
+  if (content) {
+    fs.mkdirSync(path.dirname(configPath), { recursive: true });
+    fs.writeFileSync(configPath, content, "utf-8");
+    return;
+  }
+
+  try {
+    fs.unlinkSync(configPath);
+  } catch {
+    // File may not exist.
+  }
+}
+
+/**
+ * Remove a key from a config section.
+ * Used by `interval default` to remove the project override.
+ */
+export function removeSupiConfigKey(
+  loc: SupiConfigLocation,
+  key: string,
+  options?: SupiConfigOptions,
+): void {
+  const configPath = getSupiConfigPath(loc.scope, loc.cwd, options);
+
+  const existing = readJsonFile(configPath);
+  if (!existing) return;
+
+  const sectionData = existing[loc.section] as Record<string, unknown> | undefined;
+  if (!sectionData) return;
+
+  delete sectionData[key];
+
+  if (Object.keys(sectionData).length === 0) {
+    delete existing[loc.section];
+  }
+
+  const dir = path.dirname(configPath);
+  fs.mkdirSync(dir, { recursive: true });
+
+  const content = Object.keys(existing).length > 0 ? `${JSON.stringify(existing, null, 2)}\n` : "";
+
+  if (content) {
+    // Directory guaranteed to exist since we just read from it
+    fs.writeFileSync(configPath, content, "utf-8");
+  } else {
+    try {
+      fs.unlinkSync(configPath);
+    } catch {
+      // File may not exist
+    }
+  }
+}
+
+function extractSection(
+  config: Record<string, unknown> | null,
+  section: string,
+): Record<string, unknown> | null {
+  if (!config) return null;
+  const data = config[section];
+  if (typeof data === "object" && data !== null && !Array.isArray(data)) {
+    return data as Record<string, unknown>;
+  }
+  return null;
+}