pi-web-kit 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -6,6 +6,24 @@ This project follows the spirit of [Keep a Changelog](https://keepachangelog.com
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-10-09
+
+### Added
+
+- Declare bounded structured results for all five research tools, plus `web` namespace discovery and behavior annotations. Codemode now receives typed objects without `JSON.parse`; direct JSON text remains available.
+
+### Fixed
+
+- Project provider responses to public fields and mask credentials consistently across text, structured data, details and progress. Omit arbitrary metadata, Context7 rules and raw backend error bodies.
+- Restrict Basic/Bearer masking to authorization-header values so ordinary authentication documentation remains readable, while short header credentials stay masked.
+- Honor explicit default-tool exclusions during late registration and retain manual deactivation across reload instead of force-activating optional research tools.
+- Activate newly available optional tools when credentials arrive on reload, including existing positive selections, without reactivating previously disabled tools. Remember only fixed tool names in non-model session metadata; older sessions without that history use a conservative first-reload fallback.
+- Reject cancelled calls even when a provider converts cancellation into per-page failure data.
+
+### Changed
+
+- Update the shared Pi development and contract-test baseline to 1.1.0; require Node.js >=22.19.0 to match the host runtime. Pi remains a host-supplied peer dependency.
+
 ## [0.3.0] - 2026-08-31
 
 ### Changed

package/README.md

@@ -10,6 +10,7 @@ Give [Pi](https://pi.dev) current web knowledge, authoritative library docs, and
 - **Use docs that match the task** — resolve libraries and retrieve focused, version-aware documentation with code examples.
 - **Find proven implementation patterns** — search practical usage, setup, migrations, and error context across real code.
 - **Spend context wisely** — compact search results, chunked page reads, bounded output, and fetch caching keep research useful without overwhelming the model.
+- **Compose research in scripts** — typed results let codemode filter and join data without parsing model-facing text.
 - **Choose your providers** — mix Exa, TinyFish, Brave, Firecrawl, markdown.new, Context7, and Exa Code based on coverage, cost, and credentials.
 
 ## Installation
@@ -39,6 +40,7 @@ pi -e /path/to/pi-mono/packages/pi-web-kit --web-provider-fetch markdown_new --p
 ```
 
 This is an npm-compatible TypeScript Pi package. Bun is not required.
+Use Pi 1.1.0 or newer and Node.js >=22.19.0 for the structured tool contracts.
 
 ## Quick usage
 
@@ -234,11 +236,87 @@ Finds practical code examples, implementation context, setup snippets, migration
 
 Cache keys include the provider, canonical URL, fetch-affecting parameters, relevant provider defaults, and an opaque SHA-256 API-key/account scope. Internal cache keys are never returned in tool output. `refresh: true` bypasses and replaces the cached entry.
 
+## Structured results and discovery
+
+All five tools declare `outputSchema` and return objects through `structuredContent`.
+Codemode receives those objects directly; remove old `JSON.parse(await tools.…())`
+wrappers. Direct calls still return useful JSON text with exactly the same
+bounded, redacted value. Renderer `details` remain compact and are not the
+script API.
+
+| Tool | Script result |
+| --- | --- |
+| `web_search` | `{ provider, queries: [{ query, requestedResultLimit, effectiveResultLimit, requestedContextTokens?, effectiveContextTokens, contextCharacters, omittedContextCharacters?, resultCount, omittedResultCount?, results }] }`. Each result has `url` and optional `title`, `snippet`, `content`, `contentFormat`, `siteName`, `position`. |
+| `web_fetch` | `{ provider, results }`. Each item is either `{ url, error }` or `{ url, fetchedUrl, title?, content, format, cached, refreshed, range }`. `range` has `offset`, `limit`, `returned`, `total`, `truncated`, `hasPrevious`, `hasNext`, and optional `nextOffset`. |
+| `library_search` | `{ provider: "context7", libraryName, query, searchFilterApplied?, results }`. Each candidate has `id`; title, description, branch, dates, state, scores, counts, stars and versions are optional. |
+| `library_docs` | `{ provider: "context7", libraryId, query, codeSnippets, infoSnippets }`. Code snippets expose optional title/description/language/ID/page/source/token fields and `codeList: [{ language, code }]`; info snippets require `content` with optional page/breadcrumb/token fields. |
+| `code_search` | `{ provider: "exa", query, response, resultsCount?, searchTime?, outputTokens?, requestId? }`. Provider token counts are informational, not a second Pi usage report. |
+
+Every result also allows the top-level fallback `{ truncated: true, message }`
+when metadata alone cannot fit. Check this before reading ordinary fields.
+Both text and structured payloads stay within the same 50,000-byte JSON budget.
+Fetch slices retain range continuation; search results report omitted counts.
+Missing/invalid optional provider fields are omitted. Invalid required fields
+fail the call rather than yielding a guessed success object.
+
+```js
+const search = await tools.web_search({ query: "Pi extension API", numResults: 3 });
+if ("truncated" in search) throw new Error(search.message);
+const urls = search.queries.flatMap(q => q.results.map(r => r.url));
+if (urls.length) {
+  const pages = await tools.web_fetch({ urls: urls.slice(0, 10) });
+  if ("truncated" in pages) throw new Error(pages.message);
+  text(pages.results.filter(r => !("error" in r)).map(r => ({
+    url: r.url, excerpt: r.content.slice(0, 500), nextOffset: r.range.nextOffset
+  })));
+}
+```
+
+Validation, whole-request/provider, malformed-output and cancellation failures
+throw: direct calls become Pi tool errors and scripts must use `try`/`catch` or
+`Promise.allSettled`. Individual fetch failures are successful result data with
+an `error` field, not `isError: true`. These tools do not return structured success
+objects alongside `isError: true`. A trusted result hook can replace that contract;
+annotations do not bypass hooks or approvals.
+
+The namespace is `web`, not a name prefix: calls remain `tools.web_fetch(…)`.
+`await describeNamespace("web")`, `await describeTool("web_fetch")`, and
+`await searchTools("page content", { namespace: "web" })` work even with
+`codemode.inlineBudget: 0`. Independent reads can run in parallel. The advisory
+hints are read-only, non-destructive, idempotent, and open-world: repeated reads
+can return different content and incur provider charges.
+
+Direct exposure remains the default, and codemode is optional. No package-level
+deferred/codemode exposure switch is added: it would make inactive direct tools
+nested-callable and change the meaning of disabling them. Use Pi's `codemode.mode`
+(`on` or `only`) and `inlineBudget` to reduce declarations while retaining the
+active-tool boundary. Optional research tools still require their provider keys.
+Explicit `defaultTools` `-name` entries are honored at late registration; reload
+restores Pi's active selection rather than re-enabling manually disabled tools.
+New optional tools activate when credentials first become available on reload,
+including existing positive `defaultTools` selections. Previously registered
+tools are not force-reactivated, even if settings still positively select them.
+The extension records only previously registered tool names in branch-local
+`pi-web-kit:registered-tools` session metadata, outside model context; it does
+not store activation choices or credentials. The record is refreshed before
+reload so navigating to older branch history cannot revive a manually disabled
+tool. Older sessions without this metadata preserve inactivity on their first
+reload, so they may require explicit activation or a restart.
+Malformed/unknown-version metadata uses the same
+conservative fallback.
+CLI exclusions remain host-enforced.
+
 ## Privacy and security
 
 `pi-web-kit` sends search queries and fetched URLs to the configured provider. Developer-search tools send library/doc queries to Context7 and code-context queries to Exa when those tools are enabled. Fetch providers may also receive provider-specific options. API keys are read from environment variables or local config files and are used only for provider requests.
 
-The extension rejects non-HTTP(S) URLs and URLs with embedded username/password credentials. Provider responses are not sandboxed; they are returned to Pi as tool output.
+The extension rejects non-HTTP(S) URLs and URLs with embedded username/password credentials.
+Returned data is projected to declared public fields; raw provider metadata,
+Context7 `rules`, cache keys and HTTP error bodies are not returned. Known API
+keys, URL credential patterns and Basic/Bearer authorization-header values are masked in text, structured data,
+renderer details and progress. This is not a general sensitive-data scanner:
+page content remains untrusted, and Pi retains caller-supplied arguments in its
+own transcript. Do not put credentials in queries or URLs.
 
 Report security issues privately. See [SECURITY.md](SECURITY.md).
 
@@ -257,7 +335,7 @@ Report security issues privately. See [SECURITY.md](SECURITY.md).
 
 Requirements:
 
-- Node.js >= 20.6.0
+- Node.js >= 22.19.0
 - npm
 
 Common commands:

package/SECURITY.md

@@ -26,3 +26,25 @@ The maintainer will acknowledge reports as soon as practical and coordinate disc
 On startup, `@mocito/install-telemetry` sends a best-effort install/update telemetry ping to the configured telemetry endpoint once per package version unless Pi telemetry is disabled or offline mode is enabled. The ping includes only the package name, version, and parsed platform/runtime/architecture from its User-Agent; it does not include prompts, queries, fetched URLs, file paths, config values, or API keys.
 
 The extension validates URLs before fetch calls and only accepts `http:` and `https:` URLs without embedded credentials. This validation reduces accidental misuse but does not sandbox provider responses or the local Pi process.
+
+Tool results use an allowlisted output schema before publication. Model text and
+`structuredContent` share the same 50,000-byte JSON budget and credential masking;
+renderer details and progress also mask configured API keys, URL credential
+patterns and Basic/Bearer authorization-header values (not ordinary auth prose).
+Arbitrary provider metadata, Context7 `rules`, cache keys,
+and HTTP failure bodies are excluded. Untrusted per-page error bodies become
+generic diagnostics; controlled HTTP status/timeout messages remain useful.
+This does not classify all private information in fetched content or sanitize
+Pi's stored caller arguments. Do not send secrets in queries or URLs.
+
+The `web` namespace and read-only/idempotent/open-world annotations are advisory.
+They do not approve provider spending, bypass tool hooks, or make an inactive
+direct tool reachable from nested dispatch. Project trust, URL validation,
+provider request limits and cancellation still apply.
+
+Branch-local `pi-web-kit:registered-tools` custom entries remember only this
+package's fixed tool names as they first become available. They contain no
+credentials, configuration values, activation decisions or usage data and do
+not enter model context. This distinguishes a newly available optional tool
+from re-registration of a manually disabled tool; Pi still owns activation
+and exclusions. Missing or invalid history does not force tools active on reload.

package/extensions/index.ts

@@ -1,7 +1,7 @@
 import { createHash } from "node:crypto";
-import { type ExtensionAPI, getAgentDir } from "@earendil-works/pi-coding-agent";
+import { type ExtensionAPI, type ExtensionContext, type ToolDefinition } from "@earendil-works/pi-coding-agent";
 import { Text } from "@earendil-works/pi-tui";
-import { Type } from "typebox";
+import { Type, type TSchema } from "typebox";
 import { fetchCache, type CachedPage } from "../src/cache.js";
 import { resolveConfig } from "../src/config.js";
 import { applySearchContextBudget, capSearchResultLimit, DEFAULT_FETCH_LIMIT, DEFAULT_NUM_RESULTS, DEFAULT_SEARCH_CONTEXT_TOKENS, MAX_LIMIT, MAX_NUM_RESULTS, MAX_OFFSET, MAX_QUERY_COUNT, MAX_SEARCH_CONTEXT_TOKENS, MAX_URL_COUNT, MULTI_FETCH_LIMIT, safePrefix, TINYFISH_MAX_PAGE } from "../src/limits.js";
@@ -10,6 +10,9 @@ import { mapFetchResults } from "../src/providers/fallback.js";
 import type { FetchProviderName, SearchProviderName, WebFetchResult } from "../src/types.js";
 import { canonicalWebUrl, normalizeUrlInput } from "../src/urls.js";
 import { reportInstallTelemetry } from "../src/install-telemetry.js";
+import { projectOutput, publicPageError, redactOutput, redactText, WEB_OUTPUT_SCHEMAS, WEB_TOOL_METADATA } from "../src/output.js";
+
+const REGISTERED_TOOLS_ENTRY = "pi-web-kit:registered-tools";
 
 export default function (pi: ExtensionAPI) {
   void reportInstallTelemetry();
@@ -23,18 +26,95 @@ export default function (pi: ExtensionAPI) {
   });
 
   let registeredConfig = "";
-  pi.on("session_start", (_event, ctx) => {
+  let rememberedTools: string[] | undefined;
+  pi.on("session_start", (event, ctx) => {
     const startupConfig = runtimeConfig(pi, ctx.cwd, projectIsTrusted(ctx));
     const signature = toolConfigSignature(startupConfig);
     if (signature === registeredConfig) return;
-    registerTools(pi, startupConfig);
-    syncOptionalTools(pi, startupConfig);
+    const available = [
+      "web_search", "web_fetch",
+      ...(startupConfig.apiKeys.context7 ? ["library_search", "library_docs"] : []),
+      ...(startupConfig.apiKeys.exa ? ["code_search"] : []),
+    ];
+    const previous = registrationHistory(ctx);
+    // Remember availability, not activation. Pi owns active/pending selection.
+    // Without history (an older session), conservatively preserve inactivity on
+    // the first reload instead of guessing which tools were manually disabled.
+    registerTools(pi, startupConfig, event.reason === "reload" ? new Set(previous ?? available) : undefined);
+    rememberedTools = recordRegisteredTools(pi, ctx, available);
+    // A session change can drop provider credentials without replacing this
+    // extension instance. Retained definitions must not remain active then.
+    if (typeof pi.getActiveTools === "function" && typeof pi.setActiveTools === "function") {
+      const unavailable = new Set([
+        ...(!startupConfig.apiKeys.context7 ? ["library_search", "library_docs"] : []),
+        ...(!startupConfig.apiKeys.exa ? ["code_search"] : []),
+      ]);
+      const active = pi.getActiveTools();
+      if (active.some(name => unavailable.has(name))) pi.setActiveTools(active.filter(name => !unavailable.has(name)));
+    }
     registeredConfig = signature;
   });
+  pi.on("session_shutdown", (event, ctx) => {
+    // Tree navigation can restore an older branch record while the live
+    // registry still knows newer tools. Save that fact before runtime teardown.
+    if (event.reason === "reload" && rememberedTools) recordRegisteredTools(pi, ctx, rememberedTools);
+  });
+}
+
+function registrationHistory(ctx: ExtensionContext): string[] | undefined {
+  const entry = ctx.sessionManager?.getBranch().filter(item => item.type === "custom" && item.customType === REGISTERED_TOOLS_ENTRY).at(-1);
+  return entry?.type === "custom" ? registeredToolNames(entry.data) : undefined;
+}
+
+function recordRegisteredTools(pi: ExtensionAPI, ctx: ExtensionContext, tools: readonly string[]): string[] {
+  const previous = registrationHistory(ctx);
+  const remembered = [...new Set([...(previous ?? []), ...tools])].sort();
+  if (!previous || JSON.stringify(previous) !== JSON.stringify(remembered)) {
+    pi.appendEntry?.(REGISTERED_TOOLS_ENTRY, { version: 1, tools: remembered });
+  }
+  return remembered;
+}
+
+function registeredToolNames(data: unknown): string[] | undefined {
+  if (!data || typeof data !== "object" || !("version" in data) || data.version !== 1
+    || !("tools" in data) || !Array.isArray(data.tools)
+    || data.tools.length > Object.keys(WEB_OUTPUT_SCHEMAS).length
+    || !data.tools.every(name => typeof name === "string" && Object.hasOwn(WEB_OUTPUT_SCHEMAS, name))) return undefined;
+  return [...new Set(data.tools)].sort();
 }
 
-function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolveConfig>) {
-  pi.registerTool({
+function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolveConfig>, previouslyRegistered?: ReadonlySet<string>) {
+  // All five tools stay direct. Metadata must not make excluded/inactive tools callable.
+  type DataTool = Omit<ToolDefinition, "execute"> & {
+    execute(...args: Parameters<ToolDefinition["execute"]>): Promise<unknown>;
+  };
+  const settings = pi.getSettings?.().defaultTools;
+  const disabledByDefault = (name: string) => Array.isArray(settings)
+    && settings.filter(entry => entry === `+${name}` || entry === `-${name}`).at(-1) === `-${name}`;
+  const register = (tool: DataTool) => pi.registerTool({
+    ...WEB_TOOL_METADATA,
+    ...tool,
+    // Late registration must not undo explicit settings or session deactivation.
+    // Pi restores previously active/pending names on reload even when this is false.
+    // Only first availability uses the default; re-registration cannot undo a
+    // manual deactivation, even if settings positively select this tool.
+    defaultActive: !previouslyRegistered?.has(tool.name) && !disabledByDefault(tool.name),
+    outputSchema: WEB_OUTPUT_SCHEMAS[tool.name],
+    async execute(id, args, signal, onUpdate, ctx) {
+      const secrets = Object.values(runtimeConfig(pi, ctx.cwd, projectIsTrusted(ctx)).apiKeys);
+      try {
+        const result = await tool.execute(id, args, signal,
+          onUpdate ? update => onUpdate(redactOutput(update, secrets)) : undefined, ctx);
+        // Some providers represent an aborted fetch as per-page failures.
+        // Cancellation is still a failed call, never a successful data snapshot.
+        if (signal?.aborted) throw new Error("Web tool cancelled.");
+        return jsonToolResult(result, WEB_OUTPUT_SCHEMAS[tool.name], secrets);
+      } catch (error) {
+        throw new Error(redactText(error instanceof Error ? error.message : "Web tool failed.", secrets).slice(0, 1000));
+      }
+    },
+  });
+  register({
     name: "web_search",
     label: "Web Search",
     description: buildSearchDescription(startupConfig.provider_search),
@@ -90,7 +170,7 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
         emitProgress(onUpdate, progress);
       }
       const result = { provider: config.provider_search, queries: grouped };
-      return jsonToolResult(result);
+      return result;
     },
     renderCall(args, theme) {
       return new Text(renderWebCall("search", args as Record<string, any>, theme), 0, 0);
@@ -100,7 +180,7 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
     },
   });
 
-  pi.registerTool({
+  register({
     name: "web_fetch",
     label: "Web Fetch",
     description: buildFetchDescription(startupConfig.provider_fetch),
@@ -122,7 +202,7 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
         updateFetchProgress(progress, event);
         emitProgress(onUpdate, progress);
       });
-      return jsonToolResult(result);
+      return result;
     },
     renderCall(args, theme) {
       return new Text(renderWebCall("fetch", args as Record<string, any>, theme), 0, 0);
@@ -133,7 +213,7 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
   });
 
   if (startupConfig.apiKeys.context7) {
-    pi.registerTool({
+    register({
       name: "library_search",
       label: "Library Search",
       description: "Resolve library, package, framework, SDK, API, or CLI names to canonical library IDs with version, trust, and snippet metadata. Use when you need to inspect candidate matches (official sources, versions, forks); library_docs resolves names automatically.",
@@ -147,11 +227,11 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
         const limit = parseInteger(params.limit, 10, "limit", 1, MAX_NUM_RESULTS);
         const provider = createContext7Provider(runtimeConfig(pi, ctx.cwd, projectIsTrusted(ctx)));
         const result = await provider.searchLibraries({ libraryName, query, fast: params.fast === true, limit }, signal);
-        return jsonToolResult(result);
+        return result;
       },
     });
 
-    pi.registerTool({
+    register({
       name: "library_docs",
       label: "Library Docs",
       description: "Fetch current, version-aware documentation and code examples for a library. Pass libraryName to resolve it automatically, or a known libraryId.",
@@ -171,13 +251,13 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
           if (!libraryId) throw new Error(`No library found for '${libraryName}'. Try library_search with a more specific name.`);
         }
         const result = await provider.getDocs({ libraryId, query, version: optionalString(params.version, "version"), type: "json", fast: params.fast === true, limit }, signal);
-        return jsonToolResult(result);
+        return result;
       },
     });
   }
 
   if (startupConfig.apiKeys.exa) {
-    pi.registerTool({
+    register({
       name: "code_search",
       label: "Code Search",
       description: "Find practical code examples, usage patterns, setup snippets, migrations, and error context.",
@@ -193,14 +273,12 @@ function registerTools(pi: ExtensionAPI, startupConfig: ReturnType<typeof resolv
         const tokensNum = parseTokensNum(params.tokensNum);
         const provider = createCodeSearchProvider(runtimeConfig(pi, ctx.cwd, projectIsTrusted(ctx)));
         const result = await provider.searchCode({ query, tokensNum }, signal);
-        return jsonToolResult(result);
+        return result;
       },
     });
   }
 }
 
-const OPTIONAL_TOOLS = ["library_search", "library_docs", "code_search"];
-
 function toolConfigSignature(config: ReturnType<typeof resolveConfig>): string {
   return JSON.stringify({
     search: config.provider_search,
@@ -210,16 +288,6 @@ function toolConfigSignature(config: ReturnType<typeof resolveConfig>): string {
   });
 }
 
-function syncOptionalTools(pi: ExtensionAPI, config: ReturnType<typeof resolveConfig>) {
-  if (typeof pi.getActiveTools !== "function" || typeof pi.setActiveTools !== "function") return;
-  const enabled = [
-    ...(config.apiKeys.context7 ? ["library_search", "library_docs"] : []),
-    ...(config.apiKeys.exa ? ["code_search"] : []),
-  ];
-  const active = pi.getActiveTools().filter((name) => !OPTIONAL_TOOLS.includes(name));
-  pi.setActiveTools([...new Set([...active, ...enabled])]);
-}
-
 type ProgressKind = "search" | "fetch";
 type ProgressItem = { label: string; status: "pending" | "current" | "done" | "error"; note?: string; error?: string };
 type WebProgress = { kind: ProgressKind; provider: string; total: number; completed: number; items: ProgressItem[] };
@@ -379,7 +447,7 @@ export async function fetchWithCache(providerName: FetchProviderName, params: Re
     for (const requestedUrl of missing) {
       const item = mapped.get(requestedUrl);
       if (!item || item.error) {
-        const error = item?.error ?? "No content returned.";
+        const error = publicPageError(item?.error ?? "No content returned.");
         pages.set(requestedUrl, { error, cached: false, refreshed: refresh });
         onProgress?.({ status: "error", url: requestedUrl, error });
         continue;
@@ -440,19 +508,28 @@ function projectIsTrusted(ctx: { isProjectTrusted?: () => boolean }): boolean {
 }
 
 function runtimeConfig(pi: ExtensionAPI, cwd: string, projectTrusted: boolean) {
-  return resolveConfig(
-    { providerSearch: pi.getFlag("web-provider-search"), providerFetch: pi.getFlag("web-provider-fetch") },
-    cwd,
-    process.env,
-    { includeProject: projectTrusted },
-  );
+  try {
+    return resolveConfig(
+      { providerSearch: pi.getFlag("web-provider-search"), providerFetch: pi.getFlag("web-provider-fetch") },
+      cwd,
+      process.env,
+      { includeProject: projectTrusted },
+    );
+  } catch {
+    // JSON parse and invalid-provider errors can echo credential-bearing config.
+    throw new Error("Web configuration is invalid. Check provider names, flags and config JSON.");
+  }
 }
 
 const MAX_OUTPUT_BYTES = 50_000;
 
-export function jsonToolResult(result: unknown) {
-  const bounded = boundStructuredResult(result);
-  return { content: [{ type: "text" as const, text: JSON.stringify(bounded) }], details: boundedDetails(bounded) };
+export function jsonToolResult(result: unknown, schema?: TSchema, secrets: readonly string[] = []) {
+  const publicResult = schema ? projectOutput(schema, result) : result;
+  const bounded = boundStructuredResult(redactOutput(publicResult, secrets));
+  const text = JSON.stringify(bounded);
+  // The model and scripts receive exactly the same bounded, redacted JSON value.
+  const structuredContent = JSON.parse(text);
+  return { content: [{ type: "text" as const, text }], structuredContent, details: boundedDetails(structuredContent) };
 }
 
 function boundStructuredResult(result: unknown): unknown {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-web-kit",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Context-efficient web search and fetch tools for Pi.",
   "type": "module",
   "license": "MIT",
@@ -58,19 +58,19 @@
     "typebox": "*"
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "^0.84.1",
-    "@earendil-works/pi-coding-agent": "^0.84.1",
-    "@earendil-works/pi-tui": "^0.84.1",
-    "@types/node": "^26.2.0",
-    "tsx": "^4.23.5",
-    "typebox": "^1.3.10",
+    "@earendil-works/pi-ai": "1.1.0",
+    "@earendil-works/pi-coding-agent": "1.1.0",
+    "@earendil-works/pi-tui": "1.1.0",
+    "@types/node": "^26.6.3",
+    "tsx": "^4.23.15",
+    "typebox": "^1.3.34",
     "typescript": "^7.0.2"
   },
   "publishConfig": {
     "access": "public"
   },
   "engines": {
-    "node": ">=20.6.0"
+    "node": ">=22.19.0"
   },
   "dependencies": {
     "@mocito/install-telemetry": "0.1.1"

package/src/http.ts

@@ -26,9 +26,10 @@ export async function fetchWithTimeout(url: string, init: RequestInit & { timeou
 export async function requestJson<T>(url: string, init: RequestInit & { timeoutMs?: number } = {}): Promise<T> {
   const res = await fetchWithTimeout(url, init);
   const text = await res.text();
-  if (!res.ok) throw new Error(`${res.status} ${res.statusText}: ${text.slice(0, 1000)}`);
+  if (!res.ok) throw new Error(`Provider request failed (HTTP ${res.status}).`);
   if (!text) return undefined as T;
-  return JSON.parse(text) as T;
+  try { return JSON.parse(text) as T; }
+  catch { throw new Error("Provider returned invalid JSON."); }
 }
 
 export function asSnippet(value: unknown): string | undefined {

package/src/output.ts

@@ -0,0 +1,140 @@
+import { Type, type TSchema } from "typebox";
+import { Check } from "typebox/value";
+
+const object = (properties: Record<string, TSchema>) => Type.Object(properties, { additionalProperties: false });
+const string = () => Type.Optional(Type.String());
+const number = () => Type.Optional(Type.Number());
+const boolean = () => Type.Optional(Type.Boolean());
+const truncated = object({ truncated: Type.Literal(true), message: Type.String() });
+const output = (schema: TSchema) => Type.Union([schema, truncated]);
+
+// Only explicitly selected fields cross the tool boundary. Provider metadata,
+// cache keys, response headers and Context7's untyped `rules` are not a contract.
+export const WEB_OUTPUT_SCHEMAS: Record<string, TSchema> = {
+  web_search: output(object({
+    provider: Type.String(),
+    queries: Type.Array(object({
+      query: Type.String(), requestedResultLimit: Type.Integer(), effectiveResultLimit: Type.Integer(),
+      requestedContextTokens: Type.Optional(Type.Integer()), effectiveContextTokens: Type.Integer(),
+      contextCharacters: Type.Integer(), omittedContextCharacters: Type.Optional(Type.Integer()),
+      resultCount: Type.Integer(), omittedResultCount: Type.Optional(Type.Integer()),
+      results: Type.Array(object({
+        url: Type.String(), title: string(), snippet: string(), content: string(),
+        contentFormat: Type.Optional(Type.Union([Type.Literal("markdown"), Type.Literal("text")])),
+        siteName: string(), position: number(),
+      })),
+    })),
+  })),
+  web_fetch: output(object({
+    provider: Type.String(),
+    results: Type.Array(Type.Union([
+      object({ url: Type.String(), error: Type.String() }),
+      object({
+        url: Type.String(), fetchedUrl: Type.String(), title: string(), content: Type.String(),
+        format: Type.Union([Type.Literal("markdown"), Type.Literal("html"), Type.Literal("json")]),
+        cached: Type.Boolean(), refreshed: Type.Boolean(),
+        range: object({
+          offset: Type.Integer(), limit: Type.Integer(), returned: Type.Integer(), total: Type.Integer(),
+          truncated: Type.Boolean(), hasPrevious: Type.Boolean(), hasNext: Type.Boolean(),
+          nextOffset: Type.Optional(Type.Integer()),
+        }),
+      }),
+    ])),
+  })),
+  library_search: output(object({
+    provider: Type.Literal("context7"), libraryName: Type.String(), query: Type.String(),
+    searchFilterApplied: boolean(),
+    results: Type.Array(object({
+      id: Type.String(), title: string(), description: string(), branch: string(),
+      lastUpdateDate: string(), state: string(), totalTokens: number(), totalSnippets: number(),
+      stars: number(), trustScore: number(), benchmarkScore: number(),
+      versions: Type.Optional(Type.Array(Type.String())),
+    })),
+  })),
+  library_docs: output(object({
+    provider: Type.Literal("context7"), libraryId: Type.String(), query: Type.String(),
+    codeSnippets: Type.Array(object({
+      codeTitle: string(), codeDescription: string(), codeLanguage: string(), codeTokens: number(),
+      codeId: string(), pageTitle: string(), sourceFile: string(), isDynamic: boolean(),
+      codeList: Type.Optional(Type.Array(object({ language: Type.String(), code: Type.String() }))),
+    })),
+    infoSnippets: Type.Array(object({
+      pageId: string(), breadcrumb: string(), content: Type.String(), contentTokens: number(),
+    })),
+  })),
+  code_search: output(object({
+    provider: Type.Literal("exa"), query: Type.String(), response: Type.String(),
+    resultsCount: number(), searchTime: number(), outputTokens: number(), requestId: string(),
+  })),
+};
+
+export const WEB_TOOL_METADATA = {
+  namespace: {
+    name: "web",
+    description: "Bounded web research, library documentation and code examples.",
+    instructions: "Keep existing tool names: web_search, web_fetch, library_search, library_docs, code_search. Await describeTool(name) for provider-specific inputs and output types. Calls return objects, not JSON strings. Fetch failures are per-item error strings; check them before using content. A top-level truncated:true/message result asks you to refine the request. Validation, cancellation and request failures reject. Parallel independent reads are supported; provider requests can incur costs. Availability depends on configured providers and active tools. These hints do not grant approval.",
+  },
+  annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
+} as const;
+
+/** Project to an allowlist before bounding or publishing. Invalid required data fails closed. */
+export function projectOutput(schema: TSchema, value: unknown): any {
+  const shape = schema as TSchema & {
+    anyOf?: TSchema[]; type?: string; properties?: Record<string, TSchema>;
+    required?: string[]; items?: TSchema;
+  };
+  if (shape.anyOf) {
+    for (const variant of shape.anyOf) {
+      try { return projectOutput(variant, value); } catch { /* try the next declared shape */ }
+    }
+    throw new Error("Provider returned an invalid result shape.");
+  }
+  let projected = value;
+  if (shape.type === "object" && value && typeof value === "object" && !Array.isArray(value)) {
+    projected = Object.fromEntries(Object.entries(shape.properties ?? {}).flatMap(([key, field]) => {
+      if (!Object.hasOwn(value, key) || (value as any)[key] == null) return [];
+      try { return [[key, projectOutput(field as TSchema, (value as any)[key])]]; }
+      catch {
+        if (shape.required?.includes(key)) throw new Error("Provider returned an invalid result shape.");
+        return [];
+      }
+    }));
+  } else if (shape.type === "array" && shape.items && Array.isArray(value)) {
+    projected = value.map(item => projectOutput(shape.items!, item));
+  }
+  if (!Check(schema, projected)) throw new Error("Provider returned an invalid result shape.");
+  return projected;
+}
+
+export function redactText(text: string, secrets: readonly string[] = []): string {
+  let redacted = text;
+  for (const secret of secrets.filter(Boolean).sort((a, b) => b.length - a.length)) {
+    redacted = redacted.split(secret).join("*".repeat(secret.length));
+  }
+  return redacted
+    .replace(/(https?:\/\/)([^\s/@"<>]+)@/gi, (_match, prefix, secret) => `${prefix}${"*".repeat(secret.length)}@`)
+    // Header context distinguishes credentials from prose such as "Basic
+    // authentication" or "Bearer token", without exempting short credentials.
+    .replace(/(\b(?:proxy-)?authorization\b["']?\s*[:=]\s*["'`]?(?:Bearer|Basic)[ \t]+)([A-Za-z0-9._~+/=-]+)/gi, (_match, prefix, secret) => `${prefix}${"*".repeat(secret.length)}`)
+    .replace(/([?&](?:api[-_]?key|access[-_]?token|token|secret|password)=)([^&#\s"'<>]+)/gi, (_match, prefix, secret) => `${prefix}${"*".repeat(secret.length)}`);
+}
+
+export function redactOutput(value: any, secrets: readonly string[]): any {
+  if (typeof value === "string") return redactText(value, secrets);
+  if (Array.isArray(value)) return value.map(item => redactOutput(item, secrets));
+  if (value && typeof value === "object") {
+    return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, redactOutput(item, secrets)]));
+  }
+  return value;
+}
+
+/** Provider error bodies are not page content and must not become tool output. */
+export function publicPageError(error: unknown): string {
+  if (typeof error === "string" && (
+    /^Provider request failed \(HTTP \d{3}\)\.$/.test(error)
+    || /^Request timed out after \d+ms$/.test(error)
+    || /^No content returned(?: by (?:TinyFish|Exa(?: contents endpoint)?))?\.$/.test(error)
+    || error === "Provider returned invalid JSON."
+  )) return error;
+  return "Provider could not fetch this page.";
+}

package/src/providers/markdown-new.ts

@@ -21,7 +21,7 @@ export class MarkdownNewProvider implements FetchProvider {
           timeoutMs: 45_000,
         });
         const text = await res.text();
-        if (!res.ok) throw new Error(`${res.status} ${res.statusText}: ${text.slice(0, 1000)}`);
+        if (!res.ok) throw new Error(`Provider request failed (HTTP ${res.status}).`);
         return {
           url,
           content: text,

package/src/urls.ts

@@ -6,10 +6,10 @@ export function normalizeWebUrl(value: string): string {
   try {
     parsed = new URL(value.trim());
   } catch {
-    throw new Error(`Malformed URL: ${value}`);
+    throw new Error("Malformed URL.");
   }
-  if (parsed.protocol !== "http:" && parsed.protocol !== "https:") throw new Error(`URL scheme must be http or https: ${value}`);
-  if (parsed.username || parsed.password) throw new Error(`URL credentials are not allowed: ${value}`);
+  if (parsed.protocol !== "http:" && parsed.protocol !== "https:") throw new Error("URL scheme must be http or https.");
+  if (parsed.username || parsed.password) throw new Error("URL credentials are not allowed.");
   parsed.hash = "";
   return parsed.toString();
 }