privateer-agent 0.12.12 → 0.12.13

package/README.md

@@ -72,7 +72,7 @@ What Privateer adds is a *moat* of Pi extensions layered on top:
 | `privateer-privacy` | `pi-privacy` — TEE attestation, ZDR routing, on-device PII gate — bound to the account tier resolver |
 | `privateer-account` | `/signin` billed inference against a Privateer account (device flow) |
 | `privateer-posture`, `privateer-tools` | live attestation shield + Privateer tool pack |
-| `rpiv-web-tools` | private-by-default web search (self-hosted SearXNG, no WebView) |
+| `privateer-web` | `web_search`/`web_fetch` — your account's search once signed in, or your own provider (`rpiv-web-tools`: self-hosted SearXNG, Brave, Tavily…) |
 | `rpiv-ask-user-question` | `ask_user_question` — a structured questionnaire the agent puts to you instead of guessing |
 | `pi-mcp-adapter`, `pi-subagents` | MCP servers · bounded parallel sub-agents |
 
@@ -612,8 +612,15 @@ drop your own into `~/.privateer/agent/extensions/` and it loads the same way, g
   approve it once.
 - **Workflows** — declarative multi-step pipelines the harbor executes; see
   [Workflows](#workflows).
-- **Web tools** (`rpiv-web-tools`) — private-by-default web search/fetch with pluggable backends
-  (self-hosted SearXNG for fully private search).
+- **Web tools** (`privateer-web`) — `web_search` and `web_fetch`, by one of two routes. Sign in
+  and they run on your Privateer account: no API key to obtain or keep on the machine, metered
+  against the account's daily web allowance, with the search itself made server-side (the derived
+  query is visible to Privateer — the page you fetch and the conversation around it are not).
+  Configure a provider of your own with `/web-tools` and that wins instead — `rpiv-web-tools` with
+  pluggable backends, including self-hosted SearXNG for fully private search, which is why a
+  provider you chose is never overridden. `PRIVATEER_WEB_SEARCH=privateer` picks the account route
+  even when you hold a key. Unattended runs (harbor, channels, ACP) always take the account route:
+  a routine must not hold a provider key it could be prompt-injected into leaking.
 - **Ask user question** (`rpiv-ask-user-question`) — when a request is underspecified the agent
   raises a structured questionnaire (typed options, multi-select, markdown previews, or type your
   own answer) instead of guessing. Ungated by design — it only asks you something.
@@ -631,6 +638,7 @@ drop your own into `~/.privateer/agent/extensions/` and it loads the same way, g
 | `/connect` · `/mcp` | add, enable, or remove MCP connectors / see what actually connected |
 | `/speak` · `/talk` | read answers aloud / voice input (**alt+t** is push-to-talk) |
 | `/extensions` | list loaded Pi extensions |
+| `/web-tools` | point `web_search`/`web_fetch` at a search provider of your own (signed in, they already work on your account) |
 | `/init` | scaffold a starter `PRIVATEER.md` in this directory |
 | `/update` · `/privateer` | update to the latest release / Privateer status and posture |
 

package/extensions/privateer-web.ts

@@ -0,0 +1,100 @@
+// `web_search` / `web_fetch` for the terminal — one route, chosen at load.
+//
+// WHAT CHANGED. The TUI used to load @juicesharp/rpiv-web-tools and nothing else, so web
+// access at a prompt meant going and getting a provider key first. Being signed in to a
+// Privateer account bought you nothing here, even though the same account's unattended
+// runs (harbor, channels, ACP) have searched through /api/rag/* all along. Now signing in
+// is enough: with no provider of your own configured, the account's search is what the
+// tools call — Brave server-side, no key on the machine, the account's daily allowance
+// and metering, and the query visible to Privateer's servers (never say a session that
+// searches this way is fully private).
+//
+// PRECEDENCE, AND WHY THIS FILE EXISTS. Both packs register the SAME two tool names, and
+// Pi resolves duplicates first-registration-wins — a session holding both would silently
+// get whichever loaded first. So exactly one of them may register, and something has to
+// decide which: this extension, once per process, via tools/webMode.ts. A configured
+// provider always wins. Someone who set up a self-hosted SearXNG picked the more private
+// route deliberately, and re-pointing their searches at our servers (billing their account
+// for them) would quietly undo that. `PRIVATEER_WEB_SEARCH=privateer` or `"provider":
+// "privateer"` in the rpiv config is the way back for someone who holds a key and wants
+// the account path anyway.
+//
+// WHAT IS STILL LIVE, AND WHAT NEEDS A RELAUNCH. `/signin` mid-session works: the account
+// tools are registered up front and check for credentials per call, so a signed-out
+// terminal says how to fix it rather than silently lacking a web tool. Configuring a
+// provider mid-session does NOT switch routes — the registration already happened — so it
+// takes effect at the next launch. `/web-tools` is registered either way, so that
+// configuration is always reachable.
+//
+// THE PACK IS OPTIONAL. rpiv-web-tools is a dependency the launcher drops if it didn't
+// resolve (a partial or hoisted install). Every import of it here is dynamic and
+// failure-tolerant: without it there is no user provider to choose, so the account route
+// is the only one left — which is the honest answer, not a fallback.
+//
+// The specifiers are built rather than written as literals on purpose: it keeps tsc from
+// pulling that package's own .ts sources into our typecheck, the same trick config/moat.ts
+// uses for pi-mcp-adapter.
+import { guardedWebToolDefinitions } from "../src/tools/web.ts";
+import { readWebToolsConfig, resolveWebMode, type ProviderMetaLike } from "../src/tools/webMode.ts";
+
+const RPIV_WEB_TOOLS = "@juicesharp/rpiv-web-tools";
+
+// What a signed-out terminal hears when the model reaches for the web. Both exits are
+// named because both are real: sign in and it works on the account's allowance, or bring
+// a provider of your own and the query never touches our servers.
+const SIGNED_OUT_HINT =
+  "Web access needs a Privateer account: run /signin to search on your account's " +
+  "allowance, or /web-tools to use your own search provider (a Brave/Tavily key, or a " +
+  "self-hosted SearXNG) instead.";
+
+interface RpivWebTools {
+  registerWebSearchTool(pi: unknown): void;
+  registerWebFetchTool(pi: unknown): void;
+  registerWebSearchConfigCommand(pi: unknown): void;
+  providers: readonly ProviderMetaLike[];
+}
+
+async function loadRpivWebTools(): Promise<RpivWebTools | null> {
+  try {
+    // Two entry points: the package root registers the tools and the `/web-tools`
+    // command, and providers/ carries the metadata (which env var holds which provider's
+    // key, which providers are self-hosted) that the precedence rules read. Taking that
+    // metadata from the package rather than restating it here is what keeps our notion of
+    // "configured" from drifting from theirs when it gains a provider.
+    const [tools, providers] = await Promise.all([
+      import(`${RPIV_WEB_TOOLS}/index.ts`),
+      import(`${RPIV_WEB_TOOLS}/providers/index.ts`),
+    ]);
+    if (typeof tools?.registerWebSearchTool !== "function") return null;
+    return {
+      registerWebSearchTool: tools.registerWebSearchTool,
+      registerWebFetchTool: tools.registerWebFetchTool,
+      registerWebSearchConfigCommand: tools.registerWebSearchConfigCommand,
+      providers: Array.isArray(providers?.PROVIDERS) ? providers.PROVIDERS : [],
+    };
+  } catch {
+    return null;
+  }
+}
+
+export default async function privateerWeb(pi: any): Promise<void> {
+  const rpiv = await loadRpivWebTools();
+  const decision = resolveWebMode({ providers: rpiv?.providers ?? [], config: readWebToolsConfig() });
+
+  if (rpiv && decision.mode === "own") {
+    // The user's own provider, run by the package that owns it — its interceptors, its
+    // guidance overrides, its truncation. We are choosing between packs, not wrapping one.
+    rpiv.registerWebSearchTool(pi);
+    rpiv.registerWebFetchTool(pi);
+  } else {
+    for (const def of guardedWebToolDefinitions(SIGNED_OUT_HINT)) pi.registerTool?.(def);
+  }
+
+  // Always available, whichever pack won: it is how a user on the account route configures
+  // a provider of their own, and the only place to see which keys are resolving.
+  try {
+    rpiv?.registerWebSearchConfigCommand(pi);
+  } catch {
+    /* the command is a convenience — never let it cost the session its web tools */
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "privateer-agent",
-  "version": "0.12.12",
+  "version": "0.12.13",
   "description": "Privacy-first terminal coding agent — bring your own model across 20 providers (Anthropic, OpenAI, OpenRouter, Google, local Ollama…). Safe-by-default permissions, MCP, sub-agents, workflows, and verifiable TEE inference. Built on the Pi toolkit.",
   "type": "module",
   "license": "MIT",

package/src/config/moatManifest.json

@@ -14,12 +14,12 @@
     { "name": "privateer-hints", "entry": "extensions/privateer-hints.ts", "note": "rotating tips in the working line + /hints" },
     { "name": "privateer-update", "entry": "extensions/privateer-update.ts", "note": "tool pack updates in place — banner flag + /update" },
     { "name": "privateer-speak", "entry": "extensions/privateer-speak.ts", "note": "spoken responses (/speak) + voice input (/talk) — pi-speak + confidential account TTS/STT" },
-    { "name": "rpiv-web-tools", "dep": ["@juicesharp/rpiv-web-tools", "index.ts"], "note": "private web tools (user's own provider key)" },
+    { "name": "privateer-web", "entry": "extensions/privateer-web.ts", "note": "web_search/web_fetch — account search when signed in, else the user's own provider (rpiv-web-tools)" },
     { "name": "rpiv-ask-user-question", "dep": ["@juicesharp/rpiv-ask-user-question", "index.ts"], "note": "ask_user_question" },
     { "name": "pi-mcp-adapter", "dep": ["pi-mcp-adapter", "index.ts"], "note": "MCP servers as first-class tools" },
     { "name": "pi-subagents", "dep": ["pi-subagents", "src", "extension", "index.ts"], "note": "bounded parallel sub-agents" }
   ],
-  "retired": ["pi-privacy", "pi-web-access", "pi-hypa"],
+  "retired": ["pi-privacy", "pi-web-access", "pi-hypa", "rpiv-web-tools"],
   "reservedAliases": [
     "@juicesharp/rpiv-web-tools",
     "@juicesharp/rpiv-ask-user-question"

package/src/tools/web.ts

@@ -18,20 +18,29 @@
 // access is a switch on the agent rather than an unconditional capability. Never
 // describe a routine that searches as fully private.
 //
-// SCOPE. These are the UNATTENDED paths' web tools: the harbor (routines, tasks,
-// workflow agent steps) and channels, both of which build a session from an explicit
-// extensionFactories list. The interactive TUI is deliberately untouched — the launcher
-// loads @juicesharp/rpiv-web-tools for it instead, which registers tools by these same
-// two names against a provider key the user configures themselves. A person at a terminal
-// choosing their own search provider is fine; an unattended run holding that key is not.
+// SCOPE. Two kinds of session reach these tools, by two different routes.
 //
-// The two must never meet: Pi resolves duplicate tool names first-registration-wins, so a
-// session that loaded both would silently get whichever came first. That is why the moat's
-// packs reach the TUI as `-e` args and the unattended paths as factories — one route each,
-// chosen per process, and config/moat.ts's per-kind table is where the choice is recorded.
+//   Unattended — the harbor (routines, tasks, workflow agent steps), channels and ACP
+//   build their session from an explicit extensionFactories list, and makeWebTools() is
+//   in it whenever webEnabled(). These paths get the account API and nothing else, for
+//   the reason above: an unattended run must not hold a provider key.
+//
+//   Interactive — the TUI takes ONE of these tools and @juicesharp/rpiv-web-tools, which
+//   registers the same two names against a provider the user configured themselves.
+//   extensions/privateer-web.ts makes that choice once per process: a configured provider
+//   wins (it is the user's own decision, and a self-hosted SearXNG is more private than
+//   this path), otherwise these tools serve a signed-in account. See tools/webMode.ts for
+//   the precedence and the escape hatches. Signing in mid-session is enough to start
+//   searching — guardedWebToolDefinitions() checks credentials per call, not at load.
+//
+// The two implementations must never meet: Pi resolves duplicate tool names
+// first-registration-wins, so a session that loaded both would silently get whichever came
+// first. That is why the moat's packs reach the TUI as `-e` args and the unattended paths
+// as factories — one route each, chosen per process, and config/moat.ts's per-kind table
+// (plus privateer-web.ts, for the TUI) is where the choice is recorded.
 
 import { Type } from "typebox";
-import { apiRequest } from "../auth/privateer.ts";
+import { apiRequest, hasCredentials } from "../auth/privateer.ts";
 
 /** Tool names these definitions register, for allow-list construction. */
 export const WEB_TOOL_NAMES = ["web_search", "web_fetch"] as const;
@@ -229,8 +238,8 @@ export const webFetchToolDefinition = {
 
 /**
  * Extension factory registering both tools. Used by the unattended paths, which build
- * their session from an explicit `extensionFactories` list (see config/moat.ts); the
- * interactive TUI gets rpiv-web-tools instead, per the SCOPE note above.
+ * their session from an explicit `extensionFactories` list (see config/moat.ts) only once
+ * webEnabled() has confirmed there are credentials to search with.
  */
 export function makeWebTools() {
   return (pi: { registerTool?: (def: unknown) => void }): void => {
@@ -238,3 +247,33 @@ export function makeWebTools() {
     pi.registerTool?.(webFetchToolDefinition);
   };
 }
+
+interface WebToolDefinition {
+  name: string;
+  execute(toolCallId: string, params: any): Promise<unknown>;
+}
+
+/**
+ * The same two tools, with a sign-in check in front of each call — the INTERACTIVE form.
+ *
+ * The unattended paths decide once, at session build, whether web access exists at all
+ * (webEnabled() is credentials-gated, and a run with no account simply has no such tool).
+ * A terminal can't work that way: the session outlives the decision, `/signin` happens in
+ * the middle of it, and each extension gets its own module instance under jiti — so the
+ * question has to be asked when the tool RUNS. loadCredentials() deliberately doesn't
+ * memoize a negative result, which is what makes that cheap and correct.
+ *
+ * `hint` is what a signed-out session hears. It belongs to the caller because only the
+ * caller knows what the user can do about it — the terminal can offer `/signin` and
+ * `/web-tools`, a headless host has neither.
+ */
+export function guardedWebToolDefinitions(hint: string): unknown[] {
+  const guard = <T extends WebToolDefinition>(def: T): T => ({
+    ...def,
+    async execute(toolCallId: string, params: any) {
+      if (!hasCredentials()) return text(hint);
+      return def.execute(toolCallId, params);
+    },
+  });
+  return [guard(webSearchToolDefinition), guard(webFetchToolDefinition)];
+}

package/src/tools/webMode.ts

@@ -0,0 +1,198 @@
+// Which web route an INTERACTIVE session takes — the user's own search provider, or
+// Privateer's account-backed search.
+//
+// WHY THERE IS A CHOICE AT ALL. Two implementations of `web_search`/`web_fetch` ship
+// with this agent and they buy different things:
+//
+//   own       @juicesharp/rpiv-web-tools against a provider the user configured
+//             themselves (a Brave/Tavily/… key, or a self-hosted SearXNG). The query
+//             reaches that provider and nobody else — a self-hosted SearXNG is the most
+//             private web access we can offer — but it only exists once the user has
+//             gone and got a key.
+//   privateer tools/web.ts against the account API (/api/rag/*, Brave server-side). No
+//             key to obtain, no key on the machine, per-account daily caps and metering —
+//             but the derived query is visible to Privateer's servers.
+//
+// Before this, the terminal only ever got the first one, so a signed-in user at a prompt
+// had no web access until they went and configured a provider, while the same account's
+// unattended runs (harbor, channels, ACP) searched fine. Signing in is now enough.
+//
+// PRECEDENCE: A CONFIGURED PROVIDER ALWAYS WINS. Account search fills a gap; it never
+// overrides a decision the user already made. Someone who stood up a SearXNG chose the
+// more private route on purpose, and quietly re-pointing their searches at our servers
+// (and billing their account for them) would undo that choice without telling them.
+// So: configured ⇒ own, otherwise ⇒ privateer, with `PRIVATEER_WEB_SEARCH` or a
+// `"provider": "privateer"` in the rpiv config as the escape hatch for someone who
+// holds a key but wants the account path anyway.
+//
+// ONE ROUTE PER PROCESS. Pi resolves duplicate tool names first-registration-wins, so a
+// session that loaded both packs would silently get whichever came first. The decision
+// is therefore made ONCE, at extension load (extensions/privateer-web.ts), and only the
+// winning pack registers its tools — see the SCOPE note in tools/web.ts. The consequence
+// to know: configuring a provider mid-session takes effect at the next launch. Signing IN
+// mid-session does not need one, because the account tools are already registered and
+// only check credentials when called.
+//
+// This module is the decision, kept separate from the extension so it can be tested
+// without Pi or jiti in the picture. No Pi imports, no side effects on load.
+
+import { existsSync, readFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { join } from "node:path";
+
+export type WebMode = "own" | "privateer";
+
+/** The fields we read off one of rpiv-web-tools' provider descriptors. */
+export interface ProviderMetaLike {
+  name: string;
+  /** API-key env var, if the provider takes a key (BRAVE_SEARCH_API_KEY, …). */
+  envVar?: string;
+  /** Base-URL env var. Only self-hosted providers (SearXNG, Ollama) declare one. */
+  baseUrlEnvVar?: string;
+}
+
+/** The fields we read out of ~/.config/rpiv-web-tools/config.json. */
+export interface WebToolsConfigLike {
+  provider?: string;
+  /** Legacy top-level Brave key, still honoured by rpiv-web-tools' own resolver. */
+  apiKey?: string;
+  apiKeys?: Record<string, string>;
+  baseUrls?: Record<string, string>;
+}
+
+/** Config-file / env value that pins the account path even when a provider is configured. */
+export const PRIVATEER_PROVIDER_NAME = "privateer";
+
+/** Force one route or the other: PRIVATEER_WEB_SEARCH=privateer | own. */
+export const WEB_MODE_ENV = "PRIVATEER_WEB_SEARCH";
+
+// Mirrors rpiv-web-tools' own DEFAULT_PROVIDER_NAME. It matters because the config file
+// may name no provider at all, in which case that package still resolves a Brave key —
+// so an unset `provider` with BRAVE_SEARCH_API_KEY in the environment is a configured
+// user, and we must not take their searches off it.
+const DEFAULT_PROVIDER_NAME = "brave";
+
+// The one provider whose key was historically stored at the top level, again mirroring
+// rpiv-web-tools: `config.apiKey` is still a working Brave key there until the next save
+// migrates it into `apiKeys.brave`.
+const LEGACY_TOP_LEVEL_KEY_PROVIDER = "brave";
+
+/** Where rpiv-web-tools keeps its config (@juicesharp/rpiv-config's `configPath`). */
+export function webToolsConfigPath(): string {
+  return join(homedir(), ".config", "rpiv-web-tools", "config.json");
+}
+
+/**
+ * Read that config, tolerantly. Every failure — absent, unreadable, malformed, not an
+ * object — degrades to `{}`, i.e. "nothing configured", which is the same fail-soft
+ * policy rpiv-web-tools applies to its own file. A broken config must not decide the
+ * route by throwing at extension load.
+ */
+export function readWebToolsConfig(path: string = webToolsConfigPath()): WebToolsConfigLike {
+  try {
+    if (!existsSync(path)) return {};
+    const parsed: unknown = JSON.parse(readFileSync(path, "utf8"));
+    if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return {};
+    return parsed as WebToolsConfigLike;
+  } catch {
+    return {};
+  }
+}
+
+const trimmed = (v: unknown): string | undefined => {
+  const s = typeof v === "string" ? v.trim() : "";
+  return s ? s : undefined;
+};
+
+/**
+ * Does the user have a working credential for `provider`? Deliberately the same order
+ * rpiv-web-tools' resolveProviderApiKey uses — env, then the per-provider map, then the
+ * legacy top-level field for Brave — because agreeing with it is the whole point: we are
+ * answering "would that package be able to search?", not inventing our own notion of
+ * configured.
+ */
+function resolveKey(
+  provider: string,
+  meta: ProviderMetaLike | undefined,
+  config: WebToolsConfigLike,
+  env: Record<string, string | undefined>,
+): string | undefined {
+  const envKey = meta?.envVar ? trimmed(env[meta.envVar]) : undefined;
+  if (envKey) return envKey;
+  const configKey = trimmed(config.apiKeys?.[provider]);
+  if (configKey) return configKey;
+  if (provider === LEGACY_TOP_LEVEL_KEY_PROVIDER) return trimmed(config.apiKey);
+  return undefined;
+}
+
+export interface WebModeDecision {
+  mode: WebMode;
+  /** The rpiv-web-tools provider the decision was made about ("brave" when unset). */
+  provider: string;
+  /** Why, in a few words — for tests and for the `/web-tools` diagnostic line. */
+  reason: string;
+}
+
+/**
+ * Decide the route for this process.
+ *
+ * `providers` is rpiv-web-tools' PROVIDERS metadata, passed in rather than imported so
+ * this stays testable and so a missing package (the pack is optional — the launcher drops
+ * a shim whose target isn't installed) degrades to "no provider is configured", which is
+ * the truth: without the package there is nothing to configure.
+ */
+export function resolveWebMode(opts: {
+  providers: readonly ProviderMetaLike[];
+  config: WebToolsConfigLike;
+  env?: Record<string, string | undefined>;
+}): WebModeDecision {
+  const { providers, config } = opts;
+  const env = opts.env ?? process.env;
+
+  const configured = trimmed(config.provider);
+  const provider = configured ?? DEFAULT_PROVIDER_NAME;
+
+  // 1. The explicit override, either way. Note "own" is honoured even with nothing
+  //    configured: the user gets rpiv-web-tools' own "run /web-tools" prompt, which is
+  //    the right answer for someone who has just told us they want their own provider.
+  const forced = trimmed(env[WEB_MODE_ENV])?.toLowerCase();
+  if (forced === PRIVATEER_PROVIDER_NAME) return { mode: "privateer", provider, reason: `${WEB_MODE_ENV} set` };
+  if (forced === "own") return { mode: "own", provider, reason: `${WEB_MODE_ENV} set` };
+
+  // 2. `"provider": "privateer"` in the config file — the persistent form of the same
+  //    choice, for someone who holds a key but wants the account path anyway. It is not a
+  //    provider rpiv-web-tools knows (its factory would throw on it), which is exactly why
+  //    it is safe to borrow as our marker: it can only have been set to mean this.
+  if (configured?.toLowerCase() === PRIVATEER_PROVIDER_NAME) {
+    return { mode: "privateer", provider, reason: "account search pinned in config" };
+  }
+
+  const meta = providers.find((p) => p.name === provider);
+
+  // 3. A key for the active provider — from the environment or the config file.
+  if (resolveKey(provider, meta, config, env)) {
+    return { mode: "own", provider, reason: `${provider} key configured` };
+  }
+
+  // 4. Self-hosted providers (SearXNG, Ollama) need no key — SELECTING one is the whole
+  //    configuration, since rpiv-web-tools falls back to the provider's default URL. So a
+  //    bare `"provider": "searxng"` is a working setup and must not be overridden.
+  //
+  //    Only a SELECTED provider counts, here and above. A URL env var alone is not read as
+  //    intent, however tempting: Ollama's is OLLAMA_HOST, which is set on any machine
+  //    running a local model for reasons that have nothing to do with search. Honouring it
+  //    would hand that user rpiv-web-tools with its unconfigured default provider — a
+  //    terminal whose every search fails — instead of the account search they can actually
+  //    use. Matching what rpiv-web-tools itself would resolve is the rule; guessing at
+  //    half-configured states is not.
+  if (configured && meta?.baseUrlEnvVar) {
+    return { mode: "own", provider, reason: `${provider} selected (self-hosted)` };
+  }
+
+  // 5. A provider we don't recognise was named — a newer rpiv-web-tools than the metadata
+  //    we were handed, or a typo. Either way the user made a choice about their own
+  //    provider; let that package own the outcome, including telling them it's unknown.
+  if (configured && !meta) return { mode: "own", provider, reason: `unrecognised provider "${provider}"` };
+
+  return { mode: "privateer", provider, reason: "no provider configured" };
+}