agentic-pi 0.2.2 → 0.2.3

package/README.md

@@ -42,8 +42,8 @@ single line you parse.
 
 Pi explicitly does not support MCP. agentic-pi ships a native Pi extension
 exposing **31 GitHub tools** ported from lastlight's `mcp-github-app`:
-clone/push, issues, PRs, reviews, labels, search. Tools are registered with
-the `github_` prefix to match opencode's MCP-server-name convention.
+clone/push, issues, PRs, reviews, labels, search. Tool names are prefixed
+with `github_`.
 
 Auth is opinionated: **GitHub App credentials preferred**, static
 `GITHUB_TOKEN` only as a low-trust fallback. JWT-minted installation tokens
@@ -81,38 +81,29 @@ The `extension_status` JSONL event always reports `status`, `reason`,
 `message`, `profile`, and `toolCount` so the orchestrator can log the
 outcome programmatically without parsing stderr.
 
-### 5. Models named the way opencode names them
+### 5. Model selection
 
-`--model provider/id` accepts the exact string format opencode used
-(`openai/gpt-5.5`, `anthropic/claude-opus-4-5`, etc.). Credentials come from
-environment variables (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
-`OPENROUTER_API_KEY`) or Pi's `~/.pi/agent/auth.json` if you've logged in
-interactively. Provider/id mapping is delegated to `@earendil-works/pi-ai`'s
-`getModel()`.
+`--model provider/id` (e.g. `anthropic/claude-opus-4-5`, `openai/gpt-4o`).
+Credentials come from environment variables (`OPENAI_API_KEY`,
+`ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`) or Pi's `~/.pi/agent/auth.json`
+if you've logged in interactively. Provider/id mapping is delegated to
+`@earendil-works/pi-ai`'s `getModel()`.
 
 `--thinking <level>` maps directly to Pi's `thinkingLevel`
 (`off`/`minimal`/`low`/`medium`/`high`/`xhigh`). Per-provider effort is
 handled by Pi.
 
-### 6. Things accepted but ignored for caller-side compatibility
-
-- `--dangerously-skip-permissions` — Pi has no permission prompts to skip
-  ("run in a container" is Pi's design stance). The flag is accepted so a
-  caller that previously spawned opencode does not need to strip it.
-- `--variant <level>` — alias for `--thinking`.
-
-### 7. Defaults that match a containerized sandbox
+### 6. Defaults that match a containerized sandbox
 
 - **`--no-session`** is intended to be the default in sandboxed runs (state
   lives outside the container).
 - **Built-in tools** (read, write, edit, bash, grep, find, ls) are enabled
   by default. Add `--no-builtin-tools` if you want a GitHub-only agent.
 - **`AGENTS.md`** in the working directory is auto-loaded as the agent's
-  system prompt — same convention Pi and opencode share. Drop your
-  workflow's `AGENTS.md` into the mounted workspace and the agent picks it
-  up.
+  system prompt — same convention Pi uses. Drop your workflow's
+  `AGENTS.md` into the mounted workspace and the agent picks it up.
 
-### 8. Optional micro-VM sandboxing via `--sandbox gondolin`
+### 7. Optional micro-VM sandboxing via `--sandbox gondolin`
 
 By default Pi's file and bash tools run on the host. Pass `--sandbox gondolin`
 and they get routed through a per-run [Gondolin](https://github.com/earendil-works/gondolin)
@@ -195,14 +186,73 @@ The **App PEM is never copied into the VM** — only the resulting token,
 which is short-lived. User-supplied `--sandbox-env GITHUB_TOKEN=…`
 overrides the auto-injected value if you need to scope down further.
 
+### 8. Safe web search via the `web-search` extension
+
+agentic-pi can register two native Pi tools — `web_search` and `web_fetch` —
+so the agent can do general-purpose research. Backed by a configurable
+provider:
+
+| Provider | API key env var | Native content extraction |
+| --- | --- | --- |
+| Tavily (default) | `TAVILY_API_KEY` | yes (search + extract) |
+| Exa | `EXA_API_KEY` | yes (search + contents) |
+| Brave Search | `BRAVE_SEARCH_API_KEY` | no — `web_fetch` falls back to a safe HTML→text extractor |
+
+**Auto-enable.** When at least one API key env var is present, the
+extension is configured automatically. With multiple keys set, priority is
+**Tavily → Exa → Brave**; override with `--web-search-provider` or the
+`WEB_SEARCH_PROVIDER` env var. Pass `--no-web-search` to suppress the
+tools entirely.
+
+**Host-process egress.** Both tools run in the agentic-pi process, **not**
+inside the Gondolin guest. That means:
+
+- The provider API host is **not** added to the Gondolin egress
+  allowlist, and the API key is **never** injected into the VM.
+- Behavior is identical under `--sandbox=none`, `--sandbox=gondolin`, and
+  when agentic-pi itself is containerized. The host's own network policy
+  controls reachability to the provider + arbitrary http(s) URLs.
+
+**Safety rails (built-in, non-configurable in v1).**
+
+| Rail | Default |
+| --- | --- |
+| URL scheme allowlist | `http`, `https` only (`web_fetch`) |
+| Request timeout | 15 s |
+| Max response bytes | 1 MiB (streamed, aborted on overflow) |
+| Max redirects | 3 (scheme re-checked at each hop) |
+| Content-type gate (`web_fetch`) | `text/*`, `application/(xhtml+xml\|xml\|json)` |
+| Max search results | 10 (regardless of `max_results` arg) |
+| Extracted text cap | ~200 KiB |
+| HTML cleaning | `<script>`, `<style>`, `<noscript>`, `<iframe>`, comments stripped before extraction |
+| Per-run call budget | 30 combined `web_search` + `web_fetch` calls (override with `--web-search-max-calls`) |
+
+When the call budget is hit, further invocations return a structured
+rate-limit error result so the agent can recover; the run is **not**
+aborted.
+
+**No SSRF blocking.** Loopback / private IP ranges are **not** blocked by
+default. Operators who care should run agentic-pi behind their own
+egress firewall.
+
+**Event stream.** A second `extension_status` event mirrors GitHub's:
+
+```jsonl
+{"type":"extension_status","extension":"web-search","status":"configured","provider":"tavily","toolCount":2,"maxCalls":30,"sessionId":"…","timestamp":"…"}
+```
+
+When skipped (no keys / `--no-web-search`), `status: "skipped"` carries a
+`reason` of `disabled-by-flag` or `no-credentials`. Misconfigurations
+(explicit provider whose key is missing, or an unknown provider name)
+surface as a warning before the run starts.
+
 ## When to use this
 
 - You have an orchestrator that calls a coding agent once per workflow
   phase, in a container, and parses a JSONL stream.
-- You used to call `opencode run --format json` and want a less-opaque
-  replacement built on a more hackable substrate.
 - You need GitHub repo operations available to the agent without standing
   up an MCP server.
+- You want safe, sandbox-mode-agnostic web search available to the agent.
 
 ## When **not** to use this
 
@@ -245,16 +295,19 @@ GITHUB_TOKEN=ghp_…
 | --- | --- |
 | `--model <provider/id>` | Required. e.g. `anthropic/claude-opus-4-5`, `openai/gpt-4o`. |
 | `--thinking <level>` | `off` \| `minimal` \| `low` \| `medium` \| `high` \| `xhigh`. |
-| `--variant <level>` | Alias for `--thinking`. |
 | `--profile <name>` | `read` \| `issues-write` \| `review-write` \| `repo-write`. Omit to disable GitHub tools entirely. |
 | `--cwd <path>` | Working directory for the agent. Default: `$PWD`. |
 | `--no-session` | Ephemeral run — do not persist session jsonl. Recommended in sandboxed containers. |
 | `--session-dir <path>` | Override session storage location. |
 | `--no-builtin-tools` | Disable Pi's `read,write,edit,bash,grep,find,ls`. |
 | `--tools <a,b,c>` | Explicit tool allowlist (combined with profile if set). |
-| `--sandbox <none\|gondolin>` | Route `read`/`write`/`edit`/`bash` through a sandbox backend. Default `none`. `gondolin` boots a QEMU micro-VM mounting cwd at `/workspace`. Requires QEMU on the host; native-only (not Docker-in-Docker). See section 8. |
+| `--sandbox <none\|gondolin>` | Route `read`/`write`/`edit`/`bash` through a sandbox backend. Default `none`. `gondolin` boots a QEMU micro-VM mounting cwd at `/workspace`. Requires QEMU on the host; native-only (not Docker-in-Docker). See section 7. |
 | `--sandbox-env KEY=VAL` | Inject env var into the sandbox VM (repeatable). Ignored when `--sandbox=none`. Auto-injects a minted `GITHUB_TOKEN`/`GH_TOKEN` when `--profile` is also active. |
-| `--dangerously-skip-permissions` | Accepted for caller-side compatibility. No-op. |
+| `--allow-host <host>` | Add host to the sandbox HTTP egress allowlist (repeatable). Ignored when `--sandbox=none`. |
+| `--no-network` | Disable sandbox HTTP egress entirely. Ignored when `--sandbox=none`. |
+| `--web-search-provider <p>` | Force web-search provider: `tavily` \| `brave` \| `exa`. Default: auto-detect by env. See section 8. |
+| `--no-web-search` | Disable the web-search extension (no `web_search`/`web_fetch` tools). |
+| `--web-search-max-calls <n>` | Cap combined `web_search` + `web_fetch` calls per run. Default: 30. |
 
 Reads the prompt from stdin. Emits JSONL on stdout. Exits 0 on `agent_end`,
 1 on fatal error.
@@ -265,6 +318,7 @@ Reads the prompt from stdin. Emits JSONL on stdout. Exits 0 on `agent_end`,
 {"type":"session","version":3,"id":"<uuid>","timestamp":"…","cwd":"…"}
 {"type":"sandbox_status","backend":"none","status":{"backend":"none"},"sessionId":"<uuid>","timestamp":"…"}
 {"type":"extension_status","extension":"github","status":"configured","profile":"read","toolCount":18,"sessionId":"<uuid>","timestamp":"…"}
+{"type":"extension_status","extension":"web-search","status":"configured","provider":"tavily","toolCount":2,"maxCalls":30,"sessionId":"<uuid>","timestamp":"…"}
 {"type":"agent_start","sessionId":"<uuid>","timestamp":"…"}
 {"type":"turn_start","sessionId":"<uuid>","timestamp":"…"}
 {"type":"message_start","message":{…},"sessionId":"<uuid>","timestamp":"…"}
@@ -343,7 +397,8 @@ console.log(result.records.length);      // full event log
 | `messages` | `unknown[]` | Full Pi message array from `agent_end`. |
 | `stats` | `{userMessages, assistantMessages, toolCalls, toolResults, tokens: {input, output, cacheRead, cacheWrite, total}, cost}` \| `undefined` | Token + cost rollup. |
 | `sandbox` | `{backend, status}` \| `undefined` | Mirror of the `sandbox_status` event. |
-| `github` | `{status, reason, profile, toolCount}` \| `undefined` | Mirror of the `extension_status` event. |
+| `github` | `{status, reason, profile, toolCount}` \| `undefined` | Mirror of the GitHub `extension_status` event. |
+| `webSearch` | `{status, reason, provider, toolCount, maxCalls}` \| `undefined` | Mirror of the web-search `extension_status` event. |
 | `records` | `EmitterRecord[]` | Every JSONL record in order. Same shape that the CLI writes. |
 | `warnings` | `string[]` | Warnings that would have gone to stderr in CLI mode. |
 
@@ -394,6 +449,7 @@ which walks `test/` for `*.test.ts`.
 | `test/models.test.ts` | `provider/id` parsing including openrouter triple-slash | — |
 | `test/extensions/github/profiles.test.ts` | Profile → tool allowlist (counts, superset structure, scope tiering) | — |
 | `test/extensions/github/credentials.test.ts` | `assertSafeToken` and `credentialsFilePath` validation | — |
+| `test/extensions/web-search/*.test.ts` | Provider selection, extension wiring, safe-fetch rails, HTML extraction, rate limiter, per-provider normalization (all with injected `fetchImpl`) | — |
 | `test/sandbox/preflight.test.ts` | Preflight returns a structured ok\|error result | — |
 | `test/run.integration.test.ts` | Programmatic `run()`: RunResult populated, onEvent fires for every record, **child-process check confirms zero stdout/stderr leak from library** | `OPENAI_API_KEY` not set |
 | `test/run-sandbox.integration.test.ts` | `run({ sandbox: "gondolin" })` boots a VM, agent's `write` tool produces a host file via the mount | `OPENAI_API_KEY` not set OR QEMU/preflight unavailable |

package/dist/args.d.ts

@@ -60,6 +60,22 @@ export interface RunConfig {
      * entirely. Ignored when `sandbox === "none"`.
      */
     allowedHttpHosts?: string[] | null;
+    /**
+     * Web-search extension toggle. Default: true (auto-enables when a
+     * provider API key env var is present). Pass `--no-web-search` to
+     * force-disable.
+     */
+    webSearch: boolean;
+    /**
+     * Explicit web-search provider. Overrides auto-detection by env var.
+     * Set via `--web-search-provider <tavily|brave|exa>`.
+     */
+    webSearchProvider?: string;
+    /**
+     * Per-run cap on combined web_search + web_fetch calls. Default: 30.
+     * Set via `--web-search-max-calls <n>`.
+     */
+    webSearchMaxCalls?: number;
 }
 export declare function printHelp(): void;
 export declare function parseArgs(argv: string[]): RunConfig;

package/dist/args.js

@@ -34,6 +34,15 @@ Flags:
                               Ignored when --sandbox=none.
   --no-network               Disable HTTP egress from the sandbox entirely.
                               Ignored when --sandbox=none.
+  --web-search-provider <p>  Force a web-search provider: tavily | brave | exa.
+                              Default: auto-detect from env (Tavily > Exa > Brave).
+                              Provider's API key env var must be set:
+                              TAVILY_API_KEY, EXA_API_KEY, or BRAVE_SEARCH_API_KEY.
+  --no-web-search            Disable the web-search extension entirely
+                              (web_search / web_fetch tools not registered).
+  --web-search-max-calls <n> Cap combined web_search + web_fetch calls per run.
+                              Default: 30. When exceeded, further calls return a
+                              structured error result.
   --sandbox-image <name>     Image to boot when --sandbox=gondolin. Values:
                               'default' (recommended) — bundled agentic-pi-dev image
                                 with git/gh/node/python/rust baked in (auto-downloaded).
@@ -54,6 +63,7 @@ export function parseArgs(argv) {
         noBuiltinTools: false,
         dangerouslySkipPermissions: false,
         sandbox: "none",
+        webSearch: true,
     };
     for (let i = 0; i < argv.length; i++) {
         const arg = argv[i];
@@ -147,6 +157,25 @@ export function parseArgs(argv) {
             case "--no-network":
                 config.allowedHttpHosts = null;
                 break;
+            case "--no-web-search":
+                config.webSearch = false;
+                break;
+            case "--web-search-provider": {
+                const v = next().trim();
+                if (!v)
+                    throw new Error("--web-search-provider requires a value");
+                config.webSearchProvider = v;
+                break;
+            }
+            case "--web-search-max-calls": {
+                const v = next();
+                const n = Number(v);
+                if (!Number.isFinite(n) || Math.floor(n) !== n || n < 1) {
+                    throw new Error(`--web-search-max-calls must be a positive integer (got '${v}')`);
+                }
+                config.webSearchMaxCalls = n;
+                break;
+            }
             case "-h":
             case "--help":
                 printHelp();

package/dist/args.js.map

@@ -1 +1 @@
-{"version":3,"file":"args.js","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AA2DH,MAAM,UAAU,SAAS;IACvB,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAsCtB,CAAC,CAAC;AACH,CAAC;AAED,MAAM,UAAU,SAAS,CAAC,IAAc;IACtC,MAAM,MAAM,GAAc;QACxB,KAAK,EAAE,EAAE;QACT,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;QAClB,SAAS,EAAE,KAAK;QAChB,cAAc,EAAE,KAAK;QACrB,0BAA0B,EAAE,KAAK;QACjC,OAAO,EAAE,MAAM;KAChB,CAAC;IAEF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,MAAM,IAAI,GAAG,GAAW,EAAE;YACxB,MAAM,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACpB,IAAI,CAAC,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,QAAQ,GAAG,mBAAmB,CAAC,CAAC;YACrE,OAAO,CAAC,CAAC;QACX,CAAC,CAAC;QACF,QAAQ,GAAG,EAAE,CAAC;YACZ,KAAK,SAAS,CAAC;YACf,KAAK,IAAI;gBACP,MAAM,CAAC,KAAK,GAAG,IAAI,EAAE,CAAC;gBACtB,MAAM;YACR,KAAK,YAAY,CAAC;YAClB,KAAK,WAAW,CAAC,CAAC,CAAC;gBACjB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,CAAC,KAAK,EAAE,SAAS,EAAE,KAAK,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC;oBACtE,MAAM,IAAI,KAAK,CAAC,6BAA6B,CAAC,EAAE,CAAC,CAAC;gBACpD,CAAC;gBACD,MAAM,CAAC,QAAQ,GAAG,CAA0B,CAAC;gBAC7C,MAAM;YACR,CAAC;YACD,KAAK,WAAW;gBACd,MAAM,CAAC,OAAO,GAAG,IAAI,EAAE,CAAC;gBACxB,MAAM;YACR,KAAK,OAAO;gBACV,MAAM,CAAC,GAAG,GAAG,IAAI,EAAE,CAAC;gBACpB,MAAM;YACR,KAAK,cAAc;gBACjB,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;gBACxB,MAAM;YACR,KAAK,eAAe;gBAClB,MAAM,CAAC,UAAU,GAAG,IAAI,EAAE,CAAC;gBAC3B,MAAM;YACR,KAAK,oBAAoB;gBACvB,MAAM,CAAC,cAAc,GAAG,IAAI,CAAC;gBAC7B,MAAM;YACR,KAAK,SAAS;gBACZ,MAAM,CAAC,KAAK,GAAG,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;gBACtE,MAAM;YACR,KAAK,gCAAgC;gBACnC,MAAM,CAAC,0BAA0B,GAAG,IAAI,CAAC;gBACzC,MAAM;YACR,KAAK,WAAW,CAAC,CAAC,CAAC;gBACjB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,KAAK,MAAM,IAAI,CAAC,KAAK,UAAU,EAAE,CAAC;oBACrC,MAAM,IAAI,KAAK,CAAC,sBAAsB,CAAC,8BAA8B,CAAC,CAAC;gBACzE,CAAC;gBACD,MAAM,CAAC,OAAO,GAAG,CAAC,CAAC;gBACnB,MAAM;YACR,CAAC;YACD,KAAK,iBAAiB,CAAC,CAAC,CAAC;gBACvB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBACnB,MAAM,IAAI,KAAK,CAAC,4CAA4C,CAAC,CAAC;gBAChE,CAAC;gBACD,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC;gBACxB,MAAM;YACR,CAAC;YACD,KAAK,eAAe,CAAC,CAAC,CAAC;gBACrB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,MAAM,EAAE,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;gBAC1B,IAAI,EAAE,GAAG,CAAC,EAAE,CAAC;oBACX,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,IAAI,CAAC,CAAC;gBAChE,CAAC;gBACD,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBAC3B,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;gBAC5B,IAAI,CAAC,0BAA0B,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;oBAC1C,MAAM,IAAI,KAAK,CAAC,6DAA6D,GAAG,IAAI,CAAC,CAAC;gBACxF,CAAC;gBACD,MAAM,CAAC,UAAU,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,UAAU,IAAI,EAAE,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC;gBACjE,MAAM;YACR,CAAC;YACD,KAAK,cAAc,CAAC,CAAC,CAAC;gBACpB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;gBACxB,IAAI,CAAC,CAAC;oBAAE,MAAM,IAAI,KAAK,CAAC,wCAAwC,CAAC,CAAC;gBAClE,IAAI,CAAC,oBAAoB,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;oBAClC,MAAM,IAAI,KAAK,CAAC,6CAA6C,CAAC,IAAI,CAAC,CAAC;gBACtE,CAAC;gBACD,gEAAgE;gBAChE,gEAAgE;gBAChE,+DAA+D;gBAC/D,gEAAgE;gBAChE,kEAAkE;gBAClE,MAAM,GAAG,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,EAAE,CAAC;gBAClF,MAAM,CAAC,gBAAgB,GAAG,CAAC,GAAG,GAAG,EAAE,CAAC,CAAC,CAAC;gBACtC,MAAM;YACR,CAAC;YACD,KAAK,cAAc;gBACjB,MAAM,CAAC,gBAAgB,GAAG,IAAI,CAAC;gBAC/B,MAAM;YACR,KAAK,IAAI,CAAC;YACV,KAAK,QAAQ;gBACX,SAAS,EAAE,CAAC;gBACZ,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YAClB;gBACE,MAAM,IAAI,KAAK,CAAC,iBAAiB,GAAG,EAAE,CAAC,CAAC;QAC5C,CAAC;IACH,CAAC;IAED,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QAClB,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAC;IAC3E,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAChC,MAAM,IAAI,KAAK,CAAC,uCAAuC,MAAM,CAAC,KAAK,GAAG,CAAC,CAAC;IAC1E,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC"}
+{"version":3,"file":"args.js","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AA2EH,MAAM,UAAU,SAAS;IACvB,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CA+CtB,CAAC,CAAC;AACH,CAAC;AAED,MAAM,UAAU,SAAS,CAAC,IAAc;IACtC,MAAM,MAAM,GAAc;QACxB,KAAK,EAAE,EAAE;QACT,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;QAClB,SAAS,EAAE,KAAK;QAChB,cAAc,EAAE,KAAK;QACrB,0BAA0B,EAAE,KAAK;QACjC,OAAO,EAAE,MAAM;QACf,SAAS,EAAE,IAAI;KAChB,CAAC;IAEF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,MAAM,IAAI,GAAG,GAAW,EAAE;YACxB,MAAM,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACpB,IAAI,CAAC,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,QAAQ,GAAG,mBAAmB,CAAC,CAAC;YACrE,OAAO,CAAC,CAAC;QACX,CAAC,CAAC;QACF,QAAQ,GAAG,EAAE,CAAC;YACZ,KAAK,SAAS,CAAC;YACf,KAAK,IAAI;gBACP,MAAM,CAAC,KAAK,GAAG,IAAI,EAAE,CAAC;gBACtB,MAAM;YACR,KAAK,YAAY,CAAC;YAClB,KAAK,WAAW,CAAC,CAAC,CAAC;gBACjB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,CAAC,KAAK,EAAE,SAAS,EAAE,KAAK,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC;oBACtE,MAAM,IAAI,KAAK,CAAC,6BAA6B,CAAC,EAAE,CAAC,CAAC;gBACpD,CAAC;gBACD,MAAM,CAAC,QAAQ,GAAG,CAA0B,CAAC;gBAC7C,MAAM;YACR,CAAC;YACD,KAAK,WAAW;gBACd,MAAM,CAAC,OAAO,GAAG,IAAI,EAAE,CAAC;gBACxB,MAAM;YACR,KAAK,OAAO;gBACV,MAAM,CAAC,GAAG,GAAG,IAAI,EAAE,CAAC;gBACpB,MAAM;YACR,KAAK,cAAc;gBACjB,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;gBACxB,MAAM;YACR,KAAK,eAAe;gBAClB,MAAM,CAAC,UAAU,GAAG,IAAI,EAAE,CAAC;gBAC3B,MAAM;YACR,KAAK,oBAAoB;gBACvB,MAAM,CAAC,cAAc,GAAG,IAAI,CAAC;gBAC7B,MAAM;YACR,KAAK,SAAS;gBACZ,MAAM,CAAC,KAAK,GAAG,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;gBACtE,MAAM;YACR,KAAK,gCAAgC;gBACnC,MAAM,CAAC,0BAA0B,GAAG,IAAI,CAAC;gBACzC,MAAM;YACR,KAAK,WAAW,CAAC,CAAC,CAAC;gBACjB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,KAAK,MAAM,IAAI,CAAC,KAAK,UAAU,EAAE,CAAC;oBACrC,MAAM,IAAI,KAAK,CAAC,sBAAsB,CAAC,8BAA8B,CAAC,CAAC;gBACzE,CAAC;gBACD,MAAM,CAAC,OAAO,GAAG,CAAC,CAAC;gBACnB,MAAM;YACR,CAAC;YACD,KAAK,iBAAiB,CAAC,CAAC,CAAC;gBACvB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBACnB,MAAM,IAAI,KAAK,CAAC,4CAA4C,CAAC,CAAC;gBAChE,CAAC;gBACD,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC;gBACxB,MAAM;YACR,CAAC;YACD,KAAK,eAAe,CAAC,CAAC,CAAC;gBACrB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,MAAM,EAAE,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;gBAC1B,IAAI,EAAE,GAAG,CAAC,EAAE,CAAC;oBACX,MAAM,IAAI,KAAK,CAAC,uCAAuC,CAAC,IAAI,CAAC,CAAC;gBAChE,CAAC;gBACD,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBAC3B,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;gBAC5B,IAAI,CAAC,0BAA0B,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;oBAC1C,MAAM,IAAI,KAAK,CAAC,6DAA6D,GAAG,IAAI,CAAC,CAAC;gBACxF,CAAC;gBACD,MAAM,CAAC,UAAU,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,UAAU,IAAI,EAAE,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC;gBACjE,MAAM;YACR,CAAC;YACD,KAAK,cAAc,CAAC,CAAC,CAAC;gBACpB,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;gBACxB,IAAI,CAAC,CAAC;oBAAE,MAAM,IAAI,KAAK,CAAC,wCAAwC,CAAC,CAAC;gBAClE,IAAI,CAAC,oBAAoB,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;oBAClC,MAAM,IAAI,KAAK,CAAC,6CAA6C,CAAC,IAAI,CAAC,CAAC;gBACtE,CAAC;gBACD,gEAAgE;gBAChE,gEAAgE;gBAChE,+DAA+D;gBAC/D,gEAAgE;gBAChE,kEAAkE;gBAClE,MAAM,GAAG,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,EAAE,CAAC;gBAClF,MAAM,CAAC,gBAAgB,GAAG,CAAC,GAAG,GAAG,EAAE,CAAC,CAAC,CAAC;gBACtC,MAAM;YACR,CAAC;YACD,KAAK,cAAc;gBACjB,MAAM,CAAC,gBAAgB,GAAG,IAAI,CAAC;gBAC/B,MAAM;YACR,KAAK,iBAAiB;gBACpB,MAAM,CAAC,SAAS,GAAG,KAAK,CAAC;gBACzB,MAAM;YACR,KAAK,uBAAuB,CAAC,CAAC,CAAC;gBAC7B,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC;gBACxB,IAAI,CAAC,CAAC;oBAAE,MAAM,IAAI,KAAK,CAAC,wCAAwC,CAAC,CAAC;gBAClE,MAAM,CAAC,iBAAiB,GAAG,CAAC,CAAC;gBAC7B,MAAM;YACR,CAAC;YACD,KAAK,wBAAwB,CAAC,CAAC,CAAC;gBAC9B,MAAM,CAAC,GAAG,IAAI,EAAE,CAAC;gBACjB,MAAM,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;gBACpB,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;oBACxD,MAAM,IAAI,KAAK,CAAC,2DAA2D,CAAC,IAAI,CAAC,CAAC;gBACpF,CAAC;gBACD,MAAM,CAAC,iBAAiB,GAAG,CAAC,CAAC;gBAC7B,MAAM;YACR,CAAC;YACD,KAAK,IAAI,CAAC;YACV,KAAK,QAAQ;gBACX,SAAS,EAAE,CAAC;gBACZ,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YAClB;gBACE,MAAM,IAAI,KAAK,CAAC,iBAAiB,GAAG,EAAE,CAAC,CAAC;QAC5C,CAAC;IACH,CAAC;IAED,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QAClB,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAC;IAC3E,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAChC,MAAM,IAAI,KAAK,CAAC,uCAAuC,MAAM,CAAC,KAAK,GAAG,CAAC,CAAC;IAC1E,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC"}

package/dist/extensions/web-search/extract.d.ts

@@ -0,0 +1,18 @@
+/**
+ * Minimal HTML → readable text extractor. No dependencies.
+ *
+ * Approach:
+ *   1. Strip <script>, <style>, <noscript>, <iframe>, and HTML comments
+ *      so the agent never sees code or hidden trackers.
+ *   2. Replace block-level tags with newlines so paragraphs stay separated.
+ *   3. Drop all other tags.
+ *   4. Decode the common named entities and any &#NN; / &#xHH; numeric
+ *      escapes.
+ *   5. Collapse runs of whitespace; cap output at MAX_BYTES.
+ *
+ * Not a Readability-style content-only extractor — for that, use Tavily or
+ * Exa's native extraction, which the provider clients invoke directly.
+ */
+export declare const EXTRACT_MAX_BYTES: number;
+export declare function extractTitle(html: string): string | undefined;
+export declare function htmlToText(html: string, maxBytes?: number): string;

package/dist/extensions/web-search/extract.js

@@ -0,0 +1,110 @@
+/**
+ * Minimal HTML → readable text extractor. No dependencies.
+ *
+ * Approach:
+ *   1. Strip <script>, <style>, <noscript>, <iframe>, and HTML comments
+ *      so the agent never sees code or hidden trackers.
+ *   2. Replace block-level tags with newlines so paragraphs stay separated.
+ *   3. Drop all other tags.
+ *   4. Decode the common named entities and any &#NN; / &#xHH; numeric
+ *      escapes.
+ *   5. Collapse runs of whitespace; cap output at MAX_BYTES.
+ *
+ * Not a Readability-style content-only extractor — for that, use Tavily or
+ * Exa's native extraction, which the provider clients invoke directly.
+ */
+export const EXTRACT_MAX_BYTES = 200 * 1024;
+const NAMED_ENTITIES = {
+    amp: "&",
+    lt: "<",
+    gt: ">",
+    quot: '"',
+    apos: "'",
+    nbsp: " ",
+    copy: "(c)",
+    reg: "(R)",
+    trade: "(TM)",
+    hellip: "…",
+    mdash: "—",
+    ndash: "–",
+    lsquo: "‘",
+    rsquo: "’",
+    ldquo: "“",
+    rdquo: "”",
+};
+function decodeEntities(s) {
+    return s.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (_m, body) => {
+        if (body.startsWith("#x") || body.startsWith("#X")) {
+            const code = parseInt(body.slice(2), 16);
+            return Number.isFinite(code) ? safeFromCodePoint(code) : _m;
+        }
+        if (body.startsWith("#")) {
+            const code = parseInt(body.slice(1), 10);
+            return Number.isFinite(code) ? safeFromCodePoint(code) : _m;
+        }
+        const mapped = NAMED_ENTITIES[body];
+        return mapped ?? _m;
+    });
+}
+function safeFromCodePoint(code) {
+    if (code < 0 || code > 0x10ffff)
+        return "";
+    try {
+        return String.fromCodePoint(code);
+    }
+    catch {
+        return "";
+    }
+}
+const BLOCK_TAGS = new Set([
+    "p", "br", "div", "section", "article", "header", "footer", "main",
+    "nav", "aside", "ul", "ol", "li", "table", "tr", "td", "th",
+    "h1", "h2", "h3", "h4", "h5", "h6", "blockquote", "hr", "pre", "code",
+    "dl", "dt", "dd", "figure", "figcaption",
+]);
+export function extractTitle(html) {
+    const m = /<title[^>]*>([\s\S]*?)<\/title>/i.exec(html);
+    if (!m)
+        return undefined;
+    const t = decodeEntities(m[1]).replace(/\s+/g, " ").trim();
+    return t.length > 0 ? t : undefined;
+}
+export function htmlToText(html, maxBytes = EXTRACT_MAX_BYTES) {
+    // 1. Strip dangerous / noise sections wholesale.
+    let s = html.replace(/<!--[\s\S]*?-->/g, "");
+    s = s.replace(/<script\b[\s\S]*?<\/script\s*>/gi, "");
+    s = s.replace(/<style\b[\s\S]*?<\/style\s*>/gi, "");
+    s = s.replace(/<noscript\b[\s\S]*?<\/noscript\s*>/gi, "");
+    s = s.replace(/<iframe\b[\s\S]*?<\/iframe\s*>/gi, "");
+    // 2. Convert block-level open/close tags to newlines so paragraphs survive.
+    s = s.replace(/<\/?([a-zA-Z][a-zA-Z0-9]*)\b[^>]*>/g, (_match, tag) => {
+        if (BLOCK_TAGS.has(tag.toLowerCase()))
+            return "\n";
+        return "";
+    });
+    // 3. Decode entities.
+    s = decodeEntities(s);
+    // 4. Normalize whitespace: collapse runs of spaces/tabs; trim per line;
+    //    collapse runs of blank lines.
+    s = s.replace(/\r\n?/g, "\n");
+    s = s
+        .split("\n")
+        .map((line) => line.replace(/[ \t\f\v]+/g, " ").trim())
+        .filter((line, i, arr) => {
+        // collapse 2+ blank lines down to 1
+        if (line !== "")
+            return true;
+        return arr[i - 1] !== "";
+    })
+        .join("\n")
+        .trim();
+    // 5. Byte cap. UTF-8 size approximation by encoding; on overflow, slice
+    //    on code-point boundary then re-decode.
+    const encoder = new TextEncoder();
+    const bytes = encoder.encode(s);
+    if (bytes.byteLength <= maxBytes)
+        return s;
+    // Decode just the prefix.
+    return new TextDecoder("utf-8", { fatal: false }).decode(bytes.subarray(0, maxBytes));
+}
+//# sourceMappingURL=extract.js.map

package/dist/extensions/web-search/extract.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"extract.js","sourceRoot":"","sources":["../../../src/extensions/web-search/extract.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAEH,MAAM,CAAC,MAAM,iBAAiB,GAAG,GAAG,GAAG,IAAI,CAAC;AAE5C,MAAM,cAAc,GAA2B;IAC7C,GAAG,EAAE,GAAG;IACR,EAAE,EAAE,GAAG;IACP,EAAE,EAAE,GAAG;IACP,IAAI,EAAE,GAAG;IACT,IAAI,EAAE,GAAG;IACT,IAAI,EAAE,GAAG;IACT,IAAI,EAAE,KAAK;IACX,GAAG,EAAE,KAAK;IACV,KAAK,EAAE,MAAM;IACb,MAAM,EAAE,GAAG;IACX,KAAK,EAAE,GAAG;IACV,KAAK,EAAE,GAAG;IACV,KAAK,EAAE,GAAG;IACV,KAAK,EAAE,GAAG;IACV,KAAK,EAAE,GAAG;IACV,KAAK,EAAE,GAAG;CACX,CAAC;AAEF,SAAS,cAAc,CAAC,CAAS;IAC/B,OAAO,CAAC,CAAC,OAAO,CAAC,gCAAgC,EAAE,CAAC,EAAE,EAAE,IAAY,EAAE,EAAE;QACtE,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YACnD,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YACzC,OAAO,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAC9D,CAAC;QACD,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YACzB,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YACzC,OAAO,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAC9D,CAAC;QACD,MAAM,MAAM,GAAG,cAAc,CAAC,IAAI,CAAC,CAAC;QACpC,OAAO,MAAM,IAAI,EAAE,CAAC;IACtB,CAAC,CAAC,CAAC;AACL,CAAC;AAED,SAAS,iBAAiB,CAAC,IAAY;IACrC,IAAI,IAAI,GAAG,CAAC,IAAI,IAAI,GAAG,QAAQ;QAAE,OAAO,EAAE,CAAC;IAC3C,IAAI,CAAC;QACH,OAAO,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,CAAC;IACpC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,EAAE,CAAC;IACZ,CAAC;AACH,CAAC;AAED,MAAM,UAAU,GAAG,IAAI,GAAG,CAAC;IACzB,GAAG,EAAE,IAAI,EAAE,KAAK,EAAE,SAAS,EAAE,SAAS,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM;IAClE,KAAK,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI;IAC3D,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,KAAK,EAAE,MAAM;IACrE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,QAAQ,EAAE,YAAY;CACzC,CAAC,CAAC;AAEH,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,MAAM,CAAC,GAAG,kCAAkC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxD,IAAI,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IACzB,MAAM,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3D,OAAO,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AACtC,CAAC;AAED,MAAM,UAAU,UAAU,CAAC,IAAY,EAAE,QAAQ,GAAG,iBAAiB;IACnE,iDAAiD;IACjD,IAAI,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,kBAAkB,EAAE,EAAE,CAAC,CAAC;IAC7C,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,kCAAkC,EAAE,EAAE,CAAC,CAAC;IACtD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,gCAAgC,EAAE,EAAE,CAAC,CAAC;IACpD,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,sCAAsC,EAAE,EAAE,CAAC,CAAC;IAC1D,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,kCAAkC,EAAE,EAAE,CAAC,CAAC;IAEtD,4EAA4E;IAC5E,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,qCAAqC,EAAE,CAAC,MAAM,EAAE,GAAW,EAAE,EAAE;QAC3E,IAAI,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC,WAAW,EAAE,CAAC;YAAE,OAAO,IAAI,CAAC;QACnD,OAAO,EAAE,CAAC;IACZ,CAAC,CAAC,CAAC;IAEH,sBAAsB;IACtB,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC;IAEtB,wEAAwE;IACxE,mCAAmC;IACnC,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;IAC9B,CAAC,GAAG,CAAC;SACF,KAAK,CAAC,IAAI,CAAC;SACX,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;SACtD,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,GAAG,EAAE,EAAE;QACvB,oCAAoC;QACpC,IAAI,IAAI,KAAK,EAAE;YAAE,OAAO,IAAI,CAAC;QAC7B,OAAO,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC;IAC3B,CAAC,CAAC;SACD,IAAI,CAAC,IAAI,CAAC;SACV,IAAI,EAAE,CAAC;IAEV,wEAAwE;IACxE,4CAA4C;IAC5C,MAAM,OAAO,GAAG,IAAI,WAAW,EAAE,CAAC;IAClC,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;IAChC,IAAI,KAAK,CAAC,UAAU,IAAI,QAAQ;QAAE,OAAO,CAAC,CAAC;IAC3C,0BAA0B;IAC1B,OAAO,IAAI,WAAW,CAAC,OAAO,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,CAAC,CAAC,MAAM,CACtD,KAAK,CAAC,QAAQ,CAAC,CAAC,EAAE,QAAQ,CAAC,CAC5B,CAAC;AACJ,CAAC"}

package/dist/extensions/web-search/index.d.ts

@@ -0,0 +1,43 @@
+/**
+ * Web-search extension entry point.
+ *
+ * Mirrors `src/extensions/github/index.ts`:
+ *   - silent skip when no provider is selected / no key present
+ *   - loud (warning-worthy) skip when the user explicitly asked for a
+ *     provider but didn't supply its key, or set an unknown name
+ *   - on success, hands a typed customTools list back to the runner
+ *
+ * Selection logic lives in `selection.ts`; this module wires the chosen
+ * provider to its tool builder and a per-run RateLimiter.
+ */
+import type { ToolDefinition } from "@earendil-works/pi-coding-agent";
+import type { ProviderName, WebSearchSkipReason } from "./types.js";
+export declare const DEFAULT_MAX_CALLS = 30;
+export interface WebSearchExtensionConfig {
+    /** When false, the extension is force-skipped (disabled-by-flag). Default: true. */
+    webSearch?: boolean;
+    /** Explicit provider override. */
+    webSearchProvider?: string;
+    /** Per-run call budget shared across web_search + web_fetch. Default: 30. */
+    webSearchMaxCalls?: number;
+    /** Env override (defaults to process.env). Injected by tests. */
+    env?: Record<string, string | undefined>;
+}
+export interface WebSearchExtensionResult {
+    /** Tools to merge into createAgentSession({ customTools }). */
+    customTools: ToolDefinition<any>[];
+    toolNames: string[];
+    status: "configured" | "skipped";
+    reason?: WebSearchSkipReason;
+    message?: string;
+    provider?: ProviderName;
+    /** The cap actually enforced (echoed for observability). */
+    maxCalls?: number;
+}
+export declare function loadWebSearchExtension(config?: WebSearchExtensionConfig): WebSearchExtensionResult;
+/**
+ * True if the skip is something the user almost certainly wants surfaced
+ * as a warning (vs. the silent "no keys set" case).
+ */
+export declare function isMisconfigurationSkip(result: WebSearchExtensionResult): boolean;
+export type { ProviderName, WebSearchSkipReason } from "./types.js";

package/dist/extensions/web-search/index.js

@@ -0,0 +1,86 @@
+/**
+ * Web-search extension entry point.
+ *
+ * Mirrors `src/extensions/github/index.ts`:
+ *   - silent skip when no provider is selected / no key present
+ *   - loud (warning-worthy) skip when the user explicitly asked for a
+ *     provider but didn't supply its key, or set an unknown name
+ *   - on success, hands a typed customTools list back to the runner
+ *
+ * Selection logic lives in `selection.ts`; this module wires the chosen
+ * provider to its tool builder and a per-run RateLimiter.
+ */
+import { createBraveProvider } from "./providers/brave.js";
+import { createExaProvider } from "./providers/exa.js";
+import { createTavilyProvider } from "./providers/tavily.js";
+import { RateLimiter } from "./rate-limit.js";
+import { selectProvider } from "./selection.js";
+import { buildWebSearchTools } from "./tools.js";
+export const DEFAULT_MAX_CALLS = 30;
+export function loadWebSearchExtension(config = {}) {
+    const env = config.env ?? process.env;
+    const input = {
+        webSearch: config.webSearch ?? true,
+        webSearchProvider: config.webSearchProvider,
+        env,
+    };
+    const selection = selectProvider(input);
+    if (selection.status === "skipped") {
+        return {
+            customTools: [],
+            toolNames: [],
+            status: "skipped",
+            reason: selection.reason,
+            message: selection.message,
+            provider: selection.provider,
+        };
+    }
+    const provider = instantiateProvider(selection.provider, selection.apiKey);
+    const maxCalls = clampMaxCalls(config.webSearchMaxCalls);
+    const limiter = new RateLimiter(maxCalls);
+    const tools = buildWebSearchTools(provider, limiter);
+    return {
+        customTools: tools,
+        toolNames: tools.map((t) => t.name),
+        status: "configured",
+        provider: selection.provider,
+        message: selection.message,
+        maxCalls,
+    };
+}
+/**
+ * True if the skip is something the user almost certainly wants surfaced
+ * as a warning (vs. the silent "no keys set" case).
+ */
+export function isMisconfigurationSkip(result) {
+    if (result.status !== "skipped")
+        return false;
+    if (result.reason === "invalid-config")
+        return true;
+    // Explicit provider asked for, but no key — louder than the generic
+    // "no creds at all" skip.
+    if (result.reason === "no-credentials" && result.provider !== undefined)
+        return true;
+    return false;
+}
+function instantiateProvider(name, apiKey) {
+    switch (name) {
+        case "tavily":
+            return createTavilyProvider({ apiKey });
+        case "brave":
+            return createBraveProvider({ apiKey });
+        case "exa":
+            return createExaProvider({ apiKey });
+    }
+}
+function clampMaxCalls(v) {
+    if (v === undefined || !Number.isFinite(v))
+        return DEFAULT_MAX_CALLS;
+    const n = Math.floor(v);
+    if (n < 1)
+        return 1;
+    if (n > 1000)
+        return 1000;
+    return n;
+}
+//# sourceMappingURL=index.js.map

package/dist/extensions/web-search/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/extensions/web-search/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAIH,OAAO,EAAE,mBAAmB,EAAE,MAAM,sBAAsB,CAAC;AAC3D,OAAO,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AACvD,OAAO,EAAE,oBAAoB,EAAE,MAAM,uBAAuB,CAAC;AAC7D,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAC;AAC9C,OAAO,EAAE,cAAc,EAAuB,MAAM,gBAAgB,CAAC;AACrE,OAAO,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAC;AAGjD,MAAM,CAAC,MAAM,iBAAiB,GAAG,EAAE,CAAC;AA0BpC,MAAM,UAAU,sBAAsB,CACpC,SAAmC,EAAE;IAErC,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,IAAK,OAAO,CAAC,GAA0C,CAAC;IAC9E,MAAM,KAAK,GAAmB;QAC5B,SAAS,EAAE,MAAM,CAAC,SAAS,IAAI,IAAI;QACnC,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;QAC3C,GAAG;KACJ,CAAC;IAEF,MAAM,SAAS,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC;IACxC,IAAI,SAAS,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;QACnC,OAAO;YACL,WAAW,EAAE,EAAE;YACf,SAAS,EAAE,EAAE;YACb,MAAM,EAAE,SAAS;YACjB,MAAM,EAAE,SAAS,CAAC,MAAM;YACxB,OAAO,EAAE,SAAS,CAAC,OAAO;YAC1B,QAAQ,EAAE,SAAS,CAAC,QAAQ;SAC7B,CAAC;IACJ,CAAC;IAED,MAAM,QAAQ,GAAG,mBAAmB,CAAC,SAAS,CAAC,QAAQ,EAAE,SAAS,CAAC,MAAM,CAAC,CAAC;IAC3E,MAAM,QAAQ,GAAG,aAAa,CAAC,MAAM,CAAC,iBAAiB,CAAC,CAAC;IACzD,MAAM,OAAO,GAAG,IAAI,WAAW,CAAC,QAAQ,CAAC,CAAC;IAC1C,MAAM,KAAK,GAAG,mBAAmB,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IAErD,OAAO;QACL,WAAW,EAAE,KAAK;QAClB,SAAS,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;QACnC,MAAM,EAAE,YAAY;QACpB,QAAQ,EAAE,SAAS,CAAC,QAAQ;QAC5B,OAAO,EAAE,SAAS,CAAC,OAAO;QAC1B,QAAQ;KACT,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,sBAAsB,CAAC,MAAgC;IACrE,IAAI,MAAM,CAAC,MAAM,KAAK,SAAS;QAAE,OAAO,KAAK,CAAC;IAC9C,IAAI,MAAM,CAAC,MAAM,KAAK,gBAAgB;QAAE,OAAO,IAAI,CAAC;IACpD,oEAAoE;IACpE,0BAA0B;IAC1B,IAAI,MAAM,CAAC,MAAM,KAAK,gBAAgB,IAAI,MAAM,CAAC,QAAQ,KAAK,SAAS;QAAE,OAAO,IAAI,CAAC;IACrF,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,mBAAmB,CAAC,IAAkB,EAAE,MAAc;IAC7D,QAAQ,IAAI,EAAE,CAAC;QACb,KAAK,QAAQ;YACX,OAAO,oBAAoB,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;QAC1C,KAAK,OAAO;YACV,OAAO,mBAAmB,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;QACzC,KAAK,KAAK;YACR,OAAO,iBAAiB,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;IACzC,CAAC;AACH,CAAC;AAED,SAAS,aAAa,CAAC,CAAqB;IAC1C,IAAI,CAAC,KAAK,SAAS,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QAAE,OAAO,iBAAiB,CAAC;IACrE,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IACxB,IAAI,CAAC,GAAG,CAAC;QAAE,OAAO,CAAC,CAAC;IACpB,IAAI,CAAC,GAAG,IAAI;QAAE,OAAO,IAAI,CAAC;IAC1B,OAAO,CAAC,CAAC;AACX,CAAC"}

package/dist/extensions/web-search/providers/brave.d.ts

@@ -0,0 +1,21 @@
+/**
+ * Brave Search provider. https://api.search.brave.com/app/documentation
+ *
+ * Endpoint used:
+ *   GET https://api.search.brave.com/res/v1/web/search?q=…&count=…
+ *   Header: X-Subscription-Token: <key>
+ *
+ * Brave has no content-extraction endpoint, so this provider has no
+ * `fetch()` method — the tool layer falls back to safeFetch + the HTML
+ * extractor for `web_fetch`.
+ *
+ * `include_domains` / `exclude_domains` are honored via client-side
+ * post-filtering so the tool's schema behaves uniformly across providers.
+ */
+import type { FetchImpl, Provider } from "../types.js";
+export interface BraveOptions {
+    apiKey: string;
+    fetchImpl?: FetchImpl;
+    baseUrl?: string;
+}
+export declare function createBraveProvider(options: BraveOptions): Provider;

package/dist/extensions/web-search/providers/brave.js

@@ -0,0 +1,73 @@
+/**
+ * Brave Search provider. https://api.search.brave.com/app/documentation
+ *
+ * Endpoint used:
+ *   GET https://api.search.brave.com/res/v1/web/search?q=…&count=…
+ *   Header: X-Subscription-Token: <key>
+ *
+ * Brave has no content-extraction endpoint, so this provider has no
+ * `fetch()` method — the tool layer falls back to safeFetch + the HTML
+ * extractor for `web_fetch`.
+ *
+ * `include_domains` / `exclude_domains` are honored via client-side
+ * post-filtering so the tool's schema behaves uniformly across providers.
+ */
+function hostMatches(url, pattern) {
+    let host;
+    try {
+        host = new URL(url).hostname.toLowerCase();
+    }
+    catch {
+        return false;
+    }
+    const p = pattern.toLowerCase();
+    return host === p || host.endsWith(`.${p}`);
+}
+export function createBraveProvider(options) {
+    const fetchImpl = options.fetchImpl ?? globalThis.fetch;
+    const baseUrl = options.baseUrl ?? "https://api.search.brave.com/res/v1";
+    return {
+        name: "brave",
+        supportsExtractedContent: false,
+        async search(params) {
+            const url = new URL(`${baseUrl}/web/search`);
+            url.searchParams.set("q", params.query);
+            // Brave's `count` caps at 20; we then post-filter and slice.
+            url.searchParams.set("count", String(Math.min(20, Math.max(1, params.maxResults * 2))));
+            const r = await fetchImpl(url.toString(), {
+                method: "GET",
+                headers: {
+                    accept: "application/json",
+                    "x-subscription-token": options.apiKey,
+                },
+            });
+            if (!r.ok) {
+                const text = await r.text().catch(() => "");
+                throw new Error(`brave search failed: http ${r.status} ${text.slice(0, 200)}`);
+            }
+            const data = (await r.json());
+            const items = (data.web?.results ?? []).map((it) => ({
+                title: it.title ?? "",
+                url: it.url ?? "",
+                snippet: it.description,
+                publishedDate: it.age,
+            }));
+            let filtered = items;
+            if (params.includeDomains?.length) {
+                const list = params.includeDomains;
+                filtered = filtered.filter((it) => it.url && list.some((d) => hostMatches(it.url, d)));
+            }
+            if (params.excludeDomains?.length) {
+                const list = params.excludeDomains;
+                filtered = filtered.filter((it) => !(it.url && list.some((d) => hostMatches(it.url, d))));
+            }
+            filtered = filtered.slice(0, params.maxResults);
+            return {
+                provider: "brave",
+                query: params.query,
+                results: filtered,
+            };
+        },
+    };
+}
+//# sourceMappingURL=brave.js.map