@demigodmode/pi-web-agent 1.8.0 → 1.10.0

package/CHANGELOG.md

@@ -18,9 +18,37 @@ The format is intentionally simple and release-oriented.
 ### Breaking
 - None.
 
+## [1.10.0] - 2026-08-25
+### Added
+- The zero-config default has a safety net now. When DuckDuckGo blocks a search (which happens fast on VPS and datacenter IPs), web search falls back to Tavily's keyless endpoint, no account or API key needed, so a fresh install still returns results instead of an error. Set `PI_WEB_AGENT_DISABLE_KEYLESS_FALLBACK=1` if you'd rather it just fail. (#42)
+
+### Changed
+- The keyless DuckDuckGo default is a lot harder to bot-wall. It sends real browser headers and retries once when a page looks blocked, instead of going out as an obvious script and getting bounced on the first request. (#42)
+
+### Fixed
+- Blocked searches are labeled honestly. A bot-wall page comes back as "blocked" (which triggers the fallback) instead of "couldn't parse", including the 200-status pages that mix a no-results message with a bot-wall notice. (#42)
+
+### Breaking
+- None.
+
+## [1.9.0] - 2026-08-15
+### Added
+- Search fanout. Ask a hard question and web_explore can now hit several of your configured search backends at once, dedupe the merged results, and rank the pages that more than one provider agreed on to the top. Off by default. Flip it to on or auto in Settings → Backends (auto only fans out when the first provider comes back thin, so easy queries stay cheap). Preview/verbose shows which providers ran and which got skipped. (#35)
+
+### Changed
+- None.
+
+### Fixed
+- web_explore can finally summarize a link you hand it. Paste a GitHub file/issue/PR, a PDF, or a YouTube URL and ask for a summary: it now reads the actual content (raw source, extracted PDF text, the video transcript) and gives the model the whole thing. 1.8.0 added these readers but then squeezed everything down to a ~180-char snippet before the model ever saw it, so summaries were basically guesswork. Now they're grounded in the real text (long content still capped around 24k characters). (#40, #41)
+- Fanout won't stall on a dead provider anymore. Each one gets an 8s timeout, so an unreachable self-hosted SearXNG can't hang a whole research pass. (#35)
+- Fanout only offers and queries the providers you've actually set up (keyless DuckDuckGo, SearXNG when you give it a URL, the hosted ones only when their key is present) instead of acting like all six are on. (#35)
+
+### Breaking
+- None.
+
 ## [1.8.0] - 2026-08-14
 ### Added
-- Read the content behind GitHub, PDF, and YouTube links directly instead of the page shell. GitHub files, issues and PRs come from the API/raw endpoints (optional `GITHUB_TOKEN` raises the rate limit), PDFs are parsed with unpdf, and YouTube links return the transcript. All keyless. Scanned PDFs and caption-less videos are caveated rather than failing. (#39, #40, #41)
+- Read the content behind GitHub, PDF, and YouTube links directly instead of the page shell. GitHub files, issues and PRs come from the API/raw endpoints (optional `GITHUB_TOKEN` raises the rate limit), PDFs are parsed with unpdf, and YouTube links return the transcript (long content capped around 24k characters). All keyless. Scanned PDFs and caption-less videos are caveated rather than failing. (#39, #40, #41)
 
 ### Changed
 - None.

package/README.md

@@ -1,131 +1,121 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/demigodmode/pi-web-agent/main/docs/public/pi-web-agent-banner.png" alt="pi-web-agent: bounded web research for Pi" width="100%">
+</p>
+
 # pi-web-agent
 
 [![CI](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml)
 [![npm version](https://img.shields.io/npm/v/@demigodmode/pi-web-agent)](https://www.npmjs.com/package/@demigodmode/pi-web-agent)
 [![Docs](https://img.shields.io/badge/docs-github%20pages-blue)](https://demigodmode.github.io/pi-web-agent/)
 
-`@demigodmode/pi-web-agent` is a Pi package for web access.
+One public tool, `web_explore`, that does bounded web research for Pi: search, fetch, targeted browser rendering, ranking, and honest caveats, all behind a single call.
+
+> Most agent web tooling blurs search, fetch, rendering, and synthesis into one vague thing. `pi-web-agent` keeps that boundary simple, and it is stricter about what it actually did: bot-check pages, narrow source sets, unreadable threads, and conflicting evidence show up as caveats instead of fake confidence.
+
+## What you get
+
+- **One tool.** `web_explore` handles direct links, discovery, HTTP reads, targeted headless rendering, source ranking, source-quality checks, and caveats internally.
+- **Reads the real content behind links.** Paste a GitHub, PDF, or YouTube URL and it pulls the actual thing (GitHub files/issues/PRs from the API, PDF text, YouTube transcripts), keyless. So "summarize this PDF" or "what does this repo do" works off the source, not the page shell.
+- **Six search backends.** DuckDuckGo (keyless default), SearXNG, Brave, You.com, Exa, Tavily.
+- **Optional search fanout.** Query several backends at once, dedupe, and rank pages that more than one provider agreed on to the top. Off by default; flip it to `on` or `auto`.
+- **Honest by default.** Weak, narrow, blocked, or cautionary evidence gets flagged instead of dressed up as confidence.
+- **Bounded output.** `compact` / `preview` / `verbose` transcript modes.
+- **Zero-config to start.** Runs keyless out of the box (DuckDuckGo search, local browser, the built-in readers). Opt into hosted backends, fallback, search fanout, and per-tool output modes through config when you want more control.
 
-Most agent web tools blur search, fetch, browser rendering, and research into one vague thing. `pi-web-agent` exposes one public research tool, `web_explore`, and keeps search/fetch/headless work inside that bounded workflow.
+## Why pi-web-agent
 
-The point is keeping the model-facing boundary simple: ask `web_explore` to research a question, and it handles direct links, discovery, HTTP reads, targeted browser rendering, source ranking, source-quality checks, and caveats internally.
+Compared to other web tooling for agents:
 
-That sounds obvious, but a lot of agent tooling gets fuzzy right there. This package is meant to be stricter about what it actually did and more willing to say when a read was not good enough to trust. Bot-check pages, narrow source sets, unreadable threads, and cautionary/conflicting evidence should show up as caveats instead of fake confidence.
+- **Hands-off.** No curator, no browser windows to approve, no step that pops you out of your session. Ask `web_explore` once and the answer comes back with caveats. Nothing to babysit.
+- **Keyless by default.** Search, page reads, and the GitHub/PDF/YouTube readers all work with no API keys. Add hosted providers only when you want them.
+- **Bounded and honest.** Compact output by default, and it says when a read was not good enough instead of returning fake confidence.
 
 ## Install
 
-Compatibility notice: current `pi-web-agent` requires Pi 0.74+ because Pi packages moved to the `@earendil-works/*` scope. Update Pi before updating this package. If you are on an older Pi version, stay on `@demigodmode/pi-web-agent@0.6.x` until Pi is updated.
+> `pi-web-agent` requires Pi 0.74+ (Pi packages moved to the `@earendil-works/*` scope). Update Pi before updating this package. On older Pi, stay on `@demigodmode/pi-web-agent@0.6.x`.
 
 ```bash
 pi install npm:@demigodmode/pi-web-agent
 ```
 
-After installing, reload or restart Pi. Run `/web-agent` for the action menu, or `/web-agent doctor` to check whether the package loaded cleanly and which web backends are configured.
+Reload or restart Pi after installing, then:
 
-Headless rendering first tries a detectable Chromium-family browser: Chrome, Chromium, Edge, or Brave. If none is found, it falls back to Playwright-managed Chromium and still launches headless. Firefox/Safari-only systems can still use search and plain HTTP reads; browser-rendered fallback uses Chromium.
-
-Later on, update installed packages with:
-
-```bash
-pi update --extensions
+```text
+/web-agent doctor   # check it loaded and show configured backends
+/web-agent          # action menu
 ```
 
-## Docs
+Update later with `pi update --extensions`.
 
-Docs site:
+**Browser rendering:** headless first tries a detected Chromium-family browser (Chrome, Chromium, Edge, Brave). If none is found, it falls back to Playwright-managed Chromium. Firefox/Safari-only systems still get search and plain HTTP reads.
 
-- https://demigodmode.github.io/pi-web-agent/
+## Usage
 
-Work on the docs locally:
+Ask `web_explore` a web question:
 
-```bash
-npm run docs:dev
-```
+> Find current docs and discussions on configuring Vitest coverage with the v8 provider.
 
-Build the docs:
+Or hand it a link to read:
 
-```bash
-npm run docs:build
-```
+> Summarize this PDF: https://arxiv.org/pdf/1706.03762
 
-## Presentation modes
+If a pass comes back thin, call `web_explore` again with a narrower query.
 
-`pi-web-agent` renders web tool output in one visible mode at a time:
+## Backends
 
-- `compact` — short summary, default everywhere
-- `preview` — slightly richer bounded view
-- `verbose` — fuller bounded view
+Defaults are DuckDuckGo search, plain HTTP fetch, and local-browser headless. Switch providers from `/web-agent settings → Backends`. API keys stay in environment variables, never in config files.
 
-See the `v0.3.0` release notes for a before/after of the transcript cleanup:
+| Backend | Role | Enable with |
+| --- | --- | --- |
+| DuckDuckGo | search (default) | nothing, keyless |
+| SearXNG | search (self-hosted) | base URL |
+| Brave | search (hosted) | `PI_WEB_AGENT_BRAVE_API_KEY` |
+| You.com | search (hosted) | `YDC_API_KEY` |
+| Exa | search (hosted) | `EXA_API_KEY` |
+| Tavily | search (hosted) | `TAVILY_API_KEY` |
+| Firecrawl | fetch (self-hosted) | base URL + `PI_WEB_AGENT_FIRECRAWL_API_KEY` |
+| GitHub reader | content | `GITHUB_TOKEN` (optional, raises the rate limit) |
 
-- https://github.com/demigodmode/pi-web-agent/releases/tag/v0.3.0
+Full config shape (fallback, SearXNG/Firecrawl options, fanout): see the [self-hosted backends docs](https://demigodmode.github.io/pi-web-agent/self-hosted-backends).
 
 ## Settings
 
-Primary UI:
-
 ```text
-/web-agent settings
-```
-
-Helper commands:
-
-```text
-/web-agent doctor
-/web-agent show
+/web-agent settings                    # main UI
+/web-agent doctor                      # health check
+/web-agent show                        # effective config
 /web-agent changelog
-/web-agent reset project
-/web-agent reset global
-/web-agent mode preview
-/web-agent mode web_explore verbose
-/web-agent mode web_explore inherit
+/web-agent mode web_explore verbose    # per-tool presentation mode
+/web-agent reset project | global
 ```
 
-Config files:
+Config is JSON, and project config overrides global:
 
 ```text
 Global:  ~/.pi/agent/extensions/pi-web-agent/config.json
 Project: .pi/extensions/pi-web-agent/config.json
 ```
 
-Precedence:
-
-- built-in defaults
-- global config
-- project config
-
-Project config overrides global config.
-
-Example:
-
 ```json
 {
   "presentation": {
     "defaultMode": "compact",
-    "tools": {
-      "web_explore": { "mode": "verbose" }
-    }
+    "tools": { "web_explore": { "mode": "verbose" } }
   }
 }
 ```
 
-Backend config is also supported. Defaults remain DuckDuckGo search, plain HTTP fetch, and local-browser headless fallback with managed Chromium fallback configured. If you have a Brave Search or You.com API key, either can be selected as a hosted search backend while `web_explore` still handles page reading, ranking, and caveats itself.
+Presentation modes:
 
-Search backends: DuckDuckGo (default), SearXNG (self-hosted), Brave Search (hosted), You.com (hosted).
+- `compact`: short summary, the default everywhere
+- `preview`: slightly richer bounded view
+- `verbose`: fuller bounded view
 
-Backend settings can be changed from:
-
-```text
-/web-agent settings
-```
-
-Choose **Backends** to edit search/fetch providers, fallback behavior, and SearXNG or Firecrawl base URLs interactively. Brave Search uses `PI_WEB_AGENT_BRAVE_API_KEY`, You.com uses `YDC_API_KEY`, Exa uses `EXA_API_KEY`, and Tavily uses `TAVILY_API_KEY`. Firecrawl API keys should also stay in environment variables rather than being written into config files.
-
-For the full backend config shape, including SearXNG, Brave, You.com, Firecrawl, and fallback behavior, see:
+## Docs
 
-- https://demigodmode.github.io/pi-web-agent/self-hosted-backends
+Full docs: <https://demigodmode.github.io/pi-web-agent/>. Work on them locally with `npm run docs:dev`.
 
-## Local development
+## Development
 
 ```bash
 npm install
@@ -134,10 +124,8 @@ npm run lint
 npm run build
 ```
 
-For local Pi work, this repo includes `.pi/extensions/pi-web-agent.ts`.
-
-If Pi is already running, use `/reload` after changes.
+Local Pi work uses `.pi/extensions/pi-web-agent.ts`; run `/reload` after changes.
 
 ## License
 
-AGPL-3.0-only. See `LICENSE`.
+AGPL-3.0-only. See [LICENSE](LICENSE).

package/dist/backends/config.d.ts

@@ -1,8 +1,13 @@
+import type { FanoutMode, SearchProviderName } from '../types.js';
 export type SearxngOptions = {
     categories?: string[];
     language?: string;
     safesearch?: 0 | 1 | 2;
 };
+export type FanoutConfig = {
+    mode: FanoutMode;
+    providers?: SearchProviderName[];
+};
 export type FirecrawlOptions = {
     formats?: string[];
     onlyMainContent?: boolean;
@@ -12,6 +17,7 @@ export type SearchBackendConfig = {
     baseUrl?: string;
     fallback?: 'duckduckgo';
     options?: SearxngOptions;
+    fanout?: FanoutConfig;
 };
 export type FetchBackendConfig = {
     provider: 'http' | 'firecrawl';
@@ -40,6 +46,7 @@ export type BackendConfigFile = {
             baseUrl?: unknown;
             fallback?: unknown;
             options?: unknown;
+            fanout?: unknown;
         };
         fetch?: {
             provider?: unknown;
@@ -54,6 +61,7 @@ export type BackendConfigFile = {
     };
 };
 export declare const DEFAULT_BACKEND_CONFIG: BackendConfig;
+export declare function usableSearchProviders(search: SearchBackendConfig, env?: NodeJS.ProcessEnv): SearchProviderName[];
 export declare function extractBackendConfigOverride(file: BackendConfigFile | null | undefined): BackendConfigOverride;
 export declare function validateBackendConfig(config: BackendConfig): string[];
 export declare function mergeBackendConfigLayers(...layers: Array<BackendConfig | BackendConfigOverride | undefined>): BackendConfig;

package/dist/backends/config.js

@@ -35,6 +35,38 @@ function extractFirecrawlOptions(value) {
         options.onlyMainContent = raw.onlyMainContent;
     return Object.keys(options).length > 0 ? options : undefined;
 }
+const PROVIDER_NAMES = ['duckduckgo', 'searxng', 'brave', 'youcom', 'exa', 'tavily'];
+export function usableSearchProviders(search, env = process.env) {
+    // Match the provider implementations, which treat a blank/whitespace key as unconfigured.
+    const usable = ['duckduckgo']; // keyless, always usable
+    if (search.baseUrl?.trim())
+        usable.push('searxng');
+    if (env.PI_WEB_AGENT_BRAVE_API_KEY?.trim())
+        usable.push('brave');
+    if (env.YDC_API_KEY?.trim())
+        usable.push('youcom');
+    if (env.EXA_API_KEY?.trim())
+        usable.push('exa');
+    if (env.TAVILY_API_KEY?.trim())
+        usable.push('tavily');
+    return usable;
+}
+function extractFanoutConfig(value) {
+    if (!value || typeof value !== 'object')
+        return undefined;
+    const raw = value;
+    if (raw.mode !== 'off' && raw.mode !== 'on' && raw.mode !== 'auto')
+        return undefined;
+    const config = { mode: raw.mode };
+    if (Array.isArray(raw.providers)) {
+        const providers = raw.providers.filter((p) => typeof p === 'string' && PROVIDER_NAMES.includes(p));
+        if (providers.length !== raw.providers.length)
+            return undefined; // fail loud on any invalid entry
+        if (providers.length > 0)
+            config.providers = providers;
+    }
+    return config;
+}
 export function extractBackendConfigOverride(file) {
     const backends = file?.backends;
     const override = {};
@@ -58,6 +90,10 @@ export function extractBackendConfigOverride(file) {
             }
         }
     }
+    const fanout = extractFanoutConfig(backends?.search?.fanout);
+    if (fanout) {
+        override.search = { ...(override.search ?? {}), fanout };
+    }
     if (backends?.fetch?.provider === 'http' || backends?.fetch?.provider === 'firecrawl') {
         override.fetch = { provider: backends.fetch.provider };
         if (typeof backends.fetch.baseUrl === 'string') {
@@ -106,6 +142,15 @@ export function validateBackendConfig(config) {
     if (config.fetch.options?.formats && config.fetch.options.formats.length === 0) {
         issues.push('fetch options.formats must contain at least one format when provided');
     }
+    const fanout = config.search.fanout;
+    if (fanout) {
+        if (fanout.mode !== 'off' && fanout.mode !== 'on' && fanout.mode !== 'auto') {
+            issues.push('search fanout.mode must be off, on, or auto');
+        }
+        if (fanout.providers?.includes('searxng') && !config.search.baseUrl) {
+            issues.push('search fanout with searxng requires backends.search.baseUrl');
+        }
+    }
     return issues;
 }
 function mergeSearchConfig(current, override) {

package/dist/backends/doctor.js

@@ -208,6 +208,39 @@ export async function checkBackendHealth(config, { fetchImpl = fetch, timeoutMs
     if (config.search.fallback) {
         lines.push(`search fallback: ${config.search.fallback}`);
     }
+    if (config.search.fanout && config.search.fanout.mode !== 'off') {
+        const providers = config.search.fanout.providers || ['duckduckgo', 'searxng', 'brave', 'youcom', 'exa', 'tavily'];
+        lines.push(`search fanout: ${config.search.fanout.mode} (${providers.join(', ')})`);
+        for (const provider of providers) {
+            if (provider === 'searxng' && !config.search.baseUrl) {
+                lines.push('search fanout provider searxng warning (missing baseUrl)');
+            }
+            else if (provider === 'brave') {
+                const apiKey = process.env.PI_WEB_AGENT_BRAVE_API_KEY;
+                if (!apiKey?.trim()) {
+                    lines.push('search fanout provider brave warning (missing PI_WEB_AGENT_BRAVE_API_KEY)');
+                }
+            }
+            else if (provider === 'youcom') {
+                const apiKey = process.env.YDC_API_KEY;
+                if (!apiKey?.trim()) {
+                    lines.push('search fanout provider youcom warning (missing YDC_API_KEY)');
+                }
+            }
+            else if (provider === 'exa') {
+                const apiKey = process.env.EXA_API_KEY;
+                if (!apiKey?.trim()) {
+                    lines.push('search fanout provider exa warning (missing EXA_API_KEY)');
+                }
+            }
+            else if (provider === 'tavily') {
+                const apiKey = process.env.TAVILY_API_KEY;
+                if (!apiKey?.trim()) {
+                    lines.push('search fanout provider tavily warning (missing TAVILY_API_KEY)');
+                }
+            }
+        }
+    }
     if (config.fetch.provider === 'http') {
         lines.push('fetch backend: http');
     }

package/dist/backends/factory.js

@@ -4,12 +4,13 @@ import { createYouComSearchTool } from '../search/youcom.js';
 import { createExaSearchTool } from '../search/exa.js';
 import { createTavilySearchTool } from '../search/tavily.js';
 import { createSearxngSearchTool } from '../search/searxng.js';
+import { createFanoutSearch } from '../search/fanout.js';
 import { buildFetchPresentation } from '../presentation/fetch-presentation.js';
 import { buildSearchPresentation } from '../presentation/search-presentation.js';
 import { createWebFetchHeadlessTool } from '../tools/web-fetch-headless.js';
 import { createWebFetchTool } from '../tools/web-fetch.js';
 import { createWebSearchTool } from '../tools/web-search.js';
-import { DEFAULT_BACKEND_CONFIG } from './config.js';
+import { DEFAULT_BACKEND_CONFIG, usableSearchProviders } from './config.js';
 import { createSpecialContentResolver } from '../readers/resolver.js';
 import { createGithubReader } from '../readers/github-reader.js';
 import { createPdfReader } from '../readers/pdf-reader.js';
@@ -53,7 +54,10 @@ function withSearchFallback(primary, fallback, fallbackFrom) {
             metadata: {
                 ...second.metadata,
                 fallbackFrom,
-                fallbackReason: first.error?.message ?? `${fallbackFrom} search failed.`
+                fallbackReason: first.error?.message ?? `${fallbackFrom} search failed.`,
+                // Keep the primary's fanout provenance (which providers were tried/skipped) even though
+                // the answer came from the fallback backend.
+                ...(first.metadata.fanout ? { fanout: first.metadata.fanout } : {})
             }
         };
         return { ...result, presentation: buildSearchPresentation(result) };
@@ -86,6 +90,25 @@ export function createBackendSet(config = DEFAULT_BACKEND_CONFIG, deps = {}) {
     const createHttpFetch = deps.createHttpFetch ?? createWebFetchTool;
     const createFirecrawlFetch = deps.createFirecrawlFetch ?? createFirecrawlFetcher;
     const createHeadlessFetch = deps.createHeadlessFetch ?? createWebFetchHeadlessTool;
+    function buildProviderSearch(name) {
+        switch (name) {
+            case 'searxng':
+                return config.search.baseUrl
+                    ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options })
+                    : invalidSearxngSearch();
+            case 'brave':
+                return createBraveSearch({ apiKey: process.env.PI_WEB_AGENT_BRAVE_API_KEY });
+            case 'youcom':
+                return createYouComSearch({ apiKey: process.env.YDC_API_KEY });
+            case 'exa':
+                return createExaSearch({ apiKey: process.env.EXA_API_KEY });
+            case 'tavily':
+                return createTavilySearch({ apiKey: process.env.TAVILY_API_KEY });
+            case 'duckduckgo':
+            default:
+                return createDuckDuckGoSearch();
+        }
+    }
     let search = config.search.provider === 'searxng'
         ? config.search.baseUrl
             ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options })
@@ -114,6 +137,29 @@ export function createBackendSet(config = DEFAULT_BACKEND_CONFIG, deps = {}) {
     if (config.search.provider === 'tavily' && config.search.fallback === 'duckduckgo') {
         search = withSearchFallback(search, createDuckDuckGoSearch(), 'tavily');
     }
+    const fanoutConfig = config.search.fanout;
+    if (fanoutConfig && fanoutConfig.mode !== 'off') {
+        const baseNames = fanoutConfig.providers && fanoutConfig.providers.length > 0
+            ? fanoutConfig.providers
+            : usableSearchProviders(config.search);
+        // A configured DuckDuckGo fallback must still be honored under fanout: fold it into the set.
+        const providerNames = config.search.fallback === 'duckduckgo' && !baseNames.includes('duckduckgo')
+            ? [...baseNames, 'duckduckgo']
+            : baseNames;
+        const ordered = [config.search.provider, ...providerNames.filter((n) => n !== config.search.provider)].filter((n, i, arr) => arr.indexOf(n) === i);
+        search = createFanoutSearch({
+            providers: ordered.map((name) => ({ name, search: buildProviderSearch(name) })),
+            mode: fanoutConfig.mode
+        });
+    }
+    // Keep the keyless Tavily safety net for the no-key DuckDuckGo default, even under fanout —
+    // it wraps whatever search ended up being (plain DDG or the fanout set) so a total failure
+    // still has somewhere to go. Opt out with PI_WEB_AGENT_DISABLE_KEYLESS_FALLBACK=1.
+    const keylessFallbackDisabled = process.env.PI_WEB_AGENT_DISABLE_KEYLESS_FALLBACK === '1';
+    const usingDuckDuckGoDefault = config.search.provider === 'duckduckgo' || !config.search.provider;
+    if (usingDuckDuckGoDefault && !keylessFallbackDisabled) {
+        search = withSearchFallback(search, createTavilySearch({ keyless: true }), 'duckduckgo');
+    }
     const httpFetch = createHttpFetch();
     let fetchPage = config.fetch.provider === 'firecrawl'
         ? config.fetch.baseUrl

package/dist/commands/web-agent-config.js

@@ -1,4 +1,4 @@
-import { DEFAULT_BACKEND_CONFIG, mergeBackendConfigLayers, validateBackendConfig } from '../backends/config.js';
+import { DEFAULT_BACKEND_CONFIG, mergeBackendConfigLayers, validateBackendConfig, usableSearchProviders } from '../backends/config.js';
 import { checkBackendHealth } from '../backends/doctor.js';
 import { DynamicBorder, getSettingsListTheme } from '@earendil-works/pi-coding-agent';
 import { Container, Input, SelectList, SettingsList, Text } from '@earendil-works/pi-tui';
@@ -20,7 +20,10 @@ function cloneBackendConfig(config) {
     return {
         search: {
             ...config.search,
-            options: config.search.options ? { ...config.search.options } : undefined
+            options: config.search.options ? { ...config.search.options } : undefined,
+            fanout: config.search.fanout
+                ? { mode: config.search.fanout.mode, providers: config.search.fanout.providers ? [...config.search.fanout.providers] : undefined }
+                : undefined
         },
         fetch: {
             ...config.fetch,
@@ -54,12 +57,16 @@ async function defaultCheckTypebox() {
     }
 }
 function formatSearchOptions(config) {
-    return [
+    const parts = [
         config.fallback ? `fallback ${config.fallback}` : undefined,
         config.options?.categories?.length ? `categories ${config.options.categories.join(',')}` : undefined,
         config.options?.language ? `language ${config.options.language}` : undefined,
-        config.options?.safesearch !== undefined ? `safesearch ${config.options.safesearch}` : undefined
-    ].filter(Boolean).join(' ');
+        config.options?.safesearch !== undefined ? `safesearch ${config.options.safesearch}` : undefined,
+        config.fanout && config.fanout.mode !== 'off'
+            ? `fanout ${config.fanout.mode} (${(config.fanout.providers || usableSearchProviders(config)).join(', ')})`
+            : undefined
+    ].filter(Boolean);
+    return parts.join(' ');
 }
 function formatFetchOptions(config) {
     return [
@@ -155,6 +162,9 @@ export function createBackendUrlEditor(theme, label, placeholderUrl, onOpenChang
     };
 }
 function buildBackendSettingsItems(scope, backends, theme, onUrlEditorOpenChange) {
+    const usable = usableSearchProviders(backends.search);
+    const fanoutMode = backends.search.fanout?.mode ?? 'off';
+    const fanoutProviders = backends.search.fanout?.providers ?? usable;
     return [
         {
             id: 'scope',
@@ -180,6 +190,18 @@ function buildBackendSettingsItems(scope, backends, theme, onUrlEditorOpenChange
             currentValue: backends.search.provider === 'searxng' || backends.search.provider === 'brave' || backends.search.provider === 'youcom' || backends.search.provider === 'exa' || backends.search.provider === 'tavily' ? backends.search.fallback ?? 'off' : 'off',
             values: backends.search.provider === 'searxng' || backends.search.provider === 'brave' || backends.search.provider === 'youcom' || backends.search.provider === 'exa' || backends.search.provider === 'tavily' ? ['off', 'duckduckgo'] : ['off']
         },
+        {
+            id: 'backend:search:fanout:mode',
+            label: 'Search fanout',
+            currentValue: fanoutMode,
+            values: ['off', 'on', 'auto']
+        },
+        ...(fanoutMode !== 'off' ? usable.map((provider) => ({
+            id: `backend:search:fanout:provider:${provider}`,
+            label: `  ${provider}`,
+            currentValue: fanoutProviders.includes(provider) ? 'included' : 'excluded',
+            values: ['included', 'excluded']
+        })) : []),
         {
             id: 'backend:secret:brave',
             label: 'Brave API key',
@@ -333,6 +355,40 @@ export function applySettingsValue(state, id, newValue) {
             delete currentBackends.search.fallback;
         }
     }
+    if (id === 'backend:search:fanout:mode') {
+        if (newValue === 'off') {
+            currentBackends.search.fanout = { mode: 'off' };
+        }
+        else if (newValue === 'on' || newValue === 'auto') {
+            currentBackends.search.fanout = {
+                mode: newValue,
+                providers: undefined
+            };
+        }
+    }
+    if (id.startsWith('backend:search:fanout:provider:')) {
+        const providerName = id.slice('backend:search:fanout:provider:'.length);
+        const usableProviders = usableSearchProviders(currentBackends.search);
+        // Ensure fanout exists
+        if (!currentBackends.search.fanout) {
+            currentBackends.search.fanout = { mode: 'on', providers: undefined };
+        }
+        // Materialize the provider list if it's currently undefined; use usable providers instead of all
+        const currentProviders = currentBackends.search.fanout.providers ?? usableProviders;
+        if (newValue === 'excluded') {
+            // Remove the provider from the list
+            const filtered = currentProviders.filter((p) => p !== providerName);
+            // never allow an empty set: keep at least one provider (empty would be read as "all")
+            if (filtered.length > 0) {
+                currentBackends.search.fanout.providers = filtered;
+            }
+        }
+        else if (newValue === 'included') {
+            // Add the provider back, maintaining canonical order using usable providers
+            const result = usableProviders.filter((p) => currentProviders.includes(p) || p === providerName);
+            currentBackends.search.fanout.providers = result;
+        }
+    }
     if (id === 'backend:search:baseUrl') {
         if (newValue.trim()) {
             currentBackends.search.provider = 'searxng';
@@ -400,7 +456,8 @@ export function collapseBackendConfigToOverride(config, inheritedConfig) {
             : {
                 ...(config.search.baseUrl !== inheritedConfig.search.baseUrl ? { baseUrl: config.search.baseUrl } : {}),
                 ...(config.search.fallback !== inheritedConfig.search.fallback ? { fallback: config.search.fallback } : {}),
-                ...(!sameJson(config.search.options, inheritedConfig.search.options) ? { options: config.search.options } : {})
+                ...(!sameJson(config.search.options, inheritedConfig.search.options) ? { options: config.search.options } : {}),
+                ...(!sameJson(config.search.fanout, inheritedConfig.search.fanout) ? { fanout: config.search.fanout } : {})
             };
         if (config.search.provider !== inheritedConfig.search.provider) {
             override.search.provider = config.search.provider;