@demigodmode/pi-web-agent 1.10.0 → 1.11.0

package/CHANGELOG.md

@@ -18,6 +18,19 @@ The format is intentionally simple and release-oriented.
 ### Breaking
 - None.
 
+## [1.11.0] - 2026-09-16
+### Added
+- Optional proxy support. Point web_explore at an HTTP/HTTPS proxy and everything outbound goes through it: search, fetch, Firecrawl, the GitHub/PDF/YouTube readers, doctor health checks, and the headless browser. Off unless you set it, and nothing changes if you don't. Set it in Settings → Backends, and keep credentials in `PI_WEB_AGENT_PROXY_USERNAME` / `PI_WEB_AGENT_PROXY_PASSWORD` rather than in the URL. Thanks to @lo-tp for building this. (#50)
+
+### Changed
+- `/web-agent doctor` reports whether the jsdom compat patch is in place, and prints the command to reapply it if it isn't.
+
+### Fixed
+- Pi failing to start with `Cannot find module 'punycode/'` or `Set operation called on non-Set object`. The patch for this was only applied at install time, but every pi extension shares one node_modules tree, so installing or updating anything else quietly reverted it. It now reapplies every time the extension loads. Also fixes it picking the wrong copy of the dependency when the shared tree holds more than one. (#34)
+
+### Breaking
+- None.
+
 ## [1.10.0] - 2026-08-25
 ### Added
 - The zero-config default has a safety net now. When DuckDuckGo blocks a search (which happens fast on VPS and datacenter IPs), web search falls back to Tavily's keyless endpoint, no account or API key needed, so a fresh install still returns results instead of an error. Set `PI_WEB_AGENT_DISABLE_KEYLESS_FALLBACK=1` if you'd rather it just fail. (#42)

package/README.md

@@ -4,9 +4,24 @@
 
 # pi-web-agent
 
-[![CI](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml)
-[![npm version](https://img.shields.io/npm/v/@demigodmode/pi-web-agent)](https://www.npmjs.com/package/@demigodmode/pi-web-agent)
-[![Docs](https://img.shields.io/badge/docs-github%20pages-blue)](https://demigodmode.github.io/pi-web-agent/)
+<p align="center">
+  <a href="https://github.com/demigodmode/pi-web-agent/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/demigodmode/pi-web-agent/ci.yml?branch=main&style=flat-square&logo=github&label=CI" alt="CI"></a>
+  <a href="https://www.npmjs.com/package/@demigodmode/pi-web-agent"><img src="https://img.shields.io/npm/v/@demigodmode/pi-web-agent?style=flat-square&logo=npm&logoColor=white&color=CB3837" alt="npm version"></a>
+  <a href="https://www.npmjs.com/package/@demigodmode/pi-web-agent"><img src="https://img.shields.io/npm/dm/@demigodmode/pi-web-agent?style=flat-square&color=0A7BBB&label=downloads" alt="npm downloads"></a>
+  <img src="https://img.shields.io/badge/platform-macOS%20%7C%20Linux%20%7C%20Windows-475569?style=flat-square" alt="Platform: macOS, Linux, Windows">
+  <a href="https://demigodmode.github.io/pi-web-agent/"><img src="https://img.shields.io/badge/docs-github%20pages-2088FF?style=flat-square&logo=readthedocs&logoColor=white" alt="Docs"></a>
+  <a href="https://github.com/demigodmode/pi-web-agent/blob/main/LICENSE"><img src="https://img.shields.io/npm/l/@demigodmode/pi-web-agent?style=flat-square&color=6F42C1" alt="License"></a>
+</p>
+
+<p align="center">
+  <img src="https://img.shields.io/badge/search%20backends-475569?style=flat-square" alt="Search backends">
+  <img src="https://img.shields.io/badge/DuckDuckGo-475569?style=flat-square&logo=duckduckgo&logoColor=white" alt="DuckDuckGo">
+  <img src="https://img.shields.io/badge/SearXNG-475569?style=flat-square&logo=searxng&logoColor=white" alt="SearXNG">
+  <img src="https://img.shields.io/badge/Brave-475569?style=flat-square&logo=brave&logoColor=white" alt="Brave Search">
+  <img src="https://img.shields.io/badge/You.com-475569?style=flat-square" alt="You.com">
+  <img src="https://img.shields.io/badge/Exa-475569?style=flat-square" alt="Exa">
+  <img src="https://img.shields.io/badge/Tavily-475569?style=flat-square" alt="Tavily">
+</p>
 
 One public tool, `web_explore`, that does bounded web research for Pi: search, fetch, targeted browser rendering, ranking, and honest caveats, all behind a single call.
 
@@ -26,7 +41,7 @@ One public tool, `web_explore`, that does bounded web research for Pi: search, f
 
 Compared to other web tooling for agents:
 
-- **Hands-off.** No curator, no browser windows to approve, no step that pops you out of your session. Ask `web_explore` once and the answer comes back with caveats. Nothing to babysit.
+- **Hands-off.** No curator, no browser windows to approve, no step that pops you out of your session. Ask `web_explore` once and the answer comes back cleanly. Nothing to babysit.
 - **Keyless by default.** Search, page reads, and the GitHub/PDF/YouTube readers all work with no API keys. Add hosted providers only when you want them.
 - **Bounded and honest.** Compact output by default, and it says when a read was not good enough instead of returning fake confidence.
 

package/dist/backends/config.d.ts

@@ -12,6 +12,11 @@ export type FirecrawlOptions = {
     formats?: string[];
     onlyMainContent?: boolean;
 };
+export type ProxyConfig = {
+    url: string;
+    username?: string;
+    password?: string;
+};
 export type SearchBackendConfig = {
     provider: 'duckduckgo' | 'searxng' | 'brave' | 'youcom' | 'exa' | 'tavily';
     baseUrl?: string;
@@ -33,11 +38,13 @@ export type BackendConfig = {
     search: SearchBackendConfig;
     fetch: FetchBackendConfig;
     headless: HeadlessBackendConfig;
+    proxy?: ProxyConfig;
 };
 export type BackendConfigOverride = {
     search?: Partial<SearchBackendConfig>;
     fetch?: Partial<FetchBackendConfig>;
     headless?: Partial<HeadlessBackendConfig>;
+    proxy?: ProxyConfig;
 };
 export type BackendConfigFile = {
     backends?: {
@@ -58,9 +65,29 @@ export type BackendConfigFile = {
         headless?: {
             provider?: unknown;
         };
+        proxy?: {
+            url?: unknown;
+            username?: unknown;
+            password?: unknown;
+        };
     };
 };
 export declare const DEFAULT_BACKEND_CONFIG: BackendConfig;
+/**
+ * Remove any credentials (user:pass@) embedded in a proxy URL. pi-web-agent
+ * never reads or sends credentials from the URL; proxy auth comes from
+ * backends.proxy.username/password or PI_WEB_AGENT_PROXY_USERNAME /
+ * PI_WEB_AGENT_PROXY_PASSWORD. Stripping them here keeps them out of logs,
+ * doctor output, the settings UI, and the proxy connection itself.
+ */
+export declare function stripProxyCredentials(url: string): string;
+/**
+ * Whether a proxy url passes validation: a blank url is the "disable proxy"
+ * marker and passes; anything else must parse as an http(s) URL. This is a
+ * pure syntax check — no connectivity check is performed.
+ */
+export declare function isValidProxyUrl(url: string): boolean;
+export declare function extractProxyConfig(value: unknown): ProxyConfig | undefined;
 export declare function usableSearchProviders(search: SearchBackendConfig, env?: NodeJS.ProcessEnv): SearchProviderName[];
 export declare function extractBackendConfigOverride(file: BackendConfigFile | null | undefined): BackendConfigOverride;
 export declare function validateBackendConfig(config: BackendConfig): string[];

package/dist/backends/config.js

@@ -9,6 +9,76 @@ function extractStringArray(value) {
     const strings = value.filter((item) => typeof item === 'string');
     return strings.length === value.length ? strings : undefined;
 }
+/**
+ * Remove any credentials (user:pass@) embedded in a proxy URL. pi-web-agent
+ * never reads or sends credentials from the URL; proxy auth comes from
+ * backends.proxy.username/password or PI_WEB_AGENT_PROXY_USERNAME /
+ * PI_WEB_AGENT_PROXY_PASSWORD. Stripping them here keeps them out of logs,
+ * doctor output, the settings UI, and the proxy connection itself.
+ */
+export function stripProxyCredentials(url) {
+    let parsed;
+    try {
+        parsed = new URL(url);
+    }
+    catch {
+        return url;
+    }
+    if (!parsed.username && !parsed.password)
+        return url; // already credential-free
+    parsed.username = '';
+    parsed.password = '';
+    return parsed.toString().replace(/\/$/, '');
+}
+/**
+ * Whether a proxy url passes validation: a blank url is the "disable proxy"
+ * marker and passes; anything else must parse as an http(s) URL. This is a
+ * pure syntax check — no connectivity check is performed.
+ */
+export function isValidProxyUrl(url) {
+    if (url.trim() === '')
+        return true;
+    try {
+        const parsed = new URL(url);
+        return parsed.protocol === 'http:' || parsed.protocol === 'https:';
+    }
+    catch {
+        return false;
+    }
+}
+export function extractProxyConfig(value) {
+    if (!value || typeof value !== 'object')
+        return undefined;
+    const raw = value;
+    // An explicitly present but blank url is the "disable proxy" marker: it lets a
+    // higher-priority layer (e.g. a project) clear a proxy set in a lower layer.
+    if (typeof raw.url === 'string' && raw.url.trim() === '') {
+        return { url: '' };
+    }
+    if (typeof raw.url !== 'string' || !raw.url.trim())
+        return undefined;
+    const url = raw.url.trim();
+    let parsed;
+    try {
+        parsed = new URL(url);
+    }
+    catch {
+        parsed = undefined;
+    }
+    // A valid url is normalized. A malformed url is kept as-is (rather than
+    // dropped) so validation can flag it and the backend factory can fail loudly
+    // instead of silently sending traffic direct to the websites.
+    const config = {
+        url: parsed && (parsed.protocol === 'http:' || parsed.protocol === 'https:')
+            ? parsed.toString().replace(/\/$/, '')
+            : url
+    };
+    if (typeof raw.username === 'string' && raw.username.trim())
+        config.username = raw.username;
+    if (typeof raw.password === 'string')
+        config.password = raw.password;
+    return config;
+}
 function extractSearxngOptions(value) {
     if (!value || typeof value !== 'object')
         return undefined;
@@ -113,6 +183,10 @@ export function extractBackendConfigOverride(file) {
     if (backends?.headless?.provider === 'local-browser') {
         override.headless = { provider: 'local-browser' };
     }
+    const proxy = extractProxyConfig(backends?.proxy);
+    if (proxy) {
+        override.proxy = proxy;
+    }
     return override;
 }
 export function validateBackendConfig(config) {
@@ -120,6 +194,23 @@ export function validateBackendConfig(config) {
     if (config.search.provider === 'searxng' && !config.search.baseUrl) {
         issues.push('search provider searxng requires backends.search.baseUrl');
     }
+    if (config.proxy && config.proxy.url.trim() !== '') {
+        let parsed;
+        try {
+            parsed = new URL(config.proxy.url);
+        }
+        catch {
+            parsed = undefined;
+        }
+        if (!isValidProxyUrl(config.proxy.url)) {
+            issues.push('backends.proxy.url must be an http or https URL');
+        }
+        else if (parsed && (parsed.username || parsed.password)) {
+            // Credentials belong in backends.proxy.username/password or the env vars
+            // below, never in the URL itself.
+            issues.push('backends.proxy.url must not include credentials (user:pass@); set PI_WEB_AGENT_PROXY_USERNAME and PI_WEB_AGENT_PROXY_PASSWORD (or backends.proxy.username / backends.proxy.password) instead');
+        }
+    }
     if (config.fetch.provider === 'firecrawl' && !config.fetch.baseUrl) {
         issues.push('fetch provider firecrawl requires backends.fetch.baseUrl');
     }
@@ -173,6 +264,11 @@ export function mergeBackendConfigLayers(...layers) {
     return layers.reduce((merged, layer) => ({
         search: mergeSearchConfig(merged.search, layer?.search),
         fetch: mergeFetchConfig(merged.fetch, layer?.fetch),
-        headless: { ...merged.headless, ...layer?.headless }
+        headless: { ...merged.headless, ...layer?.headless },
+        proxy: layer?.proxy
+            ? layer.proxy.url === ''
+                ? undefined // explicit disable overrides any proxy from lower layers
+                : { ...merged.proxy, ...layer.proxy }
+            : merged.proxy
     }), DEFAULT_BACKEND_CONFIG);
 }

package/dist/backends/factory.d.ts

@@ -8,7 +8,7 @@ import { createWebFetchHeadlessTool } from '../tools/web-fetch-headless.js';
 import { createWebFetchTool } from '../tools/web-fetch.js';
 import { createWebSearchTool } from '../tools/web-search.js';
 import type { WebFetchHeadlessResponse, WebFetchResponse, WebSearchResponse } from '../types.js';
-import { type BackendConfig } from './config.js';
+import { type BackendConfig, type ProxyConfig } from './config.js';
 export type BackendSet = {
     search: (input: {
         query: string;
@@ -30,5 +30,6 @@ export type BackendFactoryDeps = {
     createHttpFetch?: typeof createWebFetchTool;
     createFirecrawlFetch?: typeof createFirecrawlFetcher;
     createHeadlessFetch?: typeof createWebFetchHeadlessTool;
+    createProxyFetch?: (proxy: ProxyConfig) => typeof fetch;
 };
 export declare function createBackendSet(config?: BackendConfig, deps?: BackendFactoryDeps): BackendSet;

package/dist/backends/factory.js

@@ -1,6 +1,10 @@
 import { createFirecrawlFetcher } from '../fetch/firecrawl-fetch.js';
+import { createHttpFetcher } from '../fetch/http-fetch.js';
+import { createProxyFetch, resolveProxyCredentials } from '../fetch/proxy-fetch.js';
+import { headlessFetch } from '../fetch/headless-fetch.js';
 import { createBraveSearchTool } from '../search/brave.js';
 import { createYouComSearchTool } from '../search/youcom.js';
+import { fetchDuckDuckGoHtml } from '../search/duckduckgo.js';
 import { createExaSearchTool } from '../search/exa.js';
 import { createTavilySearchTool } from '../search/tavily.js';
 import { createSearxngSearchTool } from '../search/searxng.js';
@@ -10,7 +14,7 @@ import { buildSearchPresentation } from '../presentation/search-presentation.js'
 import { createWebFetchHeadlessTool } from '../tools/web-fetch-headless.js';
 import { createWebFetchTool } from '../tools/web-fetch.js';
 import { createWebSearchTool } from '../tools/web-search.js';
-import { DEFAULT_BACKEND_CONFIG, usableSearchProviders } from './config.js';
+import { DEFAULT_BACKEND_CONFIG, isValidProxyUrl, stripProxyCredentials, usableSearchProviders } from './config.js';
 import { createSpecialContentResolver } from '../readers/resolver.js';
 import { createGithubReader } from '../readers/github-reader.js';
 import { createPdfReader } from '../readers/pdf-reader.js';
@@ -90,52 +94,104 @@ export function createBackendSet(config = DEFAULT_BACKEND_CONFIG, deps = {}) {
     const createHttpFetch = deps.createHttpFetch ?? createWebFetchTool;
     const createFirecrawlFetch = deps.createFirecrawlFetch ?? createFirecrawlFetcher;
     const createHeadlessFetch = deps.createHeadlessFetch ?? createWebFetchHeadlessTool;
+    const makeProxyFetch = deps.createProxyFetch ?? createProxyFetch;
+    // A blank url is the "disable proxy" marker: treat it as no proxy at all.
+    const proxy = config.proxy && config.proxy.url.trim() !== '' ? config.proxy : undefined;
+    // A configured proxy whose url fails validation must not be silently ignored
+    // — that would send traffic direct to the websites. Every request errors out
+    // instead. No connectivity check is needed: the url itself is the problem.
+    if (proxy && !isValidProxyUrl(proxy.url)) {
+        const message = `backends.proxy.url (${proxy.url}) is not a valid http or https URL. ` +
+            'Web requests are blocked until it is fixed; set backends.proxy.url to "" to disable the proxy.';
+        return {
+            search: async () => {
+                const result = {
+                    status: 'error',
+                    results: [],
+                    metadata: { backend: config.search.provider, cacheHit: false },
+                    error: { code: 'BACKEND_CONFIG_INVALID', message }
+                };
+                return { ...result, presentation: buildSearchPresentation(result) };
+            },
+            fetchPage: async ({ url }) => {
+                const result = {
+                    status: 'error',
+                    url,
+                    metadata: { method: 'http', cacheHit: false },
+                    error: { code: 'BACKEND_CONFIG_INVALID', message }
+                };
+                return { ...result, presentation: buildFetchPresentation(result) };
+            },
+            headlessFetch: async ({ url }) => {
+                const result = {
+                    status: 'error',
+                    url,
+                    metadata: { method: 'headless', cacheHit: false },
+                    error: { code: 'BACKEND_CONFIG_INVALID', message }
+                };
+                return { ...result, presentation: buildFetchPresentation(result) };
+            }
+        };
+    }
+    // When a proxy is configured, every outbound HTTP request (search, fetch,
+    // readers, and doctor-style checks) goes through it; headless browser
+    // traffic gets the same proxy via Playwright launch options.
+    const fetchImpl = proxy ? makeProxyFetch(proxy) : fetch;
+    const proxyCredentials = proxy ? resolveProxyCredentials(proxy) : undefined;
+    const proxyBrowserOptions = proxy
+        ? {
+            server: stripProxyCredentials(proxy.url),
+            ...(proxyCredentials?.username !== undefined ? { username: proxyCredentials.username } : {}),
+            ...(proxyCredentials?.password !== undefined ? { password: proxyCredentials.password } : {})
+        }
+        : undefined;
+    const createDuckDuckGo = () => createDuckDuckGoSearch({ searchHtml: (query) => fetchDuckDuckGoHtml(query, { fetchImpl }) });
     function buildProviderSearch(name) {
         switch (name) {
             case 'searxng':
                 return config.search.baseUrl
-                    ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options })
+                    ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options, fetchImpl })
                     : invalidSearxngSearch();
             case 'brave':
-                return createBraveSearch({ apiKey: process.env.PI_WEB_AGENT_BRAVE_API_KEY });
+                return createBraveSearch({ apiKey: process.env.PI_WEB_AGENT_BRAVE_API_KEY, fetchImpl });
             case 'youcom':
-                return createYouComSearch({ apiKey: process.env.YDC_API_KEY });
+                return createYouComSearch({ apiKey: process.env.YDC_API_KEY, fetchImpl });
             case 'exa':
-                return createExaSearch({ apiKey: process.env.EXA_API_KEY });
+                return createExaSearch({ apiKey: process.env.EXA_API_KEY, fetchImpl });
             case 'tavily':
-                return createTavilySearch({ apiKey: process.env.TAVILY_API_KEY });
+                return createTavilySearch({ apiKey: process.env.TAVILY_API_KEY, fetchImpl });
             case 'duckduckgo':
             default:
-                return createDuckDuckGoSearch();
+                return createDuckDuckGo();
         }
     }
     let search = config.search.provider === 'searxng'
         ? config.search.baseUrl
-            ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options })
+            ? createSearxngSearch({ baseUrl: config.search.baseUrl, options: config.search.options, fetchImpl })
             : invalidSearxngSearch()
         : config.search.provider === 'brave'
-            ? createBraveSearch({ apiKey: process.env.PI_WEB_AGENT_BRAVE_API_KEY })
+            ? createBraveSearch({ apiKey: process.env.PI_WEB_AGENT_BRAVE_API_KEY, fetchImpl })
             : config.search.provider === 'youcom'
-                ? createYouComSearch({ apiKey: process.env.YDC_API_KEY })
+                ? createYouComSearch({ apiKey: process.env.YDC_API_KEY, fetchImpl })
                 : config.search.provider === 'exa'
-                    ? createExaSearch({ apiKey: process.env.EXA_API_KEY })
+                    ? createExaSearch({ apiKey: process.env.EXA_API_KEY, fetchImpl })
                     : config.search.provider === 'tavily'
-                        ? createTavilySearch({ apiKey: process.env.TAVILY_API_KEY })
-                        : createDuckDuckGoSearch();
+                        ? createTavilySearch({ apiKey: process.env.TAVILY_API_KEY, fetchImpl })
+                        : createDuckDuckGo();
     if (config.search.provider === 'searxng' && config.search.fallback === 'duckduckgo') {
-        search = withSearchFallback(search, createDuckDuckGoSearch(), 'searxng');
+        search = withSearchFallback(search, createDuckDuckGo(), 'searxng');
     }
     if (config.search.provider === 'brave' && config.search.fallback === 'duckduckgo') {
-        search = withSearchFallback(search, createDuckDuckGoSearch(), 'brave');
+        search = withSearchFallback(search, createDuckDuckGo(), 'brave');
     }
     if (config.search.provider === 'youcom' && config.search.fallback === 'duckduckgo') {
-        search = withSearchFallback(search, createDuckDuckGoSearch(), 'youcom');
+        search = withSearchFallback(search, createDuckDuckGo(), 'youcom');
     }
     if (config.search.provider === 'exa' && config.search.fallback === 'duckduckgo') {
-        search = withSearchFallback(search, createDuckDuckGoSearch(), 'exa');
+        search = withSearchFallback(search, createDuckDuckGo(), 'exa');
     }
     if (config.search.provider === 'tavily' && config.search.fallback === 'duckduckgo') {
-        search = withSearchFallback(search, createDuckDuckGoSearch(), 'tavily');
+        search = withSearchFallback(search, createDuckDuckGo(), 'tavily');
     }
     const fanoutConfig = config.search.fanout;
     if (fanoutConfig && fanoutConfig.mode !== 'off') {
@@ -158,16 +214,17 @@ export function createBackendSet(config = DEFAULT_BACKEND_CONFIG, deps = {}) {
     const keylessFallbackDisabled = process.env.PI_WEB_AGENT_DISABLE_KEYLESS_FALLBACK === '1';
     const usingDuckDuckGoDefault = config.search.provider === 'duckduckgo' || !config.search.provider;
     if (usingDuckDuckGoDefault && !keylessFallbackDisabled) {
-        search = withSearchFallback(search, createTavilySearch({ keyless: true }), 'duckduckgo');
+        search = withSearchFallback(search, createTavilySearch({ keyless: true, fetchImpl }), 'duckduckgo');
     }
-    const httpFetch = createHttpFetch();
+    const httpFetch = createHttpFetch({ fetchPage: createHttpFetcher({ fetchImpl }) });
     let fetchPage = config.fetch.provider === 'firecrawl'
         ? config.fetch.baseUrl
             ? createHttpFetch({
                 fetchPage: createFirecrawlFetch({
                     baseUrl: config.fetch.baseUrl,
                     apiKey: config.fetch.apiKey ?? process.env.PI_WEB_AGENT_FIRECRAWL_API_KEY,
-                    options: config.fetch.options
+                    options: config.fetch.options,
+                    fetchImpl
                 })
             })
             : createHttpFetch({ fetchPage: invalidFirecrawlFetch() })
@@ -176,12 +233,13 @@ export function createBackendSet(config = DEFAULT_BACKEND_CONFIG, deps = {}) {
         fetchPage = withFetchFallback(fetchPage, httpFetch);
     }
     const fetchPageWithReaders = createSpecialContentResolver({
-        readers: [createGithubReader(), createPdfReader(), createYoutubeReader()],
+        readers: [createGithubReader({ fetchImpl }), createPdfReader({ fetchImpl }), createYoutubeReader({ fetchImpl })],
         fallback: fetchPage
     });
+    const headlessPage = (url) => proxyBrowserOptions ? headlessFetch(url, { proxy: proxyBrowserOptions }) : headlessFetch(url);
     return {
         search,
         fetchPage: fetchPageWithReaders,
-        headlessFetch: createHeadlessFetch()
+        headlessFetch: createHeadlessFetch({ fetchPage: headlessPage })
     };
 }

package/dist/commands/web-agent-config.d.ts

@@ -16,6 +16,9 @@ type CommandDeps = {
         arch: string;
     };
     checkTypebox?: () => Promise<boolean>;
+    checkJitiCompat?: () => {
+        pending: string[];
+    };
     checkBackends?: (config: BackendConfig) => Promise<string[]>;
     getChangelog?: () => Promise<string | undefined>;
 };

package/dist/commands/web-agent-config.js

@@ -1,10 +1,12 @@
-import { DEFAULT_BACKEND_CONFIG, mergeBackendConfigLayers, validateBackendConfig, usableSearchProviders } from '../backends/config.js';
+import { DEFAULT_BACKEND_CONFIG, isValidProxyUrl, mergeBackendConfigLayers, stripProxyCredentials, validateBackendConfig, usableSearchProviders } from '../backends/config.js';
 import { checkBackendHealth } from '../backends/doctor.js';
 import { DynamicBorder, getSettingsListTheme } from '@earendil-works/pi-coding-agent';
 import { Container, Input, SelectList, SettingsList, Text } from '@earendil-works/pi-tui';
 import { DEFAULT_PRESENTATION_CONFIG, mergePresentationConfigLayers, resolvePresentationMode } from '../presentation/config.js';
 import { loadPresentationConfigLayers, resetPresentationConfigScope, saveBackendConfigScope, savePresentationConfigScope } from '../presentation/config-store.js';
 import { resolveBrowserExecutable } from '../fetch/browser-resolution.js';
+import { createProxyFetch } from '../fetch/proxy-fetch.js';
+import { checkJitiCompat } from '../jiti-compat.js';
 import { getLatestChangelogEntry } from '../changelog-notice.js';
 const PRESENTATION_TOOL_NAMES = ['web_explore'];
 function parseScopeToken(token) {
@@ -29,7 +31,8 @@ function cloneBackendConfig(config) {
             ...config.fetch,
             options: config.fetch.options ? { ...config.fetch.options } : undefined
         },
-        headless: { ...config.headless }
+        headless: { ...config.headless },
+        proxy: config.proxy ? { ...config.proxy } : undefined
     };
 }
 function sameJson(left, right) {
@@ -47,6 +50,21 @@ export function validateBackendUrl(value) {
         return { ok: false, message: 'Invalid URL. Include http:// or https://.' };
     }
 }
+/**
+ * The extension re-applies this patch on load, so `pending` here means the
+ * patch could not be written (read-only install, permissions, or a dependency
+ * whose shape changed). Point at the manual script in that case: it is the
+ * same recovery step people have been finding by digging through issue #34.
+ */
+function formatJitiCompatLine(status) {
+    if (status.pending.length === 0)
+        return 'jsdom compat patch: ok';
+    return [
+        `jsdom compat patch: needed (${status.pending.join(', ')})`,
+        'Run: node ~/.pi/agent/npm/node_modules/@demigodmode/pi-web-agent/scripts/patch-jiti-compat.mjs',
+        'then restart Pi.'
+    ].join('\n');
+}
 async function defaultCheckTypebox() {
     try {
         await import('typebox');
@@ -87,8 +105,11 @@ function formatBackendSummary(config = DEFAULT_BACKEND_CONFIG) {
     return [
         searchSuffix ? `${searchBase} ${searchSuffix}` : searchBase,
         fetchSuffix ? `${fetchBase} ${fetchSuffix}` : fetchBase,
-        `headless: ${config.headless.provider}`
-    ].join('\n');
+        `headless: ${config.headless.provider}`,
+        config.proxy ? `proxy: ${stripProxyCredentials(config.proxy.url)}` : undefined
+    ]
+        .filter(Boolean)
+        .join('\n');
 }
 function formatConfigSummary(config) {
     const lines = [`defaultMode: ${config.defaultMode}`];
@@ -249,6 +270,12 @@ function buildBackendSettingsItems(scope, backends, theme, onUrlEditorOpenChange
             label: 'Firecrawl API key',
             currentValue: 'env var',
             values: ['env var']
+        },
+        {
+            id: 'backend:proxy:url',
+            label: 'Proxy URL',
+            currentValue: backends.proxy ? stripProxyCredentials(backends.proxy.url) : 'not set',
+            submenu: createBackendUrlEditor(theme, 'HTTP proxy URL', 'http://127.0.0.1:7890', onUrlEditorOpenChange)
         }
     ];
 }
@@ -421,6 +448,14 @@ export function applySettingsValue(state, id, newValue) {
             delete currentBackends.fetch.baseUrl;
         }
     }
+    if (id === 'backend:proxy:url') {
+        if (newValue.trim()) {
+            currentBackends.proxy = { ...currentBackends.proxy, url: newValue.trim() };
+        }
+        else {
+            delete currentBackends.proxy;
+        }
+    }
     nextDrafts[nextScope] = currentDraft;
     nextBackendDrafts[nextScope] = currentBackends;
     return {
@@ -485,6 +520,17 @@ export function collapseBackendConfigToOverride(config, inheritedConfig) {
     if (!sameJson(config.headless, inheritedConfig.headless)) {
         override.headless = { ...config.headless };
     }
+    if (!sameJson(config.proxy, inheritedConfig.proxy)) {
+        if (config.proxy) {
+            const { password: _password, ...proxy } = config.proxy;
+            override.proxy = { ...proxy };
+        }
+        else if (inheritedConfig.proxy) {
+            // The proxy was cleared at this scope; record an explicit disable so it
+            // overrides a proxy set in a lower (e.g. global) layer.
+            override.proxy = { url: '' };
+        }
+    }
     return override;
 }
 export function handleSettingsShortcut(data) {
@@ -669,7 +715,12 @@ export function registerWebAgentConfigCommands(pi, deps = {}) {
         arch: process.arch
     };
     const checkTypebox = deps.checkTypebox ?? defaultCheckTypebox;
-    const checkBackends = deps.checkBackends ?? ((config) => checkBackendHealth(config));
+    const checkCompat = deps.checkJitiCompat ?? checkJitiCompat;
+    const checkBackends = deps.checkBackends ?? ((config) => checkBackendHealth(config, {
+        // An invalid proxy url is reported by validateBackendConfig below; never
+        // build a proxy agent from it (no connectivity check is attempted).
+        fetchImpl: config.proxy && isValidProxyUrl(config.proxy.url) ? createProxyFetch(config.proxy) : fetch
+    }));
     const getChangelog = deps.getChangelog ?? (() => getLatestChangelogEntry());
     pi.registerCommand('web-agent', {
         description: 'Open settings or manage pi-web-agent presentation config',
@@ -700,6 +751,7 @@ export function registerWebAgentConfigCommands(pi, deps = {}) {
                     'pi-web-agent: loaded',
                     `runtime: node ${runtime.nodeVersion} ${runtime.platform} ${runtime.arch}`,
                     `typebox: ${typeboxOk ? 'ok' : 'missing'}`,
+                    formatJitiCompatLine(checkCompat()),
                     formatBackendSummary(backendConfig),
                     backendIssues.length > 0 ? `backend config: warning\n${backendIssues.join('\n')}` : 'backend config: ok',
                     ...backendHealth

package/dist/extension.d.ts

@@ -1,2 +1,3 @@
+import './jiti-compat-run.js';
 import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
 export default function extension(pi: ExtensionAPI): void;

package/dist/extension.js

@@ -1,3 +1,5 @@
+// Keep first: re-applies the jsdom/jiti compat patch before jsdom loads (#34).
+import './jiti-compat-run.js';
 import { DEFAULT_BACKEND_CONFIG } from './backends/config.js';
 import { Type } from 'typebox';
 import { registerWebAgentConfigCommands } from './commands/web-agent-config.js';

package/dist/fetch/headless-fetch.d.ts

@@ -1,13 +1,20 @@
 import { type BrowserResolutionResult } from './browser-resolution.js';
 import type { WebFetchHeadlessResponse } from '../types.js';
-export declare function headlessFetch(url: string, { configuredPath, resolveBrowser, launchBrowser, now }?: {
+export type BrowserProxyOptions = {
+    server: string;
+    username?: string;
+    password?: string;
+};
+export declare function headlessFetch(url: string, { configuredPath, proxy, resolveBrowser, launchBrowser, now }?: {
     configuredPath?: string;
+    proxy?: BrowserProxyOptions;
     resolveBrowser?: (options?: {
         configuredPath?: string;
     }) => Promise<BrowserResolutionResult>;
     launchBrowser?: (options: {
         executablePath?: string;
         headless: true;
+        proxy?: BrowserProxyOptions;
     }) => Promise<{
         newContext: () => Promise<{
             newPage: () => Promise<any>;

package/dist/fetch/headless-fetch.js

@@ -7,7 +7,7 @@ function cleanupRenderedText(text) {
     cleaned = cleaned.replace(/\s+/g, ' ').trim();
     return cleaned;
 }
-export async function headlessFetch(url, { configuredPath, resolveBrowser = (options) => resolveBrowserExecutable({ configuredPath: options?.configuredPath }), launchBrowser = ({ executablePath, headless }) => chromium.launch(executablePath ? { executablePath, headless } : { headless }), now = () => Date.now() } = {}) {
+export async function headlessFetch(url, { configuredPath, proxy, resolveBrowser = (options) => resolveBrowserExecutable({ configuredPath: options?.configuredPath }), launchBrowser = ({ executablePath, headless, proxy }) => chromium.launch(executablePath ? { executablePath, headless, ...(proxy ? { proxy } : {}) } : { headless, ...(proxy ? { proxy } : {}) }), now = () => Date.now() } = {}) {
     const resolved = await resolveBrowser({ configuredPath });
     if (!resolved.ok && resolved.error.code === 'CONFIGURED_BROWSER_NOT_FOUND') {
         return {
@@ -19,8 +19,8 @@ export async function headlessFetch(url, { configuredPath, resolveBrowser = (opt
     }
     const browserName = resolved.ok ? resolved.browser : 'chromium';
     const launchOptions = resolved.ok
-        ? { executablePath: resolved.executablePath, headless: true }
-        : { headless: true };
+        ? { executablePath: resolved.executablePath, headless: true, ...(proxy ? { proxy } : {}) }
+        : { headless: true, ...(proxy ? { proxy } : {}) };
     let browser;
     let context;
     let page;

package/dist/fetch/proxy-fetch.d.ts

@@ -0,0 +1,22 @@
+import { type ProxyConfig } from '../backends/config.js';
+export type ProxyCredentials = {
+    username?: string;
+    password?: string;
+};
+export type ProxyFetchOptions = {
+    /** TLS options for the tunneled connection to the origin (https targets). */
+    tls?: {
+        rejectUnauthorized?: boolean;
+    };
+};
+/**
+ * Resolve proxy credentials from the config, falling back to environment
+ * variables so secrets can stay out of config files (same policy as API keys).
+ */
+export declare function resolveProxyCredentials(proxy: ProxyConfig, env?: NodeJS.ProcessEnv): ProxyCredentials;
+/**
+ * Build a fetch implementation that routes all traffic through the configured
+ * HTTP/HTTPS proxy. Uses undici (the engine behind Node's global fetch) with a
+ * ProxyAgent dispatcher while keeping the standard fetch API surface.
+ */
+export declare function createProxyFetch(proxy: ProxyConfig, options?: ProxyFetchOptions): typeof fetch;

package/dist/fetch/proxy-fetch.js

@@ -0,0 +1,46 @@
+import { fetch as undiciFetch, ProxyAgent } from 'undici';
+import { stripProxyCredentials } from '../backends/config.js';
+/**
+ * Resolve proxy credentials from the config, falling back to environment
+ * variables so secrets can stay out of config files (same policy as API keys).
+ */
+export function resolveProxyCredentials(proxy, env = process.env) {
+    const username = proxy.username ?? env.PI_WEB_AGENT_PROXY_USERNAME;
+    const password = proxy.password ?? env.PI_WEB_AGENT_PROXY_PASSWORD;
+    return {
+        username: username?.trim() ? username : undefined,
+        password: password ? password : undefined
+    };
+}
+/**
+ * Build a fetch implementation that routes all traffic through the configured
+ * HTTP/HTTPS proxy. Uses undici (the engine behind Node's global fetch) with a
+ * ProxyAgent dispatcher while keeping the standard fetch API surface.
+ */
+export function createProxyFetch(proxy, options = {}) {
+    const credentials = resolveProxyCredentials(proxy);
+    const agent = new ProxyAgent({
+        // Never trust credentials embedded in the URL; they come from config fields
+        // or the PI_WEB_AGENT_PROXY_* env vars via resolveProxyCredentials.
+        uri: stripProxyCredentials(proxy.url),
+        // undici's `token` option is used verbatim as the Proxy-Authorization header
+        // value for both CONNECT tunnels and forwarded HTTP requests.
+        ...(credentials.username !== undefined
+            ? {
+                token: `Basic ${Buffer.from(credentials.password !== undefined
+                    ? `${credentials.username}:${credentials.password}`
+                    : `${credentials.username}:`).toString('base64')}`
+            }
+            : {}),
+        ...(options.tls ? { requestTls: { rejectUnauthorized: options.tls.rejectUnauthorized } } : {})
+    });
+    // undici's Request/Response types are structurally near-identical to Node's
+    // global fetch types but not nominatively identical, so the wrapper is cast
+    // to the standard fetch signature (runtime behavior is identical: Node's
+    // global fetch is built on this same undici engine).
+    const proxyFetch = (input, init) => undiciFetch(input, {
+        ...init,
+        dispatcher: agent
+    });
+    return proxyFetch;
+}

package/dist/jiti-compat-run.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/jiti-compat-run.js

@@ -0,0 +1,9 @@
+import { ensureJitiCompat } from './jiti-compat.js';
+// Side-effect-only module. It exists so `extension.ts` can run the compat
+// patch as its *first* import: ESM evaluates a module's dependency subtree,
+// including that module's body, before moving on to the next import. A plain
+// `ensureJitiCompat()` call in the extension body would run too late, after
+// jsdom (and therefore tr46/cssstyle) had already been evaluated.
+//
+// Keep this import first in extension.ts.
+ensureJitiCompat();

package/dist/jiti-compat.d.ts

@@ -0,0 +1,32 @@
+/**
+ * Filesystem/resolver seam so tests can point this at a throwaway directory
+ * instead of the real `node_modules/tr46` and `node_modules/cssstyle`. All
+ * production call sites use the defaults (real `fs` + real module
+ * resolution) and never pass this in.
+ */
+export type JitiCompatDeps = {
+    /**
+     * Resolve `specifier` as `fromFile` would. Omitting `fromFile` resolves from
+     * this module, which is only correct when nothing else claims the package.
+     */
+    resolve: (specifier: string, fromFile?: string) => string;
+    existsSync: (path: string) => boolean;
+    readFileSync: (path: string) => string;
+    writeFileSync: (path: string, contents: string) => void;
+};
+export type JitiCompatStatus = {
+    /** Files that still need patching. Empty means the tree is healthy. */
+    pending: string[];
+    /** Files patched during this call. */
+    patched: string[];
+};
+/**
+ * Apply both patches if they are missing. Safe to call repeatedly: an already
+ * patched tree is a few `readFileSync` calls and no writes. Never throws.
+ */
+export declare function ensureJitiCompat(deps?: JitiCompatDeps): JitiCompatStatus;
+/**
+ * Read-only view of the same checks, for `/web-agent doctor`. Never throws and
+ * never writes.
+ */
+export declare function checkJitiCompat(deps?: JitiCompatDeps): JitiCompatStatus;

package/dist/jiti-compat.js

@@ -0,0 +1,215 @@
+import { existsSync, readFileSync, renameSync, unlinkSync, writeFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { dirname, join, relative, sep } from 'node:path';
+/**
+ * Works around two incompatibilities between pi's extension loader (a patched
+ * jiti) and jsdom's dependency tree:
+ *
+ *   1. jiti can't resolve the trailing-slash bare specifier require("punycode/")
+ *      used by tr46.
+ *   2. jiti wraps `module.exports = new Set(...)` (cssstyle) in a Proxy, which
+ *      breaks native Set methods on the exported value.
+ *
+ * Both files live in the shared `~/.pi/agent/npm/node_modules` tree, so
+ * installing or updating any *other* pi extension re-extracts them and reverts
+ * the patch. A postinstall hook alone therefore can't keep this healthy, which
+ * is why `ensureJitiCompat()` also runs on extension load, before jsdom is
+ * evaluated. See https://github.com/demigodmode/pi-web-agent/issues/34.
+ *
+ * scripts/patch-jiti-compat.mjs duplicates this logic for the postinstall
+ * hook. That hook runs before `npm run build`, so it cannot import dist/.
+ * Keep the two in sync.
+ */
+const requireFromHere = createRequire(import.meta.url);
+/**
+ * Write via a temp file + rename rather than in place. Two reasons, both of
+ * which bite harder now that this runs on every extension load instead of once
+ * per install:
+ *
+ *   - `writeFileSync` truncates first. A crash, a full disk, or an OOM kill
+ *     mid-write leaves a half-written file that still contains the marker
+ *     comment, so every later run would skip it as "already patched" and the
+ *     doctor would report a broken tree as healthy.
+ *   - Another Pi session can be `require`-ing the same file while we write it.
+ *     `rename` is atomic within a filesystem, so readers see the old file or
+ *     the new one, never a partial one.
+ *
+ * The rename also breaks a pnpm-style hardlink instead of writing through it
+ * into the shared content-addressable store.
+ */
+function writeFileAtomic(path, contents) {
+    const temp = `${path}.pi-web-agent-${process.pid}.tmp`;
+    try {
+        writeFileSync(temp, contents);
+        renameSync(temp, path);
+    }
+    catch (err) {
+        try {
+            if (existsSync(temp))
+                unlinkSync(temp);
+        }
+        catch {
+            // Best effort. Leaving a stray temp file is better than masking the
+            // original write failure.
+        }
+        throw err;
+    }
+}
+const defaultDeps = {
+    resolve: (specifier, fromFile) => (fromFile ? createRequire(fromFile) : requireFromHere).resolve(specifier),
+    existsSync,
+    readFileSync: (path) => readFileSync(path, 'utf8'),
+    writeFileSync: writeFileAtomic
+};
+const SET_SHIM_MARKER = 'pi/jiti workaround';
+const SET_SHIM = `
+// ${SET_SHIM_MARKER}: expose bound native Set methods as own properties so a
+// Proxy wrapper around this export does not break Set brand checks.
+for (const k of ["has", "add", "delete", "forEach", "keys", "values", "entries"]) {
+  module.exports[k] = Set.prototype[k].bind(module.exports);
+}
+module.exports[Symbol.iterator] = Set.prototype[Symbol.iterator].bind(module.exports);
+`;
+const PUNYCODE_SPECIFIER = 'require("punycode/")';
+const PUNYCODE_REPLACEMENT = 'require("punycode/punycode.js")';
+function findPackageRoot(deps, entryFile, packageName) {
+    let directory = dirname(entryFile);
+    for (;;) {
+        const manifestFile = join(directory, 'package.json');
+        if (deps.existsSync(manifestFile)) {
+            try {
+                const manifest = JSON.parse(deps.readFileSync(manifestFile));
+                if (manifest.name === packageName)
+                    return directory;
+            }
+            catch {
+                // An unreadable or malformed package.json on the way up is not our
+                // problem to report. Keep walking.
+            }
+        }
+        const parent = dirname(directory);
+        if (parent === directory) {
+            throw new Error(`could not find the ${packageName} package root from ${entryFile}`);
+        }
+        directory = parent;
+    }
+}
+/**
+ * Resolve the copies jsdom actually loads, not whichever copy happens to sit
+ * highest in the tree.
+ *
+ * `~/.pi/agent/npm/node_modules` is shared by every pi extension, so version
+ * conflicts routinely push a second copy of a package into a nested
+ * `node_modules`. Resolving `cssstyle` or `tr46` from *this* module can then
+ * find a hoisted copy that jsdom never requires: the patch lands on the wrong
+ * file, the load still fails, and the doctor reports a healthy tree because it
+ * checked the same wrong file. Walk the real import chain instead
+ * (jsdom -> cssstyle, jsdom -> whatwg-url -> tr46) and only fall back to a
+ * direct resolve when jsdom is not resolvable at all.
+ */
+function resolveFromJsdom(deps, specifier, via) {
+    try {
+        const jsdomEntry = deps.resolve('jsdom');
+        const importer = via ? deps.resolve(via, jsdomEntry) : jsdomEntry;
+        return deps.resolve(specifier, importer);
+    }
+    catch {
+        // jsdom (or the intermediate package) is not resolvable from here, e.g. a
+        // layout we do not recognize. Fall back to a plain resolve rather than
+        // giving up on the patch entirely.
+        return deps.resolve(specifier);
+    }
+}
+function tr46Entry(deps) {
+    return resolveFromJsdom(deps, 'tr46', 'whatwg-url');
+}
+function cssstyleTargets(deps) {
+    const packageRoot = findPackageRoot(deps, resolveFromJsdom(deps, 'cssstyle'), 'cssstyle');
+    return [
+        join(packageRoot, 'lib', 'allExtraProperties.js'),
+        join(packageRoot, 'lib', 'generated', 'allProperties.js'),
+        join(packageRoot, 'lib', 'generated', 'implementedProperties.js')
+    ].map((file) => ({
+        file,
+        label: `cssstyle/${relative(packageRoot, file).split(sep).join('/')}`
+    }));
+}
+/**
+ * Apply both patches if they are missing. Safe to call repeatedly: an already
+ * patched tree is a few `readFileSync` calls and no writes. Never throws.
+ */
+export function ensureJitiCompat(deps = defaultDeps) {
+    const status = { pending: [], patched: [] };
+    try {
+        const file = tr46Entry(deps);
+        const contents = deps.readFileSync(file);
+        if (contents.includes(PUNYCODE_SPECIFIER)) {
+            deps.writeFileSync(file, contents.replaceAll(PUNYCODE_SPECIFIER, PUNYCODE_REPLACEMENT));
+            status.patched.push('tr46/index.js');
+        }
+    }
+    catch {
+        // A read-only install, a missing dependency, or a future dependency bump
+        // that changes the shape. Report it rather than blocking extension load.
+        status.pending.push('tr46/index.js');
+    }
+    try {
+        for (const { file, label } of cssstyleTargets(deps)) {
+            // Guard the read too: one unreadable file (EACCES, odd permissions) must
+            // not abandon the other two, which may be perfectly writable.
+            try {
+                if (!deps.existsSync(file))
+                    continue;
+                const contents = deps.readFileSync(file);
+                if (contents.includes(SET_SHIM_MARKER))
+                    continue;
+                if (!contents.includes('module.exports = new Set('))
+                    continue;
+                deps.writeFileSync(file, contents + SET_SHIM);
+                status.patched.push(label);
+            }
+            catch {
+                status.pending.push(label);
+            }
+        }
+    }
+    catch {
+        status.pending.push('cssstyle');
+    }
+    return status;
+}
+/**
+ * Read-only view of the same checks, for `/web-agent doctor`. Never throws and
+ * never writes.
+ */
+export function checkJitiCompat(deps = defaultDeps) {
+    const status = { pending: [], patched: [] };
+    try {
+        const contents = deps.readFileSync(tr46Entry(deps));
+        if (contents.includes(PUNYCODE_SPECIFIER))
+            status.pending.push('tr46/index.js');
+    }
+    catch {
+        status.pending.push('tr46/index.js');
+    }
+    try {
+        for (const { file, label } of cssstyleTargets(deps)) {
+            try {
+                if (!deps.existsSync(file))
+                    continue;
+                const contents = deps.readFileSync(file);
+                if (contents.includes(SET_SHIM_MARKER))
+                    continue;
+                if (contents.includes('module.exports = new Set('))
+                    status.pending.push(label);
+            }
+            catch {
+                status.pending.push(label);
+            }
+        }
+    }
+    catch {
+        status.pending.push('cssstyle');
+    }
+    return status;
+}

package/dist/presentation/config-store.js

@@ -61,6 +61,10 @@ function serializeBackendConfigOverride(config) {
     if (config.headless && Object.keys(config.headless).length > 0) {
         backends.headless = { ...config.headless };
     }
+    if (config.proxy && Object.keys(config.proxy).length > 0) {
+        const { password: _password, ...proxy } = config.proxy;
+        backends.proxy = { ...proxy };
+    }
     return { backends };
 }
 async function readConfigFileForWrite(filePath) {

package/dist/readers/youtube-reader.d.ts

@@ -5,6 +5,8 @@ type Subtitle = {
     text: string;
 };
 type YoutubeReaderDeps = {
+    /** Fetch implementation used for YouTube's requests (defaults to global fetch). */
+    fetchImpl?: typeof fetch;
     fetchSubtitles?: (input: {
         videoID: string;
         lang: string;
@@ -17,5 +19,5 @@ type YoutubeReaderDeps = {
         description?: string;
     }>;
 };
-export declare function createYoutubeReader({ fetchSubtitles, fetchDetails }?: YoutubeReaderDeps): SpecialContentReader;
+export declare function createYoutubeReader({ fetchImpl, fetchSubtitles, fetchDetails }?: YoutubeReaderDeps): SpecialContentReader;
 export {};

package/dist/readers/youtube-reader.js

@@ -21,7 +21,15 @@ function extractVideoId(url) {
     }
     return undefined;
 }
-export function createYoutubeReader({ fetchSubtitles = getSubtitles, fetchDetails = getVideoDetails } = {}) {
+export function createYoutubeReader({ fetchImpl = fetch, fetchSubtitles, fetchDetails } = {}) {
+    // Route YouTube's caption/metadata requests through the configured fetch
+    // client (e.g. the proxy) unless an explicit override is provided.
+    const doFetchSubtitles = fetchSubtitles ?? ((input) => getSubtitles({ ...input, fetch: fetchImpl }));
+    const doFetchDetails = fetchDetails ??
+        ((input) => getVideoDetails({ ...input, fetch: fetchImpl }).then((details) => ({
+            title: details.title,
+            description: details.description
+        })));
     return {
         name: 'youtube',
         canHandle(url) {
@@ -39,8 +47,8 @@ export function createYoutubeReader({ fetchSubtitles = getSubtitles, fetchDetail
             }
             try {
                 const [subtitles, details] = await Promise.all([
-                    fetchSubtitles({ videoID, lang: 'en' }),
-                    fetchDetails({ videoID, lang: 'en' }).catch(() => ({ title: undefined }))
+                    doFetchSubtitles({ videoID, lang: 'en' }),
+                    doFetchDetails({ videoID, lang: 'en' }).catch(() => ({ title: undefined }))
                 ]);
                 if (!subtitles || subtitles.length === 0) {
                     return {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@demigodmode/pi-web-agent",
-  "version": "1.10.0",
+  "version": "1.11.0",
   "description": "Pi package for reliable web access with explicit search, fetch, and headless boundaries.",
   "type": "module",
   "main": "./dist/extension.js",
@@ -68,6 +68,7 @@
     "jsdom": "^26.0.0",
     "playwright": "^1.60.0",
     "typebox": "^1.1.37",
+    "undici": "^8.10.2",
     "unpdf": "^1.8.1",
     "youtube-caption-extractor": "^1.10.2"
   },

package/scripts/patch-jiti-compat.mjs

@@ -10,19 +10,63 @@
 // when this package is installed as a pi extension via `pi install npm:...`.
 // Safe to run multiple times and safe to no-op if the target files or
 // patterns are missing (e.g. a future dependency bump changes the shape).
-import { readFileSync, writeFileSync, existsSync } from "node:fs";
+//
+// src/jiti-compat.ts does the same two patches at extension load time, because
+// the shared ~/.pi/agent/npm tree means another extension's install can revert
+// them long after this hook ran (#34). The logic is deliberately duplicated
+// rather than shared: postinstall runs before `npm run build` in CI and in a
+// fresh clone, so this file cannot depend on dist/. Keep the two in sync.
+import { readFileSync, writeFileSync, existsSync, renameSync, unlinkSync } from "node:fs";
 import { createRequire } from "node:module";
 import { dirname, join, relative, sep } from "node:path";
 
-const resolveFromHere = createRequire(import.meta.url).resolve;
+const requireFromHere = createRequire(import.meta.url);
+const resolveFromHere = requireFromHere.resolve;
+
+// Resolve the copies jsdom actually loads. The shared ~/.pi/agent/npm tree
+// often holds a second, nested copy of a package when extensions disagree on
+// versions, and patching a hoisted copy jsdom never requires would fix
+// nothing. Mirrors resolveFromJsdom in src/jiti-compat.ts.
+function resolveFromJsdom(specifier, via) {
+  try {
+    const jsdomEntry = resolveFromHere("jsdom");
+    const importer = via ? createRequire(jsdomEntry).resolve(via) : jsdomEntry;
+    return createRequire(importer).resolve(specifier);
+  } catch {
+    return resolveFromHere(specifier);
+  }
+}
+
+// Temp file + rename, not an in-place write. `writeFileSync` truncates first,
+// so an interrupted write would leave a half-patched file that still contains
+// the marker comment and would be skipped as "already patched" forever.
+// Mirrors writeFileAtomic in src/jiti-compat.ts.
+function writeFileAtomic(path, contents) {
+  const temp = `${path}.pi-web-agent-${process.pid}.tmp`;
+  try {
+    writeFileSync(temp, contents);
+    renameSync(temp, path);
+  } catch (err) {
+    try {
+      if (existsSync(temp)) unlinkSync(temp);
+    } catch {
+      // Best effort.
+    }
+    throw err;
+  }
+}
 
 function findPackageRoot(entryFile, packageName) {
   let directory = dirname(entryFile);
   while (true) {
     const manifestFile = join(directory, "package.json");
     if (existsSync(manifestFile)) {
-      const manifest = JSON.parse(readFileSync(manifestFile, "utf8"));
-      if (manifest.name === packageName) return directory;
+      try {
+        const manifest = JSON.parse(readFileSync(manifestFile, "utf8"));
+        if (manifest.name === packageName) return directory;
+      } catch {
+        // Malformed package.json on the way up. Keep walking.
+      }
     }
 
     const parent = dirname(directory);
@@ -34,7 +78,7 @@ function findPackageRoot(entryFile, packageName) {
 }
 
 function patchTr46() {
-  const file = resolveFromHere("tr46");
+  const file = resolveFromJsdom("tr46", "whatwg-url");
   if (!existsSync(file)) {
     console.debug("patch-jiti-compat: tr46/index.js not found, skipping");
     return;
@@ -44,7 +88,7 @@ function patchTr46() {
     console.debug("patch-jiti-compat: tr46/index.js does not match expected pattern, skipping");
     return;
   }
-  writeFileSync(
+  writeFileAtomic(
     file,
     contents.replaceAll('require("punycode/")', 'require("punycode/punycode.js")'),
   );
@@ -61,7 +105,7 @@ module.exports[Symbol.iterator] = Set.prototype[Symbol.iterator].bind(module.exp
 `;
 
 function patchCssstyleSetExports() {
-  const packageRoot = findPackageRoot(resolveFromHere("cssstyle"), "cssstyle");
+  const packageRoot = findPackageRoot(resolveFromJsdom("cssstyle"), "cssstyle");
   const files = [
     join(packageRoot, "lib", "allExtraProperties.js"),
     join(packageRoot, "lib", "generated", "allProperties.js"),
@@ -79,7 +123,7 @@ function patchCssstyleSetExports() {
       console.debug(`patch-jiti-compat: ${label} does not match expected pattern, skipping`);
       continue;
     }
-    writeFileSync(file, contents + SET_SHIM);
+    writeFileAtomic(file, contents + SET_SHIM);
     console.log(`patch-jiti-compat: patched ${label}`);
   }
 }