@bacnh85/pi-web 0.6.2 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,47 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.7.1 (2026-09-12)
4
+
5
+ ### Fixed
6
+
7
+ - `before_agent_start` routing guidance now falls back to `pi.getActiveTools()`
8
+ when the host omits `systemPromptOptions` (mirrors pi-fff), so guidance no
9
+ longer silently skips injection.
10
+ - README: corrected default backend URLs to `127.0.0.1` (matching code defaults
11
+ in `lib/config.ts`), removed the nonexistent `npm run test:unit` script
12
+ reference, documented the `web_search` `timeout_ms` parameter.
13
+
14
+ ## 0.7.0 (2026-09-11)
15
+
16
+ ### Added
17
+
18
+ - **Local capture engine** — `web_screenshot` and `web_pdf` now capture
19
+ `localhost`/LAN/`file://` URLs via the locally installed Chrome/Chromium
20
+ (headless CLI, zero dependencies). The Crawl4AI daemon's browser runs on the
21
+ daemon host and SSRF-blocks private addresses, so local dev servers were
22
+ uncapturable before. Routing is automatic (`engine="auto"` default):
23
+ private URLs → local Chrome, public URLs → daemon, and a daemon SSRF-block
24
+ on an otherwise-public URL falls back to local Chrome automatically.
25
+ `engine="local"`/`"daemon"` forces one. New `web_screenshot` params:
26
+ `width` (1280), `height` (800), `full_page` (tall 8000px window — the
27
+ Chrome CLI has no true full-page flag). Binary discovery: `CHROME_PATH` env
28
+ → standard per-OS paths (Edge as Windows fallback). Captures run in an
29
+ isolated temp profile with a 30s timeout; `wait_for` maps to
30
+ `--virtual-time-budget`. Chrome versions that write the capture but never
31
+ exit (fresh `--user-data-dir` on macOS) are handled by polling for a
32
+ size-stable output file instead of requiring a clean exit. Review-hardened:
33
+ capture URLs are scheme-validated (http/https/file) before spawn so
34
+ switch-like strings can't be injected as Chrome flags; IPv6 loopback/ULA/
35
+ link-local (`[::1]`, `fc00::/7`, `fe80::/10`) route to local Chrome (Node
36
+ `URL.hostname` keeps brackets); daemon `details` payloads are preserved
37
+ (mime/artifact/full result) alongside the new `engine` key; timeout is
38
+ always a failure (a complete capture is caught by the stability poll first).
39
+ `web_status` reports the discovered local Chrome path.
40
+ - New `extensions/lib/chrome.ts` (engine + `isLocalUrl`/`resolveEngine`/
41
+ `isSsrfBlocked` helpers) with unit tests in `test/unit/chrome.test.ts`;
42
+ live-verified against a local `http.server` (PNG magic, PDF magic, inline
43
+ image block, tmp cleanup, no orphan processes).
44
+
3
45
  ## 0.6.2 (2026-09-06)
4
46
 
5
47
  ### Changed
package/README.md CHANGED
@@ -25,10 +25,10 @@ Variables:
25
25
  | Variable | Required | Default | Notes |
26
26
  |---|---|---|---|
27
27
  | `BRAVE_API_KEY` | No (1) | — | Brave Search API key |
28
- | `SEARXNG_BASE_URL` | No | `http://172.30.55.22:8888` | Self-hosted SearXNG |
28
+ | `SEARXNG_BASE_URL` | No | `http://127.0.0.1:8888` | Self-hosted SearXNG |
29
29
  | `FIRECRAWL_API_URL` | No | `https://api.firecrawl.dev/v2` | Self-hosted or hosted |
30
30
  | `FIRECRAWL_API_KEY` | No (2) | — | Required for hosted Firecrawl |
31
- | `CRAWL4AI_API_URL` | No | `http://172.30.55.22:11235` | Self-hosted Crawl4AI |
31
+ | `CRAWL4AI_API_URL` | No | `http://127.0.0.1:11235` | Self-hosted Crawl4AI |
32
32
  | `CRAWL4AI_API_TOKEN` | No (3) | — | Required if Crawl4AI auth enabled |
33
33
 
34
34
  > (1) At least one search backend (SearXNG, Brave, or Firecrawl) must be configured for `web_search`.
@@ -66,6 +66,7 @@ Parameters:
66
66
  | `engines` | string | — | SearXNG engine override, e.g. `google,github` |
67
67
  | `include_content` | boolean | false | Fetch page content alongside results |
68
68
  | `content_chars` | number | 5000 | Max content chars per result |
69
+ | `timeout_ms` | number | per-backend | Request timeout in ms (SearXNG/static 15000, Firecrawl/Crawl4AI 60000) |
69
70
 
70
71
  **Auto-selection behavior:**
71
72
 
@@ -142,21 +143,39 @@ web_crawl url="https://example.com" mode=light poll=true # Poll for completio
142
143
 
143
144
  ### `web_screenshot` — Page screenshot
144
145
 
145
- Captures a full-page PNG screenshot using Crawl4AI. Returns the PNG inline as an image block (multimodal models see it); text summary includes artifact/MIME/size.
146
+ Captures a full-page PNG screenshot using the Crawl4AI daemon, or **local headless Chrome for localhost/LAN/file URLs** (auto-detected; see [Local capture](#local-capture)). Returns the PNG inline as an image block (multimodal models see it); text summary includes engine/MIME/size.
146
147
 
147
148
  ```
148
149
  web_screenshot url="https://example.com"
149
150
  web_screenshot url="https://example.com" wait_for=5 wait_for_images=true
151
+ web_screenshot url="http://localhost:3000" # local Chrome, auto-detected
152
+ web_screenshot url="http://localhost:3000" full_page=true width=1280
153
+ web_screenshot url="https://example.com" engine="daemon" # force the daemon
150
154
  ```
151
155
 
156
+ Local-engine params: `width` (default 1280), `height` (default 800), `full_page` (captures a tall 8000px window — the Chrome CLI has no true full-page flag).
157
+
152
158
  ### `web_pdf` — Page PDF
153
159
 
154
- Generates a PDF document using Crawl4AI. Returns base64-encoded PDF.
160
+ Generates a PDF document using the Crawl4AI daemon, or **local headless Chrome** for localhost/LAN/file URLs (auto-detected). Returns base64-encoded PDF.
155
161
 
156
162
  ```
157
163
  web_pdf url="https://example.com/article"
164
+ web_pdf url="http://localhost:3000" # local Chrome, auto-detected
158
165
  ```
159
166
 
167
+ ### Local capture
168
+
169
+ The Crawl4AI daemon's browser runs on the daemon host — it cannot reach (and SSRF-blocks) your `localhost`. pi-web therefore routes private URLs to a **locally installed Chrome/Chromium** in headless mode:
170
+
171
+ | URL | Engine |
172
+ |-----|--------|
173
+ | `localhost`, `127.0.0.1`, LAN IPs (10/8, 172.16/12, 192.168/16, 169.254/16), `file://` | local Chrome |
174
+ | public URLs | Crawl4AI daemon |
175
+ | daemon SSRF-blocks a URL | automatic local-Chrome retry |
176
+
177
+ Override with `engine="local"` / `engine="daemon"`. Binary discovery: `CHROME_PATH` env, then standard Chrome/Chromium paths per OS (Edge as a Windows fallback). Captures use an isolated temp profile, a 30s timeout, and `--virtual-time-budget` for `wait_for`.
178
+
160
179
  ### `web_status` — Provider status
161
180
 
162
181
  Shows all provider configuration status and Crawl4AI server health.
@@ -170,14 +189,15 @@ Typical output:
170
189
  ```json
171
190
  {
172
191
  "brave": { "apiKeyFound": true, "apiKeySource": "process.env" },
173
- "searxng": { "baseUrl": "http://172.30.55.22:8888", ... },
174
- "firecrawl": { "baseUrl": "http://172.30.55.22:3002/v2", ... },
192
+ "searxng": { "baseUrl": "http://127.0.0.1:8888", ... },
193
+ "firecrawl": { "baseUrl": "http://127.0.0.1:3002/v2", ... },
175
194
  "crawl4ai": {
176
- "baseUrl": "http://172.30.55.22:11235",
195
+ "baseUrl": "http://127.0.0.1:11235",
177
196
  ...
178
197
  "health": { "status": "healthy", "version": "0.5.0", ... }
179
198
  },
180
- "agy": { "installed": true }
199
+ "agy": { "installed": true },
200
+ "localChrome": { "path": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" }
181
201
  }
182
202
  ```
183
203
 
@@ -229,7 +249,4 @@ See [CHANGELOG.md](CHANGELOG.md) for release history.
229
249
  ```bash
230
250
  # Run all tests
231
251
  npm test
232
-
233
- # Run only unit tests
234
- npm run test:unit
235
252
  ```
@@ -29,6 +29,13 @@ import {
29
29
  fetchCrawl4aiPdf,
30
30
  fetchCrawl4aiHealth,
31
31
  } from "./lib/crawl4ai";
32
+ import {
33
+ capturePdf as captureLocalPdf,
34
+ captureScreenshot as captureLocalScreenshot,
35
+ findChromeBinary,
36
+ isSsrfBlocked,
37
+ resolveEngine,
38
+ } from "./lib/chrome";
32
39
 
33
40
  // ---------------------------------------------------------------------------
34
41
  // Shared schema fragment
@@ -48,6 +55,14 @@ const crawl4aiControlSchema = {
48
55
  crawl4ai_api_token: Type.Optional(Type.String({ description: "Override $CRAWL4AI_API_TOKEN." })),
49
56
  };
50
57
 
58
+ const engineSchema = {
59
+ engine: Type.Optional(Type.Union([
60
+ Type.Literal("auto"),
61
+ Type.Literal("local"),
62
+ Type.Literal("daemon"),
63
+ ], { default: "auto", description: "auto routes localhost/private/file URLs to local Chrome, the rest to the Crawl4AI daemon; local/daemon force one." })),
64
+ };
65
+
51
66
  // ---------------------------------------------------------------------------
52
67
  // Always-on routing guidance (injected only when a web_* tool is active)
53
68
  // ---------------------------------------------------------------------------
@@ -286,33 +301,80 @@ export default function piWebExtension(pi: ExtensionAPI) {
286
301
  name: "web_screenshot",
287
302
  label: "Web Page Screenshot",
288
303
  description:
289
- "Full-page PNG screenshot via Crawl4AI. The PNG is returned inline as an image block.",
304
+ "Full-page PNG screenshot via the Crawl4AI daemon, or via local headless Chrome for localhost/private/file URLs (auto-detected, engine overridable). The PNG is returned inline as an image block.",
290
305
  promptSnippet: "Screenshot a webpage",
291
- promptGuidelines: ["Full-page PNG returned inline (multimodal models see it); use when web_extract fails on JS-heavy pages, or to visually inspect a built UI."],
306
+ promptGuidelines: ["PNG returned inline (multimodal models see it); use when web_extract fails on JS-heavy pages, or to visually inspect a built UI. Local dev servers (localhost/LAN/file://) capture automatically via local Chrome."],
292
307
  parameters: Type.Object({
293
308
  url: Type.String(),
294
309
  wait_for: Type.Optional(Type.Number({ default: 2, description: "Seconds to wait before capture." })),
295
310
  wait_for_images: Type.Optional(Type.Boolean({ default: false })),
311
+ engine: Type.Optional(engineSchema.engine),
312
+ width: Type.Optional(Type.Number({ default: 1280, description: "Local engine: viewport width." })),
313
+ height: Type.Optional(Type.Number({ default: 800, description: "Local engine: viewport height (full_page uses 8000)." })),
314
+ full_page: Type.Optional(Type.Boolean({ default: false, description: "Local engine: capture a tall 8000px window to approximate full page." })),
296
315
  ...crawl4aiControlSchema,
297
316
  ...sharedControlSchema,
298
317
  }),
299
318
  async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
300
- const config = loadCrawl4aiConfig(params as Record<string, unknown>, cwdFromContext(ctx), includeProjectEnv(ctx));
301
- const result = await fetchCrawl4aiScreenshot(
302
- config,
303
- params.url as string,
304
- params.wait_for as number | undefined,
305
- params.wait_for_images as boolean | undefined,
306
- signal,
307
- );
308
- if (result.success === false) {
309
- throw new Error(String(result.error_message ?? "Crawl4AI screenshot failed"));
319
+ const url = params.url as string;
320
+ let engine = resolveEngine(params.engine as string | undefined, url);
321
+ let screenshot: string | undefined;
322
+ let mime: string | undefined;
323
+ let size: number | undefined;
324
+ let artifactUrl: string | undefined;
325
+ let details: Record<string, unknown> = {};
326
+
327
+ if (engine === "local") {
328
+ const cap = await captureLocalScreenshot({
329
+ url,
330
+ width: params.width as number | undefined,
331
+ height: params.height as number | undefined,
332
+ fullPage: params.full_page as boolean | undefined,
333
+ waitForSec: params.wait_for as number | undefined,
334
+ signal,
335
+ });
336
+ screenshot = cap.base64;
337
+ mime = cap.mime;
338
+ size = cap.size;
339
+ details = { mime: cap.mime, size: cap.size };
340
+ } else {
341
+ const config = loadCrawl4aiConfig(params as Record<string, unknown>, cwdFromContext(ctx), includeProjectEnv(ctx));
342
+ try {
343
+ const result = await fetchCrawl4aiScreenshot(
344
+ config,
345
+ url,
346
+ params.wait_for as number | undefined,
347
+ params.wait_for_images as boolean | undefined,
348
+ signal,
349
+ );
350
+ if (result.success === false) {
351
+ throw new Error(String(result.error_message ?? "Crawl4AI screenshot failed"));
352
+ }
353
+ screenshot = result.screenshot as string | undefined;
354
+ artifactUrl = result.url as string | undefined;
355
+ mime = result.mime as string | undefined;
356
+ size = result.size as number | undefined;
357
+ details = { ...result };
358
+ } catch (err) {
359
+ // Daemon can't render this URL (SSRF-blocked); retry via local Chrome.
360
+ if (!isSsrfBlocked(err) || !findChromeBinary()) throw err;
361
+ engine = "local";
362
+ const cap = await captureLocalScreenshot({
363
+ url,
364
+ width: params.width as number | undefined,
365
+ height: params.height as number | undefined,
366
+ fullPage: params.full_page as boolean | undefined,
367
+ waitForSec: params.wait_for as number | undefined,
368
+ signal,
369
+ });
370
+ screenshot = cap.base64;
371
+ mime = cap.mime;
372
+ size = cap.size;
373
+ details = { mime: cap.mime, size: cap.size, fallback: "daemon SSRF-blocked this URL" };
374
+ }
310
375
  }
311
- const screenshot = result.screenshot as string | undefined;
312
- const artifactUrl = result.url as string | undefined;
313
- const mime = result.mime as string | undefined;
314
- const size = result.size as number | undefined;
315
- let text = `Screenshot: ${params.url}\n`;
376
+
377
+ let text = `Screenshot: ${url}\nEngine: ${engine === "local" ? "local-chrome" : "crawl4ai"}\n`;
316
378
  if (artifactUrl) text += `Artifact: ${artifactUrl}\n`;
317
379
  if (mime) text += `MIME: ${mime}\n`;
318
380
  if (size) text += `Size: ${size} bytes\n`;
@@ -321,7 +383,7 @@ export default function piWebExtension(pi: ExtensionAPI) {
321
383
  { type: "text", text: truncateText(text) },
322
384
  ];
323
385
  if (screenshot) content.push({ type: "image", data: screenshot, mimeType: mime || "image/png" });
324
- return { content, details: { ...result, url: params.url } };
386
+ return { content, details: { ...details, url, engine } };
325
387
  },
326
388
  });
327
389
 
@@ -330,25 +392,51 @@ export default function piWebExtension(pi: ExtensionAPI) {
330
392
  name: "web_pdf",
331
393
  label: "Web Page PDF",
332
394
  description:
333
- "PDF document via Crawl4AI.",
395
+ "PDF document via the Crawl4AI daemon, or via local headless Chrome for localhost/private/file URLs (auto-detected, engine overridable).",
334
396
  promptSnippet: "PDF a webpage",
335
- promptGuidelines: ["Printable/archivable page snapshot; returns base64 PDF."],
397
+ promptGuidelines: ["Printable/archivable page snapshot; returns base64 PDF. Local dev servers capture automatically via local Chrome."],
336
398
  parameters: Type.Object({
337
399
  url: Type.String(),
400
+ engine: Type.Optional(engineSchema.engine),
338
401
  ...crawl4aiControlSchema,
339
402
  ...sharedControlSchema,
340
403
  }),
341
404
  async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
342
- const config = loadCrawl4aiConfig(params as Record<string, unknown>, cwdFromContext(ctx), includeProjectEnv(ctx));
343
- const result = await fetchCrawl4aiPdf(config, params.url as string, signal);
344
- const pdf = result.pdf as string | undefined;
345
- const artifactUrl = result.url as string | undefined;
346
- const size = result.size as number | undefined;
347
- let text = `PDF: ${params.url}\n`;
405
+ const url = params.url as string;
406
+ let engine = resolveEngine(params.engine as string | undefined, url);
407
+ let pdf: string | undefined;
408
+ let artifactUrl: string | undefined;
409
+ let size: number | undefined;
410
+ let details: Record<string, unknown> = {};
411
+
412
+ if (engine === "local") {
413
+ const cap = await captureLocalPdf({ url, signal });
414
+ pdf = cap.base64;
415
+ size = cap.size;
416
+ details = { mime: cap.mime, size: cap.size };
417
+ } else {
418
+ const config = loadCrawl4aiConfig(params as Record<string, unknown>, cwdFromContext(ctx), includeProjectEnv(ctx));
419
+ try {
420
+ const result = await fetchCrawl4aiPdf(config, url, signal);
421
+ pdf = result.pdf as string | undefined;
422
+ artifactUrl = result.url as string | undefined;
423
+ size = result.size as number | undefined;
424
+ details = { ...result };
425
+ } catch (err) {
426
+ if (!isSsrfBlocked(err) || !findChromeBinary()) throw err;
427
+ engine = "local";
428
+ const cap = await captureLocalPdf({ url, signal });
429
+ pdf = cap.base64;
430
+ size = cap.size;
431
+ details = { mime: cap.mime, size: cap.size, fallback: "daemon SSRF-blocked this URL" };
432
+ }
433
+ }
434
+
435
+ let text = `PDF: ${url}\nEngine: ${engine === "local" ? "local-chrome" : "crawl4ai"}\n`;
348
436
  if (pdf) text += `Data: base64 PDF (${pdf.length} chars)\n`;
349
437
  if (artifactUrl) text += `Artifact: ${artifactUrl}\n`;
350
438
  if (size) text += `Size: ${size} bytes\n`;
351
- return { content: [{ type: "text" as const, text: truncateText(text) }], details: { ...result, url: params.url } };
439
+ return { content: [{ type: "text" as const, text: truncateText(text) }], details: { ...details, url, engine } };
352
440
  },
353
441
  });
354
442
 
@@ -396,6 +484,7 @@ export default function piWebExtension(pi: ExtensionAPI) {
396
484
  apiTokenSource: c4aiToken.value ? c4aiToken.source : "not set",
397
485
  },
398
486
  agy: { installed: isAgyInstalled() },
487
+ localChrome: { path: findChromeBinary() ?? "not found" },
399
488
  };
400
489
 
401
490
  // Crawl4AI health check
@@ -416,7 +505,7 @@ export default function piWebExtension(pi: ExtensionAPI) {
416
505
  // Inject the backend-selection protocol only when a web_* tool is actually
417
506
  // active, so recon agents / sessions without pi-web carry zero overhead.
418
507
  pi.on("before_agent_start", async (event) => {
419
- const active = event.systemPromptOptions?.selectedTools ?? [];
508
+ const active = event.systemPromptOptions?.selectedTools ?? pi.getActiveTools();
420
509
  if (!active.some((t) => t.startsWith("web_"))) return;
421
510
  return { systemPrompt: `${event.systemPrompt}\n\n${WEB_ROUTING_GUIDANCE}` };
422
511
  });
@@ -0,0 +1,296 @@
1
+ // Local headless Chrome capture — for localhost/private/file:// URLs the
2
+ // remote Crawl4AI daemon cannot reach (its SSRF protection blocks them).
3
+ // Zero dependencies: drives the locally installed Chrome/Chromium binary.
4
+
5
+ import { spawn } from "node:child_process";
6
+ import { existsSync, mkdtempSync, readFileSync, rmSync, statSync } from "node:fs";
7
+ import { tmpdir } from "node:os";
8
+ import path from "node:path";
9
+
10
+ export interface LocalCapture {
11
+ base64: string;
12
+ mime: string;
13
+ size: number;
14
+ }
15
+
16
+ const CHROME_TIMEOUT_MS = 30_000;
17
+ // ponytail: tall-window approximates full page (CLI has no fullPage flag) —
18
+ // Playwright tier if this proves insufficient.
19
+ const FULL_PAGE_HEIGHT = 8000;
20
+
21
+ /** Locate a locally installed Chrome/Chromium (or Edge as a Windows fallback). */
22
+ export function findChromeBinary(): string | null {
23
+ const candidates: string[] = [];
24
+ if (process.env.CHROME_PATH) candidates.push(process.env.CHROME_PATH);
25
+ switch (process.platform) {
26
+ case "darwin": {
27
+ const apps = ["/Applications", path.join(process.env.HOME ?? "", "Applications")];
28
+ for (const app of apps) {
29
+ candidates.push(
30
+ path.join(app, "Google Chrome.app/Contents/MacOS/Google Chrome"),
31
+ path.join(app, "Chromium.app/Contents/MacOS/Chromium"),
32
+ );
33
+ }
34
+ break;
35
+ }
36
+ case "win32": {
37
+ const roots = [
38
+ "C:\\Program Files",
39
+ "C:\\Program Files (x86)",
40
+ process.env.LOCALAPPDATA ?? "",
41
+ ].filter(Boolean);
42
+ for (const root of roots) {
43
+ candidates.push(
44
+ path.join(root, "Google\\Chrome\\Application\\chrome.exe"),
45
+ path.join(root, "Microsoft\\Edge\\Application\\msedge.exe"),
46
+ );
47
+ }
48
+ break;
49
+ }
50
+ default: {
51
+ const dirs = [
52
+ ...(process.env.PATH ?? "").split(":").filter(Boolean),
53
+ "/usr/bin",
54
+ "/usr/local/bin",
55
+ "/snap/bin",
56
+ ];
57
+ for (const dir of dirs) {
58
+ candidates.push(
59
+ path.join(dir, "google-chrome"),
60
+ path.join(dir, "google-chrome-stable"),
61
+ path.join(dir, "chromium"),
62
+ path.join(dir, "chromium-browser"),
63
+ );
64
+ }
65
+ }
66
+ }
67
+ return candidates.find((p) => existsSync(p)) ?? null;
68
+ }
69
+
70
+ /** True for URLs a remote daemon provably cannot render: file://, localhost, loopback, private ranges. */
71
+ export function isLocalUrl(raw: string): boolean {
72
+ let u: URL;
73
+ try {
74
+ u = new URL(raw);
75
+ } catch {
76
+ return false;
77
+ }
78
+ if (u.protocol === "file:") return true;
79
+ const host = u.hostname.toLowerCase().replace(/^\[|\]$/g, "");
80
+ if (host === "localhost" || host.endsWith(".localhost")) return true;
81
+ if (host === "::1") return true;
82
+ if (host.includes(":")) {
83
+ // IPv6 ULA fc00::/7 and link-local fe80::/10 are private too.
84
+ if (/^f[cd]/.test(host) || /^fe[89ab]/.test(host)) return true;
85
+ return false;
86
+ }
87
+ const m = host.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/);
88
+ if (m) {
89
+ const a = Number(m[1]);
90
+ const b = Number(m[2]);
91
+ if (a === 127 || a === 10 || a === 0) return true;
92
+ if (a === 172 && b >= 16 && b <= 31) return true; // 172.16/12
93
+ if (a === 192 && b === 168) return true;
94
+ if (a === 169 && b === 254) return true; // link-local
95
+ }
96
+ return false;
97
+ }
98
+
99
+ /** Daemon SSRF/URL-blocked failures that a local-Chrome retry can rescue. */
100
+ export function isSsrfBlocked(err: unknown): boolean {
101
+ const msg = err instanceof Error ? err.message : String(err);
102
+ return /SSRF|URL blocked/i.test(msg);
103
+ }
104
+
105
+ /** Pick the capture engine: local Chrome or the remote Crawl4AI daemon. */
106
+ export function resolveEngine(engine: string | undefined, url: string): "local" | "daemon" {
107
+ if (engine === "local" || engine === "daemon") return engine;
108
+ return isLocalUrl(url) ? "local" : "daemon";
109
+ }
110
+
111
+ export interface ScreenshotArgsOpts {
112
+ chromePath: string;
113
+ outPath: string;
114
+ userDataDir: string;
115
+ url: string;
116
+ width: number;
117
+ height: number;
118
+ fullPage?: boolean;
119
+ waitForSec?: number;
120
+ }
121
+
122
+ export function buildScreenshotArgs(opts: ScreenshotArgsOpts): string[] {
123
+ const height = opts.fullPage ? FULL_PAGE_HEIGHT : opts.height;
124
+ return [
125
+ opts.chromePath,
126
+ "--headless",
127
+ "--no-first-run",
128
+ "--disable-gpu",
129
+ `--user-data-dir=${opts.userDataDir}`,
130
+ "--hide-scrollbars",
131
+ `--window-size=${opts.width},${height}`,
132
+ ...(opts.waitForSec ? [`--virtual-time-budget=${Math.round(opts.waitForSec * 1000)}`] : []),
133
+ `--screenshot=${opts.outPath}`,
134
+ opts.url,
135
+ ];
136
+ }
137
+
138
+ export function buildPdfArgs(opts: {
139
+ chromePath: string;
140
+ outPath: string;
141
+ userDataDir: string;
142
+ url: string;
143
+ }): string[] {
144
+ return [
145
+ opts.chromePath,
146
+ "--headless",
147
+ "--no-first-run",
148
+ "--disable-gpu",
149
+ `--user-data-dir=${opts.userDataDir}`,
150
+ "--no-pdf-header-footer",
151
+ `--print-to-pdf=${opts.outPath}`,
152
+ opts.url,
153
+ ];
154
+ }
155
+
156
+ function runChrome(
157
+ args: string[],
158
+ outPath: string,
159
+ signal?: AbortSignal,
160
+ timeoutMs: number = CHROME_TIMEOUT_MS,
161
+ ): Promise<void> {
162
+ return new Promise((resolve, reject) => {
163
+ const child = spawn(args[0], args.slice(1), { stdio: ["ignore", "ignore", "pipe"] });
164
+ let stderr = "";
165
+ let settled = false;
166
+ let lastSize = -1;
167
+ let stablePolls = 0;
168
+ let fileTimer: ReturnType<typeof setInterval> | undefined;
169
+ const finish = (err?: Error) => {
170
+ if (settled) return;
171
+ settled = true;
172
+ clearTimeout(killTimer);
173
+ clearInterval(fileTimer);
174
+ signal?.removeEventListener("abort", onAbort);
175
+ if (err) reject(err);
176
+ else resolve();
177
+ };
178
+ const onAbort = () => {
179
+ child.kill("SIGKILL");
180
+ finish(new Error("Local capture aborted"));
181
+ };
182
+ const killTimer = setTimeout(() => {
183
+ child.kill("SIGKILL");
184
+ // Always a failure: a complete capture is caught earlier by the
185
+ // size-stability poll, so surviving to the timeout means the output
186
+ // never settled (or never appeared) — a file here may be mid-write.
187
+ finish(new Error(`Local capture timed out after ${timeoutMs / 1000}s`));
188
+ }, timeoutMs);
189
+ signal?.addEventListener("abort", onAbort, { once: true });
190
+ child.stderr?.on("data", (d: Buffer) => {
191
+ stderr += d.toString();
192
+ });
193
+ child.on("error", (err) => finish(err));
194
+ child.on("close", (code) => {
195
+ if (existsSync(outPath)) finish();
196
+ else finish(new Error(`Chrome exited with code ${code}: ${stderr.slice(-400)}`));
197
+ });
198
+ // Resolve as soon as the capture file is written and stable, then kill —
199
+ // don't require a clean Chrome exit (some versions hang after writing,
200
+ // e.g. fresh --user-data-dir on macOS).
201
+ fileTimer = setInterval(() => {
202
+ if (!existsSync(outPath)) return;
203
+ const size = statSync(outPath).size;
204
+ if (size > 0 && size === lastSize) {
205
+ if (++stablePolls >= 2) {
206
+ child.kill("SIGKILL");
207
+ finish();
208
+ }
209
+ } else {
210
+ stablePolls = 0;
211
+ lastSize = size;
212
+ }
213
+ }, 250);
214
+ });
215
+ }
216
+
217
+ async function readCapture(outPath: string, mime: string): Promise<LocalCapture> {
218
+ const buf = readFileSync(outPath);
219
+ return { base64: buf.toString("base64"), mime, size: buf.length };
220
+ }
221
+
222
+ function assertCaptureUrl(url: string): void {
223
+ // Trust boundary: the URL becomes a spawn argv element — a scheme check
224
+ // keeps strings like "--proxy-server=http://evil" from parsing as switches.
225
+ if (!/^https?:\/\//i.test(url) && !/^file:\/\//i.test(url)) {
226
+ throw new Error(`Invalid capture URL (${url.slice(0, 80)}): must be http://, https://, or file://`);
227
+ }
228
+ }
229
+
230
+ export async function captureScreenshot(opts: {
231
+ url: string;
232
+ width?: number;
233
+ height?: number;
234
+ fullPage?: boolean;
235
+ waitForSec?: number;
236
+ signal?: AbortSignal;
237
+ timeoutMs?: number;
238
+ }): Promise<LocalCapture> {
239
+ assertCaptureUrl(opts.url);
240
+ const chromePath = findChromeBinary();
241
+ if (!chromePath) {
242
+ throw new Error("No local Chrome/Chromium found — install Chrome or set CHROME_PATH.");
243
+ }
244
+ const dir = mkdtempSync(path.join(tmpdir(), "pi-web-capture-"));
245
+ try {
246
+ const outPath = path.join(dir, "screenshot.png");
247
+ await runChrome(
248
+ buildScreenshotArgs({
249
+ chromePath,
250
+ outPath,
251
+ userDataDir: path.join(dir, "profile"),
252
+ url: opts.url,
253
+ width: opts.width ?? 1280,
254
+ height: opts.height ?? 800,
255
+ fullPage: opts.fullPage,
256
+ waitForSec: opts.waitForSec,
257
+ }),
258
+ outPath,
259
+ opts.signal,
260
+ opts.timeoutMs,
261
+ );
262
+ return await readCapture(outPath, "image/png");
263
+ } finally {
264
+ rmSync(dir, { recursive: true, force: true });
265
+ }
266
+ }
267
+
268
+ export async function capturePdf(opts: {
269
+ url: string;
270
+ signal?: AbortSignal;
271
+ timeoutMs?: number;
272
+ }): Promise<LocalCapture> {
273
+ assertCaptureUrl(opts.url);
274
+ const chromePath = findChromeBinary();
275
+ if (!chromePath) {
276
+ throw new Error("No local Chrome/Chromium found — install Chrome or set CHROME_PATH.");
277
+ }
278
+ const dir = mkdtempSync(path.join(tmpdir(), "pi-web-capture-"));
279
+ try {
280
+ const outPath = path.join(dir, "page.pdf");
281
+ await runChrome(
282
+ buildPdfArgs({
283
+ chromePath,
284
+ outPath,
285
+ userDataDir: path.join(dir, "profile"),
286
+ url: opts.url,
287
+ }),
288
+ outPath,
289
+ opts.signal,
290
+ opts.timeoutMs,
291
+ );
292
+ return await readCapture(outPath, "application/pdf");
293
+ } finally {
294
+ rmSync(dir, { recursive: true, force: true });
295
+ }
296
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@bacnh85/pi-web",
3
- "version": "0.6.2",
3
+ "version": "0.7.1",
4
4
  "description": "Pi extension for web search, page extraction, Firecrawl scraping/crawling, and Crawl4AI headless browser crawling.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -15,8 +15,8 @@ Use the **7 unified tools** from the `pi-web` extension for all web-related task
15
15
  | `web_extract` | Extract readable content from a URL | Static (JSDOM) → Dynamic (Firecrawl) → Full (Crawl4AI) → agy (model-backed) |
16
16
  | `web_map` | Discover URLs from a site | Firecrawl Map (only option) |
17
17
  | `web_crawl` | Crawl multiple pages from a site | Light (Firecrawl) or Full (Crawl4AI) |
18
- | `web_screenshot` | Capture page screenshot as PNG | Crawl4AI (only option) |
19
- | `web_pdf` | Generate page PDF | Crawl4AI (only option) |
18
+ | `web_screenshot` | Capture page screenshot as PNG | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
19
+ | `web_pdf` | Generate page PDF | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
20
20
  | `web_status` | Check provider configuration and health | — |
21
21
 
22
22
  ## Decision Tree