@bacnh85/pi-web 0.7.1 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,52 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.0 (2026-09-13)
4
+
5
+ ### Added
6
+
7
+ - **`web_image` tool** — image generation from text via free upstream
8
+ providers, all direct-to-upstream (no self-host services), with automatic
9
+ fallback: `gemini` (gemini.google.com web tier via `gemini-reverse`, guest
10
+ or cookie auth) → `zai` (official `api.z.ai` CogView-4 via `ZAI_API_KEY`)
11
+ → `custom` (any OpenAI-compatible `/images/generations` endpoint via
12
+ `WEB_IMAGE_API_BASE_URL`). `provider: "auto"` walks the chain and the
13
+ result reports every fallback attempt.
14
+ - Results are saved as files (`out_dir`, default fresh temp dir) and returned
15
+ as **inline image blocks** so multimodal models see their own output;
16
+ `model`/`n` (1–4) parameters per call.
17
+ - **Soft ToS guardrails** — per-provider `WEB_IMAGE_MIN_INTERVAL_MS`
18
+ (default 5000) and a `WEB_IMAGE_DAILY_CAP` (default 20/day) on the Gemini
19
+ web tier; successes-only counting, UTC-day reset, usage surfaced in
20
+ `web_status.imageProviders.rate`.
21
+ - `web_status` now reports `imageProviders` (gemini/zai/custom config + rate
22
+ snapshot).
23
+ - Smoke script: `image` and `zai` modes (prints saved paths + PNG magic-byte
24
+ check); ask/research modes now print **full answers and source URLs**
25
+ (previously sliced to a 200/400-char preview).
26
+
27
+ ## 0.8.0 (2026-09-13)
28
+
29
+ ### Added
30
+
31
+ - **`web_research` tool** — AI-synthesized web research via Gemini's web tier
32
+ (gemini.google.com), cookie-authed with `__Secure-1PSID`.
33
+ `mode: "ask"` returns a quick grounded answer with extracted source links
34
+ (works in guest mode without any cookie, Flash-only);
35
+ `mode: "research"` runs Gemini **Deep Research** — an autonomous agent that
36
+ browses the web for minutes and returns a comprehensive report (requires the
37
+ cookie and a Gemini Advanced subscription; default timeout 10 min, cap 30).
38
+ - New `lib/gemini.ts` wrapper over the `gemini-reverse` npm package (lazy
39
+ dynamic import, injectable client for tests, one AuthError retry that
40
+ re-absorbs rotated Set-Cookies). Sources are extracted from markdown links
41
+ in the answer/report text (the web protocol exposes no structured citations).
42
+ - `web_status` now reports `geminiWeb` (configured/cookieSource/proxy).
43
+ - Env config: `GEMINI_WEB_SECURE_1PSID` (required for authed/research mode),
44
+ optional `GEMINI_WEB_PROXY` (escape hatch if Google blocks the IP).
45
+ - **Header-cap fix (authed mode)** — Google ships ~25 KB of response headers on
46
+ Gemini pages (giant `content-security-policy`), over Node's default 16 KB
47
+ parser cap; the wrapper injects a per-request `maxHeaderSize` for
48
+ gemini.google.com hosts only (lazy, no global flag needed).
49
+
3
50
  ## 0.7.1 (2026-09-12)
4
51
 
5
52
  ### Fixed
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # @bacnh85/pi-web
2
2
 
3
- Pi extension for **unified web search, content extraction, site crawling, and page capture**.
3
+ Pi extension for **unified web search, content extraction, site crawling, page capture, Gemini web-tier research, and free upstream image generation**.
4
4
 
5
- Auto-selects the best backend from SearXNG (self-hosted), Brave Search, Firecrawl, Crawl4AI, and agy (Gemini/Claude, when installed) — so agents don't have to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive searches and inline content prefer Brave.
5
+ Auto-selects the best backend from SearXNG (self-hosted), Brave Search, Firecrawl, Crawl4AI, and agy (Gemini/Claude, when installed) — so agents don't have to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive searches and inline content prefer Brave. `web_research` adds AI-synthesized research with citations via your gemini.google.com session.
6
6
 
7
7
  ## Install
8
8
 
@@ -30,10 +30,20 @@ Variables:
30
30
  | `FIRECRAWL_API_KEY` | No (2) | — | Required for hosted Firecrawl |
31
31
  | `CRAWL4AI_API_URL` | No | `http://127.0.0.1:11235` | Self-hosted Crawl4AI |
32
32
  | `CRAWL4AI_API_TOKEN` | No (3) | — | Required if Crawl4AI auth enabled |
33
+ | `GEMINI_WEB_SECURE_1PSID` | No (4) | — | `__Secure-1PSID` cookie from gemini.google.com — enables authed `web_research` (Deep Research) |
34
+ | `GEMINI_WEB_PROXY` | No | — | Proxy URL for Gemini web calls (escape hatch if Google blocks the IP) |
35
+ | `ZAI_API_KEY` | No (5) | — | Z.ai API key — enables the `web_image` `zai` provider (CogView-4 via the official `api.z.ai`); `Z_AI_API_KEY` also accepted |
36
+ | `WEB_IMAGE_API_BASE_URL` | No | — | `web_image` `custom` provider: any OpenAI-compatible images endpoint (e.g. `https://api.openai.com/v1`) |
37
+ | `WEB_IMAGE_API_KEY` | No | — | Bearer key for the `custom` endpoint |
38
+ | `WEB_IMAGE_API_LABEL` | No | — | Display label for the `custom` endpoint (default: host name) |
39
+ | `WEB_IMAGE_MIN_INTERVAL_MS` | No | `5000` | Min interval between `web_image` calls per provider |
40
+ | `WEB_IMAGE_DAILY_CAP` | No | `20` | Daily soft cap for the Gemini **web tier** `web_image` provider (keyed APIs stay uncapped) |
33
41
 
34
42
  > (1) At least one search backend (SearXNG, Brave, or Firecrawl) must be configured for `web_search`.
35
43
  > (2) Required for hosted Firecrawl; optional for self-hosted instances without auth.
36
44
  > (3) Required for Crawl4AI v0.9+ default config.
45
+ > (4) Without it `web_research mode=ask` still works in guest mode (Flash-only); `mode=research` errors with setup steps.
46
+ > (5) `web_image` works with zero config via Gemini guest mode (availability varies by region/account); `zai` activates when `ZAI_API_KEY` is present, `custom` when `WEB_IMAGE_API_BASE_URL` is set.
37
47
 
38
48
  Secrets are never printed; `web_status` reports only presence/source.
39
49
 
@@ -197,10 +207,105 @@ Typical output:
197
207
  "health": { "status": "healthy", "version": "0.5.0", ... }
198
208
  },
199
209
  "agy": { "installed": true },
210
+ "geminiWeb": { "configured": true, "cookieSource": "process.env", "proxy": false },
200
211
  "localChrome": { "path": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" }
201
212
  }
202
213
  ```
203
214
 
215
+ ### `web_research` — Gemini web-tier research
216
+
217
+ AI-synthesized research through your gemini.google.com session (powered by the
218
+ [`gemini-reverse`](https://github.com/rynn-k/Gemini-Reverse) client, lazy-loaded).
219
+
220
+ ```
221
+ web_research(query="compare the top 3 cloud providers' AI offerings", mode="research", timeout_ms=600000)
222
+ ```
223
+
224
+ - **`mode: "ask"`** (default) — quick grounded answer (Gemini auto-grounds with
225
+ Google Search). Works **without any cookie** in guest mode (Flash-only).
226
+ Sent as a temporary chat so your Gemini history stays clean.
227
+ - **`mode: "research"`** — full Gemini **Deep Research**: plan → autonomous web
228
+ browsing (minutes) → cited report. Requires the cookie **and a Gemini
229
+ Advanced subscription** on the account. Default timeout 600 s, cap 1 800 000.
230
+
231
+ Both modes return the text plus **Sources** — URLs extracted from the
232
+ answer/report markdown (the web protocol exposes no structured citations field).
233
+
234
+ Setup (authed mode):
235
+
236
+ 1. Sign in at [gemini.google.com](https://gemini.google.com/).
237
+ 2. `F12` → **Application** → **Cookies** → `https://gemini.google.com`.
238
+ 3. Copy the `__Secure-1PSID` value into `~/.pi/agent/.env.local`:
239
+
240
+ ```bash
241
+ GEMINI_WEB_SECURE_1PSID=your-cookie-value
242
+ # optional, if Google blocks your IP:
243
+ # GEMINI_WEB_PROXY=http://host:port
244
+ ```
245
+ 4. Restart pi; `web_status` shows `geminiWeb.configured: true`.
246
+
247
+ Live verification script (also proves the header-cap patch end-to-end — an
248
+ authed failure would surface `HPE_HEADER_OVERFLOW`):
249
+
250
+ ```bash
251
+ npx tsx extensions/scripts/gemini-smoke.ts "test query" # ask (authed or guest)
252
+ npx tsx extensions/scripts/gemini-smoke.ts "topic" research # Deep Research
253
+ ```
254
+
255
+ ⚠️ **Unofficial, at your own risk.** Cookie auth uses your real Google session
256
+ against gemini.google.com's internal web API and may not comply with Google's
257
+ ToS; the protocol can break when Google changes it. `ask` mode errors map to
258
+ actionable steps (expired cookie → re-copy; IP block → set `GEMINI_WEB_PROXY`).
259
+
260
+ Troubleshooting:
261
+
262
+ - *"cookie expired or invalid"* — re-copy `__Secure-1PSID` (it rotates).
263
+ - *"temporarily blocked this IP"* — set `GEMINI_WEB_PROXY`.
264
+ - *research mode: "Unknown API error: 1184"* — on this account Deep Research
265
+ was rejected; usually means no Gemini Advanced subscription on the account
266
+ (Deep Research is Advanced-only), or Google changed the protocol. `ask`
267
+ mode is unaffected.
268
+
269
+ ### `web_image` — free upstream image generation
270
+
271
+ Text → image with automatic provider fallback (all direct-to-upstream, no
272
+ self-host services):
273
+
274
+ ```
275
+ web_image(prompt="isometric cutaway of a container ship, technical illustration")
276
+ web_image(prompt="...", provider="zai") # pin CogView-4 via api.z.ai
277
+ web_image(prompt="...", model="cogview-4", n=2, out_dir="/tmp/imgs")
278
+ ```
279
+
280
+ **Provider chain** (`provider: "auto"` tries in order; pin one to skip):
281
+
282
+ | Provider | Upstream | Auth | Notes |
283
+ |---|---|---|---|
284
+ | `gemini` (default) | gemini.google.com web tier | none (guest) or `GEMINI_WEB_SECURE_1PSID` | free ≈ 20 images/day; availability varies by region/account |
285
+ | `zai` | `https://api.z.ai/api/paas/v4` (official API) | `ZAI_API_KEY` | CogView-4 (`model` default), fully ToS-compliant |
286
+ | `custom` | any OpenAI-compatible `/images/generations` endpoint | `WEB_IMAGE_API_KEY` | e.g. official OpenAI `https://api.openai.com/v1` |
287
+
288
+ Results are saved to `out_dir` (default: fresh temp dir) and returned as file
289
+ paths **plus inline image blocks** (multimodal models see the render
290
+ immediately). `details` reports the winning provider, model, and fallback
291
+ attempts.
292
+
293
+ **Guardrails** (soft, in-memory): per-provider `WEB_IMAGE_MIN_INTERVAL_MS`
294
+ (default 5 s) and a `WEB_IMAGE_DAILY_CAP` (default 20/day, applied to the
295
+ Gemini web tier only — keyed APIs are billed upstream and stay uncapped).
296
+ Counters reset on restart; `web_status.imageProviders.rate` shows usage.
297
+
298
+ ⚠️ **ToS reality (read once)**: *every* AI chatbot's terms prohibit automated
299
+ access to its web UI (Google, OpenAI, xAI "unauthorized automated or
300
+ non-human means", Z.ai alike). This tool therefore follows a risk ladder:
301
+
302
+ 1. **Official APIs** (`zai`, `custom`) — fully compliant; prefer them when a key exists.
303
+ 2. **Guest mode** (Gemini without a cookie) — no account at stake, lowest risk, Flash-tier.
304
+ 3. **Personal cookie** (Gemini authed) — your own account, single session, low volume; same accepted-risk stance as the web bridges: use a burner/low-value account, never a valued one. No account pools, no commercial use, keep volume human-scale.
305
+
306
+ Smoke test: `npx tsx extensions/scripts/gemini-smoke.ts "a red cube on white background" image`
307
+ (or `… zai` for the Z.ai path).
308
+
204
309
  ## Library structure
205
310
 
206
311
  | Module | Contents |
@@ -1,5 +1,9 @@
1
1
  /// <reference path="./types.d.ts" />
2
2
 
3
+ import fs from "node:fs";
4
+ import os from "node:os";
5
+ import path from "node:path";
6
+
3
7
  import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
4
8
  import { Type } from "typebox";
5
9
 
@@ -21,6 +25,19 @@ import {
21
25
  formatUnifiedSearchResults,
22
26
  } from "./lib/format";
23
27
  import { searchWithDiagnostics } from "./lib/search";
28
+ import {
29
+ loadGeminiWebConfig,
30
+ geminiAsk,
31
+ geminiResearch,
32
+ describeGeminiError,
33
+ } from "./lib/gemini";
34
+ import {
35
+ generateImageWithFallback,
36
+ loadImageApiConfig,
37
+ loadImageRateConfig,
38
+ imageRateSnapshot,
39
+ type ImageProvider,
40
+ } from "./lib/imageapi";
24
41
  import { extractWithDiagnostics, type ExtractMode } from "./lib/extract";
25
42
  import { firecrawlRequest, type FirecrawlResult } from "./lib/firecrawl";
26
43
  import {
@@ -63,6 +80,19 @@ const engineSchema = {
63
80
  ], { default: "auto", description: "auto routes localhost/private/file URLs to local Chrome, the rest to the Crawl4AI daemon; local/daemon force one." })),
64
81
  };
65
82
 
83
+ // Saved image file → inline image block (the 0.6.2 vision-loop lesson: the
84
+ // generating model should see its own output).
85
+ async function toImageBlock(file: string): Promise<{ type: "image"; data: string; mimeType: string }> {
86
+ const data = (await fs.promises.readFile(file)).toString("base64");
87
+ const lower = file.toLowerCase();
88
+ const mimeType = lower.endsWith(".jpg") || lower.endsWith(".jpeg")
89
+ ? "image/jpeg"
90
+ : lower.endsWith(".webp")
91
+ ? "image/webp"
92
+ : "image/png";
93
+ return { type: "image" as const, data, mimeType };
94
+ }
95
+
66
96
  // ---------------------------------------------------------------------------
67
97
  // Always-on routing guidance (injected only when a web_* tool is active)
68
98
  // ---------------------------------------------------------------------------
@@ -77,6 +107,8 @@ const WEB_ROUTING_GUIDANCE = `## Web Tool Routing (pi-web)
77
107
  - **web_map** — discover site URLs (Firecrawl Map).
78
108
  - **web_crawl** — multi-page crawl: \`mode: "light"\` (Firecrawl, url) or \`mode: "full"\` (Crawl4AI, urls[]).
79
109
  - **web_screenshot** / **web_pdf** — page capture (Crawl4AI).
110
+ - **web_research** — AI-synthesized research via Gemini web (mode "ask" = grounded answer, guest OK; mode "research" = Deep Research report, needs cookie + Gemini Advanced, takes minutes).
111
+ - **web_image** — text→image generation via free upstreams (auto: Gemini web → Z.ai CogView-4 → custom OpenAI-images endpoint; \`model\`/\`n\` params).
80
112
  - **web_status** — provider config + health.
81
113
 
82
114
  Rules: Firecrawl Search is weak on domain-specific queries — prefer SearXNG/Brave; Firecrawl Scrape fails on bot-protected sites — use Crawl4AI (\`mode: "full"\`) then agy (\`mode: "agy"\`); cite source URLs.`;
@@ -440,6 +472,115 @@ export default function piWebExtension(pi: ExtensionAPI) {
440
472
  },
441
473
  });
442
474
 
475
+ // ── web_research ─────────────────────────────────────────────────────
476
+ pi.registerTool({
477
+ name: "web_research",
478
+ label: "Web Research (Gemini)",
479
+ description:
480
+ "AI-synthesized web research via Gemini (gemini.google.com web tier, cookie auth). Mode 'ask' returns a quick grounded answer with source links (works guest-mode, Flash only). Mode 'research' runs Gemini Deep Research — an autonomous agent browses the web for minutes and returns a comprehensive cited report (requires GEMINI_WEB_SECURE_1PSID cookie and a Gemini Advanced subscription).",
481
+ promptSnippet: "AI-synthesized research with citations",
482
+ promptGuidelines: [
483
+ "Use for AI-synthesized research with sources (mode ask = quick grounded answer; mode research = multi-minute Deep Research report). NOT for URL-list searches (web_search) or single-URL extraction (web_extract). Cite the returned source URLs.",
484
+ ],
485
+ parameters: Type.Object({
486
+ query: Type.String({ description: "Research question or topic." }),
487
+ mode: Type.Optional(Type.Union(
488
+ [Type.Literal("ask"), Type.Literal("research")],
489
+ { default: "ask", description: "ask = quick grounded answer (guest OK); research = full Deep Research report (cookie + Gemini Advanced required, takes minutes)." },
490
+ )),
491
+ model: Type.Optional(Type.String({ description: "Gemini model for ask mode (e.g. gemini-3-flash). Discovered from the account by default." })),
492
+ ...sharedControlSchema,
493
+ }),
494
+ async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
495
+ const config = loadGeminiWebConfig(cwdFromContext(ctx), includeProjectEnv(ctx));
496
+ const mode = (params.mode as string) || "ask";
497
+ const query = params.query as string;
498
+ try {
499
+ if (mode === "research") {
500
+ const timeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 600_000, 30_000), 1_800_000);
501
+ const result = await geminiResearch(query, { config, timeoutMs, signal });
502
+ const meta = [
503
+ "Mode: research (Gemini Deep Research)",
504
+ result.title ? `Title: ${result.title}` : null,
505
+ result.eta ? `ETA: ${result.eta}` : null,
506
+ ].filter(Boolean).join("\n");
507
+ const sources = result.sources.length ? result.sources.map((s, i) => `${i + 1}. ${s}`).join("\n") : "(none found in report text)";
508
+ const text = `${meta}\n\n${result.text}\n\n--- Sources (extracted from report) ---\n${sources}`;
509
+ return { content: [{ type: "text" as const, text: truncateText(text) }], details: { mode, ...result } };
510
+ }
511
+ const askTimeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 120_000, 30_000), 600_000);
512
+ const result = await geminiAsk(query, { config, model: params.model as string | undefined, timeoutMs: askTimeoutMs, signal });
513
+ const meta = [
514
+ "Mode: ask",
515
+ `Model: ${result.model ?? "unknown"}`,
516
+ result.guest ? "Guest mode (no cookie — Flash only; set GEMINI_WEB_SECURE_1PSID for full access)" : "Cookie auth",
517
+ ].join("\n");
518
+ const sources = result.sources.length ? result.sources.map((s, i) => `${i + 1}. ${s}`).join("\n") : "(none found in answer text)";
519
+ const text = `${meta}\n\n${result.text}\n\n--- Sources (extracted from answer) ---\n${sources}`;
520
+ return { content: [{ type: "text" as const, text: truncateText(text) }], details: { mode, ...result } };
521
+ } catch (err) {
522
+ throw new Error(describeGeminiError(err));
523
+ }
524
+ },
525
+ });
526
+
527
+ // ── web_image ────────────────────────────────────────────────────────
528
+ pi.registerTool({
529
+ name: "web_image",
530
+ label: "Web Image Generation",
531
+ description:
532
+ "Generate images from text via free upstream providers, with fallback: Gemini web (gemini.google.com, guest or cookie auth), Z.ai official API (CogView-4 via ZAI_API_KEY), or any custom OpenAI-compatible images endpoint (WEB_IMAGE_API_BASE_URL). Returns saved file paths plus the images inline.",
533
+ promptSnippet: "Generate images via free upstreams (Gemini web, Z.ai CogView)",
534
+ promptGuidelines: [
535
+ "Use for image GENERATION from a text prompt. provider auto falls back gemini → zai → custom. Capturing an EXISTING page is web_screenshot, not this.",
536
+ ],
537
+ parameters: Type.Object({
538
+ prompt: Type.String({ description: "Image description." }),
539
+ provider: Type.Optional(Type.Union(
540
+ [Type.Literal("auto"), Type.Literal("gemini"), Type.Literal("zai"), Type.Literal("custom")],
541
+ { default: "auto", description: "auto = gemini → zai (if ZAI_API_KEY) → custom (if WEB_IMAGE_API_BASE_URL); pin one to skip fallback." },
542
+ )),
543
+ model: Type.Optional(Type.String({ description: "Provider-specific model (e.g. cogview-4, or a Gemini image-capable model id). Omit for the provider default." })),
544
+ n: Type.Optional(Type.Number({ default: 1, description: "Number of images, 1-4." })),
545
+ out_dir: Type.Optional(Type.String({ description: "Directory for saved images (default: fresh temp dir)." })),
546
+ ...sharedControlSchema,
547
+ }),
548
+ async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
549
+ const cwd = cwdFromContext(ctx);
550
+ const trusted = includeProjectEnv(ctx);
551
+ const prompt = params.prompt as string;
552
+ const n = Math.min(Math.max(Math.trunc((params.n as number) ?? 1) || 1, 1), 4);
553
+ const timeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 180_000, 10_000), 600_000);
554
+ const outDir = params.out_dir
555
+ ? path.resolve(String(params.out_dir))
556
+ : await fs.promises.mkdtemp(path.join(os.tmpdir(), "pi-web-image-"));
557
+ const result = await generateImageWithFallback({
558
+ prompt,
559
+ model: params.model as string | undefined,
560
+ n,
561
+ outDir,
562
+ provider: (params.provider as "auto" | ImageProvider) ?? "auto",
563
+ geminiConfig: loadGeminiWebConfig(cwd, trusted),
564
+ apiConfig: loadImageApiConfig(cwd, trusted),
565
+ rateConfig: loadImageRateConfig(cwd, trusted),
566
+ timeoutMs,
567
+ signal,
568
+ });
569
+ const blocks = await Promise.all(result.paths.map(toImageBlock));
570
+ const text = [
571
+ `Provider: ${result.provider}${result.model ? ` (${result.model})` : ""}`,
572
+ `Saved: ${result.paths.length} image(s)`,
573
+ ...result.paths.map((p) => ` ${p}`),
574
+ result.attempts.length ? `Fallback attempts: ${result.attempts.join(" | ")}` : null,
575
+ ].filter(Boolean).join("\n");
576
+ const content: Array<{ type: "text"; text: string } | { type: "image"; data: string; mimeType: string }> = [
577
+ { type: "text" as const, text },
578
+ ...blocks,
579
+ ];
580
+ return { content, details: { provider: result.provider, model: result.model, paths: result.paths, attempts: result.attempts } };
581
+ },
582
+ });
583
+
443
584
  // ── web_status ───────────────────────────────────────────────────────
444
585
  pi.registerTool({
445
586
  name: "web_status",
@@ -460,6 +601,8 @@ export default function piWebExtension(pi: ExtensionAPI) {
460
601
  const fireUrl = findEnvValue("FIRECRAWL_API_URL", cwd, trusted);
461
602
  const c4aiUrl = findEnvValue("CRAWL4AI_API_URL", cwd, trusted);
462
603
  const c4aiToken = findEnvValue("CRAWL4AI_API_TOKEN", cwd, trusted);
604
+ const geminiCfg = loadGeminiWebConfig(cwd, trusted);
605
+ const imageApiCfg = loadImageApiConfig(cwd, trusted);
463
606
 
464
607
  const { isAgyInstalled } = await import("./lib/agy");
465
608
 
@@ -484,6 +627,15 @@ export default function piWebExtension(pi: ExtensionAPI) {
484
627
  apiTokenSource: c4aiToken.value ? c4aiToken.source : "not set",
485
628
  },
486
629
  agy: { installed: isAgyInstalled() },
630
+ geminiWeb: { configured: Boolean(geminiCfg.psid), cookieSource: geminiCfg.psidSource, proxy: Boolean(geminiCfg.proxy) },
631
+ imageProviders: {
632
+ gemini: { configured: Boolean(geminiCfg.psid), guestPossible: true },
633
+ zai: { configured: Boolean(imageApiCfg.zai) },
634
+ custom: imageApiCfg.custom
635
+ ? { configured: true, label: imageApiCfg.custom.label }
636
+ : { configured: false },
637
+ rate: imageRateSnapshot(),
638
+ },
487
639
  localChrome: { path: findChromeBinary() ?? "not found" },
488
640
  };
489
641
 
@@ -0,0 +1,407 @@
1
+ // Gemini web-tier (gemini.google.com) research backend, cookie-authed via the
2
+ // __Secure-1PSID cookie. Thin wrapper over the `gemini-reverse` npm package
3
+ // (CJS), lazily dynamic-imported so pi startup pays zero cost when unused.
4
+
5
+ import http from "node:http";
6
+ import https from "node:https";
7
+ import { urlToHttpOptions } from "node:url";
8
+ import { findEnvValue } from "./config";
9
+
10
+ // ---------------------------------------------------------------------------
11
+ // Config
12
+ // ---------------------------------------------------------------------------
13
+
14
+ export interface GeminiWebConfig {
15
+ psid?: string;
16
+ psidSource: string;
17
+ proxy?: string;
18
+ }
19
+
20
+ export function loadGeminiWebConfig(cwd = process.cwd(), includeCwdEnv = false): GeminiWebConfig {
21
+ const psid = findEnvValue("GEMINI_WEB_SECURE_1PSID", cwd, includeCwdEnv);
22
+ const proxy = findEnvValue("GEMINI_WEB_PROXY", cwd, includeCwdEnv);
23
+ return { psid: psid.value, psidSource: psid.value ? psid.source : "not set", proxy: proxy.value };
24
+ }
25
+
26
+ // ---------------------------------------------------------------------------
27
+ // Minimal structural view of the gemini-reverse client (keeps tests injectable
28
+ // and decouples us from upstream type drift).
29
+ // ---------------------------------------------------------------------------
30
+
31
+ export interface GeminiOutputLike {
32
+ text?: string | null;
33
+ model?: string;
34
+ candidates?: Array<{ text?: string | null }>;
35
+ }
36
+
37
+ export interface GeminiResearchLike {
38
+ text?: string | null;
39
+ plan?: { title?: string | null; eta_text?: string | null };
40
+ statuses?: Array<Record<string, unknown>>;
41
+ }
42
+
43
+ export interface GeminiImageLike {
44
+ save(opts?: { path?: string; filename?: string }): Promise<string>;
45
+ url?: string;
46
+ alt?: string;
47
+ }
48
+
49
+ export interface GeminiImageOutputLike {
50
+ text?: string | null;
51
+ model?: string;
52
+ images?: GeminiImageLike[];
53
+ generated_images?: GeminiImageLike[];
54
+ }
55
+
56
+ export interface GeminiClientLike {
57
+ ask(prompt: string, opts?: Record<string, unknown>): Promise<GeminiOutputLike>;
58
+ research(
59
+ prompt: string,
60
+ opts?: { wait?: boolean; pollInterval?: number; timeout?: number; onStatus?: (s: Record<string, unknown>) => void },
61
+ ): Promise<GeminiResearchLike>;
62
+ // Optional so pre-0.9 fake clients (ask+research only) keep compiling.
63
+ newChat?(opts?: { model?: string }): {
64
+ generateContent(o: { prompt: string }): Promise<GeminiImageOutputLike>;
65
+ };
66
+ }
67
+
68
+ export type GeminiClientFactory = (
69
+ opts: { secure_1psid?: string; proxy?: string },
70
+ ) => GeminiClientLike | Promise<GeminiClientLike>;
71
+
72
+ // Cached per config (psid|proxy) so a config change re-creates the client.
73
+ let cached: { key: string; client: GeminiClientLike } | null = null;
74
+
75
+ // Google ships ~25KB of response headers on every Gemini page (a 19.7KB
76
+ // content-security-policy + 3.9KB reporting-endpoints) — over Node's default
77
+ // 16KB parser cap, so the http parser rejects them (HPE_HEADER_OVERFLOW; the
78
+ // same cap exists in undici, but gemini-reverse uses axios's node http adapter).
79
+ // Node honors a per-request maxHeaderSize override, but axios doesn't forward
80
+ // it — so lazily patch http.request/https.request to inject it for
81
+ // gemini.google.com hosts only.
82
+ // ponytail: process-wide patch, scoped to one hostname; if it ever misbehaves,
83
+ // revert to launching pi with NODE_OPTIONS=--max-http-header-size=262144.
84
+
85
+ /**
86
+ * Returns the request options to pass through with the cap injected when the
87
+ * target host is gemini.google.com, or null when the call must pass through
88
+ * untouched. Normalizes all http.request input forms (options object, string,
89
+ * URL) — string/URL forms become a fresh options object.
90
+ *
91
+ * @internal exported for tests
92
+ */
93
+ export function injectGeminiHeaderCap(options: unknown): Record<string, unknown> | null {
94
+ let opts: Record<string, unknown>;
95
+ if (typeof options === "string") {
96
+ opts = urlToHttpOptions(new URL(options)) as Record<string, unknown>;
97
+ } else if (options instanceof URL) {
98
+ opts = urlToHttpOptions(options) as Record<string, unknown>;
99
+ } else if (options && typeof options === "object") {
100
+ opts = options as Record<string, unknown>;
101
+ } else {
102
+ return null;
103
+ }
104
+ const host = String(opts.hostname ?? opts.host ?? "").split(":")[0];
105
+ if (host !== "gemini.google.com" || opts.maxHeaderSize) return null;
106
+ opts.maxHeaderSize = 256 * 1024;
107
+ return opts;
108
+ }
109
+
110
+ let headerCapPatched = false;
111
+ function patchHeaderCap(): void {
112
+ if (headerCapPatched) return;
113
+ headerCapPatched = true;
114
+ for (const mod of [http, https]) {
115
+ const real = mod.request as unknown as (...args: unknown[]) => unknown;
116
+ const patched = function (this: unknown, options: unknown, ...rest: unknown[]) {
117
+ try {
118
+ const override = injectGeminiHeaderCap(options);
119
+ if (override) return real.call(this, override, ...rest);
120
+ } catch { /* malformed input — let the real request surface the error */ }
121
+ return real.call(this, options, ...rest);
122
+ } as typeof mod.request;
123
+ mod.request = patched;
124
+ }
125
+ }
126
+
127
+ /** @internal exported for tests — resolves the real gemini-reverse module */
128
+ export async function loadDefaultFactory(): Promise<GeminiClientFactory> {
129
+ patchHeaderCap();
130
+ const mod = (await import("gemini-reverse")) as unknown as Record<string, unknown>;
131
+ // CJS interop: named export usually works, but fall back to default.Gemini.
132
+ const Gemini = (mod.Gemini ?? (mod as { default?: Record<string, unknown> }).default?.Gemini) as
133
+ | (new (opts: Record<string, unknown>) => GeminiClientLike)
134
+ | undefined;
135
+ if (typeof Gemini !== "function") {
136
+ throw new Error("gemini-reverse: Gemini export not found (unexpected package shape)");
137
+ }
138
+ // ponytail: generous per-request cap (covers research); per-mode ask/research
139
+ // timeouts are enforced by raceGuard below.
140
+ return (opts) => new Gemini({ secure_1psid: opts.secure_1psid, proxy: opts.proxy ?? null, timeout: 1_800_000 });
141
+ }
142
+
143
+ async function getClient(config: GeminiWebConfig, factory?: GeminiClientFactory): Promise<GeminiClientLike> {
144
+ const key = `${config.psid ?? ""}|${config.proxy ?? ""}`;
145
+ if (cached?.key === key) return cached.client;
146
+ const make = factory ?? (await loadDefaultFactory());
147
+ const client = await make({ secure_1psid: config.psid, proxy: config.proxy });
148
+ cached = { key, client };
149
+ return client;
150
+ }
151
+
152
+ /** @internal test hook */
153
+ export function __resetGeminiClientCache(): void {
154
+ cached = null;
155
+ }
156
+
157
+ function errorName(err: unknown): string {
158
+ const e = err as { name?: string; constructor?: { name?: string } } | null;
159
+ const name = e?.name;
160
+ // Subclasses that don't set this.name inherit the generic "Error" — prefer
161
+ // the constructor name in that case (how we detect upstream error classes).
162
+ if (name && name !== "Error") return name;
163
+ return e?.constructor?.name ?? name ?? "";
164
+ }
165
+
166
+ function isAuthError(err: unknown): boolean {
167
+ return errorName(err) === "AuthError";
168
+ }
169
+
170
+ // One AuthError retry: re-creating the client re-runs init, which absorbs the
171
+ // rotated Set-Cookies (incl. __Secure-1PSIDTS) Google hands back.
172
+ export async function withGeminiClient<T>(
173
+ config: GeminiWebConfig,
174
+ run: (client: GeminiClientLike) => Promise<T>,
175
+ factory?: GeminiClientFactory,
176
+ ): Promise<T> {
177
+ try {
178
+ return await run(await getClient(config, factory));
179
+ } catch (err) {
180
+ if (isAuthError(err) && config.psid) {
181
+ cached = null;
182
+ return run(await getClient(config, factory));
183
+ }
184
+ throw err;
185
+ }
186
+ }
187
+
188
+ // ponytail: gemini-reverse polls aren't cancellable — abort/timeout rejects the
189
+ // tool call promptly, but the underlying client poll finishes/times out in the
190
+ // background (ceiling; real cancellation needs upstream AbortSignal support).
191
+ // Shared with lib/imageapi.ts (same abort semantics for plain fetch calls).
192
+ export function raceGuard<T>(
193
+ promise: Promise<T>,
194
+ opts: { signal?: AbortSignal; timeoutMs?: number; label: string },
195
+ ): Promise<T> {
196
+ const { signal, timeoutMs, label } = opts;
197
+ if (!signal && !timeoutMs) return promise;
198
+ return new Promise<T>((resolve, reject) => {
199
+ let timer: ReturnType<typeof setTimeout> | undefined;
200
+ const onAbort = () => {
201
+ clearTimeout(timer);
202
+ const err = new Error(`${label} aborted`);
203
+ err.name = "AbortError";
204
+ reject(err);
205
+ };
206
+ const onTimeout = () => {
207
+ signal?.removeEventListener("abort", onAbort);
208
+ const err = new Error(`${label} timed out after ${timeoutMs}ms`);
209
+ err.name = "TimeoutError";
210
+ reject(err);
211
+ };
212
+ if (signal) {
213
+ if (signal.aborted) return onAbort();
214
+ signal.addEventListener("abort", onAbort, { once: true });
215
+ }
216
+ if (timeoutMs) timer = setTimeout(onTimeout, timeoutMs);
217
+ promise.then(
218
+ (v) => {
219
+ signal?.removeEventListener("abort", onAbort);
220
+ clearTimeout(timer);
221
+ resolve(v);
222
+ },
223
+ (e) => {
224
+ signal?.removeEventListener("abort", onAbort);
225
+ clearTimeout(timer);
226
+ reject(e);
227
+ },
228
+ );
229
+ });
230
+ }
231
+
232
+ // ---------------------------------------------------------------------------
233
+ // Source extraction (parsed web output has no structured citations field —
234
+ // URLs are pulled from the answer/report markdown text)
235
+ // ---------------------------------------------------------------------------
236
+
237
+ const MD_LINK_RE = /\[[^\]]*\]\((https?:\/\/[^)\s]+)\)/g;
238
+ const BARE_URL_RE = /https?:\/\/[^\s<>()\[\]{}"'`]+/g;
239
+
240
+ export function extractSources(text: string, cap = 30): string[] {
241
+ if (!text) return [];
242
+ const seen = new Set<string>();
243
+ const sources: string[] = [];
244
+ const push = (raw: string) => {
245
+ const url = raw.replace(/[.,;:!?)\]]+$/, "");
246
+ if (!seen.has(url)) {
247
+ seen.add(url);
248
+ sources.push(url);
249
+ }
250
+ };
251
+ for (const m of text.matchAll(MD_LINK_RE)) push(m[1]);
252
+ for (const m of text.matchAll(BARE_URL_RE)) push(m[0]);
253
+ return sources.slice(0, cap);
254
+ }
255
+
256
+ // ---------------------------------------------------------------------------
257
+ // Error mapping
258
+ // ---------------------------------------------------------------------------
259
+
260
+ export function describeGeminiError(err: unknown): string {
261
+ switch (errorName(err)) {
262
+ case "AuthError":
263
+ return "Gemini web cookie expired or invalid. Re-copy __Secure-1PSID from gemini.google.com (F12 → Application → Cookies) into GEMINI_WEB_SECURE_1PSID in ~/.pi/agent/.env.local, then restart pi.";
264
+ case "UsageLimitExceeded":
265
+ return "Gemini web usage limit reached. Try again later or pick a different model.";
266
+ case "TemporarilyBlocked":
267
+ return "Gemini web temporarily blocked this IP. Wait a while or set GEMINI_WEB_PROXY (e.g. http://host:port).";
268
+ case "ModelInvalid":
269
+ return "Gemini model unavailable for this account. Try another model or drop the model parameter.";
270
+ case "AbortError":
271
+ case "TimeoutError":
272
+ return err instanceof Error ? err.message : String(err);
273
+ default: {
274
+ const msg = err instanceof Error ? err.message : String(err);
275
+ if (/Unknown API error/.test(msg)) {
276
+ return `Gemini web rejected the request (${msg}). In research mode this usually means the account lacks a Gemini Advanced subscription (Deep Research is Advanced-only) or the web protocol changed. mode=ask still works.`;
277
+ }
278
+ return `Gemini web error: ${msg}`;
279
+ }
280
+ }
281
+ }
282
+
283
+ // ---------------------------------------------------------------------------
284
+ // Public API
285
+ // ---------------------------------------------------------------------------
286
+
287
+ export interface GeminiAskResult {
288
+ text: string;
289
+ model?: string;
290
+ guest: boolean;
291
+ sources: string[];
292
+ }
293
+
294
+ export async function geminiAsk(
295
+ query: string,
296
+ opts: { config: GeminiWebConfig; model?: string; timeoutMs?: number; signal?: AbortSignal; factory?: GeminiClientFactory },
297
+ ): Promise<GeminiAskResult> {
298
+ const out = await raceGuard(
299
+ withGeminiClient(
300
+ opts.config,
301
+ (client) => client.ask(query, { temporary: true, ...(opts.model ? { model: opts.model } : {}) }),
302
+ opts.factory,
303
+ ),
304
+ { signal: opts.signal, timeoutMs: opts.timeoutMs ?? 120_000, label: "web_research ask" },
305
+ );
306
+ const text = String(out?.text ?? out?.candidates?.[0]?.text ?? "");
307
+ return { text, model: out?.model, guest: !opts.config.psid, sources: extractSources(text) };
308
+ }
309
+
310
+ export interface GeminiResearchResult {
311
+ text: string;
312
+ title?: string | null;
313
+ eta?: string | null;
314
+ guest: boolean;
315
+ sources: string[];
316
+ }
317
+
318
+ export async function geminiResearch(
319
+ query: string,
320
+ opts: {
321
+ config: GeminiWebConfig;
322
+ timeoutMs?: number;
323
+ signal?: AbortSignal;
324
+ onStatus?: (s: Record<string, unknown>) => void;
325
+ factory?: GeminiClientFactory;
326
+ },
327
+ ): Promise<GeminiResearchResult> {
328
+ if (!opts.config.psid) {
329
+ throw new Error(
330
+ "Deep Research requires GEMINI_WEB_SECURE_1PSID (gemini.google.com cookie) in ~/.pi/agent/.env.local — guest mode does not support it. Deep Research also needs a Gemini Advanced subscription on the account.",
331
+ );
332
+ }
333
+ const timeoutMs = opts.timeoutMs ?? 600_000;
334
+ const result = await raceGuard(
335
+ withGeminiClient(
336
+ opts.config,
337
+ (client) =>
338
+ client.research(query, {
339
+ wait: true,
340
+ pollInterval: 10_000,
341
+ timeout: timeoutMs,
342
+ ...(opts.onStatus ? { onStatus: opts.onStatus } : {}),
343
+ }),
344
+ opts.factory,
345
+ ),
346
+ // +5s grace so the client's own poll-timeout (better semantics) fires first.
347
+ { signal: opts.signal, timeoutMs: timeoutMs + 5_000, label: "web_research research" },
348
+ );
349
+ const text = String(result?.text ?? "");
350
+ return {
351
+ text,
352
+ title: result?.plan?.title ?? null,
353
+ eta: result?.plan?.eta_text ?? null,
354
+ guest: false,
355
+ sources: extractSources(text),
356
+ };
357
+ }
358
+
359
+ // ---------------------------------------------------------------------------
360
+ // Image generation (free web tier): prompt → GeneratedImage.save() paths
361
+ // ---------------------------------------------------------------------------
362
+
363
+ export interface GeminiImageResult {
364
+ paths: string[];
365
+ model?: string;
366
+ guest: boolean;
367
+ text: string;
368
+ }
369
+
370
+ export async function geminiGenerateImage(
371
+ prompt: string,
372
+ opts: {
373
+ config: GeminiWebConfig;
374
+ outDir: string;
375
+ model?: string;
376
+ timeoutMs?: number;
377
+ signal?: AbortSignal;
378
+ factory?: GeminiClientFactory;
379
+ },
380
+ ): Promise<GeminiImageResult> {
381
+ const out = await raceGuard(
382
+ withGeminiClient(
383
+ opts.config,
384
+ async (client) => {
385
+ if (typeof client.newChat !== "function") {
386
+ throw new Error("gemini-reverse client exposes no newChat (unexpected package shape)");
387
+ }
388
+ const chat = client.newChat(opts.model ? { model: opts.model } : undefined);
389
+ return chat.generateContent({ prompt });
390
+ },
391
+ opts.factory,
392
+ ),
393
+ { signal: opts.signal, timeoutMs: opts.timeoutMs ?? 180_000, label: "web_image gemini" },
394
+ );
395
+ const images = out?.generated_images ?? out?.images ?? [];
396
+ const text = String(out?.text ?? "");
397
+ if (!images.length) {
398
+ throw new Error(
399
+ text
400
+ ? `Gemini replied with text but no images: ${text.slice(0, 200)} — image generation is likely unavailable for this account/region (it is for some Google accounts); try provider=zai (ZAI_API_KEY) or provider=custom.`
401
+ : "Gemini returned no images — generation may be unavailable for this account/region (guest mode may not support it; set GEMINI_WEB_SECURE_1PSID).",
402
+ );
403
+ }
404
+ const paths: string[] = [];
405
+ for (const img of images) paths.push(await img.save({ path: opts.outDir }));
406
+ return { paths, model: out?.model, guest: !opts.config.psid, text };
407
+ }
@@ -0,0 +1,354 @@
1
+ // Generic OpenAI-compatible images client + fallback chain for web_image.
2
+ // Serves the `zai` preset (official api.z.ai, CogView-4) and any `custom`
3
+ // OpenAI-images endpoint — direct-to-upstream plain fetch, no self-host.
4
+
5
+ import fs from "node:fs";
6
+ import path from "node:path";
7
+ import { randomUUID } from "node:crypto";
8
+ import { findEnvValue } from "./config";
9
+ import {
10
+ describeGeminiError,
11
+ geminiGenerateImage,
12
+ raceGuard,
13
+ type GeminiClientFactory,
14
+ type GeminiWebConfig,
15
+ } from "./gemini";
16
+
17
+ // ---------------------------------------------------------------------------
18
+ // Config
19
+ // ---------------------------------------------------------------------------
20
+
21
+ export const ZAI_PRESET = { baseUrl: "https://api.z.ai/api/paas/v4", defaultModel: "cogview-4" } as const;
22
+
23
+ export interface ImageApiConfig {
24
+ zai?: { apiKey: string; source: string };
25
+ custom?: { baseUrl: string; apiKey?: string; label: string; source: string };
26
+ }
27
+
28
+ export function loadImageApiConfig(cwd = process.cwd(), includeCwdEnv = false): ImageApiConfig {
29
+ const zaiKey = findEnvValue("ZAI_API_KEY", cwd, includeCwdEnv);
30
+ const zaiFound = zaiKey.value ? zaiKey : findEnvValue("Z_AI_API_KEY", cwd, includeCwdEnv);
31
+ const base = findEnvValue("WEB_IMAGE_API_BASE_URL", cwd, includeCwdEnv);
32
+ const key = findEnvValue("WEB_IMAGE_API_KEY", cwd, includeCwdEnv);
33
+ const label = findEnvValue("WEB_IMAGE_API_LABEL", cwd, includeCwdEnv);
34
+ const cfg: ImageApiConfig = {};
35
+ if (zaiFound.value) cfg.zai = { apiKey: zaiFound.value, source: zaiFound.source };
36
+ if (base.value) {
37
+ cfg.custom = {
38
+ baseUrl: base.value.replace(/\/+$/, ""),
39
+ ...(key.value ? { apiKey: key.value } : {}),
40
+ label: label.value || hostOf(base.value),
41
+ source: base.source,
42
+ };
43
+ }
44
+ return cfg;
45
+ }
46
+
47
+ function hostOf(url: string): string {
48
+ try {
49
+ return new URL(url).host;
50
+ } catch {
51
+ return url;
52
+ }
53
+ }
54
+
55
+ export interface ImageRateConfig {
56
+ minIntervalMs: number;
57
+ dailyCap: number; // applies to the gemini web tier only — keyed APIs stay uncapped
58
+ }
59
+
60
+ export function loadImageRateConfig(cwd = process.cwd(), includeCwdEnv = false): ImageRateConfig {
61
+ const interval = findEnvValue("WEB_IMAGE_MIN_INTERVAL_MS", cwd, includeCwdEnv);
62
+ const cap = findEnvValue("WEB_IMAGE_DAILY_CAP", cwd, includeCwdEnv);
63
+ const intervalNum = Number(interval.value);
64
+ const capNum = Number(cap.value);
65
+ return {
66
+ minIntervalMs: interval.value && Number.isFinite(intervalNum) ? Math.max(0, Math.trunc(intervalNum)) : 5000,
67
+ dailyCap: cap.value && Number.isFinite(capNum) ? Math.max(1, Math.trunc(capNum)) : 20,
68
+ };
69
+ }
70
+
71
+ // ---------------------------------------------------------------------------
72
+ // Soft rate guardrails (in-memory, reset on restart). Successful generations
73
+ // only — failures don't consume quota.
74
+ // ---------------------------------------------------------------------------
75
+
76
+ interface RateState {
77
+ lastAt: number;
78
+ day: string;
79
+ count: number;
80
+ }
81
+
82
+ const rate = new Map<string, RateState>();
83
+ let nowMs = () => Date.now();
84
+
85
+ /** @internal test hooks */
86
+ export function __setImageRateClock(fn: () => number): void {
87
+ nowMs = fn;
88
+ }
89
+
90
+ /** @internal test hooks */
91
+ export function __resetImageRate(): void {
92
+ rate.clear();
93
+ nowMs = () => Date.now();
94
+ }
95
+
96
+ const utcDay = (ts: number) => new Date(ts).toISOString().slice(0, 10);
97
+
98
+ function msUntilUtcRoll(ts: number): number {
99
+ const d = new Date(ts);
100
+ return Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate() + 1) - ts;
101
+ }
102
+
103
+ export type RateVerdict = { ok: true } | { ok: false; reason: string; retryAfterMs: number };
104
+
105
+ export function imageRateCheck(provider: "gemini" | "zai" | "custom", rateCfg: ImageRateConfig): RateVerdict {
106
+ const state = rate.get(provider);
107
+ const ts = nowMs();
108
+ if (state && rateCfg.minIntervalMs > 0) {
109
+ const elapsed = ts - state.lastAt;
110
+ if (elapsed < rateCfg.minIntervalMs) {
111
+ return {
112
+ ok: false,
113
+ reason: `min interval ${rateCfg.minIntervalMs}ms between calls (elapsed ${elapsed}ms, WEB_IMAGE_MIN_INTERVAL_MS)`,
114
+ retryAfterMs: rateCfg.minIntervalMs - elapsed,
115
+ };
116
+ }
117
+ }
118
+ if (provider === "gemini" && state && state.day === utcDay(ts) && state.count >= rateCfg.dailyCap) {
119
+ return {
120
+ ok: false,
121
+ reason: `daily soft cap reached (${rateCfg.dailyCap}/day, WEB_IMAGE_DAILY_CAP)`,
122
+ retryAfterMs: msUntilUtcRoll(ts),
123
+ };
124
+ }
125
+ return { ok: true };
126
+ }
127
+
128
+ export function imageRateRecord(provider: "gemini" | "zai" | "custom", count = 1): void {
129
+ const ts = nowMs();
130
+ const day = utcDay(ts);
131
+ const prev = rate.get(provider);
132
+ const sameDay = prev?.day === day;
133
+ rate.set(provider, { lastAt: ts, day, count: (sameDay ? prev!.count : 0) + count });
134
+ }
135
+
136
+ export function imageRateSnapshot(): Record<string, { count: number; day: string; msSinceLast: number }> {
137
+ const ts = nowMs();
138
+ const out: Record<string, { count: number; day: string; msSinceLast: number }> = {};
139
+ for (const [provider, s] of rate) {
140
+ out[provider] = { count: s.day === utcDay(ts) ? s.count : 0, day: s.day, msSinceLast: Math.max(0, ts - s.lastAt) };
141
+ }
142
+ return out;
143
+ }
144
+
145
+ // ---------------------------------------------------------------------------
146
+ // Single-provider OpenAI-images call (POST {base}/images/generations)
147
+ // ---------------------------------------------------------------------------
148
+
149
+ export interface FetchLike {
150
+ (
151
+ url: string,
152
+ init?: { method?: string; headers?: Record<string, string>; body?: string; signal?: AbortSignal },
153
+ ): Promise<{ ok: boolean; status: number; statusText?: string; json(): Promise<unknown>; arrayBuffer(): Promise<ArrayBuffer> }>;
154
+ }
155
+
156
+ export interface ApiImageResult {
157
+ paths: string[];
158
+ model?: string;
159
+ }
160
+
161
+ export class ImageApiError extends Error {
162
+ constructor(
163
+ public status: number,
164
+ message: string,
165
+ ) {
166
+ super(message);
167
+ }
168
+ }
169
+
170
+ export function describeImageApiError(err: unknown): string {
171
+ if (err instanceof ImageApiError) {
172
+ if (err.status === 401 || err.status === 403) return `upstream rejected the API key (HTTP ${err.status}): ${err.message}`;
173
+ if (err.status === 429) return `upstream rate limit/quota exhausted (HTTP 429): ${err.message}`;
174
+ if (err.status >= 500) return `upstream server error (HTTP ${err.status}): ${err.message}`;
175
+ return `upstream error (HTTP ${err.status}): ${err.message}`;
176
+ }
177
+ return err instanceof Error ? err.message : String(err);
178
+ }
179
+
180
+ export async function apiGenerateImage(opts: {
181
+ baseUrl: string;
182
+ apiKey?: string;
183
+ model?: string;
184
+ prompt: string;
185
+ n?: number;
186
+ outDir: string;
187
+ timeoutMs?: number;
188
+ signal?: AbortSignal;
189
+ fetchImpl?: FetchLike;
190
+ }): Promise<ApiImageResult> {
191
+ const fetchImpl = opts.fetchImpl ?? (fetch as FetchLike);
192
+ const url = `${opts.baseUrl.replace(/\/+$/, "")}/images/generations`;
193
+ const res = await raceGuard(
194
+ fetchImpl(url, {
195
+ method: "POST",
196
+ headers: { "Content-Type": "application/json", ...(opts.apiKey ? { Authorization: `Bearer ${opts.apiKey}` } : {}) },
197
+ body: JSON.stringify({ model: opts.model, prompt: opts.prompt, n: opts.n ?? 1 }),
198
+ signal: opts.signal,
199
+ }),
200
+ { signal: opts.signal, timeoutMs: opts.timeoutMs ?? 180_000, label: "web_image api" },
201
+ );
202
+ const payload = (await res.json().catch(() => null)) as Record<string, unknown> | null;
203
+ if (!res.ok) {
204
+ const errObj = payload?.error as { message?: string } | undefined;
205
+ const msg =
206
+ errObj?.message ??
207
+ (typeof payload?.message === "string" ? payload.message : undefined) ??
208
+ (payload ? JSON.stringify(payload).slice(0, 300) : res.statusText ?? "");
209
+ throw new ImageApiError(res.status, String(msg));
210
+ }
211
+ const items = Array.isArray(payload?.data) ? (payload!.data as Array<Record<string, unknown>>) : [];
212
+ if (!items.length) throw new Error(`upstream returned no image data (model ${opts.model ?? "default"})`);
213
+ fs.mkdirSync(opts.outDir, { recursive: true });
214
+ const paths: string[] = [];
215
+ for (let i = 0; i < items.length; i++) {
216
+ const item = items[i];
217
+ if (typeof item?.b64_json === "string" && item.b64_json) {
218
+ paths.push(writeB64(opts.outDir, item.b64_json, i));
219
+ } else if (typeof item?.url === "string" && item.url) {
220
+ paths.push(await downloadImage(fetchImpl, item.url, opts.outDir, i, opts.signal, opts.timeoutMs));
221
+ } else {
222
+ throw new Error(`image item ${i} had neither b64_json nor url`);
223
+ }
224
+ }
225
+ return { paths, model: typeof payload?.model === "string" ? payload.model : opts.model };
226
+ }
227
+
228
+ function writeB64(outDir: string, b64: string, i: number): string {
229
+ const file = path.join(outDir, `pi-web-image-${randomUUID().slice(0, 8)}-${i}.png`);
230
+ fs.writeFileSync(file, Buffer.from(b64, "base64"));
231
+ return file;
232
+ }
233
+
234
+ async function downloadImage(
235
+ fetchImpl: FetchLike,
236
+ url: string,
237
+ outDir: string,
238
+ i: number,
239
+ signal?: AbortSignal,
240
+ timeoutMs?: number,
241
+ ): Promise<string> {
242
+ const res = await raceGuard(fetchImpl(url, { method: "GET", signal }), {
243
+ signal,
244
+ timeoutMs: timeoutMs ?? 120_000,
245
+ label: "web_image download",
246
+ });
247
+ if (!res.ok) throw new ImageApiError(res.status, `image download failed (HTTP ${res.status})`);
248
+ const buf = Buffer.from(await res.arrayBuffer());
249
+ const file = path.join(outDir, `pi-web-image-${randomUUID().slice(0, 8)}-${i}${extFor(url)}`);
250
+ fs.writeFileSync(file, buf);
251
+ return file;
252
+ }
253
+
254
+ function extFor(url: string): string {
255
+ const m = /\.(png|jpe?g|webp|gif)(\?|$)/i.exec(url);
256
+ const ext = (m?.[1] ?? "png").toLowerCase();
257
+ return `.${ext === "jpeg" ? "jpg" : ext}`;
258
+ }
259
+
260
+ // ---------------------------------------------------------------------------
261
+ // Fallback chain: gemini (free web tier) → zai (official API) → custom
262
+ // ---------------------------------------------------------------------------
263
+
264
+ export type ImageProvider = "gemini" | "zai" | "custom";
265
+
266
+ export interface ImageChainResult {
267
+ provider: ImageProvider;
268
+ model?: string;
269
+ paths: string[];
270
+ attempts: string[];
271
+ }
272
+
273
+ export interface ImageChainParams {
274
+ prompt: string;
275
+ model?: string;
276
+ n?: number;
277
+ outDir: string;
278
+ provider: "auto" | ImageProvider;
279
+ geminiConfig: GeminiWebConfig;
280
+ apiConfig: ImageApiConfig;
281
+ rateConfig: ImageRateConfig;
282
+ timeoutMs?: number;
283
+ signal?: AbortSignal;
284
+ /** @internal test injection */
285
+ geminiFactory?: GeminiClientFactory;
286
+ /** @internal test injection */
287
+ fetchImpl?: FetchLike;
288
+ }
289
+
290
+ function chainFor(provider: "auto" | ImageProvider): ImageProvider[] {
291
+ // Auto includes ALL providers: unconfigured ones contribute "not configured
292
+ // (set …)" hints to the aggregated error instead of vanishing silently.
293
+ return provider === "auto" ? ["gemini", "zai", "custom"] : [provider];
294
+ }
295
+
296
+ export async function generateImageWithFallback(params: ImageChainParams): Promise<ImageChainResult> {
297
+ const chain = chainFor(params.provider);
298
+ const attempts: string[] = [];
299
+ for (const provider of chain) {
300
+ const configured =
301
+ provider === "gemini" ? true : provider === "zai" ? Boolean(params.apiConfig.zai) : Boolean(params.apiConfig.custom);
302
+ if (!configured) {
303
+ attempts.push(`${provider}: not configured${provider === "zai" ? " (set ZAI_API_KEY)" : " (set WEB_IMAGE_API_BASE_URL)"}`);
304
+ continue;
305
+ }
306
+ const rate = imageRateCheck(provider, params.rateConfig);
307
+ if (!rate.ok) {
308
+ attempts.push(`${provider}: skipped — ${rate.reason}`);
309
+ continue;
310
+ }
311
+ try {
312
+ let result: { paths: string[]; model?: string };
313
+ if (provider === "gemini") {
314
+ result = await geminiGenerateImage(params.prompt, {
315
+ config: params.geminiConfig,
316
+ outDir: params.outDir,
317
+ model: params.model,
318
+ timeoutMs: params.timeoutMs,
319
+ signal: params.signal,
320
+ factory: params.geminiFactory,
321
+ });
322
+ } else if (provider === "zai") {
323
+ result = await apiGenerateImage({
324
+ baseUrl: ZAI_PRESET.baseUrl,
325
+ apiKey: params.apiConfig.zai!.apiKey,
326
+ model: params.model ?? ZAI_PRESET.defaultModel,
327
+ prompt: params.prompt,
328
+ n: params.n,
329
+ outDir: params.outDir,
330
+ timeoutMs: params.timeoutMs,
331
+ signal: params.signal,
332
+ fetchImpl: params.fetchImpl,
333
+ });
334
+ } else {
335
+ result = await apiGenerateImage({
336
+ baseUrl: params.apiConfig.custom!.baseUrl,
337
+ apiKey: params.apiConfig.custom!.apiKey,
338
+ model: params.model,
339
+ prompt: params.prompt,
340
+ n: params.n,
341
+ outDir: params.outDir,
342
+ timeoutMs: params.timeoutMs,
343
+ signal: params.signal,
344
+ fetchImpl: params.fetchImpl,
345
+ });
346
+ }
347
+ imageRateRecord(provider);
348
+ return { provider, model: result.model, paths: result.paths, attempts };
349
+ } catch (err) {
350
+ attempts.push(`${provider}: ${provider === "gemini" ? describeGeminiError(err) : describeImageApiError(err)}`);
351
+ }
352
+ }
353
+ throw new Error(`All image providers failed:\n${attempts.map((a) => `- ${a}`).join("\n")}`);
354
+ }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@bacnh85/pi-web",
3
- "version": "0.7.1",
4
- "description": "Pi extension for web search, page extraction, Firecrawl scraping/crawling, and Crawl4AI headless browser crawling.",
3
+ "version": "0.9.0",
4
+ "description": "Pi extension for web search, page extraction, Firecrawl scraping/crawling, Crawl4AI headless browser crawling, Gemini web-tier research, and free upstream image generation.",
5
5
  "type": "module",
6
6
  "license": "MIT",
7
7
  "publishConfig": {
@@ -21,7 +21,12 @@
21
21
  "firecrawl",
22
22
  "crawl4ai",
23
23
  "scraping",
24
- "crawling"
24
+ "crawling",
25
+ "gemini",
26
+ "deep-research",
27
+ "image-generation",
28
+ "zai",
29
+ "cogview"
25
30
  ],
26
31
  "scripts": {
27
32
  "test": "cd extensions && npx mocha"
@@ -44,6 +49,7 @@
44
49
  },
45
50
  "dependencies": {
46
51
  "@mozilla/readability": "^0.6.0",
52
+ "gemini-reverse": "^2.1.0",
47
53
  "jsdom": "^27.0.1",
48
54
  "turndown": "^7.2.2",
49
55
  "turndown-plugin-gfm": "^1.0.2"
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  name: pi-web
3
- description: Web search, content extraction, site crawling, and page capture via the pi-web extension. Use when the user needs current web search results, documentation lookup, factual research, source discovery, URL-to-markdown extraction, JSON extraction from websites, site URL discovery, site crawling, or page screenshots/PDFs. Use when the user mentions searching the web, finding docs, looking something up, researching, scraping/extracting content from a URL, or capturing a page.
3
+ description: Web search, content extraction, site crawling, page capture, Gemini web-tier research, and free upstream image generation via the pi-web extension. Use when the user needs current web search results, documentation lookup, factual research, AI-synthesized research with sources (Gemini Deep Research), image generation from text (Gemini web, Z.ai CogView-4), source discovery, URL-to-markdown extraction, JSON extraction from websites, site URL discovery, site crawling, or page screenshots/PDFs. Use when the user mentions searching the web, finding docs, looking something up, researching deeply, generating/creating an image, scraping/extracting content from a URL, or capturing a page.
4
4
  ---
5
5
 
6
6
  # pi-web — Unified Web Tools
7
7
 
8
- Use the **7 unified tools** from the `pi-web` extension for all web-related tasks. These tools automatically select the best backend from SearXNG, Brave Search, Firecrawl, Crawl4AI, and agy (when installed) — you don't need to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive queries and inline content prefer Brave.
8
+ Use the **9 unified tools** from the `pi-web` extension for all web-related tasks. These tools automatically select the best backend from SearXNG, Brave Search, Firecrawl, Crawl4AI, and agy (when installed) — you don't need to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive queries and inline content prefer Brave.
9
9
 
10
10
  ## Quick Reference
11
11
 
@@ -17,6 +17,8 @@ Use the **7 unified tools** from the `pi-web` extension for all web-related task
17
17
  | `web_crawl` | Crawl multiple pages from a site | Light (Firecrawl) or Full (Crawl4AI) |
18
18
  | `web_screenshot` | Capture page screenshot as PNG | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
19
19
  | `web_pdf` | Generate page PDF | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
20
+ | `web_research` | AI-synthesized research with sources | Gemini web tier: ask = grounded answer (guest OK); research = Deep Research report (cookie + Gemini Advanced) |
21
+ | `web_image` | Generate images from a text prompt | Gemini web (guest/cookie) → Z.ai CogView-4 (`ZAI_API_KEY`) → custom OpenAI-images endpoint |
20
22
  | `web_status` | Check provider configuration and health | — |
21
23
 
22
24
  ## Decision Tree
@@ -53,6 +55,16 @@ What do you need?
53
55
  ├── Printable/archivable PDF of a page
54
56
  │ → web_pdf
55
57
 
58
+ ├── AI-synthesized research with sources (synthesis, comparisons, reports)
59
+ │ → web_research
60
+ │ ├─ quick grounded answer: mode=ask (default; guest OK, Flash-only)
61
+ │ └─ multi-minute Deep Research report: mode=research (needs cookie + Gemini Advanced)
62
+
63
+ ├── Generate an image from a text prompt (NOT capturing an existing page)
64
+ │ → web_image
65
+ │ ├─ default: provider=auto (Gemini web → Z.ai CogView-4 → custom endpoint)
66
+ │ └─ pin/model: provider=zai model=cogview-4, or any custom OpenAI-images endpoint
67
+
56
68
  └── Check what web tools are configured
57
69
  → web_status
58
70
  ```