@bacnh85/pi-web 0.7.1 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/README.md +107 -2
- package/extensions/index.ts +152 -0
- package/extensions/lib/gemini.ts +407 -0
- package/extensions/lib/imageapi.ts +354 -0
- package/package.json +9 -3
- package/skills/pi-web/SKILL.md +14 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,52 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.0 (2026-09-13)
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **`web_image` tool** — image generation from text via free upstream
|
|
8
|
+
providers, all direct-to-upstream (no self-host services), with automatic
|
|
9
|
+
fallback: `gemini` (gemini.google.com web tier via `gemini-reverse`, guest
|
|
10
|
+
or cookie auth) → `zai` (official `api.z.ai` CogView-4 via `ZAI_API_KEY`)
|
|
11
|
+
→ `custom` (any OpenAI-compatible `/images/generations` endpoint via
|
|
12
|
+
`WEB_IMAGE_API_BASE_URL`). `provider: "auto"` walks the chain and the
|
|
13
|
+
result reports every fallback attempt.
|
|
14
|
+
- Results are saved as files (`out_dir`, default fresh temp dir) and returned
|
|
15
|
+
as **inline image blocks** so multimodal models see their own output;
|
|
16
|
+
`model`/`n` (1–4) parameters per call.
|
|
17
|
+
- **Soft ToS guardrails** — per-provider `WEB_IMAGE_MIN_INTERVAL_MS`
|
|
18
|
+
(default 5000) and a `WEB_IMAGE_DAILY_CAP` (default 20/day) on the Gemini
|
|
19
|
+
web tier; successes-only counting, UTC-day reset, usage surfaced in
|
|
20
|
+
`web_status.imageProviders.rate`.
|
|
21
|
+
- `web_status` now reports `imageProviders` (gemini/zai/custom config + rate
|
|
22
|
+
snapshot).
|
|
23
|
+
- Smoke script: `image` and `zai` modes (prints saved paths + PNG magic-byte
|
|
24
|
+
check); ask/research modes now print **full answers and source URLs**
|
|
25
|
+
(previously sliced to a 200/400-char preview).
|
|
26
|
+
|
|
27
|
+
## 0.8.0 (2026-09-13)
|
|
28
|
+
|
|
29
|
+
### Added
|
|
30
|
+
|
|
31
|
+
- **`web_research` tool** — AI-synthesized web research via Gemini's web tier
|
|
32
|
+
(gemini.google.com), cookie-authed with `__Secure-1PSID`.
|
|
33
|
+
`mode: "ask"` returns a quick grounded answer with extracted source links
|
|
34
|
+
(works in guest mode without any cookie, Flash-only);
|
|
35
|
+
`mode: "research"` runs Gemini **Deep Research** — an autonomous agent that
|
|
36
|
+
browses the web for minutes and returns a comprehensive report (requires the
|
|
37
|
+
cookie and a Gemini Advanced subscription; default timeout 10 min, cap 30).
|
|
38
|
+
- New `lib/gemini.ts` wrapper over the `gemini-reverse` npm package (lazy
|
|
39
|
+
dynamic import, injectable client for tests, one AuthError retry that
|
|
40
|
+
re-absorbs rotated Set-Cookies). Sources are extracted from markdown links
|
|
41
|
+
in the answer/report text (the web protocol exposes no structured citations).
|
|
42
|
+
- `web_status` now reports `geminiWeb` (configured/cookieSource/proxy).
|
|
43
|
+
- Env config: `GEMINI_WEB_SECURE_1PSID` (required for authed/research mode),
|
|
44
|
+
optional `GEMINI_WEB_PROXY` (escape hatch if Google blocks the IP).
|
|
45
|
+
- **Header-cap fix (authed mode)** — Google ships ~25 KB of response headers on
|
|
46
|
+
Gemini pages (giant `content-security-policy`), over Node's default 16 KB
|
|
47
|
+
parser cap; the wrapper injects a per-request `maxHeaderSize` for
|
|
48
|
+
gemini.google.com hosts only (lazy, no global flag needed).
|
|
49
|
+
|
|
3
50
|
## 0.7.1 (2026-09-12)
|
|
4
51
|
|
|
5
52
|
### Fixed
|
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# @bacnh85/pi-web
|
|
2
2
|
|
|
3
|
-
Pi extension for **unified web search, content extraction, site crawling, and
|
|
3
|
+
Pi extension for **unified web search, content extraction, site crawling, page capture, Gemini web-tier research, and free upstream image generation**.
|
|
4
4
|
|
|
5
|
-
Auto-selects the best backend from SearXNG (self-hosted), Brave Search, Firecrawl, Crawl4AI, and agy (Gemini/Claude, when installed) — so agents don't have to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive searches and inline content prefer Brave.
|
|
5
|
+
Auto-selects the best backend from SearXNG (self-hosted), Brave Search, Firecrawl, Crawl4AI, and agy (Gemini/Claude, when installed) — so agents don't have to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive searches and inline content prefer Brave. `web_research` adds AI-synthesized research with citations via your gemini.google.com session.
|
|
6
6
|
|
|
7
7
|
## Install
|
|
8
8
|
|
|
@@ -30,10 +30,20 @@ Variables:
|
|
|
30
30
|
| `FIRECRAWL_API_KEY` | No (2) | — | Required for hosted Firecrawl |
|
|
31
31
|
| `CRAWL4AI_API_URL` | No | `http://127.0.0.1:11235` | Self-hosted Crawl4AI |
|
|
32
32
|
| `CRAWL4AI_API_TOKEN` | No (3) | — | Required if Crawl4AI auth enabled |
|
|
33
|
+
| `GEMINI_WEB_SECURE_1PSID` | No (4) | — | `__Secure-1PSID` cookie from gemini.google.com — enables authed `web_research` (Deep Research) |
|
|
34
|
+
| `GEMINI_WEB_PROXY` | No | — | Proxy URL for Gemini web calls (escape hatch if Google blocks the IP) |
|
|
35
|
+
| `ZAI_API_KEY` | No (5) | — | Z.ai API key — enables the `web_image` `zai` provider (CogView-4 via the official `api.z.ai`); `Z_AI_API_KEY` also accepted |
|
|
36
|
+
| `WEB_IMAGE_API_BASE_URL` | No | — | `web_image` `custom` provider: any OpenAI-compatible images endpoint (e.g. `https://api.openai.com/v1`) |
|
|
37
|
+
| `WEB_IMAGE_API_KEY` | No | — | Bearer key for the `custom` endpoint |
|
|
38
|
+
| `WEB_IMAGE_API_LABEL` | No | — | Display label for the `custom` endpoint (default: host name) |
|
|
39
|
+
| `WEB_IMAGE_MIN_INTERVAL_MS` | No | `5000` | Min interval between `web_image` calls per provider |
|
|
40
|
+
| `WEB_IMAGE_DAILY_CAP` | No | `20` | Daily soft cap for the Gemini **web tier** `web_image` provider (keyed APIs stay uncapped) |
|
|
33
41
|
|
|
34
42
|
> (1) At least one search backend (SearXNG, Brave, or Firecrawl) must be configured for `web_search`.
|
|
35
43
|
> (2) Required for hosted Firecrawl; optional for self-hosted instances without auth.
|
|
36
44
|
> (3) Required for Crawl4AI v0.9+ default config.
|
|
45
|
+
> (4) Without it `web_research mode=ask` still works in guest mode (Flash-only); `mode=research` errors with setup steps.
|
|
46
|
+
> (5) `web_image` works with zero config via Gemini guest mode (availability varies by region/account); `zai` activates when `ZAI_API_KEY` is present, `custom` when `WEB_IMAGE_API_BASE_URL` is set.
|
|
37
47
|
|
|
38
48
|
Secrets are never printed; `web_status` reports only presence/source.
|
|
39
49
|
|
|
@@ -197,10 +207,105 @@ Typical output:
|
|
|
197
207
|
"health": { "status": "healthy", "version": "0.5.0", ... }
|
|
198
208
|
},
|
|
199
209
|
"agy": { "installed": true },
|
|
210
|
+
"geminiWeb": { "configured": true, "cookieSource": "process.env", "proxy": false },
|
|
200
211
|
"localChrome": { "path": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" }
|
|
201
212
|
}
|
|
202
213
|
```
|
|
203
214
|
|
|
215
|
+
### `web_research` — Gemini web-tier research
|
|
216
|
+
|
|
217
|
+
AI-synthesized research through your gemini.google.com session (powered by the
|
|
218
|
+
[`gemini-reverse`](https://github.com/rynn-k/Gemini-Reverse) client, lazy-loaded).
|
|
219
|
+
|
|
220
|
+
```
|
|
221
|
+
web_research(query="compare the top 3 cloud providers' AI offerings", mode="research", timeout_ms=600000)
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
- **`mode: "ask"`** (default) — quick grounded answer (Gemini auto-grounds with
|
|
225
|
+
Google Search). Works **without any cookie** in guest mode (Flash-only).
|
|
226
|
+
Sent as a temporary chat so your Gemini history stays clean.
|
|
227
|
+
- **`mode: "research"`** — full Gemini **Deep Research**: plan → autonomous web
|
|
228
|
+
browsing (minutes) → cited report. Requires the cookie **and a Gemini
|
|
229
|
+
Advanced subscription** on the account. Default timeout 600 s, cap 1 800 000.
|
|
230
|
+
|
|
231
|
+
Both modes return the text plus **Sources** — URLs extracted from the
|
|
232
|
+
answer/report markdown (the web protocol exposes no structured citations field).
|
|
233
|
+
|
|
234
|
+
Setup (authed mode):
|
|
235
|
+
|
|
236
|
+
1. Sign in at [gemini.google.com](https://gemini.google.com/).
|
|
237
|
+
2. `F12` → **Application** → **Cookies** → `https://gemini.google.com`.
|
|
238
|
+
3. Copy the `__Secure-1PSID` value into `~/.pi/agent/.env.local`:
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
GEMINI_WEB_SECURE_1PSID=your-cookie-value
|
|
242
|
+
# optional, if Google blocks your IP:
|
|
243
|
+
# GEMINI_WEB_PROXY=http://host:port
|
|
244
|
+
```
|
|
245
|
+
4. Restart pi; `web_status` shows `geminiWeb.configured: true`.
|
|
246
|
+
|
|
247
|
+
Live verification script (also proves the header-cap patch end-to-end — an
|
|
248
|
+
authed failure would surface `HPE_HEADER_OVERFLOW`):
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
npx tsx extensions/scripts/gemini-smoke.ts "test query" # ask (authed or guest)
|
|
252
|
+
npx tsx extensions/scripts/gemini-smoke.ts "topic" research # Deep Research
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
⚠️ **Unofficial, at your own risk.** Cookie auth uses your real Google session
|
|
256
|
+
against gemini.google.com's internal web API and may not comply with Google's
|
|
257
|
+
ToS; the protocol can break when Google changes it. `ask` mode errors map to
|
|
258
|
+
actionable steps (expired cookie → re-copy; IP block → set `GEMINI_WEB_PROXY`).
|
|
259
|
+
|
|
260
|
+
Troubleshooting:
|
|
261
|
+
|
|
262
|
+
- *"cookie expired or invalid"* — re-copy `__Secure-1PSID` (it rotates).
|
|
263
|
+
- *"temporarily blocked this IP"* — set `GEMINI_WEB_PROXY`.
|
|
264
|
+
- *research mode: "Unknown API error: 1184"* — on this account Deep Research
|
|
265
|
+
was rejected; usually means no Gemini Advanced subscription on the account
|
|
266
|
+
(Deep Research is Advanced-only), or Google changed the protocol. `ask`
|
|
267
|
+
mode is unaffected.
|
|
268
|
+
|
|
269
|
+
### `web_image` — free upstream image generation
|
|
270
|
+
|
|
271
|
+
Text → image with automatic provider fallback (all direct-to-upstream, no
|
|
272
|
+
self-host services):
|
|
273
|
+
|
|
274
|
+
```
|
|
275
|
+
web_image(prompt="isometric cutaway of a container ship, technical illustration")
|
|
276
|
+
web_image(prompt="...", provider="zai") # pin CogView-4 via api.z.ai
|
|
277
|
+
web_image(prompt="...", model="cogview-4", n=2, out_dir="/tmp/imgs")
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
**Provider chain** (`provider: "auto"` tries in order; pin one to skip):
|
|
281
|
+
|
|
282
|
+
| Provider | Upstream | Auth | Notes |
|
|
283
|
+
|---|---|---|---|
|
|
284
|
+
| `gemini` (default) | gemini.google.com web tier | none (guest) or `GEMINI_WEB_SECURE_1PSID` | free ≈ 20 images/day; availability varies by region/account |
|
|
285
|
+
| `zai` | `https://api.z.ai/api/paas/v4` (official API) | `ZAI_API_KEY` | CogView-4 (`model` default), fully ToS-compliant |
|
|
286
|
+
| `custom` | any OpenAI-compatible `/images/generations` endpoint | `WEB_IMAGE_API_KEY` | e.g. official OpenAI `https://api.openai.com/v1` |
|
|
287
|
+
|
|
288
|
+
Results are saved to `out_dir` (default: fresh temp dir) and returned as file
|
|
289
|
+
paths **plus inline image blocks** (multimodal models see the render
|
|
290
|
+
immediately). `details` reports the winning provider, model, and fallback
|
|
291
|
+
attempts.
|
|
292
|
+
|
|
293
|
+
**Guardrails** (soft, in-memory): per-provider `WEB_IMAGE_MIN_INTERVAL_MS`
|
|
294
|
+
(default 5 s) and a `WEB_IMAGE_DAILY_CAP` (default 20/day, applied to the
|
|
295
|
+
Gemini web tier only — keyed APIs are billed upstream and stay uncapped).
|
|
296
|
+
Counters reset on restart; `web_status.imageProviders.rate` shows usage.
|
|
297
|
+
|
|
298
|
+
⚠️ **ToS reality (read once)**: *every* AI chatbot's terms prohibit automated
|
|
299
|
+
access to its web UI (Google, OpenAI, xAI "unauthorized automated or
|
|
300
|
+
non-human means", Z.ai alike). This tool therefore follows a risk ladder:
|
|
301
|
+
|
|
302
|
+
1. **Official APIs** (`zai`, `custom`) — fully compliant; prefer them when a key exists.
|
|
303
|
+
2. **Guest mode** (Gemini without a cookie) — no account at stake, lowest risk, Flash-tier.
|
|
304
|
+
3. **Personal cookie** (Gemini authed) — your own account, single session, low volume; same accepted-risk stance as the web bridges: use a burner/low-value account, never a valued one. No account pools, no commercial use, keep volume human-scale.
|
|
305
|
+
|
|
306
|
+
Smoke test: `npx tsx extensions/scripts/gemini-smoke.ts "a red cube on white background" image`
|
|
307
|
+
(or `… zai` for the Z.ai path).
|
|
308
|
+
|
|
204
309
|
## Library structure
|
|
205
310
|
|
|
206
311
|
| Module | Contents |
|
package/extensions/index.ts
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
/// <reference path="./types.d.ts" />
|
|
2
2
|
|
|
3
|
+
import fs from "node:fs";
|
|
4
|
+
import os from "node:os";
|
|
5
|
+
import path from "node:path";
|
|
6
|
+
|
|
3
7
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
4
8
|
import { Type } from "typebox";
|
|
5
9
|
|
|
@@ -21,6 +25,19 @@ import {
|
|
|
21
25
|
formatUnifiedSearchResults,
|
|
22
26
|
} from "./lib/format";
|
|
23
27
|
import { searchWithDiagnostics } from "./lib/search";
|
|
28
|
+
import {
|
|
29
|
+
loadGeminiWebConfig,
|
|
30
|
+
geminiAsk,
|
|
31
|
+
geminiResearch,
|
|
32
|
+
describeGeminiError,
|
|
33
|
+
} from "./lib/gemini";
|
|
34
|
+
import {
|
|
35
|
+
generateImageWithFallback,
|
|
36
|
+
loadImageApiConfig,
|
|
37
|
+
loadImageRateConfig,
|
|
38
|
+
imageRateSnapshot,
|
|
39
|
+
type ImageProvider,
|
|
40
|
+
} from "./lib/imageapi";
|
|
24
41
|
import { extractWithDiagnostics, type ExtractMode } from "./lib/extract";
|
|
25
42
|
import { firecrawlRequest, type FirecrawlResult } from "./lib/firecrawl";
|
|
26
43
|
import {
|
|
@@ -63,6 +80,19 @@ const engineSchema = {
|
|
|
63
80
|
], { default: "auto", description: "auto routes localhost/private/file URLs to local Chrome, the rest to the Crawl4AI daemon; local/daemon force one." })),
|
|
64
81
|
};
|
|
65
82
|
|
|
83
|
+
// Saved image file → inline image block (the 0.6.2 vision-loop lesson: the
|
|
84
|
+
// generating model should see its own output).
|
|
85
|
+
async function toImageBlock(file: string): Promise<{ type: "image"; data: string; mimeType: string }> {
|
|
86
|
+
const data = (await fs.promises.readFile(file)).toString("base64");
|
|
87
|
+
const lower = file.toLowerCase();
|
|
88
|
+
const mimeType = lower.endsWith(".jpg") || lower.endsWith(".jpeg")
|
|
89
|
+
? "image/jpeg"
|
|
90
|
+
: lower.endsWith(".webp")
|
|
91
|
+
? "image/webp"
|
|
92
|
+
: "image/png";
|
|
93
|
+
return { type: "image" as const, data, mimeType };
|
|
94
|
+
}
|
|
95
|
+
|
|
66
96
|
// ---------------------------------------------------------------------------
|
|
67
97
|
// Always-on routing guidance (injected only when a web_* tool is active)
|
|
68
98
|
// ---------------------------------------------------------------------------
|
|
@@ -77,6 +107,8 @@ const WEB_ROUTING_GUIDANCE = `## Web Tool Routing (pi-web)
|
|
|
77
107
|
- **web_map** — discover site URLs (Firecrawl Map).
|
|
78
108
|
- **web_crawl** — multi-page crawl: \`mode: "light"\` (Firecrawl, url) or \`mode: "full"\` (Crawl4AI, urls[]).
|
|
79
109
|
- **web_screenshot** / **web_pdf** — page capture (Crawl4AI).
|
|
110
|
+
- **web_research** — AI-synthesized research via Gemini web (mode "ask" = grounded answer, guest OK; mode "research" = Deep Research report, needs cookie + Gemini Advanced, takes minutes).
|
|
111
|
+
- **web_image** — text→image generation via free upstreams (auto: Gemini web → Z.ai CogView-4 → custom OpenAI-images endpoint; \`model\`/\`n\` params).
|
|
80
112
|
- **web_status** — provider config + health.
|
|
81
113
|
|
|
82
114
|
Rules: Firecrawl Search is weak on domain-specific queries — prefer SearXNG/Brave; Firecrawl Scrape fails on bot-protected sites — use Crawl4AI (\`mode: "full"\`) then agy (\`mode: "agy"\`); cite source URLs.`;
|
|
@@ -440,6 +472,115 @@ export default function piWebExtension(pi: ExtensionAPI) {
|
|
|
440
472
|
},
|
|
441
473
|
});
|
|
442
474
|
|
|
475
|
+
// ── web_research ─────────────────────────────────────────────────────
|
|
476
|
+
pi.registerTool({
|
|
477
|
+
name: "web_research",
|
|
478
|
+
label: "Web Research (Gemini)",
|
|
479
|
+
description:
|
|
480
|
+
"AI-synthesized web research via Gemini (gemini.google.com web tier, cookie auth). Mode 'ask' returns a quick grounded answer with source links (works guest-mode, Flash only). Mode 'research' runs Gemini Deep Research — an autonomous agent browses the web for minutes and returns a comprehensive cited report (requires GEMINI_WEB_SECURE_1PSID cookie and a Gemini Advanced subscription).",
|
|
481
|
+
promptSnippet: "AI-synthesized research with citations",
|
|
482
|
+
promptGuidelines: [
|
|
483
|
+
"Use for AI-synthesized research with sources (mode ask = quick grounded answer; mode research = multi-minute Deep Research report). NOT for URL-list searches (web_search) or single-URL extraction (web_extract). Cite the returned source URLs.",
|
|
484
|
+
],
|
|
485
|
+
parameters: Type.Object({
|
|
486
|
+
query: Type.String({ description: "Research question or topic." }),
|
|
487
|
+
mode: Type.Optional(Type.Union(
|
|
488
|
+
[Type.Literal("ask"), Type.Literal("research")],
|
|
489
|
+
{ default: "ask", description: "ask = quick grounded answer (guest OK); research = full Deep Research report (cookie + Gemini Advanced required, takes minutes)." },
|
|
490
|
+
)),
|
|
491
|
+
model: Type.Optional(Type.String({ description: "Gemini model for ask mode (e.g. gemini-3-flash). Discovered from the account by default." })),
|
|
492
|
+
...sharedControlSchema,
|
|
493
|
+
}),
|
|
494
|
+
async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
|
|
495
|
+
const config = loadGeminiWebConfig(cwdFromContext(ctx), includeProjectEnv(ctx));
|
|
496
|
+
const mode = (params.mode as string) || "ask";
|
|
497
|
+
const query = params.query as string;
|
|
498
|
+
try {
|
|
499
|
+
if (mode === "research") {
|
|
500
|
+
const timeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 600_000, 30_000), 1_800_000);
|
|
501
|
+
const result = await geminiResearch(query, { config, timeoutMs, signal });
|
|
502
|
+
const meta = [
|
|
503
|
+
"Mode: research (Gemini Deep Research)",
|
|
504
|
+
result.title ? `Title: ${result.title}` : null,
|
|
505
|
+
result.eta ? `ETA: ${result.eta}` : null,
|
|
506
|
+
].filter(Boolean).join("\n");
|
|
507
|
+
const sources = result.sources.length ? result.sources.map((s, i) => `${i + 1}. ${s}`).join("\n") : "(none found in report text)";
|
|
508
|
+
const text = `${meta}\n\n${result.text}\n\n--- Sources (extracted from report) ---\n${sources}`;
|
|
509
|
+
return { content: [{ type: "text" as const, text: truncateText(text) }], details: { mode, ...result } };
|
|
510
|
+
}
|
|
511
|
+
const askTimeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 120_000, 30_000), 600_000);
|
|
512
|
+
const result = await geminiAsk(query, { config, model: params.model as string | undefined, timeoutMs: askTimeoutMs, signal });
|
|
513
|
+
const meta = [
|
|
514
|
+
"Mode: ask",
|
|
515
|
+
`Model: ${result.model ?? "unknown"}`,
|
|
516
|
+
result.guest ? "Guest mode (no cookie — Flash only; set GEMINI_WEB_SECURE_1PSID for full access)" : "Cookie auth",
|
|
517
|
+
].join("\n");
|
|
518
|
+
const sources = result.sources.length ? result.sources.map((s, i) => `${i + 1}. ${s}`).join("\n") : "(none found in answer text)";
|
|
519
|
+
const text = `${meta}\n\n${result.text}\n\n--- Sources (extracted from answer) ---\n${sources}`;
|
|
520
|
+
return { content: [{ type: "text" as const, text: truncateText(text) }], details: { mode, ...result } };
|
|
521
|
+
} catch (err) {
|
|
522
|
+
throw new Error(describeGeminiError(err));
|
|
523
|
+
}
|
|
524
|
+
},
|
|
525
|
+
});
|
|
526
|
+
|
|
527
|
+
// ── web_image ────────────────────────────────────────────────────────
|
|
528
|
+
pi.registerTool({
|
|
529
|
+
name: "web_image",
|
|
530
|
+
label: "Web Image Generation",
|
|
531
|
+
description:
|
|
532
|
+
"Generate images from text via free upstream providers, with fallback: Gemini web (gemini.google.com, guest or cookie auth), Z.ai official API (CogView-4 via ZAI_API_KEY), or any custom OpenAI-compatible images endpoint (WEB_IMAGE_API_BASE_URL). Returns saved file paths plus the images inline.",
|
|
533
|
+
promptSnippet: "Generate images via free upstreams (Gemini web, Z.ai CogView)",
|
|
534
|
+
promptGuidelines: [
|
|
535
|
+
"Use for image GENERATION from a text prompt. provider auto falls back gemini → zai → custom. Capturing an EXISTING page is web_screenshot, not this.",
|
|
536
|
+
],
|
|
537
|
+
parameters: Type.Object({
|
|
538
|
+
prompt: Type.String({ description: "Image description." }),
|
|
539
|
+
provider: Type.Optional(Type.Union(
|
|
540
|
+
[Type.Literal("auto"), Type.Literal("gemini"), Type.Literal("zai"), Type.Literal("custom")],
|
|
541
|
+
{ default: "auto", description: "auto = gemini → zai (if ZAI_API_KEY) → custom (if WEB_IMAGE_API_BASE_URL); pin one to skip fallback." },
|
|
542
|
+
)),
|
|
543
|
+
model: Type.Optional(Type.String({ description: "Provider-specific model (e.g. cogview-4, or a Gemini image-capable model id). Omit for the provider default." })),
|
|
544
|
+
n: Type.Optional(Type.Number({ default: 1, description: "Number of images, 1-4." })),
|
|
545
|
+
out_dir: Type.Optional(Type.String({ description: "Directory for saved images (default: fresh temp dir)." })),
|
|
546
|
+
...sharedControlSchema,
|
|
547
|
+
}),
|
|
548
|
+
async execute(_id: string, params: Record<string, unknown>, signal: AbortSignal, _onUpdate: unknown, ctx: any) {
|
|
549
|
+
const cwd = cwdFromContext(ctx);
|
|
550
|
+
const trusted = includeProjectEnv(ctx);
|
|
551
|
+
const prompt = params.prompt as string;
|
|
552
|
+
const n = Math.min(Math.max(Math.trunc((params.n as number) ?? 1) || 1, 1), 4);
|
|
553
|
+
const timeoutMs = Math.min(Math.max((params.timeout_ms as number) ?? 180_000, 10_000), 600_000);
|
|
554
|
+
const outDir = params.out_dir
|
|
555
|
+
? path.resolve(String(params.out_dir))
|
|
556
|
+
: await fs.promises.mkdtemp(path.join(os.tmpdir(), "pi-web-image-"));
|
|
557
|
+
const result = await generateImageWithFallback({
|
|
558
|
+
prompt,
|
|
559
|
+
model: params.model as string | undefined,
|
|
560
|
+
n,
|
|
561
|
+
outDir,
|
|
562
|
+
provider: (params.provider as "auto" | ImageProvider) ?? "auto",
|
|
563
|
+
geminiConfig: loadGeminiWebConfig(cwd, trusted),
|
|
564
|
+
apiConfig: loadImageApiConfig(cwd, trusted),
|
|
565
|
+
rateConfig: loadImageRateConfig(cwd, trusted),
|
|
566
|
+
timeoutMs,
|
|
567
|
+
signal,
|
|
568
|
+
});
|
|
569
|
+
const blocks = await Promise.all(result.paths.map(toImageBlock));
|
|
570
|
+
const text = [
|
|
571
|
+
`Provider: ${result.provider}${result.model ? ` (${result.model})` : ""}`,
|
|
572
|
+
`Saved: ${result.paths.length} image(s)`,
|
|
573
|
+
...result.paths.map((p) => ` ${p}`),
|
|
574
|
+
result.attempts.length ? `Fallback attempts: ${result.attempts.join(" | ")}` : null,
|
|
575
|
+
].filter(Boolean).join("\n");
|
|
576
|
+
const content: Array<{ type: "text"; text: string } | { type: "image"; data: string; mimeType: string }> = [
|
|
577
|
+
{ type: "text" as const, text },
|
|
578
|
+
...blocks,
|
|
579
|
+
];
|
|
580
|
+
return { content, details: { provider: result.provider, model: result.model, paths: result.paths, attempts: result.attempts } };
|
|
581
|
+
},
|
|
582
|
+
});
|
|
583
|
+
|
|
443
584
|
// ── web_status ───────────────────────────────────────────────────────
|
|
444
585
|
pi.registerTool({
|
|
445
586
|
name: "web_status",
|
|
@@ -460,6 +601,8 @@ export default function piWebExtension(pi: ExtensionAPI) {
|
|
|
460
601
|
const fireUrl = findEnvValue("FIRECRAWL_API_URL", cwd, trusted);
|
|
461
602
|
const c4aiUrl = findEnvValue("CRAWL4AI_API_URL", cwd, trusted);
|
|
462
603
|
const c4aiToken = findEnvValue("CRAWL4AI_API_TOKEN", cwd, trusted);
|
|
604
|
+
const geminiCfg = loadGeminiWebConfig(cwd, trusted);
|
|
605
|
+
const imageApiCfg = loadImageApiConfig(cwd, trusted);
|
|
463
606
|
|
|
464
607
|
const { isAgyInstalled } = await import("./lib/agy");
|
|
465
608
|
|
|
@@ -484,6 +627,15 @@ export default function piWebExtension(pi: ExtensionAPI) {
|
|
|
484
627
|
apiTokenSource: c4aiToken.value ? c4aiToken.source : "not set",
|
|
485
628
|
},
|
|
486
629
|
agy: { installed: isAgyInstalled() },
|
|
630
|
+
geminiWeb: { configured: Boolean(geminiCfg.psid), cookieSource: geminiCfg.psidSource, proxy: Boolean(geminiCfg.proxy) },
|
|
631
|
+
imageProviders: {
|
|
632
|
+
gemini: { configured: Boolean(geminiCfg.psid), guestPossible: true },
|
|
633
|
+
zai: { configured: Boolean(imageApiCfg.zai) },
|
|
634
|
+
custom: imageApiCfg.custom
|
|
635
|
+
? { configured: true, label: imageApiCfg.custom.label }
|
|
636
|
+
: { configured: false },
|
|
637
|
+
rate: imageRateSnapshot(),
|
|
638
|
+
},
|
|
487
639
|
localChrome: { path: findChromeBinary() ?? "not found" },
|
|
488
640
|
};
|
|
489
641
|
|
|
@@ -0,0 +1,407 @@
|
|
|
1
|
+
// Gemini web-tier (gemini.google.com) research backend, cookie-authed via the
|
|
2
|
+
// __Secure-1PSID cookie. Thin wrapper over the `gemini-reverse` npm package
|
|
3
|
+
// (CJS), lazily dynamic-imported so pi startup pays zero cost when unused.
|
|
4
|
+
|
|
5
|
+
import http from "node:http";
|
|
6
|
+
import https from "node:https";
|
|
7
|
+
import { urlToHttpOptions } from "node:url";
|
|
8
|
+
import { findEnvValue } from "./config";
|
|
9
|
+
|
|
10
|
+
// ---------------------------------------------------------------------------
|
|
11
|
+
// Config
|
|
12
|
+
// ---------------------------------------------------------------------------
|
|
13
|
+
|
|
14
|
+
export interface GeminiWebConfig {
|
|
15
|
+
psid?: string;
|
|
16
|
+
psidSource: string;
|
|
17
|
+
proxy?: string;
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
export function loadGeminiWebConfig(cwd = process.cwd(), includeCwdEnv = false): GeminiWebConfig {
|
|
21
|
+
const psid = findEnvValue("GEMINI_WEB_SECURE_1PSID", cwd, includeCwdEnv);
|
|
22
|
+
const proxy = findEnvValue("GEMINI_WEB_PROXY", cwd, includeCwdEnv);
|
|
23
|
+
return { psid: psid.value, psidSource: psid.value ? psid.source : "not set", proxy: proxy.value };
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
// ---------------------------------------------------------------------------
|
|
27
|
+
// Minimal structural view of the gemini-reverse client (keeps tests injectable
|
|
28
|
+
// and decouples us from upstream type drift).
|
|
29
|
+
// ---------------------------------------------------------------------------
|
|
30
|
+
|
|
31
|
+
export interface GeminiOutputLike {
|
|
32
|
+
text?: string | null;
|
|
33
|
+
model?: string;
|
|
34
|
+
candidates?: Array<{ text?: string | null }>;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export interface GeminiResearchLike {
|
|
38
|
+
text?: string | null;
|
|
39
|
+
plan?: { title?: string | null; eta_text?: string | null };
|
|
40
|
+
statuses?: Array<Record<string, unknown>>;
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export interface GeminiImageLike {
|
|
44
|
+
save(opts?: { path?: string; filename?: string }): Promise<string>;
|
|
45
|
+
url?: string;
|
|
46
|
+
alt?: string;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
export interface GeminiImageOutputLike {
|
|
50
|
+
text?: string | null;
|
|
51
|
+
model?: string;
|
|
52
|
+
images?: GeminiImageLike[];
|
|
53
|
+
generated_images?: GeminiImageLike[];
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
export interface GeminiClientLike {
|
|
57
|
+
ask(prompt: string, opts?: Record<string, unknown>): Promise<GeminiOutputLike>;
|
|
58
|
+
research(
|
|
59
|
+
prompt: string,
|
|
60
|
+
opts?: { wait?: boolean; pollInterval?: number; timeout?: number; onStatus?: (s: Record<string, unknown>) => void },
|
|
61
|
+
): Promise<GeminiResearchLike>;
|
|
62
|
+
// Optional so pre-0.9 fake clients (ask+research only) keep compiling.
|
|
63
|
+
newChat?(opts?: { model?: string }): {
|
|
64
|
+
generateContent(o: { prompt: string }): Promise<GeminiImageOutputLike>;
|
|
65
|
+
};
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
export type GeminiClientFactory = (
|
|
69
|
+
opts: { secure_1psid?: string; proxy?: string },
|
|
70
|
+
) => GeminiClientLike | Promise<GeminiClientLike>;
|
|
71
|
+
|
|
72
|
+
// Cached per config (psid|proxy) so a config change re-creates the client.
|
|
73
|
+
let cached: { key: string; client: GeminiClientLike } | null = null;
|
|
74
|
+
|
|
75
|
+
// Google ships ~25KB of response headers on every Gemini page (a 19.7KB
|
|
76
|
+
// content-security-policy + 3.9KB reporting-endpoints) — over Node's default
|
|
77
|
+
// 16KB parser cap, so the http parser rejects them (HPE_HEADER_OVERFLOW; the
|
|
78
|
+
// same cap exists in undici, but gemini-reverse uses axios's node http adapter).
|
|
79
|
+
// Node honors a per-request maxHeaderSize override, but axios doesn't forward
|
|
80
|
+
// it — so lazily patch http.request/https.request to inject it for
|
|
81
|
+
// gemini.google.com hosts only.
|
|
82
|
+
// ponytail: process-wide patch, scoped to one hostname; if it ever misbehaves,
|
|
83
|
+
// revert to launching pi with NODE_OPTIONS=--max-http-header-size=262144.
|
|
84
|
+
|
|
85
|
+
/**
|
|
86
|
+
* Returns the request options to pass through with the cap injected when the
|
|
87
|
+
* target host is gemini.google.com, or null when the call must pass through
|
|
88
|
+
* untouched. Normalizes all http.request input forms (options object, string,
|
|
89
|
+
* URL) — string/URL forms become a fresh options object.
|
|
90
|
+
*
|
|
91
|
+
* @internal exported for tests
|
|
92
|
+
*/
|
|
93
|
+
export function injectGeminiHeaderCap(options: unknown): Record<string, unknown> | null {
|
|
94
|
+
let opts: Record<string, unknown>;
|
|
95
|
+
if (typeof options === "string") {
|
|
96
|
+
opts = urlToHttpOptions(new URL(options)) as Record<string, unknown>;
|
|
97
|
+
} else if (options instanceof URL) {
|
|
98
|
+
opts = urlToHttpOptions(options) as Record<string, unknown>;
|
|
99
|
+
} else if (options && typeof options === "object") {
|
|
100
|
+
opts = options as Record<string, unknown>;
|
|
101
|
+
} else {
|
|
102
|
+
return null;
|
|
103
|
+
}
|
|
104
|
+
const host = String(opts.hostname ?? opts.host ?? "").split(":")[0];
|
|
105
|
+
if (host !== "gemini.google.com" || opts.maxHeaderSize) return null;
|
|
106
|
+
opts.maxHeaderSize = 256 * 1024;
|
|
107
|
+
return opts;
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
let headerCapPatched = false;
|
|
111
|
+
function patchHeaderCap(): void {
|
|
112
|
+
if (headerCapPatched) return;
|
|
113
|
+
headerCapPatched = true;
|
|
114
|
+
for (const mod of [http, https]) {
|
|
115
|
+
const real = mod.request as unknown as (...args: unknown[]) => unknown;
|
|
116
|
+
const patched = function (this: unknown, options: unknown, ...rest: unknown[]) {
|
|
117
|
+
try {
|
|
118
|
+
const override = injectGeminiHeaderCap(options);
|
|
119
|
+
if (override) return real.call(this, override, ...rest);
|
|
120
|
+
} catch { /* malformed input — let the real request surface the error */ }
|
|
121
|
+
return real.call(this, options, ...rest);
|
|
122
|
+
} as typeof mod.request;
|
|
123
|
+
mod.request = patched;
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
/** @internal exported for tests — resolves the real gemini-reverse module */
|
|
128
|
+
export async function loadDefaultFactory(): Promise<GeminiClientFactory> {
|
|
129
|
+
patchHeaderCap();
|
|
130
|
+
const mod = (await import("gemini-reverse")) as unknown as Record<string, unknown>;
|
|
131
|
+
// CJS interop: named export usually works, but fall back to default.Gemini.
|
|
132
|
+
const Gemini = (mod.Gemini ?? (mod as { default?: Record<string, unknown> }).default?.Gemini) as
|
|
133
|
+
| (new (opts: Record<string, unknown>) => GeminiClientLike)
|
|
134
|
+
| undefined;
|
|
135
|
+
if (typeof Gemini !== "function") {
|
|
136
|
+
throw new Error("gemini-reverse: Gemini export not found (unexpected package shape)");
|
|
137
|
+
}
|
|
138
|
+
// ponytail: generous per-request cap (covers research); per-mode ask/research
|
|
139
|
+
// timeouts are enforced by raceGuard below.
|
|
140
|
+
return (opts) => new Gemini({ secure_1psid: opts.secure_1psid, proxy: opts.proxy ?? null, timeout: 1_800_000 });
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
async function getClient(config: GeminiWebConfig, factory?: GeminiClientFactory): Promise<GeminiClientLike> {
|
|
144
|
+
const key = `${config.psid ?? ""}|${config.proxy ?? ""}`;
|
|
145
|
+
if (cached?.key === key) return cached.client;
|
|
146
|
+
const make = factory ?? (await loadDefaultFactory());
|
|
147
|
+
const client = await make({ secure_1psid: config.psid, proxy: config.proxy });
|
|
148
|
+
cached = { key, client };
|
|
149
|
+
return client;
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
/** @internal test hook */
|
|
153
|
+
export function __resetGeminiClientCache(): void {
|
|
154
|
+
cached = null;
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
function errorName(err: unknown): string {
|
|
158
|
+
const e = err as { name?: string; constructor?: { name?: string } } | null;
|
|
159
|
+
const name = e?.name;
|
|
160
|
+
// Subclasses that don't set this.name inherit the generic "Error" — prefer
|
|
161
|
+
// the constructor name in that case (how we detect upstream error classes).
|
|
162
|
+
if (name && name !== "Error") return name;
|
|
163
|
+
return e?.constructor?.name ?? name ?? "";
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
function isAuthError(err: unknown): boolean {
|
|
167
|
+
return errorName(err) === "AuthError";
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
// One AuthError retry: re-creating the client re-runs init, which absorbs the
|
|
171
|
+
// rotated Set-Cookies (incl. __Secure-1PSIDTS) Google hands back.
|
|
172
|
+
export async function withGeminiClient<T>(
|
|
173
|
+
config: GeminiWebConfig,
|
|
174
|
+
run: (client: GeminiClientLike) => Promise<T>,
|
|
175
|
+
factory?: GeminiClientFactory,
|
|
176
|
+
): Promise<T> {
|
|
177
|
+
try {
|
|
178
|
+
return await run(await getClient(config, factory));
|
|
179
|
+
} catch (err) {
|
|
180
|
+
if (isAuthError(err) && config.psid) {
|
|
181
|
+
cached = null;
|
|
182
|
+
return run(await getClient(config, factory));
|
|
183
|
+
}
|
|
184
|
+
throw err;
|
|
185
|
+
}
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
// ponytail: gemini-reverse polls aren't cancellable — abort/timeout rejects the
|
|
189
|
+
// tool call promptly, but the underlying client poll finishes/times out in the
|
|
190
|
+
// background (ceiling; real cancellation needs upstream AbortSignal support).
|
|
191
|
+
// Shared with lib/imageapi.ts (same abort semantics for plain fetch calls).
|
|
192
|
+
export function raceGuard<T>(
|
|
193
|
+
promise: Promise<T>,
|
|
194
|
+
opts: { signal?: AbortSignal; timeoutMs?: number; label: string },
|
|
195
|
+
): Promise<T> {
|
|
196
|
+
const { signal, timeoutMs, label } = opts;
|
|
197
|
+
if (!signal && !timeoutMs) return promise;
|
|
198
|
+
return new Promise<T>((resolve, reject) => {
|
|
199
|
+
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
200
|
+
const onAbort = () => {
|
|
201
|
+
clearTimeout(timer);
|
|
202
|
+
const err = new Error(`${label} aborted`);
|
|
203
|
+
err.name = "AbortError";
|
|
204
|
+
reject(err);
|
|
205
|
+
};
|
|
206
|
+
const onTimeout = () => {
|
|
207
|
+
signal?.removeEventListener("abort", onAbort);
|
|
208
|
+
const err = new Error(`${label} timed out after ${timeoutMs}ms`);
|
|
209
|
+
err.name = "TimeoutError";
|
|
210
|
+
reject(err);
|
|
211
|
+
};
|
|
212
|
+
if (signal) {
|
|
213
|
+
if (signal.aborted) return onAbort();
|
|
214
|
+
signal.addEventListener("abort", onAbort, { once: true });
|
|
215
|
+
}
|
|
216
|
+
if (timeoutMs) timer = setTimeout(onTimeout, timeoutMs);
|
|
217
|
+
promise.then(
|
|
218
|
+
(v) => {
|
|
219
|
+
signal?.removeEventListener("abort", onAbort);
|
|
220
|
+
clearTimeout(timer);
|
|
221
|
+
resolve(v);
|
|
222
|
+
},
|
|
223
|
+
(e) => {
|
|
224
|
+
signal?.removeEventListener("abort", onAbort);
|
|
225
|
+
clearTimeout(timer);
|
|
226
|
+
reject(e);
|
|
227
|
+
},
|
|
228
|
+
);
|
|
229
|
+
});
|
|
230
|
+
}
|
|
231
|
+
|
|
232
|
+
// ---------------------------------------------------------------------------
|
|
233
|
+
// Source extraction (parsed web output has no structured citations field —
|
|
234
|
+
// URLs are pulled from the answer/report markdown text)
|
|
235
|
+
// ---------------------------------------------------------------------------
|
|
236
|
+
|
|
237
|
+
const MD_LINK_RE = /\[[^\]]*\]\((https?:\/\/[^)\s]+)\)/g;
|
|
238
|
+
const BARE_URL_RE = /https?:\/\/[^\s<>()\[\]{}"'`]+/g;
|
|
239
|
+
|
|
240
|
+
export function extractSources(text: string, cap = 30): string[] {
|
|
241
|
+
if (!text) return [];
|
|
242
|
+
const seen = new Set<string>();
|
|
243
|
+
const sources: string[] = [];
|
|
244
|
+
const push = (raw: string) => {
|
|
245
|
+
const url = raw.replace(/[.,;:!?)\]]+$/, "");
|
|
246
|
+
if (!seen.has(url)) {
|
|
247
|
+
seen.add(url);
|
|
248
|
+
sources.push(url);
|
|
249
|
+
}
|
|
250
|
+
};
|
|
251
|
+
for (const m of text.matchAll(MD_LINK_RE)) push(m[1]);
|
|
252
|
+
for (const m of text.matchAll(BARE_URL_RE)) push(m[0]);
|
|
253
|
+
return sources.slice(0, cap);
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
// ---------------------------------------------------------------------------
|
|
257
|
+
// Error mapping
|
|
258
|
+
// ---------------------------------------------------------------------------
|
|
259
|
+
|
|
260
|
+
export function describeGeminiError(err: unknown): string {
|
|
261
|
+
switch (errorName(err)) {
|
|
262
|
+
case "AuthError":
|
|
263
|
+
return "Gemini web cookie expired or invalid. Re-copy __Secure-1PSID from gemini.google.com (F12 → Application → Cookies) into GEMINI_WEB_SECURE_1PSID in ~/.pi/agent/.env.local, then restart pi.";
|
|
264
|
+
case "UsageLimitExceeded":
|
|
265
|
+
return "Gemini web usage limit reached. Try again later or pick a different model.";
|
|
266
|
+
case "TemporarilyBlocked":
|
|
267
|
+
return "Gemini web temporarily blocked this IP. Wait a while or set GEMINI_WEB_PROXY (e.g. http://host:port).";
|
|
268
|
+
case "ModelInvalid":
|
|
269
|
+
return "Gemini model unavailable for this account. Try another model or drop the model parameter.";
|
|
270
|
+
case "AbortError":
|
|
271
|
+
case "TimeoutError":
|
|
272
|
+
return err instanceof Error ? err.message : String(err);
|
|
273
|
+
default: {
|
|
274
|
+
const msg = err instanceof Error ? err.message : String(err);
|
|
275
|
+
if (/Unknown API error/.test(msg)) {
|
|
276
|
+
return `Gemini web rejected the request (${msg}). In research mode this usually means the account lacks a Gemini Advanced subscription (Deep Research is Advanced-only) or the web protocol changed. mode=ask still works.`;
|
|
277
|
+
}
|
|
278
|
+
return `Gemini web error: ${msg}`;
|
|
279
|
+
}
|
|
280
|
+
}
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
// ---------------------------------------------------------------------------
|
|
284
|
+
// Public API
|
|
285
|
+
// ---------------------------------------------------------------------------
|
|
286
|
+
|
|
287
|
+
export interface GeminiAskResult {
|
|
288
|
+
text: string;
|
|
289
|
+
model?: string;
|
|
290
|
+
guest: boolean;
|
|
291
|
+
sources: string[];
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
export async function geminiAsk(
|
|
295
|
+
query: string,
|
|
296
|
+
opts: { config: GeminiWebConfig; model?: string; timeoutMs?: number; signal?: AbortSignal; factory?: GeminiClientFactory },
|
|
297
|
+
): Promise<GeminiAskResult> {
|
|
298
|
+
const out = await raceGuard(
|
|
299
|
+
withGeminiClient(
|
|
300
|
+
opts.config,
|
|
301
|
+
(client) => client.ask(query, { temporary: true, ...(opts.model ? { model: opts.model } : {}) }),
|
|
302
|
+
opts.factory,
|
|
303
|
+
),
|
|
304
|
+
{ signal: opts.signal, timeoutMs: opts.timeoutMs ?? 120_000, label: "web_research ask" },
|
|
305
|
+
);
|
|
306
|
+
const text = String(out?.text ?? out?.candidates?.[0]?.text ?? "");
|
|
307
|
+
return { text, model: out?.model, guest: !opts.config.psid, sources: extractSources(text) };
|
|
308
|
+
}
|
|
309
|
+
|
|
310
|
+
export interface GeminiResearchResult {
|
|
311
|
+
text: string;
|
|
312
|
+
title?: string | null;
|
|
313
|
+
eta?: string | null;
|
|
314
|
+
guest: boolean;
|
|
315
|
+
sources: string[];
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
export async function geminiResearch(
|
|
319
|
+
query: string,
|
|
320
|
+
opts: {
|
|
321
|
+
config: GeminiWebConfig;
|
|
322
|
+
timeoutMs?: number;
|
|
323
|
+
signal?: AbortSignal;
|
|
324
|
+
onStatus?: (s: Record<string, unknown>) => void;
|
|
325
|
+
factory?: GeminiClientFactory;
|
|
326
|
+
},
|
|
327
|
+
): Promise<GeminiResearchResult> {
|
|
328
|
+
if (!opts.config.psid) {
|
|
329
|
+
throw new Error(
|
|
330
|
+
"Deep Research requires GEMINI_WEB_SECURE_1PSID (gemini.google.com cookie) in ~/.pi/agent/.env.local — guest mode does not support it. Deep Research also needs a Gemini Advanced subscription on the account.",
|
|
331
|
+
);
|
|
332
|
+
}
|
|
333
|
+
const timeoutMs = opts.timeoutMs ?? 600_000;
|
|
334
|
+
const result = await raceGuard(
|
|
335
|
+
withGeminiClient(
|
|
336
|
+
opts.config,
|
|
337
|
+
(client) =>
|
|
338
|
+
client.research(query, {
|
|
339
|
+
wait: true,
|
|
340
|
+
pollInterval: 10_000,
|
|
341
|
+
timeout: timeoutMs,
|
|
342
|
+
...(opts.onStatus ? { onStatus: opts.onStatus } : {}),
|
|
343
|
+
}),
|
|
344
|
+
opts.factory,
|
|
345
|
+
),
|
|
346
|
+
// +5s grace so the client's own poll-timeout (better semantics) fires first.
|
|
347
|
+
{ signal: opts.signal, timeoutMs: timeoutMs + 5_000, label: "web_research research" },
|
|
348
|
+
);
|
|
349
|
+
const text = String(result?.text ?? "");
|
|
350
|
+
return {
|
|
351
|
+
text,
|
|
352
|
+
title: result?.plan?.title ?? null,
|
|
353
|
+
eta: result?.plan?.eta_text ?? null,
|
|
354
|
+
guest: false,
|
|
355
|
+
sources: extractSources(text),
|
|
356
|
+
};
|
|
357
|
+
}
|
|
358
|
+
|
|
359
|
+
// ---------------------------------------------------------------------------
|
|
360
|
+
// Image generation (free web tier): prompt → GeneratedImage.save() paths
|
|
361
|
+
// ---------------------------------------------------------------------------
|
|
362
|
+
|
|
363
|
+
export interface GeminiImageResult {
|
|
364
|
+
paths: string[];
|
|
365
|
+
model?: string;
|
|
366
|
+
guest: boolean;
|
|
367
|
+
text: string;
|
|
368
|
+
}
|
|
369
|
+
|
|
370
|
+
export async function geminiGenerateImage(
|
|
371
|
+
prompt: string,
|
|
372
|
+
opts: {
|
|
373
|
+
config: GeminiWebConfig;
|
|
374
|
+
outDir: string;
|
|
375
|
+
model?: string;
|
|
376
|
+
timeoutMs?: number;
|
|
377
|
+
signal?: AbortSignal;
|
|
378
|
+
factory?: GeminiClientFactory;
|
|
379
|
+
},
|
|
380
|
+
): Promise<GeminiImageResult> {
|
|
381
|
+
const out = await raceGuard(
|
|
382
|
+
withGeminiClient(
|
|
383
|
+
opts.config,
|
|
384
|
+
async (client) => {
|
|
385
|
+
if (typeof client.newChat !== "function") {
|
|
386
|
+
throw new Error("gemini-reverse client exposes no newChat (unexpected package shape)");
|
|
387
|
+
}
|
|
388
|
+
const chat = client.newChat(opts.model ? { model: opts.model } : undefined);
|
|
389
|
+
return chat.generateContent({ prompt });
|
|
390
|
+
},
|
|
391
|
+
opts.factory,
|
|
392
|
+
),
|
|
393
|
+
{ signal: opts.signal, timeoutMs: opts.timeoutMs ?? 180_000, label: "web_image gemini" },
|
|
394
|
+
);
|
|
395
|
+
const images = out?.generated_images ?? out?.images ?? [];
|
|
396
|
+
const text = String(out?.text ?? "");
|
|
397
|
+
if (!images.length) {
|
|
398
|
+
throw new Error(
|
|
399
|
+
text
|
|
400
|
+
? `Gemini replied with text but no images: ${text.slice(0, 200)} — image generation is likely unavailable for this account/region (it is for some Google accounts); try provider=zai (ZAI_API_KEY) or provider=custom.`
|
|
401
|
+
: "Gemini returned no images — generation may be unavailable for this account/region (guest mode may not support it; set GEMINI_WEB_SECURE_1PSID).",
|
|
402
|
+
);
|
|
403
|
+
}
|
|
404
|
+
const paths: string[] = [];
|
|
405
|
+
for (const img of images) paths.push(await img.save({ path: opts.outDir }));
|
|
406
|
+
return { paths, model: out?.model, guest: !opts.config.psid, text };
|
|
407
|
+
}
|
|
@@ -0,0 +1,354 @@
|
|
|
1
|
+
// Generic OpenAI-compatible images client + fallback chain for web_image.
|
|
2
|
+
// Serves the `zai` preset (official api.z.ai, CogView-4) and any `custom`
|
|
3
|
+
// OpenAI-images endpoint — direct-to-upstream plain fetch, no self-host.
|
|
4
|
+
|
|
5
|
+
import fs from "node:fs";
|
|
6
|
+
import path from "node:path";
|
|
7
|
+
import { randomUUID } from "node:crypto";
|
|
8
|
+
import { findEnvValue } from "./config";
|
|
9
|
+
import {
|
|
10
|
+
describeGeminiError,
|
|
11
|
+
geminiGenerateImage,
|
|
12
|
+
raceGuard,
|
|
13
|
+
type GeminiClientFactory,
|
|
14
|
+
type GeminiWebConfig,
|
|
15
|
+
} from "./gemini";
|
|
16
|
+
|
|
17
|
+
// ---------------------------------------------------------------------------
|
|
18
|
+
// Config
|
|
19
|
+
// ---------------------------------------------------------------------------
|
|
20
|
+
|
|
21
|
+
export const ZAI_PRESET = { baseUrl: "https://api.z.ai/api/paas/v4", defaultModel: "cogview-4" } as const;
|
|
22
|
+
|
|
23
|
+
export interface ImageApiConfig {
|
|
24
|
+
zai?: { apiKey: string; source: string };
|
|
25
|
+
custom?: { baseUrl: string; apiKey?: string; label: string; source: string };
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
export function loadImageApiConfig(cwd = process.cwd(), includeCwdEnv = false): ImageApiConfig {
|
|
29
|
+
const zaiKey = findEnvValue("ZAI_API_KEY", cwd, includeCwdEnv);
|
|
30
|
+
const zaiFound = zaiKey.value ? zaiKey : findEnvValue("Z_AI_API_KEY", cwd, includeCwdEnv);
|
|
31
|
+
const base = findEnvValue("WEB_IMAGE_API_BASE_URL", cwd, includeCwdEnv);
|
|
32
|
+
const key = findEnvValue("WEB_IMAGE_API_KEY", cwd, includeCwdEnv);
|
|
33
|
+
const label = findEnvValue("WEB_IMAGE_API_LABEL", cwd, includeCwdEnv);
|
|
34
|
+
const cfg: ImageApiConfig = {};
|
|
35
|
+
if (zaiFound.value) cfg.zai = { apiKey: zaiFound.value, source: zaiFound.source };
|
|
36
|
+
if (base.value) {
|
|
37
|
+
cfg.custom = {
|
|
38
|
+
baseUrl: base.value.replace(/\/+$/, ""),
|
|
39
|
+
...(key.value ? { apiKey: key.value } : {}),
|
|
40
|
+
label: label.value || hostOf(base.value),
|
|
41
|
+
source: base.source,
|
|
42
|
+
};
|
|
43
|
+
}
|
|
44
|
+
return cfg;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function hostOf(url: string): string {
|
|
48
|
+
try {
|
|
49
|
+
return new URL(url).host;
|
|
50
|
+
} catch {
|
|
51
|
+
return url;
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
export interface ImageRateConfig {
|
|
56
|
+
minIntervalMs: number;
|
|
57
|
+
dailyCap: number; // applies to the gemini web tier only — keyed APIs stay uncapped
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
export function loadImageRateConfig(cwd = process.cwd(), includeCwdEnv = false): ImageRateConfig {
|
|
61
|
+
const interval = findEnvValue("WEB_IMAGE_MIN_INTERVAL_MS", cwd, includeCwdEnv);
|
|
62
|
+
const cap = findEnvValue("WEB_IMAGE_DAILY_CAP", cwd, includeCwdEnv);
|
|
63
|
+
const intervalNum = Number(interval.value);
|
|
64
|
+
const capNum = Number(cap.value);
|
|
65
|
+
return {
|
|
66
|
+
minIntervalMs: interval.value && Number.isFinite(intervalNum) ? Math.max(0, Math.trunc(intervalNum)) : 5000,
|
|
67
|
+
dailyCap: cap.value && Number.isFinite(capNum) ? Math.max(1, Math.trunc(capNum)) : 20,
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
// ---------------------------------------------------------------------------
|
|
72
|
+
// Soft rate guardrails (in-memory, reset on restart). Successful generations
|
|
73
|
+
// only — failures don't consume quota.
|
|
74
|
+
// ---------------------------------------------------------------------------
|
|
75
|
+
|
|
76
|
+
interface RateState {
|
|
77
|
+
lastAt: number;
|
|
78
|
+
day: string;
|
|
79
|
+
count: number;
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
const rate = new Map<string, RateState>();
|
|
83
|
+
let nowMs = () => Date.now();
|
|
84
|
+
|
|
85
|
+
/** @internal test hooks */
|
|
86
|
+
export function __setImageRateClock(fn: () => number): void {
|
|
87
|
+
nowMs = fn;
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/** @internal test hooks */
|
|
91
|
+
export function __resetImageRate(): void {
|
|
92
|
+
rate.clear();
|
|
93
|
+
nowMs = () => Date.now();
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
const utcDay = (ts: number) => new Date(ts).toISOString().slice(0, 10);
|
|
97
|
+
|
|
98
|
+
function msUntilUtcRoll(ts: number): number {
|
|
99
|
+
const d = new Date(ts);
|
|
100
|
+
return Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate() + 1) - ts;
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
export type RateVerdict = { ok: true } | { ok: false; reason: string; retryAfterMs: number };
|
|
104
|
+
|
|
105
|
+
export function imageRateCheck(provider: "gemini" | "zai" | "custom", rateCfg: ImageRateConfig): RateVerdict {
|
|
106
|
+
const state = rate.get(provider);
|
|
107
|
+
const ts = nowMs();
|
|
108
|
+
if (state && rateCfg.minIntervalMs > 0) {
|
|
109
|
+
const elapsed = ts - state.lastAt;
|
|
110
|
+
if (elapsed < rateCfg.minIntervalMs) {
|
|
111
|
+
return {
|
|
112
|
+
ok: false,
|
|
113
|
+
reason: `min interval ${rateCfg.minIntervalMs}ms between calls (elapsed ${elapsed}ms, WEB_IMAGE_MIN_INTERVAL_MS)`,
|
|
114
|
+
retryAfterMs: rateCfg.minIntervalMs - elapsed,
|
|
115
|
+
};
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
if (provider === "gemini" && state && state.day === utcDay(ts) && state.count >= rateCfg.dailyCap) {
|
|
119
|
+
return {
|
|
120
|
+
ok: false,
|
|
121
|
+
reason: `daily soft cap reached (${rateCfg.dailyCap}/day, WEB_IMAGE_DAILY_CAP)`,
|
|
122
|
+
retryAfterMs: msUntilUtcRoll(ts),
|
|
123
|
+
};
|
|
124
|
+
}
|
|
125
|
+
return { ok: true };
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
export function imageRateRecord(provider: "gemini" | "zai" | "custom", count = 1): void {
|
|
129
|
+
const ts = nowMs();
|
|
130
|
+
const day = utcDay(ts);
|
|
131
|
+
const prev = rate.get(provider);
|
|
132
|
+
const sameDay = prev?.day === day;
|
|
133
|
+
rate.set(provider, { lastAt: ts, day, count: (sameDay ? prev!.count : 0) + count });
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
export function imageRateSnapshot(): Record<string, { count: number; day: string; msSinceLast: number }> {
|
|
137
|
+
const ts = nowMs();
|
|
138
|
+
const out: Record<string, { count: number; day: string; msSinceLast: number }> = {};
|
|
139
|
+
for (const [provider, s] of rate) {
|
|
140
|
+
out[provider] = { count: s.day === utcDay(ts) ? s.count : 0, day: s.day, msSinceLast: Math.max(0, ts - s.lastAt) };
|
|
141
|
+
}
|
|
142
|
+
return out;
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
// ---------------------------------------------------------------------------
|
|
146
|
+
// Single-provider OpenAI-images call (POST {base}/images/generations)
|
|
147
|
+
// ---------------------------------------------------------------------------
|
|
148
|
+
|
|
149
|
+
export interface FetchLike {
|
|
150
|
+
(
|
|
151
|
+
url: string,
|
|
152
|
+
init?: { method?: string; headers?: Record<string, string>; body?: string; signal?: AbortSignal },
|
|
153
|
+
): Promise<{ ok: boolean; status: number; statusText?: string; json(): Promise<unknown>; arrayBuffer(): Promise<ArrayBuffer> }>;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
export interface ApiImageResult {
|
|
157
|
+
paths: string[];
|
|
158
|
+
model?: string;
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
export class ImageApiError extends Error {
|
|
162
|
+
constructor(
|
|
163
|
+
public status: number,
|
|
164
|
+
message: string,
|
|
165
|
+
) {
|
|
166
|
+
super(message);
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
export function describeImageApiError(err: unknown): string {
|
|
171
|
+
if (err instanceof ImageApiError) {
|
|
172
|
+
if (err.status === 401 || err.status === 403) return `upstream rejected the API key (HTTP ${err.status}): ${err.message}`;
|
|
173
|
+
if (err.status === 429) return `upstream rate limit/quota exhausted (HTTP 429): ${err.message}`;
|
|
174
|
+
if (err.status >= 500) return `upstream server error (HTTP ${err.status}): ${err.message}`;
|
|
175
|
+
return `upstream error (HTTP ${err.status}): ${err.message}`;
|
|
176
|
+
}
|
|
177
|
+
return err instanceof Error ? err.message : String(err);
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
export async function apiGenerateImage(opts: {
|
|
181
|
+
baseUrl: string;
|
|
182
|
+
apiKey?: string;
|
|
183
|
+
model?: string;
|
|
184
|
+
prompt: string;
|
|
185
|
+
n?: number;
|
|
186
|
+
outDir: string;
|
|
187
|
+
timeoutMs?: number;
|
|
188
|
+
signal?: AbortSignal;
|
|
189
|
+
fetchImpl?: FetchLike;
|
|
190
|
+
}): Promise<ApiImageResult> {
|
|
191
|
+
const fetchImpl = opts.fetchImpl ?? (fetch as FetchLike);
|
|
192
|
+
const url = `${opts.baseUrl.replace(/\/+$/, "")}/images/generations`;
|
|
193
|
+
const res = await raceGuard(
|
|
194
|
+
fetchImpl(url, {
|
|
195
|
+
method: "POST",
|
|
196
|
+
headers: { "Content-Type": "application/json", ...(opts.apiKey ? { Authorization: `Bearer ${opts.apiKey}` } : {}) },
|
|
197
|
+
body: JSON.stringify({ model: opts.model, prompt: opts.prompt, n: opts.n ?? 1 }),
|
|
198
|
+
signal: opts.signal,
|
|
199
|
+
}),
|
|
200
|
+
{ signal: opts.signal, timeoutMs: opts.timeoutMs ?? 180_000, label: "web_image api" },
|
|
201
|
+
);
|
|
202
|
+
const payload = (await res.json().catch(() => null)) as Record<string, unknown> | null;
|
|
203
|
+
if (!res.ok) {
|
|
204
|
+
const errObj = payload?.error as { message?: string } | undefined;
|
|
205
|
+
const msg =
|
|
206
|
+
errObj?.message ??
|
|
207
|
+
(typeof payload?.message === "string" ? payload.message : undefined) ??
|
|
208
|
+
(payload ? JSON.stringify(payload).slice(0, 300) : res.statusText ?? "");
|
|
209
|
+
throw new ImageApiError(res.status, String(msg));
|
|
210
|
+
}
|
|
211
|
+
const items = Array.isArray(payload?.data) ? (payload!.data as Array<Record<string, unknown>>) : [];
|
|
212
|
+
if (!items.length) throw new Error(`upstream returned no image data (model ${opts.model ?? "default"})`);
|
|
213
|
+
fs.mkdirSync(opts.outDir, { recursive: true });
|
|
214
|
+
const paths: string[] = [];
|
|
215
|
+
for (let i = 0; i < items.length; i++) {
|
|
216
|
+
const item = items[i];
|
|
217
|
+
if (typeof item?.b64_json === "string" && item.b64_json) {
|
|
218
|
+
paths.push(writeB64(opts.outDir, item.b64_json, i));
|
|
219
|
+
} else if (typeof item?.url === "string" && item.url) {
|
|
220
|
+
paths.push(await downloadImage(fetchImpl, item.url, opts.outDir, i, opts.signal, opts.timeoutMs));
|
|
221
|
+
} else {
|
|
222
|
+
throw new Error(`image item ${i} had neither b64_json nor url`);
|
|
223
|
+
}
|
|
224
|
+
}
|
|
225
|
+
return { paths, model: typeof payload?.model === "string" ? payload.model : opts.model };
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
function writeB64(outDir: string, b64: string, i: number): string {
|
|
229
|
+
const file = path.join(outDir, `pi-web-image-${randomUUID().slice(0, 8)}-${i}.png`);
|
|
230
|
+
fs.writeFileSync(file, Buffer.from(b64, "base64"));
|
|
231
|
+
return file;
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
async function downloadImage(
|
|
235
|
+
fetchImpl: FetchLike,
|
|
236
|
+
url: string,
|
|
237
|
+
outDir: string,
|
|
238
|
+
i: number,
|
|
239
|
+
signal?: AbortSignal,
|
|
240
|
+
timeoutMs?: number,
|
|
241
|
+
): Promise<string> {
|
|
242
|
+
const res = await raceGuard(fetchImpl(url, { method: "GET", signal }), {
|
|
243
|
+
signal,
|
|
244
|
+
timeoutMs: timeoutMs ?? 120_000,
|
|
245
|
+
label: "web_image download",
|
|
246
|
+
});
|
|
247
|
+
if (!res.ok) throw new ImageApiError(res.status, `image download failed (HTTP ${res.status})`);
|
|
248
|
+
const buf = Buffer.from(await res.arrayBuffer());
|
|
249
|
+
const file = path.join(outDir, `pi-web-image-${randomUUID().slice(0, 8)}-${i}${extFor(url)}`);
|
|
250
|
+
fs.writeFileSync(file, buf);
|
|
251
|
+
return file;
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
function extFor(url: string): string {
|
|
255
|
+
const m = /\.(png|jpe?g|webp|gif)(\?|$)/i.exec(url);
|
|
256
|
+
const ext = (m?.[1] ?? "png").toLowerCase();
|
|
257
|
+
return `.${ext === "jpeg" ? "jpg" : ext}`;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
// ---------------------------------------------------------------------------
|
|
261
|
+
// Fallback chain: gemini (free web tier) → zai (official API) → custom
|
|
262
|
+
// ---------------------------------------------------------------------------
|
|
263
|
+
|
|
264
|
+
export type ImageProvider = "gemini" | "zai" | "custom";
|
|
265
|
+
|
|
266
|
+
export interface ImageChainResult {
|
|
267
|
+
provider: ImageProvider;
|
|
268
|
+
model?: string;
|
|
269
|
+
paths: string[];
|
|
270
|
+
attempts: string[];
|
|
271
|
+
}
|
|
272
|
+
|
|
273
|
+
export interface ImageChainParams {
|
|
274
|
+
prompt: string;
|
|
275
|
+
model?: string;
|
|
276
|
+
n?: number;
|
|
277
|
+
outDir: string;
|
|
278
|
+
provider: "auto" | ImageProvider;
|
|
279
|
+
geminiConfig: GeminiWebConfig;
|
|
280
|
+
apiConfig: ImageApiConfig;
|
|
281
|
+
rateConfig: ImageRateConfig;
|
|
282
|
+
timeoutMs?: number;
|
|
283
|
+
signal?: AbortSignal;
|
|
284
|
+
/** @internal test injection */
|
|
285
|
+
geminiFactory?: GeminiClientFactory;
|
|
286
|
+
/** @internal test injection */
|
|
287
|
+
fetchImpl?: FetchLike;
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
function chainFor(provider: "auto" | ImageProvider): ImageProvider[] {
|
|
291
|
+
// Auto includes ALL providers: unconfigured ones contribute "not configured
|
|
292
|
+
// (set …)" hints to the aggregated error instead of vanishing silently.
|
|
293
|
+
return provider === "auto" ? ["gemini", "zai", "custom"] : [provider];
|
|
294
|
+
}
|
|
295
|
+
|
|
296
|
+
export async function generateImageWithFallback(params: ImageChainParams): Promise<ImageChainResult> {
|
|
297
|
+
const chain = chainFor(params.provider);
|
|
298
|
+
const attempts: string[] = [];
|
|
299
|
+
for (const provider of chain) {
|
|
300
|
+
const configured =
|
|
301
|
+
provider === "gemini" ? true : provider === "zai" ? Boolean(params.apiConfig.zai) : Boolean(params.apiConfig.custom);
|
|
302
|
+
if (!configured) {
|
|
303
|
+
attempts.push(`${provider}: not configured${provider === "zai" ? " (set ZAI_API_KEY)" : " (set WEB_IMAGE_API_BASE_URL)"}`);
|
|
304
|
+
continue;
|
|
305
|
+
}
|
|
306
|
+
const rate = imageRateCheck(provider, params.rateConfig);
|
|
307
|
+
if (!rate.ok) {
|
|
308
|
+
attempts.push(`${provider}: skipped — ${rate.reason}`);
|
|
309
|
+
continue;
|
|
310
|
+
}
|
|
311
|
+
try {
|
|
312
|
+
let result: { paths: string[]; model?: string };
|
|
313
|
+
if (provider === "gemini") {
|
|
314
|
+
result = await geminiGenerateImage(params.prompt, {
|
|
315
|
+
config: params.geminiConfig,
|
|
316
|
+
outDir: params.outDir,
|
|
317
|
+
model: params.model,
|
|
318
|
+
timeoutMs: params.timeoutMs,
|
|
319
|
+
signal: params.signal,
|
|
320
|
+
factory: params.geminiFactory,
|
|
321
|
+
});
|
|
322
|
+
} else if (provider === "zai") {
|
|
323
|
+
result = await apiGenerateImage({
|
|
324
|
+
baseUrl: ZAI_PRESET.baseUrl,
|
|
325
|
+
apiKey: params.apiConfig.zai!.apiKey,
|
|
326
|
+
model: params.model ?? ZAI_PRESET.defaultModel,
|
|
327
|
+
prompt: params.prompt,
|
|
328
|
+
n: params.n,
|
|
329
|
+
outDir: params.outDir,
|
|
330
|
+
timeoutMs: params.timeoutMs,
|
|
331
|
+
signal: params.signal,
|
|
332
|
+
fetchImpl: params.fetchImpl,
|
|
333
|
+
});
|
|
334
|
+
} else {
|
|
335
|
+
result = await apiGenerateImage({
|
|
336
|
+
baseUrl: params.apiConfig.custom!.baseUrl,
|
|
337
|
+
apiKey: params.apiConfig.custom!.apiKey,
|
|
338
|
+
model: params.model,
|
|
339
|
+
prompt: params.prompt,
|
|
340
|
+
n: params.n,
|
|
341
|
+
outDir: params.outDir,
|
|
342
|
+
timeoutMs: params.timeoutMs,
|
|
343
|
+
signal: params.signal,
|
|
344
|
+
fetchImpl: params.fetchImpl,
|
|
345
|
+
});
|
|
346
|
+
}
|
|
347
|
+
imageRateRecord(provider);
|
|
348
|
+
return { provider, model: result.model, paths: result.paths, attempts };
|
|
349
|
+
} catch (err) {
|
|
350
|
+
attempts.push(`${provider}: ${provider === "gemini" ? describeGeminiError(err) : describeImageApiError(err)}`);
|
|
351
|
+
}
|
|
352
|
+
}
|
|
353
|
+
throw new Error(`All image providers failed:\n${attempts.map((a) => `- ${a}`).join("\n")}`);
|
|
354
|
+
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bacnh85/pi-web",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Pi extension for web search, page extraction, Firecrawl scraping/crawling,
|
|
3
|
+
"version": "0.9.0",
|
|
4
|
+
"description": "Pi extension for web search, page extraction, Firecrawl scraping/crawling, Crawl4AI headless browser crawling, Gemini web-tier research, and free upstream image generation.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"publishConfig": {
|
|
@@ -21,7 +21,12 @@
|
|
|
21
21
|
"firecrawl",
|
|
22
22
|
"crawl4ai",
|
|
23
23
|
"scraping",
|
|
24
|
-
"crawling"
|
|
24
|
+
"crawling",
|
|
25
|
+
"gemini",
|
|
26
|
+
"deep-research",
|
|
27
|
+
"image-generation",
|
|
28
|
+
"zai",
|
|
29
|
+
"cogview"
|
|
25
30
|
],
|
|
26
31
|
"scripts": {
|
|
27
32
|
"test": "cd extensions && npx mocha"
|
|
@@ -44,6 +49,7 @@
|
|
|
44
49
|
},
|
|
45
50
|
"dependencies": {
|
|
46
51
|
"@mozilla/readability": "^0.6.0",
|
|
52
|
+
"gemini-reverse": "^2.1.0",
|
|
47
53
|
"jsdom": "^27.0.1",
|
|
48
54
|
"turndown": "^7.2.2",
|
|
49
55
|
"turndown-plugin-gfm": "^1.0.2"
|
package/skills/pi-web/SKILL.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: pi-web
|
|
3
|
-
description: Web search, content extraction, site crawling,
|
|
3
|
+
description: Web search, content extraction, site crawling, page capture, Gemini web-tier research, and free upstream image generation via the pi-web extension. Use when the user needs current web search results, documentation lookup, factual research, AI-synthesized research with sources (Gemini Deep Research), image generation from text (Gemini web, Z.ai CogView-4), source discovery, URL-to-markdown extraction, JSON extraction from websites, site URL discovery, site crawling, or page screenshots/PDFs. Use when the user mentions searching the web, finding docs, looking something up, researching deeply, generating/creating an image, scraping/extracting content from a URL, or capturing a page.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# pi-web — Unified Web Tools
|
|
7
7
|
|
|
8
|
-
Use the **
|
|
8
|
+
Use the **9 unified tools** from the `pi-web` extension for all web-related tasks. These tools automatically select the best backend from SearXNG, Brave Search, Firecrawl, Crawl4AI, and agy (when installed) — you don't need to know which backend to use. Search selection is adaptive: broad discovery prefers self-hosted SearXNG, while precision-sensitive queries and inline content prefer Brave.
|
|
9
9
|
|
|
10
10
|
## Quick Reference
|
|
11
11
|
|
|
@@ -17,6 +17,8 @@ Use the **7 unified tools** from the `pi-web` extension for all web-related task
|
|
|
17
17
|
| `web_crawl` | Crawl multiple pages from a site | Light (Firecrawl) or Full (Crawl4AI) |
|
|
18
18
|
| `web_screenshot` | Capture page screenshot as PNG | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
|
|
19
19
|
| `web_pdf` | Generate page PDF | Crawl4AI daemon (public URLs) or local headless Chrome (localhost/LAN/file URLs — auto-detected) |
|
|
20
|
+
| `web_research` | AI-synthesized research with sources | Gemini web tier: ask = grounded answer (guest OK); research = Deep Research report (cookie + Gemini Advanced) |
|
|
21
|
+
| `web_image` | Generate images from a text prompt | Gemini web (guest/cookie) → Z.ai CogView-4 (`ZAI_API_KEY`) → custom OpenAI-images endpoint |
|
|
20
22
|
| `web_status` | Check provider configuration and health | — |
|
|
21
23
|
|
|
22
24
|
## Decision Tree
|
|
@@ -53,6 +55,16 @@ What do you need?
|
|
|
53
55
|
├── Printable/archivable PDF of a page
|
|
54
56
|
│ → web_pdf
|
|
55
57
|
│
|
|
58
|
+
├── AI-synthesized research with sources (synthesis, comparisons, reports)
|
|
59
|
+
│ → web_research
|
|
60
|
+
│ ├─ quick grounded answer: mode=ask (default; guest OK, Flash-only)
|
|
61
|
+
│ └─ multi-minute Deep Research report: mode=research (needs cookie + Gemini Advanced)
|
|
62
|
+
│
|
|
63
|
+
├── Generate an image from a text prompt (NOT capturing an existing page)
|
|
64
|
+
│ → web_image
|
|
65
|
+
│ ├─ default: provider=auto (Gemini web → Z.ai CogView-4 → custom endpoint)
|
|
66
|
+
│ └─ pin/model: provider=zai model=cogview-4, or any custom OpenAI-images endpoint
|
|
67
|
+
│
|
|
56
68
|
└── Check what web tools are configured
|
|
57
69
|
→ web_status
|
|
58
70
|
```
|