pi-multimodal-proxy 1.19.1 → 1.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,27 @@ All notable changes to this project will be documented in this file.
4
4
 
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
6
6
 
7
+ ## [1.21.0] - 2026-10-03
8
+
9
+ ### Added
10
+
11
+ - **Compact transcript rows via `pi.registerToolRenderer()` (Pi ≥ 1.0.1, feature-detected)** — `analyze_image` and `generate_image` calls now render as one compact, themed line each instead of the default JSON-args dump. The `analyze_image` call line shows the question (word-boundary truncated, surrogate-pair safe), image references (recall ids → `image="deadbeef…"`, file paths → file name, `+N more` beyond two shown), a crop summary (`bottom-right@0` / `25%,13% 50%×50%@1` / `128,64 256×256@0` / `crops ×N`), and a model override; `generate_image` shows the prompt and override. Result lines summarize `✓ provider/model · cached · 1.2s` for analyses and `✓ N images via provider/model · $0.0311 · image="<id8>…"` for generations, or `✗ <error>`; expanding a row reveals the full text content. Generated images still render inline from the result's image blocks (independent of `renderResult`), reliable across Kitty/Ghostty/WezTerm/Warp via pi 1.0.1's JPEG/WebP/GIF `Image` fixes.
12
+ - **Renderer coverage for unregistered tools** — registering through `registerToolRenderer` rather than inline `renderCall`/`renderResult` on the tool definitions means the compact rows also apply to calls replayed when the tool is *not registered* at render time: sessions resumed with the tool disabled (`/multimodal-proxy tool off`, mode off), transcript rows rendered before extension load, and HTML exports. Resolver chaining (`next() ?? {}`) preserves other extensions' renderers for the keys we don't set.
13
+ - **Tool-result `details` payloads** — both tools now return a display-sized `details` object (analyze: `ok`/`error`/`cached`/`provider`/`model`/`latencyMs`/`groundingFormat`, with the heavy `text`/`images` stripped; generate: the structured payload as-is — ids/dims/cost) that feeds the renderer and persists in the session log without ever reaching the model.
14
+ - Defensive renderer plumbing: pi-tui's `Text` (ANSI-aware wrapping) is loaded lazily with a width-truncating plain-lines fallback (ANSI-aware, rough wcwidth — pi's differential renderer stops on overwide lines) so an unresolvable package can never block extension load nor crash rendering; the expanded result view renders **every** text block, so a `generate_image` row on a text-only base model (where the `tool_result` hook replaces image blocks with description fences) still shows the per-image descriptions; renderers tolerate partial/garbage streaming args and missing result objects. (PR #37 review: CodeRabbit.)
15
+ - New `tool-render.test.ts` (24 tests: truncation with a surrogate-pair regression trap, ref/crop summarization, call and result lines, garbage tolerance, details stripping) plus a fake-host smoke script (`tools/smoke-renderers.mjs`, real ANSI theme) that loads the real extension module and exercises the resolver contract end-to-end — 540 tests total, all green; also verified loading under the real pi 1.0.1 RPC loader.
16
+
17
+ ### Changed
18
+
19
+ - Nothing user-visible on pi < 1.0.1 hosts: registration is gated on `typeof pi.registerToolRenderer === "function"`, so older runtimes keep the default tool rendering with zero behavior change.
20
+
21
+ ## [1.20.0] - 2026-10-02
22
+
23
+ ### Added
24
+
25
+ - **`generate_image` tool (Pi ≥ 1.0, feature-detected)** — agent-facing image creation through `ModelRegistry.generateImages()` with the session's credentials and request-time auth; usage counts toward session cost. Parameters: `prompt` (1..4000 chars), optional `model` (`provider/model-id` of an image-generation model). Model resolution walks `getAvailableOfType("image")` (cached per session): without an override the first keyed image model wins; a mistyped override errors with recovery-style close matches and a malformed one gets the expected format (Pi 1.0 codemode-error philosophy). The prompt leaves only after the data-egress consent gate (`/multimodal-proxy consent yes` / `allowed-providers`). Results carry the generated image blocks (vision-capable base models see the pixels directly) plus a summary text naming each `image="<id>"` id, and an `outputSchema`-shaped `structuredContent` (provider, model, per-image id/mimeType/dimensions, `costUSD` when the provider reported usage; `usage` passed through on the tool result). Generated images are persisted in the per-session recall stores (`imageData`/`imageMeta`) with `generated-<hash8>.<ext>` filename hints, so the existing pipeline takes over: for text-only base models the `tool_result` hook auto-describes them via the vision model and appends `DescriptionEntry`s (making `#` recall work), and `analyze_image` can re-query or crop them by id regardless of base-model modality. Shares the per-turn tool-call budget with `analyze_image` (10/turn); annotated `readOnlyHint: false` (cost-incurring content creation) and namespaced `multimodal-proxy`. Registered only when the runtime exposes `generateImages` — pi < 1.0 never sees the tool.
26
+ - Five new tests for `resolveImageGenModelChoice` (default-first, exact override, close-match suggestions on typos, expected-format error, actionable hint when nothing is keyed) — 512 total, all green.
27
+
7
28
  ## [1.19.1] - 2026-10-02
8
29
 
9
30
  ### Changed
package/README.md CHANGED
@@ -8,10 +8,35 @@ When **video or audio files** are detected, they are routed to a **multimodal mo
8
8
 
9
9
  **YouTube links** are detected too: paste a URL (`youtube.com/watch?v=…`, `youtu.be/…`, `/shorts/…`, etc.) and the video is downloaded with [`yt-dlp`](https://github.com/yt-dlp/yt-dlp) and analyzed exactly like a local file.
10
10
 
11
+ **Compact tool rows (Pi ≥ 1.0.1)** — `analyze_image` and `generate_image` calls render as one tidy line each (`analyze_image "What does the legend say?" · 2 images (image="deadbeef…", shot.png) · crop bottom-right@0`) with themed result summaries (`✓ zai/glm-5.3-flash · cached · 1.2s`, `✓ 1 image via google/imagen-4 · $0.0311`), instead of the default JSON-args dump. Generated images still render inline from the result blocks; expand a row to see the full analysis fence.
12
+
13
+ ## What's new in 1.21.0
14
+
15
+ Adopting Pi 1.0.1's new `pi.registerToolRenderer()` API:
16
+
17
+ - **Compact transcript rows for `analyze_image`** — the call line shows the question (truncated on a word boundary), the image refs (recall ids shortened to `image="deadbeef…"`, paths to their file name, `+N more` beyond two), a crop summary (`bottom-right@0`, `25%,13% 50%×50%@1`, `128,64 256×256@0`, or `crops ×N`), and any model override — instead of the raw JSON arguments. The result line summarizes `✓ provider/model · cached · latency` (or `✗ <error>`), and expanding the row reveals the full analysis fence.
18
+ - **Compact transcript rows for `generate_image`** — the call line shows the prompt (and any model override); the result line summarizes `✓ N images via provider/model · $cost · image="<id8>…"`. The generated images themselves keep rendering inline — pi draws image blocks from the result content next to the renderer output — so this is purely a better header/summary, now reliable across Kitty, Ghostty, WezTerm, and Warp thanks to pi 1.0.1's JPEG/WebP/GIF image fixes.
19
+ - **Works even when the tool isn't registered** — registering through `registerToolRenderer` (instead of inline `renderCall`/`renderResult` on the tool definitions) covers rows replayed from a session where the tool is now disabled (`/multimodal-proxy tool off`, mode off), resumed sessions, and HTML exports. `next()` chaining preserves renderers from other extensions for the keys we don't set.
20
+ - **Renderer state via tool `details`** — results now carry a display-sized `details` payload (provider/model, `cached`, `latencyMs`, image ids, `costUSD`; heavy fields stripped) that feeds the renderer but never reaches the model.
21
+ - Fully feature-gated (`typeof pi.registerToolRenderer === "function"`): pi < 1.0.1 hosts keep the default rendering, zero behavior change. pi-tui's `Text` component is loaded lazily with a plain-lines fallback so a missing package can never block extension load.
22
+
23
+ Also benefits from pi 1.0.1 host fixes (no code needed here): Anthropic prompt caches survive mid-session tool toggles (tools added/redefined mid-conversation are now defined inline), "model at capacity" errors are retried instead of ending the turn, and Bedrock Claude thinking-signature failures after tool/prompt changes are fixed.
24
+
25
+ ## What's new in 1.20.0
26
+
27
+ Image generation, riding Pi 1.0's new `models.generateImages()` API:
28
+
29
+ - **`generate_image` tool** — agent-facing image creation with the session's image-generation models (feature-detected; silently absent on pi < 1.0). Prompt leaves only after the data-egress consent gate; the call reports usage/cost through the normal session accounting.
30
+ - **Generated images join the pipeline** — results return as image blocks (vision base models see the pixels) **and** are persisted in the recall stores, so a text-only base model gets an automatic vision-model description (same as any tool-returned image), `#` recall works once described, and `analyze_image` can re-query or crop them via their `image="<id>"` ids.
31
+ - **Codemode-friendly** — `outputSchema`-shaped `structuredContent` (provider, model, per-image ids/dimensions, `costUSD`) so scripts consume results programmatically; leans into Pi 1.0's leaner codemode with a compact description and recovery-style errors (close-match model suggestions, expected `provider/model-id` format, actionable `pi --login` hint when nothing is keyed).
32
+ - **Shared cost guard** — `generate_image` counts against the same per-turn tool-call budget as `analyze_image`.
33
+
11
34
  ## What's new in 1.19.0
12
35
 
13
36
  Deep integration with recent Pi releases (0.87 / 0.99):
14
37
 
38
+ - **`generate_image` tool (Pi ≥ 1.0)** — the agent can now **create** images, not just understand them. Calls run through `models.generateImages()` with the session's credentials (usage counts toward session cost), the model defaults to the first keyed image-generation model (override per call with `model: "provider/model-id"`), results carry image blocks plus stable `image="<id>"` ids, and the existing pipeline takes over automatically: text-only base models get an auto-description via your vision model, and every generated image is recallable (`#`) and re-analyzable/croppable with `analyze_image` like any attachment. Data-egress consent gates the provider before the first call.
39
+ - **Pi 1.0 alignment (1.19.1)** — leaner `analyze_image` description (~40% smaller — codemode lists tools under a 3000-token budget and drops what doesn't fit), recovery-style errors that name close matches (model typos, recall ids) and expected shapes (crops), and a dynamic implicit-vision-model fallback via `getAvailableOfType()` when the curated chain is entirely unkeyed.
15
40
  - **Virtual model — `Multimodal Auto`** (Pi ≥ 0.99, on by default): a new entry in `/model` that routes each request for you — turns whose pending input carries images go **natively** to your configured vision model (real pixels, no text fence), everything else stays on your base model (auto = whatever physical model handled the session before, or pin one with `/multimodal-proxy virtual-base`). Sticky routing keeps prompt caches valid within a turn; the footer shows the routed model and `/session` lists cost per physical model. Disable with `/multimodal-proxy virtual off`.
16
41
  - **`analyze_image` speaks the 0.99 tool API** — errors now surface as real tool errors (`isError`) instead of fake text, results carry a machine-readable `structuredContent` (analysis text, provider/model, latency, grounding format, per-image ids/dimensions/crop) so **codemode scripts** can consume them programmatically, and the tool is annotated (`readOnlyHint`) and namespaced (`multimodal-proxy`) for codemode listings.
17
42
  - **Model-aware upload downscale** (Pi ≥ 0.87) — when the configured vision model declares `inputLimits.images.resize` in its catalog entry, the proxy derives the downscale target, JPEG quality, and byte budget from it (tightening, never loosening, your `max-upload` settings). No more double-resizing against provider limits.
@@ -81,6 +81,7 @@ import {
81
81
  selectVisionModels,
82
82
  closeMatches,
83
83
  pickDynamicVisionCandidate,
84
+ resolveImageGenModelChoice,
84
85
  splitSubcommand,
85
86
  stripImagePaths,
86
87
  stripMediaPaths,
@@ -436,6 +437,56 @@ describe("pickDynamicVisionCandidate", () => {
436
437
  });
437
438
  });
438
439
 
440
+ describe("resolveImageGenModelChoice", () => {
441
+ const available = [
442
+ { provider: "openrouter", modelId: "gpt-image-1" },
443
+ { provider: "openrouter", modelId: "flux-schnell" },
444
+ { provider: "google", modelId: "imagen-4" },
445
+ ];
446
+
447
+ it("defaults to the first available model without an override", () => {
448
+ assert.deepEqual(resolveImageGenModelChoice(undefined, available), {
449
+ ok: true,
450
+ provider: "openrouter",
451
+ modelId: "gpt-image-1",
452
+ });
453
+ assert.deepEqual(resolveImageGenModelChoice("", available), {
454
+ ok: true,
455
+ provider: "openrouter",
456
+ modelId: "gpt-image-1",
457
+ });
458
+ });
459
+
460
+ it("accepts an exact override", () => {
461
+ assert.deepEqual(resolveImageGenModelChoice("google/imagen-4", available), {
462
+ ok: true,
463
+ provider: "google",
464
+ modelId: "imagen-4",
465
+ });
466
+ });
467
+
468
+ it("mistyped overrides carry recovery-style close matches", () => {
469
+ const out = resolveImageGenModelChoice("openrouter/gpt-imge-1", available);
470
+ assert.equal(out.ok, false);
471
+ if (!out.ok) {
472
+ assert.match(out.error, /not found/);
473
+ assert.ok(out.suggestions.includes("openrouter/gpt-image-1"));
474
+ }
475
+ });
476
+
477
+ it("rejects malformed model strings with the expected format", () => {
478
+ const out = resolveImageGenModelChoice("no-slash", available);
479
+ assert.equal(out.ok, false);
480
+ if (!out.ok) assert.match(out.error, /Expected format: provider\/model-id/);
481
+ });
482
+
483
+ it("errors with an actionable hint when nothing is keyed", () => {
484
+ const out = resolveImageGenModelChoice(undefined, []);
485
+ assert.equal(out.ok, false);
486
+ if (!out.ok) assert.match(out.error, /pi --login/);
487
+ });
488
+ });
489
+
439
490
  describe("sanitizeProviderHeaders", () => {
440
491
  it("drops null deletion markers", () => {
441
492
  assert.deepEqual(
@@ -0,0 +1,250 @@
1
+ /**
2
+ * Unit tests for the tool-transcript rendering helpers (1.21.0).
3
+ *
4
+ * These back the pi ≥ 1.0.1 registerToolRenderer() integration: compact
5
+ * analyze_image / generate_image call and result lines. Pure data in, plain
6
+ * segments out — theming happens in vision-proxy.ts.
7
+ *
8
+ * Run:
9
+ * node --experimental-strip-types --test extensions/__tests__/tool-render.test.ts
10
+ */
11
+
12
+ import { strict as assert } from "node:assert";
13
+ import { describe, it } from "node:test";
14
+ import {
15
+ analyzeDetailsForTranscript,
16
+ formatAnalyzeImageCall,
17
+ formatAnalyzeImageResult,
18
+ formatGenerateImageCall,
19
+ formatGenerateImageResult,
20
+ summarizeCrop,
21
+ summarizeForTranscript,
22
+ summarizeImageRef,
23
+ } from "../internal.ts";
24
+
25
+ const HASH32 = "deadbeefdeadbeefdeadbeefdeadbeef";
26
+
27
+ function plain(segments: ReadonlyArray<{ text: string }>): string {
28
+ return segments.map((s) => s.text).join("");
29
+ }
30
+
31
+ describe("summarizeForTranscript", () => {
32
+ it("passes short text through with collapsed whitespace", () => {
33
+ assert.equal(summarizeForTranscript(" hello world\n\tnext ", 50), "hello world next");
34
+ });
35
+
36
+ it("truncates long text on a word boundary", () => {
37
+ const out = summarizeForTranscript("alpha bravo charlie delta echo foxtrot golf hotel", 20);
38
+ assert.ok(out.length <= 20, `length ${out.length}: ${out}`);
39
+ assert.ok(out.endsWith("…"));
40
+ assert.ok(!out.endsWith(" …"), "no dangling space before ellipsis");
41
+ });
42
+
43
+ it("never splits a surrogate pair", () => {
44
+ const emoji = "😀".repeat(60); // each emoji is a surrogate pair
45
+ const out = summarizeForTranscript(emoji, 20); // odd max → cut lands mid-pair
46
+ assert.ok(out.length <= 20);
47
+ // No lone high surrogate anywhere: the string still round-trips as text.
48
+ // (With the guard removed, the hard cut leaves an unpaired high surrogate
49
+ // right before the ellipsis, which this regex detects.)
50
+ assert.ok(!/[\uD800-\uDBFF](?![\uDC00-\uDFFF])/.test(out));
51
+ });
52
+ });
53
+
54
+ describe("summarizeImageRef", () => {
55
+ it("shortens a quoted recall reference", () => {
56
+ assert.equal(summarizeImageRef(`image="${HASH32}"`), `image="deadbeef…"`);
57
+ });
58
+
59
+ it("shortens an unquoted recall reference and sha256:/#crop forms", () => {
60
+ assert.equal(summarizeImageRef(`image=${HASH32}`), `image="deadbeef…"`);
61
+ assert.equal(summarizeImageRef(`sha256:${HASH32}#x0y0`), `image="deadbeef…"`);
62
+ assert.equal(summarizeImageRef(HASH32), `image="deadbeef…"`);
63
+ });
64
+
65
+ it("reduces file paths to the file name (posix + windows)", () => {
66
+ assert.equal(summarizeImageRef("/home/u/pics/cat.png"), "cat.png");
67
+ assert.equal(summarizeImageRef("D:\\Users\\u\\Downloads\\shot (1).jpg"), "shot (1).jpg");
68
+ });
69
+
70
+ it("truncates very long file names", () => {
71
+ const long = "a".repeat(120) + ".png";
72
+ const out = summarizeImageRef(long);
73
+ assert.ok(out.length <= 48);
74
+ assert.ok(out.endsWith("…"));
75
+ });
76
+
77
+ it("leaves non-hash short junk as-is", () => {
78
+ assert.equal(summarizeImageRef("not-a-hash"), "not-a-hash");
79
+ });
80
+ });
81
+
82
+ describe("summarizeCrop", () => {
83
+ it("renders named regions with the image index", () => {
84
+ assert.equal(summarizeCrop({ image_index: 0, region: "bottom-right" }), "bottom-right@0");
85
+ });
86
+
87
+ it("renders normalized crops as percentages", () => {
88
+ assert.equal(
89
+ summarizeCrop({ image_index: 1, normalized: { x: 0.25, y: 0.133, width: 0.5, height: 0.5 } }),
90
+ "25%,13.3% 50%×50%@1",
91
+ );
92
+ });
93
+
94
+ it("renders pixel crops rounded", () => {
95
+ assert.equal(
96
+ summarizeCrop({ image_index: 0, pixels: { x: 128.4, y: 64.2, width: 256, height: 256 } }),
97
+ "128,64 256×256@0",
98
+ );
99
+ });
100
+
101
+ it("returns empty for unrecognized input", () => {
102
+ assert.equal(summarizeCrop(undefined), "");
103
+ assert.equal(summarizeCrop("nonsense"), "");
104
+ assert.equal(summarizeCrop({ image_index: 0 }), "crop@0");
105
+ });
106
+ });
107
+
108
+ describe("formatAnalyzeImageCall", () => {
109
+ it("builds a full call line", () => {
110
+ const segs = formatAnalyzeImageCall({
111
+ images: [`image="${HASH32}"`, "/tmp/chart.png"],
112
+ question: "What does the legend say?",
113
+ crop: [{ image_index: 0, region: "center" }],
114
+ model: "zai/glm-5.3-flash",
115
+ });
116
+ assert.equal(
117
+ plain(segs),
118
+ `"What does the legend say?" · 2 images (image="deadbeef…", chart.png) · crop center@0 · zai/glm-5.3-flash`,
119
+ );
120
+ assert.equal(segs[0]!.style, "accent");
121
+ });
122
+
123
+ it("collapses many images to two shown plus a count", () => {
124
+ const segs = formatAnalyzeImageCall({
125
+ images: ["/a.png", "/b.png", "/c.png", "/d.png"],
126
+ question: "q",
127
+ });
128
+ assert.ok(plain(segs).includes("4 images (a.png, b.png +2)"));
129
+ });
130
+
131
+ it("summarizes multiple crops as a count", () => {
132
+ const segs = formatAnalyzeImageCall({
133
+ images: ["/a.png", "/b.png"],
134
+ question: "q",
135
+ crop: [
136
+ { image_index: 0, region: "top" },
137
+ { image_index: 1, region: "bottom" },
138
+ ],
139
+ });
140
+ assert.ok(plain(segs).includes("crops ×2"));
141
+ });
142
+
143
+ it("truncates long questions and tolerates garbage args", () => {
144
+ const long = formatAnalyzeImageCall({ images: ["/a.png"], question: "x".repeat(300) });
145
+ assert.ok(plain(long).length < 200);
146
+
147
+ assert.deepEqual(formatAnalyzeImageCall("junk"), [{ text: '"junk"', style: "muted" }]);
148
+ assert.deepEqual(formatAnalyzeImageCall(null), [{ text: "null", style: "muted" }]);
149
+ });
150
+
151
+ it("handles missing question/images defensively", () => {
152
+ const segs = formatAnalyzeImageCall({});
153
+ assert.equal(plain(segs), '""');
154
+ });
155
+ });
156
+
157
+ describe("formatGenerateImageCall", () => {
158
+ it("builds prompt + model line", () => {
159
+ const segs = formatGenerateImageCall({ prompt: "A red panda astronaut, watercolor", model: "google/imagen-4" });
160
+ assert.equal(plain(segs), '"A red panda astronaut, watercolor" · google/imagen-4');
161
+ });
162
+
163
+ it("prompt-only line and garbage fallback", () => {
164
+ assert.equal(plain(formatGenerateImageCall({ prompt: "cat" })), '"cat"');
165
+ assert.deepEqual(formatGenerateImageCall(42), [{ text: "42", style: "muted" }]);
166
+ });
167
+ });
168
+
169
+ describe("formatAnalyzeImageResult", () => {
170
+ it("summarizes a successful analysis", () => {
171
+ const segs = formatAnalyzeImageResult(
172
+ { ok: true, provider: "zai", model: "glm-5.3-flash", cached: true, latencyMs: 1234 },
173
+ false,
174
+ );
175
+ assert.equal(plain(segs), "✓ zai/glm-5.3-flash · cached · 1.2s");
176
+ });
177
+
178
+ it("summarizes without optional fields and with model only", () => {
179
+ assert.equal(plain(formatAnalyzeImageResult({ ok: true }, false)), "✓");
180
+ assert.equal(plain(formatAnalyzeImageResult({ ok: true, model: "glm-5.3-flash" }, false)), "✓ glm-5.3-flash");
181
+ });
182
+
183
+ it("renders errors from details, isError flag, or a generic fallback", () => {
184
+ assert.equal(
185
+ plain(formatAnalyzeImageResult({ ok: false, error: "Error: consent required" }, true)),
186
+ "✗ Error: consent required",
187
+ );
188
+ // isError without details
189
+ assert.equal(plain(formatAnalyzeImageResult(undefined, true)), "✗ analysis failed");
190
+ // ok:false wins even without the flag (older rows)
191
+ assert.equal(plain(formatAnalyzeImageResult({ ok: false }, false)), "✗ analysis failed");
192
+ });
193
+
194
+ it("marks the summary segment styles", () => {
195
+ const segs = formatAnalyzeImageResult({ ok: true, provider: "zai", model: "m" }, false);
196
+ assert.equal(segs[0]!.style, "success");
197
+ assert.equal(segs[1]!.style, "muted");
198
+ });
199
+ });
200
+
201
+ describe("formatGenerateImageResult", () => {
202
+ it("summarizes generation with images, cost, and first id", () => {
203
+ const segs = formatGenerateImageResult(
204
+ {
205
+ ok: true,
206
+ provider: "google",
207
+ model: "imagen-4",
208
+ images: [
209
+ { id: HASH32, mimeType: "image/png", width: 1024, height: 768 },
210
+ { id: "f".repeat(32), mimeType: "image/png", width: 1024, height: 768 },
211
+ ],
212
+ costUSD: 0.0311,
213
+ },
214
+ false,
215
+ );
216
+ assert.equal(plain(segs), `✓ 2 images via google/imagen-4 · $0.0311 · image="deadbeef…" +1`);
217
+ });
218
+
219
+ it("renders errors from details or generically", () => {
220
+ assert.equal(
221
+ plain(formatGenerateImageResult({ ok: false, error: "Error: no keyed image model" }, true)),
222
+ "✗ Error: no keyed image model",
223
+ );
224
+ assert.equal(plain(formatGenerateImageResult(undefined, true)), "✗ generation failed");
225
+ });
226
+
227
+ it("falls back to 'done' when details carry no image list", () => {
228
+ assert.equal(plain(formatGenerateImageResult({ ok: true, provider: "google", model: "imagen-4" }, false)), "✓ done via google/imagen-4");
229
+ });
230
+ });
231
+
232
+ describe("analyzeDetailsForTranscript", () => {
233
+ it("strips text and images but keeps display fields", () => {
234
+ const out = analyzeDetailsForTranscript({
235
+ ok: true,
236
+ text: "a".repeat(2000),
237
+ images: [{ id: HASH32 }],
238
+ provider: "zai",
239
+ model: "glm-5.3-flash",
240
+ latencyMs: 120,
241
+ });
242
+ assert.deepEqual(out, { ok: true, provider: "zai", model: "glm-5.3-flash", latencyMs: 120 });
243
+ });
244
+
245
+ it("does not mutate the input", () => {
246
+ const input = { ok: true, text: "keep-me", images: [1, 2] };
247
+ analyzeDetailsForTranscript(input);
248
+ assert.deepEqual(input, { ok: true, text: "keep-me", images: [1, 2] });
249
+ });
250
+ });
@@ -2435,6 +2435,50 @@ export function pickDynamicVisionCandidate<T extends DynamicCandidateLike>(model
2435
2435
  return vision.find((m) => CHEAP_MODEL_RE.test(m.modelId)) ?? vision[0];
2436
2436
  }
2437
2437
 
2438
+ /** Outcome of image-generation model resolution (generate_image tool). */
2439
+ export type ImageGenModelChoice =
2440
+ | { ok: true; provider: string; modelId: string }
2441
+ | { ok: false; error: string; suggestions: readonly string[] };
2442
+
2443
+ /**
2444
+ * Resolve the image-generation model for one `generate_image` call (Pi ≥ 1.0).
2445
+ * An explicit `provider/model-id` override must match an available (keyed)
2446
+ * image model exactly — otherwise the error carries recovery-style close
2447
+ * matches. Without an override the first available model wins.
2448
+ */
2449
+ export function resolveImageGenModelChoice(
2450
+ override: string | undefined,
2451
+ available: ReadonlyArray<{ provider: string; modelId: string }>,
2452
+ ): ImageGenModelChoice {
2453
+ const labels = available.map((m) => `${m.provider}/${m.modelId}`);
2454
+ if (override === undefined || override === "") {
2455
+ const first = available[0];
2456
+ if (!first) {
2457
+ return {
2458
+ ok: false,
2459
+ error: "no image-generation model is available (keyed) in this session — add a provider key (e.g. pi --login openrouter)",
2460
+ suggestions: [],
2461
+ };
2462
+ }
2463
+ return { ok: true, provider: first.provider, modelId: first.modelId };
2464
+ }
2465
+ const parsed = parseModelString(override);
2466
+ if (!parsed) {
2467
+ return {
2468
+ ok: false,
2469
+ error: `invalid model string "${override}". Expected format: provider/model-id`,
2470
+ suggestions: [],
2471
+ };
2472
+ }
2473
+ const exact = available.find((m) => m.provider === parsed.provider && m.modelId === parsed.modelId);
2474
+ if (exact) return { ok: true, provider: exact.provider, modelId: exact.modelId };
2475
+ return {
2476
+ ok: false,
2477
+ error: `model "${parsed.provider}/${parsed.modelId}" not found among available image-generation models`,
2478
+ suggestions: closeMatches(`${parsed.provider}/${parsed.modelId}`, labels, 3),
2479
+ };
2480
+ }
2481
+
2438
2482
  export function shouldStripImages(config: VisionConfig, modelInput: readonly string[] | undefined): boolean {
2439
2483
  if (config.mode === "off") return false;
2440
2484
  if (config.mode === "always") return true;
@@ -3219,6 +3263,201 @@ export function buildToolCacheKey(
3219
3263
  return `${sortedHashes.join("+")}${cropSig ? "#crop:" + cropSig : ""}?q=${questionHash}&m=${modelId}`;
3220
3264
  }
3221
3265
 
3266
+ // ── Tool transcript rendering (pi ≥ 1.0.1 registerToolRenderer) ────────────
3267
+
3268
+ /**
3269
+ * Styling hints for one segment of a tool-transcript line, mapped to
3270
+ * theme.fg tokens by the renderer in vision-proxy.ts. Kept as plain data so
3271
+ * these helpers stay pure and unit-testable without a terminal theme.
3272
+ */
3273
+ export type TranscriptStyle = "plain" | "accent" | "muted" | "dim" | "success" | "warning" | "error";
3274
+
3275
+ export interface TranscriptSegment {
3276
+ text: string;
3277
+ style: TranscriptStyle;
3278
+ }
3279
+
3280
+ /** Collapse whitespace and truncate on a word boundary (surrogate-pair safe). */
3281
+ export function summarizeForTranscript(text: string, max: number): string {
3282
+ const t = text.replace(/\s+/g, " ").trim();
3283
+ if (t.length <= max) return t;
3284
+ let cut = t.slice(0, Math.max(1, max - 1));
3285
+ // Never split a surrogate pair on a hard cut (lone high surrogate → U+FFFD).
3286
+ if (/[\uD800-\uDBFF]$/.test(cut)) cut = cut.slice(0, -1);
3287
+ const ws = cut.lastIndexOf(" ");
3288
+ if (ws > cut.length * 0.6) cut = cut.slice(0, ws);
3289
+ return `${cut.trimEnd()}…`;
3290
+ }
3291
+
3292
+ /** Defensive finite-number check for values coming from model-authored args. */
3293
+ function isFiniteNumber(v: unknown): v is number {
3294
+ return typeof v === "number" && Number.isFinite(v);
3295
+ }
3296
+
3297
+ /** Compact percentage for crop summaries: integral values stay integral. */
3298
+ function pct(v: number): string {
3299
+ const p = v * 100;
3300
+ return Math.abs(p - Math.round(p)) < 0.05 ? String(Math.round(p)) : String(Math.round(p * 10) / 10);
3301
+ }
3302
+
3303
+ /** Safe one-line JSON rendering of an arbitrary value (circular-safe). */
3304
+ function safeJson(value: unknown, max = 120): string {
3305
+ try {
3306
+ return summarizeForTranscript(JSON.stringify(value) ?? String(value), max);
3307
+ } catch {
3308
+ return String(value);
3309
+ }
3310
+ }
3311
+
3312
+ /**
3313
+ * Shorten one analyze_image image reference for display: a recall id
3314
+ * (`image="<hash>"`, `sha256:<hash>#…`, or a bare hash) becomes
3315
+ * `image="deadbeef…"`; a file path becomes its file name (Windows or POSIX).
3316
+ */
3317
+ export function summarizeImageRef(ref: string): string {
3318
+ const s = ref.trim().replace(/^["']|["']$/g, "");
3319
+ const quoted = /image="?([a-f0-9]{16,64})"?/i.exec(s);
3320
+ if (quoted) return `image="${quoted[1]!.slice(0, 8)}…"`;
3321
+ const bare = parseRecallRef(s);
3322
+ if (bare) return `image="${bare.slice(0, 8)}…"`;
3323
+ const base = s.replace(/\\/g, "/").split("/").filter(Boolean).pop() ?? s;
3324
+ return summarizeForTranscript(base, 48);
3325
+ }
3326
+
3327
+ /**
3328
+ * Compact crop descriptor for the transcript: `bottom-right@0`,
3329
+ * `25%,13% 50%×50%@1`, or `128,64 256×256@0`. Empty string when nothing
3330
+ * recognizable.
3331
+ */
3332
+ export function summarizeCrop(crop: unknown): string {
3333
+ if (typeof crop !== "object" || crop === null) return "";
3334
+ const c = crop as Record<string, unknown>;
3335
+ const idx =
3336
+ isFiniteNumber(c.image_index) ? `@${Math.trunc(c.image_index)}` : "";
3337
+ if (typeof c.region === "string" && c.region.trim()) return `${c.region.trim()}${idx}`;
3338
+ const box = (c.normalized ?? c.pixels) as Record<string, unknown> | undefined;
3339
+ if (
3340
+ typeof box === "object" &&
3341
+ box !== null &&
3342
+ isFiniteNumber(box.x) &&
3343
+ isFiniteNumber(box.y) &&
3344
+ isFiniteNumber(box.width) &&
3345
+ isFiniteNumber(box.height)
3346
+ ) {
3347
+ if (c.normalized !== undefined) {
3348
+ return `${pct(box.x)}%,${pct(box.y)}% ${pct(box.width)}%×${pct(box.height)}%${idx}`;
3349
+ }
3350
+ return `${Math.round(box.x)},${Math.round(box.y)} ${Math.round(box.width)}×${Math.round(box.height)}${idx}`;
3351
+ }
3352
+ return idx ? `crop${idx}` : "";
3353
+ }
3354
+
3355
+ /** One-line transcript summary of an analyze_image call (pi ≥ 1.0.1 renderer). */
3356
+ export function formatAnalyzeImageCall(args: unknown): TranscriptSegment[] {
3357
+ if (typeof args !== "object" || args === null || Array.isArray(args)) {
3358
+ return [{ text: safeJson(args), style: "muted" }];
3359
+ }
3360
+ const a = args as Record<string, unknown>;
3361
+ const out: TranscriptSegment[] = [];
3362
+ const question = typeof a.question === "string" ? a.question : "";
3363
+ out.push({ text: `"${summarizeForTranscript(question, 120)}"`, style: "accent" });
3364
+ const images = Array.isArray(a.images)
3365
+ ? a.images.filter((r): r is string => typeof r === "string" && r.trim() !== "")
3366
+ : [];
3367
+ if (images.length > 0) {
3368
+ const shown = images.slice(0, 2).map(summarizeImageRef);
3369
+ const more = images.length - shown.length;
3370
+ out.push({
3371
+ text:
3372
+ ` · ${images.length} image${images.length === 1 ? "" : "s"}` +
3373
+ (shown.length ? ` (${shown.join(", ")}${more > 0 ? ` +${more}` : ""})` : ""),
3374
+ style: "muted",
3375
+ });
3376
+ }
3377
+ const crops = Array.isArray(a.crop) ? a.crop.map(summarizeCrop).filter((s) => s !== "") : [];
3378
+ if (crops.length === 1) out.push({ text: ` · crop ${crops[0]}`, style: "dim" });
3379
+ else if (crops.length > 1) out.push({ text: ` · crops ×${crops.length}`, style: "dim" });
3380
+ const model = typeof a.model === "string" && a.model.trim() ? a.model.trim() : undefined;
3381
+ if (model) out.push({ text: ` · ${summarizeForTranscript(model, 60)}`, style: "muted" });
3382
+ return out;
3383
+ }
3384
+
3385
+ /** One-line transcript summary of a generate_image call (pi ≥ 1.0.1 renderer). */
3386
+ export function formatGenerateImageCall(args: unknown): TranscriptSegment[] {
3387
+ if (typeof args !== "object" || args === null || Array.isArray(args)) {
3388
+ return [{ text: safeJson(args), style: "muted" }];
3389
+ }
3390
+ const a = args as Record<string, unknown>;
3391
+ const prompt = typeof a.prompt === "string" ? a.prompt : "";
3392
+ const out: TranscriptSegment[] = [{ text: `"${summarizeForTranscript(prompt, 120)}"`, style: "accent" }];
3393
+ const model = typeof a.model === "string" && a.model.trim() ? a.model.trim() : undefined;
3394
+ if (model) out.push({ text: ` · ${summarizeForTranscript(model, 60)}`, style: "muted" });
3395
+ return out;
3396
+ }
3397
+
3398
+ function recordOf(details: unknown): Record<string, unknown> | undefined {
3399
+ return typeof details === "object" && details !== null && !Array.isArray(details)
3400
+ ? (details as Record<string, unknown>)
3401
+ : undefined;
3402
+ }
3403
+
3404
+ /** One-line transcript summary of an analyze_image result from its `details`. */
3405
+ export function formatAnalyzeImageResult(details: unknown, isError: boolean): TranscriptSegment[] {
3406
+ const d = recordOf(details);
3407
+ if (isError || d?.ok === false) {
3408
+ const msg = typeof d?.error === "string" && d.error ? d.error : "analysis failed";
3409
+ return [{ text: `✗ ${summarizeForTranscript(msg, 160)}`, style: "error" }];
3410
+ }
3411
+ const segments: TranscriptSegment[] = [{ text: "✓", style: "success" }];
3412
+ const provider = typeof d?.provider === "string" && d.provider ? d.provider : undefined;
3413
+ const modelId = typeof d?.model === "string" && d.model ? d.model : undefined;
3414
+ if (provider && modelId) segments.push({ text: ` ${summarizeForTranscript(`${provider}/${modelId}`, 80)}`, style: "muted" });
3415
+ else if (modelId) segments.push({ text: ` ${summarizeForTranscript(modelId, 80)}`, style: "muted" });
3416
+ if (d?.cached === true) segments.push({ text: " · cached", style: "dim" });
3417
+ if (isFiniteNumber(d?.latencyMs)) {
3418
+ segments.push({ text: ` · ${(Math.max(0, d.latencyMs) / 1000).toFixed(1)}s`, style: "dim" });
3419
+ }
3420
+ return segments;
3421
+ }
3422
+
3423
+ /** One-line transcript summary of a generate_image result from its `details`. */
3424
+ export function formatGenerateImageResult(details: unknown, isError: boolean): TranscriptSegment[] {
3425
+ const d = recordOf(details);
3426
+ if (isError || d?.ok === false) {
3427
+ const msg = typeof d?.error === "string" && d.error ? d.error : "generation failed";
3428
+ return [{ text: `✗ ${summarizeForTranscript(msg, 160)}`, style: "error" }];
3429
+ }
3430
+ const images = Array.isArray(d?.images)
3431
+ ? (d!.images as unknown[]).filter((x): x is Record<string, unknown> => typeof x === "object" && x !== null)
3432
+ : [];
3433
+ const provider = typeof d?.provider === "string" && d.provider ? d.provider : undefined;
3434
+ const modelId = typeof d?.model === "string" && d.model ? d.model : undefined;
3435
+ const count = images.length > 0 ? `${images.length} image${images.length === 1 ? "" : "s"}` : "done";
3436
+ const segments: TranscriptSegment[] =
3437
+ provider && modelId
3438
+ ? [{ text: `✓ ${count} via ${summarizeForTranscript(`${provider}/${modelId}`, 80)}`, style: "success" }]
3439
+ : [{ text: `✓ ${count}`, style: "success" }];
3440
+ if (isFiniteNumber(d?.costUSD)) segments.push({ text: ` · $${Math.max(0, d.costUSD).toFixed(4)}`, style: "dim" });
3441
+ const firstId = typeof images[0]?.id === "string" && images[0].id ? images[0].id : undefined;
3442
+ if (firstId) {
3443
+ const more = images.length > 1 ? ` +${images.length - 1}` : "";
3444
+ segments.push({ text: ` · image="${firstId.slice(0, 8)}…"${more}`, style: "dim" });
3445
+ }
3446
+ return segments;
3447
+ }
3448
+
3449
+ /**
3450
+ * Copy of a structured analyze_image result without the fields the
3451
+ * transcript renderer never reads — `details` must stay small because it is
3452
+ * persisted in the session log alongside the (already stored) text fence.
3453
+ */
3454
+ export function analyzeDetailsForTranscript(structured: Record<string, unknown>): Record<string, unknown> {
3455
+ const rest: Record<string, unknown> = { ...structured };
3456
+ delete rest.text;
3457
+ delete rest.images;
3458
+ return rest;
3459
+ }
3460
+
3222
3461
  // ── Fence builders ────────────────────────────────────────────────────────
3223
3462
 
3224
3463
  export interface VideoDescriptionEntry {
@@ -256,6 +256,12 @@ import {
256
256
  buildVideoProxySection,
257
257
  extractXaiResponsesText,
258
258
  formatXaiSttTranscript,
259
+ analyzeDetailsForTranscript,
260
+ formatAnalyzeImageCall,
261
+ formatAnalyzeImageResult,
262
+ formatGenerateImageCall,
263
+ formatGenerateImageResult,
264
+ type TranscriptSegment,
259
265
  isTranscriptionRequest,
260
266
  isXaiProvider,
261
267
  bufferToPiAiImage,
@@ -284,6 +290,7 @@ import {
284
290
  applyDefaultModelFallback,
285
291
  closeMatches,
286
292
  pickDynamicVisionCandidate,
293
+ resolveImageGenModelChoice,
287
294
  DEFAULT_MODEL_FALLBACKS,
288
295
  applyRecallCompletion,
289
296
  buildRecallItems,
@@ -469,6 +476,52 @@ const TOOL_DESCRIPTION = [
469
476
  /** Maximum analyze_image tool calls per agent turn. Prevents cost runaway. */
470
477
  const MAX_TOOL_CALLS_PER_TURN = 10;
471
478
 
479
+ /**
480
+ * generate_image parameters (Pi ≥ 1.0). Lean on purpose: pi 1.0's codemode
481
+ * lists tool declarations under a 3000-token inline budget.
482
+ */
483
+ const GenerateImageParams = Type.Object({
484
+ prompt: Type.String({
485
+ description: "Required, non-empty, max 4000 chars. What to draw; include style and aspect-ratio cues the provider supports.",
486
+ }),
487
+ model: Type.Optional(
488
+ Type.String({
489
+ description: "Optional; provider/model-id of an image-generation model. Defaults to the first available (keyed) image model.",
490
+ }),
491
+ ),
492
+ reason: Type.Optional(Type.String({ description: "Optional; logged for analytics only" })),
493
+ });
494
+
495
+ /**
496
+ * outputSchema of generate_image: codemode scripts receive this
497
+ * structuredContent instead of the text + image blocks.
498
+ */
499
+ const GenerateImageOutput = Type.Object({
500
+ ok: Type.Boolean({ description: "Whether generation succeeded." }),
501
+ error: Type.Optional(Type.String({ description: "Failure message, present when ok is false." })),
502
+ provider: Type.Optional(Type.String({ description: "Provider that generated the images." })),
503
+ model: Type.Optional(Type.String({ description: "Model id that generated the images." })),
504
+ images: Type.Optional(
505
+ Type.Array(
506
+ Type.Object({
507
+ id: Type.String({ description: "Stable image id usable for recall (image=\"...\") and analyze_image." }),
508
+ mimeType: Type.String(),
509
+ width: Type.Optional(Type.Number()),
510
+ height: Type.Optional(Type.Number()),
511
+ }),
512
+ ),
513
+ ),
514
+ costUSD: Type.Optional(Type.Number({ description: "Reported generation cost, when the provider returned usage." })),
515
+ });
516
+
517
+ const GENERATE_IMAGE_DESCRIPTION = [
518
+ "Generate images with the session's image-generation models (Pi ≥ 1.0, request-time credentials).",
519
+ "",
520
+ "`prompt` describes what to draw; `model` optionally picks an image-generation model (provider/model-id — defaults to the first available).",
521
+ "Returns image blocks attached to this result plus stable `image=\"<id>\"` ids that analyze_image can re-analyze or crop like any other image.",
522
+ "Each call costs money — use it when the user asks for an image to be created, not speculatively.",
523
+ ].join("\n");
524
+
472
525
  /** Default tool-result cache size; resized to config.cacheSize on first use. */
473
526
  const DEFAULT_TOOL_CACHE_SIZE = 50;
474
527
 
@@ -508,6 +561,8 @@ interface SessionState {
508
561
  keyProbe?: Map<string, boolean>;
509
562
  /** Cached dynamic (registry-keyed) vision candidate; null = none or runtime without the API (Pi ≥ 1.0). */
510
563
  dynamicVisionProbe?: { provider: string; modelId: string } | null;
564
+ /** Cached available (keyed) image-generation models (Pi ≥ 1.0). */
565
+ imageGenModels?: Array<{ provider: string; modelId: string }>;
511
566
  /** Last implicit-default substitution already notified this session. */
512
567
  notifiedDefaultSub?: string;
513
568
  /**
@@ -1741,6 +1796,188 @@ function recallSuggestions(imageData: ImageDataStore, imageMeta: ImageMetaStore,
1741
1796
  return closeMatches(query, candidates, limit);
1742
1797
  }
1743
1798
 
1799
+ /** Content block shapes a tool result may carry (text and/or image). */
1800
+ type ToolContentBlock = { type: "text"; text: string } | { type: "image"; data: string; mimeType: string };
1801
+
1802
+ /** Structured outcome of one generate_image call (1.20.0). */
1803
+ interface GenerateImageOutcome {
1804
+ content: ToolContentBlock[];
1805
+ isError?: boolean;
1806
+ structured?: Record<string, unknown>;
1807
+ usage?: unknown;
1808
+ }
1809
+
1810
+ /** Wrap a failure message as a GenerateImageOutcome. */
1811
+ function generationError(message: string): GenerateImageOutcome {
1812
+ return { content: [{ type: "text", text: message }], isError: true, structured: { ok: false, error: message } };
1813
+ }
1814
+
1815
+ /**
1816
+ * Available (keyed) image-generation models via getAvailableOfType("image")
1817
+ * (Pi ≥ 1.0, feature-detected). Cached per session; probe failures cache as
1818
+ * an empty list so a transient error cannot wedge the tool.
1819
+ */
1820
+ async function availableImageGenModels(ctx: ExtensionContext): Promise<Array<{ provider: string; modelId: string }>> {
1821
+ const state = getSessionState(ctx);
1822
+ if (state.imageGenModels) return state.imageGenModels;
1823
+ let out: Array<{ provider: string; modelId: string }> = [];
1824
+ const reg = ctx.modelRegistry as ModelRegistry & {
1825
+ getAvailableOfType?: (type: string) => Promise<ReadonlyArray<unknown>>;
1826
+ };
1827
+ if (typeof reg.getAvailableOfType === "function") {
1828
+ try {
1829
+ const models = (await reg.getAvailableOfType("image")) as ReadonlyArray<{ provider?: unknown; id?: unknown }>;
1830
+ out = models
1831
+ .filter((m): m is { provider: string; id: string } => typeof m.provider === "string" && typeof m.id === "string")
1832
+ .map((m) => ({ provider: m.provider, modelId: m.id }));
1833
+ } catch {
1834
+ out = [];
1835
+ }
1836
+ }
1837
+ state.imageGenModels = out;
1838
+ return out;
1839
+ }
1840
+
1841
+ /** Minimal response shape of ModelRegistry.generateImages() (Pi ≥ 1.0). */
1842
+ interface GenerateImagesResponseLike {
1843
+ output?: ReadonlyArray<{ type?: unknown; data?: unknown; mimeType?: unknown }>;
1844
+ usage?: { costUSD?: unknown; inputTokens?: unknown; outputTokens?: unknown };
1845
+ stopReason?: unknown;
1846
+ errorMessage?: unknown;
1847
+ }
1848
+
1849
+ /**
1850
+ * Generate one or more images with the session's image-generation models
1851
+ * (Pi ≥ 1.0 `models.generateImages()`): resolve the model (explicit override
1852
+ * with recovery-style suggestions, else the first keyed image model), gate
1853
+ * on data-egress consent (the prompt leaves), call the registry, and return
1854
+ * image blocks the conversation can see. Generated images are persisted in
1855
+ * the recall stores, so analyze_image (and `#` recall once described) work
1856
+ * on them exactly like attached images — including when the base model sees
1857
+ * the pixels directly and the tool_result auto-description path never fires.
1858
+ */
1859
+ async function handleGenerateImage(
1860
+ params: { prompt: string; model?: string; reason?: string },
1861
+ ctx: ExtensionContext,
1862
+ config: VisionConfig,
1863
+ ): Promise<GenerateImageOutcome> {
1864
+ const prompt = typeof params.prompt === "string" ? params.prompt.trim() : "";
1865
+ if (!prompt) return generationError("Error: prompt is required and must be non-empty.");
1866
+ if (prompt.length > 4000) return generationError("Error: prompt must be at most 4000 characters.");
1867
+
1868
+ // Resolve the image-generation model.
1869
+ const available = await availableImageGenModels(ctx);
1870
+ const choice = resolveImageGenModelChoice(params.model, available);
1871
+ if (!choice.ok) {
1872
+ return generationError(
1873
+ `Error: ${choice.error}.` +
1874
+ (choice.suggestions.length > 0 ? ` Close matches: ${choice.suggestions.join(", ")}.` : ""),
1875
+ );
1876
+ }
1877
+
1878
+ // Data-egress consent: the prompt leaves for the image provider.
1879
+ const entries = ctx.sessionManager.getEntries();
1880
+ if (!hasConsent(entries, choice.provider, config.allowedProviders, config.deniedProviders)) {
1881
+ return generationError(
1882
+ `Error: consent required before sending the prompt to ${choice.provider}. Please tell the user to run the following command and then retry:\n\n/multimodal-proxy consent yes\n\n(To pre-consent this provider permanently: /multimodal-proxy allowed-providers add ${choice.provider})`,
1883
+ );
1884
+ }
1885
+
1886
+ // Fetch the ImageModel object for the chosen pair (sync registry lookups).
1887
+ const reg = ctx.modelRegistry as ModelRegistry & {
1888
+ getModelsOfType?: (type: string) => ReadonlyArray<unknown>;
1889
+ findOfType?: (type: string, provider: string, modelId: string) => unknown;
1890
+ generateImages?: (
1891
+ model: unknown,
1892
+ context: { input: Array<{ type: "text"; text: string }> },
1893
+ options?: { signal?: AbortSignal },
1894
+ ) => Promise<GenerateImagesResponseLike>;
1895
+ };
1896
+ if (typeof reg.generateImages !== "function") {
1897
+ return generationError("Error: this pi runtime does not expose models.generateImages() (needs pi ≥ 1.0). Update pi.");
1898
+ }
1899
+ let imageModel: unknown;
1900
+ if (typeof reg.getModelsOfType === "function") {
1901
+ const all = reg.getModelsOfType("image") as ReadonlyArray<{ provider?: unknown; id?: unknown }>;
1902
+ imageModel = all.find(
1903
+ (m) =>
1904
+ typeof m.provider === "string" &&
1905
+ typeof m.id === "string" &&
1906
+ m.provider === choice.provider &&
1907
+ m.id === choice.modelId,
1908
+ );
1909
+ }
1910
+ if (!imageModel && typeof reg.findOfType === "function") {
1911
+ imageModel = reg.findOfType("image", choice.provider, choice.modelId);
1912
+ }
1913
+ if (!imageModel) {
1914
+ return generationError(
1915
+ `Error: image model "${choice.provider}/${choice.modelId}" could not be resolved in the registry.`,
1916
+ );
1917
+ }
1918
+
1919
+ let response: GenerateImagesResponseLike;
1920
+ try {
1921
+ response = await reg.generateImages(imageModel, { input: [{ type: "text", text: prompt }] }, { signal: ctx.signal });
1922
+ } catch (err) {
1923
+ return generationError(`Error: image generation failed: ${err instanceof Error ? err.message : String(err)}`);
1924
+ }
1925
+ if (response.stopReason === "aborted") return generationError("Error: generation was cancelled.");
1926
+ if (response.stopReason === "error") {
1927
+ return generationError(
1928
+ `Error: image generation failed: ${typeof response.errorMessage === "string" ? response.errorMessage : "unknown provider error"}`,
1929
+ );
1930
+ }
1931
+
1932
+ const blocks = (response.output ?? []).filter(
1933
+ (b): b is { type: "image"; data: string; mimeType: string } =>
1934
+ b.type === "image" && typeof b.data === "string" && typeof b.mimeType === "string",
1935
+ );
1936
+ if (blocks.length === 0) {
1937
+ return generationError(
1938
+ "Error: the model returned no images (text-only response). Try a different prompt or model.",
1939
+ );
1940
+ }
1941
+
1942
+ // Persist recall data so analyze_image (and # recall, once a description
1943
+ // exists) work on generated images even when the base model sees the
1944
+ // pixels directly and the tool_result auto-description path never fires.
1945
+ const { imageMeta, imageData } = getSessionState(ctx);
1946
+ const structuredImages: Array<Record<string, unknown>> = [];
1947
+ for (const b of blocks) {
1948
+ const hash = hashImageData(b.data);
1949
+ const ext = b.mimeType.split("/")[1]?.replace(/[^a-z0-9]/gi, "") || "png";
1950
+ storeImageMeta(imageMeta, hash, b.data, `generated-${hash.slice(0, 8)}.${ext}`);
1951
+ storeImageData(imageData, hash, b.data, b.mimeType);
1952
+ const dims = imageMeta.get(hash);
1953
+ structuredImages.push({
1954
+ id: hash,
1955
+ mimeType: b.mimeType,
1956
+ ...(dims ? { width: dims.width, height: dims.height } : {}),
1957
+ });
1958
+ }
1959
+
1960
+ const costUSD = typeof response.usage?.costUSD === "number" ? response.usage.costUSD : undefined;
1961
+ const ids = structuredImages.map((i) => `image="${i.id}"`).join(", ");
1962
+ const text =
1963
+ `[generate_image] ${pluralImages(blocks.length)} generated via ${choice.provider}/${choice.modelId}` +
1964
+ (ids ? ` (${ids})` : "") +
1965
+ (costUSD !== undefined ? ` — cost $${costUSD.toFixed(4)}` : "") +
1966
+ ". The images are attached as image blocks; their ids can be re-analyzed or cropped with analyze_image.";
1967
+
1968
+ return {
1969
+ content: [{ type: "text", text }, ...blocks],
1970
+ structured: {
1971
+ ok: true,
1972
+ provider: choice.provider,
1973
+ model: choice.modelId,
1974
+ images: structuredImages,
1975
+ ...(costUSD !== undefined ? { costUSD } : {}),
1976
+ },
1977
+ ...(response.usage ? { usage: response.usage } : {}),
1978
+ };
1979
+ }
1980
+
1744
1981
  /**
1745
1982
  * Answer a targeted analyze_image question about one or more images: resolve
1746
1983
  * references (session recall ids or file paths), validate and apply crops,
@@ -2105,6 +2342,7 @@ interface EditorAutocompleteProvider {
2105
2342
 
2106
2343
  export default function (pi: ExtensionAPI) {
2107
2344
  let _toolRegistered = false;
2345
+ let _generateToolRegistered = false;
2108
2346
  let _autocompleteRegistered = false;
2109
2347
 
2110
2348
  /**
@@ -2149,6 +2387,227 @@ export default function (pi: ExtensionAPI) {
2149
2387
  );
2150
2388
  }
2151
2389
 
2390
+ // ── Tool transcript renderers (pi ≥ 1.0.1) ─────────────────────────────
2391
+
2392
+ /** Minimal structural shape of the TUI component the renderers return. */
2393
+ interface ComponentLike {
2394
+ render(width: number): string[];
2395
+ invalidate?(): void;
2396
+ }
2397
+
2398
+ /** Theme subset the renderers use (pi's Theme helpers). */
2399
+ interface RenderThemeLike {
2400
+ fg(token: string, text: string): string;
2401
+ bold(text: string): string;
2402
+ }
2403
+
2404
+ /** ToolRenderContext subset available to renderCall/renderResult. */
2405
+ interface RenderContextLike {
2406
+ isError?: boolean;
2407
+ isPartial?: boolean;
2408
+ expanded?: boolean;
2409
+ showImages?: boolean;
2410
+ }
2411
+
2412
+ /** AgentToolResult subset the result renderer receives. */
2413
+ interface RenderResultLike {
2414
+ content?: ReadonlyArray<{ type?: string; text?: string }>;
2415
+ details?: unknown;
2416
+ }
2417
+
2418
+ type ToolRenderersLike = {
2419
+ renderCall?: (args: unknown, theme: RenderThemeLike, context: RenderContextLike) => ComponentLike;
2420
+ renderResult?: (
2421
+ result: RenderResultLike,
2422
+ options: { expanded: boolean; isPartial: boolean },
2423
+ theme: RenderThemeLike,
2424
+ context: RenderContextLike,
2425
+ ) => ComponentLike;
2426
+ };
2427
+
2428
+ /**
2429
+ * pi-tui's Text (ANSI-aware word wrap), loaded lazily so an unresolvable
2430
+ * package can never block extension load — the crash-loop lesson from
2431
+ * 1.16.0. Warmed at registration time, long before the first tool row
2432
+ * renders.
2433
+ */
2434
+ let _textCtor: (new (text: string, paddingX?: number, paddingY?: number) => ComponentLike) | undefined;
2435
+ let _textLoadTried = false;
2436
+
2437
+ function preloadTextComponent(): void {
2438
+ if (_textLoadTried) return;
2439
+ _textLoadTried = true;
2440
+ void import("@earendil-works/pi-tui")
2441
+ .then((m) => {
2442
+ const ctor = (
2443
+ m as { Text?: new (text: string, paddingX?: number, paddingY?: number) => ComponentLike }
2444
+ ).Text;
2445
+ if (ctor) _textCtor = ctor;
2446
+ })
2447
+ .catch(() => {
2448
+ // pi-tui unreachable — the plain-lines fallback below keeps rendering
2449
+ });
2450
+ }
2451
+
2452
+ /**
2453
+ * Fixed-lines component used only when pi-tui's Text cannot load.
2454
+ * Truncates each line to the render width (ANSI-aware, rough wcwidth) —
2455
+ * pi's differential renderer stops on "rendered line exceeds terminal
2456
+ * width", so an overwide fallback line is worse than an ugly one.
2457
+ */
2458
+ function plainLines(text: string): ComponentLike {
2459
+ return {
2460
+ render: (width: number) => text.split("\n").map((line) => truncateAnsiLine(line, Math.max(1, width))),
2461
+ invalidate: () => {},
2462
+ };
2463
+ }
2464
+
2465
+ /** Rough wcwidth: 2 for the common wide ranges, 0 for combining marks, else 1. */
2466
+ function codePointWidth(cp: number): number {
2467
+ if (cp >= 0x0300 && cp <= 0x036f) return 0; // combining diacritics
2468
+ if (
2469
+ (cp >= 0x1100 && cp <= 0x115f) || // Hangul Jamo
2470
+ (cp >= 0x2e80 && cp <= 0xa4cf) || // CJK radicals … Yi
2471
+ (cp >= 0xac00 && cp <= 0xd7a3) || // Hangul syllables
2472
+ (cp >= 0xf900 && cp <= 0xfaff) || // CJK compat ideographs
2473
+ (cp >= 0xfe30 && cp <= 0xfe4f) ||
2474
+ (cp >= 0xff00 && cp <= 0xff60) ||
2475
+ (cp >= 0xffe0 && cp <= 0xffe6) ||
2476
+ (cp >= 0x1f300 && cp <= 0x1faff) || // emoji blocks
2477
+ (cp >= 0x20000 && cp <= 0x3fffd) // CJK ext
2478
+ ) {
2479
+ return 2;
2480
+ }
2481
+ return 1;
2482
+ }
2483
+
2484
+ /** ANSI-preserving truncation of one styled line to a visible width. */
2485
+ function truncateAnsiLine(line: string, width: number): string {
2486
+ let out = "";
2487
+ let w = 0;
2488
+ let i = 0;
2489
+ while (i < line.length) {
2490
+ if (line[i] === "\x1b") {
2491
+ const m = /^\x1b\[[0-9;]*m/.exec(line.slice(i));
2492
+ const seq = m ? m[0] : line[i]!;
2493
+ out += seq;
2494
+ i += seq.length;
2495
+ continue;
2496
+ }
2497
+ const cp = line.codePointAt(i)!;
2498
+ const s = String.fromCodePoint(cp);
2499
+ const cw = codePointWidth(cp);
2500
+ if (w + cw > width) break;
2501
+ out += s;
2502
+ w += cw;
2503
+ i += s.length;
2504
+ }
2505
+ return out;
2506
+ }
2507
+
2508
+ function textComponent(text: string): ComponentLike {
2509
+ return _textCtor ? new _textCtor(text, 0, 0) : plainLines(text);
2510
+ }
2511
+
2512
+ function styleSegments(segments: readonly TranscriptSegment[], theme: RenderThemeLike): string {
2513
+ let out = "";
2514
+ for (const seg of segments) {
2515
+ out += seg.style === "plain" ? seg.text : theme.fg(seg.style, seg.text);
2516
+ }
2517
+ return out;
2518
+ }
2519
+
2520
+ function textBlocks(content: ReadonlyArray<{ type?: string; text?: string }> | undefined): string[] {
2521
+ if (!content) return [];
2522
+ const out: string[] = [];
2523
+ for (const block of content) {
2524
+ if (block?.type === "text" && typeof block.text === "string" && block.text) out.push(block.text);
2525
+ }
2526
+ return out;
2527
+ }
2528
+
2529
+ function toolTitle(name: string, theme: RenderThemeLike): string {
2530
+ return theme.fg("toolTitle", theme.bold(`${name} `));
2531
+ }
2532
+
2533
+ /** Compact result component: summary line, plus the full text content when expanded. */
2534
+ function resultComponent(
2535
+ summary: readonly TranscriptSegment[],
2536
+ result: RenderResultLike,
2537
+ options: { expanded: boolean },
2538
+ theme: RenderThemeLike,
2539
+ ): ComponentLike {
2540
+ let text = styleSegments(summary, theme);
2541
+ if (options.expanded) {
2542
+ // All text blocks: a generate_image result on a text-only base model
2543
+ // carries the header plus per-image description fences that replaced
2544
+ // the image blocks — the first block alone would hide them.
2545
+ const body = textBlocks(result.content).join("\n\n");
2546
+ if (body) {
2547
+ // Re-apply the style per line: pi resets styling after every line.
2548
+ text += `\n${body.split("\n").map((line) => theme.fg("toolOutput", line)).join("\n")}`;
2549
+ }
2550
+ }
2551
+ return textComponent(text);
2552
+ }
2553
+
2554
+ /**
2555
+ * Register transcript renderers for analyze_image/generate_image (pi ≥
2556
+ * 1.0.1). Registered through pi.registerToolRenderer — instead of inline
2557
+ * renderCall/renderResult on the tool definitions — so the compact rows
2558
+ * also cover tools that are *not registered* at render time: calls
2559
+ * replayed from a session where the tool is now disabled (`tool off` /
2560
+ * mode off), resumed sessions rendered before extension load, and HTML
2561
+ * exports. `next() ?? {}` keeps renderers from later resolvers / the
2562
+ * registered tool for the keys we don't set. Generated images keep
2563
+ * rendering inline regardless: pi draws image blocks from the result
2564
+ * content next to the renderer output, independent of renderResult.
2565
+ */
2566
+ function registerToolRenderers() {
2567
+ const api = pi as unknown as {
2568
+ registerToolRenderer?: (
2569
+ resolver: (toolName: string, next: () => ToolRenderersLike | undefined) => ToolRenderersLike | undefined,
2570
+ ) => void;
2571
+ };
2572
+ if (typeof api.registerToolRenderer !== "function") return; // pi < 1.0.1
2573
+ preloadTextComponent();
2574
+ api.registerToolRenderer((toolName, next) => {
2575
+ if (toolName === "analyze_image") {
2576
+ return {
2577
+ ...(next() ?? {}),
2578
+ renderCall: (args, theme) =>
2579
+ textComponent(toolTitle("analyze_image", theme) + styleSegments(formatAnalyzeImageCall(args), theme)),
2580
+ renderResult: (result, options, theme, context) => {
2581
+ if (options.isPartial) return textComponent(theme.fg("warning", "Analyzing…"));
2582
+ return resultComponent(
2583
+ formatAnalyzeImageResult(result?.details, context?.isError === true),
2584
+ result ?? {},
2585
+ options,
2586
+ theme,
2587
+ );
2588
+ },
2589
+ };
2590
+ }
2591
+ if (toolName === "generate_image") {
2592
+ return {
2593
+ ...(next() ?? {}),
2594
+ renderCall: (args, theme) =>
2595
+ textComponent(toolTitle("generate_image", theme) + styleSegments(formatGenerateImageCall(args), theme)),
2596
+ renderResult: (result, options, theme, context) => {
2597
+ if (options.isPartial) return textComponent(theme.fg("warning", "Generating…"));
2598
+ return resultComponent(
2599
+ formatGenerateImageResult(result?.details, context?.isError === true),
2600
+ result ?? {},
2601
+ options,
2602
+ theme,
2603
+ );
2604
+ },
2605
+ };
2606
+ }
2607
+ return next();
2608
+ });
2609
+ }
2610
+
2152
2611
  // ── Virtual model (pi ≥ 0.99) ─────────────────────────────────────────────
2153
2612
 
2154
2613
  const VIRTUAL_PROVIDER = "multimodal-proxy";
@@ -2339,8 +2798,12 @@ export default function (pi: ExtensionAPI) {
2339
2798
  // unreadable config file — session_start surfaces the error path
2340
2799
  });
2341
2800
 
2801
+ // Transcript renderers for analyze_image/generate_image rows (pi ≥ 1.0.1).
2802
+ // Registered once at load; the resolver serves any later session state.
2803
+ registerToolRenderers();
2804
+
2342
2805
  /** Register or unregister the analyze_image tool based on config. */
2343
- function syncToolRegistration(config: VisionConfig) {
2806
+ function syncToolRegistration(config: VisionConfig, runtimeCtx?: ExtensionContext) {
2344
2807
  const shouldHaveTool = config.mode !== "off" && config.tool === "on";
2345
2808
  if (shouldHaveTool && !_toolRegistered) {
2346
2809
  pi.registerTool({
@@ -2398,11 +2861,93 @@ export default function (pi: ExtensionAPI) {
2398
2861
  // codemode scripts. Older hosts ignore both fields.
2399
2862
  ...(outcome.isError ? { isError: true } : {}),
2400
2863
  ...(outcome.structured ? { structuredContent: outcome.structured } : {}),
2864
+ // `details` feeds the transcript renderer (pi ≥ 1.0.1) without
2865
+ // the model seeing it; heavy fields (text, images) are stripped.
2866
+ ...(outcome.structured ? { details: analyzeDetailsForTranscript(outcome.structured) } : {}),
2401
2867
  };
2402
2868
  },
2403
2869
  });
2404
2870
  _toolRegistered = true;
2405
2871
  }
2872
+ // generate_image (Pi ≥ 1.0 only): registered alongside analyze_image
2873
+ // when the runtime exposes ModelRegistry.generateImages; silently
2874
+ // skipped on older hosts so the tool never appears there.
2875
+ if (
2876
+ shouldHaveTool &&
2877
+ !_generateToolRegistered &&
2878
+ runtimeCtx &&
2879
+ typeof (runtimeCtx.modelRegistry as ModelRegistry & { generateImages?: unknown }).generateImages ===
2880
+ "function"
2881
+ ) {
2882
+ pi.registerTool({
2883
+ name: "generate_image",
2884
+ label: "Generate Image",
2885
+ description: GENERATE_IMAGE_DESCRIPTION,
2886
+ promptSnippet: "Generate images with the session's image-generation models",
2887
+ promptGuidelines: [
2888
+ "Use generate_image when the user explicitly asks to create an image; each call costs money.",
2889
+ 'Generated images carry image="<id>" ids and can be re-analyzed or cropped with analyze_image like any other image.',
2890
+ ],
2891
+ // Cost-incurring content creation: not read-only. openWorldHint — the
2892
+ // prompt leaves for the configured image provider.
2893
+ annotations: {
2894
+ readOnlyHint: false,
2895
+ destructiveHint: false,
2896
+ idempotentHint: false,
2897
+ openWorldHint: true,
2898
+ },
2899
+ namespace: {
2900
+ name: "multimodal-proxy",
2901
+ description: "pi-multimodal-proxy media analysis (targeted image re-analysis with crop and grounding)",
2902
+ },
2903
+ outputSchema: GenerateImageOutput,
2904
+ parameters: GenerateImageParams,
2905
+ execute: async (_toolCallId, params, _signal, _onUpdate, extCtx) => {
2906
+ const entries = extCtx.sessionManager.getEntries();
2907
+ const config = await withModelFallback(resolveConfig(entries, process.env, _fileConfig), extCtx);
2908
+
2909
+ // Runtime check - tool may have been disabled mid-session
2910
+ if (config.tool !== "on" || config.mode === "off") {
2911
+ return {
2912
+ content: [
2913
+ {
2914
+ type: "text" as const,
2915
+ text: "Error: generate_image tool is currently disabled. Use /multimodal-proxy tool on to enable.",
2916
+ },
2917
+ ],
2918
+ isError: true,
2919
+ };
2920
+ }
2921
+
2922
+ // Share the per-turn cost-runaway budget with analyze_image.
2923
+ const state = getSessionState(extCtx);
2924
+ state.toolCallCount++;
2925
+ if (state.toolCallCount > MAX_TOOL_CALLS_PER_TURN) {
2926
+ return {
2927
+ content: [
2928
+ {
2929
+ type: "text" as const,
2930
+ text: `Error: tool call limit reached (${MAX_TOOL_CALLS_PER_TURN} per turn). Try again in the next turn.`,
2931
+ },
2932
+ ],
2933
+ isError: true,
2934
+ };
2935
+ }
2936
+
2937
+ const outcome = await handleGenerateImage(params, extCtx, config);
2938
+ return {
2939
+ content: outcome.content,
2940
+ ...(outcome.isError ? { isError: true } : {}),
2941
+ ...(outcome.structured ? { structuredContent: outcome.structured } : {}),
2942
+ // `details` feeds the transcript renderer (pi ≥ 1.0.1); the
2943
+ // structured payload is already display-sized (ids, dims, cost).
2944
+ ...(outcome.structured ? { details: outcome.structured } : {}),
2945
+ ...(outcome.usage !== undefined ? { usage: outcome.usage } : {}),
2946
+ };
2947
+ },
2948
+ });
2949
+ _generateToolRegistered = true;
2950
+ }
2406
2951
  // Note: Pi's extension API doesn't have unregisterTool - tool registration
2407
2952
  // persists for the session. The tool's execute handler checks the current
2408
2953
  // config at runtime and returns an error if disabled.
@@ -2418,6 +2963,8 @@ export default function (pi: ExtensionAPI) {
2418
2963
  clearImageData(state.imageData);
2419
2964
  state.compaction = undefined;
2420
2965
  state.keyProbe = undefined;
2966
+ state.dynamicVisionProbe = undefined;
2967
+ state.imageGenModels = undefined;
2421
2968
  state.notifiedDefaultSub = undefined;
2422
2969
 
2423
2970
  _fileConfig = await readPersistentFile();
@@ -2427,7 +2974,7 @@ export default function (pi: ExtensionAPI) {
2427
2974
  ctx.ui.setStatus("multimodal-proxy", steadyStatusText(config, ctx.modelRegistry));
2428
2975
 
2429
2976
  // Register tool if enabled
2430
- syncToolRegistration(config);
2977
+ syncToolRegistration(config, ctx);
2431
2978
 
2432
2979
  // Register the multimodal-proxy/auto virtual model (pi ≥ 0.99)
2433
2980
  syncVirtualModelRegistration(config);
@@ -3319,7 +3866,7 @@ export default function (pi: ExtensionAPI) {
3319
3866
  next.mode === "off" ? "warning" : "info",
3320
3867
  );
3321
3868
  // Sync tool registration on mode change
3322
- syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig));
3869
+ syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig), ctx);
3323
3870
  return;
3324
3871
  }
3325
3872
 
@@ -3731,7 +4278,7 @@ Use "*" or "all" to grant consent for all providers globally.`,
3731
4278
  }
3732
4279
  if (valueLower === "on") {
3733
4280
  const next = writePersisted({ ...persisted, tool: "on" });
3734
- syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig));
4281
+ syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig), ctx);
3735
4282
  ctx.ui.notify(`[multimodal-proxy] analyze_image tool: ON`, "info");
3736
4283
  return;
3737
4284
  }
@@ -4472,7 +5019,7 @@ Use "*" or "all" to grant consent for all providers globally.`,
4472
5019
  if (modeChoice !== "fallback" && modeChoice !== "always" && modeChoice !== "off") return;
4473
5020
  const next = writePersisted({ ...persisted, mode: modeChoice });
4474
5021
  ctx.ui.notify(`Mode set to: ${next.mode}`, "info");
4475
- syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig));
5022
+ syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig), ctx);
4476
5023
  return;
4477
5024
  }
4478
5025
 
@@ -4501,7 +5048,7 @@ Use "*" or "all" to grant consent for all providers globally.`,
4501
5048
  }
4502
5049
  const nextTool = effective.tool === "on" ? "off" : "on";
4503
5050
  writePersisted({ ...persisted, tool: nextTool });
4504
- syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig));
5051
+ syncToolRegistration(resolveConfig(ctx.sessionManager.getEntries(), process.env, _fileConfig), ctx);
4505
5052
  ctx.ui.notify(`Tool: ${nextTool}`, nextTool === "on" ? "info" : "warning");
4506
5053
  return;
4507
5054
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-multimodal-proxy",
3
- "version": "1.19.1",
3
+ "version": "1.21.0",
4
4
  "description": "Automatic image, video and audio description for any model in Pi. Routes media to a multimodal model and injects descriptions into context.",
5
5
  "keywords": [
6
6
  "pi-package"