pi-roundtable 0.9.11 → 0.9.12

package/CHANGELOG.md

@@ -5,6 +5,13 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [0.9.12] - 2026-10-11
+
+### Added
+
+- `generate_image({ prompt, reference_paths? })`: a core session tool, offered to agent and owner sessions only when a plugin fills the `images` provider, that draws a new picture from a description, saves it under `<scratch dir or workspace>/drawings/`, attaches it to the reply, and names the saved path. Up to 4 reference images (PNG, JPEG, WebP or GIF by content, 20 MiB each, 40 MiB together) resolve inside the workspace or the scratch dir like `attach_file` paths. A provider error is a tool error with its message. Owner-only by default; the agents' prompt names it only when offered.
+- `ImageDrawer` takes an optional third argument `{ signal? }`, the asking turn's abort signal; an existing two-argument drawer still fits.
+
 ## [0.9.11] - 2026-10-11
 
 ### Added

package/docs/plugins.md

@@ -693,6 +693,8 @@ A Pi package should use the host's peer dependency on `pi-roundtable`, not bundl
 Transient `SessionContext.runTask` tasks return only text and cannot attach to their parent's reply.
 
 The model attaches a file it already has on disk with the core's `attach_file({ path, filename? })`. A session with a workspace (`SessionContext.workspace`, an agent's) registers it; the path must resolve, symlinks followed, inside the workspace or the scratch dir, a relative path resolves in the workspace, and `filename` defaults to the path's base name. It queues the file with `attachReplyFile`, so `REPLY_FILE_LIMITS` apply and a surface without reply files refuses it as a tool error; its result tells the model the file will appear with its reply. It holds nothing, since it only posts in the channel the turn answers in, and its tier defaults to the owner, like any tool the core's table does not name.
+
+With an `images` provider, agent and owner sessions also offer the core's `generate_image({ prompt, reference_paths? })`, which draws a new picture from a description (`prompt`, at most 4,000 characters). `reference_paths` lists up to 4 local images, checked like `attach_file` paths (inside the workspace or the scratch dir, symlinks followed, regular files only), each a PNG, JPEG, WebP or GIF by content, at most 20 MiB each and 40 MiB together; a refused reference is a tool error before the provider is called. It passes them to the provider as `ReferenceImage`s with the turn's signal, saves the picture under `<scratch dir or workspace>/drawings/generated-<ISO time>.<ext>`, and attaches it with `attachReplyFile`; when it cannot be attached the file is kept and the result says so. The result names the saved path, never the bytes, so the model can send it on or edit it. A provider error, such as a refusal, is a tool error carrying the provider's message. It holds nothing, and its tier defaults to the owner; `toolTiers` lowers it. The agents' prompt mentions it only when it is offered.
 The Discord tools that upload, `discord_send_message` and `discord_edit_message`, take each file as exactly one of `path` (a local file, read when the call runs, with the same roots; `filename` defaults to its base name) or `dataBase64` with a `filename`, at most 8 MiB in all. A session without a workspace takes only `dataBase64`, and so does remote MCP, whose schemas stay inline only. A path outside the roots, a missing file, or a directory is a tool error naming the path. A held call's card (`approvalCard(call, workspace?, limits?)`) lists each file it sends by path with the file's size now, never its bytes, and shows an input within `limits.totalChars` (1,500) whole; a longer input has its string values cut, longest first and never below `limits.valueChars` (200), to their start plus `… [N chars]` (N in code points) until it fits, and if it still does not, the rest is dropped and the keys no longer shown are listed. `approvalDetails(call, workspace?)` gives the same call as data (`{ action, tool, input, files? }`, the whole input, files as `{ path, bytes? }`); the gate passes it to `Prompts.confirm` as `details`, for a surface that is not text.
 Calling the helper outside `context.turns.run` or an agent-team turn, or after that turn finishes, throws `ReplyFileError`; direct standalone runtime calls have no reply collector.
 Await work that produces files before returning from the tool.
@@ -1279,10 +1281,11 @@ An unknown or misspelled slot stops startup with an error listing the valid name
 | Slot | The core's default | Your replacement |
 |---|---|---|
 | `judge` | Asks the configured model small questions: does this reply approve the held actions, how hard is this turn, which agents does this group message concern | An object with `askYesNo`, `askChoice`, and `askScore` |
-| `images` | No drawing; each agent gets an avatar generated from its display name | `async (prompt, references) => bytes` returning PNG bytes |
+| `images` | No drawing; each agent gets an avatar generated from its display name, and no session offers `generate_image` | `async (prompt, references, options?) => bytes` returning PNG, JPEG, WebP or GIF bytes; `options.signal` aborts with the turn that asked |
 | `runtime` | None: the runtime plugin builds the Pi runtime itself | `(deps) => runtime`, an [`AgentRuntime`](#the-runtime-slot-replace-pi) that runs every conversation |
 
-When no `images` provider is configured, `agent_create` has no `avatar_prompt` parameter and `agent_avatar` is unavailable.
+A filled `images` slot draws agents' avatars and also powers the core's [`generate_image`](#attach-files-to-the-agents-reply) tool.
+When no `images` provider is configured, `agent_create` has no `avatar_prompt` parameter, `agent_avatar` is unavailable, and no session offers `generate_image`.
 The owner's profile panel reports the missing provider and offers no redraw option.
 Each agent gets a picture generated from its display name and the assistant's icon, so you can tell agents apart.
 `bunx roundtable doctor` reports whether the slot is filled.

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pi-roundtable",
-	"version": "0.9.11",
+	"version": "0.9.12",
 	"description": "A plugin-driven Pi agent server for one owner, reached through Discord or the web",
 	"type": "module",
 	"license": "MIT",

package/src/core/agents/agent-prompt.ts

@@ -25,6 +25,8 @@ export function agentSystemPrompt(input: {
 	speaker?: Speaker;
 	/** Whether agents can redraw avatars; false without an image provider. Default true. */
 	avatars?: boolean;
+	/** Whether the session offers generate_image; true only with an image provider. Default false. */
+	images?: boolean;
 }): string {
 	const { agent, coordinator, shared, workDir, owner, shellUser, group } =
 		input;
@@ -37,7 +39,7 @@ export function agentSystemPrompt(input: {
 		`You are "${agent.displayName}" (agent name \`${agent.name}\`), one of ${o.name}'s agents in ${o.his} Discord agent server. Each agent owns one channel and one conversation; you all share ${o.his} tools and ${o.his} memory.`,
 		roleText(agent, coordinator, who),
 		`The team: agent_list shows every agent and group; agent_get and agent_update read and improve any agent's prompt, display name, model, and thinking level, yours included; agent_create adds an agent with its own channel; ${input.avatars === false ? "" : "agent_avatar redraws an avatar; "}schedule_list with agent reads another agent's schedules. A message marked as coming from another agent is that agent speaking, not ${w.name}; only ${w.name} approves held actions.`,
-		`Your shell and file tools run on ${o.his} VPS as the user ${shellUser}, in the shared workspace ${workDir}.${input.scratchDir ? ` Write temporary files under the scratch dir ${input.scratchDir} ($TMPDIR), not elsewhere in /tmp.` : ""} Commands that are destructive or change the system, and writes outside the workspace${input.scratchDir ? " and the scratch dir" : ""}, are held for ${w.his} confirmation: tell ${w.him} exactly what will run and ask ${w.him} to confirm. To show a file you have, attach it to your reply with attach_file or give its path to discord_send_message; never read an image or base64 a file to send it.`,
+		`Your shell and file tools run on ${o.his} VPS as the user ${shellUser}, in the shared workspace ${workDir}.${input.scratchDir ? ` Write temporary files under the scratch dir ${input.scratchDir} ($TMPDIR), not elsewhere in /tmp.` : ""} Commands that are destructive or change the system, and writes outside the workspace${input.scratchDir ? " and the scratch dir" : ""}, are held for ${w.his} confirmation: tell ${w.him} exactly what will run and ask ${w.him} to confirm. To show a file you have, attach it to your reply with attach_file or give its path to discord_send_message; never read an image or base64 a file to send it.${input.images ? " To make a new picture, describe it to generate_image, which saves it and attaches it to your reply." : ""}`,
 	];
 	if (group) {
 		const others = group.members

package/src/core/agents/agent-team.ts

@@ -149,6 +149,7 @@ export class DiscordAgentTeam implements AgentOps, AgentTeam {
 			shellUser,
 			...(scratchDir ? { scratchDir } : {}),
 			avatars: this.#avatars(),
+			images: this.#avatars(),
 			...(group
 				? {
 						group: {

package/src/core/contract/providers.ts

@@ -56,10 +56,14 @@ export interface ReferenceImage {
 	mimeType: string;
 }
 
-/** Draws one image from a prompt and reference images; returns PNG or other decodable bytes. */
+/**
+ * Draws one image from a prompt and reference images; returns PNG or other decodable bytes.
+ * `options.signal`, when given, aborts with the turn that asked; a drawer may ignore it.
+ */
 export type ImageDrawer = (
 	prompt: string,
 	references: readonly ReferenceImage[],
+	options?: { signal?: AbortSignal },
 ) => Promise<Uint8Array>;
 
 /** The replaceable parts the core runs on; a plugin provides at most the ones it replaces. */

package/src/core/runtime/runtime-plugin.ts

@@ -168,6 +168,9 @@ export function runtimePlugin(
 						}),
 						confirmations: heldActions,
 						toolTiers: context.toolTiers,
+						...(providers.filled.has("images")
+							? { images: providers.images }
+							: {}),
 						stopping: () => context.queue.closed,
 						get agents() {
 							return agents();

package/src/core/runtime/runtime-types.ts

@@ -4,6 +4,7 @@ import type {
 	ExtensionFactory,
 	ModelRuntime,
 } from "@earendil-works/pi-coding-agent";
+import type { ImageDrawer } from "../contract/providers.ts";
 import type { AgentSessions, LoadedSkill } from "../contract/runtime.ts";
 import type { ChannelKey } from "../domain/conversation.ts";
 import type { InterimTextMode } from "../domain/interim.ts";
@@ -97,6 +98,8 @@ export interface PiAgentRuntimeOptions {
 	interimPrimaryChars?: number;
 	/** How long a new session waits for its MCP tools to register. */
 	mcpConnectTimeoutMs?: number;
+	/** The host's image provider, when one fills the slot; sessions then offer `generate_image`. */
+	images?: ImageDrawer;
 	/** Defaults to true: bridge shared turns refuse raw private exchanges inaccessible to the reader. */
 	memory?: boolean;
 }
@@ -113,6 +116,8 @@ export interface CoreExtensions {
 	readAttachment: ExtensionFactory;
 	/** `attach_file`, for a session with a workspace. */
 	attachFile?: ExtensionFactory;
+	/** `generate_image`, for a host with an image provider. */
+	generateImage?: ExtensionFactory;
 	confirmationGate: ExtensionFactory;
 	askUser: ExtensionFactory;
 	selfCompactGuard: ExtensionFactory;
@@ -144,6 +149,9 @@ export function sessionExtensions(
 		...(core.attachFile
 			? [{ name: "attach-file", factory: core.attachFile }]
 			: []),
+		...(core.generateImage
+			? [{ name: "generate-image", factory: core.generateImage }]
+			: []),
 		{ name: "confirmation-gate", factory: core.confirmationGate },
 		{ name: "ask-user", factory: core.askUser },
 		{ name: "self-compact-guard", factory: core.selfCompactGuard },

package/src/core/runtime/session-factory.ts

@@ -10,6 +10,7 @@ import {
 	type ToolDefinition,
 } from "@earendil-works/pi-coding-agent";
 import { channelSegment } from "../attachments/attachment-dir.ts";
+import type { ImageDrawer } from "../contract/providers.ts";
 import type { ClientPort } from "../domain/client.ts";
 import type { ChannelKey } from "../domain/conversation.ts";
 import { ConfigError } from "../domain/errors.ts";
@@ -28,8 +29,13 @@ import {
 	ATTACH_FILE_TOOL,
 	attachFileExtension,
 } from "../shared/attach-file-tool.ts";
+import {
+	GENERATE_IMAGE_TOOL,
+	generateImageExtension,
+} from "../shared/generate-image-tool.ts";
 import { packageDir } from "../shared/package-dir.ts";
 import { readAttachmentExtension } from "../shared/read-attachment-tool.ts";
+import type { WorkspaceRoots } from "../shared/workspace-files.ts";
 import { type Speaker, THE_SPEAKER } from "../speakers.ts";
 import { currentToolNames } from "../tool-tiers.ts";
 import { toolTurn } from "../tool-turn.ts";
@@ -56,6 +62,7 @@ import type { PromptSlot } from "./prompt-slot.ts";
 import type {
 	ChannelSession,
 	CompactionEnd,
+	CoreExtensions,
 	PiAgentRuntimeOptions,
 } from "./runtime-types.ts";
 import {
@@ -142,6 +149,8 @@ export class SessionFactory {
 			ASK_USER_TOOL,
 			// Registered only by a session with a workspace.
 			ATTACH_FILE_TOOL,
+			// Registered only with an image provider.
+			GENERATE_IMAGE_TOOL,
 			...groups
 				.filter((group) => selection.groups.includes(group.name))
 				.flatMap((group) => group.tools),
@@ -381,6 +390,10 @@ export class SessionFactory {
 				...(context.workspace
 					? { attachFile: attachFileExtension(context.workspace) }
 					: {}),
+				...imageTools(
+					this.#options.images,
+					context.workspace ?? { workspace: cwd },
+				),
 				confirmationGate: confirmationGateExtension(gate, slot, (signal) =>
 					toolTurn(context, signal),
 				),
@@ -442,6 +455,14 @@ export class SessionFactory {
 	}
 }
 
+/** `generate_image` for a session of a host with an image provider; nothing without one. */
+export function imageTools(
+	images: ImageDrawer | undefined,
+	roots: WorkspaceRoots,
+): Pick<CoreExtensions, "generateImage"> {
+	return images ? { generateImage: generateImageExtension(images, roots) } : {};
+}
+
 /** Pi's bash tool with TMPDIR set to `scratchDir`, so `mktemp` and tools write there. */
 export function scratchBash(cwd: string, scratchDir: string): ToolDefinition {
 	// SAFETY: Pi's bash definition is a ToolDefinition typed by its own parameter schema, which the session takes as any custom tool.

package/src/core/shared/generate-image-tool.ts

@@ -0,0 +1,173 @@
+import { mkdir, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import type { ExtensionFactory } from "@earendil-works/pi-coding-agent";
+import { Type } from "typebox";
+import type { ImageDrawer, ReferenceImage } from "../contract/providers.ts";
+import {
+	attachReplyFile,
+	REPLY_FILE_LIMITS,
+	ReplyFileError,
+} from "../reply-files.ts";
+import { toolError, toolText } from "./tool-result.ts";
+import {
+	findWorkspaceFile,
+	readWorkspaceFile,
+	WorkspaceFileError,
+	type WorkspaceRoots,
+} from "./workspace-files.ts";
+
+export const GENERATE_IMAGE_TOOL = "generate_image";
+
+/** The limits on the reference images one call passes to the drawer. */
+export const REFERENCE_IMAGE_LIMITS = Object.freeze({
+	maxImages: 4,
+	maxImageBytes: 20 * 1024 * 1024,
+	maxTotalBytes: 40 * 1024 * 1024,
+});
+
+const MAX_PROMPT_CHARS = 4000;
+
+/** The image type of `data` by its content, or undefined for anything else. */
+export function imageTypeOf(
+	data: Uint8Array,
+): { mimeType: string; ext: string } | undefined {
+	const starts = (bytes: readonly number[], at = 0) =>
+		bytes.every((byte, i) => data[at + i] === byte);
+	if (starts([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]))
+		return { mimeType: "image/png", ext: "png" };
+	if (starts([0xff, 0xd8, 0xff])) return { mimeType: "image/jpeg", ext: "jpg" };
+	if (starts([0x47, 0x49, 0x46, 0x38]))
+		return { mimeType: "image/gif", ext: "gif" };
+	if (starts([0x52, 0x49, 0x46, 0x46]) && starts([0x57, 0x45, 0x42, 0x50], 8))
+		return { mimeType: "image/webp", ext: "webp" };
+	return undefined;
+}
+
+async function readReferences(
+	paths: readonly string[],
+	roots: WorkspaceRoots,
+): Promise<ReferenceImage[]> {
+	const { maxImages, maxImageBytes, maxTotalBytes } = REFERENCE_IMAGE_LIMITS;
+	if (paths.length > maxImages)
+		throw new WorkspaceFileError(
+			`${paths.length} reference images given; at most ${maxImages} are allowed.`,
+		);
+	const files = await Promise.all(
+		paths.map((path) => findWorkspaceFile(path, roots)),
+	);
+	let total = 0;
+	for (const file of files) {
+		if (file.size > maxImageBytes)
+			throw new WorkspaceFileError(
+				`${file.path} is ${file.size} bytes; a reference image may contain at most ${maxImageBytes} bytes.`,
+			);
+		total += file.size;
+	}
+	if (total > maxTotalBytes)
+		throw new WorkspaceFileError(
+			`the reference images total ${total} bytes; at most ${maxTotalBytes} are allowed.`,
+		);
+	const references: ReferenceImage[] = [];
+	for (const file of files) {
+		const data = await readWorkspaceFile(file, maxImageBytes);
+		const type = imageTypeOf(data);
+		if (!type)
+			throw new WorkspaceFileError(
+				`${file.path} is not a PNG, JPEG, WebP or GIF image.`,
+			);
+		references.push({
+			data: Buffer.from(data).toString("base64"),
+			mimeType: type.mimeType,
+		});
+	}
+	return references;
+}
+
+/**
+ * Registers `generate_image`, which draws a new picture with the host's image provider, saves it
+ * under `<scratch dir or workspace>/drawings`, and attaches it to the turn's reply.
+ */
+export function generateImageExtension(
+	draw: ImageDrawer,
+	roots: WorkspaceRoots,
+): ExtensionFactory {
+	return (pi) => {
+		pi.registerTool({
+			name: GENERATE_IMAGE_TOOL,
+			label: "Generate image",
+			description:
+				'Generate a new picture from a description. Describe the subject, style and composition; say "no text" when you will add text to it later. The picture is saved to a file whose path the result names, and attached to your reply.',
+			parameters: Type.Object(
+				{
+					prompt: Type.String({
+						minLength: 1,
+						maxLength: MAX_PROMPT_CHARS,
+						description:
+							"What to draw: subject, style, composition, and anything to leave out.",
+					}),
+					reference_paths: Type.Optional(
+						Type.Array(Type.String({ minLength: 1, maxLength: 4096 }), {
+							maxItems: REFERENCE_IMAGE_LIMITS.maxImages,
+							description:
+								"Paths of PNG, JPEG, WebP or GIF images inside the workspace or the scratch dir to draw from.",
+						}),
+					),
+				},
+				{ additionalProperties: false },
+			),
+			execute: async (_toolCallId, params, signal) => {
+				let references: ReferenceImage[];
+				try {
+					references = await readReferences(
+						params.reference_paths ?? [],
+						roots,
+					);
+				} catch (error) {
+					if (error instanceof WorkspaceFileError)
+						return toolError(`Not drawn: ${error.message}`);
+					throw error;
+				}
+				signal?.throwIfAborted();
+				let data: Uint8Array;
+				try {
+					data = await draw(
+						params.prompt,
+						references,
+						signal ? { signal } : {},
+					);
+				} catch (error) {
+					signal?.throwIfAborted();
+					return toolError(
+						`The image provider did not draw it: ${error instanceof Error ? error.message : String(error)}`,
+					);
+				}
+				signal?.throwIfAborted();
+				const type = imageTypeOf(data);
+				if (!type)
+					return toolError(
+						"The image provider returned something that is not a PNG, JPEG, WebP or GIF image.",
+					);
+				const dir = join(roots.scratchDir ?? roots.workspace, "drawings");
+				await mkdir(dir, { recursive: true });
+				const name = `generated-${new Date().toISOString().replace(/[:.]/g, "-")}.${type.ext}`;
+				const path = join(dir, name);
+				await writeFile(path, data);
+				try {
+					if (data.byteLength > REPLY_FILE_LIMITS.maxFileBytes)
+						throw new ReplyFileError(
+							`it is ${data.byteLength} bytes; a reply file may contain at most ${REPLY_FILE_LIMITS.maxFileBytes} bytes.`,
+						);
+					attachReplyFile({ name, data });
+				} catch (error) {
+					if (!(error instanceof ReplyFileError)) throw error;
+					return toolText(
+						`Drew the picture and saved it to ${path} (${data.byteLength} bytes), but it is not attached: ${error.message}`,
+					);
+				}
+				return toolText(
+					`Drew the picture, saved it to ${path} (${data.byteLength} bytes); it will appear with your reply.`,
+				);
+			},
+		});
+	};
+}