pi-roundtable 0.12.1 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,23 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.13.0] - 2026-10-13
9
+
10
+ ### Added
11
+
12
+ - `formatChannelContext` (`formatChannelContext(messages)`) is exported from `pi-roundtable/discord`, the function that writes `ChannelContext.text` (the `<channel-context>` block with each author in attributes and every `<` neutralised), so a host can test what the model sees without a deep import. It is the function the reader already used; nothing about the block changes.
13
+ - `squarePng`, `SquarePngOptions` and `SQUARE_PNG_MAX_SIDE` (4096) are exported from `pi-roundtable/kit`. `squarePng(bytes, { size?, maxBytes?, maxSide? })` is the centered square crop, scaled down to at most `size` pixels per side (default 512) as PNG, that the agent avatars are made with. The header is read before decoding: a declared width or height over 4096 (or `maxSide`) rejects with `ImagePreparationError` (`pixel-limit`, with a message that names the side limit), bytes of no PNG, JPEG or GIF with `invalid-image`; a WebP is refused the same way, since the canvas cannot decode it, and so is any picture that fails to decode. With `maxBytes` the side steps down by a quarter until the PNG fits, and a picture that does not fit at 16 pixels throws a `RangeError`.
14
+ - `discord.attachmentDownload: { deadlineMs?, idleMs? }` (each a whole number of milliseconds from 1,000 to 600,000) sets the limits of the downloads of the files of an owner or agent channel message on the agent server: `deadlineMs` is the overall deadline for the message and its reference together (default 60,000) and `idleMs` the time one file may send no byte before it is given up (default 10,000). `attachmentsOf` takes the same two as `limits.deadlineMs` and `limits.idleMs`, and `AgentServerOptions.attachmentDownload` passes them on for a host built without `defineRoundtable`. A file that goes quiet is recorded as `download failed: idle_timeout` and the next file is tried at once.
15
+
16
+ ### Changed
17
+
18
+ - The avatar studio refuses a drawn picture that declares a side over 4096 (`ImagePreparationError`, `pixel-limit`), where it used to decode it. The configured `avatar` is not held to that: a photo of any side within the 64 MP decoded-pixel budget is accepted, and one that cannot be read as a picture is logged (`the configured avatar could not be used`, with its error code) and replaced by the plain bot, where it would stop the host from starting.
19
+ - The default overall deadline for downloading the files of a message and its reference rises from 20 to 60 seconds, since a stalled download is now caught by the 10 second idle timeout instead of by the deadline: a body that sends no byte for 10 seconds (the wait for the headers counts, also for a fetch that ignores its abort signal) ends that file alone, where it used to hold every file still to come until the deadline. A slow download that keeps sending bytes now has up to 60 seconds. A host that wants the old bound sets `discord.attachmentDownload.deadlineMs: 20000`.
20
+
21
+ ### Fixed
22
+
23
+ - An attachment's failure reason never quotes a file system path. A download that fails for any reason other than the ones with their own wording (a size over 25 MB, an HTTP status, the deadline) is recorded as `download failed: <code>`, where the code is the error's `code` (such as `ENOTDIR`, for a directory that could not be made, or `ECONNRESET`) or its `name` (such as `TimeoutError`), and `error` when it has neither; it used to be the error's own text, and the text of a failed `mkdir` quotes the directory. The reason is shown to the model in the turn's attachment block and to the channel, so a path of the host no longer reaches either.
24
+
8
25
  ## [0.12.1] - 2026-10-13
9
26
 
10
27
  ### Fixed
package/README.md CHANGED
@@ -197,6 +197,7 @@ When a conversation is started over in a server channel or thread, the assistant
197
197
  | `false` | | Post no divider |
198
198
 
199
199
  An approval card or an `ask_user` question holds its turn for `discord.cardGraceSeconds` (default `120`).
200
+ The files of a message in an agent's channel download within `discord.attachmentDownload.deadlineMs` (default 60,000) for the message and the one it replies to together, and a file that sends no byte for `discord.attachmentDownload.idleMs` (default 10,000) is given up; the turn goes on without a file it could not get.
200
201
  The copy button of an approval card keeps its text in `card-copies.json` under the data directory, the newest 100 for 30 days; `discord.cardCopies` (`{ keepDays?, keep? }`) changes that, and `false` keeps them in memory only.
201
202
  Answered within it, the turn goes on with the answer; otherwise the turn ends saying it is waiting, the card stays open with no deadline, and answering it later starts a new turn in that conversation with the answer, or with the result of the approved action.
202
203
  A coding worker's cards keep the worker waiting instead.
package/README.zh-TW.md CHANGED
@@ -185,6 +185,7 @@ export default {
185
185
  | `attachments` | `{ images: 0, maxBytes: 10 MiB }` | `images`:替這一輪抓取保留訊息中的幾張圖片,最多 4 張 |
186
186
 
187
187
  核准卡和 `ask_user` 問題會讓這一輪等 `discord.cardGraceSeconds` 秒(預設 `120`)。
188
+ agent 頻道裡訊息的附件,整則訊息(連同它回覆的那則)要在 `discord.attachmentDownload.deadlineMs`(預設 60,000)毫秒內下載完;單一檔案連續 `discord.attachmentDownload.idleMs`(預設 10,000)毫秒沒收到任何位元組就放棄,這一輪會略過收不到的檔案繼續。
188
189
  核准卡「複製」按鈕的文字存在資料目錄的 `card-copies.json`,預設留最新 100 張、30 天;`discord.cardCopies`(`{ keepDays?, keep? }`)可以調整,設成 `false` 則只留在記憶體。
189
190
  在這段時間內回答,這一輪就帶著答案繼續;沒回答的話,這一輪會說它在等你,然後結束,卡片繼續開著、沒有期限,之後再回答就會在同一個對話開新的一輪,帶上你的答案,或核准後那個動作的結果。
190
191
  coding worker 的卡片則會讓 worker 一直等下去。
package/docs/plugins.md CHANGED
@@ -60,6 +60,9 @@ Use the kit's building blocks for a plugin that runs Pi itself, such as a coding
60
60
  `ImagePreparationError.reason` is `byte-limit`, `pixel-limit`, or `invalid-image`; hosts can localize refusals without parsing native decoder errors.
61
61
  Ordinary 48 MP images and thin 9000×1 images remain admitted.
62
62
  This helper is not a downloader or filesystem validator.
63
+ - Square pictures: `squarePng(bytes, { size?, maxBytes?, maxSide? })` crops a PNG, JPEG or GIF to its centered square and scales it down to at most `size` pixels per side (default 512; a smaller picture is not enlarged), as PNG: the helper the agent avatars are made with, for a plugin that needs a profile picture or an icon.
64
+ The header is read first, and nothing is decoded for a picture that declares a width or height over `SQUARE_PNG_MAX_SIDE` (4096), which rejects with `ImagePreparationError` (`pixel-limit`, the message naming the side limit; `maxSide` sets another limit, and the 64 MP decoded-pixel budget always applies), or for bytes of no such type, a WebP (the canvas cannot decode it) or a picture that fails to decode (`invalid-image`).
65
+ With `maxBytes` the side steps down by a quarter at a time until the PNG is no larger; a picture that does not fit even at 16 pixels throws a `RangeError`, as does a `size` or `maxBytes` that is not a whole number in range. `SquarePngOptions` types the options.
63
66
  - Work: `promptSlot` (how a run asks the owner while it works), `workTimeout` (a time limit that does not count the time spent waiting on the owner), `runWorkerTask`, `archiveSessions`, and `approvalCard`, `approvalDetails` and `canonicalJson` for the cards of held actions.
64
67
  `InterimPoster(posts, { logger, channel, primaryChars?, editMs? })` posts a run's [interim text](#interim-text-what-a-turn-writes-before-its-final-answer) to an `InterimPosts`, such as a dispatch thread's `interim`: feed it `messageEnd(message)` and `toolStart(name)` from the session's events, bind `() => poster.flush()` as the slot's `beforeCard`, and `await poster.flush()` before the final report.
65
68
  - Diagnostics: `scrubDiagnostic(text, max = 600)` masks credentials (URL userinfo, token shapes, secret-named assignments and JSON fields, `Authorization`/`Cookie`/`x-api-key` headers, JWTs, PEM blocks), turns control characters other than tab and newline into spaces, and cuts the result at `max` characters.
@@ -553,7 +556,7 @@ A key the contract does not have stops the start and names the closest one.
553
556
  ### `context.attachments`: files from outside a turn
554
557
 
555
558
  A surface that takes a file as it arrives, such as Discord, hands the core a link, and the core downloads it into the conversation's attachment directory.
556
- The downloads of one message and of the message it replies to share a 20 second deadline; a file still to come when it passes, or whose directory cannot be made, is reported as `download failed` and the turn goes on without it.
559
+ The downloads of one message and of the message it replies to share a 60 second deadline, and a file that sends no byte for 10 seconds (the wait for its headers counts) is given up at once, so a stalled download costs 10 seconds, not the whole deadline. A file still to come when the deadline passes, a file that went quiet (`download failed: idle_timeout`), or one whose directory cannot be made, is reported as `download failed` and the turn goes on without it. `attachmentsOf(message, dir, logger, fetch?, { deadlineMs?, idleMs?, signal? })` takes both limits; on the agent server they are `discord.attachmentDownload: { deadlineMs?, idleMs? }` (each 1,000 to 600,000 ms; defaults 60,000 and 10,000), for a host on a slow link or with large files. A failure reason carries a code only (`download failed: ENOTDIR`, `download failed: TimeoutError`, `download failed: error`), never the error's text, which could quote a path of the host.
557
560
  A plugin whose clients upload a file themselves, over HTTP for instance, has the file before any turn exists.
558
561
  `context.attachments` keeps such a file for the conversation until a turn uses it:
559
562
 
@@ -2317,6 +2320,7 @@ A surface a plugin contributes may implement the optional `ChatSurface.markFresh
2317
2320
  The block gives each author as a name, their Discord id and a role (`owner`, `bot` or `member`) in separate attributes, and says the messages were not addressed to the assistant. A name loses the characters `"`, `'`, `=`, `<`, `>`, `(`, `)` and `,`, so it cannot imitate another author, an attribute or a role, and every `<` (also the full-width `<`) in a message's text is replaced by `‹`, so a message can neither close the block nor open a message of its own.
2318
2321
  `ChannelContext` is `{ messages, text, images? }`: `messages` are `ChannelContextMessage`s (`id`, `authorId`, `authorName`, `bot`, `owner`, `text`, `at`, `attachments`), oldest first, `text` is the block, and `images` the fetched images (see above).
2319
2322
  `ChannelContextOptions` holds the options above, and `CHANNEL_CONTEXT_DEFAULTS` their defaults.
2323
+ `formatChannelContext(messages)` is the function that writes `ChannelContext.text`, exported so a host can test what the model sees without reaching into the core: it takes `ChannelContextMessage`s (only `authorId`, `authorName`, `bot`, `owner` and `text` are read) and returns the block, headed and delimited by `<channel-context>`. Each message is a `<message from="…" id="…" role="owner|member|bot">` line whose author is in the attributes, never in the text, and every `<` of a name or text (plain or full-width) is replaced by `‹`, so no message can pass as markup. It does not fetch, cut or select anything, and it is the same function the reader calls, so a block it returns is the block a turn carries.
2320
2324
  In a test, `fakeDiscord({ channelContext })` answers `DISCORD.channelContext` as the test says; by default with no context.
2321
2325
 
2322
2326
  <!-- example: examples/channel-context.ts -->
@@ -3727,6 +3731,9 @@ Import from the entries listed below; source area files are internal.
3727
3731
  | `withReplyFiles` | `pi-roundtable` | value |
3728
3732
  | `prepareImageBytes` | `pi-roundtable/kit` | value |
3729
3733
  | `ImagePreparationError` | `pi-roundtable/kit` | value |
3734
+ | `squarePng` | `pi-roundtable/kit` | value |
3735
+ | `SQUARE_PNG_MAX_SIDE` | `pi-roundtable/kit` | value |
3736
+ | `SquarePngOptions` | `pi-roundtable/kit` | type |
3730
3737
  | `scrubDiagnostic` | `pi-roundtable/kit` | value |
3731
3738
  | `channelKey` | `pi-roundtable` | value |
3732
3739
  | `definePlugin` | `pi-roundtable` | value |
@@ -3923,6 +3930,7 @@ Import from the entries listed below; source area files are internal.
3923
3930
  | `ChannelContextAttachmentOptions` | `pi-roundtable/discord` | type |
3924
3931
  | `ChannelContextMessage` | `pi-roundtable/discord` | type |
3925
3932
  | `ChannelContextOptions` | `pi-roundtable/discord` | type |
3933
+ | `formatChannelContext` | `pi-roundtable/discord` | value |
3926
3934
  | `ChannelExecutor` | `pi-roundtable/discord` | type |
3927
3935
  | `ChannelInfo` | `pi-roundtable/discord` | type |
3928
3936
  | `ChannelOperation` | `pi-roundtable/discord` | type |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-roundtable",
3
- "version": "0.12.1",
3
+ "version": "0.13.0",
4
4
  "description": "A plugin-driven Pi agent server for one owner, reached through Discord or the web",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -60,6 +60,8 @@ export interface AgentClaimOptions {
60
60
  surface: Pick<SurfacePort, "react" | "unreact">;
61
61
  /** Where a channel's attachments are saved. */
62
62
  attachmentDir: (channel: ChannelKey) => string;
63
+ /** The downloads' limits: all of a message's files together (default 60 s), and one file without a byte (default 10 s). */
64
+ attachmentDownload?: { deadlineMs?: number; idleMs?: number };
63
65
  logger: Logger;
64
66
  /** Replaceable in tests. */
65
67
  fetchImpl?: typeof fetch;
@@ -80,7 +82,13 @@ export interface AgentClaimOptions {
80
82
  export function agentClaim(options: AgentClaimOptions): ChannelClaim {
81
83
  const { team, runtime, surface, attachmentDir, logger, fetchImpl } = options;
82
84
  const attachments = (message: InboundMessage) =>
83
- attachmentsOf(message, attachmentDir(message.channel), logger, fetchImpl);
85
+ attachmentsOf(
86
+ message,
87
+ attachmentDir(message.channel),
88
+ logger,
89
+ fetchImpl,
90
+ options.attachmentDownload,
91
+ );
84
92
  /** The channel context of a message in an agent's channel; none, logged, when it cannot be read. */
85
93
  const around = async (
86
94
  message: InboundMessage,
@@ -1,7 +1,7 @@
1
1
  import { createHash } from "node:crypto";
2
2
  import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
3
3
  import { join } from "node:path";
4
- import { createCanvas, loadImage } from "canvas";
4
+ import { squarePng } from "../attachments/square-png.ts";
5
5
  import type { ImageDrawer } from "../contract/providers.ts";
6
6
  import { ProviderError } from "../errors.ts";
7
7
  import type { HttpRoute } from "../http/listeners.ts";
@@ -10,6 +10,8 @@ import { fallbackAvatar } from "./fallback-avatar.ts";
10
10
 
11
11
  const AVATAR_SIDE = 512;
12
12
  const REFERENCE_SIDE = 512;
13
+ /** The configured reference is the host's own photo, so only the decoded-pixel budget limits it. */
14
+ const REFERENCE_MAX_SIDE = Number.MAX_SAFE_INTEGER;
13
15
  const HASH_PATH = /^\/avatars\/([0-9a-f]{64})\.png$/;
14
16
 
15
17
  export type { ImageDrawer };
@@ -21,32 +23,14 @@ export interface AvatarStudioOptions {
21
23
  publicUrl: string;
22
24
  /** The assistant's neutral avatar: the style reference and the picture of an agent without one. */
23
25
  referencePath: string;
26
+ /** The neutral avatar shipped with the package; the reference is replaced by it when the reference cannot be read as a picture. */
27
+ neutralPath?: string;
28
+ /** Told the error code of a reference that was replaced by `neutralPath`. */
29
+ onReferenceFailed?: (code: string) => void;
24
30
  /** Draws the pictures. Without one, the studio only makes each agent a picture from its name. */
25
31
  draw?: ImageDrawer;
26
32
  }
27
33
 
28
- /** A centered square crop scaled to at most `side` pixels, as PNG. */
29
- async function squarePng(bytes: Uint8Array, side: number): Promise<Buffer> {
30
- const image = await loadImage(Buffer.from(bytes));
31
- const crop = Math.min(image.width, image.height);
32
- const size = Math.min(side, crop);
33
- const canvas = createCanvas(size, size);
34
- canvas
35
- .getContext("2d")
36
- .drawImage(
37
- image,
38
- (image.width - crop) / 2,
39
- (image.height - crop) / 2,
40
- crop,
41
- crop,
42
- 0,
43
- 0,
44
- size,
45
- size,
46
- );
47
- return canvas.toBuffer("image/png");
48
- }
49
-
50
34
  /**
51
35
  * Agent avatars: an image provider draws them in the assistant's style, and without one each agent
52
36
  * gets a picture generated from its display name. They are kept and served by content hash, so a
@@ -55,17 +39,37 @@ async function squarePng(bytes: Uint8Array, side: number): Promise<Buffer> {
55
39
  export class FileAvatarStudio implements AvatarStudio {
56
40
  readonly #options: AvatarStudioOptions;
57
41
  #defaultHash: string | undefined;
42
+ #referencePath: string;
58
43
 
59
44
  constructor(options: AvatarStudioOptions) {
60
45
  this.#options = options;
46
+ this.#referencePath = options.referencePath;
61
47
  mkdirSync(options.dir, { recursive: true });
62
48
  }
63
49
 
64
50
  /** Stores the assistant's neutral avatar, the picture of an agent that has none yet. */
65
51
  async init(): Promise<void> {
66
- this.#defaultHash = await this.#store(
67
- readFileSync(this.#options.referencePath),
68
- );
52
+ try {
53
+ this.#defaultHash = await this.#referenceStore();
54
+ } catch (error) {
55
+ const { neutralPath, onReferenceFailed } = this.#options;
56
+ if (!neutralPath || neutralPath === this.#referencePath) throw error;
57
+ const code =
58
+ (error as { code?: unknown }).code ??
59
+ (error as { reason?: unknown }).reason ??
60
+ (error as Error).name;
61
+ onReferenceFailed?.(
62
+ /^[A-Za-z][A-Za-z0-9_-]{0,40}$/.test(String(code))
63
+ ? String(code)
64
+ : "error",
65
+ );
66
+ this.#referencePath = neutralPath;
67
+ this.#defaultHash = await this.#referenceStore();
68
+ }
69
+ }
70
+
71
+ async #referenceStore(): Promise<string> {
72
+ return this.#store(readFileSync(this.#referencePath), REFERENCE_MAX_SIDE);
69
73
  }
70
74
 
71
75
  /** Whether an image provider is configured; without one, only `fallback` makes pictures. */
@@ -87,10 +91,10 @@ export class FileAvatarStudio implements AvatarStudio {
87
91
  */
88
92
  async draw(avatarPrompt: string): Promise<string> {
89
93
  const draw = this.#drawer();
90
- const reference = await squarePng(
91
- readFileSync(this.#options.referencePath),
92
- REFERENCE_SIDE,
93
- );
94
+ const reference = await squarePng(readFileSync(this.#referencePath), {
95
+ size: REFERENCE_SIDE,
96
+ maxSide: REFERENCE_MAX_SIDE,
97
+ });
94
98
  const bytes = await draw(
95
99
  `Draw a square Discord avatar of the person in the reference image, redrawn as an anime character: keep their face, short dark hair, and features recognizable as the same person, but in vivid anime cel-shading with a big, exaggerated expression and a dramatic pose that shows the role. Dress them and paint the background exactly in the colours the description names; ignore the reference's green jacket and teal background. Head-and-shoulders framing that still reads at 40 px. No text, no border. The role: ${avatarPrompt}`,
96
100
  [{ data: reference.toString("base64"), mimeType: "image/png" }],
@@ -119,7 +123,7 @@ export class FileAvatarStudio implements AvatarStudio {
119
123
  return this.#store(
120
124
  await fallbackAvatar(
121
125
  displayName,
122
- readFileSync(this.#options.referencePath),
126
+ readFileSync(this.#referencePath),
123
127
  name,
124
128
  ),
125
129
  );
@@ -171,8 +175,11 @@ export class FileAvatarStudio implements AvatarStudio {
171
175
  return join(this.#options.dir, `${hash}.png`);
172
176
  }
173
177
 
174
- async #store(bytes: Uint8Array): Promise<string> {
175
- const png = await squarePng(bytes, AVATAR_SIDE);
178
+ async #store(bytes: Uint8Array, maxSide?: number): Promise<string> {
179
+ const png = await squarePng(bytes, {
180
+ size: AVATAR_SIDE,
181
+ ...(maxSide === undefined ? {} : { maxSide }),
182
+ });
176
183
  const hash = createHash("sha256").update(png).digest("hex");
177
184
  const path = this.#path(hash);
178
185
  if (!existsSync(path)) writeFileSync(path, png);
@@ -9,6 +9,9 @@ import { plainText } from "./attachment-name.ts";
9
9
 
10
10
  export const MAX_ATTACHMENT_BYTES = 25 * 1024 * 1024;
11
11
 
12
+ /** How long one download may go without a byte (the headers count as the first) before it is given up, by default. */
13
+ export const DEFAULT_IDLE_MS = 10_000;
14
+
12
15
  export interface FetchRequest {
13
16
  refs: readonly AttachmentRef[];
14
17
  /** Directory of the channel's attachment store; created when missing. */
@@ -18,6 +21,8 @@ export interface FetchRequest {
18
21
  fromReference: boolean;
19
22
  /** Aborts the download of the files still to come, and the one in flight. */
20
23
  signal?: AbortSignal;
24
+ /** Gives up on a file that sends no byte for this long, and goes on to the next; default 10 s. */
25
+ idleMs?: number;
21
26
  }
22
27
 
23
28
  export interface FetchResult {
@@ -58,6 +63,70 @@ export function safeFileName(name: string): string {
58
63
  return cleaned || "file";
59
64
  }
60
65
 
66
+ /**
67
+ * A failure's code, never its message: a message can quote the directory a file was to be written
68
+ * to, a host path the model and the channel have no use for. The code is an error's `code` (such
69
+ * as `ENOTDIR`) or its `name` (such as `TimeoutError`), and `error` when it has neither.
70
+ */
71
+ function failureCode(error: unknown): string {
72
+ const word = /^[A-Za-z][A-Za-z0-9_]{0,40}$/;
73
+ if (typeof error !== "object" || error === null) return "error";
74
+ const { code, name } = error as { code?: unknown; name?: unknown };
75
+ if (typeof code === "string" && word.test(code)) return code;
76
+ if (typeof name === "string" && name !== "Error" && word.test(name))
77
+ return name;
78
+ return "error";
79
+ }
80
+
81
+ /**
82
+ * Ends a download that has gone quiet: `signal` aborts, with the reason `idle`, once `idleMs` pass
83
+ * without a `touch`, and also when the outer signal does.
84
+ */
85
+ class IdleWatch {
86
+ readonly signal: AbortSignal;
87
+ readonly #own = new AbortController();
88
+ readonly #idleMs: number;
89
+ #timer: ReturnType<typeof setTimeout> | undefined;
90
+
91
+ constructor(idleMs: number, outer?: AbortSignal) {
92
+ this.#idleMs = idleMs;
93
+ this.signal = outer
94
+ ? AbortSignal.any([outer, this.#own.signal])
95
+ : this.#own.signal;
96
+ this.touch();
97
+ }
98
+
99
+ /** Whether the download was ended for want of bytes, not by the outer signal. */
100
+ get idled(): boolean {
101
+ return this.#own.signal.aborted;
102
+ }
103
+
104
+ touch(): void {
105
+ clearTimeout(this.#timer);
106
+ this.#timer = setTimeout(
107
+ () => this.#own.abort(new Error("idle")),
108
+ this.#idleMs,
109
+ );
110
+ }
111
+
112
+ stop(): void {
113
+ clearTimeout(this.#timer);
114
+ }
115
+ }
116
+
117
+ /** Rejects with the signal's reason when it aborts, so a fetch that ignores the signal still ends. */
118
+ function untilAborted<T>(work: Promise<T>, signal: AbortSignal): Promise<T> {
119
+ if (signal.aborted) return Promise.reject(signal.reason);
120
+ let onAbort: (() => void) | undefined;
121
+ const aborted = new Promise<never>((_, reject) => {
122
+ onAbort = () => reject(signal.reason);
123
+ signal.addEventListener("abort", onAbort, { once: true });
124
+ });
125
+ return Promise.race([work, aborted]).finally(() => {
126
+ if (onAbort) signal.removeEventListener("abort", onAbort);
127
+ });
128
+ }
129
+
61
130
  /** Downloads attachments one by one; a failed file is reported without stopping the others. */
62
131
  export async function fetchAttachments(
63
132
  request: FetchRequest,
@@ -66,6 +135,7 @@ export async function fetchAttachments(
66
135
  const files: StoredAttachment[] = [];
67
136
  const failures: AttachmentFailure[] = [];
68
137
  const { refs, dir, prefix, fromReference, signal } = request;
138
+ const idleMs = request.idleMs ?? DEFAULT_IDLE_MS;
69
139
  if (refs.length === 0) return { files, failures };
70
140
 
71
141
  for (const [index, ref] of refs.entries()) {
@@ -79,19 +149,20 @@ export async function fetchAttachments(
79
149
  fail(`larger than 25 MB (${ref.size} bytes)`);
80
150
  continue;
81
151
  }
152
+ const watch = new IdleWatch(idleMs, signal);
82
153
  try {
83
154
  // A directory that cannot be made fails the file, not the turn.
84
155
  mkdirSync(dir, { recursive: true, mode: 0o700 });
85
- const response = await fetchImpl(
86
- ref.url,
87
- signal ? { signal } : undefined,
156
+ const response = await untilAborted(
157
+ fetchImpl(ref.url, { signal: watch.signal }),
158
+ watch.signal,
88
159
  );
89
160
  if (!response.ok) {
90
161
  await response.body?.cancel();
91
162
  fail(`download failed with HTTP ${response.status}`);
92
163
  continue;
93
164
  }
94
- const data = await readAttachmentBody(response, signal);
165
+ const data = await readAttachmentBody(response, watch);
95
166
  if (typeof data === "number") {
96
167
  fail(`larger than 25 MB (${data} bytes)`);
97
168
  continue;
@@ -111,7 +182,11 @@ export async function fetchAttachments(
111
182
  fromReference,
112
183
  });
113
184
  } catch (error) {
114
- fail(`download failed: ${String(error)}`);
185
+ fail(
186
+ `download failed: ${watch.idled && !signal?.aborted ? "idle_timeout" : failureCode(error)}`,
187
+ );
188
+ } finally {
189
+ watch.stop();
115
190
  }
116
191
  }
117
192
  return { files, failures };
@@ -120,20 +195,22 @@ export async function fetchAttachments(
120
195
  /** Returns the bytes, or the observed size once the stream exceeds the limit. */
121
196
  async function readAttachmentBody(
122
197
  response: Response,
123
- signal?: AbortSignal,
198
+ watch: IdleWatch,
124
199
  ): Promise<Uint8Array | number> {
125
200
  if (!response.body) return new Uint8Array();
201
+ const { signal } = watch;
126
202
  const reader = response.body.getReader();
127
203
  const chunks: Uint8Array[] = [];
128
204
  let size = 0;
129
205
  // A body that stalls ends with the signal, also when the fetch itself ignores it.
130
- const stop = () => void reader.cancel(signal?.reason).catch(() => {});
131
- signal?.addEventListener("abort", stop, { once: true });
206
+ const stop = () => void reader.cancel(signal.reason).catch(() => {});
207
+ signal.addEventListener("abort", stop, { once: true });
132
208
  try {
133
- if (signal?.aborted) throw signal.reason ?? new Error("aborted");
209
+ if (signal.aborted) throw signal.reason ?? new Error("aborted");
134
210
  for (;;) {
135
211
  const { done, value } = await reader.read();
136
212
  if (done) break;
213
+ watch.touch();
137
214
  size += value.byteLength;
138
215
  if (size > MAX_ATTACHMENT_BYTES) {
139
216
  await reader.cancel();
@@ -142,9 +219,9 @@ async function readAttachmentBody(
142
219
  chunks.push(value);
143
220
  }
144
221
  // A cancelled reader reads as the end of the body, so a cut-short body is not a whole file.
145
- if (signal?.aborted) throw signal.reason ?? new Error("aborted");
222
+ if (signal.aborted) throw signal.reason ?? new Error("aborted");
146
223
  } finally {
147
- signal?.removeEventListener("abort", stop);
224
+ signal.removeEventListener("abort", stop);
148
225
  reader.releaseLock();
149
226
  }
150
227
  const data = new Uint8Array(size);
@@ -2,10 +2,22 @@ import { ImagePreparationError } from "./image-preparation-error.ts";
2
2
 
3
3
  const invalid = () => new ImagePreparationError("invalid-image");
4
4
 
5
- /** Inspect only encoded headers/blocks. Never allocate a raster or follow a media URL. */
6
- export function assertImagePixelBudget(bytes: Buffer, budget: number): void {
5
+ /**
6
+ * Inspect only encoded headers/blocks. Never allocate a raster or follow a media URL. With
7
+ * `maxSide`, a declared width or height (of a canvas or of any frame) over it is `pixel-limit` too.
8
+ */
9
+ export function assertImagePixelBudget(
10
+ bytes: Buffer,
11
+ budget: number,
12
+ maxSide?: number,
13
+ ): void {
7
14
  const pixels = (width: number, height: number): number => {
8
15
  if (width < 1 || height < 1) throw invalid();
16
+ if (maxSide !== undefined && (width > maxSide || height > maxSide))
17
+ throw new ImagePreparationError(
18
+ "pixel-limit",
19
+ `Image exceeds the ${maxSide}-pixel side limit.`,
20
+ );
9
21
  const area = width * height;
10
22
  check(area);
11
23
  return area;
@@ -2,13 +2,17 @@
2
2
  export class ImagePreparationError extends Error {
3
3
  override readonly name = "ImagePreparationError";
4
4
 
5
- constructor(readonly reason: "byte-limit" | "pixel-limit" | "invalid-image") {
5
+ constructor(
6
+ readonly reason: "byte-limit" | "pixel-limit" | "invalid-image",
7
+ detail?: string,
8
+ ) {
6
9
  super(
7
- {
8
- "byte-limit": "Image is larger than 25 MiB.",
9
- "pixel-limit": "Image exceeds the 64 MP decoded-pixel budget.",
10
- "invalid-image": "Image has an invalid or unsupported raster header.",
11
- }[reason],
10
+ detail ??
11
+ {
12
+ "byte-limit": "Image is larger than 25 MiB.",
13
+ "pixel-limit": "Image exceeds the 64 MP decoded-pixel budget.",
14
+ "invalid-image": "Image has an invalid or unsupported raster header.",
15
+ }[reason],
12
16
  );
13
17
  }
14
18
  }
@@ -0,0 +1,94 @@
1
+ import { createCanvas, loadImage } from "canvas";
2
+ import { assertImagePixelBudget } from "./image-budget.ts";
3
+ import { ImagePreparationError } from "./image-preparation-error.ts";
4
+
5
+ /** The longest side a picture may declare before `squarePng` refuses to decode it. */
6
+ export const SQUARE_PNG_MAX_SIDE = 4096;
7
+ /** The side `squarePng` scales to when it is not told one. */
8
+ const DEFAULT_SIDE = 512;
9
+ /** Below this side `squarePng` gives up fitting `maxBytes`. */
10
+ const SMALLEST_SIDE = 16;
11
+ /** Frames of an animation add up, so the whole stays within the budget `prepareImageBytes` allows. */
12
+ const DECODED_PIXEL_BUDGET = 8000 * 8000;
13
+
14
+ export interface SquarePngOptions {
15
+ /** The most pixels per side of the result; default 512. A smaller picture is not enlarged. */
16
+ size?: number;
17
+ /** The most bytes of the result; the side steps down by a quarter until it fits. Default none. */
18
+ maxBytes?: number;
19
+ /**
20
+ * The longest side the picture may declare before it is refused, default 4096
21
+ * (`SQUARE_PNG_MAX_SIDE`). A picture within the 64 MP decoded-pixel budget but over it, such as a
22
+ * camera photo, is admitted by a larger value.
23
+ */
24
+ maxSide?: number;
25
+ }
26
+
27
+ const wholeNumber = (name: string, value: number, max: number) => {
28
+ if (!Number.isInteger(value) || value < 1 || value > max)
29
+ throw new RangeError(
30
+ `squarePng: ${name} must be a whole number from 1 to ${max}`,
31
+ );
32
+ };
33
+
34
+ /**
35
+ * A centered square crop of a PNG, JPEG or GIF, scaled down to at most `size` pixels per
36
+ * side, as PNG. The header is read before anything is decoded: a declared width or height over
37
+ * 4096 (`SQUARE_PNG_MAX_SIDE`, or `maxSide`) rejects with `ImagePreparationError` (`pixel-limit`),
38
+ * and bytes of none of the three types (WebP included) with `invalid-image`, as does a picture
39
+ * that cannot be decoded. With `maxBytes` the side steps down (by a quarter
40
+ * each time) until the PNG is no larger; a picture that does not fit even at 16 pixels throws a
41
+ * `RangeError`.
42
+ */
43
+ export async function squarePng(
44
+ bytes: Uint8Array,
45
+ options: SquarePngOptions = {},
46
+ ): Promise<Buffer> {
47
+ const side = options.size ?? DEFAULT_SIDE;
48
+ wholeNumber("size", side, SQUARE_PNG_MAX_SIDE);
49
+ if (options.maxBytes !== undefined)
50
+ wholeNumber("maxBytes", options.maxBytes, Number.MAX_SAFE_INTEGER);
51
+ if (options.maxSide !== undefined)
52
+ wholeNumber("maxSide", options.maxSide, Number.MAX_SAFE_INTEGER);
53
+ const source = Buffer.from(bytes);
54
+ // The canvas has no WebP decoder, so a WebP is refused here rather than failing in the decode.
55
+ if (
56
+ source.toString("ascii", 0, 4) === "RIFF" &&
57
+ source.toString("ascii", 8, 12) === "WEBP"
58
+ )
59
+ throw new ImagePreparationError("invalid-image");
60
+ assertImagePixelBudget(
61
+ source,
62
+ DECODED_PIXEL_BUDGET,
63
+ options.maxSide ?? SQUARE_PNG_MAX_SIDE,
64
+ );
65
+ const image = await loadImage(source).catch(() => {
66
+ throw new ImagePreparationError("invalid-image");
67
+ });
68
+ const crop = Math.min(image.width, image.height);
69
+ let size = Math.min(side, crop);
70
+ for (;;) {
71
+ const canvas = createCanvas(size, size);
72
+ canvas
73
+ .getContext("2d")
74
+ .drawImage(
75
+ image,
76
+ (image.width - crop) / 2,
77
+ (image.height - crop) / 2,
78
+ crop,
79
+ crop,
80
+ 0,
81
+ 0,
82
+ size,
83
+ size,
84
+ );
85
+ const png = canvas.toBuffer("image/png");
86
+ if (options.maxBytes === undefined || png.byteLength <= options.maxBytes)
87
+ return png;
88
+ size = Math.floor(size * 0.75);
89
+ if (size < SMALLEST_SIDE)
90
+ throw new RangeError(
91
+ `squarePng: the picture does not fit ${options.maxBytes} bytes even at ${SMALLEST_SIDE} pixels`,
92
+ );
93
+ }
94
+ }
@@ -4,8 +4,11 @@ import type { Logger } from "../log.ts";
4
4
  import { fetchAttachments } from "./attachment-fetcher.ts";
5
5
  import { modelImagesOf } from "./model-images.ts";
6
6
 
7
- /** The most milliseconds all of a message's downloads take together by default. */
8
- const DEADLINE_MS = 20_000;
7
+ /**
8
+ * The most milliseconds all of a message's downloads take together by default. A download that
9
+ * stalls ends at `DEFAULT_IDLE_MS` without bytes, so the deadline only has to cover slow ones.
10
+ */
11
+ export const DEFAULT_DEADLINE_MS = 60_000;
9
12
 
10
13
  /** Saves the message's and its reference's files, and turns up to four images into model images. */
11
14
  export async function collectAttachments(
@@ -14,17 +17,22 @@ export async function collectAttachments(
14
17
  options: {
15
18
  logger: Logger;
16
19
  fetchImpl?: typeof fetch;
17
- /** One deadline for the message's and its reference's files together; default 20 s. */
20
+ /** One deadline for the message's and its reference's files together; default 60 s. */
18
21
  deadlineMs?: number;
22
+ /** How long one file may send no byte before it is given up and the next one tried; default 10 s. */
23
+ idleMs?: number;
19
24
  /** Aborts the downloads, as the deadline does. */
20
25
  signal?: AbortSignal;
21
26
  },
22
27
  ): Promise<TurnAttachments> {
23
28
  const { fetchImpl, logger } = options;
24
- const deadline = AbortSignal.timeout(options.deadlineMs ?? DEADLINE_MS);
29
+ const deadline = AbortSignal.timeout(
30
+ options.deadlineMs ?? DEFAULT_DEADLINE_MS,
31
+ );
25
32
  const signal = options.signal
26
33
  ? AbortSignal.any([deadline, options.signal])
27
34
  : deadline;
35
+ const idle = options.idleMs === undefined ? {} : { idleMs: options.idleMs };
28
36
  const own = await fetchAttachments(
29
37
  {
30
38
  refs: message.attachments,
@@ -32,6 +40,7 @@ export async function collectAttachments(
32
40
  prefix: message.messageId,
33
41
  fromReference: false,
34
42
  signal,
43
+ ...idle,
35
44
  },
36
45
  fetchImpl,
37
46
  );
@@ -42,6 +51,7 @@ export async function collectAttachments(
42
51
  prefix: `${message.messageId}-ref`,
43
52
  fromReference: true,
44
53
  signal,
54
+ ...idle,
45
55
  },
46
56
  fetchImpl,
47
57
  );
@@ -70,7 +80,7 @@ export function attachmentsOf(
70
80
  dir: string,
71
81
  logger: Logger,
72
82
  fetchImpl?: typeof fetch,
73
- limits: { deadlineMs?: number; signal?: AbortSignal } = {},
83
+ limits: { deadlineMs?: number; idleMs?: number; signal?: AbortSignal } = {},
74
84
  ): Promise<TurnAttachments> {
75
85
  return collectAttachments(message, dir, {
76
86
  logger,
@@ -69,6 +69,12 @@ export interface AgentServerOptions {
69
69
  * `"owners"`, which leaves a speaker below the owner tier unanswered.
70
70
  */
71
71
  speakers?: "everyone" | "owners";
72
+ /**
73
+ * Limits of the downloads of a message's files (`discord.attachmentDownload`): `deadlineMs`
74
+ * for all of a message's files together (default 60 s), `idleMs` for one file to send no byte
75
+ * (default 10 s).
76
+ */
77
+ attachmentDownload?: { deadlineMs?: number; idleMs?: number };
72
78
  /** The prompt every agent starts with, and the one for speakers other than the owner. */
73
79
  prompts: { shared: string; guest?: string };
74
80
  /** The listener the avatar pictures are served on. */
@@ -77,6 +83,8 @@ export interface AgentServerOptions {
77
83
  avatarUrl: string;
78
84
  /** The assistant's neutral avatar: the style reference and the picture of an agent without one. */
79
85
  avatarReference: string;
86
+ /** The picture used instead when `avatarReference` cannot be read as one; the failure is logged. */
87
+ avatarNeutral?: string;
80
88
  /** Reports the process's own errors to an agent; without one nothing is reported. */
81
89
  errorReporter?: ErrorReporter;
82
90
  }
@@ -155,6 +163,16 @@ export function agentServerPlugin(
155
163
  dir: join(options.dataDir, "avatars"),
156
164
  publicUrl: options.avatarUrl,
157
165
  referencePath: options.avatarReference,
166
+ ...(options.avatarNeutral
167
+ ? {
168
+ neutralPath: options.avatarNeutral,
169
+ onReferenceFailed: (code: string) =>
170
+ logger.warn(
171
+ { code },
172
+ "the configured avatar could not be used; the neutral avatar is used instead",
173
+ ),
174
+ }
175
+ : {}),
158
176
  // Without an image provider the studio makes each agent a picture from its name.
159
177
  ...(providers.filled.has("images") ? { draw: providers.images } : {}),
160
178
  });
@@ -293,6 +311,9 @@ export function agentServerPlugin(
293
311
  runtime: running,
294
312
  surface: context.surfaces,
295
313
  attachmentDir,
314
+ ...(options.attachmentDownload
315
+ ? { attachmentDownload: options.attachmentDownload }
316
+ : {}),
296
317
  logger,
297
318
  channelContext: (message) => discord.channelContext(message),
298
319
  }),
@@ -71,6 +71,14 @@ export interface DiscordConfig {
71
71
  * remove them; they age out.
72
72
  */
73
73
  cardCopies?: { keepDays?: number; keep?: number } | false;
74
+ /**
75
+ * How the files of an owner's or an agent's channel message are downloaded: `deadlineMs`
76
+ * (default 60,000) is the most time all of a message's files, and those of the message it
77
+ * replies to, may take together; `idleMs` (default 10,000) is how long one file may send no
78
+ * byte before it is given up. A file still to come at the deadline, or one that went quiet, is
79
+ * reported as `download failed` and the turn goes on without it.
80
+ */
81
+ attachmentDownload?: { deadlineMs?: number; idleMs?: number };
74
82
  }
75
83
 
76
84
  /** The Discord settings with every default filled in. */
@@ -89,6 +97,8 @@ export interface ResolvedDiscord {
89
97
  cardGraceSeconds: number;
90
98
  /** As configured: `{}` when left out, so the defaults apply, or `false`. */
91
99
  cardCopies: NonNullable<DiscordConfig["cardCopies"]> | false;
100
+ /** As configured: left out unless set, so the defaults apply. */
101
+ attachmentDownload?: NonNullable<DiscordConfig["attachmentDownload"]>;
92
102
  }
93
103
 
94
104
  /** The configured Discord settings, defaults filled in; `rootCommand` is the host's slug. */
@@ -117,6 +127,9 @@ export function resolvedDiscord(
117
127
  : { freshMarker: discord.freshMarker }),
118
128
  cardGraceSeconds: discord.cardGraceSeconds ?? 120,
119
129
  cardCopies: discord.cardCopies ?? {},
130
+ ...(discord.attachmentDownload === undefined
131
+ ? {}
132
+ : { attachmentDownload: discord.attachmentDownload }),
120
133
  };
121
134
  }
122
135
 
@@ -163,6 +176,12 @@ export const discordShape = shape({
163
176
  }),
164
177
  ),
165
178
  ),
179
+ attachmentDownload: optional(
180
+ shape({
181
+ deadlineMs: optional(integer(1_000, 600_000)),
182
+ idleMs: optional(integer(1_000, 600_000)),
183
+ }),
184
+ ),
166
185
  });
167
186
 
168
187
  /** The adapters `adapters` takes, by the name their factory gives them. */
@@ -146,7 +146,11 @@ export interface RoundtableConfig {
146
146
  socketPath?: string;
147
147
  socketMode?: number;
148
148
  };
149
- /** A picture of your own for agents without one, and the style reference for drawing them; default a plain bot. */
149
+ /**
150
+ * A picture of your own for agents without one, and the style reference for drawing them; default
151
+ * a plain bot. A PNG, JPEG or GIF of any side within the 64 MP decoded-pixel budget (an ordinary
152
+ * camera photo); one that cannot be read as a picture is logged and the plain bot is used.
153
+ */
150
154
  avatar?: string;
151
155
  /** The agents' shared working directory; default `<dataDir>/work`. */
152
156
  workDir?: string;
@@ -202,10 +202,14 @@ function discordAssembly(
202
202
  prompts: { shared: sharedPrompt, guest },
203
203
  ...(discord.agentMemory ? { memory: discord.agentMemory } : {}),
204
204
  ...(discord.agentSpeakers ? { speakers: discord.agentSpeakers } : {}),
205
+ ...(discord.attachmentDownload
206
+ ? { attachmentDownload: discord.attachmentDownload }
207
+ : {}),
205
208
  avatarListener: "public",
206
209
  // resolveConfig refuses Discord without a public address.
207
210
  avatarUrl: config.http?.publicUrl ?? "",
208
211
  avatarReference: config.avatar ?? join(ASSETS, "neutral.png"),
212
+ avatarNeutral: join(ASSETS, "neutral.png"),
209
213
  // An ops agent's reports are the agent server's to deliver.
210
214
  ...(errorReporter && "agent" in errorReporter.destination
211
215
  ? { errorReporter }
@@ -21,7 +21,10 @@ export type {
21
21
  AgentPanelOptions,
22
22
  } from "../core/discord/agent-panel.ts";
23
23
  export { agentPanel } from "../core/discord/agent-panel.ts";
24
- export { CHANNEL_CONTEXT_DEFAULTS } from "../core/discord/channel-context.ts";
24
+ export {
25
+ CHANNEL_CONTEXT_DEFAULTS,
26
+ formatChannelContext,
27
+ } from "../core/discord/channel-context.ts";
25
28
  export type { ManagedChannel } from "../core/discord/channel-executor.ts";
26
29
  export {
27
30
  fetchManagedChannel,
@@ -9,6 +9,11 @@ export {
9
9
  export { prepareImageBytes } from "../core/attachments/image-prep.ts";
10
10
  export { ImagePreparationError } from "../core/attachments/image-preparation-error.ts";
11
11
  export { withAttachmentsBlock } from "../core/attachments/prompt-block.ts";
12
+ export {
13
+ SQUARE_PNG_MAX_SIDE,
14
+ type SquarePngOptions,
15
+ squarePng,
16
+ } from "../core/attachments/square-png.ts";
12
17
  export { attachmentsOf } from "../core/attachments/turn-attachments.ts";
13
18
  export type { ChannelQueue } from "../core/routing/channel-queue.ts";
14
19
  export { channelQueue } from "../core/routing/channel-queue.ts";
package/src/kit/index.ts CHANGED
@@ -10,7 +10,10 @@ export {
10
10
  outcome,
11
11
  ownerAttachmentDir,
12
12
  prepareImageBytes,
13
+ SQUARE_PNG_MAX_SIDE,
14
+ type SquarePngOptions,
13
15
  settleTurn,
16
+ squarePng,
14
17
  withAttachmentsBlock,
15
18
  withReference,
16
19
  } from "./channels.ts";