pi-roundtable 0.12.0 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.md +1 -0
- package/README.zh-TW.md +1 -0
- package/docs/plugins.md +10 -1
- package/package.json +1 -1
- package/src/core/agents/agent-claim.ts +9 -1
- package/src/core/agents/avatar-studio.ts +40 -33
- package/src/core/attachments/attachment-fetcher.ts +100 -5
- package/src/core/attachments/image-budget.ts +14 -2
- package/src/core/attachments/image-preparation-error.ts +10 -6
- package/src/core/attachments/square-png.ts +94 -0
- package/src/core/attachments/turn-attachments.ts +29 -1
- package/src/core/builtin/agent-server.ts +21 -0
- package/src/core/builtin/discord.ts +5 -1
- package/src/core/config/adapters.ts +19 -0
- package/src/core/config/config.ts +5 -1
- package/src/core/define-roundtable.ts +4 -0
- package/src/core/discord/card-copies.ts +35 -4
- package/src/core/discord/owner-cards.ts +2 -0
- package/src/discord/index.ts +4 -1
- package/src/kit/channels.ts +5 -0
- package/src/kit/index.ts +3 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,30 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [0.13.0] - 2026-10-13
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- `formatChannelContext` (`formatChannelContext(messages)`) is exported from `pi-roundtable/discord`, the function that writes `ChannelContext.text` (the `<channel-context>` block with each author in attributes and every `<` neutralised), so a host can test what the model sees without a deep import. It is the function the reader already used; nothing about the block changes.
|
|
13
|
+
- `squarePng`, `SquarePngOptions` and `SQUARE_PNG_MAX_SIDE` (4096) are exported from `pi-roundtable/kit`. `squarePng(bytes, { size?, maxBytes?, maxSide? })` is the centered square crop, scaled down to at most `size` pixels per side (default 512) as PNG, that the agent avatars are made with. The header is read before decoding: a declared width or height over 4096 (or `maxSide`) rejects with `ImagePreparationError` (`pixel-limit`, with a message that names the side limit), bytes of no PNG, JPEG or GIF with `invalid-image`; a WebP is refused the same way, since the canvas cannot decode it, and so is any picture that fails to decode. With `maxBytes` the side steps down by a quarter until the PNG fits, and a picture that does not fit at 16 pixels throws a `RangeError`.
|
|
14
|
+
- `discord.attachmentDownload: { deadlineMs?, idleMs? }` (each a whole number of milliseconds from 1,000 to 600,000) sets the limits of the downloads of the files of an owner or agent channel message on the agent server: `deadlineMs` is the overall deadline for the message and its reference together (default 60,000) and `idleMs` the time one file may send no byte before it is given up (default 10,000). `attachmentsOf` takes the same two as `limits.deadlineMs` and `limits.idleMs`, and `AgentServerOptions.attachmentDownload` passes them on for a host built without `defineRoundtable`. A file that goes quiet is recorded as `download failed: idle_timeout` and the next file is tried at once.
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
|
|
18
|
+
- The avatar studio refuses a drawn picture that declares a side over 4096 (`ImagePreparationError`, `pixel-limit`), where it used to decode it. The configured `avatar` is not held to that: a photo of any side within the 64 MP decoded-pixel budget is accepted, and one that cannot be read as a picture is logged (`the configured avatar could not be used`, with its error code) and replaced by the plain bot, where it would stop the host from starting.
|
|
19
|
+
- The default overall deadline for downloading the files of a message and its reference rises from 20 to 60 seconds, since a stalled download is now caught by the 10 second idle timeout instead of by the deadline: a body that sends no byte for 10 seconds (the wait for the headers counts, also for a fetch that ignores its abort signal) ends that file alone, where it used to hold every file still to come until the deadline. A slow download that keeps sending bytes now has up to 60 seconds. A host that wants the old bound sets `discord.attachmentDownload.deadlineMs: 20000`.
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
|
|
23
|
+
- An attachment's failure reason never quotes a file system path. A download that fails for any reason other than the ones with their own wording (a size over 25 MB, an HTTP status, the deadline) is recorded as `download failed: <code>`, where the code is the error's `code` (such as `ENOTDIR`, for a directory that could not be made, or `ECONNRESET`) or its `name` (such as `TimeoutError`), and `error` when it has neither; it used to be the error's own text, and the text of a failed `mkdir` quotes the directory. The reason is shown to the model in the turn's attachment block and to the channel, so a path of the host no longer reaches either.
|
|
24
|
+
|
|
25
|
+
## [0.12.1] - 2026-10-13
|
|
26
|
+
|
|
27
|
+
### Fixed
|
|
28
|
+
|
|
29
|
+
- Expired card copy texts leave `card-copies.json`, not only memory. They are dropped from the file when the host starts, when a copy button finds one past `keepDays`, and on an hourly sweep (an unref'd timer that the Discord plugin starts and stops with the host), without waiting for a new card.
|
|
30
|
+
- Downloading the files of an owner or agent channel message has an overall 20 second deadline for the message and its reference together, and takes an `AbortSignal`: `attachmentsOf(message, dir, logger, fetchImpl?, limits?)` gains `limits: { deadlineMs?: number; signal?: AbortSignal }`. Files still to come when it passes, a body that stalls, and a directory that cannot be created are recorded as `download failed` failures instead of holding the turn or throwing.
|
|
31
|
+
|
|
8
32
|
## [0.12.0] - 2026-10-13
|
|
9
33
|
|
|
10
34
|
### Added
|
package/README.md
CHANGED
|
@@ -197,6 +197,7 @@ When a conversation is started over in a server channel or thread, the assistant
|
|
|
197
197
|
| `false` | | Post no divider |
|
|
198
198
|
|
|
199
199
|
An approval card or an `ask_user` question holds its turn for `discord.cardGraceSeconds` (default `120`).
|
|
200
|
+
The files of a message in an agent's channel download within `discord.attachmentDownload.deadlineMs` (default 60,000) for the message and the one it replies to together, and a file that sends no byte for `discord.attachmentDownload.idleMs` (default 10,000) is given up; the turn goes on without a file it could not get.
|
|
200
201
|
The copy button of an approval card keeps its text in `card-copies.json` under the data directory, the newest 100 for 30 days; `discord.cardCopies` (`{ keepDays?, keep? }`) changes that, and `false` keeps them in memory only.
|
|
201
202
|
Answered within it, the turn goes on with the answer; otherwise the turn ends saying it is waiting, the card stays open with no deadline, and answering it later starts a new turn in that conversation with the answer, or with the result of the approved action.
|
|
202
203
|
A coding worker's cards keep the worker waiting instead.
|
package/README.zh-TW.md
CHANGED
|
@@ -185,6 +185,7 @@ export default {
|
|
|
185
185
|
| `attachments` | `{ images: 0, maxBytes: 10 MiB }` | `images`:替這一輪抓取保留訊息中的幾張圖片,最多 4 張 |
|
|
186
186
|
|
|
187
187
|
核准卡和 `ask_user` 問題會讓這一輪等 `discord.cardGraceSeconds` 秒(預設 `120`)。
|
|
188
|
+
agent 頻道裡訊息的附件,整則訊息(連同它回覆的那則)要在 `discord.attachmentDownload.deadlineMs`(預設 60,000)毫秒內下載完;單一檔案連續 `discord.attachmentDownload.idleMs`(預設 10,000)毫秒沒收到任何位元組就放棄,這一輪會略過收不到的檔案繼續。
|
|
188
189
|
核准卡「複製」按鈕的文字存在資料目錄的 `card-copies.json`,預設留最新 100 張、30 天;`discord.cardCopies`(`{ keepDays?, keep? }`)可以調整,設成 `false` 則只留在記憶體。
|
|
189
190
|
在這段時間內回答,這一輪就帶著答案繼續;沒回答的話,這一輪會說它在等你,然後結束,卡片繼續開著、沒有期限,之後再回答就會在同一個對話開新的一輪,帶上你的答案,或核准後那個動作的結果。
|
|
190
191
|
coding worker 的卡片則會讓 worker 一直等下去。
|
package/docs/plugins.md
CHANGED
|
@@ -60,6 +60,9 @@ Use the kit's building blocks for a plugin that runs Pi itself, such as a coding
|
|
|
60
60
|
`ImagePreparationError.reason` is `byte-limit`, `pixel-limit`, or `invalid-image`; hosts can localize refusals without parsing native decoder errors.
|
|
61
61
|
Ordinary 48 MP images and thin 9000×1 images remain admitted.
|
|
62
62
|
This helper is not a downloader or filesystem validator.
|
|
63
|
+
- Square pictures: `squarePng(bytes, { size?, maxBytes?, maxSide? })` crops a PNG, JPEG or GIF to its centered square and scales it down to at most `size` pixels per side (default 512; a smaller picture is not enlarged), as PNG: the helper the agent avatars are made with, for a plugin that needs a profile picture or an icon.
|
|
64
|
+
The header is read first, and nothing is decoded for a picture that declares a width or height over `SQUARE_PNG_MAX_SIDE` (4096), which rejects with `ImagePreparationError` (`pixel-limit`, the message naming the side limit; `maxSide` sets another limit, and the 64 MP decoded-pixel budget always applies), or for bytes of no such type, a WebP (the canvas cannot decode it) or a picture that fails to decode (`invalid-image`).
|
|
65
|
+
With `maxBytes` the side steps down by a quarter at a time until the PNG is no larger; a picture that does not fit even at 16 pixels throws a `RangeError`, as does a `size` or `maxBytes` that is not a whole number in range. `SquarePngOptions` types the options.
|
|
63
66
|
- Work: `promptSlot` (how a run asks the owner while it works), `workTimeout` (a time limit that does not count the time spent waiting on the owner), `runWorkerTask`, `archiveSessions`, and `approvalCard`, `approvalDetails` and `canonicalJson` for the cards of held actions.
|
|
64
67
|
`InterimPoster(posts, { logger, channel, primaryChars?, editMs? })` posts a run's [interim text](#interim-text-what-a-turn-writes-before-its-final-answer) to an `InterimPosts`, such as a dispatch thread's `interim`: feed it `messageEnd(message)` and `toolStart(name)` from the session's events, bind `() => poster.flush()` as the slot's `beforeCard`, and `await poster.flush()` before the final report.
|
|
65
68
|
- Diagnostics: `scrubDiagnostic(text, max = 600)` masks credentials (URL userinfo, token shapes, secret-named assignments and JSON fields, `Authorization`/`Cookie`/`x-api-key` headers, JWTs, PEM blocks), turns control characters other than tab and newline into spaces, and cuts the result at `max` characters.
|
|
@@ -553,6 +556,7 @@ A key the contract does not have stops the start and names the closest one.
|
|
|
553
556
|
### `context.attachments`: files from outside a turn
|
|
554
557
|
|
|
555
558
|
A surface that takes a file as it arrives, such as Discord, hands the core a link, and the core downloads it into the conversation's attachment directory.
|
|
559
|
+
The downloads of one message and of the message it replies to share a 60 second deadline, and a file that sends no byte for 10 seconds (the wait for its headers counts) is given up at once, so a stalled download costs 10 seconds, not the whole deadline. A file still to come when the deadline passes, a file that went quiet (`download failed: idle_timeout`), or one whose directory cannot be made, is reported as `download failed` and the turn goes on without it. `attachmentsOf(message, dir, logger, fetch?, { deadlineMs?, idleMs?, signal? })` takes both limits; on the agent server they are `discord.attachmentDownload: { deadlineMs?, idleMs? }` (each 1,000 to 600,000 ms; defaults 60,000 and 10,000), for a host on a slow link or with large files. A failure reason carries a code only (`download failed: ENOTDIR`, `download failed: TimeoutError`, `download failed: error`), never the error's text, which could quote a path of the host.
|
|
556
560
|
A plugin whose clients upload a file themselves, over HTTP for instance, has the file before any turn exists.
|
|
557
561
|
`context.attachments` keeps such a file for the conversation until a turn uses it:
|
|
558
562
|
|
|
@@ -2316,6 +2320,7 @@ A surface a plugin contributes may implement the optional `ChatSurface.markFresh
|
|
|
2316
2320
|
The block gives each author as a name, their Discord id and a role (`owner`, `bot` or `member`) in separate attributes, and says the messages were not addressed to the assistant. A name loses the characters `"`, `'`, `=`, `<`, `>`, `(`, `)` and `,`, so it cannot imitate another author, an attribute or a role, and every `<` (also the full-width `<`) in a message's text is replaced by `‹`, so a message can neither close the block nor open a message of its own.
|
|
2317
2321
|
`ChannelContext` is `{ messages, text, images? }`: `messages` are `ChannelContextMessage`s (`id`, `authorId`, `authorName`, `bot`, `owner`, `text`, `at`, `attachments`), oldest first, `text` is the block, and `images` the fetched images (see above).
|
|
2318
2322
|
`ChannelContextOptions` holds the options above, and `CHANNEL_CONTEXT_DEFAULTS` their defaults.
|
|
2323
|
+
`formatChannelContext(messages)` is the function that writes `ChannelContext.text`, exported so a host can test what the model sees without reaching into the core: it takes `ChannelContextMessage`s (only `authorId`, `authorName`, `bot`, `owner` and `text` are read) and returns the block, headed and delimited by `<channel-context>`. Each message is a `<message from="…" id="…" role="owner|member|bot">` line whose author is in the attributes, never in the text, and every `<` of a name or text (plain or full-width) is replaced by `‹`, so no message can pass as markup. It does not fetch, cut or select anything, and it is the same function the reader calls, so a block it returns is the block a turn carries.
|
|
2319
2324
|
In a test, `fakeDiscord({ channelContext })` answers `DISCORD.channelContext` as the test says; by default with no context.
|
|
2320
2325
|
|
|
2321
2326
|
<!-- example: examples/channel-context.ts -->
|
|
@@ -2563,7 +2568,7 @@ A surface or a `Prompts` of your own may ignore it: nothing requires reading it.
|
|
|
2563
2568
|
|
|
2564
2569
|
`details.copy` (`{ label?, text }`) is a text the approval offers to copy, such as a command or a message the call would send; whoever builds `details` for a `confirm` of their own sets it.
|
|
2565
2570
|
It is informational and never an answer: using it does not approve, decline or otherwise settle the prompt, and a surface that cannot show it ignores it, as the web chat's own client does (the frame carries it as `approval.copy`, left out when it would not fit beside the call's `input` in the frame's 256 KiB: the input is never cut for it).
|
|
2566
|
-
On Discord the card gets a third button (`label`, default "Copy", or its Traditional Chinese equivalent on a host in that language) that any member who can see the card may press, any number of times, also after the card has ended; it replies ephemerally with the text in a code block, or as a `copy.txt` file when the text does not fit a message (2,000 characters) or holds a code fence. The button's custom id holds the card's id, never the text: the text is kept under that id in `card-copies.json` in the host's `dataDir`, in plain text and readable by the host's user only, so the button works after a restart even though the card itself no longer answers; a card whose text is gone says so. `discord.cardCopies` (`{ keepDays?, keep? }`, default the newest 100 cards for 30 days) sets how long and how many; `false` keeps the texts in memory only, with the same limits, so they are gone at a restart. The file is written in the background, one write at a time, so a card is not held up by it. The attachments' `retention` and discarding a conversation do not remove the texts: they age out, or set `cardCopies` to `false` on a host that wants none on disk: the host then deletes an existing `card-copies.json` when it starts. A text over 20,000 characters gets no button, and a warning is logged.
|
|
2571
|
+
On Discord the card gets a third button (`label`, default "Copy", or its Traditional Chinese equivalent on a host in that language) that any member who can see the card may press, any number of times, also after the card has ended; it replies ephemerally with the text in a code block, or as a `copy.txt` file when the text does not fit a message (2,000 characters) or holds a code fence. The button's custom id holds the card's id, never the text: the text is kept under that id in `card-copies.json` in the host's `dataDir`, in plain text and readable by the host's user only, so the button works after a restart even though the card itself no longer answers; a card whose text is gone says so. `discord.cardCopies` (`{ keepDays?, keep? }`, default the newest 100 cards for 30 days) sets how long and how many; `false` keeps the texts in memory only, with the same limits, so they are gone at a restart. The file is written in the background, one write at a time, so a card is not held up by it. A text past `keepDays` leaves the file too, not only memory: when the host starts or a button finds one expired, and on an hourly sweep that does not keep the process alive. The attachments' `retention` and discarding a conversation do not remove the texts: they age out, or set `cardCopies` to `false` on a host that wants none on disk: the host then deletes an existing `card-copies.json` when it starts. A text over 20,000 characters gets no button, and a warning is logged.
|
|
2567
2572
|
Because anyone who can see the card can copy the text, it holds nothing they may not read.
|
|
2568
2573
|
`confirm` resolves `approved`, `declined`, `expired` (a card was shown and went unanswered), `unavailable` (no card could be shown: no one may approve the call, the speaker is below its tier, or the card could not be posted), `cancelled` (the turn stopped), or `pending` (with a `wait`). Both `expired` and `unavailable` leave the call held for the owner's next message; the turn's progress reports them as `refused: "expired"` and `refused: "held"`.
|
|
2569
2574
|
|
|
@@ -3726,6 +3731,9 @@ Import from the entries listed below; source area files are internal.
|
|
|
3726
3731
|
| `withReplyFiles` | `pi-roundtable` | value |
|
|
3727
3732
|
| `prepareImageBytes` | `pi-roundtable/kit` | value |
|
|
3728
3733
|
| `ImagePreparationError` | `pi-roundtable/kit` | value |
|
|
3734
|
+
| `squarePng` | `pi-roundtable/kit` | value |
|
|
3735
|
+
| `SQUARE_PNG_MAX_SIDE` | `pi-roundtable/kit` | value |
|
|
3736
|
+
| `SquarePngOptions` | `pi-roundtable/kit` | type |
|
|
3729
3737
|
| `scrubDiagnostic` | `pi-roundtable/kit` | value |
|
|
3730
3738
|
| `channelKey` | `pi-roundtable` | value |
|
|
3731
3739
|
| `definePlugin` | `pi-roundtable` | value |
|
|
@@ -3922,6 +3930,7 @@ Import from the entries listed below; source area files are internal.
|
|
|
3922
3930
|
| `ChannelContextAttachmentOptions` | `pi-roundtable/discord` | type |
|
|
3923
3931
|
| `ChannelContextMessage` | `pi-roundtable/discord` | type |
|
|
3924
3932
|
| `ChannelContextOptions` | `pi-roundtable/discord` | type |
|
|
3933
|
+
| `formatChannelContext` | `pi-roundtable/discord` | value |
|
|
3925
3934
|
| `ChannelExecutor` | `pi-roundtable/discord` | type |
|
|
3926
3935
|
| `ChannelInfo` | `pi-roundtable/discord` | type |
|
|
3927
3936
|
| `ChannelOperation` | `pi-roundtable/discord` | type |
|
package/package.json
CHANGED
|
@@ -60,6 +60,8 @@ export interface AgentClaimOptions {
|
|
|
60
60
|
surface: Pick<SurfacePort, "react" | "unreact">;
|
|
61
61
|
/** Where a channel's attachments are saved. */
|
|
62
62
|
attachmentDir: (channel: ChannelKey) => string;
|
|
63
|
+
/** The downloads' limits: all of a message's files together (default 60 s), and one file without a byte (default 10 s). */
|
|
64
|
+
attachmentDownload?: { deadlineMs?: number; idleMs?: number };
|
|
63
65
|
logger: Logger;
|
|
64
66
|
/** Replaceable in tests. */
|
|
65
67
|
fetchImpl?: typeof fetch;
|
|
@@ -80,7 +82,13 @@ export interface AgentClaimOptions {
|
|
|
80
82
|
export function agentClaim(options: AgentClaimOptions): ChannelClaim {
|
|
81
83
|
const { team, runtime, surface, attachmentDir, logger, fetchImpl } = options;
|
|
82
84
|
const attachments = (message: InboundMessage) =>
|
|
83
|
-
attachmentsOf(
|
|
85
|
+
attachmentsOf(
|
|
86
|
+
message,
|
|
87
|
+
attachmentDir(message.channel),
|
|
88
|
+
logger,
|
|
89
|
+
fetchImpl,
|
|
90
|
+
options.attachmentDownload,
|
|
91
|
+
);
|
|
84
92
|
/** The channel context of a message in an agent's channel; none, logged, when it cannot be read. */
|
|
85
93
|
const around = async (
|
|
86
94
|
message: InboundMessage,
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { createHash } from "node:crypto";
|
|
2
2
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
3
3
|
import { join } from "node:path";
|
|
4
|
-
import {
|
|
4
|
+
import { squarePng } from "../attachments/square-png.ts";
|
|
5
5
|
import type { ImageDrawer } from "../contract/providers.ts";
|
|
6
6
|
import { ProviderError } from "../errors.ts";
|
|
7
7
|
import type { HttpRoute } from "../http/listeners.ts";
|
|
@@ -10,6 +10,8 @@ import { fallbackAvatar } from "./fallback-avatar.ts";
|
|
|
10
10
|
|
|
11
11
|
const AVATAR_SIDE = 512;
|
|
12
12
|
const REFERENCE_SIDE = 512;
|
|
13
|
+
/** The configured reference is the host's own photo, so only the decoded-pixel budget limits it. */
|
|
14
|
+
const REFERENCE_MAX_SIDE = Number.MAX_SAFE_INTEGER;
|
|
13
15
|
const HASH_PATH = /^\/avatars\/([0-9a-f]{64})\.png$/;
|
|
14
16
|
|
|
15
17
|
export type { ImageDrawer };
|
|
@@ -21,32 +23,14 @@ export interface AvatarStudioOptions {
|
|
|
21
23
|
publicUrl: string;
|
|
22
24
|
/** The assistant's neutral avatar: the style reference and the picture of an agent without one. */
|
|
23
25
|
referencePath: string;
|
|
26
|
+
/** The neutral avatar shipped with the package; the reference is replaced by it when the reference cannot be read as a picture. */
|
|
27
|
+
neutralPath?: string;
|
|
28
|
+
/** Told the error code of a reference that was replaced by `neutralPath`. */
|
|
29
|
+
onReferenceFailed?: (code: string) => void;
|
|
24
30
|
/** Draws the pictures. Without one, the studio only makes each agent a picture from its name. */
|
|
25
31
|
draw?: ImageDrawer;
|
|
26
32
|
}
|
|
27
33
|
|
|
28
|
-
/** A centered square crop scaled to at most `side` pixels, as PNG. */
|
|
29
|
-
async function squarePng(bytes: Uint8Array, side: number): Promise<Buffer> {
|
|
30
|
-
const image = await loadImage(Buffer.from(bytes));
|
|
31
|
-
const crop = Math.min(image.width, image.height);
|
|
32
|
-
const size = Math.min(side, crop);
|
|
33
|
-
const canvas = createCanvas(size, size);
|
|
34
|
-
canvas
|
|
35
|
-
.getContext("2d")
|
|
36
|
-
.drawImage(
|
|
37
|
-
image,
|
|
38
|
-
(image.width - crop) / 2,
|
|
39
|
-
(image.height - crop) / 2,
|
|
40
|
-
crop,
|
|
41
|
-
crop,
|
|
42
|
-
0,
|
|
43
|
-
0,
|
|
44
|
-
size,
|
|
45
|
-
size,
|
|
46
|
-
);
|
|
47
|
-
return canvas.toBuffer("image/png");
|
|
48
|
-
}
|
|
49
|
-
|
|
50
34
|
/**
|
|
51
35
|
* Agent avatars: an image provider draws them in the assistant's style, and without one each agent
|
|
52
36
|
* gets a picture generated from its display name. They are kept and served by content hash, so a
|
|
@@ -55,17 +39,37 @@ async function squarePng(bytes: Uint8Array, side: number): Promise<Buffer> {
|
|
|
55
39
|
export class FileAvatarStudio implements AvatarStudio {
|
|
56
40
|
readonly #options: AvatarStudioOptions;
|
|
57
41
|
#defaultHash: string | undefined;
|
|
42
|
+
#referencePath: string;
|
|
58
43
|
|
|
59
44
|
constructor(options: AvatarStudioOptions) {
|
|
60
45
|
this.#options = options;
|
|
46
|
+
this.#referencePath = options.referencePath;
|
|
61
47
|
mkdirSync(options.dir, { recursive: true });
|
|
62
48
|
}
|
|
63
49
|
|
|
64
50
|
/** Stores the assistant's neutral avatar, the picture of an agent that has none yet. */
|
|
65
51
|
async init(): Promise<void> {
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
)
|
|
52
|
+
try {
|
|
53
|
+
this.#defaultHash = await this.#referenceStore();
|
|
54
|
+
} catch (error) {
|
|
55
|
+
const { neutralPath, onReferenceFailed } = this.#options;
|
|
56
|
+
if (!neutralPath || neutralPath === this.#referencePath) throw error;
|
|
57
|
+
const code =
|
|
58
|
+
(error as { code?: unknown }).code ??
|
|
59
|
+
(error as { reason?: unknown }).reason ??
|
|
60
|
+
(error as Error).name;
|
|
61
|
+
onReferenceFailed?.(
|
|
62
|
+
/^[A-Za-z][A-Za-z0-9_-]{0,40}$/.test(String(code))
|
|
63
|
+
? String(code)
|
|
64
|
+
: "error",
|
|
65
|
+
);
|
|
66
|
+
this.#referencePath = neutralPath;
|
|
67
|
+
this.#defaultHash = await this.#referenceStore();
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
async #referenceStore(): Promise<string> {
|
|
72
|
+
return this.#store(readFileSync(this.#referencePath), REFERENCE_MAX_SIDE);
|
|
69
73
|
}
|
|
70
74
|
|
|
71
75
|
/** Whether an image provider is configured; without one, only `fallback` makes pictures. */
|
|
@@ -87,10 +91,10 @@ export class FileAvatarStudio implements AvatarStudio {
|
|
|
87
91
|
*/
|
|
88
92
|
async draw(avatarPrompt: string): Promise<string> {
|
|
89
93
|
const draw = this.#drawer();
|
|
90
|
-
const reference = await squarePng(
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
);
|
|
94
|
+
const reference = await squarePng(readFileSync(this.#referencePath), {
|
|
95
|
+
size: REFERENCE_SIDE,
|
|
96
|
+
maxSide: REFERENCE_MAX_SIDE,
|
|
97
|
+
});
|
|
94
98
|
const bytes = await draw(
|
|
95
99
|
`Draw a square Discord avatar of the person in the reference image, redrawn as an anime character: keep their face, short dark hair, and features recognizable as the same person, but in vivid anime cel-shading with a big, exaggerated expression and a dramatic pose that shows the role. Dress them and paint the background exactly in the colours the description names; ignore the reference's green jacket and teal background. Head-and-shoulders framing that still reads at 40 px. No text, no border. The role: ${avatarPrompt}`,
|
|
96
100
|
[{ data: reference.toString("base64"), mimeType: "image/png" }],
|
|
@@ -119,7 +123,7 @@ export class FileAvatarStudio implements AvatarStudio {
|
|
|
119
123
|
return this.#store(
|
|
120
124
|
await fallbackAvatar(
|
|
121
125
|
displayName,
|
|
122
|
-
readFileSync(this.#
|
|
126
|
+
readFileSync(this.#referencePath),
|
|
123
127
|
name,
|
|
124
128
|
),
|
|
125
129
|
);
|
|
@@ -171,8 +175,11 @@ export class FileAvatarStudio implements AvatarStudio {
|
|
|
171
175
|
return join(this.#options.dir, `${hash}.png`);
|
|
172
176
|
}
|
|
173
177
|
|
|
174
|
-
async #store(bytes: Uint8Array): Promise<string> {
|
|
175
|
-
const png = await squarePng(bytes,
|
|
178
|
+
async #store(bytes: Uint8Array, maxSide?: number): Promise<string> {
|
|
179
|
+
const png = await squarePng(bytes, {
|
|
180
|
+
size: AVATAR_SIDE,
|
|
181
|
+
...(maxSide === undefined ? {} : { maxSide }),
|
|
182
|
+
});
|
|
176
183
|
const hash = createHash("sha256").update(png).digest("hex");
|
|
177
184
|
const path = this.#path(hash);
|
|
178
185
|
if (!existsSync(path)) writeFileSync(path, png);
|
|
@@ -9,6 +9,9 @@ import { plainText } from "./attachment-name.ts";
|
|
|
9
9
|
|
|
10
10
|
export const MAX_ATTACHMENT_BYTES = 25 * 1024 * 1024;
|
|
11
11
|
|
|
12
|
+
/** How long one download may go without a byte (the headers count as the first) before it is given up, by default. */
|
|
13
|
+
export const DEFAULT_IDLE_MS = 10_000;
|
|
14
|
+
|
|
12
15
|
export interface FetchRequest {
|
|
13
16
|
refs: readonly AttachmentRef[];
|
|
14
17
|
/** Directory of the channel's attachment store; created when missing. */
|
|
@@ -16,6 +19,10 @@ export interface FetchRequest {
|
|
|
16
19
|
/** Prefix that keeps files of different messages apart, such as the message ID. */
|
|
17
20
|
prefix: string;
|
|
18
21
|
fromReference: boolean;
|
|
22
|
+
/** Aborts the download of the files still to come, and the one in flight. */
|
|
23
|
+
signal?: AbortSignal;
|
|
24
|
+
/** Gives up on a file that sends no byte for this long, and goes on to the next; default 10 s. */
|
|
25
|
+
idleMs?: number;
|
|
19
26
|
}
|
|
20
27
|
|
|
21
28
|
export interface FetchResult {
|
|
@@ -56,6 +63,70 @@ export function safeFileName(name: string): string {
|
|
|
56
63
|
return cleaned || "file";
|
|
57
64
|
}
|
|
58
65
|
|
|
66
|
+
/**
|
|
67
|
+
* A failure's code, never its message: a message can quote the directory a file was to be written
|
|
68
|
+
* to, a host path the model and the channel have no use for. The code is an error's `code` (such
|
|
69
|
+
* as `ENOTDIR`) or its `name` (such as `TimeoutError`), and `error` when it has neither.
|
|
70
|
+
*/
|
|
71
|
+
function failureCode(error: unknown): string {
|
|
72
|
+
const word = /^[A-Za-z][A-Za-z0-9_]{0,40}$/;
|
|
73
|
+
if (typeof error !== "object" || error === null) return "error";
|
|
74
|
+
const { code, name } = error as { code?: unknown; name?: unknown };
|
|
75
|
+
if (typeof code === "string" && word.test(code)) return code;
|
|
76
|
+
if (typeof name === "string" && name !== "Error" && word.test(name))
|
|
77
|
+
return name;
|
|
78
|
+
return "error";
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
/**
|
|
82
|
+
* Ends a download that has gone quiet: `signal` aborts, with the reason `idle`, once `idleMs` pass
|
|
83
|
+
* without a `touch`, and also when the outer signal does.
|
|
84
|
+
*/
|
|
85
|
+
class IdleWatch {
|
|
86
|
+
readonly signal: AbortSignal;
|
|
87
|
+
readonly #own = new AbortController();
|
|
88
|
+
readonly #idleMs: number;
|
|
89
|
+
#timer: ReturnType<typeof setTimeout> | undefined;
|
|
90
|
+
|
|
91
|
+
constructor(idleMs: number, outer?: AbortSignal) {
|
|
92
|
+
this.#idleMs = idleMs;
|
|
93
|
+
this.signal = outer
|
|
94
|
+
? AbortSignal.any([outer, this.#own.signal])
|
|
95
|
+
: this.#own.signal;
|
|
96
|
+
this.touch();
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
/** Whether the download was ended for want of bytes, not by the outer signal. */
|
|
100
|
+
get idled(): boolean {
|
|
101
|
+
return this.#own.signal.aborted;
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
touch(): void {
|
|
105
|
+
clearTimeout(this.#timer);
|
|
106
|
+
this.#timer = setTimeout(
|
|
107
|
+
() => this.#own.abort(new Error("idle")),
|
|
108
|
+
this.#idleMs,
|
|
109
|
+
);
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
stop(): void {
|
|
113
|
+
clearTimeout(this.#timer);
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
/** Rejects with the signal's reason when it aborts, so a fetch that ignores the signal still ends. */
|
|
118
|
+
function untilAborted<T>(work: Promise<T>, signal: AbortSignal): Promise<T> {
|
|
119
|
+
if (signal.aborted) return Promise.reject(signal.reason);
|
|
120
|
+
let onAbort: (() => void) | undefined;
|
|
121
|
+
const aborted = new Promise<never>((_, reject) => {
|
|
122
|
+
onAbort = () => reject(signal.reason);
|
|
123
|
+
signal.addEventListener("abort", onAbort, { once: true });
|
|
124
|
+
});
|
|
125
|
+
return Promise.race([work, aborted]).finally(() => {
|
|
126
|
+
if (onAbort) signal.removeEventListener("abort", onAbort);
|
|
127
|
+
});
|
|
128
|
+
}
|
|
129
|
+
|
|
59
130
|
/** Downloads attachments one by one; a failed file is reported without stopping the others. */
|
|
60
131
|
export async function fetchAttachments(
|
|
61
132
|
request: FetchRequest,
|
|
@@ -63,25 +134,35 @@ export async function fetchAttachments(
|
|
|
63
134
|
): Promise<FetchResult> {
|
|
64
135
|
const files: StoredAttachment[] = [];
|
|
65
136
|
const failures: AttachmentFailure[] = [];
|
|
66
|
-
const { refs, dir, prefix, fromReference } = request;
|
|
137
|
+
const { refs, dir, prefix, fromReference, signal } = request;
|
|
138
|
+
const idleMs = request.idleMs ?? DEFAULT_IDLE_MS;
|
|
67
139
|
if (refs.length === 0) return { files, failures };
|
|
68
|
-
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
|
69
140
|
|
|
70
141
|
for (const [index, ref] of refs.entries()) {
|
|
71
142
|
const fail = (reason: string) =>
|
|
72
143
|
failures.push({ name: ref.name, reason, fromReference });
|
|
144
|
+
if (signal?.aborted) {
|
|
145
|
+
fail("download failed: deadline passed or download aborted");
|
|
146
|
+
continue;
|
|
147
|
+
}
|
|
73
148
|
if (ref.size > MAX_ATTACHMENT_BYTES) {
|
|
74
149
|
fail(`larger than 25 MB (${ref.size} bytes)`);
|
|
75
150
|
continue;
|
|
76
151
|
}
|
|
152
|
+
const watch = new IdleWatch(idleMs, signal);
|
|
77
153
|
try {
|
|
78
|
-
|
|
154
|
+
// A directory that cannot be made fails the file, not the turn.
|
|
155
|
+
mkdirSync(dir, { recursive: true, mode: 0o700 });
|
|
156
|
+
const response = await untilAborted(
|
|
157
|
+
fetchImpl(ref.url, { signal: watch.signal }),
|
|
158
|
+
watch.signal,
|
|
159
|
+
);
|
|
79
160
|
if (!response.ok) {
|
|
80
161
|
await response.body?.cancel();
|
|
81
162
|
fail(`download failed with HTTP ${response.status}`);
|
|
82
163
|
continue;
|
|
83
164
|
}
|
|
84
|
-
const data = await readAttachmentBody(response);
|
|
165
|
+
const data = await readAttachmentBody(response, watch);
|
|
85
166
|
if (typeof data === "number") {
|
|
86
167
|
fail(`larger than 25 MB (${data} bytes)`);
|
|
87
168
|
continue;
|
|
@@ -101,7 +182,11 @@ export async function fetchAttachments(
|
|
|
101
182
|
fromReference,
|
|
102
183
|
});
|
|
103
184
|
} catch (error) {
|
|
104
|
-
fail(
|
|
185
|
+
fail(
|
|
186
|
+
`download failed: ${watch.idled && !signal?.aborted ? "idle_timeout" : failureCode(error)}`,
|
|
187
|
+
);
|
|
188
|
+
} finally {
|
|
189
|
+
watch.stop();
|
|
105
190
|
}
|
|
106
191
|
}
|
|
107
192
|
return { files, failures };
|
|
@@ -110,15 +195,22 @@ export async function fetchAttachments(
|
|
|
110
195
|
/** Returns the bytes, or the observed size once the stream exceeds the limit. */
|
|
111
196
|
async function readAttachmentBody(
|
|
112
197
|
response: Response,
|
|
198
|
+
watch: IdleWatch,
|
|
113
199
|
): Promise<Uint8Array | number> {
|
|
114
200
|
if (!response.body) return new Uint8Array();
|
|
201
|
+
const { signal } = watch;
|
|
115
202
|
const reader = response.body.getReader();
|
|
116
203
|
const chunks: Uint8Array[] = [];
|
|
117
204
|
let size = 0;
|
|
205
|
+
// A body that stalls ends with the signal, also when the fetch itself ignores it.
|
|
206
|
+
const stop = () => void reader.cancel(signal.reason).catch(() => {});
|
|
207
|
+
signal.addEventListener("abort", stop, { once: true });
|
|
118
208
|
try {
|
|
209
|
+
if (signal.aborted) throw signal.reason ?? new Error("aborted");
|
|
119
210
|
for (;;) {
|
|
120
211
|
const { done, value } = await reader.read();
|
|
121
212
|
if (done) break;
|
|
213
|
+
watch.touch();
|
|
122
214
|
size += value.byteLength;
|
|
123
215
|
if (size > MAX_ATTACHMENT_BYTES) {
|
|
124
216
|
await reader.cancel();
|
|
@@ -126,7 +218,10 @@ async function readAttachmentBody(
|
|
|
126
218
|
}
|
|
127
219
|
chunks.push(value);
|
|
128
220
|
}
|
|
221
|
+
// A cancelled reader reads as the end of the body, so a cut-short body is not a whole file.
|
|
222
|
+
if (signal.aborted) throw signal.reason ?? new Error("aborted");
|
|
129
223
|
} finally {
|
|
224
|
+
signal.removeEventListener("abort", stop);
|
|
130
225
|
reader.releaseLock();
|
|
131
226
|
}
|
|
132
227
|
const data = new Uint8Array(size);
|
|
@@ -2,10 +2,22 @@ import { ImagePreparationError } from "./image-preparation-error.ts";
|
|
|
2
2
|
|
|
3
3
|
const invalid = () => new ImagePreparationError("invalid-image");
|
|
4
4
|
|
|
5
|
-
/**
|
|
6
|
-
|
|
5
|
+
/**
|
|
6
|
+
* Inspect only encoded headers/blocks. Never allocate a raster or follow a media URL. With
|
|
7
|
+
* `maxSide`, a declared width or height (of a canvas or of any frame) over it is `pixel-limit` too.
|
|
8
|
+
*/
|
|
9
|
+
export function assertImagePixelBudget(
|
|
10
|
+
bytes: Buffer,
|
|
11
|
+
budget: number,
|
|
12
|
+
maxSide?: number,
|
|
13
|
+
): void {
|
|
7
14
|
const pixels = (width: number, height: number): number => {
|
|
8
15
|
if (width < 1 || height < 1) throw invalid();
|
|
16
|
+
if (maxSide !== undefined && (width > maxSide || height > maxSide))
|
|
17
|
+
throw new ImagePreparationError(
|
|
18
|
+
"pixel-limit",
|
|
19
|
+
`Image exceeds the ${maxSide}-pixel side limit.`,
|
|
20
|
+
);
|
|
9
21
|
const area = width * height;
|
|
10
22
|
check(area);
|
|
11
23
|
return area;
|
|
@@ -2,13 +2,17 @@
|
|
|
2
2
|
export class ImagePreparationError extends Error {
|
|
3
3
|
override readonly name = "ImagePreparationError";
|
|
4
4
|
|
|
5
|
-
constructor(
|
|
5
|
+
constructor(
|
|
6
|
+
readonly reason: "byte-limit" | "pixel-limit" | "invalid-image",
|
|
7
|
+
detail?: string,
|
|
8
|
+
) {
|
|
6
9
|
super(
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
10
|
+
detail ??
|
|
11
|
+
{
|
|
12
|
+
"byte-limit": "Image is larger than 25 MiB.",
|
|
13
|
+
"pixel-limit": "Image exceeds the 64 MP decoded-pixel budget.",
|
|
14
|
+
"invalid-image": "Image has an invalid or unsupported raster header.",
|
|
15
|
+
}[reason],
|
|
12
16
|
);
|
|
13
17
|
}
|
|
14
18
|
}
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
import { createCanvas, loadImage } from "canvas";
|
|
2
|
+
import { assertImagePixelBudget } from "./image-budget.ts";
|
|
3
|
+
import { ImagePreparationError } from "./image-preparation-error.ts";
|
|
4
|
+
|
|
5
|
+
/** The longest side a picture may declare before `squarePng` refuses to decode it. */
|
|
6
|
+
export const SQUARE_PNG_MAX_SIDE = 4096;
|
|
7
|
+
/** The side `squarePng` scales to when it is not told one. */
|
|
8
|
+
const DEFAULT_SIDE = 512;
|
|
9
|
+
/** Below this side `squarePng` gives up fitting `maxBytes`. */
|
|
10
|
+
const SMALLEST_SIDE = 16;
|
|
11
|
+
/** Frames of an animation add up, so the whole stays within the budget `prepareImageBytes` allows. */
|
|
12
|
+
const DECODED_PIXEL_BUDGET = 8000 * 8000;
|
|
13
|
+
|
|
14
|
+
export interface SquarePngOptions {
|
|
15
|
+
/** The most pixels per side of the result; default 512. A smaller picture is not enlarged. */
|
|
16
|
+
size?: number;
|
|
17
|
+
/** The most bytes of the result; the side steps down by a quarter until it fits. Default none. */
|
|
18
|
+
maxBytes?: number;
|
|
19
|
+
/**
|
|
20
|
+
* The longest side the picture may declare before it is refused, default 4096
|
|
21
|
+
* (`SQUARE_PNG_MAX_SIDE`). A picture within the 64 MP decoded-pixel budget but over it, such as a
|
|
22
|
+
* camera photo, is admitted by a larger value.
|
|
23
|
+
*/
|
|
24
|
+
maxSide?: number;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
const wholeNumber = (name: string, value: number, max: number) => {
|
|
28
|
+
if (!Number.isInteger(value) || value < 1 || value > max)
|
|
29
|
+
throw new RangeError(
|
|
30
|
+
`squarePng: ${name} must be a whole number from 1 to ${max}`,
|
|
31
|
+
);
|
|
32
|
+
};
|
|
33
|
+
|
|
34
|
+
/**
|
|
35
|
+
* A centered square crop of a PNG, JPEG or GIF, scaled down to at most `size` pixels per
|
|
36
|
+
* side, as PNG. The header is read before anything is decoded: a declared width or height over
|
|
37
|
+
* 4096 (`SQUARE_PNG_MAX_SIDE`, or `maxSide`) rejects with `ImagePreparationError` (`pixel-limit`),
|
|
38
|
+
* and bytes of none of the three types (WebP included) with `invalid-image`, as does a picture
|
|
39
|
+
* that cannot be decoded. With `maxBytes` the side steps down (by a quarter
|
|
40
|
+
* each time) until the PNG is no larger; a picture that does not fit even at 16 pixels throws a
|
|
41
|
+
* `RangeError`.
|
|
42
|
+
*/
|
|
43
|
+
export async function squarePng(
|
|
44
|
+
bytes: Uint8Array,
|
|
45
|
+
options: SquarePngOptions = {},
|
|
46
|
+
): Promise<Buffer> {
|
|
47
|
+
const side = options.size ?? DEFAULT_SIDE;
|
|
48
|
+
wholeNumber("size", side, SQUARE_PNG_MAX_SIDE);
|
|
49
|
+
if (options.maxBytes !== undefined)
|
|
50
|
+
wholeNumber("maxBytes", options.maxBytes, Number.MAX_SAFE_INTEGER);
|
|
51
|
+
if (options.maxSide !== undefined)
|
|
52
|
+
wholeNumber("maxSide", options.maxSide, Number.MAX_SAFE_INTEGER);
|
|
53
|
+
const source = Buffer.from(bytes);
|
|
54
|
+
// The canvas has no WebP decoder, so a WebP is refused here rather than failing in the decode.
|
|
55
|
+
if (
|
|
56
|
+
source.toString("ascii", 0, 4) === "RIFF" &&
|
|
57
|
+
source.toString("ascii", 8, 12) === "WEBP"
|
|
58
|
+
)
|
|
59
|
+
throw new ImagePreparationError("invalid-image");
|
|
60
|
+
assertImagePixelBudget(
|
|
61
|
+
source,
|
|
62
|
+
DECODED_PIXEL_BUDGET,
|
|
63
|
+
options.maxSide ?? SQUARE_PNG_MAX_SIDE,
|
|
64
|
+
);
|
|
65
|
+
const image = await loadImage(source).catch(() => {
|
|
66
|
+
throw new ImagePreparationError("invalid-image");
|
|
67
|
+
});
|
|
68
|
+
const crop = Math.min(image.width, image.height);
|
|
69
|
+
let size = Math.min(side, crop);
|
|
70
|
+
for (;;) {
|
|
71
|
+
const canvas = createCanvas(size, size);
|
|
72
|
+
canvas
|
|
73
|
+
.getContext("2d")
|
|
74
|
+
.drawImage(
|
|
75
|
+
image,
|
|
76
|
+
(image.width - crop) / 2,
|
|
77
|
+
(image.height - crop) / 2,
|
|
78
|
+
crop,
|
|
79
|
+
crop,
|
|
80
|
+
0,
|
|
81
|
+
0,
|
|
82
|
+
size,
|
|
83
|
+
size,
|
|
84
|
+
);
|
|
85
|
+
const png = canvas.toBuffer("image/png");
|
|
86
|
+
if (options.maxBytes === undefined || png.byteLength <= options.maxBytes)
|
|
87
|
+
return png;
|
|
88
|
+
size = Math.floor(size * 0.75);
|
|
89
|
+
if (size < SMALLEST_SIDE)
|
|
90
|
+
throw new RangeError(
|
|
91
|
+
`squarePng: the picture does not fit ${options.maxBytes} bytes even at ${SMALLEST_SIDE} pixels`,
|
|
92
|
+
);
|
|
93
|
+
}
|
|
94
|
+
}
|
|
@@ -4,19 +4,43 @@ import type { Logger } from "../log.ts";
|
|
|
4
4
|
import { fetchAttachments } from "./attachment-fetcher.ts";
|
|
5
5
|
import { modelImagesOf } from "./model-images.ts";
|
|
6
6
|
|
|
7
|
+
/**
|
|
8
|
+
* The most milliseconds all of a message's downloads take together by default. A download that
|
|
9
|
+
* stalls ends at `DEFAULT_IDLE_MS` without bytes, so the deadline only has to cover slow ones.
|
|
10
|
+
*/
|
|
11
|
+
export const DEFAULT_DEADLINE_MS = 60_000;
|
|
12
|
+
|
|
7
13
|
/** Saves the message's and its reference's files, and turns up to four images into model images. */
|
|
8
14
|
export async function collectAttachments(
|
|
9
15
|
message: InboundMessage,
|
|
10
16
|
dir: string,
|
|
11
|
-
options: {
|
|
17
|
+
options: {
|
|
18
|
+
logger: Logger;
|
|
19
|
+
fetchImpl?: typeof fetch;
|
|
20
|
+
/** One deadline for the message's and its reference's files together; default 60 s. */
|
|
21
|
+
deadlineMs?: number;
|
|
22
|
+
/** How long one file may send no byte before it is given up and the next one tried; default 10 s. */
|
|
23
|
+
idleMs?: number;
|
|
24
|
+
/** Aborts the downloads, as the deadline does. */
|
|
25
|
+
signal?: AbortSignal;
|
|
26
|
+
},
|
|
12
27
|
): Promise<TurnAttachments> {
|
|
13
28
|
const { fetchImpl, logger } = options;
|
|
29
|
+
const deadline = AbortSignal.timeout(
|
|
30
|
+
options.deadlineMs ?? DEFAULT_DEADLINE_MS,
|
|
31
|
+
);
|
|
32
|
+
const signal = options.signal
|
|
33
|
+
? AbortSignal.any([deadline, options.signal])
|
|
34
|
+
: deadline;
|
|
35
|
+
const idle = options.idleMs === undefined ? {} : { idleMs: options.idleMs };
|
|
14
36
|
const own = await fetchAttachments(
|
|
15
37
|
{
|
|
16
38
|
refs: message.attachments,
|
|
17
39
|
dir,
|
|
18
40
|
prefix: message.messageId,
|
|
19
41
|
fromReference: false,
|
|
42
|
+
signal,
|
|
43
|
+
...idle,
|
|
20
44
|
},
|
|
21
45
|
fetchImpl,
|
|
22
46
|
);
|
|
@@ -26,6 +50,8 @@ export async function collectAttachments(
|
|
|
26
50
|
dir,
|
|
27
51
|
prefix: `${message.messageId}-ref`,
|
|
28
52
|
fromReference: true,
|
|
53
|
+
signal,
|
|
54
|
+
...idle,
|
|
29
55
|
},
|
|
30
56
|
fetchImpl,
|
|
31
57
|
);
|
|
@@ -54,9 +80,11 @@ export function attachmentsOf(
|
|
|
54
80
|
dir: string,
|
|
55
81
|
logger: Logger,
|
|
56
82
|
fetchImpl?: typeof fetch,
|
|
83
|
+
limits: { deadlineMs?: number; idleMs?: number; signal?: AbortSignal } = {},
|
|
57
84
|
): Promise<TurnAttachments> {
|
|
58
85
|
return collectAttachments(message, dir, {
|
|
59
86
|
logger,
|
|
87
|
+
...limits,
|
|
60
88
|
...(fetchImpl ? { fetchImpl } : {}),
|
|
61
89
|
});
|
|
62
90
|
}
|
|
@@ -69,6 +69,12 @@ export interface AgentServerOptions {
|
|
|
69
69
|
* `"owners"`, which leaves a speaker below the owner tier unanswered.
|
|
70
70
|
*/
|
|
71
71
|
speakers?: "everyone" | "owners";
|
|
72
|
+
/**
|
|
73
|
+
* Limits of the downloads of a message's files (`discord.attachmentDownload`): `deadlineMs`
|
|
74
|
+
* for all of a message's files together (default 60 s), `idleMs` for one file to send no byte
|
|
75
|
+
* (default 10 s).
|
|
76
|
+
*/
|
|
77
|
+
attachmentDownload?: { deadlineMs?: number; idleMs?: number };
|
|
72
78
|
/** The prompt every agent starts with, and the one for speakers other than the owner. */
|
|
73
79
|
prompts: { shared: string; guest?: string };
|
|
74
80
|
/** The listener the avatar pictures are served on. */
|
|
@@ -77,6 +83,8 @@ export interface AgentServerOptions {
|
|
|
77
83
|
avatarUrl: string;
|
|
78
84
|
/** The assistant's neutral avatar: the style reference and the picture of an agent without one. */
|
|
79
85
|
avatarReference: string;
|
|
86
|
+
/** The picture used instead when `avatarReference` cannot be read as one; the failure is logged. */
|
|
87
|
+
avatarNeutral?: string;
|
|
80
88
|
/** Reports the process's own errors to an agent; without one nothing is reported. */
|
|
81
89
|
errorReporter?: ErrorReporter;
|
|
82
90
|
}
|
|
@@ -155,6 +163,16 @@ export function agentServerPlugin(
|
|
|
155
163
|
dir: join(options.dataDir, "avatars"),
|
|
156
164
|
publicUrl: options.avatarUrl,
|
|
157
165
|
referencePath: options.avatarReference,
|
|
166
|
+
...(options.avatarNeutral
|
|
167
|
+
? {
|
|
168
|
+
neutralPath: options.avatarNeutral,
|
|
169
|
+
onReferenceFailed: (code: string) =>
|
|
170
|
+
logger.warn(
|
|
171
|
+
{ code },
|
|
172
|
+
"the configured avatar could not be used; the neutral avatar is used instead",
|
|
173
|
+
),
|
|
174
|
+
}
|
|
175
|
+
: {}),
|
|
158
176
|
// Without an image provider the studio makes each agent a picture from its name.
|
|
159
177
|
...(providers.filled.has("images") ? { draw: providers.images } : {}),
|
|
160
178
|
});
|
|
@@ -293,6 +311,9 @@ export function agentServerPlugin(
|
|
|
293
311
|
runtime: running,
|
|
294
312
|
surface: context.surfaces,
|
|
295
313
|
attachmentDir,
|
|
314
|
+
...(options.attachmentDownload
|
|
315
|
+
? { attachmentDownload: options.attachmentDownload }
|
|
316
|
+
: {}),
|
|
296
317
|
logger,
|
|
297
318
|
channelContext: (message) => discord.channelContext(message),
|
|
298
319
|
}),
|
|
@@ -202,7 +202,11 @@ export function discordPlugin(options: DiscordOptions): RoundtablePlugin {
|
|
|
202
202
|
// The sweep reads threads through the connection, so it follows the surface's start.
|
|
203
203
|
{ name: "threads", start: () => void threads.sweep() },
|
|
204
204
|
// The copy texts of the last cards are on disk before the host exits.
|
|
205
|
-
{
|
|
205
|
+
{
|
|
206
|
+
name: "card-copies",
|
|
207
|
+
start: () => cards.startCopies(),
|
|
208
|
+
stop: () => cards.stopCopies(),
|
|
209
|
+
},
|
|
206
210
|
],
|
|
207
211
|
};
|
|
208
212
|
},
|
|
@@ -71,6 +71,14 @@ export interface DiscordConfig {
|
|
|
71
71
|
* remove them; they age out.
|
|
72
72
|
*/
|
|
73
73
|
cardCopies?: { keepDays?: number; keep?: number } | false;
|
|
74
|
+
/**
|
|
75
|
+
* How the files of an owner's or an agent's channel message are downloaded: `deadlineMs`
|
|
76
|
+
* (default 60,000) is the most time all of a message's files, and those of the message it
|
|
77
|
+
* replies to, may take together; `idleMs` (default 10,000) is how long one file may send no
|
|
78
|
+
* byte before it is given up. A file still to come at the deadline, or one that went quiet, is
|
|
79
|
+
* reported as `download failed` and the turn goes on without it.
|
|
80
|
+
*/
|
|
81
|
+
attachmentDownload?: { deadlineMs?: number; idleMs?: number };
|
|
74
82
|
}
|
|
75
83
|
|
|
76
84
|
/** The Discord settings with every default filled in. */
|
|
@@ -89,6 +97,8 @@ export interface ResolvedDiscord {
|
|
|
89
97
|
cardGraceSeconds: number;
|
|
90
98
|
/** As configured: `{}` when left out, so the defaults apply, or `false`. */
|
|
91
99
|
cardCopies: NonNullable<DiscordConfig["cardCopies"]> | false;
|
|
100
|
+
/** As configured: left out unless set, so the defaults apply. */
|
|
101
|
+
attachmentDownload?: NonNullable<DiscordConfig["attachmentDownload"]>;
|
|
92
102
|
}
|
|
93
103
|
|
|
94
104
|
/** The configured Discord settings, defaults filled in; `rootCommand` is the host's slug. */
|
|
@@ -117,6 +127,9 @@ export function resolvedDiscord(
|
|
|
117
127
|
: { freshMarker: discord.freshMarker }),
|
|
118
128
|
cardGraceSeconds: discord.cardGraceSeconds ?? 120,
|
|
119
129
|
cardCopies: discord.cardCopies ?? {},
|
|
130
|
+
...(discord.attachmentDownload === undefined
|
|
131
|
+
? {}
|
|
132
|
+
: { attachmentDownload: discord.attachmentDownload }),
|
|
120
133
|
};
|
|
121
134
|
}
|
|
122
135
|
|
|
@@ -163,6 +176,12 @@ export const discordShape = shape({
|
|
|
163
176
|
}),
|
|
164
177
|
),
|
|
165
178
|
),
|
|
179
|
+
attachmentDownload: optional(
|
|
180
|
+
shape({
|
|
181
|
+
deadlineMs: optional(integer(1_000, 600_000)),
|
|
182
|
+
idleMs: optional(integer(1_000, 600_000)),
|
|
183
|
+
}),
|
|
184
|
+
),
|
|
166
185
|
});
|
|
167
186
|
|
|
168
187
|
/** The adapters `adapters` takes, by the name their factory gives them. */
|
|
@@ -146,7 +146,11 @@ export interface RoundtableConfig {
|
|
|
146
146
|
socketPath?: string;
|
|
147
147
|
socketMode?: number;
|
|
148
148
|
};
|
|
149
|
-
/**
|
|
149
|
+
/**
|
|
150
|
+
* A picture of your own for agents without one, and the style reference for drawing them; default
|
|
151
|
+
* a plain bot. A PNG, JPEG or GIF of any side within the 64 MP decoded-pixel budget (an ordinary
|
|
152
|
+
* camera photo); one that cannot be read as a picture is logged and the plain bot is used.
|
|
153
|
+
*/
|
|
150
154
|
avatar?: string;
|
|
151
155
|
/** The agents' shared working directory; default `<dataDir>/work`. */
|
|
152
156
|
workDir?: string;
|
|
@@ -202,10 +202,14 @@ function discordAssembly(
|
|
|
202
202
|
prompts: { shared: sharedPrompt, guest },
|
|
203
203
|
...(discord.agentMemory ? { memory: discord.agentMemory } : {}),
|
|
204
204
|
...(discord.agentSpeakers ? { speakers: discord.agentSpeakers } : {}),
|
|
205
|
+
...(discord.attachmentDownload
|
|
206
|
+
? { attachmentDownload: discord.attachmentDownload }
|
|
207
|
+
: {}),
|
|
205
208
|
avatarListener: "public",
|
|
206
209
|
// resolveConfig refuses Discord without a public address.
|
|
207
210
|
avatarUrl: config.http?.publicUrl ?? "",
|
|
208
211
|
avatarReference: config.avatar ?? join(ASSETS, "neutral.png"),
|
|
212
|
+
avatarNeutral: join(ASSETS, "neutral.png"),
|
|
209
213
|
// An ops agent's reports are the agent server's to deliver.
|
|
210
214
|
...(errorReporter && "agent" in errorReporter.destination
|
|
211
215
|
? { errorReporter }
|
|
@@ -11,6 +11,8 @@ export const COPY_TEXT_LIMIT = 20_000;
|
|
|
11
11
|
const COPY_KEEP = 100;
|
|
12
12
|
/** How many days a copy text is kept by default. */
|
|
13
13
|
const COPY_KEEP_DAYS = 30;
|
|
14
|
+
/** How often expired texts are dropped from the file, an hour. */
|
|
15
|
+
const SWEEP_MS = 60 * 60_000;
|
|
14
16
|
|
|
15
17
|
interface Entry {
|
|
16
18
|
text: string;
|
|
@@ -27,6 +29,12 @@ export interface CardCopiesOptions {
|
|
|
27
29
|
logger: Logger;
|
|
28
30
|
/** Replaceable in tests. */
|
|
29
31
|
now?: () => number;
|
|
32
|
+
/** How often expired texts are dropped from the file; default `SWEEP_MS`. */
|
|
33
|
+
sweepMs?: number;
|
|
34
|
+
/** Replaceable in tests. */
|
|
35
|
+
setInterval?: typeof setInterval;
|
|
36
|
+
/** Replaceable in tests, with `setInterval`. */
|
|
37
|
+
clearInterval?: typeof clearInterval;
|
|
30
38
|
}
|
|
31
39
|
|
|
32
40
|
/**
|
|
@@ -42,6 +50,7 @@ export class CardCopies {
|
|
|
42
50
|
readonly #entries = new Map<string, Entry>();
|
|
43
51
|
#writing: Promise<void> | undefined;
|
|
44
52
|
#dirty = false;
|
|
53
|
+
#timer: ReturnType<typeof setInterval> | undefined;
|
|
45
54
|
|
|
46
55
|
constructor(options: CardCopiesOptions) {
|
|
47
56
|
this.#options = options;
|
|
@@ -56,7 +65,7 @@ export class CardCopies {
|
|
|
56
65
|
if (typeof text === "string" && typeof at === "number")
|
|
57
66
|
this.#entries.set(id, { text, at });
|
|
58
67
|
}
|
|
59
|
-
this.#prune();
|
|
68
|
+
if (this.#prune()) this.#schedule();
|
|
60
69
|
} catch (error) {
|
|
61
70
|
logger.warn(
|
|
62
71
|
{ path, err: error },
|
|
@@ -77,12 +86,31 @@ export class CardCopies {
|
|
|
77
86
|
const entry = this.#entries.get(id);
|
|
78
87
|
if (!entry) return undefined;
|
|
79
88
|
if (this.#now() - entry.at > this.#keepMs()) {
|
|
80
|
-
this.#
|
|
89
|
+
this.#prune();
|
|
90
|
+
this.#schedule();
|
|
81
91
|
return undefined;
|
|
82
92
|
}
|
|
83
93
|
return entry.text;
|
|
84
94
|
}
|
|
85
95
|
|
|
96
|
+
/** Starts the hourly sweep that drops expired texts from the file too; the timer does not hold the process. */
|
|
97
|
+
start(): void {
|
|
98
|
+
if (this.#timer || !this.#options.path) return;
|
|
99
|
+
const every = this.#options.setInterval ?? setInterval;
|
|
100
|
+
this.#timer = every(() => {
|
|
101
|
+
if (this.#prune()) this.#schedule();
|
|
102
|
+
}, this.#options.sweepMs ?? SWEEP_MS);
|
|
103
|
+
this.#timer.unref?.();
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
/** Stops the sweep and resolves once what was remembered so far is on disk; for a host that stops and for tests. */
|
|
107
|
+
async stop(): Promise<void> {
|
|
108
|
+
if (this.#timer)
|
|
109
|
+
(this.#options.clearInterval ?? clearInterval)(this.#timer);
|
|
110
|
+
this.#timer = undefined;
|
|
111
|
+
await this.flush();
|
|
112
|
+
}
|
|
113
|
+
|
|
86
114
|
/** Resolves once what was remembered so far is on disk; for a host that stops and for tests. */
|
|
87
115
|
async flush(): Promise<void> {
|
|
88
116
|
while (this.#writing) await this.#writing;
|
|
@@ -96,15 +124,18 @@ export class CardCopies {
|
|
|
96
124
|
return (this.#options.keepDays ?? COPY_KEEP_DAYS) * 24 * 60 * 60_000;
|
|
97
125
|
}
|
|
98
126
|
|
|
99
|
-
|
|
127
|
+
/** Drops expired and surplus entries; true when any went. */
|
|
128
|
+
#prune(): boolean {
|
|
129
|
+
const before = this.#entries.size;
|
|
100
130
|
const now = this.#now();
|
|
101
131
|
const keep = this.#options.keep ?? COPY_KEEP;
|
|
102
132
|
for (const [id, entry] of this.#entries)
|
|
103
133
|
if (now - entry.at > this.#keepMs()) this.#entries.delete(id);
|
|
104
134
|
for (const id of this.#entries.keys()) {
|
|
105
|
-
if (this.#entries.size <= keep)
|
|
135
|
+
if (this.#entries.size <= keep) break;
|
|
106
136
|
this.#entries.delete(id);
|
|
107
137
|
}
|
|
138
|
+
return this.#entries.size < before;
|
|
108
139
|
}
|
|
109
140
|
|
|
110
141
|
/** Writes the file after the call that asked, once at a time; asks made meanwhile share the next write. */
|
|
@@ -111,6 +111,8 @@ export class OwnerCards implements InteractionModule {
|
|
|
111
111
|
}
|
|
112
112
|
/** Resolves once the remembered copy texts are on disk. */
|
|
113
113
|
flushCopies = (): Promise<void> => this.#copies.flush();
|
|
114
|
+
startCopies = (): void => this.#copies.start();
|
|
115
|
+
stopCopies = (): Promise<void> => this.#copies.stop();
|
|
114
116
|
|
|
115
117
|
commands() {
|
|
116
118
|
return [];
|
package/src/discord/index.ts
CHANGED
|
@@ -21,7 +21,10 @@ export type {
|
|
|
21
21
|
AgentPanelOptions,
|
|
22
22
|
} from "../core/discord/agent-panel.ts";
|
|
23
23
|
export { agentPanel } from "../core/discord/agent-panel.ts";
|
|
24
|
-
export {
|
|
24
|
+
export {
|
|
25
|
+
CHANNEL_CONTEXT_DEFAULTS,
|
|
26
|
+
formatChannelContext,
|
|
27
|
+
} from "../core/discord/channel-context.ts";
|
|
25
28
|
export type { ManagedChannel } from "../core/discord/channel-executor.ts";
|
|
26
29
|
export {
|
|
27
30
|
fetchManagedChannel,
|
package/src/kit/channels.ts
CHANGED
|
@@ -9,6 +9,11 @@ export {
|
|
|
9
9
|
export { prepareImageBytes } from "../core/attachments/image-prep.ts";
|
|
10
10
|
export { ImagePreparationError } from "../core/attachments/image-preparation-error.ts";
|
|
11
11
|
export { withAttachmentsBlock } from "../core/attachments/prompt-block.ts";
|
|
12
|
+
export {
|
|
13
|
+
SQUARE_PNG_MAX_SIDE,
|
|
14
|
+
type SquarePngOptions,
|
|
15
|
+
squarePng,
|
|
16
|
+
} from "../core/attachments/square-png.ts";
|
|
12
17
|
export { attachmentsOf } from "../core/attachments/turn-attachments.ts";
|
|
13
18
|
export type { ChannelQueue } from "../core/routing/channel-queue.ts";
|
|
14
19
|
export { channelQueue } from "../core/routing/channel-queue.ts";
|