pi-roundtable 0.18.0 → 0.19.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,24 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [0.19.1] - 2026-10-18
9
+
10
+ ### Fixed
11
+
12
+ - The core's tests no longer depend on the host's `mktemp` or on a free-port probe. The check that the agents' `bash` writes into the scratch dir ran a bare `mktemp`, which macOS's BSD `mktemp` sends to the user temp dir whatever `TMPDIR` says; it now runs `mktemp -p "$TMPDIR"`, which both implementations honour, and the bare `mktemp` is held to the scratch dir only where GNU `mktemp` is the default. The listener tests found a free port by opening one and closing it, and another process could take it before the listener bound it (`EADDRINUSE`); they now let the listener bind port 0 and ask which port it got.
13
+ - The docs say what the scratch dir promises: the agents' shell runs with `TMPDIR` pointing to it, so tools that read `TMPDIR` and GNU `mktemp` write there, while macOS's BSD `mktemp` ignores `TMPDIR` for a bare `mktemp` and needs `mktemp -p "$TMPDIR"`. The agents' prompt is unchanged: it already sends temporary files to `$TMPDIR` and never asks for a bare `mktemp`.
14
+
15
+ ## [0.19.0] - 2026-10-17
16
+
17
+ ### Added
18
+
19
+ - `drawCodexImage`, `createCodexImages`, `ImageNotGeneratedError`, `CODEX_IMAGE_MODEL`, `CODEX_IMAGES_PROVIDER`, `CodexImage`, `CodexImageRequest`, `CodexImagesOptions` and `CodexFetch` (`pi-roundtable/kit`): the Codex image client and the `codex-images` plugin built on it, so a project imports one implementation instead of copying one. `drawCodexImage({ token, prompt, references?, model?, signal?, fetch? })` draws one image through ChatGPT's Codex backend and returns `{ data, revisedPrompt? }`, `data` being PNG bytes and `revisedPrompt` the prompt Codex says it drew. It aborts with `signal`, both before the response arrives and while its stream is read, and rejects with the signal's own reason, so a turn's `signal?.throwIfAborted()` still sees its own abort; it stops after five minutes with a `TimeoutError`, refuses a stream over 32 MiB (`Codex image response too large`), and reads only the first image. A call that ran and returned no image, and a completed response with no image at all, both throw `ImageNotGeneratedError`: asking again with the same prompt fails again. `createCodexImages({ fetch?, model?, onMissingLogin? })` is the plugin `codex-images`: it fills `providers.images`, reads the `openai-codex` login through `context.apiKey` for each drawing, and passes the drawing's `options.signal` on, so the turn that asked for an image can abort it. `CODEX_IMAGES_PROVIDER` is `"openai-codex"` and `CODEX_IMAGE_MODEL` is `"gpt-6-sol"`, the model that routes the request. The backend is ChatGPT's own, signed in with the owner's ChatGPT subscription, and OpenAI does not document it for this use: it can stop working without notice, and OpenAI's terms for the subscription apply.
20
+ - `jevJudge` and `JevJudgeOptions` (`pi-roundtable/kit`): a `Judge` over Jev's hosted service, built on pi-jev-compaction's client (no new dependency), for the `judge` provider. `jevJudge({ apiKey, model?, baseUrl?, timeoutMs?, fetch? })` sends each question as one request: a yes-or-no question as Jev's own question type, a choice answer checked against the options listed, and a score read by criterion index, every probability a number from 0 to 1. Every failure throws a `JudgeError`. A question has its own limit of `timeoutMs` (default 20,000) that also covers reading the answer. An empty or blank `apiKey` throws a `JudgeError` when the judge is made, because the client would otherwise use the key in the environment.
21
+
22
+ ### Changed
23
+
24
+ - The `codex-images` template (`roundtable add plugin codex-images`) is now a short file that imports its client and plugin from `pi-roundtable/kit` (`createCodexImages`, built on `drawCodexImage`) and makes the `codexImages` plugin your config lists, under the same warning head and still exporting `createCodexImages`, `ImageNotGeneratedError`, `CodexFetch` and `CodexImagesOptions`. Its `images` provider now aborts with the turn that asked (it ignored the `signal` the core's `generate_image` tool hands to a drawer), and it refuses a response stream over 32 MiB; a PNG of about 12 MiB, which the completed event may repeat, can be refused. Its copied test gains one that aborts a drawing through `options.signal`. A plugin already copied is unchanged until it is copied again.
25
+
8
26
  ## [0.18.0] - 2026-10-17
9
27
 
10
28
  ### Added
package/docs/plugins.md CHANGED
@@ -63,6 +63,12 @@ Use the kit's building blocks for a plugin that runs Pi itself, such as a coding
63
63
  - Square pictures: `squarePng(bytes, { size?, maxBytes?, maxSide? })` crops a PNG, JPEG or GIF to its centered square and scales it down to at most `size` pixels per side (default 512; a smaller picture is not enlarged), as PNG: the helper the agent avatars are made with, for a plugin that needs a profile picture or an icon.
64
64
  The header is read first, and nothing is decoded for a picture that declares a width or height over `SQUARE_PNG_MAX_SIDE` (4096), which rejects with `ImagePreparationError` (`pixel-limit`, the message naming the side limit; `maxSide` sets another limit, and the 64 MP decoded-pixel budget always applies), or for bytes of no such type, a WebP (the canvas cannot decode it) or a picture that fails to decode (`invalid-image`).
65
65
  With `maxBytes` the side steps down by a quarter at a time until the PNG is no larger; a picture that does not fit even at 16 pixels throws a `RangeError`, as does a `size` or `maxBytes` that is not a whole number in range. `SquarePngOptions` types the options.
66
+ - Codex images: `drawCodexImage({ token, prompt, references?, model?, signal?, fetch? })` draws one image through ChatGPT's Codex backend with the owner's `openai-codex` login (`CODEX_IMAGES_PROVIDER`) and returns `{ data, revisedPrompt? }`: `data` is PNG bytes, and `revisedPrompt` is the prompt Codex says it drew, when it says. `model` defaults to `CODEX_IMAGE_MODEL` (`gpt-6-sol`), which only routes the request; the backend picks the image model. `fetch` is a `CodexFetch`, which `typeof fetch` satisfies, for tests.
67
+ The call aborts with `signal`, before the response arrives and while its stream is read, and rejects with the signal's own reason, so a turn's `signal?.throwIfAborted()` still sees its own abort. It stops after five minutes with a `TimeoutError`, refuses a stream over 32 MiB (`Codex image response too large`; the completed event may repeat the image, so a PNG of about 12 MiB can be refused), and reads only the first image. An HTTP error carries its status and the first 300 characters of the body, never the token.
68
+ A call that ran and returned no image, and a response that completed without one, both throw `ImageNotGeneratedError`, usually because Codex refused the prompt; asking again with the same prompt fails again.
69
+ `createCodexImages({ fetch?, model?, onMissingLogin? })` is the `codex-images` plugin built on it. It fills the `images` provider, reads the login through `context.apiKey` for each drawing (the login refreshes), and passes the drawing's `options.signal` on, so the turn that asked for an image can abort it. Without a login, setup throws a `PluginError`; `onMissingLogin: "warn"` logs once and lets the host start, and a drawing then fails with the same `PluginError`. The kit exports no ready-made plugin object, because a plugin object shared between `check()` and `run()` should come from your own module; the `codex-images` template makes one.
70
+ The types are `CodexFetch`, `CodexImage`, `CodexImageRequest` and `CodexImagesOptions`.
71
+ This uses ChatGPT's Codex backend, signed in with the owner's ChatGPT subscription. OpenAI does not document it for this use, so it can stop working without notice, and OpenAI's terms for the subscription apply.
66
72
  - Work: `promptSlot` (how a run asks the owner while it works), `workTimeout` (a time limit that does not count the time spent waiting on the owner), `runWorkerTask`, `archiveSessions`, and `approvalCard`, `approvalDetails` and `canonicalJson` for the cards of held actions.
67
73
  `InterimPoster(posts, { logger, channel, primaryChars?, editMs? })` posts a run's [interim text](#interim-text-what-a-turn-writes-before-its-final-answer) to an `InterimPosts`, such as a dispatch thread's `interim`: feed it `messageEnd(message)` and `toolStart(name)` from the session's events, bind `() => poster.flush()` as the slot's `beforeCard`, and `await poster.flush()` before the final report.
68
74
  - Diagnostics: `scrubDiagnostic(text, max = 600)` masks credentials (URL userinfo, token shapes, secret-named assignments and JSON fields, `Authorization`/`Cookie`/`x-api-key` headers, JWTs, PEM blocks), turns control characters other than tab and newline into spaces, and cuts the result at `max` characters.
@@ -80,6 +86,9 @@ Use the kit's building blocks for a plugin that runs Pi itself, such as a coding
80
86
  Two adapters log each skip as `Jev leaves the compaction to Pi's summary` with `reason`, `detail` and `tokensBefore` through the `logger` they take: `jevCompactionExtension({ logger, ...options })`, the `compaction` session tool's extension, placed as `session.compaction.wrap(jevCompactionExtension({ logger }))` under the engine `JEV_COMPACTION_ENGINE` (`pi-jev-compaction`), and `jevCompactor({ logger, ...options })`, a `JevCompactor` for pi-roundtable-sandbox's `compaction` option that returns the compaction or `undefined` and logs with the `channel`.
81
87
  A `no_key` skip is logged once per adapter, as `Jev is not configured, so compaction uses Pi's summary`, and then not again; every other skip is logged each time.
82
88
  The types are `JevCompactInput`, `JevCompactOptions`, `JevCompactOutcome`, `JevCompactor`, `JevCompactRequest`, `JevExtensionOptions` and `JevSkipReason`.
89
+ - Jev judge: `jevJudge({ apiKey, model?, baseUrl?, timeoutMs?, fetch? })` is a `Judge` (the `judge` provider) that asks Jev, built on pi-jev-compaction's client. Each question is one request: a yes-or-no question goes out as Jev's own question type and its probability is read from Jev's answer, a choice question's answer must be one of the options you listed, and a score question's probabilities are read by criterion index, so a missing index is an error. Probabilities must be numbers from 0 to 1.
90
+ Every failure throws a `JudgeError`: a failed request, an answer that is missing or malformed, a choice that is not an option, and a question that outlasts `timeoutMs` (default 20,000, a limit of its own for each question that also covers reading the answer), as `Jev did not answer within <n> ms`.
91
+ The `apiKey` is required, and an empty or blank one throws a `JudgeError` when `jevJudge` is called, because pi-jev-compaction's client would otherwise fall back to the key in the environment. `baseUrl` and `model` default to pi-jev-compaction's; `fetch` replaces the request function, for tests. The option type is `JevJudgeOptions`.
83
92
  - Effort: `effortJudge` picks a turn's thinking level from a message with your own brief (`EffortBrief`, `JUDGE_WORK`). A host that wants other turns decided differently sets `effortPolicy` (an `EffortPolicy`) in its configuration: it gets the turn's `channel`, `kind`, `agent` and `origin` and answers `{ fixed: level }` (no judge call), `{ requiredMass: [0, medium, high, xhigh] }` (the highest level whose share of the judge's answer at or above it meets its entry; the previous level does not carry over and a failed judge gives low), or `undefined` for the default. The `kind` of an agent-server agent's turn is `"agent"`; a policy that throws, or a `requiredMass` that is not four finite numbers in [0, 1], fails the turn.
84
93
  - Presentation and small helpers: `thinkingLine`, `zonedStamp(date, timeZone)`, `channelQueue()` (a queue of your own, so work does not wait behind a running turn), `checkRepoName` and `SKILL_LIST_TOOL` with `skillListExtension` for repositories and skills, and `searchTerms` for memory search.
85
94
 
@@ -909,7 +918,7 @@ defineTool({
909
918
 
910
919
  `shellHoldRule` in `pi-roundtable/kit` judges the agents' `bash`, `write` and `edit` calls; the agent server links it, and a session without a workspace has no shell.
911
920
  Its scratch roots are the shared workspace (`HoldContext.workspace`) and the scratch dir (`HoldContext.scratchDir`).
912
- The scratch dir is the config's `scratchDir`, by default `<os temp dir>/<discord.rootCommand>-scratch` (such as `/tmp/roundtable-scratch`); the agent server creates it with mode 0700 at startup and refuses one that is a symlink or another user's, and the agents' `bash` runs with `TMPDIR` pointing to it, so `mktemp` and tools write there.
921
+ The scratch dir is the config's `scratchDir`, by default `<os temp dir>/<discord.rootCommand>-scratch` (such as `/tmp/roundtable-scratch`); the agent server creates it with mode 0700 at startup and refuses one that is a symlink or another user's, and the agents' `bash` runs with `TMPDIR` pointing to it, so tools that read `TMPDIR` and GNU `mktemp` write there. macOS's BSD `mktemp` ignores `TMPDIR` for a bare `mktemp`; `mktemp -p "$TMPDIR"` works with both.
913
922
  `AgentSessions.scratchDir` carries it to a runtime, and the agents' prompt tells them to write temporary files there, and to show a file with `attach_file` or a `discord_send_message` file `path` rather than reading an image or base64-encoding it.
914
923
 
915
924
  - A `>`, `>>` or `tee` target, and a `write` or `edit` path, inside a scratch root run; anything else, the rest of `/tmp` included, is held.
@@ -1397,7 +1406,7 @@ When no `images` provider is configured, `agent_create` has no `avatar_prompt` p
1397
1406
  The owner's profile panel reports the missing provider and offers no redraw option.
1398
1407
  Each agent gets a picture generated from its display name and the assistant's icon, so you can tell agents apart.
1399
1408
  `bunx roundtable doctor` reports whether the slot is filled.
1400
- The copy-in [`codex-images` provider template][codex-images-template] is a complete implementation of the `images` slot.
1409
+ The copy-in [`codex-images` provider template][codex-images-template] fills the `images` slot with `createCodexImages` from `pi-roundtable/kit`, which also exports the client it is built on, `drawCodexImage`.
1401
1410
 
1402
1411
  <!-- example: examples/providers.ts -->
1403
1412
  ```ts
@@ -3160,6 +3169,10 @@ Install only the packages your host uses; the core does not depend on these work
3160
3169
  It sends requests to ChatGPT's Codex backend using the owner's ChatGPT subscription login.
3161
3170
  OpenAI doesn't document the backend for this use, so it can stop working without notice; the subscription's terms apply.
3162
3171
  The copied file begins with this warning.
3172
+ Its client and plugin come from `pi-roundtable/kit` (`createCodexImages`, built on `drawCodexImage`): the copied `plugins/codex-images.ts` imports them and makes the plugin you list, so a fix to the protocol arrives with an upgrade of the package, not with a new copy.
3173
+ A drawing aborts with the turn that asked for it, a request stops after five minutes, and a response stream over 32 MiB is refused.
3174
+ A copy made before 0.19.0 holds the whole client and keeps working as it was, without the abort and the stream limit, until you copy the template again.
3175
+ To change the model, pass `createCodexImages({ model })`; the default is `CODEX_IMAGE_MODEL`.
3163
3176
 
3164
3177
  `release-notice` listens to two [events](#events-hear-what-the-core-does).
3165
3178
  When the agent server's team service reports `ready`, it reads the release file and posts if the release's `sha` differs from the one it last announced (`announced-release` in the data directory), or if the previous shutdown cut work short.
@@ -3947,12 +3960,18 @@ Import from the entries listed below; source area files are internal.
3947
3960
  | `AgentTurnRunner` | `pi-roundtable/kit` | type |
3948
3961
  | `AssistantLike` | `pi-roundtable/kit` | type |
3949
3962
  | `Backlog` | `pi-roundtable/kit` | type |
3963
+ | `CODEX_IMAGES_PROVIDER` | `pi-roundtable/kit` | value |
3964
+ | `CODEX_IMAGE_MODEL` | `pi-roundtable/kit` | value |
3950
3965
  | `CardLimits` | `pi-roundtable/kit` | type |
3951
3966
  | `CategoryLayout` | `pi-roundtable/kit` | type |
3952
3967
  | `ChannelMessage` | `pi-roundtable/kit` | type |
3953
3968
  | `ChannelQueue` | `pi-roundtable/kit` | type |
3954
3969
  | `ChoiceAnswer` | `pi-roundtable/kit` | type |
3955
3970
  | `ChoiceQuestion` | `pi-roundtable/kit` | type |
3971
+ | `CodexFetch` | `pi-roundtable/kit` | type |
3972
+ | `CodexImage` | `pi-roundtable/kit` | type |
3973
+ | `CodexImageRequest` | `pi-roundtable/kit` | type |
3974
+ | `CodexImagesOptions` | `pi-roundtable/kit` | type |
3956
3975
  | `DELEGATE_TOOL` | `pi-roundtable/kit` | value |
3957
3976
  | `DELEGATE_TOOL_SPEC` | `pi-roundtable/kit` | value |
3958
3977
  | `DashboardBoard` | `pi-roundtable/kit` | type |
@@ -3968,7 +3987,9 @@ Import from the entries listed below; source area files are internal.
3968
3987
  | `EffortRequest` | `pi-roundtable/kit` | type |
3969
3988
  | `EffortRule` | `pi-roundtable/kit` | type |
3970
3989
  | `GroupMessage` | `pi-roundtable/kit` | type |
3990
+ | `ImageNotGeneratedError` | `pi-roundtable/kit` | value |
3971
3991
  | `JUDGE_WORK` | `pi-roundtable/kit` | value |
3992
+ | `JevJudgeOptions` | `pi-roundtable/kit` | type |
3972
3993
  | `McpEndpoint` | `pi-roundtable/kit` | type |
3973
3994
  | `ModelRef` | `pi-roundtable/kit` | type |
3974
3995
  | `OwnerIdentity` | `pi-roundtable/kit` | type |
@@ -3980,6 +4001,9 @@ Import from the entries listed below; source area files are internal.
3980
4001
  | `SCHEDULE_TOOLS` | `pi-roundtable/kit` | value |
3981
4002
  | `SHELL_TOOLS` | `pi-roundtable/kit` | value |
3982
4003
  | `SKILL_LIST_TOOL` | `pi-roundtable/kit` | value |
4004
+ | `createCodexImages` | `pi-roundtable/kit` | value |
4005
+ | `drawCodexImage` | `pi-roundtable/kit` | value |
4006
+ | `jevJudge` | `pi-roundtable/kit` | value |
3983
4007
  | `packageDir` | `pi-roundtable/kit` | value |
3984
4008
  | `serveUnix` | `pi-roundtable/kit` | value |
3985
4009
  | `ScheduleToolContext` | `pi-roundtable/kit` | type |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-roundtable",
3
- "version": "0.18.0",
3
+ "version": "0.19.1",
4
4
  "description": "A plugin-driven Pi agent server for one owner, reached through Discord or the web",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -225,6 +225,7 @@ export class HttpListeners {
225
225
  readonly #sockets: SocketSet;
226
226
  readonly #closeTimeoutMs: number;
227
227
  #servers: Pick<Server<unknown>, "stop">[] = [];
228
+ readonly #ports = new Map<string, number>();
228
229
 
229
230
  /** Validates every route before any socket opens; `closeTimeoutMs` bounds how long a stop waits on WebSocket handlers. */
230
231
  constructor(
@@ -271,9 +272,15 @@ export class HttpListeners {
271
272
  );
272
273
  chmodSync(listener.socketPath, listener.mode ?? 0o660);
273
274
  } else {
274
- this.#servers.push(
275
- serveTcp(listener.port, listener.hostname, handle, websocket),
275
+ const server = serveTcp(
276
+ listener.port,
277
+ listener.hostname,
278
+ handle,
279
+ websocket,
276
280
  );
281
+ this.#servers.push(server);
282
+ if (server.port !== undefined)
283
+ this.#ports.set(listener.id, server.port);
277
284
  }
278
285
  }
279
286
  } catch (error) {
@@ -282,6 +289,11 @@ export class HttpListeners {
282
289
  }
283
290
  }
284
291
 
292
+ /** The port an open TCP listener is bound to, which differs from the configured one when that was 0. */
293
+ portOf(listener: string): number | undefined {
294
+ return this.#ports.get(listener);
295
+ }
296
+
285
297
  /** The listener's WebSocket handler, sized for its largest route limits; none without WebSocket routes. */
286
298
  #websocketHandler(
287
299
  listener: string,
@@ -314,6 +326,7 @@ export class HttpListeners {
314
326
  this.#sockets.closeAll();
315
327
  for (const server of this.#servers) void server.stop(true);
316
328
  this.#servers = [];
329
+ this.#ports.clear();
317
330
  await this.#sockets.drain(this.#closeTimeoutMs);
318
331
  }
319
332
  }
@@ -0,0 +1,257 @@
1
+ // A client for ChatGPT's undocumented Codex backend, signed in with an openai-codex login, that draws one image.
2
+
3
+ import type { ReferenceImage } from "../contract/providers.ts";
4
+
5
+ const CODEX_RESPONSES_URL = "https://chatgpt.com/backend-api/codex/responses";
6
+ const JWT_CLAIM_PATH = "https://api.openai.com/auth";
7
+ const REQUEST_TIMEOUT_MS = 5 * 60_000;
8
+ const MAX_IMAGE_BYTES = 32 * 1024 * 1024;
9
+ const MAX_IMAGE_BASE64 = Math.ceil(MAX_IMAGE_BYTES / 3) * 4;
10
+
11
+ /** The Pi provider whose login pays for the images. */
12
+ export const CODEX_IMAGES_PROVIDER = "openai-codex";
13
+ /** Routes the Codex request; the backend picks the image model itself. */
14
+ export const CODEX_IMAGE_MODEL = "gpt-6-sol";
15
+
16
+ /**
17
+ * Codex ran the image tool but returned no image, or answered without one; usually it refused
18
+ * the prompt. Asking again with the same prompt fails again.
19
+ */
20
+ export class ImageNotGeneratedError extends Error {
21
+ override name = "ImageNotGeneratedError";
22
+ }
23
+
24
+ /** The part of `fetch` the client uses, so a test can stand in for the network. `typeof fetch` is assignable. */
25
+ export type CodexFetch = (url: string, init: RequestInit) => Promise<Response>;
26
+
27
+ export interface CodexImageRequest {
28
+ /** The openai-codex login's access token; read it again for each drawing, because the login refreshes. */
29
+ token: string;
30
+ prompt: string;
31
+ references?: readonly ReferenceImage[];
32
+ /** The model that routes the request; default `CODEX_IMAGE_MODEL`. */
33
+ model?: string;
34
+ /** Aborts the request and the read of its stream; the five-minute limit applies as well. */
35
+ signal?: AbortSignal;
36
+ /** The request function; default the global `fetch`. */
37
+ fetch?: CodexFetch;
38
+ }
39
+
40
+ export interface CodexImage {
41
+ /** PNG bytes. */
42
+ data: Uint8Array;
43
+ /** The prompt Codex says it drew, when it tells. */
44
+ revisedPrompt?: string;
45
+ }
46
+
47
+ /** What bounds one drawing; the core's tests lower these, the public client always uses the defaults. */
48
+ export interface CodexImageLimits {
49
+ timeoutMs: number;
50
+ maxStreamBytes: number;
51
+ }
52
+
53
+ export const CODEX_IMAGE_LIMITS: CodexImageLimits = {
54
+ timeoutMs: REQUEST_TIMEOUT_MS,
55
+ maxStreamBytes: MAX_IMAGE_BYTES,
56
+ };
57
+
58
+ /** The ChatGPT account the login token belongs to, from the token's claims. */
59
+ function chatGptAccountId(token: string): string {
60
+ const [, payload] = token.split(".");
61
+ if (!payload) throw new Error("the Codex token is not a JWT");
62
+ let claims: Record<string, unknown>;
63
+ try {
64
+ claims = JSON.parse(Buffer.from(payload, "base64url").toString("utf8"));
65
+ } catch {
66
+ throw new Error("the Codex token's claims are not JSON");
67
+ }
68
+ const auth = claims[JWT_CLAIM_PATH] as
69
+ | { chatgpt_account_id?: unknown }
70
+ | undefined;
71
+ const id = auth?.chatgpt_account_id;
72
+ if (typeof id !== "string" || !id)
73
+ throw new Error("the Codex token has no ChatGPT account");
74
+ return id;
75
+ }
76
+
77
+ function object(value: unknown): Record<string, unknown> {
78
+ return value !== null && typeof value === "object" && !Array.isArray(value)
79
+ ? (value as Record<string, unknown>)
80
+ : {};
81
+ }
82
+
83
+ /** The one image an `image_generation_call` item carries; anything else is not an image. */
84
+ function imageOf(value: unknown): CodexImage | undefined {
85
+ const item = object(value);
86
+ if (item.type !== "image_generation_call") return undefined;
87
+ if (
88
+ item.status !== "completed" ||
89
+ typeof item.result !== "string" ||
90
+ !item.result
91
+ ) {
92
+ const { result: _result, ...detail } = item;
93
+ throw new ImageNotGeneratedError(
94
+ `Codex image generation did not complete (status ${String(item.status)}): ${JSON.stringify(detail).slice(0, 300)}`,
95
+ );
96
+ }
97
+ if (item.result.length > MAX_IMAGE_BASE64)
98
+ throw new Error("the image exceeds 32 MiB");
99
+ const data = new Uint8Array(Buffer.from(item.result, "base64"));
100
+ return typeof item.revised_prompt === "string" && item.revised_prompt
101
+ ? { data, revisedPrompt: item.revised_prompt }
102
+ : { data };
103
+ }
104
+
105
+ /** The frames of a server-sent event stream: the text between blank lines. */
106
+ async function* frames(
107
+ body: ReadableStream<Uint8Array>,
108
+ signal: AbortSignal,
109
+ maxBytes: number,
110
+ ): AsyncGenerator<string> {
111
+ const decoder = new TextDecoder();
112
+ let buffer = "";
113
+ let read = 0;
114
+ for await (const chunk of body) {
115
+ signal.throwIfAborted();
116
+ read += chunk.byteLength;
117
+ if (read > maxBytes) throw new Error("Codex image response too large");
118
+ buffer += decoder.decode(chunk, { stream: true });
119
+ let match = /\r?\n\r?\n/.exec(buffer);
120
+ while (match) {
121
+ yield buffer.slice(0, match.index);
122
+ buffer = buffer.slice(match.index + match[0].length);
123
+ match = /\r?\n\r?\n/.exec(buffer);
124
+ }
125
+ }
126
+ if (buffer.trim()) yield buffer;
127
+ }
128
+
129
+ /** The event a frame carries, or undefined for a frame without data. */
130
+ function eventOf(frame: string): Record<string, unknown> | undefined {
131
+ const data = frame
132
+ .split(/\r?\n/)
133
+ .flatMap((line) => (line.startsWith("data:") ? [line.slice(5).trim()] : []))
134
+ .join("\n");
135
+ if (!data || data === "[DONE]") return undefined;
136
+ try {
137
+ return object(JSON.parse(data));
138
+ } catch {
139
+ throw new Error(`Codex sent a malformed event: ${data.slice(0, 200)}`);
140
+ }
141
+ }
142
+
143
+ /** Reads the events until the response completes, and returns its first image. */
144
+ async function parseImageStream(
145
+ response: Response,
146
+ signal: AbortSignal,
147
+ maxBytes: number,
148
+ ): Promise<CodexImage> {
149
+ if (!response.body) throw new Error("Codex returned no body");
150
+ let image: CodexImage | undefined;
151
+ for await (const frame of frames(response.body, signal, maxBytes)) {
152
+ const event = eventOf(frame);
153
+ switch (event?.type) {
154
+ case "error":
155
+ case "response.failed": {
156
+ const error = object(object(event.response).error ?? event.error);
157
+ throw new Error(
158
+ `Codex failed: ${String(error.message ?? error.code ?? "unknown error")}`,
159
+ );
160
+ }
161
+ case "response.incomplete":
162
+ throw new Error("the Codex response was incomplete");
163
+ case "response.output_item.done":
164
+ image ??= imageOf(event.item);
165
+ break;
166
+ case "response.completed": {
167
+ const output = object(event.response).output;
168
+ if (Array.isArray(output))
169
+ for (const item of output) image ??= imageOf(item);
170
+ if (!image)
171
+ throw new ImageNotGeneratedError("Codex answered without an image");
172
+ return image;
173
+ }
174
+ default:
175
+ break;
176
+ }
177
+ }
178
+ throw new Error("the Codex stream ended before completion");
179
+ }
180
+
181
+ async function send(
182
+ request: CodexImageRequest,
183
+ signal: AbortSignal,
184
+ maxBytes: number,
185
+ ): Promise<CodexImage> {
186
+ const fetchImpl = request.fetch ?? fetch;
187
+ const response = await fetchImpl(CODEX_RESPONSES_URL, {
188
+ method: "POST",
189
+ redirect: "error",
190
+ signal,
191
+ headers: {
192
+ authorization: `Bearer ${request.token}`,
193
+ "chatgpt-account-id": chatGptAccountId(request.token),
194
+ originator: "pi",
195
+ "openai-beta": "responses=experimental",
196
+ accept: "text/event-stream",
197
+ "content-type": "application/json",
198
+ },
199
+ body: JSON.stringify({
200
+ model: request.model ?? CODEX_IMAGE_MODEL,
201
+ store: false,
202
+ stream: true,
203
+ instructions:
204
+ "You are generating bitmap image assets. For this request, call the image_generation tool exactly once. Do not answer with only text unless image generation is unavailable.",
205
+ input: [
206
+ {
207
+ role: "user",
208
+ content: [
209
+ { type: "input_text", text: request.prompt },
210
+ ...(request.references ?? []).map((image) => ({
211
+ type: "input_image",
212
+ image_url: `data:${image.mimeType};base64,${image.data}`,
213
+ })),
214
+ ],
215
+ },
216
+ ],
217
+ tools: [{ type: "image_generation", output_format: "png" }],
218
+ tool_choice: "auto",
219
+ parallel_tool_calls: false,
220
+ text: { verbosity: "low" },
221
+ }),
222
+ });
223
+ if (!response.ok) {
224
+ const detail = (await response.text()).slice(0, 300);
225
+ throw new Error(`Codex answered ${response.status}: ${detail}`);
226
+ }
227
+ return parseImageStream(response, signal, maxBytes);
228
+ }
229
+
230
+ /** `drawCodexImage` with its limits given, for the core's tests. */
231
+ export async function requestCodexImage(
232
+ request: CodexImageRequest,
233
+ limits: CodexImageLimits,
234
+ ): Promise<CodexImage> {
235
+ const timeout = AbortSignal.timeout(limits.timeoutMs);
236
+ const signal = request.signal
237
+ ? AbortSignal.any([request.signal, timeout])
238
+ : timeout;
239
+ signal.throwIfAborted();
240
+ try {
241
+ return await send(request, signal, limits.maxStreamBytes);
242
+ } catch (error) {
243
+ // The abort reaches the caller as its own reason, not as whatever the stream wrapped it in.
244
+ if (signal.aborted) throw signal.reason;
245
+ throw error;
246
+ }
247
+ }
248
+
249
+ /**
250
+ * Draws one image through the Codex backend. It aborts with `request.signal`, before the response
251
+ * and while its stream is read, stops after five minutes, and refuses a stream over 32 MiB.
252
+ */
253
+ export function drawCodexImage(
254
+ request: CodexImageRequest,
255
+ ): Promise<CodexImage> {
256
+ return requestCodexImage(request, CODEX_IMAGE_LIMITS);
257
+ }
@@ -0,0 +1,76 @@
1
+ // The codex-images plugin: fills the images slot with drawCodexImage.
2
+
3
+ import { definePlugin } from "../define.ts";
4
+ import { PluginError } from "../errors.ts";
5
+ import type { RoundtablePlugin } from "../plugin.ts";
6
+ import {
7
+ CODEX_IMAGE_MODEL,
8
+ CODEX_IMAGES_PROVIDER,
9
+ type CodexFetch,
10
+ drawCodexImage,
11
+ } from "./codex-image.ts";
12
+
13
+ export interface CodexImagesOptions {
14
+ /** The request function; default the global `fetch`. */
15
+ fetch?: CodexFetch;
16
+ /** The model that routes the request; default `CODEX_IMAGE_MODEL`. */
17
+ model?: string;
18
+ /**
19
+ * What setup does when the openai-codex login is missing or unusable: "throw" (the default)
20
+ * stops the host; "warn" logs a warning, and a drawing then fails with the same PluginError.
21
+ */
22
+ onMissingLogin?: "throw" | "warn";
23
+ }
24
+
25
+ const NO_LOGIN = `the host has no login for ${CODEX_IMAGES_PROVIDER}, the provider that pays for the images. Log in to the ${CODEX_IMAGES_PROVIDER} provider with Pi so the agent directory's auth.json holds it, or remove this plugin.`;
26
+
27
+ /**
28
+ * The `codex-images` plugin. It fills `providers.images`, reads the login's token for each drawing
29
+ * through `context.apiKey`, and aborts a drawing with the turn that asked for it.
30
+ */
31
+ export function createCodexImages(
32
+ options: CodexImagesOptions = {},
33
+ ): RoundtablePlugin {
34
+ const {
35
+ fetch: fetchImpl,
36
+ model = CODEX_IMAGE_MODEL,
37
+ onMissingLogin = "throw",
38
+ } = options;
39
+ let apiKey: ((provider: string) => Promise<string | undefined>) | undefined;
40
+ // The login refreshes its token, so each image reads it again instead of keeping the first.
41
+ const token = async (): Promise<string> => {
42
+ const value = await apiKey?.(CODEX_IMAGES_PROVIDER);
43
+ if (!value) throw new PluginError(`plugin codex-images: ${NO_LOGIN}`);
44
+ return value;
45
+ };
46
+ return definePlugin({
47
+ name: "codex-images",
48
+ providers: {
49
+ // The images slot draws an agent's avatar from a prompt and reference pictures.
50
+ images: async (prompt, references, drawing) =>
51
+ (
52
+ await drawCodexImage({
53
+ token: await token(),
54
+ prompt,
55
+ references,
56
+ model,
57
+ signal: drawing?.signal,
58
+ fetch: fetchImpl,
59
+ })
60
+ ).data,
61
+ },
62
+ async setup(context) {
63
+ apiKey = context.apiKey;
64
+ try {
65
+ await token();
66
+ } catch (error) {
67
+ if (onMissingLogin !== "warn" || !(error instanceof PluginError))
68
+ throw error;
69
+ context.logger.warn(
70
+ `${NO_LOGIN} Drawing images fails until the login exists.`,
71
+ );
72
+ }
73
+ return {};
74
+ },
75
+ });
76
+ }
@@ -472,7 +472,7 @@ export function imageTools(
472
472
  return images ? { generateImage: generateImageExtension(images, roots) } : {};
473
473
  }
474
474
 
475
- /** Pi's bash tool with TMPDIR set to `scratchDir`, so `mktemp` and tools write there. */
475
+ /** Pi's bash tool with TMPDIR set to `scratchDir`, so tools that read TMPDIR and GNU `mktemp` write there (BSD `mktemp` needs `-p "$TMPDIR"`). */
476
476
  export function scratchBash(cwd: string, scratchDir: string): ToolDefinition {
477
477
  // SAFETY: Pi's bash definition is a ToolDefinition typed by its own parameter schema, which the session takes as any custom tool.
478
478
  return createBashToolDefinition(cwd, {
@@ -0,0 +1,15 @@
1
+ // Plugin helpers, versioned like the main entry; see the plugin guide.
2
+
3
+ export type {
4
+ CodexFetch,
5
+ CodexImage,
6
+ CodexImageRequest,
7
+ } from "../core/images/codex-image.ts";
8
+ export {
9
+ CODEX_IMAGE_MODEL,
10
+ CODEX_IMAGES_PROVIDER,
11
+ drawCodexImage,
12
+ ImageNotGeneratedError,
13
+ } from "../core/images/codex-image.ts";
14
+ export type { CodexImagesOptions } from "../core/images/codex-images-plugin.ts";
15
+ export { createCodexImages } from "../core/images/codex-images-plugin.ts";
package/src/kit/index.ts CHANGED
@@ -56,6 +56,19 @@ export {
56
56
  thinkingLabel,
57
57
  } from "./domain.ts";
58
58
  export { HOLD_DESCRIBE_TIMEOUT_MS, holdChain } from "./holds.ts";
59
+ export type {
60
+ CodexFetch,
61
+ CodexImage,
62
+ CodexImageRequest,
63
+ CodexImagesOptions,
64
+ } from "./images.ts";
65
+ export {
66
+ CODEX_IMAGE_MODEL,
67
+ CODEX_IMAGES_PROVIDER,
68
+ createCodexImages,
69
+ drawCodexImage,
70
+ ImageNotGeneratedError,
71
+ } from "./images.ts";
59
72
  export type {
60
73
  JevCompactInput,
61
74
  JevCompactOptions,
@@ -74,6 +87,8 @@ export {
74
87
  jevCompactionExtension,
75
88
  jevCompactor,
76
89
  } from "./jev.ts";
90
+ export type { JevJudgeOptions } from "./jev-judge.ts";
91
+ export { jevJudge } from "./jev-judge.ts";
77
92
  export type {
78
93
  EffortBrief,
79
94
  EffortJudgeOptions,
@@ -0,0 +1,153 @@
1
+ // A Judge over Jev's hosted service, built on pi-jev-compaction's client.
2
+
3
+ import { JevClient, type JevQuestions } from "pi-jev-compaction";
4
+ import type {
5
+ ChoiceAnswer,
6
+ ChoiceQuestion,
7
+ Judge,
8
+ ScoreQuestion,
9
+ YesNoQuestion,
10
+ } from "../core/contract/providers.ts";
11
+ import { JudgeError } from "../core/errors.ts";
12
+
13
+ /** The service's name for the yes-or-no question and for its answer's probability. */
14
+ const WIRE_YES_NO = "noul";
15
+ type JevQuestion = JevQuestions[string];
16
+ const DEFAULT_TIMEOUT_MS = 20_000;
17
+
18
+ export interface JevJudgeOptions {
19
+ /** The service's API key. It is required: an empty one is refused, not looked up in the environment. */
20
+ apiKey: string;
21
+ /** The service's model; default pi-jev-compaction's. */
22
+ model?: string;
23
+ /** The service's endpoint; default pi-jev-compaction's. */
24
+ baseUrl?: string;
25
+ /** How long one question may take, the read of the answer included; default 20,000. */
26
+ timeoutMs?: number;
27
+ /** The request function; default the global `fetch`. */
28
+ fetch?: typeof fetch;
29
+ }
30
+
31
+ // pi-lens-ignore: no-unknown-parameters — a type guard over parsed JSON
32
+ const isRecord = (value: unknown): value is Record<string, unknown> =>
33
+ typeof value === "object" && value !== null && !Array.isArray(value);
34
+
35
+ // pi-lens-ignore: no-unknown-parameters — a type guard over parsed JSON
36
+ const isProbability = (value: unknown): value is number =>
37
+ typeof value === "number" && value >= 0 && value <= 1;
38
+
39
+ /**
40
+ * A `Judge` that asks Jev. Each question is one request with its own time limit of `timeoutMs`,
41
+ * which also covers reading the answer. Every failure throws a `JudgeError`: a missing answer,
42
+ * a probability outside 0 to 1, a choice that is not one of the listed options, a failed request,
43
+ * and a limit that ran out. An empty `apiKey` is refused when the judge is made.
44
+ */
45
+ export function jevJudge(options: JevJudgeOptions): Judge {
46
+ const apiKey = options.apiKey.trim();
47
+ if (!apiKey)
48
+ throw new JudgeError(
49
+ "jevJudge needs an apiKey; an empty one would fall back to the environment's key",
50
+ );
51
+ const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
52
+
53
+ async function ask(
54
+ state: Record<string, unknown>,
55
+ questions: JevQuestions,
56
+ ): Promise<Record<string, unknown>> {
57
+ const signal = AbortSignal.timeout(timeoutMs);
58
+ const client = new JevClient({
59
+ apiKey,
60
+ model: options.model,
61
+ baseUrl: options.baseUrl,
62
+ fetch: options.fetch,
63
+ signal,
64
+ });
65
+ try {
66
+ return (await client.ask(state, questions)).answers;
67
+ } catch (error) {
68
+ if (error instanceof JudgeError) throw error;
69
+ if (signal.aborted)
70
+ throw new JudgeError(`Jev did not answer within ${timeoutMs} ms`, {
71
+ cause: error,
72
+ });
73
+ const message = error instanceof Error ? error.message : String(error);
74
+ throw new JudgeError(
75
+ /^Jev/.test(message) ? message : `Jev request failed: ${message}`,
76
+ { cause: error },
77
+ );
78
+ }
79
+ }
80
+
81
+ const malformed = (name: string, detail = "") =>
82
+ new JudgeError(`Jev's answer for ${name} is malformed${detail}`);
83
+
84
+ return {
85
+ async askYesNo(
86
+ state: Record<string, unknown>,
87
+ questions: Record<string, YesNoQuestion>,
88
+ ): Promise<Record<string, number>> {
89
+ const names = Object.keys(questions);
90
+ if (names.length === 0) return {};
91
+ const answers = await ask(
92
+ state,
93
+ Object.fromEntries(
94
+ names.map((name) => {
95
+ const { type: _type, ...question } = questions[
96
+ name
97
+ ] as YesNoQuestion;
98
+ return [name, { ...question, type: WIRE_YES_NO } as JevQuestion];
99
+ }),
100
+ ),
101
+ );
102
+ // pi-lens-ignore: no-known-value-widening — filled per question name below
103
+ const result: Record<string, number> = {};
104
+ for (const name of names) {
105
+ const answer = answers[name];
106
+ const probability = isRecord(answer) ? answer[WIRE_YES_NO] : undefined;
107
+ if (!isProbability(probability)) throw malformed(name);
108
+ result[name] = probability;
109
+ }
110
+ return result;
111
+ },
112
+
113
+ async askChoice(
114
+ state: Record<string, unknown>,
115
+ name: string,
116
+ question: ChoiceQuestion,
117
+ ): Promise<ChoiceAnswer> {
118
+ const answer = (await ask(state, { [name]: question }))[name];
119
+ if (
120
+ !isRecord(answer) ||
121
+ typeof answer.choice !== "string" ||
122
+ !isProbability(answer.confidence)
123
+ )
124
+ throw malformed(name);
125
+ if (!Object.hasOwn(question.criteria, answer.choice))
126
+ throw new JudgeError(
127
+ `Jev chose ${answer.choice}, which is not an option of ${name}`,
128
+ );
129
+ return { choice: answer.choice, confidence: answer.confidence };
130
+ },
131
+
132
+ async askScore(
133
+ state: Record<string, unknown>,
134
+ name: string,
135
+ question: ScoreQuestion,
136
+ ): Promise<number[]> {
137
+ if (question.criteria.length === 0) return [];
138
+ const answer = (
139
+ await ask(state, {
140
+ [name]: { ...question, criteria: [...question.criteria] },
141
+ })
142
+ )[name];
143
+ const probabilities = isRecord(answer) ? answer.probabilities : undefined;
144
+ if (!isRecord(probabilities)) throw malformed(name);
145
+ return question.criteria.map((_, index) => {
146
+ const value = probabilities[String(index)];
147
+ if (!isProbability(value))
148
+ throw malformed(name, `: no probability for criterion ${index}`);
149
+ return value;
150
+ });
151
+ },
152
+ };
153
+ }
@@ -107,6 +107,37 @@ test("codex-images reports an HTTP error with its status, and never the token",
107
107
  await harness.stop();
108
108
  });
109
109
 
110
+ test("codex-images aborts a drawing with the turn that asked for it", async () => {
111
+ const signals: AbortSignal[] = [];
112
+ const plugin = createCodexImages({
113
+ fetch: (_url, init) => {
114
+ const signal = init.signal as AbortSignal;
115
+ signals.push(signal);
116
+ return new Promise((_resolve, reject) => {
117
+ signal.addEventListener("abort", () => reject(signal.reason));
118
+ });
119
+ },
120
+ });
121
+ const harness = await testPlugin(plugin, {
122
+ apiKeys: { "openai-codex": token("acct-1") },
123
+ });
124
+ const turn = new AbortController();
125
+ const reason = new Error("the turn ended");
126
+ const drawing = plugin.providers?.images?.("a fox", [], {
127
+ signal: turn.signal,
128
+ });
129
+ const outcome = drawing?.then(
130
+ () => undefined,
131
+ (error: unknown) => error,
132
+ );
133
+ await new Promise((done) => setTimeout(done, 20));
134
+ expect(signals[0]?.aborted).toBe(false);
135
+ turn.abort(reason);
136
+ expect(await outcome).toBe(reason);
137
+ expect(signals[0]?.aborted).toBe(true);
138
+ await harness.stop();
139
+ });
140
+
110
141
  test("codex-images refuses to start without an openai-codex login", async () => {
111
142
  const failure = await testPlugin(codexImages).then(
112
143
  () => undefined,
@@ -3,238 +3,14 @@
3
3
  // as an API, so it can stop working without notice, and OpenAI's terms for the subscription
4
4
  // apply to it. Use it only where you accept that risk.
5
5
 
6
- import { definePlugin, PluginError, type ReferenceImage } from "pi-roundtable";
6
+ import { createCodexImages } from "pi-roundtable/kit";
7
7
 
8
- const CODEX_RESPONSES_URL = "https://chatgpt.com/backend-api/codex/responses";
9
- /** The Pi provider whose login pays for the images. */
10
- const CODEX_PROVIDER = "openai-codex";
11
- /** Routes the Codex request; the backend picks the image model itself. */
12
- const CODEX_MODEL = "gpt-6-sol";
13
- const JWT_CLAIM_PATH = "https://api.openai.com/auth";
14
- const REQUEST_TIMEOUT_MS = 5 * 60_000;
15
- const MAX_IMAGE_BYTES = 32 * 1024 * 1024;
16
- const MAX_IMAGE_BASE64 = Math.ceil(MAX_IMAGE_BYTES / 3) * 4;
17
-
18
- /**
19
- * Codex ran the image tool but returned no image, usually because it refused the prompt.
20
- * Asking again with the same prompt fails again.
21
- */
22
- export class ImageNotGeneratedError extends Error {
23
- override name = "ImageNotGeneratedError";
24
- }
25
-
26
- /** The part of `fetch` the plugin uses, so a test can stand in for the network. */
27
- export type CodexFetch = (url: string, init: RequestInit) => Promise<Response>;
28
-
29
- export interface CodexImagesOptions {
30
- /** The request function; default the global `fetch`. */
31
- fetch?: CodexFetch;
32
- /** The model that routes the request; default `gpt-6-sol`. */
33
- model?: string;
34
- /**
35
- * What setup does when the openai-codex login is missing or unusable: "throw" (the default)
36
- * stops the host; "warn" logs a warning, and a drawing then fails with the same PluginError.
37
- */
38
- onMissingLogin?: "throw" | "warn";
39
- }
40
-
41
- /** The ChatGPT account the login token belongs to, from the token's claims. */
42
- function chatGptAccountId(token: string): string {
43
- const [, payload] = token.split(".");
44
- if (!payload) throw new Error("the Codex token is not a JWT");
45
- let claims: Record<string, unknown>;
46
- try {
47
- claims = JSON.parse(Buffer.from(payload, "base64url").toString("utf8"));
48
- } catch {
49
- throw new Error("the Codex token's claims are not JSON");
50
- }
51
- const auth = claims[JWT_CLAIM_PATH] as
52
- | { chatgpt_account_id?: unknown }
53
- | undefined;
54
- const id = auth?.chatgpt_account_id;
55
- if (typeof id !== "string" || !id)
56
- throw new Error("the Codex token has no ChatGPT account");
57
- return id;
58
- }
59
-
60
- function object(value: unknown): Record<string, unknown> {
61
- return value !== null && typeof value === "object" && !Array.isArray(value)
62
- ? (value as Record<string, unknown>)
63
- : {};
64
- }
65
-
66
- /** The one image an `image_generation_call` item carries; anything else is not an image. */
67
- function imageOf(value: unknown): Uint8Array | undefined {
68
- const item = object(value);
69
- if (item.type !== "image_generation_call") return undefined;
70
- if (
71
- item.status !== "completed" ||
72
- typeof item.result !== "string" ||
73
- !item.result
74
- ) {
75
- const { result: _result, ...detail } = item;
76
- throw new ImageNotGeneratedError(
77
- `Codex image generation did not complete (status ${String(item.status)}): ${JSON.stringify(detail).slice(0, 300)}`,
78
- );
79
- }
80
- if (item.result.length > MAX_IMAGE_BASE64)
81
- throw new Error("the image exceeds 32 MiB");
82
- return new Uint8Array(Buffer.from(item.result, "base64"));
83
- }
84
-
85
- /** The frames of a server-sent event stream: the text between blank lines. */
86
- async function* frames(
87
- body: ReadableStream<Uint8Array>,
88
- ): AsyncGenerator<string> {
89
- const decoder = new TextDecoder();
90
- let buffer = "";
91
- for await (const chunk of body) {
92
- buffer += decoder.decode(chunk, { stream: true });
93
- let match = /\r?\n\r?\n/.exec(buffer);
94
- while (match) {
95
- yield buffer.slice(0, match.index);
96
- buffer = buffer.slice(match.index + match[0].length);
97
- match = /\r?\n\r?\n/.exec(buffer);
98
- }
99
- }
100
- if (buffer.trim()) yield buffer;
101
- }
102
-
103
- /** The event a frame carries, or undefined for a frame without data. */
104
- function eventOf(frame: string): Record<string, unknown> | undefined {
105
- const data = frame
106
- .split(/\r?\n/)
107
- .flatMap((line) => (line.startsWith("data:") ? [line.slice(5).trim()] : []))
108
- .join("\n");
109
- if (!data || data === "[DONE]") return undefined;
110
- try {
111
- return object(JSON.parse(data));
112
- } catch {
113
- throw new Error(`Codex sent a malformed event: ${data.slice(0, 200)}`);
114
- }
115
- }
116
-
117
- /** Reads the events until the response completes, and returns its one image. */
118
- async function parseImageStream(response: Response): Promise<Uint8Array> {
119
- if (!response.body) throw new Error("Codex returned no body");
120
- let image: Uint8Array | undefined;
121
- for await (const frame of frames(response.body)) {
122
- const event = eventOf(frame);
123
- switch (event?.type) {
124
- case "error":
125
- case "response.failed": {
126
- const error = object(object(event.response).error ?? event.error);
127
- throw new Error(
128
- `Codex failed: ${String(error.message ?? error.code ?? "unknown error")}`,
129
- );
130
- }
131
- case "response.incomplete":
132
- throw new Error("the Codex response was incomplete");
133
- case "response.output_item.done":
134
- image ??= imageOf(event.item);
135
- break;
136
- case "response.completed": {
137
- const output = object(event.response).output;
138
- if (Array.isArray(output))
139
- for (const item of output) image ??= imageOf(item);
140
- if (!image)
141
- throw new ImageNotGeneratedError("Codex answered without an image");
142
- return image;
143
- }
144
- default:
145
- break;
146
- }
147
- }
148
- throw new Error("the Codex stream ended before completion");
149
- }
150
-
151
- async function generateImage(
152
- prompt: string,
153
- references: readonly ReferenceImage[],
154
- token: string,
155
- model: string,
156
- fetchImpl: CodexFetch,
157
- ): Promise<Uint8Array> {
158
- const response = await fetchImpl(CODEX_RESPONSES_URL, {
159
- method: "POST",
160
- redirect: "error",
161
- signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
162
- headers: {
163
- authorization: `Bearer ${token}`,
164
- "chatgpt-account-id": chatGptAccountId(token),
165
- originator: "pi",
166
- "openai-beta": "responses=experimental",
167
- accept: "text/event-stream",
168
- "content-type": "application/json",
169
- },
170
- body: JSON.stringify({
171
- model,
172
- store: false,
173
- stream: true,
174
- instructions:
175
- "You are generating bitmap image assets. For this request, call the image_generation tool exactly once. Do not answer with only text unless image generation is unavailable.",
176
- input: [
177
- {
178
- role: "user",
179
- content: [
180
- { type: "input_text", text: prompt },
181
- ...references.map((image) => ({
182
- type: "input_image",
183
- image_url: `data:${image.mimeType};base64,${image.data}`,
184
- })),
185
- ],
186
- },
187
- ],
188
- tools: [{ type: "image_generation", output_format: "png" }],
189
- tool_choice: "auto",
190
- parallel_tool_calls: false,
191
- text: { verbosity: "low" },
192
- }),
193
- });
194
- if (!response.ok) {
195
- const detail = (await response.text()).slice(0, 300);
196
- throw new Error(`Codex answered ${response.status}: ${detail}`);
197
- }
198
- return parseImageStream(response);
199
- }
200
-
201
- const NO_LOGIN = `the host has no login for ${CODEX_PROVIDER}, the provider that pays for the images. Log in to the ${CODEX_PROVIDER} provider with Pi so the agent directory's auth.json holds it, or remove this plugin.`;
202
-
203
- /** The plugin, with its request function and model replaceable for a test. */
204
- export function createCodexImages(options: CodexImagesOptions = {}) {
205
- const {
206
- fetch: fetchImpl = fetch,
207
- model = CODEX_MODEL,
208
- onMissingLogin = "throw",
209
- } = options;
210
- let apiKey: ((provider: string) => Promise<string | undefined>) | undefined;
211
- // The login refreshes its token, so each image reads it again instead of keeping the first.
212
- const token = async (): Promise<string> => {
213
- const value = await apiKey?.(CODEX_PROVIDER);
214
- if (!value) throw new PluginError(`plugin codex-images: ${NO_LOGIN}`);
215
- return value;
216
- };
217
- return definePlugin({
218
- name: "codex-images",
219
- providers: {
220
- // The images slot draws an agent's avatar from a prompt and reference pictures.
221
- images: async (prompt, references) =>
222
- generateImage(prompt, references, await token(), model, fetchImpl),
223
- },
224
- async setup(context) {
225
- apiKey = context.apiKey;
226
- try {
227
- await token();
228
- } catch (error) {
229
- if (onMissingLogin !== "warn" || !(error instanceof PluginError))
230
- throw error;
231
- context.logger.warn(
232
- `${NO_LOGIN} Drawing images fails until the login exists.`,
233
- );
234
- }
235
- return {};
236
- },
237
- });
238
- }
8
+ export {
9
+ type CodexFetch,
10
+ type CodexImagesOptions,
11
+ createCodexImages,
12
+ ImageNotGeneratedError,
13
+ } from "pi-roundtable/kit";
239
14
 
15
+ /** The plugin the project lists. The client and the plugin are the kit's; this file is yours to change. */
240
16
  export const codexImages = createCodexImages();