agello 0.2.1 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,15 +5,16 @@
5
5
  - Chat with the agent from a web page; terminal input, browser input and agent replies are shown as separate bubbles
6
6
  - Live session status, tool activity, and prompts queued while the agent is busy (same order as the terminal)
7
7
  - Conversation saved per pane in the browser, kept across reloads
8
+ - Interactive terminal view of the existing herdr pane: live cursor, keyboard input, and automatic resize
8
9
  - Optional live view of a `terminal-browser` screen, with an on/off switch to control it yourself
9
10
  - Embeddable `<agent-bridge>` web component
10
11
 
11
- Currently supports **Claude Code** sessions.
12
+ Chat currently supports **Claude Code** sessions. The terminal view connects directly to the pane, including ordinary shells and other terminal applications.
12
13
 
13
14
  ## Requirements
14
15
 
15
16
  - [Bun](https://bun.sh) >= 1.1
16
- - [herdr](https://herdr.dev) (the agent must run inside a herdr pane)
17
+ - [herdr](https://herdr.dev) (the agent must run inside a herdr pane); interactive terminal view requires `herdr terminal session control`
17
18
  - Optional: `terminal-browser` for the screen panel
18
19
 
19
20
  ## Install
@@ -69,13 +70,25 @@ The approve / reject buttons send `action=approve` / `action=reject`.
69
70
  | `agello open [--port N \| --pane ID]` | Open the page (default: this pane's server) |
70
71
  | `agello present [x,y,w,h] [--port N \| --pane ID]` | Presentation mode (see below) |
71
72
  | `agello present stop` | End presentation mode |
72
- | `agello script load <file> \| show` | Load / show a presentation script |
73
- | `agello present resume \| pause \| goto <n[.m]>` | Play the script from where it stopped / stop it / move to a step |
74
73
 
75
74
  `start` options: `--pane <id>`, `--port <n>` (default: first free from 8765), `--session <id>`, `--browser <terminal-browser key>`, `--allow-origin <origin>` (repeatable), `--open`, `--foreground`.
76
75
 
77
76
  State and logs: `~/.local/state/agello/<port>.json`, `<port>.log`.
78
77
 
78
+ ### Pane picker
79
+
80
+ Click the status in the header to open a drawer with herdr's workspace -> tab -> pane tree (the current pane's workspace is expanded; a workspace with a single tab lists its panes directly). Search filters by workspace, title, folder, or agent. Picking a pane switches the page to that pane's server, starting it if needed (`?server=` keeps the choice across reloads). Panes without Claude Code open as a terminal, since chat supports Claude Code only.
81
+
82
+ `+` adds a workspace (top), a tab (workspace row), or splits a pane (pane row); new panes open as a terminal. Nothing can be closed from the page: closing a herdr pane ends its processes.
83
+
84
+ ### Interactive terminal
85
+
86
+ Click the terminal icon in the header to replace the chat with the pane's live terminal (the screen panel stays). Keyboard input (including Korean text, arrow keys, and Ctrl+C) goes directly to that terminal. The viewport follows the browser size, and the mouse wheel scrolls through Herdr. Click the icon again or close the page to release control; the pane and its process keep running. If disconnected, use **다시 연결** to restore the current screen.
87
+
88
+ Only one browser connection can control a pane through this server at a time. Agello never forcibly takes over another Herdr controller. While connected, resizing the browser also changes the source terminal size. Terminal input is raw input, without the chat's `[browser]` prefix.
89
+
90
+ The xterm.js renderer and its styles are served locally; no terminal CDN is required. Herdr sends an initial ANSI screen and subsequent frame updates over the WebSocket, including cursor state. Terminal frames are not stored in browser chat history.
91
+
79
92
  ### Presentation mode
80
93
 
81
94
  `agello present x,y,w,h` (CSS px of the browser's visible viewport; omit for the whole screen):
@@ -89,23 +102,6 @@ State and logs: `~/.local/state/agello/<port>.json`, `<port>.log`.
89
102
 
90
103
  Run `present` again to move the crop. User control is off while presenting.
91
104
 
92
- #### Scripts
93
-
94
- Write the talk ahead so each line appears the moment its screen does:
95
-
96
- ```json
97
- { "rect": "28,157,688,477",
98
- "steps": [
99
- { "go": "#1", "say": ["First line", "Second line"] },
100
- { "go": "#2", "say": ["..."], "hold": 6 } ] }
101
- ```
102
-
103
- - step: `go` brings the screen there (`#n` sets `location.hash`, anything else is JS run in the page; agello knows nothing about the deck), `say` lines are one bubble each, `hold` seconds a line stays on screen (default from length, 10 to 20s; never below 10s). Pacing: screen change, 1s, line, fade out, 0.3s, next line; after a step's last line 0.7s before the next screen
104
- - `present resume` plays from where it stopped (starts presentation mode with the script's `rect`); it first re-runs the current step's `go`, so the agent may move the screen freely while answering
105
- - raising a hand pauses at once; the agent gets `action=hand-raise` with the position (`마지막 표시 2.1 · 다음 2.2`)
106
- - `script load` again after editing keeps the position; `present goto 2.2` then `present resume` replays from a fixed line
107
- - at the end the agent gets `action=present-done`; while the script is paused and the agent is working, the page shows a small loader
108
-
109
105
  ## Embed
110
106
 
111
107
  ```html
@@ -129,11 +125,17 @@ See `web/embed-example.html`.
129
125
  - **Send**: `herdr agent prompt <pane> "<text>"` (kept to 3 lines so Claude Code does not treat it as a paste)
130
126
  - **Status**: `herdr agent get` / `herdr pane get`, polled every 1.5s
131
127
  - **Chat**: tails the Claude Code transcript (`$CLAUDE_CONFIG_DIR/projects/*/<session>.jsonl`) and streams it over SSE
128
+ - **Panes**: `GET /panes` (`herdr workspace/tab/pane list`), `POST /panes/connect` (`agello start --pane`), `POST /panes/create` (`herdr workspace create`, `tab create`, `pane split`, always `--no-focus`)
129
+ - **Terminal**: `/terminal` WebSocket relays `herdr terminal session control <pane>` ANSI frames and validated input, resize, and scroll commands to xterm.js
132
130
  - **Screen**: finds the terminal-browser CDP port via `terminal-browser ls --json` and relays `Page.startScreencast` frames; user control forwards `Input.*` events only
133
131
 
134
132
  ## Security
135
133
 
136
- The server listens on `127.0.0.1` only. Data and input endpoints accept requests from the same origin and `localhost` pages only; other origins get `403` unless added with `--allow-origin`. Anything allowed to post to `/send` can type into your agent session, so keep the allow list short.
134
+ The server listens on `127.0.0.1` only. Data and input endpoints accept requests from the same origin and `localhost` pages only; other origins get `403` unless added with `--allow-origin`. Allowed origins can send chat prompts and connect to `/terminal` to type directly into the pane, so keep the allow list short.
135
+
136
+ ## Development
137
+
138
+ Run `bun install` and `bun test`. Tests cover the pane tree and creation (against a mock herdr), local terminal assets, origin rejection, ANSI frames, keyboard bytes, resize, exclusive control, and reconnect.
137
139
 
138
140
  ## License
139
141
 
package/bin/agello.ts CHANGED
@@ -27,13 +27,6 @@ Usage:
27
27
  whole screen), with agent replies as bubbles that fade after 10s.
28
28
  Run again to move the crop.
29
29
  ${NAME} present stop [--port N|--pane ID] End presentation mode (bubbles stay in the chat)
30
- ${NAME} script load <file.json> Load a presentation script (keeps the current position):
31
- {"rect": "x,y,w,h", "steps": [{"go": "#1", "say": ["line", ...], "hold": 6}]}
32
- go: "#n" sets location.hash, otherwise JS run in the page; one line = one bubble
33
- ${NAME} script show Script with positions (step.line) and player state
34
- ${NAME} present resume Play the script from where it stopped (starts presentation mode)
35
- ${NAME} present pause Stop the script (raising a hand on the page also pauses)
36
- ${NAME} present goto <n[.m]> Move to step n (line m) and show its screen
37
30
  ${NAME} help | --version
38
31
 
39
32
  start options:
@@ -291,56 +284,6 @@ async function cmdOpen(argv: string[]) {
291
284
  console.log(inst.url);
292
285
  }
293
286
 
294
- async function api(url: string, path: string, body?: object): Promise<any> {
295
- try {
296
- const r = await fetch(`${url}${path}`, {
297
- method: body ? "POST" : "GET",
298
- headers: { "Content-Type": "application/json" },
299
- body: body ? JSON.stringify(body) : undefined,
300
- signal: AbortSignal.timeout(8000),
301
- });
302
- return await r.json();
303
- } catch {
304
- return fail(`server unreachable: ${url}`);
305
- }
306
- }
307
-
308
- const playerLine = (p: any) =>
309
- `script: ${p.state}${p.last ? ` last=${p.last}` : ""}${p.next ? ` next=${p.next}` : ""} steps=${p.steps}`;
310
-
311
- async function cmdScript(argv: string[]) {
312
- const { values: o, positionals } = parseArgs({
313
- args: argv,
314
- options: { port: { type: "string" }, pane: { type: "string" } },
315
- allowPositionals: true,
316
- });
317
- const [sub, file] = positionals;
318
- const inst = await resolveTarget(o);
319
- if (sub === "load") {
320
- if (!file) fail("usage: script load <file.json>");
321
- let script: unknown;
322
- try {
323
- script = await Bun.file(file!).json();
324
- } catch (e) {
325
- fail(`cannot read ${file}: ${(e as Error).message}`);
326
- }
327
- const res = await api(inst.url, "/script", { script });
328
- if (!res.ok) fail(res.error);
329
- console.log(playerLine(res.player));
330
- } else if (sub === "show" || !sub) {
331
- const res = await api(inst.url, "/script");
332
- console.log(playerLine(res.player));
333
- for (const [i, st] of (res.script?.steps ?? []).entries()) {
334
- console.log(`${i + 1}${st.go ? ` go=${st.go}` : ""}${st.hold ? ` hold=${st.hold}s` : ""}`);
335
- st.say.forEach((l: string, j: number) => {
336
- const at = `${i + 1}.${j + 1}`;
337
- const mark = at === res.player.next ? ">" : at === res.player.last ? "*" : " ";
338
- console.log(` ${mark} ${at} ${l}`);
339
- });
340
- }
341
- } else fail(`unknown script command: ${sub}`);
342
- }
343
-
344
287
  async function cmdPresent(argv: string[]) {
345
288
  const { values: o, positionals } = parseArgs({
346
289
  args: argv,
@@ -348,14 +291,6 @@ async function cmdPresent(argv: string[]) {
348
291
  allowPositionals: true,
349
292
  });
350
293
  const inst = await resolveTarget(o);
351
- const [sub, at] = positionals;
352
- if (sub === "resume" || sub === "pause" || sub === "goto") {
353
- if (sub === "goto" && !at) fail("usage: present goto <n[.m]>");
354
- const res = await api(inst.url, "/player", { cmd: sub, at });
355
- if (!res.ok) fail(`${res.error}${res.player ? ` (${playerLine(res.player)})` : ""}`);
356
- console.log(playerLine(res.player));
357
- return;
358
- }
359
294
  const arg = positionals.join(",");
360
295
  let body: object;
361
296
  if (arg === "stop") body = { stop: true };
@@ -402,9 +337,6 @@ switch (cmd) {
402
337
  case "present":
403
338
  await cmdPresent(rest);
404
339
  break;
405
- case "script":
406
- await cmdScript(rest);
407
- break;
408
340
  case "--version":
409
341
  case "-v":
410
342
  console.log(pkg.version);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agello",
3
- "version": "0.2.1",
3
+ "version": "0.4.1",
4
4
  "description": "Talk to a coding agent in a herdr pane from the browser: chat, live status, and a shared browser screen.",
5
5
  "keywords": [
6
6
  "herdr",
@@ -31,5 +31,9 @@
31
31
  "engines": {
32
32
  "bun": ">=1.1"
33
33
  },
34
- "license": "MIT"
34
+ "license": "MIT",
35
+ "dependencies": {
36
+ "@xterm/addon-fit": "0.11.0",
37
+ "@xterm/xterm": "6.0.0"
38
+ }
35
39
  }
package/src/panes.ts ADDED
@@ -0,0 +1,86 @@
1
+ // herdr workspace -> tab -> pane tree, creation, and switching to a pane's own
2
+ // agello server (one server per pane; started on demand through the CLI).
3
+ // Deliberately no close/delete: closing a herdr pane kills its processes.
4
+
5
+ import { join } from "node:path";
6
+ import { run, runJson } from "./run.ts";
7
+
8
+ const CLI = join(import.meta.dir, "..", "bin", "agello.ts");
9
+ const ID = /^[A-Za-z0-9]+:[A-Za-z0-9]+$/; // herdr ids: w23, w23:t1, w23:p3
10
+ const WS_ID = /^[A-Za-z0-9]+$/;
11
+
12
+ // Resolve herdr on the current PATH (Bun.spawn uses the PATH from process start).
13
+ const bin = () => Bun.which("herdr", { PATH: process.env.PATH }) ?? "herdr";
14
+ const herdr = async (...cmd: string[]) => (await runJson([bin(), ...cmd], 5000)) ?? { error: { code: "herdr_failed" } };
15
+
16
+ export type TreePane = { id: string; agent?: string; status: string; title?: string; cwd?: string; focused: boolean };
17
+ export type TreeTab = { id: string; label: string; number: number; panes: TreePane[] };
18
+ export type TreeWorkspace = { id: string; label: string; number: number; tabs: TreeTab[] };
19
+
20
+ export async function paneTree(): Promise<TreeWorkspace[] | null> {
21
+ const [w, t, p] = await Promise.all([herdr("workspace", "list"), herdr("tab", "list"), herdr("pane", "list")]);
22
+ if (!w.result || !t.result || !p.result) return null;
23
+ const tabs = new Map<string, TreeTab>();
24
+ for (const tab of t.result.tabs)
25
+ tabs.set(tab.tab_id, { id: tab.tab_id, label: tab.label, number: tab.number, panes: [] });
26
+ for (const pane of p.result.panes)
27
+ tabs.get(pane.tab_id)?.panes.push({
28
+ id: pane.pane_id,
29
+ agent: pane.agent,
30
+ status: pane.agent_status,
31
+ title: pane.terminal_title_stripped || undefined,
32
+ cwd: pane.foreground_cwd || pane.cwd,
33
+ focused: pane.focused,
34
+ });
35
+ return w.result.workspaces.map((ws: any) => ({
36
+ id: ws.workspace_id,
37
+ label: ws.label,
38
+ number: ws.number,
39
+ tabs: t.result.tabs
40
+ .filter((tab: any) => tab.workspace_id === ws.workspace_id)
41
+ .map((tab: any) => tabs.get(tab.tab_id)!),
42
+ }));
43
+ }
44
+
45
+ const text = (v: unknown, max = 200) => (typeof v === "string" && v.trim() && v.length <= max ? v.trim() : undefined);
46
+ const fail = (error: string, status = 400) => Response.json({ ok: false, error }, { status });
47
+
48
+ // POST /panes/connect {pane} -> {ok, url}: the pane's server, started if needed.
49
+ async function connect(pane: unknown): Promise<Response> {
50
+ if (typeof pane !== "string" || !ID.test(pane)) return fail("invalid_pane");
51
+ if (!(await herdr("pane", "get", pane)).result) return fail("pane_not_found", 404);
52
+ const out = await run([process.execPath, CLI, "start", "--pane", pane], 20000);
53
+ const url = out?.match(/https?:\/\/127\.0\.0\.1:\d+/)?.[0];
54
+ return url ? Response.json({ ok: true, pane, url }) : fail("server_start_failed", 502);
55
+ }
56
+
57
+ // POST /panes/create {kind: workspace|tab|pane, ...} -> {ok, pane}: the new root pane.
58
+ async function create(d: any): Promise<Response> {
59
+ const cwd = text(d.cwd, 1024);
60
+ if (cwd && !cwd.startsWith("/")) return fail("invalid_cwd");
61
+ const label = text(d.label, 80);
62
+ const opt = [...(cwd ? ["--cwd", cwd] : []), ...(label ? ["--label", label] : []), "--no-focus"];
63
+ let res: any;
64
+ if (d.kind === "workspace") res = await herdr("workspace", "create", ...opt);
65
+ else if (d.kind === "tab") {
66
+ if (typeof d.workspace !== "string" || !WS_ID.test(d.workspace)) return fail("invalid_workspace");
67
+ res = await herdr("tab", "create", "--workspace", d.workspace, ...opt);
68
+ } else if (d.kind === "pane") {
69
+ if (typeof d.pane !== "string" || !ID.test(d.pane)) return fail("invalid_pane");
70
+ if (d.direction !== "right" && d.direction !== "down") return fail("invalid_direction");
71
+ res = await herdr("pane", "split", d.pane, "--direction", d.direction, ...(cwd ? ["--cwd", cwd] : []), "--no-focus");
72
+ } else return fail("invalid_kind");
73
+ const pane = res.result?.root_pane?.pane_id ?? res.result?.pane?.pane_id;
74
+ return pane ? Response.json({ ok: true, pane }) : fail(res.error?.code ?? "herdr_failed", 502);
75
+ }
76
+
77
+ export async function panesRoute(req: Request, path: string, current: string): Promise<Response | null> {
78
+ if (path === "/panes" && req.method === "GET") {
79
+ const tree = await paneTree();
80
+ return tree ? Response.json({ ok: true, current, workspaces: tree }) : fail("herdr_failed", 502);
81
+ }
82
+ if (req.method !== "POST" || (path !== "/panes/connect" && path !== "/panes/create")) return null;
83
+ const d = (await req.json().catch(() => null)) as any;
84
+ if (!d || typeof d !== "object") return fail("invalid_body");
85
+ return path === "/panes/connect" ? connect(d.pane) : create(d);
86
+ }
package/src/screencast.ts CHANGED
@@ -268,12 +268,5 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
268
268
  return browser ? { connected: false, browser: browser.key } : { connected: false, reason };
269
269
  }
270
270
 
271
- // Run JS in the relayed tab (presentation script `go`). Needs the relay to be
272
- // connected, i.e. at least one viewer. Returns false if it could not run.
273
- async function evaluate(expression: string): Promise<boolean> {
274
- const r = await request("Runtime.evaluate", { expression, awaitPromise: true });
275
- return !!r && !r.exceptionDetails;
276
- }
277
-
278
- return { handle, input, describe, setClip, evaluate };
271
+ return { handle, input, describe, setClip };
279
272
  }
package/src/server.ts CHANGED
@@ -4,21 +4,26 @@
4
4
  // GET /agent-bridge.js <agent-bridge> web component
5
5
  // GET /status session liveness (herdr agent get + pane get)
6
6
  // GET /events SSE: status, chat messages, tool start/end (from transcript JSONL)
7
- // POST /send {action, text} -> herdr agent prompt
7
+ // POST /send {action, text, images?: [{type, data(base64)}]} -> herdr agent prompt
8
+ // (images are saved to $TMPDIR/agello-uploads and sent as [image: <path>])
8
9
  // GET /screen SSE: terminal-browser screen frames (CDP screencast)
10
+ // POST /upload {images: [{type, data(base64)}]} -> {paths}: terminal image paste
11
+ // WS /terminal interactive herdr terminal frames, keyboard input, resize
12
+ // GET /panes herdr workspace -> tab -> pane tree
13
+ // POST /panes/connect {pane} -> url of that pane's server (started if needed)
14
+ // POST /panes/create {kind: workspace|tab|pane, cwd?, label?, workspace?, pane?, direction?}
9
15
  // WS /input user control: CDP Input.* commands forwarded to the page
10
16
  // GET /present presentation state
11
17
  // POST /present {rect?: {x,y,w,h}} start (or move the crop), {stop: true} end
12
18
  // ({stop: true, from: "viewer"}: ended from the page, the agent is told
13
19
  // with "[browser] action=present-stop")
14
- // GET /script presentation script + player position
15
- // POST /script {script: {rect?, steps: [{go?, say: string[], hold?}]}} load (keeps position)
16
- // POST /player {cmd: "resume" | "pause" | "goto", at?: "3" | "3.2"}
17
20
 
18
- import { homedir } from "node:os";
21
+ import { createTerminal, type SocketData } from "./terminal.ts";
22
+ import { panesRoute } from "./panes.ts";
23
+ import { mkdir } from "node:fs/promises";
24
+ import { homedir, tmpdir } from "node:os";
19
25
  import { join } from "node:path";
20
26
  import { runJson } from "./run.ts";
21
- import { createPlayer, parsePos, parseScript, type PlayerInfo } from "./player.ts";
22
27
  import { createScreencast, type Rect } from "./screencast.ts";
23
28
 
24
29
  export type ServerOptions = {
@@ -30,6 +35,30 @@ export type ServerOptions = {
30
35
  };
31
36
 
32
37
  export const BROWSER_PREFIX = "[browser]";
38
+ const UPLOAD_DIR = join(tmpdir(), "agello-uploads");
39
+ const IMAGE_EXT: Record<string, string> = { "image/png": "png", "image/jpeg": "jpg", "image/gif": "gif", "image/webp": "webp" };
40
+ const MAX_IMAGES = 5;
41
+ const MAX_IMAGE_BYTES = 10 * 1024 * 1024;
42
+
43
+ // Save pasted/dropped chat images so the agent can open them by path
44
+ // (herdr agent prompt carries text only). null: invalid input.
45
+ export async function saveImages(list: unknown): Promise<string[] | null> {
46
+ if (list === undefined) return [];
47
+ if (!Array.isArray(list) || list.length > MAX_IMAGES) return null;
48
+ const files = list.map((img: any) => {
49
+ const ext = IMAGE_EXT[img?.type];
50
+ if (!ext || typeof img.data !== "string") return null;
51
+ const bytes = Buffer.from(img.data, "base64");
52
+ return bytes.length && bytes.length <= MAX_IMAGE_BYTES ? { ext, bytes } : null;
53
+ });
54
+ if (files.includes(null)) return null;
55
+ await mkdir(UPLOAD_DIR, { recursive: true, mode: 0o700 });
56
+ return Promise.all(files.map(async (f) => {
57
+ const path = join(UPLOAD_DIR, `${crypto.randomUUID()}.${f!.ext}`);
58
+ await Bun.write(path, f!.bytes);
59
+ return path;
60
+ }));
61
+ }
33
62
  const WEB_DIR = join(import.meta.dir, "..", "web");
34
63
 
35
64
  // ---------- pure helpers ----------
@@ -94,7 +123,7 @@ export function parseRect(r: any): Rect | null {
94
123
  return { x, y, w, h };
95
124
  }
96
125
 
97
- export type Present = { on: boolean; rect?: Rect; since?: string; player?: PlayerInfo };
126
+ export type Present = { on: boolean; rect?: Rect; since?: string };
98
127
 
99
128
  type Status =
100
129
  | { alive: false; reason: string; pane: string }
@@ -108,6 +137,12 @@ export async function startServer(opts: ServerOptions) {
108
137
  const HTML = Bun.file(join(WEB_DIR, "index.html"));
109
138
  const COMPONENT = Bun.file(join(WEB_DIR, "agent-bridge.js"));
110
139
 
140
+ const terminal = createTerminal(PANE);
141
+ const bundle = await Bun.build({ entrypoints: [join(WEB_DIR, "terminal.js")], target: "browser", minify: true });
142
+ if (!bundle.success) throw new Error(`Terminal bundle failed: ${bundle.logs.join("\n")}`);
143
+ const terminalJS = await bundle.outputs[0].text();
144
+ const terminalCSS = Bun.file(new URL(import.meta.resolve("@xterm/xterm/css/xterm.css")));
145
+
111
146
  const agentInfo = async (): Promise<any | null> =>
112
147
  (await herdr(3000, "agent", "get", PANE))?.result?.agent ?? null;
113
148
 
@@ -307,7 +342,8 @@ export async function startServer(opts: ServerOptions) {
307
342
 
308
343
  const screen = createScreencast({
309
344
  browserKey: opts.browser,
310
- herdrTab: async () => (await agentInfo())?.tab_id,
345
+ // shell panes have no agent: fall back to the pane's own tab
346
+ herdrTab: async () => (await agentInfo())?.tab_id ?? (await herdr(3000, "pane", "get", PANE))?.result?.pane?.tab_id,
311
347
  });
312
348
 
313
349
  // ----- presentation mode -----
@@ -315,67 +351,10 @@ export async function startServer(opts: ServerOptions) {
315
351
  // short-lived bubbles. rect: crop of the visible viewport (unset = whole screen).
316
352
 
317
353
  let present: Present = { on: false };
318
- const presentView = (): Present => ({ ...present, player: player.info() });
319
354
  function setPresent(next: Present) {
320
355
  present = next;
321
356
  screen.setClip(next.on ? (next.rect ?? null) : null);
322
- if (!next.on) player.stop();
323
- broadcast("present", presentView());
324
- }
325
-
326
- // Script player: agello owns order and timing, the agent owns the deck
327
- // (each step's `go`) and decides when to resume after a question.
328
- const player = createPlayer({
329
- async go(expr) {
330
- const js = expr.startsWith("#") ? `location.hash = ${JSON.stringify(expr)}` : expr;
331
- if (!(await screen.evaluate(js))) console.error(`present: go failed: ${expr}`);
332
- },
333
- say(text, at, hold) {
334
- broadcast("message", { role: "assistant", text, ts: new Date().toISOString(), script: at, hold });
335
- },
336
- done() {
337
- const n = player.info().steps;
338
- promptAgent("present-done", `대본 끝까지 재생함 (${n}장)`);
339
- },
340
- changed() {
341
- broadcast("present", presentView());
342
- },
343
- });
344
-
345
- // "마지막 표시 3.2 · 다음 3.3" for agent notices
346
- const where = () => {
347
- const i = player.info();
348
- if (i.state === "empty") return "";
349
- return [i.last && `마지막 표시 ${i.last}`, i.next && `다음 ${i.next}`].filter(Boolean).join(" · ");
350
- };
351
-
352
- async function scriptRequest(req: Request): Promise<Response> {
353
- if (req.method !== "POST") return Response.json({ script: player.script(), player: player.info() });
354
- const data = (await req.json().catch(() => ({}))) as { script?: unknown };
355
- const script = parseScript(data.script);
356
- if (typeof script === "string") return Response.json({ ok: false, error: script }, { status: 400 });
357
- if (script.rect && !parseRect(script.rect))
358
- return Response.json({ ok: false, error: "invalid_rect" }, { status: 400 });
359
- player.load(script);
360
- return Response.json({ ok: true, player: player.info() });
361
- }
362
-
363
- async function playerRequest(req: Request): Promise<Response> {
364
- const data = (await req.json().catch(() => ({}))) as { cmd?: string; at?: string };
365
- let error: string | null = null;
366
- if (data.cmd === "resume") {
367
- if (!present.on) {
368
- const rect = player.script()?.rect;
369
- setPresent({ on: true, rect: rect ? parseRect(rect)! : undefined, since: new Date().toISOString() });
370
- }
371
- error = player.resume();
372
- } else if (data.cmd === "pause") player.pause();
373
- else if (data.cmd === "goto") {
374
- const pos = parsePos(data.at ?? "");
375
- error = pos ? await player.goto(pos) : "invalid_position";
376
- } else error = "unknown_cmd";
377
- if (error) return Response.json({ ok: false, error, player: player.info() }, { status: 400 });
378
- return Response.json({ ok: true, player: player.info() });
357
+ broadcast("present", present);
379
358
  }
380
359
 
381
360
  async function presentRequest(req: Request): Promise<Response> {
@@ -387,11 +366,10 @@ export async function startServer(opts: ServerOptions) {
387
366
  // (The CLI stop comes from the agent itself and needs no notice.)
388
367
  let notified: string | undefined;
389
368
  if (was && data.from === "viewer") {
390
- const at = where();
391
- const r = await promptAgent("present-stop", `사용자가 페이지에서 발표를 종료함${at ? ` (대본 정지: ${at})` : ""}`);
369
+ const r = await promptAgent("present-stop", "사용자가 페이지에서 발표를 종료함");
392
370
  notified = r.ok ? "ok" : r.error;
393
371
  }
394
- return Response.json({ ok: true, present: presentView(), notified });
372
+ return Response.json({ ok: true, present, notified });
395
373
  }
396
374
  let rect: Rect | undefined;
397
375
  if (data.rect != null) {
@@ -399,7 +377,7 @@ export async function startServer(opts: ServerOptions) {
399
377
  if (!rect) return Response.json({ ok: false, error: "invalid_rect" }, { status: 400 });
400
378
  }
401
379
  setPresent({ on: true, rect, since: present.on ? present.since : new Date().toISOString() });
402
- return Response.json({ ok: true, present: presentView() });
380
+ return Response.json({ ok: true, present });
403
381
  }
404
382
 
405
383
  // ----- HTTP -----
@@ -440,7 +418,7 @@ export async function startServer(opts: ServerOptions) {
440
418
  tools: [...pendingTools.keys()],
441
419
  });
442
420
  if (lastStatus) send(c, "status", lastStatus);
443
- send(c, "present", presentView());
421
+ send(c, "present", present);
444
422
  for (const t of pendingTools.values()) send(c, "tool_start", t);
445
423
  ping = setInterval(() => send(c, "ping", {}), 15000);
446
424
  },
@@ -465,33 +443,36 @@ export async function startServer(opts: ServerOptions) {
465
443
  }
466
444
 
467
445
  async function sendPrompt(req: Request): Promise<Response> {
468
- const data = (await req.json().catch(() => ({}))) as { action?: string; text?: string };
446
+ const data = (await req.json().catch(() => ({}))) as { action?: string; text?: string; images?: unknown };
469
447
  const action = /^[\w-]{1,32}$/.test(data.action ?? "") ? data.action! : "message";
470
- let text = data.text ?? "";
471
- // Raised hand: stop the script at once (no agent round trip) and tell the
472
- // agent where it stopped.
473
- if (action === "hand-raise") {
474
- player.pause();
475
- const at = where();
476
- if (at) text += ` (대본 정지: ${at})`;
477
- }
448
+ const images = await saveImages(data.images);
449
+ if (!images) return Response.json({ ok: false, error: "invalid_images" }, { status: 400 });
450
+ const text = [data.text ?? "", ...images.map((p) => `[image: ${p}]`)].join("\n");
478
451
  const r = await promptAgent(action, text);
479
452
  return r.ok ? Response.json({ ok: true }) : Response.json({ ok: false, error: r.error }, { status: r.status });
480
453
  }
481
454
 
482
455
  async function route(req: Request, url: URL): Promise<Response> {
456
+ if (url.pathname.startsWith("/panes")) {
457
+ const res = await panesRoute(req, url.pathname, PANE);
458
+ if (res) return res;
459
+ }
483
460
  switch (url.pathname) {
484
461
  case "/status":
485
462
  return Response.json({
486
463
  ...(await checkStatus()),
487
464
  browser: opts.browser,
488
465
  screen: await screen.describe(),
489
- present: presentView(),
466
+ present,
490
467
  });
491
468
  case "/events":
492
469
  return events();
493
470
  case "/screen":
494
471
  return screen.handle();
472
+ case "/terminal.js":
473
+ return new Response(terminalJS, { headers: { "Content-Type": "text/javascript; charset=utf-8" } });
474
+ case "/terminal.css":
475
+ return new Response(terminalCSS, { headers: { "Content-Type": "text/css; charset=utf-8" } });
495
476
  case "/agent-bridge.js":
496
477
  return new Response(COMPONENT, {
497
478
  headers: { "Content-Type": "text/javascript; charset=utf-8", "Cache-Control": "no-store" },
@@ -499,30 +480,36 @@ export async function startServer(opts: ServerOptions) {
499
480
  case "/send":
500
481
  if (req.method === "POST") return sendPrompt(req);
501
482
  break;
502
- case "/present":
503
- return req.method === "POST" ? presentRequest(req) : Response.json(presentView());
504
- case "/script":
505
- return scriptRequest(req);
506
- case "/player":
507
- if (req.method === "POST") return playerRequest(req);
483
+ case "/upload":
484
+ if (req.method === "POST") {
485
+ const data = (await req.json().catch(() => ({}))) as { images?: unknown };
486
+ const paths = data.images === undefined ? null : await saveImages(data.images);
487
+ return paths ? Response.json({ ok: true, paths }) : Response.json({ ok: false, error: "invalid_images" }, { status: 400 });
488
+ }
508
489
  break;
490
+ case "/present":
491
+ return req.method === "POST" ? presentRequest(req) : Response.json(present);
509
492
  }
510
493
  return new Response(HTML, { headers: { "Content-Type": "text/html; charset=utf-8", "Cache-Control": "no-store" } });
511
494
  }
512
495
 
513
- const server = Bun.serve({
496
+ const server = Bun.serve<SocketData>({
514
497
  hostname: "127.0.0.1",
515
498
  port: opts.port,
516
499
  idleTimeout: 0, // keep SSE open
517
500
  websocket: {
518
- message(_ws, msg) {
519
- screen.input(String(msg));
501
+ maxPayloadLength: 128 * 1024,
502
+ open(ws) { if (ws.data.kind === "terminal") terminal.open(ws); },
503
+ message(ws, msg) {
504
+ if (ws.data.kind === "terminal") terminal.message(ws, String(msg));
505
+ else screen.input(String(msg));
520
506
  },
507
+ close(ws) { if (ws.data.kind === "terminal") terminal.close(ws); },
521
508
  },
522
509
  async fetch(req, server) {
523
510
  const url = new URL(req.url);
524
511
  const origin = req.headers.get("Origin");
525
- const isScript = url.pathname === "/agent-bridge.js";
512
+ const isScript = ["/agent-bridge.js", "/terminal.js", "/terminal.css"].includes(url.pathname);
526
513
 
527
514
  // The component script itself is public; data and input endpoints are origin-checked.
528
515
  if (!isScript && !originAllowed(origin, url.origin))
@@ -541,8 +528,9 @@ export async function startServer(opts: ServerOptions) {
541
528
  origin,
542
529
  );
543
530
 
544
- if (url.pathname === "/input")
545
- return server.upgrade(req) ? undefined : new Response("websocket required", { status: 426 });
531
+ if (url.pathname === "/input" || url.pathname === "/terminal")
532
+ return server.upgrade(req, { data: { kind: url.pathname === "/terminal" ? "terminal" : "screen" } })
533
+ ? undefined : new Response("websocket required", { status: 426 });
546
534
 
547
535
  const res = await route(req, url);
548
536
  return isScript ? (res.headers.set("Access-Control-Allow-Origin", "*"), res) : withCors(res, origin);
@@ -555,6 +543,7 @@ export async function startServer(opts: ServerOptions) {
555
543
  session: SESSION,
556
544
  transcript,
557
545
  stop() {
546
+ terminal.stop();
558
547
  timers.forEach(clearInterval);
559
548
  server.stop(true);
560
549
  },