agello 0.4.2 → 0.4.4

package/README.md

@@ -8,6 +8,7 @@
 - Interactive terminal view of the existing herdr pane: live cursor, keyboard input, and automatic resize
 - Optional live view of a `terminal-browser` screen, with an on/off switch to control it yourself
 - Annotate: point at an element on that screen and send a request about it, with its CSS selector
+- Tab sound: hear that screen's audio on the page instead of the agent's machine
 - Embeddable `<agent-bridge>` web component
 
 Chat currently supports **Claude Code** sessions. The terminal view connects directly to the pane, including ordinary shells and other terminal applications.
@@ -73,6 +74,10 @@ The `주석` button on the screen panel turns on annotate mode: hovering tints t
 
 Selector: nearest unique `id` / `data-testid` ancestor, then `tag:nth-of-type` steps (light DOM only). Coordinates are CSS px of the viewport. Off while presenting and while "내 조작" is on.
 
+### Tab sound
+
+The `소리` button on the screen panel plays the relayed tab's audio on that page. While at least one page has it on, the tab is muted in the agent's terminal-browser; when the last one turns it off or closes, local playback returns within 4 seconds. agello reroutes the page's Web Audio output and `<audio>`/`<video>` elements into PCM chunks (`GET /screen/audio`, SSE), since CDP has no audio stream. Not captured: Web Audio connected before it was turned on (reload the tab), cross-origin media without CORS (keeps playing locally), cross-site iframes. Audio trails the screen by about 0.2s.
+
 ### Commands
 
 | Command | Description |
@@ -113,7 +118,8 @@ The xterm.js renderer and its styles are served locally; no terminal CDN is requ
 - agent replies appear over it as bubbles that fade after 10s
 - `agello present stop` returns to the normal layout; the replies are in the chat as usual
 - the viewer can ask a question with the raise-hand button (its bubble fades after 10s too) or end the presentation with the end button; the agent then receives `[browser] action=present-stop`
-- raising the hand tells the agent at once (`action=hand-raise`) so it can pause; closing the input without asking sends `action=hand-lower`
+- raising the hand tells the agent at once (`action=hand-raise`) so it can pause; closing it with X (or Esc) before asking sends `action=hand-lower`
+- the input stays open for the whole question: after the agent answers, follow-up questions go in the same input (no need to raise the hand again); X (or Esc) then sends `action=hand-done`, and the agent resumes
 
 Run `present` again to move the crop. User control is off while presenting.
 
@@ -130,10 +136,35 @@ Write the talk ahead so each line appears the moment its screen does:
 
 - step: `go` brings the screen there (`#n` sets `location.hash`, anything else is JS run in the page; agello knows nothing about the deck), `say` lines are one bubble each, `hold` seconds a line stays on screen (default from length, 10 to 20s; never below 10s). Pacing: screen change, 1s, line, fade out, 0.3s, next line; after a step's last line 0.7s before the next screen
 - `present resume` plays from where it stopped (starts presentation mode with the script's `rect`); it first re-runs the current step's `go`, so the agent may move the screen freely while answering
-- raising a hand pauses at once; the agent gets `action=hand-raise` with the position (`마지막 표시 2.1 · 다음 2.2`)
+- raising a hand pauses at once and rewinds one line, so `present resume` replays the interrupted line; the agent gets `action=hand-raise` with the position (`마지막 표시 2.1 · 다음 2.1`)
+- raising, cancelling (X before asking) and finishing a question (X after an answer) show a small light event bubble for 5s right away, apart from the conversation bubbles; sending a question shows none
 - `script load` again after editing keeps the position; `present goto 2.2` then `present resume` replays from a fixed line
 - at the end the agent gets `action=present-done`; while the script is paused and the agent is working, the page shows a small loader
 
+#### Optional TTS and recording
+
+The screen panel's **발표 설정** remains available during presentation. TTS and recording both start OFF and can be used independently:
+
+| TTS | Recording | Result |
+|---|---|---|
+| OFF | OFF | Existing timed script captions |
+| OFF | ON | Presentation images and captions in a silent video |
+| ON | OFF | Spoken script, with captions synchronized to audio playback |
+| ON | ON | Presentation images, captions, and TTS audio in the controlling viewer's video |
+
+1. Open **발표 설정**. Enter a Typecast API key and optionally change the voice ID, then click **키·목소리 적용**. The default voice is `tc_6a0e85a97f7750959b970d5d`; the provider uses Korean (`kor`) and model `ssfm-v30`.
+2. Alternatively, **로컬 키 불러오기** reads only `TYPECAST_API_KEY` from `../yt-outlier/.env`, relative to the server's working directory. It does not execute the file or import other settings. A server started with `TYPECAST_API_KEY` in its environment also has a key ready, with TTS still OFF.
+3. Click **TTS 켜기** in the viewer that should play sound. This click grants browser audio playback permission. Click **대본 재생** after loading a script; **일시정지** stops playback, and **다시 재생** starts a completed script again from its first step.
+4. Click **녹화 시작**, then **녹화 종료** and **영상 저장** to download the recording. Recording can start before presentation if a screen image is already available. Stopping recording leaves script and audio playback running.
+
+With TTS enabled, each script caption appears after that viewer starts the actual audio, and closes when the audio finishes. The audio's duration replaces the script's `hold`; ordinary agent replies still use their existing caption timer. Raising a hand immediately stops audio locally, and resumed playback repeats the interrupted line. Pausing, changing steps, updating a script, or disconnecting cancels outdated speech. A TTS failure pauses the script; fix the key or connection, or turn TTS OFF, then resume.
+
+Only the viewer that enabled TTS plays audio and acknowledges speech progress. Other viewers receive the same captions and can record a silent video. To include TTS audio, record in the viewer that enabled TTS. Audio settings are fixed when recording starts: stop recording before changing TTS. If another viewer changes the controlling audio role, a recording whose audio setting changes is finalized automatically.
+
+Recording uses a 1280×720 canvas that fits the relayed browser image with letterboxing and draws the current captions below it. It records these images and plain caption text, rather than the page's entire DOM: chat, settings, question input, and control buttons are excluded; caption Markdown styling is flattened. Long captions are wrapped and truncated to the safe area. The canvas targets 30fps, while source screen updates can be slower (presentation screenshots are approximately 10fps). A background or minimized tab may render more slowly. Use a current Chromium browser for the verified recording path; other browsers depend on their `canvas.captureStream`, Web Audio, and `MediaRecorder` support. MP4 is preferred when supported, with WebM as a fallback. Streaming WebM duration metadata is finalized when its header supports the correction; otherwise the original browser recording is retained.
+
+Recording stops at 15 minutes or 256MB, at presentation end, on server changes, or on connection loss. Keep the page open until **영상 저장** appears and download it before closing or reloading; recording data is held in browser memory. A later recording replaces the earlier download link.
+
 ## Embed
 
 ```html
@@ -165,9 +196,11 @@ See `web/embed-example.html`.
 
 The server listens on `127.0.0.1` only. Data and input endpoints accept requests from the same origin and `localhost` pages only; other origins get `403` unless added with `--allow-origin`. Allowed origins can send chat prompts and connect to `/terminal` to type directly into the pane, so keep the allow list short.
 
+TTS keys remain in server memory until replaced, cleared with **키 삭제**, or the server stops. Password inputs are cleared after applying; keys and speech-driver tokens are never put in browser storage, chat history, scripts, or SSE broadcasts. Keys are sent only to the local server and Typecast. Script text is sent to Typecast when TTS is ON, and generated audio is cached in server memory for reuse. Allowed origins can also configure TTS and read generated audio, so allow only trusted pages.
+
 ## Development
 
-Run `bun install` and `bun test`. Tests cover the pane tree and creation (against a mock herdr), local terminal assets, origin rejection, ANSI frames, keyboard bytes, resize, exclusive control, and reconnect.
+Run `bun install` and `bun test`. Tests cover the pane tree and creation (against a mock herdr), local terminal assets, origin rejection, ANSI frames, keyboard bytes, resize, exclusive control, reconnect, TTS coordination, stale speech cancellation, and recording lifecycle and metadata.
 
 ## License
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agello",
-  "version": "0.4.2",
+  "version": "0.4.4",
   "description": "Talk to a coding agent in a herdr pane from the browser: chat, live status, and a shared browser screen.",
   "keywords": [
     "herdr",

package/src/page-audio.ts

@@ -0,0 +1,159 @@
+// Page audio relay, page side. Injected into the relayed terminal-browser tab
+// through CDP while at least one viewer listens (GET /screen/audio).
+//
+// CDP has no audio stream and terminal-browser (Electron) denies tab capture,
+// so the page's own audio graph is rerouted:
+//   - AudioNode.connect(ctx.destination) goes to a per-context tap instead
+//   - <audio>/<video> playing while remote are moved into a WebAudio graph
+//     (createMediaElementSource), which also takes them off the speakers
+//   tap -> out (gain 1 local, 0 remote) -> real destination
+//   tap -> ScriptProcessor -> 16-bit stereo PCM -> binding -> agello server
+//
+// Remote mode lasts while the server keeps calling ping() (every ~1.5s); after
+// 4s without a ping (server gone, tab no longer relayed) local playback returns.
+//
+// Not captured: connections made before injection (reload the page),
+// cross-origin media without CORS (routing them would play silence, so they
+// keep playing locally), cross-site iframes (separate CDP targets).
+
+export const AUDIO_BINDING = "__agelloAudioPcm";
+
+export const PAGE_AUDIO_JS = `(() => {
+  if (window.__agelloAudio) return;
+  const AC = window.AudioContext || window.webkitAudioContext;
+  if (!AC || !window.AudioNode) return;
+  const BINDING = ${JSON.stringify(AUDIO_BINDING)};
+  const QUIET_CHUNKS = 24; // ~2s of silence before chunks stop being sent
+  const origConnect = AudioNode.prototype.connect;
+  const origDisconnect = AudioNode.prototype.disconnect;
+  const origCreateSource = AC.prototype.createMediaElementSource;
+  const origPlay = HTMLMediaElement.prototype.play;
+  const taps = new WeakMap(); // context -> tap record
+  const records = new Set();
+  const owned = new WeakSet(); // media elements moved into our graph
+  let remote = false, lastPing = 0, nextId = 1, mediaCtx = null;
+
+  function send(t, buf) {
+    const n = buf.length;
+    const L = buf.getChannelData(0), R = buf.numberOfChannels > 1 ? buf.getChannelData(1) : L;
+    const pcm = new Int16Array(n * 2);
+    let loud = false;
+    for (let i = 0; i < n; i++) {
+      const l = L[i], r = R[i];
+      if (l || r) loud = true;
+      pcm[2 * i] = (l < -1 ? -1 : l > 1 ? 1 : l) * 32767;
+      pcm[2 * i + 1] = (r < -1 ? -1 : r > 1 ? 1 : r) * 32767;
+    }
+    t.quiet = loud ? 0 : t.quiet + 1;
+    if (t.quiet > QUIET_CHUNKS) return;
+    const bytes = new Uint8Array(pcm.buffer);
+    let s = "";
+    for (let i = 0; i < bytes.length; i += 0x8000) s += String.fromCharCode.apply(null, bytes.subarray(i, i + 0x8000));
+    try { window[BINDING](JSON.stringify({ c: t.id, r: t.ctx.sampleRate, d: btoa(s) })); } catch {}
+  }
+
+  function tapOf(ctx) {
+    let t = taps.get(ctx);
+    if (t) return t;
+    const tap = ctx.createGain(), out = ctx.createGain();
+    out.gain.value = remote ? 0 : 1;
+    origConnect.call(tap, out);
+    origConnect.call(out, ctx.destination);
+    t = { id: nextId++, ctx, tap, out, quiet: QUIET_CHUNKS + 1 };
+    if (ctx.createScriptProcessor) {
+      const proc = ctx.createScriptProcessor(4096, 2, 2), sink = ctx.createGain();
+      sink.gain.value = 0;
+      origConnect.call(tap, proc);
+      origConnect.call(proc, sink);
+      origConnect.call(sink, ctx.destination);
+      proc.onaudioprocess = (e) => { if (remote) send(t, e.inputBuffer); };
+      t.proc = proc;
+    }
+    taps.set(ctx, t);
+    records.add(t);
+    return t;
+  }
+
+  AudioNode.prototype.connect = function (dest, output) {
+    const ctx = this.context;
+    if (dest && ctx instanceof AC && dest === ctx.destination) {
+      origConnect.call(this, tapOf(ctx).tap, output || 0);
+      return dest;
+    }
+    return origConnect.apply(this, arguments);
+  };
+  AudioNode.prototype.disconnect = function (dest) {
+    const ctx = this.context, t = ctx && taps.get(ctx);
+    if (t && dest && dest === ctx.destination) {
+      const args = [...arguments];
+      args[0] = t.tap;
+      if (args.length > 2) args.length = 2; // the tap has one input
+      return origDisconnect.apply(this, args);
+    }
+    return origDisconnect.apply(this, arguments);
+  };
+
+  function capturable(el) {
+    if (el.srcObject) return true;
+    const src = el.currentSrc || el.src;
+    if (!src) return false;
+    if (el.crossOrigin != null) return true;
+    try {
+      const u = new URL(src, location.href);
+      return u.protocol === "data:" || u.origin === location.origin;
+    } catch { return false; }
+  }
+  function own(el) {
+    if (!remote || !(el instanceof HTMLMediaElement) || owned.has(el) || !capturable(el)) return;
+    try {
+      mediaCtx = mediaCtx || new AC();
+      const src = origCreateSource.call(mediaCtx, el);
+      owned.add(el);
+      origConnect.call(src, tapOf(mediaCtx).tap);
+      if (mediaCtx.state !== "running") mediaCtx.resume().catch(() => {});
+    } catch {}
+  }
+  HTMLMediaElement.prototype.play = function () {
+    own(this);
+    return origPlay.apply(this, arguments);
+  };
+  for (const type of ["play", "playing"]) document.addEventListener(type, (e) => own(e.target), true);
+
+  function setRemote(on) {
+    if (on) lastPing = Date.now();
+    if (remote === on) return;
+    remote = on;
+    for (const t of records) {
+      t.out.gain.value = on ? 0 : 1;
+      t.quiet = QUIET_CHUNKS + 1;
+    }
+    if (on) {
+      document.querySelectorAll("audio,video").forEach((el) => { if (!el.paused) own(el); });
+      if (mediaCtx && mediaCtx.state !== "running") mediaCtx.resume().catch(() => {});
+    }
+  }
+  setInterval(() => { if (remote && Date.now() - lastPing > 4000) setRemote(false); }, 1000);
+
+  window.__agelloAudio = {
+    ping: () => setRemote(true),
+    off: () => setRemote(false),
+    state: () => ({ remote, contexts: records.size, media: !!mediaCtx }),
+  };
+})()`;
+
+// Viewer-bound chunk, validated (the binding is callable by the page itself).
+export type PcmChunk = { c: number; r: number; d: string };
+export function parsePcm(payload: string): PcmChunk | null {
+  if (payload.length > 512 * 1024) return null;
+  let m: any;
+  try {
+    m = JSON.parse(payload);
+  } catch {
+    return null;
+  }
+  const { c, r, d } = m ?? {};
+  if (!Number.isInteger(c) || c < 1 || c > 1e6) return null;
+  if (!Number.isFinite(r) || r < 8000 || r > 192000) return null;
+  if (typeof d !== "string" || !d || d.length % 4 || !/^[A-Za-z0-9+/]+={0,2}$/.test(d)) return null;
+  return { c, r, d };
+}

package/src/player.ts

@@ -9,6 +9,7 @@
 // line is sent and held for its reading time (see PACE). pause() stops at once and keeps
 // the position, resume() re-runs the current step's `go` (the agent may have
 // moved the screen meanwhile) and continues from the next unshown line.
+// rewind() (after a raised hand) moves back so the last shown line plays again.
 // goto() moves the position and shows that step without playing.
 //
 // Positions are "step.line", 1-based, in everything that leaves this module.
@@ -22,6 +23,7 @@ export type PlayerInfo = {
   next?: string; // next line to show ("3.2")
   last?: string; // last line shown
   steps: number;
+  error?: string;
 };
 
 // Pacing: the audience looks at a new screen first, then reads and thinks.
@@ -79,7 +81,10 @@ export function parsePos(v: string): { step: number; line: number } | null {
 
 export function createPlayer(deps: {
   go: (expr: string) => Promise<void>;
-  say: (text: string, at: string, hold: number) => void;
+  say: (text: string, at: string, hold: number, narration?: string) => void;
+  // null selects text pacing; a promise resolves only after actual playback ends.
+  narrate?: (text: string, at: string, signal: AbortSignal,
+    started: (duration: number, id: string) => void) => Promise<void> | null;
   done: () => void;
   changed: () => void;
 }) {
@@ -90,6 +95,8 @@ export function createPlayer(deps: {
   let entered = false; // `go` of cur.step already run in this play run
   let run = 0; // bumps on every pause/goto/load, stale timers check it
   let timer: Timer | null = null;
+  let speech: AbortController | null = null;
+  let error: string | undefined;
 
   const fmt = (p: { step: number; line: number }) => `${p.step + 1}.${p.line + 1}`;
   const set = (s: PlayerInfo["state"]) => {
@@ -100,6 +107,8 @@ export function createPlayer(deps: {
     run++;
     if (timer) clearTimeout(timer);
     timer = null;
+    speech?.abort();
+    speech = null;
   };
   const wait = (ms: number, id: number, fn: () => void) => {
     timer = setTimeout(() => id === run && fn(), ms);
@@ -116,6 +125,7 @@ export function createPlayer(deps: {
       steps,
       next: script && p.step < steps ? fmt(p) : undefined,
       last: last ? fmt(last) : undefined,
+      ...(error ? { error } : {}),
     };
   }
 
@@ -154,13 +164,38 @@ export function createPlayer(deps: {
       return;
     }
     const text = step.say[cur.line];
-    const hold = lineHold(text, step.hold);
+    const position = { ...cur };
     const lastLine = cur.line === step.say.length - 1;
+    const gap = lastLine ? PACE.afterStep : PACE.betweenLines;
+    speech = new AbortController();
+    const narration = deps.narrate?.(text, fmt(cur), speech.signal, (duration, narrationId) => {
+      if (id !== run || state !== "playing") return;
+      last = position;
+      deps.say(text, fmt(position), duration, narrationId);
+      deps.changed();
+    });
+    if (narration) {
+      try {
+        await narration;
+        if (id !== run || state !== "playing") return;
+        speech = null;
+        cur = { step: position.step, line: position.line + 1 };
+        deps.changed();
+        wait(PACE.fade + gap, id, () => tick(id));
+      } catch {
+        if (id !== run) return;
+        cancel();
+        error = "narration_failed";
+        set("paused");
+      }
+      return;
+    }
+    speech = null;
+    const hold = lineHold(text, step.hold);
     last = { ...cur };
     deps.say(text, fmt(cur), hold);
     cur = { step: cur.step, line: cur.line + 1 };
     deps.changed();
-    const gap = lastLine ? PACE.afterStep : PACE.betweenLines;
     wait(hold * 1000 + PACE.fade + gap, id, () => tick(id));
   }
 
@@ -172,6 +207,7 @@ export function createPlayer(deps: {
     load(next: Script) {
       const playing = state === "playing";
       cancel();
+      error = undefined;
       script = next;
       if (cur.step > next.steps.length) cur = { step: next.steps.length, line: 0 };
       normalize();
@@ -189,6 +225,7 @@ export function createPlayer(deps: {
       if (state === "playing") return null;
       if (state === "done") return "done";
       cancel();
+      error = undefined;
       entered = false; // bring the screen back to the current step first
       set("playing");
       tick(run);
@@ -201,6 +238,15 @@ export function createPlayer(deps: {
       set("paused");
     },
 
+    // Paused by an interruption (raised hand): the next resume replays the
+    // last shown line, since the viewer may have missed it.
+    rewind() {
+      if (state !== "paused" || !last) return;
+      cur = { ...last };
+      entered = false;
+      deps.changed();
+    },
+
     // Move to a step (and line) and show its screen. Does not play; if it was
     // playing, it keeps playing from there.
     async goto(pos: { step: number; line: number }): Promise<string | null> {
@@ -210,6 +256,7 @@ export function createPlayer(deps: {
       if (pos.line > 0 && pos.line >= step.say.length) return "no_such_line";
       const playing = state === "playing";
       cancel();
+      error = undefined;
       cur = { ...pos };
       entered = false;
       if (playing) {
@@ -225,8 +272,8 @@ export function createPlayer(deps: {
 
     // Presentation ended: stop where it is.
     stop() {
+      cancel();
       if (state === "playing") {
-        cancel();
         set("paused");
       }
     },

package/src/screencast.ts

@@ -1,4 +1,5 @@
 import { runJson } from "./run.ts";
+import { AUDIO_BINDING, PAGE_AUDIO_JS, parsePcm } from "./page-audio.ts";
 
 // terminal-browser screen relay: CDP Page.startScreencast -> SSE (/screen).
 //
@@ -6,6 +7,10 @@ import { runJson } from "./run.ts";
 // follows its active tab, and streams JPEG frames only while at least one
 // viewer is connected. input() forwards whitelisted CDP Input.* commands
 // from the viewer to the page (user control).
+//
+// Page audio (GET /screen/audio, SSE): while at least one viewer listens, the
+// tab's audio is rerouted into PCM chunks for those viewers and muted locally
+// (see page-audio.ts). The last listener leaving restores local playback.
 
 type Browser = {
   key: string;
@@ -24,10 +29,12 @@ export type ScreenMeta = {
   url?: string;
   title?: string;
   agentControlled?: boolean;
+  audioListeners?: number; // status only
 };
 
 export function createScreencast(opts: { browserKey?: string; herdrTab?: () => Promise<string | undefined> }) {
   const viewers = new Set<ReadableStreamDefaultController>();
+  const listeners = new Set<ReadableStreamDefaultController>(); // page audio
   const enc = new TextEncoder();
 
   let meta: ScreenMeta = { connected: false };
@@ -39,6 +46,7 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
   let wsTarget = "";
   let msgId = 0;
   let timer: Timer | null = null;
+  let audioOn = false; // audio hook installed on the current CDP session
   const pending = new Map<number, (result: any) => void>();
 
   const emit = (c: ReadableStreamDefaultController, event: string, data: unknown) => {
@@ -46,9 +54,12 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
       c.enqueue(enc.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`));
     } catch {
       viewers.delete(c);
+      listeners.delete(c);
     }
   };
   const broadcast = (event: string, data: unknown) => viewers.forEach((c) => emit(c, event, data));
+  const AUDIO_OFF = "window.__agelloAudio?.off()";
+  const AUDIO_PING = "window.__agelloAudio?.ping()";
 
   function setMeta(next: ScreenMeta) {
     if (JSON.stringify(next) === JSON.stringify(meta)) return;
@@ -78,6 +89,10 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
   }
 
   function disconnect() {
+    // the old tab plays locally again (its own 4s timeout covers a lost message)
+    if (audioOn && ws?.readyState === WebSocket.OPEN)
+      ws.send(JSON.stringify({ id: ++msgId, method: "Runtime.evaluate", params: { expression: AUDIO_OFF } }));
+    audioOn = false;
     ws?.close();
     ws = null;
     wsTarget = "";
@@ -153,6 +168,7 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
       call("Page.enable");
       call("Page.startScreencast", { format: "jpeg", quality: 70, maxWidth: 1600, maxHeight: 1600 });
       if (clip) capture();
+      if (listeners.size) audioStart();
     };
     sock.onmessage = (ev) => {
       const msg = JSON.parse(String(ev.data));
@@ -162,6 +178,11 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
         resolve(msg.result ?? null);
         return;
       }
+      if (msg.method === "Runtime.bindingCalled" && msg.params?.name === AUDIO_BINDING) {
+        const chunk = listeners.size ? parsePcm(String(msg.params.payload ?? "")) : null;
+        if (chunk) listeners.forEach((c) => emit(c, "pcm", chunk));
+        return;
+      }
       if (msg.method !== "Page.screencastFrame") return;
       const { data, metadata, sessionId } = msg.params;
       call("Page.screencastFrameAck", { sessionId });
@@ -177,6 +198,25 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
     };
   }
 
+  // Install the page hook (new documents too) and switch the tab to remote.
+  function audioStart() {
+    const sock = ws;
+    if (audioOn || !sock || sock.readyState !== WebSocket.OPEN) return;
+    audioOn = true;
+    const call = (method: string, params: object = {}) =>
+      sock.send(JSON.stringify({ id: ++msgId, method, params }));
+    call("Runtime.enable");
+    call("Runtime.addBinding", { name: AUDIO_BINDING });
+    call("Page.addScriptToEvaluateOnNewDocument", { source: PAGE_AUDIO_JS });
+    call("Runtime.evaluate", { expression: PAGE_AUDIO_JS });
+    call("Runtime.evaluate", { expression: AUDIO_PING });
+  }
+  function audioPing() {
+    if (!listeners.size || !ws || ws.readyState !== WebSocket.OPEN) return;
+    if (!audioOn) return audioStart();
+    ws.send(JSON.stringify({ id: ++msgId, method: "Runtime.evaluate", params: { expression: AUDIO_PING } }));
+  }
+
   let syncing = false; // skip a tick while the previous sync is still running
   async function sync() {
     if (syncing) return;
@@ -200,6 +240,7 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
     }
     const url = `ws://127.0.0.1:${b.cdpPort}/devtools/page/${tab.targetId}`;
     if (url !== wsTarget || !ws) connect(b.cdpPort, tab.targetId);
+    else audioPing();
     setMeta({
       connected: true,
       browser: b.key,
@@ -248,6 +289,31 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
     });
   }
 
+  // SSE of page audio for one listener. Only meaningful alongside /screen
+  // (the relay runs while screen viewers are connected).
+  function handleAudio(): Response {
+    let ctrl!: ReadableStreamDefaultController;
+    let ping: Timer;
+    const stream = new ReadableStream({
+      start(c) {
+        ctrl = c;
+        listeners.add(c);
+        emit(c, "ready", {});
+        ping = setInterval(() => emit(c, "ping", {}), 15000);
+        audioPing();
+      },
+      cancel() {
+        clearInterval(ping);
+        listeners.delete(ctrl);
+        if (listeners.size === 0 && audioOn && ws?.readyState === WebSocket.OPEN)
+          ws.send(JSON.stringify({ id: ++msgId, method: "Runtime.evaluate", params: { expression: AUDIO_OFF } }));
+      },
+    });
+    return new Response(stream, {
+      headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", Connection: "keep-alive" },
+    });
+  }
+
   const INPUT_METHODS = new Set(["Input.dispatchMouseEvent", "Input.dispatchKeyEvent", "Input.insertText"]);
   function input(raw: string) {
     let m: any;
@@ -263,7 +329,7 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
 
   // Which browser the relay uses (or would use), for status reporting.
   async function describe(): Promise<ScreenMeta> {
-    if (meta.connected) return meta;
+    if (meta.connected) return { ...meta, audioListeners: listeners.size };
     const { browser, reason } = await pickBrowser();
     return browser ? { connected: false, browser: browser.key } : { connected: false, reason };
   }
@@ -287,7 +353,7 @@ export function createScreencast(opts: { browserKey?: string; herdrTab?: () => P
     return r && !r.exceptionDetails ? (r.result?.value ?? null) : null;
   }
 
-  return { handle, input, describe, setClip, evaluate, inspect };
+  return { handle, handleAudio, input, describe, setClip, evaluate, inspect };
 }
 
 export type InspectQuery = { x?: number; y?: number; selector?: string; full?: boolean };