@mineo71/showtime 0.2.1 → 0.4.0

package/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 0.4.0 — 2026-10-13
+
+- Narration: `show.say` on any step is spoken by a TTS engine (`audio.voice`: the macOS/Linux
+  system voice, OpenAI, ElevenLabs or any command). The video holds the frame until a line ends.
+  Lines are cached; editing them needs no new take.
+- `audio.music`: a looped music bed that ducks under the voice. `audio.sounds`: click and
+  keystroke sounds (takes now log keystrokes; older takes spread them over each `type` step).
+- Subtitles from the narration: an `.srt` next to the video, or burned in (`audio.subtitles`).
+- `--aspect 9:16 | 1:1` on `render`, `make` and `frames`: cropped cuts with a camera that follows
+  the action, from the same take. Stills of a crop go to `frames-9x16/` or `frames-1x1/`.
+- `--format gif | webp` (or an `-o` with that extension) and `--width`; `--no-audio`. Silent
+  outputs skip narration (no voice needed) unless subtitles are burned in.
+- `render --json` adds `subtitles`; the brand file accepts `audio`.
+- `lint`: `narration-stretch` flags lines that hold the frame for long.
+
+## 0.3.0 — 2026-10-13
+
+- `showtime frames` also renders a contact sheet (`frames/sheet.png`) of every step.
+- `showtime lint`: long captions, pauses that should be waits, zoom/spotlight that cannot apply,
+  and, with a take, captions too brief to read and overall length.
+- `--json` on every command.
+- `hooks.before` / `hooks.after`: shell commands or HTTP calls around every `check` and `record`,
+  so state can be reset between rehearsal and take.
+- `app.mock` (canned responses for matching requests), `app.clock` (the page clock starts at a fixed date),
+  `app.dialogs` (accept or dismiss native dialogs; accepted by default).
+- Safe re-takes: a failed `record` leaves the previous take untouched (`take.tmp` keeps the failure).
+- Settling ignores event streams; relative `goto` URLs resolve against `app.url`.
+- Captions never overlap, and move to the top when the action is at the bottom of the screen.
+- `meta.outro` closing card and `meta.logo` on the intro and outro cards.
+- `showtime.config.json` brand file with default `style` and `meta` for every script below it.
+- Takes recorded with 0.2 stay valid.
+
 ## 0.2.1 — 2026-10-13
 
 - Releases are automated: `npm version patch|minor|major` dates the changelog, tags and pushes;

package/README.md

@@ -8,7 +8,8 @@
 
 Your AI agent (Claude Code or Codex) writes a short JSON script. showTime clicks through your web app, films it and edits it
 the way a motion designer would (auto-zoom, a smooth cursor, click ripples, spotlights, captions,
-sped-up loading) and hands you a polished 1080p MP4.<br />
+sped-up loading, narration) and hands you a polished video: a 1080p MP4, a 9:16 cut for social or
+a GIF for your README.<br />
 No timeline editor. No re-take when a caption changes.
 
 [![tests](https://img.shields.io/github/actions/workflow/status/mineo71/showTime/ci.yml?branch=main&style=flat-square&label=tests)](https://github.com/mineo71/showTime/actions/workflows/ci.yml)
@@ -26,7 +27,7 @@ No timeline editor. No re-take when a caption changes.
 <div align="center">
   <a href="docs/media/demo.mp4"><img src="docs/media/hero.webp" width="860" alt="A demo of a fictional invoicing app, filmed and edited by showTime: the camera zooms on a chart, a cursor fills in a new invoice, a spotlight lands on the total, and the sent invoice appears in the table" /></a>
   <br />
-  <sub>Made by <a href="examples/fennel/demo.json">one 44-line script</a> and one command, with no manual editing. <a href="docs/media/demo.mp4">▶ Full-quality MP4</a></sub>
+  <sub>Made by <a href="examples/fennel/demo.json">a 39-line script</a>, a <a href="examples/fennel/showtime.config.json">brand file</a> and one command, with no manual editing. <a href="docs/media/demo.mp4">▶ Full-quality MP4, with sound</a></sub>
 </div>
 
 ---
@@ -121,7 +122,40 @@ again.
     <td valign="top">
       <img src="docs/media/intro.jpg" alt="Intro card with title and subtitle" /><br />
       <b>🎬 Intro card and window chrome</b><br />
-      <sub>Title card, a browser frame with your URL, a gradient backdrop, rounded corners and a shadow. Each of them is a <code>style</code> field.</sub>
+      <sub>Title card with your logo, a browser frame with your URL, a gradient backdrop, rounded corners and a shadow. Each of them is a <code>style</code> or <code>meta</code> field.</sub>
+    </td>
+  </tr>
+  <tr>
+    <td valign="top">
+<pre lang="json">
+"show": { "say": "Send it before lunch." }
+
+"audio": {
+  "voice": { "engine": "openai", "name": "nova" },
+  "music": { "file": "bed.mp3" },
+  "sounds": true,
+  "subtitles": "burn"
+}
+</pre>
+      <b>🎙️ Narration, music and clicks</b><br />
+      <sub>Give a step a line and a voice reads it: the built-in voice of macOS or Linux, OpenAI, ElevenLabs or any TTS command. The video holds the frame until the line ends, music ducks under the voice, and subtitles come as an <code>.srt</code> or burned in.</sub>
+    </td>
+    <td valign="top">
+      <img src="docs/media/formats.jpg" alt="The same moment in 16:9, 9:16 and 1:1" /><br />
+      <b>📱 Every format from one take</b><br />
+      <sub><code>--aspect 9:16</code> or <code>1:1</code> crops the window and a follow camera keeps the action in view. <code>--format gif</code> or <code>webp</code> for READMEs and docs. Nothing is re-recorded.</sub>
+    </td>
+  </tr>
+  <tr>
+    <td valign="top">
+      <img src="docs/media/outro.jpg" alt="Outro card with logo, title and URL" /><br />
+      <b>📣 Ends on your call to action</b><br />
+      <sub>An outro card with your logo, a headline and the URL to visit. Put it in a brand file once and every demo ends the same way.</sub>
+    </td>
+    <td valign="top">
+      <img src="docs/media/sheet.jpg" alt="Contact sheet of every step" /><br />
+      <b>🧪 Reviews its own work</b><br />
+      <sub><code>showtime frames</code> renders a contact sheet of every step, and <code>showtime lint</code> flags captions that are too long or too brief to read. The agent fixes them before you look.</sub>
     </td>
   </tr>
 </table>
@@ -132,8 +166,16 @@ And under the hood:
   When a step fails it prints an ARIA snapshot of the page, so Claude can fix its own selectors.
 - 🔁 **Stale-take detection.** The take stores a hash of the actions. Edit a caption and `render`
   reuses the take. Edit a click and `render` tells you to `record` again.
-- 🧪 **Self-review.** `showtime frames` renders a still per step so the agent can *look* at its
-  work before you do.
+- 🧹 **Clean state for every take.** `hooks` run a reset command or call an endpoint before and
+  after each rehearsal and take, so a flow that creates data works twice in a row.
+- 🎭 **Deterministic pages.** `app.mock` answers flaky or slow APIs with canned JSON, `app.clock`
+  pins the date, and `app.dialogs` answers `confirm()` the way the demo needs.
+- 🎨 **One brand file.** `showtime.config.json` holds your colours, logo and outro; every script
+  below it inherits them.
+- 🛟 **Safe re-takes.** A take that fails halfway leaves the last good one untouched.
+- 🤖 **JSON everywhere.** Every command takes `--json`, so agents read results instead of parsing
+  prose. Captions also move to the top on their own when the action happens at the bottom of the
+  screen.
 - 📐 **Typed scripts.** Point `"$schema"` at [`docs/script.schema.json`](docs/script.schema.json)
   for autocomplete and validation in your editor.
 - 🖥️ **Crisp at 1080p.** The app is captured at 2× device scale, so zoomed shots stay sharp.
@@ -169,7 +211,8 @@ The agent takes it from there:
 3. **Rehearses** with `showtime check` until every step passes, fixing selectors from the ARIA
    snapshot.
 4. **Records and renders** with `showtime record` and `showtime render`.
-5. **Reviews** the stills from `showtime frames`, tightens captions and pacing, and re-renders.
+5. **Reviews** the contact sheet from `showtime frames` and the warnings from `showtime lint`,
+   tightens captions and pacing, and re-renders.
 
 The skill ([`skill/SKILL.md`](skill/SKILL.md)) is a plain-Markdown
 [Agent Skill](https://agentskills.io). Any agent that can run shell commands can follow it, and so
@@ -185,16 +228,19 @@ flowchart LR
   R --> E["events.json<br/>targets · clicks · idle"]
   S --> T["timeline.json<br/>camera · cursor · captions · speed"]
   E --> T
+  T --> A["mix.wav<br/>voice · music · clicks"]
   T --> M["Remotion"]
+  A --> M
   V --> M
-  M --> O["demo.mp4"]
+  M --> O["demo.mp4 · 9:16 · GIF"]
 ```
 
 The core idea is to **separate capture from direction**. `record` drives a real Chromium through
 the script and writes two things: a clean video with no cursor, and an event log of where every
 target was, when each click landed and when the page sat idle. A pure function,
 `buildTimeline(script, events)`, turns that log into speed segments, camera keyframes, cursor
-paths, captions and spotlights. Remotion renders the result frame by frame.
+paths, captions and spotlights, and holds the frame wherever a narration line needs more time.
+The soundtrack is mixed from the same timeline, and Remotion renders the result frame by frame.
 
 So presentation is cheap to change and recording stays honest: the app really is clicked, typed
 into and waited on, and everything cinematic is computed afterwards.
@@ -224,8 +270,9 @@ into and waited on, and everything cinematic is computed afterwards.
 }
 ```
 
-`meta.title` adds an intro card. `setup` steps run before the camera starts (cookie banners,
-onboarding). Scenes group steps for readability and for naming the stills.
+`meta.title` adds an intro card, `meta.outro` a closing card and `meta.logo` puts your logo on
+both. `setup` steps run before the camera starts (cookie banners, onboarding). Scenes group steps
+for readability and for naming the stills.
 
 <details>
 <summary><b>Actions</b> (<code>do</code>)</summary>
@@ -257,6 +304,7 @@ Targets are Playwright selectors: `role=button[name='Save']`, `text=Save`, `#id`
 | `spotlight` | boolean        | dim everything except the target |
 | `hold`      | ms             | extra real time after the step (changes the take) |
 | `speed`     | 0.25–16        | playback speed for this step |
+| `say`       | string         | narration spoken from the start of the step; the next narrated step waits for it |
 
 </details>
 
@@ -291,6 +339,83 @@ Targets are Playwright selectors: `role=button[name='Save']`, `text=Save`, `#id`
 | `colorScheme`       | none       | `light` or `dark`; also picks the browser-bar theme |
 | `locale`, `timezoneId` | none    | passed to the browser context |
 | `mask`              | `[]`       | CSS selectors to blur (plain CSS, not `text=`) |
+| `mock`              | `[]`       | `[{ url, method?, status?, json? \| body?, headers?, delayMs? }]`: answer matching requests (`url` is a glob like `**/api/stats*`; the first match wins) |
+| `clock`             | none       | start the page clock at this ISO date-time; it then runs normally |
+| `dialogs`           | `accept`   | answer `alert`/`confirm`/`prompt` with `accept` or `dismiss` |
+
+</details>
+
+<details>
+<summary><b>Sound</b> (<code>audio</code>)</summary>
+
+```json
+"audio": {
+  "voice": { "engine": "system", "name": "Samantha", "speed": 1 },
+  "music": { "file": "music/bed.mp3", "volume": 0.25 },
+  "sounds": true,
+  "subtitles": "file"
+}
+```
+
+| field                | default    | |
+|----------------------|------------|-|
+| `voice.engine`       | `system`   | `system` (macOS `say`, Linux `espeak-ng`), `openai`, `elevenlabs`, or `command` |
+| `voice.name`         | engine default | voice name (`system`, `openai`), or voice id (`elevenlabs`, required) |
+| `voice.speed`        | `1`        | 0.5–2 |
+| `voice.model`        | engine default | `gpt-4o-mini-tts` (OpenAI), `eleven_multilingual_v2` (ElevenLabs) |
+| `voice.instructions` | none       | OpenAI only: how to speak ("warm, confident") |
+| `voice.command`      | none       | `command` only: a shell command that reads the text on stdin and writes audio to `{out}` (inserted already quoted, so write it bare; also in `$SHOWTIME_OUT`), e.g. `piper -m en_US.onnx -f {out}` |
+| `music`              | none       | `{ file, volume? }`: a bed under the whole video, looped and ducked under the voice |
+| `sounds`             | `false`    | click and keystroke sounds |
+| `subtitles`          | `file`     | `file`: an `.srt` next to the video; `burn`: also on the video (captions move up); `off` |
+
+`openai` and `elevenlabs` read `OPENAI_API_KEY` / `ELEVENLABS_API_KEY` from the environment
+(`OPENAI_BASE_URL` points at any OpenAI-compatible server). Each line is synthesized once and
+cached; edits to `say` or `audio` only need `render`. Silent outputs (GIF, WebP, `--no-audio`) skip
+the narration and its pauses, unless subtitles are burned in.
+
+</details>
+
+<details>
+<summary><b>Cards</b> (<code>meta</code>)</summary>
+
+| field      | |
+|------------|-|
+| `title`, `subtitle` | intro card |
+| `logo`     | SVG, PNG, JPEG or WebP (≤ 2 MB), relative to the file that names it; shown on the intro and outro |
+| `outro`    | `{ title, subtitle?, url? }`: closing card after the video |
+
+</details>
+
+<details>
+<summary><b>Hooks</b> (<code>hooks</code>)</summary>
+
+```json
+"hooks": {
+  "before": [{ "run": "npm run db:reset" }],
+  "after": [{ "request": { "url": "http://localhost:3000/api/test/reset", "method": "POST" } }]
+}
+```
+
+`before` hooks run at the start of every `check` and `record` (so `make` resets between the
+rehearsal and the take); `after` hooks run when they end, even after a failure, and a failing
+`after` hook is a warning that never costs you the take. `run` is a shell
+command in the script's folder (120 s timeout); `request` is an HTTP call that must answer 2xx.
+
+</details>
+
+<details>
+<summary><b>Brand file</b> (<code>showtime.config.json</code>)</summary>
+
+Put it next to your scripts or in any folder above them; the nearest one applies. Its `style`,
+`meta` and `audio` are defaults, and a script's own fields win key by key.
+
+```json
+{
+  "style": { "background": "linear-gradient(135deg, #0f172a, #312e81)", "accent": "#6366f1", "address": "app.acme.com" },
+  "meta": { "logo": "brand/logo.svg", "outro": { "title": "Try Acme free", "url": "acme.com" } }
+}
+```
 
 </details>
 
@@ -299,14 +424,17 @@ Targets are Playwright selectors: `role=button[name='Save']`, `text=Save`, `#id`
 | command | what it does |
 |---|---|
 | `showtime setup [--agent claude\|codex\|all] [--no-browsers]` | download the browsers, link the skill |
-| `showtime make <script> [-o out.mp4]` | check → record → render |
+| `showtime make <script> [render options]` | check → record → render |
 | `showtime check <script> [--json] [--headed]` | rehearse; a screenshot per step; the first failure with an ARIA snapshot |
 | `showtime record <script> [--headed] [--keep-frames]` | record a take into `.showtime/<name>/take` |
-| `showtime render <script> [-o out.mp4]` | render from the last take, without re-recording |
-| `showtime frames <script>` | QA stills: the intro plus one per step |
+| `showtime render <script> [-o out] [--aspect 16:9\|9:16\|1:1] [--format mp4\|gif\|webp] [--width px] [--no-audio]` | render from the last take, without re-recording |
+| `showtime frames <script> [--aspect …]` | QA stills (intro, one per step, outro) and a contact sheet of them |
+| `showtime lint <script>` | warnings about captions, pauses and length; timing checks need a take |
 | `showtime schema` | print the script's JSON Schema |
 
-The video lands next to the script (`demo/pitch.json` → `demo/pitch.mp4`). Working files live in
+Every command takes `--json` and then prints exactly one JSON document. The video lands next to
+the script (`demo/pitch.json` → `demo/pitch.mp4`, `demo/pitch-9x16.mp4`, `demo/pitch.gif`), with
+`demo/pitch.srt` when it is narrated. Working files live in
 `.showtime/<name>/`, which is worth adding to your `.gitignore`.
 
 ## Tips for a great demo
@@ -317,15 +445,22 @@ The video lands next to the script (`demo/pitch.json` → `demo/pitch.mp4`). Wor
   `spotlight` on the one number that sells it.
 - **Never `pause` for loading.** Use `wait` with a selector and auto-speed compresses the spinner.
 - **Start clean.** Use `setup` for cookie banners, `app.storageState` to skip the login, `app.mask`
-  for anything personal, and seed data you can reset between `check` and `record`.
+  for anything personal, and `hooks` to reset the data between `check` and `record`.
+- **Pin the world.** Mock slow or random APIs with `app.mock` and pin the date with `app.clock`,
+  so every take looks the same.
+- **Narrate the why, not the clicks.** One short sentence per moment. If `showtime lint` says a
+  line holds the frame for seconds, shorten it or give the step more to show.
 
 ## FAQ
 
 **Which apps does it work with?** Any web app Chromium can open: a localhost dev server, staging
 or production. Logged-in flows work through `app.storageState`.
 
-**Does showTime call an AI model?** No. showTime is deterministic and needs no API keys. The AI
-part (writing, fixing and reviewing the script) happens in your agent, guided by the skill.
+**Does showTime call an AI model?** Not for directing: the AI part (writing, fixing and reviewing
+the script) happens in your agent, guided by the skill. Narration is text-to-speech. The built-in
+voice of macOS or Linux needs no key and nothing leaves your machine; with `openai` or
+`elevenlabs`, only the narration text is sent to that provider, with your key from the
+environment.
 
 **Why not just screen-record?** You can, once. A screen recording can't be re-edited when the copy
 changes, the cursor is wherever your hand was, and the spinner plays at full length. With
@@ -336,8 +471,9 @@ needs Playwright and ffmpeg. Windows is untested.
 
 ## Roadmap
 
-- 🎙️ Narration: text-to-speech per scene, with steps stretched to fit the audio, and burnt-in subtitles.
-- 📱 Vertical 9:16 and square 1:1 cuts with a follow-cam, from the same take.
+- 🧑‍🎨 `showtime studio`: a local page with a live player and a timeline to drag captions, zooms and
+  speed, saving back to the JSON.
+- 🎥 `showtime record-human <url>`: click through the app yourself and get a script back.
 - 🎞️ Deterministic frame-by-frame capture in virtual time, for a perfect 60 fps.
 - 🧩 An MCP server for Claude Desktop, and a desktop studio for timeline tweaks.
 - 📲 Mobile (iOS Simulator) and desktop-app drivers that produce the same `events.json`.