hyperframes 0.2.2 → 0.2.3-alpha.1

package/dist/skills/{hyperframes-compose → hyperframes}/SKILL.md

@@ -1,22 +1,11 @@
 ---
-name: hyperframes-compose
-description: Create video compositions, animations, title cards, or overlays in HyperFrames HTML. Use when asked to build any HTML-based video content.
+name: hyperframes
+description: Create video compositions, animations, title cards, overlays, captions, voiceovers, audio-reactive visuals, and scene transitions in HyperFrames HTML. Use when asked to build any HTML-based video content, add captions or subtitles synced to audio, generate text-to-speech narration, create audio-reactive animation (beat sync, glow, pulse driven by music), add animated text highlighting (marker sweeps, hand-drawn circles, burst lines, scribble, sketchout), or add transitions between scenes (crossfades, wipes, reveals, shader transitions). Covers composition authoring, timing, media, and the full video production workflow. For CLI commands (init, lint, preview, render, transcribe, tts) see the hyperframes-cli skill.
 ---
 
-# Compose Video
+# HyperFrames
 
-HTML is the source of truth for video. A composition is an HTML file with `data-*` attributes for timing, a GSAP timeline for animation, and CSS for appearance. The framework handles clip visibility, media playback, and timeline sync.
-
-## Approach
-
-Before writing HTML, think at a high level:
-
-1. **What** — what should the viewer experience? Identify the narrative arc, key moments, and emotional beats.
-2. **Structure** — how many compositions, which are sub-compositions vs inline, what tracks carry what (video, audio, overlays, captions).
-3. **Timing** — which clips drive the duration, where do transitions land, what's the pacing.
-4. **Execute** — then implement using the rules below.
-
-For small edits (fix a color, adjust timing, add one element), skip straight to the rules.
+HTML is the source of truth for video. A composition is an HTML file with `data-*` attributes for timing, a GSAP timeline for animation, and CSS for appearance.
 
 When no `visual-style.md` or animation direction is provided, follow [house-style.md](./house-style.md) for motion defaults, sizing, and color palettes.
 
@@ -29,7 +18,7 @@ When no `visual-style.md` or animation direction is provided, follow [house-styl
 | `id`               | Yes                               | Unique identifier                                      |
 | `data-start`       | Yes                               | Seconds or clip ID reference (`"el-1"`, `"intro + 2"`) |
 | `data-duration`    | Required for img/div/compositions | Seconds. Video/audio defaults to media duration.       |
-| `data-track-index` | Yes                               | Integer. Same-track clips **cannot overlap**.          |
+| `data-track-index` | Yes                               | Integer. Same-track clips cannot overlap.              |
 | `data-media-start` | No                                | Trim offset into source (seconds)                      |
 | `data-volume`      | No                                | 0-1 (default 1)                                        |
 
@@ -46,7 +35,7 @@ When no `visual-style.md` or animation direction is provided, follow [house-styl
 
 ## Composition Structure
 
-Every composition is a `<template>` wrapping a `<div>` with `data-composition-id`. Each must include its own GSAP script and register its timeline:
+Every composition is a `<template>` wrapping a `<div>` with `data-composition-id`:
 
 ```html
 <template id="my-comp-template">
@@ -100,15 +89,14 @@ Video must be `muted playsinline`. Audio is always a separate `<audio>` element:
 - Register every timeline: `window.__timelines["<composition-id>"] = tl`
 - Framework auto-nests sub-timelines — do NOT manually add them
 - Duration comes from `data-duration`, not from GSAP timeline length
-- Never create empty tweens to set duration
 
 ## Rules (Non-Negotiable)
 
-**Deterministic:** No `Math.random()`, `Date.now()`, or time-based logic. The renderer must produce identical output every time.
+**Deterministic:** No `Math.random()`, `Date.now()`, or time-based logic.
 
 **GSAP:** Only animate visual properties (`opacity`, `x`, `y`, `scale`, `rotation`, `color`, `backgroundColor`, `borderRadius`, transforms). Do NOT animate `visibility`, `display`, or call `video.play()`/`audio.play()`.
 
-**Animation conflicts:** Never animate the same property on the same element from multiple timelines simultaneously — causes flickering in headless renders.
+**Animation conflicts:** Never animate the same property on the same element from multiple timelines simultaneously.
 
 **Never do:**
 
@@ -120,36 +108,48 @@ Video must be `muted playsinline`. Audio is always a separate `<audio>` element:
 6. Call play/pause/seek on media — framework owns playback
 7. Create a top-level container without `data-composition-id`
 
-## Editing Existing Compositions
-
-- Read the full composition first — match existing fonts, colors, animation patterns
-- Only change what was requested — don't rewrite untouched sections
-- Don't rewrite entire files for small changes
-- Preserve timing of unrelated clips
-
 ## Typography and Assets
 
-- Every composition loads its own fonts (`@import` or `@font-face` in `<style>`)
-- Use `font-display: block` for local fonts — renderer needs fonts loaded before capturing
-- Add `crossorigin="anonymous"` to media loaded from external URLs
+- Every composition loads its own fonts (`@import` or `@font-face`)
+- Use `font-display: block` for local fonts
+- Add `crossorigin="anonymous"` to external media
 - Minimum readable text: 20px landscape, 18px portrait
-- For dynamic text that may overflow its container, use `window.__hyperframes.fitTextFontSize(text, { maxWidth, fontFamily, fontWeight })` to compute the largest font size that fits on one line. Returns `{ fontSize, fits }`. Options: `baseFontSize` (default 78), `minFontSize` (default 42), `step` (default 2). `fontFamily` and `fontWeight` must match the CSS applied to the element.
-- All files (video, audio, fonts, images) live at the project root alongside `index.html`
-- From sub-compositions, use `../` to reference root files
+- For dynamic text overflow, use `window.__hyperframes.fitTextFontSize(text, { maxWidth, fontFamily, fontWeight })` — returns `{ fontSize, fits }`
+- All files live at the project root alongside `index.html`; sub-compositions use `../`
 
-For PiP, title cards, and slide show patterns, see [patterns.md](./patterns.md).
-For data, stats, and infographics, see [data-in-motion.md](./data-in-motion.md).
-For typewriter text and other GSAP animation effects, see the `gsap-effects` skill.
-For audio-driven animation (beat sync, glow, pulse), see the `audio-reactive` skill.
+## Editing Existing Compositions
+
+- Read the full composition first — match existing fonts, colors, animation patterns
+- Only change what was requested
+- Preserve timing of unrelated clips
 
 ## Output Checklist
 
-- [ ] Every top-level container has `data-composition-id`
-- [ ] Every composition has `data-width`, `data-height`, `data-duration`
+- [ ] Every top-level container has `data-composition-id`, `data-width`, `data-height`, `data-duration`
 - [ ] Compositions in own HTML files, loaded via `data-composition-src`
 - [ ] `<template>` wrapper on sub-compositions
 - [ ] `window.__timelines` registered for every composition
-- [ ] 100% deterministic — no randomness
-- [ ] Each composition includes GSAP: `<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>`
-- [ ] `npx hyperframes lint` passes with 0 errors
-- [ ] `npx hyperframes validate` passes with 0 errors (run both before opening the studio)
+- [ ] 100% deterministic
+- [ ] Each composition includes GSAP script tag
+- [ ] `npx hyperframes lint` and `npx hyperframes validate` both pass
+
+---
+
+## References (loaded on demand)
+
+- **[references/captions.md](references/captions.md)** — Captions, subtitles, lyrics, karaoke synced to audio. Tone-adaptive style detection, per-word styling, text overflow prevention, caption exit guarantees, word grouping. Read when adding any text synced to audio timing.
+- **[references/tts.md](references/tts.md)** — Text-to-speech with Kokoro-82M. Voice selection, speed tuning, TTS+captions workflow. Read when generating narration or voiceover.
+- **[references/audio-reactive.md](references/audio-reactive.md)** — Audio-reactive animation: map frequency bands and amplitude to GSAP properties. Read when visuals should respond to music, voice, or sound.
+- **[references/marker-highlight.md](references/marker-highlight.md)** — Animated text highlighting via canvas overlays: marker pen, circle, burst, scribble, sketchout. Read when adding visual emphasis to text.
+- **[house-style.md](house-style.md)** — Default motion, sizing, and color palettes when no style is specified.
+- **[patterns.md](patterns.md)** — PiP, title cards, slide show patterns.
+- **[data-in-motion.md](data-in-motion.md)** — Data, stats, and infographic patterns.
+- **[references/transcript-guide.md](references/transcript-guide.md)** — Transcription commands, whisper models, external APIs, troubleshooting.
+- **[references/dynamic-techniques.md](references/dynamic-techniques.md)** — Dynamic caption animation techniques (karaoke, clip-path, slam, scatter, elastic, 3D).
+
+- **[references/transitions.md](references/transitions.md)** — Scene transitions: crossfades, wipes, reveals, shader transitions. Energy/mood selection, narrative position, CSS vs WebGL guidance. Read when a composition has multiple scenes that need visual handoffs.
+  - [transitions/catalog.md](references/transitions/catalog.md) — Hard rules, scene template, and routing to per-type implementation code.
+  - [transitions/shader-setup.md](references/transitions/shader-setup.md) — WebGL boilerplate for shader transitions.
+  - [transitions/shader-transitions.md](references/transitions/shader-transitions.md) — 14 fragment shaders.
+
+GSAP patterns and effects are in the `/gsap` skill.

package/dist/skills/hyperframes/references/audio-reactive.md

@@ -0,0 +1,76 @@
+# Audio-Reactive Animation
+
+Drive visuals from music, voice, or sound. Any GSAP-animatable property can respond to pre-extracted audio data.
+
+## Audio Data Format
+
+```js
+var AUDIO_DATA = {
+  fps: 30,
+  totalFrames: 900,
+  frames: [{ bands: [0.82, 0.45, 0.31, ...] }, ...]
+};
+```
+
+- `frames[i].bands[]` — frequency band amplitudes, 0-1. Index 0 = bass, higher = treble.
+- Each band normalized independently across the full track.
+
+## Mapping Audio to Visuals
+
+| Audio signal           | Visual property                   | Effect                     |
+| ---------------------- | --------------------------------- | -------------------------- |
+| Bass (bands[0])        | `scale`                           | Pulse on beat              |
+| Treble (bands[12-14])  | `textShadow`, `boxShadow`         | Glow intensity             |
+| Overall amplitude      | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift |
+| Mid-range (bands[4-8]) | `borderRadius`, `width`           | Shape morphing             |
+
+Any GSAP-tweenable property works — `clipPath`, `filter`, SVG attributes, CSS custom properties.
+
+## Content, Not Medium
+
+Audio provides **timing and intensity**. The visual vocabulary comes from the narrative.
+
+**Never add:** equalizer bars, spectrum analyzers, waveform displays, musical notes clip art, generic particle systems, rainbow color cycling, strobing white on beats, abstract pulsing orbs.
+
+**Instead:** Let content guide the visual and audio drive its behavior. Bass makes warmth _swell_. Treble sharpens _contrast_. The visual choice comes from "what does this piece feel like?"
+
+## Sampling Pattern
+
+Audio reactivity requires per-frame sampling via a `for` loop with `tl.call()`, not a single tween:
+
+```js
+// ✅ Correct — sample every frame
+for (var f = 0; f < AUDIO_DATA.totalFrames; f++) {
+  tl.call(
+    (function (frame) {
+      return function () {
+        draw(frame);
+      };
+    })(AUDIO_DATA.frames[f]),
+    [],
+    f / AUDIO_DATA.fps,
+  );
+}
+
+// ❌ Wrong — single tween, doesn't react to audio
+gsap.to(".el", { scale: 1.2, duration: totalDuration });
+```
+
+Without per-frame sampling, the composition doesn't actually react to audio.
+
+## textShadow Gotcha
+
+`textShadow` on a parent container with semi-transparent children (e.g., inactive caption words at `rgba(255,255,255,0.3)`) renders a visible glow rectangle behind all children. Fix: apply `scale` to the container for beat pulse, but apply `textShadow` to individual active words only.
+
+## Guidelines
+
+- **Subtlety for text** — 3-6% scale variation, soft glow. Heavy pulsing makes text unreadable.
+- **Go bigger on non-text** — backgrounds and shapes can handle 10-30% swings.
+- **Match the energy** — corporate = subtle; music video = dramatic.
+- **Deterministic** — pre-extracted data, no Web Audio API, no runtime analysis.
+
+## Constraints
+
+- All audio data must be pre-extracted (use `extract-audio-data.py` from the gsap skill's scripts/)
+- No `Math.random()` or `Date.now()`
+- Audio reactivity runs on the same GSAP timeline as everything else

package/dist/skills/hyperframes/references/captions.md

@@ -0,0 +1,132 @@
+# Captions
+
+## Language Rule (Non-Negotiable)
+
+**Never use `.en` models unless the user explicitly states the audio is English.** `.en` models TRANSLATE non-English audio into English instead of transcribing it.
+
+1. User says the language → `--model small --language <code>` (no `.en`)
+2. User says English → `--model small.en`
+3. Language unknown → `--model small` (no `.en`, no `--language`) — auto-detects
+
+---
+
+Analyze spoken content to determine caption style. If user specifies a style, use that. Otherwise, detect tone from the transcript.
+
+## Transcript Source
+
+```json
+[
+  { "text": "Hello", "start": 0.0, "end": 0.5 },
+  { "text": "world.", "start": 0.6, "end": 1.2 }
+]
+```
+
+For transcription commands, whisper models, external APIs, see [transcript-guide.md](transcript-guide.md).
+
+## Style Detection (When No Style Specified)
+
+Read the full transcript before choosing. Four dimensions:
+
+**1. Visual feel** — corporate→clean; energetic→bold; storytelling→elegant; technical→precise; social→playful.
+
+**2. Color palette** — dark+bright for energy; muted for professional; high contrast for clarity; one accent color.
+
+**3. Font mood** — heavy/condensed for impact; clean sans for modern; rounded for friendly; serif for elegance.
+
+**4. Animation character** — scale-pop for punchy; gentle fade for calm; word-by-word for emphasis; typewriter for technical.
+
+## Per-Word Styling
+
+Scan for words deserving distinct treatment:
+
+- **Brand/product names** — larger size, unique color
+- **ALL CAPS** — scale boost, flash, accent color
+- **Numbers/statistics** — bold weight, accent color
+- **Emotional keywords** — exaggerated animation (overshoot, bounce)
+- **Call-to-action** — highlight, underline, color pop
+- **Marker highlight** — for beyond-color emphasis, see [marker-highlight.md](marker-highlight.md)
+
+## Script-to-Style Mapping
+
+| Tone         | Font mood                | Animation                          | Color                       | Size    |
+| ------------ | ------------------------ | ---------------------------------- | --------------------------- | ------- |
+| Hype/launch  | Heavy condensed, 800-900 | Scale-pop, back.out(1.7), 0.1-0.2s | Bright on dark              | 72-96px |
+| Corporate    | Clean sans, 600-700      | Fade+slide, power3.out, 0.3s       | White/neutral, muted accent | 56-72px |
+| Tutorial     | Mono/clean sans, 500-600 | Typewriter/fade, 0.4-0.5s          | High contrast, minimal      | 48-64px |
+| Storytelling | Serif/elegant, 400-500   | Slow fade, power2.out, 0.5-0.6s    | Warm muted tones            | 44-56px |
+| Social       | Rounded sans, 700-800    | Bounce, elastic.out, word-by-word  | Playful, colored pills      | 56-80px |
+
+## Word Grouping
+
+- **High energy:** 2-3 words. Quick turnover.
+- **Conversational:** 3-5 words. Natural phrases.
+- **Measured/calm:** 4-6 words. Longer groups.
+
+Break on sentence boundaries, 150ms+ pauses, or max word count.
+
+## Positioning
+
+- **Landscape (1920x1080):** Bottom 80-120px, centered
+- **Portrait (1080x1920):** Lower middle ~600-700px from bottom, centered
+- Never cover the subject's face
+- `position: absolute` — never relative
+- One caption group visible at a time
+
+## Text Overflow Prevention
+
+Use `window.__hyperframes.fitTextFontSize()`:
+
+```js
+var result = window.__hyperframes.fitTextFontSize(group.text.toUpperCase(), {
+  fontFamily: "Outfit",
+  fontWeight: 900,
+  maxWidth: 1600,
+});
+el.style.fontSize = result.fontSize + "px";
+```
+
+Options: `maxWidth` (1600 landscape, 900 portrait), `baseFontSize` (78), `minFontSize` (42), `fontWeight`, `fontFamily`, `step` (2).
+
+CSS safety nets: `max-width` on container, `overflow: visible` (**not** `hidden` — hidden clips scaled emphasis words and glow effects), `position: absolute`, explicit `height`. When per-word styling uses `scale > 1.0`, compute `maxWidth = safeWidth / maxScale` to leave headroom.
+
+**Container pattern:** Full-width absolute container, centered. Do **not** use `left: 50%; transform: translateX(-50%)` — causes clipping at composition edges.
+
+## Caption Exit Guarantee
+
+Every group **must** have a hard kill after exit animation:
+
+```js
+tl.to(groupEl, { opacity: 0, scale: 0.95, duration: 0.12, ease: "power2.in" }, group.end - 0.12);
+tl.set(groupEl, { opacity: 0, visibility: "hidden" }, group.end); // deterministic kill
+```
+
+Self-lint after building timeline — place **before** `window.__timelines[id] = tl` so it runs at composition init:
+
+```js
+GROUPS.forEach(function (group, gi) {
+  var el = document.getElementById("cg-" + gi);
+  if (!el) return;
+  tl.seek(group.end + 0.01);
+  var computed = window.getComputedStyle(el);
+  if (computed.opacity !== "0" && computed.visibility !== "hidden") {
+    console.warn(
+      "[caption-lint] group " + gi + " still visible at t=" + (group.end + 0.01).toFixed(2) + "s",
+    );
+  }
+});
+tl.seek(0);
+```
+
+## Further References
+
+- [dynamic-techniques.md](dynamic-techniques.md) — karaoke, clip-path reveals, slam words, scatter exits, elastic, 3D rotation
+- [transcript-guide.md](transcript-guide.md) — transcription commands, whisper models, external APIs
+- [marker-highlight.md](marker-highlight.md) — animated text emphasis paired with per-word styling
+
+## Constraints
+
+- Deterministic. No `Math.random()`, no `Date.now()`.
+- Sync to transcript timestamps.
+- One group visible at a time.
+- Every group must have a hard `tl.set` kill at `group.end`.
+- Check project root for font files before defaulting to Google Fonts.