hyperframes 0.2.2 → 0.2.3-alpha.2

package/dist/skills/hyperframes/SKILL.md

@@ -0,0 +1,175 @@
+---
+name: hyperframes
+description: Create video compositions, animations, title cards, overlays, captions, voiceovers, audio-reactive visuals, and scene transitions in HyperFrames HTML. Use when asked to build any HTML-based video content, add captions or subtitles synced to audio, generate text-to-speech narration, create audio-reactive animation (beat sync, glow, pulse driven by music), add animated text highlighting (marker sweeps, hand-drawn circles, burst lines, scribble, sketchout), or add transitions between scenes (crossfades, wipes, reveals, shader transitions). Covers composition authoring, timing, media, and the full video production workflow. For CLI commands (init, lint, preview, render, transcribe, tts) see the hyperframes-cli skill.
+---
+
+# HyperFrames
+
+HTML is the source of truth for video. A composition is an HTML file with `data-*` attributes for timing, a GSAP timeline for animation, and CSS for appearance.
+
+When no `visual-style.md` or animation direction is provided, follow [house-style.md](./house-style.md) for motion defaults, sizing, and color palettes.
+
+## Data Attributes
+
+### All Clips
+
+| Attribute          | Required                          | Values                                                 |
+| ------------------ | --------------------------------- | ------------------------------------------------------ |
+| `id`               | Yes                               | Unique identifier                                      |
+| `data-start`       | Yes                               | Seconds or clip ID reference (`"el-1"`, `"intro + 2"`) |
+| `data-duration`    | Required for img/div/compositions | Seconds. Video/audio defaults to media duration.       |
+| `data-track-index` | Yes                               | Integer. Same-track clips cannot overlap.              |
+| `data-media-start` | No                                | Trim offset into source (seconds)                      |
+| `data-volume`      | No                                | 0-1 (default 1)                                        |
+
+`data-track-index` does **not** affect visual layering — use CSS `z-index`.
+
+### Composition Clips
+
+| Attribute                    | Required | Values                                       |
+| ---------------------------- | -------- | -------------------------------------------- |
+| `data-composition-id`        | Yes      | Unique composition ID                        |
+| `data-duration`              | Yes      | Takes precedence over GSAP timeline duration |
+| `data-width` / `data-height` | Yes      | Pixel dimensions (1920x1080 or 1080x1920)    |
+| `data-composition-src`       | No       | Path to external HTML file                   |
+
+## Composition Structure
+
+Every composition is a `<template>` wrapping a `<div>` with `data-composition-id`:
+
+```html
+<template id="my-comp-template">
+  <div data-composition-id="my-comp" data-width="1920" data-height="1080">
+    <!-- content -->
+    <style>
+      [data-composition-id="my-comp"] {
+        /* scoped styles */
+      }
+    </style>
+    <script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
+    <script>
+      window.__timelines = window.__timelines || {};
+      const tl = gsap.timeline({ paused: true });
+      // tweens...
+      window.__timelines["my-comp"] = tl;
+    </script>
+  </div>
+</template>
+```
+
+Load in root: `<div id="el-1" data-composition-id="my-comp" data-composition-src="compositions/my-comp.html" data-start="0" data-duration="10" data-track-index="1"></div>`
+
+## Video and Audio
+
+Video must be `muted playsinline`. Audio is always a separate `<audio>` element:
+
+```html
+<video
+  id="el-v"
+  data-start="0"
+  data-duration="30"
+  data-track-index="0"
+  src="video.mp4"
+  muted
+  playsinline
+></video>
+<audio
+  id="el-a"
+  data-start="0"
+  data-duration="30"
+  data-track-index="2"
+  src="video.mp4"
+  data-volume="1"
+></audio>
+```
+
+## Timeline Contract
+
+- All timelines start `{ paused: true }` — the player controls playback
+- Register every timeline: `window.__timelines["<composition-id>"] = tl`
+- Framework auto-nests sub-timelines — do NOT manually add them
+- Duration comes from `data-duration`, not from GSAP timeline length
+
+## Rules (Non-Negotiable)
+
+**Deterministic:** No `Math.random()`, `Date.now()`, or time-based logic. Use a seeded PRNG if you need pseudo-random values (e.g. mulberry32).
+
+**GSAP:** Only animate visual properties (`opacity`, `x`, `y`, `scale`, `rotation`, `color`, `backgroundColor`, `borderRadius`, transforms). Do NOT animate `visibility`, `display`, or call `video.play()`/`audio.play()`.
+
+**Animation conflicts:** Never animate the same property on the same element from multiple timelines simultaneously.
+
+**No `repeat: -1`:** Infinite-repeat timelines break the capture engine. Calculate the exact repeat count from composition duration: `repeat: Math.ceil(duration / cycleDuration) - 1`.
+
+**Synchronous timeline construction:** Never build timelines inside `async`/`await`, `setTimeout`, or Promises. The capture engine reads `window.__timelines` synchronously after page load. If you need fonts loaded first, use a synchronous `document.fonts.load()` call or rely on `font-display: block` — the engine waits for the page `load` event.
+
+**Never do:**
+
+1. Forget `window.__timelines` registration
+2. Use video for audio — always muted video + separate `<audio>`
+3. Nest video inside a timed div — use a non-timed wrapper
+4. Use `data-layer` (use `data-track-index`) or `data-end` (use `data-duration`)
+5. Animate video element dimensions — animate a wrapper div
+6. Call play/pause/seek on media — framework owns playback
+7. Create a top-level container without `data-composition-id`
+8. Use `repeat: -1` on any timeline or tween — always finite repeats
+9. Build timelines asynchronously (inside `async`, `setTimeout`, `Promise`)
+
+## Typography and Assets
+
+- Load fonts via `<link>` tags with `display=block` in `<head>`, NOT via CSS `@import` — `@import` is async and may not complete before the first frame capture
+- Use `font-display: block` for `@font-face` declarations
+- Add `crossorigin="anonymous"` to external media
+- **Minimum font sizes for rendered video (1080p at DPR 1):**
+  - Body/label text: 20px minimum (landscape), 18px minimum (portrait)
+  - Data labels, axis labels, footnotes: 16px minimum — anything smaller becomes illegible after encoding
+  - Headlines: 36px+ recommended
+  - Avoid sub-14px text entirely — it will be unreadable in the final MP4
+- For dynamic text overflow, use `window.__hyperframes.fitTextFontSize(text, { maxWidth, fontFamily, fontWeight })` — returns `{ fontSize, fits }`
+- All files live at the project root alongside `index.html`; sub-compositions use `../`
+
+### Backgrounds and Color
+
+- **Avoid full-screen linear gradients on dark backgrounds** — H.264 encoding creates visible color banding. Prefer: solid colors, radial gradients with limited range, or subtle noise/texture overlays to break up banding.
+- For dark themes, use solid `#000` or `#0A0A0A` with localized radial glows rather than a linear gradient spanning the full viewport.
+
+## Editing Existing Compositions
+
+- Read the full composition first — match existing fonts, colors, animation patterns
+- Only change what was requested
+- Preserve timing of unrelated clips
+
+## Output Checklist
+
+- [ ] Every top-level container has `data-composition-id`, `data-width`, `data-height`, `data-duration`
+- [ ] Compositions in own HTML files, loaded via `data-composition-src`
+- [ ] `<template>` wrapper on sub-compositions
+- [ ] `window.__timelines` registered for every composition
+- [ ] Timeline construction is synchronous (no async/await wrapping timeline code)
+- [ ] No `repeat: -1` on any tween or nested timeline
+- [ ] No text below 16px (data labels, footnotes) or 20px (body text)
+- [ ] No full-screen linear dark gradients (use radial or solid + localized glow)
+- [ ] Fonts loaded via `<link>` with `display=block`, not CSS `@import`
+- [ ] 100% deterministic
+- [ ] Each composition includes GSAP script tag
+- [ ] `npx hyperframes lint` and `npx hyperframes validate` both pass
+
+---
+
+## References (loaded on demand)
+
+- **[references/captions.md](references/captions.md)** — Captions, subtitles, lyrics, karaoke synced to audio. Tone-adaptive style detection, per-word styling, text overflow prevention, caption exit guarantees, word grouping. Read when adding any text synced to audio timing.
+- **[references/tts.md](references/tts.md)** — Text-to-speech with Kokoro-82M. Voice selection, speed tuning, TTS+captions workflow. Read when generating narration or voiceover.
+- **[references/audio-reactive.md](references/audio-reactive.md)** — Audio-reactive animation: map frequency bands and amplitude to GSAP properties. Read when visuals should respond to music, voice, or sound.
+- **[references/marker-highlight.md](references/marker-highlight.md)** — Animated text highlighting via canvas overlays: marker pen, circle, burst, scribble, sketchout. Read when adding visual emphasis to text.
+- **[house-style.md](house-style.md)** — Default motion, sizing, and color palettes when no style is specified.
+- **[patterns.md](patterns.md)** — PiP, title cards, slide show patterns.
+- **[data-in-motion.md](data-in-motion.md)** — Data, stats, and infographic patterns.
+- **[references/transcript-guide.md](references/transcript-guide.md)** — Transcription commands, whisper models, external APIs, troubleshooting.
+- **[references/dynamic-techniques.md](references/dynamic-techniques.md)** — Dynamic caption animation techniques (karaoke, clip-path, slam, scatter, elastic, 3D).
+
+- **[references/transitions.md](references/transitions.md)** — Scene transitions: crossfades, wipes, reveals, shader transitions. Energy/mood selection, narrative position, CSS vs WebGL guidance. Read when a composition has multiple scenes that need visual handoffs.
+  - [transitions/catalog.md](references/transitions/catalog.md) — Hard rules, scene template, and routing to per-type implementation code.
+  - [transitions/shader-setup.md](references/transitions/shader-setup.md) — WebGL boilerplate for shader transitions.
+  - [transitions/shader-transitions.md](references/transitions/shader-transitions.md) — 14 fragment shaders.
+
+GSAP patterns and effects are in the `/gsap` skill.

package/dist/skills/hyperframes/references/audio-reactive.md

@@ -0,0 +1,76 @@
+# Audio-Reactive Animation
+
+Drive visuals from music, voice, or sound. Any GSAP-animatable property can respond to pre-extracted audio data.
+
+## Audio Data Format
+
+```js
+var AUDIO_DATA = {
+  fps: 30,
+  totalFrames: 900,
+  frames: [{ bands: [0.82, 0.45, 0.31, ...] }, ...]
+};
+```
+
+- `frames[i].bands[]` — frequency band amplitudes, 0-1. Index 0 = bass, higher = treble.
+- Each band normalized independently across the full track.
+
+## Mapping Audio to Visuals
+
+| Audio signal           | Visual property                   | Effect                     |
+| ---------------------- | --------------------------------- | -------------------------- |
+| Bass (bands[0])        | `scale`                           | Pulse on beat              |
+| Treble (bands[12-14])  | `textShadow`, `boxShadow`         | Glow intensity             |
+| Overall amplitude      | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift |
+| Mid-range (bands[4-8]) | `borderRadius`, `width`           | Shape morphing             |
+
+Any GSAP-tweenable property works — `clipPath`, `filter`, SVG attributes, CSS custom properties.
+
+## Content, Not Medium
+
+Audio provides **timing and intensity**. The visual vocabulary comes from the narrative.
+
+**Never add:** equalizer bars, spectrum analyzers, waveform displays, musical notes clip art, generic particle systems, rainbow color cycling, strobing white on beats, abstract pulsing orbs.
+
+**Instead:** Let content guide the visual and audio drive its behavior. Bass makes warmth _swell_. Treble sharpens _contrast_. The visual choice comes from "what does this piece feel like?"
+
+## Sampling Pattern
+
+Audio reactivity requires per-frame sampling via a `for` loop with `tl.call()`, not a single tween:
+
+```js
+// ✅ Correct — sample every frame
+for (var f = 0; f < AUDIO_DATA.totalFrames; f++) {
+  tl.call(
+    (function (frame) {
+      return function () {
+        draw(frame);
+      };
+    })(AUDIO_DATA.frames[f]),
+    [],
+    f / AUDIO_DATA.fps,
+  );
+}
+
+// ❌ Wrong — single tween, doesn't react to audio
+gsap.to(".el", { scale: 1.2, duration: totalDuration });
+```
+
+Without per-frame sampling, the composition doesn't actually react to audio.
+
+## textShadow Gotcha
+
+`textShadow` on a parent container with semi-transparent children (e.g., inactive caption words at `rgba(255,255,255,0.3)`) renders a visible glow rectangle behind all children. Fix: apply `scale` to the container for beat pulse, but apply `textShadow` to individual active words only.
+
+## Guidelines
+
+- **Subtlety for text** — 3-6% scale variation, soft glow. Heavy pulsing makes text unreadable.
+- **Go bigger on non-text** — backgrounds and shapes can handle 10-30% swings.
+- **Match the energy** — corporate = subtle; music video = dramatic.
+- **Deterministic** — pre-extracted data, no Web Audio API, no runtime analysis.
+
+## Constraints
+
+- All audio data must be pre-extracted (use `extract-audio-data.py` from the gsap skill's scripts/)
+- No `Math.random()` or `Date.now()`
+- Audio reactivity runs on the same GSAP timeline as everything else

package/dist/skills/hyperframes/references/captions.md

@@ -0,0 +1,132 @@
+# Captions
+
+## Language Rule (Non-Negotiable)
+
+**Never use `.en` models unless the user explicitly states the audio is English.** `.en` models TRANSLATE non-English audio into English instead of transcribing it.
+
+1. User says the language → `--model small --language <code>` (no `.en`)
+2. User says English → `--model small.en`
+3. Language unknown → `--model small` (no `.en`, no `--language`) — auto-detects
+
+---
+
+Analyze spoken content to determine caption style. If user specifies a style, use that. Otherwise, detect tone from the transcript.
+
+## Transcript Source
+
+```json
+[
+  { "text": "Hello", "start": 0.0, "end": 0.5 },
+  { "text": "world.", "start": 0.6, "end": 1.2 }
+]
+```
+
+For transcription commands, whisper models, external APIs, see [transcript-guide.md](transcript-guide.md).
+
+## Style Detection (When No Style Specified)
+
+Read the full transcript before choosing. Four dimensions:
+
+**1. Visual feel** — corporate→clean; energetic→bold; storytelling→elegant; technical→precise; social→playful.
+
+**2. Color palette** — dark+bright for energy; muted for professional; high contrast for clarity; one accent color.
+
+**3. Font mood** — heavy/condensed for impact; clean sans for modern; rounded for friendly; serif for elegance.
+
+**4. Animation character** — scale-pop for punchy; gentle fade for calm; word-by-word for emphasis; typewriter for technical.
+
+## Per-Word Styling
+
+Scan for words deserving distinct treatment:
+
+- **Brand/product names** — larger size, unique color
+- **ALL CAPS** — scale boost, flash, accent color
+- **Numbers/statistics** — bold weight, accent color
+- **Emotional keywords** — exaggerated animation (overshoot, bounce)
+- **Call-to-action** — highlight, underline, color pop
+- **Marker highlight** — for beyond-color emphasis, see [marker-highlight.md](marker-highlight.md)
+
+## Script-to-Style Mapping
+
+| Tone         | Font mood                | Animation                          | Color                       | Size    |
+| ------------ | ------------------------ | ---------------------------------- | --------------------------- | ------- |
+| Hype/launch  | Heavy condensed, 800-900 | Scale-pop, back.out(1.7), 0.1-0.2s | Bright on dark              | 72-96px |
+| Corporate    | Clean sans, 600-700      | Fade+slide, power3.out, 0.3s       | White/neutral, muted accent | 56-72px |
+| Tutorial     | Mono/clean sans, 500-600 | Typewriter/fade, 0.4-0.5s          | High contrast, minimal      | 48-64px |
+| Storytelling | Serif/elegant, 400-500   | Slow fade, power2.out, 0.5-0.6s    | Warm muted tones            | 44-56px |
+| Social       | Rounded sans, 700-800    | Bounce, elastic.out, word-by-word  | Playful, colored pills      | 56-80px |
+
+## Word Grouping
+
+- **High energy:** 2-3 words. Quick turnover.
+- **Conversational:** 3-5 words. Natural phrases.
+- **Measured/calm:** 4-6 words. Longer groups.
+
+Break on sentence boundaries, 150ms+ pauses, or max word count.
+
+## Positioning
+
+- **Landscape (1920x1080):** Bottom 80-120px, centered
+- **Portrait (1080x1920):** Lower middle ~600-700px from bottom, centered
+- Never cover the subject's face
+- `position: absolute` — never relative
+- One caption group visible at a time
+
+## Text Overflow Prevention
+
+Use `window.__hyperframes.fitTextFontSize()`:
+
+```js
+var result = window.__hyperframes.fitTextFontSize(group.text.toUpperCase(), {
+  fontFamily: "Outfit",
+  fontWeight: 900,
+  maxWidth: 1600,
+});
+el.style.fontSize = result.fontSize + "px";
+```
+
+Options: `maxWidth` (1600 landscape, 900 portrait), `baseFontSize` (78), `minFontSize` (42), `fontWeight`, `fontFamily`, `step` (2).
+
+CSS safety nets: `max-width` on container, `overflow: visible` (**not** `hidden` — hidden clips scaled emphasis words and glow effects), `position: absolute`, explicit `height`. When per-word styling uses `scale > 1.0`, compute `maxWidth = safeWidth / maxScale` to leave headroom.
+
+**Container pattern:** Full-width absolute container, centered. Do **not** use `left: 50%; transform: translateX(-50%)` — causes clipping at composition edges.
+
+## Caption Exit Guarantee
+
+Every group **must** have a hard kill after exit animation:
+
+```js
+tl.to(groupEl, { opacity: 0, scale: 0.95, duration: 0.12, ease: "power2.in" }, group.end - 0.12);
+tl.set(groupEl, { opacity: 0, visibility: "hidden" }, group.end); // deterministic kill
+```
+
+Self-lint after building timeline — place **before** `window.__timelines[id] = tl` so it runs at composition init:
+
+```js
+GROUPS.forEach(function (group, gi) {
+  var el = document.getElementById("cg-" + gi);
+  if (!el) return;
+  tl.seek(group.end + 0.01);
+  var computed = window.getComputedStyle(el);
+  if (computed.opacity !== "0" && computed.visibility !== "hidden") {
+    console.warn(
+      "[caption-lint] group " + gi + " still visible at t=" + (group.end + 0.01).toFixed(2) + "s",
+    );
+  }
+});
+tl.seek(0);
+```
+
+## Further References
+
+- [dynamic-techniques.md](dynamic-techniques.md) — karaoke, clip-path reveals, slam words, scatter exits, elastic, 3D rotation
+- [transcript-guide.md](transcript-guide.md) — transcription commands, whisper models, external APIs
+- [marker-highlight.md](marker-highlight.md) — animated text emphasis paired with per-word styling
+
+## Constraints
+
+- Deterministic. No `Math.random()`, no `Date.now()`.
+- Sync to transcript timestamps.
+- One group visible at a time.
+- Every group must have a hard `tl.set` kill at `group.end`.
+- Check project root for font files before defaulting to Google Fonts.