zilmate 1.9.9 → 1.10.2

package/.agents/skills/remotion-to-hyperframes/references/timing.md

@@ -0,0 +1,165 @@
+# Timing translation: interpolate, spring, easing
+
+The single highest-leverage reference. Easings and timings are what readers
+notice; getting them wrong costs more SSIM than any other translation choice.
+Empirically validated against tiers T1–T3.
+
+## Conversion: frames → seconds
+
+HF's timeline is in seconds. Remotion is frame-based. Always:
+
+```
+time_seconds = frame / fps
+```
+
+So at fps=30:
+
+- frame 15 → 0.5 s
+- frame 30 → 1.0 s
+- frame 90 → 3.0 s
+
+Do this conversion once when translating, not at runtime.
+
+## interpolate — linear
+
+```tsx
+const opacity = interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" });
+```
+
+Translates to:
+
+```js
+gsap.to(target, { opacity: 1, duration: 1.0, ease: "none" }, 0);
+// fromTo if the property starts at 0 and CSS doesn't already set it
+gsap.fromTo(target, { opacity: 0 }, { opacity: 1, duration: 1.0, ease: "none" }, 0);
+```
+
+`ease: "none"` matches Remotion's default linear interpolation. CSS sets the
+`from` value if your initial state is in CSS; otherwise use `fromTo`.
+
+`extrapolateLeft`/`extrapolateRight` defaults to `"extend"` in Remotion but
+`"clamp"` is what the agent will see most often. GSAP doesn't extend — values
+hold at the start and end of the tween. So for `clamp`, GSAP matches; for
+`extend`, you'd need to extend the input range manually before emitting.
+
+## interpolate — multi-segment
+
+```tsx
+const opacity = interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0]);
+```
+
+Three keyframed tweens at offsets `[0]/fps`, `[1]/fps`, `[2]/fps`:
+
+```js
+const tl = gsap.timeline({ paused: true });
+tl.to(target, { opacity: 1, duration: 0.5, ease: "none" }, 0);
+tl.to(target, { opacity: 1, duration: 2.0, ease: "none" }, 0.5);
+tl.to(target, { opacity: 0, duration: 0.5, ease: "none" }, 2.5);
+```
+
+Validated in T1 — mean SSIM 0.974 against Remotion baseline.
+
+## spring → GSAP back.out
+
+Remotion's `spring()` is the most lossy translation. The mapping is approximate
+but close enough that real-world compositions hold ≥ 0.92 SSIM (T2: 0.985, T3: 0.953).
+
+| Remotion `spring` config                          | GSAP equivalent                                      | Validated in                     |
+| ------------------------------------------------- | ---------------------------------------------------- | -------------------------------- |
+| `{damping: 12, stiffness: 100, mass: 1}` (snappy) | `back.out(1.4)` over ~0.7 s                          | T2, T3 (TitleScene)              |
+| `{damping: 14, stiffness: 90, mass: 1}` (calmer)  | `back.out(1.2)` over ~0.7 s                          | T3 (StatCard)                    |
+| `{damping: 8, stiffness: 200}` (very bouncy)      | `back.out(2.0)` or `elastic.out(1, 0.5)` over ~0.6 s | not validated; budget ~0.05 SSIM |
+| `{overshootClamping: true}`                       | `power3.out` over ~0.6 s (no overshoot)              | not validated                    |
+
+**Rule of thumb**: `back.out(N)` overshoot ratio ≈ `(stiffness / damping^2) * 1.4`. For
+`damping:12, stiffness:100` that gives `1.4 * 100/144 = 0.97`, which is close to
+the validated 1.4 (the formula is rough; tune by visual). Default duration is
+~0.7 s for the typical config.
+
+When the spring's `delay`/`from`/`to` are non-default, scale the duration
+proportionally.
+
+## interpolate with custom easing
+
+```tsx
+import { Easing } from "remotion";
+interpolate(frame, [0, 30], [0, 1], { easing: Easing.out(Easing.cubic) });
+```
+
+| Remotion                     | GSAP                                                                                   |
+| ---------------------------- | -------------------------------------------------------------------------------------- |
+| `Easing.in(Easing.linear)`   | `ease: "none"`                                                                         |
+| `Easing.out(Easing.cubic)`   | `ease: "power3.out"`                                                                   |
+| `Easing.inOut(Easing.cubic)` | `ease: "power3.inOut"`                                                                 |
+| `Easing.out(Easing.poly(N))` | `ease: "power<N>.out"` (N=2 quad, 3 cubic, 4 quart, 5 quint)                           |
+| `Easing.bezier(a,b,c,d)`     | `CustomEase.create("c", "M0,0 C${a},${b} ${c},${d} 1,1")` (requires CustomEase plugin) |
+| `Easing.elastic(bounciness)` | `ease: "elastic.out(${bounciness}, 0.3)"`                                              |
+| `Easing.bounce`              | `ease: "bounce.out"`                                                                   |
+| `Easing.back(overshoot)`     | `ease: "back.out(${overshoot * 1.7})"` (Remotion's overshoot scale differs)            |
+
+## interpolate driving non-numeric properties
+
+```tsx
+const color = interpolateColors(frame, [0, 30], ["#ff0000", "#0000ff"]);
+```
+
+GSAP does color tweens natively:
+
+```js
+gsap.to(target, { color: "#0000ff", duration: 1.0, ease: "none" }, 0);
+```
+
+Same for `backgroundColor`, `borderColor`. The `from` value is read from CSS
+or the inline style.
+
+## Custom count-up / number tweens
+
+When Remotion uses a frame-driven number ramp (`Math.round(value * eased)`):
+
+```tsx
+const t = interpolate(frame, [0, 45], [0, 1]);
+const eased = 1 - (1 - t) ** 3; // cubic ease-out
+const value = Math.round(target * eased);
+return <div>{value.toLocaleString()}</div>;
+```
+
+GSAP equivalent — tween a counter object, write `textContent` on update:
+
+```js
+const counter = { v: 0 };
+tl.to(
+  counter,
+  {
+    v: target,
+    duration: 1.5,
+    ease: "power3.out",
+    onUpdate: () => {
+      el.textContent = Math.round(counter.v).toLocaleString();
+    },
+  },
+  0,
+);
+```
+
+`power3.out` matches `1 - (1-t)^3` exactly. Validated in T3 (mean SSIM 0.953).
+Per-frame digit mismatches occur on sub-frame timing offsets but final values
+converge — no SSIM impact above the noise floor.
+
+## Stagger via per-instance prop
+
+When custom subcomponents take a `delayInFrames` prop:
+
+```tsx
+<StatCard delayInFrames={i * 12} value={...} />
+```
+
+Translate to GSAP timeline offsets:
+
+```js
+cards.forEach((card, i) => {
+  const start = base + i * (12 / fps); // i * 0.4s at fps=30
+  tl.to(card, { ... }, start);
+});
+```
+
+Validated in T3 — three StatCards staggered at 0.0/0.4/0.8 s.

package/.agents/skills/remotion-to-hyperframes/references/transitions.md

@@ -0,0 +1,114 @@
+# Transitions translation: @remotion/transitions → HF crossfades / shader-transitions
+
+The `@remotion/transitions` package is Remotion's library of pre-built
+scene-to-scene transitions. HF has two paths to translate them:
+
+1. **Manual GSAP crossfade** — for simple opacity/transform transitions. Free, no extra package.
+2. **HF shader-transitions package** — for visually-rich transitions that match the @remotion/transitions presets.
+
+## Pattern: `<TransitionSeries>` is `<Series>` with overlap
+
+```tsx
+<TransitionSeries>
+  <TransitionSeries.Sequence durationInFrames={60}>
+    <SceneA />
+  </TransitionSeries.Sequence>
+  <TransitionSeries.Transition
+    presentation={fade()}
+    timing={linearTiming({ durationInFrames: 15 })}
+  />
+  <TransitionSeries.Sequence durationInFrames={60}>
+    <SceneB />
+  </TransitionSeries.Sequence>
+</TransitionSeries>
+```
+
+Translates to scenes that overlap by the transition duration:
+
+- SceneA: [0, 60] = `data-start="0" data-duration="2"`
+- SceneB: [60-15, 60-15+60] = `data-start="1.5" data-duration="2"` (the transition window overlaps the end of A and start of B)
+
+Then drive the transition with GSAP:
+
+```js
+// Manual fade (presentation={fade()})
+tl.to(sceneA, { opacity: 0, duration: 0.5, ease: "none" }, 1.5);
+tl.fromTo(sceneB, { opacity: 0 }, { opacity: 1, duration: 0.5, ease: "none" }, 1.5);
+```
+
+## Presentation table
+
+| Remotion `presentation`            | HF translation                                                                            |
+| ---------------------------------- | ----------------------------------------------------------------------------------------- |
+| `fade()`                           | manual `gsap.to(opacity)` crossfade                                                       |
+| `slide({direction: "from-right"})` | `gsap.fromTo(translateX: "100%" → 0)` on incoming + `to(translateX: "-100%")` on outgoing |
+| `wipe({direction: "from-left"})`   | `gsap.fromTo(clip-path: inset(0 100% 0 0) → inset(0 0 0 0))` on incoming                  |
+| `clockWipe()`                      | use HF's `sdf-iris` shader-transition (`npx hyperframes add sdf-iris`)                    |
+| `flip()`                           | `gsap.to(rotateY)` 180° split between scenes                                              |
+| `cube()`                           | use HF's `cinematic-zoom` or build manually with `rotateY` + `transform-origin`           |
+| `iris()`                           | use HF's `sdf-iris` shader-transition                                                     |
+| `none()`                           | no transition; hard cut at the boundary                                                   |
+
+## Timing translations
+
+```tsx
+linearTiming({durationInFrames: 15})              → ease: "none"
+linearTiming({durationInFrames: 15, easing: ...}) → ease per the easing table in timing.md
+springTiming({config: {damping: 12}})             → ease: "back.out(1.4)" (~0.7 s)
+```
+
+Convert `durationInFrames` to seconds (`/fps`).
+
+## When to use HF shader-transitions
+
+For transitions Remotion presets that have visually-rich GLSL equivalents
+(iris, ripple, zoom, glitch), use HF's [shader-transitions](https://hyperframes.heygen.com/catalog/blocks)
+package. They produce richer output than manual GSAP transforms.
+
+```bash
+npx hyperframes add sdf-iris
+```
+
+Then in the composition:
+
+```html
+<div id="iris-transition" class="hf-shader-transition" data-start="1.5" data-duration="0.5">
+  <!-- bound scenes via the shader-transition's data-from / data-to -->
+</div>
+```
+
+Each shader-transition has its own data attributes; see the catalog page
+for the specific block.
+
+## When the source uses a custom Presentation
+
+Remotion supports custom `presentation` implementations:
+
+```tsx
+const customPresentation: PresentationComponent = ({
+  children,
+  presentationProgress,
+  presentationDirection,
+}) => {
+  return (
+    <div
+      style={
+        {
+          /* compute transform from progress */
+        }
+      }
+    >
+      {children}
+    </div>
+  );
+};
+```
+
+Translation: extract the math from the `style={...}` block and emit
+equivalent GSAP tweens. Specifically the transform formula maps directly
+to a `gsap.to(target, { transform: ... })` parameterized by `progress`.
+
+If the custom presentation uses `useCurrentFrame()` internally to
+animate something _outside_ the simple progress curve, treat the source
+as untranslatable and bow out to the runtime interop pattern (see
+[escape-hatch.md](escape-hatch.md)).

package/.agents/skills/remotion-to-hyperframes/scripts/frame_strip.sh

@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+# frame_strip.sh — produce a side-by-side comparison strip from two videos.
+#
+# Used to debug failing render_diff.sh runs visually: pick a sample timestamp
+# range, extract frames from both videos, lay them out as a grid for review.
+#
+# Usage:
+#   frame_strip.sh <baseline.mp4> <translated.mp4> [output-dir] [samples]
+#
+# Defaults: output-dir=./strip-out, samples=8 (evenly spaced across duration).
+# Output:
+#   strip.png         — single PNG with `samples` rows, each row is
+#                       (baseline frame | translated frame) at one timestamp
+#   timestamps.txt    — the timestamps sampled
+
+set -euo pipefail
+
+if [[ $# -lt 2 || $# -gt 4 ]]; then
+  echo "usage: $0 <baseline.mp4> <translated.mp4> [output-dir] [samples]" >&2
+  exit 2
+fi
+
+BASELINE="$1"
+TRANSLATED="$2"
+OUTDIR="${3:-./strip-out}"
+SAMPLES="${4:-8}"
+
+if ! command -v ffmpeg >/dev/null 2>&1 || ! command -v ffprobe >/dev/null 2>&1; then
+  echo "error: ffmpeg/ffprobe not on PATH" >&2
+  exit 2
+fi
+
+mkdir -p "$OUTDIR"
+
+# Build the strip in a single ffmpeg invocation. Two inputs (baseline and
+# translated) are sampled at N evenly-spaced timestamps via the `select`
+# filter, then assembled with hstack (per-row pairs) + vstack (rows).
+# Earlier versions spawned 3 ffmpeg calls per timestamp + a final vstack;
+# this is one call regardless of N.
+python3 - "$BASELINE" "$TRANSLATED" "$OUTDIR" "$SAMPLES" <<'PY'
+import json
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+baseline, translated, outdir, samples = sys.argv[1], sys.argv[2], Path(sys.argv[3]), int(sys.argv[4])
+
+# Read fps + duration from the baseline so we can map timestamps to frame
+# indexes for the `select` filter (frame-accurate, doesn't depend on
+# keyframe alignment).
+probe = subprocess.run(
+    ["ffprobe", "-v", "error", "-select_streams", "v:0",
+     "-show_entries", "stream=r_frame_rate,nb_read_frames,duration",
+     "-show_entries", "format=duration",
+     "-of", "json", "-count_frames", baseline],
+    check=True, capture_output=True, text=True,
+)
+data = json.loads(probe.stdout)
+stream = data["streams"][0]
+num, den = stream["r_frame_rate"].split("/")
+fps = float(num) / float(den)
+nb_frames = int(stream.get("nb_read_frames") or 0)
+if nb_frames <= 0:
+    duration = float(stream.get("duration") or data["format"]["duration"])
+    nb_frames = int(duration * fps)
+
+# Even-spaced sample frames in the 5%-95% window (skip fade-in/out noise).
+start = max(0, int(nb_frames * 0.05))
+end = max(start, int(nb_frames * 0.95) - 1)
+if samples == 1:
+    frames = [start]
+else:
+    step = (end - start) / (samples - 1)
+    frames = [int(start + i * step) for i in range(samples)]
+
+(outdir / "timestamps.txt").write_text(
+    "\n".join(f"{f / fps:.3f}" for f in frames) + "\n"
+)
+
+# select='eq(n,F1)+eq(n,F2)+...' picks exactly the listed frames from each
+# input. We then hstack per-frame pairs and vstack the result.
+select_expr = "+".join(f"eq(n,{f})" for f in frames)
+n = len(frames)
+filter_parts = [
+    f"[0:v]select='{select_expr}',setpts=N/FRAME_RATE/TB,split={n}"
+    + "".join(f"[b{i}]" for i in range(n)),
+    f"[1:v]select='{select_expr}',setpts=N/FRAME_RATE/TB,split={n}"
+    + "".join(f"[t{i}]" for i in range(n)),
+]
+for i in range(n):
+    filter_parts.append(f"[b{i}][t{i}]hstack=inputs=2[row{i}]")
+filter_parts.append(
+    "".join(f"[row{i}]" for i in range(n)) + f"vstack=inputs={n}[out]"
+)
+filter_graph = ";".join(filter_parts)
+
+cmd = [
+    "ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
+    "-i", baseline, "-i", translated,
+    "-filter_complex", filter_graph,
+    "-map", "[out]", "-frames:v", "1",
+    str(outdir / "strip.png"),
+]
+subprocess.run(cmd, check=True)
+print(f"wrote {outdir / 'strip.png'} ({n} samples)")
+PY