ctxora 6.2.2 → 6.4.0

package/src/harness_context/skills/ecc/remotion-video-creation/SKILL.md

@@ -0,0 +1,43 @@
+---
+name: remotion-video-creation
+description: Best practices for Remotion - Video creation in React. 29 domain-specific rules covering 3D, animations, audio, captions, charts, transitions, and more. Use when building video in React with Remotion — animations, audio, captions, charts, or transitions.
+metadata:
+  tags: remotion, video, react, animation, composition, three.js, lottie
+---
+
+## When to use
+
+Use this skills whenever you are dealing with Remotion code to obtain the domain-specific knowledge.
+
+## How to use
+
+Read individual rule files for detailed explanations and code examples:
+
+- [rules/3d.md](rules/3d.md) - 3D content in Remotion using Three.js and React Three Fiber
+- [rules/animations.md](rules/animations.md) - Fundamental animation skills for Remotion
+- [rules/assets.md](rules/assets.md) - Importing images, videos, audio, and fonts into Remotion
+- [rules/audio.md](rules/audio.md) - Using audio and sound in Remotion - importing, trimming, volume, speed, pitch
+- [rules/calculate-metadata.md](rules/calculate-metadata.md) - Dynamically set composition duration, dimensions, and props
+- [rules/can-decode.md](rules/can-decode.md) - Check if a video can be decoded by the browser using Mediabunny
+- [rules/charts.md](rules/charts.md) - Chart and data visualization patterns for Remotion
+- [rules/compositions.md](rules/compositions.md) - Defining compositions, stills, folders, default props and dynamic metadata
+- [rules/display-captions.md](rules/display-captions.md) - Displaying captions in Remotion with TikTok-style pages and word highlighting
+- [rules/extract-frames.md](rules/extract-frames.md) - Extract frames from videos at specific timestamps using Mediabunny
+- [rules/fonts.md](rules/fonts.md) - Loading Google Fonts and local fonts in Remotion
+- [rules/get-audio-duration.md](rules/get-audio-duration.md) - Getting the duration of an audio file in seconds with Mediabunny
+- [rules/get-video-dimensions.md](rules/get-video-dimensions.md) - Getting the width and height of a video file with Mediabunny
+- [rules/get-video-duration.md](rules/get-video-duration.md) - Getting the duration of a video file in seconds with Mediabunny
+- [rules/gifs.md](rules/gifs.md) - Displaying GIFs synchronized with Remotion's timeline
+- [rules/images.md](rules/images.md) - Embedding images in Remotion using the Img component
+- [rules/import-srt-captions.md](rules/import-srt-captions.md) - Importing .srt subtitle files into Remotion using @remotion/captions
+- [rules/lottie.md](rules/lottie.md) - Embedding Lottie animations in Remotion
+- [rules/measuring-dom-nodes.md](rules/measuring-dom-nodes.md) - Measuring DOM element dimensions in Remotion
+- [rules/measuring-text.md](rules/measuring-text.md) - Measuring text dimensions, fitting text to containers, and checking overflow
+- [rules/sequencing.md](rules/sequencing.md) - Sequencing patterns for Remotion - delay, trim, limit duration of items
+- [rules/tailwind.md](rules/tailwind.md) - Using TailwindCSS in Remotion
+- [rules/text-animations.md](rules/text-animations.md) - Typography and text animation patterns for Remotion
+- [rules/timing.md](rules/timing.md) - Interpolation curves in Remotion - linear, easing, spring animations
+- [rules/transcribe-captions.md](rules/transcribe-captions.md) - Transcribing audio to generate captions in Remotion
+- [rules/transitions.md](rules/transitions.md) - Scene transition patterns for Remotion
+- [rules/trimming.md](rules/trimming.md) - Trimming patterns for Remotion - cut the beginning or end of animations
+- [rules/videos.md](rules/videos.md) - Embedding videos in Remotion - trimming, volume, speed, looping, pitch

package/src/harness_context/skills/ecc/remotion-video-creation/rules/3d.md

@@ -0,0 +1,86 @@
+---
+name: 3d
+description: 3D content in Remotion using Three.js and React Three Fiber.
+metadata:
+  tags: 3d, three, threejs
+---
+
+# Using Three.js and React Three Fiber in Remotion
+
+Follow React Three Fiber and Three.js best practices.
+Only the following Remotion-specific rules need to be followed:
+
+## Prerequisites
+
+First, the `@remotion/three` package needs to be installed.
+If it is not, use the following command:
+
+```bash
+npx remotion add @remotion/three # If project uses npm
+bunx remotion add @remotion/three # If project uses bun
+yarn remotion add @remotion/three # If project uses yarn
+pnpm exec remotion add @remotion/three # If project uses pnpm
+```
+
+## Using ThreeCanvas
+
+You MUST wrap 3D content in `<ThreeCanvas>` and include proper lighting.
+`<ThreeCanvas>` MUST have a `width` and `height` prop.
+
+```tsx
+import { ThreeCanvas } from "@remotion/three";
+import { useVideoConfig } from "remotion";
+
+const { width, height } = useVideoConfig();
+
+<ThreeCanvas width={width} height={height}>
+  <ambientLight intensity={0.4} />
+  <directionalLight position={[5, 5, 5]} intensity={0.8} />
+  <mesh>
+    <sphereGeometry args={[1, 32, 32]} />
+    <meshStandardMaterial color="red" />
+  </mesh>
+</ThreeCanvas>
+```
+
+## No animations not driven by `useCurrentFrame()`
+
+Shaders, models etc MUST NOT animate by themselves.
+No animations are allowed unless they are driven by `useCurrentFrame()`.
+Otherwise, it will cause flickering during rendering.
+
+Using `useFrame()` from `@react-three/fiber` is forbidden.
+
+## Animate using `useCurrentFrame()`
+
+Use `useCurrentFrame()` to perform animations.
+
+```tsx
+const frame = useCurrentFrame();
+const rotationY = frame * 0.02;
+
+<mesh rotation={[0, rotationY, 0]}>
+  <boxGeometry args={[2, 2, 2]} />
+  <meshStandardMaterial color="#4a9eff" />
+</mesh>
+```
+
+## Using `<Sequence>` inside `<ThreeCanvas>`
+
+The `layout` prop of any `<Sequence>` inside a `<ThreeCanvas>` must be set to `none`.
+
+```tsx
+import { Sequence } from "remotion";
+import { ThreeCanvas } from "@remotion/three";
+
+const { width, height } = useVideoConfig();
+
+<ThreeCanvas width={width} height={height}>
+  <Sequence layout="none">
+    <mesh>
+      <boxGeometry args={[2, 2, 2]} />
+      <meshStandardMaterial color="#4a9eff" />
+    </mesh>
+  </Sequence>
+</ThreeCanvas>
+```

package/src/harness_context/skills/ecc/remotion-video-creation/rules/animations.md

@@ -0,0 +1,29 @@
+---
+name: animations
+description: Fundamental animation skills for Remotion
+metadata:
+  tags: animations, transitions, frames, useCurrentFrame
+---
+
+All animations MUST be driven by the `useCurrentFrame()` hook.
+Write animations in seconds and multiply them by the `fps` value from `useVideoConfig()`.
+
+```tsx
+import { useCurrentFrame } from "remotion";
+
+export const FadeIn = () => {
+  const frame = useCurrentFrame();
+  const { fps } = useVideoConfig();
+
+  const opacity = interpolate(frame, [0, 2 * fps], [0, 1], {
+    extrapolateRight: 'clamp',
+  });
+
+  return (
+    <div style={{ opacity }}>Hello World!</div>
+  );
+};
+```
+
+CSS transitions or animations are FORBIDDEN - they will not render correctly.
+Tailwind animation class names are FORBIDDEN - they will not render correctly.

package/src/harness_context/skills/ecc/remotion-video-creation/rules/assets/charts-bar-chart.tsx

@@ -0,0 +1,173 @@
+import {loadFont} from '@remotion/google-fonts/Inter';
+import {AbsoluteFill, spring, useCurrentFrame, useVideoConfig} from 'remotion';
+
+const {fontFamily} = loadFont();
+
+const COLOR_BAR = '#D4AF37';
+const COLOR_TEXT = '#ffffff';
+const COLOR_MUTED = '#888888';
+const COLOR_BG = '#0a0a0a';
+const COLOR_AXIS = '#333333';
+
+// Ideal composition size: 1280x720
+
+const Title: React.FC<{children: React.ReactNode}> = ({children}) => (
+	<div style={{textAlign: 'center', marginBottom: 40}}>
+		<div style={{color: COLOR_TEXT, fontSize: 48, fontWeight: 600}}>
+			{children}
+		</div>
+	</div>
+);
+
+const YAxis: React.FC<{steps: number[]; height: number}> = ({
+	steps,
+	height,
+}) => (
+	<div
+		style={{
+			display: 'flex',
+			flexDirection: 'column',
+			justifyContent: 'space-between',
+			height,
+			paddingRight: 16,
+		}}
+	>
+		{steps
+			.slice()
+			.reverse()
+			.map((step) => (
+				<div
+					key={step}
+					style={{
+						color: COLOR_MUTED,
+						fontSize: 20,
+						textAlign: 'right',
+					}}
+				>
+					{step.toLocaleString()}
+				</div>
+			))}
+	</div>
+);
+
+const Bar: React.FC<{
+	height: number;
+	progress: number;
+}> = ({height, progress}) => (
+	<div
+		style={{
+			flex: 1,
+			display: 'flex',
+			flexDirection: 'column',
+			justifyContent: 'flex-end',
+		}}
+	>
+		<div
+			style={{
+				width: '100%',
+				height,
+				backgroundColor: COLOR_BAR,
+				borderRadius: '8px 8px 0 0',
+				opacity: progress,
+			}}
+		/>
+	</div>
+);
+
+const XAxis: React.FC<{
+	children: React.ReactNode;
+	labels: string[];
+	height: number;
+}> = ({children, labels, height}) => (
+	<div style={{flex: 1, display: 'flex', flexDirection: 'column'}}>
+		<div
+			style={{
+				display: 'flex',
+				alignItems: 'flex-end',
+				gap: 16,
+				height,
+				borderLeft: `2px solid ${COLOR_AXIS}`,
+				borderBottom: `2px solid ${COLOR_AXIS}`,
+				paddingLeft: 16,
+			}}
+		>
+			{children}
+		</div>
+		<div
+			style={{
+				display: 'flex',
+				gap: 16,
+				paddingLeft: 16,
+				marginTop: 12,
+			}}
+		>
+			{labels.map((label) => (
+				<div
+					key={label}
+					style={{
+						flex: 1,
+						textAlign: 'center',
+						color: COLOR_MUTED,
+						fontSize: 20,
+					}}
+				>
+					{label}
+				</div>
+			))}
+		</div>
+	</div>
+);
+
+export const MyAnimation = () => {
+	const frame = useCurrentFrame();
+	const {fps, height} = useVideoConfig();
+
+	const data = [
+		{month: 'Jan', price: 2039},
+		{month: 'Mar', price: 2160},
+		{month: 'May', price: 2327},
+		{month: 'Jul', price: 2426},
+		{month: 'Sep', price: 2634},
+		{month: 'Nov', price: 2672},
+	];
+
+	const minPrice = 2000;
+	const maxPrice = 2800;
+	const priceRange = maxPrice - minPrice;
+	const chartHeight = height - 280;
+	const yAxisSteps = [2000, 2400, 2800];
+
+	return (
+		<AbsoluteFill
+			style={{
+				backgroundColor: COLOR_BG,
+				padding: 60,
+				display: 'flex',
+				flexDirection: 'column',
+				fontFamily,
+			}}
+		>
+			<Title>Gold Price 2024</Title>
+
+			<div style={{display: 'flex', flex: 1}}>
+				<YAxis steps={yAxisSteps} height={chartHeight} />
+				<XAxis height={chartHeight} labels={data.map((d) => d.month)}>
+					{data.map((item, i) => {
+						const progress = spring({
+							frame: frame - i * 5 - 10,
+							fps,
+							config: {damping: 18, stiffness: 80},
+						});
+
+						const barHeight =
+							((item.price - minPrice) / priceRange) * chartHeight * progress;
+
+						return (
+							<Bar key={item.month} height={barHeight} progress={progress} />
+						);
+					})}
+				</XAxis>
+			</div>
+		</AbsoluteFill>
+	);
+};

package/src/harness_context/skills/ecc/remotion-video-creation/rules/assets/text-animations-typewriter.tsx

@@ -0,0 +1,100 @@
+import {
+	AbsoluteFill,
+	interpolate,
+	useCurrentFrame,
+	useVideoConfig,
+} from 'remotion';
+
+const COLOR_BG = '#ffffff';
+const COLOR_TEXT = '#000000';
+const FULL_TEXT = 'From prompt to motion graphics. This is Remotion.';
+const PAUSE_AFTER = 'From prompt to motion graphics.';
+const FONT_SIZE = 72;
+const FONT_WEIGHT = 700;
+const CHAR_FRAMES = 2;
+const CURSOR_BLINK_FRAMES = 16;
+const PAUSE_SECONDS = 1;
+
+// Ideal composition size: 1280x720
+
+const getTypedText = ({
+	frame,
+	fullText,
+	pauseAfter,
+	charFrames,
+	pauseFrames,
+}: {
+	frame: number;
+	fullText: string;
+	pauseAfter: string;
+	charFrames: number;
+	pauseFrames: number;
+}): string => {
+	const pauseIndex = fullText.indexOf(pauseAfter);
+	const preLen =
+		pauseIndex >= 0 ? pauseIndex + pauseAfter.length : fullText.length;
+
+	let typedChars = 0;
+	if (frame < preLen * charFrames) {
+		typedChars = Math.floor(frame / charFrames);
+	} else if (frame < preLen * charFrames + pauseFrames) {
+		typedChars = preLen;
+	} else {
+		const postPhase = frame - preLen * charFrames - pauseFrames;
+		typedChars = Math.min(
+			fullText.length,
+			preLen + Math.floor(postPhase / charFrames),
+		);
+	}
+	return fullText.slice(0, typedChars);
+};
+
+const Cursor: React.FC<{
+	frame: number;
+	blinkFrames: number;
+	symbol?: string;
+}> = ({frame, blinkFrames, symbol = '\u258C'}) => {
+	const opacity = interpolate(
+		frame % blinkFrames,
+		[0, blinkFrames / 2, blinkFrames],
+		[1, 0, 1],
+		{extrapolateLeft: 'clamp', extrapolateRight: 'clamp'},
+	);
+
+	return <span style={{opacity}}>{symbol}</span>;
+};
+
+export const MyAnimation = () => {
+	const frame = useCurrentFrame();
+	const {fps} = useVideoConfig();
+
+	const pauseFrames = Math.round(fps * PAUSE_SECONDS);
+
+	const typedText = getTypedText({
+		frame,
+		fullText: FULL_TEXT,
+		pauseAfter: PAUSE_AFTER,
+		charFrames: CHAR_FRAMES,
+		pauseFrames,
+	});
+
+	return (
+		<AbsoluteFill
+			style={{
+				backgroundColor: COLOR_BG,
+			}}
+		>
+			<div
+				style={{
+					color: COLOR_TEXT,
+					fontSize: FONT_SIZE,
+					fontWeight: FONT_WEIGHT,
+					fontFamily: 'sans-serif',
+				}}
+			>
+				<span>{typedText}</span>
+				<Cursor frame={frame} blinkFrames={CURSOR_BLINK_FRAMES} />
+			</div>
+		</AbsoluteFill>
+	);
+};

package/src/harness_context/skills/ecc/remotion-video-creation/rules/assets/text-animations-word-highlight.tsx

@@ -0,0 +1,108 @@
+import {loadFont} from '@remotion/google-fonts/Inter';
+import React from 'react';
+import {
+	AbsoluteFill,
+	spring,
+	useCurrentFrame,
+	useVideoConfig,
+} from 'remotion';
+
+/*
+ * Highlight a word in a sentence with a spring-animated wipe effect.
+ */
+
+// Ideal composition size: 1280x720
+
+const COLOR_BG = '#ffffff';
+const COLOR_TEXT = '#000000';
+const COLOR_HIGHLIGHT = '#A7C7E7';
+const FULL_TEXT = 'This is Remotion.';
+const HIGHLIGHT_WORD = 'Remotion';
+const FONT_SIZE = 72;
+const FONT_WEIGHT = 700;
+const HIGHLIGHT_START_FRAME = 30;
+const HIGHLIGHT_WIPE_DURATION = 18;
+
+const {fontFamily} = loadFont();
+
+const Highlight: React.FC<{
+	word: string;
+	color: string;
+	delay: number;
+	durationInFrames: number;
+}> = ({word, color, delay, durationInFrames}) => {
+	const frame = useCurrentFrame();
+	const {fps} = useVideoConfig();
+
+	const highlightProgress = spring({
+		fps,
+		frame,
+		config: {damping: 200},
+		delay,
+		durationInFrames,
+	});
+	const scaleX = Math.max(0, Math.min(1, highlightProgress));
+
+	return (
+		<span style={{position: 'relative', display: 'inline-block'}}>
+			<span
+				style={{
+					position: 'absolute',
+					left: 0,
+					right: 0,
+					top: '50%',
+					height: '1.05em',
+					transform: `translateY(-50%) scaleX(${scaleX})`,
+					transformOrigin: 'left center',
+					backgroundColor: color,
+					borderRadius: '0.18em',
+					zIndex: 0,
+				}}
+			/>
+			<span style={{position: 'relative', zIndex: 1}}>{word}</span>
+		</span>
+	);
+};
+
+export const MyAnimation = () => {
+	const highlightIndex = FULL_TEXT.indexOf(HIGHLIGHT_WORD);
+	const hasHighlight = highlightIndex >= 0;
+	const preText = hasHighlight ? FULL_TEXT.slice(0, highlightIndex) : FULL_TEXT;
+	const postText = hasHighlight
+		? FULL_TEXT.slice(highlightIndex + HIGHLIGHT_WORD.length)
+		: '';
+
+	return (
+		<AbsoluteFill
+			style={{
+				backgroundColor: COLOR_BG,
+				alignItems: 'center',
+				justifyContent: 'center',
+				fontFamily,
+			}}
+		>
+			<div
+				style={{
+					color: COLOR_TEXT,
+					fontSize: FONT_SIZE,
+					fontWeight: FONT_WEIGHT,
+				}}
+			>
+				{hasHighlight ? (
+					<>
+						<span>{preText}</span>
+						<Highlight
+							word={HIGHLIGHT_WORD}
+							color={COLOR_HIGHLIGHT}
+							delay={HIGHLIGHT_START_FRAME}
+							durationInFrames={HIGHLIGHT_WIPE_DURATION}
+						/>
+						<span>{postText}</span>
+					</>
+				) : (
+					<span>{FULL_TEXT}</span>
+				)}
+			</div>
+		</AbsoluteFill>
+	);
+};

package/src/harness_context/skills/ecc/remotion-video-creation/rules/assets.md

@@ -0,0 +1,78 @@
+---
+name: assets
+description: Importing images, videos, audio, and fonts into Remotion
+metadata:
+  tags: assets, staticFile, images, fonts, public
+---
+
+# Importing assets in Remotion
+
+## The public folder
+
+Place assets in the `public/` folder at your project root.
+
+## Using staticFile()
+
+You MUST use `staticFile()` to reference files from the `public/` folder:
+
+```tsx
+import {Img, staticFile} from 'remotion';
+
+export const MyComposition = () => {
+  return <Img src={staticFile('logo.png')} />;
+};
+```
+
+The function returns an encoded URL that works correctly when deploying to subdirectories.
+
+## Using with components
+
+**Images:**
+
+```tsx
+import {Img, staticFile} from 'remotion';
+
+<Img src={staticFile('photo.png')} />;
+```
+
+**Videos:**
+
+```tsx
+import {Video} from '@remotion/media';
+import {staticFile} from 'remotion';
+
+<Video src={staticFile('clip.mp4')} />;
+```
+
+**Audio:**
+
+```tsx
+import {Audio} from '@remotion/media';
+import {staticFile} from 'remotion';
+
+<Audio src={staticFile('music.mp3')} />;
+```
+
+**Fonts:**
+
+```tsx
+import {staticFile} from 'remotion';
+
+const fontFamily = new FontFace('MyFont', `url(${staticFile('font.woff2')})`);
+await fontFamily.load();
+document.fonts.add(fontFamily);
+```
+
+## Remote URLs
+
+Remote URLs can be used directly without `staticFile()`:
+
+```tsx
+<Img src="https://example.com/image.png" />
+<Video src="https://remotion.media/video.mp4" />
+```
+
+## Important notes
+
+- Remotion components (`<Img>`, `<Video>`, `<Audio>`) ensure assets are fully loaded before rendering
+- Special characters in filenames (`#`, `?`, `&`) are automatically encoded