@ericrisco/rsc 3.1.2 → 3.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -4
- package/manifest.json +28 -3
- package/package.json +1 -1
- package/scripts/lib/domains.js +1 -1
- package/skills/ai-media/SKILL.md +2 -2
- package/skills/cost-tracking/SKILL.md +2 -1
- package/skills/design/references/motion-and-interaction.md +4 -0
- package/skills/llm-pipeline/SKILL.md +1 -1
- package/skills/scroll-film/SKILL.md +220 -0
- package/skills/scroll-film/assets/page-template.html +128 -0
- package/skills/scroll-film/assets/scrub-engine.js +352 -0
- package/skills/scroll-film/evals/README.md +26 -0
- package/skills/scroll-film/evals/cases.yaml +81 -0
- package/skills/scroll-film/references/chain.md +218 -0
- package/skills/scroll-film/references/engine.md +132 -0
- package/skills/scroll-film/scripts/verify.sh +158 -0
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
# Building the chain — prompts, probe, batch loops
|
|
2
|
+
|
|
3
|
+
Everything the body of `scroll-film` points at for sections 1 to 4. Fill the slots; keep the
|
|
4
|
+
clauses marked **verbatim** exactly as written, because they are what the seams depend on.
|
|
5
|
+
|
|
6
|
+
## The brief to write down before anything renders
|
|
7
|
+
|
|
8
|
+
```yaml
|
|
9
|
+
subject: "a bubble-tea brand, from the tea field to the last sip" # the user's words
|
|
10
|
+
brand: { name: "Pearl & Leaf", tone: "warm, playful" }
|
|
11
|
+
palette: { bg: "#F3EDE4", accent: "#6D55B8", others: ["#8FB98A", "#C88A5A", "#3A2E48"] }
|
|
12
|
+
look: clay-diorama # becomes the style preamble below
|
|
13
|
+
camera: dive-and-hop # walkthrough | fixed-angle | dive-and-hop → shape A, A, B
|
|
14
|
+
phones: true # true = a second, 9:16 chain
|
|
15
|
+
scenes: # 4–7, in camera order; the last carries the CTA
|
|
16
|
+
- { id: field, subject: "terraced tea field, pickers with baskets, a small drying shed", focus: "the drying racks" }
|
|
17
|
+
- { id: kitchen, subject: "a kitchen with copper pots of simmering tapioca pearls", focus: "the pearl pots" }
|
|
18
|
+
- { id: shop, subject: "the corner shop with a striped awning and a queue", focus: "the counter" }
|
|
19
|
+
model: { endpoint: "<from the probe>", seconds_leg: 8, seconds_hop: 5, price_per_second: "<read today>" }
|
|
20
|
+
estimate_usd: "<clips × seconds × price + 20%>"
|
|
21
|
+
approved: false # nothing renders until this is true
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Style preamble — one block, pasted unchanged into every still prompt
|
|
25
|
+
|
|
26
|
+
Write it once from the look the user picked. The background colour is the page background.
|
|
27
|
+
|
|
28
|
+
```text
|
|
29
|
+
Isometric miniature diorama on a plain solid #F3EDE4 background, a soft shadow beneath it.
|
|
30
|
+
Matte clay materials, rounded toy-like shapes, warm low studio light, tilt-shift depth of field.
|
|
31
|
+
Palette limited to #6D55B8, #8FB98A, #C88A5A, #3A2E48. Subject centred. No text, letters,
|
|
32
|
+
numbers or logos anywhere.
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Other looks change the first two sentences only and keep the palette and no-text lines:
|
|
36
|
+
|
|
37
|
+
- **Paper cut:** "Layered paper-craft diorama, matte card, clean die-cut edges, thin shadows between layers."
|
|
38
|
+
- **Vinyl toy:** "Glossy vinyl-toy diorama, smooth plastic shading, soft rim light."
|
|
39
|
+
- **Night miniature:** "Miniature town at night, warm lit windows, neon signs, wet reflective streets."
|
|
40
|
+
- **Photoreal:** "Architectural photograph of one continuous space, wide lens, late-afternoon light, natural
|
|
41
|
+
materials, no people." Photoreal scenes are full-bleed (no floating island), the camera moves
|
|
42
|
+
through doorways and glass instead of opening a roof, and it suits shape A. Do not pass one
|
|
43
|
+
approved still as a style reference here: it clones the same room into every scene.
|
|
44
|
+
|
|
45
|
+
## Still prompt
|
|
46
|
+
|
|
47
|
+
```text
|
|
48
|
+
<STYLE PREAMBLE>
|
|
49
|
+
Subject: <scene.subject>. Framed for a 16:9 picture with the focal point, <scene.focus>, in the centre.
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
- Render every still from **one** source and model. Two image models drift in warmth and line,
|
|
53
|
+
and the chain inherits the drift.
|
|
54
|
+
- Prepare each still at the clip's aspect before it becomes a start image: 16:9 for desktop, and a
|
|
55
|
+
1080×1920 canvas in the background colour for the phone chain. Handed an off-aspect image, many
|
|
56
|
+
endpoints choose their own framing and frame 0 stops matching.
|
|
57
|
+
- Review the set before any video: same angle, same light, same palette. Re-roll the odd one out.
|
|
58
|
+
|
|
59
|
+
## Leg prompt — shape A
|
|
60
|
+
|
|
61
|
+
Start frame: the previous leg's real last frame (leg 0: the first still). **No end frame.**
|
|
62
|
+
|
|
63
|
+
```text
|
|
64
|
+
One continuous camera move, no cuts. Continue the same slow, steady forward drift.
|
|
65
|
+
<MID-LEG MOVE, optional>. The camera travels into <scene.subject> toward <scene.focus>.
|
|
66
|
+
In the final second, settle into a slow, steady forward drift toward <where the next scene lies>.
|
|
67
|
+
<last two sentences of the style preamble>. Smooth, unhurried motion. No text.
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
The first and the last sentence of that block are **verbatim**: they are the handoff contract
|
|
71
|
+
that keeps velocity continuous across the seam.
|
|
72
|
+
|
|
73
|
+
### Mid-leg moves by concept
|
|
74
|
+
|
|
75
|
+
A reversal inside one leg is safe, since one render has no seam. Pick from the scene's own logic:
|
|
76
|
+
|
|
77
|
+
| Concept | Mid-leg move |
|
|
78
|
+
| --- | --- |
|
|
79
|
+
| Product, luxury | "circling the <hero object> in a slow half orbit, keeping it centred, then moving past it" |
|
|
80
|
+
| Real estate, hotels | "gliding through the doorway at eye height, rising gently as the room opens up" |
|
|
81
|
+
| Production, logistics | "tracking low and level alongside the <line>, foreground objects sliding past" |
|
|
82
|
+
| Travel, campus, outdoors | "climbing over the <terrain>, then swooping down toward the <next focus>" |
|
|
83
|
+
| Food, craft | "pushing in until the <craft moment> nearly fills the frame, then easing back" |
|
|
84
|
+
| The scene itself is the show | omit the slot: a plain forward drift is the safest leg |
|
|
85
|
+
|
|
86
|
+
Expressive moves raise the re-roll rate: the model can end mid-orbit instead of in a clean forward
|
|
87
|
+
drift. Budget one extra render for each expressive leg, and check its last frame before chaining.
|
|
88
|
+
|
|
89
|
+
### Fixed-angle clause
|
|
90
|
+
|
|
91
|
+
When the user picked "one fixed angle", put this in the mid-leg slot of **every** leg, **verbatim**:
|
|
92
|
+
|
|
93
|
+
```text
|
|
94
|
+
The camera holds the same high isometric angle the whole time, with no rotation, orbit or tilt.
|
|
95
|
+
It only travels straight and level while the world slides past beneath it.
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Long legs drift the angle slightly. When checking each last frame, also check the view has not
|
|
99
|
+
turned; re-roll a leg that has.
|
|
100
|
+
|
|
101
|
+
## Dive and hop prompts — shape B
|
|
102
|
+
|
|
103
|
+
Dive. Start frame: the scene's still (the solid-background version, so the frame is full).
|
|
104
|
+
|
|
105
|
+
```text
|
|
106
|
+
One continuous camera move, no cuts. Start high and far, looking down at the whole <scene.subject>
|
|
107
|
+
like a small model. The camera glides forward and descends toward <scene.focus>; as it closes in,
|
|
108
|
+
the roof lifts gently away to reveal the interior. <style tail>. Smooth, unhurried motion. No text.
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
For an open scene with no roof (a field, a square), swap the last clause for "the camera skims low
|
|
112
|
+
across <the scene> toward <scene.focus>".
|
|
113
|
+
|
|
114
|
+
Hop. Start frame: dive k's real last frame. End frame: dive k+1's real first frame.
|
|
115
|
+
|
|
116
|
+
```text
|
|
117
|
+
One continuous camera move, no cuts. The camera rises up and out of <scene k>, climbs into the sky,
|
|
118
|
+
crosses the connected miniature world and arrives above <scene k+1>, starting to descend toward it.
|
|
119
|
+
<style tail>. Smooth, unhurried motion. No text.
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
For the hop into a hero-product finale, end it on "…and the world falls away toward a single giant
|
|
123
|
+
<product> floating in open <bg colour> space."
|
|
124
|
+
|
|
125
|
+
## Phone chain clause
|
|
126
|
+
|
|
127
|
+
Prepend to every leg, dive and hop of the 9:16 chain:
|
|
128
|
+
|
|
129
|
+
```text
|
|
130
|
+
Vertical composition: the scene centred with open <bg colour> space above and below it.
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
The phone chain hands off frames from its own renders only.
|
|
134
|
+
|
|
135
|
+
## The probe — one cheap clip before the batch
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
ssim() { ffmpeg -i "$1" -i "$2" -lavfi "[0]scale=960:540,setsar=1[a];[1]scale=960:540,setsar=1[b];[a][b]ssim" \
|
|
139
|
+
-f null - 2>&1 | grep -oE 'All:[0-9.]+'; }
|
|
140
|
+
ffmpeg -v error -y -i probe.mp4 -frames:v 2 -vsync 0 probe_%d.png
|
|
141
|
+
ssim probe_1.png still_01.png # frame 0 against the still it was given
|
|
142
|
+
ssim probe_1.png probe_2.png # the clip's own frame-to-frame step, the yardstick
|
|
143
|
+
# Hop duty: the last frame against the end frame it was given.
|
|
144
|
+
ffmpeg -v error -y -sseof -1 -i probe_hop.mp4 -update 1 probe_last.png
|
|
145
|
+
ssim probe_last.png still_02.png
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
- Frame 0 against the still should score about as high as the clip's own frame step. Judge it
|
|
149
|
+
against that yardstick, never a fixed number: how alike two frames look depends on the picture.
|
|
150
|
+
On a soft landscape photo, a fresh render of the same scene still scored 0.89. Clearly below the
|
|
151
|
+
step means the input steered a new render instead of starting the clip: the endpoint cannot
|
|
152
|
+
hold a seam.
|
|
153
|
+
- The hop's last frame only has to land on the same composition. Look at it; a near miss is normal.
|
|
154
|
+
- Compare the bill for the probe with the advertised price before multiplying it by the chain.
|
|
155
|
+
- Re-run the probe whenever the endpoint's input schema changes. Catalogues gain and lose
|
|
156
|
+
start-frame support without notice.
|
|
157
|
+
|
|
158
|
+
## Batch loops
|
|
159
|
+
|
|
160
|
+
`make_clip` is the one function you write against the chosen endpoint (the call itself →
|
|
161
|
+
`../../fal/SKILL.md` or `../../replicate/SKILL.md`). Its contract: `make_clip out.mp4 prompt.txt
|
|
162
|
+
start.png [end.png]` renders one clip, downloads it at once (result URLs expire within hours) and
|
|
163
|
+
appends the billed cost to `costs.log`. Run the loops as a background job and poll; a clip takes
|
|
164
|
+
minutes. Keep them in a `bash` script: zsh arrays start at 1, and an inline loop in a zsh shell
|
|
165
|
+
pairs each hop with the wrong scene.
|
|
166
|
+
|
|
167
|
+
Shape A, one leg at a time:
|
|
168
|
+
|
|
169
|
+
```bash
|
|
170
|
+
#!/bin/bash
|
|
171
|
+
set -e
|
|
172
|
+
SCENES="field kitchen shop"
|
|
173
|
+
start="stills/field.png"
|
|
174
|
+
for s in $SCENES; do
|
|
175
|
+
make_clip "legs/$s.mp4" "prompts/leg_$s.txt" "$start"
|
|
176
|
+
ffmpeg -v error -y -sseof -1 -i "legs/$s.mp4" -update 1 -q:v 2 "frames/${s}_last.png"
|
|
177
|
+
start="frames/${s}_last.png" # look at it before the next leg spends money
|
|
178
|
+
done
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
Shape B, dives in parallel, then hops:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
#!/bin/bash
|
|
185
|
+
set -e
|
|
186
|
+
SCENES="field kitchen shop"
|
|
187
|
+
for s in $SCENES; do make_clip "dives/$s.mp4" "prompts/dive_$s.txt" "stills/$s.png" & done
|
|
188
|
+
wait
|
|
189
|
+
for s in $SCENES; do
|
|
190
|
+
ffmpeg -v error -y -i "dives/$s.mp4" -frames:v 1 -q:v 2 "frames/${s}_first.png"
|
|
191
|
+
ffmpeg -v error -y -sseof -1 -i "dives/$s.mp4" -update 1 -q:v 2 "frames/${s}_last.png"
|
|
192
|
+
done
|
|
193
|
+
prev=""; k=0
|
|
194
|
+
for s in $SCENES; do
|
|
195
|
+
if [ -n "$prev" ]; then
|
|
196
|
+
k=$((k + 1))
|
|
197
|
+
make_clip "hops/hop_$k.mp4" "prompts/hop_$k.txt" "frames/${prev}_last.png" "frames/${s}_first.png" &
|
|
198
|
+
fi
|
|
199
|
+
prev="$s"
|
|
200
|
+
done
|
|
201
|
+
wait
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
Five or six renders at once is usually fine. Many more invites transient rate and credit errors:
|
|
205
|
+
re-roll the one that failed, never the batch.
|
|
206
|
+
|
|
207
|
+
## When a content filter blocks a harmless clip
|
|
208
|
+
|
|
209
|
+
Video filters flag ordinary interiors, mostly bedrooms, pools, spas and bars, from the prompt words
|
|
210
|
+
and from the frames themselves. In order:
|
|
211
|
+
|
|
212
|
+
1. Re-roll. The verdict is often not deterministic.
|
|
213
|
+
2. Take out the trigger words and add "empty, unoccupied, architectural".
|
|
214
|
+
3. Render that one clip on another model that also passed the probe, with the same start and end
|
|
215
|
+
frames. Its render character differs slightly, which a short hop behind the seam dissolve
|
|
216
|
+
usually hides.
|
|
217
|
+
4. Drop that hop. List the two dives next to each other in the engine config and it dissolves
|
|
218
|
+
between them directly. The film completes; that seam reads as a dissolve, not a flight.
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# The scrub engine — options, wiring, pacing
|
|
2
|
+
|
|
3
|
+
`assets/scrub-engine.js` is plain JavaScript with no dependencies. Loaded as a script it defines
|
|
4
|
+
`window.ScrollFilm`; under a bundler it also exports the same object. `assets/page-template.html` is
|
|
5
|
+
a complete page that uses it.
|
|
6
|
+
|
|
7
|
+
## Mounting
|
|
8
|
+
|
|
9
|
+
```js
|
|
10
|
+
const film = ScrollFilm.mount(container, options);
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`container` is the element the film lives in, usually a `<section>`. The engine adds a sticky stage
|
|
14
|
+
(the clips and an overlay slot for your copy) and a spacer that gives the section its scroll length.
|
|
15
|
+
Content after the section scrolls normally once the film ends.
|
|
16
|
+
|
|
17
|
+
### `options`
|
|
18
|
+
|
|
19
|
+
| Option | Default | What it does |
|
|
20
|
+
| --- | --- | --- |
|
|
21
|
+
| `shots` | required | The clips in chain order (fields below). |
|
|
22
|
+
| `fade` | `0.1` | Width of the seam dissolve, in viewport heights. |
|
|
23
|
+
| `preload` | `1.5` | A clip is fetched when the scroll comes this many viewports from it. |
|
|
24
|
+
| `ease` | `0.2` | How fast the playhead catches up with the scroll each frame (1 = instantly). |
|
|
25
|
+
| `phoneWidth` | `860` | At or below this width, or on any touch-only device, the phone sources are used. |
|
|
26
|
+
| `stillsOnly` | `false` | Posters only, no video. Also forced by reduced motion and by Save-Data. |
|
|
27
|
+
| `listen` | `true` | `false` = the engine adds no scroll listener; call `film.update()` from your own scroll source. |
|
|
28
|
+
| `onChange` | none | Called with the state object whenever the position changes. |
|
|
29
|
+
|
|
30
|
+
### Fields of each shot
|
|
31
|
+
|
|
32
|
+
| Field | What it is |
|
|
33
|
+
| --- | --- |
|
|
34
|
+
| `src` | The desktop clip (`-g 8` encode). |
|
|
35
|
+
| `srcMobile` | The phone clip (`-m.mp4`, 9:16, `-g 4`). Optional; falls back to `src`. |
|
|
36
|
+
| `poster` / `posterMobile` | The clip's first frame. Shown until the video paints, and as the fallback. |
|
|
37
|
+
| `length` | Scroll distance in viewport heights. More length, longer dwell. Default 1. |
|
|
38
|
+
| `hold` | 0 to 1. Slows the camera mid-shot and speeds it near the seams; the seam frames never move. Above 0.6 the stop starts to read as a stall. |
|
|
39
|
+
| `section` | Which copy block this clip belongs to. A dive and the hop that leaves it share a section. Defaults to the shot's index. |
|
|
40
|
+
|
|
41
|
+
### What `mount` returns
|
|
42
|
+
|
|
43
|
+
| Member | Use |
|
|
44
|
+
| --- | --- |
|
|
45
|
+
| `overlay` | Append your copy here; it rides the sticky stage above the clips. |
|
|
46
|
+
| `stage` | The sticky element, for styling. |
|
|
47
|
+
| `stillsOnly` | Whether this visitor gets posters only. |
|
|
48
|
+
| `update()` | Re-read the scroll position. Needed only with `listen: false`. |
|
|
49
|
+
| `relayout()` | Recompute positions after content above the film changes height (a late banner, a font swap). |
|
|
50
|
+
| `jumpTo(section)` | Smooth-scroll to the middle of a section; instant under reduced motion. |
|
|
51
|
+
| `destroy()` | Remove listeners, cancel fetches, release the Blob URLs, remove the added nodes. |
|
|
52
|
+
|
|
53
|
+
### The state object passed to `onChange`
|
|
54
|
+
|
|
55
|
+
```js
|
|
56
|
+
{ shot: 2, section: 1, sectionProgress: 0.42, progress: 0.37 }
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
`sectionProgress` runs 0 to 1 across every clip of that section. Fade a section's copy in over its
|
|
60
|
+
first 15% and out over its last 15%; let the first block start visible and the last one hold its
|
|
61
|
+
call to action. The template does exactly this.
|
|
62
|
+
|
|
63
|
+
## Pacing
|
|
64
|
+
|
|
65
|
+
- The hero and the finale earn more `length` (1.4 to 1.6) and a `hold` of 0.3 to 0.5, so the camera
|
|
66
|
+
settles while the copy is readable.
|
|
67
|
+
- Hops are transit: `length` 0.8 to 1.0, no `hold`.
|
|
68
|
+
- Expressive camera work belongs in the clip, restraint in the scroll mapping. Both compound.
|
|
69
|
+
|
|
70
|
+
## Theming and layout
|
|
71
|
+
|
|
72
|
+
- `--film-bg` is the colour behind the stage; set it to the scene background so posters and
|
|
73
|
+
dissolves sit on it. `--film-focus` (default `50% 45%`) is where `object-fit: cover` keeps the
|
|
74
|
+
subject when the viewport crops a clip.
|
|
75
|
+
- The engine's own CSS sits in `@layer scroll-film`, so any unlayered page rule wins without
|
|
76
|
+
specificity tricks.
|
|
77
|
+
- The stage is `position: sticky`. Any ancestor with `overflow: hidden` breaks it; use
|
|
78
|
+
`overflow-x: clip` on `body` if the page needs to stop horizontal overflow.
|
|
79
|
+
- The stage is `100lvh` tall: the film fills the screen when the URL bar is hidden, and the bottom
|
|
80
|
+
edge sits under the bar while it shows. Keep copy clear of the bottom with `svh` units and
|
|
81
|
+
`env(safe-area-inset-bottom)`, as the template does, and add `viewport-fit=cover` to the
|
|
82
|
+
viewport meta tag.
|
|
83
|
+
|
|
84
|
+
## Wiring it into a framework
|
|
85
|
+
|
|
86
|
+
React or Next.js, in a client component (the effect never runs on the server):
|
|
87
|
+
|
|
88
|
+
```tsx
|
|
89
|
+
'use client';
|
|
90
|
+
import { useEffect, useRef } from 'react';
|
|
91
|
+
|
|
92
|
+
export function Film({ shots }: { shots: object[] }) {
|
|
93
|
+
const ref = useRef<HTMLElement>(null);
|
|
94
|
+
useEffect(() => {
|
|
95
|
+
let film: { destroy(): void } | undefined;
|
|
96
|
+
let cancelled = false;
|
|
97
|
+
import('./scrub-engine.js').then((mod) => {
|
|
98
|
+
const ScrollFilm = (mod as any).default ?? (window as any).ScrollFilm;
|
|
99
|
+
if (!cancelled && ref.current) film = ScrollFilm.mount(ref.current, { shots });
|
|
100
|
+
});
|
|
101
|
+
return () => { cancelled = true; film?.destroy(); };
|
|
102
|
+
}, [shots]);
|
|
103
|
+
return <section ref={ref} aria-label="The journey" />;
|
|
104
|
+
}
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
The `cancelled` flag matters: a component that unmounts before the import resolves would otherwise
|
|
108
|
+
mount a film nobody can destroy. Keep `shots` stable (a module constant or `useMemo`) or every
|
|
109
|
+
render rebuilds the film.
|
|
110
|
+
|
|
111
|
+
Vue: mount in `onMounted`, keep the returned object, call `destroy()` in `onBeforeUnmount`. A
|
|
112
|
+
server-rendered page in any other stack: serve the assets and add two `<script>` tags, as the
|
|
113
|
+
template does.
|
|
114
|
+
|
|
115
|
+
To drive it from a scroll source the page already owns:
|
|
116
|
+
|
|
117
|
+
```js
|
|
118
|
+
const film = ScrollFilm.mount(el, { shots, listen: false });
|
|
119
|
+
ScrollTrigger.create({ trigger: el, start: 'top top', end: 'bottom bottom', onUpdate: () => film.update() });
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## Accessibility and weight
|
|
123
|
+
|
|
124
|
+
- The clip layer is `aria-hidden`. Everything a visitor needs is in the copy, which is ordinary
|
|
125
|
+
HTML in reading order, available to screen readers and search engines with or without the film.
|
|
126
|
+
- The template adds a skip link past the film and, when keyboard focus enters a faded copy block,
|
|
127
|
+
scrolls to that block's section.
|
|
128
|
+
- Preload the first poster with `fetchpriority="high"`; it is the LCP element. Clips load by
|
|
129
|
+
proximity, so first paint costs one image and one or two clips, not the whole chain.
|
|
130
|
+
- Measure the chain before shipping (`du -ch assets/vid/*.mp4`). A 1080p clip of several seconds
|
|
131
|
+
at GOP 8 runs to megabytes, so a whole chain runs to tens of them. That is why the phone chain is
|
|
132
|
+
720 wide and why a clip is fetched only when the scroll gets near it.
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# verify.sh — check a scroll-film clip chain before it ships. Read-only.
|
|
3
|
+
#
|
|
4
|
+
# bash verify.sh [--strict] [--gop N] [--min-ssim X] [--portrait] [--frames DIR] clip1.mp4 clip2.mp4 ...
|
|
5
|
+
#
|
|
6
|
+
# Pass the clips in chain order (the order the engine plays them). Per clip it checks the
|
|
7
|
+
# encode a scrubbed video needs: H.264 yuv420p, no audio track, the index (moov) ahead of the
|
|
8
|
+
# media so the first frame paints early, and a keyframe at least every N frames (default 8; use
|
|
9
|
+
# --gop 4 for phone encodes). Across the chain it checks every clip shares one size and frame
|
|
10
|
+
# rate, and scores each seam: the true last frame of clip i against the first frame of clip i+1.
|
|
11
|
+
#
|
|
12
|
+
# A seam is judged against the clips around it, not against a fixed number: how alike two
|
|
13
|
+
# frames look depends on the picture, so a soft landscape scores high even across a cut. The
|
|
14
|
+
# seam's SSIM must stay within --margin (default 0.03) of the ordinary frame-to-frame SSIM at
|
|
15
|
+
# the end of clip i and the start of clip i+1. A frame-locked seam looks like one more frame; a
|
|
16
|
+
# fresh render of the same scene drops below. --frames DIR writes each pair side by side.
|
|
17
|
+
#
|
|
18
|
+
# Warns by default and exits 0. --strict exits 1 on any warning. Exits 2 on bad usage.
|
|
19
|
+
# bash 3.2-safe: no associative arrays, no mapfile.
|
|
20
|
+
|
|
21
|
+
set -u
|
|
22
|
+
|
|
23
|
+
STRICT=0
|
|
24
|
+
GOP=8
|
|
25
|
+
MARGIN=0.03
|
|
26
|
+
PORTRAIT=0
|
|
27
|
+
FRAMES_DIR=""
|
|
28
|
+
CLIPS=()
|
|
29
|
+
|
|
30
|
+
while [ $# -gt 0 ]; do
|
|
31
|
+
case "$1" in
|
|
32
|
+
--strict) STRICT=1 ;;
|
|
33
|
+
--gop) shift; GOP="${1:-}" ;;
|
|
34
|
+
--margin) shift; MARGIN="${1:-}" ;;
|
|
35
|
+
--portrait) PORTRAIT=1 ;;
|
|
36
|
+
--frames) shift; FRAMES_DIR="${1:-}" ;;
|
|
37
|
+
-h|--help) sed -n '2,19p' "$0"; exit 0 ;;
|
|
38
|
+
-*) echo "verify.sh: unknown option $1" >&2; exit 2 ;;
|
|
39
|
+
*) CLIPS+=("$1") ;;
|
|
40
|
+
esac
|
|
41
|
+
shift
|
|
42
|
+
done
|
|
43
|
+
|
|
44
|
+
for tool in ffmpeg ffprobe awk; do
|
|
45
|
+
command -v "$tool" >/dev/null 2>&1 || { echo "verify.sh: $tool not found on PATH" >&2; exit 2; }
|
|
46
|
+
done
|
|
47
|
+
if [ "${#CLIPS[@]}" -eq 0 ]; then
|
|
48
|
+
echo "verify.sh: pass the clips in chain order (see --help)" >&2
|
|
49
|
+
exit 2
|
|
50
|
+
fi
|
|
51
|
+
|
|
52
|
+
WARNINGS=0
|
|
53
|
+
ok() { echo "ok $*"; }
|
|
54
|
+
warn() { echo "WARN $*"; WARNINGS=$((WARNINGS + 1)); }
|
|
55
|
+
|
|
56
|
+
TMP=$(mktemp -d "${TMPDIR:-/tmp}/scroll-film-verify.XXXXXX")
|
|
57
|
+
trap 'rm -rf "$TMP"' EXIT
|
|
58
|
+
[ -n "$FRAMES_DIR" ] && mkdir -p "$FRAMES_DIR"
|
|
59
|
+
|
|
60
|
+
# Every decoded frame of the last second, one file each: the final file is the real last frame.
|
|
61
|
+
# (A plain -sseof seek with -frames:v 1 stops a few frames short and judges the wrong frame.)
|
|
62
|
+
# The two last and the two first frames give the clips' own frame-to-frame step at the seam.
|
|
63
|
+
edge_frames() { # clip index
|
|
64
|
+
mkdir -p "$TMP/tail_$2" "$TMP/head_$2"
|
|
65
|
+
ffmpeg -v error -y -sseof -1 -i "$1" -vsync 0 "$TMP/tail_$2/%04d.png"
|
|
66
|
+
ffmpeg -v error -y -i "$1" -frames:v 2 -vsync 0 "$TMP/head_$2/%04d.png"
|
|
67
|
+
ls "$TMP/tail_$2" | tail -2 > "$TMP/tail_$2.list"
|
|
68
|
+
}
|
|
69
|
+
ssim() { ffmpeg -v info -i "$1" -i "$2" -lavfi "[0][1]ssim" -f null - 2>&1 | grep -oE 'All:[0-9.]+' | tail -1 | cut -d: -f2; }
|
|
70
|
+
|
|
71
|
+
FIRST_SHAPE=""
|
|
72
|
+
i=0
|
|
73
|
+
for clip in "${CLIPS[@]}"; do
|
|
74
|
+
i=$((i + 1))
|
|
75
|
+
name=$(basename "$clip")
|
|
76
|
+
if [ ! -f "$clip" ]; then warn "$name: file not found"; continue; fi
|
|
77
|
+
|
|
78
|
+
video=$(ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,pix_fmt,width,height,r_frame_rate -of csv=p=0 "$clip")
|
|
79
|
+
if [ -z "$video" ]; then warn "$name: no video stream"; continue; fi
|
|
80
|
+
codec=$(echo "$video" | cut -d, -f1)
|
|
81
|
+
width=$(echo "$video" | cut -d, -f2)
|
|
82
|
+
height=$(echo "$video" | cut -d, -f3)
|
|
83
|
+
pixfmt=$(echo "$video" | cut -d, -f4)
|
|
84
|
+
rate=$(echo "$video" | cut -d, -f5)
|
|
85
|
+
|
|
86
|
+
[ "$codec" = "h264" ] && ok "$name: h264" || warn "$name: codec $codec (h264 decodes and seeks everywhere)"
|
|
87
|
+
[ "$pixfmt" = "yuv420p" ] && ok "$name: yuv420p" || warn "$name: pixel format $pixfmt (Safari wants yuv420p)"
|
|
88
|
+
|
|
89
|
+
audio=$(ffprobe -v error -select_streams a -show_entries stream=index -of csv=p=0 "$clip")
|
|
90
|
+
[ -z "$audio" ] && ok "$name: no audio track" || warn "$name: has an audio track (dead weight on a muted scrub; encode with -an)"
|
|
91
|
+
|
|
92
|
+
first_atom=$(ffprobe -v trace "$clip" 2>&1 | grep -oE "type:'(moov|mdat)'" | head -1)
|
|
93
|
+
[ "$first_atom" = "type:'moov'" ] && ok "$name: moov before mdat (faststart)" \
|
|
94
|
+
|| warn "$name: index after the media (encode with -movflags +faststart)"
|
|
95
|
+
|
|
96
|
+
max_gap=$(ffprobe -v error -select_streams v:0 -show_entries packet=flags -of csv=p=0 "$clip" \
|
|
97
|
+
| awk '{ n++; if (substr($0,1,1) == "K") { if (last && n - last > gap) gap = n - last; last = n } }
|
|
98
|
+
END { if (n - last + 1 > gap) gap = n - last + 1; print gap + 0 }')
|
|
99
|
+
if [ "$max_gap" -le "$GOP" ]; then ok "$name: keyframe every <= $max_gap frames"
|
|
100
|
+
else warn "$name: $max_gap frames between keyframes (> $GOP); every scrub seek decodes that many"; fi
|
|
101
|
+
|
|
102
|
+
if [ "$PORTRAIT" -eq 1 ]; then
|
|
103
|
+
[ "$height" -gt "$width" ] && ok "$name: portrait ${width}x${height}" \
|
|
104
|
+
|| warn "$name: ${width}x${height} is not portrait (the phone chain is rendered at 9:16, not cropped)"
|
|
105
|
+
fi
|
|
106
|
+
|
|
107
|
+
shape="${width}x${height}@${rate}"
|
|
108
|
+
if [ -z "$FIRST_SHAPE" ]; then FIRST_SHAPE="$shape"
|
|
109
|
+
elif [ "$shape" != "$FIRST_SHAPE" ]; then
|
|
110
|
+
warn "$name: $shape differs from the first clip's $FIRST_SHAPE (one model and one encode for the whole chain)"
|
|
111
|
+
fi
|
|
112
|
+
|
|
113
|
+
edge_frames "$clip" "$i"
|
|
114
|
+
done
|
|
115
|
+
|
|
116
|
+
n=${#CLIPS[@]}
|
|
117
|
+
s=1
|
|
118
|
+
while [ "$s" -lt "$n" ]; do
|
|
119
|
+
next=$((s + 1))
|
|
120
|
+
label="seam $s→$next ($(basename "${CLIPS[$((s - 1))]}") → $(basename "${CLIPS[$s]}"))"
|
|
121
|
+
a_prev=""; a=""; b="$TMP/head_$next/0001.png"; b_next="$TMP/head_$next/0002.png"
|
|
122
|
+
if [ -s "$TMP/tail_$s.list" ]; then
|
|
123
|
+
a="$TMP/tail_$s/$(tail -1 "$TMP/tail_$s.list")"
|
|
124
|
+
[ "$(wc -l < "$TMP/tail_$s.list")" -ge 2 ] && a_prev="$TMP/tail_$s/$(head -1 "$TMP/tail_$s.list")"
|
|
125
|
+
fi
|
|
126
|
+
if [ -z "$a" ] || [ ! -f "$a" ] || [ ! -f "$b" ]; then
|
|
127
|
+
warn "$label: could not extract both frames"
|
|
128
|
+
else
|
|
129
|
+
score=$(ssim "$a" "$b")
|
|
130
|
+
step_out=""; step_in=""
|
|
131
|
+
[ -n "$a_prev" ] && step_out=$(ssim "$a_prev" "$a")
|
|
132
|
+
[ -f "$b_next" ] && step_in=$(ssim "$b" "$b_next")
|
|
133
|
+
step=$(echo "$step_out $step_in" | awk '{ m = ""; for (k = 1; k <= NF; k++) if (m == "" || $k < m) m = $k; print m }')
|
|
134
|
+
if [ -z "$score" ]; then
|
|
135
|
+
warn "$label: frames could not be compared (different sizes?)"
|
|
136
|
+
elif [ -z "$step" ]; then
|
|
137
|
+
warn "$label: SSIM $score, but the clips are too short to measure their own frame step; judge it by eye"
|
|
138
|
+
elif awk -v s="$score" -v t="$step" -v m="$MARGIN" 'BEGIN { exit !(s >= t - m) }'; then
|
|
139
|
+
ok "$label: SSIM $score (frame to frame inside the clips: $step)"
|
|
140
|
+
else
|
|
141
|
+
warn "$label: SSIM $score against $step frame to frame inside the clips; look at it. A different picture means the handoff used the wrong frame"
|
|
142
|
+
fi
|
|
143
|
+
if [ -n "$FRAMES_DIR" ]; then
|
|
144
|
+
ffmpeg -v error -y -i "$a" -i "$b" -filter_complex hstack "$FRAMES_DIR/seam_$(printf '%02d' "$s").png" \
|
|
145
|
+
&& echo " side by side: $FRAMES_DIR/seam_$(printf '%02d' "$s").png"
|
|
146
|
+
fi
|
|
147
|
+
fi
|
|
148
|
+
s=$next
|
|
149
|
+
done
|
|
150
|
+
|
|
151
|
+
echo
|
|
152
|
+
if [ "$WARNINGS" -eq 0 ]; then
|
|
153
|
+
echo "RESULT: PASS ($n clip(s), $((n - 1)) seam(s))"
|
|
154
|
+
exit 0
|
|
155
|
+
fi
|
|
156
|
+
echo "RESULT: $WARNINGS warning(s)"
|
|
157
|
+
[ "$STRICT" -eq 1 ] && exit 1
|
|
158
|
+
exit 0
|