song2mv 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- song2mv-0.1.0/.claude/skills/song2mv/SKILL.md +687 -0
- song2mv-0.1.0/.gitattributes +1 -0
- song2mv-0.1.0/.gitignore +21 -0
- song2mv-0.1.0/.mergetrain.yaml +43 -0
- song2mv-0.1.0/AGENTS.md +86 -0
- song2mv-0.1.0/AGENTS.mergetrain.md +39 -0
- song2mv-0.1.0/CHANGELOG.md +27 -0
- song2mv-0.1.0/CLAUDE.md +10 -0
- song2mv-0.1.0/CONTRIBUTING.md +88 -0
- song2mv-0.1.0/LICENSE +21 -0
- song2mv-0.1.0/PKG-INFO +407 -0
- song2mv-0.1.0/README.md +363 -0
- song2mv-0.1.0/SECURITY.md +12 -0
- song2mv-0.1.0/bench/jamendolyrics.py +595 -0
- song2mv-0.1.0/bench/whatif_first_word.py +162 -0
- song2mv-0.1.0/discovery/metadata.yaml +23 -0
- song2mv-0.1.0/docs/benchmark.md +721 -0
- song2mv-0.1.0/docs/contract.md +156 -0
- song2mv-0.1.0/docs/install.md +55 -0
- song2mv-0.1.0/docs/merge-queue.md +152 -0
- song2mv-0.1.0/docs/portrait-acting.md +153 -0
- song2mv-0.1.0/docs/pre-release-log.md +813 -0
- song2mv-0.1.0/docs/product-scope.md +56 -0
- song2mv-0.1.0/docs/release.md +36 -0
- song2mv-0.1.0/docs/timing-layer.md +156 -0
- song2mv-0.1.0/examples/song.example.json +22 -0
- song2mv-0.1.0/llms.txt +39 -0
- song2mv-0.1.0/notes.md +20 -0
- song2mv-0.1.0/pyproject.toml +92 -0
- song2mv-0.1.0/scripts/mergetrain/daemon.sh +102 -0
- song2mv-0.1.0/scripts/mergetrain/land-pr.sh +38 -0
- song2mv-0.1.0/scripts/mergetrain/pre-push +12 -0
- song2mv-0.1.0/scripts/mergetrain/setup.sh +21 -0
- song2mv-0.1.0/scripts/mergetrain/verify-ci.sh +34 -0
- song2mv-0.1.0/src/song2mv/__init__.py +2 -0
- song2mv-0.1.0/src/song2mv/__main__.py +3 -0
- song2mv-0.1.0/src/song2mv/check/__init__.py +0 -0
- song2mv-0.1.0/src/song2mv/check/font.py +140 -0
- song2mv-0.1.0/src/song2mv/check/goals.json +45 -0
- song2mv-0.1.0/src/song2mv/check/score.py +481 -0
- song2mv-0.1.0/src/song2mv/check/sheet.py +250 -0
- song2mv-0.1.0/src/song2mv/cli.py +361 -0
- song2mv-0.1.0/src/song2mv/contract.py +44 -0
- song2mv-0.1.0/src/song2mv/errors.py +13 -0
- song2mv-0.1.0/src/song2mv/render/__init__.py +0 -0
- song2mv-0.1.0/src/song2mv/render/runner.py +234 -0
- song2mv-0.1.0/src/song2mv/render/template/package-lock.json +4811 -0
- song2mv-0.1.0/src/song2mv/render/template/package.json +28 -0
- song2mv-0.1.0/src/song2mv/render/template/public/.gitkeep +0 -0
- song2mv-0.1.0/src/song2mv/render/template/render.mjs +240 -0
- song2mv-0.1.0/src/song2mv/render/template/src/Concepts.tsx +49 -0
- song2mv-0.1.0/src/song2mv/render/template/src/Contact.tsx +91 -0
- song2mv-0.1.0/src/song2mv/render/template/src/Gallery.tsx +1878 -0
- song2mv-0.1.0/src/song2mv/render/template/src/LyricShort.tsx +82 -0
- song2mv-0.1.0/src/song2mv/render/template/src/LyricVideo.tsx +45 -0
- song2mv-0.1.0/src/song2mv/render/template/src/PortraitDemo.tsx +49 -0
- song2mv-0.1.0/src/song2mv/render/template/src/Root.tsx +44 -0
- song2mv-0.1.0/src/song2mv/render/template/src/fonts.d.ts +16 -0
- song2mv-0.1.0/src/song2mv/render/template/src/index.ts +4 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/backgrounds.tsx +260 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/camera.tsx +120 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/cash.tsx +91 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/city.tsx +713 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/color.ts +20 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/crowd.tsx +815 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/effects.tsx +684 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/flames.tsx +205 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/font.ts +39 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/hit.tsx +156 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/index.ts +43 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/math.ts +29 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/nature.tsx +461 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/objects.tsx +124 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/people.tsx +749 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/phone.tsx +492 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/plan.ts +382 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/portrait-body.tsx +433 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/portrait-cutout.tsx +41 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/portrait-interaction.tsx +485 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/portrait-motion.ts +171 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/recipes.tsx +52 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/room.tsx +641 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/sky.tsx +336 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/song.tsx +194 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/storm.tsx +253 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/storyboard.tsx +203 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/comic.tsx +544 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/finish.tsx +178 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/line.tsx +140 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/paint.tsx +106 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/paper.tsx +187 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/pixel.tsx +609 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/poster.tsx +669 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/styles/toon.tsx +72 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/symbols.tsx +524 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/tears.tsx +161 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/text.tsx +187 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/vehicles.tsx +589 -0
- song2mv-0.1.0/src/song2mv/render/template/src/kit/vertical.tsx +103 -0
- song2mv-0.1.0/src/song2mv/render/template/src/sample-song.json +182 -0
- song2mv-0.1.0/src/song2mv/render/template/src/scenes/LyricScene.tsx +60 -0
- song2mv-0.1.0/src/song2mv/render/template/src/scenes/Starter.tsx +300 -0
- song2mv-0.1.0/src/song2mv/render/template/src/scenes/concepts/index.ts +4 -0
- song2mv-0.1.0/src/song2mv/render/template/src/scenes/index.ts +8 -0
- song2mv-0.1.0/src/song2mv/render/template/test-assets.mjs +367 -0
- song2mv-0.1.0/src/song2mv/render/template/test-interactions.mjs +1366 -0
- song2mv-0.1.0/src/song2mv/render/template/tsconfig.json +15 -0
- song2mv-0.1.0/src/song2mv/timing/__init__.py +0 -0
- song2mv-0.1.0/src/song2mv/timing/align.py +378 -0
- song2mv-0.1.0/src/song2mv/timing/analyze_music.py +72 -0
- song2mv-0.1.0/src/song2mv/timing/beat_grid.py +581 -0
- song2mv-0.1.0/src/song2mv/timing/build_lyrics.py +308 -0
- song2mv-0.1.0/src/song2mv/timing/click_check.py +97 -0
- song2mv-0.1.0/src/song2mv/timing/extract_lyrics.py +47 -0
- song2mv-0.1.0/src/song2mv/timing/lib.py +428 -0
- song2mv-0.1.0/src/song2mv/timing/numbers.py +115 -0
- song2mv-0.1.0/src/song2mv/timing/pipeline.py +396 -0
- song2mv-0.1.0/src/song2mv/timing/reprompt.py +287 -0
- song2mv-0.1.0/src/song2mv/timing/sections.py +124 -0
- song2mv-0.1.0/src/song2mv/timing/to_lrc.py +42 -0
- song2mv-0.1.0/src/song2mv/timing/to_srt.py +38 -0
- song2mv-0.1.0/src/song2mv/timing/transcribe.py +115 -0
- song2mv-0.1.0/tests/test_align.py +54 -0
- song2mv-0.1.0/tests/test_beats.py +49 -0
- song2mv-0.1.0/tests/test_cache.py +65 -0
- song2mv-0.1.0/tests/test_check.py +449 -0
- song2mv-0.1.0/tests/test_cli.py +32 -0
- song2mv-0.1.0/tests/test_click.py +101 -0
- song2mv-0.1.0/tests/test_config.py +55 -0
- song2mv-0.1.0/tests/test_errors.py +56 -0
- song2mv-0.1.0/tests/test_formats.py +75 -0
- song2mv-0.1.0/tests/test_guide.py +75 -0
- song2mv-0.1.0/tests/test_lyrics.py +278 -0
- song2mv-0.1.0/tests/test_numbers.py +71 -0
- song2mv-0.1.0/tests/test_relisten.py +203 -0
- song2mv-0.1.0/tests/test_render.py +221 -0
- song2mv-0.1.0/tests/test_repeats.py +130 -0
- song2mv-0.1.0/tests/test_sections.py +83 -0
- song2mv-0.1.0/tests/test_sheet.py +171 -0
- song2mv-0.1.0/tests/test_swing.py +127 -0
- song2mv-0.1.0/tests/test_template.py +804 -0
- song2mv-0.1.0/tests/test_tempo.py +194 -0
- song2mv-0.1.0/tests/test_video_sheet.py +225 -0
|
@@ -0,0 +1,687 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: song2mv
|
|
3
|
+
description: Make a beat-synced music video or lyric video for a song, drawn in code — lyric and beat timing, scene design, 16:9 and 9:16 renders, objective checks, fixes. Also for synced lyrics only (LRC, SRT, word-timed JSON) or a song's beat grid. Use when the user asks for a music video, lyric video, visualizer, karaoke video, Shorts/Reels clip, LRC/SRT or beat timing for a song file.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Music video from a song (song2mv)
|
|
7
|
+
|
|
8
|
+
You turn one audio file into a music video whose every frame is code (React/SVG in Remotion), synced to the song's
|
|
9
|
+
beat and lyrics. `song2mv` has the tools; this file is the order to use them in and what to watch for.
|
|
10
|
+
|
|
11
|
+
The loop is: **timing → reading the lyrics → scene design → render → check → fix**. You cannot hear the song, so
|
|
12
|
+
every judgement that needs ears goes to the user as a short clip or click track that makes it a ten-second check.
|
|
13
|
+
|
|
14
|
+
**Checkpoints don't block.** At each one, show the user what you have, with your pick and the reason in one line,
|
|
15
|
+
and keep going; they can redirect you at any time. Decide timing by measurement, send the ear-check clips anyway and
|
|
16
|
+
list the spots nobody has heard when you deliver. Wait only for what you can neither measure nor recommend: the
|
|
17
|
+
song's rights, or a lyric only the user knows.
|
|
18
|
+
|
|
19
|
+
## 0. Before anything
|
|
20
|
+
|
|
21
|
+
- **Rights.** Ask whose song it is and whether the user may publish a video of it (AI music services' plan terms
|
|
22
|
+
count). Never copy the song, its lyrics or renders into a repository; everything song-specific lives in the work
|
|
23
|
+
folder (`song2mv-work/<song>/`).
|
|
24
|
+
- **Setup**, once: `uv tool install --python 3.13 "song2mv[sync]"` (faster-whisper has no Python 3.15
|
|
25
|
+
wheels). Needs ffmpeg; `render` needs Node 18+ and installs Remotion
|
|
26
|
+
and a headless Chrome (~700 MB) on its first run.
|
|
27
|
+
- **What they want**: full 16:9 video, 9:16 short(s), LRC/SRT only, or all. Title text. Mood or references. If you
|
|
28
|
+
make the song too (Suno and the like), name a BPM in the style prompt: it tends to give a steady tempo.
|
|
29
|
+
|
|
30
|
+
## 1. Timing
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
song2mv sync <audio> --lyrics <lyrics.txt> --language <code> --title "<title>" --json # run in the background; minutes
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
A Suno download also carries timed lines, a subtitle track (`ffmpeg -i song.m4a -map 0:s:0 suno.srt`): on two songs
|
|
37
|
+
they agreed with `sync`'s line starts to a median 0.07 s and 0.32 s and were 1.3–2.8 s apart on three lines, where
|
|
38
|
+
the whisper runs sided with `sync` twice. Read them as a second opinion on `check_by_ear`, not as the timing.
|
|
39
|
+
|
|
40
|
+
`--lyrics` is optional when the audio has a lyrics tag (Suno writes one). If you also have a lyric sheet, pass it:
|
|
41
|
+
where it and the tag differ, `warnings` names the lines and says which of the two the whisper runs support there (for
|
|
42
|
+
one song the sheet was right, for another the tag). `off_sheet` lists stretches where the runs agree on words that no
|
|
43
|
+
line matched: chants, ad-libs, changed lines, a repeat the sheet has once, or a line of the sheet placed elsewhere
|
|
44
|
+
(its words are on the sheet; that line is then usually on `check_by_ear`). Sync again with a sheet that matches what
|
|
45
|
+
the runs heard (`<work>/asr/run*.json` has every word), and put the stretches you add into the user's ear check. A
|
|
46
|
+
chant without words to read ("oh, oh, oh") may stay off the sheet and the captions when the song map gives its stretch
|
|
47
|
+
an event of its own (a break, step 3). Read the progress log and the JSON:
|
|
48
|
+
|
|
49
|
+
- **Section parsing** (`── [Chorus] → ch1 (hot)`): are the kinds right? A section named calm (a bridge, an outro)
|
|
50
|
+
that is about as loud as the verses becomes `mid`, and `warnings` says so. Fix with `section_kinds` in `song.json`.
|
|
51
|
+
Lines in `( )` were skipped as stage directions unless their section is in `keep_parenthesized_in`. A `/` or `–`
|
|
52
|
+
between words stays in the line's `text`, where a scene can split on it, but is not a word: it has no time and is
|
|
53
|
+
left out of the captions, `song.lrc` and `song.srt`. A sheet without `[Section]` lines gets sections proposed from
|
|
54
|
+
its repeated lines, listed in `warnings` and written to `<work>/lyrics.sections.txt`: if one is wrong, edit that file
|
|
55
|
+
and sync again with `--lyrics` pointing to it.
|
|
56
|
+
- **`◀ no run found this line` / `◀ only run [k]` / `◀ runs disagree`**: weak spots. More runs help
|
|
57
|
+
(`--runs 5`, finished runs are reused). A bigger model (`--model medium`) helps more but is slower.
|
|
58
|
+
`◀ back in lyric order (was at T)`: the merged runs had put the line on another copy of its words, so it was
|
|
59
|
+
merged again from the runs that kept the sheet's order; check it by ear.
|
|
60
|
+
- **Tempo line** (`→ 90.00 BPM ... 53% of lyric line gaps are whole bars`): if the fit is low or a close runner-up
|
|
61
|
+
exists, say so. **Bar start** (`beat % 4 == 3`): a guess from the kick drum.
|
|
62
|
+
- **Grid fit** (`grid fit, 4-bar windows …: off 3:02–4:10 (shift, 90 BPM)`, and a warning per stretch): the song
|
|
63
|
+
leaves the grid's one tempo there, and `grid.json` lists it under `off_grid`. AI-made songs can drift in tempo;
|
|
64
|
+
`sync` follows the drift with a beat map (`beat map: … beats` in the log, `beats` in `grid.json`), and words,
|
|
65
|
+
`grid.pulse`, `grid.bar` and the clicks follow it. Without a map (`no beat map`), words there are not snapped, and
|
|
66
|
+
motion driven by `grid.pulse` or `grid.bar` will miss the music there: drive those stretches by line and word
|
|
67
|
+
times instead. Put a click track over the start of the stretch. A warning that the grid does not fit the whole
|
|
68
|
+
song names a `--bpm` to try; `the tempo pick is uncertain` names the tempo the onsets prefer.
|
|
69
|
+
- **Swing** (`swing: the hats put the 2nd and 4th 16th notes … at 0.325 and 0.829`, `swing` in `grid.json`): words
|
|
70
|
+
and `grid.snap(t)` sit on those swung 16ths; let the user hear the word ticks of `click --words` over a dense verse.
|
|
71
|
+
- **`check_by_ear`** in the JSON: lines filled by re-listening (`reprompt`), estimated, placed by only one whisper
|
|
72
|
+
run, out of the lyric sheet's order (a repeated line on another copy of its words), with a word no run timed
|
|
73
|
+
between words far apart (the log names the word: `◀ no run timed "…"`; the words around it may be off too), or
|
|
74
|
+
spread over far more time than they are sung in (`◀ … spread out: …` in the log: usually split between two copies
|
|
75
|
+
of a repeat the sheet has once; look for an `off_sheet` stretch or a tag warning next to it), with three words at
|
|
76
|
+
almost one time or two that another copy of the line has apart (`◀ … stacked: …`), with runs that disagree by more
|
|
77
|
+
than 0.7 s (`◀ runs disagree by …`; the log flags it from 0.4 s, and below 0.7 s such lines mostly start on time) or
|
|
78
|
+
on a word (`◀ no run times "…" near …`), or heard alike by the runs but not as written while another copy of it is
|
|
79
|
+
(a warning `… is heard as "…"`: Suno changed that repeat, or whisper misheard it the same way in every run, which
|
|
80
|
+
is what every such warning on the JamendoLyrics benchmark was; put the line into the user's ear check with both
|
|
81
|
+
versions, as written and as heard, and change the sheet only after the user confirms). Most likely wrong. A warning
|
|
82
|
+
"heard N times" names the other times a re-heard line was heard: make the click track cover them all.
|
|
83
|
+
|
|
84
|
+
Then make the ear check cheap for the user:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
song2mv click <work> --from <chorus start - 4> --to <+12> --words # → <work>/click-<from>-<to>.wav
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Send it with the question: do the clicks sit on the drums, is the high click on the "one", do the word ticks land on
|
|
91
|
+
the syllables? Meanwhile decide by measurement: where the whisper runs agree, the grid fit, a repeated section's
|
|
92
|
+
timing. Fix with `"bpm"`, `"downbeat_mod4"` and `"manual": {"<line id>": [word start times]}` in `<work>/song.json`,
|
|
93
|
+
then `song2mv sync <work>/song.json` (only cheap steps rerun). Shouts and chants nearly always need a few `manual`
|
|
94
|
+
values. A pin only constrains: re-heard lines around it keep their times while they still fit between the pins, so
|
|
95
|
+
pin only the lines that are wrong; a re-heard line a pin contradicts is heard again and stays on `check_by_ear`.
|
|
96
|
+
|
|
97
|
+
Outputs in `<work>/`: `grid.json`, `lyrics.json`, `mv.json` (the template's input), `song.lrc`, `song.words.lrc`,
|
|
98
|
+
`song.srt`, `music.json` (loudness and section boundaries). If the user only wanted subtitles, you are done here.
|
|
99
|
+
|
|
100
|
+
## 2. Baseline render
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
song2mv render <work>/mv.json --from <a hot section start - 6> --to <+10> --aspect both
|
|
104
|
+
song2mv render <work>/mv.json --still <time> --aspect both # PNGs you can look at
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Look at the stills yourself. This plain lyric video is the floor: captions in time, sections coloured by energy,
|
|
108
|
+
choruses big. Show it to the user early — sometimes it is all they need.
|
|
109
|
+
|
|
110
|
+
## 3. Lyrics, concept, then the song map
|
|
111
|
+
|
|
112
|
+
**Read the lyrics first, line by line,** into `<work>/lyrics-map.md`: a row per line id from `mv.json`, with what the
|
|
113
|
+
line says and what it does. Most lines just say something. The lines that do something (a pun, a metaphor, a parody)
|
|
114
|
+
are the jokes and images the song is made of, and drawing their nouns misses them. One trial's first take gave each
|
|
115
|
+
line a prop for its nouns: 6 of its 17 wordplay lines never reached the screen, 4 showed one of their two meanings,
|
|
116
|
+
and its on-screen tags read out what the picture already showed. The user's first note was that the jokes were
|
|
117
|
+
missing. For each line that does something, write the device, the picture and the text it needs:
|
|
118
|
+
|
|
119
|
+
| The line … | Show |
|
|
120
|
+
|---|---|
|
|
121
|
+
| makes a metaphor or simile ("my heart's a stereo") | its image (a heart-shaped boombox), not the singer saying it |
|
|
122
|
+
| means two things (a pun, a double meaning) | both meanings, one turning into the other |
|
|
123
|
+
| sounds like another word ("I knead you") | the other word's picture (hands in dough) and both words as text, "knead → need": a picture cannot show that two words sound alike |
|
|
124
|
+
| bends a famous line (a parody, a quote) | the famous image, then the twist |
|
|
125
|
+
| names a person, brand or work | what the song uses the name for (its sound, its trait), never the real person |
|
|
126
|
+
| lists or rhymes several things | one picture per word, each on its word |
|
|
127
|
+
| comes back (a callback, the hook again) | the earlier picture, changed |
|
|
128
|
+
|
|
129
|
+
A line you would rather not draw (a slur, a sexual line) gets the harmless reading the song itself offers, such as
|
|
130
|
+
the word its rhyme turns it into, or a censor gag; never nothing, since a skipped punchline reads as a video that
|
|
131
|
+
missed the joke. On-screen text names the twist ("knead → need"), not what the picture shows. Choose the concept with
|
|
132
|
+
the map in hand: a song with a device in every other line needs a frame where a new picture per line fits (a TV
|
|
133
|
+
show's screen, a comic strip, a quiz), not one long scene. Show the user the lines you found doing something, with
|
|
134
|
+
your reading of each, and carry on: a slang word or an in-joke you misread costs a row of the map now and a scene
|
|
135
|
+
later.
|
|
136
|
+
|
|
137
|
+
Every song gets its own look. One kit or template applied to every song makes every video look alike, so the look is
|
|
138
|
+
decided before you touch the kit.
|
|
139
|
+
|
|
140
|
+
**Propose two or three concepts and pick one.** They must differ in more than colour: a different art style and a
|
|
141
|
+
different central image. For each, write:
|
|
142
|
+
|
|
143
|
+
- **Idea:** one line, the image that carries the song, taken from its lyrics and mood.
|
|
144
|
+
- **Art style:** what fits this song, not what is quickest to build. Some options:
|
|
145
|
+
- flat illustration (the kit's own style);
|
|
146
|
+
- paper cut-out with drop shadows (the kit has a style pack for it);
|
|
147
|
+
- line art: ink on paper, chalk on a blackboard, a blueprint or neon tubes on brick (the kit has a style pack for
|
|
148
|
+
all four, which traces any drawing, kit pieces included);
|
|
149
|
+
- pixel art (the kit has a style pack for it too);
|
|
150
|
+
- toon: an ink outline round each character and prop, shading from a lit side and a shine on round forms (the kit
|
|
151
|
+
has a style pack for it, which outlines kit pieces as they are);
|
|
152
|
+
- watercolour washes over a pencil drawing (the kit has a style pack for it, which paints kit pieces as they are),
|
|
153
|
+
or collage;
|
|
154
|
+
- an isometric miniature;
|
|
155
|
+
- kinetic typography only;
|
|
156
|
+
- over any of these, an old film, a VHS tape or a halftone print (the kit's `finish` pack).
|
|
157
|
+
- **Palette, texture and typeface.** The typeface is part of the look: rather than a system font, use an OFL font
|
|
158
|
+
file imported from `src/` (`song2mv render` ignores `public/`) and loaded with the kit's `loadFont` at the top of a
|
|
159
|
+
module: `import lilita from "../fonts/LilitaOne-Regular.ttf"` and `export const DISPLAY = loadFont("Lilita One",
|
|
160
|
+
lilita)` give `fontFamily` its value, and every frame waits for the file. Name the family in `theme.font` to set the
|
|
161
|
+
captions in it; the module loads only if a scene file imports it (one that nothing imported left captions in a serif). Not a `delayRender()` of your own at the top of a module: Remotion drops its handle, and the timer
|
|
162
|
+
left behind failed one full render and stalled the next.
|
|
163
|
+
- **Main character or motif,** and how it changes over the song.
|
|
164
|
+
- **First chorus:** how it hits, the song's biggest moment.
|
|
165
|
+
|
|
166
|
+
**What the user judged fun.** Four new-user runs were ranked by fun, not by the checks. Two judged fun did all of
|
|
167
|
+
these; the one judged less fun did none of the first three and saved minutes with a plainer concept (27 against 40 and
|
|
168
|
+
43): don't buy minutes that way. The latest, made from this list and the lyric map without any earlier run's clips or
|
|
169
|
+
notes, was judged more fun than the run before it on the same song ("both good, the newer one more to my taste"): a
|
|
170
|
+
device on the key word of 32 of its 33 lines and five kinds of shot (a telestrator, a photo finish, a jumbotron
|
|
171
|
+
replay, a press conference, a thermal camera), though only its cat's face was in view.
|
|
172
|
+
|
|
173
|
+
1. **Something happens on the key word of each line,** not a picture of the line's noun: a paw pushes the mug off the
|
|
174
|
+
desk on "three", a telestrator draws the play as it is rapped. The lyric map says which word and what.
|
|
175
|
+
2. **Reactions.** Cut from the act to someone reacting: a crowd, an old man yelling, a mug sweating. Props with faces
|
|
176
|
+
are cheap extra reactors.
|
|
177
|
+
3. **A viewpoint that shows faces.** Top-down or far wide shots hide the acting: the dull run's toy-town view turned
|
|
178
|
+
its people into dots.
|
|
179
|
+
4. **A frame that turns the everyday into an event:** a sports broadcast, a heist, a nature film, a period cartoon, a
|
|
180
|
+
TV show. It brings its own overlays, sound words and title cards.
|
|
181
|
+
5. **Change the kind of shot, not just its zoom:** a diagram, a side view, a close-up, a new world for a punchline.
|
|
182
|
+
|
|
183
|
+
Make each concept concrete enough to picture: one rough still each, shown side by side with your pick and the reason
|
|
184
|
+
in one line. Copy the template now (step 4's first command), draw each concept as a component of `T` in
|
|
185
|
+
`<work>/video/src/scenes/concepts/`, export it by name from `src/scenes/concepts/index.ts`, and render them all at one
|
|
186
|
+
moment, such as the first chorus' downbeat: write that time into `<work>/video/concepts.json` (`{"T": 33.1}`) and run
|
|
187
|
+
this in `<work>/video`. It prints the PNG's path. Each concept is drawn alone, without the template's sky, captions
|
|
188
|
+
or vignette:
|
|
189
|
+
|
|
190
|
+
```bash
|
|
191
|
+
node render.mjs ../mv.json --composition Concepts --props concepts.json
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
A copy made by an older song2mv has a `render.mjs` that does not know `node render.mjs --composition`: it renders the whole song as a 16:9 video into
|
|
195
|
+
`out/` instead, and prints no PNG path. Stop it and use the `npx remotion still … --props` command in that copy's
|
|
196
|
+
`src/Concepts.tsx` (`src/Contact.tsx` for contact sheets), or make a new `--template` folder and move `src/scenes/`
|
|
197
|
+
and any kit changes into it.
|
|
198
|
+
|
|
199
|
+
**Then the song map** for the chosen concept, `<work>/map.md`. Show it to the user and carry on:
|
|
200
|
+
|
|
201
|
+
- Per section (from `mv.json` sections; `<work>/music.json` has the loudness curve and likely boundaries): bars,
|
|
202
|
+
energy (rising / peak / falling), what the lyrics say, **one visual idea**, colour, and where the lyric map's
|
|
203
|
+
pictures go in it.
|
|
204
|
+
- **One main character or motif** that carries through the whole song and changes with it. A video with a
|
|
205
|
+
protagonist is more watchable than a slideshow of effects.
|
|
206
|
+
- **Hit points**: the first beat of each chorus, drops, the last word. These get the biggest moment (cut, flash,
|
|
207
|
+
slam). Use `grid.bar(k)` and line/word times — never hand-typed seconds.
|
|
208
|
+
- **Breaks**: list every stretch of 4 s or more without a sung line in `mv.json` (an intro, an instrumental, an
|
|
209
|
+
"oh-oh" that `off_sheet` names) and give each an event of its own: everything dances, someone joins in, a new
|
|
210
|
+
world. One world held through a break is the dullest part of a chorus clip.
|
|
211
|
+
- **Contrast**: chorus 2 must not look like chorus 1 (the check measures it). The same words in the same place
|
|
212
|
+
look alike to it even over another sky: move or restyle chorus 2's big type and boxes too. Verses cut on an uneven
|
|
213
|
+
rhythm (2 bars, 1 bar, 3 bars …), not every bar.
|
|
214
|
+
- **Text**: which lines become big type in the scene (`Slam`, `BigLine`), which stay as bottom captions.
|
|
215
|
+
|
|
216
|
+
**Then get the pieces, in this order:**
|
|
217
|
+
|
|
218
|
+
1. **The concept decides.** Pick it first (above), then look in the kit for what fits it, not for a concept that fits
|
|
219
|
+
the kit.
|
|
220
|
+
2. **Reuse what fits the concept's style.** A style pack in `src/kit/styles/` puts any object in one art style
|
|
221
|
+
(`paper`: cut-out pieces with shadows and grain; `pixel`: sprites on a canvas of big pixels). The kit's own pieces
|
|
222
|
+
are flat illustration: they fit a flat concept and look out of place in any other, unless a pack restyles them
|
|
223
|
+
(`pixel.Pixelate` turns any of them into pixels).
|
|
224
|
+
3. **Change your copy.** The song's template copy (`--template`, step 4) is yours: change a kit piece or a pack there
|
|
225
|
+
(a colour, a shape, another pose) instead of working around it. Upgrades never touch it.
|
|
226
|
+
4. **Draw what is new** in the concept's style. With a pack, draw it from the pack's pieces. Without one, build the
|
|
227
|
+
style first (an ink line, a watercolour wash) and draw the song's objects on top. The kit's timing and
|
|
228
|
+
motion code works in any style:
|
|
229
|
+
- `useSong()` and `grid`;
|
|
230
|
+
- `Camera` and `cameraAt()`, `Drift`, `liveLine`, and the easing and `rng` helpers in `math`;
|
|
231
|
+
- the helpers that take their colours and shapes as props: `Hit`, `Flames`, `Lightning` and `Storm`, `Tears`,
|
|
232
|
+
`Cash`;
|
|
233
|
+
- how `effects.tsx` times its particles.
|
|
234
|
+
|
|
235
|
+
Before rendering the whole song, test a frame: if it could sit in the kit's gallery unchanged, it is not this song's
|
|
236
|
+
video yet.
|
|
237
|
+
|
|
238
|
+
Ask yourself honestly: would a stranger watch this past the first chorus? If the answer is "it is correct", it is
|
|
239
|
+
not done — make it funnier, stranger, more specific to this song.
|
|
240
|
+
|
|
241
|
+
## 4. Build scenes
|
|
242
|
+
|
|
243
|
+
```bash
|
|
244
|
+
song2mv render <work>/mv.json --template <work>/video --still 1 # first use copies the template there
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Write the song's scenes in `<work>/video/src/scenes/` and export them from `src/scenes/index.ts` (`Scene` and
|
|
248
|
+
`linesDrawnByScene`). Draw them in the chosen concept's style, and take what fits from the kit in `src/kit/`.
|
|
249
|
+
|
|
250
|
+
**A whole song goes faster split among agents** (two songs, audio to videos, in about two hours): shared props first
|
|
251
|
+
from one brief and palette, then one builder per section in its own `--template` copy, rendering only that copy. After
|
|
252
|
+
merging, check the whole song and a 9:16 sheet of every board: builders' stretches miss lines and `tall` framings.
|
|
253
|
+
|
|
254
|
+
**Plan the cuts with the kit; don't write cut logic.** In five trial videos every agent wrote 750–1200 lines of scene
|
|
255
|
+
code and rebuilt the same cut machinery, and one lost half an hour iterating cut plans against `check`. The kit has it:
|
|
256
|
+
|
|
257
|
+
- `planShots(ctx, from, to, framings)` (`src/kit/plan.ts`) plans a stretch's cuts, which framing each takes, its drift
|
|
258
|
+
and the hits, by section kind: verses cut unevenly on bar lines and a 16th before a line's first word, never twice
|
|
259
|
+
in a short line, at least every 5.2 s; calm sections hold slow shots; hot sections cut on downbeats with a punch;
|
|
260
|
+
copy n of a section opens on framing n − 1 (chorus 2 on the second). It returns the kit camera's `shots` and `hits`;
|
|
261
|
+
`pattern` sets your own bar counts.
|
|
262
|
+
- A `Board` (`src/kit/storyboard.tsx`) is a stretch that starts at a line id, a section id or a time: a `world` (SVG of
|
|
263
|
+
`T` on the stage), its `framings` (and `tall` ones for 9:16), and optionally `pattern`, `drift`, `hits`, an
|
|
264
|
+
`overlay` (text over the camera) and `big` (its lines as big type, where the captions go). A section cuts in the
|
|
265
|
+
`pattern` of the board it opens on, one at the section, at its first line or at a line sung into it just before;
|
|
266
|
+
a video render's log names a pattern no section takes.
|
|
267
|
+
`<Storyboard T={T} boards={…} />` draws the board on at T (`wrap` holds the filmed world in a style pack's
|
|
268
|
+
`paper.Stack` or the like), the same with `vertical` is the `VerticalScene`, and `linesDrawnBy(song, boards)` gives
|
|
269
|
+
`linesDrawnByScene`. A board's start is a cut, planned with its section's other cuts: the rhythm counts it, and a
|
|
270
|
+
section with boards at its lines cuts inside as few lines as its rhythm allows, inside none under 2.5 s if it can.
|
|
271
|
+
Key boards to the lyric map's lines whose picture needs a new place or subject, every 2 lines or more and unevenly:
|
|
272
|
+
in one outro three boards 2 lines apart cut too evenly for `cuts.rhythm`. The map's other lines change something
|
|
273
|
+
inside the board on their own line (a prop, a sign, a screen; `liveLine(span.lines, T)` is the line sung at T).
|
|
274
|
+
Take a board's lines from `span.lines`, not from the song by time: a board keyed to a line opens only a 16th
|
|
275
|
+
before its first word (0.1 s at 150 BPM). One song's overlay tags started 0.12 s before each first word and were
|
|
276
|
+
dropped when that fell before the board's start, so the tag of the line each of 11 boards was keyed to never
|
|
277
|
+
showed. A board at a line opens on its first framing in every copy of the section; a board at a section opens on
|
|
278
|
+
framing n − 1 in copy n (`v2` on `framings[1]`), so put that copy's opening there.
|
|
279
|
+
So does a board at the same line of every copy with the same framings (`ch1-04`, `ch2-04`): a board repeated in
|
|
280
|
+
each chorus needs as many framings as there are copies, each composing the subject differently (off-centre,
|
|
281
|
+
rolled, close). Another background alone measured alike in one chorus (0.81 → 0.73); another composition passed
|
|
282
|
+
(0.36). A video render's log names boards at one line of two copies that open on the same framing.
|
|
283
|
+
- A cut counts only when the picture changes: give a world framings that show different parts of it (the sky, the
|
|
284
|
+
ground, the subject up close), not one layout at two zooms. A large still overlay hides cuts from `check`. Framings
|
|
285
|
+
next to each other in the list need 1.35× apart in zoom, another part of the world or 4° of roll: the planner skips
|
|
286
|
+
a framing too alike to the one before it (a close-up at 2.4 after one at 1.8 never showed), and a draft's render
|
|
287
|
+
log names one it skipped that never shows (`storyboard: the board at v2-08 never shows framing 2 …`). The 9:16
|
|
288
|
+
frame skips a `tall` framing too alike to the one before it the same way.
|
|
289
|
+
- **Point a framing at its subject's middle**: the character or the thing the line is about. Both frames put the
|
|
290
|
+
point in the middle of their picture, above the captions and big type: y 0–840 of the 16:9 frame (`WIDE` in the
|
|
291
|
+
kit), y 269–1080 of the 9:16 one (`VERTICAL`). Zoom so the subject fits the picture, also when the shot's drift
|
|
292
|
+
pushes in by up to 15 %: a `Person` 300·s px tall fits up to zoom 2.3 / s. A closer shot runs on behind the
|
|
293
|
+
captions; let that be the ground, never a face or the line's prop. Both frames show the world past the stage's
|
|
294
|
+
bottom edge (16:9 by 120 px at zoom 1): draw the ground on below y 1080. A contact sheet draws the captions' top as
|
|
295
|
+
a dashed line.
|
|
296
|
+
|
|
297
|
+
**The starter** (`src/scenes/Starter.tsx`) is a whole video built that way from kit worlds (a street, a room, a stage,
|
|
298
|
+
fireworks, a beach, hills) that passes `check` on any song as it is. To start from it, put the line given in
|
|
299
|
+
`src/scenes/index.ts` in place of the `LyricScene` one. Then make it this song's: edit its marked block (`SHOW`: what
|
|
300
|
+
each section, copy or line shows and which lines are big type; `PALETTE`; `HERO`) and replace its worlds with the
|
|
301
|
+
concept's. Its look is the kit's: a starting point, never the video. When a line runs into the next section (a
|
|
302
|
+
count-in such as "one, two, three" that ends after the chorus starts), that section's board starts at its own first
|
|
303
|
+
line instead, so the line stays on screen to its last word.
|
|
304
|
+
|
|
305
|
+
**The frame comes from the song config.** Around your scenes the template draws a background, the title and the
|
|
306
|
+
captions in `theme`'s colours and font; in 9:16 its starry sky fills the screen above and below the stage. Over the
|
|
307
|
+
scenes it lays a dark vignette, and a white flash at the start of each hot section, placed on the one-tempo grid.
|
|
308
|
+
Left at the defaults, they put the kit's neon night around every concept, and the title is the file name. Set `title`
|
|
309
|
+
and `theme` (`bg`, `stars`, the text colours, `font`; `unsungShadow: "none"` on a light `bg`) in `song.json` to fit
|
|
310
|
+
the concept; `"vignette": false` keeps the
|
|
311
|
+
scene's corners as drawn, and `"flashes": [[song time, strength 0–1], …]` puts the flashes on the song's real hits
|
|
312
|
+
(`[]` when the scenes flash themselves). Then run `song2mv sync <work>/song.json --from build`.
|
|
313
|
+
|
|
314
|
+
### The scene kit
|
|
315
|
+
|
|
316
|
+
Look at it before you design anything (in `<work>/video`):
|
|
317
|
+
|
|
318
|
+
```bash
|
|
319
|
+
node render.mjs src/sample-song.json --composition Kit-Gallery # out/src-Kit-Gallery.png: all 32 scenes
|
|
320
|
+
node render.mjs src/sample-song.json --composition Kit-Gallery --frame 4 # out/src-Kit-Gallery-4.png: one (1–32)
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
`src/Gallery.tsx` shows how each of those scenes is put together. Every piece is a pure function of `T`; its doc
|
|
324
|
+
comment in `src/kit/` lists what each prop does. Scenes 1–12 are the kit's pieces, in one flat illustrated style.
|
|
325
|
+
Scene 13 is the paper style pack. Scenes 14–20 are helpers that fit any style, their colours and shapes being props
|
|
326
|
+
(most of those tiles draw the helper again, flat on paper), and a person in bed: one world filmed as shots (14), hits
|
|
327
|
+
(15), flames (16), a storm (17), tears (18), lying awake in bed (19), falling cash (20). Scene 21 is a recipe that
|
|
328
|
+
combines them. Scene 22 is the pixel style pack, scene 23 the comic style pack, scene 24 the pixel pack's type.
|
|
329
|
+
Scenes 25–28 are a body for a head of your own, such as a portrait: walking and gestures (25), the poster props (26),
|
|
330
|
+
a call and writing (27), a hand at the temple (28). Scenes 29–32 are the toon, line, paint and finish style packs.
|
|
331
|
+
|
|
332
|
+
| family | pieces | for |
|
|
333
|
+
|---|---|---|
|
|
334
|
+
| sky and light | `SkyGradient` (dawn, day, dusk, night; `to` + `k` blend; `stars`), `Sun`, `Moon`, `Clouds` (`time`, `light`, `size`), `Rays`, `BeatGlow`, `Flash`, `Bokeh`, `Dust`, `Snow` | time of day, mood, the beat in light |
|
|
335
|
+
| camera | `Camera` (one world filmed as `shots`: each cuts in at a bar or word time and looks at `x`, `y` with `zoom` and `rotate`, may drift `to` the next cut or `glide` in; `hits` punch the zoom in), `cameraAt()` (the same camera as numbers and as SVG or CSS transforms), `Drift` (a slow zoom), `planShots` and `Storyboard` (above) | cutting on the music |
|
|
336
|
+
| places | `City` (night, `dusk`, `day`; `shift` pans by depth), `Street` (pavement, road, lamps), `Room` (`view` through the window; `bed` + `sleeper`), `Bed` (a sleeper between mattress and blanket, `cover`, `head`; `over` for what lies on the bedding), `Ground` (ridges, town, trees), `Sea` (`light` path, `beach`), `GridFloor` | where it happens |
|
|
337
|
+
| people | `Person` (15 poses, `lie` on the back among them; `walk`/`run` by `phase`; `look`: outfit, hair, skin, colours; `prop` + `personHand()`; `personEye()` for tears or a glint), `Crowd` (`handsUp`, `jump`, `idle`, `lights`), `MiniPerson`; for a head of your own, such as a photo portrait: `PortraitCutout` (clips the head and neck out of a bust, so its shoulders don't double the body's), `PortraitBody` (from the front; arm angles, `feet`, `lean`; `gesture` `phone`, `point`, `smoke` or `hold` a `frontProp`), `portraitMotion()` (walk and run on planted feet, gestures that settle; turn the head by its `headAngle`), `PortraitInteraction` (a call, writing, a hand at the temple: hands, prop and contact worked out together from the head's `profile`; check them with the real portrait at the real camera size) | the protagonist, the audience |
|
|
338
|
+
| objects | `Phone` (`messages`, `typing`, `call`), `Clock` (`spin`, `ticks` on the beat, alarm `ring`), `Heart` (`beat`, `crack`), `Car` (`distance` turns the wheels), `Plane` (`trail`), `Lantern`/`Lanterns` | what the lyrics are about |
|
|
339
|
+
| effects | `Rain` (`strength`, splashes on `ground`), `Fireworks` (`bursts` at times), `Confetti`, `Hit` (`burst`, `sparkle` or `ring` at a point and time), `Flames` (a fire to a candle; `flat`, `wind`, `strength`) and `FlamesAlong` (an object's edge on fire), `Storm` (a storm deck of `Clouds` lit from inside by its `strikes`) and `Lightning` (`fork` or `zigzag` bolts; a strike's `to` aims it at a point), `strikeFlashes()` (for `Flash`), `Tears` (well up, run down a cheek, drip; `spot` dampens where they land), `Cash` (banknotes tumbling down) | weather, hits and the big moments |
|
|
340
|
+
| text | `Slam`, `Quiet`, `BigLine`, `Captions` | lines in the scene |
|
|
341
|
+
|
|
342
|
+
**Style packs** (`src/kit/styles/`, one namespace each: `import { paper, pixel, comic, poster } from "../kit"`).
|
|
343
|
+
Their filters cost little on a Mac, where `song2mv render` has Chrome draw on the GPU; in software (Linux, Windows,
|
|
344
|
+
`SONG2MV_GL=off`) Chrome draws them one frame at a time on one core: 16 s of the starter with `toon` over the whole
|
|
345
|
+
frame rendered in 21 s on the GPU and 5.6 min in software. A filter of your own: shift an `feOffset` a hair over a
|
|
346
|
+
whole or half number (1.01, not 1). Offsets land on whole pixels, and Chrome rounds an exact half up in software and
|
|
347
|
+
down on the GPU, where a line-pack draft (half size, so a shift of half a pixel) came out blank.
|
|
348
|
+
`paper`, for paper cut-out:
|
|
349
|
+
- outlines cut by hand, each with its own seed: `paper.rect`, `paper.ellipse`, `paper.poly`, `paper.strip` (stems,
|
|
350
|
+
cords, steam); join outlines to cut holes;
|
|
351
|
+
- `paper.Cut`: one piece, placed with `x`, `y`, `rotate`, `s`; `lift` raises it off the paper under it and its
|
|
352
|
+
shadow grows; a Cut inside a Cut is glued onto it;
|
|
353
|
+
- `paper.Stack id`: the grain and the light for every piece inside it, one per scene;
|
|
354
|
+
- `paper.colors` and `paper.matte(c)` for paper colours, `paper.step(T)` for the held steps of cut-out motion.
|
|
355
|
+
|
|
356
|
+
`pixel`, for pixel art. One pixel size per shot: draw the whole scene on one grid and let only the camera scale it,
|
|
357
|
+
by whole numbers; never `scale()` a sprite or a layer, or blocks of two sizes meet and edges go soft.
|
|
358
|
+
- `new pixel.Canvas()`: a picture in big pixels, drawn with `box`, `disc`, `poly`, `line` (1 px wide it is pixel
|
|
359
|
+
perfect: no doubled pixel where it turns), `dither` (a tone of a colour) and `text` (a 5×7 font: Latin letters,
|
|
360
|
+
digits and the rest of ASCII); positions round to whole pixels;
|
|
361
|
+
- `pixel.sprite(rows, palette)`: a character from a character map, a string per row and a character per pixel
|
|
362
|
+
(`"..HHhh.."` with `{ H: hair, h: shine }`; others stay clear). `canvas.stamp(sprite, outline, x, y)` puts it on
|
|
363
|
+
with a clean 1-pixel outline (inside corners filled, so the ring turns them on a diagonal). Draw people, animals
|
|
364
|
+
and anything round this way, a frame per pose, not from discs or sampled curves: eyes, ears, a tail and a wheel
|
|
365
|
+
need their pixels placed (scene 22: a walk in four frames, a cat asleep, a bicycle, a car). Part shapes that touch
|
|
366
|
+
inside a sprite (leaves, an arm over the body) with a darker shade, not a gap, since every gap gets an ink line,
|
|
367
|
+
and stamp what a character walks past before the character, so its outline stays whole. Check them at 1:1 and 2×;
|
|
368
|
+
- `pixel.Pixels`: a canvas on the stage, `px` stage px to a pixel (at 8, 240 × 135 pixels fill it), rounded to
|
|
369
|
+
whole stage px;
|
|
370
|
+
- `pixel.cameraAt(T, shots)`: the kit's camera for a pixel scene, `<g transform={pixel.cameraAt(…).svg}>` round it:
|
|
371
|
+
the zoom rounds to a whole number and the frame to whole pixels, kept on the stage, so a close-up is 2× or 3× with
|
|
372
|
+
one pixel size. A shot that changes the zoom cuts (blocks change size in one jump), so give each zoom a beat or
|
|
373
|
+
more; a glide or drift at one zoom pans by whole pixels;
|
|
374
|
+
- `pixel.BigText`: big type on the grid, `x` its centre and `y` its top in blocks of a `px` grid as in `Pixels`: the
|
|
375
|
+
5×7 font at `scale` blocks a bit, two-tone with `lower`, a `shadow` one bit down, one block of `outline` round all
|
|
376
|
+
of it with no specks in the letters; `width` wraps it (and steps the scale down while a word alone is wider), `T`
|
|
377
|
+
with `t` pops it in once. Not `text` + `stamp`: `stamp` rings every gap, so a text with a shadow gets ink specks
|
|
378
|
+
between the letters and their shadow. Text the 5×7 font lacks, such as Korean, comes from `font` (default: the
|
|
379
|
+
theme's) at 16 blocks to the em per scale (Hangul 14 blocks tall): it reads from scale 1, and with a shadow from
|
|
380
|
+
scale 2 if the shadow is dark under a light face; a bright shadow leaves its colour inside syllables with two final
|
|
381
|
+
consonants at any scale. The 16 blocks were measured with Pretendard, the theme's first font; Apple SD Gothic Neo,
|
|
382
|
+
where a Mac without it falls back, reads rougher (thin strokes, short bars as one-block stubs), and Noto Sans CJK
|
|
383
|
+
KR (Linux) reads cleanly. For the same letters on every machine, import an OFL Korean font file from `src/`, load
|
|
384
|
+
it with `loadFont` and pass what it returns as `font`;
|
|
385
|
+
- `pixel.Dialog`: a game dialog box that letters the line being sung word by word (sung words lit, the rest dim),
|
|
386
|
+
wrapped to its `width` and set once per line, so no word moves; a line longer than `rows` turns the page. Put the
|
|
387
|
+
lines it shows in `linesDrawnByScene`. Its 5×7 letters are 7 blocks tall: `px` 6 keeps them over 36 px in 16:9
|
|
388
|
+
and in a `VerticalScene`, but the short shows a `Scene` in its band at 0.76, so there use `px` 7 or more (8, the
|
|
389
|
+
default, to be safe). The 5×7 font has all of ASCII and drops symbols it lacks (music notes, emoji); when a line
|
|
390
|
+
has letters it lacks, such as Korean, every line comes from the font in rows of 19 blocks, Hangul 14 tall: draw
|
|
391
|
+
the box at `px` 4 in 16:9 (Hangul stays over 36 px in the band too), at `px` 6 in a 9:16 frame;
|
|
392
|
+
- `pixel.Pixelate id`: anything inside turns into blocks on the same grid, `outline` rings them one block wide: any
|
|
393
|
+
kit drawing, or type in a font of your own. Pop or slide what is inside, not a group around it, so the blocks stay
|
|
394
|
+
put. One colour per Pixelate gives clean blocks. `shadow` adds a drop shadow one block down in that colour, inside
|
|
395
|
+
the outline (a dark one keeps diagonal edges clean). Not a second Pixelate under an outlined one: the ring leaves
|
|
396
|
+
each block's diagonal corner open, so a shadow offset sideways shows through there in specks. Korean type smears
|
|
397
|
+
under about 18 blocks to the em here. Pass the size it is drawn at on screen as `stageScale`: the
|
|
398
|
+
camera's zoom under `pixel.cameraAt` (`stageScale={camera.zoom}`), times 0.76 in the 9:16 band; otherwise blocks
|
|
399
|
+
flicker, vanish or come out a pixel short;
|
|
400
|
+
- `pixel.colors`, and `pixel.step(T)` for the held 8 steps a second of sprite animation.
|
|
401
|
+
|
|
402
|
+
`comic`, for a comic page: `comic.Page` holds `comic.Panel`s, each filming the world with its own `shots`; `comic.Ink`
|
|
403
|
+
prints colour off register under ink lines (one Ink per layer, back to front), with `comic.Dots` for Ben-Day dots;
|
|
404
|
+
`comic.Balloon` and `comic.Caption` letter a `Line` as it is sung, unsung words ghosted; `comic.Sfx` slams once.
|
|
405
|
+
Draw characters and small props in `comic.Ink contour`: one heavy outline round all of it, colour in register, and
|
|
406
|
+
inside only the lines you place: `comic.Brush` (tapered: features, folds, an arm's edge over the body), `comic.Edge`
|
|
407
|
+
(eyes, a mouth) and `fill="none"` paths. `comic.brushPath` makes limbs and tails to fill. A form in front of another
|
|
408
|
+
(a forearm over the body) goes in a later contour Ink; check the joints at 2×. The contour fills narrow gaps and sharp
|
|
409
|
+
notches: give a small form (an ear) a lighter `line`, draw a hole (a mug handle's) as a filled shape in an `Edge`,
|
|
410
|
+
round the dips between tufts of fur, and keep balloons, tails and trailing bubbles clear of the figure.
|
|
411
|
+
|
|
412
|
+
`poster`, office and street props drawn flat as on a poster, in `poster.colors`: `poster.Phone` (its `children` on
|
|
413
|
+
the screen, such as `poster.Bubble`s with a `tail`), `poster.Notebook` (`write` 0–1), `poster.CoffeeCup`,
|
|
414
|
+
`poster.Glass` (`fill`), `poster.Lighter`, `poster.Smoke` (steam, or a `PortraitBody`'s cigarette from
|
|
415
|
+
`portraitPropTip()` given the body's props), `poster.Building` (`sign`), `poster.Light` (`lit`), `poster.Cloud`, and
|
|
416
|
+
`poster.SleevedHand` going into a `poster.Pocket`. Check a held prop at both ends of its move and in between.
|
|
417
|
+
|
|
418
|
+
`toon`, an ink outline and shading for any drawing (gallery tile 29: kit pieces as drawn, then in the pack):
|
|
419
|
+
- `toon.Defs id` once in the scene's `<Svg>`, then `<toon.Ink id>` round each character or prop: one outline round
|
|
420
|
+
its silhouette, kit pieces included (a `Person`, a `Car`); their soft ground shadows and glows stay without a line.
|
|
421
|
+
One Ink per character or prop: an Ink round a whole group outlines only the group's outer edge;
|
|
422
|
+
- `toon.Shade id color` (`round` for a ball lit from the top left) with `fill={toon.fill(id)}` shades a form from its
|
|
423
|
+
lit side, and `toon.Shine` puts a highlight there;
|
|
424
|
+
- `toon.Line` draws a limb, cord or handle with an outline of its own, outside an Ink.
|
|
425
|
+
|
|
426
|
+
In one trial the user's first note on a flat first take was that its pictures were lacking; the second take drew
|
|
427
|
+
every character and prop this way, and the user liked the outlines. Outlines are fine high-contrast detail, which
|
|
428
|
+
measures as motion under drift: that video's boards ran at `drift` 0.5–0.6.
|
|
429
|
+
|
|
430
|
+
`line`, line art traced from any drawing (gallery tile 30: kit pieces in its four looks):
|
|
431
|
+
- `<line.Lines id look>` round a character, a prop or a whole world traces its outline and the edges inside it where
|
|
432
|
+
the brightness changes, and draws only those lines. `look` is `ink`, `chalk`, `blueprint` or `neon`; `color` gives
|
|
433
|
+
another line colour (one neon tube per colour). Soft shadows and glows leave no line, detail too dense for the
|
|
434
|
+
lines (a wheel's spokes at half size) is drawn round instead of filled in, and `soften` drops small detail. A world
|
|
435
|
+
wider than the frame takes `region`, as `paint.Paint` does, or the trace covers all of it;
|
|
436
|
+
- `<line.Paper id look>` lays the look's ground: grained paper, a chalkboard, blueprint paper with its grid, a brick
|
|
437
|
+
wall in the dark;
|
|
438
|
+
- `line.looks` has each look's paper and line colours, for type and anything drawn beside the traced pieces.
|
|
439
|
+
|
|
440
|
+
Shading steps become lines too, so pieces drawn flat trace cleanest, and two colours of one brightness meet without a
|
|
441
|
+
line inside a piece (its outline always draws). Traced worlds are fine high-contrast detail, so calm their drift
|
|
442
|
+
(step 5). It traces at any size, a concept board's tile or a draft under a camera that pulls back included. On a
|
|
443
|
+
12-core Mac's GPU a still of the street traced over the whole frame takes 2.2 s against 2.0 s untraced, 16 half-size
|
|
444
|
+
pieces 2.3 s; in software 10.5 s and 3.6 s. Concept boards in six new-user runs proposed these looks five times (an
|
|
445
|
+
ink diagram or blueprint three times, chalk and neon once each) and dropped them for the drawing they needed; the
|
|
446
|
+
sixth run's blueprint tile came out blank, before the pack traced at that size. One trial video drew its navy ink by
|
|
447
|
+
hand.
|
|
448
|
+
|
|
449
|
+
`paint`, watercolour for any drawing (gallery tile 31: a street world painted on paper):
|
|
450
|
+
- `<paint.Paper id>` lays cold-press paper, and `<paint.Paint id>` paints what it holds on it, kit pieces included:
|
|
451
|
+
edges that wander (`wobble`), thinned and uneven washes (`seed` moves the blooms), pigment pooling darker at the
|
|
452
|
+
edges of each wash, and a pencil underdrawing traced from the same drawing (`pencil={false}` leaves it out);
|
|
453
|
+
- paint a world in one `Paint`, so that what is in front covers what is behind: separate Paints lay their washes over
|
|
454
|
+
each other like glazes, and buildings showed through the people. A world wider than the frame takes `region`, such
|
|
455
|
+
as the stage `[-60, -60, 2040, 1200]`, or the filter covers all of it;
|
|
456
|
+
- on a 12-core Mac's GPU a still of the painted street takes 2.3 s against 2.0 s unpainted; in software 8.0 s.
|
|
457
|
+
|
|
458
|
+
`finish`, a whole frame as an old film, a VHS tape or a halftone print, over any style (gallery tile 32). Wrap the
|
|
459
|
+
frame, camera and all, in `<finish.Film id T>` (`tone` `sepia` or `bw`; grain, scratches, dust, flicker, gate weave,
|
|
460
|
+
a vignette), `<finish.Vhs id T>` (colour fringing, scanlines, a rolling band, a `label` and a counter) or
|
|
461
|
+
`<finish.Print id>` (cyan, magenta, yellow and black halftone dots, off register). Each clips what it holds to its
|
|
462
|
+
`w` × `h`, so a finish on a screen inside a scene stays on the screen. `finish.step(T)` holds the picture for a
|
|
463
|
+
twelfth of a second, for film-era motion: pass it to the scene as its T. Keep the song's captions outside. On a
|
|
464
|
+
Mac's GPU a still of the street takes 2.1–2.2 s in each against 2.0 s; in software 3.2 s as film, 2.5 s as VHS and
|
|
465
|
+
5.4 s as print.
|
|
466
|
+
|
|
467
|
+
Concept boards in five new-user runs proposed a watercolour dreamland and a monster-movie poster once each, and one
|
|
468
|
+
run drew a 1930s rubber-hose cartoon by hand (about 1,250 lines): `toon` under `finish.Film` comes close to that look.
|
|
469
|
+
|
|
470
|
+
**A photo as the head.** Import the image from the song's `src/` (PNG or JPEG; `song2mv render` ignores `public/`)
|
|
471
|
+
and draw it with Remotion's `<Img>` in a `<foreignObject>`: a missing file then stops the render, where an SVG
|
|
472
|
+
`<image>` leaves Chrome's broken-image icon in the frame. `PortraitCutout` clips it to the head and neck and puts its
|
|
473
|
+
neckline (`anchor`) on the collar at (0, −235). Give the profile's ear and temple in body units,
|
|
474
|
+
`neck + s × (point in the photo − anchor)`, and pick `s` so that the ear is 40–100 above the neck, where a call
|
|
475
|
+
reaches it. A call, writing or a hand at the head goes through `PortraitInteraction`, not a scene's own joint
|
|
476
|
+
coordinates: `PortraitBody`'s `phone` gesture holds the phone in one place, by the gallery head's jaw, and misses a
|
|
477
|
+
photo's ear. On a `PortraitBody`, `foregroundCollar` covers the cut, and `hold` grips at (±72, −116), the lower
|
|
478
|
+
corners of what it holds. `renderHead(angle, pitch)` can use `portraitHeadTransform({neck, angle, pitch})` to nod a
|
|
479
|
+
flat head about its neckline; a photo keeps its expression, so the acting is the head's turn, the lean and the hands.
|
|
480
|
+
`docs/portrait-acting.md` has an example, the checks and how to add an action.
|
|
481
|
+
|
|
482
|
+
Putting a scene together:
|
|
483
|
+
|
|
484
|
+
- **Order:** draw back to front inside `<Svg>`: sky, far places, near places, people, then effects in front. Give
|
|
485
|
+
every instance an `id` of its own when two of a kind share the stage (gradients are looked up by id).
|
|
486
|
+
- **Camera:** draw the world once, wrap it in `<Camera>` (inside an `<Svg>`: `<g transform={cameraAt(…).svg}>`) and
|
|
487
|
+
cut its `shots` on bars and words; put `Hit`s and the camera's `hits` on the same times. Keep text outside the
|
|
488
|
+
camera, so that a punch or a roll never moves it.
|
|
489
|
+
- **Any style:** `Hit`, `Flames`, `Lightning`/`Storm`, `Tears` and `Cash` take their colours and shapes as props. For
|
|
490
|
+
a flat or paper look pass the style's colours with `glow={0}` or `flat`; inside a `paper.Stack`, wrapping one in
|
|
491
|
+
`<paper.Cut d="" fill="none" lift={6}>` gives it the paper's shadow (gallery tiles 15–18 and 20). Inside
|
|
492
|
+
`<pixel.Pixelate outline={…}>`, any kit drawing turns into outlined pixels, with in-between colours where it is
|
|
493
|
+
shaded or soft; draw a sprite for anything seen up close.
|
|
494
|
+
- **Recipes:** parts fit together by the same scale `s` (1 = a Person 300 px tall), the line things stand on and
|
|
495
|
+
anchors (`personHand()`, `personEye()`, a strike's `to`, `FlamesAlong`), so a new asset can be a recipe of a few
|
|
496
|
+
lines that combines them, like `SleeplessStorm` in `src/kit/recipes.tsx` (scene 21); recipes are welcome as pull
|
|
497
|
+
requests too.
|
|
498
|
+
- **One light:** choose the scene's key light and pass it everywhere it is asked for: the same `time` for sky,
|
|
499
|
+
clouds, city and vehicles; `light` on `Clouds` and `Sea`.
|
|
500
|
+
- **Motion from data:**
|
|
501
|
+
- a walk covers `STRIDE × s` per `phase` cycle, so move x by that and the feet don't slide;
|
|
502
|
+
- a car's `distance` is the distance moved divided by s;
|
|
503
|
+
- put bursts, cuts and pans on `grid.bar(k)` and word times.
|
|
504
|
+
- **Rain:** fade it with `strength`, not with an opacity group around it.
|
|
505
|
+
- **Drawing next to the kit:** when the concept uses the kit's flat style, the song's own character and objects
|
|
506
|
+
must match it:
|
|
507
|
+
- flat shapes, no black outlines (an edge is a darker tone of the fill);
|
|
508
|
+
- one tone step of shade, a lighter rim on the lit side, glow around lights;
|
|
509
|
+
- far things lighter and hazier.
|
|
510
|
+
|
|
511
|
+
Rules that held up:
|
|
512
|
+
|
|
513
|
+
1. **Pure function of song time `T`.** Never the frame number or state; any `--from` must render identically.
|
|
514
|
+
2. **Text pops once, then holds still** while it is read. No wobble, bob or beat-shake on letters. The beat moves
|
|
515
|
+
lights, particles, shapes, the camera (`Camera`, `Drift`) — not text.
|
|
516
|
+
3. **Every line a scene draws itself** goes into `linesDrawnByScene`, so the bottom captions don't double it.
|
|
517
|
+
4. **Lay out on the 1920×1080 stage.** In 9:16 the stage is a band; read `VerticalCtx` to stack side-by-side
|
|
518
|
+
layouts, or export a `VerticalScene` that draws the whole 9:16 frame (step 6). Keep text ≥ 36 px on screen:
|
|
519
|
+
`check` reads only the size literals in the source, not what a camera or a `scale()` makes of them.
|
|
520
|
+
5. Pull timing from data: `useSong().lineById["ch1-01"].words[0].t`, `grid.bar(k)`, `grid.pulse(T)`.
|
|
521
|
+
|
|
522
|
+
Typecheck (`npx tsc --noEmit` in `<work>/video`), then look at stills of the new section before rendering video. A
|
|
523
|
+
contact sheet draws the scene at many song times in one render, each frame labelled with its time: write its props
|
|
524
|
+
into `<work>/video/sheet.json`, such as `{"times": [33.1, 34.6, 36.2, 37.7]}` (`"cols"`: frames per row;
|
|
525
|
+
`"vertical": true`: the 9:16 frame), and run this in `<work>/video`. It draws the bare scene, for scene work; once
|
|
526
|
+
there is a video, `song2mv sheet` shows it as rendered (step 5):
|
|
527
|
+
|
|
528
|
+
```bash
|
|
529
|
+
node render.mjs ../mv.json --composition Contact --props sheet.json
|
|
530
|
+
```
|
|
531
|
+
|
|
532
|
+
A sheet draws every tile in one page, where an SVG id names one definition for all of them: a gradient, clip or filter
|
|
533
|
+
that changes with T needs T in its id (`id={"glow-" + Math.round(T * 100)}`), or every tile shows the first tile's.
|
|
534
|
+
|
|
535
|
+
**Before the first full render, look closer.** A whole frame on a sheet hides rough lines. `"crop": [x, y, w, h]`
|
|
536
|
+
shows only that region of the frame (its pixels: 1920×1080, or 1080×1920 with `"vertical"`), scaled to the tile
|
|
537
|
+
width: `"cols"` sets the zoom and each label shows it. Make one crop sheet per character and per big-type moment, at
|
|
538
|
+
1× or more, and fix rough lines, seams, gaps between parts and clipped text. Judge soft glows at 1×: above it the
|
|
539
|
+
browser cuts a wide glow (the kit's 90 px text glow) with hard edges the video does not have.
|
|
540
|
+
|
|
541
|
+
```bash
|
|
542
|
+
node render.mjs ../mv.json --composition Contact --props hero.json --name hero # hero.json: {"times": [33.1, 41.2, 52], "cols": 3, "crop": [700, 380, 312, 400]}, 2×
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
## 5. Render and check
|
|
546
|
+
|
|
547
|
+
Iterate on drafts:
|
|
548
|
+
|
|
549
|
+
```bash
|
|
550
|
+
song2mv render <work>/mv.json --template <work>/video --from A --to B --draft
|
|
551
|
+
song2mv check <work>/renders/<name>-16x9-A-B-draft.mp4 --song <work>/mv.json --from A --to B --src "<work>/video/src/scenes/**/*.tsx" --json > <work>/check.json
|
|
552
|
+
song2mv sheet <work>/renders/<name>-16x9-A-B-draft.mp4 --cuts <work>/check.json
|
|
553
|
+
```
|
|
554
|
+
|
|
555
|
+
A draft is the video at 10 fps and half size (960×540), enough for `check`, which measures 10 frames a second at
|
|
556
|
+
320×180: 30 s of a paper-cut scene renders in 15 s instead of 55–70 s at full size (and about 2.5 min as the finished
|
|
557
|
+
video), and `check` gives it the same passes, fails and cuts as the full render. Per-line motion is up to 0.3 higher
|
|
558
|
+
at full size, never lower (a line that opens on a cut can differ by about 1), so a draft's `warnings` name the lines
|
|
559
|
+
within 0.3 under `motion.max`: bring them lower, or the full render may fail them (a line at 8.94 in a grainy draft
|
|
560
|
+
measured 9.20 at full size). When the stretch passes, render it once without `--draft` and check that too; the user
|
|
561
|
+
only ever sees full renders, since a draft looks choppy and soft.
|
|
562
|
+
Before that full render, look at a few frames at full size (`--still T` renders one): a contact sheet's small tiles
|
|
563
|
+
hid a scene drawn twice, a band across the picture that cost one trial its final render.
|
|
564
|
+
Run one render, still or sync at a time: each uses every core, and two renders in one template copy have hung.
|
|
565
|
+
Take the video's path from `outputs` in `render --json` instead of building it (the file name starts with the
|
|
566
|
+
work folder's name, or `--name`). `--src` should cover the video's scene files only: keep review helpers such as
|
|
567
|
+
contact sheets outside `src/scenes/`. Fix every FAIL, re-measure, and say which numbers moved. Checks under `skipped`
|
|
568
|
+
did not run on this stretch; they did not pass. If a FAIL is the ruler's fault (a deliberately still, quiet
|
|
569
|
+
intro line), copy the goals file, add the line id to `motion.quiet_allow`, pass `--goals` and tell the user why —
|
|
570
|
+
don't quietly bend the goals.
|
|
571
|
+
|
|
572
|
+
Each failing line says where: the stretch measured, the cuts in it, and the part under the goal (a still or empty
|
|
573
|
+
line) or the biggest change and the motion without the cuts (a frantic line). Change that moment, not the whole scene.
|
|
574
|
+
In a pixel scene a whole-pixel camera step (`pixel.cameraAt`) can count as a cut, in the list and in that motion.
|
|
575
|
+
|
|
576
|
+
**Look at what you rendered.** `song2mv sheet` tiles the video's own frames into a PNG, captions, title and strips
|
|
577
|
+
included, each labelled with its song time and the line sung then (or the section between lines). With `--cuts` it
|
|
578
|
+
takes a frame 0.3 s into each shot from the saved check result, whose `--from` and `--song` it reuses; `--every S`
|
|
579
|
+
takes one from the middle of every S seconds, `--times` the song times you list, `--lines` one in the last word of
|
|
580
|
+
each line id you list, and `--crop X,Y,W,H` zooms into one region as `Contact`'s `crop` does. A long sheet goes on
|
|
581
|
+
more pages, all listed in `outputs` (`--json`): look at every page. No number catches two lyric lines on screen at
|
|
582
|
+
once, a caption over big type or a cut-off title; a sheet does. Make one of every full render too (`--every 3 --song
|
|
583
|
+
<work>/mv.json`), and one of the lyric map's lines (`--lines` with their ids): each frame must show its row's
|
|
584
|
+
picture, or the line's joke is not in the video. The frame is in the line's last word, where an event on a late key
|
|
585
|
+
word has landed: in one run the lines' middle frames showed the pictures of 26 of 32 rows, frames on the key words all
|
|
586
|
+
32.
|
|
587
|
+
|
|
588
|
+
- **Held shots** measure still; a slow drift (the shot's `to`) keeps one alive. On one pixel street a 4–6 pixel drift
|
|
589
|
+
over a 2–3 s close-up measured 2–5; a 14-pixel drift measured up to 16 (frantic). A small thing moving across a
|
|
590
|
+
wide shot measures still too (a car crossing a street measured 0.14): frame it closer or pan with it. Fine
|
|
591
|
+
high-contrast detail (window grids, railings, stripes, stair treads) measures as motion under any drift: over one
|
|
592
|
+
brownstone street the planner's drift alone measured 8–9 on most verse lines, and `check` says "the whole frame
|
|
593
|
+
changes" when such a line is frantic without a cut. Draw that detail coarser or lower in contrast, or calm the
|
|
594
|
+
drift: `drift: 0.5` on the street's boards (`planShots(…, { drift })`) took its verse lines from 5.4 to 3.6 on
|
|
595
|
+
average, none of them still.
|
|
596
|
+
- **Shot variety.** `check` does not see a shot coming back: one video cut back to the same wide street shot about
|
|
597
|
+
30 times. The `--cuts` sheet has a frame of every shot. A shot that continues across neighbouring tiles is one shot;
|
|
598
|
+
reframe a framing that returns after other shots.
|
|
599
|
+
- **Dense lines** (a line every 1.5 s or less, as in rap): don't cut on every line of a mid section. A short line
|
|
600
|
+
that holds two cuts reads frantic, and even spacing fails `cuts.rhythm`; cut every 2–3 lines, unevenly, and keep
|
|
601
|
+
the fast cutting for hot sections (`planShots` does all this). Big type that changes on every line counts as a cut,
|
|
602
|
+
and so does a large transform such as a flip; a reframe of a flat or dark set may not, so change the set or its
|
|
603
|
+
palette between shots. `cuts.max_gap` names the sudden changes in each long stretch that did not count as a cut,
|
|
604
|
+
with their correlation (a cut is under 0.62) and difference (over 2.5): a high correlation is a reframe too alike to count, a
|
|
605
|
+
low difference one too dim. A thermal-camera board's two framings left one run 8.5 s without a cut; on a made-up
|
|
606
|
+
frame like it, a 10 px reframe measured correlation 0.85. Cut a 16th before a line's first word or on it, not just after it: `check` counts that cut
|
|
607
|
+
as the line's own, not as motion in the line before.
|
|
608
|
+
- **Fast change inside one shot** measures frantic without any cut. In one video a rewind gag that flipped between
|
|
609
|
+
old frames measured 29.5 (goal ≤ 9) and one crossfade under a slow scan bar 4.7; a windmill spinning in a close
|
|
610
|
+
shot 14.2, and filmed wide at 70° a second 5.5; a wobbling stack of burgers with new ones popping in 14.5, and the
|
|
611
|
+
stack still with one burger held up per line (`drift: 0.5`) 6.8. Keep the gag and slow its motion: the line still
|
|
612
|
+
needs its event.
|
|
613
|
+
|
|
614
|
+
**Passing is necessary, not proof of fun.** After the numbers pass, send the user a clip of the full render and ask
|
|
615
|
+
what is boring. Their answer outranks the check.
|
|
616
|
+
|
|
617
|
+
## 6. Shorts (9:16)
|
|
618
|
+
|
|
619
|
+
Pick 10–35 s around the hook (usually the first chorus). The first two seconds decide whether people stay: start
|
|
620
|
+
on the hit and use `--hold` so the first frame shows the payoff fully drawn; check it with `--still` at from + hold.
|
|
621
|
+
|
|
622
|
+
```bash
|
|
623
|
+
song2mv render <work>/mv.json --template <work>/video --from <hit> --to <+33> --aspect 9x16 --hold <s>
|
|
624
|
+
```
|
|
625
|
+
|
|
626
|
+
While you fix the short, render and check it as a draft (`--draft`), as in step 5.
|
|
627
|
+
|
|
628
|
+
By default the short shows `Scene` shrunk into a band, with the template's sky above and below it and the title on
|
|
629
|
+
top. That frame puts the template's look around every concept, and a scene that draws its own lyrics leaves the
|
|
630
|
+
bottom of the screen empty. To fill the phone screen instead, export a `VerticalScene` from `src/scenes/index.ts`:
|
|
631
|
+
a component of `T` that draws the whole 1080×1920 frame. The template then adds only the captions of lines the
|
|
632
|
+
scenes don't draw, the flashes and the fade-out; the background and any title are the scene's. It is usually the
|
|
633
|
+
same world and timing filmed for a tall frame, which `verticalView` in the kit does:
|
|
634
|
+
|
|
635
|
+
```tsx
|
|
636
|
+
const cam = verticalView(cameraAt(T, shots, { hits })); // a pixel scene: pixel.cameraAt(T, shots, { px, vertical: true })
|
|
637
|
+
<Svg w={1080} h={1920}><g transform={cam.svg}>…the world, as drawn for 16:9…</g></Svg>
|
|
638
|
+
```
|
|
639
|
+
|
|
640
|
+
Where things go (`VERTICAL` in the kit has the numbers):
|
|
641
|
+
|
|
642
|
+
- **Text** inside the safe box, x 120–960, y 269–1248, clear of the app buttons and caption overlay on Reels, Shorts
|
|
643
|
+
and TikTok. The template's captions take its bottom, y 1082–1240; a scene that draws its own lyrics puts them there
|
|
644
|
+
too, and a title or big type at the top of the box.
|
|
645
|
+
- **The subject** in the picture area above the captions, y 269–1080. `verticalView` puts the view's point in its
|
|
646
|
+
middle (540, 675), as large as in the 16:9 frame; pass a larger zoom to come closer. A wide shot's point is
|
|
647
|
+
usually the middle of the stage, not its subject: give the tall frame's wide shots the subject's point
|
|
648
|
+
(`shots.map(s => (s.zoom ?? 1) > 1.5 ? s : { ...s, x: 900, y: 820 })`), or the subject drops under the captions.
|
|
649
|
+
Point a board's `tall` framings at the subject's middle too (step 4), with the starter's `tall()`, not `at()`, which
|
|
650
|
+
keeps a framing on the 16:9 stage and so moves a tall frame's subject off its middle.
|
|
651
|
+
A close-up shows more below its point than in 16:9, and that part sits under the captions: if it is the subject's
|
|
652
|
+
body, zoom out until the subject fits above y 1080.
|
|
653
|
+
- **Below y 1248** the apps cover the frame: picture only. The view keeps the frame on the stage's sides and top but
|
|
654
|
+
runs on past its bottom (the kit's places end at y 1080), so draw the ground on down there, or a foreground such
|
|
655
|
+
as a crowd. How far it runs on depends on the zoom: the frame ends 1245 / zoom stage px below the view's point, so
|
|
656
|
+
a 2× close-up of a subject at y 760 shows only about 300 px below y 1080. Put the foreground just under y 1080, not
|
|
657
|
+
350–700 px down, where only wide shots reach. The view also stops at the stage's top edge, so a subject near it
|
|
658
|
+
lands above (540, 675); a scene whose background reaches above the stage passes the area it fills as `world`
|
|
659
|
+
(`verticalView(cam, { world: [0, -900, 1920, Infinity] })`, or with boards
|
|
660
|
+
`<Storyboard T={T} boards={BOARDS} vertical world={[0, -900, 1920, Infinity]} />`).
|
|
661
|
+
|
|
662
|
+
A contact sheet draws the captions' top as a dashed line, and with `"vertical": true` the safe box too (a crop sheet
|
|
663
|
+
leaves them out): check them there before rendering the short. `check` measures the top 56 % of a short, above the
|
|
664
|
+
captions, also with a `VerticalScene`; a large still overlay there (a HUD, a dialogue box) hides cuts from it, so cut
|
|
665
|
+
between shots that look different.
|
|
666
|
+
|
|
667
|
+
## 7. Deliver
|
|
668
|
+
|
|
669
|
+
Full render(s), short(s), `song.srt` / `song.lrc`, a short list of what was hand-fixed (manual timing values, goals
|
|
670
|
+
changed and why) and the spots nobody has heard yet, each with its clip. Nothing from the work folder gets committed.
|
|
671
|
+
|
|
672
|
+
For uploading, ask where the user will post it:
|
|
673
|
+
- a platform with a size limit gets `song2mv render <work>/mv.json --template <work>/video --max-mb N`, which encodes the
|
|
674
|
+
finished video again to fit and keeps its sound;
|
|
675
|
+
- an album or playlist page gets a square cover from `--cover T` at a title or hook frame;
|
|
676
|
+
- music platforms that show synced lyrics take `<work>/song.lrc` (`song.words.lrc` has word times).
|
|
677
|
+
|
|
678
|
+
A finished render is already capped at 8 Mbit/s, about 60 MB a minute at most; pass `--max-mb` when the limit is
|
|
679
|
+
lower than that.
|
|
680
|
+
|
|
681
|
+
**Give back what other songs can use.** Look over what you drew for the video. Some pieces are generic, not tied to
|
|
682
|
+
this song: an object, a pose, a place, a style pack. Offer those to the user as a pull request to song2mv, following
|
|
683
|
+
`CONTRIBUTING.md` ("Adding an asset or a style pack"): original drawings, no media files, the kit's rules, a gallery
|
|
684
|
+
tile and a still. Open it only if the user agrees, and leave the song, its lyrics and its own scenes out. Give back
|
|
685
|
+
what the video taught as well: what the user found fun or dull and what fixed it, with the numbers that moved, as a
|
|
686
|
+
change to this skill, so the next agent starts where you finished. Describe the song in general terms and make up
|
|
687
|
+
the examples.
|