freecodego 0.1.7-alpha.2.2 → 0.1.7-rc.2.2

package/dist/assets/design/skills/embedded-captions/references/bespoke-vs-presets.md

@@ -0,0 +1,178 @@
+# Bespoke design vs. presets — when to override, when to clone
+
+The 5 preset styles (`intro / phrase / emph / dream / crown`) and the 3 templates (`wall-embed`, `corner-column-crown`, `portrait-header`) are **scaffolds**, not rules. The best renders we've shipped all override presets for specific groups because **typography is a per-scene decision**, not a general rule.
+
+If you only use presets, your render will look generic. If you only copy existing renders, your skill won't adapt. The right workflow is:
+
+1. **Decide the shape first** (template choice, plane position, blend mode).
+2. **Check if a canonical example is close enough** → clone and tweak words + timings.
+3. **Otherwise start from presets** → override per-group via `custom_css`.
+
+---
+
+## Canonical example renders
+
+Full HTML for two validated renders is in `references/example-renders/`:
+
+| File               | Scene                                                   | What makes it work                                                                                                                                                                                         |
+| ------------------ | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `memory-wall.html` | Introspective monologue, right-side foam wall, mid-tone | Right-aligned cascade, per-group bespoke sizes (`cap-1` 78 italic / `cap-2` 66 italic + right hanging-indent / `cap-3` 72 upright / `cap-4` 90 uppercase). `mix-blend-mode: screen` for the dark-ish foam. |
+| `champion.html`    | Podcast interview, cluttered bookshelf, 1920×1080       | Upper-left column + center-stage crown. Tuned preset class sizes (`cap-intro` 52 / `cap-phrase` 60 / `cap-emph` 70 / `cap-crown` 140). `screen` blend reads the shelves through text.                      |
+
+**When a new scene matches one of these closely** (similar framing, similar subject-center, similar backdrop type): clone the HTML and only replace the GROUPS array + word timings. Don't re-derive the design from presets — you'll lose the specific choices that took many iterations to validate.
+
+---
+
+## When presets are wrong
+
+You'll reach for presets like `"style": "emph"` when what the scene really needs is:
+
+### "This cap is at position N and deserves its own treatment"
+
+`memory-wall.html` uses `cap-1 / cap-2 / cap-3 / cap-4` — **position-indexed**, not role-indexed. Each one is a bespoke design for a specific phrase at a specific point in the arc:
+
+- cap-1 (soft opener, 4 words): 78px italic 600 — feels like a whisper
+- cap-2 (dreamy modifier, 3 words): 66px italic 500 + `padding-right: 44px` — hanging indent creates ragged right-edge stagger
+- cap-3 (turn, 2 words): 72px upright 700 — the syntactic pivot, no italic
+- cap-4 (climax, 4 words): 90px uppercase 900 — three lines cascade right-aligned
+
+`phrase/emph/intro` can't express "this cap has a hanging indent" or "this cap is the syntactic pivot". When that matters, invent your own class names:
+
+```json
+{
+  "template": "wall-embed",
+  "custom_css": "
+    .cap-1 { font-size: 78px; font-weight: 600; font-style: italic;
+             letter-spacing: -0.01em; }
+    .cap-2 { font-size: 66px; font-weight: 500; font-style: italic;
+             padding-right: 44px; }
+    .cap-3 { font-size: 72px; font-weight: 700; letter-spacing: -0.015em; }
+    .cap-4 { font-size: 90px; font-weight: 900; letter-spacing: -0.03em;
+             text-transform: uppercase; line-height: 1.0; }
+  ",
+  "groups": [
+    { "id": "cg-0", "style": "1", "words": [...] },
+    { "id": "cg-1", "style": "2", "words": [...] },
+    ...
+  ]
+}
+```
+
+The `"style": "1"` field becomes `class="cap-1"` on the element — any string works, no validation.
+
+### "The template's blend doesn't suit this backdrop" → pick a different template, do NOT override
+
+**Cinematic mode does not override colour or blend.** A template's `mix-blend-mode` + fill are **locked DNA** — `make-composition.cjs` ignores `plan.cap_color` / `blend_mode` / `text_shadow` / `text_filter`. Selecting a template commits to its look; the only agent-authored things are layout (planes/positions) and per-group typography.
+
+So use the caption-region luminance to **choose** a template that already fits — never to recolour one:
+
+| Region luminance                  | What fits                                                                                     | Why                                                                        |
+| --------------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
+| < 60 (dark / low-key)             | a cream + `screen` template (`cinematic-cream`, `memory-wall`, `champion`, `portrait-header`) | light text glows, picks up the scene                                       |
+| 60–180 (mid-tone)                 | a cream + `screen` template still reads (add a scrim via Standard if marginal)                | text picks up texture                                                      |
+| > 180 (bright: window, pale wall) | **none of the cream/`screen` Cinematic templates — they wash out**                            | → use **Standard mode** (opaque rail, set per the chosen template) instead |
+
+If the scene is bright and the cream/`screen` look washes out, that's the signal to switch to **Standard mode** (which sets opaque colour in the HTML), not to recolour a Cinematic template into something it isn't.
+
+### "Hanging indent / outdent / letter-width tweak"
+
+These are per-group affordances you'll occasionally need. Express via `custom_css`:
+
+```css
+#cg-2 {
+  padding-right: 44px;
+} /* right-aligned: shrinks right edge, creating left outdent */
+#cg-2 {
+  padding-left: 44px;
+} /* left-aligned: offset right start */
+.cap-emph .w:first-child {
+  font-size: 110%;
+} /* oversize first word only */
+```
+
+The `#cg-N` selector always works because `make-composition.cjs` writes `<div id="cg-N" ...>` for every group.
+
+### "Caps should accumulate (flex stack) instead of swap"
+
+All three templates default to `position: absolute` on `.cap` inside their plane — caps stack at one spot and only the active one shows (single-caption swap). This is correct for `portrait-header` and `corner-column-crown`, where each caption replaces the last.
+
+`memory-wall` uses **flex column accumulation** — captions pile up like a poem. The template doesn't default to this, so override via custom_css:
+
+```css
+.wall-plane {
+  display: flex;
+  flex-direction: column;
+  justify-content: center;
+  align-items: flex-end; /* or flex-start for left-aligned */
+  text-align: right;
+  gap: 14px;
+}
+.wall-plane .cap {
+  position: static; /* un-do template's absolute */
+  top: auto;
+  right: auto;
+  max-width: 100%;
+}
+```
+
+With this + staggered `in` / `out` times, cap-0 fades in at t=0.2, cap-1 at t=2.55 (below cap-0 in flex order), cap-2 at t=4.90 (replaces both as they fade out together at t=4.85) — this is how the memory-wall poem pages work.
+
+### "Font size doesn't match the scene"
+
+Template preset sizes are tuned for a specific column width + frame size. Don't fight them — override:
+
+```json
+"custom_css": ".cap-intro { font-size: 52px; } .cap-phrase { font-size: 60px; }"
+```
+
+Then check [typography-presets.md § Font-size scales with column width](typography-presets.md) for what to aim at given your plane's actual dimensions.
+
+---
+
+## The clone-and-tweak workflow
+
+For a new video that's clearly similar to an existing canonical example:
+
+```bash
+# 1. Scaffold the project
+hyperframes init <project> --non-interactive --video <video.mp4> --skill=embedded-captions
+
+# 2. Matte + transcribe
+node scripts/matte.cjs <project>
+node scripts/transcribe.cjs <project>
+
+# 3. Copy the canonical HTML instead of writing plan.json
+cp references/example-renders/memory-wall.html <project>/index.html
+
+# 4. Replace GROUPS array with the new transcript's grouping (hand-edit index.html)
+
+# 5. Render directly (skip make-composition.cjs since we're not using plan.json)
+bash scripts/render-and-composite.sh <project>
+```
+
+This skips the preset-based plan.json entirely. Use when:
+
+- Subject framing, shot composition, and backdrop type are similar to the example
+- You just need to swap the words and timings
+- The bespoke typography from the example is what you want
+
+**Don't clone when**:
+
+- Subject position differs significantly (e.g. centered vs off-center)
+- Scene luminance / blend-mode needs are different
+- You want to experiment with new typography
+
+In those cases, start with plan.json + custom_css and iterate.
+
+---
+
+## Rendering history
+
+`render-and-composite.sh` now snapshots `index.html` + `plan.json` into `<project>/history/` with a timestamp before every render. If the user says "the previous one was better", diff against the latest snapshot:
+
+```bash
+ls <project>/history/
+diff <project>/history/index-20260422-203947.html <project>/index.html
+```
+
+This lets you recover a design you iterated away from, without re-reading the agent transcript.

package/dist/assets/design/skills/embedded-captions/references/caption-grouping.md

@@ -0,0 +1,130 @@
+# Caption Grouping
+
+How to turn the Whisper word-level transcript into the `groups[]` array of plan.json.
+
+## Goal
+
+**Each group is one visual phrase**: enters, reveals word-by-word, exits. Rule of thumb: 1 group ≈ 1 comma-to-comma clause or 1 breath of speech.
+
+## Input
+
+`transcript.json` from `transcribe.cjs`:
+
+```json
+{
+  "words": [
+    { "text": "Some", "start": 0.24, "end": 0.44, "type": "word" },
+    { "text": " ", "start": 0.44, "end": 0.48, "type": "spacing" },
+    { "text": "memories", "start": 0.48, "end": 0.82, "type": "word" }
+  ]
+}
+```
+
+Drop `type: "spacing"` entries; you only need words.
+
+## Break boundaries
+
+Cut a new group at ANY of:
+
+1. **Pause ≥ 500ms** (gap between word.end[i] and word.start[i+1]) — speaker took a breath.
+2. **Sentence terminator** — word ends with `.`, `?`, `!`, or an em-dash-like pause.
+3. **Strong comma** — `,` followed by pause ≥ 250ms.
+4. **Discourse reset** — words like "but", "so", "and then", "you know" starting a clause often merit their own or new group.
+5. **Group reaches 6 words OR 2.5 seconds** — whichever first. Long groups feel like subtitles, not embedded typography.
+
+Hard constraints:
+
+- Minimum 2 words per group (1-word exceptions: interjections like "Wait." or the crown line).
+- Minimum 0.5s on screen. If a group is less, merge into neighbor.
+- No overlapping groups — at most one visible at a time.
+
+## Timing the group
+
+For a group with words `w[0]..w[n-1]`:
+
+- `in` = `w[0].start - 0.08` (enter slightly before first word)
+- `out` = `min(next_group.in - 0.05, w[n-1].end + 0.6)` (linger ~0.6s after last word, but don't collide with next)
+
+The last group extends to the video end if needed.
+
+## Style & tone (cross-reference)
+
+See `typography-presets.md` for how to pick `style` and `tone` per group. Work left-to-right through the groups and:
+
+1. First group: default `intro` + `soft`.
+2. Watch for emphasis signals (ALL CAPS in transcript is rare but possible; more often it's semantic — superlatives, proper nouns).
+3. Escalate tone into `present` once the monologue shifts from setup to statement.
+4. Reserve `crown` for at most ONE group, typically the final line.
+
+## Editorial surgery is allowed
+
+You do NOT have to caption every word. It's fine to:
+
+- **Drop filler** like extra "you know"s, "um"s, "I mean"s if they bloat the visual pace.
+- **Condense** a 6-word run into 4 by cutting function words, as long as the meaning and timing remain truthful.
+- **Skip the whole thing** during obvious silence or non-speech (laugh, music interlude).
+
+Editorial rule: you are writing typography to support the speech, not a court transcript. Keep meaning, trim noise.
+
+## Example (champion)
+
+Transcript: "You know, for me I've had this kind of upbringing, had the great foundation and, you know, I've achieved incredible things. I was dreaming of becoming number one in the world and becoming a Wimbledon champion"
+
+Groups after editorial pass:
+
+```json
+[
+  {
+    "id": "cg-0",
+    "style": "intro",
+    "tone": "soft",
+    "words": ["You", "know", "for", "me"],
+    "in": 0.1,
+    "out": 1.45
+  },
+  {
+    "id": "cg-1",
+    "style": "phrase",
+    "tone": "soft",
+    "words": ["I've", "had", "this", "kind", "of", "upbringing"],
+    "in": 1.4,
+    "out": 3.35
+  },
+  {
+    "id": "cg-2",
+    "style": "phrase",
+    "tone": "soft",
+    "words": ["the", "great", "foundation"],
+    "in": 3.5,
+    "out": 5.35
+  },
+  {
+    "id": "cg-3",
+    "style": "emph",
+    "tone": "present",
+    "words": ["I've", "achieved", "incredible", "things"],
+    "in": 6.05,
+    "out": 8.3
+  },
+  {
+    "id": "cg-4",
+    "style": "dream",
+    "tone": "present",
+    "words": ["dreaming", "of", "becoming", "number", "one"],
+    "in": 8.5,
+    "out": 10.4
+  }
+]
+```
+
+Plus the crown:
+
+```json
+{ "id": "cg-crown", "style": "crown", "words": ["Wimbledon", "Champion"], "in": 10.8, "out": 12.08 }
+```
+
+Notice "had" was dropped from cg-2 ("had the great foundation" → "the great foundation"), "I was" was dropped from cg-4, and "a" was dropped from crown — all for visual cadence.
+
+## word.start/end inside groups
+
+Pass through the original timestamps from the transcript. Don't retime individual words — only the group `in`/`out`. The word-level karaoke reveal inside each group uses the original w.start.