@lalalic/markcut 2.2.3 → 2.2.5

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lalalic/markcut",
-  "version": "2.2.3",
+  "version": "2.2.5",
   "description": "Markdown-to-video engine. Describe scenes in markdown, get a rendered video.",
   "bin": {
     "markcut": "bin/markcut"

package/skills/markcut/SKILL.md

@@ -28,6 +28,14 @@ see [docs/markdown-descriptive.md](docs/markdown-descriptive.md) for full detail
 - Templates in ``docs/templates/`` provide ready-to-use video structures for `orchestrator` agents, use them as starting point to create your own videos.
 
 
+### Viral Story requires
+- **Hooks** : why should the viewer watch this?
+- **Conflict** : what challenges or obstacles do the characters face?
+- **Resolution** : how are the conflicts resolved?
+- **Emotion** : what feelings are evoked in the viewer?
+- **Call to action** : what should the viewer do next?
+- **Open ending** : does the story leave room for interpretation or continuation?
+
 ## 3. CLI
 
 ```bash
@@ -66,4 +74,11 @@ some common issues (photo or video can't be displayed, audio missing), take belo
   - `rehype-katex` — render math formulas with KaTeX
 
 - `@remotion/shapes` — render shapes like arrows, circles, rectangles, etc
-- `@remotion/starburst` — render starburst animations
+- `@remotion/starburst` — render starburst animations
+
+## Golden rule
+- always check stream start and duration to avoid
+  - audio cut off
+  - video cut off
+  - subtitle mismatch
+  - sync issues between audio, video, and subtitles

package/skills/markcut/docs/markdown-descriptive.md

@@ -35,7 +35,11 @@ The line after `# video` contains all video configuration as space-separated
 
 ```markdown
 # video
-width:1920 height:1080 fps:30 layout:series tts:"edge-tts --voice 'en-US-GuyNeural' --text '{input}' --write-media '{output}'" stylesheet:".bg { background: #000; }" subtitle:captions.vtt
+width:1920 height:1080 fps:30 layout:series 
+~~~css stylesheet
+.bg { background: #000; }
+~~~
+subtitle:captions.vtt
 ```
 
 Supported keys: `width`, `height`, `fps`, `layout`, `tts`, `stt`, `tti`, `ttv`,
@@ -145,8 +149,10 @@ For compatibility, `import { Name } from "spec"` also works and produces the sam
 | `height` | canvas height | root |
 | `fps` | frame rate | root |
 | `theme` | *removed — use `style` on root* | root |
-| `tts` | CLI template string (e.g. `edge-tts --voice "en-US-GuyNeural" --text "{input}" --write-media "{output}"`) | root |
-| `stt` | CLI template string (e.g. `whisper "{input}" --output_format vtt --output_dir "{output}"`) | root |
+| `tts` | text-to-speech CLI template string  | root |
+| `stt` | speech-to-text CLI template string  | root |
+| `tti` | text-to-image CLI template string  | root |
+| `ttv` | text-to-video CLI template string  | root |
 | `layout` | `series\|parallel\|transitionSeries` | root, scene | 
 | `transition` | `fade\|slide\|wipe\|flip\|clockWipe` | transitionSeries |
 | `transitionTime` | seconds | transitionSeries |
@@ -176,7 +182,6 @@ For compatibility, `import { Name } from "spec"` also works and produces the sam
 | `script` | narration/dialogue text; TTS source; NOT rendered directly | audio | Only on audio nodes — see Narration section below |
 | `speaker` | speaker name for multi-turn dialogue; set automatically by dialogue expansion | audio | Used for per-speaker voice lookup and subtitle prefix |
 | `voices` | JSON object mapping speaker names to extra TTS CLI flags e.g. `{"Ray":"--voice en-US-GuyNeural"}` | root | Flags are appended to the resolved TTS CLI template |
-| `tts` | CLI template string e.g. `"edge-tts --voice en-US-GuyNeural --text {input} --write-media {output}"` | root | Set once at the root level; all TTS uses this template |
 | `metadata` | arbitrary metadata string | root |
 | `stylesheet` | global CSS string; selectors use `.className` on elements | root |
 | `style` | inline CSS applied to the node's container div e.g. `"border-radius:12px"` | any |
@@ -752,45 +757,49 @@ markcut verify courseware.md
 - `@remotion/shapes` — render shapes like arrows, circles, rectangles, etc
 - `@remotion/starburst` — render starburst animations
 
-## Example
-
-```md
----
-width: 1080
-height: 1920
-fps: 30
----
-# video
-layout:series
-~~~js imports
-export { StatCounter } from "stat-counter"
-export { Logo } from "github:myorg/design-system#Logo"
-
-export function Greeting({ name }) {
-  return <div style={{color: '#fff', fontSize: 28, textAlign: 'center'}}>Hello {name}!</div>
-}
+# best practices
+~~~ example - avoid audio cut
+- parallel
+  - image|video background:true
+  - audio|script 
 ~~~
 
-## Hook
-layout:parallel
-- script "Set location and emotional tone"
-- image src:cover.jpg duration:2
+~~~ example - global streams
+#video
+- audio background:true loop:true src:bgm.mp3
+- component background:true loop:true jsx:"<Logo />" style:"position:fixed;top:10px;left:10px;width:100px;height:100px;"
+- parallel title:"lip sync" style:"position:fixed;bottom:100px;right:100px;width:100px;height:100px;"
+  - video src:background.mp4 loop:true background:true
+  - video src:lip_sync.mp4
+~~~
 
-## Journey
-layout:transitionSeries transition:fade transitionTime:0.4
-- script "Move through moments"
-- video src:clips/arrival.mp4 startFrom:0 endAt:3.5
-- video src:clips/fire.mp4 startFrom:1 endAt:4
+~~~ example - styling VTT captions
+```vtt
+000:00:00.000 --> 00:00:05.000
+It's a <span style="color:#ff6b6b;font-weight:bold">Bear</span>.
 
-## Stat
-layout:parallel
-- component duration:2 jsx:"<StatCounter value={42} label='S-mores' />"
+000:00:05.000 --> 00:00:10.000
+And now it's <span class="emoji-dead">dead</span>.
 
-## Logo
-layout:parallel
-- component duration:1 jsx:"<Logo />"
+000:00:10.000 --> 00:00:15.000
+<span class="bong">Bong!</span>
+```
 
-## Route
-layout:parallel
-- map duration:3 travelMode:DRIVING waypoints:[37.7749,-122.4194,"SF";34.0522,-118.2437,"LA"]
+#root
+```css stylesheet
+.emoji-dead {
+  font-family: "Funny Emoji", sans-serif;
+  font-size: 48px;
+}
+.bong {
+  color: #ff6b6b;
+  font-weight: bold;
+  position: absolute;
+  top: 50%;
+  left: 50%;
+  transform: translate(-50%, -50%);
+  animation: bong 1s ease-in-out infinite;
+}
 ```
+~~~
+

package/skills/markcut/docs/template.md

@@ -59,7 +59,6 @@ What tools/skills the agent needs before starting. Always include:
 
 - The engine itself: `npx @lalalic/markcut` runnable
 - Required markcut skill (`SKILL.md` + `docs/markdown-descriptive.md`) — the agent must read these
-- Required CLIs: `edge-tts`, `ffmpeg`/`ffprobe`, whisper, image-understanding (for reviewer)
 - Any domain-specific tools
 
 ### §1 Inputs — the contract with the user

package/skills/markcut/docs/templates/vlog/TEMPLATE.md

@@ -9,26 +9,6 @@ engine: "@lalalic/markcut — run via `npx @lalalic/markcut`"
 
 Follow this file top to bottom. Read the markcut skill (`SKILL.md` → `docs/markdown-descriptive.md`) first if you have not.
 
-```mermaid
-flowchart LR
-    Profile[👤 user.md<br/>profile] --> Writer
-    History[📜 vlog_history.json<br/>past entries] --> Writer
-    Media[📁 Media folder] --> Vision[🧠 markcut vision --label]
-    Vision --> Labels[📝 metadata.json]
-    Labels --> Group[🔍 group-clips.md<br/>filter + group]
-    Group --> Groups[📋 grouped scenes]
-    Locale[🌍 local-context.md<br/>weather, events, news] --> Groups
-    Groups --> Writer[✍️ story-writer agent<br/>narration + title]
-    Writer --> Storyboard[📄 course.md]
-    BGM[🎵 bgm-select.md<br/>mandatory BGM] --> Assemble[📦 assemble + render]
-    Storyboard --> Assemble
-    Assemble --> MP4[📺 final.mp4]
-    MP4 --> Reviewer[🔍 reviewer agent]
-    Reviewer -->|FAIL| Fix[🔧 fix]
-    Fix --> Assemble
-    Reviewer -->|PASS| HistoryUpdate[📝 append to<br/>vlog_history.json]
-```
-
 | Path | Runs in | Purpose |
 |---|---|---|
 | `TEMPLATE.md` | your context | everything |
@@ -38,10 +18,7 @@ flowchart LR
 ## 0. Prerequisites
 
 - `npx @lalalic/markcut` runnable
-- TTS CLI (default: `edge-tts`)
-- `ffmpeg`/`ffprobe` on PATH
-- For vision pipeline: ITT/VTT/STT CLIs configured (see `docs/markdown-descriptive.md` for env vars)
-- For reviewer: image-understanding capability, STT CLI
+- `ffmpeg`/`ffprobe`/`exiftool` on PATH
 
 ## 1. Inputs — collect before starting
 
@@ -50,13 +27,13 @@ flowchart LR
 | Media folder | **yes** | — | absolute path to folder of photos/videos |
 | Theme / event name | **yes** | — | e.g. "birthday camping", "Tokyo trip", "product unboxing" |
 | Date range | no | from media EXIF | — |
-| Language | no | en | narrator's language |
+| Language | no | zh | narrator's language |
 | Style | no | `daily` | `daily` (casual), `lyrical` (cinematic), `humorous` (funny) |
 | Target duration | no | 60s | 15–120s for short vlogs |
-| Profile | **parse from** | `~/.pi/agent/user.md` | user's personality, region, family members, voice ref, life context. Read this file at the start — it's the authoritative source for who the creator is. The creator's name, region, family, and regular activities feed into narration naturally ("my daughter", "drove up from Ottawa", "my third camping trip this year"). |
+| Profile | **parse from** | context | user's personality, region, family members, voice ref, life context. Read this file at the start — it's the authoritative source for who the creator is. The creator's name, region, family, and regular activities feed into narration naturally ("my daughter", "drove up from Ottawa", "my third camping trip this year"). |
 | Voice | no | en: `en-US-GuyNeural` | edge-tts voice. If profile has a voice reference URL, use TTS voice cloning instead |
 | BGM style | no | `ambient` | mood/genre for background music. BGM is **mandatory** — see §2 for root-level audio node |
-| Vlog history | **read from** | `vlog_history.json` | JSON array of past vlogs in the project. Used for story continuity ("last time I was here...", "my third camping trip this year"). If the file doesn't exist, start fresh. See §3 for history rules. |
+| Vlog history | **read from** | `vlog_history.json` or folder | JSON array of past vlogs in the project. Used for story continuity ("last time I was here...", "my third camping trip this year"). If the file doesn't exist, start fresh. See §3 for history rules. |
 
 **Rule:** The golden rule of vlogs — **every word in narration, title, or captions must be traceable to real data**: (1) user-stated facts, (2) clip VLM descriptions, (3) GPS/timestamp metadata, (4) user labels from interactive labeling, (5) local news/searched events for the date/location. Never invent people, dialogue, events, or place names. When unsure, paraphrase conservatively ("a path", "someone") rather than guessing.
 
@@ -65,6 +42,13 @@ flowchart LR
 ### Overview
 
 ```
+---
+storyoutline: # it's your logic to design story.
+  hook:
+  confliction:
+  emotion:
+  open-ending:
+---
 # video                          ← root: width:1920 height:1080 fps:30 layout:series
 │                                  tts:"<edge-tts CLI template>"
 ├── ## Route                     ← (only if GPS data available)
@@ -100,9 +84,11 @@ width:1920 height:1080 fps:30 layout:series
 - Volume: 0.10–0.20 for ambient BGM; 0.05–0.10 for lyrical music.
 - Source BGM from royalty-free libraries (see `prompts/bgm-select.md`).
 
-### Core design: one scene per story beat, media + narration as parallel
+### Core design: one scene per story beat
+- determine **Hook**, **Confliction**, **Emotion**, **Open-ending**
+- the vlog's story arc: hook → chronological story beats → close.
 
-Each scene is a **story beat** — a single moment or idea that stands on its own. Inside a scene, `layout:parallel` means all media and the subtitle narration play simultaneously.
+- Each scene is a **story beat** — a single moment or idea that stands on its own. Inside a scene, `layout:parallel` means all media and the subtitle narration play simultaneously.
 
 ```
 ## Into the Woods
@@ -113,7 +99,7 @@ layout:parallel
 ```
 
 **Rules:**
-
+- determine story arc first, then group clips into scenes. Each scene is a single story beat.
 - **Scene count**: 5–12 scenes. Each 3–20 seconds. Total duration matches target ±15%.
 - **Hook → Core → Vibe → Close arc**: The first scene hooks the viewer (best visual + intriguing line). Middle scenes are chronological story beats. The final scene provides emotional/humorous closure.
 - **Transition between scenes**: The root `layout:series` plays scenes sequentially. For smooth transitions, add `transition:fade(0.5)` to the root config line.
@@ -123,19 +109,6 @@ layout:parallel
 - **BGM is mandatory**: Always add an `- audio isBackground:true foreground:true` at root level (see BGM section above).
 - **No bullet-reveal** — vlogs are linear media, not slides. No `current`/`on:` events needed.
 
-### Scene media node style
-
-For images:
-```markdown
-- image src:<relative-path> duration:<s> start:<s>
-```
-
-For video clips (trimmed):
-```markdown
-- video src:<relative-path> startFrom:<s> endAt:<s> start:<s>
-```
-
-The `src` path is relative to the media folder. During assembly, media is symlinked/copied into the markcut project so `npx markcut render` can serve it.
 
 ## 2.b Production polish — route sync, overlays, effects
 
@@ -177,14 +150,14 @@ Use per-scene map overlays when GPS data is available for that specific clip. Us
 
 ## 3. Authoring rules — the professional bar
 
-### Filter rules (see `prompts/group-clips.md`)
+### Filter rules (use `prompts/group-clips.md`)
 
-- Keep all clips unless: aesthetic score < 0.4, near-duplicate of a better clip, or off-topic.
+- Keep all clips unless: near-duplicate of a better clip, or off-topic.
 - Always keep clips with a positive user label from the interactive labeling step.
 - Max ~20% drops. Never drop below 5 clips total.
 - List every dropped clip with a reason.
 
-### Grouping rules (see `prompts/group-clips.md`)
+### Grouping rules (use `prompts/group-clips.md`)
 
 - **Visual continuity rule**: clips sharing the same background/outfit/subject-state form one scene. Exhaust one scene before moving to the next. No jumping A→B→A.
 - 1–4 clips per scene (a single long take >10s may be its own scene).
@@ -210,7 +183,7 @@ Before writing narration, search for local news, events, or weather for the vlog
 - **Local event**: "Turns out the Gatineau Park was running the fall colours festival that weekend."
 - **Seasonal context**: "Full moon that night, so the trail was brighter than usual."
 
-Rules:
+**Rules:**
 - Only use facts confirmed by at least one search result. Never invent weather/events.
 - Keep it to 1–2 local-context references per vlog. Don't overwhelm the narration.
 - The context should **amplify** the story, not replace it. "Drove up during the fall colours festival so the park was packed" tells us more than just the festival fact.
@@ -230,7 +203,7 @@ Rules:
 - Keep references brief (one line per vlog max). The story should stand alone.
 - Never fabricate a past event. Only what exists in the history file.
 
-### Creator profile
+### User profile
 
 Read `~/.pi/agent/user.md` at the start and weave the creator's identity into narration naturally:
 
@@ -249,44 +222,6 @@ One title, 3–15 words, poetic/suspenseful/summarizing, social-media friendly.
 - Same pattern as courseware: `# <lang>` variant block at file end with per-language TTS voice.
 - Every `script`/`subtitle` node gets a `<lang>:"..."` twin.
 
-## 4. Components & styles
-
-Vlogs don't need custom JSX components — they use markcut's built-in `image`, `video`, `audio`, `subtitle`, and `map` nodes. However, you may add these optional enhancements:
-
-### Optional: NarratorBox component
-
-If adding a narrator overlay with name label, include this in the `~~~js imports` block:
-
-```jsx
-import { delayRender, continueRender } from 'remotion'
-
-export function NarratorBox({ src, name = '', size = 100 }) {
-  return (
-    <div style={{
-      position: 'absolute', bottom: 40, left: 40,
-      display: 'flex', flexDirection: 'column', alignItems: 'center', gap: 6,
-    }}>
-      <div style={{
-        width: size, height: size, borderRadius: '50%', overflow: 'hidden',
-        border: '3px solid rgba(255,255,255,.6)',
-        boxShadow: '0 0 20px rgba(0,0,0,.4)',
-      }}>
-        <img src={src} style={{ width: '100%', height: '100%', objectFit: 'cover' }} />
-      </div>
-      {name && (
-        <span style={{
-          fontSize: 14, color: '#fff', textShadow: '0 2px 8px rgba(0,0,0,.6)',
-          whiteSpace: 'nowrap', fontWeight: 500,
-        }}>{name}</span>
-      )}
-    </div>
-  )
-}
-```
-
-### Stylesheet
-
-No stylesheet needed for standard vlogs. The built-in markcut rendering handles video/image filling, subtitle positioning, and audio mixing. Add one only if you need custom overlays.
 
 ## 5. Workflow
 
@@ -307,7 +242,7 @@ No stylesheet needed for standard vlogs. The built-in markcut rendering handles
      }
    ]
    ```
-   If the file doesn't exist, start fresh.
+   If the file doesn't exist, try to extract for project folder.
 
 ### Phase 1: Prepare media
 
@@ -326,7 +261,7 @@ No stylesheet needed for standard vlogs. The built-in markcut rendering handles
 5. **Search local news/events** — Fill `prompts/local-context.md` with the vlog's location, date range, and theme. This searches for relevant weather, events, news, or seasonal information. The result feeds into narration with 1–2 natural references.
 
 6. **Group and filter clips** — Fill `prompts/group-clips.md` with clip data from `metadata.json`. This produces:
-   - Dropped clips with reasons
+   - Speed up less-interesting clips vs Dropped clips
    - Visual-continuity scene groups
    - Trim boundaries for video clips
    - GPS waypoints for per-scene map overlays
@@ -348,16 +283,7 @@ No stylesheet needed for standard vlogs. The built-in markcut rendering handles
 
 8. **Select BGM** — Choose background music that matches the vlog's mood. Fill `prompts/bgm-select.md` or search using the `audio-sourcing` skill. Download the track and reference it as `src:bgm.mp3` in the root-level audio node.
 
-9. **Assemble course.md** — Write the markcut markdown file using §2 grammar. The root config line includes TTS voice. BGM is at root level (`- audio isBackground:true foreground:true src:bgm.mp3 volume:0.15`). For scenes with GPS data, add per-scene map overlays (see §2.b).
-
-   **Media staging**: Make media accessible to the renderer:
-   ```
-   mkdir -p .markcut/generated/media/
-   cp <media-folder>/clips_normalized/*.mp4 .markcut/generated/media/
-   cp <media-folder>/*.jpg .markcut/generated/media/
-   cp /path/to/bgm.mp3 .markcut/generated/media/bgm.mp3
-   ```
-   Reference as `src:.markcut/generated/media/<file>` in the markdown.
+9. **Assemble .md** — Write the markcut markdown file using §2 grammar. For scenes with GPS data, add per-scene map overlays (see §2.b).
 
 10. **Render** — `npx @lalalic/markcut render course.md`. On engine errors: fix and re-render, max 3 attempts per error, then ask the user.
 
@@ -394,12 +320,7 @@ Done only when ALL hold:
 - [ ] BGM is **present** (ffprobe confirms audio stream or mixed track) and audible (not silent, not clipping)
 - [ ] BGM ducked under voice — TTS audible above music in ≥90% of spoken segments
 - [ ] local context references (if any) verified against search results — no invented weather/events
-- [ ] history references (if any) match actual entries in `vlog_history.json` — no fabricated past events
+- [ ] history references — no fabricated past events
 - [ ] media `src` paths resolve correctly (no broken links in rendered output)
 - [ ] profile used: narration tone matches creator personality, family names from profile appear correctly
-- [ ] per-scene map overlays (if GPS available) positioned without obstructing main content
-
-## 7. Reference — worked examples
-
-- Golden example: See `tests/fixtures/templates/courseware.md` for the template format.
-- Reference vlog storyboard: The bullx vlog project at `free2/bullx/packages/vlog/storyboard.md` shows a real vlog (birthday camping at Lac Philippe) with 13 chronological scenes, route map, clip trimming, and first-person narration. The project's `.pi/agents/story-writer.md` and `.pi/prompts/vlog-storyboard.md` contain the source of truth for the story-writing methodology documented here.
+- [ ] per-scene map overlays (if GPS available) positioned without obstructing main conten