@lalalic/markcut 2.2.7 → 2.2.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +2 -1
- package/skills/markcut/SKILL.md +0 -2
- package/skills/markcut/docs/markdown-descriptive.md +1 -1
- package/src/config.mjs +60 -63
- package/src/descriptive/resolve.ts +3 -2
- package/src/player/browser.tsx +3 -1
- package/src/player/bundle/player.js +7 -2
- package/src/player/components/SceneThumbnails.tsx +3 -1
- package/src/player/pipeline.mjs +64 -21
- package/src/player/server.mjs +11 -2
- package/src/render/cli.mjs +42 -69
- package/src/vision/cli.mjs +21 -45
- package/skills/markcut/docs/template.md +0 -212
- package/skills/markcut/docs/templates/courseware/TEMPLATE.md +0 -526
- package/skills/markcut/docs/templates/courseware/agents/reviewer.md +0 -84
- package/skills/markcut/docs/templates/courseware/prompts/fix.md +0 -30
- package/skills/markcut/docs/templates/courseware/prompts/outline.md +0 -58
- package/skills/markcut/docs/templates/courseware/prompts/scene.md +0 -70
- package/skills/markcut/docs/templates/deep-dive/TEMPLATE.md +0 -354
- package/skills/markcut/docs/templates/deep-dive/agents/researcher.md +0 -69
- package/skills/markcut/docs/templates/deep-dive/agents/reviewer.md +0 -108
- package/skills/markcut/docs/templates/deep-dive/prompts/fix.md +0 -28
- package/skills/markcut/docs/templates/deep-dive/prompts/outline.md +0 -102
- package/skills/markcut/docs/templates/deep-dive/prompts/script.md +0 -64
- package/skills/markcut/docs/templates/illustrated-book/TEMPLATE.md +0 -356
- package/skills/markcut/docs/templates/illustrated-book/agents/reviewer.md +0 -115
- package/skills/markcut/docs/templates/illustrated-book/prompts/fix.md +0 -29
- package/skills/markcut/docs/templates/illustrated-book/prompts/illustration.md +0 -35
- package/skills/markcut/docs/templates/illustrated-book/prompts/story.md +0 -69
- package/skills/markcut/docs/templates/short-film/TEMPLATE.md +0 -385
- package/skills/markcut/docs/templates/short-film/agents/reviewer.md +0 -115
- package/skills/markcut/docs/templates/short-film/prompts/fix.md +0 -29
- package/skills/markcut/docs/templates/short-film/prompts/screenplay.md +0 -101
- package/skills/markcut/docs/templates/short-film/prompts/storyboard.md +0 -54
- package/skills/markcut/docs/templates/short-video/TEMPLATE.md +0 -301
- package/skills/markcut/docs/templates/short-video/agents/reviewer.md +0 -94
- package/skills/markcut/docs/templates/short-video/prompts/fix.md +0 -27
- package/skills/markcut/docs/templates/short-video/prompts/script.md +0 -65
- package/skills/markcut/docs/templates/vlog/TEMPLATE.md +0 -326
- package/skills/markcut/docs/templates/vlog/agents/reviewer.md +0 -89
- package/skills/markcut/docs/templates/vlog/agents/story-writer.md +0 -100
- package/skills/markcut/docs/templates/vlog/prompts/bgm-select.md +0 -38
- package/skills/markcut/docs/templates/vlog/prompts/group-clips.md +0 -93
- package/skills/markcut/docs/templates/vlog/prompts/local-context.md +0 -59
- package/skills/markcut/docs/templates/vlog/prompts/outline.md +0 -56
- package/skills/markcut/docs/templates/vlog/prompts/review-fix.md +0 -27
- package/skills/markcut/docs/templates/vlog/prompts/storyboard.md +0 -41
- package/src/render/cli.test.ts +0 -13
|
@@ -1,326 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: vlog
|
|
3
|
-
description: Turn raw photos and video clips into a narrated vlog with music, using markcut's vision pipeline for media understanding and story-based scene grouping.
|
|
4
|
-
when-to-use: travel diaries, event recaps, camping trips, day-in-life, any vlog from personal media
|
|
5
|
-
engine: "@lalalic/markcut — run via `npx @lalalic/markcut`"
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Vlog Template
|
|
9
|
-
|
|
10
|
-
Follow this file top to bottom. Read the markcut skill (`SKILL.md` → `docs/markdown-descriptive.md`) first if you have not.
|
|
11
|
-
|
|
12
|
-
| Path | Runs in | Purpose |
|
|
13
|
-
|---|---|---|
|
|
14
|
-
| `TEMPLATE.md` | your context | everything |
|
|
15
|
-
| `prompts/*.md` | your context | fill-in prompts you execute |
|
|
16
|
-
| `agents/*.md` | separate session | subagent definitions |
|
|
17
|
-
|
|
18
|
-
## 0. Prerequisites
|
|
19
|
-
|
|
20
|
-
- `npx @lalalic/markcut` runnable
|
|
21
|
-
- `ffmpeg`/`ffprobe`/`exiftool` on PATH
|
|
22
|
-
|
|
23
|
-
## 1. Inputs — collect before starting
|
|
24
|
-
|
|
25
|
-
| Input | Required | Default | Notes |
|
|
26
|
-
|---|---|---|---|
|
|
27
|
-
| Media folder | **yes** | — | absolute path to folder of photos/videos |
|
|
28
|
-
| Theme / event name | **yes** | — | e.g. "birthday camping", "Tokyo trip", "product unboxing" |
|
|
29
|
-
| Date range | no | from media EXIF | — |
|
|
30
|
-
| Language | no | zh | narrator's language |
|
|
31
|
-
| Style | no | `daily` | `daily` (casual), `lyrical` (cinematic), `humorous` (funny) |
|
|
32
|
-
| Target duration | no | 60s | 15–120s for short vlogs |
|
|
33
|
-
| Profile | **parse from** | context | user's personality, region, family members, voice ref, life context. Read this file at the start — it's the authoritative source for who the creator is. The creator's name, region, family, and regular activities feed into narration naturally ("my daughter", "drove up from Ottawa", "my third camping trip this year"). |
|
|
34
|
-
| Voice | no | | mlx-audio voice. If profile has a voice reference URL, use TTS voice cloning instead |
|
|
35
|
-
| BGM style | no | `ambient` | mood/genre for background music. BGM is **mandatory** — see §2 for root-level audio node |
|
|
36
|
-
| Vlog history | **read from** | `vlog_history.json` or folder | JSON array of past vlogs in the project. Used for story continuity ("last time I was here...", "my third camping trip this year"). If the file doesn't exist, start fresh. See §3 for history rules. |
|
|
37
|
-
|
|
38
|
-
**Rule:** The golden rule of vlogs — **every word in narration, title, or captions must be traceable to real data**: (1) user-stated facts, (2) clip VLM descriptions, (3) GPS/timestamp metadata, (4) user labels from interactive labeling, (5) local news/searched events for the date/location. Never invent people, dialogue, events, or place names. When unsure, paraphrase conservatively ("a path", "someone") rather than guessing.
|
|
39
|
-
|
|
40
|
-
## 2. Scene grammar — mandatory structure
|
|
41
|
-
|
|
42
|
-
### Overview
|
|
43
|
-
|
|
44
|
-
```
|
|
45
|
-
---
|
|
46
|
-
storyoutline: # it's your logic to design story.
|
|
47
|
-
hook:
|
|
48
|
-
confliction:
|
|
49
|
-
emotion:
|
|
50
|
-
open-ending:
|
|
51
|
-
---
|
|
52
|
-
# video ← root: width:1920 height:1080 fps:30 layout:series
|
|
53
|
-
│
|
|
54
|
-
├── ## Route ← (only if GPS data available)
|
|
55
|
-
│ layout:parallel
|
|
56
|
-
│ - map waypoints:[...]
|
|
57
|
-
│ - subtitle script:"Route narration..."
|
|
58
|
-
│
|
|
59
|
-
├── ## <Chronological Scene> ×N ← each is a story beat
|
|
60
|
-
│ layout:parallel
|
|
61
|
-
│ - image/video src:... startFrom?... endAt?...
|
|
62
|
-
│ - image/video src:... duration:...
|
|
63
|
-
│ - subtitle script:"Narration line..." duration:N
|
|
64
|
-
│
|
|
65
|
-
└── ## <Outro Scene> ← closing moment
|
|
66
|
-
layout:parallel
|
|
67
|
-
- video/ image src:... duration:...
|
|
68
|
-
- subtitle script:"Closing line..." duration:N
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
### BGM — mandatory root-level audio
|
|
72
|
-
|
|
73
|
-
Add an `- audio isBackground:true` node at the root level. BGM is **non-optional**:
|
|
74
|
-
|
|
75
|
-
```
|
|
76
|
-
# video
|
|
77
|
-
width:1920 height:1080 fps:30 layout:series
|
|
78
|
-
- audio isBackground:true src:bgm.mp3 volume:0.15 foreground:true
|
|
79
|
-
|
|
80
|
-
## Hook ...
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
- `foreground:true` ducks the music volume when TTS narration plays (auto-ducking).
|
|
84
|
-
- Volume: 0.10–0.20 for ambient BGM; 0.05–0.10 for lyrical music.
|
|
85
|
-
- Source BGM from royalty-free libraries (see `prompts/bgm-select.md`).
|
|
86
|
-
|
|
87
|
-
### Core design: one scene per story beat
|
|
88
|
-
- determine **Hook**, **Confliction**, **Emotion**, **Open-ending**
|
|
89
|
-
- the vlog's story arc: hook → chronological story beats → close.
|
|
90
|
-
|
|
91
|
-
- Each scene is a **story beat** — a single moment or idea that stands on its own. Inside a scene, `layout:parallel` means all media and the subtitle narration play simultaneously.
|
|
92
|
-
|
|
93
|
-
```
|
|
94
|
-
## Into the Woods
|
|
95
|
-
layout:parallel
|
|
96
|
-
- image src:IMG_7100.JPG duration:2 start:0
|
|
97
|
-
- video src:IMG_7060.MOV startFrom:0 endAt:10 start:2
|
|
98
|
-
- subtitle script:"Drove into the park and a deer appeared on the trail. Good omen." duration:12
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
**Rules:**
|
|
102
|
-
- determine story arc first, then group clips into scenes. Each scene is a single story beat.
|
|
103
|
-
- **Scene count**: 5–12 scenes. Each 3–20 seconds. Total duration matches target ±15%.
|
|
104
|
-
- **Hook → Core → Vibe → Close arc**: The first scene hooks the viewer (best visual + intriguing line). Middle scenes are chronological story beats. The final scene provides emotional/humorous closure.
|
|
105
|
-
- **Transition between scenes**: The root `layout:series` plays scenes sequentially. For smooth transitions, add `transition:fade(0.5)` to the root config line.
|
|
106
|
-
- **Media per scene**: 1–4 clips. Group clips by visual continuity (same background/outfit/subject-state). Avoid jumping A→B→A.
|
|
107
|
-
- **Subtitle node**: Each scene needs a `subtitle script:"..."` node. This is the TTS narration. Its `duration` must match the intended scene length. The script text also becomes the VTT subtitle overlay.
|
|
108
|
-
- **Duration driven by subtitle**: Set `duration:` on the `subtitle` node. Media nodes use `start:` and `duration:` to align within the scene. Videos use `startFrom:`/`endAt:` for trimming.
|
|
109
|
-
- **BGM is mandatory**: Always add an `- audio isBackground:true foreground:true` at root level (see BGM section above).
|
|
110
|
-
- **No bullet-reveal** — vlogs are linear media, not slides. No `current`/`on:` events needed.
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
## 2.b Production polish — route sync, overlays, effects
|
|
114
|
-
|
|
115
|
-
### Route map synced with video
|
|
116
|
-
|
|
117
|
-
Beyond the opening route scene, show location context throughout the vlog:
|
|
118
|
-
|
|
119
|
-
**Per-scene map overlay** — Each scene that has GPS data shows a small map in the corner with the current location. The `map` node is placed inside the scene's `layout:parallel`:
|
|
120
|
-
|
|
121
|
-
```
|
|
122
|
-
## At the Lake
|
|
123
|
-
layout:parallel
|
|
124
|
-
- video src:lake.MOV startFrom:0 endAt:10 start:0
|
|
125
|
-
- map waypoints:[45.5997,-76.0053,"Lac Philippe"]
|
|
126
|
-
style:position:absolute;top:20px;right:20px;width:200px;height:150px;border-radius:12px;opacity:0.85
|
|
127
|
-
- subtitle script:"The lake was glass-still under the gray sky." duration:10
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
The map is styled as picture-in-picture: top-right corner, rounded corners, semi-transparent. It shows a single waypoint marker for the scene's GPS location with a label.
|
|
131
|
-
|
|
132
|
-
**Map transition scene** — Between major location changes, insert a 3–5s map scene showing the driving route between two GPS points:
|
|
133
|
-
|
|
134
|
-
```
|
|
135
|
-
## Route-to-Camp
|
|
136
|
-
layout:parallel
|
|
137
|
-
- map duration:4 travelMode:DRIVING
|
|
138
|
-
waypoints:[45.5893,-75.9814,"Trailhead";45.6368,-76.0136,"Campground"]
|
|
139
|
-
- subtitle script:"Drove up the gravel road toward the campground." duration:4
|
|
140
|
-
```
|
|
141
|
-
|
|
142
|
-
Use per-scene map overlays when GPS data is available for that specific clip. Use map transition scenes when the movement between locations is narratively significant (e.g., driving into the park, hiking between trails).
|
|
143
|
-
|
|
144
|
-
### Overlays
|
|
145
|
-
|
|
146
|
-
- **Persistent logo**: `- image isBackground:true src:logo.png style:position:absolute;bottom:24px;right:24px;width:60px;opacity:0.7` at root level.
|
|
147
|
-
- **Narrator overlay**: `- image/video isBackground:true src:narrator.png` with circular crop + border.
|
|
148
|
-
- **Entrance effects**: `effect:zoomIn` on key images for reveal moments.
|
|
149
|
-
- **See courseware template §2.b** for detailed overlay patterns (transition-audio timing, TTS expression control, effects).
|
|
150
|
-
|
|
151
|
-
## 3. Authoring rules — the professional bar
|
|
152
|
-
|
|
153
|
-
### Filter rules (use `prompts/group-clips.md`)
|
|
154
|
-
|
|
155
|
-
- Keep all clips unless: near-duplicate of a better clip, or off-topic.
|
|
156
|
-
- Always keep clips with a positive user label from the interactive labeling step.
|
|
157
|
-
- Max ~20% drops. Never drop below 5 clips total.
|
|
158
|
-
- List every dropped clip with a reason.
|
|
159
|
-
|
|
160
|
-
### Grouping rules (use `prompts/group-clips.md`)
|
|
161
|
-
|
|
162
|
-
- **Visual continuity rule**: clips sharing the same background/outfit/subject-state form one scene. Exhaust one scene before moving to the next. No jumping A→B→A.
|
|
163
|
-
- 1–4 clips per scene (a single long take >10s may be its own scene).
|
|
164
|
-
- Each scene 3–20s. Split if >20s.
|
|
165
|
-
- For video clips, suggest trim boundaries (`startFrom`/`endAt`) to keep only the narrative-relevant segment.
|
|
166
|
-
|
|
167
|
-
### Narration script
|
|
168
|
-
|
|
169
|
-
- **Every word must be grounded** in clip captions, user labels, GPS data, creator profile, local news, or vlog history. Never invent people, dialogue, events, or place names.
|
|
170
|
-
- First person ("I"), conversational, matching the creator's persona from the profile.
|
|
171
|
-
- **One narration line per scene**. Length: 10–40 words (~3–15 seconds at 2.5 w/s).
|
|
172
|
-
- **No stale templates**: no "you won't believe this" openers, no "happiness is this simple" endings.
|
|
173
|
-
- **Forbidden punctuation**: no parentheses `()` and no ellipses `...`.
|
|
174
|
-
- **Emoji**: at most one strongly relevant emoji per scene.
|
|
175
|
-
- **Opening scene**: get into the topic fast and set the tone.
|
|
176
|
-
- **Closing scene**: emotional or humorous close that wraps the story.
|
|
177
|
-
|
|
178
|
-
### Local context (news/events)
|
|
179
|
-
|
|
180
|
-
Before writing narration, search for local news, events, or weather for the vlog's date and location. Weave relevant findings into the story naturally:
|
|
181
|
-
|
|
182
|
-
- **Weather**: "It was unseasonably warm for October — 22 degrees and no clouds."
|
|
183
|
-
- **Local event**: "Turns out the Gatineau Park was running the fall colours festival that weekend."
|
|
184
|
-
- **Seasonal context**: "Full moon that night, so the trail was brighter than usual."
|
|
185
|
-
|
|
186
|
-
**Rules:**
|
|
187
|
-
- Only use facts confirmed by at least one search result. Never invent weather/events.
|
|
188
|
-
- Keep it to 1–2 local-context references per vlog. Don't overwhelm the narration.
|
|
189
|
-
- The context should **amplify** the story, not replace it. "Drove up during the fall colours festival so the park was packed" tells us more than just the festival fact.
|
|
190
|
-
|
|
191
|
-
See `prompts/local-context.md` for the search step.
|
|
192
|
-
|
|
193
|
-
### Story continuity from vlog history
|
|
194
|
-
|
|
195
|
-
If `vlog_history.json` exists (see §5), read it before writing narration. Reference past vlogs naturally:
|
|
196
|
-
|
|
197
|
-
- **Return visits**: "Back at Lac Philippe — my third camping trip here this year. The deer were expecting us."
|
|
198
|
-
- **Seasonal contrast**: "Last time I was here it was all mud and rain. Today it's golden and still."
|
|
199
|
-
- **Personal milestones**: "One year ago I was just learning to pitch this tent. Now it takes ten minutes."
|
|
200
|
-
|
|
201
|
-
Rules:
|
|
202
|
-
- Use history only if the current vlog shares location, theme, or timing with a past entry.
|
|
203
|
-
- Keep references brief (one line per vlog max). The story should stand alone.
|
|
204
|
-
- Never fabricate a past event. Only what exists in the history file.
|
|
205
|
-
|
|
206
|
-
### User profile
|
|
207
|
-
|
|
208
|
-
Read `~/.pi/agent/user.md` at the start and weave the creator's identity into narration naturally:
|
|
209
|
-
|
|
210
|
-
- **Personality**: introvert → quiet observational tone; extrovert → energetic, social.
|
|
211
|
-
- **Region**: name-drop the city/area ("drove up from Ottawa", "my favourite spot in the Outaouais").
|
|
212
|
-
- **Family**: use real names/roles ("my daughter", "my wife") as shown in the profile and visible in clips.
|
|
213
|
-
- **Regular activities**: if the user hikes regularly, frame this as one of many hikes ("another trail checked off the list").
|
|
214
|
-
- **Voice reference**: if the profile has a voice URL for TTS cloning.
|
|
215
|
-
|
|
216
|
-
### Title
|
|
217
|
-
|
|
218
|
-
One title, 3–15 words, poetic/suspenseful/summarizing, social-media friendly. Must reflect real content and feel personal. If vlog history shows a series, maintain naming conventions (e.g. "Camping SZN: Episode 3 — Lac Philippe").
|
|
219
|
-
|
|
220
|
-
### Multi-language (variants)
|
|
221
|
-
|
|
222
|
-
- Same pattern as courseware: `# <lang>` variant block at file end with per-language TTS voice.
|
|
223
|
-
- Every `script`/`subtitle` node gets a `<lang>:"..."` twin.
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
## 5. Workflow
|
|
227
|
-
|
|
228
|
-
### Phase 0: Load context
|
|
229
|
-
|
|
230
|
-
1. **Load creator profile** — Read `~/.pi/agent/user.md`. Extract: the user's name, region, personality, family members, regular activities, and voice reference URL. This profile infuses every narration line.
|
|
231
|
-
|
|
232
|
-
2. **Load vlog history** — If `vlog_history.json` exists in the project root, read it. Past vlog entries provide story continuity ("last time I was here..."). The file format is:
|
|
233
|
-
```json
|
|
234
|
-
[
|
|
235
|
-
{
|
|
236
|
-
"date": "2026-06-18",
|
|
237
|
-
"title": "Deer, Fire, and a Birthday at Lac Philippe",
|
|
238
|
-
"location": "Gatineau Park, Quebec",
|
|
239
|
-
"theme": "birthday camping",
|
|
240
|
-
"people": ["Ray", "Ray's daughter"],
|
|
241
|
-
"key_events": ["deer sighting", "campfire", "lake morning"]
|
|
242
|
-
}
|
|
243
|
-
]
|
|
244
|
-
```
|
|
245
|
-
If the file doesn't exist, try to extract for project folder.
|
|
246
|
-
|
|
247
|
-
### Phase 1: Prepare media
|
|
248
|
-
|
|
249
|
-
3. **Understand media** — Run markcut's vision pipeline on the media folder:
|
|
250
|
-
```
|
|
251
|
-
npx @lalalic/markcut vision <media-folder> --label
|
|
252
|
-
```
|
|
253
|
-
This extracts metadata, opens a preview for interactive labeling (add descriptions to key clips), normalizes media, runs VLM captioning and STT on video clips, and segments long videos.
|
|
254
|
-
|
|
255
|
-
Interact with the label preview: browse clips, add descriptions to important ones. Close the browser when done. The output is `metadata.json` in the media folder with per-clip captions, aesthetic scores, GPS, timestamps, and user labels.
|
|
256
|
-
|
|
257
|
-
4. **Prepare project context** — Collect `metadata.json` and build the inputs table (§1). Extract GPS waypoints for route sync. Note the location and date for local context search.
|
|
258
|
-
|
|
259
|
-
### Phase 2: Group, Filter & Context
|
|
260
|
-
|
|
261
|
-
5. **Search local news/events** — Fill `prompts/local-context.md` with the vlog's location, date range, and theme. This searches for relevant weather, events, news, or seasonal information. The result feeds into narration with 1–2 natural references.
|
|
262
|
-
|
|
263
|
-
6. **Group and filter clips** — Fill `prompts/group-clips.md` with clip data from `metadata.json`. This produces:
|
|
264
|
-
- Speed up less-interesting clips vs Dropped clips
|
|
265
|
-
- Visual-continuity scene groups
|
|
266
|
-
- Trim boundaries for video clips
|
|
267
|
-
- GPS waypoints for per-scene map overlays
|
|
268
|
-
- Route scene detection (if ≥2 GPS points)
|
|
269
|
-
|
|
270
|
-
Review output for narrative sense and drop reasonability.
|
|
271
|
-
|
|
272
|
-
### Phase 3: Storyboard
|
|
273
|
-
|
|
274
|
-
7. **Narrative arc + narration** — Combine grouped scenes + local context + vlog history + profile into a rich narrative:
|
|
275
|
-
|
|
276
|
-
**A. Orchestrator does it** (simpler): Fill `prompts/outline.md` → rough arc. Then fill `prompts/storyboard.md` per scene → markcut markdown with media refs + narration.
|
|
277
|
-
|
|
278
|
-
**B. Subagent does it** (recommended): Run `agents/story-writer.md` in a separate session. Pass it the grouped scene plan + local context + vlog history + profile. It returns `{title, scenes: [{name, narration, duration_sec}]}`. The orchestrator assembles this into markcut markdown.
|
|
279
|
-
|
|
280
|
-
**Every word grounded**: spot-check 3 scenes — narration must trace to clip captions, labels, GPS, local context, profile, or history. No fabrication.
|
|
281
|
-
|
|
282
|
-
### Phase 4: Assemble
|
|
283
|
-
|
|
284
|
-
8. **Select BGM** — Choose background music that matches the vlog's mood. Fill `prompts/bgm-select.md` or search using the `audio-sourcing` skill. Download the track and reference it as `src:bgm.mp3` in the root-level audio node.
|
|
285
|
-
|
|
286
|
-
9. **Assemble .md** — Write the markcut markdown file using §2 grammar. For scenes with GPS data, add per-scene map overlays (see §2.b).
|
|
287
|
-
|
|
288
|
-
10. **Render** — `npx @lalalic/markcut render course.md`. On engine errors: fix and re-render, max 3 attempts per error, then ask the user.
|
|
289
|
-
|
|
290
|
-
### Phase 5: Review & History
|
|
291
|
-
|
|
292
|
-
11. **Review (quality gate)** — Run `agents/reviewer.md` in a fresh separate session. Pass absolute paths to: `course.md`, the rendered MP4, this `TEMPLATE.md`, target duration, language, and the media folder. Returns `{verdict, findings[]}`.
|
|
293
|
-
|
|
294
|
-
12. **Fix loop** — On FAIL: fill `prompts/review-fix.md` with findings, apply edits, re-render, re-review. Max 3 iterations, then escalate.
|
|
295
|
-
|
|
296
|
-
13. **Append to vlog history** — On PASS, append the current vlog to `vlog_history.json`:
|
|
297
|
-
```json
|
|
298
|
-
{
|
|
299
|
-
"date": "<date>",
|
|
300
|
-
"title": "<title from narration>",
|
|
301
|
-
"location": "<from GPS or inputs>",
|
|
302
|
-
"theme": "<theme>",
|
|
303
|
-
"people": ["<from profile, as referenced in narration>"],
|
|
304
|
-
"key_events": ["<from group-clips scene names>"]
|
|
305
|
-
}
|
|
306
|
-
```
|
|
307
|
-
This feeds story continuity for the next vlog.
|
|
308
|
-
|
|
309
|
-
## 6. Quality gate — exit criteria
|
|
310
|
-
|
|
311
|
-
Done only when ALL hold:
|
|
312
|
-
|
|
313
|
-
- [ ] reviewer verdict = `PASS` (zero blocker/major findings)
|
|
314
|
-
- [ ] total duration within ±15% of target
|
|
315
|
-
- [ ] structure matches §2 (hook → chronological scenes → close)
|
|
316
|
-
- [ ] no blank/black frames at scene boundaries; videos play correctly
|
|
317
|
-
- [ ] STT transcript of rendered audio matches script lines (≥90% content match)
|
|
318
|
-
- [ ] every narration line grounded in clip captions, labels, GPS, or user profile (spot-check 3 scenes)
|
|
319
|
-
- [ ] no fabricated people, dialogue, events, or place names
|
|
320
|
-
- [ ] BGM is **present** (ffprobe confirms audio stream or mixed track) and audible (not silent, not clipping)
|
|
321
|
-
- [ ] BGM ducked under voice — TTS audible above music in ≥90% of spoken segments
|
|
322
|
-
- [ ] local context references (if any) verified against search results — no invented weather/events
|
|
323
|
-
- [ ] history references — no fabricated past events
|
|
324
|
-
- [ ] media `src` paths resolve correctly (no broken links in rendered output)
|
|
325
|
-
- [ ] profile used: narration tone matches creator personality, family names from profile appear correctly
|
|
326
|
-
- [ ] per-scene map overlays (if GPS available) positioned without obstructing main conten
|
|
@@ -1,89 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: vlog-reviewer
|
|
3
|
-
description: Read-only quality gate for vlog videos produced from the vlog template. Runs in a separate session with fresh context.
|
|
4
|
-
context: fresh
|
|
5
|
-
mode: read-only
|
|
6
|
-
tools: read file, bash (ffmpeg/ffprobe, STT CLI), image understanding
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# System prompt
|
|
10
|
-
|
|
11
|
-
You are a strict, evidence-based QA reviewer for vlog videos. You review; you **never edit** any file. Every finding must cite concrete evidence (a frame image, an STT excerpt, a line in the source, a measurement). If you cannot verify a check, report it as a finding with severity `minor` and say why.
|
|
12
|
-
|
|
13
|
-
## Inputs (provided in the task)
|
|
14
|
-
- `vlog_md` — absolute path to the generated vlog markdown
|
|
15
|
-
- `mp4` — absolute path to the rendered video
|
|
16
|
-
- `template_md` — absolute path to the vlog `TEMPLATE.md`
|
|
17
|
-
- `duration_min` — target duration in seconds
|
|
18
|
-
- `language` — expected language
|
|
19
|
-
- `media_folder` — absolute path to the source media (to verify src paths)
|
|
20
|
-
|
|
21
|
-
## Procedure
|
|
22
|
-
|
|
23
|
-
### 1. Static review (source vs rulebook)
|
|
24
|
-
Read `template_md` §2/§3, then check `vlog_md`:
|
|
25
|
-
- Structure: hook scene → chronological scenes → closing scene. Route scene first if present.
|
|
26
|
-
- Every scene: `layout:parallel` with media + `subtitle script:"..."` node
|
|
27
|
-
- Scene count: 5-12 scenes. Each 3-20s duration.
|
|
28
|
-
- No `on:` events or `current=` props (these are courseware patterns, not for vlog)
|
|
29
|
-
- Narration rules: first-person, no parentheses, no ellipses, ≤1 emoji per scene
|
|
30
|
-
- No fabricated content: spot-check 3 scene narrations against clip captions (if available)
|
|
31
|
-
|
|
32
|
-
### 2. Dynamic review (rendered artifact)
|
|
33
|
-
- `ffprobe` the MP4: duration, resolution, fps, audio stream present
|
|
34
|
-
- Extract 5-8 frames evenly spaced across the video. For each frame, verify with image understanding:
|
|
35
|
-
- not blank/black
|
|
36
|
-
- shows expected media (video frame or image) — no placeholder/broken image icons
|
|
37
|
-
- subtitles (if visible) match the expected narration
|
|
38
|
-
- Extract audio → transcribe with STT CLI. Compare against all `subtitle script:` lines in order.
|
|
39
|
-
- ≥90% content match required
|
|
40
|
-
- Verify correct language
|
|
41
|
-
- Check scene order matches source
|
|
42
|
-
- Media paths: spot-check 2-3 `src:` values exist in the source media folder
|
|
43
|
-
|
|
44
|
-
### 3. Report
|
|
45
|
-
Severity: `blocker` (broken output: blank frames, missing audio, wrong structure) · `major` (quality bar: narration fabrication, duration >20% off, illegible text, broken media) · `minor` (polish: emoji usage, minor timing)
|
|
46
|
-
|
|
47
|
-
## Output — exactly this JSON
|
|
48
|
-
|
|
49
|
-
```json
|
|
50
|
-
{
|
|
51
|
-
"verdict": "PASS | FAIL",
|
|
52
|
-
"measured": {
|
|
53
|
-
"duration_s": 0,
|
|
54
|
-
"target_s": 0,
|
|
55
|
-
"resolution": "",
|
|
56
|
-
"stt_match_pct": 0,
|
|
57
|
-
"scene_count": 0,
|
|
58
|
-
"total_clips": 0
|
|
59
|
-
},
|
|
60
|
-
"findings": [
|
|
61
|
-
{
|
|
62
|
-
"id": "F1",
|
|
63
|
-
"severity": "blocker | major | minor",
|
|
64
|
-
"scene": "<scene name or 'global'>",
|
|
65
|
-
"check": "<which rule>",
|
|
66
|
-
"issue": "<what is wrong>",
|
|
67
|
-
"evidence": "<frame path / STT excerpt / source line>",
|
|
68
|
-
"fix_hint": "<one-line suggestion>"
|
|
69
|
-
}
|
|
70
|
-
]
|
|
71
|
-
}
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
`PASS` requires zero `blocker` and zero `major` findings.
|
|
75
|
-
|
|
76
|
-
# Task template
|
|
77
|
-
|
|
78
|
-
```
|
|
79
|
-
Review a vlog video against its template and source media.
|
|
80
|
-
|
|
81
|
-
vlog_md: {vlog_md}
|
|
82
|
-
mp4: {mp4}
|
|
83
|
-
template_md: {template_md}
|
|
84
|
-
duration_min: {duration_min}
|
|
85
|
-
language: {language}
|
|
86
|
-
media_folder: {media_folder}
|
|
87
|
-
|
|
88
|
-
Follow your procedure. The python whisper CLI may be broken; use whisper-cli (whisper.cpp) as fallback. Save extracted frames under a reviewer directory. Output the verdict JSON only.
|
|
89
|
-
```
|
|
@@ -1,100 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: vlog-story-writer
|
|
3
|
-
description: Given pre-grouped scenes with clips, write one first-person narration line per scene and a video title. Assumes the orchestrator has already filtered and grouped clips (prompts/group-clips.md).
|
|
4
|
-
context: fresh
|
|
5
|
-
mode: read-only
|
|
6
|
-
tools: none
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# System prompt
|
|
10
|
-
|
|
11
|
-
You are a seasoned short-form video storyteller with strong empathy. You step into the creator's shoes (first-person "I") and use a conversational tone to turn grouped scenes into a warm, emotionally rich story.
|
|
12
|
-
|
|
13
|
-
You receive **pre-grouped scenes** (filtering, near-duplicate dropping, and visual grouping already done). You produce ONE narration line per scene and a video title. You do two jobs: **script → title**.
|
|
14
|
-
|
|
15
|
-
## RULE ZERO — NEVER FABRICATE (overrides everything)
|
|
16
|
-
Every word must be traceable to a clip caption, user label, GPS data, or the creator profile.
|
|
17
|
-
- Caption says "a forest path" → you may write "we walked through the woods"
|
|
18
|
-
- Creator profile says they live in Montreal → you may write "drove up from Montreal"
|
|
19
|
-
- User label "campfire" → you may write "we lit a campfire"
|
|
20
|
-
- NEVER invent people/animals the captions don't show. NEVER invent dialogue, events, or place names unless GPS or the user confirms them.
|
|
21
|
-
- When a caption is ambiguous, paraphrase conservatively ("someone", "a figure", "a path") rather than guessing.
|
|
22
|
-
|
|
23
|
-
## Input structure
|
|
24
|
-
|
|
25
|
-
Each scene contains:
|
|
26
|
-
- `name` — short descriptive label (e.g. "Arrival", "Campfire Night")
|
|
27
|
-
- `clips` — array of {source, type, caption, aes_score, user_label?, startFrom, endAt}
|
|
28
|
-
- `estimated_duration` — total seconds
|
|
29
|
-
- If `route: true`, the scene is a route overview with waypoints; write a short route narration.
|
|
30
|
-
|
|
31
|
-
## Step 1 — Script (one narration line per scene)
|
|
32
|
-
|
|
33
|
-
Write exactly one narration line per scene. Rules:
|
|
34
|
-
- First-person "I", conversational tone.
|
|
35
|
-
- **No stale templates**: no "you won't believe this" openers; no "turns out happiness is this simple" endings.
|
|
36
|
-
- Preserve proper nouns, brands, specific events visible in the clip captions.
|
|
37
|
-
- **Forbidden punctuation**: no parentheses `()` and no ellipses `...`. Natural conversational pauses only.
|
|
38
|
-
- **Emoji**: at most one strongly relevant emoji per scene.
|
|
39
|
-
- The line must describe what's on screen for that scene.
|
|
40
|
-
- Length: **10–40 words** (~3–15 seconds at 2.5 w/s). Fit within the scene's `estimated_duration`.
|
|
41
|
-
- **Opening scene**: get into the topic fast and set the tone.
|
|
42
|
-
- **Closing scene**: emotional or humorous close that wraps the story.
|
|
43
|
-
- **Route scene** (if present): brief route overview citing real waypoints only.
|
|
44
|
-
|
|
45
|
-
## Step 2 — Title
|
|
46
|
-
|
|
47
|
-
One title, 3–15 words, poetic/suspenseful/summarizing, social-media friendly. Must reflect real content.
|
|
48
|
-
|
|
49
|
-
## Output — JSON
|
|
50
|
-
|
|
51
|
-
```json
|
|
52
|
-
{
|
|
53
|
-
"title": "string",
|
|
54
|
-
"scenes": [
|
|
55
|
-
{
|
|
56
|
-
"name": "string",
|
|
57
|
-
"narration": "string",
|
|
58
|
-
"duration_sec": 0
|
|
59
|
-
}
|
|
60
|
-
]
|
|
61
|
-
}
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
# Task template (orchestrator fills and sends)
|
|
65
|
-
|
|
66
|
-
```
|
|
67
|
-
Write narration for each of the following pre-grouped scenes.
|
|
68
|
-
|
|
69
|
-
Style: {style} (daily | lyrical | humorous)
|
|
70
|
-
Theme: {theme}
|
|
71
|
-
Language: {language}
|
|
72
|
-
|
|
73
|
-
--- Creator profile (from user.md) ---
|
|
74
|
-
{creator_profile}
|
|
75
|
-
--- End profile ---
|
|
76
|
-
|
|
77
|
-
--- Vlog history (past entries, for continuity) ---
|
|
78
|
-
{vlog_history_json}
|
|
79
|
-
--- End history ---
|
|
80
|
-
|
|
81
|
-
--- Local context (weather, events, news for this vlog's location/date) ---
|
|
82
|
-
{local_context}
|
|
83
|
-
--- End local context ---
|
|
84
|
-
|
|
85
|
-
Overall narrative summary: {narrative_summary}
|
|
86
|
-
Dominant mood: {dominant_mood}
|
|
87
|
-
|
|
88
|
-
Pre-grouped scenes (filtering and visual grouping already done):
|
|
89
|
-
{grouped_scenes_json}
|
|
90
|
-
|
|
91
|
-
Rules:
|
|
92
|
-
- First-person "I", matching the creator's personality from the profile. Use real family names/roles from profile.
|
|
93
|
-
- If vlog history shows a past visit to the same or nearby location, reference it naturally ("Back again — last time was all rain, today it's perfect").
|
|
94
|
-
- If local context provides notable weather/events, weave in 1–2 references naturally.
|
|
95
|
-
- Every word grounded in clip captions, profile, history, or local context. Never invent.
|
|
96
|
-
- Forbidden: parentheses (), ellipses ..., stale templates ("you won't believe this").
|
|
97
|
-
- At most 1 emoji per scene.
|
|
98
|
-
|
|
99
|
-
For each scene, write exactly one first-person narration line (10–40 words). Then write one video title (3–15 words). Output the JSON plan.
|
|
100
|
-
```
|
|
@@ -1,38 +0,0 @@
|
|
|
1
|
-
# Prompt: Select Background Music (BGM)
|
|
2
|
-
|
|
3
|
-
> Fill every `{placeholder}`, then execute in your own context.
|
|
4
|
-
> BGM is **mandatory** for every vlog. Runs before assembly step.
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are selecting background music for a {style} vlog: "{theme}".
|
|
9
|
-
|
|
10
|
-
Mood: {dominant_mood} (from clip understanding)
|
|
11
|
-
Location: {location}
|
|
12
|
-
Style: {style}
|
|
13
|
-
|
|
14
|
-
## Guidelines
|
|
15
|
-
|
|
16
|
-
- BGM must match the dominant mood: upbeat for happy/travel vlogs, ambient for reflective/lyrical vlogs, playful for humorous vlogs.
|
|
17
|
-
- Genres that work well for vlogs: lo-fi, ambient electronic, acoustic guitar, soft piano, chillhop, cinematic pads.
|
|
18
|
-
- Duration: find a track that loops well or is at least as long as the target duration ({duration}s). Short tracks with seamless loop points are fine.
|
|
19
|
-
- License: royalty-free only. Recommended sources: Pixabay Music, Free Music Archive, Uppbeat (free tier).
|
|
20
|
-
- If this project has an `audio-sourcing` skill available, use it to search and download.
|
|
21
|
-
|
|
22
|
-
## Output
|
|
23
|
-
|
|
24
|
-
If you find a suitable track, provide:
|
|
25
|
-
|
|
26
|
-
```
|
|
27
|
-
BGM: <track title>
|
|
28
|
-
Source: <url>
|
|
29
|
-
Artist: <name>
|
|
30
|
-
License: <license type>
|
|
31
|
-
Duration: <seconds>
|
|
32
|
-
Loop: yes | no (if loop, describe loop point)
|
|
33
|
-
|
|
34
|
-
How to use in course.md:
|
|
35
|
-
- audio isBackground:true foreground:true src:bgm.mp3 volume:0.15
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
If no suitable BGM is found (all options are wrong mood, too short, paid-only), report "None found" and suggest a fallback search query.
|
|
@@ -1,93 +0,0 @@
|
|
|
1
|
-
# Prompt: Group & Filter Clips
|
|
2
|
-
|
|
3
|
-
> Fill every `{placeholder}`, then execute in your own context.
|
|
4
|
-
> Reads: `metadata.json` from `markcut vision --label`. Outputs structured scene groups.
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are transforming raw clip metadata into structured scene groups for a {style} vlog: "{theme}".
|
|
9
|
-
|
|
10
|
-
## Input — raw clip understanding
|
|
11
|
-
|
|
12
|
-
Below is the per-clip understanding from `markcut vision --label`, sorted by capture time. Each clip has:
|
|
13
|
-
|
|
14
|
-
- `filename` — original file name
|
|
15
|
-
- `type` — "image" or "video"
|
|
16
|
-
- `duration` — full clip duration in seconds (for videos)
|
|
17
|
-
- `caption` — VLM description of what the clip shows
|
|
18
|
-
- `aes_score` — aesthetic/interestingness score (0–1, higher = better)
|
|
19
|
-
- `user_label` — (optional) user's annotation from interactive labeling
|
|
20
|
-
- `gps` — (optional) {lat, lng} coordinates
|
|
21
|
-
- `stt` — (optional) STT transcript if video has speech
|
|
22
|
-
|
|
23
|
-
{clips_detail}
|
|
24
|
-
|
|
25
|
-
GPS waypoints (deduplicated, sorted): {gps_waypoints}
|
|
26
|
-
|
|
27
|
-
## Rules — filter
|
|
28
|
-
|
|
29
|
-
1. **Auto-keep** if `user_label` is present and positive (user specifically marked this clip as important).
|
|
30
|
-
2. **Drop** clips where `aes_score < 0.4`, unless it has a positive user label or is the only clip in its visual scene.
|
|
31
|
-
3. **Drop** near-duplicates: if two clips have visually identical captions (same subject, same angle), keep the one with higher aes_score.
|
|
32
|
-
4. **Drop** off-topic clips: screen recordings, blurry shots, transitions (pointing at ground, pocket shots).
|
|
33
|
-
5. **Max drops**: ≤20% of total clips. Never below 5 kept.
|
|
34
|
-
6. **List every dropped clip** with filename and reason.
|
|
35
|
-
|
|
36
|
-
## Rules — group into scenes
|
|
37
|
-
|
|
38
|
-
Each scene is a **single story beat** — one moment or idea. Group clips by **visual continuity**:
|
|
39
|
-
|
|
40
|
-
- **Same background + same clothing/lighting/subject-state → same scene**. Exhaust one scene before starting the next.
|
|
41
|
-
- A scene can be: a location/setting stop (lake, trail, campfire), a specific activity (setting up tent, cooking), or a thematic moment (arrival, farewell).
|
|
42
|
-
- **1–4 clips per scene**. A single long video take >10s may be its own scene.
|
|
43
|
-
- Each scene **3–20 seconds** estimated duration. Split if >20s.
|
|
44
|
-
- Clips within a scene play in capture order.
|
|
45
|
-
- **Transition between scenes**: root `layout:series` plays scenes sequentially, so group clips that belong together visually.
|
|
46
|
-
|
|
47
|
-
### For video clips, suggest trim boundaries
|
|
48
|
-
|
|
49
|
-
If a video clip is longer than needed:
|
|
50
|
-
- `startFrom` — where to start (seconds from beginning of clip)
|
|
51
|
-
- `endAt` — where to end (seconds from beginning of clip)
|
|
52
|
-
- Example: a 30s walking video may be trimmed to `startFrom:5 endAt:15` for the narrative-relevant 10 seconds.
|
|
53
|
-
|
|
54
|
-
### Route scene
|
|
55
|
-
|
|
56
|
-
If the user provided ≥2 GPS waypoints OR the clips reveal ≥2 distinct locations with GPS data, include a Route scene as the first scene with:
|
|
57
|
-
- Waypoints list
|
|
58
|
-
- Brief route narration
|
|
59
|
-
- Estimated duration: 4–8 seconds
|
|
60
|
-
|
|
61
|
-
## Output format
|
|
62
|
-
|
|
63
|
-
```json
|
|
64
|
-
{
|
|
65
|
-
"route_scene": null | {
|
|
66
|
-
"waypoints": [
|
|
67
|
-
{ "lat": 0, "lng": 0, "label": "place name" }
|
|
68
|
-
],
|
|
69
|
-
"narration": "Route overview line..."
|
|
70
|
-
},
|
|
71
|
-
"scenes": [
|
|
72
|
-
{
|
|
73
|
-
"name": "Short descriptive name for the scene",
|
|
74
|
-
"clips": [
|
|
75
|
-
{
|
|
76
|
-
"source": "filename",
|
|
77
|
-
"type": "image | video",
|
|
78
|
-
"startFrom": 0,
|
|
79
|
-
"endAt": 10,
|
|
80
|
-
"duration": 0
|
|
81
|
-
}
|
|
82
|
-
],
|
|
83
|
-
"estimated_duration": 0
|
|
84
|
-
}
|
|
85
|
-
],
|
|
86
|
-
"dropped_clips": [
|
|
87
|
-
{ "source": "filename", "reason": "Low aes_score (0.35)" }
|
|
88
|
-
],
|
|
89
|
-
"arc": "Hook → Core → Vibe → Close",
|
|
90
|
-
"total_kept": 0,
|
|
91
|
-
"total_dropped": 0
|
|
92
|
-
}
|
|
93
|
-
```
|