@lalalic/markcut 2.1.0 → 2.2.1

package/skills/markcut/docs/templates/vlog/prompts/group-clips.md

@@ -0,0 +1,93 @@
+# Prompt: Group & Filter Clips
+
+> Fill every `{placeholder}`, then execute in your own context.
+> Reads: `metadata.json` from `markcut vision --label`. Outputs structured scene groups.
+
+---
+
+You are transforming raw clip metadata into structured scene groups for a {style} vlog: "{theme}".
+
+## Input — raw clip understanding
+
+Below is the per-clip understanding from `markcut vision --label`, sorted by capture time. Each clip has:
+
+- `filename` — original file name
+- `type` — "image" or "video"
+- `duration` — full clip duration in seconds (for videos)
+- `caption` — VLM description of what the clip shows
+- `aes_score` — aesthetic/interestingness score (0–1, higher = better)
+- `user_label` — (optional) user's annotation from interactive labeling
+- `gps` — (optional) {lat, lng} coordinates
+- `stt` — (optional) STT transcript if video has speech
+
+{clips_detail}
+
+GPS waypoints (deduplicated, sorted): {gps_waypoints}
+
+## Rules — filter
+
+1. **Auto-keep** if `user_label` is present and positive (user specifically marked this clip as important).
+2. **Drop** clips where `aes_score < 0.4`, unless it has a positive user label or is the only clip in its visual scene.
+3. **Drop** near-duplicates: if two clips have visually identical captions (same subject, same angle), keep the one with higher aes_score.
+4. **Drop** off-topic clips: screen recordings, blurry shots, transitions (pointing at ground, pocket shots).
+5. **Max drops**: ≤20% of total clips. Never below 5 kept.
+6. **List every dropped clip** with filename and reason.
+
+## Rules — group into scenes
+
+Each scene is a **single story beat** — one moment or idea. Group clips by **visual continuity**:
+
+- **Same background + same clothing/lighting/subject-state → same scene**. Exhaust one scene before starting the next.
+- A scene can be: a location/setting stop (lake, trail, campfire), a specific activity (setting up tent, cooking), or a thematic moment (arrival, farewell).
+- **1–4 clips per scene**. A single long video take >10s may be its own scene.
+- Each scene **3–20 seconds** estimated duration. Split if >20s.
+- Clips within a scene play in capture order.
+- **Transition between scenes**: root `layout:series` plays scenes sequentially, so group clips that belong together visually.
+
+### For video clips, suggest trim boundaries
+
+If a video clip is longer than needed:
+- `startFrom` — where to start (seconds from beginning of clip)
+- `endAt` — where to end (seconds from beginning of clip)
+- Example: a 30s walking video may be trimmed to `startFrom:5 endAt:15` for the narrative-relevant 10 seconds.
+
+### Route scene
+
+If the user provided ≥2 GPS waypoints OR the clips reveal ≥2 distinct locations with GPS data, include a Route scene as the first scene with:
+- Waypoints list
+- Brief route narration
+- Estimated duration: 4–8 seconds
+
+## Output format
+
+```json
+{
+  "route_scene": null | {
+    "waypoints": [
+      { "lat": 0, "lng": 0, "label": "place name" }
+    ],
+    "narration": "Route overview line..."
+  },
+  "scenes": [
+    {
+      "name": "Short descriptive name for the scene",
+      "clips": [
+        {
+          "source": "filename",
+          "type": "image | video",
+          "startFrom": 0,
+          "endAt": 10,
+          "duration": 0
+        }
+      ],
+      "estimated_duration": 0
+    }
+  ],
+  "dropped_clips": [
+    { "source": "filename", "reason": "Low aes_score (0.35)" }
+  ],
+  "arc": "Hook → Core → Vibe → Close",
+  "total_kept": 0,
+  "total_dropped": 0
+}
+```

package/skills/markcut/docs/templates/vlog/prompts/local-context.md

@@ -0,0 +1,59 @@
+# Prompt: Search Local Context (News, Weather, Events)
+
+> Fill every `{placeholder}`, then execute in your own context.
+> Runs after Phase 1 (media understanding) before storyboard writing.
+> Uses web search to find real facts about the vlog's location and date.
+
+---
+
+You are researching local context for a vlog about "{theme}" at {location}.
+
+Date range: {date_range} (or approximate dates from media timestamps)
+Location description: {location} (from GPS reverse-geocode or user input)
+Vlog style: {style}
+
+## Task
+
+Search for the following and report real facts that could naturally weave into the narration:
+
+1. **Weather** — What was the weather like on those dates? High/low temps, conditions (sunny, rainy, cloudy), notable patterns (heatwave, storm, full moon).
+2. **Local events** — Any festivals, markets, sports events, holidays, or community gatherings happening at that location during the date range.
+3. **Seasonal context** — Seasonal highlights: fall colours, cherry blossoms, ski season opening, migration patterns, park hours.
+4. **Recent news** — Any notable local news near that location within the week before the vlog dates (park closures, trail openings, wildlife alerts, cultural events).
+
+Use multiple search queries for coverage. Prefer official sources (weather service, park website, local news).
+
+## Output format
+
+```
+## Local Context
+
+Weather:
+- <fact 1> (source: <url>)
+- <fact 2> (source: <url>)
+
+Events:
+- <fact> (source: <url>)
+
+Seasonal:
+- <fact> (source: <url>)
+
+News:
+- <fact> (source: <url>)
+
+## Amplification ideas (optional, for narrator)
+
+- <How a fact amplifies the story: "The fall colours festival meant the park was busier than usual, which explains the crowded trail clip">
+- <Another idea>
+```
+
+## Rules
+
+- Only include facts confirmed by at least one web source. If a search yields nothing useful for a category, write "None found."
+- Do not invent weather, events, or news.
+- The amplification ideas are suggestions only — the story-writer decides whether to use them.
+- Focus on facts that **connect** to what the media shows (weather matching the sky in photos, events explaining crowds, seasonal features matching the scenery).
+
+## After this step
+
+Pass the result to the storyboard phase (Phase 3). The story-writer agent or outline prompt should receive the local context as an additional input. Use at most **2 local-context references** in the final narration — enough to add color, not enough to overwhelm the personal story.

package/skills/markcut/docs/templates/vlog/prompts/outline.md

@@ -0,0 +1,56 @@
+# Prompt: Vlog Outline — Narrative Arc
+
+> Fill every `{placeholder}`, then execute in your own context.
+> Reads: `metadata.json` (per-clip captions, aes_score, GPS), project inputs.
+
+---
+
+You are planning a {duration_min}-second {style} vlog: "{theme}".
+
+Creator: {creator_profile}
+Language: {language}
+
+Given the clip understanding below, plan the narrative arc.
+
+Clip understanding (from `markcut vision --label`):
+- Total clips: {total_clips}
+- Dominant mood: {dominant_mood}
+- Narrative summary: {narrative_summary}
+- Key subjects: {key_subjects}
+{clips_detail}
+
+Requirements:
+
+1. **Hook → Core → Vibe → Close arc**: First scene hooks the viewer with the best visual + intriguing line. Middle scenes are chronological story beats. Final scene provides closure.
+
+2. **Visual continuity**: Clips sharing same background/outfit/subject-state form one scene. No jumping A→B→A.
+
+3. **Drop rules**: Keep all clips unless aes_score < 0.4, near-duplicate, or off-topic. Max ~20% drops. Never below 5 clips.
+
+4. **GPS**: {gps_info} — if ≥2 GPS waypoints, include a route scene with map.
+
+Output format:
+
+```
+## Scene Plan
+
+### Route Scene (if applicable)
+- waypoints: <list>
+- route narration: <one line describing the route>
+
+### Scene 1: <title>
+- clips: <filenames with time ranges>
+- narration: <first-person line, 10-40 words>
+- mood: <mood word>
+- duration: <estimated seconds>
+
+### Scene 2: <title>
+...
+
+### Scene N: <title>
+
+Dropped clips:
+- <filename>: <reason>
+
+Title suggestion: <3-15 words>
+```

package/skills/markcut/docs/templates/vlog/prompts/review-fix.md

@@ -0,0 +1,27 @@
+# Prompt: Apply Reviewer Findings
+
+> Fill every `{placeholder}`, then execute in your own context.
+> Used after a FAIL verdict. You edit; the reviewer never does.
+
+---
+
+Apply the reviewer findings below to `{vlog_md_path}` with minimal, targeted edits.
+
+Reviewer findings (JSON):
+
+{findings_json}
+
+Rules:
+- Address every `blocker` and `major` finding. Minor findings only when trivially fixable.
+- Each fix must respect TEMPLATE.md §2 scene grammar and §3 authoring rules.
+- Narration fabrication → rewrite the line to use conservative paraphrase of clip captions.
+- Broken src → fix the path or add a placeholder image.
+- Duration mismatch → shorten/lengthen narration and adjust subtitle `duration:`.
+- Blank frames → check startFrom/endAt values, clip ordering, and file existence.
+
+After editing, list what changed:
+```
+- <finding id>: <scene> — <one-line description of the edit>
+```
+
+Then re-render and re-review.

package/skills/markcut/docs/templates/vlog/prompts/storyboard.md

@@ -0,0 +1,41 @@
+# Prompt: Vlog Scene → Markcut Markdown
+
+> Fill every `{placeholder}`, then execute in your own context.
+> Run once per scene in the outline. Constraints mirror TEMPLATE.md §2/§3.
+
+---
+
+You are writing one scene of a {style} vlog. The scene will be rendered with markcut markdown syntax.
+
+Theme: {theme}
+Creator: {creator_profile}
+Language: {language}
+
+Scene from outline:
+
+{outline_scene}
+
+All clips are in the media directory `{media_dir}`. Use filenames as `src:` values relative to this directory.
+
+**A. Media selection:**
+- 1-4 clips for this scene. Use imaging for photos, videos for clips. See `startFrom`/`endAt` for trimming.
+- Image: `- image src:{filename} duration:{s} start:{s}`
+- Video: `- video src:{filename} startFrom:{trim_start} endAt:{trim_end} start:{s}`
+- First clip starts at `start:0`. Subsequent clips start at the cumulative previous duration.
+
+**B. Narration:**
+- One `subtitle script:"..."` node per scene. Duration matches total scene length.
+- First-person, conversational, matching creator profile.
+- Every word grounded in clip captions, user labels, or GPS. No fabrication.
+- Length: 10-40 words (~3-15 seconds at 2.5 w/s).
+- No parentheses, no ellipses. At most one emoji.
+
+Output format for ONE scene:
+
+```
+## <SceneTitle (PascalCase)>
+layout:parallel
+- image src:<filename> duration:<s> start:<s>
+- video src:<filename> startFrom:<s> endAt:<s> start:<s>
+- subtitle script:"<narration line>" duration:<total_s>
+```

package/src/config.mjs

@@ -75,14 +75,22 @@ export const DEFAULT_TTS_CLI =
 
 /**
  * Default agent CLI — general-purpose text LLM.
- * Used for text-only tasks like detect-scenes and editing.
+ * Used for text-only tasks like detect-scenes.
+ * Placeholder:
+ *   {prompt} = user prompt (current context + edit request)
+ * Override with MARKCUT_AGENT_CLI env var.
+ */
+export const DEFAULT_AGENT_CLI = process.env.MARKCUT_AGENT_CLI || 'npx pi -p {prompt}';
+
+/**
+ * Default edit agent CLI — used by the edit-mode persistent rpc session.
  * Placeholders:
  *   {systemprompt} = system prompt (role, instructions, knowledge)
  *   {prompt}       = user prompt (current context + edit request)
  *   {sessionid}    = unique session ID for conversation continuity
- * Override with MARKCUT_AGENT_CLI env var.
+ * Override with MARKCUT_EDIT_CLI env var.
  */
-export const DEFAULT_AGENT_CLI = process.env.MARKCUT_AGENT_CLI || 'npx pi --session-id {sessionid} --system-prompt {systemprompt} -p {prompt}';
+export const DEFAULT_EDIT_CLI = process.env.MARKCUT_EDIT_CLI || 'npx pi --session-id {sessionid} --system-prompt {systemprompt} -p {prompt}';
 
 // ── Render-only pipeline CLI templates ─────────────────────────────────────
 // These are specific to the render pipeline (no vision pipeline equivalent).