@lalalic/markcut 2.8.0 → 2.9.0

package/README.md

@@ -48,6 +48,7 @@ storyboard.md  ──[parse]──▶  DescriptiveRoot  ──[compile]──▶
 | **Built-in Mermaid** | `<Mermaid source="..." theme="dark" />` renders Mermaid diagrams as inline SVG. No imports needed |
 | **Styling** | Inline `style` strings on any node for CSS. JSX components use inline React styles |
 | **Live edit** | `--edit` watches the input file, re-runs pipeline, auto-reloads player |
+| **Player language** | Player UI supports English/Chinese. Use `?lang=en` or `?lang=zh` |
 | **Storyboard** | `--storyboard` fast structure preview: replaces TTI/TTV prompts with placeholder components, skips slow generation. Implies `--edit` |
 | **Label mode** | `--label` interactive player with per-scene label input, saves to labels.json |
 | **CLI** | `render`, `preview` commands for MP4 export and Remotion Studio |
@@ -159,6 +160,34 @@ Compound variant labels (like `zh-tiktok`) are split on `-` to form a **variant
 
 The browser player reads `window.VARIANT` from the URL path and fetches the correct compiled data via `/api/video-data?variant=<name>`. A **variant switcher bar** at the top of the player lets you jump between variants instantly.
 
+### Player UI Language (EN / ZH)
+
+The player UI (edit panel, placeholders, tooltips) supports English and Chinese.
+
+- `?lang=en` for English
+- `?lang=zh` for Chinese
+
+Examples:
+
+- `http://localhost:3001?lang=en`
+- `http://localhost:3001?lang=zh`
+
+If `lang` is not provided, the player falls back to browser language (Chinese browsers default to `zh`, otherwise `en`).
+
+### Map Language (EN / ZH)
+
+`map` nodes support `language`/`lang` and `region` keys.
+
+```md
+- map duration:3 lang:zh region:CN waypoints:[31.23,121.47,"上海";39.90,116.40,"北京"]
+- map duration:3 lang:en region:US waypoints:[37.77,-122.41,"SF";34.05,-118.24,"LA"]
+```
+
+Locale shorthand normalization:
+
+- `lang:zh` / `language:zh` → `zh-CN` (default region `CN`)
+- `lang:en` / `language:en` → `en` (default region `US`)
+
 ## `.markcut/` Directory Layout
 
 When you run `preview` or `render`, all generated artifacts live under `.markcut/` next to the source file. With multiple variants, each gets its own subdirectory:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lalalic/markcut",
-  "version": "2.8.0",
+  "version": "2.9.0",
   "description": "Markdown-to-video engine. Describe scenes in markdown, get a rendered video.",
   "bin": {
     "markcut": "bin/markcut"

package/skills/markcut/SKILL.md

@@ -80,52 +80,15 @@ some common issues (photo or video can't be displayed, audio missing), take belo
 | Topic | File |
 |-------|------|
 | Markdown descriptive format (primary authoring format) | [docs/markdown-descriptive.md](docs/markdown-descriptive.md) |
+| Built-in components & common npm packages | [docs/components.md](docs/components.md) |
+| Sound effects | [docs/sound-effects.md](docs/sound-effects.md) |
 
 
-## Built-in Components (no imports needed)
 
-These components are available directly in `jsx:".."` fields or importable from the `~~~js imports` block via `@lalalic/markcut/components`:
+## Built-in Components
 
-### `<Markdown />` — render markdown content
-```md
-- component jsx:"<Markdown source='# Hello\n\n**bold** text.' />"
-```
-- Uses `react-markdown` + `remark-gfm` (tables, strikethrough, task lists)
-- Supports `plugins` and `components` props: define custom renderers in imports block and pass them in
-- ` ```mermaid ` code fences inside markdown are automatically rendered as Mermaid diagrams
-- Importable: `import {Markdown} from "@lalalic/markcut/components"`
-
-### `<Mermaid />` — render Mermaid diagrams as SVG
-```md
-- component jsx:"<Mermaid source='graph TD; A-->B; A-->C; B-->D;' />"
-```
-- Theme prop: `theme="default" | "dark" | "forest" | "neutral"` (default: `dark`)
-- Uses `delayRender`/`continueRender` for async rendering — diagram is ready before Remotion captures the frame
-- Errors shown inline in the output
-- Importable: `import {Mermaid} from "@lalalic/markcut/components"`
-
-### Wrapping built-ins in custom components
-```js
-import {Markdown, Mermaid} from "@lalalic/markcut/components"
-
-export function SuperMarkdown({ source }) {
-  return (
-    <Markdown
-      source={source}
-      components={{
-        li: ({children}) => <li style={{color:'#ffd700'}}>{children}</li>,
-      }}
-    />
-  )
-}
-```
+Built-in components available via `@lalalic/markcut/components`. See [docs/components.md](docs/components.md) for full reference.
 
-## Common npm packages (used via imports block)
-- `react-markdown` + `remark-gfm` — already bundled as built-in `<Markdown />` above
-- `remark-toc` — generate table of contents
-- `remark-math` + `rehype-katex` — render math formulas with KaTeX
-- `@remotion/shapes` — render shapes like arrows, circles, rectangles, etc
-- `@remotion/starburst` — render starburst animations
 
 ## Golden rule
 - always check stream start and duration to avoid
@@ -133,8 +96,12 @@ export function SuperMarkdown({ source }) {
   - video cut off
   - subtitle mismatch
   - sync issues between audio, video, and subtitles
-- don't set duration for script or stream's duration depending on audio script
-  - markcut resolver will automatically calculate the duration based on the audio script length
-- **don't** rm `.markcut` directory, which served as cache for all generated content. cache will auto update according to the content change. rm `.markcut` will cause all content to be regenerated, which is time consuming and wasteful.
+according to the content change. rm `.markcut` will cause all content to be regenerated, which is time consuming and wasteful.
 - put all manual assets in `assets` folder, such as bgm, logo, watermark, etc. don't put them in `.markcut` folder, which is auto generated and will be deleted when `markcut clean` command is run.
-- `npx @lalalic/markcut preview` stuck until user close the preview window.
+
+### Don'ts
+- **don't** set duration for script or stream's duration depending on audio script
+  - markcut resolver will automatically calculate the duration based on the audio script length
+- **don't** rm `.markcut` directory, which served as cache for all generated content. cache will auto update 
+- **don't** set timeout for `preview`, `vision`, `render` markcut commands, which may take long time to generate medias.
+- **don't** use skill to understand vision media. use `npx @lalalic/markcut vision <folder>`.

package/skills/markcut/docs/components.md

@@ -0,0 +1,46 @@
+## Built-in Components (no imports needed)
+
+These components are available directly in `jsx:".."` fields or importable from the `~~~js imports` block via `@lalalic/markcut/components`:
+
+### `<Markdown />` — render markdown content
+```md
+- component jsx:"<Markdown source='# Hello\n\n**bold** text.' />"
+```
+- Uses `react-markdown` + `remark-gfm` (tables, strikethrough, task lists)
+- Supports `plugins` and `components` props: define custom renderers in imports block and pass them in
+- ` ```mermaid ` code fences inside markdown are automatically rendered as Mermaid diagrams
+- Importable: `import {Markdown} from "@lalalic/markcut/components"`
+
+### `<Mermaid />` — render Mermaid diagrams as SVG
+```md
+- component jsx:"<Mermaid source='graph TD; A-->B; A-->C; B-->D;' />"
+```
+- Theme prop: `theme="default" | "dark" | "forest" | "neutral"` (default: `dark`)
+- Uses `delayRender`/`continueRender` for async rendering — diagram is ready before Remotion captures the frame
+- Errors shown inline in the output
+- Importable: `import {Mermaid} from "@lalalic/markcut/components"`
+
+### Wrapping built-ins in custom components
+```js
+import {Markdown, Mermaid} from "@lalalic/markcut/components"
+
+export function SuperMarkdown({ source }) {
+  return (
+    <Markdown
+      source={source}
+      components={{
+        li: ({children}) => <li style={{color:'#ffd700'}}>{children}</li>,
+      }}
+    />
+  )
+}
+```
+
+## Common npm packages (used via imports block)
+- `react-markdown` + `remark-gfm` — already bundled as built-in `<Markdown />` above
+    - `remark-toc` — generate table of contents
+    - `remark-math` + `rehype-katex` — render math formulas with KaTeX
+- `@remotion/shapes` — render shapes like arrows, circles, rectangles, etc
+- `@remotion/starburst` — render starburst animations
+- `react-webcam-pro` — render webcam video
+- `react-chartjs-2` — render charts with Chart.js at https://react-chartjs-2.js.org/components

package/skills/markcut/docs/markdown-descriptive.md

@@ -331,6 +331,9 @@ Subtitles are configured at the root level as a VTT overlay. Set via `subtitle:`
 | `zoom` | int (default 10) | map |
 | `center` | `{lat:n,lng:n}` JSON | map |
 | `mapType` | `roadmap\|satellite\|hybrid\|terrain` | map |
+| `language` | map label/UI language. Supports `en`, `zh`, `zh-CN`, etc. | map |
+| `lang` | alias of `language` (e.g. `lang:zh`) | map |
+| `region` | region code hint (e.g. `CN`, `US`) | map |
 | `routeMarker` | emoji string e.g. `"🚗"` | map |
 | `title` | display title | scene |
 | `instruction` | visual intent / style / any prompt; NOT rendered | any |
@@ -433,6 +436,15 @@ When: animated route. Required: `duration`, `waypoints`.
 
 `- map duration:3 travelMode:DRIVING waypoints:[37.77,-122.41,"SF";34.05,-118.24,"LA"]`
 
+Language and region are optional:
+
+`- map duration:3 language:zh region:CN waypoints:[31.23,121.47,"上海";39.90,116.40,"北京"]`
+
+Shorthand aliases:
+
+- `language:zh` or `lang:zh` → normalized to `zh-CN` with `region:CN` default
+- `language:en` or `lang:en` → keeps English labels with `region:US` default
+
 ### `include`
 
 When: embed an external markdown file as a sub-video. The sub-video is

package/skills/markcut/docs/sound-effects.md

@@ -0,0 +1,45 @@
+# 🎧 Remotion Sound Effects
+
+## 1. URL Construction
+
+All sound effect URLs follow this pattern:
+
+`https://remotion.media/<name>.wav`
+
+such as `https://remotion.media/sanctuary-guardian-what.wav`
+---
+
+## 2. Available Names
+
+whip  
+whoosh  
+page-turn  
+switch  
+mouse-click  
+shutter-modern  
+shutter-old  
+ding  
+bruh  
+vine-boom  
+windows-xp-error  
+fah  
+spongebob-fail  
+omg-hell-nah  
+price-is-right-fail  
+romance-meme  
+bone-crack  
+anime-wow  
+yippee  
+loading-lag  
+wilhelm-scream  
+mac-quack  
+skedaddle  
+snapchat-notification  
+nelly-ahh  
+sanctuary-guardian-what  
+minecraft-hurt  
+oh-my-god-vine  
+illuminati-confirmed  
+dramatic-boomer  
+triggered  
+record-scratch  

package/src/config.mjs

@@ -85,9 +85,9 @@ export const DEFAULT_VTT_SAMPLE_INTERVAL = Number(process.env.MARKCUT_VTT_SAMPLE
 
 
 /** Speech-to-text CLI. Override via --stt flag or MARKCUT_STT_CLI env var. */
-export const DEFAULT_STT_CLI = args.cliOverrides.stt || process.env.MARKCUT_STT_CLI || 'uvx --from openai-whisper whisper "{input}" --output_format vtt --output_dir "{output}"';
+export const DEFAULT_STT_CLI = args.cliOverrides.stt || process.env.MARKCUT_STT_CLI || 'uvx --from openai-whisper whisper "{input}" --output_format vtt --output_dir "{output}" --word_timestamps True --max_line_count 1 --max_line_width 14';
 /** Text-to-speech CLI. Override via --tts flag or MARKCUT_TTS_CLI env var. */
-export const DEFAULT_TTS_CLI = args.cliOverrides.tts || process.env.MARKCUT_TTS_CLI || 'uvx edge-tts --voice "en-US-GuyNeural" --text "{input}" --write-media "{output}"';
+export const DEFAULT_TTS_CLI = args.cliOverrides.tts || process.env.MARKCUT_TTS_CLI || 'uvx edge-tts --voice "zh-CN-YunxiNeural" --text "{input}" --write-media "{output}"';
 /** Default agent CLI. Override via --agent flag or MARKCUT_AGENT_CLI env var. */
 export const DEFAULT_AGENT_CLI = args.cliOverrides.agent || process.env.MARKCUT_AGENT_CLI || 'npx pi -p {prompt}';
 /** Default edit agent CLI. Override via --edit-cli flag or MARKCUT_EDIT_CLI env var. */

package/src/descriptive/compiler.ts

@@ -147,6 +147,8 @@ export interface DescriptiveMap extends DescriptiveBaseNode {
   zoom?: number;
   center?: { lat: number; lng: number };
   mapType?: "roadmap" | "satellite" | "hybrid" | "terrain";
+  language?: string;
+  region?: string;
   travelMode?: "DRIVING" | "WALKING" | "BICYCLING" | "TRANSIT";
   routeMarker?: string;
 }
@@ -593,6 +595,8 @@ function compileLeaf(node: Exclude<DescriptiveNode, DescriptiveContainer | Descr
         zoom: node.zoom ?? 10,
         center: node.center,
         mapType: node.mapType ?? "roadmap",
+        language: node.language,
+        region: node.region,
         travelMode: node.travelMode ?? "DRIVING",
         routeMarker: node.routeMarker ?? "🚗",
         googleMapsApiKey: ctx.googleMapsApiKey,

package/src/descriptive/markdown.ts

@@ -327,6 +327,8 @@ function parseNodeLine(content: string, lineNum?: number): DescriptiveNode {
         zoom: attrs.zoom as any,
         center: attrs.center as any,
         mapType: attrs.mapType as any,
+        language: (attrs.language as any) ?? (attrs.lang as any),
+        region: attrs.region as any,
         instruction: attrs.instruction as any,
         visible: attrs.visible as any,
         isBackground: attrs.isBackground as any,

package/src/player/browser.tsx

@@ -14,7 +14,8 @@ import * as ReactDOM from "react-dom";
 import * as Remotion from "remotion";
 import { Player } from "@remotion/player";
 import { MarkCut, getDurationInSeconds } from "../entry";
-import { HeaderBar, EditControls, LabelControls, SceneThumbnails, VariantBar } from "./components/index";
+import { HeaderBar, EditControls, EditMessagePanel, LabelControls, SceneThumbnails, VariantBar } from "./components/index";
+import type { EditEntry } from "./components/EditMessagePanel";
 import * as MarkcutComponents from "../components/index";
 
 /**
@@ -161,6 +162,13 @@ function PlayerApp() {
   const urlParams = new URLSearchParams(typeof window !== "undefined" ? window.location.search : "");
   const autoPlay = urlParams.get("autoplay") === "true";
   const startAt = parseFloat(urlParams.get("start") || urlParams.get("t") || "0") || 0;
+  const locale: "en" | "zh" = (() => {
+    const raw = (urlParams.get("lang") || "").toLowerCase();
+    if (raw === "zh" || raw.startsWith("zh-")) return "zh";
+    if (raw === "en" || raw.startsWith("en-")) return "en";
+    if (typeof navigator !== "undefined" && navigator.language.toLowerCase().startsWith("zh")) return "zh";
+    return "en";
+  })();
 
   // Derive player config from data early so effects can reference them safely.
   const fps = data?.fps ?? 30;
@@ -389,10 +397,30 @@ function PlayerApp() {
     (typeof window !== "undefined" ? (window as any).MODE : null) || "preview";
 
   // Shared state for header info
-  const [editStatus, setEditStatus] = React.useState("");
   const [sseConnected, setSseConnected] = React.useState(false);
   const [labelSceneInfo, setLabelSceneInfo] = React.useState("");
 
+  // ── Edit message panel state ─────────────────────────────────────────
+  const [editEntries, setEditEntries] = React.useState<EditEntry[]>([]);
+  const [showEditOverlay, setShowEditOverlay] = React.useState(false);
+  const nextEditIdRef = React.useRef(1);
+  const hideEditOverlayTimerRef = React.useRef<number | null>(null);
+
+  const clearHideOverlayTimer = React.useCallback(() => {
+    if (hideEditOverlayTimerRef.current !== null) {
+      window.clearTimeout(hideEditOverlayTimerRef.current);
+      hideEditOverlayTimerRef.current = null;
+    }
+  }, []);
+
+  const scheduleHideEditOverlay = React.useCallback(() => {
+    clearHideOverlayTimer();
+    hideEditOverlayTimerRef.current = window.setTimeout(() => {
+      setShowEditOverlay(false);
+      hideEditOverlayTimerRef.current = null;
+    }, 5000);
+  }, [clearHideOverlayTimer]);
+
   // SSE connection — shared across all modes as a server-liveness monitor
   // Edit mode also listens for "reload" messages (auto-refresh on file change)
   const suppressReloadRef = React.useRef(false);
@@ -404,8 +432,57 @@ function PlayerApp() {
       evtSource.onmessage = (e: MessageEvent) => {
         try {
           const msg = JSON.parse(e.data);
+
+          // Reload event (file changed on disk or agent finished editing)
           if (msg.type === "reload" && !suppressReloadRef.current) {
             window.dispatchEvent(new Event("refresh-player"));
+            return;
+          }
+
+          // Edit progress events (live from agent RPC)
+          if (msg.type === "edit:start") {
+            clearHideOverlayTimer();
+            setShowEditOverlay(true);
+            const id = nextEditIdRef.current++;
+            setEditEntries((prev) => [
+              ...prev,
+              { id, request: msg.request || "", progress: "", status: "thinking" },
+            ]);
+            return;
+          }
+          if (msg.type === "edit:progress") {
+            setEditEntries((prev) => {
+              const last = prev[prev.length - 1];
+              if (!last || last.status !== "thinking") return prev;
+              return prev.map((e) =>
+                e.id === last.id ? { ...e, progress: msg.text || "" } : e
+              );
+            });
+            return;
+          }
+          if (msg.type === "edit:done") {
+            setEditEntries((prev) => {
+              const last = prev[prev.length - 1];
+              if (!last) return prev;
+              return prev.map((e) =>
+                e.id === last.id
+                  ? { ...e, status: "done", progress: e.progress || msg.summary || "" }
+                  : e
+              );
+            });
+            scheduleHideEditOverlay();
+            return;
+          }
+          if (msg.type === "edit:error") {
+            setEditEntries((prev) => {
+              const last = prev[prev.length - 1];
+              if (!last) return prev;
+              return prev.map((e) =>
+                e.id === last.id ? { ...e, status: "error", error: msg.error } : e
+              );
+            });
+            scheduleHideEditOverlay();
+            return;
           }
         } catch {}
       };
@@ -414,10 +491,11 @@ function PlayerApp() {
       setSseConnected(false);
     }
     return () => {
+      clearHideOverlayTimer();
       evtSource?.close();
       setSseConnected(false);
     };
-  }, []);
+  }, [clearHideOverlayTimer, scheduleHideEditOverlay]);
 
   if (error) {
     return <div style={{ color: "red", padding: 40, fontFamily: "sans-serif" }}>Error: {error}</div>;
@@ -440,9 +518,9 @@ function PlayerApp() {
       {/* ── Header (close button + mode info) ── */}
       <HeaderBar
         mode={mode}
-        editStatus={editStatus}
         sseConnected={sseConnected}
         sceneInfo={mode === "label" ? labelSceneInfo : undefined}
+        locale={locale}
       />
 
       {/* ── Variant switcher ── */}
@@ -466,6 +544,18 @@ function PlayerApp() {
           doubleClickToFullscreen={true}
           autoPlay={autoPlay}
         />
+
+        {/* ── Edit message overlay (shown while waiting, then auto-hides) ── */}
+        {mode === "edit" && showEditOverlay && (
+          <div id="edit-message-overlay">
+            <EditMessagePanel
+              entries={editEntries}
+              minimized={false}
+              locale={locale}
+              onToggleMinimize={() => setShowEditOverlay(false)}
+            />
+          </div>
+        )}
       </div>
 
       {/* ── Scene thumbnails (shared across all modes) ── */}
@@ -483,10 +573,10 @@ function PlayerApp() {
       {/* ── Mode-specific controls ── */}
       {mode === "edit" && (
         <EditControls
-          onStatusChange={setEditStatus}
           suppressReloadRef={suppressReloadRef}
           currentTime={currentTime}
           activeScene={activeScene}
+          locale={locale}
         />
       )}
       {mode === "label" && (