npm - @lalalic/markcut - Versions diffs - 2.2.9 → 2.3.0 - Mend

@lalalic/markcut 2.2.9 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (4) hide show

package/package.json +1 -1
package/skills/markcut/SKILL.md +11 -7
package/skills/markcut/docs/edit-mode.md +0 -220
package/skills/markcut/docs/label-mode.md +0 -112

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "@lalalic/markcut",
-  "version": "2.2.9",
+  "version": "2.3.0",
   "description": "Markdown-to-video engine. Describe scenes in markdown, get a rendered video.",
   "bin": {
     "markcut": "bin/markcut"

package/skills/markcut/SKILL.md CHANGED Viewed

@@ -23,11 +23,6 @@ see [docs/markdown-descriptive.md](docs/markdown-descriptive.md) for full detail
 ### Video Variants
 - define variants for different video configurations, such as different languages, aspects, platforms
-### Templates
-- Templates in ``docs/templates/`` provide ready-to-use video structures for `orchestrator` agents, use them as starting point to create your own videos.
 ### Viral Story requires
 - **Hooks** : why should the viewer watch this?
 - **Conflict** : what challenges or obstacles do the characters face?
@@ -36,11 +31,22 @@ see [docs/markdown-descriptive.md](docs/markdown-descriptive.md) for full detail
 - **Call to action** : what should the viewer do next?
 - **Open ending** : does the story leave room for interpretation or continuation?
+### TTS, TTI, TTV, STT media generation
+markcut implements a **media generation pipeline** to generate TTS, TTI, TTV, STT media automatically.
+- `npx @lalalic/markcut --show-clis` to see the default CLIs for TTS, TTI, TTV, STT media generation.
+- `orchestrator` agent DONT generate media directly
 ## 3. CLI
 ```bash
 npx @lalalic/markcut <command> [options]
 npx @lalalic/markcut --help # get overall information
+npx @lalalic/markcut --show-clis # get command specific information
+npx @lalalic/markcut preview <file> # assemble and preview the video with a local server, and chat to edit the video and auto refresh
+npx @lalalic/markcut render <file> # render the video to mp4
+npx @lalalic/markcut vision <folder> # vision understanding medias in folder
+npx @lalalic/markcut vision <folder> --label # full pipeline vision understanding medias in folder, and provide ui to label the medias with text, time ranges
 ```
 ---
@@ -60,8 +66,6 @@ some common issues (photo or video can't be displayed, audio missing), take belo
 | Topic | File |
 |-------|------|
 | Markdown descriptive format (primary authoring format) | [docs/markdown-descriptive.md](docs/markdown-descriptive.md) |
-| Label system (browse, label, export labels.json) | [docs/label-mode.md](docs/label-mode.md) |
-| Player servers (label + edit mode) | [docs/edit-mode.md](docs/edit-mode.md) |
 ## common used components

package/skills/markcut/docs/edit-mode.md DELETED Viewed

@@ -1,220 +0,0 @@
-# Watch Mode
-> `preview <file.md> --edit` — live editing loop with AI-assisted JSON editing.
-Edit mode gives you a browser player that **automatically reloads** when the source file changes, plus an **edit input** that calls an AI coding agent (pi) to make changes on your behalf.
-Generated artifacts (TTS audio, compiled includes, component bundles) are
-stored under `.markcut/` next to the source file — see [SKILL.md](../SKILL.md) for the full layout.
-## Quickstart
-```bash
-cd remotion-engine
-npm run preview -- sample-visual.json --edit
-# → Browser opens, terminal blocks until you close it
-```
-## Architecture
-```
-┌─────────────┐     HTTP      ┌──────────────────┐    spawn     ┌──────────┐
-│   Browser   │◄──────────────►│  player/server   │◄────────────►│  pi CLI  │
-│  (Chrome)   │     SSE        │   .mjs (Node)    │  one-shot    │ (agent)  │
-└──────┬──────┘                └────────┬─────────┘              └──────────┘
-       │                                │
-       │  POST /api/edit                │  fs.watchFile
-       │  {text: "make text bigger"}    │  (poll every 500ms)
-       │                                │
-       │  GET /api/video-data           ▼
-       │  ← stream tree JSON    ┌──────────────┐
-       │                        │ stream tree  │
-       │  EventSource           │ .json (file) │
-       │  /api/events           └──────────────┘
-       │  ← {type:"reload"}           ▲
-       │                              │
-       │  window.dispatchEvent        │ pi edits
-       │  ("refresh-player")          │ this file
-       │                              │
-```
-## File: `src/player/server.mjs`
-### Server startup (`cli.mjs` → `server.mjs`)
-```
-node src/render/cli.mjs preview file.json --edit --port 3031
-```
-1. `cli.mjs` spawns `server.mjs` with `--edit` flag
-2. `server.mjs` parses `file.json` to build a recursive tree description
-3. Starts `fs.watchFile(file.json, {interval:500})` — polls every 500ms for changes
-4. Starts HTTP server on specified port
-5. Auto-opens browser via `open http://localhost:PORT`
-### Page served
-The HTML page loads two scripts:
-| Script | Source | Role |
-|--------|--------|------|
-| `player.js` | Bundled esbuild output from `src/player/browser.tsx` | Renders the stream tree using `@remotion/player` with `MarkCut` |
-| Inline script | Generated by `getHtml()` in `server.mjs` | Watch-mode UI controls (close button, edit input, SSE) |
-### Edit flow
-```
-Browser                          Server                          pi CLI
-  │                                │                              │
-  │── POST /api/edit ────────────► │                              │
-  │   {text: "make text bigger"}   │                              │
-  │                                │── spawn("pi", ["-p", prompt])│
-  │                                │   prompt includes:           │
-  │                                │   • full tree description   │
-  │                                │     (recursive, any depth)  │
-  │                                │   • edit history             │
-  │                                │   • knowledge (all stream   │
-  │                                │     types, components,      │
-  │                                │     schema, styling guide)  │
-  │                                │   • edit request             │
-  │                                │                              │
-  │                                │   pi can edit ANY field on  │
-  │                                │   ANY node in the tree      │
-  │                                │                              ├── edits file.json
-  │                                │                              └── exits
-  │                                │◄── exit code 0              │
-  │◄── {done:true, output:"..."} ──┤                              │
-  │                                │                              │
-  │ suppressReload = true          │                              │
-  │ header shows "Changed text..." │                              │
-  │                                │                              │
-  │  (fs.watchFile fires SSE       │                              │
-  │   but suppressReload=true      │                              │
-  │   so it's ignored)             │                              │
-  │                                │                              │
-  │ after 4s:                      │                              │
-  │ dispatchEvent("refresh-player")│                              │
-  │ → player re-fetches /api/data  │                              │
-  │ → re-renders with new JSON     │                              │
-```
-### Key design decisions
-**Why in-place refresh instead of `location.reload()`?**
-- Timeline playback continues uninterrupted
-- Status text persists until next edit
-- No page flash
-**Why `suppressReload`?**
-- SSE fires ~500ms after pi edits the file
-- API response arrives at the same time
-- `suppressReload` flag ignores the SSE event during edit
-- The client reloads itself after showing the summary
-## File: `src/player/browser.tsx`
-The bundled player app:
-1. Fetches `/api/video-data` on mount
-2. Renders the stream tree using `@remotion/player` with `MarkCut`
-3. Listens for `"refresh-player"` custom DOM event
-4. On refresh event: re-fetches data, re-renders in-place
-## File: `src/render/cli.mjs`
-The CLI routes `--edit` flag to the player server:
-```javascript
-if (args.edit) {
-  const serverArgs = ["src/player/server.mjs", file, "--edit", `--port=${port}`];
-  const child = spawn("node", serverArgs, { cwd: ROOT, stdio: "inherit" });
-  // ...blocks until server exits...
-}
-```
-## API Endpoints
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/api/video-data` | GET | Returns the raw stream tree JSON file |
-| `/api/scenes` | GET | Returns parsed scenes array with timing |
-| `/api/video-info` | GET | Returns scenes + totalDuration + mode flags |
-| `/api/edit` | POST | Accepts `{text}` → spawns pi to edit the JSON → file watcher triggers → SSE reload |
-| `/api/shutdown` | POST | Kills the server process, unblocking the agent |
-| `/api/events` | GET | SSE stream — sends `{type:"reload"}` when file changes |
-| `/api/labels` | GET/POST | (label mode only) Save/load scene labels |
-## The pi prompt
-Each edit call sends a structured prompt with a **recursive tree description** instead of a flat scene list:
-```
-You are editing sample-visual.json, a Remotion stream tree JSON.
-The stream tree (indentation shows nesting; timing in seconds):
-  root "root" (1080×1920, 30fps, series, transition:fade)
-    folder "scene-1" (parallel)
-      component GradientBackground {type:"radial",animated:true,noise:true} [0ÔåÆ5s, bg]
-      component AnimatedHeadline {"text":"Hello Remotion",...} [0.5ÔåÆ4.5s]
-    folder "scene-2" (parallel)
-      component GradientBackground {type:"linear",animated:true} [0ÔåÆ5s, bg]
-      subtitle "This text is styled with CSS" fontSize:52 [0.5ÔåÆ4.5s]
-    ...
-Previous edits:
-1. make scene 1 headline text orange
-Edit request: make scene 1 headline text bigger
---- Knowledge ---
-You can edit ANY field on ANY node in the JSON...
-Stream types: root, folder, video, audio, image, subtitle, component, effect, rhythm, map
-...
-IMPORTANT: Read the full existing JSON file before editing. Only edit the JSON file. You can change, add, or remove any field on any node. Output ONLY a one-line summary of what specific change you made.
-```
-The tree description walks the JSON recursively, showing:
-- **Nesting depth** via indentation
-- **Type** and **id/name** of each node
-- **Key fields** (dimensions, fps, theme, series/parallel, transitions)
-- **Timing** from leaf span (`[start→end]`)
-- **isBackground** flag (`[bg]`)
-- **Component props** and node-specific fields (fontSize, fit, style, volume, etc.)
-This means pi can edit **any aspect of the JSON** ÔÇö not just scene props. Nested folders, effects, rhythms, maps, actions, styles, themes, global stylesheets, audio config ÔÇö everything is visible and editable.
-## Session context
-The server maintains `editHistory[]` — every edit is appended. On each call, previous edits are included so pi knows what was done before:
-```
-Previous edits:
-1. make scene 1 headline text orange
-2. change background from radial to linear gradient
-3. add 3D angle to device mockup in scene 4
-```
-This gives pi enough context to make coherent changes across multiple edit rounds.
-## Close / tab-close
-| Action | What happens |
-|--------|-------------|
-| Click **✕** | `sendBeacon("/api/shutdown")` → server exits → agent unblocks |
-| Close **tab** | SSE connection drops → 3s grace → server exits |
-| Close **browser** | Same as tab close |
-## Label mode (`--label`)
-`preview <file.json> --label` starts the same player but with a **label input** instead of the edit input. Labels are saved to `labels.json` next to the source file. No AI editing, no file watching.
-## Build
-The browser bundle is built with esbuild:
-```bash
-bash scripts/build-player.sh
-# → src/player/bundle/player.js
-```
-The bundle includes React, @remotion/player, MarkCut, all 13 built-in components, themes, templates, and the duration utility. It is served as a static file by the player server.

package/skills/markcut/docs/label-mode.md DELETED Viewed

@@ -1,112 +0,0 @@
-# Label System
-Interactive media labeling before understanding clips in json stream tree.
-The stream tree follow rules, see [docs/json-descriptive.md](docs/json-descriptive.md) for full details .
-## Overview
-The label player lets you load a stream tree, browse its clips, and attach descriptive labels to each clip's media.
-The output is an updated `labels.json` file with `description` fields populated.
-## Usage
-```bash
-# Label from a stream tree JSON or labels.json
-npx markcut preview labels.json --label
-# Specify port
-npx markcut preview labels.json --label --port 3031
-```
-### Labels Persistence
-The stream tree follow rules, see [docs/json-descriptive.md](docs/json-descriptive.md) for full details .
-Labels are saved to the source file.
-```json
-{
-  "root": {
-    "id": "root",
-    "type": "root",
-    "children": [
-      {
-        "id": "scene-1",
-        "type": "folder",
-        "children": [
-          {
-            "id": "scene-1-media",
-            "type": "image",
-            "src": "/media/photo.jpg",
-            "start": 0, "end": 5,
-            "description": "Best group shot from the campfire"
-          }
-        ]
-      },
-      {
-        "id": "scene-2",
-        "type": "folder",
-        "children": [
-          {
-            "id": "scene-2-media",
-            "type": "video",
-            "src": "/media/clip.mp4",
-            "start": 0, "end": 5,
-            "description": "Sunset timelapse — use for intro"
-          }
-        ]
-      }
-    ]
-  }
-}
-```
-The `description` field on each media leaf node holds the label text.
-## Workflow
-```
-Stream tree JSON (labels.json)
-       │
-       ▼
-─── preview --label ───→ Browse & label clips
-       │                        │
-       │                        ▼
-       │                  Updated labels.json
-       │                        │
-       ▼                        ▼
-    Use in Storyboard ──────→ Assemble → Render
-```
-1. **Prepare** a stream tree JSON with scenes
-2. **Label** with `preview --label` to add descriptions to each scene
-3. **Export** labels are saved alongside the source file
-4. **Use** the labeled scenes in your storyboard or assembly workflow
-## API Endpoints
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/api/video-data` | GET | Returns the raw stream tree JSON |
-| `/api/video-info` | GET | Returns scenes array with timing + totalDuration |
-| `/api/labels` | GET | Returns the current stream tree with descriptions |
-| `/api/labels` | POST | Saves updated descriptions ({descriptions: {index: "text"}}) |
-| `/api/shutdown` | POST | Kills the server |
-| `/api/events` | GET | SSE events (reload on file change) |
-## Implementation
-Source: `src/player/server.mjs` (unified server, use `--label` flag)
-UI controls: `src/player/components/LabelControls.tsx` (React component)
-- Built on Node.js `http` module (no Express dependency)
-- Serves bundled `player.js` (esbuild output from `src/player/browser.tsx`)
-- Labels saved atomically to disk on each POST
-- Supports stream tree JSON, descriptive JSON, and markdown input
-- React components in `src/player/components/` replace the old HTML-based `ui/` directory
-## Reference
-| Topic | Link |
-|-------|------|
-| Storyboard (video planning) | [storyboard.md](storyboard.md) |
-| Stream tree types | `SKILL.md` |
-| Dynamic components | [dynamic-components.md](dynamic-components.md) |