@program-video/cli 0.1.11 → 0.2.1

package/AGENTS.md

@@ -1,132 +1,252 @@
-# Program CLI - AI Agent Instructions
+# Program CLI - AI Agent Guide
 
-> CLI for Program.video - create and manage AI-generated videos programmatically.
+> Local-only CLI for the Program desktop app. All operations go through the local Convex backend at `http://127.0.0.1:3210`. No cloud, no auth, no cost.
 
-## Quick Start
+## Prerequisites
 
-```bash
-# 1. Authenticate (opens browser, user enters code)
-npx @program-video/cli auth login
-
-# 2. Create a project
-npx @program-video/cli project create --title "My Video" --json
+The Program desktop app must be running. The CLI talks to its local backend.
 
-# 3. Execute tools to build the video
-npx @program-video/cli tool exec addScene --project <projectId> --params '{"title":"Intro","duration":5,"code":"export default () => <h1>Hello</h1>"}' --json
+```bash
+# Verify the backend is reachable
+program template list --json
 ```
 
-## Authentication
+If you get "Local backend not available", the desktop app isn't running.
 
-**Always check auth status first, then authenticate if needed.**
+## Tool Execution
+
+All composition and generation operations use the `tool exec` pattern:
 
 ```bash
-npx @program-video/cli auth status --json
+program tool exec <toolName> --project <compositionId> --params '<json>' --json
 ```
 
-If not authenticated, run login and **relay the code to the user**:
+### List available tools
 
 ```bash
-npx @program-video/cli auth login
+program tool list --json
 ```
 
-The command outputs a URL and code. **You MUST tell the user both:**
-- The URL to visit (e.g., `https://program.video/device`)
-- The code to enter (e.g., `ABCD-1234`)
+## Node Graph Editing
 
-Example: "Please visit https://program.video/device and enter code **ABCD-1234** to authorize."
+Video and image scenes can carry a first-class node graph in `scene.metadata.nodeGraph`.
+Use the dedicated `program node ...` commands for graph work instead of hand-editing the raw JSON.
 
-Wait for the command to complete - it confirms when authenticated. Credentials are stored in `~/.program/credentials.json`.
+```bash
+# Inspect a composition, then inspect the scene graph
+program tool exec assessComposition --project <id> --json
+program node list --project <id> --scene <sceneId> --json
+
+# Add nodes
+program node add prompt --project <id> --scene <sceneId> --config '{"prompt":"Describe the shot"}' --json
+program node add reference-image --project <id> --scene <sceneId> --config '{"imageUrl":"localasset://localhost/..."}' --json
+program node add reference-video --project <id> --scene <sceneId> --config '{"videoUrl":"localasset://localhost/..."}' --json
+program node add model-select --project <id> --scene <sceneId> --config '{"model":"alibaba/wan-v2.6-t2v","provider":"ai-gateway"}' --json
+program node add generate-video --project <id> --scene <sceneId> --json
+program node add generate-image --project <id> --scene <sceneId> --json
+program node add number-value --project <id> --scene <sceneId> --config '{"value":5}' --json
+
+# Update node config
+program node config <nodeId> --project <id> --scene <sceneId> --set '{"prompt":"Updated prompt"}' --json
+
+# Connect nodes by handle
+program node connect <promptNodeId> <referenceImageNodeId> --project <id> --scene <sceneId> --source-handle prompt --target-handle prompt --json
+program node connect <promptNodeId> <generateNodeId> --project <id> --scene <sceneId> --source-handle prompt --target-handle prompt --json
+program node connect <referenceImageNodeId> <generateNodeId> --project <id> --scene <sceneId> --source-handle imageUrl --target-handle imageUrl --json
+program node connect <referenceVideoNodeId> <sceneOutputNodeId> --project <id> --scene <sceneId> --source-handle videoUrl --target-handle media --json
+program node connect <generateVideoNodeId> <sceneOutputNodeId> --project <id> --scene <sceneId> --source-handle videoUrl --target-handle media --json
+program node connect <generateVideoNodeId> <sceneOutputNodeId> --project <id> --scene <sceneId> --source-handle duration --target-handle duration --json
+
+# Clean up / arrange / run
+program node disconnect <sourceNodeId> <targetNodeId> --project <id> --scene <sceneId> --json
+program node remove <nodeId> --project <id> --scene <sceneId> --json
+program node autolayout --project <id> --scene <sceneId> --json
+program node execute --project <id> --scene <sceneId> --json
+```
+
+Important graph conventions:
+- `scene-output` uses `media` as the canonical asset input for both image and video sources.
+- Prompt nodes, reference-image nodes, reference-video nodes, model-select nodes, and number-value nodes are legitimate graph primitives and should not be treated as legacy.
+- Prefer the node commands when the user wants to scaffold or edit generation pipelines for a scene.
 
-## Commands
+## Available Tools
 
-### Projects
+### Composition Tools
 
 ```bash
-# Create a new project
-npx @program-video/cli project create --title "Title" --json
+# Read full composition data
+program tool exec readComposition --project <id> --json
+
+# Get lightweight overview with scene summaries
+program tool exec assessComposition --project <id> --json
+
+# Add a code scene
+program tool exec addScene --project <id> --params '{
+  "title": "Intro",
+  "duration": 5,
+  "code": "export default () => <h1>Hello</h1>"
+}' --json
+
+# Add a video scene
+program tool exec addScene --project <id> --params '{
+  "type": "video_scene",
+  "title": "B-Roll",
+  "url": "https://example.com/video.mp4",
+  "duration": 5,
+  "start": 0,
+  "layer": 1
+}' --json
+
+# Add an audio scene
+program tool exec addScene --project <id> --params '{
+  "type": "audio_scene",
+  "title": "Narration",
+  "url": "/audio-cache/scene-trimmed.wav",
+  "duration": 10
+}' --json
+
+# Update scene code
+program tool exec updateScene --project <id> --params '{
+  "sceneId": "<scene-id>",
+  "code": "export default () => <h1>Updated</h1>"
+}' --json
+
+# Delete a scene
+program tool exec deleteScene --project <id> --params '{"sceneId": "<scene-id>"}' --json
+
+# Delete all scenes
+program tool exec deleteAllScenes --project <id> --json
+
+# Update composition metadata
+program tool exec setMetadata --project <id> --params '{
+  "title": "My Video",
+  "duration": 30,
+  "fps": 30,
+  "width": 1080,
+  "height": 1920
+}' --json
+```
 
-# List all projects
-npx @program-video/cli project list --json
+### Generation Tools
 
-# Get project details
-npx @program-video/cli project get <projectId> --json
+```bash
+# Generate speech (TTS via Replicate chatterbox-turbo)
+program tool exec generateSpeech --project <id> --params '{
+  "text": "Welcome to the show",
+  "voice": "Ethan"
+}' --json
+
+# Generate speech and attach to existing audio scene
+program tool exec generateSpeech --project <id> --params '{
+  "text": "Scene narration",
+  "voice": "Ethan",
+  "sceneId": "<existing-audio-scene-id>"
+}' --json
+
+# Transcribe audio (local whisper, word-level timestamps)
+program tool exec transcribeAudio --params '{
+  "filePath": "/tmp/program-video-cache/audio/scene-trimmed.wav",
+  "model": "base"
+}' --json
 ```
 
-### Tools (Video Composition)
+### Utility Tools
 
 ```bash
-# List available tools
-npx @program-video/cli tool list --json
-
-# Execute a tool
-npx @program-video/cli tool exec <toolName> --project <projectId> --params '<json>' --json
+# List all compositions
+program tool exec listCompositions --json
+program tool exec listCompositions --params '{"limit": 10}' --json
+
+# Trim silence from composition audio scenes
+program tool exec trimSilence --project <id> --json
+program tool exec trimSilence --project <id> --params '{
+  "sceneId": "<audio-scene-id>",
+  "silenceDuration": "0.25",
+  "threshold": "-30dB"
+}' --json
+
+# Build caption overlay from transcripts
+program tool exec buildCaptions --project <id> --json
+program tool exec buildCaptions --project <id> --params '{
+  "accentWords": "important,highlight",
+  "style": "tiktok"
+}' --json
+
+# Generate subtitles end to end (local transcription + captions scene)
+program tool exec generateSubtitles --project <id> --json
+program tool exec generateSubtitles --project <id> --params '{
+  "sceneId": "<audio-or-video-scene-id>",
+  "style": "tiktok",
+  "accentWords": "ai,program"
+}' --json
+
+# List available models
+program tool exec listSpeechModels --json
+program tool exec listVideoModels --json
+program tool exec listImageModels --json
 ```
 
-**Available tools:**
-- `addScene` - Add a scene with React/Remotion code
-- `updateScene` - Modify an existing scene
-- `deleteScene` - Remove a scene
-- `generateSpeech` - Generate text-to-speech audio
-- `readComposition` - Get current composition state
+## Templates
 
-### Messages
+Templates are reusable workflow definitions with ordered steps.
 
 ```bash
-# Send a message to the project chat (appears in web editor)
-npx @program-video/cli message send "Status update" --project <projectId> --json
-
-# List messages
-npx @program-video/cli message list --project <projectId> --json
+program template list --json
+program template use <id>
+program template get <id> --json
+program template create --name "My Workflow" --steps '[]' --json
+program template update <id> --name "New Name" --json
+program template run <id> --json
 ```
 
-### Workflows
+## Research
 
 ```bash
-# List workflow templates
-npx @program-video/cli workflow list --json
-
-# Run a workflow
-npx @program-video/cli workflow run <templateId> --project <projectId> --input '<json>' --json
-
-# Check workflow status
-npx @program-video/cli workflow status <executionId> --json
+program research add <url> [--note "Why this matters"] --json
+program research list [--platform youtube] [--limit 20] --json
+program research analyze <id-or-url> --json
+program research probe <id-or-url> --json
+program research to-template <id> --json
 ```
 
-## JSON Output
+## Audio Production Workflow
 
-**Always use `--json` flag** for structured, parseable output:
+The recommended sequence for producing audio in a composition:
 
-```json
-{
-  "success": true,
-  "data": { ... }
-}
-```
+1. **Generate TTS** for each scene's narration
+   ```bash
+   program tool exec generateSpeech --project <id> --params '{
+     "text": "Scene narration",
+     "voice": "Ethan"
+   }' --json
+   ```
 
-Errors return:
-```json
-{
-  "success": false,
-  "error": "Error message"
-}
-```
+2. **Trim silence** from audio scenes
+   ```bash
+   program tool exec trimSilence --project <id> --json
+   ```
 
-## Typical Workflow
+3. **Transcribe** each trimmed audio file
+   ```bash
+   program tool exec transcribeAudio --params '{
+     "filePath": "/tmp/program-video-cache/audio/<scene-id>-trimmed.wav",
+     "output": "/tmp/program-video-cache/audio/<scene-id>-transcript.json"
+   }' --json
+   ```
 
-1. **Authenticate**: `auth login`
-2. **Create project**: `project create --title "Video Title" --json`
-3. **Read composition**: `tool exec readComposition --project <id> --json`
-4. **Add scenes**: `tool exec addScene --project <id> --params '...' --json`
-5. **Generate speech**: `tool exec generateSpeech --project <id> --params '{"text":"..."}' --json`
-6. **Send status**: `message send "Video complete!" --project <id> --json`
+4. **Build captions** from transcripts
+   ```bash
+   program tool exec buildCaptions --project <id> --json
+   ```
 
 ## Important Notes
 
-- All commands support `--json` for machine-readable output
-- Project IDs are returned when creating projects
-- Scene code uses React/Remotion - export a default component
-- Changes sync to the web editor at program.video in real-time
-
-## Documentation
-
-Full documentation: https://docs.program.video/docs/cli
+- **Always use `--json`** for structured, parseable output
+- **`--project` accepts a composition ID** (in local mode, compositions are the projects)
+- **Voice names are case-sensitive** — "Ethan" not "ethan"
+- **Audio files cached at** `/tmp/program-video-cache/audio/`
+- **Replicate URLs expire** within hours — files are downloaded and cached immediately
+- **Local whisper** requires `pip install openai-whisper`
+- **ffmpeg required** for trim-silence and audio duration detection
+- **Trim before transcribe** — silence at the start throws off timestamp alignment
+- **Audio file validation** — if a downloaded file is < 1000 bytes, the Replicate URL likely expired

package/dist/binary-manager-HSC2WG4R.js

@@ -0,0 +1,17 @@
+#!/usr/bin/env node
+import {
+  checkSetup,
+  getBinPath,
+  getWhisperModelPath,
+  resolveBinary,
+  setupAll
+} from "./chunk-WMJSGRMP.js";
+import "./chunk-2H7UOFLK.js";
+export {
+  checkSetup,
+  getBinPath,
+  getWhisperModelPath,
+  resolveBinary,
+  setupAll
+};
+//# sourceMappingURL=binary-manager-HSC2WG4R.js.map

package/dist/binary-manager-HSC2WG4R.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}

package/dist/captions-V3I4YG3Q.js

@@ -0,0 +1,14 @@
+#!/usr/bin/env node
+import {
+  buildCaptionEntries,
+  captionsCommand,
+  generateCaptionCode
+} from "./chunk-Z5WNLASJ.js";
+import "./chunk-2PMNEXGG.js";
+import "./chunk-2H7UOFLK.js";
+export {
+  buildCaptionEntries,
+  captionsCommand,
+  generateCaptionCode
+};
+//# sourceMappingURL=captions-V3I4YG3Q.js.map

package/dist/captions-V3I4YG3Q.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}

package/dist/chunk-2H7UOFLK.js

@@ -0,0 +1,11 @@
+#!/usr/bin/env node
+var __defProp = Object.defineProperty;
+var __export = (target, all) => {
+  for (var name in all)
+    __defProp(target, name, { get: all[name], enumerable: true });
+};
+
+export {
+  __export
+};
+//# sourceMappingURL=chunk-2H7UOFLK.js.map

package/dist/chunk-2H7UOFLK.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":[],"sourcesContent":[],"mappings":"","names":[]}