@t3lnet/sceneforge 1.0.9 → 1.0.11

package/context/templates/stages/stage3-balancing.md

@@ -0,0 +1,229 @@
+# Stage 3: Step Balancing
+
+This stage focuses on analyzing timing data and adjusting scripts/actions to achieve optimal synchronization between voiceover and video.
+
+## Objectives
+
+1. Analyze script JSON output for timing data
+2. Compare estimated vs. actual durations
+3. Adjust scripts or add wait actions
+4. Achieve smooth synchronization
+
+## Understanding Timing Data
+
+After running `sceneforge record`, timing data is saved to:
+```
+output/<demo-name>/scripts/<demo-name>-scripts.json
+```
+
+### Script JSON Format
+
+```json
+{
+  "demoName": "my-demo",
+  "generatedAt": "2024-01-15T10:30:00Z",
+  "steps": [
+    {
+      "stepId": "intro",
+      "script": "Welcome to this demo...",
+      "wordCount": 12,
+      "estimatedDurationMs": 4800,
+      "stepTimingMs": 3000,
+      "actions": [
+        {
+          "action": "wait",
+          "duration": 3000
+        }
+      ]
+    }
+  ],
+  "totalEstimatedDurationMs": 45000,
+  "totalWordCount": 112
+}
+```
+
+### Key Metrics
+
+| Metric | Description |
+|--------|-------------|
+| `estimatedDurationMs` | Expected voiceover duration (wordCount × 400ms) |
+| `stepTimingMs` | Actual recorded step duration |
+| `wordCount` | Number of words in script |
+
+## Balancing Strategy
+
+### Step Duration Analysis
+
+For each step, compare:
+```
+Difference = stepTimingMs - estimatedDurationMs
+```
+
+**If positive (actions longer than script):**
+- Video has "dead air"
+- Options: Expand script OR add wait action at start
+
+**If negative (script longer than actions):**
+- Voiceover will overlap next step
+- Options: Shorten script OR add wait action at end
+
+### Adjustment Options
+
+#### 1. Expand Script (actions > script)
+
+**Before:**
+```yaml
+- id: create-project
+  script: "Click Create to make a new project."
+  actions:
+    - action: click
+      target: { type: button, text: "Create" }
+    - action: wait
+      waitFor: { type: selector, value: ".modal" }
+    - action: type
+      target: { type: input, name: "name" }
+      text: "My Project"
+```
+
+**After (expanded script):**
+```yaml
+- id: create-project
+  script: "Click the Create button to start a new project. In the dialog that appears, enter a descriptive name for your project."
+  actions:
+    - action: click
+      target: { type: button, text: "Create" }
+    - action: wait
+      waitFor: { type: selector, value: ".modal" }
+    - action: type
+      target: { type: input, name: "name" }
+      text: "My Project"
+```
+
+#### 2. Add Wait Action (actions < script)
+
+**Before:**
+```yaml
+- id: quick-action
+  script: "This powerful feature allows you to quickly process multiple items at once, saving significant time in your workflow."
+  actions:
+    - action: click
+      target: { type: button, text: "Process All" }
+```
+
+**After (with wait):**
+```yaml
+- id: quick-action
+  script: "This powerful feature allows you to quickly process multiple items at once, saving significant time in your workflow."
+  actions:
+    - action: click
+      target: { type: button, text: "Process All" }
+    - action: wait
+      duration: 3000  # Allow voiceover to complete
+```
+
+#### 3. Shorten Script (script too long)
+
+**Before:**
+```yaml
+- id: save-settings
+  script: "Now we need to click the Save button located at the bottom of the form to save all of our configuration changes to the system."
+  actions:
+    - action: click
+      target: { type: button, text: "Save" }
+```
+
+**After (shortened):**
+```yaml
+- id: save-settings
+  script: "Click Save to apply your changes."
+  actions:
+    - action: click
+      target: { type: button, text: "Save" }
+```
+
+## Timing Formulas
+
+### Estimate Script Duration
+```
+estimatedMs = wordCount × 400
+```
+
+### Calculate Required Wait
+```
+waitMs = estimatedDurationMs - stepTimingMs
+```
+If `waitMs > 0`, add wait action with that duration.
+
+### Words Per Second
+```
+wordsPerSecond = 2.5  (150 WPM)
+```
+
+## Balancing Workflow
+
+1. **Record demo:**
+   ```bash
+   sceneforge record -d demo.yaml -b http://localhost:3000
+   ```
+
+2. **Review timing data:**
+   ```bash
+   cat output/my-demo/scripts/my-demo-scripts.json
+   ```
+
+3. **Identify imbalances:**
+   - Steps where `stepTimingMs << estimatedDurationMs`
+   - Steps where `stepTimingMs >> estimatedDurationMs`
+
+4. **Make adjustments:**
+   - Edit YAML to modify scripts
+   - Add/remove wait actions
+
+5. **Re-record and verify:**
+   ```bash
+   sceneforge record -d demo.yaml -b http://localhost:3000
+   ```
+
+## Common Adjustments
+
+### Add Intro Padding
+```yaml
+- id: intro
+  script: "Welcome to the Product Tour. In this demo, we'll explore the key features."
+  actions:
+    - action: wait
+      duration: 4000  # Match script duration
+```
+
+### Extend Transition
+```yaml
+- id: navigate-dashboard
+  script: "Navigate to the dashboard to see your analytics overview."
+  actions:
+    - action: click
+      target: { type: link, text: "Dashboard" }
+    - action: wait
+      waitFor: { type: idle }
+    - action: wait
+      duration: 1500  # Extra time for voiceover
+```
+
+### Quick Action Padding
+```yaml
+- id: single-click
+  script: "Enable notifications to stay updated on project changes."
+  actions:
+    - action: click
+      target: { type: selector, selector: "[data-testid='notifications-toggle']" }
+    - action: wait
+      duration: 2500  # Voiceover needs time
+```
+
+## Checklist Before Stage 4
+
+- [ ] All steps reviewed for timing balance
+- [ ] Scripts match action durations (within ~20%)
+- [ ] Wait actions added where needed
+- [ ] No rapid-fire short steps
+- [ ] Intro/outro have adequate duration
+- [ ] Demo flows naturally when watched

package/context/templates/stages/stage4-rebalancing.md

@@ -0,0 +1,228 @@
+# Stage 4: Rebalancing
+
+This stage focuses on the iteration cycle after audio generation, using actual audio durations to fine-tune synchronization.
+
+## Objectives
+
+1. Analyze audio manifest with actual durations
+2. Compare actual audio vs. video timing
+3. Make final timing adjustments
+4. Achieve polished final output
+
+## Audio Manifest Format
+
+After running `sceneforge voiceover`, the audio manifest is saved to:
+```
+output/<demo-name>/audio/manifest.json
+```
+
+### Manifest Structure
+
+```json
+{
+  "demoName": "my-demo",
+  "generatedAt": "2024-01-15T11:00:00Z",
+  "voiceId": "21m00Tcm4TlvDq8ikWAM",
+  "segments": [
+    {
+      "stepId": "intro",
+      "script": "Welcome to this demo...",
+      "audioFile": "intro.mp3",
+      "durationMs": 4850,
+      "estimatedDurationMs": 4800,
+      "variance": 50
+    }
+  ],
+  "totalDurationMs": 47200,
+  "totalEstimatedMs": 45000
+}
+```
+
+### Key Fields
+
+| Field | Description |
+|-------|-------------|
+| `durationMs` | Actual audio duration from ElevenLabs |
+| `estimatedDurationMs` | Our word count estimate |
+| `variance` | Difference (actual - estimated) |
+| `audioFile` | Generated audio file path |
+
+## Rebalancing Workflow
+
+### 1. Generate Voiceover
+```bash
+sceneforge voiceover --demo my-demo
+```
+
+### 2. Analyze Manifest
+```bash
+cat output/my-demo/audio/manifest.json | jq '.segments[] | {stepId, durationMs, variance}'
+```
+
+### 3. Identify Problem Steps
+
+**Audio longer than video:**
+```
+stepTimingMs: 3000
+audioDurationMs: 4500
+Problem: Voiceover will be cut off or overlap
+```
+
+**Audio shorter than video:**
+```
+stepTimingMs: 5000
+audioDurationMs: 2000
+Problem: Silent gap in demo
+```
+
+### 4. Make Adjustments
+
+Adjust the YAML based on actual audio durations.
+
+### 5. Re-run Pipeline
+```bash
+sceneforge pipeline -d demo.yaml -b http://localhost:3000 --resume
+```
+
+The `--resume` flag skips completed steps and re-processes from where changes are needed.
+
+## Adjustment Strategies
+
+### Audio Too Long
+
+**Option A: Add wait action to extend step**
+```yaml
+- id: explain-feature
+  script: "This feature provides comprehensive analytics..."
+  actions:
+    - action: hover
+      target: { type: selector, selector: ".analytics-panel" }
+    - action: wait
+      duration: 2000  # Added to match audio duration
+```
+
+**Option B: Shorten script**
+```yaml
+# Before (estimated 6s, actual 7.2s)
+script: "This feature provides comprehensive analytics including detailed breakdowns of user engagement metrics."
+
+# After (shorter to match available time)
+script: "This feature provides comprehensive analytics on user engagement."
+```
+
+### Audio Too Short
+
+**Option A: Expand script**
+```yaml
+# Before (estimated 3s, actual 2.5s, but step is 5s)
+script: "Click Save to finish."
+
+# After (expanded to fill time)
+script: "Click the Save button to store your changes. The system will confirm once saved."
+```
+
+**Option B: Split into multiple steps**
+```yaml
+# Before: One step with long video and short script
+- id: complex-workflow
+  script: "Complete the workflow."
+  actions:
+    # ... many actions
+
+# After: Split into logical parts
+- id: workflow-part-1
+  script: "First, configure the basic settings."
+  actions:
+    # ... first set of actions
+
+- id: workflow-part-2
+  script: "Next, set up the advanced options."
+  actions:
+    # ... second set of actions
+```
+
+## Iteration Cycle
+
+```
+┌─────────────────────────────────┐
+│  1. Edit YAML (scripts/waits)   │
+└───────────────┬─────────────────┘
+                │
+                ▼
+┌─────────────────────────────────┐
+│  2. Run pipeline with --resume  │
+│     (re-records if needed)      │
+└───────────────┬─────────────────┘
+                │
+                ▼
+┌─────────────────────────────────┐
+│  3. Generate new voiceover      │
+│     (or use cache if unchanged) │
+└───────────────┬─────────────────┘
+                │
+                ▼
+┌─────────────────────────────────┐
+│  4. Check manifest for timing   │
+└───────────────┬─────────────────┘
+                │
+                ▼
+        ┌───────┴───────┐
+        │  Acceptable?  │
+        └───────┬───────┘
+           No ──┼── Yes
+                │    │
+                │    ▼
+         Loop   │  Done!
+         back ──┘
+```
+
+## Using --resume Flag
+
+The `--resume` flag is essential for efficient iteration:
+
+```bash
+# First run: full pipeline
+sceneforge pipeline -d demo.yaml -b http://localhost:3000 --clean
+
+# After editing scripts only (no action changes)
+sceneforge pipeline -d demo.yaml -b http://localhost:3000 --resume --skip-record
+
+# After editing actions
+sceneforge pipeline -d demo.yaml -b http://localhost:3000 --resume
+```
+
+**Resume behavior:**
+- Detects which steps have changes
+- Re-uses cached voiceover for unchanged scripts
+- Only re-records steps with action changes
+- Regenerates final video
+
+## Quality Checks
+
+### Visual Sync Check
+Watch the final video and note:
+- [ ] Voiceover starts match action starts
+- [ ] No awkward silences
+- [ ] No cut-off narration
+- [ ] Transitions feel natural
+
+### Timing Tolerance
+Aim for variance within ±500ms per step:
+- Under 500ms: Usually acceptable
+- 500-1000ms: Consider adjustment
+- Over 1000ms: Definitely adjust
+
+### Audio Quality Check
+- [ ] No clipped words at step boundaries
+- [ ] Volume consistent across steps
+- [ ] No unexpected pauses
+
+## Final Checklist
+
+- [ ] All steps have acceptable timing variance
+- [ ] Final video plays smoothly
+- [ ] Voiceover syncs with actions
+- [ ] No dead air or overlap issues
+- [ ] Intro provides adequate context
+- [ ] Outro concludes naturally
+- [ ] Overall demo flows professionally

package/context/tests/context-builder.test.ts

@@ -0,0 +1,237 @@
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import * as fs from "fs/promises";
+import * as path from "path";
+import * as os from "os";
+import {
+  buildContext,
+  previewContext,
+  deployContext,
+  listDeployedContext,
+  removeContext,
+  getSkill,
+  listSkills,
+  hasTemplates,
+} from "../context-builder";
+
+describe("context-builder", () => {
+  let tempDir: string;
+
+  beforeEach(async () => {
+    tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "sceneforge-test-"));
+  });
+
+  afterEach(async () => {
+    await fs.rm(tempDir, { recursive: true, force: true });
+  });
+
+  describe("hasTemplates", () => {
+    it("returns true when templates exist", async () => {
+      const result = await hasTemplates();
+      expect(result).toBe(true);
+    });
+  });
+
+  describe("buildContext", () => {
+    it("builds context for claude with all stages", async () => {
+      const content = await buildContext("claude", "all");
+      expect(content).toContain("SceneForge");
+      expect(content).toContain("Claude Code");
+    });
+
+    it("builds context for specific stage", async () => {
+      const content = await buildContext("cursor", "actions");
+      expect(content).toContain("Cursor");
+    });
+
+    it("includes base templates in output", async () => {
+      const content = await buildContext("claude", "all");
+      expect(content).toContain("YAML");
+      expect(content).toContain("DemoAction");
+    });
+
+    it("applies variable interpolation", async () => {
+      const content = await buildContext("claude", "all", {
+        customVar: "CustomValue",
+      });
+      // Variables not in templates won't appear, but function should not throw
+      expect(content.length).toBeGreaterThan(0);
+    });
+  });
+
+  describe("previewContext", () => {
+    it("returns preview result with content", async () => {
+      const result = await previewContext("claude", "all");
+      expect(result.tool).toBe("claude");
+      expect(result.stage).toBeUndefined();
+      expect(result.content.length).toBeGreaterThan(0);
+    });
+
+    it("includes stage in result when specified", async () => {
+      const result = await previewContext("cursor", "actions");
+      expect(result.tool).toBe("cursor");
+      expect(result.stage).toBe("actions");
+    });
+  });
+
+  describe("deployContext", () => {
+    it("deploys combined file for single tool", async () => {
+      const results = await deployContext({
+        target: "claude",
+        stage: "all",
+        format: "combined",
+        outputDir: tempDir,
+      });
+
+      expect(results.length).toBe(1);
+      expect(results[0].created).toBe(true);
+      expect(results[0].tool).toBe("claude");
+
+      const filePath = path.join(tempDir, "CLAUDE.md");
+      const content = await fs.readFile(filePath, "utf-8");
+      expect(content).toContain("SceneForge");
+    });
+
+    it("deploys files for all tools", async () => {
+      const results = await deployContext({
+        target: "all",
+        stage: "all",
+        format: "combined",
+        outputDir: tempDir,
+      });
+
+      expect(results.length).toBe(4);
+      expect(results.every((r) => r.created)).toBe(true);
+
+      // Check each file exists
+      const claudeFile = await fs.readFile(path.join(tempDir, "CLAUDE.md"), "utf-8");
+      expect(claudeFile).toContain("Claude Code");
+
+      const cursorFile = await fs.readFile(path.join(tempDir, ".cursorrules"), "utf-8");
+      expect(cursorFile).toContain("Cursor");
+    });
+
+    it("creates directory structure for split format", async () => {
+      const results = await deployContext({
+        target: "claude",
+        stage: "actions",
+        format: "split",
+        outputDir: tempDir,
+      });
+
+      expect(results.length).toBe(1);
+      expect(results[0].created).toBe(true);
+
+      const splitDir = path.join(tempDir, ".claude/rules");
+      const files = await fs.readdir(splitDir);
+      expect(files.length).toBeGreaterThan(0);
+    });
+
+    it("deploys multiple stages in split format", async () => {
+      const results = await deployContext({
+        target: "cursor",
+        stage: "all",
+        format: "split",
+        outputDir: tempDir,
+      });
+
+      expect(results.length).toBe(4);
+      expect(results.every((r) => r.created)).toBe(true);
+    });
+  });
+
+  describe("listDeployedContext", () => {
+    it("returns empty list for empty directory", async () => {
+      const result = await listDeployedContext(tempDir);
+      expect(result.files.every((f) => !f.exists)).toBe(true);
+    });
+
+    it("detects deployed files", async () => {
+      // First deploy
+      await deployContext({
+        target: "claude",
+        stage: "all",
+        format: "combined",
+        outputDir: tempDir,
+      });
+
+      const result = await listDeployedContext(tempDir);
+      const existing = result.files.filter((f) => f.exists);
+      expect(existing.length).toBeGreaterThan(0);
+      expect(existing.some((f) => f.tool === "claude")).toBe(true);
+    });
+  });
+
+  describe("removeContext", () => {
+    it("removes deployed files for specific tool", async () => {
+      // Deploy first
+      await deployContext({
+        target: "claude",
+        stage: "all",
+        format: "combined",
+        outputDir: tempDir,
+      });
+
+      // Verify file exists
+      const filePath = path.join(tempDir, "CLAUDE.md");
+      await expect(fs.access(filePath)).resolves.toBeUndefined();
+
+      // Remove
+      const results = await removeContext(tempDir, "claude");
+      expect(results.some((r) => r.removed)).toBe(true);
+
+      // Verify file removed
+      await expect(fs.access(filePath)).rejects.toThrow();
+    });
+
+    it("removes all deployed files when target is all", async () => {
+      // Deploy all
+      await deployContext({
+        target: "all",
+        stage: "all",
+        format: "combined",
+        outputDir: tempDir,
+      });
+
+      // Remove all
+      await removeContext(tempDir, "all");
+
+      // Verify all removed
+      const result = await listDeployedContext(tempDir);
+      const existing = result.files.filter((f) => f.exists);
+      expect(existing.length).toBe(0);
+    });
+
+    it("handles non-existent files gracefully", async () => {
+      const results = await removeContext(tempDir, "claude");
+      // Should not error, just not mark as removed
+      expect(results.every((r) => !r.error || r.error === undefined)).toBe(true);
+    });
+  });
+
+  describe("getSkill", () => {
+    it("returns skill content for valid skill", async () => {
+      const skill = await getSkill("generate-actions");
+      expect(skill).not.toBeNull();
+      expect(skill?.name).toBe("generate-actions");
+      expect(skill?.content).toContain("Generate");
+    });
+
+    it("returns null for invalid skill", async () => {
+      const skill = await getSkill("non-existent-skill");
+      expect(skill).toBeNull();
+    });
+  });
+
+  describe("listSkills", () => {
+    it("returns all available skills", async () => {
+      const skills = await listSkills();
+      expect(skills).toContain("generate-actions");
+      expect(skills).toContain("write-step-script");
+      expect(skills).toContain("balance-timing");
+      expect(skills).toContain("review-demo-yaml");
+      expect(skills).toContain("debug-selector");
+      expect(skills).toContain("optimize-demo");
+      expect(skills.length).toBe(6);
+    });
+  });
+});