@mevdragon/vidfarm-devcli 0.2.12 → 0.2.14

package/auto-create-hyperframe-templates/runbook.md

@@ -0,0 +1,198 @@
+# Auto-Create HyperFrame Template Runbook
+
+Use this runbook when a simple MP4 or slideshow should become an editable HyperFrames composition and production Lambda site.
+
+## 1. Create A Run Folder
+
+```bash
+RUN_ID="<format-slug>-$(date -u +%Y%m%dT%H%M%SZ)"
+mkdir -p "auto-create-hyperframe-templates/runs/${RUN_ID}"/{source,frames,audio,transcripts,ocr,motion,extraction,plan,output}
+```
+
+Copy source media and notes into `source/`.
+
+If running through automation, create `manifest.json` matching `input-manifest.schema.json` before copying media:
+
+```json
+{
+  "mode": "autocreate-hyperframe-template",
+  "input_path": "drafts/example/preview/source.mp4",
+  "output_path": "drafts/example",
+  "slug_id": "example",
+  "source_mode": "auto",
+  "render_defaults": {
+    "width": 720,
+    "height": 1280,
+    "fps": 30
+  },
+  "lambda": {
+    "stack_name": "hyperframes-vidfarm-prod",
+    "region": "us-east-1",
+    "memory_strategy": "estimate"
+  },
+  "publish": {
+    "prestage_site": false,
+    "output_url_policy": "signed_https_final_output_only"
+  }
+}
+```
+
+## 2. Extract Media Artifacts
+
+For MP4:
+
+```bash
+ffprobe -v error -show_entries format=duration:stream=codec_type,width,height,avg_frame_rate -of json source/source.mp4 \
+  > "auto-create-hyperframe-templates/runs/${RUN_ID}/source/ffprobe.json"
+
+ffmpeg -y -i source/source.mp4 \
+  -vf "fps=1/2,scale=240:-1,tile=3x4" \
+  -frames:v 1 "auto-create-hyperframe-templates/runs/${RUN_ID}/frames/contact-sheet.jpg"
+
+ffmpeg -y -i source/source.mp4 \
+  -vn -ac 1 -ar 16000 "auto-create-hyperframe-templates/runs/${RUN_ID}/audio/source.wav"
+```
+
+For slideshows, probe each image and create a contact sheet:
+
+```bash
+ffprobe -v error -show_entries stream=width,height -of json source/slide-01.png \
+  > "auto-create-hyperframe-templates/runs/${RUN_ID}/source/slide-01.ffprobe.json"
+```
+
+Add scene detection, OCR, STT, and layout/subject-region detection with whichever local or provider tool is available. The important part is to write concrete timing, text, and bounding boxes into the run folder before planning.
+
+## 3. Decide Source Mode
+
+Classify the input:
+
+- `single_video`: one MP4 drives the full timeline
+- `scene_split_video`: one MP4 should be split into editable timed scene clips
+- `slideshow`: images become timed slides
+- `mixed_media`: video, images, audio, and HTML overlays need explicit composition
+
+Record whether source media contains baked text that should be extracted as deterministic captions/HTML overlays.
+
+## 4. Generate Production Graph
+
+Send `extractor-system-prompt.md`, source notes, artifacts, contact sheet, transcript, OCR, and scene cuts to a multimodal model.
+
+Write:
+
+```txt
+auto-create-hyperframe-templates/runs/<run_id>/extraction/production_graph.json
+```
+
+Validate against:
+
+```txt
+auto-create-hyperframe-templates/production-graph.schema.json
+```
+
+## 5. Generate HyperFrame Template Plan
+
+Send `template-planner-prompt.md` plus `production_graph.json`.
+
+Write:
+
+```txt
+auto-create-hyperframe-templates/runs/<run_id>/plan/template_plan.json
+```
+
+Validate against:
+
+```txt
+auto-create-hyperframe-templates/template-plan.schema.json
+```
+
+## 6. Write Composition Artifacts
+
+Create or update the target source folder:
+
+```txt
+drafts/<format_slug>/
+  composition.html
+  composition.json
+  lambda-site.json
+```
+
+For simple MP4/slideshow inputs, the local bridge can do this step directly:
+
+```bash
+node auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs drafts/<format_slug>/preview/source.mp4
+node auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs drafts/<format_slug>/preview --stage-site
+```
+
+Implementation order:
+
+1. write `composition.json` from `template_plan.layers`
+2. write `composition.html` from the same layer manifest
+3. include root `data-composition-id`, `data-width`, `data-height`, `data-duration`, and `data-source-folder`
+4. write `lambda-site.json` with stack, region, memory, render defaults, and site upload status
+5. run local editor smoke test
+
+## 7. HyperFrames Layer Rules
+
+Use these data attributes in `composition.html`:
+
+- `data-hf-id`
+- `data-layer-kind`
+- `data-layer-mode`
+- `data-start`
+- `data-duration`
+- `data-track-index`
+- `data-label`
+- `data-media-start` for timed media slices
+- `data-playback-start` for editor/publish compatibility
+- `data-src` when using a timeline proxy
+
+For video scene clips, prefer timed real `<video>` elements or a timeline proxy that publish normalization rewrites into a real `<video>`. A preview-only `data-vf-playback-source` may help editor scrubbing, but it must not be the publish source of truth.
+
+## 8. Estimate Lambda Memory
+
+Use `lambda-memory-estimator.md`.
+
+Recommended baseline:
+
+- `2048` MB: static images, simple slideshows, minimal HTML, no video decode pressure
+- `3072` MB: default for vertical social video with one decoded video stream and deterministic overlays
+- `4096` MB: 1080p video, multiple streams, heavy captions, large shadows, masks, or long audio
+- `6144+` MB: high-resolution, many active layers, complex filters, or measured failures
+
+Write both the recommendation and the reasoning into `composition.json` and `lambda-site.json`.
+
+## 9. Stage Production Lambda Site
+
+After the project folder is valid, pre-stage it for repeat publishing:
+
+```bash
+hyperframes lambda sites create drafts/<format_slug> \
+  --stack-name hyperframes-vidfarm-prod \
+  --region us-east-1
+```
+
+If using the SDK or demo API instead of the CLI, write the returned content-addressed site id into `lambda-site.json`.
+
+The frontend should show the final signed output URL after publish. Do not expose S3 Console or Step Functions Console links as user-facing CTAs.
+
+## 10. Validate Publish
+
+Run one local render and one Lambda publish:
+
+```bash
+hyperframes lambda render drafts/<format_slug> \
+  --stack-name hyperframes-vidfarm-prod \
+  --region us-east-1 \
+  --site-id <site_id> \
+  --width 720 \
+  --height 1280 \
+  --wait
+```
+
+Check:
+
+- final output reflects deleted/hidden scene layers
+- caption timing matches the editor timeline
+- blank gaps remain blank when no media layer covers them
+- audio does not silently disappear after scene splitting
+- output URL is accessible without AWS Console login when surfaced in the product

package/auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs

@@ -0,0 +1,368 @@
+#!/usr/bin/env node
+import { spawn } from "node:child_process";
+import { cp, copyFile, mkdir, readFile, stat, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const scriptDir = path.dirname(fileURLToPath(import.meta.url));
+const harnessRoot = path.resolve(scriptDir, "..");
+const repoRoot = path.resolve(harnessRoot, "..");
+const defaultStackName = "hyperframes-vidfarm-prod";
+const defaultRegion = "us-east-1";
+const memoryBuckets = [2048, 3072, 4096, 6144, 8192, 10240];
+const videoExts = new Set([".mp4", ".mov", ".m4v", ".webm"]);
+const imageExts = new Set([".png", ".jpg", ".jpeg", ".webp"]);
+
+const args = process.argv.slice(2);
+const stageSite = args.includes("--stage-site");
+const inputArg = args.find((arg) => !arg.startsWith("--"));
+
+if (!inputArg) {
+  console.error("Usage: node auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs <media-path|input-manifest.json> [--stage-site]");
+  process.exit(1);
+}
+
+const manifest = await loadInputManifest(inputArg);
+const outputDir = path.resolve(repoRoot, manifest.output_path);
+const decomposeInput = await prepareInputForDecomposer(manifest);
+
+await runCommand(process.execPath, [
+  path.join(repoRoot, "demo", "scripts", "decompose-static.mjs"),
+  path.relative(repoRoot, decomposeInput)
+]);
+
+const decompositionPath = path.join(outputDir, "composition.json");
+const decomposition = JSON.parse(await readFile(decompositionPath, "utf8"));
+const memoryEstimate = estimateLambdaMemory(decomposition, manifest);
+const lambdaSite = await buildLambdaSiteManifest(manifest, outputDir, memoryEstimate, stageSite);
+
+decomposition.schema = "vidfarm.hyperframe-decomposition.v1";
+decomposition.lambda_memory_estimate = memoryEstimate;
+decomposition.lambda_site = lambdaSite;
+
+await writeFile(decompositionPath, `${JSON.stringify(decomposition, null, 2)}\n`);
+await writeFile(path.join(outputDir, "lambda-site.json"), `${JSON.stringify(lambdaSite, null, 2)}\n`);
+
+console.log(`Wrote ${path.relative(repoRoot, path.join(outputDir, "composition.html"))}`);
+console.log(`Wrote ${path.relative(repoRoot, decompositionPath)}`);
+console.log(`Wrote ${path.relative(repoRoot, path.join(outputDir, "lambda-site.json"))}`);
+console.log(`Lambda memory estimate: ${memoryEstimate.recommended_memory_mb} MB (${memoryEstimate.confidence} confidence)`);
+if (lambdaSite.site_id) {
+  console.log(`Lambda site id: ${lambdaSite.site_id}`);
+}
+
+async function loadInputManifest(input) {
+  const inputPath = path.resolve(repoRoot, input);
+  const inputStats = await stat(inputPath);
+  if (inputStats.isFile() && path.extname(inputPath).toLowerCase() === ".json") {
+    const parsed = JSON.parse(await readFile(inputPath, "utf8"));
+    if (parsed.mode === "autocreate-hyperframe-template") {
+      return normalizeManifest(parsed, inputPath);
+    }
+  }
+
+  const outputPath = inputStats.isDirectory() ? inputPath : path.dirname(inputPath);
+  return normalizeManifest({
+    mode: "autocreate-hyperframe-template",
+    input_path: path.relative(repoRoot, inputPath),
+    output_path: path.relative(repoRoot, outputPath),
+    slug_id: path.basename(outputPath),
+    source_mode: "auto",
+    render_defaults: {
+      width: 720,
+      height: 1280,
+      fps: 30
+    },
+    lambda: {
+      stack_name: defaultStackName,
+      region: defaultRegion,
+      memory_strategy: "estimate"
+    },
+    publish: {
+      prestage_site: false,
+      output_url_policy: "signed_https_final_output_only"
+    }
+  }, inputPath);
+}
+
+function normalizeManifest(manifest, manifestPath) {
+  if (manifest.mode !== "autocreate-hyperframe-template") {
+    throw new Error(`Invalid manifest mode in ${manifestPath}`);
+  }
+  return {
+    ...manifest,
+    output_path: manifest.output_path || path.dirname(manifest.input_path),
+    source_mode: manifest.source_mode || "auto",
+    render_defaults: {
+      width: Number(manifest.render_defaults?.width || 720),
+      height: Number(manifest.render_defaults?.height || 1280),
+      fps: Number(manifest.render_defaults?.fps || 30)
+    },
+    lambda: {
+      stack_name: manifest.lambda?.stack_name || defaultStackName,
+      region: manifest.lambda?.region || defaultRegion,
+      memory_strategy: manifest.lambda?.memory_strategy || "estimate",
+      fixed_memory_mb: manifest.lambda?.fixed_memory_mb
+    },
+    publish: {
+      prestage_site: Boolean(manifest.publish?.prestage_site),
+      output_url_policy: "signed_https_final_output_only"
+    }
+  };
+}
+
+async function prepareInputForDecomposer(manifest) {
+  const inputPath = path.resolve(repoRoot, manifest.input_path);
+  const outputPath = path.resolve(repoRoot, manifest.output_path);
+  const inputStats = await stat(inputPath);
+
+  if (inputStats.isDirectory()) {
+    if (path.resolve(inputPath) !== path.resolve(outputPath)) {
+      await mkdir(outputPath, { recursive: true });
+      await cp(inputPath, outputPath, { recursive: true, force: true });
+      return outputPath;
+    }
+    return inputPath;
+  }
+
+  const ext = path.extname(inputPath).toLowerCase();
+  if (!videoExts.has(ext) && !imageExts.has(ext)) {
+    throw new Error(`Unsupported input media: ${inputPath}`);
+  }
+
+  if (path.resolve(path.dirname(inputPath)) !== path.resolve(outputPath)) {
+    await mkdir(outputPath, { recursive: true });
+    const target = path.join(outputPath, path.basename(inputPath));
+    await copyFile(inputPath, target);
+    return target;
+  }
+
+  return inputPath;
+}
+
+function estimateLambdaMemory(decomposition, manifest) {
+  const renderDefaults = manifest.render_defaults;
+  const layers = Array.isArray(decomposition.editor_layers) ? decomposition.editor_layers : [];
+  const assets = Array.isArray(decomposition.source_assets) ? decomposition.source_assets : [];
+  const firstAsset = assets[0] || {};
+  const width = Number(renderDefaults?.width || firstAsset.width || 720);
+  const height = Number(renderDefaults?.height || firstAsset.height || 1280);
+  const outputPixels = width * height;
+  const durationSeconds = layers.reduce((max, layer) => {
+    return Math.max(max, Number(layer.start || 0) + Number(layer.duration || 0));
+  }, Number(firstAsset.duration || firstAsset.duration_seconds || 0));
+  const videoLayers = layers.filter((layer) => layer.type === "video" || layer.kind === "video");
+  const audioLayers = layers.filter((layer) => layer.type === "audio" || layer.kind === "audio");
+  const textLayers = layers.filter((layer) => ["caption", "text", "shape", "html"].includes(layer.type || layer.kind));
+  const uniqueVideoSources = new Set(videoLayers.map((layer) => layer.src || layer.media?.src).filter(Boolean));
+  const maxSimultaneousVideos = maxOverlap(videoLayers);
+
+  const outputPixelScoreMb = outputPixels <= 1_000_000 ? 0 : outputPixels <= 2_100_000 ? 512 : outputPixels <= 4_000_000 ? 1024 : 2048;
+  let mediaScoreMb = 0;
+  if (videoLayers.length > 0 || assets.some((asset) => asset.type === "video")) {
+    mediaScoreMb += 768;
+    mediaScoreMb += Math.max(0, maxSimultaneousVideos - 1) * 512;
+    mediaScoreMb += Math.max(0, uniqueVideoSources.size - 1) * 256;
+    if (assets.some((asset) => Number(asset.width || 0) * Number(asset.height || 0) > outputPixels * 1.75)) {
+      mediaScoreMb += 512;
+    }
+  }
+
+  let domCaptionScoreMb = layers.length <= 10 ? 0 : layers.length <= 30 ? 256 : 512;
+  if (textLayers.length >= 4) domCaptionScoreMb += 256;
+  if (textLayers.some((layer) => String(layer.textBackgroundStyle || layer.background_style || "").includes("highlight"))) {
+    domCaptionScoreMb += 256;
+  }
+
+  let effectScoreMb = 0;
+  if (textLayers.some((layer) => ["outline", "radial_halo"].includes(layer.textBackgroundStyle || layer.background_style))) {
+    effectScoreMb += 256;
+  }
+  if (layers.some((layer) => layer.filter || layer.mask || layer.mixBlendMode || layer.blur)) {
+    effectScoreMb += 512;
+  }
+
+  let durationAudioScoreMb = durationSeconds <= 30 ? 0 : durationSeconds <= 60 ? 256 : durationSeconds <= 180 ? 512 : 1024;
+  if (audioLayers.length > 0 || assets.some((asset) => asset.type === "audio" || asset.type === "video")) {
+    durationAudioScoreMb += 256;
+  }
+  if (audioLayers.length > 1) durationAudioScoreMb += 256;
+
+  const rawEstimateMb = 1536 + outputPixelScoreMb + mediaScoreMb + domCaptionScoreMb + effectScoreMb + durationAudioScoreMb;
+  let roundedBucketMb = memoryBuckets.find((bucket) => bucket >= rawEstimateMb) || 10240;
+  if (videoLayers.length === 0 && !assets.some((asset) => asset.type === "video")) {
+    roundedBucketMb = Math.min(roundedBucketMb, 2048);
+  }
+  if ((videoLayers.length > 0 || assets.some((asset) => asset.type === "video")) && maxSimultaneousVideos <= 1 && roundedBucketMb < 3072) {
+    roundedBucketMb = 3072;
+  }
+  if (roundedBucketMb > 4096 && rawEstimateMb < 6144) {
+    roundedBucketMb = 4096;
+  }
+  const fixedMemory = manifest.lambda?.memory_strategy === "fixed"
+    ? Number(manifest.lambda?.fixed_memory_mb || 0)
+    : 0;
+  const recommendedMemoryMb = memoryBuckets.includes(fixedMemory) ? fixedMemory : roundedBucketMb;
+  const estimatedBilledSeconds = estimateBilledSeconds(durationSeconds, layers.length, videoLayers.length);
+  const estimatedLambdaGbSeconds = (recommendedMemoryMb / 1024) * estimatedBilledSeconds;
+  const estimatedCostUsd = estimatedLambdaGbSeconds * 0.0000166667 + 40 * 0.000025;
+
+  const riskFlags = [];
+  if (videoLayers.length > 0 || assets.some((asset) => asset.type === "video")) riskFlags.push("source_video_decode");
+  if (maxSimultaneousVideos > 1) riskFlags.push("multiple_simultaneous_video_streams");
+  if (textLayers.length >= 4) riskFlags.push("caption_or_html_overlay_density");
+  if (audioLayers.length > 0 || assets.some((asset) => asset.type === "video")) riskFlags.push("continuous_audio");
+  if (outputPixels > 2_100_000) riskFlags.push("high_output_resolution");
+
+  return {
+    recommended_memory_mb: recommendedMemoryMb,
+    confidence: riskFlags.length >= 3 ? "medium" : "high",
+    complexity_scores: {
+      output_pixels: outputPixels,
+      output_pixel_score_mb: outputPixelScoreMb,
+      max_simultaneous_video_streams: maxSimultaneousVideos,
+      unique_video_sources: uniqueVideoSources.size,
+      media_score_mb: mediaScoreMb,
+      layer_count: layers.length,
+      text_layer_count: textLayers.length,
+      dom_caption_score_mb: domCaptionScoreMb,
+      effect_score_mb: effectScoreMb,
+      duration_seconds: Number(durationSeconds.toFixed(3)),
+      duration_audio_score_mb: durationAudioScoreMb,
+      raw_estimate_mb: rawEstimateMb,
+      rounded_bucket_mb: roundedBucketMb,
+      estimated_billed_seconds: estimatedBilledSeconds
+    },
+    risk_flags: riskFlags,
+    rationale: rationaleForEstimate(recommendedMemoryMb, maxSimultaneousVideos, videoLayers.length, outputPixels, textLayers.length),
+    benchmark_override: null,
+    estimated_lambda_gb_seconds: Number(estimatedLambdaGbSeconds.toFixed(3)),
+    estimated_cost_usd: Number(estimatedCostUsd.toFixed(4))
+  };
+}
+
+function maxOverlap(layers) {
+  if (layers.length === 0) return 0;
+  const points = [];
+  for (const layer of layers) {
+    const start = Number(layer.start || 0);
+    const end = start + Number(layer.duration || 0);
+    if (end > start) {
+      points.push(start + (end - start) / 2);
+      points.push(start + 0.001);
+    }
+  }
+  return points.reduce((max, point) => {
+    const active = layers.filter((layer) => {
+      const start = Number(layer.start || 0);
+      const end = start + Number(layer.duration || 0);
+      return point >= start && point < end;
+    }).length;
+    return Math.max(max, active);
+  }, 0);
+}
+
+function estimateBilledSeconds(durationSeconds, layerCount, videoLayerCount) {
+  const base = videoLayerCount > 0 ? 90 : 20;
+  const durationFactor = durationSeconds * (videoLayerCount > 0 ? 8 : 4);
+  const layerFactor = layerCount * 2;
+  return Math.ceil(Math.max(base, durationFactor + layerFactor));
+}
+
+function rationaleForEstimate(memoryMb, maxVideos, videoLayerCount, outputPixels, textLayerCount) {
+  if (videoLayerCount === 0) {
+    return "No active video decode pressure; simple static/slideshow output fits the 2GB bucket.";
+  }
+  if (memoryMb === 3072) {
+    return "One active social-video stream with deterministic overlays fits the measured 3GB production baseline.";
+  }
+  if (memoryMb === 4096) {
+    return `Raised to 4GB for ${maxVideos} simultaneous video stream(s), ${Math.round(outputPixels / 1_000_000 * 10) / 10}MP output, and ${textLayerCount} text/html layer(s).`;
+  }
+  return "Raised above 4GB because composition complexity crosses the normal social-video range.";
+}
+
+async function buildLambdaSiteManifest(manifest, outputDir, memoryEstimate, shouldStageSite) {
+  const relativeOutput = path.relative(repoRoot, outputDir);
+  const stackName = manifest.lambda.stack_name || defaultStackName;
+  const region = manifest.lambda.region || defaultRegion;
+  const siteUploadCommand = `demo/node_modules/.bin/hyperframes lambda sites create ${shellQuote(relativeOutput)} --stack-name ${shellQuote(stackName)} --region ${shellQuote(region)}`;
+  const site = {
+    schema: "vidfarm.hyperframe.lambda-site.v1",
+    stack_name: stackName,
+    region,
+    site_id: null,
+    project_dir: relativeOutput,
+    composition_file: "composition.html",
+    memory_mb: memoryEstimate.recommended_memory_mb,
+    render_defaults: manifest.render_defaults,
+    output_url_policy: "signed_https_final_output_only",
+    prestage_site_requested: Boolean(manifest.publish?.prestage_site),
+    site_upload_command: siteUploadCommand,
+    estimated_lambda_gb_seconds: memoryEstimate.estimated_lambda_gb_seconds,
+    estimated_cost_usd: memoryEstimate.estimated_cost_usd
+  };
+
+  if (!shouldStageSite) {
+    return site;
+  }
+
+  try {
+    const hyperframesBin = path.join(repoRoot, "demo", "node_modules", ".bin", "hyperframes");
+    const stdout = await runCommand(hyperframesBin, [
+      "lambda",
+      "sites",
+      "create",
+      relativeOutput,
+      "--stack-name",
+      stackName,
+      "--region",
+      region
+    ]);
+    site.site_id = parseSiteId(stdout);
+    site.site_upload_stdout = stdout.trim();
+  } catch (error) {
+    site.site_upload_error = error instanceof Error ? error.message : String(error);
+  }
+
+  return site;
+}
+
+function parseSiteId(stdout) {
+  const text = String(stdout || "");
+  const labeled = text.match(/Site ID:\s*([A-Za-z0-9_-]+)/i);
+  if (labeled) return labeled[1];
+  const generic = text.match(/\b([a-f0-9]{12,64}|[A-Za-z0-9_-]{12,64})\b/);
+  return generic ? generic[1] : null;
+}
+
+function runCommand(command, commandArgs) {
+  return new Promise((resolve, reject) => {
+    const child = spawn(command, commandArgs, {
+      cwd: repoRoot,
+      env: process.env,
+      stdio: ["ignore", "pipe", "pipe"]
+    });
+    let stdout = "";
+    let stderr = "";
+    child.stdout.on("data", (chunk) => {
+      stdout += String(chunk);
+      process.stdout.write(chunk);
+    });
+    child.stderr.on("data", (chunk) => {
+      stderr += String(chunk);
+      process.stderr.write(chunk);
+    });
+    child.on("error", reject);
+    child.on("close", (code) => {
+      if (code === 0) resolve(stdout);
+      else reject(new Error(`${command} exited ${code}: ${stderr}`));
+    });
+  });
+}
+
+function shellQuote(value) {
+  const text = String(value);
+  if (/^[A-Za-z0-9_./:=@-]+$/.test(text)) return text;
+  return `'${text.replaceAll("'", "'\\''")}'`;
+}