@amaster.ai/pi-video-gen 0.1.27 → 0.1.29

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@amaster.ai/pi-video-gen",
-  "version": "0.1.27",
+  "version": "0.1.29",
   "description": "Pi extension for direct video-model clips, recoverable mixed-shot rendering, and local concat, FFmpeg timeline or Remotion composition with subtitles and QC.",
   "keywords": [
     "pi-package",
@@ -54,12 +54,12 @@
   "dependencies": {
     "sharp": "^0.34.0",
     "msedge-tts": "^2.0.7",
-    "@amaster.ai/pi-shared": "0.1.27"
+    "@amaster.ai/pi-shared": "0.1.29"
   },
   "optionalDependencies": {
     "@amaster.ai/pi-video-gen-ffmpeg-darwin-arm64": "0.1.8",
-    "@amaster.ai/pi-video-gen-ffmpeg-linux-arm64": "0.1.8",
     "@amaster.ai/pi-video-gen-ffmpeg-darwin-x64": "0.1.8",
+    "@amaster.ai/pi-video-gen-ffmpeg-linux-arm64": "0.1.8",
     "@amaster.ai/pi-video-gen-ffmpeg-win32-x64": "0.1.8",
     "@amaster.ai/pi-video-gen-ffmpeg-linux-x64": "0.1.8"
   },
@@ -80,8 +80,8 @@
     }
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "1.2.0",
-    "@earendil-works/pi-coding-agent": "1.2.0",
+    "@earendil-works/pi-ai": "1.3.0",
+    "@earendil-works/pi-coding-agent": "1.3.0",
     "typebox": "1.3.27",
     "vitest": "^4.0.0",
     "ffmpeg-static": "^5.2.0"

package/skills/video-gen/SKILL.md

@@ -5,7 +5,7 @@ description: "Create video from existing clips, direct video-model calls, batch
 
 # Video generation
 
-Choose each shot's source independently: existing video, a model-generated clip, or graphics in a Remotion project. Then choose the assembly path:
+Choose each shot's source independently: existing video, a model-generated clip, or graphics in a Remotion project. In a proposed workflow, name the actual tool at each handoff, including `image_generate` for missing portraits or reference frames; a package or skill name alone does not specify the executable tool. Then choose the assembly path:
 
 - **C0 local concat** — `video_compose` joins compatible existing MP4 clips.
 - **Timeline render** — `video_compose` locally turns images/screenshots and existing video clips into a video with overlays, motion, transitions, soft or burned subtitles, source audio, and optional BGM. Narration is an optional network feature.
@@ -27,7 +27,9 @@ The AI video model is fixed by `pi-video-gen.defaultModel`; generated images use
    | Promo/explainer from images, screenshots & clips | `video_compose` (TimelineSpec — local mixed-media render; optional network TTS) |
    | Precise UI, charts, typography or editable frame animation | `video_compose` with a trusted Remotion project, or `video_render` with Remotion assembly when the script also generates shots |
 
-1. **Local flows stop here.** For C0 follow §A0; for Timeline follow §A1; for Remotion read its handoff. All-local scripts need no video-model credentials or paid-model confirmation. Use `image_generate` only when source images are missing.
+   **Creative planning.** For a product launch film, product demo, or authored explainer, read [film direction](references/film-direction.md) before designing scenes, including when all media is local. Follow its direction, rhythm, sound and review workflow alongside the selected assembly path. A single requested clip or a straightforward assembly follows its supplied instructions without this planning stage.
+
+1. **Local flows use their assembly guide.** For C0 follow §A0; for Timeline follow §A1; for Remotion read its handoff. All-local scripts need no video-model credentials or paid-model confirmation. Use `image_generate` only when source images are missing.
 2. **AI preflight only for generated shots.** Call `video_capabilities` and respect the active model's duration, audio, frame and trusted-asset support. Confirm `image_generate` is available only when source frames need to be generated (`/video-gen doctor` checks; config health is `/image-gen list`).
 3. **Pick direct or scripted generation.** Use `video_generate` directly for each model clip when the agent is controlling the calls. Use `video_render` for a prepared script that automates per-shot generation, persistence, recovery and assembly. A project may mix generated shots with existing footage without generating it again.
 4. **Plan only what the film needs.** For a story with recurring characters or shot-to-shot continuity, write a shot book in conversation using §B. For other clips, work from the supplied media or shot instructions. Existing frames can be reused; generate missing portraits or first/last frames with `image_generate` using §C when they improve consistency.
@@ -223,3 +225,5 @@ Then call `video_render` with that path. Interrupted? Call it again with the sam
 ## E. Completion
 
 Before reporting success, verify the requested clip/segment count and order, inspect the final video or available QC evidence, and report the returned absolute path and any accepted degradation. For Timeline, perform the QC-frame and subtitle checks in §A1. A successful tool call alone does not establish visual acceptance; state any verification limitation.
+
+For films using [film direction](references/film-direction.md), complete its delivery review and revision checks against the actual exported version. Report separately what was checked in stills, in motion and in audio.

package/skills/video-gen/evals.json

@@ -67,14 +67,17 @@
     },
     {
       "id": "multi-shot-film",
-      "prompt": "I want a short three-shot AI film with a recurring character and visual continuity between shots. Describe the planning, frame, confirmation, and render stages in Pi, including how many times the final render tool is invoked. Do not call tools.",
-      "expected_output": "Use a shot book, generate character/shot frames with image_generate, obtain confirmations, then call video_render once.",
+      "prompt": "I want a 30-second, three-shot AI-assisted product launch film with a recurring mascot and visual continuity. AI assistance may generate missing mascot stills; model-generated video clips are optional. The product is a task manager; I have supplied actual UI screenshots and brand assets but no measured performance data. You may choose a sensible visual direction and proceed without waiting for a style selection. Explain the product evidence, rhythm and key-frame plan, frame generation, paid confirmation and render stages, including how many times the final render tool is invoked. I also want a music/effects treatment aligned to the reveals. Explain how you would achieve that using the existing Timeline or Remotion path and how you would review the export. Do not call task tools or create files; you may read skill references.",
+      "expected_output": "Read film-direction.md; ground claims in supplied product material without inventing performance numbers, recommend a direction without a mandatory selection, and provide a structured multi-shot plan covering intent, timing, readable key frames and mascot continuity. Use image_generate only for missing frames and confirm any paid scope. Choose a supported route: one video_compose call for local/Remotion assembly, or one video_render call for a prepared generated-shot batch with assembly. Explain feasible audio timing in Remotion or disclose Timeline limitations; review actual stills, motion and audio with honest verification limits.",
       "expectations": [
         {
-          "text": "Uses a shot book for the multi-shot plan",
+          "text": "Provides a structured multi-shot plan with shot intent, timing and recurring mascot continuity; a shot book, beat map or equivalent scene plan is acceptable without a required artifact name",
           "includes_any": [
             "shot book",
-            "shot-book"
+            "shot-book",
+            "shot plan",
+            "beat map",
+            "scene plan"
           ]
         },
         {
@@ -84,14 +87,53 @@
           ]
         },
         {
-          "text": "Calls video_render only once",
-          "includes": [
+          "text": "Chooses a supported route and specifies one initial final-assembly invocation: video_compose for local/Remotion assembly, or video_render for a prepared generated-shot batch with assembly; any paid generation retains scoped approval",
+          "includes_any": [
+            "video_compose",
             "video_render"
+          ]
+        },
+        {
+          "text": "Grounds claims and visuals in actual supplied product evidence without inventing performance results",
+          "includes_any": [
+            "evidence",
+            "supplied",
+            "actual"
+          ]
+        },
+        {
+          "text": "Plans rhythm, readable holds and representative key frames before building the full film",
+          "includes_any": [
+            "rhythm",
+            "beat map",
+            "beat-map"
           ],
+          "includes": [
+            "hold"
+          ]
+        },
+        {
+          "text": "Recommends a visual direction and proceeds without requiring the user to choose among styles",
+          "includes_any": [
+            "recommend",
+            "default",
+            "proceed"
+          ]
+        },
+        {
+          "text": "Explains a supported way to time music/effects using Remotion or identifies that Timeline cannot place arbitrary timed effects or dynamically duck BGM",
+          "includes_any": [
+            "Remotion",
+            "fixed",
+            "duck"
+          ]
+        },
+        {
+          "text": "Reviews the exported film in stills, at normal speed and in audio, stating unavailable visual or listening checks honestly",
           "includes_any": [
-            "call once",
-            "once",
-            "one call"
+            "normal speed",
+            "normal-speed",
+            "playback"
           ]
         }
       ]

package/skills/video-gen/references/film-direction.md

@@ -0,0 +1,63 @@
+# Film direction
+
+Use this workflow for product launch films, product demos and authored explainers. Scale the plan to the requested film: a short supplied edit can use a few lines in chat; a film with several scenes or collaborators benefits from a saved `treatment.md` beside its editable source. Keep planning/source files outside generated job directories. The selected Timeline or Remotion guide remains the authority for executable inputs, trust, paid scope and recovery.
+
+## 1. Ground the film in the product
+
+Read the supplied site, UI, brand assets, data and footage before designing scenes. State the audience, the one idea the viewer should retain, the target duration/aspect ratio and the evidence that demonstrates that idea. Trace on-screen claims, numbers and UI states to those materials. Label illustrative mockups as such; use actual product states for demonstrations. If access is unavailable, identify the missing evidence and work from the material available without inventing product capabilities.
+
+Resolve only decisions that change the result and remain unspecified. Use an established style and supplied media first. State reasonable defaults for missing creative preferences and continue; ask when a missing fact or choice prevents a faithful result. Record whether sound comes from existing footage, supplied music, narration, or a music/effects treatment. Follow the entry skill's authorization rules before any paid generation or narration upload.
+
+When visual direction is genuinely open, outline up to three directions that differ in material or motion language, explaining how each serves the product. Recommend one and proceed with that default unless the user requested a selection. For an agreed style, develop it directly. Make actual key frames for materially different directions only when they will help decide the result; respect the chosen image workflow and authorization.
+
+**Ready to proceed:** the central claim has evidence, the material and visual direction support it, and any decision that blocks faithful production is resolved.
+
+## 2. Design the rhythm before building every scene
+
+Choose the route that carries the explanation: continuous movement through one world, a montage linked by a beat or visual anchor, or a continuous spine with a short montage burst. For a demo, preserve causal continuity across UI/device changes: the same request, object or person should remain identifiable through the handoff.
+
+Write a beat map in chat or the treatment. Each row describes what changes for the viewer, with enough reading time after the movement lands:
+
+| Time window | New information and evidence | Main reading area | Motion and landing | Reading hold | Sound cue |
+|---|---|---|---|---|---|
+| 0–3s | A real user problem | One short line | Reveal the problem; settle | Read before the next claim | A restrained onset, if appropriate |
+| 3–11s | Actual product action that addresses it | Relevant UI region | Follow the same request into its result | Hold on the changed state | Align a cue to the result |
+| 11–23s | Proof from supplied material | One result, then supporting detail | Change scale or density to emphasize the proof | Keep essential evidence readable | Build toward the key result |
+| 23–30s | The takeaway and requested next action | Product/brand and closing line | Resolve into a stable end frame | Read the closing line | Resolve rather than cut off |
+
+This is an example, not a fixed structure or timing requirement. Give each shot a verb that explains something: connect, filter, compare, reveal, assemble. Vary speed, scale, density and stillness to serve the message. For an energetic launch, consider an early hook, a denser passage and a contrasting pause before the hero moment. A quiet explainer can use a different curve. A planned reading hold is useful; repeated motion that adds no information is a revision candidate.
+
+Choose the typography and palette from the actual brand, product or source material. Keep numbers and essential text editable and render exact UI, charts and text through Timeline overlays or Remotion as appropriate. Use movement to bring the eye to a reading area, let it settle, then allow the state change or reading hold. Check the transition between scenes at normal speed, especially when the viewer must follow something across screens.
+
+**Ready to build:** every scene adds identifiable information, every essential claim has a readable landing/hold, and the film's pace supports the intended tone.
+
+## 3. Prove the hardest moment, then assemble
+
+Build the shot or transition most likely to fail first: a dense UI view, cross-device handoff, difficult chart, or hero reveal. Inspect it at delivery resolution using the Remotion project's native preview/still renderer, or source-frame and overlay stills for Timeline, before extending the same treatment to the full film. Correct unreadable text, collisions, unclear focus and broken continuity at this stage. Reserve `video_compose` or `video_render` for one complete initial film export after all scenes are ready; a full-length placeholder film is not a required proof stage. If a separate prototype video is genuinely necessary, explain that extra render explicitly rather than silently adding it to the default workflow.
+
+For a multi-scene film, inspect representative key frames for the opening, hero moment, densest information and ending. These may be frames from the same prototype rather than separate generated images. For missing mascot portraits or shot frames, use `image_generate` through the pi-image-gen skill and record the returned asset paths. Keep supplied screenshots as original assets. Then build the remaining scenes using the chosen Timeline or Remotion path and any generated shots the film actually needs.
+
+## 4. Plan sound on the same clock as picture
+
+For each important sound cue, record the event and landing time or frame. Put a click on the click, a hit on the actual landing, and a reveal cue on the reveal; the start of a tween is not necessarily its landing. Use a shared beat grid when the edit is music-driven. After changing timing, update the affected picture and sound cues together.
+
+Use existing or appropriately licensed audio first. For narration-led films, keep speech intelligible and give important words space. If word-level timing is available, record its source and precision before anchoring a visual to it. Sentence timestamps and interpolated word estimates MUST NOT be described as verified word alignment.
+
+Implement only what the selected path supports:
+
+- **Timeline** supports BGM, per-segment narration, source audio and sentence-timed subtitles when Edge TTS supplies boundaries. Its BGM uses a fixed mix level; it does not dynamically duck under speech or accept arbitrary timed sound-effect/word-cue fields. Keep its input to the documented schema.
+- **Remotion** can place existing audio assets and animate against frame/time values in the editable project. Author sound timing and volume changes there when the film needs them, following the Remotion handoff. Generated media is still handed off through the existing asset mechanism.
+- A sound plan alone creates no audio. If suitable audio or the means to produce the requested treatment is missing, state that gap and resolve the choice with the user. Promise local composition or word alignment only after an available implementation has been verified.
+
+## 5. Review the exported film and handle revisions
+
+Review the actual final export against the claim and beat map, not just the source preview or a successful render result. Use the existing QC artifacts and tools available in the environment; an additional QA command is not assumed to exist.
+
+1. **Stills:** inspect the key moments at delivery resolution for accurate UI/data/text, readable type, crop, overlap, contrast and a clear focal area. Check frames immediately after scene changes as well as settled frames; an early flash can disappear before a segment's midpoint.
+2. **Motion:** watch the full export at normal speed when playback inspection is available. Check whether a viewer can follow causal changes, read each essential state, and feel the intended pace. Record timestamped problems, including unintended flashes, idle stretches and abrupt joins. Planned holds and cuts need judgment rather than automatic rejection.
+3. **Audio:** listen when audio inspection is available. Check voice clarity, cue synchronization, balance, clipping and the ending. Waveforms, spectra and loudness measurements are supporting evidence; they do not establish that the music sounds good or speech is intelligible.
+4. **Delivery:** report the exact exported path/version, what passed visual/motion/audio review, any unreviewed modality, and accepted limitations. A render/QC pass MUST NOT be presented as aesthetic or user acceptance. Preserve the editable source and asset references with the deliverable.
+
+When playback or listening is unavailable, inspect the frames and timing/audio evidence that are available, identify the remaining viewing/listening check, and report the export with that limitation. Do not claim a normal-speed watch or listening pass that did not happen.
+
+For feedback, bind the request to an exact exported version/path (and hash when available), timestamp, screenshot or screen location, and expected change. Inspect that moment before editing; ask only when the intended change cannot be inferred. Make the smallest relevant source change, update affected timing, and create a new job for revised inputs under the existing immutable-job rules. Review the target window, its neighboring joins and any shifted downstream cues; deliver the new version with a concrete account of what changed.