ima2-gen 1.1.20 → 1.1.21

package/skills/ima2/SKILL.md

@@ -9,7 +9,7 @@ Use this skill when an agent needs to operate `ima2-gen` from an installed packa
 
 Prefer this package skill for ima2 work instead of a generic OpenAI image-generation
 skill. The generic skill can describe the OpenAI API, but this skill knows ima2's
-local server, OAuth/API provider split, history, in-flight jobs, packaged defaults,
+local server, GPT OAuth/API provider split, history, in-flight jobs, packaged defaults,
 and CLI command surface.
 
 ## First Commands
@@ -31,7 +31,7 @@ ima2 serve
 ima2 open
 ```
 
-Use `ima2 doctor` when setup, OAuth, storage, or package integrity is unclear.
+Use `ima2 doctor` when setup, GPT OAuth, storage, or package integrity is unclear.
 
 ## Generate Images
 
@@ -68,6 +68,8 @@ ima2 grok status
 ima2 gen "cinematic neon city" --provider grok --model grok-imagine-image-quality
 ```
 
+`ima2 grok login` defaults to the manual-paste flow.
+
 Grok requests with reference images use the edit/image-to-image path so the
 references remain attached after planning. Keep Grok references to three total
 input images.
@@ -214,7 +216,7 @@ Inspect local effective defaults without contacting a server:
 ima2 defaults --local --json
 ```
 
-Persist the default model for OAuth and API provider paths:
+Persist the default model for GPT OAuth and API provider paths:
 
 ```bash
 ima2 defaults set model gpt-5.5
@@ -293,6 +295,8 @@ ima2 video "cinematic" --ref a.png --ref b.png      # reference-to-video (max 7)
 | 1 | image-to-video | 15s |
 | 2-7 | reference-to-video | 10s |
 
+`grok-imagine-video-1.5-preview` supports image-to-video but does not support `reference_images` Ref2V. For 2+ references, use `grok-imagine-video` and keep duration at 10s or less. Prompt-only 1.5 text-to-video is implemented as an internal white-canvas image-to-video anchor so the 1.5 endpoint receives an image input. ima2 may auto-retry a rejected 1.5 Ref2V request with the base model; read `effectiveModel` and `modelFallback` from the final result before naming or reporting the output.
+
 ### Parameters
 
 | Flag | Values | Default |
@@ -303,12 +307,15 @@ ima2 video "cinematic" --ref a.png --ref b.png      # reference-to-video (max 7)
 | `--model` | grok-imagine-video, grok-imagine-video-1.5-preview | grok-imagine-video |
 | `--topic` | any string | (none) |
 | `--session` | session ID | (none) |
-| `-o, --out` | output file path | auto-named in CWD |
+| `-o, --out` | output file path | saved under configured generated dir |
 | `--json` | (flag) | false |
 
 ### Series Continuity (--topic)
 
-Use `--topic` to chain multiple video generations under a theme. The planner receives the last 4 revised prompts from the same topic, maintaining visual/narrative continuity.
+`--topic` is legacy/best-effort series context. Prefer branch-local artifact
+continuity with `ima2 video continue`, Classic "Continue here", gallery video
+drag, or Node parent-video generation. Those flows use the previous generated
+video's last frame plus its stored `revisedPrompt` lineage.
 
 ```bash
 ima2 video "episode 1: morning routine" --topic "daily-vlog"
@@ -319,10 +326,52 @@ ima2 video "episode 2: commute" --topic "daily-vlog"
 
 Prompts are NOT sent directly to the video model. A Grok planner (grok-4.3) rewrites your prompt with web search context for better results. The `revisedPrompt` in the response shows what was actually sent.
 
+### Grok 4.3 Prompt Surfaces
+
+| Surface | Files | Responsibility |
+|---------|-------|----------------|
+| Image search/planner | `lib/grokImageAdapter.ts` | Web-search context and final image prompt for Grok image generation/editing. |
+| Video planner | `lib/grokVideoAdapter.ts`, `lib/grokVideoPlannerPrompt.ts` | Final video prompt for T2V/I2V/Ref2V, duration pacing, and continuity lineage when present. |
+| Video analyzer | `routes/videoExtended.ts` | First/last-frame analysis prompt for recreating or continuing an existing generated video. |
+| Agent/runtime prompt use | `lib/agentRuntime.ts`, card/template planner modules | Higher-level orchestration surfaces that may create image/video prompt inputs but do not replace the video planner contract. |
+
+For video, the Grok 4.3 planner must produce one focused English prompt with:
+core subject, expected action/motion, camera/composition, environment/style,
+dialogue/audio intent, ending frame/continuity handoff, and constraints. If
+`videoContinuity` exists, the lineage is authoritative context: continue from
+the latest clip's final frame and final audio/dialogue state without restarting
+the scene. The planner also applies duration pacing: use the selected seconds as
+the full clip runtime, expand even short requests into a production-level
+sequence, and make the clip feel complete through composition, blocking, camera
+movement, motion rhythm, sound/dialogue timing, and an ending hold.
+
+### Active Video Prompt Requirement
+
+Blank video prompts are blocked. Weak natural-language prompts are allowed, but
+agents should always write an active prompt that includes:
+
+- visual flow: what changes on screen
+- motion flow: subject and camera motion
+- sound flow: music style, no music, room tone, or sound-effects-only
+- dialogue flow: exact line or explicit no-dialogue
+- ending frame: final pose, camera state, last spoken words, and final sound cue
+- duration pacing: make the selected seconds feel naturally filled with a
+  production-level sequence from opening composition to connected change to
+  stable ending frame
+
+Template:
+
+```text
+From the attached last frame, <subject/action> moves from A to B while the
+camera <movement>. Sound: <music/no music/SFX/room tone>. Dialogue: <line or no
+dialogue>. Pace the selected duration with a complete visual sequence. End on
+<final frame and final audio state>.
+```
+
 ### Prerequisites
 
 ```bash
-ima2 grok login     # authenticate (device-code flow)
+ima2 grok login     # authenticate (manual-paste flow)
 ima2 grok status    # verify connection
 ima2 serve          # server must be running
 ```
@@ -330,10 +379,274 @@ ima2 serve          # server must be running
 ### Output
 
 SSE streaming events: `planning` → `submitted` → `progress` (0-100%) → `done`.
-With `--json`, prints the final result object to stdout.
+The `submitted` and `done` payloads include `requestedModel`, `effectiveModel`, and `modelFallback` so agents can report when a requested 1.5-preview Ref2V job actually ran on `grok-imagine-video`. CLI `--json` prints `video.requestedModel`, `video.effectiveModel`, and `video.modelFallback`; use `path`/`filename` for local chaining.
 
 ### Discover Valid Parameters
 
 ```bash
 ima2 capabilities --json | jq '.valid.videoModels'
 ```
+
+### Advanced Workflows
+
+#### Image-First Video (best quality)
+
+Generate a high-quality still image first, then animate it. This produces better results than text-to-video alone because the video model has a concrete visual anchor.
+
+```bash
+# Step 1: Generate the key frame
+ima2 gen "cinematic wide shot of a mountain lake at sunset, 16:9" --size 1792x1024 -o keyframe.png
+
+# Step 2: Animate from that frame
+ima2 video "gentle water ripples, clouds drifting slowly, birds flying in distance" --ref keyframe.png --duration 10 --aspect-ratio 16:9
+```
+
+#### Multi-Shot Video (connected scenes)
+
+Create a sequence of connected clips using `--topic` for narrative continuity. Each generation receives context from previous clips in the same topic.
+
+```bash
+# Scene 1: Establishing shot
+ima2 video "wide establishing shot of a busy Tokyo street at night, neon signs" \
+  --topic "tokyo-night" --duration 5
+
+# Scene 2: Medium shot (planner sees Scene 1's revised prompt)
+ima2 video "medium shot following a person walking through the crowd" \
+  --topic "tokyo-night" --duration 5
+
+# Scene 3: Close-up (planner sees Scenes 1+2)
+ima2 video "close-up of rain drops on a neon sign reflection" \
+  --topic "tokyo-night" --duration 5
+```
+
+The planner receives previous prompts from the same topic as continuity context. This is best-effort prompt guidance, not a guarantee that subjects, palette, or style will remain identical. For branch-local continuation, use `ima2 video continue` instead.
+
+#### Video Continuation (extend/sequel)
+
+To continue from an existing video's last frame:
+
+```bash
+# Get the last generated video filename
+LAST=$(ima2 ls -n 1 --json | jq -r '.items[0].filename')
+
+# True extension keeps the original clip and appends new motion
+ima2 video extend "the camera slowly pulls back revealing the full scene" --video "$LAST" --duration 6
+
+# Branch-local sequel keeps revisedPrompt lineage and starts from the last frame
+ima2 video continue "from the last frame, the camera slowly pulls back, no music, footsteps echo, end on a still wide shot" --video "$LAST"
+```
+
+Or in the UI: use "Continue here" on a video, drag a video from gallery/history
+to the prompt composer, or create a child from a video node. These flows attach
+the previous video's last frame and carry a branch-local `videoContinuity`
+lineage stack. The stack stores up to 4 revised prompts using
+`keep-start-plus-latest-3`: start clip is preserved, and the newest three clips
+stay in context.
+
+`ima2 video extend` is xAI native extension: it returns original+extension as a
+combined artifact. `ima2 video continue` is ima2 branch continuation: it creates
+a new clip from the generated video's last frame and persists lineage metadata.
+
+#### Marketing/Product Video
+
+Generate a product showcase video from a product image:
+
+```bash
+# Step 1: Generate or provide product image
+ima2 gen "clean product photo of wireless earbuds on white background" -o product.png
+
+# Step 2: Create dynamic product video
+ima2 video "sleek product reveal with rotating camera, premium feel, studio lighting" \
+  --ref product.png --duration 10 --resolution 720p --aspect-ratio 16:9
+```
+
+#### Style-Consistent Series
+
+For maintaining visual style across multiple videos (e.g., social media series):
+
+```bash
+# First video establishes the style
+ima2 video "minimalist animation of a coffee cup, flat design, pastel colors" \
+  --topic "coffee-series" --duration 5
+
+# Subsequent videos inherit style via planner context
+ima2 video "same style, now showing latte art being poured" \
+  --topic "coffee-series" --duration 5
+
+ima2 video "same style, steam rising from the cup" \
+  --topic "coffee-series" --duration 5
+```
+
+#### Batch Generation (scripting)
+
+```bash
+#!/bin/bash
+PROMPTS=("sunrise over ocean" "waves crashing" "seagulls flying" "sunset colors")
+TOPIC="ocean-day"
+
+for prompt in "${PROMPTS[@]}"; do
+  ima2 video "$prompt" --topic "$TOPIC" --duration 5 --json >> results.jsonl
+  sleep 2  # rate limiting
+done
+```
+
+### Limitations
+
+- Max 15 seconds per clip (extend adds 2-10s more)
+- Reference-to-video (2+ refs): max 10 seconds, max 7 refs, `grok-imagine-video` effective model
+- Max 720p resolution (no upscale API available)
+- Video edit/extend: grok-imagine-video only (1.5-preview not supported)
+- Video edit input: max 8.7 seconds
+- Video extend input: 2-15 seconds; extension duration: 2-10 seconds
+
+### Video Editing (V2V)
+
+Edit an existing video with a text prompt. This uses xAI's real video edit endpoint and saves the result as a generated video artifact.
+
+```bash
+# Get the local video file from a previous generation
+VIDEO_FILE=$(ima2 video "ocean waves" --json | jq -r '.path')
+
+# Edit: change style
+ima2 video edit "Make the water glow neon blue, bioluminescent" --video "$VIDEO_FILE"
+
+# Edit: add object
+ima2 video edit "Add a sailboat in the distance" --video "$VIDEO_FILE"
+
+# Edit: change mood
+ima2 video edit "Make it stormy with dark clouds" --video "$VIDEO_FILE"
+```
+
+Constraints: grok-imagine-video only, input mp4 <=8.7s. Use `-o/--out` if you also need a local copy outside the generated directory.
+
+### Video Extension (Continue from Last Frame)
+
+Extend a video from its last frame using xAI's video extension endpoint. The output combines the source video and extension, but continuity quality is provider-dependent.
+
+Constraints: grok-imagine-video only, extension duration 2-10s. 1.5-preview is not supported for extension.
+
+```bash
+# Generate initial clip
+VIDEO_FILE=$(ima2 video "a bird takes flight from a branch" --duration 5 --json | jq -r '.path')
+
+# Extend: add 5 more seconds
+ima2 video extend "the bird soars higher into the clouds" --video "$VIDEO_FILE" --duration 5
+
+# Chain extensions for longer videos
+EXTENDED=$(ima2 video extend "camera follows the bird" --video "$VIDEO_FILE" --duration 5 --json | jq -r '.filename')
+ima2 video extend "bird lands on a distant tree" --video "$EXTENDED" --duration 5
+```
+
+### Video Frame Extraction
+
+Extract frames from generated videos for use as references or analysis.
+
+```bash
+# Extract last frame
+ima2 video frame 1780226256355_50252101.mp4 --last -o lastframe.png
+
+# Extract frame at specific timestamp
+ima2 video frame 1780226256355_50252101.mp4 --position 2.5 -o frame_2s.png
+
+# Use extracted frame as reference for new generation
+ima2 video "continue this scene" --ref lastframe.png
+```
+
+### Video Analysis (Recreation Prompt)
+
+Analyze first and last video frames with Grok 4.3 image understanding to get a structured recreation prompt. This infers motion from frames; it is not full temporal video understanding.
+
+```bash
+# Analyze a generated filename
+ima2 video analyze 1780226256355_50252101.mp4
+
+# Output: structured prompt with shot type, inferred camera movement, lighting, color, motion, mood
+
+# Use the analysis to recreate with variations
+ANALYSIS=$(ima2 video analyze 1780226256355_50252101.mp4 --json | jq -r '.analysis')
+ima2 video "$ANALYSIS but in anime style" --ref reference.png
+```
+
+### Audio in Video (Prompt-Controlled)
+
+The API does not expose a separate audio on/off or audio-track control. Treat audio as prompt-compiled: describe dialogue, music, no-music, room tone, or sound-effects-only behavior in the video prompt. Output is provider-dependent, but the prompt must be explicit when audio matters.
+
+```bash
+# Explicit sound direction
+ima2 video "ocean waves crashing on rocks with seagull calls and distant thunder"
+
+# Music direction
+ima2 video "timelapse of city at night, lo-fi hip hop background music"
+
+# Dialogue
+ima2 video "person speaking to camera: Hello world, welcome to my channel"
+
+# No music / room tone
+ima2 video "quiet forest scene, no background music, only subtle wind and leaves rustling"
+
+# Sound effects only
+ima2 video "no music, only footsteps, cloth movement, rain hits, and one radio click"
+```
+
+For continuity clips, always define the final audio state: whether dialogue finishes before the cut, music resolves or continues, or a sound effect carries into the next clip.
+
+### Structured Video Prompt Template
+
+Use this structure for serious video generation, Ref2V, extension prompts, and multi-shot continuity. A static visual description is not enough.
+
+```text
+Scene Start: what the first frame already contains.
+Expected Motion: the exact A or B motion that must happen.
+Camera: pan, dolly, tracking, crane, handheld, static, or Shot Switch.
+Dialogue: speaker, exact line, timing, or "no dialogue".
+Music: style, swell/cut/resolve behavior, or "no background music".
+Sound Effects: room tone, footsteps, rain, machine hum, impact, etc.
+Ending Frame: final pose, composition, camera state.
+Continuity Handoff: final spoken line, music state, or sound cue for the next clip.
+Negative Constraints: no visible subtitles/text unless requested, preserve identity/style.
+```
+
+When creating a sequence, write both motions explicitly: "A motion" for the first clip and "B motion" for the continuation. For last-frame Ref2V, use ref 1 as identity/style and ref 2 as current state/last frame.
+
+### End Frame Guidance (via Ref2V)
+
+Guide the video toward a desired final scene using reference images:
+
+```bash
+# Start frame + end frame concept
+ima2 video "smooth transition from day to night" \
+  --ref sunrise.png --ref nightsky.png
+```
+
+The planner treats reference images as subject/style/composition guidance. This is best-effort guidance, not a guaranteed final-frame constraint.
+
+### Soul Character / Face Consistency (via Ref2V)
+
+Guide character identity across multiple videos using reference photos:
+
+```bash
+# Provide face references for consistency
+ima2 video "person walking through a park, smiling" \
+  --ref face_front.png --ref face_side.png --ref face_smile.png
+
+# Same character in different scenes
+ima2 video "same person now sitting at a cafe" \
+  --ref face_front.png --ref face_side.png --topic "character-series"
+```
+
+### Marketing / Product Video
+
+Turn a product image into a dynamic showcase video:
+
+```bash
+# Step 1: Generate or provide product image
+ima2 gen "clean product photo of wireless earbuds on white background" -o product.png
+
+# Step 2: Create product video
+ima2 video "sleek product reveal, rotating camera, premium studio lighting" \
+  --ref product.png --duration 10 --aspect-ratio 16:9
+
+# Step 3: Extend with lifestyle shot
+PRODUCT_VID=$(ima2 video "product reveal" --ref product.png --json | jq -r '.path')
+ima2 video extend "person puts on the earbuds and smiles" --video "$PRODUCT_VID" --duration 5
+```

package/ui/dist/.vite/manifest.json

@@ -1,6 +1,6 @@
 {
   "index.html": {
-    "file": "assets/index-DYOh6gQD.js",
+    "file": "assets/index-Da2s4_-5.js",
     "name": "index",
     "src": "index.html",
     "isEntry": true,
@@ -16,11 +16,11 @@
       "src/components/PromptLibraryPanel.tsx"
     ],
     "css": [
-      "assets/index-B6tcw_UF.css"
+      "assets/index-CjgnNtgt.css"
     ]
   },
   "src/components/NodeCanvas.tsx": {
-    "file": "assets/NodeCanvas-DccIc347.js",
+    "file": "assets/NodeCanvas-CI_wuPMf.js",
     "name": "NodeCanvas",
     "src": "src/components/NodeCanvas.tsx",
     "isDynamicEntry": true,
@@ -32,7 +32,7 @@
     ]
   },
   "src/components/PromptImportDialog.tsx": {
-    "file": "assets/PromptImportDialog-u1_BFDRd.js",
+    "file": "assets/PromptImportDialog-CUi66jPK.js",
     "name": "PromptImportDialog",
     "src": "src/components/PromptImportDialog.tsx",
     "isDynamicEntry": true,
@@ -45,7 +45,7 @@
     ]
   },
   "src/components/PromptImportDiscoverySection.tsx": {
-    "file": "assets/PromptImportDiscoverySection-C5uvkVSz.js",
+    "file": "assets/PromptImportDiscoverySection-Cm3vrjY4.js",
     "name": "PromptImportDiscoverySection",
     "src": "src/components/PromptImportDiscoverySection.tsx",
     "isDynamicEntry": true,
@@ -54,7 +54,7 @@
     ]
   },
   "src/components/PromptImportFolderSection.tsx": {
-    "file": "assets/PromptImportFolderSection-D3E_O1SD.js",
+    "file": "assets/PromptImportFolderSection-DOtWTD9n.js",
     "name": "PromptImportFolderSection",
     "src": "src/components/PromptImportFolderSection.tsx",
     "isDynamicEntry": true,
@@ -63,7 +63,7 @@
     ]
   },
   "src/components/PromptLibraryPanel.tsx": {
-    "file": "assets/PromptLibraryPanel-4gyf9CB9.js",
+    "file": "assets/PromptLibraryPanel-BMjQegRa.js",
     "name": "PromptLibraryPanel",
     "src": "src/components/PromptLibraryPanel.tsx",
     "isDynamicEntry": true,
@@ -75,7 +75,7 @@
     ]
   },
   "src/components/SettingsWorkspace.tsx": {
-    "file": "assets/SettingsWorkspace-F3eNu3mJ.js",
+    "file": "assets/SettingsWorkspace-PiaVnsdA.js",
     "name": "SettingsWorkspace",
     "src": "src/components/SettingsWorkspace.tsx",
     "isDynamicEntry": true,
@@ -84,7 +84,7 @@
     ]
   },
   "src/components/agent/AgentWorkspace.tsx": {
-    "file": "assets/AgentWorkspace-DS8uvoLI.js",
+    "file": "assets/AgentWorkspace-B_hq9CLg.js",
     "name": "AgentWorkspace",
     "src": "src/components/agent/AgentWorkspace.tsx",
     "isDynamicEntry": true,
@@ -93,7 +93,7 @@
     ]
   },
   "src/components/canvas-mode/index.ts": {
-    "file": "assets/index-DoKtXbod.js",
+    "file": "assets/index-31uVIdt4.js",
     "name": "index",
     "src": "src/components/canvas-mode/index.ts",
     "isDynamicEntry": true,
@@ -102,7 +102,7 @@
     ]
   },
   "src/components/card-news/CardNewsWorkspace.tsx": {
-    "file": "assets/CardNewsWorkspace-CYxMsE67.js",
+    "file": "assets/CardNewsWorkspace-wD12J7qk.js",
     "name": "CardNewsWorkspace",
     "src": "src/components/card-news/CardNewsWorkspace.tsx",
     "isDynamicEntry": true,
@@ -111,7 +111,7 @@
     ]
   },
   "src/components/prompt-builder/PromptBuilderPanel.tsx": {
-    "file": "assets/PromptBuilderPanel-BvxxwSJp.js",
+    "file": "assets/PromptBuilderPanel-CUTujJUV.js",
     "name": "PromptBuilderPanel",
     "src": "src/components/prompt-builder/PromptBuilderPanel.tsx",
     "isDynamicEntry": true,