@sogni-ai/sogni-intelligence-client 3.24.2 → 3.24.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/client/SogniClientWrapper.d.ts +1 -0
- package/dist/client/SogniClientWrapper.d.ts.map +1 -1
- package/dist/client/SogniClientWrapper.js +19 -0
- package/dist/client/SogniClientWrapper.js.map +1 -1
- package/dist/contracts/data/promptContracts.d.ts.map +1 -1
- package/dist/contracts/data/promptContracts.js +3 -5
- package/dist/contracts/data/promptContracts.js.map +1 -1
- package/dist/contracts/imagePrompt.d.ts.map +1 -1
- package/dist/contracts/imagePrompt.js +2 -1
- package/dist/contracts/imagePrompt.js.map +1 -1
- package/dist/contracts/toolPromptMarkers.d.ts +1 -1
- package/dist/contracts/toolPromptMarkers.d.ts.map +1 -1
- package/dist/contracts/toolPromptMarkers.js +1 -1
- package/dist/contracts/toolPromptMarkers.js.map +1 -1
- package/dist/openai-tools/_manifests.generated.d.ts.map +1 -1
- package/dist/openai-tools/_manifests.generated.js +25 -62
- package/dist/openai-tools/_manifests.generated.js.map +1 -1
- package/dist/openai-tools/generation-tools.json +25 -62
- package/dist/public-skill-runtime/index.d.ts.map +1 -1
- package/dist/public-skill-runtime/index.js +108 -31
- package/dist/public-skill-runtime/index.js.map +1 -1
- package/dist/schemas/tools/animate_photo.schema.json +8 -8
- package/dist/schemas/tools/generate_video.schema.json +6 -6
- package/dist/schemas/tools/sound_to_video.schema.json +1 -1
- package/dist/schemas/tools/video_to_video.schema.json +10 -47
- package/dist/tools/definitions/animate-photo/definition.js +1 -1
- package/dist/tools/definitions/animate-photo/definition.js.map +1 -1
- package/dist/tools/definitions/generate-video/definition.js +3 -3
- package/dist/tools/definitions/generate-video/definition.js.map +1 -1
- package/dist/tools/definitions/sound-to-video/definition.js +1 -1
- package/dist/tools/definitions/sound-to-video/definition.js.map +1 -1
- package/dist/tools/definitions/video-to-video/definition.d.ts.map +1 -1
- package/dist/tools/definitions/video-to-video/definition.js +11 -39
- package/dist/tools/definitions/video-to-video/definition.js.map +1 -1
- package/dist/tools/shared/modelRegistry.d.ts.map +1 -1
- package/dist/tools/shared/modelRegistry.js +0 -1
- package/dist/tools/shared/modelRegistry.js.map +1 -1
- package/dist-esm/client/SogniClientWrapper.js +19 -0
- package/dist-esm/client/SogniClientWrapper.js.map +1 -1
- package/dist-esm/contracts/data/promptContracts.js +3 -5
- package/dist-esm/contracts/data/promptContracts.js.map +1 -1
- package/dist-esm/contracts/imagePrompt.js +2 -1
- package/dist-esm/contracts/imagePrompt.js.map +1 -1
- package/dist-esm/contracts/toolPromptMarkers.js +1 -1
- package/dist-esm/contracts/toolPromptMarkers.js.map +1 -1
- package/dist-esm/openai-tools/_manifests.generated.js +25 -62
- package/dist-esm/openai-tools/_manifests.generated.js.map +1 -1
- package/dist-esm/openai-tools/generation-tools.json +25 -62
- package/dist-esm/public-skill-runtime/index.js +108 -31
- package/dist-esm/public-skill-runtime/index.js.map +1 -1
- package/dist-esm/schemas/tools/animate_photo.schema.json +8 -8
- package/dist-esm/schemas/tools/generate_video.schema.json +6 -6
- package/dist-esm/schemas/tools/sound_to_video.schema.json +1 -1
- package/dist-esm/schemas/tools/video_to_video.schema.json +10 -47
- package/dist-esm/tools/definitions/animate-photo/definition.js +1 -1
- package/dist-esm/tools/definitions/animate-photo/definition.js.map +1 -1
- package/dist-esm/tools/definitions/generate-video/definition.js +3 -3
- package/dist-esm/tools/definitions/generate-video/definition.js.map +1 -1
- package/dist-esm/tools/definitions/sound-to-video/definition.js +1 -1
- package/dist-esm/tools/definitions/sound-to-video/definition.js.map +1 -1
- package/dist-esm/tools/definitions/video-to-video/definition.js +12 -40
- package/dist-esm/tools/definitions/video-to-video/definition.js.map +1 -1
- package/dist-esm/tools/shared/modelRegistry.js +0 -1
- package/dist-esm/tools/shared/modelRegistry.js.map +1 -1
- package/package.json +3 -3
|
@@ -681,7 +681,7 @@ exports.generationToolsManifest = {
|
|
|
681
681
|
"type": "function",
|
|
682
682
|
"function": {
|
|
683
683
|
"name": "generate_video",
|
|
684
|
-
"description": "Generate a video from text or Seedance multimodal references. LTX 2.5 is the default and generates audio natively; LTX 2.3 remains available as rollback and also generates audio natively (dialogue, sounds, ambient music) — describe audio in the prompt. If the user provides exact speech, include it in double quotes; if they only imply speech, describe the performance and voice without inventing quoted words. Never use placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". PERSONA VOICE: Only when the user explicitly asks to use/clone a registered persona voice clip, call resolve_personas first, then set voicePersonaName to select which persona's voice clip to use. Do not set voicePersonaName for ordinary character dialogue or inferred voices; describe those voices in the prompt for native LTX audio. For cross-persona narration (e.g. David narrates a video of Aleyna), resolve both personas and set voicePersonaName to the narrator only if that registered voice was requested. Persona voice requires ltx23 because LTX 2.5 has no compatible ID-LoRA and WAN 2.2 does not support voice identity. For non-Seedance syncing to a specific song or audio track, use sound_to_video instead. For non-Seedance animation from a locked source photo, use animate_photo. Do NOT use for My Personas unless generating a Seedance reference-based video — standard persona videos use resolve_personas → edit_image → animate_photo. SEEDANCE DEFAULT: For seedance2, seedance2-mini, or seedance2-5, default to exactly one video (4-15s on 2.0/Mini, 4-30s on seedance2-5) unless the user explicitly asks for multiple separate outputs. Multiple beats, shots, or scene descriptions in one up-to-15s Seedance prompt are still one video. If the user requests one continuous Seedance video longer than 15s, prefer seedance2-5, which renders up to 30s in a single call; beyond 30s (or on 2.0/Mini) preserve the requested total duration in the prompt/context and let chat orchestration split it into supported segment renders and stitch them instead of clamping it to a short excerpt. Uploaded/generated storyboard, shot-sheet, or trailer-concept images used as Seedance references should become one Seedance generate_video call by default; do not extract panels with edit_image and do not animate the storyboard sheet with LTX unless the user explicitly asks for separate non-Seedance clips. Seedance loose image, video, and audio references go through this tool; do not use animate_photo sourceImageIndex/frameRole/endImageIndex for Seedance. If an uploaded video is the source clip to transform, upscale, enhance, restyle, or remaster, use video_to_video with controlMode=\"seedance-v2v\" instead of generate_video referenceVideoIndices. If the uploaded audio is the primary sync target, lip-sync target, or requested as sound-to-video/audio-sync, use sound_to_video with videoModel=\"seedance2-mini\" instead of this tool unless the user asks for full Seedance. Use referenceAudioIndices here only when audio is a loose reference under an image/video-anchored Seedance shot. For Seedance, every image — first frame, last frame, or loose reference — is passed through referenceImageIndices (auto-uploaded as referenceImageUrls). Anchor frame intent in the prompt with @Image tags such as \"Use @Image1 as the opening shot reference. Begin the video with a composition, subject placement, lighting, mood, and camera framing that closely match @Image1.\" (or @Image2 as the final shot reference). For seamless-loop or \"first frame and last frame identical\" requests with a single uploaded image, anchor it explicitly as both: \"Use @Image1 as both the first frame and last frame so the video loops cleanly back to the opening composition.\" Assign each useful @Image/@Video/@Audio tag a role. APPROVED STORYBOARD PRODUCTION: When the user asks for a production workflow from an approved storyboard, the chat orchestrator should use the durable CampaignStoryboard contract: render the composite board, audit it, generate per-scene GPT Image 2 keyframes, then render Seedance scene clips and stitch them. Do not replace that with a generic storyboard-reference video unless the user asks for a fast draft. PARTIAL VIDEO EDITS: Do NOT call generate_video to re-render an existing rendered/uploaded video just to change part of it (the bumper, the intro, the end card, a single scene, the last few seconds, etc.). Use replace_video_segment for that — it preserves the unchanged portion, keeps the original audio outside the replaced window, and costs far less. Likewise use extend_video to add new time to the end without rewriting the rest. If the request is vague, ask about vision/mood/style first. Only call once you have clear creative intent. WAN 3 uses the exact selector wan3.0-video: use this tool for text-to-video or loose Image 1/Video 1/Audio 1 references; use animate_photo for native first/last frames
|
|
684
|
+
"description": "Generate a video from text or Seedance multimodal references. LTX 2.5 is the default and generates audio natively; LTX 2.3 remains available as rollback and also generates audio natively (dialogue, sounds, ambient music) — describe audio in the prompt. If the user provides exact speech, include it in double quotes; if they only imply speech, describe the performance and voice without inventing quoted words. Never use placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". PERSONA VOICE: Only when the user explicitly asks to use/clone a registered persona voice clip, call resolve_personas first, then set voicePersonaName to select which persona's voice clip to use. Do not set voicePersonaName for ordinary character dialogue or inferred voices; describe those voices in the prompt for native LTX audio. For cross-persona narration (e.g. David narrates a video of Aleyna), resolve both personas and set voicePersonaName to the narrator only if that registered voice was requested. Persona voice requires ltx23 because LTX 2.5 has no compatible ID-LoRA and WAN 2.2 does not support voice identity. For non-Seedance syncing to a specific song or audio track, use sound_to_video instead. For non-Seedance animation from a locked source photo, use animate_photo. Do NOT use for My Personas unless generating a Seedance reference-based video — standard persona videos use resolve_personas → edit_image → animate_photo. SEEDANCE DEFAULT: For seedance2, seedance2-mini, or seedance2-5, default to exactly one video (4-15s on 2.0/Mini, 4-30s on seedance2-5) unless the user explicitly asks for multiple separate outputs. Multiple beats, shots, or scene descriptions in one up-to-15s Seedance prompt are still one video. If the user requests one continuous Seedance video longer than 15s, prefer seedance2-5, which renders up to 30s in a single call; beyond 30s (or on 2.0/Mini) preserve the requested total duration in the prompt/context and let chat orchestration split it into supported segment renders and stitch them instead of clamping it to a short excerpt. Uploaded/generated storyboard, shot-sheet, or trailer-concept images used as Seedance references should become one Seedance generate_video call by default; do not extract panels with edit_image and do not animate the storyboard sheet with LTX unless the user explicitly asks for separate non-Seedance clips. Seedance loose image, video, and audio references go through this tool; do not use animate_photo sourceImageIndex/frameRole/endImageIndex for Seedance. If an uploaded video is the source clip to transform, upscale, enhance, restyle, or remaster, use video_to_video with controlMode=\"seedance-v2v\" instead of generate_video referenceVideoIndices. If the uploaded audio is the primary sync target, lip-sync target, or requested as sound-to-video/audio-sync, use sound_to_video with videoModel=\"seedance2-mini\" instead of this tool unless the user asks for full Seedance. Use referenceAudioIndices here only when audio is a loose reference under an image/video-anchored Seedance shot. For Seedance, every image — first frame, last frame, or loose reference — is passed through referenceImageIndices (auto-uploaded as referenceImageUrls). Anchor frame intent in the prompt with @Image tags such as \"Use @Image1 as the opening shot reference. Begin the video with a composition, subject placement, lighting, mood, and camera framing that closely match @Image1.\" (or @Image2 as the final shot reference). For seamless-loop or \"first frame and last frame identical\" requests with a single uploaded image, anchor it explicitly as both: \"Use @Image1 as both the first frame and last frame so the video loops cleanly back to the opening composition.\" Assign each useful @Image/@Video/@Audio tag a role. APPROVED STORYBOARD PRODUCTION: When the user asks for a production workflow from an approved storyboard, the chat orchestrator should use the durable CampaignStoryboard contract: render the composite board, audit it, generate per-scene GPT Image 2 keyframes, then render Seedance scene clips and stitch them. Do not replace that with a generic storyboard-reference video unless the user asks for a fast draft. PARTIAL VIDEO EDITS: Do NOT call generate_video to re-render an existing rendered/uploaded video just to change part of it (the bumper, the intro, the end card, a single scene, the last few seconds, etc.). Use replace_video_segment for that — it preserves the unchanged portion, keeps the original audio outside the replaced window, and costs far less. Likewise use extend_video to add new time to the end without rewriting the rest. If the request is vague, ask about vision/mood/style first. Only call once you have clear creative intent. WAN 3 uses the exact selector wan3.0-video: use this tool for text-to-video or loose Image 1/Video 1/Audio 1 references; use animate_photo for native first/last frames and sound_to_video when audio drives timing. A Wan 3 video reference conditions a new generation; it does not invoke a provider-backed edit or extend mode.",
|
|
685
685
|
"parameters": {
|
|
686
686
|
"type": "object",
|
|
687
687
|
"properties": {
|
|
@@ -717,7 +717,7 @@ exports.generationToolsManifest = {
|
|
|
717
717
|
"3:4",
|
|
718
718
|
"9:16"
|
|
719
719
|
],
|
|
720
|
-
"description": "Wan 3 only. Output ratio. Use \"adaptive\" to
|
|
720
|
+
"description": "Wan 3 only. Output ratio. Use \"adaptive\" to derive the shape from input/context media. Omit for automatic behavior."
|
|
721
721
|
},
|
|
722
722
|
"watermark": {
|
|
723
723
|
"type": "boolean",
|
|
@@ -753,7 +753,7 @@ exports.generationToolsManifest = {
|
|
|
753
753
|
"minimax-h3-r2v-turbo",
|
|
754
754
|
"wan3.0-video"
|
|
755
755
|
],
|
|
756
|
-
"description": "
|
|
756
|
+
"description": "\"ltx25\" (default): LTX 2.5 with native audio; Fast, HQ, and Pro currently use the release-validated official Distilled INT8 workflow. The Dev checkpoints are not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe. Video model. \"ltx23\": LTX 2.3 rollback with native audio. \"wan22\": quick simple motion without audio. \"minimax-h3-t2v\": standard 20-step MiniMax H3 text-to-video; \"minimax-h3-t2v-turbo\": 4-step Turbo text-to-video. Both use native audio, fixed 24fps, 5.17-15.08s, and a 768p-class 32px-grid canvas; use animate_photo for H3 image-conditioned modes. Base and Turbo T2V/I2V/FLF2V prompts use the exact ordered fields integrated_multimodal_description, overall_soundscape, and non_diegetic_music; I2V/FLF2V prepend the official alignment line. \"minimax-h3-r2v\": standard 20-step MiniMax H3 reference-to-video; \"minimax-h3-r2v-turbo\": the dedicated LightX2V 4-step Ref2VA Turbo workflow using Euler/simple and a 960x544 default. Both accept up to 9 images, 3 videos, and 3 audios (12 files total); at least one visual reference (image or video) is required and audio alone is invalid. Select references with referenceImageIndices/referenceVideoIndices/referenceAudioIndices and address them with the official <Subject N>/<Picture N>/<Video N>/<Audio N> semantics. Seedance quality is selected only by model: use \"seedance2-mini\" for Seedance 2.0 Mini or faster/lower-cost 720p iteration, and use \"seedance2\" for the full Seedance 2.0 model, explicit full-quality requests, 1080p/4K requests, or generated/uploaded storyboard images unless the user explicitly asks for a draft or Mini. Do not use Default Media Quality Fast/HQ/Pro or targetResolution to represent Seedance quality. \"seedance2-5\": Seedance 2.5, the newest Seedance generation — 480p and 720p ONLY (it cannot render 1080p or 4K), 4-30s per clip at a fixed 24 fps, native audio, first-and-last-frame conditioning, and a much larger reference budget than the 2.0 family: up to 30 images, 10 videos, and 10 audios, with up to 50 reference media files total, subject to those per-modality caps. Choose \"seedance2-5\" when the user asks for Seedance 2.5, wants a single continuous Seedance clip longer than 15s (2.5 renders up to 30s in one call instead of being split and stitched), or wants a first-and-last-frame Seedance transition. Keep \"seedance2\" for 1080p/4K requests, which Seedance 2.5 cannot satisfy. Seedance supports multimodal loose reference assets: images (up to 9), videos (up to 3), and audios (up to 3), with no more than 12 asset files total. Use @Image1/@Video1/@Audio1 style references in creative briefs when assigning roles. Assign every useful reference asset a role and prefer positive preservation constraints. If an uploaded video is the source clip to transform, upscale, enhance, restyle, or remaster, use video_to_video with controlMode=\"seedance-v2v\" instead of generate_video referenceVideoIndices. Alibaba HappyHorse 1.1 video models (third-party vendor — requires Premium Spark). Select by mode: \"happyhorse-1.1-t2v\" for text-to-video, \"happyhorse-1.1-i2v\" for image-to-video from one first-frame image, and \"happyhorse-1.1-r2v\" for reference-to-video with up to 9 reference images. Resolutions 720P and 1080P; duration 3-15 seconds at 24 fps; native synchronized audio is always generated (do not set generateAudio or negativePrompt). Supported aspect ratios: 16:9, 9:16, 1:1, 4:3, 3:4, 4:5, 5:4, 9:21, 21:9. HappyHorse 1.1 takes image references only and renders a native synchronized audio track (always on; do not set generateAudio or a negative prompt). Pick the model by mode: happyhorse-1.1-t2v for text-to-video (no reference image), happyhorse-1.1-i2v for image-to-video from a single first frame, and happyhorse-1.1-r2v for reference-to-video with 1 to 9 reference images. For r2v, tag the images in the prompt as [Image 1]…[Image 9] and assign each a clear role. HappyHorse does not accept reference videos or reference audios. \"wan3.0-video\" is Alibaba Wan 3: one canonical premium-vendor model for text, first/last frames, loose multimodal references, audio-driven generation, and document/web context. It renders 2-30s or smart duration at fixed 30 fps with optional native audio, prompt expansion control, optional watermark, 480p/720p/1080p, and adaptive/16:9/4:3/1:1/3:4/9:16 ratios. It accepts up to 10 images, 5 videos, 5 audios, one file, or one webpage. Do not send negativePrompt. Use animate_photo for native first/last frames, generate_video for text or loose references, and sound_to_video when audio drives timing. Video references provide loose conditioning for a new result, not source-video editing or extension."
|
|
757
757
|
},
|
|
758
758
|
"generateAudio": {
|
|
759
759
|
"type": "boolean",
|
|
@@ -764,21 +764,21 @@ exports.generationToolsManifest = {
|
|
|
764
764
|
"items": {
|
|
765
765
|
"type": "number"
|
|
766
766
|
},
|
|
767
|
-
"description": "Seedance or MiniMax H3
|
|
767
|
+
"description": "Seedance or MiniMax H3 r2v image references. Use negative indices for uploaded images and non-negative indices for generated image results. Seedance uses @Image tags. H3 Ref2VA requires at least one visual reference—an image or video—and uses the official <Subject N>/<Picture N> semantics; these are loose references, not locked first frames. Wan 3 loose images use Image 1, Image 2, and so on, with up to 10 images."
|
|
768
768
|
},
|
|
769
769
|
"referenceVideoIndices": {
|
|
770
770
|
"type": "array",
|
|
771
771
|
"items": {
|
|
772
772
|
"type": "number"
|
|
773
773
|
},
|
|
774
|
-
"description": "Seedance or MiniMax H3
|
|
774
|
+
"description": "Seedance or MiniMax H3 r2v loose video references. Use negative indices for uploaded videos and non-negative indices for generated video results. Seedance uses @Video tags; H3 uses <Video 1>, <Video 2>, and so on in selection order. A video may be the only H3 Ref2VA visual reference. Do not use this for source-video transforms; use video_to_video instead. Wan 3 loose videos use Video 1, Video 2, and so on, with up to 5 videos."
|
|
775
775
|
},
|
|
776
776
|
"referenceAudioIndices": {
|
|
777
777
|
"type": "array",
|
|
778
778
|
"items": {
|
|
779
779
|
"type": "number"
|
|
780
780
|
},
|
|
781
|
-
"description": "Seedance or MiniMax H3
|
|
781
|
+
"description": "Seedance or MiniMax H3 r2v loose audio references. Use negative indices for uploaded audio files and non-negative indices for generated audio results. Seedance uses @Audio tags; H3 uses <Audio 1>, <Audio 2>, and so on in selection order. H3 Ref2VA audio may accompany an image or video, but audio alone is invalid. Wan 3 loose audios use Audio 1, Audio 2, and so on, with up to 5 audios."
|
|
782
782
|
},
|
|
783
783
|
"width": {
|
|
784
784
|
"type": "number",
|
|
@@ -1156,13 +1156,13 @@ exports.generationToolsManifest = {
|
|
|
1156
1156
|
"type": "function",
|
|
1157
1157
|
"function": {
|
|
1158
1158
|
"name": "animate_photo",
|
|
1159
|
-
"description": "Animate a photo into video with motion, audio, and dialogue using LTX 2.5 by default, LTX 2.3 as rollback, or WAN 2.2. Do NOT use this tool for seedance2, seedance2-mini, or seedance2-5. Seedance media references — including Seedance 2.5 first-and-last-frame requests — must go through generate_video with referenceImageIndices/referenceVideoIndices/referenceAudioIndices and @Image/@Video/@Audio role text in the prompt; for seamless-loop Seedance requests with one uploaded image, the prompt should anchor it as both the first frame and last frame. LTX/WAN NOTE: uploaded audio files are not loose references for ltx25/ltx23/wan22; use sound_to_video when uploaded audio is the primary sync target. DANCE REQUESTS (\"make them dance\", \"do the X dance\"): use dance_montage — NOT this tool. LTX 2.5 and LTX 2.3 generate audio natively — describe dialogue and ambient sounds directly in the prompt (do NOT pre-generate audio for this tool). If the user provides exact speech, include it in double quotes; if they only imply speech, describe the performance and voice without inventing quoted words. Avoid placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". PERSONA VOICE: Only when the user explicitly asks to use/clone a registered persona voice clip, call resolve_personas first, then set voicePersonaName to select which persona's voice clip to use. Do not set voicePersonaName for ordinary character dialogue or inferred voices; describe those voices in the prompt for native LTX audio. For cross-persona narration (e.g. David narrates a video of Aleyna), resolve both personas and set voicePersonaName to the narrator only if that registered voice was requested. Persona voice requires ltx23 because LTX 2.5 has no compatible ID-LoRA and WAN 2.2 does not support voice identity. PERSONA PIPELINE: For persona videos, ensure an image of the persona exists before calling animate_photo. The standard pipeline is: resolve_personas → edit_image → animate_photo. If a suitable persona image already exists (user uploaded one, a prior edit_image/generate_image result, OR the user explicitly says to use the Persona image/reference photo directly), skip edit_image and animate directly. After resolve_personas, this tool can animate the injected persona image directly when that explicit direct-use instruction is given. Auto-uses the latest result image (from any prior tool) unless sourceImageIndex is set. Supports start-frame (default), end-frame, and start+end interpolation modes for LTX/WAN
|
|
1159
|
+
"description": "Animate a photo into video with motion, audio, and dialogue using LTX 2.5 by default, LTX 2.3 as rollback, or WAN 2.2. Do NOT use this tool for seedance2, seedance2-mini, or seedance2-5. Seedance media references — including Seedance 2.5 first-and-last-frame requests — must go through generate_video with referenceImageIndices/referenceVideoIndices/referenceAudioIndices and @Image/@Video/@Audio role text in the prompt; for seamless-loop Seedance requests with one uploaded image, the prompt should anchor it as both the first frame and last frame. LTX/WAN NOTE: uploaded audio files are not loose references for ltx25/ltx23/wan22; use sound_to_video when uploaded audio is the primary sync target. DANCE REQUESTS (\"make them dance\", \"do the X dance\"): use dance_montage — NOT this tool. LTX 2.5 and LTX 2.3 generate audio natively — describe dialogue and ambient sounds directly in the prompt (do NOT pre-generate audio for this tool). If the user provides exact speech, include it in double quotes; if they only imply speech, describe the performance and voice without inventing quoted words. Avoid placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". PERSONA VOICE: Only when the user explicitly asks to use/clone a registered persona voice clip, call resolve_personas first, then set voicePersonaName to select which persona's voice clip to use. Do not set voicePersonaName for ordinary character dialogue or inferred voices; describe those voices in the prompt for native LTX audio. For cross-persona narration (e.g. David narrates a video of Aleyna), resolve both personas and set voicePersonaName to the narrator only if that registered voice was requested. Persona voice requires ltx23 because LTX 2.5 has no compatible ID-LoRA and WAN 2.2 does not support voice identity. PERSONA PIPELINE: For persona videos, ensure an image of the persona exists before calling animate_photo. The standard pipeline is: resolve_personas → edit_image → animate_photo. If a suitable persona image already exists (user uploaded one, a prior edit_image/generate_image result, OR the user explicitly says to use the Persona image/reference photo directly), skip edit_image and animate directly. After resolve_personas, this tool can animate the injected persona image directly when that explicit direct-use instruction is given. Auto-uses the latest result image (from any prior tool) unless sourceImageIndex is set. Supports start-frame (default), end-frame, and start+end interpolation modes for LTX/WAN — ask the user which frame role their image should play if they mention \"end frame\", \"last frame\", or provide two images. FIRST+LAST FRAME WORKFLOW: When the user wants a non-Seedance video using two different scenes as start and end frames, prefer generating both images in a single generate_image/edit_image call with numberOfVariations=2 and Dynamic Prompts, then call animate_photo with frameRole=\"both\", sourceImageIndex=0, endImageIndex=1. If the user explicitly wants separately created frame assets, preserve that staged instruction while keeping indices correct. In frameRole=\"both\", the handler automatically inspects both images and upgrades the base prompt into a scene-aware smooth transition prompt, so your prompt should state the desired transition style, action, dialogue, and audio rather than trying to list every visible object. If the request is vague, analyze the image first and suggest 2-3 specific animation ideas tailored to what you see. Call once you have clear creative intent. N-VIDEOS PATTERN: Avoid sequential animate_photo calls for N outputs. For a single fixed source/end frame where only prompt text varies, use sourceImageIndex + numberOfVariations=N + one Dynamic Prompt branch in prompt so Sogni submits one project with multiple jobs. If the user explicitly asks for Dynamic Prompt or Dynamic Template syntax, prefer this one-project path whenever every output uses the same source/end frames and shared settings, even if they also ask to stitch the completed clips afterward. Use sourceImageIndices/prompts for multi-segment stitched non-Seedance video, different source/end assets, different audio windows, different durations/dimensions, isolated retry lifecycle, or other per-output parameters. sourceImageIndices supports up to 16 entries; there is NO 3-clip cap, so do not split one planned batch into \"first 3\" and \"remaining\" calls. For a dialogue-heavy total-duration request with no explicit per-clip duration, prefer 15-second clips on ltx25 by default (or ltx23 rollback) (30s total = 2 clips × 15s) and 10-second clips on wan22 (60s total on wan22 = 6 clips × 10s; do NOT pick 4 clips × 15s on wan22 — the wan22 worker rejects clips longer than 10s). Multi-source flavors: (A) SHARED CONTENT — when all N clips have the same dialogue/motion but different source visuals (different scenes, outfits, environments, persona looks), first generate N distinct images via ONE edit_image/generate_image call with numberOfVariations=N + Dynamic Prompts {|}, then call animate_photo with sourceImageIndices=[start..start+N-1] and a single shared `prompt`. If all segments intentionally reuse the primary uploaded image and only prompt text varies, use sourceImageIndex=-1, frameRole=\"both\" if requested, endImageIndex=-1 if requested, numberOfVariations=N, and one Dynamic Prompt branch in prompt. Each branch option must be a complete natural-language motion prompt; do not include \"clip N\", source-frame boilerplate, \"overall request context\", or instructions to follow the user request. For a long scripted/dialogue/storyboard video from a single supplied/uploaded image where each segment needs isolated exact dialogue or per-segment wiring, use sourceImageIndices=[-1,-1,...] and per-clip prompts. Only set frameRole=\"both\" and endImageIndex=-1 when the user explicitly says the same uploaded/source/original image should be both the first and last frame of every segment. If the user requests generated source images first, honor that image stage, then animate the generated result indices. When using generated scene keyframes and each clip should begin and end on its own scene image for stitching, call animate_photo with frameRole=\"both\" and sourceImageIndices=[start..end] but OMIT endImageIndex; do not set endImageIndex=-1 unless every source is the uploaded image. (B) PER-CLIP CONTENT — when source/end asset wiring or other per-output parameters differ, pass BOTH sourceImageIndices AND `prompts` (an array of N strings, one per clip) in the same single call. Each prompt must independently anchor the visible characters, scene action, camera, audio, exact screenplay-style speaker tags, and exact quoted dialogue for that segment. If you just wrote or displayed a script/table, copy the exact dialogue lines into the corresponding per-clip prompts; do not summarize them as speech activity. If using named speaker tags with any multi-person reference image or generated scene keyframe, include one explicit cast map in each prompt that binds each name to visible position, clothing, and props/actions, e.g. SPEAKER_A = left person holding a prop; SPEAKER_B = center person with tablet; SPEAKER_C = right person near table. Do not also describe the same people again as generic man/boy/girl/woman/character subjects. For screenplay, storyboard, commercial, series, or other longer-form tasks with recurring characters, preserve the same character names and repeated visual anchors in every per-clip prompt where each character appears. Use the standard single-source path (numberOfVariations only) when the user wants motion variety from a single fixed frame. Wan 3 first-frame and first+last-frame generation is supported with videoModel=\"wan3.0-video\".",
|
|
1160
1160
|
"parameters": {
|
|
1161
1161
|
"type": "object",
|
|
1162
1162
|
"properties": {
|
|
1163
1163
|
"prompt": {
|
|
1164
1164
|
"type": "string",
|
|
1165
|
-
"description": "I2V RULE: Do NOT re-describe what is visible in the input image. Focus on the transition from stillness — motion, expression changes, what happens next, camera movement, and sound.\n\nLITERAL PROMPT OVERRIDE: If the user explicitly says not to modify the prompt, or to use it exactly/verbatim/as-is, copy the identified prompt text verbatim instead of applying these construction rules unless a hard requirement is missing. Set skipPromptProcessing=true; for Seedance or Wan 3 also set expandPrompt=false.\n\nSTRUCTURE: \"[How the subject begins to move]. [What changes next]. [Camera behavior]. [Audio].\"\n\nMOTION PACING: Scale complexity to duration. <=6s: 1 main action beat + 1 simple camera move. Around 10s: 2-3 clear action beats + 1 camera move. >10s: up to 4 action beats in clear sequence. Prefer fewer readable beats over dense micro-actions, especially in short clips.\n\nBLOCKING: Use the image as the anchor and direct only meaningful layout changes. If the prompt introduces multiple moving subjects, state left/right placement, foreground/background, facing toward/away, and relative distance.\n\nACTION: One flowing paragraph. Describe motion beat by beat with temporal connectors (\"as\", \"then\", \"while\"). Specify who moves, what moves, how it moves, and what the camera does. One main thread — avoid too many actions at once or generic phrases like \"comes alive.\"\n\nDIALOGUE: Put user-provided spoken lines in double quotes. For screenplay-style or longer-form tasks, prefix each spoken line with a stable speaker tag outside the quotes, e.g. CHARACTER: \"We made it.\" Break long speech into short quoted phrases with acting beats between them (gestures, pauses, glances). If the user asks for speech but provides no exact words, describe the visible delivery, voice quality, and emotion without inventing quoted dialogue; ask only when exact wording is the point of the request. Never write placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". Show emotion through visible behavior, not labels. LTX 2.5 and LTX 2.3 generate audio natively. QUOTING RULE: ONLY use double quotes for spoken dialogue. Never quote on-screen text, overlay text, titles, captions, signs, or any visual text — describe them without quotes (e.g. bold white text reading CONGRATULATIONS overlays the lower third).\n\nAUDIO: Prompt sound intentionally — voice quality, volume, room tone, ambience, music, weather, footsteps. Include language or accent if relevant. Useful voice/volume anchors: whisper, mutter, shout, scream, energetic announcer, resonant voice with gravitas, distorted radio-style, robotic monotone, childlike curiosity.\n\nCAMERA: Cinematic terms — slow push-in, static tripod, handheld, slow arc, dolly in. Describe movement relative to subject.\n\nFor first+last-frame transitions (frameRole=\"both\"), write a concise base request for the transition style, action, dialogue, and audio. The handler will inspect both frames and expand it into a scene-aware prompt that maps visible objects and subjects between frames.\n\nFor specific characters (movies, TV): describe visual appearance — don't rely on names alone.\n\nFor complex/creative scenes (characters talking, skits), capture full creative intent — system auto-expands into detailed prompt.\n\nAVOID: Re-describing the image, vague prompts, too many actions at once, abstract emotions without visible behavior, rigid numeric constraints, readable text or logos.\n\nPOSITIVE CONSTRAINT TRANSLATION: For LTX 2.3 and WAN 2.2, the prompt field is a positive prompt. Translate user avoid/no/don't constraints into affirmative production constraints instead of copying negative phrasing. Examples: \"no people in background\" -> single subject focus with an empty background; \"no text\" -> clean blank surfaces; \"don't make it blurry\" -> crisp sharp focus; \"no weird hands\" -> natural anatomically consistent hands; \"no mouth movement, no talking, no lip syncing\" -> silent expression-only physical performance with facial motion independent of speech timing; \"don't change the room\" -> the same room and layout remain consistent; \"keep flames consistent\" -> flame and ember movement remains consistent with the source scene. Preserve exact quoted visible text or dialogue when the user explicitly requests it, and keep surrounding surfaces blank. For Dynamic Prompt batches, put these translated shared constraints before the \"{...}\" branch so every variation inherits them.\n\nWAN 2.2 (\"wan22\"): 30-150 words, subtle natural movements. Motion-only visual prompt; omit soundtrack, ambience, room tone, music, hums, sighs, spoken words, voice, and SFX cues because WAN does not generate audio.\n\nBATCH VARIATIONS: When numberOfVariations > 1, use Dynamic Prompt syntax to vary motion, camera, or atmosphere while preserving the user's specified elements. This is one Sogni project with multiple jobs, so prefer it when all outputs share the same source/end frames and generation parameters and only prompt text varies. Example: \"{gentle sway with drifting embers|slow paw wave with a tiny head tilt|small hop with soft fur motion}\"
|
|
1165
|
+
"description": "I2V RULE: Do NOT re-describe what is visible in the input image. Focus on the transition from stillness — motion, expression changes, what happens next, camera movement, and sound.\n\nLITERAL PROMPT OVERRIDE: If the user explicitly says not to modify the prompt, or to use it exactly/verbatim/as-is, copy the identified prompt text verbatim instead of applying these construction rules unless a hard requirement is missing. Set skipPromptProcessing=true; for Seedance or Wan 3 also set expandPrompt=false.\n\nSTRUCTURE: \"[How the subject begins to move]. [What changes next]. [Camera behavior]. [Audio].\"\n\nMOTION PACING: Scale complexity to duration. <=6s: 1 main action beat + 1 simple camera move. Around 10s: 2-3 clear action beats + 1 camera move. >10s: up to 4 action beats in clear sequence. Prefer fewer readable beats over dense micro-actions, especially in short clips.\n\nBLOCKING: Use the image as the anchor and direct only meaningful layout changes. If the prompt introduces multiple moving subjects, state left/right placement, foreground/background, facing toward/away, and relative distance.\n\nACTION: One flowing paragraph. Describe motion beat by beat with temporal connectors (\"as\", \"then\", \"while\"). Specify who moves, what moves, how it moves, and what the camera does. One main thread — avoid too many actions at once or generic phrases like \"comes alive.\"\n\nDIALOGUE: Put user-provided spoken lines in double quotes. For screenplay-style or longer-form tasks, prefix each spoken line with a stable speaker tag outside the quotes, e.g. CHARACTER: \"We made it.\" Break long speech into short quoted phrases with acting beats between them (gestures, pauses, glances). If the user asks for speech but provides no exact words, describe the visible delivery, voice quality, and emotion without inventing quoted dialogue; ask only when exact wording is the point of the request. Never write placeholders such as \"while speaking\", \"dialogue begins\", \"explaining\", or \"final line lands\". Show emotion through visible behavior, not labels. LTX 2.5 and LTX 2.3 generate audio natively. QUOTING RULE: ONLY use double quotes for spoken dialogue. Never quote on-screen text, overlay text, titles, captions, signs, or any visual text — describe them without quotes (e.g. bold white text reading CONGRATULATIONS overlays the lower third).\n\nAUDIO: Prompt sound intentionally — voice quality, volume, room tone, ambience, music, weather, footsteps. Include language or accent if relevant. Useful voice/volume anchors: whisper, mutter, shout, scream, energetic announcer, resonant voice with gravitas, distorted radio-style, robotic monotone, childlike curiosity.\n\nCAMERA: Cinematic terms — slow push-in, static tripod, handheld, slow arc, dolly in. Describe movement relative to subject.\n\nFor first+last-frame transitions (frameRole=\"both\"), write a concise base request for the transition style, action, dialogue, and audio. The handler will inspect both frames and expand it into a scene-aware prompt that maps visible objects and subjects between frames.\n\nFor specific characters (movies, TV): describe visual appearance — don't rely on names alone.\n\nFor complex/creative scenes (characters talking, skits), capture full creative intent — system auto-expands into detailed prompt.\n\nAVOID: Re-describing the image, vague prompts, too many actions at once, abstract emotions without visible behavior, rigid numeric constraints, readable text or logos.\n\nPOSITIVE CONSTRAINT TRANSLATION: For LTX 2.3 and WAN 2.2, the prompt field is a positive prompt. Translate user avoid/no/don't constraints into affirmative production constraints instead of copying negative phrasing. Examples: \"no people in background\" -> single subject focus with an empty background; \"no text\" -> clean blank surfaces; \"don't make it blurry\" -> crisp sharp focus; \"no weird hands\" -> natural anatomically consistent hands; \"no mouth movement, no talking, no lip syncing\" -> silent expression-only physical performance with facial motion independent of speech timing; \"don't change the room\" -> the same room and layout remain consistent; \"keep flames consistent\" -> flame and ember movement remains consistent with the source scene. Preserve exact quoted visible text or dialogue when the user explicitly requests it, and keep surrounding surfaces blank. For Dynamic Prompt batches, put these translated shared constraints before the \"{...}\" branch so every variation inherits them.\n\nWAN 2.2 (\"wan22\"): 30-150 words, subtle natural movements. Motion-only visual prompt; omit soundtrack, ambience, room tone, music, hums, sighs, spoken words, voice, and SFX cues because WAN does not generate audio.\n\nBATCH VARIATIONS: When numberOfVariations > 1, use Dynamic Prompt syntax to vary motion, camera, or atmosphere while preserving the user's specified elements. This is one Sogni project with multiple jobs, so prefer it when all outputs share the same source/end frames and generation parameters and only prompt text varies. Example: \"{gentle sway with drifting embers|slow paw wave with a tiny head tilt|small hop with soft fur motion}\"."
|
|
1166
1166
|
},
|
|
1167
1167
|
"expandPrompt": {
|
|
1168
1168
|
"type": "boolean",
|
|
@@ -1186,7 +1186,7 @@ exports.generationToolsManifest = {
|
|
|
1186
1186
|
"minimax-h3-flf2v-turbo",
|
|
1187
1187
|
"wan3.0-video"
|
|
1188
1188
|
],
|
|
1189
|
-
"description": "
|
|
1189
|
+
"description": "\"ltx25\" (default): LTX 2.5 I2V or first/last-frame video with native audio; Fast, HQ, and Pro currently use the release-validated official Distilled INT8 workflow. The Dev checkpoints are not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe. Which video model to use. \"ltx23\": LTX 2.3 rollback with native audio. \"wan22\": quick simple motion without audio, up to 10s. \"minimax-h3-i2v\" and \"minimax-h3-i2v-turbo\": standard and 4-step Turbo MiniMax H3 from one first frame. \"minimax-h3-flf2v\" and \"minimax-h3-flf2v-turbo\": standard and 4-step Turbo MiniMax H3 between required first and last frames; use frameRole=\"both\" and provide the end frame. H3 generates native audio at fixed 24fps for 5.17-15.08s and has no negative-prompt input. H3 Base and Turbo prompts use the exact three-field contract and the official mode-specific alignment line. Do not set Seedance here; use generate_video with Seedance references. \"wan3.0-video\" is Alibaba Wan 3: one canonical premium-vendor model for text-to-video, first-frame and first+last-frame animation, loose multimodal references, and audio-driven generation. It renders 2-30s at fixed 30 fps with optional native audio, supports 480p/720p/1080p and 16:9/4:3/1:1/3:4/9:16, accepts up to 10 reference images, 5 reference videos, and 5 reference audios, and uses plain per-type prompt labels Image 1, Video 1, and Audio 1. Do not send negativePrompt. Use animate_photo for native first/last frames, generate_video for text or loose references, and sound_to_video when audio is the primary driver. Video references are loose conditioning, not source-video edit or extend modes."
|
|
1190
1190
|
},
|
|
1191
1191
|
"negativePrompt": {
|
|
1192
1192
|
"type": "string",
|
|
@@ -1226,7 +1226,7 @@ exports.generationToolsManifest = {
|
|
|
1226
1226
|
},
|
|
1227
1227
|
"sourceImageIndex": {
|
|
1228
1228
|
"type": "number",
|
|
1229
|
-
"description": "Which image to use as the START frame. Use 0-based non-negative indices for generated result images. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. Omit to auto-select: uses the latest result for \"start\"/\"end\" modes, or the FIRST result for \"both\" mode. IMPORTANT: When frameRole is \"both\", set this to the start frame image index and endImageIndex to the end frame image index
|
|
1229
|
+
"description": "Which image to use as the START frame. Use 0-based non-negative indices for generated result images. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. Omit to auto-select: uses the latest result for \"start\"/\"end\" modes, or the FIRST result for \"both\" mode. IMPORTANT: When frameRole is \"both\", set this to the start frame image index and endImageIndex to the end frame image index."
|
|
1230
1230
|
},
|
|
1231
1231
|
"sourceImageIndices": {
|
|
1232
1232
|
"type": "array",
|
|
@@ -1235,7 +1235,7 @@ exports.generationToolsManifest = {
|
|
|
1235
1235
|
},
|
|
1236
1236
|
"minItems": 1,
|
|
1237
1237
|
"maxItems": 16,
|
|
1238
|
-
"description": "Array of source frame indices — one video is generated per entry as its own SDK project, all running in PARALLEL. Use this when outcomes need different source images, different end frames, isolated retry lifecycle, or other per-clip asset wiring/parameters. If every outcome uses the same source/end frames and only prompt text differs, prefer sourceImageIndex with numberOfVariations=N and one Dynamic Prompt branch in `prompt` so Sogni creates one project with multiple jobs. Use 0-based non-negative result indices for generated images. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. Repeating -1 is allowed for true multi-project workflows that intentionally reuse the same uploaded image while varying per-clip assets or parameters. By default all projects share the `prompt`/`voice`/`duration`, but you can pass `prompts` (array) to give each clip its own dialogue/motion when multi-project fan-out is required. Avoid sequential animate_photo calls for N outputs. Do NOT combine with `numberOfVariations` or `sourceImageIndex`. Use frameRole=\"end\" with sourceImageIndices only when the user explicitly says the repeated uploaded/generated image is the last/end frame for each clip and no first/start frame should be supplied; in that case omit endImageIndex/endImageIndices because each sourceImageIndices entry is the end frame. You MAY combine with frameRole=\"both\" when clips need start and end frames. For adjacent transition chains across generated images, use sourceImageIndices=[start..end-1] and endImageIndices=[start+1..end] so N images produce N-1 transition clips. If the uploaded/original image starts the chain and generated results are the remaining frames, use sourceImageIndices=[-1,start..end-1] and endImageIndices=[start..end]. If the user supplies multiple uploaded images as the actual keyframe sequence, use adjacent negative uploaded indices, e.g. 5 uploaded images become sourceImageIndices=[-1,-2,-3,-4], endImageIndices=[-2,-3,-4,-5], frameRole=\"both\", prompts length 4, then stitch_video. If the user specifies transition motion, camera behavior, actions, dialogue, or audio, copy those instructions into every corresponding per-clip prompt; only invent a generic smooth transition when the user does not specify one. If the user asks for a seamless loop or final transition from the last image back to the first, close the chain by including the last image as a source and the first image as the final end frame, e.g. 5 uploaded images become sourceImageIndices=[-1,-2,-3,-4,-5], endImageIndices=[-2,-3,-4,-5,-1]. For generated scene keyframes that should each loop to themselves, omit endImageIndex/endImageIndices so each source image is also its own end frame. Set endImageIndex=-1 only when every sourceImageIndices entry is also -1 and every segment reuses the first uploaded image. Range: 1–16 indices. For generated image batches, values MUST be read from the latest edit_image/generate_image tool result's `startIndex` field. If startIndex=3 and 4 images were generated in that batch, pass `[3,4,5,6]` (NOT `[0,1,2,3]`). Do NOT assume generated indices start at 0 — they don't if there are prior results in the conversation
|
|
1238
|
+
"description": "Array of source frame indices — one video is generated per entry as its own SDK project, all running in PARALLEL. Use this when outcomes need different source images, different end frames, isolated retry lifecycle, or other per-clip asset wiring/parameters. If every outcome uses the same source/end frames and only prompt text differs, prefer sourceImageIndex with numberOfVariations=N and one Dynamic Prompt branch in `prompt` so Sogni creates one project with multiple jobs. Use 0-based non-negative result indices for generated images. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. Repeating -1 is allowed for true multi-project workflows that intentionally reuse the same uploaded image while varying per-clip assets or parameters. By default all projects share the `prompt`/`voice`/`duration`, but you can pass `prompts` (array) to give each clip its own dialogue/motion when multi-project fan-out is required. Avoid sequential animate_photo calls for N outputs. Do NOT combine with `numberOfVariations` or `sourceImageIndex`. Use frameRole=\"end\" with sourceImageIndices only when the user explicitly says the repeated uploaded/generated image is the last/end frame for each clip and no first/start frame should be supplied; in that case omit endImageIndex/endImageIndices because each sourceImageIndices entry is the end frame. You MAY combine with frameRole=\"both\" when clips need start and end frames. For adjacent transition chains across generated images, use sourceImageIndices=[start..end-1] and endImageIndices=[start+1..end] so N images produce N-1 transition clips. If the uploaded/original image starts the chain and generated results are the remaining frames, use sourceImageIndices=[-1,start..end-1] and endImageIndices=[start..end]. If the user supplies multiple uploaded images as the actual keyframe sequence, use adjacent negative uploaded indices, e.g. 5 uploaded images become sourceImageIndices=[-1,-2,-3,-4], endImageIndices=[-2,-3,-4,-5], frameRole=\"both\", prompts length 4, then stitch_video. If the user specifies transition motion, camera behavior, actions, dialogue, or audio, copy those instructions into every corresponding per-clip prompt; only invent a generic smooth transition when the user does not specify one. If the user asks for a seamless loop or final transition from the last image back to the first, close the chain by including the last image as a source and the first image as the final end frame, e.g. 5 uploaded images become sourceImageIndices=[-1,-2,-3,-4,-5], endImageIndices=[-2,-3,-4,-5,-1]. For generated scene keyframes that should each loop to themselves, omit endImageIndex/endImageIndices so each source image is also its own end frame. Set endImageIndex=-1 only when every sourceImageIndices entry is also -1 and every segment reuses the first uploaded image. Range: 1–16 indices. For generated image batches, values MUST be read from the latest edit_image/generate_image tool result's `startIndex` field. If startIndex=3 and 4 images were generated in that batch, pass `[3,4,5,6]` (NOT `[0,1,2,3]`). Do NOT assume generated indices start at 0 — they don't if there are prior results in the conversation."
|
|
1239
1239
|
},
|
|
1240
1240
|
"prompts": {
|
|
1241
1241
|
"type": "array",
|
|
@@ -1263,11 +1263,11 @@ exports.generationToolsManifest = {
|
|
|
1263
1263
|
"end",
|
|
1264
1264
|
"both"
|
|
1265
1265
|
],
|
|
1266
|
-
"description": "How to use the source image(s). \"start\" (default): first frame. \"end\": last frame. \"both\": interpolate between first and last frames. For
|
|
1266
|
+
"description": "How to use the source image(s). \"start\" (default): first frame. \"end\": last frame. \"both\": interpolate between first and last frames. For end-frame fan-out, use frameRole=\"end\" with sourceImageIndices. MiniMax H3 i2v supports only \"start\"; MiniMax H3 flf2v requires \"both\" plus an end image. For single clips using \"both\", set sourceImageIndex and endImageIndex; fan-out can use matching sourceImageIndices/endImageIndices."
|
|
1267
1267
|
},
|
|
1268
1268
|
"endImageIndex": {
|
|
1269
1269
|
"type": "number",
|
|
1270
|
-
"description": "Which image to use as the END frame. Use 0-based non-negative indices for generated results. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. For a single frameRole=\"both\" transition between two different images, set this to the desired end frame. For sourceImageIndices fan-out where each generated keyframe should also be its own last frame, OMIT this field. Use a shared uploaded endImageIndex only when every sourceImageIndices entry is also an uploaded image; otherwise use endImageIndices for per-clip end frames
|
|
1270
|
+
"description": "Which image to use as the END frame. Use 0-based non-negative indices for generated results. Use negative indices for uploaded images: -1 = first/primary upload, -2 = second upload, -3 = third upload, etc. For a single frameRole=\"both\" transition between two different images, set this to the desired end frame. For sourceImageIndices fan-out where each generated keyframe should also be its own last frame, OMIT this field. Use a shared uploaded endImageIndex only when every sourceImageIndices entry is also an uploaded image; otherwise use endImageIndices for per-clip end frames."
|
|
1271
1271
|
},
|
|
1272
1272
|
"endImageIndices": {
|
|
1273
1273
|
"type": "array",
|
|
@@ -1276,7 +1276,7 @@ exports.generationToolsManifest = {
|
|
|
1276
1276
|
},
|
|
1277
1277
|
"minItems": 1,
|
|
1278
1278
|
"maxItems": 16,
|
|
1279
|
-
"description": "Per-clip END frame indices for sourceImageIndices fan-out. Use ONLY with frameRole=\"both\". Length MUST exactly match sourceImageIndices. Use 0-based non-negative indices for generated results and negative indices for uploaded images (-1 first upload, -2 second upload, etc.). Use this for transition chains between generated images, e.g. 5 generated images at indices [0,1,2,3,4] should become 4 transition clips with sourceImageIndices=[0,1,2,3], endImageIndices=[1,2,3,4], prompts length 4, duration as requested, then stitch_video. If the chain starts on the uploaded image and continues through generated results [0,1,2,3], use sourceImageIndices=[-1,0,1,2] and endImageIndices=[0,1,2,3]. If the user supplies 5 uploaded images as the sequence, use sourceImageIndices=[-1,-2,-3,-4] and endImageIndices=[-2,-3,-4,-5]. If the user requests a seamless loop or final transition back to the first image, append that loop closure: sourceImageIndices=[-1,-2,-3,-4,-5], endImageIndices=[-2,-3,-4,-5,-1]. Do NOT also set endImageIndex when using this
|
|
1279
|
+
"description": "Per-clip END frame indices for sourceImageIndices fan-out. Use ONLY with frameRole=\"both\". Length MUST exactly match sourceImageIndices. Use 0-based non-negative indices for generated results and negative indices for uploaded images (-1 first upload, -2 second upload, etc.). Use this for transition chains between generated images, e.g. 5 generated images at indices [0,1,2,3,4] should become 4 transition clips with sourceImageIndices=[0,1,2,3], endImageIndices=[1,2,3,4], prompts length 4, duration as requested, then stitch_video. If the chain starts on the uploaded image and continues through generated results [0,1,2,3], use sourceImageIndices=[-1,0,1,2] and endImageIndices=[0,1,2,3]. If the user supplies 5 uploaded images as the sequence, use sourceImageIndices=[-1,-2,-3,-4] and endImageIndices=[-2,-3,-4,-5]. If the user requests a seamless loop or final transition back to the first image, append that loop closure: sourceImageIndices=[-1,-2,-3,-4,-5], endImageIndices=[-2,-3,-4,-5,-1]. Do NOT also set endImageIndex when using this."
|
|
1280
1280
|
},
|
|
1281
1281
|
"voicePersonaName": {
|
|
1282
1282
|
"type": "string",
|
|
@@ -1343,17 +1343,17 @@ exports.generationToolsManifest = {
|
|
|
1343
1343
|
"type": "function",
|
|
1344
1344
|
"function": {
|
|
1345
1345
|
"name": "video_to_video",
|
|
1346
|
-
"description": "Transform an existing video using AI. Uses WAN 2.2 Animate (move/replace) with a reference image, LTX 2.5 V2V controls by default (canny/pose/depth/detailer plus distilled inpaint/outpaint), LTX 2.3 as rollback, or Seedance V2V when explicitly requested. LTX 2.5 Fast, HQ, and Pro use the release-validated official Distilled workflow for canny/pose/depth/detailer/inpaint/outpaint; Dev is not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe. Requires an uploaded video file. Use when the user wants to animate a photo with video motion, replace subjects, restyle footage, extend its canvas, regenerate a region, or enhance quality. Wan 3
|
|
1346
|
+
"description": "Transform an existing video using AI. Uses WAN 2.2 Animate (move/replace) with a reference image, LTX 2.5 V2V controls by default (canny/pose/depth/detailer plus distilled inpaint/outpaint), LTX 2.3 as rollback, or Seedance V2V when explicitly requested. LTX 2.5 Fast, HQ, and Pro use the release-validated official Distilled workflow for canny/pose/depth/detailer/inpaint/outpaint; Dev is not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe. Requires an uploaded video file. Use when the user wants to animate a photo with video motion, replace subjects, restyle footage, extend its canvas, regenerate a region, or enhance quality. Wan 3 is not a source-video editing model; its video inputs are loose references handled by generate_video.",
|
|
1347
1347
|
"parameters": {
|
|
1348
1348
|
"type": "object",
|
|
1349
1349
|
"properties": {
|
|
1350
1350
|
"prompt": {
|
|
1351
1351
|
"type": "string",
|
|
1352
|
-
"description": "Describe the TARGET appearance (not the transformation process). 2-4 present-tense sentences.\n\nLITERAL PROMPT OVERRIDE: If the user explicitly says not to modify the prompt, or to use it exactly/verbatim/as-is, copy the identified prompt text verbatim instead of applying these construction rules unless a hard requirement is missing. For Seedance
|
|
1352
|
+
"description": "Describe the TARGET appearance (not the transformation process). 2-4 present-tense sentences.\n\nLITERAL PROMPT OVERRIDE: If the user explicitly says not to modify the prompt, or to use it exactly/verbatim/as-is, copy the identified prompt text verbatim instead of applying these construction rules unless a hard requirement is missing. For Seedance, set expandPrompt=false.\n\nFor LTX 2.5 or 2.3 canny/depth/pose modes, the source video preserves composition, depth, or motion. Spend prompt detail on style, atmosphere, lighting, surface texture, color palette, scale, and pacing. LTX 2.5 Fast, HQ, and Pro use the release-validated official Distilled workflow for canny/pose/depth/detailer/inpaint/outpaint; Dev is not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe.\n\nExamples by mode:\n- animate-move (DEFAULT — WAN 2.2 Animate Move: applies camera/motion from source video to reference image): \"Smooth cinematic camera movement following the subject through the scene.\"\n- animate-replace (WAN 2.2 Animate Replace: replaces the subject in the source video with the reference image): \"The person from the reference photo performing the actions from the video.\"\n- canny (LTX 2.5 default; LTX 2.3 rollback — edge-detection restyle): \"Hand-drawn watercolor anime style with soft ink edges, muted teal and coral palette, rain mist, neon reflections, warm rim light, preserving original silhouettes and composition.\"\n- pose (LTX 2.5 or LTX 2.3 — tracks skeleton and transfers the reference-image subject): \"A glossy cartoon robot from the reference image performs the source video's motion, with brushed metal texture, glowing cyan joints, and energetic stage lighting.\" This mode requires a reference image as well as the source video.\n- depth (LTX 2.5 default; LTX 2.3 rollback — depth-map restyle): \"A misty alpine valley at golden hour, expansive scale, volumetric haze, cool blue shadows, warm rim light, cinematic depth, lingering continuous shot.\"\n- detailer (LTX 2.5 default; LTX 2.3 rollback — enhance quality): DESCRIBE THE SOURCE, do not request changes. Append quality qualifiers only. E.g. \"The same scene, ultra-sharp and clean, crisp high-resolution detail, preserving all original content, composition, and color.\" Avoid words like \"enhanced textures\", \"restyled\", or any new subjects/objects — they cause drift.\n- seedance-v2v (BytePlus Dreamina Seedance 2.0 V2V): \"Restyle the source clip in a watercolor look with soft ink edges, while preserving its motion and composition.\" Use natural prose; Seedance reads the reference video holistically rather than via control-net constraints, so describe target style/mood/dialogue rather than control strength.\n- outpaint (LTX 2.5 default; LTX 2.3 rollback — canvas extension): describe what fills the NEWLY REVEALED area around the original frame, consistent with the source scene. E.g. \"The same street scene continues seamlessly into the newly revealed space — more wet asphalt, parked cars, and glowing shopfronts, matching the original lighting and perspective.\" Set outpaintPosition (and optionally outpaintAspectRatio); no mask needed.\n- inpaint (LTX 2.5 default; LTX 2.3 rollback — masked region regeneration): describe ONLY what the inpainted region should become; the rest of the frame is preserved. E.g. \"A vintage red convertible parked at the curb, matching the scene's lighting and shadows.\" If the user supplied a mask, set maskImageIndex. If no mask was supplied, omit maskImageIndex so execution derives a mask from the source video and prompt.\n\nPresent tense. Positive phrasing. Concrete visual details.\n\nNON-SEEDANCE POSITIVE CONSTRAINTS: For LTX 2.5, LTX 2.3, and WAN 2.2 modes, prompt is a positive prompt. Translate user avoid/no/don't constraints into affirmative production constraints instead of copying negative phrasing. Preserve exact quoted visible text when the user explicitly requests it; keep surrounding surfaces blank.\n\nBATCH VARIATIONS: When numberOfVariations > 1, use Dynamic Prompt syntax to vary the artistic treatment while keeping control mode and structural intent consistent. Example: \"transform to {watercolor with soft edges|oil painting with bold strokes|anime with clean lines} style\"."
|
|
1353
1353
|
},
|
|
1354
1354
|
"expandPrompt": {
|
|
1355
1355
|
"type": "boolean",
|
|
1356
|
-
"description": "Seedance
|
|
1356
|
+
"description": "Seedance only. Whether to run Sogni's exact-model prompt shaper before dispatch. Defaults to true. Set false when the supplied prompt is already model-ready and must remain exact."
|
|
1357
1357
|
},
|
|
1358
1358
|
"videoSourceIndex": {
|
|
1359
1359
|
"type": "number",
|
|
@@ -1372,11 +1372,11 @@ exports.generationToolsManifest = {
|
|
|
1372
1372
|
"inpaint",
|
|
1373
1373
|
"seedance-v2v"
|
|
1374
1374
|
],
|
|
1375
|
-
"description": "How the source video and reference image interact. Pick by user intent:\n• \"animate-move\" (DEFAULT) — WAN 2.2 Animate Move. Applies camera movement and motion from the source video to the reference image, bringing a still photo to life. Requires sourceImageIndex.\n• \"animate-replace\" — WAN 2.2 Animate Replace. Replaces the subject in the source video with the person/character from the reference image, keeping the video's background and motion. Requires sourceImageIndex.\n• \"canny\" — LTX 2.5 (default) or 2.3 edge-detection control. Best for restyling while preserving exact composition and silhouettes. Video-only.\n• \"pose\" — LTX 2.5 (default) or 2.3 skeletal tracking. Best for
|
|
1375
|
+
"description": "How the source video and reference image interact. Pick by user intent:\n• \"animate-move\" (DEFAULT) — WAN 2.2 Animate Move. Applies camera movement and motion from the source video to the reference image, bringing a still photo to life. Requires sourceImageIndex.\n• \"animate-replace\" — WAN 2.2 Animate Replace. Replaces the subject in the source video with the person/character from the reference image, keeping the video's background and motion. Requires sourceImageIndex.\n• \"canny\" — LTX 2.5 (default) or 2.3 edge-detection control. Best for restyling while preserving exact composition and silhouettes. Video-only.\n• \"pose\" — LTX 2.5 (default) or 2.3 skeletal tracking. Best for transferring the person/character from a required reference image while keeping the source video's motion. Requires sourceImageIndex (or the sole available reference image).\n• \"depth\" — LTX 2.5 (default) or 2.3 depth-map control. Best for scenes with perspective, camera movement, or volumetric content. Video-only.\n• \"detailer\" — LTX 2.5 (default) or 2.3 quality enhancement. Describe the original scene with quality qualifiers and do not request content changes.\n• \"outpaint\" — distilled LTX 2.5 (default) or LTX 2.3 canvas extension. Set outpaintPosition and optionally outpaintAspectRatio; Pro/dev is not supported for this mode.\n• \"inpaint\" — distilled LTX 2.5 (default) or LTX 2.3 masked region regeneration. Set maskImageIndex when supplied; Pro/dev is not supported for this mode.\n• \"seedance-v2v\" — BytePlus Dreamina Seedance 2.0 video-to-video. Use only when the user explicitly asks for Seedance on the uploaded source video, such as a Seedance upscale, enhance, remaster, restyle, or transform. High-fidelity quality, native audio, time-coded scene control. Seedance V2V reads @Video1 holistically. Use it for restyling, motion transfer, extension, subject replacement, or scene transformation, and assign @Video1 a clear role such as source clip, camera movement, action timing, edit rhythm, or continuation anchor. Distinct from canny/depth/pose which use control-net constraints — Seedance treats the reference video holistically.\nCanny vs depth: canny preserves silhouettes and fine outlines — pick it for subject-led scenes and graphic restyles. Depth preserves 3D structure — pick it for scenes where the camera moves or spatial layout matters more than edge fidelity. Default: \"animate-move\"."
|
|
1376
1376
|
},
|
|
1377
1377
|
"negativePrompt": {
|
|
1378
1378
|
"type": "string",
|
|
1379
|
-
"description": "Advanced non-Seedance only. Use this field only when the user explicitly asks to set a separate negative prompt. For ordinary avoid/no/don't constraints on LTX 2.3 or WAN 2.2, translate them into affirmative production constraints inside prompt instead; do not move them here. Do not set when controlMode is seedance-v2v or videoModel is seedance2/seedance2-mini/seedance2-5
|
|
1379
|
+
"description": "Advanced non-Seedance only. Use this field only when the user explicitly asks to set a separate negative prompt. For ordinary avoid/no/don't constraints on LTX 2.3 or WAN 2.2, translate them into affirmative production constraints inside prompt instead; do not move them here. Do not set when controlMode is seedance-v2v or videoModel is seedance2/seedance2-mini/seedance2-5."
|
|
1380
1380
|
},
|
|
1381
1381
|
"videoModel": {
|
|
1382
1382
|
"type": "string",
|
|
@@ -1386,10 +1386,9 @@ exports.generationToolsManifest = {
|
|
|
1386
1386
|
"wan22-animate",
|
|
1387
1387
|
"seedance2",
|
|
1388
1388
|
"seedance2-mini",
|
|
1389
|
-
"seedance2-5"
|
|
1390
|
-
"wan3.0-video"
|
|
1389
|
+
"seedance2-5"
|
|
1391
1390
|
],
|
|
1392
|
-
"description": "Model selector for this video-to-video request. Usually omit:
|
|
1391
|
+
"description": "Model selector for this video-to-video request. Usually omit: non-Seedance controls default to \"ltx25-v2v\"; use \"ltx23-v2v\" only for rollback. LTX 2.5 Fast, HQ, and Pro currently use the release-validated official Distilled workflow for canny, pose, depth, detailer, inpaint, and outpaint. Dev is not publicly routed until upstream publishes and Sogni validates an official ComfyUI Dev recipe. For controlMode=\"seedance-v2v\", Seedance quality is selected only by model: use \"seedance2-mini\" for faster/lower-cost drafts and use \"seedance2\" for full-quality Seedance or 1080p/4K. \"seedance2-5\" supports 480p/720p, 4-30s at 24 fps, native audio, and first/last-frame conditioning; keep \"seedance2\" for 1080p/4K."
|
|
1393
1392
|
},
|
|
1394
1393
|
"generateAudio": {
|
|
1395
1394
|
"type": "boolean",
|
|
@@ -1397,11 +1396,11 @@ exports.generationToolsManifest = {
|
|
|
1397
1396
|
},
|
|
1398
1397
|
"targetResolution": {
|
|
1399
1398
|
"type": "number",
|
|
1400
|
-
"description": "Seedance
|
|
1399
|
+
"description": "Seedance V2V only. Short-side output resolution target in pixels. Use when the user asks for a bare named resolution such as \"480p\", \"720p\", \"1080p\", \"2160p\", or \"4K\" without exact dimensions. Seedance V2V full supports 4K; Seedance V2V Mini, Fast, and Seedance 2.5 support 480p and 720p only, so never set 1080p or 4K for \"seedance2-5\". Preserve the source video shape instead of forcing landscape pixels."
|
|
1401
1400
|
},
|
|
1402
1401
|
"sourceImageIndex": {
|
|
1403
1402
|
"type": "number",
|
|
1404
|
-
"description": "
|
|
1403
|
+
"description": "Optional 0-based reference-image index. Required for \"animate-move\", \"animate-replace\", and \"pose\" when more than one image is available; the sole available image may be auto-selected. LTX pose always dispatches both the source video and a reference image. Ignored by \"canny\", \"depth\", \"detailer\", \"outpaint\", and \"inpaint\"."
|
|
1405
1404
|
},
|
|
1406
1405
|
"outpaintPosition": {
|
|
1407
1406
|
"type": "string",
|
|
@@ -1432,46 +1431,10 @@ exports.generationToolsManifest = {
|
|
|
1432
1431
|
},
|
|
1433
1432
|
"duration": {
|
|
1434
1433
|
"type": "number",
|
|
1435
|
-
"description": "Output video duration in seconds. Per-model range: WAN 2.2/LTX modes = 2-20s;
|
|
1434
|
+
"description": "Output video duration in seconds. Per-model range: WAN 2.2/LTX modes = 2-20s; Seedance 2.0 and Mini = 4-15s; Seedance 2.5 = 4-30s. If omitted, the tool matches the uploaded source video duration when available (capped to the selected model range); otherwise it falls back to 10s for WAN Animate Move/Replace and 5s for LTX/Seedance modes. For long stitched/bulk WAN Animate Move/Replace work with no explicit per-clip length, prefer about 10s clips rather than 5s chunks. Only pass this when the user explicitly requests a different length.",
|
|
1436
1435
|
"minimum": 2,
|
|
1437
1436
|
"maximum": 30
|
|
1438
1437
|
},
|
|
1439
|
-
"smartDuration": {
|
|
1440
|
-
"type": "boolean",
|
|
1441
|
-
"description": "Wan 3 only. Let Wan 3 choose 2-30 output seconds. Do not also set duration; input plus output must stay within the provider's 30-second limit. Sogni reserves 30 seconds and settles down to actual duration."
|
|
1442
|
-
},
|
|
1443
|
-
"wan3TaskType": {
|
|
1444
|
-
"type": "string",
|
|
1445
|
-
"enum": [
|
|
1446
|
-
"edit",
|
|
1447
|
-
"extend"
|
|
1448
|
-
],
|
|
1449
|
-
"description": "Wan 3 only. Use \"edit\" to transform the source or \"extend\" to continue it. Extension automatically uses ratio=\"adaptive\" and the prompt should explicitly describe continuation intent."
|
|
1450
|
-
},
|
|
1451
|
-
"ratio": {
|
|
1452
|
-
"type": "string",
|
|
1453
|
-
"enum": [
|
|
1454
|
-
"adaptive",
|
|
1455
|
-
"16:9",
|
|
1456
|
-
"4:3",
|
|
1457
|
-
"1:1",
|
|
1458
|
-
"3:4",
|
|
1459
|
-
"9:16"
|
|
1460
|
-
],
|
|
1461
|
-
"description": "Wan 3 only. Extension requires \"adaptive\"; edit defaults to adaptive."
|
|
1462
|
-
},
|
|
1463
|
-
"referenceFileUrl": {
|
|
1464
|
-
"type": "string",
|
|
1465
|
-
"description": "Wan 3 only. One public HTTPS document URL for additional edit/extension context (DOCX/DOC/XLSX/XLS/PPTX/PPT/PDF/TXT/KEY/PAGES/NUMBERS/Markdown, up to 100 MB; PDF/DOCX/DOC/PPTX/PPT/KEY/PAGES up to 50 pages). Mutually exclusive with referenceLinkUrl."
|
|
1466
|
-
},
|
|
1467
|
-
"referenceLinkUrl": {
|
|
1468
|
-
"type": "string",
|
|
1469
|
-
"description": "Wan 3 only. One public HTTPS webpage URL for additional edit/extension context. Mutually exclusive with referenceFileUrl."
|
|
1470
|
-
},
|
|
1471
|
-
"watermark": {
|
|
1472
|
-
"type": "boolean",
|
|
1473
|
-
"description": "Wan 3 only. Add Alibaba's visible watermark. Defaults to false."
|
|
1474
|
-
},
|
|
1475
1438
|
"numberOfVariations": {
|
|
1476
1439
|
"type": "number",
|
|
1477
1440
|
"description": "Number of video variations to generate (1-16). Default: 1.",
|
|
@@ -1737,7 +1700,7 @@ exports.generationToolsManifest = {
|
|
|
1737
1700
|
"ltx23-a2v",
|
|
1738
1701
|
"wan3.0-video"
|
|
1739
1702
|
],
|
|
1740
|
-
"description": "\"ltx25-ia2v\" (default with image) and \"ltx25-a2v\" (default without image)
|
|
1703
|
+
"description": "\"ltx25-ia2v\" (default with image) and \"ltx25-a2v\" (default without image): LTX 2.5 image+audio and audio-only modes; Fast, HQ, and Pro currently use the release-validated official Distilled INT8 workflows. The Dev checkpoints are not publicly routed until upstream publishes and Sogni validates official ComfyUI Dev recipes. Video model. \"ltx23-ia2v\" (rollback with image): LTX 2.3 image+audio to video, audio-reactive with a reference image; Fast/HQ use the distilled 8-step worker and Default Media Quality Pro uses the non-distilled dev worker. \"ltx23-a2v\" (rollback without image): LTX 2.3 audio-only to video, no image needed, creates video purely from text prompt + audio with the same quality-tier routing. \"wan-s2v\": WAN 2.2 sound-to-video, best for lip-sync with a face image, fast 4-step. \"seedance2\": full Seedance 2.0 audio-reference video, 4-15s. \"seedance2-mini\": Seedance 2.0 Mini, 720p cap, fastest/lower-cost Seedance option. Seedance quality is selected only by this model value: pick \"seedance2-mini\" for faster/lower-cost drafts or explicit Mini requests, and pick \"seedance2\" for full-quality Seedance or 1080p/4K. Do not infer the Seedance model from Default Media Quality Fast/HQ/Pro. \"seedance2-5\": Seedance 2.5, the newest Seedance generation — 480p and 720p ONLY (it cannot render 1080p or 4K), 4-30s per clip at a fixed 24 fps, native audio, first-and-last-frame conditioning, and a much larger reference budget than the 2.0 family: up to 30 images, 10 videos, and 10 audios, with up to 50 reference media files total, subject to those per-modality caps. Choose \"seedance2-5\" when the user asks for Seedance 2.5, wants a single continuous Seedance clip longer than 15s (2.5 renders up to 30s in one call instead of being split and stitched), or wants a first-and-last-frame Seedance transition. Keep \"seedance2\" for 1080p/4K requests, which Seedance 2.5 cannot satisfy. For Seedance audio-reference prompts, preserve exact spoken dialogue when the user supplied it, and assign @Image1/@Audio1 roles. If the user asks for speech without words, describe the vocal performance without inventing quoted dialogue. Treat lip-sync, voice cloning, and real-human reference behavior as provider-sensitive rather than guaranteed. Omit to auto-select based on whether an image is present. \"wan3.0-video\" is Alibaba Wan 3: one canonical premium-vendor model for text-to-video, first-frame and first+last-frame animation, loose multimodal references, and audio-driven generation. It renders 2-30s at fixed 30 fps with optional native audio, supports 480p/720p/1080p and 16:9/4:3/1:1/3:4/9:16, accepts up to 10 reference images, 5 reference videos, and 5 reference audios, and uses plain per-type prompt labels Image 1, Video 1, and Audio 1. Do not send negativePrompt. Use animate_photo for native first/last frames, generate_video for text or loose references, and sound_to_video when audio is the primary driver. Video references are loose conditioning, not source-video edit or extend modes."
|
|
1741
1704
|
},
|
|
1742
1705
|
"generateAudio": {
|
|
1743
1706
|
"type": "boolean",
|