@mindstudio-ai/agent 0.1.151 → 0.1.153

package/llms.txt

@@ -285,8 +285,10 @@ Analyze an image using a vision model based on a text prompt.
 
 #### analyzeVideo
 Analyze a video using a video analysis model based on a text prompt.
-- Uses the configured video analysis model to generate a text analysis of the video.
-- The prompt should describe what to look for or extract from the video.
+- The prompt describes what to look for or extract from the video.
+- The default model is gemini-3.8-flash. Any Gemini chat model reads video (the 3.x Flash and Flash Lite models, gemini-3.1-pro, and the 2.5 family), as does TwelveLabs' pegasus-1.6; set one with `videoAnalysisModelOverride`, with `temperature` and `maxResponseTokens` beside `model`.
+- `videoUrl` is a video file's URL (an app file, a Google Drive share link, or any public URL). Gemini also reads a public YouTube link; Pegasus doesn't.
+- Length: Pegasus takes 1 second to 1 hour. Gemini takes about 3 hours at its default resolution; the 3.x models read a second of video as about 66 tokens and the 2.5 models as about 258, which is what a long video costs.
 - Input: `{ prompt: string, videoUrl: string, videoAnalysisModelOverride?: { model: string, config?: object } | { model: string, temperature: number, maxResponseTokens: number, ignorePreamble?: boolean, userMessagePreprocessor?: { dataSource?: string, messageTemplate?: string, maxResults?: number, enabled?: boolean, shouldInherit?: boolean }, preamble?: string, multiModelEnabled?: boolean, editResponseEnabled?: boolean, promptCaching?: boolean, config?: object } }`
 - Output: `{ analysis: string }`
 
@@ -332,10 +334,11 @@ Create a draft email in the connected Gmail account.
 
 #### decomposeImage
 Split an image into a base image and transparent layers.
-- Uses Seedream 5.0 Flash layer decomposition. There is no model picker.
-- An optional prompt can name the objects or groupings to separate.
-- The output is an array of hosted image URLs. PNG output keeps layer transparency.
-- Input: `{ image: string, prompt?: string, intermediateAsset?: boolean }`
+- Uses Seedream 5.0 Flash layer decomposition by default, and Seedream 5.0 Pro when type is "advanced", which separates more cleanly and costs more (and twice as much at 2K).
+- `resolution` is "1k", "1.5k" or "2k" (the default).
+- An optional prompt names the objects or groupings to separate. Left out, the model separates the main elements itself: text, subjects, decorations and background.
+- The output is an array of hosted PNG image URLs: first the base image at full size, then each separated element as a transparent layer cropped to the element. The layers come with no position, so placing one back over the base means finding where it sat. Flash returned 13 layers for one photo and Pro 16.
+- Input: `{ image: string, prompt?: string, type?: "standard" | "advanced", resolution?: "1k" | "1.5k" | "2k", intermediateAsset?: boolean }`
 - Output: `{ imageUrls: string[] }`
 
 #### deleteDataSource
@@ -455,8 +458,12 @@ Fetch recent message history from a Slack channel.
 
 #### generate3dModel
 Generate a 3D model using a 3D generation model.
-- Text-to-3D models use the prompt field.
-- Image-to-3D and multi-view models take image URLs through the selected model's configuration inputs.
+- The default model is wavespeed-tripo3d-h3.1 (Tripo v3.1), which makes a model from the prompt, from one `image`, or from `images`: 2–4 views of the same object, in order front, left, back, right.
+- Models and what they take, set in `threeDModelOverride.config`:
+- From text or an image: wavespeed-tripo3d-h3.1, wavespeed-meshy7.1 (also `images`, 1–4 views), wavespeed-hunyuan3d-v3 (an `image`, plus optional `back_image_url`, `left_image_url`, `right_image_url`), wavespeed-hunyuan-3d-v3.1-rapid, wavespeed-tripo3d-p2, wavespeed-rodin-v2.5 (`images`, 1–5).
+- From an image only: trellis-2, wavespeed-meshy7 (or `images`, 1–4 views), wavespeed-tripo3d-v2.5, seed3d-2.0, wavespeed-sam-3d-objects (with optional `mask_images`).
+- From views: wavespeed-hi3d-v3.0 (`front_image`, and optional `back_image`, `left_image`, `right_image`).
+- `$mode` in the config picks a model's mode (`text-to-3d`, `image-to-3d`, or its multi-view mode). Left out, `images` picks the multi-view mode and `image` picks image-to-3d; with neither, the model's default mode. A text mode needs the prompt; in the others the prompt is optional guidance. A prompt longer than the model reads is refused, not cut.
 - The output is standardized as a GLB URL plus optional FBX/OBJ/USDZ, thumbnail, texture maps, and provider task ID.
 - Input: `{ prompt?: string, intermediateAsset?: boolean, threeDModelOverride?: { model: string, config?: object } }`
 - Output: `{ prompt?: string, glbUrl: string, fbxUrl?: string, objUrl?: string, usdzUrl?: string, thumbnailUrl?: string, textureUrls?: object[], providerTaskId?: string, resolvedConfig?: object }`
@@ -489,14 +496,23 @@ Generate a lip sync video from audio and an image or video.
 
 #### generateMusic
 Generate an audio file from provided instructions (text) using a music model.
-- The text field contains the instructions (prompt) for the music generation.
+- `text` describes the song: genre, mood, instruments, tempo, vocals, and for Lyria its structure (`[Verse]`, `[Chorus]`, timestamps such as `[0:00-0:20] Intro`). A text longer than the model reads is refused, not cut.
+- The default model is elevenlabs-music_v2.5. Options go in `musicModelOverride.config`, and differ by model:
+- elevenlabs-music_v2.5, elevenlabs-music_v2: `musicLengthMs`, the song's length from 3,000 to 600,000 (the model chooses one from the text otherwise, and holds a length it's given), and `forceInstrumental: true` for no vocals.
+- lyria-3.5: songs of up to about three minutes, their length steered from the text; `image`, the URL of an image for the song to take its mood and subject from. lyria-3 makes 30-second clips, lyria-3-pro songs of up to about three minutes.
+- minimax-music-3.0: `lyrics` (written from the text when left out), `is_instrumental: true`, `bitrate`, `sample_rate`. minimax-music-2.5 requires `lyrics`.
 - In foreground mode, the audio is displayed to the user. In background mode, the URL is saved to a variable.
 - Input: `{ text: string, intermediateAsset?: boolean, musicModelOverride?: { model: string, config?: object } }`
 - Output: `{ audioUrl: string }`
 
 #### generateSound
 Generate a sound effect from a text description using a sound model.
-- The text field contains the description of the sound effect to generate.
+- `text` describes the sound effect to generate.
+- The default model is elevenlabs-sound_v2. Options go in `soundModelOverride.config`:
+- `durationSeconds` — the length, from 0.5 to 30. Left out, the model picks one from the text, and the effect is billed as 5 seconds whatever its length.
+- `loop: true` — a sound that repeats with no audible start or end, for ambience and backgrounds.
+- `promptInfluence` — how literally the text is followed, from 0 to 1 (default 0.3).
+- `outputFormat` — MP3 (`mp3_44100_128` by default, up to `mp3_44100_192`) or Opus (`opus_48000_32` to `opus_48000_192`).
 - In foreground mode, the audio is displayed to the user. In background mode, the URL is saved to a variable.
 - Input: `{ text: string, intermediateAsset?: boolean, soundModelOverride?: { model: string, config?: object } }`
 - Output: `{ audioUrl: string }`
@@ -525,8 +541,10 @@ the server-side plumbing.
 - Output: `{ content: string }`
 
 #### generateVideo
-Generate a video from a text prompt using an AI model.
-- Prompts should be descriptive but concise (roughly 3–6 sentences).
+Generate a video from a text prompt, an image or references, using an AI model.
+- Prompts should be descriptive but concise (roughly 3–6 sentences): the subject, the motion, the camera and the sound. A prompt longer than the model reads is refused, not cut.
+- The default model is wan3.0-video. Options go in `videoModelOverride.config` under the model's own names, and differ by model, so look the model up before writing the call. Images, videos and audio go there too, as URLs: a first frame is `image` on most models, `image_url` on Veo, gemini-omni-flash, Runway's and minimax-h3-max, `source_image` on Grok Imagine, `start_frame` on ray-2 and `first_frame_image` on skyreels-v4.
+- A model that makes video more than one way takes `$mode` in its config (`text-to-video`, `image-to-video`, `reference-to-video`, `video-edit` and others it lists). Left out, a first frame or reference images in the config pick their mode, and otherwise the model's default runs.
 - Videos are hosted on the public CDN by default, or in one of the app's own file stores when the call passes `store` — a private store keeps the video private, with no public copy left behind.
 - In foreground mode, the video is displayed to the user. In background mode, the URL is saved to a variable.
 - When generateVariants is true with numVariants > 1, multiple videos are generated in parallel.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mindstudio-ai/agent",
-  "version": "0.1.151",
+  "version": "0.1.153",
   "description": "TypeScript SDK for MindStudio direct step execution",
   "type": "module",
   "main": "./dist/index.js",