@sogni-ai/sogni-creative-agent-skill 3.6.0 → 3.6.1

package/CHANGELOG.md

@@ -5,6 +5,30 @@ All notable changes to this project are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.6.1] - 2026-06-15
+
+### Changed
+
+- **Hosted-API guidance now recommends client-side planning over hosted re-planning.** The skill is driven by a
+  frontier LLM that out-plans Sogni's hosted planning model, so steering it to delegate planning through
+  `--api-chat` was a downgrade. `SKILL.md`, `references/hosted-api.md`, and `README.md` now tell the calling agent
+  to plan and select tools itself, use `--api-workflow` with an explicit `--workflow-input` step graph for durable
+  multi-step work (the server executes the authored plan without re-planning), and reserve `--api-chat` /
+  `--durable-chat` for deliberately offloading a long server-side loop or uploading several local files in one
+  turn. `--api-chat` and all hosted modes remain fully supported — only the recommended default changed.
+
+### Fixed
+
+- **Local Seedance reference images via `-c`/`--context` now auto-upload in direct CLI mode.** Local
+  loose-reference images were rejected with an HTTPS-only error that pushed users onto the unreliable
+  `--api-chat` / `--durable-chat` path; local `--ref-audio` and `--ref-video` already auto-uploaded through the
+  `/v2` presigned-POST flow, so images were the only modality missing it and one broken branch cascaded into
+  downstream failures (vision 1024px cap, HTTP timeout, no-content, missing durable SDK package). Local
+  `-c`/`--context` images now upload through the same `/v2/image` presigned flow and forward as Sogni-hosted URLs.
+  MIME type is resolved by magic-byte sniffing (falling back to extension), and the accepted set
+  (PNG/JPEG/WebP/GIF) mirrors the backend's `allowedContentTypes`. Adds local-PNG-upload and mislabeled-WebP
+  byte-sniff regression tests; verified end-to-end with a real Seedance 2.0 render from a local `-c` PNG.
+
 ## [3.6.0] - 2026-06-12
 
 ### Added

package/README.md

@@ -600,7 +600,9 @@ Stored at `~/.config/sogni/personality.txt`.
 
 Hosted API modes require `SOGNI_API_KEY`.
 
-- **`--api-chat`** targets `/v1/chat/completions` with Sogni creative-agent tools — best for text-first natural-language workflows. The CLI sanitizes prompt-injection markers before forwarding messages and can use the current server-side creative-agent media tools, including video extension, segment replacement, overlays, subtitles, stitch/orbit/dance composition, and generated artifact indexing. Tune with `--api-tools creative-agent|creative-tools|none`, `--no-api-tool-execution`, `--llm-model`, and `--system`.
+**Choosing a mode.** Whatever is driving this CLI is usually a more capable planner than Sogni's hosted model, so prefer to plan yourself and let the server execute: direct-to-SDK flags for one-shot work, and `--api-workflow` with an explicit `--workflow-input` step graph for multi-step/durable work (you author the plan; the server runs it durably with replay — no hosted re-planning). Use `--api-chat` / `--durable-chat` when you deliberately want the hosted model to own a long server-side loop, or when several local files must be uploaded for one turn.
+
+- **`--api-chat`** targets `/v1/chat/completions` with Sogni creative-agent tools and **delegates planning/tool-selection to the hosted model** — reach for it when the caller is a thin client, when you want the hosted model to drive a long server-side tool loop, or when several local files must be uploaded for one turn. The CLI sanitizes prompt-injection markers before forwarding messages and can use the current server-side creative-agent media tools, including video extension, segment replacement, overlays, subtitles, stitch/orbit/dance composition, and generated artifact indexing. Tune with `--api-tools creative-agent|creative-tools|none`, `--no-api-tool-execution`, `--llm-model`, and `--system`.
 - **Sogni Intelligence controls** include `--task-profile general|coding|reasoning`, `--max-tokens`, and `--thinking` / `--no-thinking`, which forward to `/v1/chat/completions` as `task_profile`, `max_tokens`, and `chat_template_kwargs.enable_thinking`. Use `--list-api-models` or `--get-api-model <id>` to inspect `/v1/models`.
 - **`--durable-chat`** starts a hosted `/v1/chat/runs` record through the SDK transport. Set `SOGNI_SKILL_USE_SDK_TRANSPORT=1` before using it. The CLI streams assistant deltas and de-duplicated per-job progress / ETA / result lines from hosted run events.
 - **`--api-workflow`** targets `/v1/creative-agent/workflows` for durable, async workflow records with event streaming and cancellation. Requests carry `input.steps` plus snake_case controls such as `token_type`, `media_references`, `max_estimated_capacity_units`, and `confirm_cost`.

package/SKILL.md

@@ -2,7 +2,7 @@
 name: sogni-creative-agent-skill
 description: "Sogni Creative Agent Skill: agent skill and CLI for image, video, and music generation using Sogni AI's decentralized GPU network. Supports personas (named people with saved reference photos and voice clips), persistent memories, custom personality, style transfer, angle synthesis, Seedance/LTX/WAN video, music/lyrics, hosted chat, durable workflows, replay records, and multi-step creative workflows. Ask the agent to \"draw\", \"generate\", \"create an image\", \"make a video/animate\", \"make music\", \"apply a style\", or \"generate me as a superhero\"."
 metadata:
-  version: "3.6.0"
+  version: "3.6.1"
   homepage: https://sogni.ai
   openclaw:
     emoji: "🎨"
@@ -98,28 +98,31 @@ sogni-agent -o /tmp/cat.png "a cat wearing a hat"    # ✗ avoid — user can't
 - Media listing for `--list-media` (read): `~/.openclaw/media/inbound`, falling back to the legacy `~/.clawdbot/media/inbound` when only it exists (`SOGNI_MEDIA_INBOUND_DIR`)
 - Custom ffmpeg binary: `FFMPEG_PATH`
 
-## Recommended path: hosted Sogni Intelligence endpoints
+## Recommended path: you plan, Sogni executes
 
-For any natural-language creative request that should be planned, multi-step, resumable, or benefit from server-side tool selection and repair, prefer the hosted endpoints over direct-to-SDK flags — **read [`references/hosted-api.md`](./references/hosted-api.md) first** for the full contract (tool surfaces, durable workflows, templates, replays, Seedance reference modes, media-reference uploads, cost controls):
+You (the calling LLM) are almost always more capable than Sogni's hosted planning model, so **do the planning and tool selection yourself** and let the hosted endpoints do what only the server can — run on the GPU network, persist assets/manifests, orchestrate durable multi-step runs with replay, and apply structured-contract repair. Don't flatten a rich request into a single natural-language string and hand planning back to a weaker model. Match the mode to the work:
 
-```bash
-# Natural-language creative request (LLM picks the tool, dispatches, repairs)
-sogni-agent --api-chat "Turn the attached product photo into a launch poster" --ref product.jpg
+- **One-shot generation** → direct-to-SDK flags (the Core Commands below). You already know the tool, model, and prompt — just run it. No LLM round-trip, lowest latency/cost.
+- **Multi-step / durable / resumable** → `--api-workflow` with an explicit step graph via `--workflow-input <json|@path>`. *You* author the exact plan — `steps[]` with `toolName`, `arguments`, and `dependsOn` bindings (e.g. `sourceStepId`, `targetArgument`, `transform: "artifact_url"`) — and the server executes it durably with replay/resumability, **without re-planning through the hosted LLM**. Presets like `--api-workflow storyboard-video` are fine when they already match the request.
+- **`--api-chat` / `--durable-chat` (hosted LLM owns the loop)** → reserve for when you deliberately *want* the hosted model to drive a long server-side tool loop (saves client round-trips on long async jobs), when structured-contract repair recipes should govern, or when several local files must be uploaded for a single turn (multi-file local upload is only supported here). These delegate planning to the hosted model — choose them on purpose, not by default.
 
-# Durable hosted chat run (persisted event log + SSE stream)
-SOGNI_SKILL_USE_SDK_TRANSPORT=1 sogni-agent --durable-chat "Create a launch campaign and animate the hero clip"
+**Read [`references/hosted-api.md`](./references/hosted-api.md) first** for the full hosted contract (tool surfaces, durable workflows, templates, replays, Seedance reference modes, media-reference uploads, cost controls).
 
-# Durable workflow (resumable, server-orchestrated)
-sogni-agent --api-workflow --video-prompt "The camera slowly pushes in" "A graphite robot sketch on a drafting table"
+```bash
+# One-shot: you pick the tool, the server just executes (see Core Commands below)
+sogni-agent -q -Q hq -o ./poster.png "Turn the product photo into a launch poster"
 
-# Storyboard → GPT Image 2 sheet → Seedance video, all server-side
+# Multi-step durable: you author the step graph, the server executes it (no hosted re-planning)
+sogni-agent --api-workflow --workflow-input @plan.json
 sogni-agent --api-workflow storyboard-video --storyboard-frames 6 -Q hq "9:16 bakery launch video"
+
+# Deliberately hand the whole loop to the hosted model (long async job, or multi local-file upload)
+sogni-agent --api-chat "Turn the attached product photo into a launch poster" --ref product.jpg
+SOGNI_SKILL_USE_SDK_TRANSPORT=1 sogni-agent --durable-chat "Create a launch campaign and animate the hero clip"
 ```
 
 Hosted modes require `SOGNI_API_KEY`. Local file references are uploaded to Sogni media storage and forwarded as retrievable URLs — **use direct CLI mode for private media that must not leave the local machine.**
 
-Use the direct-to-SDK commands below for explicit one-shot generation when you already know the model, dimensions, and prompt.
-
 ## Core Commands (direct-to-SDK)
 
 ```bash

package/openclaw.plugin.json

@@ -2,7 +2,7 @@
   "id": "sogni-creative-agent-skill",
   "name": "Sogni Creative Agent Skill — Image, Video & Music Generation",
   "description": "Agent skill and CLI for Sogni AI image, video, and music generation.",
-  "version": "3.6.0",
+  "version": "3.6.1",
   "skills": [
     "."
   ],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sogni-ai/sogni-creative-agent-skill",
-  "version": "3.6.0",
+  "version": "3.6.1",
   "description": "Sogni Creative Agent Skill: agent skill and CLI for Sogni AI image, video, and music generation.",
   "type": "module",
   "main": "sogni-agent.mjs",

package/references/hosted-api.md

@@ -7,35 +7,46 @@ All hosted modes require `SOGNI_API_KEY`.
 
 ## When to prefer the hosted path
 
-For any natural-language creative request that benefits from tool selection,
-repair, or durable workflows, prefer the hosted Sogni Intelligence endpoints
-over direct-to-SDK media flags. They are the canonical home for
-OpenAI-compatible chat, server-side creative tool dispatch, Structured
-Contracts v1 (gating policies, repair recipes, prompt contracts), durable chat
-runs, durable workflows, workflow templates, replay, and asset-manifest
-mapping.
+The thing calling this API is usually a frontier LLM that is **more capable
+than Sogni's hosted planning model**. So the default split is: *you* do the
+planning and tool selection, and the hosted endpoints do what only the server
+can — run on the GPU network, persist assets/manifests, orchestrate durable
+multi-step runs with replay, and apply Structured Contracts v1 (gating
+policies, repair recipes, prompt contracts). Routing a request through
+`--api-chat` so a weaker model re-plans it is usually a downgrade; reach for the
+hosted *planner* deliberately, not by default.
+
+- **You already know the single tool + args** → direct-to-SDK flags. Lowest
+  latency/cost, no LLM round-trip.
+- **Multi-step, durable, resumable** → `--api-workflow` with an explicit
+  `--workflow-input` step graph that *you* author (`steps[]` with `toolName`,
+  `arguments`, and `dependsOn` bindings). The server executes and repairs it
+  deterministically with replay/resumability and **no hosted-LLM re-planning**.
+  This is the best fit when a frontier client drives the work.
+- **You want the hosted model to own a long loop** → `--api-chat` /
+  `--durable-chat`. Worth it when offloading a long async tool loop server-side
+  saves client round-trips, when structured-contract repair should govern, or
+  when several local files must be uploaded for one turn (only supported here).
 
 ```bash
-# Natural-language creative request (LLM picks the tool, dispatches, repairs)
+# You author the exact durable plan; the server executes it (no hosted re-planning)
+sogni-agent --api-workflow --workflow-input @plan.json
+
+# Storyboard → GPT Image 2 sheet → Seedance, all server-side (preset plan)
+sogni-agent --api-workflow storyboard-video --storyboard-frames 6 -Q hq \
+  "Create a 9:16 bakery launch video with a neon street-window reveal"
+
+# Deliberately hand planning to the hosted model (long async job / multi local-file upload)
 sogni-agent --api-chat "Turn the attached product photo into a launch poster" --ref product.jpg
 
 # Durable hosted chat run (persisted event log + SSE stream)
 SOGNI_SKILL_USE_SDK_TRANSPORT=1 sogni-agent --durable-chat \
   "Create a four-shot launch campaign, generate the key art, and animate the hero clip"
-
-# Multi-step durable workflow (resumable, replay-friendly, server-orchestrated)
-sogni-agent --api-workflow \
-  --video-prompt "The camera slowly pushes in" \
-  "A graphite robot sketch on a drafting table"
-
-# Storyboard → GPT Image 2 sheet → Seedance, all server-side
-sogni-agent --api-workflow storyboard-video --storyboard-frames 6 -Q hq \
-  "Create a 9:16 bakery launch video with a neon street-window reveal"
 ```
 
-The direct-to-SDK flags remain available for explicit one-shot generation when
-you already know the exact model, dimensions, and prompt and don't need LLM
-planning — use them when latency or cost rules out the LLM round-trip.
+The direct-to-SDK flags remain the right call for explicit one-shot generation
+when you already know the exact model, dimensions, and prompt — use them
+whenever latency or cost rules out an LLM round-trip.
 
 ## --api-chat (`POST /v1/chat/completions`)
 
@@ -156,19 +167,24 @@ per video request:
 - **Loose reference mode — `-c/--context` plus optional `--ref-audio` and
   `--ref-video` extras.** Anchor frame intent in the prompt with `@Image1` /
   `@Video1` / `@Audio1` etc. (e.g. *"Use @Image1 as the opening shot
-  reference"*). Supports up to 9 image refs, 3 video refs, 3 audio refs, and
-  12 total reference assets per request (canonical caps come from
+  reference"*). Each `-c/--context` image may be a **local file or an HTTPS
+  URL** (PNG, JPEG, WebP, or GIF) — local files are uploaded to Sogni media
+  storage automatically, so you do **not** need `--api-chat` / `--durable-chat`
+  just to attach a local loose-reference image. Supports up to 9 image refs, 3 video refs, 3 audio
+  refs, and 12 total reference assets per request (canonical caps come from
   `SEEDANCE_REFERENCE_LIMITS` / `validateSeedanceReferenceCounts()` in
   `@sogni-ai/sogni-intelligence-client/tools`).
 
 Combining `--ref` / `--ref-end` with `-c/--context` on Seedance is rejected
-client-side with an error pointing at the correct mode. In CLI direct-gen
-mode, additional `--ref-audio` / `--ref-video` entries beyond the first must
-be HTTPS URLs (the primary entry can still be a local file); for local
-multi-file Seedance uploads, use `--api-chat` / `--durable-chat` instead.
-Seedance accepts public HTTPS image, video, and audio references that pass the
-CLI URL safety checks; localhost and private-network URLs are rejected before
-forwarding. Audio references must be paired with an image or video reference.
+client-side with an error pointing at the correct mode. In CLI direct-gen mode,
+local `-c/--context` images and the primary `--ref-audio` / `--ref-video` are
+uploaded to Sogni media storage automatically and forwarded as HTTPS URLs; only
+*additional* `--ref-audio` / `--ref-video` entries beyond the first must already
+be HTTPS URLs (use `--api-chat` / `--durable-chat` when you need to attach
+several local audio or video files in one request). Seedance accepts public
+HTTPS image, video, and audio references that pass the CLI URL safety checks;
+localhost and private-network URLs are rejected before forwarding. Audio
+references must be paired with an image or video reference.
 
 ## Models, replays, and contract debugging
 

package/sogni-agent.mjs

@@ -4242,6 +4242,8 @@ function extensionForApiMediaReference(mimeType, kind) {
   const normalized = String(mimeType || '').split(';')[0].trim().toLowerCase();
   if (normalized === 'image/jpeg' || normalized === 'image/jpg') return 'jpg';
   if (normalized === 'image/png') return 'png';
+  if (normalized === 'image/webp') return 'webp';
+  if (normalized === 'image/gif') return 'gif';
   if (normalized === 'audio/mpeg' || normalized === 'audio/mp3') return 'mp3';
   if (normalized === 'audio/mp4' || normalized === 'audio/m4a' || normalized === 'audio/x-m4a') return 'm4a';
   if (normalized === 'audio/wav' || normalized === 'audio/x-wav' || normalized === 'audio/wave') return 'wav';
@@ -6285,6 +6287,84 @@ async function uploadSeedanceReferenceVideoUrl(pathOrUrl, apiKey, index = 0) {
   return uploaded.url;
 }
 
+// Content types the Sogni media pipeline accepts for image references, mirroring
+// the `allowedContentTypes` the /v2/image/uploadUrl presigned-POST endpoint
+// returns. Kept as a constant so the skill validates exactly what the backend
+// will store rather than imposing a narrower client-side policy.
+const SEEDANCE_REFERENCE_IMAGE_MIME_TYPES = Object.freeze([
+  'image/png', 'image/jpeg', 'image/webp', 'image/gif',
+]);
+
+// Identify an image's MIME type from its leading bytes (magic numbers). Reliable
+// because we already hold the buffer, so it works regardless of file extension.
+function sniffSeedanceReferenceImageMimeType(buffer) {
+  if (!buffer || buffer.length < 4) return null;
+  if (buffer[0] === 0x89 && buffer[1] === 0x50 && buffer[2] === 0x4e && buffer[3] === 0x47) return 'image/png';
+  if (buffer[0] === 0xff && buffer[1] === 0xd8 && buffer[2] === 0xff) return 'image/jpeg';
+  if (
+    buffer.length >= 12
+    && buffer[0] === 0x52 && buffer[1] === 0x49 && buffer[2] === 0x46 && buffer[3] === 0x46
+    && buffer[8] === 0x57 && buffer[9] === 0x45 && buffer[10] === 0x42 && buffer[11] === 0x50
+  ) return 'image/webp';
+  if (buffer[0] === 0x47 && buffer[1] === 0x49 && buffer[2] === 0x46 && buffer[3] === 0x38) return 'image/gif';
+  return null;
+}
+
+// Resolve a Seedance loose-reference image's MIME type from its bytes first,
+// falling back to the file extension. Unsupported files fail fast with an
+// actionable message instead of uploading bytes the render backend will reject.
+function seedanceReferenceImageMimeType(pathOrUrl, buffer) {
+  const sniffed = sniffSeedanceReferenceImageMimeType(buffer);
+  if (sniffed) return sniffed;
+  const byPath = mimeTypeForPath(pathOrUrl, '');
+  const normalizedByPath = byPath === 'image/jpg' ? 'image/jpeg' : byPath;
+  if (SEEDANCE_REFERENCE_IMAGE_MIME_TYPES.includes(normalizedByPath)) return normalizedByPath;
+  const err = new Error(
+    `Seedance reference image "${pathOrUrl}" must be a PNG, JPEG, WebP, or GIF file (or an HTTPS URL to one).`,
+  );
+  err.code = 'UNSUPPORTED_MEDIA_TYPE';
+  err.hint = 'Convert the image to PNG, JPEG, or WebP, or pass an HTTPS URL.';
+  err.details = { source: pathOrUrl };
+  throw err;
+}
+
+async function prepareSeedanceReferenceImageUploadFile(pathOrUrl, buffer) {
+  const data = Buffer.from(buffer);
+  const mimeType = seedanceReferenceImageMimeType(pathOrUrl, data);
+  const filename = withMediaExtension(
+    mediaFilenameFromSource(pathOrUrl, 'reference-image'),
+    extensionForApiMediaReference(mimeType, 'image'),
+  );
+  const maxBytes = apiMediaReferenceMaxBytes();
+  if (data.length > maxBytes) {
+    const err = new Error(
+      `Seedance reference image "${pathOrUrl}" is ${data.length} bytes, above the ${maxBytes} byte upload limit.`,
+    );
+    err.code = 'MEDIA_REFERENCE_TOO_LARGE';
+    err.details = { source: pathOrUrl, byteLength: data.length, maxBytes };
+    throw err;
+  }
+  return {
+    buffer: data,
+    filename,
+    byteLength: data.length,
+    mimeType,
+  };
+}
+
+// Upload a local (non-HTTPS) Seedance loose-reference image and return its
+// hosted HTTPS download URL. The Client SDK's loose-reference arrays accept only
+// URL strings, so this is what lets `-c <local image>` work in direct generation
+// without forcing the user onto the --api-chat / --durable-chat path. Mirrors
+// uploadSeedanceReferenceAudioUrl / uploadSeedanceReferenceVideoUrl.
+async function uploadSeedanceReferenceImageUrl(pathOrUrl, apiKey, index = 0) {
+  const ref = { flag: '-c/--context', value: pathOrUrl, kind: 'image' };
+  const buffer = await fetchMediaBuffer(pathOrUrl);
+  const file = await prepareSeedanceReferenceImageUploadFile(pathOrUrl, buffer);
+  const uploaded = await uploadPreparedApiMediaReferenceV2(ref, index, apiKey, file);
+  return uploaded.url;
+}
+
 async function trimSeedanceV2VSourceVideoBuffer(buffer, sourceLabel, startOffset, requestedDuration) {
   const ffmpegPath = await ensureFfmpegAvailable();
   const tempDir = createTrackedTempDir('sogni-seedance-v2v-');
@@ -8086,19 +8166,24 @@ async function main() {
       // Seedance loose-reference extras: -c/--context images beyond start/end,
       // plus repeated --ref-audio / --ref-video entries past the first. The
       // Sogni Client SDK accepts only URL arrays for these (createJobRequestMessage),
-      // so extras MUST be HTTPS URLs. For multi-file local uploads, use --api-chat /
-      // --durable-chat where the LLM upload pipeline handles per-file uploads.
+      // so each entry must resolve to an HTTPS URL. HTTPS inputs are forwarded as-is
+      // (SSRF-validated); local files are uploaded to a Sogni-hosted URL first, the
+      // same way the primary --ref-audio / --ref-video locals are handled. This lets
+      // `-c <local image>` work in direct generation without a detour through
+      // --api-chat / --durable-chat.
       if (isSeedanceVideo) {
-        for (const ctxImage of (Array.isArray(options.contextImages) ? options.contextImages : [])) {
+        for (const [ctxIndex, ctxImage] of (Array.isArray(options.contextImages) ? options.contextImages : []).entries()) {
           if (!ctxImage) continue;
-          if (!isHttpsUrl(ctxImage)) {
-            fatalCliError(
-              `Seedance extra image reference "${ctxImage}" must be an HTTPS URL. ` +
-              'Local file uploads beyond --ref / --ref-end are only supported in --api-chat / --durable-chat mode.',
-              { code: 'INVALID_ARGUMENT', details: { flag: '-c/--context', value: ctxImage } },
+          if (isHttpsUrl(ctxImage)) {
+            await appendSafeSeedanceReferenceUrl(seedanceReferenceImageUrls, ctxImage, 'Seedance image reference');
+          } else {
+            const uploadedImageUrl = await uploadSeedanceReferenceImageUrl(
+              ctxImage,
+              creds.SOGNI_API_KEY,
+              ctxIndex,
             );
+            seedanceReferenceImageUrls.push(uploadedImageUrl);
           }
-          await appendSafeSeedanceReferenceUrl(seedanceReferenceImageUrls, ctxImage, 'Seedance image reference');
         }
         for (const [extraAudioIndex, extraAudio] of options.refAudios.entries()) {
           if (!isHttpsUrl(extraAudio)) {

package/version.mjs

@@ -1 +1 @@
-export const PACKAGE_VERSION = '3.6.0';
+export const PACKAGE_VERSION = '3.6.1';