@civitai/app-sdk 0.77.0 → 0.78.0

package/README.md

@@ -666,7 +666,7 @@ The starters in `civitai/civitai-app-starters` wire this into framework-specific
 
 ## Choosing a workflow step type
 
-The orchestrator is a workflow API: each request submits a list of typed steps. `WORKFLOW_STEP_TYPES` is the in-code catalog of every step `$type` it accepts, with a one-line description for each — `textToImage`, `imageGen`, `videoGen`, `comfy`, `customComfy`, `textToSpeech`, `aceStepAudio`, `transcription`, `imageUpscaler` among them (55 in total).
+The orchestrator is a workflow API: each request submits a list of typed steps. `WORKFLOW_STEP_TYPES` is the in-code catalog of every step `$type` it accepts, with a one-line description for each — `textToImage`, `imageGen`, `videoGen`, `comfy`, `customComfy`, `textToSpeech`, `aceStepAudio`, `transcription`, `imageUpscaler` among them (58 in total).
 
 The catalog is pinned to the orchestrator's published OpenAPI spec two ways — an offline unit test against a transcribed copy of the spec's `WorkflowStepTemplate` discriminator mapping, and a CI job (`pnpm check:catalogs`) that re-fetches the live spec and diffs it. If a `$type` is listed here, the orchestrator accepts it.
 
@@ -741,15 +741,15 @@ What it exports:
 
 | Export | What |
 |---|---|
-| `WorkflowStepTemplates` | `$type` → template type, for 47 of the catalog's 55 step types. Keyed by the WIRE name, which the generated type names don't always match (`model3DPreview` → `Model3dPreviewStepTemplate`). |
+| `WorkflowStepTemplates` | `$type` → template type, for 47 of the catalog's 58 step types. Keyed by the WIRE name, which the generated type names don't always match (`model3DPreview` → `Model3dPreviewStepTemplate`). |
 | `WorkflowStepTemplateFor<'videoGen'>` | One step's template. |
 | `WorkflowStepInputFor<'videoGen'>` | One step's `input` shape, without needing the generated `*Input` name. |
 | `AnyWorkflowStepTemplate` | Discriminated union of all 47 mapped templates — `Extract<…, { $type: 'comfy' }>` and exhaustive `switch` work. `@civitai/client`'s base `WorkflowStepTemplate` has `$type` as a bare `string`, so it narrows nothing. |
 | `TypedWorkflowTemplate` | The submit envelope with `steps` narrowed to that union. Pass it straight to `submitWorkflow` / `estimateWorkflow`. |
 
-> 🔴 **The map is not total over the catalog, and that is the expected state.** `WORKFLOW_STEP_TYPES` documents 55 `$type`s; this map covers 47. The 8 with no generated template in the pinned `@civitai/client` are `imageScanning`, `liveTextToSpeech`, `liveTranscription`, `merge`, `preprocessVideo`, `promptModeration`, `soniloAudioGen`, `yuE2`, and `WorkflowStepTemplateFor<…>` is a compile error for each of them.
+> 🔴 **The map is not total over the catalog, and that is the expected state.** `WORKFLOW_STEP_TYPES` documents 58 `$type`s; this map covers 47. The 11 with no generated template in the pinned `@civitai/client` are `archiveExtraction`, `audioModeration`, `imageScanning`, `liveTextToSpeech`, `liveTranscription`, `merge`, `multimodalEmbedding`, `preprocessVideo`, `promptModeration`, `soniloAudioGen`, `yuE2`, and `WorkflowStepTemplateFor<…>` is a compile error for each of them.
 >
-> The two surfaces move independently on purpose: the catalog tracks the **live** orchestrator spec (a daily job syncs it), while these types track whatever `@civitai/client` was last published from. So the catalog runs ahead and the client catches up. The gap is never silent — `test/orchestrator/step-templates.test-d.ts` carries it as a `never` ledger plus one `@ts-expect-error` per gap `$type`, and `test/orchestrator/step-count-prose.test.ts` derives all four numbers (55, 47, 8, and the names) from `WORKFLOW_STEP_TYPES` and the map's own AST, then fails if this paragraph or its twin in `src/orchestrator/steps.ts` disagrees by one character.
+> The two surfaces move independently on purpose: the catalog tracks the **live** orchestrator spec (a daily job syncs it), while these types track whatever `@civitai/client` was last published from. So the catalog runs ahead and the client catches up. The gap is never silent — `test/orchestrator/step-templates.test-d.ts` carries it as a `never` ledger plus one `@ts-expect-error` per gap `$type`, and `test/orchestrator/step-count-prose.test.ts` derives all four numbers (58, 47, 11, and the names) from `WORKFLOW_STEP_TYPES` and the map's own AST, then fails if this paragraph or its twin in `src/orchestrator/steps.ts` disagrees by one character.
 
 The generated `*StepTemplate` and `*Input` types are **not** re-exported individually. Using this subpath already requires `@civitai/client` installed, so if you want one by name, import it straight from there — `import type { TextToImageStepTemplate } from '@civitai/client'`.
 
@@ -788,7 +788,7 @@ Several of the 47 exist to serve Civitai's own pipelines rather than third-party
 
 `training` and `imageResourceTraining` are the exception: an App Block's host explicitly **allows** them on the pass-through arm (see `WorkflowBodyPassThroughStep` under Primitives above for the `maxBuzz` bound, and how a trained epoch is published). For a standalone app, the orchestrator's own authorization still decides.
 
-Note that `WORKFLOW_STEP_TYPES` does **not** mark most of them: of its 55 entries exactly three — `comfyNodepackSnapshot`, `qwenImageBench` and `promptModeration` — sit under its "Platform internals" heading, and the rest are ordinary documented entries (`webScrape` even carries usage notes). The reason the platform steps are typed anyway is not that the catalog flags them as internal; it's that the catalog *documents* them, so skipping them would make `WorkflowStepTemplateFor<'training'>` a compile error for a step type the SDK documents — which is exactly what is live today for each `$type` the pinned client cannot type, and is why that gap is spelled out above rather than left to be discovered.
+Note that `WORKFLOW_STEP_TYPES` does **not** mark most of them: of its 58 entries exactly three — `comfyNodepackSnapshot`, `qwenImageBench` and `promptModeration` — sit under its "Platform internals" heading, and the rest are ordinary documented entries (`webScrape` even carries usage notes). The reason the platform steps are typed anyway is not that the catalog flags them as internal; it's that the catalog *documents* them, so skipping them would make `WorkflowStepTemplateFor<'training'>` a compile error for a step type the SDK documents — which is exactly what is live today for each `$type` the pinned client cannot type, and is why that gap is spelled out above rather than left to be discovered.
 
 ## Public vs. confidential clients
 

package/dist/orchestrator/index.d.ts

@@ -196,6 +196,14 @@ export declare const WORKFLOW_STEP_TYPES: {
     readonly imageScanning: "Unified image moderation scan (NSFW level, AI/anime recognition, tagging, age detection)";
     /** Shieldstral text/prompt safety moderation (`mode: 'prompt' | 'text'`). */
     readonly shieldstralModeration: "Text / prompt safety moderation (Shieldstral)";
+    /**
+     * Transcribe the audio at `mediaUrl` and classify the lyrics for sexualized
+     * minors. `transcriber` is `auto`, `aceStep` or `asr`; `auto` sends speech in
+     * a supported language to ASR and everything else to ACE-Step. The output
+     * carries the `lyrics`, a `score`, `sexualizedMinors` and `blocked`, plus
+     * `hasAudio` and a `skippedReason`.
+     */
+    readonly audioModeration: "Audio moderation — transcribe lyrics and flag sexualized minors";
     /** ClamAV scan a model file for malware. */
     readonly modelClamScan: "Antivirus scan a model file";
     /** Pickle-scan a model file for unsafe pickles. */
@@ -206,6 +214,13 @@ export declare const WORKFLOW_STEP_TYPES: {
     readonly chatCompletion: "LLM chat completion";
     /** Generate a richer prompt from a short seed prompt. */
     readonly promptEnhancement: "Prompt expansion via LLM";
+    /**
+     * Embed 1–128 `inputs` (each a unique `id` carrying `text`, an `imageUrl` or
+     * a `videoUrl` of at most 60 s / 100 MiB, audio ignored) into normalized
+     * vectors with `wemm-2b`. `dimensions` is 64–2048 in powers of two, default
+     * 256.
+     */
+    readonly multimodalEmbedding: "Text / image / video embeddings (wemm-2b)";
     /**
      * Fetch one page and return its content. `url` and `formats` are both
      * REQUIRED; `formats` selects which representations come back — the spec
@@ -226,6 +241,13 @@ export declare const WORKFLOW_STEP_TYPES: {
     readonly echo: "Echo step — round-trip the input for testing";
     /** Package multiple blobs into a zip archive. */
     readonly blobArchive: "Zip multiple blobs into an archive";
+    /**
+     * Unpack a `.zip`, `.tar` or `.tar.gz` at `sourceUrl`. `mediaTypes` limits
+     * which kinds of entry are extracted (default all) and `maxEntries` caps the
+     * count (1–2000, default 2000); anything filtered or over the cap comes back
+     * in `rejected` rather than failing the step. `entries` are in archive order.
+     */
+    readonly archiveExtraction: "Extract the entries of a zip / tar / tar.gz archive";
     /** Combine fields from earlier steps' outputs into a single result. */
     readonly merge: "Merge — combine fields from earlier steps' outputs into one result";
     /** Snapshot the installed ComfyUI custom-node packs on a worker. */
@@ -272,6 +294,14 @@ export declare const IMAGE_GEN_ENGINES: {
      * workers) — picking this engine pins the first-party vendor path.
      */
     readonly krea: "Krea (Krea 2, first-party API)";
+    /**
+     * Krea 3: text-to-image, or an edit of up to six `images`. `resolution` is
+     * `1k` or `2k` (default `2k`), `thinkingMode` is `instant` / `medium` /
+     * `ultra`, and `quantity` is capped at 4. Omit `aspectRatio` to let Krea
+     * choose. Note the spec marks `images` required (no minimum length) while
+     * its own description says to omit it for text-to-image.
+     */
+    readonly krea3: "Krea 3 (generation and editing, 1K / 2K)";
     /** Self-hosted SDCpp (stable-diffusion.cpp) gen. */
     readonly sdcpp: "SDCpp (self-hosted diffusion)";
     /** fal.ai routed gen. */

package/dist/orchestrator/index.js

@@ -204,6 +204,14 @@ export const WORKFLOW_STEP_TYPES = {
     imageScanning: 'Unified image moderation scan (NSFW level, AI/anime recognition, tagging, age detection)',
     /** Shieldstral text/prompt safety moderation (`mode: 'prompt' | 'text'`). */
     shieldstralModeration: 'Text / prompt safety moderation (Shieldstral)',
+    /**
+     * Transcribe the audio at `mediaUrl` and classify the lyrics for sexualized
+     * minors. `transcriber` is `auto`, `aceStep` or `asr`; `auto` sends speech in
+     * a supported language to ASR and everything else to ACE-Step. The output
+     * carries the `lyrics`, a `score`, `sexualizedMinors` and `blocked`, plus
+     * `hasAudio` and a `skippedReason`.
+     */
+    audioModeration: 'Audio moderation — transcribe lyrics and flag sexualized minors',
     /** ClamAV scan a model file for malware. */
     modelClamScan: 'Antivirus scan a model file',
     /** Pickle-scan a model file for unsafe pickles. */
@@ -215,6 +223,13 @@ export const WORKFLOW_STEP_TYPES = {
     chatCompletion: 'LLM chat completion',
     /** Generate a richer prompt from a short seed prompt. */
     promptEnhancement: 'Prompt expansion via LLM',
+    /**
+     * Embed 1–128 `inputs` (each a unique `id` carrying `text`, an `imageUrl` or
+     * a `videoUrl` of at most 60 s / 100 MiB, audio ignored) into normalized
+     * vectors with `wemm-2b`. `dimensions` is 64–2048 in powers of two, default
+     * 256.
+     */
+    multimodalEmbedding: 'Text / image / video embeddings (wemm-2b)',
     // ----- Web ---------------------------------------------------------------
     /**
      * Fetch one page and return its content. `url` and `formats` are both
@@ -237,6 +252,13 @@ export const WORKFLOW_STEP_TYPES = {
     echo: 'Echo step — round-trip the input for testing',
     /** Package multiple blobs into a zip archive. */
     blobArchive: 'Zip multiple blobs into an archive',
+    /**
+     * Unpack a `.zip`, `.tar` or `.tar.gz` at `sourceUrl`. `mediaTypes` limits
+     * which kinds of entry are extracted (default all) and `maxEntries` caps the
+     * count (1–2000, default 2000); anything filtered or over the cap comes back
+     * in `rejected` rather than failing the step. `entries` are in archive order.
+     */
+    archiveExtraction: 'Extract the entries of a zip / tar / tar.gz archive',
     /** Combine fields from earlier steps' outputs into a single result. */
     merge: "Merge — combine fields from earlier steps' outputs into one result",
     // ----- Platform internals ------------------------------------------------
@@ -286,6 +308,14 @@ export const IMAGE_GEN_ENGINES = {
      * workers) — picking this engine pins the first-party vendor path.
      */
     krea: 'Krea (Krea 2, first-party API)',
+    /**
+     * Krea 3: text-to-image, or an edit of up to six `images`. `resolution` is
+     * `1k` or `2k` (default `2k`), `thinkingMode` is `instant` / `medium` /
+     * `ultra`, and `quantity` is capped at 4. Omit `aspectRatio` to let Krea
+     * choose. Note the spec marks `images` required (no minimum length) while
+     * its own description says to omit it for text-to-image.
+     */
+    krea3: 'Krea 3 (generation and editing, 1K / 2K)',
     /** Self-hosted SDCpp (stable-diffusion.cpp) gen. */
     sdcpp: 'SDCpp (self-hosted diffusion)',
     /** fal.ai routed gen. */

package/dist/orchestrator/steps.d.ts

@@ -3,7 +3,7 @@
  * own generated client (`@civitai/client`).
  *
  * The sibling `@civitai/app-sdk/orchestrator` module gives you the *catalog*
- * (`WORKFLOW_STEP_TYPES` — 55 `$type` names and what each one does) and the
+ * (`WORKFLOW_STEP_TYPES` — 58 `$type` names and what each one does) and the
  * fetch helpers (`submitWorkflow`, `estimateWorkflow`, …), but its body
  * builders take `input: unknown`. This module is the missing half: the actual
  * per-step input shapes, tracked against the orchestrator's OpenAPI spec by
@@ -78,7 +78,7 @@
  * orchestrator's own authorization still decides.
  *
  * ⚠️ `WORKFLOW_STEP_TYPES` does NOT mark most of them. Counted at this commit:
- * of its 55 entries, exactly THREE sit under its "Platform internals" heading —
+ * of its 58 entries, exactly THREE sit under its "Platform internals" heading —
  * `comfyNodepackSnapshot`, `qwenImageBench` and `promptModeration`. `training`,
  * `webScrape`, `xGuardModeration`, `modelPickleScan` and the `model*` / `media*` steps are
  * ordinary documented entries under ordinary headings, and `webScrape` carries
@@ -97,10 +97,11 @@
  * (#315) without anything going red. The measured state, derived rather than
  * typed:
  *
- * `WORKFLOW_STEP_TYPES` documents 55 `$type`s; this map covers 47. The 8 with
- * no generated template in the pinned `@civitai/client` are `imageScanning`,
- * `liveTextToSpeech`, `liveTranscription`, `merge`, `preprocessVideo`,
- * `promptModeration`, `soniloAudioGen`, `yuE2`, and
+ * `WORKFLOW_STEP_TYPES` documents 58 `$type`s; this map covers 47. The 11 with
+ * no generated template in the pinned `@civitai/client` are `archiveExtraction`,
+ * `audioModeration`, `imageScanning`, `liveTextToSpeech`, `liveTranscription`,
+ * `merge`, `multimodalEmbedding`, `preprocessVideo`, `promptModeration`,
+ * `soniloAudioGen`, `yuE2`, and
  * `WorkflowStepTemplateFor<…>` is a compile error for each of them.
  *
  * That gap is EXPECTED and is not a defect in either surface. The catalog
@@ -309,7 +310,7 @@ interface StepTemplateMap {
     xGuardModeration: XGuardModerationStepTemplate;
 }
 /**
- * `$type` → its step-template type, for 47 of the catalog's 55 step types.
+ * `$type` → its step-template type, for 47 of the catalog's 58 step types.
  *
  * Keyed by the WIRE name rather than the generated type name, because the wire
  * name is what you actually have in hand and the generator does not always

package/dist/orchestrator/steps.js

@@ -3,7 +3,7 @@
  * own generated client (`@civitai/client`).
  *
  * The sibling `@civitai/app-sdk/orchestrator` module gives you the *catalog*
- * (`WORKFLOW_STEP_TYPES` — 55 `$type` names and what each one does) and the
+ * (`WORKFLOW_STEP_TYPES` — 58 `$type` names and what each one does) and the
  * fetch helpers (`submitWorkflow`, `estimateWorkflow`, …), but its body
  * builders take `input: unknown`. This module is the missing half: the actual
  * per-step input shapes, tracked against the orchestrator's OpenAPI spec by
@@ -78,7 +78,7 @@
  * orchestrator's own authorization still decides.
  *
  * ⚠️ `WORKFLOW_STEP_TYPES` does NOT mark most of them. Counted at this commit:
- * of its 55 entries, exactly THREE sit under its "Platform internals" heading —
+ * of its 58 entries, exactly THREE sit under its "Platform internals" heading —
  * `comfyNodepackSnapshot`, `qwenImageBench` and `promptModeration`. `training`,
  * `webScrape`, `xGuardModeration`, `modelPickleScan` and the `model*` / `media*` steps are
  * ordinary documented entries under ordinary headings, and `webScrape` carries
@@ -97,10 +97,11 @@
  * (#315) without anything going red. The measured state, derived rather than
  * typed:
  *
- * `WORKFLOW_STEP_TYPES` documents 55 `$type`s; this map covers 47. The 8 with
- * no generated template in the pinned `@civitai/client` are `imageScanning`,
- * `liveTextToSpeech`, `liveTranscription`, `merge`, `preprocessVideo`,
- * `promptModeration`, `soniloAudioGen`, `yuE2`, and
+ * `WORKFLOW_STEP_TYPES` documents 58 `$type`s; this map covers 47. The 11 with
+ * no generated template in the pinned `@civitai/client` are `archiveExtraction`,
+ * `audioModeration`, `imageScanning`, `liveTextToSpeech`, `liveTranscription`,
+ * `merge`, `multimodalEmbedding`, `preprocessVideo`, `promptModeration`,
+ * `soniloAudioGen`, `yuE2`, and
  * `WorkflowStepTemplateFor<…>` is a compile error for each of them.
  *
  * That gap is EXPECTED and is not a defect in either surface. The catalog

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@civitai/app-sdk",
-  "version": "0.77.0",
+  "version": "0.78.0",
   "description": "OAuth + PKCE, encrypted-cookie sessions, scopes, and orchestrator helpers for building third-party Civitai apps.",
   "license": "MIT",
   "type": "module",