@sogni-ai/sogni-client 4.0.0-alpha.4 → 4.0.0-alpha.41

package/README.md

@@ -1,9 +1,28 @@
 # Sogni SDK for JavaScript & Node.js
-This library provides an easy way to interact with the [Sogni AI Supernet](https://www.sogni.ai/supernet) - a DePIN protocol for creative AI. It is written in TypeScript and can be used 
+This library provides an easy way to interact with the [Sogni AI Supernet](https://www.sogni.ai/supernet) - a DePIN protocol for creative AI. It is written in TypeScript and can be used
 in both TypeScript and JavaScript projects such as backend Node.js and browser environments.
 
 Behind the scenes this SDK uses WebSocket connection for communication between clients, server, and workers. It harnesses an event-based API to interact with Supernet to make things super efficient.
 
+## Features
+- 🎨 **Image Generation** - Create images with the latest frontier Open Source models like Stable Diffusion and Flux
+- 🎬 **Video Generation** - Generate videos using **Wan 2.2 14B FP8** models with five workflow types:
+  - Text-to-Video (t2v) - Generate videos from text prompts
+  - Image-to-Video (i2v) - Animate static images
+  - Sound-to-Video (s2v) - Generate videos synchronized with audio
+  - Animate-Move - Transfer motion from reference video to image subject
+  - Animate-Replace - Replace subjects in videos while preserving motion
+- ⚡ **Fast & Relaxed Networks** - Choose between high-speed GPU network or cost-effective Mac network
+- 🔄 **Real-time Progress** - Event-based API with progress tracking and live updates
+- 🎯 **Advanced Controls** - Fine-tune generation with samplers, schedulers, ControlNets, and more
+## Migration notes
+### v3.x.x to v4.x.x
+Version 4 adds support for video generation, including the new **Wan 2.2 14B FP8** model family with five workflow types (text-to-video, image-to-video, sound-to-video, animate-move, and animate-replace). There are the following breaking changes:
+- `type` is required when calling `sogni.projects.create(params)`, valid values are `image` and `video`. See code examples below.
+- `numberOfImages` renamed to `numberOfMedia`
+- `hasResultImage` in `Job` class is now `hasResultMedia`
+- `Job` and `Project` classes now have `type` property that can be `image` or `video`
+
 ## Installation
 Add library to your project using npm or yarn:
 ```bash
@@ -23,17 +42,17 @@ Your account is tied to a [Base](https://www.base.org/) Wallet that is created d
 
 ### Supernet Types
 There are 2 worker network types available:
-- `fast` - this network runs on high-end GPUs and is optimized for speed. It is more expensive than `relaxed` network.
-- `relaxed` - this network runs on Apple Mac devices and is optimized for cost. It is cheaper than `fast` network.
+- `fast` - this network runs on high-end GPUs and is optimized for speed. It is more expensive than `relaxed` network. **Required for video generation**.
+- `relaxed` - this network runs on Apple Mac devices and is optimized for cost. It is cheaper than `fast` network. Supports image generation only.
 
 In both options, the more complex your query is (the more steps), the higher the cost in tokens.
 
 ### Inference definitions: Projects and Jobs
-One request for image generation is called a **Project**. Project can generate one or more images. 
-Each image is represented by a **Job**.
+One request for image or video generation is called a **Project**. A project can generate one or more images or videos.
+Each generated image or video is represented by a **Job**.
 
-When you send a project to Supernet, it will be processed by one or more workers. The resulting images will be encrypted and 
-uploaded to Sogni servers where it will be stored for 24 hours. After this period images will be auto-deleted.
+When you send a project to Supernet, it will be processed by one or more workers. The resulting media will be encrypted and
+uploaded to Sogni servers where it will be stored for 24 hours. After this period, media files will be auto-deleted.
 
 ## Client initialization
 To initialize a client, you need to provide `appId`, and account credentials.
@@ -49,13 +68,13 @@ const options = {
   network: 'fast', // Network to use, 'fast' or 'relaxed'
 }
 
-const client = await SogniClient.createInstance(options);
+const sogni = await SogniClient.createInstance(options);
 // Login to Sogni account and establish WebSocket connection to Supernet
-await client.account.login(USERNAME, PASSWORD);
+await sogni.account.login(USERNAME, PASSWORD);
 // Now wait until list of available models is received.
 // This step is only needed if you want to create project immediately.
-const models = await client.projects.waitForModels();
-// You can get list of available models any time from `client.projects.availableModels`
+const models = await sogni.projects.waitForModels();
+// You can get list of available models any time from `sogni.projects.availableModels`
 ```
 **Important Note:** 
 - This sample assume you are using ES modules, which allow `await` on the top level, if you are CommomJS you will need to wrap `await` calls in an async function.
@@ -64,16 +83,44 @@ const models = await client.projects.waitForModels();
 
 ## Usage
 After calling `login` method, the client will establish a WebSocket connection to Sogni Supernet. Within a short period of time the
-client will receive the current balance and list of available models. After this you can start using the client to generate images.
+client will receive the current balance and list of available models. After this you can start using the client to generate images or videos.
+
+It is advised to watch for `connected` and `disconnected` events on the client instance to be notified when the connection is established or lost:
+```typescript
+// Will be triggered when the client is connected to Supernet
+sogni.client.on('connected', ({network}) => {
+  console.log('Connected to Supernet:', network);
+});
+
+// Will be triggered when websocket connection is lost or the client is disconnected from Supernet
+sogni.client.on('disconnected', ({code, reason}) => {
+  console.log('Disconnected from Supernet:', code, reason);
+});
+```
+
+## Image Generation
+
+Sogni supports a wide range of models for image generation. You can find a list of available models in 
+`sogni.projects.availableModels` property during runtime or query it using `sogni.projects.getAvailableModels()` method.
+
+For a start, you can try FLUX.1 \[schnell\] with the following parameters:
+```javascript
+const fluxDefaults = {
+  modelId: 'flux1-schnell-fp8',
+  steps: 4,
+  guidance: 1
+}
+```
 
-### Creating project
+### Creating an image project
 ```javascript
 // Find model that has the most workers
-const mostPopularModel = client.projects.availableModels.reduce((a, b) =>
+const mostPopularModel = sogni.projects.availableModels.reduce((a, b) =>
   a.workerCount > b.workerCount ? a : b
 );
 // Create a project using the most popular model
-const project = await client.projects.create({
+const project = await sogni.projects.create({
+  type: 'image',
   modelId: mostPopularModel.id,
   positivePrompt: 'A cat wearing a hat',
   negativePrompt:
@@ -81,7 +128,7 @@ const project = await client.projects.create({
   stylePrompt: 'anime',
   steps: 20, 
   guidance: 7.5, 
-  numberOfImages: 1,
+  numberOfMedia: 1,
   outputFormat: 'jpg' // Can be 'png' or 'jpg', defaults to 'png'
 });
 ```
@@ -94,7 +141,8 @@ In general, there are 2 ways to work with API:
 
 #### Using promises
 ```javascript
-const project = await client.projects.create({
+const project = await sogni.projects.create({
+  type: 'image',
   modelId: mostPopularModel.id,
   steps: 20,
   guidance: 7.5,
@@ -102,7 +150,7 @@ const project = await client.projects.create({
   negativePrompt:
     'malformation, bad anatomy, bad hands, missing fingers, cropped, low quality, bad quality, jpeg artifacts, watermark',
   stylePrompt: 'anime',
-  numberOfImages: 4
+  numberOfMedia: 4
 });
 
 project.on('progress', (progress) => {
@@ -117,7 +165,8 @@ console.log('Image URLs:', imageUrls);
 
 #### Using events
 ```javascript
-const project = await client.projects.create({
+const project = await sogni.projects.create({
+  type: 'image',
   modelId: mostPopularModel.id,
   steps: 20,
   guidance: 7.5,
@@ -125,7 +174,7 @@ const project = await client.projects.create({
   negativePrompt:
     'malformation, bad anatomy, bad hands, missing fingers, cropped, low quality, bad quality, jpeg artifacts, watermark',
   stylePrompt: 'anime',
-  numberOfImages: 4
+  numberOfMedia: 4
 });
 
 // Fired when one of project jobs completed, you can get the resultUrl from the job
@@ -182,7 +231,7 @@ Turbo and LCM models are designed for quality output in as little as 1 step. ([M
 - `disableNSFWFilter` - disable NSFW filter for this project. NSFW filter is enabled by default and workers won't upload resulting images if they are detected as NSFW.
 - `seed` - uint32 number to use as seed. If not provided, random seed will be used. If `numberOfImages` is greater than 1, provided seed will be user only for one of them. ([More info](https://docs.sogni.ai/learn/basics/generation-seed)).
 - `numberOfPreviews` - number of preview images to generate. If not provided, no preview images will be generated.
-- `sampler` - time step spacing algorithm ([More info](https://docs.sogni.ai/sogni-studio/advanced/samplers-and-schedulers)). For available options, see the **"Samplers"** section below.
+- `sampler` - sampler algorithm ([More info](https://docs.sogni.ai/sogni-studio/advanced/samplers-and-schedulers)). For available options, see the **"Samplers"** section below.
 - `scheduler` - scheduler to use ([More info](https://docs.sogni.ai/sogni-studio/advanced/samplers-and-schedulers)). For available options, see the **"Schedulers"** section below.
 - `startingImage` - guide image in PNG format. Can be [File](https://developer.mozilla.org/en-US/docs/Web/API/File), [Blob](https://developer.mozilla.org/en-US/docs/Web/API/Blob) or [Buffer](https://nodejs.org/api/buffer.html)
 - `startingImageStrength` - strong effect of starting image should be. From 0 to 1, default 0.5. 
@@ -192,9 +241,9 @@ Turbo and LCM models are designed for quality output in as little as 1 step. ([M
 TypeScript type definitions for project parameters can be found in [ProjectParams](https://sdk-docs.sogni.ai/interfaces/ProjectParams.html) docs.
 
 ### Detecting available output presets
-You can get a list of available output presets for a specific network and model using `client.projects.getOutputPresets` method.
+You can get a list of available output presets for a specific network and model using `sogni.projects.getOutputPresets` method.
 ```javascript
-const presets = await client.projects.getSizePresets('fast', 'flux1-schnell-fp8');
+const presets = await sogni.projects.getSizePresets('fast', 'flux1-schnell-fp8');
 console.log('Available output presets:', presets);
 ```
 Sample response:
@@ -290,17 +339,14 @@ Available sampler options:
 
 | Option          | Description                         |
 |-----------------|-------------------------------------|
-| `dfs_sd3`       | Discrete Flow Scheduler (SD3)       |
+| `dfs_sd3`       | Discrete Flow Sampler (SD3)         |
 | `dpm_pp`        | DPM Solver Multistep (DPM-Solver++) |
 | `dpm_pp_sde`    | DPM++ SDE                           |
 | `dpm_pp_2m`     | DPM++ 2M                            |
-| `dpm_pp_2m_sde` | DPM++ 2M SDE                        |
 | `euler`         | Euler                               |
 | `euler_a`       | Euler a                             |
-| `heun`          | Heun                                |
 | `lcm`           | LCM (Latent Consistency Model)      |
 | `pndm_plms`     | PNDM (Pseudo-linear multi-step)     |
-| `uni_pc`        | UniPC                               |
 
 **IMPORTANT:** Sampler compatibility depends on model and network. See [Samplers and Schedulers](https://docs.sogni.ai/sogni-studio/advanced/samplers-and-schedulers) docs for more info.
 
@@ -356,10 +402,11 @@ To use ControlNet in your project, you need to provide `controlNet` object with
 Example:
 ```javascript
 const cnImage = fs.readFileSync('./cn.jpg');
-const project = await client.projects.create({
+const project = await sogni.projects.create({
+  type: 'image',
   network: 'fast',
   modelId: 'coreml-cyberrealistic_v70_768',
-  numberOfImages: 1,
+  numberOfMedia: 1,
   positivePrompt: 'make men look older',
   steps: 20,
   guidance: 7.5,
@@ -400,5 +447,189 @@ export interface ControlNetParams {
 ```
 
 
-## Code examples
-You can find more code examples in the [examples](https://github.com/Sogni-AI/sogni-client/tree/main/examples) directory.
+## Video Generation with Wan 2.2 Models
+
+The Sogni SDK supports advanced video generation workflows powered by **Wan 2.2 14B FP8** models. These models are available on the `fast` network and support various video generation workflows.
+
+### Available Wan 2.2 Workflows
+
+The Wan 2.2 model family supports five distinct video generation workflows:
+
+1. **Text-to-Video (t2v)** - Generate videos from text prompts
+2. **Image-to-Video (i2v)** - Animate a static image into a video (First and Last Frame supported)
+3. **Sound-to-Video (s2v)** - Bring a character in an image to life with video and audio synchronization including lip syncing
+4. **Animate-Move** - Transfer character motion and emotion from a reference video to a subject from an image into a new video
+5. **Animate-Replace** - Replace a subject in a video while preserving motion
+
+### Model Variants
+
+Each workflow has two model variants optimized for different use cases:
+
+- **Speed variant** (with `_lightx2v` suffix) - Faster inference (4-step), good quality
+- **Quality variant** (without `_lightx2v`) - Slower inference, best quality
+
+Example model IDs:
+- `wan_v2.2-14b-fp8_t2v_lightx2v` (Text-to-Video, speed)
+- `wan_v2.2-14b-fp8_t2v` (Text-to-Video, quality)
+- `wan_v2.2-14b-fp8_i2v_lightx2v` (Image-to-Video, speed)
+- `wan_v2.2-14b-fp8_i2v` (Image-to-Video, quality)
+- `wan_v2.2-14b-fp8_s2v_lightx2v` (Sound-to-Video, speed)
+- `wan_v2.2-14b-fp8_s2v` (Sound-to-Video, quality)
+- `wan_v2.2-14b-fp8_animate-move_lightx2v` (Animate-Move, speed)
+- `wan_v2.2-14b-fp8_animate-replace_lightx2v` (Animate-Replace, speed)
+
+### Video Parameters
+
+When creating video projects, you can specify:
+
+- `fps` - Frames per second: 16 or 32 (default: 16)
+- `frames` - Number of frames: 17-161 (default: 81, which is ~5 seconds at 16fps)
+- `width` - Video width in pixels
+- `height` - Video height in pixels
+- `steps` - Increase inference steps to increase quality
+- `seed` - Random seed for reproducibility
+- `referenceImage` - Reference image for workflows that require it (i2v, s2v, animate-move, animate-replace)
+- `referenceVideo` - Reference video for animate workflows (animate-move, animate-replace)
+- `referenceAudio` - Reference audio for sound-to-video workflow
+
+### Text-to-Video Example
+
+```javascript
+const project = await sogni.projects.create({
+  type: 'video',
+  network: 'fast',
+  modelId: 'wan_v2.2-14b-fp8_t2v_lightx2v',
+  positivePrompt: 'A serene ocean wave crashing on a beach at sunset',
+  fps: 16,
+  frames: 81,
+  width: 512,
+  height: 512
+});
+
+const videoUrls = await project.waitForCompletion();
+console.log('Video URL:', videoUrls[0]);
+```
+
+### Image-to-Video Example
+
+```javascript
+const referenceImage = fs.readFileSync('./input-image.png');
+
+const project = await sogni.projects.create({
+  type: 'video',
+  network: 'fast',
+  modelId: 'wan_v2.2-14b-fp8_i2v_lightx2v',
+  positivePrompt: 'camera zooms in slowly',
+  referenceImage: referenceImage,
+  fps: 16,
+  frames: 81
+});
+
+const videoUrls = await project.waitForCompletion();
+```
+
+### Sound-to-Video Example
+
+```javascript
+const referenceImage = fs.readFileSync('./image.jpg');
+const referenceAudio = fs.readFileSync('./audio.m4a');
+
+const project = await sogni.projects.create({
+  type: 'video',
+  network: 'fast',
+  modelId: 'wan_v2.2-14b-fp8_s2v_lightx2v',
+  referenceImage: referenceImage,
+  referenceAudio: referenceAudio,
+  fps: 16,
+  frames: 81
+});
+
+const videoUrls = await project.waitForCompletion();
+```
+
+### Animate-Move Example
+
+Transfer motion from a reference video to a subject in an image:
+
+```javascript
+const referenceImage = fs.readFileSync('./subject.jpg');
+const referenceVideo = fs.readFileSync('./motion-source.mp4');
+
+const project = await sogni.projects.create({
+  type: 'video',
+  network: 'fast',
+  modelId: 'wan_v2.2-14b-fp8_animate-move_lightx2v',
+  referenceImage: referenceImage,
+  referenceVideo: referenceVideo,
+  fps: 16,
+  frames: 81
+});
+
+const videoUrls = await project.waitForCompletion();
+```
+
+### Animate-Replace Example
+
+Replace a subject in a video while preserving the original motion:
+
+```javascript
+const referenceImage = fs.readFileSync('./new-subject.jpg');
+const referenceVideo = fs.readFileSync('./original-video.mp4');
+
+const project = await sogni.projects.create({
+  type: 'video',
+  network: 'fast',
+  modelId: 'wan_v2.2-14b-fp8_animate-replace_lightx2v',
+  referenceImage: referenceImage,
+  referenceVideo: referenceVideo,
+  fps: 16,
+  frames: 81
+});
+
+const videoUrls = await project.waitForCompletion();
+```
+
+## Code Examples
+
+The [examples](https://github.com/Sogni-AI/sogni-client/tree/main/examples) directory contains working examples for all workflows:
+
+### Image Workflow Examples
+- **`workflow_text_to_image.mjs`** - Text-to-image generation with multiple model options
+- **`workflow_image_edit.mjs`** - Reference-based image generation using context images
+
+### Video Workflow Examples
+- **`workflow_text_to_video.mjs`** - Text-to-video generation with WAN 2.2 models
+- **`workflow_image_to_video.mjs`** - Animate static images into videos
+- **`workflow_sound_to_video.mjs`** - Audio-synchronized video generation with lip-sync
+- **`workflow_video_to_video.mjs`** - Motion transfer and character replacement (Animate-Move/Animate-Replace)
+
+### Basic Examples
+- **`promise_based.mjs`** - Image generation using promises/async-await
+- **`event_driven.js`** - Image generation using event listeners
+
+### Featured Models
+
+The workflow examples showcase powerful new models:
+
+| Model ID | Description | Use Case |
+|----------|-------------|----------|
+| `z_image_turbo_bf16` | Z-Image Turbo - Fast 4-step generation | Quick text-to-image prototyping |
+| `qwen_image_edit_2511_fp8_lightning` | Qwen Image Edit Lightning - Fast 4-step | Quick reference-based generation |
+| `qwen_image_edit_2511_fp8` | Qwen Image Edit - High quality 20-step | Professional image editing |
+| `flux2_dev_fp8` | Flux.2 Dev - Context-aware generation | High-quality images with reference support |
+
+All workflow examples include:
+- Interactive model and parameter selection
+- Balance checking and cost confirmation
+- Real-time progress tracking with ETA
+- Error handling with detailed feedback
+- Automatic file download and preview
+
+Run any workflow example:
+```bash
+cd examples
+npm install
+node workflow_text_to_image.mjs
+node workflow_image_edit.mjs
+node workflow_text_to_video.mjs
+```

package/dist/Account/index.d.ts

@@ -6,11 +6,11 @@ import CurrentAccount from './CurrentAccount';
 import { SupernetType } from '../ApiClient/WebSocketClient/types';
 /**
  * Account API methods that let you interact with the user's account.
- * Can be accessed via `client.account`. Look for more samples below.
+ * Can be accessed via `sogni.account`. Look for more samples below.
  *
  * @example Retrieve the current account balance
  * ```typescript
- * const balance = await client.account.refreshBalance();
+ * const balance = await sogni.account.refreshBalance();
  * console.log(balance);
  * ```
  *
@@ -19,6 +19,7 @@ declare class AccountApi extends ApiGroup {
     readonly currentAccount: CurrentAccount;
     constructor(config: ApiConfig);
     private handleBalanceUpdate;
+    private handleChangeNetwork;
     private handleServerConnected;
     private handleServerDisconnected;
     private handleAuthUpdated;
@@ -35,7 +36,7 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Create a wallet from username and password
      * ```typescript
-     * const wallet = client.account.getWallet('username', 'password');
+     * const wallet = sogni.account.getWallet('username', 'password');
      * console.log(wallet.address);
      * ```
      *
@@ -53,7 +54,7 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Login with username and password
      * ```typescript
-     * await client.account.login('username', 'password');
+     * await sogni.account.login('username', 'password');
      * console.log('Logged in');
      * ```
      *
@@ -68,7 +69,7 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Logout the user
      * ```typescript
-     * await client.account.logout();
+     * await sogni.account.logout();
      * console.log('Logged out');
      * ```
      */
@@ -82,7 +83,7 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Refresh user account balance
      * ```typescript
-     * const balance = await client.account.refreshBalance();
+     * const balance = await sogni.account.refreshBalance();
      * console.log(balance);
      * ```
      */
@@ -93,7 +94,7 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Get the account balance of the current user
      * ```typescript
-     * const balance = await client.account.accountBalance();
+     * const balance = await sogni.account.accountBalance();
      * console.log(balance);
      * ```
      */
@@ -105,15 +106,16 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Get the balance of the wallet address
      * ```typescript
-     * const address = client.account.currentAccount.walletAddress;
-     * const balance = await client.account.walletBalance(address);
+     * const address = sogni.account.currentAccount.walletAddress;
+     * const balance = await sogni.account.walletBalance(address);
      * console.log(balance);
      * // { token: '100.000000', ether: '0.000000' }
      * ```
      *
      * @param walletAddress
+     * @param provider - blockchain provider, 'base' or 'etherlink' defaults to 'base'
      */
-    walletBalance(walletAddress: string): Promise<{
+    walletBalance(walletAddress: string, provider?: 'base' | 'etherlink'): Promise<{
         sogni: string;
         spark: string;
         ether: string;
@@ -132,9 +134,9 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Switch to the fast network
      * ```typescript
-     * await client.account.switchNetwork('fast');
+     * await sogni.account.switchNetwork('fast');
      * console.log('Switched to the fast network, now lets wait until we get list of models');
-     * await client.projects.waitForModels();
+     * await sogni.projects.waitForModels();
      * ```
      * @param network - Network type to switch to
      */
@@ -144,10 +146,10 @@ declare class AccountApi extends ApiGroup {
      *
      * @example Get the transaction history
      * ```typescript
-     * const { entries, next } = await client.account.transactionHistory({
+     * const { entries, next } = await sogni.account.transactionHistory({
      *  status: 'completed',
      *  limit: 10,
-     *  address: client.account.currentAccount.walletAddress
+     *  address: sogni.account.currentAccount.walletAddress
      * });
      * ```
      *
@@ -176,7 +178,7 @@ declare class AccountApi extends ApiGroup {
      * Withdraw funds from the current account to wallet.
      * @example withdraw to current wallet address
      * ```typescript
-     * await client.account.withdraw('your-account-password', 100);
+     * await sogni.account.withdraw('your-account-password', 100);
      * ```
      *
      * @param password - account password
@@ -187,7 +189,7 @@ declare class AccountApi extends ApiGroup {
      * Deposit tokens from wallet to account
      * @example withdraw to current wallet address
      * ```typescript
-     * await client.account.deposit('your-account-password', 100);
+     * await sogni.account.deposit('your-account-password', 100);
      * ```
      *
      * @param password - account password