@vargai/sdk 0.1.1

package/SKILLS.md

@@ -0,0 +1,157 @@
+# agent skills
+
+this sdk includes claude code agent skills for each service. each skill is co-located with its service code.
+
+## available skills
+
+### service skills
+
+located in `service/<name>/SKILL.md`:
+
+1. **image-generation** (`service/image/`)
+   - generate ai images using fal (flux models) or higgsfield soul characters
+   - cli: `bun run service/image fal|soul <prompt> [options]`
+
+2. **video-generation** (`service/video/`)
+   - generate videos from images (local or url) or text prompts using fal.ai
+   - supports local image files - automatically uploads to fal storage
+   - cli: `bun run service/video from_image|from_text <args>`
+
+3. **voice-synthesis** (`service/voice/`)
+   - generate realistic text-to-speech audio using elevenlabs
+   - cli: `bun run service/voice generate|elevenlabs <text> [options]`
+
+3b. **music-generation** (`lib/elevenlabs.ts`)
+   - generate music from text prompts using elevenlabs
+   - generate sound effects from descriptions
+   - cli: `bun run lib/elevenlabs.ts music|sfx <prompt> [options]`
+
+4. **video-lipsync** (`service/sync/`)
+   - sync video with audio using wav2lip or simple overlay
+   - cli: `bun run service/sync sync|wav2lip|overlay <args>`
+
+5. **video-captions** (`service/captions/`)
+   - add auto-generated or custom subtitles to videos
+   - cli: `bun run service/captions <videoPath> [options]`
+
+6. **video-editing** (`service/edit/`)
+   - edit videos with ffmpeg (resize, trim, concat, social media prep)
+   - cli: `bun run service/edit social|montage|trim|resize|merge_audio <args>`
+
+7. **audio-transcription** (`service/transcribe/`)
+   - transcribe audio to text or subtitles using groq/fireworks
+   - cli: `bun run service/transcribe <audioUrl> <provider> [outputPath]`
+
+### utility skills
+
+8. **telegram-send** (external: `/Users/aleks/Github/Badaboom1995/rumble-b2c`)
+   - send videos to telegram users/channels as round videos
+   - automatically converts to 512x512 square format for telegram
+   - cli: `cd /Users/aleks/Github/Badaboom1995/rumble-b2c && bun run scripts/telegram-send-video.ts <videoPath> <@username>`
+   - example: `cd /Users/aleks/Github/Badaboom1995/rumble-b2c && bun run scripts/telegram-send-video.ts /path/to/video.mp4 @caffeinum`
+
+### pipeline skills
+
+located in `pipeline/cookbooks/SKILL.md`:
+
+9. **talking-character-pipeline** (`pipeline/cookbooks/`)
+   - complete workflow to create talking character videos
+   - combines: character generation → voiceover → animation → lipsync → captions → social prep
+
+10. **round-video-character** (`pipeline/cookbooks/round-video-character.md`)
+   - create realistic round selfie videos for telegram using nano banana pro + wan 2.5
+   - workflow: generate selfie first frame (person in setting) → voiceover → wan 2.5 video
+   - uses: `bun run lib/fal.ts`, `bun run lib/replicate.ts`, `bun run lib/elevenlabs.ts`
+   - input: text script + profile photo
+   - output: extreme close-up selfie video with authentic camera shake, lighting, and audio
+
+## structure
+
+each skill follows this pattern:
+
+```
+service/<name>/
+├── index.ts      # service implementation
+└── SKILL.md      # claude code agent skill
+```
+
+## how skills work
+
+skills are **model-invoked** - claude autonomously decides when to use them based on your request and the skill's description.
+
+**example:**
+- you say: "create a talking character video"
+- claude reads `talking-character-pipeline` skill
+- claude executes the workflow using the pipeline steps
+
+## using skills
+
+### in claude code
+
+skills are automatically discovered when you're in the sdk directory:
+
+```
+user: create an image of a sunset
+claude: [uses image-generation skill]
+        bun run service/image fal "beautiful sunset over mountains"
+```
+
+### manually
+
+you can also run services directly:
+
+```bash
+# generate image
+bun run service/image fal "sunset over mountains" true
+
+# generate video from that image
+bun run service/video from_image "camera pan" https://image-url.jpg 5 true
+
+# add voice
+bun run service/voice elevenlabs "this is a beautiful sunset" rachel true
+
+# sync with video
+bun run service/sync wav2lip https://video-url.mp4 https://audio-url.mp3
+```
+
+## skill features
+
+each skill includes:
+
+- **name**: unique skill identifier
+- **description**: when claude should use this skill
+- **allowed-tools**: restricted to Read, Bash for safety
+- **usage examples**: cli and programmatic examples
+- **when to use**: specific use cases
+- **tips**: best practices
+- **environment variables**: required api keys
+
+## benefits
+
+- **discoverability**: claude knows all available services
+- **context**: skills provide usage examples and best practices
+- **safety**: `allowed-tools` limits to read-only and bash execution
+- **documentation**: skills serve as living documentation
+
+## skill reference
+
+| skill | service | primary use case |
+|-------|---------|------------------|
+| image-generation | image | create ai images, character headshots |
+| video-generation | video | animate images, generate video clips |
+| voice-synthesis | voice | text-to-speech, voiceovers |
+| music-generation | elevenlabs | generate music, create sound effects |
+| video-lipsync | sync | sync audio with video, talking characters |
+| video-captions | captions | add subtitles, accessibility |
+| video-editing | edit | resize, trim, social media optimization |
+| audio-transcription | transcribe | speech-to-text, subtitle generation |
+| telegram-send | external | send videos to telegram as round videos |
+| talking-character-pipeline | pipeline | end-to-end talking character videos |
+| round-video-character | pipeline | telegram round selfie videos with wan 2.5 |
+
+## see also
+
+- [README.md](README.md) - sdk overview and installation
+- [STRUCTURE.md](STRUCTURE.md) - detailed module organization
+- [pipeline/cookbooks/talking-character.md](pipeline/cookbooks/talking-character.md) - talking character workflow
+- [pipeline/cookbooks/round-video-character.md](pipeline/cookbooks/round-video-character.md) - telegram round selfie video cookbook

package/STRUCTURE.md

@@ -0,0 +1,92 @@
+# sdk structure
+
+## lib/ - two fal implementations
+
+### lib/ai-sdk/fal.ts
+uses `@ai-sdk/fal` with vercel ai sdk
+
+**when to use:**
+- standard image generation
+- need consistent api across providers
+- want automatic image format handling
+- prefer typed aspect ratios
+
+**commands:**
+```bash
+bun run lib/ai-sdk/fal.ts generate_image <prompt> [model] [aspectRatio]
+```
+
+### lib/fal.ts
+uses `@fal-ai/client` directly
+
+**when to use:**
+- video generation (image-to-video, text-to-video)
+- advanced fal features
+- need queue/streaming updates
+- custom api parameters
+
+**commands:**
+```bash
+bun run lib/fal.ts generate_image <prompt> [model] [imageSize]
+bun run lib/fal.ts image_to_video <prompt> <imageUrl> [duration]
+bun run lib/fal.ts text_to_video <prompt> [duration]
+```
+
+### lib/higgsfield.ts
+uses `@higgsfield/client` for soul character generation
+
+**commands:**
+```bash
+bun run lib/higgsfield.ts generate_soul <prompt> [customReferenceId]
+bun run lib/higgsfield.ts create_character <name> <imageUrl1> [imageUrl2...]
+bun run lib/higgsfield.ts list_styles
+```
+
+## service/ - high-level wrappers
+
+### service/image.ts
+combines fal + higgsfield for image generation
+
+```bash
+bun run service/image.ts fal <prompt> [model] [upload]
+bun run service/image.ts soul <prompt> [customReferenceId] [upload]
+```
+
+### service/video.ts
+video generation with optional s3 upload
+
+```bash
+bun run service/video.ts from_image <prompt> <imageUrl> [duration] [upload]
+bun run service/video.ts from_text <prompt> [duration] [upload]
+```
+
+## utilities/
+
+### utilities/s3.ts
+cloudflare r2 / s3 storage operations
+
+```bash
+bun run utilities/s3.ts upload <filePath> <objectKey>
+bun run utilities/s3.ts upload_from_url <url> <objectKey>
+bun run utilities/s3.ts presigned_url <objectKey> [expiresIn]
+```
+
+## pipeline/cookbooks/
+markdown guides for complex workflows
+
+- `talking-character.md`: create talking character videos
+
+## dependencies
+
+- `@ai-sdk/fal` - vercel ai sdk fal provider
+- `@fal-ai/client` - official fal client
+- `@higgsfield/client` - higgsfield api client
+- `@aws-sdk/client-s3` - s3 storage
+- `ai` - vercel ai sdk core
+
+## key decisions
+
+1. **two fal implementations** - ai-sdk for simplicity, client for power
+2. **all scripts are cli + library** - can be run directly or imported
+3. **consistent logging** - `[module] message` format
+4. **auto image opening** - ai-sdk version opens images automatically

package/TEST_RESULTS.md

@@ -0,0 +1,122 @@
+# test results
+
+## ✅ both fal approaches working
+
+### approach 1: lib/ai-sdk/fal.ts (vercel ai sdk)
+
+```bash
+$ bun run lib/ai-sdk/fal.ts generate_image "futuristic spaceship" "fal-ai/flux/dev" "16:9"
+
+[ai-sdk/fal] generating image with fal-ai/flux/dev
+[ai-sdk/fal] prompt: futuristic spaceship interior
+[ai-sdk/fal] aspect ratio: 16:9
+[ai-sdk/fal] completed!
+
+image saved to: /tmp/fal-ai-sdk-1763772836608.png
+
+metadata:
+{
+  "images": [
+    {
+      "width": 1024,
+      "height": 576,
+      "contentType": "image/jpeg",
+      "nsfw": false
+    }
+  ]
+}
+```
+
+✅ benefits:
+- clean typed api
+- auto image save + open
+- aspect ratio support
+- consistent with other ai-sdk providers
+
+### approach 2: lib/fal.ts (fal client direct)
+
+```bash
+$ bun run lib/fal.ts generate_image "ancient temple ruins"
+
+[fal] generating image with fal-ai/flux-pro/v1.1
+[fal] prompt: ancient temple ruins at sunset
+[fal] processing...
+[fal] completed!
+
+{
+  "data": {
+    "images": [
+      {
+        "url": "https://v3b.fal.media/files/b/koala/L5LYGCHZ4aZ_CKZsmPbUe.jpg",
+        "width": 1024,
+        "height": 768,
+        "content_type": "image/jpeg"
+      }
+    ],
+    "seed": 2946158106
+  }
+}
+```
+
+✅ benefits:
+- full api access
+- queue updates
+- video support
+- custom parameters
+
+## cli tests ✅
+
+all help menus working:
+
+```bash
+bun run lib/ai-sdk/fal.ts                    # ✓
+bun run lib/fal.ts                            # ✓
+bun run lib/higgsfield.ts                     # ✓
+bun run service/image.ts                      # ✓
+bun run service/video.ts                      # ✓
+bun run utilities/s3.ts                       # ✓
+```
+
+## library imports ✅
+
+```typescript
+import { generateImage } from "./index"
+import * as aiSdkFal from "./index"
+
+// both approaches available
+```
+
+## actual generation tests ✅
+
+successfully generated and opened:
+- cyberpunk city (16:9, ai-sdk)
+- spaceship interior (16:9, ai-sdk)
+- temple ruins (4:3, fal client)
+- aurora borealis (4:3, fal client)
+
+all images ~15-20 seconds generation time
+
+## what works
+
+1. **dual fal implementations** - ai-sdk for simplicity, client for power ✓
+2. **all cli scripts executable** with proper help menus ✓
+3. **library imports functional** ✓
+4. **actual image generation working** ✓
+5. **automatic image opening** (ai-sdk version) ✓
+6. **queue progress updates** (fal client) ✓
+
+## file structure
+
+```
+lib/
+├── ai-sdk/
+│   └── fal.ts          # vercel ai sdk approach
+├── fal.ts              # fal client approach
+└── higgsfield.ts       # soul character generation
+```
+
+## recommendations
+
+- **use lib/ai-sdk/fal.ts** for standard image generation
+- **use lib/fal.ts** for video or advanced features
+- **use service/**.ts for high-level operations with s3 upload

package/action/captions/SKILL.md

@@ -0,0 +1,170 @@
+---
+name: video-captions
+description: add auto-generated or custom subtitles to videos using groq/fireworks transcription and ffmpeg overlay. use when adding captions, subtitles, or text overlays to videos for accessibility or social media.
+allowed-tools: Read, Bash
+---
+
+# video captions
+
+automatically generate and overlay subtitles on videos with customizable styling.
+
+## features
+
+- **auto-generation**: transcribe video audio using groq or fireworks
+- **custom srt support**: use existing subtitle files
+- **styling**: customize font, size, colors, position
+- **word-level timing**: fireworks provides precise word timestamps
+- **instant overlay**: ffmpeg-based subtitle rendering
+
+## usage
+
+### auto-generate captions
+```bash
+bun run service/captions.ts <videoPath> [outputPath] [options]
+```
+
+**basic example:**
+```bash
+bun run service/captions.ts media/video.mp4
+# outputs: media/video-captioned.mp4
+```
+
+**with options:**
+```bash
+bun run service/captions.ts media/video.mp4 output.mp4 --provider fireworks --font Arial --size 28
+```
+
+### use existing srt file
+```bash
+bun run service/captions.ts media/video.mp4 output.mp4 --srt media/video.srt
+```
+
+## options
+
+- `--srt <path>` - use existing srt file instead of auto-generating
+- `--provider <name>` - groq or fireworks (default: fireworks)
+- `--font <name>` - font name (default: Arial)
+- `--size <number>` - font size (default: 24)
+- `--color <hex>` - primary color in ASS format (default: &HFFFFFF white)
+- `--outline <hex>` - outline color in ASS format (default: &H000000 black)
+
+## as library
+
+```typescript
+import { addCaptions } from "./service/captions"
+
+const result = await addCaptions({
+  videoPath: "media/video.mp4",
+  output: "captioned.mp4",
+  provider: "fireworks", // or "groq"
+  style: {
+    fontName: "Helvetica",
+    fontSize: 28,
+    primaryColor: "&HFFFFFF",
+    outlineColor: "&H000000",
+    bold: true,
+    alignment: 2, // bottom center
+    marginV: 20
+  }
+})
+```
+
+## providers
+
+### fireworks (recommended)
+- **word-level timestamps** for precise timing
+- generates `.srt` format with detailed timing
+- better for social media content
+- slightly slower transcription
+
+### groq
+- **ultra-fast** transcription
+- plain text output (converted to srt)
+- sentence-level timing
+- great for quick previews
+
+## styling options
+
+```typescript
+interface SubtitleStyle {
+  fontName?: string      // default: Arial
+  fontSize?: number      // default: 24
+  primaryColor?: string  // default: &HFFFFFF (white)
+  outlineColor?: string  // default: &H000000 (black)
+  bold?: boolean         // default: true
+  alignment?: number     // 1-9, default: 2 (bottom center)
+  marginV?: number       // vertical margin, default: 20
+}
+```
+
+**alignment values:**
+```
+1 = bottom left    2 = bottom center    3 = bottom right
+4 = middle left    5 = middle center    6 = middle right
+7 = top left       8 = top center       9 = top right
+```
+
+## when to use
+
+use this skill when:
+- preparing videos for social media (tiktok, instagram, youtube)
+- adding accessibility features
+- creating educational or tutorial content
+- need word-level caption timing
+- translating videos with custom srt files
+
+## typical workflow
+
+1. create or edit video
+2. add captions with auto-transcription (this service)
+3. customize style for platform
+4. prepare for social media (edit service)
+
+## examples
+
+**tiktok/instagram style captions:**
+```bash
+bun run service/captions.ts video.mp4 captioned.mp4 \
+  --provider fireworks \
+  --font "Arial Black" \
+  --size 32 \
+  --color "&H00FFFF"
+```
+
+**professional style:**
+```bash
+bun run service/captions.ts video.mp4 output.mp4 \
+  --provider fireworks \
+  --font "Helvetica" \
+  --size 24
+```
+
+**with existing subtitles:**
+```bash
+bun run service/captions.ts video.mp4 final.mp4 \
+  --srt custom-subtitles.srt \
+  --font "Arial" \
+  --size 26
+```
+
+## output
+
+- generates `.srt` file if auto-transcribing
+- creates new video file with burned-in subtitles
+- preserves original video quality
+- audio is copied without re-encoding
+
+## environment variables
+
+required (for auto-transcription):
+- `GROQ_API_KEY` - for groq provider
+- `FIREWORKS_API_KEY` - for fireworks provider
+
+**system requirements:**
+- ffmpeg must be installed
+
+## processing time
+
+- transcription: 5-30 seconds (depending on video length)
+- overlay: 5-15 seconds (depending on video length)
+- total: typically under 1 minute

package/action/captions/index.ts

@@ -0,0 +1,169 @@
+#!/usr/bin/env bun
+
+/**
+ * video captioning service
+ * generates and overlays subtitles on videos using ffmpeg
+ * supports auto-generation via groq/fireworks or custom srt files
+ */
+
+import { existsSync } from "node:fs";
+import ffmpeg from "fluent-ffmpeg";
+import type { ActionMeta } from "../../cli/types";
+import { transcribe } from "../transcribe";
+
+export const meta: ActionMeta = {
+  name: "captions",
+  type: "action",
+  description: "add subtitles to video",
+  inputType: "video",
+  outputType: "video",
+  schema: {
+    input: {
+      type: "object",
+      required: ["video", "output"],
+      properties: {
+        video: {
+          type: "string",
+          format: "file-path",
+          description: "input video file",
+        },
+        output: {
+          type: "string",
+          format: "file-path",
+          description: "output video path",
+        },
+        srt: {
+          type: "string",
+          format: "file-path",
+          description: "existing srt file (auto-generates if not provided)",
+        },
+        provider: {
+          type: "string",
+          enum: ["groq", "fireworks"],
+          default: "fireworks",
+          description: "transcription provider for auto-generation",
+        },
+      },
+    },
+    output: { type: "string", format: "file-path", description: "video path" },
+  },
+  async run(options) {
+    const { video, output, srt, provider } = options as {
+      video: string;
+      output: string;
+      srt?: string;
+      provider?: "groq" | "fireworks";
+    };
+    return addCaptions({ videoPath: video, output, srtPath: srt, provider });
+  },
+};
+
+// types
+export interface AddCaptionsOptions {
+  videoPath: string;
+  srtPath?: string; // optional existing srt file
+  output: string;
+  provider?: "groq" | "fireworks"; // only used if srtPath not provided
+  style?: SubtitleStyle;
+}
+
+export interface SubtitleStyle {
+  fontName?: string; // default: Arial
+  fontSize?: number; // default: 24
+  primaryColor?: string; // default: &HFFFFFF (white)
+  outlineColor?: string; // default: &H000000 (black)
+  bold?: boolean; // default: true
+  alignment?: number; // 1-9, default: 2 (bottom center)
+  marginV?: number; // vertical margin, default: 20
+}
+
+// default subtitle style
+const DEFAULT_STYLE: Required<SubtitleStyle> = {
+  fontName: "Arial",
+  fontSize: 24,
+  primaryColor: "&HFFFFFF", // white
+  outlineColor: "&H000000", // black
+  bold: true,
+  alignment: 2, // bottom center
+  marginV: 20,
+};
+
+// main function to add captions to video
+export async function addCaptions(
+  options: AddCaptionsOptions,
+): Promise<string> {
+  const { videoPath, srtPath, output, provider = "fireworks", style } = options;
+
+  if (!videoPath) {
+    throw new Error("videoPath is required");
+  }
+  if (!output) {
+    throw new Error("output is required");
+  }
+  if (!existsSync(videoPath)) {
+    throw new Error(`video file not found: ${videoPath}`);
+  }
+
+  console.log("[captions] adding captions to video...");
+
+  // determine srt file path
+  let finalSrtPath = srtPath;
+
+  // if no srt file provided, auto-generate it
+  if (!finalSrtPath) {
+    console.log(
+      `[captions] no srt file provided, auto-generating with ${provider}...`,
+    );
+
+    // generate srt file from video audio
+    const tempSrtPath = videoPath.replace(/\.[^.]+$/, ".srt");
+
+    const result = await transcribe({
+      audioUrl: videoPath,
+      provider,
+      outputFormat: "srt",
+      outputPath: tempSrtPath,
+    });
+
+    if (!result.success) {
+      throw new Error(`failed to generate subtitles: ${result.error}`);
+    }
+
+    finalSrtPath = tempSrtPath;
+    console.log(`[captions] generated subtitles at ${finalSrtPath}`);
+  }
+
+  if (!existsSync(finalSrtPath)) {
+    throw new Error(`srt file not found: ${finalSrtPath}`);
+  }
+
+  // merge style with defaults
+  const finalStyle = { ...DEFAULT_STYLE, ...style };
+
+  // build subtitle filter with style
+  const subtitlesFilter = `subtitles=${finalSrtPath}:force_style='FontName=${finalStyle.fontName},FontSize=${finalStyle.fontSize},PrimaryColour=${finalStyle.primaryColor},OutlineColour=${finalStyle.outlineColor},Bold=${finalStyle.bold ? -1 : 0},Alignment=${finalStyle.alignment},MarginV=${finalStyle.marginV}'`;
+
+  console.log("[captions] overlaying subtitles on video...");
+
+  return new Promise((resolve, reject) => {
+    ffmpeg(videoPath)
+      .videoFilters(subtitlesFilter)
+      .outputOptions(["-c:a", "copy"]) // copy audio without re-encoding
+      .output(output)
+      .on("end", () => {
+        console.log(`[captions] saved to ${output}`);
+        resolve(output);
+      })
+      .on("error", (err) => {
+        console.error("[captions] error:", err);
+        reject(err);
+      })
+      .run();
+  });
+}
+
+// cli
+if (import.meta.main) {
+  const { runCli } = await import("../../cli/runner");
+  runCli(meta);
+}