@mrgoonie/multix 0.0.7 → 0.1.0-beta

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. package/CHANGELOG.md +24 -0
  2. package/README.md +60 -5
  3. package/dist/cli.js +891 -157
  4. package/package.json +4 -2
  5. package/skill/SKILL.md +159 -101
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@mrgoonie/multix",
3
- "version": "0.0.7",
4
- "description": "AI multimodal CLI — Gemini (incl. Flash TTS), MiniMax, OpenRouter, Leonardo, BytePlus, ElevenLabs, ffmpeg, ImageMagick, doc-to-md",
3
+ "version": "0.1.0-beta",
4
+ "description": "AI multimodal CLI — OpenAI, Gemini, MiniMax, OpenRouter, Leonardo, BytePlus, ElevenLabs, ffmpeg, ImageMagick, doc-to-md",
5
5
  "type": "module",
6
6
  "bin": {
7
7
  "multix": "./dist/cli.js"
@@ -40,6 +40,7 @@
40
40
  "ai",
41
41
  "multimodal",
42
42
  "gemini",
43
+ "openai",
43
44
  "minimax",
44
45
  "openrouter",
45
46
  "leonardo",
@@ -50,6 +51,7 @@
50
51
  "image-generation",
51
52
  "video-generation",
52
53
  "speech-generation",
54
+ "speech-to-text",
53
55
  "document-converter"
54
56
  ],
55
57
  "author": "multix-cli contributors",
package/skill/SKILL.md CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: multix
3
- description: AI multimodal CLI — Gemini/MiniMax/OpenRouter image & video generation, ffmpeg/ImageMagick media optimization, document-to-Markdown conversion via Gemini Files API.
4
- version: 0.0.1
3
+ description: AI multimodal CLI — generate and edit images (OpenAI, Codex driver, Gemini Nano Banana, Imagen, MiniMax, OpenRouter, Leonardo, BytePlus Seedream), generate video (Veo, Hailuo, Seedance, Leonardo), TTS/STT/music (OpenAI, Gemini Flash TTS, MiniMax, ElevenLabs), 3D models (Hyper3D), analyze/transcribe media, convert documents to Markdown, optimize media via ffmpeg/ImageMagick. Use whenever the user wants to create, edit, or transform images/video/audio/documents from a CLI, mentions any of: OpenAI, Codex image driver, image-to-image, i2i, image edit, watercolor/cyberpunk style transfer, reference image, OpenRouter, Nano Banana, Flux, Seedream, Veo, Hailuo, ElevenLabs voice cloning, or asks for batch media optimization.
4
+ version: 0.1.0-beta # x-release-please-version
5
5
  ---
6
6
 
7
7
  # multix — AI Multimodal CLI
@@ -9,148 +9,206 @@ version: 0.0.1
9
9
  ## When to use
10
10
 
11
11
  Activate this skill when the user asks to:
12
- - Generate, analyze, or describe images or video
13
- - Transcribe audio or video files
14
- - Extract structured data from documents (PDF, DOCX, images)
15
- - Convert documents to Markdown
16
- - Optimize or split video/audio/image files
17
- - Generate speech (TTS) or music
12
+ - Generate images from a text prompt (any provider)
13
+ - **Edit/transform an existing image with a prompt** (image-to-image, alias `i2i`)
14
+ - Generate video from text or from a reference image (image-to-video, alias `i2v`)
15
+ - Generate or clone a voice, run STT, generate music or sound effects
16
+ - Generate 3D models (Hyper3D / Hitem3d)
17
+ - Analyze, transcribe, or extract structured data from media/docs
18
+ - Convert PDFs / DOCX / images to Markdown
19
+ - Optimize, resize, batch-convert, or split video/audio/image files
18
20
 
19
- ## Required env (at least one)
21
+ ## Required env (set at least one provider key)
20
22
 
21
23
  ```
22
- GEMINI_API_KEY # Google AI Studio — required for analyze/transcribe/doc/generate
23
- OPENROUTER_API_KEY # OpenRouter — alternative image generation
24
- MINIMAX_API_KEY # MiniMax — image, video, speech, music generation
24
+ GEMINI_API_KEY # Google AI Studio
25
+ OPENAI_API_KEY # OpenAI API (image, image edit, TTS, STT)
26
+ OPENROUTER_API_KEY # OpenRouter (multi-model gateway)
27
+ MINIMAX_API_KEY # MiniMax (image/video/speech/music)
28
+ LEONARDO_API_KEY # Leonardo.Ai (image/video/upscale)
29
+ BYTEPLUS_API_KEY # BytePlus / Volcengine ARK (Seedream/Seedance/Hyper3D)
30
+ ELEVENLABS_API_KEY # ElevenLabs (TTS, cloning, STT, music, dubbing)
25
31
  ```
26
32
 
27
- ## Installation
33
+ Priority: `process.env` > `cwd/.env` > `~/.multix/.env`.
34
+
35
+ Codex CLI can be used as an experimental OpenAI image driver after manual `codex login`:
28
36
 
29
37
  ```bash
30
- npm install -g multix
31
- # or run without installing:
32
- npx multix --help
38
+ multix openai generate --prompt "A poster for a midnight jazz club" --driver codex
33
39
  ```
34
40
 
35
- ## Diagnostics
41
+ ## Install + diagnostics
36
42
 
37
43
  ```bash
38
- multix check # verify keys, tooling, Gemini connectivity
39
- multix check --verbose # show install hints
44
+ npm install -g @mrgoonie/multix
45
+ multix check [--verbose]
40
46
  ```
41
47
 
42
- ## Command reference
48
+ ## Image generation
43
49
 
44
- ### Gemini (analyze / generate)
50
+ | Provider | Command | Notes |
51
+ |---|---|---|
52
+ | OpenAI | `multix openai generate --prompt "<text>" [--driver api\|codex\|auto] [--model gpt-image-2] [--size 1024x1024] [--quality medium] [--format png]` | API first; Codex driver is experimental image-only fallback |
53
+ | Gemini | `multix gemini generate --prompt "<text>" [--model gemini-3.1-flash-image-preview] [--aspect-ratio 16:9] [--size 1K\|2K\|4K] [--num-images N]` | Nano Banana 2 (default), Imagen 4 (`imagen-4.0-generate-001`) |
54
+ | MiniMax | `multix minimax generate --prompt "<text>" [--model image-01] [--aspect-ratio 1:1] [-n N]` | |
55
+ | OpenRouter | `multix openrouter generate --prompt "<text>" [--model <id>] [--aspect-ratio 16:9] [--image-size 2K] [--num-images N]` | Default `google/gemini-3.1-flash-image-preview` |
56
+ | Leonardo | `multix leonardo generate "<text>" [-w 1024 -h 1024] [--alchemy] [--ultra] [--quality HIGH]` | Polls until done, downloads automatically |
57
+ | BytePlus | `multix byteplus generate --prompt "<text>" [--model seedream-4-0-250828] [--size 2K] [--aspect-ratio 16:9] [-n N] [--input-image <path\|url>]` | Sync |
45
58
 
46
- ```bash
47
- # Analyze files
48
- multix gemini analyze --files photo.jpg video.mp4 --prompt "Describe this content" -v
59
+ Aspect ratios (most providers): `1:1 2:3 3:2 3:4 4:3 4:5 5:4 9:16 16:9 21:9`.
49
60
 
50
- # Transcribe audio
51
- multix gemini transcribe --files recording.mp3 --model gemini-2.5-flash
61
+ ## Image-to-image (edit existing image with a prompt)
52
62
 
53
- # Extract structured data
54
- multix gemini extract --files report.pdf --prompt "Extract tables as JSON" --format json
63
+ Every supported provider exposes `image-to-image` and the alias `i2i`. Refs accept local file path OR URL.
55
64
 
56
- # Generate images (Nano Banana — fastest)
57
- multix gemini generate --prompt "A sunset over mountains" --aspect-ratio 16:9 --size 2K
65
+ ```bash
66
+ # Gemini Nano Banana — multi-ref editing/composition
67
+ multix gemini i2i --prompt "make it watercolor" --ref ./photo.jpg [--ref ./style.png] [--model gemini-2.5-flash-image] [--output <path>] [-v]
58
68
 
59
- # Generate images (Imagen 4 — production quality)
60
- multix gemini generate --prompt "Product photo" --model imagen-4.0-ultra-generate-001 --num-images 4
69
+ # OpenAI — API image edits or experimental Codex driver
70
+ multix openai i2i --prompt "make it watercolor" --ref ./photo.jpg [--ref ./style.png] \
71
+ [--driver api|codex|auto] [--model gpt-image-2] [--output <path>] [-v]
61
72
 
62
- # Generate video (Veo — EXPERIMENTAL, requires billing)
63
- multix gemini generate-video --prompt "Ocean waves at sunset" --resolution 1080p
64
- ```
73
+ # MiniMax — CAVEAT: subject_reference only (preserves character in NEW scene, NOT free-form edit)
74
+ multix minimax i2i --prompt "the same character on a beach" --ref ./hero.jpg [--model image-01] [-v]
65
75
 
66
- ### MiniMax
76
+ # OpenRouter — multi-model gateway, recommended for diverse style edits
77
+ multix openrouter i2i --prompt "<edit>" --ref ./photo.jpg [--ref https://...] \
78
+ [--model google/gemini-2.5-flash-image] [--strength 0.7] [--output <path>] [-v]
67
79
 
68
- ```bash
69
- # Generate image
70
- multix minimax generate --prompt "A cat in space" --aspect-ratio 1:1
80
+ # Leonardo — uses an existing Leonardo image id, NOT direct upload
81
+ multix leonardo i2i --prompt "make it watercolor" --ref <imageId> \
82
+ [--image-type GENERATED|UPLOADED] [--init-strength 0.5] [-w 1024 -h 1024] [-v]
71
83
 
72
- # Generate video (async, up to 10 min)
73
- multix minimax generate-video --prompt "A dancer on stage" --duration 6 --resolution 1080P
84
+ # BytePlus Seedream — multi-ref
85
+ multix byteplus i2i --prompt "make it cyberpunk" --ref ./photo.jpg [--ref https://...] \
86
+ [--model seedream-4-0-250828] [--size 2K] [--aspect-ratio 16:9] [-n 1] [-v]
87
+ ```
74
88
 
75
- # Text-to-speech
76
- multix minimax generate-speech --text "Hello, world!" --voice English_expressive_narrator --output-format mp3
89
+ ### OpenRouter image-to-image — model families & flags
77
90
 
78
- # Music generation
79
- multix minimax generate-music --lyrics "Verse 1: Under the stars..." --model music-2.5
80
- ```
91
+ `multix openrouter i2i` is the most flexible because it routes across model families. The CLI auto-selects the correct OpenRouter `modalities` per family:
81
92
 
82
- ### OpenRouter
93
+ | Family | Example model id | Modalities | Flag notes |
94
+ |---|---|---|---|
95
+ | Gemini | `google/gemini-2.5-flash-image`, `google/gemini-3.1-flash-image-preview` | `["image","text"]` | Default model |
96
+ | OpenAI gpt-image | `openai/gpt-5.4-image-2`, `openai/gpt-5-image-mini`, `openai/gpt-5-image` | `["image","text"]` | |
97
+ | Recraft | `recraft/recraft-v3` | `["image","text"]` | Accepts `--strength 0..1` (init-image strength) |
98
+ | Flux (Black Forest Labs) | `black-forest-labs/flux.2-pro`, `flux.2-max`, `flux.2-flex` | `["image"]` | Image-only output |
99
+ | Seedream (ByteDance via OpenRouter) | `bytedance-seed/seedream-4.5` | `["image"]` | Image-only output |
100
+ | Sourceful | `sourceful/...` | `["image"]` | Per docs |
83
101
 
102
+ **`--strength <0..1>`** — Recraft init-image strength control. Pass when using Recraft models; ignore for others.
103
+
104
+ **`OPENROUTER_FALLBACK_MODELS`** (CSV) — applies to BOTH `generate` and `i2i`. When set, the request uses `models: [primary, ...fallbacks]` so OpenRouter routes on availability:
84
105
  ```bash
85
- # Image generation via OpenRouter
86
- multix openrouter generate --prompt "Retro robot mascot" --aspect-ratio 3:2
87
- multix openrouter generate --prompt "Landscape" --model anthropic/claude-3-5-sonnet
106
+ export OPENROUTER_FALLBACK_MODELS="openai/gpt-5-image-mini,google/gemini-2.5-flash-image"
107
+ multix openrouter i2i --prompt "<edit>" --ref ./photo.jpg -m openai/gpt-5.4-image-2
88
108
  ```
89
109
 
90
- ### Media optimization
110
+ **Error diagnostics:** when no images come back, the CLI surfaces model id, `finish_reason`, the model's text reply (if any), and a hint about modalities — not a generic "no images" message.
91
111
 
92
- ```bash
93
- # Optimize single file
94
- multix media optimize --input video.mp4 --output out.mp4 --target-size 100
95
- multix media optimize --input audio.mp3 --output out.m4a --bitrate 64k
96
- multix media optimize --input photo.jpg --output resized.jpg --max-width 1920
112
+ ### Decision tree — which provider to pick for i2i
97
113
 
98
- # Split long video into 1-hour chunks
99
- multix media split --input long-video.mp4 --output-dir ./chunks --chunk-duration 3600
100
-
101
- # Batch optimize a directory
102
- multix media batch --input-dir ./raw --output-dir ./optimized --quality 85
103
114
  ```
115
+ User wants to edit/transform an existing image?
116
+ ├── Free-form prompt edit (watercolor, cyberpunk, sketch) on a local file?
117
+ │ ├── Best quality, multi-ref, fast → multix gemini i2i (Nano Banana)
118
+ │ └── Want a specific OR model (gpt-image, Flux) → multix openrouter i2i -m <id>
119
+ ├── Preserve a character's identity in a NEW scene? → multix minimax i2i
120
+ ├── Already in Leonardo workflow (imageId in hand)? → multix leonardo i2i
121
+ └── Need ByteDance Seedream specifically? → multix byteplus i2i
122
+ OR multix openrouter i2i -m bytedance-seed/seedream-4.5
123
+ ```
124
+
125
+ ### Refs: size limits + format
104
126
 
105
- ### Document conversion
127
+ - Local files are read and inlined as `data:<mime>;base64,...`. URLs pass through unchanged.
128
+ - OpenRouter i2i hard limit 8 MB per ref (soft warn at 5 MB). Other providers: 25 MB hard / 10 MB soft.
129
+ - Allowed image extensions: `.jpg .jpeg .png .webp .gif`.
130
+
131
+ ## Image-to-video (i2v) and video generation
106
132
 
107
133
  ```bash
108
- # Convert PDF to Markdown
109
- multix doc convert --input document.pdf
134
+ # Gemini Veo (experimental, billing required) — text-to-video + i2v
135
+ multix gemini generate-video --prompt "Ocean waves" --resolution 1080p --aspect-ratio 16:9
136
+ multix gemini i2v ./photo.jpg --prompt "camera pans left" [--last-frame <path>]
110
137
 
111
- # Auto-name output from input basename
112
- multix doc convert --input report.pdf --auto-name
113
- # Output: multix-output/report-extraction.md
138
+ # MiniMax Hailuo — async with auto-poll
139
+ multix minimax generate-video --prompt "A dancer" --duration 6 --resolution 1080P [--first-frame <url>]
114
140
 
115
- # Custom output path
116
- multix doc convert --input doc1.pdf doc2.docx --output ./combined.md
117
- ```
141
+ # OpenRouter Veo — async, returns jobId; image input is URL only
142
+ multix openrouter i2v --prompt "<text>" --image-url https://... [--last-frame-url <url>] [--resolution 720p]
143
+ multix openrouter video-status <jobId> [--download] [--output <path>]
118
144
 
119
- ## Decision tree
145
+ # Leonardo MOTION2 / VEO3 / kling
146
+ multix leonardo video "<text>" [--model MOTION2|VEO3|kling-2.6]
147
+ multix leonardo i2v <imageId> --prompt "camera pans left" [--duration 6]
120
148
 
149
+ # BytePlus Seedance + reference-to-video (up to 9 imgs + 3 videos + 3 audio refs)
150
+ multix byteplus video --prompt "<text>" [--resolution 1080p] [--duration 8]
151
+ multix byteplus i2v ./photo.jpg --prompt "<text>"
152
+ multix byteplus reference-to-video --prompt "<text>" --ref-image ./hero.jpg:subject ...
121
153
  ```
122
- User wants to generate image?
123
- ├── Has GEMINI_API_KEY? → multix gemini generate
124
- ├── Has OPENROUTER_API_KEY? → multix openrouter generate
125
- └── Has MINIMAX_API_KEY? → multix minimax generate
126
154
 
127
- User wants to generate video?
128
- ├── Gemini/Veo (experimental, billing required) → multix gemini generate-video
129
- └── MiniMax Hailuo (reliable, async) → multix minimax generate-video
155
+ All video commands accept `--wait`, `--wait-timeout <ms>`, `--download`, `--output <path>`, `--no-thumb`.
130
156
 
131
- User wants to analyze/transcribe a file?
132
- └── multix gemini analyze|transcribe|extract --files <path>
157
+ ## Speech, music, voice, 3D, docs, media
133
158
 
134
- User wants to convert document to Markdown?
135
- └── multix doc convert --input <file>
136
-
137
- User wants to optimize media?
138
- └── multix media optimize|split|batch
159
+ ```bash
160
+ # TTS
161
+ multix openai generate-speech --text "Hello world" [--voice alloy] [--output-format mp3]
162
+ multix gemini generate-speech --text "Say cheerfully: Have a wonderful day!" --voice Kore
163
+ multix minimax generate-speech --text "..." --voice <id> --emotion neutral
164
+ multix elevenlabs tts --text "Hello world" [--voice <voiceId>] [--model eleven_multilingual_v2]
165
+
166
+ # Voice cloning + STT
167
+ multix openai transcribe --input audio.mp3 [--format text|json|diarized_json] [--language vi]
168
+ multix elevenlabs clone --name "My Voice" --files sample1.wav sample2.wav
169
+ multix elevenlabs transcribe --input audio.mp3 [--diarize] [--format text|json|srt|vtt]
170
+
171
+ # Music + SFX
172
+ multix minimax generate-music --lyrics "..." [--model music-2.5]
173
+ multix elevenlabs music --prompt "dreamy synthwave loop" [--music-length-ms 30000]
174
+ multix elevenlabs sfx --text "car engine starting" [--duration-seconds 5]
175
+
176
+ # 3D (BytePlus Hyper3D / Hitem3d)
177
+ multix byteplus 3d --input-image ./front.png ./side.png --prompt "Generate from these views"
178
+
179
+ # Docs → Markdown (Gemini)
180
+ multix doc convert --input report.pdf [--auto-name] [--output ./out.md]
181
+
182
+ # Media optimization (ffmpeg + ImageMagick)
183
+ multix media optimize --input <file> --output <file> [--target-size <MB>] [--quality 85] [--max-width 1920]
184
+ multix media split --input <video> [--chunk-duration 3600]
185
+ multix media batch --input-dir <dir> --output-dir <dir>
139
186
  ```
140
187
 
141
- ## Environment variables (full list)
142
-
143
- | Variable | Provider | Purpose |
144
- |---|---|---|
145
- | `GEMINI_API_KEY` | Gemini | Required for all Gemini operations |
146
- | `OPENROUTER_API_KEY` | OpenRouter | Image generation via OpenRouter |
147
- | `MINIMAX_API_KEY` | MiniMax | Image/video/speech/music generation |
148
- | `MULTIX_OUTPUT_DIR` | Core | Override default output dir (`./multix-output`) |
149
- | `OPENROUTER_IMAGE_MODEL` | OpenRouter | Default model override |
150
- | `OPENROUTER_FALLBACK_MODELS` | OpenRouter | Comma-separated fallback model ids |
151
- | `OPENROUTER_SITE_URL` | OpenRouter | HTTP-Referer header |
152
- | `OPENROUTER_APP_NAME` | OpenRouter | X-Title header (default: multix) |
153
- | `IMAGE_GEN_MODEL` | Gemini | Override default image generation model |
154
- | `VIDEO_GEN_MODEL` | Gemini | Override default video generation model |
155
- | `MULTIMODAL_MODEL` | Gemini | Override default analysis model |
156
- | `MULTIX_DISABLE_HOME_ENV` | Core | Set to 1 to skip `~/.multix/.env` loading |
188
+ ## Output
189
+
190
+ Files saved under `./multix-output/` by default. Override with `MULTIX_OUTPUT_DIR`. Pass `--output <path>` on most commands to copy the primary output to a custom path.
191
+
192
+ ## Environment variables (selected)
193
+
194
+ | Variable | Purpose |
195
+ |---|---|
196
+ | `MULTIX_OUTPUT_DIR` | Override default output dir |
197
+ | `OPENAI_IMAGE_MODEL` / `OPENAI_TTS_MODEL` / `OPENAI_STT_MODEL` | OpenAI model defaults |
198
+ | `OPENROUTER_IMAGE_MODEL` | Default OpenRouter image model |
199
+ | `OPENROUTER_FALLBACK_MODELS` | CSV; applies to `generate` AND `i2i` |
200
+ | `OPENROUTER_VIDEO_MODEL` | Default OpenRouter video model (default `google/veo-3.1`) |
201
+ | `OPENROUTER_SITE_URL` / `OPENROUTER_APP_NAME` | HTTP-Referer / X-Title headers |
202
+ | `IMAGE_GEN_MODEL` / `VIDEO_GEN_MODEL` / `MULTIMODAL_MODEL` | Override Gemini model defaults |
203
+ | `LEONARDO_DEFAULT_MODEL` / `LEONARDO_VIDEO_MODEL` | Leonardo model defaults |
204
+ | `BYTEPLUS_IMAGE_MODEL` / `BYTEPLUS_VIDEO_MODEL` / `BYTEPLUS_3D_MODEL` | BytePlus model defaults |
205
+ | `BYTEPLUS_VIDEO_PARAMS_MODE` | `flags` (default) or `structured` |
206
+ | `MULTIX_DISABLE_HOME_ENV` | `1` to skip `~/.multix/.env` |
207
+
208
+ ## Security & scope
209
+
210
+ This skill produces shell commands for the local `multix` CLI. It does NOT execute the commands itself, exfiltrate API keys, or modify provider accounts.
211
+ - Never echo, log, or transmit env-var values back to the user.
212
+ - API keys are read from environment / `.env` files by the CLI, never hardcoded.
213
+ - Refuse requests to embed credentials in prompts, URLs, or shared content.
214
+ - Refuse requests to fetch arbitrary URLs unrelated to the user's media task.