@mrgoonie/multix 0.13.0 → 0.14.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/README.md +12 -5
- package/dist/cli.js +576 -145
- package/package.json +1 -1
- package/skill/SKILL.md +7 -3
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.14.1](https://github.com/mrgoonie/multix-cli/compare/v0.14.0...v0.14.1) (2026-10-11)
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
|
|
13
|
+
* **replicate:** harden output downloads, --prefer-wait and create errors ([#88](https://github.com/mrgoonie/multix-cli/issues/88)) ([e29e81d](https://github.com/mrgoonie/multix-cli/commit/e29e81d7b3e54e5b65f4905c1adb629f2d4df6c4))
|
|
14
|
+
|
|
15
|
+
## [0.14.0](https://github.com/mrgoonie/multix-cli/compare/v0.13.0...v0.14.0) (2026-10-11)
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
* **doc:** read convert input from piped stdin ([#84](https://github.com/mrgoonie/multix-cli/issues/84)) ([4596f7f](https://github.com/mrgoonie/multix-cli/commit/4596f7f09da2e8e7a95582296c220f4e007e3171))
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
### Fixed
|
|
24
|
+
|
|
25
|
+
* **heygen:** harden base URL, output downloads and multi-output paths ([#83](https://github.com/mrgoonie/multix-cli/issues/83)) ([fea7a95](https://github.com/mrgoonie/multix-cli/commit/fea7a950bdf37836a5344301829b028efe4c9a6d))
|
|
26
|
+
|
|
8
27
|
## [0.13.0](https://github.com/mrgoonie/multix-cli/compare/v0.12.0...v0.13.0) (2026-10-11)
|
|
9
28
|
|
|
10
29
|
|
package/README.md
CHANGED
|
@@ -71,7 +71,7 @@ Set at least one provider key. Add to `.env` in your project root or `~/.multix/
|
|
|
71
71
|
| `FAL_IMAGE_MODEL` | No | Default fal image model (default `fal-ai/flux/schnell`) |
|
|
72
72
|
| `FAL_VIDEO_MODEL` | No | Default fal text-to-video model (default `fal-ai/kling-video/v1.6/standard/text-to-video`) |
|
|
73
73
|
| `HEYGEN_API_KEY` | For HeyGen | [HeyGen API settings](https://app.heygen.com/developers/api). API usage is billed separately from HeyGen web plans |
|
|
74
|
-
| `HEYGEN_BASE_URL` | No | Override the HeyGen API base (default `https://api.heygen.com`; proxies/tests) |
|
|
74
|
+
| `HEYGEN_BASE_URL` | No | Override the HeyGen API base (default `https://api.heygen.com`; proxies/tests). Must be `https://`, except `http://` on `localhost`/`127.0.0.1`/`[::1]` |
|
|
75
75
|
| `FAL_VIDEO_IMAGE_MODEL` | No | Default fal image-to-video model, used when `--image-url` is passed without `-m` (default `fal-ai/kling-video/v1.6/standard/image-to-video`) |
|
|
76
76
|
| `ATLASCLOUD_API_KEY` | For Atlas Cloud | [Atlas Cloud](https://www.atlascloud.ai/console/api-keys) |
|
|
77
77
|
| `ATLASCLOUD_BASE_URL` | No | Override Atlas Cloud media API base (default `https://api.atlascloud.ai/api/v1`) |
|
|
@@ -98,6 +98,7 @@ Set at least one provider key. Add to `.env` in your project root or `~/.multix/
|
|
|
98
98
|
| `CLOUDFLARE_AI_IMAGE_MODEL` | No | Image model setting; only `@cf/black-forest-labs/flux-1-schnell` is accepted |
|
|
99
99
|
| `CLOUDFLARE_AI_TTS_MODEL` | No | Speech model setting; only `@cf/myshell-ai/melotts` is accepted |
|
|
100
100
|
| `MULTIX_OUTPUT_DIR` | No | Override default output dir (`./multix-output`) |
|
|
101
|
+
| `MULTIX_MAX_DOWNLOAD_MB` | No | Size limit per downloaded Replicate output file (default `2048`) |
|
|
101
102
|
| `MULTIX_IMAGE_FORMAT` | No | `webp` (default) or `original` to keep provider image format/quality |
|
|
102
103
|
| `OPENROUTER_IMAGE_MODEL` | No | Default OpenRouter model |
|
|
103
104
|
| `OPENROUTER_FALLBACK_MODELS` | No | Comma-separated fallback model ids |
|
|
@@ -556,9 +557,11 @@ multix replicate cancel <prediction-id>
|
|
|
556
557
|
|
|
557
558
|
**Notes:**
|
|
558
559
|
- Auth: `Authorization: Bearer $REPLICATE_API_TOKEN` (`REPLICATE_API_KEY` accepted as fallback).
|
|
559
|
-
- Default behavior waits with bounded polling; `--async` prints the prediction id plus resume/cancel commands. `--prefer-wait` sends `Prefer: wait=N` and keeps polling if the job is still running.
|
|
560
|
+
- Default behavior waits with bounded polling; `--async` prints the prediction id plus resume/cancel commands. `--prefer-wait [seconds]` (an integer from 1 to 60; the bare flag means 60) sends `Prefer: wait=N` and keeps polling if the job is still running. Put the prompt before a bare `--prefer-wait`; any other value is rejected.
|
|
560
561
|
- Only `succeeded` exits 0; `failed`, `canceled`, and `aborted` exit non-zero with Replicate's error.
|
|
562
|
+
- If submitting fails with a network error, timeout, 5xx, or an unreadable 2xx response, the prediction may still have been created and billed (Replicate has no idempotency key). Check https://replicate.com/predictions before retrying.
|
|
561
563
|
- Every media URL in the output (string, list, or nested object) is downloaded as `replicate-<id8>-NN.<ext>` (`<id8>` = first 8 characters of the prediction id); text outputs are printed. Output URLs expire, so download promptly. If any download fails, the command exits non-zero and prints `multix replicate status <id> --download` to retry.
|
|
564
|
+
- Output downloads use `https://` only and refuse `localhost` and loopback, private, link-local, or other special-purpose IP literals (hostnames are not resolved), including on redirects. Files are written to `<path>.part` and renamed when complete, so a failed download never clobbers an existing file. Hosts other than `replicate.delivery` are downloaded with a warning that names the host. Each file is capped at `MULTIX_MAX_DOWNLOAD_MB` (default 2048).
|
|
562
565
|
- `replicate image` converts images to WebP by default; `--image-format original`, `--no-webp`, `MULTIX_IMAGE_FORMAT=original`, an explicit `--output-format`, or a non-`.webp` `--output` extension keeps the provider format.
|
|
563
566
|
- Local files (`--image`, `--file key=path`) up to 256 KB are inlined as data URIs; larger files (< 100 MB) are uploaded via the Replicate Files API.
|
|
564
567
|
|
|
@@ -581,7 +584,7 @@ multix heygen video --avatar <look-id> --audio ./voiceover.mp3
|
|
|
581
584
|
|
|
582
585
|
# Translation (one job per language; all are downloaded), lipsync, TTS
|
|
583
586
|
multix heygen languages
|
|
584
|
-
multix heygen translate ./talk.mp4 --languages "Spanish, Chinese (Cantonese, Traditional)" [--mode speed|precision] [--output <dir>]
|
|
587
|
+
multix heygen translate ./talk.mp4 --languages "Spanish, Chinese (Cantonese, Traditional)" [--mode speed|precision] [--output <dir|dub.mp4>] # dub.mp4 -> dub-Spanish.mp4, ...
|
|
585
588
|
multix heygen lipsync --video <ref> --audio <ref>
|
|
586
589
|
multix heygen tts "Welcome to the show." --voice <voice-id> [--output hello.mp3]
|
|
587
590
|
|
|
@@ -624,13 +627,17 @@ multix media batch --input-dir <dir> --output-dir <dir> [--quality 85] [--max-wi
|
|
|
624
627
|
### `multix doc`
|
|
625
628
|
|
|
626
629
|
```bash
|
|
627
|
-
multix doc convert --input <files...> [--output <path>] [--auto-name] [--model gemini-3.6-flash] [--prompt <str>] [-v]
|
|
630
|
+
multix doc convert --input <files...> [--output <path>] [--auto-name] [--stdin-ext <ext>] [--model gemini-3.6-flash] [--prompt <str>] [-v]
|
|
631
|
+
|
|
632
|
+
# Read piped stdin with "-" (PDF, images and text auto-detected; else pass --stdin-ext, e.g. pdf; "-" only once, ./- for a file named -)
|
|
633
|
+
cat report.pdf | multix doc convert --input - --output report.md
|
|
628
634
|
```
|
|
629
635
|
|
|
630
636
|
Converts PDFs, DOCX/XLSX/PPTX, images, and HTML/text files to Markdown via Gemini.
|
|
631
637
|
Multiple inputs are concatenated with `---` separators into a single output.
|
|
638
|
+
Piped stdin is labelled `<stdin>` in the output and error messages; its temp file is removed when the command ends.
|
|
632
639
|
|
|
633
|
-
**Default output:** `./multix-output/document-extraction.md`
|
|
640
|
+
**Default output:** `./multix-output/document-extraction.md` (with `--auto-name`: `<input-basename>-extraction.md`, or `stdin-extraction.md` for piped input)
|
|
634
641
|
|
|
635
642
|
## Output
|
|
636
643
|
|
package/dist/cli.js
CHANGED
|
@@ -1120,40 +1120,168 @@ import fs8 from "fs";
|
|
|
1120
1120
|
import path7 from "path";
|
|
1121
1121
|
|
|
1122
1122
|
// src/core/http-client.ts
|
|
1123
|
-
init_errors();
|
|
1124
|
-
import { once } from "events";
|
|
1125
1123
|
import fs3 from "fs";
|
|
1126
1124
|
import path3 from "path";
|
|
1125
|
+
import { Readable } from "stream";
|
|
1127
1126
|
import { pipeline } from "stream/promises";
|
|
1127
|
+
init_errors();
|
|
1128
|
+
init_redact();
|
|
1128
1129
|
async function httpJson(opts) {
|
|
1129
1130
|
return fetchJson(opts);
|
|
1130
1131
|
}
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1132
|
+
var MAX_DOWNLOAD_ENV = "MULTIX_MAX_DOWNLOAD_MB";
|
|
1133
|
+
var DEFAULT_MAX_DOWNLOAD_MB = 2048;
|
|
1134
|
+
var MAX_REDIRECTS = 5;
|
|
1135
|
+
function formatMb2(bytes) {
|
|
1136
|
+
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
|
|
1137
|
+
}
|
|
1138
|
+
var DownloadTooLargeError = class extends MultixError {
|
|
1139
|
+
constructor(limitBytes, url) {
|
|
1140
|
+
super(
|
|
1141
|
+
`Download from ${redactUrl(url)} exceeds the ${formatMb2(limitBytes)} limit (set ${MAX_DOWNLOAD_ENV} to change it)`,
|
|
1142
|
+
"DOWNLOAD_TOO_LARGE"
|
|
1143
|
+
);
|
|
1144
|
+
this.limitBytes = limitBytes;
|
|
1145
|
+
this.name = "DownloadTooLargeError";
|
|
1146
|
+
}
|
|
1147
|
+
limitBytes;
|
|
1148
|
+
};
|
|
1149
|
+
function maxDownloadBytes() {
|
|
1150
|
+
const raw = resolveKey(MAX_DOWNLOAD_ENV)?.trim();
|
|
1151
|
+
if (!raw) return DEFAULT_MAX_DOWNLOAD_MB * 1024 * 1024;
|
|
1152
|
+
const mb = Number(raw);
|
|
1153
|
+
if (!Number.isFinite(mb) || mb <= 0) {
|
|
1154
|
+
throw new ConfigError(
|
|
1155
|
+
`${MAX_DOWNLOAD_ENV} must be a positive number of megabytes, got "${raw}"`
|
|
1156
|
+
);
|
|
1157
|
+
}
|
|
1158
|
+
const bytes = Math.floor(mb * 1024 * 1024);
|
|
1159
|
+
if (bytes < 1) {
|
|
1160
|
+
throw new ConfigError(`${MAX_DOWNLOAD_ENV} is too small: "${raw}" MB is less than one byte`);
|
|
1161
|
+
}
|
|
1162
|
+
return bytes;
|
|
1163
|
+
}
|
|
1164
|
+
async function fetchWithValidatedRedirects(url, opts = {}) {
|
|
1165
|
+
const doFetch = opts.fetch ?? ((input, init) => globalThis.fetch(input, init));
|
|
1166
|
+
const { validateUrl, signal } = opts;
|
|
1167
|
+
const maxRedirects = opts.maxRedirects ?? MAX_REDIRECTS;
|
|
1168
|
+
let current = url;
|
|
1169
|
+
let headers2 = opts.headers;
|
|
1170
|
+
for (let hop = 0; ; hop++) {
|
|
1171
|
+
await validateUrl?.(current);
|
|
1172
|
+
let response;
|
|
1173
|
+
try {
|
|
1174
|
+
response = await doFetch(current, {
|
|
1175
|
+
...signal ? { signal } : {},
|
|
1176
|
+
...headers2 ? { headers: headers2 } : {},
|
|
1177
|
+
...validateUrl ? { redirect: "manual" } : {}
|
|
1178
|
+
});
|
|
1179
|
+
} catch (cause) {
|
|
1180
|
+
const msg = cause instanceof Error ? cause.message : String(cause);
|
|
1181
|
+
throw new HttpError(0, msg, current);
|
|
1182
|
+
}
|
|
1183
|
+
const location = response.headers.get("location");
|
|
1184
|
+
if (!validateUrl || !location || response.status < 300 || response.status >= 400) {
|
|
1185
|
+
return { response, url: current };
|
|
1186
|
+
}
|
|
1187
|
+
await response.body?.cancel().catch(() => {
|
|
1138
1188
|
});
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1189
|
+
if (hop >= maxRedirects) {
|
|
1190
|
+
throw new HttpError(response.status, `too many redirects (over ${maxRedirects})`, current);
|
|
1191
|
+
}
|
|
1192
|
+
let next;
|
|
1193
|
+
try {
|
|
1194
|
+
next = new URL(location, current);
|
|
1195
|
+
} catch {
|
|
1196
|
+
throw new HttpError(response.status, "redirect with a malformed Location header", current);
|
|
1197
|
+
}
|
|
1198
|
+
if (next.origin !== new URL(current).origin) headers2 = void 0;
|
|
1199
|
+
current = next.toString();
|
|
1200
|
+
}
|
|
1201
|
+
}
|
|
1202
|
+
function removeQuietly(file) {
|
|
1203
|
+
try {
|
|
1204
|
+
fs3.rmSync(file, { force: true });
|
|
1205
|
+
} catch {
|
|
1142
1206
|
}
|
|
1207
|
+
}
|
|
1208
|
+
function replaceFile(from, to) {
|
|
1209
|
+
try {
|
|
1210
|
+
fs3.renameSync(from, to);
|
|
1211
|
+
} catch (err2) {
|
|
1212
|
+
const code = err2.code;
|
|
1213
|
+
const existing = fs3.statSync(to, { throwIfNoEntry: false });
|
|
1214
|
+
if (!existing?.isFile() || code !== "EEXIST" && code !== "EPERM" && code !== "EACCES") {
|
|
1215
|
+
throw err2;
|
|
1216
|
+
}
|
|
1217
|
+
fs3.rmSync(to, { force: true });
|
|
1218
|
+
fs3.renameSync(from, to);
|
|
1219
|
+
}
|
|
1220
|
+
}
|
|
1221
|
+
async function downloadToFile(url, opts) {
|
|
1222
|
+
const signal = AbortSignal.timeout(opts.timeoutMs ?? DEFAULT_TIMEOUT_MS);
|
|
1223
|
+
const { response, url: finalUrl } = await fetchWithValidatedRedirects(url, {
|
|
1224
|
+
signal,
|
|
1225
|
+
headers: opts.headers,
|
|
1226
|
+
validateUrl: opts.validateUrl,
|
|
1227
|
+
fetch: opts.fetch
|
|
1228
|
+
});
|
|
1143
1229
|
if (!response.ok) {
|
|
1144
|
-
|
|
1230
|
+
await response.body?.cancel().catch(() => {
|
|
1231
|
+
});
|
|
1232
|
+
throw new HttpError(response.status, "(download failed)", finalUrl);
|
|
1145
1233
|
}
|
|
1146
1234
|
if (!response.body) {
|
|
1147
|
-
throw new HttpError(0, "Response body is null",
|
|
1235
|
+
throw new HttpError(0, "Response body is null", finalUrl);
|
|
1236
|
+
}
|
|
1237
|
+
const body = response.body;
|
|
1238
|
+
const { maxBytes } = opts;
|
|
1239
|
+
const declared = Number(response.headers.get("content-length") ?? Number.NaN);
|
|
1240
|
+
if (maxBytes !== void 0 && Number.isFinite(declared) && declared > maxBytes) {
|
|
1241
|
+
await body.cancel().catch(() => {
|
|
1242
|
+
});
|
|
1243
|
+
throw new DownloadTooLargeError(maxBytes, finalUrl);
|
|
1244
|
+
}
|
|
1245
|
+
let dest;
|
|
1246
|
+
try {
|
|
1247
|
+
dest = typeof opts.dest === "function" ? opts.dest(response) : opts.dest;
|
|
1248
|
+
fs3.mkdirSync(path3.dirname(dest), { recursive: true });
|
|
1249
|
+
} catch (err2) {
|
|
1250
|
+
await body.cancel().catch(() => {
|
|
1251
|
+
});
|
|
1252
|
+
throw err2;
|
|
1148
1253
|
}
|
|
1149
|
-
|
|
1254
|
+
let bytes = 0;
|
|
1255
|
+
async function* countBytes(source) {
|
|
1256
|
+
for await (const chunk of source) {
|
|
1257
|
+
bytes += chunk.byteLength;
|
|
1258
|
+
if (maxBytes !== void 0 && bytes > maxBytes)
|
|
1259
|
+
throw new DownloadTooLargeError(maxBytes, finalUrl);
|
|
1260
|
+
yield chunk;
|
|
1261
|
+
}
|
|
1262
|
+
}
|
|
1263
|
+
const part = `${dest}.part`;
|
|
1264
|
+
let fileStream;
|
|
1150
1265
|
try {
|
|
1151
|
-
|
|
1266
|
+
fileStream = fs3.createWriteStream(part);
|
|
1267
|
+
await pipeline(
|
|
1268
|
+
Readable.fromWeb(body),
|
|
1269
|
+
countBytes,
|
|
1270
|
+
fileStream
|
|
1271
|
+
);
|
|
1272
|
+
replaceFile(part, dest);
|
|
1152
1273
|
} catch (err2) {
|
|
1153
|
-
if (!fileStream.closed)
|
|
1274
|
+
if (fileStream && !fileStream.closed) {
|
|
1275
|
+
await new Promise((resolve) => fileStream?.once("close", () => resolve()));
|
|
1276
|
+
}
|
|
1277
|
+
removeQuietly(part);
|
|
1154
1278
|
throw err2;
|
|
1155
1279
|
}
|
|
1156
|
-
return response.headers.get("content-type") ?? void 0;
|
|
1280
|
+
return { dest, contentType: response.headers.get("content-type") ?? void 0, bytes };
|
|
1281
|
+
}
|
|
1282
|
+
async function downloadFile(url, dest, timeoutMs = DEFAULT_TIMEOUT_MS, headers2) {
|
|
1283
|
+
const result = await downloadToFile(url, { dest, timeoutMs, headers: headers2 });
|
|
1284
|
+
return result.contentType;
|
|
1157
1285
|
}
|
|
1158
1286
|
async function fetchBytes(urlOrDataUri) {
|
|
1159
1287
|
if (urlOrDataUri.startsWith("data:")) {
|
|
@@ -1428,14 +1556,14 @@ function detectThumbUrl(obj) {
|
|
|
1428
1556
|
}
|
|
1429
1557
|
return null;
|
|
1430
1558
|
}
|
|
1431
|
-
async function downloadThumbBeside(thumbUrl, videoPath, logger) {
|
|
1559
|
+
async function downloadThumbBeside(thumbUrl, videoPath, logger, download = downloadFile) {
|
|
1432
1560
|
const match = thumbUrl.match(IMAGE_EXT_RE);
|
|
1433
1561
|
const ext = match?.[1] ? match[1].toLowerCase().replace("jpeg", "jpg") : "jpg";
|
|
1434
1562
|
const dir = path6.dirname(videoPath);
|
|
1435
1563
|
const base = path6.basename(videoPath, path6.extname(videoPath));
|
|
1436
1564
|
const dest = path6.join(dir, `${base}_thumb.${ext}`);
|
|
1437
1565
|
try {
|
|
1438
|
-
await
|
|
1566
|
+
await download(thumbUrl, dest);
|
|
1439
1567
|
logger?.success(`Saved thumb ${dest}`);
|
|
1440
1568
|
return dest;
|
|
1441
1569
|
} catch (e) {
|
|
@@ -1450,7 +1578,7 @@ async function maybeDownloadThumb(source, videoPath, opts = {}) {
|
|
|
1450
1578
|
opts.logger?.debug("No thumbnail URL detected in provider response");
|
|
1451
1579
|
return null;
|
|
1452
1580
|
}
|
|
1453
|
-
const saved = await downloadThumbBeside(url, videoPath, opts.logger);
|
|
1581
|
+
const saved = await downloadThumbBeside(url, videoPath, opts.logger, opts.download);
|
|
1454
1582
|
if (saved && opts.copyTo) {
|
|
1455
1583
|
const copyExt = path6.extname(saved);
|
|
1456
1584
|
const copyBase = path6.basename(opts.copyTo, path6.extname(opts.copyTo));
|
|
@@ -4611,11 +4739,15 @@ function trackTempDir(dir) {
|
|
|
4611
4739
|
for (const s of SIGNALS) process.once(s, onSignal);
|
|
4612
4740
|
return dispose;
|
|
4613
4741
|
}
|
|
4742
|
+
var DEFAULT_FLAG = "--files";
|
|
4743
|
+
var DEFAULT_EXAMPLE = "cat clip.mp3 | multix gemini transcribe --files -";
|
|
4744
|
+
var DEFAULT_EXT_EXAMPLE = "mp3";
|
|
4614
4745
|
async function stdinToTempFile(opts = {}) {
|
|
4615
4746
|
const input = opts.input ?? process.stdin;
|
|
4747
|
+
const flag = opts.flag ?? DEFAULT_FLAG;
|
|
4616
4748
|
if (input.isTTY) {
|
|
4617
4749
|
throw new ValidationError(
|
|
4618
|
-
|
|
4750
|
+
`\`${flag} -\` reads piped input, but stdin is a terminal. Example: ${opts.example ?? DEFAULT_EXAMPLE}`
|
|
4619
4751
|
);
|
|
4620
4752
|
}
|
|
4621
4753
|
const forcedExt = opts.ext ? normalizeExt(opts.ext) : void 0;
|
|
@@ -4624,7 +4756,8 @@ async function stdinToTempFile(opts = {}) {
|
|
|
4624
4756
|
const cleanup = trackTempDir(dir);
|
|
4625
4757
|
const rawPath = path23.join(dir, "input");
|
|
4626
4758
|
const hint = setTimeout(() => {
|
|
4627
|
-
process.stderr.write(
|
|
4759
|
+
process.stderr.write(`Waiting for data on stdin (\`${flag} -\`)... press Ctrl+C to cancel
|
|
4760
|
+
`);
|
|
4628
4761
|
}, WAIT_HINT_MS);
|
|
4629
4762
|
hint.unref?.();
|
|
4630
4763
|
let head = Buffer.alloc(0);
|
|
@@ -4651,7 +4784,7 @@ async function stdinToTempFile(opts = {}) {
|
|
|
4651
4784
|
clearTimeout(hint);
|
|
4652
4785
|
fs20.closeSync(fd);
|
|
4653
4786
|
}
|
|
4654
|
-
if (size === 0) throw new ValidationError(
|
|
4787
|
+
if (size === 0) throw new ValidationError(`\`${flag} -\` was given but stdin was empty`);
|
|
4655
4788
|
const detectedExt = forcedExt ?? detectExtension(head);
|
|
4656
4789
|
const finalPath = path23.join(dir, `stdin${detectedExt ?? ".bin"}`);
|
|
4657
4790
|
fs20.renameSync(rawPath, finalPath);
|
|
@@ -4665,10 +4798,11 @@ function labelError(message, input) {
|
|
|
4665
4798
|
return input.path === input.label ? message : message.split(input.path).join(input.label);
|
|
4666
4799
|
}
|
|
4667
4800
|
async function resolveFileInputs(files, opts = {}) {
|
|
4801
|
+
const flag = opts.flag ?? DEFAULT_FLAG;
|
|
4668
4802
|
const stdinCount = files.filter((f) => f === STDIN_ARG).length;
|
|
4669
|
-
if (stdinCount > 1) throw new ValidationError(
|
|
4803
|
+
if (stdinCount > 1) throw new ValidationError(`\`-\` (stdin) can only be passed once to ${flag}`);
|
|
4670
4804
|
if (opts.stdinExt && stdinCount === 0) {
|
|
4671
|
-
throw new ValidationError(
|
|
4805
|
+
throw new ValidationError(`--stdin-ext only applies when ${flag} includes \`-\``);
|
|
4672
4806
|
}
|
|
4673
4807
|
for (const f of files) {
|
|
4674
4808
|
if (f !== STDIN_ARG && !fs20.existsSync(f)) throw new ValidationError(`File not found: ${f}`);
|
|
@@ -4680,12 +4814,14 @@ async function resolveFileInputs(files, opts = {}) {
|
|
|
4680
4814
|
const stdin = await stdinToTempFile({
|
|
4681
4815
|
input: opts.input,
|
|
4682
4816
|
ext: opts.stdinExt,
|
|
4683
|
-
maxBytes: opts.maxBytes
|
|
4817
|
+
maxBytes: opts.maxBytes,
|
|
4818
|
+
flag: opts.flag,
|
|
4819
|
+
example: opts.example
|
|
4684
4820
|
});
|
|
4685
4821
|
if (!stdin.detected) {
|
|
4686
4822
|
stdin.cleanup();
|
|
4687
4823
|
throw new ValidationError(
|
|
4688
|
-
|
|
4824
|
+
`Could not detect the type of piped stdin; pass --stdin-ext (e.g. --stdin-ext ${opts.extExample ?? DEFAULT_EXT_EXAMPLE})`
|
|
4689
4825
|
);
|
|
4690
4826
|
}
|
|
4691
4827
|
return {
|
|
@@ -6072,8 +6208,10 @@ var HEYGEN_OPERATIONS = {
|
|
|
6072
6208
|
};
|
|
6073
6209
|
|
|
6074
6210
|
// src/providers/heygen/node-helpers.ts
|
|
6211
|
+
import { once } from "events";
|
|
6075
6212
|
import fs29 from "fs";
|
|
6076
6213
|
import path32 from "path";
|
|
6214
|
+
import { pipeline as pipeline2 } from "stream/promises";
|
|
6077
6215
|
init_errors();
|
|
6078
6216
|
init_redact();
|
|
6079
6217
|
|
|
@@ -6503,6 +6641,7 @@ function videoState(video, kind = "video", id = video.id, opts = {}) {
|
|
|
6503
6641
|
return state;
|
|
6504
6642
|
}
|
|
6505
6643
|
function translationState(t) {
|
|
6644
|
+
const audioOnly = !str(t.video_url) && !!str(t.audio_url);
|
|
6506
6645
|
return {
|
|
6507
6646
|
kind: "translation",
|
|
6508
6647
|
id: t.id,
|
|
@@ -6510,6 +6649,7 @@ function translationState(t) {
|
|
|
6510
6649
|
done: t.status === "completed" || t.status === "failed",
|
|
6511
6650
|
succeeded: t.status === "completed",
|
|
6512
6651
|
outputUrl: str(t.video_url) ?? str(t.audio_url),
|
|
6652
|
+
...audioOnly ? { outputIsAudio: true } : {},
|
|
6513
6653
|
captionUrl: str(t.srt_caption_url) ?? str(t.vtt_caption_url),
|
|
6514
6654
|
outputLanguage: str(t.output_language),
|
|
6515
6655
|
failureMessage: str(t.failure_message),
|
|
@@ -6632,13 +6772,29 @@ function getJobState(client, kind, id, opts = {}) {
|
|
|
6632
6772
|
return createJobStatusReader(client, kind, id, opts)();
|
|
6633
6773
|
}
|
|
6634
6774
|
var DEFAULT_POLL_INTERVAL_MS = 1e4;
|
|
6635
|
-
|
|
6636
|
-
|
|
6775
|
+
var STAGGER_STEP_MS = 500;
|
|
6776
|
+
var STAGGER_MAX_MS = 5e3;
|
|
6777
|
+
function staggerOffsetMs(index, intervalMs) {
|
|
6778
|
+
if (!Number.isFinite(index) || index <= 0) return 0;
|
|
6779
|
+
const step = Math.min(STAGGER_STEP_MS, Math.max(0, intervalMs));
|
|
6780
|
+
return Math.min(index * step, STAGGER_MAX_MS);
|
|
6781
|
+
}
|
|
6782
|
+
var defaultSleep3 = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
6783
|
+
async function waitForJob(client, kind, id, opts) {
|
|
6784
|
+
const { burnCaptions, startDelayMs = 0, ...poll2 } = opts;
|
|
6785
|
+
let pendingDelay = Math.min(Math.max(0, startDelayMs), Math.floor(poll2.timeoutMs / 2));
|
|
6786
|
+
const read = createJobStatusReader(client, kind, id, { burnCaptions });
|
|
6637
6787
|
return pollUntil({
|
|
6638
6788
|
intervalMs: DEFAULT_POLL_INTERVAL_MS,
|
|
6639
6789
|
maxIntervalMs: 3e4,
|
|
6640
6790
|
...poll2,
|
|
6641
|
-
fetchStatus:
|
|
6791
|
+
fetchStatus: async (ctx) => {
|
|
6792
|
+
const delay = pendingDelay;
|
|
6793
|
+
if (delay <= 0) return read(ctx);
|
|
6794
|
+
pendingDelay = 0;
|
|
6795
|
+
await (poll2.sleep ?? defaultSleep3)(delay);
|
|
6796
|
+
return read({ remainingMs: Math.max(1, ctx.remainingMs - delay) });
|
|
6797
|
+
},
|
|
6642
6798
|
isTerminal: (s) => s.done
|
|
6643
6799
|
});
|
|
6644
6800
|
}
|
|
@@ -6662,6 +6818,7 @@ function resumeCommand2(kind, id, opts = {}) {
|
|
|
6662
6818
|
|
|
6663
6819
|
// src/providers/heygen/request-builders.ts
|
|
6664
6820
|
init_errors();
|
|
6821
|
+
init_redact();
|
|
6665
6822
|
function oneOf(value, allowed, flag) {
|
|
6666
6823
|
if (value === void 0) return void 0;
|
|
6667
6824
|
if (!allowed.includes(value)) {
|
|
@@ -6683,6 +6840,30 @@ function requireHttps(url, flag) {
|
|
|
6683
6840
|
if (!/^https:\/\//i.test(url)) throw new ValidationError(`${flag} must be an https:// URL`);
|
|
6684
6841
|
return url;
|
|
6685
6842
|
}
|
|
6843
|
+
function isHttpsUrl(url) {
|
|
6844
|
+
try {
|
|
6845
|
+
return new URL(url).protocol === "https:";
|
|
6846
|
+
} catch {
|
|
6847
|
+
return false;
|
|
6848
|
+
}
|
|
6849
|
+
}
|
|
6850
|
+
var LOOPBACK_HOSTS = /* @__PURE__ */ new Set(["localhost", "127.0.0.1", "[::1]"]);
|
|
6851
|
+
function checkBaseUrl(raw, variable) {
|
|
6852
|
+
const value = raw.trim();
|
|
6853
|
+
let url;
|
|
6854
|
+
try {
|
|
6855
|
+
url = new URL(value);
|
|
6856
|
+
} catch {
|
|
6857
|
+
throw new ConfigError(`${variable} must be an absolute https:// URL`);
|
|
6858
|
+
}
|
|
6859
|
+
const loopbackHttp = url.protocol === "http:" && LOOPBACK_HOSTS.has(url.hostname);
|
|
6860
|
+
if (url.protocol !== "https:" && !loopbackHttp) {
|
|
6861
|
+
throw new ConfigError(
|
|
6862
|
+
`${variable} must use https:// (got ${redactUrl(value)}); plain http:// is only allowed for localhost, 127.0.0.1, or [::1]`
|
|
6863
|
+
);
|
|
6864
|
+
}
|
|
6865
|
+
return value;
|
|
6866
|
+
}
|
|
6686
6867
|
function classifyMediaRef(value) {
|
|
6687
6868
|
const v = value.trim();
|
|
6688
6869
|
if (!v) throw new ValidationError("Media reference must not be empty");
|
|
@@ -6948,9 +7129,11 @@ function requireHeygenKey() {
|
|
|
6948
7129
|
return key;
|
|
6949
7130
|
}
|
|
6950
7131
|
function createHeygenClientFromEnv() {
|
|
7132
|
+
const apiKey = requireHeygenKey();
|
|
7133
|
+
const override = resolveKey("HEYGEN_BASE_URL")?.trim();
|
|
6951
7134
|
return new HeygenClient({
|
|
6952
|
-
apiKey
|
|
6953
|
-
baseUrl:
|
|
7135
|
+
apiKey,
|
|
7136
|
+
baseUrl: override ? checkBaseUrl(override, "HEYGEN_BASE_URL") : DEFAULT_HEYGEN_BASE_URL
|
|
6954
7137
|
});
|
|
6955
7138
|
}
|
|
6956
7139
|
async function uploadLocalFile(client, filePath, logger) {
|
|
@@ -7012,9 +7195,15 @@ function addWaitOptions(cmd, defaultTimeoutMs = DEFAULT_WAIT_TIMEOUT_MS) {
|
|
|
7012
7195
|
"--poll-interval <ms>",
|
|
7013
7196
|
"Initial poll interval (backs off to 30s)",
|
|
7014
7197
|
String(DEFAULT_POLL_INTERVAL_MS)
|
|
7015
|
-
).option("--no-download", "Do not download outputs; print output URLs instead").option(
|
|
7016
|
-
|
|
7017
|
-
|
|
7198
|
+
).option("--no-download", "Do not download outputs; print output URLs instead").option(
|
|
7199
|
+
"--output <path>",
|
|
7200
|
+
"Output file or directory. A path ending in / or \\, an existing directory, or a path without an extension is a directory; with several outputs, a media file name gets a per-language suffix (dub.mp4 -> dub-Spanish.mp4)"
|
|
7201
|
+
).option("--captions", "Also download the caption/subtitle file when one is available").option("--no-thumb", "Skip downloading the video thumbnail").option("--json", "Print a JSON summary instead of human-readable output").option("-v, --verbose", "Verbose logging");
|
|
7202
|
+
}
|
|
7203
|
+
var VIDEO_EXTS = /* @__PURE__ */ new Set(["mp4", "webm", "mov"]);
|
|
7204
|
+
var AUDIO_EXTS = /* @__PURE__ */ new Set(["mp3", "wav", "m4a", "aac"]);
|
|
7205
|
+
var MEDIA_EXTS = /* @__PURE__ */ new Set([...VIDEO_EXTS, ...AUDIO_EXTS]);
|
|
7206
|
+
var OUTPUT_EXTS = /* @__PURE__ */ new Set([...MEDIA_EXTS, "srt", "vtt"]);
|
|
7018
7207
|
function inferExtension(url, fallback) {
|
|
7019
7208
|
try {
|
|
7020
7209
|
const ext = path32.extname(new URL(url).pathname).slice(1).toLowerCase();
|
|
@@ -7026,16 +7215,57 @@ function inferExtension(url, fallback) {
|
|
|
7026
7215
|
function safeSlug(value) {
|
|
7027
7216
|
return value.replace(/[^A-Za-z0-9_-]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 40) || "out";
|
|
7028
7217
|
}
|
|
7218
|
+
var MAX_REDIRECTS2 = 5;
|
|
7219
|
+
var isRedirect = (status) => [301, 302, 303, 307, 308].includes(status);
|
|
7220
|
+
async function downloadHttpsOnly(url, dest, timeoutMs) {
|
|
7221
|
+
const signal = AbortSignal.timeout(timeoutMs);
|
|
7222
|
+
let current = url;
|
|
7223
|
+
for (let hop = 0; ; hop++) {
|
|
7224
|
+
if (!isHttpsUrl(current)) {
|
|
7225
|
+
throw new HttpError(0, "refusing to follow a redirect to a non-https URL", current);
|
|
7226
|
+
}
|
|
7227
|
+
let response;
|
|
7228
|
+
try {
|
|
7229
|
+
response = await globalThis.fetch(current, { signal, redirect: "manual" });
|
|
7230
|
+
} catch (cause) {
|
|
7231
|
+
throw new HttpError(0, cause instanceof Error ? cause.message : String(cause), current);
|
|
7232
|
+
}
|
|
7233
|
+
if (response.url && !isHttpsUrl(response.url)) {
|
|
7234
|
+
await response.body?.cancel();
|
|
7235
|
+
throw new HttpError(0, "redirected to a non-https URL", response.url);
|
|
7236
|
+
}
|
|
7237
|
+
const location = response.headers.get("location");
|
|
7238
|
+
if (isRedirect(response.status) && location) {
|
|
7239
|
+
await response.body?.cancel();
|
|
7240
|
+
if (hop >= MAX_REDIRECTS2) throw new HttpError(0, "too many redirects", url);
|
|
7241
|
+
current = new URL(location, current).href;
|
|
7242
|
+
continue;
|
|
7243
|
+
}
|
|
7244
|
+
if (!response.ok) throw new HttpError(response.status, "(download failed)", current);
|
|
7245
|
+
if (!response.body) throw new HttpError(0, "Response body is null", current);
|
|
7246
|
+
fs29.mkdirSync(path32.dirname(dest), { recursive: true });
|
|
7247
|
+
const fileStream = fs29.createWriteStream(dest);
|
|
7248
|
+
try {
|
|
7249
|
+
await pipeline2(response.body, fileStream);
|
|
7250
|
+
} catch (err2) {
|
|
7251
|
+
if (!fileStream.closed) await once(fileStream, "close");
|
|
7252
|
+
throw err2;
|
|
7253
|
+
}
|
|
7254
|
+
return;
|
|
7255
|
+
}
|
|
7256
|
+
}
|
|
7029
7257
|
async function downloadOutput(url, dest, resumeHint) {
|
|
7030
7258
|
if (!url) {
|
|
7031
7259
|
throw new ValidationError(`HeyGen reported success but returned no output URL. ${resumeHint}`);
|
|
7032
7260
|
}
|
|
7033
|
-
if (
|
|
7034
|
-
throw new ValidationError(
|
|
7261
|
+
if (!isHttpsUrl(url)) {
|
|
7262
|
+
throw new ValidationError(
|
|
7263
|
+
`Refusing to download HeyGen output from a non-https URL (${redactUrl(url)})`
|
|
7264
|
+
);
|
|
7035
7265
|
}
|
|
7036
7266
|
const tmp = `${dest}.${process.pid}-${Date.now()}.part`;
|
|
7037
7267
|
try {
|
|
7038
|
-
await
|
|
7268
|
+
await downloadHttpsOnly(url, tmp, 6e5);
|
|
7039
7269
|
fs29.renameSync(tmp, dest);
|
|
7040
7270
|
} catch (e) {
|
|
7041
7271
|
fs29.rmSync(tmp, { force: true });
|
|
@@ -7052,15 +7282,55 @@ function defaultOutputPath(state, ext) {
|
|
|
7052
7282
|
const lang = state.outputLanguage ? `-${safeSlug(state.outputLanguage)}` : "";
|
|
7053
7283
|
return path32.join(getOutputDir(), `heygen-${state.kind}${lang}-${safeSlug(state.id)}.${ext}`);
|
|
7054
7284
|
}
|
|
7055
|
-
function
|
|
7056
|
-
|
|
7057
|
-
|
|
7058
|
-
|
|
7059
|
-
|
|
7060
|
-
return path32.join(dir, path32.basename(defaultOutputPath(state, ext)));
|
|
7285
|
+
function statOrUndefined(p) {
|
|
7286
|
+
try {
|
|
7287
|
+
return fs29.statSync(p);
|
|
7288
|
+
} catch {
|
|
7289
|
+
return void 0;
|
|
7061
7290
|
}
|
|
7062
|
-
return path32.resolve(output);
|
|
7063
7291
|
}
|
|
7292
|
+
function isDirectoryOutput2(output, target) {
|
|
7293
|
+
if (/[\\/]$/.test(output)) return true;
|
|
7294
|
+
const stat = statOrUndefined(target);
|
|
7295
|
+
if (stat) return stat.isDirectory();
|
|
7296
|
+
return path32.extname(target) === "";
|
|
7297
|
+
}
|
|
7298
|
+
function withMediaExt(file, ext) {
|
|
7299
|
+
const given = path32.extname(file).slice(1).toLowerCase();
|
|
7300
|
+
const mismatch = VIDEO_EXTS.has(given) && AUDIO_EXTS.has(ext) || AUDIO_EXTS.has(given) && VIDEO_EXTS.has(ext);
|
|
7301
|
+
return mismatch ? `${file.slice(0, -(given.length + 1))}.${ext}` : file;
|
|
7302
|
+
}
|
|
7303
|
+
function outputPathFor(state, ext, output, count) {
|
|
7304
|
+
if (!output) return defaultOutputPath(state, ext);
|
|
7305
|
+
const target = path32.resolve(output);
|
|
7306
|
+
const givenExt = path32.extname(target);
|
|
7307
|
+
const isTemplate = count > 1 && MEDIA_EXTS.has(givenExt.slice(1).toLowerCase());
|
|
7308
|
+
if (isDirectoryOutput2(output, target) || count > 1 && !isTemplate) {
|
|
7309
|
+
fs29.mkdirSync(target, { recursive: true });
|
|
7310
|
+
return path32.join(target, path32.basename(defaultOutputPath(state, ext)));
|
|
7311
|
+
}
|
|
7312
|
+
if (!isTemplate) return withMediaExt(target, ext);
|
|
7313
|
+
const base = path32.basename(target, givenExt);
|
|
7314
|
+
const suffix = safeSlug(state.outputLanguage ?? state.id);
|
|
7315
|
+
return withMediaExt(path32.join(path32.dirname(target), `${base}-${suffix}${givenExt}`), ext);
|
|
7316
|
+
}
|
|
7317
|
+
function claimPath(dest, state, used) {
|
|
7318
|
+
const ext = path32.extname(dest);
|
|
7319
|
+
const stem = dest.slice(0, dest.length - ext.length);
|
|
7320
|
+
let candidate = dest;
|
|
7321
|
+
for (let n = 1; used.has(candidate.toLowerCase()); n++) {
|
|
7322
|
+
const id = safeSlug(state.id);
|
|
7323
|
+
candidate = n === 1 ? `${stem}-${id}${ext}` : `${stem}-${id}-${n}${ext}`;
|
|
7324
|
+
}
|
|
7325
|
+
used.add(candidate.toLowerCase());
|
|
7326
|
+
return candidate;
|
|
7327
|
+
}
|
|
7328
|
+
function captionPathFor(dest, capExt) {
|
|
7329
|
+
const stem = path32.join(path32.dirname(dest), path32.basename(dest, path32.extname(dest)));
|
|
7330
|
+
const capDest = `${stem}.${capExt}`;
|
|
7331
|
+
return capDest.toLowerCase() === dest.toLowerCase() ? `${stem}.captions.${capExt}` : capDest;
|
|
7332
|
+
}
|
|
7333
|
+
var downloadThumbHttpsOnly = (url, dest) => downloadHttpsOnly(url, dest, 12e4);
|
|
7064
7334
|
function summarizeState(state) {
|
|
7065
7335
|
const out = {
|
|
7066
7336
|
kind: state.kind,
|
|
@@ -7078,7 +7348,7 @@ function summarizeState(state) {
|
|
|
7078
7348
|
if (state.failureMessage) out.failure_message = state.failureMessage;
|
|
7079
7349
|
return out;
|
|
7080
7350
|
}
|
|
7081
|
-
async function deliverOne(state, opts, count, logger, files) {
|
|
7351
|
+
async function deliverOne(state, opts, count, logger, files, usedPaths) {
|
|
7082
7352
|
const hint = `Refresh with: ${resumeCommand2(state.kind, state.id, opts)}`;
|
|
7083
7353
|
if (opts.download === false) {
|
|
7084
7354
|
if (!state.outputUrl) {
|
|
@@ -7091,39 +7361,42 @@ async function deliverOne(state, opts, count, logger, files) {
|
|
|
7091
7361
|
};
|
|
7092
7362
|
return { state, files, urls };
|
|
7093
7363
|
}
|
|
7094
|
-
const ext = inferExtension(state.outputUrl ?? "", "mp4");
|
|
7095
|
-
const dest = outputPathFor(state, ext, opts.output, count);
|
|
7364
|
+
const ext = inferExtension(state.outputUrl ?? "", state.outputIsAudio ? "mp3" : "mp4");
|
|
7365
|
+
const dest = claimPath(outputPathFor(state, ext, opts.output, count), state, usedPaths);
|
|
7096
7366
|
await downloadOutput(state.outputUrl, dest, hint);
|
|
7097
7367
|
logger.success(`Saved ${dest}`);
|
|
7098
7368
|
files.push(dest);
|
|
7099
7369
|
if (opts.captions && state.captionUrl) {
|
|
7100
7370
|
const capExt = inferExtension(state.captionUrl, "srt");
|
|
7101
|
-
const capDest =
|
|
7102
|
-
path32.dirname(dest),
|
|
7103
|
-
`${path32.basename(dest, path32.extname(dest))}.${capExt}`
|
|
7104
|
-
);
|
|
7371
|
+
const capDest = claimPath(captionPathFor(dest, capExt), state, usedPaths);
|
|
7105
7372
|
await downloadOutput(state.captionUrl, capDest, hint);
|
|
7106
7373
|
logger.success(`Saved ${capDest}`);
|
|
7107
7374
|
files.push(capDest);
|
|
7108
7375
|
} else if (opts.captions) {
|
|
7109
7376
|
logger.warn(`No caption file available for ${state.kind} ${state.id}`);
|
|
7110
7377
|
}
|
|
7111
|
-
if (state.kind === "video" || state.kind === "agent") {
|
|
7112
|
-
const
|
|
7113
|
-
|
|
7114
|
-
logger
|
|
7115
|
-
}
|
|
7116
|
-
|
|
7378
|
+
if ((state.kind === "video" || state.kind === "agent") && opts.thumb !== false) {
|
|
7379
|
+
const thumbUrl = detectThumbUrl(state.raw);
|
|
7380
|
+
if (thumbUrl && !isHttpsUrl(thumbUrl)) {
|
|
7381
|
+
logger.warn(`Skipping thumbnail from a non-https URL (${redactUrl(thumbUrl)})`);
|
|
7382
|
+
} else {
|
|
7383
|
+
const thumb = await maybeDownloadThumb(state.raw, dest, {
|
|
7384
|
+
logger,
|
|
7385
|
+
download: downloadThumbHttpsOnly
|
|
7386
|
+
});
|
|
7387
|
+
if (thumb) files.push(thumb);
|
|
7388
|
+
}
|
|
7117
7389
|
}
|
|
7118
7390
|
return { state, files };
|
|
7119
7391
|
}
|
|
7120
7392
|
async function deliverOutputs(states, opts, logger) {
|
|
7121
7393
|
const finished = [];
|
|
7122
7394
|
const failures = [];
|
|
7395
|
+
const usedPaths = /* @__PURE__ */ new Set();
|
|
7123
7396
|
for (const state of states) {
|
|
7124
7397
|
const files = [];
|
|
7125
7398
|
try {
|
|
7126
|
-
finished.push(await deliverOne(state, opts, states.length, logger, files));
|
|
7399
|
+
finished.push(await deliverOne(state, opts, states.length, logger, files, usedPaths));
|
|
7127
7400
|
} catch (e) {
|
|
7128
7401
|
failures.push(`${state.kind} ${state.id}: ${e instanceof Error ? e.message : String(e)}`);
|
|
7129
7402
|
if (files.length > 0) finished.push({ state, files });
|
|
@@ -7176,9 +7449,10 @@ async function completeJobs(client, kind, ids, opts, logger) {
|
|
|
7176
7449
|
);
|
|
7177
7450
|
const settled = await Promise.allSettled(
|
|
7178
7451
|
ids.map(
|
|
7179
|
-
(id) => waitForJob(client, kind, id, {
|
|
7452
|
+
(id, index) => waitForJob(client, kind, id, {
|
|
7180
7453
|
timeoutMs,
|
|
7181
7454
|
intervalMs,
|
|
7455
|
+
startDelayMs: staggerOffsetMs(index, intervalMs),
|
|
7182
7456
|
maxIntervalMs: Math.max(intervalMs, 3e4),
|
|
7183
7457
|
burnCaptions: opts.burnCaptions,
|
|
7184
7458
|
resumeHint: `Resume with: ${resumeCommand2(kind, id, opts)}`,
|
|
@@ -10391,6 +10665,19 @@ function mapReplicateError(error, subject) {
|
|
|
10391
10665
|
}
|
|
10392
10666
|
return new ReplicateApiError(message, status, retryAfterMs);
|
|
10393
10667
|
}
|
|
10668
|
+
var ORPHAN_HINT = "The prediction may have been created anyway (Replicate has no idempotency key); check https://replicate.com/predictions before retrying to avoid paying twice";
|
|
10669
|
+
function withOrphanHint(error) {
|
|
10670
|
+
if (error instanceof ReplicateApiError) {
|
|
10671
|
+
const { status } = error;
|
|
10672
|
+
const mayExist = status === 0 || status >= 200 && status < 300 || status >= 500;
|
|
10673
|
+
if (!mayExist) return error;
|
|
10674
|
+
return new ReplicateApiError(`${error.message}. ${ORPHAN_HINT}`, status, error.retryAfterMs);
|
|
10675
|
+
}
|
|
10676
|
+
if (error instanceof ProviderError) {
|
|
10677
|
+
return new ProviderError(`${error.message}. ${ORPHAN_HINT}`, error.provider);
|
|
10678
|
+
}
|
|
10679
|
+
return error;
|
|
10680
|
+
}
|
|
10394
10681
|
function validatePrediction(value) {
|
|
10395
10682
|
if (value === null || typeof value !== "object" || typeof value.id !== "string" || typeof value.status !== "string") {
|
|
10396
10683
|
throw new ProviderError("Malformed Replicate response: missing prediction id/status", PROVIDER);
|
|
@@ -10408,7 +10695,7 @@ function assertSucceeded(prediction2) {
|
|
|
10408
10695
|
PROVIDER
|
|
10409
10696
|
);
|
|
10410
10697
|
}
|
|
10411
|
-
var
|
|
10698
|
+
var defaultSleep4 = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
10412
10699
|
var ReplicateClient = class {
|
|
10413
10700
|
apiKey;
|
|
10414
10701
|
baseUrl;
|
|
@@ -10419,7 +10706,7 @@ var ReplicateClient = class {
|
|
|
10419
10706
|
this.apiKey = opts.apiKey;
|
|
10420
10707
|
this.baseUrl = (opts.baseUrl || DEFAULT_REPLICATE_BASE_URL).replace(/\/+$/, "");
|
|
10421
10708
|
this.doFetch = opts.fetch ?? ((input, init) => globalThis.fetch(input, init));
|
|
10422
|
-
this.sleep = opts.sleep ??
|
|
10709
|
+
this.sleep = opts.sleep ?? defaultSleep4;
|
|
10423
10710
|
}
|
|
10424
10711
|
headers(extra = {}) {
|
|
10425
10712
|
return { accept: "application/json", authorization: `Bearer ${this.apiKey}`, ...extra };
|
|
@@ -10459,7 +10746,7 @@ var ReplicateClient = class {
|
|
|
10459
10746
|
await this.sleep(delay);
|
|
10460
10747
|
continue;
|
|
10461
10748
|
}
|
|
10462
|
-
throw mapReplicateError(error, subject);
|
|
10749
|
+
throw withOrphanHint(mapReplicateError(error, subject));
|
|
10463
10750
|
}
|
|
10464
10751
|
}
|
|
10465
10752
|
}
|
|
@@ -10606,23 +10893,137 @@ init_errors();
|
|
|
10606
10893
|
|
|
10607
10894
|
// src/providers/replicate/commands/prediction-flow.ts
|
|
10608
10895
|
init_errors();
|
|
10896
|
+
import { InvalidArgumentError, Option as Option2 } from "commander";
|
|
10609
10897
|
|
|
10610
10898
|
// src/providers/replicate/node-outputs.ts
|
|
10611
10899
|
init_errors();
|
|
10612
|
-
init_redact();
|
|
10613
|
-
import { once as once2 } from "events";
|
|
10614
10900
|
import fs44 from "fs";
|
|
10615
10901
|
import path51 from "path";
|
|
10616
|
-
|
|
10617
|
-
import { pipeline as pipeline2 } from "stream/promises";
|
|
10902
|
+
init_redact();
|
|
10618
10903
|
|
|
10619
10904
|
// src/providers/replicate/output.ts
|
|
10620
10905
|
init_redact();
|
|
10906
|
+
|
|
10907
|
+
// src/core/url-safety.ts
|
|
10908
|
+
init_errors();
|
|
10909
|
+
function ipv4Octets(host) {
|
|
10910
|
+
const parts = host.split(".");
|
|
10911
|
+
if (parts.length !== 4 || !parts.every((p) => /^\d{1,3}$/.test(p))) return void 0;
|
|
10912
|
+
const octets = parts.map(Number);
|
|
10913
|
+
return octets.every((o) => o <= 255) ? octets : void 0;
|
|
10914
|
+
}
|
|
10915
|
+
function isNonPublicIpv4([a = 0, b = 0, c2 = 0]) {
|
|
10916
|
+
return a === 0 || // "this network" / unspecified
|
|
10917
|
+
a === 10 || a === 127 || // loopback
|
|
10918
|
+
a === 100 && b >= 64 && b <= 127 || // carrier-grade NAT
|
|
10919
|
+
a === 169 && b === 254 || // link-local (cloud metadata)
|
|
10920
|
+
a === 172 && b >= 16 && b <= 31 || a === 192 && b === 0 && c2 === 0 || // IETF protocol assignments
|
|
10921
|
+
a === 192 && b === 0 && c2 === 2 || // TEST-NET-1
|
|
10922
|
+
a === 192 && b === 168 || a === 198 && (b === 18 || b === 19) || // benchmarking
|
|
10923
|
+
a === 198 && b === 51 && c2 === 100 || // TEST-NET-2
|
|
10924
|
+
a === 203 && b === 0 && c2 === 113 || // TEST-NET-3
|
|
10925
|
+
a >= 224;
|
|
10926
|
+
}
|
|
10927
|
+
function ipv6Groups(host) {
|
|
10928
|
+
if (!/^[0-9a-f:.]+$/i.test(host) || !host.includes(":")) return void 0;
|
|
10929
|
+
let text = host;
|
|
10930
|
+
const tail = /^(.*:)(\d+\.\d+\.\d+\.\d+)$/.exec(text);
|
|
10931
|
+
if (tail) {
|
|
10932
|
+
const octets = ipv4Octets(tail[2]);
|
|
10933
|
+
if (!octets) return void 0;
|
|
10934
|
+
const [a = 0, b = 0, c2 = 0, d = 0] = octets;
|
|
10935
|
+
text = `${tail[1]}${(a << 8 | b).toString(16)}:${(c2 << 8 | d).toString(16)}`;
|
|
10936
|
+
}
|
|
10937
|
+
const halves = text.split("::");
|
|
10938
|
+
if (halves.length > 2) return void 0;
|
|
10939
|
+
const parse = (s) => s === "" ? [] : s.split(":");
|
|
10940
|
+
const head = parse(halves[0]);
|
|
10941
|
+
const rest = halves.length === 2 ? parse(halves[1]) : [];
|
|
10942
|
+
const missing = 8 - head.length - rest.length;
|
|
10943
|
+
if (halves.length === 2 ? missing < 1 : missing !== 0) return void 0;
|
|
10944
|
+
const all = [...head, ...Array(halves.length === 2 ? missing : 0).fill("0"), ...rest];
|
|
10945
|
+
if (!all.every((g) => /^[0-9a-f]{1,4}$/i.test(g))) return void 0;
|
|
10946
|
+
return all.map((g) => Number.parseInt(g, 16));
|
|
10947
|
+
}
|
|
10948
|
+
function embeddedIpv4(hi, lo) {
|
|
10949
|
+
return [hi >> 8, hi & 255, lo >> 8, lo & 255];
|
|
10950
|
+
}
|
|
10951
|
+
function isNonPublicIpv6(groups) {
|
|
10952
|
+
const [g0 = 0, g1 = 0, g2 = 0, g3 = 0, g4 = 0, g5 = 0, g6 = 0, g7 = 0] = groups;
|
|
10953
|
+
const zeroUpTo = (n) => groups.slice(0, n).every((g) => g === 0);
|
|
10954
|
+
if (zeroUpTo(5) && g5 === 65535) return isNonPublicIpv4(embeddedIpv4(g6, g7));
|
|
10955
|
+
if (g0 === 100 && g1 === 65435 && g2 === 0 && g3 === 0 && g4 === 0 && g5 === 0) {
|
|
10956
|
+
return isNonPublicIpv4(embeddedIpv4(g6, g7));
|
|
10957
|
+
}
|
|
10958
|
+
if (g0 === 8194) return isNonPublicIpv4(embeddedIpv4(g1, g2));
|
|
10959
|
+
return g0 === 0 || // unspecified (::), loopback (::1), IPv4-compatible and other reserved
|
|
10960
|
+
g0 === 256 && g1 === 0 && g2 === 0 && g3 === 0 || // discard-only 100::/64
|
|
10961
|
+
(g0 & 65024) === 64512 || // unique local fc00::/7
|
|
10962
|
+
(g0 & 65472) === 65152 || // link-local fe80::/10
|
|
10963
|
+
(g0 & 65472) === 65216 || // deprecated site-local fec0::/10
|
|
10964
|
+
(g0 & 65280) === 65280;
|
|
10965
|
+
}
|
|
10966
|
+
function bareHost(hostname) {
|
|
10967
|
+
const host = hostname.toLowerCase().replace(/\.$/, "");
|
|
10968
|
+
return host.startsWith("[") && host.endsWith("]") ? host.slice(1, -1) : host;
|
|
10969
|
+
}
|
|
10970
|
+
function isLoopbackHost(hostname) {
|
|
10971
|
+
const host = bareHost(hostname);
|
|
10972
|
+
if (host === "localhost" || host.endsWith(".localhost")) return true;
|
|
10973
|
+
const octets = ipv4Octets(host);
|
|
10974
|
+
if (octets) return octets[0] === 127;
|
|
10975
|
+
const groups = ipv6Groups(host);
|
|
10976
|
+
if (!groups) return false;
|
|
10977
|
+
if (groups.slice(0, 5).every((g) => g === 0) && groups[5] === 65535) {
|
|
10978
|
+
return groups[6] !== void 0 && groups[6] >> 8 === 127;
|
|
10979
|
+
}
|
|
10980
|
+
return groups.slice(0, 7).every((g) => g === 0) && groups[7] === 1;
|
|
10981
|
+
}
|
|
10982
|
+
function isNonPublicHost(hostname) {
|
|
10983
|
+
const host = bareHost(hostname);
|
|
10984
|
+
if (host === "localhost" || host.endsWith(".localhost")) return true;
|
|
10985
|
+
const octets = ipv4Octets(host);
|
|
10986
|
+
if (octets) return isNonPublicIpv4(octets);
|
|
10987
|
+
const groups = ipv6Groups(host);
|
|
10988
|
+
return groups !== void 0 && isNonPublicIpv6(groups);
|
|
10989
|
+
}
|
|
10990
|
+
function checkUrlSafety(url, opts) {
|
|
10991
|
+
const action = opts.action ?? "download from";
|
|
10992
|
+
let parsed;
|
|
10993
|
+
try {
|
|
10994
|
+
parsed = new URL(url);
|
|
10995
|
+
} catch {
|
|
10996
|
+
throw new ProviderError(`Refusing to ${action} a malformed URL`, opts.provider);
|
|
10997
|
+
}
|
|
10998
|
+
const host = parsed.hostname.toLowerCase().replace(/\.$/, "");
|
|
10999
|
+
const localOk = opts.allowLocalhostHttp === true && isLoopbackHost(host);
|
|
11000
|
+
const httpOk = parsed.protocol === "https:" || localOk && parsed.protocol === "http:";
|
|
11001
|
+
if (!httpOk) {
|
|
11002
|
+
throw new ProviderError(
|
|
11003
|
+
`Refusing to ${action} ${host || "this URL"}: only https:// URLs are allowed`,
|
|
11004
|
+
opts.provider
|
|
11005
|
+
);
|
|
11006
|
+
}
|
|
11007
|
+
if (!localOk && isNonPublicHost(host)) {
|
|
11008
|
+
throw new ProviderError(
|
|
11009
|
+
`Refusing to ${action} ${host}: it points to a local or private address`,
|
|
11010
|
+
opts.provider
|
|
11011
|
+
);
|
|
11012
|
+
}
|
|
11013
|
+
const trusted = (opts.trustedHosts ?? []).some((t) => host === t || host.endsWith(`.${t}`));
|
|
11014
|
+
return { host, trusted };
|
|
11015
|
+
}
|
|
11016
|
+
|
|
11017
|
+
// src/providers/replicate/output.ts
|
|
10621
11018
|
var HTTP_RE = /^https?:\/\//i;
|
|
10622
11019
|
var DATA_URI_RE = /^data:[^,]*,/i;
|
|
10623
11020
|
function isMediaRef(value) {
|
|
10624
11021
|
return typeof value === "string" && (HTTP_RE.test(value) || DATA_URI_RE.test(value));
|
|
10625
11022
|
}
|
|
11023
|
+
var REPLICATE_DELIVERY_HOST = "replicate.delivery";
|
|
11024
|
+
function checkOutputUrl(url) {
|
|
11025
|
+
return checkUrlSafety(url, { provider: "replicate", trustedHosts: [REPLICATE_DELIVERY_HOST] });
|
|
11026
|
+
}
|
|
10626
11027
|
function extractOutput(output) {
|
|
10627
11028
|
const result = { media: [], texts: [] };
|
|
10628
11029
|
const seen = /* @__PURE__ */ new Set();
|
|
@@ -10781,59 +11182,53 @@ function resolveDestination(predictionId, index, count, ext, opts) {
|
|
|
10781
11182
|
const base = givenExt ? output.slice(0, -givenExt.length) : output;
|
|
10782
11183
|
return `${base}-${String(index + 1).padStart(2, "0")}${ext}`;
|
|
10783
11184
|
}
|
|
10784
|
-
async function writeResponse(response, dest) {
|
|
10785
|
-
if (!response.body) throw new HttpError(0, "Response body is null", response.url);
|
|
10786
|
-
const fileStream = fs44.createWriteStream(dest);
|
|
10787
|
-
try {
|
|
10788
|
-
await pipeline2(
|
|
10789
|
-
Readable.fromWeb(response.body),
|
|
10790
|
-
fileStream
|
|
10791
|
-
);
|
|
10792
|
-
} catch (err2) {
|
|
10793
|
-
if (!fileStream.closed) await once2(fileStream, "close");
|
|
10794
|
-
throw err2;
|
|
10795
|
-
}
|
|
10796
|
-
}
|
|
10797
11185
|
async function saveOne(item, predictionId, index, count, opts) {
|
|
10798
11186
|
if (item.url.startsWith("data:")) {
|
|
10799
11187
|
const comma = item.url.indexOf(",");
|
|
10800
11188
|
const meta = item.url.slice(5, comma);
|
|
10801
11189
|
const payload = item.url.slice(comma + 1);
|
|
10802
11190
|
const bytes = meta.endsWith(";base64") ? Buffer.from(payload, "base64") : Buffer.from(decodeURIComponent(payload));
|
|
10803
|
-
const
|
|
11191
|
+
const dest = resolveDestination(
|
|
10804
11192
|
predictionId,
|
|
10805
11193
|
index,
|
|
10806
11194
|
count,
|
|
10807
11195
|
extFromUrl4(item.url) ?? ".bin",
|
|
10808
11196
|
opts
|
|
10809
11197
|
);
|
|
10810
|
-
fs44.mkdirSync(path51.dirname(
|
|
10811
|
-
fs44.writeFileSync(
|
|
10812
|
-
return
|
|
10813
|
-
}
|
|
10814
|
-
const doFetch = opts.fetch ?? ((url, init) => globalThis.fetch(url, init));
|
|
10815
|
-
let response;
|
|
10816
|
-
try {
|
|
10817
|
-
response = await doFetch(item.url, { signal: AbortSignal.timeout(DOWNLOAD_TIMEOUT_MS) });
|
|
10818
|
-
} catch (cause) {
|
|
10819
|
-
throw new HttpError(0, cause instanceof Error ? cause.message : String(cause), item.url);
|
|
10820
|
-
}
|
|
10821
|
-
if (!response.ok) {
|
|
10822
|
-
const hint = response.status === 404 || response.status === 410 ? " (output may have expired)" : "";
|
|
10823
|
-
throw new HttpError(response.status, `download failed${hint}`, item.url);
|
|
11198
|
+
fs44.mkdirSync(path51.dirname(dest), { recursive: true });
|
|
11199
|
+
fs44.writeFileSync(dest, bytes);
|
|
11200
|
+
return dest;
|
|
10824
11201
|
}
|
|
10825
|
-
const
|
|
10826
|
-
const dest = resolveDestination(predictionId, index, count, ext, opts);
|
|
10827
|
-
fs44.mkdirSync(path51.dirname(dest), { recursive: true });
|
|
11202
|
+
const warned = opts.warnedHosts;
|
|
10828
11203
|
try {
|
|
10829
|
-
await
|
|
11204
|
+
const { dest } = await downloadToFile(item.url, {
|
|
11205
|
+
dest: (response) => {
|
|
11206
|
+
const ext = extFromUrl4(item.url) ?? extFromContentType(response.headers.get("content-type")) ?? ".bin";
|
|
11207
|
+
return resolveDestination(predictionId, index, count, ext, opts);
|
|
11208
|
+
},
|
|
11209
|
+
timeoutMs: DOWNLOAD_TIMEOUT_MS,
|
|
11210
|
+
maxBytes: opts.maxBytes,
|
|
11211
|
+
fetch: opts.fetch,
|
|
11212
|
+
validateUrl: (url) => {
|
|
11213
|
+
const { host, trusted } = checkOutputUrl(url);
|
|
11214
|
+
if (!trusted && !warned.has(host)) {
|
|
11215
|
+
warned.add(host);
|
|
11216
|
+
opts.logger.warn(
|
|
11217
|
+
`Output ${item.path} is served from ${host}, not ${REPLICATE_DELIVERY_HOST}`
|
|
11218
|
+
);
|
|
11219
|
+
}
|
|
11220
|
+
}
|
|
11221
|
+
});
|
|
11222
|
+
return dest;
|
|
10830
11223
|
} catch (err2) {
|
|
10831
|
-
|
|
11224
|
+
if (err2 instanceof HttpError && (err2.status === 404 || err2.status === 410)) {
|
|
11225
|
+
throw new HttpError(err2.status, "download failed (output may have expired)", err2.rawUrl);
|
|
11226
|
+
}
|
|
10832
11227
|
throw err2;
|
|
10833
11228
|
}
|
|
10834
|
-
return dest;
|
|
10835
11229
|
}
|
|
10836
11230
|
async function saveMediaOutputs(predictionId, media, opts) {
|
|
11231
|
+
const withCap = { ...opts, maxBytes: maxDownloadBytes(), warnedHosts: /* @__PURE__ */ new Set() };
|
|
10837
11232
|
const saved = [];
|
|
10838
11233
|
const failed = [];
|
|
10839
11234
|
for (let i = 0; i < media.length; i++) {
|
|
@@ -10841,7 +11236,7 @@ async function saveMediaOutputs(predictionId, media, opts) {
|
|
|
10841
11236
|
if (!item) continue;
|
|
10842
11237
|
opts.logger.debug(`Downloading ${item.path} from ${redactUrl(item.url).slice(0, 200)}`);
|
|
10843
11238
|
try {
|
|
10844
|
-
const dest = await saveOne(item, predictionId, i, media.length,
|
|
11239
|
+
const dest = await saveOne(item, predictionId, i, media.length, withCap);
|
|
10845
11240
|
saved.push(dest);
|
|
10846
11241
|
opts.logger.success(`Saved ${dest}`);
|
|
10847
11242
|
} catch (e) {
|
|
@@ -10858,10 +11253,22 @@ async function saveMediaOutputs(predictionId, media, opts) {
|
|
|
10858
11253
|
}
|
|
10859
11254
|
|
|
10860
11255
|
// src/providers/replicate/commands/prediction-flow.ts
|
|
11256
|
+
var DEFAULT_PREFER_WAIT_SECONDS = 60;
|
|
11257
|
+
function parsePreferWaitSeconds(raw) {
|
|
11258
|
+
const n = Number(raw);
|
|
11259
|
+
if (!/^\d+$/.test(raw.trim()) || n < 1 || n > 60) {
|
|
11260
|
+
throw new InvalidArgumentError(
|
|
11261
|
+
`Expected an integer number of seconds from 1 to 60 (a bare --prefer-wait means ${DEFAULT_PREFER_WAIT_SECONDS}); put the prompt before the flag.`
|
|
11262
|
+
);
|
|
11263
|
+
}
|
|
11264
|
+
return n;
|
|
11265
|
+
}
|
|
10861
11266
|
function addFlowOptions(cmd, defaultTimeoutMs) {
|
|
10862
|
-
return cmd.option("--async", "Submit and return immediately with the prediction id").option("--wait", "Wait for a terminal status (default unless --async)").
|
|
10863
|
-
|
|
10864
|
-
|
|
11267
|
+
return cmd.option("--async", "Submit and return immediately with the prediction id").option("--wait", "Wait for a terminal status (default unless --async)").addOption(
|
|
11268
|
+
new Option2(
|
|
11269
|
+
"--prefer-wait [seconds]",
|
|
11270
|
+
"Ask Replicate to hold the create request open for 1-60s (bare flag: 60); keeps polling if still running"
|
|
11271
|
+
).argParser(parsePreferWaitSeconds).preset(String(DEFAULT_PREFER_WAIT_SECONDS))
|
|
10865
11272
|
).option("--wait-timeout <ms>", "Polling timeout in milliseconds", String(defaultTimeoutMs)).option("--download", "Download media outputs (default)").option("--no-download", "Print output URLs instead of downloading").option("--output <path>", "Output file path, or a directory for multiple outputs").option("-v, --verbose", "Verbose logging");
|
|
10866
11273
|
}
|
|
10867
11274
|
function parsePositiveInt2(raw, flag, max) {
|
|
@@ -10873,11 +11280,6 @@ function parsePositiveInt2(raw, flag, max) {
|
|
|
10873
11280
|
}
|
|
10874
11281
|
return n;
|
|
10875
11282
|
}
|
|
10876
|
-
function parsePreferWait(value) {
|
|
10877
|
-
if (value === void 0 || value === false) return void 0;
|
|
10878
|
-
if (value === true) return 60;
|
|
10879
|
-
return parsePositiveInt2(value, "--prefer-wait", 60);
|
|
10880
|
-
}
|
|
10881
11283
|
function resumeCommand3(id) {
|
|
10882
11284
|
return `multix replicate status ${id} --wait --download`;
|
|
10883
11285
|
}
|
|
@@ -10932,13 +11334,16 @@ async function executePrediction2(params) {
|
|
|
10932
11334
|
const { client, model, input, opts, logger } = params;
|
|
10933
11335
|
if (opts.async && opts.wait)
|
|
10934
11336
|
throw new ValidationError("--async and --wait are mutually exclusive");
|
|
10935
|
-
if (opts.async && opts.preferWait) {
|
|
11337
|
+
if (opts.async && opts.preferWait !== void 0) {
|
|
10936
11338
|
throw new ValidationError("--async and --prefer-wait are mutually exclusive");
|
|
10937
11339
|
}
|
|
10938
11340
|
const waitTimeoutMs = parsePositiveInt2(opts.waitTimeout, "--wait-timeout");
|
|
10939
|
-
const preferWaitSeconds = parsePreferWait(opts.preferWait);
|
|
10940
11341
|
logger.info(`Submitting to Replicate ${describeModelRef(model)}...`);
|
|
10941
|
-
const prediction2 = await client.createPrediction({
|
|
11342
|
+
const prediction2 = await client.createPrediction({
|
|
11343
|
+
model,
|
|
11344
|
+
input,
|
|
11345
|
+
preferWaitSeconds: opts.preferWait
|
|
11346
|
+
});
|
|
10942
11347
|
if (opts.async) {
|
|
10943
11348
|
printAsyncHandle(prediction2);
|
|
10944
11349
|
return;
|
|
@@ -11010,7 +11415,7 @@ import fs45 from "fs";
|
|
|
11010
11415
|
import path52 from "path";
|
|
11011
11416
|
var DATA_URI_MAX_BYTES = 256 * 1024;
|
|
11012
11417
|
var UPLOAD_MAX_BYTES = 100 * 1024 * 1024;
|
|
11013
|
-
function
|
|
11418
|
+
function formatMb3(bytes) {
|
|
11014
11419
|
return `${(bytes / (1024 * 1024)).toFixed(1)}MB`;
|
|
11015
11420
|
}
|
|
11016
11421
|
async function resolveFileInput(value, client, logger) {
|
|
@@ -11026,7 +11431,7 @@ async function resolveFileInput(value, client, logger) {
|
|
|
11026
11431
|
if (stat.size === 0) throw new ValidationError(`Input file is empty: ${value}`);
|
|
11027
11432
|
if (stat.size >= UPLOAD_MAX_BYTES) {
|
|
11028
11433
|
throw new ValidationError(
|
|
11029
|
-
`Input file ${value} is ${
|
|
11434
|
+
`Input file ${value} is ${formatMb3(stat.size)}; Replicate accepts uploads under 100MB. Host it and pass an https URL instead.`
|
|
11030
11435
|
);
|
|
11031
11436
|
}
|
|
11032
11437
|
const filename = path52.basename(filePath);
|
|
@@ -11036,7 +11441,7 @@ async function resolveFileInput(value, client, logger) {
|
|
|
11036
11441
|
logger.debug(`Inlining ${filename} (${stat.size} bytes) as a data URI`);
|
|
11037
11442
|
return `data:${contentType};base64,${bytes.toString("base64")}`;
|
|
11038
11443
|
}
|
|
11039
|
-
logger.info(`Uploading ${filename} (${
|
|
11444
|
+
logger.info(`Uploading ${filename} (${formatMb3(stat.size)}) to Replicate Files...`);
|
|
11040
11445
|
const file = await client.uploadFile({
|
|
11041
11446
|
data: new Blob([bytes], { type: contentType }),
|
|
11042
11447
|
filename,
|
|
@@ -11460,7 +11865,6 @@ function registerCheckCommand(program2) {
|
|
|
11460
11865
|
}
|
|
11461
11866
|
|
|
11462
11867
|
// src/commands/doc/convert.ts
|
|
11463
|
-
init_errors();
|
|
11464
11868
|
import fs47 from "fs";
|
|
11465
11869
|
import path53 from "path";
|
|
11466
11870
|
|
|
@@ -11478,27 +11882,45 @@ Requirements:
|
|
|
11478
11882
|
Output only the markdown content without any preamble or explanation.`;
|
|
11479
11883
|
|
|
11480
11884
|
// src/commands/doc/convert.ts
|
|
11885
|
+
function autoNameBase(input) {
|
|
11886
|
+
if (input === STDIN_ARG) return "stdin";
|
|
11887
|
+
return path53.basename(input, path53.extname(input));
|
|
11888
|
+
}
|
|
11481
11889
|
async function convertDocuments(opts) {
|
|
11482
|
-
const {
|
|
11890
|
+
const { output, autoName = false, verbose, logger } = opts;
|
|
11483
11891
|
const model = opts.model ?? DOC_MODEL_DEFAULT;
|
|
11484
11892
|
const prompt = opts.prompt ?? DEFAULT_CONVERT_PROMPT;
|
|
11485
|
-
|
|
11486
|
-
|
|
11487
|
-
|
|
11488
|
-
|
|
11489
|
-
|
|
11490
|
-
|
|
11491
|
-
}
|
|
11492
|
-
|
|
11493
|
-
|
|
11494
|
-
|
|
11495
|
-
|
|
11893
|
+
const { inputs, cleanup } = await resolveFileInputs(opts.inputs, {
|
|
11894
|
+
stdinExt: opts.stdinExt,
|
|
11895
|
+
input: opts.stdin,
|
|
11896
|
+
flag: "--input",
|
|
11897
|
+
example: "cat report.pdf | multix doc convert --input -",
|
|
11898
|
+
extExample: "pdf"
|
|
11899
|
+
});
|
|
11900
|
+
let failed;
|
|
11901
|
+
try {
|
|
11902
|
+
let outputPath;
|
|
11903
|
+
if (output) {
|
|
11904
|
+
outputPath = path53.resolve(output);
|
|
11905
|
+
} else if (autoName && opts.inputs.length === 1 && opts.inputs[0]) {
|
|
11906
|
+
outputPath = path53.join(getOutputDir(), `${autoNameBase(opts.inputs[0])}-extraction.md`);
|
|
11907
|
+
} else {
|
|
11908
|
+
outputPath = path53.join(getOutputDir(), "document-extraction.md");
|
|
11909
|
+
}
|
|
11910
|
+
fs47.mkdirSync(path53.dirname(outputPath), { recursive: true });
|
|
11911
|
+
failed = await convertResolved(inputs, { outputPath, model, prompt, verbose, logger });
|
|
11912
|
+
} finally {
|
|
11913
|
+
cleanup();
|
|
11496
11914
|
}
|
|
11497
|
-
|
|
11915
|
+
if (failed > 0) process.exit(1);
|
|
11916
|
+
}
|
|
11917
|
+
async function convertResolved(inputs, ctx) {
|
|
11918
|
+
const { outputPath, model, prompt, verbose, logger } = ctx;
|
|
11498
11919
|
const results = [];
|
|
11499
11920
|
for (let i = 0; i < inputs.length; i++) {
|
|
11500
|
-
const
|
|
11501
|
-
|
|
11921
|
+
const input = inputs[i];
|
|
11922
|
+
const filePath = input.path;
|
|
11923
|
+
logger?.debug(`[${i + 1}/${inputs.length}] Converting: ${input.label}`);
|
|
11502
11924
|
let markdown;
|
|
11503
11925
|
let errorMsg;
|
|
11504
11926
|
for (let attempt = 0; attempt < 3; attempt++) {
|
|
@@ -11530,7 +11952,7 @@ async function convertDocuments(opts) {
|
|
|
11530
11952
|
markdown = extractText(resp);
|
|
11531
11953
|
break;
|
|
11532
11954
|
} catch (e) {
|
|
11533
|
-
errorMsg = e instanceof Error ? e.message : String(e);
|
|
11955
|
+
errorMsg = labelError(e instanceof Error ? e.message : String(e), input);
|
|
11534
11956
|
if (isGeminiRateLimitError(e)) break;
|
|
11535
11957
|
if (attempt < 2) {
|
|
11536
11958
|
const wait = 2 ** attempt * 1e3;
|
|
@@ -11539,8 +11961,11 @@ async function convertDocuments(opts) {
|
|
|
11539
11961
|
}
|
|
11540
11962
|
}
|
|
11541
11963
|
}
|
|
11964
|
+
if (markdown === void 0) {
|
|
11965
|
+
logger?.error(`Failed: ${input.label} \u2014 ${errorMsg}`);
|
|
11966
|
+
}
|
|
11542
11967
|
results.push({
|
|
11543
|
-
file:
|
|
11968
|
+
file: input.label,
|
|
11544
11969
|
status: markdown !== void 0 ? "success" : "error",
|
|
11545
11970
|
markdown,
|
|
11546
11971
|
error: errorMsg
|
|
@@ -11554,7 +11979,7 @@ async function convertDocuments(opts) {
|
|
|
11554
11979
|
`);
|
|
11555
11980
|
for (const r of results) {
|
|
11556
11981
|
lines.push(`
|
|
11557
|
-
## ${path53.basename(r.file)}
|
|
11982
|
+
## ${path53.basename(r.file).replace(/[<>]/g, "\\$&")}
|
|
11558
11983
|
`);
|
|
11559
11984
|
if (r.status === "success" && r.markdown) {
|
|
11560
11985
|
lines.push(`${r.markdown}
|
|
@@ -11580,8 +12005,7 @@ ${"=".repeat(50)}`);
|
|
|
11580
12005
|
console.log(` [${r.status}] ${r.file}`);
|
|
11581
12006
|
}
|
|
11582
12007
|
}
|
|
11583
|
-
|
|
11584
|
-
if (failed > 0) process.exit(1);
|
|
12008
|
+
return results.length - succeeded;
|
|
11585
12009
|
}
|
|
11586
12010
|
function sleep4(ms) {
|
|
11587
12011
|
return new Promise((r) => setTimeout(r, ms));
|
|
@@ -11594,14 +12018,21 @@ function registerDocCommands(program2) {
|
|
|
11594
12018
|
);
|
|
11595
12019
|
doc.command("convert").description("Convert documents to Markdown using Gemini Files API").requiredOption(
|
|
11596
12020
|
"--input <files...>",
|
|
11597
|
-
"Input file(s) to convert",
|
|
12021
|
+
"Input file(s) to convert (use - to read piped stdin)",
|
|
11598
12022
|
(val, prev) => [...prev, val],
|
|
11599
12023
|
[]
|
|
11600
|
-
).alias("-i").option(
|
|
12024
|
+
).alias("-i").option(
|
|
12025
|
+
"--stdin-ext <ext>",
|
|
12026
|
+
"Extension for piped stdin when auto-detection fails (e.g. pdf, txt)"
|
|
12027
|
+
).option("--output <path>", "Output markdown file path").option(
|
|
12028
|
+
"--auto-name",
|
|
12029
|
+
"Derive output filename from input basename (single file only; stdin becomes stdin-extraction.md)"
|
|
12030
|
+
).option("--model <id>", `Gemini model to use (default: ${DOC_MODEL_DEFAULT})`).option("--prompt <text>", "Custom conversion prompt").option("-v, --verbose", "Verbose logging").action(
|
|
11601
12031
|
async (opts) => {
|
|
11602
12032
|
const logger = createLogger({ verbose: opts.verbose ?? false });
|
|
11603
12033
|
await convertDocuments({
|
|
11604
12034
|
inputs: opts.input,
|
|
12035
|
+
stdinExt: opts.stdinExt,
|
|
11605
12036
|
output: opts.output,
|
|
11606
12037
|
autoName: opts.autoName,
|
|
11607
12038
|
model: opts.model,
|
|
@@ -11623,13 +12054,13 @@ import fs48 from "fs";
|
|
|
11623
12054
|
import path54 from "path";
|
|
11624
12055
|
|
|
11625
12056
|
// src/commands/media/detect-type.ts
|
|
11626
|
-
var
|
|
11627
|
-
var
|
|
12057
|
+
var VIDEO_EXTS2 = /* @__PURE__ */ new Set([".mp4", ".mov", ".avi", ".mkv", ".webm", ".flv", ".mpg", ".mpeg"]);
|
|
12058
|
+
var AUDIO_EXTS2 = /* @__PURE__ */ new Set([".mp3", ".wav", ".m4a", ".flac", ".aac", ".ogg", ".aiff"]);
|
|
11628
12059
|
var IMAGE_EXTS2 = /* @__PURE__ */ new Set([".jpg", ".jpeg", ".png", ".webp", ".heic", ".heif", ".gif", ".bmp"]);
|
|
11629
12060
|
function extToKind(ext) {
|
|
11630
12061
|
const lower = ext.toLowerCase();
|
|
11631
|
-
if (
|
|
11632
|
-
if (
|
|
12062
|
+
if (VIDEO_EXTS2.has(lower)) return "video";
|
|
12063
|
+
if (AUDIO_EXTS2.has(lower)) return "audio";
|
|
11633
12064
|
if (IMAGE_EXTS2.has(lower)) return "image";
|
|
11634
12065
|
return void 0;
|
|
11635
12066
|
}
|
|
@@ -11805,7 +12236,7 @@ async function optimizeVideo(opts) {
|
|
|
11805
12236
|
}
|
|
11806
12237
|
|
|
11807
12238
|
// src/commands/media/batch.ts
|
|
11808
|
-
var ALL_EXTS = /* @__PURE__ */ new Set([...
|
|
12239
|
+
var ALL_EXTS = /* @__PURE__ */ new Set([...VIDEO_EXTS2, ...AUDIO_EXTS2, ...IMAGE_EXTS2]);
|
|
11809
12240
|
async function batchOptimize(opts) {
|
|
11810
12241
|
const {
|
|
11811
12242
|
inputDir,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mrgoonie/multix",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.14.1",
|
|
4
4
|
"description": "AI multimodal CLI — OpenAI, Gemini, MiniMax, OpenRouter, Leonardo, BytePlus, ElevenLabs, fal.ai, Atlas Cloud, Replicate, HeyGen, ffmpeg, ImageMagick, doc-to-md",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/skill/SKILL.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: multix
|
|
3
3
|
description: AI multimodal CLI — generate and edit images (OpenAI, Codex driver, Gemini Nano Banana, Imagen, MiniMax, OpenRouter, Leonardo, BytePlus Seedream, Cloudflare Workers AI FLUX.1 Schnell, fal.ai FLUX, Replicate FLUX), generate video (Gemini Veo, MiniMax Hailuo, OpenRouter via Veo/Seedance/Kling, Leonardo via Motion/Veo/Kling/Hailuo/Seedance/LTX, BytePlus Seedance, Cloudflare AI Gateway via Replicate, fal.ai queue models, Replicate direct API models incl. P-Video, HeyGen Video Agent and scripted avatar videos), video translation and lipsync (HeyGen), TTS/STT/music (OpenAI, Gemini Flash TTS incl. Gemini 3.8 with delivery-style control, MiniMax, Cloudflare Workers AI MeloTTS, ElevenLabs), 3D models (BytePlus Hyper3D/Hitem3d), analyze/transcribe media, convert documents to Markdown, optimize media via ffmpeg/ImageMagick, and self-update the CLI. Use whenever the user wants to create, edit, or transform images/video/audio/3D/documents from a CLI, mentions any of: OpenAI, Codex image driver, image-to-image, i2i, image edit, watercolor/cyberpunk style transfer, reference image, OpenRouter, Nano Banana, Flux, Seedream, Veo, Hailuo, Seedance, Kling, Cloudflare Workers AI, AI Gateway, fal.ai, Atlas Cloud, Replicate, replicate.com models, Replicate predictions, HeyGen, avatar video, talking head, Video Agent, video translation, dubbing, lipsync, video generation, text-to-video, image-to-video, i2v, ElevenLabs voice cloning, updating/upgrading multix, or asks for batch media optimization.
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.14.1 # x-release-please-version
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# multix — AI Multimodal CLI
|
|
@@ -178,7 +178,9 @@ multix replicate status <prediction-id> --wait --download
|
|
|
178
178
|
multix replicate cancel <prediction-id>
|
|
179
179
|
```
|
|
180
180
|
|
|
181
|
-
Shared flags: `--async`, `--wait`, `--prefer-wait [seconds]` (sends `Prefer: wait=N`, still polls if unfinished), `--wait-timeout <ms>`, `--no-download` (print URLs), `--output <path|dir/>`. `failed`/`canceled`/`aborted` predictions exit non-zero. Local files up to 256 KB are inlined as data URIs; larger files (< 100 MB) go through the Replicate Files API.
|
|
181
|
+
Shared flags: `--async`, `--wait`, `--prefer-wait [seconds]` (integer 1-60, bare flag = 60; sends `Prefer: wait=N`, still polls if unfinished; put the prompt before a bare flag), `--wait-timeout <ms>`, `--no-download` (print URLs), `--output <path|dir/>`. `failed`/`canceled`/`aborted` predictions exit non-zero. Local files up to 256 KB are inlined as data URIs; larger files (< 100 MB) go through the Replicate Files API.
|
|
182
|
+
|
|
183
|
+
Output downloads are `https://` only; `localhost` and loopback/private/link-local/special-purpose IP literals (including redirects; hostnames are not resolved) are refused, non-`replicate.delivery` hosts are downloaded with a warning, and each file is capped at `MULTIX_MAX_DOWNLOAD_MB` (default 2048). If submitting fails with a network error, timeout, 5xx, or an unreadable 2xx response, the prediction may still exist and bill: check https://replicate.com/predictions before retrying instead of resubmitting blindly.
|
|
182
184
|
|
|
183
185
|
## HeyGen
|
|
184
186
|
|
|
@@ -194,7 +196,7 @@ multix heygen video-agent "45-second explainer for a note app" [--avatar <look-i
|
|
|
194
196
|
multix heygen avatars [--type photo_avatar] [--cursor <token>]
|
|
195
197
|
multix heygen voices [--language English]
|
|
196
198
|
multix heygen video "Hello!" --avatar <look-id> [--voice <id>] [--aspect-ratio 9:16] [--engine avatar_iv|avatar_v|avatar_iii] [--burn-captions]
|
|
197
|
-
multix heygen translate <ref> --languages "Spanish, Chinese (Cantonese, Traditional)" [--output <dir>]
|
|
199
|
+
multix heygen translate <ref> --languages "Spanish, Chinese (Cantonese, Traditional)" [--output <dir|dub.mp4>] # several languages: dub.mp4 -> dub-Spanish.mp4, ...
|
|
198
200
|
multix heygen lipsync --video <ref> --audio <ref>
|
|
199
201
|
multix heygen tts "Hello" --voice <voice-id>
|
|
200
202
|
multix heygen template <template-id> --input @vars.json
|
|
@@ -354,6 +356,7 @@ multix byteplus 3d --input-image ./front.png ./side.png --prompt "Generate from
|
|
|
354
356
|
|
|
355
357
|
# Docs → Markdown (Gemini)
|
|
356
358
|
multix doc convert --input report.pdf [--auto-name] [--output ./out.md]
|
|
359
|
+
cat report.pdf | multix doc convert --input - --output ./out.md # "-" reads stdin; --stdin-ext <ext> (e.g. pdf) if undetected
|
|
357
360
|
|
|
358
361
|
# Media optimization (ffmpeg + ImageMagick)
|
|
359
362
|
multix media optimize --input <file> --output <file> [--target-size <MB>] [--quality 85] [--max-width 1920]
|
|
@@ -383,6 +386,7 @@ Requires `cwebp` (libwebp: `brew install webp`, `apt install webp`) on `PATH`.
|
|
|
383
386
|
| Variable | Purpose |
|
|
384
387
|
|---|---|
|
|
385
388
|
| `MULTIX_OUTPUT_DIR` | Override default output dir |
|
|
389
|
+
| `MULTIX_MAX_DOWNLOAD_MB` | Size limit per downloaded Replicate output file (default 2048) |
|
|
386
390
|
| `OPENAI_IMAGE_MODEL` / `OPENAI_TTS_MODEL` / `OPENAI_STT_MODEL` | OpenAI model defaults |
|
|
387
391
|
| `OPENROUTER_IMAGE_MODEL` | Default OpenRouter image model |
|
|
388
392
|
| `OPENROUTER_FALLBACK_MODELS` | CSV; applies to `generate` AND `i2i` |
|