nanoodle 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -2
- package/bin/nanoodle.mjs +7 -1
- package/package.json +1 -1
- package/src/browser.mjs +1 -1
- package/src/catalog.mjs +19 -0
- package/src/cleanvoice.mjs +104 -0
- package/src/client.mjs +66 -8
- package/src/decide.mjs +160 -0
- package/src/endpoint.mjs +406 -0
- package/src/estimate.mjs +141 -20
- package/src/graph.mjs +12 -2
- package/src/index.mjs +4 -0
- package/src/io.mjs +28 -0
- package/src/local-media.mjs +38 -0
- package/src/media.mjs +3 -0
- package/src/nodes.mjs +184 -18
- package/src/prompt-caps.mjs +169 -0
- package/src/workflow.mjs +96 -7
package/README.md
CHANGED
|
@@ -170,13 +170,68 @@ Failures in lanes no output depends on only appear in `result.errors`.
|
|
|
170
170
|
Unknown node types, missing required inputs, bad keys, and a missing API key
|
|
171
171
|
all fail **before** anything is spent.
|
|
172
172
|
|
|
173
|
+
### Gates (Decide said no)
|
|
174
|
+
|
|
175
|
+
A ⚖️ `decide` node in yes/no mode with **gate** on stops its branch when the
|
|
176
|
+
answer is no. That is a result, not an error, so `run()` resolves:
|
|
177
|
+
|
|
178
|
+
```js
|
|
179
|
+
result.gated
|
|
180
|
+
// [{ nodeId: "n4", name: "Decide", yes: 0.12, skipped: ["n5", "n6"],
|
|
181
|
+
// message: "gate closed — the answer was no (yes 12%), so nothing downstream ran" }]
|
|
182
|
+
result.nodes.n4 // { status: "gated", out: { text: "no", decision }, gate: { yes, message }, costUsd, ... }
|
|
183
|
+
result.nodes.n5 // { status: "skipped", gatedBy: "n4", costUsd: null, ... } — never ran, never billed
|
|
184
|
+
result.get("Image") // throws NanoodleError code "gated": the gate "Decide" answered no
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
`result.errors` stays empty, progress emits `node-gated` / `node-skipped`, and
|
|
188
|
+
the CLI exits 0 with `gated` in its JSON summary. A node fed by both a closed
|
|
189
|
+
gate and a real failure is still an error.
|
|
190
|
+
|
|
191
|
+
### Prompt length caps
|
|
192
|
+
|
|
193
|
+
Many image and video models reject an over-long prompt at NanoGPT's route
|
|
194
|
+
(`400 prompt_too_long`) — `qwen-image-3` stops at 800 characters,
|
|
195
|
+
`step-image-edit-2` at 512, 45 of 183 probed image models cap somewhere. The
|
|
196
|
+
catalogs don't advertise any of it, and in a graph the prompt is written by an
|
|
197
|
+
upstream LLM, so there's nothing for a caller to shorten. So `run()`:
|
|
198
|
+
|
|
199
|
+
1. **trims** a prompt over the cap, at a sentence boundary (never mid-word,
|
|
200
|
+
never below 70% of the cap), rather than sending a request certain to fail;
|
|
201
|
+
2. **reports** every trim — a `prompt-trimmed` progress event
|
|
202
|
+
(`{ nodeId, name, from, to, cap }`) and a process warning. Never silent;
|
|
203
|
+
3. **learns** a cap from a live 400 and keeps it on the `Workflow` instance, so
|
|
204
|
+
a retry of an unknown or newly-added model trims up front instead of failing
|
|
205
|
+
again.
|
|
206
|
+
|
|
207
|
+
It does **not** rewrite the graph's own prompts to avoid a trim: an `llm` node's
|
|
208
|
+
system prompt goes to the model exactly as its author wrote it. Expect the trim
|
|
209
|
+
to fire routinely on tight models — measured against five chat models writing an
|
|
210
|
+
image prompt, all five overshot an 800-char cap and the trim kept 29–62%. Handle
|
|
211
|
+
`prompt-trimmed` if that matters to you.
|
|
212
|
+
|
|
213
|
+
Audio caps come from `supported_parameters.max_chars` when you pass a
|
|
214
|
+
[catalog](#supported-nodes); image/video caps come from a probed table. A model
|
|
215
|
+
with no known cap is left completely alone.
|
|
216
|
+
|
|
217
|
+
The pieces are exported if you orchestrate graphs yourself:
|
|
218
|
+
`promptCap`, `fitPromptText`, `isPromptTooLong`, `promptCapFromError`,
|
|
219
|
+
`PROMPT_CAPS`.
|
|
220
|
+
|
|
173
221
|
## Supported nodes
|
|
174
222
|
|
|
175
223
|
| runs | node types |
|
|
176
224
|
|---|---|
|
|
177
|
-
| local | text, upload (image/audio/video), choice, join, comment |
|
|
225
|
+
| local | text, upload (image/audio/video), mupload (3D .glb), choice, join, comment |
|
|
178
226
|
| local media† | resize, vframes, combine, soundtrack, trim, extractaudio |
|
|
179
|
-
| NanoGPT | llm (incl. vision + audio input), image, edit, inpaint*, vision, tvideo, ivideo, vedit, lipsync, music, remix, tts, transcribe |
|
|
227
|
+
| NanoGPT | llm (incl. vision + audio input), image, edit, inpaint*, vision, decide†, tvideo, ivideo, model3d‡, vedit, lipsync, music, remix, tts, transcribe, cleanvoice§ |
|
|
228
|
+
| custom URL | endpoint (POST to a URL in the graph; no NanoGPT key) |
|
|
229
|
+
|
|
230
|
+
† `decide` asks a NanoGPT decision model one typed question (`POST /api/v1/decisions`): pick the best of the wired `img1…` images, choose a label, score on a scale, or yes/no. Text-only decisions need nothing extra; wired images are shrunk to the model's limits with ffmpeg. A yes/no gate that answers no is not a failure: that node settles as `gated`, everything downstream is `skipped` unbilled, and the run succeeds (see [Gates](#gates-decide-said-no)).
|
|
231
|
+
|
|
232
|
+
‡ `model3d` turns an image and/or a prompt into a GLB (`POST /api/generate-video`, then `GET /api/video/status`). The output port is `model`. A blank model id uses `tripo3d/v2.5` (image only). Polling stops at 25 minutes.
|
|
233
|
+
|
|
234
|
+
§ `cleanvoice` strips noise and music from hosted audio or video (`POST /api/v1/audio/speech` with `elevenlabs/audio-isolation` or `veed/clean-audio`, then `GET /api/tts/status`). The source must be a public `http(s)` URL — `data:` and `blob:` are refused before any request. Priced from the source length (default model is $0.121/min).
|
|
180
235
|
|
|
181
236
|
† **local media** prefers a pure-JS path that matches the browser (lossless mp4 remux, PCM-WAV trim, PNG resize). **ffmpeg** on `PATH` is the fallback for everything else (soft dependency — not an npm package); clear error if it’s required and missing.
|
|
182
237
|
|
package/bin/nanoodle.mjs
CHANGED
|
@@ -242,6 +242,8 @@ then: NANOGPT_API_KEY=... nanoodle run ${dest} --input Text="your idea"`);
|
|
|
242
242
|
if (e.type === "node-start") console.error(`▶ ${e.name} (${e.nodeId})`);
|
|
243
243
|
if (e.type === "node-done") console.error(`✔ ${e.name} (${e.nodeId}) ${e.ms}ms${e.costUsd != null ? ` $${e.costUsd}` : ""}`);
|
|
244
244
|
if (e.type === "node-error") console.error(`✖ ${e.name} (${e.nodeId}): ${e.error}`);
|
|
245
|
+
if (e.type === "node-gated") console.error(`⛔ ${e.name} (${e.nodeId}) gated: ${e.message}${e.costUsd != null ? ` $${e.costUsd}` : ""}`);
|
|
246
|
+
if (e.type === "node-skipped") console.error(`⤼ ${e.name} (${e.nodeId}) skipped — gate ${e.gatedBy} said no`);
|
|
245
247
|
},
|
|
246
248
|
});
|
|
247
249
|
} catch (e) {
|
|
@@ -284,7 +286,11 @@ then: NANOGPT_API_KEY=... nanoodle run ${dest} --input Text="your idea"`);
|
|
|
284
286
|
costExact: result.costExact,
|
|
285
287
|
remainingBalance: result.remainingBalance,
|
|
286
288
|
errors: result.errors,
|
|
287
|
-
|
|
289
|
+
gated: result.gated || [],
|
|
290
|
+
nodes: Object.fromEntries(Object.entries(result.nodes).map(([id, r]) => [id, {
|
|
291
|
+
status: r.status, ms: r.ms, costUsd: r.costUsd, error: r.error,
|
|
292
|
+
...(r.gate ? { gate: r.gate } : {}), ...(r.gatedBy ? { gatedBy: r.gatedBy } : {}),
|
|
293
|
+
}])),
|
|
288
294
|
}, null, 2));
|
|
289
295
|
if (!quiet) {
|
|
290
296
|
const approx = result.costExact ? "" : "≥ ";
|
package/package.json
CHANGED
package/src/browser.mjs
CHANGED
|
@@ -29,7 +29,7 @@ export {
|
|
|
29
29
|
} from "./media.mjs";
|
|
30
30
|
export { NanoClient, httpError, costFromJson, costFromHeaders, costWithHeaders, sleep } from "./client.mjs";
|
|
31
31
|
export { NODE_TYPES, displayName, materialize, topoSort, wiredFramesFloor, MAX_FRAMES } from "./graph.mjs";
|
|
32
|
-
export { RUNNERS } from "./nodes.mjs"; // per-node executors — the play delegation shim drives these directly
|
|
32
|
+
export { RUNNERS, IMG_INPUT_ROLES } from "./nodes.mjs"; // per-node executors — the play delegation shim drives these directly
|
|
33
33
|
export { catItem, chatModelCan } from "./catalog.mjs";
|
|
34
34
|
export { deriveInputs, deriveOutputs, deriveSettings, INPUT_SPECS, SETTING_SPECS } from "./io.mjs";
|
|
35
35
|
export { decodeShareUrl, decodeShareFragment, isShareRef } from "./share.mjs";
|
package/src/catalog.mjs
CHANGED
|
@@ -15,6 +15,25 @@ export function catItem(catalog, kind, id) {
|
|
|
15
15
|
return (Array.isArray(raw) && raw.find((m) => m && m.id === id)) || null;
|
|
16
16
|
}
|
|
17
17
|
|
|
18
|
+
/**
|
|
19
|
+
* True when a video model's PRICING block prices reference images even though its
|
|
20
|
+
* supported_parameters never names the param (minimax-h3 ships that way). A billing
|
|
21
|
+
* line for refs is the catalog stating the model takes them, so the ref gates accept
|
|
22
|
+
* it as evidence; models with neither the param nor the pricing stay "no refs",
|
|
23
|
+
* because an ignored-but-sent ref array is still charged.
|
|
24
|
+
*
|
|
25
|
+
* Also true for a billed `reference_to_video` MODE under per_second_by_mode or
|
|
26
|
+
* per_second_by_mode_and_resolution (Gemini Omni Flash v1 / v1.1) — same class of
|
|
27
|
+
* evidence as extra_reference_image. Exact key only; kling-o1's
|
|
28
|
+
* reference_to_video_image / _video stay on the mode gate (different shape).
|
|
29
|
+
*/
|
|
30
|
+
export function pricingAdvertisesRefs(pricing) {
|
|
31
|
+
if (!pricing) return false;
|
|
32
|
+
if (pricing.included_reference_images != null || pricing.extra_reference_image != null) return true;
|
|
33
|
+
const mm = pricing.per_second_by_mode || pricing.per_second_by_mode_and_resolution;
|
|
34
|
+
return !!(mm && mm.reference_to_video != null);
|
|
35
|
+
}
|
|
36
|
+
|
|
18
37
|
/** Permissive capability probe: true unless the model is in the catalog AND lacks the flag. */
|
|
19
38
|
export function chatModelCan(catalog, model, flag) {
|
|
20
39
|
const m = catItem(catalog, "chat", model);
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* 🎧 Clean voice — strip background noise / music, keep the voice.
|
|
3
|
+
* Twin of the editor/play block (nanoodle #709). NanoGPT's noise_reduction
|
|
4
|
+
* models (ElevenLabs Audio Isolation, VEED Clean Audio) ride
|
|
5
|
+
* POST /api/v1/audio/speech with { model, audio:<public URL>, duration }
|
|
6
|
+
* and answer 202 + runId (charged up front) → GET /api/tts/status → hosted audio.
|
|
7
|
+
* The source MUST be a public http(s) URL: data: and blob: are refused before
|
|
8
|
+
* any request. duration is only a quote; the server re-measures and bills that.
|
|
9
|
+
*/
|
|
10
|
+
import { NanoodleError } from "./errors.mjs";
|
|
11
|
+
|
|
12
|
+
export const CLEANVOICE_DEFAULT_MODEL = "elevenlabs/audio-isolation"; // $0.121/min
|
|
13
|
+
export const CLEANVOICE_FALLBACK_SECS = 60; // length unknown → quote a minute
|
|
14
|
+
const CLEANVOICE_VIDEO_EXT = /\.(mp4|m4v|mov|webm|mkv|avi)(\?|#|$)/i;
|
|
15
|
+
|
|
16
|
+
/** Which file to clean: a wired audio clip, else a wired video, else the pasted link. */
|
|
17
|
+
export function cleanVoiceSource(inp, f) {
|
|
18
|
+
const a = inp && typeof inp.audio === "string" ? inp.audio.trim() : "";
|
|
19
|
+
const v = inp && typeof inp.video === "string" ? inp.video.trim() : "";
|
|
20
|
+
const link = String((f && f.url) || "").trim();
|
|
21
|
+
const url = a || v || link;
|
|
22
|
+
const video = a ? false : (v ? true : CLEANVOICE_VIDEO_EXT.test(link));
|
|
23
|
+
if (!url) throw new NanoodleError("no audio — wire audio or a video in, or paste a public link to the recording");
|
|
24
|
+
if (!/^https?:\/\//i.test(url)) {
|
|
25
|
+
throw new NanoodleError(
|
|
26
|
+
"Clean voice needs a hosted file — NanoGPT downloads it from a public link, and a clip uploaded or made in your browser has none. Wire a generated video or track, or paste a public https link to the recording.");
|
|
27
|
+
}
|
|
28
|
+
return { url, video };
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/** Duration quote NanoGPT requires (0.6–3600 s); unknown → CLEANVOICE_FALLBACK_SECS. */
|
|
32
|
+
export function cleanVoiceSeconds(secs) {
|
|
33
|
+
const s = +secs;
|
|
34
|
+
if (!isFinite(s) || s <= 0) return CLEANVOICE_FALLBACK_SECS;
|
|
35
|
+
return Math.min(3600, Math.max(0.6, Math.round(s * 100) / 100));
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
export function cleanVoiceExtra(url, secs) {
|
|
39
|
+
return { audio: url, duration: cleanVoiceSeconds(secs) };
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
/** Seconds the cost forecast assumes: the wired source node's duration knob, else 30 s. */
|
|
43
|
+
export function cleanVoiceEstSeconds(srcFields) {
|
|
44
|
+
const d = parseFloat(srcFields && srcFields.duration);
|
|
45
|
+
return (isFinite(d) && d > 0) ? d : 30;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/** NanoGPT's 400s, reworded into the fix. Unrelated errors (and aborts) pass through. */
|
|
49
|
+
export function cleanVoiceError(e) {
|
|
50
|
+
if (e && (e.name === "AbortError" || e.code === "aborted")) return e;
|
|
51
|
+
const m = String((e && e.message) || e || "");
|
|
52
|
+
if (/verify the source (audio )?duration/i.test(m)) {
|
|
53
|
+
return new NanoodleError("NanoGPT couldn't read this file's length — use MP3, WAV, M4A/AAC or MP4 (OGG isn't accepted).");
|
|
54
|
+
}
|
|
55
|
+
if (/unable to download/i.test(m)) {
|
|
56
|
+
return new NanoodleError("NanoGPT couldn't download the file — the link must be public (no sign-in, not expired).");
|
|
57
|
+
}
|
|
58
|
+
if (/public http\(?s?\)? source/i.test(m)) {
|
|
59
|
+
return new NanoodleError("Clean voice needs a public https link — clips uploaded in the browser can't reach it.");
|
|
60
|
+
}
|
|
61
|
+
return e;
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* The source's length from media metadata. Browsers can read it without CORS;
|
|
66
|
+
* Node has no media element, so this resolves null and the quote falls back to
|
|
67
|
+
* CLEANVOICE_FALLBACK_SECS (the server re-measures before billing).
|
|
68
|
+
*/
|
|
69
|
+
export function cleanVoiceMediaSeconds(url, isVideo) {
|
|
70
|
+
return new Promise((res) => {
|
|
71
|
+
const doc = globalThis.document;
|
|
72
|
+
if (!doc || typeof doc.createElement !== "function") { res(null); return; }
|
|
73
|
+
let el = null;
|
|
74
|
+
let done = false;
|
|
75
|
+
const fin = (v) => {
|
|
76
|
+
if (done) return;
|
|
77
|
+
done = true;
|
|
78
|
+
try { el.removeAttribute("src"); el.load(); } catch { /* element may already be gone */ }
|
|
79
|
+
res(v);
|
|
80
|
+
};
|
|
81
|
+
try { el = doc.createElement(isVideo ? "video" : "audio"); }
|
|
82
|
+
catch { res(null); return; }
|
|
83
|
+
el.preload = "metadata";
|
|
84
|
+
el.muted = true;
|
|
85
|
+
el.onloadedmetadata = () => {
|
|
86
|
+
const d = el.duration;
|
|
87
|
+
fin(isFinite(d) && d > 0 ? d : null);
|
|
88
|
+
};
|
|
89
|
+
el.onerror = () => fin(null);
|
|
90
|
+
setTimeout(() => fin(null), 10000);
|
|
91
|
+
el.src = url;
|
|
92
|
+
});
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/** send(model, extra) → audio URL (the runtime's audio submit + poll). */
|
|
96
|
+
export async function cleanVoiceRun(model, inp, f, send) {
|
|
97
|
+
const src = cleanVoiceSource(inp, f);
|
|
98
|
+
const secs = await cleanVoiceMediaSeconds(src.url, src.video);
|
|
99
|
+
try {
|
|
100
|
+
return { audio: await send(model || CLEANVOICE_DEFAULT_MODEL, cleanVoiceExtra(src.url, secs)) };
|
|
101
|
+
} catch (e) {
|
|
102
|
+
throw cleanVoiceError(e);
|
|
103
|
+
}
|
|
104
|
+
}
|
package/src/client.mjs
CHANGED
|
@@ -3,6 +3,10 @@ import { b64ImageMime, bytesToDataUrl, dataUrlBytes, MEDIA_INLINE_MAX } from "./
|
|
|
3
3
|
import { assertPaymentOption, parseNanoInvoice, looksLikeResult } from "./x402.mjs";
|
|
4
4
|
|
|
5
5
|
const AUDIO_MIME = { mp3: "audio/mpeg", opus: "audio/ogg", aac: "audio/aac", flac: "audio/flac", wav: "audio/wav", pcm: "audio/wav" };
|
|
6
|
+
// Native music endpoints use prompt for musical direction; lyrics remain separate.
|
|
7
|
+
const AUDIO_PROMPT_MODEL_RE = /(?:^|\/)(?:prompt-to-song|generate-bgm)$|(?:^|\/)mureka-ai\/[^/]+\/generate-song$|^minimax\/music-3$/i;
|
|
8
|
+
// Yue2 catalogs take `style` + `lyrics` (+ `audio` for music-to-music), not OpenAI-style `input`.
|
|
9
|
+
const AUDIO_STYLE_MODEL_RE = /yue2-3b\/(?:text|music)-to-music$/i;
|
|
6
10
|
|
|
7
11
|
/** Map an HTTP failure to an actionable error (mirrors the app's httpRunError). Never leaks the key. */
|
|
8
12
|
export function httpError(status, bodyText) {
|
|
@@ -75,6 +79,22 @@ export function sleep(ms, signal) {
|
|
|
75
79
|
* NanoGPT transport. `baseUrl` and `fetch` are injectable (that's how the offline test harness runs);
|
|
76
80
|
* pollIntervals / timeouts are per-media-kind knobs (ms).
|
|
77
81
|
*/
|
|
82
|
+
/**
|
|
83
|
+
* GLB URL from a 3D status poll. A completed payload is 3D only when
|
|
84
|
+
* output.kind is "3d" or output.format is "glb" — a bare video url is not a model.
|
|
85
|
+
* Prefers model_url, then model.url, video.url, then videoUrls[0].
|
|
86
|
+
*/
|
|
87
|
+
export function model3dStatusUrl(s) {
|
|
88
|
+
const out = (s && s.data && s.data.output) || (s && s.output) || {};
|
|
89
|
+
const kind = String(out.kind || "").toLowerCase();
|
|
90
|
+
const format = String(out.format || "").toLowerCase();
|
|
91
|
+
if (kind !== "3d" && format !== "glb") return "";
|
|
92
|
+
const list = Array.isArray(out.videoUrls) ? out.videoUrls : [];
|
|
93
|
+
const first = list.length ? list[0] : null;
|
|
94
|
+
const fromList = typeof first === "string" ? first : (first && first.url) || "";
|
|
95
|
+
return out.model_url || (out.model && out.model.url) || (out.video && out.video.url) || fromList || "";
|
|
96
|
+
}
|
|
97
|
+
|
|
78
98
|
export class NanoClient {
|
|
79
99
|
constructor({ apiKey, baseUrl = "https://nano-gpt.com", fetch = globalThis.fetch, pollIntervals = {}, timeouts = {}, payment } = {}) {
|
|
80
100
|
// non-enumerable: console.log/util.inspect/JSON.stringify of a client (or a Workflow holding
|
|
@@ -85,7 +105,11 @@ export class NanoClient {
|
|
|
85
105
|
this.baseUrl = String(baseUrl).replace(/\/+$/, "");
|
|
86
106
|
this.fetch = fetch;
|
|
87
107
|
this.pollIntervals = { video: 5000, audio: 3000, x402: 3000, ...pollIntervals };
|
|
88
|
-
|
|
108
|
+
// video: no default deadline — NanoGPT jobs keep running server-side and long renders
|
|
109
|
+
// routinely exceed 10 min. Pass timeouts.video (ms) to cap a headless/CI run. Audio keeps
|
|
110
|
+
// a 5-min default (shorter jobs; still overridable). 3D stops at 25 min (the editor's
|
|
111
|
+
// MODEL3D_DEADLINE_MS); the job may still be running after that.
|
|
112
|
+
this.timeouts = { video: Infinity, audio: 300000, model3d: 25 * 60 * 1000, ...timeouts };
|
|
89
113
|
}
|
|
90
114
|
|
|
91
115
|
_auth() {
|
|
@@ -196,6 +220,19 @@ export class NanoClient {
|
|
|
196
220
|
return out;
|
|
197
221
|
}
|
|
198
222
|
|
|
223
|
+
/**
|
|
224
|
+
* POST /api/v1/decisions (⚖️ Decide). Returns the JSON ({ answers, usage:{ input_tokens, output_tokens, cost } }).
|
|
225
|
+
* Decision models bill input tokens only; the real price rides usage.cost.
|
|
226
|
+
*/
|
|
227
|
+
async decide(body, { onCost, signal } = {}) {
|
|
228
|
+
const r = await this._postJson("/api/v1/decisions", body, signal);
|
|
229
|
+
if (!r.ok) throw httpError(r.status, await r.text());
|
|
230
|
+
const j = await r.json();
|
|
231
|
+
const c = j && j.usage && typeof j.usage.cost === "number" ? j.usage.cost : null;
|
|
232
|
+
if (onCost) onCost(costWithHeaders(c != null ? { ...j, cost: c } : j, r));
|
|
233
|
+
return j;
|
|
234
|
+
}
|
|
235
|
+
|
|
199
236
|
/** POST /v1/images/generations (NOTE: not /api/v1). Returns data: / https URL(s). */
|
|
200
237
|
async image({ prompt, model, size, imageDataUrl, maskDataUrl, extra, n = 1, multi = false }, { onCost, signal } = {}) {
|
|
201
238
|
const body = { model, size: size || "1024x1024", n, response_format: "b64_json" };
|
|
@@ -220,7 +257,10 @@ export class NanoClient {
|
|
|
220
257
|
// io.resume (a runId) skips the submit — and its charge — and goes straight to
|
|
221
258
|
// polling: how a timed-out job is picked back up without paying twice.
|
|
222
259
|
async video(model, prompt, opts = {}, imageDataUrl, { onCost, onPoll, onRunId, resume, signal } = {}) {
|
|
223
|
-
const
|
|
260
|
+
const is3d = opts.mediaKind === "model3d";
|
|
261
|
+
const body = { model };
|
|
262
|
+
// 3D omits an empty prompt (image-only models). Video still sends a blank prompt.
|
|
263
|
+
if (!(is3d && !String(prompt || "").trim())) body.prompt = prompt;
|
|
224
264
|
if (opts.duration) body.duration = opts.duration;
|
|
225
265
|
if (opts.aspect_ratio) body.aspect_ratio = opts.aspect_ratio;
|
|
226
266
|
if (opts.resolution) body.resolution = opts.resolution;
|
|
@@ -256,28 +296,38 @@ export class NanoClient {
|
|
|
256
296
|
}
|
|
257
297
|
|
|
258
298
|
const t0 = Date.now();
|
|
259
|
-
|
|
260
|
-
|
|
299
|
+
const videoCap = is3d ? this.timeouts.model3d : this.timeouts.video;
|
|
300
|
+
const pollEvery = is3d ? (this.pollIntervals.model3d || this.pollIntervals.video) : this.pollIntervals.video;
|
|
301
|
+
// Infinity/non-finite = wait forever (default). Finite ms = headless/CI cap.
|
|
302
|
+
while (!Number.isFinite(videoCap) || Date.now() - t0 < videoCap) {
|
|
303
|
+
await sleep(pollEvery, signal);
|
|
261
304
|
let s;
|
|
262
305
|
try {
|
|
263
306
|
s = await (await this._get("/api/video/status?requestId=" + encodeURIComponent(runId), signal)).json();
|
|
264
307
|
} catch (e) {
|
|
265
308
|
if (signal && signal.aborted) throw abortError(signal.reason);
|
|
266
|
-
continue; // transient poll failure — keep polling until
|
|
309
|
+
continue; // transient poll failure — keep polling until complete or optional cap
|
|
267
310
|
}
|
|
268
311
|
const st = String((s.data && s.data.status) || s.status || "").toUpperCase();
|
|
269
312
|
if (onPoll) onPoll({ status: st, elapsedMs: Date.now() - t0, runId });
|
|
270
313
|
if (st === "COMPLETED" || st === "SUCCEEDED") {
|
|
314
|
+
if (is3d) {
|
|
315
|
+
const url = model3dStatusUrl(s);
|
|
316
|
+
if (!url) throw new NanoodleError("completed but no model url");
|
|
317
|
+
return url;
|
|
318
|
+
}
|
|
271
319
|
const out = (s.data && s.data.output) || s.output || {};
|
|
272
320
|
const url = (out.video && out.video.url) || out.url || (Array.isArray(out.video) ? out.video[0] && out.video[0].url : null);
|
|
273
321
|
if (!url) throw new NanoodleError("completed but no video url");
|
|
274
322
|
return url;
|
|
275
323
|
}
|
|
276
324
|
if (["FAILED", "ERROR", "CANCELED"].includes(st)) {
|
|
277
|
-
|
|
325
|
+
const why = (s.data && s.data.error) || st;
|
|
326
|
+
throw new NanoodleError((is3d ? "3D failed: " : "video failed: ") + why);
|
|
278
327
|
}
|
|
279
328
|
}
|
|
280
|
-
|
|
329
|
+
const label = is3d ? "3D" : "video";
|
|
330
|
+
throw new NanoodleError(`${label} timed out (${Math.round(videoCap / 1000)}s) — the job may still be running on NanoGPT's side`, { code: "timeout" });
|
|
281
331
|
}
|
|
282
332
|
|
|
283
333
|
/**
|
|
@@ -293,6 +343,14 @@ export class NanoClient {
|
|
|
293
343
|
return url;
|
|
294
344
|
}
|
|
295
345
|
const body = Object.assign({ model, input }, extra);
|
|
346
|
+
if (AUDIO_PROMPT_MODEL_RE.test(String(model || "").trim())) {
|
|
347
|
+
if (body.prompt == null || String(body.prompt).trim() === "") body.prompt = input;
|
|
348
|
+
delete body.input;
|
|
349
|
+
} else if (AUDIO_STYLE_MODEL_RE.test(String(model || "").trim())) {
|
|
350
|
+
if (body.style == null || String(body.style).trim() === "") body.style = input;
|
|
351
|
+
delete body.input;
|
|
352
|
+
delete body.prompt;
|
|
353
|
+
}
|
|
296
354
|
const r = await this._postJson("/api/v1/audio/speech", body, signal);
|
|
297
355
|
if (!r.ok) throw httpError(r.status, await r.text());
|
|
298
356
|
const ct = (r.headers && r.headers.get && r.headers.get("content-type")) || "";
|
|
@@ -312,7 +370,7 @@ export class NanoClient {
|
|
|
312
370
|
const bytes = new Uint8Array(await r.arrayBuffer());
|
|
313
371
|
let mime = ct.split(";")[0].trim().toLowerCase();
|
|
314
372
|
if (!mime || mime === "application/octet-stream" || mime === "binary/octet-stream") {
|
|
315
|
-
mime = AUDIO_MIME[extra.response_format || "mp3"] || "audio/mpeg";
|
|
373
|
+
mime = AUDIO_MIME[(extra && (extra.output_format || extra.response_format)) || "mp3"] || "audio/mpeg";
|
|
316
374
|
}
|
|
317
375
|
return bytesToDataUrl(bytes, mime);
|
|
318
376
|
}
|
package/src/decide.mjs
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
// ⚖️ Decide — typed judgments from NanoGPT decision models (POST /api/v1/decisions).
|
|
2
|
+
// Port of the editor/play twin block (nanoodle index.html / play.html "⚖️ DECIDE"): same
|
|
3
|
+
// question shapes, same image state parts, same in-order + reversed pick debias, same
|
|
4
|
+
// outputs. Only the on-device image shrink differs: browsers use canvas, this library
|
|
5
|
+
// uses ffmpeg (fitImageJpeg) — text-only decisions need no ffmpeg at all.
|
|
6
|
+
import { NanoodleError } from "./errors.mjs";
|
|
7
|
+
|
|
8
|
+
export const DECIDE_DEFAULT_MODEL = "perplexity/pplx-decider-v1.1-27b";
|
|
9
|
+
export const DECIDE_SCALE_DEFAULT = "poor\nokay\ngood\ngreat";
|
|
10
|
+
/** The live limits of every image-capable decision model at launch (catalog decision_input.image_limits). */
|
|
11
|
+
export const DECIDE_IMG_FALLBACK = { maxImages: 4, maxDimension: 512, maxEncodedBytes: 240000 };
|
|
12
|
+
|
|
13
|
+
export function decideMode(f) {
|
|
14
|
+
const m = f && f.mode;
|
|
15
|
+
return (m === "choose" || m === "score" || m === "yesno") ? m : "pick";
|
|
16
|
+
}
|
|
17
|
+
export function decideLines(s) {
|
|
18
|
+
return String(s == null ? "" : s).split("\n").map((x) => x.trim()).filter(Boolean);
|
|
19
|
+
}
|
|
20
|
+
export function decideDefaultQuestion(mode) {
|
|
21
|
+
if (mode === "pick") return "Which image best matches the brief?";
|
|
22
|
+
if (mode === "choose") return "Which label fits best?";
|
|
23
|
+
if (mode === "score") return "How good is it?";
|
|
24
|
+
return "Is it good enough to use?";
|
|
25
|
+
}
|
|
26
|
+
/** Image limits from a catalog decision_input; known=false (absent / offline) → permissive launch limits; known text-only → null. */
|
|
27
|
+
export function decideImageLimits(di, known) {
|
|
28
|
+
if (!known) return DECIDE_IMG_FALLBACK;
|
|
29
|
+
if (!di || !di.image_input) return null;
|
|
30
|
+
const l = di.image_limits || {};
|
|
31
|
+
return {
|
|
32
|
+
maxImages: l.maxImages > 0 ? l.maxImages : DECIDE_IMG_FALLBACK.maxImages,
|
|
33
|
+
maxDimension: l.maxDimension > 0 ? l.maxDimension : DECIDE_IMG_FALLBACK.maxDimension,
|
|
34
|
+
maxEncodedBytes: l.maxEncodedBytes > 0 ? l.maxEncodedBytes : DECIDE_IMG_FALLBACK.maxEncodedBytes,
|
|
35
|
+
};
|
|
36
|
+
}
|
|
37
|
+
/** The one typed question this node asks. Throws (before any request) on an unanswerable setup. */
|
|
38
|
+
export function decideQuestionFor(mode, f, nImgs) {
|
|
39
|
+
const q = String((f && f.question) || "").trim() || decideDefaultQuestion(mode);
|
|
40
|
+
if (mode === "pick") {
|
|
41
|
+
if (nImgs < 2) throw new NanoodleError("pick needs at least two images — wire 2 or more into the image ports");
|
|
42
|
+
const c = {};
|
|
43
|
+
for (let i = 1; i <= nImgs; i++) c["image_" + i] = "image " + i;
|
|
44
|
+
return { type: "choice", instructions: q, criteria: c };
|
|
45
|
+
}
|
|
46
|
+
if (mode === "choose") {
|
|
47
|
+
const labels = decideLines(f && f.options).filter((x, i, a) => a.indexOf(x) === i);
|
|
48
|
+
if (labels.length < 2) throw new NanoodleError("add at least two labels to choose from (one per line)");
|
|
49
|
+
if (labels.length > 255) throw new NanoodleError("too many labels — 255 at most");
|
|
50
|
+
const cc = {};
|
|
51
|
+
labels.forEach((x) => { cc[x] = null; });
|
|
52
|
+
return { type: "choice", instructions: q, criteria: cc };
|
|
53
|
+
}
|
|
54
|
+
if (mode === "score") {
|
|
55
|
+
const lv = decideLines((f && f.levels != null && String(f.levels).trim()) ? f.levels : DECIDE_SCALE_DEFAULT);
|
|
56
|
+
if (lv.length < 2 || lv.length > 10) throw new NanoodleError("the scale needs 2 to 10 levels, worst first (one per line)");
|
|
57
|
+
return { type: "score", instructions: q, criteria: lv };
|
|
58
|
+
}
|
|
59
|
+
return { type: "noul", instructions: q };
|
|
60
|
+
}
|
|
61
|
+
/** state: plain text, or (with images) text + labelled inline image_url parts. */
|
|
62
|
+
export function decideState(text, imgs) {
|
|
63
|
+
if (!imgs.length) return text;
|
|
64
|
+
const s = [];
|
|
65
|
+
if (text) s.push(text);
|
|
66
|
+
imgs.forEach((u, i) => {
|
|
67
|
+
if (imgs.length > 1) s.push("image_" + (i + 1) + ":");
|
|
68
|
+
s.push({ type: "image_url", image_url: { url: u } });
|
|
69
|
+
});
|
|
70
|
+
return s;
|
|
71
|
+
}
|
|
72
|
+
/** API answer → node outputs: text, image (pick: the winner; else the first image, passed through), decision detail. */
|
|
73
|
+
export function decideOutputs(mode, ans, q, imgs, usage, model) {
|
|
74
|
+
if (!ans || !ans.type) throw new NanoodleError("the decision model returned no answer");
|
|
75
|
+
const d = { mode, model, cost: (usage && typeof usage.cost === "number") ? usage.cost : null, rows: [] };
|
|
76
|
+
const probs = ans.probabilities || {};
|
|
77
|
+
if (mode === "pick") {
|
|
78
|
+
const k = String(ans.choice || ""), idx = parseInt(k.replace(/^image_/, ""), 10) || 1;
|
|
79
|
+
d.pick = idx; d.confidence = ans.confidence;
|
|
80
|
+
d.rows = Object.keys(q.criteria).map((key, i) => ({ label: "image " + (i + 1), p: +probs[key] || 0, win: key === k }));
|
|
81
|
+
return { text: "image " + idx, image: imgs[idx - 1] || "", decision: d };
|
|
82
|
+
}
|
|
83
|
+
if (mode === "choose") {
|
|
84
|
+
d.confidence = ans.confidence;
|
|
85
|
+
d.rows = Object.keys(q.criteria).map((key) => ({ label: key, p: +probs[key] || 0, win: key === ans.choice }));
|
|
86
|
+
return { text: String(ans.choice || ""), image: imgs[0] || "", decision: d };
|
|
87
|
+
}
|
|
88
|
+
if (mode === "score") {
|
|
89
|
+
const sc = (+ans.score || 0) + 1; // 1 = the first (worst) level
|
|
90
|
+
d.score = sc; d.levels = q.criteria.length; d.confidence = ans.confidence;
|
|
91
|
+
let top = 0;
|
|
92
|
+
q.criteria.forEach((x, i) => { if ((+probs[String(i)] || 0) > (+probs[String(top)] || 0)) top = i; });
|
|
93
|
+
d.rows = q.criteria.map((x, i) => ({ label: (i + 1) + " · " + x, p: +probs[String(i)] || 0, win: i === top }));
|
|
94
|
+
return { text: String(Math.round(sc * 100) / 100), image: imgs[0] || "", decision: d };
|
|
95
|
+
}
|
|
96
|
+
const py = Math.max(0, Math.min(1, +ans.noul || 0));
|
|
97
|
+
d.yes = py;
|
|
98
|
+
d.rows = [{ label: "yes", p: py, win: py >= 0.5 }, { label: "no", p: 1 - py, win: py < 0.5 }];
|
|
99
|
+
return { text: py >= 0.5 ? "yes" : "no", image: imgs[0] || "", decision: d };
|
|
100
|
+
}
|
|
101
|
+
/**
|
|
102
|
+
* A closed yes/no gate: a deliberate stop, not a failure. The runner throws this (so the browser
|
|
103
|
+
* twin and any direct RUNNERS caller see err.gate), and Workflow.run() settles it as node status
|
|
104
|
+
* "gated" — the decision ran and billed, `out` carries its answer, everything downstream is
|
|
105
|
+
* "skipped" unbilled, and the run itself succeeds.
|
|
106
|
+
*/
|
|
107
|
+
export function decideGateError(d, out) {
|
|
108
|
+
return new NanoodleError(
|
|
109
|
+
"gate closed — the answer was no (yes " + Math.round(d.yes * 100) + "%), so nothing downstream ran",
|
|
110
|
+
{ code: "decide-gate", gate: true, decision: d, out: out || null });
|
|
111
|
+
}
|
|
112
|
+
/** Average pick probabilities across runs that saw the candidates in different orders; costs add up. */
|
|
113
|
+
export function decideMergeOrders(js, orders) {
|
|
114
|
+
const n = orders[0].length, sum = new Array(n).fill(0);
|
|
115
|
+
let cost = 0, costKnown = true;
|
|
116
|
+
js.forEach((j, r) => {
|
|
117
|
+
const a = j && j.answers && j.answers.answer;
|
|
118
|
+
if (!a || !a.probabilities) throw new NanoodleError("the decision model returned no answer");
|
|
119
|
+
orders[r].forEach((orig, k) => { sum[orig] += (+a.probabilities["image_" + (k + 1)] || 0) / js.length; });
|
|
120
|
+
if (j.usage && typeof j.usage.cost === "number") cost += j.usage.cost; else costKnown = false;
|
|
121
|
+
});
|
|
122
|
+
let best = 0;
|
|
123
|
+
const probs = {};
|
|
124
|
+
sum.forEach((p, i) => { probs["image_" + (i + 1)] = p; if (p > sum[best]) best = i; });
|
|
125
|
+
return {
|
|
126
|
+
answers: { answer: { type: "choice", choice: "image_" + (best + 1), confidence: sum[best], probabilities: probs } },
|
|
127
|
+
usage: { cost: costKnown ? cost : null }, runs: js.length,
|
|
128
|
+
};
|
|
129
|
+
}
|
|
130
|
+
/**
|
|
131
|
+
* The whole run. send(body) POSTs /api/v1/decisions and returns the JSON; fit(url, maxDim, budget)
|
|
132
|
+
* returns a data: URL that fits (inlining remote URLs as needed).
|
|
133
|
+
*/
|
|
134
|
+
export async function decideRun(f, model, text, imgs, lim, send, fit) {
|
|
135
|
+
const mode = decideMode(f);
|
|
136
|
+
text = String(text == null ? "" : text).trim();
|
|
137
|
+
imgs = (imgs || []).filter(Boolean);
|
|
138
|
+
if (imgs.length && !lim) throw new NanoodleError("this decision model can’t see images — pick one that can (e.g. PPLX Decider or Clef)");
|
|
139
|
+
if (lim && imgs.length > lim.maxImages) throw new NanoodleError("this decision model takes at most " + lim.maxImages + " images — unwire the extras");
|
|
140
|
+
const q = decideQuestionFor(mode, f, imgs.length);
|
|
141
|
+
if (!text && !imgs.length && !String((f && f.question) || "").trim()) throw new NanoodleError("nothing to judge — wire text or an image into Decide");
|
|
142
|
+
const sent = [];
|
|
143
|
+
if (imgs.length) {
|
|
144
|
+
const budget = Math.floor(lim.maxEncodedBytes * 0.95 / imgs.length);
|
|
145
|
+
for (const u of imgs) sent.push(await fit(u, lim.maxDimension, budget));
|
|
146
|
+
}
|
|
147
|
+
let j;
|
|
148
|
+
if (mode === "pick") {
|
|
149
|
+
// decision models lean toward the first image they see: ask in order and reversed, average
|
|
150
|
+
const fwd = sent.map((x, i) => i), rev = fwd.slice().reverse();
|
|
151
|
+
const both = await Promise.all([fwd, rev].map((ord) =>
|
|
152
|
+
send({ model, state: decideState(text, ord.map((i) => sent[i])), questions: { answer: q } })));
|
|
153
|
+
j = decideMergeOrders(both, [fwd, rev]);
|
|
154
|
+
} else {
|
|
155
|
+
j = await send({ model, state: decideState(text, sent), questions: { answer: q } });
|
|
156
|
+
}
|
|
157
|
+
const out = decideOutputs(mode, j && j.answers && j.answers.answer, q, imgs, j && j.usage, model);
|
|
158
|
+
if (mode === "yesno" && (f.gate === true || f.gate === "true") && out.decision.yes < 0.5) throw decideGateError(out.decision, out);
|
|
159
|
+
return out;
|
|
160
|
+
}
|