badgr-cli 1.0.41 → 1.0.43

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -66,10 +66,14 @@ badgr down <deployment-id>
66
66
  **Shortcuts** — wrappers around `run` / `serve` for common workloads:
67
67
 
68
68
  | `badgr comfyui run <workflow.json>` | Launch ComfyUI, queue workflow, return endpoint URL |
69
+ | `badgr comfyui batch --workflow ...` | Productized batch image generation — no ComfyUI setup, blessed workflow only |
69
70
  | `badgr train <config.yaml>` | LoRA / fine-tuning job, stream logs |
71
+ | `badgr train lora --base-model ...` | Productized LoRA training — preset + dataset, no config file needed |
70
72
  | `badgr transcribe <audio>` | Whisper transcription, print transcript |
71
73
  | `badgr embed <model> <input>` | Text embeddings, output JSONL |
72
74
 
75
+ `badgr serve --list-aliases` lists the blessed vLLM model shortcuts (`qwen-7b`, `llama-8b`, `qwen-coder-7b`) that can be used in place of a full model ID.
76
+
73
77
  ---
74
78
 
75
79
  ## `badgr serve` options
@@ -91,6 +95,9 @@ badgr serve meta-llama/Llama-3.1-8B-Instruct --gpu L40S --region EU
91
95
  | `--max-cost <$>` | — | Auto-stop when total spend reaches this amount |
92
96
  | `--health-path <path>` | auto | Readiness path to poll (auto-detected for ComfyUI → `/system_stats`) |
93
97
  | `--no-wait` | — | Skip endpoint health check and return immediately |
98
+ | `--list-aliases` | — | List blessed vLLM model aliases (`qwen-7b`, `llama-8b`, `qwen-coder-7b`) and exit — no provisioning, no API key required |
99
+
100
+ Blessed aliases expand to a full model ID + preset GPU, e.g. `badgr serve qwen-7b` → `Qwen/Qwen2.5-7B-Instruct` on an RTX 4090. Run `badgr serve --list-aliases` to see the current list.
94
101
 
95
102
  ---
96
103
 
@@ -153,13 +160,39 @@ Requires either `--max-cost` or `--persistent` to prevent runaway billing.
153
160
 
154
161
  ---
155
162
 
163
+ ## `badgr comfyui batch` options
164
+
165
+ Productized batch image generation — runs a list of prompts through a **blessed** ComfyUI workflow and returns image URLs. No ComfyUI setup, no workflow file, no manual teardown.
166
+
167
+ ```bash
168
+ badgr comfyui batch --workflow sdxl-basic --prompts prompts.txt --max-cost 10
169
+ badgr comfyui batch --workflow sdxl-basic --prompt "a cat on a beach" --prompt "a dog in the park" --max-cost 5
170
+ ```
171
+
172
+ Blessed workflows: `sdxl-basic` (SDXL text-to-image, default sampler settings). Max 20 prompts per batch.
173
+
174
+ | Flag | Default | Description |
175
+ |------|---------|-------------|
176
+ | `--workflow <name>` | — | Blessed workflow ID (required) — currently `sdxl-basic` |
177
+ | `--prompts <file>` | — | Text file, one prompt per line |
178
+ | `--prompt <text>` | — | Inline prompt (repeatable) — combine with `--prompts` if needed |
179
+ | `--max-cost <$>` | — | Auto-stop when total spend reaches this amount (required unless `--dry-run`) |
180
+ | `--max-runtime <min>` | 60 | Auto-stop after N minutes |
181
+ | `--gpu-type <type>` | workflow default | GPU type override |
182
+ | `--tier 1\|2` | 1 | Provider tier |
183
+ | `--dry-run` | — | Preview the batch (workflow, GPU, prompt count, cost) without provisioning |
184
+
185
+ Polls until complete and prints image URLs, or detaches with `badgr status` guidance if it outlives `--max-runtime`.
186
+
187
+ ---
188
+
156
189
  ## `badgr train` options
157
190
 
158
191
  ```bash
159
192
  badgr train config.yaml --gpu A100 --max-runtime 240 --env HF_TOKEN=$HF_TOKEN
160
193
  ```
161
194
 
162
- Auto-detects framework (axolotl, unsloth, trl) from config content. Default max-runtime is 120 min.
195
+ Detects framework (axolotl, unsloth, trl) from config content, but **only Axolotl configs run today** — the container command for `unsloth`/`trl`/unrecognized configs isn't wired up yet, so `badgr train` blocks before provisioning rather than billing a GPU that's guaranteed to fail. Use `--framework axolotl` to force it, or use `badgr train lora` for a config-free productized path. Default max-runtime is 120 min.
163
196
 
164
197
  | Flag | Default | Description |
165
198
  |------|---------|-------------|
@@ -169,12 +202,46 @@ Auto-detects framework (axolotl, unsloth, trl) from config content. Default max-
169
202
  | `--max-price <$/hr>` | — | Hard spend cap per GPU-hour |
170
203
  | `--tier 1\|2` | 1 | Provider tier |
171
204
  | `--region US\|EU\|AU` | — | Region preference |
172
- | `--framework <name>` | auto-detect | Force framework: `axolotl`, `unsloth`, `trl` |
205
+ | `--framework <name>` | auto-detect | Force framework: `axolotl`, `unsloth`, `trl` (only `axolotl` currently runs) |
173
206
  | `--env KEY=VALUE` | — | Environment variable (repeatable) |
174
207
  | `--detach` | — | Launch and return immediately, don't stream logs |
175
208
 
176
209
  ---
177
210
 
211
+ ## `badgr train lora` options
212
+
213
+ Productized LoRA training — pass a base model and dataset, no Axolotl config file needed. Badgr generates the config from a preset and returns a downloadable adapter.
214
+
215
+ ```bash
216
+ badgr train lora --base-model mistralai/Mistral-7B-v0.1 --dataset ./train.jsonl --preset small --max-cost 20
217
+ badgr train lora --base-model meta-llama/Llama-3.1-8B-Instruct --dataset https://example.com/data.jsonl --preset medium --max-cost 40
218
+ ```
219
+
220
+ Dataset sources: local file (uploaded first), direct URL (`https://`, `s3://`), or `--file-id` from a prior `badgr` upload.
221
+
222
+ | Flag | Default | Description |
223
+ |------|---------|-------------|
224
+ | `--base-model <id>` | — | HuggingFace model ID (required) — validated to exist before provisioning |
225
+ | `--dataset <path\|url>` | — | Local file, direct URL, or `s3://` URI |
226
+ | `--file-id <id>` | — | Badgr upload ID instead of `--dataset` |
227
+ | `--preset small\|medium` | `small` | Training profile — see below |
228
+ | `--max-cost <$>` | — | Auto-stop when total spend reaches this amount (required unless `--dry-run`) |
229
+ | `--max-runtime <min>` | 240 | Auto-stop after N minutes |
230
+ | `--gpu-type <type>` | preset default | GPU type override |
231
+ | `--tier 1\|2` | 1 | Provider tier |
232
+ | `--dry-run` | — | Preview the job (preset, GPU, rank, epochs, cost) without provisioning |
233
+
234
+ **Presets:**
235
+
236
+ | Preset | GPU | LoRA rank | Epochs | Best for |
237
+ |--------|-----|-----------|--------|----------|
238
+ | `small` (default) | RTX 4090 | 16 | 3 | Fast, low-cost — good default for most datasets |
239
+ | `medium` | A100 | 32 | 5 | Larger rank/more epochs — bigger datasets or higher quality |
240
+
241
+ On completion, prints an `adapter_url` — download with `GET /v1/jobs/{job_id}/adapter`, or via `badgr workload info` if saved.
242
+
243
+ ---
244
+
178
245
  ## `badgr transcribe` options
179
246
 
180
247
  ```bash
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "badgr-cli",
3
- "version": "1.0.41",
3
+ "version": "1.0.43",
4
4
  "description": "Badgr — run or serve GPU workloads from one command",
5
5
  "type": "module",
6
6
  "bin": {
@@ -13,6 +13,7 @@
13
13
  },
14
14
  "dependencies": {
15
15
  "@inquirer/prompts": "^8.5.2",
16
+ "archiver": "^7.0.1",
16
17
  "chalk": "^5.3.0"
17
18
  },
18
19
  "devDependencies": {
package/src/badgr.js CHANGED
@@ -42,12 +42,15 @@ ${chalk.bold('COMMANDS')}
42
42
 
43
43
  ${chalk.bold('SHORTCUTS')} ${chalk.dim('(wrappers around run / serve for common workloads)')}
44
44
  ${chalk.cyan('badgr comfyui run <workflow.json>')} Launch ComfyUI, return endpoint URL
45
+ ${chalk.cyan('badgr comfyui batch --workflow ...')} Blessed-workflow batch image gen (no ComfyUI setup needed)
45
46
  ${chalk.cyan('badgr train <config.yaml>')} LoRA / fine-tuning job, stream logs
47
+ ${chalk.cyan('badgr train lora --base-model ...')} Productized LoRA training (preset + dataset, no config file)
46
48
  ${chalk.cyan('badgr transcribe <audio>')} Whisper transcription, print transcript
47
49
  ${chalk.cyan('badgr embed <model> <input>')} Text embeddings, output JSONL
48
50
  ${chalk.cyan('badgr serve template <name>')} Launch an endpoint template (vllm, invokeai, comfyui, …)
49
51
  ${chalk.cyan('badgr run template <name>')} Launch a job template (axolotl, unsloth)
50
52
  ${chalk.cyan('badgr template list')} Browse all pre-built templates
53
+ ${chalk.cyan('badgr serve --list-aliases')} List blessed vLLM model shortcuts (qwen-7b, llama-8b, …)
51
54
 
52
55
  ${chalk.bold('EXAMPLES')}
53
56
  ${chalk.dim('# Verify the stack works end-to-end:')}
package/src/catalog.js CHANGED
@@ -442,6 +442,44 @@ export const TEMPLATES = [
442
442
 
443
443
  export const TEMPLATE_MAP = Object.fromEntries(TEMPLATES.map(t => [t.name, t]));
444
444
 
445
+ // ---------------------------------------------------------------------------
446
+ // Productized runner catalogs
447
+ // ---------------------------------------------------------------------------
448
+
449
+ /** Blessed vLLM model aliases. `badgr serve qwen-7b` expands to the full model ID. */
450
+ export const BLESSED_VLLM_MODELS = {
451
+ 'qwen-7b': {
452
+ model_id: 'Qwen/Qwen2.5-7B-Instruct',
453
+ gpu_type: 'RTX_4090',
454
+ image: 'vllm/vllm-openai:latest',
455
+ health_path: '/v1/models',
456
+ description: 'Qwen 2.5 7B Instruct — fast, multilingual',
457
+ },
458
+ 'llama-8b': {
459
+ model_id: 'meta-llama/Llama-3.1-8B-Instruct',
460
+ gpu_type: 'RTX_4090',
461
+ image: 'vllm/vllm-openai:latest',
462
+ health_path: '/v1/models',
463
+ description: 'Llama 3.1 8B Instruct — Meta flagship 8B',
464
+ },
465
+ 'qwen-coder-7b': {
466
+ model_id: 'Qwen/Qwen2.5-Coder-7B-Instruct',
467
+ gpu_type: 'RTX_4090',
468
+ image: 'vllm/vllm-openai:latest',
469
+ health_path: '/v1/models',
470
+ description: 'Qwen 2.5 Coder 7B — code generation specialist',
471
+ },
472
+ };
473
+
474
+ /** Blessed ComfyUI workflows accepted by `POST /v1/jobs` comfy.batch. */
475
+ export const BLESSED_COMFY_WORKFLOWS = {
476
+ 'sdxl-basic': {
477
+ description: 'SDXL text-to-image with default sampler settings',
478
+ gpu_type: 'RTX_4090',
479
+ output_type: 'images',
480
+ },
481
+ };
482
+
445
483
  /**
446
484
  * Build the args array passed to serveCommand / runCommand.
447
485
  * Template defaults are applied first; CLI overrides win.
@@ -11,6 +11,7 @@ import { callApi, listDeployments } from '../api.js';
11
11
  import { addDeployment, addReceipt, updateReceipt, generateReceiptId } from '../store.js';
12
12
  import { normalizeTier, callWithFallback, HIGH_RATE_THRESHOLD } from '../fallback.js';
13
13
  import { formatCliError } from '../errors.js';
14
+ import { BLESSED_COMFY_WORKFLOWS } from '../catalog.js';
14
15
 
15
16
  const COMFYUI_IMAGE = process.env.COMFYUI_IMAGE || 'yanwk/comfyui-boot:cu126-megapak';
16
17
  const HEALTH_PATH = '/system_stats';
@@ -110,11 +111,169 @@ async function validateComfyNodes(endpointUrl, nodeList, chalk) {
110
111
  }
111
112
  }
112
113
 
114
+ // ---------------------------------------------------------------------------
115
+ // badgr comfyui batch — productized batch via POST /v1/jobs comfy.batch
116
+ // ---------------------------------------------------------------------------
117
+
118
+ export function parseComfyBatchArgs(args) {
119
+ const flags = {};
120
+ let i = 0;
121
+ while (i < args.length) {
122
+ const a = args[i];
123
+ if (a === '--workflow') { flags.workflow = args[++i]; i++; continue; }
124
+ if (a === '--prompts') { flags.prompts = args[++i]; i++; continue; }
125
+ if (a === '--prompt') { if (!flags.inlinePrompts) flags.inlinePrompts = []; flags.inlinePrompts.push(args[++i]); i++; continue; }
126
+ if (a === '--max-cost') { flags.maxCost = parseFloat(args[++i]); i++; continue; }
127
+ if (a === '--max-runtime') { flags.maxRuntime = parseFloat(args[++i]); i++; continue; }
128
+ if (a === '--tier') { flags.tier = args[++i]; i++; continue; }
129
+ if (a === '--gpu-type') { flags.gpuType = args[++i]; i++; continue; }
130
+ if (a === '--dry-run') { flags.dryRun = true; i++; continue; }
131
+ i++;
132
+ }
133
+ return flags;
134
+ }
135
+
136
+ export async function comfyBatchCommand(config, args, chalk) {
137
+ const { readFileSync, existsSync } = await import('fs');
138
+ const { callApi } = await import('../api.js');
139
+ const { addReceipt, generateReceiptId } = await import('../store.js');
140
+ const flags = parseComfyBatchArgs(args);
141
+
142
+ if (!flags.workflow) {
143
+ console.error(chalk.red('\n Usage: badgr comfyui batch --workflow sdxl-basic --prompts prompts.txt --max-cost 10\n'));
144
+ console.error(chalk.dim(' Runs a batch of prompts through a blessed ComfyUI workflow and returns image URLs.\n'));
145
+ console.error(chalk.dim(' Blessed workflows: sdxl-basic\n'));
146
+ process.exitCode = 1;
147
+ return;
148
+ }
149
+
150
+ if (!flags.maxCost && !flags.dryRun) {
151
+ console.error(chalk.red('\n ✗ --max-cost is required.\n'));
152
+ process.exitCode = 1;
153
+ return;
154
+ }
155
+
156
+ // Collect prompts from file or --prompt flags
157
+ let prompts = flags.inlinePrompts || [];
158
+ if (flags.prompts) {
159
+ if (!existsSync(flags.prompts)) {
160
+ console.error(chalk.red(`\n ✗ Prompts file not found: ${flags.prompts}\n`));
161
+ process.exitCode = 1;
162
+ return;
163
+ }
164
+ const lines = readFileSync(flags.prompts, 'utf8')
165
+ .split('\n')
166
+ .map(l => l.trim())
167
+ .filter(Boolean);
168
+ prompts = prompts.concat(lines);
169
+ }
170
+
171
+ if (flags.dryRun) {
172
+ const workflowSpec = BLESSED_COMFY_WORKFLOWS[flags.workflow];
173
+ console.log(chalk.bold('\n⚡ Dry run — no GPU will be provisioned\n'));
174
+ console.log(` ${chalk.bold('Workflow:')} ${flags.workflow}${workflowSpec ? '' : chalk.yellow(' (unknown — server will reject this)')}`);
175
+ if (workflowSpec) {
176
+ console.log(` ${chalk.bold('GPU:')} ${flags.gpuType || workflowSpec.gpu_type}`);
177
+ console.log(` ${chalk.dim(workflowSpec.description)}`);
178
+ }
179
+ console.log(` ${chalk.bold('Prompts:')} ${prompts.length}${prompts.length > 20 ? chalk.yellow(' (exceeds the 20-prompt limit — server will reject this)') : ''}`);
180
+ if (flags.maxCost) console.log(` ${chalk.bold('Max cost:')} $${flags.maxCost}`);
181
+ console.log(` ${chalk.bold('Max runtime:')} ${flags.maxRuntime ?? 60}min`);
182
+ console.log(chalk.dim('\n Remove --dry-run to submit.\n'));
183
+ return;
184
+ }
185
+
186
+ requireApiKey(config);
187
+
188
+ if (prompts.length === 0) {
189
+ console.error(chalk.red('\n ✗ No prompts provided. Use --prompts file.txt or --prompt "text"\n'));
190
+ process.exitCode = 1;
191
+ return;
192
+ }
193
+
194
+ if (prompts.length > 20) {
195
+ console.error(chalk.red(`\n ✗ Max 20 prompts per batch (got ${prompts.length})\n`));
196
+ process.exitCode = 1;
197
+ return;
198
+ }
199
+
200
+ const input = { workflow_id: flags.workflow, prompts };
201
+ if (flags.gpuType) input.gpu_type = flags.gpuType;
202
+
203
+ const rcptId = generateReceiptId();
204
+ const maxRuntime = flags.maxRuntime ?? 60;
205
+
206
+ console.log(chalk.bold('\n⚡ Starting ComfyUI batch\n'));
207
+ console.log(` ${chalk.bold('Workflow:')} ${flags.workflow}`);
208
+ console.log(` ${chalk.bold('Prompts:')} ${prompts.length}`);
209
+ console.log(` ${chalk.bold('Max cost:')} $${flags.maxCost}`);
210
+ console.log(` ${chalk.bold('Max runtime:')} ${maxRuntime} min\n`);
211
+
212
+ let job;
213
+ try {
214
+ job = await callApi('/jobs', {
215
+ method: 'POST',
216
+ apiKey: config.apiKey,
217
+ baseUrl: config.baseUrl,
218
+ body: {
219
+ type: 'comfy.batch',
220
+ input,
221
+ policy: { max_cost: flags.maxCost, max_runtime_minutes: maxRuntime, tier: flags.tier },
222
+ },
223
+ });
224
+ } catch (err) {
225
+ console.error(chalk.red(`\n ✗ Failed to submit job: ${err.message}\n`));
226
+ process.exitCode = 1;
227
+ return;
228
+ }
229
+
230
+ addReceipt({ id: rcptId, type: 'comfy.batch', job_id: job.job_id, started_at: Date.now() });
231
+ console.log(` ${chalk.bold('Job ID:')} ${job.job_id}`);
232
+ console.log(` ${chalk.bold('Receipt:')} ${chalk.dim(rcptId)}`);
233
+ console.log(chalk.dim('\n Polling for completion…\n'));
234
+
235
+ const startMs = Date.now();
236
+ const maxMs = maxRuntime * 60 * 1000;
237
+ while (Date.now() - startMs < maxMs) {
238
+ await new Promise(r => setTimeout(r, 15_000));
239
+ let detail;
240
+ try {
241
+ detail = await callApi(`/jobs/${job.job_id}`, {
242
+ apiKey: config.apiKey,
243
+ baseUrl: config.baseUrl,
244
+ });
245
+ } catch { continue; }
246
+ process.stdout.write(`\r Status: ${detail.status} elapsed: ${Math.floor((Date.now() - startMs) / 1000)}s `);
247
+ if (detail.status === 'completed') {
248
+ const out = detail.output || {};
249
+ console.log(chalk.green('\n\n ✓ Batch complete\n'));
250
+ if (out.image_urls && out.image_urls.length > 0) {
251
+ console.log(` ${chalk.bold('Images (${out.image_urls.length}):')}`);
252
+ out.image_urls.forEach((url, i) => console.log(` ${i + 1}. ${url}`));
253
+ }
254
+ console.log(`\n ${chalk.bold('Receipt:')} ${chalk.dim(rcptId)}\n`);
255
+ return;
256
+ }
257
+ if (detail.status === 'failed') {
258
+ console.error(chalk.red(`\n\n ✗ Batch failed: ${detail.error_code || ''} — ${detail.error_message || ''}\n`));
259
+ process.exitCode = 1;
260
+ return;
261
+ }
262
+ }
263
+ console.error(chalk.yellow('\n\n Batch still running — detached. Check status:\n'));
264
+ console.error(chalk.dim(` badgr status\n`));
265
+ }
266
+
113
267
  export async function comfyuiCommand(config, args, chalk) {
268
+ // Route subcommands
269
+ const sub = args[0];
270
+ if (sub === 'batch') return comfyBatchCommand(config, args.slice(1), chalk);
271
+
114
272
  const { workflow, flags } = parseComfyuiArgs(args);
115
273
 
116
274
  if (!workflow) {
117
275
  console.error(chalk.red('\n Usage: badgr comfyui run workflow.json\n'));
276
+ console.error(chalk.dim(' Batch mode: badgr comfyui batch --workflow sdxl-basic --prompts prompts.txt --max-cost 10\n'));
118
277
  console.error(chalk.dim(' Launches ComfyUI, queues your workflow, and returns the URL.\n'));
119
278
  process.exitCode = 1;
120
279
  return;
@@ -383,11 +383,10 @@ function _isGitHubUrl(arg) {
383
383
 
384
384
  /**
385
385
  * Zip a local directory into a temp file, returning the temp file path.
386
- * Uses Node's built-in APIs no external zip library required.
386
+ * Uses the `archiver` package so the CLI never depends on a system `zip`
387
+ * binary being present (Windows, minimal containers, etc).
387
388
  */
388
389
  async function _zipDirectory(dirPath, chalk) {
389
- // Use archiver if available, otherwise fall back to a manual approach via
390
- // a child process calling `zip` (available on Linux/macOS) or PowerShell on Windows.
391
390
  const absDir = path.resolve(dirPath);
392
391
  if (!fs.existsSync(absDir)) {
393
392
  throw new Error(`Directory not found: ${absDir}`);
@@ -395,37 +394,19 @@ async function _zipDirectory(dirPath, chalk) {
395
394
 
396
395
  const tmpFile = path.join(os.tmpdir(), `badgr-upload-${Date.now()}.zip`);
397
396
 
398
- // Attempt to use archiver (optional peer dep) first; fall back to system zip.
399
- try {
400
- const { default: archiver } = await import('archiver');
401
- await new Promise((resolve, reject) => {
402
- const output = createWriteStream(tmpFile);
403
- const archive = archiver('zip', { zlib: { level: 6 } });
404
- output.on('close', resolve);
405
- archive.on('error', reject);
406
- archive.pipe(output);
407
- archive.glob('**/*', {
408
- cwd: absDir,
409
- dot: false,
410
- ignore: [..._ZIP_EXCLUDES].map(e => `**/${e}/**`).concat([..._ZIP_EXCLUDES].map(e => e)),
411
- });
412
- archive.finalize();
413
- });
414
- return tmpFile;
415
- } catch {
416
- // archiver not installed — fall back to system zip command
417
- }
418
-
419
- const { spawn } = await import('child_process');
397
+ const { default: archiver } = await import('archiver');
420
398
  await new Promise((resolve, reject) => {
421
- // Build exclude args — zip uses -x patterns
422
- const excludeArgs = [];
423
- for (const ex of _ZIP_EXCLUDES) {
424
- excludeArgs.push('-x', `*/${ex}/*`, '-x', `${ex}/*`, '-x', `${ex}`);
425
- }
426
- const child = spawn('zip', ['-r', '-q', tmpFile, '.', ...excludeArgs], { cwd: absDir });
427
- child.on('close', code => code === 0 ? resolve() : reject(new Error(`zip exited ${code}`)));
428
- child.on('error', reject);
399
+ const output = createWriteStream(tmpFile);
400
+ const archive = archiver('zip', { zlib: { level: 6 } });
401
+ output.on('close', resolve);
402
+ archive.on('error', reject);
403
+ archive.pipe(output);
404
+ archive.glob('**/*', {
405
+ cwd: absDir,
406
+ dot: false,
407
+ ignore: [..._ZIP_EXCLUDES].map(e => `**/${e}/**`).concat([..._ZIP_EXCLUDES].map(e => e)),
408
+ });
409
+ archive.finalize();
429
410
  });
430
411
 
431
412
  return tmpFile;
@@ -3,7 +3,7 @@ import { callApi, listDeployments } from '../api.js';
3
3
  import { addDeployment, addReceipt, updateReceipt, generateReceiptId } from '../store.js';
4
4
  import { normalizeTier, callWithFallback, HIGH_RATE_THRESHOLD } from '../fallback.js';
5
5
  import { formatCliError } from '../errors.js';
6
- import { TEMPLATE_MAP, buildTemplateFlags, parseTemplateOverrides } from '../catalog.js';
6
+ import { TEMPLATE_MAP, buildTemplateFlags, parseTemplateOverrides, BLESSED_VLLM_MODELS } from '../catalog.js';
7
7
 
8
8
  const LLAMA_CPP_IMAGE = 'michaelmanleyx/llama-cpp:server-cuda';
9
9
 
@@ -42,6 +42,7 @@ export function parseServeArgs(args) {
42
42
  if (args[i] === '--runtime') { flags.runtime = args[++i]; i++; continue; }
43
43
  if (args[i] === '--hf-repo') { flags.hfRepo = args[++i]; i++; continue; }
44
44
  if (args[i] === '--hf-file') { flags.hfFile = args[++i]; i++; continue; }
45
+ if (args[i] === '--list-aliases') { flags.listAliases = true; i++; continue; }
45
46
  if (args[i] === '--env') {
46
47
  const kv = args[++i]; i++;
47
48
  if (!flags.env) flags.env = [];
@@ -97,20 +98,17 @@ function _inferGgufProfile(ggufPath) {
97
98
  return { label: 'GGUF inference (36B+, llama.cpp)', vram: '48+ GB', gpus: ['A6000', 'L40S', 'A100'] };
98
99
  }
99
100
 
100
- function _serveStageLabel(elapsedSec, healthPath = '/models') {
101
- if (healthPath === '/models') {
102
- if (elapsedSec < 45) return 'Starting vLLM…';
103
- if (elapsedSec < 150) return 'Downloading model…';
104
- return 'Waiting for /v1/models…';
101
+ // Label derived from the backend's real readiness signal (readiness_reason),
102
+ // not a fixed timer — the timer only breaks ties when we have no signal yet.
103
+ function _serveStageLabel(elapsedSec, healthPath, readinessReason) {
104
+ switch (readinessReason) {
105
+ case 'port_unreachable': return 'Waiting for container to start…';
106
+ case 'http_404': return 'Model loading — waiting for the API to come up…';
107
+ case 'http_error': return `Waiting for ${healthPath || 'endpoint'} (non-200 response)…`;
108
+ default: break;
105
109
  }
106
- if (healthPath === '/health') {
107
- if (elapsedSec < 60) return 'Starting server…';
108
- if (elapsedSec < 180) return 'Downloading model…';
109
- return 'Waiting for /health…';
110
- }
111
- if (elapsedSec < 30) return 'Starting container…';
112
- if (elapsedSec < 120) return 'Container starting…';
113
- return `Waiting for ${healthPath}…`;
110
+ if (elapsedSec < 30) return 'Starting container…';
111
+ return `Waiting for ${healthPath || 'endpoint'}…`;
114
112
  }
115
113
 
116
114
  function _detectHealthPath(image) {
@@ -121,16 +119,21 @@ function _detectHealthPath(image) {
121
119
  }
122
120
 
123
121
  /**
124
- * Poll the model endpoint until /models returns 200, or timeout.
125
- * Also checks deployment status on each iteration fails fast if the dep is dead.
122
+ * Wait for the deployment's app-level endpoint to become ready by polling Badgr's
123
+ * own deployment status (GET /deployments/{id})never the RunPod proxy/pod
124
+ * directly. The backend does the real health_path probing (see
125
+ * DeploymentService.check_endpoint_readiness); a pod reporting RUNNING only
126
+ * means infrastructure is up, not that the app inside is serving.
126
127
  * Returns { ready: boolean, timedOut: boolean, depFailed: boolean, failReason?: string }
127
128
  */
128
- async function waitForEndpoint(endpointUrl, deploymentId, config, timeoutMs = 5 * 60 * 1000, chalk, healthPath = '/models') {
129
+ // vLLM cold start (model download + load) often exceeds 5 min on first boot.
130
+ const VLLM_SERVE_WAIT_MS = 15 * 60 * 1000;
131
+
132
+ async function waitForEndpoint(deploymentId, config, timeoutMs = VLLM_SERVE_WAIT_MS, chalk, healthPath = '/models') {
129
133
  const startMs = Date.now();
130
134
  const deadline = startMs + timeoutMs;
131
135
 
132
136
  while (Date.now() < deadline) {
133
- // Check deployment status first — fail fast on OOM/crash before waiting more
134
137
  try {
135
138
  const dep = await callApi(`/deployments/${deploymentId}`, {
136
139
  apiKey: config.apiKey,
@@ -139,23 +142,19 @@ async function waitForEndpoint(endpointUrl, deploymentId, config, timeoutMs = 5
139
142
  });
140
143
  if (['failed', 'terminated', 'error', 'stopped'].includes(dep.status)) {
141
144
  process.stdout.write('\n');
142
- return { ready: false, timedOut: false, depFailed: true, failReason: dep.error || dep.status };
145
+ return { ready: false, timedOut: false, depFailed: true, failReason: dep.fix_hint || dep.error || dep.status };
143
146
  }
147
+ if (dep.endpoint_ready) {
148
+ process.stdout.write('\n');
149
+ return { ready: true, timedOut: false, depFailed: false };
150
+ }
151
+ const elapsed = Math.round((Date.now() - startMs) / 1000);
152
+ process.stdout.write(
153
+ `\r ${chalk.dim(_serveStageLabel(elapsed, dep.health_path || healthPath, dep.readiness_reason) + ` (${elapsed}s)`)} `
154
+ );
144
155
  } catch {
145
- // status check failed — continue to endpoint check
146
- }
147
-
148
- try {
149
- const res = await fetch(`${endpointUrl}${healthPath}`, { signal: AbortSignal.timeout(8000) });
150
- if (res.ok) { process.stdout.write('\n'); return { ready: true, timedOut: false, depFailed: false }; }
151
- } catch {
152
- // still starting
156
+ // status check failed (transient network/API issue) retry next tick
153
157
  }
154
-
155
- const elapsed = Math.round((Date.now() - startMs) / 1000);
156
- process.stdout.write(
157
- `\r ${chalk.dim(_serveStageLabel(elapsed, healthPath) + ` (${elapsed}s)`)} `
158
- );
159
158
  await new Promise(r => setTimeout(r, 8000));
160
159
  }
161
160
 
@@ -217,10 +216,25 @@ export async function serveCommand(config, args, chalk) {
217
216
  return serveCommand(config, expandedArgs, chalk);
218
217
  }
219
218
 
219
+ if (args.includes('--list-aliases')) {
220
+ console.log(chalk.bold('\n⚡ Blessed vLLM aliases\n'));
221
+ console.log(chalk.dim(' Use any of these in place of a full model ID: badgr serve <alias>\n'));
222
+ for (const [alias, spec] of Object.entries(BLESSED_VLLM_MODELS)) {
223
+ console.log(` ${chalk.cyan(alias.padEnd(16))} ${spec.model_id}`);
224
+ console.log(` ${''.padEnd(16)} ${chalk.dim(spec.description)}`);
225
+ }
226
+ console.log();
227
+ return;
228
+ }
229
+
220
230
  const { model, flags } = parseServeArgs(args);
221
231
  const customImage = flags.image || null;
222
232
  const isLlamaCpp = flags.runtime === 'llama.cpp';
223
233
 
234
+ // Expand blessed alias (qwen-7b, llama-8b, qwen-coder-7b) to full model ID + GPU.
235
+ const vllmAlias = model && !customImage && !isLlamaCpp ? BLESSED_VLLM_MODELS[model] : null;
236
+ const effectiveModel = vllmAlias ? vllmAlias.model_id : model;
237
+
224
238
  // Detect flags that ended up as positional args due to broken shell line continuation
225
239
  // (e.g. `\ ` with a trailing space instead of `\<newline>`).
226
240
  // After parseServeArgs, only the model name should be in positional. Any extra token
@@ -293,7 +307,9 @@ export async function serveCommand(config, args, chalk) {
293
307
 
294
308
  const envObj = parseEnvFlag(flags.env);
295
309
 
296
- const gpu = flags.gpu ? flags.gpu.toUpperCase().replace('-', '_') : (customImage ? 'L40S' : 'AUTO');
310
+ const gpu = flags.gpu
311
+ ? flags.gpu.toUpperCase().replace('-', '_')
312
+ : (vllmAlias ? vllmAlias.gpu_type : (customImage ? 'L40S' : 'AUTO'));
297
313
  const gpuLabel = gpu === 'AUTO' ? 'auto' : gpu;
298
314
 
299
315
  const effectiveTier = normalizeTier(flags.tier);
@@ -319,13 +335,17 @@ export async function serveCommand(config, args, chalk) {
319
335
  if (flags.env?.length) console.log(` ${chalk.bold('Env:')} ${flags.env.join(', ')}`);
320
336
  } else {
321
337
  console.log(chalk.bold('\n⚡ Serving model\n'));
322
- console.log(` ${chalk.bold('Model:')} ${model}`);
338
+ if (vllmAlias) {
339
+ console.log(` ${chalk.bold('Alias:')} ${model} ${chalk.dim(`→ ${effectiveModel}`)}`);
340
+ } else {
341
+ console.log(` ${chalk.bold('Model:')} ${effectiveModel}`);
342
+ }
323
343
  console.log(` ${chalk.bold('GPU:')} ${gpuLabel}`);
324
344
  if (flags.task) console.log(` ${chalk.bold('Task:')} ${flags.task}`);
325
345
  if (flags.env?.length) console.log(` ${chalk.bold('Env:')} ${flags.env.join(', ')}`);
326
346
 
327
347
  if (gpu === 'AUTO') {
328
- const prof = _inferServeProfile(model);
348
+ const prof = _inferServeProfile(effectiveModel);
329
349
  console.log();
330
350
  console.log(` ${chalk.bold('Estimated workload:')} ${prof.label}`);
331
351
  console.log(` ${chalk.bold('Estimated minimum VRAM:')} ${prof.vram}`);
@@ -348,7 +368,7 @@ export async function serveCommand(config, args, chalk) {
348
368
  ACTIVE.has(d.status) &&
349
369
  d.workload_type === 'endpoint' &&
350
370
  (
351
- (model && d.model === model) ||
371
+ (effectiveModel && d.model === effectiveModel) ||
352
372
  (customImage && d.image === customImage) ||
353
373
  (isLlamaCpp && d.image === LLAMA_CPP_IMAGE &&
354
374
  d.env?.LLAMA_ARG_HF_REPO === flags.hfRepo && d.env?.LLAMA_ARG_HF_FILE === flags.hfFile)
@@ -381,7 +401,7 @@ export async function serveCommand(config, args, chalk) {
381
401
  ? { LLAMA_ARG_HF_REPO: flags.hfRepo, LLAMA_ARG_HF_FILE: flags.hfFile, ...envObj }
382
402
  : envObj;
383
403
  return {
384
- ...(model ? { model } : {}),
404
+ ...(effectiveModel ? { model: effectiveModel } : {}),
385
405
  ...(isLlamaCpp ? { image: LLAMA_CPP_IMAGE } : customImage ? { image: customImage } : {}),
386
406
  ...(flags.task ? { task: flags.task } : {}),
387
407
  gpu: gpuOverride || gpu,
@@ -392,6 +412,7 @@ export async function serveCommand(config, args, chalk) {
392
412
  tier: tierOverride || effectiveTier,
393
413
  ...(Object.keys(effectiveEnv).length > 0 ? { env: effectiveEnv } : {}),
394
414
  ...(flags.maxCost ? { max_cost_usd: flags.maxCost } : {}),
415
+ ...(flags.healthPath ? { health_path: flags.healthPath } : {}),
395
416
  };
396
417
  }
397
418
 
@@ -424,7 +445,7 @@ export async function serveCommand(config, args, chalk) {
424
445
  id: dep.deployment_id,
425
446
  name: dep.name,
426
447
  type: 'endpoint',
427
- model: dep.model || model,
448
+ model: dep.model || effectiveModel,
428
449
  gpu: dep.gpu_type,
429
450
  count: dep.gpu_count,
430
451
  status: dep.status,
@@ -510,7 +531,7 @@ export async function serveCommand(config, args, chalk) {
510
531
  // status check failed — proceed with endpoint poll anyway
511
532
  }
512
533
 
513
- const healthResult = await waitForEndpoint(endpointUrl, dep.deployment_id, config, 5 * 60 * 1000, chalk, resolvedHealthPath);
534
+ const healthResult = await waitForEndpoint(dep.deployment_id, config, VLLM_SERVE_WAIT_MS, chalk, resolvedHealthPath);
514
535
  process.stdout.write('\n');
515
536
 
516
537
  if (healthResult.depFailed) {
@@ -556,7 +577,7 @@ export async function serveCommand(config, args, chalk) {
556
577
  console.log(` ${chalk.bold('HF Repo:')} ${flags.hfRepo}`);
557
578
  console.log(` ${chalk.bold('HF File:')} ${flags.hfFile}`);
558
579
  }
559
- else if (dep.model || model) console.log(` ${chalk.bold('Model:')} ${dep.model || model}`);
580
+ else if (dep.model || effectiveModel) console.log(` ${chalk.bold('Model:')} ${dep.model || effectiveModel}`);
560
581
  if (customImage) console.log(` ${chalk.bold('Image:')} ${customImage}`);
561
582
  console.log(` ${chalk.bold('GPU:')} ${dep.gpu_type} × ${dep.gpu_count}`);
562
583
  if (serveRate > 0) console.log(` ${chalk.bold('Rate:')} $${serveRate.toFixed(2)}/hr`);
@@ -574,7 +595,7 @@ export async function serveCommand(config, args, chalk) {
574
595
 
575
596
  if (endpointReady && !customImage) {
576
597
  const keySnip = config.apiKey?.slice(0, 4) || 'sk-...';
577
- const sdkModel = isLlamaCpp ? 'default' : (dep.model || model);
598
+ const sdkModel = isLlamaCpp ? 'default' : (dep.model || effectiveModel);
578
599
  console.log(` ${chalk.bold('Use with OpenAI SDK:')}`);
579
600
  console.log(chalk.dim(` from openai import OpenAI`));
580
601
  console.log(chalk.dim(` client = OpenAI(base_url="${endpointUrl}", api_key="${keySnip}...")`));