runbios-mcp 0.5.0 → 0.6.0-dev.368
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.md +4 -4
- package/dist/config.d.ts +1 -1
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +1 -1
- package/dist/config.js.map +1 -1
- package/dist/http/assistant.js +1 -1
- package/dist/http/assistant.js.map +1 -1
- package/dist/http/metadata.js +1 -1
- package/dist/http/metadata.js.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +117 -221
- package/dist/server.js.map +1 -1
- package/dist/training-contract.d.ts +6 -0
- package/dist/training-contract.d.ts.map +1 -1
- package/dist/training-contract.js +6 -0
- package/dist/training-contract.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.d.ts.map +1 -1
- package/dist/version.js +1 -1
- package/dist/version.js.map +1 -1
- package/package.json +1 -1
package/dist/server.js
CHANGED
|
@@ -844,8 +844,6 @@ launchGates = resolveLaunchGates(), allowedTools) {
|
|
|
844
844
|
epochs: z.number().positive().max(1000).optional().describe("Number of training epochs. Governs the full run when max_steps is omitted (the default)."),
|
|
845
845
|
max_steps: z.number().int().min(1).max(10_000_000).optional().describe("Optional hard step cap that overrides epochs. Omit for no cap and all configured epochs; setting less than one epoch intentionally trains only part of the data."),
|
|
846
846
|
learning_rate: z.number().positive().max(1).optional().describe("Learning rate."),
|
|
847
|
-
batch_size: z.number().int().min(1).max(65536).optional().describe("Per-device train batch size."),
|
|
848
|
-
gradient_accumulation_steps: z.number().int().min(1).max(65536).optional(),
|
|
849
847
|
max_seq_length: trainingSeqLengthSchema(),
|
|
850
848
|
lora_rank: z.number().int().min(1).max(4096).optional(),
|
|
851
849
|
lora_alpha: z.number().positive().max(1_000_000).optional(),
|
|
@@ -857,6 +855,12 @@ launchGates = resolveLaunchGates(), allowedTools) {
|
|
|
857
855
|
.describe("Learning-rate scheduler."),
|
|
858
856
|
job_name: z.string().max(255).optional(),
|
|
859
857
|
eval_split_ratio: z.number().min(0).max(0.99).optional(),
|
|
858
|
+
validation_rows: z.number().int().min(0).max(50_000_000).optional()
|
|
859
|
+
.describe("Validation rows held out and never trained on (0 turns validation off). Allowed: 100 (or 5% when that is fewer) up to 40% of the training rows; unset, 5% capped at 1,000. Preflight returns the exact range. Do not send with eval_split_ratio."),
|
|
860
|
+
precision: z.enum(["bf16", "fp8"]).optional()
|
|
861
|
+
.describe("Training precision. Offer only what get_managed_training_options lists for the model and adapter; bf16 is the default."),
|
|
862
|
+
optim: z.enum(["adamw_torch_fused", "adamw_8bit"]).optional()
|
|
863
|
+
.describe("Optimizer state: adamw_torch_fused (16-bit, default, most accurate) or adamw_8bit (less memory). Offer only what get_managed_training_options lists."),
|
|
860
864
|
eval_steps: z.number().int().positive().optional().describe("Evaluate every N optimizer steps when a validation split exists."),
|
|
861
865
|
save_steps: z.number().int().positive().optional().describe("Checkpoint interval in optimizer steps."),
|
|
862
866
|
early_stopping_patience: z.number().int().min(0).max(20).optional()
|
|
@@ -918,7 +922,7 @@ on.
|
|
|
918
922
|
8. loop_list_models - The models a pipeline can train, and the platform's fixed sample minimums (floors). The only list a pipeline's model may come from; unavailable says, in one plain sentence per model, why one is not offered.
|
|
919
923
|
9. loop_create_pipeline - One use case: a name, a model (optionally a second model, a different one), its tags, when to train (optionally a schedule; the sample minimums are fixed by the platform) and how a new version goes live. It switches on the source named by its pipeline ID, so an app can send samples at once. SPEND CONSENT: saving it authorises trainings on the workspace's wallet whenever its data has grown enough; the platform chooses the GPUs, and what is charged is the GPU time its trainings use and, where the platform scores samples, the judge's scoring of unreviewed samples, shown on the training it was for. Say so and wait for a yes.
|
|
920
924
|
10. loop_list_pipelines / loop_get_pipeline / loop_list_pipeline_attempts / loop_update_pipeline / loop_delete_pipeline - Each pipeline's versions (only trainings that went live: v1, v2 ...: version version numbers count up; nothing untested goes live any more, and each version's tested says whether it was), its trainings and how each ended (the latest on the pipeline, every one a page at a time with loop_list_pipeline_attempts, each with an explanation in plain words), the live version and whether its deployment answers (live_deployment), status and next_reason, and its settings. Deleting a pipeline stops its live version's deployment (unless another pipeline or model name answers from it too) and deletes its history.
|
|
921
|
-
11. loop_run_pipeline / loop_import_into_pipeline - Train now (skips the schedule, never the minimum), and bring a file in as this pipeline's samples.
|
|
925
|
+
11. loop_run_pipeline / loop_import_into_pipeline - Train now (skips the schedule, never the minimum), and bring a file in as this pipeline's samples. loop_list_live_deployments / loop_use_live_deployment - when the live version's deployment was deleted, find a deployment of its weights and answer the app from it; nothing is trained again.
|
|
922
926
|
12. loop_create_benchmark / loop_list_benchmarks / loop_get_benchmark / loop_list_benchmark_items / loop_get_benchmark_history / loop_retire_benchmark / loop_get_benchmark_run / loop_list_benchmark_run_items / loop_benchmark_questions - Benchmarks: fixed sets of the user's questions with right answers that every training is tested on, so "is it getting better" has a yardstick that does not move, and each benchmark run's report: its scores, how many questions each model got right (score 0.70 or more), and every question behind them. loop_benchmark_questions shows each question with how a pipeline's recent trainings did on it.
|
|
923
927
|
13. loop_list_pipeline_benchmarks / loop_attach_pipeline_benchmark / loop_move_pipeline_benchmark / loop_remove_pipeline_benchmark / loop_compare_versions / loop_compare_attempts - A pipeline's benchmarks in priority order (1 is the primary), its versions side by side (v1 against v5), and any two trainings side by side, live or not, question by question on the automatic-test questions both were scored on.
|
|
924
928
|
14. loop_list_training_runs / loop_get_training_run / loop_get_training_curve - Every training of every pipeline (result filters by what became of it; each item's 'attempt' is the training as its pipeline's own list says it): its timeline, why it went live or did not (its going-live record, and verdict_checks: every bar its test needed, what it got and whether it was met), what each part cost, and available_actions — what this reader may actually do next, rather than what the state looks like it allows. loop_get_training_curve shows how one went while it trained, step by step, and the step it kept.
|
|
@@ -1172,6 +1176,7 @@ get_inference_status instead of asserting them.
|
|
|
1172
1176
|
5. get_model_config - Get recommended settings for your chosen model
|
|
1173
1177
|
6. list_supported_architectures - Check which architectures can be trained or served
|
|
1174
1178
|
7. get_training_capabilities - Read the authoritative training contract; call this before building automated payloads
|
|
1179
|
+
7a. get_managed_training_options - With managed_training on: the context lengths, precisions and optimizer states a model can be trained with; offer only these
|
|
1175
1180
|
8. get_gpu_pricing - See GPU options and pricing
|
|
1176
1181
|
9. get_recommended_gpu - Read default-config sizing; use full preflight for the actual run
|
|
1177
1182
|
9a. recommend_training_config - Ask the trainer's sizing model for batch, accumulation, LR, predicted peak memory, minimum GPU count and wall clock on the GPU you picked; check your own batch size against it
|
|
@@ -1232,25 +1237,18 @@ Option B: Import from HuggingFace — call import_huggingface_dataset with a rep
|
|
|
1232
1237
|
|
|
1233
1238
|
Dataset upload accepts one file type: JSONL. SFT rows may use messages, conversations, instruction/output, prompt/completion, query/response, input/output, question/answer, or problem/solution. CPT rows require a non-empty text field. Preference, reward, KTO, GRPO, PPO, and GKD datasets are refused while those training methods are unavailable.
|
|
1234
1239
|
|
|
1235
|
-
###
|
|
1236
|
-
Call
|
|
1240
|
+
### Managed training (check first)
|
|
1241
|
+
Call get_training_capabilities. If its features list has managed_training enabled, the platform chooses the hardware: do NOT choose or send a GPU, GPU count, per-device batch, gradient accumulation, recompute level, checkpoint count or price cap -- they are ignored (preflight warns FIELD_IGNORED). Call get_managed_training_options for the model, method and adapter, and offer only what it returns: the context length (max_length, one of 2048-131072), precision (bf16 or fp8) and optimizer state (optim: adamw_torch_fused = 16-bit, adamw_8bit = 8-bit). Send only: model, method (sft/cpt), adapter (lora/qlora/full), dataset(s), max_length, precision, optim, learning_rate, lr_scheduler_type, warmup_ratio, weight_decay, epochs or max_steps, lora_rank (8-64), lora_alpha, and optionally validation_rows (0 turns validation off) and early_stopping_patience. On preflight_training_job and create_training_job these are the inputs max_seq_length, precision, optim, scheduler, validation_rows. Validation rows must be between 100 (or 5% of the rows when that is under 100) and 40% of the training rows; unset, the job holds out 5% capped at 1,000 rows, and preflight returns the exact rows and range in validation. Unset, the schedule is cosine; it keeps its best (lowest validation loss) and latest checkpoints, is charged per trained token only, and waits unbilled in the queue, first come first served, when no machine is free (the customer is emailed when it starts and may cancel it while queued). Training logs and machine metrics (memory, MFU, throughput) are not available to the customer: report progress from get_training_status and training metrics from get_training_metrics. Samples longer than max_length lose their oldest turns, never the answer; image samples that do not fit are skipped. Preflight returns the placement validity and, when token_billing is enabled, an estimate (estimated tokens and cost at the model's per-token price; the charge is the exact tokens trained). Customers never see hardware; do not describe GPUs to them.
|
|
1237
1242
|
|
|
1238
|
-
### Step
|
|
1239
|
-
Call preflight_training_job with the model, dataset(s), method, adapter and
|
|
1243
|
+
### Step 4: Preflight
|
|
1244
|
+
Call preflight_training_job with the model, dataset(s), method, adapter and training settings. It returns valid, the validation rows and their allowed range, warnings and the per-token cost estimate. Show the estimate to the user before creating.
|
|
1240
1245
|
|
|
1241
|
-
### Step
|
|
1242
|
-
Call create_training_job with
|
|
1243
|
-
- model: the model name from step 2
|
|
1244
|
-
- dataset_id: the ID from step 3
|
|
1245
|
-
- method: "sft" for instruction tuning (most common)
|
|
1246
|
-
- adapter: "lora" (fastest, recommended for first jobs)
|
|
1247
|
-
- Leave other params as defaults unless you have specific needs
|
|
1246
|
+
### Step 5: Create the training job
|
|
1247
|
+
Call create_training_job with the same request. The job is placed on tested machines; when none is free it is queued (first come first served, unbilled) and starts automatically. The user is emailed when it starts and can cancel at any time with stop_training_job.
|
|
1248
1248
|
|
|
1249
|
-
### Step
|
|
1250
|
-
|
|
1251
|
-
|
|
1252
|
-
If a started job loses its pod it rests at 'interrupted' (billing already stopped, not 'failed') — call resume_training_job to continue from the last checkpoint.
|
|
1253
|
-
Call get_training_metrics for semantic summaries first; request a bounded series when you need to inspect curve shape. Follow each metric's returned goal instead of assuming every number should decrease.
|
|
1249
|
+
### Step 6: Monitor progress
|
|
1250
|
+
Call get_training_status for progress (queued, getting ready, downloading the model, preparing data, training step N of M, saving). Call get_training_metrics for loss, eval loss, gradient norm, token accuracy, learning rate and tokens trained. The job keeps two checkpoints: the best (lowest validation loss) and the last.
|
|
1251
|
+
If a started job loses its machine it rests at 'interrupted' (nothing is charged for that) -- call resume_training_job to continue from the last checkpoint.
|
|
1254
1252
|
|
|
1255
1253
|
### Tips
|
|
1256
1254
|
- Start with LoRA, not full fine-tune — it's 10x faster and 5x cheaper
|
|
@@ -1357,25 +1355,21 @@ When to use: Before SFT, when your domain has specialized vocabulary/knowledge.
|
|
|
1357
1355
|
- A VLM architecture does not imply support for image-only pretraining or multimodal preference objectives.
|
|
1358
1356
|
- DPO, PPO and GRPO are not enabled by these training tools. They require separately verified training implementations; PPO/GRPO also require rollout inference and reward handling.
|
|
1359
1357
|
- Training starts from catalog models. Do not promise a CPT-to-SFT pipeline using a customer's trained checkpoint as a new training source: that workflow is not exposed here.`,
|
|
1360
|
-
gpu_selection: `#
|
|
1361
|
-
|
|
1362
|
-
|
|
1363
|
-
|
|
1364
|
-
|
|
1365
|
-
|
|
1366
|
-
|
|
1367
|
-
|
|
1368
|
-
|
|
1369
|
-
##
|
|
1370
|
-
-
|
|
1371
|
-
-
|
|
1372
|
-
|
|
1373
|
-
|
|
1374
|
-
|
|
1375
|
-
## Memory and quality
|
|
1376
|
-
- Memory depends on the full configuration and input shapes, not parameter count alone. Do not invent a smaller minimum or claim that quantization always halves total memory.
|
|
1377
|
-
- Full fine-tuning is not a guarantee of higher accuracy than adapters. Use representative validation to compare outcomes.
|
|
1378
|
-
- Preflight admission is not proof of universal freedom from hardware faults or OOM. Report actual tested configurations and measured headroom.`,
|
|
1358
|
+
gpu_selection: `# Machines (managed training)
|
|
1359
|
+
|
|
1360
|
+
The platform chooses the machines. Do not choose, ask for, or describe GPUs, GPU counts, batch size, gradient accumulation, memory settings, checkpoint counts or price caps: the training tools do not take them, and the API ignores them with a FIELD_IGNORED warning.
|
|
1361
|
+
|
|
1362
|
+
## How a job is placed
|
|
1363
|
+
- Every offered combination of model, adapter, context length, precision and optimizer was measured on our machines first. get_managed_training_options lists exactly what is offered for a model and adapter; anything else is refused before any money is spent, with a reason.
|
|
1364
|
+
- The platform picks the fastest tested setup that fits the job, with batch and memory settings that keep it from running out of memory, and checks the run's memory again before it starts.
|
|
1365
|
+
- Short samples are packed together into each context window, so a longer context costs no speed.
|
|
1366
|
+
|
|
1367
|
+
## When no machine is free
|
|
1368
|
+
- The job waits in the queue, first come first served with fair turns per user. Waiting is never charged and a queued request expires after seven days.
|
|
1369
|
+
- It starts automatically and the user is emailed when it does. It can be cancelled at any time, queued or running.
|
|
1370
|
+
|
|
1371
|
+
## What the user pays
|
|
1372
|
+
- Only for the exact tokens trained, at the model's per-token price. preflight_training_job returns the estimate.`,
|
|
1379
1373
|
serverless_inference: `# Serverless Inference
|
|
1380
1374
|
|
|
1381
1375
|
Serverless is the immediate, per-token inference product. It does NOT create or manage a deployment and does not reserve a GPU.
|
|
@@ -1490,149 +1484,55 @@ ignored rather than honored — the server derives them again either way.
|
|
|
1490
1484
|
- Training and deployment use the shared capacity broker. It evaluates priority first, then rotates fairly across users at equal priority while preserving FIFO inside each user's own requests. This prevents a high-volume user from starving others; do not describe the queue as strict global FIFO.`,
|
|
1491
1485
|
hyperparameters: `# Hyperparameter Guide
|
|
1492
1486
|
|
|
1493
|
-
|
|
1494
|
-
|
|
1495
|
-
### Learning
|
|
1496
|
-
|
|
1497
|
-
|
|
1498
|
-
|
|
1499
|
-
|
|
1500
|
-
|
|
1501
|
-
|
|
1502
|
-
|
|
1503
|
-
|
|
1504
|
-
|
|
1505
|
-
|
|
1506
|
-
|
|
1507
|
-
|
|
1508
|
-
|
|
1509
|
-
|
|
1510
|
-
|
|
1511
|
-
|
|
1512
|
-
|
|
1513
|
-
|
|
1514
|
-
|
|
1515
|
-
|
|
1516
|
-
|
|
1517
|
-
|
|
1518
|
-
- Default: 16
|
|
1519
|
-
- Range: 4-256
|
|
1520
|
-
- Higher = more trainable parameters, better quality, but slower
|
|
1521
|
-
- 8-16 is sufficient for most tasks
|
|
1522
|
-
- 32-64 for complex tasks requiring more model capacity
|
|
1523
|
-
- 128+ for near-full-fine-tune quality
|
|
1524
|
-
|
|
1525
|
-
### LoRA Alpha (lora_alpha)
|
|
1526
|
-
- Default: 32 (typically 2x lora_rank)
|
|
1527
|
-
- Controls the scaling of LoRA updates
|
|
1528
|
-
- Rule of thumb: set to 2x your lora_rank
|
|
1529
|
-
|
|
1530
|
-
### Max Sequence Length
|
|
1531
|
-
- Default: 2048
|
|
1532
|
-
- Range: 512-8192
|
|
1533
|
-
- Must cover your longest training example
|
|
1534
|
-
- Longer = more VRAM usage
|
|
1535
|
-
- Set to slightly above your longest example
|
|
1536
|
-
|
|
1537
|
-
### LR Scheduler
|
|
1538
|
-
- cosine (default) — gradually decreases, good for most tasks
|
|
1539
|
-
- linear — constant decrease, predictable
|
|
1540
|
-
- constant_with_warmup — stays flat after warmup, good for short training
|
|
1541
|
-
|
|
1542
|
-
### Warmup Ratio
|
|
1543
|
-
- Default: 0.03 (3% of total steps)
|
|
1544
|
-
- Prevents early instability by slowly ramping up learning rate
|
|
1545
|
-
|
|
1546
|
-
## Recommended Defaults by Use Case
|
|
1547
|
-
| Parameter | Quick Test | Standard | High Quality |
|
|
1548
|
-
|-----------|-----------|----------|--------------|
|
|
1549
|
-
| Adapter | LoRA | LoRA | Full |
|
|
1550
|
-
| Learning Rate | 3e-4 | 2e-4 | 2e-5 |
|
|
1551
|
-
| Epochs | 1 | 3 | 5 |
|
|
1552
|
-
| Batch Size | 4 | 4 | 8 |
|
|
1553
|
-
| LoRA Rank | 8 | 16 | 64 |
|
|
1554
|
-
| Scheduler | cosine | cosine | cosine |`,
|
|
1487
|
+
Offer only the settings below. Ranges and the context lengths, precisions and optimizers a model offers come from get_training_capabilities and get_managed_training_options.
|
|
1488
|
+
|
|
1489
|
+
### Learning rate (learning_rate)
|
|
1490
|
+
How big a step the model takes each time it learns. Start with 2e-4 for LoRA and QLoRA and 1e-5 to 2e-5 for full fine-tuning. Lower it if the loss jumps around.
|
|
1491
|
+
|
|
1492
|
+
### Epochs (epochs) and max steps (max_steps)
|
|
1493
|
+
How many times the model reads the whole dataset. One to three is usual: small datasets (under 1,000 rows) 3 to 5, large datasets 1 to 3. max_steps is an optional cap; leave it empty to train every epoch.
|
|
1494
|
+
|
|
1495
|
+
### Context length (max_length)
|
|
1496
|
+
The longest text read in one go, in tokens: 2K, 4K, 8K, 16K, 32K, 64K or 128K where the model offers it. Pick one that fits the longest examples. Longer samples lose their oldest turns, never the answer.
|
|
1497
|
+
|
|
1498
|
+
### Precision (precision)
|
|
1499
|
+
bf16 is accurate and fast and right for most runs. fp8 can be faster on large models and is offered only where it was tested.
|
|
1500
|
+
|
|
1501
|
+
### Optimizer state (optim)
|
|
1502
|
+
adamw_torch_fused keeps 16-bit state and is the most accurate. adamw_8bit uses less memory, which helps large full fine-tunes, and is offered only where it was tested.
|
|
1503
|
+
|
|
1504
|
+
### LoRA rank (lora_rank) and alpha (lora_alpha)
|
|
1505
|
+
Rank is how much the adapter can learn (8 to 64; 16 is a good start). Alpha is usually twice the rank.
|
|
1506
|
+
|
|
1507
|
+
### Warmup ratio, weight decay and schedule
|
|
1508
|
+
warmup_ratio (default 0.03) eases the first steps in. weight_decay (around 0.01) helps the model generalize. lr_scheduler_type defaults to cosine.
|
|
1509
|
+
|
|
1510
|
+
### Validation rows (split_dataset_samples) and early stopping (early_stopping_patience)
|
|
1511
|
+
Rows held out and never trained on, used to measure real progress and keep the best checkpoint. Allowed: 100 rows (or 5% when that is fewer) up to 40% of the rows; default 5% capped at 1,000. Early stopping ends the run after that many evaluations without improvement (default 3).`,
|
|
1555
1512
|
monitoring: `# Training Monitoring Guide
|
|
1556
1513
|
|
|
1557
|
-
##
|
|
1558
|
-
|
|
1559
|
-
|
|
1560
|
-
|
|
1561
|
-
|
|
1562
|
-
|
|
1563
|
-
|
|
1564
|
-
|
|
1565
|
-
|
|
1566
|
-
|
|
1567
|
-
-
|
|
1568
|
-
|
|
1569
|
-
|
|
1570
|
-
-
|
|
1571
|
-
|
|
1572
|
-
|
|
1573
|
-
|
|
1574
|
-
-
|
|
1575
|
-
-
|
|
1576
|
-
|
|
1577
|
-
|
|
1578
|
-
|
|
1579
|
-
2. get_training_metrics — Semantic metric summaries plus bounded curve data on request
|
|
1580
|
-
3. get_training_logs — Raw training output for debugging
|
|
1581
|
-
4. get_training_checkpoints — Saved model snapshots at intervals
|
|
1582
|
-
|
|
1583
|
-
## When to Stop Early
|
|
1584
|
-
- Loss has plateaued for many steps (no improvement)
|
|
1585
|
-
- Eval loss is increasing (overfitting)
|
|
1586
|
-
- You've reached satisfactory quality and want to save GPU cost
|
|
1587
|
-
- The model is producing good outputs in eval samples
|
|
1588
|
-
|
|
1589
|
-
## Checkpoints
|
|
1590
|
-
- Saved automatically at regular intervals during training
|
|
1591
|
-
- Each checkpoint is a complete model snapshot you can use
|
|
1592
|
-
- If training is interrupted, resume from the last checkpoint
|
|
1593
|
-
- Best practice: compare outputs from different checkpoints to pick the best one`,
|
|
1594
|
-
cost_optimization: `# Cost Optimization Guide
|
|
1595
|
-
|
|
1596
|
-
## Per-Second Billing
|
|
1597
|
-
Run BiOS charges per-SECOND of GPU usage, not per hour. This means:
|
|
1598
|
-
- You only pay for actual compute time
|
|
1599
|
-
- Stopping a job immediately stops billing
|
|
1600
|
-
- No wasted time rounding up to the next hour
|
|
1601
|
-
|
|
1602
|
-
## Cost Estimation
|
|
1603
|
-
Before launching a job, estimate cost:
|
|
1604
|
-
1. Call get_recommended_gpu for your model — it includes a cost estimate
|
|
1605
|
-
2. Rough formula: (dataset_rows / batch_size) × epochs × seconds_per_step × gpu_price_per_second
|
|
1606
|
-
|
|
1607
|
-
## Money-Saving Strategies
|
|
1608
|
-
|
|
1609
|
-
### 1. Start Small
|
|
1610
|
-
- Train for 1 epoch first to validate your setup
|
|
1611
|
-
- Check if the loss is decreasing and outputs look reasonable
|
|
1612
|
-
- Only then train for the full 3-5 epochs
|
|
1613
|
-
|
|
1614
|
-
### 2. Use LoRA
|
|
1615
|
-
- LoRA is 5-10x cheaper than full fine-tune
|
|
1616
|
-
- Quality is comparable for most tasks
|
|
1617
|
-
- QLoRA is even cheaper (uses 4-bit quantization)
|
|
1618
|
-
|
|
1619
|
-
### 3. Optimize Batch Size
|
|
1620
|
-
- Larger batch sizes = fewer total steps = less time
|
|
1621
|
-
- Use gradient accumulation to simulate large batches on limited VRAM
|
|
1622
|
-
|
|
1623
|
-
### 4. Choose the Right GPU
|
|
1624
|
-
- Don't pick H100 for a 7B LoRA job — A100 is plenty and cheaper
|
|
1625
|
-
- Use get_recommended_gpu to find the cost-optimal choice
|
|
1626
|
-
|
|
1627
|
-
### 5. Monitor and Stop Early
|
|
1628
|
-
- Read the summaries from get_training_metrics, then request a bounded series for any curve that needs inspection
|
|
1629
|
-
- Treat a plateau as evidence to review alongside eval metrics and the configured run bound, not an automatic stop verdict
|
|
1630
|
-
- Check wallet balance with get_wallet_balance before long jobs
|
|
1631
|
-
|
|
1632
|
-
### 6. Check Your Balance First
|
|
1633
|
-
- Call get_wallet_balance before creating a job
|
|
1634
|
-
- Ensure you have enough credits for the estimated training time
|
|
1635
|
-
- Set up auto top-up to avoid interruptions mid-training`,
|
|
1514
|
+
## Tools
|
|
1515
|
+
1. get_training_status -- where the job is: queued, getting ready, downloading the model, preparing data, training step N of M, saving, finished.
|
|
1516
|
+
2. get_training_metrics -- training metrics only: loss, eval loss, gradient norm, token accuracy, perplexity, learning rate, epoch and tokens trained. Read the summaries first and ask for a bounded series when you need the curve.
|
|
1517
|
+
3. get_training_checkpoints -- the kept checkpoints: the best (lowest validation loss) and the last.
|
|
1518
|
+
|
|
1519
|
+
Training logs and machine metrics (memory, MFU, throughput) are not available to customers.
|
|
1520
|
+
|
|
1521
|
+
## Reading the curves
|
|
1522
|
+
- Training loss should fall. A sharp early drop is normal.
|
|
1523
|
+
- Eval loss should follow it. If eval loss rises while training loss falls the model is starting to memorize; early stopping ends the run and the best checkpoint is kept.
|
|
1524
|
+
- The learning rate follows the schedule: warmup, then the decay.`,
|
|
1525
|
+
cost_optimization: `# Cost Guide
|
|
1526
|
+
|
|
1527
|
+
## Per-token pricing
|
|
1528
|
+
Training is charged for the exact tokens it trained, at the model's per-token price for the adapter. Waiting in the queue, setup and machine time are never charged. preflight_training_job returns the estimate; the charge after the run is the exact count.
|
|
1529
|
+
|
|
1530
|
+
## Spending less
|
|
1531
|
+
- Train 1 epoch first to check the setup, then the full run.
|
|
1532
|
+
- LoRA and QLoRA cost less per token than full fine-tuning and are close in quality for most tasks.
|
|
1533
|
+
- Early stopping (on by default when validation rows are held out) ends a run once it stops improving.
|
|
1534
|
+
- max_steps caps a run when you only want part of the data.
|
|
1535
|
+
- Check get_wallet_balance before long runs.`,
|
|
1636
1536
|
capacity_errors: `# Standard GPU Rejection Errors
|
|
1637
1537
|
|
|
1638
1538
|
Training and inference speak ONE GPU rejection contract: one body shape, and a
|
|
@@ -2559,6 +2459,21 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2559
2459
|
return json(stripBuildIdentity(data));
|
|
2560
2460
|
});
|
|
2561
2461
|
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2462
|
+
/* TOOL: get_managed_training_options */
|
|
2463
|
+
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2464
|
+
server.tool("get_managed_training_options", "Managed training only (get_training_capabilities lists managed_training enabled): what this model, method and adapter can be trained with. Returns options, each a precision (bf16 or fp8) and optimizer state (adamw_torch_fused = 16-bit, adamw_8bit = 8-bit) with the context lengths (2048 ... 131072) a certified setup covers. Offer the customer only these; every option listed is accepted by preflight_training_job and create_training_job. Empty options means the model is not trainable with that method and adapter yet. No hardware is named and none should be described to the customer.", {
|
|
2465
|
+
model_id: z.string().describe("Model id, e.g. Qwen/Qwen3.5-9B."),
|
|
2466
|
+
method: z.enum(TRAINING_METHODS).optional().describe("Training method: sft (default), or pt/cpt for continued pre-training."),
|
|
2467
|
+
adapter: z.enum(TRAINING_ADAPTERS).optional().describe("lora (default), qlora or full."),
|
|
2468
|
+
}, async ({ model_id, method, adapter }) => {
|
|
2469
|
+
const q = new URLSearchParams({
|
|
2470
|
+
model_id,
|
|
2471
|
+
training_method: method === "cpt" ? "pt" : method || "sft",
|
|
2472
|
+
train_type: adapter || "lora",
|
|
2473
|
+
});
|
|
2474
|
+
return json(await client.api(`/api/training/managed-options?${q}`));
|
|
2475
|
+
});
|
|
2476
|
+
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2562
2477
|
/* TOOL: preflight_training_job */
|
|
2563
2478
|
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2564
2479
|
/**
|
|
@@ -2602,7 +2517,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2602
2517
|
...(params.eval_dataset_ids ? { eval_dataset_ids: await Promise.all(params.eval_dataset_ids.map((ref) => resolveDatasetRef(ref, "eval_dataset_ids"))) } : {}),
|
|
2603
2518
|
};
|
|
2604
2519
|
}
|
|
2605
|
-
server.tool("preflight_training_job", "Validate and canonicalize a proposed training job without creating a job
|
|
2520
|
+
server.tool("preflight_training_job", "Validate and canonicalize a proposed training job without creating a job or charging the wallet. You choose the model, adapter, datasets and training settings; the platform chooses the machines, batch and memory settings from tested setups. Returns dataset compatibility, the per-token cost estimate, the validation rows it will hold out (and their allowed range), warnings (FIELD_IGNORED for anything the platform sets) and a request hash.", {
|
|
2606
2521
|
model: z.string().describe("Base model id from the Run BiOS catalog (e.g., 'meta-llama/Llama-3.1-8B-Instruct'). Must be hosted on Run BiOS — pick from list_models/search_models. Training always starts from a catalog base model; start_from (where enabled) adds earlier LoRA versions of that same base on top of it, and training from any other fine-tuned model is not supported."),
|
|
2607
2522
|
model_revision: z.string().max(256).optional().describe("Requested model branch, tag, or commit. Copy the exact 40-hex value returned in canonical_request into create."),
|
|
2608
2523
|
dataset_id: z.string().optional().describe("Dataset id from list_datasets. An exact, unique dataset name is also accepted and resolved to its id."),
|
|
@@ -2610,25 +2525,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2610
2525
|
eval_dataset_ids: z.array(z.string()).min(1).max(64).optional().describe("Separate evaluation datasets, composed sequentially. Mutually exclusive with eval_split_ratio."),
|
|
2611
2526
|
method: z.enum(TRAINING_METHODS),
|
|
2612
2527
|
adapter: z.enum(TRAINING_ADAPTERS).optional(),
|
|
2613
|
-
gpu_type: z.string().optional(),
|
|
2614
|
-
gpu_count: z.number().int().min(1).max(8).optional(),
|
|
2615
|
-
gpu_priorities: z
|
|
2616
|
-
.array(z.object({
|
|
2617
|
-
gpu_type: z.string(), gpu_count: z.number().int().min(1).max(8),
|
|
2618
|
-
provider: z.string().min(1).max(30).optional(), region: z.string().min(1).max(64).optional(),
|
|
2619
|
-
tier: z.literal("secure").optional(),
|
|
2620
|
-
}))
|
|
2621
|
-
.min(1)
|
|
2622
|
-
.max(5)
|
|
2623
|
-
.optional()
|
|
2624
|
-
.describe("Ranked GPU placement choices. Copy these unchanged from preflight; queueing accepts one exact placement through five ranked choices, and entry one must match gpu_type/gpu_count."),
|
|
2625
|
-
queue_if_unavailable: z.boolean().optional().describe("Explicit consent to wait in the capacity queue when none of the ranked GPU choices are free; accepts 1 to 5 gpu_priorities because some training configurations have one valid shape."),
|
|
2626
|
-
queue_deadline: z.string().min(1).optional().describe("Optional RFC 3339 deadline from one minute through seven days in the future; requires queue_if_unavailable=true."),
|
|
2627
|
-
max_price_hour_cents: priceCapSchema(true),
|
|
2628
|
-
storage_gb: z.number().int().min(1).max(10000).optional(),
|
|
2629
|
-
num_checkpoints: z.number().int().min(1).max(100).optional(),
|
|
2630
2528
|
integration_id: z.string().optional(),
|
|
2631
|
-
network_volume_id: z.string().optional(),
|
|
2632
2529
|
dataset_mixing: z.enum(["shuffle", "sequential", "interleave"]).optional(),
|
|
2633
2530
|
mixing: datasetMixingSchema.optional().describe("Composition plan. Dataset IDs define source order and weight positions; default seed is 42. The source pins, plan and checksum are retained for metadata-only resume."),
|
|
2634
2531
|
start_from: startFromSchema.optional(),
|
|
@@ -2649,7 +2546,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2649
2546
|
// server-side (403 TRAINING_COMING_SOON before any billing or booking) as the
|
|
2650
2547
|
// authority. At launch the flag opens and the registration returns verbatim.
|
|
2651
2548
|
if (!launchGates.training) {
|
|
2652
|
-
server.tool("create_training_job", "Create a training job
|
|
2549
|
+
server.tool("create_training_job", "Create a training job with the same request preflight_training_job validated. The platform places it on tested machines; when none is free it waits in the queue (first come first served, unbilled, expires after seven days), starts automatically and emails the user when it starts. It is charged for the exact tokens it trained. Report progress from get_training_status and training metrics from get_training_metrics; cancel any time with stop_training_job. Reuse one idempotency key after timeouts.", {
|
|
2653
2550
|
model: z.string().describe("Base model id from the Run BiOS catalog (e.g., 'meta-llama/Llama-3.1-8B-Instruct'). Must be hosted on Run BiOS — pick from list_models/search_models. Training always starts from a catalog base model; start_from (where enabled) adds earlier LoRA versions of that same base on top of it, and training from any other fine-tuned model is not supported."),
|
|
2654
2551
|
model_revision: z.string().max(256).optional().describe("Exact model_revision from preflight canonical_request; a branch or tag is accepted but resolved again before any paid mutation."),
|
|
2655
2552
|
idempotency_key: z
|
|
@@ -2664,25 +2561,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2664
2561
|
eval_dataset_ids: z.array(z.string()).min(1).max(64).optional().describe("Separate evaluation datasets, composed sequentially. Mutually exclusive with eval_split_ratio."),
|
|
2665
2562
|
method: z.enum(TRAINING_METHODS).describe("Training method: sft, or pt/cpt for continued pre-training."),
|
|
2666
2563
|
adapter: z.enum(TRAINING_ADAPTERS).optional().describe("Any adapter supported by the Run BiOS contract."),
|
|
2667
|
-
gpu_type: z.string().optional().describe("GPU type returned by preflight_training_job after the complete training configuration is known. Do not guess or use a pre-configuration recommendation."),
|
|
2668
|
-
gpu_count: z.number().int().min(1).max(8).optional().describe("Number of GPUs for the primary choice."),
|
|
2669
|
-
gpu_priorities: z
|
|
2670
|
-
.array(z.object({
|
|
2671
|
-
gpu_type: z.string(), gpu_count: z.number().int().min(1).max(8),
|
|
2672
|
-
provider: z.string().min(1).max(30).optional(), region: z.string().min(1).max(64).optional(),
|
|
2673
|
-
tier: z.literal("secure").optional(),
|
|
2674
|
-
}))
|
|
2675
|
-
.min(1)
|
|
2676
|
-
.max(5)
|
|
2677
|
-
.optional()
|
|
2678
|
-
.describe("Ranked GPU placement choices. Copy these unchanged from preflight; queueing accepts one exact placement through five ranked choices, and entry one must match gpu_type/gpu_count."),
|
|
2679
|
-
queue_if_unavailable: z.boolean().optional().describe("Explicit consent to wait in the capacity queue when none of the ranked GPU choices are free; accepts 1 to 5 gpu_priorities because some training configurations have one valid shape."),
|
|
2680
|
-
queue_deadline: z.string().min(1).optional().describe("Optional RFC 3339 deadline from one minute through seven days in the future; requires queue_if_unavailable=true."),
|
|
2681
|
-
max_price_hour_cents: priceCapSchema(true),
|
|
2682
|
-
storage_gb: z.number().int().min(1).max(10000).optional(),
|
|
2683
|
-
num_checkpoints: z.number().int().min(1).max(100).optional(),
|
|
2684
2564
|
integration_id: z.string().optional(),
|
|
2685
|
-
network_volume_id: z.string().optional(),
|
|
2686
2565
|
dataset_mixing: z.enum(["shuffle", "sequential", "interleave"]).optional(),
|
|
2687
2566
|
mixing: datasetMixingSchema.optional().describe("Composition plan. Dataset IDs define source order and weight positions; default seed is 42. The source pins, plan and checksum are retained for metadata-only resume."),
|
|
2688
2567
|
start_from: startFromSchema.optional(),
|
|
@@ -2766,7 +2645,7 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2766
2645
|
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2767
2646
|
/* TOOL: get_training_logs */
|
|
2768
2647
|
/* ══════════════════════════════════════════════════════════════════════════ */
|
|
2769
|
-
server.tool("get_training_logs", "Get structured recent training log entries (level, message, timestamp). Results are newest first and bounded by tail.", {
|
|
2648
|
+
server.tool("get_training_logs", "Get structured recent training log entries (level, message, timestamp). Results are newest first and bounded by tail. Not available when managed_training is enabled (the platform runs the machines): use get_training_status for progress and get_training_metrics for loss, eval loss, gradient norm and accuracy.", {
|
|
2770
2649
|
job_id: z.string().describe("Training job ID"),
|
|
2771
2650
|
tail: z
|
|
2772
2651
|
.number()
|
|
@@ -2777,9 +2656,21 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
2777
2656
|
.describe("Number of most recent log entries to return (default: 200, max: 1000)"),
|
|
2778
2657
|
}, async ({ job_id, tail }) => {
|
|
2779
2658
|
validateId(job_id, "job_id");
|
|
2780
|
-
|
|
2781
|
-
|
|
2782
|
-
|
|
2659
|
+
let data;
|
|
2660
|
+
try {
|
|
2661
|
+
data = await client.api(`/api/training/jobs/${job_id}/logs`, {
|
|
2662
|
+
params: { tail: tail?.toString() },
|
|
2663
|
+
});
|
|
2664
|
+
}
|
|
2665
|
+
catch (error) {
|
|
2666
|
+
if (/\bLOGS_NOT_AVAILABLE\b/.test(error instanceof Error ? error.message : String(error))) {
|
|
2667
|
+
return json({
|
|
2668
|
+
logs: [],
|
|
2669
|
+
note: "Training logs are not available under managed training. Use get_training_status for progress and get_training_metrics for training metrics.",
|
|
2670
|
+
});
|
|
2671
|
+
}
|
|
2672
|
+
throw error;
|
|
2673
|
+
}
|
|
2783
2674
|
// Log lines are written by the training engine, so a boot/pull line can
|
|
2784
2675
|
// quote the image reference it started from. Redact by content: the
|
|
2785
2676
|
// customer keeps every line, minus any build identity inside it.
|
|
@@ -3428,6 +3319,11 @@ create (1-5 choices), and the queue itself stays opt-in.`,
|
|
|
3428
3319
|
method: "POST",
|
|
3429
3320
|
body: {},
|
|
3430
3321
|
})));
|
|
3322
|
+
server.tool("loop_list_live_deployments", "For a pipeline whose live version's deployment was DELETED (live_deployment.status deleted on loop_get_pipeline): the deployments of this workspace that serve exactly the live version's weights, with their status, and the live version itself (live.training_job_id and live.checkpoint_id, what the user deploys again from Deployments). Nothing has to be trained again: the weights are kept. 409 NO_LIVE_VERSION when the pipeline has none. Read-only.", { pipeline_id: z.string() }, async ({ pipeline_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/live/deployments`)));
|
|
3323
|
+
server.tool("loop_use_live_deployment", "Answer the pipeline's model name from this deployment of the live version's weights (one loop_list_live_deployments named), after the pipeline's own deployment was deleted. Nothing is trained again and the version number does not change; the deployment bills the user by the hour, like any deployment, so say so. Refused while the pipeline's own deployment still exists (409 LIVE_DEPLOYMENT_PRESENT: the user starts it again in Deployments instead), for other weights (422 DEPLOYMENT_WEIGHTS_DIFFER) and for a deployment that is not running (409 DEPLOYMENT_NOT_RUNNING). A later rollback stops it.", { pipeline_id: z.string(), deployment_id: z.string() }, async ({ pipeline_id, deployment_id }) => json(await client.api(`/api/loop/pipelines/${encodeURIComponent(pipeline_id)}/live/deployment`, {
|
|
3324
|
+
method: "POST",
|
|
3325
|
+
body: { deployment_id },
|
|
3326
|
+
})));
|
|
3431
3327
|
server.tool("loop_import_into_pipeline", "loop_import_rows with every row given this pipeline's ID (own_tag), so the file becomes this pipeline's samples. source defaults to the pipeline ID. default_feedback 'good' records every answered row that brought no feedback of its own as a good example to learn from — use it only when the user says the file's answers are good; without it such rows wait for review. The result, the retry token (import_id) and the 5000-row limit are loop_import_rows' own.", {
|
|
3432
3328
|
pipeline_id: z.string(),
|
|
3433
3329
|
rows: z.array(z.record(z.string(), z.any())).describe("The rows, already parsed, at most 5000, each in the shape it already has"),
|