cli-jaw 0.1.6 → 0.1.8

package/skills_ref/hugging-face-model-trainer/references/training_patterns.md

@@ -1,203 +0,0 @@
-# Common Training Patterns
-
-This guide provides common training patterns and use cases for TRL on Hugging Face Jobs.
-
-## Multi-GPU Training
-
-Automatic distributed training across multiple GPUs. TRL/Accelerate handles distribution automatically:
-
-```python
-hf_jobs("uv", {
-    "script": """
-# Your training script here (same as single GPU)
-# No changes needed - Accelerate detects multiple GPUs
-""",
-    "flavor": "a10g-largex2",  # 2x A10G GPUs
-    "timeout": "4h",
-    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
-})
-```
-
-**Tips for multi-GPU:**
-- No code changes needed
-- Use `per_device_train_batch_size` (per GPU, not total)
-- Effective batch size = `per_device_train_batch_size` × `num_gpus` × `gradient_accumulation_steps`
-- Monitor GPU utilization to ensure both GPUs are being used
-
-## DPO Training (Preference Learning)
-
-Train with preference data for alignment:
-
-```python
-hf_jobs("uv", {
-    "script": """
-# /// script
-# dependencies = ["trl>=0.12.0", "trackio"]
-# ///
-
-from datasets import load_dataset
-from trl import DPOTrainer, DPOConfig
-import trackio
-
-dataset = load_dataset("trl-lib/ultrafeedback_binarized", split="train")
-
-# Create train/eval split
-dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
-
-config = DPOConfig(
-    output_dir="dpo-model",
-    push_to_hub=True,
-    hub_model_id="username/dpo-model",
-    num_train_epochs=1,
-    beta=0.1,  # KL penalty coefficient
-    eval_strategy="steps",
-    eval_steps=50,
-    report_to="trackio",
-    run_name="baseline_run", # use a meaningful run name
-    # max_length=1024,  # Default - only set if you need different sequence length
-)
-
-trainer = DPOTrainer(
-    model="Qwen/Qwen2.5-0.5B-Instruct",  # Use instruct model as base
-    train_dataset=dataset_split["train"],
-    eval_dataset=dataset_split["test"],  # IMPORTANT: Provide eval_dataset when eval_strategy is enabled
-    args=config,
-)
-
-trainer.train()
-trainer.push_to_hub()
-trackio.finish()
-""",
-    "flavor": "a10g-large",
-    "timeout": "3h",
-    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
-})
-```
-
-**For DPO documentation:** Use `hf_doc_fetch("https://huggingface.co/docs/trl/dpo_trainer")`
-
-## GRPO Training (Online RL)
-
-Group Relative Policy Optimization for online reinforcement learning:
-
-```python
-hf_jobs("uv", {
-    "script": "https://raw.githubusercontent.com/huggingface/trl/main/examples/scripts/grpo.py",
-    "script_args": [
-        "--model_name_or_path", "Qwen/Qwen2.5-0.5B-Instruct",
-        "--dataset_name", "trl-lib/math_shepherd",
-        "--output_dir", "grpo-model",
-        "--push_to_hub",
-        "--hub_model_id", "username/grpo-model"
-    ],
-    "flavor": "a10g-large",
-    "timeout": "4h",
-    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
-})
-```
-
-**For GRPO documentation:** Use `hf_doc_fetch("https://huggingface.co/docs/trl/grpo_trainer")`
-
-## Trackio Configuration
-
-**Use sensible defaults for trackio setup.** See `references/trackio_guide.md` for complete documentation including grouping runs for experiments.
-
-### Basic Pattern
-
-```python
-import trackio
-
-trackio.init(
-    project="my-training",
-    run_name="baseline-run",             # Descriptive name user will recognize
-    space_id="username/trackio",     # Default space: {username}/trackio
-    config={
-        # Keep config minimal - hyperparameters and model/dataset info only
-        "model": "Qwen/Qwen2.5-0.5B",
-        "dataset": "trl-lib/Capybara",
-        "learning_rate": 2e-5,
-    }
-)
-
-# Your training code...
-
-trackio.finish()
-```
-
-### Grouping for Experiments (Optional)
-
-When user wants to compare related runs, use the `group` parameter:
-
-```python
-# Hyperparameter sweep
-trackio.init(project="hyperparam-sweep", run_name="lr-0.001", group="lr_0.001")
-trackio.init(project="hyperparam-sweep", run_name="lr-0.01", group="lr_0.01")
-```
-
-## Pattern Selection Guide
-
-| Use Case | Pattern | Hardware | Time |
-|----------|---------|----------|------|
-| SFT training | `scripts/train_sft_example.py` | a10g-large | 2-6 hours |
-| Large dataset (>10K) | Multi-GPU | a10g-largex2 | 4-12 hours |
-| Preference learning | DPO Training | a10g-large | 2-4 hours |
-| Online RL | GRPO Training | a10g-large | 3-6 hours |
-
-## Critical: Evaluation Dataset Requirements
-
-**⚠️ IMPORTANT**: If you set `eval_strategy="steps"` or `eval_strategy="epoch"`, you **MUST** provide an `eval_dataset` to the trainer, or the training will hang.
-
-### ✅ CORRECT - With eval dataset:
-```python
-dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
-
-trainer = SFTTrainer(
-    model="Qwen/Qwen2.5-0.5B",
-    train_dataset=dataset_split["train"],
-    eval_dataset=dataset_split["test"],  # ← MUST provide when eval_strategy is enabled
-    args=SFTConfig(eval_strategy="steps", ...),
-)
-```
-
-### ❌ WRONG - Will hang:
-```python
-trainer = SFTTrainer(
-    model="Qwen/Qwen2.5-0.5B",
-    train_dataset=dataset,
-    # NO eval_dataset but eval_strategy="steps" ← WILL HANG
-    args=SFTConfig(eval_strategy="steps", ...),
-)
-```
-
-### Option: Disable evaluation if no eval dataset
-```python
-config = SFTConfig(
-    eval_strategy="no",  # ← Explicitly disable evaluation
-    # ... other config
-)
-
-trainer = SFTTrainer(
-    model="Qwen/Qwen2.5-0.5B",
-    train_dataset=dataset,
-    # No eval_dataset needed
-    args=config,
-)
-```
-
-## Best Practices
-
-1. **Use train/eval splits** - Create evaluation split for monitoring progress
-2. **Enable Trackio** - Monitor progress in real-time
-3. **Add 20-30% buffer to timeout** - Account for loading/saving overhead
-4. **Test with TRL official scripts first** - Use maintained examples before custom code
-5. **Always provide eval_dataset** - When using eval_strategy, or set to "no"
-6. **Use multi-GPU for large models** - 7B+ models benefit significantly
-
-## See Also
-
-- `scripts/train_sft_example.py` - Complete SFT template with Trackio and eval split
-- `scripts/train_dpo_example.py` - Complete DPO template
-- `scripts/train_grpo_example.py` - Complete GRPO template
-- `references/hardware_guide.md` - Detailed hardware specifications
-- `references/training_methods.md` - Overview of all TRL training methods
-- `references/troubleshooting.md` - Common issues and solutions

package/skills_ref/hugging-face-model-trainer/references/troubleshooting.md

@@ -1,282 +0,0 @@
-# Troubleshooting TRL Training Jobs
-
-Common issues and solutions when training with TRL on Hugging Face Jobs.
-
-## Training Hangs at "Starting training..." Step
-
-**Problem:** Job starts but hangs at the training step - never progresses, never times out, just sits there.
-
-**Root Cause:** Using `eval_strategy="steps"` or `eval_strategy="epoch"` without providing an `eval_dataset` to the trainer.
-
-**Solution:**
-
-**Option A: Provide eval_dataset (recommended)**
-```python
-# Create train/eval split
-dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
-
-trainer = SFTTrainer(
-    model="Qwen/Qwen2.5-0.5B",
-    train_dataset=dataset_split["train"],
-    eval_dataset=dataset_split["test"],  # ← MUST provide when eval_strategy is enabled
-    args=SFTConfig(
-        eval_strategy="steps",
-        eval_steps=50,
-        ...
-    ),
-)
-```
-
-**Option B: Disable evaluation**
-```python
-trainer = SFTTrainer(
-    model="Qwen/Qwen2.5-0.5B",
-    train_dataset=dataset,
-    # No eval_dataset
-    args=SFTConfig(
-        eval_strategy="no",  # ← Explicitly disable
-        ...
-    ),
-)
-```
-
-**Prevention:**
-- Always create train/eval split for better monitoring
-- Use `dataset.train_test_split(test_size=0.1, seed=42)`
-- Check example scripts: `scripts/train_sft_example.py` includes proper eval setup
-
-## Job Times Out
-
-**Problem:** Job terminates before training completes, all progress lost.
-
-**Solutions:**
-- Increase timeout parameter (e.g., `"timeout": "4h"`)
-- Reduce `num_train_epochs` or use smaller dataset slice
-- Use smaller model or enable LoRA/PEFT to speed up training
-- Add 20-30% buffer to estimated time for loading/saving overhead
-
-**Prevention:**
-- Always start with a quick demo run to estimate timing
-- Use `scripts/estimate_cost.py` to get time estimates
-- Monitor first runs closely via Trackio or logs
-
-## Model Not Saved to Hub
-
-**Problem:** Training completes but model doesn't appear on Hub - all work lost.
-
-**Check:**
-- [ ] `push_to_hub=True` in training config
-- [ ] `hub_model_id` specified with username (e.g., `"username/model-name"`)
-- [ ] `secrets={"HF_TOKEN": "$HF_TOKEN"}` in job submission
-- [ ] User has write access to target repo
-- [ ] Token has write permissions (check at https://huggingface.co/settings/tokens)
-- [ ] Training script calls `trainer.push_to_hub()` at the end
-
-**See:** `references/hub_saving.md` for detailed Hub authentication troubleshooting
-
-## Out of Memory (OOM)
-
-**Problem:** Job fails with CUDA out of memory error.
-
-**Solutions (in order of preference):**
-1. **Reduce batch size:** Lower `per_device_train_batch_size` (try 4 → 2 → 1)
-2. **Increase gradient accumulation:** Raise `gradient_accumulation_steps` to maintain effective batch size
-3. **Disable evaluation:** Remove `eval_dataset` and `eval_strategy` (saves ~40% memory, good for demos)
-4. **Enable LoRA/PEFT:** Use `peft_config=LoraConfig(r=8, lora_alpha=16)` to train adapters only (smaller rank = less memory)
-5. **Use larger GPU:** Switch from `t4-small` → `l4x1` → `a10g-large` → `a100-large`
-6. **Enable gradient checkpointing:** Set `gradient_checkpointing=True` in config (slower but saves memory)
-7. **Use smaller model:** Try a smaller variant (e.g., 0.5B instead of 3B)
-
-**Memory guidelines:**
-- T4 (16GB): <1B models with LoRA
-- A10G (24GB): 1-3B models with LoRA, <1B full fine-tune
-- A100 (40GB/80GB): 7B+ models with LoRA, 3B full fine-tune
-
-## Parameter Naming Issues
-
-**Problem:** `TypeError: SFTConfig.__init__() got an unexpected keyword argument 'max_seq_length'`
-
-**Cause:** TRL config classes use `max_length`, not `max_seq_length`.
-
-**Solution:**
-```python
-# ✅ CORRECT - TRL uses max_length
-SFTConfig(max_length=512)
-DPOConfig(max_length=512)
-
-# ❌ WRONG - This will fail
-SFTConfig(max_seq_length=512)
-```
-
-**Note:** Most TRL configs don't require explicit max_length - the default (1024) works well. Only set if you need a specific value.
-
-## Dataset Format Error
-
-**Problem:** Training fails with dataset format errors or missing fields.
-
-**Solutions:**
-1. **Check format documentation:**
-   ```python
-   hf_doc_fetch("https://huggingface.co/docs/trl/dataset_formats")
-   ```
-
-2. **Validate dataset before training:**
-   ```bash
-   uv run https://huggingface.co/datasets/mcp-tools/skills/raw/main/dataset_inspector.py \
-     --dataset <dataset-name> --split train
-   ```
-   Or via hf_jobs:
-   ```python
-   hf_jobs("uv", {
-       "script": "https://huggingface.co/datasets/mcp-tools/skills/raw/main/dataset_inspector.py",
-       "script_args": ["--dataset", "dataset-name", "--split", "train"]
-   })
-   ```
-
-3. **Verify field names:**
-   - **SFT:** Needs "messages" field (conversational), OR "text" field, OR "prompt"/"completion"
-   - **DPO:** Needs "chosen" and "rejected" fields
-   - **GRPO:** Needs prompt-only format
-
-4. **Check dataset split:**
-   - Ensure split exists (e.g., `split="train"`)
-   - Preview dataset: `load_dataset("name", split="train[:5]")`
-
-## Import/Module Errors
-
-**Problem:** Job fails with "ModuleNotFoundError" or import errors.
-
-**Solutions:**
-1. **Add PEP 723 header with dependencies:**
-   ```python
-   # /// script
-   # dependencies = [
-   #     "trl>=0.12.0",
-   #     "peft>=0.7.0",
-   #     "transformers>=4.36.0",
-   # ]
-   # ///
-   ```
-
-2. **Verify exact format:**
-   - Must have `# ///` delimiters (with space after `#`)
-   - Dependencies must be valid PyPI package names
-   - Check spelling and version constraints
-
-3. **Test locally first:**
-   ```bash
-   uv run train.py  # Tests if dependencies are correct
-   ```
-
-## Authentication Errors
-
-**Problem:** Job fails with authentication or permission errors when pushing to Hub.
-
-**Solutions:**
-1. **Verify authentication:**
-   ```python
-   mcp__huggingface__hf_whoami()  # Check who's authenticated
-   ```
-
-2. **Check token permissions:**
-   - Go to https://huggingface.co/settings/tokens
-   - Ensure token has "write" permission
-   - Token must not be "read-only"
-
-3. **Verify token in job:**
-   ```python
-   "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # Must be in job config
-   ```
-
-4. **Check repo permissions:**
-   - User must have write access to target repo
-   - If org repo, user must be member with write access
-   - Repo must exist or user must have permission to create
-
-## Job Stuck or Not Starting
-
-**Problem:** Job shows "pending" or "starting" for extended period.
-
-**Solutions:**
-- Check Jobs dashboard for status: https://huggingface.co/jobs
-- Verify hardware availability (some GPU types may have queues)
-- Try different hardware flavor if one is heavily utilized
-- Check for account billing issues (Jobs requires paid plan)
-
-**Typical startup times:**
-- CPU jobs: 10-30 seconds
-- GPU jobs: 30-90 seconds
-- If >3 minutes: likely queued or stuck
-
-## Training Loss Not Decreasing
-
-**Problem:** Training runs but loss stays flat or doesn't improve.
-
-**Solutions:**
-1. **Check learning rate:** May be too low (try 2e-5 to 5e-5) or too high (try 1e-6)
-2. **Verify dataset quality:** Inspect examples to ensure they're reasonable
-3. **Check model size:** Very small models may not have capacity for task
-4. **Increase training steps:** May need more epochs or larger dataset
-5. **Verify dataset format:** Wrong format may cause degraded training
-
-## Logs Not Appearing
-
-**Problem:** Cannot see training logs or progress.
-
-**Solutions:**
-1. **Wait 30-60 seconds:** Initial logs can be delayed
-2. **Check logs via MCP tool:**
-   ```python
-   hf_jobs("logs", {"job_id": "your-job-id"})
-   ```
-3. **Use Trackio for real-time monitoring:** See `references/trackio_guide.md`
-4. **Verify job is actually running:**
-   ```python
-   hf_jobs("inspect", {"job_id": "your-job-id"})
-   ```
-
-## Checkpoint/Resume Issues
-
-**Problem:** Cannot resume from checkpoint or checkpoint not saved.
-
-**Solutions:**
-1. **Enable checkpoint saving:**
-   ```python
-   SFTConfig(
-       save_strategy="steps",
-       save_steps=100,
-       hub_strategy="every_save",  # Push each checkpoint
-   )
-   ```
-
-2. **Verify checkpoints pushed to Hub:** Check model repo for checkpoint folders
-
-3. **Resume from checkpoint:**
-   ```python
-   trainer = SFTTrainer(
-       model="username/model-name",  # Can be checkpoint path
-       resume_from_checkpoint="username/model-name/checkpoint-1000",
-   )
-   ```
-
-## Getting Help
-
-If issues persist:
-
-1. **Check TRL documentation:**
-   ```python
-   hf_doc_search("your issue", product="trl")
-   ```
-
-2. **Check Jobs documentation:**
-   ```python
-   hf_doc_fetch("https://huggingface.co/docs/huggingface_hub/guides/jobs")
-   ```
-
-3. **Review related guides:**
-   - `references/hub_saving.md` - Hub authentication issues
-   - `references/hardware_guide.md` - Hardware selection and specs
-   - `references/training_patterns.md` - Eval dataset requirements
-   - SKILL.md "Working with Scripts" section - Script format and URL issues
-
-4. **Ask in HF forums:** https://discuss.huggingface.co/