tribunal-kit 9.2.3 → 9.2.5

package/.agent/skills/huggingface-llm-trainer/references/training_patterns.md

@@ -0,0 +1,203 @@
+# Common Training Patterns
+
+This guide provides common training patterns and use cases for TRL on Hugging Face Jobs.
+
+## Multi-GPU Training
+
+Automatic distributed training across multiple GPUs. TRL/Accelerate handles distribution automatically:
+
+```python
+hf_jobs("uv", {
+    "script": """
+# Your training script here (same as single GPU)
+# No changes needed - Accelerate detects multiple GPUs
+""",
+    "flavor": "a10g-largex2",  # 2x A10G GPUs
+    "timeout": "4h",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
+})
+```
+
+**Tips for multi-GPU:**
+- No code changes needed
+- Use `per_device_train_batch_size` (per GPU, not total)
+- Effective batch size = `per_device_train_batch_size` × `num_gpus` × `gradient_accumulation_steps`
+- Monitor GPU utilization to ensure both GPUs are being used
+
+## DPO Training (Preference Learning)
+
+Train with preference data for alignment:
+
+```python
+hf_jobs("uv", {
+    "script": """
+# /// script
+# dependencies = ["trl>=0.12.0", "trackio"]
+# ///
+
+from datasets import load_dataset
+from trl import DPOTrainer, DPOConfig
+import trackio
+
+dataset = load_dataset("trl-lib/ultrafeedback_binarized", split="train")
+
+# Create train/eval split
+dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
+
+config = DPOConfig(
+    output_dir="dpo-model",
+    push_to_hub=True,
+    hub_model_id="username/dpo-model",
+    num_train_epochs=1,
+    beta=0.1,  # KL penalty coefficient
+    eval_strategy="steps",
+    eval_steps=50,
+    report_to="trackio",
+    run_name="baseline_run", # use a meaningful run name
+    # max_length=1024,  # Default - only set if you need different sequence length
+)
+
+trainer = DPOTrainer(
+    model="Qwen/Qwen2.5-0.5B-Instruct",  # Use instruct model as base
+    train_dataset=dataset_split["train"],
+    eval_dataset=dataset_split["test"],  # IMPORTANT: Provide eval_dataset when eval_strategy is enabled
+    args=config,
+)
+
+trainer.train()
+trainer.push_to_hub()
+trackio.finish()
+""",
+    "flavor": "a10g-large",
+    "timeout": "3h",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
+})
+```
+
+**For DPO documentation:** Use `hf_doc_fetch("https://huggingface.co/docs/trl/dpo_trainer")`
+
+## GRPO Training (Online RL)
+
+Group Relative Policy Optimization for online reinforcement learning:
+
+```python
+hf_jobs("uv", {
+    "script": "https://raw.githubusercontent.com/huggingface/trl/main/examples/scripts/grpo.py",
+    "script_args": [
+        "--model_name_or_path", "Qwen/Qwen2.5-0.5B-Instruct",
+        "--dataset_name", "trl-lib/math_shepherd",
+        "--output_dir", "grpo-model",
+        "--push_to_hub",
+        "--hub_model_id", "username/grpo-model"
+    ],
+    "flavor": "a10g-large",
+    "timeout": "4h",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
+})
+```
+
+**For GRPO documentation:** Use `hf_doc_fetch("https://huggingface.co/docs/trl/grpo_trainer")`
+
+## Trackio Configuration
+
+**Use sensible defaults for trackio setup.** See `references/trackio_guide.md` for complete documentation including grouping runs for experiments.
+
+### Basic Pattern
+
+```python
+import trackio
+
+trackio.init(
+    project="my-training",
+    run_name="baseline-run",             # Descriptive name user will recognize
+    space_id="username/trackio",     # Default space: {username}/trackio
+    config={
+        # Keep config minimal - hyperparameters and model/dataset info only
+        "model": "Qwen/Qwen2.5-0.5B",
+        "dataset": "trl-lib/Capybara",
+        "learning_rate": 2e-5,
+    }
+)
+
+# Your training code...
+
+trackio.finish()
+```
+
+### Grouping for Experiments (Optional)
+
+When user wants to compare related runs, use the `group` parameter:
+
+```python
+# Hyperparameter sweep
+trackio.init(project="hyperparam-sweep", run_name="lr-0.001", group="lr_0.001")
+trackio.init(project="hyperparam-sweep", run_name="lr-0.01", group="lr_0.01")
+```
+
+## Pattern Selection Guide
+
+| Use Case | Pattern | Hardware | Time |
+|----------|---------|----------|------|
+| SFT training | `scripts/train_sft_example.py` | a10g-large | 2-6 hours |
+| Large dataset (>10K) | Multi-GPU | a10g-largex2 | 4-12 hours |
+| Preference learning | DPO Training | a10g-large | 2-4 hours |
+| Online RL | GRPO Training | a10g-large | 3-6 hours |
+
+## Critical: Evaluation Dataset Requirements
+
+**⚠️ IMPORTANT**: If you set `eval_strategy="steps"` or `eval_strategy="epoch"`, you **MUST** provide an `eval_dataset` to the trainer, or the training will hang.
+
+### ✅ CORRECT - With eval dataset:
+```python
+dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
+
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset_split["train"],
+    eval_dataset=dataset_split["test"],  # ← MUST provide when eval_strategy is enabled
+    args=SFTConfig(eval_strategy="steps", ...),
+)
+```
+
+### ❌ WRONG - Will hang:
+```python
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset,
+    # NO eval_dataset but eval_strategy="steps" ← WILL HANG
+    args=SFTConfig(eval_strategy="steps", ...),
+)
+```
+
+### Option: Disable evaluation if no eval dataset
+```python
+config = SFTConfig(
+    eval_strategy="no",  # ← Explicitly disable evaluation
+    # ... other config
+)
+
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset,
+    # No eval_dataset needed
+    args=config,
+)
+```
+
+## Best Practices
+
+1. **Use train/eval splits** - Create evaluation split for monitoring progress
+2. **Enable Trackio** - Monitor progress in real-time
+3. **Add 20-30% buffer to timeout** - Account for loading/saving overhead
+4. **Test with TRL official scripts first** - Use maintained examples before custom code
+5. **Always provide eval_dataset** - When using eval_strategy, or set to "no"
+6. **Use multi-GPU for large models** - 7B+ models benefit significantly
+
+## See Also
+
+- `scripts/train_sft_example.py` - Complete SFT template with Trackio and eval split
+- `scripts/train_dpo_example.py` - Complete DPO template
+- `scripts/train_grpo_example.py` - Complete GRPO template
+- `references/hardware_guide.md` - Detailed hardware specifications
+- `references/training_methods.md` - Overview of all TRL training methods
+- `references/troubleshooting.md` - Common issues and solutions

package/.agent/skills/huggingface-llm-trainer/references/troubleshooting.md

@@ -0,0 +1,282 @@
+# Troubleshooting TRL Training Jobs
+
+Common issues and solutions when training with TRL on Hugging Face Jobs.
+
+## Training Hangs at "Starting training..." Step
+
+**Problem:** Job starts but hangs at the training step - never progresses, never times out, just sits there.
+
+**Root Cause:** Using `eval_strategy="steps"` or `eval_strategy="epoch"` without providing an `eval_dataset` to the trainer.
+
+**Solution:**
+
+**Option A: Provide eval_dataset (recommended)**
+```python
+# Create train/eval split
+dataset_split = dataset.train_test_split(test_size=0.1, seed=42)
+
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset_split["train"],
+    eval_dataset=dataset_split["test"],  # ← MUST provide when eval_strategy is enabled
+    args=SFTConfig(
+        eval_strategy="steps",
+        eval_steps=50,
+        ...
+    ),
+)
+```
+
+**Option B: Disable evaluation**
+```python
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset,
+    # No eval_dataset
+    args=SFTConfig(
+        eval_strategy="no",  # ← Explicitly disable
+        ...
+    ),
+)
+```
+
+**Prevention:**
+- Always create train/eval split for better monitoring
+- Use `dataset.train_test_split(test_size=0.1, seed=42)`
+- Check example scripts: `scripts/train_sft_example.py` includes proper eval setup
+
+## Job Times Out
+
+**Problem:** Job terminates before training completes, all progress lost.
+
+**Solutions:**
+- Increase timeout parameter (e.g., `"timeout": "4h"`)
+- Reduce `num_train_epochs` or use smaller dataset slice
+- Use smaller model or enable LoRA/PEFT to speed up training
+- Add 20-30% buffer to estimated time for loading/saving overhead
+
+**Prevention:**
+- Always start with a quick demo run to estimate timing
+- Use `scripts/estimate_cost.py` to get time estimates
+- Monitor first runs closely via Trackio or logs
+
+## Model Not Saved to Hub
+
+**Problem:** Training completes but model doesn't appear on Hub - all work lost.
+
+**Check:**
+- [ ] `push_to_hub=True` in training config
+- [ ] `hub_model_id` specified with username (e.g., `"username/model-name"`)
+- [ ] `secrets={"HF_TOKEN": "$HF_TOKEN"}` in job submission
+- [ ] User has write access to target repo
+- [ ] Token has write permissions (check at https://huggingface.co/settings/tokens)
+- [ ] Training script calls `trainer.push_to_hub()` at the end
+
+**See:** `references/hub_saving.md` for detailed Hub authentication troubleshooting
+
+## Out of Memory (OOM)
+
+**Problem:** Job fails with CUDA out of memory error.
+
+**Solutions (in order of preference):**
+1. **Reduce batch size:** Lower `per_device_train_batch_size` (try 4 → 2 → 1)
+2. **Increase gradient accumulation:** Raise `gradient_accumulation_steps` to maintain effective batch size
+3. **Disable evaluation:** Remove `eval_dataset` and `eval_strategy` (saves ~40% memory, good for demos)
+4. **Enable LoRA/PEFT:** Use `peft_config=LoraConfig(r=8, lora_alpha=16)` to train adapters only (smaller rank = less memory)
+5. **Use larger GPU:** Switch from `t4-small` → `l4x1` → `a10g-large` → `a100-large`
+6. **Enable gradient checkpointing:** Set `gradient_checkpointing=True` in config (slower but saves memory)
+7. **Use smaller model:** Try a smaller variant (e.g., 0.5B instead of 3B)
+
+**Memory guidelines:**
+- T4 (16GB): <1B models with LoRA
+- A10G (24GB): 1-3B models with LoRA, <1B full fine-tune
+- A100 (40GB/80GB): 7B+ models with LoRA, 3B full fine-tune
+
+## Parameter Naming Issues
+
+**Problem:** `TypeError: SFTConfig.__init__() got an unexpected keyword argument 'max_seq_length'`
+
+**Cause:** TRL config classes use `max_length`, not `max_seq_length`.
+
+**Solution:**
+```python
+# ✅ CORRECT - TRL uses max_length
+SFTConfig(max_length=512)
+DPOConfig(max_length=512)
+
+# ❌ WRONG - This will fail
+SFTConfig(max_seq_length=512)
+```
+
+**Note:** Most TRL configs don't require explicit max_length - the default (1024) works well. Only set if you need a specific value.
+
+## Dataset Format Error
+
+**Problem:** Training fails with dataset format errors or missing fields.
+
+**Solutions:**
+1. **Check format documentation:**
+   ```python
+   hf_doc_fetch("https://huggingface.co/docs/trl/dataset_formats")
+   ```
+
+2. **Validate dataset before training:**
+   ```bash
+   uv run https://huggingface.co/datasets/mcp-tools/skills/raw/main/dataset_inspector.py \
+     --dataset <dataset-name> --split train
+   ```
+   Or via hf_jobs:
+   ```python
+   hf_jobs("uv", {
+       "script": "https://huggingface.co/datasets/mcp-tools/skills/raw/main/dataset_inspector.py",
+       "script_args": ["--dataset", "dataset-name", "--split", "train"]
+   })
+   ```
+
+3. **Verify field names:**
+   - **SFT:** Needs "messages" field (conversational), OR "text" field, OR "prompt"/"completion"
+   - **DPO:** Needs "chosen" and "rejected" fields
+   - **GRPO:** Needs prompt-only format
+
+4. **Check dataset split:**
+   - Ensure split exists (e.g., `split="train"`)
+   - Preview dataset: `load_dataset("name", split="train[:5]")`
+
+## Import/Module Errors
+
+**Problem:** Job fails with "ModuleNotFoundError" or import errors.
+
+**Solutions:**
+1. **Add PEP 723 header with dependencies:**
+   ```python
+   # /// script
+   # dependencies = [
+   #     "trl>=0.12.0",
+   #     "peft>=0.7.0",
+   #     "transformers>=4.36.0",
+   # ]
+   # ///
+   ```
+
+2. **Verify exact format:**
+   - Must have `# ///` delimiters (with space after `#`)
+   - Dependencies must be valid PyPI package names
+   - Check spelling and version constraints
+
+3. **Test locally first:**
+   ```bash
+   uv run train.py  # Tests if dependencies are correct
+   ```
+
+## Authentication Errors
+
+**Problem:** Job fails with authentication or permission errors when pushing to Hub.
+
+**Solutions:**
+1. **Verify authentication:**
+   ```python
+   mcp__huggingface__hf_whoami()  # Check who's authenticated
+   ```
+
+2. **Check token permissions:**
+   - Go to https://huggingface.co/settings/tokens
+   - Ensure token has "write" permission
+   - Token must not be "read-only"
+
+3. **Verify token in job:**
+   ```python
+   "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # Must be in job config
+   ```
+
+4. **Check repo permissions:**
+   - User must have write access to target repo
+   - If org repo, user must be member with write access
+   - Repo must exist or user must have permission to create
+
+## Job Stuck or Not Starting
+
+**Problem:** Job shows "pending" or "starting" for extended period.
+
+**Solutions:**
+- Check Jobs dashboard for status: https://huggingface.co/jobs
+- Verify hardware availability (some GPU types may have queues)
+- Try different hardware flavor if one is heavily utilized
+- Check for account billing issues (Jobs requires paid plan)
+
+**Typical startup times:**
+- CPU jobs: 10-30 seconds
+- GPU jobs: 30-90 seconds
+- If >3 minutes: likely queued or stuck
+
+## Training Loss Not Decreasing
+
+**Problem:** Training runs but loss stays flat or doesn't improve.
+
+**Solutions:**
+1. **Check learning rate:** May be too low (try 2e-5 to 5e-5) or too high (try 1e-6)
+2. **Verify dataset quality:** Inspect examples to ensure they're reasonable
+3. **Check model size:** Very small models may not have capacity for task
+4. **Increase training steps:** May need more epochs or larger dataset
+5. **Verify dataset format:** Wrong format may cause degraded training
+
+## Logs Not Appearing
+
+**Problem:** Cannot see training logs or progress.
+
+**Solutions:**
+1. **Wait 30-60 seconds:** Initial logs can be delayed
+2. **Check logs via MCP tool:**
+   ```python
+   hf_jobs("logs", {"job_id": "your-job-id"})
+   ```
+3. **Use Trackio for real-time monitoring:** See `references/trackio_guide.md`
+4. **Verify job is actually running:**
+   ```python
+   hf_jobs("inspect", {"job_id": "your-job-id"})
+   ```
+
+## Checkpoint/Resume Issues
+
+**Problem:** Cannot resume from checkpoint or checkpoint not saved.
+
+**Solutions:**
+1. **Enable checkpoint saving:**
+   ```python
+   SFTConfig(
+       save_strategy="steps",
+       save_steps=100,
+       hub_strategy="every_save",  # Push each checkpoint
+   )
+   ```
+
+2. **Verify checkpoints pushed to Hub:** Check model repo for checkpoint folders
+
+3. **Resume from checkpoint:**
+   ```python
+   trainer = SFTTrainer(
+       model="username/model-name",  # Can be checkpoint path
+       resume_from_checkpoint="username/model-name/checkpoint-1000",
+   )
+   ```
+
+## Getting Help
+
+If issues persist:
+
+1. **Check TRL documentation:**
+   ```python
+   hf_doc_search("your issue", product="trl")
+   ```
+
+2. **Check Jobs documentation:**
+   ```python
+   hf_doc_fetch("https://huggingface.co/docs/huggingface_hub/guides/jobs")
+   ```
+
+3. **Review related guides:**
+   - `references/hub_saving.md` - Hub authentication issues
+   - `references/hardware_guide.md` - Hardware selection and specs
+   - `references/training_patterns.md` - Eval dataset requirements
+   - SKILL.md "Working with Scripts" section - Script format and URL issues
+
+4. **Ask in HF forums:** https://discuss.huggingface.co/