tribunal-kit 9.2.3 → 9.2.5

package/.agent/skills/huggingface-llm-trainer/references/hub_saving.md

@@ -0,0 +1,364 @@
+# Saving Training Results to Hugging Face Hub
+
+**⚠️ CRITICAL:** Training environments are ephemeral. ALL results are lost when a job completes unless pushed to the Hub.
+
+## Why Hub Push is Required
+
+When running on Hugging Face Jobs:
+- Environment is temporary
+- All files deleted on job completion
+- No local disk persistence
+- Cannot access results after job ends
+
+**Without Hub push, training is completely wasted.**
+
+## Required Configuration
+
+### 1. Training Configuration
+
+In your SFTConfig or trainer config:
+
+```python
+SFTConfig(
+    push_to_hub=True,                    # Enable Hub push
+    hub_model_id="username/model-name",   # Target repository
+)
+```
+
+### 2. Job Configuration
+
+When submitting the job:
+
+```python
+hf_jobs("uv", {
+    "script": "train.py",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # Provide authentication
+})
+```
+
+**The `$HF_TOKEN` placeholder is automatically replaced with your Hugging Face token.**
+
+## Complete Example
+
+```python
+# train.py
+# /// script
+# dependencies = ["trl"]
+# ///
+
+from trl import SFTTrainer, SFTConfig
+from datasets import load_dataset
+
+dataset = load_dataset("trl-lib/Capybara", split="train")
+
+# Configure with Hub push
+config = SFTConfig(
+    output_dir="my-model",
+    num_train_epochs=3,
+    
+    # ✅ CRITICAL: Hub push configuration
+    push_to_hub=True,
+    hub_model_id="myusername/my-trained-model",
+    
+    # Optional: Push strategy
+    push_to_hub_model_id="myusername/my-trained-model",
+    push_to_hub_organization=None,
+    push_to_hub_token=None,  # Uses environment token
+)
+
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset,
+    args=config,
+)
+
+trainer.train()
+
+# ✅ Push final model
+trainer.push_to_hub()
+
+print("✅ Model saved to: https://huggingface.co/myusername/my-trained-model")
+```
+
+**Submit with authentication:**
+
+```python
+hf_jobs("uv", {
+    "script": "train.py",
+    "flavor": "a10g-large",
+    "timeout": "2h",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # ✅ Required!
+})
+```
+
+## What Gets Saved
+
+When `push_to_hub=True`:
+
+1. **Model weights** - Final trained parameters
+2. **Tokenizer** - Associated tokenizer
+3. **Configuration** - Model config (config.json)
+4. **Training arguments** - Hyperparameters used
+5. **Model card** - Auto-generated documentation
+6. **Checkpoints** - If `save_strategy="steps"` enabled
+
+## Checkpoint Saving
+
+Save intermediate checkpoints during training:
+
+```python
+SFTConfig(
+    output_dir="my-model",
+    push_to_hub=True,
+    hub_model_id="username/my-model",
+    
+    # Checkpoint configuration
+    save_strategy="steps",
+    save_steps=100,              # Save every 100 steps
+    save_total_limit=3,          # Keep only last 3 checkpoints
+)
+```
+
+**Benefits:**
+- Resume training if job fails
+- Compare checkpoint performance
+- Use intermediate models
+
+**Checkpoints are pushed to:** `username/my-model` (same repo)
+
+## Authentication Methods
+
+### Method 1: Automatic Token (Recommended)
+
+```python
+"secrets": {"HF_TOKEN": "$HF_TOKEN"}
+```
+
+Uses your logged-in Hugging Face token automatically.
+
+### Method 2: Explicit Token
+
+```python
+"secrets": {"HF_TOKEN": "hf_abc123..."}
+```
+
+Provide token explicitly (not recommended for security).
+
+### Method 3: Environment Variable
+
+```python
+"env": {"HF_TOKEN": "hf_abc123..."}
+```
+
+Pass as regular environment variable (less secure than secrets).
+
+**Always prefer Method 1** for security and convenience.
+
+## Verification Checklist
+
+Before submitting any training job, verify:
+
+- [ ] `push_to_hub=True` in training config
+- [ ] `hub_model_id` is specified (format: `username/model-name`)
+- [ ] `secrets={"HF_TOKEN": "$HF_TOKEN"}` in job config
+- [ ] Repository name doesn't conflict with existing repos
+- [ ] You have write access to the target namespace
+
+## Repository Setup
+
+### Automatic Creation
+
+If repository doesn't exist, it's created automatically when first pushing.
+
+### Manual Creation
+
+Create repository before training:
+
+```python
+from huggingface_hub import HfApi
+
+api = HfApi()
+api.create_repo(
+    repo_id="username/model-name",
+    repo_type="model",
+    private=False,  # or True for private repo
+)
+```
+
+### Repository Naming
+
+**Valid names:**
+- `username/my-model`
+- `username/model-name`
+- `organization/model-name`
+
+**Invalid names:**
+- `model-name` (missing username)
+- `username/model name` (spaces not allowed)
+- `username/MODEL` (uppercase discouraged)
+
+## Troubleshooting
+
+### Error: 401 Unauthorized
+
+**Cause:** HF_TOKEN not provided or invalid
+
+**Solutions:**
+1. Verify `secrets={"HF_TOKEN": "$HF_TOKEN"}` in job config
+2. Check you're logged in: `hf auth whoami`
+3. Re-login: `hf auth login`
+
+### Error: 403 Forbidden
+
+**Cause:** No write access to repository
+
+**Solutions:**
+1. Check repository namespace matches your username
+2. Verify you're a member of organization (if using org namespace)
+3. Check repository isn't private (if accessing org repo)
+
+### Error: Repository not found
+
+**Cause:** Repository doesn't exist and auto-creation failed
+
+**Solutions:**
+1. Manually create repository first
+2. Check repository name format
+3. Verify namespace exists
+
+### Error: Push failed during training
+
+**Cause:** Network issues or Hub unavailable
+
+**Solutions:**
+1. Training continues but final push fails
+2. Checkpoints may be saved
+3. Re-run push manually after job completes
+
+### Issue: Model saved but not visible
+
+**Possible causes:**
+1. Repository is private—check https://huggingface.co/username
+2. Wrong namespace—verify `hub_model_id` matches login
+3. Push still in progress—wait a few minutes
+
+## Manual Push After Training
+
+If training completes but push fails, push manually:
+
+```python
+from transformers import AutoModel, AutoTokenizer
+
+# Load from local checkpoint
+model = AutoModel.from_pretrained("./output_dir")
+tokenizer = AutoTokenizer.from_pretrained("./output_dir")
+
+# Push to Hub
+model.push_to_hub("username/model-name", token="hf_abc123...")
+tokenizer.push_to_hub("username/model-name", token="hf_abc123...")
+```
+
+**Note:** Only possible if job hasn't completed (files still exist).
+
+## Best Practices
+
+1. **Always enable `push_to_hub=True`**
+2. **Use checkpoint saving** for long training runs
+3. **Verify Hub push** in logs before job completes
+4. **Set appropriate `save_total_limit`** to avoid excessive checkpoints
+5. **Use descriptive repo names** (e.g., `qwen-capybara-sft` not `model1`)
+6. **Add model card** with training details
+7. **Tag models** with relevant tags (e.g., `text-generation`, `fine-tuned`)
+
+## Monitoring Push Progress
+
+Check logs for push progress:
+
+```python
+hf_jobs("logs", {"job_id": "your-job-id"})
+```
+
+**Look for:**
+```
+Pushing model to username/model-name...
+Upload file pytorch_model.bin: 100%
+✅ Model pushed successfully
+```
+
+## Example: Full Production Setup
+
+```python
+# production_train.py
+# /// script
+# dependencies = ["trl>=0.12.0", "peft>=0.7.0"]
+# ///
+
+from datasets import load_dataset
+from peft import LoraConfig
+from trl import SFTTrainer, SFTConfig
+import os
+
+# Verify token is available
+assert "HF_TOKEN" in os.environ, "HF_TOKEN not found in environment!"
+
+# Load dataset
+dataset = load_dataset("trl-lib/Capybara", split="train")
+print(f"✅ Dataset loaded: {len(dataset)} examples")
+
+# Configure with comprehensive Hub settings
+config = SFTConfig(
+    output_dir="qwen-capybara-sft",
+    
+    # Hub configuration
+    push_to_hub=True,
+    hub_model_id="myusername/qwen-capybara-sft",
+    hub_strategy="checkpoint",  # Push checkpoints
+    
+    # Checkpoint configuration
+    save_strategy="steps",
+    save_steps=100,
+    save_total_limit=3,
+    
+    # Training settings
+    num_train_epochs=3,
+    per_device_train_batch_size=4,
+    
+    # Logging
+    logging_steps=10,
+    logging_first_step=True,
+)
+
+# Train with LoRA
+trainer = SFTTrainer(
+    model="Qwen/Qwen2.5-0.5B",
+    train_dataset=dataset,
+    args=config,
+    peft_config=LoraConfig(r=16, lora_alpha=32),
+)
+
+print("🚀 Starting training...")
+trainer.train()
+
+print("💾 Pushing final model to Hub...")
+trainer.push_to_hub()
+
+print("✅ Training complete!")
+print(f"Model available at: https://huggingface.co/myusername/qwen-capybara-sft")
+```
+
+**Submit:**
+
+```python
+hf_jobs("uv", {
+    "script": "production_train.py",
+    "flavor": "a10g-large",
+    "timeout": "6h",
+    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
+})
+```
+
+## Key Takeaway
+
+**Without `push_to_hub=True` and `secrets={"HF_TOKEN": "$HF_TOKEN"}`, all training results are permanently lost.**
+
+Always verify both are configured before submitting any training job.

package/.agent/skills/huggingface-llm-trainer/references/local_training_macos.md

@@ -0,0 +1,231 @@
+# Local Training on macOS (Apple Silicon)
+
+Run small LoRA fine-tuning jobs locally on Mac for smoke tests and quick iteration before submitting to HF Jobs.
+
+## When to Use Local Mac vs HF Jobs
+
+| Local Mac | HF Jobs / Cloud GPU |
+|-----------|-------------------|
+| Model ≤3B, text-only | Model 7B+ |
+| LoRA/PEFT only | QLoRA 4-bit (CUDA/bitsandbytes) |
+| Short context (≤1024) | Long context / full fine-tuning |
+| Smoke tests, dataset validation | Production runs, VLMs |
+
+**Typical workflow:** local smoke test → HF Jobs with same config → export/quantize ([gguf_conversion.md](gguf_conversion.md))
+
+## Recommended Defaults
+
+| Setting | Value | Notes |
+|---------|-------|-------|
+| Model size | 0.5B–1.5B first run | Scale up after verifying |
+| Max seq length | 512–1024 | Lower = less memory |
+| Batch size | 1 | Scale via gradient accumulation |
+| Gradient accumulation | 8–16 | Effective batch = 8–16 |
+| LoRA rank (r) | 8–16 | alpha = 2×r |
+| Dtype | float32 | fp16 causes NaN on MPS; bf16 only on M1 Pro+ and M2/M3/M4 |
+
+### Memory by hardware
+
+| Unified RAM | Max Model Size |
+|-------------|---------------|
+| 16 GB | ~0.5B–1.5B |
+| 32 GB | ~1.5B–3B |
+| 64 GB | ~3B (short context) |
+
+## Setup
+
+```bash
+xcode-select --install
+python3 -m venv .venv && source .venv/bin/activate
+pip install -U "torch>=2.2" "transformers>=4.40" "trl>=0.12" "peft>=0.10" \
+    datasets accelerate safetensors huggingface_hub
+```
+
+Verify MPS:
+```bash
+python -c "import torch; print(torch.__version__, '| MPS:', torch.backends.mps.is_available())"
+```
+
+Optional — configure Accelerate for local Mac (no distributed, no mixed precision, MPS device):
+```bash
+accelerate config
+```
+
+## Training Script
+
+<details>
+<summary><strong>train_lora_sft.py</strong></summary>
+
+```python
+import os
+from dataclasses import dataclass
+from typing import Optional
+import torch
+from datasets import load_dataset
+from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
+from peft import LoraConfig
+from trl import SFTTrainer, SFTConfig
+
+set_seed(42)
+
+@dataclass
+class Cfg:
+    model_id: str = os.environ.get("MODEL_ID", "Qwen/Qwen2.5-0.5B-Instruct")
+    dataset_id: str = os.environ.get("DATASET_ID", "HuggingFaceH4/ultrachat_200k")
+    dataset_split: str = os.environ.get("DATASET_SPLIT", "train_sft[:500]")
+    data_files: Optional[str] = os.environ.get("DATA_FILES", None)
+    text_field: str = os.environ.get("TEXT_FIELD", "")
+    messages_field: str = os.environ.get("MESSAGES_FIELD", "messages")
+    out_dir: str = os.environ.get("OUT_DIR", "outputs/local-lora")
+    max_seq_length: int = int(os.environ.get("MAX_SEQ_LENGTH", "512"))
+    max_steps: int = int(os.environ.get("MAX_STEPS", "-1"))
+
+cfg = Cfg()
+device = "mps" if torch.backends.mps.is_available() else "cpu"
+
+tokenizer = AutoTokenizer.from_pretrained(cfg.model_id, use_fast=True)
+if tokenizer.pad_token is None:
+    tokenizer.pad_token = tokenizer.eos_token
+tokenizer.padding_side = "right"
+
+model = AutoModelForCausalLM.from_pretrained(cfg.model_id, torch_dtype=torch.float32)
+model.to(device)
+model.config.use_cache = False
+
+if cfg.data_files:
+    ds = load_dataset("json", data_files=cfg.data_files, split="train")
+else:
+    ds = load_dataset(cfg.dataset_id, split=cfg.dataset_split)
+
+def format_example(ex):
+    if cfg.text_field and isinstance(ex.get(cfg.text_field), str):
+        ex["text"] = ex[cfg.text_field]
+        return ex
+    msgs = ex.get(cfg.messages_field)
+    if isinstance(msgs, list):
+        if hasattr(tokenizer, "apply_chat_template"):
+            try:
+                ex["text"] = tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)
+                return ex
+            except Exception:
+                pass
+        ex["text"] = "\n".join([str(m) for m in msgs])
+        return ex
+    ex["text"] = str(ex)
+    return ex
+
+ds = ds.map(format_example)
+ds = ds.remove_columns([c for c in ds.column_names if c != "text"])
+
+lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, bias="none",
+                  task_type="CAUSAL_LM", target_modules=["q_proj", "k_proj", "v_proj", "o_proj"])
+
+sft_kwargs = dict(
+    output_dir=cfg.out_dir, per_device_train_batch_size=1, gradient_accumulation_steps=8,
+    learning_rate=2e-4, logging_steps=10, save_steps=200, save_total_limit=2,
+    gradient_checkpointing=True, report_to="none", fp16=False, bf16=False,
+    max_seq_length=cfg.max_seq_length, dataset_text_field="text",
+)
+if cfg.max_steps > 0:
+    sft_kwargs["max_steps"] = cfg.max_steps
+else:
+    sft_kwargs["num_train_epochs"] = 1
+
+trainer = SFTTrainer(model=model, train_dataset=ds, peft_config=lora,
+                     args=SFTConfig(**sft_kwargs), processing_class=tokenizer)
+trainer.train()
+trainer.save_model(cfg.out_dir)
+print(f"✅ Saved to: {cfg.out_dir}")
+```
+
+</details>
+
+### Run
+
+```bash
+python train_lora_sft.py
+```
+
+**Env overrides:**
+
+```bash
+MODEL_ID="Qwen/Qwen2.5-1.5B-Instruct" python train_lora_sft.py   # different model
+MAX_STEPS=50 python train_lora_sft.py                              # quick 50-step test
+DATA_FILES="my_data.jsonl" python train_lora_sft.py                # local JSONL file
+PYTORCH_ENABLE_MPS_FALLBACK=1 python train_lora_sft.py             # MPS op fallback to CPU
+PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 python train_lora_sft.py      # disable MPS memory limit (use with caution)
+```
+
+**Local JSONL format** — chat messages or plain text:
+```jsonl
+{"messages": [{"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi!"}]}
+```
+```jsonl
+{"text": "User: Hello\nAssistant: Hi!"}
+```
+For plain text: `DATA_FILES="file.jsonl" TEXT_FIELD="text" MESSAGES_FIELD="" python train_lora_sft.py`
+
+### Verify Success
+
+- Loss decreases over steps
+- `outputs/local-lora/` contains `adapter_config.json` + `*.safetensors`
+
+## Quick Evaluation
+
+<details>
+<summary><strong>eval_generate.py</strong></summary>
+
+```python
+import os, torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+BASE = os.environ.get("MODEL_ID", "Qwen/Qwen2.5-0.5B-Instruct")
+ADAPTER = os.environ.get("ADAPTER_DIR", "outputs/local-lora")
+device = "mps" if torch.backends.mps.is_available() else "cpu"
+
+tokenizer = AutoTokenizer.from_pretrained(BASE, use_fast=True)
+model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float32)
+model.to(device)
+model = PeftModel.from_pretrained(model, ADAPTER)
+
+prompt = os.environ.get("PROMPT", "Explain gradient accumulation in 3 bullet points.")
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**inputs, max_new_tokens=120, do_sample=True, temperature=0.7, top_p=0.9)
+print(tokenizer.decode(out[0], skip_special_tokens=True))
+```
+
+</details>
+
+## Troubleshooting (macOS-Specific)
+
+For general training issues, see [troubleshooting.md](troubleshooting.md).
+
+| Problem | Fix |
+|---------|-----|
+| MPS unsupported op / crash | `PYTORCH_ENABLE_MPS_FALLBACK=1` |
+| OOM / system instability | Reduce `MAX_SEQ_LENGTH`, use smaller model, set `PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0` (caution) |
+| fp16 NaN / loss explosion | Keep `fp16=False` (default), lower learning rate |
+| LoRA "module not found" | Print `model.named_modules()` to find correct target names |
+| TRL TypeError on args | Check TRL version; script uses `SFTConfig` + `processing_class` (TRL ≥0.12) |
+| Intel Mac | No MPS — use HF Jobs instead |
+
+**Common LoRA target modules by architecture:**
+
+| Architecture | target_modules |
+|-------------|---------------|
+| Llama/Qwen/Mistral | `q_proj`, `k_proj`, `v_proj`, `o_proj` |
+| GPT-2/GPT-J | `c_attn`, `c_proj` |
+| BLOOM | `query_key_value`, `dense` |
+
+## MLX Alternative
+
+[MLX](https://github.com/ml-explore/mlx) offers tighter Apple Silicon integration but has a smaller ecosystem and less mature training APIs. For this skill's workflow (local validation → HF Jobs), PyTorch + MPS is recommended for consistency. See [mlx-lm](https://github.com/ml-explore/mlx-lm) for MLX-based fine-tuning.
+
+## See Also
+
+- [troubleshooting.md](troubleshooting.md) — General TRL troubleshooting
+- [hardware_guide.md](hardware_guide.md) — GPU selection for HF Jobs
+- [gguf_conversion.md](gguf_conversion.md) — Export for on-device inference
+- [training_methods.md](training_methods.md) — SFT, DPO, GRPO overview