aigroup-workflow 2.1.1 → 2.2.0

package/skills/fine-tuning-expert/references/lora-peft.md

@@ -1,347 +0,0 @@
-# LoRA and Parameter-Efficient Fine-Tuning
-
----
-
-## Overview
-
-Parameter-Efficient Fine-Tuning (PEFT) methods train only a small subset of model parameters while keeping the base model frozen. This dramatically reduces memory requirements and enables fine-tuning of large models on consumer hardware.
-
-## When to Use PEFT vs Full Fine-Tuning
-
-| Method | Use When | Avoid When |
-|--------|----------|------------|
-| **LoRA** | 7B+ models, limited VRAM, need multiple task adapters | Very small models (<1B), need maximum quality |
-| **QLoRA** | 13B+ models, single GPU, memory-constrained | High-throughput training, inference speed critical |
-| **Full Fine-Tuning** | Small models, abundant compute, maximum performance needed | Large models, limited resources |
-| **Prefix Tuning** | Generation tasks, need interpretable soft prompts | Complex reasoning tasks |
-| **IA3** | Extreme efficiency needed, inference overhead critical | Tasks needing high adapter capacity |
-
-## LoRA Configuration
-
-```python
-from peft import LoraConfig, get_peft_model, TaskType
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-# Load base model
-model = AutoModelForCausalLM.from_pretrained(
-    "meta-llama/Llama-3.1-8B",
-    torch_dtype=torch.bfloat16,
-    device_map="auto",
-    attn_implementation="flash_attention_2"  # Use Flash Attention if available
-)
-
-# LoRA configuration for instruction tuning
-lora_config = LoraConfig(
-    task_type=TaskType.CAUSAL_LM,
-    r=16,                          # Rank - start with 8-16, increase if underfitting
-    lora_alpha=32,                 # Alpha - typically 2x rank
-    lora_dropout=0.05,             # Dropout for regularization
-    target_modules=[               # Target attention layers
-        "q_proj", "k_proj", "v_proj", "o_proj",  # Attention
-        "gate_proj", "up_proj", "down_proj"       # MLP (optional, increases capacity)
-    ],
-    bias="none",                   # "none", "all", or "lora_only"
-    modules_to_save=None           # Modules to train fully (e.g., embed_tokens for new tokens)
-)
-
-# Create PEFT model
-model = get_peft_model(model, lora_config)
-model.print_trainable_parameters()
-# Output: trainable params: 13,631,488 || all params: 8,043,163,648 || trainable%: 0.1695
-```
-
-### Rank Selection Guide
-
-```python
-def recommend_lora_rank(task_complexity: str, dataset_size: int, model_size_b: float) -> int:
-    """
-    Recommend LoRA rank based on task and resources.
-
-    Args:
-        task_complexity: "simple" (classification), "moderate" (QA), "complex" (creative)
-        dataset_size: Number of training examples
-        model_size_b: Model size in billions of parameters
-    """
-    base_rank = {
-        "simple": 8,
-        "moderate": 16,
-        "complex": 32
-    }[task_complexity]
-
-    # Adjust for dataset size
-    if dataset_size < 1000:
-        rank = max(4, base_rank // 2)  # Reduce rank to prevent overfitting
-    elif dataset_size > 50000:
-        rank = min(64, base_rank * 2)  # Can support higher rank
-    else:
-        rank = base_rank
-
-    # Adjust for model size (larger models may need lower rank)
-    if model_size_b > 30:
-        rank = max(4, rank // 2)
-
-    return rank
-
-# Example usage
-rank = recommend_lora_rank("moderate", dataset_size=10000, model_size_b=8)
-print(f"Recommended rank: {rank}")  # 16
-```
-
-## QLoRA Configuration
-
-QLoRA combines 4-bit quantization with LoRA for extreme memory efficiency.
-
-```python
-from transformers import BitsAndBytesConfig
-import torch
-
-# 4-bit quantization config
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",           # NormalFloat4 for better quality
-    bnb_4bit_compute_dtype=torch.bfloat16,
-    bnb_4bit_use_double_quant=True        # Nested quantization for more savings
-)
-
-# Load quantized model
-model = AutoModelForCausalLM.from_pretrained(
-    "meta-llama/Llama-3.1-70B",
-    quantization_config=bnb_config,
-    device_map="auto",
-    attn_implementation="flash_attention_2"
-)
-
-# Prepare model for kbit training
-from peft import prepare_model_for_kbit_training
-model = prepare_model_for_kbit_training(model, use_gradient_checkpointing=True)
-
-# Apply LoRA
-lora_config = LoraConfig(
-    r=16,
-    lora_alpha=32,
-    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
-    lora_dropout=0.05,
-    bias="none",
-    task_type=TaskType.CAUSAL_LM
-)
-model = get_peft_model(model, lora_config)
-```
-
-### Memory Comparison
-
-| Model | Full FT | LoRA (r=16) | QLoRA (r=16) |
-|-------|---------|-------------|--------------|
-| Llama 3.1 8B | ~64 GB | ~18 GB | ~6 GB |
-| Llama 3.1 70B | ~560 GB | ~160 GB | ~48 GB |
-| Mistral 7B | ~56 GB | ~16 GB | ~5 GB |
-
-## Training with PEFT
-
-```python
-from transformers import TrainingArguments, Trainer
-from trl import SFTTrainer
-
-training_args = TrainingArguments(
-    output_dir="./lora-output",
-    num_train_epochs=3,
-    per_device_train_batch_size=4,
-    gradient_accumulation_steps=4,      # Effective batch size = 16
-    learning_rate=2e-4,                  # Higher LR for LoRA than full FT
-    lr_scheduler_type="cosine",
-    warmup_ratio=0.03,
-    logging_steps=10,
-    save_strategy="steps",
-    save_steps=100,
-    evaluation_strategy="steps",
-    eval_steps=100,
-    bf16=True,
-    gradient_checkpointing=True,
-    gradient_checkpointing_kwargs={"use_reentrant": False},
-    optim="paged_adamw_8bit",            # Memory-efficient optimizer
-    max_grad_norm=0.3,
-    group_by_length=True,                # Group similar length sequences
-    report_to="wandb"
-)
-
-# Using TRL's SFTTrainer for instruction tuning
-trainer = SFTTrainer(
-    model=model,
-    args=training_args,
-    train_dataset=train_dataset,
-    eval_dataset=eval_dataset,
-    tokenizer=tokenizer,
-    max_seq_length=2048,
-    packing=True,                        # Pack short sequences for efficiency
-    dataset_text_field="text"
-)
-
-trainer.train()
-```
-
-## Target Module Selection
-
-Different architectures have different module names:
-
-```python
-# Common target modules by architecture
-TARGET_MODULES = {
-    "llama": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
-    "mistral": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
-    "falcon": ["query_key_value", "dense", "dense_h_to_4h", "dense_4h_to_h"],
-    "gpt2": ["c_attn", "c_proj", "c_fc"],
-    "phi": ["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"],
-    "qwen2": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
-}
-
-def get_target_modules(model_name: str, include_mlp: bool = True) -> list[str]:
-    """Get appropriate target modules for a model architecture."""
-    name_lower = model_name.lower()
-
-    for arch, modules in TARGET_MODULES.items():
-        if arch in name_lower:
-            if include_mlp:
-                return modules
-            # Return only attention modules
-            attention_keywords = ["q_proj", "k_proj", "v_proj", "o_proj", "query", "key", "value", "attn"]
-            return [m for m in modules if any(kw in m.lower() for kw in attention_keywords)]
-
-    # Default for unknown architectures - inspect model
-    raise ValueError(f"Unknown architecture: {model_name}. Inspect model.named_modules() to find target modules.")
-```
-
-## Adapter Merging Strategies
-
-```python
-from peft import PeftModel
-
-# Load base model and adapter
-base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
-model = PeftModel.from_pretrained(base_model, "path/to/lora-adapter")
-
-# Method 1: Merge adapter weights into base model
-merged_model = model.merge_and_unload()
-merged_model.save_pretrained("./merged-model")
-
-# Method 2: Merge multiple adapters (weighted combination)
-from peft import add_weighted_adapter
-
-# Load multiple adapters
-model = PeftModel.from_pretrained(base_model, "adapter1", adapter_name="adapter1")
-model.load_adapter("adapter2", adapter_name="adapter2")
-model.load_adapter("adapter3", adapter_name="adapter3")
-
-# Combine with weights
-model.add_weighted_adapter(
-    adapters=["adapter1", "adapter2", "adapter3"],
-    weights=[0.5, 0.3, 0.2],
-    adapter_name="combined",
-    combination_type="linear"  # or "svd", "cat"
-)
-model.set_adapter("combined")
-```
-
-## DoRA (Weight-Decomposed LoRA)
-
-DoRA improves on LoRA by decomposing weights into magnitude and direction components.
-
-```python
-from peft import LoraConfig
-
-# DoRA configuration
-dora_config = LoraConfig(
-    r=16,
-    lora_alpha=32,
-    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
-    lora_dropout=0.05,
-    use_dora=True,  # Enable DoRA
-    task_type=TaskType.CAUSAL_LM
-)
-
-# Training is identical to LoRA
-model = get_peft_model(model, dora_config)
-```
-
-## rsLoRA (Rank-Stabilized LoRA)
-
-Proper scaling for higher ranks:
-
-```python
-from peft import LoraConfig
-
-# rsLoRA for high-rank training
-rslora_config = LoraConfig(
-    r=64,  # Higher rank
-    lora_alpha=64,
-    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
-    use_rslora=True,  # Rank-stabilized scaling
-    task_type=TaskType.CAUSAL_LM
-)
-```
-
-## Common Issues and Solutions
-
-### Issue: Loss Not Decreasing
-
-```python
-# Check 1: Verify adapter is training
-for name, param in model.named_parameters():
-    if param.requires_grad:
-        print(f"Training: {name}")
-
-# Check 2: Increase rank or alpha
-config = LoraConfig(r=32, lora_alpha=64, ...)
-
-# Check 3: Reduce learning rate
-training_args = TrainingArguments(learning_rate=1e-4, ...)
-```
-
-### Issue: Out of Memory
-
-```python
-# Solution 1: Use QLoRA
-bnb_config = BitsAndBytesConfig(load_in_4bit=True, ...)
-
-# Solution 2: Enable gradient checkpointing
-model.gradient_checkpointing_enable()
-
-# Solution 3: Reduce batch size, increase gradient accumulation
-training_args = TrainingArguments(
-    per_device_train_batch_size=1,
-    gradient_accumulation_steps=16
-)
-
-# Solution 4: Use 8-bit optimizer
-training_args = TrainingArguments(optim="paged_adamw_8bit")
-```
-
-### Issue: Adapter Not Loading
-
-```python
-# Ensure architecture matches
-from peft import PeftModel, PeftConfig
-
-# Check adapter config
-config = PeftConfig.from_pretrained("path/to/adapter")
-print(f"Base model: {config.base_model_name_or_path}")
-print(f"Target modules: {config.target_modules}")
-
-# Load with correct base model
-base_model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path)
-model = PeftModel.from_pretrained(base_model, "path/to/adapter")
-```
-
-## Quick Reference
-
-| Parameter | Typical Range | Effect |
-|-----------|---------------|--------|
-| `r` (rank) | 4-64 | Adapter capacity; higher = more expressive |
-| `lora_alpha` | r to 2*r | Scaling factor; higher = larger updates |
-| `lora_dropout` | 0.0-0.1 | Regularization; increase for small datasets |
-| `learning_rate` | 1e-5 to 3e-4 | LoRA tolerates higher LR than full FT |
-| `target_modules` | attention + MLP | More modules = more capacity + memory |
-
-## Related References
-
-- `hyperparameter-tuning.md` - Learning rate schedules, batch sizes
-- `deployment-optimization.md` - Adapter merging, quantization for inference
-- `dataset-preparation.md` - Training data formatting

package/skills/finishing-a-development-branch/SKILL.md

@@ -1,200 +0,0 @@
----
-name: finishing-a-development-branch
-description: Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup
----
-
-# Finishing a Development Branch
-
-## Overview
-
-Guide completion of development work by presenting clear options and handling chosen workflow.
-
-**Core principle:** Verify tests → Present options → Execute choice → Clean up.
-
-**Announce at start:** "I'm using the finishing-a-development-branch skill to complete this work."
-
-## The Process
-
-### Step 1: Verify Tests
-
-**Before presenting options, verify tests pass:**
-
-```bash
-# Run project's test suite
-npm test / cargo test / pytest / go test ./...
-```
-
-**If tests fail:**
-```
-Tests failing (<N> failures). Must fix before completing:
-
-[Show failures]
-
-Cannot proceed with merge/PR until tests pass.
-```
-
-Stop. Don't proceed to Step 2.
-
-**If tests pass:** Continue to Step 2.
-
-### Step 2: Determine Base Branch
-
-```bash
-# Try common base branches
-git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null
-```
-
-Or ask: "This branch split from main - is that correct?"
-
-### Step 3: Present Options
-
-Present exactly these 4 options:
-
-```
-Implementation complete. What would you like to do?
-
-1. Merge back to <base-branch> locally
-2. Push and create a Pull Request
-3. Keep the branch as-is (I'll handle it later)
-4. Discard this work
-
-Which option?
-```
-
-**Don't add explanation** - keep options concise.
-
-### Step 4: Execute Choice
-
-#### Option 1: Merge Locally
-
-```bash
-# Switch to base branch
-git checkout <base-branch>
-
-# Pull latest
-git pull
-
-# Merge feature branch
-git merge <feature-branch>
-
-# Verify tests on merged result
-<test command>
-
-# If tests pass
-git branch -d <feature-branch>
-```
-
-Then: Cleanup worktree (Step 5)
-
-#### Option 2: Push and Create PR
-
-```bash
-# Push branch
-git push -u origin <feature-branch>
-
-# Create PR
-gh pr create --title "<title>" --body "$(cat <<'EOF'
-## Summary
-<2-3 bullets of what changed>
-
-## Test Plan
-- [ ] <verification steps>
-EOF
-)"
-```
-
-Then: Cleanup worktree (Step 5)
-
-#### Option 3: Keep As-Is
-
-Report: "Keeping branch <name>. Worktree preserved at <path>."
-
-**Don't cleanup worktree.**
-
-#### Option 4: Discard
-
-**Confirm first:**
-```
-This will permanently delete:
-- Branch <name>
-- All commits: <commit-list>
-- Worktree at <path>
-
-Type 'discard' to confirm.
-```
-
-Wait for exact confirmation.
-
-If confirmed:
-```bash
-git checkout <base-branch>
-git branch -D <feature-branch>
-```
-
-Then: Cleanup worktree (Step 5)
-
-### Step 5: Cleanup Worktree
-
-**For Options 1, 2, 4:**
-
-Check if in worktree:
-```bash
-git worktree list | grep $(git branch --show-current)
-```
-
-If yes:
-```bash
-git worktree remove <worktree-path>
-```
-
-**For Option 3:** Keep worktree.
-
-## Quick Reference
-
-| Option | Merge | Push | Keep Worktree | Cleanup Branch |
-|--------|-------|------|---------------|----------------|
-| 1. Merge locally | ✓ | - | - | ✓ |
-| 2. Create PR | - | ✓ | ✓ | - |
-| 3. Keep as-is | - | - | ✓ | - |
-| 4. Discard | - | - | - | ✓ (force) |
-
-## Common Mistakes
-
-**Skipping test verification**
-- **Problem:** Merge broken code, create failing PR
-- **Fix:** Always verify tests before offering options
-
-**Open-ended questions**
-- **Problem:** "What should I do next?" → ambiguous
-- **Fix:** Present exactly 4 structured options
-
-**Automatic worktree cleanup**
-- **Problem:** Remove worktree when might need it (Option 2, 3)
-- **Fix:** Only cleanup for Options 1 and 4
-
-**No confirmation for discard**
-- **Problem:** Accidentally delete work
-- **Fix:** Require typed "discard" confirmation
-
-## Red Flags
-
-**Never:**
-- Proceed with failing tests
-- Merge without verifying tests on result
-- Delete work without confirmation
-- Force-push without explicit request
-
-**Always:**
-- Verify tests before offering options
-- Present exactly 4 options
-- Get typed confirmation for Option 4
-- Clean up worktree for Options 1 & 4 only
-
-## Integration
-
-**Called by:**
-- Subagent dispatch flow per `docs/rules/agents.md` - After all tasks complete
-- **executing-plans** (Step 5) - After all batches complete
-
-**Pairs with:**
-- **using-git-worktrees** - Cleans up worktree created by that skill

package/skills/flutter-expert/SKILL.md

@@ -1,138 +0,0 @@
----
-name: flutter-expert
-description: Use when building cross-platform applications with Flutter 3+ and Dart. Invoke for widget development, Riverpod/Bloc state management, GoRouter navigation, platform-specific implementations, performance optimization.
-license: MIT
-metadata:
-  author: https://github.com/Jeffallan
-  version: "1.1.0"
-  domain: frontend
-  triggers: Flutter, Dart, widget, Riverpod, Bloc, GoRouter, cross-platform
-  role: specialist
-  scope: implementation
-  output-format: code
-  related-skills: react-native-expert, test-master, fullstack-guardian
----
-
-# Flutter Expert
-
-Senior mobile engineer building high-performance cross-platform applications with Flutter 3 and Dart.
-
-## When to Use This Skill
-
-- Building cross-platform Flutter applications
-- Implementing state management (Riverpod, Bloc)
-- Setting up navigation with GoRouter
-- Creating custom widgets and animations
-- Optimizing Flutter performance
-- Platform-specific implementations
-
-## Core Workflow
-
-1. **Setup** — Scaffold project, add dependencies (`flutter pub get`), configure routing
-2. **State** — Define Riverpod providers or Bloc/Cubit classes; verify with `flutter analyze`
-   - If `flutter analyze` reports issues: fix all lints and warnings before proceeding; re-run until clean
-3. **Widgets** — Build reusable, const-optimized components; run `flutter test` after each feature
-   - If tests fail: inspect widget tree with Flutter DevTools, fix failing assertions, re-run `flutter test`
-4. **Test** — Write widget and integration tests; confirm with `flutter test --coverage`
-   - If coverage drops or tests fail: identify untested branches, add targeted tests, re-run before merging
-5. **Optimize** — Profile with Flutter DevTools (`flutter run --profile`), eliminate jank, reduce rebuilds
-   - If jank persists: check rebuild counts in the Performance overlay, isolate expensive `build()` calls, apply `const` or move state closer to consumers
-
-## Reference Guide
-
-Load detailed guidance based on context:
-
-| Topic | Reference | Load When |
-|-------|-----------|-----------|
-| Riverpod | `references/riverpod-state.md` | State management, providers, notifiers |
-| Bloc | `references/bloc-state.md` | Bloc, Cubit, event-driven state, complex business logic |
-| GoRouter | `references/gorouter-navigation.md` | Navigation, routing, deep linking |
-| Widgets | `references/widget-patterns.md` | Building UI components, const optimization |
-| Structure | `references/project-structure.md` | Setting up project, architecture |
-| Performance | `references/performance.md` | Optimization, profiling, jank fixes |
-
-## Code Examples
-
-### Riverpod Provider + ConsumerWidget (correct pattern)
-
-```dart
-// provider definition
-final counterProvider = StateNotifierProvider<CounterNotifier, int>(
-  (ref) => CounterNotifier(),
-);
-
-class CounterNotifier extends StateNotifier<int> {
-  CounterNotifier() : super(0);
-  void increment() => state = state + 1; // new instance, never mutate
-}
-
-// consuming widget — use ConsumerWidget, not StatefulWidget
-class CounterView extends ConsumerWidget {
-  const CounterView({super.key});
-
-  @override
-  Widget build(BuildContext context, WidgetRef ref) {
-    final count = ref.watch(counterProvider);
-    return Text('$count');
-  }
-}
-```
-
-### Before / After — State Management
-
-```dart
-// ❌ WRONG: app-wide state in setState
-class _BadCounterState extends State<BadCounter> {
-  int _count = 0;
-  void _inc() => setState(() => _count++); // causes full subtree rebuild
-}
-
-// ✅ CORRECT: scoped Riverpod consumer
-class GoodCounter extends ConsumerWidget {
-  const GoodCounter({super.key});
-  @override
-  Widget build(BuildContext context, WidgetRef ref) {
-    final count = ref.watch(counterProvider);
-    return IconButton(
-      onPressed: () => ref.read(counterProvider.notifier).increment(),
-      icon: const Icon(Icons.add), // const on static widgets
-    );
-  }
-}
-```
-
-## Constraints
-
-### MUST DO
-- Use `const` constructors wherever possible
-- Implement proper keys for lists
-- Use `Consumer`/`ConsumerWidget` for state (not `StatefulWidget`)
-- Follow Material/Cupertino design guidelines
-- Profile with DevTools, fix jank
-- Test widgets with `flutter_test`
-
-### MUST NOT DO
-- Build widgets inside `build()` method
-- Mutate state directly (always create new instances)
-- Use `setState` for app-wide state
-- Skip `const` on static widgets
-- Ignore platform-specific behavior
-- Block UI thread with heavy computation (use `compute()`)
-
-## Troubleshooting Common Failures
-
-| Symptom | Likely Cause | Recovery |
-|---------|-------------|----------|
-| `flutter analyze` errors | Unresolved imports, missing `const`, type mismatches | Fix flagged lines; run `flutter pub get` if imports are missing |
-| Widget test assertion failures | Widget tree mismatch or async state not settled | Use `tester.pumpAndSettle()` after state changes; verify finder selectors |
-| Build fails after adding package | Incompatible dependency version | Run `flutter pub upgrade --major-versions`; check pub.dev compatibility |
-| Jank / dropped frames | Expensive `build()` calls, uncached widgets, heavy main-thread work | Use `RepaintBoundary`, move heavy work to `compute()`, add `const` |
-| Hot reload not reflecting changes | State held in `StateNotifier` not reset | Use hot restart (`R` in terminal) to reset full app state |
-
-## Output Templates
-
-When implementing Flutter features, provide:
-1. Widget code with proper `const` usage
-2. Provider/Bloc definitions
-3. Route configuration if needed
-4. Test file structure