@zigrivers/scaffold 3.7.0 → 3.9.0

package/content/knowledge/ml/ml-training-patterns.md

@@ -0,0 +1,269 @@
+---
+name: ml-training-patterns
+description: Data loaders, training loops, distributed training with DDP and FSDP, checkpointing strategies, and hyperparameter tuning patterns
+topics: [ml, training, data-loaders, distributed-training, ddp, fsdp, checkpointing, hyperparameter-tuning]
+---
+
+The training loop is the heart of every ML project, but it is also where most bugs hide: data leaking between splits, gradients not zeroed, mixed precision overflows, checkpoints saved incorrectly, and distributed training hanging on a single slow worker. These are not exotic edge cases — they are the standard bugs that every ML engineer encounters. A well-structured training pipeline prevents them through clear separation of concerns and defensive coding.
+
+## Summary
+
+Build training pipelines with properly configured data loaders (worker count, pinned memory, prefetch), clean training loops with explicit gradient management, mixed precision for efficiency, and robust checkpointing. For large models or large datasets, use PyTorch DDP for multi-GPU training or FSDP for models too large to fit on a single GPU. Manage hyperparameter search with a systematic tool (Optuna, Ray Tune, W&B Sweeps) rather than manual iteration.
+
+## Deep Guidance
+
+### Data Loaders
+
+`torch.utils.data.DataLoader` is the standard interface for batched data loading. Configure it correctly:
+
+```python
+from torch.utils.data import DataLoader
+from src.data.dataset import MyDataset
+
+def build_dataloader(
+    dataset: MyDataset,
+    batch_size: int,
+    split: str,
+    num_workers: int = 4,
+) -> DataLoader:
+    is_train = split == "train"
+    return DataLoader(
+        dataset,
+        batch_size=batch_size,
+        shuffle=is_train,           # Shuffle only training data
+        num_workers=num_workers,     # Parallel data loading workers
+        pin_memory=True,             # Pin CPU memory for faster GPU transfer
+        prefetch_factor=2,           # Prefetch 2 batches per worker
+        persistent_workers=True,     # Keep workers alive between epochs
+        drop_last=is_train,          # Drop incomplete final batch (training only)
+    )
+```
+
+**`num_workers` guidance**:
+- Start with `min(os.cpu_count(), 8)` and tune from there
+- Set to 0 for debugging (single-process, easier stack traces)
+- On Windows, set to 0 if you encounter multiprocessing issues
+- Bottleneck check: if GPU utilisation < 80%, increase workers or enable prefetch
+
+**Common data loader bugs**:
+- Using `shuffle=True` on validation/test sets (breaks reproducibility checks)
+- Not setting `worker_init_fn` when using random augmentation in workers (workers share the same seed without this)
+- `pin_memory=True` on a machine without GPU (no-op but wastes memory)
+
+### Training Loop Structure
+
+```python
+def train_epoch(
+    model: nn.Module,
+    loader: DataLoader,
+    optimizer: torch.optim.Optimizer,
+    criterion: nn.Module,
+    scaler: torch.cuda.amp.GradScaler,
+    device: torch.device,
+) -> dict[str, float]:
+    model.train()
+    total_loss = 0.0
+    n_batches = 0
+
+    for batch in loader:
+        inputs, targets = batch
+        inputs = inputs.to(device, non_blocking=True)
+        targets = targets.to(device, non_blocking=True)
+
+        # Zero gradients BEFORE forward pass
+        optimizer.zero_grad(set_to_none=True)  # Faster than zero_grad()
+
+        # Mixed precision forward pass
+        with torch.autocast(device_type="cuda", dtype=torch.float16):
+            outputs = model(inputs)
+            loss = criterion(outputs, targets)
+
+        # Scaled backward pass
+        scaler.scale(loss).backward()
+
+        # Gradient clipping (before unscaling)
+        scaler.unscale_(optimizer)
+        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
+
+        # Optimizer step
+        scaler.step(optimizer)
+        scaler.update()
+
+        total_loss += loss.item()
+        n_batches += 1
+
+    return {"loss": total_loss / n_batches}
+```
+
+**Critical training loop rules**:
+1. `model.train()` before training, `model.eval()` before evaluation — these affect BatchNorm and Dropout
+2. `optimizer.zero_grad()` at the start of each batch, not the end
+3. Clip gradients before the optimizer step
+4. Use `loss.item()` (not `loss`) when accumulating — `.item()` detaches from the computation graph
+
+### Mixed Precision Training
+
+Mixed precision (float16/bfloat16 for computation, float32 for parameters) typically provides 2–3x speedup on modern GPUs with minimal accuracy impact:
+
+```python
+# Setup
+scaler = torch.cuda.amp.GradScaler()
+
+# Training step (shown above)
+with torch.autocast(device_type="cuda", dtype=torch.float16):
+    outputs = model(inputs)
+    loss = criterion(outputs, targets)
+
+scaler.scale(loss).backward()
+scaler.unscale_(optimizer)
+torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
+scaler.step(optimizer)
+scaler.update()
+```
+
+**bfloat16 vs float16**:
+- `bfloat16`: Same dynamic range as float32, less precision. Better for training stability. Requires Ampere GPU (A100, A30, RTX 30xx) or newer.
+- `float16`: Better precision than bfloat16 but narrower dynamic range (overflow risk). Works on all CUDA GPUs.
+- Default to `bfloat16` on Ampere+, `float16` on older GPUs.
+
+### Checkpointing
+
+Save and restore training state completely — not just model weights:
+
+```python
+def save_checkpoint(
+    path: str,
+    model: nn.Module,
+    optimizer: torch.optim.Optimizer,
+    scheduler,
+    scaler: torch.cuda.amp.GradScaler,
+    epoch: int,
+    metrics: dict,
+) -> None:
+    torch.save({
+        "epoch": epoch,
+        "model_state_dict": model.state_dict(),
+        "optimizer_state_dict": optimizer.state_dict(),
+        "scheduler_state_dict": scheduler.state_dict(),
+        "scaler_state_dict": scaler.state_dict(),
+        "metrics": metrics,
+    }, path)
+
+def load_checkpoint(path: str, model, optimizer, scheduler, scaler):
+    checkpoint = torch.load(path, map_location="cpu")
+    model.load_state_dict(checkpoint["model_state_dict"])
+    optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
+    scheduler.load_state_dict(checkpoint["scheduler_state_dict"])
+    scaler.load_state_dict(checkpoint["scaler_state_dict"])
+    return checkpoint["epoch"], checkpoint["metrics"]
+```
+
+**Checkpoint strategy**:
+- Save every N epochs AND on best validation metric (two separate files)
+- Keep last K checkpoints (delete older ones to save disk)
+- Always test checkpoint resume — bugs in resume code are discovered in production during long training runs, not in testing
+
+### Distributed Training: DDP
+
+PyTorch DistributedDataParallel (DDP) is the standard for multi-GPU training. Each GPU runs an independent process with a full model copy; gradients are averaged across GPUs after each backward pass:
+
+```python
+# Launch: torchrun --nproc_per_node=4 train.py
+import torch.distributed as dist
+from torch.nn.parallel import DistributedDataParallel as DDP
+from torch.utils.data.distributed import DistributedSampler
+
+def train_distributed():
+    dist.init_process_group(backend="nccl")
+    rank = dist.get_rank()
+    local_rank = int(os.environ["LOCAL_RANK"])
+    world_size = dist.get_world_size()
+
+    device = torch.device(f"cuda:{local_rank}")
+    torch.cuda.set_device(device)
+
+    model = MyModel().to(device)
+    model = DDP(model, device_ids=[local_rank])
+
+    # Each rank gets a different data partition
+    sampler = DistributedSampler(dataset, num_replicas=world_size, rank=rank)
+    loader = DataLoader(dataset, sampler=sampler, batch_size=batch_size_per_gpu)
+
+    for epoch in range(epochs):
+        sampler.set_epoch(epoch)  # Required for shuffle to work correctly
+        train_epoch(model, loader, ...)
+
+    # Save only from rank 0
+    if rank == 0:
+        torch.save(model.module.state_dict(), "model.pt")  # .module unwraps DDP
+
+    dist.destroy_process_group()
+```
+
+**DDP best practices**:
+- Use `torchrun` (not `torch.multiprocessing.spawn`) for launch
+- Effective batch size = `batch_size_per_gpu × world_size` — scale learning rate accordingly (linear scaling rule)
+- Always call `sampler.set_epoch(epoch)` or shuffle is deterministic across epochs
+- Log and save only from rank 0
+
+### Distributed Training: FSDP
+
+Fully Sharded Data Parallel (FSDP) shards model parameters across GPUs, enabling training of models too large for a single GPU:
+
+```python
+from torch.distributed.fsdp import FullyShardedDataParallel as FSDP
+from torch.distributed.fsdp import MixedPrecision
+import torch
+
+bf16_policy = MixedPrecision(
+    param_dtype=torch.bfloat16,
+    reduce_dtype=torch.bfloat16,
+    buffer_dtype=torch.bfloat16,
+)
+
+model = FSDP(
+    model,
+    mixed_precision=bf16_policy,
+    auto_wrap_policy=transformer_auto_wrap_policy,  # Wrap each transformer layer
+)
+```
+
+Use FSDP when model parameters exceed single-GPU memory. Use DDP when the model fits on one GPU — DDP is simpler and has less communication overhead.
+
+### Hyperparameter Tuning
+
+Never tune hyperparameters manually at scale. Use a systematic search tool:
+
+**Optuna** (open source, flexible):
+```python
+import optuna
+
+def objective(trial: optuna.Trial) -> float:
+    lr = trial.suggest_float("lr", 1e-5, 1e-2, log=True)
+    batch_size = trial.suggest_categorical("batch_size", [16, 32, 64])
+    dropout = trial.suggest_float("dropout", 0.0, 0.5)
+
+    model = build_model(dropout=dropout)
+    val_loss = train_and_evaluate(model, lr=lr, batch_size=batch_size)
+    return val_loss  # Optuna minimises by default
+
+study = optuna.create_study(direction="minimize", sampler=optuna.samplers.TPESampler())
+study.optimize(objective, n_trials=50, n_jobs=4)
+
+print(f"Best params: {study.best_params}")
+print(f"Best value: {study.best_value:.4f}")
+```
+
+**Key hyperparameters to tune** (in order of impact):
+1. Learning rate (most impactful — always tune first)
+2. Batch size (affects generalisation and training speed)
+3. Architecture (model size, depth, width)
+4. Regularisation (dropout, weight decay)
+5. Learning rate schedule (warmup steps, decay type)
+
+**Search strategies**:
+- Random search: Surprisingly effective, easy to parallelise
+- Bayesian optimisation (TPE in Optuna): More efficient for small budgets
+- Grid search: Only for 1–2 hyperparameters with small ranges
+
+Report the best result with multiple seeds (mean ± std) — a single seed result may be a lucky draw.

package/content/knowledge/mobile-app/mobile-app-architecture.md

@@ -0,0 +1,283 @@
+---
+name: mobile-app-architecture
+description: MVVM/MVI/TCA patterns, navigation architecture, dependency injection, and state management for iOS and Android mobile apps
+topics: [mobile-app, architecture, mvvm, mvi, tca, navigation, dependency-injection, state-management]
+---
+
+Mobile app architecture determines testability, scalability, and developer velocity. The wrong architecture is expensive to reverse — a monolithic ViewController or God Activity becomes unmaintainable at scale. Both iOS and Android ecosystems have converged on unidirectional data flow patterns: TCA and MVVM+Combine/async for iOS, MVI and MVVM+Flow for Android. Choose the pattern that matches your team's size and complexity requirements, not the most sophisticated available option.
+
+## Summary
+
+iOS architectures: MVVM with SwiftUI/Combine for mid-size apps, TCA (The Composable Architecture) for large apps requiring strict testability and state isolation. Android architectures: MVVM with StateFlow for most apps, MVI for complex state management. Both platforms benefit from clean architecture layers — presentation, domain, data — with dependency injection (Hilt for Android, constructor injection or a container for iOS). Navigation architecture is separate from view architecture: use Coordinator (iOS) or Navigation Component (Android).
+
+## Deep Guidance
+
+### iOS Architecture Patterns
+
+**MVVM with SwiftUI**
+
+The standard pattern for new iOS apps. The ViewModel is an `@Observable` class (iOS 17+) or `ObservableObject` (iOS 13+) that holds and transforms state:
+
+```swift
+@Observable
+final class UserProfileViewModel {
+    var user: User?
+    var isLoading = false
+    var error: Error?
+
+    private let repository: UserRepository
+
+    init(repository: UserRepository) {
+        self.repository = repository
+    }
+
+    func loadUser(id: String) async {
+        isLoading = true
+        defer { isLoading = false }
+        do {
+            user = try await repository.fetchUser(id: id)
+        } catch {
+            self.error = error
+        }
+    }
+}
+
+struct UserProfileView: View {
+    @State private var viewModel = UserProfileViewModel(repository: LiveUserRepository())
+
+    var body: some View {
+        Group {
+            if viewModel.isLoading { ProgressView() }
+            else if let user = viewModel.user { UserDetailView(user: user) }
+            else if viewModel.error != nil { ErrorView() }
+        }
+        .task { await viewModel.loadUser(id: userId) }
+    }
+}
+```
+
+Rules for healthy MVVM:
+- ViewModels must not import UIKit or SwiftUI — they are platform-agnostic
+- One ViewModel per screen/feature, not per view hierarchy level
+- ViewModels receive dependencies via constructor injection — no singletons
+- ViewModels hold only UI state, not business logic — business logic belongs in services/repositories
+- Test ViewModels by injecting fake dependencies and asserting state transitions
+
+**TCA (The Composable Architecture)**
+
+For large apps with complex state, strict testability requirements, or large teams. TCA provides a single-direction state mutation model:
+
+```swift
+@Reducer
+struct UserProfileFeature {
+    @ObservableState
+    struct State: Equatable {
+        var user: User?
+        var isLoading = false
+        var error: String?
+    }
+
+    enum Action {
+        case loadUser(String)
+        case userLoaded(Result<User, Error>)
+    }
+
+    @Dependency(\.userRepository) var userRepository
+
+    var body: some ReducerOf<Self> {
+        Reduce { state, action in
+            switch action {
+            case .loadUser(let id):
+                state.isLoading = true
+                return .run { send in
+                    await send(.userLoaded(Result { try await userRepository.fetchUser(id: id) }))
+                }
+            case .userLoaded(.success(let user)):
+                state.isLoading = false
+                state.user = user
+                return .none
+            case .userLoaded(.failure(let error)):
+                state.isLoading = false
+                state.error = error.localizedDescription
+                return .none
+            }
+        }
+    }
+}
+```
+
+TCA benefits: every state mutation is explicit, side effects are isolated and cancellable, testing is deterministic. TCA costs: steep learning curve, boilerplate-heavy for simple features, requires team-wide adoption to be consistent.
+
+**Clean Architecture layers for iOS**
+```
+Presentation Layer: Views + ViewModels
+    ↓ calls
+Domain Layer: Use Cases + Domain Models + Repository Protocols
+    ↓ calls
+Data Layer: Repository Implementations + Network + Persistence
+```
+
+- Domain layer has zero dependencies on UIKit, SwiftUI, or any specific framework
+- Repository protocols defined in the domain layer, implemented in the data layer
+- Use cases encapsulate single business operations: `FetchUserProfileUseCase`, `SubmitOrderUseCase`
+
+### Android Architecture Patterns
+
+**MVVM with StateFlow**
+
+The Google-recommended pattern, aligning with Android's official architecture guidance:
+
+```kotlin
+data class UserProfileUiState(
+    val user: User? = null,
+    val isLoading: Boolean = false,
+    val error: String? = null
+)
+
+@HiltViewModel
+class UserProfileViewModel @Inject constructor(
+    private val userRepository: UserRepository
+) : ViewModel() {
+
+    private val _uiState = MutableStateFlow(UserProfileUiState())
+    val uiState: StateFlow<UserProfileUiState> = _uiState.asStateFlow()
+
+    fun loadUser(userId: String) {
+        viewModelScope.launch {
+            _uiState.update { it.copy(isLoading = true) }
+            userRepository.fetchUser(userId)
+                .onSuccess { user ->
+                    _uiState.update { it.copy(user = user, isLoading = false) }
+                }
+                .onFailure { error ->
+                    _uiState.update { it.copy(error = error.message, isLoading = false) }
+                }
+        }
+    }
+}
+
+@Composable
+fun UserProfileScreen(viewModel: UserProfileViewModel = hiltViewModel()) {
+    val uiState by viewModel.uiState.collectAsStateWithLifecycle()
+    // render based on uiState
+}
+```
+
+**MVI (Model-View-Intent)**
+
+For features with complex state machines where MVVM state updates become hard to reason about:
+
+```kotlin
+sealed class UserProfileIntent {
+    data class LoadUser(val userId: String) : UserProfileIntent()
+    data object Refresh : UserProfileIntent()
+}
+
+sealed class UserProfileEffect {
+    data class ShowError(val message: String) : UserProfileEffect()
+    data object NavigateToLogin : UserProfileEffect()
+}
+```
+
+MVI separates user intentions from state mutations. The `SharedFlow` channel (`_effect`) handles one-shot events (navigation, toasts) that must not survive recomposition — a critical distinction from `StateFlow`.
+
+**One-shot events vs. state**
+- Use `StateFlow` for persistent UI state: loading, data, errors that survive recomposition
+- Use `SharedFlow` or `Channel` (as `Flow`) for one-shot effects: navigation commands, snackbar messages, dialog triggers
+- Never put navigation events in `StateFlow` — they replay on configuration change, causing double navigation
+
+**Clean Architecture layers for Android**
+```
+UI Layer: Composables + ViewModels
+    ↓ calls
+Domain Layer: Use Cases + Domain Models + Repository Interfaces
+    ↓ calls
+Data Layer: Repository Implementations + RemoteDataSource + LocalDataSource
+```
+
+- Domain layer: pure Kotlin module (`core/domain`) with no Android dependencies
+- Use cases: single `operator fun invoke()` or `execute()` function
+- Repository pattern: the domain defines the interface; data layer implements it
+
+### Dependency Injection
+
+**Android: Hilt**
+
+Hilt is the recommended DI framework for Android:
+
+```kotlin
+// Define a module
+@Module
+@InstallIn(SingletonComponent::class)
+object NetworkModule {
+    @Provides
+    @Singleton
+    fun provideOkHttpClient(): OkHttpClient = OkHttpClient.Builder().build()
+
+    @Provides
+    @Singleton
+    fun provideRetrofit(client: OkHttpClient): Retrofit = Retrofit.Builder()
+        .baseUrl(BuildConfig.BASE_URL)
+        .client(client)
+        .addConverterFactory(GsonConverterFactory.create())
+        .build()
+}
+
+// Inject into ViewModel
+@HiltViewModel
+class HomeViewModel @Inject constructor(
+    private val userRepository: UserRepository,
+    private val analyticsService: AnalyticsService
+) : ViewModel()
+```
+
+Hilt scopes: `SingletonComponent` (app lifetime), `ActivityRetainedComponent` (ViewModel lifetime), `ViewModelComponent` (ViewModel scope), `FragmentComponent`, `ActivityComponent`. Match the scope to the dependency's actual lifetime.
+
+**iOS: Constructor injection + container**
+
+Swift does not have a dominant DI framework. Use constructor injection as the default:
+
+```swift
+// Dependency container
+final class AppDependencies {
+    static let shared = AppDependencies()
+
+    lazy var networkClient: NetworkClient = URLSessionNetworkClient()
+    lazy var userRepository: UserRepository = NetworkUserRepository(client: networkClient)
+    lazy var analyticsService: AnalyticsService = FirebaseAnalyticsService()
+}
+
+// Inject at the composition root (app entry point or Coordinator)
+let viewModel = UserProfileViewModel(
+    repository: AppDependencies.shared.userRepository
+)
+```
+
+For testing: define protocols for all dependencies and inject fakes in tests. Never call `AppDependencies.shared` inside a ViewModel — inject via constructor.
+
+### State Management
+
+**iOS state scoping**
+- `@State`: view-local ephemeral state (animation flags, text field values) — does not survive view destruction
+- `@Binding`: two-way binding from parent to child — child can mutate parent's state
+- `@Observable` / `@ObservableObject`: shared mutable state in a ViewModel — survives view re-renders
+- `@Environment`: dependency injection through the view tree (theme, locale, custom services)
+- `@EnvironmentObject`: globally shared state accessed without explicit passing — use sparingly, only for truly app-wide state (user session, theme)
+- Avoid prop-drilling state through 4+ view layers — use `@Environment` or restructure to lift state to a shared ancestor ViewModel
+
+**Android state scoping**
+- `remember { }`: view-local ephemeral state in Compose — survives recomposition, not configuration change
+- `rememberSaveable { }`: survives configuration change by saving to Bundle
+- `StateFlow` in ViewModel: survives configuration change automatically (ViewModel lifecycle)
+- `SavedStateHandle` in ViewModel: persists through process death for critical state (form data, scroll position)
+- Hoist state to the lowest ancestor that needs it — do not hoist everything to the ViewModel
+
+**Handling configuration changes (Android)**
+- ViewModel automatically survives rotation and theme change — the primary benefit of the ViewModel
+- Always collect `StateFlow` with `collectAsStateWithLifecycle()` in Compose — stops collection when UI is not visible, preventing wasted work and crashes in background
+
+**Background state handling (iOS)**
+- ScenePhase: observe `\.scenePhase` in SwiftUI to react to foreground/background transitions
+- Save in-progress work when entering background: `@Environment(\.scenePhase) var scenePhase`
+- `@AppStorage`: lightweight persistence backed by UserDefaults for small values
+- Combine state from multiple sources with `Publishers.CombineLatest` or async/await TaskGroup