@devo-bmad-custom/agent-orchestration 1.0.1 → 1.0.3

package/src/.agents/skills/ui-ux-pro-custom/data/swift-ios-skills/apple-on-device-ai/references/mlx-swift.md

@@ -1,285 +1,285 @@
-# MLX Swift & llama.cpp Reference
-
-Complete reference for running open-source LLMs on Apple platforms using
-MLX Swift and llama.cpp.
-
-## Contents
-
-- [MLX Swift](#mlx-swift)
-- [llama.cpp](#llamacpp)
-- [Multi-Backend Architecture](#multi-backend-architecture)
-- [Built-in Apple Frameworks](#built-in-apple-frameworks)
-- [Performance Best Practices](#performance-best-practices)
-- [Review Checklist](#review-checklist)
-
-## MLX Swift
-
-Apple's ML framework for Swift. Highest sustained generation throughput on
-Apple Silicon via unified memory architecture.
-
-### Key Characteristics
-
-- Unified memory: operations run on CPU or GPU without data transfer
-- Lazy computation: operations computed only when needed
-- Automatic differentiation for training
-- Metal GPU acceleration
-- Research-oriented but increasingly used in production
-
-### Loading and Running LLMs
-
-```swift
-import MLX
-import MLXLLM
-
-let config = ModelConfiguration(
-    id: "mlx-community/Mistral-7B-Instruct-v0.3-4bit"
-)
-let model = try await LLMModelFactory.shared.loadContainer(
-    configuration: config
-)
-
-try await model.perform { context in
-    let input = try await context.processor.prepare(
-        input: UserInput(prompt: "Hello")
-    )
-    let stream = try generate(
-        input: input,
-        parameters: GenerateParameters(temperature: 0.0),
-        context: context
-    )
-    for await part in stream {
-        print(part.chunk ?? "", terminator: "")
-    }
-}
-```
-
-### Recommended Models by Device
-
-| Device | RAM | Recommended Model | Disk Size | RAM Usage |
-|---|---|---|---|---|
-| iPhone 12-14 | 4-6 GB | SmolLM2-135M or Qwen 2.5 0.5B | ~278 MB | ~0.3 GB |
-| iPhone 15 Pro+ | 8 GB | Gemma 3n E4B 4-bit | ~2.7 GB | ~3.5 GB |
-| Mac 8 GB | 8 GB | Llama 3.2 3B 4-bit | ~1.8 GB | ~3 GB |
-| Mac 16 GB+ | 16 GB+ | Mistral 7B 4-bit | ~4 GB | ~6 GB |
-
-### Memory Management Rules
-
-1. Never exceed 60% of total RAM on iOS
-2. Set GPU cache limits:
-   ```swift
-   MLX.GPU.set(cacheLimit: 512 * 1024 * 1024) // 512 MB
-   ```
-3. Monitor memory pressure and reduce cache under pressure
-4. Unload models on app backgrounding
-5. Use "Increased Memory Limit" entitlement for larger models on iOS
-6. Pre-flight memory checks before loading models
-7. Physical device required (no simulator support for Metal GPU)
-
-### Model Lifecycle Management
-
-```swift
-@Observable
-class ModelManager {
-    private var model: LLMModelContainer?
-    private var generationCount = 0
-
-    func loadModel() async throws {
-        let config = ModelConfiguration(
-            id: "mlx-community/Llama-3.2-3B-Instruct-4bit"
-        )
-        model = try await LLMModelFactory.shared.loadContainer(
-            configuration: config
-        )
-    }
-
-    func unloadModel() {
-        model = nil
-        MLX.GPU.set(cacheLimit: 0)
-    }
-}
-```
-
-Key lifecycle patterns:
-- Track active generation count to distinguish "loaded but idle" from
-  "generating"
-- Unconditional cancellation on app backgrounding
-- 5-second delayed force-unload after backgrounding
-- Platform-specific memory monitoring (UIKit on iOS, DispatchSource on macOS)
-
-### Background Handling
-
-```swift
-// iOS: Observe app lifecycle
-NotificationCenter.default.addObserver(
-    forName: UIApplication.didEnterBackgroundNotification,
-    object: nil, queue: .main
-) { _ in
-    modelManager.cancelGeneration()
-    Task {
-        try await Task.sleep(for: .seconds(5))
-        modelManager.unloadModel()
-    }
-}
-```
-
-## llama.cpp
-
-C/C++ LLM inference engine. Best cross-platform support. Uses GGUF model format.
-
-### Swift Integration (swift-llama-cpp)
-
-```swift
-import SwiftLlamaCpp
-
-let service = LlamaService(
-    modelUrl: modelURL,
-    config: .init(
-        batchSize: 256,
-        maxTokenCount: 4096,
-        useGPU: true
-    )
-)
-
-let messages = [
-    LlamaChatMessage(role: .system, content: "You are helpful."),
-    LlamaChatMessage(role: .user, content: "Hello")
-]
-
-let stream = try await service.streamCompletion(
-    of: messages,
-    samplingConfig: .init(temperature: 0.8)
-)
-for try await token in stream {
-    print(token, terminator: "")
-}
-```
-
-### GGUF Quantization Levels
-
-| Level | Quality | Size | Use Case |
-|---|---|---|---|
-| Q2_K | Lowest | Smallest | Extreme memory constraints |
-| Q4_K_M | Good | Balanced | Mobile devices (recommended) |
-| Q5_K_M | Higher | Larger | When quality matters more |
-| Q8_0 | Near-original | Largest | Desktop with ample RAM |
-
-### llama.cpp vs MLX Swift
-
-| Aspect | llama.cpp | MLX Swift |
-|---|---|---|
-| Model format | GGUF | Hugging Face / MLX format |
-| Platform support | Cross-platform | Apple only |
-| Throughput (Apple Silicon) | Good | Best |
-| Model ecosystem | Broadest | mlx-community models |
-| Maturity | Very mature | Evolving |
-| Memory efficiency | Excellent | Good |
-
-## Multi-Backend Architecture
-
-When an app needs multiple AI backends:
-
-### Fallback Chain Pattern
-
-```swift
-func respond(to prompt: String) async throws -> String {
-    // Try Foundation Models first (zero setup, best integration)
-    if SystemLanguageModel.default.isAvailable {
-        return try await foundationModelsRespond(prompt)
-    }
-
-    // Fall back to MLX Swift (best throughput)
-    if canLoadMLXModel() {
-        return try await mlxRespond(prompt)
-    }
-
-    // Fall back to llama.cpp (broadest compatibility)
-    if llamaModelAvailable() {
-        return try await llamaRespond(prompt)
-    }
-
-    throw AIError.noBackendAvailable
-}
-```
-
-### Architecture Guidelines
-
-1. Create a router that checks Foundation Models availability first
-2. Fall back to MLX or llama.cpp when Foundation Models is unavailable
-3. Define model tiers based on device capabilities
-4. Serialize all model access through a coordinator actor to prevent contention
-5. Ensure tool systems work across backends (schema translation may be needed)
-
-### Coordinator Actor
-
-```swift
-actor ModelCoordinator {
-    private var activeBackend: Backend?
-
-    func withExclusiveAccess<T>(
-        _ work: () async throws -> T
-    ) async rethrows -> T {
-        try await work()
-    }
-
-    enum Backend {
-        case foundationModels
-        case mlx
-        case llamaCpp
-    }
-}
-```
-
-## Built-in Apple Frameworks
-
-Before reaching for custom models, consider built-in frameworks:
-
-### Natural Language Framework
-
-No model downloads required:
-
-- `NLLanguageRecognizer` -- Language detection
-- `NLTokenizer` -- Word, sentence, paragraph tokenization
-- `NLTagger` -- Parts of speech, named entity recognition, sentiment
-- `NLEmbedding` -- Word and sentence vectors, similarity search
-
-### Vision Framework
-
-Built-in computer vision (legacy `VN*` API; for iOS 18+ prefer modern Swift equivalents like `RecognizeTextRequest`):
-
-- `VNRecognizeTextRequest` -- OCR
-- `VNClassifyImageRequest` -- Image classification
-- `VNDetectFaceRectanglesRequest` -- Face detection
-- `VNDetectHumanBodyPoseRequest` -- Body pose estimation
-
-### Create ML
-
-Training custom classifiers directly on device or Mac:
-
-- Image classification
-- Text classification
-- Tabular data models
-- Sound classification
-
-## Performance Best Practices
-
-1. Run outside debugger for accurate benchmarks (Xcode: Cmd-Opt-R, uncheck
-   "Debug Executable")
-2. Use `session.prewarm()` for Foundation Models before user interaction
-3. Batch Vision framework requests in a single `perform()` call
-4. Use `.fast` recognition level for real-time camera processing
-5. Neural Engine (Core ML) is most energy-efficient for compatible operations
-6. For MLX Swift, monitor token generation speed and adjust model size if
-   below acceptable thresholds
-
-## Review Checklist
-
-- [ ] Model size appropriate for target device RAM
-- [ ] Memory pressure monitoring implemented
-- [ ] Models unloaded on app backgrounding
-- [ ] GPU cache limits set appropriately
-- [ ] Pre-flight memory check before loading large models
-- [ ] Fallback strategy when model unavailable
-- [ ] All model access serialized through coordinator
-- [ ] Quantization level appropriate for quality/size tradeoff
-- [ ] Energy efficiency considered (Neural Engine vs GPU)
-- [ ] Physical device testing (not simulator) for Metal-dependent code
+# MLX Swift & llama.cpp Reference
+
+Complete reference for running open-source LLMs on Apple platforms using
+MLX Swift and llama.cpp.
+
+## Contents
+
+- [MLX Swift](#mlx-swift)
+- [llama.cpp](#llamacpp)
+- [Multi-Backend Architecture](#multi-backend-architecture)
+- [Built-in Apple Frameworks](#built-in-apple-frameworks)
+- [Performance Best Practices](#performance-best-practices)
+- [Review Checklist](#review-checklist)
+
+## MLX Swift
+
+Apple's ML framework for Swift. Highest sustained generation throughput on
+Apple Silicon via unified memory architecture.
+
+### Key Characteristics
+
+- Unified memory: operations run on CPU or GPU without data transfer
+- Lazy computation: operations computed only when needed
+- Automatic differentiation for training
+- Metal GPU acceleration
+- Research-oriented but increasingly used in production
+
+### Loading and Running LLMs
+
+```swift
+import MLX
+import MLXLLM
+
+let config = ModelConfiguration(
+    id: "mlx-community/Mistral-7B-Instruct-v0.3-4bit"
+)
+let model = try await LLMModelFactory.shared.loadContainer(
+    configuration: config
+)
+
+try await model.perform { context in
+    let input = try await context.processor.prepare(
+        input: UserInput(prompt: "Hello")
+    )
+    let stream = try generate(
+        input: input,
+        parameters: GenerateParameters(temperature: 0.0),
+        context: context
+    )
+    for await part in stream {
+        print(part.chunk ?? "", terminator: "")
+    }
+}
+```
+
+### Recommended Models by Device
+
+| Device | RAM | Recommended Model | Disk Size | RAM Usage |
+|---|---|---|---|---|
+| iPhone 12-14 | 4-6 GB | SmolLM2-135M or Qwen 2.5 0.5B | ~278 MB | ~0.3 GB |
+| iPhone 15 Pro+ | 8 GB | Gemma 3n E4B 4-bit | ~2.7 GB | ~3.5 GB |
+| Mac 8 GB | 8 GB | Llama 3.2 3B 4-bit | ~1.8 GB | ~3 GB |
+| Mac 16 GB+ | 16 GB+ | Mistral 7B 4-bit | ~4 GB | ~6 GB |
+
+### Memory Management Rules
+
+1. Never exceed 60% of total RAM on iOS
+2. Set GPU cache limits:
+   ```swift
+   MLX.GPU.set(cacheLimit: 512 * 1024 * 1024) // 512 MB
+   ```
+3. Monitor memory pressure and reduce cache under pressure
+4. Unload models on app backgrounding
+5. Use "Increased Memory Limit" entitlement for larger models on iOS
+6. Pre-flight memory checks before loading models
+7. Physical device required (no simulator support for Metal GPU)
+
+### Model Lifecycle Management
+
+```swift
+@Observable
+class ModelManager {
+    private var model: LLMModelContainer?
+    private var generationCount = 0
+
+    func loadModel() async throws {
+        let config = ModelConfiguration(
+            id: "mlx-community/Llama-3.2-3B-Instruct-4bit"
+        )
+        model = try await LLMModelFactory.shared.loadContainer(
+            configuration: config
+        )
+    }
+
+    func unloadModel() {
+        model = nil
+        MLX.GPU.set(cacheLimit: 0)
+    }
+}
+```
+
+Key lifecycle patterns:
+- Track active generation count to distinguish "loaded but idle" from
+  "generating"
+- Unconditional cancellation on app backgrounding
+- 5-second delayed force-unload after backgrounding
+- Platform-specific memory monitoring (UIKit on iOS, DispatchSource on macOS)
+
+### Background Handling
+
+```swift
+// iOS: Observe app lifecycle
+NotificationCenter.default.addObserver(
+    forName: UIApplication.didEnterBackgroundNotification,
+    object: nil, queue: .main
+) { _ in
+    modelManager.cancelGeneration()
+    Task {
+        try await Task.sleep(for: .seconds(5))
+        modelManager.unloadModel()
+    }
+}
+```
+
+## llama.cpp
+
+C/C++ LLM inference engine. Best cross-platform support. Uses GGUF model format.
+
+### Swift Integration (swift-llama-cpp)
+
+```swift
+import SwiftLlamaCpp
+
+let service = LlamaService(
+    modelUrl: modelURL,
+    config: .init(
+        batchSize: 256,
+        maxTokenCount: 4096,
+        useGPU: true
+    )
+)
+
+let messages = [
+    LlamaChatMessage(role: .system, content: "You are helpful."),
+    LlamaChatMessage(role: .user, content: "Hello")
+]
+
+let stream = try await service.streamCompletion(
+    of: messages,
+    samplingConfig: .init(temperature: 0.8)
+)
+for try await token in stream {
+    print(token, terminator: "")
+}
+```
+
+### GGUF Quantization Levels
+
+| Level | Quality | Size | Use Case |
+|---|---|---|---|
+| Q2_K | Lowest | Smallest | Extreme memory constraints |
+| Q4_K_M | Good | Balanced | Mobile devices (recommended) |
+| Q5_K_M | Higher | Larger | When quality matters more |
+| Q8_0 | Near-original | Largest | Desktop with ample RAM |
+
+### llama.cpp vs MLX Swift
+
+| Aspect | llama.cpp | MLX Swift |
+|---|---|---|
+| Model format | GGUF | Hugging Face / MLX format |
+| Platform support | Cross-platform | Apple only |
+| Throughput (Apple Silicon) | Good | Best |
+| Model ecosystem | Broadest | mlx-community models |
+| Maturity | Very mature | Evolving |
+| Memory efficiency | Excellent | Good |
+
+## Multi-Backend Architecture
+
+When an app needs multiple AI backends:
+
+### Fallback Chain Pattern
+
+```swift
+func respond(to prompt: String) async throws -> String {
+    // Try Foundation Models first (zero setup, best integration)
+    if SystemLanguageModel.default.isAvailable {
+        return try await foundationModelsRespond(prompt)
+    }
+
+    // Fall back to MLX Swift (best throughput)
+    if canLoadMLXModel() {
+        return try await mlxRespond(prompt)
+    }
+
+    // Fall back to llama.cpp (broadest compatibility)
+    if llamaModelAvailable() {
+        return try await llamaRespond(prompt)
+    }
+
+    throw AIError.noBackendAvailable
+}
+```
+
+### Architecture Guidelines
+
+1. Create a router that checks Foundation Models availability first
+2. Fall back to MLX or llama.cpp when Foundation Models is unavailable
+3. Define model tiers based on device capabilities
+4. Serialize all model access through a coordinator actor to prevent contention
+5. Ensure tool systems work across backends (schema translation may be needed)
+
+### Coordinator Actor
+
+```swift
+actor ModelCoordinator {
+    private var activeBackend: Backend?
+
+    func withExclusiveAccess<T>(
+        _ work: () async throws -> T
+    ) async rethrows -> T {
+        try await work()
+    }
+
+    enum Backend {
+        case foundationModels
+        case mlx
+        case llamaCpp
+    }
+}
+```
+
+## Built-in Apple Frameworks
+
+Before reaching for custom models, consider built-in frameworks:
+
+### Natural Language Framework
+
+No model downloads required:
+
+- `NLLanguageRecognizer` -- Language detection
+- `NLTokenizer` -- Word, sentence, paragraph tokenization
+- `NLTagger` -- Parts of speech, named entity recognition, sentiment
+- `NLEmbedding` -- Word and sentence vectors, similarity search
+
+### Vision Framework
+
+Built-in computer vision (legacy `VN*` API; for iOS 18+ prefer modern Swift equivalents like `RecognizeTextRequest`):
+
+- `VNRecognizeTextRequest` -- OCR
+- `VNClassifyImageRequest` -- Image classification
+- `VNDetectFaceRectanglesRequest` -- Face detection
+- `VNDetectHumanBodyPoseRequest` -- Body pose estimation
+
+### Create ML
+
+Training custom classifiers directly on device or Mac:
+
+- Image classification
+- Text classification
+- Tabular data models
+- Sound classification
+
+## Performance Best Practices
+
+1. Run outside debugger for accurate benchmarks (Xcode: Cmd-Opt-R, uncheck
+   "Debug Executable")
+2. Use `session.prewarm()` for Foundation Models before user interaction
+3. Batch Vision framework requests in a single `perform()` call
+4. Use `.fast` recognition level for real-time camera processing
+5. Neural Engine (Core ML) is most energy-efficient for compatible operations
+6. For MLX Swift, monitor token generation speed and adjust model size if
+   below acceptable thresholds
+
+## Review Checklist
+
+- [ ] Model size appropriate for target device RAM
+- [ ] Memory pressure monitoring implemented
+- [ ] Models unloaded on app backgrounding
+- [ ] GPU cache limits set appropriately
+- [ ] Pre-flight memory check before loading large models
+- [ ] Fallback strategy when model unavailable
+- [ ] All model access serialized through coordinator
+- [ ] Quantization level appropriate for quality/size tradeoff
+- [ ] Energy efficiency considered (Neural Engine vs GPU)
+- [ ] Physical device testing (not simulator) for Metal-dependent code