buildanything 1.7.1 → 1.8.0

package/skills/ios/apple-on-device-ai/references/mlx-swift.md

@@ -0,0 +1,285 @@
+# MLX Swift & llama.cpp Reference
+
+Complete reference for running open-source LLMs on Apple platforms using
+MLX Swift and llama.cpp.
+
+## Contents
+
+- [MLX Swift](#mlx-swift)
+- [llama.cpp](#llamacpp)
+- [Multi-Backend Architecture](#multi-backend-architecture)
+- [Built-in Apple Frameworks](#built-in-apple-frameworks)
+- [Performance Best Practices](#performance-best-practices)
+- [Review Checklist](#review-checklist)
+
+## MLX Swift
+
+Apple's ML framework for Swift. Highest sustained generation throughput on
+Apple Silicon via unified memory architecture.
+
+### Key Characteristics
+
+- Unified memory: operations run on CPU or GPU without data transfer
+- Lazy computation: operations computed only when needed
+- Automatic differentiation for training
+- Metal GPU acceleration
+- Research-oriented but increasingly used in production
+
+### Loading and Running LLMs
+
+```swift
+import MLX
+import MLXLLM
+
+let config = ModelConfiguration(
+    id: "mlx-community/Mistral-7B-Instruct-v0.3-4bit"
+)
+let model = try await LLMModelFactory.shared.loadContainer(
+    configuration: config
+)
+
+try await model.perform { context in
+    let input = try await context.processor.prepare(
+        input: UserInput(prompt: "Hello")
+    )
+    let stream = try generate(
+        input: input,
+        parameters: GenerateParameters(temperature: 0.0),
+        context: context
+    )
+    for await part in stream {
+        print(part.chunk ?? "", terminator: "")
+    }
+}
+```
+
+### Recommended Models by Device
+
+| Device | RAM | Recommended Model | Disk Size | RAM Usage |
+|---|---|---|---|---|
+| iPhone 12-14 | 4-6 GB | SmolLM2-135M or Qwen 2.5 0.5B | ~278 MB | ~0.3 GB |
+| iPhone 15 Pro+ | 8 GB | Gemma 3n E4B 4-bit | ~2.7 GB | ~3.5 GB |
+| Mac 8 GB | 8 GB | Llama 3.2 3B 4-bit | ~1.8 GB | ~3 GB |
+| Mac 16 GB+ | 16 GB+ | Mistral 7B 4-bit | ~4 GB | ~6 GB |
+
+### Memory Management Rules
+
+1. Never exceed 60% of total RAM on iOS
+2. Set GPU cache limits:
+   ```swift
+   MLX.GPU.set(cacheLimit: 512 * 1024 * 1024) // 512 MB
+   ```
+3. Monitor memory pressure and reduce cache under pressure
+4. Unload models on app backgrounding
+5. Use "Increased Memory Limit" entitlement for larger models on iOS
+6. Pre-flight memory checks before loading models
+7. Physical device required (no simulator support for Metal GPU)
+
+### Model Lifecycle Management
+
+```swift
+@Observable
+class ModelManager {
+    private var model: LLMModelContainer?
+    private var generationCount = 0
+
+    func loadModel() async throws {
+        let config = ModelConfiguration(
+            id: "mlx-community/Llama-3.2-3B-Instruct-4bit"
+        )
+        model = try await LLMModelFactory.shared.loadContainer(
+            configuration: config
+        )
+    }
+
+    func unloadModel() {
+        model = nil
+        MLX.GPU.set(cacheLimit: 0)
+    }
+}
+```
+
+Key lifecycle patterns:
+- Track active generation count to distinguish "loaded but idle" from
+  "generating"
+- Unconditional cancellation on app backgrounding
+- 5-second delayed force-unload after backgrounding
+- Platform-specific memory monitoring (UIKit on iOS, DispatchSource on macOS)
+
+### Background Handling
+
+```swift
+// iOS: Observe app lifecycle
+NotificationCenter.default.addObserver(
+    forName: UIApplication.didEnterBackgroundNotification,
+    object: nil, queue: .main
+) { _ in
+    modelManager.cancelGeneration()
+    Task {
+        try await Task.sleep(for: .seconds(5))
+        modelManager.unloadModel()
+    }
+}
+```
+
+## llama.cpp
+
+C/C++ LLM inference engine. Best cross-platform support. Uses GGUF model format.
+
+### Swift Integration (swift-llama-cpp)
+
+```swift
+import SwiftLlamaCpp
+
+let service = LlamaService(
+    modelUrl: modelURL,
+    config: .init(
+        batchSize: 256,
+        maxTokenCount: 4096,
+        useGPU: true
+    )
+)
+
+let messages = [
+    LlamaChatMessage(role: .system, content: "You are helpful."),
+    LlamaChatMessage(role: .user, content: "Hello")
+]
+
+let stream = try await service.streamCompletion(
+    of: messages,
+    samplingConfig: .init(temperature: 0.8)
+)
+for try await token in stream {
+    print(token, terminator: "")
+}
+```
+
+### GGUF Quantization Levels
+
+| Level | Quality | Size | Use Case |
+|---|---|---|---|
+| Q2_K | Lowest | Smallest | Extreme memory constraints |
+| Q4_K_M | Good | Balanced | Mobile devices (recommended) |
+| Q5_K_M | Higher | Larger | When quality matters more |
+| Q8_0 | Near-original | Largest | Desktop with ample RAM |
+
+### llama.cpp vs MLX Swift
+
+| Aspect | llama.cpp | MLX Swift |
+|---|---|---|
+| Model format | GGUF | Hugging Face / MLX format |
+| Platform support | Cross-platform | Apple only |
+| Throughput (Apple Silicon) | Good | Best |
+| Model ecosystem | Broadest | mlx-community models |
+| Maturity | Very mature | Evolving |
+| Memory efficiency | Excellent | Good |
+
+## Multi-Backend Architecture
+
+When an app needs multiple AI backends:
+
+### Fallback Chain Pattern
+
+```swift
+func respond(to prompt: String) async throws -> String {
+    // Try Foundation Models first (zero setup, best integration)
+    if SystemLanguageModel.default.isAvailable {
+        return try await foundationModelsRespond(prompt)
+    }
+
+    // Fall back to MLX Swift (best throughput)
+    if canLoadMLXModel() {
+        return try await mlxRespond(prompt)
+    }
+
+    // Fall back to llama.cpp (broadest compatibility)
+    if llamaModelAvailable() {
+        return try await llamaRespond(prompt)
+    }
+
+    throw AIError.noBackendAvailable
+}
+```
+
+### Architecture Guidelines
+
+1. Create a router that checks Foundation Models availability first
+2. Fall back to MLX or llama.cpp when Foundation Models is unavailable
+3. Define model tiers based on device capabilities
+4. Serialize all model access through a coordinator actor to prevent contention
+5. Ensure tool systems work across backends (schema translation may be needed)
+
+### Coordinator Actor
+
+```swift
+actor ModelCoordinator {
+    private var activeBackend: Backend?
+
+    func withExclusiveAccess<T>(
+        _ work: () async throws -> T
+    ) async rethrows -> T {
+        try await work()
+    }
+
+    enum Backend {
+        case foundationModels
+        case mlx
+        case llamaCpp
+    }
+}
+```
+
+## Built-in Apple Frameworks
+
+Before reaching for custom models, consider built-in frameworks:
+
+### Natural Language Framework
+
+No model downloads required:
+
+- `NLLanguageRecognizer` -- Language detection
+- `NLTokenizer` -- Word, sentence, paragraph tokenization
+- `NLTagger` -- Parts of speech, named entity recognition, sentiment
+- `NLEmbedding` -- Word and sentence vectors, similarity search
+
+### Vision Framework
+
+Built-in computer vision (legacy `VN*` API; for iOS 18+ prefer modern Swift equivalents like `RecognizeTextRequest`):
+
+- `VNRecognizeTextRequest` -- OCR
+- `VNClassifyImageRequest` -- Image classification
+- `VNDetectFaceRectanglesRequest` -- Face detection
+- `VNDetectHumanBodyPoseRequest` -- Body pose estimation
+
+### Create ML
+
+Training custom classifiers directly on device or Mac:
+
+- Image classification
+- Text classification
+- Tabular data models
+- Sound classification
+
+## Performance Best Practices
+
+1. Run outside debugger for accurate benchmarks (Xcode: Cmd-Opt-R, uncheck
+   "Debug Executable")
+2. Use `session.prewarm()` for Foundation Models before user interaction
+3. Batch Vision framework requests in a single `perform()` call
+4. Use `.fast` recognition level for real-time camera processing
+5. Neural Engine (Core ML) is most energy-efficient for compatible operations
+6. For MLX Swift, monitor token generation speed and adjust model size if
+   below acceptable thresholds
+
+## Review Checklist
+
+- [ ] Model size appropriate for target device RAM
+- [ ] Memory pressure monitoring implemented
+- [ ] Models unloaded on app backgrounding
+- [ ] GPU cache limits set appropriately
+- [ ] Pre-flight memory check before loading large models
+- [ ] Fallback strategy when model unavailable
+- [ ] All model access serialized through coordinator
+- [ ] Quantization level appropriate for quality/size tradeoff
+- [ ] Energy efficiency considered (Neural Engine vs GPU)
+- [ ] Physical device testing (not simulator) for Metal-dependent code

package/skills/ios/ios-26-platform/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: ios-26-platform
+description: Use when implementing iOS 26 APIs EXCEPT Liquid Glass (WebView, Chart3D, @Animatable, toolbar morphing, AlarmKit, FoundationModels, Visual Intelligence), deploying iOS 26+ apps, or supporting backward compatibility with iOS 17/18. For Liquid Glass, see `skills/ios/swiftui-liquid-glass/`.
+---
+
+# iOS 26 Platform
+
+iOS 26 introduces new SwiftUI APIs, platform frameworks, and automatic modernization at compile time. This skill covers iOS 26 APIs EXCEPT Liquid Glass.
+
+Liquid Glass → see `skills/ios/swiftui-liquid-glass/`
+
+## Overview
+
+iOS 26 modernizes UI with new SwiftUI APIs (WebView, Chart3D, @Animatable), and advanced features (@BackoffAnimation, free-form windows). The core principle: modern UI gets updated automatically at compile time from recompiling with Xcode 26.
+
+## Reference Loading Guide
+
+**ALWAYS load reference files if there is even a small chance the content may be required.** It's better to have the context than to miss a pattern or make a mistake.
+
+| Reference | Load When |
+|-----------|-----------|
+| **Liquid Glass** | → see `skills/ios/swiftui-liquid-glass/` |
+| **[Automatic Adoption](references/automatic-adoption.md)** | Understanding what iOS 26 changes automatically vs what requires code |
+| **[SwiftUI APIs](references/swiftui-apis.md)** | Using WebView, Chart3D, `@Animatable`, AttributedString, or new view modifiers |
+| **[Toolbar & Navigation](references/toolbar-navigation.md)** | Customizing toolbars with spacers, morphing, glass button styles, or search |
+| **[Backward Compatibility](references/backward-compat.md)** | Supporting iOS 17/18 alongside iOS 26, or using UIDesignRequiresCompatibility |
+
+## Core Workflow
+
+1. **Check deployment target** — iOS 26+ required for new APIs
+2. **Recompile with Xcode 26** — Standard controls get automatic modernization
+3. **Identify new API opportunities** — WebView, Chart3D, @Animatable, toolbar morphing
+4. **Add @available guards** — For backward compatibility with iOS 17/18
+5. **Test accessibility** — Verify Reduce Transparency, Increase Contrast, Reduce Motion
+
+Liquid Glass → see `skills/ios/swiftui-liquid-glass/`
+
+## Common Mistakes
+
+1. **Ignoring backward compatibility** — Targeting iOS 26+ without `@available` guards breaks iOS 17/18 support. Always use `if #available(iOS 26, *)` for new APIs.
+
+2. **Animation performance issues** — New iOS 26 animations can be expensive. Respect Reduce Motion accessibility setting and profile with Instruments 26 before shipping.
+
+3. **Not testing on actual devices** — Simulator rendering differs from hardware. Test new visual APIs on iPhone 15 Pro, iPad, and Mac to verify quality.
+
+4. **Using old UIView patterns with new SwiftUI APIs** — Mixing UIView-based navigation with iOS 26 SwiftUI creates inconsistent appearances. Migrate fully to SwiftUI or wrap carefully with UIViewRepresentable.
+
+Liquid Glass-specific pitfalls → see `skills/ios/swiftui-liquid-glass/`
+
+
+---
+
+Vendored from: https://github.com/johnrogers/claude-swift-engineering/tree/main/plugins/swift-engineering/skills/ios-26-platform

package/skills/ios/ios-26-platform/references/automatic-adoption.md

@@ -0,0 +1,161 @@
+# Automatic vs Explicit Adoption
+
+**iOS 26+**
+
+## What Changes Automatically (Recompile Only)
+
+### Standard Components Get Liquid Glass
+Simply recompile with Xcode 26 — no code changes:
+
+```swift
+// All these get glass automatically:
+NavigationView { ... }
+TabView { ... }
+.toolbar { ... }
+.sheet { ... }
+.popover { ... }
+List { ... }
+```
+
+### Controls Get New Appearance
+- Toggles, segmented pickers, sliders
+- Knobs transform into glass during interaction
+- Buttons morph into menus/popovers
+
+### Search Placement
+```swift
+NavigationSplitView {
+    List { }
+        .searchable(text: $searchText)
+}
+// Automatically:
+// - Bottom-aligned on iPhone (ergonomic)
+// - Top trailing on iPad
+```
+
+### Performance Improvements
+- **6x faster loading** for 100k+ item lists (macOS)
+- **16x faster updates** for large lists
+- Reduced dropped frames during scrolling
+- Better nested ScrollView performance with lazy stacks
+
+## What Requires Code Changes
+
+### Toolbar Customization
+```swift
+.toolbar {
+    ToolbarItemGroup(placement: .topBarTrailing) {
+        Button("Up") { }
+        Button("Down") { }
+
+        Spacer(.fixed) // NEW: Separates button groups
+
+        Button("Settings") { }
+    }
+}
+```
+
+### Prominent Toolbar Buttons
+```swift
+Button("Add Trip") {
+    addTrip()
+}
+.buttonStyle(.borderedProminent)
+.tint(.blue) // Tinting in glass toolbars
+```
+
+### Glass Effect for Custom Views
+```swift
+CustomPhotoGrid()
+    .glassBackgroundEffect() // Reflects surrounding content
+```
+
+### Search Tab Role
+```swift
+TabView {
+    SearchView()
+        .tabItem { Label("Search", systemImage: "magnifyingglass") }
+        .tabRole(.search) // Separates from tabs, morphs into search
+}
+```
+
+### Tab Bar Minimization
+```swift
+TabView { ... }
+    .tabBarMinimizationBehavior(.onScrollDown) // Recedes on scroll
+```
+
+## Audit Checklist
+
+### Remove These (Interfere with Glass)
+```swift
+// Remove custom backgrounds on navigation
+NavigationView { }
+    .background(Color.blue.opacity(0.5)) // Remove
+
+// Remove custom blur effects
+.background(
+    VisualEffectView(effect: UIBlurEffect(style: .systemMaterial))
+) // Remove — let system handle
+
+// Remove hard-coded dimensions
+Slider(value: $volume)
+    .frame(width: 250, height: 44) // Remove — new metrics
+```
+
+### Update These
+```swift
+// Section headers: Use title-style capitalization
+Section(header: Text("User Settings")) { } // Not "user settings"
+
+// Forms: Use grouped style
+Form { ... }
+    .formStyle(.grouped) // Platform-optimized metrics
+```
+
+## Safe Area Considerations
+
+```swift
+// When glass extends edge-to-edge with .ignoresSafeArea()
+// Use .safeAreaPadding() for content (not .padding())
+
+ZStack {
+    RoundedRectangle(cornerRadius: 12)
+        .fill(.thinMaterial)
+        .ignoresSafeArea()
+
+    VStack {
+        content
+    }
+    .safeAreaPadding(.horizontal, 20) // Respects notch/home indicator
+}
+```
+
+## Testing Required
+
+1. **Light/dark modes** — Glass adapts independently
+2. **Reduce Transparency** — Glass becomes frostier
+3. **Increase Contrast** — Elements become black/white
+4. **Reduce Motion** — Elastic properties disabled
+5. **Dynamic Type** — Larger text sizes
+6. **Content scrolling** — Verify scroll edge effects
+
+## Platform-Specific Changes
+
+### iPad
+- Tab bar can adapt to sidebar
+- Resizable windows (UIRequiresFullscreen deprecated)
+- Menu bar via swipe down
+
+### macOS
+- Synchronized window resize animations
+- Window controls adapt automatically
+- Glassy sidebars reflect content
+
+### watchOS
+- Minimal changes, automatic adoption
+- Use standard toolbar APIs and button styles
+
+### tvOS
+- Focus-based glass appearance
+- Apple TV 4K 2nd gen+ required for full effects