@mmerterden/multi-agent-pipeline 20.7.0 → 20.8.1

package/pipeline/skills/shared/external/apple-on-device-ai/SKILL.md

@@ -1,493 +1,468 @@
 ---
 name: apple-on-device-ai
-description: "Integrate on-device AI using Foundation Models framework, Core ML, and open-source LLM runtimes on Apple Silicon. Covers Foundation Models (LanguageModelSession, @Generable, @Guide, SystemLanguageModel, structured output, tool calling), Core ML (coremltools, model conversion, quantization, palettization, pruning, Neural Engine, MLTensor), MLX Swift (transformer inference, unified memory), and llama.cpp (GGUF, cross-platform LLM). Use when building tool-calling AI features, working with guided generation schemas, converting models, or running on-device inference."
+description: "On-device AI on Apple hardware: Foundation Models (SystemLanguageModel, LanguageModelSession, @Generable and @Guide structured output, tool calling), coremltools conversion, quantization, palettization, pruning, Neural Engine, MLX Swift transformer inference on unified memory, llama.cpp GGUF models. Use when building tool-calling AI features, designing guided generation schemas, converting a model or running inference on device. Not for Swift-side Core ML integration (coreml), OCR, sentiment, NER or classifier training."
 metadata:
-  source: "dpearson2699/swift-ios-skills (PolyForm Perimeter 1.0.0)"
+  source: multi-agent-pipeline
 ---
 
-# On-Device AI for Apple Platforms
+# On-device AI on Apple platforms
 
-Guide for selecting, deploying, and optimizing on-device ML models. Covers Apple
-Foundation Models, Core ML, MLX Swift, and llama.cpp.
+Four stacks run models locally on Apple hardware. Pick by the job, the OS floor
+and the model you need, then follow the stack's rules for availability, memory
+and testing.
 
 ## Contents
 
-- [Framework Selection Router](#framework-selection-router)
-- [Apple Foundation Models Overview](#apple-foundation-models-overview)
-- [Core ML Overview](#core-ml-overview)
-- [MLX Swift Overview](#mlx-swift-overview)
-- [Multi-Backend Architecture](#multi-backend-architecture)
-- [Performance Best Practices](#performance-best-practices)
-- [Common Mistakes](#common-mistakes)
-- [Review Checklist](#review-checklist)
-- [References](#references)
+- [Choosing a stack](#choosing-a-stack)
+- [Foundation Models](#foundation-models)
+- [Core ML](#core-ml)
+- [MLX Swift](#mlx-swift)
+- [Running several backends](#running-several-backends)
+- [Performance habits](#performance-habits)
+- [Mistakes to catch](#mistakes-to-catch)
+- [Before approving](#before-approving)
+- [Reference files](#reference-files)
 
-## Framework Selection Router
+## Choosing a stack
 
-Use this decision tree to pick the right framework for your use case.
+**Foundation Models** gives apps the system language model. It runs on iOS 26+
+and macOS 26+ where Apple Intelligence is enabled, and covers summaries, text
+generation, typed output, pulling entities out of text, and brief back-and-forth
+chat. The app ships no API key, runs no server and makes no network call, yet it
+still has to cope with model assets that are not downloaded yet.
 
-### Apple Foundation Models
+- Fits: typed output with `@Generable`; summarizing, classifying and tagging;
+  generation that calls app code through the `Tool` protocol; features where
+  data must stay on the device.
+- Does not fit: complex math, writing code, answers that must be factually
+  right, or apps that still support releases before iOS 26.
 
-**When to use:** Text generation, summarization, entity extraction, structured
-output, and short dialog on iOS 26+ / macOS 26+ devices with Apple Intelligence
-enabled. No app-managed API key, network round trip, or model hosting; still
-handle system model asset readiness.
+**Core ML** runs models you trained yourself (vision, text, audio) on every
+Apple platform, after coremltools converts them from scikit-learn, TensorFlow
+or PyTorch.
 
-**Best for:**
-- Generating text or structured data with `@Generable` types
-- Summarization, classification, content tagging
-- Tool-augmented generation with the `Tool` protocol
-- Apps that need guaranteed on-device privacy
+- Fits: image classification, detection and segmentation; custom text or
+  sentiment classifiers; audio through SoundAnalysis; tuning for the Neural
+  Engine; models that need compressing by quantization, pruning or
+  palettization.
 
-**Not suited for:** Complex math, code generation, factual accuracy tasks,
-or apps targeting pre-iOS 26 devices.
+**MLX Swift** runs particular open-weight models (Gemma, Qwen, Mistral, Llama)
+and reaches the best sustained speed on Apple Silicon. It is also a good
+research and prototyping tool.
 
-### Core ML
+- Fits: highest token rate on Apple Silicon; models from the `mlx-community`
+  organisation on Hugging Face; research that needs automatic differentiation;
+  fine-tuning on a Mac.
 
-**When to use:** Deploying custom trained models (vision, NLP, audio) across all
-Apple platforms. Converting models from PyTorch, TensorFlow, or scikit-learn
-with coremltools.
+**llama.cpp** runs GGUF models on nearly any platform and suits production apps
+that need wide device coverage.
 
-**Best for:**
-- Image classification, object detection, segmentation
-- Custom NLP classifiers, sentiment analysis models
-- Audio/speech models via SoundAnalysis integration
-- Any scenario needing Neural Engine optimization
-- Models requiring quantization, palettization, or pruning
+- Fits: quantized GGUF files (Q4_K_M, Q5_K_M, Q8_0); one engine across iOS,
+  Android and desktop; the broadest open model ecosystem.
 
-### MLX Swift
+Quick routing:
 
-**When to use:** Running specific open-source LLMs (Llama, Mistral, Qwen, Gemma)
-on Apple Silicon with maximum throughput. Research and prototyping.
-
-**Best for:**
-- Highest sustained token generation on Apple Silicon
-- Running Hugging Face models from `mlx-community`
-- Research requiring automatic differentiation
-- Fine-tuning workflows on Mac
-
-### llama.cpp
-
-**When to use:** Cross-platform LLM inference using GGUF model format. Production
-deployments needing broad device support.
-
-**Best for:**
-- GGUF quantized models (Q4_K_M, Q5_K_M, Q8_0)
-- Cross-platform apps (iOS + Android + desktop)
-- Maximum compatibility with open-source model ecosystem
-
-### Quick Reference
-
-| Scenario | Framework |
+| Need | Go to |
 |---|---|
-| Text generation on Apple Intelligence devices (iOS 26+) | Foundation Models |
-| Structured output from on-device LLM | Foundation Models (`@Generable`) |
-| Image classification, object detection | Core ML |
-| Custom model from PyTorch/TensorFlow | Core ML + coremltools |
-| Running specific open-source LLMs | MLX Swift or llama.cpp |
-| Maximum throughput on Apple Silicon | MLX Swift |
-| Cross-platform LLM inference | llama.cpp |
-| OCR and text recognition | Vision framework |
-| Sentiment analysis, NER, tokenization | Natural Language framework |
-| Training custom classifiers on device | Create ML |
+| Generate text where Apple Intelligence is on (iOS 26+) | Foundation Models |
+| Typed LLM output | Foundation Models, `@Generable` |
+| Image classification or object detection | Core ML |
+| Bring a PyTorch or TensorFlow model | Core ML, converted with coremltools |
+| Top speed on Apple Silicon | MLX Swift |
+| A named open-weight LLM | MLX Swift or llama.cpp |
+| One LLM engine on every platform | llama.cpp |
+| Read text in images (OCR) | Vision |
+| Sentiment, named entities, tokenization | Natural Language |
+| Train a custom classifier on device | Create ML |
 
-## Apple Foundation Models Overview
+## Foundation Models
 
-On-device language model optimized for Apple Silicon. Available on devices
-supporting Apple Intelligence (iOS 26+, macOS 26+).
+Facts that shape every design:
 
-- Token budget covers input + output; check `contextSize` for the limit
-- Resolve locale before generation by checking `supportsLocale(_:)` against
-  `Locale.current` and preferred fallbacks; do not raw-match `supportedLanguages`
-- Guardrails always enforced, cannot be disabled
+- Prompt and reply draw on one token budget. `contextSize` gives its size.
+- Choose the locale before generating: try `Locale.current`, then the user's
+  other preferred languages, with `supportsLocale(_:)`. Do not match against
+  `supportedLanguages` yourself.
+- Guardrails are always on and cannot be disabled.
 
-### Availability Checking (Required)
+### Check availability first
 
-Always check before using. Never crash on unavailability.
+Every entry point checks availability and falls back without crashing.
 
 ```swift
+import Foundation
 import FoundationModels
-
-switch SystemLanguageModel.default.availability {
-case .available:
-    guard SystemLanguageModel.default.supportsLocale(Locale.current) else {
-        // Use locale fallback before generating
-        break
+import os
+
+private let aiLog = Logger(subsystem: "StudyApp", category: "AI")
+
+enum SummaryState { case ready, openSettings, downloading, unsupported }
+
+func summaryState() -> SummaryState {
+    let model = SystemLanguageModel.default
+    switch model.availability {
+    case .available:
+        return model.supportsLocale(Locale.current) ? .ready : .unsupported
+    case .unavailable(.appleIntelligenceNotEnabled):
+        return .openSettings          // point the user to Settings
+    case .unavailable(.modelNotReady):
+        return .downloading           // assets still arriving; show progress
+    case .unavailable(.deviceNotEligible):
+        return .unsupported           // hardware cannot run it; use a fallback
+    case .unavailable(let other):
+        aiLog.notice("Model unavailable: \(String(describing: other))")
+        return .unsupported
     }
-    // Proceed with model usage
-case .unavailable(.appleIntelligenceNotEnabled):
-    // Guide user to enable Apple Intelligence in Settings
-case .unavailable(.modelNotReady):
-    // System model assets are not ready; show loading state
-case .unavailable(.deviceNotEligible):
-    // Device cannot run Apple Intelligence; use fallback
-case .unavailable(let reason):
-    // Unknown or future unavailable reason; use fallback and log reason
 }
 ```
 
-### Session Management
+### Sessions
 
 ```swift
-// Basic session
-let session = LanguageModelSession()
+let plain = LanguageModelSession()
 
-// Session with instructions
-let session = LanguageModelSession {
-    "You are a helpful cooking assistant."
+let tutor = LanguageModelSession {
+    "You write short study notes for high-school biology."
 }
 
-// Session with tools
-let session = LanguageModelSession(
-    tools: [weatherTool, recipeTool]
-) {
-    "You are a helpful assistant with access to tools."
+let withTools = LanguageModelSession(tools: [GlossaryTool()]) {
+    "Explain terms using the glossary when a term is unfamiliar."
 }
+
+let resumed = LanguageModelSession(model: .default, tools: [], transcript: savedTranscript)
 ```
 
-Key rules:
-- Sessions are stateful -- multi-turn conversations maintain context automatically
-- One request at a time per session (check `session.isResponding`)
-- Call `session.prewarm()` before user interaction for faster first response
-- Save/restore transcripts: `LanguageModelSession(model: model, tools: [], transcript: savedTranscript)`
+- A session remembers the conversation; each turn sees the ones before it.
+- A session serves a single request at once. `isResponding` is true while one
+  is in flight.
+- Prewarm with `session.prewarm()` ahead of the user's first question so the
+  first reply arrives sooner.
+- Restore a conversation by passing a saved `Transcript`.
 
-### Structured Output with `@Generable`
+### Structured output
 
-The `@Generable` macro creates compile-time schemas for type-safe output:
+`@Generable` builds the output schema at compile time, and the response comes
+back as your type.
 
 ```swift
 @Generable
-struct Recipe {
-    @Guide(description: "The recipe name")
-    var name: String
-
-    @Guide(description: "Cooking steps", .count(3))
-    var steps: [String]
-
-    @Guide(description: "Prep time in minutes", .range(1...120))
-    var prepTime: Int
+struct StudyCard {
+    @Guide(description: "The term being studied, two to four words")
+    var term: String
+    @Guide(description: "Plain-language definitions", .count(2))
+    var definitions: [String]
+    @Guide(.range(1...5))
+    var difficulty: Int
 }
 
-let response = try await session.respond(
-    to: "Suggest a quick pasta recipe",
-    generating: Recipe.self
-)
-print(response.content.name)
+let reply = try await tutor.respond(to: "Make a card for osmosis",
+                                    generating: StudyCard.self)
+let card = reply.content
 ```
 
-#### `@Guide` Constraints
+`@Guide` options:
 
-| Constraint | Purpose |
+| Guide | Effect |
 |---|---|
-| `description:` | Natural language hint for generation |
-| `.anyOf([values])` | Restrict to enumerated string values |
-| `.count(n)` | Fixed array length |
-| `.range(min...max)` | Numeric range |
-| `.minimum(n)` / `.maximum(n)` | One-sided numeric bound |
-| `.minimumCount(n)` / `.maximumCount(n)` | Array length bounds |
-| `.constant(value)` | Always returns this value |
-| `.pattern(regex)` | String format enforcement |
-| `.element(guide)` | Guide applied to each array element |
-
-Properties generate in declaration order. Place foundational data before
-dependent data for better results.
-
-### Streaming Structured Output
+| `description:` | Natural-language hint for the field |
+| `.anyOf([...])` | String must be one of the listed values |
+| `.count(n)` | Array has exactly `n` elements |
+| `.range(a...b)` | Number within a closed range |
+| `.minimum(n)` / `.maximum(n)` | Lower or upper limit on a number |
+| `.minimumCount(n)` / `.maximumCount(n)` | Limits on how many elements an array has |
+| `.constant(value)` | Field is always exactly this |
+| `.pattern(regex)` | String matches a pattern |
+| `.element(guide)` | Applies a guide to every array element |
+
+Fields are generated in declaration order. Put the fields that others depend on
+first.
+
+Streaming gives partial values as they form; each property of
+`StudyCard.PartiallyGenerated` is optional:
 
 ```swift
-let stream = session.streamResponse(
-    to: "Suggest a recipe",
-    generating: Recipe.self
-)
-for try await snapshot in stream {
-    // snapshot.content is Recipe.PartiallyGenerated (all properties optional)
-    if let name = snapshot.content.name { updateNameLabel(name) }
+let cardStream = tutor.streamResponse(to: "Card for diffusion", generating: StudyCard.self)
+for try await partial in cardStream {
+    draft.term = partial.content.term ?? draft.term
+    draft.definitions = partial.content.definitions ?? draft.definitions
 }
 ```
 
-### Tool Calling
+### Tools
 
 ```swift
-struct WeatherTool: Tool {
-    let name = "weather"
-    let description = "Get current weather for a city."
+struct GlossaryTool: Tool {
+    let name = "lookUpTerm"
+    let description = "Returns the course glossary entry for a biology term."
 
     @Generable
     struct Arguments {
-        @Guide(description: "The city name")
-        var city: String
+        @Guide(description: "Singular form of the term to look up")
+        var term: String
+        @Guide(.range(1...3))
+        var gradeLevel: Int
     }
 
     func call(arguments: Arguments) async throws -> String {
-        let weather = try await fetchWeather(arguments.city)
-        return weather.description
+        await Glossary.shared.entry(for: arguments.term, level: arguments.gradeLevel)
+            ?? "No entry found."
     }
 }
 ```
 
-Register only necessary tools at session creation. `Tool` is `Sendable`; tool
-descriptors and `@Generable` schemas consume the shared context window. The
-model chooses when to call tools, so prefetch deterministic required data into
-the prompt and reserve autonomous tools for dynamic lookups.
+- Register tools when creating the session, and only the ones the task needs.
+- `Tool` is `Sendable`; captured state must be safe to share.
+- Tool names, descriptions and argument schemas, like `@Generable` schemas,
+  use up part of the context budget.
+- Whether a tool runs is the model's decision. Data the answer always needs
+  should be fetched beforehand and put in the prompt; keep tools for lookups
+  that depend on the conversation.
 
-### Error Handling
+### Errors
 
 ```swift
 do {
-    let response = try await session.respond(to: prompt)
-} catch let error as LanguageModelSession.GenerationError {
-    switch error {
-    case .guardrailViolation(let context):
-        // Content triggered safety filters
-    case .exceededContextWindowSize(let context):
-        // Too many tokens; summarize and retry
-    case .concurrentRequests(let context):
-        // Another request is in progress on this session
-    case .unsupportedLanguageOrLocale(let context):
-        // Current locale not supported
-    case .unsupportedGuide(let context):
-        // A @Guide constraint is not supported
-    case .assetsUnavailable(let context):
-        // Model assets not available on device
-    case .refusal(let refusal, _):
-        // Model refused; stream refusal.explanation for details
-    case .rateLimited(let context):
-        // Too many requests; back off and retry
-    case .decodingFailure(let context):
-        // Response could not be decoded into the expected type
-    default: break
+    let answer = try await tutor.respond(to: question)
+    show(answer.content)
+} catch let failure as LanguageModelSession.GenerationError {
+    switch failure {
+    case .exceededContextWindowSize:   restartWithSummary()         // too many tokens
+    case .guardrailViolation:          showRephraseHint()           // safety filter hit
+    case .concurrentRequests:          queueUntilIdle()             // session already busy
+    case .unsupportedLanguageOrLocale: showLanguageNotice()
+    case .unsupportedGuide:            logSchemaProblem()           // a @Guide the model cannot honour
+    case .assetsUnavailable:           showDownloadingState()
+    case .refusal(let refusal, _):     explain(refusal)             // stream refusal.explanation
+    case .rateLimited:                 retryLater()                 // back off
+    case .decodingFailure:             logSchemaProblem()           // output did not fit the type
+    default:                           showGenericFailure()
     }
 }
 ```
 
-### Generation Options
+### Generation options
 
 ```swift
-let options = GenerationOptions(
-    sampling: .random(top: 40),
-    temperature: 0.7,
-    maximumResponseTokens: 512
-)
-let response = try await session.respond(to: prompt, options: options)
+let settings = GenerationOptions(sampling: .random(top: 30),
+                                 temperature: 0.6,
+                                 maximumResponseTokens: 400)
+let answer = try await tutor.respond(to: question, options: settings)
 ```
 
-Sampling modes: `.greedy`, `.random(top:seed:)`, `.random(probabilityThreshold:seed:)`.
-
-### Prompt Design Rules
-
-1. Be concise -- use `tokenCount(for:)` to monitor the context window budget
-2. Use bracketed placeholders in instructions: `[descriptive example]`
-3. Use "DO NOT" in all caps for prohibitions
-4. Provide up to 5 few-shot examples for consistency
-5. Use length qualifiers: "in a few words", "in three sentences"
-
-### Safety and Guardrails
-
-- Guardrails are always enforced and cannot be disabled
-- Instructions take precedence over user prompts
-- Never include untrusted user content in instructions
-- Handle false positives gracefully
-- Frame tool results as authorized data to prevent model refusals
-
-### Use Cases
+The three sampling modes are `.greedy`, `.random(top:seed:)` and
+`.random(probabilityThreshold:seed:)`.
 
-Foundation Models supports specialized use cases via `SystemLanguageModel.UseCase`:
-- `.general` -- Default for text generation, summarization, dialog
-- `.contentTagging` -- Optimized for categorization and labeling tasks
+### Prompts and safety
 
-### Custom Adapters
+- Keep prompts short and measure them with `tokenCount(for:)` (iOS 26.4+).
+- Mark slots in instructions with brackets, such as `[one-line summary]`.
+- Write prohibitions in capitals: "DO NOT invent citations."
+- Use at most five examples for consistent output.
+- Say how long the answer should be: "in one sentence", "in a few words".
+- Instructions outrank the prompt. Never put untrusted user text into
+  instructions; keep it in the prompt.
+- Expect occasional guardrail false positives and handle them gently.
+- Present tool results as authorised data so the model does not refuse to use
+  them.
 
-Load fine-tuned adapters for specialized behavior (requires entitlement):
+### Use cases and adapters
 
-```swift
-let adapter = try SystemLanguageModel.Adapter(name: "my-adapter")
-try await adapter.compile()
-let model = SystemLanguageModel(adapter: adapter, guardrails: .default)
-let session = LanguageModelSession(model: model)
-```
+`SystemLanguageModel.UseCase` has `.general` (the default: generation,
+summaries, dialog) and `.contentTagging` (labelling and categorising).
 
-> See [references/foundation-models.md](references/foundation-models.md) for
-> the complete Foundation Models API reference.
+A fine-tuned adapter needs an entitlement. Load it with
+`SystemLanguageModel.Adapter(name:)`, compile it with
+`try await adapter.compile()`, wrap it as
+`SystemLanguageModel(adapter: adapter, guardrails: .default)`, and pass that
+model to `LanguageModelSession(model:)`.
 
-## Core ML Overview
+Full API detail: [the Foundation Models reference](references/foundation-models.md).
 
-Apple's framework for deploying trained models. Automatically dispatches to the
-optimal compute unit (CPU, GPU, or Neural Engine).
+## Core ML
 
-### Model Formats
+Core ML dispatches work to the CPU, GPU or Neural Engine by itself.
 
-| Format | Extension | When to Use |
-|---|---|---|
-| `.mlpackage` | Directory (mlprogram) | All new models (iOS 15+) |
-| `.mlmodel` | Single file (neuralnetwork) | Legacy only (iOS 11-14) |
-| `.mlmodelc` | Compiled | Pre-compiled for faster loading |
-
-Always use mlprogram (`.mlpackage`) for new work.
+| Format | Use |
+|---|---|
+| `.mlpackage` (ML program) | Every new model; iOS 15+ |
+| `.mlmodel` (neural network) | Older format, the only choice for iOS 11 to 14; do not use for new work |
+| `.mlmodelc` | Compiled form that loads faster |
 
-### Conversion Pipeline (coremltools)
+New conversions always target the ML program format:
 
 ```python
+import torch
 import coremltools as ct
 
-# PyTorch conversion (torch.jit.trace)
-model.eval()  # CRITICAL: always call eval() before tracing
-traced = torch.jit.trace(model, example_input)
-mlmodel = ct.convert(
+net = build_segmenter()
+net.eval()                                   # never trace in training mode
+sample = torch.rand(1, 3, 256, 256)
+traced = torch.jit.trace(net, sample)
+
+segmenter = ct.convert(
     traced,
-    inputs=[ct.TensorType(shape=(1, 3, 224, 224), name="image")],
+    convert_to="mlprogram",
     minimum_deployment_target=ct.target.iOS18,
-    convert_to='mlprogram',
+    inputs=[ct.TensorType(name="frame", shape=sample.shape)],
 )
-mlmodel.save("Model.mlpackage")
+segmenter.save("Segmenter.mlpackage")
 ```
 
-### Optimization Techniques
+Compression at a glance:
 
-| Technique | Size Reduction | Accuracy Impact | Best Compute Unit |
+| Technique | Size cut | Accuracy risk | Best on |
 |---|---|---|---|
-| INT8 per-channel | ~4x | Low | CPU/GPU |
-| INT4 per-block | ~8x | Medium | GPU |
-| Palettization 4-bit | ~8x | Low-Medium | Neural Engine |
-| W8A8 (weights+activations) | ~4x | Low | ANE (A17 Pro/M4+) |
-| Pruning 75% | ~4x | Medium | CPU/ANE |
-
-### Boundary with `coreml`
-
-This skill owns Python-side conversion, compression, profiling, and framework
-selection. Use the sibling `coreml` skill for Swift app integration, prediction
-APIs, runtime configuration, Vision request wiring, and detailed model loading.
+| 4-bit palettization | about 8x | low to medium | Neural Engine |
+| INT8, per channel | about 4x | low | CPU and GPU |
+| INT4, per block | about 8x | medium | GPU |
+| W8A8 (weights and activations) | about 4x | low | Neural Engine on A17 Pro and M4 or later |
+| 75% pruning | about 4x | medium | CPU, Neural Engine |
 
-> See [references/coreml-conversion.md](references/coreml-conversion.md) for the
-> full conversion pipeline and [references/coreml-optimization.md](references/coreml-optimization.md)
-> for optimization techniques.
+Boundary: this skill decides which framework to use and owns the Python side
+(conversion, compression, profiling). The `coreml` skill owns the Swift side:
+loading models in the app, runtime settings, calling predictions, `MLTensor`
+pre- and post-processing, and hooking models into Vision requests. The
+[optimization guide](references/coreml-optimization.md#mltensor) keeps a short
+`MLTensor` example.
 
-## MLX Swift Overview
+Conversion: [the conversion guide](references/coreml-conversion.md).
+Compression and profiling: [the optimization guide](references/coreml-optimization.md).
 
-Apple's ML framework for Swift. Highest sustained generation throughput on
-Apple Silicon via unified memory architecture.
+## MLX Swift
 
-### Loading and Running LLMs
+MLX Swift gets the highest sustained throughput on Apple Silicon because CPU
+and GPU share unified memory.
 
 ```swift
-import MLX
-import MLXLLM
 import MLXLMCommon
+import MLXLLM
 import MLXLMHFAPI
+import MLX
 
-let container = try await LLMModelFactory.shared.loadContainer(
+let factory = LLMModelFactory.shared
+let modelContainer = try await factory.loadContainer(
     from: HubClient.default,
     using: TokenizersLoader(),
-    configuration: .init(id: "mlx-community/Qwen3-4B-4bit")
+    configuration: ModelConfiguration(id: "mlx-community/Qwen3-1.7B-4bit")
 )
-let session = ChatSession(container)
-print(try await session.respond(to: "Hello"))
+let chat = ChatSession(modelContainer)
+let text = try await chat.respond(to: "List three uses of chlorophyll.")
 ```
 
-### Model Selection by Device
+Starting points by device:
 
-| Device | RAM | Recommended Model | RAM Usage |
-|---|---|---|---|
-| iPhone 12-14 | 4-6 GB | SmolLM2-135M or Qwen 2.5 0.5B | ~0.3 GB |
-| iPhone 15 Pro+ | 8 GB | Gemma 3n E4B 4-bit | ~3.5 GB |
-| Mac 8 GB | 8 GB | Llama 3.2 3B 4-bit | ~3 GB |
-| Mac 16 GB+ | 16 GB+ | Mistral 7B 4-bit | ~6 GB |
+| Device and RAM | Resident memory | Suggested model |
+|---|---|---|
+| iPhone 12 to 14, 4 to 6 GB | about 0.3 GB | SmolLM2-135M, or Qwen 2.5 at 0.5B |
+| iPhone 15 Pro or newer, 8 GB | about 3.5 GB | Gemma 3n E4B at 4 bits |
+| Mac, 8 GB | about 3 GB | Llama 3.2 3B at 4 bits |
+| Mac, 16 GB or more | about 6 GB | Mistral 7B at 4 bits |
 
-### Memory Management
+Memory rules:
 
-1. Never exceed 60% of total RAM on iOS
-2. Set MLX cache limits: `Memory.cacheLimit = 512 * 1024 * 1024`
-3. Unload MLX and llama.cpp models on backgrounding or memory pressure; for MLX,
-   also call `Memory.clearCache()` after generation-heavy phases
-4. Use "Increased Memory Limit" entitlement for larger models
-5. Validate MLX Swift and llama.cpp on physical Apple Silicon; Simulator cannot
-   exercise Metal-dependent inference, memory, or performance
+- On iOS stay under 60% of total RAM.
+- Cap the MLX cache: `Memory.cacheLimit = 512 * 1024 * 1024`.
+- Release llama.cpp and MLX models on backgrounding or a memory warning, and
+  after heavy MLX generation also call `Memory.clearCache()`.
+- Larger models need the Increased Memory Limit entitlement.
+- Test MLX and llama.cpp code on real Apple Silicon hardware. The Simulator
+  does not run Metal inference and shows neither real memory limits nor real
+  speed.
 
-> See [references/mlx-swift.md](references/mlx-swift.md) for full MLX Swift
-> patterns and llama.cpp integration.
+MLX lifecycle, llama.cpp and GGUF: [the MLX and llama.cpp guide](references/mlx-swift.md).
 
-## Multi-Backend Architecture
+## Running several backends
 
-When an app needs multiple AI backends (e.g., Foundation Models + MLX fallback):
+Try the system model first, then a local open model, and fail clearly when
+neither is there:
 
 ```swift
-func respond(to prompt: String) async throws -> String {
-    if SystemLanguageModel.default.isAvailable {
-        return try await foundationModelsRespond(prompt)
-    } else if canLoadMLXModel() {
-        return try await mlxRespond(prompt)
-    } else {
-        throw AIError.noBackendAvailable
-    }
+func pickBackend() async throws -> Backend {
+    let systemModel = SystemLanguageModel.default
+    if systemModel.isAvailable { return .foundationModels }
+    if await mlxModelCanLoad() { return .mlx }
+    throw AIError.noBackendAvailable
 }
 ```
 
-Serialize all model access through a coordinator actor to prevent contention:
+Send every model call through one actor so requests never compete for the same
+hardware. Actors are reentrant: a method that awaits lets the next caller in,
+so an actor that only forwards the call does not serialise anything. Chain the
+work instead:
 
 ```swift
-actor ModelCoordinator {
-    func withExclusiveAccess<T>(_ work: () async throws -> T) async rethrows -> T {
-        try await work()
+actor InferenceGate {
+    private var last: Task<Void, Never>?
+
+    func exclusive<Value: Sendable>(
+        _ work: @escaping @Sendable () async throws -> Value
+    ) async throws -> Value {
+        let previous = last
+        let current = Task { () async throws -> Value in
+            await previous?.value
+            return try await work()
+        }
+        last = Task { _ = try? await current.value }
+        return try await current.value
     }
 }
 ```
 
-For custom Core ML models, name only the conversion/optimization handoff here:
-send Swift app integration, model loading, Vision wiring, and prediction
-lifecycle to `coreml`. Keep private user content, such as journals, on device
-unless product explicitly opts into a nonlocal fallback.
-
-## Performance Best Practices
-
-1. Run outside debugger for accurate benchmarks (Xcode: Cmd-Opt-R, uncheck
-   "Debug Executable")
-2. Call `session.prewarm()` for Foundation Models before user interaction
-3. Pre-compile Core ML models to `.mlmodelc` for faster loading
-4. Use EnumeratedShapes over RangeDim for Neural Engine optimization
-5. Use 4-bit palettization for best Neural Engine memory/latency gains
-6. Hand off detailed Vision, Natural Language, and Swift Core ML runtime
-   integration to the sibling framework skills
-
-## Common Mistakes
-
-1. **No availability check.** Starting generation without checking
-   `SystemLanguageModel.default.availability` leaves unsupported devices with
-   failures instead of fallback UI.
-2. **No fallback UI.** Users on pre-iOS 26 or devices without Apple Intelligence
-   see nothing. Always provide a graceful degradation path.
-3. **Exceeding the context window.** The token budget covers input + output.
-   Monitor usage via `tokenCount(for:)` and summarize when needed.
-4. **Concurrent requests on one session.** `LanguageModelSession` supports one
-   request at a time. Check `session.isResponding` or serialize access.
-5. **Untrusted content in instructions.** User input placed in the instructions
-   parameter bypasses guardrail boundaries. Keep user content in the prompt.
-6. **Forgetting `model.eval()` before Core ML tracing.** PyTorch models must be
-   in eval mode before `torch.jit.trace`. Training-mode artifacts corrupt output.
-7. **Using neuralnetwork format.** Always use `mlprogram` (.mlpackage) for new
-   Core ML models. The legacy neuralnetwork format is deprecated.
-8. **Exceeding 60% RAM on iOS (MLX Swift).** Large models cause OOM kills.
-9. **Trusting MLX simulator results.** Validate Metal-dependent behavior on
-   physical devices; Simulator is only a UI/control-flow smoke test.
-10. **Not clearing MLX caches.** Pair model unload with `Memory.clearCache()`.
-
-## Review Checklist
-
-- [ ] Framework selection matches use case and target OS version
-- [ ] Foundation Models: availability checked before every API call
-- [ ] Foundation Models: graceful fallback when model unavailable
-- [ ] Foundation Models: session prewarm called before user interaction
-- [ ] Foundation Models: `@Generable` properties in logical generation order
-- [ ] Foundation Models: token budget accounted for (check `contextSize`)
-- [ ] Core ML: model format is mlprogram (.mlpackage) for iOS 15+
-- [ ] Core ML: conversion, deployment target, and compression validated
-- [ ] MLX Swift: model size appropriate for target device RAM
-- [ ] MLX Swift: cache limits set, caches cleared, models unloaded
-- [ ] All model access serialized through coordinator actor
-- [ ] Concurrency: model types and tool implementations are `Sendable`-conformant or `@MainActor`-isolated
-- [ ] Physical device testing performed (not simulator)
-
-## References
-
-- [Foundation Models API](references/foundation-models.md) -- LanguageModelSession, `@Generable`, tool calling, prompt design
-- [Core ML Conversion](references/coreml-conversion.md) -- Model conversion from PyTorch, TensorFlow, other frameworks
-- [Core ML Optimization](references/coreml-optimization.md) -- Quantization, palettization, pruning, performance tuning
-- [MLX Swift & llama.cpp](references/mlx-swift.md) -- MLX Swift patterns, llama.cpp integration, memory management
+- For a custom Core ML model, this skill stops at conversion and
+  optimisation. Swift integration, loading, Vision hookup and running
+  predictions are for `coreml`.
+- Private material such as journals or health notes stays on the device unless
+  the product has deliberately chosen a non-local fallback.
+
+## Performance habits
+
+1. Benchmark without the debugger: in the scheme's Run action (Cmd-Opt-R),
+   clear "Debug executable".
+2. Prewarm Foundation Models sessions before the user needs them.
+3. Ship Core ML models precompiled to `.mlmodelc`.
+4. For the Neural Engine, prefer `EnumeratedShapes` to `RangeDim`.
+5. Palettize to 4 bits for the lowest Neural Engine memory and latency.
+6. Runtime code for Vision, Natural Language and Core ML in Swift belongs to
+   the skills for those frameworks.
+
+## Mistakes to catch
+
+| Mistake | Consequence | Fix |
+|---|---|---|
+| No availability check | Unsupported devices fail instead of degrading | Switch on `availability` first |
+| No fallback UI | Users on older systems or with Apple Intelligence off get a blank feature | Always offer a degraded path |
+| Overflowing the context | Input plus output exceed the window | Measure with `tokenCount(for:)` (iOS 26.4+), summarise |
+| Two requests on one session | `concurrentRequests` error | Check `isResponding` or serialise |
+| User text in instructions | Weakens the instruction boundary and guardrails | Keep user text in the prompt |
+| Tracing without `model.eval()` | Dropout and batch-norm training behaviour baked in | Call `eval()` before trace or export |
+| Converting to neural network format | Deprecated, missing features | Use ML program |
+| MLX using over 60% RAM on iOS | App killed for memory | Pick a smaller model, cap caches |
+| Trusting Simulator numbers for MLX | Misleading; it only proves UI and control flow | Measure on hardware |
+| Releasing an MLX model but not its cache | Memory stays high | Call `Memory.clearCache()` as part of unloading |
+
+## Before approving
+
+- [ ] The framework matches the use case and the deployment target.
+- [ ] Foundation Models: every call is preceded by an availability check, and
+      a degraded path exists.
+- [ ] Foundation Models: sessions prewarmed, `@Generable` fields in dependency
+      order, token use compared with `contextSize`.
+- [ ] Core ML: ML program `.mlpackage` (iOS 15+); the conversion, its
+      deployment target and any compression were verified.
+- [ ] MLX: the model fits in the device's memory, the cache has a limit, and
+      caches are cleared when models are released.
+- [ ] One coordinating actor mediates every model call.
+- [ ] Model types and tools are `Sendable` or isolated to `@MainActor`.
+- [ ] Tested on physical devices, not only the Simulator.
+
+## Reference files
+
+- [Foundation Models reference](references/foundation-models.md): the full API,
+  long conversations, feedback.
+- [Conversion guide](references/coreml-conversion.md): coremltools from each
+  source framework, input types, shapes, deployment targets, stateful and
+  multifunction models.
+- [Optimization guide](references/coreml-optimization.md): quantizing,
+  palettizing, pruning, QAT, Swift loading basics, profiling.
+- [MLX and llama.cpp guide](references/mlx-swift.md): MLX Swift lifecycle,
+  llama.cpp, GGUF levels, routing between backends, built-in frameworks.