@mmerterden/multi-agent-pipeline 20.7.0 → 20.8.0

package/pipeline/skills/shared/external/coreml/SKILL.md

@@ -1,500 +1,456 @@
 ---
 name: coreml
-description: "Integrate Core ML models in iOS apps for on-device machine learning inference. Covers model loading (.mlmodel, .mlpackage, .mlmodelc), predictions with auto-generated classes and MLFeatureProvider, compute unit configuration (CPU, GPU, Neural Engine), MLTensor, VNCoreMLRequest, MLComputePlan, multi-model pipelines, and deployment strategies. Use when loading Core ML models, making predictions, configuring compute units, or profiling model performance."
+description: "Swift-side Core ML: loading .mlmodel, .mlpackage and .mlmodelc, predictions via generated classes and MLFeatureProvider, compute units, async and stateful prediction with MLState, MLTensor, MLMultiArray, Vision CoreMLRequest and VNCoreMLRequest, MLComputePlan, multi-model pipelines, deployment and profiling. Use when integrating, reviewing, deploying or profiling a Core ML model in an app. Not for Python conversion (apple-on-device-ai)."
 metadata:
-  source: "dpearson2699/swift-ios-skills (PolyForm Perimeter 1.0.0)"
+  source: multi-agent-pipeline
 ---
-# Core ML Swift Integration
 
-Load, configure, and run Core ML models in iOS apps. This skill covers the
-Swift side: model loading, prediction, MLTensor, profiling, and deployment.
-Target iOS 26+ with Swift 6.3, backward-compatible to iOS 14 unless noted.
+# Core ML
 
-> **Scope boundary:** Python-side model conversion, optimization (quantization,
-> palettization, pruning), and framework selection live in the `apple-on-device-ai`
-> skill. This skill owns Swift integration only.
+This skill covers the app side of Core ML: getting a model into memory,
+feeding it, reading its output, and shipping it. Baseline: iOS 26 and Swift
+6.3, with each API's lower floor stated where it matters (much of Core ML goes
+back to iOS 14 or earlier).
 
-See [references/coreml-swift-integration.md](references/coreml-swift-integration.md) for complete code patterns including
-actor-based caching, batch inference, image preprocessing, and testing.
+Out of scope here: converting or compressing models in Python (quantization,
+palettization, pruning) and choosing between on-device AI frameworks; those
+belong to `apple-on-device-ai`. Text recognition, barcodes and document
+scanning with Vision belong to `vision-framework`.
 
-## Contents
+Longer code (an actor that owns models, SwiftUI wiring, custom feature
+providers, stateful sessions, pixel buffers, tensors, Vision pipelines,
+NaturalLanguage, compute plans, lifecycle, errors, tests) is in
+`references/coreml-swift-integration.md`.
 
-- [Loading Models](#loading-models)
-- [Model Configuration](#model-configuration)
-- [Making Predictions](#making-predictions)
-- [MLTensor (iOS 18+)](#mltensor-ios-18)
-- [Working with MLMultiArray](#working-with-mlmultiarray)
-- [Image Preprocessing](#image-preprocessing)
-- [Multi-Model Pipelines](#multi-model-pipelines)
-- [Vision Integration](#vision-integration)
-- [Performance Profiling](#performance-profiling)
-- [Model Deployment](#model-deployment)
-- [Memory Management](#memory-management)
-- [Common Mistakes](#common-mistakes)
-- [Review Checklist](#review-checklist)
-- [References](#references)
+## Loading a model
 
-## Loading Models
-
-### Auto-Generated Classes
-
-When you add a `.mlmodel` or `.mlpackage` to an app target, Xcode generates a Swift
-class with typed input/output. Use this whenever possible.
+**Generated class.** Drop a `.mlmodel` or `.mlpackage` into a target and Xcode
+generates a Swift class with typed inputs and outputs. Use it whenever it
+exists.
 
 ```swift
 import CoreML
 
-let config = MLModelConfiguration()
-config.computeUnits = .all
-
-let model = try MyImageClassifier(configuration: config)
+func makeDenoiser() throws -> PhotoDenoiser {
+    let settings = MLModelConfiguration()
+    settings.computeUnits = .all
+    return try PhotoDenoiser(configuration: settings)
+}
 ```
 
-### Manual Loading
-
-Load from a URL when the model is downloaded at runtime or stored outside the
-bundle.
+**From a URL.** For models that are downloaded or stored outside the bundle:
 
 ```swift
-let modelURL = Bundle.main.url(
-    forResource: "MyModel", withExtension: "mlmodelc"
-)!
-let model = try MLModel(contentsOf: modelURL, configuration: config)
-```
-
-### Async Loading (iOS 15+)
-
-Load models without blocking the main thread. Prefer this for large models.
+import CoreML
 
-```swift
-let model = try await MLModel.load(
-    contentsOf: modelURL,
-    configuration: config
-)
+func bundledModel(named name: String) throws -> MLModel {
+    guard let url = Bundle.main.url(forResource: name, withExtension: "mlmodelc") else {
+        throw CocoaError(.fileNoSuchFile)
+    }
+    return try MLModel(contentsOf: url, configuration: MLModelConfiguration())
+}
 ```
 
-### Compile at Runtime (iOS 16+)
-
-Compile a `.mlpackage` or `.mlmodel` to `.mlmodelc` on device. Useful for
-models downloaded from a server. Do this once per model version, not on every
-launch.
+**Asynchronously.** `MLModel.load(contentsOf:configuration:)` (async, iOS 15+)
+keeps large models from stalling the main thread and is the preferred loader.
 
-```swift
-let compiledURL = try await MLModel.compileModel(at: packageURL)
-let model = try await MLModel.load(contentsOf: compiledURL, configuration: config)
-```
+**Compiling on device.** `MLModel.compileModel(at:)` (async form, iOS 16+)
+turns a `.mlmodel` or `.mlpackage` into a `.mlmodelc` on the device, which is
+how server-delivered models become loadable. Two rules travel together:
 
-Cache the compiled URL -- recompiling on every launch is a bug. Copy
-`compiledURL` to a persistent location (e.g., Application Support). When
-reviewing runtime-loaded models, call out both facts together: async
-`MLModel.compileModel(at:)` is iOS 16+, and compiled models must be cached so the
-app does not recompile on every launch.
+1. Compile once per model version, never on every launch; recompiling at each
+   start is a bug.
+2. The returned URL points into a temporary location. Move the `.mlmodelc`
+   somewhere durable such as Application Support and reuse it.
 
-## Model Configuration
+When reviewing runtime-loaded models, state both: the async compile API needs
+iOS 16, and its output must be cached.
 
-`MLModelConfiguration` controls compute units, GPU access, and model parameters.
+## Configuration and compute units
 
-### Compute Units Decision Table
+`MLModelConfiguration` chooses hardware, GPU precision and model parameters.
 
-| Value | Uses | When to Choose |
+| `computeUnits` | Hardware | When |
 |---|---|---|
-| `.all` | CPU + GPU + Neural Engine | Default. Let the system decide. |
-| `.cpuOnly` | CPU | Deterministic tests, CPU-only fallbacks, or constrained work after profiling shows accelerator policy, contention, thermal state, or energy budget is the limiting factor. |
-| `.cpuAndGPU` | CPU + GPU | Need GPU but model has ops unsupported by ANE. |
-| `.cpuAndNeuralEngine` (iOS 16+) | CPU + Neural Engine | Best energy efficiency for compatible models. |
+| `.all` (default) | CPU, GPU, Neural Engine; the system decides | Almost always |
+| `.cpuOnly` | CPU | Deterministic tests, CPU-only fallbacks, or when profiling shows accelerator policy, contention, thermal state or energy budget is the constraint |
+| `.cpuAndGPU` | CPU and GPU | The model needs the GPU but has operations the Neural Engine cannot run |
+| `.cpuAndNeuralEngine` (iOS 16+) | CPU and Neural Engine | Best energy efficiency for models the Neural Engine fully supports |
 
 ```swift
-let config = MLModelConfiguration()
-config.computeUnits = .cpuAndNeuralEngine
-
-// Optional fallback for constrained work after profiling and policy review
-config.computeUnits = .cpuOnly
-```
-
-### Configuration Properties
+import CoreML
 
-```swift
-let config = MLModelConfiguration()
-config.computeUnits = .all
-config.allowLowPrecisionAccumulationOnGPU = true // faster, slight precision loss
+func efficientSettings(fallbackToCPU: Bool) -> MLModelConfiguration {
+    let settings = MLModelConfiguration()
+    settings.computeUnits = fallbackToCPU ? .cpuOnly : .cpuAndNeuralEngine
+    settings.allowLowPrecisionAccumulationOnGPU = true
+    return settings
+}
 ```
 
-## Making Predictions
+`allowLowPrecisionAccumulationOnGPU` buys speed for a small loss of precision.
+Choose `.cpuOnly` only after profiling says so.
 
-### With Auto-Generated Classes
+## Predictions
 
-The generated class provides typed input/output structs.
+**Generated class:**
 
 ```swift
-let model = try MyImageClassifier(configuration: config)
-let input = MyImageClassifierInput(image: pixelBuffer)
-let output = try model.prediction(input: input)
-print(output.classLabel)        // "golden_retriever"
-print(output.classLabelProbs)   // ["golden_retriever": 0.95, ...]
+import CoreML
+
+func tagScene(_ frame: CVPixelBuffer, with model: SceneTagger) throws -> (String, Double) {
+    let result = try model.prediction(input: SceneTaggerInput(image: frame))
+    return (result.classLabel, result.classLabelProbs[result.classLabel] ?? 0)
+}
 ```
 
-### With MLDictionaryFeatureProvider
+`classLabel` is the winning label and `classLabelProbs` maps every label to
+its probability.
 
-Use when inputs are dynamic or not known at compile time.
+**Dictionary provider**, for inputs only known at run time:
 
 ```swift
-let inputFeatures = try MLDictionaryFeatureProvider(dictionary: [
-    "image": MLFeatureValue(pixelBuffer: pixelBuffer),
-    "confidence_threshold": MLFeatureValue(double: 0.5),
-])
-let output = try model.prediction(from: inputFeatures)
-let label = output.featureValue(for: "classLabel")?.stringValue
-```
+import CoreML
 
-### Prediction Inside Async Workflows
+func tagWithThreshold(_ frame: CVPixelBuffer, cutoff: Double, model: MLModel) async throws -> String? {
+    let inputs = try MLDictionaryFeatureProvider(dictionary: [
+        "image": MLFeatureValue(pixelBuffer: frame),
+        "threshold": MLFeatureValue(double: cutoff)
+    ])
+    let outputs = try await model.prediction(from: inputs)
+    return outputs.featureValue(for: "label")?.stringValue
+}
+```
 
-`MLModel.prediction(...)` is synchronous. In async pipelines, keep model loading
-async, then run prediction from an actor or non-main task without adding `await`
-to the prediction call.
+### Sync or async prediction
 
-```swift
-let output = try model.prediction(from: inputFeatures)
-```
+- `prediction(from:)` and `prediction(from:options:)` exist in a synchronous
+  throwing form on every supported OS.
+- From iOS 17 there is also `prediction(from:options:) async throws`, which
+  may be called concurrently on the same model. Inside an `async` function the
+  compiler resolves to that overload, so the call needs `try await`; leaving
+  out `await` there does not compile.
+- In synchronous code (a serial queue, a synchronous actor method) the
+  synchronous form is used without `await`.
+- Loading stays async either way (`MLModel.load`).
 
-### Batch Prediction
+### Batches
 
-Process multiple inputs in one call for better throughput.
+Processing several inputs per call raises throughput:
 
 ```swift
-let batchInputs = try MLArrayBatchProvider(array: inputs.map { input in
-    try MLDictionaryFeatureProvider(dictionary: ["image": MLFeatureValue(pixelBuffer: input)])
-})
-let batchOutput = try model.predictions(fromBatch: batchInputs)
-for i in 0..<batchOutput.count {
-    let result = batchOutput.features(at: i)
-    print(result.featureValue(for: "classLabel")?.stringValue ?? "unknown")
+import CoreML
+
+func labels(for frames: [CVPixelBuffer], model: MLModel) throws -> [String] {
+    let rows = try frames.map { try MLDictionaryFeatureProvider(dictionary: ["image": MLFeatureValue(pixelBuffer: $0)]) }
+    let answers = try model.predictions(fromBatch: MLArrayBatchProvider(array: rows))
+    return (0..<answers.count).compactMap { answers.features(at: $0).featureValue(for: "label")?.stringValue }
 }
 ```
 
-Use `predictions(fromBatch:)` when batching without explicit
-`MLPredictionOptions`. Use `predictions(from:options:)` only when passing both an
-`MLBatchProvider` and `MLPredictionOptions`; `predictions(from:)` by itself is
-not the no-options batch API.
+Two batch spellings are valid: `predictions(fromBatch:)` without options, and
+`predictions(from:options:)` when you pass an `MLBatchProvider` together with
+`MLPredictionOptions`. There is no options-free `predictions(from:)`; do not
+write one.
 
-### Stateful Prediction (iOS 18+)
+### Stateful models with MLState (iOS 18+)
 
-Use `MLState` for models that maintain state across predictions (sequence models,
-LLMs, audio accumulators). Create state once and pass it to each prediction call.
+Sequence models (language models, streaming audio, time series) keep context
+between calls. Create a state once per stream and pass it to every call.
 
 ```swift
-let state = model.makeState()
+import CoreML
 
-// Each synchronous prediction carries forward the internal model state
-for frame in audioFrames {
-    let input = try MLDictionaryFeatureProvider(dictionary: [
-        "audio_features": MLFeatureValue(multiArray: frame)
-    ])
-    let output = try model.prediction(from: input, using: state)
-    let classification = output.featureValue(for: "label")?.stringValue
+func classifyStream(_ windows: [MLMultiArray], model: MLModel) throws -> [String] {
+    let memory = model.makeState()
+    return try windows.compactMap { window in
+        let input = try MLDictionaryFeatureProvider(dictionary: ["samples": MLFeatureValue(multiArray: window)])
+        return try model.prediction(from: input, using: memory).featureValue(for: "label")?.stringValue
+    }
 }
 ```
 
-`MLState` is `Sendable`, but `Sendable` does not make one state safe for
-concurrent inference. Predictions using the same state must be serialized; do
-not read or write state buffers while a prediction is in flight. Call
-`model.makeState()` for each independent concurrent stream. If you need
-`MLPredictionOptions`, iOS 18+ also provides the async
-`prediction(from:using:options:)` overload; the same one-in-flight-per-state rule
-still applies.
+Rules for `MLState`:
+
+- It is `Sendable`, which lets it move between concurrency domains. It does
+  not make concurrent predictions on one state safe: run them one at a time.
+- Do not read or write the state's buffers while a prediction is running.
+- Independent concurrent streams each get their own `model.makeState()`.
+- The async `prediction(from:using:options:)` (iOS 18) is there when you need
+  options or are in async code; the same one-prediction-in-flight rule holds.
+  The synchronous `prediction(from:using:)` is for synchronous loops.
 
 ## MLTensor (iOS 18+)
 
-`MLTensor` is a Swift-native multidimensional array for pre/post-processing.
-Operations run lazily -- call `await tensor.shapedArray(of:)` to materialize results.
+`MLTensor` is a Swift multidimensional array for pre- and post-processing.
+Operations are lazy; results materialize only when you await
+`shapedArray(of:)`.
 
 ```swift
 import CoreML
 
-// Creation
-let tensor = MLTensor([1.0, 2.0, 3.0, 4.0])
-let zeros = MLTensor(zeros: [3, 224, 224], scalarType: Float.self)
-
-// Reshaping
-let reshaped = tensor.reshaped(to: [2, 2])
-
-// Math operations
-let softmaxed = tensor.softmax(alongAxis: -1)
-let centered = tensor - tensor.mean()
+func normalizedScores(_ logits: [Float]) async -> MLMultiArray {
+    let raw = MLTensor(logits).reshaped(to: [1, logits.count])
+    let centered = raw - raw.mean()
+    let probabilities = centered.softmax(alongAxis: -1)
+    let shaped = await probabilities.shapedArray(of: Float.self)
+    return MLMultiArray(shaped)
+}
 
-// Interop with MLShapedArray / MLMultiArray
-let shaped = await tensor.shapedArray(of: Float.self)
-let multiArray = try MLMultiArray(shaped)
-let shapedAgain = MLShapedArray<Float>(multiArray)
+func tensor(from array: MLMultiArray) -> MLTensor {
+    MLTensor(MLShapedArray<Float>(array))
+}
 ```
 
-Do not invent `MLTensor` APIs for statistics or bridging. Avoid examples such as
-`MLTensor(multiArray)`, `tensor.std()`, `tensor.standardDeviation()`, direct
-lazy-buffer access, or synchronous extraction; perform unsupported DSP/statistics
-outside the tensor pipeline or with source-confirmed tensor operations.
+Stay with documented operations. There is no `MLTensor(multiArray)`
+initializer, no `std()` or `standardDeviation()`, no direct access to the lazy
+storage, and no synchronous extraction. Go through `MLShapedArray` for
+conversions, and build statistics from documented pieces (`mean()`,
+`squareRoot()`, arithmetic) or do them outside the tensor pipeline.
+`MLTensor(zeros:scalarType:)` and `MLTensor(ones:scalarType:)` create filled
+tensors.
 
-## Working with MLMultiArray
+## MLMultiArray
 
-`MLMultiArray` is the primary data exchange type for non-image model inputs and
-outputs. Use it when the auto-generated class expects array-type features.
+`MLMultiArray` is the exchange type for non-image inputs and outputs; generated
+classes use it for array features.
 
 ```swift
-// Create a 3D array: [batch, sequence, features]
-let array = try MLMultiArray(shape: [1, 128, 768], dataType: .float32)
+import CoreML
 
-// Write values
-for i in 0..<128 {
-    array[[0, i, 0] as [NSNumber]] = NSNumber(value: Float(i))
+func embeddingInput(tokens: Int, width: Int) throws -> MLMultiArray {
+    let grid = try MLMultiArray(shape: [1, NSNumber(value: tokens), NSNumber(value: width)], dataType: .float32)
+    grid[[0, 0, 0] as [NSNumber]] = 0.25
+    return grid
 }
 
-// Read values
-let value = array[[0, 0, 0] as [NSNumber]].floatValue
+func firstValue(of grid: MLMultiArray) -> Float {
+    grid[[0, 0, 0] as [NSNumber]].floatValue
+}
 
-let data: [Float] = [1.0, 2.0, 3.0]
-let shaped = MLShapedArray(scalars: data, shape: [3])
-let fromShaped = try MLMultiArray(shaped)
+func packed(_ values: [Float]) -> MLMultiArray {
+    MLMultiArray(MLShapedArray<Float>(scalars: values, shape: [1, values.count]))
+}
 ```
 
-See [references/coreml-swift-integration.md](references/coreml-swift-integration.md) for advanced MLMultiArray patterns
-including NLP tokenization and audio feature extraction.
+The shape reads batch, sequence, features. Tokenized text and audio feature
+patterns are in the reference.
 
-## Image Preprocessing
+## Image input
 
-Image models expect `CVPixelBuffer` input. Use `CGImage` conversion for photos
-from the camera or photo library. Vision's `VNCoreMLRequest` handles this
-automatically; manual conversion is needed only for direct `MLModel` prediction.
+Image models take a `CVPixelBuffer`. When Vision drives the model
+(`CoreMLRequest` or `VNCoreMLRequest`), conversion happens automatically; the
+manual path is only for direct `MLModel` predictions.
 
 ```swift
+import CoreGraphics
 import CoreVideo
 
-func createPixelBuffer(from cgImage: CGImage, width: Int, height: Int) -> CVPixelBuffer? {
-    var pixelBuffer: CVPixelBuffer?
-    let attrs: [CFString: Any] = [
+func pixelBuffer(from picture: CGImage, side: Int) -> CVPixelBuffer? {
+    let options = [
         kCVPixelBufferCGImageCompatibilityKey: true,
-        kCVPixelBufferCGBitmapContextCompatibilityKey: true,
-    ]
-    CVPixelBufferCreate(kCFAllocatorDefault, width, height,
-                        kCVPixelFormatType_32ARGB, attrs as CFDictionary, &pixelBuffer)
-
-    guard let buffer = pixelBuffer else { return nil }
+        kCVPixelBufferCGBitmapContextCompatibilityKey: true
+    ] as CFDictionary
+    var buffer: CVPixelBuffer?
+    guard CVPixelBufferCreate(kCFAllocatorDefault, side, side, kCVPixelFormatType_32ARGB, options, &buffer) == kCVReturnSuccess,
+          let buffer else { return nil }
     CVPixelBufferLockBaseAddress(buffer, [])
+    defer { CVPixelBufferUnlockBaseAddress(buffer, []) }
     let context = CGContext(
-        data: CVPixelBufferGetBaseAddress(buffer),
-        width: width, height: height,
+        data: CVPixelBufferGetBaseAddress(buffer), width: side, height: side,
         bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(buffer),
-        space: CGColorSpaceCreateDeviceRGB(),
-        bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
+        space: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
     )
-    context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height))
-    CVPixelBufferUnlockBaseAddress(buffer, [])
+    context?.draw(picture, in: CGRect(x: 0, y: 0, width: side, height: side))
     return buffer
 }
 ```
 
-For additional preprocessing patterns (normalization, center-cropping), see
-[references/coreml-swift-integration.md](references/coreml-swift-integration.md).
+Normalization and center-crop helpers are in the reference.
 
-## Multi-Model Pipelines
+## Chaining models
 
-Chain models when preprocessing or postprocessing requires a separate model.
+When pre- or post-processing is its own model, call them in order:
 
 ```swift
-// Sequential inference: preprocessor -> main model -> postprocessor
-let preprocessed = try preprocessor.prediction(from: rawInput)
-let mainOutput = try mainModel.prediction(from: preprocessed)
-let finalOutput = try postprocessor.prediction(from: mainOutput)
-```
+import CoreML
 
-For Xcode-managed pipelines, use the pipeline model type in the `.mlpackage`.
-Each sub-model runs on its optimal compute unit.
+func runChain(_ input: MLFeatureProvider, stages: [MLModel]) async throws -> MLFeatureProvider {
+    var current = input
+    for stage in stages {
+        current = try await stage.prediction(from: current)
+    }
+    return current
+}
+```
 
-## Vision Integration
+For chains that Xcode manages, build a pipeline model inside the `.mlpackage`;
+each sub-model then runs on the compute unit that suits it.
 
-Use Vision to run Core ML image models with automatic image preprocessing
-(resizing, normalization, color space, orientation).
+## Through Vision
 
-### Modern: CoreMLRequest (iOS 18+)
+Vision takes care of resizing, cropping, normalization, color space and
+orientation.
 
 ```swift
-import Vision
 import CoreML
+import Vision
 
-let model = try MLModel(contentsOf: modelURL, configuration: config)
-let request = CoreMLRequest(model: .init(model))
-let results = try await request.perform(on: cgImage)
-
-if let classification = results.first as? ClassificationObservation {
-    print("\(classification.identifier): \(classification.confidence)")
+func dominantLabel(in photo: CGImage, model: MLModel) async throws -> (String, Float)? {
+    let request = CoreMLRequest(model: try CoreMLModelContainer(model: model))
+    let observations = try await request.perform(on: photo)
+    guard let top = observations.first as? ClassificationObservation else { return nil }
+    return (top.identifier, top.confidence)
 }
 ```
 
-### Legacy: VNCoreMLRequest
+`CoreMLRequest` needs iOS 18. The legacy route for detection models:
 
 ```swift
-let vnModel = try VNCoreMLModel(for: model)
-let request = VNCoreMLRequest(model: vnModel) { request, error in
-    guard let results = request.results as? [VNRecognizedObjectObservation] else { return }
-    for observation in results {
-        let label = observation.labels.first?.identifier ?? "unknown"
-        let confidence = observation.labels.first?.confidence ?? 0
-        let boundingBox = observation.boundingBox // normalized coordinates
-        print("\(label): \(confidence) at \(boundingBox)")
+import CoreML
+import Vision
+
+func legacyDetections(in frame: CVPixelBuffer, model: MLModel) throws -> [(String, Float, CGRect)] {
+    let request = VNCoreMLRequest(model: try VNCoreMLModel(for: model))
+    request.imageCropAndScaleOption = .scaleFill
+    try VNImageRequestHandler(cvPixelBuffer: frame).perform([request])
+    let objects = request.results as? [VNRecognizedObjectObservation] ?? []
+    return objects.compactMap { object in
+        guard let label = object.labels.first else { return nil }
+        return (label.identifier, label.confidence, object.boundingBox)
     }
 }
-request.imageCropAndScaleOption = .scaleFill
-
-let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
-try handler.perform([request])
 ```
 
-> For complete Vision framework patterns (text recognition, barcode detection,
-> document scanning), see the `vision-framework` skill.
-
-## Performance Profiling
+`boundingBox` is normalized. Everything else about Vision (OCR, barcodes,
+documents) is in `vision-framework`.
 
-### MLComputePlan (iOS 17.4+)
+## Profiling
 
-Inspect which compute device each operation will use before running predictions.
+`MLComputePlan` (iOS 17.4+) tells you, before any prediction, which device
+each operation prefers and how expensive it is:
 
 ```swift
-let computePlan = try await MLComputePlan.load(
-    contentsOf: modelURL, configuration: config
-)
-guard case let .program(program) = computePlan.modelStructure else { return }
-guard let mainFunction = program.functions["main"] else { return }
-
-for operation in mainFunction.block.operations {
-    let deviceUsage = computePlan.deviceUsage(for: operation)
-    let estimatedCost = computePlan.estimatedCost(of: operation)
-    print("\(operation.operatorName): \(String(describing: deviceUsage?.preferred))")
+import CoreML
+
+func opsOffAccelerators(modelURL: URL) async throws -> [String] {
+    let plan = try await MLComputePlan.load(contentsOf: modelURL, configuration: MLModelConfiguration())
+    guard case .program(let program) = plan.modelStructure,
+          let entry = program.functions["main"] else { return [] }
+    return entry.block.operations.compactMap { op in
+        guard let usage = plan.deviceUsage(for: op), case .cpu = usage.preferred else { return nil }
+        let cost = plan.estimatedCost(of: op)?.weight ?? 0
+        return "\(op.operatorName) cost \(cost)"
+    }
 }
 ```
 
-### Instruments
-
-Use the **Core ML** instrument template in Instruments to profile:
-- Model load time
-- Prediction latency (per-operation breakdown)
-- Compute device dispatch (CPU/GPU/ANE per operation)
-- Memory allocation
-
-Run outside the debugger for accurate results (Xcode: Product > Profile).
-
-## Model Deployment
+The Instruments Core ML template shows load time, per-prediction latency with a
+per-operation breakdown, which device each operation ran on, and memory.
+Profile with Product > Profile, not under the debugger, or the numbers are
+not representative.
 
-### Bundle vs Downloaded Assets
+## Deployment
 
-| Strategy | Pros | Cons |
+| Strategy | Strength | Cost |
 |---|---|---|
-| Bundle in app | Instant availability, works offline | Increases app download size |
-| Background Assets | Preferred for large or updateable model assets | Requires asset-pack setup |
-| On-demand resources | Smaller initial download for existing ODR apps | Legacy technology; prefer Background Assets for new work |
-| CloudKit / server | Maximum flexibility | Requires network, longer setup |
-
-### Size Considerations
-
-- For iOS/iPadOS 18+, App Store Connect lists a 4 GB thinned app bundle limit
-  and 8 GB thinned ODR asset-pack limit.
-- Prefer Background Assets for new large or updateable model assets; keep ODR
-  guidance for existing projects that already use it.
-- Pre-compile to `.mlmodelc` to skip on-device compilation
-- For downloaded `.mlmodel` or `.mlpackage` files, compile once with
-  `MLModel.compileModel(at:)`, move the resulting `.mlmodelc` out of Core ML's
-  temporary location, and cache it by model version.
-- Validate memory and performance on physical target devices, especially the
-  lowest-memory supported device. Check model load, first prediction, repeated
-  predictions, background/foreground transitions, and low-memory behavior.
-
-For Background Assets, make the asset pack locally available, resolve the model
-URL, then load the compiled model with `MLModel.load(contentsOf:configuration:)`.
+| In the app bundle | Available instantly and offline | Bigger download |
+| Background Assets | Preferred for large or updatable models | Asset pack setup |
+| On-demand resources | Smaller first download for apps already on ODR | Legacy; choose Background Assets for new work |
+| CloudKit or own server | Most flexible | Needs network, more plumbing |
+
+- Size ceilings change; check the current App Store Connect limits. For
+  iOS/iPadOS 18 and later they list 4 GB for the thinned app bundle and 8 GB
+  per thinned on-demand-resource asset pack.
+- Keep on-demand-resource guidance for projects that already use it.
+- Ship `.mlmodelc` (precompiled) so the device does not compile at first use.
+- For downloaded `.mlmodel`/`.mlpackage` files: compile once with
+  `compileModel(at:)`, move the result out of Core ML's temporary directory,
+  and key the cache by model version.
+- With Background Assets: make the asset pack locally available, resolve the
+  model's URL inside it, then `MLModel.load(contentsOf:configuration:)`.
+- Test on physical devices, above all the lowest-memory one you support: load
+  time, first prediction, repeated predictions, background and foreground
+  transitions, and low-memory behavior.
+
+On-demand resources, for apps that already use them:
 
 ```swift
-// Existing On-Demand Resources project
-let request = NSBundleResourceRequest(tags: ["ml-model-v2"])
-try await request.beginAccessingResources()
-let modelURL = Bundle.main.url(forResource: "LargeModel", withExtension: "mlmodelc")!
-let model = try await MLModel.load(contentsOf: modelURL, configuration: config)
-// Call request.endAccessingResources() when done
+import CoreML
+
+func loadTaggedModel(tag: String, name: String) async throws -> (MLModel, NSBundleResourceRequest) {
+    let request = NSBundleResourceRequest(tags: [tag])
+    try await request.beginAccessingResources()
+    guard let url = Bundle.main.url(forResource: name, withExtension: "mlmodelc") else {
+        request.endAccessingResources()
+        throw CocoaError(.fileNoSuchFile)
+    }
+    let model = try await MLModel.load(contentsOf: url, configuration: MLModelConfiguration())
+    return (model, request)
+}
 ```
 
-## Memory Management
-
-- **Unload on background:** Release model references when the app enters background
-  to free GPU/ANE memory. Reload on foreground return.
-- **Choose compute units by context:** use `.all` by default. Consider `.cpuOnly`
-  only when profiling or app policy shows accelerator contention, thermal state,
-  energy budget, deterministic testing, or a legitimate background execution
-  constraint makes CPU the right tradeoff.
-- **Share model instances:** Never create multiple `MLModel` instances from the same
-  compiled model. Use an actor to provide shared access.
-- **Monitor memory pressure:** Large models (>100 MB) can trigger memory warnings.
-  Register for `UIApplication.didReceiveMemoryWarningNotification` and release
-  cached models when under pressure.
-
-See [references/coreml-swift-integration.md](references/coreml-swift-integration.md) for an actor-based model manager with
-lifecycle-aware loading and cache eviction.
-
-## Common Mistakes
-
-**DON'T:** Load models on the main thread.
-**DO:** Use `MLModel.load(contentsOf:configuration:)` async API or load on a background actor.
-**Why:** Large models can take seconds to load, freezing the UI.
-
-**DON'T:** Recompile `.mlpackage` to `.mlmodelc` on every app launch.
-**DO:** Compile once with `MLModel.compileModel(at:)` and cache the compiled URL persistently.
-**Why:** Compilation is expensive. Cache the `.mlmodelc` in Application Support.
-
-**DON'T:** Hardcode `.cpuOnly` unless you have a specific reason.
-**DO:** Use `.all` and let the system choose the optimal compute unit.
-**Why:** `.all` enables Neural Engine and GPU, which are faster and more energy-efficient.
-
-**DON'T:** Claim GPU or Neural Engine are categorically unavailable for all
-background-adjacent work.
-**DO:** Treat background execution as policy-, mode-, contention-, thermal-, and
-energy-dependent, and profile the actual workload on device.
-**Why:** Apps may be suspended, throttled, or limited by their background mode;
-`.cpuOnly` is a tradeoff, not a universal requirement.
-
-**DON'T:** Ignore `MLFeatureValue` type mismatches between input and model expectations.
-**DO:** Match types exactly -- use `MLFeatureValue(pixelBuffer:)` for images, not raw data.
-**Why:** Type mismatches cause cryptic runtime crashes or silent incorrect results.
-
-**DON'T:** Create a new `MLModel` instance for every prediction.
-**DO:** Load once and reuse. Use an actor to manage the model lifecycle.
-**Why:** Model loading allocates significant memory and compute resources.
-
-**DON'T:** Skip error handling for model loading and prediction.
-**DO:** Catch errors and provide fallback behavior when the model fails.
-**Why:** Models can fail to load on older devices or when resources are constrained.
-
-**DON'T:** Assume all operations run on the Neural Engine.
-**DO:** Use `MLComputePlan` (iOS 17.4+) to verify device dispatch per operation.
-**Why:** Unsupported operations fall back to CPU, which may bottleneck the pipeline.
-
-**DON'T:** Process images manually before passing to Vision + Core ML.
-**DO:** Use `CoreMLRequest` (iOS 18+) or `VNCoreMLRequest` (legacy) to let Vision handle preprocessing.
-**Why:** Vision handles orientation, scaling, and pixel format conversion correctly.
-
-## Review Checklist
-
-- [ ] Model loaded asynchronously (not blocking main thread)
-- [ ] `MLModelConfiguration.computeUnits` set appropriately for use case
-- [ ] Model instance reused across predictions (not recreated each time)
-- [ ] Auto-generated class used when available (typed inputs/outputs)
-- [ ] Error handling for model loading and prediction failures
-- [ ] Compiled model cached persistently if compiled at runtime
-- [ ] Image inputs use Vision pipeline (`CoreMLRequest` iOS 18+ or `VNCoreMLRequest`) for correct preprocessing
-- [ ] `MLComputePlan` checked to verify compute device dispatch (iOS 17.4+)
-- [ ] Batch predictions used when processing multiple inputs
-- [ ] Model size appropriate for deployment strategy (bundle, Background Assets, ODR)
-- [ ] Memory tested on target devices (especially older devices with less RAM)
-- [ ] Predictions run outside debugger for accurate performance measurement
+Keep the returned request alive while the model is in use and call
+`endAccessingResources()` when you are done with it.
+
+## Memory
+
+- Drop model references when the app goes to the background to release GPU
+  and Neural Engine memory; load again on return to the foreground.
+- `.all` is the default. Use `.cpuOnly` only when profiling or policy points
+  to contention, thermal limits, energy, deterministic tests or a real
+  background-execution constraint.
+- Keep one `MLModel` per compiled model and share it through an actor.
+- Models above roughly 100 MB can trigger memory warnings; listen for
+  `UIApplication.didReceiveMemoryWarningNotification` and release cached
+  models.
+
+## Common mistakes
+
+1. **Loading on the main thread.** Large models take seconds; use
+   `MLModel.load(contentsOf:configuration:)` or a background actor.
+2. **Recompiling every launch.** Compilation is expensive; compile once and
+   keep the `.mlmodelc` in Application Support.
+3. **Hard-coding `.cpuOnly`.** `.all` lets Core ML use the Neural Engine and
+   GPU, usually faster and more efficient.
+4. **Declaring GPU or Neural Engine off-limits for anything near the
+   background.** Access depends on system policy, execution mode, contention,
+   thermal state and energy; profile on a device. `.cpuOnly` is a tradeoff,
+   not a rule, and a backgrounded app may be suspended or throttled anyway.
+5. **Feature type mismatches.** Images go in as `MLFeatureValue(pixelBuffer:)`,
+   not raw bytes; mismatches crash obscurely or give silently wrong output.
+6. **A new `MLModel` per prediction.** Loading costs memory and time; load once
+   and reuse through an actor.
+7. **No error handling.** Loading and prediction can fail on older or
+   constrained devices; catch and fall back.
+8. **Assuming everything runs on the Neural Engine.** Unsupported operations
+   fall back to the CPU and can dominate latency; check with `MLComputePlan`.
+9. **Hand-preprocessing images for Vision.** `CoreMLRequest` (iOS 18) and
+   `VNCoreMLRequest` already handle orientation, scaling and pixel format.
+10. **Dropping `await` on prediction in async code.** On iOS 17 and later the
+    async overload is selected there; write `try await`.
+
+## Review checklist
+
+- [ ] Model loaded asynchronously
+- [ ] `computeUnits` chosen deliberately
+- [ ] One model instance reused across predictions
+- [ ] Generated class used where Xcode provides one
+- [ ] Load and prediction failures handled
+- [ ] Runtime-compiled models cached in a durable location
+- [ ] Image inputs routed through Vision (`CoreMLRequest` on iOS 18+, otherwise `VNCoreMLRequest`)
+- [ ] Device dispatch checked with `MLComputePlan` (iOS 17.4+)
+- [ ] Batch API used for many inputs
+- [ ] Model size matches the delivery strategy (bundle, Background Assets, ODR)
+- [ ] Memory measured on target devices, especially older low-RAM ones
+- [ ] Latency measured outside the debugger
 
 ## References
 
-- Patterns and code: [references/coreml-swift-integration.md](references/coreml-swift-integration.md)
-- Model conversion and optimization (Python-side): covered in the `apple-on-device-ai` skill
-- Apple docs: [Core ML](https://sosumi.ai/documentation/coreml) |
-  [MLModel](https://sosumi.ai/documentation/coreml/mlmodel) |
-  [MLTensor](https://sosumi.ai/documentation/coreml/mltensor) |
-  [MLComputePlan](https://sosumi.ai/documentation/coreml/mlcomputeplan-1w21n) |
-  [Background Assets](https://sosumi.ai/documentation/backgroundassets)
+- `references/coreml-swift-integration.md`
+- `apple-on-device-ai` for conversion and compression in Python
+- https://developer.apple.com/documentation/coreml
+- https://developer.apple.com/documentation/coreml/mlmodel
+- https://developer.apple.com/documentation/coreml/mltensor
+- https://developer.apple.com/documentation/coreml/mlcomputeplan-1w21n
+- https://developer.apple.com/documentation/backgroundassets