@mmerterden/multi-agent-pipeline 20.6.0 → 20.8.0

package/pipeline/skills/shared/external/background-processing/references/background-task-patterns.md

@@ -1,415 +1,267 @@
-# Background Task Patterns  -  Extended Reference
+# Background Task Patterns
 
-Overflow reference for the background-processing skill. Contains debugging tips,
-advanced background URLSession patterns, background push best practices, and
-SwiftUI integration patterns.
+Deeper material for the `background-processing` skill: debugging, error codes,
+checkpointed processing, background transfer details, silent push limits, the
+SwiftUI modifier and continued-processing specifics.
 
-## Contents
+## Debugging
 
-- [Debugging Background Tasks](#debugging-background-tasks)
-- [Advanced BGProcessingTask Patterns](#advanced-bgprocessingtask-patterns)
-- [Background URLSession  -  Extended Patterns](#background-urlsession--extended-patterns)
-- [Background Push  -  Extended Patterns](#background-push--extended-patterns)
-- [SwiftUI BackgroundTask Modifier](#swiftui-backgroundtask-modifier)
-- [BGContinuedProcessingTask  -  Extended Patterns](#bgcontinuedprocessingtask--extended-patterns)
+### Forcing a launch from the debugger
 
-## Debugging Background Tasks
+Background tasks rarely fire while you are watching. With the app running
+under Xcode, pause it and type in the LLDB console:
 
-### Simulating Task Launches in Xcode
-
-Use the LLDB console to trigger tasks instantly during development. The app must
-be running on a device in the debugger with a breakpoint hit or paused.
-
-These are Apple-documented private functions for development only. Do not
-include references to them in App Store-submitted code.
-
-```swift
-e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.example.app.refresh"]
 ```
-
-For processing tasks:
-
-```swift
-e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.example.app.db-cleanup"]
+e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"org.sample.photos.refresh"]
 ```
 
-To simulate early termination (expiration):
+Resume, and the registered handler runs. To test the expiry path, start the
+task that way, pause again and send:
 
-```swift
-e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateExpirationForTaskWithIdentifier:@"com.example.app.refresh"]
+```
+e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateExpirationForTaskWithIdentifier:@"org.sample.photos.refresh"]
 ```
 
-### Verifying Pending Tasks
+These selectors are private. Apple documents them for development only; they
+must never appear in code that ships to the App Store.
 
-Check what tasks are currently scheduled:
+### What is scheduled right now
 
 ```swift
-BGTaskScheduler.shared.getPendingTaskRequests { requests in
-    for request in requests {
-        print("Pending: \(request.identifier), earliest: \(String(describing: request.earliestBeginDate))")
+BGTaskScheduler.shared.getPendingTaskRequests { pending in
+    for request in pending {
+        Log.background.debug("\(request.identifier) not before \(String(describing: request.earliestBeginDate))")
     }
 }
 ```
 
-### Common Debugging Issues
+### Why a task never runs
 
-| Symptom | Cause | Fix |
-|---|---|---|
-| Task never fires | Identifier not in Info.plist | Add to `BGTaskSchedulerPermittedIdentifiers` |
-| Task never fires | Background modes not enabled | Enable `fetch` and/or `processing` in capabilities |
-| Task never fires on device | Background App Refresh disabled or Low Power Mode active | Check `UIApplication.shared.backgroundRefreshStatus`; Low Power Mode reduces background runtime |
-| `.notPermitted` error | Identifier mismatch | Verify exact string match between code and plist |
-| `.unavailable` error | Running in extension | BGTaskScheduler not available in app extensions |
-| `.tooManyPendingTaskRequests` | More than 1 refresh task or 10 processing tasks scheduled in total | Cancel old requests before submitting new ones |
+| Symptom | Likely cause | Fix |
+|---------|--------------|-----|
+| Never fires anywhere | Identifier absent from `BGTaskSchedulerPermittedIdentifiers` | Add the exact string |
+| Never fires anywhere | Background modes off | Enable `fetch` or `processing` |
+| Never fires on one device | Background App Refresh turned off, or Low Power Mode on | Read `UIApplication.shared.backgroundRefreshStatus`; Low Power Mode also shortens runtime |
+| `submit` throws `.notPermitted` | Code and Info.plist spell the identifier differently | Make them match |
+| `submit` throws `.unavailable` | Called from an app extension | Schedule from the app; extensions cannot use `BGTaskScheduler` |
+| `submit` throws `.tooManyPendingTaskRequests` | Over the limit of 1 refresh and 10 processing requests pending | Cancel stale requests before adding new ones |
+| `submit` throws `.immediateRunIneligible` | A continued processing request used `.fail` and could not start now | Tell the user, or submit with `.queue` |
 
-Submitting an unexecuted task request with the same identifier replaces the
-previous request.
+A new request with the same identifier as one that has not run yet replaces
+the old one, so resubmitting is safe.
 
-## Advanced BGProcessingTask Patterns
+## Processing tasks in depth
 
-### Conditional Requirements
-
-Use `requiresExternalPower` and `requiresNetworkConnectivity` to ensure the
-system only launches your task when conditions are met:
+### Overnight work on power and network
 
 ```swift
-func scheduleSyncTask() {
-    let request = BGProcessingTaskRequest(
-        identifier: "com.example.app.full-sync"
-    )
-    // Only run when charging and connected to network
+func scheduleNightlyBackup() {
+    let request = BGProcessingTaskRequest(identifier: "org.sample.photos.backup")
     request.requiresExternalPower = true
     request.requiresNetworkConnectivity = true
-    // Don't run before 2 AM
-    var components = DateComponents()
-    components.hour = 2
-    if let twoAM = Calendar.current.nextDate(
-        after: Date(),
-        matching: components,
+    request.earliestBeginDate = Calendar.current.nextDate(
+        after: .now,
+        matching: DateComponents(hour: 2, minute: 0),
         matchingPolicy: .nextTime
-    ) {
-        request.earliestBeginDate = twoAM
-    }
-
-    do {
-        try BGTaskScheduler.shared.submit(request)
-    } catch {
-        print("Failed to schedule sync: \(error)")
-    }
+    )
+    try? BGTaskScheduler.shared.submit(request)
 }
 ```
 
-### Incremental Work with Checkpointing
+`nextDate(after:matching:matchingPolicy:)` finds the next 2 AM, which becomes
+the lower bound for the run.
+
+### Checkpointing
 
-Design tasks to save progress so they can resume if terminated:
+Long jobs rarely finish in one window. Save a cursor after each item and pick
+up where the last run stopped:
 
 ```swift
-func handleMigration(task: BGProcessingTask) {
+func backUp(_ task: BGProcessingTask, assets: [PhotoAsset]) {
+    nonisolated(unsafe) let task = task
+    let cursorKey = "backup.cursor"
     let work = Task {
-        let lastProcessed = UserDefaults.standard.integer(
-            forKey: "migrationLastIndex"
-        )
-        let items = try await loadItems()
-
-        for (index, item) in items.dropFirst(lastProcessed).enumerated() {
-            try Task.checkCancellation()
-            try await migrate(item)
-            // Checkpoint progress
-            UserDefaults.standard.set(
-                lastProcessed + index + 1,
-                forKey: "migrationLastIndex"
-            )
+        do {
+            let start = UserDefaults.standard.integer(forKey: cursorKey)
+            for (offset, asset) in assets.dropFirst(start).enumerated() {
+                try Task.checkCancellation()
+                try await upload(asset)
+                UserDefaults.standard.set(start + offset + 1, forKey: cursorKey)
+            }
+            UserDefaults.standard.removeObject(forKey: cursorKey)
+            task.setTaskCompleted(success: true)
+        } catch {
+            task.setTaskCompleted(success: false)
         }
-
-        task.setTaskCompleted(success: true)
     }
-
     task.expirationHandler = {
         work.cancel()
-        // Progress is saved -- next launch picks up where we left off
         task.setTaskCompleted(success: false)
     }
 }
 ```
 
-## Background URLSession  -  Extended Patterns
+When the window closes, the saved cursor lets the next run skip what is done.
+The `nonisolated(unsafe)` copy lets the non-`Sendable` `BGTask` be shared by
+the work `Task` and the expiration handler under Swift 6.
 
-### Configuration Best Practices
+## Background URLSession details
 
-```swift
-let config = URLSessionConfiguration.background(
-    withIdentifier: "com.example.app.background-transfer"
-)
+### Configuration options
 
-// isDiscretionary = true: system picks optimal time (WiFi, power)
-// Use for non-urgent transfers
+```swift
+let config = URLSessionConfiguration.background(withIdentifier: "org.sample.podcasts.transfers")
 config.isDiscretionary = true
-
-// sessionSendsLaunchEvents = true: app relaunched when transfer completes
 config.sessionSendsLaunchEvents = true
-
-// Set reasonable timeouts
-config.timeoutIntervalForResource = 60 * 60 * 24 * 7  // 7 days
-
-// Allow cellular (default is true)
+config.timeoutIntervalForResource = 7 * 24 * 60 * 60
 config.allowsCellularAccess = true
 ```
 
-### Upload with Background Session
+- `isDiscretionary`: for transfers that are not urgent; the system waits for
+  Wi-Fi and power.
+- `sessionSendsLaunchEvents`: relaunch the app when transfers complete.
+- `timeoutIntervalForResource`: how long a transfer may take overall; one week
+  here.
+- `allowsCellularAccess` is already `true` by default; set it to `false` to
+  keep large transfers on Wi-Fi.
 
-```swift
-func uploadFile(at fileURL: URL) {
-    var request = URLRequest(url: URL(string: "https://api.example.com/upload")!)
-    request.httpMethod = "POST"
-    let uploadTask = session.uploadTask(with: request, fromFile: fileURL)
-    uploadTask.resume()
-}
-```
-
-**Important:** Background sessions only support `uploadTask(with:fromFile:)` and
-`downloadTask(with:)`. Data tasks, `uploadTask(with:from:)` (Data), and
-closure/async-based tasks are **not** supported.
-
-### Handling App Relaunch
-
-When the system completes a background transfer and your app is not running, it
-relaunches the app. You must:
-
-1. Store the completion handler from
-   `application(_:handleEventsForBackgroundURLSession:completionHandler:)`
-2. Recreate the `URLSession` with the **same identifier**
-3. Call the stored completion handler in `urlSessionDidFinishEvents`
+### Uploads
 
 ```swift
-// AppDelegate
-func application(
-    _ application: UIApplication,
-    handleEventsForBackgroundURLSession identifier: String,
-    completionHandler: @escaping () -> Void
-) {
-    // Recreating session with the same identifier reconnects to the transfer
-    _ = DownloadManager.shared.session  // trigger lazy init
-    DownloadManager.shared.completionHandler = completionHandler
-}
-
-// In your URLSessionDelegate
-func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
-    Task { @MainActor in
-        self.completionHandler?()
-        self.completionHandler = nil
-    }
+@MainActor
+func sendRecording(at file: URL, to endpoint: URL) {
+    var request = URLRequest(url: endpoint)
+    request.httpMethod = "PUT"
+    let job = EpisodeDownloader.shared.session.uploadTask(with: request, fromFile: file)
+    job.resume()
 }
 ```
 
-### Handling Download Errors and Retries
+Background sessions accept only `uploadTask(with:fromFile:)` and
+`downloadTask(with:)` style tasks. Data tasks, uploads from in-memory `Data`
+(`uploadTask(with:from:)`) and every completion-handler or async variant are
+rejected.
 
-```swift
-func urlSession(
-    _ session: URLSession,
-    task: URLSessionTask,
-    didCompleteWithError error: (any Error)?
-) {
-    guard let error else { return }  // Success handled in didFinishDownloadingTo
-
-    let nsError = error as NSError
-
-    // Check if download can be resumed
-    if let resumeData = nsError.userInfo[NSURLSessionDownloadTaskResumeData] as? Data {
-        // Store resumeData and retry later
-        let downloadTask = session.downloadTask(withResumeData: resumeData)
-        downloadTask.resume()
-        return
-    }
+### Relaunch sequence
 
-    // Non-resumable error -- retry from scratch or notify user
-    if nsError.code == NSURLErrorNetworkConnectionLost {
-        // Re-enqueue the download
-        if let url = task.originalRequest?.url {
-            let newTask = session.downloadTask(with: url)
-            newTask.resume()
-        }
-    }
-}
-```
+1. The system calls `application(_:handleEventsForBackgroundURLSession:completionHandler:)`.
+   Keep the handler.
+2. Create the session again with the same identifier; touching the lazy
+   `session` property is enough to reconnect to the running transfers.
+3. The delegate receives the queued events, then
+   `urlSessionDidFinishEvents(forBackgroundURLSession:)`, where you call the
+   stored handler on the main actor.
 
-### Progress Tracking
+### Resuming after failure
 
 ```swift
-func urlSession(
-    _ session: URLSession,
-    downloadTask: URLSessionDownloadTask,
-    didWriteData bytesWritten: Int64,
-    totalBytesWritten: Int64,
-    totalBytesExpectedToWrite: Int64
-) {
-    let progress = Double(totalBytesWritten) / Double(totalBytesExpectedToWrite)
-    // Update UI on main actor if app is in foreground
-    Task { @MainActor in
-        DownloadProgressStore.shared.update(
-            taskID: downloadTask.taskIdentifier,
-            progress: progress
-        )
+nonisolated func urlSession(_ s: URLSession, task job: URLSessionTask, didCompleteWithError error: (any Error)?) {
+    guard let error = error as NSError? else { return }
+    if let partial = error.userInfo[NSURLSessionDownloadTaskResumeData] as? Data {
+        s.downloadTask(withResumeData: partial).resume()
+    } else if error.code == NSURLErrorNetworkConnectionLost, let url = job.originalRequest?.url {
+        s.downloadTask(with: url).resume()
     }
 }
 ```
 
-## Background Push  -  Extended Patterns
+Resume data lets the transfer continue from where it stopped. Without it, a
+dropped connection is retried from the start using the original URL.
 
-### Push Payload Requirements
+### Progress
 
-The `content-available: 1` flag is required. You can include custom data:
-
-```json
-{
-    "aps": {
-        "content-available": 1
-    },
-    "type": "new-message",
-    "conversation-id": "abc-123"
+```swift
+nonisolated func urlSession(_ s: URLSession, downloadTask job: URLSessionDownloadTask,
+                didWriteData chunk: Int64, totalBytesWritten written: Int64,
+                totalBytesExpectedToWrite expected: Int64) {
+    guard expected > 0 else { return }
+    let fraction = Double(written) / Double(expected)
+    Task { @MainActor in DownloadProgress.shared.update(job.taskIdentifier, fraction) }
 }
 ```
 
-**Do not** include `alert`, `badge`, or `sound` if you only want a silent push.
-Including visual notification keys changes the push behavior.
-
-Send background notification requests with APNs headers:
-
-```http
-apns-push-type: background
-apns-priority: 5
-```
-
-### Rate Limiting
-
-Apple throttles background push delivery. Guidelines:
-
-- Delivery is low priority and not guaranteed.
-- Do not send more than two or three background notifications per hour.
-- The system may hold only the newest background notification and discard older
-  held notifications.
-- If the user force-quits the app, background pushes stop until the next manual
-  launch.
-- Use `apns-priority: 5`; high-priority pushes are for user-visible
-  notifications, not silent refresh.
-
-### Handling Push with Async Work
+UI updates go through the main actor. This method and the failure handler
+above are `nonisolated` because they belong to the `@MainActor` downloader from
+the parent skill, while the session calls them on its own queue.
+
+## Silent push details
+
+- `content-available: 1` is required; custom keys sit beside `aps` at the top
+  level.
+- Leave out `alert`, `badge` and `sound`. Any visible key turns the push into a
+  user-facing notification with different handling.
+- APNs treats these pushes as low priority with no delivery promise. Send no
+  more than two or three an hour. When several are held, the system may keep
+  only the newest and drop the rest.
+- If the user force-quits the app, silent pushes stop until they open it
+  again by hand.
+- Use priority `5`; priority `10` is meant for notifications the user sees.
+- The app has about 30 seconds to call the completion handler. Missing that
+  shrinks its future background push budget.
 
 ```swift
-func application(
-    _ application: UIApplication,
-    didReceiveRemoteNotification userInfo: [AnyHashable: Any],
-    fetchCompletionHandler completionHandler:
-        @escaping (UIBackgroundFetchResult) -> Void
-) {
-    guard let type = userInfo["type"] as? String else {
-        completionHandler(.noData)
-        return
-    }
-
-    Task {
-        do {
-            switch type {
-            case "new-message":
-                let conversationID = userInfo["conversation-id"] as? String
-                let fetched = try await MessageService.shared
-                    .fetchMessages(for: conversationID)
-                completionHandler(fetched ? .newData : .noData)
-
-            case "config-update":
-                try await ConfigService.shared.refreshConfig()
-                completionHandler(.newData)
-
-            default:
-                completionHandler(.noData)
-            }
-        } catch {
-            completionHandler(.failed)
-        }
+func handleSilentPush(_ payload: [AnyHashable: Any]) async -> UIBackgroundFetchResult {
+    switch payload["kind"] as? String {
+    case "album-changed":
+        return await LibrarySync.shared.syncChangedAlbums() ? .newData : .noData
+    case "purge-cache":
+        ThumbnailCache.shared.removeAll()
+        return .newData
+    default:
+        return .noData
     }
 }
 ```
 
-**Important:** You have approximately 30 seconds to call `completionHandler`.
-Failure to do so causes the system to penalize your app's background push
-budget.
+The fetch-completion delegate method calls this and forwards the result to its
+handler; a payload with no known kind reports `.noData`.
 
-## SwiftUI BackgroundTask Modifier
+## SwiftUI `.backgroundTask`
 
-SwiftUI provides a `.backgroundTask` modifier as an alternative to manual
-`BGTaskScheduler` registration:
+A SwiftUI app can declare the handler on the scene instead of registering it
+by hand:
 
 ```swift
 @main
-struct MyApp: App {
+struct PhotosApp: App {
     var body: some Scene {
-        WindowGroup { ContentView() }
-            .backgroundTask(.appRefresh("com.example.app.refresh")) {
-                await refreshFeed()
-                // Schedule the next one
-                scheduleAppRefresh()
+        WindowGroup { LibraryView() }
+            .backgroundTask(.appRefresh("org.sample.photos.refresh")) {
+                _ = await LibrarySync.shared.syncChangedAlbums()
+                LibrarySync.shared.scheduleRefresh()
             }
     }
 }
 ```
 
-This is a SwiftUI handler for matching background tasks. You still need the
-Info.plist identifiers, background modes, scheduling, and cancellation-safe work
-patterns.
+The rest of the contract is unchanged: the identifier is still listed in
+Info.plist, background modes are still on, the next run is still scheduled,
+and the work must still stop cleanly when cancelled.
 
-## BGContinuedProcessingTask  -  Extended Patterns
+## Continued processing specifics
 
-### Checking Supported Resources
+### Resources
 
-Before requesting GPU or other resources, verify the device supports them and
-enable Background GPU Access
-(`com.apple.developer.background-tasks.continued-processing.gpu`) for GPU work:
+Call `BGTaskScheduler.supportedResources` and request `.gpu` only when it is
+listed. GPU use also depends on the Background GPU Access entitlement.
 
-```swift
-let supported = BGTaskScheduler.supportedResources
-if supported.contains(.gpu) {
-    request.requiredResources = .gpu
-}
-```
+### Submission strategy
 
-### Submission Strategies
+`BGContinuedProcessingTaskRequest.SubmissionStrategy` has two cases:
 
-`BGContinuedProcessingTaskRequest.SubmissionStrategy` controls behavior when the
-system cannot run the task immediately:
+| Strategy | Behaviour | Pick it when |
+|----------|-----------|--------------|
+| `.queue` | Accepted and started as soon as the system allows | The job can start a little later without harm |
+| `.fail` | `submit` fails unless the task can start now | The user is waiting and a delayed start is useless |
 
-| Strategy | Behavior |
-|---|---|
-| `.queue` | Task is queued and starts as soon as possible |
-| `.fail` | Submission fails immediately if can't run now |
+### Cancellation from the Live Activity
 
-Use `.fail` when the work is only relevant in the current moment (e.g., a user
-is waiting). Use `.queue` for work that can start whenever the system allows.
+The user can stop the job from the system Live Activity. That arrives through
+`expirationHandler`: cancel the work, delete partial output, and complete with
+`success: false`.
 
-### Cancellation by the User
+### Progress drives survival
 
-The system shows a Live Activity for continued processing tasks. The user can
-cancel the task from there. Handle this in your expiration handler:
-
-```swift
-task.expirationHandler = {
-    // Clean up partial work
-    cleanupPartialExport()
-    task.setTaskCompleted(success: false)
-}
-```
-
-### Progress Reporting
-
-The system uses your `Progress` object to decide termination priority. Tasks
-with no progress updates are terminated first under resource pressure:
-
-```swift
-// Report fine-grained progress
-let progress = task.progress
-progress.totalUnitCount = Int64(totalItems)
-
-for (index, item) in items.enumerated() {
-    try Task.checkCancellation()
-    await process(item)
-    progress.completedUnitCount = Int64(index + 1)
-}
-```
+When resources run short, the system reads each task's `Progress` to decide
+what to end first. A task that never updates progress is the first to go, so
+report small steps often.

package/pipeline/skills/shared/external/backlog/BACKLOG.md

@@ -1,6 +1,6 @@
 # Backlog
 
-Deferred-but-approved work, kept whole. Entries move `deferred` -> `ready` -> `in-progress` -> `done` and are never deleted. The full spec travels verbatim from when the job was deferred.
+Deferred-but-approved work, kept whole. Entries move `deferred` -> `ready` -> `in-progress` -> `done`. An entry leaves only when a re-audit finds its subject gone, or when it is done and its lesson lives in the skill or doc that owns that area. The full spec travels verbatim from when the job was deferred.
 
 ## Ready