@mmerterden/multi-agent-pipeline 20.7.0 → 20.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (264) hide show
  1. package/CHANGELOG.md +9 -0
  2. package/LICENSE +0 -10
  3. package/docs/facts.json +1 -1
  4. package/manifest.json +266 -267
  5. package/package.json +2 -2
  6. package/pipeline/scripts/_notices.mjs +1 -1
  7. package/pipeline/skills/.skill-manifest.json +68 -68
  8. package/pipeline/skills/shared/README.md +70 -70
  9. package/pipeline/skills/shared/external/alarmkit/SKILL.md +373 -381
  10. package/pipeline/skills/shared/external/alarmkit/evals/evals.json +23 -18
  11. package/pipeline/skills/shared/external/alarmkit/references/alarmkit-patterns.md +328 -378
  12. package/pipeline/skills/shared/external/app-clips/SKILL.md +260 -160
  13. package/pipeline/skills/shared/external/app-clips/evals/evals.json +27 -27
  14. package/pipeline/skills/shared/external/app-clips/references/data-handoff-notifications-location.md +150 -83
  15. package/pipeline/skills/shared/external/app-clips/references/routing-and-experiences.md +135 -83
  16. package/pipeline/skills/shared/external/app-clips/references/size-capabilities-and-promotion.md +143 -85
  17. package/pipeline/skills/shared/external/app-intents/SKILL.md +302 -304
  18. package/pipeline/skills/shared/external/app-intents/evals/evals.json +21 -21
  19. package/pipeline/skills/shared/external/app-intents/references/appintents-advanced.md +594 -894
  20. package/pipeline/skills/shared/external/app-store-optimization/SKILL.md +339 -277
  21. package/pipeline/skills/shared/external/app-store-optimization/evals/evals.json +27 -23
  22. package/pipeline/skills/shared/external/app-store-optimization/references/keyword-research-methodology.md +105 -122
  23. package/pipeline/skills/shared/external/app-store-optimization/references/product-page-variants.md +143 -166
  24. package/pipeline/skills/shared/external/app-store-review/SKILL.md +307 -326
  25. package/pipeline/skills/shared/external/app-store-review/evals/evals.json +21 -21
  26. package/pipeline/skills/shared/external/app-store-review/references/privacy-manifest.md +105 -67
  27. package/pipeline/skills/shared/external/app-store-review/references/review-checklists.md +114 -101
  28. package/pipeline/skills/shared/external/apple-on-device-ai/SKILL.md +333 -360
  29. package/pipeline/skills/shared/external/apple-on-device-ai/evals/evals.json +24 -27
  30. package/pipeline/skills/shared/external/apple-on-device-ai/references/coreml-conversion.md +215 -322
  31. package/pipeline/skills/shared/external/apple-on-device-ai/references/coreml-optimization.md +161 -256
  32. package/pipeline/skills/shared/external/apple-on-device-ai/references/foundation-models.md +277 -387
  33. package/pipeline/skills/shared/external/apple-on-device-ai/references/mlx-swift.md +196 -210
  34. package/pipeline/skills/shared/external/authentication/SKILL.md +265 -381
  35. package/pipeline/skills/shared/external/authentication/evals/evals.json +25 -25
  36. package/pipeline/skills/shared/external/authentication/references/keychain-biometric.md +133 -178
  37. package/pipeline/skills/shared/external/authentication/references/passkeys.md +111 -147
  38. package/pipeline/skills/shared/external/avkit/SKILL.md +267 -364
  39. package/pipeline/skills/shared/external/avkit/evals/evals.json +26 -26
  40. package/pipeline/skills/shared/external/avkit/references/avkit-patterns.md +375 -493
  41. package/pipeline/skills/shared/external/background-processing/SKILL.md +270 -382
  42. package/pipeline/skills/shared/external/background-processing/evals/evals.json +22 -22
  43. package/pipeline/skills/shared/external/background-processing/references/background-task-patterns.md +169 -317
  44. package/pipeline/skills/shared/external/callkit-voip/SKILL.md +290 -371
  45. package/pipeline/skills/shared/external/callkit-voip/evals/evals.json +24 -24
  46. package/pipeline/skills/shared/external/callkit-voip/references/callkit-patterns.md +175 -343
  47. package/pipeline/skills/shared/external/cloudkit-sync/SKILL.md +292 -381
  48. package/pipeline/skills/shared/external/cloudkit-sync/evals/evals.json +33 -30
  49. package/pipeline/skills/shared/external/cloudkit-sync/references/cloudkit-patterns.md +227 -355
  50. package/pipeline/skills/shared/external/contacts-framework/SKILL.md +197 -346
  51. package/pipeline/skills/shared/external/contacts-framework/evals/evals.json +19 -21
  52. package/pipeline/skills/shared/external/contacts-framework/references/contacts-patterns.md +169 -308
  53. package/pipeline/skills/shared/external/core-bluetooth/SKILL.md +226 -376
  54. package/pipeline/skills/shared/external/core-bluetooth/evals/evals.json +25 -22
  55. package/pipeline/skills/shared/external/core-bluetooth/references/ble-patterns.md +257 -337
  56. package/pipeline/skills/shared/external/core-data/SKILL.md +292 -368
  57. package/pipeline/skills/shared/external/core-data/evals/evals.json +30 -27
  58. package/pipeline/skills/shared/external/core-motion/SKILL.md +235 -324
  59. package/pipeline/skills/shared/external/core-motion/evals/evals.json +31 -27
  60. package/pipeline/skills/shared/external/core-motion/references/motion-patterns.md +210 -310
  61. package/pipeline/skills/shared/external/core-nfc/SKILL.md +292 -366
  62. package/pipeline/skills/shared/external/core-nfc/evals/evals.json +22 -24
  63. package/pipeline/skills/shared/external/core-nfc/references/nfc-patterns.md +233 -329
  64. package/pipeline/skills/shared/external/coreml/SKILL.md +323 -367
  65. package/pipeline/skills/shared/external/coreml/evals/evals.json +24 -21
  66. package/pipeline/skills/shared/external/coreml/references/coreml-swift-integration.md +562 -565
  67. package/pipeline/skills/shared/external/cryptokit/SKILL.md +253 -394
  68. package/pipeline/skills/shared/external/cryptokit/evals/evals.json +20 -18
  69. package/pipeline/skills/shared/external/cryptokit/references/cryptokit-patterns.md +299 -488
  70. package/pipeline/skills/shared/external/debugging-instruments/SKILL.md +270 -323
  71. package/pipeline/skills/shared/external/debugging-instruments/evals/evals.json +27 -30
  72. package/pipeline/skills/shared/external/debugging-instruments/references/instruments-guide.md +167 -315
  73. package/pipeline/skills/shared/external/debugging-instruments/references/lldb-patterns.md +140 -193
  74. package/pipeline/skills/shared/external/device-integrity/SKILL.md +230 -353
  75. package/pipeline/skills/shared/external/device-integrity/evals/evals.json +25 -21
  76. package/pipeline/skills/shared/external/device-integrity/references/device-integrity-patterns.md +159 -197
  77. package/pipeline/skills/shared/external/energykit/SKILL.md +225 -392
  78. package/pipeline/skills/shared/external/energykit/evals/evals.json +29 -28
  79. package/pipeline/skills/shared/external/energykit/references/energykit-patterns.md +174 -470
  80. package/pipeline/skills/shared/external/eventkit-calendar/SKILL.md +261 -383
  81. package/pipeline/skills/shared/external/eventkit-calendar/evals/evals.json +25 -22
  82. package/pipeline/skills/shared/external/eventkit-calendar/references/eventkit-patterns.md +165 -268
  83. package/pipeline/skills/shared/external/healthkit/SKILL.md +252 -303
  84. package/pipeline/skills/shared/external/healthkit/evals/evals.json +24 -23
  85. package/pipeline/skills/shared/external/healthkit/references/healthkit-patterns.md +369 -523
  86. package/pipeline/skills/shared/external/homekit-matter/SKILL.md +233 -348
  87. package/pipeline/skills/shared/external/homekit-matter/evals/evals.json +27 -22
  88. package/pipeline/skills/shared/external/homekit-matter/references/matter-commissioning.md +199 -305
  89. package/pipeline/skills/shared/external/ios-accessibility/SKILL.md +368 -340
  90. package/pipeline/skills/shared/external/ios-accessibility/evals/evals.json +28 -27
  91. package/pipeline/skills/shared/external/ios-accessibility/references/a11y-patterns.md +314 -260
  92. package/pipeline/skills/shared/external/ios-accessibility/references/media-accessibility.md +97 -67
  93. package/pipeline/skills/shared/external/ios-accessibility/references/nutrition-labels.md +165 -101
  94. package/pipeline/skills/shared/external/ios-localization/SKILL.md +258 -371
  95. package/pipeline/skills/shared/external/ios-localization/evals/evals.json +23 -23
  96. package/pipeline/skills/shared/external/ios-localization/references/formatstyle-locale.md +283 -491
  97. package/pipeline/skills/shared/external/ios-localization/references/string-catalogs.md +313 -440
  98. package/pipeline/skills/shared/external/ios-networking/SKILL.md +265 -341
  99. package/pipeline/skills/shared/external/ios-networking/evals/evals.json +24 -24
  100. package/pipeline/skills/shared/external/ios-networking/references/background-websocket.md +425 -652
  101. package/pipeline/skills/shared/external/ios-networking/references/file-storage-patterns.md +143 -285
  102. package/pipeline/skills/shared/external/ios-networking/references/lightweight-clients.md +93 -53
  103. package/pipeline/skills/shared/external/ios-networking/references/network-framework.md +231 -456
  104. package/pipeline/skills/shared/external/ios-networking/references/urlsession-patterns.md +517 -784
  105. package/pipeline/skills/shared/external/ios-simulator/SKILL.md +265 -393
  106. package/pipeline/skills/shared/external/ios-simulator/evals/evals.json +21 -21
  107. package/pipeline/skills/shared/external/ios-simulator/references/simctl-commands.md +177 -270
  108. package/pipeline/skills/shared/external/live-activities/SKILL.md +318 -360
  109. package/pipeline/skills/shared/external/live-activities/evals/evals.json +21 -21
  110. package/pipeline/skills/shared/external/live-activities/references/activitykit-patterns.md +478 -710
  111. package/pipeline/skills/shared/external/mapkit-location/SKILL.md +295 -267
  112. package/pipeline/skills/shared/external/mapkit-location/evals/evals.json +28 -24
  113. package/pipeline/skills/shared/external/mapkit-location/references/mapkit-corelocation-patterns.md +378 -532
  114. package/pipeline/skills/shared/external/mapkit-location/references/mapkit-patterns.md +397 -499
  115. package/pipeline/skills/shared/external/metrickit-diagnostics/SKILL.md +165 -348
  116. package/pipeline/skills/shared/external/metrickit-diagnostics/evals/evals.json +26 -23
  117. package/pipeline/skills/shared/external/metrickit-diagnostics/references/metrickit-patterns.md +123 -130
  118. package/pipeline/skills/shared/external/musickit-audio/SKILL.md +189 -315
  119. package/pipeline/skills/shared/external/musickit-audio/evals/evals.json +22 -21
  120. package/pipeline/skills/shared/external/musickit-audio/references/musickit-patterns.md +181 -270
  121. package/pipeline/skills/shared/external/natural-language/SKILL.md +188 -340
  122. package/pipeline/skills/shared/external/natural-language/evals/evals.json +21 -21
  123. package/pipeline/skills/shared/external/natural-language/references/translation-patterns.md +171 -225
  124. package/pipeline/skills/shared/external/passkit-wallet/SKILL.md +258 -392
  125. package/pipeline/skills/shared/external/passkit-wallet/evals/evals.json +30 -29
  126. package/pipeline/skills/shared/external/passkit-wallet/references/wallet-passes.md +164 -231
  127. package/pipeline/skills/shared/external/pdfkit/SKILL.md +312 -344
  128. package/pipeline/skills/shared/external/pdfkit/evals/evals.json +19 -19
  129. package/pipeline/skills/shared/external/pdfkit/references/pdfkit-patterns.md +413 -624
  130. package/pipeline/skills/shared/external/pencilkit-drawing/SKILL.md +242 -358
  131. package/pipeline/skills/shared/external/pencilkit-drawing/evals/evals.json +25 -21
  132. package/pipeline/skills/shared/external/pencilkit-drawing/references/pencilkit-patterns.md +161 -226
  133. package/pipeline/skills/shared/external/permissionkit/SKILL.md +282 -400
  134. package/pipeline/skills/shared/external/permissionkit/evals/evals.json +27 -30
  135. package/pipeline/skills/shared/external/permissionkit/references/permissionkit-patterns.md +237 -350
  136. package/pipeline/skills/shared/external/photos-camera-media/SKILL.md +276 -325
  137. package/pipeline/skills/shared/external/photos-camera-media/references/av-playback.md +299 -545
  138. package/pipeline/skills/shared/external/photos-camera-media/references/camera-capture.md +344 -588
  139. package/pipeline/skills/shared/external/photos-camera-media/references/image-loading-caching.md +316 -660
  140. package/pipeline/skills/shared/external/photos-camera-media/references/photokit-patterns.md +270 -416
  141. package/pipeline/skills/shared/external/push-notifications/SKILL.md +312 -340
  142. package/pipeline/skills/shared/external/push-notifications/evals/evals.json +27 -26
  143. package/pipeline/skills/shared/external/push-notifications/references/notification-patterns.md +328 -485
  144. package/pipeline/skills/shared/external/push-notifications/references/rich-notifications.md +327 -560
  145. package/pipeline/skills/shared/external/realitykit-ar/SKILL.md +218 -410
  146. package/pipeline/skills/shared/external/realitykit-ar/evals/evals.json +24 -27
  147. package/pipeline/skills/shared/external/realitykit-ar/references/realitykit-patterns.md +221 -348
  148. package/pipeline/skills/shared/external/shareplay-activities/SKILL.md +222 -393
  149. package/pipeline/skills/shared/external/shareplay-activities/evals/evals.json +23 -24
  150. package/pipeline/skills/shared/external/shareplay-activities/references/shareplay-patterns.md +280 -420
  151. package/pipeline/skills/shared/external/speech-recognition/SKILL.md +217 -421
  152. package/pipeline/skills/shared/external/speech-recognition/evals/evals.json +23 -26
  153. package/pipeline/skills/shared/external/speech-recognition/references/speechanalyzer-patterns.md +133 -125
  154. package/pipeline/skills/shared/external/storekit/SKILL.md +228 -204
  155. package/pipeline/skills/shared/external/storekit/evals/evals.json +27 -24
  156. package/pipeline/skills/shared/external/storekit/references/app-review-guidelines.md +98 -109
  157. package/pipeline/skills/shared/external/storekit/references/core-patterns.md +298 -242
  158. package/pipeline/skills/shared/external/storekit/references/storekit-advanced.md +356 -649
  159. package/pipeline/skills/shared/external/swift-api-design-guidelines/SKILL.md +274 -399
  160. package/pipeline/skills/shared/external/swift-api-design-guidelines/evals/evals.json +22 -24
  161. package/pipeline/skills/shared/external/swift-api-design-guidelines/references/argument-labels-and-parameters.md +107 -108
  162. package/pipeline/skills/shared/external/swift-api-design-guidelines/references/conventions-and-special-rules.md +93 -165
  163. package/pipeline/skills/shared/external/swift-api-design-guidelines/references/naming-and-clarity.md +99 -137
  164. package/pipeline/skills/shared/external/swift-api-design-guidelines/references/side-effects-and-mutating-pairs.md +77 -120
  165. package/pipeline/skills/shared/external/swift-architecture/SKILL.md +334 -350
  166. package/pipeline/skills/shared/external/swift-architecture/evals/evals.json +22 -22
  167. package/pipeline/skills/shared/external/swift-charts/SKILL.md +208 -394
  168. package/pipeline/skills/shared/external/swift-charts/evals/evals.json +27 -30
  169. package/pipeline/skills/shared/external/swift-charts/references/charts-patterns.md +351 -762
  170. package/pipeline/skills/shared/external/swift-codable/SKILL.md +339 -343
  171. package/pipeline/skills/shared/external/swift-codable/evals/evals.json +20 -20
  172. package/pipeline/skills/shared/external/swift-concurrency/SKILL.md +303 -351
  173. package/pipeline/skills/shared/external/swift-concurrency/evals/evals.json +27 -24
  174. package/pipeline/skills/shared/external/swift-concurrency/references/approachable-concurrency.md +65 -80
  175. package/pipeline/skills/shared/external/swift-concurrency/references/async-algorithms.md +48 -84
  176. package/pipeline/skills/shared/external/swift-concurrency/references/bridging-interop.md +134 -79
  177. package/pipeline/skills/shared/external/swift-concurrency/references/concurrency-patterns.md +145 -167
  178. package/pipeline/skills/shared/external/swift-concurrency/references/diagnostics.md +62 -50
  179. package/pipeline/skills/shared/external/swift-concurrency/references/swiftui-concurrency.md +92 -121
  180. package/pipeline/skills/shared/external/swift-concurrency/references/synchronization-primitives.md +177 -241
  181. package/pipeline/skills/shared/external/swift-formatstyle/SKILL.md +258 -234
  182. package/pipeline/skills/shared/external/swift-language/SKILL.md +342 -382
  183. package/pipeline/skills/shared/external/swift-language/evals/evals.json +24 -27
  184. package/pipeline/skills/shared/external/swift-language/references/swift-attributes-interop.md +79 -56
  185. package/pipeline/skills/shared/external/swift-language/references/swift-patterns-extended.md +297 -340
  186. package/pipeline/skills/shared/external/swift-security/SKILL.md +180 -161
  187. package/pipeline/skills/shared/external/swift-security/evals/evals.json +25 -25
  188. package/pipeline/skills/shared/external/swift-security/references/biometric-authentication.md +314 -469
  189. package/pipeline/skills/shared/external/swift-security/references/certificate-trust.md +408 -476
  190. package/pipeline/skills/shared/external/swift-security/references/common-anti-patterns.md +260 -530
  191. package/pipeline/skills/shared/external/swift-security/references/compliance-owasp-mapping.md +270 -477
  192. package/pipeline/skills/shared/external/swift-security/references/credential-storage-patterns.md +573 -571
  193. package/pipeline/skills/shared/external/swift-security/references/cryptokit-public-key.md +370 -441
  194. package/pipeline/skills/shared/external/swift-security/references/cryptokit-symmetric.md +332 -433
  195. package/pipeline/skills/shared/external/swift-security/references/keychain-access-control.md +346 -468
  196. package/pipeline/skills/shared/external/swift-security/references/keychain-fundamentals.md +352 -472
  197. package/pipeline/skills/shared/external/swift-security/references/keychain-item-classes.md +431 -432
  198. package/pipeline/skills/shared/external/swift-security/references/keychain-sharing.md +328 -425
  199. package/pipeline/skills/shared/external/swift-security/references/migration-legacy-stores.md +341 -579
  200. package/pipeline/skills/shared/external/swift-security/references/secure-enclave.md +396 -457
  201. package/pipeline/skills/shared/external/swift-security/references/testing-security-code.md +354 -614
  202. package/pipeline/skills/shared/external/swift-testing/SKILL.md +188 -175
  203. package/pipeline/skills/shared/external/swift-testing/evals/evals.json +26 -24
  204. package/pipeline/skills/shared/external/swift-testing/references/testing-advanced.md +80 -84
  205. package/pipeline/skills/shared/external/swift-testing/references/testing-patterns.md +317 -433
  206. package/pipeline/skills/shared/external/swiftdata/SKILL.md +392 -256
  207. package/pipeline/skills/shared/external/swiftdata/evals/evals.json +24 -24
  208. package/pipeline/skills/shared/external/swiftdata/references/core-data-coexistence.md +206 -402
  209. package/pipeline/skills/shared/external/swiftdata/references/indexing.md +59 -52
  210. package/pipeline/skills/shared/external/swiftdata/references/predicate-pitfalls.md +57 -33
  211. package/pipeline/skills/shared/external/swiftdata/references/swiftdata-advanced.md +354 -747
  212. package/pipeline/skills/shared/external/swiftdata/references/swiftdata-queries.md +300 -508
  213. package/pipeline/skills/shared/external/swiftlint/SKILL.md +175 -226
  214. package/pipeline/skills/shared/external/swiftlint/references/adoption-and-configuration.md +141 -208
  215. package/pipeline/skills/shared/external/swiftlint/references/custom-rules-and-analyze.md +100 -109
  216. package/pipeline/skills/shared/external/swiftlint/references/plugins-run-scripts-and-integrations.md +159 -179
  217. package/pipeline/skills/shared/external/swiftlint/references/rule-reference.md +383 -18
  218. package/pipeline/skills/shared/external/swiftlint/references/rules-suppressions-and-baselines.md +143 -229
  219. package/pipeline/skills/shared/external/swiftui-animation/SKILL.md +283 -366
  220. package/pipeline/skills/shared/external/swiftui-animation/references/animation-advanced.md +396 -608
  221. package/pipeline/skills/shared/external/swiftui-animation/references/core-animation-bridge.md +336 -385
  222. package/pipeline/skills/shared/external/swiftui-gestures/SKILL.md +239 -349
  223. package/pipeline/skills/shared/external/swiftui-gestures/references/gesture-patterns.md +228 -310
  224. package/pipeline/skills/shared/external/swiftui-layout-components/SKILL.md +260 -249
  225. package/pipeline/skills/shared/external/swiftui-layout-components/references/form.md +92 -74
  226. package/pipeline/skills/shared/external/swiftui-layout-components/references/grids.md +112 -177
  227. package/pipeline/skills/shared/external/swiftui-layout-components/references/list.md +61 -64
  228. package/pipeline/skills/shared/external/swiftui-layout-components/references/scrollview.md +94 -134
  229. package/pipeline/skills/shared/external/swiftui-liquid-glass/SKILL.md +193 -225
  230. package/pipeline/skills/shared/external/swiftui-liquid-glass/references/liquid-glass.md +173 -327
  231. package/pipeline/skills/shared/external/swiftui-navigation/SKILL.md +193 -168
  232. package/pipeline/skills/shared/external/swiftui-navigation/references/deeplinks.md +127 -150
  233. package/pipeline/skills/shared/external/swiftui-navigation/references/navigationstack.md +132 -133
  234. package/pipeline/skills/shared/external/swiftui-navigation/references/sheets.md +152 -117
  235. package/pipeline/skills/shared/external/swiftui-navigation/references/tabview.md +106 -140
  236. package/pipeline/skills/shared/external/swiftui-patterns/SKILL.md +316 -252
  237. package/pipeline/skills/shared/external/swiftui-patterns/references/architecture-patterns.md +341 -332
  238. package/pipeline/skills/shared/external/swiftui-patterns/references/deprecated-migration.md +547 -854
  239. package/pipeline/skills/shared/external/swiftui-patterns/references/design-polish.md +485 -537
  240. package/pipeline/skills/shared/external/swiftui-patterns/references/platform-and-sharing.md +417 -499
  241. package/pipeline/skills/shared/external/swiftui-performance/SKILL.md +213 -376
  242. package/pipeline/skills/shared/external/swiftui-performance/references/demystify-swiftui-performance-wwdc23.md +86 -175
  243. package/pipeline/skills/shared/external/swiftui-performance/references/optimizing-swiftui-performance-instruments.md +89 -195
  244. package/pipeline/skills/shared/external/swiftui-performance/references/understanding-hangs-in-your-app.md +95 -182
  245. package/pipeline/skills/shared/external/swiftui-performance/references/understanding-improving-swiftui-performance.md +71 -149
  246. package/pipeline/skills/shared/external/swiftui-performance/references/wwdc-session-sources.md +21 -27
  247. package/pipeline/skills/shared/external/swiftui-uikit-interop/SKILL.md +303 -295
  248. package/pipeline/skills/shared/external/swiftui-uikit-interop/references/hosting-migration.md +204 -387
  249. package/pipeline/skills/shared/external/swiftui-uikit-interop/references/representable-recipes.md +469 -683
  250. package/pipeline/skills/shared/external/swiftui-webkit/SKILL.md +140 -186
  251. package/pipeline/skills/shared/external/swiftui-webkit/references/loading-and-observation.md +75 -86
  252. package/pipeline/skills/shared/external/swiftui-webkit/references/local-content-and-custom-schemes.md +63 -60
  253. package/pipeline/skills/shared/external/swiftui-webkit/references/migration-and-fallbacks.md +69 -137
  254. package/pipeline/skills/shared/external/swiftui-webkit/references/navigation-and-javascript.md +95 -67
  255. package/pipeline/skills/shared/external/tipkit/SKILL.md +220 -335
  256. package/pipeline/skills/shared/external/tipkit/references/tipkit-patterns.md +356 -494
  257. package/pipeline/skills/shared/external/vision-framework/SKILL.md +260 -375
  258. package/pipeline/skills/shared/external/vision-framework/references/vision-requests.md +393 -515
  259. package/pipeline/skills/shared/external/vision-framework/references/visionkit-scanner.md +363 -539
  260. package/pipeline/skills/shared/external/weatherkit/SKILL.md +152 -310
  261. package/pipeline/skills/shared/external/weatherkit/references/weatherkit-patterns.md +288 -407
  262. package/pipeline/skills/shared/external/widgetkit/SKILL.md +216 -288
  263. package/pipeline/skills/shared/external/widgetkit/references/widgetkit-advanced.md +414 -719
  264. package/pipeline/skills/shared/external/NOTICE-swift-ios-skills.md +0 -39
@@ -1,500 +1,296 @@
1
1
  ---
2
2
  name: speech-recognition
3
- description: "Transcribe speech to text using Apple's Speech framework. Use when implementing live microphone transcription with AVAudioEngine, recognizing recorded audio files, handling speech and microphone authorization, choosing on-device vs server-backed SFSpeechRecognizer behavior, or adopting SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, AssetInventory, and async result streams on iOS 26+."
3
+ description: "Speech framework speech-to-text: live microphone transcription with AVAudioEngine, recorded audio files, speech and microphone permission, on-device versus server SFSpeechRecognizer, iOS 26 SpeechAnalyzer with SpeechTranscriber, DictationTranscriber, SpeechDetector, AssetInventory and async results. Use when transcribing live or recorded speech or migrating to SpeechAnalyzer."
4
4
  metadata:
5
- source: "dpearson2699/swift-ios-skills (PolyForm Perimeter 1.0.0)"
5
+ source: multi-agent-pipeline
6
6
  ---
7
- # Speech Recognition
8
-
9
- Transcribe live and pre-recorded audio to text using Apple's Speech framework.
10
- Covers `SpeechAnalyzer` / `SpeechTranscriber` (iOS 26+) and
11
- `SFSpeechRecognizer` (iOS 10+). Targets Swift 6.3 / iOS 26+ while preserving
12
- fallback guidance for apps that support older OS versions.
13
-
14
- **Scope boundary:** Use this skill for speech-to-text recognition, speech
15
- authorization, microphone capture plumbing, and result handling. Hand off text
16
- analysis, language identification after transcription, sentiment, embeddings,
17
- and translation to `natural-language`; hand off audio playback UI to `avkit`;
18
- hand off summarization or generation over transcripts to `apple-on-device-ai`.
19
-
20
- ## Contents
21
-
22
- - [SpeechAnalyzer Strategy (iOS 26+)](#speechanalyzer-strategy-ios-26)
23
- - [SFSpeechRecognizer Setup](#sfspeechrecognizer-setup)
24
- - [Authorization](#authorization)
25
- - [Live Microphone Transcription](#live-microphone-transcription)
26
- - [Pre-Recorded Audio File Recognition](#pre-recorded-audio-file-recognition)
27
- - [On-Device vs Server Recognition](#on-device-vs-server-recognition)
28
- - [Handling Results](#handling-results)
29
- - [Common Mistakes](#common-mistakes)
30
- - [Review Checklist](#review-checklist)
31
- - [References](#references)
32
-
33
- ## SpeechAnalyzer Strategy (iOS 26+)
34
-
35
- Use `SpeechAnalyzer` for modern iOS 26+ speech analysis, especially long-form
36
- recordings, live transcription, time-indexed transcripts, and fully on-device
37
- flows. Keep `SFSpeechRecognizer` for iOS 10+ deployment targets, server-backed
38
- locale coverage, or existing callback/delegate implementations.
39
-
40
- Read [SpeechAnalyzer patterns](references/speechanalyzer-patterns.md) when
41
- implementing an iOS 26+ transcription pipeline, model asset handling, volatile
42
- results, or file/buffer examples.
43
-
44
- ### SpeechAnalyzer setup checklist
45
-
46
- 1. Choose the module:
47
- - `SpeechTranscriber` for the newer general-purpose on-device model.
48
- - `DictationTranscriber` when `SpeechTranscriber` is unavailable for the
49
- current device or locale and dictation-compatible support is acceptable.
50
- - `SpeechDetector` only in conjunction with a transcriber when voice
51
- activity detection is worth the accuracy/power tradeoff.
52
- 2. Check support before creating the session:
53
- - `SpeechTranscriber.isAvailable`
54
- - `SpeechTranscriber.supportedLocale(equivalentTo:)`
55
- - `SpeechTranscriber.installedLocales` / `supportedLocales` when showing
56
- language choices.
57
- 3. Pick a documented preset:
58
- - `.transcription` for basic accurate transcription.
59
- - `.progressiveTranscription` for live UI updates.
60
- - `.timeIndexedProgressiveTranscription` when playback highlighting needs
61
- `audioTimeRange`.
62
- 4. Install required assets with `AssetInventory.assetInstallationRequest`.
63
- 5. Convert live audio buffers to
64
- `SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith:)` before yielding
65
- `AnalyzerInput`.
66
- 6. Consume module results from their `AsyncSequence` in a separate task.
67
- 7. Finish explicitly with `finalizeAndFinish(through:)`,
68
- `finalizeAndFinishThroughEndOfInput()`, or `cancelAndFinishNow()`.
69
-
70
- Do not use an `offlineTranscription` preset; Apple does not document one.
71
- Finishing an `AsyncStream` input sequence does not finish the analyzer session.
72
-
73
- ## SFSpeechRecognizer Setup
74
-
75
- ### Creating a recognizer with locale
7
+
8
+ # Speech recognition
9
+
10
+ The Speech framework has two generations. `SpeechAnalyzer` with
11
+ `SpeechTranscriber` arrived in iOS 26 and runs entirely on device with async
12
+ result streams. `SFSpeechRecognizer` dates back to iOS 10 and uses callbacks.
13
+ Code here assumes Swift 6.3 and an iOS 26 deployment target, with notes for
14
+ older targets.
15
+
16
+ ## Scope
17
+
18
+ This skill covers speech permissions, microphone capture plumbing,
19
+ transcription and result handling. Hand off:
20
+
21
+ | Need | Skill |
22
+ |---|---|
23
+ | Analyzing the transcript: language, sentiment, entities, embeddings, translation | `natural-language` |
24
+ | Playback UI and captions | `avkit` |
25
+ | Summaries or other generation over a transcript | `apple-on-device-ai` |
26
+
27
+ ## Which API
28
+
29
+ Choose `SpeechAnalyzer` on iOS 26+ for long recordings, live transcription,
30
+ transcripts indexed by time, and flows that must stay on the device.
31
+
32
+ Keep `SFSpeechRecognizer` when the app still supports older systems, when a
33
+ locale is only covered by server recognition, or when a working
34
+ callback-based implementation does not need to change yet.
35
+
36
+ ### SpeechAnalyzer checklist
37
+
38
+ 1. **Pick the module.** `SpeechTranscriber` is the newer general-purpose
39
+ on-device model. Fall back to `DictationTranscriber` when the transcriber
40
+ is unavailable for the device or locale and dictation-style output is
41
+ acceptable. Add `SpeechDetector` (voice activity detection) only alongside
42
+ a transcriber, and only if the power saving is worth the risk to accuracy.
43
+ 2. **Check support.** `SpeechTranscriber.isAvailable` and
44
+ `SpeechTranscriber.supportedLocale(equivalentTo:)`; language pickers read
45
+ `installedLocales` and `supportedLocales`.
46
+ 3. **Pick a preset.** `.transcription` for plain accurate text,
47
+ `.progressiveTranscription` for live UI, `.timeIndexedProgressiveTranscription`
48
+ when playback should highlight words through `audioTimeRange`.
49
+ 4. **Install assets** with `AssetInventory.assetInstallationRequest(supporting:)`.
50
+ 5. **Convert live audio** to `SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith:)`
51
+ before yielding `AnalyzerInput` values.
52
+ 6. **Read results** from the module's async sequence in a separate task.
53
+ 7. **Finish on purpose** with `finalizeAndFinish(through:)`,
54
+ `finalizeAndFinishThroughEndOfInput()` or `cancelAndFinishNow()`.
55
+
56
+ There is no `.offlineTranscription` preset, whatever some samples claim.
57
+ Closing your `AsyncStream` of input does not finish the analyzer either.
58
+ Full code is in [analyzer recipes](references/speechanalyzer-patterns.md).
59
+
60
+ ## SFSpeechRecognizer setup
76
61
 
77
62
  ```swift
78
63
  import Speech
79
64
 
80
- // Default locale (user's current language)
81
- let recognizer = SFSpeechRecognizer()
65
+ let deviceLocale = SFSpeechRecognizer() // current locale
66
+ let british = SFSpeechRecognizer(locale: Locale(identifier: "en-GB")) // specific locale
82
67
 
83
- // Specific locale
84
- let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))
85
-
86
- // Check if recognition is available for this locale
87
- guard let recognizer, recognizer.isAvailable else {
88
- print("Speech recognition not available")
68
+ guard let recognizer = british, recognizer.isAvailable else {
69
+ // show "speech input unavailable"
89
70
  return
90
71
  }
91
72
  ```
92
73
 
93
- ### Monitoring availability changes
74
+ The initializer is failable. Availability also changes at runtime (network
75
+ loss, locale switch), so adopt `SFSpeechRecognizerDelegate`. The callback runs
76
+ on the recognizer's `queue`, the main queue unless you change it, so a
77
+ main-actor model can conform with `@preconcurrency`; without it Swift 6
78
+ rejects the conformance as crossing into main-actor code.
94
79
 
95
80
  ```swift
96
- final class SpeechManager: NSObject, SFSpeechRecognizerDelegate {
97
- private let recognizer = SFSpeechRecognizer()!
98
-
99
- override init() {
100
- super.init()
101
- recognizer.delegate = self
102
- }
81
+ @MainActor
82
+ final class DictationModel: NSObject, @preconcurrency SFSpeechRecognizerDelegate {
83
+ var recordButtonEnabled = false
103
84
 
104
- func speechRecognizer(
105
- _ speechRecognizer: SFSpeechRecognizer,
106
- availabilityDidChange available: Bool
107
- ) {
108
- // Update UI - disable record button when unavailable
85
+ func speechRecognizer(_ source: SFSpeechRecognizer, availabilityDidChange isUsable: Bool) {
86
+ recordButtonEnabled = isUsable
109
87
  }
110
88
  }
111
89
  ```
112
90
 
113
91
  ## Authorization
114
92
 
115
- Request **both** speech recognition and microphone permissions before starting
116
- live transcription. Add these keys to `Info.plist`:
93
+ Live transcription needs **two** permissions: speech recognition and the
94
+ microphone. Add both Info.plist strings:
117
95
 
118
96
  - `NSSpeechRecognitionUsageDescription`
119
97
  - `NSMicrophoneUsageDescription`
120
98
 
121
99
  ```swift
122
- import Speech
123
- import AVFoundation
124
-
125
- func requestPermissions() async -> Bool {
126
- let speechStatus = await withCheckedContinuation { continuation in
100
+ func speechAllowed() async -> Bool {
101
+ await withCheckedContinuation { continuation in
127
102
  SFSpeechRecognizer.requestAuthorization { status in
128
- continuation.resume(returning: status)
129
- }
130
- }
131
- guard speechStatus == .authorized else { return false }
132
-
133
- let micStatus: Bool
134
- if #available(iOS 17, *) {
135
- micStatus = await AVAudioApplication.requestRecordPermission()
136
- } else {
137
- micStatus = await withCheckedContinuation { continuation in
138
- AVAudioSession.sharedInstance().requestRecordPermission { granted in
139
- continuation.resume(returning: granted)
140
- }
103
+ continuation.resume(returning: status == .authorized)
141
104
  }
142
105
  }
143
- return micStatus
106
+ }
107
+
108
+ func microphoneAllowed() async -> Bool {
109
+ await AVAudioApplication.requestRecordPermission() // iOS 17+
144
110
  }
145
111
  ```
146
112
 
147
- ## Live Microphone Transcription
113
+ Before iOS 17, ask through `AVAudioSession.sharedInstance().requestRecordPermission(_:)`.
148
114
 
149
- The standard pattern: `AVAudioEngine` captures microphone audio → buffers are
150
- appended to `SFSpeechAudioBufferRecognitionRequest` → results stream in.
115
+ ## Live microphone transcription
116
+
117
+ `AVAudioEngine` captures audio, a tap appends each buffer to an
118
+ `SFSpeechAudioBufferRecognitionRequest`, and the recognition task reports
119
+ results.
151
120
 
152
121
  ```swift
153
- import Speech
154
122
  import AVFoundation
123
+ import Speech
155
124
 
156
- final class LiveTranscriber {
157
- private let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!
158
- private let audioEngine = AVAudioEngine()
159
- private var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?
160
- private var recognitionTask: SFSpeechRecognitionTask?
125
+ @MainActor
126
+ final class LiveCaptioner {
127
+ private let engine = AVAudioEngine()
128
+ private let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-AU"))
129
+ private var request: SFSpeechAudioBufferRecognitionRequest?
130
+ private var task: SFSpeechRecognitionTask?
131
+ private(set) var caption = ""
161
132
 
162
- func startTranscribing() throws {
163
- // Cancel any in-progress task
164
- recognitionTask?.cancel()
165
- recognitionTask = nil
133
+ func begin() throws {
134
+ task?.cancel()
135
+ task = nil
166
136
 
167
- // Configure audio session
168
- let audioSession = AVAudioSession.sharedInstance()
169
- try audioSession.setCategory(.record, mode: .measurement, options: .duckOthers)
170
- try audioSession.setActive(true, options: .notifyOthersOnDeactivation)
137
+ let session = AVAudioSession.sharedInstance()
138
+ try session.setCategory(.record, mode: .measurement, options: [.duckOthers])
139
+ try session.setActive(true, options: .notifyOthersOnDeactivation)
171
140
 
172
- // Create request
173
141
  let request = SFSpeechAudioBufferRecognitionRequest()
174
142
  request.shouldReportPartialResults = true
175
- self.recognitionRequest = request
176
-
177
- // Start recognition task
178
- recognitionTask = recognizer.recognitionTask(with: request) { result, error in
179
- if let result {
180
- let text = result.bestTranscription.formattedString
181
- print("Transcription: \(text)")
182
-
183
- if result.isFinal {
184
- self.stopTranscribing()
185
- }
186
- }
187
- if let error {
188
- print("Recognition error: \(error)")
189
- self.stopTranscribing()
143
+ self.request = request
144
+
145
+ task = recognizer?.recognitionTask(with: request) { [weak self] result, error in
146
+ let text = result?.bestTranscription.formattedString
147
+ let done = error != nil || (result?.isFinal ?? false)
148
+ Task { @MainActor in
149
+ guard let self else { return }
150
+ if let text { self.caption = text }
151
+ if done { self.end() }
190
152
  }
191
153
  }
192
154
 
193
- // Install audio tap
194
- let inputNode = audioEngine.inputNode
195
- let recordingFormat = inputNode.outputFormat(forBus: 0)
196
- inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) {
197
- buffer, _ in
155
+ feedMicrophone(engine.inputNode, into: request)
156
+ engine.prepare()
157
+ try engine.start()
158
+ }
159
+
160
+ // nonisolated so the tap closure is not main-actor isolated: it runs on the audio thread
161
+ private nonisolated func feedMicrophone(_ mic: AVAudioInputNode,
162
+ into request: SFSpeechAudioBufferRecognitionRequest) {
163
+ let native = mic.outputFormat(forBus: 0)
164
+ mic.installTap(onBus: 0, bufferSize: 2048, format: native) { buffer, _ in
198
165
  request.append(buffer)
199
166
  }
200
-
201
- audioEngine.prepare()
202
- try audioEngine.start()
203
167
  }
204
168
 
205
- func stopTranscribing() {
206
- audioEngine.stop()
207
- audioEngine.inputNode.removeTap(onBus: 0)
208
- recognitionRequest?.endAudio()
209
- recognitionRequest = nil
210
- recognitionTask?.cancel()
211
- recognitionTask = nil
169
+ func end() {
170
+ engine.stop()
171
+ engine.inputNode.removeTap(onBus: 0)
172
+ request?.endAudio()
173
+ request = nil
174
+ task?.cancel()
175
+ task = nil
212
176
  }
213
177
  }
214
178
  ```
215
179
 
216
- ## Pre-Recorded Audio File Recognition
180
+ The tap block runs on a real-time audio thread. Under Swift 6 a closure
181
+ written inside a `@MainActor` method inherits that isolation and traps when
182
+ called elsewhere, which is why the tap is installed from a `nonisolated`
183
+ helper.
217
184
 
218
- Use `SFSpeechURLRecognitionRequest` for audio files on disk:
185
+ ## Transcribing a recorded file
219
186
 
220
187
  ```swift
221
- func transcribeFile(at url: URL) async throws -> String {
222
- guard let recognizer = SFSpeechRecognizer(), recognizer.isAvailable else {
223
- throw SpeechError.unavailable
224
- }
188
+ func transcribeFile(at url: URL, with recognizer: SFSpeechRecognizer) async throws -> String {
225
189
  let request = SFSpeechURLRecognitionRequest(url: url)
226
190
  request.shouldReportPartialResults = false
227
191
 
228
- return try await withCheckedThrowingContinuation { continuation in
229
- var didResume = false
230
- recognizer.recognitionTask(with: request) { result, error in
231
- guard !didResume else { return }
232
- if let error {
233
- didResume = true
234
- continuation.resume(throwing: error)
235
- } else if let result, result.isFinal {
236
- didResume = true
237
- continuation.resume(
238
- returning: result.bestTranscription.formattedString
239
- )
192
+ let text: String = try await withCheckedThrowingContinuation { continuation in
193
+ var settled = false
194
+ recognizer.recognitionTask(with: request) { outcome, failure in
195
+ guard !settled else { return }
196
+ if let failure {
197
+ settled = true
198
+ continuation.resume(throwing: failure)
199
+ } else if let outcome, outcome.isFinal {
200
+ settled = true
201
+ continuation.resume(returning: outcome.bestTranscription.formattedString)
240
202
  }
241
203
  }
242
204
  }
205
+ return text
243
206
  }
244
207
  ```
245
208
 
246
- ## On-Device vs Server Recognition
247
-
248
- `SFSpeechRecognizer` can use on-device recognition for supported locales on
249
- iOS 13+. If `supportsOnDeviceRecognition` is false, the recognizer requires a
250
- network connection. `requiresOnDeviceRecognition` only has effect when the
251
- recognizer supports it.
209
+ The `settled` flag stops a second resume if the handler fires again.
252
210
 
253
- ```swift
254
- let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!
211
+ ## On device or on the server
255
212
 
256
- // Check if on-device is supported for this locale
257
- if recognizer.supportsOnDeviceRecognition {
258
- let request = SFSpeechAudioBufferRecognitionRequest()
259
- request.requiresOnDeviceRecognition = true // Force on-device
260
- }
261
- ```
262
-
263
- `SFSpeechRecognizer` requests may still be a poor fit for long-form capture.
264
- Apple documents a roughly one-minute task limit for speech recognition and
265
- other service limits. For long recordings on iOS 26+, prefer `SpeechAnalyzer`;
266
- otherwise chunk or restart recognition before the limit and preserve transcript
267
- state across tasks.
268
-
269
- ## Handling Results
270
-
271
- ### Partial vs final results
213
+ - Since iOS 13 many locales can be recognized on the device. When
214
+ `supportsOnDeviceRecognition` is `false`, recognition needs the network.
215
+ - `requiresOnDeviceRecognition = true` is only honoured when the recognizer
216
+ supports it, so check first:
272
217
 
273
218
  ```swift
274
- let request = SFSpeechAudioBufferRecognitionRequest()
275
- request.shouldReportPartialResults = true // default is true
276
-
277
- recognizer.recognitionTask(with: request) { result, error in
278
- guard let result else { return }
279
-
280
- if result.isFinal {
281
- // Final transcription - recognition is complete
282
- let final = result.bestTranscription.formattedString
283
- } else {
284
- // Partial result - may change as more audio is processed
285
- let partial = result.bestTranscription.formattedString
286
- }
287
- }
288
- ```
289
-
290
- ### Accessing alternative transcriptions and confidence
291
-
292
- ```swift
293
- recognizer.recognitionTask(with: request) { result, error in
294
- guard let result else { return }
295
-
296
- // Best transcription
297
- let best = result.bestTranscription
298
-
299
- // All alternatives (sorted by confidence, descending)
300
- for transcription in result.transcriptions {
301
- for segment in transcription.segments {
302
- print("\(segment.substring): \(segment.confidence)")
303
- }
304
- }
305
- }
306
- ```
307
-
308
- ### Adding punctuation (iOS 16+)
309
-
310
- ```swift
311
- let request = SFSpeechAudioBufferRecognitionRequest()
312
- request.addsPunctuation = true
313
- ```
314
-
315
- ### Contextual strings
316
-
317
- Improve recognition of domain-specific terms:
318
-
319
- ```swift
320
- let request = SFSpeechAudioBufferRecognitionRequest()
321
- request.contextualStrings = ["SwiftUI", "Xcode", "CloudKit"]
322
- ```
323
-
324
- ## Common Mistakes
325
-
326
- ### Not requesting both speech and microphone authorization
327
-
328
- ```swift
329
- // ❌ DON'T: Only request speech authorization for live audio
330
- SFSpeechRecognizer.requestAuthorization { status in
331
- // Missing microphone permission - audio engine will fail
332
- self.startRecording()
333
- }
334
-
335
- // ✅ DO: Request both permissions before recording
336
- SFSpeechRecognizer.requestAuthorization { status in
337
- guard status == .authorized else { return }
338
- AVAudioSession.sharedInstance().requestRecordPermission { granted in
339
- guard granted else { return }
340
- self.startRecording()
341
- }
342
- }
343
- ```
344
-
345
- ### Not handling availability changes
346
-
347
- ```swift
348
- // ❌ DON'T: Assume recognizer stays available after initial check
349
- let recognizer = SFSpeechRecognizer()!
350
- // Recognition may fail if network drops or locale changes
351
-
352
- // ✅ DO: Monitor availability via delegate
353
- recognizer.delegate = self
354
- func speechRecognizer(
355
- _ speechRecognizer: SFSpeechRecognizer,
356
- availabilityDidChange available: Bool
357
- ) {
358
- recordButton.isEnabled = available
359
- }
360
- ```
361
-
362
- ### Not stopping the audio engine when recognition ends
363
-
364
- ```swift
365
- // ❌ DON'T: Leave audio engine running after recognition finishes
366
- recognizer.recognitionTask(with: request) { result, error in
367
- if result?.isFinal == true {
368
- // Audio engine still running, wasting resources and battery
369
- }
370
- }
371
-
372
- // ✅ DO: Clean up all audio resources
373
- recognizer.recognitionTask(with: request) { result, error in
374
- if result?.isFinal == true || error != nil {
375
- self.audioEngine.stop()
376
- self.audioEngine.inputNode.removeTap(onBus: 0)
377
- self.recognitionRequest?.endAudio()
378
- self.recognitionRequest = nil
379
- }
380
- }
381
- ```
382
-
383
- ### Assuming on-device recognition is available for all locales
384
-
385
- ```swift
386
- // ❌ DON'T: Force on-device without checking support
387
- let request = SFSpeechAudioBufferRecognitionRequest()
388
- request.requiresOnDeviceRecognition = true // Ignored unless the recognizer supports it
389
-
390
- // ✅ DO: Check support before requiring on-device
391
219
  if recognizer.supportsOnDeviceRecognition {
392
220
  request.requiresOnDeviceRecognition = true
393
221
  } else {
394
- // Fall back to server-based or inform user
395
- }
396
- ```
397
-
398
- ### Not handling the one-minute recognition limit
399
-
400
- ```swift
401
- // ❌ DON'T: Start one long continuous recognition session
402
- func startRecording() {
403
- // SFSpeechRecognizer tasks can be cut off after about 60 seconds
404
- }
405
-
406
- // ✅ DO: roll the segment before the limit and let cleanup end audio once
407
- func scheduleRecognitionRollover() {
408
- recognitionTimer = Timer.scheduledTimer(withTimeInterval: 55, repeats: false) { [weak self] _ in
409
- self?.commitLatestPartialText()
410
- self?.stopTranscribing() // owns endAudio(), tap removal, and task cancellation
411
- try? self?.startTranscribing()
412
- }
413
- }
414
- ```
415
- `SFSpeechRecognitionTask` exposes `finish()`, `cancel()`, `state`, and `error`;
416
- do not invent task properties such as `recognitionTask` to restart work. Keep
417
- the active `SFSpeechAudioBufferRecognitionRequest` in your manager and call
418
- `endAudio()` from one cleanup path only.
419
-
420
- ### Treating SpeechAnalyzer input completion as session completion
421
-
422
- ```swift
423
- // ❌ DON'T: Only finish the AsyncStream and expect result streams to close
424
- inputBuilder.finish()
425
-
426
- // ✅ DO: explicitly finish or cancel the analyzer session
427
- let lastSampleTime = try await analyzer.analyzeSequence(inputSequence)
428
- if let lastSampleTime {
429
- try await analyzer.finalizeAndFinish(through: lastSampleTime)
430
- } else {
431
- try analyzer.cancelAndFinishNow()
432
- }
433
- ```
434
-
435
- ### Duplicating volatile SpeechAnalyzer results
436
-
437
- ```swift
438
- // ✅ Replace volatile text with the finalized result for the same audio range
439
- for try await result in transcriber.results {
440
- if result.isFinal {
441
- volatileTranscript = AttributedString()
442
- finalizedTranscript.append(result.text)
443
- } else {
444
- volatileTranscript = result.text
445
- }
446
- }
447
- ```
448
-
449
- ### Creating multiple simultaneous recognition tasks
450
-
451
- ```swift
452
- // ❌ DON'T: Start a new task without canceling the previous one
453
- func startRecording() {
454
- recognitionTask = recognizer.recognitionTask(with: request) { ... }
455
- // Previous task is still running - undefined behavior
456
- }
457
-
458
- // ✅ DO: Cancel existing task before creating a new one
459
- func startRecording() {
460
- recognitionTask?.cancel()
461
- recognitionTask = nil
462
- recognitionTask = recognizer.recognitionTask(with: request) { ... }
222
+ // tell the user audio will be sent for recognition, or pick another locale
463
223
  }
464
224
  ```
465
225
 
466
- ## Review Checklist
467
-
468
- - [ ] `NSSpeechRecognitionUsageDescription` is in Info.plist
469
- - [ ] `NSMicrophoneUsageDescription` is in Info.plist (if using live audio)
470
- - [ ] Authorization is requested before starting recognition
471
- - [ ] `SFSpeechRecognizerDelegate` is set to handle `availabilityDidChange`
472
- - [ ] Audio engine is stopped and tap removed when recognition ends
473
- - [ ] `recognitionRequest.endAudio()` is called when done recording
474
- - [ ] Previous `recognitionTask` is canceled before starting a new one
475
- - [ ] `supportsOnDeviceRecognition` is checked before requiring on-device mode
476
- - [ ] Partial results are handled separately from final (`isFinal`) results
477
- - [ ] `SFSpeechRecognizer` one-minute/service limits are accounted for
478
- - [ ] For iOS 26+: `AssetInventory` assets are installed before using `SpeechAnalyzer`
479
- - [ ] For iOS 26+: `SpeechTranscriber.isAvailable` and locale support are checked
480
- - [ ] For iOS 26+: live buffers are converted to the analyzer-compatible format
481
- - [ ] For iOS 26+: analyzer sessions are explicitly finalized or canceled
482
- - [ ] For iOS 26+: volatile results are replaced by finalized results, not duplicated
226
+ - Server-backed recognition has limits, including roughly one minute of
227
+ audio per task. On iOS 26+ prefer `SpeechAnalyzer` for long recordings;
228
+ otherwise split the audio and restart before the limit, carrying the
229
+ transcript forward.
230
+
231
+ ## Results
232
+
233
+ - `shouldReportPartialResults` defaults to `true`. Partial text can still
234
+ change; `isFinal` marks the last result of a task.
235
+ - `result.transcriptions` holds alternatives, most confident first. Each
236
+ transcription's `segments` expose `substring` and `confidence`.
237
+ - `request.addsPunctuation = true` (iOS 16+) inserts punctuation.
238
+ - `request.contextualStrings = ["Kubernetes", "Grafana"]` nudges the
239
+ recognizer toward domain vocabulary.
240
+
241
+ ## Common mistakes
242
+
243
+ - **Asking only for speech permission.** Live audio also needs the
244
+ microphone, or the engine fails to start. Request both before recording.
245
+ - **Ignoring availability changes.** Watch the delegate; network loss or a
246
+ locale change can make the recognizer unavailable.
247
+ - **Leaving the engine running.** On a final result or an error, stop the
248
+ engine, remove the tap and end the audio.
249
+ - **Forcing on-device without support.** The flag is ignored; check
250
+ `supportsOnDeviceRecognition` and fall back or inform the user.
251
+ - **Running into the one-minute ceiling.** Roll over before it, for example
252
+ with a timer at 55 seconds: keep the latest partial text, stop through the
253
+ single cleanup path (the only place that calls `endAudio()`, removes the tap
254
+ and cancels), then start a fresh task.
255
+ - **Inventing task APIs.** `SFSpeechRecognitionTask` offers `finish()`,
256
+ `cancel()`, `state` and `error`, nothing to restart itself. Hold the live
257
+ request as a property of the owning object, and let exactly one code path
258
+ call `endAudio()`.
259
+ - **Assuming the analyzer finishes when input ends.** Finishing your input
260
+ builder does not close result streams. After `analyzeSequence(_:)` returns
261
+ the last sample time, call `finalizeAndFinish(through:)`; if there is none,
262
+ `cancelAndFinishNow()`.
263
+ - **Duplicating volatile text.** When a result is final, clear the volatile
264
+ part and append the final text; otherwise replace the volatile part.
265
+ - **Overlapping tasks.** Cancel the previous recognition task before creating
266
+ a new one; concurrent tasks behave unpredictably.
267
+
268
+ ## Review checklist
269
+
270
+ - [ ] `NSSpeechRecognitionUsageDescription` present, plus `NSMicrophoneUsageDescription` for live audio
271
+ - [ ] Authorization requested before any recognition
272
+ - [ ] Delegate reacts to `availabilityDidChange`
273
+ - [ ] Engine stopped, tap removed and `endAudio()` called when finished
274
+ - [ ] Previous task cancelled before a new one starts
275
+ - [ ] `supportsOnDeviceRecognition` checked before requiring on-device
276
+ - [ ] Partial and final results handled differently
277
+ - [ ] Task duration and service limits planned for
278
+ - [ ] iOS 26+: assets installed through `AssetInventory`, `isAvailable` and locale support checked, live buffers converted, sessions finalized or cancelled, volatile text replaced rather than duplicated
483
279
 
484
280
  ## References
485
281
 
486
- - [Speech framework](https://sosumi.ai/documentation/speech)
487
- - [SpeechAnalyzer](https://sosumi.ai/documentation/speech/speechanalyzer)
488
- - [SpeechTranscriber](https://sosumi.ai/documentation/speech/speechtranscriber)
489
- - [SpeechTranscriber.Preset](https://sosumi.ai/documentation/speech/speechtranscriber/preset)
490
- - [DictationTranscriber](https://sosumi.ai/documentation/speech/dictationtranscriber)
491
- - [SpeechDetector](https://sosumi.ai/documentation/speech/speechdetector)
492
- - [SFSpeechRecognizer](https://sosumi.ai/documentation/speech/sfspeechrecognizer)
493
- - [SFSpeechAudioBufferRecognitionRequest](https://sosumi.ai/documentation/speech/sfspeechaudiobufferrecognitionrequest)
494
- - [SFSpeechURLRecognitionRequest](https://sosumi.ai/documentation/speech/sfspeechurlrecognitionrequest)
495
- - [SFSpeechRecognitionResult](https://sosumi.ai/documentation/speech/sfspeechrecognitionresult)
496
- - [SFSpeechRecognitionRequest](https://sosumi.ai/documentation/speech/sfspeechrecognitionrequest)
497
- - [AssetInventory](https://sosumi.ai/documentation/speech/assetinventory)
498
- - [Asking Permission to Use Speech Recognition](https://sosumi.ai/documentation/speech/asking-permission-to-use-speech-recognition)
499
- - [Recognizing Speech in Live Audio](https://sosumi.ai/documentation/speech/recognizing-speech-in-live-audio)
500
- - [Bring advanced speech-to-text to your app with SpeechAnalyzer](https://sosumi.ai/videos/play/wwdc2025/277)
282
+ - [analyzer recipes](references/speechanalyzer-patterns.md)
283
+ - https://developer.apple.com/documentation/speech
284
+ - https://developer.apple.com/documentation/speech/speechanalyzer
285
+ - https://developer.apple.com/documentation/speech/speechtranscriber
286
+ - https://developer.apple.com/documentation/speech/speechtranscriber/preset
287
+ - https://developer.apple.com/documentation/speech/dictationtranscriber
288
+ - https://developer.apple.com/documentation/speech/speechdetector
289
+ - https://developer.apple.com/documentation/speech/assetinventory
290
+ - https://developer.apple.com/documentation/speech/sfspeechrecognizer
291
+ - https://developer.apple.com/documentation/speech/sfspeechrecognitionrequest
292
+ - https://developer.apple.com/documentation/speech/sfspeechaudiobufferrecognitionrequest
293
+ - https://developer.apple.com/documentation/speech/sfspeechurlrecognitionrequest
294
+ - https://developer.apple.com/documentation/speech/sfspeechrecognitionresult
295
+ - Apple articles on requesting speech-recognition permission and on recognizing live audio, in the Speech documentation
296
+ - WWDC25 session 277 (SpeechAnalyzer introduction)