@mmerterden/multi-agent-pipeline 20.6.0 → 20.8.0

package/pipeline/skills/shared/external/ios-networking/evals/evals.json

@@ -4,46 +4,46 @@
     {
       "id": 1,
       "name": "urlsession-api-client-review",
-      "prompt": "Review this Swift API client plan: every request uses URLSession.shared.data(for:) and decodes without checking HTTP status, retries every error including cancellation and all 4xx responses, stores bearer tokens in UserDefaults, and tests by mocking the APIClient protocol only. Give a concise correction plan with Swift where useful.",
-      "expected_output": "A URLSession API client review that fixes response validation, retry policy, cancellation, token handling boundaries, and test strategy.",
+      "prompt": "Our new API layer calls URLSession.shared.data(for:) from every service, decodes the body straight away without looking at the status code, wraps every call in a retry loop that retries on any error (cancellation and every 4xx included), keeps the bearer token in UserDefaults, and our tests only replace the APIClient protocol with a fake. Give me a short correction plan with Swift snippets.",
+      "expected_output": "A correction plan covering status validation before decoding, a retry policy that skips cancellation and client errors, a configured injected session, token handling through middleware with Keychain storage handed to the security boundary, and transport-level testing.",
       "files": [],
       "assertions": [
-        "Requires validating HTTPURLResponse status codes before decoding and treating 4xx/5xx separately from transport failures.",
-        "Recommends a configured URLSession for production rather than URLSession.shared when timeouts, caching, delegates, or connectivity behavior matter.",
-        "Excludes cancellation and most 4xx client errors from retry logic while allowing appropriate retries such as 429 or 5xx.",
-        "Uses request middleware or a centralized request builder for auth headers rather than hardcoded tokens.",
-        "States that sensitive tokens belong in Keychain and keeps deep credential-storage implementation in the security boundary.",
-        "Mentions URLProtocol or transport-level testing when request/response behavior needs verification."
+        "Validates the HTTPURLResponse status before decoding and treats 4xx and 5xx responses separately from transport failures",
+        "Replaces URLSession.shared with a configured, injected URLSession whenever the app needs its own timeouts, cache, delegate or offline waiting",
+        "Excludes cancellation and most 4xx responses from retry, while allowing 429 and 5xx",
+        "Adds auth headers through middleware or one central request builder rather than per request",
+        "Moves tokens to the Keychain and leaves credential-storage internals to the security boundary",
+        "Recommends URLProtocol-based or other transport-level tests to verify requests and responses"
       ]
     },
     {
       "id": 2,
       "name": "background-transfer-websocket-boundary",
-      "prompt": "A team wants one networking manager that uses a background URLSession for JSON polling, large file downloads, file uploads from memory, and a chat WebSocket that must keep receiving after the app is force-quit. Review the plan and say what should change.",
-      "expected_output": "A review that separates background URLSession transfer support from foreground WebSocket behavior and corrects force-quit/background limitations.",
+      "prompt": "I want one manager built on a background URLSession that polls a JSON endpoint every minute, downloads large video files, uploads photos straight from Data in memory, and also runs our chat WebSocket, which should stay connected even once the user swipes the app away in the switcher. What should change?",
+      "expected_output": "Separates background HTTP transfers from foreground WebSocket work and corrects the limits of background sessions and force-quit.",
       "files": [],
       "assertions": [
-        "States that background URLSession is for HTTP/HTTPS upload and download transfers, not WebSocket durability.",
-        "States that background uploads must be file-backed rather than Data or stream uploads after app exit.",
-        "Says background sessions require delegates and task APIs rather than async convenience APIs as the durable transfer pattern.",
-        "Explains that user force-quit cancels background transfers and prevents automatic relaunch until the user opens the app.",
-        "Requires moving or opening a downloaded temporary file before didFinishDownloadingTo returns.",
-        "Keeps WebSocket guidance to foreground/default-session realtime networking with reconnect when the app is active again."
+        "States that a background URLSession is for HTTP and HTTPS uploads and downloads, not for keeping a WebSocket alive",
+        "Requires background uploads to come from a file, not from Data or a stream",
+        "Uses delegates and task APIs on the background session instead of the async conveniences",
+        "Explains that swiping the app away cancels its background transfers, and that the system will not relaunch it for them until it is opened manually",
+        "Moves or opens the downloaded file before didFinishDownloadingTo returns",
+        "Limits the WebSocket to foreground, default-session use and reconnects when the app becomes active"
       ]
     },
     {
       "id": 3,
       "name": "network-framework-ats-security-boundary",
-      "prompt": "Review this iOS 26 networking design: use Network.framework for all REST calls to avoid ATS, disable NSAllowsArbitraryLoads globally, implement certificate pinning by hashing SecKeyCopyExternalRepresentation bytes, wrap NWPathMonitor in a custom AsyncStream helper, and sketch NetworkConnection<QUIC> stream work as synchronous callbacks. What should change, and which parts belong to neighboring skills?",
-      "expected_output": "A boundary-aware review that routes REST to URLSession/ATS, low-level networking to Network.framework, certificate trust details to swift-security, uses modern NWPathMonitor guidance, and treats iOS 26 NetworkConnection QUIC stream APIs correctly.",
+      "prompt": "For an iOS 26 app we plan to send all REST traffic over Network.framework so ATS does not apply, set NSAllowsArbitraryLoads to true for the whole app, pin certificates by hashing the bytes from SecKeyCopyExternalRepresentation, wrap NWPathMonitor in our own AsyncStream, and treat NetworkConnection<QUIC> stream opening as plain synchronous callbacks. What needs to change, and which pieces should another skill own?",
+      "expected_output": "Routes REST back to URLSession under ATS, keeps Network.framework for low-level work, sends trust and pinning work to swift-security, iterates NWPathMonitor directly, and treats QUIC stream calls as async throwing.",
       "files": [],
       "assertions": [
-        "Recommends URLSession for ordinary REST/HTTP API calls and Network.framework only for lower-level TCP, UDP, listeners, Bonjour, path monitoring, WebSocket protocol work, or iOS 26 NetworkConnection needs.",
-        "States that ATS applies to URL Loading/URLSession and that Network.framework callers must take responsibility for secure configuration.",
-        "Rejects global NSAllowsArbitraryLoads as a default and recommends fixing servers or using the narrowest justified ATS exception.",
-        "Corrects raw SecKeyCopyExternalRepresentation hashing as not sufficient for SPKI pinning and routes deep certificate-trust implementation to swift-security.",
-        "Uses NWPathMonitor as AsyncSequence on current OS targets or explicitly labels a wrapper as fallback/custom projection.",
-        "For NetworkConnection QUIC streams, treats openStream and inboundStreams as async throwing APIs."
+        "Keeps ordinary REST on URLSession and limits Network.framework to raw TCP or UDP, listeners, Bonjour discovery, path monitoring, protocol-level WebSocket work, or NetworkConnection on iOS 26",
+        "Explains that ATS is enforced by the URL Loading System (URLSession) only, so code using Network.framework has to set up TLS and trust on its own",
+        "Rejects a global NSAllowsArbitraryLoads and asks for HTTPS fixes or the narrowest justified exception",
+        "States that hashing raw SecKeyCopyExternalRepresentation output is not SPKI pinning and routes trust implementation to swift-security",
+        "Iterates NWPathMonitor as an AsyncSequence on current SDKs, or labels a custom wrapper as a fallback or projection",
+        "Treats openStream and inboundStreams on NetworkConnection<QUIC> as async throwing calls"
       ]
     }
   ]