@mmerterden/multi-agent-pipeline 20.7.0 → 20.8.0

package/pipeline/skills/shared/external/core-bluetooth/evals/evals.json

@@ -3,42 +3,45 @@
   "evals": [
     {
       "id": 1,
-      "prompt": "I'm building an iOS heart-rate monitor screen that scans for service 180D, connects, subscribes to 2A37, parses the BPM, and sends a small command to a writable characteristic. Show the Core Bluetooth manager code and the safety checks I should include.",
-      "expected_output": "Guidance creates a CBCentralManager, waits for poweredOn, checks Bluetooth authorization, scans with service UUID 180D, retains the discovered peripheral, sets the peripheral delegate before discovery, subscribes to notify characteristics, parses heart-rate data with byte-count guards, and chooses write types using characteristic properties plus maximumWriteValueLength and canSendWriteWithoutResponse.",
+      "name": "central-heart-rate-review",
+      "prompt": "Our fitness app creates a CBCentralManager at launch and immediately calls scanForPeripherals(withServices: nil). In didDiscover we connect to the first device without storing it, then call discoverServices before didConnect. We read heart rate as Int(data[1]) and write config blobs with .withoutResponse whenever we like. Review it.",
+      "expected_output": "Checks authorization and waits for .poweredOn, scans for 180D instead of nil, keeps a strong reference and sets the delegate before discovery in didConnect, parses heart rate using the flags byte with length guards for 8- and 16-bit values, and gates writes on properties, maximumWriteValueLength(for:) and canSendWriteWithoutResponse.",
       "files": [],
       "assertions": [
-        "The output checks Bluetooth authorization and gates BLE operations on the manager reaching poweredOn.",
-        "The output scans for service UUID 180D rather than using a nil production scan.",
-        "The output retains the discovered CBPeripheral and sets its delegate before service discovery.",
-        "The output parses heart-rate measurement data with explicit guards for the flags byte and 16-bit BPM payload length.",
-        "The output checks characteristic write properties, maximumWriteValueLength(for:), and canSendWriteWithoutResponse before using writeValue."
+        "Checks manager authorization and only scans after state becomes .poweredOn",
+        "Scans for the heart rate service 180D rather than nil",
+        "Retains the discovered peripheral and sets its delegate before discovering services in didConnect",
+        "Parses the measurement using flag bit 0 for 8-bit versus 16-bit values with byte-count guards",
+        "Checks characteristic properties, maximumWriteValueLength(for:) and canSendWriteWithoutResponse before writing"
       ]
     },
     {
       "id": 2,
-      "prompt": "Review this BLE background plan: scan for all peripherals with nil services while in the background, rely on allowDuplicates for live RSSI, advertise a local name from the phone, skip bluetooth-peripheral because the service was added before suspension, and restore only the peripheral identifier after relaunch.",
-      "expected_output": "Review flags each background/state-restoration issue. It explains that background central scans require explicit service UUIDs and scan options have no effect; background peripheral advertising omits the local name and puts service UUIDs in overflow; published service contents are disabled without bluetooth-peripheral; and restoration should use Core Bluetooth restoration identifiers plus restored state dictionaries for peripherals, scan services/options, services, and advertisement data.",
+      "name": "background-and-restoration",
+      "prompt": "Our glucose app must keep reading a sensor in the background and survive being terminated. In the background we scan with nil services and AllowDuplicates on, we expect our peripheral-mode companion to advertise its local name while backgrounded without the bluetooth-peripheral mode, and on relaunch we just reconnect using a peripheral UUID saved in UserDefaults. What breaks?",
+      "expected_output": "Explains nil scans do not run in the background and scan options are ignored there, the local name is not advertised in the background and service UUIDs land in the overflow area seen only by explicit scans, service contents are unavailable while suspended without bluetooth-peripheral, and restoration identifiers with willRestoreState and the restored-state keys are needed instead of only a saved UUID.",
       "files": [],
       "assertions": [
-        "The output rejects nil-service background scanning and requires explicit service UUIDs for bluetooth-central background scans.",
-        "The output states scan options such as allowDuplicates have no effect while scanning in the background.",
-        "The output states background peripheral advertising does not include the local name and service UUIDs are discoverable only through explicit service scans.",
-        "The output states published service contents are disabled in the background without bluetooth-peripheral mode.",
-        "The output recommends Core Bluetooth restoration identifiers and restored state dictionary keys instead of restoring only an app-stored peripheral identifier."
+        "Rejects nil-service scans in the background",
+        "Notes scan options such as AllowDuplicates are ignored in the background",
+        "States the local name is not advertised in the background and service UUIDs are discoverable only by explicit service scans",
+        "Says published service contents are unavailable while suspended without the bluetooth-peripheral mode",
+        "Uses restoration identifiers, willRestoreState and the restored-state dictionary keys rather than relying on a stored identifier alone"
       ]
     },
     {
       "id": 3,
-      "prompt": "We need a privacy-preserving first-run picker for a BLE thermostat, then after the user picks it we need to read and write GATT characteristics. Should this be handled entirely by Core Bluetooth, entirely by AccessorySetupKit, or split? Give the implementation boundary.",
-      "expected_output": "The answer splits responsibility: AccessorySetupKit owns the privacy-preserving picker/discovery/authorization step, and Core Bluetooth owns post-setup GATT communication using the selected accessory identifier. It avoids broadening Core Bluetooth into picker setup and avoids using AccessorySetupKit for characteristic read/write logic.",
+      "name": "accessory-setup-boundary",
+      "prompt": "We make a smart plant pot. First-run setup should use the system accessory picker so we do not need broad Bluetooth access, and afterwards the app reads soil moisture and writes watering schedules. Our plan is to scan for every nearby device with Core Bluetooth to build the picker and then use AccessorySetupKit to read the characteristics. Is that right?",
+      "expected_output": "Swaps the roles: AccessorySetupKit handles the privacy-preserving picker and setup, Core Bluetooth handles GATT reads and writes afterwards, no broad scanning to build a picker, AccessorySetupKit is not used for characteristic I/O, and the selected accessory's identity is handed to Core Bluetooth to connect.",
       "files": [],
       "assertions": [
-        "The output routes the first-run privacy-preserving accessory picker to AccessorySetupKit.",
-        "The output routes post-selection GATT reads, writes, notifications, services, and characteristics to Core Bluetooth.",
-        "The output does not recommend broad direct Core Bluetooth scanning for the initial privacy-preserving picker use case.",
-        "The output does not expand AccessorySetupKit into characteristic read/write implementation.",
-        "The output describes the handoff from selected accessory identity to Core Bluetooth communication."
+        "Uses AccessorySetupKit for the first-run privacy-preserving picker",
+        "Uses Core Bluetooth for GATT reads and writes after the accessory is chosen",
+        "Advises against broad Core Bluetooth scanning to build the picker",
+        "States AccessorySetupKit is not the API for characteristic I/O",
+        "Describes passing the selected accessory's Bluetooth identifier to Core Bluetooth to retrieve and connect to it"
       ]
     }
   ]
-}
+}