@mmerterden/multi-agent-pipeline 8.6.2 → 10.0.6

package/pipeline/skills/shared/external/device-integrity/references/device-integrity-patterns.md

@@ -0,0 +1,240 @@
+# Device Integrity Extended Patterns
+
+Overflow reference for the `device-integrity` skill. Contains server verification details, advanced error handling, and integration patterns.
+
+## Contents
+
+- [Server-Side Attestation Verification](#server-side-attestation-verification)
+- [Server-Side Assertion Verification](#server-side-assertion-verification)
+- [Server Architecture](#server-architecture)
+- [Error Handling](#error-handling)
+- [Retry Strategy](#retry-strategy)
+- [Handling Rejected Keys](#handling-rejected-keys)
+- [Full Integration Manager](#full-integration-manager)
+- [Gradual Rollout](#gradual-rollout)
+- [Environment Entitlement](#environment-entitlement)
+
+## Server-Side Attestation Verification
+
+Your server must:
+1. Verify the attestation object is a valid CBOR-encoded structure.
+2. Extract the certificate chain and validate it against Apple's App Attest root CA.
+3. Compute `clientDataHash = SHA256(challenge)`, append it to the decoded
+   `authData`, then compute `nonce = SHA256(authData || clientDataHash)`.
+4. Extract the credential certificate extension with OID
+   `1.2.840.113635.100.8.2` and verify its octet string equals `nonce`.
+5. Verify the public-key hash matches the app-provided `keyId`.
+6. Verify the `RP ID` hash matches `SHA256(teamID + "." + bundleID)`.
+7. Verify the initial counter is `0`, the `aaguid` matches the expected
+   development or production environment, and `credentialId` equals `keyId`.
+8. Store the verified public key and receipt for future assertion verification.
+9. Mark the challenge consumed only after every verification step succeeds,
+   ideally in the same transaction that stores the key state.
+
+See [Validating apps that connect to your server](https://sosumi.ai/documentation/devicecheck/validating-apps-that-connect-to-your-server) for the full server verification algorithm.
+
+## Server-Side Assertion Verification
+
+Your server must:
+1. Decode the assertion (CBOR).
+2. Recompute `clientDataHash = SHA256(clientData)`, where `clientData`
+   includes a one-time server challenge and request context.
+3. Verify the signature using the stored public key over
+   `SHA256(authenticatorData || clientDataHash)`.
+4. Verify the `RP ID` hash and the counter (greater than the stored counter, or
+   greater than `0` for the first assertion).
+5. Confirm the embedded challenge matches the issued challenge and the request
+   context binds the assertion to the received request.
+6. Mark the challenge consumed and update the stored counter only after every
+   verification step succeeds, ideally atomically.
+
+## Server Architecture
+
+### Attestation vs. Assertion
+
+| Phase | When | What It Proves | Frequency |
+|-------|------|---------------|-----------|
+| **Attestation** | After key generation | The key lives on a genuine Apple device running a legitimate instance of your app | Once per key |
+| **Assertion** | With each sensitive request | The request came from the attested app instance | Per request |
+
+### Recommended Server Architecture
+
+1. **Challenge endpoint** -- generate a random nonce with at least 16 bytes of entropy, store it server-side with a short TTL (e.g., 5 minutes), purpose, and expected request/key context.
+2. **Attestation verification endpoint** -- validate the attestation object, store the public key and receipt keyed by `keyId`.
+3. **Assertion verification middleware** -- verify assertions on sensitive endpoints (purchases, account changes).
+
+Reject expired, missing, mismatched, or already-consumed challenges. Consume a
+challenge only after the corresponding attestation or assertion is fully
+verified; consuming on receipt can block safe retries after transient failures.
+
+### Risk Assessment
+
+Combine App Attest with [fraud risk assessment](https://sosumi.ai/documentation/devicecheck/assessing-fraud-risk) for defense in depth. App Attest alone does not guarantee the user is not abusing the app -- it confirms the app is genuine.
+
+App Attest is not a user authentication, session, entitlement, TLS, certificate
+pinning, or subscription validation system. Keep those controls in the
+appropriate authentication, networking, or broader security layer, and require
+them in addition to App Attest on protected endpoints.
+
+## Error Handling
+
+### DCError Codes
+
+```swift
+import DeviceCheck
+
+func handleAttestError(_ error: Error) {
+    if let dcError = error as? DCError {
+        switch dcError.code {
+        case .unknownSystemFailure:
+            // Transient system error -- retry with exponential backoff
+            break
+        case .featureUnsupported:
+            // Device or OS does not support this feature
+            // Fall back to alternative verification
+            break
+        case .invalidKey:
+            // Already-attested key, unattested assertion key, or service rejection
+            // Inspect local/server state; discard and regenerate only when bad
+            break
+        case .invalidInput:
+            // The clientDataHash or keyId was malformed
+            break
+        case .serverUnavailable:
+            // Retry attestation later with the same keyId and clientDataHash
+            break
+        @unknown default:
+            break
+        }
+    }
+}
+```
+
+## Retry Strategy
+
+```swift
+import CryptoKit
+
+extension AppAttestManager {
+    func attestKeyWithRetry(challenge: Data, maxAttempts: Int = 3) async throws -> Data {
+        guard let keyId else {
+            throw DeviceIntegrityError.keyNotGenerated
+        }
+
+        let clientDataHash = Data(SHA256.hash(data: challenge))
+        var lastError: Error?
+
+        for attempt in 0..<maxAttempts {
+            do {
+                return try await service.attestKey(keyId, clientDataHash: clientDataHash)
+            } catch let error as DCError where error.code == .serverUnavailable {
+                lastError = error
+                if attempt < maxAttempts - 1 {
+                    try await Task.sleep(for: .seconds(pow(2.0, Double(attempt + 1))))
+                }
+            } catch {
+                throw error // Non-retryable errors propagate immediately
+            }
+        }
+
+        throw lastError ?? DeviceIntegrityError.attestationFailed
+    }
+}
+```
+
+Use the same `challenge`, `keyId`, and `clientDataHash` for each retry after
+`.serverUnavailable`. Do not fetch a fresh challenge for that retry loop unless
+you are also starting over with a new attestation attempt.
+
+## Handling Rejected Keys
+
+`DCError.invalidKey` means the app called `attestKey` for an already-attested
+key, called `generateAssertion` with an unattested key, or the App Attest service
+rejected the key. If local/server state confirms the key cannot be used, delete
+the stored `keyId` and generate a new key:
+
+```swift
+extension AppAttestManager {
+    func handleRejectedKey() async throws -> String {
+        deleteKeyIdFromKeychain()
+        keyId = nil
+        return try await generateKeyIfNeeded()
+    }
+
+    private func deleteKeyIdFromKeychain() {
+        let query: [String: Any] = [
+            kSecClass as String: kSecClassGenericPassword,
+            kSecAttrAccount as String: "app-attest-key-id",
+            kSecAttrService as String: Bundle.main.bundleIdentifier ?? ""
+        ]
+        SecItemDelete(query as CFDictionary)
+    }
+}
+```
+
+## Full Integration Manager
+
+Combine the patterns above into a single `actor` that manages the full lifecycle:
+1. Check `isSupported` and fall back to `DCDevice` tokens on unsupported devices.
+2. Call `generateKeyIfNeeded()` for each user account on each device, reuse the
+   account/device-scoped `keyId`, and limit new key generation to new
+   account/device/install enrollment or confirmed bad-key recovery.
+3. Attest once per key; if `.serverUnavailable` occurs, retry with the same
+   challenge, key, and `clientDataHash`.
+4. For each sensitive request, obtain a one-time assertion challenge and sign
+   client data that includes the challenge plus request context.
+5. Handle `DCError.invalidKey` by checking whether the key was already attested,
+   not yet attested, or rejected before regenerating.
+
+## Gradual Rollout
+
+Apple recommends a gradual rollout. Gate App Attest behind a remote feature
+flag and fall back to `DCDevice` tokens on unsupported devices. For large apps,
+ramp production adoption gradually and be prepared to reduce attestation traffic
+if `.serverUnavailable` or rate-limit behavior increases during rollout.
+
+## Environment Entitlement
+
+Set the App Attest environment in your entitlements file. Use `development`
+during testing and `production` for App Store builds:
+
+```xml
+<key>com.apple.developer.devicecheck.appattest-environment</key>
+<string>production</string>
+```
+
+When the entitlement is omitted during development, the app uses the App Attest
+sandbox by default. After distribution through TestFlight, the App Store, or the
+Apple Developer Enterprise Program, the app ignores the entitlement value and
+uses production. Sandbox keys and receipts do not work in production, and
+production keys and receipts do not work in sandbox.
+
+If an App Clip or extension uses App Attest, configure the capability for that
+target too. App Attest is supported only in Action, extensible SSO, and watchOS
+extensions; other extension types are unsupported even if `isSupported` returns
+`true`.
+
+### Error Type
+
+```swift
+enum DeviceIntegrityError: Error {
+    case deviceCheckUnsupported
+    case keyNotGenerated
+    case attestationFailed
+    case attestationVerificationFailed
+    case assertionFailed
+    case serverVerificationFailed
+}
+```
+
+## Apple Documentation Links
+
+- [DeviceCheck framework](https://sosumi.ai/documentation/devicecheck)
+- [DCDevice](https://sosumi.ai/documentation/devicecheck/dcdevice)
+- [DCAppAttestService](https://sosumi.ai/documentation/devicecheck/dcappattestservice)
+- [Establishing your app's integrity](https://sosumi.ai/documentation/devicecheck/establishing-your-app-s-integrity)
+- [Validating apps that connect to your server](https://sosumi.ai/documentation/devicecheck/validating-apps-that-connect-to-your-server)
+- [Assessing fraud risk](https://sosumi.ai/documentation/devicecheck/assessing-fraud-risk)
+- [Preparing to use the app attest service](https://sosumi.ai/documentation/devicecheck/preparing-to-use-the-app-attest-service)
+- [Attestation Object Validation Guide](https://sosumi.ai/documentation/devicecheck/attestation-object-validation-guide)
+- [App Attest Environment](https://sosumi.ai/documentation/bundleresources/entitlements/com.apple.developer.devicecheck.appattest-environment)

package/pipeline/skills/shared/external/energykit/SKILL.md

@@ -6,9 +6,12 @@ description: "Query grid electricity forecasts and submit load events using Ener
 # EnergyKit
 
 Provide grid electricity forecasts to help users choose when to use electricity.
-EnergyKit identifies times when there is relatively cleaner or less expensive
-electricity on the grid, enabling apps to shift or reduce load accordingly.
-Targets Swift 6.2 / iOS 26+.
+EnergyKit identifies times when grid electricity is relatively cleaner and,
+when cost information is available, less expensive. Apps use that guidance to
+shift or reduce managed device load. Targets Swift 6.3 / iOS 26+.
+
+> **Beta-sensitive.** EnergyKit is new in iOS 26 and may change before GM.
+> Re-check current Apple documentation before relying on specific API details.
 
 ## Contents
 
@@ -27,8 +30,11 @@ Targets Swift 6.2 / iOS 26+.
 
 ### Entitlement
 
-EnergyKit requires the `com.apple.developer.energykit` entitlement. Add it
-to your app's entitlements file.
+EnergyKit requires the `com.apple.developer.energykit` entitlement. Enable the
+EnergyKit capability in Xcode so the entitlement is added to the app target.
+Treat this as a top-level setup prerequisite before writing guidance queries,
+venue lookup, load-event submission, or insight code. Missing permission can
+surface as `EnergyKitError.permissionDenied`.
 
 ### Import
 
@@ -36,16 +42,19 @@ to your app's entitlements file.
 import EnergyKit
 ```
 
-**Platform availability:** iOS 26+, iPadOS 26+.
+**Platform availability:** Core EnergyKit APIs are iOS 26.0+ and iPadOS 26.0+.
+Some insight breakdown APIs, including grid cleanliness categories, are iOS
+26.1+ / iPadOS 26.1+ and need availability guards.
 
 ## Core Concepts
 
 EnergyKit provides two main capabilities:
 
 1. **Electricity Guidance** -- time-weighted forecasts telling apps when
-   electricity is cleaner or cheaper, so devices can shift or reduce consumption
-2. **Load Events** -- telemetry from devices (EV chargers, HVAC) submitted back
-   to the system to track how well the app follows guidance
+   electricity is cleaner and, when rate data is available, less expensive
+2. **Load Events** -- telemetry from managed devices (EV chargers, HVAC)
+   submitted by the same device/app that requested guidance so EnergyKit can
+   generate insights
 
 ### Key Types
 
@@ -59,7 +68,7 @@ EnergyKit provides two main capabilities:
 | `ElectricVehicleLoadEvent` | Load event for EV charger telemetry |
 | `ElectricHVACLoadEvent` | Load event for HVAC system telemetry |
 | `ElectricityInsightService` | Service for querying energy/runtime insights |
-| `ElectricityInsightRecord` | Historical energy data broken down by cleanliness/tariff |
+| `ElectricityInsightRecord` | Historical energy or runtime data, optionally broken down by tariff or 26.1+ grid cleanliness |
 | `ElectricityInsightQuery` | Query for historical insight data |
 
 ### Suggested Actions
@@ -78,7 +87,7 @@ import EnergyKit
 
 func observeGuidance(venueID: UUID) async throws {
     let query = ElectricityGuidance.Query(suggestedAction: .shift)
-    let service = ElectricityGuidance.sharedService  // Verify access pattern against Xcode 26 SDK
+    let service = ElectricityGuidance.sharedService
 
     let guidanceStream = service.guidance(using: query, at: venueID)
 
@@ -172,8 +181,8 @@ struct RatingIndicator: View {
 
     var body: some View {
         Text(label)
-            .padding(.horizontal, 8)
-            .padding(.vertical, 4)
+            .padding(.horizontal)
+            .padding(.vertical)
             .background(color.opacity(0.2))
             .foregroundStyle(color)
             .clipShape(Capsule())
@@ -213,7 +222,9 @@ print("Venue name: \(venue.name)")
 ## Submitting Load Events
 
 Report device consumption data back to the system. This helps the system
-improve future guidance accuracy.
+generate electricity insights. The same EnergyKit-capable device/app that
+requested electricity guidance must submit the corresponding load events, using
+the guidance token returned by EnergyKit. Do not invent a token.
 
 ### EV Charger Load Events
 
@@ -289,20 +300,42 @@ func submitHVACEvent(
 | `.active` | Device is actively consuming (periodic updates) |
 | `.end` | Device stops consuming electricity |
 
+For EV charging, record begin/end, one steady sample about every 15 minutes,
+and extra samples for user actions, pauses, new guidance, or rapid power
+changes. For HVAC, submit separate events when equipment starts, when heating
+or cooling stage changes (heat stage 1 -> 2, heat -> cool, cool -> idle), and
+when equipment stops. Batch events when practical for performance. Insights
+are only available for submitted events, and load events for an `EnergyVenue`
+are visible to people who share the associated Home in the Home app.
+
 ## Electricity Insights
 
 Query historical energy and runtime data for devices using
-`ElectricityInsightService`.
+`ElectricityInsightService`. An empty `ElectricityInsightQuery.Options` option
+set returns totals only; it does not populate cleanliness or tariff breakdowns.
+Request `.cleanliness` and/or `.tariff` only when the UI needs those breakdowns.
+Do not substitute MetricKit app power metrics for EnergyKit insights; EnergyKit
+insights depend on EnergyKit load events submitted for the managed device.
+
+Choose insight granularity from the requested range. For a seven-day view,
+query `.hourly`; use `.daily` only when the query covers at least a calendar
+month.
 
 ```swift
 func queryEnergyInsights(deviceID: String, venueID: UUID) async throws {
+    let sevenDaysAgo = Calendar.current.date(
+        byAdding: .day,
+        value: -7,
+        to: Date()
+    )!
+
     let query = ElectricityInsightQuery(
         options: [.cleanliness, .tariff],
         range: DateInterval(
-            start: Calendar.current.date(byAdding: .day, value: -7, to: Date())!,
+            start: sevenDaysAgo,
             end: Date()
         ),
-        granularity: .daily,
+        granularity: .hourly,
         flowDirection: .imported
     )
 
@@ -313,7 +346,9 @@ func queryEnergyInsights(deviceID: String, venueID: UUID) async throws {
 
     for await record in stream {
         if let total = record.totalEnergy { print("Total: \(total)") }
-        if let cleaner = record.dataByGridCleanliness?.cleaner {
+
+        if #available(iOS 26.1, iPadOS 26.1, *),
+           let cleaner = record.dataByGridCleanliness?.cleaner {
             print("Cleaner: \(cleaner)")
         }
     }
@@ -322,22 +357,18 @@ func queryEnergyInsights(deviceID: String, venueID: UUID) async throws {
 
 Use `runtimeInsights(forDeviceID:using:atVenue:)` for runtime data instead
 of energy. Granularity options: `.hourly`, `.daily`, `.weekly`, `.monthly`,
-`.yearly`. See `references/energykit-patterns.md` for full insight examples.
+`.yearly`. Choose a range that matches Apple's minimum aggregation windows:
+hourly for at least a calendar week, daily for at least a calendar month,
+weekly for at least six months, and monthly or yearly for at least a calendar
+year. See [references/energykit-patterns.md](references/energykit-patterns.md) for full insight examples.
 
 ## Common Mistakes
 
 ### DON'T: Forget the EnergyKit entitlement
 
-Without the entitlement, all EnergyKit calls fail silently or throw errors.
-
-```swift
-// WRONG: No entitlement configured
-let service = ElectricityGuidance.sharedService  // Will fail
-
-// CORRECT: Add com.apple.developer.energykit to entitlements
-// Then use the service
-let service = ElectricityGuidance.sharedService
-```
+Without the entitlement, EnergyKit APIs can fail with permission errors such as
+`EnergyKitError.permissionDenied`. Treat the EnergyKit capability as setup, not
+as an implementation detail to discover after writing queries.
 
 ### DON'T: Ignore unsupported regions
 
@@ -377,13 +408,14 @@ do {
 
 ### DON'T: Discard the guidance token
 
-The `guidanceToken` links load events to the guidance that influenced them.
-Always store and pass it through to load event submissions.
+The `guidanceToken` links load events to the guidance that was in effect. Store
+the token returned from EnergyKit on the device that fetched it and pass that
+real token to load event submissions.
 
 ```swift
 // WRONG: Ignore the guidance token
 for try await guidance in guidanceStream {
-    startCharging()
+    startCharging(followingGuidanceToken: UUID())  // fabricated token
 }
 
 // CORRECT: Store the token for load events
@@ -434,17 +466,24 @@ let guidanceStream = service.guidance(using: query, at: venue.id)
 - [ ] `EnergyKitError.unsupportedRegion` handled with user-facing message
 - [ ] `EnergyKitError.permissionDenied` handled gracefully
 - [ ] Guidance token stored and passed to load event submissions
+- [ ] No placeholder or fabricated guidance tokens are used in load events
+- [ ] The same EnergyKit-capable device/app that requested guidance submits the corresponding load events
 - [ ] Venues discovered via `EnergyVenue.venues()` before querying guidance
 - [ ] Load event sessions follow `.begin` -> `.active` -> `.end` lifecycle
+- [ ] EV/HVAC event cadence follows Apple guidance and events are batched when practical
 - [ ] `ElectricityGuidance.Value.rating` interpreted correctly (lower is better)
 - [ ] `SuggestedAction` matches the device type (`.shift` for EV, `.reduce` for HVAC)
-- [ ] Insight queries use appropriate granularity for the time range
+- [ ] Insight queries use appropriate minimum ranges for their granularity
+- [ ] Empty insight query options are treated as totals-only, not as cleanliness or tariff requests
+- [ ] MetricKit power telemetry is not used as a substitute for EnergyKit load events or insights
+- [ ] Grid cleanliness insight fields are guarded for iOS/iPadOS 26.1+
+- [ ] Users understand load events are shared with people who share the Home
 - [ ] Rate limiting handled via `EnergyKitError.rateLimitExceeded`
 - [ ] Service unavailability handled with retry logic
 
 ## References
 
-- Extended patterns (full app architecture, SwiftUI dashboard): `references/energykit-patterns.md`
+- Extended patterns (full app architecture, SwiftUI dashboard): [references/energykit-patterns.md](references/energykit-patterns.md)
 - [EnergyKit framework](https://sosumi.ai/documentation/energykit)
 - [ElectricityGuidance](https://sosumi.ai/documentation/energykit/electricityguidance)
 - [ElectricityGuidance.Service](https://sosumi.ai/documentation/energykit/electricityguidance/service)

package/pipeline/skills/shared/external/energykit/evals/evals.json

@@ -0,0 +1,45 @@
+{
+  "skill_name": "energykit",
+  "evals": [
+    {
+      "id": 1,
+      "name": "ev-guidance-load-events",
+      "prompt": "Review this EnergyKit EV charging integration. It enables the capability, fetches ElectricityGuidance for an EnergyVenue, builds a charging schedule, then submits one ElectricVehicleLoadEvent at the end of the session with `GuidanceState(wasFollowingGuidance: true, guidanceToken: UUID())`. Give corrected Swift-level guidance without turning this into a HomeKit or BackgroundTasks tutorial.",
+      "expected_output": "A concise review that preserves the EnergyKit boundary, uses the real guidance token, and explains the load event lifecycle and cadence for EV charging.",
+      "files": [],
+      "assertions": [
+        "Lists the EnergyKit capability or `com.apple.developer.energykit` entitlement as a setup prerequisite, not only as an error-handling detail.",
+        "Rejects fabricating a guidance token and says the event must use the `guidanceToken` returned by EnergyKit guidance on the device/app that requested it.",
+        "Requires a session lifecycle with `.begin`, `.active`, and `.end` events rather than only one final event.",
+        "Mentions EV sampling about every 15 minutes when charging is steady and additional events for significant changes such as user actions, pauses, new guidance, or rapid power changes.",
+        "Keeps the answer focused on EnergyKit rather than expanding into HomeKit automation or BackgroundTasks implementation details."
+      ]
+    },
+    {
+      "id": 2,
+      "name": "insight-availability-granularity",
+      "prompt": "Fix this EnergyKit insight snippet for an app that supports iOS 26.0: it requests `.daily` insights for the last 7 days and immediately reads `record.dataByGridCleanliness?.cleaner`. Include the right availability and range guidance.",
+      "expected_output": "Corrects the query granularity/range mismatch and guards grid cleanliness insight access for iOS/iPadOS 26.1+.",
+      "files": [],
+      "assertions": [
+        "States that `dataByGridCleanliness` / grid cleanliness breakdowns require iOS/iPadOS 26.1+ and should be guarded or omitted on 26.0.",
+        "Does not claim all EnergyKit insight APIs are 26.1-only.",
+        "Pairs granularity to the requested range, using `.hourly` for a seven-day or calendar-week view and `.daily` only for at least a calendar month.",
+        "Mentions that empty `ElectricityInsightQuery.Options` returns totals without cleanliness or tariff breakdown."
+      ]
+    },
+    {
+      "id": 3,
+      "name": "energykit-boundary-review",
+      "prompt": "A teammate wrote: 'EnergyKit works anywhere, no entitlement is needed, and if there are no insights we can infer usage from MetricKit power metrics.' Review this plan for a smart thermostat app and identify the minimal corrections.",
+      "expected_output": "Identifies EnergyKit entitlement, region/venue limitations, submitted load-event dependency for insights, HVAC stage-change event guidance, and avoids MetricKit substitution.",
+      "files": [],
+      "assertions": [
+        "Requires the `com.apple.developer.energykit` entitlement and handles `EnergyKitError.permissionDenied` rather than saying no entitlement is needed.",
+        "Handles unsupported regions and unavailable or restricted venues instead of claiming EnergyKit works anywhere.",
+        "States that EnergyKit insights depend on submitted EnergyKit load events and cannot be replaced by MetricKit power metrics.",
+        "For HVAC, recommends separate load events for heating/cooling stage transitions rather than only fixed timer samples or a final summary."
+      ]
+    }
+  ]
+}