reis-mobile 0.7.3 → 0.8.0

package/skills/mobile-observability/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: mobile-observability
+description: Designs and reviews production observability for mobile apps on every stack — Flutter, native Android, native iOS, React Native and Kotlin Multiplatform. Covers crash and error reporting with usable context, breadcrumbs, structured logs, traces and custom metrics, release and cohort segmentation, correlation with backend requests, alerting thresholds, sampling and cost, and what must never be collected. Use when production problems cannot be explained from the available data, when setting up crash reporting or telemetry, or when defining release health alerts.
+intents: [observability]
+stacks: ["*"]
+---
+
+# Mobile Observability
+
+What you can learn about the app after it has shipped, on every stack. This file holds the model, the segmentation rules and the privacy limits; each platform's SDKs, symbol upload and field-metric sources live in its reference.
+
+## 1. Pick the reference
+
+| Project | Read |
+|---|---|
+| Flutter | [references/flutter.md](references/flutter.md) |
+| Native Android | [references/android.md](references/android.md) |
+| Native iOS | [references/ios.md](references/ios.md) |
+| React Native or Expo | [references/react-native.md](references/react-native.md) |
+| Kotlin Multiplatform | [references/kotlin-multiplatform.md](references/kotlin-multiplatform.md) |
+
+Firebase Crashlytics, Analytics and Performance Monitoring setup per stack is in [mobile-firebase](../mobile-firebase/SKILL.md). Release gating on these signals is in [mobile-release](../mobile-release/SKILL.md); the lab counterpart of the performance metrics is in [mobile-performance](../mobile-performance/SKILL.md).
+
+## 2. Mobile is not a server
+
+Everything below follows from four differences, and ignoring them is what produces a telemetry setup that costs money and answers nothing:
+
+1. **You cannot log in to the device.** There is no `kubectl logs`. If the data was not collected and sent before the user gave up, it does not exist.
+2. **The old version never goes away.** A server rolls back in minutes; an app version lives on devices for months. Every signal must be sliced by app version, or a fixed bug and a live bug look identical in the aggregate.
+3. **The device is the constraint.** Telemetry costs the user's battery, data plan and storage. A payload sent per event is a wakeup per event — see [background.md](../mobile-performance/references/background.md).
+4. **Reports arrive late and out of order.** A crash is reported on the *next* launch; a device offline for a week uploads a week later. Never read the last hour of a metric as "what is happening now" without accounting for the reporting delay.
+
+## 3. The signals, and what each one answers
+
+| Signal | Answers | Usual failure |
+|---|---|---|
+| **Crashes** | The process died. Where, on which OS/device/version | Obfuscated stack traces — symbols not uploaded |
+| **Non-fatal errors** | A caught failure the user saw as a broken screen | Everything caught and logged, so the important one is invisible |
+| **ANRs / hangs / terminations** | The app froze or was killed, without a crash report | Only watching crashes and missing the majority of bad sessions |
+| **Breadcrumbs** | What the user did in the seconds before the failure | Free text with no structure, useless for grouping |
+| **Structured logs** | The detail behind one session | Sent for every session, at full volume, at full cost |
+| **Traces / spans** | Where the time went in one flow, across app and backend | Instrumented only in the app, so the backend half is invisible |
+| **Metrics** | The aggregate: latency percentiles, success rates, adoption | Averages instead of percentiles |
+| **Product analytics** | Whether the user completed the journey at all | Treated as observability; it shows the drop, never the cause |
+
+A production incident needs at least three of these to be explainable: **what broke** (crash/error), **what led to it** (breadcrumbs/logs), and **how many users** (metrics segmented by version).
+
+## 4. Segmentation: the dimensions that must be on every signal
+
+Attach these to every crash, error, span and metric, or the data cannot be sliced when it matters:
+
+- **App version and build number** — non-negotiable; every comparison is version-over-version.
+- **OS version and device model** — most "random" crashes are one OS version or one manufacturer.
+- **Release channel / rollout stage** — internal, beta, staged rollout percentage, full.
+- **A stable install id**, not a user id, for counting affected users. It must be resettable and must not follow the person across reinstalls or apps (see §8).
+- **Locale and region** where formatting, right-to-left layout or regulation differ.
+- **Connectivity class** at the time (offline, cellular, Wi-Fi) for anything network-related.
+- **Feature flag / experiment state** for flags that change the code path. A crash present only in one variant is otherwise unattributable.
+
+Keep the cardinality bounded. A dimension whose value is a user id, a timestamp or a free-text message turns a metric into millions of series and is the standard way to make an observability bill unpayable.
+
+## 5. Error context: make one report enough
+
+A stack trace alone rarely explains a mobile failure. A report is useful when it carries:
+
+- **The stack trace, symbolicated.** Verify symbol upload in CI for every build variant — an unsymbolicated release build is the most common and most expensive gap. The reference names the mechanism per stack.
+- **Custom keys** for the state that decides the code path: the screen, the entity id being acted on, the auth state, whether the outbox was non-empty, the flag values.
+- **Breadcrumbs as structured events**, not sentences: `screen_view{name}`, `network{method, host, status, ms}`, `tap{target}`, `state_change{from, to}`. Cap them (the last 50–100) and drop the payloads.
+- **The grouping key.** Reports group by stack signature; a generic wrapper (`throw AppException(e)`) collapses a hundred distinct causes into one issue and hides all of them. Preserve the cause chain, and put the variable detail in keys, not in the exception message.
+- **A correlation id** shared with the backend (§6).
+- **Whether the user saw it.** A caught-and-recovered error and a blank screen are different severities and should be distinguishable without reading code.
+
+## 6. Correlation with the backend
+
+Mobile latency is mostly backend latency, and the app's "the request was slow" is unactionable on its own.
+
+- Generate a **request id** per outbound call on the device, send it in a header, log it on both sides, and put it on the client-side span and on any error report from that call. One id makes "slow for this user at that minute" a single backend query instead of a guess.
+- Use **W3C Trace Context** (`traceparent`) when the backend is already traced, so the mobile span is the parent of the server spans rather than a disconnected record. OpenTelemetry clients exist for mobile; the reference names the current state per stack.
+- Attach a **session id** that spans the app session, so several failing calls can be recognized as one bad session.
+- **Propagate nothing sensitive** in these headers, and never put a token in a trace attribute.
+
+Without correlation, the mobile team and the backend team argue from two datasets. With it, the argument is a join.
+
+## 7. Alerting: on the few things that mean "stop the rollout"
+
+Most mobile alerts are noise because they fire on a raw count that tracks daily usage. Alert on **rates**, segmented by version, with the reporting delay in mind:
+
+| Alert | Reasonable shape |
+|---|---|
+| Crash-free sessions drop | Below your baseline (commonly 99.5%+ for a mature app) **for the new version**, compared against the previous one |
+| ANR / hang rate | Above the platform's own bad-behaviour threshold; both stores act on this |
+| New issue in a new release | Any issue whose first occurrence is this version and which affects more than N users — this is the one that stops a staged rollout |
+| Critical journey success rate | Login, checkout, sync drain: success percentage per version, not per absolute count |
+| Startup and key-screen latency | P90, per version, against the previous release's value |
+
+Rules that keep alerts trustworthy: compare a version against the previous version and not against last week; require a minimum session count before a percentage can fire; suppress alerts during the first minutes of a rollout when the denominator is tiny; and route each alert to someone who can act — a page for release health, a dashboard for everything else. An alert nobody acts on should be deleted, not muted.
+
+## 8. Sampling and cost
+
+Full-fidelity telemetry from every session is neither affordable nor kind to the device.
+
+- **Never sample crashes, ANRs or fatal errors.** They are rare and each one matters.
+- **Sample traces and verbose logs** — a small percentage of sessions at full detail answers most questions. Make the decision **per session, not per event**, so a sampled session is complete and readable end to end.
+- **Always keep the sessions that failed.** Sampling that discards the broken session is worse than no sampling: raise the rate for sessions with an error, or buffer locally and upload only on failure.
+- **Batch uploads**, with a size and a time bound, and flush on background/terminate. One flush per event is a radio wakeup per event.
+- **Bound the local buffer** and drop the oldest when it is full; an unbounded telemetry queue fills the device and turns a monitoring feature into a support ticket.
+- **Respect the network.** Do not upload telemetry on metered or constrained connections unless it is a crash report.
+- **Record the sampling rate with the data.** A percentile computed from a sampled set without knowing the rate is wrong, and someone will read it as absolute.
+
+## 9. What must never be collected
+
+Observability is where privacy violations happen by accident, because logging "the request" logs whatever was in it.
+
+- **Never log:** passwords, tokens, refresh tokens, API keys, full payment data, government ids, health data, precise location, message or document contents, or full request/response bodies from authenticated endpoints.
+- **Redact at the source**, in the SDK's callback before send (every reporter has one), not in the backend pipeline. Data that left the device is already disclosed.
+- **URLs carry secrets** in query strings — strip query parameters, or allow-list them. A URL is an attribute, not a free-text field.
+- **Crash reports include more than you sent**: register values, thread names, and sometimes fragments of memory in native crashes. Treat the whole report as sensitive and restrict who can read it.
+- **Consent gates collection.** On both stores, telemetry and analytics must match the declared data-safety/privacy-nutrition disclosures and the app's consent state. A user who declined analytics must not have analytics sent, and the SDK must be initialized accordingly — not initialized and then asked to stay quiet.
+- **Prefer aggregation and short retention** for anything user-linked, and be able to delete a person's data on request, which requires the install id to be mappable and resettable.
+
+This overlaps with [mobile-security](../mobile-security/SKILL.md); when in doubt, the answer is to send less.
+
+## 10. Reviewing an existing setup
+
+- [ ] Symbol/mapping upload happens in CI for every shipped variant, and a release build's crash is verifiably readable.
+- [ ] App version, OS, device, channel and flag state are on every signal.
+- [ ] ANRs, hangs and background terminations are collected, not just crashes.
+- [ ] Caught errors are distinguishable from user-visible failures, and grouping is not collapsed by a generic wrapper.
+- [ ] Breadcrumbs are structured and capped; logs are sampled; crashes are not.
+- [ ] A client-generated correlation id reaches the backend and appears in error reports.
+- [ ] Alerts are rate-based, per version, with a minimum-volume guard, and each one has an owner.
+- [ ] Telemetry uploads are batched, bounded, flushed on background, and off on metered connections.
+- [ ] A redaction callback exists and is tested; no secrets, bodies or precise location are sent.
+- [ ] Collection respects consent and matches the store privacy declarations.
+- [ ] Someone can answer "how many users on 4.2.0 hit this, and what were they doing" without adding code.
+
+The last line is the real test. If answering an ordinary production question requires shipping a new build and waiting two weeks for adoption, the setup has failed regardless of how many SDKs are installed.

package/skills/mobile-observability/references/android.md

@@ -0,0 +1,105 @@
+# Observability on native Android
+
+Android is the only mobile platform that gives you meaningful field data **without installing anything**: Play Console's Android vitals reports crashes, ANRs, wakeups, wake locks and rendering for your installed base, sliced by version, device and OS. Start there, then add an SDK for the context vitals cannot give you.
+
+## What you get for free
+
+| Source | Gives |
+|---|---|
+| Play Console → Android vitals | Crash rate, **ANR rate** and user-perceived ANR rate, excessive wakeups, stuck partial wake locks, slow/frozen frames, app size — all per version and per device model, with bad-behaviour thresholds the store itself acts on |
+| `ApplicationExitInfo` | Why the process last died — crash, ANR, low memory, user request, excessive resource use — readable **on the next launch**, including the ANR trace |
+| Play Console → Reach and devices | The device/OS distribution you actually ship to, which decides what a "rare" crash really costs |
+
+`ApplicationExitInfo` is the cheapest large win in Android observability and is frequently missing: it is the only way to see background terminations and OOM kills, which produce no crash report at all.
+
+```kotlin
+val am = getSystemService(ActivityManager::class.java)
+am.getHistoricalProcessExitReasons(packageName, /* pid = */ 0, /* maxNum = */ 5)
+    .firstOrNull()
+    ?.let { info ->
+        crashReporter.log(
+            "last_exit reason=${info.reason} status=${info.status} " +
+                "importance=${info.importance} description=${info.description}",
+        )
+        // REASON_ANR carries the ANR trace in info.traceInputStream
+    }
+```
+
+Report it on the next launch, attached to the session, so an OOM kill becomes a searchable event rather than an unexplained gap.
+
+## Crash and error reporting
+
+Crashlytics (see [mobile-firebase](../../mobile-firebase/SKILL.md)), Sentry, Bugsnag and Embrace all cover JVM crashes, native (NDK) crashes and non-fatals. What matters is the configuration, not the vendor:
+
+- **Mapping file upload for every release variant.** With R8/Proguard enabled, an un-uploaded mapping file produces an unreadable stack trace. The Crashlytics Gradle plugin uploads automatically when `mappingFileUploadEnabled` is on; other SDKs have a Gradle task. Verify it in CI by failing the build when the upload task did not run, and prove it once by symbolicating a deliberate release crash.
+- **Native symbols** need a separate upload (`firebaseCrashlytics { nativeSymbolUploadEnabled = true }`, or the vendor's equivalent) or every NDK/Kotlin-Native frame is an address.
+- **Custom keys** on every session: screen, auth state, feature-flag values, outbox depth, connectivity class.
+- **Non-fatals** for caught failures that the user saw: `FirebaseCrashlytics.getInstance().recordException(e)` — and *not* for every caught exception, or the important one disappears.
+- **Preserve the cause chain.** `throw AppException("failed", cause = e)` keeps grouping meaningful; `throw AppException("failed")` collapses every cause into one issue.
+
+## ANRs and hangs
+
+ANRs are the signal Android developers most often miss, because they produce no crash callback.
+
+- Play Console's ANR rate is authoritative and is a store policy threshold.
+- `ApplicationExitInfo` with `REASON_ANR` gives the trace on the device.
+- Crash reporters detect ANRs with their own watchdogs; treat their counts as an early indicator and Play Console as the truth.
+- Diagnose the cause with a system trace and the main-thread stack — see [mobile-performance/references/android.md](../../mobile-performance/references/android.md) and the ANR guide in [mobile-debug](../../mobile-debug/SKILL.md).
+
+## Logs, traces and metrics
+
+- **Structured logs.** `Log.d` goes nowhere in production. Use the reporter's breadcrumb API, or a `Timber`-style tree that forwards structured events to the reporter, and keep the tags bounded.
+- **Traces.** Firebase Performance Monitoring gives automatic app-start, screen-rendering and HTTP traces plus `Trace` for custom spans, with no backend work. OpenTelemetry's Android instrumentation is the option when you already run an OTel backend and want `traceparent` propagation into the server traces; check the current stability of the artifacts before adopting them.
+- **Custom metrics.** Prefer a small number of counters and durations with bounded attributes (version, screen, result) over free-form events.
+- **Network correlation.** An OkHttp `Interceptor` is the natural place to generate and attach the request id, record the span, and add a breadcrumb — one implementation covers Retrofit, Coil and anything else on the same client.
+
+```kotlin
+class ObservabilityInterceptor(private val reporter: Reporter) : Interceptor {
+    override fun intercept(chain: Interceptor.Chain): Response {
+        val requestId = UUID.randomUUID().toString()
+        val request = chain.request().newBuilder().header("X-Request-Id", requestId).build()
+        val start = SystemClock.elapsedRealtime()
+        return try {
+            chain.proceed(request).also { response ->
+                reporter.breadcrumb(
+                    "network",
+                    mapOf(
+                        "method" to request.method,
+                        "host" to request.url.host,            // host, not the full URL
+                        "status" to response.code,
+                        "ms" to SystemClock.elapsedRealtime() - start,
+                        "request_id" to requestId,
+                    ),
+                )
+            }
+        } catch (e: IOException) {
+            reporter.breadcrumb("network_error", mapOf("host" to request.url.host, "request_id" to requestId))
+            throw e
+        }
+    }
+}
+```
+
+Note what is absent: no full URL (query strings carry tokens), no headers, no bodies.
+
+## Privacy and consent
+
+- Play requires an accurate **Data safety** declaration covering everything the SDKs collect, including the ones you did not configure yourself. Audit the merged manifest and the dependency list, not just your own code.
+- Gate initialization on consent. Analytics collection can be disabled by default in the manifest and enabled after consent (`firebase_analytics_collection_enabled`, `firebase_crashlytics_collection_enabled` and the equivalent runtime setters) — initializing and then hoping the SDK stays quiet is not the same thing.
+- Use a resettable id (the reporter's install id, or `AdvertisingIdClient` only if advertising is genuinely the purpose and it is declared). `ANDROID_ID` and any hardware identifier are not acceptable install ids.
+- Register a redaction callback / `beforeSend` equivalent and unit-test it against a payload containing a token.
+
+## CI
+
+- Fail the build when the mapping or native-symbol upload task did not run for a release variant.
+- Post the build's version name and code into the reporter so a release appears in the dashboard the moment it ships.
+- Check Android vitals as part of the staged-rollout gate — see [mobile-release](../../mobile-release/SKILL.md).
+
+## Official documentation
+
+- Android vitals: https://developer.android.com/topic/performance/vitals
+- ApplicationExitInfo: https://developer.android.com/reference/android/app/ApplicationExitInfo
+- ANRs: https://developer.android.com/topic/performance/vitals/anr
+- Crashlytics for Android: https://firebase.google.com/docs/crashlytics/get-started?platform=android
+- Firebase Performance Monitoring: https://firebase.google.com/docs/perf-mon/get-started-android
+- Play Data safety: https://support.google.com/googleplay/android-developer/answer/10787469

package/skills/mobile-observability/references/flutter.md

@@ -0,0 +1,86 @@
+# Observability in Flutter
+
+Flutter's specific problem is that a failure can happen in three places with three different capture mechanisms: **Dart** (uncaught exceptions in the framework, in your code, or in an isolate), the **Flutter engine/native layer** (a real native crash), and the **platform channels** between them. A setup that only wires `FlutterError.onError` misses two of the three.
+
+## Capture all three layers
+
+```dart
+Future<void> main() async {
+  WidgetsFlutterBinding.ensureInitialized();
+  await Firebase.initializeApp();
+
+  // 1. Errors the Flutter framework catches (build, layout, paint, gestures).
+  FlutterError.onError = FirebaseCrashlytics.instance.recordFlutterFatalError;
+
+  // 2. Errors from the Dart zone that the framework does not catch:
+  //    async gaps, futures without a catch, platform-channel replies.
+  PlatformDispatcher.instance.onError = (error, stack) {
+    FirebaseCrashlytics.instance.recordError(error, stack, fatal: true);
+    return true;
+  };
+
+  runApp(const App());
+}
+```
+
+- `FlutterError.onError` alone leaves every unhandled async error unreported.
+- A background `Isolate` has its own error handling: add an `Isolate.current.addErrorListener` (or the reporter's helper) inside the isolate entry point, or its crashes are silent.
+- Native crashes in the engine or in a plugin's Android/iOS code are captured by the native SDK the Flutter package installs — which is why the plugin's native setup steps matter, not just the Dart ones.
+
+Sentry, Bugsnag, Embrace and Crashlytics all follow this shape; the reference for the Firebase path is [mobile-firebase](../../mobile-firebase/SKILL.md).
+
+## Symbols
+
+A release Flutter build is AOT-compiled, and its Dart stack traces are **obfuscated addresses** unless the debug info is uploaded.
+
+```bash
+flutter build appbundle --obfuscate --split-debug-info=build/symbols
+flutter build ipa       --obfuscate --split-debug-info=build/symbols
+```
+
+Then upload `build/symbols` with the vendor's command (`firebase crashlytics:symbols:upload`, `sentry-cli debug-files upload`, …) **for every build you ship**, and keep the directory per version — symbols from a different build do not resolve. On Android you still need the R8 mapping file for the Kotlin side, and on iOS the dSYMs for the native side: three uploads, not one.
+
+The most common production gap in Flutter apps is exactly this: Dart symbols uploaded during a manual first setup, never wired into CI, and unreadable stack traces from then on. Verify by crashing a release build on purpose once.
+
+## Context and breadcrumbs
+
+- **Navigation breadcrumbs** come free from a `NavigatorObserver`:
+
+```dart
+class ObservabilityObserver extends NavigatorObserver {
+  @override
+  void didPush(Route route, Route? previous) =>
+      reporter.breadcrumb('screen_view', {'name': route.settings.name ?? 'unnamed'});
+}
+```
+
+Route names must be set for this to be useful — an app built entirely from anonymous `MaterialPageRoute`s produces `unnamed` breadcrumbs.
+
+- **Custom keys** per session: `setCustomKey` for the screen, auth state, flag values, outbox depth, connectivity.
+- **Network breadcrumbs** belong in a Dio `Interceptor` or an `http` `BaseClient` wrapper, where the request id is also generated and attached. Record method, host, status and duration — never the full URL or the body.
+- **State-management errors:** `Bloc.observer` (`onError`), Riverpod's `ProviderObserver`, or the equivalent hook forwards state-layer failures with the state that caused them, which a bare stack trace does not carry.
+- Keep the cause chain: `Error.throwWithStackTrace(AppException(e), stack)` preserves the original trace, while `throw AppException('failed')` in a `catch` discards the only useful part of the report.
+
+## Performance signals
+
+- **Firebase Performance Monitoring** for Flutter gives app-start and custom traces; HTTP traces are collected automatically on the native side, and Dart-side calls need explicit `HttpMetric` instrumentation.
+- The lab counterparts — frame timings, startup traces, `integration_test` timeline summaries — are in [mobile-performance/references/flutter.md](../../mobile-performance/references/flutter.md). Use the same metric names in both so a CI regression and a field regression are comparable.
+- `SchedulerBinding.instance.addTimingsCallback` exposes real frame timings in a release build; sampled and aggregated, it is a usable field jank metric when a vendor SDK does not provide one.
+
+## Platform field data still applies
+
+Flutter does not replace Android vitals, Play Console ANR data, Xcode Organizer or MetricKit — the app is still an Android and an iOS process. ANRs and watchdog terminations in a Flutter app show up there and nowhere in the Dart layer. Read [android.md](android.md) and [ios.md](ios.md) alongside this file.
+
+## Privacy and consent
+
+- Both stores' declarations (Data safety, privacy nutrition label and `PrivacyInfo.xcprivacy`) must cover what the Flutter plugins collect natively, not just what your Dart code sends.
+- Gate collection: `FirebaseCrashlytics.instance.setCrashlyticsCollectionEnabled(false)` until consent, or the vendor's equivalent, set **before** the reporter would send anything.
+- Implement the SDK's redaction hook (`beforeSend` in Sentry, a record filter elsewhere) and unit-test it with a payload containing a token.
+
+## Official documentation
+
+- Handling errors in Flutter: https://docs.flutter.dev/testing/errors
+- Obfuscating Dart code: https://docs.flutter.dev/deployment/obfuscate
+- Crashlytics for Flutter: https://firebase.google.com/docs/crashlytics/get-started?platform=flutter
+- Firebase Performance Monitoring for Flutter: https://firebase.google.com/docs/perf-mon/flutter/get-started
+- Flutter DevTools: https://docs.flutter.dev/tools/devtools/overview

package/skills/mobile-observability/references/ios.md

@@ -0,0 +1,88 @@
+# Observability on native iOS
+
+Apple's own telemetry is good and under-used: **MetricKit** delivers per-device diagnostics and metrics to your app, and **Xcode Organizer** aggregates crashes, hangs, launch time, memory, disk writes and battery per release from users who opted into sharing. Both are free, privacy-reviewed and need no SDK. Add a third-party reporter for the context they do not carry — breadcrumbs, custom keys, correlation ids.
+
+## What you get from Apple
+
+| Source | Gives |
+|---|---|
+| Xcode Organizer → Crashes | Symbolicated crash reports per version, grouped, from opted-in users |
+| Xcode Organizer → Hangs, Launches, Memory, Disk Writes, Battery | Field metrics per version, with a comparison to the previous release |
+| MetricKit (`MXMetricManager`) | Daily metric payloads (`MXAppLaunchMetric`, `MXAnimationMetric`, `MXAppExitMetric`, `MXSignpostMetric`, and more) and diagnostic payloads (crash, hang, disk-write, CPU exception) delivered **into your app**, so you can forward them to your own backend |
+
+`MXAppExitMetric` is the one to wire up first: it reports how sessions ended, including **memory-pressure terminations and watchdog kills**, which produce no crash report and are otherwise invisible.
+
+```swift
+final class MetricsSubscriber: NSObject, MXMetricManagerSubscriber {
+    func didReceive(_ payloads: [MXMetricPayload]) {
+        for payload in payloads {
+            backend.upload(payload.jsonRepresentation(), kind: "metric")
+        }
+    }
+
+    func didReceive(_ payloads: [MXDiagnosticPayload]) {
+        for payload in payloads {
+            backend.upload(payload.jsonRepresentation(), kind: "diagnostic")   // hangs and crashes
+        }
+    }
+}
+
+// In application(_:didFinishLaunchingWithOptions:)
+MXMetricManager.shared.add(subscriber)
+```
+
+Payloads arrive **once a day, about the previous day** — plan the dashboard around that delay rather than expecting live data.
+
+## Crash and error reporting
+
+- **dSYMs must reach the reporter for every build.** With bitcode gone this is a straightforward upload, but it still breaks constantly: Xcode Cloud, a CI cache, or `DEBUG_INFORMATION_FORMAT` set to `dwarf` instead of `dwarf-with-dsym` will all produce unreadable release crashes. Run the vendor's upload script as a build phase *and* verify in CI, then prove it once with a deliberate release crash.
+- **Swift errors are not exceptions.** A crash reporter sees `fatalError`, force-unwraps, array bounds, and signals — not a `throw` you caught. User-visible failures from `catch` blocks must be reported explicitly as non-fatals, with the state that explains them.
+- **Custom keys** per session: scene/screen, auth state, flag values, outbox depth, connectivity (`NWPathMonitor`'s current path).
+- **Grouping** follows the stack signature. A generic `AppError.wrapped(Error)` rethrown from one place collapses unrelated failures into one issue; keep the original error type and domain in the report.
+
+## Hangs
+
+Apple treats the main thread being unresponsive beyond roughly 250 ms as a hang, and Organizer reports a hang rate per hour of use.
+
+- Organizer → Hangs and MetricKit's hang diagnostics give the field data with a stack.
+- The Thread Performance Checker (a debug-time diagnostic) catches main-thread file I/O and priority inversions before they ship.
+- Diagnose with Instruments — see [mobile-performance/references/ios.md](../../mobile-performance/references/ios.md).
+
+Watchdog terminations (`0x8badf00d`) are hangs that lasted long enough to be killed; they appear in `MXAppExitMetric` and in Organizer, not as ordinary crashes.
+
+## Logs, traces and metrics
+
+- **`OSLog` / `Logger` is the system logger**: structured, privacy-aware, and cheap. It does **not** leave the device — treat it as what you read on a connected device or in a sysdiagnose, and forward what you need to your reporter separately.
+- Mark values explicitly: `logger.info("order \(id, privacy: .private) failed with \(code, privacy: .public)")`. Dynamic strings default to private and are redacted in collected logs — deliberately marking a secret `.public` is the mistake to look for in review.
+- **Signposts** (`OSSignposter`) mark intervals that Instruments can read *and* that MetricKit aggregates as `MXSignpostMetric` from the field — the same instrumentation serves the lab and production.
+- **Network correlation:** a `URLProtocol`, a custom `URLSessionTaskDelegate`, or a thin client wrapper generates the request id, adds the header, records the duration and drops a breadcrumb with **host and status only** — never the full URL or the body.
+
+```swift
+var request = URLRequest(url: url)
+let requestID = UUID().uuidString
+request.setValue(requestID, forHTTPHeaderField: "X-Request-Id")
+```
+
+- **OpenTelemetry** has a Swift implementation for teams already running an OTel backend and wanting `traceparent` propagation into server traces; check the current maturity of the package before adopting it.
+
+## Privacy and consent
+
+- The **privacy nutrition label** and the **privacy manifest** (`PrivacyInfo.xcprivacy`) must cover everything collected, including by third-party SDKs — and SDKs on Apple's required-reason list must ship their own manifest and signature. A reporter added for observability changes your declared data collection.
+- **App Tracking Transparency** applies to tracking across apps and websites, not to first-party crash reporting; do not conflate them, but do check what your SDK's default configuration sends.
+- Gate SDK initialization on consent where consent is required, and use the reporter's own resettable install id rather than `identifierForVendor` if the id must be resettable independently of app removal.
+- Implement the vendor's `beforeSend` callback and test it against a payload containing a token.
+
+## CI
+
+- Fail the archive job when dSYM upload did not run, and when `DEBUG_INFORMATION_FORMAT` is not `dwarf-with-dsym` for Release.
+- Upload the build's version and build number to the reporter so the release appears immediately.
+- Check Organizer's hang rate and crash rate for the previous release before promoting a new one — see [mobile-release](../../mobile-release/SKILL.md).
+
+## Official documentation
+
+- MetricKit: https://developer.apple.com/documentation/metrickit
+- Diagnosing issues using crash reports and device logs: https://developer.apple.com/documentation/xcode/diagnosing-issues-using-crash-reports-and-device-logs
+- Understanding hangs in your app: https://developer.apple.com/documentation/xcode/understanding-hangs-in-your-app
+- Logging with OSLog: https://developer.apple.com/documentation/os/logging
+- Recording performance data with signposts: https://developer.apple.com/documentation/os/recording-performance-data
+- Privacy manifest files: https://developer.apple.com/documentation/bundleresources/privacy-manifest-files

package/skills/mobile-observability/references/kotlin-multiplatform.md

@@ -0,0 +1,76 @@
+# Observability in Kotlin Multiplatform
+
+Two things are shared and one is not. The **instrumentation** — what you record, the event names, the attributes, the sampling decision, the redaction rules — belongs in `commonMain`, so both apps report the same names and a dashboard can compare them. The **reporting SDK** is per host: Crashlytics/Sentry on Android through the Android SDK, the same vendor on iOS through its Apple SDK. There is no cross-platform crash reporter that covers Kotlin/Native crashes as well as each platform's native reporter does.
+
+## The boundary
+
+```kotlin
+// commonMain — the only API the shared code knows about
+interface Telemetry {
+    fun breadcrumb(name: String, attributes: Map<String, String> = emptyMap())
+    fun setKey(key: String, value: String)
+    fun recordError(throwable: Throwable, fatal: Boolean = false)
+    fun startSpan(name: String): Span
+}
+```
+
+`androidMain` implements it over the Android SDK; `iosMain` implements it over the Apple SDK, or the iOS app injects a Swift implementation through the interface. Prefer injection from the host over `expect`/`actual` here: it keeps the vendor out of the shared dependency graph and makes `commonTest` a matter of passing a recording fake.
+
+Everything else — which events exist, their attribute keys, the sampling rate, the redaction — lives in `commonMain` and is tested once.
+
+## Kotlin/Native crashes on iOS
+
+This is the part that surprises teams:
+
+- An **unhandled Kotlin exception on iOS terminates the process**. By default it is reported as a native crash, and the report's usefulness depends entirely on the framework's dSYM being uploaded alongside the app's.
+- `setUnhandledExceptionHook` lets you record the Kotlin exception through the host reporter *before* the process dies — use it to attach the shared-layer context (the screen, the operation, the outbox depth) that a raw native crash report will not carry. Do not use it to swallow the exception.
+- Kotlin frames appear as addresses unless the framework is built with debug symbols and those symbols are uploaded. Check `binaryOptions` and the framework build configuration, and verify once with a deliberate release crash from shared code.
+- `kotlin.native.binary.` options that strip symbols will make every shared-code crash unreadable. Confirm what the release framework actually ships.
+
+On Android the shared code is ordinary JVM code: Kotlin exceptions reach the default uncaught handler and the reporter sees them normally, symbolicated through the R8 mapping file.
+
+## Instrumenting the shared layer
+
+- **Ktor:** a client plugin is the single place to generate the request id, add the header, record the span and drop a network breadcrumb — for both platforms at once.
+
+```kotlin
+val client = HttpClient {
+    install(createClientPlugin("Observability") {
+        onRequest { request, _ ->
+            request.headers.append("X-Request-Id", uuid4().toString())
+        }
+        onResponse { response ->
+            telemetry.breadcrumb("network", mapOf(
+                "host" to response.request.url.host,          // host only, never the full URL
+                "status" to response.status.value.toString(),
+            ))
+        }
+    })
+}
+```
+
+- **Repository and sync errors:** the shared sync engine ([mobile-offline-sync](../../mobile-offline-sync/SKILL.md)) is where the most valuable context lives — outbox depth, attempt counts, conflict outcomes. Record it as keys and breadcrumbs from `commonMain` so both apps report it identically.
+- **Coroutine failures:** an exception in a shared `CoroutineScope` with no handler is lost. Install a `CoroutineExceptionHandler` that routes to `Telemetry.recordError`, and remember that a failing child in a plain `Job` cancels the scope — the report must say which.
+- **Spans:** `OSSignposter` on iOS and trace sections on Android are host concerns; expose `startSpan` and let each `actual` map it, so the same span name shows up in Instruments and in a system trace.
+
+## Segmentation
+
+The dimensions in the skill's §4 apply, plus one KMP-specific pair worth setting as keys: the **shared module version** (if it is versioned independently of the apps) and the **target**. "Only on iOS" is the single most useful filter in a KMP dashboard, and it only exists if the target is an attribute on every signal.
+
+## Platform field data
+
+Android vitals, Play Console ANRs, Xcode Organizer and MetricKit all apply unchanged — the shared code runs inside two ordinary apps. See [android.md](android.md) and [ios.md](ios.md). The lab counterpart, including how to tell shared cost from host cost, is [mobile-performance/references/kotlin-multiplatform.md](../../mobile-performance/references/kotlin-multiplatform.md).
+
+## Testing
+
+- A recording fake `Telemetry` in `commonTest` asserts that a failing sync drain records the error once, with the expected keys, and does not log the payload.
+- The redaction rules are shared code: test them against a payload containing a token, on both JVM and Native test targets.
+- Sampling decisions should be deterministic given an injected seed or clock, so the test can assert "a failed session is always kept".
+
+## Official documentation
+
+- Kotlin Multiplatform: https://kotlinlang.org/docs/multiplatform.html
+- Kotlin/Native and Swift/Objective-C interop: https://kotlinlang.org/docs/native-objc-interop.html
+- Ktor client plugins: https://ktor.io/docs/client-custom-plugins.html
+- Crashlytics: https://firebase.google.com/docs/crashlytics
+- MetricKit: https://developer.apple.com/documentation/metrickit

package/skills/mobile-observability/references/react-native.md

@@ -0,0 +1,62 @@
+# Observability in React Native and Expo
+
+A React Native app fails in two runtimes. A **JavaScript error** produces a red screen in development and, in production, either a caught error boundary or a fatal that the native crash reporter never sees on its own. A **native crash** produces a normal Android/iOS crash with no JavaScript stack. A setup that covers only one of them explains half the incidents — and the half it misses is usually the harder one.
+
+## Cover both runtimes
+
+Use a reporter with a React Native SDK (`@sentry/react-native`, Bugsnag, Crashlytics via `@react-native-firebase/crashlytics`, Embrace). What to check in the configuration:
+
+- **JavaScript fatals are reported as fatals**, not swallowed by a global handler that logs and continues. `ErrorUtils.setGlobalHandler` is the underlying mechanism; the SDK wraps it, and an app that installs its own handler afterwards silently disables reporting.
+- **Unhandled promise rejections** are captured. In Hermes they do not reach the global error handler by default, and an SDK option or a rejection tracker is needed.
+- **Native crashes** are captured by the SDK's Android and iOS layers — which means the native install steps (autolinking plus any Gradle/Podfile lines the SDK documents) must actually be present. Verify on a release build, not in Metro.
+- **React error boundaries** report and then render a fallback. A boundary that catches and shows a blank screen without reporting is the most common invisible failure in RN apps.
+
+## Source maps are the whole game
+
+A production JavaScript stack trace is minified Hermes bytecode offsets. Without source maps uploaded per build, every report is unreadable:
+
+```bash
+# The release build emits the bundle and its map; upload both, per platform, per version.
+npx react-native bundle --platform android --dev false \
+  --entry-file index.js \
+  --bundle-output android/app/build/generated/.../index.android.bundle \
+  --sourcemap-output index.android.bundle.map
+```
+
+- The SDK's Gradle plugin / Xcode build phase usually automates the upload; confirm it ran for the shipped variant, and that the **release identifier** (version + build number + bundle hash) matches what the app reports at runtime. A mismatched release is the usual reason a correctly uploaded map still does not symbolicate.
+- **Expo:** `eas build` uploads source maps when the SDK's config plugin is installed; **`eas update` publishes a new bundle without a new build**, so each OTA update needs its own source map upload and its own release identifier. An OTA-heavy app with build-time-only uploads has unreadable traces for every update after the first.
+- Android also needs the R8 mapping file and iOS the dSYMs for native frames. Three artifacts, as in every cross-platform stack.
+
+## OTA updates change what "app version" means
+
+With `expo-updates` or CodePush, the native build and the JavaScript bundle version independently. Every signal must carry **both** — the native version/build and the update/bundle id — or you cannot tell which JavaScript is actually running on a device reporting a crash. Set the update id as a custom key at startup and include it in the release identifier used for source maps.
+
+This is the single most important RN-specific item in the skill's §4 segmentation list.
+
+## Context and breadcrumbs
+
+- **Navigation:** React Navigation's `onStateChange`, or Expo Router's equivalent, produces a screen breadcrumb per transition. Most SDKs ship an integration that does it for you.
+- **Network:** wrap `fetch`/`axios` once to generate the request id, add the `X-Request-Id` header, and record method, host, status and duration. SDK network breadcrumbs are on by default in some vendors and capture full URLs — review and restrict that before shipping, since query strings carry tokens.
+- **Custom keys:** screen, auth state, flag values, update id, connectivity class from NetInfo, outbox depth.
+- **Redux/Zustand/TanStack Query:** an action or query-error middleware that forwards failures with the state slice that caused them. Beware the vendor Redux integrations that attach the **entire state tree** to every event — that is both a cost and a privacy problem.
+
+## Performance and native field data
+
+- The RN SDKs report app-start, slow/frozen frames and, in newer versions, some JS-thread metrics. Treat them as indicators and read the platform's own data as the truth.
+- Android vitals and Play Console ANR data, Xcode Organizer and MetricKit all still apply and still see the real process — see [android.md](android.md) and [ios.md](ios.md). An ANR in a React Native app is an Android ANR, reported by Play, and invisible from JavaScript.
+- The lab counterparts are in [mobile-performance/references/react-native.md](../../mobile-performance/references/react-native.md).
+
+## Privacy and consent
+
+- The Play Data safety form and the iOS privacy nutrition label plus `PrivacyInfo.xcprivacy` must cover the native SDKs the JS packages pull in. Expo config plugins add native SDKs you never wrote a line against.
+- Gate initialization on consent; most SDKs support initializing with sending disabled and enabling it later.
+- Implement `beforeSend` (or the vendor's equivalent) to strip tokens, bodies and query strings, and unit-test it — it is plain JavaScript, so this is a cheap test with real value.
+- `console.log` left in production ships strings into the device log and, with some SDK configurations, into breadcrumbs. Strip logs in release (`babel-plugin-transform-remove-console` or equivalent) or make the logger deliberate.
+
+## Official documentation
+
+- React Native source maps and symbolication: https://reactnative.dev/docs/symbolication
+- Expo EAS Update: https://docs.expo.dev/eas-update/introduction/
+- React Native Firebase Crashlytics: https://rnfirebase.io/crashlytics/usage
+- Sentry for React Native: https://docs.sentry.io/platforms/react-native/
+- NetInfo: https://github.com/react-native-netinfo/react-native-netinfo