@cometchat/skills 4.1.0 → 4.2.0

package/skills/cometchat-native-calls/references/picture-in-picture.md

@@ -0,0 +1,276 @@
+# Picture-in-Picture on React Native
+
+Both Android and iOS support PiP for video calls — but the integration paths are different. Android exposes `PictureInPictureParams` via `Activity` APIs; iOS exposes `AVPictureInPictureController` (and CallKit handles it for VoIP calls). RN bridges both via native modules.
+
+---
+
+## Android — Activity-level PiP
+
+Android PiP is an Activity attribute. The activity declares it supports PiP, and you transition into PiP by calling `enterPictureInPictureMode()` from native (RN exposes a bridge).
+
+### Manifest
+
+```xml
+<!-- android/app/src/main/AndroidManifest.xml -->
+<activity
+  android:name=".MainActivity"
+  android:supportsPictureInPicture="true"
+  android:configChanges="screenSize|smallestScreenSize|screenLayout|orientation|keyboardHidden">
+  <!-- ... -->
+</activity>
+```
+
+`configChanges` is critical — without it, Android destroys + recreates the activity on PiP transition, which kills the WebRTC connection.
+
+### Bridge
+
+`react-native-webrtc` ships a helper module (in some versions) for entering PiP. If your version doesn't have it, write a tiny native module:
+
+```kotlin
+// android/app/src/main/java/com/yourapp/PictureInPictureModule.kt
+package com.yourapp
+
+import android.app.PictureInPictureParams
+import android.os.Build
+import android.util.Rational
+import com.facebook.react.bridge.ReactApplicationContext
+import com.facebook.react.bridge.ReactContextBaseJavaModule
+import com.facebook.react.bridge.ReactMethod
+import com.facebook.react.bridge.Promise
+
+class PictureInPictureModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaModule(reactContext) {
+  override fun getName() = "PictureInPicture"
+
+  @ReactMethod
+  fun enterPip(promise: Promise) {
+    val activity = currentActivity
+    if (activity == null) {
+      promise.reject("NO_ACTIVITY", "No current activity")
+      return
+    }
+    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) {
+      promise.reject("UNSUPPORTED", "Android < 8.0 doesn't support PiP")
+      return
+    }
+
+    val aspectRatio = Rational(16, 9)
+    val params = PictureInPictureParams.Builder()
+      .setAspectRatio(aspectRatio)
+      .build()
+
+    val ok = activity.enterPictureInPictureMode(params)
+    if (ok) promise.resolve(null) else promise.reject("FAILED", "Couldn't enter PiP")
+  }
+}
+```
+
+Register in `MainApplication.kt`:
+
+```kotlin
+override fun getPackages(): List<ReactPackage> {
+  val packages = PackageList(this).packages
+  packages.add(PictureInPicturePackage())          // your package wrapping the module
+  return packages
+}
+```
+
+### React side
+
+```ts
+import { NativeModules, Platform } from "react-native";
+
+const PictureInPicture = NativeModules.PictureInPicture;
+
+async function enterPiP() {
+  if (Platform.OS !== "android") return;
+  try {
+    await PictureInPicture.enterPip();
+  } catch (err) {
+    console.warn("PiP failed:", err);
+  }
+}
+```
+
+### Listening for PiP transitions
+
+On Android, the activity gets `onPictureInPictureModeChanged` callback. RN exposes this via `AppState`:
+
+```ts
+import { AppState, type AppStateStatus } from "react-native";
+
+useEffect(() => {
+  const handler = (state: AppStateStatus) => {
+    // 'background' fires when entering PiP on some Android versions
+    // 'active' fires on returning
+    if (state === "active") {
+      // restore full UI
+    }
+  };
+  const sub = AppState.addEventListener("change", handler);
+  return () => sub.remove();
+}, []);
+```
+
+Better — emit a custom event from your native module on `onPictureInPictureModeChanged` and listen in JS:
+
+```ts
+import { NativeEventEmitter, NativeModules } from "react-native";
+
+const emitter = new NativeEventEmitter(NativeModules.PictureInPicture);
+useEffect(() => {
+  const sub = emitter.addListener("pipModeChanged", ({ inPip }: { inPip: boolean }) => {
+    setInPip(inPip);
+  });
+  return () => sub.remove();
+}, []);
+```
+
+---
+
+## iOS — CallKit handles it for VoIP calls
+
+The simplest iOS PiP path: **don't write any.** If you've integrated CallKit (rule 1.2 in the SKILL.md, mandatory for standalone-mode VoIP), iOS auto-handles PiP for active video calls when the user backgrounds the app. The system shows your CallKit-reported call in a small floating window.
+
+For non-CallKit calls (in-foreground only), you need `AVPictureInPictureController`. This is involved on iOS because `AVPictureInPictureController` requires an `AVSampleBufferDisplayLayer` — it doesn't work directly with `<RTCView />`.
+
+### AVPictureInPictureController bridge
+
+`react-native-webrtc` ≥ 119 ships an `RTCVideoView` that supports PiP via a system-managed display layer. The bridge:
+
+```ts
+import { RTCView } from "react-native-webrtc";
+
+// react-native-webrtc 119+
+<RTCView
+  streamURL={remoteStream.toURL()}
+  style={styles.remoteVideo}
+  pictureInPicture={true}
+  onPictureInPictureStart={() => setInPip(true)}
+  onPictureInPictureStop={() => setInPip(false)}
+/>
+```
+
+Older versions: write a wrapper around `AVPictureInPictureController` and a `WebRTCVideoView` subclass that exposes its `AVSampleBufferDisplayLayer` to the controller. Substantial native work; deferring to CallKit is the right call for most apps.
+
+### Info.plist
+
+Background modes must include `audio` (rule 1.3 in the SKILL.md). PiP itself doesn't need a separate capability; it inherits from "audio" + active call.
+
+---
+
+## App state preservation across PiP
+
+When the app enters PiP (Android) or background-with-active-call (iOS), the call state must persist. Practical rules:
+
+1. **Don't tear down the WebRTC connection** on `componentWillUnmount` of the call screen. The screen survives PiP; tearing down kills the call.
+2. **Hide the in-page video** when PiP is active — same rule as web. Two videos rendering = bad UX, double battery.
+3. **Restore on PiP exit** — listen to `pipModeChanged` and re-render the full call surface.
+
+```tsx
+function CallScreen() {
+  const [inPip, setInPip] = useState(false);
+
+  return (
+    <View style={[styles.container, inPip && styles.minimalPip]}>
+      <RTCView
+        streamURL={remoteStream?.toURL()}
+        style={inPip ? styles.pipFullSize : styles.normalSize}
+        pictureInPicture={true}
+        onPictureInPictureStart={() => setInPip(true)}
+        onPictureInPictureStop={() => setInPip(false)}
+      />
+      {!inPip && <ControlPanel onEnd={endCall} />}
+    </View>
+  );
+}
+```
+
+The control panel is only visible in non-PiP mode; PiP is video-only by design.
+
+---
+
+## Auto-PiP on app backgrounding
+
+Common UX: user presses Home button or switches apps → call auto-enters PiP so video keeps playing.
+
+### Android
+
+In `MainActivity.kt`:
+
+```kotlin
+override fun onUserLeaveHint() {
+  super.onUserLeaveHint()
+  // User pressed Home — auto-enter PiP if a call is active
+  if (isCallActive() && Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+    enterPictureInPictureMode(PictureInPictureParams.Builder().build())
+  }
+}
+
+private fun isCallActive(): Boolean {
+  // bridge to JS state — store this in a SharedPreferences flag
+  return getSharedPreferences("call", MODE_PRIVATE).getBoolean("active", false)
+}
+```
+
+JS side: write the flag whenever the call starts/ends:
+
+```ts
+import { NativeModules } from "react-native";
+
+useEffect(() => {
+  NativeModules.PictureInPicture.setCallActive(true);
+  return () => NativeModules.PictureInPicture.setCallActive(false);
+}, []);
+```
+
+### iOS
+
+CallKit handles this — if you've reported the call to CallKit (rule 1.2), iOS auto-PiPs on background. No bridge code needed for the auto-PiP path.
+
+For non-CallKit calls (rare in production calling apps), use `react-native-webrtc`'s `pictureInPicture` prop with auto-enter behavior at the native level.
+
+---
+
+## Anti-patterns
+
+1. **Skipping `configChanges` on the manifest activity.** PiP transition destroys + recreates the activity → call dies.
+2. **Tearing down WebRTC in PiP transition.** Same outcome — call dies.
+3. **No iOS Background Modes `audio` capability.** PiP video plays without sound; user thinks it's broken.
+4. **Auto-PiP on every app background, including non-call screens.** User opens settings, comes back to a PiP'd home screen. Gate on `isCallActive()`.
+5. **Rendering control panel inside PiP window.** PiP windows are small; controls don't fit. Hide them.
+6. **Using two separate `<RTCView />` instances** — one for normal, one for PiP. The `pictureInPicture` prop on a single view is the right pattern.
+7. **iOS PiP without CallKit.** Doesn't work for backgrounded calls — only foregrounded. CallKit is required for "real" PiP.
+
+---
+
+## Verification checklist
+
+**Android:**
+- [ ] `android:supportsPictureInPicture="true"` on the activity
+- [ ] `android:configChanges` includes screenSize/smallestScreenSize/screenLayout/orientation
+- [ ] Native PiP module registered + bridged
+- [ ] `onUserLeaveHint` auto-PiP gated on `isCallActive()`
+- [ ] PiP entry uses 16:9 aspect ratio (or matches stream resolution)
+- [ ] PiP mode change emits to JS; controls hidden in PiP
+
+**iOS:**
+- [ ] CallKit integrated (rule 1.2 in SKILL.md)
+- [ ] Background Modes: audio + voip + remote-notification
+- [ ] `react-native-webrtc` ≥ 119 if using `pictureInPicture` prop
+- [ ] PiP transitions don't tear down call
+
+**Real-device smoke:**
+- [ ] Android: press Home during call → PiP appears, video keeps playing, audio continues
+- [ ] Android: tap PiP window → call screen restored, controls visible
+- [ ] iOS: press Home during call → CallKit-managed PiP appears
+- [ ] iOS: tap PiP → app foregrounds, full call screen visible
+
+---
+
+## Pointers
+
+- `references/custom-ui.md` — composing custom call surfaces (PiP integration via the `pictureInPicture` prop pattern)
+- `references/voip-push-end-to-end.md` — CallKit which handles iOS PiP for free
+- `cometchat-android-v5-calls/references/picture-in-picture.md` — Android-native PiP reference (deeper Android-specific patterns)
+- `cometchat-react-calls/references/picture-in-picture.md` — Web PiP (different APIs but same UX shape)
+- `cometchat-native-calls` SKILL.md — base hard rules

package/skills/cometchat-native-calls/references/raise-hand.md

@@ -0,0 +1,203 @@
+# Raise hand on React Native
+
+Same SDK API as web (`@cometchat/calls-sdk-react-native` exposes the same calls). The difference is the UI surface — native components instead of HTML, RN re-renders with state, no `aria-pressed` (use `accessibilityState` instead).
+
+**Canonical docs:** https://www.cometchat.com/docs/calls/react-native/raise-hand
+**Read first:** `cometchat-react-calls/references/raise-hand.md` — the integration shape and anti-patterns are identical; this reference covers RN-specific wiring.
+
+---
+
+## SDK API
+
+```ts
+import { CometChatCalls } from "@cometchat/calls-sdk-react-native";
+
+CometChatCalls.raiseHand();
+CometChatCalls.lowerHand();
+
+CometChatCalls.addEventListener("onParticipantHandRaised", (p) => { ... });
+CometChatCalls.addEventListener("onParticipantHandLowered", (p) => { ... });
+
+const settings = new CometChatCalls.CallSettingsBuilder()
+  .setSessionID(sessionId)
+  .hideRaiseHandButton(true)
+  .build();
+```
+
+---
+
+## RN-specific UX patterns
+
+### Toggle button (TouchableOpacity)
+
+```tsx
+import { useState } from "react";
+import { TouchableOpacity, Text, AccessibilityInfo } from "react-native";
+import { CometChatCalls } from "@cometchat/calls-sdk-react-native";
+
+function RaiseHandButton() {
+  const [raised, setRaised] = useState(false);
+
+  function toggle() {
+    if (raised) {
+      CometChatCalls.lowerHand();
+      setRaised(false);
+      AccessibilityInfo.announceForAccessibility("Hand lowered");
+    } else {
+      CometChatCalls.raiseHand();
+      setRaised(true);
+      AccessibilityInfo.announceForAccessibility("Hand raised");
+    }
+  }
+
+  return (
+    <TouchableOpacity
+      onPress={toggle}
+      accessibilityRole="button"
+      accessibilityState={{ selected: raised }}
+      accessibilityLabel={raised ? "Lower hand" : "Raise hand"}
+      style={{
+        padding: 12,
+        borderRadius: 32,
+        backgroundColor: raised ? "#FFD60A" : "rgba(255,255,255,0.2)",
+      }}
+    >
+      <Text style={{ color: raised ? "#000" : "#FFF" }}>
+        ✋ {raised ? "Lower" : "Raise"}
+      </Text>
+    </TouchableOpacity>
+  );
+}
+```
+
+`AccessibilityInfo.announceForAccessibility` is the RN equivalent of web's `aria-live` regions — TalkBack / VoiceOver pick up the announcement. This is the canonical RN a11y pattern (cf. `cometchat-a11y`).
+
+### Raised-hands roster (BottomSheet pattern)
+
+For a host's "raised hands" list, the natural RN UX is a bottom sheet rather than a permanent sidebar (vertical phone screens are tight on horizontal real estate):
+
+```tsx
+import { useEffect, useRef, useState } from "react";
+import { View, Text, FlatList } from "react-native";
+import BottomSheet from "@gorhom/bottom-sheet";
+import { CometChatCalls } from "@cometchat/calls-sdk-react-native";
+
+interface RaisedParticipant { uid: string; name: string; raisedAt: number; }
+
+function RaisedHandsSheet() {
+  const sheetRef = useRef<BottomSheet>(null);
+  const [raised, setRaised] = useState<Map<string, RaisedParticipant>>(new Map());
+
+  useEffect(() => {
+    const onRaised = (p: { uid: string; name: string }) => {
+      setRaised(prev => new Map(prev).set(p.uid, { ...p, raisedAt: Date.now() }));
+      // Auto-expand the sheet on first hand raised
+      if (raised.size === 0) sheetRef.current?.snapToIndex(1);
+    };
+    const onLowered = (p: { uid: string }) => {
+      setRaised(prev => {
+        const next = new Map(prev);
+        next.delete(p.uid);
+        return next;
+      });
+    };
+
+    CometChatCalls.addEventListener("onParticipantHandRaised", onRaised);
+    CometChatCalls.addEventListener("onParticipantHandLowered", onLowered);
+    return () => {
+      CometChatCalls.removeEventListener("onParticipantHandRaised", onRaised);
+      CometChatCalls.removeEventListener("onParticipantHandLowered", onLowered);
+    };
+  }, [raised.size]);
+
+  const sorted = Array.from(raised.values()).sort((a, b) => a.raisedAt - b.raisedAt);
+
+  return (
+    <BottomSheet ref={sheetRef} snapPoints={["10%", "40%"]} index={raised.size > 0 ? 1 : 0}>
+      <View style={{ padding: 16 }}>
+        <Text style={{ fontWeight: "600", marginBottom: 8 }}>
+          Raised hands ({raised.size})
+        </Text>
+        <FlatList
+          data={sorted}
+          keyExtractor={(p) => p.uid}
+          renderItem={({ item }) => (
+            <Text style={{ paddingVertical: 6 }}>
+              ✋ {item.name}
+            </Text>
+          )}
+        />
+      </View>
+    </BottomSheet>
+  );
+}
+```
+
+`@gorhom/bottom-sheet` is the canonical RN bottom-sheet library; if you've already integrated CometChat the kit's setup includes its peer deps (`react-native-gesture-handler`, `react-native-reanimated`).
+
+### Toast for the host
+
+```tsx
+import Toast from "react-native-toast-message";
+
+useEffect(() => {
+  const onRaised = (p: { name: string }) => {
+    Toast.show({
+      type: "info",
+      text1: `${p.name} raised their hand`,
+      visibilityTime: 4000,
+      position: "top",
+    });
+  };
+  CometChatCalls.addEventListener("onParticipantHandRaised", onRaised);
+  return () => CometChatCalls.removeEventListener("onParticipantHandRaised", onRaised);
+}, []);
+```
+
+---
+
+## When to gate raise-hand UI
+
+Same rule as web: hide for 1:1 calls. Gate on participant count:
+
+```tsx
+const showRaiseHand = participants.length > 2;
+return showRaiseHand ? <RaiseHandButton /> : null;
+```
+
+---
+
+## Anti-patterns
+
+Same shape as web (cf. sister reference) plus RN-specific:
+
+1. **Forgetting `accessibilityRole="button"`.** TouchableOpacity defaults to "image" or no role — TalkBack reads it wrong. Always set the role.
+2. **`Toast.show` from inside the SDK callback without UI-thread guarantee.** Toast lib usually handles this, but if you use a custom toast, wrap in `requestAnimationFrame` or check that the call from the SDK callback is on the JS thread.
+3. **BottomSheet with a single snap-point.** Use 2 snaps so users can collapse without dismissing entirely.
+4. **Animating the raise-hand badge with Reanimated `withSpring`** in a video-call context. CPU-intensive; can stutter the WebRTC pipeline. Use `withTiming` with short durations or skip the animation.
+
+---
+
+## Verification checklist
+
+- [ ] `raiseHand()` / `lowerHand()` wired to your toggle button
+- [ ] Both event listeners registered + cleaned up
+- [ ] `accessibilityState`, `accessibilityRole`, `accessibilityLabel` on the toggle button
+- [ ] `AccessibilityInfo.announceForAccessibility` on state change
+- [ ] Roster sorted by `raisedAt` ascending
+- [ ] `hideRaiseHandButton: true` in CallSettings if custom UI
+- [ ] Real-device smoke: 3 devices in same call, hand-raise from 2 → host's BottomSheet shows both
+- [ ] iOS device: VoiceOver announces "Hand raised" / "Hand lowered"
+- [ ] Android device: TalkBack announces same
+- [ ] Visual at 360×640 (small Android phone) — button + sheet still legible
+
+---
+
+## Pointers
+
+- `cometchat-react-calls/references/raise-hand.md` — sister reference (same SDK, web-flavored UI)
+- `cometchat-native-calls` SKILL.md — the seven hard rules
+- `references/group-calls.md` — group call architecture
+- `references/custom-ui.md` — RN custom call UI
+- `cometchat-a11y` — RN announcement patterns
+- Canonical docs: https://www.cometchat.com/docs/calls/react-native/raise-hand