@cometchat/skills 4.0.0 → 4.2.0

package/skills/cometchat-react-calls/references/picture-in-picture.md

@@ -0,0 +1,245 @@
+# Picture-in-Picture on web
+
+Web has two PiP APIs, used for different things:
+
+1. **Video PiP** (`HTMLVideoElement.requestPictureInPicture()`) — lets a single `<video>` element float in a system-managed window above all browser tabs. Standard since Chrome 70 / Safari 13 / Firefox 71. Fine for one remote participant.
+
+2. **Document PiP** (`window.documentPictureInPicture.requestWindow()`) — lets you put arbitrary HTML (custom call UI with controls, multi-tile grid, roster) in a floating window. Chrome 116+ only. Falls back gracefully where unsupported.
+
+This reference covers both, plus when to pick which.
+
+---
+
+## When to use Video PiP vs Document PiP
+
+| Scenario | Pick |
+|---|---|
+| 1:1 call, just want the remote face floating while user works | Video PiP |
+| Multi-party call, want the active speaker + a small roster floating | Document PiP if Chrome 116+, else fall back to Video PiP |
+| Want call controls (mute/end) visible in the PiP window | Document PiP only — Video PiP doesn't allow custom controls |
+| Cross-browser support including Safari + Firefox | Video PiP (with fallback when neither works) |
+
+The skill defaults to Video PiP for cross-browser compatibility; Document PiP is opt-in for Chromium-only apps.
+
+---
+
+## Video PiP — the simple path
+
+```tsx
+// CustomOngoingCallView.tsx — extends the version in references/custom-ui.md
+const remoteVideoRef = useRef<HTMLVideoElement>(null);
+const [pipActive, setPipActive] = useState(false);
+
+async function enterPiP() {
+  const video = remoteVideoRef.current;
+  if (!video) return;
+  if (!document.pictureInPictureEnabled) {
+    setError("Picture-in-Picture isn't supported in this browser");
+    return;
+  }
+  try {
+    await video.requestPictureInPicture();
+    setPipActive(true);
+  } catch (err) {
+    // user denied, video not yet playing, etc.
+    console.warn("PiP request failed:", err);
+  }
+}
+
+useEffect(() => {
+  const video = remoteVideoRef.current;
+  if (!video) return;
+  const onEnter = () => setPipActive(true);
+  const onLeave = () => setPipActive(false);
+  video.addEventListener("enterpictureinpicture", onEnter);
+  video.addEventListener("leavepictureinpicture", onLeave);
+  return () => {
+    video.removeEventListener("enterpictureinpicture", onEnter);
+    video.removeEventListener("leavepictureinpicture", onLeave);
+  };
+}, []);
+```
+
+The `<video>` element keeps playing — PiP doesn't pause or remount. CSS doesn't apply (the OS owns the floating window). Hide the in-page video when PiP is active to avoid the "two videos playing" UX:
+
+```tsx
+<video
+  ref={remoteVideoRef}
+  autoPlay
+  playsInline
+  style={{ display: pipActive ? "none" : "block" }}
+/>
+```
+
+---
+
+## Document PiP — the rich path (Chrome 116+)
+
+```tsx
+const [pipWindow, setPipWindow] = useState<Window | null>(null);
+
+async function enterDocumentPiP() {
+  // Feature detect
+  if (!("documentPictureInPicture" in window)) {
+    return enterPiP();      // fall through to video PiP
+  }
+
+  const pipWin = await (window as unknown as {
+    documentPictureInPicture: { requestWindow: (opts: { width: number; height: number }) => Promise<Window> };
+  }).documentPictureInPicture.requestWindow({
+    width: 360,
+    height: 480,
+  });
+
+  // Copy the call container into the PiP window
+  const container = document.getElementById("ongoing-call-root");
+  if (container) {
+    pipWin.document.body.appendChild(container);
+  }
+
+  // PiP window has its own document — copy stylesheets so kit styling works
+  for (const styleSheet of Array.from(document.styleSheets)) {
+    try {
+      const cssRules = Array.from(styleSheet.cssRules ?? []).map((r) => r.cssText).join("\n");
+      const style = pipWin.document.createElement("style");
+      style.textContent = cssRules;
+      pipWin.document.head.appendChild(style);
+    } catch {
+      // cross-origin stylesheets throw — copy <link> href instead
+      if (styleSheet.href) {
+        const link = pipWin.document.createElement("link");
+        link.rel = "stylesheet";
+        link.href = styleSheet.href;
+        pipWin.document.head.appendChild(link);
+      }
+    }
+  }
+
+  // When the user closes the PiP window (system X button), restore the container
+  pipWin.addEventListener("pagehide", () => {
+    const restored = pipWin.document.getElementById("ongoing-call-root");
+    if (restored && document.getElementById("call-host")) {
+      document.getElementById("call-host")!.appendChild(restored);
+    }
+    setPipWindow(null);
+  });
+
+  setPipWindow(pipWin);
+}
+```
+
+The `container` keeps its event handlers and React fiber attached — clicking "End" inside the PiP window still calls your React handlers. This is the magic of Document PiP that single-video PiP doesn't give you.
+
+**Caveat:** stylesheets are copied at PiP-open time. If you change the theme mid-PiP (light/dark toggle), styles in the PiP window go stale. Add a `MutationObserver` or just don't allow theme switching while PiP is active.
+
+---
+
+## Browser support matrix
+
+| Browser | Video PiP | Document PiP |
+|---|---|---|
+| Chrome 70+ desktop | ✓ | Chrome 116+ |
+| Edge 79+ desktop | ✓ | Edge 116+ |
+| Safari 13+ desktop | ✓ | ✗ (no plans yet) |
+| Firefox 71+ desktop | ✓ (custom toggle UI, not standard API) | ✗ |
+| Chrome mobile (Android) | ✓ system-PiP equivalent | ✗ |
+| Safari iOS | iPad: ✓; iPhone: limited | ✗ |
+
+Feature-detect both. Don't render the "Enter PiP" button when neither is supported.
+
+```tsx
+const canVideoPiP = typeof document !== "undefined" && document.pictureInPictureEnabled;
+const canDocPiP = typeof window !== "undefined" && "documentPictureInPicture" in window;
+const showPipButton = canVideoPiP || canDocPiP;
+```
+
+---
+
+## Auto-enter PiP on tab switch
+
+A "tab visibility" pattern many call UX teams want — auto-enter PiP when the user switches away from the call tab:
+
+```tsx
+useEffect(() => {
+  const onVisibilityChange = () => {
+    if (document.visibilityState === "hidden" && remoteVideoRef.current) {
+      remoteVideoRef.current.requestPictureInPicture().catch(() => {});
+    }
+  };
+  document.addEventListener("visibilitychange", onVisibilityChange);
+  return () => document.removeEventListener("visibilitychange", onVisibilityChange);
+}, []);
+```
+
+Browsers reject auto-PiP requests not tied to user gestures in some contexts (Safari is strictest). Use `await navigator.mediaSession.setActionHandler("enterpictureinpicture", ...)` for a cleaner API where supported.
+
+---
+
+## Auto-leave PiP on hangup
+
+When the call ends, exit PiP cleanly:
+
+```tsx
+function endCall() {
+  if (document.pictureInPictureElement) {
+    document.exitPictureInPicture();
+  }
+  if (pipWindow) {
+    pipWindow.close();
+    setPipWindow(null);
+  }
+  CometChatCalls.leaveSession();
+  // ...rest of cleanup
+}
+```
+
+Without this, the PiP window stays floating after the call ends, showing a frozen frame.
+
+---
+
+## PiP + custom UI integration
+
+If you're using `enableDefaultLayout(true)` (kit-rendered call UI), PiP works on the kit's internal `<video>` element. Reach into it via:
+
+```ts
+const callContainer = document.getElementById("calls-container");
+const video = callContainer?.querySelector("video");   // kit renders one or more
+if (video instanceof HTMLVideoElement) {
+  await video.requestPictureInPicture();
+}
+```
+
+Brittle — kit DOM structure can change between versions. Custom UI (Document PiP path above) is more stable.
+
+---
+
+## Anti-patterns
+
+1. **Calling `requestPictureInPicture()` from `useEffect` on mount.** Browsers reject — must be in response to user gesture. Wire to a button.
+2. **Forgetting to hide the in-page `<video>` while PiP is active.** Two videos play, audio doubles, layout breaks.
+3. **Document PiP without copying stylesheets.** PiP window renders unstyled; user sees raw HTML.
+4. **Not exiting PiP on hangup.** Frozen frame floats after call ends.
+5. **Document PiP detection via `'documentPictureInPicture' in document`.** It's on `window`, not `document`. Common typo.
+6. **Auto-PiP on every visibility change, including page reload.** User reloads → unintended PiP. Gate on call active + user-initiated focus loss.
+
+---
+
+## Verification checklist
+
+- [ ] PiP button only renders if `document.pictureInPictureEnabled` OR `'documentPictureInPicture' in window`
+- [ ] PiP request triggered from a click handler, not `useEffect`
+- [ ] In-page video hidden while PiP active (or repositioned)
+- [ ] `enterpictureinpicture` / `leavepictureinpicture` listeners update local state
+- [ ] Hangup path calls `document.exitPictureInPicture()` if active
+- [ ] Document PiP path copies stylesheets to the PiP window's document
+- [ ] Document PiP path restores the container to the main window on `pagehide`
+- [ ] Real-browser smoke: Chrome desktop (both APIs) + Safari desktop (Video PiP) + Firefox (Video PiP)
+
+---
+
+## Pointers
+
+- Custom UI integration: `references/custom-ui.md`
+- Kit-default layout: kit handles internal video PiP via the kit's own controls
+- Document PiP spec: https://wicg.github.io/document-picture-in-picture/
+- `cometchat-react-calls` SKILL.md — base hard rules

package/skills/cometchat-react-calls/references/raise-hand.md

@@ -0,0 +1,238 @@
+# Raise hand on web
+
+Lets participants signal they want to speak without interrupting the current speaker. The SDK ships first-party support — four method calls, two events, one settings flag. No custom signaling needed.
+
+**Canonical docs:** https://www.cometchat.com/docs/calls/javascript/raise-hand
+**Use it for:** classrooms, large group calls, town halls, any call with > ~5 participants where verbal turn-taking gets messy.
+
+---
+
+## SDK API (web Calls SDK)
+
+```ts
+import { CometChatCalls } from "@cometchat/calls-sdk-javascript";
+
+// Local user raises hand
+CometChatCalls.raiseHand();
+
+// Local user lowers hand
+CometChatCalls.lowerHand();
+
+// Subscribe to other participants' hand state
+CometChatCalls.addEventListener("onParticipantHandRaised", (participant) => {
+  // participant.uid, participant.name available
+});
+
+CometChatCalls.addEventListener("onParticipantHandLowered", (participant) => {
+  // ...
+});
+```
+
+The SDK ships a built-in raise-hand button in the default control panel. Hide it via call settings if you're rolling custom UI:
+
+```ts
+const callSettings = new CometChatCalls.CallSettingsBuilder()
+  .setSessionID(sessionId)
+  .hideRaiseHandButton(true)         // suppress the SDK's button — your UI takes over
+  .build();
+```
+
+---
+
+## When to use built-in vs custom
+
+| Scenario | Use |
+|---|---|
+| Default kit UI is fine; just want raise-hand | Built-in (don't pass `hideRaiseHandButton`) |
+| Custom call UI (your own control panel) | Custom — call `raiseHand()` / `lowerHand()` from your buttons |
+| Need different host vs participant UI | Custom — query group scope, render different controls |
+| Need raise-hand list (host sees who's raised) | Custom — maintain local Map<uid, raisedAt> via the listeners |
+
+---
+
+## Custom raise-hand UX — three pieces
+
+### 1. Local participant button (toggle)
+
+```tsx
+import { useState } from "react";
+import { CometChatCalls } from "@cometchat/calls-sdk-javascript";
+
+function RaiseHandButton() {
+  const [raised, setRaised] = useState(false);
+
+  function toggle() {
+    if (raised) {
+      CometChatCalls.lowerHand();
+      setRaised(false);
+    } else {
+      CometChatCalls.raiseHand();
+      setRaised(true);
+    }
+  }
+
+  return (
+    <button onClick={toggle} aria-pressed={raised}>
+      {raised ? "✋ Lower" : "✋ Raise hand"}
+    </button>
+  );
+}
+```
+
+Visual hint: render the icon with `aria-pressed={raised}` so screen readers announce the toggle state. (See `cometchat-a11y` for the broader rule.)
+
+### 2. Raised-hands roster (host view)
+
+```tsx
+import { useEffect, useState } from "react";
+import { CometChatCalls } from "@cometchat/calls-sdk-javascript";
+
+interface RaisedParticipant { uid: string; name: string; raisedAt: number; }
+
+function RaisedHandsList() {
+  const [raised, setRaised] = useState<Map<string, RaisedParticipant>>(new Map());
+
+  useEffect(() => {
+    const onRaised = (p: { uid: string; name: string }) => {
+      setRaised(prev => {
+        const next = new Map(prev);
+        next.set(p.uid, { ...p, raisedAt: Date.now() });
+        return next;
+      });
+    };
+    const onLowered = (p: { uid: string }) => {
+      setRaised(prev => {
+        const next = new Map(prev);
+        next.delete(p.uid);
+        return next;
+      });
+    };
+
+    CometChatCalls.addEventListener("onParticipantHandRaised", onRaised);
+    CometChatCalls.addEventListener("onParticipantHandLowered", onLowered);
+
+    return () => {
+      CometChatCalls.removeEventListener("onParticipantHandRaised", onRaised);
+      CometChatCalls.removeEventListener("onParticipantHandLowered", onLowered);
+    };
+  }, []);
+
+  // Sort oldest-first — fairness queue
+  const sorted = Array.from(raised.values()).sort((a, b) => a.raisedAt - b.raisedAt);
+  if (sorted.length === 0) return null;
+
+  return (
+    <ul aria-label="Raised hands queue">
+      {sorted.map(p => (
+        <li key={p.uid}>
+          ✋ {p.name} <span style={{ color: "#888" }}>{secondsAgo(p.raisedAt)}</span>
+        </li>
+      ))}
+    </ul>
+  );
+}
+
+function secondsAgo(t: number) { return `${Math.round((Date.now() - t) / 1000)}s ago`; }
+```
+
+Sort by `raisedAt` ascending = first-raised-first-called, which feels fair to participants. Don't sort alphabetically.
+
+### 3. Toast notification for the host
+
+```tsx
+useEffect(() => {
+  const onRaised = (p: { name: string }) => {
+    toast.info(`${p.name} raised their hand`, { duration: 4000 });
+  };
+  CometChatCalls.addEventListener("onParticipantHandRaised", onRaised);
+  return () => CometChatCalls.removeEventListener("onParticipantHandRaised", onRaised);
+}, []);
+```
+
+Use `aria-live="polite"` on the toast region so screen readers announce — same a11y pattern as new-message announcements (cf. `cometchat-a11y`).
+
+---
+
+## Lower-by-host pattern
+
+The SDK exposes `lowerHand()` only for the local user. To let a host lower someone else's hand, you need a moderator action via the participant-management API:
+
+```ts
+// Host action — requires moderator/admin scope on the group
+async function lowerParticipantHand(uid: string) {
+  // SDK doesn't expose remoteLowerHand directly. Two options:
+  // A) Send a custom message to the participant; their client lowers itself
+  // B) Use the moderator mute/kick API as the boundary
+
+  // Option A: lightweight, requires the participant's client to listen
+  await CometChat.sendCustomMessage(new CometChat.CustomMessage(
+    uid, CometChat.RECEIVER_TYPE.USER, "lower_hand", {}
+  ));
+}
+```
+
+On the receiving side:
+
+```ts
+CometChat.addMessageListener("raise-hand-control", new CometChat.MessageListener({
+  onCustomMessageReceived: (msg) => {
+    if (msg.getType() === "lower_hand") {
+      CometChatCalls.lowerHand();
+    }
+  },
+}));
+```
+
+This is application-level signaling, not SDK-built-in. Document the contract in your team's call protocols.
+
+---
+
+## Hide button on rendered surfaces
+
+If using the kit's `<CometChatOngoingCall />` and want raise-hand off entirely (e.g. 1:1 calls don't need it):
+
+```tsx
+import { CometChatCalls } from "@cometchat/calls-sdk-javascript";
+
+const settings = new CometChatCalls.CallSettingsBuilder()
+  .setSessionID(sessionId)
+  .hideRaiseHandButton(true)
+  .build();
+```
+
+For 1:1 calls, default to hidden. For group calls > 5 participants, default to shown.
+
+---
+
+## Anti-patterns
+
+1. **Polling for raised-hand state.** The SDK fires events on change — listen, don't poll. Polling burns CPU.
+2. **Sorting raised-hands alphabetically.** First-raised-first-called is the fair queue. Sort by `raisedAt` ascending.
+3. **Auto-lowering hands after a timer.** Some people raise their hand and wait. Letting the SDK manage state means the participant lowers when called or via their own button.
+4. **Showing raise-hand button in 1:1 calls.** Visually noisy and pointless. Gate on `participantCount > 2`.
+5. **Skipping the listener cleanup in `useEffect`'s return.** Stacked listeners fire multiple times when the component re-mounts — duplicate toasts, duplicate roster entries.
+6. **Treating "hand raised" as a permission grant.** Raise-hand is a request, not a mute override. The host still has to unmute the participant separately.
+
+---
+
+## Verification checklist
+
+- [ ] `raiseHand()` / `lowerHand()` calls in your code (not just relying on the SDK button)
+- [ ] Both `onParticipantHandRaised` and `onParticipantHandLowered` listeners registered
+- [ ] Listeners cleaned up in component unmount (return from `useEffect`)
+- [ ] Roster sorted by `raisedAt` ascending
+- [ ] `hideRaiseHandButton: true` in call settings IF custom UI is used (otherwise duplicates)
+- [ ] Toast / badge UI uses `aria-live="polite"` (a11y)
+- [ ] Browser smoke: 3 tabs, hand-raise from 2 of them, host's roster shows both in raise-order
+- [ ] Lower-hand smoke: tab A raises, lowers, host's roster updates without page refresh
+
+---
+
+## Pointers
+
+- `cometchat-react-calls` SKILL.md — the seven hard rules (still apply for raise-hand UI)
+- `references/group-calls.md` — group call architecture (raise-hand is a group-call feature)
+- `references/custom-ui.md` — custom call UI patterns
+- `cometchat-a11y` — toast announcements for raised-hand events
+- Canonical docs: https://www.cometchat.com/docs/calls/javascript/raise-hand
+- For deeper SDK reference (other event types, presenter-mode interplay): query the docs MCP at `https://www.cometchat.com/docs/mcp`

package/skills/cometchat-react-calls/references/recording-screen-share.md

@@ -0,0 +1,143 @@
+# Recording + screen sharing on web
+
+Both features ship with the Calls SDK; both have web-specific gotchas the kit's defaults don't handle.
+
+---
+
+## Recording
+
+### Server-side: enable in the dashboard first
+
+Recording is a paid feature gated by your CometChat plan. Enable it in **Dashboard → Chat & Messaging → Calls → Recording**. Without that, the client-side flag below is a no-op.
+
+### Client-side: opt-in per session
+
+```ts
+const settings = new CometChatCalls.CallSettingsBuilder()
+  .setSessionID(sessionId)
+  .setIsAudioOnly(false)
+  .enableRecording(true)                       // ← server starts recording when session begins
+  .setShowRecordingButton(true)                // ← user-toggleable mid-call
+  .build();
+```
+
+Two flags, two behaviors:
+
+- `enableRecording(true)` — recording starts the moment the session begins. Server-side flag.
+- `setShowRecordingButton(true)` — exposes a "Record" toggle in the default control panel. User decides when to start/stop. Custom-UI code must wire its own button.
+
+**Compliance note:** in some jurisdictions you must notify all participants before recording starts. The default kit UI shows a small "Recording" indicator; if you're using custom UI, you must render this yourself. The skill's verification checklist flags this.
+
+### Recording lifecycle events
+
+```ts
+const listener = new CometChatCalls.OngoingCallListener({
+  onRecordingStarted: (rec: unknown) => {
+    // server confirmed recording is active
+  },
+  onRecordingStopped: (rec: unknown) => {
+    // server stopped — file will appear in dashboard within ~30 seconds
+  },
+  onRecordingFailed: (error: unknown) => {
+    // surface to UI — usually plan limits or storage quota
+  },
+});
+```
+
+### Where the recordings go
+
+CometChat hosts the file. It appears in **Dashboard → Calls → Recordings** with a download link. The skill points users at the dashboard path; there is no client-side download API.
+
+---
+
+## Screen sharing
+
+### Two roles: presenter + viewer
+
+- **Presenter** (the user sharing their screen) — calls `CometChatCalls.startScreenShare()` and receives a `MediaStream` from `getDisplayMedia`
+- **Viewer** (everyone else) — sees the presenter's screen as another video tile, no special API call needed
+
+Browser support: Chrome/Edge (full), Firefox (full), Safari 13+ (full). On mobile browsers, `getDisplayMedia` is supported on iOS 16+ Safari and recent Android Chrome.
+
+### Presenter — start sharing
+
+```ts
+async function startScreenShare() {
+  try {
+    await CometChatCalls.startScreenShare();
+    // SDK handled getDisplayMedia + signaling; UI updates via onScreenShareStarted
+  } catch (err: unknown) {
+    if ((err as Error).name === "NotAllowedError") {
+      // user clicked "Cancel" on the picker — no error UI needed
+      return;
+    }
+    setError("Couldn't start screen share");
+  }
+}
+```
+
+Stop sharing:
+
+```ts
+CometChatCalls.endScreenShare();
+```
+
+The browser also fires its own "Stop sharing" button (the system overlay Chrome shows during a screen-share). The SDK listens for this too; `onScreenShareEnded` fires either way.
+
+### Viewer — listen for screen-share events
+
+```ts
+const listener = new CometChatCalls.OngoingCallListener({
+  onScreenShareStarted: (presenterUid: string, stream: MediaStream) => {
+    // attach the stream to a <video> element
+    if (screenShareVideoRef.current) {
+      screenShareVideoRef.current.srcObject = stream;
+    }
+  },
+  onScreenShareEnded: () => {
+    if (screenShareVideoRef.current) {
+      screenShareVideoRef.current.srcObject = null;
+    }
+  },
+});
+```
+
+Compose the screen-share tile alongside the camera tiles in your custom layout.
+
+### Audio passthrough during screen share
+
+By default, `getDisplayMedia` captures video only. To capture system audio (for sharing a video with sound), pass `audio: true`:
+
+```ts
+// Browser-level API — the SDK's startScreenShare wraps this internally
+const stream = await navigator.mediaDevices.getDisplayMedia({
+  video: true,
+  audio: true,
+});
+```
+
+Browser support is uneven — Chrome desktop has it; Firefox does not; Safari has it for tab capture but not full-screen.
+
+The Calls SDK's `startScreenShare()` does NOT request audio by default in v4. If you need audio passthrough, use the lower-level `getDisplayMedia` API directly + pipe the audio track via a custom track-add hook (covered in `custom-ui.md`).
+
+---
+
+## Combining recording + screen-share
+
+Server-side recording captures the active video composition, including screen-share when a participant is sharing. The recording file is one MP4 with the layout the kit was rendering at the time.
+
+If you're using custom UI, the recording captures what the SDK sends to the server — not your custom DOM. The composition is determined by the SDK's internal layout, not your CSS.
+
+---
+
+## Browser permissions for screen-share
+
+Like `getUserMedia`, `getDisplayMedia` requires HTTPS or `localhost`. It also requires an active user gesture (click/tap) — you cannot start it from a `useEffect` or timer. The skill scaffolds the API call inside an onClick handler.
+
+System-level: macOS 10.15+ asks the user once to grant Chrome/Safari/Firefox permission to record the screen (System Preferences → Security & Privacy → Screen Recording). If the user denies, `getDisplayMedia` throws `NotAllowedError` with no remediation path inside the browser — surface a "Open System Preferences" instruction.
+
+---
+
+## Cleanup
+
+Both recording and screen-share are part of the call session. `CometChatCalls.leaveSession()` stops both automatically. Custom UI must NOT separately call `getTracks().forEach(t => t.stop())` on the SDK's screen-share stream — the SDK owns it. Stop only the streams YOUR code created (e.g. local preview).