@graineai/inapp-react-native 0.15.0 → 0.17.0

package/INTEGRATION.md

@@ -8,6 +8,87 @@ Package: `@graineai/inapp-react-native`
 
 ---
 
+## How it works, end to end
+
+Two things travel between your app and the agent, and keeping them straight
+explains most of this document.
+
+**Media** is the customer's voice and the agent's. **Control** is everything the
+agent knows about your app: the screen, who is on it, what just happened, and
+what it is allowed to do about it. They are different problems — media wants
+pacing and loss tolerance, control wants ordered, reliable delivery — so they are
+carried differently, and the SDK is what hides that from you.
+
+```
+  YOUR APP                                    GRAINE
+  ─────────────────────────────────────       ──────────────────────────────
+
+  ┌───────────────────────────────────┐
+  │ your screens                      │
+  │   useGraineScreen(...)  ──────┐   │
+  │   useGraineAction(...)  ──────┤   │
+  │   identify() / trackEvent()   │   │
+  └───────────────────────────────┼───┘
+                                  │  one client, one source of truth
+                                  ▼
+                     ┌─────────────────────────┐
+                     │  GraineInAppClient      │
+                     │  screen · identity      │
+                     │  events · actions       │
+                     │  buildContextFrame()    │
+                     │  maskDeep() ← PII never │
+                     │    leaves the device    │
+                     └───────────┬─────────────┘
+                                 │
+                     ┌───────────▼─────────────┐
+                     │  GraineVoiceLauncher    │
+                     │  mic · lifecycle · relay│
+                     └───────────┬─────────────┘
+                                 │  postMessage
+                     ┌───────────▼─────────────┐
+                     │  hidden WebView (1×1)   │           ┌──────────────┐
+                     │  the audio engine       │══ media ══▶│  the agent   │
+                     │  capture · playback     │           │              │
+                     │  jitter buffer · barge-in│──control ─▶│  runtime     │
+                     └─────────────────────────┘           └──────┬───────┘
+                                                                  │
+                                 ▲                                │
+                                 └──────── app_action ────────────┘
+                                    "open the offers screen"
+                                    (the agent BLOCKS on your answer)
+```
+
+**Why a WebView.** React Native has no `getUserMedia`. The alternative is a
+native audio module, which means a store release for every audio fix — and the
+audio path is exactly where the fixes are. The page is one pixel, invisible and
+unreachable; your customer only ever sees your own UI. See *Why there is no
+LiveKit step* at the end.
+
+### What travels, and when
+
+| Frame | Direction | When |
+|---|---|---|
+| `app_context` | app → agent | Every screen change and field edit, throttled. Masked first. |
+| `app_action` | agent → app | The agent wants to do something. **It blocks on your reply** (~6s). |
+| `app_action_result` | app → agent | Your handler's answer: `ok`, `refused`, or `error`. |
+| `app_event` | app → agent | Something happened that may be worth speaking about unprompted. |
+
+### The one rule that matters
+
+**One client.** `GraineProvider` builds a client unless you hand it one. If your
+app already has a module-level instance — and any app of size does, because
+`identify()` and `trackEvent()` are not all called from React — pass it in:
+
+```tsx
+<GraineProvider client={myClient} autoConnect={false}>
+```
+
+Two clients is the failure that looks like a broken product: your screens write
+the context into one, the launcher reads the other and finds nothing, and the
+agent connects, greets, and says it cannot see the screen.
+
+---
+
 ## What you need
 
 | | |
@@ -89,12 +170,18 @@ export default function App() {
       baseUrl="https://www.graine.ai"
       publishableKey="pk_live_..."
       navigationRef={navigationRef}
+      // The launcher owns the connection — see below.
+      autoConnect={false}
     >
       <NavigationContainer ref={navigationRef}>
         <RootNavigator />
       </NavigationContainer>
 
-      <GraineVoiceLauncher webView={WebView} autoStart>
+      <GraineVoiceLauncher
+        webView={WebView}
+        autoStart
+        onError={(m) => console.warn("[graine]", m)}
+      >
         {() => <GraineAgentBar />}
       </GraineVoiceLauncher>
     </GraineProvider>
@@ -102,6 +189,16 @@ export default function App() {
 }
 ```
 
+**`autoConnect={false}` is not optional here.** The provider opens a connection
+of its own by default, and `GraineVoiceLauncher` opens the one that carries the
+call. Leave both on and the agent answers twice — the customer hears two
+greetings over each other, and you are billed for two conversations. Drop the
+launcher and use text only, and `autoConnect` goes back to its default.
+
+**Pass `onError`.** A rejected key or an agent that has not been enabled for apps
+leaves the launcher rendering nothing at all, forever, with nothing in the
+console. This is the only way to find out.
+
 `navigationRef` is the line that matters. With it the SDK reads the current route
 itself, so the agent knows which screen the customer is on without you calling
 anything per screen — including the screen you add next year when nobody
@@ -153,14 +250,43 @@ specific than a default. Both are masked on the device before they leave it.
 
 ### It reaches the call record, not only the prompt
 
-Traits are stored on the conversation as `app_user`, and the name labels the row
-in Call History. A web call has no phone number to identify it by, so without
-this every in-app conversation shows a session id.
+Traits are stored on the conversation as `app_user`, and they are what labels
+the row in Call History. **A web call has no phone number**, so without this
+every in-app conversation shows a bare session id — a page of uuids.
 
 Read from the LIVE identity rather than the init frame, so a customer who signs
 in halfway through is identified from that moment rather than from a frame that
 predates them.
 
+#### Tag the session with whatever you file people under
+
+You do not have to have a `name`. Call History looks for a label in this order:
+
+`name` → `full_name` / `fullName` → `display_name` / `displayName` → `label` →
+`username` / `user_name` → `customer_name` → `company` → `email` → **any other
+string trait**
+
+So an app that files people by policy number, customer id or internal handle is
+still legible:
+
+```tsx
+identify({ label: `Policy ${policy.number}` });   // → "Policy HDFC-4471"
+identify({ customer_name: acct.holder });          // → "R. Bhanot"
+identify({ crm_id: lead.id });                     // → falls through to the id
+```
+
+Two rules worth knowing:
+
+- **Redacted values are skipped.** `phone` and `email` are masked on the device,
+  so a row never renders `[PHONE]` as somebody's name. Send the phone anyway if
+  it is useful — the agent is told a number is on file without being told the
+  digits — just do not rely on it as the label.
+- **Identity alone is enough.** You do not need `useGraineScreen` for this. An
+  app that only calls `identify()` still tags every conversation.
+
+`identify(null)` on sign-out. An agent still greeting the previous account by
+name after a switch is worse than one that never knew it.
+
 ---
 
 ## Step 3 — Report what is ON the screen
@@ -520,6 +646,55 @@ Echo. The SDK suppresses the agent's own voice from the microphone, but a very
 loud speaker defeats a level-based guard. Lower the volume, or use earphones to
 confirm that is what it is.
 
+**Nothing at all happens, and `onError` never fires either.**
+You did not pass `onError`. Without it a failed session — a rejected key, an
+agent not enabled for apps — leaves the launcher rendering nothing, silently.
+Pass it before you debug anything else.
+
+**On `rtc`: every call reports a fallback to `ws`.**
+`onTransport` gives a `reason`, and `message` carries the server's own sentence.
+
+| `reason` | Means | Retry helps? |
+|---|---|---|
+| `not_configured` | A setting is wrong — see below. Also raised through `onError`. | Never |
+| `unreachable` | No SIP realm on the org, or the SBC is not reachable from this network. | Sometimes |
+| `screen_channel` | Audio connected, the screen channel did not, so the call restarted rather than run blind. | Sometimes |
+
+**`not_configured`: which setting?** The call still works — it is carried on the
+WebSocket — so nothing is on fire, but WebRTC will never be used until this is
+fixed. Ask the mint endpoint directly, without rebuilding anything:
+
+```bash
+curl -i -X POST https://www.graine.ai/api/embed/rtc-session \
+  -H 'Content-Type: application/json' \
+  -d '{"publishableKey":"pk_live_YOUR_KEY"}'
+```
+
+| Response | `code` | Fix |
+|---|---|---|
+| 401 | `key_missing` | No key reached the endpoint. Update the SDK — before 0.17.0 it did not put `pk` in the page URL. |
+| 401 | `key_malformed` | Not a publishable key. It starts with `pk_live_`, from Agent → Embed & Widgets. |
+| 401 | `key_unknown` | No agent registered to that key. |
+| 403 | `no_origin` | **The common one for apps.** A native app sends no `Origin`, so add the literal entry `app://` to that agent's Allowed domains. |
+| 403 | `no_domains_configured` | The allowlist is empty, which denies everything. |
+| 403 | `origin_not_allowed` | Add your site to Allowed domains. |
+| 503 | `not_configured` | WebRTC is not enabled on that deployment. Not fixable from the app. |
+| 200 | — | The mint works; look at `onTransport` for `unreachable` instead. |
+
+That curl sends **no** `Origin` header, exactly as a native app does — so a 403
+from it is precisely the 403 your app is getting.
+
+**On `rtc`: the greeting plays twice, occasionally.**
+That is the `screen_channel` fallback restarting the call. It only happens when
+the control channel fails to join within five seconds, and it is deliberate: an
+in-app agent that cannot see the screen is not a lesser version of the product.
+If it happens on every call, fix the control channel rather than living with it.
+
+**Audio is muffled or crackly on Android but fine on iOS.**
+Update the SDK. Versions before 0.15.0 asked the platform for the stream's own
+sample rate; iOS refuses and falls back to the device rate, Android grants it
+and resamples in its output stage, which is where the artefacts came from.
+
 ---
 
 ## Reference
@@ -534,12 +709,68 @@ confirm that is what it is.
 | `useGraineEvents()` | conversation lifecycle |
 | `useGraineVoice()` | only if you bring your own native audio instead of the WebView |
 
-`GraineProvider`: `baseUrl`, `publishableKey`, `navigationRef`, `includeScreens`,
-`launcherDelayMs`, `visibility`, `autoConnect`, `onProactive`.
+### `GraineProvider`
 
-`GraineVoiceLauncher`: `webView` (required), `autoStart`, `onCaption`,
-`onCallState`, and a child function receiving `{ connected, connecting, muted,
-start, end, setMuted }`.
+| Prop | Type | Notes |
+|---|---|---|
+| `publishableKey` | `string` | Required unless you pass `client`. |
+| `baseUrl` | `string` | Defaults to Graine's host. |
+| `client` | `GraineInAppClient` | Use **your** instance instead of building one. Pass this if your app has a module-level client. |
+| `navigationRef` | `ref` | The same ref you give `NavigationContainer`. Makes the agent screen-aware with no per-screen code. |
+| `includeScreens` | `string[]` | Route names where the launcher may appear. Omit for everywhere. |
+| `launcherDelayMs` | `number` | Delay before the launcher appears on an eligible screen. |
+| `visibility` | `LauncherVisibility` | Groups, continuity, per-group delays and insets. See *Controlling where the launcher appears*. |
+| `autoConnect` | `boolean` | Default `true`. Set `false` when `GraineVoiceLauncher` owns the connection — see below. |
+| `onProactive` | `(text) => void` | The agent spoke first; draw your own nudge. |
+| `endOnBackground` | `boolean` | Default `true`. You want it on. |
+
+### `GraineVoiceLauncher`
+
+| Prop | Type | Notes |
+|---|---|---|
+| `webView` | `ComponentType` | **Required.** Pass `react-native-webview`'s `WebView`. Passed in rather than imported so the package installs without it. |
+| `autoStart` | `boolean` | Start talking as soon as the page is ready. |
+| `transport` | `"ws" \| "rtc"` | Which transport carries **audio**. See below. |
+| `onTransport` | `({transport, requested}) => void` | Which transport actually carried the call, including after a fallback. |
+| `onCaption` | `({role, text, live}) => void` | Each caption line, for your own UI. |
+| `onCallState` | `({connected, connecting}) => void` | Connection state. |
+| `onEnded` | `() => void` | The call is over. Fires once. Give the goodbye ~400ms to play before you unmount. |
+| `onMicDenied` | `(reason) => void` | `NotAllowedError` is a refusal; `NotFoundError` is a device with no usable microphone. Different sentences for a customer. |
+| `onError` | `(message) => void` | Voice could not be set up at all — key rejected, agent not enabled for apps, no network. Without this the launcher renders nothing and you cannot tell why. |
+| `children` | `(api) => ReactNode` | Receives `{ connected, connecting, muted, start, end, setMuted }`. |
+
+**The launcher owns the connection.** It asks for the microphone, resolves the
+session, holds the socket and relays your screen, actions and events onto it. So
+set `autoConnect={false}` on the provider when you use it: two live connections
+on one agent is two conversations, and the customer hears both greetings.
+
+### `transport` — how audio travels
+
+**Omit it and the dashboard decides.** Agent → Embed & Widgets → *Call audio*
+sets this for every app and site running the agent, so the transport can be
+changed without shipping a release. Passing the prop overrides it.
+
+`ws` (the default) sends raw PCM on the same socket that carries screen context
+and in-app actions.
+
+`rtc` hands media to jambonz's SBC — Opus, DTLS-SRTP, an adaptive jitter buffer,
+and the platform's own echo canceller running below the microphone with the
+playout signal as its reference. Better audio on a bad network.
+
+**`rtc` carries the screen too.** The browser holds two connections — media to
+the SBC, and a control socket for `app_context`, `app_action` and `app_event` —
+and the server joins them by a session id the browser puts on the INVITE. So an
+in-app agent on `rtc` sees exactly what it sees on `ws`. Frames sent before the
+join completes are queued rather than dropped, because the first screen frame
+almost always arrives while the phone is still ringing, and it is the one the
+agent's opening line is composed from.
+
+`rtc` **falls back.** WebRTC has more ways to be unavailable than a WebSocket —
+no SIP realm on the org, an unreachable SBC, a network that eats UDP — and each
+is a reason to carry the call differently rather than refuse it. The page
+reconnects on the WebSocket and tells you through `onTransport`, so a permanent
+misconfiguration is visible instead of being merely audible. A refused
+microphone is **not** a transport problem and does not fall back.
 
 ---
 

package/README.md

@@ -114,6 +114,45 @@ the outcome through the next `useGraineScreen` update.
 
 ## Voice
 
+The short way: no native audio module, no rebuild, and audio fixes reach you
+over the air.
+
+```tsx
+import { WebView } from "react-native-webview";
+import { GraineProvider, GraineVoiceLauncher, GraineAgentBar } from "@graineai/inapp-react-native";
+
+<GraineProvider
+  baseUrl="…"
+  publishableKey="pk_live_…"
+  navigationRef={navigationRef}
+  autoConnect={false}          // the launcher owns the connection
+>
+  <RootNavigator />
+  <GraineVoiceLauncher
+    webView={WebView}
+    autoStart
+    onError={(m) => console.warn("[graine]", m)}
+  >
+    {() => <GraineAgentBar />}
+  </GraineVoiceLauncher>
+</GraineProvider>
+```
+
+`react-native-webview` is the only peer dependency and it autolinks. The
+microphone and speaker live in a hidden one-pixel WebView — your customer never
+sees a web page. Set `autoConnect={false}`: leave the provider connecting too
+and the agent answers twice, so the customer hears two greetings and you are
+billed for two conversations.
+
+Audio travels on a WebSocket by default. Set **Embed & Widgets → Call audio** to
+*High quality* for WebRTC — Opus, a jitter buffer and the device's own echo
+canceller — which carries screen context just the same and falls back on its own
+if it cannot connect. See `INTEGRATION.md`.
+
+### Bringing your own audio
+
+Only if you already have a native audio module and want it used instead.
+
 ```tsx
 <GraineProvider baseUrl="…" publishableKey="pk_live_…" voice>
 ```

package/dist/client.d.ts

@@ -53,6 +53,7 @@ export declare class GraineInAppClient {
     executeAction(name: string, args?: Record<string, unknown>): Promise<AppActionResult>;
     private runAction;
     private invokeAction;
+    get isConnected(): boolean;
     getScreen(): ScreenContext | null;
     getIdentity(): Record<string, string>;
     getRecentEvents(): Array<{
@@ -67,6 +68,12 @@ export declare class GraineInAppClient {
     reportEvent(event: AppEvent): void;
     track(name: string, data?: Record<string, unknown>): void;
     private scheduleScreen;
+    buildContextFrame(): {
+        type: "app_context";
+        context: Record<string, unknown>;
+        available_actions: string[];
+        seq: number;
+    } | null;
     private flushScreen;
     private armStall;
     identify(user: Record<string, unknown> | null): void;

package/dist/client.js

@@ -259,6 +259,9 @@ export class GraineInAppClient {
             };
         }
     }
+    get isConnected() {
+        return !!this.ws && this.ws.readyState === 1;
+    }
     getScreen() {
         return this.screen;
     }
@@ -293,6 +296,7 @@ export class GraineInAppClient {
         this.scheduleScreen();
     }
     reportEvent(event) {
+        this.emit("app_event", event);
         this.send({ type: "app_event", event });
     }
     track(name, data) {
@@ -320,21 +324,28 @@ export class GraineInAppClient {
             }
         }, CONTEXT_THROTTLE_MS);
     }
-    flushScreen() {
-        if (!this.screen)
-            return false;
-        const { availableActions, ...rest } = this.screen;
-        return this.send({
+    buildContextFrame() {
+        const hasIdentity = Object.keys(this.identity).length > 0;
+        if (!this.screen && !hasIdentity && this.recentEvents.length === 0)
+            return null;
+        const { availableActions, ...rest } = (this.screen ?? {});
+        return {
             type: "app_context",
             context: maskDeep({
                 ...rest,
-                idle_ms: Date.now() - this.screenSince,
+                ...(this.screen ? { idle_ms: Date.now() - this.screenSince } : {}),
                 ...(Object.keys(this.identity).length ? { user: this.identity } : {}),
                 ...(this.recentEvents.length ? { recent_events: this.recentEvents } : {}),
             }),
             available_actions: (availableActions ?? this.availableActions).filter((a) => this.actions.has(a)),
             seq: ++this.screenSeq,
-        });
+        };
+    }
+    flushScreen() {
+        const frame = this.buildContextFrame();
+        if (!frame)
+            return false;
+        return this.send(frame);
     }
     armStall() {
         clearTimeout(this.stallTimer);
@@ -369,8 +380,7 @@ export class GraineInAppClient {
             flat[k] = typeof v === "string" ? v : JSON.stringify(v);
         }
         this.identity = maskDeep(flat);
-        if (this.screen)
-            this.scheduleScreen();
+        this.scheduleScreen();
     }
     noteInteraction() {
         this.screenSince = Date.now();

package/dist/react-native/index.d.ts

@@ -60,8 +60,9 @@ export interface Turn {
     text: string;
     live?: boolean;
 }
-export interface GraineProviderProps extends GraineInAppOptions {
+export interface GraineProviderProps extends Partial<GraineInAppOptions> {
     children: React.ReactNode;
+    client?: GraineInAppClient;
     autoConnect?: boolean;
     onProactive?: (text: string) => void;
     endOnBackground?: boolean;
@@ -72,7 +73,7 @@ export interface GraineProviderProps extends GraineInAppOptions {
     launcherDelayMs?: number;
     visibility?: LauncherVisibility;
 }
-export declare function GraineProvider({ children, autoConnect, onProactive, endOnBackground, navigationRef, includeScreens, launcherDelayMs, visibility, ...options }: GraineProviderProps): React.JSX.Element;
+export declare function GraineProvider({ children, autoConnect, onProactive, endOnBackground, navigationRef, includeScreens, launcherDelayMs, visibility, client: providedClient, ...options }: GraineProviderProps): React.JSX.Element;
 export declare function useGraineAgent(): GraineContextValue;
 export declare function useGraineEvents(handler: (e: GraineEvent) => void): void;
 export declare function useGraineTrack(): (name: string, data?: Record<string, unknown>) => void;

package/dist/react-native/index.js

@@ -5,10 +5,12 @@ import { GraineInAppClient } from "../client.js";
 import { VoiceSession } from "../voice.js";
 import { LauncherVisibilityTracker, activeRouteName, } from "./navigation.js";
 const Ctx = createContext(null);
-export function GraineProvider({ children, autoConnect = true, onProactive, endOnBackground = true, navigationRef, includeScreens, launcherDelayMs = 0, visibility, ...options }) {
+export function GraineProvider({ children, autoConnect = true, onProactive, endOnBackground = true, navigationRef, includeScreens, launcherDelayMs = 0, visibility, client: providedClient, ...options }) {
     const clientRef = useRef(null);
-    if (!clientRef.current)
-        clientRef.current = new GraineInAppClient(options);
+    if (!clientRef.current) {
+        clientRef.current =
+            providedClient ?? new GraineInAppClient(options);
+    }
     const client = clientRef.current;
     const [connected, setConnected] = useState(false);
     const [connecting, setConnecting] = useState(false);

package/dist/react-native/voice-launcher.d.ts

@@ -1,5 +1,12 @@
 import React from "react";
 export interface GraineVoiceLauncherProps {
+    transport?: "ws" | "rtc";
+    onTransport?: (info: {
+        transport: "ws" | "rtc";
+        requested: "ws" | "rtc";
+        reason?: "not_configured" | "unreachable" | "screen_channel";
+        message?: string;
+    }) => void;
     webView: React.ComponentType<any>;
     autoStart?: boolean;
     onCaption?: (c: {
@@ -13,6 +20,7 @@ export interface GraineVoiceLauncherProps {
     }) => void;
     onEnded?: () => void;
     onMicDenied?: (reason: string) => void;
+    onError?: (message: string) => void;
     children?: (api: GraineVoiceApi) => React.ReactNode;
 }
 export interface GraineVoiceApi {
@@ -23,4 +31,4 @@ export interface GraineVoiceApi {
     end: () => void;
     setMuted: (m: boolean) => void;
 }
-export declare function GraineVoiceLauncher({ webView: WebView, autoStart, onCaption, onCallState, onMicDenied, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;
+export declare function GraineVoiceLauncher({ webView: WebView, autoStart, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;

package/dist/react-native/voice-launcher.js

@@ -1,8 +1,8 @@
 import { Fragment as _Fragment, jsx as _jsx, jsxs as _jsxs } from "react/jsx-runtime";
 import { useCallback, useEffect, useRef, useState } from "react";
-import { AppState, View } from "react-native";
+import { AppState, PermissionsAndroid, Platform, View } from "react-native";
 import { useGraineAgent } from "./index.js";
-export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCaption, onCallState, onMicDenied, onEnded, children, }) {
+export function GraineVoiceLauncher({ webView: WebView, autoStart = false, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }) {
     const { client, appearance } = useGraineAgent();
     const ref = useRef(null);
     const startedRef = useRef(false);
@@ -13,22 +13,86 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
     const post = useCallback((msg) => {
         ref.current?.injectJavaScript(`window.postMessage(${JSON.stringify(msg)}, "*"); true;`);
     }, []);
+    const [voiceAgentId, setVoiceAgentId] = useState(client.config ? client.config.webcallAgentId || client.config.agentId : null);
+    const onErrorRef = useRef(onError);
+    onErrorRef.current = onError;
+    useEffect(() => {
+        if (voiceAgentId)
+            return;
+        let cancelled = false;
+        (async () => {
+            try {
+                const cfg = client.config ?? (await client.session());
+                if (!cancelled)
+                    setVoiceAgentId(cfg.webcallAgentId || cfg.agentId);
+            }
+            catch (err) {
+                if (!cancelled) {
+                    onErrorRef.current?.(String(err?.message || err || "Voice is unavailable."));
+                }
+            }
+        })();
+        return () => { cancelled = true; };
+    }, [client, voiceAgentId]);
+    const ensureMic = useCallback(async () => {
+        if (Platform.OS !== "android")
+            return true;
+        try {
+            const perm = PermissionsAndroid?.PERMISSIONS?.RECORD_AUDIO;
+            if (!perm)
+                return true;
+            if (await PermissionsAndroid.check(perm))
+                return true;
+            const res = await PermissionsAndroid.request(perm, {
+                title: "Microphone",
+                message: "Allow the microphone so you can talk to the assistant.",
+                buttonPositive: "Allow",
+            });
+            return res === PermissionsAndroid.RESULTS.GRANTED;
+        }
+        catch {
+            return true;
+        }
+    }, []);
     const pushContext = useCallback(() => {
-        const screen = client.getScreen();
-        if (!screen)
+        const frame = client.buildContextFrame();
+        if (!frame)
             return;
-        const { availableActions, ...rest } = screen;
         post({
             type: "graine:app-context",
-            context: rest,
-            availableActions: (availableActions ?? client.getAvailableActions()).filter((a) => client.getAvailableActions().includes(a)),
-            seq: Date.now(),
+            context: frame.context,
+            availableActions: frame.available_actions,
+            seq: frame.seq,
         });
     }, [client, post]);
     useEffect(() => {
         if (ready)
             pushContext();
     }, [ready, pushContext]);
+    const beginCall = useCallback(async () => {
+        if (client.isConnected) {
+            console.warn("[graine] The provider has its own connection open while GraineVoiceLauncher " +
+                "is starting a call. Pass autoConnect={false} to <GraineProvider> — otherwise " +
+                "the agent answers twice and the customer hears two greetings.");
+        }
+        if (!(await ensureMic())) {
+            setConnecting(false);
+            onMicDenied?.("NotAllowedError");
+            return;
+        }
+        post({ type: "graine:start-call" });
+    }, [client, ensureMic, onMicDenied, post]);
+    useEffect(() => {
+        if (!connected)
+            return;
+        return client.on("app_event", (event) => {
+            pushContext();
+            const { name, ...rest } = (event ?? {});
+            if (!name)
+                return;
+            post({ type: "graine:app-event", name, data: rest });
+        });
+    }, [client, connected, post, pushContext]);
     const onMessage = useCallback(async (e) => {
         let msg;
         try {
@@ -38,12 +102,23 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
             return;
         }
         switch (msg.type) {
+            case "graine:transport":
+                onTransport?.({
+                    transport: msg.transport === "rtc" ? "rtc" : "ws",
+                    requested: msg.requested === "rtc" ? "rtc" : "ws",
+                    reason: msg.reason,
+                    message: msg.message,
+                });
+                if (msg.reason === "not_configured" && msg.message) {
+                    onError?.(String(msg.message));
+                }
+                return;
             case "graine:ready":
                 setReady(true);
                 pushContext();
                 if (autoStart && !startedRef.current) {
                     startedRef.current = true;
-                    post({ type: "graine:start-call" });
+                    void beginCall();
                 }
                 return;
             case "graine:call":
@@ -80,15 +155,15 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
                 return;
             }
         }
-    }, [autoStart, client, onCaption, onCallState, onMicDenied, onEnded, post, pushContext]);
+    }, [autoStart, beginCall, client, onCaption, onCallState, onMicDenied, onTransport, onError, onEnded, post, pushContext]);
     const api = {
         connected,
         connecting,
         muted,
         start: useCallback(() => {
             setConnecting(true);
-            post({ type: "graine:start-call" });
-        }, [post]),
+            void beginCall();
+        }, [beginCall]),
         end: useCallback(() => post({ type: "graine:end-call" }), [post]),
         setMuted: useCallback((m) => {
             setMutedState(m);
@@ -121,11 +196,12 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
         return () => { if (pending)
             clearTimeout(pending); sub.remove(); };
     }, [post]);
-    const agentId = client.config?.agentId;
-    if (!agentId)
+    if (!voiceAgentId)
         return _jsx(_Fragment, { children: children?.(api) });
-    const src = `${client.opts.baseUrl.replace(/\/$/, "")}/embed/${agentId}` +
-        `?mode=voice&branding=0&embed=1` +
+    const effectiveTransport = transport ?? (appearance?.audioTransport === "rtc" ? "rtc" : "ws");
+    const src = `${client.opts.baseUrl.replace(/\/$/, "")}/embed/${voiceAgentId}` +
+        `?mode=voice&branding=0&embed=1&transport=${effectiveTransport}` +
+        `&pk=${encodeURIComponent(client.opts.publishableKey)}` +
         (appearance?.accent ? `&accent=${encodeURIComponent(String(appearance.accent).replace("#", ""))}` : "");
     return (_jsxs(_Fragment, { children: [_jsx(View, { style: { position: "absolute", width: 1, height: 1, opacity: 0, bottom: 0, left: 0 }, pointerEvents: "none", children: _jsx(WebView, { ref: ref, source: { uri: src }, originWhitelist: ["*"], javaScriptEnabled: true, domStorageEnabled: true, allowsInlineMediaPlayback: true, mediaPlaybackRequiresUserAction: false, mediaCapturePermissionGrantType: "grant", onMessage: onMessage, style: { width: 1, height: 1, backgroundColor: "transparent" } }) }), children?.(api)] }));
 }

package/package.json

@@ -1,8 +1,8 @@
 {
   "name": "@graineai/inapp-react-native",
-  "version": "0.15.0",
+  "version": "0.17.0",
   "type": "module",
-  "description": "Graine in-app agent for React Native — an agent that sees the screen your customer is on and can act on it.",
+  "description": "Graine in-app agent for React Native \u2014 an agent that sees the screen your customer is on and can act on it.",
   "license": "MIT",
   "private": false,
   "main": "dist/index.js",