@graineai/inapp-react-native 0.15.0 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/INTEGRATION.md CHANGED
@@ -8,6 +8,87 @@ Package: `@graineai/inapp-react-native`
8
8
 
9
9
  ---
10
10
 
11
+ ## How it works, end to end
12
+
13
+ Two things travel between your app and the agent, and keeping them straight
14
+ explains most of this document.
15
+
16
+ **Media** is the customer's voice and the agent's. **Control** is everything the
17
+ agent knows about your app: the screen, who is on it, what just happened, and
18
+ what it is allowed to do about it. They are different problems — media wants
19
+ pacing and loss tolerance, control wants ordered, reliable delivery — so they are
20
+ carried differently, and the SDK is what hides that from you.
21
+
22
+ ```
23
+ YOUR APP GRAINE
24
+ ───────────────────────────────────── ──────────────────────────────
25
+
26
+ ┌───────────────────────────────────┐
27
+ │ your screens │
28
+ │ useGraineScreen(...) ──────┐ │
29
+ │ useGraineAction(...) ──────┤ │
30
+ │ identify() / trackEvent() │ │
31
+ └───────────────────────────────┼───┘
32
+ │ one client, one source of truth
33
+
34
+ ┌─────────────────────────┐
35
+ │ GraineInAppClient │
36
+ │ screen · identity │
37
+ │ events · actions │
38
+ │ buildContextFrame() │
39
+ │ maskDeep() ← PII never │
40
+ │ leaves the device │
41
+ └───────────┬─────────────┘
42
+
43
+ ┌───────────▼─────────────┐
44
+ │ GraineVoiceLauncher │
45
+ │ mic · lifecycle · relay│
46
+ └───────────┬─────────────┘
47
+ │ postMessage
48
+ ┌───────────▼─────────────┐
49
+ │ hidden WebView (1×1) │ ┌──────────────┐
50
+ │ the audio engine │══ media ══▶│ the agent │
51
+ │ capture · playback │ │ │
52
+ │ jitter buffer · barge-in│──control ─▶│ runtime │
53
+ └─────────────────────────┘ └──────┬───────┘
54
+
55
+ ▲ │
56
+ └──────── app_action ────────────┘
57
+ "open the offers screen"
58
+ (the agent BLOCKS on your answer)
59
+ ```
60
+
61
+ **Why a WebView.** React Native has no `getUserMedia`. The alternative is a
62
+ native audio module, which means a store release for every audio fix — and the
63
+ audio path is exactly where the fixes are. The page is one pixel, invisible and
64
+ unreachable; your customer only ever sees your own UI. See *Why there is no
65
+ LiveKit step* at the end.
66
+
67
+ ### What travels, and when
68
+
69
+ | Frame | Direction | When |
70
+ |---|---|---|
71
+ | `app_context` | app → agent | Every screen change and field edit, throttled. Masked first. |
72
+ | `app_action` | agent → app | The agent wants to do something. **It blocks on your reply** (~6s). |
73
+ | `app_action_result` | app → agent | Your handler's answer: `ok`, `refused`, or `error`. |
74
+ | `app_event` | app → agent | Something happened that may be worth speaking about unprompted. |
75
+
76
+ ### The one rule that matters
77
+
78
+ **One client.** `GraineProvider` builds a client unless you hand it one. If your
79
+ app already has a module-level instance — and any app of size does, because
80
+ `identify()` and `trackEvent()` are not all called from React — pass it in:
81
+
82
+ ```tsx
83
+ <GraineProvider client={myClient} autoConnect={false}>
84
+ ```
85
+
86
+ Two clients is the failure that looks like a broken product: your screens write
87
+ the context into one, the launcher reads the other and finds nothing, and the
88
+ agent connects, greets, and says it cannot see the screen.
89
+
90
+ ---
91
+
11
92
  ## What you need
12
93
 
13
94
  | | |
@@ -89,12 +170,18 @@ export default function App() {
89
170
  baseUrl="https://www.graine.ai"
90
171
  publishableKey="pk_live_..."
91
172
  navigationRef={navigationRef}
173
+ // The launcher owns the connection — see below.
174
+ autoConnect={false}
92
175
  >
93
176
  <NavigationContainer ref={navigationRef}>
94
177
  <RootNavigator />
95
178
  </NavigationContainer>
96
179
 
97
- <GraineVoiceLauncher webView={WebView} autoStart>
180
+ <GraineVoiceLauncher
181
+ webView={WebView}
182
+ autoStart
183
+ onError={(m) => console.warn("[graine]", m)}
184
+ >
98
185
  {() => <GraineAgentBar />}
99
186
  </GraineVoiceLauncher>
100
187
  </GraineProvider>
@@ -102,6 +189,16 @@ export default function App() {
102
189
  }
103
190
  ```
104
191
 
192
+ **`autoConnect={false}` is not optional here.** The provider opens a connection
193
+ of its own by default, and `GraineVoiceLauncher` opens the one that carries the
194
+ call. Leave both on and the agent answers twice — the customer hears two
195
+ greetings over each other, and you are billed for two conversations. Drop the
196
+ launcher and use text only, and `autoConnect` goes back to its default.
197
+
198
+ **Pass `onError`.** A rejected key or an agent that has not been enabled for apps
199
+ leaves the launcher rendering nothing at all, forever, with nothing in the
200
+ console. This is the only way to find out.
201
+
105
202
  `navigationRef` is the line that matters. With it the SDK reads the current route
106
203
  itself, so the agent knows which screen the customer is on without you calling
107
204
  anything per screen — including the screen you add next year when nobody
@@ -153,14 +250,43 @@ specific than a default. Both are masked on the device before they leave it.
153
250
 
154
251
  ### It reaches the call record, not only the prompt
155
252
 
156
- Traits are stored on the conversation as `app_user`, and the name labels the row
157
- in Call History. A web call has no phone number to identify it by, so without
158
- this every in-app conversation shows a session id.
253
+ Traits are stored on the conversation as `app_user`, and they are what labels
254
+ the row in Call History. **A web call has no phone number**, so without this
255
+ every in-app conversation shows a bare session id — a page of uuids.
159
256
 
160
257
  Read from the LIVE identity rather than the init frame, so a customer who signs
161
258
  in halfway through is identified from that moment rather than from a frame that
162
259
  predates them.
163
260
 
261
+ #### Tag the session with whatever you file people under
262
+
263
+ You do not have to have a `name`. Call History looks for a label in this order:
264
+
265
+ `name` → `full_name` / `fullName` → `display_name` / `displayName` → `label` →
266
+ `username` / `user_name` → `customer_name` → `company` → `email` → **any other
267
+ string trait**
268
+
269
+ So an app that files people by policy number, customer id or internal handle is
270
+ still legible:
271
+
272
+ ```tsx
273
+ identify({ label: `Policy ${policy.number}` }); // → "Policy HDFC-4471"
274
+ identify({ customer_name: acct.holder }); // → "R. Bhanot"
275
+ identify({ crm_id: lead.id }); // → falls through to the id
276
+ ```
277
+
278
+ Two rules worth knowing:
279
+
280
+ - **Redacted values are skipped.** `phone` and `email` are masked on the device,
281
+ so a row never renders `[PHONE]` as somebody's name. Send the phone anyway if
282
+ it is useful — the agent is told a number is on file without being told the
283
+ digits — just do not rely on it as the label.
284
+ - **Identity alone is enough.** You do not need `useGraineScreen` for this. An
285
+ app that only calls `identify()` still tags every conversation.
286
+
287
+ `identify(null)` on sign-out. An agent still greeting the previous account by
288
+ name after a switch is worse than one that never knew it.
289
+
164
290
  ---
165
291
 
166
292
  ## Step 3 — Report what is ON the screen
@@ -520,6 +646,29 @@ Echo. The SDK suppresses the agent's own voice from the microphone, but a very
520
646
  loud speaker defeats a level-based guard. Lower the volume, or use earphones to
521
647
  confirm that is what it is.
522
648
 
649
+ **Nothing at all happens, and `onError` never fires either.**
650
+ You did not pass `onError`. Without it a failed session — a rejected key, an
651
+ agent not enabled for apps — leaves the launcher rendering nothing, silently.
652
+ Pass it before you debug anything else.
653
+
654
+ **On `rtc`: every call reports a fallback to `ws`.**
655
+ `onTransport` gives a `reason`. `unreachable` means WebRTC could not be reached
656
+ at all — usually no SIP realm on the org, or the SBC is not reachable from the
657
+ customer's network. `screen_channel` means the audio connected but the control
658
+ socket did not join, so the call was restarted on the WebSocket rather than run
659
+ blind. Both are org configuration, not integration code.
660
+
661
+ **On `rtc`: the greeting plays twice, occasionally.**
662
+ That is the `screen_channel` fallback restarting the call. It only happens when
663
+ the control channel fails to join within five seconds, and it is deliberate: an
664
+ in-app agent that cannot see the screen is not a lesser version of the product.
665
+ If it happens on every call, fix the control channel rather than living with it.
666
+
667
+ **Audio is muffled or crackly on Android but fine on iOS.**
668
+ Update the SDK. Versions before 0.15.0 asked the platform for the stream's own
669
+ sample rate; iOS refuses and falls back to the device rate, Android grants it
670
+ and resamples in its output stage, which is where the artefacts came from.
671
+
523
672
  ---
524
673
 
525
674
  ## Reference
@@ -534,12 +683,68 @@ confirm that is what it is.
534
683
  | `useGraineEvents()` | conversation lifecycle |
535
684
  | `useGraineVoice()` | only if you bring your own native audio instead of the WebView |
536
685
 
537
- `GraineProvider`: `baseUrl`, `publishableKey`, `navigationRef`, `includeScreens`,
538
- `launcherDelayMs`, `visibility`, `autoConnect`, `onProactive`.
686
+ ### `GraineProvider`
539
687
 
540
- `GraineVoiceLauncher`: `webView` (required), `autoStart`, `onCaption`,
541
- `onCallState`, and a child function receiving `{ connected, connecting, muted,
542
- start, end, setMuted }`.
688
+ | Prop | Type | Notes |
689
+ |---|---|---|
690
+ | `publishableKey` | `string` | Required unless you pass `client`. |
691
+ | `baseUrl` | `string` | Defaults to Graine's host. |
692
+ | `client` | `GraineInAppClient` | Use **your** instance instead of building one. Pass this if your app has a module-level client. |
693
+ | `navigationRef` | `ref` | The same ref you give `NavigationContainer`. Makes the agent screen-aware with no per-screen code. |
694
+ | `includeScreens` | `string[]` | Route names where the launcher may appear. Omit for everywhere. |
695
+ | `launcherDelayMs` | `number` | Delay before the launcher appears on an eligible screen. |
696
+ | `visibility` | `LauncherVisibility` | Groups, continuity, per-group delays and insets. See *Controlling where the launcher appears*. |
697
+ | `autoConnect` | `boolean` | Default `true`. Set `false` when `GraineVoiceLauncher` owns the connection — see below. |
698
+ | `onProactive` | `(text) => void` | The agent spoke first; draw your own nudge. |
699
+ | `endOnBackground` | `boolean` | Default `true`. You want it on. |
700
+
701
+ ### `GraineVoiceLauncher`
702
+
703
+ | Prop | Type | Notes |
704
+ |---|---|---|
705
+ | `webView` | `ComponentType` | **Required.** Pass `react-native-webview`'s `WebView`. Passed in rather than imported so the package installs without it. |
706
+ | `autoStart` | `boolean` | Start talking as soon as the page is ready. |
707
+ | `transport` | `"ws" \| "rtc"` | Which transport carries **audio**. See below. |
708
+ | `onTransport` | `({transport, requested}) => void` | Which transport actually carried the call, including after a fallback. |
709
+ | `onCaption` | `({role, text, live}) => void` | Each caption line, for your own UI. |
710
+ | `onCallState` | `({connected, connecting}) => void` | Connection state. |
711
+ | `onEnded` | `() => void` | The call is over. Fires once. Give the goodbye ~400ms to play before you unmount. |
712
+ | `onMicDenied` | `(reason) => void` | `NotAllowedError` is a refusal; `NotFoundError` is a device with no usable microphone. Different sentences for a customer. |
713
+ | `onError` | `(message) => void` | Voice could not be set up at all — key rejected, agent not enabled for apps, no network. Without this the launcher renders nothing and you cannot tell why. |
714
+ | `children` | `(api) => ReactNode` | Receives `{ connected, connecting, muted, start, end, setMuted }`. |
715
+
716
+ **The launcher owns the connection.** It asks for the microphone, resolves the
717
+ session, holds the socket and relays your screen, actions and events onto it. So
718
+ set `autoConnect={false}` on the provider when you use it: two live connections
719
+ on one agent is two conversations, and the customer hears both greetings.
720
+
721
+ ### `transport` — how audio travels
722
+
723
+ **Omit it and the dashboard decides.** Agent → Embed & Widgets → *Call audio*
724
+ sets this for every app and site running the agent, so the transport can be
725
+ changed without shipping a release. Passing the prop overrides it.
726
+
727
+ `ws` (the default) sends raw PCM on the same socket that carries screen context
728
+ and in-app actions.
729
+
730
+ `rtc` hands media to jambonz's SBC — Opus, DTLS-SRTP, an adaptive jitter buffer,
731
+ and the platform's own echo canceller running below the microphone with the
732
+ playout signal as its reference. Better audio on a bad network.
733
+
734
+ **`rtc` carries the screen too.** The browser holds two connections — media to
735
+ the SBC, and a control socket for `app_context`, `app_action` and `app_event` —
736
+ and the server joins them by a session id the browser puts on the INVITE. So an
737
+ in-app agent on `rtc` sees exactly what it sees on `ws`. Frames sent before the
738
+ join completes are queued rather than dropped, because the first screen frame
739
+ almost always arrives while the phone is still ringing, and it is the one the
740
+ agent's opening line is composed from.
741
+
742
+ `rtc` **falls back.** WebRTC has more ways to be unavailable than a WebSocket —
743
+ no SIP realm on the org, an unreachable SBC, a network that eats UDP — and each
744
+ is a reason to carry the call differently rather than refuse it. The page
745
+ reconnects on the WebSocket and tells you through `onTransport`, so a permanent
746
+ misconfiguration is visible instead of being merely audible. A refused
747
+ microphone is **not** a transport problem and does not fall back.
543
748
 
544
749
  ---
545
750
 
package/README.md CHANGED
@@ -114,6 +114,45 @@ the outcome through the next `useGraineScreen` update.
114
114
 
115
115
  ## Voice
116
116
 
117
+ The short way: no native audio module, no rebuild, and audio fixes reach you
118
+ over the air.
119
+
120
+ ```tsx
121
+ import { WebView } from "react-native-webview";
122
+ import { GraineProvider, GraineVoiceLauncher, GraineAgentBar } from "@graineai/inapp-react-native";
123
+
124
+ <GraineProvider
125
+ baseUrl="…"
126
+ publishableKey="pk_live_…"
127
+ navigationRef={navigationRef}
128
+ autoConnect={false} // the launcher owns the connection
129
+ >
130
+ <RootNavigator />
131
+ <GraineVoiceLauncher
132
+ webView={WebView}
133
+ autoStart
134
+ onError={(m) => console.warn("[graine]", m)}
135
+ >
136
+ {() => <GraineAgentBar />}
137
+ </GraineVoiceLauncher>
138
+ </GraineProvider>
139
+ ```
140
+
141
+ `react-native-webview` is the only peer dependency and it autolinks. The
142
+ microphone and speaker live in a hidden one-pixel WebView — your customer never
143
+ sees a web page. Set `autoConnect={false}`: leave the provider connecting too
144
+ and the agent answers twice, so the customer hears two greetings and you are
145
+ billed for two conversations.
146
+
147
+ Audio travels on a WebSocket by default. Set **Embed & Widgets → Call audio** to
148
+ *High quality* for WebRTC — Opus, a jitter buffer and the device's own echo
149
+ canceller — which carries screen context just the same and falls back on its own
150
+ if it cannot connect. See `INTEGRATION.md`.
151
+
152
+ ### Bringing your own audio
153
+
154
+ Only if you already have a native audio module and want it used instead.
155
+
117
156
  ```tsx
118
157
  <GraineProvider baseUrl="…" publishableKey="pk_live_…" voice>
119
158
  ```
package/dist/client.d.ts CHANGED
@@ -53,6 +53,7 @@ export declare class GraineInAppClient {
53
53
  executeAction(name: string, args?: Record<string, unknown>): Promise<AppActionResult>;
54
54
  private runAction;
55
55
  private invokeAction;
56
+ get isConnected(): boolean;
56
57
  getScreen(): ScreenContext | null;
57
58
  getIdentity(): Record<string, string>;
58
59
  getRecentEvents(): Array<{
@@ -67,6 +68,12 @@ export declare class GraineInAppClient {
67
68
  reportEvent(event: AppEvent): void;
68
69
  track(name: string, data?: Record<string, unknown>): void;
69
70
  private scheduleScreen;
71
+ buildContextFrame(): {
72
+ type: "app_context";
73
+ context: Record<string, unknown>;
74
+ available_actions: string[];
75
+ seq: number;
76
+ } | null;
70
77
  private flushScreen;
71
78
  private armStall;
72
79
  identify(user: Record<string, unknown> | null): void;
package/dist/client.js CHANGED
@@ -259,6 +259,9 @@ export class GraineInAppClient {
259
259
  };
260
260
  }
261
261
  }
262
+ get isConnected() {
263
+ return !!this.ws && this.ws.readyState === 1;
264
+ }
262
265
  getScreen() {
263
266
  return this.screen;
264
267
  }
@@ -293,6 +296,7 @@ export class GraineInAppClient {
293
296
  this.scheduleScreen();
294
297
  }
295
298
  reportEvent(event) {
299
+ this.emit("app_event", event);
296
300
  this.send({ type: "app_event", event });
297
301
  }
298
302
  track(name, data) {
@@ -320,21 +324,28 @@ export class GraineInAppClient {
320
324
  }
321
325
  }, CONTEXT_THROTTLE_MS);
322
326
  }
323
- flushScreen() {
324
- if (!this.screen)
325
- return false;
326
- const { availableActions, ...rest } = this.screen;
327
- return this.send({
327
+ buildContextFrame() {
328
+ const hasIdentity = Object.keys(this.identity).length > 0;
329
+ if (!this.screen && !hasIdentity && this.recentEvents.length === 0)
330
+ return null;
331
+ const { availableActions, ...rest } = (this.screen ?? {});
332
+ return {
328
333
  type: "app_context",
329
334
  context: maskDeep({
330
335
  ...rest,
331
- idle_ms: Date.now() - this.screenSince,
336
+ ...(this.screen ? { idle_ms: Date.now() - this.screenSince } : {}),
332
337
  ...(Object.keys(this.identity).length ? { user: this.identity } : {}),
333
338
  ...(this.recentEvents.length ? { recent_events: this.recentEvents } : {}),
334
339
  }),
335
340
  available_actions: (availableActions ?? this.availableActions).filter((a) => this.actions.has(a)),
336
341
  seq: ++this.screenSeq,
337
- });
342
+ };
343
+ }
344
+ flushScreen() {
345
+ const frame = this.buildContextFrame();
346
+ if (!frame)
347
+ return false;
348
+ return this.send(frame);
338
349
  }
339
350
  armStall() {
340
351
  clearTimeout(this.stallTimer);
@@ -369,8 +380,7 @@ export class GraineInAppClient {
369
380
  flat[k] = typeof v === "string" ? v : JSON.stringify(v);
370
381
  }
371
382
  this.identity = maskDeep(flat);
372
- if (this.screen)
373
- this.scheduleScreen();
383
+ this.scheduleScreen();
374
384
  }
375
385
  noteInteraction() {
376
386
  this.screenSince = Date.now();
@@ -60,8 +60,9 @@ export interface Turn {
60
60
  text: string;
61
61
  live?: boolean;
62
62
  }
63
- export interface GraineProviderProps extends GraineInAppOptions {
63
+ export interface GraineProviderProps extends Partial<GraineInAppOptions> {
64
64
  children: React.ReactNode;
65
+ client?: GraineInAppClient;
65
66
  autoConnect?: boolean;
66
67
  onProactive?: (text: string) => void;
67
68
  endOnBackground?: boolean;
@@ -72,7 +73,7 @@ export interface GraineProviderProps extends GraineInAppOptions {
72
73
  launcherDelayMs?: number;
73
74
  visibility?: LauncherVisibility;
74
75
  }
75
- export declare function GraineProvider({ children, autoConnect, onProactive, endOnBackground, navigationRef, includeScreens, launcherDelayMs, visibility, ...options }: GraineProviderProps): React.JSX.Element;
76
+ export declare function GraineProvider({ children, autoConnect, onProactive, endOnBackground, navigationRef, includeScreens, launcherDelayMs, visibility, client: providedClient, ...options }: GraineProviderProps): React.JSX.Element;
76
77
  export declare function useGraineAgent(): GraineContextValue;
77
78
  export declare function useGraineEvents(handler: (e: GraineEvent) => void): void;
78
79
  export declare function useGraineTrack(): (name: string, data?: Record<string, unknown>) => void;
@@ -5,10 +5,12 @@ import { GraineInAppClient } from "../client.js";
5
5
  import { VoiceSession } from "../voice.js";
6
6
  import { LauncherVisibilityTracker, activeRouteName, } from "./navigation.js";
7
7
  const Ctx = createContext(null);
8
- export function GraineProvider({ children, autoConnect = true, onProactive, endOnBackground = true, navigationRef, includeScreens, launcherDelayMs = 0, visibility, ...options }) {
8
+ export function GraineProvider({ children, autoConnect = true, onProactive, endOnBackground = true, navigationRef, includeScreens, launcherDelayMs = 0, visibility, client: providedClient, ...options }) {
9
9
  const clientRef = useRef(null);
10
- if (!clientRef.current)
11
- clientRef.current = new GraineInAppClient(options);
10
+ if (!clientRef.current) {
11
+ clientRef.current =
12
+ providedClient ?? new GraineInAppClient(options);
13
+ }
12
14
  const client = clientRef.current;
13
15
  const [connected, setConnected] = useState(false);
14
16
  const [connecting, setConnecting] = useState(false);
@@ -1,5 +1,10 @@
1
1
  import React from "react";
2
2
  export interface GraineVoiceLauncherProps {
3
+ transport?: "ws" | "rtc";
4
+ onTransport?: (info: {
5
+ transport: "ws" | "rtc";
6
+ requested: "ws" | "rtc";
7
+ }) => void;
3
8
  webView: React.ComponentType<any>;
4
9
  autoStart?: boolean;
5
10
  onCaption?: (c: {
@@ -13,6 +18,7 @@ export interface GraineVoiceLauncherProps {
13
18
  }) => void;
14
19
  onEnded?: () => void;
15
20
  onMicDenied?: (reason: string) => void;
21
+ onError?: (message: string) => void;
16
22
  children?: (api: GraineVoiceApi) => React.ReactNode;
17
23
  }
18
24
  export interface GraineVoiceApi {
@@ -23,4 +29,4 @@ export interface GraineVoiceApi {
23
29
  end: () => void;
24
30
  setMuted: (m: boolean) => void;
25
31
  }
26
- export declare function GraineVoiceLauncher({ webView: WebView, autoStart, onCaption, onCallState, onMicDenied, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;
32
+ export declare function GraineVoiceLauncher({ webView: WebView, autoStart, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;
@@ -1,8 +1,8 @@
1
1
  import { Fragment as _Fragment, jsx as _jsx, jsxs as _jsxs } from "react/jsx-runtime";
2
2
  import { useCallback, useEffect, useRef, useState } from "react";
3
- import { AppState, View } from "react-native";
3
+ import { AppState, PermissionsAndroid, Platform, View } from "react-native";
4
4
  import { useGraineAgent } from "./index.js";
5
- export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCaption, onCallState, onMicDenied, onEnded, children, }) {
5
+ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }) {
6
6
  const { client, appearance } = useGraineAgent();
7
7
  const ref = useRef(null);
8
8
  const startedRef = useRef(false);
@@ -13,22 +13,86 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
13
13
  const post = useCallback((msg) => {
14
14
  ref.current?.injectJavaScript(`window.postMessage(${JSON.stringify(msg)}, "*"); true;`);
15
15
  }, []);
16
+ const [voiceAgentId, setVoiceAgentId] = useState(client.config ? client.config.webcallAgentId || client.config.agentId : null);
17
+ const onErrorRef = useRef(onError);
18
+ onErrorRef.current = onError;
19
+ useEffect(() => {
20
+ if (voiceAgentId)
21
+ return;
22
+ let cancelled = false;
23
+ (async () => {
24
+ try {
25
+ const cfg = client.config ?? (await client.session());
26
+ if (!cancelled)
27
+ setVoiceAgentId(cfg.webcallAgentId || cfg.agentId);
28
+ }
29
+ catch (err) {
30
+ if (!cancelled) {
31
+ onErrorRef.current?.(String(err?.message || err || "Voice is unavailable."));
32
+ }
33
+ }
34
+ })();
35
+ return () => { cancelled = true; };
36
+ }, [client, voiceAgentId]);
37
+ const ensureMic = useCallback(async () => {
38
+ if (Platform.OS !== "android")
39
+ return true;
40
+ try {
41
+ const perm = PermissionsAndroid?.PERMISSIONS?.RECORD_AUDIO;
42
+ if (!perm)
43
+ return true;
44
+ if (await PermissionsAndroid.check(perm))
45
+ return true;
46
+ const res = await PermissionsAndroid.request(perm, {
47
+ title: "Microphone",
48
+ message: "Allow the microphone so you can talk to the assistant.",
49
+ buttonPositive: "Allow",
50
+ });
51
+ return res === PermissionsAndroid.RESULTS.GRANTED;
52
+ }
53
+ catch {
54
+ return true;
55
+ }
56
+ }, []);
16
57
  const pushContext = useCallback(() => {
17
- const screen = client.getScreen();
18
- if (!screen)
58
+ const frame = client.buildContextFrame();
59
+ if (!frame)
19
60
  return;
20
- const { availableActions, ...rest } = screen;
21
61
  post({
22
62
  type: "graine:app-context",
23
- context: rest,
24
- availableActions: (availableActions ?? client.getAvailableActions()).filter((a) => client.getAvailableActions().includes(a)),
25
- seq: Date.now(),
63
+ context: frame.context,
64
+ availableActions: frame.available_actions,
65
+ seq: frame.seq,
26
66
  });
27
67
  }, [client, post]);
28
68
  useEffect(() => {
29
69
  if (ready)
30
70
  pushContext();
31
71
  }, [ready, pushContext]);
72
+ const beginCall = useCallback(async () => {
73
+ if (client.isConnected) {
74
+ console.warn("[graine] The provider has its own connection open while GraineVoiceLauncher " +
75
+ "is starting a call. Pass autoConnect={false} to <GraineProvider> — otherwise " +
76
+ "the agent answers twice and the customer hears two greetings.");
77
+ }
78
+ if (!(await ensureMic())) {
79
+ setConnecting(false);
80
+ onMicDenied?.("NotAllowedError");
81
+ return;
82
+ }
83
+ post({ type: "graine:start-call" });
84
+ }, [client, ensureMic, onMicDenied, post]);
85
+ useEffect(() => {
86
+ if (!connected)
87
+ return;
88
+ return client.on("app_event", (event) => {
89
+ pushContext();
90
+ const { name, ...rest } = (event ?? {});
91
+ if (!name)
92
+ return;
93
+ post({ type: "graine:app-event", name, data: rest });
94
+ });
95
+ }, [client, connected, post, pushContext]);
32
96
  const onMessage = useCallback(async (e) => {
33
97
  let msg;
34
98
  try {
@@ -38,12 +102,18 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
38
102
  return;
39
103
  }
40
104
  switch (msg.type) {
105
+ case "graine:transport":
106
+ onTransport?.({
107
+ transport: msg.transport === "rtc" ? "rtc" : "ws",
108
+ requested: msg.requested === "rtc" ? "rtc" : "ws",
109
+ });
110
+ return;
41
111
  case "graine:ready":
42
112
  setReady(true);
43
113
  pushContext();
44
114
  if (autoStart && !startedRef.current) {
45
115
  startedRef.current = true;
46
- post({ type: "graine:start-call" });
116
+ void beginCall();
47
117
  }
48
118
  return;
49
119
  case "graine:call":
@@ -80,15 +150,15 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
80
150
  return;
81
151
  }
82
152
  }
83
- }, [autoStart, client, onCaption, onCallState, onMicDenied, onEnded, post, pushContext]);
153
+ }, [autoStart, beginCall, client, onCaption, onCallState, onMicDenied, onTransport, onEnded, post, pushContext]);
84
154
  const api = {
85
155
  connected,
86
156
  connecting,
87
157
  muted,
88
158
  start: useCallback(() => {
89
159
  setConnecting(true);
90
- post({ type: "graine:start-call" });
91
- }, [post]),
160
+ void beginCall();
161
+ }, [beginCall]),
92
162
  end: useCallback(() => post({ type: "graine:end-call" }), [post]),
93
163
  setMuted: useCallback((m) => {
94
164
  setMutedState(m);
@@ -121,11 +191,11 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, onCap
121
191
  return () => { if (pending)
122
192
  clearTimeout(pending); sub.remove(); };
123
193
  }, [post]);
124
- const agentId = client.config?.agentId;
125
- if (!agentId)
194
+ if (!voiceAgentId)
126
195
  return _jsx(_Fragment, { children: children?.(api) });
127
- const src = `${client.opts.baseUrl.replace(/\/$/, "")}/embed/${agentId}` +
128
- `?mode=voice&branding=0&embed=1` +
196
+ const effectiveTransport = transport ?? (appearance?.audioTransport === "rtc" ? "rtc" : "ws");
197
+ const src = `${client.opts.baseUrl.replace(/\/$/, "")}/embed/${voiceAgentId}` +
198
+ `?mode=voice&branding=0&embed=1&transport=${effectiveTransport}` +
129
199
  (appearance?.accent ? `&accent=${encodeURIComponent(String(appearance.accent).replace("#", ""))}` : "");
130
200
  return (_jsxs(_Fragment, { children: [_jsx(View, { style: { position: "absolute", width: 1, height: 1, opacity: 0, bottom: 0, left: 0 }, pointerEvents: "none", children: _jsx(WebView, { ref: ref, source: { uri: src }, originWhitelist: ["*"], javaScriptEnabled: true, domStorageEnabled: true, allowsInlineMediaPlayback: true, mediaPlaybackRequiresUserAction: false, mediaCapturePermissionGrantType: "grant", onMessage: onMessage, style: { width: 1, height: 1, backgroundColor: "transparent" } }) }), children?.(api)] }));
131
201
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@graineai/inapp-react-native",
3
- "version": "0.15.0",
3
+ "version": "0.16.0",
4
4
  "type": "module",
5
5
  "description": "Graine in-app agent for React Native — an agent that sees the screen your customer is on and can act on it.",
6
6
  "license": "MIT",