@graineai/inapp-react-native 0.40.0 → 0.40.1

package/INTEGRATION.md

@@ -49,7 +49,7 @@ carried differently, and the SDK is what hides that from you.
                      │  hidden WebView (1×1)   │           ┌──────────────┐
                      │  the audio engine       │══ media ══▶│  the agent   │
                      │  capture · playback     │           │              │
-                     │  jitter buffer · barge-in│──control ─▶│  runtime     │
+                     │  jitter buffer · barge-in│──control ─▶│  (Graine)    │
                      └─────────────────────────┘           └──────┬───────┘
                                                                   │
                                  ▲                                │
@@ -62,7 +62,7 @@ carried differently, and the SDK is what hides that from you.
 native audio module, which means a store release for every audio fix — and the
 audio path is exactly where the fixes are. The page is one pixel, invisible and
 unreachable; your customer only ever sees your own UI. See *Why there is no
-LiveKit step* at the end.
+native setup* at the end.
 
 ### What travels, and when
 
@@ -362,11 +362,11 @@ may do in your app". The name must match exactly. Saving pushes it to the agent
 immediately.
 
 Neither half works alone. A handler with no declaration is dead code; a
-declaration with no handler is worse, because the agent will promise it and the
-runtime answers "this app build does not implement it".
+declaration with no handler is worse, because the agent will promise it and then
+have to say "this app build does not implement it".
 
 Return a message saying what actually happened — the agent repeats it, so a vague
-result becomes a vague promise. **The runtime blocks on your handler**, so return
+result becomes a vague promise. **The agent waits for your handler**, so return
 promptly; anything slow should return immediately and report completion by
 updating the screen.
 
@@ -413,7 +413,7 @@ you wrote `[]`.)
 
 **A handler has six seconds.** `ACTION_DEADLINE_MS`. Return promptly — do
 the slow part after you have answered. A handler that has not resolved by
-then is answered as a timeout on the runtime side, and the agent tells the
+then is answered as a timeout, and the agent tells the
 customer it was *not* done rather than guessing. A handler that throws is
 answered as `error` for the same reason.
 
@@ -510,7 +510,7 @@ explains a conversation: a declined payment, a rejected document, an OTP retried
 three times.
 
 The last 10 are kept. To make the agent speak first, report an event instead —
-and the runtime decides whether it is worth interrupting for.
+and the agent decides whether it is worth interrupting for.
 
 ---
 
@@ -532,7 +532,7 @@ Reserve it for a dead end the customer would want addressed without asking: a
 save that never reached the server, an upload that failed twice, a payment
 declined. Not a tap, not a navigation, not a save that worked.
 
-The runtime still decides whether to speak: silent while it is talking, while a
+The agent still decides whether to speak: silent while it is talking, while a
 reply is generating, for 20s after the last intervention, and after three in a
 session.
 
@@ -641,7 +641,7 @@ export default function App() {
 ```
 
 Taps land in the screen frame as `tap:screen` events with the tap's position;
-the runtime reads them as "tapped the screen" and Call History shows them on the
+the agent reads them as "tapped the screen" and Call History shows them on the
 call's timeline. Named taps from `useGraineTap` are unaffected and still carry
 their names. (0.27.4 — earlier versions did not export `GraineTouchCapture`.)
 
@@ -791,8 +791,13 @@ call that asked for it.
 <GraineProvider … endOnBackground={false}>   {/* default: true */}
 ```
 
-Turn it off only if a conversation should genuinely survive a switch to another
-app, and you are prepared to explain the open microphone.
+It governs the call as well as the chat — the voice launcher honours it from
+**0.40.1**; before, a call ended two seconds after the app left the foreground
+whatever the provider said. Turn it off only when your app keeps the call alive
+itself: a call screen of its own and, on Android, a microphone foreground
+service. On iOS, test on real devices first — whether a web view keeps its
+microphone in the background varies between iOS versions. See *A call like
+WhatsApp* below.
 
 Text sessions reconnect when the app returns. **Voice does not auto-resume** — a
 call is something a person chose to start, and starting one again because they
@@ -800,9 +805,8 @@ came back to the app is a microphone opening unasked. `connected` goes false;
 draw from that.
 
 **`connect()` is idempotent.** Calling it twice returns the same session rather
-than opening a second socket. This matters under React 18 strict mode, which
-mounts effects twice: previously the first socket was orphaned with no reference
-to close it, and the runtime kept it open and billing until its own timeout.
+than opening a second one — which matters under React 18 strict mode, which
+mounts effects twice.
 
 **Nothing is left holding the call.** An outstanding action's waiter is released
 in a `finally`, so a customer closing the app mid-action releases it too, and
@@ -816,38 +820,74 @@ for you on unmount and on backgrounding.
 
 ---
 
+## A call like WhatsApp
+
+A call that rings the customer's phone — CallKit and VoIP pushes on iOS, a
+ringing call notification and call screen on Android — and goes on with the
+screen off runs on Graine's native call engine, in its own package:
+`@graineai/inapp-react-native-calls`. It carries native code, so it needs a new
+store build; this SDK stays over-the-air.
+
+```tsx
+await GraineCalls.configure({ publishableKey: "pk_live_…", displayName: "Acme Assistant" });
+GraineCalls.enableVoIPPushes((token) => myServer.saveVoipToken(token));   // iOS
+// Android: GraineCalls.handlePush(message.data) first in your Firebase handlers
+await GraineCalls.startCall({ reason: "card_blocked" });                 // the customer calls the agent
+```
+
+**Attach this SDK's client** inside `GraineProvider` and the call gets the same
+screen context, identity, variables and actions as the assistant bar:
+
+```tsx
+function CallsBridge() {
+  const client = useGraineClient();
+  useEffect(() => GraineCalls.attach(client), [client]);
+  return null;
+}
+```
+
+**Two prompt variables carry the reason**: `who_called` (`agent` or
+`customer`) and `call_reason`, set by the engine on every call; give both a
+default under Embed & Widgets → Variables. The pushes, the iOS capabilities and
+your server's side are at https://www.graine.ai/docs/in-app/calling#react-native.
+
+**Without a native module** (an Expo Go build, a release shipped only over the
+air), a call can still run on this SDK's in-app engine: the launcher on a call
+screen of your own, with `endOnBackground={false}` on the provider (honoured
+for calls from 0.40.1). The ring is then a notification of your own — not
+CallKit, because the engine's microphone runs in WebKit — and Android needs a
+microphone foreground service for the length of the call. On iOS, whether a web
+view keeps its microphone in the background varies between iOS versions: test
+it on the versions you support.
+
+---
+
 ## How long "Connecting…" takes, and why
 
-From the tap to the agent's first word there are four waits. Three of them
-are paid before the tap on a current SDK; the fourth is bounded. If the
-launcher sits on "Connecting…" for more than about two seconds, one of these
-is not happening, and the table says which.
+From the tap to the agent's first word, four things happen. On a current SDK,
+three of them are done before the tap. If the launcher sits on "Connecting…"
+for more than about two seconds, one of them is not happening, and the table
+says which.
 
-| Wait | What it is | When it is paid | Typical |
+| Step | What it is | When it happens | Typical |
 |---|---|---|---|
-| Page load | The voice engine is a page in a WebView | **Before the tap** — `GraineFab` mounts the engine on open; a custom bar mounts `GraineVoiceLauncher` at launch with `active={false}` and flips it on open (0.27.5) | ~4s, hidden |
-| Mint + REGISTER | A 120-second SIP credential, then registration on the SBC | **Before the tap**, re-warmed as it nears expiry and whenever the app comes back to the foreground (0.36.0) | ~1s, hidden |
-| Microphone | The OS permission prompt, first time only | At the tap | 0 once granted |
-| INVITE → answer | The call itself, on a network path prepared before the tap | At the tap | ~0.2s |
-
-**The INVITE is bounded, not left to the browser.** WebRTC gathers network
-candidates before dialling, and Chrome does not report gathering complete
-until STUN has been tried on *every* interface a device has — a VPN tunnel,
-a link-local IPv6 address, a cellular path the app may not use. Those never
-answer, and Chrome retries each for about forty seconds before giving up.
-Measured on the embed page with the SIP frames timestamped: registered at
-8.2s, the usable candidate in hand at 8.3s, the INVITE not sent until 48.3s,
-then answered in 300ms. On a phone with Wi-Fi and cellular that is the
-ten-second "Connecting…" that was reported. The engine now dials the moment a
-server-reflexive candidate exists, or 1.5 seconds after gathering starts if
-none arrives — a network with no STUN at all still gets a call,
-with host candidates, rather than the pause and then the same call.
-
-**On an app that is still on 0.26.x** the first two rows are paid *after* the
-tap: the page loads and registers only once the customer has pressed the
-microphone, so "Connecting…" is honest and about six seconds long. Moving to
-0.27.4 is what hides them. The dial bound needs no app change — it lives in
-the engine page, so every install gets it when the page is deployed.
+| The engine loads | The voice engine is a page in a WebView | **Before the tap** — `GraineFab` loads it when it opens; a custom bar mounts `GraineVoiceLauncher` at launch with `active={false}` and turns it on when it opens (0.27.5) | ~4 s, out of sight |
+| The line is prepared | A short-lived connection to Graine, made ready | **Before the tap**, renewed before it expires and whenever the app returns to the foreground (0.36.0) | ~1 s, out of sight |
+| The microphone | The OS permission prompt, first time only | At the tap | none once granted |
+| The call connects | The call itself, on a network route found before the tap | At the tap | ~0.2 s |
+
+**Connecting never waits on a network route it does not need.** A device can
+have interfaces that never answer — a VPN tunnel, a link-local IPv6 address, a
+cellular path the app may not use — and trying every one of them can take forty
+seconds. The engine connects on the first route that works, or after 1.5
+seconds with the routes it has, so a phone on Wi-Fi and cellular connects in
+about a second rather than ten.
+
+**On an app that is still on 0.26.x** the first two steps happen *after* the
+tap: the page loads and prepares the line only once the customer has pressed
+the microphone, so "Connecting…" is honest and about six seconds long. Moving
+to 0.27.4 is what hides them. The route bound needs no app change — it lives in
+the engine page, so every install already has it.
 
 What "Connecting…" should look like on a current app, then: a tap, the
 microphone prompt once, and the greeting inside a second.
@@ -896,12 +936,12 @@ Pass it before you debug anything else.
 | `reason` | Means | Retry helps? |
 |---|---|---|
 | `not_configured` | A setting is wrong — see below. Also raised through `onError`. | Never |
-| `unreachable` | No SIP realm on the org, or the SBC is not reachable from this network. | Sometimes |
+| `unreachable` | WebRTC is not set up for your organisation, or Graine's media servers cannot be reached from this network. | Sometimes |
 | `screen_channel` | Audio connected, the screen channel did not, so the call restarted rather than run blind. | Sometimes |
 
 **`not_configured`: which setting?** The call still works — it is carried on the
 WebSocket — so nothing is on fire, but WebRTC will never be used until this is
-fixed. Ask the mint endpoint directly, without rebuilding anything:
+fixed. Ask the WebRTC endpoint directly, without rebuilding anything:
 
 ```bash
 curl -i -X POST https://www.graine.ai/api/embed/rtc-session \
@@ -917,8 +957,8 @@ curl -i -X POST https://www.graine.ai/api/embed/rtc-session \
 | 403 | `no_origin` | **The common one for apps.** A native app sends no `Origin`, so add the literal entry `app://` to that agent's Allowed domains. |
 | 403 | `no_domains_configured` | The allowlist is empty, which denies everything. |
 | 403 | `origin_not_allowed` | Add your site to Allowed domains. |
-| 503 | `not_configured` | WebRTC is not enabled on that deployment. Not fixable from the app. |
-| 200 | — | The mint works; look at `onTransport` for `unreachable` instead. |
+| 503 | `not_configured` | WebRTC is not enabled for your organisation. Ask Graine support — it is not fixable from the app. |
+| 200 | — | Your key and Allowed domains are fine; look at `onTransport` for `unreachable` instead. |
 
 That curl sends **no** `Origin` header, exactly as a native app does — so a 403
 from it is precisely the 403 your app is getting.
@@ -985,9 +1025,9 @@ A customer who taps through four screens and then asks "where am I?" gets the
 fourth. The guarantees, so you do not have to think about them:
 
 - **Immediate.** `useGraineScreen` reports a screen the moment it mounts, and
-  the voice launcher relays it to the runtime at once — the 400ms coalescing
-  applies only to the SDK's own text socket, never to a host's channel.
-- **Ordered.** Every context frame carries a monotonic `seq`; the runtime keeps
+  the voice launcher passes it on at once — the 400ms coalescing applies only
+  to the SDK's own chat connection, never to a host's channel.
+- **Ordered.** Every context frame carries a monotonic `seq`; the agent keeps
   the newest and drops anything that arrives late. The last screen always wins.
 - **Cleanup-safe.** The description is re-sent whenever its state changes and
   withdrawn only on unmount — never on a change (0.26.2; earlier versions sent
@@ -1001,9 +1041,9 @@ fourth. The guarantees, so you do not have to think about them:
   a call and when the app returns to the foreground, with its original
   sequence number — so a frame lost in flight is repaired within seconds, not
   at the next screen change; one that arrived is simply ignored.
-- **Applied every turn.** The runtime puts the last-reported screen on the
-  prompt at every generation, so nothing that rewrites the prompt mid-call — a
-  language switch, a summary — can make the agent forget where the customer is.
+- **Applied every turn.** The agent is given the last-reported screen on every
+  turn, so nothing that changes the conversation mid-call — a language switch,
+  a summary — can make it forget where the customer is.
 
 Every reporting hook works **outside a provider** too — against the client the
 provider registered, or one you pass to `setDefaultClient(client)` once — so an
@@ -1033,48 +1073,12 @@ screens already report.
 ## Captions on WebRTC
 
 Captions — the agent's line and what the customer is saying — reach the host
-on both transports. On WebRTC the audio carries no text, so the runtime
-mirrors the same caption frames over the control channel; a native bar using
+on both transports. On WebRTC the captions arrive on a channel of their own; a
+native bar using
 `onCaption` sees `role: "customer"` and `role: "agent"` lines either way.
 
 ---
 
-## Coming from RevRag
-
-Same shape, different names. If you have a RevRag integration, this is the
-whole translation:
-
-| RevRag | Graine |
-| --- | --- |
-| `useInitialize({ apiKey })` → `{ isInitialized, error }` | `useGraineReady()` → `{ ready, error }` (the key goes on `GraineProvider`) |
-| `EmbedProvider navigationRef appVersion` | `GraineProvider navigationRef appVersion` |
-| `includeScreens` | `includeScreens` |
-| `embedButtonDelayMs` | `launcherDelayMs` |
-| `embedButtonVisibilityConfig` `{ defaultDelayMs, defaultInset, groups }` | `visibility` `{ defaultDelayMs, defaultInset, groups }` |
-| `EmbedButtonGroupConfig` `{ id, screens, continuity, delayMs, delayPolicy, inset }` | `LauncherGroup` — identical fields |
-| `continuous` / `perScreen` | same |
-| `perScreen` / `oncePerGroupEntry` / `oncePerAppSession` | same |
-| `EmbedButtonInset` | `LauncherInset` — same shape |
-| `EmbedButton` / the provider mounts the FAB | `fab={{ webView }}` on `GraineProvider`, or `GraineFab` yourself |
-| Best-effort click tracking on touchables | `captureTaps` (on by default with `fab`) — `tap:screen` events |
-| `useInitialize` → `{ isInitialized, error }` | `useGraineInit()` |
-| `Embed.Event(USER_DATA, { app_user_id, data })` | `useGraineIdentify()` / `client.identify({ id, name, … })` |
-| `Embed.Event(SCREEN_STATE, { screen, data })` | `useGraineScreen({ screen, fields })` |
-| `Embed.Event(CUSTOM_EVENT, data)` / `ANALYTICS_DATA` | `useGraineTrack()` / `client.track(name, data)` |
-| `embedOnAgent(AGENT_CONVERSATION_STARTED / ENDED)` | `useGraineEvents(e => e.type === "conversation_started" / "conversation_ended")` |
-| `AgentEvent.MICROPHONE_PERMISSION_DENIED` | `{ type: "mic_denied", reason }` on the same stream |
-| `AgentEvent.POPUP_MESSAGE_VISIBLE` | `{ type: "launcher_shown", screen }` |
-| `checkPermissions()` | `requestMicrophonePermission()` |
-| server `widget_config` | `appearance` from the session, via `resolveNativeAppearance` |
-| LiveKit native setup | none — see *Why there is no LiveKit step* |
-| Friction detection (dashboard switch) | Per-screen JSON rules — `friction` on the provider, or Embed & Widgets → Friction; five named signals, `friction_detected` event |
-
-Two things RevRag has that are deliberately absent: a LiveKit install step
-(audio rides a WebView on a hosted page, so audio fixes ship without an app
-release), and click tracking on every touchable by default (`useGraineTap` is
-opt-in per element, because a stream of every tap is noise the agent has to be
-told to ignore).
-
 ## Reference
 
 | Hook | For |
@@ -1104,7 +1108,7 @@ told to ignore).
 | `visibility` | `LauncherVisibility` | Groups, continuity, per-group delays and insets. See *Controlling where the launcher appears*. |
 | `autoConnect` | `boolean` | Default `true`. Set `false` when `GraineVoiceLauncher` owns the connection — see below. |
 | `onProactive` | `(text) => void` | The agent spoke first; draw your own nudge. |
-| `endOnBackground` | `boolean` | Default `true`. You want it on. |
+| `endOnBackground` | `boolean` | Default `true`: the session — and, from 0.40.1, the call — ends two seconds after the app leaves the foreground. Turn it off only when your app keeps the call alive itself; see *A call like WhatsApp*. |
 | `appVersion` | `string` | Which build the conversation happened in. Lands on the call record. |
 | `friction` | `FrictionConfig` | This app's friction rules, layered over the dashboard's. See *Friction detection*. |
 | `loadFont` | `(fonts) => Promise` | expo-font's `Font.loadAsync`. The SDK loads the design's typeface itself — only the faces it uses, from Graine. **0.38.0** |
@@ -1137,8 +1141,8 @@ told to ignore).
 | `onEnded` | `() => void` | The call is over. Fires once. Give the goodbye ~400ms to play before you unmount. |
 | `onMicDenied` | `(reason) => void` | `NotAllowedError` is a refusal; `NotFoundError` is a device with no usable microphone. Different sentences for a customer. |
 | `onError` | `(message) => void` | Voice could not be set up at all — key rejected, agent not enabled for apps, no network. Without this the launcher renders nothing and you cannot tell why. |
-| `children` | `(api) => ReactNode` | Receives `{ connected, connecting, muted, start, end, setMuted }`. |
-| `active` | `boolean` | Default `true`. Mount early with `false` so the engine page loads before the customer needs it (it registers nothing while idle); flip to `true` when your UI opens — with `autoStart` that places the call, without it it warms the registration so the tap is instant. Back to `false` ends a live call. (0.27.5; 0.27.6 makes that hangup unconditional) |
+| `children` | `(api) => ReactNode` | Receives the voice API: `start`, `end`, `setMuted`, `connected`, `connecting`, `connectStage`, `slowConnect`, `callFailed`, `captions`, `quality`, `reconnecting`, `micDenied`, `error` and more — draw your own call from it (*Captions and hanging up*). |
+| `active` | `boolean` | Default `true`. Mount early with `false` so the engine page loads before the customer needs it (it connects nothing while idle); flip to `true` when your UI opens — with `autoStart` that places the call, without it it prepares the line so the tap is instant. Back to `false` ends a live call. (0.27.5; 0.27.6 makes that hangup unconditional) |
 
 **The launcher owns the connection.** It asks for the microphone, resolves the
 session, holds the socket and relays your screen, actions and events onto it. So
@@ -1154,20 +1158,20 @@ changed without shipping a release. Passing the prop overrides it.
 `ws` (the default) sends raw PCM on the same socket that carries screen context
 and in-app actions.
 
-`rtc` hands media to jambonz's SBC — Opus, DTLS-SRTP, an adaptive jitter buffer,
-and the platform's own echo canceller running below the microphone with the
-playout signal as its reference. Better audio on a bad network.
+`rtc` sends audio over WebRTC to Graine's media servers — Opus, encrypted
+(DTLS-SRTP), an adaptive jitter buffer, and the platform's own echo canceller.
+Better audio on a bad network.
 
-**`rtc` carries the screen too.** The browser holds two connections — media to
-the SBC, and a control socket for `app_context`, `app_action` and `app_event` —
-and the server joins them by a session id the browser puts on the INVITE. So an
-in-app agent on `rtc` sees exactly what it sees on `ws`. Frames sent before the
+**`rtc` carries the screen too.** Screen context, actions and events travel on
+a second connection that Graine joins to the call, so an in-app agent on `rtc`
+sees exactly what it sees on `ws`. Frames sent before the
 join completes are queued rather than dropped, because the first screen frame
 almost always arrives while the phone is still ringing, and it is the one the
 agent's opening line is composed from.
 
 `rtc` **falls back.** WebRTC has more ways to be unavailable than a WebSocket —
-no SIP realm on the org, an unreachable SBC, a network that eats UDP — and each
+WebRTC not set up for your organisation, media servers out of reach, a network
+that blocks UDP — and each
 is a reason to carry the call differently rather than refuse it. The page
 reconnects on the WebSocket and tells you through `onTransport`, so a permanent
 misconfiguration is visible instead of being merely audible. A refused
@@ -1175,17 +1179,13 @@ microphone is **not** a transport problem and does not fall back.
 
 ---
 
-## Why there is no LiveKit step
+## Why there is no native setup
 
-Other in-app agent SDKs run voice over LiveKit, which means native
-initialisation in `MainApplication.kt` and `AppDelegate.swift`, a pod install, a
-Lottie dependency, ProGuard rules, and a clean rebuild of both platforms. Their
-own documentation leads its troubleshooting with the error you get when one of
-those is missed.
+The SDK needs no edit to `MainApplication.kt` or `AppDelegate.swift`, no pod
+install and no native rebuild. The audio runs in the same hosted engine as the
+web widget, so an improvement to the audio — where nearly every fix lands —
+reaches your app over the air, and integration is an npm install.
 
-We put the audio in a WebView instead. The trade is real and worth stating: WebRTC
-gets acoustic echo cancellation from the platform's audio stack, and our
-level-based echo guard is an approximation of it that gives up when the speaker
-is very loud. What you get in exchange is that the audio path — where nearly
-every fix lands — ships over the air, and integration is an npm install rather
-than an afternoon in Xcode.
+The trade, stated plainly: a WebRTC call uses the phone's own echo canceller;
+on the WebSocket path a level-based guard stands in for it, and gives up when
+the speaker is very loud. Earphones or a lower volume settle it.

package/README.md

@@ -180,9 +180,8 @@ varies with the network and a player with no cushion runs dry in that gap. Hold
 queue does run dry, refill to that again before resuming — otherwise one hiccup
 becomes continuous stuttering for the rest of the turn.
 
-Report what you are holding through `bufferedSeconds()` and the SDK will
-acknowledge the agent's audio at playout rather than on arrival, which is what
-the runtime uses to know when it has finished speaking.
+Report what you are holding through `bufferedSeconds()`, so the agent knows
+when its reply has actually been heard rather than just received.
 
 ## Privacy
 
@@ -205,10 +204,19 @@ requests a permission on your behalf.
 
 > **A publishable key inside a mobile app is public.** Anyone can extract it
 > from an APK or IPA, and the web domain allowlist does not apply to mobile
-> apps. Rate limits, single-use connection tickets and your account's credit
-> limits are what bound it. See the
+> apps. Rate limits, single-use connection credentials and your account's
+> credit limits are what bound it. See the
 > [security notes](https://www.graine.ai/docs/in-app/privacy) before launch.
 
+## Calls like WhatsApp
+
+A call that rings the customer's phone — CallKit and VoIP pushes on iOS, a
+ringing call notification and call screen on Android — and goes on with the
+screen off comes in its own package, `@graineai/inapp-react-native-calls`,
+which runs Graine's native call engine and shares this SDK's screen context and
+actions (`GraineCalls.attach(client)`). It needs a new store build; this SDK
+stays over-the-air. https://www.graine.ai/docs/in-app/calling#react-native
+
 ## Requirements
 
 - React Native 0.68+

package/dist/client.d.ts

@@ -55,6 +55,10 @@ export declare class GraineInAppClient {
     private legacyStall;
     private pingTimer;
     private agentSpeaking;
+    private openedAsVoice;
+    private lastAgentAudioAt;
+    private speakingTimer;
+    private spokenTurn;
     private bargedIn;
     private muted;
     private playoutClock;
@@ -145,6 +149,8 @@ export declare class GraineInAppClient {
     setPlayoutClock(fn: (() => number) | null): void;
     get isAgentSpeaking(): boolean;
     submitWidget(widgetId: string, data: Record<string, unknown>, summary: string): void;
+    private agentAudioArrived;
+    private scheduleSpeakingEnd;
     private stopTimers;
     close(): void;
 }

package/dist/client.js

@@ -1,8 +1,9 @@
-import { contextToken, readAgentChunk, variablesForTicket, classifyQuality, ACTION_DEADLINE_MS, CONTEXT_THROTTLE_MS, } from "./protocol.js";
+import { contextToken, readAgentChunk, spokenJoinGap, variablesForTicket, classifyQuality, ACTION_DEADLINE_MS, CONTEXT_THROTTLE_MS, } from "./protocol.js";
 import { maskDeep } from "./mask.js";
 import { FrictionDetector, resolveFriction } from "./friction.js";
 export const SUBPROTOCOL = "graine.embed.v1";
 const PING_MS = 25000;
+const SPEAKING_TAIL_MS = 300;
 export class GraineInAppClient {
     allVariables() {
         return {
@@ -79,6 +80,10 @@ export class GraineInAppClient {
         this.legacyStall = false;
         this.pingTimer = null;
         this.agentSpeaking = false;
+        this.openedAsVoice = false;
+        this.lastAgentAudioAt = 0;
+        this.speakingTimer = null;
+        this.spokenTurn = "";
         this.bargedIn = false;
         this.muted = false;
         this.playoutClock = null;
@@ -248,6 +253,7 @@ export class GraineInAppClient {
             await this.session();
         const agentId = this.config.webcallAgentId || this.config.agentId;
         const t = await this.ticket();
+        this.openedAsVoice = this.socketIsVoice;
         let url;
         let protocols;
         if (t?.wsUrl && t.ticket) {
@@ -319,15 +325,24 @@ export class GraineInAppClient {
         switch (msg.type) {
             case "text": {
                 const chunk = readAgentChunk(msg.data);
-                if (chunk.startsTurn && !this.agentSpeaking) {
+                if (!this.openedAsVoice && (chunk.startsTurn || chunk.text) && !this.agentSpeaking) {
                     this.agentSpeaking = true;
                     this.emit("agent_speaking", true);
                 }
-                if (chunk.text)
-                    this.emit("chunk", chunk.text);
-                if (chunk.endsTurn && this.agentSpeaking) {
-                    this.agentSpeaking = false;
-                    this.emit("agent_speaking", false);
+                let words = chunk.text;
+                if (this.openedAsVoice && words) {
+                    words = spokenJoinGap(this.spokenTurn, words) + words;
+                    this.spokenTurn += words;
+                }
+                if (words)
+                    this.emit("chunk", words);
+                if (chunk.endsTurn) {
+                    this.spokenTurn = "";
+                    this.emit("turn_end");
+                    if (!this.openedAsVoice && this.agentSpeaking) {
+                        this.agentSpeaking = false;
+                        this.emit("agent_speaking", false);
+                    }
                 }
                 break;
             }
@@ -341,6 +356,7 @@ export class GraineInAppClient {
                 void this.runAction(msg.action);
                 break;
             case "audio":
+                this.agentAudioArrived();
                 this.emit("audio", {
                     data: msg.data,
                     format: msg.meta_info?.format ?? null,
@@ -357,6 +373,9 @@ export class GraineInAppClient {
                 break;
             case "clear":
             case "interruption":
+                clearTimeout(this.speakingTimer);
+                this.speakingTimer = null;
+                this.spokenTurn = "";
                 this.agentSpeaking = false;
                 this.emit("clear");
                 break;
@@ -684,7 +703,33 @@ export class GraineInAppClient {
         if (summary)
             this.say(summary);
     }
+    agentAudioArrived() {
+        this.lastAgentAudioAt = Date.now();
+        if (!this.agentSpeaking) {
+            this.agentSpeaking = true;
+            this.emit("agent_speaking", true);
+        }
+        this.scheduleSpeakingEnd(SPEAKING_TAIL_MS);
+    }
+    scheduleSpeakingEnd(delayMs) {
+        clearTimeout(this.speakingTimer);
+        this.speakingTimer = setTimeout(() => {
+            this.speakingTimer = null;
+            if (!this.agentSpeaking)
+                return;
+            const quietMs = Date.now() - this.lastAgentAudioAt;
+            const buffered = this.playoutClock?.() ?? 0;
+            if (quietMs >= SPEAKING_TAIL_MS && buffered <= 0.02) {
+                this.agentSpeaking = false;
+                this.emit("agent_speaking", false);
+                return;
+            }
+            this.scheduleSpeakingEnd(Math.min(1000, Math.max(50, buffered * 1000, SPEAKING_TAIL_MS - quietMs)));
+        }, delayMs);
+    }
     stopTimers() {
+        clearTimeout(this.speakingTimer);
+        this.speakingTimer = null;
         clearInterval(this.pingTimer);
         clearTimeout(this.stallTimer);
         clearTimeout(this.contextTimer);

package/dist/index.d.ts

@@ -20,7 +20,7 @@ export { PENDING_WIDGET_MS, AWAITING_REPLY_MS, MIN_LOADER_MS } from "./react-nat
 export type { GraineWidgetProps, GraineWidgetTheme } from "./react-native/widget.js";
 export { GraineInAppClient, type GraineInAppOptions, type SessionConfig, type ActionDiagnosis, } from "./client.js";
 export { FrictionDetector, resolveFriction, rulesFor, DEFAULT_FRICTION, type FrictionConfig, type FrictionScreenRules, type FrictionNudge, type FrictionSignal, type FrictionKind, } from "./friction.js";
-export { VoiceSession, liveAudioStreamAdapter, base64ToPcm16, CAPTURE_SAMPLE_RATE, type AudioAdapter, type VoiceSessionOptions, } from "./voice.js";
+export { VoiceSession, AGENT_PCM_RATE, liveAudioStreamAdapter, base64ToPcm16, CAPTURE_SAMPLE_RATE, type AudioAdapter, type VoiceSessionOptions, } from "./voice.js";
 export { EchoGuard, decodeAgentAudio, decodeMuLaw, decodePcm16, base64ToBytes, type AudioFormat, type DecodedAudio, } from "./audio.js";
 export { addMaskRule, maskDeep, maskString } from "./mask.js";
 export { GraineProvider, useGraineAgent, useGraineSurface, useGraineConnection, useGraineVariables, useKeyboardLift, liftFor, useDraggablePlacement, clampOffset, offsetDuringDrag, isTap, DRAG_SLOP_PX, useGraineScreen, useGraineAction, useGraineVoice, useGraineIdentify, useGraineEvents, useGraineField, useGraineClient, setDefaultClient, useGraineTrack, useGraineTap, useGraineHighlight, useGraineWidget, useGraineReady, useGraineInit, useGraineTheme, type GraineEvent, type GraineProviderProps, type SvgKit, type Turn, type Caption, } from "./react-native/index.js";

package/dist/index.js

@@ -11,7 +11,7 @@ export { GraineLoader, useGraineLoader } from "./react-native/loader.js";
 export { PENDING_WIDGET_MS, AWAITING_REPLY_MS, MIN_LOADER_MS } from "./react-native/index.js";
 export { GraineInAppClient, } from "./client.js";
 export { FrictionDetector, resolveFriction, rulesFor, DEFAULT_FRICTION, } from "./friction.js";
-export { VoiceSession, liveAudioStreamAdapter, base64ToPcm16, CAPTURE_SAMPLE_RATE, } from "./voice.js";
+export { VoiceSession, AGENT_PCM_RATE, liveAudioStreamAdapter, base64ToPcm16, CAPTURE_SAMPLE_RATE, } from "./voice.js";
 export { EchoGuard, decodeAgentAudio, decodeMuLaw, decodePcm16, base64ToBytes, } from "./audio.js";
 export { addMaskRule, maskDeep, maskString } from "./mask.js";
 export { GraineProvider, useGraineAgent, useGraineSurface, useGraineConnection, useGraineVariables, useKeyboardLift, liftFor, useDraggablePlacement, clampOffset, offsetDuringDrag, isTap, DRAG_SLOP_PX, useGraineScreen, useGraineAction, useGraineVoice, useGraineIdentify, useGraineEvents, useGraineField, useGraineClient, setDefaultClient, useGraineTrack, useGraineTap, useGraineHighlight, useGraineWidget, useGraineReady, useGraineInit, useGraineTheme, } from "./react-native/index.js";

package/dist/protocol.d.ts

@@ -194,6 +194,7 @@ export interface AgentChunk {
     startsTurn: boolean;
     endsTurn: boolean;
 }
+export declare function spokenJoinGap(soFar: string, next: string): string;
 export declare function readAgentChunk(raw: unknown): AgentChunk;
 export type ConnectionQuality = "excellent" | "good" | "poor" | "lost" | "unknown";
 export interface QualitySample {

package/dist/protocol.js

@@ -322,6 +322,17 @@ export function resolveSurface(appearance, prefer) {
 }
 export const STREAM_START = "<beginning_of_stream>";
 export const STREAM_END = "<end_of_stream>";
+export function spokenJoinGap(soFar, next) {
+    if (!soFar || !next)
+        return "";
+    if (/\s$/.test(soFar) || /^\s/.test(next))
+        return "";
+    if (/^[,.;:!?)\]}…%'’”»।॥]/.test(next))
+        return "";
+    if (/[(\[{“‘«\/-]$/.test(soFar))
+        return "";
+    return " ";
+}
 export function readAgentChunk(raw) {
     const text = raw == null ? "" : String(raw);
     if (!text)

package/dist/react-native/index.d.ts

@@ -43,6 +43,7 @@ interface GraineContextValue {
     fonts: Record<string, string> | null;
     faces: LoadedFaces;
     svg: SvgKit | null;
+    endOnBackground: boolean;
     sessionReady: boolean;
     onEvent: (fn: (e: GraineEvent) => void) => () => void;
     track: (name: string, data?: Record<string, unknown>) => void;

package/dist/react-native/index.js

@@ -390,9 +390,10 @@ export function GraineProvider({ children, autoConnect = true, surface: surfaceP
         connectionQuality,
         network,
         setVariables: (vars) => client.setVariables(vars),
+        endOnBackground,
     }), [client, connected, connecting, error, open, messages, widgets, muted, agentSpeaking, caption,
         launcherVisible, launcherInset, currentScreen, onEvent, appearance, design, fonts, faces, svgKit, sessionReady,
-        surface, surfacePolicy, setSurface, connectionQuality, network]);
+        surface, surfacePolicy, setSurface, connectionQuality, network, endOnBackground]);
     const wantsCapture = captureTaps ?? !!fab;
     const body = fab || wantsCapture ? (_jsxs(GraineTouchCapture, { enabled: wantsCapture, children: [children, fab ? _jsx(GraineFab, { ...fab }) : null] })) : children;
     return _jsx(Ctx.Provider, { value: value, children: body });

package/dist/react-native/voice-launcher.js

@@ -1,12 +1,15 @@
 import { Fragment as _Fragment, jsx as _jsx, jsxs as _jsxs } from "react/jsx-runtime";
-import { useCallback, useEffect, useRef, useState } from "react";
+import { useCallback, useContext, useEffect, useRef, useState } from "react";
 import { AppState, View } from "react-native";
-import { useGraineAgent } from "./index.js";
+import { Ctx, useGraineAgent } from "./index.js";
 import { requestMicrophonePermission } from "./permissions.js";
 export const CAPTION_HISTORY = 12;
 export const SLOW_CONNECT_MS = 4000;
 export function GraineVoiceLauncher({ webView: WebView, autoStart = false, active: activeProp = true, transport, onTransport, onCaption, onCallState, onMicDenied, onError, timing, onEnded, children, }) {
     const { client, appearance, surface, surfaces } = useGraineAgent();
+    const endOnBackground = useContext(Ctx)?.endOnBackground ?? true;
+    const endOnBackgroundRef = useRef(endOnBackground);
+    endOnBackgroundRef.current = endOnBackground;
     const active = activeProp && surface !== "chat";
     const ref = useRef(null);
     useEffect(() => {
@@ -518,7 +521,7 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, activ
                     post({ type: "graine:warm" });
                 return;
             }
-            if (next === "background" && wasActive && !pending) {
+            if (next === "background" && wasActive && !pending && endOnBackgroundRef.current) {
                 pending = setTimeout(() => {
                     pending = null;
                     if (AppState.currentState !== "active")

package/dist/voice.d.ts

@@ -1,5 +1,6 @@
 import type { GraineInAppClient } from "./client.js";
 export declare const CAPTURE_SAMPLE_RATE = 16000;
+export declare const AGENT_PCM_RATE = 24000;
 export declare const MIN_PLAYOUT_BUFFER_SECONDS = 0.15;
 export interface AudioAdapter {
     startCapture(onChunk: (base64: string) => void): Promise<void> | void;
@@ -23,6 +24,7 @@ export declare class VoiceSession {
     private echo;
     private agentAudible;
     private lastAudioAt;
+    private audibleUntil;
     constructor(client: GraineInAppClient, adapter: AudioAdapter, options?: VoiceSessionOptions);
     get active(): boolean;
     start(): Promise<void>;

package/dist/voice.js

@@ -1,5 +1,6 @@
 import { EchoGuard, base64ToBytes, decodeAgentAudio, decodePcm16 } from "./audio.js";
 export const CAPTURE_SAMPLE_RATE = 16000;
+export const AGENT_PCM_RATE = 24000;
 export const MIN_PLAYOUT_BUFFER_SECONDS = 0.15;
 export function base64ToPcm16(b64) {
     return decodePcm16(base64ToBytes(b64));
@@ -15,6 +16,7 @@ export class VoiceSession {
         this.echo = new EchoGuard();
         this.agentAudible = false;
         this.lastAudioAt = 0;
+        this.audibleUntil = 0;
     }
     get active() {
         return this.capturing;
@@ -25,13 +27,14 @@ export class VoiceSession {
         this.capturing = true;
         this.offs = [
             this.client.on("audio", ({ data, format, sampleRate, }) => {
-                const decoded = decodeAgentAudio(base64ToBytes(data), { format, sampleRate }, this.format);
+                const decoded = decodeAgentAudio(base64ToBytes(data), { format, sampleRate }, this.format, AGENT_PCM_RATE);
                 if (!decoded)
                     return;
                 if (decoded.latch)
                     this.format = decoded.latch;
                 this.lastAudioAt = Date.now();
                 this.adapter.play(decoded.pcm16, decoded.sampleRate);
+                this.audibleUntil = Date.now() + (this.adapter.bufferedSeconds?.() ?? 0) * 1000;
             }),
             this.client.on("clear", () => {
                 this.adapter.clear();
@@ -59,7 +62,7 @@ export class VoiceSession {
         });
     }
     agentIsAudible() {
-        if (Date.now() - this.lastAudioAt > 1500)
+        if (Date.now() - Math.max(this.lastAudioAt, this.audibleUntil) > 1500)
             return false;
         const buffered = this.adapter.bufferedSeconds?.();
         return buffered !== undefined ? buffered > 0.02 || this.agentAudible : this.agentAudible;
@@ -71,6 +74,7 @@ export class VoiceSession {
         this.format = null;
         this.agentAudible = false;
         this.lastAudioAt = 0;
+        this.audibleUntil = 0;
         this.echo.reset();
         this.offs.forEach((off) => off());
         this.offs = [];

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@graineai/inapp-react-native",
-  "version": "0.40.0",
+  "version": "0.40.1",
   "type": "module",
   "description": "Graine in-app agent for React Native — an agent that sees the screen your customer is on and can act on it.",
   "license": "MIT",