@graineai/inapp-react-native 0.27.2 → 0.27.5

package/INTEGRATION.md

@@ -389,6 +389,46 @@ nowhere. Save is what pushes the catalogue to the agent.
 
 ---
 
+### The contract, exactly
+
+Four rules decide what the agent can do and what the customer is told. Each
+one has bitten a real integration.
+
+**Registration is per mounted screen.** `useGraineAction` registers the
+handler when the component mounts and unregisters it when it unmounts, so the
+agent is offered only what the screen in front of the customer can actually
+do. Register in the screen that performs the action, not at the root — an
+action registered at the root is offered on every screen, including the ones
+where it makes no sense, and the agent will call it there.
+
+**`availableActions` narrows; omitting it is the default you want.**
+On `useGraineScreen`, leave it out and every action registered on the mounted
+screen is offered. Pass a list to offer a subset. Pass `[]` to offer
+*nothing* — say that deliberately, for a read-only screen. It can only ever
+narrow: an action the dashboard never declared is not callable whatever
+appears here, and the server enforces that, not this field. (Before 0.27.3
+the SDK could send `[]` by accident while handlers were still registering,
+which disarmed the agent for the whole screen; it now sends `[]` only when
+you wrote `[]`.)
+
+**A handler has six seconds.** `ACTION_DEADLINE_MS`. Return promptly — do
+the slow part after you have answered. A handler that has not resolved by
+then is answered as a timeout on the runtime side, and the agent tells the
+customer it was *not* done rather than guessing. A handler that throws is
+answered as `error` for the same reason.
+
+**The status is what the customer is told.** `{ status: "ok" }` makes the
+agent say it is done, so return it only when it is. `{ status: "refused",
+message }` is often the most useful answer you can give — "this field is
+locked after submission" — and the agent passes the reason on. `message` is
+for the agent, which decides how to phrase it, not text read to the customer.
+
+**Where it all ends up.** Every call the agent makes lands in Call History
+with the app's answer beside it — "Ran set_ring (state: off) → ok" or
+"→ refused: …" — on the timeline against the turn it happened in, and in the
+execution log. A refusal there is the moment a customer was told something
+that did not happen, which is exactly the row a product team wants to find.
+
 ## Step 4b — Show them where, and draw them things
 
 ### Pointing at something on screen
@@ -518,6 +558,42 @@ your funnel.
 
 ---
 
+## Tap capture when your screens sit above the provider
+
+`GraineProvider` wraps its children in `GraineTouchCapture`, so an app whose
+whole tree hangs below the provider gets best-effort tap tracking for free
+(`captureTaps`, on by default with `fab`).
+
+That covers fewer apps than it sounds. An app that builds its client at startup
+and registers it with `setDefaultClient` can mount the provider anywhere — often
+deep, around a call bar or a single screen — while the screens the customer
+actually taps sit *above* it. Capture inside the provider then wraps the bar's
+own chrome and sees none of the taps that matter.
+
+For that layout, mount capture at your own root. Both components are exported
+and work without a provider above them: the client resolves through
+`setDefaultClient`, and the launcher-visibility gate simply does not apply when
+there is no provider context to read it from.
+
+```tsx
+import { GraineTouchCapture } from "@graineai/inapp-react-native";
+
+export default function App() {
+  return (
+    <SafeAreaProvider>
+      <GraineTouchCapture>
+        <YourNavigator />
+      </GraineTouchCapture>
+    </SafeAreaProvider>
+  );
+}
+```
+
+Taps land in the screen frame as `tap:screen` events with the tap's position;
+the runtime reads them as "tapped the screen" and Call History shows them on the
+call's timeline. Named taps from `useGraineTap` are unaffected and still carry
+their names. (0.27.4 — earlier versions did not export `GraineTouchCapture`.)
+
 ## Controlling where the launcher appears
 
 `includeScreens` for a simple allowlist:
@@ -682,6 +758,42 @@ for you on unmount and on backgrounding.
 
 ---
 
+## How long "Connecting…" takes, and why
+
+From the tap to the agent's first word there are four waits. Three of them
+are paid before the tap on a current SDK; the fourth is bounded. If the
+launcher sits on "Connecting…" for more than about two seconds, one of these
+is not happening, and the table says which.
+
+| Wait | What it is | When it is paid | Typical |
+|---|---|---|---|
+| Page load | The voice engine is a page in a WebView | **Before the tap** — `GraineFab` mounts the engine on open; a custom bar mounts `GraineVoiceLauncher` at launch with `active={false}` and flips it on open (0.27.5) | ~4s, hidden |
+| Mint + REGISTER | A 120-second SIP credential, then registration on the SBC | **Before the tap**, re-warmed as it nears expiry | ~1.6s, hidden |
+| Microphone | The OS permission prompt, first time only | At the tap | 0 once granted |
+| INVITE → answer | The call itself | At the tap | ~0.3s |
+
+**The INVITE is bounded, not left to the browser.** WebRTC gathers network
+candidates before dialling, and Chrome does not report gathering complete
+until STUN has been tried on *every* interface a device has — a VPN tunnel,
+a link-local IPv6 address, a cellular path the app may not use. Those never
+answer, and Chrome retries each for about forty seconds before giving up.
+Measured on the embed page with the SIP frames timestamped: registered at
+8.2s, the usable candidate in hand at 8.3s, the INVITE not sent until 48.3s,
+then answered in 300ms. On a phone with Wi-Fi and cellular that is the
+ten-second "Connecting…" that was reported. The engine now dials the moment a
+server-reflexive candidate exists, or 1.5 seconds after gathering starts if
+none arrives — a network with no STUN at all still gets a call,
+with host candidates, rather than the pause and then the same call.
+
+**On an app that is still on 0.26.x** the first two rows are paid *after* the
+tap: the page loads and registers only once the customer has pressed the
+microphone, so "Connecting…" is honest and about six seconds long. Moving to
+0.27.4 is what hides them. The dial bound needs no app change — it lives in
+the engine page, so every install gets it when the page is deployed.
+
+What "Connecting…" should look like on a current app, then: a tap, the
+microphone prompt once, and the greeting inside a second.
+
 ## Troubleshooting
 
 **Nothing renders at all, and the console says `no_origin`.**
@@ -965,6 +1077,7 @@ told to ignore).
 | `onMicDenied` | `(reason) => void` | `NotAllowedError` is a refusal; `NotFoundError` is a device with no usable microphone. Different sentences for a customer. |
 | `onError` | `(message) => void` | Voice could not be set up at all — key rejected, agent not enabled for apps, no network. Without this the launcher renders nothing and you cannot tell why. |
 | `children` | `(api) => ReactNode` | Receives `{ connected, connecting, muted, start, end, setMuted }`. |
+| `active` | `boolean` | Default `true`. Mount early with `false` so the engine page loads before the customer needs it (it registers nothing while idle); flip to `true` when your UI opens — with `autoStart` that places the call, without it it warms the registration so the tap is instant. Back to `false` ends a live call. (0.27.5) |
 
 **The launcher owns the connection.** It asks for the microphone, resolves the
 session, holds the socket and relays your screen, actions and events onto it. So

package/dist/client.d.ts

@@ -79,6 +79,7 @@ export declare class GraineInAppClient {
     private performAction;
     get isConnected(): boolean;
     getScreen(): ScreenContext | null;
+    private availableActionsFrameField;
     getIdentity(): Record<string, string>;
     getRecentEvents(): Array<{
         name: string;
@@ -106,7 +107,7 @@ export declare class GraineInAppClient {
     buildContextFrame(): {
         type: "app_context";
         context: Record<string, unknown>;
-        available_actions: string[];
+        available_actions?: string[];
         seq: number;
     } | null;
     private flushScreen;

package/dist/client.js

@@ -327,6 +327,18 @@ export class GraineInAppClient {
     getScreen() {
         return this.screen;
     }
+    availableActionsFrameField(declared) {
+        const usable = (declared ?? this.availableActions).filter((a) => this.actions.has(a));
+        if (usable.length)
+            return { available_actions: usable };
+        if (Array.isArray(declared))
+            return { available_actions: [] };
+        if (this.actions.size === 0) {
+            console.warn("[Graine] No in-app action handlers are registered. Falling back to the dashboard's " +
+                "catalogue so the agent is not silently disarmed — register handlers with useGraineAction.");
+        }
+        return {};
+    }
     getIdentity() {
         return { ...this.identity };
     }
@@ -478,7 +490,7 @@ export class GraineInAppClient {
                 ...(this.recentEvents.length ? { recent_events: this.recentEvents } : {}),
                 ...(this.highlightables.size ? { highlightable: [...this.highlightables] } : {}),
             }),
-            available_actions: (availableActions ?? this.availableActions).filter((a) => this.actions.has(a)),
+            ...this.availableActionsFrameField(availableActions),
             seq: ++this.screenSeq,
         };
     }

package/dist/react-native/index.d.ts

@@ -2,6 +2,8 @@ import React from "react";
 import { GraineInAppClient, type GraineInAppOptions } from "../client.js";
 import { type AudioAdapter, type VoiceSessionOptions } from "../voice.js";
 import type { ActionHandler, ScreenContext, ScreenField } from "../protocol.js";
+export { GraineFab, GraineTouchCapture } from "./fab.js";
+export type { GraineFabProps, GraineTouchCaptureProps } from "./fab.js";
 import type { FrictionConfig } from "../friction.js";
 import { type LauncherInset, type LauncherVisibility } from "./navigation.js";
 interface GraineContextValue {
@@ -133,4 +135,3 @@ export declare function useGraineReady(): {
     ready: boolean;
     error: string | null;
 };
-export {};

package/dist/react-native/index.js

@@ -4,6 +4,7 @@ import { AppState } from "react-native";
 import { GraineInAppClient } from "../client.js";
 import { VoiceSession } from "../voice.js";
 import { GraineFab, GraineTouchCapture } from "./fab.js";
+export { GraineFab, GraineTouchCapture } from "./fab.js";
 import { LauncherVisibilityTracker, activeRouteName, } from "./navigation.js";
 export const Ctx = createContext(null);
 let defaultClient = null;

package/dist/react-native/voice-launcher.d.ts

@@ -9,6 +9,7 @@ export interface GraineVoiceLauncherProps {
     }) => void;
     webView: React.ComponentType<any>;
     autoStart?: boolean;
+    active?: boolean;
     onCaption?: (c: {
         role: string;
         text: string;
@@ -32,4 +33,4 @@ export interface GraineVoiceApi {
     end: () => void;
     setMuted: (m: boolean) => void;
 }
-export declare function GraineVoiceLauncher({ webView: WebView, autoStart, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;
+export declare function GraineVoiceLauncher({ webView: WebView, autoStart, active, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }: GraineVoiceLauncherProps): React.JSX.Element;

package/dist/react-native/voice-launcher.js

@@ -3,10 +3,13 @@ import { useCallback, useEffect, useRef, useState } from "react";
 import { AppState, View } from "react-native";
 import { useGraineAgent } from "./index.js";
 import { requestMicrophonePermission } from "./permissions.js";
-export function GraineVoiceLauncher({ webView: WebView, autoStart = false, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }) {
+export function GraineVoiceLauncher({ webView: WebView, autoStart = false, active = true, transport, onTransport, onCaption, onCallState, onMicDenied, onError, onEnded, children, }) {
     const { client, appearance } = useGraineAgent();
     const ref = useRef(null);
     const startedRef = useRef(false);
+    const activeRef = useRef(active);
+    activeRef.current = active;
+    const mountedIdleRef = useRef(!active);
     const [connected, setConnected] = useState(false);
     const [connecting, setConnecting] = useState(false);
     const [muted, setMutedState] = useState(false);
@@ -113,7 +116,7 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, trans
             case "graine:ready":
                 setReady(true);
                 pushContext();
-                if ((autoStart || pendingStartRef.current) && !startedRef.current) {
+                if ((autoStart || pendingStartRef.current) && !startedRef.current && activeRef.current) {
                     pendingStartRef.current = false;
                     startedRef.current = true;
                     void beginCall();
@@ -180,6 +183,33 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, trans
         }, [post]),
     };
     useEffect(() => () => { post({ type: "graine:end-call" }); }, [post]);
+    const activeSeenRef = useRef(null);
+    useEffect(() => {
+        const first = activeSeenRef.current === null;
+        activeSeenRef.current = active;
+        if (active) {
+            if (autoStart) {
+                if (startedRef.current)
+                    return;
+                if (ready) {
+                    startedRef.current = true;
+                    void beginCall();
+                }
+                else
+                    pendingStartRef.current = true;
+            }
+            else if (!first) {
+                post({ type: "graine:warm" });
+            }
+            return;
+        }
+        if (first)
+            return;
+        if (startedRef.current || pendingStartRef.current)
+            post({ type: "graine:end-call" });
+        startedRef.current = false;
+        pendingStartRef.current = false;
+    }, [active]);
     useEffect(() => {
         let last = AppState.currentState;
         let pending = null;
@@ -211,6 +241,7 @@ export function GraineVoiceLauncher({ webView: WebView, autoStart = false, trans
     const effectiveTransport = transport ?? (appearance?.audioTransport === "rtc" ? "rtc" : "ws");
     const src = `${client.opts.baseUrl.replace(/\/$/, "")}/embed/${voiceAgentId}` +
         `?mode=voice&branding=0&embed=1&transport=${effectiveTransport}` +
+        (mountedIdleRef.current ? "&idle=1" : "") +
         `&pk=${encodeURIComponent(client.opts.publishableKey)}` +
         (appearance?.accent ? `&accent=${encodeURIComponent(String(appearance.accent).replace("#", ""))}` : "");
     return (_jsxs(_Fragment, { children: [_jsx(View, { style: { position: "absolute", width: 1, height: 1, opacity: 0, bottom: 0, left: 0 }, pointerEvents: "none", children: _jsx(WebView, { ref: ref, source: { uri: src }, originWhitelist: ["*"], javaScriptEnabled: true, domStorageEnabled: true, allowsInlineMediaPlayback: true, mediaPlaybackRequiresUserAction: false, mediaCapturePermissionGrantType: "grant", onMessage: onMessage, style: { width: 1, height: 1, backgroundColor: "transparent" } }) }), children?.(api)] }));

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@graineai/inapp-react-native",
-  "version": "0.27.2",
+  "version": "0.27.5",
   "type": "module",
   "description": "Graine in-app agent for React Native — an agent that sees the screen your customer is on and can act on it.",
   "license": "MIT",