@navai/voice-frontend 0.1.0 → 0.1.2

package/README.en.md

@@ -0,0 +1,320 @@
+# @navai/voice-frontend
+
+<p>
+  <a href="./README.es.md"><img alt="Idioma Espanol" src="https://img.shields.io/badge/Idioma-ES-0A66C2?style=for-the-badge"></a>
+  <a href="./README.en.md"><img alt="Language English" src="https://img.shields.io/badge/Language-EN-1D9A6C?style=for-the-badge"></a>
+</p>
+
+Frontend package to build Navai voice agents in web applications.
+
+It removes repeated boilerplate for:
+
+1. Realtime client secret requests.
+2. Route-aware navigation tools.
+3. Dynamic local function loading.
+4. Optional backend function bridging.
+5. React hook lifecycle for connect/disconnect.
+
+## Installation
+
+```bash
+npm install @navai/voice-frontend @openai/agents zod
+npm install react
+```
+
+## Package Architecture
+
+This package is intentionally split by concern:
+
+1. `src/backend.ts`
+HTTP client for backend routes:
+- `POST /navai/realtime/client-secret`
+- `GET /navai/functions`
+- `POST /navai/functions/execute`
+
+2. `src/runtime.ts`
+Runtime resolver for:
+- route module selection
+- function module filtering by `NAVAI_FUNCTIONS_FOLDERS`
+- optional model override
+
+3. `src/functions.ts`
+Local function loader:
+- imports modules from generated loaders
+- converts exports into normalized callable tool definitions
+
+4. `src/agent.ts`
+Agent builder:
+- creates `RealtimeAgent`
+- injects built-in tools (`navigate_to`, `execute_app_function`)
+- optionally adds direct alias tools for each allowed function
+
+5. `src/useWebVoiceAgent.ts`
+React lifecycle wrapper:
+- builds runtime config
+- requests client secret
+- discovers backend functions
+- builds agent
+- opens/closes `RealtimeSession`
+
+6. `src/routes.ts`
+Route matching helpers for natural language to path resolution.
+
+## End-to-End Runtime Flow
+
+Hook-driven runtime flow (`useWebVoiceAgent`):
+
+1. Resolve runtime config from `moduleLoaders` + `defaultRoutes` + env/options.
+2. Create backend client with `apiBaseUrl` or `NAVAI_API_URL`.
+3. On `start()`:
+- request client secret.
+- fetch backend function list.
+- build Navai agent with local + backend functions.
+- connect `RealtimeSession`.
+4. On `stop()`:
+- close session and reset state.
+
+State machine exposed by hook:
+
+- `idle`
+- `connecting`
+- `connected`
+- `error`
+
+Agent voice state exposed by the hook:
+
+- `agentVoiceState`: `idle | speaking`
+- `isAgentSpeaking`: `boolean`
+
+`agentVoiceState` is driven by realtime events `audio_start`, `audio_stopped`, and `audio_interrupted`.
+
+## Public API
+
+Main exports:
+
+- `buildNavaiAgent(...)`
+- `createNavaiBackendClient(...)`
+- `resolveNavaiFrontendRuntimeConfig(...)`
+- `loadNavaiFunctions(...)`
+- `useWebVoiceAgent(...)`
+- `resolveNavaiRoute(...)`
+- `getNavaiRoutePromptLines(...)`
+
+Useful types:
+
+- `NavaiRoute`
+- `NavaiFunctionDefinition`
+- `NavaiFunctionsRegistry`
+- `NavaiBackendFunctionDefinition`
+- `UseWebVoiceAgentOptions`
+
+## Tool Model and Behavior
+
+`buildNavaiAgent` always registers:
+
+- `navigate_to`
+- `execute_app_function`
+
+Optional direct alias tools:
+
+- for each allowed function name, a direct tool can be created.
+- reserved names are never used as direct tools (`navigate_to`, `execute_app_function`).
+- invalid tool ids are skipped (kept accessible via `execute_app_function`).
+
+Execution precedence:
+
+1. Try frontend/local function first.
+2. If missing, try backend function.
+3. If both exist with same name, frontend wins and backend is ignored with warning.
+
+## Dynamic Function Loading Internals
+
+`loadNavaiFunctions` supports module export shapes:
+
+1. Exported function.
+2. Exported class (instance methods become functions).
+3. Exported object (callable members become functions).
+
+Name normalization rules:
+
+- snake_case lowercase.
+- invalid chars removed.
+- collisions are renamed with suffixes (`_2`, `_3`, ...).
+
+Argument mapping rules:
+
+- `payload.args` or `payload.arguments` as direct args.
+- else `payload.value` as first arg.
+- else full payload as first arg.
+- context appended when arity indicates one more argument.
+
+For class methods:
+
+- constructor args: `payload.constructorArgs`.
+- method args: `payload.methodArgs`.
+
+## Runtime Resolution and Env Precedence
+
+`resolveNavaiFrontendRuntimeConfig` input priority:
+
+1. Explicit function args.
+2. Env object keys.
+3. Package defaults.
+
+Keys used:
+
+- `NAVAI_ROUTES_FILE`
+- `NAVAI_FUNCTIONS_FOLDERS`
+- `NAVAI_REALTIME_MODEL`
+
+Defaults:
+
+- routes file: `src/ai/routes.ts`
+- functions folder: `src/ai/functions-modules`
+
+Path matcher formats:
+
+- folder: `src/ai/functions-modules`
+- recursive: `src/ai/functions-modules/...`
+- wildcard: `src/features/*/voice-functions`
+- explicit file: `src/ai/functions-modules/secret.ts`
+- CSV list: `a,b,c`
+
+Fallback behavior:
+
+- if configured folders match no modules, warning is emitted.
+- resolver falls back to default functions folder.
+
+## Backend Client Behavior
+
+`createNavaiBackendClient` base URL priority:
+
+1. `apiBaseUrl` option.
+2. `env.NAVAI_API_URL`.
+3. fallback `http://localhost:3000`.
+
+Methods:
+
+- `createClientSecret(input?)`
+- `listFunctions()`
+- `executeFunction({ functionName, payload })`
+
+Error handling:
+
+- network/HTTP failures throw for create/execute.
+- function listing returns warnings and empty list on failures.
+
+## Generated Module Loader CLI
+
+This package ships:
+
+- `navai-generate-web-loaders`
+
+Default command behavior:
+
+1. Reads `.env` and process env.
+2. Resolves `NAVAI_FUNCTIONS_FOLDERS` and `NAVAI_ROUTES_FILE`.
+3. Selects modules only from configured function paths.
+4. Optionally includes configured route module if it differs from default route module.
+5. Writes `src/ai/generated-module-loaders.ts`.
+
+Manual usage:
+
+```bash
+navai-generate-web-loaders
+```
+
+Useful flags:
+
+- `--project-root <path>`
+- `--src-root <path>`
+- `--output-file <path>`
+- `--env-file <path>`
+- `--default-functions-folder <path>`
+- `--default-routes-file <path>`
+- `--type-import <module>`
+- `--export-name <identifier>`
+
+## Auto Setup on npm Install
+
+Postinstall script can auto-add missing scripts in consumer app:
+
+- `generate:module-loaders` -> `navai-generate-web-loaders`
+- `predev` -> `npm run generate:module-loaders`
+- `prebuild` -> `npm run generate:module-loaders`
+- `pretypecheck` -> `npm run generate:module-loaders`
+- `prelint` -> `npm run generate:module-loaders`
+
+Rules:
+
+- only missing scripts are added.
+- existing scripts are never overwritten.
+
+Disable auto setup:
+
+- `NAVAI_SKIP_AUTO_SETUP=1`
+- or `NAVAI_SKIP_FRONTEND_AUTO_SETUP=1`
+
+Manual setup runner:
+
+```bash
+npx navai-setup-voice-frontend
+```
+
+## Integration Examples
+
+Imperative integration:
+
+```ts
+import { RealtimeSession } from "@openai/agents/realtime";
+import { buildNavaiAgent, createNavaiBackendClient } from "@navai/voice-frontend";
+import { NAVAI_ROUTE_ITEMS } from "./ai/routes";
+import { NAVAI_WEB_MODULE_LOADERS } from "./ai/generated-module-loaders";
+
+const backend = createNavaiBackendClient({ apiBaseUrl: "http://localhost:3000" });
+const secret = await backend.createClientSecret();
+const backendList = await backend.listFunctions();
+
+const { agent, warnings } = await buildNavaiAgent({
+  navigate: (path) => router.navigate(path),
+  routes: NAVAI_ROUTE_ITEMS,
+  functionModuleLoaders: NAVAI_WEB_MODULE_LOADERS,
+  backendFunctions: backendList.functions,
+  executeBackendFunction: backend.executeFunction
+});
+
+warnings.forEach((w) => console.warn(w));
+
+const session = new RealtimeSession(agent);
+await session.connect({ apiKey: secret.value });
+```
+
+React hook integration:
+
+```ts
+import { useWebVoiceAgent } from "@navai/voice-frontend";
+import { NAVAI_WEB_MODULE_LOADERS } from "./ai/generated-module-loaders";
+import { NAVAI_ROUTE_ITEMS } from "./ai/routes";
+
+const voice = useWebVoiceAgent({
+  navigate: (path) => router.navigate(path),
+  moduleLoaders: NAVAI_WEB_MODULE_LOADERS,
+  defaultRoutes: NAVAI_ROUTE_ITEMS,
+  env: import.meta.env as Record<string, string | undefined>
+});
+```
+
+## Operational Notes
+
+- warnings are emitted with `console.warn` from runtime, backend list, and agent builder.
+- unknown function execution returns structured `ok: false` payload.
+- if route module fails to load or has invalid shape, resolver falls back to default routes.
+
+## Related Docs
+
+- Spanish version: `README.es.md`
+- English version: `README.en.md`
+- Backend package: `../voice-backend/README.md`
+- Mobile package: `../voice-mobile/README.md`
+- Playground Web: `../../apps/playground-web/README.md`
+- Playground API: `../../apps/playground-api/README.md`

package/README.es.md

@@ -0,0 +1,320 @@
+# @navai/voice-frontend
+
+<p>
+  <a href="./README.es.md"><img alt="Idioma Espanol" src="https://img.shields.io/badge/Idioma-ES-0A66C2?style=for-the-badge"></a>
+  <a href="./README.en.md"><img alt="Language English" src="https://img.shields.io/badge/Language-EN-1D9A6C?style=for-the-badge"></a>
+</p>
+
+Paquete frontend para construir agentes de voz Navai en aplicaciones web.
+
+El objetivo es evitar boilerplate repetido para:
+
+1. Solicitud de `client_secret` realtime.
+2. Tools de navegacion basadas en rutas permitidas.
+3. Carga dinamica de funciones locales.
+4. Puente opcional con funciones backend.
+5. Ciclo de vida React para conectar/desconectar sesion.
+
+## Instalacion
+
+```bash
+npm install @navai/voice-frontend @openai/agents zod
+npm install react
+```
+
+## Arquitectura del Paquete
+
+El paquete esta separado por responsabilidades:
+
+1. `src/backend.ts`
+Cliente HTTP para rutas backend:
+- `POST /navai/realtime/client-secret`
+- `GET /navai/functions`
+- `POST /navai/functions/execute`
+
+2. `src/runtime.ts`
+Resolver de runtime para:
+- seleccion de modulo de rutas
+- filtrado de modulos de funciones por `NAVAI_FUNCTIONS_FOLDERS`
+- override opcional de modelo
+
+3. `src/functions.ts`
+Loader de funciones locales:
+- importa modulos desde loaders generados
+- transforma exports en definiciones de tools normalizadas y ejecutables
+
+4. `src/agent.ts`
+Builder del agente:
+- crea `RealtimeAgent`
+- inyecta tools base (`navigate_to`, `execute_app_function`)
+- agrega aliases directos por funcion permitida cuando aplica
+
+5. `src/useWebVoiceAgent.ts`
+Wrapper de ciclo de vida React:
+- construye runtime config
+- solicita client secret
+- descubre funciones backend
+- construye el agente
+- abre/cierra `RealtimeSession`
+
+6. `src/routes.ts`
+Helpers para resolver texto natural hacia rutas permitidas.
+
+## Flujo Runtime de Punta a Punta
+
+Flujo del hook (`useWebVoiceAgent`):
+
+1. Resuelve runtime config desde `moduleLoaders` + `defaultRoutes` + env/opciones.
+2. Crea backend client con `apiBaseUrl` o `NAVAI_API_URL`.
+3. En `start()`:
+- solicita client secret.
+- solicita listado de funciones backend.
+- construye agente Navai con funciones locales + backend.
+- conecta `RealtimeSession`.
+4. En `stop()`:
+- cierra sesion y resetea estado.
+
+Maquina de estados expuesta por el hook:
+
+- `idle`
+- `connecting`
+- `connected`
+- `error`
+
+Estado de voz del agente expuesto por el hook:
+
+- `agentVoiceState`: `idle | speaking`
+- `isAgentSpeaking`: `boolean`
+
+`agentVoiceState` se actualiza con los eventos realtime `audio_start`, `audio_stopped` y `audio_interrupted`.
+
+## API Publica
+
+Exports principales:
+
+- `buildNavaiAgent(...)`
+- `createNavaiBackendClient(...)`
+- `resolveNavaiFrontendRuntimeConfig(...)`
+- `loadNavaiFunctions(...)`
+- `useWebVoiceAgent(...)`
+- `resolveNavaiRoute(...)`
+- `getNavaiRoutePromptLines(...)`
+
+Tipos utiles:
+
+- `NavaiRoute`
+- `NavaiFunctionDefinition`
+- `NavaiFunctionsRegistry`
+- `NavaiBackendFunctionDefinition`
+- `UseWebVoiceAgentOptions`
+
+## Modelo de Tools y Comportamiento
+
+`buildNavaiAgent` siempre registra:
+
+- `navigate_to`
+- `execute_app_function`
+
+Aliases directos opcionales:
+
+- para cada funcion permitida se puede crear una tool directa.
+- nombres reservados nunca se exponen como tool directa (`navigate_to`, `execute_app_function`).
+- ids invalidos de tool se omiten (la funcion sigue accesible via `execute_app_function`).
+
+Precedencia de ejecucion:
+
+1. Intenta funcion local/frontend.
+2. Si no existe, intenta funcion backend.
+3. Si ambas tienen el mismo nombre, gana frontend y backend se ignora con warning.
+
+## Internos de Carga Dinamica de Funciones
+
+`loadNavaiFunctions` soporta estos formatos de export:
+
+1. Funcion exportada.
+2. Clase exportada (metodos de instancia se vuelven funciones).
+3. Objeto exportado (miembros callables se vuelven funciones).
+
+Reglas de normalizacion de nombre:
+
+- snake_case en minusculas.
+- elimina caracteres invalidos.
+- colisiones se renombran con sufijos (`_2`, `_3`, ...).
+
+Reglas de mapeo de argumentos:
+
+- usa `payload.args` o `payload.arguments` como argumentos directos.
+- si no, usa `payload.value` como primer argumento.
+- si no, usa payload completo como primer argumento.
+- agrega contexto cuando la aridad indica un argumento adicional.
+
+Para metodos de clase:
+
+- args de constructor: `payload.constructorArgs`.
+- args del metodo: `payload.methodArgs`.
+
+## Resolucion Runtime y Precedencia de Env
+
+Prioridad de entrada en `resolveNavaiFrontendRuntimeConfig`:
+
+1. Argumentos explicitos de funcion.
+2. Claves del objeto env.
+3. Defaults del paquete.
+
+Claves usadas:
+
+- `NAVAI_ROUTES_FILE`
+- `NAVAI_FUNCTIONS_FOLDERS`
+- `NAVAI_REALTIME_MODEL`
+
+Defaults:
+
+- archivo de rutas: `src/ai/routes.ts`
+- carpeta de funciones: `src/ai/functions-modules`
+
+Formatos aceptados en matcher de rutas:
+
+- carpeta: `src/ai/functions-modules`
+- recursivo: `src/ai/functions-modules/...`
+- wildcard: `src/features/*/voice-functions`
+- archivo explicito: `src/ai/functions-modules/secret.ts`
+- CSV: `a,b,c`
+
+Comportamiento fallback:
+
+- si `NAVAI_FUNCTIONS_FOLDERS` no matchea modulos, emite warning.
+- hace fallback a carpeta de funciones por defecto.
+
+## Comportamiento del Backend Client
+
+Prioridad de base URL en `createNavaiBackendClient`:
+
+1. Opcion `apiBaseUrl`.
+2. `env.NAVAI_API_URL`.
+3. Fallback `http://localhost:3000`.
+
+Metodos:
+
+- `createClientSecret(input?)`
+- `listFunctions()`
+- `executeFunction({ functionName, payload })`
+
+Manejo de errores:
+
+- fallos de red/HTTP lanzan error en create/execute.
+- el listado de funciones retorna warnings + lista vacia en fallos.
+
+## CLI Generador de Module Loaders
+
+Este paquete incluye:
+
+- `navai-generate-web-loaders`
+
+Comportamiento por defecto:
+
+1. Lee `.env` y env del proceso.
+2. Resuelve `NAVAI_FUNCTIONS_FOLDERS` y `NAVAI_ROUTES_FILE`.
+3. Selecciona modulos solo en rutas de funciones configuradas.
+4. Incluye modulo de rutas configurado cuando difiere del modulo por defecto.
+5. Escribe `src/ai/generated-module-loaders.ts`.
+
+Uso manual:
+
+```bash
+navai-generate-web-loaders
+```
+
+Flags utiles:
+
+- `--project-root <path>`
+- `--src-root <path>`
+- `--output-file <path>`
+- `--env-file <path>`
+- `--default-functions-folder <path>`
+- `--default-routes-file <path>`
+- `--type-import <module>`
+- `--export-name <identifier>`
+
+## Auto Configuracion al Instalar desde npm
+
+El `postinstall` puede agregar scripts faltantes en el consumidor:
+
+- `generate:module-loaders` -> `navai-generate-web-loaders`
+- `predev` -> `npm run generate:module-loaders`
+- `prebuild` -> `npm run generate:module-loaders`
+- `pretypecheck` -> `npm run generate:module-loaders`
+- `prelint` -> `npm run generate:module-loaders`
+
+Reglas:
+
+- solo agrega scripts faltantes.
+- nunca sobreescribe scripts existentes.
+
+Desactivar auto setup:
+
+- `NAVAI_SKIP_AUTO_SETUP=1`
+- o `NAVAI_SKIP_FRONTEND_AUTO_SETUP=1`
+
+Ejecutor manual de setup:
+
+```bash
+npx navai-setup-voice-frontend
+```
+
+## Ejemplos de Integracion
+
+Integracion imperativa:
+
+```ts
+import { RealtimeSession } from "@openai/agents/realtime";
+import { buildNavaiAgent, createNavaiBackendClient } from "@navai/voice-frontend";
+import { NAVAI_ROUTE_ITEMS } from "./ai/routes";
+import { NAVAI_WEB_MODULE_LOADERS } from "./ai/generated-module-loaders";
+
+const backend = createNavaiBackendClient({ apiBaseUrl: "http://localhost:3000" });
+const secret = await backend.createClientSecret();
+const backendList = await backend.listFunctions();
+
+const { agent, warnings } = await buildNavaiAgent({
+  navigate: (path) => router.navigate(path),
+  routes: NAVAI_ROUTE_ITEMS,
+  functionModuleLoaders: NAVAI_WEB_MODULE_LOADERS,
+  backendFunctions: backendList.functions,
+  executeBackendFunction: backend.executeFunction
+});
+
+warnings.forEach((w) => console.warn(w));
+
+const session = new RealtimeSession(agent);
+await session.connect({ apiKey: secret.value });
+```
+
+Integracion con hook React:
+
+```ts
+import { useWebVoiceAgent } from "@navai/voice-frontend";
+import { NAVAI_WEB_MODULE_LOADERS } from "./ai/generated-module-loaders";
+import { NAVAI_ROUTE_ITEMS } from "./ai/routes";
+
+const voice = useWebVoiceAgent({
+  navigate: (path) => router.navigate(path),
+  moduleLoaders: NAVAI_WEB_MODULE_LOADERS,
+  defaultRoutes: NAVAI_ROUTE_ITEMS,
+  env: import.meta.env as Record<string, string | undefined>
+});
+```
+
+## Notas Operativas
+
+- los warnings se emiten con `console.warn` desde runtime, backend list y agent builder.
+- una funcion desconocida retorna payload estructurado `ok: false`.
+- si el modulo de rutas falla o tiene shape invalido, el resolver hace fallback a rutas por defecto.
+
+## Documentacion Relacionada
+
+- Version en espanol: `README.es.md`
+- Version en ingles: `README.en.md`
+- Paquete backend: `../voice-backend/README.md`
+- Paquete mobile: `../voice-mobile/README.md`
+- Playground Web: `../../apps/playground-web/README.md`
+- Playground API: `../../apps/playground-api/README.md`