esap-aiui-react 1.0.0 → 1.0.2

package/README.md

@@ -1,15 +1,22 @@
 # AIUI React SDK
 
-[![npm version](https://img.shields.io/npm/v/@espai/aiui-react-sdk.svg?style=flat-square)](https://www.npmjs.com/package/@espai/aiui-react-sdk)
-[![npm downloads](https://img.shields.io/npm/dm/@espai/aiui-react-sdk.svg?style=flat-square)](https://www.npmjs.com/package/@espai/aiui-react-sdk)
+[![npm version](https://img.shields.io/npm/v/esap-aiui-react.svg?style=flat-square)](https://www.npmjs.com/package/esap-aiui-react)
+[![npm downloads](https://img.shields.io/npm/dm/esap-aiui-react.svg?style=flat-square)](https://www.npmjs.com/package/esap-aiui-react)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](https://opensource.org/licenses/MIT)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.3-blue.svg?style=flat-square)](https://www.typescriptlang.org/)
-[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/espai/aiui-react-sdk/pulls)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://github.com/espai/aiui-react/pulls)
 
-**Zero-configuration voice and chat control for React applications through autonomous semantic UI discovery.**
+**Zero-configuration voice and chat control for any React applications through autonomous semantic UI discovery.**
 
 AIUI React SDK enables natural language interaction with any React application through both voice commands and text chat, without manual UI annotation or intent mapping. The framework employs real-time DOM observation and semantic element discovery to automatically understand your application's interface, allowing users to control your app through conversational voice or text-based commands.
 
+### Enterprise features at a glance
+
+- Security: API-key protected WebSocket channels and safety-rule gates
+- Privacy: client-side redaction and sensitive-field filtering
+- Auditability: server-side action logs and context update tracing
+- Deployment: cloud, private VPC, or fully on-prem
+
 ## Overview
 
 Traditional voice control solutions require extensive manual configuration, predefined intent schemas, or explicit UI element annotation. AIUI eliminates this overhead through a novel semantic discovery architecture that automatically maps UI elements to their contextual meaning, enabling immediate voice interaction with zero setup.
@@ -26,11 +33,11 @@ The SDK implements a **hybrid discovery engine** combining MutationObserver-base
 - **Privacy-preserving architecture** — Client-side filtering with configurable redaction patterns
 - **Multi-backend AI support** — Compatible with OpenAI GPT-4, Anthropic Claude, Google Gemini, and local models
 
-```markdown
+
 ## Architecture
 
-![AIUI Architecture](./assets/architecture-diagram.png)
-```
+![AIUI Architecture](https://i.imgur.com/Pvmi0Aq.png)
+
 ### Protocol Design
 
 The SDK implements a multi-channel WebSocket architecture to optimize for both latency and bandwidth:
@@ -46,7 +53,7 @@ This separation prevents JSON parsing overhead from blocking time-sensitive audi
 ## Installation
 
 ```bash
-npm install @espai/aiui-react-sdk
+npm install esap-aiui-react
 ```
 
 ### Prerequisites
@@ -151,17 +158,121 @@ your-application/
 │   ├── worklet-processor.js   # Microphone input processor
 │   └── index.html
 ├── src/
+│   ├── aiui.config.json       # AIUI configuration (REQUIRED)
 │   └── App.tsx
 └── package.json
 ```
 
+### Required: AIUI Configuration File
+
+You **must create** an `aiui.config.json` file in your **`src/`** directory. This file defines how the AIUI SDK connects to your backend and which actions are allowed on each page.
+
+#### Create `src/aiui.config.json`:
+
+```json
+{
+  "applicationId": "your-app-name",
+  "serverUrl": "http://localhost:8000",
+  "apiKey": "your-secret-key",
+  
+  "pages": [
+    {
+      "route": "/",
+      "title": "Home Page",
+      "safeActions": ["click", "fill", "navigate"],
+      "dangerousActions": []
+    },
+    {
+      "route": "/dashboard",
+      "title": "Dashboard",
+      "safeActions": ["view", "filter", "export"],
+      "dangerousActions": ["delete"]
+    },
+    {
+      "route": "/users/:id/edit",
+      "title": "Edit User",
+      "safeActions": ["edit", "save"],
+      "dangerousActions": ["delete user", "deactivate"]
+    }
+  ],
+  
+  "safetyRules": {
+    "requireConfirmation": [
+      "delete user",
+      "delete item",
+      "cancel order"
+    ],
+    "blockedSelectors": [
+      "input[type=\"password\"]",
+      "[data-sensitive=\"true\"]",
+      ".admin-only"
+    ],
+    "allowedDomains": [
+      "localhost",
+      "yourdomain.com"
+    ]
+  },
+  
+  "privacy": {
+    "exposePasswords": false,
+    "exposeCreditCards": false,
+    "redactPatterns": [
+      "[a-z0-9._%+-]+@[a-z0-9.-]+\\.[a-z]{2,}",
+      "\\b\\d{3}[-.]?\\d{3}[-.]?\\d{4}\\b"
+    ]
+  }
+}
+```
+
+#### Configuration Properties:
+
+| Property | Required | Description |
+|----------|----------|-------------|
+| `applicationId` | ✅ | Unique identifier for your application |
+| `serverUrl` | ✅ | Backend API endpoint (WebSocket server URL) |
+| `apiKey` | ✅ | Authentication key for backend connection |
+| `pages` | ✅ | Array of page routes with allowed actions |
+| `safetyRules` | ❌ | Safety configurations for dangerous actions |
+| `privacy` | ❌ | Privacy settings for sensitive data |
+
+#### Page Configuration Details:
+
+Each page in the `pages` array defines:
+- `route`: URL path (supports dynamic routes like `/users/:id`)
+- `title`: Human-readable page name
+- `safeActions`: Actions users can perform via voice/chat (e.g., "click submit", "fill email")
+- `dangerousActions`: Actions requiring user confirmation before execution
+
+#### Load the configuration in your app:
+
+```tsx
+import { AIUIProvider } from 'esap-aiui-react';
+import aiuiConfig from './aiui.config.json';
+
+function App() {
+  return (
+    <AIUIProvider config={aiuiConfig}>
+      <YourApplication />
+    </AIUIProvider>
+  );
+}
+```
+
+**Why this file is required:**
+- ✅ Backend connection - SDK needs `serverUrl` and `apiKey` to connect
+- ✅ Route mapping - Defines which actions are available on each page
+- ✅ Security - Prevents dangerous actions without confirmation
+- ✅ Navigation - Enables voice commands like "navigate to dashboard"
+
+Without this file, the SDK cannot connect to your backend or understand your application's structure.
+
 ## Quick Start
 
 ### Basic Integration
 
 ```tsx
-import { AIUIProvider, useAIUI } from '@espai/aiui-react-sdk';
-import type { AIUIConfig } from '@espai/aiui-react-sdk';
+import { AIUIProvider, useAIUI } from 'esap-aiui-react';
+import type { AIUIConfig } from 'esap-aiui-react';
 
 const config: AIUIConfig = {
   applicationId: 'production-app-v1',
@@ -204,7 +315,7 @@ function App() {
 ### Voice Control Component
 
 ```tsx
-import { useAIUI } from '@espai/aiui-react-sdk';
+import { useAIUI } from 'esap-aiui-react';
 
 function VoiceController() {
   const { 
@@ -238,7 +349,7 @@ function VoiceController() {
 ### Chat Interface Component
 
 ```tsx
-import { useAIUI } from '@espai/aiui-react-sdk';
+import { useAIUI } from 'esap-aiui-react';
 import { useState } from 'react';
 
 function ChatController() {
@@ -486,7 +597,7 @@ interface ChatMessage {
 Execute UI actions programmatically without voice input:
 
 ```typescript
-import { useAIUI } from '@espai/aiui-react-sdk';
+import { useAIUI } from 'esap-aiui-react';
 
 function DataTable() {
   const { executeAction } = useAIUI();
@@ -536,7 +647,7 @@ function DataTable() {
 Combine both voice and chat interfaces for flexible user interaction:
 
 ```tsx
-import { useAIUI } from '@espai/aiui-react-sdk';
+import { useAIUI } from 'esap-aiui-react';
 import { useState } from 'react';
 
 function AIAssistant() {
@@ -751,7 +862,7 @@ The `data-select-options` attribute defines available options using `|||` as a d
 ### Programmatic Navigation
 
 ```typescript
-import { useAIUI } from '@espai/aiui-react-sdk';
+import { useAIUI } from 'esap-aiui-react';
 
 function NavigationHandler() {
   const { executeAction } = useAIUI();
@@ -922,208 +1033,6 @@ The SDK communicates with a backend server implementing the AIUI protocol. The s
 }
 ```
 
-### Reference Server Implementation
-
-```javascript
-const WebSocket = require('ws');
-
-// Create WebSocket server with three endpoints
-const contextServer = new WebSocket.Server({ port: 8080, path: '/context' });
-const audioServer = new WebSocket.Server({ port: 8080, path: '/audio' });
-const chatServer = new WebSocket.Server({ port: 8080, path: '/chat' });
-
-// Session management
-const sessions = new Map();
-
-// Context channel handler
-contextServer.on('connection', (ws, req) => {
-  const url = new URL(req.url, 'ws://localhost');
-  const applicationId = url.searchParams.get('applicationId');
-  const apiKey = url.searchParams.get('apiKey');
-  
-  // Validate API key
-  if (!validateApiKey(apiKey)) {
-    ws.close(1008, 'Invalid API key');
-    return;
-  }
-  
-  console.log(`Context connected: ${applicationId}`);
-  
-  // Store session
-  if (!sessions.has(applicationId)) {
-    sessions.set(applicationId, {});
-  }
-  sessions.get(applicationId).contextWs = ws;
-  
-  ws.on('message', async (data) => {
-    const message = JSON.parse(data.toString());
-    
-    if (message.type === 'context_update' || message.type === 'context_append') {
-      // Store UI context for LLM
-      const session = sessions.get(applicationId);
-      session.uiContext = message.context || message.elements;
-      
-      console.log(`UI context updated: ${session.uiContext.elements?.length || 0} elements`);
-    }
-  });
-  
-  ws.on('close', () => {
-    console.log(`Context disconnected: ${applicationId}`);
-  });
-});
-
-// Audio channel handler (for voice mode)
-audioServer.on('connection', (ws, req) => {
-  const url = new URL(req.url, 'ws://localhost');
-  const applicationId = url.searchParams.get('applicationId');
-  
-  console.log(`Audio connected: ${applicationId}`);
-  
-  const session = sessions.get(applicationId);
-  if (session) {
-    session.audioWs = ws;
-  }
-  
-  let audioBuffer = [];
-  
-  ws.on('message', async (data) => {
-    if (data instanceof Buffer) {
-      // Accumulate audio chunks
-      audioBuffer.push(data);
-      
-      // Process when sufficient audio accumulated (e.g., 1 second)
-      if (audioBuffer.length >= 50) {  // 50 * 20ms = 1 second
-        const audioData = Buffer.concat(audioBuffer);
-        audioBuffer = [];
-        
-        // Send to Speech-to-Text service
-        const transcript = await speechToText(audioData);
-        
-        if (transcript && session?.uiContext) {
-          // Process with LLM
-          const action = await processWithLLM(transcript, session.uiContext);
-          
-          // Send action command via context channel
-          session.contextWs?.send(JSON.stringify({
-            type: 'action',
-            action: action.type,
-            params: action.params,
-            timestamp: new Date().toISOString()
-          }));
-          
-          // Generate TTS response
-          const audioResponse = await textToSpeech(action.response);
-          
-          // Send audio response
-          ws.send(audioResponse);
-        }
-      }
-    }
-  });
-  
-  ws.on('close', () => {
-    console.log(`Audio disconnected: ${applicationId}`);
-  });
-});
-
-// Chat channel handler (for text mode)
-chatServer.on('connection', (ws, req) => {
-  const url = new URL(req.url, 'ws://localhost');
-  const applicationId = url.searchParams.get('applicationId');
-  
-  console.log(`Chat connected: ${applicationId}`);
-  
-  const session = sessions.get(applicationId);
-  if (session) {
-    session.chatWs = ws;
-  }
-  
-  // Confirm connection
-  ws.send(JSON.stringify({ type: 'chat_connected' }));
-  
-  ws.on('message', async (data) => {
-    const message = JSON.parse(data.toString());
-    
-    if (message.type === 'chat_message') {
-      const userMessage = message.content;
-      console.log(`Chat message from ${applicationId}: ${userMessage}`);
-      
-      // Send typing indicator
-      ws.send(JSON.stringify({ type: 'chat_typing', typing: true }));
-      
-      if (session?.uiContext) {
-        // Process with LLM
-        const response = await processWithLLM(userMessage, session.uiContext);
-        
-        // Stop typing indicator
-        ws.send(JSON.stringify({ type: 'chat_typing', typing: false }));
-        
-        // If action required, send to context channel
-        if (response.action) {
-          session.contextWs?.send(JSON.stringify({
-            type: 'action',
-            action: response.type,
-            params: response.params,
-            timestamp: new Date().toISOString()
-          }));
-        }
-        
-        // Send chat response
-        ws.send(JSON.stringify({
-          type: 'chat_message',
-          role: 'assistant',
-          content: response.message,
-          timestamp: new Date().toISOString()
-        }));
-      } else {
-        ws.send(JSON.stringify({
-          type: 'chat_message',
-          role: 'assistant',
-          content: 'I don\'t have access to the UI context yet. Please ensure the page is loaded.',
-          timestamp: new Date().toISOString()
-        }));
-      }
-    }
-  });
-  
-  ws.on('close', () => {
-    console.log(`Chat disconnected: ${applicationId}`);
-    if (session) {
-      session.chatWs = null;
-    }
-  });
-});
-
-async function processWithLLM(userInput, uiContext) {
-  const prompt = `
-    You are controlling a web application via ${typeof userInput === 'string' ? 'chat' : 'voice'}.
-    
-    Current UI Context:
-    ${JSON.stringify(uiContext, null, 2)}
-    
-    User input: "${userInput}"
-    
-    Determine if an action is needed and respond in JSON format:
-    {
-      "action": true/false,
-      "type": "click" | "set_value" | "select_from_dropdown" | "navigate",
-      "params": { "semantic": "element description", "value": "..." },
-      "message": "Response to user (what you did or information about the UI)"
-    }
-    
-    If the user is just asking for information about the UI, set action to false and provide the information in message.
-  `;
-  
-  const response = await openai.chat.completions.create({
-    model: "gpt-4",
-    messages: [{ role: "user", content: prompt }],
-    response_format: { type: "json_object" }
-  });
-  
-  return JSON.parse(response.choices[0].message.content);
-}
-```
-
 ### Production Deployment Considerations
 
 **Security:**
@@ -1267,8 +1176,8 @@ We welcome contributions from the community. Please review our contribution guid
 
 ```bash
 # Clone repository
-git clone https://github.com/espai/aiui-react-sdk.git
-cd aiui-react-sdk
+git clone https://github.com/espai/aiui-react.git
+cd aiui-react
 
 # Install dependencies
 npm install
@@ -1329,7 +1238,7 @@ If you use AIUI in academic research, please cite:
   title={AIUI: Autonomous Voice-Controlled UI Framework with Zero-Configuration Semantic Discovery},
   author={Atik, Md Mahabube Alahi},
   year={2025},
-  url={https://www.npmjs.com/package/@espai/aiui-react-sdk},
+  url={https://www.npmjs.com/package/esap-aiui-react},
   version={1.0.21}
 }
 ```
@@ -1349,8 +1258,8 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI
 ## Support
 
 **Community Support:**
-- GitHub Issues: [Report bugs and request features](https://github.com/espai/aiui-react-sdk/issues)
-- GitHub Discussions: [Community Q&A and ideas](https://github.com/espai/aiui-react-sdk/discussions)
+- GitHub Issues: [Report bugs and request features](https://github.com/espai/aiui-react/issues)
+- GitHub Discussions: [Community Q&A and ideas](https://github.com/espai/aiui-react/discussions)
 - Stack Overflow: Tag questions with `aiui-react`
 
 **Commercial Support:**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "esap-aiui-react",
-  "version": "1.0.0",
+  "version": "1.0.2",
   "type": "module",
   "description": "Zero-configuration voice and chat control for React applications through autonomous semantic UI discovery",
   "main": "dist/index.js",

package/dist/auuichat.d.ts

@@ -1,57 +0,0 @@
-import * as React from 'react';
-import type { ReactNode } from 'react';
-export interface AIUIConfig {
-    applicationId: string;
-    serverUrl: string;
-    apiKey?: string;
-    pages: MinimalPageConfig[];
-    safetyRules?: SafetyRules;
-    privacy?: PrivacyConfig;
-    onNavigate?: (route: string) => void | Promise<void>;
-}
-export interface MinimalPageConfig {
-    route: string;
-    title?: string;
-    safeActions?: string[];
-    dangerousActions?: string[];
-}
-export interface SafetyRules {
-    requireConfirmation?: string[];
-    blockedSelectors?: string[];
-    allowedDomains?: string[];
-}
-export interface PrivacyConfig {
-    redactPatterns?: string[];
-    exposePasswords?: boolean;
-    exposeCreditCards?: boolean;
-}
-interface AIUIContextValue {
-    config: AIUIConfig;
-    isConnected: boolean;
-    isListening: boolean;
-    currentPage: string | null;
-    connect: () => void;
-    disconnect: () => void;
-    startListening: () => Promise<void>;
-    stopListening: () => void;
-    sendChatMessage: (message: string) => Promise<void>;
-    chatMessages: ChatMessage[];
-    isChatConnected: boolean;
-    executeAction: (action: string, params: any) => Promise<any>;
-    getComponentValue: (selector: string) => any;
-    registerComponent: (componentId: string, element: HTMLElement) => void;
-    unregisterComponent: (componentId: string) => void;
-}
-export interface ChatMessage {
-    role: 'user' | 'assistant';
-    content: string;
-    timestamp: string;
-}
-export declare const useAIUI: () => AIUIContextValue;
-export interface AIUIProviderProps {
-    config: AIUIConfig;
-    children: ReactNode;
-}
-export declare const AIUIProvider: React.FC<AIUIProviderProps>;
-export default AIUIProvider;
-//# sourceMappingURL=auuichat.d.ts.map

package/dist/auuichat.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"auuichat.d.ts","sourceRoot":"","sources":["../src/auuichat.tsx"],"names":[],"mappings":"AACA,OAAO,KAAK,KAAK,MAAM,OAAO,CAAC;AAE/B,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,OAAO,CAAC;AAOvC,MAAM,WAAW,UAAU;IACvB,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE,iBAAiB,EAAE,CAAC;IAC3B,WAAW,CAAC,EAAE,WAAW,CAAC;IAC1B,OAAO,CAAC,EAAE,aAAa,CAAC;IACxB,UAAU,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACxD;AAED,MAAM,WAAW,iBAAiB;IAC9B,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IACvB,gBAAgB,CAAC,EAAE,MAAM,EAAE,CAAC;CAC/B;AAED,MAAM,WAAW,WAAW;IACxB,mBAAmB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC/B,gBAAgB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC5B,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;CAC7B;AAED,MAAM,WAAW,aAAa;IAC1B,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;IAC1B,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC/B;AAED,UAAU,gBAAgB;IACtB,MAAM,EAAE,UAAU,CAAC;IACnB,WAAW,EAAE,OAAO,CAAC;IACrB,WAAW,EAAE,OAAO,CAAC;IACrB,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IAE3B,OAAO,EAAE,MAAM,IAAI,CAAC;IACpB,UAAU,EAAE,MAAM,IAAI,CAAC;IACvB,cAAc,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IACpC,aAAa,EAAE,MAAM,IAAI,CAAC;IAE1B,eAAe,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACpD,YAAY,EAAE,WAAW,EAAE,CAAC;IAC5B,eAAe,EAAE,OAAO,CAAC;IAEzB,aAAa,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,KAAK,OAAO,CAAC,GAAG,CAAC,CAAC;IAC7D,iBAAiB,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,GAAG,CAAC;IAC7C,iBAAiB,EAAE,CAAC,WAAW,EAAE,MAAM,EAAE,OAAO,EAAE,WAAW,KAAK,IAAI,CAAC;IACvE,mBAAmB,EAAE,CAAC,WAAW,EAAE,MAAM,KAAK,IAAI,CAAC;CACtD;AAED,MAAM,WAAW,WAAW;IACxB,IAAI,EAAE,MAAM,GAAG,WAAW,CAAC;IAC3B,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,MAAM,CAAC;CACrB;AAID,eAAO,MAAM,OAAO,QAAO,gBAM1B,CAAC;AAu3DF,MAAM,WAAW,iBAAiB;IAC9B,MAAM,EAAE,UAAU,CAAC;IACnB,QAAQ,EAAE,SAAS,CAAC;CACvB;AAED,eAAO,MAAM,YAAY,EAAE,KAAK,CAAC,EAAE,CAAC,iBAAiB,CA6RpD,CAAC;AA6BF,eAAe,YAAY,CAAC"}