guideai-app 0.3.4 → 0.4.1

package/README.md

@@ -12,6 +12,8 @@ yarn add guideai-app
 
 ## Usage
 
+### Basic Usage
+
 ```jsx
 import GuideAI from 'guideai-app';
 
@@ -28,11 +30,31 @@ function App() {
 }
 ```
 
+### With Workflow Triggers
+
+```jsx
+import GuideAI from 'guideai-app';
+
+function App() {
+  return (
+    <div>
+      <GuideAI 
+        organizationKey="your-organization-key"
+        workflowKey="customer-support"
+        position={{ bottom: '2rem', right: '2rem' }}
+        onError={(error) => console.error(error)}
+      />
+    </div>
+  );
+}
+```
+
 ## Props
 
 | Prop | Type | Required | Description |
 |------|------|----------|-------------|
 | `organizationKey` | `string` | Yes | Your organization key for workflows |
+| `workflowKey` | `string` | No | Workflow key for trigger-based workflows |
 | `position` | `object` | No | Positioning configuration for the component |
 | `onError` | `function` | No | Error handler callback |
 | `transcript` | `object` | No | Transcript feature configuration |
@@ -192,6 +214,63 @@ The transcript uses modern CSS features including:
 - Responsive design that adapts to different screen sizes
 - Custom scrollbar styling for a polished look
 
+## Element Interaction Features
+
+GuideAI includes powerful element interaction capabilities that allow the AI to guide users by highlighting, hovering over, and clicking elements on the page.
+
+### Features
+
+- **Element Highlighting**: AI can highlight and click on specific page elements
+- **Hover Interactions**: AI can hover over elements with visual feedback
+- **Position Tracking**: Cursor follows elements that move or expand during interactions
+- **Multiple Selector Support**: Supports CSS selectors, XPath, and text-based selectors
+- **Extended Visual Feedback**: 3-second highlighting with enhanced animations
+
+### Supported Selector Types
+
+| Selector Type | Format | Example |
+|---------------|--------|---------|
+| **ID** | `#elementId` | `#sidebarMenu-button-goals` |
+| **Class** | `.className` | `.submit-button` |
+| **Attribute** | `[attribute="value"]` | `[data-testid="submit"]` |
+| **Complex CSS** | Any valid CSS | `button.primary[aria-label="Save"]` |
+| **Text-based** | `text:cssSelector:textContent` | `text:.MuiListItemText-root:Goal Sets` |
+| **XPath** | `//xpath` | `//button[contains(text(), 'Submit')]` |
+
+### Available Tools
+
+#### `highlight` Tool
+- **Purpose**: Moves cursor to elements and clicks them
+- **Use Case**: Guiding users through workflows by clicking interactive elements
+- **Visual Effect**: Blue cursor with click animations and visual feedback
+
+#### `hoverThenHighlight` Tool  
+- **Purpose**: Moves cursor to elements and hovers over them
+- **Use Case**: Demonstrating or highlighting elements without triggering clicks
+- **Visual Effect**: Orange hover effects with position tracking for dynamic elements
+
+### How It Works
+
+1. **AI Detection**: The AI analyzes user requests and determines which elements to interact with
+2. **Element Finding**: Uses advanced selector logic to locate elements on the page
+3. **Visual Animation**: Animated cursor moves from viewport center to target elements
+4. **Interaction**: Performs click or hover action with appropriate visual feedback
+5. **Position Tracking**: For hover actions, tracks element position changes for 3 seconds
+
+### Example Usage
+
+The AI can respond to user requests like:
+- "Click the save button" → Uses `highlight` tool to click elements
+- "Show me the navigation menu" → Uses `hoverThenHighlight` tool to highlight elements
+- "Test hover on the goals section" → Triggers predefined test sequence
+
+### Technical Implementation
+
+- **Files**: `selectElementAndClick.ts` (clicking) and `hoverAndHighlight.ts` (hovering)
+- **State Management**: Separate state tracking for click and hover operations
+- **Error Handling**: Graceful fallback when elements are not found
+- **Performance**: Optimized with 50ms position tracking intervals
+
 ## Migration from Voice-Only Versions
 
 If you're upgrading from a voice-only version:
@@ -200,3 +279,53 @@ If you're upgrading from a voice-only version:
 - **Text Input Enabled**: Text input is now available by default
 - **Disable if Needed**: Set `input={{ enableTextInput: false }}` to disable text input
 - **Voice Still Default**: Voice mode remains the default interaction method
+
+## Workflow Trigger Feature
+
+GuideAI now supports workflow triggers based on trigger words detected in user messages. This feature allows you to create dynamic workflows that are activated when specific keywords or phrases are mentioned by users.
+
+### Features
+
+- **Trigger Word Detection**: Automatically detect trigger words in user messages
+- **Workflow Integration**: Seamlessly integrate with your backend workflow system
+- **Dynamic Prompts**: Workflow-specific prompts from the initialize-session API
+- **Tool-based Triggers**: AI can call workflow triggers through function calls
+- **Configurable**: Support for multiple workflow keys and trigger patterns
+
+### How It Works
+
+1. **Session Initialization**: When a conversation starts, the system calls `/initialize-session` with the `workflowKey` and receives a workflow-specific prompt
+2. **Trigger Word Detection**: The AI automatically analyzes user messages for trigger words and calls the `trigger_workflow` function when triggers are found
+3. **Workflow Execution**: Your backend receives the trigger request and executes workflow-specific actions
+
+### Example Workflows
+
+**Customer Support**: Trigger words like "help", "support", "issue", "problem"
+**Sales Inquiry**: Trigger words like "pricing", "cost", "buy", "purchase"
+**Technical Support**: Trigger words like "error", "bug", "crash", "not working"
+
+### Backend Requirements
+
+You only need to implement the `initialize-session` endpoint, which returns workflows along with the prompt:
+
+```json
+{
+  "id": "conversation-uuid",
+  "ephemeralToken": "openai-realtime-token",
+  "prompt": "organization-prompt",
+  "workflows": [
+    {
+      "id": "workflow-uuid",
+      "name": "Customer Support Workflow",
+      "triggers": ["help", "support", "customer service"],
+      "organizationKey": "org-key-123",
+      "prompt": "You are a helpful customer support agent...",
+      "archived": false,
+      "createdAt": "2024-01-01T00:00:00.000Z",
+      "updatedAt": "2024-01-01T00:00:00.000Z"
+    }
+  ]
+}
+```
+
+For detailed implementation guide, see [workflow-trigger-usage.md](./workflow-trigger-usage.md).

package/dist/GuideAI.d.ts

@@ -1,4 +1,3 @@
-import React from 'react';
 import { GuideAIProps } from './types/GuideAI.types';
-declare const GuideAI: (props: GuideAIProps) => React.JSX.Element | null;
+declare const GuideAI: (props: GuideAIProps) => import("react").JSX.Element | null;
 export default GuideAI;