@superatomai/sdk-node 0.0.3 → 0.0.4-mds

package/README.md

@@ -1,675 +1,942 @@
-# @superatomai/sdk-node
-
-A TypeScript/Node.js SDK for building AI-powered data applications with Superatom - a platform for creating data-driven, intelligent applications with real-time WebSocket communication, LLM integration, and comprehensive user management.
-
-## Features
-
-- **WebSocket Communication** - Real-time bidirectional messaging with automatic reconnection
-- **Collection Handlers** - Register custom data operation handlers (CRUD, queries, mutations)
-- **User Management** - Built-in authentication and user storage with file-based persistence
-- **Dashboard & Report Management** - Create and manage dashboards and reports with DSL-based rendering
-- **LLM Integration** - Unified interface for Anthropic Claude and Groq models with streaming support
-- **Thread & UI Block Management** - Organize conversations and UI components with automatic cleanup
-- **Log Collection** - Capture and send runtime logs to the UI
-- **Prompt Loader** - Load custom prompts from the file system
-- **Cleanup Service** - Automatic memory management and old data removal
-- **TypeScript First** - Full type safety with comprehensive type definitions
-
-## Installation
-
-```bash
-npm install @superatomai/sdk-node
-```
-
-Or with pnpm:
-
-```bash
-pnpm add @superatomai/sdk-node
-```
-
-## Quick Start
-
-```typescript
-import { SuperatomSDK } from '@superatomai/sdk-node';
-
-// Initialize the SDK
-const sdk = new SuperatomSDK({
-  apiKey: 'your-api-key',
-  projectId: 'your-project-id',
-  userId: 'user-123', // optional, defaults to 'anonymous'
-  type: 'data-agent', // optional, defaults to 'data-agent'
-});
-
-// The SDK automatically connects to the Superatom WebSocket service
-// Wait for connection before using
-await sdk.connect();
-
-// Register a collection handler for data operations
-sdk.addCollection('users', 'getMany', async (params) => {
-  return {
-    data: [
-      { id: 1, name: 'John Doe', email: 'john@example.com' },
-      { id: 2, name: 'Jane Smith', email: 'jane@example.com' },
-    ],
-  };
-});
-
-// Listen to messages
-sdk.onMessage((message) => {
-  console.log('Received:', message.type);
-});
-
-// Cleanup when done
-await sdk.destroy();
-```
-
-## Configuration
-
-### SuperatomSDKConfig
-
-```typescript
-interface SuperatomSDKConfig {
-  apiKey: string;                    // Required: Your Superatom API key
-  projectId: string;                 // Required: Your project ID
-  userId?: string;                   // Optional: User identifier (default: 'anonymous')
-  type?: string;                     // Optional: Agent type (default: 'data-agent')
-  url?: string;                      // Optional: Custom WebSocket URL
-  bundleDir?: string;                // Optional: Directory for bundle requests
-  promptsDir?: string;               // Optional: Custom prompts directory (default: .prompts)
-  ANTHROPIC_API_KEY?: string;        // Optional: Anthropic API key for LLM
-  GROQ_API_KEY?: string;             // Optional: Groq API key for LLM
-  LLM_PROVIDERS?: LLMProvider[];     // Optional: Custom LLM providers
-}
-```
-
-### Environment Variables
-
-- `SA_WEBSOCKET_URL` - WebSocket URL (default: `wss://ws.superatom.ai/websocket`)
-- `ANTHROPIC_API_KEY` - Anthropic API key for Claude models
-- `GROQ_API_KEY` - Groq API key for open-source models
-
-## Core Features
-
-### 1. Collection Handlers
-
-Register custom handlers for data operations. Supports standard CRUD operations and custom operations.
-
-```typescript
-// Register a getMany handler
-sdk.addCollection<ParamsType, ResultType>(
-  'products',
-  'getMany',
-  async (params) => {
-    // params includes filters, pagination, etc.
-    const products = await database.products.find(params);
-    return { data: products };
-  }
-);
-
-// Register a createOne handler
-sdk.addCollection('products', 'createOne', async (params) => {
-  const newProduct = await database.products.create(params);
-  return { data: newProduct };
-});
-
-// Register a custom query handler
-sdk.addCollection('analytics', 'query', async (params) => {
-  const results = await runAnalyticsQuery(params);
-  return { data: results };
-});
-```
-
-**Supported Operations:**
-- `getMany` - Fetch multiple records
-- `getOne` - Fetch single record
-- `createOne` - Create a record
-- `updateOne` - Update a record
-- `deleteOne` - Delete a record
-- `query` - Custom queries
-- `mutation` - Custom mutations
-
-### 2. Message Handling
-
-Listen to all messages or specific message types:
-
-```typescript
-// Listen to all messages
-const unsubscribe = sdk.onMessage((message) => {
-  console.log('Message:', message.type, message.payload);
-});
-
-// Listen to specific message types
-sdk.onMessageType('DATA_REQ', (message) => {
-  console.log('Data request received:', message.payload);
-});
-
-// Unsubscribe when needed
-unsubscribe();
-
-// Send messages
-sdk.send({
-  type: 'CUSTOM_MESSAGE',
-  from: { id: 'sdk', type: 'agent' },
-  payload: { data: 'Hello' },
-});
-```
-
-**Built-in Message Types:**
-- `DATA_REQ` - Data request from runtime
-- `BUNDLE_REQ` - Bundle/asset request
-- `AUTH_LOGIN_REQ` - Authentication login request
-- `AUTH_VERIFY_REQ` - Token verification request
-- `USER_PROMPT_REQ` - User prompt for LLM processing
-- `USER_PROMPT_SUGGESTIONS_REQ` - Request for prompt suggestions
-- `ACTIONS` - Component actions request
-- `COMPONENT_LIST_RES` - Available components list
-- `USERS` - User management operations
-- `DASHBOARDS` - Dashboard CRUD operations
-- `REPORTS` - Report CRUD operations
-
-### 3. User Management
-
-Built-in user authentication and management with file-based storage:
-
-```typescript
-const userManager = sdk.getUserManager();
-
-// Create a user
-await userManager.createUser({
-  username: 'john_doe',
-  password: 'secure_password',
-});
-
-// Get a user
-const user = await userManager.getUser('john_doe');
-
-// Update a user
-await userManager.updateUser('john_doe', {
-  password: 'new_password',
-});
-
-// Delete a user
-await userManager.deleteUser('john_doe');
-
-// Get all users
-const users = await userManager.getAllUsers();
-```
-
-**Features:**
-- Automatic password hashing with bcrypt
-- File-based storage (JSON)
-- Token-based authentication
-- Auto-save on changes
-- Memory management with configurable limits
-
-### 4. Dashboard Management
-
-Create and manage dashboards with DSL-based rendering:
-
-```typescript
-const dashboardManager = sdk.getDashboardManager();
-
-// Create a dashboard
-const dashboard = {
-  id: 'dashboard-1',
-  title: 'Sales Dashboard',
-  description: 'Overview of sales metrics',
-  dsl: {
-    type: 'container',
-    children: [
-      { type: 'chart', props: { chartType: 'line', data: [...] } },
-      { type: 'table', props: { columns: [...], data: [...] } },
-    ],
-  },
-};
-
-await dashboardManager.createDashboard(dashboard);
-
-// Get a dashboard
-const retrieved = await dashboardManager.getDashboard('dashboard-1');
-
-// Update a dashboard
-await dashboardManager.updateDashboard('dashboard-1', {
-  title: 'Updated Sales Dashboard',
-});
-
-// Delete a dashboard
-await dashboardManager.deleteDashboard('dashboard-1');
-
-// Get all dashboards
-const allDashboards = await dashboardManager.getAllDashboards();
-```
-
-### 5. Report Management
-
-Similar to dashboards, but for reports:
-
-```typescript
-const reportManager = sdk.getReportManager();
-
-// Create a report
-const report = {
-  id: 'report-1',
-  title: 'Monthly Report',
-  description: 'Monthly performance report',
-  dsl: {
-    type: 'report',
-    sections: [
-      { type: 'summary', content: '...' },
-      { type: 'charts', data: [...] },
-    ],
-  },
-};
-
-await reportManager.createReport(report);
-
-// CRUD operations work the same as dashboards
-const retrieved = await reportManager.getReport('report-1');
-await reportManager.updateReport('report-1', { title: 'Q1 Report' });
-await reportManager.deleteReport('report-1');
-const allReports = await reportManager.getAllReports();
-```
-
-### 6. LLM Integration
-
-Unified interface for multiple LLM providers with streaming support:
-
-```typescript
-import { LLM } from '@superatomai/sdk-node';
-
-// Text generation
-const response = await LLM.text(
-  {
-    sys: 'You are a helpful assistant.',
-    user: 'What is the meaning of life?',
-  },
-  {
-    model: 'anthropic/claude-sonnet-4-5', // or 'groq/llama3-70b'
-    apiKey: 'your-api-key',
-    maxTokens: 1000,
-    temperature: 0.7,
-  }
-);
-
-// Streaming responses
-const result = await LLM.stream(
-  {
-    sys: 'You are a storyteller.',
-    user: 'Tell me a short story.',
-  },
-  {
-    model: 'anthropic/claude-sonnet-4-5',
-    apiKey: 'your-api-key',
-    partial: (chunk) => {
-      // Called for each chunk
-      process.stdout.write(chunk);
-    },
-  }
-);
-
-// JSON output
-const jsonResult = await LLM.stream(
-  {
-    sys: 'Return a JSON object with user data.',
-    user: 'Create a user profile for John Doe.',
-  },
-  {
-    model: 'groq/llama3-70b',
-    apiKey: 'your-api-key',
-  },
-  true // Enable JSON parsing
-);
-```
-
-**Supported Models:**
-- **Anthropic:** `claude-sonnet-4-5`, `claude-opus-4`, etc.
-- **Groq:** `llama3-70b`, `mixtral-8x7b`, etc.
-
-**Model Format:**
-- With provider: `anthropic/model-name` or `groq/model-name`
-- Without provider: Defaults to Anthropic
-
-### 7. Thread & UI Block Management
-
-Organize conversations and UI components:
-
-```typescript
-import { Thread, UIBlock, ThreadManager } from '@superatomai/sdk-node';
-
-const threadManager = ThreadManager.getInstance();
-
-// Create a thread
-const thread = threadManager.createThread({
-  userId: 'user-123',
-  metadata: { source: 'chat' },
-});
-
-// Add UI blocks
-const block1 = new UIBlock(
-  'block-1',
-  'text',
-  { content: 'Hello!' }
-);
-thread.addBlock(block1);
-
-const block2 = new UIBlock(
-  'block-2',
-  'table',
-  {
-    columns: ['Name', 'Age'],
-    data: [['John', 30], ['Jane', 25]]
-  }
-);
-thread.addBlock(block2);
-
-// Get thread
-const retrieved = threadManager.getThread(thread.getId());
-
-// Get all threads
-const allThreads = threadManager.getAllThreads();
-
-// Delete thread
-threadManager.deleteThread(thread.getId());
-```
-
-**UI Block Types:**
-- `text` - Text content
-- `table` - Tabular data
-- `chart` - Charts and visualizations
-- `form` - Interactive forms
-- Custom types as needed
-
-### 8. Log Collection
-
-Capture and send logs to the UI:
-
-```typescript
-import { UILogCollector } from '@superatomai/sdk-node';
-
-const collector = new UILogCollector();
-
-// Start collecting logs
-collector.start();
-
-// Logs are automatically captured
-console.log('This will be captured');
-console.error('Error messages too');
-
-// Add custom logs
-collector.addLog({
-  level: 'info',
-  message: 'Processing data...',
-  type: 'explanation',
-  data: { progress: 50 },
-});
-
-// Get captured logs
-const logs = collector.getLogs();
-
-// Send logs to UI (via SDK)
-const message = collector.createMessage('uiblock-123');
-sdk.send(message);
-
-// Stop collecting
-collector.stop();
-
-// Clear logs
-collector.clear();
-```
-
-### 9. Cleanup Service
-
-Automatic memory management for threads and UI blocks:
-
-```typescript
-import { CleanupService } from '@superatomai/sdk-node';
-
-const cleanupService = CleanupService.getInstance();
-
-// Start automatic cleanup (runs every 24 hours by default)
-cleanupService.startAutoCleanup(24);
-
-// Manual cleanup
-const stats = cleanupService.runFullCleanup();
-console.log('Cleanup stats:', stats);
-// Output: { threadsDeleted: 5, uiblocksDeleted: {...}, dataCleared: 10 }
-
-// Get memory statistics
-const memStats = cleanupService.getMemoryStats();
-console.log('Memory:', memStats);
-// Output: { threadCount: 42, totalUIBlocks: 156, avgUIBlocksPerThread: 3.7 }
-
-// Stop automatic cleanup
-cleanupService.stopAutoCleanup();
-```
-
-**Configuration:**
-
-```typescript
-import { STORAGE_CONFIG } from '@superatomai/sdk-node';
-
-// Thread retention (default: 30 days)
-STORAGE_CONFIG.THREAD_RETENTION_DAYS = 60;
-
-// UI block retention (default: 7 days)
-STORAGE_CONFIG.UIBLOCK_RETENTION_DAYS = 14;
-
-// Max component data size (default: 100KB)
-STORAGE_CONFIG.MAX_COMPONENT_DATA_SIZE = 200 * 1024;
-```
-
-### 10. Prompt Loader
-
-Load custom prompts from the file system:
-
-```typescript
-// Prompts are automatically loaded from .prompts/ directory
-// Or specify custom directory in config:
-const sdk = new SuperatomSDK({
-  apiKey: 'your-api-key',
-  projectId: 'your-project-id',
-  promptsDir: './my-prompts',
-});
-
-// Prompts are loaded and cached on initialization
-// They can be accessed by handlers internally
-```
-
-**Prompt File Structure:**
-```
-.prompts/
-  ├── system.txt          # System prompts
-  ├── user-greeting.txt   # User greeting prompts
-  └── analysis.txt        # Analysis prompts
-```
-
-## API Reference
-
-### SuperatomSDK
-
-#### Methods
-
-- `connect(): Promise<void>` - Connect to WebSocket service
-- `disconnect(): void` - Disconnect from WebSocket
-- `destroy(): Promise<void>` - Cleanup and disconnect permanently
-- `isConnected(): boolean` - Check connection status
-- `send(message: Message): void` - Send a message
-- `onMessage(handler): () => void` - Register message handler (returns unsubscribe function)
-- `onMessageType(type, handler): () => void` - Register type-specific handler
-- `addCollection(name, operation, handler): void` - Register collection handler
-- `getUserManager(): UserManager` - Get user manager instance
-- `getDashboardManager(): DashboardManager` - Get dashboard manager instance
-- `getReportManager(): ReportManager` - Get report manager instance
-
-### UserManager
-
-#### Methods
-
-- `createUser(data): Promise<User>` - Create a new user
-- `getUser(username): Promise<User | null>` - Get user by username
-- `updateUser(username, data): Promise<User>` - Update user
-- `deleteUser(username): Promise<boolean>` - Delete user
-- `getAllUsers(): Promise<User[]>` - Get all users
-- `verifyPassword(username, password): Promise<boolean>` - Verify password
-- `generateToken(username): Promise<string>` - Generate auth token
-- `verifyToken(token): Promise<User | null>` - Verify auth token
-
-### DashboardManager
-
-#### Methods
-
-- `createDashboard(dashboard): Promise<DSLRendererProps>` - Create dashboard
-- `getDashboard(id): Promise<DSLRendererProps | null>` - Get dashboard
-- `updateDashboard(id, updates): Promise<DSLRendererProps>` - Update dashboard
-- `deleteDashboard(id): Promise<boolean>` - Delete dashboard
-- `getAllDashboards(): Promise<DSLRendererProps[]>` - Get all dashboards
-
-### ReportManager
-
-Same methods as DashboardManager but for reports.
-
-### LLM (Static Class)
-
-#### Methods
-
-- `text(messages, options): Promise<string>` - Generate text response
-- `stream(messages, options, json?): Promise<string | object>` - Stream response
-
-### ThreadManager
-
-#### Methods
-
-- `getInstance(): ThreadManager` - Get singleton instance
-- `createThread(options): Thread` - Create new thread
-- `getThread(id): Thread | undefined` - Get thread by ID
-- `getAllThreads(): Thread[]` - Get all threads
-- `deleteThread(id): boolean` - Delete thread
-
-### CleanupService
-
-#### Methods
-
-- `getInstance(): CleanupService` - Get singleton instance
-- `startAutoCleanup(intervalHours?): void` - Start automatic cleanup
-- `stopAutoCleanup(): void` - Stop automatic cleanup
-- `runFullCleanup(): CleanupStats` - Run manual cleanup
-- `getMemoryStats(): MemoryStats` - Get current memory statistics
-
-### UILogCollector
-
-#### Methods
-
-- `start(): void` - Start collecting logs
-- `stop(): void` - Stop collecting
-- `clear(): void` - Clear collected logs
-- `getLogs(): CapturedLog[]` - Get all logs
-- `addLog(log): void` - Add custom log
-- `createMessage(uiBlockId): UILogsMessage` - Create message for SDK
-
-## Advanced Usage
-
-### Custom Message Handlers
-
-```typescript
-// Handle custom message types
-sdk.onMessageType('CUSTOM_EVENT', async (message) => {
-  console.log('Custom event:', message.payload);
-
-  // Process and respond
-  sdk.send({
-    type: 'CUSTOM_RESPONSE',
-    from: { id: 'sdk', type: 'agent' },
-    payload: { result: 'processed' },
-  });
-});
-```
-
-### Error Handling
-
-```typescript
-try {
-  await sdk.connect();
-} catch (error) {
-  console.error('Connection failed:', error);
-}
-
-// Handle WebSocket errors
-sdk.onMessageType('error', (message) => {
-  console.error('Error message:', message.payload);
-});
-```
-
-### Reconnection
-
-The SDK automatically handles reconnection with exponential backoff:
-- Max attempts: 5
-- Delay: 1s, 2s, 4s, 8s, 10s (capped at 10s)
-
-### Bundle Requests
-
-Serve static files via bundle requests:
-
-```typescript
-const sdk = new SuperatomSDK({
-  apiKey: 'your-api-key',
-  projectId: 'your-project-id',
-  bundleDir: './public', // Directory to serve files from
-});
-
-// Files in ./public will be served when requested
-```
-
-## TypeScript Support
-
-This SDK is written in TypeScript and includes full type definitions. All exports are fully typed for the best development experience.
-
-```typescript
-import type {
-  Message,
-  IncomingMessage,
-  SuperatomSDKConfig,
-  CollectionHandler,
-  User,
-  DSLRendererProps,
-} from '@superatomai/sdk-node';
-```
-
-## Examples
-
-Check out the [examples](./examples) directory for complete examples:
-- Basic WebSocket communication
-- Data collection handlers
-- User authentication flow
-- LLM integration
-- Thread management
-- Dashboard creation
-
-## Development
-
-```bash
-# Install dependencies
-pnpm install
-
-# Build the SDK
-pnpm run build
-
-# Watch mode
-pnpm run dev
-
-# Run tests
-pnpm test
-```
-
-## License
-
-MIT
-
-## Support
-
-For issues and questions:
-- **GitHub Issues:** [Report an issue](https://github.com/superatomai/sdk-nodejs/issues)
-- **Email:** ashish@superatom.ai
-
-## Contributing
-
-Contributions are welcome! Please feel free to submit a Pull Request.
+# @superatomai/sdk-node
+
+A TypeScript/Node.js SDK for building AI-powered data applications with Superatom - a platform for creating data-driven, intelligent applications with real-time WebSocket communication, LLM integration, and comprehensive user management.
+
+## Features
+
+- **WebSocket Communication** - Real-time bidirectional messaging with automatic reconnection
+- **AI-Powered Component Generation** - Automatically generate dashboard components and visualizations from user questions
+- **Intelligent Text Responses** - LLM-powered conversational responses with SQL query execution and retry logic
+- **Component Matching & Classification** - Smart component selection based on question type and visualization needs
+- **Collection Handlers** - Register custom data operation handlers (CRUD, queries, mutations)
+- **User Management** - Built-in authentication and user storage with file-based persistence
+- **Dashboard & Report Management** - Create and manage dashboards and reports with DSL-based rendering
+- **Multi-Provider LLM Integration** - Unified interface for Anthropic Claude and Groq models with automatic fallback
+- **Thread & UI Block Management** - Organize conversations and UI components with automatic cleanup
+- **Log Collection** - Capture and send runtime logs to the UI with configurable log levels
+- **Prompt Loader** - Load and cache custom prompts from the file system with template variable support
+- **Database Schema Management** - Automatic schema documentation generation for LLM context
+- **Cleanup Service** - Automatic memory management and old data removal
+- **TypeScript First** - Full type safety with comprehensive type definitions
+
+## Installation
+
+```bash
+npm install @superatomai/sdk-node
+```
+
+Or with pnpm:
+
+```bash
+pnpm add @superatomai/sdk-node
+```
+
+## Quick Start
+
+```typescript
+import { SuperatomSDK } from '@superatomai/sdk-node';
+
+// Initialize the SDK
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  userId: 'user-123', // optional, defaults to 'anonymous'
+  type: 'data-agent', // optional, defaults to 'data-agent'
+});
+
+// The SDK automatically connects to the Superatom WebSocket service
+// Wait for connection before using
+await sdk.connect();
+
+// Register a collection handler for data operations
+sdk.addCollection('users', 'getMany', async (params) => {
+  return {
+    data: [
+      { id: 1, name: 'John Doe', email: 'john@example.com' },
+      { id: 2, name: 'Jane Smith', email: 'jane@example.com' },
+    ],
+  };
+});
+
+// Listen to messages
+sdk.onMessage((message) => {
+  console.log('Received:', message.type);
+});
+
+// Cleanup when done
+await sdk.destroy();
+```
+
+## Configuration
+
+### SuperatomSDKConfig
+
+```typescript
+interface SuperatomSDKConfig {
+  apiKey: string;                    // Required: Your Superatom API key
+  projectId: string;                 // Required: Your project ID
+  userId?: string;                   // Optional: User identifier (default: 'anonymous')
+  type?: string;                     // Optional: Agent type (default: 'data-agent')
+  url?: string;                      // Optional: Custom WebSocket URL
+  bundleDir?: string;                // Optional: Directory for bundle requests
+  promptsDir?: string;               // Optional: Custom prompts directory (default: .prompts)
+  ANTHROPIC_API_KEY?: string;        // Optional: Anthropic API key for LLM
+  GROQ_API_KEY?: string;             // Optional: Groq API key for LLM
+  LLM_PROVIDERS?: LLMProvider[];     // Optional: Custom LLM providers
+  logLevel?: LogLevel;               // Optional: Log level (errors, warnings, info, verbose) (default: 'info')
+}
+```
+
+### Environment Variables
+
+- `SA_WEBSOCKET_URL` - WebSocket URL (default: `wss://ws.superatom.ai/websocket`)
+- `ANTHROPIC_API_KEY` - Anthropic API key for Claude models
+- `GROQ_API_KEY` - Groq API key for open-source models
+- `SUPERATOM_LOG_LEVEL` - Log level for SDK logging (errors, warnings, info, verbose) (default: 'info')
+
+## Core Features
+
+### 1. Collection Handlers
+
+Register custom handlers for data operations. Supports standard CRUD operations and custom operations.
+
+```typescript
+// Register a getMany handler
+sdk.addCollection<ParamsType, ResultType>(
+  'products',
+  'getMany',
+  async (params) => {
+    // params includes filters, pagination, etc.
+    const products = await database.products.find(params);
+    return { data: products };
+  }
+);
+
+// Register a createOne handler
+sdk.addCollection('products', 'createOne', async (params) => {
+  const newProduct = await database.products.create(params);
+  return { data: newProduct };
+});
+
+// Register a custom query handler
+sdk.addCollection('analytics', 'query', async (params) => {
+  const results = await runAnalyticsQuery(params);
+  return { data: results };
+});
+```
+
+**Supported Operations:**
+- `getMany` - Fetch multiple records
+- `getOne` - Fetch single record
+- `createOne` - Create a record
+- `updateOne` - Update a record
+- `deleteOne` - Delete a record
+- `query` - Custom queries
+- `mutation` - Custom mutations
+
+### 2. Message Handling
+
+Listen to all messages or specific message types:
+
+```typescript
+// Listen to all messages
+const unsubscribe = sdk.onMessage((message) => {
+  console.log('Message:', message.type, message.payload);
+});
+
+// Listen to specific message types
+sdk.onMessageType('DATA_REQ', (message) => {
+  console.log('Data request received:', message.payload);
+});
+
+// Unsubscribe when needed
+unsubscribe();
+
+// Send messages
+sdk.send({
+  type: 'CUSTOM_MESSAGE',
+  from: { id: 'sdk', type: 'agent' },
+  payload: { data: 'Hello' },
+});
+```
+
+**Built-in Message Types:**
+- `DATA_REQ` - Data request from runtime
+- `BUNDLE_REQ` - Bundle/asset request
+- `AUTH_LOGIN_REQ` - Authentication login request
+- `AUTH_VERIFY_REQ` - Token verification request
+- `USER_PROMPT_REQ` - User prompt for LLM processing
+- `USER_PROMPT_SUGGESTIONS_REQ` - Request for prompt suggestions
+- `ACTIONS` - Component actions request
+- `COMPONENT_LIST_RES` - Available components list
+- `USERS` - User management operations
+- `DASHBOARDS` - Dashboard CRUD operations
+- `REPORTS` - Report CRUD operations
+
+### 3. User Management
+
+Built-in user authentication and management with file-based storage:
+
+```typescript
+const userManager = sdk.getUserManager();
+
+// Create a user
+await userManager.createUser({
+  username: 'john_doe',
+  password: 'secure_password',
+});
+
+// Get a user
+const user = await userManager.getUser('john_doe');
+
+// Update a user
+await userManager.updateUser('john_doe', {
+  password: 'new_password',
+});
+
+// Delete a user
+await userManager.deleteUser('john_doe');
+
+// Get all users
+const users = await userManager.getAllUsers();
+```
+
+**Features:**
+- Automatic password hashing with bcrypt
+- File-based storage (JSON)
+- Token-based authentication
+- Auto-save on changes
+- Memory management with configurable limits
+
+### 4. Dashboard Management
+
+Create and manage dashboards with DSL-based rendering:
+
+```typescript
+const dashboardManager = sdk.getDashboardManager();
+
+// Create a dashboard
+const dashboard = {
+  id: 'dashboard-1',
+  title: 'Sales Dashboard',
+  description: 'Overview of sales metrics',
+  dsl: {
+    type: 'container',
+    children: [
+      { type: 'chart', props: { chartType: 'line', data: [...] } },
+      { type: 'table', props: { columns: [...], data: [...] } },
+    ],
+  },
+};
+
+await dashboardManager.createDashboard(dashboard);
+
+// Get a dashboard
+const retrieved = await dashboardManager.getDashboard('dashboard-1');
+
+// Update a dashboard
+await dashboardManager.updateDashboard('dashboard-1', {
+  title: 'Updated Sales Dashboard',
+});
+
+// Delete a dashboard
+await dashboardManager.deleteDashboard('dashboard-1');
+
+// Get all dashboards
+const allDashboards = await dashboardManager.getAllDashboards();
+```
+
+### 5. Report Management
+
+Similar to dashboards, but for reports:
+
+```typescript
+const reportManager = sdk.getReportManager();
+
+// Create a report
+const report = {
+  id: 'report-1',
+  title: 'Monthly Report',
+  description: 'Monthly performance report',
+  dsl: {
+    type: 'report',
+    sections: [
+      { type: 'summary', content: '...' },
+      { type: 'charts', data: [...] },
+    ],
+  },
+};
+
+await reportManager.createReport(report);
+
+// CRUD operations work the same as dashboards
+const retrieved = await reportManager.getReport('report-1');
+await reportManager.updateReport('report-1', { title: 'Q1 Report' });
+await reportManager.deleteReport('report-1');
+const allReports = await reportManager.getAllReports();
+```
+
+### 6. LLM Integration
+
+Unified interface for multiple LLM providers with streaming support:
+
+```typescript
+import { LLM } from '@superatomai/sdk-node';
+
+// Text generation
+const response = await LLM.text(
+  {
+    sys: 'You are a helpful assistant.',
+    user: 'What is the meaning of life?',
+  },
+  {
+    model: 'anthropic/claude-sonnet-4-5', // or 'groq/llama3-70b'
+    apiKey: 'your-api-key',
+    maxTokens: 1000,
+    temperature: 0.7,
+  }
+);
+
+// Streaming responses
+const result = await LLM.stream(
+  {
+    sys: 'You are a storyteller.',
+    user: 'Tell me a short story.',
+  },
+  {
+    model: 'anthropic/claude-sonnet-4-5',
+    apiKey: 'your-api-key',
+    partial: (chunk) => {
+      // Called for each chunk
+      process.stdout.write(chunk);
+    },
+  }
+);
+
+// JSON output
+const jsonResult = await LLM.stream(
+  {
+    sys: 'Return a JSON object with user data.',
+    user: 'Create a user profile for John Doe.',
+  },
+  {
+    model: 'groq/llama3-70b',
+    apiKey: 'your-api-key',
+  },
+  true // Enable JSON parsing
+);
+```
+
+**Supported Models:**
+- **Anthropic:** `claude-sonnet-4-5`, `claude-opus-4`, etc.
+- **Groq:** `llama3-70b`, `mixtral-8x7b`, etc.
+
+**Model Format:**
+- With provider: `anthropic/model-name` or `groq/model-name`
+- Without provider: Defaults to Anthropic
+
+### 7. AI-Powered User Response System
+
+The SDK includes a powerful AI system for generating intelligent responses to user questions with two modes: **component generation** and **text responses**.
+
+#### Component Generation Mode
+
+Automatically match and generate dashboard components based on user questions:
+
+```typescript
+import { get_user_response } from '@superatomai/sdk-node/userResponse';
+
+const components = [
+  { id: 'sales-chart', name: 'SalesChart', type: 'BarChart', /* ... */ },
+  { id: 'kpi-card', name: 'RevenueKPI', type: 'KPICard', /* ... */ },
+  // ... more components
+];
+
+const result = await get_user_response(
+  'Show me sales by region',
+  components,
+  anthropicApiKey,
+  groqApiKey,
+  ['anthropic', 'groq'], // Provider fallback order
+  logCollector,
+  conversationHistory,
+  'component' // Component generation mode
+);
+
+if (result.success) {
+  console.log('Generated component:', result.data.component);
+  console.log('Reasoning:', result.data.reasoning);
+}
+```
+
+**Features:**
+- **Question Classification** - Automatically determines question type (analytical, data_modification, general)
+- **Visualization Type Detection** - Identifies required visualization types (charts, tables, KPIs)
+- **Multi-Component Dashboards** - Generates multiple components for comprehensive analysis
+- **Props Modification** - Intelligently modifies component props including SQL queries
+- **SQL Query Validation** - Ensures queries have proper LIMIT clauses and fixes scalar subqueries
+
+#### Text Response Mode
+
+Generate conversational text responses with SQL query execution:
+
+```typescript
+const result = await get_user_response(
+  'What were the top 5 selling products last month?',
+  components,
+  anthropicApiKey,
+  groqApiKey,
+  ['anthropic', 'groq'],
+  logCollector,
+  conversationHistory,
+  'text', // Text response mode
+  (chunk) => {
+    // Stream text chunks in real-time
+    process.stdout.write(chunk);
+  },
+  collections // Required for query execution
+);
+
+if (result.success) {
+  console.log('Text response:', result.data.text);
+  console.log('Matched components:', result.data.matchedComponents);
+  console.log('Container component:', result.data.component);
+}
+```
+
+**Features:**
+- **SQL Query Execution** - Automatically generates and executes SQL queries via tool calling
+- **Automatic Retry Logic** - Up to 6 retry attempts with query correction on errors
+- **Streaming Responses** - Real-time text streaming with query execution status updates
+- **Component Suggestions** - Parses component suggestions from text and matches with available components
+- **Layout Discovery** - Automatically selects appropriate dashboard layouts based on component metadata
+- **Performance Tracking** - Measures and logs total time taken for request processing
+
+#### Using BaseLLM Classes Directly
+
+For more control, use the BaseLLM implementations directly:
+
+```typescript
+import { anthropicLLM, groqLLM } from '@superatomai/sdk-node/userResponse';
+
+// Classify user question
+const classification = await anthropicLLM.classifyUserQuestion(
+  'Show me sales trends',
+  apiKey,
+  logCollector,
+  conversationHistory
+);
+
+console.log('Question type:', classification.questionType);
+console.log('Visualizations needed:', classification.visualizations);
+
+// Generate analytical component
+const result = await anthropicLLM.generateAnalyticalComponent(
+  'Show me sales by region',
+  components,
+  'BarChart', // Preferred visualization type
+  apiKey,
+  logCollector,
+  conversationHistory
+);
+
+// Match existing component
+const matchResult = await anthropicLLM.matchComponent(
+  'Update the sales dashboard',
+  components,
+  apiKey,
+  logCollector,
+  conversationHistory
+);
+
+// Generate next questions
+const nextQuestions = await anthropicLLM.generateNextQuestions(
+  originalPrompt,
+  generatedComponent,
+  componentData,
+  apiKey,
+  logCollector,
+  conversationHistory
+);
+```
+
+**BaseLLM Methods:**
+- `handleUserRequest()` - Main orchestration method (supports both component and text modes)
+- `classifyUserQuestion()` - Classify question type and identify visualizations
+- `generateAnalyticalComponent()` - Generate single analytical component
+- `generateMultipleAnalyticalComponents()` - Generate multiple components in parallel
+- `matchComponent()` - Match and modify existing component
+- `validateAndModifyProps()` - Validate and modify component props
+- `generateTextResponse()` - Generate text with tool calling support
+- `matchComponentsFromTextResponse()` - Match components from text suggestions
+- `generateNextQuestions()` - Generate follow-up question suggestions
+
+### 8. Prompt Loader System
+
+The SDK includes a sophisticated prompt loading system with caching and template variable support:
+
+```typescript
+import { promptLoader } from '@superatomai/sdk-node/userResponse';
+
+// Initialize with custom directory (default: .prompts)
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  promptsDir: './my-custom-prompts',
+});
+
+// Prompts are automatically loaded and cached on initialization
+// Access prompt cache size
+const cacheSize = promptLoader.getCacheSize();
+console.log(`Loaded ${cacheSize} prompts`);
+
+// Load specific prompts with variables
+const prompts = await promptLoader.loadPrompts('classify', {
+  USER_PROMPT: userQuestion,
+  CONVERSATION_HISTORY: history || 'No previous conversation'
+});
+
+console.log('System prompt:', prompts.system);
+console.log('User prompt:', prompts.user);
+```
+
+**Prompt Directory Structure:**
+```
+.prompts/
+  ├── classify/
+  │   ├── system.md     # System prompt for classification
+  │   └── user.md       # User prompt template
+  ├── match-component/
+  │   ├── system.md
+  │   └── user.md
+  ├── modify-props/
+  │   ├── system.md
+  │   └── user.md
+  ├── text-response/
+  │   ├── system.md
+  │   └── user.md
+  └── match-text-components/
+      ├── system.md
+      └── user.md
+```
+
+**Template Variables:**
+Variables in prompts are replaced using the `{{VARIABLE_NAME}}` syntax:
+
+```markdown
+# system.md
+You are analyzing this question: {{USER_PROMPT}}
+
+Previous conversation:
+{{CONVERSATION_HISTORY}}
+
+Available components:
+{{AVAILABLE_COMPONENTS}}
+```
+
+**Built-in Prompt Types:**
+- `classify` - Question classification and visualization type detection
+- `match-component` - Component matching and selection
+- `modify-props` - Props validation and modification
+- `single-component` - Single analytical component generation
+- `text-response` - Text response with tool calling
+- `match-text-components` - Component matching from text suggestions
+- `container-metadata` - Container title and description generation
+- `actions` - Next question generation
+
+### 9. Thread & UI Block Management
+
+Organize conversations and UI components:
+
+```typescript
+import { Thread, UIBlock, ThreadManager } from '@superatomai/sdk-node';
+
+const threadManager = ThreadManager.getInstance();
+
+// Create a thread
+const thread = threadManager.createThread({
+  userId: 'user-123',
+  metadata: { source: 'chat' },
+});
+
+// Add UI blocks
+const block1 = new UIBlock(
+  'block-1',
+  'text',
+  { content: 'Hello!' }
+);
+thread.addBlock(block1);
+
+const block2 = new UIBlock(
+  'block-2',
+  'table',
+  {
+    columns: ['Name', 'Age'],
+    data: [['John', 30], ['Jane', 25]]
+  }
+);
+thread.addBlock(block2);
+
+// Get thread
+const retrieved = threadManager.getThread(thread.getId());
+
+// Get all threads
+const allThreads = threadManager.getAllThreads();
+
+// Delete thread
+threadManager.deleteThread(thread.getId());
+```
+
+**UI Block Types:**
+- `text` - Text content
+- `table` - Tabular data
+- `chart` - Charts and visualizations
+- `form` - Interactive forms
+- Custom types as needed
+
+### 10. Logging System
+
+The SDK includes a comprehensive logging system with environment-based log levels for both terminal and UI log collection.
+
+#### Log Levels
+
+The SDK supports hierarchical log levels:
+
+- **`errors`** - Only error logs are shown
+- **`warnings`** - Warning and error logs are shown
+- **`info`** - Info, warning, and error logs are shown (default)
+- **`verbose`** - All logs including debug messages are shown
+
+#### Configuring Log Level
+
+You can set the log level in three ways:
+
+**1. Via environment variable (recommended for deployment):**
+
+```bash
+export SUPERATOM_LOG_LEVEL=verbose
+node your-app.js
+```
+
+**2. Via SDK configuration (overrides environment variable):**
+
+```typescript
+import { SuperatomSDK } from '@superatomai/sdk-node';
+
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  logLevel: 'verbose', // errors | warnings | info | verbose
+});
+```
+
+**3. Programmatically at runtime:**
+
+```typescript
+import { logger } from '@superatomai/sdk-node';
+
+// Change log level at runtime
+logger.setLogLevel('verbose');
+
+// Get current log level
+const currentLevel = logger.getLogLevel(); // returns 'verbose'
+```
+
+#### Using the Logger
+
+```typescript
+import { logger } from '@superatomai/sdk-node';
+
+// These will be filtered based on the configured log level
+logger.error('Critical error occurred');        // Shown at all levels
+logger.warn('Something might be wrong');        // Hidden when level is 'errors'
+logger.info('Processing completed');            // Hidden when level is 'errors' or 'warnings'
+logger.debug('Detailed debug information');     // Only shown when level is 'verbose'
+```
+
+#### UI Log Collection
+
+The `UILogCollector` also respects the log level configuration. Logs are sent to the frontend and displayed in the terminal based on the configured level.
+
+```typescript
+import { UILogCollector } from '@superatomai/sdk-node';
+
+const collector = new UILogCollector(
+  clientId,
+  (msg) => sdk.send(msg),
+  'uiblock-123'
+);
+
+// These logs are filtered based on log level before being sent to UI
+collector.error('Error message');      // Always sent
+collector.warn('Warning message');     // Sent unless level is 'errors'
+collector.info('Info message');        // Sent unless level is 'errors' or 'warnings'
+collector.debug('Debug message');      // Only sent when level is 'verbose'
+
+// Get all collected logs
+const logs = collector.getLogs();
+```
+
+**Note:** The log level affects both terminal output and UI log collection, ensuring consistent logging behavior across your application.
+
+### 11. Cleanup Service
+
+Automatic memory management for threads and UI blocks:
+
+```typescript
+import { CleanupService } from '@superatomai/sdk-node';
+
+const cleanupService = CleanupService.getInstance();
+
+// Start automatic cleanup (runs every 24 hours by default)
+cleanupService.startAutoCleanup(24);
+
+// Manual cleanup
+const stats = cleanupService.runFullCleanup();
+console.log('Cleanup stats:', stats);
+// Output: { threadsDeleted: 5, uiblocksDeleted: {...}, dataCleared: 10 }
+
+// Get memory statistics
+const memStats = cleanupService.getMemoryStats();
+console.log('Memory:', memStats);
+// Output: { threadCount: 42, totalUIBlocks: 156, avgUIBlocksPerThread: 3.7 }
+
+// Stop automatic cleanup
+cleanupService.stopAutoCleanup();
+```
+
+**Configuration:**
+
+```typescript
+import { STORAGE_CONFIG } from '@superatomai/sdk-node';
+
+// Thread retention (default: 30 days)
+STORAGE_CONFIG.THREAD_RETENTION_DAYS = 60;
+
+// UI block retention (default: 7 days)
+STORAGE_CONFIG.UIBLOCK_RETENTION_DAYS = 14;
+
+// Max component data size (default: 100KB)
+STORAGE_CONFIG.MAX_COMPONENT_DATA_SIZE = 200 * 1024;
+```
+
+## API Reference
+
+### SuperatomSDK
+
+#### Methods
+
+- `connect(): Promise<void>` - Connect to WebSocket service
+- `disconnect(): void` - Disconnect from WebSocket
+- `destroy(): Promise<void>` - Cleanup and disconnect permanently
+- `isConnected(): boolean` - Check connection status
+- `send(message: Message): void` - Send a message
+- `onMessage(handler): () => void` - Register message handler (returns unsubscribe function)
+- `onMessageType(type, handler): () => void` - Register type-specific handler
+- `addCollection(name, operation, handler): void` - Register collection handler
+- `getUserManager(): UserManager` - Get user manager instance
+- `getDashboardManager(): DashboardManager` - Get dashboard manager instance
+- `getReportManager(): ReportManager` - Get report manager instance
+
+### UserManager
+
+#### Methods
+
+- `createUser(data): Promise<User>` - Create a new user
+- `getUser(username): Promise<User | null>` - Get user by username
+- `updateUser(username, data): Promise<User>` - Update user
+- `deleteUser(username): Promise<boolean>` - Delete user
+- `getAllUsers(): Promise<User[]>` - Get all users
+- `verifyPassword(username, password): Promise<boolean>` - Verify password
+- `generateToken(username): Promise<string>` - Generate auth token
+- `verifyToken(token): Promise<User | null>` - Verify auth token
+
+### DashboardManager
+
+#### Methods
+
+- `createDashboard(dashboard): Promise<DSLRendererProps>` - Create dashboard
+- `getDashboard(id): Promise<DSLRendererProps | null>` - Get dashboard
+- `updateDashboard(id, updates): Promise<DSLRendererProps>` - Update dashboard
+- `deleteDashboard(id): Promise<boolean>` - Delete dashboard
+- `getAllDashboards(): Promise<DSLRendererProps[]>` - Get all dashboards
+
+### ReportManager
+
+Same methods as DashboardManager but for reports.
+
+### LLM (Static Class)
+
+#### Methods
+
+- `text(messages, options): Promise<string>` - Generate text response
+- `stream(messages, options, json?): Promise<string | object>` - Stream response
+
+### ThreadManager
+
+#### Methods
+
+- `getInstance(): ThreadManager` - Get singleton instance
+- `createThread(options): Thread` - Create new thread
+- `getThread(id): Thread | undefined` - Get thread by ID
+- `getAllThreads(): Thread[]` - Get all threads
+- `deleteThread(id): boolean` - Delete thread
+
+### CleanupService
+
+#### Methods
+
+- `getInstance(): CleanupService` - Get singleton instance
+- `startAutoCleanup(intervalHours?): void` - Start automatic cleanup
+- `stopAutoCleanup(): void` - Stop automatic cleanup
+- `runFullCleanup(): CleanupStats` - Run manual cleanup
+- `getMemoryStats(): MemoryStats` - Get current memory statistics
+
+### UILogCollector
+
+#### Methods
+
+- `start(): void` - Start collecting logs
+- `stop(): void` - Stop collecting
+- `clear(): void` - Clear collected logs
+- `getLogs(): CapturedLog[]` - Get all logs
+- `addLog(log): void` - Add custom log
+- `createMessage(uiBlockId): UILogsMessage` - Create message for SDK
+
+## Advanced Usage
+
+### Custom Message Handlers
+
+```typescript
+// Handle custom message types
+sdk.onMessageType('CUSTOM_EVENT', async (message) => {
+  console.log('Custom event:', message.payload);
+
+  // Process and respond
+  sdk.send({
+    type: 'CUSTOM_RESPONSE',
+    from: { id: 'sdk', type: 'agent' },
+    payload: { result: 'processed' },
+  });
+});
+```
+
+### Error Handling
+
+```typescript
+try {
+  await sdk.connect();
+} catch (error) {
+  console.error('Connection failed:', error);
+}
+
+// Handle WebSocket errors
+sdk.onMessageType('error', (message) => {
+  console.error('Error message:', message.payload);
+});
+```
+
+### Reconnection
+
+The SDK automatically handles reconnection with exponential backoff:
+- Max attempts: 5
+- Delay: 1s, 2s, 4s, 8s, 10s (capped at 10s)
+
+### Bundle Requests
+
+Serve static files via bundle requests:
+
+```typescript
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  bundleDir: './public', // Directory to serve files from
+});
+
+// Files in ./public will be served when requested
+```
+
+## TypeScript Support
+
+This SDK is written in TypeScript and includes full type definitions. All exports are fully typed for the best development experience.
+
+```typescript
+import type {
+  Message,
+  IncomingMessage,
+  SuperatomSDKConfig,
+  CollectionHandler,
+  CollectionOperation,
+  User,
+  UsersData,
+  Component,
+  T_RESPONSE,
+  LLMProvider,
+  LogLevel,
+  CapturedLog,
+  Action,
+} from '@superatomai/sdk-node';
+
+// Import LLM and utility classes
+import {
+  LLM,
+  UserManager,
+  UILogCollector,
+  Thread,
+  UIBlock,
+  ThreadManager,
+  CleanupService,
+  logger,
+} from '@superatomai/sdk-node';
+
+// Import user response utilities
+import {
+  get_user_response,
+  anthropicLLM,
+  groqLLM,
+} from '@superatomai/sdk-node/userResponse';
+```
+
+## Examples
+
+Check out the [examples](./examples) directory for complete examples:
+- Basic WebSocket communication
+- Data collection handlers
+- User authentication flow
+- LLM integration with streaming
+- AI-powered component generation
+- Text response with query execution
+- Component matching and classification
+- Thread management
+- Dashboard creation
+- Prompt customization
+
+## Development
+
+```bash
+# Install dependencies
+pnpm install
+
+# Build the SDK
+pnpm run build
+
+# Watch mode
+pnpm run dev
+
+# Run tests
+pnpm test
+```
+
+## License
+
+MIT
+
+## Support
+
+For issues and questions:
+- **GitHub Issues:** [Report an issue](https://github.com/superatomai/sdk-nodejs/issues)
+- **Email:** ashish@superatom.ai
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.