@superatomai/sdk-node 0.0.2 → 0.0.4

package/README.md

@@ -1,23 +1,36 @@
-# @superatomai/sdk
+# @superatomai/sdk-node
 
-A TypeScript SDK for building applications with Superatom - a platform for creating data-driven, AI-powered applications.
+A TypeScript/Node.js SDK for building AI-powered data applications with Superatom - a platform for creating data-driven, intelligent applications with real-time WebSocket communication, LLM integration, and comprehensive user management.
+
+## Features
+
+- **WebSocket Communication** - Real-time bidirectional messaging with automatic reconnection
+- **Collection Handlers** - Register custom data operation handlers (CRUD, queries, mutations)
+- **User Management** - Built-in authentication and user storage with file-based persistence
+- **Dashboard & Report Management** - Create and manage dashboards and reports with DSL-based rendering
+- **LLM Integration** - Unified interface for Anthropic Claude and Groq models with streaming support
+- **Thread & UI Block Management** - Organize conversations and UI components with automatic cleanup
+- **Log Collection** - Capture and send runtime logs to the UI
+- **Prompt Loader** - Load custom prompts from the file system
+- **Cleanup Service** - Automatic memory management and old data removal
+- **TypeScript First** - Full type safety with comprehensive type definitions
 
 ## Installation
 
 ```bash
-npm install @superatomai/sdk
+npm install @superatomai/sdk-node
 ```
 
 Or with pnpm:
 
 ```bash
-pnpm add @superatomai/sdk
+pnpm add @superatomai/sdk-node
 ```
 
 ## Quick Start
 
 ```typescript
-import { SuperatomSDK } from '@superatomai/sdk';
+import { SuperatomSDK } from '@superatomai/sdk-node';
 
 // Initialize the SDK
 const sdk = new SuperatomSDK({
@@ -32,25 +45,18 @@ const sdk = new SuperatomSDK({
 await sdk.connect();
 
 // Register a collection handler for data operations
-sdk.addCollection('users', 'list', async (params) => {
-  // Your custom logic to fetch users
+sdk.addCollection('users', 'getMany', async (params) => {
   return {
     data: [
-      { id: 1, name: 'John Doe' },
-      { id: 2, name: 'Jane Smith' },
+      { id: 1, name: 'John Doe', email: 'john@example.com' },
+      { id: 2, name: 'Jane Smith', email: 'jane@example.com' },
     ],
   };
 });
 
-// Listen to all incoming messages
+// Listen to messages
 sdk.onMessage((message) => {
-  console.log('Received message:', message);
-});
-
-// Send messages to the Superatom service
-sdk.send({
-  type: 'custom_message',
-  data: { foo: 'bar' },
+  console.log('Received:', message.type);
 });
 
 // Cleanup when done
@@ -67,104 +73,270 @@ interface SuperatomSDKConfig {
   projectId: string;                 // Required: Your project ID
   userId?: string;                   // Optional: User identifier (default: 'anonymous')
   type?: string;                     // Optional: Agent type (default: 'data-agent')
-  bundleDir?: string;                // Optional: Directory for bundle requests
   url?: string;                      // Optional: Custom WebSocket URL
+  bundleDir?: string;                // Optional: Directory for bundle requests
+  promptsDir?: string;               // Optional: Custom prompts directory (default: .prompts)
   ANTHROPIC_API_KEY?: string;        // Optional: Anthropic API key for LLM
   GROQ_API_KEY?: string;             // Optional: Groq API key for LLM
   LLM_PROVIDERS?: LLMProvider[];     // Optional: Custom LLM providers
 }
 ```
 
-Environment variables are also supported:
-- `SA_WEBSOCKET_URL`: WebSocket URL (default: `wss://ws.superatom.ai/websocket`)
-- `ANTHROPIC_API_KEY`: Anthropic API key
-- `GROQ_API_KEY`: Groq API key
+### Environment Variables
+
+- `SA_WEBSOCKET_URL` - WebSocket URL (default: `wss://ws.superatom.ai/websocket`)
+- `ANTHROPIC_API_KEY` - Anthropic API key for Claude models
+- `GROQ_API_KEY` - Groq API key for open-source models
 
-## Key Features
+## Core Features
 
-### Collection Handlers
+### 1. Collection Handlers
 
-Register handlers for data operations:
+Register custom handlers for data operations. Supports standard CRUD operations and custom operations.
 
 ```typescript
+// Register a getMany handler
 sdk.addCollection<ParamsType, ResultType>(
-  'collection-name',
-  'operation',
+  'products',
+  'getMany',
   async (params) => {
-    // Your custom logic
-    return result;
+    // params includes filters, pagination, etc.
+    const products = await database.products.find(params);
+    return { data: products };
   }
 );
+
+// Register a createOne handler
+sdk.addCollection('products', 'createOne', async (params) => {
+  const newProduct = await database.products.create(params);
+  return { data: newProduct };
+});
+
+// Register a custom query handler
+sdk.addCollection('analytics', 'query', async (params) => {
+  const results = await runAnalyticsQuery(params);
+  return { data: results };
+});
 ```
 
-### Message Handling
+**Supported Operations:**
+- `getMany` - Fetch multiple records
+- `getOne` - Fetch single record
+- `createOne` - Create a record
+- `updateOne` - Update a record
+- `deleteOne` - Delete a record
+- `query` - Custom queries
+- `mutation` - Custom mutations
+
+### 2. Message Handling
 
-Listen to all messages:
+Listen to all messages or specific message types:
 
 ```typescript
+// Listen to all messages
 const unsubscribe = sdk.onMessage((message) => {
-  console.log('Message received:', message);
+  console.log('Message:', message.type, message.payload);
+});
+
+// Listen to specific message types
+sdk.onMessageType('DATA_REQ', (message) => {
+  console.log('Data request received:', message.payload);
 });
 
 // Unsubscribe when needed
 unsubscribe();
-```
-
-Listen to specific message types:
 
-```typescript
-const unsubscribe = sdk.onMessageType('custom_type', (message) => {
-  console.log('Custom message:', message);
+// Send messages
+sdk.send({
+  type: 'CUSTOM_MESSAGE',
+  from: { id: 'sdk', type: 'agent' },
+  payload: { data: 'Hello' },
 });
 ```
 
-### User Management
+**Built-in Message Types:**
+- `DATA_REQ` - Data request from runtime
+- `BUNDLE_REQ` - Bundle/asset request
+- `AUTH_LOGIN_REQ` - Authentication login request
+- `AUTH_VERIFY_REQ` - Token verification request
+- `USER_PROMPT_REQ` - User prompt for LLM processing
+- `USER_PROMPT_SUGGESTIONS_REQ` - Request for prompt suggestions
+- `ACTIONS` - Component actions request
+- `COMPONENT_LIST_RES` - Available components list
+- `USERS` - User management operations
+- `DASHBOARDS` - Dashboard CRUD operations
+- `REPORTS` - Report CRUD operations
+
+### 3. User Management
+
+Built-in user authentication and management with file-based storage:
 
 ```typescript
 const userManager = sdk.getUserManager();
 
 // Create a user
 await userManager.createUser({
-  id: 'user-123',
-  name: 'John Doe',
-  email: 'john@example.com',
+  username: 'john_doe',
+  password: 'secure_password',
 });
 
 // Get a user
-const user = await userManager.getUser('user-123');
+const user = await userManager.getUser('john_doe');
 
 // Update a user
-await userManager.updateUser('user-123', { name: 'John Updated' });
+await userManager.updateUser('john_doe', {
+  password: 'new_password',
+});
 
 // Delete a user
-await userManager.deleteUser('user-123');
+await userManager.deleteUser('john_doe');
+
+// Get all users
+const users = await userManager.getAllUsers();
 ```
 
-### LLM Integration
+**Features:**
+- Automatic password hashing with bcrypt
+- File-based storage (JSON)
+- Token-based authentication
+- Auto-save on changes
+- Memory management with configurable limits
+
+### 4. Dashboard Management
+
+Create and manage dashboards with DSL-based rendering:
 
 ```typescript
-import { LLM } from '@superatomai/sdk';
+const dashboardManager = sdk.getDashboardManager();
+
+// Create a dashboard
+const dashboard = {
+  id: 'dashboard-1',
+  title: 'Sales Dashboard',
+  description: 'Overview of sales metrics',
+  dsl: {
+    type: 'container',
+    children: [
+      { type: 'chart', props: { chartType: 'line', data: [...] } },
+      { type: 'table', props: { columns: [...], data: [...] } },
+    ],
+  },
+};
+
+await dashboardManager.createDashboard(dashboard);
 
-const llm = new LLM({
-  apiKey: 'your-anthropic-or-groq-api-key',
-  provider: 'anthropic', // or 'groq'
+// Get a dashboard
+const retrieved = await dashboardManager.getDashboard('dashboard-1');
+
+// Update a dashboard
+await dashboardManager.updateDashboard('dashboard-1', {
+  title: 'Updated Sales Dashboard',
 });
 
-// Generate text
-const response = await llm.text('What is the meaning of life?');
+// Delete a dashboard
+await dashboardManager.deleteDashboard('dashboard-1');
 
-// Stream responses
-for await (const chunk of llm.stream('Tell me a story')) {
-  process.stdout.write(chunk);
-}
+// Get all dashboards
+const allDashboards = await dashboardManager.getAllDashboards();
+```
+
+### 5. Report Management
+
+Similar to dashboards, but for reports:
+
+```typescript
+const reportManager = sdk.getReportManager();
+
+// Create a report
+const report = {
+  id: 'report-1',
+  title: 'Monthly Report',
+  description: 'Monthly performance report',
+  dsl: {
+    type: 'report',
+    sections: [
+      { type: 'summary', content: '...' },
+      { type: 'charts', data: [...] },
+    ],
+  },
+};
+
+await reportManager.createReport(report);
+
+// CRUD operations work the same as dashboards
+const retrieved = await reportManager.getReport('report-1');
+await reportManager.updateReport('report-1', { title: 'Q1 Report' });
+await reportManager.deleteReport('report-1');
+const allReports = await reportManager.getAllReports();
+```
+
+### 6. LLM Integration
+
+Unified interface for multiple LLM providers with streaming support:
+
+```typescript
+import { LLM } from '@superatomai/sdk-node';
+
+// Text generation
+const response = await LLM.text(
+  {
+    sys: 'You are a helpful assistant.',
+    user: 'What is the meaning of life?',
+  },
+  {
+    model: 'anthropic/claude-sonnet-4-5', // or 'groq/llama3-70b'
+    apiKey: 'your-api-key',
+    maxTokens: 1000,
+    temperature: 0.7,
+  }
+);
+
+// Streaming responses
+const result = await LLM.stream(
+  {
+    sys: 'You are a storyteller.',
+    user: 'Tell me a short story.',
+  },
+  {
+    model: 'anthropic/claude-sonnet-4-5',
+    apiKey: 'your-api-key',
+    partial: (chunk) => {
+      // Called for each chunk
+      process.stdout.write(chunk);
+    },
+  }
+);
+
+// JSON output
+const jsonResult = await LLM.stream(
+  {
+    sys: 'Return a JSON object with user data.',
+    user: 'Create a user profile for John Doe.',
+  },
+  {
+    model: 'groq/llama3-70b',
+    apiKey: 'your-api-key',
+  },
+  true // Enable JSON parsing
+);
 ```
 
-### Thread Management
+**Supported Models:**
+- **Anthropic:** `claude-sonnet-4-5`, `claude-opus-4`, etc.
+- **Groq:** `llama3-70b`, `mixtral-8x7b`, etc.
+
+**Model Format:**
+- With provider: `anthropic/model-name` or `groq/model-name`
+- Without provider: Defaults to Anthropic
+
+### 7. Thread & UI Block Management
+
+Organize conversations and UI components:
 
 ```typescript
-import { Thread, UIBlock, ThreadManager } from '@superatomai/sdk';
+import { Thread, UIBlock, ThreadManager } from '@superatomai/sdk-node';
 
-const threadManager = new ThreadManager();
+const threadManager = ThreadManager.getInstance();
 
 // Create a thread
 const thread = threadManager.createThread({
@@ -172,55 +344,322 @@ const thread = threadManager.createThread({
   metadata: { source: 'chat' },
 });
 
-// Add UI blocks to thread
-thread.addBlock(
-  new UIBlock('block-1', 'text', { content: 'Hello, World!' })
+// Add UI blocks
+const block1 = new UIBlock(
+  'block-1',
+  'text',
+  { content: 'Hello!' }
 );
+thread.addBlock(block1);
+
+const block2 = new UIBlock(
+  'block-2',
+  'table',
+  {
+    columns: ['Name', 'Age'],
+    data: [['John', 30], ['Jane', 25]]
+  }
+);
+thread.addBlock(block2);
+
+// Get thread
+const retrieved = threadManager.getThread(thread.getId());
 
 // Get all threads
-const threads = threadManager.getAllThreads();
+const allThreads = threadManager.getAllThreads();
 
-// Cleanup old threads
-await threadManager.cleanup();
+// Delete thread
+threadManager.deleteThread(thread.getId());
 ```
 
-### Cleanup Service
+**UI Block Types:**
+- `text` - Text content
+- `table` - Tabular data
+- `chart` - Charts and visualizations
+- `form` - Interactive forms
+- Custom types as needed
+
+### 8. Log Collection
+
+Capture and send logs to the UI:
 
 ```typescript
-import { CleanupService } from '@superatomai/sdk';
+import { UILogCollector } from '@superatomai/sdk-node';
 
-const cleanupService = new CleanupService();
+const collector = new UILogCollector();
 
-// Register cleanup tasks
-cleanupService.registerTask('threads', async () => {
-  // Your cleanup logic
+// Start collecting logs
+collector.start();
+
+// Logs are automatically captured
+console.log('This will be captured');
+console.error('Error messages too');
+
+// Add custom logs
+collector.addLog({
+  level: 'info',
+  message: 'Processing data...',
+  type: 'explanation',
+  data: { progress: 50 },
 });
 
-// Start periodic cleanup (every 5 minutes by default)
-cleanupService.start();
+// Get captured logs
+const logs = collector.getLogs();
+
+// Send logs to UI (via SDK)
+const message = collector.createMessage('uiblock-123');
+sdk.send(message);
 
-// Stop cleanup
-cleanupService.stop();
+// Stop collecting
+collector.stop();
+
+// Clear logs
+collector.clear();
+```
+
+### 9. Cleanup Service
+
+Automatic memory management for threads and UI blocks:
+
+```typescript
+import { CleanupService } from '@superatomai/sdk-node';
+
+const cleanupService = CleanupService.getInstance();
+
+// Start automatic cleanup (runs every 24 hours by default)
+cleanupService.startAutoCleanup(24);
+
+// Manual cleanup
+const stats = cleanupService.runFullCleanup();
+console.log('Cleanup stats:', stats);
+// Output: { threadsDeleted: 5, uiblocksDeleted: {...}, dataCleared: 10 }
+
+// Get memory statistics
+const memStats = cleanupService.getMemoryStats();
+console.log('Memory:', memStats);
+// Output: { threadCount: 42, totalUIBlocks: 156, avgUIBlocksPerThread: 3.7 }
+
+// Stop automatic cleanup
+cleanupService.stopAutoCleanup();
+```
+
+**Configuration:**
+
+```typescript
+import { STORAGE_CONFIG } from '@superatomai/sdk-node';
+
+// Thread retention (default: 30 days)
+STORAGE_CONFIG.THREAD_RETENTION_DAYS = 60;
+
+// UI block retention (default: 7 days)
+STORAGE_CONFIG.UIBLOCK_RETENTION_DAYS = 14;
+
+// Max component data size (default: 100KB)
+STORAGE_CONFIG.MAX_COMPONENT_DATA_SIZE = 200 * 1024;
+```
+
+### 10. Prompt Loader
+
+Load custom prompts from the file system:
+
+```typescript
+// Prompts are automatically loaded from .prompts/ directory
+// Or specify custom directory in config:
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  promptsDir: './my-prompts',
+});
+
+// Prompts are loaded and cached on initialization
+// They can be accessed by handlers internally
+```
+
+**Prompt File Structure:**
+```
+.prompts/
+  ├── system.txt          # System prompts
+  ├── user-greeting.txt   # User greeting prompts
+  └── analysis.txt        # Analysis prompts
 ```
 
 ## API Reference
 
-### SuperatomSDK Methods
+### SuperatomSDK
 
-- `connect(): Promise<void>` - Connect to the WebSocket service
-- `disconnect(): void` - Disconnect from the WebSocket service
+#### Methods
+
+- `connect(): Promise<void>` - Connect to WebSocket service
+- `disconnect(): void` - Disconnect from WebSocket
 - `destroy(): Promise<void>` - Cleanup and disconnect permanently
 - `isConnected(): boolean` - Check connection status
-- `send(message: Message): void` - Send a message to Superatom
-- `onMessage(handler): () => void` - Register a message handler
-- `onMessageType(type, handler): () => void` - Register a type-specific handler
-- `addCollection(name, operation, handler): void` - Register a collection handler
-- `getUserManager(): UserManager` - Get the user manager instance
+- `send(message: Message): void` - Send a message
+- `onMessage(handler): () => void` - Register message handler (returns unsubscribe function)
+- `onMessageType(type, handler): () => void` - Register type-specific handler
+- `addCollection(name, operation, handler): void` - Register collection handler
+- `getUserManager(): UserManager` - Get user manager instance
+- `getDashboardManager(): DashboardManager` - Get dashboard manager instance
+- `getReportManager(): ReportManager` - Get report manager instance
+
+### UserManager
+
+#### Methods
+
+- `createUser(data): Promise<User>` - Create a new user
+- `getUser(username): Promise<User | null>` - Get user by username
+- `updateUser(username, data): Promise<User>` - Update user
+- `deleteUser(username): Promise<boolean>` - Delete user
+- `getAllUsers(): Promise<User[]>` - Get all users
+- `verifyPassword(username, password): Promise<boolean>` - Verify password
+- `generateToken(username): Promise<string>` - Generate auth token
+- `verifyToken(token): Promise<User | null>` - Verify auth token
+
+### DashboardManager
+
+#### Methods
+
+- `createDashboard(dashboard): Promise<DSLRendererProps>` - Create dashboard
+- `getDashboard(id): Promise<DSLRendererProps | null>` - Get dashboard
+- `updateDashboard(id, updates): Promise<DSLRendererProps>` - Update dashboard
+- `deleteDashboard(id): Promise<boolean>` - Delete dashboard
+- `getAllDashboards(): Promise<DSLRendererProps[]>` - Get all dashboards
+
+### ReportManager
+
+Same methods as DashboardManager but for reports.
+
+### LLM (Static Class)
+
+#### Methods
+
+- `text(messages, options): Promise<string>` - Generate text response
+- `stream(messages, options, json?): Promise<string | object>` - Stream response
+
+### ThreadManager
+
+#### Methods
+
+- `getInstance(): ThreadManager` - Get singleton instance
+- `createThread(options): Thread` - Create new thread
+- `getThread(id): Thread | undefined` - Get thread by ID
+- `getAllThreads(): Thread[]` - Get all threads
+- `deleteThread(id): boolean` - Delete thread
+
+### CleanupService
+
+#### Methods
+
+- `getInstance(): CleanupService` - Get singleton instance
+- `startAutoCleanup(intervalHours?): void` - Start automatic cleanup
+- `stopAutoCleanup(): void` - Stop automatic cleanup
+- `runFullCleanup(): CleanupStats` - Run manual cleanup
+- `getMemoryStats(): MemoryStats` - Get current memory statistics
+
+### UILogCollector
+
+#### Methods
+
+- `start(): void` - Start collecting logs
+- `stop(): void` - Stop collecting
+- `clear(): void` - Clear collected logs
+- `getLogs(): CapturedLog[]` - Get all logs
+- `addLog(log): void` - Add custom log
+- `createMessage(uiBlockId): UILogsMessage` - Create message for SDK
+
+## Advanced Usage
+
+### Custom Message Handlers
+
+```typescript
+// Handle custom message types
+sdk.onMessageType('CUSTOM_EVENT', async (message) => {
+  console.log('Custom event:', message.payload);
+
+  // Process and respond
+  sdk.send({
+    type: 'CUSTOM_RESPONSE',
+    from: { id: 'sdk', type: 'agent' },
+    payload: { result: 'processed' },
+  });
+});
+```
+
+### Error Handling
+
+```typescript
+try {
+  await sdk.connect();
+} catch (error) {
+  console.error('Connection failed:', error);
+}
+
+// Handle WebSocket errors
+sdk.onMessageType('error', (message) => {
+  console.error('Error message:', message.payload);
+});
+```
+
+### Reconnection
+
+The SDK automatically handles reconnection with exponential backoff:
+- Max attempts: 5
+- Delay: 1s, 2s, 4s, 8s, 10s (capped at 10s)
+
+### Bundle Requests
+
+Serve static files via bundle requests:
+
+```typescript
+const sdk = new SuperatomSDK({
+  apiKey: 'your-api-key',
+  projectId: 'your-project-id',
+  bundleDir: './public', // Directory to serve files from
+});
+
+// Files in ./public will be served when requested
+```
 
 ## TypeScript Support
 
 This SDK is written in TypeScript and includes full type definitions. All exports are fully typed for the best development experience.
 
+```typescript
+import type {
+  Message,
+  IncomingMessage,
+  SuperatomSDKConfig,
+  CollectionHandler,
+  User,
+  DSLRendererProps,
+} from '@superatomai/sdk-node';
+```
+
+## Examples
+
+Check out the [examples](./examples) directory for complete examples:
+- Basic WebSocket communication
+- Data collection handlers
+- User authentication flow
+- LLM integration
+- Thread management
+- Dashboard creation
+
+## Development
+
+```bash
+# Install dependencies
+pnpm install
+
+# Build the SDK
+pnpm run build
+
+# Watch mode
+pnpm run dev
+
+# Run tests
+pnpm test
+```
+
 ## License
 
 MIT
@@ -228,5 +667,9 @@ MIT
 ## Support
 
 For issues and questions:
-- GitHub Issues: [Report an issue](https://github.com/superatomai/sdk-nodejs/issues)
-- Email: ashish@superatom.ai
+- **GitHub Issues:** [Report an issue](https://github.com/superatomai/sdk-nodejs/issues)
+- **Email:** ashish@superatom.ai
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.