@minded-ai/mindedjs 1.0.0-ec2-beta-1

package/docs/sdk/agent-api.md

@@ -0,0 +1,540 @@
+# Agent API
+
+The Agent class is the core component of MindedJS that orchestrates conversations, manages state, and coordinates flows. This document covers the initialization options and key methods available on the Agent instance.
+
+## Constructor
+
+The Agent constructor accepts configuration options to customize its behavior:
+
+```typescript
+new Agent({
+  memorySchema: z.ZodSchema,
+  config: MindedSDKConfig,
+  tools: Tool[],
+  platformToken?: string,
+  memorySaver?: BaseCheckpointSaver,
+  interruptSessionManager?: InterruptSessionManager,
+  parseSessionIdFromTrigger?: (trigger: any) => string,
+});
+```
+
+### Options
+
+- **memorySchema** (required): Zod schema defining the agent's memory structure
+- **config** (required): SDK configuration including flows and LLM settings
+- **tools** (required): Array of custom tools available to the agent
+- **platformToken**: Optional token for platform authentication
+- **memorySaver**: Optional custom checkpoint saver for conversation memory
+- **interruptSessionManager**: Optional custom interrupt session manager
+- **parseSessionIdFromTrigger**: Optional function to extract session ID from triggers (defaults to `triggerBody.sessionId`)
+
+## Key Methods
+
+### updateState
+
+Updates the state of an active session. This method allows you to programmatically modify the conversation state, including messages, memory, and other state properties.
+
+#### Signature
+
+```typescript
+async updateState({
+  sessionId: string,
+  state: Partial<State>
+}): Promise<void>
+```
+
+#### Parameters
+
+- `sessionId` (string): The unique identifier of the session to update
+- `state` (Partial<State>): An object containing the state properties to update. Can include:
+  - `messages`: Array of messages to add or update
+  - `memory`: Partial memory object to merge with existing memory
+  - `goto`: Node ID to override the next execution point
+  - Any other state properties defined in your agent
+
+#### Usage Examples
+
+##### Update Memory
+
+```typescript
+// Update specific memory fields
+await agent.updateState({
+  sessionId: 'session-123',
+  state: {
+    memory: {
+      user: {
+        name: 'John Doe',
+        preferences: {
+          language: 'en',
+        },
+      },
+    },
+  },
+});
+```
+
+##### Add Messages
+
+```typescript
+import { HumanMessage, AIMessage } from '@langchain/core/messages';
+
+// Add a new message to the conversation
+await agent.updateState({
+  sessionId: 'session-123',
+  state: {
+    messages: [new HumanMessage('User input'), new AIMessage('Agent response')],
+  },
+});
+```
+
+##### Update Multiple State Properties
+
+```typescript
+// Update both memory and messages
+await agent.updateState({
+  sessionId: 'session-123',
+  state: {
+    memory: {
+      conversation: {
+        topic: 'Order inquiry',
+        urgency: 'high',
+      },
+    },
+    messages: [new HumanMessage('I need help with my order')],
+  },
+});
+```
+
+#### Common Use Cases
+
+1. **Tool State Updates**: Tools can update state after execution
+
+   ```typescript
+   // Inside a tool's execute method
+   async execute({ input, state, agent }) {
+     // Perform tool logic
+     const result = await processOrder(input.orderId);
+
+     // Update state with results
+     await agent.updateState({
+       sessionId: state.sessionId,
+       state: {
+         memory: {
+           order: {
+             id: result.orderId,
+             status: result.status
+           }
+         }
+       }
+     });
+
+     return { result };
+   }
+   ```
+
+2. **External Event Handling**: Update state based on external events
+
+   ```typescript
+   // Handle webhook or external trigger
+   async handleWebhook(sessionId: string, webhookData: any) {
+     await agent.updateState({
+       sessionId,
+       state: {
+         memory: {
+           externalData: webhookData,
+           lastWebhookTime: new Date().toISOString()
+         }
+       }
+     });
+   }
+   ```
+
+3. **Voice Session Corrections**: Update messages during voice conversations
+   ```typescript
+   // Correct a voice transcription
+   await agent.updateState({
+     sessionId: voiceSessionId,
+     state: {
+       messages: [correctedMessage],
+       goto: currentNodeId,
+     },
+   });
+   ```
+
+### getState
+
+Retrieves the current state of a session.
+
+#### Signature
+
+```typescript
+async getState(sessionId: string): Promise<State<Memory>>
+```
+
+#### Parameters
+
+- `sessionId` (string): The unique identifier of the session
+
+#### Returns
+
+A `State` object containing:
+
+- `sessionId`: The session identifier
+- `memory`: The current memory state
+- `messages`: Array of conversation messages
+- `sessionType`: The type of session (TEXT, VOICE, etc.)
+- Other state properties
+
+#### Usage Example
+
+```typescript
+// Get current state
+const state = await agent.getState('session-123');
+
+// Access memory
+console.log('User name:', state.memory.user?.name);
+
+// Check conversation history
+console.log('Message count:', state.messages.length);
+
+// Get session type
+console.log('Session type:', state.sessionType);
+```
+
+### compiledGraph
+
+Access to the underlying LangGraph compiled graph for advanced operations.
+
+#### Usage Example
+
+```typescript
+// Get LangGraph state directly
+const graphState = await agent.compiledGraph.getState(agent.getLangraphConfig(sessionId));
+
+// Update state with LangGraph API
+await agent.compiledGraph.updateState(agent.getLangraphConfig(sessionId), {
+  messages: [newMessage],
+  goto: 'specific-node',
+});
+```
+
+## State Management Best Practices
+
+### 1. Partial Updates
+
+Always use partial updates to avoid overwriting existing state:
+
+```typescript
+// ✅ Good - Partial update
+await agent.updateState({
+  sessionId,
+  state: {
+    memory: {
+      user: { name: 'Jane' },
+    },
+  },
+});
+
+// ❌ Avoid - Don't replace entire memory
+await agent.updateState({
+  sessionId,
+  state: {
+    memory: completeMemoryObject, // This might lose other fields
+  },
+});
+```
+
+### 2. Atomic Operations
+
+Group related updates together:
+
+```typescript
+// ✅ Good - Single atomic update
+await agent.updateState({
+  sessionId,
+  state: {
+    memory: {
+      order: { id: '123', status: 'confirmed' },
+      lastAction: 'order_confirmed',
+    },
+    messages: [confirmationMessage],
+  },
+});
+
+// ❌ Avoid - Multiple separate updates
+await agent.updateState({ sessionId, state: { memory: { order: { id: '123' } } } });
+await agent.updateState({ sessionId, state: { memory: { order: { status: 'confirmed' } } } });
+await agent.updateState({ sessionId, state: { messages: [confirmationMessage] } });
+```
+
+### 3. State Validation
+
+The state updates are validated against your memory schema:
+
+```typescript
+try {
+  await agent.updateState({
+    sessionId,
+    state: {
+      memory: {
+        user: {
+          email: 'invalid-email', // Will fail validation if email has format validation
+        },
+      },
+    },
+  });
+} catch (err) {
+  logger.error({ message: 'State validation failed', err });
+}
+```
+
+### 4. Tool State Updates
+
+Tools should return state updates rather than calling `updateState` directly when possible:
+
+```typescript
+// ✅ Preferred - Return state updates
+async execute({ input, state }) {
+  const result = await processData(input);
+  return {
+    result,
+    state: {
+      memory: {
+        processedData: result.data
+      }
+    }
+  };
+}
+
+// ⚠️ Use only when necessary - Direct update
+async execute({ input, state, agent }) {
+  const result = await processData(input);
+  await agent.updateState({
+    sessionId: state.sessionId,
+    state: {
+      memory: { processedData: result.data }
+    }
+  });
+  return { result };
+}
+```
+
+## Tool Execution API
+
+### ToolExecutor
+
+The `ToolExecutor` class enables standalone tool execution, particularly useful for browser automation and external tool invocations.
+
+#### Setting Up Tool Execution
+
+```typescript
+import { ToolExecutor } from '@minded-ai/mindedjs';
+
+// Create a tool executor
+const toolExecutor = new ToolExecutor(agent);
+
+// Register tools that can be executed via requests
+// Only tools with allowExecutionRequests: true will be registered
+toolExecutor.registerTools(tools);
+```
+
+#### Tool Configuration for Execution
+
+Tools must explicitly opt-in to be executable via requests:
+
+```typescript
+import { Tool } from '@minded-ai/mindedjs';
+
+const myBrowserTool: Tool<typeof schema, Memory> = {
+  name: 'browserAction',
+  description: 'Performs browser automation',
+  input: schema,
+  allowExecutionRequests: true, // Required for ToolExecutor
+  execute: async ({ input, state, agent }) => {
+    // Access CDP URL if available (browser-use context)
+    if (state.cdp?.url) {
+      // Connect to browser using Playwright or similar
+      const browser = await chromium.connectOverCDP(state.cdp.url);
+      const page = browser.contexts()[0].pages()[0];
+
+      // Perform browser automation
+      await page.goto(input.url);
+      const title = await page.title();
+
+      return {
+        result: { pageTitle: title },
+        state: {
+          memory: {
+            lastVisitedUrl: input.url,
+          },
+        },
+      };
+    }
+
+    // Fallback for non-browser context
+    return { result: 'Browser context not available' };
+  },
+};
+```
+
+#### Executing Tools
+
+```typescript
+const response = await toolExecutor.executeTool({
+  toolName: 'browserAction',
+  toolParams: { url: 'https://example.com' },
+  sessionId: 'session-123',
+  requestId: 'req-456',
+  cdpUrl: 'ws://localhost:9222/devtools/browser/...', // Optional, for browser-use
+});
+
+if (response.error) {
+  logger.error({ message: 'Tool execution failed', error: response.error });
+} else {
+  logger.info({ message: 'Tool result', result: response.result });
+}
+```
+
+### CDP URL and Browser Automation
+
+When tools are executed in a browser-use context, the Chrome DevTools Protocol (CDP) URL is available in the state:
+
+#### Accessing CDP URL in Tools
+
+```typescript
+const browserTool: Tool<typeof schema, Memory> = {
+  name: 'advancedBrowserAction',
+  description: 'Advanced browser automation with direct CDP access',
+  input: schema,
+  allowExecutionRequests: true,
+  execute: async ({ input, state, agent }) => {
+    const cdp = state.cdp;
+
+    if (!cdp?.url) {
+      return {
+        result: 'This tool requires browser context',
+        error: 'CDP URL not available',
+      };
+    }
+
+    // Use Playwright with CDP
+    const browser = await playwright.chromium.connectOverCDP(cdp.url);
+    const context = browser.contexts()[0];
+    const page = context.pages()[0];
+
+    // Perform complex browser automation
+    await page.evaluate(() => {
+      // Execute JavaScript in the browser
+      document.querySelector('#submit')?.click();
+    });
+
+    // Take screenshot
+    const screenshot = await page.screenshot();
+
+    return {
+      result: {
+        success: true,
+        screenshot: screenshot.toString('base64'),
+      },
+      state: {
+        memory: {
+          lastAction: 'browser_automation',
+          timestamp: new Date().toISOString(),
+        },
+      },
+    };
+  },
+};
+```
+
+#### Browser-Use Integration Flow
+
+1. Browser-use requests tool execution via socket
+2. ToolExecutor receives request with CDP URL
+3. CDP info is injected into state (`state.cdp`)
+4. Tool connects to browser using CDP URL from `state.cdp.url`
+5. Tool performs automation and returns results
+6. State updates are automatically applied
+
+#### Security Considerations
+
+- Tools must have `allowExecutionRequests: true` to be executable via ToolExecutor
+- CDP URLs are only available during browser-use execution context
+- Tools should validate CDP URL availability before attempting browser operations
+- Always handle connection failures gracefully
+
+## Integration with Other Features
+
+### With Browser Use Tools
+
+When using browser automation tools, state updates happen automatically through the ToolExecutor:
+
+```typescript
+// Browser tools can update state via the ToolExecutor
+const toolExecutor = new ToolExecutor(agent);
+toolExecutor.registerTools(browserTools);
+
+// State updates are handled internally when tools return state changes
+// The CDP info is automatically injected when available in state.cdp
+```
+
+### With Voice Sessions
+
+Voice sessions use `updateState` for real-time corrections:
+
+```typescript
+// Voice session correction flow
+const voiceSession = new VoiceSession(agent, sessionId);
+// Corrections use updateState internally
+```
+
+### With Events
+
+State can be updated in event handlers:
+
+```typescript
+agent.on(AgentEvents.TRIGGER_EVENT, async ({ triggerName, sessionId }) => {
+  if (triggerName === 'external_update') {
+    await agent.updateState({
+      sessionId,
+      state: {
+        memory: {
+          lastTrigger: triggerName,
+          triggerTime: new Date().toISOString(),
+        },
+      },
+    });
+  }
+});
+```
+
+## Error Handling
+
+Always handle potential errors when updating state:
+
+```typescript
+try {
+  await agent.updateState({
+    sessionId,
+    state,
+  });
+} catch (err) {
+  if (err.message.includes('Session not found')) {
+    // Handle missing session
+    logger.error({ message: 'Session does not exist', sessionId, err });
+  } else if (err.message.includes('Validation')) {
+    // Handle validation errors
+    logger.error({ message: 'Invalid state update', err });
+  } else {
+    // Handle other errors
+    logger.error({ message: 'Failed to update state', err });
+  }
+}
+```
+
+## Related Documentation
+
+- [Memory](./memory.md) - Learn about memory schemas and management
+- [Events](./events.md) - Handle agent events and triggers
+- [Debugging](./debugging.md) - Debug state updates and transitions
+- [Tools](../low-code-editor/tools.md) - Create tools that update state

package/docs/sdk/debugging.md

@@ -0,0 +1,80 @@
+# Debugging
+
+MindedJS provides comprehensive debugging capabilities to help you understand and troubleshoot your agent's behavior during development and production. This guide covers various debugging techniques and tools available in the SDK.
+
+## Debug Logging
+
+Set the log level to debug in your environment:
+
+```env
+LOG_LEVEL=debug
+```
+
+## Debugging Logical Conditions
+
+You can debug and breakpoint on logical conditions during development:
+
+```typescript
+// Listen to condition evaluation events
+import { AgentEvents } from 'mindedjs';
+
+agent.on(AgentEvents.ON_LOGICAL_CONDITION, async ({ edge, state, condition }) => {
+  console.log('[Debug] Evaluating condition:', condition);
+  console.log('[Debug] Current memory:', state.memory);
+});
+
+agent.on(AgentEvents.ON_LOGICAL_CONDITION_RESULT, async ({ condition, result, executionTimeMs }) => {
+  console.log('[Debug] Result:', result);
+  console.log('[Debug] Execution time:', executionTimeMs, 'ms');
+});
+```
+
+## LLM Debug Callback Handler
+
+A good practice is to periodically inspect the actual messages being sent to the LLM to ensure they match your expectations.
+You can breakpoint and view the final prompt messages after compilation using `LLMDebugCallbackHandler`.
+
+### Using the LLM Debug Callback Handler
+
+The `LLMDebugCallbackHandler` can be imported directly from the main package:
+
+```typescript
+import { Agent, LLMDebugCallbackHandler } from '@minded-ai/mindedjs';
+
+// Create the debug handler
+const debugHandler = new LLMDebugCallbackHandler();
+
+// Configure your agent with the debug handler
+const agent = new Agent({
+  config: {
+    ...config,
+    llm: {
+      ...config.llm,
+      properties: {
+        ...config.llm.properties,
+        callbacks: [debugHandler],
+      },
+    },
+  },
+  tools,
+  memorySchema,
+});
+```
+
+### Advanced Usage - Custom Debug Handler
+
+You can extend the `LLMDebugCallbackHandler` to add your own debugging logic:
+
+```typescript
+import { LLMDebugCallbackHandler } from '@minded-ai/mindedjs';
+
+class CustomDebugHandler extends LLMDebugCallbackHandler {
+  async handleLLMEnd(output: any, ...args: any[]): Promise<void> {
+    // Log token usage if available
+    const tokenUsage = output.llmOutput?.tokenUsage;
+    console.log(`Token usage: ${tokenUsage.totalTokens}`);
+  }
+}
+```
+
+The `LLMDebugCallbackHandler` extends LangChain's `BaseCallbackHandler` and can override different mathods in langchain flow. See the [LangChain documentation](https://js.langchain.com/docs/modules/callbacks/).