massgen 0.0.3__py3-none-any.whl → 0.1.0a1__py3-none-any.whl

massgen/backend/docs/MCP_IMPLEMENTATION_CLAUDE_BACKEND.md

@@ -0,0 +1,177 @@
+# MCP Integration in ClaudeBackend
+
+## Overview
+
+The `ClaudeBackend` in `massgen/backend/claude.py` features a robust and resilient Model Context Protocol (MCP) integration. Unlike other backends that might differentiate between multiple transport types, the Claude backend uses a **unified recursive model** to handle all external tools. It leverages a single `MultiMCPClient` to manage both `stdio` and `streamable-http` servers, providing a streamlined and powerful way to extend Claude's capabilities with custom tools.
+
+The core of this implementation is the `_stream_mcp_recursive` function, which allows for multiple rounds of tool execution within a single user request, enabling complex, multi-step agentic workflows.
+
+## Key Features
+
+-   **Recursive Execution Loop**: Enables multi-step tool usage where the model can call tools, process results, and call more tools in a continuous loop until it arrives at a final answer.
+-   **Unified Tool Management**: Seamlessly combines built-in Claude tools (Web Search, Code Execution), user-defined functions, and external MCP tools into a single, coherent toolset for the model.
+-   **Claude-Specific Format Conversion**: Automatically converts all tools into the specific `input_schema` format required by the Anthropic Messages API.
+-   **Resilience and Fault Tolerance**: Integrates a circuit breaker pattern to prevent calls to failing MCP servers and a retry mechanism with exponential backoff for individual function calls.
+-   **Advanced Streaming & UI Feedback**: Provides real-time status updates for MCP operations (connection, tool calls, completion) directly in the stream, allowing for a transparent user experience.
+-   **Graceful Error Handling**: If MCP servers fail during setup or execution, the backend automatically falls back to a non-MCP mode, ensuring the user still receives a response from the model.
+
+## Architecture
+
+### Architectural Flow
+
+The Claude backend uses a simplified, unified architecture compared to multi-transport backends. All MCP tools (`stdio` and `streamable-http`) are managed by a single `MultiMCPClient`, and the core logic is contained within a recursive loop.
+
+```mermaid
+graph TD
+    subgraph "Configuration Layer"
+        A1[YAML Configuration] --> A2[mcp_servers]
+    end
+
+    subgraph "ClaudeBackend Layer"
+        B1[__init__] --> B2[Circuit Breaker Setup]
+        B2 --> B3[_setup_mcp_tools]
+        B3 --> B4[MultiMCPClient<br/>(stdio/streamable-http)]
+    end
+
+    subgraph "Recursive Execution Loop"
+        C1[_stream_mcp_recursive] --> C2{Model Needs Tool?}
+        C2 -->|Yes| C3[_execute_mcp_function_with_retry]
+        C3 --> C4[Append Result to History]
+        C4 --> C1
+        C2 -->|No| C5[Yield Final Response]
+    end
+
+    subgraph "External Systems"
+        D1[Anthropic API]
+        D2[MCP Servers]
+    end
+
+    A1 --> B1
+    B4 --> D2
+    C1 --> D1
+    C3 --> B4
+```
+
+### Unified MCP Client
+
+The Claude backend does not separate MCP servers by transport type. Instead, it uses a single `MultiMCPClient` instance from `massgen.mcp_tools` to manage all configured `stdio` and `streamable-http` servers. This simplifies the architecture and allows for consistent handling of all external tools.
+
+```python
+# A single client manages all stdio and streamable-http servers
+self._mcp_client: Optional[MultiMCPClient] = None
+
+# A single function registry holds all discovered tools
+self.functions: Dict[str, Function] = {}
+```
+
+### Recursive Execution Model
+
+The backend's power lies in its recursive design. When the model uses an MCP tool, the backend doesn't simply return the result. Instead, it executes the tool, appends the result to the conversation history, and **calls the model again with the updated history**. This loop continues until the model generates a text response instead of another tool call.
+
+This approach allows the agent to perform complex tasks that require sequential tool use, such as reading a file, analyzing its content, and then writing a summary to a new file.
+
+## Core Components
+
+-   **`ClaudeBackend.__init__`**: Initializes the backend, loads MCP server configurations from the agent config, and prepares the single circuit breaker for all MCP tools.
+
+-   **`_setup_mcp_tools`**: This async method is the heart of the connection process. It normalizes server configurations, filters out any servers currently blocked by the circuit breaker, and initializes the `MultiMCPClient` to connect to the available servers and discover their tools.
+
+-   **`_convert_mcp_tools_to_claude_format`**: A utility that transforms the discovered MCP functions into the JSON format that the Anthropic API expects for its `tools` parameter.
+
+-   **`_build_claude_api_params`**: Constructs the final payload for the Anthropic API call. It gathers all tools—built-in, user-defined, and MCP—and converts them into the correct format. It also handles message history conversion.
+
+-   **`_stream_mcp_recursive`**: The core of the agentic loop. This function:
+    1.  Sends the current message history to the Claude API.
+    2.  Streams the response, identifying any `tool_use` blocks.
+    3.  If an MCP tool is called, it executes it via `_execute_mcp_function_with_retry`.
+    4.  Appends the tool result to the message history.
+    5.  **Recursively calls itself** with the new, updated message history.
+    6.  If no MCP tools are called, it yields the final text response and terminates the loop.
+
+-   **`_execute_mcp_function_with_retry`**: Executes a single MCP function, handling JSON parsing, retries with backoff, and reporting success or failure to the circuit breaker.
+
+-   **`_handle_mcp_error_and_fallback`**: A crucial error handler. If the MCP client fails to connect or a tool fails catastrophically, this function catches the error, yields a user-friendly message, and re-runs the request in a non-MCP mode.
+
+-   **Resource Management (`__aenter__`/`__aexit__`)**: The backend uses an async context manager to ensure that MCP resources are properly managed.
+    -   The `__aenter__` method is responsible for calling `_setup_mcp_tools`, which establishes connections to the MCP servers before a request is processed.
+    -   The `__aexit__` method ensures that `cleanup_mcp` is called when the context is exited, guaranteeing that connections are closed and resources are released, even if errors occur.
+
+## Execution Workflow
+
+A typical request involving an MCP tool follows this sequence:
+
+```mermaid
+sequenceDiagram
+    participant U as User
+    participant CB as ClaudeBackend
+    participant API as Anthropic API
+    participant MCP as MCP Server
+
+    U->>CB: stream_with_tools(messages)
+    activate CB
+    CB->>CB: _setup_mcp_tools()
+    CB->>MCP: Connect & Discover Tools
+    MCP-->>CB: Tools Registered
+    CB->>CB: _stream_mcp_recursive(messages)
+    activate CB
+    CB->>API: messages.create(messages, tools)
+    API-->>CB: Stream [tool_use block]
+    CB->>MCP: _execute_mcp_function_with_retry()
+    activate MCP
+    MCP-->>CB: Tool Result
+    deactivate MCP
+    CB->>CB: Append Result to History
+    CB->>API: messages.create(updated_messages, tools)
+    API-->>CB: Stream [text content]
+    deactivate CB
+    CB-->>U: yield StreamChunk(content)
+    deactivate CB
+```
+
+## Design Choices and Implications
+
+-   **Recursive Model**: The recursive approach is extremely powerful for creating autonomous agents that can solve multi-step problems. However, it carries a risk of long-running loops and high token costs. This is mitigated by the `MCPMessageManager`, which trims the conversation history to prevent uncontrolled growth.
+
+-   **Unified Client**: By using a single client for `stdio` and `streamable-http`, the architecture is simplified. This means the backend has full control over the connection and can support local development servers, but it does not support a native HTTP transport if Anthropic were to introduce one in the future.
+
+## Security and Performance
+
+-   **Network Flexibility**: Because the implementation uses the `mcp_tools` library for `streamable-http`, it is not subject to the same network restrictions as a native API implementation. It can connect to servers on `localhost`, private networks, and corporate VPNs, making it ideal for development and internal enterprise tools.
+-   **Asynchronous Operations**: The entire workflow is built on `asyncio` for non-blocking I/O, ensuring efficient handling of network requests and subprocesses.
+-   **Circuit Breaker**: Prevents the system from repeatedly trying to connect to a failing server, improving stability.
+-   **Bounded History**: The `MCPMessageManager` is used to trim the conversation history within the recursive loop, preventing infinite growth and excessive token consumption.
+-   **Thread Safety**: A lock (`_stats_lock`) is used to safely increment tool usage counters in the async environment.
+
+## Configuration Example
+
+The `ClaudeBackend` is configured via the same `mcp_servers` block in the agent's YAML configuration file. It supports `stdio` and `streamable-http` transport types.
+
+```yaml
+# Example configuration for the Claude backend
+mcp_servers:
+  # stdio transport: for local command-line tools
+  file_system_tools:
+    type: "stdio"
+    command: "python"
+    args: ["-m", "mcp_file_server"]
+    # Optional: specify which tools to use from this server
+    allowed_tools:
+      - "read_file"
+      - "write_file"
+      - "list_directory"
+
+  # streamable-http transport: for web-based streaming services
+  web_search_service:
+    type: "streamable-http"
+    url: "https://api.custom_search.example.com/mcp"
+    # Optional: add authorization headers
+    authorization: "Bearer ${SEARCH_API_TOKEN}"
+```
+
+### Key Configuration Options
+
+-   **`type`**: Specifies the transport. For the `ClaudeBackend`, this can be `stdio` or `streamable-http`.
+-   **`command` / `args`**: Used for `stdio` servers to define the command to execute the local tool server.
+-   **`url`**: Used for `streamable-http` servers to specify the endpoint of the remote tool server.
+-   **`authorization`**: (Optional) Provides an authorization header for `streamable-http` servers. It supports environment variable substitution with the `${VAR_NAME}` syntax.
+-   **`allowed_tools` / `exclude_tools`**: (Optional) A list of tool names to explicitly include or exclude from a given server, allowing for fine-grained control over the available tools.

massgen/backend/docs/MCP_INTEGRATION_RESPONSE_BACKEND.md

@@ -0,0 +1,352 @@
+# MCP Integration in ResponseBackend (OpenAI Response API)
+
+## Overview
+
+The `ResponseBackend` class in `massgen/backend/response.py` implements a sophisticated **three-transport Model Context Protocol (MCP)** integration system that allows seamless connectivity with different types of MCP servers while maintaining fault tolerance and optimal performance.
+
+## Three Transport Types
+
+The system supports three distinct transport types, each serving different use cases:
+
+### 1. **stdio Transport**
+- **Purpose**: Process-based MCP servers running as local subprocesses
+- **Implementation**: Uses `MultiMCPClient` from `massgen.mcp_tools`
+- **Communication**: Standard input/output streams
+- **Use Cases**: Local tools, file system operations, command-line utilities
+- **Examples**: File operations, local development tools, system utilities
+
+### 2. **streamable-http Transport**
+- **Purpose**: Web-based MCP servers with streaming capabilities
+- **Implementation**: Uses `MultiMCPClient` from `massgen.mcp_tools`
+- **Communication**: HTTP with streaming support
+- **Use Cases**: Remote web services, cloud APIs, streaming data sources
+- **Examples**: Weather APIs, stock data, real-time information services
+
+### 3. **http Transport**
+- **Purpose**: Direct OpenAI-native MCP server connections
+- **Implementation**: Uses OpenAI's built-in MCP client
+- **Communication**: Direct HTTP between OpenAI and external servers
+- **Use Cases**: Enterprise MCP servers, high-performance services
+- **Examples**: Corporate MCP servers, specialized AI services
+
+## Why Three Transport Types?
+
+### 1. **Flexibility and Compatibility**
+- **stdio**: Supports traditional command-line tools and legacy systems
+- **streamable-http**: Enables modern web-based services with real-time capabilities
+- **http**: Provides direct, high-performance connectivity for enterprise deployments
+
+### 2. **Optimized Performance**
+- **Local Execution**: stdio servers run locally with minimal latency
+- **Streaming Efficiency**: streamable-http supports real-time data streaming
+- **Direct Connectivity**: http transport eliminates intermediate processing
+
+### 3. **Reliability and Fault Tolerance**
+- **Circuit Breakers**: Each transport type has dedicated circuit breaker protection
+- **Graceful Degradation**: Failure in one transport doesn't affect others
+- **Automatic Fallback**: System continues operating even if some transports fail
+
+## Architecture
+
+### Transport Separation
+```python
+# Servers are separated by transport type during initialization
+self._mcp_tools_servers: List[Dict[str, Any]] = []    # stdio + streamable-http
+self._http_servers: List[Dict[str, Any]] = []         # Native OpenAI HTTP
+```
+
+### Dual Execution Modes
+The backend operates in two distinct modes based on available MCP servers:
+
+#### Mode 1: MCP Tools Mode (stdio + streamable-http)
+- Uses custom `MultiMCPClient` for server management
+- Implements function execution loop with retry logic
+- Supports complex multi-turn conversations with tool calls
+- Handles streaming responses with real-time function execution
+
+#### Mode 2: HTTP-Only Mode (http transport)
+- Uses OpenAI's native MCP client
+- Direct server-to-server communication
+- Simplified execution model
+- Better performance for simple request-response patterns
+
+## Key Components
+
+### 1. **MCP Server Management**
+```python
+def _separate_mcp_servers_by_transport_type(self) -> None:
+    """Separate MCP servers into local execution and HTTP transport types."""
+```
+
+### 2. **Circuit Breaker System**
+- **Dedicated Circuit Breakers**: Separate protection for each transport type
+- **Automatic Failover**: Servers are skipped when circuit breakers trip
+- **Health Monitoring**: Real-time tracking of server availability
+
+```python
+# Circuit breakers for different transport types
+self._mcp_tools_circuit_breaker = None  # For stdio + streamable-http
+self._http_circuit_breaker = None       # For OpenAI native http
+```
+
+### 3. **Function Conversion and Execution**
+- **Automatic Tool Discovery**: Converts MCP tools to OpenAI function format
+- **Retry Logic**: Exponential backoff for failed function calls
+- **Error Handling**: Comprehensive error recovery mechanisms
+
+```python
+async def _execute_mcp_function_with_retry(
+    self, function_name: str, arguments_json: str, max_retries: int = 3
+) -> str:
+```
+
+## Execution Flow
+
+### 1. **Enhanced Initialization with Error Recovery**
+1. Parse MCP server configurations
+2. Separate servers by transport type
+3. Initialize circuit breakers for each transport
+4. **Setup MultiMCPClient with error capture**:
+   - Try to establish MCP connections
+   - **Catch setup errors at top level** (no longer silently swallowed)
+   - **Emit user-visible status messages** for setup failures
+   - **Fall back gracefully** to non-MCP mode if setup fails
+5. **Report connection status** to user via StreamChunk messages
+
+### 2. **Intelligent Tool Processing**
+1. Convert stdio/streamable-http tools to OpenAI function format
+2. Convert http servers to OpenAI native MCP format
+3. Apply circuit breaker filtering
+4. **Check MCP availability** and notify if tools are unavailable
+5. Combine with provider tools (web search, code interpreter)
+
+### 3. **Request Processing with Multi-Level Fallback**
+1. Choose execution mode based on available servers
+2. Build API parameters with appropriate tool formats
+3. **Handle setup-phase errors**: If MCP setup failed, use non-MCP streaming
+4. Execute request with chosen transport
+5. **Handle streaming-phase errors**: If MCP fails during execution, fall back to non-MCP
+6. Handle streaming responses and function calls
+
+### 4. **Robust Function Execution Loop** (stdio + streamable-http)
+1. Detect function calls in streaming response
+2. Execute functions with retry logic and exponential backoff
+3. Update message history with results
+4. **Handle function execution failures**: Continue with other functions or fall back
+5. Continue iteration until completion or max iterations reached
+6. **Provide status updates** for each execution step
+
+## Error Handling and Recovery
+
+### 1. **Multi-Phase Error Detection**
+The system implements comprehensive error handling across two critical phases:
+
+#### Setup Phase Error Handling (`__aenter__`)
+- **Silent Error Capture**: MCP setup errors in `__aenter__` are now caught at the top level
+- **User-Visible Notifications**: Clear status messages inform users when MCP setup fails
+- **Graceful Fallback**: Automatic fallback to non-MCP streaming when setup errors occur
+- **Error Propagation**: Setup errors no longer get silently swallowed
+
+#### Runtime Phase Error Handling (Streaming)
+- **Circuit Breaker Integration**: Automatic detection of server failures during streaming
+- **Temporary Exclusion**: Failed servers are temporarily excluded with status tracking
+- **Gradual Recovery**: Servers are gradually reintroduced after cooldown periods
+
+### 2. **Enhanced Retry Mechanisms**
+- **Exponential Backoff**: Increasing delays between retries for function calls
+- **Maximum Retry Limits**: Prevent infinite retry loops with configurable limits
+- **Error Categorization**: Different handling for MCP-specific vs generic errors
+- **Function-Level Retries**: Individual MCP function calls have dedicated retry logic
+
+### 3. **Advanced Graceful Degradation**
+- **Transport Isolation**: Failure in one transport doesn't affect others
+- **Multi-Level Fallback**: Setup errors → streaming errors → non-MCP mode
+- **User-Friendly Messages**: Clear, contextual error communication:
+  ```python
+  # Setup failure notification
+  "⚠️ [MCP] Setup failed; continuing without MCP"
+
+  # Runtime error with MCP tools
+  "⚠️ [MCP Tool] Connection failed; falling back to non-MCP tools"
+
+  # Successful MCP connection
+  "✅ [MCP] Connected to {n} servers"
+  ```
+- **Provider Tool Preservation**: Web search and code interpreter tools remain available during MCP failures
+
+## Configuration Example
+
+```yaml
+# Example configuration showing all three transport types
+mcp_servers:
+  # stdio transport - local command-line tools
+  file_tools:
+    type: "stdio"
+    command: "python"
+    args: ["-m", "mcp_file_server"]
+
+  # streamable-http transport - web-based streaming services
+  weather_service:
+    type: "streamable-http"
+    url: "https://api.weather.example.com/mcp"
+
+  # http transport - direct OpenAI MCP servers with authorization
+  enterprise_tools:
+    type: "http"
+    url: "https://mcp.enterprise.example.com"
+    authorization: "${ENTERPRISE_MCP_TOKEN}"  # Environment variable substitution
+    require_approval: "never"  # Optional: "never" (default) or "always"
+    allowed_tools:              # Optional: limit to specific tools
+      - "create_report"
+      - "get_data"
+
+  # HTTP MCP server with always requiring approval
+  secure_payment_service:
+    type: "http"
+    url: "https://mcp.stripe.com"
+    authorization: "${STRIPE_OAUTH_ACCESS_TOKEN}"
+    require_approval: "always"  # User must approve each tool call
+```
+
+### Configuration Options
+
+#### require_approval
+- **"never"**: Tools are executed automatically (default behavior)
+- **"always"**: User must approve each tool execution before it runs
+- **Purpose**: Control user interaction level for sensitive operations
+
+#### authorization
+- **Format**: Supports environment variable substitution with `${VAR_NAME}` pattern
+- **Purpose**: Authenticate with HTTP MCP servers requiring API keys or OAuth tokens
+- **Examples**: `"Bearer ${API_TOKEN}"`, `"${OAUTH_ACCESS_TOKEN}"`
+
+#### allowed_tools
+- **Purpose**: Limit server to specific tools (optional)
+- **Format**: List of tool names to include
+- **Use Case**: Restrict access to sensitive operations
+
+## Performance Optimization
+
+### 1. **Resource Management**
+- **Connection Pooling**: Efficient connection reuse
+- **Memory Management**: Bounded message history to prevent memory leaks
+- **Async Processing**: Non-blocking operations for better performance
+
+### 2. **Monitoring and Metrics**
+- **Call Tracking**: Statistics on tool calls and failures
+- **Performance Logging**: Detailed logging for optimization
+- **Circuit Breaker Status**: Real-time health monitoring
+
+### 3. **Scalability Features**
+- **Multi-Server Support**: Simultaneous connections to multiple servers
+- **Tool Filtering**: Selective tool inclusion/exclusion
+- **Concurrent Execution**: Parallel tool execution when possible
+
+## Security Considerations
+
+### 1. **URL Validation**
+- **Security Checks**: Validation of HTTP server URLs
+- **Localhost Support**: Allowance for development environments
+- **Private Network Support**: Support for internal corporate networks
+
+### 2. **Command Sanitization**
+- **Safe Execution**: Secure handling of stdio commands
+- **Input Validation**: Validation of all function arguments
+- **Sandboxing**: Isolation of potentially dangerous operations
+
+## Limitations of OpenAI Native HTTP Transport
+
+While the http transport type offers direct connectivity and high performance, it has significant limitations compared to stdio and streamable-http transports:
+
+### 1. **Network Accessibility Restrictions**
+- **No Local Servers**: OpenAI's native MCP client cannot access local HTTP servers running on `localhost`, `127.0.0.1`, or private IP addresses
+- **Corporate Network Restrictions**: Servers behind corporate firewalls or in private networks are inaccessible
+- **VPN/Proxy Limitations**: Some VPN configurations may block access to MCP servers
+- **Geographic Restrictions**: OpenAI's network may have geographic limitations for certain server locations
+
+### 2. **Security and Compliance Constraints**
+- **OpenAI Security Policies**: All HTTP MCP servers must comply with OpenAI's security requirements and acceptable use policies
+- **Content Filtering**: OpenAI may filter or block certain types of content or server responses
+- **Rate Limiting**: Subject to OpenAI's rate limiting and usage policies
+- **Data Privacy**: All server communications go through OpenAI's infrastructure
+
+### 3. **Server Configuration Requirements**
+- **HTTPS Mandatory**: All HTTP MCP servers must use HTTPS with valid SSL certificates
+- **Domain Registration**: Servers must have properly registered domain names
+- **Public Accessibility**: Servers must be publicly accessible on the internet
+- **CORS Configuration**: Proper Cross-Origin Resource Sharing setup may be required
+
+### 4. **Development and Testing Challenges**
+- **Local Development**: Cannot test with local MCP servers during development
+- **Staging Environments**: Staging servers in private networks are inaccessible
+- **Debugging Complexity**: Harder to debug HTTP MCP issues due to limited visibility
+- **Testing Overhead**: Requires deploying servers to public environments for testing
+
+### 5. **Feature Limitations**
+- **Streaming Support**: Limited or no support for streaming responses compared to streamable-http
+- **Custom Headers**: Restrictions on custom HTTP headers and authentication methods
+- **Session Management**: Limited control over session persistence and state management
+- **Timeout Handling**: Fixed timeout configurations that cannot be customized
+
+### 6. **Reliability and Control**
+- **Dependency on OpenAI**: Relies on OpenAI's infrastructure and network stability
+- **Limited Monitoring**: Reduced visibility into connection health and performance
+- **Circuit Breaker Control**: Less granular control over circuit breaker behavior
+- **Failover Options**: Limited failover capabilities when OpenAI's MCP client has issues
+
+## When to Use Each Transport Type
+
+### Use stdio Transport When:
+- Working with local development tools
+- Accessing file system operations
+- Running command-line utilities
+- Needing maximum control and visibility
+- Working in isolated environments
+
+### Use streamable-http Transport When:
+- Accessing remote web services
+- Needing real-time streaming capabilities
+- Working with cloud-based APIs
+- Requiring custom authentication
+- Needing detailed debugging information
+
+### Use http Transport When:
+- Working with enterprise MCP servers
+- Needing maximum performance
+- Servers are publicly accessible
+- Compliance with OpenAI policies is acceptable
+- No local or private network access is required
+
+## Migration Considerations
+
+When migrating from stdio/streamable-http to http transport, consider:
+
+1. **Server Deployment**: MCP servers must be deployed to public infrastructure
+2. **Security Updates**: Ensure servers meet OpenAI's security requirements
+3. **Testing Strategy**: Implement comprehensive testing for public accessibility
+4. **Monitoring**: Add monitoring for public server availability and performance
+5. **Fallback Plan**: Maintain stdio/streamable-http options as backup
+
+## Benefits
+
+### 1. **Maximum Compatibility**
+- Supports all major MCP server types
+- Works with existing tools and services
+- Seamless integration with OpenAI's ecosystem
+
+### 2. **High Reliability**
+- Fault tolerance through multiple transport options
+- Automatic recovery from failures
+- Graceful degradation when issues occur
+
+### 3. **Developer Experience**
+- Simple configuration with YAML files
+- Automatic tool discovery and conversion
+- Comprehensive error handling and logging
+
+### 4. **Production Ready**
+- Circuit breaker patterns for stability
+- Resource cleanup and memory management
+- Comprehensive monitoring and metrics
+
+This three-transport architecture provides a robust, flexible, and production-ready MCP integration system that can handle diverse use cases while maintaining high reliability and performance.