claude-mpm 4.16.0__py3-none-any.whl → 4.25.10__py3-none-any.whl

claude_mpm/services/event_bus/README.md

@@ -0,0 +1,244 @@
+# EventBus Service
+
+A decoupled event handling system for claude-mpm that enables loose coupling between event producers (like hooks) and consumers (like Socket.IO).
+
+## Overview
+
+The EventBus service provides a centralized publish-subscribe system that:
+- Decouples event producers from consumers
+- Supports both synchronous and asynchronous handlers
+- Provides event filtering and routing capabilities
+- Isolates failures in individual consumers
+- Offers comprehensive monitoring and statistics
+
+## Architecture
+
+```
+Hook Handler → EventBus → Multiple Consumers
+                         ├── Socket.IO Relay
+                         ├── Logging Service
+                         ├── Analytics
+                         └── Custom Handlers
+```
+
+## Key Components
+
+### EventBus (`event_bus.py`)
+- Singleton pattern for centralized event coordination
+- Thread-safe event publishing from any context
+- Support for wildcard subscriptions (`hook.*`)
+- Event filtering and routing
+- Built-in statistics and history tracking
+
+### SocketIORelay (`relay.py`)
+- Consumes events from EventBus
+- Relays to Socket.IO clients
+- Handles connection failures gracefully
+- Single point of Socket.IO management
+
+### Configuration (`config.py`)
+- Environment variable support
+- Runtime configuration changes
+- Easy testing with different settings
+
+## Installation
+
+The EventBus requires `pyee` (Python EventEmitter):
+
+```bash
+pip install pyee>=13.0.0
+```
+
+## Usage
+
+### Basic Publishing
+
+```python
+from claude_mpm.services.event_bus import EventBus
+
+# Get singleton instance
+event_bus = EventBus.get_instance()
+
+# Publish an event
+event_bus.publish("hook.pre_tool", {
+    "tool_name": "Read",
+    "file_path": "/example.py",
+    "sessionId": "abc-123"
+})
+```
+
+### Subscribing to Events
+
+```python
+# Subscribe to specific event
+def handle_pre_tool(data):
+    print(f"Tool: {data.get('tool_name')}")
+
+event_bus.on("hook.pre_tool", handle_pre_tool)
+
+# Subscribe to wildcard pattern
+def handle_all_hooks(event_type, data):
+    print(f"Hook event: {event_type}")
+
+event_bus.on("hook.*", handle_all_hooks)
+```
+
+### Async Handlers
+
+```python
+async def async_handler(data):
+    await process_async(data)
+
+event_bus.on("app.async_event", async_handler)
+```
+
+### Event Filtering
+
+```python
+# Only allow specific events
+event_bus.add_filter("hook.*")
+event_bus.add_filter("system.critical.*")
+
+# Events outside filters will be dropped
+event_bus.publish("app.user_action", {})  # Filtered out
+```
+
+### Socket.IO Integration
+
+```python
+from claude_mpm.services.event_bus import SocketIORelay
+
+# Create and start relay
+relay = SocketIORelay(port=8765)
+relay.start()
+
+# Events published to EventBus will be relayed to Socket.IO
+event_bus.publish("hook.subagent_stop", {
+    "agent_type": "Engineer",
+    "result": "Success"
+})
+```
+
+## Configuration
+
+Configure via environment variables:
+
+```bash
+# EventBus settings
+export CLAUDE_MPM_EVENTBUS_ENABLED=true
+export CLAUDE_MPM_EVENTBUS_DEBUG=false
+export CLAUDE_MPM_EVENTBUS_HISTORY_SIZE=100
+export CLAUDE_MPM_EVENTBUS_FILTERS="hook.*,system.critical.*"
+
+# Relay settings
+export CLAUDE_MPM_RELAY_ENABLED=true
+export CLAUDE_MPM_SOCKETIO_PORT=8765
+export CLAUDE_MPM_RELAY_DEBUG=false
+export CLAUDE_MPM_RELAY_MAX_RETRIES=3
+```
+
+## Migration from Direct Socket.IO
+
+### Before (Direct Socket.IO)
+```python
+# In hook_handler.py
+client = socketio.Client()
+client.connect("http://localhost:8765")
+client.emit("claude_event", normalized_data)
+```
+
+### After (EventBus)
+```python
+# In hook_handler.py
+event_bus = EventBus.get_instance()
+event_bus.publish("hook.pre_tool", data)
+
+# Socket.IO handled by relay (separate concern)
+```
+
+## Benefits
+
+1. **Decoupling**: Hooks don't need to know about Socket.IO
+2. **Testability**: Easy to test without Socket.IO running
+3. **Reliability**: Failures in one consumer don't affect others
+4. **Flexibility**: Add new consumers without modifying producers
+5. **Performance**: Events processed asynchronously
+6. **Monitoring**: Built-in statistics and event history
+
+## Testing
+
+Run the test suite:
+
+```bash
+# Integration tests
+python scripts/test_eventbus_integration.py
+
+# Usage examples
+python scripts/eventbus_usage_example.py
+
+# Complete flow test
+python scripts/test_eventbus_flow.py
+```
+
+## Monitoring
+
+Get EventBus statistics:
+
+```python
+stats = event_bus.get_stats()
+print(f"Events published: {stats['events_published']}")
+print(f"Events filtered: {stats['events_filtered']}")
+
+# Recent event history
+recent = event_bus.get_recent_events(10)
+for event in recent:
+    print(f"{event['type']} at {event['timestamp']}")
+```
+
+## Error Handling
+
+Errors in consumers are isolated:
+
+```python
+def good_handler(data):
+    process(data)  # Always runs
+
+def bad_handler(data):
+    raise Exception("Error")  # Doesn't affect good_handler
+
+event_bus.on("app.event", good_handler)
+event_bus.on("app.event", bad_handler)
+```
+
+## Performance
+
+- Handles 30,000+ events/second
+- Minimal memory overhead
+- Thread-safe for concurrent publishing
+- Efficient wildcard matching
+
+## Troubleshooting
+
+### Events not being received
+1. Check if EventBus is enabled: `CLAUDE_MPM_EVENTBUS_ENABLED=true`
+2. Verify no filters are blocking: `event_bus.clear_filters()`
+3. Check handler registration: `event_bus.on("event_type", handler)`
+
+### Socket.IO relay not working
+1. Verify Socket.IO server is running on configured port
+2. Check relay is started: `relay.start()`
+3. Enable debug logging: `CLAUDE_MPM_RELAY_DEBUG=true`
+
+### High memory usage
+1. Reduce history size: `CLAUDE_MPM_EVENTBUS_HISTORY_SIZE=50`
+2. Clear history periodically: `event_bus.clear_history()`
+
+## Future Enhancements
+
+- [ ] Persistent event storage
+- [ ] Event replay capabilities
+- [ ] Advanced routing rules
+- [ ] Event prioritization
+- [ ] Distributed EventBus (Redis backend)
+- [ ] Event schema validation
+- [ ] Rate limiting per consumer

claude_mpm/services/event_bus/direct_relay.py

@@ -256,9 +256,9 @@ class DirectSocketIORelay:
                                         event_type, broadcast_data
                                     )
                                     self.stats["events_relayed"] += 1
-                                    self.stats[
-                                        "events_failed"
-                                    ] -= 1  # Undo the failure count
+                                    self.stats["events_failed"] -= (
+                                        1  # Undo the failure count
+                                    )
                                     logger.info(
                                         f"[DirectRelay] Retry successful for {event_type}"
                                     )

claude_mpm/services/event_bus/event_bus.py

@@ -68,6 +68,9 @@ class EventBus:
         # Track async handler tasks to prevent garbage collection
         self._handler_tasks: Set[asyncio.Task] = set()
 
+        # Track handler wrappers for removal
+        self._handler_wrappers: Dict[tuple, Callable] = {}
+
         logger.info("EventBus initialized")
 
     @classmethod
@@ -266,8 +269,20 @@ class EventBus:
 
             logger.debug(f"Registered wildcard handler for: {event_type}")
         else:
-            # Regular event registration
-            self._emitter.on(event_type, handler)
+            # Wrap handler to catch exceptions and prevent them from stopping other handlers
+            def safe_handler(data):
+                try:
+                    handler(data)
+                except Exception as e:
+                    if self._debug:
+                        logger.debug(f"Handler error for {event_type}: {e}")
+
+            # Store mapping for later removal
+            wrapper_key = (event_type, handler)
+            self._handler_wrappers[wrapper_key] = safe_handler
+
+            # Regular event registration with wrapped handler
+            self._emitter.on(event_type, safe_handler)
             logger.debug(f"Registered handler for: {event_type}")
 
     def once(self, event_type: str, handler: Callable) -> None:
@@ -287,7 +302,17 @@ class EventBus:
             event_type: The event type
             handler: The handler to remove
         """
-        self._emitter.remove_listener(event_type, handler)
+        # Check if we have a wrapped version of this handler
+        wrapper_key = (event_type, handler)
+        if wrapper_key in self._handler_wrappers:
+            # Remove the wrapped handler from pyee
+            wrapped_handler = self._handler_wrappers[wrapper_key]
+            self._emitter.remove_listener(event_type, wrapped_handler)
+            # Remove from our tracking dict
+            del self._handler_wrappers[wrapper_key]
+        else:
+            # No wrapper, remove directly (e.g., for wildcard handlers or direct registrations)
+            self._emitter.remove_listener(event_type, handler)
         logger.debug(f"Removed handler for: {event_type}")
 
     def remove_all_listeners(self, event_type: Optional[str] = None) -> None:
@@ -298,9 +323,17 @@ class EventBus:
         """
         if event_type:
             self._emitter.remove_all_listeners(event_type)
+            # Clean up wrappers for this event type
+            wrappers_to_remove = [
+                key for key in self._handler_wrappers if key[0] == event_type
+            ]
+            for key in wrappers_to_remove:
+                del self._handler_wrappers[key]
             logger.debug(f"Removed all handlers for: {event_type}")
         else:
             self._emitter.remove_all_listeners()
+            # Clean up all wrappers
+            self._handler_wrappers.clear()
             logger.debug("Removed all event handlers")
 
     def _record_event(self, event_type: str, data: Any) -> None:

claude_mpm/services/event_bus/relay.py

@@ -9,6 +9,7 @@ WHY separate relay component:
 """
 
 import os
+import threading
 import time
 from datetime import datetime, timezone
 from typing import Any, Dict, Optional
@@ -271,11 +272,15 @@ class SocketIORelay:
 
 # Global relay instance
 _relay_instance: Optional[SocketIORelay] = None
+_relay_lock = threading.Lock()
 
 
 def get_relay(port: Optional[int] = None) -> SocketIORelay:
     """Get or create the global SocketIO relay instance.
 
+    Thread-safe implementation using double-checked locking pattern to
+    prevent race conditions during concurrent initialization.
+
     Args:
         port: Optional port number
 
@@ -283,9 +288,16 @@ def get_relay(port: Optional[int] = None) -> SocketIORelay:
         SocketIORelay: The relay instance
     """
     global _relay_instance
-    if _relay_instance is None:
-        _relay_instance = SocketIORelay(port)
-    return _relay_instance
+
+    # Fast path - check without lock
+    if _relay_instance is not None:
+        return _relay_instance
+
+    # Slow path - acquire lock and double-check
+    with _relay_lock:
+        if _relay_instance is None:
+            _relay_instance = SocketIORelay(port)
+        return _relay_instance
 
 
 def start_relay(port: Optional[int] = None) -> SocketIORelay:
@@ -303,8 +315,12 @@ def start_relay(port: Optional[int] = None) -> SocketIORelay:
 
 
 def stop_relay() -> None:
-    """Stop the global SocketIO relay."""
+    """Stop the global SocketIO relay.
+
+    Thread-safe implementation ensures proper cleanup.
+    """
     global _relay_instance
-    if _relay_instance:
-        _relay_instance.stop()
-        _relay_instance = None
+    with _relay_lock:
+        if _relay_instance:
+            _relay_instance.stop()
+            _relay_instance = None

claude_mpm/services/events/README.md

@@ -0,0 +1,303 @@
+# Event Bus System
+
+A decoupled, reliable event system for Claude MPM that separates event producers from consumers, enabling better testing, maintenance, and extensibility.
+
+## Overview
+
+The Event Bus system provides a central hub for all events in the Claude MPM framework. Instead of components directly calling Socket.IO or other services, they publish events to the bus, which routes them to interested consumers.
+
+## Architecture
+
+```
+Producers → Event Bus → Consumers
+```
+
+### Key Components
+
+- **EventBus**: Central routing and queuing system
+- **Event**: Standard event format with metadata
+- **Producers**: Publish events without knowing consumers
+- **Consumers**: Subscribe to topics and process events
+- **Topics**: Hierarchical event categorization
+
+## Benefits
+
+### 1. Decoupling
+- Producers don't know about consumers
+- Socket.IO is just another consumer
+- Easy to add/remove consumers without affecting producers
+
+### 2. Reliability
+- Events queued when consumers unavailable
+- Retry logic with exponential backoff
+- Dead letter queue for failed events
+- Graceful degradation
+
+### 3. Testability
+- Test producers without Socket.IO
+- Mock event bus for unit tests
+- Test consumers in isolation
+- Deterministic event flow
+
+### 4. Observability
+- Central metrics collection
+- Event flow monitoring
+- Performance tracking
+- Debug logging
+
+### 5. Flexibility
+- Topic-based filtering
+- Event transformation
+- Batch processing
+- Priority queuing
+
+## Usage
+
+### Basic Setup
+
+```python
+from claude_mpm.services.events import EventBus, SocketIOConsumer, LoggingConsumer
+
+# Create and start event bus
+bus = EventBus()
+await bus.start()
+
+# Add consumers
+socketio_consumer = SocketIOConsumer()
+await bus.subscribe(socketio_consumer)
+
+logging_consumer = LoggingConsumer(topics=["hook.**"])
+await bus.subscribe(logging_consumer)
+```
+
+### Publishing Events
+
+```python
+from claude_mpm.services.events import HookEventProducer
+
+# Create producer
+producer = HookEventProducer(bus)
+
+# Publish events
+await producer.publish_response({
+    "content": "Assistant response text",
+    "model": "claude-3",
+})
+
+await producer.publish_tool_use(
+    tool_name="Read",
+    tool_params={"file_path": "/path/to/file"},
+    tool_result="File contents",
+)
+```
+
+### Creating Custom Consumers
+
+```python
+from claude_mpm.services.events import IEventConsumer, ConsumerConfig
+
+class CustomConsumer(IEventConsumer):
+    def __init__(self):
+        self._config = ConsumerConfig(
+            name="CustomConsumer",
+            topics=["system.**"],  # Subscribe to system events
+            priority=ConsumerPriority.NORMAL,
+        )
+    
+    async def consume(self, event: Event) -> bool:
+        # Process event
+        print(f"Processing: {event.topic} - {event.type}")
+        return True
+    
+    # Implement other required methods...
+```
+
+## Topic Hierarchy
+
+Events are organized in a hierarchical topic structure:
+
+```
+hook.*          # Hook system events
+  .response     # Assistant responses
+  .tool         # Tool usage
+  .subagent.*   # Subagent events
+  .error        # Hook errors
+
+system.*        # System events
+  .lifecycle.*  # Service startup/shutdown
+  .health       # Health status
+  .config       # Configuration changes
+  .performance  # Performance metrics
+  .error        # System errors
+  .warning      # System warnings
+
+cli.*           # CLI command events
+agent.*         # Agent events
+build.*         # Build monitoring events
+```
+
+### Topic Patterns
+
+- `hook.*` - Matches `hook.response` but not `hook.tool.usage`
+- `hook.**` - Matches all hook events including nested
+- `**` - Matches all events
+
+## Event Format
+
+```python
+@dataclass
+class Event:
+    id: str                    # Unique event ID
+    topic: str                 # Event topic
+    type: str                  # Event type
+    timestamp: datetime        # Creation time
+    source: str               # Event source
+    data: Dict[str, Any]      # Event payload
+    metadata: EventMetadata   # Processing metadata
+    correlation_id: str       # Track related events
+    priority: EventPriority   # Processing priority
+```
+
+## Consumers
+
+### Built-in Consumers
+
+#### SocketIOConsumer
+- Emits events via Socket.IO
+- Batching for efficiency
+- Automatic reconnection
+- Connection pooling
+
+#### LoggingConsumer
+- Logs events for debugging
+- Configurable log levels
+- JSON formatting
+- Topic filtering
+
+#### MetricsConsumer
+- Collects event statistics
+- Rate calculation
+- Latency tracking
+- Top event analysis
+
+#### DeadLetterConsumer
+- Stores failed events
+- Event replay capability
+- Retention policies
+- Failure analysis
+
+## Producers
+
+### Built-in Producers
+
+#### HookEventProducer
+- Publishes hook system events
+- Helper methods for common events
+- Correlation tracking
+
+#### SystemEventProducer
+- Publishes system-level events
+- Service lifecycle events
+- Health and metrics
+- Configuration changes
+
+## Configuration
+
+### EventBus Options
+
+```python
+EventBus(
+    max_queue_size=10000,      # Maximum queued events
+    process_interval=0.01,     # Processing frequency
+    batch_timeout=0.1,         # Batch wait time
+    enable_metrics=True,       # Track metrics
+    enable_persistence=False,  # Persist to disk
+)
+```
+
+### Consumer Configuration
+
+```python
+ConsumerConfig(
+    name="MyConsumer",
+    topics=["hook.**"],           # Topics to subscribe
+    priority=ConsumerPriority.HIGH,
+    batch_size=10,                # Process in batches
+    batch_timeout=0.5,            # Batch wait time
+    max_retries=3,                # Retry failed events
+    retry_backoff=2.0,            # Backoff multiplier
+    filter_func=my_filter,        # Event filter
+    transform_func=my_transform,  # Event transformer
+)
+```
+
+## Migration Guide
+
+### Updating Hook Handler
+
+Before (Direct Socket.IO):
+```python
+# Old: Direct Socket.IO dependency
+socketio_client = socketio.Client()
+socketio_client.emit("hook_event", event_data)
+```
+
+After (Event Bus):
+```python
+# New: Publish to event bus
+producer = HookEventProducer(event_bus)
+await producer.publish_response(response_data)
+```
+
+### Testing
+
+Before:
+```python
+# Hard to test - needs Socket.IO server
+def test_hook_handler():
+    # Complex setup with Socket.IO
+    pass
+```
+
+After:
+```python
+# Easy to test - mock event bus
+def test_hook_handler():
+    mock_bus = Mock()
+    producer = HookEventProducer(mock_bus)
+    await producer.publish_response(data)
+    mock_bus.publish.assert_called_once()
+```
+
+## Performance
+
+- **Publish Latency**: <1ms average
+- **Processing Throughput**: 10,000+ events/sec
+- **Memory Usage**: ~100 bytes per queued event
+- **CPU Overhead**: <5% for typical load
+
+## Best Practices
+
+1. **Use Topics Wisely**: Create logical topic hierarchies
+2. **Set Priorities**: Critical events should have higher priority
+3. **Handle Errors**: Consumers should handle their own errors
+4. **Monitor Metrics**: Track queue size and processing rates
+5. **Test in Isolation**: Test producers and consumers separately
+6. **Batch When Possible**: Improve efficiency with batching
+7. **Clean Shutdown**: Always call `bus.stop()` for graceful shutdown
+
+## Examples
+
+See the `scripts/` directory for complete examples:
+- `test_event_bus.py` - Basic event bus demonstration
+- `hook_handler_with_event_bus.py` - Hook handler integration
+
+## Future Enhancements
+
+- [ ] Persistent event storage
+- [ ] Event replay from timestamp
+- [ ] Distributed event bus (Redis/RabbitMQ backend)
+- [ ] Event schema validation
+- [ ] Advanced routing rules
+- [ ] Event compression
+- [ ] WebSocket consumer for real-time streaming

claude_mpm/services/events/consumers/logging.py

@@ -133,8 +133,7 @@ class LoggingConsumer(IEventConsumer):
         """Format an event for logging."""
         # Build base message
         message = (
-            f"[{event.topic}] {event.type} "
-            f"(id={event.id[:8]}, source={event.source})"
+            f"[{event.topic}] {event.type} (id={event.id[:8]}, source={event.source})"
         )
 
         # Add data if configured