bunny_farm 0.1.2

data/docs/architecture/overview.md

@@ -0,0 +1,448 @@
+# Architecture Overview
+
+BunnyFarm provides a clean, layered architecture that separates concerns and enables scalable message processing. This section covers the overall system design and how components interact.
+
+<img src="../assets/architecture_overview.svg" alt="Architecture Overview" width="100%">
+
+## Architectural Layers
+
+### 1. Ruby Application Layer
+
+The top layer consists of your Ruby applications that produce and consume messages:
+
+**Producers:**
+- Web applications handling user requests
+- API servers processing external requests  
+- Scheduled jobs and cron tasks
+- Other microservices
+
+**Consumers:**
+- Background worker processes
+- Microservices handling specific domains
+- Event processors
+- Data pipeline components
+
+### 2. BunnyFarm Library Layer
+
+The BunnyFarm library provides the core abstractions and functionality:
+
+**Core Components:**
+
+#### Message Classes
+- Base class for all message types
+- DSL for defining data structure (fields)
+- DSL for defining operations (actions)
+- Built-in serialization and validation
+
+#### Publisher
+- Message publishing functionality
+- Connection management
+- Routing key generation
+- Error handling
+
+#### Consumer
+- Message consumption and processing
+- Automatic deserialization
+- Action method routing
+- ACK/NACK management
+
+#### Configuration
+- Environment-based configuration
+- YAML configuration file support
+- Programmatic configuration
+- Connection parameter management
+
+#### Router
+- Routing key parsing and generation
+- Action method dispatching
+- Message type resolution
+
+### 3. RabbitMQ Message Broker
+
+RabbitMQ serves as the message broker layer:
+
+**Key Components:**
+
+#### Exchange
+- Routes messages based on routing keys
+- Typically uses topic exchange for flexible routing
+- Handles message durability and persistence
+
+#### Queues
+- Store messages until consumed
+- Support various configurations (durable, auto-delete, etc.)
+- Handle message ordering and priority
+
+#### Management Interface
+- Web-based administration
+- Queue and exchange monitoring
+- Performance metrics and alerts
+
+### 4. Persistence & External Services
+
+The bottom layer includes external dependencies:
+
+- **Databases** - For persistent data storage
+- **Email Services** - For notification sending
+- **File Storage** - For document and media handling
+- **External APIs** - For third-party integrations
+- **Caching** - Redis/Memcached for performance
+
+## Message Flow Architecture
+
+<img src="../assets/message_flow.svg" alt="Message Flow" width="100%">
+
+The message processing follows a predictable flow:
+
+### 1. Message Creation
+```ruby
+message = OrderMessage.new
+message[:order_id] = 12345
+message[:customer] = customer_data
+```
+
+### 2. Publishing
+```ruby
+message.publish('process') # Creates routing key: OrderMessage.process
+```
+
+### 3. RabbitMQ Routing
+- Exchange receives message with routing key
+- Routes to appropriate queue(s) based on bindings
+- Message persisted until consumed
+
+### 4. Consumption
+```ruby
+BunnyFarm.manage # Consumer polls for messages
+```
+
+### 5. Processing
+- Message deserialized from JSON
+- Appropriate action method called
+- Business logic executed
+
+### 6. Acknowledgment
+- Success returns `true` → ACK sent to RabbitMQ
+- Failure returns `false` → NACK sent (message requeued or dead-lettered)
+
+## Design Principles
+
+### Message-Centric Design
+
+Messages are first-class objects that encapsulate both data and behavior:
+
+```ruby
+class OrderMessage < BunnyFarm::Message
+  # Data structure
+  fields :order_id, :customer, :items
+  
+  # Behavior  
+  actions :validate, :process, :ship
+  
+  def validate
+    # Validation logic
+  end
+end
+```
+
+This approach provides:
+- **Encapsulation** - Data and behavior stay together
+- **Discoverability** - Easy to understand what a message can do
+- **Testability** - Individual message types can be unit tested
+
+### Predictable Routing
+
+Routing keys follow a consistent pattern:
+
+```
+MessageClassName.action
+```
+
+Examples:
+- `OrderMessage.process`
+- `EmailMessage.send`
+- `ReportMessage.generate`
+
+Benefits:
+- **Transparency** - Easy to understand message flow
+- **Debugging** - Clear routing paths
+- **Monitoring** - Trackable message types
+
+### JSON Serialization
+
+All message data uses JSON serialization:
+
+```json
+{
+  "order_id": 12345,
+  "customer": {
+    "name": "John Doe", 
+    "email": "john@example.com"
+  },
+  "items": [
+    {"product_id": 1, "quantity": 2}
+  ]
+}
+```
+
+Advantages:
+- **Human readable** - Easy debugging and monitoring
+- **Language agnostic** - Other systems can integrate
+- **Tooling support** - JSON tools widely available
+
+## Scalability Patterns
+
+### Horizontal Scaling
+
+Add more consumers to handle increased load:
+
+```ruby
+# Worker 1
+BunnyFarm.config { app_id 'worker_1' }
+BunnyFarm.manage
+
+# Worker 2  
+BunnyFarm.config { app_id 'worker_2' }
+BunnyFarm.manage
+
+# Worker N...
+```
+
+Multiple consumers can process messages from the same queue concurrently.
+
+### Queue Partitioning
+
+Use different queues for different message types or priorities:
+
+```ruby
+# High priority queue
+BunnyFarm.config do
+  queue_name 'high_priority_orders'
+  routing_key 'OrderMessage.urgent'
+end
+
+# Normal priority queue
+BunnyFarm.config do  
+  queue_name 'normal_orders'
+  routing_key 'OrderMessage.process'
+end
+```
+
+### Load Balancing
+
+RabbitMQ automatically load balances messages across multiple consumers using round-robin distribution.
+
+## Fault Tolerance
+
+### Message Persistence
+
+Messages can be persisted to disk:
+
+```ruby
+BunnyFarm.config do
+  exchange_options durable: true
+  queue_options durable: true
+  message_options persistent: true
+end
+```
+
+### Error Handling
+
+Failed messages can be:
+- **Retried** - Requeued automatically
+- **Dead lettered** - Sent to error queue
+- **Logged** - For manual inspection
+
+```ruby
+def risky_operation
+  begin
+    perform_work
+    success!
+  rescue => e
+    failure("Operation failed: #{e.message}")
+  end
+  
+  successful?
+end
+```
+
+### Circuit Breaker Pattern
+
+Prevent cascading failures:
+
+```ruby
+def external_service_call
+  if circuit_breaker.open?
+    failure("Circuit breaker open - service unavailable")
+    return
+  end
+  
+  begin
+    result = external_service.call
+    circuit_breaker.record_success
+    success!
+  rescue => e
+    circuit_breaker.record_failure
+    failure("External service failed: #{e.message}")
+  end
+  
+  successful?
+end
+```
+
+## Monitoring and Observability
+
+### RabbitMQ Management
+
+Built-in monitoring via RabbitMQ management interface:
+- Queue depths and rates
+- Consumer activity  
+- Message flow metrics
+- Connection status
+
+### Application Metrics
+
+Track application-level metrics:
+
+```ruby
+def process_order
+  StatsD.increment 'orders.processing.started'
+  
+  result = perform_processing
+  
+  if successful?
+    StatsD.increment 'orders.processing.success'
+  else
+    StatsD.increment 'orders.processing.failed'
+  end
+  
+  result
+end
+```
+
+### Logging
+
+Structured logging for message processing:
+
+```ruby
+def process_order
+  logger.info "Processing order", order_id: @items[:order_id]
+  
+  # ... processing
+  
+  if successful?
+    logger.info "Order processed successfully", order_id: @items[:order_id]
+  else
+    logger.error "Order processing failed", 
+                 order_id: @items[:order_id], 
+                 errors: errors
+  end
+end
+```
+
+## Integration Patterns
+
+### Event-Driven Architecture
+
+Use BunnyFarm for event-driven communication between services:
+
+```ruby
+# Service A publishes events
+order_created = OrderCreatedEvent.new
+order_created[:order_id] = order.id
+order_created.publish('notify_services')
+
+# Service B subscribes to events
+class OrderCreatedEvent < BunnyFarm::Message
+  actions :update_inventory, :send_confirmation
+  
+  def update_inventory
+    # Update inventory service
+  end
+  
+  def send_confirmation  
+    # Send customer confirmation
+  end
+end
+```
+
+### Saga Pattern
+
+Implement distributed transactions:
+
+```ruby
+class OrderSaga < BunnyFarm::Message
+  actions :start, :process_payment, :reserve_inventory, 
+          :ship_order, :complete, :compensate
+          
+  def start
+    # Begin saga
+    next_step('process_payment')
+  end
+  
+  def process_payment
+    if payment_successful?
+      next_step('reserve_inventory')
+    else
+      compensate('payment_failed')
+    end
+  end
+  
+  private
+  
+  def next_step(action)
+    self.class.new(@items).publish(action)
+  end
+end
+```
+
+## Performance Considerations
+
+### Connection Pooling
+
+For high-throughput scenarios:
+
+```ruby
+BunnyFarm.config do
+  connection_pool_size 10
+  channel_pool_size 20
+end
+```
+
+### Batch Processing
+
+Process multiple messages together:
+
+```ruby
+def process_batch
+  messages = consume_batch(size: 100)
+  
+  ActiveRecord::Base.transaction do
+    messages.each(&:process)
+  end
+end
+```
+
+### Memory Management
+
+For long-running consumers:
+
+```ruby
+def process_with_memory_management
+  GC.start if processed_count % 1000 == 0
+  
+  # Process message
+  
+  ensure
+    # Cleanup resources
+end
+```
+
+## Next Steps
+
+Explore specific architectural topics:
+
+- **[Message Flow](message-flow.md)** - Detailed message processing flow
+- **[Scaling Patterns](scaling.md)** - Horizontal and vertical scaling strategies  
+- **[Integration Patterns](integration.md)** - Common integration scenarios
+- **[Configuration](../configuration/overview.md)** - Architecture configuration options

data/docs/architecture/scaling.md

@@ -0,0 +1,8 @@
+# Scaling Patterns
+
+Horizontal and vertical scaling strategies.
+
+## Horizontal Scaling
+- Multiple consumer instances
+- Queue partitioning
+- Load balancing

data/docs/assets/actions_dsl_flow.svg

@@ -0,0 +1,109 @@
+<svg width="900" height="600" xmlns="http://www.w3.org/2000/svg">
+  <rect width="900" height="600" fill="transparent"/>
+  
+  <!-- Title -->
+  <text x="450" y="30" text-anchor="middle" font-family="Arial, sans-serif" font-size="18" font-weight="bold" fill="#e0e0e0">
+    Actions DSL: Defining Message Behaviors
+  </text>
+  
+  <!-- Action Declaration -->
+  <g>
+    <text x="450" y="70" text-anchor="middle" font-family="Arial, sans-serif" font-size="16" font-weight="bold" fill="#f6e05e">
+      Action Declaration and Implementation
+    </text>
+    
+    <rect x="50" y="90" width="800" height="140" rx="8" fill="#2d3748" stroke="#f6e05e" stroke-width="2"/>
+    <text x="450" y="115" text-anchor="middle" font-family="Arial, sans-serif" font-size="14" font-weight="bold" fill="#f6e05e">Complete Action Definition</text>
+    
+    <text x="70" y="140" font-family="monospace" font-size="12" fill="#4299e1">class OrderMessage extends BunnyFarm::Message</text>
+    <text x="80" y="160" font-family="monospace" font-size="12" fill="#68d391">fields</text>
+    <text x="140" y="160" font-family="monospace" font-size="12" fill="#e0e0e0">:order_id, :customer_id, :items, :total</text>
+    <text x="80" y="180" font-family="monospace" font-size="12" fill="#f6e05e">actions</text>
+    <text x="150" y="180" font-family="monospace" font-size="12" fill="#e0e0e0">:validate, :process, :ship, :cancel</text>
+    <text x="80" y="200" font-family="monospace" font-size="12" fill="#9f7aea">def validate</text>
+    <text x="90" y="220" font-family="monospace" font-size="12" fill="#e0e0e0">check_inventory and validate_payment</text>
+    <text x="80" y="240" font-family="monospace" font-size="12" fill="#9f7aea">end</text>
+    <text x="70" y="260" font-family="monospace" font-size="12" fill="#4299e1">end</text>
+  </g>
+  
+  <!-- Action Types -->
+  <g>
+    <text x="450" y="300" text-anchor="middle" font-family="Arial, sans-serif" font-size="16" font-weight="bold" fill="#e0e0e0">
+      Action Types and Patterns
+    </text>
+    
+    <!-- Simple Actions -->
+    <rect x="50" y="320" width="200" height="100" rx="6" fill="#2d3748" stroke="#68d391" stroke-width="2"/>
+    <text x="150" y="345" text-anchor="middle" font-family="Arial, sans-serif" font-size="13" font-weight="bold" fill="#68d391">Simple Actions</text>
+    <text x="60" y="365" font-family="monospace" font-size="10" fill="#e0e0e0">def send_email</text>
+    <text x="70" y="380" font-family="monospace" font-size="10" fill="#e0e0e0">EmailService.send(</text>
+    <text x="80" y="395" font-family="monospace" font-size="10" fill="#e0e0e0">self[:recipient],</text>
+    <text x="80" y="410" font-family="monospace" font-size="10" fill="#e0e0e0">self[:message])</text>
+    <text x="60" y="425" font-family="monospace" font-size="10" fill="#e0e0e0">end</text>
+    
+    <!-- Workflow Actions -->
+    <rect x="270" y="320" width="200" height="100" rx="6" fill="#2d3748" stroke="#4299e1" stroke-width="2"/>
+    <text x="370" y="345" text-anchor="middle" font-family="Arial, sans-serif" font-size="13" font-weight="bold" fill="#4299e1">Workflow Actions</text>
+    <text x="280" y="365" font-family="monospace" font-size="10" fill="#e0e0e0">def process_order</text>
+    <text x="290" y="380" font-family="monospace" font-size="10" fill="#e0e0e0">publish('validate')</text>
+    <text x="290" y="395" font-family="monospace" font-size="10" fill="#e0e0e0">publish('charge')</text>
+    <text x="290" y="410" font-family="monospace" font-size="10" fill="#e0e0e0">publish('fulfill')</text>
+    <text x="280" y="425" font-family="monospace" font-size="10" fill="#e0e0e0">end</text>
+    
+    <!-- Conditional Actions -->
+    <rect x="490" y="320" width="200" height="100" rx="6" fill="#2d3748" stroke="#9f7aea" stroke-width="2"/>
+    <text x="590" y="345" text-anchor="middle" font-family="Arial, sans-serif" font-size="13" font-weight="bold" fill="#9f7aea">Conditional Actions</text>
+    <text x="500" y="365" font-family="monospace" font-size="10" fill="#e0e0e0">def handle_payment</text>
+    <text x="510" y="380" font-family="monospace" font-size="10" fill="#e0e0e0">if premium_customer?</text>
+    <text x="520" y="395" font-family="monospace" font-size="10" fill="#e0e0e0">fast_checkout</text>
+    <text x="510" y="410" font-family="monospace" font-size="10" fill="#e0e0e0">else standard_flow</text>
+    <text x="500" y="425" font-family="monospace" font-size="10" fill="#e0e0e0">end</text>
+    
+    <!-- Error Handling Actions -->
+    <rect x="710" y="320" width="180" height="100" rx="6" fill="#2d3748" stroke="#e53e3e" stroke-width="2"/>
+    <text x="800" y="345" text-anchor="middle" font-family="Arial, sans-serif" font-size="13" font-weight="bold" fill="#e53e3e">Error Actions</text>
+    <text x="720" y="365" font-family="monospace" font-size="10" fill="#e0e0e0">def retry_payment</text>
+    <text x="730" y="380" font-family="monospace" font-size="10" fill="#e0e0e0">retry_count += 1</text>
+    <text x="730" y="395" font-family="monospace" font-size="10" fill="#e0e0e0">if retry_count > 3</text>
+    <text x="740" y="410" font-family="monospace" font-size="10" fill="#e0e0e0">publish('failed')</text>
+    <text x="720" y="425" font-family="monospace" font-size="10" fill="#e0e0e0">end</text>
+  </g>
+  
+  <!-- Action Execution Flow -->
+  <g>
+    <text x="450" y="470" text-anchor="middle" font-family="Arial, sans-serif" font-size="16" font-weight="bold" fill="#e0e0e0">
+      Action Execution and Routing
+    </text>
+    
+    <!-- Publishing -->
+    <rect x="50" y="490" width="160" height="60" rx="6" fill="#1a202c" stroke="#ff9800" stroke-width="1"/>
+    <text x="130" y="515" text-anchor="middle" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="#ff9800">1. Publish</text>
+    <text x="60" y="535" font-family="monospace" font-size="10" fill="#e0e0e0">message.publish('validate')</text>
+    
+    <!-- Routing -->
+    <rect x="240" y="490" width="160" height="60" rx="6" fill="#1a202c" stroke="#f6e05e" stroke-width="1"/>
+    <text x="320" y="515" text-anchor="middle" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="#f6e05e">2. Route</text>
+    <text x="250" y="535" font-family="monospace" font-size="10" fill="#e0e0e0">OrderMessage.validate</text>
+    
+    <!-- Consume -->
+    <rect x="430" y="490" width="160" height="60" rx="6" fill="#1a202c" stroke="#4299e1" stroke-width="1"/>
+    <text x="510" y="515" text-anchor="middle" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="#4299e1">3. Consume</text>
+    <text x="440" y="535" font-family="monospace" font-size="10" fill="#e0e0e0">Worker picks up message</text>
+    
+    <!-- Execute -->
+    <rect x="620" y="490" width="160" height="60" rx="6" fill="#1a202c" stroke="#68d391" stroke-width="1"/>
+    <text x="700" y="515" text-anchor="middle" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="#68d391">4. Execute</text>
+    <text x="630" y="535" font-family="monospace" font-size="10" fill="#e0e0e0">message.validate()</text>
+  </g>
+  
+  <!-- Flow arrows -->
+  <defs>
+    <marker id="arrow-action" markerWidth="8" markerHeight="6" refX="7" refY="3" orient="auto">
+      <polygon points="0 0, 8 3, 0 6" fill="#ff9800"/>
+    </marker>
+  </defs>
+  
+  <line x1="210" y1="520" x2="240" y2="520" stroke="#ff9800" stroke-width="2" marker-end="url(#arrow-action)"/>
+  <line x1="400" y1="520" x2="430" y2="520" stroke="#ff9800" stroke-width="2" marker-end="url(#arrow-action)"/>
+  <line x1="590" y1="520" x2="620" y2="520" stroke="#ff9800" stroke-width="2" marker-end="url(#arrow-action)"/>
+</svg>