smart_message 0.0.8 → 0.0.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e9fe12143456d1bc485554ad2d9f465a93e1d751860805bb2b5ee7496cccf031
-  data.tar.gz: bf4a18a9506120da4283a091e7ee6b921d02a01696c07bec92217883bea4e330
+  metadata.gz: 5f02a7e60919734663addbef7ae2b5582bc142993654ef33fe17c90e1a70d138
+  data.tar.gz: 50e7250d3e217faba23eb2a1d995b6edff0d9e0d7935bf2ea31602aff483a092
 SHA512:
-  metadata.gz: ad5f81928567ffdbc6a980b02777c2468804a7a4f248c4c7b34690942c356f69859f525aaef02b463a02b2dd1efe76e321194117e7730a28ac99257f501ee9a8
-  data.tar.gz: 2bdab0d12acb85994e1121ef1308386f776bf36e08a086102cebdd9d1279198b2c89d9a160250ab9dca32e648305abffe0b32c87d63c2c5b4ec2ee43d77787d1
+  metadata.gz: 7c157a60a127d4bd7a86742ccee8dff5385f11110d1449bc859742c497cc79d9a0e528dd39645d66a8e2b413602c31c4c79f73c9c560b5aacb37acc9610d3183
+  data.tar.gz: 31fc0e365c549326e9a9314f435d50715c4c980e7380311e19a7428638198d2ce499f8acd104381a631b2a40cd66d2deb8bb7fa3e16c59a61a443cc980698b56

data/.gitignore

@@ -1,3 +1,4 @@
+/log/
 /*.log
 /.bundle/
 /.yardoc

data/.irbrc

@@ -0,0 +1,24 @@
+# smart_message/.irbrc
+
+require_relative './lib/smart_message'
+
+SmartMessage.configure do |config|
+  config.logger = STDOUT
+  config.log_level = :info
+  config.log_format = :text
+  config.log_colorize = true
+end
+
+LOG = SmartMessage.configuration.default_logger
+
+class TheMessage < SmartMessage::Base
+  version 3
+  description "my simple message"
+
+  header do
+    from 'dewayne'
+    to 'madbomber'
+  end
+
+  property :data, required: true
+end

data/CHANGELOG.md

@@ -7,6 +7,102 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.0.9] 2025-08-20
+### Added
+- **Advanced Logging Configuration System**: Comprehensive logger configuration with multiple output formats and options
+  - New SmartMessage configuration options: `log_level`, `log_format`, `log_colorize`, `log_include_source`, `log_structured_data`, `log_options`
+  - Support for symbol-based log levels: `:info`, `:debug`, `:warn`, `:error`, `:fatal`
+  - Colorized console output with customizable color schemes using ANSI terminal colors
+  - File-based logging without colorization for clean log files
+  - JSON structured logging support with metadata inclusion
+  - Log rolling capabilities: size-based and date-based rolling with configurable retention
+  - Source location tracking for debugging (file, line number, method)
+  - Complete Lumberjack logger integration with SmartMessage configuration system
+  - New comprehensive logger demonstration: `examples/show_logger.rb`
+  - Application-level logger access through `SmartMessage.configuration.default_logger`
+- **Transport Layer Abstraction**: Complete refactoring from legacy Broker to modern Transport architecture
+  - New `SmartMessage::Transport` module providing pluggable message delivery backends
+  - Transport registry system for dynamic plugin discovery and registration
+  - Base transport class with standardized interface for all implementations
+  - Memory, STDOUT, and Redis transport implementations with production-ready features
+  - File-based transport support for inter-process communication
+- **Message Wrapper Architecture**: Unified message envelope system for cleaner dataflow
+  - New `SmartMessage::Wrapper::Base` class consolidating header and payload
+  - Simplified method signatures throughout message processing pipeline
+  - `_sm_` prefixed properties to avoid collision with user message definitions
+  - Support for different initialization patterns and automatic header generation
+- **HeaderDSL for Configuration**: Simplified message class setup with declarative syntax
+  - New `SmartMessage::Addressing::HeaderDSL` providing cleaner class-level configuration
+  - Declarative `from`, `to`, `reply_to` methods for entity addressing
+  - Automatic header field configuration without boilerplate code
+- **Advanced Message Filtering**: Regular expression support for flexible subscription patterns
+  - Regex pattern matching for `from:` and `to:` filters (e.g., `from: /^payment-.*/`)
+  - Array filters combining exact strings and patterns: `from: ['admin', /^system-.*/]`
+  - Environment-based routing patterns for dev/staging/production separation
+  - Service pattern routing for microservices architecture
+  - New example demonstrating regex filtering in microservices (`09_regex_filtering_microservices.rb`)
+- **Performance Benchmarking Tools**: Comprehensive performance measurement infrastructure
+  - New benchmark suite comparing thread pool and Ractor implementations
+  - JSON-based benchmark result storage with detailed metrics
+  - Benchmark comparison tool for analyzing performance across runs
+  - Ractor-based message processing implementation for parallel execution
+- **Improved Documentation Structure**: Complete documentation overhaul
+  - New modular documentation in `docs/` directory covering all major features
+  - Dedicated guides for transports, serializers, dispatcher, and filtering
+  - Architecture overview with component relationships
+  - Troubleshooting guide for common issues
+  - Example README with comprehensive usage patterns
+
+### Changed
+- **BREAKING: Transport API Migration**: Complete replacement of Broker with Transport
+  - All `broker` references must be updated to `transport`
+  - `SmartMessage::Broker` removed in favor of `SmartMessage::Transport`
+  - Transport implementations now in `lib/smart_message/transport/` directory
+  - Updated all examples and tests to use new transport terminology
+- **Simplified Module Structure**: Cleaner separation of concerns
+  - Extracted addressing logic into `SmartMessage::Addressing` module
+  - Separated messaging functionality into `SmartMessage::Messaging` module
+  - Created dedicated `SmartMessage::Plugins` module for plugin management
+  - Moved versioning logic to `SmartMessage::Versioning` module
+  - Utilities consolidated in `SmartMessage::Utilities` module
+- **Enhanced Error Messages**: More descriptive error handling throughout
+  - Improved header validation error messages with context
+  - Better transport registration error reporting
+  - Clearer serialization failure messages
+
+### Fixed
+- **Message Processing Simplification**: Removed unnecessary complexity in message wrapper
+  - Eliminated redundant processing steps in wrapper initialization
+  - Streamlined header and payload handling
+  - Fixed potential race conditions in concurrent message processing
+- **Header Method Handling**: Improved reliability of header field access
+  - Fixed edge cases in header method delegation
+  - Ensured consistent behavior across all header field operations
+  - Better error messages for invalid header operations
+
+### Enhanced
+- **Developer Experience**: Significant improvements to API usability
+  - More intuitive transport registration and discovery
+  - Cleaner message class definition with HeaderDSL
+  - Simplified subscription syntax with powerful filtering
+  - Better separation between framework internals and user code
+- **Testing Infrastructure**: Expanded test coverage
+  - New tests for transport layer abstraction
+  - Comprehensive filtering tests including regex patterns
+  - Performance benchmark test suite
+  - Proc handler integration tests
+- **Example Programs**: Enhanced educational value
+  - Added performance benchmarking examples
+  - New regex filtering demonstration for microservices
+  - Dead letter queue demonstration with retry scenarios
+  - Updated all examples to use modern transport architecture
+
+### Deprecated
+- **Broker System**: Legacy broker implementation marked for removal
+  - `SmartMessage::Broker` and all subclasses deprecated
+  - Migration path provided through transport layer
+  - Broker terminology replaced with transport throughout codebase
+
 ## [0.0.8] 2025-08-19
 
 ### Added

data/Gemfile.lock

@@ -1,12 +1,15 @@
 PATH
   remote: .
   specs:
-    smart_message (0.0.7)
+    smart_message (0.0.9)
       activesupport
       breaker_machines
+      colorize
       concurrent-ruby
       hashie
+      lumberjack
       redis
+      zeitwerk
 
 GEM
   remote: https://rubygems.org/
@@ -33,6 +36,7 @@ GEM
       concurrent-ruby (~> 1.3)
       state_machines (>= 0.50.0)
       zeitwerk (~> 2.7)
+    colorize (1.1.0)
     concurrent-ruby (1.3.5)
     connection_pool (2.5.3)
     debug_me (1.1.1)
@@ -41,6 +45,7 @@ GEM
     i18n (1.14.7)
       concurrent-ruby (~> 1.0)
     logger (1.7.0)
+    lumberjack (1.4.0)
     minitest (5.25.5)
     minitest-power_assert (0.3.1)
       minitest

data/README.md

@@ -10,15 +10,19 @@ SmartMessage is a message abstraction framework that decouples business logic fr
 - **Transport Abstraction**: Plugin architecture supporting multiple message transports (Redis, RabbitMQ, Kafka, etc.)
 - **Serialization Flexibility**: Pluggable serialization formats (JSON, MessagePack, etc.)
 - **Entity-to-Entity Addressing**: Built-in FROM/TO/REPLY_TO addressing for point-to-point and broadcast messaging patterns
+- **Advanced Message Filtering**: Filter subscriptions using exact strings, regular expressions, or mixed arrays for precise message routing
 - **Schema Versioning**: Built-in version management with automatic compatibility validation
 - **Comprehensive Validation**: Property validation with custom error messages and automatic validation before publishing
 - **Message Documentation**: Built-in documentation support for message classes and properties with automatic defaults
 - **Flexible Message Handlers**: Multiple subscription patterns - default methods, custom methods, blocks, procs, and lambdas
 - **Dual-Level Configuration**: Class and instance-level plugin overrides for gateway patterns
 - **Concurrent Processing**: Thread-safe message routing using `Concurrent::CachedThreadPool`
+- **Advanced Logging System**: Comprehensive logging with colorized console output, JSON structured logging, and file rolling
 - **Built-in Statistics**: Message processing metrics and monitoring
 - **Development Tools**: STDOUT and in-memory transports for testing
 - **Production Ready**: Redis transport with automatic reconnection and error handling
+- **Dead Letter Queue**: File-based DLQ with JSON Lines format for failed message capture and replay
+- **Circuit Breaker Integration**: Production-grade reliability with BreakerMachines for automatic fallback and recovery
 
 ## Installation
 
@@ -48,11 +52,18 @@ class OrderMessage < SmartMessage::Base
   # Add a description for the message class
   description "Represents customer order data for processing and fulfillment"
   
-  # Configure entity addressing
+  # Configure entity addressing (Method 1: Direct methods)
   from 'order-service'
   to 'fulfillment-service'  # Point-to-point message
   reply_to 'order-service'  # Responses come back here
   
+  # Alternative Method 2: Using header block
+  # header do
+  #   from 'order-service'
+  #   to 'fulfillment-service'
+  #   reply_to 'order-service'
+  # end
+  
   # Required properties with validation
   property :order_id, 
     required: true,
@@ -159,12 +170,43 @@ end
 OrderMessage.subscribe(audit_handler)
 ```
 
-### 4. Entity Addressing
+### 4. Message Filtering (NEW!)
 
-SmartMessage supports entity-to-entity addressing with FROM/TO/REPLY_TO fields for advanced message routing:
+SmartMessage supports powerful message filtering using exact strings, regular expressions, or arrays:
 
 ```ruby
-# Point-to-point messaging
+# Filter by exact sender
+OrderMessage.subscribe(from: 'payment-service')
+
+# Filter by sender pattern (all payment services)
+OrderMessage.subscribe(from: /^payment-.*/)
+
+# Filter by multiple senders
+OrderMessage.subscribe(from: ['admin', 'system', 'monitoring'])
+
+# Mixed exact and pattern matching
+OrderMessage.subscribe(from: ['admin', /^system-.*/, 'legacy-service'])
+
+# Filter by recipient patterns
+OrderMessage.subscribe(to: /^(dev|staging)-.*/)
+
+# Combined filtering
+OrderMessage.subscribe(
+  from: /^admin-.*/, 
+  to: ['order-service', /^fulfillment-.*/]
+)
+
+# Environment-based routing
+DevService.subscribe(to: /^(dev|staging)-.*/)
+ProdService.subscribe(to: /^prod-.*/)
+```
+
+### 5. Entity Addressing
+
+SmartMessage supports entity-to-entity addressing with FROM/TO/REPLY_TO fields for advanced message routing. You can configure addressing using three different approaches:
+
+#### Method 1: Direct Class Methods
+```ruby
 class PaymentMessage < SmartMessage::Base
   version 1
   from 'payment-service'     # Required: sender identity
@@ -174,26 +216,67 @@ class PaymentMessage < SmartMessage::Base
   property :amount, required: true
   property :account_id, required: true
 end
+```
 
-# Broadcast messaging (no 'to' specified)
-class SystemAnnouncementMessage < SmartMessage::Base
+#### Method 2: Header Block DSL
+```ruby
+class PaymentMessage < SmartMessage::Base
   version 1
-  from 'admin-service'       # Required: sender identity
-  # No 'to' field = broadcast to all subscribers
   
-  property :message, required: true
-  property :priority, default: 'normal'
+  # Configure all addressing in a single block
+  header do
+    from 'payment-service'
+    to 'bank-gateway'
+    reply_to 'payment-service'
+  end
+  
+  property :amount, required: true
+  property :account_id, required: true
 end
+```
+
+#### Method 3: Instance-Level Configuration
+```ruby
 
-# Instance-level addressing override
+# Create payment instance
 payment = PaymentMessage.new(amount: 100.00, account_id: "ACCT-123")
-payment.to('backup-gateway')  # Override destination for this instance
-payment.publish
 
-# Access addressing information
-puts payment._sm_header.from      # => 'payment-service'
+# Override addressing at instance level
+payment.to('backup-gateway')  # Method chaining supported
+payment.from('urgent-processor')
+
+# Alternative setter syntax
+payment.from = 'urgent-processor'
+payment.to = 'backup-gateway'
+
+# Access addressing (shortcut methods)
+puts payment.from      # => 'urgent-processor'
+puts payment.to        # => 'backup-gateway'
+puts payment.reply_to  # => 'payment-service'
+
+# Access via header (full path)
+puts payment._sm_header.from      # => 'urgent-processor'
 puts payment._sm_header.to        # => 'backup-gateway'
 puts payment._sm_header.reply_to  # => 'payment-service'
+
+# Publish with updated addressing
+payment.publish
+```
+
+#### Broadcast Messaging Example
+```ruby
+class SystemAnnouncementMessage < SmartMessage::Base
+  version 1
+  
+  # Using header block for broadcast configuration
+  header do
+    from 'admin-service'  # Required: sender identity
+    # No 'to' field = broadcast to all subscribers
+  end
+  
+  property :message, required: true
+  property :priority, default: 'normal'
+end
 ```
 
 #### Messaging Patterns Supported
@@ -203,6 +286,104 @@ puts payment._sm_header.reply_to  # => 'payment-service'
 - **Request-Reply**: Use `reply_to` field to specify response routing
 - **Gateway Patterns**: Override addressing at instance level for message forwarding
 
+## Logging Configuration
+
+SmartMessage includes a comprehensive logging system with support for multiple output formats, colorization, and file rolling capabilities.
+
+### Basic Logging Configuration
+
+```ruby
+# Configure SmartMessage logging
+SmartMessage.configure do |config|
+  config.logger = STDOUT              # Output destination (file path, STDOUT, STDERR)
+  config.log_level = :info           # Log level (:debug, :info, :warn, :error, :fatal)
+  config.log_format = :text          # Output format (:text, :json)
+  config.log_colorize = true         # Enable colorized console output
+  config.log_include_source = false  # Include source file/line information
+  config.log_structured_data = false # Enable structured data logging
+end
+
+# Access the configured logger in your application
+logger = SmartMessage.configuration.default_logger
+logger.info("Application started", component: "main", pid: Process.pid)
+```
+
+### Advanced Logging Features
+
+#### Colorized Console Output
+```ruby
+SmartMessage.configure do |config|
+  config.logger = STDOUT
+  config.log_colorize = true
+  config.log_format = :text
+end
+
+logger = SmartMessage.configuration.default_logger
+logger.debug("Debug message")    # Green background, white text
+logger.info("Info message")      # White text
+logger.warn("Warning message")   # Yellow background, white bold text
+logger.error("Error message")    # Light red background, white bold text
+logger.fatal("Fatal message")    # Light red background, yellow bold text
+```
+
+#### JSON Structured Logging
+```ruby
+SmartMessage.configure do |config|
+  config.logger = "log/application.log"
+  config.log_format = :json
+  config.log_structured_data = true
+  config.log_include_source = true
+end
+
+logger = SmartMessage.configuration.default_logger
+logger.info("User action", 
+            user_id: 12345, 
+            action: "login", 
+            ip_address: "192.168.1.1")
+# Output: {"timestamp":"2025-01-15T10:30:45.123Z","level":"INFO","message":"User action","user_id":12345,"action":"login","ip_address":"192.168.1.1","source":"app.rb:42:in `authenticate`"}
+```
+
+#### File Rolling Configuration
+```ruby
+SmartMessage.configure do |config|
+  config.logger = "log/application.log"
+  config.log_options = {
+    # Size-based rolling
+    roll_by_size: true,
+    max_file_size: 10 * 1024 * 1024,  # 10 MB
+    keep_files: 5,                     # Keep 5 old files
+    
+    # Date-based rolling (alternative to size-based)
+    roll_by_date: false,               # Set to true for date-based
+    date_pattern: '%Y-%m-%d'           # Daily rolling pattern
+  }
+end
+```
+
+### SmartMessage Integration
+
+SmartMessage classes automatically use the configured logger:
+
+```ruby
+class OrderMessage < SmartMessage::Base
+  property :order_id, required: true
+  property :amount, required: true
+  
+  def process
+    # Logger is automatically available
+    logger.info("Processing order", 
+                order_id: order_id, 
+                amount: amount,
+                header: _sm_header.to_h,
+                payload: _sm_payload)
+  end
+end
+
+# Messages inherit the global logger configuration
+message = OrderMessage.new(order_id: "123", amount: 99.99)
+message.publish  # Uses configured logger for any internal logging
+```
+
 ## Architecture
 
 ### Core Components
@@ -644,6 +825,99 @@ puts message._sm_header.to
 puts message._sm_header.reply_to
 ```
 
+### Dead Letter Queue
+
+SmartMessage includes a comprehensive file-based Dead Letter Queue system for handling failed messages:
+
+```ruby
+# Configure global DLQ (optional - defaults to 'dead_letters.jsonl')
+SmartMessage::DeadLetterQueue.configure_default('/var/log/app/dlq.jsonl')
+
+# Or use environment-based configuration
+SmartMessage::DeadLetterQueue.configure_default(
+  ENV.fetch('SMART_MESSAGE_DLQ_PATH', 'dead_letters.jsonl')
+)
+
+# Access the default DLQ instance
+dlq = SmartMessage::DeadLetterQueue.default
+
+# Create a custom DLQ instance for specific needs
+custom_dlq = SmartMessage::DeadLetterQueue.new('/tmp/critical_failures.jsonl')
+```
+
+#### DLQ Operations
+
+```ruby
+# Messages are automatically captured when circuit breakers trip
+# But you can also manually enqueue failed messages:
+dlq.enqueue(
+  message._sm_header,
+  message_payload,
+  error: "Connection timeout",
+  transport: "Redis",
+  retry_count: 3
+)
+
+# Inspect queue status
+puts "Queue size: #{dlq.size}"
+puts "Next message: #{dlq.peek}"  # Look without removing
+
+# Get statistics
+stats = dlq.statistics
+puts "Total messages: #{stats[:total]}"
+puts "By error type: #{stats[:by_error]}"
+puts "By message class: #{stats[:by_class]}"
+```
+
+#### Message Replay
+
+```ruby
+# Replay messages back through their original transport
+dlq.replay_one           # Replay oldest message
+dlq.replay_batch(10)      # Replay next 10 messages
+dlq.replay_all            # Replay entire queue
+
+# Replay with a different transport
+redis_transport = SmartMessage::Transport.create(:redis)
+dlq.replay_one(redis_transport)  # Override original transport
+```
+
+#### Administrative Functions
+
+```ruby
+# Filter messages for analysis
+failed_orders = dlq.filter_by_class('OrderMessage')
+timeout_errors = dlq.filter_by_error_pattern(/timeout/i)
+
+# Export messages within a time range
+yesterday = Time.now - 86400
+today = Time.now
+recent_failures = dlq.export_range(yesterday, today)
+
+# Clear the queue when needed
+dlq.clear  # Remove all messages
+```
+
+#### Integration with Circuit Breakers
+
+Dead Letter Queue is automatically integrated with circuit breakers:
+
+```ruby
+class PaymentMessage < SmartMessage::Base
+  config do
+    transport SmartMessage::Transport.create(:redis)
+    # Messages automatically go to DLQ when circuit breaker trips
+  end
+end
+
+# Monitor circuit breaker status
+transport = PaymentMessage.transport
+stats = transport.transport_circuit_stats
+if stats[:transport_publish][:open]
+  puts "Circuit open - messages going to DLQ"
+end
+```
+
 ## Development
 
 After checking out the repo, run:

data/docs/README.md

@@ -17,11 +17,13 @@ Welcome to the comprehensive documentation for SmartMessage, a Ruby gem that abs
 - [SmartMessage::Base](base.md)
 - [Property System](properties.md)
 - [Entity Addressing](addressing.md)
+- [Message Filtering](message_filtering.md)
 - [Transport Layer](transports.md)
 - [Serializers](serializers.md)
 - [Logging System](logging.md)
 - [Dispatcher & Routing](dispatcher.md)
 - [Message Headers](headers.md)
+- [Dead Letter Queue](dead_letter_queue.md)
 
 ### Advanced Topics
 - [Custom Transports](custom-transports.md)
@@ -50,6 +52,6 @@ Welcome to the comprehensive documentation for SmartMessage, a Ruby gem that abs
 
 ## Version
 
-This documentation is for SmartMessage v0.0.2.
+This documentation is for SmartMessage v0.0.8.
 
 For older versions, please check the git tags and corresponding documentation.