smart_message 0.0.12 → 0.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (117) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +1 -0
  3. data/CHANGELOG.md +155 -1
  4. data/Gemfile.lock +6 -6
  5. data/README.md +71 -25
  6. data/docs/core-concepts/architecture.md +5 -10
  7. data/docs/getting-started/examples.md +0 -12
  8. data/docs/getting-started/quick-start.md +4 -9
  9. data/docs/index.md +6 -4
  10. data/docs/reference/serializers.md +160 -488
  11. data/docs/reference/transports.md +47 -146
  12. data/docs/transports/memory-transport.md +2 -1
  13. data/docs/transports/multi-transport.md +484 -0
  14. data/docs/transports/redis-transport-comparison.md +215 -350
  15. data/docs/transports/redis-transport.md +3 -22
  16. data/examples/README.md +6 -9
  17. data/examples/city_scenario/README.md +1 -1
  18. data/examples/city_scenario/messages/emergency_911_message.rb +0 -1
  19. data/examples/city_scenario/messages/emergency_resolved_message.rb +0 -1
  20. data/examples/city_scenario/messages/fire_dispatch_message.rb +0 -1
  21. data/examples/city_scenario/messages/fire_emergency_message.rb +0 -1
  22. data/examples/city_scenario/messages/health_check_message.rb +0 -1
  23. data/examples/city_scenario/messages/health_status_message.rb +0 -1
  24. data/examples/city_scenario/messages/police_dispatch_message.rb +0 -1
  25. data/examples/city_scenario/messages/silent_alarm_message.rb +0 -1
  26. data/examples/file/00_run_all_file_demos.rb +260 -0
  27. data/examples/file/01_basic_file_transport_demo.rb +237 -0
  28. data/examples/file/02_fifo_transport_demo.rb +289 -0
  29. data/examples/file/03_file_watching_demo.rb +332 -0
  30. data/examples/file/04_multi_transport_file_demo.rb +432 -0
  31. data/examples/file/README.md +257 -0
  32. data/examples/memory/00_run_all_demos.rb +317 -0
  33. data/examples/memory/01_message_deduplication_demo.rb +18 -32
  34. data/examples/memory/02_dead_letter_queue_demo.rb +9 -12
  35. data/examples/memory/03_point_to_point_orders.rb +3 -5
  36. data/examples/memory/04_publish_subscribe_events.rb +15 -16
  37. data/examples/memory/05_many_to_many_chat.rb +19 -22
  38. data/examples/memory/06_stdout_publish_only.rb +145 -0
  39. data/examples/memory/07_proc_handlers_demo.rb +13 -14
  40. data/examples/memory/08_custom_logger_demo.rb +136 -140
  41. data/examples/memory/09_error_handling_demo.rb +7 -10
  42. data/examples/memory/10_entity_addressing_basic.rb +25 -31
  43. data/examples/memory/11_entity_addressing_with_filtering.rb +32 -36
  44. data/examples/memory/12_regex_filtering_microservices.rb +10 -11
  45. data/examples/memory/13_header_block_configuration.rb +0 -5
  46. data/examples/memory/14_global_configuration_demo.rb +12 -14
  47. data/examples/memory/15_logger_demo.rb +0 -1
  48. data/examples/memory/README.md +37 -20
  49. data/examples/memory/log/demo_app.log.1 +100 -0
  50. data/examples/memory/log/demo_app.log.2 +100 -0
  51. data/examples/multi_transport_example.rb +114 -0
  52. data/examples/redis/01_smart_home_iot_demo.rb +20 -24
  53. data/examples/redis/README.md +0 -2
  54. data/examples/utilities/box_it.rb +12 -0
  55. data/examples/utilities/doing.rb +19 -0
  56. data/examples/utilities/temp.md +28 -0
  57. data/lib/smart_message/base.rb +24 -17
  58. data/lib/smart_message/configuration.rb +2 -23
  59. data/lib/smart_message/dead_letter_queue.rb +1 -1
  60. data/lib/smart_message/errors.rb +3 -0
  61. data/lib/smart_message/header.rb +1 -1
  62. data/lib/smart_message/logger/default.rb +1 -1
  63. data/lib/smart_message/messaging.rb +37 -66
  64. data/lib/smart_message/plugins.rb +42 -41
  65. data/lib/smart_message/serializer/base.rb +1 -1
  66. data/lib/smart_message/serializer.rb +3 -2
  67. data/lib/smart_message/subscription.rb +18 -20
  68. data/lib/smart_message/transport/async_publish_queue.rb +284 -0
  69. data/lib/smart_message/transport/base.rb +42 -8
  70. data/lib/smart_message/transport/fifo_operations.rb +264 -0
  71. data/lib/smart_message/transport/file_operations.rb +200 -0
  72. data/lib/smart_message/transport/file_transport.rb +149 -0
  73. data/lib/smart_message/transport/file_watching.rb +72 -0
  74. data/lib/smart_message/transport/memory_transport.rb +23 -4
  75. data/lib/smart_message/transport/partitioned_files.rb +46 -0
  76. data/lib/smart_message/transport/redis_transport.rb +11 -0
  77. data/lib/smart_message/transport/registry.rb +0 -1
  78. data/lib/smart_message/transport/stdout_transport.rb +73 -41
  79. data/lib/smart_message/transport/stdout_transport.rb.backup +88 -0
  80. data/lib/smart_message/transport.rb +0 -1
  81. data/lib/smart_message/version.rb +1 -1
  82. metadata +25 -37
  83. data/docs/guides/redis-queue-getting-started.md +0 -697
  84. data/docs/guides/redis-queue-patterns.md +0 -889
  85. data/docs/guides/redis-queue-production.md +0 -1091
  86. data/docs/transports/redis-enhanced-transport.md +0 -524
  87. data/docs/transports/redis-queue-transport.md +0 -1304
  88. data/examples/redis_enhanced/README.md +0 -319
  89. data/examples/redis_enhanced/enhanced_01_basic_patterns.rb +0 -233
  90. data/examples/redis_enhanced/enhanced_02_fluent_api.rb +0 -331
  91. data/examples/redis_enhanced/enhanced_03_dual_publishing.rb +0 -281
  92. data/examples/redis_enhanced/enhanced_04_advanced_routing.rb +0 -419
  93. data/examples/redis_queue/01_basic_messaging.rb +0 -221
  94. data/examples/redis_queue/01_comprehensive_examples.rb +0 -508
  95. data/examples/redis_queue/02_pattern_routing.rb +0 -405
  96. data/examples/redis_queue/03_fluent_api.rb +0 -422
  97. data/examples/redis_queue/04_load_balancing.rb +0 -486
  98. data/examples/redis_queue/05_microservices.rb +0 -735
  99. data/examples/redis_queue/06_emergency_alerts.rb +0 -777
  100. data/examples/redis_queue/07_queue_management.rb +0 -587
  101. data/examples/redis_queue/README.md +0 -366
  102. data/examples/redis_queue/enhanced_01_basic_patterns.rb +0 -233
  103. data/examples/redis_queue/enhanced_02_fluent_api.rb +0 -331
  104. data/examples/redis_queue/enhanced_03_dual_publishing.rb +0 -281
  105. data/examples/redis_queue/enhanced_04_advanced_routing.rb +0 -419
  106. data/examples/redis_queue/redis_queue_architecture.svg +0 -148
  107. data/ideas/README.md +0 -41
  108. data/ideas/agents.md +0 -1001
  109. data/ideas/database_transport.md +0 -980
  110. data/ideas/improvement.md +0 -359
  111. data/ideas/meshage.md +0 -1788
  112. data/ideas/message_discovery.md +0 -178
  113. data/ideas/message_schema.md +0 -1381
  114. data/lib/smart_message/transport/redis_enhanced_transport.rb +0 -399
  115. data/lib/smart_message/transport/redis_queue_transport.rb +0 -555
  116. data/lib/smart_message/wrapper.rb.bak +0 -132
  117. /data/examples/memory/{06_pretty_print_demo.rb → 16_pretty_print_demo.rb} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2e7bd096698c7b656fc3800dcbbd88366dccb0ae36be0a9a1f50240c944a6de3
4
- data.tar.gz: 13235c27c12c4f76bdae091b0a432b505f0cfe49cd571fde387358aba98cf7da
3
+ metadata.gz: b0f06621d7e381b6069aba0fad1c329b941349e95488bdf5d8d39fd66e1d0fd2
4
+ data.tar.gz: d1ef64216d07f3e6005db87194590984004bf68f923c09bec93aadb889909cb1
5
5
  SHA512:
6
- metadata.gz: 2b9b74f2203c1c6cd95a981e4f7c89d9fb6e1aad112c99b61420521ce95e89b4f3c378936ba9f435f67ee548b88e2e2ea28d70216856eaf898c028f1058f5d04
7
- data.tar.gz: f7a9636706cb07e2b8df17e0e25b13128a2aed8a4ddd3d80ab28fadee3e27fc1ac52fafa9593f9854ffa54fb5920fe93334fb7f361c3394929c1fe4ea9be2adb
6
+ metadata.gz: f06e42c967e586521659b24e132001a64e51d64541214b06c7d5fa7bf6bfc48782a13bec11ace183ae50ff93b3840a5344ea2c59f3b44f0227fe8162b5452c44
7
+ data.tar.gz: bd2b874f6574600002a5b1cf845ca37076aef74f53fde34b7bae183af5c8978c5dd7c7fdb4d9f442014d7fb146371655e4f3092216e285fc7b0f5fba65dd254f
data/.gitignore CHANGED
@@ -13,3 +13,4 @@ site
13
13
  .aigcm_msg
14
14
  /log/
15
15
  /logs/
16
+ *.jsonl
data/CHANGELOG.md CHANGED
@@ -7,10 +7,164 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.0.16] 2025-09-10
11
+
12
+ ### Added
13
+ - **FileTransport Demo Suite**: Comprehensive demonstration programs for FileTransport functionality
14
+ - **Basic FileTransport Demo** (`examples/file/01_basic_file_transport_demo.rb`): Complete demonstration of core FileTransport features including different output formats (JSON, YAML, raw), append vs overwrite modes, custom serialization, and file rotation examples
15
+ - **FIFO Transport Demo** (`examples/file/02_fifo_transport_demo.rb`): FIFO (named pipe) demonstrations covering FIFO creation, properties, and concepts with educational content about inter-process communication
16
+ - **File Watching Demo** (`examples/file/03_file_watching_demo.rb`): File monitoring and change detection examples with polling-based file watching
17
+ - **Multi-Transport File Demo** (`examples/file/04_multi_transport_file_demo.rb`): Advanced patterns including fan-out to multiple files, conditional routing, multi-transport combinations, archival strategies, and performance monitoring
18
+ - **Interactive Demo Runner** (`examples/file/00_run_all_file_demos.rb`): Menu-driven demo launcher with comprehensive error handling and user-friendly navigation
19
+
20
+ ### Fixed
21
+ - **Demo Configuration Syntax**: Fixed incorrect `.configure` method usage throughout FileTransport demos
22
+ - **Issue**: Demos were using non-existent `MessageClass.configure` method causing "undefined method 'configure'" errors
23
+ - **Solution**: Updated all configuration calls to use `MessageClass.class_eval { transport transport_instance }` pattern consistent with working examples
24
+ - **Files Updated**: All FileTransport demo files now use correct transport configuration syntax
25
+ - **Message Instantiation Arguments**: Fixed keyword argument errors in message creation
26
+ - **Issue**: Hash variables being passed to `MessageClass.new(hash_var)` instead of using keyword argument expansion
27
+ - **Solution**: Updated all message instantiation calls to use `MessageClass.new(**hash_var)` for proper keyword argument passing
28
+ - **Impact**: Resolves "wrong number of arguments (given 1, expected 0)" errors throughout demo suite
29
+ - **Method Name Consistency**: Updated all demo files to use correct `publish` method instead of deprecated `send`
30
+ - **Issue**: Some demos were using `.send` method which is not the correct SmartMessage publishing method
31
+ - **Solution**: Replaced all instances of `.send` with `.publish` across FileTransport demos
32
+ - **Files Affected**: 02_fifo_transport_demo.rb, 04_multi_transport_file_demo.rb
33
+ - **Missing `from` Property Declarations**: Added required `from` declarations to all message classes
34
+ - **Issue**: Message classes missing required `from` property causing "The property 'from' From entity ID is required" errors
35
+ - **Solution**: Added `from 'demo_source'` declarations to all message class definitions including dynamically created classes
36
+ - **Impact**: Ensures proper message routing and header initialization
37
+ - **FIFO Blocking Issues**: Resolved program hanging when writing to FIFOs without concurrent readers
38
+ - **Issue**: FIFO operations block indefinitely when no reader process is available
39
+ - **Solution**: Simplified FIFO examples to demonstrate FIFO concepts and properties without blocking operations, replaced complex multi-process examples with educational demonstrations
40
+ - **Result**: Demos now complete successfully without hanging while still teaching FIFO fundamentals
41
+ - **Variable Scope Issues**: Fixed transport variable accessibility problems in custom routing logic
42
+ - **Issue**: Complex custom routing classes couldn't access transport variables from outer scope
43
+ - **Solution**: Simplified conditional routing examples to use separate message classes for different severity levels instead of dynamic transport switching
44
+ - **Benefit**: Cleaner, more maintainable code that demonstrates routing concepts without scope complications
45
+
46
+ ### Enhanced
47
+ - **Demo Educational Value**: Improved all FileTransport demos with comprehensive documentation and error handling
48
+ - **Documentation**: Added detailed comments explaining FileTransport concepts, FIFO characteristics, and usage patterns
49
+ - **Error Handling**: Enhanced error reporting and graceful degradation in all demo scenarios
50
+ - **User Experience**: Simplified complex examples to focus on core functionality while maintaining educational value
51
+ - **Output Clarity**: Improved console output formatting and messaging for better understanding of demo operations
52
+
53
+ ### Documentation
54
+ - **FileTransport Demo README**: Comprehensive documentation for all FileTransport demonstration programs
55
+ - **Usage Instructions**: Step-by-step guide for running individual demos and the interactive demo runner
56
+ - **Concept Explanations**: Detailed explanations of FileTransport features, FIFO operations, and advanced patterns
57
+ - **Troubleshooting**: Common issues and solutions for FileTransport usage
58
+ - **Example Outputs**: Sample console outputs showing expected demo results
59
+
60
+ ## [0.0.15] 2025-09-10
61
+
62
+ ### Added
63
+ - **Multi-Transport Publishing**: Messages can now be configured to publish to multiple transports simultaneously
64
+ - **Core Feature**: Configure messages with an array of transports using `transport [transport1, transport2, transport3]` syntax
65
+ - **Resilient Publishing**: Publishing succeeds if ANY configured transport works; only fails if ALL transports fail
66
+ - **Error Handling**: Individual transport failures are logged but don't prevent publishing to remaining transports
67
+ - **Backward Compatibility**: Single transport configuration continues to work unchanged (`transport single_transport`)
68
+ - **Utility Methods**: Added `transports()`, `single_transport?()`, and `multiple_transports?()` for transport introspection
69
+ - **Use Cases**: Supports redundancy, integration, migration, and fan-out messaging patterns
70
+
71
+ ### Enhanced
72
+ - **Transport Configuration API**: Extended transport configuration methods to handle both single transports and transport arrays
73
+ - **Plugins Module**: Updated `transport()` method in `lib/smart_message/plugins.rb` to accept arrays while maintaining backward compatibility
74
+ - **Internal Storage**: Transport arrays stored internally but `transport()` getter returns first transport for existing code compatibility
75
+ - **Instance Overrides**: Instance-level transport configuration can override class-level multi-transport settings
76
+ - **Publishing Pipeline**: Enhanced message publishing in `lib/smart_message/messaging.rb` for multi-transport support
77
+ - **Parallel Publishing**: Iterates through all configured transports and publishes to each
78
+ - **Comprehensive Logging**: Logs successful and failed transports with detailed error information
79
+ - **Error Aggregation**: Collects all transport errors and raises `PublishError` only when all transports fail
80
+ - **Success Tracking**: Continues processing even when individual transports fail
81
+
82
+ ### Added - Error Handling
83
+ - **PublishError**: New error class `SmartMessage::Errors::PublishError` for multi-transport failure scenarios
84
+ - **Condition**: Only raised when ALL configured transports fail to publish
85
+ - **Error Detail**: Aggregates error messages from all failed transports for comprehensive debugging
86
+ - **Added to**: `lib/smart_message/errors.rb`
87
+
88
+ ### Added - Documentation
89
+ - **Comprehensive Guide**: Created `docs/transports/multi-transport.md` with complete multi-transport documentation
90
+ - **Real-World Examples**: High-availability, development/production dual publishing, monitoring integration, A/B testing
91
+ - **Best Practices**: Performance considerations, environment-specific configuration, health monitoring
92
+ - **Troubleshooting**: Common issues, debugging techniques, and solutions
93
+ - **Integration Updates**: Updated main transport documentation and index to reference multi-transport capability
94
+ - **Transport Layer**: Added multi-transport section to `docs/reference/transports.md`
95
+ - **Table of Contents**: Added multi-transport link to main documentation index
96
+
97
+ ### Added - Testing
98
+ - **Comprehensive Test Suite**: Created `test/multi_transport_test.rb` with 12 test cases covering all functionality
99
+ - **Backward Compatibility**: Verified single transport behavior remains unchanged
100
+ - **Multi-Transport Scenarios**: Tests for successful publishing, partial failures, complete failures
101
+ - **Configuration Testing**: Class vs instance configuration, transport introspection, method chaining
102
+ - **Mock Transports**: Added `FailingTransport` and `CountingTransport` for controlled testing
103
+ - **Integration Tests**: Real transport combinations (Memory + STDOUT)
104
+
105
+ ### Added - Examples
106
+ - **README Integration**: Added multi-transport example to main README.md with practical use case
107
+ - **Features Section**: Added multi-transport publishing bullet point to feature list
108
+ - **Transport Implementations**: Complete section with `CriticalOrderMessage` example showing 3 transports
109
+ - **Key Benefits**: Highlighted redundancy, integration, migration, and resilience benefits
110
+ - **Example File**: Created `examples/multi_transport_example.rb` demonstrating all multi-transport functionality
111
+ - **Usage Patterns**: Single transport (backward compatibility), multiple transports, instance overrides
112
+ - **Error Scenarios**: Partial transport failure resilience and complete failure handling
113
+ - **Utility Demonstrations**: Transport counting, type checking, and configuration inspection
114
+
115
+ ## [0.0.14] 2025-09-10
116
+
117
+ ### Fixed
118
+ - **Configuration Architecture**: Fixed `SmartMessage::Serializer.default` method to properly reflect transport-owned serialization architecture
119
+ - **Issue**: `SmartMessage::Serializer.default` was trying to access a non-existent `default_serializer` method on global configuration
120
+ - **Root Cause**: Serialization belongs to transports, not global configuration, but the method was attempting to check global config first
121
+ - **Solution**: Updated method to return framework default (`SmartMessage::Serializer::Json`) directly without depending on global configuration
122
+ - **Architecture Clarification**: Each transport owns and configures its own serializer (line 19 in transport/base.rb: `@serializer = options[:serializer] || default_serializer`)
123
+ - **Demo Fix**: Updated `examples/memory/14_global_configuration_demo.rb` to remove incorrect `config.serializer =` attempts and added architecture comments
124
+ - **Test Method Signatures**: Completed wrapper-to-message architecture transition by fixing remaining method signature issues
125
+ - **dispatcher_test.rb**: Changed `processer_one` and `processer_two` from instance methods to class methods (`def self.`)
126
+ - **Message Format**: Updated methods to use flat message structure (`message._sm_header.message_class` and `message.to_json`) instead of old two-tier format (`message_header, encoded_message = message`)
127
+ - **subscribe_test.rb**: Fixed `business_logic` method to be a class method (`def self.business_logic`) to match subscription call pattern
128
+ - **Result**: All tests now pass (`176 runs, 578 assertions, 0 failures, 0 errors, 1 skips`) completing the architectural transition
129
+
130
+ ## [0.0.13] 2025-09-10
131
+
132
+ ### Changed
133
+ - **BREAKING: Transport-Level Serialization Architecture**: Major architectural rework moving serializer responsibility from message classes to transport layer
134
+ - **Core Change**: Serializers now associated with transports instead of message classes, enabling different transports to use different serialization formats
135
+ - **Message Structure**: Eliminated wrapper objects in favor of flat message structure with `_sm_header` routing metadata
136
+ - **Serialization Format**: Messages now serialized as single JSON objects containing both header and payload data (e.g., `{"_sm_header": {...}, "content": "value", "property": "data"}`)
137
+ - **Transport Flexibility**: Each transport can now use its own serializer (JSON, MessagePack, etc.) independently
138
+ - **Backward Compatibility**: Added `_sm_payload` and `split` methods to maintain compatibility with existing proc handlers
139
+ - **Performance**: Simplified message processing pipeline with direct message instance passing instead of complex wrapper handling
140
+
141
+ ### Fixed
142
+ - **Message Reconstruction**: Fixed critical bug in `SmartMessage::Base#initialize` where payload properties were not properly extracted from flat serialized structure
143
+ - **Root Cause**: Initialization logic looked for `props[:_sm_payload]` but flat structure stores properties directly in `props`
144
+ - **Solution**: Changed to `payload_props = props.except(:_sm_header)` to extract all non-header properties
145
+ - **Impact**: Proc handlers now receive correct message content, resolving nil property values in `_sm_payload`
146
+ - **Circuit Breaker Test Compatibility**: Updated circuit breaker tests to use new transport method signatures
147
+ - **Issue**: Tests were calling `transport.publish(header, payload)` with two arguments
148
+ - **Fix**: Updated to `transport.publish(message_instance)` with single message parameter
149
+ - **Result**: All circuit breaker tests now pass with new architecture
150
+ - **Deduplication Test Format**: Updated deduplication tests from old wrapper format to new flat structure
151
+ - **Changed**: `_sm_payload: { content: "value" }` → `content: "value"`
152
+ - **Maintained**: Full deduplication functionality with handler-scoped DDQ system
153
+
154
+ ### Enhanced
155
+ - **Transport Serialization Control**: Each transport implementation can now specify its preferred serializer
156
+ - **Memory Transport**: Returns message objects directly without serialization for performance
157
+ - **STDOUT Transport**: Uses JSON serializer with pretty printing for human readability
158
+ - **Redis Transport**: Uses MessagePack with JSON fallback for efficient network transmission
159
+ - **Message Processing Pipeline**: Streamlined architecture with direct message instance routing
160
+ - **Eliminated**: Complex wrapper object creation and management overhead
161
+ - **Simplified**: Transport `receive` method now directly reconstructs message instances from flat data
162
+ - **Improved**: Cleaner separation between transport concerns and message processing logic
163
+
10
164
  ### Added
11
165
  - **Emergency Services Multi-Program Demo**: Complete emergency dispatch simulation system
12
166
  - Added comprehensive emergency services message definitions with validation
13
- - Emergency Dispatch Center (911) routing calls to appropriate departments
167
+ - Emergency Dispatch Center (911) routing calls to appropriate departments
14
168
  - Fire Department, Police Department, and Health Department service implementations
15
169
  - AI-powered visitor generating realistic emergency scenarios with retry logic
16
170
  - Health monitoring system across all city services with status reporting
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- smart_message (0.0.12)
4
+ smart_message (0.0.16)
5
5
  activesupport
6
6
  async
7
7
  async-redis
@@ -30,7 +30,7 @@ GEM
30
30
  tzinfo (~> 2.0, >= 2.0.5)
31
31
  uri (>= 0.13.1)
32
32
  amazing_print (1.8.1)
33
- async (2.29.0)
33
+ async (2.31.0)
34
34
  console (~> 1.29)
35
35
  fiber-annotation
36
36
  io-event (~> 1.11)
@@ -69,12 +69,12 @@ GEM
69
69
  i18n (1.14.7)
70
70
  concurrent-ruby (~> 1.0)
71
71
  io-endpoint (0.15.2)
72
- io-event (1.13.0)
72
+ io-event (1.14.0)
73
73
  io-stream (0.10.0)
74
74
  json (2.13.2)
75
75
  logger (1.7.0)
76
76
  lumberjack (1.4.1)
77
- metrics (0.14.0)
77
+ metrics (0.15.0)
78
78
  minitest (5.25.5)
79
79
  minitest-power_assert (0.3.1)
80
80
  minitest
@@ -85,7 +85,7 @@ GEM
85
85
  rake (13.3.0)
86
86
  redis (5.4.1)
87
87
  redis-client (>= 0.22.0)
88
- redis-client (0.25.2)
88
+ redis-client (0.25.3)
89
89
  connection_pool
90
90
  securerandom (0.4.1)
91
91
  shoulda (4.0.0)
@@ -95,7 +95,7 @@ GEM
95
95
  shoulda-matchers (4.5.1)
96
96
  activesupport (>= 4.2.0)
97
97
  state_machines (0.100.1)
98
- traces (0.18.1)
98
+ traces (0.18.2)
99
99
  tzinfo (2.0.6)
100
100
  concurrent-ruby (~> 1.0)
101
101
  uri (1.0.3)
data/README.md CHANGED
@@ -23,6 +23,7 @@ Think of SmartMessage as ActiveRecord for messaging - just as ActiveRecord frees
23
23
  ## Features
24
24
 
25
25
  - **Transport Abstraction**: Plugin architecture supporting multiple message transports (Redis, RabbitMQ, Kafka, etc.)
26
+ - **Multi-Transport Publishing**: Send messages to multiple transports simultaneously for redundancy, integration, and migration scenarios
26
27
  - **🌟 Redis Queue Transport**: Advanced transport with RabbitMQ-style routing patterns, persistent FIFO queues, load balancing, and 10x faster performance than traditional message brokers. Built on Ruby's Async framework for fiber-based concurrency supporting thousands of concurrent subscriptions - [see full documentation](docs/transports/redis-queue.md)
27
28
  - **Serialization Flexibility**: Pluggable serialization formats (JSON, MessagePack, etc.)
28
29
  - **Entity-to-Entity Addressing**: Built-in FROM/TO/REPLY_TO addressing for point-to-point and broadcast messaging patterns
@@ -36,7 +37,7 @@ Think of SmartMessage as ActiveRecord for messaging - just as ActiveRecord frees
36
37
  - **Advanced Logging System**: Comprehensive logging with colorized console output, JSON structured logging, and file rolling
37
38
  - **Built-in Statistics**: Message processing metrics and monitoring
38
39
  - **Message Deduplication**: Handler-scoped deduplication queues (DDQ) with memory or Redis storage for preventing duplicate message processing
39
- - **Development Tools**: STDOUT and in-memory transports for testing
40
+ - **Development Tools**: STDOUT transport for publish-only scenarios and in-memory transport for testing with local processing
40
41
  - **Production Ready**: Redis transport with automatic reconnection and error handling
41
42
  - **Dead Letter Queue**: File-based DLQ with JSON Lines format for failed message capture and replay
42
43
  - **Circuit Breaker Integration**: Production-grade reliability with BreakerMachines for automatic fallback and recovery
@@ -131,8 +132,8 @@ class OrderMessage < SmartMessage::Base
131
132
 
132
133
  # Configure transport and serializer at class level
133
134
  config do
134
- # Option 1: Simple STDOUT for development
135
- transport SmartMessage::Transport.create(:stdout, loopback: true)
135
+ # Option 1: Memory transport for local development with message processing
136
+ transport SmartMessage::Transport::MemoryTransport.new
136
137
 
137
138
  # Option 2: Redis Queue for production (10x faster than RabbitMQ!)
138
139
  # transport SmartMessage::Transport.create(:redis_queue,
@@ -196,24 +197,19 @@ OrderMessage.subscribe
196
197
  OrderMessage.subscribe("PaymentService.process_order")
197
198
 
198
199
  # 3. Block handler (NEW!)
199
- OrderMessage.subscribe do |wrapper|
200
- header, payload = wrapper.split
201
- order_data = JSON.parse(payload)
202
- puts "Quick processing: Order #{order_data['order_id']}"
200
+ OrderMessage.subscribe do |message|
201
+ puts "Quick processing: Order #{message.order_id}"
203
202
  end
204
203
 
205
204
  # 4. Proc handler (NEW!)
206
- order_processor = proc do |wrapper|
207
- header, payload = wrapper.split
208
- order_data = JSON.parse(payload)
209
- EmailService.send_confirmation(order_data['customer_id'])
205
+ order_processor = proc do |message|
206
+ EmailService.send_confirmation(message.customer_id)
210
207
  end
211
208
  OrderMessage.subscribe(order_processor)
212
209
 
213
210
  # 5. Lambda handler (NEW!)
214
- audit_handler = lambda do |wrapper|
215
- header, payload = wrapper.split
216
- AuditLog.record("Order processed at #{header.published_at}")
211
+ audit_handler = lambda do |message|
212
+ AuditLog.record("Order processed at #{message._sm_header.published_at}")
217
213
  end
218
214
  OrderMessage.subscribe(audit_handler)
219
215
  ```
@@ -542,8 +538,8 @@ The foundation class that all messages inherit from. Built on `Hashie::Dash` wit
542
538
  #### Transport Layer
543
539
  Pluggable message delivery system with built-in implementations:
544
540
 
545
- - **StdoutTransport**: Development and testing transport
546
- - **MemoryTransport**: In-memory queuing for testing
541
+ - **StdoutTransport**: Publish-only transport for debugging and external integration
542
+ - **MemoryTransport**: In-memory queuing for testing with local message processing
547
543
  - **RedisTransport**: Redis pub/sub transport for production messaging
548
544
  - **Custom Transports**: Implement `SmartMessage::Transport::Base`
549
545
 
@@ -586,6 +582,38 @@ This enables gateway patterns where messages can be received from one transport/
586
582
 
587
583
  ## Transport Implementations
588
584
 
585
+ ### Multi-Transport Messages
586
+
587
+ SmartMessage supports publishing to multiple transports simultaneously for redundancy, integration, and migration scenarios:
588
+
589
+ ```ruby
590
+ class CriticalOrderMessage < SmartMessage::Base
591
+ property :order_id, required: true
592
+ property :amount, required: true
593
+
594
+ # Configure multiple transports - message goes to ALL of them
595
+ transport [
596
+ SmartMessage::Transport.create(:redis_queue, url: 'redis://primary:6379'),
597
+ SmartMessage::Transport.create(:redis, url: 'redis://backup:6379'),
598
+ SmartMessage::Transport::StdoutTransport.new(format: :json)
599
+ ]
600
+ end
601
+
602
+ # Publishes to Redis Queue, Redis Pub/Sub, and STDOUT simultaneously
603
+ message = CriticalOrderMessage.new(order_id: "ORD-123", amount: 99.99)
604
+ message.publish # ✅ Succeeds if ANY transport works
605
+
606
+ # Check transport configuration
607
+ puts message.multiple_transports? # => true
608
+ puts message.transports.length # => 3
609
+ ```
610
+
611
+ **Key Benefits:**
612
+ - **Redundancy**: Messages reach multiple destinations for reliability
613
+ - **Integration**: Simultaneously log to STDOUT, send to Redis, and webhook external systems
614
+ - **Migration**: Gradually transition between transport systems
615
+ - **Resilient**: Publishing succeeds as long as ANY transport works
616
+
589
617
  ### Redis Queue Transport (Featured) 🌟
590
618
 
591
619
  The Redis Queue Transport provides enterprise-grade message routing with exceptional performance:
@@ -637,17 +665,35 @@ OrderMessage.new(
637
665
 
638
666
  📚 **Full Documentation:** [Redis Queue Transport Guide](docs/transports/redis-queue.md) | [Getting Started](docs/guides/redis-queue-getting-started.md) | [Examples](examples/redis_queue/)
639
667
 
640
- ### STDOUT Transport (Development)
668
+ ### STDOUT Transport (Publish-Only)
669
+
670
+ The STDOUT transport is designed for publish-only scenarios - perfect for debugging, logging, or integration with external systems.
641
671
 
642
672
  ```ruby
643
- # Basic STDOUT output
644
- transport = SmartMessage::Transport.create(:stdout)
673
+ # Basic STDOUT output (publish-only)
674
+ transport = SmartMessage::Transport::StdoutTransport.new
675
+
676
+ # Pretty-printed format for human reading (default)
677
+ transport = SmartMessage::Transport::StdoutTransport.new(format: :pretty)
645
678
 
646
- # With loopback for testing subscriptions
647
- transport = SmartMessage::Transport.create(:stdout, loopback: true)
679
+ # JSON format for machine processing
680
+ transport = SmartMessage::Transport::StdoutTransport.new(format: :json)
648
681
 
649
- # Output to file
650
- transport = SmartMessage::Transport.create(:stdout, output: "messages.log")
682
+ # Output to file instead of STDOUT
683
+ transport = SmartMessage::Transport::StdoutTransport.new(output: "messages.log")
684
+ ```
685
+
686
+ **Key Features:**
687
+ - **Publish-only**: No message processing or loopback
688
+ - **Subscription attempts are ignored** with warning logs
689
+ - **Two formats**: `:pretty` for debugging, `:json` for integration
690
+ - **Perfect for**: debugging, logging, piping to external tools
691
+ - **Use cases**: `./app | jq`, `./app | fluentd`, development monitoring
692
+
693
+ **For local message processing, use MemoryTransport instead:**
694
+ ```ruby
695
+ # Use Memory transport if you need local message processing
696
+ transport = SmartMessage::Transport::MemoryTransport.new
651
697
  ```
652
698
 
653
699
  ### Memory Transport (Testing)
@@ -952,8 +998,8 @@ You can also set descriptions within configuration blocks:
952
998
  class ConfiguredMessage < SmartMessage::Base
953
999
  config do
954
1000
  description "Set within config block"
955
- transport SmartMessage::Transport.create(:stdout)
956
- serializer SmartMessage::Serializer::JSON.new
1001
+ transport SmartMessage::Transport::MemoryTransport.new
1002
+ serializer SmartMessage::Serializer::Json.new
957
1003
  end
958
1004
  end
959
1005
  ```
@@ -9,10 +9,10 @@ SmartMessage is designed around the principle that **messages should be independ
9
9
  ### Core Principles
10
10
 
11
11
  1. **Separation of Concerns**: Message content, transport, and serialization are independent
12
- 2. **Plugin Architecture**: Pluggable transports and serializers
12
+ 2. **Plugin Architecture**: Pluggable transports
13
13
  3. **Dual Configuration**: Both class-level and instance-level configuration
14
14
  4. **Thread Safety**: Concurrent message processing with thread pools
15
- 5. **Gateway Support**: Messages can flow between different transports/serializers
15
+ 5. **Gateway Support**: Messages can flow between different transports
16
16
 
17
17
  ## Architecture Overview
18
18
 
@@ -26,7 +26,7 @@ The foundation class that all messages inherit from, built on `Hashie::Dash`.
26
26
 
27
27
  **Key Responsibilities:**
28
28
  - Property management and validation
29
- - Plugin configuration (transport, serializer, logger)
29
+ - Plugin configuration (transport, logger)
30
30
  - Message lifecycle management
31
31
  - Header generation and management
32
32
 
@@ -40,7 +40,6 @@ class MyMessage < SmartMessage::Base
40
40
 
41
41
  config do
42
42
  transport MyTransport.new
43
- serializer MySerializer.new
44
43
  end
45
44
  end
46
45
  ```
@@ -163,7 +162,6 @@ puts header.from # "payment-service"
163
162
  puts header.to # "order-service"
164
163
  puts header.reply_to # "payment-service" (defaults to from)
165
164
  puts header.version # 1
166
- puts header.serializer # "SmartMessage::Serializer::JSON"
167
165
  ```
168
166
 
169
167
  ## Message Lifecycle
@@ -176,7 +174,6 @@ class OrderMessage < SmartMessage::Base
176
174
 
177
175
  config do
178
176
  transport SmartMessage::Transport.create(:memory)
179
- serializer SmartMessage::Serializer::Json.new
180
177
  end
181
178
  end
182
179
  ```
@@ -201,7 +198,7 @@ order = OrderMessage.new(order_id: "123", amount: 99.99)
201
198
  order.from("order-service").to("payment-service")
202
199
  order.publish
203
200
  # 1. Creates header with UUID, timestamp, addressing
204
- # 2. Encodes message via serializer
201
+ # 2. Encodes message via transport's serializer
205
202
  # 3. Sends via transport
206
203
  # 4. Circuit breaker monitors for failures
207
204
  ```
@@ -210,7 +207,7 @@ order.publish
210
207
  ```ruby
211
208
  # Transport receives serialized message
212
209
  transport.receive(message_class, serialized_message)
213
- # 1. Decodes message using class's configured serializer
210
+ # 1. Decodes message using transport's serializer
214
211
  # 2. Routes decoded message to dispatcher
215
212
  # 3. Dispatcher checks DDQ for duplicates per handler
216
213
  # 4. Applies message filters (from/to/broadcast)
@@ -271,7 +268,6 @@ SmartMessage supports configuration at both class and instance levels:
271
268
  class PaymentMessage < SmartMessage::Base
272
269
  config do
273
270
  transport ProductionTransport.new
274
- serializer SecureSerializer.new
275
271
  end
276
272
  end
277
273
 
@@ -459,7 +455,6 @@ Configuration uses method-based DSL:
459
455
  ```ruby
460
456
  config do
461
457
  transport MyTransport.new(option1: value1)
462
- serializer MySerializer.new(option2: value2)
463
458
  logger MyLogger.new(level: :debug)
464
459
  end
465
460
  ```
@@ -20,7 +20,6 @@ class NotificationMessage < SmartMessage::Base
20
20
 
21
21
  config do
22
22
  transport SmartMessage::Transport.create(:stdout, loopback: true)
23
- serializer SmartMessage::Serializer::Json.new
24
23
  end
25
24
 
26
25
  def self.process(decoded_message)
@@ -87,7 +86,6 @@ class UserRegisteredEvent < SmartMessage::Base
87
86
 
88
87
  config do
89
88
  transport SmartMessage::Transport.create(:memory, auto_process: true)
90
- serializer SmartMessage::Serializer::Json.new
91
89
  end
92
90
 
93
91
  def self.process(decoded_message)
@@ -152,7 +150,6 @@ class WelcomeEmailMessage < SmartMessage::Base
152
150
 
153
151
  config do
154
152
  transport SmartMessage::Transport.create(:stdout)
155
- serializer SmartMessage::Serializer::Json.new
156
153
  end
157
154
 
158
155
  def self.process(decoded_message)
@@ -171,7 +168,6 @@ class AnalyticsMessage < SmartMessage::Base
171
168
 
172
169
  config do
173
170
  transport SmartMessage::Transport.create(:stdout)
174
- serializer SmartMessage::Serializer::Json.new
175
171
  end
176
172
 
177
173
  def self.process(decoded_message)
@@ -209,7 +205,6 @@ class OrderCreatedMessage < SmartMessage::Base
209
205
 
210
206
  config do
211
207
  transport SmartMessage::Transport.create(:memory, auto_process: true)
212
- serializer SmartMessage::Serializer::Json.new
213
208
  end
214
209
 
215
210
  def self.process(decoded_message)
@@ -252,7 +247,6 @@ class InventoryReservationMessage < SmartMessage::Base
252
247
 
253
248
  config do
254
249
  transport SmartMessage::Transport.create(:memory, auto_process: true)
255
- serializer SmartMessage::Serializer::Json.new
256
250
  end
257
251
 
258
252
  def self.process(decoded_message)
@@ -289,7 +283,6 @@ class PaymentProcessingMessage < SmartMessage::Base
289
283
 
290
284
  config do
291
285
  transport SmartMessage::Transport.create(:memory, auto_process: true)
292
- serializer SmartMessage::Serializer::Json.new
293
286
  end
294
287
 
295
288
  def self.process(decoded_message)
@@ -362,7 +355,6 @@ class LogMessage < SmartMessage::Base
362
355
 
363
356
  config do
364
357
  transport SmartMessage::Transport.create(:stdout, output: "application.log")
365
- serializer SmartMessage::Serializer::Json.new
366
358
  end
367
359
 
368
360
  def self.process(decoded_message)
@@ -412,7 +404,6 @@ class MetricMessage < SmartMessage::Base
412
404
 
413
405
  config do
414
406
  transport SmartMessage::Transport.create(:memory, auto_process: true)
415
- serializer SmartMessage::Serializer::Json.new
416
407
  end
417
408
 
418
409
  def self.process(decoded_message)
@@ -496,7 +487,6 @@ class MessageGateway < SmartMessage::Base
496
487
  # Receive from one transport
497
488
  config do
498
489
  transport SmartMessage::Transport.create(:memory, auto_process: true)
499
- serializer SmartMessage::Serializer::Json.new
500
490
  end
501
491
 
502
492
  def self.process(decoded_message)
@@ -621,7 +611,6 @@ class ResilientMessage < SmartMessage::Base
621
611
 
622
612
  config do
623
613
  transport SmartMessage::Transport.create(:memory, auto_process: true)
624
- serializer SmartMessage::Serializer::Json.new
625
614
  end
626
615
 
627
616
  def self.process(decoded_message)
@@ -689,7 +678,6 @@ class DeadLetterMessage < SmartMessage::Base
689
678
 
690
679
  config do
691
680
  transport SmartMessage::Transport.create(:stdout, output: "dead_letter_queue.log")
692
- serializer SmartMessage::Serializer::Json.new
693
681
  end
694
682
 
695
683
  def self.process(decoded_message)
@@ -92,7 +92,6 @@ class WelcomeMessage < SmartMessage::Base
92
92
  # Configure the transport (where messages go)
93
93
  config do
94
94
  transport SmartMessage::Transport.create(:stdout, loopback: true)
95
- serializer SmartMessage::Serializer::Json.new
96
95
  end
97
96
 
98
97
  # Define how to process received messages
@@ -213,12 +212,10 @@ transport SmartMessage::Transport.create(:memory, auto_process: true)
213
212
  ```
214
213
 
215
214
  ### Serializers
216
- Serializers handle message encoding/decoding:
217
-
218
- ```ruby
219
- # JSON serialization (built-in)
220
- serializer SmartMessage::Serializer::Json.new
221
- ```
215
+ Transports handle their own serialization automatically:
216
+ - **MemoryTransport**: No serialization (objects passed directly)
217
+ - **StdoutTransport**: JSON serialization for readability
218
+ - **RedisTransport**: MessagePack (with JSON fallback) for efficiency
222
219
 
223
220
  ### Message Handlers
224
221
 
@@ -321,7 +318,6 @@ class NotificationMessage < SmartMessage::Base
321
318
 
322
319
  config do
323
320
  transport SmartMessage::Transport.create(:memory, auto_process: true)
324
- serializer SmartMessage::Serializer::Json.new
325
321
  end
326
322
 
327
323
  def self.process(decoded_message)
@@ -364,7 +360,6 @@ class EventMessage < SmartMessage::Base
364
360
 
365
361
  config do
366
362
  transport SmartMessage::Transport.create(:stdout, output: 'events.log')
367
- serializer SmartMessage::Serializer::Json.new
368
363
  end
369
364
  end
370
365
 
data/docs/index.md CHANGED
@@ -31,13 +31,15 @@ Think of SmartMessage as ActiveRecord for messaging - just as ActiveRecord frees
31
31
 
32
32
  ### Transports
33
33
  - [Transport Layer](reference/transports.md)
34
- - [Redis Queue Transport](transports/redis-queue-transport.md) ⭐ **Featured**
34
+ - [Multi-Transport Publishing](transports/multi-transport.md)
35
+ - [Redis Transport](transports/redis-transport.md)
35
36
  - [Redis Transport Comparison](transports/redis-transport-comparison.md)
37
+ - [Memory Transport](transports/memory-transport.md)
36
38
 
37
39
  ### Guides
38
- - [Redis Queue Getting Started](guides/redis-queue-getting-started.md)
39
- - [Advanced Routing Patterns](guides/redis-queue-patterns.md)
40
- - [Production Deployment](guides/redis-queue-production.md)
40
+ - [Examples & Use Cases](getting-started/examples.md)
41
+ - [Architecture Overview](core-concepts/architecture.md)
42
+ - [Transport Configuration](reference/transports.md)
41
43
 
42
44
  ### Reference
43
45
  - [Serializers](reference/serializers.md)