pub_sub_model_sync 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0839d268607d201c9f77bbbca404f32aa5af02621b7582256eb384b42b9d3985'
-  data.tar.gz: 0ad3d2cfdce3ce99eb5d0b95a4292496927befaca2a8e2d83b865af411be2988
+  metadata.gz: ce57b6483f8a2f3fb1a65a2b12a19dd3d7e8ce75afea225ead398980dbfbbf5c
+  data.tar.gz: b86d05d225cb4729c5299d7cd8711fe7523f2b7eb851fd1229e99a6caa008c3d
 SHA512:
-  metadata.gz: c86be499e21572b796060fc07f5d95191971b87366bbebe3b22737d5eb906f25a6e850f4b4b6f2c08f2790541169f36401fb67b95df0c5da365fda1f76e8751f
-  data.tar.gz: 3e91e306f83eb4ba8dd5c494aadc90b36a702935d7fb716424af80efee6fe6bce9ff895ae6231aa1c32327d7f468ed2148aa69a5f37842c10e0f50e206dad82a
+  metadata.gz: ce613eb94bde316e2a8211f2efb1f0afcc86542607f80a7f53476afde59d6b6757793de4d2ca658a5c39b690c46a7bc8399e913119f632e06353f71278a55e2c
+  data.tar.gz: d012f277cb251a96e5baee5292c40dd63fdf4710e7fd900a93a1b642ff2b33f2f11f80b4663ea0867fe44f60cfba33886910b49d90d95f853e2fed3aece5ac02

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    pub_sub_model_sync (1.5.0)
+    pub_sub_model_sync (1.6.0)
       rails
 
 GEM

data/README.md

@@ -271,8 +271,10 @@ PubSubModelSync::Payload.new({ ids: [my_user.id] }, { klass: 'User', action: :ba
   - `as_klass:` (String, default current class name): Class name of the notification
   - `headers:` (Hash, optional): header settings (More in Payload.headers)
 
-- `ps_perform_publish(action = :create)` Permits to perform manually the callback of a specific `ps_after_action`
+- `ps_perform_publish(action = :create, parents_actions: false)` Permits to perform manually the callback of a specific `ps_after_action`
   - `action` (Symbol, default: :create) Only :create|:update|:destroy
+  - `parents_actions` (Boolean, default: false) When `true`, includes inherited PubSub-callbacks from parent classes
+    
   
 #### **Publisher helpers**
 - Publish or republish a notification
@@ -317,6 +319,7 @@ Any notification before delivering is transformed as a Payload for a better port
       Note: Final `ordering_key` is calculated as: `payload.headers[:forced_ordering_key] || current_transaction&.key || payload.headers[:ordering_key]`         
     - `topic_name`: (String|Array<String>, optional): Specific topic name where to deliver the notification (default `PubSubModelSync::Config.topic_name`).
     - `forced_ordering_key`: (String, optional): Overrides `ordering_key` with the provided value even withing transactions. Default `nil`.
+    - `target_app_key`: (String, optional): Allows to send the notification to a specific app (includes the application key, separated by comma when multiple apps). Default `nil`.
     - `cache` (Boolean | Hash, Default false) Cache settings   
         - `true`: Skip publishing similar payloads
         - `Hash<required: Array<Symbol>>`: Same as `true` and enables payload optimization to exclude unchanged non important attributes. Sample: `{ required: %i[id email] }`
@@ -379,16 +382,17 @@ Note: To reduce Payload size, some header info are not delivered (Enable debug m
   - Manual transactions   
     `PubSubModelSync::MessagePublisher::transaction(key, max_buffer: , &block)`
     - `key` (String|nil) Key used as the ordering_key for all inner notifications (When nil, will use `ordering_key` of the first notification)  
-    - `max_buffer:` (Integer, default: `PubSubModelSync::Config.transactions_max_buffer`) Transaction buffer size (more details in #transactions_max_buffer).     
+    - `max_buffer:` (Integer, default: `PubSubModelSync::Config.transactions_max_buffer`) Transaction buffer size (DEPRECATED).     
+    - `headers:` (Hash) Header settings to be added to each Payload's header inside this transaction     
     Sample:
     ```ruby
-      PubSubModelSync::MessagePublisher::transaction('my-custom-key') do
+      PubSubModelSync::MessagePublisher::transaction('my-custom-key', headers: { key: 'my-key' }) do
         user = User.create(name: 'test') # `User`:`:create` notification
         post = Post.create(title: 'sample') # `Post`:`:create` notification
         PubSubModelSync::Payload.new({ ids: [user.id] }, { klass: 'User', action: :send_welcome, mode: :klass }).publish! # `User`:`:send_welcome` notification
       end
     ```
-    All notifications uses `ordering_key: 'my-custom-key'` and will be processed in the same order they were published.
+    All notifications uses `ordering_key: 'my-custom-key'` and will be processed in the same order they were published (Payload headers will include `key="my-key"`).
 
 ## **Testing with RSpec**
 - Config: (spec/rails_helper.rb)
@@ -536,6 +540,8 @@ config.debug = true
     (Proc) => called after publishing a notification
 - ```.on_error_publish = ->(exception, {payload:}) { payload.delay(...).publish! }```
     (Proc) => called when failed publishing a notification (delayed_job or similar can be used for retrying)
+- ```.skip_cache = false```
+    (true/false*) => Allow to skip payload optimization (cache settings)
 - ```.transactions_max_buffer = 1``` (Integer, default 1) Controls the maximum quantity of notifications to be enqueued to the transaction-buffer before delivering them and thus adds the ability to rollback notifications if the transaction fails.        
     Once this quantity of notifications is reached, then all notifications of the current transaction will immediately be delivered (can be customized per transaction).    
     Note: There is no way to rollback delivered notifications if current transaction fails later.      

data/lib/pub_sub_model_sync/message_processor.rb

@@ -23,15 +23,15 @@ module PubSubModelSync
     end
 
     def process
-      retries ||= 0
       process!
     rescue => e
-      retry_process?(e, retries += 1) ? retry : notify_error(e)
+      notify_error(e)
     end
 
     private
 
-    def run_subscriber(subscriber)
+    def run_subscriber(subscriber) # rubocop:disable Metrics/AbcSize
+      retries ||= 0
       processor = PubSubModelSync::RunSubscriber.new(subscriber, payload)
       return unless processable?(subscriber)
 
@@ -39,6 +39,8 @@ module PubSubModelSync
       processor.call
       res = config.on_success_processing.call(payload, { subscriber: subscriber })
       log "processed message with: #{payload.inspect}" if res != :skip_log
+    rescue => e
+      retry_process?(e, retries += 1) ? retry : raise(e)
     end
 
     def processable?(subscriber)
@@ -47,7 +49,7 @@ module PubSubModelSync
       !cancel
     end
 
-    # @param error (StandardError)
+    # @param error (StandardError, Exception)
     def notify_error(error)
       error_msg = 'Error processing message: '
       error_details = [payload, error.message, error.backtrace]
@@ -59,9 +61,10 @@ module PubSubModelSync
       raise(e)
     end
 
+    # @param error [StandardError]
     def lost_db_connection?(error)
-      connection_lost_classes = %w[ActiveRecord::ConnectionTimeoutError PG::UnableToSend]
-      connection_lost_classes.include?(error.class.name) || error.message.match?(/lost connection/i)
+      classes = %w[ActiveRecord::ConnectionTimeoutError PG::UnableToSend ActiveRecord::ConnectionNotEstablished]
+      classes.include?(error.class.name) || error.message.match?(/Lost connection to MySQL server/i)
     end
 
     def retry_process?(error, retries) # rubocop:disable Metrics/MethodLength

data/lib/pub_sub_model_sync/message_publisher.rb

@@ -18,6 +18,9 @@ module PubSubModelSync
       #   worker/thread.
       # @see Transaction.new(...)
       # @param key (String|Nil)
+      # @param settings (Hash<:headers, :max_buffer>)
+      #   @option headers [Hash] Headers to be merged for each payload inside this transaction
+      #   @option max_buffer [Integer] Deprecated
       # @param block (Yield) block to be executed
       def transaction(key, settings = {}, &block)
         t = init_transaction(key, settings)
@@ -71,7 +74,7 @@ module PubSubModelSync
       # @return Payload
       # Raises error if exist
       def publish!(payload, &block)
-        payload.headers[:ordering_key] = ordering_key_for(payload)
+        add_transaction_headers(payload)
         return unless ensure_publish(payload)
 
         current_transaction ? current_transaction.add_payload(payload) : connector_publish(payload)
@@ -106,8 +109,10 @@ module PubSubModelSync
         !cancelled
       end
 
-      def ordering_key_for(payload)
-        payload.headers[:forced_ordering_key] || current_transaction&.key || payload.headers[:ordering_key]
+      def add_transaction_headers(payload)
+        key = payload.headers[:forced_ordering_key] || current_transaction&.key || payload.headers[:ordering_key]
+        payload.headers[:ordering_key] = key
+        payload.headers.merge!(current_transaction.headers) if current_transaction
       end
 
       def ensure_model_publish(model, action, payload)

data/lib/pub_sub_model_sync/publisher_concern.rb

@@ -38,10 +38,10 @@ module PubSubModelSync
 
     # Permits to perform manually the callback for a specific action
     # @param action (Symbol, default: :create) Only :create|:update|:destroy
-    def ps_perform_publish(action = :create)
-      items = self.class.ps_cache_publish_callbacks.select { |item| item[:actions].include?(action) }
-      raise(StandardError, "No callback found for action :#{action}") if items.empty?
-
+    def ps_perform_publish(action = :create, parents_actions: false)
+      callbacks = self.class.ps_cache_publish_callbacks
+      callbacks = self.class.ancestors.map { |p| p.try(:ps_cache_publish_callbacks) }.compact.flatten if parents_actions
+      items = callbacks.select { |item| item[:actions].include?(action) }
       items.each { |item| instance_exec(action, &item[:callback]) }
       self
     end

data/lib/pub_sub_model_sync/service_base.rb

@@ -34,7 +34,7 @@ module PubSubModelSync
     def process_message(payload_info)
       payload = decode_payload(payload_info)
       return unless payload
-      return if same_app_message?(payload)
+      return if same_app_message?(payload) || !target_app_message?(payload)
 
       payload.process
     end
@@ -57,5 +57,10 @@ module PubSubModelSync
       log("Skipping message from same origin: #{[payload]}") if res && config.debug
       res
     end
+
+    def target_app_message?(payload)
+      key = payload.headers[:target_app_key].to_s
+      !key.present? || key.split(',').include?(config.subscription_key)
+    end
   end
 end

data/lib/pub_sub_model_sync/transaction.rb

@@ -3,17 +3,18 @@
 module PubSubModelSync
   class Transaction < Base
     PUBLISHER_KLASS = PubSubModelSync::MessagePublisher
-    attr_accessor :key, :payloads, :max_buffer, :root, :children, :finished
+    attr_accessor :key, :payloads, :max_buffer, :root, :children, :finished, :headers
 
     # @param key (String,Nil) Transaction key, if empty will use the ordering_key from first payload
     # @param max_buffer (Integer) Once this quantity of notifications is reached, then all notifications
     #   will immediately be delivered.
     #   Note: There is no way to rollback delivered notifications if current transaction fails
-    def initialize(key, max_buffer: config.transactions_max_buffer)
+    def initialize(key, max_buffer: config.transactions_max_buffer, headers: {})
       @key = key
       @max_buffer = max_buffer
       @children = []
       @payloads = []
+      @headers = headers
     end
 
     # @param payload (Payload)

data/lib/pub_sub_model_sync/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module PubSubModelSync
-  VERSION = '1.5.0'
+  VERSION = '1.6.0'
 end

data/pub_sub_model_sync.gemspec

@@ -4,15 +4,18 @@ lib = File.expand_path('lib', __dir__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'pub_sub_model_sync/version'
 
-Gem::Specification.new do |spec|
+Gem::Specification.new do |spec| # rubocop:disable Metrics/BlockLength
   spec.required_ruby_version = '>= 2.4' # rubocop:disable Gemspec/RequiredRubyVersion
   spec.name          = 'pub_sub_model_sync'
   spec.version       = PubSubModelSync::VERSION
   spec.authors       = ['Owen']
   spec.email         = ['owenperedo@gmail.com']
 
-  spec.summary       = 'Permit to sync models between apps through pub/sub'
-  spec.description   = 'Permit to sync models between apps through pub/sub'
+  spec.summary       = 'This gem permits to sync automatically models and custom data between multiple Rails
+    applications by publishing notifications via pubsub (Google PubSub, RabbitMQ, or Apache Kafka) and automatically
+    processed by all connected applications. Out of the scope, this gem includes transactions to keep Data consistency
+    by processing notifications in the order they were delivered.'
+  spec.description   = spec.summary
   spec.homepage      = 'https://github.com/owen2345/pub_sub_model_sync'
   spec.license       = 'MIT'
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pub_sub_model_sync
 version: !ruby/object:Gem::Version
-  version: 1.5.0
+  version: 1.6.0
 platform: ruby
 authors:
 - Owen
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-06-02 00:00:00.000000000 Z
+date: 2022-10-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -80,7 +80,11 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: Permit to sync models between apps through pub/sub
+description: This gem permits to sync automatically models and custom data between
+  multiple Rails applications by publishing notifications via pubsub (Google PubSub,
+  RabbitMQ, or Apache Kafka) and automatically processed by all connected applications.
+  Out of the scope, this gem includes transactions to keep Data consistency by processing
+  notifications in the order they were delivered.
 email:
 - owenperedo@gmail.com
 executables: []
@@ -159,5 +163,9 @@ requirements: []
 rubygems_version: 3.0.8
 signing_key:
 specification_version: 4
-summary: Permit to sync models between apps through pub/sub
+summary: This gem permits to sync automatically models and custom data between multiple
+  Rails applications by publishing notifications via pubsub (Google PubSub, RabbitMQ,
+  or Apache Kafka) and automatically processed by all connected applications. Out
+  of the scope, this gem includes transactions to keep Data consistency by processing
+  notifications in the order they were delivered.
 test_files: []