dionysus-rb 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 51b0fe501a9678e2fc887a1525a40a0ddc6e04d9a2d49e8211305370eef4e843
-  data.tar.gz: 82597c73fa710cbac12f091e6f8c26462aae3dfc7572de180cbc474c70cedf1a
+  metadata.gz: 8297895cea885c4df1b824188297d199eed0eb00f4c52f102842e8468c02728a
+  data.tar.gz: b71aff2bf67cffe2a5be1e9befd29f3651a23501ee107806b7c2c6eef85f744f
 SHA512:
-  metadata.gz: c7f05381d3fc7e3eba08e4666a46b71e1bc52425d56a3a0d46c5fde4287fdcc77bc64c4521d4f76307849e1cb017e07ab9d612dc127c72bc3fe5e23d919c994e
-  data.tar.gz: 13e9c0664deb5453e9510c0cba835222cb487ff3511696bdcfce4ec09f8677eb16fd4b25bafd4748275cc0533bd62ac3a524761d673d368666153ff79892cb40
+  metadata.gz: 3ad54974b1e4956cd76f94e865c225dfe1beb2b339a29b6a594faf1671f59b4388d9a39a420788f57692e168edb1fc067e351a96d7d5883ec237f051f319eb21
+  data.tar.gz: 428894822478c3ed6296eb51c7440049fe39dd3ec54220abc038e1786b3f4468e48ee745cd2cadaaf7a419b9b093e79628cfb0be476602d352c6317280771685

data/CHANGELOG.md

@@ -1,5 +1,11 @@
 ## [Unreleased]
 
+## [0.3.0]
+- Allow to provide multiple message filters. `message_filter` stays for backwards compatibility and depends on `message_filters`.
+
+## [0.2.0]
+- Fix creation of outbox records when updating a record that was soft-deleted
+
 ## [0.1.0]
 
 - Turn private gem `bookingsync-prometheus` into `dionysus-rb` and release it publicly

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    dionysus-rb (0.1.0)
+    dionysus-rb (0.3.0)
       activerecord (>= 5)
       activesupport (>= 3.2)
       concurrent-ruby

data/README.md

@@ -24,8 +24,8 @@ If bundler is not being used to manage dependencies, install the gem by executin
 
 Please read [this article first](https://www.smily.com/engineering/integration-patterns-for-distributed-architecture-how-we-use-kafka-in-smily-and-why) to understand the context how this gem was built. Also, it's just recently been made public, so some part of the docs might require clarification. If you find any section like that, don't hesitate to submit an issue.
 
-### TODO - update the article is published.
-Also, [read this article], which is an introduction to the gem.
+
+Also, [read this article](https://www.smily.com/engineering/integration-patterns-for-distributed-architecture-intro-to-dionysus-rb), which is an introduction to the gem.
 
 
 Any application can be both consumer and the producer of Karafka events, so let's take a look how to handle configuration for both scenario.
@@ -115,7 +115,7 @@ end
 
 
 There are a couple of important things to understand here.
-- A namespace might be used for versioning so that you can have e.g., `v3` and v4` format working at the same time and consumers consumings from different ones as they need. Namespace is a part of the topic name, in the example above the following topics are declared: `v3_accounts`, `v3_rentals`, `v3_bookings`, `v3_los_records`. Most likely you will need to create them manually in the production environement, depending the Kafka cluster configuration.
+- A namespace might be used for versioning so that you can have e.g., `v3` and `v4` format working at the same time and consumers consumings from different ones as they need. Namespace is a part of the topic name, in the example above the following topics are declared: `v3_accounts`, `v3_rentals`, `v3_bookings`, `v3_los_records`. Most likely you will need to create them manually in the production environement, depending the Kafka cluster configuration.
 - `topic` is a declaration of Kafka `topics`. To understand more about topics and what would be some rule of thumbs when designing them, please [read this article](https://www.smily.com/engineering/integration-patterns-for-distributed-architecture-intro-to-kafka).
 - Some entities might have attributes depending on other entities (computed properties) or might need to be always published together (kind of like Domain-Driven Design Aggregate). For these cases, use `with` directive which is an equivalent of sideloading from REST APIs.  E.g., Booking could have `final_price` attribute that depends on other models, like BookingsFee or BookingsTax, which contribute to that price. Publishing these items separately, e.g. first Bookings Fee first and then Booking with the changed final price might lead to inconsistency on the consumer side where `final_price` value doesn't match the value that would be obtained by summing all elements of the price. That's why all these records need to be published together. That's what `with` option is supposed to cover: `publish Booking, with: [BookingsFee, BookingsTax]`. Thanks to that declaration, any update to Booking or change its dependencies (BookingsFee, BookingsTax) such as creation/update/deletion will result in publishing `booking_updated` event.
 
@@ -1136,7 +1136,8 @@ Dionysus::Consumer.configure do |config|
   config.synced_data_attribute = :synced_data # required, default: :synced_data
   config.resolve_synced_data_hash_proc = ->(record) { record.synced_data_model.synced_data_hash } # optional, defaults to ->(record) { record.public_send(Dionysus::Consumer.configuration.synced_data_attribute).to_h }
   config.sidekiq_queue = :default # optional, defaults to `:dionysus`
-  config.message_filter = FilterIgnoringLargeMessageToAvoidOutofMemoryErrors.new(error_handler: Sentry) # not required, defaults to Dionysus::Utils::DefaultMessageFilter, which doesn't ignore any messages. It can be useful when you want to ignore some messages, e.g. some very large ones that would cause OOM error. Check the implementation of `Dionysus::Utils::DefaultMessageFilter for more details to understand what kind of arguments are available to set the condition. `error_handler` needs to implement Sentry-like interface.
+  config.message_filter = FilterIgnoringLargeMessageToAvoidOutofMemoryErrors.new(error_handler: Sentry) #  DEPRECATED - not required, defaults to Dionysus::Utils::DefaultMessageFilter, which doesn't ignore any messages. It can be useful when you want to ignore some messages, e.g. some very large ones that would cause OOM error. Check the implementation of `Dionysus::Utils::DefaultMessageFilter for more details to understand what kind of arguments are available to set the condition. `error_handler` needs to implement Sentry-like interface. Kept for backwards compatibility, please use `message_filters` instead.
+  config.message_filters = [FilterIgnoringLargeMessageToAvoidOutofMemoryErrors.new(error_handler: Sentry)] # not required, defaults to [Dionysus::Utils::DefaultMessageFilter], which doesn't ignore any messages. It can be useful when you want to ignore some messages, e.g. some very large ones that would cause OOM error. Check the implementation of `Dionysus::Utils::DefaultMessageFilter for more details to understand what kind of arguments are available to set the condition. `error_handler` needs to implement Sentry-like interface. 
 
   # if you ever need to provide mapping:
 
@@ -1203,4 +1204,3 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/Bookin
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-["NAME_OF_THE_APP"]:

data/lib/dionysus/consumer/config.rb

@@ -6,7 +6,7 @@ class Dionysus::Consumer::Config
     :instrumenter, :event_bus, :soft_delete_strategy, :soft_deleted_at_timestamp_attribute,
     :synced_created_at_timestamp_attribute, :synced_updated_at_timestamp_attribute, :synced_id_attribute,
     :synced_data_attribute, :consumer_base_class, :retry_provider, :resolve_synced_data_hash_proc, :sidekiq_queue,
-    :message_filter
+    :message_filters
 
   def self.default_sidekiq_queue
     :dionysus
@@ -94,4 +94,13 @@ class Dionysus::Consumer::Config
     @message_filter || Dionysus::Utils::DefaultMessageFilter.new(error_handler:
       Dionysus::Utils::NullErrorHandler)
   end
+
+  def message_filter=(val)
+    @message_filter = val
+    self.message_filters = [val]
+  end
+
+  def message_filters
+    @message_filters || Array.wrap(message_filter)
+  end
 end

data/lib/dionysus/consumer/params_batch_processor.rb

@@ -27,8 +27,9 @@ class Dionysus::Consumer::ParamsBatchProcessor
             transformed_data = Dionysus::Consumer::Deserializer.new(data).deserialize
           end
 
-          if ignore_message?(topic: topic, message: message, transformed_data: transformed_data)
-            notify_about_ignored_message(topic: topic, message: message, transformed_data: transformed_data)
+          if (applicable_message_filter = find_applicable_message_filter(topic, message, transformed_data))
+            applicable_message_filter.notify_about_ignored_message(topic: topic, message: message,
+              transformed_data: transformed_data)
             next
           end
 
@@ -50,8 +51,11 @@ class Dionysus::Consumer::ParamsBatchProcessor
 
   private
 
-  delegate :message_filter, to: :config
-  delegate :ignore_message?, :notify_about_ignored_message, to: :message_filter
+  delegate :message_filters, to: :config
+
+  def find_applicable_message_filter(topic, message, transformed_data)
+    message_filters.find { |f| f.ignore_message?(topic: topic, message: message, transformed_data: transformed_data) }
+  end
 
   def instrument(label, options = {}, &block)
     config.instrumenter.instrument(label, options, &block)

data/lib/dionysus/producer/outbox.rb

@@ -82,10 +82,10 @@ class Dionysus::Producer::Outbox
   def event_type_for_update_of_soft_deletable_record_for_soft_delete_state_change(publishable)
     if publishable.previous_changes_uncanceled?
       :created
-    elsif publishable.previous_changes_canceled?
+    elsif publishable.previous_changes_canceled? || publishable.previous_changes_still_canceled?
       :destroyed
-    elsif publishable.previous_changes_still_canceled? || publishable.previous_changed_still_visible?
-      nil
+    elsif publishable.previous_changed_still_visible?
+      raise "that should never happen: a cannot be still visible when it was soft-deleted"
     else
       raise "that should never happen"
     end
@@ -95,7 +95,7 @@ class Dionysus::Producer::Outbox
     if publishable.visible? || (publishable.soft_deleted? && publishable.dionysus_publish_updates_after_soft_delete?)
       :updated
     elsif publishable.soft_deleted?
-      nil
+      nil # deliberately not returning any event because the record is configured as such
     else
       raise "that should never happen"
     end

data/lib/dionysus/version.rb

@@ -3,5 +3,5 @@
 module Dionysus
   module Version
   end
-  VERSION = "0.1.0"
+  VERSION = "0.3.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dionysus-rb
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Karol Galanciak
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-12-11 00:00:00.000000000 Z
+date: 2024-02-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord