waterdrop 2.3.3 → 2.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b1482f0d2955c43c2ab479372c9cbb4a8c039723121c26b2ff75d0e26700d71e
-  data.tar.gz: 1bb3e675c0c632d4940b2b661bdb75a74c44358a7e53fa9dc7051fdad62218b0
+  metadata.gz: 0bd7288d6c0c7a6f356f050af309899cad41e6fcf202c945229231d7058194e1
+  data.tar.gz: 727738b727dccf8e2d7b4eb7c3f25b1c7ed761eb958eae4ba818c7402839a6c7
 SHA512:
-  metadata.gz: 8921844cc44625916974c63478562b66a7f30a833b6f95e71e8c3448109d673bd3520a8a48bdf0fe7296e5e00b988c15c3dd92cab2e11ee37914f667966a14de
-  data.tar.gz: 6c6038c4f9a718d107fd885e66bf29cc3dde09c6f6e0be2ba41651f3494672ddd925e71574f53ec8ec70d4e766024018f73a43a5fadf99402117ac3a0751b0c0
+  metadata.gz: 95f69c8cd00d33e04747f1447ca9fe40de88941869925662116427b14da7f0eb0a04c7eada0e00ea6c226dd8be90484b7e97e6d34f9c7f0a11333bfede37e03c
+  data.tar.gz: fa43e25469180c9d9e65f31b4d79554b35c090e79d8812586d614a77acafe04bb873f24ed58f739c4523696282bef8dfa05e0fa0358ca808bfd2df038bf3d55b

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 # WaterDrop changelog
 
+## 2.4.0 (2022-07-28)
+- Small refactor of the DataDog/Statsd listener to align for future extraction to `karafka-common`.
+- Replace `dry-monitor` with home-brew notification layer (API compatible) and allow for usage with `ActiveSupport::Notifications`.
+- Remove all the common code into `karafka-core` and add it as a dependency.
+
 ## 2.3.3 (2022-07-18)
 - Replace `dry-validation` with home-brew validation layer and drop direct dependency on `dry-validation`.
 - Remove indirect dependency on dry-configurable from DataDog listener (no changes required).

data/Gemfile.lock

@@ -1,16 +1,15 @@
 PATH
   remote: .
   specs:
-    waterdrop (2.3.3)
-      concurrent-ruby (>= 1.1)
-      dry-monitor (~> 0.5)
+    waterdrop (2.4.0)
+      karafka-core (~> 2.0)
       rdkafka (>= 0.10)
       zeitwerk (~> 2.3)
 
 GEM
   remote: https://rubygems.org/
   specs:
-    activesupport (7.0.3)
+    activesupport (7.0.3.1)
       concurrent-ruby (~> 1.0, >= 1.0.2)
       i18n (>= 1.6, < 2)
       minitest (>= 5.1)
@@ -19,26 +18,15 @@ GEM
     concurrent-ruby (1.1.10)
     diff-lcs (1.5.0)
     docile (1.4.0)
-    dry-configurable (0.15.0)
-      concurrent-ruby (~> 1.0)
-      dry-core (~> 0.6)
-    dry-core (0.8.0)
-      concurrent-ruby (~> 1.0)
-    dry-events (0.3.0)
-      concurrent-ruby (~> 1.0)
-      dry-core (~> 0.5, >= 0.5)
-    dry-monitor (0.6.1)
-      dry-configurable (~> 0.13, >= 0.13.0)
-      dry-core (~> 0.5, >= 0.5)
-      dry-events (~> 0.2)
-      zeitwerk (~> 2.5)
     factory_bot (6.2.1)
       activesupport (>= 5.0.0)
     ffi (1.15.5)
-    i18n (1.10.0)
+    i18n (1.12.0)
       concurrent-ruby (~> 1.0)
+    karafka-core (2.0.0)
+      concurrent-ruby (>= 1.1)
     mini_portile2 (2.8.0)
-    minitest (5.16.0)
+    minitest (5.16.2)
     rake (13.0.6)
     rdkafka (0.12.0)
       ffi (~> 1.15)
@@ -63,12 +51,11 @@ GEM
       simplecov_json_formatter (~> 0.1)
     simplecov-html (0.12.3)
     simplecov_json_formatter (0.1.4)
-    tzinfo (2.0.4)
+    tzinfo (2.0.5)
       concurrent-ruby (~> 1.0)
     zeitwerk (2.6.0)
 
 PLATFORMS
-  arm64-darwin-21
   x86_64-linux
 
 DEPENDENCIES

data/README.md

@@ -2,7 +2,7 @@
 
 **Note**: Documentation presented here refers to WaterDrop `2.x`.
 
-WaterDrop `2.x` does **not** work with Karafka `1.*` and aims to either work as a standalone producer outside of Karafka `1.*` ecosystem or as a part of soon to be released Karafka `2.0.*`.
+WaterDrop `2.x` works with Karafka `2.*` and aims to either work as a standalone producer or as a part of the Karafka `2.*`.
 
 Please refer to [this](https://github.com/karafka/waterdrop/tree/1.4) branch and its documentation for details about WaterDrop `1.*` usage.
 
@@ -10,17 +10,17 @@ Please refer to [this](https://github.com/karafka/waterdrop/tree/1.4) branch and
 [![Gem Version](https://badge.fury.io/rb/waterdrop.svg)](http://badge.fury.io/rb/waterdrop)
 [![Join the chat at https://slack.karafka.io](https://raw.githubusercontent.com/karafka/misc/master/slack.svg)](https://slack.karafka.io)
 
-Gem used to send messages to Kafka in an easy way with an extra validation layer. It is a part of the [Karafka](https://github.com/karafka/karafka) ecosystem.
+A gem to send messages to Kafka easily with an extra validation layer. It is a part of the [Karafka](https://github.com/karafka/karafka) ecosystem.
 
 It:
 
- - Is thread safe
+ - Is thread-safe
  - Supports sync producing
  - Supports async producing
  - Supports buffering
  - Supports producing messages to multiple clusters
  - Supports multiple delivery policies
- - Works with Kafka 1.0+ and Ruby 2.6+
+ - Works with Kafka `1.0+` and Ruby `2.7+`
 
 ## Table of contents
 
@@ -30,6 +30,8 @@ It:
   * [Kafka configuration options](#kafka-configuration-options)
 - [Usage](#usage)
   * [Basic usage](#basic-usage)
+  * [Using WaterDrop across the application and with Ruby on Rails](#using-waterdrop-across-the-application-and-with-ruby-on-rails)
+  * [Using WaterDrop with a connection-pool](#using-waterdrop-with-a-connection-pool)
   * [Buffering](#buffering)
       + [Using WaterDrop to buffer messages based on the application logic](#using-waterdrop-to-buffer-messages-based-on-the-application-logic)
       + [Using WaterDrop with rdkafka buffers to achieve periodic auto-flushing](#using-waterdrop-with-rdkafka-buffers-to-achieve-periodic-auto-flushing)
@@ -42,6 +44,8 @@ It:
 
 ## Installation
 
+**Note**: If you want to both produce and consume messages, please use [Karafka](https://github.com/karafka/karafka/). It integrates WaterDrop automatically.
+
 Add this to your Gemfile:
 
 ```ruby
@@ -56,7 +60,7 @@ bundle install
 
 ## Setup
 
-WaterDrop is a complex tool, that contains multiple configuration options. To keep everything organized, all the configuration options were divided into two groups:
+WaterDrop is a complex tool that contains multiple configuration options. To keep everything organized, all the configuration options were divided into two groups:
 
 - WaterDrop options - options directly related to WaterDrop and its components
 - Kafka driver options - options related to `rdkafka`
@@ -103,8 +107,6 @@ You can create producers with different `kafka` settings. Documentation of the a
 
 ## Usage
 
-Please refer to the [documentation](https://www.rubydoc.info/gems/waterdrop) in case you're interested in the more advanced API.
-
 ### Basic usage
 
 To send Kafka messages, just create a producer and use it:
@@ -157,6 +159,41 @@ Here are all the things you can provide in the message hash:
 
 Keep in mind, that message you want to send should be either binary or stringified (to_s, to_json, etc).
 
+### Using WaterDrop across the application and with Ruby on Rails
+
+If you plan to both produce and consume messages using Kafka, you should install and use [Karafka](https://github.com/karafka/karafka). It integrates automatically with Ruby on Rails applications and auto-configures WaterDrop producer to make it accessible via `Karafka#producer` method.
+
+If you want to only produce messages from within your application, since WaterDrop is thread-safe you can create a single instance in an initializer like so:
+
+```ruby
+KAFKA_PRODUCER = WaterDrop::Producer.new
+
+KAFKA_PRODUCER.setup do |config|
+  config.kafka = { 'bootstrap.servers': 'localhost:9092' }
+end
+
+# And just dispatch messages
+KAFKA_PRODUCER.produce_sync(topic: 'my-topic', payload: 'my message')
+```
+
+### Using WaterDrop with a connection-pool
+
+While WaterDrop is thread-safe, there is no problem in using it with a connection pool inside high-intensity applications. The only thing worth keeping in mind, is that WaterDrop instances should be shutdown before the application is closed.
+
+```ruby
+KAFKA_PRODUCERS_CP = ConnectionPool.new do
+  WaterDrop::Producer.new do |config|
+    config.kafka = { 'bootstrap.servers': 'localhost:9092' }
+  end
+end
+
+KAFKA_PRODUCERS_CP.with do |producer|
+  producer.produce_async(topic: 'my-topic', payload: 'my message')
+end
+
+KAFKA_PRODUCERS_CP.shutdown { |producer| producer.close }
+```
+
 ### Buffering
 
 WaterDrop producers support buffering messages in their internal buffers and on the `rdkafka` level via `queue.buffering.*` set of settings.
@@ -316,6 +353,8 @@ producer.monitor.subscribe(listener)
 
 You can also find [here](https://github.com/karafka/waterdrop/blob/master/lib/waterdrop/instrumentation/vendors/datadog/dashboard.json) a ready to import DataDog dashboard configuration file that you can use to monitor all of your producers.
 
+![Example WaterDrop DD dashboard](https://raw.githubusercontent.com/karafka/misc/master/printscreens/waterdrop_dd_dashboard_example.png)
+
 ### Error notifications
 
 WaterDrop allows you to listen to all errors that occur while producing messages and in its internal background threads. Things like reconnecting to Kafka upon network errors and others unrelated to publishing messages are all available under `error.occurred` notification key. You can subscribe to this event to ensure your setup is healthy and without any problems that would otherwise go unnoticed as long as messages are delivered.

data/lib/waterdrop/config.rb

@@ -5,7 +5,7 @@
 module WaterDrop
   # Configuration object for setting up all options required by WaterDrop
   class Config
-    include Configurable
+    include ::Karafka::Core::Configurable
 
     # Defaults for kafka settings, that will be overwritten only if not present already
     KAFKA_DEFAULTS = {
@@ -77,7 +77,7 @@ module WaterDrop
     # Propagates the kafka setting defaults unless they are already present
     # This makes it easier to set some values that users usually don't change but still allows them
     # to overwrite the whole hash if they want to
-    # @param config [Dry::Configurable::Config] dry config of this producer
+    # @param config [WaterDrop::Configurable::Node] config of this producer
     def merge_kafka_defaults!(config)
       KAFKA_DEFAULTS.each do |key, value|
         next if config.kafka.key?(key)

data/lib/waterdrop/contracts/config.rb

@@ -3,7 +3,7 @@
 module WaterDrop
   module Contracts
     # Contract with validation rules for WaterDrop configuration details
-    class Config < Contractable::Contract
+    class Config < ::Karafka::Core::Contractable::Contract
       configure do |config|
         config.error_messages = YAML.safe_load(
           File.read(
@@ -12,13 +12,13 @@ module WaterDrop
         ).fetch('en').fetch('validations').fetch('config')
       end
 
-      required(:id) { |id| id.is_a?(String) && !id.empty? }
-      required(:logger) { |logger| !logger.nil? }
-      required(:deliver) { |deliver| [true, false].include?(deliver) }
-      required(:max_payload_size) { |ps| ps.is_a?(Integer) && ps >= 1 }
-      required(:max_wait_timeout) { |mwt| mwt.is_a?(Numeric) && mwt >= 0 }
-      required(:wait_timeout) { |wt| wt.is_a?(Numeric) && wt.positive? }
-      required(:kafka) { |kafka| kafka.is_a?(Hash) && !kafka.empty? }
+      required(:id) { |val| val.is_a?(String) && !val.empty? }
+      required(:logger) { |val| !val.nil? }
+      required(:deliver) { |val| [true, false].include?(val) }
+      required(:max_payload_size) { |val| val.is_a?(Integer) && val >= 1 }
+      required(:max_wait_timeout) { |val| val.is_a?(Numeric) && val >= 0 }
+      required(:wait_timeout) { |val| val.is_a?(Numeric) && val.positive? }
+      required(:kafka) { |val| val.is_a?(Hash) && !val.empty? }
 
       # rdkafka allows both symbols and strings as keys for config but then casts them to strings
       # This can be confusing, so we expect all keys to be symbolized

data/lib/waterdrop/contracts/message.rb

@@ -4,7 +4,7 @@ module WaterDrop
   module Contracts
     # Contract with validation rules for validating that all the message options that
     # we provide to producer ale valid and usable
-    class Message < Contractable::Contract
+    class Message < ::Karafka::Core::Contractable::Contract
       configure do |config|
         config.error_messages = YAML.safe_load(
           File.read(
@@ -26,13 +26,13 @@ module WaterDrop
         @max_payload_size = max_payload_size
       end
 
-      required(:topic) { |topic| topic.is_a?(String) && TOPIC_REGEXP.match?(topic) }
-      required(:payload) { |payload| payload.is_a?(String) }
-      optional(:key) { |key| key.nil? || (key.is_a?(String) && !key.empty?) }
-      optional(:partition) { |partition| partition.is_a?(Integer) && partition >= -1 }
-      optional(:partition_key) { |p_key| p_key.nil? || (p_key.is_a?(String) && !p_key.empty?) }
-      optional(:timestamp) { |ts| ts.nil? || (ts.is_a?(Time) || ts.is_a?(Integer)) }
-      optional(:headers) { |headers| headers.nil? || headers.is_a?(Hash) }
+      required(:topic) { |val| val.is_a?(String) && TOPIC_REGEXP.match?(val) }
+      required(:payload) { |val| val.is_a?(String) }
+      optional(:key) { |val| val.nil? || (val.is_a?(String) && !val.empty?) }
+      optional(:partition) { |val| val.is_a?(Integer) && val >= -1 }
+      optional(:partition_key) { |val| val.nil? || (val.is_a?(String) && !val.empty?) }
+      optional(:timestamp) { |val| val.nil? || (val.is_a?(Time) || val.is_a?(Integer)) }
+      optional(:headers) { |val| val.nil? || val.is_a?(Hash) }
 
       virtual do |config, errors|
         next true unless errors.empty?

data/lib/waterdrop/instrumentation/monitor.rb

@@ -2,41 +2,18 @@
 
 module WaterDrop
   module Instrumentation
-    # Monitor is used to hookup external monitoring services to monitor how WaterDrop works
-    # Since it is a pub-sub based on dry-monitor, you can use as many subscribers/loggers at the
-    # same time, which means that you might have for example file logging and NewRelic at the same
-    # time
-    # @note This class acts as a singleton because we are only permitted to have single monitor
-    #   per running process (just as logger)
-    class Monitor < Dry::Monitor::Notifications
-      # List of events that we support in the system and to which a monitor client can hook up
-      # @note The non-error once support timestamp benchmarking
-      EVENTS = %w[
-        producer.closed
-
-        message.produced_async
-        message.produced_sync
-        message.acknowledged
-        message.buffered
-
-        messages.produced_async
-        messages.produced_sync
-        messages.buffered
-
-        buffer.flushed_async
-        buffer.flushed_sync
-
-        statistics.emitted
-
-        error.occurred
-      ].freeze
-
-      private_constant :EVENTS
-
-      # @return [WaterDrop::Instrumentation::Monitor] monitor instance for system instrumentation
-      def initialize
-        super(:waterdrop)
-        EVENTS.each(&method(:register_event))
+    # WaterDrop instrumentation monitor that we use to publish events
+    # By default uses our internal notifications bus but can be used with
+    # `ActiveSupport::Notifications` as well
+    class Monitor < ::Karafka::Core::Monitoring::Monitor
+      # @param notifications_bus [Object] either our internal notifications bus or
+      #   `ActiveSupport::Notifications`
+      # @param namespace [String, nil] namespace for events or nil if no namespace
+      def initialize(
+        notifications_bus = WaterDrop::Instrumentation::Notifications.new,
+        namespace = nil
+      )
+        super(notifications_bus, namespace)
       end
     end
   end

data/lib/waterdrop/instrumentation/notifications.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+module WaterDrop
+  module Instrumentation
+    # Instrumented is used to hookup external monitoring services to monitor how WaterDrop works
+    class Notifications < ::Karafka::Core::Monitoring::Notifications
+      # List of events that we support in the system and to which a monitor client can hook up
+      # @note The non-error once support timestamp benchmarking
+      EVENTS = %w[
+        producer.closed
+
+        message.produced_async
+        message.produced_sync
+        message.acknowledged
+        message.buffered
+
+        messages.produced_async
+        messages.produced_sync
+        messages.buffered
+
+        buffer.flushed_async
+        buffer.flushed_sync
+
+        statistics.emitted
+
+        error.occurred
+      ].freeze
+
+      private_constant :EVENTS
+
+      # @return [WaterDrop::Instrumentation::Monitor] monitor instance for system instrumentation
+      def initialize
+        super
+        EVENTS.each { |event| register_event(event) }
+      end
+    end
+  end
+end

data/lib/waterdrop/instrumentation/vendors/datadog/listener.rb

@@ -11,7 +11,7 @@ module WaterDrop
         #
         # @note You need to setup the `dogstatsd-ruby` client and assign it
         class Listener
-          include WaterDrop::Configurable
+          include ::Karafka::Core::Configurable
           extend Forwardable
 
           def_delegators :config, :client, :rd_kafka_metrics, :namespace, :default_tags
@@ -66,7 +66,7 @@ module WaterDrop
 
           # Hooks up to WaterDrop instrumentation for emitted statistics
           #
-          # @param event [Dry::Events::Event]
+          # @param event [WaterDrop::Monitor::Event]
           def on_statistics_emitted(event)
             statistics = event[:statistics]
 
@@ -77,22 +77,15 @@ module WaterDrop
 
           # Increases the errors count by 1
           #
-          # @param _event [Dry::Events::Event]
+          # @param _event [WaterDrop::Monitor::Event]
           def on_error_occurred(_event)
-            client.count(
-              namespaced_metric('error_occurred'),
-              1,
-              tags: default_tags
-            )
+            count('error_occurred', 1, tags: default_tags)
           end
 
           # Increases acknowledged messages counter
-          # @param _event [Dry::Events::Event]
+          # @param _event [WaterDrop::Monitor::Event]
           def on_message_acknowledged(_event)
-            client.increment(
-              namespaced_metric('acknowledged'),
-              tags: default_tags
-            )
+            increment('acknowledged', tags: default_tags)
           end
 
           %i[
@@ -100,12 +93,12 @@ module WaterDrop
             produced_async
           ].each do |event_scope|
             class_eval <<~METHODS, __FILE__, __LINE__ + 1
-              # @param event [Dry::Events::Event]
+              # @param event [WaterDrop::Monitor::Event]
               def on_message_#{event_scope}(event)
                 report_message(event[:message][:topic], :#{event_scope})
               end
 
-              # @param event [Dry::Events::Event]
+              # @param event [WaterDrop::Monitor::Event]
               def on_messages_#{event_scope}(event)
                 event[:messages].each do |message|
                   report_message(message[:topic], :#{event_scope})
@@ -120,10 +113,10 @@ module WaterDrop
             messages_buffered
           ].each do |event_scope|
             class_eval <<~METHODS, __FILE__, __LINE__ + 1
-              # @param event [Dry::Events::Event]
+              # @param event [WaterDrop::Monitor::Event]
               def on_#{event_scope}(event)
-                client.histogram(
-                  namespaced_metric('buffer.size'),
+                histogram(
+                  'buffer.size',
                   event[:buffer].size,
                   tags: default_tags
                 )
@@ -138,7 +131,7 @@ module WaterDrop
             flushed_async
           ].each do |event_scope|
             class_eval <<~METHODS, __FILE__, __LINE__ + 1
-              # @param event [Dry::Events::Event]
+              # @param event [WaterDrop::Monitor::Event]
               def on_buffer_#{event_scope}(event)
                 event[:messages].each do |message|
                   report_message(message[:topic], :#{event_scope})
@@ -149,14 +142,28 @@ module WaterDrop
 
           private
 
+          %i[
+            count
+            gauge
+            histogram
+            increment
+            decrement
+          ].each do |metric_type|
+            class_eval <<~METHODS, __FILE__, __LINE__ + 1
+              def #{metric_type}(key, *args)
+                client.#{metric_type}(
+                  namespaced_metric(key),
+                  *args
+                )
+              end
+            METHODS
+          end
+
           # Report that a message has been produced to a topic.
           # @param topic [String] Kafka topic
           # @param method_name [Symbol] method from which this message operation comes
           def report_message(topic, method_name)
-            client.increment(
-              namespaced_metric(method_name),
-              tags: default_tags + ["topic:#{topic}"]
-            )
+            increment(method_name, tags: default_tags + ["topic:#{topic}"])
           end
 
           # Wraps metric name in listener's namespace
@@ -172,9 +179,9 @@ module WaterDrop
           def report_metric(metric, statistics)
             case metric.scope
             when :root
-              client.public_send(
+              public_send(
                 metric.type,
-                namespaced_metric(metric.name),
+                metric.name,
                 statistics.fetch(*metric.key_location),
                 tags: default_tags
               )
@@ -185,9 +192,9 @@ module WaterDrop
                 # node ids
                 next if broker_statistics['nodeid'] == -1
 
-                client.public_send(
+                public_send(
                   metric.type,
-                  namespaced_metric(metric.name),
+                  metric.name,
                   broker_statistics.dig(*metric.key_location),
                   tags: default_tags + ["broker:#{broker_statistics['nodename']}"]
                 )

data/lib/waterdrop/version.rb

@@ -3,5 +3,5 @@
 # WaterDrop library
 module WaterDrop
   # Current WaterDrop version
-  VERSION = '2.3.3'
+  VERSION = '2.4.0'
 end

data/lib/waterdrop.rb

@@ -3,11 +3,8 @@
 # External components
 # delegate should be removed because we don't need it, we just add it because of ruby-kafka
 %w[
+  karafka-core
   forwardable
-  concurrent/array
-  concurrent/hash
-  yaml
-  dry/monitor/notifications
   rdkafka
   json
   zeitwerk

data/waterdrop.gemspec

@@ -16,8 +16,7 @@ Gem::Specification.new do |spec|
   spec.description   = spec.summary
   spec.license       = 'MIT'
 
-  spec.add_dependency 'concurrent-ruby', '>= 1.1'
-  spec.add_dependency 'dry-monitor', '~> 0.5'
+  spec.add_dependency 'karafka-core', '~> 2.0'
   spec.add_dependency 'rdkafka', '>= 0.10'
   spec.add_dependency 'zeitwerk', '~> 2.3'
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: waterdrop
 version: !ruby/object:Gem::Version
-  version: 2.3.3
+  version: 2.4.0
 platform: ruby
 authors:
 - Maciej Mensfeld
@@ -34,36 +34,22 @@ cert_chain:
   R2P11bWoCtr70BsccVrN8jEhzwXngMyI2gVt750Y+dbTu1KgRqZKp/ECe7ZzPzXj
   pIy9vHxTANKYVyI4qj8OrFdEM5BQNu8oQpL0iQ==
   -----END CERTIFICATE-----
-date: 2022-07-18 00:00:00.000000000 Z
+date: 2022-07-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: concurrent-ruby
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '1.1'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '1.1'
-- !ruby/object:Gem::Dependency
-  name: dry-monitor
+  name: karafka-core
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.5'
+        version: '2.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.5'
+        version: '2.0'
 - !ruby/object:Gem::Dependency
   name: rdkafka
   requirement: !ruby/object:Gem::Requirement
@@ -116,13 +102,6 @@ files:
 - docker-compose.yml
 - lib/waterdrop.rb
 - lib/waterdrop/config.rb
-- lib/waterdrop/configurable.rb
-- lib/waterdrop/configurable/leaf.rb
-- lib/waterdrop/configurable/node.rb
-- lib/waterdrop/contractable.rb
-- lib/waterdrop/contractable/contract.rb
-- lib/waterdrop/contractable/result.rb
-- lib/waterdrop/contractable/rule.rb
 - lib/waterdrop/contracts.rb
 - lib/waterdrop/contracts/config.rb
 - lib/waterdrop/contracts/message.rb
@@ -135,6 +114,7 @@ files:
 - lib/waterdrop/instrumentation/callbacks_manager.rb
 - lib/waterdrop/instrumentation/logger_listener.rb
 - lib/waterdrop/instrumentation/monitor.rb
+- lib/waterdrop/instrumentation/notifications.rb
 - lib/waterdrop/instrumentation/vendors/datadog/dashboard.json
 - lib/waterdrop/instrumentation/vendors/datadog/listener.rb
 - lib/waterdrop/patches/rdkafka/bindings.rb

data/lib/waterdrop/configurable/leaf.rb

@@ -1,8 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Configurable
-    # Single end config value representation
-    Leaf = Struct.new(:name, :default, :constructor)
-  end
-end

data/lib/waterdrop/configurable/node.rb

@@ -1,100 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Configurable
-    # Single non-leaf node
-    # This is a core component for the configurable settings
-    #
-    # The idea here is simple: we collect settings (leafs) and children (nodes) information and we
-    # only compile/initialize the values prior to user running the `#configure` API. This API needs
-    # to run prior to using the result stuff even if there is nothing to configure
-    class Node
-      attr_reader :name, :nestings
-
-      # We need to be able to redefine children for deep copy
-      attr_accessor :children
-
-      # @param name [Symbol] node name
-      # @param nestings [Proc] block for nested settings
-      def initialize(name, nestings = ->(_) {})
-        @name = name
-        @children = []
-        @nestings = nestings
-        instance_eval(&nestings)
-      end
-
-      # Allows for a single leaf or nested node definition
-      #
-      # @param name [Symbol] setting or nested node name
-      # @param default [Object] default value
-      # @param constructor [#call, nil] callable or nil
-      # @param block [Proc] block for nested settings
-      def setting(name, default: nil, constructor: nil, &block)
-        @children << if block
-                       Node.new(name, block)
-                     else
-                       Leaf.new(name, default, constructor)
-                     end
-      end
-
-      # Allows for the configuration and setup of the settings
-      #
-      # Compile settings, allow for overrides via yielding
-      # @return [Node] returns self after configuration
-      def configure
-        compile
-        yield(self) if block_given?
-        self
-      end
-
-      # @return [Hash] frozen config hash representation
-      def to_h
-        config = {}
-
-        @children.each do |value|
-          config[value.name] = if value.is_a?(Leaf)
-                                 public_send(value.name)
-                               else
-                                 value.to_h
-                               end
-        end
-
-        config.freeze
-      end
-
-      # Deep copies all the children nodes to allow us for templates building on a class level and
-      # non-side-effect usage on an instance/inherited.
-      # @return [Node] duplicated node
-      def deep_dup
-        dupped = Node.new(name, nestings)
-
-        dupped.children += children.map do |value|
-          value.is_a?(Leaf) ? value.dup : value.deep_dup
-        end
-
-        dupped
-      end
-
-      # Converts the settings definitions into end children
-      # @note It runs once, after things are compiled, they will not be recompiled again
-      def compile
-        @children.each do |value|
-          # Do not redefine something that was already set during compilation
-          # This will allow us to reconfigure things and skip override with defaults
-          next if respond_to?(value.name)
-
-          singleton_class.attr_accessor value.name
-
-          initialized = if value.is_a?(Leaf)
-                          value.constructor ? value.constructor.call(value.default) : value.default
-                        else
-                          value.compile
-                          value
-                        end
-
-          public_send("#{value.name}=", initialized)
-        end
-      end
-    end
-  end
-end

data/lib/waterdrop/configurable.rb

@@ -1,71 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  # A simple dry-configuration API compatible module for defining settings with defaults and a
-  # constructor.
-  module Configurable
-    # A simple settings layer that works similar to dry-configurable
-    # It allows us to define settings on a class and per instance level with templating on a class
-    # level. It handles inheritance and allows for nested settings.
-    #
-    # @note The core settings template needs to be defined on a class level
-    class << self
-      # Sets up all the class methods and inits the core root node.
-      # Useful when only per class settings are needed as does not include instance methods
-      # @param base [Class] class that we extend
-      def extended(base)
-        base.extend ClassMethods
-      end
-
-      # Sets up all the class and instance methods and inits the core root node
-      #
-      # @param base [Class] class to which we want to add configuration
-      #
-      # Needs to be used when per instance configuration is needed
-      def included(base)
-        base.include InstanceMethods
-        base.extend self
-      end
-    end
-
-    # Instance related methods
-    module InstanceMethods
-      # @return [Node] config root node
-      def config
-        @config ||= self.class.config.deep_dup
-      end
-
-      # Allows for a per instance configuration (if needed)
-      # @param block [Proc] block for configuration
-      def configure(&block)
-        config.configure(&block)
-      end
-    end
-
-    # Class related methods
-    module ClassMethods
-      # @return [Node] root node for the settings
-      def config
-        return @config if @config
-
-        # This will handle inheritance
-        @config = if superclass.respond_to?(:config)
-                    superclass.config.deep_dup
-                  else
-                    Node.new(:root)
-                  end
-      end
-
-      # Allows for a per class configuration (if needed)
-      # @param block [Proc] block for configuration
-      def configure(&block)
-        config.configure(&block)
-      end
-
-      # Pipes the settings setup to the config root node
-      def setting(...)
-        config.setting(...)
-      end
-    end
-  end
-end

data/lib/waterdrop/contractable/contract.rb

@@ -1,183 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Contractable
-    # Base contract for all the contracts that check data format
-    #
-    # @note This contract does NOT support rules inheritance as it was never needed in Karafka
-    class Contract
-      extend Configurable
-
-      # Yaml based error messages data
-      setting(:error_messages)
-
-      # Class level API definitions
-      class << self
-        # @return [Array<Rule>] all the validation rules defined for a given contract
-        attr_reader :rules
-
-        # Allows for definition of a scope/namespace for nested validations
-        #
-        # @param path [Symbol] path in the hash for nesting
-        # @param block [Proc] nested rule code or more nestings inside
-        #
-        # @example
-        #   nested(:key) do
-        #     required(:inside) { |inside| inside.is_a?(String) }
-        #   end
-        def nested(path, &block)
-          init_accu
-          @nested << path
-          instance_eval(&block)
-          @nested.pop
-        end
-
-        # Defines a rule for a required field (required means, that will automatically create an
-        # error if missing)
-        #
-        # @param keys [Array<Symbol>] single or full path
-        # @param block [Proc] validation rule
-        def required(*keys, &block)
-          init_accu
-          @rules << Rule.new(@nested + keys, :required, block).freeze
-        end
-
-        # @param keys [Array<Symbol>] single or full path
-        # @param block [Proc] validation rule
-        def optional(*keys, &block)
-          init_accu
-          @rules << Rule.new(@nested + keys, :optional, block).freeze
-        end
-
-        # @param block [Proc] validation rule
-        #
-        # @note Virtual rules have different result expectations. Please see contracts or specs for
-        #   details.
-        def virtual(&block)
-          init_accu
-          @rules << Rule.new([], :virtual, block).freeze
-        end
-
-        private
-
-        # Initializes nestings and rules building accumulator
-        def init_accu
-          @nested ||= []
-          @rules ||= []
-        end
-      end
-
-      # Runs the validation
-      #
-      # @param data [Hash] hash with data we want to validate
-      # @return [Result] validaton result
-      def call(data)
-        errors = []
-
-        self.class.rules.map do |rule|
-          case rule.type
-          when :required
-            validate_required(data, rule, errors)
-          when :optional
-            validate_optional(data, rule, errors)
-          when :virtual
-            validate_virtual(data, rule, errors)
-          end
-        end
-
-        Result.new(errors, self)
-      end
-
-      # @param data [Hash] data for validation
-      # @param error_class [Class] error class that should be used when validation fails
-      # @return [Boolean] true
-      # @raise [StandardError] any error provided in the error_class that inherits from the
-      #   standard error
-      def validate!(data, error_class)
-        result = call(data)
-
-        return true if result.success?
-
-        raise error_class, result.errors
-      end
-
-      private
-
-      # Runs validation for rules on fields that are required and adds errors (if any) to the
-      # errors array
-      #
-      # @param data [Hash] input hash
-      # @param rule [Rule] validation rule
-      # @param errors [Array] array with errors from previous rules (if any)
-      def validate_required(data, rule, errors)
-        for_checking = dig(data, rule.path)
-
-        if for_checking.first == :match
-          result = rule.validator.call(for_checking.last, data, errors, self)
-
-          return if result == true
-
-          errors << [rule.path, result || :format]
-        else
-          errors << [rule.path, :missing]
-        end
-      end
-
-      # Runs validation for rules on fields that are optional and adds errors (if any) to the
-      # errors array
-      #
-      # @param data [Hash] input hash
-      # @param rule [Rule] validation rule
-      # @param errors [Array] array with errors from previous rules (if any)
-      def validate_optional(data, rule, errors)
-        for_checking = dig(data, rule.path)
-
-        return unless for_checking.first == :match
-
-        result = rule.validator.call(for_checking.last, data, errors, self)
-
-        return if result == true
-
-        errors << [rule.path, result || :format]
-      end
-
-      # Runs validation for rules on virtual fields (aggregates, etc) and adds errors (if any) to
-      # the errors array
-      #
-      # @param data [Hash] input hash
-      # @param rule [Rule] validation rule
-      # @param errors [Array] array with errors from previous rules (if any)
-      def validate_virtual(data, rule, errors)
-        result = rule.validator.call(data, errors, self)
-
-        return if result == true
-
-        errors.push(*result)
-      end
-
-      # Tries to dig for a given key in a hash and returns it with indication whether or not it was
-      # possible to find it (dig returns nil and we don't know if it wasn't the digged key value)
-      #
-      # @param data [Hash]
-      # @param keys [Array<Symbol>]
-      # @return [Array<Symbol, Object>] array where the first element is `:match` or `:miss` and
-      #   the digged value or nil if not found
-      def dig(data, keys)
-        current = data
-        result = :match
-
-        keys.each do |nesting|
-          unless current.key?(nesting)
-            result = :miss
-
-            break
-          end
-
-          current = current[nesting]
-        end
-
-        [result, current]
-      end
-    end
-  end
-end

data/lib/waterdrop/contractable/result.rb

@@ -1,57 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Contractable
-    # Representation of a validaton result with resolved error messages
-    class Result
-      attr_reader :errors
-
-      # Builds a result object and remaps (if needed) error keys to proper error messages
-      #
-      # @param errors [Array<Array>] array with sub-arrays with paths and error keys
-      # @param contract [Object] contract that generated the error
-      def initialize(errors, contract)
-        # Short track to skip object allocation for the happy path
-        if errors.empty?
-          @errors = errors
-          return
-        end
-
-        hashed = {}
-
-        errors.each do |error|
-          scope = error.first.map(&:to_s).join('.').to_sym
-
-          # This will allow for usage of custom messages instead of yaml keys if needed
-          hashed[scope] = if error.last.is_a?(String)
-                            error.last
-                          else
-                            build_message(contract, scope, error.last)
-                          end
-        end
-
-        @errors = hashed
-      end
-
-      # @return [Boolean] true if no errors
-      def success?
-        errors.empty?
-      end
-
-      private
-
-      # Builds message based on the error messages
-      # @param contract [Object] contract for which we build the result
-      # @param scope [Symbol] path to the key that has an error
-      # @param error_key [Symbol] error key for yaml errors lookup
-      # @return [String] error message
-      def build_message(contract, scope, error_key)
-        messages = contract.class.config.error_messages
-
-        messages.fetch(error_key.to_s) do
-          messages.fetch("#{scope}_#{error_key}")
-        end
-      end
-    end
-  end
-end

data/lib/waterdrop/contractable/rule.rb

@@ -1,8 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Contractable
-    # Representation of a single validation rule
-    Rule = Struct.new(:path, :type, :validator)
-  end
-end

data/lib/waterdrop/contractable.rb

@@ -1,13 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  # Contract layer for WaterDrop and Karafka
-  # It aims to be "dry-validation" like but smaller and easier to handle + without dependencies
-  #
-  # It allows for nested validations, etc
-  #
-  # @note It is thread-safe to run but validations definitions should happen before threads are
-  #   used.
-  module Contractable
-  end
-end