waterdrop 2.0.0 → 2.4.0

data/LICENSE

@@ -1,165 +0,0 @@
-                   GNU LESSER GENERAL PUBLIC LICENSE
-                       Version 3, 29 June 2007
-
- Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
-
-  This version of the GNU Lesser General Public License incorporates
-the terms and conditions of version 3 of the GNU General Public
-License, supplemented by the additional permissions listed below.
-
-  0. Additional Definitions.
-
-  As used herein, "this License" refers to version 3 of the GNU Lesser
-General Public License, and the "GNU GPL" refers to version 3 of the GNU
-General Public License.
-
-  "The Library" refers to a covered work governed by this License,
-other than an Application or a Combined Work as defined below.
-
-  An "Application" is any work that makes use of an interface provided
-by the Library, but which is not otherwise based on the Library.
-Defining a subclass of a class defined by the Library is deemed a mode
-of using an interface provided by the Library.
-
-  A "Combined Work" is a work produced by combining or linking an
-Application with the Library.  The particular version of the Library
-with which the Combined Work was made is also called the "Linked
-Version".
-
-  The "Minimal Corresponding Source" for a Combined Work means the
-Corresponding Source for the Combined Work, excluding any source code
-for portions of the Combined Work that, considered in isolation, are
-based on the Application, and not on the Linked Version.
-
-  The "Corresponding Application Code" for a Combined Work means the
-object code and/or source code for the Application, including any data
-and utility programs needed for reproducing the Combined Work from the
-Application, but excluding the System Libraries of the Combined Work.
-
-  1. Exception to Section 3 of the GNU GPL.
-
-  You may convey a covered work under sections 3 and 4 of this License
-without being bound by section 3 of the GNU GPL.
-
-  2. Conveying Modified Versions.
-
-  If you modify a copy of the Library, and, in your modifications, a
-facility refers to a function or data to be supplied by an Application
-that uses the facility (other than as an argument passed when the
-facility is invoked), then you may convey a copy of the modified
-version:
-
-   a) under this License, provided that you make a good faith effort to
-   ensure that, in the event an Application does not supply the
-   function or data, the facility still operates, and performs
-   whatever part of its purpose remains meaningful, or
-
-   b) under the GNU GPL, with none of the additional permissions of
-   this License applicable to that copy.
-
-  3. Object Code Incorporating Material from Library Header Files.
-
-  The object code form of an Application may incorporate material from
-a header file that is part of the Library.  You may convey such object
-code under terms of your choice, provided that, if the incorporated
-material is not limited to numerical parameters, data structure
-layouts and accessors, or small macros, inline functions and templates
-(ten or fewer lines in length), you do both of the following:
-
-   a) Give prominent notice with each copy of the object code that the
-   Library is used in it and that the Library and its use are
-   covered by this License.
-
-   b) Accompany the object code with a copy of the GNU GPL and this license
-   document.
-
-  4. Combined Works.
-
-  You may convey a Combined Work under terms of your choice that,
-taken together, effectively do not restrict modification of the
-portions of the Library contained in the Combined Work and reverse
-engineering for debugging such modifications, if you also do each of
-the following:
-
-   a) Give prominent notice with each copy of the Combined Work that
-   the Library is used in it and that the Library and its use are
-   covered by this License.
-
-   b) Accompany the Combined Work with a copy of the GNU GPL and this license
-   document.
-
-   c) For a Combined Work that displays copyright notices during
-   execution, include the copyright notice for the Library among
-   these notices, as well as a reference directing the user to the
-   copies of the GNU GPL and this license document.
-
-   d) Do one of the following:
-
-       0) Convey the Minimal Corresponding Source under the terms of this
-       License, and the Corresponding Application Code in a form
-       suitable for, and under terms that permit, the user to
-       recombine or relink the Application with a modified version of
-       the Linked Version to produce a modified Combined Work, in the
-       manner specified by section 6 of the GNU GPL for conveying
-       Corresponding Source.
-
-       1) Use a suitable shared library mechanism for linking with the
-       Library.  A suitable mechanism is one that (a) uses at run time
-       a copy of the Library already present on the user's computer
-       system, and (b) will operate properly with a modified version
-       of the Library that is interface-compatible with the Linked
-       Version.
-
-   e) Provide Installation Information, but only if you would otherwise
-   be required to provide such information under section 6 of the
-   GNU GPL, and only to the extent that such information is
-   necessary to install and execute a modified version of the
-   Combined Work produced by recombining or relinking the
-   Application with a modified version of the Linked Version. (If
-   you use option 4d0, the Installation Information must accompany
-   the Minimal Corresponding Source and Corresponding Application
-   Code. If you use option 4d1, you must provide the Installation
-   Information in the manner specified by section 6 of the GNU GPL
-   for conveying Corresponding Source.)
-
-  5. Combined Libraries.
-
-  You may place library facilities that are a work based on the
-Library side by side in a single library together with other library
-facilities that are not Applications and are not covered by this
-License, and convey such a combined library under terms of your
-choice, if you do both of the following:
-
-   a) Accompany the combined library with a copy of the same work based
-   on the Library, uncombined with any other library facilities,
-   conveyed under the terms of this License.
-
-   b) Give prominent notice with the combined library that part of it
-   is a work based on the Library, and explaining where to find the
-   accompanying uncombined form of the same work.
-
-  6. Revised Versions of the GNU Lesser General Public License.
-
-  The Free Software Foundation may publish revised and/or new versions
-of the GNU Lesser General Public License from time to time. Such new
-versions will be similar in spirit to the present version, but may
-differ in detail to address new problems or concerns.
-
-  Each version is given a distinguishing version number. If the
-Library as you received it specifies that a certain numbered version
-of the GNU Lesser General Public License "or any later version"
-applies to it, you have the option of following the terms and
-conditions either of that published version or of any later version
-published by the Free Software Foundation. If the Library as you
-received it does not specify a version number of the GNU Lesser
-General Public License, you may choose any version of the GNU Lesser
-General Public License ever published by the Free Software Foundation.
-
-  If the Library as you received it specifies that a proxy can decide
-whether future versions of the GNU Lesser General Public License shall
-apply, that proxy's public statement of acceptance of any version is
-permanent authorization for you to choose that version for the
-Library.

data/lib/water_drop/contracts/config.rb

@@ -1,26 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Contracts
-    # Contract with validation rules for WaterDrop configuration details
-    class Config < Dry::Validation::Contract
-      # Ensure valid format of each seed broker so that rdkafka doesn't fail silently
-      SEED_BROKER_FORMAT_REGEXP = %r{\A([^:/,]+:[0-9]+)(,[^:/,]+:[0-9]+)*\z}.freeze
-
-      private_constant :SEED_BROKER_FORMAT_REGEXP
-
-      params do
-        required(:id).filled(:str?)
-        required(:logger).filled
-        required(:deliver).filled(:bool?)
-        required(:max_payload_size).filled(:int?, gteq?: 1)
-        required(:max_wait_timeout).filled(:number?, gteq?: 0)
-        required(:wait_timeout).filled(:number?, gt?: 0)
-
-        required(:kafka).schema do
-          required(:'bootstrap.servers').filled(:str?, format?: SEED_BROKER_FORMAT_REGEXP)
-        end
-      end
-    end
-  end
-end

data/lib/water_drop/contracts/message.rb

@@ -1,41 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  module Contracts
-    # Contract with validation rules for validating that all the message options that
-    # we provide to producer ale valid and usable
-    class Message < Dry::Validation::Contract
-      # Regex to check that topic has a valid format
-      TOPIC_REGEXP = /\A(\w|-|\.)+\z/.freeze
-
-      # Checks, that the given value is a string
-      STRING_ASSERTION = ->(value) { value.is_a?(String) }.to_proc
-
-      private_constant :TOPIC_REGEXP, :STRING_ASSERTION
-
-      config.messages.load_paths << File.join(WaterDrop.gem_root, 'config', 'errors.yml')
-
-      option :max_payload_size
-
-      params do
-        required(:topic).filled(:str?, format?: TOPIC_REGEXP)
-        required(:payload).filled(:str?)
-        optional(:key).maybe(:str?, :filled?)
-        optional(:partition).filled(:int?, gteq?: -1)
-        optional(:timestamp).maybe { time? | int? }
-        optional(:headers).maybe(:hash?)
-      end
-
-      rule(:headers) do
-        next unless value.is_a?(Hash)
-
-        key.failure(:invalid_key_type) unless value.keys.all?(&STRING_ASSERTION)
-        key.failure(:invalid_value_type) unless value.values.all?(&STRING_ASSERTION)
-      end
-
-      rule(:payload) do
-        key.failure(:max_payload_size) if value.bytesize > max_payload_size
-      end
-    end
-  end
-end

data/lib/water_drop/instrumentation.rb

@@ -1,7 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  # Namespace for all the things related with WaterDrop instrumentation process
-  module Instrumentation
-  end
-end

data/lib/water_drop/producer/builder.rb

@@ -1,63 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  class Producer
-    # Class used to construct the rdkafka producer client
-    class Builder
-      # @param producer [Producer] not yet configured producer for which we want to
-      #   build the client
-      # @param config [Object] dry-configurable based configuration object
-      # @return [Rdkafka::Producer, Producer::DummyClient] raw rdkafka producer or a dummy producer
-      #   when we don't want to dispatch any messages
-      def call(producer, config)
-        return DummyClient.new unless config.deliver
-
-        Rdkafka::Config.logger = config.logger
-        Rdkafka::Config.statistics_callback = build_statistics_callback(producer, config.monitor)
-
-        client = Rdkafka::Config.new(config.kafka.to_h).producer
-        client.delivery_callback = build_delivery_callback(producer, config.monitor)
-        client
-      end
-
-      private
-
-      # Creates a proc that we want to run upon each successful message delivery
-      #
-      # @param producer [Producer]
-      # @param monitor [Object] monitor we want to use
-      # @return [Proc] delivery callback
-      def build_delivery_callback(producer, monitor)
-        lambda do |delivery_report|
-          monitor.instrument(
-            'message.acknowledged',
-            producer: producer,
-            offset: delivery_report.offset,
-            partition: delivery_report.partition
-          )
-        end
-      end
-
-      # Creates a proc that we want to run upon each statistics callback execution
-      #
-      # @param producer [Producer]
-      # @param monitor [Object] monitor we want to use
-      # @return [Proc] statistics callback
-      # @note We decorate the statistics with our own decorator because some of the metrics from
-      #   rdkafka are absolute. For example number of sent messages increases not in reference to
-      #   previous statistics emit but from the beginning of the process. We decorate it with diff
-      #   of all the numeric values against the data from the previous callback emit
-      def build_statistics_callback(producer, monitor)
-        statistics_decorator = StatisticsDecorator.new
-
-        lambda do |statistics|
-          monitor.instrument(
-            'statistics.emitted',
-            producer: producer,
-            statistics: statistics_decorator.call(statistics)
-          )
-        end
-      end
-    end
-  end
-end

data/lib/water_drop/producer/statistics_decorator.rb

@@ -1,71 +0,0 @@
-# frozen_string_literal: true
-
-module WaterDrop
-  class Producer
-    # Many of the librdkafka statistics are absolute values instead of a gauge.
-    # This means, that for example number of messages sent is an absolute growing value
-    # instead of being a value of messages sent from the last statistics report.
-    # This decorator calculates the diff against previously emited stats, so we get also
-    # the diff together with the original values
-    class StatisticsDecorator
-      def initialize
-        @previous = {}.freeze
-      end
-
-      # @param emited_stats [Hash] original emited statistics
-      # @return [Hash] emited statistics extended with the diff data
-      # @note We modify the emited statistics, instead of creating new. Since we don't expose
-      #   any API to get raw data, users can just assume that the result of this decoration is the
-      #   proper raw stats that they can use
-      def call(emited_stats)
-        diff(
-          @previous,
-          emited_stats
-        )
-
-        @previous = emited_stats
-
-        emited_stats.freeze
-      end
-
-      private
-
-      # Calculates the diff of the provided values and modifies in place the emited statistics
-      #
-      # @param previous [Object] previous value from the given scope in which
-      #   we are
-      # @param current [Object] current scope from emitted statistics
-      # @return [Object] the diff if the values were numerics or the current scope
-      def diff(previous, current)
-        if current.is_a?(Hash)
-          # @note We cannot use #each_key as we modify the content of the current scope
-          #   in place (in case it's a hash)
-          current.keys.each do |key|
-            append(
-              current,
-              key,
-              diff((previous || {})[key], (current || {})[key])
-            )
-          end
-        end
-
-        if current.is_a?(Numeric) && previous.is_a?(Numeric)
-          current - previous
-        else
-          current
-        end
-      end
-
-      # Appends the result of the diff to a given key as long as the result is numeric
-      #
-      # @param current [Hash] current scope
-      # @param key [Symbol] key based on which we were diffing
-      # @param result [Object] diff result
-      def append(current, key, result)
-        return unless result.is_a?(Numeric)
-
-        current["#{key}_d"] = result
-      end
-    end
-  end
-end

data/lib/water_drop.rb

@@ -1,30 +0,0 @@
-# frozen_string_literal: true
-
-# External components
-# delegate should be removed because we don't need it, we just add it because of ruby-kafka
-%w[
-  concurrent/array
-  dry-configurable
-  dry/monitor/notifications
-  dry-validation
-  rdkafka
-  json
-  zeitwerk
-  securerandom
-].each { |lib| require lib }
-
-# WaterDrop library
-module WaterDrop
-  class << self
-    # @return [String] root path of this gem
-    def gem_root
-      Pathname.new(File.expand_path('..', __dir__))
-    end
-  end
-end
-
-Zeitwerk::Loader
-  .for_gem
-  .tap { |loader| loader.ignore("#{__dir__}/waterdrop.rb") }
-  .tap(&:setup)
-  .tap(&:eager_load)