phobos 1.9.0.pre.beta2 → 2.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f2b2f9eeb44e40d9cb01add121e7b388c66e99f5482e5f2ec78bb7bc3263bd32
-  data.tar.gz: 796484971553a2b19edc18f0e4192ffaece1de0c27894bfa0344b0af29fc3f23
+  metadata.gz: 77558904b4cc06321149de51f2b13edeb97704a93128749214ba8454ff997db3
+  data.tar.gz: c88ded6d626d55f56eef5645147f102867097bbc5d32c8c4d3604a24b53f8d57
 SHA512:
-  metadata.gz: 0f2f422bff9b56354c1378c283090277a1af164b2b8a3f9a038c85a25157131e1e2cd005d37e72ba21d921620521210020859368b42851dc110043395052ab11
-  data.tar.gz: dbd23e7210111e63921854bf279c0f64aa9a4ba426f24af897924a1c6a76cb24a19be4c35db1105013c0d0de3c6a6e2bb73272df6cce9b24e4dce9c9a445becf
+  metadata.gz: e2c541adb24d1a42be9c99958508ebc143c078c3294df9245094c954de8c5d240be89fdf8a1afcf05ed3ad556290b2a7928b5722b4e1b56a25b2095f9e3d2b69
+  data.tar.gz: 49af36a25cc903e7bcc87e6a7a453dfd32127111ce99896b77bf31f50c1967cc8550b9e3740d47d42febaae44ffa92c7dd5e3959c96a0b637bcc5bb20ddf0c66

data/.gitignore

@@ -12,3 +12,4 @@ spec/examples.txt
 config/*.yml
 log/*.log
 .byebug_history
+.idea

data/CHANGELOG.md

@@ -6,6 +6,28 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 ``
 ## UNRELEASED
 
+## [2.0.2] - 2021-01-28
+- Additional fixes for Ruby 3.0
+
+## [2.0.1] - 2021-01-14
+- Fix bug with Ruby 3.0 around OpenStruct
+
+## [2.0.0-beta1] - 2020-05-04
+
+- Remove deprecated patterns:
+-- `before_consume` method
+-- `around_consume` as a class method or without yielding values
+-- `publish` and `async_publish` with positional arguments
+
+## [1.9.0] - 2020-03-05
+- Bumped version to 1.9.0.
+
+## [1.9.0-beta3] - 2020-02-05
+
+- Fix bug where deprecation errors would be shown when receiving nil payloads
+  even if `around_consume` was updated to yield them.
+
+
 ## [1.9.0-beta2] - 2020-01-09
 
 - Allow `around_consume` to yield payload and metadata, and deprecate

data/README.md

@@ -21,7 +21,6 @@ With Phobos by your side, all this becomes smooth sailing.
 ## Table of Contents
 
 1. [Installation](#installation)
-  1. [Upgrade Notes](#upgrade-notes)
 1. [Usage](#usage)
   1. [Standalone apps](#usage-standalone-apps)
   1. [Consuming messages from Kafka](#usage-consuming-messages-from-kafka)
@@ -32,6 +31,7 @@ With Phobos by your side, all this becomes smooth sailing.
 1. [Plugins](#plugins)
 1. [Development](#development)
 1. [Test](#test)
+1. [Upgrade Notes](#upgrade-notes)
 
 ## <a name="installation"></a> Installation
 
@@ -53,10 +53,6 @@ Or install it yourself as:
 $ gem install phobos
 ```
 
-### <a name="upgrade-notes"></a> Upgrade Notes
-
-Version 1.8.2 introduced a new `persistent_connections` setting for regular producers. This reduces the number of connections used to produce messages and you should consider setting it to true. This does require a manual shutdown call -  please see [Producers with persistent connections](#persistent-connection).
-
 ## <a name="usage"></a> Usage
 
 Phobos can be used in two ways: as a standalone application or to support Kafka features in your existing project - including Rails apps. It provides a CLI tool to run it.
@@ -197,8 +193,6 @@ class MyHandler
 end
 ```
 
-Note: Previous versions used a `before_consume` method to pre-process the payload. This is still supported, but deprecated. Going forward, `around_consume` should yield the payload and metadata back to the calling code, allowing it to act as a pre-processor, e.g. by decoding Avro messages into Ruby hashes.
-
 Take a look at the examples folder for some ideas.
 
 The hander life cycle can be illustrated as:
@@ -287,8 +281,6 @@ my = MyProducer.new
 my.producer.publish(topic: 'topic', payload: 'message-payload', key: 'partition and message key', headers: { header_1: 'value 1' })
 ```
 
-Older versions of Phobos provided a `publish` method that accepted positional arguments. That version is still supported but it's soon to be deprecated in favour of the keyword arguments version.
-
 It is also possible to publish several messages at once:
 
 ```ruby
@@ -604,8 +596,16 @@ Phobos exports a spec helper that can help you test your consumer. The Phobos li
 * `process_message(handler:, payload:, metadata: {}, encoding: nil)` - Invokes your handler with payload and metadata, using a dummy listener (encoding and metadata are optional).
 
 ```ruby
-require 'spec_helper'
+### spec_helper.rb
+require 'phobos/test/helper'
+RSpec.configure do |config|
+  config.include Phobos::Test::Helper
+  config.before(:each) do
+    Phobos.configure(path_to_my_config_file)
+  end
+end 
 
+### Spec file
 describe MyConsumer do
   let(:payload) { 'foo' }
   let(:metadata) { Hash(foo: 'bar') }
@@ -619,6 +619,57 @@ describe MyConsumer do
 end
 ```
 
+## <a name="upgrade-notes"></a> Upgrade Notes
+
+Version 2.0 removes deprecated ways of defining producers and consumers:
+* The `before_consume` method has been removed. You can have this behavior in the first part of an `around_consume` method.
+* `around_consume` is now only available as an instance method, and it must yield the values to pass to the `consume` method.
+* `publish` and `async_publish` now only accept keyword arguments, not positional arguments.
+
+Example pre-2.0:
+```ruby
+class MyHandler
+  include Phobos::Handler
+
+  def before_consume(payload, metadata)
+    payload[:id] = 1
+  end
+
+  def self.around_consume(payload, metadata)
+    metadata[:key] = 5
+    yield
+  end
+end
+```
+
+In 2.0:
+```ruby
+class MyHandler
+  include Phobos::Handler
+
+  def around_consume(payload, metadata)
+    new_payload = payload.dup
+    new_metadata = metadata.dup
+    new_payload[:id] = 1
+    new_metadata[:key] = 5
+    yield new_payload, new_metadata
+  end
+end
+```
+
+Producer, 1.9:
+```ruby
+  producer.publish('my-topic', { payload_value: 1}, 5, 3, {header_val: 5})
+```
+
+Producer 2.0:
+```ruby
+  producer.publish(topic: 'my-topic', payload: { payload_value: 1}, key: 5, 
+     partition_key: 3, headers: { header_val: 5})
+```
+
+Version 1.8.2 introduced a new `persistent_connections` setting for regular producers. This reduces the number of connections used to produce messages and you should consider setting it to true. This does require a manual shutdown call -  please see [Producers with persistent connections](#persistent-connection).
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/klarna/phobos.

data/lib/phobos.rb

@@ -67,7 +67,7 @@ module Phobos
     end
 
     def create_kafka_client
-      Kafka.new(config.kafka.to_hash.merge(logger: @ruby_kafka_logger))
+      Kafka.new(**config.kafka.to_hash.merge(logger: @ruby_kafka_logger))
     end
 
     def create_exponential_backoff(backoff_config = nil)

data/lib/phobos/actions/process_batch_inline.rb

@@ -47,32 +47,13 @@ module Phobos
         )
       end
 
-      def preprocess(batch, handler)
-        if handler.respond_to?(:before_consume_batch)
-          Phobos.deprecate('before_consume_batch is deprecated and will be removed in 2.0. \
-                            Use around_consume_batch and yield payloads and metadata objects.')
-          handler.before_consume_batch(batch, @metadata)
-        else
-          batch
-        end
-      end
-
       def process_batch(batch)
         instrument('listener.process_batch_inline', @metadata) do |_metadata|
           handler = @listener.handler_class.new
 
-          preprocessed_batch = preprocess(batch, handler)
-          consume_block = proc { |around_batch, around_metadata|
-            if around_batch
-              handler.consume_batch(around_batch, around_metadata)
-            else
-              Phobos.deprecate('Calling around_consume_batch without yielding payloads \
-                                and metadata is deprecated and will be removed in 2.0.')
-              handler.consume_batch(preprocessed_batch, @metadata)
-            end
-          }
-
-          handler.around_consume_batch(preprocessed_batch, @metadata, &consume_block)
+          handler.around_consume_batch(batch, @metadata) do |around_batch, around_metadata|
+            handler.consume_batch(around_batch, around_metadata)
+          end
         end
       end
     end

data/lib/phobos/actions/process_message.rb

@@ -35,47 +35,12 @@ module Phobos
 
       private
 
-      def preprocess(payload, handler)
-        if handler.respond_to?(:before_consume)
-          Phobos.deprecate('before_consume is deprecated and will be removed in 2.0. \
-                            Use around_consume and yield payload and metadata objects.')
-          begin
-            handler.before_consume(payload, @metadata)
-          rescue ArgumentError
-            handler.before_consume(payload)
-          end
-        else
-          payload
-        end
-      end
-
-      def consume_block(payload, handler)
-        proc { |around_payload, around_metadata|
-          if around_payload
-            handler.consume(around_payload, around_metadata)
-          else
-            Phobos.deprecate('Calling around_consume without yielding payload and metadata \
-                              is deprecated and will be removed in 2.0.')
-            handler.consume(payload, @metadata)
-          end
-        }
-      end
-
       def process_message(payload)
         instrument('listener.process_message', @metadata) do
           handler = @listener.handler_class.new
 
-          preprocessed_payload = preprocess(payload, handler)
-          block = consume_block(preprocessed_payload, handler)
-
-          if @listener.handler_class.respond_to?(:around_consume)
-            # around_consume class method implementation
-            Phobos.deprecate('around_consume has been moved to instance method, please update '\
-                             'your consumer. This will not be backwards compatible in the future.')
-            @listener.handler_class.around_consume(preprocessed_payload, @metadata, &block)
-          else
-            # around_consume instance method implementation
-            handler.around_consume(preprocessed_payload, @metadata, &block)
+          handler.around_consume(payload, @metadata) do |around_payload, around_metadata|
+            handler.consume(around_payload, around_metadata)
           end
         end
       end

data/lib/phobos/deep_struct.rb

@@ -8,7 +8,7 @@ module Phobos
     # Based on
     # https://docs.omniref.com/ruby/2.3.0/files/lib/ostruct.rb#line=88
     def initialize(hash = nil)
-      @table = {}
+      super
       @hash_table = {}
 
       hash&.each_pair do |key, value|

data/lib/phobos/executor.rb

@@ -13,7 +13,7 @@ module Phobos
         max_concurrency = listener_configs[:max_concurrency] || 1
         Array.new(max_concurrency).map do
           configs = listener_configs.select { |k| Constants::LISTENER_OPTS.include?(k) }
-          Phobos::Listener.new(configs.merge(handler: handler_class))
+          Phobos::Listener.new(**configs.merge(handler: handler_class))
         end
       end
     end

data/lib/phobos/listener.rb

@@ -93,7 +93,7 @@ module Phobos
     def start_listener
       instrument('listener.start', listener_metadata) do
         @consumer = create_kafka_consumer
-        @consumer.subscribe(topic, @subscribe_opts)
+        @consumer.subscribe(topic, **@subscribe_opts)
 
         # This is done here because the producer client is bound to the current thread and
         # since "start" blocks a thread might be used to call it
@@ -135,7 +135,7 @@ module Phobos
     end
 
     def consume_each_batch
-      @consumer.each_batch(@message_processing_opts) do |batch|
+      @consumer.each_batch(**@message_processing_opts) do |batch|
         batch_processor = Phobos::Actions::ProcessBatch.new(
           listener: self,
           batch: batch,
@@ -149,7 +149,7 @@ module Phobos
     end
 
     def consume_each_batch_inline
-      @consumer.each_batch(@message_processing_opts) do |batch|
+      @consumer.each_batch(**@message_processing_opts) do |batch|
         batch_processor = Phobos::Actions::ProcessBatchInline.new(
           listener: self,
           batch: batch,
@@ -163,7 +163,7 @@ module Phobos
     end
 
     def consume_each_message
-      @consumer.each_message(@message_processing_opts) do |message|
+      @consumer.each_message(**@message_processing_opts) do |message|
         message_processor = Phobos::Actions::ProcessMessage.new(
           listener: self,
           message: message,
@@ -181,7 +181,7 @@ module Phobos
         Constants::KAFKA_CONSUMER_OPTS.include?(k)
       end
       configs.merge!(@kafka_consumer_opts)
-      @kafka_client.consumer({ group_id: group_id }.merge(configs))
+      @kafka_client.consumer(**{ group_id: group_id }.merge(configs))
     end
 
     def compact(hash)

data/lib/phobos/producer.rb

@@ -11,28 +11,24 @@ module Phobos
     end
 
     class PublicAPI
-      MissingRequiredArgumentsError = Class.new(StandardError) do
-        def initialize
-          super('You need to provide a topic name and a payload')
-        end
-      end
-
       def initialize(host_obj)
         @host_obj = host_obj
       end
 
-      def publish(*args, **kwargs)
-        Phobos.deprecate(deprecate_positional_args_message('publish')) if kwargs.empty?
-
-        args = normalize_arguments(*args, **kwargs)
-        class_producer.publish(**args)
+      def publish(topic:, payload:, key: nil, partition_key: nil, headers: nil)
+        class_producer.publish(topic: topic,
+                               payload: payload,
+                               key: key,
+                               partition_key: partition_key,
+                               headers: headers)
       end
 
-      def async_publish(*args, **kwargs)
-        Phobos.deprecate(deprecate_positional_args_message('async_publish')) if kwargs.empty?
-
-        args = normalize_arguments(*args, **kwargs)
-        class_producer.async_publish(**args)
+      def async_publish(topic:, payload:, key: nil, partition_key: nil, headers: nil)
+        class_producer.async_publish(topic: topic,
+                                     payload: payload,
+                                     key: key,
+                                     partition_key: partition_key,
+                                     headers: headers)
       end
 
       # @param messages [Array(Hash(:topic, :payload, :key, :headers))]
@@ -54,36 +50,6 @@ module Phobos
       def class_producer
         @host_obj.class.producer
       end
-
-      # rubocop:disable Metrics/ParameterLists
-      def normalize_arguments(p_topic = nil, p_payload = nil, p_key = nil,
-                              p_partition_key = nil, p_headers = {},
-                              **kwargs)
-        {}.tap do |args|
-          {
-            topic: p_topic,
-            payload: p_payload,
-            key: p_key,
-            partition_key: p_partition_key,
-            headers: p_headers
-          }.each { |k, v| args[k] = kwargs[k] || v }
-
-          raise MissingRequiredArgumentsError if [:topic, :payload].any? { |k| args[k].nil? }
-
-          kwargs.each do |k, v|
-            next if args.key?(k)
-
-            args[:headers][k] = v
-          end
-        end
-      end
-      # rubocop:enable Metrics/ParameterLists
-
-      def deprecate_positional_args_message(method_name)
-        "The `#{method_name}` method should now receive keyword arguments " \
-          'rather than positional ones. Please update your publishers. This will ' \
-          'not be backwards compatible in the future.'
-      end
     end
 
     module ClassMethods
@@ -112,7 +78,7 @@ module Phobos
 
         def create_sync_producer
           client = kafka_client || configure_kafka_client(Phobos.create_kafka_client)
-          sync_producer = client.producer(regular_configs)
+          sync_producer = client.producer(**regular_configs)
           if Phobos.config.producer_hash[:persistent_connections]
             producer_store[:sync_producer] = sync_producer
           end
@@ -143,7 +109,7 @@ module Phobos
 
         def create_async_producer
           client = kafka_client || configure_kafka_client(Phobos.create_kafka_client)
-          async_producer = client.async_producer(async_configs)
+          async_producer = client.async_producer(**async_configs)
           producer_store[:async_producer] = async_producer
         end
 

data/lib/phobos/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Phobos
-  VERSION = '1.9.0-beta2'
+  VERSION = '2.0.2'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: phobos
 version: !ruby/object:Gem::Version
-  version: 1.9.0.pre.beta2
+  version: 2.0.2
 platform: ruby
 authors:
 - Túlio Ornelas
@@ -12,10 +12,10 @@ authors:
 - Francisco Juan
 - Tommy Gustafsson
 - Daniel Orner
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-01-09 00:00:00.000000000 Z
+date: 2021-01-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -299,7 +299,7 @@ licenses:
 - Apache License Version 2.0
 metadata:
   allowed_push_host: https://rubygems.org
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -310,12 +310,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '2.3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
-rubygems_version: 3.0.2
-signing_key: 
+rubygems_version: 3.2.3
+signing_key:
 specification_version: 4
 summary: Simplifying Kafka for ruby apps
 test_files: []