pheromone 0.5.0 → 0.5.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 3e21ddb46a7940af3aa6428f01c768b484c8cd08
-  data.tar.gz: e25484b58e3f22a92d4829c17c850d45e6207273
+  metadata.gz: fdb8d44235a3ccf8f853460d9b49ea652a0bd3ce
+  data.tar.gz: 1c4694ebbc1e013a37159b88733ad56bcbfb5af9
 SHA512:
-  metadata.gz: d222ca050b37759c2a85d3d5aff1d3e3a95a29fdf741dc916762cde97ed95ce67632fe443c1eafaf74e5771fc6dbdab7e2d9def782f8129f108d4da142fade11
-  data.tar.gz: 6b18b8348d1a54b2e5a9b3b4c9e87ccb85aebba4908c7a03b5378c305bc18e9dbc91ab07113348ed3da0a56274ff97eb57cefefa98166d8b268aad4a838c110d
+  metadata.gz: 1c5e66513aa2078d9f94aa62f0e0c3f3b9a73fb571d3b8ae87584141e4dcb0a09b012a8a745f60dfe30c6952b857b1e7d36b76e0a717f3b607d9f0bde963b55d
+  data.tar.gz: 0313d08e66fed868d7aaa3781a9482e3b6102f1f02f3bcc735723030200d790147eb213bf2be74bdbac3ba35d6a57fca09652f6a0935b57a688c92fce214ff48

data/README.md

@@ -36,7 +36,7 @@ Pheromone.setup do |pheromone_config|
   # pheromone_config.background_processor.klass = 'BackgroundWorker'
   # pheromone_config.timezone = 'UTC'
   pheromone_config.message_format = :json
-    WaterDrop.setup do |waterdrop_config|    
+    WaterDrop.setup do |waterdrop_config|
       waterdrop_config.deliver = Rails.env.production?
       waterdrop_config.kafka.seed_brokers = [Rails.env.production? ? ENV['KAFKA_HOST'] : 'localhost:9092']
     end
@@ -46,10 +46,11 @@ end
 Edit this file to modify the default config. The following configuration options are available:
 
 
-| Option                        | Value type    | Description                      | 
+| Option                        | Value type    | Description                      |
 |-------------------------------|---------------|----------------------------------|
 | background_processor.name     | Symbol        | Choose :sidekiq or :resque as the background processor only if messages need to be sent to kafka asynchronously |
 | background_processor.klass    | String        | Background processor class name that sends messages to kafka |
+| background_processor.custom_processor |Proc| Custom processor (only works when you specify name as :custom) |
 | timezone_format               | String        | Valid timezone name for timestamps sent to kafka |
 | message_format                | Symbol        | Only supports :json format currently |
 | enabled                       | Boolean       | Defaults to true. When this is set to false, no messages will be sent to Kafka |
@@ -66,7 +67,7 @@ The underlying Kafka client used by `pheromone` is `ruby-kafka`. This client pro
 It is advisable to use the normal producer in production systems because async producer provides no guarantees that the messages will be delivered. To read more on this, refer the `ruby-kafka` [documentation](https://github.com/zendesk/ruby-kafka#asynchronously-producing-messages)
 
 Even while using a synchronous producer, sometimes there might be a need to run send messages to Kafka in a background task. This is especially true for batch processing tasks that send a high message volume to Kafka. To allow for this, `pheromone` provides an `async` mode that can be specified as an option to `publish` by specifying `dispatch_method` as `:async`. By default, `dispatch_method` will be `:sync`. Specifying `:async` will still use the normal producer and NOT the async_producer.
-  
+
 ```
 class PublishableModel < ActiveRecord::Base
   include Pheromone::Publishable
@@ -117,6 +118,18 @@ Create a new class and add the name under `Pheromone.config.background_processor
    end
  end
 ```
+#### 1.c. Implement your own processor
+You can also implement your own processor in addition to resque and sidekiq.
+```ruby
+  Pheromone.setup do |config|
+    config.background_processor.name = :custom
+    config.background_processor.klass = 'CustomJob'
+    config.background_processor.custom_processor = -> (klass, msg) do
+      klass.perform(msg)
+    end
+  end
+```
+
 `pheromone` will invoke the class name specified in the config with the message object. This mode can be used if you don't want to block a request that ends up sending messages to Kafka.
 
 ### 2. Supported events
@@ -300,7 +313,7 @@ end
 
 Pheromone makes it easier to standardise the format of messages sent to Kafka while affording the flexibility to add other custom fields using the metadata option.
 
-By default, the messages sent to Kafka have an event_type and timestamp attached to it as shown below: 
+By default, the messages sent to Kafka have an event_type and timestamp attached to it as shown below:
 ```
   {
     'event_type' => 'create',
@@ -333,7 +346,7 @@ end
 ```
 
 The Kafka message will have the original metadata in addition to the new fields:
- 
+
 
 ```
   {
@@ -351,15 +364,15 @@ The Kafka message will have the original metadata in addition to the new fields:
 `pheromone` provides a custom message object that sends messages to Kafka in a predefined format, to maintain consistency in the message fields.
 
 `Pheromone::Messaging::Message` can be initialized with the following arguments:
- - `topic`: name of the topic to which the message is produced 
+ - `topic`: name of the topic to which the message is produced
  - `blob`: the actual message itself
  - `metadata`: any additional fields that must be sent along with the message
  - `options`: producer options as described in Section 6
- 
+
  Of these fields, only `topic` and `message` are compulsory and the remaining two are optional.
- 
+
  Example usage:
- 
+
  ```
    Pheromone::Messaging::Message.new(
      topic: 'test_topic',
@@ -368,9 +381,9 @@ The Kafka message will have the original metadata in addition to the new fields:
      producer_options: { max_retries: 5 }
    ).send!
  ```
- 
- This will send a message to `test_topic` in Kafka in the following format: 
- 
+
+ This will send a message to `test_topic` in Kafka in the following format:
+
  ```
   {
     'event_type' => 'create',
@@ -389,9 +402,9 @@ As seen above `timestamp` will be added automatically to the main attributes alo
 
 `publish` takes the `encoder` and `message_format` as options for encoding to be specified. Encoder is called with the entire message object and performs encoding on this message.
 
-```        
+```
   publish(
-    [ 
+    [
       {
         topic: :test_topic,
         event_types: [:update, :create],
@@ -419,7 +432,7 @@ The message published to Kafka looks like this:
    }
  }
  ```
- From `0.5` version onwards, the message format looks like this: 
+ From `0.5` version onwards, the message format looks like this:
  ```
  {
    metadata: {
@@ -435,10 +448,10 @@ The message published to Kafka looks like this:
  `event`, `entity`, and `timestamp` are determined and added by `pheromone`. `timestamp` will be in UTC by default and will use `timezone_format` value specified in `config/initializers/pheromone.rb`. Contents of `metadata` can be modified by using these [guidelines](https://github.com/ankitagupta12/pheromone#7-specifying-message-metadata). Message contents are placed under the key `blob`.
 
   In order to use the format with `blob` embedded inside `metadata`, specify option `embed_blob` as `true` inside `publish` options like this:
-  
-  ```        
+
+  ```
   publish(
-    [ 
+    [
       {
         topic: :test_topic,
         event_types: [:update, :create],
@@ -456,17 +469,17 @@ The message published to Kafka looks like this:
 #### RSpec
 
 `pheromone` makes it easy to control when it is enabled during testing with `pheromone/frameworks/rspec`
- 
+
  ```
  # spec/rails_helper.rb
  require 'rspec/rails'
  # ...
  require 'pheromone/frameworks/rspec'
- 
+
  ```
- 
+
  With this helper, `publishable` is disabled by default. To turn it on, pass `publishable: true` to the spec block like this:
- 
+
  ```
  describe 'PublishableModel' do
    before do
@@ -474,7 +487,7 @@ The message published to Kafka looks like this:
      allow(WaterDrop::SyncProducer).to receive(:call) do
      @invocation_count += 1
    end
-    
+
    it 'sends messages to kafka', publishable: true do
      PublishableModel.create
      expect(@invocation_count).to eq(1)
@@ -482,7 +495,7 @@ The message published to Kafka looks like this:
  end
  ```
  Alternatively, `Pheromone.enabled?` can be used to check if `pheromone` is enabled
- 
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/lib/pheromone/config.rb

@@ -8,10 +8,15 @@ module Pheromone
     setting :message_format, :json
     setting :enabled, true
     setting :background_processor do
+      # type: symbol
       # accepts :sidekiq or :resque as a value
       setting :name
+      # type: string
       # specify the background job handling message send to kafka
       setting :klass
+      # type: proc/lambda
+      # specify custom background job
+      setting :custom_processor
     end
     # timezone names should match a valid timezone defined here:
     # http://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html
@@ -25,4 +30,4 @@ module Pheromone
       end
     end
   end
-end
+end

data/lib/pheromone/messaging/message_dispatcher.rb

@@ -9,6 +9,8 @@ require 'pheromone/messaging/message'
 module Pheromone
   module Messaging
     class MessageDispatcher
+      class NoSpecifiedProcessor < StandardError; end
+
       def initialize(message_parameters:, dispatch_method:)
         @message_parameters = message_parameters
         @dispatch_method = dispatch_method
@@ -38,6 +40,10 @@ module Pheromone
           Resque.enqueue(background_processor_klass, message_body)
         elsif background_processor.name == :sidekiq
           background_processor_klass.perform_async(message_body)
+        elsif background_processor.name == :custom
+          background_processor.custom_processor.call(background_processor_klass, message_body)
+        else
+          raise NoSpecifiedProcessor
         end
       end
 
@@ -60,4 +66,4 @@ module Pheromone
       end
     end
   end
-end
+end

data/lib/pheromone/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Pheromone
-  VERSION = '0.5.0'.freeze
+  VERSION = '0.5.1'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pheromone
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.5.1
 platform: ruby
 authors:
 - Ankita Gupta
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-06-18 00:00:00.000000000 Z
+date: 2018-09-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: active_model_serializers