pheromone 0.0.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 8a1139f6f89ed56b3c6403f3ec53e6e484129306
+  data.tar.gz: 6d57ed3461f221dbb859e909c78843b1996d249a
+SHA512:
+  metadata.gz: db3e5e83ddfb4dd2f95ee7502fc958dcad89d38754ff901c8b3894b37503a6021bbeb7e68e1f55dad3fc34279017da71bb34adbc50611e4713bec2c0ed34be32
+  data.tar.gz: 64cba76f28557a3cdb5672ef29539a5c21650a9cebc42e1edabe7419f13ac554d280946db82de90b6194375f7d38e9357d3c99a74de5a9733c1984d72709abbf

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,49 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, and in the interest of
+fostering an open and welcoming community, we pledge to respect all people who
+contribute through reporting issues, posting feature requests, updating
+documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free
+experience for everyone, regardless of level of experience, gender, gender
+identity and expression, sexual orientation, disability, personal appearance,
+body size, race, ethnicity, age, religion, or nationality.
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery
+* Personal attacks
+* Trolling or insulting/derogatory comments
+* Public or private harassment
+* Publishing other's private information, such as physical or electronic
+  addresses, without explicit permission
+* Other unethical or unprofessional conduct
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+By adopting this Code of Conduct, project maintainers commit themselves to
+fairly and consistently applying these principles to every aspect of managing
+this project. Project maintainers who do not follow or enforce the Code of
+Conduct may be permanently removed from the project team.
+
+This code of conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting a project maintainer at ankitagupta12391@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. Maintainers are
+obligated to maintain confidentiality with regard to the reporter of an
+incident.
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 1.3.0, available at
+[http://contributor-covenant.org/version/1/3/0/][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/3/0/

data/README.md

@@ -0,0 +1,211 @@
+# Pheromone
+
+`pheromone` provides a utility function to easily send `ActiveRecord` updates to Kafka. 
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'pheromone'
+```
+
+And then execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install pheromone
+
+## Waterdrop Setup
+
+Pheromone depends on `waterdrop` to send messages to Kafka. `waterdrop` settings can be added by following the Setup step on [waterdrop](https://github.com/karafka/waterdrop/blob/master/README.md)
+
+WaterDrop has following configuration options:
+
+| Option                  | Value type    | Description                      |
+|-------------------------|---------------|----------------------------------|
+| send_messages           | Boolean       | Should we send messages to Kafka |
+| kafka.hosts             | Array<String> | Kafka servers hosts with ports   |
+| connection_pool_size    | Integer       | Kafka connection pool size       |
+| connection_pool_timeout | Integer       | Kafka connection pool timeout    |
+| raise_on_failure        | Boolean       | Should we raise an exception when we cannot send message to Kafka - if false will silently ignore failures (will just ignore them) |
+
+To apply this configuration, you need to use a *setup* method:
+
+```ruby
+WaterDrop.setup do |config|
+  config.send_messages = true
+  config.connection_pool_size = 20
+  config.connection_pool_timeout = 1
+  config.kafka.hosts = ['localhost:9092']
+  config.raise_on_failure = true
+end
+```
+
+This configuration can be placed in *config/initializers* and can vary based on the environment:
+
+```ruby
+WaterDrop.setup do |config|
+  config.send_messages = Rails.env.production?
+  config.connection_pool_size = 20
+  config.connection_pool_timeout = 1
+  config.kafka.hosts = [Rails.env.production? ? 'prod-host:9091' : 'localhost:9092']
+  config.raise_on_failure = Rails.env.production?
+end
+```
+ 
+## Usage
+
+### 1. Supported events
+#### 1.a. To send messages for model `create` event, add the following lines to your ActiveRecord model
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:create],
+      topic: :topic1,
+      message: ->(obj) { { name: obj.name } }
+    }
+  ]
+end
+```
+
+#### 1.b. To send messages for model `update` event, specify `update` in the `event_types` array:
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:update],
+      topic: :topic1,
+      message: ->(obj) { { name: obj.name } }
+    }
+  ]
+end
+``` 
+
+Messages can be published for multiple event types by defining `events_types: [:create, :update]`.
+
+### 2. Supported message formats
+
+#### 2.a. Using a proc in `message` 
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:create],
+      topic: :topic1,
+      message: ->(obj) { { name: obj.name } }
+    }
+  ]
+end
+```
+
+#### 2.a. Using a defined function in `message`
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:update],
+      topic: :topic1,
+      message: message 
+    }
+  ]
+  
+  def message
+    { name: self.name }
+  end
+end
+```
+
+### 3. Sending messages conditionally
+
+#### 3.a. Using a proc in `if`
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:update],
+      topic: :topic1,
+      message: message,
+      if: ->(data) { data.condition },
+    }
+  ] 
+   
+  def message
+    { name: self.name }
+  end
+end
+```
+#### 3.b. Using a defined function in `if`
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:update],
+      topic: :topic1,
+      message: message,
+      if: pre_condition
+    }
+  ] 
+  
+  def pre_condition
+    name.present?
+  end
+  
+  def message
+    { name: self.name }
+  end
+end
+```
+
+### 4. Specifying the topic
+
+The kafka topic can be specified in the `topic` option to `publish`. To publish to `topic_test`, use the following:
+
+
+```
+class PublishableModel < ActiveRecord::Base
+  include Pheromone
+  publish message_options: [
+    {
+      event_types: [:create],
+      topic: :topic_test,
+      message: ->(obj) { { name: obj.name } }
+    }
+  ]
+end
+```
+
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+
+=======
+# pheromone
+

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "pheromone"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/pheromone/options_validator.rb

@@ -0,0 +1,58 @@
+# validate message options provided to publish method in Publishable concern
+class OptionsValidator
+  ACCEPTED_EVENT_TYPES = [:create, :update].freeze
+
+  def initialize(message_options)
+    @errors = {}
+    @message_options = message_options
+  end
+
+  def validate
+    validate_message_options
+    return @errors if @errors.present?
+    validate_topic
+    validate_event_types
+    validate_message_attributes
+    @errors
+  end
+
+  private
+
+  def validate_message_options
+    return if @message_options.is_a?(Array)
+    add_error_message(:message_options, 'Message options should be an array')
+  end
+
+  def validate_topic
+    return if @message_options.all? { |options| options[:topic].present? }
+    add_error_message(:topic, 'Topic name missing')
+  end
+
+  # :reek:FeatureEnvy
+  def validate_event_types
+    return if @message_options.all? do |options|
+      event_types = options[:event_types]
+      next true unless event_types
+      event_types.present? &&
+          event_types.is_a?(Array) &&
+          (event_types - ACCEPTED_EVENT_TYPES).empty?
+    end
+
+    add_error_message(
+        :event_types,
+        "Event types must be a non-empty array with types #{ACCEPTED_EVENT_TYPES.join(',')}"
+    )
+  end
+
+  def validate_message_attributes
+    return if @message_options.all? do |options|
+      options[:serializer].present? || options[:message].present?
+    end
+
+    add_error_message(:message_attributes, 'Either serializer or message should be specified')
+  end
+
+  def add_error_message(key, value)
+    @errors.merge!(key => value)
+  end
+end

data/lib/pheromone/setup/waterdrop.rb

@@ -0,0 +1,7 @@
+WaterDrop.setup do |config|
+  config.send_messages = true
+  config.connection_pool_size = ENV['KAFKA_CONNECTION_POOL_SIZE'] || 20
+  config.connection_pool_timeout = ENV['KAFKA_CONNECTION_POOL_TIMEOUT'] || 1
+  config.kafka.hosts = (ENV['KAFKA_HOSTS'] || '').split(',')
+  config.raise_on_failure = true
+end

data/lib/pheromone/version.rb

@@ -0,0 +1,3 @@
+module Pheromone
+  VERSION = '0.0.2'
+end

data/lib/pheromone.rb

@@ -0,0 +1,98 @@
+require 'pheromone/version'
+require 'pheromone/options_validator'
+require 'active_support'
+
+# Usage: For publishing messages to kafka, include this concern
+# in the model and then add
+#
+#  include Publishable
+#  publish message_options: [
+#            {
+#              topic: :topic1,
+#              event_types: [:create, :update],
+#              message: { a: 1, b: 2 }
+#            },....
+#          ]
+#
+# Each entry in message_options will be registered as a potential
+# message to be published to kafka on the after_commit hook of the
+# including model. message_options can take an optional if: key which
+# accepts either a callback or an instance method name - which can be
+# used to decide if the message should be published or not.
+#
+# To control how the model is serialized before being published to kafka
+# either provide a Serializer via the `serializer` key or a callback or instance
+# method name via the `message` key
+
+module Pheromone
+  extend ActiveSupport::Concern
+
+  # class methods for the model including Publishable
+  module ClassMethods
+    def publish(message_options:)
+      errors = OptionsValidator.new(message_options).validate
+      raise "Errors: #{errors}" unless errors.empty?
+      __send__(:after_commit, proc { dispatch_messages(message_options: message_options) })
+    end
+  end
+
+  def dispatch_messages(message_options:)
+    message_options.each do |options|
+      next unless check_conditions(options)
+      begin
+        send_message(options)
+      rescue => error
+        puts error
+      end
+    end
+  end
+
+  private
+
+  def check_conditions(options)
+    condition_callback = options[:if]
+    return check_event(options) unless condition_callback
+    call_proc_or_instance_method(condition_callback)
+  end
+
+  def check_event(options)
+    options[:event_types].any? { |event| event == current_event }
+  end
+
+  def send_message(options)
+    WaterDrop::Message.new(
+      options[:topic],
+      message_meta_data.merge!(blob: message_blob(options)).to_json
+    ).send!
+  end
+
+  def message_meta_data
+    {
+      event: current_event,
+      entity: self.class.name,
+      timestamp: Time.now
+    }
+  end
+
+  def current_event
+    id_previously_changed? ? :create : :update
+  end
+
+  def message_blob(options)
+    message = options[:message]
+    return call_proc_or_instance_method(message) if message
+    # options[:serializer].new(self, options[:serializer_options] || {}).serializable_object
+  end
+
+  # This method has the :reek:ManualDispatch smell,
+  # which is difficult to avoid since it handles
+  # either a lambda/Proc or a named method from the including
+  # class.
+  def call_proc_or_instance_method(proc_or_symbol)
+    return proc_or_symbol.call(self) if proc_or_symbol.respond_to?(:call)
+    unless respond_to? proc_or_symbol
+      raise "Method #{proc_or_symbol} not found for #{self.class.name}"
+    end
+    __send__(proc_or_symbol)
+  end
+end

metadata

@@ -0,0 +1,164 @@
+--- !ruby/object:Gem::Specification
+name: pheromone
+version: !ruby/object:Gem::Version
+  version: 0.0.2
+platform: ruby
+authors:
+- Ankita Gupta
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2017-05-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: waterdrop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.3.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.3.2.1
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+- !ruby/object:Gem::Dependency
+  name: timecop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec-rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+- !ruby/object:Gem::Dependency
+  name: with_model
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+description: Sends messages to kafka using different formats and strategies
+email:
+- ankitagupta12391@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CODE_OF_CONDUCT.md
+- README.md
+- bin/console
+- bin/setup
+- lib/pheromone.rb
+- lib/pheromone/options_validator.rb
+- lib/pheromone/setup/waterdrop.rb
+- lib/pheromone/version.rb
+homepage: https://github.com/ankitagupta12/pheromone
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.7
+signing_key: 
+specification_version: 4
+summary: Transmits messages to kafka from active record
+test_files: []