kafka_event_hub 0.1.2 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 58e8c21c528fd01d4de7680776a595be0aaf552bf833b7ac320aa3a39f8730d3
-  data.tar.gz: 2eba7b277ea24b5498f25ca42784818d31c8fee58d9a736dbdd3c8a86933f7bc
+  metadata.gz: 376525f485f73ea729e460154f69178ed5c62eab2255dd746386e5bff0e4e7f6
+  data.tar.gz: bcc93865915d1ec526cbf70032030e0a7ceca7a5aa163ba86942e42348ae3ca6
 SHA512:
-  metadata.gz: b05083fa3651f17ecb5d389e4240a3227fbbfa08c601bb74248908addeb334782c7278a77f14a0dba0c44b5efa905d361fa2a0f38cb57973cab6d6fbe5cfffcf
-  data.tar.gz: 6ce657f9f61d3155eb47a182f25a0d0aef28acf5bd53068d69d7314b27e229b1b5e6930b845d83626e93dfeb8f136282ceb62247de3abf0679fc54983e2b351e
+  metadata.gz: 2013e2c844dd9346c1e076470967aad13621db009679c77e0873ce317c91355bcd66fe68ee887272f6b1a3d4a2c800373dc6048e26beace2e8355c0e5d600f2b
+  data.tar.gz: d0d77508e2e0c7fdbf4418e7ce4e71c0e2d00e562c04fa3e89e4e693eafe8de46e92a1abe7918fc374d20755aabc848c7849ffcb1cec40a76286d67b7beb7941

data/README.md

@@ -1,43 +1,158 @@
 # KafkaEventHub
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/kafka_event_hub`. To experiment with that code, run `bin/console` for an interactive prompt.
+[![Build Status](https://github.com/marryam-shahzad/kafka_event_hub/actions/workflows/ci.yml/badge.svg)](https://github.com/marryam-shahzad/kafka_event_hub/actions)
+[![Gem Version](https://badge.fury.io/rb/kafka_event_hub.svg)](https://badge.fury.io/rb/kafka_event_hub)
 
-TODO: Delete this and the text above, and describe your gem
+KafkaEventHub is a Ruby gem providing a clean, generic, and configurable interface for producing and consuming Kafka messages in Ruby and Rails applications. It abstracts Kafka connection management and offers flexible producer and consumer classes for easy event streaming integration.
+
+---
+
+## Features
+
+- Generic Kafka Producer with topic, key, payload, and partition support.
+- Generic Kafka Consumer supporting message polling and subscription.
+- Easily configurable via environment variables or programmatically.
+- Lightweight and reusable across multiple Ruby or Rails projects.
+- Handles message serialization/deserialization seamlessly.
+- Retry mechanisms can be implemented externally in jobs or services.
+
+---
 
 ## Installation
 
 Add this line to your application's Gemfile:
 
-```ruby
-gem 'kafka_event_hub'
-```
+    gem 'kafka_event_hub'
 
-And then execute:
+Then execute:
 
-    $ bundle
+    bundle install
 
 Or install it yourself as:
 
-    $ gem install kafka_event_hub
+    gem install kafka_event_hub
+
+---
+
+## Configuration
+
+Before using the gem, set the following environment variables (adjust as per your Kafka setup):
+
+```bash
+DEFAULT_TOPIC=default           # Default Kafka topic
+EVENT_HUB_NAMESPACE=your-namespace.servicebus.windows.net:9093
+EVENT_HUB_URL="Endpoint=sb://your-namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=your-access-key"
+EVENT_HUB_CONSUMER_GROUP=Default
+APP_NAME=your-app-name
+KAFKA_POLL_TIMEOUT=120000      # in milliseconds
+```
+---
 
 ## Usage
 
-TODO: Write usage instructions here
+### Producer
+
+Create a producer instance and send messages to Kafka topics easily:
+
+    producer = KafkaEventHub::Producer.new('your-topic')
+
+    producer.produce(
+      key: 'Entity#123',
+      payload: { event: 'event_name', data: 'your data here', timestamp: Time.now }
+    )
+
+### Consumer
+
+Create a consumer instance and process incoming messages with a block:
+
+    consumer = KafkaEventHub::Consumer.new('your-topic')
+
+    consumer.each_message do |message|
+      puts "Received message: #{message.payload}"
+      # Your message processing logic here
+    end
+
+Alternatively, use `poll_messages` with a timeout (in milliseconds):
+
+    consumer.poll_messages(1000) do |message|
+      # Process each message here
+    end
+
+---
+
+## Integration with Rails Jobs
+
+You can wrap your Kafka producer and consumer in Rails ActiveJobs for background processing and retries.
+
+### Example Producer Job
+
+    class GenericKafkaProducerJob < ActiveJob::Base
+      queue_as :kafka_stream
+
+      def perform(key, payload, topic = 'default-topic', partition = 0)
+        KafkaEventHub::Producer.new(topic).produce(
+          key: key,
+          payload: payload,
+          partition: partition
+        )
+      end
+    end
+
+Customize the job code to suit your app’s background job system or retry logic.
+
+### Example Consumer Job
+
+    class GenericKafkaConsumerJob < ActiveJob::Base
+      queue_as :kafka_consumer
+
+      def perform(topic = 'default-topic')
+        consumer = KafkaEventHub::Consumer.new(topic)
+
+        consumer.each_message do |message|
+          payload = JSON.parse(message.payload)
+          # Process your payload here
+        end
+      end
+    end
+
+---
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+To set up the development environment:
+
+    bundle install
+    rake spec
+
+To build and install the gem locally:
+
+    gem build kafka_event_hub.gemspec
+    gem install ./kafka_event_hub-0.1.0.gem
+
+To release a new version:
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+1. Update the version in `lib/kafka_event_hub/version.rb`.
+2. Run:
+
+   bundle exec rake release
+
+This will create a git tag, push commits and tags, and push the gem to [RubyGems.org](https://rubygems.org).
+
+---
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/kafka_event_hub. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+Bug reports and pull requests are welcome on [GitHub](https://github.com/marryam-shahzad/kafka_event_hub).  
+Please follow the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+---
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+This gem is licensed under the [MIT License](https://opensource.org/licenses/MIT).
+
+---
 
 ## Code of Conduct
 
-Everyone interacting in the KafkaEventHub project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/kafka_event_hub/blob/master/CODE_OF_CONDUCT.md).
+Everyone participating in this project is expected to follow the [code of conduct](https://github.com/marryam-shahzad/kafka_event_hub/blob/master/CODE_OF_CONDUCT.md).

data/kafka_event_hub.gemspec

@@ -15,14 +15,16 @@ Gem::Specification.new do |spec|
     Centralizes ENV-based Rd-kafka setup into a single gem.
   DESC
 
-  # Once this gem is published, its RubyGems page will be:
-  spec.homepage      = "https://rubygems.org/gems/kafka_event_hub"
+  # Homepage and source links
+  spec.homepage      = "https://github.com/marryam-shahzad/kafka_event_hub"
   spec.license       = "MIT"
 
-  # Only allow pushing to RubyGems.org
+  # Only allow pushing to RubyGems.org and link to GitHub
   if spec.respond_to?(:metadata)
     spec.metadata["allowed_push_host"] = "https://rubygems.org"
     spec.metadata["homepage_uri"]      = spec.homepage
+    spec.metadata["source_code_uri"]   = "https://github.com/marryam-shahzad/kafka_event_hub"
+    spec.metadata["documentation_uri"] = "https://github.com/marryam-shahzad/kafka_event_hub/blob/master/README.md"
   end
 
   # Files to include in the gem
@@ -36,5 +38,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "bundler", "~> 1.17"
   spec.add_development_dependency "rake",    "~> 10.0"
   spec.add_development_dependency "rspec",   "~> 3.0"
-  spec.add_dependency 'uri', '~> 1.0.3'
+  spec.add_dependency             "uri",     "~> 1.0.3"
 end

data/lib/kafka_event_hub/consumer.rb

@@ -0,0 +1,35 @@
+# lib/kafka_event_hub/consumer.rb
+module KafkaEventHub
+  class Consumer < Config
+    def initialize(topic = nil)
+      super('consumer' => true, 'topic' => topic)
+    end
+
+    # Generic message consuming loop - pass a block to process messages
+    def each_message
+      consumer = @kafka.consumer
+      consumer.subscribe(@topic)
+
+      consumer.each do |message|
+        yield message
+      end
+    ensure
+      consumer&.close
+    end
+
+    # Optional: poll with timeout
+    def poll_messages(timeout = POLL_TIMEOUT)
+      consumer = @kafka.consumer
+      consumer.subscribe(@topic)
+
+      loop do
+        message = consumer.poll(timeout)
+        break unless message
+
+        yield message
+      end
+    ensure
+      consumer&.close
+    end
+  end
+end

data/lib/kafka_event_hub/version.rb

@@ -1,3 +1,3 @@
 module KafkaEventHub
-  VERSION = "0.1.2"
+  VERSION = "1.1.0"
 end

data/lib/kafka_event_hub.rb

@@ -3,6 +3,7 @@ require "kafka_event_hub/version"
 require_relative "kafka_event_hub/version"
 require_relative "kafka_event_hub/config"
 require_relative "kafka_event_hub/producer"
+require_relative "kafka_event_hub/consumer"
 
 
 module KafkaEventHub

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: kafka_event_hub
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 1.1.0
 platform: ruby
 authors:
 - Marryam Shahzad
@@ -106,14 +106,17 @@ files:
 - kafka_event_hub.gemspec
 - lib/kafka_event_hub.rb
 - lib/kafka_event_hub/config.rb
+- lib/kafka_event_hub/consumer.rb
 - lib/kafka_event_hub/producer.rb
 - lib/kafka_event_hub/version.rb
-homepage: https://rubygems.org/gems/kafka_event_hub
+homepage: https://github.com/marryam-shahzad/kafka_event_hub
 licenses:
 - MIT
 metadata:
   allowed_push_host: https://rubygems.org
-  homepage_uri: https://rubygems.org/gems/kafka_event_hub
+  homepage_uri: https://github.com/marryam-shahzad/kafka_event_hub
+  source_code_uri: https://github.com/marryam-shahzad/kafka_event_hub
+  documentation_uri: https://github.com/marryam-shahzad/kafka_event_hub/blob/master/README.md
 post_install_message: 
 rdoc_options: []
 require_paths: