ears 0.3.1 → 0.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8d16a56d89359232938fc2e04641e7db0f45440bd6cee7770b8e7d0be30fb014
-  data.tar.gz: fd3f8ce9db2ec52e646593b57c0f0b868d13fc5edb09c66e4ee7f345e49cd605
+  metadata.gz: 3f0fa1b69a334f5ec5d649478a46d0c78298621a2c879c3b62457b8dd41723fe
+  data.tar.gz: e34d3bee6f15435ca5c9847b384aad75e6acd3ccd06dbb9a751a798f63f3a077
 SHA512:
-  metadata.gz: d36a71c0a123865108370bdfa167a33128331b35626556794becb972b4ead16368df897f4cb6c998926c5b353e095580a67f7687cc111a2f54c2c80f84b10fb9
-  data.tar.gz: 8ed9d81e0da9cc807f15a81faf72dc88763e0482d31db53fc0f8b720be032bb3ccc657e5caa03bbefdf1816e2b69dd069230f2ac6a223cb56814c4237ee3d8f9
+  metadata.gz: 4d914d9a7e5ce22b6595899952626bbbec3b1885a6ff0b4fbf768560d4382dadb64578a9c305d0573afba36918e0eccd8c704ccfb681a76f606a09cf8b23b2c6
+  data.tar.gz: f6f95145e4c70a565cd95755602116755e79000ca530883aadcf8d9461aacdf3d6828bf67533ed7f7458471473972839ef5745b1e44c933705678be119260f63

data/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## 0.3.2 (2021-05-21)
+
+### Changes
+
+- added YARD documentation and usage instructions to README
+- introduced `Ears::Middleware` as an abstract base class for middlewares
+
 ## 0.3.1 (2021-05-07)
 
 ### Changes

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    ears (0.3.1)
+    ears (0.3.2)
       bunny
       multi_json
 
@@ -19,6 +19,7 @@ GEM
       ast (~> 2.4.1)
     rainbow (3.0.0)
     rake (13.0.3)
+    redcarpet (3.5.1)
     regexp_parser (2.1.1)
     rexml (3.2.4)
     rspec (3.10.0)
@@ -52,6 +53,7 @@ GEM
       rubocop-ast (>= 1.1.0)
     ruby-progressbar (1.11.0)
     unicode-display_width (2.0.0)
+    yard (0.9.26)
 
 PLATFORMS
   x86_64-darwin-20
@@ -60,10 +62,12 @@ PLATFORMS
 DEPENDENCIES
   ears!
   rake
+  redcarpet
   rspec
   rubocop
   rubocop-rake
   rubocop-rspec
+  yard
 
 BUNDLED WITH
    2.2.3

data/README.md

@@ -1,6 +1,6 @@
 # Ears
 
-TODO: Write description here
+`Ears` is a small, simple library for writing RabbitMQ consumers.
 
 ## Installation
 
@@ -20,16 +20,141 @@ Or install it yourself as:
 
 ## Usage
 
-TODO: Write usage instructions here
+### Basic usage
+
+First, you should configure where `Ears` should connect to.
+
+```ruby
+require 'ears'
+
+Ears.configure do |config|
+  config.rabbitmq_url = 'amqp://user:password@myrmq:5672'
+end
+```
+
+Next, define your exchanges, queues, and consumers by calling `Ears.setup`.
+
+```ruby
+Ears.setup do
+  # define a durable topic exchange
+  my_exchange = exchange('my_exchange', :topic, durable: true)
+
+  # define a queue
+  my_queue = queue('my_queue', durable: true)
+
+  # bind your queue to the exchange
+  my_queue.bind(my_exchange, routing_key: 'my.routing.key')
+
+  # define a consumer that handles messages for that queue
+  consumer(my_queue, MyConsumer)
+end
+```
+
+Finally, you need to implement `MyConsumer` by subclassing `Ears::Consumer`.
+
+```ruby
+class MyConsumer < Ears::Consumer
+  def work(delivery_info, metadata, payload)
+    message = JSON.parse(payload)
+    do_stuff(message)
+
+    ack
+  end
+end
+```
+
+At the end of the `#work` method, you must always return `ack`, `reject`, or `requeue` to signal what should be done with the message.
+
+### Middlewares
+
+`Ears` supports middlewares that you can use for recurring tasks that you don't always want to reimplement. It comes with some built-in middlewares:
+
+- `Ears::JSON` for automatically parsing JSON payloads
+- `Ears::Appsignal` for automatically wrapping `#work` in an Appsignal transaction
+
+You can use a middleware by just calling `use` with the middleware you want to register in your consumer.
+
+```ruby
+require 'ears/middlewares/json'
+
+class MyConsumer < Ears::Consumer
+  # register the JSON middleware and don't symbolize keys (this can be omitted, the default is true)
+  use Ears::Middlewares::JSON, symbolize_keys: false
+
+  def work(delivery_info, metadata, payload)
+    return ack unless payload['data'].nil? # this now just works
+  end
+end
+```
+
+If you want to implement your own middleware, just subclass `Ears::Middleware` and implement `#call` (and if you need it `#initialize`).
+
+```ruby
+class MyMiddleware < Ears::Middleware
+  def initialize(opts = {})
+    @my_option = opts.fetch(:my_option, nil)
+  end
+
+  def call(delivery_info, metadata, payload, app)
+    do_stuff
+
+    # always call the next middleware in the chain or your consumer will never be called
+    app.call(delivery_info, metadata, payload)
+  end
+end
+```
+
+### Multiple threads
+
+If you need to handle a lot of messages, you might want to have multiple instances of the same consumer all working on a dedicated thread. This is supported out of the box. You just have to define how many consumers you want when calling `consumer` in `Ears.setup`.
+
+```ruby
+Ears.setup do
+  my_exchange = exchange('my_exchange', :topic, durable: true)
+  my_queue = queue('my_queue', durable: true)
+  my_queue.bind(my_exchange, routing_key: 'my.routing.key')
+
+  # this will instantiate MyConsumer 10 times and run every instance on a dedicated thread
+  consumer(my_queue, MyConsumer, 10)
+end
+```
+
+It may also be interesting for you to increase the prefetch amount. The default prefetch amount is 1, but if you have a lot of very small, fast to process messages, a higher prefetch is a good idea. Just set it when defining your consumer.
+
+```ruby
+Ears.setup do
+  my_exchange = exchange('my_exchange', :topic, durable: true)
+  my_queue = queue('my_queue', durable: true)
+  my_queue.bind(my_exchange, routing_key: 'my.routing.key')
+
+  # this will instantiate one consumer but with a prefetch value of 10
+  consumer(my_queue, MyConsumer, 1, prefetch: 10)
+end
+```
+
+### Setting arbitrary exchange/queue parameters
+
+If you need some custom arguments on your exchange or queue, you can just pass these to `queue` or `exchange` inside `Ears.setup`. These are then just passed on to `Bunny::Queue` and `Bunny::Exchange`.
+
+```ruby
+Ears.setup do
+  my_queue =
+    queue('my_queue', durable: true, arguments: { 'x-message-ttl' => 10_000 })
+end
+```
+
+## Documentation
+
+If you need more in-depth information, look at [our API documentation](https://www.rubydoc.info/gems/ears).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/ears. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/ears/blob/master/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome on GitHub at https://github.com/ivx/ears. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/ivx/ears/blob/master/CODE_OF_CONDUCT.md).
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+The gem is available as open-source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
 
 ## Code of Conduct
 
-Everyone interacting in the Ears project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/ears/blob/master/CODE_OF_CONDUCT.md).
+Everyone interacting in the Ears project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the [code of conduct](https://github.com/ivx/ears/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -1,6 +1,9 @@
 require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
+require 'yard'
 
 RSpec::Core::RakeTask.new(:spec)
 
+YARD::Rake::YardocTask.new { |t| t.files = ['lib/**/*.rb'] }
+
 task default: :spec

data/ears.gemspec

@@ -33,8 +33,10 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'multi_json'
 
   spec.add_development_dependency 'rake'
+  spec.add_development_dependency 'redcarpet'
   spec.add_development_dependency 'rspec'
   spec.add_development_dependency 'rubocop'
   spec.add_development_dependency 'rubocop-rake'
   spec.add_development_dependency 'rubocop-rspec'
+  spec.add_development_dependency 'yard'
 end

data/lib/ears.rb

@@ -1,26 +1,35 @@
 require 'bunny'
 require 'ears/configuration'
 require 'ears/consumer'
+require 'ears/middleware'
 require 'ears/setup'
 require 'ears/version'
 
 module Ears
-  class Error < StandardError
-  end
-
   class << self
+    # The global configuration for Ears.
+    #
+    # @return [Ears::Configuration]
     def configuration
       @configuration ||= Ears::Configuration.new
     end
 
+    # Yields the global configuration instance so you can modify it.
+    # @yieldparam configuration [Ears::Configuration] The global configuration instance.
     def configure
       yield(configuration)
     end
 
+    # The global RabbitMQ connection used by Ears.
+    #
+    # @return [Bunny::Session]
     def connection
       @connection ||= Bunny.new.tap { |conn| conn.start }
     end
 
+    # The channel for the current thread.
+    #
+    # @return [Bunny::Channel]
     def channel
       Thread.current[:ears_channel] ||=
         connection
@@ -31,10 +40,13 @@ module Ears
           end
     end
 
+    # Used to set up your exchanges, queues and consumers. See {Ears::Setup} for implementation details.
     def setup(&block)
       Ears::Setup.new.instance_eval(&block)
     end
 
+    # Blocks the calling thread until +SIGTERM+ or +SIGINT+ is received.
+    # Used to keep the process alive while processing messages.
     def run!
       running = true
       Signal.trap('INT') { running = false }
@@ -42,6 +54,7 @@ module Ears
       sleep 1 while running
     end
 
+    # Used internally for testing.
     def reset!
       @connection = nil
       Thread.current[:ears_channel] = nil

data/lib/ears/configuration.rb

@@ -1,7 +1,9 @@
 module Ears
+  # The class representing the global {Ears} configuration.
   class Configuration
     DEFAULT_RABBITMQ_URL = 'amqp://guest:guest@localhost:5672'
 
+    # @return [String] the connection string for RabbitMQ.
     attr_accessor :rabbitmq_url
 
     def initialize

data/lib/ears/consumer.rb

@@ -1,7 +1,10 @@
 require 'bunny'
 
 module Ears
+  # The abstract base class for consumers processing messages from queues.
+  # @abstract Subclass and override {#work} to implement.
   class Consumer
+    # Error that is raised when an invalid value is returned from {#work}
     class InvalidReturnError < StandardError
       def initialize(value)
         super(
@@ -10,18 +13,38 @@ module Ears
       end
     end
 
+    # List of registered middlewares. Register new middlewares with {.use}.
+    # @return [Array<Ears::Middleware>]
     def self.middlewares
       @middlewares ||= []
     end
 
+    # Registers a new middleware by instantiating +middleware+ and passing it +opts+.
+    #
+    # @param [Class<Ears::Middleware>] middleware The middleware class to instantiate and register.
+    # @param [Hash] opts The options for instantiating the middleware.
     def self.use(middleware, opts = {})
       middlewares << middleware.new(opts)
     end
 
+    # The method that is called when a message from the queue is received.
+    # Keep in mind that the parameters received can be altered by middlewares!
+    #
+    # @param [Bunny::DeliveryInfo] delivery_info The delivery info of the message.
+    # @param [Bunny::MessageProperties] metadata The metadata of the message.
+    # @param [String] payload The payload of the message.
+    #
+    # @return [:ack, :reject, :requeue] A symbol denoting what should be done with the message.
     def work(delivery_info, metadata, payload)
       raise NotImplementedError
     end
 
+    # Wraps #work to add middlewares. This is being called by Ears when a message is received for the consumer.
+    #
+    # @param [Bunny::DeliveryInfo] delivery_info The delivery info of the received message.
+    # @param [Bunny::MessageProperties] metadata The metadata of the received message.
+    # @param [String] payload The payload of the received message.
+    # @raise [InvalidReturnError] if you return something other than +:ack+, +:reject+ or +:requeue+ from {#work}.
     def process_delivery(delivery_info, metadata, payload)
       self.class.middlewares.reverse.reduce(
         work_proc,
@@ -32,14 +55,24 @@ module Ears
 
     protected
 
+    # Helper method to ack a message.
+    #
+    # @return [:ack]
     def ack
       :ack
     end
 
+    # Helper method to reject a message.
+    #
+    # @return [:reject]
+    #
     def reject
       :reject
     end
 
+    # Helper method to requeue a message.
+    #
+    # @return [:requeue]
     def requeue
       :requeue
     end

data/lib/ears/consumer_wrapper.rb

@@ -1,12 +1,23 @@
 require 'bunny'
 
 module Ears
+  # Wraps the user-defined consumer to provide the expected interface to Bunny.
   class ConsumerWrapper < Bunny::Consumer
+    # @param [Ears::Consumer] consumer The user-defined consumer implementation derived from {Ears::Consumer}.
+    # @param [Bunny::Channel] channel The channel used for the consumer.
+    # @param [Bunny::Queue] queue The queue the consumer is subscribed to.
+    # @param [String] consumer_tag A string identifying the consumer instance.
+    # @param [Hash] arguments Arguments that are passed on to +Bunny::Consumer.new+.
     def initialize(consumer, channel, queue, consumer_tag, arguments = {})
       @consumer = consumer
       super(channel, queue, consumer_tag, false, false, arguments)
     end
 
+    # Called when a message is received from the subscribed queue.
+    #
+    # @param [Bunny::DeliveryInfo] delivery_info The delivery info of the received message.
+    # @param [Bunny::MessageProperties] metadata The metadata of the received message.
+    # @param [String] payload The payload of the received message.
     def process_delivery(delivery_info, metadata, payload)
       consumer
         .process_delivery(delivery_info, metadata, payload)

data/lib/ears/middleware.rb

@@ -0,0 +1,15 @@
+module Ears
+  # The abstract base class for middlewares.
+  # @abstract Subclass and override {#call} (and maybe +#initialize+) to implement.
+  class Middleware
+    # Invokes the middleware.
+    #
+    # @param [Bunny::DeliveryInfo] delivery_info The delivery info of the received message.
+    # @param [Bunny::MessageProperties] metadata The metadata of the received message.
+    # @param [String] payload The payload of the received message.
+    # @param app The next middleware to call or the actual consumer instance.
+    def call(delivery_info, metadata, payload, app)
+      raise NotImplementedError
+    end
+  end
+end

data/lib/ears/middlewares/appsignal.rb

@@ -1,9 +1,14 @@
+require 'ears/middleware'
+
 module Ears
   module Middlewares
-    class Appsignal
-      attr_reader :transaction_name, :class_name, :method
-
+    # A middleware that automatically wraps {Ears::Consumer#work} in an Appsignal transaction.
+    class Appsignal < Middleware
+      # @param [Hash] opts The options for the middleware.
+      # @option opts [String] :transaction_name The name of the Appsignal transaction.
+      # @option opts [String] :class_name The name of the class you want to monitor.
       def initialize(opts)
+        super()
         @transaction_name = opts.fetch(:transaction_name)
         @class_name = opts.fetch(:class_name)
       end
@@ -16,6 +21,10 @@ module Ears
           queue_start: Time.now.utc,
         ) { app.call(delivery_info, metadata, payload) }
       end
+
+      private
+
+      attr_reader :transaction_name, :class_name
     end
   end
 end

data/lib/ears/middlewares/json.rb

@@ -1,11 +1,14 @@
+require 'ears/middleware'
 require 'multi_json'
 
 module Ears
   module Middlewares
-    class JSON
-      attr_reader :symbolize_keys
-
+    # A middleware that automatically parses your JSON payload.
+    class JSON < Middleware
+      # @param [Hash] opts The options for the middleware.
+      # @option opts [Boolean] :symbolize_keys Whether to symbolize the keys of your payload.
       def initialize(opts = {})
+        super()
         @symbolize_keys = opts.fetch(:symbolize_keys, true)
       end
 
@@ -13,6 +16,10 @@ module Ears
         parsed_payload = MultiJson.load(payload, symbolize_keys: symbolize_keys)
         app.call(delivery_info, metadata, parsed_payload)
       end
+
+      private
+
+      attr_reader :symbolize_keys
     end
   end
 end

data/lib/ears/setup.rb

@@ -3,15 +3,34 @@ require 'ears/consumer'
 require 'ears/consumer_wrapper'
 
 module Ears
+  # Contains methods used in {Ears.setup} to set up your exchanges, queues and consumers.
   class Setup
+    # Creates a new exchange if it does not already exist.
+    #
+    # @param [String] name The name of the exchange.
+    # @param [Symbol] type The type of the exchange (:direct, :fanout, :topic or :headers).
+    # @param [Hash] opts The options for the exchange. These are passed on to +Bunny::Exchange.new+.
+    # @return [Bunny::Exchange] The exchange that was either newly created or was already there.
     def exchange(name, type, opts = {})
       Bunny::Exchange.new(Ears.channel, type, name, opts)
     end
 
+    # Creates a new queue if it does not already exist.
+    #
+    # @param [String] name The name of the queue.
+    # @param [Hash] opts The options for the queue. These are passed on to +Bunny::Exchange.new+.
+    # @return [Bunny::Queue] The queue that was either newly created or was already there.
     def queue(name, opts = {})
       Bunny::Queue.new(Ears.channel, name, opts)
     end
 
+    # Creates and starts one or many consumers bound to the given queue.
+    #
+    # @param [Bunny::Queue] queue The queue the consumers should be subscribed to.
+    # @param [Class<Ears::Consumer>] consumer_class A class implementing {Ears::Consumer} that holds the consumer behavior.
+    # @param [Integer] threads The number of threads that should be used to process messages from the queue.
+    # @param [Hash] args The arguments for the consumer. These are passed on to +Bunny::Consumer.new+.
+    # @option args [Integer] :prefetch (1) The prefetch count used for this consumer.
     def consumer(queue, consumer_class, threads = 1, args = {})
       threads.times do |n|
         consumer_queue = create_consumer_queue(queue, args)

data/lib/ears/version.rb

@@ -1,3 +1,3 @@
 module Ears
-  VERSION = '0.3.1'
+  VERSION = '0.3.2'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ears
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.3.2
 platform: ruby
 authors:
 - Mario Mainz
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-05-07 00:00:00.000000000 Z
+date: 2021-05-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bunny
@@ -52,6 +52,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: redcarpet
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -108,6 +122,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A gem for building RabbitMQ consumers.
 email:
 - mario.mainz@invision.de
@@ -132,6 +160,7 @@ files:
 - lib/ears/configuration.rb
 - lib/ears/consumer.rb
 - lib/ears/consumer_wrapper.rb
+- lib/ears/middleware.rb
 - lib/ears/middlewares/appsignal.rb
 - lib/ears/middlewares/json.rb
 - lib/ears/setup.rb