chasqui 1.0.0.pre.rc1 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: e2a3b1a24a648748e026c9938d71f77aeda97a7f
-  data.tar.gz: cf1b2e7b5d261c31980b46edc42fcff4048374a2
+  metadata.gz: 529cc6f7637b86eb3f4e6f2186e17432b766b9a0
+  data.tar.gz: 4d0d17f7b715880207944527b13828d7272f78eb
 SHA512:
-  metadata.gz: bedd779b43a73f7df0021bf31c19aa686093432e9488f7b1d32be64a066defc788b040596d19a662ddca6715245c36f0d2eb5a810b3ad26eaaf2d05ceceefdf1
-  data.tar.gz: a06828b09503066ea658447bcc0af0099339be201538421e94923bf138c8d25926eae97264c99291aca5259be3dfcf53fccf72ca6acf8f580ab092322f1ae48f
+  metadata.gz: 2ca9133c707e182c426b59e8d94cd6bc4c95836e21ef152b55e22194cb129eeeedaccab5e1dd1aeed285d9ce36944791417c5367671c15d9940a2b6f4fe5bebd
+  data.tar.gz: 8aeff03699366f0c44c73d077ee868412cca52b0dc61016f517aa7ae2c4df3807ffe613888214f7155b23852e306e24f71ff82c072e5d1e9d6fdf6ac9435fc90

data/.gitignore

@@ -13,3 +13,4 @@
 *.a
 mkmf.log
 .vagrant/
+.ruby-version

data/README.md

@@ -3,138 +3,150 @@
 
 # Chasqui
 
-Chasqui is a simple, lightweight, persistent implementation of the
-publish-subscribe (pub-sub) messaging pattern for service oriented
-architectures.
-
-Chasqui delivers messages to subscribers in a Resque-compatible format. If you
-are already using Resque and/or Sidekiq, Chasqui will make a wonderful
-companion to your architecture.
+Chasqui adds persistent
+[publish-subscribe](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern)
+(pub-sub) messaging capabilities to Sidekiq and Resque workers.
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Add this line to your application's Gemfile
 
     gem 'chasqui'
 
-And then execute:
+then execute
 
     $ bundle
 
-Or install it yourself as:
+or install it yourself as
 
     $ gem install chasqui
 
 ## Dependencies
 
-Chasqui uses [Redis](http://redis.io/) to queue events and manage
+Chasqui uses [Redis](http://redis.io/) to store events and manage
 subscriptions. You can install Redis with your favorite package manager, such
 as homebrew, yum, or apt, or if you prefer, you can run `vagrant up` to run
-Redis in a virtual machine.
+Redis in a virtual machine. If you already have Resque or Sidekiq working, then
+you already have everything you need to get started with Chasqui.
 
 ## Quick Start
 
-Chasqui consists of two components - a client and a broker. Clients can both
-publish to a channel and subscribe to events on one or more channels. The
-broker transforms incoming events into Resque (or Sidekiq) jobs and places them
-on one or more queues according to the currently registered subscribers. Under
-the hood, a subscriber is simply a Sidekiq/Resque worker that processes jobs
-placed on a queue.
-
 ### Start the broker
 
-    chasqui -r redis://localhost:6379/0 -q my-app
+    chasqui -r redis://localhost:6379/0
 
-Your broker must use the same Redis connection as your Sidekiq/Resque workers.
-For a list of available broker options, see `chasqui --help`.
+The broker is a ruby daemon that listens for events (messages) published to
+channels (topics) and forwards those events to registered subscribers.  In
+order to work, your broker must use the same Redis database as your
+Sidekiq/Resque workers. For a list of available broker options, see `chasqui
+--help`.
 
-### Publish events
+### Publish an events
 
-Publishing events is simple.
+    Chasqui.publish 'order.purchased', user, order
 
-    # file: publisher.rb
-    require 'chasqui'
-    Chasqui.publish 'user.sign-up', name: 'Darth Vader'
+Publish an event to the `order.purchased` channel and include information
+about the user and order that triggered the event. Any arguments after the
+channel name must be JSON-serializable.
 
-Be sure to run the publisher, broker, and subscribers in separate terminal
-windows.
+### Define workers to handle events
 
-    ruby publisher.rb
+With Sidekiq
 
-### Subscribe to events
+    class OrderPublishWorker
+      include Sidekiq::Worker
+      sidekiq_options queue: 'pubsub' # you can use any options sidekiq supports
 
-Subscribing to events is also simple. In the following example, we create a
-subscriber to handle events published to the `user.sign-up` channel.
+      def perform(event, user, order_details)
+        # custom logic to handle the event
+      end
+    end
 
-    # file: subscriber1.rb
-    require 'chasqui'
+With Resque
 
-    class UserSignUpSubscriber
-      include Chasqui::Subscriber
-      subscribe channel: 'user.sign-up'
+    class OrderPublishWorker
+      @queue = 'pubsub' # choice of queue name is up to you
 
-      def perform(payload)
-        # Do something when the user signs up.
-        #
-        # User.create(name: payload[:user])
-        #  => #<User:0X00fe346 @name="Darth Vader">
+      def self.perform(event, user, order_details)
+        # custom logic to handle the event
       end
     end
 
-### Running Sidekiq subscribers
+The `OrderPublishWorker` is a normal Sidekiq (or Resque) worker. The first
+argument to the perform method is a [Chasqui::Event](#) object, and the
+remaining arguments are the same arguments you passed to `Chasqui.publish`.
+
+### Subscribe to events
+
+    Chasqui.subscribe do
+      on 'order.purchased', PurchasedOrderWorker
+      # ...more subscriptions
+    end
+
+The above code tells Chasqui to place events published to the `order.purchased`
+channel on `PurchaseOrderWorker`'s queue.
 
-Here is how you can run the subscriber as a sidekiq worker:
+You can also use a callable object instead of a worker class to handle events.
 
-    sidekiq -r subscriber.rb
+    Chasqui.subscribe queue: 'app_id:pubsub' do
+      on 'order.purchased', ->(event, user, order) {
+        logger.info event.to_json
+      }
+    end
 
-For more information on running Sidekiq, please refer to the
-[Sidekiq documentation](https://github.com/mperham/sidekiq).
+### Running Subscribers
 
-### Running Resque subscribers
+With Sidekiq
 
-To run the resque worker, you first need to create a Rakefile.
+    bundle exec sidekiq -q app_id:pubsub
 
-    # Rakefile
-    require 'resque'
-    require 'resque/tasks'
+With Resque
 
-    task 'resque:setup' => ['chasqui:subscriber']
+    QUEUES=app_id:pubsub bundle exec rake resque:work
 
-    namespace :chasqui do
-      task :subscriber do
-        require './subscriber.rb'
-      end
+Subscribers are normal Sidekiq or Resque workers, and can take advantage of all
+available features and plugins.  Please refer to the documentation for those
+libraries for detailed instructions.
+
+* [Sidekiq documentation](https://github.com/mperham/sidekiq)
+* [Resque documentation](https://github.com/resque/resque)
+
+### Configuration
+
+    Chasqui.configure do |c|
+      c.redis = 'redis://my-redis.example.com:6379'
+      ...
     end
 
-Then you can run the resque worker to start processing events.
+For a full list of configuration options, see the
+[Chasqui::Config documentation](http://www.rubydoc.info/gems/chasqui/Chasqui/Config).
+
+## Unsubscribing
 
-    rake resque:work
+    Chasqui.unsubscribe 'order.purchased', 'app_id:pubsub'
 
-For more information on running Resque workers, please refer to
-the [resque documentation](https://github.com/resque/resque).
+If you no longer wish to handle events for a channel, you should unsubscribe
+the worker so that the Chasqui broker stops placing jobs on that worker's
+queue.
 
 ## Why Chasqui?
 
-* Reduces coupling between applications.
-* Simple to learn and use.
-* Integrates with the popular Sidekiq and Resque background worker libraries.
-* Queues events for registered subscribers even if a subscriber is unavailable.
-* Subscribers can benefit from Sidekiq's built-in retry functionality for
-  failed jobs.
+* Persistent - events don't get lost when the broker restarts or your workers
+  are not running.
+* Integrates with the proven Sidekiq and Resque background worker libraries.
+* Reduces service coupling - publishers have no knowledge of subscribers and
+  subscribers have no knowledge of publishers.
 
 ## Limitations
 
-In order for Chasqui to work properly, the publisher, broker, and all
-subscribers must connect to the same Redis database. Chasqui is intentionally
-simple and may not have all of the features you would expect from a messaging
-system. If Chasqui is missing a feature that you think it should have, please
-consider [opening a GitHub issue](https://github.com/jbgo/chasqui/issues/new)
-to discuss your feature proposal. 
+Chasqui requires that the publisher, broker, and all subscribers must connect
+to the same Redis database. If your applications use separate Redis databases,
+they will not be able to communicate with each other using Chasqui.
 
 ## Contributing
 
-* For new functionality, please open an issue for discussion before creating a
-  pull request.
+* For feature requests, please [open an issue](https://github.com/jbgo/chasqui/issues/new)
+  to discuss the proposed feature.
 * For bug fixes, you are welcome to create a pull request without first opening
   an issue.
 * Except for documentation changes, tests are required with all pull requests.
@@ -143,7 +155,7 @@ to discuss your feature proposal.
 
 ## Code of Conduct
 
-If you are unsure whether or not your communication may be inappropriate,
-please consult the [Chasqui Code of Conduct](code-of-conduct.md).  If you even
+If you are unsure whether or not your communication is appropriate for Chasqui
+please consult the [Chasqui Code of Conduct](code-of-conduct.md).  If you
 suspect harassment or abuse, please report it to the email address listed in
 the Code of Conduct.

data/chasqui.gemspec

@@ -8,7 +8,7 @@ Gem::Specification.new do |spec|
   spec.version       = Chasqui::VERSION
   spec.authors       = ["Jordan Bach"]
   spec.email         = ["jordan@opensolitude.com"]
-  spec.summary       = %q{Chasqui is a simple, lightweight, persistent implementation of the publish-subscribe (pub/sub) messaging pattern for service oriented architectures.}
+  spec.summary       = %q{Chasqui adds persistent publish-subscribe (pub-sub) messaging capabilities to Sidekiq and Resque workers.}
   spec.homepage      = "https://github.com/jbgo/chasqui"
   spec.license       = "MIT"
 

data/lib/chasqui.rb

@@ -9,36 +9,114 @@ require "chasqui/config"
 require "chasqui/broker"
 require "chasqui/brokers/redis_broker"
 require "chasqui/queue_adapter"
-require "chasqui/queue_adapters/redis_queue_adapter"
+require "chasqui/queue_adapter/redis_queue_adapter"
 require "chasqui/subscriber"
 require "chasqui/subscriptions"
-require "chasqui/worker"
+require "chasqui/subscription_builder"
+require "chasqui/subscription_builder/resque_subscription_builder"
+require "chasqui/subscription_builder/sidekiq_subscription_builder"
 
+# A persistent implementation of the publish-subscribe messaging pattern for
+# Resque and Sidekiq workers.
 module Chasqui
 
-  class ConfigurationError < StandardError; end
-
   class << self
     extend Forwardable
     def_delegators :config, *CONFIG_SETTINGS
-    def_delegators :subscriptions, :register, :unregister
 
+    # Yields an object for configuring Chasqui.
+    #
+    # @example
+    #   Chasqui.configure do |c|
+    #     c.redis = 'redis://my-redis.example.com:6379'
+    #     ...
+    #   end
+    #
+    # @see Config See Chasqui::Config for a full list of configuration options.
+    #
+    # @yieldparam config [Config]
     def configure(&block)
       yield config
     end
 
+    # @visibility private
+    #
+    # Returns the Chasqui configuration object.
+    #
+    # @see Config See Chasqui::Config for a full list of configuration options.
+    #
+    # @return [Config]
     def config
       @config ||= Config.new
     end
 
+    # Publish an event to a channel.
+    #
+    # @param channel [String] the channel name
+    # @param args [Array<#to_json>] an array of JSON serializable objects that
+    #   comprise the event's payload.
     def publish(channel, *args)
       redis.lpush inbox_queue, build_event(channel, *args).to_json
     end
 
+    # Subscribe workers to channels.
+    #
+    #     Chasqui.subscribe(queue: 'high-priority') do
+    #       on 'channel1', Worker1
+    #       on 'channel2', Worker2
+    #       on 'channel3', ->(event) { ... }, queue: 'low-priority'
+    #       ...
+    #     end
+    #
+    # The +.subscribe+ method creates a context for registering workers to
+    # receive events for specified channels. Within a subscribe block you make
+    # calls to the {SubscriptionBuilder#on #on} method to create subscriptions.
+    #
+    # {SubscriptionBuilder#on #on} expects a channel name as the first argument
+    # and either a Resque/Sidekiq worker as the second argument or a callable
+    # object, such as a proc, lambda, or any object that responds to +#call+.
+    #
+    # @see SubscriptionBuilder#on
+    #
+    # @param [Hash] options default options for calls to +#on+. The defaults
+    #   will be overriden by options supplied to the +#on+ method directly.
+    #   See {Chasqui::SubscriptionBuilder#on} for available options.
+    def subscribe(options={})
+      builder = SubscriptionBuilder.builder(subscriptions, options)
+      builder.instance_eval &Proc.new
+    end
+
+    # @visibility private
+    #
+    # Returns the registered subscriptions.
+    #
+    # @return [Subscriptions]
     def subscriptions
       @subscriptions ||= Subscriptions.new build_queue_adapter
     end
 
+    # Unsubscribe workers from a channel.
+    #
+    # When you unsubscribe from a channel, the broker will stop placing jobs on
+    # the worker queue. When only given +channel+ and +queue+ arguments,
+    # +#unsubscribe+ will unsubscribe all workers using that channel and queue.
+    # When the additional +worker+ argument is given, Chasqui will only
+    # unsubscribe the given worker.
+    #
+    # @param channel [String] the channel name
+    # @param queue [String] the queue name
+    # @param worker [.perform,#perform,#call] the worker class or proc for a
+    #   a currently subscribed worker
+    def unsubscribe(channel, queue, worker=nil)
+      subscribers = if worker
+                      [Subscriber.new(channel, queue, worker)]
+                    else
+                      subscriptions.find channel, queue
+                    end
+
+      subscribers.each { |sub| subscriptions.unregister sub }
+    end
+
     private
 
     def build_event(channel, *args)

data/lib/chasqui/cli.rb

@@ -44,7 +44,7 @@ class Chasqui::CLI
     opts = {}
 
     @parser = OptionParser.new do |o|
-      o.banner = "Usage: #{argv[0]} [options]"
+      o.banner = "Usage: chasqui [options]"
 
       o.on('-f', '--logfile PATH', 'log file path') do |arg|
         opts[:logfile] = arg
@@ -71,7 +71,13 @@ class Chasqui::CLI
       end
     end
 
-    @parser.parse!(argv)
+    begin
+      @parser.parse!(argv)
+    rescue OptionParser::InvalidOption => ex
+      puts "Error: #{ex.message}\n\n#{@parser.help}"
+      exit 1
+    end
+
     @options = OpenStruct.new opts
   end
 

data/lib/chasqui/config.rb

@@ -1,16 +1,10 @@
 module Chasqui
 
-  Defaults = {
-    default_queue: 'chasqui-subscribers',
-    inbox_queue: 'inbox',
-    redis_namespace: 'chasqui',
-    broker_poll_interval: 3,
-    queue_adapter: -> { QueueAdapters::RedisQueueAdapter }
-  }.freeze
-
+  # Raised when configured settings prevent Chasqui from working correctly.
   class ConfigurationError < StandardError
   end
 
+  # @visibility private
   CONFIG_SETTINGS = [
     :broker_poll_interval,
     :channel_prefix,
@@ -22,7 +16,75 @@ module Chasqui
     :worker_backend
   ]
 
+  # Stores and manages all Chasqui configuration settings.
   class Config < Struct.new(*CONFIG_SETTINGS)
+
+    # @!attribute broker_poll_interval
+    #   How long the broker daemon waits for an event before pausing to handle
+    #   signals. Default: +3+
+    #   @return [Fixnum] seconds
+
+    # @!attribute channel_prefix
+    #   A string to prepend to channel names for all published events. This is
+    #   useful for namespacing channel names to prevent collisions with other
+    #   applications that may choose the same channel name for a different type
+    #   of event. Default: +nil+
+    #
+    #   @example
+    #       Chasqui.configure do |c|
+    #         c.channel_prefix = 'com.example.app1'
+    #       end
+    #
+    #       # publishes to channel: "com.example.app1.user.signup"
+    #       Chasqui.publish 'user.signup', user
+    #
+    #   @return [String]
+
+    # @!attribute default_queue
+    #   The queue to use when a worker class does not define a queue and a
+    #   queue option is not supplied to {Chasqui::SubscriptionBuilder#on #on}.
+    #   Default: +"chasqui-workers"+
+    #   @return [String]
+
+    # @!attribute inbox_queue
+    #   The queue that stores published events until they are delivered to
+    #   subscriber (worker) queues. Default: "chasqui-inbox"
+    #   @return [String]
+
+    # @!attribute logger
+    #   The logger to use for the Chasqui broker. Default: +Logger.new(STDOUT)+
+    #   @return [Logger]
+
+    # @!attribute [rw] queue_adapter
+    #   @api private
+    #   The queue adapter to use for binding queues to channels.
+    #   @return [Chasqui::QueueAdapter]
+
+    # @!attribute redis
+    #   Customize the Redis databse connection Chasqui uses.
+    #   Default: +"redis://localhost:6379/0"+
+    #   @return [Redis,String,Hash]
+
+    # @!attribute worker_backend
+    #   The type of worker that will handle events in class to
+    #   {Chasqui.subscribe}. Can be either +resque+ or +sidekiq+. Chasqui will
+    #   attempt to auto-detect the +worker_backend+ if either library is
+    #   loaded. Default: +nil+
+    #   @return [Symbol]
+
+    # @visibility private
+    # Default values for all configuration settings.
+    Defaults = {
+      broker_poll_interval: 3,
+      channel_prefix: nil,
+      default_queue: 'chasqui-workers',
+      inbox_queue: 'chasqui-inbox',
+      logger: STDOUT,
+      queue_adapter: -> { QueueAdapter::RedisQueueAdapter },
+      redis_namespace: 'chasqui',
+      worker_backend: nil
+    }.freeze
+
     def default_queue
       self[:default_queue] ||= Defaults.fetch(:default_queue)
     end