superbolt 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: e5ba86e1d694023338deda0d39210951edb45c5b
+  data.tar.gz: 76775a315705a55b7a0e149fc787957800cbc56f
+SHA512:
+  metadata.gz: ee50367c87dcf66d69221f20b06aa3d533ffe45b27ddd038d5de41ac00af2c8609ea11c2d54e09cf80c5bacb649dd7877c4bc4043408ad6ec0932bc8688df568
+  data.tar.gz: 07dd4657ad96cf381b7f6e8f107d0c106ae983c274faf7d8324ad4dcb958b593b6e020b1b064607e667b452ad22218271bff451c5b2066698b9c19ec92fb9db4

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in superbolt.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 SocialChorus
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,266 @@
+# Superbolt
+
+Superbolt is an easy intra-app communication system for sending messages
+between applications. It is backed by RabbitMQ and under the covers it
+uses both the Bunny gem and the AMQP gem.
+
+#### Why not just use those gems?
+
+RabbitMQ is hard. Even though it is a messaging queue, by name, a
+developer can't pick up one of these great gems and treat the queue
+like a queue. There is great ceremony in the process. In fact, it is
+quite easy to pass in the wrong queue arguments, or leave a connection
+open.
+
+Superbolt takes the ceremony away and let's developers focus on what is
+important: reading and sending messages.
+
+#### How does it make it easier?
+
+Superbolt has two main components: a queue and an app.
+
+The queue object acts like a queue. The queue can push, pop and peek. 
+The queue is able to spit out all or a subset of the messages on the 
+queue. It can clear itself. In short it makes inline operations doable.
+
+The app on the other hand is an EventMachine process that takes over the
+thread. It continually reads from a single queue until it recieves a
+signal or message shutting it down. It is smart; it handles exceptions in message
+processing by moving those problem messages to a related error queue. It also
+listens on a separate quit queue for a graceful shutdown. A graceful
+shutdown means no messages are lost.
+
+#### Simple because there are less RabbitMQ capabilities
+
+Superbolt makes intra-app communication easier via reducing
+the types of things that can be done in RabbitMQ. While Superbolt was made to
+address typical messaging patterns like RPC and work queues, it cuts out
+RabbitMQ features like exclusive binding, and uses the library itself,
+and conventions to make sure the right application gets the right
+message. If all the features of RabbitMQ are needed, then using
+Superbolt is not appropriate ... hop along.
+
+#### Conventions
+
+While Superbolt makes it possible to listen on any queue name
+as an application, or interact with any queue name as an enumerable-ish queue, it
+gets its real power from conventions.
+
+Messages are JSON. Developers can do it differently, but the gains of
+ease will be lost.
+
+By convention each application has a name. The name is used along with
+the application environment in order to communicate. The environment
+means that test and development messages don't get lumped together. The
+application name means that each application has to employ some
+filtering to figure out what end process should handle the messages.
+
+Per application filtering, is made very easy by Superbolt's message conventions.
+A Superbolt message has three keys:
+
+1. origin: the origin application
+2. event: some identifier/sort key for the message type, will be 'default' by
+   default.
+3. arguments: additional data to be passed on to the handling process
+
+Developer's can create a message without having to think to much about
+these concerns:
+
+    Superbolt.app_name = 'me'
+    Superbolt.message.to('over_there').send!({just: 'do it!'})
+
+    Superbolt.queue('over_there').pop
+    => {
+      origin: 'me',
+      event: 'default',
+      arguments: {
+        just: 'do it!'
+      }
+    }
+
+## Usage (more)
+
+###Configuring:
+
+    Superbolt.config = {
+      app_name: 'my_app', # no default
+      env: 'staging', # looks to env for information
+      connection_params: {
+        # can use anything RabbitMQ speaks here
+        host: 'my-rabbitmq-provider.com'
+      } # defaults to what is set in the ENV or localhost
+    }
+
+#### Connection Configuration
+
+Out of the box Superbolt will look to the ENV to see if there is a
+connection key, RABBITMQ\_URL. Developers can customize the connection
+key that is used. Actual connection params that RabbitMQ Bunny/AMQP use
+can also be passed in as above. If a connection key is not found in the
+ENV, localhost will be used. If the application uses these typical
+conventions, then no connection configuration is required.
+ 
+#### App Name
+
+The application name/identifier is an important default to setup in
+order to get all the goodness of related to messaging and its
+conventions. 
+
+If no app name is set up, a littlem or work is required to send a
+message:
+
+    Superbolt.message
+      .to('over_there')
+      .from('me')
+      .send!({just: 'do it!'})
+
+Without the #from call, the message will be sent without an origin. In
+our experience it is typical to process messages differently depending
+on the sender. Of course, that information can be encoded into the event
+name, but it is pretty easy to configure an application name so event
+filtering is easier for the consuming application.
+
+    Superbolt.app_name = 'my_great_app'
+    # - or -
+    Superbolt.config = {
+      app_name = 'my_great_app'
+    }
+
+### Sending messages
+
+Superbolt doesn't want to developers worrying about exchanges,
+durability or connection ceremony. Developers should be able to just
+send a message. The ease of that sending depends on whether developers
+are sticking with the Superbolt conventions or not. 
+
+#### The easiest way to message
+
+The easiest way is to use the Superbolt level helper for sending a
+message:
+
+    Superbolt.message
+      .re('dorothy')
+      .to('wicked_witch')
+      .send!('On yellow brick road; has friends!')
+
+This message can be received on a queue
+'wicked\_witch\_staging', given that the environment is
+'staging'. When it is popped off it will look like this:
+
+    {
+      origin: 'my_configured_app_name',
+      event: 'dorothy',
+      arguments: 'On yellow brick road; has friends!'
+    }
+  
+#### A more customizable messaging experience
+
+Messages can also be sent via Superbolt::Queue objects. In this case the
+message can be anything and the queue name is exactly what is passed in.
+
+    queue = Superbolt::Queue.new('dorothy')
+    queue.push({demand: 'Surrender!'})
+    queue.pop 
+    => {
+      'demand' => 'Surrender!'
+    }
+
+### Reading messages
+
+Messages can be read inline or via a standalone app. 
+
+#### Reading Inline
+
+Reading messages inline is easy and to the point. The Superbolt::Queue
+object tries to act queue-like instead of like a hard to use external
+service. 
+
+Popping messages off the queue will remove the message from the queue
+immediately. If something goes wrong with eth message processing, it is
+the responsibility of the consuming application to figure out what to
+do.
+
+    message = queue.pop # This removes the message permanently
+
+Messages can be read in non-destructive ways as well. 
+
+    message = queue.peek
+    # Ponder or process message.
+    # It is still hanging out at the top of the queue.
+    # It can be deleted with a pop, 
+    # provided another consumer hasn't already deleted it!
+
+Because of asynchronicity issues with job processing across several apps
+or workers, the best usage for non-destructive inline reads is debugging
+and information gathering.
+
+    queue.all # return all the messages on the queue
+
+    # remove messages meeting the block criteria
+    queue.delete {|m| m['level'] == 'not_important' }
+
+    # peek at messages in a certain range
+    queue.slice(2, 4) 
+
+    # get a certain message, non-destructively
+    queue[3]
+
+#### Reading via a Standalone App
+
+Reading messages inline is useful, but in general an application wants
+to read continuously for the latest and greatest intra-app
+communications.
+
+    Superbolt::App.new('dorothy_inbox').run do |message, logger|
+      HomewardBound.new(message).perform
+    end
+
+Exceptions raised in the processing block will not exit
+the Superbolt app. Errors will be logged and the message will be put on
+an error queue with information about the exception raised. Those
+messages can be seen by accessing the related error queue:
+
+    error_queue = Superbolt::App.new('dorothy_inbox').error_queue
+    error_queue.all
+
+The error queue is a Superbolt::Queue and methods like #pop, #delete,
+and #[] are available to gather more data about the exceptions being
+raised.
+
+The app can be shutdown gracefully by sending a quit message to a
+special queue:
+
+    quit_queue = Superbolt::App.new('dorothy_inbox').quit_queue
+    quit_queue.push(message: 'for a deploy')
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'superbolt'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install superbolt
+
+## TODOs:
+
+* Easy filtering/delegation of messages to classes
+* Failed messages are put on another queue so that the app is not
+  in a failure loop.
+* In code YARD stye documentation
+* CodeClimate
+* TravisCI continuout integration
+
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/lib/superbolt/adapter/amqp.rb

@@ -0,0 +1,18 @@
+module Superbolt
+  module Adapter
+    class AMQP < Base
+      def socket
+        @socket ||= ::AMQP.connect(config.connection_params)
+      end
+
+      def channel
+        @channel ||= ::AMQP::Channel.new(socket)
+      end
+
+      def close(&block)
+        socket.close(&block)
+      end
+    end
+  end
+end
+

data/lib/superbolt/adapter/base.rb

@@ -0,0 +1,28 @@
+module Superbolt
+  module Adapter
+    class Base
+      attr_reader :config
+
+      def initialize(config=nil)
+        @config = config || Superbolt.config
+      end
+
+      delegate :closed?, :open, :open?,
+        to: :socket
+
+      def close
+        response = socket.close
+        @socket = nil
+        @channel = nil
+        response
+      end
+
+      delegate :queues, :acknowledge, :reject, :queue,
+        to: :channel
+
+      def exchange
+        channel.default_exchange
+      end
+    end
+  end
+end

data/lib/superbolt/adapter/bunny.rb

@@ -0,0 +1,16 @@
+module Superbolt
+  module Adapter
+    class Bunny < Base
+      def socket
+        return @socket if @socket
+        @socket = ::Bunny.new(config.connection_params)
+        @socket.start
+        @socket
+      end
+
+      def channel
+        @channel ||= socket.create_channel
+      end
+    end
+  end
+end

data/lib/superbolt/app.rb

@@ -0,0 +1,73 @@
+module Superbolt
+  class App
+    attr_reader :config, :env
+    attr_accessor :logger
+
+    def initialize(name, options)
+      @name = name
+      @env = options[:env]
+      @logger = options[:logger] || Logger.new($stdout)
+      @config = options[:config] || Superbolt.config
+    end
+
+    def name
+      env ? "#{@name}_#{env}" : @name
+    end
+
+    def connection
+      @connection ||= Connection::App.new(name, config)
+    end
+
+    delegate :close, :closing, :exclusive?, :durable?, :auto_delete?,
+      :writer, :channel, :q,
+        to: :connection
+
+    def queue
+      connection.q
+    end
+
+    def quit_subscriber_queue
+      connection.qq
+    end
+
+    def quit_queue
+      Queue.new("#{connection.name}.quit", connection.config)
+    end
+
+    def error_queue
+      Queue.new("#{connection.name}.error", connection.config)
+    end
+
+    def run(&block)
+      EventMachine.run do
+        queue.subscribe(ack: false) do |metadata, payload|
+          message = IncomingMessage.new(metadata, payload, channel)
+          processor = Processor.new(message, logger, &block)
+          unless processor.perform
+            on_error(message.parse, processor.exception)
+          end
+        end
+
+        quit_subscriber_queue.subscribe do |message|
+          quit(message)
+        end
+      end
+    end
+
+    def on_error(message, error)
+      error_message = message.merge({error: {
+        class: error.class,
+        message: error.message,
+        backtrace: error.backtrace
+      }})
+      error_queue.push(error_message)
+    end
+
+    def quit(message='no message given')
+      logger.info "EXITING Superbolt App listening on queue #{name}: #{message}"
+      close {
+        EventMachine.stop { exit }
+      }
+    end
+  end
+end

data/lib/superbolt/config.rb

@@ -0,0 +1,30 @@
+module Superbolt
+  class Config
+    attr_reader :options
+
+    def initialize(options={})
+      @options = options
+    end
+
+    def connection_params
+      env_params || default
+    end
+
+    def env_connection_key
+      options[:connection_key] || 'RABBITMQ_URL'
+    end
+
+    def env_params
+      ENV[env_connection_key]
+    end
+
+    def default
+      options[:connection_params] || {host: '127.0.0.1'}
+    end
+
+    def ==(other)
+      other.connection_params == connection_params &&
+        other.env_connection_key == env_connection_key
+    end
+  end
+end

data/lib/superbolt/connection/app.rb

@@ -0,0 +1,21 @@
+module Superbolt
+  module Connection
+    class App < Base
+      def connection
+        @connection ||= Adapter::AMQP.new(config)
+      end
+
+      def close(&block)
+        connection.close(&block)
+        @connection = nil
+        @q = nil
+        @qq = nil
+      end
+
+      def qq
+        @qq ||= connection.queue("#{name}.quit", self.class.default_options)
+      end
+    end
+  end
+end
+

data/lib/superbolt/connection/base.rb

@@ -0,0 +1,40 @@
+module Superbolt
+  module Connection
+    class Base
+      attr_reader :name, :config
+
+      def initialize(name, config=nil)
+        @name = name
+        @config = config || Superbolt.config
+      end
+
+      def connection
+        raise NotImplementedError
+      end
+
+      def close
+        raise NotImplementedError
+      end
+
+      def q
+        @q ||= connection.queue(name, self.class.default_options)
+      end
+
+      delegate :exclusive?, :durable?, :auto_delete?,
+        to: :q
+
+      def channel
+        connection.channel
+      end
+
+      def self.default_options
+        {
+          :auto_delete => false,
+          :durable => true,
+          :exclusive => false
+        }
+      end
+    end
+  end
+end
+

data/lib/superbolt/connection/queue.rb

@@ -0,0 +1,25 @@
+module Superbolt
+  module Connection
+    class Queue < Base
+      def connection
+        @connection ||= Adapter::Bunny.new(config)
+      end
+
+      def close
+        connection.close
+        @connection = nil
+        @q = nil
+      end
+
+      def closing(&block)
+        response = block.call
+        close
+        response
+      end
+
+      def writer
+        connection.exchange
+      end
+    end
+  end
+end

data/lib/superbolt/facade.rb

@@ -0,0 +1,26 @@
+module Superbolt
+  def self.config=(options)
+    @config = Config.new(options)
+  end
+
+  def self.config
+    @config ||= Config.new
+  end
+
+  def self.queue(name)
+    Superbolt::Queue.new(name, config)
+  end
+
+  class << self
+    attr_writer :env
+    attr_accessor :app_name
+  end
+
+  def self.env
+    @env || 'development'
+  end
+
+  def self.message(args={})
+    Superbolt::Messenger.new(args)
+  end
+end

data/lib/superbolt/incoming_message.rb

@@ -0,0 +1,26 @@
+module Superbolt
+  class IncomingMessage
+    attr_reader :payload, :tag, :channel
+
+    def initialize(delivery_info, payload, channel)
+      @payload = payload
+      @tag = delivery_info.delivery_tag
+      @channel = channel
+    end
+
+    def parse
+      JSON.parse(payload)
+    rescue JSON::ParserError
+      payload
+    end
+
+    def reject
+      channel.reject(tag)
+    end
+
+    def ack
+      channel.acknowledge(tag)
+    end
+  end
+end
+

data/lib/superbolt/messenger.rb

@@ -0,0 +1,66 @@
+module Superbolt
+  class Messenger
+    attr_accessor :origin, :name, :event, :arguments, :env
+
+    def initialize(options={})
+      @name = options.delete(:to)
+      @origin = options.delete(:from) || Superbolt.app_name
+      @event = options.delete(:event) || self.class.defaultevent
+      @env = Superbolt.env
+      @arguments = options
+    end
+
+    def message
+      {
+        origin: origin,
+        event: event,
+        arguments: arguments
+      }
+    end
+
+    # chainer methods
+    #
+    def to(val)
+      self.name = val
+      self
+    end
+
+    def from(val)
+      self.origin = val
+      self
+    end
+
+    def re(val)
+      self.event = val
+      self
+    end
+
+    def data(val)
+      self.arguments = val
+      self
+    end
+
+    # alias
+    # -------
+
+    def send!(args=nil)
+      self.arguments = args if args
+      queue.push(message)
+    end
+
+    def queue
+      unless name
+        raise "no destination app name defined, please pass one in"
+      end
+      Queue.new(destination_name)
+    end
+
+    def destination_name
+      "#{name}_#{env}"
+    end
+
+    def self.defaultevent
+      'default'
+    end
+  end
+end