rails-queue 1.0.0

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright 2012 We're Probably Wrong
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,179 @@
+# rails-queue
+
+[![Build Status](https://secure.travis-ci.org/probablywrong/rails-queue.png)](https://travis-ci.org/probablywrong/rails-queue)
+
+## Description
+
+It's the `Rails.queue` [from Rails 4](https://github.com/rails/rails/blob/master/activesupport/lib/active_support/queueing.rb), backported to Rails 3.2+ for your Queueing pleasure.
+
+> `Rails.queue` is the application's queue. You can push a job onto the queue by:
+>
+> ```
+> Rails.queue.push job
+> ```
+>
+> A job is an object that responds to `#run`. Queue consumers will pop jobs off of the queue and invoke the queue's `run` method.
+>
+> Note that depending on your queue implementation, jobs may not be executed in the same process as they were created in, and are never executed in the same thread as they were created in.
+>
+> If necessary, a queue implementation may need to serialize your job for distribution to another process. The documentation of your queue will specify the requirements for that serialization
+
+
+## Requirements
+
+This gem requires Rails 3.2+ and has been tested on the following versions:
+
+* 3.2.8
+* 3.2.9
+
+This gem has been tested against the following Ruby versions:
+
+* 1.9.2
+* 1.9.3
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem "rails-queue"
+```
+
+And then execute:
+
+```
+$ bundle
+```
+
+Or install it yourself as:
+
+```
+$ gem install rails-queue
+```
+
+
+## Usage
+
+The following queueing strategies are provided by default:
+
+* `SynchronousQueue`
+* `TestQueue`
+
+The following queue consumer strategies are provided by default:
+
+* `ThreadedQueueConsumer`
+
+The threaded consumer will run jobs in a background thread in development mode or in a VM where running jobs on a thread in production mode makes sense.
+
+When the process exits, the consumer pushes a `nil` onto the queue and joins the thread, which will ensure that all jobs are executed before the process finally dies.
+
+_Note: In this port, classes that live under the `ActiveSupport::` namespace in Rails 4 have been moved to the `Rails::Queue::` namespace._
+
+### Configuration
+
+Rails will now have the following options available:
+
+* `config.queue`
+* `config.queue_consumer`
+* `config.action_mailer.queue`
+
+These can be customized in `config/application.rb` or by environment:
+
+```ruby
+# config/environments/production.rb
+
+# Default the production mode queue to an synchronous queue. You will probably
+# want to replace this with an out-of-process queueing solution.
+config.queue = Rails::Queue::SynchronousQueue.new
+
+# You will probably want to change the job queue consumer from the default.
+config.queue_consumer = Rails::Queue::ThreadedQueueConsumer.new
+```
+
+
+### ActionMailer
+
+Mail delivery from ActionMailer can be processed asynchronously using the Rails.queue.
+
+By default, ActionMailer will use the application default Rails.queue strategy. This can be configured:
+
+```ruby
+# The queue that will be used to deliver the mail. The queue should expect a job
+# that responds to run.
+config.action_mailer.queue = Rails::Queue::SynchronousQueue.new
+```
+
+
+### Testing
+
+In test mode, the Rails queue is backed by an Array so that assertions can be made about its contents. The test queue provides a `jobs` method to make assertions about the queue's contents and a `drain` method to drain the queue and run the jobs.
+
+Jobs are run in a separate thread to catch mistakes where code assumes that the job is run in the same thread.
+
+```ruby
+# config/environments/test.rb
+
+config.queue = Rails::Queue::TestQueue.new
+```
+
+### Alternate Providers
+
+The Rails Queue implementation is designed to be pluggable and you may want to investigate other queue providers, such as:
+
+* [resque-rails](https://github.com/jeremy/resque-rails)
+* [Sidekiq](https://github.com/mperham/sidekiq/tree/rails4)
+
+
+## Resources
+
+The following are some resources from around the Internet which introduce the Rails Queue:
+
+_Note: Many of these are, or will be soon, out of date._
+
+* [Ruby on Rails 4.0 Release Notes](http://edgeguides.rubyonrails.org/4_0_release_notes.html)
+* [Rails 4.0 Sneak Peek: Queueing](http://reefpoints.dockyard.com/ruby/2012/06/25/rails-4-sneak-peek-queueing.html)
+* [New in Rails 4.0: Rails Queue](http://www.3magine.com/blog/new-in-rails-4-0-rails-queue/)
+
+
+## A note on spelling
+
+Yes, there are two spelling options. We defer to research provided by [XKCD](http://xkcd.com/853/).
+
+
+## Authors
+
+This was written/extracted as part of an ongoing yak-shave known as "We're Probably Wrong", a production of:
+
+* [@elskwid](https://github.com/elskwid) [![endorse](http://api.coderwall.com/elskwid/endorsecount.png)](http://coderwall.com/elskwid)
+* [@phlipper](https://github.com/phlipper) [![endorse](http://api.coderwall.com/phlipper/endorsecount.png)](http://coderwall.com/phlipper)
+
+
+## Contributors
+
+Many thanks go to the following who have contributed to making this gem even better:
+
+* **[@rails](https://github.com/rails)**
+    * for making this all possible in the first place
+
+
+## Contributing
+
+1. [Fork it](https://github.com/phlipper/rails-four-queueing/fork_select)
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. [Create a Pull Request](hhttps://github.com/phlipper/rails-four-queueing/pull/new)
+
+
+## License
+
+**rails-queue**
+
+* Freely distributable and licensed under the [MIT license](http://probablywrong.mit-license.org/2012/license.html).
+* Copyright (c) 2012 We're Probably Wrong (github@wereprobablywrong.com)
+* http://wereprobablywrong.com/
+
+**rails**
+
+* Ruby on Rails is released under the [MIT License](http://www.opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,12 @@
+require "bundler/gem_tasks"
+
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "lib"
+  t.libs << "test"
+  t.pattern = "test/**/*_test.rb"
+  t.verbose = false
+end
+
+task default: :test

data/lib/rails-queue.rb

@@ -0,0 +1,15 @@
+require "active_support/concern"
+
+$: << File.expand_path("../", __FILE__)
+
+require "rails/queue/configuration"
+require "rails/queue/application"
+require "rails/queue/queue"
+require "rails/queue/action_mailer/queued_message"
+
+require "rails/queue/railtie"
+
+module Rails
+  module Queue
+  end
+end

data/lib/rails/queue/action_mailer/queued_message.rb

@@ -0,0 +1,42 @@
+# Original File: RAILS_ROOT/actionmailer/lib/action_mailer/queued_message.rb
+require "delegate"
+
+module Rails
+  module Queue
+    module ActionMailer
+      class QueuedMessage < ::Delegator
+        attr_reader :queue
+
+        def initialize(queue, mailer_class, method_name, *args)
+          @queue = queue
+          @job   = DeliveryJob.new(mailer_class, method_name, args)
+        end
+
+        def __getobj__
+          @job.message
+        end
+
+        # Queues the message for delivery.
+        def deliver
+          tap { @queue.push @job }
+        end
+
+        class DeliveryJob
+          def initialize(mailer_class, method_name, args)
+            @mailer_class = mailer_class
+            @method_name  = method_name
+            @args         = args
+          end
+
+          def message
+            @message ||= @mailer_class.send(:new, @method_name, *@args).message
+          end
+
+          def run
+            message.deliver
+          end
+        end
+      end
+    end
+  end
+end

data/lib/rails/queue/application.rb

@@ -0,0 +1,17 @@
+module Rails
+  module Queue
+    module Application
+      extend ::ActiveSupport::Concern
+
+      included do
+        attr_accessor :queue_consumer
+        attr_writer :queue
+      end
+
+      def queue #:nodoc:
+        @queue ||= config.queue || Rails::Queue::Queue.new
+      end
+
+    end
+  end
+end

data/lib/rails/queue/configuration.rb

@@ -0,0 +1,12 @@
+module Rails
+  module Queue
+    module Configuration
+      extend ::ActiveSupport::Concern
+
+      included do
+        attr_accessor :queue, :queue_consumer
+      end
+
+    end
+  end
+end

data/lib/rails/queue/queue.rb

@@ -0,0 +1,110 @@
+# Original File: RAILS_ROOT/activesupport/lib/active_support/queueing.rb
+require "delegate"
+require "thread"
+
+module Rails
+  module Queue
+
+    # A Queue that simply inherits from STDLIB's Queue. When this
+    # queue is used, Rails automatically starts a job runner in a
+    # background thread.
+    class Queue < ::Queue
+      attr_writer :consumer
+
+      def initialize(consumer_options = {})
+        super()
+        @consumer_options = consumer_options
+      end
+
+      def consumer
+        @consumer ||= ThreadedQueueConsumer.new(self, @consumer_options)
+      end
+
+      # Drain the queue, running all jobs in a different thread. This method
+      # may not be available on production queues.
+      def drain
+        # run the jobs in a separate thread so assumptions of synchronous
+        # jobs are caught in test mode.
+        consumer.drain
+      end
+    end
+
+    class SynchronousQueue < Queue
+      def push(job)
+        super.tap { drain }
+      end
+      alias <<  push
+      alias enq push
+    end
+
+    # In test mode, the Rails queue is backed by an Array so that assertions
+    # can be made about its contents. The test queue provides a +jobs+
+    # method to make assertions about the queue's contents and a +drain+
+    # method to drain the queue and run the jobs.
+    #
+    # Jobs are run in a separate thread to catch mistakes where code
+    # assumes that the job is run in the same thread.
+    class TestQueue < Queue
+      # Get a list of the jobs off this queue. This method may not be
+      # available on production queues.
+      def jobs
+        @que.dup
+      end
+
+      # Marshal and unmarshal job before pushing it onto the queue.  This will
+      # raise an exception on any attempts in tests to push jobs that can't (or
+      # shouldn't) be marshalled.
+      def push(job)
+        super Marshal.load(Marshal.dump(job))
+      end
+    end
+
+    # The threaded consumer will run jobs in a background thread in
+    # development mode or in a VM where running jobs on a thread in
+    # production mode makes sense.
+    #
+    # When the process exits, the consumer pushes a nil onto the
+    # queue and joins the thread, which will ensure that all jobs
+    # are executed before the process finally dies.
+    class ThreadedQueueConsumer
+      attr_accessor :logger
+
+      def initialize(queue, options = {})
+        @queue = queue
+        @logger = options[:logger]
+        @fallback_logger = Logger.new($stderr)
+      end
+
+      def start
+        @thread = Thread.new { consume }
+        self
+      end
+
+      def shutdown
+        @queue.push nil
+        @thread.join
+      end
+
+      def drain
+        @queue.pop.run until @queue.empty?
+      end
+
+      def consume
+        while job = @queue.pop
+          run job
+        end
+      end
+
+      def run(job)
+        job.run
+      rescue Exception => exception
+        handle_exception job, exception
+      end
+
+      def handle_exception(job, exception)
+        (logger || @fallback_logger).error "Job Error: #{job.inspect}\n#{exception.message}\n#{exception.backtrace.join("\n")}"
+      end
+    end
+
+  end
+end

data/lib/rails/queue/railtie.rb

@@ -0,0 +1,64 @@
+require "rails"
+
+module Rails
+  module Queue
+
+    # Public: Initialize Rails 4 Queue
+    class Railtie < ::Rails::Railtie
+
+      # mixin the queue and queue_consumer to Configuration
+      Rails::Application::Configuration.send :include,
+        Rails::Queue::Configuration
+
+      # mixin the queue and queue_consumer to Application with default Queue
+      Rails::Application.send :include, Rails::Queue::Application
+
+      # provide a `queue` accessor from the top-level module
+      def Rails.queue
+        application.queue
+      end
+
+      # initialize the configuration instance queue if needed
+      config.after_initialize do |app|
+        unless app.config.instance_variable_get(:"@queue")
+          app.config.instance_variable_set(
+            :"@queue", Rails::Queue::SynchronousQueue.new
+          )
+        end
+      end
+
+      # initialize the queue_consumer
+      # copied from: railties/lib/rails/application/finisher.rb
+      initializer :activate_queue_consumer, after: "finisher_hook" do |app|
+        if app.config.queue.class == Rails::Queue::Queue
+          app.queue_consumer = app.config.queue_consumer || app.config.queue.consumer
+          app.queue_consumer.logger ||= Rails.logger if app.queue_consumer.respond_to?(:logger=)
+          app.queue_consumer.start
+          at_exit { app.queue_consumer.shutdown }
+        end
+      end
+
+      # initialize ActionMailer queue configuration
+      config.after_initialize do |app|
+        app.config.action_mailer.queue ||= app.queue
+      end
+
+      # enable ActionMailer queued delivery
+      ::ActiveSupport.on_load :action_mailer do
+        class_attribute :queue
+        self.queue = Rails::Queue::SynchronousQueue.new
+
+        # override default implementation to provide a QueuedMessage
+        # copied from Rails 4
+        def self.method_missing(method_name, *args)
+          if action_methods.include?(method_name.to_s)
+            Rails::Queue::ActionMailer::QueuedMessage.new(queue, self, method_name, *args)
+          else
+            super
+          end
+        end
+      end
+    end
+
+  end
+end