hirefire 0.1.0

data/.document

@@ -0,0 +1,3 @@
+README.md
+LICENSE
+lib/**/*.rb

data/.gitignore

@@ -0,0 +1,4 @@
+.DS_Store
+*.gem
+.yardoc/*
+.yardoc/**/*

data/.infinity_test

@@ -0,0 +1,11 @@
+# encoding: utf-8
+
+##
+# Run all tests when a file gets saved inside the lib folder
+infinity_test do
+  heuristics do
+    add('lib/') do |file|
+      run :all => :tests
+    end
+  end
+end

data/.rspec

@@ -0,0 +1,3 @@
+--format Fuubar
+--color
+--tty

data/Gemfile

@@ -0,0 +1,15 @@
+# encoding: utf-8
+
+source 'http://rubygems.org'
+
+##
+# Define gems to be used in the 'test' environment
+group :test do
+  gem 'heroku'
+  gem 'rush'
+  gem 'rspec'
+  gem 'mocha'
+  gem 'infinity_test'
+  gem 'fuubar'
+  gem 'timecop'
+end

data/Gemfile.lock

@@ -0,0 +1,53 @@
+GEM
+  remote: http://rubygems.org/
+  specs:
+    configuration (1.2.0)
+    diff-lcs (1.1.2)
+    fattr (2.2.0)
+    fuubar (0.0.3)
+      rspec (~> 2.0)
+      rspec-instafail (~> 0.1.4)
+      ruby-progressbar (~> 0.0.9)
+    heroku (1.20.1)
+      launchy (~> 0.3.2)
+      rest-client (< 1.7.0, >= 1.4.0)
+    infinity_test (1.0.2)
+      notifiers (>= 1.1.0)
+      watchr (>= 0.7)
+    launchy (0.3.7)
+      configuration (>= 0.0.5)
+      rake (>= 0.8.1)
+    mime-types (1.16)
+    mocha (0.9.12)
+    notifiers (1.1.0)
+    rake (0.8.7)
+    rest-client (1.6.1)
+      mime-types (>= 1.16)
+    rspec (2.5.0)
+      rspec-core (~> 2.5.0)
+      rspec-expectations (~> 2.5.0)
+      rspec-mocks (~> 2.5.0)
+    rspec-core (2.5.1)
+    rspec-expectations (2.5.0)
+      diff-lcs (~> 1.1.2)
+    rspec-instafail (0.1.6)
+    rspec-mocks (2.5.0)
+    ruby-progressbar (0.0.9)
+    rush (0.6.7)
+      session
+    session (3.1.0)
+      fattr
+    timecop (0.3.5)
+    watchr (0.7)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  fuubar
+  heroku
+  infinity_test
+  mocha
+  rspec
+  rush
+  timecop

data/LICENSE.md

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Michael van Rooijen ( [@meskyanichi](http://twitter.com/#!/meskyanichi) )
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,100 @@
+HireFire - The Heroku Worker Manager
+====================================
+
+**HireFire automatically "hires" and "fires" (aka "scales") Delayed Job workers on Heroku**. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and then firing them again once all the jobs have been processed. It's also capable to dramatically reducing processing time by automatically hiring more workers when the queue size increases.
+
+**Low traffic example** say we have a small application that doesn't process for more than 2 hours in the background a month. Meanwhile, your worker is basically just idle the rest of the 718 hours in that month. Keeping that idle worker running costs $36/month ($0.05/hour). But, for the resources you're actually **making use of** (2 hours a month), you should be paying $0.10/month, not $36/month. This is what HireFire is for.
+
+**High traffic example** say we have a high traffic application that needs to process a lot of jobs. There may be "traffic spikes" from time to time. In this case you can take advantage of the **job\_worker\_ratio**. Since this is application-specific, HireFire allows you to define how many workers there should be running, depending on the amount of queued jobs there are (see example configuration below). HireFire will then spin up more workers as traffic increases so it can work through the queue faster, then when the jobs are all finished, it'll shut down all the workers again until the next job gets queued (in which case it'll start with only a single worker again).
+
+**Enough with the examples!** Read on to see how to set it, and configure it to your scaling and money saving needs.
+
+Author
+------
+
+**Michael van Rooijen ( [@meskyanichi](http://twitter.com/#!/meskyanichi) )**
+
+Drop me a message for any questions, suggestions, requests, bugs or submit them to the [issue log](https://github.com/meskyanichi/hirefire/issues).
+
+
+Setting it up
+-------------
+
+A painless process. In a Ruby on Rails environment you would do something like this.
+
+**Rails.root/Gemfile**
+
+    gem 'rails'
+    gem 'delayed_job'
+    gem 'hirefire'
+
+**(The order is important: Delayed Job > HireFire)**
+
+Be sure to add the following Heroku environment variables so HireFire can manage your workers.
+
+    heroku config:add HIREFIRE_EMAIL=<your_email> HIREFIRE_PASSWORD=<your_password>
+
+These are the same email and password credentials you use to log in to the Heroku web interface to manage your workers.
+
+And that's it. Next time you deploy to [Heroku](http://heroku.com/) it'll automatically hire and fire your workers. Now, there are defaults, but I highly recommend you configure it since it only takes a few seconds. Create an initializer file:
+
+**Rails.root/config/initializers/hirefire.rb**
+
+    HireFire.configure do |config|
+      config.max_workers      = 5 # default is 1
+      config.job_worker_ratio = [
+          { :jobs => 1,   :workers => 1 },
+          { :jobs => 15,  :workers => 2 },
+          { :jobs => 35,  :workers => 3 },
+          { :jobs => 60,  :workers => 4 },
+          { :jobs => 80,  :workers => 5 }
+        ]
+    end
+
+Basically what it comes down to is that we say **NEVER** to hire more than 5 workers at a time (`config.max_workers = 5`). And then we define an array of hashes that represents our **job\_worker\_ratio**. In the above example we are basically saying:
+
+* Hire 1 worker if there are 1-14 queued jobs
+* Hire 2 workers if there are 15-34 queued jobs
+* Hire 3 workers if there are 35-59 queued jobs
+* Hire 4 workers if there are 60-79 queued jobs
+* Hire 5 workers if there are more than 80 queued jobs
+
+Once all the jobs in the queue have been processed, it'll fire (shut down) all the workers and start with a single worker the next time a new job gets queued. And then the next time the queue hits 15 jobs mark, in which case the single worker isn't fast enough on it's own, it'll spin up the 2nd worker again.
+
+
+In a non-Ruby on Rails environment
+----------------------------------
+
+Almost the same setup, except that you have to initialize HireFire yourself after Delayed Job is done loading.
+
+    require 'delayed_job'
+    require 'hirefire'
+    HireFire::Initializer.initialize!
+
+**(Again, the order is important: Delayed Job > HireFire)**
+
+If all goes well you should see a message similar to this when you boot your application:
+
+    [HireFire] Delayed::Backend::ActiveRecord::Job detected!
+
+
+Mapper Support
+--------------
+
+* [ActiveRecord ORM](https://github.com/rails/rails/tree/master/activerecord)
+* [Mongoid ODM](https://github.com/mongoid/mongoid) (using [delayed_job_mongoid](https://github.com/collectiveidea/delayed_job_mongoid))
+
+
+Worker Support
+--------------
+
+Currently only [Delayed Job](https://github.com/collectiveidea/delayed_job) with either [ActiveRecord ORM](https://github.com/rails/rails/tree/master/activerecord) and [Mongoid ODM](https://github.com/mongoid/mongoid).
+Might have plans to implement this for other workers in the future.
+
+
+Other potentially interesting gems
+----------------------------------
+
+* [Backup](https://github.com/meskyanichi/backup)
+* [GitPusshuTen](https://github.com/meskyanichi/gitpusshuten)
+* [Mongoid::Paperclip](https://github.com/meskyanichi/mongoid-paperclip)

data/hirefire.gemspec

@@ -0,0 +1,38 @@
+# encoding: utf-8
+
+require File.expand_path(File.dirname(__FILE__) + '/lib/hirefire')
+
+Gem::Specification.new do |gem|
+
+  ##
+  # General configuration / information
+  gem.name        = 'hirefire'
+  gem.version     = HireFire::Version.current
+  gem.platform    = Gem::Platform::RUBY
+  gem.authors     = 'Michael van Rooijen'
+  gem.email       = 'meskyanichi@gmail.com'
+  gem.homepage    = 'http://rubygems.org/gems/hirefire'
+  gem.summary     = 'HireFire automatically "hires" and "fires" (aka "scales") Delayed Job workers on Heroku.'
+  gem.description = <<-EOS
+                      HireFire automatically "hires" and "fires" (aka "scales") Delayed Job workers on Heroku.
+                      When there are no queue jobs, HireFire will fire (shut down) all workers. If there are
+                      queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired
+                      depends on the amount of queued jobs (the ratio can be configured by you). HireFire
+                      is great for both high, mid and low traffic applications. It can save you a lot of
+                      money by only hiring workers when there are pending jobs, and then firing them again
+                      once all the jobs have been processed. It's also capable to dramatically reducing
+                      processing time by automatically hiring more workers when the queue size increases.
+                    EOS
+
+  ##
+  # Files and folder that need to be compiled in to the Ruby Gem
+  gem.files         = %x[git ls-files].split("\n")
+  gem.test_files    = %x[git ls-files -- {spec}/*].split("\n")
+  gem.require_path  = 'lib'
+
+  ##
+  # Production gem dependencies
+  gem.add_dependency 'heroku', ['~> 1.20.1']
+  gem.add_dependency 'rush',   ['~> 0.6.7']
+
+end

data/lib/hirefire.rb

@@ -0,0 +1,85 @@
+# encoding: utf-8
+
+module HireFire
+
+  ##
+  # HireFire constants
+  LIB_PATH         = File.dirname(__FILE__)
+  FREELANCER_PATH  = File.join(LIB_PATH,        'hirefire')
+  ENVIRONMENT_PATH = File.join(FREELANCER_PATH, 'environment')
+  BACKEND_PATH     = File.join(FREELANCER_PATH, 'backend')
+
+  ##
+  # HireFire namespace
+  autoload :Configuration, File.join(FREELANCER_PATH, 'configuration')
+  autoload :Environment,   File.join(FREELANCER_PATH, 'environment')
+  autoload :Initializer,   File.join(FREELANCER_PATH, 'initializer')
+  autoload :Backend,       File.join(FREELANCER_PATH, 'backend')
+  autoload :Logger,        File.join(FREELANCER_PATH, 'logger')
+  autoload :Version,       File.join(FREELANCER_PATH, 'version')
+
+  ##
+  # HireFire::Environment namespace
+  module Environment
+    autoload :Base,   File.join(ENVIRONMENT_PATH, 'base')
+    autoload :Heroku, File.join(ENVIRONMENT_PATH, 'heroku')
+    autoload :Local,  File.join(ENVIRONMENT_PATH, 'local')
+    autoload :Noop,   File.join(ENVIRONMENT_PATH, 'noop')
+  end
+
+  ##
+  # HireFire::Backend namespace
+  module Backend
+    autoload :ActiveRecord, File.join(BACKEND_PATH, 'active_record')
+    autoload :Mongoid,      File.join(BACKEND_PATH, 'mongoid')
+  end
+
+  ##
+  # This method is used to configure HireFire
+  #
+  # @yield [config] the instance of HireFire::Configuration class
+  # @yieldparam [Fixnum] max_workers default: 1 (set at least 1)
+  # @yieldparam [Array] job_worker_ratio default: see example
+  # @yieldparam [Symbol, nil] environment (:heroku, :local, :noop or nil) - default: nil
+  #
+  # @note Every param has it's own defaults. It's best to leave the environment param at "nil".
+  #   When environment is set to "nil", it'll default to the :noop environment. This basically means
+  #   that you have to run "rake jobs:work" yourself from the console to get the jobs running in development mode.
+  #   In production, it'll automatically use :heroku if deployed to the Heroku platform.
+  #
+  # @example
+  #   HireFire.configure do |config|
+  #     config.environment      = nil
+  #     config.max_workers      = 5
+  #     config.job_worker_ratio = [
+  #       { :jobs => 1,   :workers => 1 },
+  #       { :jobs => 15,  :workers => 2 },
+  #       { :jobs => 35,  :workers => 3 },
+  #       { :jobs => 60,  :workers => 4 },
+  #       { :jobs => 80,  :workers => 5 }
+  #     ]
+  #   end
+  #
+  # @return [nil]
+  def self.configure
+    yield(configuration); nil
+  end
+
+  ##
+  # Instantiates a new HireFire::Configuration
+  # instance and instance variable caches it
+  def self.configuration
+    @configuration ||= HireFire::Configuration.new
+  end
+
+end
+
+##
+# If Ruby on Rails is detected, it'll automatically initialize HireFire
+# so that the developer doesn't have to manually invoke it from an initializer file
+#
+# Users not using Ruby on Rails will have to run "HireFire::Initializer.initialize!"
+# in their application manually, after loading Delayed Job and the desired mapper (ActiveRecord or Mongoid)
+if defined?(Rails)
+  require File.join(HireFire::FREELANCER_PATH, 'railtie')
+end

data/lib/hirefire/backend.rb

@@ -0,0 +1,22 @@
+# encoding: utf-8
+
+module HireFire
+  module Backend
+
+    ##
+    # Load the correct module (ActiveRecord or Mongoid)
+    # based on which Delayed::Backend has been loaded
+    #
+    # @return [nil]
+    def self.included(base)
+      if defined?(Delayed::Backend::ActiveRecord::Job)
+        base.send(:include, ActiveRecord)
+      end
+
+      if defined?(Delayed::Backend::Mongoid::Job)
+        base.send(:include, Mongoid)
+      end
+    end
+
+  end
+end

data/lib/hirefire/backend/active_record.rb

@@ -0,0 +1,20 @@
+# encoding: utf-8
+
+module HireFire
+  module Backend
+    module ActiveRecord
+
+      ##
+      # Counts the amount of queued jobs in the database,
+      # failed jobs are excluded from the sum
+      #
+      # @return [Fixnum]
+      def jobs
+        Delayed::Job.
+        where(:failed_at => nil).
+        where('run_at <= ?', Time.now).count
+      end
+
+    end
+  end
+end

data/lib/hirefire/backend/mongoid.rb

@@ -0,0 +1,21 @@
+# encoding: utf-8
+
+module HireFire
+  module Backend
+    module Mongoid
+
+      ##
+      # Counts the amount of queued jobs in the database,
+      # failed jobs and jobs scheduled for the future are excluded
+      #
+      # @return [Fixnum]
+      def jobs
+        Delayed::Job.where(
+          :failed_at  => nil,
+          :run_at.lte => Time.now
+        ).count
+      end
+
+    end
+  end
+end

data/lib/hirefire/configuration.rb

@@ -0,0 +1,46 @@
+# encoding: utf-8
+
+module HireFire
+  class Configuration
+
+    ##
+    # Contains the max amount of workers that are allowed to run concurrently
+    #
+    # @return [Fixnum] default: 1
+    attr_accessor :max_workers
+
+    ##
+    # Contains the job/worker ratio which determines
+    # how many workers need to be running depending on
+    # the amount of pending jobs
+    #
+    # @return [Array] containing one or more hashes
+    attr_accessor :job_worker_ratio
+
+    ##
+    # Default is nil, in which case it'll auto-detect either :heroku or :noop,
+    # depending on the environment. It will never use :local, unless explicitly defined by the user.
+    #
+    # @param [Symbol, nil] environment Contains the name of the environment to run in.
+    # @return [Symbol, nil] default: nil
+    attr_accessor :environment
+
+    ##
+    # Instantiates a new HireFire::Configuration object
+    # with the default configuration. These default configurations
+    # may be overwritten using the HireFire.configure class method
+    #
+    # @return [HireFire::Configuration]
+    def initialize
+      @max_workers      = 1
+      @job_worker_ratio = [
+          { :jobs => 1,   :workers => 1 },
+          { :jobs => 25,  :workers => 2 },
+          { :jobs => 50,  :workers => 3 },
+          { :jobs => 75,  :workers => 4 },
+          { :jobs => 100, :workers => 5 }
+        ]
+    end
+
+  end
+end

data/lib/hirefire/delayed_job_extension.rb

@@ -0,0 +1,58 @@
+# encoding: utf-8
+
+module Delayed
+  class Worker
+
+    ##
+    # @note
+    #   This method gets invoked on heroku by the rake task "jobs:work"
+    #
+    #   This is basically the same method as the Delayed Job version,
+    #   except for the following:
+    #
+    #   1. All ouput will now go through the HireFire::Logger.
+    #   2. When HireFire cannot find any jobs to process it sends the "fire"
+    #      signal to all workers, ending all the processes simultaneously. The reason
+    #      we wait for all the processes to finish before sending the signal is because it'll
+    #      otherwise interrupt workers and leave jobs unfinished.
+    #
+    def start
+      HireFire::Logger.message "Starting job worker!"
+
+      trap('TERM') { HireFire::Logger.message 'Exiting...'; $exit = true }
+      trap('INT')  { HireFire::Logger.message 'Exiting...'; $exit = true }
+
+      queued = Delayed::Job.new
+
+      loop do
+        result = nil
+
+        realtime = Benchmark.realtime do
+          result = work_off
+        end
+
+        count = result.sum
+
+        break if $exit
+
+        if count.zero?
+          sleep(1)
+        else
+          HireFire::Logger.message "#{count} jobs processed at %.4f j/s, %d failed ..." % [count / realtime, result.last]
+        end
+
+        ##
+        # If there are no jobs currently queued,
+        # and the worker is still running, it'll kill itself
+        if queued.jobs == 0
+          Delayed::Job.environment.fire
+        end
+
+        break if $exit
+      end
+
+    ensure
+      Delayed::Job.clear_locks!(name)
+    end
+  end
+end

data/lib/hirefire/environment.rb

@@ -0,0 +1,50 @@
+# encoding: utf-8
+
+module HireFire
+  module Environment
+
+    ##
+    # This gets included in to the Delayed::Backend::(ActiveRecord|Mongoid)::Job
+    # classes and will add the necessary hooks (after_create, after_destroy and after_update)
+    # to spawn or kill Delayed Job worker processes on either Heroku or your local machine
+    #
+    # @param (Class) base This is the class in which this module will be included
+    def self.included(base)
+      base.send :extend, ClassMethods
+      base.class_eval do
+        after_create  'self.class.environment.hire'
+        after_destroy 'self.class.environment.fire'
+        after_update  'self.class.environment.fire',
+          :unless => Proc.new { |job| job.failed_at.nil? }
+      end
+
+      Logger.message("#{ base.name } detected!")
+    end
+
+    ##
+    # Class methods that will be added to the Delayed::Job backend
+    module ClassMethods
+
+      ##
+      # Returns the environment class method (for Delayed::Job ORM/ODM class)
+      #
+      # If HireFire.configuration.environment is nil (the default) then it'll
+      # auto-detect which environment to run in (either Heroku or Local)
+      #
+      # If HireFire.configuration.environment isn't nil (explicitly set) then
+      # it'll run in the specified environment (Heroku, Local or Noop)
+      #
+      # @return [HireFire::Environment::Heroku, HireFire::Environment::Local, HireFire::Environment::Noop]
+      def environment
+        @environment ||= HireFire::Environment.const_get(
+          if environment = HireFire.configuration.environment
+            environment.to_s.camelize
+          else
+            ENV.include?('HEROKU_UPID') ? 'Heroku' : 'Noop'
+          end
+        ).new
+      end
+    end
+
+  end
+end

data/lib/hirefire/environment/base.rb

@@ -0,0 +1,102 @@
+# encoding: utf-8
+
+module HireFire
+  module Environment
+    class Base
+
+      ##
+      # Include HireFire::Backend helpers
+      include HireFire::Backend
+
+      ##
+      # This method gets invoked when a new job has been queued
+      #
+      # Iterates through the default (or user-defined) job/worker ratio until
+      # it finds a match for the for the current situation (see example).
+      #
+      # @example
+      #   # Say we have 40 queued jobs, and we configured our job/worker ratio like so:
+      #
+      #   HireFire.configure do |config|
+      #     config.max_workers      = 5
+      #     config.job_worker_ratio = [
+      #       { :jobs => 1,   :workers => 1 },
+      #       { :jobs => 15,  :workers => 2 },
+      #       { :jobs => 35,  :workers => 3 },
+      #       { :jobs => 60,  :workers => 4 },
+      #       { :jobs => 80,  :workers => 5 }
+      #     ]
+      #   end
+      #
+      #   # It'll match at { :jobs => 35, :workers => 3 }, (35 jobs or more: hire 3 workers)
+      #   # meaning that it'll ensure there are 3 workers running.
+      #
+      #   # If there were already were 3 workers, it'll leave it as is
+      #   
+      #   # If there were more than 3 workers running (say, 4 or 5), it will NOT reduce
+      #   # the number. This is because when you reduce the number of workers, you cannot
+      #   # tell which worker Heroku will shut down, meaning you might interrupt a worker
+      #   # that's currently working, causing the job to fail. Also, consider the fact that
+      #   # there are, for example, 35 jobs still to be picked up, so the more workers,
+      #   # the faster it processes. You aren't even paying more because it doesn't matter whether
+      #   # you have 1 worker, or 5 workers processing jobs, because workers are pro-rated to the second.
+      #   # So basically 5 workers would cost 5 times more, but will also process 5 times faster.
+      #
+      #   # Once all jobs finished processing (e.g. Delayed::Job.jobs == 0), HireFire will invoke a signal
+      #   # which will set the workers back to 0 and shuts down all the workers simultaneously.
+      #
+      # @return [nil]
+      def hire
+        jobs_count    = jobs
+        workers_count = workers
+
+        ratio.each do |ratio|
+          if jobs_count >= ratio[:jobs] and max_workers >= ratio[:workers]
+            if not workers_count == ratio[:workers]
+              Logger.message("Hiring more workers so we have #{ ratio[:workers] } in total.")
+              workers(ratio[:workers])
+            end
+
+            break
+          end
+        end
+      end
+
+      ##
+      # This method gets invoked when a job is either "destroyed"
+      # or "updated, unless the job didn't fail"
+      #
+      # If there are workers active, but there are no more pending jobs,
+      # then fire all the workers
+      #
+      # @return [nil]
+      def fire
+        if jobs == 0 and workers > 0
+          Logger.message("All queued jobs have been processed. Firing all workers.")
+          workers(0)
+        end
+      end
+
+      private
+
+      ##
+      # Wrapper method for HireFire.configuration
+      # Returns the max amount of workers that may run concurrently
+      #
+      # @return [Fixnum] the max amount of workers that are allowed to run concurrently
+      def max_workers
+        HireFire.configuration.max_workers
+      end
+
+      ##
+      # Wrapper method for HireFire.configuration
+      # Returns the job/worker ratio array (in reversed order)
+      #
+      # @return [Array] the array of hashes containing the job/worker ratio (in reversed order)
+      def ratio
+        HireFire.configuration.job_worker_ratio.reverse
+      end
+
+    end
+  end
+end

data/lib/hirefire/environment/heroku.rb

@@ -0,0 +1,46 @@
+# encoding: utf-8
+
+require 'heroku'
+
+module HireFire
+  module Environment
+    class Heroku < Base
+
+      private
+
+      ##
+      # Either retrieves the amount of currently running workers,
+      # or set the amount of workers to a specific amount by providing a value
+      #
+      # @overload workers(amount = nil)
+      #   @param [Fixnum] amount will tell heroku to run N workers
+      #   @return [nil]
+      # @overload workers(amount = nil)
+      #   @param [nil] amount
+      #   @return [Fixnum] will request the amount of currently running workers from Heroku
+      def workers(amount = nil)
+
+        #
+        # Returns the amount of Delayed Job
+        # workers that are currently running on Heroku
+        if amount.nil?
+          return client.info(ENV['APP_NAME'])[:workers].to_i
+        end
+
+        ##
+        # Sets the amount of Delayed Job
+        # workers that need to be running on Heroku
+        client.set_workers(ENV['APP_NAME'], amount)
+      end
+
+      ##
+      # @return [Heroku::Client] instance of the heroku client
+      def client
+        @client ||= ::Heroku::Client.new(
+          ENV['HIREFIRE_EMAIL'], ENV['HIREFIRE_PASSWORD']
+        )
+      end
+
+    end
+  end
+end

data/lib/hirefire/environment/local.rb

@@ -0,0 +1,84 @@
+# encoding: utf-8
+
+require 'rush'
+
+module HireFire
+  module Environment
+    class Local < Base
+
+      private
+
+      ##
+      # Either retrieve the amount of currently running workers,
+      # or set the amount of workers to a specific amount by providing a value
+      #
+      # @overload workers(amount = nil)
+      #   @param [Fixnum] amount will tell the local machine to run N workers
+      #   @return [nil]
+      # @overload workers(amount = nil)
+      #   @param [nil] amount
+      #   @return [Fixnum] will request the amount of currently running workers from the local machine
+      def workers(amount = nil)
+
+        ##
+        # Returns the amount of Delayed Job workers that are currently
+        # running on the local machine if amount is nil
+        if amount.nil?
+          return Rush::Box.new.processes.filter(
+            :cmdline => /rake jobs:work WORKER=HIREFIRE/
+          ).size
+        end
+
+        ##
+        # Fire workers
+        #
+        # If the amount of workers required is set to 0
+        # then we fire all the workers and we return
+        #
+        # The worker that finished the last job will go ahead and
+        # kill all the other (if any) workers first, and then kill itself afterwards
+        if amount == 0
+
+          ##
+          # Gather process ids from all HireFire workers
+          pids = Rush::Box.new.processes.filter(
+            :cmdline => /rake jobs:work WORKER=HIREFIRE/
+          ).map(&:pid)
+
+          ##
+          # Instantiate a new local (shell) connection
+          shell = Rush::Connection::Local.new
+
+          ##
+          # Kill all Freelance workers,
+          # except the one that's doing the killing
+          (pids - [Rush.my_process.pid]).each do |pid|
+            shell.kill_process(pid)
+          end
+
+          ##
+          # Kill the last Freelance worker (self)
+          Logger.message("There are now #{ amount } workers.")
+          shell.kill_process(Rush.my_process.pid)
+          return
+        end
+
+        ##
+        # Hire workers
+        #
+        # If the amount of workers required is greater than
+        # the amount of workers already working, then hire the
+        # additional amount of workers required
+        workers_count = workers
+        if amount > workers_count
+          (amount - workers_count).times do
+            Rush::Box.new[Rails.root].bash(
+              'rake jobs:work WORKER=HIREFIRE', :background => true
+            )
+          end
+        end
+      end
+
+    end
+  end
+end

data/lib/hirefire/environment/noop.rb

@@ -0,0 +1,19 @@
+# encoding: utf-8
+
+module HireFire
+  module Environment
+    class Noop
+
+      ##
+      # Will invoke the #hire method, but won't actually do anything
+      def hire
+      end
+
+      ##
+      # Will invoke the #fire method, but won't actually do anything
+      def fire
+      end
+
+    end
+  end
+end

data/lib/hirefire/initializer.rb

@@ -0,0 +1,37 @@
+# encoding: utf-8
+
+module HireFire
+  class Initializer
+
+    ##
+    # Loads the HireFire extension in to Delayed Job and
+    # extends the Delayed Job "jobs:work" rake task command
+    #
+    # @return [nil]
+    def self.initialize!
+      ##
+      # If DelayedJob is using ActiveRecord, then include
+      # HireFire::Environment in to the ActiveRecord Delayed Job Backend
+      if defined?(Delayed::Backend::ActiveRecord::Job)
+        Delayed::Backend::ActiveRecord::Job.
+        send(:include, HireFire::Environment).
+        send(:include, HireFire::Backend)
+      end
+
+      ##
+      # If DelayedJob is using Mongoid, then include
+      # HireFire::Environment in to the Mongoid Delayed Job Backend
+      if defined?(Delayed::Backend::Mongoid::Job)
+        Delayed::Backend::Mongoid::Job.
+        send(:include, HireFire::Environment).
+        send(:include, HireFire::Backend)
+      end
+
+      ##
+      # Load Delayed Job extension, this is the start
+      # method that gets invoked when running "rake jobs:work"
+      require File.dirname(__FILE__) + '/delayed_job_extension'
+    end
+
+  end
+end

data/lib/hirefire/logger.rb

@@ -0,0 +1,98 @@
+# encoding: utf-8
+
+module HireFire
+  class Logger
+
+    ##
+    # Outputs a messages to the console
+    #
+    # @param [String] string prints a string to the console (green color)
+    # @return [nil]
+    def self.message(string)
+      puts loggify(string, :green)
+    end
+
+    ##
+    # Outputs an error to the console
+    #
+    # @param [String] string prints a string to the console (red color)
+    # @return [nil]
+    def self.error(string)
+      puts loggify(string, :red)
+    end
+
+    ##
+    # Outputs a notice to the console
+    #
+    # @param [String] string prints a string to the console (yellow color)
+    # @return [nil]
+    def self.warn(string)
+      puts loggify(string, :yellow)
+    end
+
+    ##
+    # Outputs the data as if it were a regular 'puts' command
+    #
+    # @param [String] string prints a string to the console (standard color)
+    # @return [nil]
+    def self.normal(string)
+      puts string
+    end
+
+    ##
+    # Builds the string in a log format with the date/time, the type (colorized)
+    # based on whether it's a message, notice or error, and the message itself.
+    #
+    # @param [String] string the string to print to the console
+    # @param [Symbol, false] color the color to print the string in
+    # @return [String] the log-like formatted string
+    def self.loggify(string, color = false)
+      return "[#{time}][HireFire] #{string}" unless color
+      "[#{time}][#{send(color, 'HireFire')}] #{string}"
+    end
+
+    ##
+    # @return [Time] the time in [YYYY-MM-DD HH:MM:SS] format
+    def self.time
+      Time.now.strftime("%Y-%m-%d %H:%M:%S")
+    end
+
+    ##
+    # Invokes the #colorize method with the provided string
+    # and the color code "32" (for green)
+    #
+    # @param [String] string
+    # @return [String] the provided string in special tags to color it green in the console
+    def self.green(string)
+      colorize(string, 32)
+    end
+
+    ##
+    # Invokes the #colorize method with the provided string
+    # and the color code "33" (for yellow)
+    #
+    # @param [String] string
+    # @return [String] the provided string in special tags to color it yellow in the console
+    def self.yellow(string)
+      colorize(string, 33)
+    end
+
+    ##
+    # Invokes the #colorize method the with provided string
+    # and the color code "31" (for red)
+    def self.red(string)
+      colorize(string, 31)
+    end
+
+    ##
+    # Wraps the provided string in colorizing tags to provide
+    # easier to view output to the client
+    #
+    # @param [String] string
+    # @return [String] the provided string in special tags to color it red in the console
+    def self.colorize(string, code)
+      "\e[#{code}m#{string}\e[0m"
+    end
+
+  end
+end

data/lib/hirefire/railtie.rb

@@ -0,0 +1,14 @@
+# encoding: utf-8
+
+module HireFire
+  class Railtie < Rails::Railtie
+
+    ##
+    # Initializes HireFire for Delayed Job when
+    # the Ruby on Rails web framework is done loading
+    initializer :after_initialize do
+      HireFire::Initializer.initialize!
+    end
+
+  end
+end

data/lib/hirefire/version.rb

@@ -0,0 +1,13 @@
+# encoding: utf-8
+
+module HireFire
+  module Version
+
+    ##
+    # @return [String] the current version of the HireFire gem
+    def self.current
+      '0.1.0'
+    end
+
+  end
+end

data/spec/configuration_spec.rb

@@ -0,0 +1,47 @@
+# encoding: utf-8
+
+require File.expand_path('../spec_helper', __FILE__)
+
+describe HireFire::Configuration do
+
+  it 'should have defaults' do
+    configuration = HireFire.configuration
+
+    configuration.environment.should      == nil
+    configuration.max_workers.should      == 1
+    configuration.job_worker_ratio.should == [
+        { :jobs => 1,   :workers => 1 },
+        { :jobs => 25,  :workers => 2 },
+        { :jobs => 50,  :workers => 3 },
+        { :jobs => 75,  :workers => 4 },
+        { :jobs => 100, :workers => 5 }
+      ]
+  end
+
+  it 'should be configurable' do
+    HireFire.configure do |config|
+      config.environment      = :noop
+      config.max_workers      = 10
+      config.job_worker_ratio = [
+          { :jobs => 1,   :workers => 1 },
+          { :jobs => 15,  :workers => 2 },
+          { :jobs => 35,  :workers => 3 },
+          { :jobs => 60,  :workers => 4 },
+          { :jobs => 80,  :workers => 5 }
+        ]
+    end
+
+    configuration = HireFire.configuration
+
+    configuration.environment.should      == :noop
+    configuration.max_workers.should      == 10
+    configuration.job_worker_ratio.should == [
+        { :jobs => 1,   :workers => 1 },
+        { :jobs => 15,  :workers => 2 },
+        { :jobs => 35,  :workers => 3 },
+        { :jobs => 60,  :workers => 4 },
+        { :jobs => 80,  :workers => 5 }
+      ]
+  end
+
+end

data/spec/logger_spec.rb

@@ -0,0 +1,37 @@
+# encoding: utf-8
+
+require File.expand_path('../spec_helper', __FILE__)
+
+require 'timecop'
+
+describe HireFire::Logger do
+
+  before do
+    Timecop.freeze( Time.now )
+  end
+
+  context 'when logging regular messages' do
+    it do
+      HireFire::Logger.expects(:puts).with("[#{ Time.now.strftime("%Y-%m-%d %H:%M:%S") }][\e[32mHireFire\e[0m] This has been logged.")
+
+      HireFire::Logger.message "This has been logged."
+    end
+  end
+
+  context 'when logging error messages' do
+    it do
+      HireFire::Logger.expects(:puts).with("[#{ Time.now.strftime("%Y-%m-%d %H:%M:%S") }][\e[31mHireFire\e[0m] This has been logged.")
+
+      HireFire::Logger.error "This has been logged."
+    end
+  end
+
+  context 'when logging warn messages' do
+    it do
+      HireFire::Logger.expects(:puts).with("[#{ Time.now.strftime("%Y-%m-%d %H:%M:%S") }][\e[33mHireFire\e[0m] This has been logged.")
+
+      HireFire::Logger.warn "This has been logged."
+    end
+  end
+
+end

data/spec/spec_helper.rb

@@ -0,0 +1,15 @@
+# encoding: utf-8
+
+##
+# Path to the lib directory
+LIB_PATH = File.expand_path('../../lib', __FILE__)
+
+##
+# Load the HireFire Ruby library
+require File.join(LIB_PATH, 'hirefire')
+
+##
+# Use Mocha to mock with RSpec
+RSpec.configure do |config|
+  config.mock_with :mocha
+end

metadata

@@ -0,0 +1,102 @@
+--- !ruby/object:Gem::Specification 
+name: hirefire
+version: !ruby/object:Gem::Version 
+  prerelease: 
+  version: 0.1.0
+platform: ruby
+authors: 
+- Michael van Rooijen
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2011-04-10 00:00:00 Z
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: heroku
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 1.20.1
+  type: :runtime
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
+  name: rush
+  prerelease: false
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 0.6.7
+  type: :runtime
+  version_requirements: *id002
+description: "                      HireFire automatically \"hires\" and \"fires\" (aka \"scales\") Delayed Job workers on Heroku.\n                      When there are no queue jobs, HireFire will fire (shut down) all workers. If there are\n                      queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired\n                      depends on the amount of queued jobs (the ratio can be configured by you). HireFire\n                      is great for both high, mid and low traffic applications. It can save you a lot of\n                      money by only hiring workers when there are pending jobs, and then firing them again\n                      once all the jobs have been processed. It's also capable to dramatically reducing\n                      processing time by automatically hiring more workers when the queue size increases.\n"
+email: meskyanichi@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: []
+
+files: 
+- .document
+- .gitignore
+- .infinity_test
+- .rspec
+- Gemfile
+- Gemfile.lock
+- LICENSE.md
+- README.md
+- hirefire.gemspec
+- lib/hirefire.rb
+- lib/hirefire/backend.rb
+- lib/hirefire/backend/active_record.rb
+- lib/hirefire/backend/mongoid.rb
+- lib/hirefire/configuration.rb
+- lib/hirefire/delayed_job_extension.rb
+- lib/hirefire/environment.rb
+- lib/hirefire/environment/base.rb
+- lib/hirefire/environment/heroku.rb
+- lib/hirefire/environment/local.rb
+- lib/hirefire/environment/noop.rb
+- lib/hirefire/initializer.rb
+- lib/hirefire/logger.rb
+- lib/hirefire/railtie.rb
+- lib/hirefire/version.rb
+- spec/configuration_spec.rb
+- spec/logger_spec.rb
+- spec/spec_helper.rb
+homepage: http://rubygems.org/gems/hirefire
+licenses: []
+
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.7.2
+signing_key: 
+specification_version: 3
+summary: HireFire automatically "hires" and "fires" (aka "scales") Delayed Job workers on Heroku.
+test_files: []
+
+has_rdoc: