workless 1.0.2 → 1.1.0

data/README.md

@@ -7,9 +7,10 @@ It is designed to be used when you're using Heroku as a host and have the need t
 
 By adding the gem to your project and configuring our Heroku app with some config variables workless should do the rest.
 
-## Update
+## Updates
 
-Version 1.0.0 has been released, this brings compatibility with delayed_job 3 and compatibility with Rails 2.3.x and up.
+* Version 1.1.0 has been released, this adds support for scaling using multiple workers thanks to @jaimeiniesta and @davidakachaos.
+* Version 1.0.0 has been released, this brings compatibility with delayed_job 3 and compatibility with Rails 2.3.x and up.
 
 ## Compatibility
 
@@ -48,17 +49,10 @@ gem "workless", "~> 1.0.1"
 
 If you don't specify delayed_job in your Gemfile workless will bring it in, most likly the latest version (3.0.1)
 
-Add your Heroku username / password as config vars to your Heroku instance, the same one you log in to Heroku with.
-[You may also use your API key as the password](https://github.com/heroku/heroku/issues/103).
+Add your Heroku app name / [API key](https://devcenter.heroku.com/articles/authentication) as config vars to your Heroku instance.
 
 <pre>
-heroku config:add HEROKU_USER=yourusername HEROKU_PASSWORD=yourpassword
-</pre>
-
-And for cedar apps add the app name
-
-<pre>
-heroku config:add APP_NAME=yourherokuappname
+heroku config:add HEROKU_API_KEY=yourapikey APP_NAME=yourherokuappname
 </pre>
 
 ## Failing Jobs
@@ -87,9 +81,21 @@ The local scaler uses @adamwiggins rush library http://github.com/adamwiggins/ru
 
 The heroku scaler works on the Aspen and Bamboo stacks while the heroku_cedar scaler only works on the new Cedar stack.
 
+## Scaling to multiple workers
+
+As an experimental feature for the Cedar stack, Workless can scale to more than 1 worker based on the current work load. You just need to define these config variables on your app, setting the values you want:
+
+<pre>
+heroku config:add WORKLESS_MAX_WORKERS=10
+heroku config:add WORKLESS_MIN_WORKERS=0
+heroku config:add WORKLESS_WORKERS_RATIO=50
+</pre>
+
+In this example, it will scale up to a maximum of 10 workers, firing up 1 worker for every 50 jobs on the queue. The minimum will be 0 workers, but you could set it to a higher value if you want.
+
 ## Note on Patches/Pull Requests
  
-* Please fork the project, as you can see there are no tests and at present I don't know how to go about adding them so any advice would be welcome.
+* Please fork the project.
 * Make your feature addition or bug fix.
 * Commit, do not mess with rakefile, version, or history.
   (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)

data/lib/workless.rb

@@ -1,4 +1,4 @@
-require "heroku"
+require "heroku-api"
 require File.dirname(__FILE__) + "/workless/scalers/base"
 require File.dirname(__FILE__) + "/workless/scaler"
 require File.dirname(__FILE__) + "/workless/railtie" if defined?(Rails::Railtie)

data/lib/workless/scaler.rb

@@ -19,14 +19,8 @@ module Delayed
       
       module ClassMethods
         def scaler
-          @scaler ||= if ENV.include?("HEROKU_USER")
-            client = ::Heroku::Client.new(ENV['HEROKU_USER'], ENV['HEROKU_PASSWORD'])
-            case client.info(ENV["APP_NAME"])[:stack]
-            when "cedar"
-              Scaler::HerokuCedar
-            else  
-              Scaler::Heroku
-            end
+          @scaler ||= if ENV.include?("HEROKU_API_KEY")
+            Scaler::HerokuCedar
           else
             Scaler::Local
           end

data/lib/workless/scalers/base.rb

@@ -13,7 +13,7 @@ module Delayed
       module HerokuClient
 
         def client
-          @client ||= ::Heroku::Client.new(ENV['HEROKU_USER'], ENV['HEROKU_PASSWORD'])
+          @client ||= ::Heroku::API.new(:api_key => ENV['HEROKU_API_KEY'])
         end
 
       end

data/lib/workless/scalers/heroku.rb

@@ -1,4 +1,4 @@
-require 'heroku'
+require 'heroku-api'
 
 module Delayed
   module Workless
@@ -9,15 +9,15 @@ module Delayed
         extend Delayed::Workless::Scaler::HerokuClient
 
         def self.up
-          client.set_workers(ENV['APP_NAME'], 1) if self.workers == 0
+          client.put_workers(ENV['APP_NAME'], 1) if self.workers == 0
         end
 
         def self.down
-          client.set_workers(ENV['APP_NAME'], 0) unless self.workers == 0 or self.jobs.count > 0
+          client.put_workers(ENV['APP_NAME'], 0) unless self.workers == 0 or self.jobs.count > 0
         end
 
         def self.workers
-          client.info(ENV['APP_NAME'])[:workers].to_i
+          client.get_ps(ENV['APP_NAME']).body.count { |p| p["process"] =~ /worker\.\d?/ }
         end
 
       end

data/lib/workless/scalers/heroku_cedar.rb

@@ -1,27 +1,49 @@
-require 'heroku'
+require 'heroku-api'
 
 module Delayed
   module Workless
     module Scaler
-
       class HerokuCedar < Base
-
         extend Delayed::Workless::Scaler::HerokuClient
 
         def self.up
-          client.ps_scale(ENV['APP_NAME'], :type => 'worker', :qty => 1) if self.workers == 0
+          client.post_ps_scale(ENV['APP_NAME'], 'worker', self.workers_needed) if self.workers < self.workers_needed
         end
 
         def self.down
-          client.ps_scale(ENV['APP_NAME'], :type => 'worker', :qty => 0) unless self.workers == 0 or self.jobs.count > 0
+          client.post_ps_scale(ENV['APP_NAME'], 'worker', self.min_workers) unless self.workers == self.min_workers or self.jobs.count > 0
         end
 
         def self.workers
-          client.ps(ENV['APP_NAME']).count { |p| p["process"] =~ /worker\.\d?/ }
+          client.get_ps(ENV['APP_NAME']).body.count { |p| p["process"] =~ /worker\.\d?/ }
         end
 
-      end
+        # Returns the number of workers needed based on the current number of pending jobs and the settings defined by:
+        #
+        # ENV['WORKLESS_WORKERS_RATIO']
+        # ENV['WORKLESS_MAX_WORKERS']
+        # ENV['WORKLESS_MIN_WORKERS']
+        #
+        def self.workers_needed
+          [[(self.jobs.count.to_f / self.workers_ratio).ceil, self.max_workers].min, self.min_workers].max
+        end
+
+        def self.workers_ratio
+          if ENV['WORKLESS_WORKERS_RATIO'].present? && (ENV['WORKLESS_WORKERS_RATIO'].to_i != 0)
+            ENV['WORKLESS_WORKERS_RATIO'].to_i
+          else
+            100
+          end
+        end
+
+        def self.max_workers
+          ENV['WORKLESS_MAX_WORKERS'].present? ? ENV['WORKLESS_MAX_WORKERS'].to_i : 1
+        end
 
+        def self.min_workers
+          ENV['WORKLESS_MIN_WORKERS'].present? ? ENV['WORKLESS_MIN_WORKERS'].to_i : 0
+        end
+      end
     end
   end
-end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: workless
 version: !ruby/object:Gem::Version
-  version: 1.0.2
+  version: 1.1.0
   prerelease: 
 platform: ruby
 authors:
@@ -13,7 +13,7 @@ date: 2011-06-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
-  requirement: &70192881277980 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,15 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70192881277980
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: heroku
-  requirement: &70192881277400 !ruby/object:Gem::Requirement
+  name: heroku-api
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +37,15 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70192881277400
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rush
-  requirement: &70192881276860 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,10 +53,15 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70192881276860
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: delayed_job
-  requirement: &70192881276240 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -54,10 +69,15 @@ dependencies:
         version: 2.0.7
   type: :runtime
   prerelease: false
-  version_requirements: *70192881276240
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.7
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70192881275520 !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -65,7 +85,12 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70192881275520
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Extension to Delayed Job to enable workers to scale up when needed
 email: paul.crabtree@gmail.com
 executables: []
@@ -104,7 +129,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: 1.3.6
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.6
+rubygems_version: 1.8.24
 signing_key: 
 specification_version: 3
 summary: Use delayed job workers only when theyre needed on Heroku