autoscaler 0.9.0 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c1870d13f3e44b1d9b2267f41fb8f5d94c94c6ec
-  data.tar.gz: 0a7662b031e66c6d9967567cb7612791fe6914ff
+  metadata.gz: 3d1fdf02eef4bd9e9ed8ca36ec7a88b0e298ead8
+  data.tar.gz: fd639d4797b01dc4aded9d0d11dc0d386d24f39e
 SHA512:
-  metadata.gz: 10566a1b1762379530e22dbfe1174212c7c23dc4078787fffdb9956d80d3dc33a4d1aba2c9298b1408579fbda1e336e05a9b1eb63124d6bfb48fd8b90de0cc94
-  data.tar.gz: 37fd250217015d0aabf1e99cc7773aacdbf51e6ed57bfbda0a18fe998369459b69fcbcf1126fa467ba1906539e4d20b12134ec944c17c2071849e61c5243f0ff
+  metadata.gz: 43c6d465b6bbbb1978d9ace6e5adc60db0a4c4aac72b206706f10397121dafc9ad2c59edd1a02395a10df8b01927abf445977be997d60bf0d19e172ed24aa85b
+  data.tar.gz: 27fdfa89b504de1b90a735929e96d9c96e7fd0f84e2b8d80ea05a2ec58d786af71f726109649dfa220b7d9a5cd038f34dc591d28a9218587c073a2a6b8fb9961

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 0.10.0
+
+- Require Sidekiq 3.5
+- You may use `HerokuPlatformScaler` and `HEROKU_ACCESS_TOKEN` in place of `HerkouScaler` and `HEROKU_API_KEY`
+- QueueSystem#workers returns the number of engaged SK processes.
+- Linear Scaling Strategy will not scale down past number of active workers. Assumes 1-1 SK process/dyno mapping.
+- Calls the SideKiq quiet api when shutting down
+- Count workers currently running (Joel Van Horn)
+- Update gems and use RSpec expect syntax (giviger)
+
 ## 0.9.0
 
 - CounterCacheRedis.new now takes a third parameter `worker_type`, a string used in the

data/Guardfile

@@ -4,11 +4,9 @@ end
 
 tag     = "--tag #{ENV['TAG']}"           if ENV['TAG']
 example = "--example '#{ENV['EXAMPLE']}'" if ENV['EXAMPLE']
-%w(sidekiq-2 sidekiq-3).each do |appraisal|
-  guard :rspec, :cmd => "appraisal #{appraisal} rspec --color --format d #{tag} #{example}" do
-    watch(%r{^spec/.+_spec\.rb$})
-    watch(%r{^lib/(.+).rb$}) { |m| "spec/#{m[1]}_spec.rb" }
-    watch('spec/spec_helper.rb')  { "spec" }
-  end
+guard :rspec, :cmd => "rspec --color --format d #{tag} #{example}" do
+  watch(%r{^spec/.+_spec\.rb$})
+  watch(%r{^lib/(.+).rb$}) { |m| "spec/#{m[1]}_spec.rb" }
+  watch('spec/spec_helper.rb')  { "spec" }
 end
 

data/README.md

@@ -1,10 +1,10 @@
 # Sidekiq Heroku Autoscaler
 
-[Sidekiq](https://github.com/mperham/sidekiq) performs background jobs.  While it's threading model allows it to scale easier than worker-pre-process background systems, people running test or lightly loaded systems on [Heroku](http://www.heroku.com/) still want to scale down to zero to avoid racking up charges.
+[Sidekiq](https://github.com/mperham/sidekiq) performs background jobs.  While its threading model allows it to scale easier than worker-pre-process background systems, people running test or lightly loaded systems on [Heroku](http://www.heroku.com/) still want to scale down to zero to avoid racking up charges.
 
 ## Requirements
 
-Tested on Ruby 1.9.2 and Heroku Cedar stack.
+Tested on Ruby 2.1.7 and Heroku Cedar stack.
 
 ## Installation
 
@@ -12,36 +12,38 @@ Tested on Ruby 1.9.2 and Heroku Cedar stack.
 
 ## Getting Started
 
-This gem uses the [Herkou-Api](https://github.com/heroku/heroku.rb) gem, which requires an API key from Heroku.  It will also need the heroku app name.  By default, these are specified through environment variables.  You can also pass them to HerokuScaler explicitly.
+This gem uses the [Heroku Platform-Api](https://github.com/heroku/platform-api.rb) gem, which requires an OAuth token from Heroku.  It will also need the heroku app name.  By default, these are specified through environment variables.  You can also pass them to `HerokuPlatformScaler` explicitly.
 
-    HEROKU_API_KEY=.....
+    HEROKU_ACCESS_TOKEN=.....
     HEROKU_APP=....
 
+Support is still present for [Heroku-Api](https://github.com/heroku/heroku.rb) via `HerkouScaler` and `HEROKU_API_KEY`, but may be removed in a future major version.
+
 Install the middleware in your `Sidekiq.configure_` blocks
 
     require 'autoscaler/sidekiq'
-    require 'autoscaler/heroku_scaler'
+    require 'autoscaler/heroku_platform_scaler'
 
     Sidekiq.configure_client do |config|
       config.client_middleware do |chain|
-        chain.add Autoscaler::Sidekiq::Client, 'default' => Autoscaler::HerokuScaler.new
+        chain.add Autoscaler::Sidekiq::Client, 'default' => Autoscaler::HerokuPlatformScaler.new
       end
     end
 
     Sidekiq.configure_server do |config|
       config.server_middleware do |chain|
-        chain.add(Autoscaler::Sidekiq::Server, Autoscaler::HerokuScaler.new, 60) # 60 second timeout
+        chain.add(Autoscaler::Sidekiq::Server, Autoscaler::HerokuPlatformScaler.new, 60) # 60 second timeout
       end
     end
 
 ## Limits and Challenges
 
-- HerokuScaler includes an attempt at current-worker cache that may be overcomplication, and doesn't work very well on the server
+- HerokuPlatformScaler includes an attempt at current-worker cache that may be overcomplication, and doesn't work very well on the server
 - Multiple scale-down loops may be started, particularly if there are multiple jobs queued when the servers comes up.  Heroku seems to handle multiple scale-down commands well.
 - The scale-down monitor is triggered on job completion (and server middleware is only run around jobs), so if the server nevers processes any jobs, it won't turn off.
 - The retry and schedule lists are considered - if you schedule a long-running task, the process will not scale-down.
 - If background jobs trigger jobs in other scaled processes, please note you'll need `config.client_middleware` in your `Sidekiq.configure_server` block in order to scale-up.
-- Exceptions while calling the Heroku API are caught and printed by default.  See `HerokuScaler#exception_handler` to override
+- Exceptions while calling the Heroku API are caught and printed by default.  See `HerokuPlatformScaler#exception_handler` to override
 
 ## Experimental
 
@@ -57,24 +59,28 @@ You can pass a scaling strategy object instead of the timeout to the server midd
 
 ### Working caching
 
-    scaler.counter_cache = Autoscaler::CounterCacheRedis(Sidekiq.method(:redis))
+    scaler.counter_cache = Autoscaler::CounterCacheRedis.new(Sidekiq.method(:redis))
 
 ## Tests
 
 The project is setup to run RSpec with Guard.  It expects a redis instance on a custom port, which is started by the Guardfile.
 
-The HerokuScaler is not tested by default because it makes live API requests.  Specify `HEROKU_APP` and `HEROKU_API_KEY` on the command line, and then watch your app's logs.
+The HerokuPlatformScaler is not tested by default because it makes live API requests.  Specify `HEROKU_APP` and `HEROKU_ACCESS_TOKEN` on the command line, and then watch your app's logs.
 
-    HEROKU_APP=... HEROKU_API_KEY=... guard
+    HEROKU_APP=... HEROKU_ACCESS_TOKEN=... guard
     heroku logs --app ...
 
 ## Authors
 
 Justin Love, [@wondible](http://twitter.com/wondible), [https://github.com/JustinLove](https://github.com/JustinLove)
 
-Ported to Heroku-Api by Fix Peña, [https://github.com/fixr](https://github.com/fixr)
+### Contributors
 
-Retry/schedule sets by Matt Anderson [https://github.com/tonkapark](https://github.com/tonkapark) and Thibaud Guillaume-Gentil [https://github.com/jilion](https://github.com/jilion)
+- Benjamin Kudria [https://github.com/bkudria](https://github.com/bkudria)
+- Fix Peña [https://github.com/fixr](https://github.com/fixr)
+- Gabriel Givigier Guimarães [https://github.com/givigier](https://github.com/givigier)
+- Matt Anderson [https://github.com/tonkapark](https://github.com/tonkapark)
+- Thibaud Guillaume-Gentil [https://github.com/jilion](https://github.com/jilion)
 
 ## Licence
 

data/examples/complex.rb

@@ -1,15 +1,19 @@
 require 'sidekiq'
 require 'autoscaler/sidekiq'
-require 'autoscaler/heroku_scaler'
+require 'autoscaler/heroku_platform_scaler'
+
+# This setup is for multiple queues, where each queue has a dedicated process type
 
 heroku = nil
 if ENV['HEROKU_APP']
   heroku = {}
   scaleable = %w[default import] - (ENV['ALWAYS'] || '').split(' ')
   scaleable.each do |queue|
-    heroku[queue] = Autoscaler::HerokuScaler.new(
+    # We are using the convention that worker process type is the
+    # same as the queue name
+    heroku[queue] = Autoscaler::HerokuPlatformScaler.new(
       queue,
-      ENV['HEROKU_API_KEY'],
+      ENV['HEROKU_ACCESS_TOKEN'],
       ENV['HEROKU_APP'])
   end
 end

data/examples/simple.rb

@@ -1,10 +1,12 @@
 require 'sidekiq'
 require 'autoscaler/sidekiq'
-require 'autoscaler/heroku_scaler'
+require 'autoscaler/heroku_platform_scaler'
+
+# This is setup for a single queue (default) and worker process (worker)
 
 heroku = nil
 if ENV['HEROKU_APP']
-  heroku = Autoscaler::HerokuScaler.new
+  heroku = Autoscaler::HerokuPlatformScaler.new
   #heroku.exception_handler = lambda {|exception| MyApp.logger.error(exception)}
 end
 

data/lib/autoscaler/binary_scaling_strategy.rb

@@ -20,7 +20,7 @@ module Autoscaler
 
     private
     def active?(system)
-      system.queued > 0 || system.scheduled > 0 || system.retrying > 0 || system.workers > 0
+      system.any_work?
     end
   end
 end

data/lib/autoscaler/heroku_platform_scaler.rb

@@ -0,0 +1,84 @@
+require 'platform-api'
+require 'autoscaler/counter_cache_memory'
+
+module Autoscaler
+  # Wraps the Heroku Platform API to provide just the interface that we need for scaling.
+  class HerokuPlatformScaler
+    # @param [String] type process type this scaler controls
+    # @param [String] token Heroku OAuth access token
+    # @param [String] app Heroku app name
+    def initialize(
+        type = 'worker',
+        token = ENV['HEROKU_ACCESS_TOKEN'],
+        app = ENV['HEROKU_APP'])
+      @client = PlatformAPI.connect_oauth(token)
+      @type = type
+      @app = app
+      @workers = CounterCacheMemory.new
+    end
+
+    attr_reader :app
+    attr_reader :type
+
+    # Read the current worker count (value may be cached)
+    # @return [Numeric] number of workers
+    def workers
+      @workers.counter {@workers.counter = heroku_get_workers}
+    end
+
+    # Set the number of workers (noop if workers the same)
+    # @param [Numeric] n number of workers
+    def workers=(n)
+      unknown = false
+      current = @workers.counter{unknown = true; 1}
+      if n != current || unknown
+        p "Scaling #{type} to #{n}"
+        heroku_set_workers(n)
+        @workers.counter = n
+      end
+    end
+
+    # Callable object which responds to exceptions during api calls #
+    # @example
+    #   heroku.exception_handler = lambda {|exception| MyApp.logger.error(exception)}
+    #   heroku.exception_handler = lambda {|exception| raise}
+    #   # default
+    #   lambda {|exception|
+    #     p exception
+    #     puts exception.backtrace
+    #   }
+    attr_writer :exception_handler
+
+    # Object which supports #counter and #counter=
+    # Defaults to CounterCacheMemory
+    def counter_cache=(cache)
+      @workers = cache
+    end
+
+    private
+    attr_reader :client
+
+    def heroku_get_workers
+      client.formation.list(app)
+        .select {|item| item['type'] == type}
+        .map {|item| item['quantity']}
+        .reduce(0, &:+)
+    rescue Excon::Errors::Error => e
+      exception_handler.call(e)
+      0
+    end
+
+    def heroku_set_workers(n)
+      client.formation.update(app, type, {:quantity => n})
+    rescue Excon::Errors::Error, Heroku::API::Errors::Error => e
+      exception_handler.call(e)
+    end
+
+    def exception_handler
+      @exception_handler ||= lambda {|exception|
+        p exception
+        puts exception.backtrace
+      }
+    end
+  end
+end

data/lib/autoscaler/ignore_scheduled_and_retrying.rb

@@ -1,9 +1,14 @@
 module Autoscaler
+  # - Strategy wrapper to ignore scheduled and retrying queues. Usage:
+  #  ``new_strategy = IgnoreScheduledAndRetrying.new(my_old_strategy)``
   class IgnoreScheduledAndRetrying
     def initialize(strategy)
       @strategy = strategy
     end
 
+    # @param [QueueSystem] system interface to the queuing system
+    # @param [Numeric] event_idle_time number of seconds since a job related event
+    # @return [Integer] target number of workers
     def call(system, event_idle_time)
       system.define_singleton_method(:scheduled) { 0 }
       system.define_singleton_method(:retrying)  { 0 }

data/lib/autoscaler/linear_scaling_strategy.rb

@@ -20,16 +20,20 @@ module Autoscaler
 
       # Scale requested capacity taking into account the minimum required
       scale_factor = (requested_capacity_percentage - @min_capacity_percentage) / (@total_capacity - @min_capacity_percentage)
+      scale_factor = 0 if scale_factor.nan? # Handle DIVZERO
+
       scaled_capacity_percentage = scale_factor * @total_capacity
 
       ideal_workers = ([0, scaled_capacity_percentage].max * @max_workers).ceil
+      min_workers   = [system.workers, ideal_workers].max  # Don't scale down past number of currently engaged workers
+      max_workers   = [min_workers,  @max_workers].min     # Don't scale up past number of max workers
 
-      return [ideal_workers, @max_workers].min
+      return [min_workers, max_workers].min
     end
 
     private
     def total_work(system)
-      system.queued + system.scheduled + system.retrying
+      system.total_work
     end
   end
 end

data/lib/autoscaler/sidekiq/celluloid_monitor.rb

@@ -1,4 +1,4 @@
-require 'celluloid'
+require 'celluloid/current'
 
 module Autoscaler
   module Sidekiq
@@ -6,7 +6,7 @@ module Autoscaler
     class CelluloidMonitor
       include Celluloid
 
-      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuScaler})
+      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuPlatformScaler})
       # @param [Strategy] strategy object that decides the target number of workers (e.g. {BinaryScalingStrategy})
       # @param [System] system interface to the queuing system for use by the strategy
       def initialize(scaler, strategy, system)
@@ -29,6 +29,7 @@ module Autoscaler
             target_workers = @strategy.call(@system, idle_time)
             workers = @scaler.workers = target_workers unless workers == target_workers
           end while workers > 0
+          ::Sidekiq::ProcessSet.new.each(&:quiet!)
         end
       end
 

data/lib/autoscaler/sidekiq/client.rb

@@ -6,7 +6,7 @@ module Autoscaler
     # Sidekiq client middleware
     # Performs scale-up when items are queued and there are no workers running
     class Client
-      # @param [Hash] scalers map of queue(String) => scaler (e.g. {HerokuScaler}).
+      # @param [Hash] scalers map of queue(String) => scaler (e.g. {HerokuPlatformScaler}).
       #   Which scaler to use for each sidekiq queue
       def initialize(scalers)
         @scalers = scalers

data/lib/autoscaler/sidekiq/entire_queue_system.rb

@@ -5,9 +5,10 @@ module Autoscaler
     # Interface to to interrogate the queuing system
     # Includes every queue
     class EntireQueueSystem
-      # @return [Integer] number of worker actively engaged
+      # @return [Integer] number of workers actively engaged
       def workers
-        ::Sidekiq::Workers.new.size
+        ::Sidekiq::Workers.new.map {|pid, _, _| pid}.uniq.size
+        # #size may be out-of-date.
       end
 
       # @return [Integer] amount work ready to go
@@ -25,6 +26,16 @@ module Autoscaler
         ::Sidekiq::RetrySet.new.size
       end
 
+      # @return [Boolean] if any kind of work still needs to be done
+      def any_work?
+        queued > 0 || scheduled > 0 || retrying > 0 || workers > 0
+      end
+
+      # @return [Integer] total amount of work
+      def total_work
+        queued + scheduled + retrying + workers
+      end
+
       # @return [Array[String]]
       def queue_names
         sidekiq_queues.keys

data/lib/autoscaler/sidekiq/monitor_middleware_adapter.rb

@@ -8,15 +8,17 @@ module Autoscaler
     # Shim to the existing autoscaler interface
     # Starts the monitor and notifies it of job events that may occur while it's sleeping
     class MonitorMiddlewareAdapter
-      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuScaler})
+      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuPlatformScaler})
       # @param [Strategy,Numeric] timeout strategy object that determines target workers, or a timeout  in seconds to be passed to {DelayedShutdown}+{BinaryScalingStrategy}
       # @param [Array[String]] specified_queues list of queues to monitor to determine if there is work left.  Defaults to all sidekiq queues.
       def initialize(scaler, timeout, specified_queues = nil)
         unless monitor
-          CelluloidMonitor.supervise_as(:autoscaler_monitor,
+          CelluloidMonitor.supervise :as => :autoscaler_monitor,
+                                     :args => [
                                         scaler,
                                         strategy(timeout),
-                                        QueueSystem.new(specified_queues))
+                                        QueueSystem.new(specified_queues),
+                                     ]
         end
       end
 
@@ -25,7 +27,8 @@ module Autoscaler
         monitor.async.starting_job
         yield
       ensure
-        monitor.async.finished_job
+        # monitor might have gone, e.g. if Sidekiq has received SIGTERM
+        monitor.async.finished_job if monitor
       end
 
       private

data/lib/autoscaler/sidekiq/sleep_wait_server.rb

@@ -6,7 +6,7 @@ module Autoscaler
     # Sidekiq server middleware
     # Performs scale-down when the queue is empty
     class SleepWaitServer
-      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuScaler})
+      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuPlatformScaler})
       # @param [Numeric] timeout number of seconds to wait before shutdown
       # @param [Array[String]] specified_queues list of queues to monitor to determine if there is work left.  Defaults to all sidekiq queues.
       def initialize(scaler, timeout, specified_queues = nil)
@@ -36,7 +36,7 @@ module Autoscaler
       attr_reader :system
 
       def pending_work?
-        system.queued > 0 || system.scheduled > 0 || system.retrying > 0
+        system.any_work?
       end
 
       def working!(queue, redis)

data/lib/autoscaler/sidekiq/specified_queue_system.rb

@@ -10,11 +10,11 @@ module Autoscaler
         @queue_names = specified_queues
       end
 
-      # @return [Integer] number of worker actively engaged
+      # @return [Integer] number of workers actively engaged
       def workers
-        ::Sidekiq::Workers.new.count {|name, work|
+        ::Sidekiq::Workers.new.select {|_, _, work|
           queue_names.include?(work['queue'])
-        }
+        }.map {|pid, _, _| pid}.uniq.size
       end
 
       # @return [Integer] amount work ready to go
@@ -32,6 +32,16 @@ module Autoscaler
         count_set(::Sidekiq::RetrySet.new)
       end
 
+      # @return [Boolean] if any kind of work still needs to be done
+      def any_work?
+        queued > 0 || scheduled > 0 || retrying > 0 || workers > 0
+      end
+
+      # @return [Integer] total amount of work
+      def total_work
+        queued + scheduled + retrying + workers
+      end
+
       # @return [Array[String]]
       attr_reader :queue_names
 

data/lib/autoscaler/version.rb

@@ -1,4 +1,4 @@
 module Autoscaler
   # version number
-  VERSION = "0.9.0"
+  VERSION = "0.10.0"
 end

data/spec/autoscaler/binary_scaling_strategy_spec.rb

@@ -8,12 +8,12 @@ describe Autoscaler::BinaryScalingStrategy do
   it "scales with no work" do
     system = TestSystem.new(0)
     strategy = cut.new
-    strategy.call(system, 1).should == 0
+    expect(strategy.call(system, 1)).to eq 0
   end
 
   it "does not scale with pending work" do
     system = TestSystem.new(1)
     strategy = cut.new(2)
-    strategy.call(system, 1).should == 2
+    expect(strategy.call(system, 1)).to eq 2
   end
 end

data/spec/autoscaler/counter_cache_memory_spec.rb

@@ -5,17 +5,17 @@ describe Autoscaler::CounterCacheMemory do
   let(:cut) {Autoscaler::CounterCacheMemory}
 
   it {expect{cut.new.counter}.to raise_error(cut::Expired)}
-  it {cut.new.counter{1}.should == 1}
+  it {expect(cut.new.counter{1}).to eq 1}
 
   it 'set and store' do
     cache = cut.new
     cache.counter = 1
-    cache.counter.should == 1
+    expect(cache.counter).to eq 1
   end
 
   it 'times out' do
     cache = cut.new(0)
     cache.counter = 1
-    expect{cache.counter.should}.to raise_error(cut::Expired)
+    expect{cache.counter}.to raise_error(cut::Expired)
   end
 end

data/spec/autoscaler/counter_cache_redis_spec.rb

@@ -11,11 +11,11 @@ describe Autoscaler::CounterCacheRedis do
   subject {cut.new(Sidekiq.method(:redis))}
 
   it {expect{subject.counter}.to raise_error(cut::Expired)}
-  it {subject.counter{1}.should == 1}
+  it {expect(subject.counter{1}).to eq 1}
 
   it 'set and store' do
     subject.counter = 2
-    subject.counter.should == 2
+    expect(subject.counter).to eq 2
   end
 
   it 'does not conflict with multiple worker types' do
@@ -23,7 +23,7 @@ describe Autoscaler::CounterCacheRedis do
     subject.counter = 1
     other_worker_cache.counter = 2
 
-    subject.counter.should == 1
+    expect(subject.counter).to eq 1
     other_worker_cache.counter = 2
   end
 
@@ -37,13 +37,13 @@ describe Autoscaler::CounterCacheRedis do
   it 'passed a connection pool' do
     cache = cut.new(@redis)
     cache.counter = 4
-    cache.counter.should == 4
+    expect(cache.counter).to eq 4
   end
 
   it 'passed a plain connection' do
-    connection = Redis.connect(:url => 'http://localhost:9736', :namespace => 'autoscaler')
+    connection = Redis.connect(:url => 'redis://localhost:9736', :namespace => 'autoscaler')
     cache = cut.new connection
     cache.counter = 5
-    cache.counter.should == 5
+    expect(cache.counter).to eq 5
   end
 end

data/spec/autoscaler/delayed_shutdown_spec.rb

@@ -7,17 +7,17 @@ describe Autoscaler::DelayedShutdown do
 
   it "returns normal values" do
     strategy = cut.new(lambda{|s,t| 2}, 0)
-    strategy.call(nil, 1).should == 2
+    expect(strategy.call(nil, 1)).to eq 2
   end
 
   it "delays zeros" do
     strategy = cut.new(lambda{|s,t| 0}, 60)
-    strategy.call(nil, 1).should == 1
+    expect(strategy.call(nil, 1)).to eq 1
   end
 
   it "eventually returns zero" do
     strategy = cut.new(lambda{|s,t| 0}, 60)
-    strategy.stub(:level_idle_time).and_return(61)
-    strategy.call(nil, 61).should == 0
+    allow(strategy).to receive(:level_idle_time).and_return(61)
+    expect(strategy.call(nil, 61)).to eq 0
   end
 end

data/spec/autoscaler/heroku_platform_scaler_spec.rb

@@ -0,0 +1,47 @@
+require 'spec_helper'
+require 'autoscaler/heroku_platform_scaler'
+
+describe Autoscaler::HerokuPlatformScaler, :platform_api => true do
+  let(:cut) {Autoscaler::HerokuPlatformScaler}
+  let(:client) {cut.new}
+  subject {client}
+
+  its(:workers) {should eq(0)}
+
+  describe 'scaled' do
+    around do |example|
+      client.workers = 1
+      example.call
+      client.workers = 0
+    end
+
+    its(:workers) {should eq(1)}
+  end
+
+  shared_examples 'exception handler' do |exception_class|
+    before do
+      expect(client).to receive(:client){
+        raise exception_class.new(Exception.new('oops'))
+      }
+    end
+
+    describe "default handler" do
+      it {expect{client.workers}.to_not raise_error}
+      it {expect(client.workers).to eq(0)}
+      it {expect{client.workers = 2}.to_not raise_error}
+    end
+
+    describe "custom handler" do
+      before do
+        @caught = false
+        client.exception_handler = lambda {|exception| @caught = true}
+      end
+
+      it {client.workers; expect(@caught).to be(true)}
+    end
+  end
+
+  describe 'exception handling', :focus => true do
+    it_behaves_like 'exception handler', Excon::Errors::Error
+  end
+end

data/spec/autoscaler/heroku_scaler_spec.rb

@@ -2,33 +2,33 @@ require 'spec_helper'
 require 'autoscaler/heroku_scaler'
 require 'heroku/api/errors'
 
-describe Autoscaler::HerokuScaler, :online => true do
+describe Autoscaler::HerokuScaler, :api1 => true do
   let(:cut) {Autoscaler::HerokuScaler}
   let(:client) {cut.new}
   subject {client}
 
-  its(:workers) {should == 0}
+  its(:workers) {should eq(0)}
 
   describe 'scaled' do
     around do |example|
       client.workers = 1
-      example.yield
+      example.call
       client.workers = 0
     end
 
-    its(:workers) {should == 1}
+    its(:workers) {should eq(1)}
   end
 
   shared_examples 'exception handler' do |exception_class|
     before do
-      client.should_receive(:client){
+      expect(client).to receive(:client){
         raise exception_class.new(Exception.new('oops'))
       }
     end
 
     describe "default handler" do
       it {expect{client.workers}.to_not raise_error}
-      it {client.workers.should == 0}
+      it {expect(client.workers).to eq(0)}
       it {expect{client.workers = 2}.to_not raise_error}
     end
 
@@ -38,7 +38,7 @@ describe Autoscaler::HerokuScaler, :online => true do
         client.exception_handler = lambda {|exception| @caught = true}
       end
 
-      it {client.workers; @caught.should be_true}
+      it {client.workers; expect(@caught).to be(true)}
     end
   end
 
@@ -46,4 +46,4 @@ describe Autoscaler::HerokuScaler, :online => true do
     it_behaves_like 'exception handler', Excon::Errors::SocketError
     it_behaves_like 'exception handler', Heroku::API::Errors::Error
   end
-end
+end

data/spec/autoscaler/ignore_scheduled_and_retrying_spec.rb

@@ -8,25 +8,25 @@ describe Autoscaler::IgnoreScheduledAndRetrying do
   it "passes through enqueued" do
     system = Struct.new(:enqueued).new(3)
     strategy = proc {|system, time| system.enqueued}
-    cut.new(strategy).call(system, 0).should == 3
+    expect(cut.new(strategy).call(system, 0)).to eq 3
   end
 
   it "passes through workers" do
     system = Struct.new(:workers).new(3)
     strategy = proc {|system, time| system.workers}
-    cut.new(strategy).call(system, 0).should == 3
+    expect(cut.new(strategy).call(system, 0)).to eq 3
   end
 
   it "ignores scheduled" do
     system = Struct.new(:scheduled).new(3)
     strategy = proc {|system, time| system.scheduled}
-    cut.new(strategy).call(system, 0).should == 0
+    expect(cut.new(strategy).call(system, 0)).to eq 0
   end
 
   it "ignores retrying" do
     system = Struct.new(:retrying).new(3)
     strategy = proc {|system, time| system.retrying}
-    cut.new(strategy).call(system, 0).should == 0
+    expect(cut.new(strategy).call(system, 0)).to eq 0
   end
 end
 

data/spec/autoscaler/linear_scaling_strategy_spec.rb

@@ -8,72 +8,78 @@ describe Autoscaler::LinearScalingStrategy do
   it "deactivates with no work" do
     system = TestSystem.new(0)
     strategy = cut.new(1)
-    strategy.call(system, 1).should == 0
+    expect(strategy.call(system, 1)).to eq 0
   end
 
   it "activates with some work" do
     system = TestSystem.new(1)
     strategy = cut.new(1)
-    strategy.call(system, 1).should be > 0
+    expect(strategy.call(system, 1)).to be > 0
   end
 
   it "minimally scales with minimal work" do
     system = TestSystem.new(1)
     strategy = cut.new(2, 2)
-    strategy.call(system, 1).should == 1
+    expect(strategy.call(system, 1)).to eq 1
   end
 
   it "maximally scales with too much work" do
     system = TestSystem.new(5)
     strategy = cut.new(2, 2)
-    strategy.call(system, 1).should == 2
+    expect(strategy.call(system, 1)).to eq 2
   end
 
   it "proportionally scales with some work" do
     system = TestSystem.new(5)
     strategy = cut.new(5, 2)
-    strategy.call(system, 1).should == 3
+    expect(strategy.call(system, 1)).to eq 3
   end
 
   it "doesn't scale unless minimum is met" do
     system = TestSystem.new(2)
     strategy = cut.new(10, 4, 0.5)
-    strategy.call(system, 1).should == 0
+    expect(strategy.call(system, 1)).to eq 0
   end
 
   it "scales proprotionally with a minimum" do
     system = TestSystem.new(3)
     strategy = cut.new(10, 4, 0.5)
-    strategy.call(system, 1).should == 1
+    expect(strategy.call(system, 1)).to eq 1
   end
 
   it "scales maximally with a minimum" do
     system = TestSystem.new(25)
     strategy = cut.new(5, 4, 0.5)
-    strategy.call(system, 1).should == 5
+    expect(strategy.call(system, 1)).to eq 5
   end
 
   it "scales proportionally with a minimum > 1" do
     system = TestSystem.new(12)
     strategy = cut.new(5, 4, 2)
-    strategy.call(system, 1).should == 2
+    expect(strategy.call(system, 1)).to eq 2
   end
 
   it "scales maximally with a minimum factor > 1" do
     system = TestSystem.new(30)
     strategy = cut.new(5, 4, 2)
-    strategy.call(system, 1).should == 5
+    expect(strategy.call(system, 1)).to eq 5
   end
 
-  xit "doesn't scale down engaged workers" do
+  it "doesn't scale down engaged workers" do
     system = TestSystem.new(0, 2)
     strategy = cut.new(5, 4)
-    strategy.call(system, 1).should == 2
+    expect(strategy.call(system, 1)).to eq 2
   end
 
-  xit "doesn't scale above max workers even if engaged workers is greater" do
+  it "doesn't scale above max workers even if engaged workers is greater" do
     system = TestSystem.new(40, 6)
     strategy = cut.new(5, 4)
-    strategy.call(system, 1).should == 5
+    expect(strategy.call(system, 1)).to eq 5
+  end
+
+  it "returns zero if requested capacity is zero" do
+    system = TestSystem.new(0, 0)
+    strategy = cut.new(0, 0)
+    expect(strategy.call(system, 5)).to eq 0
   end
 end

data/spec/autoscaler/sidekiq/activity_spec.rb

@@ -16,19 +16,19 @@ describe Autoscaler::Sidekiq::Activity do
       activity.idle!('queue')
       other_process.working!('other_queue')
     end
-    it {activity.should be_idle(['queue'])}
+    it {expect(activity).to be_idle(['queue'])}
   end
 
   it 'passed a connection pool' do
     activity = cut.new(5, @redis)
     activity.working!('queue')
-    activity.should_not be_idle(['queue'])
+    expect(activity).to_not be_idle(['queue'])
   end
 
   it 'passed a plain connection' do
-    connection = Redis.connect(:url => 'http://localhost:9736', :namespace => 'autoscaler')
+    connection = Redis.connect(:url => 'redis://localhost:9736', :namespace => 'autoscaler')
     activity = cut.new(5, connection)
     activity.working!('queue')
-    activity.should_not be_idle(['queue'])
+    expect(activity).to_not be_idle(['queue'])
   end
 end

data/spec/autoscaler/sidekiq/celluloid_monitor_spec.rb

@@ -16,7 +16,7 @@ describe Autoscaler::Sidekiq::CelluloidMonitor do
     system = TestSystem.new(0)
     manager = cut.new(scaler, lambda{|s,t| 0}, system)
     Timeout.timeout(1) { manager.wait_for_downscale(0.5) }
-    scaler.workers.should == 0
+    expect(scaler.workers).to eq 0
     manager.terminate
   end
 
@@ -24,7 +24,7 @@ describe Autoscaler::Sidekiq::CelluloidMonitor do
     system = TestSystem.new(1)
     manager = cut.new(scaler, lambda{|s,t| 1}, system)
     expect {Timeout.timeout(1) { manager.wait_for_downscale(0.5) }}.to raise_error Timeout::Error
-    scaler.workers.should == 1
+    expect(scaler.workers).to eq 1
     manager.terminate
   end
 
@@ -33,7 +33,7 @@ describe Autoscaler::Sidekiq::CelluloidMonitor do
     scaler = TestScaler.new(0)
     manager = cut.new(scaler, lambda{|s,t| 0}, system)
     Timeout.timeout(1) { manager.wait_for_downscale(0.5) }
-    scaler.workers.should == 0
+    expect(scaler.workers).to eq 0
     manager.terminate
   end
 end

data/spec/autoscaler/sidekiq/client_spec.rb

@@ -10,26 +10,26 @@ describe Autoscaler::Sidekiq::Client do
   describe 'call' do
     it 'scales' do
       client.call(Class, {}, 'queue') {}
-      scaler.workers.should == 1
+      expect(scaler.workers).to eq 1
     end
 
     it 'scales with a redis pool' do
       client.call(Class, {}, 'queue', ::Sidekiq.method(:redis)) {}
-      scaler.workers.should == 1
+      expect(scaler.workers).to eq 1
     end
 
-    it('yields') {client.call(Class, {}, 'queue') {:foo}.should == :foo}
+    it('yields') {expect(client.call(Class, {}, 'queue') {:foo}).to eq :foo}
   end
 
   describe 'initial workers' do
     it 'works with default arguments' do
       client.set_initial_workers
-      scaler.workers.should == 0
+      expect(scaler.workers).to eq 0
     end
 
     it 'scales when necessary' do
       client.set_initial_workers {|q| TestSystem.new(1)}
-      scaler.workers.should == 1
+      expect(scaler.workers).to eq 1
     end
   end
 end

data/spec/autoscaler/sidekiq/entire_queue_system_spec.rb

@@ -24,42 +24,42 @@ describe Autoscaler::Sidekiq::EntireQueueSystem do
 
   subject {cut.new}
 
-  it {subject.queue_names.should == []}
-  it {subject.workers.should == 0}
+  it {expect(subject.queue_names).to eq []}
+  it {expect(subject.workers).to eq 0}
 
   describe 'no queued work' do
     it "with no work" do
-      subject.stub(:sidekiq_queues).and_return({'queue' => 0, 'another_queue' => 0})
-      subject.queued.should == 0
+      allow(subject).to receive(:sidekiq_queues).and_return({'queue' => 0, 'another_queue' => 0})
+      expect(subject.queued).to eq 0
     end
 
     it "with no work and no queues" do
-      subject.queued.should == 0
+      expect(subject.queued).to eq 0
     end
 
     it "with no scheduled work" do
-      subject.scheduled.should == 0
+      expect(subject.scheduled).to eq 0
     end
 
     it "with no retry work" do
-      subject.retrying.should == 0
+      expect(subject.retrying).to eq 0
     end
   end
 
   describe 'with queued work' do
     it "with enqueued work" do
-      subject.stub(:sidekiq_queues).and_return({'queue' => 1})
-      subject.queued.should == 1
+      allow(subject).to receive(:sidekiq_queues).and_return({'queue' => 1})
+      expect(subject.queued).to eq 1
     end
 
     it "with schedule work" do
       with_scheduled_work_in('queue')
-      subject.scheduled.should == 1
+      expect(subject.scheduled).to eq 1
     end
 
     it "with retry work" do
       with_retry_work_in('queue')
-      subject.retrying.should == 1
+      expect(subject.retrying).to eq 1
     end
   end
 end

data/spec/autoscaler/sidekiq/monitor_middleware_adapter_spec.rb

@@ -11,6 +11,6 @@ describe Autoscaler::Sidekiq::MonitorMiddlewareAdapter do
   let(:scaler) {TestScaler.new(1)}
   let(:server) {cut.new(scaler, 0, ['queue'])}
 
-  it('yields') {server.call(Object.new, {}, 'queue') {:foo}.should == :foo}
-  it('yields with a redis pool') {server.call(Object.new, {}, 'queue', Sidekiq.method(:redis)) {:foo}.should == :foo}
+  it('yields') {expect(server.call(Object.new, {}, 'queue') {:foo}).to eq :foo}
+  it('yields with a redis pool') {expect(server.call(Object.new, {}, 'queue', Sidekiq.method(:redis)) {:foo}).to eq :foo}
 end

data/spec/autoscaler/sidekiq/sleep_wait_server_spec.rb

@@ -12,34 +12,34 @@ describe Autoscaler::Sidekiq::SleepWaitServer do
   let(:server) {cut.new(scaler, 0, ['queue'])}
 
   shared_examples "a sleepwait server" do
-  it "scales with no work" do
-    server.stub(:pending_work?).and_return(false)
-    when_run
-    scaler.workers.should == 0
-  end
+    it "scales with no work" do
+      allow(server).to receive(:pending_work?).and_return(false)
+      when_run
+      expect(scaler.workers).to eq 0
+    end
 
-  it "does not scale with pending work" do
-    server.stub(:pending_work?).and_return(true)
-    when_run
-    scaler.workers.should == 1
-  end
+    it "does not scale with pending work" do
+      allow(server).to receive(:pending_work?).and_return(true)
+      when_run
+      expect(scaler.workers).to eq 1
+    end
   end
 
   describe "a middleware with no redis specified" do
-  it_behaves_like "a sleepwait server" do
-  def when_run
-    server.call(Object.new, {}, 'queue') {}
-  end
-  end
+    it_behaves_like "a sleepwait server" do
+      def when_run
+        server.call(Object.new, {}, 'queue') {}
+      end
+    end
   end
 
   describe "a middleware with redis specified" do
-  it_behaves_like "a sleepwait server" do
-  def when_run
-    server.call(Object.new, {}, 'queue', Sidekiq.method(:redis)) {}
-  end
-  end
+    it_behaves_like "a sleepwait server" do
+      def when_run
+        server.call(Object.new, {}, 'queue', Sidekiq.method(:redis)) {}
+      end
+    end
   end
 
-  it('yields') {server.call(Object.new, {}, 'queue') {:foo}.should == :foo}
+  it('yields') {expect(server.call(Object.new, {}, 'queue') {:foo}).to eq :foo}
 end

data/spec/autoscaler/sidekiq/specified_queue_system_spec.rb

@@ -24,40 +24,40 @@ describe Autoscaler::Sidekiq::SpecifiedQueueSystem do
 
   subject {cut.new(['queue'])}
 
-  it {subject.queue_names.should == ['queue']}
-  it {subject.workers.should == 0}
+  it {expect(subject.queue_names).to eq ['queue']}
+  it {expect(subject.workers).to eq 0}
 
   describe 'no queued work' do
     it "with no work" do
-      subject.stub(:sidekiq_queues).and_return({'queue' => 0, 'another_queue' => 1})
-      subject.queued.should == 0
+      allow(subject).to receive(:sidekiq_queues).and_return({'queue' => 0, 'another_queue' => 1})
+      expect(subject.queued).to eq 0
     end
 
     it "with scheduled work in another queue" do
       with_scheduled_work_in('another_queue')
-      subject.scheduled.should == 0
+      expect(subject.scheduled).to eq 0
     end
 
     it "with retry work in another queue" do
       with_retry_work_in('another_queue')
-      subject.retrying.should == 0
+      expect(subject.retrying).to eq 0
     end
   end
 
   describe 'with queued work' do
     it "with enqueued work" do
-      subject.stub(:sidekiq_queues).and_return({'queue' => 1})
-      subject.queued.should == 1
+      allow(subject).to receive(:sidekiq_queues).and_return({'queue' => 1})
+      expect(subject.queued).to eq 1
     end
 
     it "with schedule work" do
       with_scheduled_work_in('queue')
-      subject.scheduled.should == 1
+      expect(subject.scheduled).to eq 1
     end
 
     it "with retry work" do
       with_retry_work_in('queue')
-      subject.retrying.should == 1
+      expect(subject.retrying).to eq 1
     end
   end
 end

data/spec/spec_helper.rb

@@ -1,10 +1,12 @@
+require 'rspec/its'
 require 'sidekiq'
-REDIS = Sidekiq::RedisConnection.create(:url => 'http://localhost:9736', :namespace => 'autoscaler')
+REDIS = Sidekiq::RedisConnection.create(:url => 'redis://localhost:9736', :namespace => 'autoscaler')
 
 RSpec.configure do |config|
   config.mock_with :rspec
 
-  config.filter_run_excluding :online => true unless ENV['HEROKU_APP']
+  config.filter_run_excluding :api1 => true unless ENV['HEROKU_API_KEY']
+  config.filter_run_excluding :platform_api => true unless ENV['HEROKU_ACCESS_TOKEN']
 end
 
 class TestScaler

data/spec/test_system.rb

@@ -8,4 +8,10 @@ class TestSystem
   def queued; @pending; end
   def scheduled; 0; end
   def retrying; 0; end
+  def total_work
+    queued + scheduled + retrying + workers
+  end
+  def any_work?
+    queued > 0 || scheduled > 0 || retrying > 0 || workers > 0
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: autoscaler
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.10.0
 platform: ruby
 authors:
 - Justin Love
@@ -9,30 +9,52 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-19 00:00:00.000000000 Z
+date: 2015-10-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sidekiq
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.7'
-    - - "<"
+        version: 3.5.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.5.1
+- !ruby/object:Gem::Dependency
+  name: celluloid
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.1'
+        version: 0.17.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.17.2
+- !ruby/object:Gem::Dependency
+  name: heroku-api
+  requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '2.7'
-    - - "<"
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '3.1'
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: heroku-api
+  name: platform-api
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -73,6 +95,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec-its
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: guard-rspec
   requirement: !ruby/object:Gem::Requirement
@@ -119,6 +155,7 @@ files:
 - lib/autoscaler/counter_cache_memory.rb
 - lib/autoscaler/counter_cache_redis.rb
 - lib/autoscaler/delayed_shutdown.rb
+- lib/autoscaler/heroku_platform_scaler.rb
 - lib/autoscaler/heroku_scaler.rb
 - lib/autoscaler/ignore_scheduled_and_retrying.rb
 - lib/autoscaler/linear_scaling_strategy.rb
@@ -137,6 +174,7 @@ files:
 - spec/autoscaler/counter_cache_memory_spec.rb
 - spec/autoscaler/counter_cache_redis_spec.rb
 - spec/autoscaler/delayed_shutdown_spec.rb
+- spec/autoscaler/heroku_platform_scaler_spec.rb
 - spec/autoscaler/heroku_scaler_spec.rb
 - spec/autoscaler/ignore_scheduled_and_retrying_spec.rb
 - spec/autoscaler/linear_scaling_strategy_spec.rb
@@ -149,7 +187,7 @@ files:
 - spec/autoscaler/sidekiq/specified_queue_system_spec.rb
 - spec/spec_helper.rb
 - spec/test_system.rb
-homepage: ''
+homepage: https://github.com/JustinLove/autoscaler
 licenses: []
 metadata: {}
 post_install_message: 
@@ -168,7 +206,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: autoscaler
-rubygems_version: 2.2.0
+rubygems_version: 2.4.8
 signing_key: 
 specification_version: 4
 summary: Start/stop Sidekiq workers on Heroku
@@ -178,6 +216,7 @@ test_files:
 - spec/autoscaler/counter_cache_memory_spec.rb
 - spec/autoscaler/counter_cache_redis_spec.rb
 - spec/autoscaler/delayed_shutdown_spec.rb
+- spec/autoscaler/heroku_platform_scaler_spec.rb
 - spec/autoscaler/heroku_scaler_spec.rb
 - spec/autoscaler/ignore_scheduled_and_retrying_spec.rb
 - spec/autoscaler/linear_scaling_strategy_spec.rb