autoscale 0.9.3 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: deaa1c361a30d17f661ab6804b158a0adfa5013c
-  data.tar.gz: 80b1396a7e61ad98f015aed0c7ee4d5bf427c13f
+  metadata.gz: d220489df83d3b499d5ba3e90f02359f106ed92e
+  data.tar.gz: f8c2289b5dab2e7b3f29c23b1d8f89657d372ad6
 SHA512:
-  metadata.gz: a9c023c43d9e0e47e127391a2d5ab05c6cbe93d1c2ed8ca1d79654342a061a6c4884d391c4a5fbd965e1339787c7882386556d45fd0114b2351c669c10e93cc4
-  data.tar.gz: a60201cf2f905f24bec6dfedcfbc8fd9c29416b9d16c07ddf53f8fe00db76d62609b041812b44275d0323b8f4234e0576c2eee4108617b3b95aaf7864f1d955f
+  metadata.gz: 56359da2856356597efaeaf4be2b4c4efa222132f1d4b78a5a8ef0eec1cb3185afee195f419305c5a6d0549ba3f96018ef106cdb8a27c0843ef31c8f55ed1e67
+  data.tar.gz: 8d69c218da237c24f6ec31643bc557212725262afcb991eb8df353eec6717e02b68584a08b2a82b085511e9080494f5d99fc80413645775cc43ced40632dd129

data/CHANGELOG.md

@@ -1,9 +1,18 @@
 # Changelog
 
+## 0.11.0
+
+- Replace celluloid monitor with thread based middleware for Sidekiq 4.0
+
 ## 0.10.0
-- Require Sidekiq 3.
-- Linear Scaling Strategy will not scale down past number of active workers. Assumes 1-1 SK process/dyno mapping.
+
+- Require Sidekiq 3.5
+- You may use `HerokuPlatformScaler` and `HEROKU_ACCESS_TOKEN` in place of `HerkouScaler` and `HEROKU_API_KEY`
 - QueueSystem#workers returns the number of engaged SK processes.
+- Linear Scaling Strategy will not scale down past number of active workers. Assumes 1-1 SK process/dyno mapping.
+- Calls the SideKiq quiet api when shutting down
+- Count workers currently running (Joel Van Horn)
+- Update gems and use RSpec expect syntax (giviger)
 
 ## 0.9.0
 

data/README.md

@@ -4,7 +4,7 @@
 
 ## Requirements
 
-Tested on Ruby 1.9.2 and Heroku Cedar stack.
+Tested on Ruby 2.1.7 and Heroku Cedar stack.
 
 ## Installation
 
@@ -12,36 +12,38 @@ Tested on Ruby 1.9.2 and Heroku Cedar stack.
 
 ## Getting Started
 
-This gem uses the [Heroku-Api](https://github.com/heroku/heroku.rb) gem, which requires an API key from Heroku.  It will also need the heroku app name.  By default, these are specified through environment variables.  You can also pass them to HerokuScaler explicitly.
+This gem uses the [Heroku Platform-Api](https://github.com/heroku/platform-api.rb) gem, which requires an OAuth token from Heroku.  It will also need the heroku app name.  By default, these are specified through environment variables.  You can also pass them to `HerokuPlatformScaler` explicitly.
 
-    HEROKU_API_KEY=.....
+    HEROKU_ACCESS_TOKEN=.....
     HEROKU_APP=....
 
+Support is still present for [Heroku-Api](https://github.com/heroku/heroku.rb) via `HerkouScaler` and `HEROKU_API_KEY`, but may be removed in a future major version.
+
 Install the middleware in your `Sidekiq.configure_` blocks
 
     require 'autoscaler/sidekiq'
-    require 'autoscaler/heroku_scaler'
+    require 'autoscaler/heroku_platform_scaler'
 
     Sidekiq.configure_client do |config|
       config.client_middleware do |chain|
-        chain.add Autoscaler::Sidekiq::Client, 'default' => Autoscaler::HerokuScaler.new
+        chain.add Autoscaler::Sidekiq::Client, 'default' => Autoscaler::HerokuPlatformScaler.new
       end
     end
 
     Sidekiq.configure_server do |config|
       config.server_middleware do |chain|
-        chain.add(Autoscaler::Sidekiq::Server, Autoscaler::HerokuScaler.new, 60) # 60 second timeout
+        chain.add(Autoscaler::Sidekiq::Server, Autoscaler::HerokuPlatformScaler.new, 60) # 60 second timeout
       end
     end
 
 ## Limits and Challenges
 
-- HerokuScaler includes an attempt at current-worker cache that may be overcomplication, and doesn't work very well on the server
+- HerokuPlatformScaler includes an attempt at current-worker cache that may be overcomplication, and doesn't work very well on the server
 - Multiple scale-down loops may be started, particularly if there are multiple jobs queued when the servers comes up.  Heroku seems to handle multiple scale-down commands well.
 - The scale-down monitor is triggered on job completion (and server middleware is only run around jobs), so if the server nevers processes any jobs, it won't turn off.
 - The retry and schedule lists are considered - if you schedule a long-running task, the process will not scale-down.
 - If background jobs trigger jobs in other scaled processes, please note you'll need `config.client_middleware` in your `Sidekiq.configure_server` block in order to scale-up.
-- Exceptions while calling the Heroku API are caught and printed by default.  See `HerokuScaler#exception_handler` to override
+- Exceptions while calling the Heroku API are caught and printed by default.  See `HerokuPlatformScaler#exception_handler` to override
 
 ## Experimental
 
@@ -57,24 +59,28 @@ You can pass a scaling strategy object instead of the timeout to the server midd
 
 ### Working caching
 
-    scaler.counter_cache = Autoscaler::CounterCacheRedis(Sidekiq.method(:redis))
+    scaler.counter_cache = Autoscaler::CounterCacheRedis.new(Sidekiq.method(:redis))
 
 ## Tests
 
 The project is setup to run RSpec with Guard.  It expects a redis instance on a custom port, which is started by the Guardfile.
 
-The HerokuScaler is not tested by default because it makes live API requests.  Specify `HEROKU_APP` and `HEROKU_API_KEY` on the command line, and then watch your app's logs.
+The HerokuPlatformScaler is not tested by default because it makes live API requests.  Specify `HEROKU_APP` and `HEROKU_ACCESS_TOKEN` on the command line, and then watch your app's logs.
 
-    HEROKU_APP=... HEROKU_API_KEY=... guard
+    HEROKU_APP=... HEROKU_ACCESS_TOKEN=... guard
     heroku logs --app ...
 
 ## Authors
 
 Justin Love, [@wondible](http://twitter.com/wondible), [https://github.com/JustinLove](https://github.com/JustinLove)
 
-Ported to Heroku-Api by Fix Peña, [https://github.com/fixr](https://github.com/fixr)
+### Contributors
 
-Retry/schedule sets by Matt Anderson [https://github.com/tonkapark](https://github.com/tonkapark) and Thibaud Guillaume-Gentil [https://github.com/jilion](https://github.com/jilion)
+- Benjamin Kudria [https://github.com/bkudria](https://github.com/bkudria)
+- Fix Peña [https://github.com/fixr](https://github.com/fixr)
+- Gabriel Givigier Guimarães [https://github.com/givigier](https://github.com/givigier)
+- Matt Anderson [https://github.com/tonkapark](https://github.com/tonkapark)
+- Thibaud Guillaume-Gentil [https://github.com/jilion](https://github.com/jilion)
 
 ## Licence
 

data/examples/complex.rb

@@ -1,15 +1,15 @@
 require 'sidekiq'
 require 'autoscaler/sidekiq'
-require 'autoscaler/heroku_scaler'
+require 'autoscaler/heroku_platform_scaler'
 
 heroku = nil
 if ENV['HEROKU_APP']
   heroku = {}
   scaleable = %w[default import] - (ENV['ALWAYS'] || '').split(' ')
   scaleable.each do |queue|
-    heroku[queue] = Autoscaler::HerokuScaler.new(
+    heroku[queue] = Autoscaler::HerokuPlatformScaler.new(
       queue,
-      ENV['HEROKU_API_KEY'],
+      ENV['HEROKU_ACCESS_TOKEN'],
       ENV['HEROKU_APP'])
   end
 end

data/examples/simple.rb

@@ -1,10 +1,10 @@
 require 'sidekiq'
 require 'autoscaler/sidekiq'
-require 'autoscaler/heroku_scaler'
+require 'autoscaler/heroku_platform_scaler'
 
 heroku = nil
 if ENV['HEROKU_APP']
-  heroku = Autoscaler::HerokuScaler.new
+  heroku = Autoscaler::HerokuPlatformScaler.new
   #heroku.exception_handler = lambda {|exception| MyApp.logger.error(exception)}
 end
 

data/lib/autoscaler/binary_scaling_strategy.rb

@@ -20,7 +20,7 @@ module Autoscaler
 
     private
     def active?(system)
-      system.queued > 0 || system.scheduled > 0 || system.retrying > 0 || system.workers > 0
+      system.any_work?
     end
   end
 end

data/lib/autoscaler/heroku_platform_scaler.rb

@@ -0,0 +1,84 @@
+require 'platform-api'
+require 'autoscaler/counter_cache_memory'
+
+module Autoscaler
+  # Wraps the Heroku Platform API to provide just the interface that we need for scaling.
+  class HerokuPlatformScaler
+    # @param [String] type process type this scaler controls
+    # @param [String] token Heroku OAuth access token
+    # @param [String] app Heroku app name
+    def initialize(
+        type = 'worker',
+        token = ENV['HEROKU_ACCESS_TOKEN'],
+        app = ENV['HEROKU_APP'])
+      @client = PlatformAPI.connect_oauth(token)
+      @type = type
+      @app = app
+      @workers = CounterCacheMemory.new
+    end
+
+    attr_reader :app
+    attr_reader :type
+
+    # Read the current worker count (value may be cached)
+    # @return [Numeric] number of workers
+    def workers
+      @workers.counter {@workers.counter = heroku_get_workers}
+    end
+
+    # Set the number of workers (noop if workers the same)
+    # @param [Numeric] n number of workers
+    def workers=(n)
+      unknown = false
+      current = @workers.counter{unknown = true; 1}
+      if n != current || unknown
+        p "Scaling #{type} to #{n}"
+        heroku_set_workers(n)
+        @workers.counter = n
+      end
+    end
+
+    # Callable object which responds to exceptions during api calls #
+    # @example
+    #   heroku.exception_handler = lambda {|exception| MyApp.logger.error(exception)}
+    #   heroku.exception_handler = lambda {|exception| raise}
+    #   # default
+    #   lambda {|exception|
+    #     p exception
+    #     puts exception.backtrace
+    #   }
+    attr_writer :exception_handler
+
+    # Object which supports #counter and #counter=
+    # Defaults to CounterCacheMemory
+    def counter_cache=(cache)
+      @workers = cache
+    end
+
+    private
+    attr_reader :client
+
+    def heroku_get_workers
+      client.formation.list(app)
+        .select {|item| item['type'] == type}
+        .map {|item| item['quantity']}
+        .reduce(0, &:+)
+    rescue Excon::Errors::Error => e
+      exception_handler.call(e)
+      0
+    end
+
+    def heroku_set_workers(n)
+      client.formation.update(app, type, {:quantity => n})
+    rescue Excon::Errors::Error, Heroku::API::Errors::Error => e
+      exception_handler.call(e)
+    end
+
+    def exception_handler
+      @exception_handler ||= lambda {|exception|
+        p exception
+        puts exception.backtrace
+      }
+    end
+  end
+end

data/lib/autoscaler/ignore_scheduled_and_retrying.rb

@@ -1,9 +1,14 @@
 module Autoscaler
+  # - Strategy wrapper to ignore scheduled and retrying queues. Usage:
+  #  ``new_strategy = IgnoreScheduledAndRetrying.new(my_old_strategy)``
   class IgnoreScheduledAndRetrying
     def initialize(strategy)
       @strategy = strategy
     end
 
+    # @param [QueueSystem] system interface to the queuing system
+    # @param [Numeric] event_idle_time number of seconds since a job related event
+    # @return [Integer] target number of workers
     def call(system, event_idle_time)
       system.define_singleton_method(:scheduled) { 0 }
       system.define_singleton_method(:retrying)  { 0 }

data/lib/autoscaler/linear_scaling_strategy.rb

@@ -33,7 +33,7 @@ module Autoscaler
 
     private
     def total_work(system)
-      system.queued + system.scheduled + system.retrying
+      system.total_work
     end
   end
 end

data/lib/autoscaler/sidekiq.rb

@@ -1,11 +1,11 @@
 require 'autoscaler/sidekiq/client'
-require 'autoscaler/sidekiq/monitor_middleware_adapter'
+require 'autoscaler/sidekiq/thread_server'
 
 module Autoscaler
   # namespace module for Sidekiq middlewares
   module Sidekiq
     # Sidekiq server middleware
     # Performs scale-down when the queue is empty
-    Server = MonitorMiddlewareAdapter
+    Server = ThreadServer
   end
 end

data/lib/autoscaler/sidekiq/client.rb

@@ -6,7 +6,7 @@ module Autoscaler
     # Sidekiq client middleware
     # Performs scale-up when items are queued and there are no workers running
     class Client
-      # @param [Hash] scalers map of queue(String) => scaler (e.g. {HerokuScaler}).
+      # @param [Hash] scalers map of queue(String) => scaler (e.g. {HerokuPlatformScaler}).
       #   Which scaler to use for each sidekiq queue
       def initialize(scalers)
         @scalers = scalers

data/lib/autoscaler/sidekiq/entire_queue_system.rb

@@ -26,6 +26,16 @@ module Autoscaler
         ::Sidekiq::RetrySet.new.size
       end
 
+      # @return [Boolean] if any kind of work still needs to be done
+      def any_work?
+        queued > 0 || scheduled > 0 || retrying > 0 || workers > 0
+      end
+
+      # @return [Integer] total amount of work
+      def total_work
+        queued + scheduled + retrying + workers
+      end
+
       # @return [Array[String]]
       def queue_names
         sidekiq_queues.keys

data/lib/autoscaler/sidekiq/sleep_wait_server.rb

@@ -6,7 +6,7 @@ module Autoscaler
     # Sidekiq server middleware
     # Performs scale-down when the queue is empty
     class SleepWaitServer
-      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuScaler})
+      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuPlatformScaler})
       # @param [Numeric] timeout number of seconds to wait before shutdown
       # @param [Array[String]] specified_queues list of queues to monitor to determine if there is work left.  Defaults to all sidekiq queues.
       def initialize(scaler, timeout, specified_queues = nil)
@@ -36,7 +36,7 @@ module Autoscaler
       attr_reader :system
 
       def pending_work?
-        system.queued > 0 || system.scheduled > 0 || system.retrying > 0
+        system.any_work?
       end
 
       def working!(queue, redis)

data/lib/autoscaler/sidekiq/specified_queue_system.rb

@@ -32,6 +32,16 @@ module Autoscaler
         count_set(::Sidekiq::RetrySet.new)
       end
 
+      # @return [Boolean] if any kind of work still needs to be done
+      def any_work?
+        queued > 0 || scheduled > 0 || retrying > 0 || workers > 0
+      end
+
+      # @return [Integer] total amount of work
+      def total_work
+        queued + scheduled + retrying + workers
+      end
+
       # @return [Array[String]]
       attr_reader :queue_names
 

data/lib/autoscaler/sidekiq/thread_server.rb

@@ -0,0 +1,90 @@
+require 'autoscaler/sidekiq/queue_system'
+require 'autoscaler/binary_scaling_strategy'
+require 'autoscaler/delayed_shutdown'
+require 'thread'
+
+module Autoscaler
+  module Sidekiq
+    # Sidekiq server middleware
+    # spawns a thread to monitor the sidekiq server for scale-down
+    class ThreadServer
+      # @param [scaler] scaler object that actually performs scaling operations (e.g. {HerokuPlatformScaler})
+      # @param [Strategy,Numeric] timeout strategy object that determines target workers, or a timeout  in seconds to be passed to {DelayedShutdown}+{BinaryScalingStrategy}
+      # @param [Array[String]] specified_queues list of queues to monitor to determine if there is work left.  Defaults to all sidekiq queues.
+      def initialize(scaler, timeout, specified_queues = nil)
+        @scaler = scaler
+        @strategy = strategy(timeout)
+        @system = QueueSystem.new(specified_queues)
+        @mutex = Mutex.new
+        @done = false
+      end
+
+      # Sidekiq middleware api entry point
+      def call(worker, msg, queue, _ = nil)
+        yield
+      ensure
+        active_now!
+        wait_for_downscale
+      end
+
+      # Start the monitoring thread if it's not running
+      def wait_for_downscale
+        @thread ||= Thread.new do
+          begin
+            run
+          rescue
+            @thread = nil
+          end
+        end
+      end
+
+      # Thread core loop
+      # Periodically update the desired number of workers
+      # @param [Numeric] interval polling interval, mostly for testing
+      def run(interval = 15)
+        active_now!
+
+        workers = :unknown
+
+        begin
+          sleep(interval)
+          target_workers = @strategy.call(@system, idle_time)
+          workers = @scaler.workers = target_workers unless workers == target_workers
+        end while !@done && workers > 0
+        ::Sidekiq::ProcessSet.new.each(&:quiet!)
+      end
+
+      # Shut down the thread, pause until complete
+      def terminate
+        @done = true
+        if @thread
+          t = @thread
+          @thread = nil
+          t.value
+        end
+      end
+
+      private
+
+      def active_now!
+        @mutex.synchronize do
+          @activity = Time.now
+        end
+      end
+
+      def idle_time
+        @mutex.synchronize do
+          Time.now - @activity
+        end
+      end
+
+      def strategy(timeout)
+        if timeout.respond_to?(:call)
+          timeout
+        else
+          DelayedShutdown.new(BinaryScalingStrategy.new, timeout)
+        end
+      end
+    end
+  end
+end

data/lib/autoscaler/version.rb

@@ -1,4 +1,4 @@
 module Autoscaler
   # version number
-  VERSION = "0.9.3"
+  VERSION = "0.11.0"
 end