sidekiq-throttled 0.17.0 → 1.0.0.alpha

data/CHANGES.md

@@ -1,300 +0,0 @@
-## 0.16.1 (2022-06-14)
-
-* [#2](https://github.com/ixti/sidekiq-throttled/pull/2)
-  Support custom ActiveJob adapters.
-  ([@longkt90])
-
-* [#107](https://github.com/sensortower/sidekiq-throttled/pull/107)
-  Log error when key sufix extraction fails.
-  ([@pjungwir])
-
-## 0.16.0 (2022-06-13)
-
-* Drop Ruby 2.6 support.
-
-* Drop Sidekiq 5.X support.
-
-* [#121](https://github.com/sensortower/sidekiq-throttled/pull/121)
-  Sidekiq 6.5 compatibility.
-  ([@hieuk09])
-
-* [#116](https://github.com/sensortower/sidekiq-throttled/pull/116)
-  Unwrap ActiveJob JobWrapper
-  ([@holstvoogd])
-
-* [#115](https://github.com/sensortower/sidekiq-throttled/pull/115)
-  Fix Redis 4.6 deprecation warnings.
-  ([@dbackeus])
-
-* [#113](https://github.com/sensortower/sidekiq-throttled/pull/113)
-  Add Ruby 3.1 to CI.
-  ([@petergoldstein])
-
-* [#103](https://github.com/sensortower/sidekiq-throttled/pull/103)
-  Rename Sidekiq::Throttled::Worker to Sidekiq::Throttled::Job, and alias it
-  as Sidekiq::Throttled::Worker.
-  ([@CHTJonas])
-
-## 0.15.0 (2021-12-16)
-
-* [#102](https://github.com/sensortower/sidekiq-throttled/pull/102)
-  Support Ruby 3.0 and Sidekiq 6.2+.
-  ([@ybiquitous])
-
-* [#97](https://github.com/sensortower/sidekiq-throttled/pull/97)
-  Fix kwargs usage in strategy collection.
-  ([@baptistejub])
-
-## 0.14.0 (2021-09-21)
-
-* [#98](https://github.com/sensortower/sidekiq-throttled/pull/98)
-  Remove warning for strategy override.
-  ([@mattiagiuffrida-st])
-
-* [#65](https://github.com/sensortower/sidekiq-throttled/pull/65)
-  Support composite (multi-key) strategies.
-  ([@holyketzer])
-
-* [#89](https://github.com/sensortower/sidekiq-throttled/pull/89),
-  [#96](https://github.com/sensortower/sidekiq-throttled/pull/96), and
-  [#93](https://github.com/sensortower/sidekiq-throttled/pull/93)
-  Improve documentation.
-  ([@hubertjakubiak], [@khaile], and [@kylerippey])
-
-## 0.13.0 (2020-07-28)
-
-* [#85](https://github.com/sensortower/sidekiq-throttled/pull/85)
-  Add Sidekiq 6.1+ support.
-  ([@hmaack])
-
-## 0.12.0 (2020-06-22)
-
-* [#80](https://github.com/sensortower/sidekiq-throttled/pull/80)
-  Allow override cooldown timeout of queues with throttled jobs.
-  ([@vaot])
-
-* [#76](https://github.com/sensortower/sidekiq-throttled/pull/76)
-  Fix warnings on Ruby 2.7
-  ([@lenon])
-
-
-## 0.11.0 (2019-08-24)
-
-* [#59](https://github.com/sensortower/sidekiq-throttled/pull/59)
-  Add throttling observer.
-  ([@ogins57])
-
-
-## 0.10.0 (2019-06-22)
-
-* [#60](https://github.com/sensortower/sidekiq-throttled/pull/60)
-  Skip throttling check in redis if limit is 0.
-  ([@mstruve])
-
-* [#58](https://github.com/sensortower/sidekiq-throttled/pull/58)
-  Improve documentation bout TTL.
-  ([@ziaulrehman40])
-
-* Improve reliability of paused queues, by resyncing list of paused queues
-  on schedule.
-  ([@ixti])
-
-
-## 0.9.0 (2018-09-11)
-
-* Add support of Sidekiq 5.2.x
-  ([@ixti])
-
-* Drop support of Sidekiq 4.x
-  ([@ixti])
-
-* Add concurrent-ruby as dependency which is no longer a dependency of
-  Sidekiq as 5.2+ version.
-  ([@ixti])
-
-
-## 0.8.2 (2018-02-14)
-
-* Extract redis LUA scripts stored procedures runner to redis-prescription gem.
-  ([@ixti])
-
-* Switch to Concurrent.monotonic_time to expire elements of ExpirableList.
-  ([@ixti])
-
-
-## 0.8.1 (2017-11-02)
-
-* Preload job class constant prior trying to get it's throttling strategy.
-  ([@ixti])
-
-
-## 0.8.0 (2017-10-11)
-
-* Refactor concurrency throttling internals to use sorted sets in order to avoid
-  starvation in case when finalize! was not called (OOM / redis issues).
-  ([@ixti])
-
-
-## 0.7.3 (2017-06-26)
-
-* [#34](https://github.com/sensortower/sidekiq-throttled/issues/34)
-  Fix reset button for sidekiq `>= 4.2`.
-  ([@ixti])
-
-
-## 0.7.2 (2017-04-02)
-
-* Fix summary bar fixer on sidekiq 4.2+.
-  ([@ixti])
-
-* Fix regexp used to fix summay bar queues link when ui was enhanced.
-  ([@ixti])
-
-
-## 0.7.1 (2017-03-30)
-
-* Fix summary bar queues link when queue ui was enhanced.
-  ([@ixti])
-
-* [#31](https://github.com/sensortower/sidekiq-throttled/pull/31)
-  [#30](https://github.com/sensortower/sidekiq-throttled/issues/30)
-  Do not throttle if limit is `nil`.
-  ([@ixti])
-
-
-## 0.7.0 (2017-03-22)
-
-* Expose pause/resume queues hidden feature to UI. This was available via API
-  since v0.6.0 and today it's finally got it's UI.
-  ([@ixti])
-
-
-## 0.6.7 (2017-03-21)
-
-* Fix fetcher causing workers starvation upon low concurrency thresholds.
-  ([@ixti])
-
-
-## 0.6.6 (2016-10-16)
-
-* [#24](https://github.com/sensortower/sidekiq-throttled/pull/24)
-  Fix dynamic `:key_suffix` issue.
-  ([@iporsut])
-
-
-## 0.6.5 (2016-09-04)
-
-* Fix concurrency throttling when redis-namespace is used.
-  ([@ixti])
-
-
-## 0.6.4 (2016-09-02)
-
-* Rename UnitOfWork throttled requeue to `#requeue_throttled`.
-  ([@ixti])
-
-
-## 0.6.3 (2016-09-02)
-
-* Enrich internal API to allow better extensibility.
-  ([@ixti])
-
-
-## 0.6.2 (2016-09-01)
-
-* Add `Fetch.bulk_requeue` used by Sidekiq upon termination.
-  ([@ixti])
-
-
-## 0.6.1 (2016-08-30)
-
-* Trivial internal API change: extracted queues list builder of `Fetch` into
-  dedicated internal method, allowing to enhance it with extra custom filters.
-  ([@ixti])
-
-
-## 0.6.0 (2016-08-27)
-
-* [#21](https://github.com/sensortower/sidekiq-throttled/pull/21)
-  Allow pause/unpause queues.
-  ([@ixti])
-
-
-## 0.5.0 (2016-08-18)
-
-* Drop Sidekiq 3.x support.
-  ([@ixti])
-
-
-## 0.4.1 (2016-08-18)
-
-* [#15](https://github.com/sensortower/sidekiq-throttled/pull/15)
-  Fix throttled web UI on older versions of sidekiq.
-  ([@palanglung])
-
-
-## 0.4.0 (2016-05-17)
-
-* [#14](https://github.com/sensortower/sidekiq-throttled/pull/14)
-  Support dynamic configuration of limits and periods.
-  ([@azach], [@ixti])
-
-
-## 0.3.2 (2016-05-16)
-
-* [#13](https://github.com/sensortower/sidekiq-throttled/issues/13)
-  Fix throttled BasicFetch with strictly ordered queues on sidekiq 4.
-  ([@palanglung], [@ixti])
-
-
-## 0.3.1 (2016-05-15)
-
-* Precalculate LUA script digests to reduce bandwidth upon nodes reload
-  _(which might (and might not) happen if you run thousands of nodes)_.
-  ([@ixti])
-
-
-## 0.3.0 (2016-05-02)
-
-* [#1](https://github.com/sensortower/sidekiq-throttled/issues/1):
-  Add Sidekiq 4.0 support.
-  ([@ixti])
-
-
-## 0.2.0 (2016-02-29)
-
-* [#6](https://github.com/sensortower/sidekiq-throttled/pull/6):
-  Add dynamic key suffix functionality.
-  ([@fhwang])
-
-
-## 0.1.0 (2015-11-03)
-
-* Initial release.
-
-
-[@ixti]: https://github.com/ixti
-[@fhwang]: https://github.com/fhwang
-[@palanglung]: https://github.com/palanglung
-[@azach]: https://github.com/azach
-[@iporsut]: https://github.com/iporsut
-[@mstruve]: https://github.com/mstruve
-[@ziaulrehman40]: https://github.com/ziaulrehman40
-[@ogins57]: https://github.com/ogins57
-[@lenon]: https://github.com/lenon
-[@vaot]: https://github.com/vaot
-[@hmaack]: https://github.com/hmaack
-[@holyketzer]: https://github.com/holyketzer
-[@hubertjakubiak]: https://github.com/hubertjakubiak
-[@kylerippey]: https://github.com/kylerippey
-[@khaile]: https://github.com/khaile
-[@mattiagiuffrida-st]: https://github.com/mattiagiuffrida-st
-[@baptistejub]: https://github.com/baptistejub
-[@ybiquitous]: https://github.com/ybiquitous
-[@hieuk09]: https://github.com/hieuk09
-[@petergoldstein]: https://github.com/petergoldstein
-[@dbackeus]: https://github.com/dbackeus
-[@holstvoogd]: https://github.com/holstvoogd
-[@CHTJonas]: https://github.com/CHTJonas
-[@pjungwir]: https://github.com/pjungwir
-[@longkt90]: https://github.com/longkt90

data/Gemfile

@@ -1,34 +0,0 @@
-# frozen_string_literal: true
-
-source "https://rubygems.org"
-
-gem "appraisal"
-gem "rake"
-gem "rspec"
-gem "sidekiq"
-
-group :development do
-  gem "byebug"
-  gem "guard",         :require => false
-  gem "guard-rspec",   :require => false
-  gem "guard-rubocop", :require => false
-end
-
-group :test do
-  gem "apparition"
-  gem "capybara"
-  gem "puma"
-  gem "rack-test"
-  gem "sinatra"
-  gem "timecop"
-end
-
-group :lint do
-  gem "rubocop",              :require => false
-  gem "rubocop-performance",  :require => false
-  gem "rubocop-rake",         :require => false
-  gem "rubocop-rspec",        :require => false
-end
-
-# Specify your gem's dependencies in sidekiq-throttled.gemspec
-gemspec

data/Guardfile

@@ -1,25 +0,0 @@
-# frozen_string_literal: true
-
-group :red_green_refactor, :halt_on_fail => true do
-  guard :rspec, :cmd => "bundle exec rspec --no-profile" do
-    require "guard/rspec/dsl"
-    dsl = Guard::RSpec::Dsl.new(self)
-
-    # Feel free to open issues for suggestions and improvements
-
-    # RSpec files
-    rspec = dsl.rspec
-    watch(rspec.spec_helper) { rspec.spec_dir }
-    watch(rspec.spec_support) { rspec.spec_dir }
-    watch(rspec.spec_files)
-
-    # Ruby files
-    ruby = dsl.ruby
-    dsl.watch_spec_files_for(ruby.lib_files)
-  end
-
-  guard :rubocop, :all_on_start => false do
-    watch(%r{.+\.rb$})
-    watch(%r{(?:.+/)?\.rubocop\.yml$}) { |m| File.dirname(m[0]) }
-  end
-end

data/Rakefile

@@ -1,27 +0,0 @@
-# frozen_string_literal: true
-
-require "bundler/gem_tasks"
-
-require "rspec/core/rake_task"
-RSpec::Core::RakeTask.new(:spec)
-
-desc "Run RuboCop"
-task :rubocop do
-  require "rubocop"
-  result = RuboCop::CLI.new.run([])
-  abort("RuboCop failed!") if result.nonzero?
-end
-
-namespace :rubocop do
-  desc "Auto-correct RuboCop offenses"
-  task :autocorrect do
-    require "rubocop"
-    result = RuboCop::CLI.new.run(["--auto-correct"])
-    abort("RuboCop failed!") if result.nonzero?
-  end
-end
-
-default_suite = ENV["CI"] ? :spec : %i[spec rubocop]
-named_suites  = { "rubocop" => :rubocop, "rspec" => :spec }
-
-task :default => named_suites.fetch(ENV.fetch("SUITE", nil), default_suite)

data/gemfiles/sidekiq_6.4.gemfile

@@ -1,33 +0,0 @@
-# This file was generated by Appraisal
-
-source "https://rubygems.org"
-
-gem "appraisal"
-gem "rake"
-gem "rspec"
-gem "sidekiq", "~> 6.4.0"
-
-group :development do
-  gem "byebug"
-  gem "guard", require: false
-  gem "guard-rspec", require: false
-  gem "guard-rubocop", require: false
-end
-
-group :test do
-  gem "apparition"
-  gem "capybara"
-  gem "puma"
-  gem "rack-test"
-  gem "sinatra"
-  gem "timecop"
-end
-
-group :lint do
-  gem "rubocop", require: false
-  gem "rubocop-performance", require: false
-  gem "rubocop-rake", require: false
-  gem "rubocop-rspec", require: false
-end
-
-gemspec path: "../"

data/gemfiles/sidekiq_6.5.gemfile

@@ -1,33 +0,0 @@
-# This file was generated by Appraisal
-
-source "https://rubygems.org"
-
-gem "appraisal"
-gem "rake"
-gem "rspec"
-gem "sidekiq", "~> 6.5.0"
-
-group :development do
-  gem "byebug"
-  gem "guard", require: false
-  gem "guard-rspec", require: false
-  gem "guard-rubocop", require: false
-end
-
-group :test do
-  gem "apparition"
-  gem "capybara"
-  gem "puma"
-  gem "rack-test"
-  gem "sinatra"
-  gem "timecop"
-end
-
-group :lint do
-  gem "rubocop", require: false
-  gem "rubocop-performance", require: false
-  gem "rubocop-rake", require: false
-  gem "rubocop-rspec", require: false
-end
-
-gemspec path: "../"

data/lib/sidekiq/throttled/communicator/callbacks.rb

@@ -1,72 +0,0 @@
-# frozen_string_literal: true
-
-require "fiber"
-
-require "sidekiq/throttled/communicator/exception_handler"
-
-module Sidekiq
-  module Throttled
-    class Communicator
-      # Callbacks registry and runner. Runs registered callbacks in dedicated
-      # Fiber solving issue with ConnectionPool and Redis client in subscriber
-      # mode.
-      #
-      # Once Redis entered subscriber mode `#subscribe` method, it can't be used
-      # for any command but pub/sub or quit, making it impossible to use for
-      # anything else. ConnectionPool binds reserved client to Thread, thus
-      # nested `#with` calls inside same thread result into a same connection.
-      # That makes it impossible to issue any normal Redis commands from
-      # within listener Thread.
-      #
-      # @private
-      class Callbacks
-        include ExceptionHandler
-
-        # Initializes callbacks registry.
-        def initialize
-          @mutex    = Mutex.new
-          @handlers = Hash.new { |h, k| h[k] = [] }
-        end
-
-        # Registers handler of given event.
-        #
-        # @example
-        #
-        #   callbacks.on "and out comes wolves" do |who|
-        #     puts "#{who} let the dogs out?!"
-        #   end
-        #
-        # @param [#to_s] event
-        # @raise [ArgumentError] if no handler block given
-        # @yield [*args] Runs given block upon `event`
-        # @yieldreturn [void]
-        # @return [self]
-        def on(event, &handler)
-          raise ArgumentError, "No block given" unless handler
-
-          @mutex.synchronize { @handlers[event.to_s] << handler }
-          self
-        end
-
-        # Runs event handlers with given args.
-        #
-        # @param [#to_s] event
-        # @param [Object] payload
-        # @return [void]
-        def run(event, payload = nil)
-          @mutex.synchronize do
-            fiber = Fiber.new do
-              @handlers[event.to_s].each do |callback|
-                callback.call(payload)
-              rescue => e
-                handle_exception(e, :context => "sidekiq:throttled")
-              end
-            end
-
-            fiber.resume
-          end
-        end
-      end
-    end
-  end
-end

data/lib/sidekiq/throttled/communicator/exception_handler.rb

@@ -1,25 +0,0 @@
-# frozen_string_literal: true
-
-require "sidekiq"
-require "sidekiq/version"
-
-module Sidekiq
-  module Throttled
-    class Communicator
-      if Sidekiq::VERSION >= "6.5.0"
-        module ExceptionHandler
-          def handle_exception(*args)
-            Sidekiq.handle_exception(*args)
-          end
-        end
-
-        # NOTE: `Sidekiq.default_error_handler` is private API
-        Sidekiq.error_handlers << Sidekiq.method(:default_error_handler)
-      else
-        require "sidekiq/exception_handler"
-
-        ExceptionHandler = ::Sidekiq::ExceptionHandler
-      end
-    end
-  end
-end

data/lib/sidekiq/throttled/communicator/listener.rb

@@ -1,109 +0,0 @@
-# frozen_string_literal: true
-
-require "sidekiq/throttled/communicator/exception_handler"
-
-module Sidekiq
-  module Throttled
-    class Communicator
-      # Redis subscription listener thread.
-      #
-      # @private
-      class Listener < Thread
-        include ExceptionHandler
-
-        # Starts listener thread.
-        #
-        # @param [String] channel Redis pub/sub channel to listen
-        # @param [Callbacks] callbacks Message callbacks registry
-        def initialize(channel, callbacks)
-          @channel    = channel
-          @callbacks  = callbacks
-          @terminated = false
-          @subscribed = false
-
-          super { listen until @terminated }
-        end
-
-        # Whenever underlying redis client subscribed to pub/sup channel.
-        #
-        # @return [Boolean]
-        def ready?
-          @subscribed
-        end
-
-        # Whenever main loop is still running.
-        #
-        # @return [Boolean]
-        def listening?
-          !@terminated
-        end
-
-        # Stops listener.
-        #
-        # @return [void]
-        def stop
-          # Raising exception while client is in subscription mode makes
-          # redis close connection and thus causing ConnectionPool reopen
-          # it (normal mode). Otherwise subscription mode client will be
-          # pushed back to ConnectionPool causing problems.
-          raise Sidekiq::Shutdown
-        end
-
-        private
-
-        # Wraps {#subscribe} with exception handlers:
-        #
-        # - `Sidekiq::Shutdown` exception marks listener as stopped and returns
-        #   making `while` loop of listener thread terminate.
-        #
-        # - `StandardError` got recorded to the log and swallowed,
-        #   making `while` loop of the listener thread restart.
-        #
-        # - `Exception` is recorded to the log and re-raised.
-        #
-        # @return [void]
-        def listen # rubocop:disable Metrics/MethodLength
-          subscribe
-        rescue Sidekiq::Shutdown
-          @terminated = true
-          @subscribed = false
-        rescue StandardError => e # rubocop:disable Style/RescueStandardError
-          @subscribed = false
-          handle_exception(e, { :context => "sidekiq:throttled" })
-          sleep 1
-        rescue Exception => e # rubocop:disable Lint/RescueException
-          @terminated = true
-          @subscribed = false
-          handle_exception(e, { :context => "sidekiq:throttled" })
-          raise
-        end
-
-        # Subscribes to channel and triggers all registerd handlers for
-        # received messages.
-        #
-        # @note Puts thread's Redis connection to subscription mode and
-        #   locks thread.
-        #
-        # @see http://redis.io/topics/pubsub
-        # @see http://redis.io/commands/subscribe
-        # @see Callbacks#run
-        # @return [void]
-        def subscribe # rubocop:disable Metrics/MethodLength
-          Sidekiq.redis do |conn|
-            conn.subscribe @channel do |on|
-              on.subscribe do
-                @subscribed = true
-                @callbacks.run("ready")
-              end
-
-              on.message do |_channel, data|
-                message, payload = Marshal.load(data) # rubocop:disable Security/MarshalLoad:
-                @callbacks.run("message:#{message}", payload)
-              end
-            end
-          end
-        end
-      end
-    end
-  end
-end