sidekiq-fair_tenant 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ff58bfdb071ee2b14924f2ff844ccd2ce488fe54e91ee19e7b96834a4b01e6c1
+  data.tar.gz: aba3f768b2b5c612936c336099ded364220943eda5b30ea02e4be2f5817b615d
+SHA512:
+  metadata.gz: fe50c83e1ba529bc0bfab3e036cc57dbe96d05285b2d939aa16c2a710a36925b383c9402d075811c365b26cfc4393933d64026e13c2413ecdc2d79dfbc9de1e1
+  data.tar.gz: 3ec788281b6dd892186cdc32bc97b09814964924a1a8feb130e92eb7f783f77bba1d0bc642d619175608accd66f9f3276ae6b43bbc04ef7e9f0f29f89011c5b3

data/CHANGELOG.md

@@ -0,0 +1,12 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2024-01-12
+
+- Initial release: multiple rules, configurable throttling window.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2024 Andrey Novikov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,160 @@
+# Sidekiq::FairTenant
+
+Throttle “greedy” clients’ jobs to ensure more or less fair distribution of resources between clients.
+
+This tiny [Sidekiq] middleware will re-route client's jobs after certain threshold to throttled queues (defined by you), where they will be processed with reduced priority.
+
+“Weighted queues” feature of Sidekiq allows to de-prioritize jobs in throttled queues, so they will not block jobs from other clients, at the same time preserving overall throughput.
+
+<a href="https://evilmartians.com/?utm_source=sidekiq-fair_tenant">
+  <picture>
+    <source
+      media="(prefers-color-scheme: dark)"
+      srcset="https://evilmartians.com/badges/sponsored-by-evil-martians_v2.0_for-dark-bg@2x.png"
+    >
+    <img
+      src="https://evilmartians.com/badges/sponsored-by-evil-martians_v2.0@2x.png"
+      alt="Sponsored by Evil Martians"
+      width="236"
+      height="54"
+    >
+  </picture>
+</a>
+
+## Installation
+
+ 1. Install the gem and add to the application's Gemfile by executing:
+
+    ```sh
+    bundle add sidekiq-fair_tenant
+    ```
+
+ 2. Add `fair_tenant_queues` section to `sidekiq_options` in your job class:
+
+    ```diff
+     class SomeJob
+       sidekiq_options \
+         queue: 'default',
+    +    fair_tenant_queues: [
+    +     { queue: 'throttled_2x', threshold: 100, per: 1.hour },
+    +     { queue: 'throttled_4x', threshold:  10, per: 1.minute },
+    +    ]
+     end
+    ```
+
+ 3. Add tenant detection login into your job class:
+
+    ```diff
+     class SomeJob
+    +  def self.fair_tenant(*_perform_arguments)
+    +    # Return any string that will be used as tenant name
+    +    "tenant_1"
+    +  end
+     end
+    ```
+
+ 4. Add throttled queues with reduced weights to your Sidekiq configuration:
+
+    ```diff
+     # config/sidekiq.yml
+     :queues:
+       - [default, 4]
+    +  - [throttled_2x, 2]
+    +  - [throttled_4x, 1]
+    ```
+
+    See [Sidekiq Advanced options for Queues](https://github.com/sidekiq/sidekiq/wiki/Advanced-Options#queues) to learn more about queue weights.
+
+## Usage
+
+### Specifying throttling rules
+
+In your job class, add `fair_tenant_queues` section to `sidekiq_options` as array of hashes with following keys:
+
+ - `queue` - throttled queue name to re-route jobs into.
+ - `threshold` - maximum number of jobs allowed to be enqueued within `per` seconds.
+ - `per` - sliding time window in seconds to count jobs (you can use ActiveSupport Durations in Rails).
+
+You can specify multiple rules and they all will be checked. _Last_ matching rule will be used, so order rules from least to most restrictive.
+
+Example:
+
+```ruby
+sidekiq_options \
+  queue: 'default',
+  fair_tenant_queues: [
+    # First rule is less restrictive, reacting to a large number of jobs enqueued in a long time window
+    { queue: 'throttled_2x', threshold: 1_000, per: 1.day },
+    # Next rule is more restrictive, reacting to spikes of jobs in a short time window
+    { queue: 'throttled_4x', threshold:    10, per: 1.minute },
+  ]
+```
+
+### Specifying tenant
+
+ 1. Explicitly during job enqueuing:
+
+    ```ruby
+    SomeJob.set(fair_tenant: 'tenant_1').perform_async
+    ```
+
+ 2. Dynamically using `fair_tenant` class-level method in your job class (receives same arguments as `perform`)
+
+    ```ruby
+    class SomeJob
+      def self.fair_tenant(*_perform_arguments)
+        # Return any string that will be used as tenant name
+        "tenant_1"
+      end
+    end
+    ```
+
+ 3. Set `fair_tenant` job option in a custom [middleware](https://github.com/sidekiq/sidekiq/wiki/Middleware) earlier in the stack.
+
+ 4. Or let this gem automatically pick tenant name from [apartment-sidekiq](https://github.com/influitive/apartment-sidekiq) if you're using apartment gem.
+
+## Configuration
+
+Configuration is handled by [anyway_config] gem. With it you can load settings from environment variables (which names are constructed from config key upcased and prefixed with `SIDEKIQ_FAIR_TENANT_`), YAML files, and other sources. See [anyway_config] docs for details.
+
+| Config key                 | Type     | Default                                                             | Description                                                                                                                                                                                                             |
+|----------------------------|----------|---------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `max_throttling_window`    | integer  | `86_400` (1 day)                                                    | Maximum throttling window in seconds                                                                                                                                                                                    |
+| `enqueues_key`             | string   | `sidekiq-fair_tenant:enqueued:%<job_class>s:tenant:%<fair_tenant>s` | Ruby [format string](https://docs.ruby-lang.org/en/3.3/format_specifications_rdoc.html) used as a name for Redis key holding job ids for throttling window. Available placeholders: `queue`, `job_class`, `fair_tenant` |
+| `logger`                   | logger   | `Sidekiq.logger`                                                    | Logger instance used for warning logging.                                                                                                                                                                               |
+
+## How it works
+
+If number of jobs enqueued by a single client exceeds some threshold per a sliding time window, their jobs would be re-routed to another queue, with lower priority.
+
+This gem tracks single client's jobs in a Redis [sorted set](https://redis.io/docs/data-types/sorted-sets/) with job id as a key and enqueuing timestamp as a score. When a new job is enqueued, it is added to the set, and then the set is trimmed to contain only jobs enqueued within the last `max_throttling_window` seconds.
+
+On every enqueue attempt, the set is checked for number of jobs enqueued within the last `per` seconds of every rule. If the number of jobs in this time window exceeds `threshold`, the job is enqueued to a throttled queue, otherwise it is enqueued to the default queue. If multiple rules match, last one is used.
+
+You are expected to configure Sidekiq to process throttled queues with lower priority using [queue weights](https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues).
+
+### Advantages
+ - If fast queues are empty then slow queues are processed at full speed (no artificial delays)
+ - If fast queues are full, slow queues are still processed, but slower (configurable), so application doesn’t “stall” for throttled users
+ - Minimal changes to the application code are required.
+
+### Disadvantages
+ - As Sidekiq does not support mixing ordered and weighted queue modes (as stated in Sidekiq Wiki on queue configuration), you can’t make the same worker process execute some super important queue always first, ignoring other queues. Run separate worker to solve this.
+ - You have to keep track of all your queues and their weights.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/sidekiq-fair_tenant.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+[sidekiq]: https://github.com/sidekiq/sidekiq "Simple, efficient background processing for Ruby"
+[anyway_config]: https://github.com/palkan/anyway_config "Configuration library for Ruby gems and applications"

data/lib/sidekiq/fair_tenant/client_middleware.rb

@@ -0,0 +1,100 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module FairTenant
+    # Client middleware re-routing jobs of overly active tenants to slower queues based on thresholds
+    class ClientMiddleware
+      # Re-routes job to the most appropriate queue, based on tenant's throttling rules
+      # rubocop:disable Metrics/MethodLength
+      def call(worker, job, queue, redis_pool)
+        job_class = job_class(worker, job)
+        arguments = job["wrapped"] ? job.dig("args", 0, "arguments") : job["args"]
+        return yield unless enabled?(job_class, job, queue)
+
+        job["fair_tenant"] ||= tenant_id(job_class, job, arguments)
+        unless job["fair_tenant"]
+          logger.warn "#{job_class} with args #{arguments.inspect} won't be throttled: missing `fair_tenant` in job"
+          return yield
+        end
+
+        redis_pool.then do |redis|
+          register_job(job_class, job, queue, redis)
+          job["queue"] = assign_queue(job_class, job, queue, redis)
+        end
+
+        yield
+      end
+      # rubocop:enable Metrics/MethodLength
+
+      private
+
+      def enabled?(job_class, job, queue)
+        return false if job["fair_tenant_queues"].blank? # Not configured for throttling
+
+        original_queue = original_queue(job_class, job, queue)
+        return false if original_queue != job["queue"] # Someone already rerouted this job, nothing to do here
+
+        true
+      end
+
+      # Writes job to sliding window sorted set
+      def register_job(job_class, job, queue, redis)
+        enqueues_key = enqueues_key(job_class, job, queue)
+        max_throttling_window = Sidekiq::FairTenant.max_throttling_window
+        redis.multi do |tx|
+          tx.zremrangebyscore(enqueues_key, "-inf", (Time.now - max_throttling_window).to_i)
+          tx.zadd(enqueues_key, Time.now.to_i, "jid:#{job["jid"]}")
+          tx.expire(enqueues_key, max_throttling_window)
+        end
+      end
+
+      # Chooses the last queue, for the most restrictive (threshold/time) rule that is met.
+      # Assumes the slowest queue, with most restrictive rule, comes last in the `fair_tenants` array.
+      def assign_queue(job_class, job, queue, redis)
+        enqueues_key = enqueues_key(job_class, job, queue)
+
+        matching_rules =
+          job["fair_tenant_queues"].map(&:symbolize_keys).filter do |config|
+            threshold = config[:threshold]
+            window_start = Time.now - (config[:per] || Sidekiq::FairTenant.max_throttling_window)
+            threshold < redis.zcount(enqueues_key, window_start.to_i, Time.now.to_i)
+          end
+
+        matching_rules.any? ? matching_rules.last[:queue] : queue
+      end
+
+      def enqueues_key(job_class, job, queue)
+        format(Sidekiq::FairTenant.enqueues_key, queue: queue, fair_tenant: job["fair_tenant"], job_class: job_class)
+      end
+
+      def job_class(worker, job)
+        job_class = job["wrapped"] || worker
+        return job_class if job_class.is_a?(Class)
+        return job_class.constantize if job_class.respond_to?(:constantize)
+
+        Object.const_get(job_class.to_s)
+      end
+
+      # Calculates tenant identifier (`fair_tenant`) for the job
+      def tenant_id(job_class, job, arguments)
+        return job_class.fair_tenant(*arguments) if job_class.respond_to?(:fair_tenant)
+
+        job["apartment"] # for compatibility with sidekiq-apartment
+      end
+
+      def original_queue(job_class, _job, queue)
+        if job_class.respond_to?(:queue_name)
+          job_class.queue_name # ActiveJob
+        elsif job_class.respond_to?(:queue)
+          job_class.queue.to_s # Sidekiq
+        else
+          queue
+        end
+      end
+
+      def logger
+        Sidekiq::FairTenant.logger
+      end
+    end
+  end
+end

data/lib/sidekiq/fair_tenant/config.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require "anyway"
+
+module Sidekiq
+  module FairTenant
+    # Runtime configuration for the sidekiq-fair_tenant gem
+    class Config < ::Anyway::Config
+      config_name :sidekiq_fair_tenant
+
+      # Maximum amount of time to store information about tenant enqueues
+      attr_config max_throttling_window: 86_400 # 1 day
+
+      # Sorted set that contains job ids enqueued by each tenant in last 1 day (max throttling window)
+      attr_config enqueues_key: "sidekiq-fair_tenant:enqueued:%<job_class>s:tenant:%<fair_tenant>s"
+
+      # Logger to use for throttling warnings
+      attr_config logger: ::Sidekiq.logger
+    end
+  end
+end

data/lib/sidekiq/fair_tenant/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module FairTenant
+    VERSION = "0.1.0"
+  end
+end

data/lib/sidekiq/fair_tenant.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+
+require_relative "fair_tenant/version"
+require_relative "fair_tenant/config"
+require_relative "fair_tenant/client_middleware"
+
+module Sidekiq
+  # Client middleware and job DSL for throttling jobs of overly active tenants
+  module FairTenant
+    class Error < ::StandardError; end
+
+    class << self
+      extend ::Forwardable
+
+      def config
+        @config ||= Config.new
+      end
+
+      def_delegators :config, :max_throttling_window, :enqueues_key, :logger
+    end
+  end
+end
+
+Sidekiq.configure_client do |config|
+  config.client_middleware do |chain|
+    chain.add Sidekiq::FairTenant::ClientMiddleware
+  end
+end
+
+Sidekiq.configure_server do |config|
+  config.client_middleware do |chain|
+    chain.add Sidekiq::FairTenant::ClientMiddleware
+  end
+end

data/sidekiq-fair_tenant.gemspec

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require_relative "lib/sidekiq/fair_tenant/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "sidekiq-fair_tenant"
+  spec.version = Sidekiq::FairTenant::VERSION
+  spec.authors = ["Andrey Novikov"]
+  spec.email = ["envek@envek.name"]
+
+  spec.summary = "Throttle Sidekiq jobs of greedy tenants"
+  spec.description = "Re-route jobs of way too active tenants to slower queues, letting other tenant's jobs to go first"
+  spec.homepage = "https://github.com/Envek/sidekiq-fair_tenant"
+  spec.license = "MIT"
+  spec.required_ruby_version = ">= 2.6.0"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = "https://github.com/Envek/sidekiq-fair_tenant"
+  spec.metadata["changelog_uri"] = "https://github.com/Envek/sidekiq-fair_tenant/blob/master/CHANGELOG.md"
+  spec.metadata["rubygems_mfa_required"] = "true"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ spec/ Gemfile Rakefile]) ||
+        f.match(/^(\.)/)
+    end
+  end
+  spec.bindir = "exe"
+  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "anyway_config", ">= 1.0", "< 3"
+  spec.add_dependency "sidekiq", ">= 5"
+end

metadata

@@ -0,0 +1,90 @@
+--- !ruby/object:Gem::Specification
+name: sidekiq-fair_tenant
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Andrey Novikov
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2024-01-12 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: anyway_config
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5'
+description: Re-route jobs of way too active tenants to slower queues, letting other
+  tenant's jobs to go first
+email:
+- envek@envek.name
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- lib/sidekiq/fair_tenant.rb
+- lib/sidekiq/fair_tenant/client_middleware.rb
+- lib/sidekiq/fair_tenant/config.rb
+- lib/sidekiq/fair_tenant/version.rb
+- sidekiq-fair_tenant.gemspec
+homepage: https://github.com/Envek/sidekiq-fair_tenant
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/Envek/sidekiq-fair_tenant
+  source_code_uri: https://github.com/Envek/sidekiq-fair_tenant
+  changelog_uri: https://github.com/Envek/sidekiq-fair_tenant/blob/master/CHANGELOG.md
+  rubygems_mfa_required: 'true'
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.6.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.3
+signing_key: 
+specification_version: 4
+summary: Throttle Sidekiq jobs of greedy tenants
+test_files: []