pause 0.4.0 → 0.5.1

data/README.md

@@ -1,11 +1,17 @@
-# Pause
 
-[![Gem Version](https://badge.fury.io/rb/pause.png)](http://badge.fury.io/rb/pause)
-[![Build Status](https://travis-ci.org/kigster/pause.svg?branch=master)](https://travis-ci.org/kigster/pause)
+[![RSpec](https://github.com/kigster/pause/actions/workflows/rspec.yml/badge.svg?style=for-the-badge)](https://github.com/kigster/pause/actions/workflows/rspec.yml)
+[![Rubocop](https://github.com/kigster/pause/actions/workflows/rubocop.yml/badge.svg?style=for-the-badge)](https://github.com/kigster/pause/actions/workflows/rubocop.yml)
 
-## In a Nutshell
+[![Gem Version](https://badge.fury.io/rb/pause@2x.png?icon=si%3Arubygems)](https://badge.fury.io/rb/pause)
 
-**Pause** is a fast and very flexible Redis-backed rate-limiter. You can use it to track events, with
+[![Downloads](https://img.shields.io/gem/dt/pause.svg?style=for-the-badge&color=0AF)](https://rubygems.org/gems/pause)
+[![License](https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge&color=0AF)](https://opensource.org/licenses/MIT)
+
+# Pause — A Redis-backed Rate Limiter
+
+## Overview
+
+**Pause** is a fast and very flexible Redis-backed rate-limiter, written originally at [Wanelo.com](https://www.crunchbase.com/organization/wanelo). You can use it to track events, with
 rules around how often they are allowed to occur within configured time checks.
 
 Sample applications include:
@@ -49,17 +55,21 @@ Pause.configure do |config|
   config.redis_host = '127.0.0.1'
   config.redis_port = 6379
   config.redis_db   = 1
-  
-  # aggregate all events into 10 minute blocks. 
-  # Larger blocks require less RAM and CPU, smaller blocks are more 
-  # computationally expensive.
-  config.resolution = 600
-       
-  # discard all events older than 1 day
-  config.history    = 86400   
+  config.resolution = 600     
+  config.history    = 7 * 86400  # discard events older than 7 days   
 end
 ```
 
+> NOTE: **resolution** is an setting that's key to understanding how Pause works. It represents the length of time during which similar events are aggregated into a Hash-like object, where the key is the identifier, and the value is the count within that period.
+> 
+> Because of this,
+>  
+>   * _Larger resolution requires less RAM and CPU and are faster to compute_
+>   * _Smaller resolution is more computationally expensive, but provides higher granularity_.
+>
+> The resolution setting must set to the smallest rate-limit period across all of your checks. Below it is set to 10 minutes, meaning that you can use Pause to **rate limit any event to no more than N times within a period of 10 minutes or more.**
+
+
 #### Define Rate Limited "Action"
 
 Next we must define the rate limited action based on the specification above. This is how easy it is:
@@ -69,13 +79,22 @@ module MyApp
   class UserNotificationLimiter < ::Pause::Action
     # this is a redis key namespace added to all data in this action
     scope 'un'  
-    check period_seconds:       120, max_allowed: 1
-    check period_seconds:     86400, max_allowed: 3
-    check period_seconds: 7 * 86400, max_allowed: 7
+    
+    check period_seconds: 120, 
+          max_allowed: 1, 
+          block_ttl: 240
+          
+    check period_seconds: 86400, 
+          max_allowed: 3
+          
+    check period_seconds: 7 * 86400, 
+          max_allowed: 7
   end
 end
 ```
 
+> NOTE: for each check, `block_ttl` defaults to `period_seconds`, and represents the duration of time the action will consider itself as "rate limited" after a particular check reaches the limit.  Note, that all actions will automatically leave the "rate limited" state after `block_ttl` seconds have passed.
+
 #### Perform operation, but only if the user is not rate-limited
 
 Now we simply instantiate this limiter by passing user ID (any unique identifier works). We can then ask the limiter, `ok?` or `rate_limited?`, or we can use two convenient methods that only execute enclosed block if the described condition is satisfied:
@@ -83,17 +102,18 @@ Now we simply instantiate this limiter by passing user ID (any unique identifier
 ```ruby
 class NotificationsWorker
   def perform(user_id)
-    limiter = MyApp::UserNotificationLimiter.new(user_id)
-    
-    limiter.unless_rate_limited do
-      user = User.find(user_id) 
-      user.send_push_notification!
-    end
-    
-    # You can also do something in case the user is rate limited:
-    limiter.if_rate_limited do |rate_limit_event|
-      Rails.logger.info("user #{user.id} has exceeded rate limit: #{rate_limit_event}") 
-    end
+    MyApp::UserNotificationLimiter.new(user_id) do 
+      unless_rate_limited do
+        # this block ONLY runs if rate limit is not reached
+        user = User.find(user_id) 
+        PushNotifications.new(user).send_push_notification!
+      end
+      
+      if_rate_limited do |rate_limit_event|
+        # this block ONLY runs if the action has reached it's rate limit.
+        Rails.logger.info("user #{user.id} has exceeded rate limit: #{rate_limit_event}") 
+      end
+    end   
   end
 end
 ```
@@ -119,12 +139,17 @@ Or install it yourself as:
 
 ### Configuration
 
-Configure Pause. This could be in a Rails initializer.
+Configure Pause. This could be in a Rails initializer. At the moment, pause can only be used as a singleton, i.e. you can not use `pause` with multiple configurations, or multiple redis backends. This is something that can be rather easily fixed, if necessary.
 
-  * resolution - The time resolution (in seconds) defining the minimum period into which action counts are
+Therefore, you configure the Pause singleton with the following options: 
+ 
+  * **redis connection parameters** - The host, port and db of the Redis instance to use.
+
+  * **resolution** - the length of time (in seconds) defining the minimum period into which action counts are
                  aggregated. This defines the size of the persistent store. The higher the number, the less data needs
                  to be persisted in Redis.
-  * history - The maximum amount of time (in seconds) that data is persisted
+
+  * **history** - The maximum amount of time (in seconds) that data is persisted.
 
 ```ruby
 Pause.configure do |config|
@@ -136,12 +161,10 @@ Pause.configure do |config|
 end
 ```
 
+
 ### Actions
 
-Define local actions for your application. Actions define a scope by
-which they are identified in the persistent store (aka "namespace"), and a set of checks.  Checks define various
-thresholds (`max_allowed`) against periods of time (`period_seconds`). When a threshold it triggered,
-the action is rate limited, and stays rate limited for the duration of `block_ttl` seconds.
+Define local actions for your application. Actions define a scope by which they are identified in the persistent store (aka "namespace"), and a set of checks.  Checks define various thresholds (`max_allowed`) against periods of time (`period_seconds`). When a threshold it triggered, the action is rate limited, and stays rate limited for the duration of `block_ttl` seconds.
 
 #### Checks
 
@@ -185,6 +208,7 @@ When an event occurs, you increment an instance of your action, optionally with
 In the example at the top of the README you saw how we used `#unless_rate_limited` and `#if_rate_limited` methods. These are the recommended API methods, but if you must get a finer-grained control over the actions, you can also use methods such as `#ok?`, `#rate_limited?`, `#increment!` to do manually what the block methods do already. Below is an example of this "manual" implementation:
 
 ```ruby
+# app/controllers/follows_controller.rb
 class FollowsController < ApplicationController
   def create
     action = FollowAction.new(user.id)
@@ -196,6 +220,7 @@ class FollowsController < ApplicationController
   end
 end
 
+# app/controlers/other_controller.rb
 class OtherController < ApplicationController
   def index
     action = OtherAction.new(params[:thing])d
@@ -217,11 +242,16 @@ while true
   action.increment!
 
   rate_limit_event = action.analyze
-  if rate_limit_event
-    puts rate_limit_event.identifier               # which key got rate limited ("thing")
-    puts rate_limit_event.sum                      # total count that triggered a rate limit
-    puts rate_limit_event.timestamp                # timestamp when rate limiting occurred
-    puts rate_limit_event.period_check             # period check object, that triggered this rate limiting event
+  
+  if rate_limit_event    # or action.ok?
+    # which key got rate limited ("thing")
+    rate_limit_event.identifier
+    # total count that triggered a rate limit
+    rate_limit_event.sum
+    # timestamp when rate limiting occurred
+    rate_limit_event.timestamp
+    # period check object, that triggered this rate limiting event
+    rate_limit_event.period_check
   else
     # not rate-limited, same as action.ok?
   end
@@ -301,21 +331,19 @@ Pause.configure do |config|
 end
 ```
 
-With this configuration, any Pause operation that we know is not supported by Twemproxy will raise
-`Pause::Redis::OperationNotSupported`. For instance, when sharding we are unable to get a list of all
-tracked identifiers.
+With this configuration, any Pause operation that we know is not supported by Twemproxy will raise `Pause::Redis::OperationNotSupported`. For instance, when sharding we are unable to get a list of all tracked identifiers.
 
 The action block list is implemented as a sorted set, so it should still be usable when sharding.
 
 ## Testing
 
-By default, `fakeredis` gem is used to emulate Redis in development. However, the same test-suite should be able to run against a real redis — however, be aware that it will flush the current db during spec run. In order to run specs against real redis, make sure you have Redis running locally on the default port, and that you are able to connect to it using `redis-cli`.
+By default, `fakeredis` gem is used to emulate Redis in development. 
 
-Please note that Travis suite, as well as the default rake task, run both.
+However, the same test-suite runs against a real redis, just be aware that using real redis server running locally on 127.0.0.1 will result in a `flush` operation on the current redis db during the spec run. In order to run specs against real redis, please make sure you have Redis running locally on the default port (6379), and that you are able to connect to it using `redis-cli -p 6379 -h 127.0.0.1`.
 
-### Unit Testing with Fakeredis
+### Unit Testing with FakeRedis
 
-Fakeredis is the default, and is also run whenever `bundle exec rspec` is executed, or `rake spec` task invoked.
+The gem `fakeredis` is the default for testing, and is also run whenever `bundle exec rspec` is executed, or `rake spec` task invoked.
 
 ```bash
 bundle exec rake spec:unit
@@ -327,6 +355,16 @@ bundle exec rake spec:unit
 bundle exec rake spec:integration
 ```
 
+OR
+
+```bash
+PAUSE_REAL_REDIS=1 bundle exec rspec
+```
+
+### Test Coverage
+
+At the time of this writing the gem has 100% test coverage. Please keep it that way ;)
+
 ## Contributing
 
 Want to make it better? Cool. Here's how:
@@ -339,8 +377,16 @@ Want to make it better? Cool. Here's how:
 
 ## Authors
 
-This gem was written by Eric Saxby, Atasay Gokkaya and Konstantin Gredeskoul at Wanelo, Inc.
+ * This gem was donated to Open Source by [Wanelo, Inc](https://www.crunchbase.com/organization/wanelo)
+
+ * The original authors are:
+   * [Atasay Gökkaya](https://github.com/atasay)
+   * [Konstantin Gredeskoul](https://github.com/kigster)
+   * [Eric Saxby](https://github.com/sax)
+   * Paul Henry
+ 
+ * The gem is currently maintained and kept up to date by [Konstantin Gredeskoul](https://kig.re/)
 
-Please see the LICENSE.txt file for further details.
+Please see the [LICENSE.txt](LICENSE.txt) file for further details.
 
 

data/Rakefile

@@ -1,10 +1,12 @@
+# frozen_string_literal: true
+
 require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
 require 'yard'
 
 RSpec::Core::RakeTask.new(:spec)
 
-task :default => %w(spec:unit spec:integration)
+task default: %w[spec:unit spec:integration]
 
 namespace :spec do
   desc 'Run specs using fakeredis'
@@ -25,23 +27,22 @@ def shell(*args)
 end
 
 task :clean do
-  shell('rm -rf pkg/ tmp/ coverage/ doc/ ' )
+  shell('rm -rf pkg/ tmp/ coverage/ doc/ ')
 end
 
-task :gem => [:build] do
+task gem: [:build] do
   shell('gem install pkg/*')
 end
 
-task :permissions => [ :clean ] do
+task permissions: [:clean] do
   shell('chmod -v o+r,g+r * */* */*/* */*/*/* */*/*/*/* */*/*/*/*/*')
-  shell("find . -type d -exec chmod o+x,g+x {} \\;")
+  shell('find . -type d -exec chmod o+x,g+x {} \\;')
 end
 
-task :build => :permissions
+task build: :permissions
 
 YARD::Rake::YardocTask.new(:doc) do |t|
-  t.files = %w(lib/**/*.rb exe/*.rb - README.md LICENSE.txt)
-  t.options.unshift('--title','"Pause - Redis-backed Rate Limiter"')
-  t.after = ->() { exec('open doc/index.html') }
+  t.files = %w[lib/**/*.rb exe/*.rb - README.md LICENSE.txt]
+  t.options.unshift('--title', '"Pause - Redis-backed Rate Limiter"')
+  t.after = -> { exec('open doc/index.html') }
 end
-

data/bin/spec

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+retry-errors() {
+  sleep 1
+  bundle exec rspec --only-failures
+}
+
+specs() {
+  bundle exec rspec && \
+    PAUSE_REAL_REDIS=true bundle exec rspec
+}
+
+specs || retry-errors
+

data/lib/pause/action.rb

@@ -1,10 +1,13 @@
+# frozen_string_literal: true
+
 module Pause
   class Action
     attr_accessor :identifier
 
-    def initialize(identifier)
-      @identifier       = identifier
+    def initialize(identifier, &block)
+      @identifier = identifier
       self.class.checks ||= []
+      instance_exec(&block) if block
     end
 
     def scope
@@ -22,9 +25,8 @@ module Pause
           #       scope "my:scope"
           #     end
           #
-          @scope = klass.name.downcase.gsub(/::/, '.')
+          @scope = klass.name.downcase.gsub('::', '.')
           class << self
-
             # @param [String] args
             def scope(*args)
               @scope = args.first if args && args.size == 1
@@ -115,7 +117,7 @@ module Pause
       end
     end
 
-    def if_rate_limited(&_block)
+    def if_rate_limited(&)
       check_result = analyze(recalculate: true)
       yield(check_result) unless check_result.nil?
     end
@@ -156,6 +158,5 @@ module Pause
     def adapter
       self.class.adapter
     end
-
   end
 end

data/lib/pause/analyzer.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require 'pause/helper/timing'
 
 module Pause
@@ -12,12 +14,14 @@ module Pause
     # @return [Pause::RateLimitedEvent]  the action was blocked as a result of this check
     def check(action, recalculate: false)
       return false if adapter.rate_limited?(action.scope, action.identifier) && !recalculate
+
       timestamp = period_marker(Pause.config.resolution, Time.now.to_i)
       set       = adapter.key_history(action.scope, action.identifier)
       action.checks.each do |period_check|
         start_time = timestamp - period_check.period_seconds
         set.reverse.inject(0) do |sum, element|
           break if element.ts < start_time
+
           sum += element.count
           if sum >= period_check.max_allowed
             adapter.rate_limit!(action.scope, action.identifier, period_check.block_ttl)

data/lib/pause/configuration.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Pause
   class Configuration
     attr_writer :redis_host, :redis_port, :redis_db, :resolution, :history, :sharded
@@ -24,7 +26,7 @@ module Pause
     end
 
     def history
-      (@history || 86400).to_i
+      (@history || 86_400).to_i
     end
 
     def sharded

data/lib/pause/helper/timing.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Pause
   module Helper
     module Timing

data/lib/pause/logger.rb

@@ -1,11 +1,18 @@
+# frozen_string_literal: true
+
 module Pause
+  # @description Logger class for Pause
   class Logger
-    def self.puts message
-      STDOUT.puts message
-    end
+    class << self
+      def puts(message)
+        $stdout.puts message
+      end
 
-    def self.fatal message
-      STDERR.puts message
+      def fatal(message)
+        # rubocop: disable Style/StderrPuts
+        $stderr.puts message.red
+        # rubocop: enable Style/StderrPuts
+      end
     end
   end
 end

data/lib/pause/rate_limited_event.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Pause
   class RateLimitedEvent
     attr_accessor :action, :identifier, :period_check, :sum, :timestamp
@@ -9,6 +11,5 @@ module Pause
       @sum = sum
       @timestamp = timestamp
     end
-
   end
 end

data/lib/pause/redis/adapter.rb

@@ -1,19 +1,20 @@
+# frozen_string_literal: true
+
 require 'pause/helper/timing'
 
 module Pause
   module Redis
-
     # This class encapsulates Redis operations used by Pause
     class Adapter
       class << self
         def redis
-          @redis_conn ||= ::Redis.new(redis_connection_opts)
+          @redis ||= ::Redis.new(redis_connection_opts)
         end
 
         def redis_connection_opts
           { host: Pause.config.redis_host,
             port: Pause.config.redis_port,
-            db:   Pause.config.redis_db }
+            db: Pause.config.redis_db }
         end
       end
 
@@ -72,6 +73,7 @@ module Pause
       # @return count [Integer] the number of items deleted
       def delete_rate_limited_keys(scope)
         return 0 unless rate_limited_keys(scope).any?
+
         delete_tracking_keys(scope, rate_limited_keys(scope))
         redis.zremrangebyscore(rate_limited_list(scope), '-inf', '+inf').tap do |_count|
           redis.del rate_limited_list(scope)
@@ -84,7 +86,7 @@ module Pause
       end
 
       def disable(scope)
-        redis.set("internal:|#{scope}|:disabled", "1")
+        redis.set("internal:|#{scope}|:disabled", '1')
       end
 
       def enable(scope)
@@ -110,11 +112,11 @@ module Pause
       end
 
       def truncate_set_for(k)
-        if redis.zcard(k) > time_blocks_to_keep
-          list      = extract_set_elements(k)
-          to_remove = list.slice(0, (list.size - time_blocks_to_keep)).map(&:ts)
-          redis.zrem(k, to_remove) if k && to_remove && to_remove.size > 0
-        end
+        return unless redis.zcard(k) > time_blocks_to_keep
+
+        list      = extract_set_elements(k)
+        to_remove = list.slice(0, (list.size - time_blocks_to_keep)).map(&:ts)
+        redis.zrem(k, to_remove) if k && to_remove&.size&.positive?
       end
 
       def delete_tracking_keys(scope, ids)
@@ -137,7 +139,7 @@ module Pause
 
       def keys(key_scope)
         redis.keys("#{key_scope}:*").map do |key|
-          key.gsub(/^#{key_scope}:/, "").tr('|', '')
+          key.gsub(/^#{key_scope}:/, '').tr('|', '')
         end
       end
 

data/lib/pause/redis/sharded_adapter.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Pause
   module Redis
     class OperationNotSupported < StandardError
@@ -7,7 +9,6 @@ module Pause
     # Operations that are not possible when data is sharded
     # raise an error.
     class ShardedAdapter < Adapter
-
       # Overrides real multi which is not possible when sharded.
       def with_multi
         yield(redis) if block_given?
@@ -23,7 +24,7 @@ module Pause
       private
 
       def keys(_key_scope)
-        raise OperationNotSupported.new('Can not be executed when Pause is configured in sharded mode')
+        raise OperationNotSupported, 'Can not be executed when Pause is configured in sharded mode'
       end
     end
   end

data/lib/pause/version.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Pause
-  VERSION = '0.4.0'
+  VERSION = '0.5.1'
 end

data/lib/pause.rb

@@ -1,4 +1,7 @@
+# frozen_string_literal: true
+
 require 'redis'
+require 'colored2'
 require 'pause/version'
 require 'pause/configuration'
 require 'pause/action'
@@ -9,15 +12,23 @@ require 'pause/redis/sharded_adapter'
 require 'pause/rate_limited_event'
 
 module Pause
-  class PeriodCheck < Struct.new(:period_seconds, :max_allowed, :block_ttl)
+  PeriodCheck = Struct.new(:period_seconds, :max_allowed, :block_ttl) do
+    def initialize(*args, period_seconds: nil, max_allowed: nil, block_ttl: nil)
+      if args.any?
+        super(*args)
+      else
+        super(period_seconds, max_allowed, block_ttl)
+      end
+    end
+
     def <=>(other)
-      self.period_seconds <=> other.period_seconds
+      period_seconds <=> other.period_seconds
     end
   end
 
-  class SetElement < Struct.new(:ts, :count)
+  SetElement = Struct.new(:ts, :count) do
     def <=>(other)
-      self.ts <=> other.ts
+      ts <=> other.ts
     end
   end
 
@@ -27,21 +38,21 @@ module Pause
     end
 
     def adapter
-      @adapter ||= config.sharded ?
-        Pause::Redis::ShardedAdapter.new(config) :
-        Pause::Redis::Adapter.new(config)
+      @adapter ||= if config.sharded
+                     Pause::Redis::ShardedAdapter.new(config)
+                   else
+                     Pause::Redis::Adapter.new(config)
+                   end
     end
 
-    def adapter=(adapter)
-      @adapter = adapter
-    end
+    attr_writer :adapter
 
-    def configure(&block)
-      @configuration ||= Pause::Configuration.new.configure(&block)
+    def configure(&)
+      @configure ||= Pause::Configuration.new.configure(&)
     end
 
-    def config(&block)
-      configure(&block)
+    def config(&)
+      configure(&)
     end
   end
 end

data/pause.gemspec

@@ -1,30 +1,27 @@
-# -*- encoding: utf-8 -*-
-lib = File.expand_path('../lib', __FILE__)
+# frozen_string_literal: true
+
+require 'English'
+lib = File.expand_path('lib', __dir__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'pause/version'
 
 Gem::Specification.new do |gem|
   gem.name          = 'pause'
   gem.version       = Pause::VERSION
-  gem.authors       = ['Atasay Gokkaya', 'Paul Henry', 'Eric Saxby', 'Konstantin Gredeskoul']
-  gem.email         = %w(atasay@wanelo.com paul@wanelo.com sax@ericsaxby.com kigster@gmail.com)
-  gem.summary       = %q(Fast, scalable, and flexible real time rate limiting library for distributed Ruby environments backed by Redis.)
-  gem.description   = %q(This gem provides highly flexible and easy to use interface to define rate limit checks, register events as they come, and verify if the rate limit is reached. Multiple checks for the same metric are easily supported. This gem is used at very high scale on several popular web sites.)
+  gem.authors       = ['Konstantin Gredeskoul', 'Atasay Gokkaya', 'Paul Henry', 'Eric Saxby']
+  gem.email         = %w[kigster@gmail.com atasay@wanelo.com paul@wanelo.com sax@ericsaxby.com]
+  gem.summary       = 'Fast, scalable, and flexible real time rate limiting library for distributed Ruby environments backed by Redis.'
+  gem.description   = 'This gem provides highly flexible and easy to use interface to define rate limit checks, register events as they come, and verify if the rate limit is reached. Multiple checks for the same metric are easily supported. This gem is used at very high scale on several popular web sites.'
   gem.homepage      = 'https://github.com/kigster/pause'
 
-  gem.files         = `git ls-files`.split($/)
-  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
-  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.files         = `git ls-files`.split($INPUT_RECORD_SEPARATOR)
+  gem.executables   = gem.files.grep(%r{^bin/}).map { |f| File.basename(f) }
   gem.require_paths = ['lib']
 
+  gem.add_dependency 'colored2'
   gem.add_dependency 'redis'
-  gem.add_dependency 'hiredis'
 
-  gem.add_development_dependency 'simplecov'
-  gem.add_development_dependency 'yard'
-  gem.add_development_dependency 'rspec'
-  gem.add_development_dependency 'fakeredis'
-  gem.add_development_dependency 'guard-rspec'
-  gem.add_development_dependency 'timecop'
-  gem.add_development_dependency 'rake'
+  # optional
+  # gem.add_dependency 'hiredis'
+  gem.metadata['rubygems_mfa_required'] = 'true'
 end