prop 1.2.0 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2601b63983949b53ff9f31664203316614466b74
-  data.tar.gz: dc5deeebea017087d28a378e83783e3d7a5001ae
+  metadata.gz: c346d4ae613e60059428af7761f47af474a02c63
+  data.tar.gz: 5eadca75e381659c036eaf348ff24181148d127b
 SHA512:
-  metadata.gz: fcd418b8e8bd9c96bb84ad902db628ce7c792a3ff3324b52c682ed989ad951595aa35564f26386f742ee07f73c3af42b490cc1731614ae255b4cc2d2988608eb
-  data.tar.gz: 43d95cc4198eacf0c3fe20182788085c778a7d20d67ffeb97c1875f3f8488275d2ea2e0aa8e997e82ffffb2aab56a7c37bbab619fb8ca930d7b3c72b94a76150
+  metadata.gz: ce9e3393a3ff39720b1083a34d96b9c3c960f0ba5b47ab1f1cc7cb131132183880a6de7b0928a9736fb93c29ea1a81bd39abc79287783b5b85633bddf36edd48
+  data.tar.gz: 1ee56b82228280ef143b2cece612d2e310da75a99685259da374b05b6fe9a39810fb6fbecd006d260e9220c1b88e172b3793919fa1ca2ce4e1e4c26c889a3abc

data/README.md

@@ -1,30 +1,29 @@
 
 # Prop [![Build Status](https://travis-ci.org/zendesk/prop.png)](https://travis-ci.org/zendesk/prop)
 
-Prop is a simple gem for rate limiting requests of any kind. It allows you to configure hooks for registering certain actions, such that you can define thresholds, register usage and finally act on exceptions once thresholds get exceeded.
+A gem to rate limit requests/actions of any kind.<br/>
+Define thresholds, register usage and finally act on exceptions once thresholds get exceeded.
 
 Prop supports two limiting strategies:
 
-* Basic strategy (default): Prop will use an interval to define a window of time using simple div arithmetic. This means that it's a worst-case throttle that will allow up to two times the specified requests within the specified interval.
-* Leaky bucket strategy: Prop also supports the [Leaky Bucket](https://en.wikipedia.org/wiki/Leaky_bucket) algorithm, which is similar to the basic strategy but also supports bursts up to a specified threshold.
+* Basic strategy (default): Prop will use an interval to define a window of time using simple div arithmetic. 
+This means that it's a worst-case throttle that will allow up to two times the specified requests within the specified interval.
+* Leaky bucket strategy: Prop also supports the [Leaky Bucket](https://en.wikipedia.org/wiki/Leaky_bucket) algorithm, 
+which is similar to the basic strategy but also supports bursts up to a specified threshold.
 
-To get going with Prop, you first define the read and write operations. These define how you write a registered request and how to read the number of requests for a given action. For example, do something like the below in a Rails initializer:
+To store values, prop needs a cache:
 
 ```ruby
-Prop.read do |key|
-  Rails.cache.read(key)
-end
-
-Prop.write do |key, value|
-  Rails.cache.write(key, value)
-end
+# config/initializers/prop.rb
+Prop.cache = Rails.cache # needs read/write/increment methods
 ```
 
-You can choose to rely on whatever you'd like to use for transient storage. Prop does not do any sort of clean up of its key space, so you would have to implement that manually should you be using anything but an LRU cache like memcached.
+Prop does not expire its used keys, so use memcached or similar, not redis.
 
 ## Setting a Callback
 
-You can define an optional callback that is invoked when a rate limit is reached. In a Rails application you could use such a handler to add notification support:
+You can define an optional callback that is invoked when a rate limit is reached. In a Rails application you 
+could use such a handler to add notification support:
 
 ```ruby
 Prop.before_throttle do |handle, key, threshold, interval|
@@ -34,14 +33,12 @@ end
 
 ## Defining thresholds
 
-Once the read and write operations are defined, you can optionally define thresholds. If, for example, you want to have a threshold on accepted emails per hour from a given user, you could define a threshold and interval (in seconds) for this like so:
+Example: Limit on accepted emails per hour from a given user, by defining a threshold and interval (in seconds):
 
 ```ruby
 Prop.configure(:mails_per_hour, threshold: 100, interval: 1.hour, description: "Mail rate limit exceeded")
 ```
 
-The `:mails_per_hour` in the above is called the "handle". You can now put the throttle to work with these values, by passing the handle to the respective methods in Prop:
-
 ```ruby
 # Throws Prop::RateLimitExceededError if the threshold/interval has been reached
 Prop.throttle!(:mails_per_hour)
@@ -52,18 +49,18 @@ Prop.throttle!(:expensive_request) { calculator.something_very_hard }
 # Returns true if the threshold/interval has been reached
 Prop.throttled?(:mails_per_hour)
 
-# Sets the throttle "count" to 0
+# Sets the throttle count to 0
 Prop.reset(:mails_per_hour)
 
 # Returns the value of this throttle, usually a count, but see below for more
 Prop.count(:mails_per_hour)
 ```
 
-Prop will raise a `RuntimeError` if you attempt to operate on an undefined handle.
+Prop will raise a `KeyError` if you attempt to operate on an undefined handle.
 
 ## Scoping a throttle
 
-In many cases you will want to tie a specific key to a defined throttle. For example, you can scope the throttling to a specific sender rather than running a global "mails per hour" throttle:
+Example: scope the throttling to a specific sender rather than running a global "mails per hour" throttle:
 
 ```ruby
 Prop.throttle!(:mails_per_hour, mail.from)
@@ -72,7 +69,7 @@ Prop.reset(:mails_per_hour, mail.from)
 Prop.query(:mails_per_hour, mail.from)
 ```
 
-The throttle scope can also be an array of values, e.g.:
+The throttle scope can also be an array of values:
 
 ```ruby
 Prop.throttle!(:mails_per_hour, [ account.id, mail.from ])
@@ -80,7 +77,10 @@ Prop.throttle!(:mails_per_hour, [ account.id, mail.from ])
 
 ## Error handling
 
-If the throttle! method gets called more than "threshold" times within "interval in seconds" for a given handle and key combination, Prop throws a `Prop::RateLimited` error which is a subclass of `StandardError`. This exception contains a "handle" reference and a "description" if specified during the configuration. The handle allows you to rescue `Prop::RateLimited` and differentiate action depending on the handle. For example, in Rails you can use this in e.g. `ApplicationController`:
+If the threshold for a given handle and key combination is exceeded, Prop throws a `Prop::RateLimited`. 
+This exception contains a "handle" reference and a "description" if specified during the configuration. 
+The handle allows you to rescue `Prop::RateLimited` and differentiate action depending on the handle. 
+For example, in Rails you can use this in e.g. `ApplicationController`:
 
 ```ruby
 rescue_from Prop::RateLimited do |e|
@@ -94,15 +94,22 @@ end
 
 ### Using the Middleware
 
-Prop ships with a built-in Rack middleware that you can use to do all the exception handling. When a `Prop::RateLimited` error is caught, it will build an HTTP [429 Too Many Requests](http://tools.ietf.org/html/draft-nottingham-http-new-status-02#section-4) response and set the following headers:
+Prop ships with a built-in Rack middleware that you can use to do all the exception handling. 
+When a `Prop::RateLimited` error is caught, it will build an HTTP 
+[429 Too Many Requests](http://tools.ietf.org/html/draft-nottingham-http-new-status-02#section-4) 
+response and set the following headers:
 
     Retry-After: 32
     Content-Type: text/plain
     Content-Length: 72
 
-Where `Retry-After` is the number of seconds the client has to wait before retrying this end point. The body of this response is whatever description Prop has configured for the throttle that got violated, or a default string if there's none configured.
+Where `Retry-After` is the number of seconds the client has to wait before retrying this end point. 
+The body of this response is whatever description Prop has configured for the throttle that got violated, 
+or a default string if there's none configured.
 
-If you wish to do manual error messaging in these cases, you can define an error handler in your Prop configuration. Here's how the default error handler looks - you use anything that responds to `.call` and takes the environment and a `RateLimited` instance as argument:
+If you wish to do manual error messaging in these cases, you can define an error handler in your Prop configuration. 
+Here's how the default error handler looks - you use anything that responds to `.call` and 
+takes the environment and a `RateLimited` instance as argument:
 
 ```ruby
 error_handler = Proc.new do |env, error|
@@ -112,7 +119,7 @@ error_handler = Proc.new do |env, error|
   [ 429, headers, [ body ]]
 end
 
-ActionController::Dispatcher.middleware.insert_before(ActionController::ParamsParser, :error_handler => error_handler)
+ActionController::Dispatcher.middleware.insert_before(ActionController::ParamsParser, error_handler: error_handler)
 ```
 
 An alternative to this, is to extend `Prop::Middleware` and override the `render_response(env, error)` method.
@@ -127,22 +134,23 @@ Prop.disabled do
 end
 ```
 
-## Threshold settings
+## Overriding threshold
 
 You can chose to override the threshold for a given key:
 
 ```ruby
-Prop.throttle!(:mails_per_hour, mail.from, :threshold => current_account.mail_throttle_threshold)
+Prop.throttle!(:mails_per_hour, mail.from, threshold: current_account.mail_throttle_threshold)
 ```
 
-When the threshold are invoked without argument, the key is nil and as such a scope of its own, i.e. these are equivalent:
+When `throttle` is invoked without argument, the key is nil and as such a scope of its own, i.e. these are equivalent:
 
 ```ruby
 Prop.throttle!(:mails_per_hour)
 Prop.throttle!(:mails_per_hour, nil)
 ```
 
-The default (and smallest possible) increment is 1, you can set that to any integer value using :increment which is handy for building time based throttles:
+The default (and smallest possible) increment is 1, you can set that to any integer value using 
+`:increment` which is handy for building time based throttles:
 
 ```ruby
 Prop.configure(:execute_time, threshold: 10, interval: 1.minute)
@@ -173,18 +181,21 @@ rescue Prop::RateLimited => e
   when :auth
     raise AuthFailure
   ...
-end 
+end
 ```
 
 ## Using Leaky Bucket Algorithm
 
-You can add two additional configurations: `:strategy` and `:burst_rate` to use the [leaky bucket algorithm](https://en.wikipedia.org/wiki/Leaky_bucket). Prop will handle the details after configured, and you don't have to specify `:strategy` again when using `throttle`, `throttle!` or any other methods.
+You can add two additional configurations: `:strategy` and `:burst_rate` to use the 
+[leaky bucket algorithm](https://en.wikipedia.org/wiki/Leaky_bucket). 
+Prop will handle the details after configured, and you don't have to specify `:strategy` 
+again when using `throttle`, `throttle!` or any other methods.
 
 ```ruby
 Prop.configure(:api_request, strategy: :leaky_bucket, burst_rate: 20, threshold: 5, interval: 1.minute)
 ```
 
-* `:threshold` value here would be the "leak rate" of leaky bucket algorithm. 
+* `:threshold` value here would be the "leak rate" of leaky bucket algorithm.
 
 
 ## License
@@ -196,4 +207,7 @@ You may obtain a copy of the License at
 
 http://www.apache.org/licenses/LICENSE-2.0
 
-Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
+Unless required by applicable law or agreed to in writing, 
+software distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
+See the License for the specific language governing permissions and limitations under the License.

data/lib/prop.rb

@@ -7,7 +7,7 @@ module Prop
   # Short hand for accessing Prop::Limiter methods
   class << self
     extend Forwardable
-    def_delegators :"Prop::Limiter", :read, :write, :configure, :configurations, :disabled, :before_throttle
+    def_delegators :"Prop::Limiter", :read, :write, :cache=, :configure, :configurations, :disabled, :before_throttle
     def_delegators :"Prop::Limiter", :throttle, :throttle!, :throttled?, :count, :query, :reset
   end
 end

data/lib/prop/interval_strategy.rb

@@ -6,20 +6,21 @@ module Prop
   class IntervalStrategy
     class << self
       def counter(cache_key, options)
-        Prop::Limiter.reader.call(cache_key).to_i
+        Prop::Limiter.cache.read(cache_key).to_i
       end
 
-      def increment(cache_key, options, counter)
+      def increment(cache_key, options)
         increment = options.fetch(:increment, 1)
-        Prop::Limiter.writer.call(cache_key, counter + increment)
+        cache = Prop::Limiter.cache
+        cache.increment(cache_key, increment) || (cache.write(cache_key, increment, raw: true) && increment) # WARNING: potential race condition
       end
 
       def reset(cache_key)
-        Prop::Limiter.writer.call(cache_key, 0)
+        Prop::Limiter.cache.write(cache_key, 0)
       end
 
-      def at_threshold?(counter, options)
-        counter >= options.fetch(:threshold)
+      def compare_threshold?(counter, operator, options)
+        counter.send operator, options.fetch(:threshold)
       end
 
       # Builds the expiring cache key
@@ -37,7 +38,7 @@ module Prop
       def threshold_reached(options)
         threshold = options.fetch(:threshold)
 
-        "#{options[:handle]} threshold of #{threshold} tries per #{options[:interval]}s exceeded for key '#{options[:key].inspect}', hash #{options[:cache_key]}"
+        "#{options[:handle]} threshold of #{threshold} tries per #{options[:interval]}s exceeded for key #{options[:key].inspect}, hash #{options[:cache_key]}"
       end
 
       def validate_options!(options)

data/lib/prop/leaky_bucket_strategy.rb

@@ -6,34 +6,31 @@ require 'prop/interval_strategy'
 module Prop
   class LeakyBucketStrategy
     class << self
-      def update_bucket(cache_key, interval, leak_rate)
-        bucket = Prop::Limiter.reader.call(cache_key) || default_bucket
+      def counter(cache_key, options)
+        bucket = Prop::Limiter.cache.read(cache_key) || default_bucket
         now = Time.now.to_i
-        leak_amount = (now - bucket[:last_updated]) / interval * leak_rate
+        leak_amount = (now - bucket.fetch(:last_updated)) / options.fetch(:interval) * options.fetch(:threshold)
 
-        bucket[:bucket] = [bucket[:bucket] - leak_amount, 0].max
+        bucket[:bucket] = [bucket.fetch(:bucket) - leak_amount, 0].max
         bucket[:last_updated] = now
-
-        Prop::Limiter.writer.call(cache_key, bucket)
         bucket
       end
 
-      def counter(cache_key, options)
-        update_bucket(cache_key, options[:interval], options[:threshold]).merge(burst_rate: options[:burst_rate])
-      end
-
-      def increment(cache_key, options, counter)
-        increment = options.fetch(:increment, 1)
-        bucket = { :bucket => counter[:bucket].to_i + increment, :last_updated => Time.now.to_i }
-        Prop::Limiter.writer.call(cache_key, bucket)
+      # WARNING: race condition
+      # this increment is not atomic, so it might miss counts when used frequently
+      def increment(cache_key, options)
+        counter = counter(cache_key, options)
+        counter[:bucket] += options.fetch(:increment, 1)
+        Prop::Limiter.cache.write(cache_key, counter)
+        counter
       end
 
       def reset(cache_key)
-        Prop::Limiter.writer.call(cache_key, default_bucket)
+        Prop::Limiter.cache.write(cache_key, default_bucket)
       end
 
-      def at_threshold?(counter, options)
-        counter[:bucket].to_i >= options.fetch(:burst_rate)
+      def compare_threshold?(counter, operator, options)
+        counter.fetch(:bucket).to_i.send operator, options.fetch(:burst_rate)
       end
 
       def build(options)
@@ -45,15 +42,11 @@ module Prop
         "prop/leaky_bucket/#{Digest::MD5.hexdigest(cache_key)}"
       end
 
-      def default_bucket
-        { :bucket => 0, :last_updated => 0 }
-      end
-
       def threshold_reached(options)
         burst_rate = options.fetch(:burst_rate)
         threshold  = options.fetch(:threshold)
 
-        "#{options[:handle]} threshold of #{threshold} tries per #{options[:interval]}s and burst rate #{burst_rate} tries exceeded for key '#{options[:key].inspect}', hash #{options[:cache_key]}"
+        "#{options[:handle]} threshold of #{threshold} tries per #{options[:interval]}s and burst rate #{burst_rate} tries exceeded for key #{options[:key].inspect}, hash #{options[:cache_key]}"
       end
 
       def validate_options!(options)
@@ -63,6 +56,12 @@ module Prop
           raise ArgumentError.new(":burst_rate must be an Integer and larger than :threshold")
         end
       end
+
+      private
+
+      def default_bucket
+        { bucket: 0, last_updated: 0 }
+      end
     end
   end
 end

data/lib/prop/limiter.rb

@@ -8,14 +8,22 @@ module Prop
   class Limiter
 
     class << self
-      attr_accessor :handles, :reader, :writer, :before_throttle_callback
+      attr_accessor :handles, :before_throttle_callback, :cache
 
       def read(&blk)
-        self.reader = blk
+        raise "Use .cache = "
       end
 
       def write(&blk)
-        self.writer = blk
+        raise "Use .cache = "
+      end
+
+      def cache=(cache)
+        [:read, :write, :increment].each do |method|
+          next if cache.respond_to?(method)
+          raise ArgumentError, "Cache needs to respond to #{method}"
+        end
+        @cache = cache
       end
 
       def before_throttle(&blk)
@@ -25,12 +33,12 @@ module Prop
       # Public: Registers a handle for rate limiting
       #
       # handle   - the name of the handle you wish to use in your code, e.g. :login_attempt
-      # defaults - the settings for this handle, e.g. { :threshold => 5, :interval => 5.minutes }
+      # defaults - the settings for this handle, e.g. { threshold: 5, interval: 5.minutes }
       #
       # Raises Prop::RateLimited if the number if the threshold for this handle has been reached
       def configure(handle, defaults)
-        raise RuntimeError.new("Invalid threshold setting") unless defaults[:threshold].to_i > 0
-        raise RuntimeError.new("Invalid interval setting")  unless defaults[:interval].to_i > 0
+        raise ArgumentError.new("Invalid threshold setting") unless defaults[:threshold].to_i > 0
+        raise ArgumentError.new("Invalid interval setting")  unless defaults[:interval].to_i > 0
 
         self.handles ||= {}
         self.handles[handle] = defaults
@@ -55,23 +63,19 @@ module Prop
       #
       # Returns true if the threshold for this handle has been reached, else returns false
       def throttle(handle, key = nil, options = {})
-        options, cache_key = prepare(handle, key, options)
-        counter = @strategy.counter(cache_key, options)
+        return false if disabled?
 
-        unless disabled?
-          if @strategy.at_threshold?(counter, options)
-            unless before_throttle_callback.nil?
-              before_throttle_callback.call(handle, key, options[:threshold], options[:interval])
-            end
-
-            true
-          else
-            @strategy.increment(cache_key, options, counter)
+        options, cache_key = prepare(handle, key, options)
+        counter = @strategy.increment(cache_key, options)
 
-            yield if block_given?
+        if @strategy.compare_threshold?(counter, :>, options)
+          before_throttle_callback &&
+            before_throttle_callback.call(handle, key, options[:threshold], options[:interval])
 
-            false
-          end
+          true
+        else
+          yield if block_given?
+          false
         end
       end
 
@@ -82,19 +86,19 @@ module Prop
       # options - request specific overrides to the defaults configured for this handle
       # (optional) a block of code that this throttle is guarding
       #
-      # Raises Prop::RateLimited if the number if the threshold for this handle has been reached
+      # Raises Prop::RateLimited if the threshold for this handle has been reached
       # Returns the value of the block if given a such, otherwise the current count of the throttle
       def throttle!(handle, key = nil, options = {})
         options, cache_key = prepare(handle, key, options)
 
         if throttle(handle, key, options)
-          raise Prop::RateLimited.new(options.merge(:cache_key => cache_key, :handle => handle))
+          raise Prop::RateLimited.new(options.merge(cache_key: cache_key, handle: handle))
         end
 
         block_given? ? yield : @strategy.counter(cache_key, options)
       end
 
-      # Public: Allows to query whether the given handle/key combination is currently throttled
+      # Public: Is the given handle/key combination currently throttled ?
       #
       # handle   - the throttle identifier
       # key      - the associated key
@@ -103,7 +107,7 @@ module Prop
       def throttled?(handle, key = nil, options = {})
         options, cache_key = prepare(handle, key, options)
         counter = @strategy.counter(cache_key, options)
-        @strategy.at_threshold?(counter, options)
+        @strategy.compare_threshold?(counter, :>=, options)
       end
 
       # Public: Resets a specific throttle
@@ -113,7 +117,7 @@ module Prop
       #
       # Returns nothing
       def reset(handle, key = nil, options = {})
-        options, cache_key = prepare(handle, key, options)
+        _options, cache_key = prepare(handle, key, options)
         @strategy.reset(cache_key)
       end
 
@@ -141,14 +145,15 @@ module Prop
       end
 
       def prepare(handle, key, params)
-        raise RuntimeError.new("No such handle configured: #{handle.inspect}") unless (handles || {}).key?(handle)
+        unless defaults = handles[handle]
+          raise KeyError.new("No such handle configured: #{handle.inspect}")
+        end
 
-        defaults  = handles[handle]
-        options   = Prop::Options.build(:key => key, :params => params, :defaults => defaults)
+        options = Prop::Options.build(key: key, params: params, defaults: defaults)
 
         @strategy = options.fetch(:strategy)
 
-        cache_key = @strategy.build(:key => key, :handle => handle, :interval => options[:interval])
+        cache_key = @strategy.build(key: key, handle: handle, interval: options[:interval])
 
         [ options, cache_key ]
       end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: prop
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 2.0.0
 platform: ruby
 authors:
 - Morten Primdahl
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-07-09 00:00:00.000000000 Z
+date: 2015-10-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -25,7 +25,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: bundler
+  name: maxitest
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -39,7 +39,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: minitest
+  name: mocha
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -53,7 +53,21 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: mocha
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: bump
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -102,7 +116,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.4.7
+rubygems_version: 2.4.5.1
 signing_key: 
 specification_version: 4
 summary: Gem for implementing rate limits.