glutton_ratelimit 0.1.0

data/.document

@@ -0,0 +1,5 @@
+README.rdoc
+lib/**/*.rb
+bin/*
+features/**/*.feature
+LICENSE

data/.gitignore

@@ -0,0 +1,21 @@
+## MAC OS
+.DS_Store
+
+## TEXTMATE
+*.tmproj
+tmtags
+
+## EMACS
+*~
+\#*
+.\#*
+
+## VIM
+*.swp
+
+## PROJECT::GENERAL
+coverage
+rdoc
+pkg
+
+## PROJECT::SPECIFIC

data/LICENSE

@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <http://unlicense.org/>

data/README.rdoc

@@ -0,0 +1,125 @@
+= glutton_ratelimit
+
+A Ruby library for limiting the number of times a method can be invoked within a specified time period.
+
+The rate-limiting is simple if somewhat naïve. Throttled methods are blocked using sleep.
+
+== Use Cases
+
+You might wish to use this library to throttle code that:
+
+* Accesses a 3rd-party API that imposes a maximum rate-limit. 
+* Scrapes data from any website where rapid repeated access is banned.
+
+For example, EchoNest API requests need to be limited to 120 every minute.
+
+== Types of Limiting
+
+There are two types of rate limiting provided by this library:
+
+* Bursty Token Bucket Limiting (GluttonRatelimit::BurstyTokenBucket)
+* Average Throttle Limiting (GluttonRatelimit::AveragedThrottle)
+
+=== Bursty Token Bucket
+
+If executions are limited to n times per m seconds then:
+
+1. n executions will be allowed immediately.
+2. Before the next execution the process will sleep for the remainder of the m second time period.
+3. The process is repeated.
+
+The amount of time slept in step 2 will depend on how long the n executions took in step 1.
+
+=== Average Throttle
+
+If executions are limited to n times per m seconds then:
+
+1. The first execution will occur immediately.
+2. Before each of the remaining n-1 executions the process will attempt to sleep so that the executions are evenly spaced within the m second time period.
+3. The process is repeated.
+
+The amount of time slept before each execution will depend on the time period m and the average elapsed time of each execution.
+
+== Instance Method Limiting Example
+    
+The rate_limit method can be used at the end of a class definition to limit specific instance methods. 
+        
+    class LimitInstanceMethod
+      # The class must be extended to permit limiting.
+      extend GluttonRatelimit
+        
+      def initialize
+        @start = Time.now
+      end
+      
+      def limit_me
+        puts "#{Time.now - @start}"
+      end
+      
+      # Throttle the limit_me method to six executions every six seconds.
+      rate_limit :limit_me, 6, 6
+    end
+    
+    t = LimitInstanceMethod.new
+    
+    10.times { t.limit_me }
+
+When using the rate_limit method the limiting defaults to GluttonRatelimit::AveragedThrottle. Token Bucket limiting can also be specified:
+
+    rate_limite :limit_me, 6, 6, GluttonRatelimit::BurstyTokenBucket
+
+== Simple Manual Limiting Example
+
+Chunks of code can also be manually throttled using the wait method of a specific GluttonRatelimit object.
+
+    # Maximum of twelve executions every five seconds.
+    rl = GluttonRatelimit::BurstyTokenBucket.new 12, 5
+    
+    25.times do
+      # BurstyTokenBucket will allow for a full burst of executions followed by a pause.
+      rl.wait
+      # Simulating a constant-time task:
+      sleep 0.1
+    end
+    
+    # Maximum of six executions every six seconds.
+    rl = GluttonRatelimit::AveragedThrottle.new 6, 6
+    13.times do
+      # AverageThrottle will attempt to evenly space executions within the allowed period.
+      rl.wait
+      # Simulating a 0 to 1 second random-time task:
+      sleep rand
+    end
+    
+== More Examples
+
+More examples can be found within the examples folder.
+
+== Warnings
+
+Before using the library you should be aware of the following warnings.
+
+=== Short Tasks and AveragedThrottle
+
+The inaccuracy of Ruby's sleep method may cause timing issues with the AveragedThrottle limiting. Specifically, the limiting accuracy may be slightly-off when trying to rate-limit quick methods (sub-millisecond tasks).
+
+See: http://codeidol.com/other/rubyckbk/Date-and-Time/Waiting-a-Certain-Amount-of-Time
+
+It is recommend that you use the BurstyTokenBucket limiting when throttling very short tasks.
+
+=== Naive Throttling
+
+As mentioned above, throttling is accomplish by blocking script execution using sleep. There is no support for dropping or queuing throttled method invocations.
+
+This library is not thread safe.
+
+== Thanks
+
+Some of the algorithms were inspired by these discussions:
+
+* http://stackoverflow.com/questions/667508/whats-a-good-rate-limiting-algorithm
+* http://stackoverflow.com/questions/1407113/throttling-method-calls-to-m-requests-in-n-seconds
+
+== License
+
+This is free and unencumbered software released into the public domain. See LICENSE for details.

data/Rakefile

@@ -0,0 +1,52 @@
+require 'rubygems'
+require 'rake'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "glutton_ratelimit"
+    gem.summary = %Q{Simple Ruby library for self-imposed rater-limiting.}
+    gem.description = %Q{A Ruby library for limiting the number of times a method can be invoked within a specified time period.}
+    gem.email = "stungeye@gmail.com"
+    gem.homepage = "http://github.com/stungeye/glutton_ratelimit"
+    gem.authors = ["Wally Glutton"]
+    # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
+  end
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
+end
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+end
+
+begin
+  require 'rcov/rcovtask'
+  Rcov::RcovTask.new do |test|
+    test.libs << 'test'
+    test.pattern = 'test/**/test_*.rb'
+    test.verbose = true
+  end
+rescue LoadError
+  task :rcov do
+    abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
+  end
+end
+
+task :test => :check_dependencies
+
+task :default => :test
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  version = File.exist?('VERSION') ? File.read('VERSION') : ""
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "glutton_ratelimit #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end

data/VERSION

@@ -0,0 +1 @@
+0.1.0

data/examples/limit_instance_methods.rb

@@ -0,0 +1,34 @@
+require 'rubygems'
+require 'glutton_ratelimit'
+
+class LimitInstanceMethods
+  extend GluttonRatelimit
+    
+  def initialize
+    @start = Time.now
+  end
+  
+  def limit_me
+    puts "#{Time.now - @start}"
+    sleep 0.001
+  end
+  
+  def cap_me
+    puts "#{Time.now - @start}"
+    sleep 0.001
+  end
+  
+  rate_limit :limit_me, 6, 6
+  rate_limit :cap_me, 6, 6, GluttonRatelimit::BurstyTokenBucket
+end
+
+t = LimitInstanceMethods.new
+
+puts "Six requests every 6 seconds (Averaged): "
+13.times { t.limit_me }
+puts "Six requests every 6 seconds (Bursty): "
+13.times { t.cap_me }
+
+# In both cases:
+# The 7th execution should occur after 6 seconds after the first.
+# The 13th execution should occur after 12 seconds after the first.

data/examples/simple_manual.rb

@@ -0,0 +1,30 @@
+require 'rubygems'
+require 'glutton_ratelimit'
+
+puts "Maximum of 12 executions every 5 seconds (Bursty):"
+rl = GluttonRatelimit::BurstyTokenBucket.new 12, 5
+
+start = Time.now
+25.times do |n|
+  # BurstyTokenBucket will allow for a full burst of executions followed by a pause.
+  rl.wait
+  puts "#{n+1} - #{Time.now - start}"
+  # Simulating a constant-time task:
+  sleep 0.1
+end
+
+# The 25th execution should occur after 10 seconds has elapsed.
+
+puts "Maximum of 3 executions every 3 seconds (Averaged):"
+rl = GluttonRatelimit::AveragedThrottle.new 3, 3
+
+start = Time.now
+7.times do |n|
+  # AverageThrottle will attempt to evenly space executions within the allowed period.
+  rl.wait
+  puts "#{n+1} - #{Time.now - start}"
+  # Simulating a 0 to 1 second random-time task:
+  sleep rand
+end
+
+# The 7th execution should occur after 6 seconds has elapsed.

data/glutton_ratelimit.gemspec

@@ -0,0 +1,64 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE DIRECTLY
+# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{glutton_ratelimit}
+  s.version = "0.1.0"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Wally Glutton"]
+  s.date = %q{2010-06-10}
+  s.description = %q{A Ruby library for limiting the number of times a method can be invoked within a specified time period.}
+  s.email = %q{stungeye@gmail.com}
+  s.extra_rdoc_files = [
+    "LICENSE",
+     "README.rdoc"
+  ]
+  s.files = [
+    ".document",
+     ".gitignore",
+     "LICENSE",
+     "README.rdoc",
+     "Rakefile",
+     "VERSION",
+     "examples/limit_instance_methods.rb",
+     "examples/simple_manual.rb",
+     "glutton_ratelimit.gemspec",
+     "lib/glutton_ratelimit.rb",
+     "lib/glutton_ratelimit/averaged_throttle.rb",
+     "lib/glutton_ratelimit/bursty_ring_buffer.rb",
+     "lib/glutton_ratelimit/bursty_token_bucket.rb",
+     "test/helper.rb",
+     "test/test_glutton_ratelimit_averaged_throttle.rb",
+     "test/test_glutton_ratelimit_bursty_ring_buffer.rb",
+     "test/test_glutton_ratelimit_bursty_token_bucket.rb",
+     "test/testing_module.rb"
+  ]
+  s.homepage = %q{http://github.com/stungeye/glutton_ratelimit}
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.6}
+  s.summary = %q{Simple Ruby library for self-imposed rater-limiting.}
+  s.test_files = [
+    "test/test_glutton_ratelimit_averaged_throttle.rb",
+     "test/helper.rb",
+     "test/test_glutton_ratelimit_bursty_ring_buffer.rb",
+     "test/testing_module.rb",
+     "test/test_glutton_ratelimit_bursty_token_bucket.rb",
+     "examples/limit_instance_methods.rb",
+     "examples/simple_manual.rb"
+  ]
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+    else
+    end
+  else
+  end
+end
+

data/lib/glutton_ratelimit.rb

@@ -0,0 +1,29 @@
+module GluttonRatelimit
+  
+  def rate_limit symbol, executions, time_period, rl_class = AveragedThrottle
+    rl = rl_class.new executions, time_period
+    old_symbol = "#{symbol}_old".to_sym
+    alias_method old_symbol, symbol
+    define_method symbol do |*args|
+      rl.wait
+      self.send old_symbol, *args
+    end
+  end
+  
+  private
+  # All the other classes extend this parent and are therefore
+  # constructed in the same manner.
+  class ParentLimiter
+    attr_reader :executions
+    
+    def initialize executions, time_period
+      @executions = executions
+      @time_period = time_period
+    end
+  end
+end
+
+dir = File.expand_path(File.dirname(__FILE__))
+require File.join(dir, "glutton_ratelimit", "bursty_ring_buffer")
+require File.join(dir, "glutton_ratelimit", "bursty_token_bucket")
+require File.join(dir, "glutton_ratelimit", "averaged_throttle")

data/lib/glutton_ratelimit/averaged_throttle.rb

@@ -0,0 +1,44 @@
+module GluttonRatelimit
+  
+  class AveragedThrottle < ParentLimiter
+    
+    def reset_bucket
+      @before_previous_execution = Time.now if @before_previous_execution.nil?
+      @oldest_timestamp = Time.now
+      @tokens = @executions
+      @total_task_time = 0
+    end
+    
+    def executed_this_period
+      @executions - @tokens
+    end
+    
+    def average_task_time
+      @total_task_time.to_f / executed_this_period
+    end
+    
+    def wait
+      reset_bucket if @tokens.nil?
+      
+      now = Time.now
+      delta_since_previous = now - @before_previous_execution
+      @total_task_time += delta_since_previous
+      remaining_time = @time_period - (now - @oldest_timestamp)
+      
+      if @tokens.zero?
+        sleep(remaining_time) if remaining_time > 0
+        reset_bucket
+      elsif executed_this_period != 0
+        throttle = (remaining_time.to_f + delta_since_previous) / (@tokens+1) - average_task_time
+        sleep throttle if throttle > 0
+      end
+      
+      @tokens -= 1
+      @before_previous_execution = Time.now
+    end
+    
+  end
+
+end
+
+

data/lib/glutton_ratelimit/bursty_ring_buffer.rb

@@ -0,0 +1,24 @@
+module GluttonRatelimit
+  
+  class BurstyRingBuffer < ParentLimiter
+    
+    def oldest_timestamp
+      @timestamps = Array.new executions, (Time.now - @time_period) if @timestamps.nil?
+      @timestamps[0]
+    end
+    
+    def current_timestamp= new_stamp
+      @timestamps.push(new_stamp).shift
+    end
+      
+    def wait
+      delta = Time.now - oldest_timestamp
+      sleep(@time_period - delta) if delta < @time_period
+      self.current_timestamp = Time.now
+    end
+    
+  end
+
+end
+
+

data/lib/glutton_ratelimit/bursty_token_bucket.rb

@@ -0,0 +1,26 @@
+module GluttonRatelimit
+  
+  class BurstyTokenBucket < ParentLimiter
+    
+    def reset_bucket
+      @oldest_timestamp = Time.now
+      @tokens = @executions
+    end
+    
+    def wait
+      reset_bucket if @tokens.nil?
+      
+      if @tokens.zero?
+        delta = Time.now - @oldest_timestamp
+        sleep(@time_period - delta) if delta < @time_period
+        reset_bucket
+      end
+      
+      @tokens -= 1
+    end
+    
+  end
+
+end
+
+

data/test/helper.rb

@@ -0,0 +1,21 @@
+require 'rubygems'
+require 'test/unit'
+
+$LOAD_PATH.unshift(File.dirname(__FILE__))
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+require 'glutton_ratelimit'
+
+class Test::Unit::TestCase
+end
+
+def timed_run ratelimiter, passes = 1
+  before_last_invocation = 0
+  start_time = Time.now
+  (passes * ratelimiter.executions + 1).times do |n|
+    ratelimiter.wait
+    before_last_invocation = Time.now
+    yield
+  end 
+  before_last_invocation - start_time
+end
+

data/test/test_glutton_ratelimit_averaged_throttle.rb

@@ -0,0 +1,15 @@
+require 'helper'
+require 'testing_module.rb'
+
+class TestGluttonRatelimitAveragedThrottle < Test::Unit::TestCase
+  include TestingModule
+  
+  def setup
+    @testClass = GluttonRatelimit::AveragedThrottle
+  end
+  
+  def test_object_creations
+    rl = @testClass.new 1, 1
+    assert_instance_of GluttonRatelimit::AveragedThrottle, rl
+  end
+end

data/test/test_glutton_ratelimit_bursty_ring_buffer.rb

@@ -0,0 +1,15 @@
+require 'helper'
+require 'testing_module.rb'
+
+class TestGluttonRatelimitBurstyRingBuffer < Test::Unit::TestCase
+  include TestingModule
+  
+  def setup
+    @testClass = GluttonRatelimit::BurstyRingBuffer
+  end
+  
+  def test_object_creations
+    rl = @testClass.new 1, 1
+    assert_instance_of GluttonRatelimit::BurstyRingBuffer, rl
+  end
+end

data/test/test_glutton_ratelimit_bursty_token_bucket.rb

@@ -0,0 +1,15 @@
+require 'helper'
+require 'testing_module.rb'
+
+class TestGluttonRatelimitBurtyTokenBucket < Test::Unit::TestCase
+  include TestingModule
+  
+  def setup
+    @testClass = GluttonRatelimit::BurstyTokenBucket
+  end
+  
+  def test_object_creations
+    rl = @testClass.new 1, 1
+    assert_instance_of GluttonRatelimit::BurstyTokenBucket, rl
+  end
+end

data/test/testing_module.rb

@@ -0,0 +1,67 @@
+module TestingModule
+  # This module assumes use as a mixin within
+  # a test class which defines @testClass.
+  
+  def test_120_tasks_every_second_with_ms_task
+    min_time = 1
+    rl = @testClass.new 120, min_time
+    delta = timed_run(rl) { sleep 0.001 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time
+  end
+  
+  def test_120_tasks_every_quarter_second_with_ms_task
+    min_time = 0.25
+    rl = @testClass.new 120, min_time
+    delta = timed_run(rl) { sleep 0.001 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time
+  end
+  
+  def test_120_tasks_every_second_with_cenisecond_task
+    min_time = 1
+    rl = @testClass.new 120, min_time
+    delta = timed_run(rl) { sleep 0.01 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time
+  end
+  
+  def test_50_tasks_every_second_with_fast_task
+    min_time = 1
+    rl = @testClass.new 50, min_time
+    delta = timed_run(rl) { sleep 0 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time
+  end
+  
+  def test_1_task_every_1_seconds_with_1_second_task
+    min_time = 1
+    rl = @testClass.new 1, min_time
+    delta = timed_run(rl) { sleep 1 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time
+  end
+  
+  def test_5_tasks_every_quarter_second_four_runs_with_ms_task
+    min_time = 0.25
+    rl = @testClass.new 1, min_time
+    delta = timed_run(rl, 4) { sleep 0.001 }
+    #puts "#{delta} >? #{min_time}"
+    assert delta > min_time * 4
+  end
+  
+  def test_120_tasks_every_half_second_two_runs_with_no_task
+    min_time = 0.5
+    rl = @testClass.new 120, min_time
+    delta = timed_run(rl, 2) { sleep 0 }
+    assert delta > min_time * 2
+  end
+  
+  def test_10_tasks_every_1_seconds_two_runs_with_short_rnd_task
+    min_time = 1
+    rl = @testClass.new 10, min_time
+    delta = timed_run(rl, 2) { sleep(0.2 * rand) }
+    assert delta > min_time * 2
+  end
+  
+end

metadata

@@ -0,0 +1,86 @@
+--- !ruby/object:Gem::Specification 
+name: glutton_ratelimit
+version: !ruby/object:Gem::Version 
+  prerelease: false
+  segments: 
+  - 0
+  - 1
+  - 0
+  version: 0.1.0
+platform: ruby
+authors: 
+- Wally Glutton
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2010-06-10 00:00:00 -05:00
+default_executable: 
+dependencies: []
+
+description: A Ruby library for limiting the number of times a method can be invoked within a specified time period.
+email: stungeye@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE
+- README.rdoc
+files: 
+- .document
+- .gitignore
+- LICENSE
+- README.rdoc
+- Rakefile
+- VERSION
+- examples/limit_instance_methods.rb
+- examples/simple_manual.rb
+- glutton_ratelimit.gemspec
+- lib/glutton_ratelimit.rb
+- lib/glutton_ratelimit/averaged_throttle.rb
+- lib/glutton_ratelimit/bursty_ring_buffer.rb
+- lib/glutton_ratelimit/bursty_token_bucket.rb
+- test/helper.rb
+- test/test_glutton_ratelimit_averaged_throttle.rb
+- test/test_glutton_ratelimit_bursty_ring_buffer.rb
+- test/test_glutton_ratelimit_bursty_token_bucket.rb
+- test/testing_module.rb
+has_rdoc: true
+homepage: http://github.com/stungeye/glutton_ratelimit
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.6
+signing_key: 
+specification_version: 3
+summary: Simple Ruby library for self-imposed rater-limiting.
+test_files: 
+- test/test_glutton_ratelimit_averaged_throttle.rb
+- test/helper.rb
+- test/test_glutton_ratelimit_bursty_ring_buffer.rb
+- test/testing_module.rb
+- test/test_glutton_ratelimit_bursty_token_bucket.rb
+- examples/limit_instance_methods.rb
+- examples/simple_manual.rb