ihasa 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 54a2fc759ca9910bc5e2153bde55d374dc3ef2bb
4
- data.tar.gz: 001131036b26d03947150db2f2ed61e63f2d8a1a
3
+ metadata.gz: bb33afd1cc7c68a0cbcbaba333ded91ad08eca2b
4
+ data.tar.gz: 63972017da74a9b3f2ee8de0e5644ef01cfdd6aa
5
5
  SHA512:
6
- metadata.gz: cfe56b58bd987f45f5b2a0214871ae637214f8d4a590e6b2ad2563c87ca0619c0dfb219896c59b5e0f1774c5577a4588f406827978ae6f52ee963eb59d14e68b
7
- data.tar.gz: 9341610417ed54fbd1c0b488031ff83140972e82fc4d9ba029fd92f184e143810b5fefd9375880ad4441ef1f29eca8d68c625db8e4f9d9fd3155e5846a432e14
6
+ metadata.gz: 1a32eabe6c7d6c0ff132dbb48e385ba5f8c8c928feacdb1c7ca0fcb313d918da4f8b1de65dc9ef6f4adeeea20cbeebdab2f9ff8fe20fe06f9fe7b89b3c936f39
7
+ data.tar.gz: 50ff73af381171104c20922562de2fcaa2f8273e0ba3bf638d656c18b58502b1870858784556b7a769dfdf326662c8c4c377ebc40f8c38077bef917f6882252c
@@ -0,0 +1 @@
1
+ 2.3.1
@@ -0,0 +1,3 @@
1
+ script: bundle exec rspec
2
+ services:
3
+ - redis-server
data/README.md CHANGED
@@ -1,16 +1,21 @@
1
- No more pounding on your APIs !
1
+ # Ihasa
2
+
3
+ [![Build Status](https://travis-ci.org/bankair/ihasa.svg?branch=master)](https://travis-ci.org/bankair/ihasa)
4
+
5
+ _No more pounding on your APIs !_
2
6
 
3
7
  Ihasa is a ruby implementation of the [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) backed-up by Redis.
4
8
 
5
9
  It provides a way to share your rate/burst limit across multiple servers, as well as a simple interface.
6
10
 
7
- *Why use Ihasa?*
11
+ **Why use Ihasa?**
8
12
 
9
- 1. It's easy to use ([go check the usage section](#usage)
13
+ 1. It's easy to use ([go check the usage section](#usage))
10
14
  2. It supports rate AND burst
11
15
  3. It does not reset all rate limit consumption each new second/minute/hour
12
16
  4. It has [namespaces](#namespaces)
13
17
 
18
+ This README file contains the following sections:
14
19
 
15
20
  - [Installation](#installation)
16
21
  - [Usage](#usage)
@@ -25,7 +30,7 @@ Installation is standard:
25
30
  $ gem install ihasa
26
31
  ```
27
32
 
28
- You can as well include it in you `Gemfile`:
33
+ You can include it in your `Gemfile` as well:
29
34
 
30
35
  ```
31
36
  gem 'ihasa', require: false
@@ -39,7 +44,7 @@ Be sure to require Ihasa:
39
44
  require 'ihasa'
40
45
  ```
41
46
 
42
- To create a new bucket accepting 5 requests per second with an allowed burst of 10
47
+ To create a new bucket that accepts 5 requests per second with an allowed burst of 10
43
48
  requests per second (the default values), use the `Ihasa.bucket` method:
44
49
 
45
50
  ```ruby
@@ -50,8 +55,8 @@ Please note that the default redis connection is built from the `REDIS_URL`
50
55
  environment variable, or use the default constructor of redis-rb
51
56
  (`redis://localhost:6379`).
52
57
 
53
- Now, you can use your token bucket to check if an incoming request can be handled
54
- or must be turned down:
58
+ Now, you can use your token bucket to check if an incoming request can be handled,
59
+ or must be declined:
55
60
 
56
61
  ```ruby
57
62
  def process(request)
@@ -64,22 +69,24 @@ def process(request)
64
69
  end
65
70
  ```
66
71
 
67
- Please note that there is also a `Ihasa::Bucket#accept?!` method that raise a
68
- `Ihasa::Bucket::EmptyBucket` errors if the limit is violated.
72
+ Please note that there is also a `Ihasa::Bucket#accept?!` method that raises an
73
+ `Ihasa::Bucket::EmptyBucket` error if the limit has already been reached.
69
74
 
70
75
  ### Advanced
71
76
 
72
77
  In this section, you will find some details on the available configuration options of the Ihasa::Bucket
73
- class, as well a some advices to run lots of Bucket simultaneously.
78
+ class, as well as advice on how to run many Buckets simultaneously.
74
79
 
75
- #### Using a lots of different buckets
80
+ #### Using multiple buckets
76
81
 
77
- If you want to enforce rate limits by customer, you have to create as many buckets as customers you have.
78
- Which can be a lot if you are successful ;).
82
+ If you want to enforce per-customer rate limits, you must create as many buckets as you have customers.
83
+ Which can be quite a few if you are successful ;).
79
84
 
80
- To have a lots of buckets in parallel and to avoid resetting your redis namespaces too often, I suggest you
81
- no longer use the `Ihasa.bucket` method. Instead, you should back-up your buckets with activerecord models
82
- (for example) and initialize them in a callback runned after the models creation.
85
+ To have many buckets in parallel, and to avoid resetting your redis namespaces too often, I suggest you
86
+ no longer use the `Ihasa.bucket` method. Instead, you should back up your buckets with activerecord models
87
+ (for example) and initialize them in an after-creation model callback.
88
+
89
+ To help you with that, we added the `save` and `delete` instance methods to the `Ihasa::Bucket` class.
83
90
 
84
91
  Example:
85
92
 
@@ -91,10 +98,14 @@ Example:
91
98
  @implementation ||= Ihasa::Bucket.new(rate, burst, prefix, $redis)
92
99
  end
93
100
 
94
- # The Ihasa::Bucket#initialize_redis_namespace set the relevant
101
+ # The Ihasa::Bucket#save set the relevant
95
102
  # keys in your redis instance to have a working bucket. Do it
96
103
  # only when you create or update your bucket's configuration.
97
- after_save { implementation.initialize_redis_namespace }
104
+ after_save { implementation.save }
105
+
106
+ # The Ihasa::Bucket#delete remove the variables stored into
107
+ # the redis instance.
108
+ after_destroy { implementation.delete }
98
109
 
99
110
  delegate :accept?, to: :implementation
100
111
  end
@@ -124,9 +135,9 @@ Example:
124
135
  end
125
136
  ```
126
137
 
127
- #### Configuring rate and burst limit
138
+ #### Configuring rate limit and burst limit
128
139
 
129
- Two configuration options exists to configure the rate and burst limits:
140
+ You can configure both the rate limit and burst limit:
130
141
 
131
142
  ```ruby
132
143
  bucket = Ihasa.bucket(rate: 20, burst: 100)
@@ -134,7 +145,7 @@ bucket = Ihasa.bucket(rate: 20, burst: 100)
134
145
 
135
146
  #### Namespaces
136
147
 
137
- You can have as many bucket as you want on the same redis instance, as long as you
148
+ You can have as many buckets as you want on the same redis instance, as long as you
138
149
  configure different namespace for each of them.
139
150
 
140
151
  Here is an example of using two different buckets for reading and writing to data:
@@ -162,11 +173,11 @@ class Controller < ActionController::Base
162
173
 
163
174
  #### Redis
164
175
 
165
- By default, all new buckets use the redis instance hosted at localhost:6379. There is
166
- however two way to configure the used redis instance:
176
+ By default, all new buckets use the redis instance hosted at localhost:6379. You can
177
+ override this default like so:
167
178
 
168
179
  1. Override the `REDIS_URL` env variable. All new buckets will use that instance
169
- 2. Override the redis url on a bucket creation basis like follow:
180
+ 2. Override the redis url on a bucket creation basis like this:
170
181
 
171
182
  ```ruby
172
183
  Ihasa.bucket(redis: Redis.new(url: 'redis://fancy_host:6379'))
@@ -174,8 +185,8 @@ Ihasa.bucket(redis: Redis.new(url: 'redis://fancy_host:6379'))
174
185
 
175
186
  ## Example
176
187
 
177
- This is an example of a rack middleware accepting 20 requests per seconds, and
178
- tolerating burst up to 100 requests per second:
188
+ This is an example of a rack middleware that accepts 20 requests per seconds, and
189
+ allows bursts up to 100 requests per second:
179
190
 
180
191
  ```ruby
181
192
  class RateLimiter
@@ -7,7 +7,7 @@ Gem::Specification.new do |s|
7
7
  s.name = 'ihasa'
8
8
  s.version = Ihasa::Version::STRING
9
9
  s.platform = Gem::Platform::RUBY
10
- s.required_ruby_version = '>= 1.9.3'
10
+ s.required_ruby_version = '>= 2.0.0'
11
11
  s.authors = ['Alexandre Ignjatovic']
12
12
  s.description = <<-EOF
13
13
  A Redis-backed rate limiter written in ruby and
@@ -1,9 +1,14 @@
1
1
  require 'redis'
2
2
  require 'ihasa/version'
3
- require 'ihasa/bucket'
4
3
 
5
4
  # Ihasa module. Root of the Ihasa::Bucket class
6
5
  module Ihasa
6
+ NOK = 0
7
+ OK = 1
8
+ OPTIONS = %i(rate burst last allowance).freeze
9
+
10
+ require 'ihasa/bucket'
11
+
7
12
  module_function
8
13
 
9
14
  def default_redis
@@ -16,6 +21,6 @@ module Ihasa
16
21
 
17
22
  DEFAULT_PREFIX = 'IHAB'.freeze
18
23
  def bucket(rate: 5, burst: 10, prefix: DEFAULT_PREFIX, redis: default_redis)
19
- Bucket.new(rate, burst, prefix, redis).tap(&:initialize_redis_namespace)
24
+ Bucket.create(rate, burst, prefix, redis)
20
25
  end
21
26
  end
@@ -1,31 +1,50 @@
1
- require 'digest/sha1'
1
+ require 'ihasa/lua'
2
2
  module Ihasa
3
- NOK = 0
4
- OK = 1
5
3
  # Bucket class. That bucket fills up to burst, by rate per
6
4
  # second. Each accept? or accept?! call decrement it from 1.
7
5
  class Bucket
8
- attr_reader :redis
6
+ class << self
7
+ def create(*args)
8
+ new(*args).tap(&:save)
9
+ end
10
+
11
+ REDIS_VERSION_WITH_REPLICATE_COMMANDS_SUPPORT = 3.2
12
+
13
+ def legacy_mode?(redis)
14
+ redis_version(redis) < REDIS_VERSION_WITH_REPLICATE_COMMANDS_SUPPORT
15
+ end
16
+
17
+ private
18
+
19
+ def redis_version(redis)
20
+ Float(redis.info['redis_version'][/\d+\.\d+/])
21
+ end
22
+ end
23
+
24
+ attr_reader :redis, :keys, :rate, :burst, :prefix
9
25
  def initialize(rate, burst, prefix, redis)
26
+ @implementation =
27
+ if self.class.legacy_mode?(redis)
28
+ require 'ihasa/bucket/legacy_implementation'
29
+ LegacyImplementation.instance
30
+ else
31
+ require 'ihasa/bucket/implementation'
32
+ Implementation.instance
33
+ end
10
34
  @prefix = prefix
11
- @keys = {}
12
- @keys[:rate] = "#{prefix}:RATE"
13
- @keys[:allowance] = "#{prefix}:ALLOWANCE"
14
- @keys[:burst] = "#{prefix}:BURST"
15
- @keys[:last] = "#{prefix}:LAST"
35
+ @keys = Ihasa::OPTIONS.map { |opt| "#{prefix}:#{opt.upcase}" }
16
36
  @redis = redis
17
37
  @rate = Float rate
18
38
  @burst = Float burst
19
- self.digest = Digest::SHA1.hexdigest statement
20
39
  end
21
40
 
22
41
  SETUP_ADVICE = 'Ensure that the method '\
23
- 'Ihasa::Bucket#initialize_redis_namespace was called.'.freeze
42
+ 'Ihasa::Bucket#save was called.'.freeze
24
43
  SETUP_ERROR = ('Redis raised an error: %{msg}. ' + SETUP_ADVICE).freeze
25
44
  class RedisNamespaceSetupError < RuntimeError; end
26
45
 
27
46
  def accept?
28
- result = redis.evalsha(digest, redis_keys) == OK
47
+ result = @implementation.accept?(self) == OK
29
48
  return yield if result && block_given?
30
49
  result
31
50
  rescue Redis::CommandError => e
@@ -36,106 +55,16 @@ module Ihasa
36
55
 
37
56
  def accept!
38
57
  result = (block_given? ? accept?(&Proc.new) : accept?)
39
- raise EmptyBucket, "Bucket #{@prefix} throttle limit" unless result
58
+ raise EmptyBucket, "Bucket #{prefix} throttle limit" unless result
40
59
  result
41
60
  end
42
61
 
43
- def initialize_redis_namespace
44
- load_statement
45
- redis_eval <<-LUA
46
- #{INTRO_STATEMENT}
47
- #{redis_set rate, @rate}
48
- #{redis_set burst, @burst}
49
- #{redis_set allowance, @burst}
50
- #{redis_set last, 'now'}
51
- LUA
52
- end
53
-
54
- protected
55
-
56
- attr_accessor :digest
57
-
58
- def load_statement
59
- sha = redis.script(:load, statement)
60
- if sha != digest
61
- raise 'SHA1 inconsistency: expected #{digest}, got #{sha}'
62
- end
63
- end
64
-
65
- require 'forwardable'
66
- extend Forwardable
67
-
68
- def_delegator :@keys, :keys
69
- def_delegator :@keys, :values, :redis_keys
70
-
71
- def index(key)
72
- keys.index(key) + 1
73
- end
74
-
75
- # Please note that the replicate_commands is mandatory when using a
76
- # non deterministic command before writing shit to the redis instance.
77
- INTRO_STATEMENT = <<-LUA.freeze
78
- redis.replicate_commands()
79
- local now = redis.call('TIME')
80
- now = now[1] + now[2] * 10 ^ -6
81
- LUA
82
-
83
- def redis_eval(statement)
84
- redis.eval(statement, redis_keys)
85
- end
86
-
87
- ELAPSED_STATEMENT = 'local elapsed = now - last'.freeze
88
-
89
- def local_statements
90
- results = %i(rate burst last allowance).map do |key|
91
- "local #{key} = tonumber(#{redis_get(redis_key(key))})"
92
- end
93
- results << ELAPSED_STATEMENT
94
- results.join "\n"
95
- end
96
-
97
- ALLOWANCE_UPDATE_STATEMENT = <<-LUA.freeze
98
- allowance = allowance + (elapsed * rate)
99
- if allowance > burst then
100
- allowance = burst
101
- end
102
- LUA
103
-
104
- def statement
105
- @statement ||= <<-LUA
106
- #{INTRO_STATEMENT}
107
- #{local_statements}
108
- #{ALLOWANCE_UPDATE_STATEMENT}
109
- local result = #{NOK}
110
- if allowance >= 1.0 then
111
- allowance = allowance - 1.0
112
- result = #{OK}
113
- end
114
- #{redis_set(last, 'now')}
115
- #{redis_set(allowance, 'allowance')}
116
- return result
117
- LUA
118
- end
119
-
120
- def redis_exists(key)
121
- "redis.call('EXISTS', #{key})"
122
- end
123
-
124
- def redis_key(key)
125
- "KEYS[#{index key}]"
126
- end
127
-
128
- def redis_get(key)
129
- "tonumber(redis.call('GET', #{key}))"
130
- end
131
-
132
- def redis_set(key, value)
133
- "redis.call('SET', #{key}, tostring(#{value}))"
62
+ def save
63
+ @implementation.save(self)
134
64
  end
135
65
 
136
- def method_missing(sym, *args, &block)
137
- super unless @keys.key?(sym)
138
- redis_key sym
66
+ def delete
67
+ redis.del(keys)
139
68
  end
140
69
  end
141
70
  end
@@ -0,0 +1,23 @@
1
+ require 'ihasa/lua'
2
+ require 'singleton'
3
+ module Ihasa
4
+ # For redis server whith version >= 3.2
5
+ class Bucket::Implementation
6
+ include Singleton
7
+
8
+ def save(bucket)
9
+ sha = bucket.redis.script(:load, Lua::TOKEN_BUCKET_ALGORITHM)
10
+ if sha != Lua::TOKEN_BUCKET_HASH
11
+ raise "SHA1 mismatch: expected #{Lua::TOKEN_BUCKET_HASH}, got #{sha}"
12
+ end
13
+ bucket.redis.eval(
14
+ Lua.configuration(bucket.rate, bucket.burst),
15
+ bucket.keys
16
+ )
17
+ end
18
+
19
+ def accept?(bucket)
20
+ bucket.redis.evalsha(Lua::TOKEN_BUCKET_HASH, bucket.keys)
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,33 @@
1
+ require 'ihasa/lua'
2
+ require 'singleton'
3
+ module Ihasa
4
+ # For redis server whose version is prior to 3.2
5
+ class Bucket::LegacyImplementation
6
+ include Singleton
7
+
8
+ def save(bucket)
9
+ bucket.redis.eval(
10
+ Lua.configuration(
11
+ bucket.rate,
12
+ bucket.burst,
13
+ Lua.now_declaration(redis_time(bucket.redis))
14
+ ),
15
+ bucket.keys
16
+ )
17
+ end
18
+
19
+ def accept?(bucket)
20
+ now = redis_time bucket.redis
21
+ script = Lua.token_bucket_algorithm_legacy(now)
22
+ bucket.redis.eval(script, bucket.keys)
23
+ end
24
+
25
+ private
26
+
27
+ MICROSECS_PER_SEC = 10**6
28
+ def redis_time(redis)
29
+ seconds, microseconds = redis.time
30
+ seconds + microseconds.to_f / MICROSECS_PER_SEC
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,99 @@
1
+ require 'digest/sha1'
2
+ module Ihasa
3
+ # Contains lua related logic
4
+ module Lua
5
+ NOW_DECLARATION = <<-LUA.freeze
6
+ redis.replicate_commands()
7
+ local now = redis.call('TIME')
8
+ now = now[1] + now[2] * 10 ^ -6
9
+ LUA
10
+
11
+ ALLOWANCE_UPDATE_STATEMENT = <<-LUA.freeze
12
+ allowance = allowance + (elapsed * rate)
13
+ if allowance > burst then
14
+ allowance = burst
15
+ end
16
+ LUA
17
+
18
+ class << self
19
+ def now_declaration(value)
20
+ "local now = #{value}"
21
+ end
22
+
23
+ # Please note that the replicate_commands is mandatory when using a
24
+ # non deterministic command before writing shit to the redis instance
25
+ # for versions >= 3.2
26
+ def configuration(rate_value, burst_value, now_declaration = NOW_DECLARATION)
27
+ <<-LUA
28
+ #{now_declaration}
29
+ #{set rate, rate_value}
30
+ #{set burst, burst_value}
31
+ #{set allowance, burst_value}
32
+ #{set last, 'now'}
33
+ LUA
34
+ end
35
+
36
+ def index(key)
37
+ Integer(Ihasa::OPTIONS.index(key)) + 1
38
+ end
39
+
40
+ def fetch(key)
41
+ "KEYS[#{index key}]"
42
+ end
43
+
44
+ def get(key)
45
+ "tonumber(redis.call('GET', #{key}))"
46
+ end
47
+
48
+ def set(key, value)
49
+ "redis.call('SET', #{key}, tostring(#{value}))"
50
+ end
51
+
52
+ def exists?(key)
53
+ "redis.call('EXISTS', #{key})"
54
+ end
55
+
56
+ def method_missing(sym, *args, &block)
57
+ super unless Ihasa::OPTIONS.include? sym
58
+ fetch sym
59
+ end
60
+
61
+ def to_local(key)
62
+ "local #{key} = tonumber(#{get(fetch(key))})"
63
+ end
64
+
65
+ def token_bucket_algorithm_legacy(now_value)
66
+ <<-LUA.freeze
67
+ #{now_declaration(now_value)}
68
+ #{TOKEN_BUCKET_ALGORITHM_BODY}
69
+ LUA
70
+ end
71
+ end
72
+
73
+ ELAPSED_STATEMENT = 'local elapsed = now - last'.freeze
74
+ SEP = "\n".freeze
75
+ LOCAL_VARIABLES = Ihasa::OPTIONS
76
+ .map { |key| to_local(key) }
77
+ .tap { |vars| vars << ELAPSED_STATEMENT }.join(SEP).freeze
78
+ TOKEN_BUCKET_ALGORITHM_BODY = <<-LUA.freeze
79
+ #{LOCAL_VARIABLES}
80
+ #{ALLOWANCE_UPDATE_STATEMENT}
81
+ local result = #{Ihasa::NOK}
82
+ if allowance >= 1.0 then
83
+ allowance = allowance - 1.0
84
+ result = #{Ihasa::OK}
85
+ end
86
+ #{set(last, 'now')}
87
+ #{set(allowance, 'allowance')}
88
+ return result
89
+ LUA
90
+ TOKEN_BUCKET_ALGORITHM = <<-LUA.freeze
91
+ #{NOW_DECLARATION}
92
+ #{TOKEN_BUCKET_ALGORITHM_BODY}
93
+ LUA
94
+ TOKEN_BUCKET_HASH = Digest::SHA1.hexdigest(
95
+ Lua::TOKEN_BUCKET_ALGORITHM
96
+ ).freeze
97
+
98
+ end
99
+ end
@@ -3,7 +3,7 @@
3
3
  module Ihasa
4
4
  # This module holds the Ihasa version information.
5
5
  module Version
6
- STRING = '1.0.0'
6
+ STRING = '1.1.0'
7
7
 
8
8
  module_function
9
9
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ihasa
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexandre Ignjatovic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-05-21 00:00:00.000000000 Z
11
+ date: 2016-07-09 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: redis
@@ -50,12 +50,17 @@ extra_rdoc_files:
50
50
  - LICENSE.txt
51
51
  - README.md
52
52
  files:
53
+ - ".ruby-version"
54
+ - ".travis.yml"
53
55
  - Gemfile.lock
54
56
  - LICENSE.txt
55
57
  - README.md
56
58
  - ihasa.gemspec
57
59
  - lib/ihasa.rb
58
60
  - lib/ihasa/bucket.rb
61
+ - lib/ihasa/bucket/implementation.rb
62
+ - lib/ihasa/bucket/legacy_implementation.rb
63
+ - lib/ihasa/lua.rb
59
64
  - lib/ihasa/version.rb
60
65
  homepage: http://github.com/bankair/ihasa
61
66
  licenses:
@@ -69,7 +74,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
69
74
  requirements:
70
75
  - - ">="
71
76
  - !ruby/object:Gem::Version
72
- version: 1.9.3
77
+ version: 2.0.0
73
78
  required_rubygems_version: !ruby/object:Gem::Requirement
74
79
  requirements:
75
80
  - - ">="